timekeeping: Use READ_ONCE/WRITE_ONCE() for xtime_sec to prevent tearing

The timekeeper update path uses a bulk memcpy() to synchronize the timekeeper structure, which is not guaranteed to be atomic. This allows for torn reads in ktime_get_real_seconds() on 64-bit systems, where the sequence counter protection is bypassed for performance. To prevent reading a torn 64-bit xtime_sec value, enforce atomic-like access by using WRITE_ONCE() for the critical field before the bulk memcpy() in timekeeping_update_from_shadow(). Correspondingly, use READ_ONCE() in ktime_get_real_seconds() to ensure a fresh, consistent load from memory. [ tglx: Format changelog and add comment ]

Dennis Moshegov committed 3 months ago

Commit d7fc1331 parent ecc330e1 file · +7 −3Browse files

Showing the changes to kernel (1 of 1 file). Show all

Unified · Split

kernel/time/timekeeping.c+7−3

 	 * the downside that the reader side does not longer benefit from
 	 * the cacheline optimized data layout of the timekeeper and requires
 	 * another indirection.
+	 *
+	 * Write xtime_sec first so that even if the memcpy() tears the store
+	 * data integrity is provided for ktime_get_real_seconds().
 	 */
+	WRITE_ONCE(tkd->timekeeper.xtime_sec, tk->xtime_sec);
 	memcpy(&tkd->timekeeper, tk, sizeof(*tk));
 	write_seqcount_end(&tkd->seq);
 }
 	unsigned int seq;
 	if (IS_ENABLED(CONFIG_64BIT))
-		return tk->xtime_sec;
+		return READ_ONCE(tk->xtime_sec);
 	do {
 		seq = read_seqcount_begin(&tk_core.seq);
-		seconds = tk->xtime_sec;
+		seconds = READ_ONCE(tk->xtime_sec);
 	} while (read_seqcount_retry(&tk_core.seq, seq));
 {
 	struct timekeeper *tk = &tk_core.timekeeper;
-	return tk->xtime_sec;
+	return READ_ONCE(tk->xtime_sec);
 }
 static inline u64 tk_clock_read_snapshot(const struct tk_read_base *tkr,