timekeeping: Use READ_ONCE/WRITE_ONCE() for xtime_sec to prevent tearing

The timekeeper update path uses a bulk memcpy() to synchronize the timekeeper structure, which is not guaranteed to be atomic. This allows for torn reads in ktime_get_real_seconds() on 64-bit systems, where the sequence counter protection is bypassed for performance. To prevent reading a torn 64-bit xtime_sec value, enforce atomic-like access by using WRITE_ONCE() for the critical field before the bulk memcpy() in timekeeping_update_from_shadow(). Correspondingly, use READ_ONCE() in ktime_get_real_seconds() to ensure a fresh, consistent load from memory. [ tglx: Format changelog and add comment ]

Dennis Moshegov committed 3 months ago

Commit d7fc1331 parent ecc330e1 file · +7 −3Browse files

Showing the changes to kernel (1 of 1 file). Show all

Unified · Split

kernel/time/timekeeping.c+7−3

@@ -858,7 +858,11 @@ static void timekeeping_update_from_shadow(struct tk_data *tkd, unsigned int act · 857 unmodified lines
858	858	* the downside that the reader side does not longer benefit from
859	859	* the cacheline optimized data layout of the timekeeper and requires
860	860	* another indirection.
	861	+ *
	862	+ * Write xtime_sec first so that even if the memcpy() tears the store
	863	+ * data integrity is provided for ktime_get_real_seconds().
861	864	*/
	865	+ WRITE_ONCE(tkd->timekeeper.xtime_sec, tk->xtime_sec);
862	866	memcpy(&tkd->timekeeper, tk, sizeof(*tk));
863	867	write_seqcount_end(&tkd->seq);
864	868	}
@@ -1186,11 +1190,11 @@ time64_t ktime_get_real_seconds(void) · 321 unmodified lines
1186	1190	unsigned int seq;
1187	1191
1188	1192	if (IS_ENABLED(CONFIG_64BIT))
1189		− return tk->xtime_sec;
	1193	+ return READ_ONCE(tk->xtime_sec);
1190	1194
1191	1195	do {
1192	1196	seq = read_seqcount_begin(&tk_core.seq);
1193		− seconds = tk->xtime_sec;
	1197	+ seconds = READ_ONCE(tk->xtime_sec);
1194	1198
1195	1199	} while (read_seqcount_retry(&tk_core.seq, seq));
1196	1200
@@ -1212,7 +1216,7 @@ noinstr time64_t __ktime_get_real_seconds(void) · 15 unmodified lines
1212	1216	{
1213	1217	struct timekeeper *tk = &tk_core.timekeeper;
1214	1218
1215		− return tk->xtime_sec;
	1219	+ return READ_ONCE(tk->xtime_sec);
1216	1220	}
1217	1221
1218	1222	static inline u64 tk_clock_read_snapshot(const struct tk_read_base *tkr,