string: Add load_unaligned_zeropad() code path to sized_strscpy()

The call to read_word_at_a_time() in sized_strscpy() is problematic with MTE because it may trigger a tag check fault when reading across a tag granule (16 bytes) boundary. To make this code MTE compatible, let's start using load_unaligned_zeropad() on architectures where it is available (i.e. architectures that define CONFIG_DCACHE_WORD_ACCESS). Because load_unaligned_zeropad() takes care of page boundaries as well as tag granule boundaries, also disable the code preventing crossing page boundaries when using load_unaligned_zeropad().

Peter Collingbourne committed 1 year ago

Commit d94c12b1 parent 8ffd0151 file · +10 −3Browse files

Showing the changes to lib/string.c (1 of 1 file). Show all

Unified · Split

lib/string.c+10−3

@@ -119,6 +119,7 @@ ssize_t sized_strscpy(char dest, const char src, size_t count) · 118 unmodified lines
119	119	if (count == 0 \|\| WARN_ON_ONCE(count > INT_MAX))
120	120	return -E2BIG;
121	121
	122	+#ifndef CONFIG_DCACHE_WORD_ACCESS
122	123	#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
123	124	/*
124	125	* If src is unaligned, don't cross a page boundary,
@@ -133,12 +134,14 @@ ssize_t sized_strscpy(char dest, const char src, size_t count) · 8 unmodified lines
133	134	/* If src or dest is unaligned, don't do word-at-a-time. */
134	135	if (((long) dest \| (long) src) & (sizeof(long) - 1))
135	136	max = 0;
	137	+#endif
136	138	#endif
137	139
138	140	/*
139		− * read_word_at_a_time() below may read uninitialized bytes after the
140		− * trailing zero and use them in comparisons. Disable this optimization
141		− * under KMSAN to prevent false positive reports.
	141	+ * load_unaligned_zeropad() or read_word_at_a_time() below may read
	142	+ * uninitialized bytes after the trailing zero and use them in
	143	+ * comparisons. Disable this optimization under KMSAN to prevent
	144	+ * false positive reports.
142	145	*/
143	146	if (IS_ENABLED(CONFIG_KMSAN))
144	147	max = 0;
@@ -146,7 +149,11 @@ ssize_t sized_strscpy(char dest, const char src, size_t count) · 1 unmodified line
146	149	while (max >= sizeof(unsigned long)) {
147	150	unsigned long c, data;
148	151
	152	+#ifdef CONFIG_DCACHE_WORD_ACCESS
	153	+ c = load_unaligned_zeropad(src+res);
	154	+#else
149	155	c = read_word_at_a_time(src+res);
	156	+#endif
150	157	if (has_zero(c, &data, &constants)) {
151	158	data = prep_zero_mask(c, data, &constants);
152	159	data = create_zero_mask(data);