Merge tag 'rust-6.11' of https://github.com/Rust-for-Linux/linux

Pull Rust updates from Miguel Ojeda: "The highlight is the establishment of a minimum version for the Rust toolchain, including 'rustc' (and bundled tools) and 'bindgen'. The initial minimum will be the pinned version we currently have, i.e. we are just widening the allowed versions. That covers three stable Rust releases: 1.78.0, 1.79.0, 1.80.0 (getting released tomorrow), plus beta, plus nightly. This should already be enough for kernel developers in distributions that provide recent Rust compiler versions routinely, such as Arch Linux, Debian Unstable (outside the freeze period), Fedora Linux, Gentoo Linux (especially the testing channel), Nix (unstable) and openSUSE Slowroll and Tumbleweed. In addition, the kernel is now being built-tested by Rust's pre-merge CI. That is, every change that is attempting to land into the Rust compiler is tested against the kernel, and it is merged only if it passes. Similarly, the bindgen tool has agreed to build the kernel in their CI too. Thus, with the pre-merge CI in place, both projects hope to avoid unintentional changes to Rust that break the kernel. This means that, in general, apart from intentional changes on their side (that we will need to workaround conditionally on our side), the upcoming Rust compiler versions should generally work. In addition, the Rust project has proposed getting the kernel into stable Rust (at least solving the main blockers) as one of its three flagship goals for 2024H2 [1]. I would like to thank Niko, Sid, Emilio et al. for their help promoting the collaboration between Rust and the kernel. Toolchain and infrastructure: - Support several Rust toolchain versions. - Support several bindgen versions. - Remove 'cargo' requirement and simplify 'rusttest', thanks to 'alloc' having been dropped last cycle. - Provide proper error reporting for the 'rust-analyzer' target. 'kernel' crate: - Add 'uaccess' module with a safe userspace pointers abstraction. - Add 'page' module with a 'struct page' abstraction. - Support more complex generics in workqueue's 'impl_has_work!' macro. 'macros' crate: - Add 'firmware' field support to the 'module!' macro. - Improve 'module!' macro documentation. Documentation: - Provide instructions on what packages should be installed to build the kernel in some popular Linux distributions. - Introduce the new kernel.org LLVM+Rust toolchains. - Explain '#[no_std]'. And a few other small bits" Link: https://rust-lang.github.io/rust-project-goals/2024h2/index.html#flagship-goals [1] * tag 'rust-6.11' of https://github.com/Rust-for-Linux/linux: (26 commits) docs: rust: quick-start: add section on Linux distributions rust: warn about `bindgen` versions 0.66.0 and 0.66.1 rust: start supporting several `bindgen` versions rust: work around `bindgen` 0.69.0 issue rust: avoid assuming a particular `bindgen` build rust: start supporting several compiler versions rust: simplify Clippy warning flags set rust: relax most deny-level lints to warnings rust: allow `dead_code` for never constructed bindings rust: init: simplify from `map_err` to `inspect_err` rust: macros: indent list item in `paste!`'s docs rust: add abstraction for `struct page` rust: uaccess: add typed accessors for userspace pointers uaccess: always export _copy_[from|to]_user with CONFIG_RUST rust: uaccess: add userspace pointers kbuild: rust-analyzer: improve comment documentation kbuild: rust-analyzer: better error handling docs: rust: no_std is used rust: alloc: add __GFP_HIGHMEM flag rust: alloc: fix typo in docs for GFP_NOWAIT ...

Linus Torvalds committed 2 years ago

Commit 910bfc22 parents ff30564 b12634125 files · +1058 −236Browse files

Unified · Split

25 files

Documentation/process/changes.rst +1 −8
Documentation/rust/general-information.rst +8 −0
Documentation/rust/quick-start.rst +100 −43
Documentation/rust/testing.rst +2 −3
Makefile +16 −14
include/linux/uaccess.h +32 −14
init/Kconfig +4 −1
lib/usercopy.c +4 −26
rust/Makefile +10 −64
rust/bindings/bindings_helper.h +1 −0
rust/bindings/lib.rs +1 −0
rust/helpers.c +34 −0
rust/kernel/alloc.rs +16 −1
rust/kernel/init.rs +4 −9
rust/kernel/lib.rs +2 −0
rust/kernel/page.rs +250 −0
rust/kernel/types.rs +64 −0
rust/kernel/uaccess.rs +388 −0
rust/kernel/workqueue.rs +9 −7
rust/macros/lib.rs +39 −6
rust/macros/module.rs +16 −2
rust/uapi/lib.rs +1 −0
scripts/rust_is_available.sh +17 −16
scripts/rust_is_available_bindgen_0_66.h +2 −0
scripts/rust_is_available_test.py +37 −22

Documentation/process/changes.rst+1−8

@@ -89,14 +89,7 @@ docs on :ref:`Building Linux with Clang/LLVM <kbuild_llvm>`. · 88 unmodified lines
89	89	Rust (optional)
90	90	---------------
91	91
92		−A particular version of the Rust toolchain is required. Newer versions may or
93		−may not work because the kernel depends on some unstable Rust features, for
94		−the moment.
95		−
96		−Each Rust toolchain comes with several "components", some of which are required
97		−(like ``rustc``) and some that are optional. The ``rust-src`` component (which
98		−is optional) needs to be installed to build the kernel. Other components are
99		−useful for developing.
	92	+A recent version of the Rust compiler is required.
100	93
101	94	Please see Documentation/rust/quick-start.rst for instructions on how to
102	95	satisfy the build requirements of Rust support. In particular, the ``Makefile``

Documentation/rust/general-information.rst+8−0

@@ -7,6 +7,14 @@ This document contains useful information to know when working with · 6 unmodified lines
7	7	the Rust support in the kernel.
8	8
9	9
	10	+``no_std``
	11	+----------
	12	+
	13	+The Rust support in the kernel can link only `core <https://doc.rust-lang.org/core/>`_,
	14	+but not `std <https://doc.rust-lang.org/std/>`_. Crates for use in the
	15	+kernel must opt into this behavior using the ``#![no_std]`` attribute.
	16	+
	17	+
10	18	Code documentation
11	19	------------------
12	20

Documentation/rust/quick-start.rst+100−43

@@ -5,17 +5,93 @@ Quick Start · 4 unmodified lines
5	5
6	6	This document describes how to get started with kernel development in Rust.
7	7
	8	+There are a few ways to install a Rust toolchain needed for kernel development.
	9	+A simple way is to use the packages from your Linux distribution if they are
	10	+suitable -- the first section below explains this approach. An advantage of this
	11	+approach is that, typically, the distribution will match the LLVM used by Rust
	12	+and Clang.
	13	+
	14	+Another way is using the prebuilt stable versions of LLVM+Rust provided on
	15	+`kernel.org <https://kernel.org/pub/tools/llvm/rust/>`_. These are the same slim
	16	+and fast LLVM toolchains from :ref:`Getting LLVM <getting_llvm>` with versions
	17	+of Rust added to them that Rust for Linux supports. Two sets are provided: the
	18	+"latest LLVM" and "matching LLVM" (please see the link for more information).
	19	+
	20	+Alternatively, the next two "Requirements" sections explain each component and
	21	+how to install them through ``rustup``, the standalone installers from Rust
	22	+and/or building them.
	23	+
	24	+The rest of the document explains other aspects on how to get started.
	25	+
	26	+
	27	+Distributions
	28	+-------------
	29	+
	30	+Arch Linux
	31	+**********
	32	+
	33	+Arch Linux provides recent Rust releases and thus it should generally work out
	34	+of the box, e.g.::
	35	+
	36	+ pacman -S rust rust-src rust-bindgen
	37	+
	38	+
	39	+Debian
	40	+******
	41	+
	42	+Debian Unstable (Sid), outside of the freeze period, provides recent Rust
	43	+releases and thus it should generally work out of the box, e.g.::
	44	+
	45	+ apt install rustc rust-src bindgen rustfmt rust-clippy
	46	+
	47	+
	48	+Fedora Linux
	49	+************
	50	+
	51	+Fedora Linux provides recent Rust releases and thus it should generally work out
	52	+of the box, e.g.::
	53	+
	54	+ dnf install rust rust-src bindgen-cli rustfmt clippy
	55	+
	56	+
	57	+Gentoo Linux
	58	+************
	59	+
	60	+Gentoo Linux (and especially the testing branch) provides recent Rust releases
	61	+and thus it should generally work out of the box, e.g.::
	62	+
	63	+ USE='rust-src rustfmt clippy' emerge dev-lang/rust dev-util/bindgen
	64	+
	65	+``LIBCLANG_PATH`` may need to be set.
	66	+
	67	+
	68	+Nix
	69	+***
	70	+
	71	+Nix (unstable channel) provides recent Rust releases and thus it should
	72	+generally work out of the box, e.g.::
	73	+
	74	+ { pkgs ? import <nixpkgs> {} }:
	75	+ pkgs.mkShell {
	76	+ nativeBuildInputs = with pkgs; [ rustc rust-bindgen rustfmt clippy ];
	77	+ RUST_LIB_SRC = "${pkgs.rust.packages.stable.rustPlatform.rustLibSrc}";
	78	+ }
	79	+
	80	+
	81	+openSUSE
	82	+********
	83	+
	84	+openSUSE Slowroll and openSUSE Tumbleweed provide recent Rust releases and thus
	85	+they should generally work out of the box, e.g.::
	86	+
	87	+ zypper install rust rust1.79-src rust-bindgen clang
	88	+
8	89
9	90	Requirements: Building
10	91	----------------------
11	92
12	93	This section explains how to fetch the tools needed for building.
13	94
14		−Some of these requirements might be available from Linux distributions
15		−under names like ``rustc``, ``rust-src``, ``rust-bindgen``, etc. However,
16		−at the time of writing, they are likely not to be recent enough unless
17		−the distribution tracks the latest releases.
18		−
19	95	To easily check whether the requirements are met, the following target
20	96	can be used::
21	97
@@ -29,16 +105,15 @@ if that is the case. · 7 unmodified lines
29	105	rustc
30	106	*****
31	107
32		−A particular version of the Rust compiler is required. Newer versions may or
33		−may not work because, for the moment, the kernel depends on some unstable
34		−Rust features.
	108	+A recent version of the Rust compiler is required.
35	109
36	110	If ``rustup`` is being used, enter the kernel build directory (or use
37		−``--path=<build-dir>`` argument to the ``set`` sub-command) and run::
	111	+``--path=<build-dir>`` argument to the ``set`` sub-command) and run,
	112	+for instance::
38	113
39		− rustup override set $(scripts/min-tool-version.sh rustc)
	114	+ rustup override set stable
40	115
41		−This will configure your working directory to use the correct version of
	116	+This will configure your working directory to use the given version of
42	117	``rustc`` without affecting your default toolchain.
43	118
44	119	Note that the override applies to the current working directory (and its
@@ -65,9 +140,9 @@ version later on requires re-adding the component. · 20 unmodified lines
65	140	Otherwise, if a standalone installer is used, the Rust source tree may be
66	141	downloaded into the toolchain's installation folder::
67	142
68		− curl -L "https://static.rust-lang.org/dist/rust-src-$(scripts/min-tool-version.sh rustc).tar.gz" \|
	143	+ curl -L "https://static.rust-lang.org/dist/rust-src-$(rustc --version \| cut -d' ' -f2).tar.gz" \|
69	144	tar -xzf - -C "$(rustc --print sysroot)/lib" \
70		− "rust-src-$(scripts/min-tool-version.sh rustc)/rust-src/lib/" \
	145	+ "rust-src-$(rustc --version \| cut -d' ' -f2)/rust-src/lib/" \
71	146	--strip-components=3
72	147
73	148	In this case, upgrading the Rust compiler version later on requires manually
@@ -101,26 +176,22 @@ bindgen · 27 unmodified lines
101	176	*******
102	177
103	178	The bindings to the C side of the kernel are generated at build time using
104		−the ``bindgen`` tool. A particular version is required.
	179	+the ``bindgen`` tool.
105	180
106		−Install it via (note that this will download and build the tool from source)::
	181	+Install it, for instance, via (note that this will download and build the tool
	182	+from source)::
107	183
108		− cargo install --locked --version $(scripts/min-tool-version.sh bindgen) bindgen-cli
	184	+ cargo install --locked bindgen-cli
109	185
110		−``bindgen`` needs to find a suitable ``libclang`` in order to work. If it is
111		−not found (or a different ``libclang`` than the one found should be used),
112		−the process can be tweaked using the environment variables understood by
113		−``clang-sys`` (the Rust bindings crate that ``bindgen`` uses to access
114		−``libclang``):
	186	+``bindgen`` uses the ``clang-sys`` crate to find a suitable ``libclang`` (which
	187	+may be linked statically, dynamically or loaded at runtime). By default, the
	188	+``cargo`` command above will produce a ``bindgen`` binary that will load
	189	+``libclang`` at runtime. If it is not found (or a different ``libclang`` than
	190	+the one found should be used), the process can be tweaked, e.g. by using the
	191	+``LIBCLANG_PATH`` environment variable. For details, please see ``clang-sys``'s
	192	+documentation at:
115	193
116		−* ``LLVM_CONFIG_PATH`` can be pointed to an ``llvm-config`` executable.
117		−
118		−* Or ``LIBCLANG_PATH`` can be pointed to a ``libclang`` shared library
119		− or to the directory containing it.
120		−
121		−* Or ``CLANG_PATH`` can be pointed to a ``clang`` executable.
122		−
123		−For details, please see ``clang-sys``'s documentation at:
	194	+ https://github.com/KyleMayes/clang-sys#linking
124	195
125	196	https://github.com/KyleMayes/clang-sys#environment-variables
126	197
@@ -164,20 +235,6 @@ can be installed manually:: · 37 unmodified lines
164	235	The standalone installers also come with ``clippy``.
165	236
166	237
167		−cargo
168		−*****
169		−
170		−``cargo`` is the Rust native build system. It is currently required to run
171		−the tests since it is used to build a custom standard library that contains
172		−the facilities provided by the custom ``alloc`` in the kernel. The tests can
173		−be run using the ``rusttest`` Make target.
174		−
175		−If ``rustup`` is being used, all the profiles already install the tool,
176		−thus nothing needs to be done.
177		−
178		−The standalone installers also come with ``cargo``.
179		−
180		−
181	238	rustdoc
182	239	*******
183	240

Documentation/rust/testing.rst+2−3

@@ -131,9 +131,8 @@ Additionally, there are the ``#[test]`` tests. These can be run using the · 130 unmodified lines
131	131
132	132	make LLVM=1 rusttest
133	133
134		−This requires the kernel ``.config`` and downloads external repositories. It
135		−runs the ``#[test]`` tests on the host (currently) and thus is fairly limited in
136		−what these tests can test.
	134	+This requires the kernel ``.config``. It runs the ``#[test]`` tests on the host
	135	+(currently) and thus is fairly limited in what these tests can test.
137	136
138	137	The Kselftests
139	138	--------------

Makefile+16−14

@@ -445,17 +445,17 @@ KBUILD_USERLDFLAGS := $(USERLDFLAGS) · 444 unmodified lines
445	445	# host programs.
446	446	export rust_common_flags := --edition=2021 \
447	447	-Zbinary_dep_depinfo=y \
448		− -Dunsafe_op_in_unsafe_fn -Drust_2018_idioms \
449		− -Dunreachable_pub -Dnon_ascii_idents \
	448	+ -Dunsafe_op_in_unsafe_fn \
	449	+ -Dnon_ascii_idents \
	450	+ -Wrust_2018_idioms \
	451	+ -Wunreachable_pub \
450	452	-Wmissing_docs \
451		− -Drustdoc::missing_crate_level_docs \
452		− -Dclippy::correctness -Dclippy::style \
453		− -Dclippy::suspicious -Dclippy::complexity \
454		− -Dclippy::perf \
455		− -Dclippy::let_unit_value -Dclippy::mut_mut \
456		− -Dclippy::needless_bitwise_bool \
457		− -Dclippy::needless_continue \
458		− -Dclippy::no_mangle_with_rust_abi \
	453	+ -Wrustdoc::missing_crate_level_docs \
	454	+ -Wclippy::all \
	455	+ -Wclippy::mut_mut \
	456	+ -Wclippy::needless_bitwise_bool \
	457	+ -Wclippy::needless_continue \
	458	+ -Wclippy::no_mangle_with_rust_abi \
459	459	-Wclippy::dbg_macro
460	460
461	461	KBUILD_HOSTCFLAGS := $(KBUILD_USERHOSTCFLAGS) $(HOST_LFS_CFLAGS) \
@@ -493,7 +493,6 @@ RUSTDOC = rustdoc · 31 unmodified lines
493	493	RUSTFMT = rustfmt
494	494	CLIPPY_DRIVER = clippy-driver
495	495	BINDGEN = bindgen
496		−CARGO = cargo
497	496	PAHOLE = pahole
498	497	RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids
499	498	LEX = flex
@@ -559,7 +558,7 @@ KBUILD_RUSTFLAGS := $(rust_common_flags) \ · 59 unmodified lines
559	558	-Csymbol-mangling-version=v0 \
560	559	-Crelocation-model=static \
561	560	-Zfunction-sections=n \
562		− -Dclippy::float_arithmetic
	561	+ -Wclippy::float_arithmetic
563	562
564	563	KBUILD_AFLAGS_KERNEL :=
565	564	KBUILD_CFLAGS_KERNEL :=
@@ -587,7 +586,7 @@ endif · 21 unmodified lines
587	586	export RUSTC_BOOTSTRAP := 1
588	587
589	588	export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC HOSTPKG_CONFIG
590		−export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN CARGO
	589	+export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
591	590	export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
592	591	export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
593	592	export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
@@ -1959,9 +1958,12 @@ quiet_cmd_tags = GEN $@ · 1365 unmodified lines
1959	1958	tags TAGS cscope gtags: FORCE
1960	1959	$(call cmd,tags)
1961	1960
1962		−# IDE support targets
	1961	+# Generate rust-project.json (a file that describes the structure of non-Cargo
	1962	+# Rust projects) for rust-analyzer (an implementation of the Language Server
	1963	+# Protocol).
1963	1964	PHONY += rust-analyzer
1964	1965	rust-analyzer:
	1966	+ $(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh
1965	1967	$(Q)$(MAKE) $(build)=rust $@
1966	1968
1967	1969	# Script to generate missing namespace dependencies

include/linux/uaccess.h+32−14

@@ -5,6 +5,7 @@ · 4 unmodified lines
5	5	#include <linux/fault-inject-usercopy.h>
6	6	#include <linux/instrumented.h>
7	7	#include <linux/minmax.h>
	8	+#include <linux/nospec.h>
8	9	#include <linux/sched.h>
9	10	#include <linux/thread_info.h>
10	11
@@ -138,13 +139,26 @@ __copy_to_user(void __user to, const void from, unsigned long n) · 127 unmodified lines
138	139	return raw_copy_to_user(to, from, n);
139	140	}
140	141
141		−#ifdef INLINE_COPY_FROM_USER
	142	+/*
	143	+ * Architectures that #define INLINE_COPY_TO_USER use this function
	144	+ * directly in the normal copy_to/from_user(), the other ones go
	145	+ * through an extern _copy_to/from_user(), which expands the same code
	146	+ * here.
	147	+ *
	148	+ * Rust code always uses the extern definition.
	149	+ */
142	150	static inline __must_check unsigned long
143		−_copy_from_user(void to, const void __user from, unsigned long n)
	151	+_inline_copy_from_user(void to, const void __user from, unsigned long n)
144	152	{
145	153	unsigned long res = n;
146	154	might_fault();
147	155	if (!should_fail_usercopy() && likely(access_ok(from, n))) {
	156	+ /*
	157	+ * Ensure that bad access_ok() speculation will not
	158	+ * lead to nasty side effects after the copy is
	159	+ * finished:
	160	+ */
	161	+ barrier_nospec();
148	162	instrument_copy_from_user_before(to, from, n);
149	163	res = raw_copy_from_user(to, from, n);
150	164	instrument_copy_from_user_after(to, from, n, res);
@@ -153,14 +167,11 @@ _copy_from_user(void to, const void __user from, unsigned long n) · 2 unmodified lines
153	167	memset(to + (n - res), 0, res);
154	168	return res;
155	169	}
156		−#else
157	170	extern __must_check unsigned long
158	171	_copy_from_user(void , const void __user , unsigned long);
159		−#endif
160	172
161		−#ifdef INLINE_COPY_TO_USER
162	173	static inline __must_check unsigned long
163		−_copy_to_user(void __user to, const void from, unsigned long n)
	174	+_inline_copy_to_user(void __user to, const void from, unsigned long n)
164	175	{
165	176	might_fault();
166	177	if (should_fail_usercopy())
@@ -171,25 +182,32 @@ _copy_to_user(void __user to, const void from, unsigned long n) · 4 unmodified lines
171	182	}
172	183	return n;
173	184	}
174		−#else
175	185	extern __must_check unsigned long
176	186	_copy_to_user(void __user , const void , unsigned long);
177		−#endif
178	187
179	188	static __always_inline unsigned long __must_check
180	189	copy_from_user(void to, const void __user from, unsigned long n)
181	190	{
182		− if (check_copy_size(to, n, false))
183		− n = _copy_from_user(to, from, n);
184		− return n;
	191	+ if (!check_copy_size(to, n, false))
	192	+ return n;
	193	+#ifdef INLINE_COPY_FROM_USER
	194	+ return _inline_copy_from_user(to, from, n);
	195	+#else
	196	+ return _copy_from_user(to, from, n);
	197	+#endif
185	198	}
186	199
187	200	static __always_inline unsigned long __must_check
188	201	copy_to_user(void __user to, const void from, unsigned long n)
189	202	{
190		− if (check_copy_size(from, n, true))
191		− n = _copy_to_user(to, from, n);
192		− return n;
	203	+ if (!check_copy_size(from, n, true))
	204	+ return n;
	205	+
	206	+#ifdef INLINE_COPY_TO_USER
	207	+ return _inline_copy_to_user(to, from, n);
	208	+#else
	209	+ return _copy_to_user(to, from, n);
	210	+#endif
193	211	}
194	212
195	213	#ifndef copy_mc_to_kernel

init/Kconfig+4−1

@@ -1924,7 +1924,10 @@ config RUSTC_VERSION_TEXT · 1923 unmodified lines
1924	1924	config BINDGEN_VERSION_TEXT
1925	1925	string
1926	1926	depends on RUST
1927		− default $(shell,command -v $(BINDGEN) >/dev/null 2>&1 && $(BINDGEN) --version \|\| echo n)
	1927	+ # The dummy parameter `workaround-for-0.69.0` is required to support 0.69.0
	1928	+ # (https://github.com/rust-lang/rust-bindgen/pull/2678). It can be removed when
	1929	+ # the minimum version is upgraded past that (0.69.1 already fixed the issue).
	1930	+ default $(shell,command -v $(BINDGEN) >/dev/null 2>&1 && $(BINDGEN) --version workaround-for-0.69.0 \|\| echo n)
1928	1931
1929	1932	#
1930	1933	# Place an empty function call at each tracepoint site. Can be

lib/usercopy.c+4−26

@@ -12,40 +12,18 @@ · 11 unmodified lines
12	12
13	13	/* out-of-line parts */
14	14
15		−#ifndef INLINE_COPY_FROM_USER
	15	+#if !defined(INLINE_COPY_FROM_USER) \|\| defined(CONFIG_RUST)
16	16	unsigned long _copy_from_user(void to, const void __user from, unsigned long n)
17	17	{
18		− unsigned long res = n;
19		− might_fault();
20		− if (!should_fail_usercopy() && likely(access_ok(from, n))) {
21		− /*
22		− * Ensure that bad access_ok() speculation will not
23		− * lead to nasty side effects after the copy is
24		− * finished:
25		− */
26		− barrier_nospec();
27		− instrument_copy_from_user_before(to, from, n);
28		− res = raw_copy_from_user(to, from, n);
29		− instrument_copy_from_user_after(to, from, n, res);
30		− }
31		− if (unlikely(res))
32		− memset(to + (n - res), 0, res);
33		− return res;
	18	+ return _inline_copy_from_user(to, from, n);
34	19	}
35	20	EXPORT_SYMBOL(_copy_from_user);
36	21	#endif
37	22
38		−#ifndef INLINE_COPY_TO_USER
	23	+#if !defined(INLINE_COPY_TO_USER) \|\| defined(CONFIG_RUST)
39	24	unsigned long _copy_to_user(void __user to, const void from, unsigned long n)
40	25	{
41		− might_fault();
42		− if (should_fail_usercopy())
43		− return n;
44		− if (likely(access_ok(to, n))) {
45		− instrument_copy_to_user(to, from, n);
46		− n = raw_copy_to_user(to, from, n);
47		− }
48		− return n;
	26	+ return _inline_copy_to_user(to, from, n);
49	27	}
50	28	EXPORT_SYMBOL(_copy_to_user);
51	29	#endif

rust/Makefile+10−64

@@ -44,17 +44,10 @@ rustc_sysroot := $(shell MAKEFLAGS= $(RUSTC) $(rust_flags) --print sysroot) · 43 unmodified lines
44	44	rustc_host_target := $(shell $(RUSTC) --version --verbose \| grep -F 'host: ' \| cut -d' ' -f2)
45	45	RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library
46	46
47		−ifeq ($(quiet),silent_)
48		−cargo_quiet=-q
	47	+ifneq ($(quiet),)
49	48	rust_test_quiet=-q
50	49	rustdoc_test_quiet=--test-args -q
51	50	rustdoc_test_kernel_quiet=>/dev/null
52		−else ifeq ($(quiet),quiet_)
53		−rust_test_quiet=-q
54		−rustdoc_test_quiet=--test-args -q
55		−rustdoc_test_kernel_quiet=>/dev/null
56		−else
57		−cargo_quiet=--verbose
58	51	endif
59	52
60	53	core-cfgs = \
@@ -135,22 +128,21 @@ quiet_cmd_rustc_test_library = RUSTC TL $< · 74 unmodified lines
135	128	@$(objtree)/include/generated/rustc_cfg $(rustc_target_flags) \
136	129	--crate-type $(if $(rustc_test_library_proc),proc-macro,rlib) \
137	130	--out-dir $(objtree)/$(obj)/test --cfg testlib \
138		− --sysroot $(objtree)/$(obj)/test/sysroot \
139	131	-L$(objtree)/$(obj)/test \
140	132	--crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<
141	133
142		−rusttestlib-build_error: $(src)/build_error.rs rusttest-prepare FORCE
	134	+rusttestlib-build_error: $(src)/build_error.rs FORCE
143	135	+$(call if_changed,rustc_test_library)
144	136
145	137	rusttestlib-macros: private rustc_target_flags = --extern proc_macro
146	138	rusttestlib-macros: private rustc_test_library_proc = yes
147		−rusttestlib-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
	139	+rusttestlib-macros: $(src)/macros/lib.rs FORCE
148	140	+$(call if_changed,rustc_test_library)
149	141
150		−rusttestlib-bindings: $(src)/bindings/lib.rs rusttest-prepare FORCE
	142	+rusttestlib-bindings: $(src)/bindings/lib.rs FORCE
151	143	+$(call if_changed,rustc_test_library)
152	144
153		−rusttestlib-uapi: $(src)/uapi/lib.rs rusttest-prepare FORCE
	145	+rusttestlib-uapi: $(src)/uapi/lib.rs FORCE
154	146	+$(call if_changed,rustc_test_library)
155	147
156	148	quiet_cmd_rustdoc_test = RUSTDOC T $<
@@ -159,7 +151,7 @@ quiet_cmd_rustdoc_test = RUSTDOC T $< · 2 unmodified lines
159	151	$(RUSTDOC) --test $(rust_common_flags) \
160	152	@$(objtree)/include/generated/rustc_cfg \
161	153	$(rustc_target_flags) $(rustdoc_test_target_flags) \
162		− --sysroot $(objtree)/$(obj)/test/sysroot $(rustdoc_test_quiet) \
	154	+ $(rustdoc_test_quiet) \
163	155	-L$(objtree)/$(obj)/test --output $(rustdoc_output) \
164	156	--crate-name $(subst rusttest-,,$@) $<
165	157
@@ -192,7 +184,6 @@ quiet_cmd_rustc_test = RUSTC T $< · 26 unmodified lines
192	184	$(RUSTC) --test $(rust_common_flags) \
193	185	@$(objtree)/include/generated/rustc_cfg \
194	186	$(rustc_target_flags) --out-dir $(objtree)/$(obj)/test \
195		− --sysroot $(objtree)/$(obj)/test/sysroot \
196	187	-L$(objtree)/$(obj)/test \
197	188	--crate-name $(subst rusttest-,,$@) $<; \
198	189	$(objtree)/$(obj)/test/$(subst rusttest-,,$@) $(rust_test_quiet) \
@@ -200,60 +191,15 @@ quiet_cmd_rustc_test = RUSTC T $< · 1 unmodified line
200	191
201	192	rusttest: rusttest-macros rusttest-kernel
202	193
203		−# This prepares a custom sysroot with our custom `alloc` instead of
204		−# the standard one.
205		−#
206		−# This requires several hacks:
207		−# - Unlike `core` and `alloc`, `std` depends on more than a dozen crates,
208		−# including third-party crates that need to be downloaded, plus custom
209		−# `build.rs` steps. Thus hardcoding things here is not maintainable.
210		−# - `cargo` knows how to build the standard library, but it is an unstable
211		−# feature so far (`-Zbuild-std`).
212		−# - `cargo` only considers the use case of building the standard library
213		−# to use it in a given package. Thus we need to create a dummy package
214		−# and pick the generated libraries from there.
215		−# - The usual ways of modifying the dependency graph in `cargo` do not seem
216		−# to apply for the `-Zbuild-std` steps, thus we have to mislead it
217		−# by modifying the sources in the sysroot.
218		−# - To avoid messing with the user's Rust installation, we create a clone
219		−# of the sysroot. However, `cargo` ignores `RUSTFLAGS` in the `-Zbuild-std`
220		−# steps, thus we use a wrapper binary passed via `RUSTC` to pass the flag.
221		−#
222		−# In the future, we hope to avoid the whole ordeal by either:
223		−# - Making the `test` crate not depend on `std` (either improving upstream
224		−# or having our own custom crate).
225		−# - Making the tests run in kernel space (requires the previous point).
226		−# - Making `std` and friends be more like a "normal" crate, so that
227		−# `-Zbuild-std` and related hacks are not needed.
228		−quiet_cmd_rustsysroot = RUSTSYSROOT
229		− cmd_rustsysroot = \
230		− rm -rf $(objtree)/$(obj)/test; \
231		− mkdir -p $(objtree)/$(obj)/test; \
232		− cp -a $(rustc_sysroot) $(objtree)/$(obj)/test/sysroot; \
233		− echo '\#!/bin/sh' > $(objtree)/$(obj)/test/rustc_sysroot; \
234		− echo "$(RUSTC) --sysroot=$(abspath $(objtree)/$(obj)/test/sysroot) \"\$$@\"" \
235		− >> $(objtree)/$(obj)/test/rustc_sysroot; \
236		− chmod u+x $(objtree)/$(obj)/test/rustc_sysroot; \
237		− $(CARGO) -q new $(objtree)/$(obj)/test/dummy; \
238		− RUSTC=$(objtree)/$(obj)/test/rustc_sysroot $(CARGO) $(cargo_quiet) \
239		− test -Zbuild-std --target $(rustc_host_target) \
240		− --manifest-path $(objtree)/$(obj)/test/dummy/Cargo.toml; \
241		− rm $(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib/*; \
242		− cp $(objtree)/$(obj)/test/dummy/target/$(rustc_host_target)/debug/deps/* \
243		− $(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib
244		−
245		−rusttest-prepare: FORCE
246		− +$(call if_changed,rustsysroot)
247		−
248	194	rusttest-macros: private rustc_target_flags = --extern proc_macro
249	195	rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro
250		−rusttest-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
	196	+rusttest-macros: $(src)/macros/lib.rs FORCE
251	197	+$(call if_changed,rustc_test)
252	198	+$(call if_changed,rustdoc_test)
253	199
254	200	rusttest-kernel: private rustc_target_flags = --extern alloc \
255	201	--extern build_error --extern macros --extern bindings --extern uapi
256		−rusttest-kernel: $(src)/kernel/lib.rs rusttest-prepare \
	202	+rusttest-kernel: $(src)/kernel/lib.rs \
257	203	rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
258	204	rusttestlib-uapi FORCE
259	205	+$(call if_changed,rustc_test)
@@ -421,7 +367,7 @@ ifneq ($(or $(CONFIG_ARM64),$(and $(CONFIG_RISCV),$(CONFIG_64BIT))),) · 161 unmodified lines
421	367	endif
422	368
423	369	$(obj)/core.o: private skip_clippy = 1
424		−$(obj)/core.o: private skip_flags = -Dunreachable_pub
	370	+$(obj)/core.o: private skip_flags = -Wunreachable_pub
425	371	$(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))
426	372	$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
427	373	$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs FORCE
@@ -435,7 +381,7 @@ $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE · 7 unmodified lines
435	381	+$(call if_changed_dep,rustc_library)
436	382
437	383	$(obj)/alloc.o: private skip_clippy = 1
438		−$(obj)/alloc.o: private skip_flags = -Dunreachable_pub
	384	+$(obj)/alloc.o: private skip_flags = -Wunreachable_pub
439	385	$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)
440	386	$(obj)/alloc.o: $(RUST_LIB_SRC)/alloc/src/lib.rs $(obj)/compiler_builtins.o FORCE
441	387	+$(call if_changed_dep,rustc_library)

rust/bindings/bindings_helper.h+1−0

@@ -30,4 +30,5 @@ const gfp_t RUST_CONST_HELPER_GFP_KERNEL = GFP_KERNEL; · 29 unmodified lines
30	30	const gfp_t RUST_CONST_HELPER_GFP_KERNEL_ACCOUNT = GFP_KERNEL_ACCOUNT;
31	31	const gfp_t RUST_CONST_HELPER_GFP_NOWAIT = GFP_NOWAIT;
32	32	const gfp_t RUST_CONST_HELPER___GFP_ZERO = __GFP_ZERO;
	33	+const gfp_t RUST_CONST_HELPER___GFP_HIGHMEM = ___GFP_HIGHMEM;
33	34	const blk_features_t RUST_CONST_HELPER_BLK_FEAT_ROTATIONAL = BLK_FEAT_ROTATIONAL;

rust/bindings/lib.rs+1−0

@@ -24,6 +24,7 @@ · 23 unmodified lines
24	24	unsafe_op_in_unsafe_fn
25	25	)]
26	26
	27	+#[allow(dead_code)]
27	28	mod bindings_raw {
28	29	// Use glob import here to expose all helpers.
29	30	// Symbols defined within the module will take precedence to the glob import.

rust/helpers.c+34−0

@@ -26,6 +26,8 @@ · 25 unmodified lines
26	26	#include <linux/device.h>
27	27	#include <linux/err.h>
28	28	#include <linux/errname.h>
	29	+#include <linux/gfp.h>
	30	+#include <linux/highmem.h>
29	31	#include <linux/mutex.h>
30	32	#include <linux/refcount.h>
31	33	#include <linux/sched/signal.h>
@@ -40,6 +42,20 @@ __noreturn void rust_helper_BUG(void) · 8 unmodified lines
40	42	}
41	43	EXPORT_SYMBOL_GPL(rust_helper_BUG);
42	44
	45	+unsigned long rust_helper_copy_from_user(void to, const void __user from,
	46	+ unsigned long n)
	47	+{
	48	+ return copy_from_user(to, from, n);
	49	+}
	50	+EXPORT_SYMBOL_GPL(rust_helper_copy_from_user);
	51	+
	52	+unsigned long rust_helper_copy_to_user(void __user to, const void from,
	53	+ unsigned long n)
	54	+{
	55	+ return copy_to_user(to, from, n);
	56	+}
	57	+EXPORT_SYMBOL_GPL(rust_helper_copy_to_user);
	58	+
43	59	void rust_helper_mutex_lock(struct mutex *lock)
44	60	{
45	61	mutex_lock(lock);
@@ -81,6 +97,24 @@ int rust_helper_signal_pending(struct task_struct *t) · 35 unmodified lines
81	97	}
82	98	EXPORT_SYMBOL_GPL(rust_helper_signal_pending);
83	99
	100	+struct page *rust_helper_alloc_pages(gfp_t gfp_mask, unsigned int order)
	101	+{
	102	+ return alloc_pages(gfp_mask, order);
	103	+}
	104	+EXPORT_SYMBOL_GPL(rust_helper_alloc_pages);
	105	+
	106	+void rust_helper_kmap_local_page(struct page page)
	107	+{
	108	+ return kmap_local_page(page);
	109	+}
	110	+EXPORT_SYMBOL_GPL(rust_helper_kmap_local_page);
	111	+
	112	+void rust_helper_kunmap_local(const void *addr)
	113	+{
	114	+ kunmap_local(addr);
	115	+}
	116	+EXPORT_SYMBOL_GPL(rust_helper_kunmap_local);
	117	+
84	118	refcount_t rust_helper_REFCOUNT_INIT(int n)
85	119	{
86	120	return (refcount_t)REFCOUNT_INIT(n);

rust/kernel/alloc.rs+16−1

@@ -20,6 +20,13 @@ pub struct AllocError; · 19 unmodified lines
20	20	#[derive(Clone, Copy)]
21	21	pub struct Flags(u32);
22	22
	23	+impl Flags {
	24	+ /// Get the raw representation of this flag.
	25	+ pub(crate) fn as_raw(self) -> u32 {
	26	+ self.0
	27	+ }
	28	+}
	29	+
23	30	impl core::ops::BitOr for Flags {
24	31	type Output = Self;
25	32	fn bitor(self, rhs: Self) -> Self::Output {
@@ -52,6 +59,14 @@ pub mod flags { · 26 unmodified lines
52	59	/// This is normally or'd with other flags.
53	60	pub const __GFP_ZERO: Flags = Flags(bindings::__GFP_ZERO);
54	61
	62	+ /// Allow the allocation to be in high memory.
	63	+ ///
	64	+ /// Allocations in high memory may not be mapped into the kernel's address space, so this can't
	65	+ /// be used with `kmalloc` and other similar methods.
	66	+ ///
	67	+ /// This is normally or'd with other flags.
	68	+ pub const __GFP_HIGHMEM: Flags = Flags(bindings::__GFP_HIGHMEM);
	69	+
55	70	/// Users can not sleep and need the allocation to succeed.
56	71	///
57	72	/// A lower watermark is applied to allow access to "atomic reserves". The current
@@ -66,7 +81,7 @@ pub mod flags { · 8 unmodified lines
66	81	/// The same as [`GFP_KERNEL`], except the allocation is accounted to kmemcg.
67	82	pub const GFP_KERNEL_ACCOUNT: Flags = Flags(bindings::GFP_KERNEL_ACCOUNT);
68	83
69		− /// Ror kernel allocations that should not stall for direct reclaim, start physical IO or
	84	+ /// For kernel allocations that should not stall for direct reclaim, start physical IO or
70	85	/// use any filesystem callback. It is very likely to fail to allocate memory, even for very
71	86	/// small allocations.
72	87	pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);

rust/kernel/init.rs+4−9

@@ -843,11 +843,8 @@ where · 842 unmodified lines
843	843	let val = unsafe { &mut *slot };
844	844	// SAFETY: `slot` is considered pinned.
845	845	let val = unsafe { Pin::new_unchecked(val) };
846		− (self.1)(val).map_err(\|e\| {
847		− // SAFETY: `slot` was initialized above.
848		− unsafe { core::ptr::drop_in_place(slot) };
849		− e
850		− })
	846	+ // SAFETY: `slot` was initialized above.
	847	+ (self.1)(val).inspect_err(\|_\| unsafe { core::ptr::drop_in_place(slot) })
851	848	}
852	849	}
853	850
@@ -941,11 +938,9 @@ where · 87 unmodified lines
941	938	// SAFETY: All requirements fulfilled since this function is `__init`.
942	939	unsafe { self.0.__pinned_init(slot)? };
943	940	// SAFETY: The above call initialized `slot` and we still have unique access.
944		− (self.1)(unsafe { &mut *slot }).map_err(\|e\| {
	941	+ (self.1)(unsafe { &mut *slot }).inspect_err(\|_\|
945	942	// SAFETY: `slot` was initialized above.
946		− unsafe { core::ptr::drop_in_place(slot) };
947		− e
948		− })
	943	+ unsafe { core::ptr::drop_in_place(slot) })
949	944	}
950	945	}
951	946

rust/kernel/lib.rs+2−0

@@ -40,6 +40,7 @@ pub mod ioctl; · 39 unmodified lines
40	40	pub mod kunit;
41	41	#[cfg(CONFIG_NET)]
42	42	pub mod net;
	43	+pub mod page;
43	44	pub mod prelude;
44	45	pub mod print;
45	46	mod static_assert;
@@ -50,6 +51,7 @@ pub mod sync; · 4 unmodified lines
50	51	pub mod task;
51	52	pub mod time;
52	53	pub mod types;
	54	+pub mod uaccess;
53	55	pub mod workqueue;
54	56
55	57	#[doc(hidden)]

rust/kernel/page.rs+250−0

@@ -0,0 +1,250 @@
	1	+// SPDX-License-Identifier: GPL-2.0
	2	+
	3	+//! Kernel page allocation and management.
	4	+
	5	+use crate::{
	6	+ alloc::{AllocError, Flags},
	7	+ bindings,
	8	+ error::code::*,
	9	+ error::Result,
	10	+ uaccess::UserSliceReader,
	11	+};
	12	+use core::ptr::{self, NonNull};
	13	+
	14	+/// A bitwise shift for the page size.
	15	+pub const PAGE_SHIFT: usize = bindings::PAGE_SHIFT as usize;
	16	+
	17	+/// The number of bytes in a page.
	18	+pub const PAGE_SIZE: usize = bindings::PAGE_SIZE;
	19	+
	20	+/// A bitmask that gives the page containing a given address.
	21	+pub const PAGE_MASK: usize = !(PAGE_SIZE - 1);
	22	+
	23	+/// A pointer to a page that owns the page allocation.
	24	+///
	25	+/// # Invariants
	26	+///
	27	+/// The pointer is valid, and has ownership over the page.
	28	+pub struct Page {
	29	+ page: NonNull<bindings::page>,
	30	+}
	31	+
	32	+// SAFETY: Pages have no logic that relies on them staying on a given thread, so moving them across
	33	+// threads is safe.
	34	+unsafe impl Send for Page {}
	35	+
	36	+// SAFETY: Pages have no logic that relies on them not being accessed concurrently, so accessing
	37	+// them concurrently is safe.
	38	+unsafe impl Sync for Page {}
	39	+
	40	+impl Page {
	41	+ /// Allocates a new page.
	42	+ ///
	43	+ /// # Examples
	44	+ ///
	45	+ /// Allocate memory for a page.
	46	+ ///
	47	+ /// ```
	48	+ /// use kernel::page::Page;
	49	+ ///
	50	+ /// # fn dox() -> Result<(), kernel::alloc::AllocError> {
	51	+ /// let page = Page::alloc_page(GFP_KERNEL)?;
	52	+ /// # Ok(()) }
	53	+ /// ```
	54	+ ///
	55	+ /// Allocate memory for a page and zero its contents.
	56	+ ///
	57	+ /// ```
	58	+ /// use kernel::page::Page;
	59	+ ///
	60	+ /// # fn dox() -> Result<(), kernel::alloc::AllocError> {
	61	+ /// let page = Page::alloc_page(GFP_KERNEL \| __GFP_ZERO)?;
	62	+ /// # Ok(()) }
	63	+ /// ```
	64	+ pub fn alloc_page(flags: Flags) -> Result<Self, AllocError> {
	65	+ // SAFETY: Depending on the value of `gfp_flags`, this call may sleep. Other than that, it
	66	+ // is always safe to call this method.
	67	+ let page = unsafe { bindings::alloc_pages(flags.as_raw(), 0) };
	68	+ let page = NonNull::new(page).ok_or(AllocError)?;
	69	+ // INVARIANT: We just successfully allocated a page, so we now have ownership of the newly
	70	+ // allocated page. We transfer that ownership to the new `Page` object.
	71	+ Ok(Self { page })
	72	+ }
	73	+
	74	+ /// Returns a raw pointer to the page.
	75	+ pub fn as_ptr(&self) -> *mut bindings::page {
	76	+ self.page.as_ptr()
	77	+ }
	78	+
	79	+ /// Runs a piece of code with this page mapped to an address.
	80	+ ///
	81	+ /// The page is unmapped when this call returns.
	82	+ ///
	83	+ /// # Using the raw pointer
	84	+ ///
	85	+ /// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for
	86	+ /// `PAGE_SIZE` bytes and for the duration in which the closure is called. The pointer might
	87	+ /// only be mapped on the current thread, and when that is the case, dereferencing it on other
	88	+ /// threads is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't
	89	+ /// cause data races, the memory may be uninitialized, and so on.
	90	+ ///
	91	+ /// If multiple threads map the same page at the same time, then they may reference with
	92	+ /// different addresses. However, even if the addresses are different, the underlying memory is
	93	+ /// still the same for these purposes (e.g., it's still a data race if they both write to the
	94	+ /// same underlying byte at the same time).
	95	+ fn with_page_mapped<T>(&self, f: impl FnOnce(*mut u8) -> T) -> T {
	96	+ // SAFETY: `page` is valid due to the type invariants on `Page`.
	97	+ let mapped_addr = unsafe { bindings::kmap_local_page(self.as_ptr()) };
	98	+
	99	+ let res = f(mapped_addr.cast());
	100	+
	101	+ // This unmaps the page mapped above.
	102	+ //
	103	+ // SAFETY: Since this API takes the user code as a closure, it can only be used in a manner
	104	+ // where the pages are unmapped in reverse order. This is as required by `kunmap_local`.
	105	+ //
	106	+ // In other words, if this call to `kunmap_local` happens when a different page should be
	107	+ // unmapped first, then there must necessarily be a call to `kmap_local_page` other than the
	108	+ // call just above in `with_page_mapped` that made that possible. In this case, it is the
	109	+ // unsafe block that wraps that other call that is incorrect.
	110	+ unsafe { bindings::kunmap_local(mapped_addr) };
	111	+
	112	+ res
	113	+ }
	114	+
	115	+ /// Runs a piece of code with a raw pointer to a slice of this page, with bounds checking.
	116	+ ///
	117	+ /// If `f` is called, then it will be called with a pointer that points at `off` bytes into the
	118	+ /// page, and the pointer will be valid for at least `len` bytes. The pointer is only valid on
	119	+ /// this task, as this method uses a local mapping.
	120	+ ///
	121	+ /// If `off` and `len` refers to a region outside of this page, then this method returns
	122	+ /// [`EINVAL`] and does not call `f`.
	123	+ ///
	124	+ /// # Using the raw pointer
	125	+ ///
	126	+ /// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for
	127	+ /// `len` bytes and for the duration in which the closure is called. The pointer might only be
	128	+ /// mapped on the current thread, and when that is the case, dereferencing it on other threads
	129	+ /// is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't cause
	130	+ /// data races, the memory may be uninitialized, and so on.
	131	+ ///
	132	+ /// If multiple threads map the same page at the same time, then they may reference with
	133	+ /// different addresses. However, even if the addresses are different, the underlying memory is
	134	+ /// still the same for these purposes (e.g., it's still a data race if they both write to the
	135	+ /// same underlying byte at the same time).
	136	+ fn with_pointer_into_page<T>(
	137	+ &self,
	138	+ off: usize,
	139	+ len: usize,
	140	+ f: impl FnOnce(*mut u8) -> Result<T>,
	141	+ ) -> Result<T> {
	142	+ let bounds_ok = off <= PAGE_SIZE && len <= PAGE_SIZE && (off + len) <= PAGE_SIZE;
	143	+
	144	+ if bounds_ok {
	145	+ self.with_page_mapped(move \|page_addr\| {
	146	+ // SAFETY: The `off` integer is at most `PAGE_SIZE`, so this pointer offset will
	147	+ // result in a pointer that is in bounds or one off the end of the page.
	148	+ f(unsafe { page_addr.add(off) })
	149	+ })
	150	+ } else {
	151	+ Err(EINVAL)
	152	+ }
	153	+ }
	154	+
	155	+ /// Maps the page and reads from it into the given buffer.
	156	+ ///
	157	+ /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	158	+ /// outside of the page, then this call returns [`EINVAL`].
	159	+ ///
	160	+ /// # Safety
	161	+ ///
	162	+ /// * Callers must ensure that `dst` is valid for writing `len` bytes.
	163	+ /// * Callers must ensure that this call does not race with a write to the same page that
	164	+ /// overlaps with this read.
	165	+ pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result {
	166	+ self.with_pointer_into_page(offset, len, move \|src\| {
	167	+ // SAFETY: If `with_pointer_into_page` calls into this closure, then
	168	+ // it has performed a bounds check and guarantees that `src` is
	169	+ // valid for `len` bytes.
	170	+ //
	171	+ // There caller guarantees that there is no data race.
	172	+ unsafe { ptr::copy_nonoverlapping(src, dst, len) };
	173	+ Ok(())
	174	+ })
	175	+ }
	176	+
	177	+ /// Maps the page and writes into it from the given buffer.
	178	+ ///
	179	+ /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	180	+ /// outside of the page, then this call returns [`EINVAL`].
	181	+ ///
	182	+ /// # Safety
	183	+ ///
	184	+ /// * Callers must ensure that `src` is valid for reading `len` bytes.
	185	+ /// * Callers must ensure that this call does not race with a read or write to the same page
	186	+ /// that overlaps with this write.
	187	+ pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Result {
	188	+ self.with_pointer_into_page(offset, len, move \|dst\| {
	189	+ // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	190	+ // bounds check and guarantees that `dst` is valid for `len` bytes.
	191	+ //
	192	+ // There caller guarantees that there is no data race.
	193	+ unsafe { ptr::copy_nonoverlapping(src, dst, len) };
	194	+ Ok(())
	195	+ })
	196	+ }
	197	+
	198	+ /// Maps the page and zeroes the given slice.
	199	+ ///
	200	+ /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	201	+ /// outside of the page, then this call returns [`EINVAL`].
	202	+ ///
	203	+ /// # Safety
	204	+ ///
	205	+ /// Callers must ensure that this call does not race with a read or write to the same page that
	206	+ /// overlaps with this write.
	207	+ pub unsafe fn fill_zero_raw(&self, offset: usize, len: usize) -> Result {
	208	+ self.with_pointer_into_page(offset, len, move \|dst\| {
	209	+ // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	210	+ // bounds check and guarantees that `dst` is valid for `len` bytes.
	211	+ //
	212	+ // There caller guarantees that there is no data race.
	213	+ unsafe { ptr::write_bytes(dst, 0u8, len) };
	214	+ Ok(())
	215	+ })
	216	+ }
	217	+
	218	+ /// Copies data from userspace into this page.
	219	+ ///
	220	+ /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	221	+ /// outside of the page, then this call returns [`EINVAL`].
	222	+ ///
	223	+ /// Like the other `UserSliceReader` methods, data races are allowed on the userspace address.
	224	+ /// However, they are not allowed on the page you are copying into.
	225	+ ///
	226	+ /// # Safety
	227	+ ///
	228	+ /// Callers must ensure that this call does not race with a read or write to the same page that
	229	+ /// overlaps with this write.
	230	+ pub unsafe fn copy_from_user_slice_raw(
	231	+ &self,
	232	+ reader: &mut UserSliceReader,
	233	+ offset: usize,
	234	+ len: usize,
	235	+ ) -> Result {
	236	+ self.with_pointer_into_page(offset, len, move \|dst\| {
	237	+ // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	238	+ // bounds check and guarantees that `dst` is valid for `len` bytes. Furthermore, we have
	239	+ // exclusive access to the slice since the caller guarantees that there are no races.
	240	+ reader.read_raw(unsafe { core::slice::from_raw_parts_mut(dst.cast(), len) })
	241	+ })
	242	+ }
	243	+}
	244	+
	245	+impl Drop for Page {
	246	+ fn drop(&mut self) {
	247	+ // SAFETY: By the type invariants, we have ownership of the page and can free it.
	248	+ unsafe { bindings::__free_pages(self.page.as_ptr(), 0) };
	249	+ }
	250	+}

rust/kernel/types.rs+64−0

@@ -409,3 +409,67 @@ pub enum Either<L, R> { · 408 unmodified lines
409	409	/// Constructs an instance of [`Either`] containing a value of type `R`.
410	410	Right(R),
411	411	}
	412	+
	413	+/// Types for which any bit pattern is valid.
	414	+///
	415	+/// Not all types are valid for all values. For example, a `bool` must be either zero or one, so
	416	+/// reading arbitrary bytes into something that contains a `bool` is not okay.
	417	+///
	418	+/// It's okay for the type to have padding, as initializing those bytes has no effect.
	419	+///
	420	+/// # Safety
	421	+///
	422	+/// All bit-patterns must be valid for this type. This type must not have interior mutability.
	423	+pub unsafe trait FromBytes {}
	424	+
	425	+// SAFETY: All bit patterns are acceptable values of the types below.
	426	+unsafe impl FromBytes for u8 {}
	427	+unsafe impl FromBytes for u16 {}
	428	+unsafe impl FromBytes for u32 {}
	429	+unsafe impl FromBytes for u64 {}
	430	+unsafe impl FromBytes for usize {}
	431	+unsafe impl FromBytes for i8 {}
	432	+unsafe impl FromBytes for i16 {}
	433	+unsafe impl FromBytes for i32 {}
	434	+unsafe impl FromBytes for i64 {}
	435	+unsafe impl FromBytes for isize {}
	436	+// SAFETY: If all bit patterns are acceptable for individual values in an array, then all bit
	437	+// patterns are also acceptable for arrays of that type.
	438	+unsafe impl<T: FromBytes> FromBytes for [T] {}
	439	+unsafe impl<T: FromBytes, const N: usize> FromBytes for [T; N] {}
	440	+
	441	+/// Types that can be viewed as an immutable slice of initialized bytes.
	442	+///
	443	+/// If a struct implements this trait, then it is okay to copy it byte-for-byte to userspace. This
	444	+/// means that it should not have any padding, as padding bytes are uninitialized. Reading
	445	+/// uninitialized memory is not just undefined behavior, it may even lead to leaking sensitive
	446	+/// information on the stack to userspace.
	447	+///
	448	+/// The struct should also not hold kernel pointers, as kernel pointer addresses are also considered
	449	+/// sensitive. However, leaking kernel pointers is not considered undefined behavior by Rust, so
	450	+/// this is a correctness requirement, but not a safety requirement.
	451	+///
	452	+/// # Safety
	453	+///
	454	+/// Values of this type may not contain any uninitialized bytes. This type must not have interior
	455	+/// mutability.
	456	+pub unsafe trait AsBytes {}
	457	+
	458	+// SAFETY: Instances of the following types have no uninitialized portions.
	459	+unsafe impl AsBytes for u8 {}
	460	+unsafe impl AsBytes for u16 {}
	461	+unsafe impl AsBytes for u32 {}
	462	+unsafe impl AsBytes for u64 {}
	463	+unsafe impl AsBytes for usize {}
	464	+unsafe impl AsBytes for i8 {}
	465	+unsafe impl AsBytes for i16 {}
	466	+unsafe impl AsBytes for i32 {}
	467	+unsafe impl AsBytes for i64 {}
	468	+unsafe impl AsBytes for isize {}
	469	+unsafe impl AsBytes for bool {}
	470	+unsafe impl AsBytes for char {}
	471	+unsafe impl AsBytes for str {}
	472	+// SAFETY: If individual values in an array have no uninitialized portions, then the array itself
	473	+// does not have any uninitialized portions either.
	474	+unsafe impl<T: AsBytes> AsBytes for [T] {}
	475	+unsafe impl<T: AsBytes, const N: usize> AsBytes for [T; N] {}

rust/kernel/uaccess.rs+388−0

@@ -0,0 +1,388 @@
	1	+// SPDX-License-Identifier: GPL-2.0
	2	+
	3	+//! Slices to user space memory regions.
	4	+//!
	5	+//! C header: [`include/linux/uaccess.h`](srctree/include/linux/uaccess.h)
	6	+
	7	+use crate::{
	8	+ alloc::Flags,
	9	+ bindings,
	10	+ error::Result,
	11	+ prelude::*,
	12	+ types::{AsBytes, FromBytes},
	13	+};
	14	+use alloc::vec::Vec;
	15	+use core::ffi::{c_ulong, c_void};
	16	+use core::mem::{size_of, MaybeUninit};
	17	+
	18	+/// The type used for userspace addresses.
	19	+pub type UserPtr = usize;
	20	+
	21	+/// A pointer to an area in userspace memory, which can be either read-only or read-write.
	22	+///
	23	+/// All methods on this struct are safe: attempting to read or write on bad addresses (either out of
	24	+/// the bound of the slice or unmapped addresses) will return [`EFAULT`]. Concurrent access,
	25	+/// including data races to/from userspace memory, is permitted, because fundamentally another
	26	+/// userspace thread/process could always be modifying memory at the same time (in the same way that
	27	+/// userspace Rust's [`std::io`] permits data races with the contents of files on disk). In the
	28	+/// presence of a race, the exact byte values read/written are unspecified but the operation is
	29	+/// well-defined. Kernelspace code should validate its copy of data after completing a read, and not
	30	+/// expect that multiple reads of the same address will return the same value.
	31	+///
	32	+/// These APIs are designed to make it difficult to accidentally write TOCTOU (time-of-check to
	33	+/// time-of-use) bugs. Every time a memory location is read, the reader's position is advanced by
	34	+/// the read length and the next read will start from there. This helps prevent accidentally reading
	35	+/// the same location twice and causing a TOCTOU bug.
	36	+///
	37	+/// Creating a [`UserSliceReader`] and/or [`UserSliceWriter`] consumes the `UserSlice`, helping
	38	+/// ensure that there aren't multiple readers or writers to the same location.
	39	+///
	40	+/// If double-fetching a memory location is necessary for some reason, then that is done by creating
	41	+/// multiple readers to the same memory location, e.g. using [`clone_reader`].
	42	+///
	43	+/// # Examples
	44	+///
	45	+/// Takes a region of userspace memory from the current process, and modify it by adding one to
	46	+/// every byte in the region.
	47	+///
	48	+/// ```no_run
	49	+/// use alloc::vec::Vec;
	50	+/// use core::ffi::c_void;
	51	+/// use kernel::error::Result;
	52	+/// use kernel::uaccess::{UserPtr, UserSlice};
	53	+///
	54	+/// fn bytes_add_one(uptr: UserPtr, len: usize) -> Result<()> {
	55	+/// let (read, mut write) = UserSlice::new(uptr, len).reader_writer();
	56	+///
	57	+/// let mut buf = Vec::new();
	58	+/// read.read_all(&mut buf, GFP_KERNEL)?;
	59	+///
	60	+/// for b in &mut buf {
	61	+/// *b = b.wrapping_add(1);
	62	+/// }
	63	+///
	64	+/// write.write_slice(&buf)?;
	65	+/// Ok(())
	66	+/// }
	67	+/// ```
	68	+///
	69	+/// Example illustrating a TOCTOU (time-of-check to time-of-use) bug.
	70	+///
	71	+/// ```no_run
	72	+/// use alloc::vec::Vec;
	73	+/// use core::ffi::c_void;
	74	+/// use kernel::error::{code::EINVAL, Result};
	75	+/// use kernel::uaccess::{UserPtr, UserSlice};
	76	+///
	77	+/// /// Returns whether the data in this region is valid.
	78	+/// fn is_valid(uptr: UserPtr, len: usize) -> Result<bool> {
	79	+/// let read = UserSlice::new(uptr, len).reader();
	80	+///
	81	+/// let mut buf = Vec::new();
	82	+/// read.read_all(&mut buf, GFP_KERNEL)?;
	83	+///
	84	+/// todo!()
	85	+/// }
	86	+///
	87	+/// /// Returns the bytes behind this user pointer if they are valid.
	88	+/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<Vec<u8>> {
	89	+/// if !is_valid(uptr, len)? {
	90	+/// return Err(EINVAL);
	91	+/// }
	92	+///
	93	+/// let read = UserSlice::new(uptr, len).reader();
	94	+///
	95	+/// let mut buf = Vec::new();
	96	+/// read.read_all(&mut buf, GFP_KERNEL)?;
	97	+///
	98	+/// // THIS IS A BUG! The bytes could have changed since we checked them.
	99	+/// //
	100	+/// // To avoid this kind of bug, don't call `UserSlice::new` multiple
	101	+/// // times with the same address.
	102	+/// Ok(buf)
	103	+/// }
	104	+/// ```
	105	+///
	106	+/// [`std::io`]: https://doc.rust-lang.org/std/io/index.html
	107	+/// [`clone_reader`]: UserSliceReader::clone_reader
	108	+pub struct UserSlice {
	109	+ ptr: UserPtr,
	110	+ length: usize,
	111	+}
	112	+
	113	+impl UserSlice {
	114	+ /// Constructs a user slice from a raw pointer and a length in bytes.
	115	+ ///
	116	+ /// Constructing a [`UserSlice`] performs no checks on the provided address and length, it can
	117	+ /// safely be constructed inside a kernel thread with no current userspace process. Reads and
	118	+ /// writes wrap the kernel APIs `copy_from_user` and `copy_to_user`, which check the memory map
	119	+ /// of the current process and enforce that the address range is within the user range (no
	120	+ /// additional calls to `access_ok` are needed). Validity of the pointer is checked when you
	121	+ /// attempt to read or write, not in the call to `UserSlice::new`.
	122	+ ///
	123	+ /// Callers must be careful to avoid time-of-check-time-of-use (TOCTOU) issues. The simplest way
	124	+ /// is to create a single instance of [`UserSlice`] per user memory block as it reads each byte
	125	+ /// at most once.
	126	+ pub fn new(ptr: UserPtr, length: usize) -> Self {
	127	+ UserSlice { ptr, length }
	128	+ }
	129	+
	130	+ /// Reads the entirety of the user slice, appending it to the end of the provided buffer.
	131	+ ///
	132	+ /// Fails with [`EFAULT`] if the read happens on a bad address.
	133	+ pub fn read_all(self, buf: &mut Vec<u8>, flags: Flags) -> Result {
	134	+ self.reader().read_all(buf, flags)
	135	+ }
	136	+
	137	+ /// Constructs a [`UserSliceReader`].
	138	+ pub fn reader(self) -> UserSliceReader {
	139	+ UserSliceReader {
	140	+ ptr: self.ptr,
	141	+ length: self.length,
	142	+ }
	143	+ }
	144	+
	145	+ /// Constructs a [`UserSliceWriter`].
	146	+ pub fn writer(self) -> UserSliceWriter {
	147	+ UserSliceWriter {
	148	+ ptr: self.ptr,
	149	+ length: self.length,
	150	+ }
	151	+ }
	152	+
	153	+ /// Constructs both a [`UserSliceReader`] and a [`UserSliceWriter`].
	154	+ ///
	155	+ /// Usually when this is used, you will first read the data, and then overwrite it afterwards.
	156	+ pub fn reader_writer(self) -> (UserSliceReader, UserSliceWriter) {
	157	+ (
	158	+ UserSliceReader {
	159	+ ptr: self.ptr,
	160	+ length: self.length,
	161	+ },
	162	+ UserSliceWriter {
	163	+ ptr: self.ptr,
	164	+ length: self.length,
	165	+ },
	166	+ )
	167	+ }
	168	+}
	169	+
	170	+/// A reader for [`UserSlice`].
	171	+///
	172	+/// Used to incrementally read from the user slice.
	173	+pub struct UserSliceReader {
	174	+ ptr: UserPtr,
	175	+ length: usize,
	176	+}
	177	+
	178	+impl UserSliceReader {
	179	+ /// Skip the provided number of bytes.
	180	+ ///
	181	+ /// Returns an error if skipping more than the length of the buffer.
	182	+ pub fn skip(&mut self, num_skip: usize) -> Result {
	183	+ // Update `self.length` first since that's the fallible part of this operation.
	184	+ self.length = self.length.checked_sub(num_skip).ok_or(EFAULT)?;
	185	+ self.ptr = self.ptr.wrapping_add(num_skip);
	186	+ Ok(())
	187	+ }
	188	+
	189	+ /// Create a reader that can access the same range of data.
	190	+ ///
	191	+ /// Reading from the clone does not advance the current reader.
	192	+ ///
	193	+ /// The caller should take care to not introduce TOCTOU issues, as described in the
	194	+ /// documentation for [`UserSlice`].
	195	+ pub fn clone_reader(&self) -> UserSliceReader {
	196	+ UserSliceReader {
	197	+ ptr: self.ptr,
	198	+ length: self.length,
	199	+ }
	200	+ }
	201	+
	202	+ /// Returns the number of bytes left to be read from this reader.
	203	+ ///
	204	+ /// Note that even reading less than this number of bytes may fail.
	205	+ pub fn len(&self) -> usize {
	206	+ self.length
	207	+ }
	208	+
	209	+ /// Returns `true` if no data is available in the io buffer.
	210	+ pub fn is_empty(&self) -> bool {
	211	+ self.length == 0
	212	+ }
	213	+
	214	+ /// Reads raw data from the user slice into a kernel buffer.
	215	+ ///
	216	+ /// For a version that uses `&mut [u8]`, please see [`UserSliceReader::read_slice`].
	217	+ ///
	218	+ /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	219	+ /// bounds of this [`UserSliceReader`]. This call may modify `out` even if it returns an error.
	220	+ ///
	221	+ /// # Guarantees
	222	+ ///
	223	+ /// After a successful call to this method, all bytes in `out` are initialized.
	224	+ pub fn read_raw(&mut self, out: &mut [MaybeUninit<u8>]) -> Result {
	225	+ let len = out.len();
	226	+ let out_ptr = out.as_mut_ptr().cast::<c_void>();
	227	+ if len > self.length {
	228	+ return Err(EFAULT);
	229	+ }
	230	+ let Ok(len_ulong) = c_ulong::try_from(len) else {
	231	+ return Err(EFAULT);
	232	+ };
	233	+ // SAFETY: `out_ptr` points into a mutable slice of length `len_ulong`, so we may write
	234	+ // that many bytes to it.
	235	+ let res =
	236	+ unsafe { bindings::copy_from_user(out_ptr, self.ptr as *const c_void, len_ulong) };
	237	+ if res != 0 {
	238	+ return Err(EFAULT);
	239	+ }
	240	+ self.ptr = self.ptr.wrapping_add(len);
	241	+ self.length -= len;
	242	+ Ok(())
	243	+ }
	244	+
	245	+ /// Reads raw data from the user slice into a kernel buffer.
	246	+ ///
	247	+ /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	248	+ /// bounds of this [`UserSliceReader`]. This call may modify `out` even if it returns an error.
	249	+ pub fn read_slice(&mut self, out: &mut [u8]) -> Result {
	250	+ // SAFETY: The types are compatible and `read_raw` doesn't write uninitialized bytes to
	251	+ // `out`.
	252	+ let out = unsafe { &mut (out as mut [u8] as *mut [MaybeUninit<u8>]) };
	253	+ self.read_raw(out)
	254	+ }
	255	+
	256	+ /// Reads a value of the specified type.
	257	+ ///
	258	+ /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	259	+ /// bounds of this [`UserSliceReader`].
	260	+ pub fn read<T: FromBytes>(&mut self) -> Result<T> {
	261	+ let len = size_of::<T>();
	262	+ if len > self.length {
	263	+ return Err(EFAULT);
	264	+ }
	265	+ let Ok(len_ulong) = c_ulong::try_from(len) else {
	266	+ return Err(EFAULT);
	267	+ };
	268	+ let mut out: MaybeUninit<T> = MaybeUninit::uninit();
	269	+ // SAFETY: The local variable `out` is valid for writing `size_of::<T>()` bytes.
	270	+ //
	271	+ // By using the _copy_from_user variant, we skip the check_object_size check that verifies
	272	+ // the kernel pointer. This mirrors the logic on the C side that skips the check when the
	273	+ // length is a compile-time constant.
	274	+ let res = unsafe {
	275	+ bindings::_copy_from_user(
	276	+ out.as_mut_ptr().cast::<c_void>(),
	277	+ self.ptr as *const c_void,
	278	+ len_ulong,
	279	+ )
	280	+ };
	281	+ if res != 0 {
	282	+ return Err(EFAULT);
	283	+ }
	284	+ self.ptr = self.ptr.wrapping_add(len);
	285	+ self.length -= len;
	286	+ // SAFETY: The read above has initialized all bytes in `out`, and since `T` implements
	287	+ // `FromBytes`, any bit-pattern is a valid value for this type.
	288	+ Ok(unsafe { out.assume_init() })
	289	+ }
	290	+
	291	+ /// Reads the entirety of the user slice, appending it to the end of the provided buffer.
	292	+ ///
	293	+ /// Fails with [`EFAULT`] if the read happens on a bad address.
	294	+ pub fn read_all(mut self, buf: &mut Vec<u8>, flags: Flags) -> Result {
	295	+ let len = self.length;
	296	+ VecExt::<u8>::reserve(buf, len, flags)?;
	297	+
	298	+ // The call to `try_reserve` was successful, so the spare capacity is at least `len` bytes
	299	+ // long.
	300	+ self.read_raw(&mut buf.spare_capacity_mut()[..len])?;
	301	+
	302	+ // SAFETY: Since the call to `read_raw` was successful, so the next `len` bytes of the
	303	+ // vector have been initialized.
	304	+ unsafe { buf.set_len(buf.len() + len) };
	305	+ Ok(())
	306	+ }
	307	+}
	308	+
	309	+/// A writer for [`UserSlice`].
	310	+///
	311	+/// Used to incrementally write into the user slice.
	312	+pub struct UserSliceWriter {
	313	+ ptr: UserPtr,
	314	+ length: usize,
	315	+}
	316	+
	317	+impl UserSliceWriter {
	318	+ /// Returns the amount of space remaining in this buffer.
	319	+ ///
	320	+ /// Note that even writing less than this number of bytes may fail.
	321	+ pub fn len(&self) -> usize {
	322	+ self.length
	323	+ }
	324	+
	325	+ /// Returns `true` if no more data can be written to this buffer.
	326	+ pub fn is_empty(&self) -> bool {
	327	+ self.length == 0
	328	+ }
	329	+
	330	+ /// Writes raw data to this user pointer from a kernel buffer.
	331	+ ///
	332	+ /// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
	333	+ /// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
	334	+ /// if it returns an error.
	335	+ pub fn write_slice(&mut self, data: &[u8]) -> Result {
	336	+ let len = data.len();
	337	+ let data_ptr = data.as_ptr().cast::<c_void>();
	338	+ if len > self.length {
	339	+ return Err(EFAULT);
	340	+ }
	341	+ let Ok(len_ulong) = c_ulong::try_from(len) else {
	342	+ return Err(EFAULT);
	343	+ };
	344	+ // SAFETY: `data_ptr` points into an immutable slice of length `len_ulong`, so we may read
	345	+ // that many bytes from it.
	346	+ let res = unsafe { bindings::copy_to_user(self.ptr as *mut c_void, data_ptr, len_ulong) };
	347	+ if res != 0 {
	348	+ return Err(EFAULT);
	349	+ }
	350	+ self.ptr = self.ptr.wrapping_add(len);
	351	+ self.length -= len;
	352	+ Ok(())
	353	+ }
	354	+
	355	+ /// Writes the provided Rust value to this userspace pointer.
	356	+ ///
	357	+ /// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
	358	+ /// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
	359	+ /// if it returns an error.
	360	+ pub fn write<T: AsBytes>(&mut self, value: &T) -> Result {
	361	+ let len = size_of::<T>();
	362	+ if len > self.length {
	363	+ return Err(EFAULT);
	364	+ }
	365	+ let Ok(len_ulong) = c_ulong::try_from(len) else {
	366	+ return Err(EFAULT);
	367	+ };
	368	+ // SAFETY: The reference points to a value of type `T`, so it is valid for reading
	369	+ // `size_of::<T>()` bytes.
	370	+ //
	371	+ // By using the _copy_to_user variant, we skip the check_object_size check that verifies the
	372	+ // kernel pointer. This mirrors the logic on the C side that skips the check when the length
	373	+ // is a compile-time constant.
	374	+ let res = unsafe {
	375	+ bindings::_copy_to_user(
	376	+ self.ptr as *mut c_void,
	377	+ (value as *const T).cast::<c_void>(),
	378	+ len_ulong,
	379	+ )
	380	+ };
	381	+ if res != 0 {
	382	+ return Err(EFAULT);
	383	+ }
	384	+ self.ptr = self.ptr.wrapping_add(len);
	385	+ self.length -= len;
	386	+ Ok(())
	387	+ }
	388	+}

rust/kernel/workqueue.rs+9−7

@@ -482,24 +482,26 @@ pub unsafe trait HasWork<T, const ID: u64 = 0> { · 481 unmodified lines
482	482	/// use kernel::sync::Arc;
483	483	/// use kernel::workqueue::{self, impl_has_work, Work};
484	484	///
485		−/// struct MyStruct {
486		−/// work_field: Work<MyStruct, 17>,
	485	+/// struct MyStruct<'a, T, const N: usize> {
	486	+/// work_field: Work<MyStruct<'a, T, N>, 17>,
	487	+/// f: fn(&'a [T; N]),
487	488	/// }
488	489	///
489	490	/// impl_has_work! {
490		−/// impl HasWork<MyStruct, 17> for MyStruct { self.work_field }
	491	+/// impl{'a, T, const N: usize} HasWork<MyStruct<'a, T, N>, 17>
	492	+/// for MyStruct<'a, T, N> { self.work_field }
491	493	/// }
492	494	/// ```
493	495	#[macro_export]
494	496	macro_rules! impl_has_work {
495		− ($(impl$(<$($implarg:ident),*>)?
	497	+ ($(impl$({$($generics:tt)*})?
496	498	HasWork<$work_type:ty $(, $id:tt)?>
497		− for $self:ident $(<$($selfarg:ident),*>)?
	499	+ for $self:ty
498	500	{ self.$field:ident }
499	501	)*) => {$(
500	502	// SAFETY: The implementation of `raw_get_work` only compiles if the field has the right
501	503	// type.
502		− unsafe impl$(<$($implarg),>)? $crate::workqueue::HasWork<$work_type $(, $id)?> for $self $(<$($selfarg),>)? {
	504	+ unsafe impl$(<$($generics)+>)? $crate::workqueue::HasWork<$work_type $(, $id)?> for $self {
503	505	const OFFSET: usize = ::core::mem::offset_of!(Self, $field) as usize;
504	506
505	507	#[inline]
@@ -515,7 +517,7 @@ macro_rules! impl_has_work { · 9 unmodified lines
515	517	pub use impl_has_work;
516	518
517	519	impl_has_work! {
518		− impl<T> HasWork<Self> for ClosureWork<T> { self.work }
	520	+ impl{T} HasWork<Self> for ClosureWork<T> { self.work }
519	521	}
520	522
521	523	unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Arc<T>

rust/macros/lib.rs+39−6

@@ -35,6 +35,7 @@ use proc_macro::TokenStream; · 34 unmodified lines
35	35	/// author: "Rust for Linux Contributors",
36	36	/// description: "My very own kernel module!",
37	37	/// license: "GPL",
	38	+/// alias: ["alternate_module_name"],
38	39	/// }
39	40	///
40	41	/// struct MyModule;
@@ -55,13 +56,45 @@ use proc_macro::TokenStream; · 14 unmodified lines
55	56	/// }
56	57	/// ```
57	58	///
	59	+/// ## Firmware
	60	+///
	61	+/// The following example shows how to declare a kernel module that needs
	62	+/// to load binary firmware files. You need to specify the file names of
	63	+/// the firmware in the `firmware` field. The information is embedded
	64	+/// in the `modinfo` section of the kernel module. For example, a tool to
	65	+/// build an initramfs uses this information to put the firmware files into
	66	+/// the initramfs image.
	67	+///
	68	+/// ```ignore
	69	+/// use kernel::prelude::*;
	70	+///
	71	+/// module!{
	72	+/// type: MyDeviceDriverModule,
	73	+/// name: "my_device_driver_module",
	74	+/// author: "Rust for Linux Contributors",
	75	+/// description: "My device driver requires firmware",
	76	+/// license: "GPL",
	77	+/// firmware: ["my_device_firmware1.bin", "my_device_firmware2.bin"],
	78	+/// }
	79	+///
	80	+/// struct MyDeviceDriverModule;
	81	+///
	82	+/// impl kernel::Module for MyDeviceDriverModule {
	83	+/// fn init() -> Result<Self> {
	84	+/// Ok(Self)
	85	+/// }
	86	+/// }
	87	+/// ```
	88	+///
58	89	/// # Supported argument types
59	90	/// - `type`: type which implements the [`Module`] trait (required).
60		−/// - `name`: byte array of the name of the kernel module (required).
61		−/// - `author`: byte array of the author of the kernel module.
62		−/// - `description`: byte array of the description of the kernel module.
63		−/// - `license`: byte array of the license of the kernel module (required).
64		−/// - `alias`: byte array of alias name of the kernel module.
	91	+/// - `name`: ASCII string literal of the name of the kernel module (required).
	92	+/// - `author`: string literal of the author of the kernel module.
	93	+/// - `description`: string literal of the description of the kernel module.
	94	+/// - `license`: ASCII string literal of the license of the kernel module (required).
	95	+/// - `alias`: array of ASCII string literals of the alias names of the kernel module.
	96	+/// - `firmware`: array of ASCII string literals of the firmware files of
	97	+/// the kernel module.
65	98	#[proc_macro]
66	99	pub fn module(ts: TokenStream) -> TokenStream {
67	100	module::module(ts)
@@ -312,7 +345,7 @@ pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream { · 244 unmodified lines
312	345	///
313	346	/// Currently supported modifiers are:
314	347	/// * `span`: change the span of concatenated identifier to the span of the specified token. By
315		−/// default the span of the `[< >]` group is used.
	348	+/// default the span of the `[< >]` group is used.
316	349	/// * `lower`: change the identifier to lower case.
317	350	/// * `upper`: change the identifier to upper case.
318	351	///

rust/macros/module.rs+16−2

@@ -97,14 +97,22 @@ struct ModuleInfo { · 96 unmodified lines
97	97	author: Option<String>,
98	98	description: Option<String>,
99	99	alias: Option<Vec<String>>,
	100	+ firmware: Option<Vec<String>>,
100	101	}
101	102
102	103	impl ModuleInfo {
103	104	fn parse(it: &mut token_stream::IntoIter) -> Self {
104	105	let mut info = ModuleInfo::default();
105	106
106		− const EXPECTED_KEYS: &[&str] =
107		− &["type", "name", "author", "description", "license", "alias"];
	107	+ const EXPECTED_KEYS: &[&str] = &[
	108	+ "type",
	109	+ "name",
	110	+ "author",
	111	+ "description",
	112	+ "license",
	113	+ "alias",
	114	+ "firmware",
	115	+ ];
108	116	const REQUIRED_KEYS: &[&str] = &["type", "name", "license"];
109	117	let mut seen_keys = Vec::new();
110	118
@@ -131,6 +139,7 @@ impl ModuleInfo { · 20 unmodified lines
131	139	"description" => info.description = Some(expect_string(it)),
132	140	"license" => info.license = expect_string_ascii(it),
133	141	"alias" => info.alias = Some(expect_string_array(it)),
	142	+ "firmware" => info.firmware = Some(expect_string_array(it)),
134	143	_ => panic!(
135	144	"Unknown key \"{}\". Valid keys are: {:?}.",
136	145	key, EXPECTED_KEYS
@@ -186,6 +195,11 @@ pub(crate) fn module(ts: TokenStream) -> TokenStream { · 49 unmodified lines
186	195	modinfo.emit("alias", &alias);
187	196	}
188	197	}
	198	+ if let Some(firmware) = info.firmware {
	199	+ for fw in firmware {
	200	+ modinfo.emit("firmware", &fw);
	201	+ }
	202	+ }
189	203
190	204	// Built-in modules also export the `file` modinfo string.
191	205	let file =

rust/uapi/lib.rs+1−0

@@ -14,6 +14,7 @@ · 13 unmodified lines
14	14	#![cfg_attr(test, allow(unsafe_op_in_unsafe_fn))]
15	15	#![allow(
16	16	clippy::all,
	17	+ dead_code,
17	18	missing_docs,
18	19	non_camel_case_types,
19	20	non_upper_case_globals,

scripts/rust_is_available.sh+17−16

@@ -117,20 +117,16 @@ if [ "$rust_compiler_cversion" -lt "$rust_compiler_min_cversion" ]; then · 116 unmodified lines
117	117	echo >&2 "***"
118	118	exit 1
119	119	fi
120		−if [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
121		− echo >&2 "***"
122		− echo >&2 "*** Rust compiler '$RUSTC' is too new. This may or may not work."
123		− echo >&2 "*** Your version: $rust_compiler_version"
124		− echo >&2 "*** Expected version: $rust_compiler_min_version"
125		− echo >&2 "***"
126		− warning=1
127		−fi
128	120
129	121	# Check that the Rust bindings generator is suitable.
130	122	#
131	123	# Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
	124	+#
	125	+# The dummy parameter `workaround-for-0.69.0` is required to support 0.69.0
	126	+# (https://github.com/rust-lang/rust-bindgen/pull/2678). It can be removed when
	127	+# the minimum version is upgraded past that (0.69.1 already fixed the issue).
132	128	rust_bindings_generator_output=$( \
133		− LC_ALL=C "$BINDGEN" --version 2>/dev/null
	129	+ LC_ALL=C "$BINDGEN" --version workaround-for-0.69.0 2>/dev/null
134	130	) \|\| rust_bindings_generator_code=$?
135	131	if [ -n "$rust_bindings_generator_code" ]; then
136	132	echo >&2 "***"
@@ -165,13 +161,18 @@ if [ "$rust_bindings_generator_cversion" -lt "$rust_bindings_generator_min_cvers · 28 unmodified lines
165	161	echo >&2 "***"
166	162	exit 1
167	163	fi
168		−if [ "$rust_bindings_generator_cversion" -gt "$rust_bindings_generator_min_cversion" ]; then
169		− echo >&2 "***"
170		− echo >&2 "*** Rust bindings generator '$BINDGEN' is too new. This may or may not work."
171		− echo >&2 "*** Your version: $rust_bindings_generator_version"
172		− echo >&2 "*** Expected version: $rust_bindings_generator_min_version"
173		− echo >&2 "***"
174		− warning=1
	164	+if [ "$rust_bindings_generator_cversion" -eq 6600 ] \|\|
	165	+ [ "$rust_bindings_generator_cversion" -eq 6601 ]; then
	166	+ # Distributions may have patched the issue (e.g. Debian did).
	167	+ if ! "$BINDGEN" $(dirname $0)/rust_is_available_bindgen_0_66.h >/dev/null; then
	168	+ echo >&2 "***"
	169	+ echo >&2 "*** Rust bindings generator '$BINDGEN' versions 0.66.0 and 0.66.1 may not"
	170	+ echo >&2 "*** work due to a bug (https://github.com/rust-lang/rust-bindgen/pull/2567),"
	171	+ echo >&2 "*** unless patched (like Debian's)."
	172	+ echo >&2 "*** Your version: $rust_bindings_generator_version"
	173	+ echo >&2 "***"
	174	+ warning=1
	175	+ fi
175	176	fi
176	177
177	178	# Check that the `libclang` used by the Rust bindings generator is suitable.

scripts/rust_is_available_bindgen_0_66.h+2−0

@@ -0,0 +1,2 @@
	1	+/* SPDX-License-Identifier: GPL-2.0 */
	2	+#define A "\0"

scripts/rust_is_available_test.py+37−22

@@ -54,18 +54,34 @@ else: · 53 unmodified lines
54	54	""")
55	55
56	56	@classmethod
57		− def generate_bindgen(cls, version_stdout, libclang_stderr):
	57	+ def generate_bindgen(cls, version_stdout, libclang_stderr, version_0_66_patched=False):
	58	+ if libclang_stderr is None:
	59	+ libclang_case = f"raise SystemExit({cls.bindgen_default_bindgen_libclang_failure_exit_code})"
	60	+ else:
	61	+ libclang_case = f"print({repr(libclang_stderr)}, file=sys.stderr)"
	62	+
	63	+ if version_0_66_patched:
	64	+ version_0_66_case = "pass"
	65	+ else:
	66	+ version_0_66_case = "raise SystemExit(1)"
	67	+
58	68	return cls.generate_executable(f"""#!/usr/bin/env python3
59	69	import sys
60	70	if "rust_is_available_bindgen_libclang.h" in " ".join(sys.argv):
61		− print({repr(libclang_stderr)}, file=sys.stderr)
	71	+ {libclang_case}
	72	+elif "rust_is_available_bindgen_0_66.h" in " ".join(sys.argv):
	73	+ {version_0_66_case}
62	74	else:
63	75	print({repr(version_stdout)})
64	76	""")
65	77
66	78	@classmethod
67		− def generate_bindgen_version(cls, stdout):
68		− return cls.generate_bindgen(stdout, cls.bindgen_default_bindgen_libclang_stderr)
	79	+ def generate_bindgen_version(cls, stdout, version_0_66_patched=False):
	80	+ return cls.generate_bindgen(stdout, cls.bindgen_default_bindgen_libclang_stderr, version_0_66_patched)
	81	+
	82	+ @classmethod
	83	+ def generate_bindgen_libclang_failure(cls):
	84	+ return cls.generate_bindgen(cls.bindgen_default_bindgen_version_stdout, None)
69	85
70	86	@classmethod
71	87	def generate_bindgen_libclang(cls, stderr):
@@ -89,6 +105,7 @@ else: · 17 unmodified lines
89	105	cls.rust_default_sysroot = subprocess.check_output(("rustc", "--print", "sysroot")).decode().strip()
90	106
91	107	cls.bindgen_default_bindgen_version_stdout = f"bindgen {cls.bindgen_default_version}"
	108	+ cls.bindgen_default_bindgen_libclang_failure_exit_code = 42
92	109	cls.bindgen_default_bindgen_libclang_stderr = f"scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {cls.llvm_default_version} [-W#pragma-messages], err: false"
93	110
94	111	cls.default_rustc = cls.generate_rustc(f"rustc {cls.rustc_default_version}")
@@ -193,11 +210,6 @@ else: · 98 unmodified lines
193	210	result = self.run_script(self.Expected.FAILURE, { "RUSTC": rustc })
194	211	self.assertIn(f"Rust compiler '{rustc}' is too old.", result.stderr)
195	212
196		− def test_rustc_new_version(self):
197		− rustc = self.generate_rustc("rustc 1.999.0 (a8314ef7d 2099-06-27)")
198		− result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "RUSTC": rustc })
199		− self.assertIn(f"Rust compiler '{rustc}' is too new. This may or may not work.", result.stderr)
200		−
201	213	def test_bindgen_nonexecutable(self):
202	214	result = self.run_script(self.Expected.FAILURE, { "BINDGEN": self.nonexecutable })
203	215	self.assertIn(f"Running '{self.nonexecutable}' to check the Rust bindings generator version failed with", result.stderr)
@@ -226,21 +238,24 @@ else: · 22 unmodified lines
226	238	result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
227	239	self.assertIn(f"Rust bindings generator '{bindgen}' is too old.", result.stderr)
228	240
229		− def test_bindgen_new_version(self):
230		− bindgen = self.generate_bindgen_version("bindgen 0.999.0")
231		− result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "BINDGEN": bindgen })
232		− self.assertIn(f"Rust bindings generator '{bindgen}' is too new. This may or may not work.", result.stderr)
	241	+ def test_bindgen_bad_version_0_66_0_and_0_66_1(self):
	242	+ for version in ("0.66.0", "0.66.1"):
	243	+ with self.subTest(version=version):
	244	+ bindgen = self.generate_bindgen_version(f"bindgen {version}")
	245	+ result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "BINDGEN": bindgen })
	246	+ self.assertIn(f"Rust bindings generator '{bindgen}' versions 0.66.0 and 0.66.1 may not", result.stderr)
	247	+
	248	+ def test_bindgen_bad_version_0_66_0_and_0_66_1_patched(self):
	249	+ for version in ("0.66.0", "0.66.1"):
	250	+ with self.subTest(version=version):
	251	+ bindgen = self.generate_bindgen_version(f"bindgen {version}", True)
	252	+ result = self.run_script(self.Expected.SUCCESS, { "BINDGEN": bindgen })
233	253
234	254	def test_bindgen_libclang_failure(self):
235		− for env in (
236		− { "LLVM_CONFIG_PATH": self.missing },
237		− { "LIBCLANG_PATH": self.missing },
238		− { "CLANG_PATH": self.missing },
239		− ):
240		− with self.subTest(env=env):
241		− result = self.run_script(self.Expected.FAILURE, env \| { "PATH": os.environ["PATH"], "BINDGEN": "bindgen" })
242		− self.assertIn("Running 'bindgen' to check the libclang version (used by the Rust", result.stderr)
243		− self.assertIn("bindings generator) failed with code ", result.stderr)
	255	+ bindgen = self.generate_bindgen_libclang_failure()
	256	+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
	257	+ self.assertIn(f"Running '{bindgen}' to check the libclang version (used by the Rust", result.stderr)
	258	+ self.assertIn(f"bindings generator) failed with code {self.bindgen_default_bindgen_libclang_failure_exit_code}. This may be caused by", result.stderr)
244	259
245	260	def test_bindgen_libclang_unexpected_version(self):
246	261	bindgen = self.generate_bindgen_libclang("scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version unexpected [-W#pragma-messages], err: false")