Merge tag 'rust-6.11' of https://github.com/Rust-for-Linux/linux

Pull Rust updates from Miguel Ojeda: "The highlight is the establishment of a minimum version for the Rust toolchain, including 'rustc' (and bundled tools) and 'bindgen'. The initial minimum will be the pinned version we currently have, i.e. we are just widening the allowed versions. That covers three stable Rust releases: 1.78.0, 1.79.0, 1.80.0 (getting released tomorrow), plus beta, plus nightly. This should already be enough for kernel developers in distributions that provide recent Rust compiler versions routinely, such as Arch Linux, Debian Unstable (outside the freeze period), Fedora Linux, Gentoo Linux (especially the testing channel), Nix (unstable) and openSUSE Slowroll and Tumbleweed. In addition, the kernel is now being built-tested by Rust's pre-merge CI. That is, every change that is attempting to land into the Rust compiler is tested against the kernel, and it is merged only if it passes. Similarly, the bindgen tool has agreed to build the kernel in their CI too. Thus, with the pre-merge CI in place, both projects hope to avoid unintentional changes to Rust that break the kernel. This means that, in general, apart from intentional changes on their side (that we will need to workaround conditionally on our side), the upcoming Rust compiler versions should generally work. In addition, the Rust project has proposed getting the kernel into stable Rust (at least solving the main blockers) as one of its three flagship goals for 2024H2 [1]. I would like to thank Niko, Sid, Emilio et al. for their help promoting the collaboration between Rust and the kernel. Toolchain and infrastructure: - Support several Rust toolchain versions. - Support several bindgen versions. - Remove 'cargo' requirement and simplify 'rusttest', thanks to 'alloc' having been dropped last cycle. - Provide proper error reporting for the 'rust-analyzer' target. 'kernel' crate: - Add 'uaccess' module with a safe userspace pointers abstraction. - Add 'page' module with a 'struct page' abstraction. - Support more complex generics in workqueue's 'impl_has_work!' macro. 'macros' crate: - Add 'firmware' field support to the 'module!' macro. - Improve 'module!' macro documentation. Documentation: - Provide instructions on what packages should be installed to build the kernel in some popular Linux distributions. - Introduce the new kernel.org LLVM+Rust toolchains. - Explain '#[no_std]'. And a few other small bits" Link: https://rust-lang.github.io/rust-project-goals/2024h2/index.html#flagship-goals [1] * tag 'rust-6.11' of https://github.com/Rust-for-Linux/linux: (26 commits) docs: rust: quick-start: add section on Linux distributions rust: warn about `bindgen` versions 0.66.0 and 0.66.1 rust: start supporting several `bindgen` versions rust: work around `bindgen` 0.69.0 issue rust: avoid assuming a particular `bindgen` build rust: start supporting several compiler versions rust: simplify Clippy warning flags set rust: relax most deny-level lints to warnings rust: allow `dead_code` for never constructed bindings rust: init: simplify from `map_err` to `inspect_err` rust: macros: indent list item in `paste!`'s docs rust: add abstraction for `struct page` rust: uaccess: add typed accessors for userspace pointers uaccess: always export _copy_[from|to]_user with CONFIG_RUST rust: uaccess: add userspace pointers kbuild: rust-analyzer: improve comment documentation kbuild: rust-analyzer: better error handling docs: rust: no_std is used rust: alloc: add __GFP_HIGHMEM flag rust: alloc: fix typo in docs for GFP_NOWAIT ...

Linus Torvalds committed 2 years ago

Commit 910bfc22 parents ff30564 b12634125 files · +1058 −236Browse files

Showing the changes to rust (14 of 25 files). Show all

Unified · Split

14 files

rust/Makefile +10 −64
rust/bindings/bindings_helper.h +1 −0
rust/bindings/lib.rs +1 −0
rust/helpers.c +34 −0
rust/kernel/alloc.rs +16 −1
rust/kernel/init.rs +4 −9
rust/kernel/lib.rs +2 −0
rust/kernel/page.rs +250 −0
rust/kernel/types.rs +64 −0
rust/kernel/uaccess.rs +388 −0
rust/kernel/workqueue.rs +9 −7
rust/macros/lib.rs +39 −6
rust/macros/module.rs +16 −2
rust/uapi/lib.rs +1 −0

rust/Makefile+10−64

@@ -44,17 +44,10 @@ rustc_sysroot := $(shell MAKEFLAGS= $(RUSTC) $(rust_flags) --print sysroot) · 43 unmodified lines
44	44	rustc_host_target := $(shell $(RUSTC) --version --verbose \| grep -F 'host: ' \| cut -d' ' -f2)
45	45	RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library
46	46
47		−ifeq ($(quiet),silent_)
48		−cargo_quiet=-q
	47	+ifneq ($(quiet),)
49	48	rust_test_quiet=-q
50	49	rustdoc_test_quiet=--test-args -q
51	50	rustdoc_test_kernel_quiet=>/dev/null
52		−else ifeq ($(quiet),quiet_)
53		−rust_test_quiet=-q
54		−rustdoc_test_quiet=--test-args -q
55		−rustdoc_test_kernel_quiet=>/dev/null
56		−else
57		−cargo_quiet=--verbose
58	51	endif
59	52
60	53	core-cfgs = \
@@ -135,22 +128,21 @@ quiet_cmd_rustc_test_library = RUSTC TL $< · 74 unmodified lines
135	128	@$(objtree)/include/generated/rustc_cfg $(rustc_target_flags) \
136	129	--crate-type $(if $(rustc_test_library_proc),proc-macro,rlib) \
137	130	--out-dir $(objtree)/$(obj)/test --cfg testlib \
138		− --sysroot $(objtree)/$(obj)/test/sysroot \
139	131	-L$(objtree)/$(obj)/test \
140	132	--crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<
141	133
142		−rusttestlib-build_error: $(src)/build_error.rs rusttest-prepare FORCE
	134	+rusttestlib-build_error: $(src)/build_error.rs FORCE
143	135	+$(call if_changed,rustc_test_library)
144	136
145	137	rusttestlib-macros: private rustc_target_flags = --extern proc_macro
146	138	rusttestlib-macros: private rustc_test_library_proc = yes
147		−rusttestlib-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
	139	+rusttestlib-macros: $(src)/macros/lib.rs FORCE
148	140	+$(call if_changed,rustc_test_library)
149	141
150		−rusttestlib-bindings: $(src)/bindings/lib.rs rusttest-prepare FORCE
	142	+rusttestlib-bindings: $(src)/bindings/lib.rs FORCE
151	143	+$(call if_changed,rustc_test_library)
152	144
153		−rusttestlib-uapi: $(src)/uapi/lib.rs rusttest-prepare FORCE
	145	+rusttestlib-uapi: $(src)/uapi/lib.rs FORCE
154	146	+$(call if_changed,rustc_test_library)
155	147
156	148	quiet_cmd_rustdoc_test = RUSTDOC T $<
@@ -159,7 +151,7 @@ quiet_cmd_rustdoc_test = RUSTDOC T $< · 2 unmodified lines
159	151	$(RUSTDOC) --test $(rust_common_flags) \
160	152	@$(objtree)/include/generated/rustc_cfg \
161	153	$(rustc_target_flags) $(rustdoc_test_target_flags) \
162		− --sysroot $(objtree)/$(obj)/test/sysroot $(rustdoc_test_quiet) \
	154	+ $(rustdoc_test_quiet) \
163	155	-L$(objtree)/$(obj)/test --output $(rustdoc_output) \
164	156	--crate-name $(subst rusttest-,,$@) $<
165	157
@@ -192,7 +184,6 @@ quiet_cmd_rustc_test = RUSTC T $< · 26 unmodified lines
192	184	$(RUSTC) --test $(rust_common_flags) \
193	185	@$(objtree)/include/generated/rustc_cfg \
194	186	$(rustc_target_flags) --out-dir $(objtree)/$(obj)/test \
195		− --sysroot $(objtree)/$(obj)/test/sysroot \
196	187	-L$(objtree)/$(obj)/test \
197	188	--crate-name $(subst rusttest-,,$@) $<; \
198	189	$(objtree)/$(obj)/test/$(subst rusttest-,,$@) $(rust_test_quiet) \
@@ -200,60 +191,15 @@ quiet_cmd_rustc_test = RUSTC T $< · 1 unmodified line
200	191
201	192	rusttest: rusttest-macros rusttest-kernel
202	193
203		−# This prepares a custom sysroot with our custom `alloc` instead of
204		−# the standard one.
205		−#
206		−# This requires several hacks:
207		−# - Unlike `core` and `alloc`, `std` depends on more than a dozen crates,
208		−# including third-party crates that need to be downloaded, plus custom
209		−# `build.rs` steps. Thus hardcoding things here is not maintainable.
210		−# - `cargo` knows how to build the standard library, but it is an unstable
211		−# feature so far (`-Zbuild-std`).
212		−# - `cargo` only considers the use case of building the standard library
213		−# to use it in a given package. Thus we need to create a dummy package
214		−# and pick the generated libraries from there.
215		−# - The usual ways of modifying the dependency graph in `cargo` do not seem
216		−# to apply for the `-Zbuild-std` steps, thus we have to mislead it
217		−# by modifying the sources in the sysroot.
218		−# - To avoid messing with the user's Rust installation, we create a clone
219		−# of the sysroot. However, `cargo` ignores `RUSTFLAGS` in the `-Zbuild-std`
220		−# steps, thus we use a wrapper binary passed via `RUSTC` to pass the flag.
221		−#
222		−# In the future, we hope to avoid the whole ordeal by either:
223		−# - Making the `test` crate not depend on `std` (either improving upstream
224		−# or having our own custom crate).
225		−# - Making the tests run in kernel space (requires the previous point).
226		−# - Making `std` and friends be more like a "normal" crate, so that
227		−# `-Zbuild-std` and related hacks are not needed.
228		−quiet_cmd_rustsysroot = RUSTSYSROOT
229		− cmd_rustsysroot = \
230		− rm -rf $(objtree)/$(obj)/test; \
231		− mkdir -p $(objtree)/$(obj)/test; \
232		− cp -a $(rustc_sysroot) $(objtree)/$(obj)/test/sysroot; \
233		− echo '\#!/bin/sh' > $(objtree)/$(obj)/test/rustc_sysroot; \
234		− echo "$(RUSTC) --sysroot=$(abspath $(objtree)/$(obj)/test/sysroot) \"\$$@\"" \
235		− >> $(objtree)/$(obj)/test/rustc_sysroot; \
236		− chmod u+x $(objtree)/$(obj)/test/rustc_sysroot; \
237		− $(CARGO) -q new $(objtree)/$(obj)/test/dummy; \
238		− RUSTC=$(objtree)/$(obj)/test/rustc_sysroot $(CARGO) $(cargo_quiet) \
239		− test -Zbuild-std --target $(rustc_host_target) \
240		− --manifest-path $(objtree)/$(obj)/test/dummy/Cargo.toml; \
241		− rm $(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib/*; \
242		− cp $(objtree)/$(obj)/test/dummy/target/$(rustc_host_target)/debug/deps/* \
243		− $(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib
244		−
245		−rusttest-prepare: FORCE
246		− +$(call if_changed,rustsysroot)
247		−
248	194	rusttest-macros: private rustc_target_flags = --extern proc_macro
249	195	rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro
250		−rusttest-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
	196	+rusttest-macros: $(src)/macros/lib.rs FORCE
251	197	+$(call if_changed,rustc_test)
252	198	+$(call if_changed,rustdoc_test)
253	199
254	200	rusttest-kernel: private rustc_target_flags = --extern alloc \
255	201	--extern build_error --extern macros --extern bindings --extern uapi
256		−rusttest-kernel: $(src)/kernel/lib.rs rusttest-prepare \
	202	+rusttest-kernel: $(src)/kernel/lib.rs \
257	203	rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
258	204	rusttestlib-uapi FORCE
259	205	+$(call if_changed,rustc_test)
@@ -421,7 +367,7 @@ ifneq ($(or $(CONFIG_ARM64),$(and $(CONFIG_RISCV),$(CONFIG_64BIT))),) · 161 unmodified lines
421	367	endif
422	368
423	369	$(obj)/core.o: private skip_clippy = 1
424		−$(obj)/core.o: private skip_flags = -Dunreachable_pub
	370	+$(obj)/core.o: private skip_flags = -Wunreachable_pub
425	371	$(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))
426	372	$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
427	373	$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs FORCE
@@ -435,7 +381,7 @@ $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE · 7 unmodified lines
435	381	+$(call if_changed_dep,rustc_library)
436	382
437	383	$(obj)/alloc.o: private skip_clippy = 1
438		−$(obj)/alloc.o: private skip_flags = -Dunreachable_pub
	384	+$(obj)/alloc.o: private skip_flags = -Wunreachable_pub
439	385	$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)
440	386	$(obj)/alloc.o: $(RUST_LIB_SRC)/alloc/src/lib.rs $(obj)/compiler_builtins.o FORCE
441	387	+$(call if_changed_dep,rustc_library)

rust/bindings/bindings_helper.h+1−0

@@ -30,4 +30,5 @@ const gfp_t RUST_CONST_HELPER_GFP_KERNEL = GFP_KERNEL; · 29 unmodified lines
30	30	const gfp_t RUST_CONST_HELPER_GFP_KERNEL_ACCOUNT = GFP_KERNEL_ACCOUNT;
31	31	const gfp_t RUST_CONST_HELPER_GFP_NOWAIT = GFP_NOWAIT;
32	32	const gfp_t RUST_CONST_HELPER___GFP_ZERO = __GFP_ZERO;
	33	+const gfp_t RUST_CONST_HELPER___GFP_HIGHMEM = ___GFP_HIGHMEM;
33	34	const blk_features_t RUST_CONST_HELPER_BLK_FEAT_ROTATIONAL = BLK_FEAT_ROTATIONAL;

rust/bindings/lib.rs+1−0

@@ -24,6 +24,7 @@ · 23 unmodified lines
24	24	unsafe_op_in_unsafe_fn
25	25	)]
26	26
	27	+#[allow(dead_code)]
27	28	mod bindings_raw {
28	29	// Use glob import here to expose all helpers.
29	30	// Symbols defined within the module will take precedence to the glob import.

rust/helpers.c+34−0

@@ -26,6 +26,8 @@ · 25 unmodified lines
26	26	#include <linux/device.h>
27	27	#include <linux/err.h>
28	28	#include <linux/errname.h>
	29	+#include <linux/gfp.h>
	30	+#include <linux/highmem.h>
29	31	#include <linux/mutex.h>
30	32	#include <linux/refcount.h>
31	33	#include <linux/sched/signal.h>
@@ -40,6 +42,20 @@ __noreturn void rust_helper_BUG(void) · 8 unmodified lines
40	42	}
41	43	EXPORT_SYMBOL_GPL(rust_helper_BUG);
42	44
	45	+unsigned long rust_helper_copy_from_user(void to, const void __user from,
	46	+ unsigned long n)
	47	+{
	48	+ return copy_from_user(to, from, n);
	49	+}
	50	+EXPORT_SYMBOL_GPL(rust_helper_copy_from_user);
	51	+
	52	+unsigned long rust_helper_copy_to_user(void __user to, const void from,
	53	+ unsigned long n)
	54	+{
	55	+ return copy_to_user(to, from, n);
	56	+}
	57	+EXPORT_SYMBOL_GPL(rust_helper_copy_to_user);
	58	+
43	59	void rust_helper_mutex_lock(struct mutex *lock)
44	60	{
45	61	mutex_lock(lock);
@@ -81,6 +97,24 @@ int rust_helper_signal_pending(struct task_struct *t) · 35 unmodified lines
81	97	}
82	98	EXPORT_SYMBOL_GPL(rust_helper_signal_pending);
83	99
	100	+struct page *rust_helper_alloc_pages(gfp_t gfp_mask, unsigned int order)
	101	+{
	102	+ return alloc_pages(gfp_mask, order);
	103	+}
	104	+EXPORT_SYMBOL_GPL(rust_helper_alloc_pages);
	105	+
	106	+void rust_helper_kmap_local_page(struct page page)
	107	+{
	108	+ return kmap_local_page(page);
	109	+}
	110	+EXPORT_SYMBOL_GPL(rust_helper_kmap_local_page);
	111	+
	112	+void rust_helper_kunmap_local(const void *addr)
	113	+{
	114	+ kunmap_local(addr);
	115	+}
	116	+EXPORT_SYMBOL_GPL(rust_helper_kunmap_local);
	117	+
84	118	refcount_t rust_helper_REFCOUNT_INIT(int n)
85	119	{
86	120	return (refcount_t)REFCOUNT_INIT(n);

rust/kernel/alloc.rs+16−1

@@ -20,6 +20,13 @@ pub struct AllocError; · 19 unmodified lines
20	20	#[derive(Clone, Copy)]
21	21	pub struct Flags(u32);
22	22
	23	+impl Flags {
	24	+ /// Get the raw representation of this flag.
	25	+ pub(crate) fn as_raw(self) -> u32 {
	26	+ self.0
	27	+ }
	28	+}
	29	+
23	30	impl core::ops::BitOr for Flags {
24	31	type Output = Self;
25	32	fn bitor(self, rhs: Self) -> Self::Output {
@@ -52,6 +59,14 @@ pub mod flags { · 26 unmodified lines
52	59	/// This is normally or'd with other flags.
53	60	pub const __GFP_ZERO: Flags = Flags(bindings::__GFP_ZERO);
54	61
	62	+ /// Allow the allocation to be in high memory.
	63	+ ///
	64	+ /// Allocations in high memory may not be mapped into the kernel's address space, so this can't
	65	+ /// be used with `kmalloc` and other similar methods.
	66	+ ///
	67	+ /// This is normally or'd with other flags.
	68	+ pub const __GFP_HIGHMEM: Flags = Flags(bindings::__GFP_HIGHMEM);
	69	+
55	70	/// Users can not sleep and need the allocation to succeed.
56	71	///
57	72	/// A lower watermark is applied to allow access to "atomic reserves". The current
@@ -66,7 +81,7 @@ pub mod flags { · 8 unmodified lines
66	81	/// The same as [`GFP_KERNEL`], except the allocation is accounted to kmemcg.
67	82	pub const GFP_KERNEL_ACCOUNT: Flags = Flags(bindings::GFP_KERNEL_ACCOUNT);
68	83
69		− /// Ror kernel allocations that should not stall for direct reclaim, start physical IO or
	84	+ /// For kernel allocations that should not stall for direct reclaim, start physical IO or
70	85	/// use any filesystem callback. It is very likely to fail to allocate memory, even for very
71	86	/// small allocations.
72	87	pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);

rust/kernel/init.rs+4−9

@@ -843,11 +843,8 @@ where · 842 unmodified lines
843	843	let val = unsafe { &mut *slot };
844	844	// SAFETY: `slot` is considered pinned.
845	845	let val = unsafe { Pin::new_unchecked(val) };
846		− (self.1)(val).map_err(\|e\| {
847		− // SAFETY: `slot` was initialized above.
848		− unsafe { core::ptr::drop_in_place(slot) };
849		− e
850		− })
	846	+ // SAFETY: `slot` was initialized above.
	847	+ (self.1)(val).inspect_err(\|_\| unsafe { core::ptr::drop_in_place(slot) })
851	848	}
852	849	}
853	850
@@ -941,11 +938,9 @@ where · 87 unmodified lines
941	938	// SAFETY: All requirements fulfilled since this function is `__init`.
942	939	unsafe { self.0.__pinned_init(slot)? };
943	940	// SAFETY: The above call initialized `slot` and we still have unique access.
944		− (self.1)(unsafe { &mut *slot }).map_err(\|e\| {
	941	+ (self.1)(unsafe { &mut *slot }).inspect_err(\|_\|
945	942	// SAFETY: `slot` was initialized above.
946		− unsafe { core::ptr::drop_in_place(slot) };
947		− e
948		− })
	943	+ unsafe { core::ptr::drop_in_place(slot) })
949	944	}
950	945	}
951	946

rust/kernel/lib.rs+2−0

@@ -40,6 +40,7 @@ pub mod ioctl; · 39 unmodified lines
40	40	pub mod kunit;
41	41	#[cfg(CONFIG_NET)]
42	42	pub mod net;
	43	+pub mod page;
43	44	pub mod prelude;
44	45	pub mod print;
45	46	mod static_assert;
@@ -50,6 +51,7 @@ pub mod sync; · 4 unmodified lines
50	51	pub mod task;
51	52	pub mod time;
52	53	pub mod types;
	54	+pub mod uaccess;
53	55	pub mod workqueue;
54	56
55	57	#[doc(hidden)]

rust/kernel/page.rs+250−0

@@ -0,0 +1,250 @@
	1	+// SPDX-License-Identifier: GPL-2.0
	2	+
	3	+//! Kernel page allocation and management.
	4	+
	5	+use crate::{
	6	+ alloc::{AllocError, Flags},
	7	+ bindings,
	8	+ error::code::*,
	9	+ error::Result,
	10	+ uaccess::UserSliceReader,
	11	+};
	12	+use core::ptr::{self, NonNull};
	13	+
	14	+/// A bitwise shift for the page size.
	15	+pub const PAGE_SHIFT: usize = bindings::PAGE_SHIFT as usize;
	16	+
	17	+/// The number of bytes in a page.
	18	+pub const PAGE_SIZE: usize = bindings::PAGE_SIZE;
	19	+
	20	+/// A bitmask that gives the page containing a given address.
	21	+pub const PAGE_MASK: usize = !(PAGE_SIZE - 1);
	22	+
	23	+/// A pointer to a page that owns the page allocation.
	24	+///
	25	+/// # Invariants
	26	+///
	27	+/// The pointer is valid, and has ownership over the page.
	28	+pub struct Page {
	29	+ page: NonNull<bindings::page>,
	30	+}
	31	+
	32	+// SAFETY: Pages have no logic that relies on them staying on a given thread, so moving them across
	33	+// threads is safe.
	34	+unsafe impl Send for Page {}
	35	+
	36	+// SAFETY: Pages have no logic that relies on them not being accessed concurrently, so accessing
	37	+// them concurrently is safe.
	38	+unsafe impl Sync for Page {}
	39	+
	40	+impl Page {
	41	+ /// Allocates a new page.
	42	+ ///
	43	+ /// # Examples
	44	+ ///
	45	+ /// Allocate memory for a page.
	46	+ ///
	47	+ /// ```
	48	+ /// use kernel::page::Page;
	49	+ ///
	50	+ /// # fn dox() -> Result<(), kernel::alloc::AllocError> {
	51	+ /// let page = Page::alloc_page(GFP_KERNEL)?;
	52	+ /// # Ok(()) }
	53	+ /// ```
	54	+ ///
	55	+ /// Allocate memory for a page and zero its contents.
	56	+ ///
	57	+ /// ```
	58	+ /// use kernel::page::Page;
	59	+ ///
	60	+ /// # fn dox() -> Result<(), kernel::alloc::AllocError> {
	61	+ /// let page = Page::alloc_page(GFP_KERNEL \| __GFP_ZERO)?;
	62	+ /// # Ok(()) }
	63	+ /// ```
	64	+ pub fn alloc_page(flags: Flags) -> Result<Self, AllocError> {
	65	+ // SAFETY: Depending on the value of `gfp_flags`, this call may sleep. Other than that, it
	66	+ // is always safe to call this method.
	67	+ let page = unsafe { bindings::alloc_pages(flags.as_raw(), 0) };
	68	+ let page = NonNull::new(page).ok_or(AllocError)?;
	69	+ // INVARIANT: We just successfully allocated a page, so we now have ownership of the newly
	70	+ // allocated page. We transfer that ownership to the new `Page` object.
	71	+ Ok(Self { page })
	72	+ }
	73	+
	74	+ /// Returns a raw pointer to the page.
	75	+ pub fn as_ptr(&self) -> *mut bindings::page {
	76	+ self.page.as_ptr()
	77	+ }
	78	+
	79	+ /// Runs a piece of code with this page mapped to an address.
	80	+ ///
	81	+ /// The page is unmapped when this call returns.
	82	+ ///
	83	+ /// # Using the raw pointer
	84	+ ///
	85	+ /// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for
	86	+ /// `PAGE_SIZE` bytes and for the duration in which the closure is called. The pointer might
	87	+ /// only be mapped on the current thread, and when that is the case, dereferencing it on other
	88	+ /// threads is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't
	89	+ /// cause data races, the memory may be uninitialized, and so on.
	90	+ ///
	91	+ /// If multiple threads map the same page at the same time, then they may reference with
	92	+ /// different addresses. However, even if the addresses are different, the underlying memory is
	93	+ /// still the same for these purposes (e.g., it's still a data race if they both write to the
	94	+ /// same underlying byte at the same time).
	95	+ fn with_page_mapped<T>(&self, f: impl FnOnce(*mut u8) -> T) -> T {
	96	+ // SAFETY: `page` is valid due to the type invariants on `Page`.
	97	+ let mapped_addr = unsafe { bindings::kmap_local_page(self.as_ptr()) };
	98	+
	99	+ let res = f(mapped_addr.cast());
	100	+
	101	+ // This unmaps the page mapped above.
	102	+ //
	103	+ // SAFETY: Since this API takes the user code as a closure, it can only be used in a manner
	104	+ // where the pages are unmapped in reverse order. This is as required by `kunmap_local`.
	105	+ //
	106	+ // In other words, if this call to `kunmap_local` happens when a different page should be
	107	+ // unmapped first, then there must necessarily be a call to `kmap_local_page` other than the
	108	+ // call just above in `with_page_mapped` that made that possible. In this case, it is the
	109	+ // unsafe block that wraps that other call that is incorrect.
	110	+ unsafe { bindings::kunmap_local(mapped_addr) };
	111	+
	112	+ res
	113	+ }
	114	+
	115	+ /// Runs a piece of code with a raw pointer to a slice of this page, with bounds checking.
	116	+ ///
	117	+ /// If `f` is called, then it will be called with a pointer that points at `off` bytes into the
	118	+ /// page, and the pointer will be valid for at least `len` bytes. The pointer is only valid on
	119	+ /// this task, as this method uses a local mapping.
	120	+ ///
	121	+ /// If `off` and `len` refers to a region outside of this page, then this method returns
	122	+ /// [`EINVAL`] and does not call `f`.
	123	+ ///
	124	+ /// # Using the raw pointer
	125	+ ///
	126	+ /// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for
	127	+ /// `len` bytes and for the duration in which the closure is called. The pointer might only be
	128	+ /// mapped on the current thread, and when that is the case, dereferencing it on other threads
	129	+ /// is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't cause
	130	+ /// data races, the memory may be uninitialized, and so on.
	131	+ ///
	132	+ /// If multiple threads map the same page at the same time, then they may reference with
	133	+ /// different addresses. However, even if the addresses are different, the underlying memory is
	134	+ /// still the same for these purposes (e.g., it's still a data race if they both write to the
	135	+ /// same underlying byte at the same time).
	136	+ fn with_pointer_into_page<T>(
	137	+ &self,
	138	+ off: usize,
	139	+ len: usize,
	140	+ f: impl FnOnce(*mut u8) -> Result<T>,
	141	+ ) -> Result<T> {
	142	+ let bounds_ok = off <= PAGE_SIZE && len <= PAGE_SIZE && (off + len) <= PAGE_SIZE;
	143	+
	144	+ if bounds_ok {
	145	+ self.with_page_mapped(move \|page_addr\| {
	146	+ // SAFETY: The `off` integer is at most `PAGE_SIZE`, so this pointer offset will
	147	+ // result in a pointer that is in bounds or one off the end of the page.
	148	+ f(unsafe { page_addr.add(off) })
	149	+ })
	150	+ } else {
	151	+ Err(EINVAL)
	152	+ }
	153	+ }
	154	+
	155	+ /// Maps the page and reads from it into the given buffer.
	156	+ ///
	157	+ /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	158	+ /// outside of the page, then this call returns [`EINVAL`].
	159	+ ///
	160	+ /// # Safety
	161	+ ///
	162	+ /// * Callers must ensure that `dst` is valid for writing `len` bytes.
	163	+ /// * Callers must ensure that this call does not race with a write to the same page that
	164	+ /// overlaps with this read.
	165	+ pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result {
	166	+ self.with_pointer_into_page(offset, len, move \|src\| {
	167	+ // SAFETY: If `with_pointer_into_page` calls into this closure, then
	168	+ // it has performed a bounds check and guarantees that `src` is
	169	+ // valid for `len` bytes.
	170	+ //
	171	+ // There caller guarantees that there is no data race.
	172	+ unsafe { ptr::copy_nonoverlapping(src, dst, len) };
	173	+ Ok(())
	174	+ })
	175	+ }
	176	+
	177	+ /// Maps the page and writes into it from the given buffer.
	178	+ ///
	179	+ /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	180	+ /// outside of the page, then this call returns [`EINVAL`].
	181	+ ///
	182	+ /// # Safety
	183	+ ///
	184	+ /// * Callers must ensure that `src` is valid for reading `len` bytes.
	185	+ /// * Callers must ensure that this call does not race with a read or write to the same page
	186	+ /// that overlaps with this write.
	187	+ pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Result {
	188	+ self.with_pointer_into_page(offset, len, move \|dst\| {
	189	+ // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	190	+ // bounds check and guarantees that `dst` is valid for `len` bytes.
	191	+ //
	192	+ // There caller guarantees that there is no data race.
	193	+ unsafe { ptr::copy_nonoverlapping(src, dst, len) };
	194	+ Ok(())
	195	+ })
	196	+ }
	197	+
	198	+ /// Maps the page and zeroes the given slice.
	199	+ ///
	200	+ /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	201	+ /// outside of the page, then this call returns [`EINVAL`].
	202	+ ///
	203	+ /// # Safety
	204	+ ///
	205	+ /// Callers must ensure that this call does not race with a read or write to the same page that
	206	+ /// overlaps with this write.
	207	+ pub unsafe fn fill_zero_raw(&self, offset: usize, len: usize) -> Result {
	208	+ self.with_pointer_into_page(offset, len, move \|dst\| {
	209	+ // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	210	+ // bounds check and guarantees that `dst` is valid for `len` bytes.
	211	+ //
	212	+ // There caller guarantees that there is no data race.
	213	+ unsafe { ptr::write_bytes(dst, 0u8, len) };
	214	+ Ok(())
	215	+ })
	216	+ }
	217	+
	218	+ /// Copies data from userspace into this page.
	219	+ ///
	220	+ /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	221	+ /// outside of the page, then this call returns [`EINVAL`].
	222	+ ///
	223	+ /// Like the other `UserSliceReader` methods, data races are allowed on the userspace address.
	224	+ /// However, they are not allowed on the page you are copying into.
	225	+ ///
	226	+ /// # Safety
	227	+ ///
	228	+ /// Callers must ensure that this call does not race with a read or write to the same page that
	229	+ /// overlaps with this write.
	230	+ pub unsafe fn copy_from_user_slice_raw(
	231	+ &self,
	232	+ reader: &mut UserSliceReader,
	233	+ offset: usize,
	234	+ len: usize,
	235	+ ) -> Result {
	236	+ self.with_pointer_into_page(offset, len, move \|dst\| {
	237	+ // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	238	+ // bounds check and guarantees that `dst` is valid for `len` bytes. Furthermore, we have
	239	+ // exclusive access to the slice since the caller guarantees that there are no races.
	240	+ reader.read_raw(unsafe { core::slice::from_raw_parts_mut(dst.cast(), len) })
	241	+ })
	242	+ }
	243	+}
	244	+
	245	+impl Drop for Page {
	246	+ fn drop(&mut self) {
	247	+ // SAFETY: By the type invariants, we have ownership of the page and can free it.
	248	+ unsafe { bindings::__free_pages(self.page.as_ptr(), 0) };
	249	+ }
	250	+}

rust/kernel/types.rs+64−0

@@ -409,3 +409,67 @@ pub enum Either<L, R> { · 408 unmodified lines
409	409	/// Constructs an instance of [`Either`] containing a value of type `R`.
410	410	Right(R),
411	411	}
	412	+
	413	+/// Types for which any bit pattern is valid.
	414	+///
	415	+/// Not all types are valid for all values. For example, a `bool` must be either zero or one, so
	416	+/// reading arbitrary bytes into something that contains a `bool` is not okay.
	417	+///
	418	+/// It's okay for the type to have padding, as initializing those bytes has no effect.
	419	+///
	420	+/// # Safety
	421	+///
	422	+/// All bit-patterns must be valid for this type. This type must not have interior mutability.
	423	+pub unsafe trait FromBytes {}
	424	+
	425	+// SAFETY: All bit patterns are acceptable values of the types below.
	426	+unsafe impl FromBytes for u8 {}
	427	+unsafe impl FromBytes for u16 {}
	428	+unsafe impl FromBytes for u32 {}
	429	+unsafe impl FromBytes for u64 {}
	430	+unsafe impl FromBytes for usize {}
	431	+unsafe impl FromBytes for i8 {}
	432	+unsafe impl FromBytes for i16 {}
	433	+unsafe impl FromBytes for i32 {}
	434	+unsafe impl FromBytes for i64 {}
	435	+unsafe impl FromBytes for isize {}
	436	+// SAFETY: If all bit patterns are acceptable for individual values in an array, then all bit
	437	+// patterns are also acceptable for arrays of that type.
	438	+unsafe impl<T: FromBytes> FromBytes for [T] {}
	439	+unsafe impl<T: FromBytes, const N: usize> FromBytes for [T; N] {}
	440	+
	441	+/// Types that can be viewed as an immutable slice of initialized bytes.
	442	+///
	443	+/// If a struct implements this trait, then it is okay to copy it byte-for-byte to userspace. This
	444	+/// means that it should not have any padding, as padding bytes are uninitialized. Reading
	445	+/// uninitialized memory is not just undefined behavior, it may even lead to leaking sensitive
	446	+/// information on the stack to userspace.
	447	+///
	448	+/// The struct should also not hold kernel pointers, as kernel pointer addresses are also considered
	449	+/// sensitive. However, leaking kernel pointers is not considered undefined behavior by Rust, so
	450	+/// this is a correctness requirement, but not a safety requirement.
	451	+///
	452	+/// # Safety
	453	+///
	454	+/// Values of this type may not contain any uninitialized bytes. This type must not have interior
	455	+/// mutability.
	456	+pub unsafe trait AsBytes {}
	457	+
	458	+// SAFETY: Instances of the following types have no uninitialized portions.
	459	+unsafe impl AsBytes for u8 {}
	460	+unsafe impl AsBytes for u16 {}
	461	+unsafe impl AsBytes for u32 {}
	462	+unsafe impl AsBytes for u64 {}
	463	+unsafe impl AsBytes for usize {}
	464	+unsafe impl AsBytes for i8 {}
	465	+unsafe impl AsBytes for i16 {}
	466	+unsafe impl AsBytes for i32 {}
	467	+unsafe impl AsBytes for i64 {}
	468	+unsafe impl AsBytes for isize {}
	469	+unsafe impl AsBytes for bool {}
	470	+unsafe impl AsBytes for char {}
	471	+unsafe impl AsBytes for str {}
	472	+// SAFETY: If individual values in an array have no uninitialized portions, then the array itself
	473	+// does not have any uninitialized portions either.
	474	+unsafe impl<T: AsBytes> AsBytes for [T] {}
	475	+unsafe impl<T: AsBytes, const N: usize> AsBytes for [T; N] {}

rust/kernel/uaccess.rs+388−0

@@ -0,0 +1,388 @@
	1	+// SPDX-License-Identifier: GPL-2.0
	2	+
	3	+//! Slices to user space memory regions.
	4	+//!
	5	+//! C header: [`include/linux/uaccess.h`](srctree/include/linux/uaccess.h)
	6	+
	7	+use crate::{
	8	+ alloc::Flags,
	9	+ bindings,
	10	+ error::Result,
	11	+ prelude::*,
	12	+ types::{AsBytes, FromBytes},
	13	+};
	14	+use alloc::vec::Vec;
	15	+use core::ffi::{c_ulong, c_void};
	16	+use core::mem::{size_of, MaybeUninit};
	17	+
	18	+/// The type used for userspace addresses.
	19	+pub type UserPtr = usize;
	20	+
	21	+/// A pointer to an area in userspace memory, which can be either read-only or read-write.
	22	+///
	23	+/// All methods on this struct are safe: attempting to read or write on bad addresses (either out of
	24	+/// the bound of the slice or unmapped addresses) will return [`EFAULT`]. Concurrent access,
	25	+/// including data races to/from userspace memory, is permitted, because fundamentally another
	26	+/// userspace thread/process could always be modifying memory at the same time (in the same way that
	27	+/// userspace Rust's [`std::io`] permits data races with the contents of files on disk). In the
	28	+/// presence of a race, the exact byte values read/written are unspecified but the operation is
	29	+/// well-defined. Kernelspace code should validate its copy of data after completing a read, and not
	30	+/// expect that multiple reads of the same address will return the same value.
	31	+///
	32	+/// These APIs are designed to make it difficult to accidentally write TOCTOU (time-of-check to
	33	+/// time-of-use) bugs. Every time a memory location is read, the reader's position is advanced by
	34	+/// the read length and the next read will start from there. This helps prevent accidentally reading
	35	+/// the same location twice and causing a TOCTOU bug.
	36	+///
	37	+/// Creating a [`UserSliceReader`] and/or [`UserSliceWriter`] consumes the `UserSlice`, helping
	38	+/// ensure that there aren't multiple readers or writers to the same location.
	39	+///
	40	+/// If double-fetching a memory location is necessary for some reason, then that is done by creating
	41	+/// multiple readers to the same memory location, e.g. using [`clone_reader`].
	42	+///
	43	+/// # Examples
	44	+///
	45	+/// Takes a region of userspace memory from the current process, and modify it by adding one to
	46	+/// every byte in the region.
	47	+///
	48	+/// ```no_run
	49	+/// use alloc::vec::Vec;
	50	+/// use core::ffi::c_void;
	51	+/// use kernel::error::Result;
	52	+/// use kernel::uaccess::{UserPtr, UserSlice};
	53	+///
	54	+/// fn bytes_add_one(uptr: UserPtr, len: usize) -> Result<()> {
	55	+/// let (read, mut write) = UserSlice::new(uptr, len).reader_writer();
	56	+///
	57	+/// let mut buf = Vec::new();
	58	+/// read.read_all(&mut buf, GFP_KERNEL)?;
	59	+///
	60	+/// for b in &mut buf {
	61	+/// *b = b.wrapping_add(1);
	62	+/// }
	63	+///
	64	+/// write.write_slice(&buf)?;
	65	+/// Ok(())
	66	+/// }
	67	+/// ```
	68	+///
	69	+/// Example illustrating a TOCTOU (time-of-check to time-of-use) bug.
	70	+///
	71	+/// ```no_run
	72	+/// use alloc::vec::Vec;
	73	+/// use core::ffi::c_void;
	74	+/// use kernel::error::{code::EINVAL, Result};
	75	+/// use kernel::uaccess::{UserPtr, UserSlice};
	76	+///
	77	+/// /// Returns whether the data in this region is valid.
	78	+/// fn is_valid(uptr: UserPtr, len: usize) -> Result<bool> {
	79	+/// let read = UserSlice::new(uptr, len).reader();
	80	+///
	81	+/// let mut buf = Vec::new();
	82	+/// read.read_all(&mut buf, GFP_KERNEL)?;
	83	+///
	84	+/// todo!()
	85	+/// }
	86	+///
	87	+/// /// Returns the bytes behind this user pointer if they are valid.
	88	+/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<Vec<u8>> {
	89	+/// if !is_valid(uptr, len)? {
	90	+/// return Err(EINVAL);
	91	+/// }
	92	+///
	93	+/// let read = UserSlice::new(uptr, len).reader();
	94	+///
	95	+/// let mut buf = Vec::new();
	96	+/// read.read_all(&mut buf, GFP_KERNEL)?;
	97	+///
	98	+/// // THIS IS A BUG! The bytes could have changed since we checked them.
	99	+/// //
	100	+/// // To avoid this kind of bug, don't call `UserSlice::new` multiple
	101	+/// // times with the same address.
	102	+/// Ok(buf)
	103	+/// }
	104	+/// ```
	105	+///
	106	+/// [`std::io`]: https://doc.rust-lang.org/std/io/index.html
	107	+/// [`clone_reader`]: UserSliceReader::clone_reader
	108	+pub struct UserSlice {
	109	+ ptr: UserPtr,
	110	+ length: usize,
	111	+}
	112	+
	113	+impl UserSlice {
	114	+ /// Constructs a user slice from a raw pointer and a length in bytes.
	115	+ ///
	116	+ /// Constructing a [`UserSlice`] performs no checks on the provided address and length, it can
	117	+ /// safely be constructed inside a kernel thread with no current userspace process. Reads and
	118	+ /// writes wrap the kernel APIs `copy_from_user` and `copy_to_user`, which check the memory map
	119	+ /// of the current process and enforce that the address range is within the user range (no
	120	+ /// additional calls to `access_ok` are needed). Validity of the pointer is checked when you
	121	+ /// attempt to read or write, not in the call to `UserSlice::new`.
	122	+ ///
	123	+ /// Callers must be careful to avoid time-of-check-time-of-use (TOCTOU) issues. The simplest way
	124	+ /// is to create a single instance of [`UserSlice`] per user memory block as it reads each byte
	125	+ /// at most once.
	126	+ pub fn new(ptr: UserPtr, length: usize) -> Self {
	127	+ UserSlice { ptr, length }
	128	+ }
	129	+
	130	+ /// Reads the entirety of the user slice, appending it to the end of the provided buffer.
	131	+ ///
	132	+ /// Fails with [`EFAULT`] if the read happens on a bad address.
	133	+ pub fn read_all(self, buf: &mut Vec<u8>, flags: Flags) -> Result {
	134	+ self.reader().read_all(buf, flags)
	135	+ }
	136	+
	137	+ /// Constructs a [`UserSliceReader`].
	138	+ pub fn reader(self) -> UserSliceReader {
	139	+ UserSliceReader {
	140	+ ptr: self.ptr,
	141	+ length: self.length,
	142	+ }
	143	+ }
	144	+
	145	+ /// Constructs a [`UserSliceWriter`].
	146	+ pub fn writer(self) -> UserSliceWriter {
	147	+ UserSliceWriter {
	148	+ ptr: self.ptr,
	149	+ length: self.length,
	150	+ }
	151	+ }
	152	+
	153	+ /// Constructs both a [`UserSliceReader`] and a [`UserSliceWriter`].
	154	+ ///
	155	+ /// Usually when this is used, you will first read the data, and then overwrite it afterwards.
	156	+ pub fn reader_writer(self) -> (UserSliceReader, UserSliceWriter) {
	157	+ (
	158	+ UserSliceReader {
	159	+ ptr: self.ptr,
	160	+ length: self.length,
	161	+ },
	162	+ UserSliceWriter {
	163	+ ptr: self.ptr,
	164	+ length: self.length,
	165	+ },
	166	+ )
	167	+ }
	168	+}
	169	+
	170	+/// A reader for [`UserSlice`].
	171	+///
	172	+/// Used to incrementally read from the user slice.
	173	+pub struct UserSliceReader {
	174	+ ptr: UserPtr,
	175	+ length: usize,
	176	+}
	177	+
	178	+impl UserSliceReader {
	179	+ /// Skip the provided number of bytes.
	180	+ ///
	181	+ /// Returns an error if skipping more than the length of the buffer.
	182	+ pub fn skip(&mut self, num_skip: usize) -> Result {
	183	+ // Update `self.length` first since that's the fallible part of this operation.
	184	+ self.length = self.length.checked_sub(num_skip).ok_or(EFAULT)?;
	185	+ self.ptr = self.ptr.wrapping_add(num_skip);
	186	+ Ok(())
	187	+ }
	188	+
	189	+ /// Create a reader that can access the same range of data.
	190	+ ///
	191	+ /// Reading from the clone does not advance the current reader.
	192	+ ///
	193	+ /// The caller should take care to not introduce TOCTOU issues, as described in the
	194	+ /// documentation for [`UserSlice`].
	195	+ pub fn clone_reader(&self) -> UserSliceReader {
	196	+ UserSliceReader {
	197	+ ptr: self.ptr,
	198	+ length: self.length,
	199	+ }
	200	+ }
	201	+
	202	+ /// Returns the number of bytes left to be read from this reader.
	203	+ ///
	204	+ /// Note that even reading less than this number of bytes may fail.
	205	+ pub fn len(&self) -> usize {
	206	+ self.length
	207	+ }
	208	+
	209	+ /// Returns `true` if no data is available in the io buffer.
	210	+ pub fn is_empty(&self) -> bool {
	211	+ self.length == 0
	212	+ }
	213	+
	214	+ /// Reads raw data from the user slice into a kernel buffer.
	215	+ ///
	216	+ /// For a version that uses `&mut [u8]`, please see [`UserSliceReader::read_slice`].
	217	+ ///
	218	+ /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	219	+ /// bounds of this [`UserSliceReader`]. This call may modify `out` even if it returns an error.
	220	+ ///
	221	+ /// # Guarantees
	222	+ ///
	223	+ /// After a successful call to this method, all bytes in `out` are initialized.
	224	+ pub fn read_raw(&mut self, out: &mut [MaybeUninit<u8>]) -> Result {
	225	+ let len = out.len();
	226	+ let out_ptr = out.as_mut_ptr().cast::<c_void>();
	227	+ if len > self.length {
	228	+ return Err(EFAULT);
	229	+ }
	230	+ let Ok(len_ulong) = c_ulong::try_from(len) else {
	231	+ return Err(EFAULT);
	232	+ };
	233	+ // SAFETY: `out_ptr` points into a mutable slice of length `len_ulong`, so we may write
	234	+ // that many bytes to it.
	235	+ let res =
	236	+ unsafe { bindings::copy_from_user(out_ptr, self.ptr as *const c_void, len_ulong) };
	237	+ if res != 0 {
	238	+ return Err(EFAULT);
	239	+ }
	240	+ self.ptr = self.ptr.wrapping_add(len);
	241	+ self.length -= len;
	242	+ Ok(())
	243	+ }
	244	+
	245	+ /// Reads raw data from the user slice into a kernel buffer.
	246	+ ///
	247	+ /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	248	+ /// bounds of this [`UserSliceReader`]. This call may modify `out` even if it returns an error.
	249	+ pub fn read_slice(&mut self, out: &mut [u8]) -> Result {
	250	+ // SAFETY: The types are compatible and `read_raw` doesn't write uninitialized bytes to
	251	+ // `out`.
	252	+ let out = unsafe { &mut (out as mut [u8] as *mut [MaybeUninit<u8>]) };
	253	+ self.read_raw(out)
	254	+ }
	255	+
	256	+ /// Reads a value of the specified type.
	257	+ ///
	258	+ /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	259	+ /// bounds of this [`UserSliceReader`].
	260	+ pub fn read<T: FromBytes>(&mut self) -> Result<T> {
	261	+ let len = size_of::<T>();
	262	+ if len > self.length {
	263	+ return Err(EFAULT);
	264	+ }
	265	+ let Ok(len_ulong) = c_ulong::try_from(len) else {
	266	+ return Err(EFAULT);
	267	+ };
	268	+ let mut out: MaybeUninit<T> = MaybeUninit::uninit();
	269	+ // SAFETY: The local variable `out` is valid for writing `size_of::<T>()` bytes.
	270	+ //
	271	+ // By using the _copy_from_user variant, we skip the check_object_size check that verifies
	272	+ // the kernel pointer. This mirrors the logic on the C side that skips the check when the
	273	+ // length is a compile-time constant.
	274	+ let res = unsafe {
	275	+ bindings::_copy_from_user(
	276	+ out.as_mut_ptr().cast::<c_void>(),
	277	+ self.ptr as *const c_void,
	278	+ len_ulong,
	279	+ )
	280	+ };
	281	+ if res != 0 {
	282	+ return Err(EFAULT);
	283	+ }
	284	+ self.ptr = self.ptr.wrapping_add(len);
	285	+ self.length -= len;
	286	+ // SAFETY: The read above has initialized all bytes in `out`, and since `T` implements
	287	+ // `FromBytes`, any bit-pattern is a valid value for this type.
	288	+ Ok(unsafe { out.assume_init() })
	289	+ }
	290	+
	291	+ /// Reads the entirety of the user slice, appending it to the end of the provided buffer.
	292	+ ///
	293	+ /// Fails with [`EFAULT`] if the read happens on a bad address.
	294	+ pub fn read_all(mut self, buf: &mut Vec<u8>, flags: Flags) -> Result {
	295	+ let len = self.length;
	296	+ VecExt::<u8>::reserve(buf, len, flags)?;
	297	+
	298	+ // The call to `try_reserve` was successful, so the spare capacity is at least `len` bytes
	299	+ // long.
	300	+ self.read_raw(&mut buf.spare_capacity_mut()[..len])?;
	301	+
	302	+ // SAFETY: Since the call to `read_raw` was successful, so the next `len` bytes of the
	303	+ // vector have been initialized.
	304	+ unsafe { buf.set_len(buf.len() + len) };
	305	+ Ok(())
	306	+ }
	307	+}
	308	+
	309	+/// A writer for [`UserSlice`].
	310	+///
	311	+/// Used to incrementally write into the user slice.
	312	+pub struct UserSliceWriter {
	313	+ ptr: UserPtr,
	314	+ length: usize,
	315	+}
	316	+
	317	+impl UserSliceWriter {
	318	+ /// Returns the amount of space remaining in this buffer.
	319	+ ///
	320	+ /// Note that even writing less than this number of bytes may fail.
	321	+ pub fn len(&self) -> usize {
	322	+ self.length
	323	+ }
	324	+
	325	+ /// Returns `true` if no more data can be written to this buffer.
	326	+ pub fn is_empty(&self) -> bool {
	327	+ self.length == 0
	328	+ }
	329	+
	330	+ /// Writes raw data to this user pointer from a kernel buffer.
	331	+ ///
	332	+ /// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
	333	+ /// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
	334	+ /// if it returns an error.
	335	+ pub fn write_slice(&mut self, data: &[u8]) -> Result {
	336	+ let len = data.len();
	337	+ let data_ptr = data.as_ptr().cast::<c_void>();
	338	+ if len > self.length {
	339	+ return Err(EFAULT);
	340	+ }
	341	+ let Ok(len_ulong) = c_ulong::try_from(len) else {
	342	+ return Err(EFAULT);
	343	+ };
	344	+ // SAFETY: `data_ptr` points into an immutable slice of length `len_ulong`, so we may read
	345	+ // that many bytes from it.
	346	+ let res = unsafe { bindings::copy_to_user(self.ptr as *mut c_void, data_ptr, len_ulong) };
	347	+ if res != 0 {
	348	+ return Err(EFAULT);
	349	+ }
	350	+ self.ptr = self.ptr.wrapping_add(len);
	351	+ self.length -= len;
	352	+ Ok(())
	353	+ }
	354	+
	355	+ /// Writes the provided Rust value to this userspace pointer.
	356	+ ///
	357	+ /// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
	358	+ /// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
	359	+ /// if it returns an error.
	360	+ pub fn write<T: AsBytes>(&mut self, value: &T) -> Result {
	361	+ let len = size_of::<T>();
	362	+ if len > self.length {
	363	+ return Err(EFAULT);
	364	+ }
	365	+ let Ok(len_ulong) = c_ulong::try_from(len) else {
	366	+ return Err(EFAULT);
	367	+ };
	368	+ // SAFETY: The reference points to a value of type `T`, so it is valid for reading
	369	+ // `size_of::<T>()` bytes.
	370	+ //
	371	+ // By using the _copy_to_user variant, we skip the check_object_size check that verifies the
	372	+ // kernel pointer. This mirrors the logic on the C side that skips the check when the length
	373	+ // is a compile-time constant.
	374	+ let res = unsafe {
	375	+ bindings::_copy_to_user(
	376	+ self.ptr as *mut c_void,
	377	+ (value as *const T).cast::<c_void>(),
	378	+ len_ulong,
	379	+ )
	380	+ };
	381	+ if res != 0 {
	382	+ return Err(EFAULT);
	383	+ }
	384	+ self.ptr = self.ptr.wrapping_add(len);
	385	+ self.length -= len;
	386	+ Ok(())
	387	+ }
	388	+}

rust/kernel/workqueue.rs+9−7

@@ -482,24 +482,26 @@ pub unsafe trait HasWork<T, const ID: u64 = 0> { · 481 unmodified lines
482	482	/// use kernel::sync::Arc;
483	483	/// use kernel::workqueue::{self, impl_has_work, Work};
484	484	///
485		−/// struct MyStruct {
486		−/// work_field: Work<MyStruct, 17>,
	485	+/// struct MyStruct<'a, T, const N: usize> {
	486	+/// work_field: Work<MyStruct<'a, T, N>, 17>,
	487	+/// f: fn(&'a [T; N]),
487	488	/// }
488	489	///
489	490	/// impl_has_work! {
490		−/// impl HasWork<MyStruct, 17> for MyStruct { self.work_field }
	491	+/// impl{'a, T, const N: usize} HasWork<MyStruct<'a, T, N>, 17>
	492	+/// for MyStruct<'a, T, N> { self.work_field }
491	493	/// }
492	494	/// ```
493	495	#[macro_export]
494	496	macro_rules! impl_has_work {
495		− ($(impl$(<$($implarg:ident),*>)?
	497	+ ($(impl$({$($generics:tt)*})?
496	498	HasWork<$work_type:ty $(, $id:tt)?>
497		− for $self:ident $(<$($selfarg:ident),*>)?
	499	+ for $self:ty
498	500	{ self.$field:ident }
499	501	)*) => {$(
500	502	// SAFETY: The implementation of `raw_get_work` only compiles if the field has the right
501	503	// type.
502		− unsafe impl$(<$($implarg),>)? $crate::workqueue::HasWork<$work_type $(, $id)?> for $self $(<$($selfarg),>)? {
	504	+ unsafe impl$(<$($generics)+>)? $crate::workqueue::HasWork<$work_type $(, $id)?> for $self {
503	505	const OFFSET: usize = ::core::mem::offset_of!(Self, $field) as usize;
504	506
505	507	#[inline]
@@ -515,7 +517,7 @@ macro_rules! impl_has_work { · 9 unmodified lines
515	517	pub use impl_has_work;
516	518
517	519	impl_has_work! {
518		− impl<T> HasWork<Self> for ClosureWork<T> { self.work }
	520	+ impl{T} HasWork<Self> for ClosureWork<T> { self.work }
519	521	}
520	522
521	523	unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Arc<T>

rust/macros/lib.rs+39−6

@@ -35,6 +35,7 @@ use proc_macro::TokenStream; · 34 unmodified lines
35	35	/// author: "Rust for Linux Contributors",
36	36	/// description: "My very own kernel module!",
37	37	/// license: "GPL",
	38	+/// alias: ["alternate_module_name"],
38	39	/// }
39	40	///
40	41	/// struct MyModule;
@@ -55,13 +56,45 @@ use proc_macro::TokenStream; · 14 unmodified lines
55	56	/// }
56	57	/// ```
57	58	///
	59	+/// ## Firmware
	60	+///
	61	+/// The following example shows how to declare a kernel module that needs
	62	+/// to load binary firmware files. You need to specify the file names of
	63	+/// the firmware in the `firmware` field. The information is embedded
	64	+/// in the `modinfo` section of the kernel module. For example, a tool to
	65	+/// build an initramfs uses this information to put the firmware files into
	66	+/// the initramfs image.
	67	+///
	68	+/// ```ignore
	69	+/// use kernel::prelude::*;
	70	+///
	71	+/// module!{
	72	+/// type: MyDeviceDriverModule,
	73	+/// name: "my_device_driver_module",
	74	+/// author: "Rust for Linux Contributors",
	75	+/// description: "My device driver requires firmware",
	76	+/// license: "GPL",
	77	+/// firmware: ["my_device_firmware1.bin", "my_device_firmware2.bin"],
	78	+/// }
	79	+///
	80	+/// struct MyDeviceDriverModule;
	81	+///
	82	+/// impl kernel::Module for MyDeviceDriverModule {
	83	+/// fn init() -> Result<Self> {
	84	+/// Ok(Self)
	85	+/// }
	86	+/// }
	87	+/// ```
	88	+///
58	89	/// # Supported argument types
59	90	/// - `type`: type which implements the [`Module`] trait (required).
60		−/// - `name`: byte array of the name of the kernel module (required).
61		−/// - `author`: byte array of the author of the kernel module.
62		−/// - `description`: byte array of the description of the kernel module.
63		−/// - `license`: byte array of the license of the kernel module (required).
64		−/// - `alias`: byte array of alias name of the kernel module.
	91	+/// - `name`: ASCII string literal of the name of the kernel module (required).
	92	+/// - `author`: string literal of the author of the kernel module.
	93	+/// - `description`: string literal of the description of the kernel module.
	94	+/// - `license`: ASCII string literal of the license of the kernel module (required).
	95	+/// - `alias`: array of ASCII string literals of the alias names of the kernel module.
	96	+/// - `firmware`: array of ASCII string literals of the firmware files of
	97	+/// the kernel module.
65	98	#[proc_macro]
66	99	pub fn module(ts: TokenStream) -> TokenStream {
67	100	module::module(ts)
@@ -312,7 +345,7 @@ pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream { · 244 unmodified lines
312	345	///
313	346	/// Currently supported modifiers are:
314	347	/// * `span`: change the span of concatenated identifier to the span of the specified token. By
315		−/// default the span of the `[< >]` group is used.
	348	+/// default the span of the `[< >]` group is used.
316	349	/// * `lower`: change the identifier to lower case.
317	350	/// * `upper`: change the identifier to upper case.
318	351	///

rust/macros/module.rs+16−2

@@ -97,14 +97,22 @@ struct ModuleInfo { · 96 unmodified lines
97	97	author: Option<String>,
98	98	description: Option<String>,
99	99	alias: Option<Vec<String>>,
	100	+ firmware: Option<Vec<String>>,
100	101	}
101	102
102	103	impl ModuleInfo {
103	104	fn parse(it: &mut token_stream::IntoIter) -> Self {
104	105	let mut info = ModuleInfo::default();
105	106
106		− const EXPECTED_KEYS: &[&str] =
107		− &["type", "name", "author", "description", "license", "alias"];
	107	+ const EXPECTED_KEYS: &[&str] = &[
	108	+ "type",
	109	+ "name",
	110	+ "author",
	111	+ "description",
	112	+ "license",
	113	+ "alias",
	114	+ "firmware",
	115	+ ];
108	116	const REQUIRED_KEYS: &[&str] = &["type", "name", "license"];
109	117	let mut seen_keys = Vec::new();
110	118
@@ -131,6 +139,7 @@ impl ModuleInfo { · 20 unmodified lines
131	139	"description" => info.description = Some(expect_string(it)),
132	140	"license" => info.license = expect_string_ascii(it),
133	141	"alias" => info.alias = Some(expect_string_array(it)),
	142	+ "firmware" => info.firmware = Some(expect_string_array(it)),
134	143	_ => panic!(
135	144	"Unknown key \"{}\". Valid keys are: {:?}.",
136	145	key, EXPECTED_KEYS
@@ -186,6 +195,11 @@ pub(crate) fn module(ts: TokenStream) -> TokenStream { · 49 unmodified lines
186	195	modinfo.emit("alias", &alias);
187	196	}
188	197	}
	198	+ if let Some(firmware) = info.firmware {
	199	+ for fw in firmware {
	200	+ modinfo.emit("firmware", &fw);
	201	+ }
	202	+ }
189	203
190	204	// Built-in modules also export the `file` modinfo string.
191	205	let file =

rust/uapi/lib.rs+1−0

@@ -14,6 +14,7 @@ · 13 unmodified lines
14	14	#![cfg_attr(test, allow(unsafe_op_in_unsafe_fn))]
15	15	#![allow(
16	16	clippy::all,
	17	+ dead_code,
17	18	missing_docs,
18	19	non_camel_case_types,
19	20	non_upper_case_globals,