Merge tag 'rust-6.11' of https://github.com/Rust-for-Linux/linux

Pull Rust updates from Miguel Ojeda: "The highlight is the establishment of a minimum version for the Rust toolchain, including 'rustc' (and bundled tools) and 'bindgen'. The initial minimum will be the pinned version we currently have, i.e. we are just widening the allowed versions. That covers three stable Rust releases: 1.78.0, 1.79.0, 1.80.0 (getting released tomorrow), plus beta, plus nightly. This should already be enough for kernel developers in distributions that provide recent Rust compiler versions routinely, such as Arch Linux, Debian Unstable (outside the freeze period), Fedora Linux, Gentoo Linux (especially the testing channel), Nix (unstable) and openSUSE Slowroll and Tumbleweed. In addition, the kernel is now being built-tested by Rust's pre-merge CI. That is, every change that is attempting to land into the Rust compiler is tested against the kernel, and it is merged only if it passes. Similarly, the bindgen tool has agreed to build the kernel in their CI too. Thus, with the pre-merge CI in place, both projects hope to avoid unintentional changes to Rust that break the kernel. This means that, in general, apart from intentional changes on their side (that we will need to workaround conditionally on our side), the upcoming Rust compiler versions should generally work. In addition, the Rust project has proposed getting the kernel into stable Rust (at least solving the main blockers) as one of its three flagship goals for 2024H2 [1]. I would like to thank Niko, Sid, Emilio et al. for their help promoting the collaboration between Rust and the kernel. Toolchain and infrastructure: - Support several Rust toolchain versions. - Support several bindgen versions. - Remove 'cargo' requirement and simplify 'rusttest', thanks to 'alloc' having been dropped last cycle. - Provide proper error reporting for the 'rust-analyzer' target. 'kernel' crate: - Add 'uaccess' module with a safe userspace pointers abstraction. - Add 'page' module with a 'struct page' abstraction. - Support more complex generics in workqueue's 'impl_has_work!' macro. 'macros' crate: - Add 'firmware' field support to the 'module!' macro. - Improve 'module!' macro documentation. Documentation: - Provide instructions on what packages should be installed to build the kernel in some popular Linux distributions. - Introduce the new kernel.org LLVM+Rust toolchains. - Explain '#[no_std]'. And a few other small bits" Link: https://rust-lang.github.io/rust-project-goals/2024h2/index.html#flagship-goals [1] * tag 'rust-6.11' of https://github.com/Rust-for-Linux/linux: (26 commits) docs: rust: quick-start: add section on Linux distributions rust: warn about `bindgen` versions 0.66.0 and 0.66.1 rust: start supporting several `bindgen` versions rust: work around `bindgen` 0.69.0 issue rust: avoid assuming a particular `bindgen` build rust: start supporting several compiler versions rust: simplify Clippy warning flags set rust: relax most deny-level lints to warnings rust: allow `dead_code` for never constructed bindings rust: init: simplify from `map_err` to `inspect_err` rust: macros: indent list item in `paste!`'s docs rust: add abstraction for `struct page` rust: uaccess: add typed accessors for userspace pointers uaccess: always export _copy_[from|to]_user with CONFIG_RUST rust: uaccess: add userspace pointers kbuild: rust-analyzer: improve comment documentation kbuild: rust-analyzer: better error handling docs: rust: no_std is used rust: alloc: add __GFP_HIGHMEM flag rust: alloc: fix typo in docs for GFP_NOWAIT ...

Linus Torvalds committed 2 years ago

Commit 910bfc22 parents ff30564 b12634125 files · +1058 −236Browse files

Showing the changes to rust/kernel (7 of 25 files). Show all

Unified · Split

7 files

rust/kernel/alloc.rs +16 −1
rust/kernel/init.rs +4 −9
rust/kernel/lib.rs +2 −0
rust/kernel/page.rs +250 −0
rust/kernel/types.rs +64 −0
rust/kernel/uaccess.rs +388 −0
rust/kernel/workqueue.rs +9 −7

rust/kernel/alloc.rs+16−1

 #[derive(Clone, Copy)]
 pub struct Flags(u32);
+impl Flags {
+    /// Get the raw representation of this flag.
+    pub(crate) fn as_raw(self) -> u32 {
+        self.0
+    }
+}
 impl core::ops::BitOr for Flags {
     type Output = Self;
     fn bitor(self, rhs: Self) -> Self::Output {
     /// This is normally or'd with other flags.
     pub const __GFP_ZERO: Flags = Flags(bindings::__GFP_ZERO);
+    /// Allow the allocation to be in high memory.
+    ///
+    /// Allocations in high memory may not be mapped into the kernel's address space, so this can't
+    /// be used with `kmalloc` and other similar methods.
+    ///
+    /// This is normally or'd with other flags.
+    pub const __GFP_HIGHMEM: Flags = Flags(bindings::__GFP_HIGHMEM);
     /// Users can not sleep and need the allocation to succeed.
     ///
     /// A lower watermark is applied to allow access to "atomic reserves". The current
     /// The same as [`GFP_KERNEL`], except the allocation is accounted to kmemcg.
     pub const GFP_KERNEL_ACCOUNT: Flags = Flags(bindings::GFP_KERNEL_ACCOUNT);
-    /// Ror kernel allocations that should not stall for direct reclaim, start physical IO or
+    /// For kernel allocations that should not stall for direct reclaim, start physical IO or
     /// use any filesystem callback.  It is very likely to fail to allocate memory, even for very
     /// small allocations.
     pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);

rust/kernel/init.rs+4−9

         let val = unsafe { &mut *slot };
         // SAFETY: `slot` is considered pinned.
         let val = unsafe { Pin::new_unchecked(val) };
-        (self.1)(val).map_err(|e| {
+        // SAFETY: `slot` was initialized above.
-            // SAFETY: `slot` was initialized above.
+        (self.1)(val).inspect_err(|_| unsafe { core::ptr::drop_in_place(slot) })
-            unsafe { core::ptr::drop_in_place(slot) };
-            e
-        })
     }
 }
         // SAFETY: All requirements fulfilled since this function is `__init`.
         unsafe { self.0.__pinned_init(slot)? };
         // SAFETY: The above call initialized `slot` and we still have unique access.
-        (self.1)(unsafe { &mut *slot }).map_err(|e| {
+        (self.1)(unsafe { &mut *slot }).inspect_err(|_|
             // SAFETY: `slot` was initialized above.
-            unsafe { core::ptr::drop_in_place(slot) };
+            unsafe { core::ptr::drop_in_place(slot) })
-            e
-        })
     }
 }

rust/kernel/lib.rs+2−0

 pub mod kunit;
 #[cfg(CONFIG_NET)]
 pub mod net;
+pub mod page;
 pub mod prelude;
 pub mod print;
 mod static_assert;
 pub mod task;
 pub mod time;
 pub mod types;
+pub mod uaccess;
 pub mod workqueue;
 #[doc(hidden)]

rust/kernel/page.rs+250−0

@@ -0,0 +1,250 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2
	3	//! Kernel page allocation and management.
	4
	5	use crate::{
	6	alloc::{AllocError, Flags},
	7	bindings,
	8	error::code::*,
	9	error::Result,
	10	uaccess::UserSliceReader,
	11	};
	12	use core::ptr::{self, NonNull};
	13
	14	/// A bitwise shift for the page size.
	15	pub const PAGE_SHIFT: usize = bindings::PAGE_SHIFT as usize;
	16
	17	/// The number of bytes in a page.
	18	pub const PAGE_SIZE: usize = bindings::PAGE_SIZE;
	19
	20	/// A bitmask that gives the page containing a given address.
	21	pub const PAGE_MASK: usize = !(PAGE_SIZE - 1);
	22
	23	/// A pointer to a page that owns the page allocation.
	24	///
	25	/// # Invariants
	26	///
	27	/// The pointer is valid, and has ownership over the page.
	28	pub struct Page {
	29	page: NonNull<bindings::page>,
	30	}
	31
	32	// SAFETY: Pages have no logic that relies on them staying on a given thread, so moving them across
	33	// threads is safe.
	34	unsafe impl Send for Page {}
	35
	36	// SAFETY: Pages have no logic that relies on them not being accessed concurrently, so accessing
	37	// them concurrently is safe.
	38	unsafe impl Sync for Page {}
	39
	40	impl Page {
	41	/// Allocates a new page.
	42	///
	43	/// # Examples
	44	///
	45	/// Allocate memory for a page.
	46	///
	47	/// ```
	48	/// use kernel::page::Page;
	49	///
	50	/// # fn dox() -> Result<(), kernel::alloc::AllocError> {
	51	/// let page = Page::alloc_page(GFP_KERNEL)?;
	52	/// # Ok(()) }
	53	/// ```
	54	///
	55	/// Allocate memory for a page and zero its contents.
	56	///
	57	/// ```
	58	/// use kernel::page::Page;
	59	///
	60	/// # fn dox() -> Result<(), kernel::alloc::AllocError> {
	61	/// let page = Page::alloc_page(GFP_KERNEL \| __GFP_ZERO)?;
	62	/// # Ok(()) }
	63	/// ```
	64	pub fn alloc_page(flags: Flags) -> Result<Self, AllocError> {
	65	// SAFETY: Depending on the value of `gfp_flags`, this call may sleep. Other than that, it
	66	// is always safe to call this method.
	67	let page = unsafe { bindings::alloc_pages(flags.as_raw(), 0) };
	68	let page = NonNull::new(page).ok_or(AllocError)?;
	69	// INVARIANT: We just successfully allocated a page, so we now have ownership of the newly
	70	// allocated page. We transfer that ownership to the new `Page` object.
	71	Ok(Self { page })
	72	}
	73
	74	/// Returns a raw pointer to the page.
	75	pub fn as_ptr(&self) -> *mut bindings::page {
	76	self.page.as_ptr()
	77	}
	78
	79	/// Runs a piece of code with this page mapped to an address.
	80	///
	81	/// The page is unmapped when this call returns.
	82	///
	83	/// # Using the raw pointer
	84	///
	85	/// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for
	86	/// `PAGE_SIZE` bytes and for the duration in which the closure is called. The pointer might
	87	/// only be mapped on the current thread, and when that is the case, dereferencing it on other
	88	/// threads is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't
	89	/// cause data races, the memory may be uninitialized, and so on.
	90	///
	91	/// If multiple threads map the same page at the same time, then they may reference with
	92	/// different addresses. However, even if the addresses are different, the underlying memory is
	93	/// still the same for these purposes (e.g., it's still a data race if they both write to the
	94	/// same underlying byte at the same time).
	95	fn with_page_mapped<T>(&self, f: impl FnOnce(*mut u8) -> T) -> T {
	96	// SAFETY: `page` is valid due to the type invariants on `Page`.
	97	let mapped_addr = unsafe { bindings::kmap_local_page(self.as_ptr()) };
	98
	99	let res = f(mapped_addr.cast());
	100
	101	// This unmaps the page mapped above.
	102	//
	103	// SAFETY: Since this API takes the user code as a closure, it can only be used in a manner
	104	// where the pages are unmapped in reverse order. This is as required by `kunmap_local`.
	105	//
	106	// In other words, if this call to `kunmap_local` happens when a different page should be
	107	// unmapped first, then there must necessarily be a call to `kmap_local_page` other than the
	108	// call just above in `with_page_mapped` that made that possible. In this case, it is the
	109	// unsafe block that wraps that other call that is incorrect.
	110	unsafe { bindings::kunmap_local(mapped_addr) };
	111
	112	res
	113	}
	114
	115	/// Runs a piece of code with a raw pointer to a slice of this page, with bounds checking.
	116	///
	117	/// If `f` is called, then it will be called with a pointer that points at `off` bytes into the
	118	/// page, and the pointer will be valid for at least `len` bytes. The pointer is only valid on
	119	/// this task, as this method uses a local mapping.
	120	///
	121	/// If `off` and `len` refers to a region outside of this page, then this method returns
	122	/// [`EINVAL`] and does not call `f`.
	123	///
	124	/// # Using the raw pointer
	125	///
	126	/// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for
	127	/// `len` bytes and for the duration in which the closure is called. The pointer might only be
	128	/// mapped on the current thread, and when that is the case, dereferencing it on other threads
	129	/// is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't cause
	130	/// data races, the memory may be uninitialized, and so on.
	131	///
	132	/// If multiple threads map the same page at the same time, then they may reference with
	133	/// different addresses. However, even if the addresses are different, the underlying memory is
	134	/// still the same for these purposes (e.g., it's still a data race if they both write to the
	135	/// same underlying byte at the same time).
	136	fn with_pointer_into_page<T>(
	137	&self,
	138	off: usize,
	139	len: usize,
	140	f: impl FnOnce(*mut u8) -> Result<T>,
	141	) -> Result<T> {
	142	let bounds_ok = off <= PAGE_SIZE && len <= PAGE_SIZE && (off + len) <= PAGE_SIZE;
	143
	144	if bounds_ok {
	145	self.with_page_mapped(move \|page_addr\| {
	146	// SAFETY: The `off` integer is at most `PAGE_SIZE`, so this pointer offset will
	147	// result in a pointer that is in bounds or one off the end of the page.
	148	f(unsafe { page_addr.add(off) })
	149	})
	150	} else {
	151	Err(EINVAL)
	152	}
	153	}
	154
	155	/// Maps the page and reads from it into the given buffer.
	156	///
	157	/// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	158	/// outside of the page, then this call returns [`EINVAL`].
	159	///
	160	/// # Safety
	161	///
	162	/// * Callers must ensure that `dst` is valid for writing `len` bytes.
	163	/// * Callers must ensure that this call does not race with a write to the same page that
	164	/// overlaps with this read.
	165	pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result {
	166	self.with_pointer_into_page(offset, len, move \|src\| {
	167	// SAFETY: If `with_pointer_into_page` calls into this closure, then
	168	// it has performed a bounds check and guarantees that `src` is
	169	// valid for `len` bytes.
	170	//
	171	// There caller guarantees that there is no data race.
	172	unsafe { ptr::copy_nonoverlapping(src, dst, len) };
	173	Ok(())
	174	})
	175	}
	176
	177	/// Maps the page and writes into it from the given buffer.
	178	///
	179	/// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	180	/// outside of the page, then this call returns [`EINVAL`].
	181	///
	182	/// # Safety
	183	///
	184	/// * Callers must ensure that `src` is valid for reading `len` bytes.
	185	/// * Callers must ensure that this call does not race with a read or write to the same page
	186	/// that overlaps with this write.
	187	pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Result {
	188	self.with_pointer_into_page(offset, len, move \|dst\| {
	189	// SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	190	// bounds check and guarantees that `dst` is valid for `len` bytes.
	191	//
	192	// There caller guarantees that there is no data race.
	193	unsafe { ptr::copy_nonoverlapping(src, dst, len) };
	194	Ok(())
	195	})
	196	}
	197
	198	/// Maps the page and zeroes the given slice.
	199	///
	200	/// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	201	/// outside of the page, then this call returns [`EINVAL`].
	202	///
	203	/// # Safety
	204	///
	205	/// Callers must ensure that this call does not race with a read or write to the same page that
	206	/// overlaps with this write.
	207	pub unsafe fn fill_zero_raw(&self, offset: usize, len: usize) -> Result {
	208	self.with_pointer_into_page(offset, len, move \|dst\| {
	209	// SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	210	// bounds check and guarantees that `dst` is valid for `len` bytes.
	211	//
	212	// There caller guarantees that there is no data race.
	213	unsafe { ptr::write_bytes(dst, 0u8, len) };
	214	Ok(())
	215	})
	216	}
	217
	218	/// Copies data from userspace into this page.
	219	///
	220	/// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
	221	/// outside of the page, then this call returns [`EINVAL`].
	222	///
	223	/// Like the other `UserSliceReader` methods, data races are allowed on the userspace address.
	224	/// However, they are not allowed on the page you are copying into.
	225	///
	226	/// # Safety
	227	///
	228	/// Callers must ensure that this call does not race with a read or write to the same page that
	229	/// overlaps with this write.
	230	pub unsafe fn copy_from_user_slice_raw(
	231	&self,
	232	reader: &mut UserSliceReader,
	233	offset: usize,
	234	len: usize,
	235	) -> Result {
	236	self.with_pointer_into_page(offset, len, move \|dst\| {
	237	// SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
	238	// bounds check and guarantees that `dst` is valid for `len` bytes. Furthermore, we have
	239	// exclusive access to the slice since the caller guarantees that there are no races.
	240	reader.read_raw(unsafe { core::slice::from_raw_parts_mut(dst.cast(), len) })
	241	})
	242	}
	243	}
	244
	245	impl Drop for Page {
	246	fn drop(&mut self) {
	247	// SAFETY: By the type invariants, we have ownership of the page and can free it.
	248	unsafe { bindings::__free_pages(self.page.as_ptr(), 0) };
	249	}
	250	}

rust/kernel/types.rs+64−0

@@ -409,3 +409,67 @@ pub enum Either<L, R> { · 408 unmodified lines
409	/// Constructs an instance of [`Either`] containing a value of type `R`.	409	/// Constructs an instance of [`Either`] containing a value of type `R`.
410	Right(R),	410	Right(R),
411	}	411	}
		412
		413	/// Types for which any bit pattern is valid.
		414	///
		415	/// Not all types are valid for all values. For example, a `bool` must be either zero or one, so
		416	/// reading arbitrary bytes into something that contains a `bool` is not okay.
		417	///
		418	/// It's okay for the type to have padding, as initializing those bytes has no effect.
		419	///
		420	/// # Safety
		421	///
		422	/// All bit-patterns must be valid for this type. This type must not have interior mutability.
		423	pub unsafe trait FromBytes {}
		424
		425	// SAFETY: All bit patterns are acceptable values of the types below.
		426	unsafe impl FromBytes for u8 {}
		427	unsafe impl FromBytes for u16 {}
		428	unsafe impl FromBytes for u32 {}
		429	unsafe impl FromBytes for u64 {}
		430	unsafe impl FromBytes for usize {}
		431	unsafe impl FromBytes for i8 {}
		432	unsafe impl FromBytes for i16 {}
		433	unsafe impl FromBytes for i32 {}
		434	unsafe impl FromBytes for i64 {}
		435	unsafe impl FromBytes for isize {}
		436	// SAFETY: If all bit patterns are acceptable for individual values in an array, then all bit
		437	// patterns are also acceptable for arrays of that type.
		438	unsafe impl<T: FromBytes> FromBytes for [T] {}
		439	unsafe impl<T: FromBytes, const N: usize> FromBytes for [T; N] {}
		440
		441	/// Types that can be viewed as an immutable slice of initialized bytes.
		442	///
		443	/// If a struct implements this trait, then it is okay to copy it byte-for-byte to userspace. This
		444	/// means that it should not have any padding, as padding bytes are uninitialized. Reading
		445	/// uninitialized memory is not just undefined behavior, it may even lead to leaking sensitive
		446	/// information on the stack to userspace.
		447	///
		448	/// The struct should also not hold kernel pointers, as kernel pointer addresses are also considered
		449	/// sensitive. However, leaking kernel pointers is not considered undefined behavior by Rust, so
		450	/// this is a correctness requirement, but not a safety requirement.
		451	///
		452	/// # Safety
		453	///
		454	/// Values of this type may not contain any uninitialized bytes. This type must not have interior
		455	/// mutability.
		456	pub unsafe trait AsBytes {}
		457
		458	// SAFETY: Instances of the following types have no uninitialized portions.
		459	unsafe impl AsBytes for u8 {}
		460	unsafe impl AsBytes for u16 {}
		461	unsafe impl AsBytes for u32 {}
		462	unsafe impl AsBytes for u64 {}
		463	unsafe impl AsBytes for usize {}
		464	unsafe impl AsBytes for i8 {}
		465	unsafe impl AsBytes for i16 {}
		466	unsafe impl AsBytes for i32 {}
		467	unsafe impl AsBytes for i64 {}
		468	unsafe impl AsBytes for isize {}
		469	unsafe impl AsBytes for bool {}
		470	unsafe impl AsBytes for char {}
		471	unsafe impl AsBytes for str {}
		472	// SAFETY: If individual values in an array have no uninitialized portions, then the array itself
		473	// does not have any uninitialized portions either.
		474	unsafe impl<T: AsBytes> AsBytes for [T] {}
		475	unsafe impl<T: AsBytes, const N: usize> AsBytes for [T; N] {}

rust/kernel/uaccess.rs+388−0

@@ -0,0 +1,388 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2
	3	//! Slices to user space memory regions.
	4	//!
	5	//! C header: [`include/linux/uaccess.h`](srctree/include/linux/uaccess.h)
	6
	7	use crate::{
	8	alloc::Flags,
	9	bindings,
	10	error::Result,
	11	prelude::*,
	12	types::{AsBytes, FromBytes},
	13	};
	14	use alloc::vec::Vec;
	15	use core::ffi::{c_ulong, c_void};
	16	use core::mem::{size_of, MaybeUninit};
	17
	18	/// The type used for userspace addresses.
	19	pub type UserPtr = usize;
	20
	21	/// A pointer to an area in userspace memory, which can be either read-only or read-write.
	22	///
	23	/// All methods on this struct are safe: attempting to read or write on bad addresses (either out of
	24	/// the bound of the slice or unmapped addresses) will return [`EFAULT`]. Concurrent access,
	25	/// including data races to/from userspace memory, is permitted, because fundamentally another
	26	/// userspace thread/process could always be modifying memory at the same time (in the same way that
	27	/// userspace Rust's [`std::io`] permits data races with the contents of files on disk). In the
	28	/// presence of a race, the exact byte values read/written are unspecified but the operation is
	29	/// well-defined. Kernelspace code should validate its copy of data after completing a read, and not
	30	/// expect that multiple reads of the same address will return the same value.
	31	///
	32	/// These APIs are designed to make it difficult to accidentally write TOCTOU (time-of-check to
	33	/// time-of-use) bugs. Every time a memory location is read, the reader's position is advanced by
	34	/// the read length and the next read will start from there. This helps prevent accidentally reading
	35	/// the same location twice and causing a TOCTOU bug.
	36	///
	37	/// Creating a [`UserSliceReader`] and/or [`UserSliceWriter`] consumes the `UserSlice`, helping
	38	/// ensure that there aren't multiple readers or writers to the same location.
	39	///
	40	/// If double-fetching a memory location is necessary for some reason, then that is done by creating
	41	/// multiple readers to the same memory location, e.g. using [`clone_reader`].
	42	///
	43	/// # Examples
	44	///
	45	/// Takes a region of userspace memory from the current process, and modify it by adding one to
	46	/// every byte in the region.
	47	///
	48	/// ```no_run
	49	/// use alloc::vec::Vec;
	50	/// use core::ffi::c_void;
	51	/// use kernel::error::Result;
	52	/// use kernel::uaccess::{UserPtr, UserSlice};
	53	///
	54	/// fn bytes_add_one(uptr: UserPtr, len: usize) -> Result<()> {
	55	/// let (read, mut write) = UserSlice::new(uptr, len).reader_writer();
	56	///
	57	/// let mut buf = Vec::new();
	58	/// read.read_all(&mut buf, GFP_KERNEL)?;
	59	///
	60	/// for b in &mut buf {
	61	/// *b = b.wrapping_add(1);
	62	/// }
	63	///
	64	/// write.write_slice(&buf)?;
	65	/// Ok(())
	66	/// }
	67	/// ```
	68	///
	69	/// Example illustrating a TOCTOU (time-of-check to time-of-use) bug.
	70	///
	71	/// ```no_run
	72	/// use alloc::vec::Vec;
	73	/// use core::ffi::c_void;
	74	/// use kernel::error::{code::EINVAL, Result};
	75	/// use kernel::uaccess::{UserPtr, UserSlice};
	76	///
	77	/// /// Returns whether the data in this region is valid.
	78	/// fn is_valid(uptr: UserPtr, len: usize) -> Result<bool> {
	79	/// let read = UserSlice::new(uptr, len).reader();
	80	///
	81	/// let mut buf = Vec::new();
	82	/// read.read_all(&mut buf, GFP_KERNEL)?;
	83	///
	84	/// todo!()
	85	/// }
	86	///
	87	/// /// Returns the bytes behind this user pointer if they are valid.
	88	/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<Vec<u8>> {
	89	/// if !is_valid(uptr, len)? {
	90	/// return Err(EINVAL);
	91	/// }
	92	///
	93	/// let read = UserSlice::new(uptr, len).reader();
	94	///
	95	/// let mut buf = Vec::new();
	96	/// read.read_all(&mut buf, GFP_KERNEL)?;
	97	///
	98	/// // THIS IS A BUG! The bytes could have changed since we checked them.
	99	/// //
	100	/// // To avoid this kind of bug, don't call `UserSlice::new` multiple
	101	/// // times with the same address.
	102	/// Ok(buf)
	103	/// }
	104	/// ```
	105	///
	106	/// [`std::io`]: https://doc.rust-lang.org/std/io/index.html
	107	/// [`clone_reader`]: UserSliceReader::clone_reader
	108	pub struct UserSlice {
	109	ptr: UserPtr,
	110	length: usize,
	111	}
	112
	113	impl UserSlice {
	114	/// Constructs a user slice from a raw pointer and a length in bytes.
	115	///
	116	/// Constructing a [`UserSlice`] performs no checks on the provided address and length, it can
	117	/// safely be constructed inside a kernel thread with no current userspace process. Reads and
	118	/// writes wrap the kernel APIs `copy_from_user` and `copy_to_user`, which check the memory map
	119	/// of the current process and enforce that the address range is within the user range (no
	120	/// additional calls to `access_ok` are needed). Validity of the pointer is checked when you
	121	/// attempt to read or write, not in the call to `UserSlice::new`.
	122	///
	123	/// Callers must be careful to avoid time-of-check-time-of-use (TOCTOU) issues. The simplest way
	124	/// is to create a single instance of [`UserSlice`] per user memory block as it reads each byte
	125	/// at most once.
	126	pub fn new(ptr: UserPtr, length: usize) -> Self {
	127	UserSlice { ptr, length }
	128	}
	129
	130	/// Reads the entirety of the user slice, appending it to the end of the provided buffer.
	131	///
	132	/// Fails with [`EFAULT`] if the read happens on a bad address.
	133	pub fn read_all(self, buf: &mut Vec<u8>, flags: Flags) -> Result {
	134	self.reader().read_all(buf, flags)
	135	}
	136
	137	/// Constructs a [`UserSliceReader`].
	138	pub fn reader(self) -> UserSliceReader {
	139	UserSliceReader {
	140	ptr: self.ptr,
	141	length: self.length,
	142	}
	143	}
	144
	145	/// Constructs a [`UserSliceWriter`].
	146	pub fn writer(self) -> UserSliceWriter {
	147	UserSliceWriter {
	148	ptr: self.ptr,
	149	length: self.length,
	150	}
	151	}
	152
	153	/// Constructs both a [`UserSliceReader`] and a [`UserSliceWriter`].
	154	///
	155	/// Usually when this is used, you will first read the data, and then overwrite it afterwards.
	156	pub fn reader_writer(self) -> (UserSliceReader, UserSliceWriter) {
	157	(
	158	UserSliceReader {
	159	ptr: self.ptr,
	160	length: self.length,
	161	},
	162	UserSliceWriter {
	163	ptr: self.ptr,
	164	length: self.length,
	165	},
	166	)
	167	}
	168	}
	169
	170	/// A reader for [`UserSlice`].
	171	///
	172	/// Used to incrementally read from the user slice.
	173	pub struct UserSliceReader {
	174	ptr: UserPtr,
	175	length: usize,
	176	}
	177
	178	impl UserSliceReader {
	179	/// Skip the provided number of bytes.
	180	///
	181	/// Returns an error if skipping more than the length of the buffer.
	182	pub fn skip(&mut self, num_skip: usize) -> Result {
	183	// Update `self.length` first since that's the fallible part of this operation.
	184	self.length = self.length.checked_sub(num_skip).ok_or(EFAULT)?;
	185	self.ptr = self.ptr.wrapping_add(num_skip);
	186	Ok(())
	187	}
	188
	189	/// Create a reader that can access the same range of data.
	190	///
	191	/// Reading from the clone does not advance the current reader.
	192	///
	193	/// The caller should take care to not introduce TOCTOU issues, as described in the
	194	/// documentation for [`UserSlice`].
	195	pub fn clone_reader(&self) -> UserSliceReader {
	196	UserSliceReader {
	197	ptr: self.ptr,
	198	length: self.length,
	199	}
	200	}
	201
	202	/// Returns the number of bytes left to be read from this reader.
	203	///
	204	/// Note that even reading less than this number of bytes may fail.
	205	pub fn len(&self) -> usize {
	206	self.length
	207	}
	208
	209	/// Returns `true` if no data is available in the io buffer.
	210	pub fn is_empty(&self) -> bool {
	211	self.length == 0
	212	}
	213
	214	/// Reads raw data from the user slice into a kernel buffer.
	215	///
	216	/// For a version that uses `&mut [u8]`, please see [`UserSliceReader::read_slice`].
	217	///
	218	/// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	219	/// bounds of this [`UserSliceReader`]. This call may modify `out` even if it returns an error.
	220	///
	221	/// # Guarantees
	222	///
	223	/// After a successful call to this method, all bytes in `out` are initialized.
	224	pub fn read_raw(&mut self, out: &mut [MaybeUninit<u8>]) -> Result {
	225	let len = out.len();
	226	let out_ptr = out.as_mut_ptr().cast::<c_void>();
	227	if len > self.length {
	228	return Err(EFAULT);
	229	}
	230	let Ok(len_ulong) = c_ulong::try_from(len) else {
	231	return Err(EFAULT);
	232	};
	233	// SAFETY: `out_ptr` points into a mutable slice of length `len_ulong`, so we may write
	234	// that many bytes to it.
	235	let res =
	236	unsafe { bindings::copy_from_user(out_ptr, self.ptr as *const c_void, len_ulong) };
	237	if res != 0 {
	238	return Err(EFAULT);
	239	}
	240	self.ptr = self.ptr.wrapping_add(len);
	241	self.length -= len;
	242	Ok(())
	243	}
	244
	245	/// Reads raw data from the user slice into a kernel buffer.
	246	///
	247	/// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	248	/// bounds of this [`UserSliceReader`]. This call may modify `out` even if it returns an error.
	249	pub fn read_slice(&mut self, out: &mut [u8]) -> Result {
	250	// SAFETY: The types are compatible and `read_raw` doesn't write uninitialized bytes to
	251	// `out`.
	252	let out = unsafe { &mut (out as mut [u8] as *mut [MaybeUninit<u8>]) };
	253	self.read_raw(out)
	254	}
	255
	256	/// Reads a value of the specified type.
	257	///
	258	/// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
	259	/// bounds of this [`UserSliceReader`].
	260	pub fn read<T: FromBytes>(&mut self) -> Result<T> {
	261	let len = size_of::<T>();
	262	if len > self.length {
	263	return Err(EFAULT);
	264	}
	265	let Ok(len_ulong) = c_ulong::try_from(len) else {
	266	return Err(EFAULT);
	267	};
	268	let mut out: MaybeUninit<T> = MaybeUninit::uninit();
	269	// SAFETY: The local variable `out` is valid for writing `size_of::<T>()` bytes.
	270	//
	271	// By using the _copy_from_user variant, we skip the check_object_size check that verifies
	272	// the kernel pointer. This mirrors the logic on the C side that skips the check when the
	273	// length is a compile-time constant.
	274	let res = unsafe {
	275	bindings::_copy_from_user(
	276	out.as_mut_ptr().cast::<c_void>(),
	277	self.ptr as *const c_void,
	278	len_ulong,
	279	)
	280	};
	281	if res != 0 {
	282	return Err(EFAULT);
	283	}
	284	self.ptr = self.ptr.wrapping_add(len);
	285	self.length -= len;
	286	// SAFETY: The read above has initialized all bytes in `out`, and since `T` implements
	287	// `FromBytes`, any bit-pattern is a valid value for this type.
	288	Ok(unsafe { out.assume_init() })
	289	}
	290
	291	/// Reads the entirety of the user slice, appending it to the end of the provided buffer.
	292	///
	293	/// Fails with [`EFAULT`] if the read happens on a bad address.
	294	pub fn read_all(mut self, buf: &mut Vec<u8>, flags: Flags) -> Result {
	295	let len = self.length;
	296	VecExt::<u8>::reserve(buf, len, flags)?;
	297
	298	// The call to `try_reserve` was successful, so the spare capacity is at least `len` bytes
	299	// long.
	300	self.read_raw(&mut buf.spare_capacity_mut()[..len])?;
	301
	302	// SAFETY: Since the call to `read_raw` was successful, so the next `len` bytes of the
	303	// vector have been initialized.
	304	unsafe { buf.set_len(buf.len() + len) };
	305	Ok(())
	306	}
	307	}
	308
	309	/// A writer for [`UserSlice`].
	310	///
	311	/// Used to incrementally write into the user slice.
	312	pub struct UserSliceWriter {
	313	ptr: UserPtr,
	314	length: usize,
	315	}
	316
	317	impl UserSliceWriter {
	318	/// Returns the amount of space remaining in this buffer.
	319	///
	320	/// Note that even writing less than this number of bytes may fail.
	321	pub fn len(&self) -> usize {
	322	self.length
	323	}
	324
	325	/// Returns `true` if no more data can be written to this buffer.
	326	pub fn is_empty(&self) -> bool {
	327	self.length == 0
	328	}
	329
	330	/// Writes raw data to this user pointer from a kernel buffer.
	331	///
	332	/// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
	333	/// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
	334	/// if it returns an error.
	335	pub fn write_slice(&mut self, data: &[u8]) -> Result {
	336	let len = data.len();
	337	let data_ptr = data.as_ptr().cast::<c_void>();
	338	if len > self.length {
	339	return Err(EFAULT);
	340	}
	341	let Ok(len_ulong) = c_ulong::try_from(len) else {
	342	return Err(EFAULT);
	343	};
	344	// SAFETY: `data_ptr` points into an immutable slice of length `len_ulong`, so we may read
	345	// that many bytes from it.
	346	let res = unsafe { bindings::copy_to_user(self.ptr as *mut c_void, data_ptr, len_ulong) };
	347	if res != 0 {
	348	return Err(EFAULT);
	349	}
	350	self.ptr = self.ptr.wrapping_add(len);
	351	self.length -= len;
	352	Ok(())
	353	}
	354
	355	/// Writes the provided Rust value to this userspace pointer.
	356	///
	357	/// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
	358	/// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
	359	/// if it returns an error.
	360	pub fn write<T: AsBytes>(&mut self, value: &T) -> Result {
	361	let len = size_of::<T>();
	362	if len > self.length {
	363	return Err(EFAULT);
	364	}
	365	let Ok(len_ulong) = c_ulong::try_from(len) else {
	366	return Err(EFAULT);
	367	};
	368	// SAFETY: The reference points to a value of type `T`, so it is valid for reading
	369	// `size_of::<T>()` bytes.
	370	//
	371	// By using the _copy_to_user variant, we skip the check_object_size check that verifies the
	372	// kernel pointer. This mirrors the logic on the C side that skips the check when the length
	373	// is a compile-time constant.
	374	let res = unsafe {
	375	bindings::_copy_to_user(
	376	self.ptr as *mut c_void,
	377	(value as *const T).cast::<c_void>(),
	378	len_ulong,
	379	)
	380	};
	381	if res != 0 {
	382	return Err(EFAULT);
	383	}
	384	self.ptr = self.ptr.wrapping_add(len);
	385	self.length -= len;
	386	Ok(())
	387	}
	388	}

rust/kernel/workqueue.rs+9−7

 /// use kernel::sync::Arc;
 /// use kernel::workqueue::{self, impl_has_work, Work};
 ///
-/// struct MyStruct {
+/// struct MyStruct<'a, T, const N: usize> {
-///     work_field: Work<MyStruct, 17>,
+///     work_field: Work<MyStruct<'a, T, N>, 17>,
+///     f: fn(&'a [T; N]),
 /// }
 ///
 /// impl_has_work! {
-///     impl HasWork<MyStruct, 17> for MyStruct { self.work_field }
+///     impl{'a, T, const N: usize} HasWork<MyStruct<'a, T, N>, 17>
+///     for MyStruct<'a, T, N> { self.work_field }
 /// }
 /// ```
 #[macro_export]
 macro_rules! impl_has_work {
-    ($(impl$(<$($implarg:ident),*>)?
+    ($(impl$({$($generics:tt)*})?
        HasWork<$work_type:ty $(, $id:tt)?>
-       for $self:ident $(<$($selfarg:ident),*>)?
+       for $self:ty
        { self.$field:ident }
     )*) => {$(
         // SAFETY: The implementation of `raw_get_work` only compiles if the field has the right
         // type.
-        unsafe impl$(<$($implarg),*>)? $crate::workqueue::HasWork<$work_type $(, $id)?> for $self $(<$($selfarg),*>)? {
+        unsafe impl$(<$($generics)+>)? $crate::workqueue::HasWork<$work_type $(, $id)?> for $self {
             const OFFSET: usize = ::core::mem::offset_of!(Self, $field) as usize;
             #[inline]
 pub use impl_has_work;
 impl_has_work! {
-    impl<T> HasWork<Self> for ClosureWork<T> { self.work }
+    impl{T} HasWork<Self> for ClosureWork<T> { self.work }
 }
 unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Arc<T>