lael-labs/linux

Documentation/process/changes.rst+1−1

@@ -31,7 +31,7 @@ you probably needn't concern yourself with pcmciautils. · 30 unmodified lines
31	====================== =============== ========================================	31	====================== =============== ========================================
32	GNU C 5.1 gcc --version	32	GNU C 5.1 gcc --version
33	Clang/LLVM (optional) 11.0.0 clang --version	33	Clang/LLVM (optional) 11.0.0 clang --version
34	Rust (optional) 1.74.1 rustc --version	34	Rust (optional) 1.76.0 rustc --version
35	bindgen (optional) 0.65.1 bindgen --version	35	bindgen (optional) 0.65.1 bindgen --version
36	GNU make 3.82 make --version	36	GNU make 3.82 make --version
37	bash 4.2 bash --version	37	bash 4.2 bash --version

Documentation/rust/general-information.rst+0−24

@@ -77,27 +77,3 @@ configuration: · 76 unmodified lines
77	#[cfg(CONFIG_X="y")] // Enabled as a built-in (`y`)	77	#[cfg(CONFIG_X="y")] // Enabled as a built-in (`y`)
78	#[cfg(CONFIG_X="m")] // Enabled as a module (`m`)	78	#[cfg(CONFIG_X="m")] // Enabled as a module (`m`)
79	#[cfg(not(CONFIG_X))] // Disabled	79	#[cfg(not(CONFIG_X))] // Disabled
80
81
82	Testing
83	-------
84
85	There are the tests that come from the examples in the Rust documentation
86	and get transformed into KUnit tests. These can be run via KUnit. For example
87	via ``kunit_tool`` (``kunit.py``) on the command line::
88
89	./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y
90
91	Alternatively, KUnit can run them as kernel built-in at boot. Refer to
92	Documentation/dev-tools/kunit/index.rst for the general KUnit documentation
93	and Documentation/dev-tools/kunit/architecture.rst for the details of kernel
94	built-in vs. command line testing.
95
96	Additionally, there are the ``#[test]`` tests. These can be run using
97	the ``rusttest`` Make target::
98
99	make LLVM=1 rusttest
100
101	This requires the kernel ``.config`` and downloads external repositories.
102	It runs the ``#[test]`` tests on the host (currently) and thus is fairly
103	limited in what these tests can test.

Documentation/rust/index.rst+1−0

@@ -40,6 +40,7 @@ configurations. · 39 unmodified lines
40	general-information	40	general-information
41	coding-guidelines	41	coding-guidelines
42	arch-support	42	arch-support
		43	testing
43		44
44	.. only:: subproject and html	45	.. only:: subproject and html
45		46

Documentation/rust/testing.rst+135−0

@@ -0,0 +1,135 @@
	1	.. SPDX-License-Identifier: GPL-2.0
	2
	3	Testing
	4	=======
	5
	6	This document contains useful information how to test the Rust code in the
	7	kernel.
	8
	9	There are two sorts of tests:
	10
	11	- The KUnit tests.
	12	- The ``#[test]`` tests.
	13
	14	The KUnit tests
	15	---------------
	16
	17	These are the tests that come from the examples in the Rust documentation. They
	18	get transformed into KUnit tests.
	19
	20	Usage
	21	*****
	22
	23	These tests can be run via KUnit. For example via ``kunit_tool`` (``kunit.py``)
	24	on the command line::
	25
	26	./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y
	27
	28	Alternatively, KUnit can run them as kernel built-in at boot. Refer to
	29	Documentation/dev-tools/kunit/index.rst for the general KUnit documentation
	30	and Documentation/dev-tools/kunit/architecture.rst for the details of kernel
	31	built-in vs. command line testing.
	32
	33	To use these KUnit doctests, the following must be enabled::
	34
	35	CONFIG_KUNIT
	36	Kernel hacking -> Kernel Testing and Coverage -> KUnit - Enable support for unit tests
	37	CONFIG_RUST_KERNEL_DOCTESTS
	38	Kernel hacking -> Rust hacking -> Doctests for the `kernel` crate
	39
	40	in the kernel config system.
	41
	42	KUnit tests are documentation tests
	43	***********************************
	44
	45	These documentation tests are typically examples of usage of any item (e.g.
	46	function, struct, module...).
	47
	48	They are very convenient because they are just written alongside the
	49	documentation. For instance:
	50
	51	.. code-block:: rust
	52
	53	/// Sums two numbers.
	54	///
	55	/// ```
	56	/// assert_eq!(mymod::f(10, 20), 30);
	57	/// ```
	58	pub fn f(a: i32, b: i32) -> i32 {
	59	a + b
	60	}
	61
	62	In userspace, the tests are collected and run via ``rustdoc``. Using the tool
	63	as-is would be useful already, since it allows verifying that examples compile
	64	(thus enforcing they are kept in sync with the code they document) and as well
	65	as running those that do not depend on in-kernel APIs.
	66
	67	For the kernel, however, these tests get transformed into KUnit test suites.
	68	This means that doctests get compiled as Rust kernel objects, allowing them to
	69	run against a built kernel.
	70
	71	A benefit of this KUnit integration is that Rust doctests get to reuse existing
	72	testing facilities. For instance, the kernel log would look like::
	73
	74	KTAP version 1
	75	1..1
	76	KTAP version 1
	77	# Subtest: rust_doctests_kernel
	78	1..59
	79	# rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
	80	ok 1 rust_doctest_kernel_build_assert_rs_0
	81	# rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
	82	ok 2 rust_doctest_kernel_build_assert_rs_1
	83	# rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
	84	ok 3 rust_doctest_kernel_init_rs_0
	85	...
	86	# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
	87	ok 59 rust_doctest_kernel_types_rs_2
	88	# rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
	89	# Totals: pass:59 fail:0 skip:0 total:59
	90	ok 1 rust_doctests_kernel
	91
	92	Tests using the `? <https://doc.rust-lang.org/reference/expressions/operator-expr.html#the-question-mark-operator>`_
	93	operator are also supported as usual, e.g.:
	94
	95	.. code-block:: rust
	96
	97	/// ```
	98	/// # use kernel::{spawn_work_item, workqueue};
	99	/// spawn_work_item!(workqueue::system(), \|\| pr_info!("x"))?;
	100	/// # Ok::<(), Error>(())
	101	/// ```
	102
	103	The tests are also compiled with Clippy under ``CLIPPY=1``, just like normal
	104	code, thus also benefitting from extra linting.
	105
	106	In order for developers to easily see which line of doctest code caused a
	107	failure, a KTAP diagnostic line is printed to the log. This contains the
	108	location (file and line) of the original test (i.e. instead of the location in
	109	the generated Rust file)::
	110
	111	# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
	112
	113	Rust tests appear to assert using the usual ``assert!`` and ``assert_eq!``
	114	macros from the Rust standard library (``core``). We provide a custom version
	115	that forwards the call to KUnit instead. Importantly, these macros do not
	116	require passing context, unlike those for KUnit testing (i.e.
	117	``struct kunit *``). This makes them easier to use, and readers of the
	118	documentation do not need to care about which testing framework is used. In
	119	addition, it may allow us to test third-party code more easily in the future.
	120
	121	A current limitation is that KUnit does not support assertions in other tasks.
	122	Thus, we presently simply print an error to the kernel log if an assertion
	123	actually failed. Additionally, doctests are not run for nonpublic functions.
	124
	125	The ``#[test]`` tests
	126	---------------------
	127
	128	Additionally, there are the ``#[test]`` tests. These can be run using the
	129	``rusttest`` Make target::
	130
	131	make LLVM=1 rusttest
	132
	133	This requires the kernel ``.config`` and downloads external repositories. It
	134	runs the ``#[test]`` tests on the host (currently) and thus is fairly limited in
	135	what these tests can test.

Makefile+2−2

@@ -1201,7 +1201,7 @@ prepare0: archprepare · 1200 unmodified lines
1201	# All the preparing..	1201	# All the preparing..
1202	prepare: prepare0	1202	prepare: prepare0
1203	ifdef CONFIG_RUST	1203	ifdef CONFIG_RUST
1204	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh	1204	+$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh
1205	$(Q)$(MAKE) $(build)=rust	1205	$(Q)$(MAKE) $(build)=rust
1206	endif	1206	endif
1207		1207
@@ -1711,7 +1711,7 @@ $(DOC_TARGETS): · 503 unmodified lines
1711	# "Is Rust available?" target	1711	# "Is Rust available?" target
1712	PHONY += rustavailable	1712	PHONY += rustavailable
1713	rustavailable:	1713	rustavailable:
1714	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh && echo "Rust is available!"	1714	+$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh && echo "Rust is available!"
1715		1715
1716	# Documentation target	1716	# Documentation target
1717	#	1717	#

rust/Makefile+24−24

rust/alloc/alloc.rs+11−1

         panic!("allocation failed");
     }
+    #[inline]
     fn rt_error(layout: Layout) -> ! {
         unsafe {
             __rust_alloc_error_handler(layout.size(), layout.align());
         }
     }
-    unsafe { core::intrinsics::const_eval_select((layout,), ct_error, rt_error) }
+    #[cfg(not(feature = "panic_immediate_abort"))]
+    unsafe {
+        core::intrinsics::const_eval_select((layout,), ct_error, rt_error)
+    }
+    #[cfg(feature = "panic_immediate_abort")]
+    ct_error(layout)
 }
 // For alloc test `std::alloc::handle_alloc_error` can be used directly.
     }
 }
+#[cfg(not(no_global_oom_handling))]
 /// Specialize clones into pre-allocated, uninitialized memory.
 /// Used by `Box::clone` and `Rc`/`Arc::make_mut`.
 pub(crate) trait WriteCloneIntoRaw: Sized {
     unsafe fn write_clone_into_raw(&self, target: *mut Self);
 }
+#[cfg(not(no_global_oom_handling))]
 impl<T: Clone> WriteCloneIntoRaw for T {
     #[inline]
     default unsafe fn write_clone_into_raw(&self, target: *mut Self) {
     }
 }
+#[cfg(not(no_global_oom_handling))]
 impl<T: Copy> WriteCloneIntoRaw for T {
     #[inline]
     unsafe fn write_clone_into_raw(&self, target: *mut Self) {

rust/alloc/boxed.rs+23−11

 use core::marker::Unsize;
 use core::mem::{self, SizedTypeProperties};
 use core::ops::{
-    CoerceUnsized, Deref, DerefMut, DispatchFromDyn, Generator, GeneratorState, Receiver,
+    CoerceUnsized, Coroutine, CoroutineState, Deref, DerefMut, DispatchFromDyn, Receiver,
 };
 use core::pin::Pin;
 use core::ptr::{self, NonNull, Unique};
     /// ```
     /// let five = Box::new(5);
     /// ```
-    #[cfg(all(not(no_global_oom_handling)))]
+    #[cfg(not(no_global_oom_handling))]
     #[inline(always)]
     #[stable(feature = "rust1", since = "1.0.0")]
     #[must_use]
     /// use std::ptr;
     ///
     /// let x = Box::new(String::from("Hello"));
-    /// let p = Box::into_raw(x);
+    /// let ptr = Box::into_raw(x);
     /// unsafe {
-    ///     ptr::drop_in_place(p);
+    ///     ptr::drop_in_place(ptr);
-    ///     dealloc(p as *mut u8, Layout::new::<String>());
+    ///     dealloc(ptr as *mut u8, Layout::new::<String>());
+    /// }
+    /// ```
+    /// Note: This is equivalent to the following:
+    /// ```
+    /// let x = Box::new(String::from("Hello"));
+    /// let ptr = Box::into_raw(x);
+    /// unsafe {
+    ///     drop(Box::from_raw(ptr));
     /// }
     /// ```
     ///
 #[stable(feature = "pin", since = "1.33.0")]
 impl<T: ?Sized, A: Allocator> Unpin for Box<T, A> where A: 'static {}
-#[unstable(feature = "generator_trait", issue = "43122")]
+#[unstable(feature = "coroutine_trait", issue = "43122")]
-impl<G: ?Sized + Generator<R> + Unpin, R, A: Allocator> Generator<R> for Box<G, A>
+impl<G: ?Sized + Coroutine<R> + Unpin, R, A: Allocator> Coroutine<R> for Box<G, A>
 where
     A: 'static,
 {
     type Yield = G::Yield;
     type Return = G::Return;
-    fn resume(mut self: Pin<&mut Self>, arg: R) -> GeneratorState<Self::Yield, Self::Return> {
+    fn resume(mut self: Pin<&mut Self>, arg: R) -> CoroutineState<Self::Yield, Self::Return> {
         G::resume(Pin::new(&mut *self), arg)
     }
 }
-#[unstable(feature = "generator_trait", issue = "43122")]
+#[unstable(feature = "coroutine_trait", issue = "43122")]
-impl<G: ?Sized + Generator<R>, R, A: Allocator> Generator<R> for Pin<Box<G, A>>
+impl<G: ?Sized + Coroutine<R>, R, A: Allocator> Coroutine<R> for Pin<Box<G, A>>
 where
     A: 'static,
 {
     type Yield = G::Yield;
     type Return = G::Return;
-    fn resume(mut self: Pin<&mut Self>, arg: R) -> GeneratorState<Self::Yield, Self::Return> {
+    fn resume(mut self: Pin<&mut Self>, arg: R) -> CoroutineState<Self::Yield, Self::Return> {
         G::resume((*self).as_mut(), arg)
     }
 }
     fn source(&self) -> Option<&(dyn core::error::Error + 'static)> {
         core::error::Error::source(&**self)
     }
+    fn provide<'b>(&'b self, request: &mut core::error::Request<'b>) {
+        core::error::Error::provide(&**self, request);
+    }
 }

rust/alloc/collections/mod.rs+1−0

@@ -150,6 +150,7 @@ impl Display for TryReserveError { · 149 unmodified lines
150		150
151	/// An intermediate trait for specialization of `Extend`.	151	/// An intermediate trait for specialization of `Extend`.
152	#[doc(hidden)]	152	#[doc(hidden)]
		153	#[cfg(not(no_global_oom_handling))]
153	trait SpecExtend<I: IntoIterator> {	154	trait SpecExtend<I: IntoIterator> {
154	/// Extends `self` with the contents of the given iterator.	155	/// Extends `self` with the contents of the given iterator.
155	fn spec_extend(&mut self, iter: I);	156	fn spec_extend(&mut self, iter: I);

rust/alloc/lib.rs+5−4

     not(no_sync),
     target_has_atomic = "ptr"
 ))]
+#![doc(rust_logo)]
+#![feature(rustdoc_internals)]
 #![no_std]
 #![needs_allocator]
 // Lints:
 #![feature(const_eval_select)]
 #![feature(const_maybe_uninit_as_mut_ptr)]
 #![feature(const_maybe_uninit_write)]
-#![feature(const_maybe_uninit_zeroed)]
 #![feature(const_pin)]
 #![feature(const_refs_to_cell)]
 #![feature(const_size_of_val)]
 #![feature(maybe_uninit_uninit_array)]
 #![feature(maybe_uninit_uninit_array_transpose)]
 #![feature(pattern)]
-#![feature(pointer_byte_offsets)]
 #![feature(ptr_internals)]
 #![feature(ptr_metadata)]
 #![feature(ptr_sub_ptr)]
 #![feature(std_internals)]
 #![feature(str_internals)]
 #![feature(strict_provenance)]
+#![feature(trusted_fused)]
 #![feature(trusted_len)]
 #![feature(trusted_random_access)]
 #![feature(try_trait_v2)]
 //
 // Language features:
 // tidy-alphabetical-start
-#![cfg_attr(not(test), feature(generator_trait))]
+#![cfg_attr(not(test), feature(coroutine_trait))]
 #![cfg_attr(test, feature(panic_update_hook))]
 #![cfg_attr(test, feature(test))]
 #![feature(allocator_internals)]
     /// seed not being the same for every RNG invocation too.
     pub(crate) fn test_rng() -> rand_xorshift::XorShiftRng {
         use std::hash::{BuildHasher, Hash, Hasher};
-        let mut hasher = std::collections::hash_map::RandomState::new().build_hasher();
+        let mut hasher = std::hash::RandomState::new().build_hasher();
         std::panic::Location::caller().hash(&mut hasher);
         let hc64 = hasher.finish();
         let seed_vec =

rust/alloc/raw_vec.rs+57−20

     Zeroed,
 }
+#[repr(transparent)]
+#[cfg_attr(target_pointer_width = "16", rustc_layout_scalar_valid_range_end(0x7fff))]
+#[cfg_attr(target_pointer_width = "32", rustc_layout_scalar_valid_range_end(0x7fff_ffff))]
+#[cfg_attr(target_pointer_width = "64", rustc_layout_scalar_valid_range_end(0x7fff_ffff_ffff_ffff))]
+struct Cap(usize);
+impl Cap {
+    const ZERO: Cap = unsafe { Cap(0) };
+}
 /// A low-level utility for more ergonomically allocating, reallocating, and deallocating
 /// a buffer of memory on the heap without having to worry about all the corner cases
 /// involved. This type is excellent for building your own data structures like Vec and VecDeque.
 #[allow(missing_debug_implementations)]
 pub(crate) struct RawVec<T, A: Allocator = Global> {
     ptr: Unique<T>,
-    cap: usize,
+    /// Never used for ZSTs; it's `capacity()`'s responsibility to return usize::MAX in that case.
+    ///
+    /// # Safety
+    ///
+    /// `cap` must be in the `0..=isize::MAX` range.
+    cap: Cap,
     alloc: A,
 }
     /// the returned `RawVec`.
     pub const fn new_in(alloc: A) -> Self {
         // `cap: 0` means "unallocated". zero-sized types are ignored.
-        Self { ptr: Unique::dangling(), cap: 0, alloc }
+        Self { ptr: Unique::dangling(), cap: Cap::ZERO, alloc }
     }
     /// Like `with_capacity`, but parameterized over the choice of
             // here should change to `ptr.len() / mem::size_of::<T>()`.
             Self {
                 ptr: unsafe { Unique::new_unchecked(ptr.cast().as_ptr()) },
-                cap: capacity,
+                cap: unsafe { Cap(capacity) },
                 alloc,
             }
         }
         // here should change to `ptr.len() / mem::size_of::<T>()`.
         Ok(Self {
             ptr: unsafe { Unique::new_unchecked(ptr.cast().as_ptr()) },
-            cap: capacity,
+            cap: unsafe { Cap(capacity) },
             alloc,
         })
     }
     /// The `ptr` must be allocated (via the given allocator `alloc`), and with the given
     /// `capacity`.
     /// The `capacity` cannot exceed `isize::MAX` for sized types. (only a concern on 32-bit
-    /// systems). ZST vectors may have a capacity up to `usize::MAX`.
+    /// systems). For ZSTs capacity is ignored.
     /// If the `ptr` and `capacity` come from a `RawVec` created via `alloc`, then this is
     /// guaranteed.
     #[inline]
     pub unsafe fn from_raw_parts_in(ptr: *mut T, capacity: usize, alloc: A) -> Self {
-        Self { ptr: unsafe { Unique::new_unchecked(ptr) }, cap: capacity, alloc }
+        let cap = if T::IS_ZST { Cap::ZERO } else { unsafe { Cap(capacity) } };
+        Self { ptr: unsafe { Unique::new_unchecked(ptr) }, cap, alloc }
     }
     /// Gets a raw pointer to the start of the allocation. Note that this is
     /// This will always be `usize::MAX` if `T` is zero-sized.
     #[inline(always)]
     pub fn capacity(&self) -> usize {
-        if T::IS_ZST { usize::MAX } else { self.cap }
+        if T::IS_ZST { usize::MAX } else { self.cap.0 }
     }
     /// Returns a shared reference to the allocator backing this `RawVec`.
     }
     fn current_memory(&self) -> Option<(NonNull<u8>, Layout)> {
-        if T::IS_ZST || self.cap == 0 {
+        if T::IS_ZST || self.cap.0 == 0 {
             None
         } else {
             // We could use Layout::array here which ensures the absence of isize and usize overflows
             let _: () = const { assert!(mem::size_of::<T>() % mem::align_of::<T>() == 0) };
             unsafe {
                 let align = mem::align_of::<T>();
-                let size = mem::size_of::<T>().unchecked_mul(self.cap);
+                let size = mem::size_of::<T>().unchecked_mul(self.cap.0);
                 let layout = Layout::from_size_align_unchecked(size, align);
                 Some((self.ptr.cast().into(), layout))
             }
     /// The same as `reserve`, but returns on errors instead of panicking or aborting.
     pub fn try_reserve(&mut self, len: usize, additional: usize) -> Result<(), TryReserveError> {
         if self.needs_to_grow(len, additional) {
-            self.grow_amortized(len, additional)
+            self.grow_amortized(len, additional)?;
-        } else {
-            Ok(())
         }
+        unsafe {
+            // Inform the optimizer that the reservation has succeeded or wasn't needed
+            core::intrinsics::assume(!self.needs_to_grow(len, additional));
+        }
+        Ok(())
     }
     /// The same as `reserve_for_push`, but returns on errors instead of panicking or aborting.
         len: usize,
         additional: usize,
     ) -> Result<(), TryReserveError> {
-        if self.needs_to_grow(len, additional) { self.grow_exact(len, additional) } else { Ok(()) }
+        if self.needs_to_grow(len, additional) {
+            self.grow_exact(len, additional)?;
+        }
+        unsafe {
+            // Inform the optimizer that the reservation has succeeded or wasn't needed
+            core::intrinsics::assume(!self.needs_to_grow(len, additional));
+        }
+        Ok(())
     }
     /// Shrinks the buffer down to the specified capacity. If the given amount
         additional > self.capacity().wrapping_sub(len)
     }
-    fn set_ptr_and_cap(&mut self, ptr: NonNull<[u8]>, cap: usize) {
+    /// # Safety:
+    ///
+    /// `cap` must not exceed `isize::MAX`.
+    unsafe fn set_ptr_and_cap(&mut self, ptr: NonNull<[u8]>, cap: usize) {
         // Allocators currently return a `NonNull<[u8]>` whose length matches
         // the size requested. If that ever changes, the capacity here should
         // change to `ptr.len() / mem::size_of::<T>()`.
         self.ptr = unsafe { Unique::new_unchecked(ptr.cast().as_ptr()) };
-        self.cap = cap;
+        self.cap = unsafe { Cap(cap) };
     }
     // This method is usually instantiated many times. So we want it to be as
         // This guarantees exponential growth. The doubling cannot overflow
         // because `cap <= isize::MAX` and the type of `cap` is `usize`.
-        let cap = cmp::max(self.cap * 2, required_cap);
+        let cap = cmp::max(self.cap.0 * 2, required_cap);
         let cap = cmp::max(Self::MIN_NON_ZERO_CAP, cap);
         let new_layout = Layout::array::<T>(cap);
         // `finish_grow` is non-generic over `T`.
         let ptr = finish_grow(new_layout, self.current_memory(), &mut self.alloc)?;
-        self.set_ptr_and_cap(ptr, cap);
+        // SAFETY: finish_grow would have resulted in a capacity overflow if we tried to allocate more than isize::MAX items
+        unsafe { self.set_ptr_and_cap(ptr, cap) };
         Ok(())
     }
         // `finish_grow` is non-generic over `T`.
         let ptr = finish_grow(new_layout, self.current_memory(), &mut self.alloc)?;
-        self.set_ptr_and_cap(ptr, cap);
+        // SAFETY: finish_grow would have resulted in a capacity overflow if we tried to allocate more than isize::MAX items
+        unsafe {
+            self.set_ptr_and_cap(ptr, cap);
+        }
         Ok(())
     }
         if cap == 0 {
             unsafe { self.alloc.deallocate(ptr, layout) };
             self.ptr = Unique::dangling();
-            self.cap = 0;
+            self.cap = Cap::ZERO;
         } else {
             let ptr = unsafe {
                 // `Layout::array` cannot overflow here because it would have
                     .shrink(ptr, layout, new_layout)
                     .map_err(|_| AllocError { layout: new_layout, non_exhaustive: () })?
             };
-            self.set_ptr_and_cap(ptr, cap);
+            // SAFETY: if the allocation is valid, then the capacity is too
+            unsafe {
+                self.set_ptr_and_cap(ptr, cap);
+            }
         }
         Ok(())
     }
 // ensure that the code generation related to these panics is minimal as there's
 // only one location which panics rather than a bunch throughout the module.
 #[cfg(not(no_global_oom_handling))]
+#[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
 fn capacity_overflow() -> ! {
     panic!("capacity overflow");
 }

rust/alloc/vec/into_iter.rs+11−5

 use core::array;
 use core::fmt;
 use core::iter::{
-    FusedIterator, InPlaceIterable, SourceIter, TrustedLen, TrustedRandomAccessNoCoerce,
+    FusedIterator, InPlaceIterable, SourceIter, TrustedFused, TrustedLen,
+    TrustedRandomAccessNoCoerce,
 };
 use core::marker::PhantomData;
 use core::mem::{self, ManuallyDrop, MaybeUninit, SizedTypeProperties};
         // Also note the implementation of `Self: TrustedRandomAccess` requires
         // that `T: Copy` so reading elements from the buffer doesn't invalidate
         // them for `Drop`.
-        unsafe {
+        unsafe { if T::IS_ZST { mem::zeroed() } else { ptr::read(self.ptr.add(i)) } }
-            if T::IS_ZST { mem::zeroed() } else { ptr::read(self.ptr.add(i)) }
-        }
     }
 }
 #[stable(feature = "fused", since = "1.26.0")]
 impl<T, A: Allocator> FusedIterator for IntoIter<T, A> {}
+#[doc(hidden)]
+#[unstable(issue = "none", feature = "trusted_fused")]
+unsafe impl<T, A: Allocator> TrustedFused for IntoIter<T, A> {}
 #[unstable(feature = "trusted_len", issue = "37572")]
 unsafe impl<T, A: Allocator> TrustedLen for IntoIter<T, A> {}
 // also refer to the vec::in_place_collect module documentation to get an overview
 #[unstable(issue = "none", feature = "inplace_iteration")]
 #[doc(hidden)]
-unsafe impl<T, A: Allocator> InPlaceIterable for IntoIter<T, A> {}
+unsafe impl<T, A: Allocator> InPlaceIterable for IntoIter<T, A> {
+    const EXPAND_BY: Option<NonZeroUsize> = NonZeroUsize::new(1);
+    const MERGE_BY: Option<NonZeroUsize> = NonZeroUsize::new(1);
+}
 #[unstable(issue = "none", feature = "inplace_iteration")]
 #[doc(hidden)]

rust/alloc/vec/mod.rs+61−20

 #[cfg(not(no_global_oom_handling))]
 use self::is_zero::IsZero;
+#[cfg(not(no_global_oom_handling))]
 mod is_zero;
 #[cfg(not(no_global_oom_handling))]
 mod set_len_on_drop;
 #[cfg(not(no_global_oom_handling))]
-use self::in_place_drop::{InPlaceDrop, InPlaceDstBufDrop};
+use self::in_place_drop::{InPlaceDrop, InPlaceDstDataSrcBufDrop};
 #[cfg(not(no_global_oom_handling))]
 mod in_place_drop;
     /// [`as_mut_ptr`]: Vec::as_mut_ptr
     /// [`as_ptr`]: Vec::as_ptr
     #[stable(feature = "vec_as_ptr", since = "1.37.0")]
-    #[cfg_attr(not(bootstrap), rustc_never_returns_null_ptr)]
+    #[rustc_never_returns_null_ptr]
     #[inline]
     pub fn as_ptr(&self) -> *const T {
         // We shadow the slice method of the same name to avoid going through
     /// [`as_mut_ptr`]: Vec::as_mut_ptr
     /// [`as_ptr`]: Vec::as_ptr
     #[stable(feature = "vec_as_ptr", since = "1.37.0")]
-    #[cfg_attr(not(bootstrap), rustc_never_returns_null_ptr)]
+    #[rustc_never_returns_null_ptr]
     #[inline]
     pub fn as_mut_ptr(&mut self) -> *mut T {
         // We shadow the slice method of the same name to avoid going through
     #[stable(feature = "rust1", since = "1.0.0")]
     pub fn swap_remove(&mut self, index: usize) -> T {
         #[cold]
-        #[inline(never)]
+        #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
+        #[track_caller]
         fn assert_failed(index: usize, len: usize) -> ! {
             panic!("swap_remove index (is {index}) should be < len (is {len})");
         }
     #[stable(feature = "rust1", since = "1.0.0")]
     pub fn insert(&mut self, index: usize, element: T) {
         #[cold]
-        #[inline(never)]
+        #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
+        #[track_caller]
         fn assert_failed(index: usize, len: usize) -> ! {
             panic!("insertion index (is {index}) should be <= len (is {len})");
         }
     #[track_caller]
     pub fn remove(&mut self, index: usize) -> T {
         #[cold]
-        #[inline(never)]
+        #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
         #[track_caller]
         fn assert_failed(index: usize, len: usize) -> ! {
             panic!("removal index (is {index}) should be < len (is {len})");
             return;
         }
-        /* INVARIANT: vec.len() > read >= write > write-1 >= 0 */
+        // Check if we ever want to remove anything.
+        // This allows to use copy_non_overlapping in next cycle.
+        // And avoids any memory writes if we don't need to remove anything.
+        let mut first_duplicate_idx: usize = 1;
+        let start = self.as_mut_ptr();
+        while first_duplicate_idx != len {
+            let found_duplicate = unsafe {
+                // SAFETY: first_duplicate always in range [1..len)
+                // Note that we start iteration from 1 so we never overflow.
+                let prev = start.add(first_duplicate_idx.wrapping_sub(1));
+                let current = start.add(first_duplicate_idx);
+                // We explicitly say in docs that references are reversed.
+                same_bucket(&mut *current, &mut *prev)
+            };
+            if found_duplicate {
+                break;
+            }
+            first_duplicate_idx += 1;
+        }
+        // Don't need to remove anything.
+        // We cannot get bigger than len.
+        if first_duplicate_idx == len {
+            return;
+        }
+        /* INVARIANT: vec.len() > read > write > write-1 >= 0 */
         struct FillGapOnDrop<'a, T, A: core::alloc::Allocator> {
             /* Offset of the element we want to check if it is duplicate */
             read: usize,
             }
         }
-        let mut gap = FillGapOnDrop { read: 1, write: 1, vec: self };
-        let ptr = gap.vec.as_mut_ptr();
         /* Drop items while going through Vec, it should be more efficient than
          * doing slice partition_dedup + truncate */
+        // Construct gap first and then drop item to avoid memory corruption if `T::drop` panics.
+        let mut gap =
+            FillGapOnDrop { read: first_duplicate_idx + 1, write: first_duplicate_idx, vec: self };
+        unsafe {
+            // SAFETY: we checked that first_duplicate_idx in bounds before.
+            // If drop panics, `gap` would remove this item without drop.
+            ptr::drop_in_place(start.add(first_duplicate_idx));
+        }
         /* SAFETY: Because of the invariant, read_ptr, prev_ptr and write_ptr
          * are always in-bounds and read_ptr never aliases prev_ptr */
         unsafe {
             while gap.read < len {
-                let read_ptr = ptr.add(gap.read);
+                let read_ptr = start.add(gap.read);
-                let prev_ptr = ptr.add(gap.write.wrapping_sub(1));
+                let prev_ptr = start.add(gap.write.wrapping_sub(1));
-                if same_bucket(&mut *read_ptr, &mut *prev_ptr) {
+                // We explicitly say in docs that references are reversed.
+                let found_duplicate = same_bucket(&mut *read_ptr, &mut *prev_ptr);
+                if found_duplicate {
                     // Increase `gap.read` now since the drop may panic.
                     gap.read += 1;
                     /* We have found duplicate, drop it in-place */
                     ptr::drop_in_place(read_ptr);
                 } else {
-                    let write_ptr = ptr.add(gap.write);
+                    let write_ptr = start.add(gap.write);
-                    /* Because `read_ptr` can be equal to `write_ptr`, we either
+                    /* read_ptr cannot be equal to write_ptr because at this point
-                     * have to use `copy` or conditional `copy_nonoverlapping`.
+                     * we guaranteed to skip at least one element (before loop starts).
-                     * Looks like the first option is faster. */
+                     */
-                    ptr::copy(read_ptr, write_ptr, 1);
+                    ptr::copy_nonoverlapping(read_ptr, write_ptr, 1);
                     /* We have filled that place, so go further */
                     gap.write += 1;
         } else {
             unsafe {
                 self.len -= 1;
+                core::intrinsics::assume(self.len < self.capacity());
                 Some(ptr::read(self.as_ptr().add(self.len())))
             }
         }
         A: Clone,
     {
         #[cold]
-        #[inline(never)]
+        #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
+        #[track_caller]
         fn assert_failed(at: usize, len: usize) -> ! {
             panic!("`at` split index (is {at}) should be <= len (is {len})");
         }
     <T as SpecFromElem>::from_elem(elem, n, alloc)
 }
+#[cfg(not(no_global_oom_handling))]
 trait ExtendFromWithinSpec {
     /// # Safety
     ///
     unsafe fn spec_extend_from_within(&mut self, src: Range<usize>);
 }
+#[cfg(not(no_global_oom_handling))]
 impl<T: Clone, A: Allocator> ExtendFromWithinSpec for Vec<T, A> {
     default unsafe fn spec_extend_from_within(&mut self, src: Range<usize>) {
         // SAFETY:
     }
 }
+#[cfg(not(no_global_oom_handling))]
 impl<T: Copy, A: Allocator> ExtendFromWithinSpec for Vec<T, A> {
     unsafe fn spec_extend_from_within(&mut self, src: Range<usize>) {
         let count = src.len();
 /// ```
 /// use std::hash::BuildHasher;
 ///
-/// let b = std::collections::hash_map::RandomState::new();
+/// let b = std::hash::RandomState::new();
 /// let v: Vec<u8> = vec![0xa8, 0x3c, 0x09];
 /// let s: &[u8] = &[0xa8, 0x3c, 0x09];
 /// assert_eq!(b.hash_one(v), b.hash_one(s));

rust/bindings/bindings_helper.h+3−2

 #include <kunit/test.h>
 #include <linux/errname.h>
 #include <linux/ethtool.h>
+#include <linux/jiffies.h>
 #include <linux/mdio.h>
 #include <linux/phy.h>
-#include <linux/slab.h>
 #include <linux/refcount.h>
-#include <linux/wait.h>
 #include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
 #include <linux/workqueue.h>
 /* `bindgen` gets confused at certain things. */

rust/kernel/allocator.rs+1−1

@@ -35,7 +35,7 @@ unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: bindings::gf · 34 unmodified lines
35	// - `ptr` is either null or a pointer returned from a previous `k{re}alloc()` by the	35	// - `ptr` is either null or a pointer returned from a previous `k{re}alloc()` by the
36	// function safety requirement.	36	// function safety requirement.
37	// - `size` is greater than 0 since it's either a `layout.size()` (which cannot be zero	37	// - `size` is greater than 0 since it's either a `layout.size()` (which cannot be zero
38	// according to the function safety requirement) or a result from `next_power_of_two()`.	38	// according to the function safety requirement) or a result from `next_power_of_two()`.
39	unsafe { bindings::krealloc(ptr as const core::ffi::c_void, size, flags) as mut u8 }	39	unsafe { bindings::krealloc(ptr as const core::ffi::c_void, size, flags) as mut u8 }
40	}	40	}
41		41

rust/kernel/error.rs+3−7

 ///     pdev: &mut PlatformDevice,
 ///     index: u32,
 /// ) -> Result<*mut core::ffi::c_void> {
-///     // SAFETY: FFI call.
+///     // SAFETY: `pdev` points to a valid platform device. There are no safety requirements
-///     unsafe {
+///     // on `index`.
-///         from_err_ptr(bindings::devm_platform_ioremap_resource(
+///     from_err_ptr(unsafe { bindings::devm_platform_ioremap_resource(pdev.to_ptr(), index) })
-///             pdev.to_ptr(),
-///             index,
-///         ))
-///     }
 /// }
 /// ```
 // TODO: Remove `dead_code` marker once an in-kernel client is available.

rust/kernel/init.rs+11−11

@@ -36,7 +36,7 @@ · 35 unmodified lines
36	//!	36	//!
37	//! ```rust	37	//! ```rust
38	//! # #![allow(clippy::disallowed_names)]	38	//! # #![allow(clippy::disallowed_names)]
39	//! use kernel::{prelude::*, sync::Mutex, new_mutex};	39	//! use kernel::sync::{new_mutex, Mutex};
40	//! # use core::pin::Pin;	40	//! # use core::pin::Pin;
41	//! #[pin_data]	41	//! #[pin_data]
42	//! struct Foo {	42	//! struct Foo {
@@ -56,7 +56,7 @@ · 13 unmodified lines
56	//!	56	//!
57	//! ```rust	57	//! ```rust
58	//! # #![allow(clippy::disallowed_names)]	58	//! # #![allow(clippy::disallowed_names)]
59	//! # use kernel::{prelude::*, sync::Mutex, new_mutex};	59	//! # use kernel::sync::{new_mutex, Mutex};
60	//! # use core::pin::Pin;	60	//! # use core::pin::Pin;
61	//! # #[pin_data]	61	//! # #[pin_data]
62	//! # struct Foo {	62	//! # struct Foo {
@@ -79,7 +79,7 @@ · 16 unmodified lines
79	//! above method only works for types where you can access the fields.	79	//! above method only works for types where you can access the fields.
80	//!	80	//!
81	//! ```rust	81	//! ```rust
82	//! # use kernel::{new_mutex, sync::{Arc, Mutex}};	82	//! # use kernel::sync::{new_mutex, Arc, Mutex};
83	//! let mtx: Result<Arc<Mutex<usize>>> = Arc::pin_init(new_mutex!(42, "example::mtx"));	83	//! let mtx: Result<Arc<Mutex<usize>>> = Arc::pin_init(new_mutex!(42, "example::mtx"));
84	//! ```	84	//! ```
85	//!	85	//!
@@ -751,10 +751,10 @@ macro_rules! try_init { · 665 unmodified lines
751	///	751	///
752	/// # Safety	752	/// # Safety
753	///	753	///
754	/// When implementing this type you will need to take great care. Also there are probably very few	754	/// When implementing this trait you will need to take great care. Also there are probably very few
755	/// cases where a manual implementation is necessary. Use [`pin_init_from_closure`] where possible.	755	/// cases where a manual implementation is necessary. Use [`pin_init_from_closure`] where possible.
756	///	756	///
757	/// The [`PinInit::__pinned_init`] function	757	/// The [`PinInit::__pinned_init`] function:
758	/// - returns `Ok(())` if it initialized every field of `slot`,	758	/// - returns `Ok(())` if it initialized every field of `slot`,
759	/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:	759	/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
760	/// - `slot` can be deallocated without UB occurring,	760	/// - `slot` can be deallocated without UB occurring,
@@ -861,10 +861,10 @@ where · 100 unmodified lines
861	///	861	///
862	/// # Safety	862	/// # Safety
863	///	863	///
864	/// When implementing this type you will need to take great care. Also there are probably very few	864	/// When implementing this trait you will need to take great care. Also there are probably very few
865	/// cases where a manual implementation is necessary. Use [`init_from_closure`] where possible.	865	/// cases where a manual implementation is necessary. Use [`init_from_closure`] where possible.
866	///	866	///
867	/// The [`Init::__init`] function	867	/// The [`Init::__init`] function:
868	/// - returns `Ok(())` if it initialized every field of `slot`,	868	/// - returns `Ok(())` if it initialized every field of `slot`,
869	/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:	869	/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
870	/// - `slot` can be deallocated without UB occurring,	870	/// - `slot` can be deallocated without UB occurring,
@@ -1013,7 +1013,7 @@ pub fn uninit<T, E>() -> impl Init<MaybeUninit<T>, E> { · 142 unmodified lines
1013	///	1013	///
1014	/// ```rust	1014	/// ```rust
1015	/// use kernel::{error::Error, init::init_array_from_fn};	1015	/// use kernel::{error::Error, init::init_array_from_fn};
1016	/// let array: Box<[usize; 1_000]>= Box::init::<Error>(init_array_from_fn(\|i\| i)).unwrap();	1016	/// let array: Box<[usize; 1_000]> = Box::init::<Error>(init_array_from_fn(\|i\| i)).unwrap();
1017	/// assert_eq!(array.len(), 1_000);	1017	/// assert_eq!(array.len(), 1_000);
1018	/// ```	1018	/// ```
1019	pub fn init_array_from_fn<I, const N: usize, T, E>(	1019	pub fn init_array_from_fn<I, const N: usize, T, E>(
@@ -1027,7 +1027,7 @@ where · 7 unmodified lines
1027	// Counts the number of initialized elements and when dropped drops that many elements from	1027	// Counts the number of initialized elements and when dropped drops that many elements from
1028	// `slot`.	1028	// `slot`.
1029	let mut init_count = ScopeGuard::new_with_data(0, \|i\| {	1029	let mut init_count = ScopeGuard::new_with_data(0, \|i\| {
1030	// We now free every element that has been initialized before:	1030	// We now free every element that has been initialized before.
1031	// SAFETY: The loop initialized exactly the values from 0..i and since we	1031	// SAFETY: The loop initialized exactly the values from 0..i and since we
1032	// return `Err` below, the caller will consider the memory at `slot` as	1032	// return `Err` below, the caller will consider the memory at `slot` as
1033	// uninitialized.	1033	// uninitialized.
@@ -1056,7 +1056,7 @@ where · 22 unmodified lines
1056	///	1056	///
1057	/// ```rust	1057	/// ```rust
1058	/// use kernel::{sync::{Arc, Mutex}, init::pin_init_array_from_fn, new_mutex};	1058	/// use kernel::{sync::{Arc, Mutex}, init::pin_init_array_from_fn, new_mutex};
1059	/// let array: Arc<[Mutex<usize>; 1_000]>=	1059	/// let array: Arc<[Mutex<usize>; 1_000]> =
1060	/// Arc::pin_init(pin_init_array_from_fn(\|i\| new_mutex!(i))).unwrap();	1060	/// Arc::pin_init(pin_init_array_from_fn(\|i\| new_mutex!(i))).unwrap();
1061	/// assert_eq!(array.len(), 1_000);	1061	/// assert_eq!(array.len(), 1_000);
1062	/// ```	1062	/// ```
@@ -1071,7 +1071,7 @@ where · 8 unmodified lines
1071	// Counts the number of initialized elements and when dropped drops that many elements from	1071	// Counts the number of initialized elements and when dropped drops that many elements from
1072	// `slot`.	1072	// `slot`.
1073	let mut init_count = ScopeGuard::new_with_data(0, \|i\| {	1073	let mut init_count = ScopeGuard::new_with_data(0, \|i\| {
1074	// We now free every element that has been initialized before:	1074	// We now free every element that has been initialized before.
1075	// SAFETY: The loop initialized exactly the values from 0..i and since we	1075	// SAFETY: The loop initialized exactly the values from 0..i and since we
1076	// return `Err` below, the caller will consider the memory at `slot` as	1076	// return `Err` below, the caller will consider the memory at `slot` as
1077	// uninitialized.	1077	// uninitialized.

rust/kernel/ioctl.rs+3−3

@@ -1,6 +1,6 @@
1	// SPDX-License-Identifier: GPL-2.0	1	// SPDX-License-Identifier: GPL-2.0
2		2
3	//! ioctl() number definitions	3	//! `ioctl()` number definitions.
4	//!	4	//!
5	//! C header: [`include/asm-generic/ioctl.h`](srctree/include/asm-generic/ioctl.h)	5	//! C header: [`include/asm-generic/ioctl.h`](srctree/include/asm-generic/ioctl.h)
6		6
@@ -28,13 +28,13 @@ pub const fn _IO(ty: u32, nr: u32) -> u32 { · 21 unmodified lines
28	_IOC(uapi::_IOC_NONE, ty, nr, 0)	28	_IOC(uapi::_IOC_NONE, ty, nr, 0)
29	}	29	}
30		30
31	/// Build an ioctl number for an read-only ioctl.	31	/// Build an ioctl number for a read-only ioctl.
32	#[inline(always)]	32	#[inline(always)]
33	pub const fn _IOR<T>(ty: u32, nr: u32) -> u32 {	33	pub const fn _IOR<T>(ty: u32, nr: u32) -> u32 {
34	_IOC(uapi::_IOC_READ, ty, nr, core::mem::size_of::<T>())	34	_IOC(uapi::_IOC_READ, ty, nr, core::mem::size_of::<T>())
35	}	35	}
36		36
37	/// Build an ioctl number for an write-only ioctl.	37	/// Build an ioctl number for a write-only ioctl.
38	#[inline(always)]	38	#[inline(always)]
39	pub const fn _IOW<T>(ty: u32, nr: u32) -> u32 {	39	pub const fn _IOW<T>(ty: u32, nr: u32) -> u32 {
40	_IOC(uapi::_IOC_WRITE, ty, nr, core::mem::size_of::<T>())	40	_IOC(uapi::_IOC_WRITE, ty, nr, core::mem::size_of::<T>())

rust/kernel/lib.rs+34−3

@@ -14,11 +14,9 @@ · 13 unmodified lines
14	#![no_std]	14	#![no_std]
15	#![feature(allocator_api)]	15	#![feature(allocator_api)]
16	#![feature(coerce_unsized)]	16	#![feature(coerce_unsized)]
17	#![feature(const_maybe_uninit_zeroed)]
18	#![feature(dispatch_from_dyn)]	17	#![feature(dispatch_from_dyn)]
19	#![feature(new_uninit)]	18	#![feature(new_uninit)]
20	#![feature(offset_of)]	19	#![feature(offset_of)]
21	#![feature(ptr_metadata)]
22	#![feature(receiver_trait)]	20	#![feature(receiver_trait)]
23	#![feature(unsize)]	21	#![feature(unsize)]
24		22
@@ -49,6 +47,7 @@ pub mod std_vendor; · 24 unmodified lines
49	pub mod str;	47	pub mod str;
50	pub mod sync;	48	pub mod sync;
51	pub mod task;	49	pub mod task;
		50	pub mod time;
52	pub mod types;	51	pub mod types;
53	pub mod workqueue;	52	pub mod workqueue;
54		53
@@ -78,7 +77,7 @@ pub trait Module: Sized + Sync { · 23 unmodified lines
78		77
79	/// Equivalent to `THIS_MODULE` in the C API.	78	/// Equivalent to `THIS_MODULE` in the C API.
80	///	79	///
81	/// C header: `include/linux/export.h`	80	/// C header: [`include/linux/export.h`](srctree/include/linux/export.h)
82	pub struct ThisModule(*mut bindings::module);	81	pub struct ThisModule(*mut bindings::module);
83		82
84	// SAFETY: `THIS_MODULE` may be used from all threads within a module.	83	// SAFETY: `THIS_MODULE` may be used from all threads within a module.
@@ -102,3 +101,35 @@ fn panic(info: &core::panic::PanicInfo<'_>) -> ! { · 17 unmodified lines
102	// SAFETY: FFI call.	101	// SAFETY: FFI call.
103	unsafe { bindings::BUG() };	102	unsafe { bindings::BUG() };
104	}	103	}
		104
		105	/// Produces a pointer to an object from a pointer to one of its fields.
		106	///
		107	/// # Safety
		108	///
		109	/// The pointer passed to this macro, and the pointer returned by this macro, must both be in
		110	/// bounds of the same allocation.
		111	///
		112	/// # Examples
		113	///
		114	/// ```
		115	/// # use kernel::container_of;
		116	/// struct Test {
		117	/// a: u64,
		118	/// b: u32,
		119	/// }
		120	///
		121	/// let test = Test { a: 10, b: 20 };
		122	/// let b_ptr = &test.b;
		123	/// // SAFETY: The pointer points at the `b` field of a `Test`, so the resulting pointer will be
		124	/// // in-bounds of the same allocation as `b_ptr`.
		125	/// let test_alias = unsafe { container_of!(b_ptr, Test, b) };
		126	/// assert!(core::ptr::eq(&test, test_alias));
		127	/// ```
		128	#[macro_export]
		129	macro_rules! container_of {
		130	($ptr:expr, $type:ty, $($f:tt)*) => {{
		131	let ptr = $ptr as const _ as const u8;
		132	let offset: usize = ::core::mem::offset_of!($type, $($f)*);
		133	ptr.sub(offset) as *const $type
		134	}}
		135	}

rust/kernel/str.rs+182−11

@@ -13,9 +13,102 @@ use crate::{ · 12 unmodified lines
13	};	13	};
14		14
15	/// Byte string without UTF-8 validity guarantee.	15	/// Byte string without UTF-8 validity guarantee.
16	///	16	#[repr(transparent)]
17	/// `BStr` is simply an alias to `[u8]`, but has a more evident semantical meaning.	17	pub struct BStr([u8]);
18	pub type BStr = [u8];	18
		19	impl BStr {
		20	/// Returns the length of this string.
		21	#[inline]
		22	pub const fn len(&self) -> usize {
		23	self.0.len()
		24	}
		25
		26	/// Returns `true` if the string is empty.
		27	#[inline]
		28	pub const fn is_empty(&self) -> bool {
		29	self.len() == 0
		30	}
		31
		32	/// Creates a [`BStr`] from a `[u8]`.
		33	#[inline]
		34	pub const fn from_bytes(bytes: &[u8]) -> &Self {
		35	// SAFETY: `BStr` is transparent to `[u8]`.
		36	unsafe { &(bytes as const [u8] as *const BStr) }
		37	}
		38	}
		39
		40	impl fmt::Display for BStr {
		41	/// Formats printable ASCII characters, escaping the rest.
		42	///
		43	/// ```
		44	/// # use kernel::{fmt, b_str, str::{BStr, CString}};
		45	/// let ascii = b_str!("Hello, BStr!");
		46	/// let s = CString::try_from_fmt(fmt!("{}", ascii)).unwrap();
		47	/// assert_eq!(s.as_bytes(), "Hello, BStr!".as_bytes());
		48	///
		49	/// let non_ascii = b_str!("🦀");
		50	/// let s = CString::try_from_fmt(fmt!("{}", non_ascii)).unwrap();
		51	/// assert_eq!(s.as_bytes(), "\\xf0\\x9f\\xa6\\x80".as_bytes());
		52	/// ```
		53	fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
		54	for &b in &self.0 {
		55	match b {
		56	// Common escape codes.
		57	b'\t' => f.write_str("\\t")?,
		58	b'\n' => f.write_str("\\n")?,
		59	b'\r' => f.write_str("\\r")?,
		60	// Printable characters.
		61	0x20..=0x7e => f.write_char(b as char)?,
		62	_ => write!(f, "\\x{:02x}", b)?,
		63	}
		64	}
		65	Ok(())
		66	}
		67	}
		68
		69	impl fmt::Debug for BStr {
		70	/// Formats printable ASCII characters with a double quote on either end,
		71	/// escaping the rest.
		72	///
		73	/// ```
		74	/// # use kernel::{fmt, b_str, str::{BStr, CString}};
		75	/// // Embedded double quotes are escaped.
		76	/// let ascii = b_str!("Hello, \"BStr\"!");
		77	/// let s = CString::try_from_fmt(fmt!("{:?}", ascii)).unwrap();
		78	/// assert_eq!(s.as_bytes(), "\"Hello, \\\"BStr\\\"!\"".as_bytes());
		79	///
		80	/// let non_ascii = b_str!("😺");
		81	/// let s = CString::try_from_fmt(fmt!("{:?}", non_ascii)).unwrap();
		82	/// assert_eq!(s.as_bytes(), "\"\\xf0\\x9f\\x98\\xba\"".as_bytes());
		83	/// ```
		84	fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
		85	f.write_char('"')?;
		86	for &b in &self.0 {
		87	match b {
		88	// Common escape codes.
		89	b'\t' => f.write_str("\\t")?,
		90	b'\n' => f.write_str("\\n")?,
		91	b'\r' => f.write_str("\\r")?,
		92	// String escape characters.
		93	b'\"' => f.write_str("\\\"")?,
		94	b'\\' => f.write_str("\\\\")?,
		95	// Printable characters.
		96	0x20..=0x7e => f.write_char(b as char)?,
		97	_ => write!(f, "\\x{:02x}", b)?,
		98	}
		99	}
		100	f.write_char('"')
		101	}
		102	}
		103
		104	impl Deref for BStr {
		105	type Target = [u8];
		106
		107	#[inline]
		108	fn deref(&self) -> &Self::Target {
		109	&self.0
		110	}
		111	}
19		112
20	/// Creates a new [`BStr`] from a string literal.	113	/// Creates a new [`BStr`] from a string literal.
21	///	114	///
@@ -33,7 +126,7 @@ pub type BStr = [u8]; · 11 unmodified lines
33	macro_rules! b_str {	126	macro_rules! b_str {
34	($str:literal) => {{	127	($str:literal) => {{
35	const S: &'static str = $str;	128	const S: &'static str = $str;
36	const C: &'static $crate::str::BStr = S.as_bytes();	129	const C: &'static $crate::str::BStr = $crate::str::BStr::from_bytes(S.as_bytes());
37	C	130	C
38	}};	131	}};
39	}	132	}
@@ -149,13 +242,13 @@ impl CStr { · 109 unmodified lines
149	self.0.as_ptr() as _	242	self.0.as_ptr() as _
150	}	243	}
151		244
152	/// Convert the string to a byte slice without the trailing 0 byte.	245	/// Convert the string to a byte slice without the trailing `NUL` byte.
153	#[inline]	246	#[inline]
154	pub fn as_bytes(&self) -> &[u8] {	247	pub fn as_bytes(&self) -> &[u8] {
155	&self.0[..self.len()]	248	&self.0[..self.len()]
156	}	249	}
157		250
158	/// Convert the string to a byte slice containing the trailing 0 byte.	251	/// Convert the string to a byte slice containing the trailing `NUL` byte.
159	#[inline]	252	#[inline]
160	pub const fn as_bytes_with_nul(&self) -> &[u8] {	253	pub const fn as_bytes_with_nul(&self) -> &[u8] {
161	&self.0	254	&self.0
@@ -191,9 +284,9 @@ impl CStr { · 29 unmodified lines
191	/// ```	284	/// ```
192	/// # use kernel::c_str;	285	/// # use kernel::c_str;
193	/// # use kernel::str::CStr;	286	/// # use kernel::str::CStr;
		287	/// let bar = c_str!("ツ");
194	/// // SAFETY: String literals are guaranteed to be valid UTF-8	288	/// // SAFETY: String literals are guaranteed to be valid UTF-8
195	/// // by the Rust compiler.	289	/// // by the Rust compiler.
196	/// let bar = c_str!("ツ");
197	/// assert_eq!(unsafe { bar.as_str_unchecked() }, "ツ");	290	/// assert_eq!(unsafe { bar.as_str_unchecked() }, "ツ");
198	/// ```	291	/// ```
199	#[inline]	292	#[inline]
@@ -271,7 +364,7 @@ impl fmt::Debug for CStr { · 71 unmodified lines
271	impl AsRef<BStr> for CStr {	364	impl AsRef<BStr> for CStr {
272	#[inline]	365	#[inline]
273	fn as_ref(&self) -> &BStr {	366	fn as_ref(&self) -> &BStr {
274	self.as_bytes()	367	BStr::from_bytes(self.as_bytes())
275	}	368	}
276	}	369	}
277		370
@@ -280,7 +373,7 @@ impl Deref for CStr { · 2 unmodified lines
280		373
281	#[inline]	374	#[inline]
282	fn deref(&self) -> &Self::Target {	375	fn deref(&self) -> &Self::Target {
283	self.as_bytes()	376	self.as_ref()
284	}	377	}
285	}	378	}
286		379
@@ -327,7 +420,7 @@ where · 40 unmodified lines
327		420
328	#[inline]	421	#[inline]
329	fn index(&self, index: Idx) -> &Self::Output {	422	fn index(&self, index: Idx) -> &Self::Output {
330	&self.as_bytes()[index]	423	&self.as_ref()[index]
331	}	424	}
332	}	425	}
333		426
@@ -357,6 +450,21 @@ macro_rules! c_str { · 23 unmodified lines
357	#[cfg(test)]	450	#[cfg(test)]
358	mod tests {	451	mod tests {
359	use super::*;	452	use super::*;
		453	use alloc::format;
		454
		455	const ALL_ASCII_CHARS: &'static str =
		456	"\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x09\\x0a\\x0b\\x0c\\x0d\\x0e\\x0f\
		457	\\x10\\x11\\x12\\x13\\x14\\x15\\x16\\x17\\x18\\x19\\x1a\\x1b\\x1c\\x1d\\x1e\\x1f \
		458	!\"#$%&'()*+,-./0123456789:;<=>?@\
		459	ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{\|}~\\x7f\
		460	\\x80\\x81\\x82\\x83\\x84\\x85\\x86\\x87\\x88\\x89\\x8a\\x8b\\x8c\\x8d\\x8e\\x8f\
		461	\\x90\\x91\\x92\\x93\\x94\\x95\\x96\\x97\\x98\\x99\\x9a\\x9b\\x9c\\x9d\\x9e\\x9f\
		462	\\xa0\\xa1\\xa2\\xa3\\xa4\\xa5\\xa6\\xa7\\xa8\\xa9\\xaa\\xab\\xac\\xad\\xae\\xaf\
		463	\\xb0\\xb1\\xb2\\xb3\\xb4\\xb5\\xb6\\xb7\\xb8\\xb9\\xba\\xbb\\xbc\\xbd\\xbe\\xbf\
		464	\\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\
		465	\\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd7\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdf\
		466	\\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\
		467	\\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf7\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff";
360		468
361	#[test]	469	#[test]
362	fn test_cstr_to_str() {	470	fn test_cstr_to_str() {
@@ -381,6 +489,69 @@ mod tests { · 18 unmodified lines
381	let unchecked_str = unsafe { checked_cstr.as_str_unchecked() };	489	let unchecked_str = unsafe { checked_cstr.as_str_unchecked() };
382	assert_eq!(unchecked_str, "🐧");	490	assert_eq!(unchecked_str, "🐧");
383	}	491	}
		492
		493	#[test]
		494	fn test_cstr_display() {
		495	let hello_world = CStr::from_bytes_with_nul(b"hello, world!\0").unwrap();
		496	assert_eq!(format!("{}", hello_world), "hello, world!");
		497	let non_printables = CStr::from_bytes_with_nul(b"\x01\x09\x0a\0").unwrap();
		498	assert_eq!(format!("{}", non_printables), "\\x01\\x09\\x0a");
		499	let non_ascii = CStr::from_bytes_with_nul(b"d\xe9j\xe0 vu\0").unwrap();
		500	assert_eq!(format!("{}", non_ascii), "d\\xe9j\\xe0 vu");
		501	let good_bytes = CStr::from_bytes_with_nul(b"\xf0\x9f\xa6\x80\0").unwrap();
		502	assert_eq!(format!("{}", good_bytes), "\\xf0\\x9f\\xa6\\x80");
		503	}
		504
		505	#[test]
		506	fn test_cstr_display_all_bytes() {
		507	let mut bytes: [u8; 256] = [0; 256];
		508	// fill `bytes` with [1..=255] + [0]
		509	for i in u8::MIN..=u8::MAX {
		510	bytes[i as usize] = i.wrapping_add(1);
		511	}
		512	let cstr = CStr::from_bytes_with_nul(&bytes).unwrap();
		513	assert_eq!(format!("{}", cstr), ALL_ASCII_CHARS);
		514	}
		515
		516	#[test]
		517	fn test_cstr_debug() {
		518	let hello_world = CStr::from_bytes_with_nul(b"hello, world!\0").unwrap();
		519	assert_eq!(format!("{:?}", hello_world), "\"hello, world!\"");
		520	let non_printables = CStr::from_bytes_with_nul(b"\x01\x09\x0a\0").unwrap();
		521	assert_eq!(format!("{:?}", non_printables), "\"\\x01\\x09\\x0a\"");
		522	let non_ascii = CStr::from_bytes_with_nul(b"d\xe9j\xe0 vu\0").unwrap();
		523	assert_eq!(format!("{:?}", non_ascii), "\"d\\xe9j\\xe0 vu\"");
		524	let good_bytes = CStr::from_bytes_with_nul(b"\xf0\x9f\xa6\x80\0").unwrap();
		525	assert_eq!(format!("{:?}", good_bytes), "\"\\xf0\\x9f\\xa6\\x80\"");
		526	}
		527
		528	#[test]
		529	fn test_bstr_display() {
		530	let hello_world = BStr::from_bytes(b"hello, world!");
		531	assert_eq!(format!("{}", hello_world), "hello, world!");
		532	let escapes = BStr::from_bytes(b"_\t_\n_\r_\\_\'_\"_");
		533	assert_eq!(format!("{}", escapes), "_\\t_\\n_\\r_\\_'_\"_");
		534	let others = BStr::from_bytes(b"\x01");
		535	assert_eq!(format!("{}", others), "\\x01");
		536	let non_ascii = BStr::from_bytes(b"d\xe9j\xe0 vu");
		537	assert_eq!(format!("{}", non_ascii), "d\\xe9j\\xe0 vu");
		538	let good_bytes = BStr::from_bytes(b"\xf0\x9f\xa6\x80");
		539	assert_eq!(format!("{}", good_bytes), "\\xf0\\x9f\\xa6\\x80");
		540	}
		541
		542	#[test]
		543	fn test_bstr_debug() {
		544	let hello_world = BStr::from_bytes(b"hello, world!");
		545	assert_eq!(format!("{:?}", hello_world), "\"hello, world!\"");
		546	let escapes = BStr::from_bytes(b"_\t_\n_\r_\\_\'_\"_");
		547	assert_eq!(format!("{:?}", escapes), "\"_\\t_\\n_\\r_\\\\_'_\\\"_\"");
		548	let others = BStr::from_bytes(b"\x01");
		549	assert_eq!(format!("{:?}", others), "\"\\x01\"");
		550	let non_ascii = BStr::from_bytes(b"d\xe9j\xe0 vu");
		551	assert_eq!(format!("{:?}", non_ascii), "\"d\\xe9j\\xe0 vu\"");
		552	let good_bytes = BStr::from_bytes(b"\xf0\x9f\xa6\x80");
		553	assert_eq!(format!("{:?}", good_bytes), "\"\\xf0\\x9f\\xa6\\x80\"");
		554	}
384	}	555	}
385		556
386	/// Allows formatting of [`fmt::Arguments`] into a raw buffer.	557	/// Allows formatting of [`fmt::Arguments`] into a raw buffer.
@@ -449,7 +620,7 @@ impl RawFormatter { · 62 unmodified lines
449	self.pos as _	620	self.pos as _
450	}	621	}
451		622
452	/// Return the number of bytes written to the formatter.	623	/// Returns the number of bytes written to the formatter.
453	pub(crate) fn bytes_written(&self) -> usize {	624	pub(crate) fn bytes_written(&self) -> usize {
454	self.pos - self.beg	625	self.pos - self.beg
455	}	626	}

rust/kernel/sync.rs+3−2

@@ -13,8 +13,9 @@ pub mod lock; · 12 unmodified lines
13	mod locked_by;	13	mod locked_by;
14		14
15	pub use arc::{Arc, ArcBorrow, UniqueArc};	15	pub use arc::{Arc, ArcBorrow, UniqueArc};
16	pub use condvar::CondVar;	16	pub use condvar::{new_condvar, CondVar, CondVarTimeoutResult};
17	pub use lock::{mutex::Mutex, spinlock::SpinLock};	17	pub use lock::mutex::{new_mutex, Mutex};
		18	pub use lock::spinlock::{new_spinlock, SpinLock};
18	pub use locked_by::LockedBy;	19	pub use locked_by::LockedBy;
19		20
20	/// Represents a lockdep class. It's a wrapper around C's `lock_class_key`.	21	/// Represents a lockdep class. It's a wrapper around C's `lock_class_key`.

rust/kernel/sync/arc.rs+14−16

@@ -30,7 +30,7 @@ use core::{ · 29 unmodified lines
30	mem::{ManuallyDrop, MaybeUninit},	30	mem::{ManuallyDrop, MaybeUninit},
31	ops::{Deref, DerefMut},	31	ops::{Deref, DerefMut},
32	pin::Pin,	32	pin::Pin,
33	ptr::{NonNull, Pointee},	33	ptr::NonNull,
34	};	34	};
35	use macros::pin_data;	35	use macros::pin_data;
36		36
@@ -56,7 +56,7 @@ mod std_vendor; · 19 unmodified lines
56	/// b: u32,	56	/// b: u32,
57	/// }	57	/// }
58	///	58	///
59	/// // Create a ref-counted instance of `Example`.	59	/// // Create a refcounted instance of `Example`.
60	/// let obj = Arc::try_new(Example { a: 10, b: 20 })?;	60	/// let obj = Arc::try_new(Example { a: 10, b: 20 })?;
61	///	61	///
62	/// // Get a new pointer to `obj` and increment the refcount.	62	/// // Get a new pointer to `obj` and increment the refcount.
@@ -239,22 +239,20 @@ impl<T: ?Sized> Arc<T> { · 176 unmodified lines
239	// binary, so its layout is not so large that it can trigger arithmetic overflow.	239	// binary, so its layout is not so large that it can trigger arithmetic overflow.
240	let val_offset = unsafe { refcount_layout.extend(val_layout).unwrap_unchecked().1 };	240	let val_offset = unsafe { refcount_layout.extend(val_layout).unwrap_unchecked().1 };
241		241
242	let metadata: <T as Pointee>::Metadata = core::ptr::metadata(ptr);	242	// Pointer casts leave the metadata unchanged. This is okay because the metadata of `T` and
243	// SAFETY: The metadata of `T` and `ArcInner<T>` is the same because `ArcInner` is a struct	243	// `ArcInner<T>` is the same since `ArcInner` is a struct with `T` as its last field.
244	// with `T` as its last field.
245	//	244	//
246	// This is documented at:	245	// This is documented at:
247	// <https://doc.rust-lang.org/std/ptr/trait.Pointee.html>.	246	// <https://doc.rust-lang.org/std/ptr/trait.Pointee.html>.
248	let metadata: <ArcInner<T> as Pointee>::Metadata =	247	let ptr = ptr as *const ArcInner<T>;
249	unsafe { core::mem::transmute_copy(&metadata) };	248
250	// SAFETY: The pointer is in-bounds of an allocation both before and after offsetting the	249	// SAFETY: The pointer is in-bounds of an allocation both before and after offsetting the
251	// pointer, since it originates from a previous call to `Arc::into_raw` and is still valid.	250	// pointer, since it originates from a previous call to `Arc::into_raw` and is still valid.
252	let ptr = unsafe { (ptr as mut u8).sub(val_offset) as mut () };	251	let ptr = unsafe { ptr.byte_sub(val_offset) };
253	let ptr = core::ptr::from_raw_parts_mut(ptr, metadata);
254		252
255	// SAFETY: By the safety requirements we know that `ptr` came from `Arc::into_raw`, so the	253	// SAFETY: By the safety requirements we know that `ptr` came from `Arc::into_raw`, so the
256	// reference count held then will be owned by the new `Arc` object.	254	// reference count held then will be owned by the new `Arc` object.
257	unsafe { Self::from_inner(NonNull::new_unchecked(ptr)) }	255	unsafe { Self::from_inner(NonNull::new_unchecked(ptr.cast_mut())) }
258	}	256	}
259		257
260	/// Returns an [`ArcBorrow`] from the given [`Arc`].	258	/// Returns an [`ArcBorrow`] from the given [`Arc`].
@@ -365,12 +363,12 @@ impl<T: ?Sized> From<Pin<UniqueArc<T>>> for Arc<T> { · 104 unmodified lines
365	/// A borrowed reference to an [`Arc`] instance.	363	/// A borrowed reference to an [`Arc`] instance.
366	///	364	///
367	/// For cases when one doesn't ever need to increment the refcount on the allocation, it is simpler	365	/// For cases when one doesn't ever need to increment the refcount on the allocation, it is simpler
368	/// to use just `&T`, which we can trivially get from an `Arc<T>` instance.	366	/// to use just `&T`, which we can trivially get from an [`Arc<T>`] instance.
369	///	367	///
370	/// However, when one may need to increment the refcount, it is preferable to use an `ArcBorrow<T>`	368	/// However, when one may need to increment the refcount, it is preferable to use an `ArcBorrow<T>`
371	/// over `&Arc<T>` because the latter results in a double-indirection: a pointer (shared reference)	369	/// over `&Arc<T>` because the latter results in a double-indirection: a pointer (shared reference)
372	/// to a pointer (`Arc<T>`) to the object (`T`). An [`ArcBorrow`] eliminates this double	370	/// to a pointer ([`Arc<T>`]) to the object (`T`). An [`ArcBorrow`] eliminates this double
373	/// indirection while still allowing one to increment the refcount and getting an `Arc<T>` when/if	371	/// indirection while still allowing one to increment the refcount and getting an [`Arc<T>`] when/if
374	/// needed.	372	/// needed.
375	///	373	///
376	/// # Invariants	374	/// # Invariants
@@ -510,7 +508,7 @@ impl<T: ?Sized> Deref for ArcBorrow<'_, T> { · 133 unmodified lines
510	/// # test().unwrap();	508	/// # test().unwrap();
511	/// ```	509	/// ```
512	///	510	///
513	/// In the following example we first allocate memory for a ref-counted `Example` but we don't	511	/// In the following example we first allocate memory for a refcounted `Example` but we don't
514	/// initialise it on allocation. We do initialise it later with a call to [`UniqueArc::write`],	512	/// initialise it on allocation. We do initialise it later with a call to [`UniqueArc::write`],
515	/// followed by a conversion to `Arc<Example>`. This is particularly useful when allocation happens	513	/// followed by a conversion to `Arc<Example>`. This is particularly useful when allocation happens
516	/// in one context (e.g., sleepable) and initialisation in another (e.g., atomic):	514	/// in one context (e.g., sleepable) and initialisation in another (e.g., atomic):
@@ -560,7 +558,7 @@ impl<T> UniqueArc<T> { · 43 unmodified lines
560	/// Tries to allocate a new [`UniqueArc`] instance.	558	/// Tries to allocate a new [`UniqueArc`] instance.
561	pub fn try_new(value: T) -> Result<Self, AllocError> {	559	pub fn try_new(value: T) -> Result<Self, AllocError> {
562	Ok(Self {	560	Ok(Self {
563	// INVARIANT: The newly-created object has a ref-count of 1.	561	// INVARIANT: The newly-created object has a refcount of 1.
564	inner: Arc::try_new(value)?,	562	inner: Arc::try_new(value)?,
565	})	563	})
566	}	564	}
@@ -574,7 +572,7 @@ impl<T> UniqueArc<T> { · 7 unmodified lines
574	data <- init::uninit::<T, AllocError>(),	572	data <- init::uninit::<T, AllocError>(),
575	}? AllocError))?;	573	}? AllocError))?;
576	Ok(UniqueArc {	574	Ok(UniqueArc {
577	// INVARIANT: The newly-created object has a ref-count of 1.	575	// INVARIANT: The newly-created object has a refcount of 1.
578	// SAFETY: The pointer from the `Box` is valid.	576	// SAFETY: The pointer from the `Box` is valid.
579	inner: unsafe { Arc::from_inner(Box::leak(inner).into()) },	577	inner: unsafe { Arc::from_inner(Box::leak(inner).into()) },
580	})	578	})

rust/kernel/sync/condvar.rs+88−22

 //! variable.
 use super::{lock::Backend, lock::Guard, LockClassKey};
-use crate::{bindings, init::PinInit, pin_init, str::CStr, types::Opaque};
+use crate::{
+    bindings,
+    init::PinInit,
+    pin_init,
+    str::CStr,
+    task::{MAX_SCHEDULE_TIMEOUT, TASK_INTERRUPTIBLE, TASK_NORMAL, TASK_UNINTERRUPTIBLE},
+    time::Jiffies,
+    types::Opaque,
+};
+use core::ffi::{c_int, c_long};
 use core::marker::PhantomPinned;
+use core::ptr;
 use macros::pin_data;
 /// Creates a [`CondVar`] initialiser with the given name and a newly-created lock class.
         $crate::sync::CondVar::new($crate::optional_name!($($name)?), $crate::static_lock_class!())
     };
 }
+pub use new_condvar;
 /// A conditional variable.
 ///
 /// The following is an example of using a condvar with a mutex:
 ///
 /// ```
-/// use kernel::sync::{CondVar, Mutex};
+/// use kernel::sync::{new_condvar, new_mutex, CondVar, Mutex};
-/// use kernel::{new_condvar, new_mutex};
 ///
 /// #[pin_data]
 /// pub struct Example {
 #[pin_data]
 pub struct CondVar {
     #[pin]
-    pub(crate) wait_list: Opaque<bindings::wait_queue_head>,
+    pub(crate) wait_queue_head: Opaque<bindings::wait_queue_head>,
     /// A condvar needs to be pinned because it contains a [`struct list_head`] that is
     /// self-referential, so it cannot be safely moved once it is initialised.
+    ///
+    /// [`struct list_head`]: srctree/include/linux/types.h
     #[pin]
     _pin: PhantomPinned,
 }
             _pin: PhantomPinned,
             // SAFETY: `slot` is valid while the closure is called and both `name` and `key` have
             // static lifetimes so they live indefinitely.
-            wait_list <- Opaque::ffi_init(|slot| unsafe {
+            wait_queue_head <- Opaque::ffi_init(|slot| unsafe {
                 bindings::__init_waitqueue_head(slot, name.as_char_ptr(), key.as_ptr())
             }),
         })
     }
-    fn wait_internal<T: ?Sized, B: Backend>(&self, wait_state: u32, guard: &mut Guard<'_, T, B>) {
+    fn wait_internal<T: ?Sized, B: Backend>(
+        &self,
+        wait_state: c_int,
+        guard: &mut Guard<'_, T, B>,
+        timeout_in_jiffies: c_long,
+    ) -> c_long {
         let wait = Opaque::<bindings::wait_queue_entry>::uninit();
         // SAFETY: `wait` points to valid memory.
         unsafe { bindings::init_wait(wait.get()) };
-        // SAFETY: Both `wait` and `wait_list` point to valid memory.
+        // SAFETY: Both `wait` and `wait_queue_head` point to valid memory.
         unsafe {
-            bindings::prepare_to_wait_exclusive(self.wait_list.get(), wait.get(), wait_state as _)
+            bindings::prepare_to_wait_exclusive(self.wait_queue_head.get(), wait.get(), wait_state)
         };
-        // SAFETY: No arguments, switches to another thread.
+        // SAFETY: Switches to another thread. The timeout can be any number.
-        guard.do_unlocked(|| unsafe { bindings::schedule() });
+        let ret = guard.do_unlocked(|| unsafe { bindings::schedule_timeout(timeout_in_jiffies) });
+        // SAFETY: Both `wait` and `wait_queue_head` point to valid memory.
+        unsafe { bindings::finish_wait(self.wait_queue_head.get(), wait.get()) };
-        // SAFETY: Both `wait` and `wait_list` point to valid memory.
+        ret
-        unsafe { bindings::finish_wait(self.wait_list.get(), wait.get()) };
     }
     /// Releases the lock and waits for a notification in uninterruptible mode.
     /// [`CondVar::notify_one`] or [`CondVar::notify_all`]. Note that it may also wake up
     /// spuriously.
     pub fn wait<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) {
-        self.wait_internal(bindings::TASK_UNINTERRUPTIBLE, guard);
+        self.wait_internal(TASK_UNINTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT);
     }
     /// Releases the lock and waits for a notification in interruptible mode.
     /// Returns whether there is a signal pending.
     #[must_use = "wait_interruptible returns if a signal is pending, so the caller must check the return value"]
     pub fn wait_interruptible<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) -> bool {
-        self.wait_internal(bindings::TASK_INTERRUPTIBLE, guard);
+        self.wait_internal(TASK_INTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT);
         crate::current!().signal_pending()
     }
-    /// Calls the kernel function to notify the appropriate number of threads with the given flags.
+    /// Releases the lock and waits for a notification in interruptible mode.
-    fn notify(&self, count: i32, flags: u32) {
+    ///
-        // SAFETY: `wait_list` points to valid memory.
+    /// Atomically releases the given lock (whose ownership is proven by the guard) and puts the
+    /// thread to sleep. It wakes up when notified by [`CondVar::notify_one`] or
+    /// [`CondVar::notify_all`], or when a timeout occurs, or when the thread receives a signal.
+    #[must_use = "wait_interruptible_timeout returns if a signal is pending, so the caller must check the return value"]
+    pub fn wait_interruptible_timeout<T: ?Sized, B: Backend>(
+        &self,
+        guard: &mut Guard<'_, T, B>,
+        jiffies: Jiffies,
+    ) -> CondVarTimeoutResult {
+        let jiffies = jiffies.try_into().unwrap_or(MAX_SCHEDULE_TIMEOUT);
+        let res = self.wait_internal(TASK_INTERRUPTIBLE, guard, jiffies);
+        match (res as Jiffies, crate::current!().signal_pending()) {
+            (jiffies, true) => CondVarTimeoutResult::Signal { jiffies },
+            (0, false) => CondVarTimeoutResult::Timeout,
+            (jiffies, false) => CondVarTimeoutResult::Woken { jiffies },
+        }
+    }
+    /// Calls the kernel function to notify the appropriate number of threads.
+    fn notify(&self, count: c_int) {
+        // SAFETY: `wait_queue_head` points to valid memory.
         unsafe {
             bindings::__wake_up(
-                self.wait_list.get(),
+                self.wait_queue_head.get(),
-                bindings::TASK_NORMAL,
+                TASK_NORMAL,
                 count,
-                flags as _,
+                ptr::null_mut(),
             )
         };
     }
+    /// Calls the kernel function to notify one thread synchronously.
+    ///
+    /// This method behaves like `notify_one`, except that it hints to the scheduler that the
+    /// current thread is about to go to sleep, so it should schedule the target thread on the same
+    /// CPU.
+    pub fn notify_sync(&self) {
+        // SAFETY: `wait_queue_head` points to valid memory.
+        unsafe { bindings::__wake_up_sync(self.wait_queue_head.get(), TASK_NORMAL) };
+    }
     /// Wakes a single waiter up, if any.
     ///
     /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
     /// completely (as opposed to automatically waking up the next waiter).
     pub fn notify_one(&self) {
-        self.notify(1, 0);
+        self.notify(1);
     }
     /// Wakes all waiters up, if any.
     /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
     /// completely (as opposed to automatically waking up the next waiter).
     pub fn notify_all(&self) {
-        self.notify(0, 0);
+        self.notify(0);
     }
 }
+/// The return type of `wait_timeout`.
+pub enum CondVarTimeoutResult {
+    /// The timeout was reached.
+    Timeout,
+    /// Somebody woke us up.
+    Woken {
+        /// Remaining sleep duration.
+        jiffies: Jiffies,
+    },
+    /// A signal occurred.
+    Signal {
+        /// Remaining sleep duration.
+        jiffies: Jiffies,
+    },
+}

rust/kernel/sync/lock.rs+13−6

 /// # Safety
 ///
 /// - Implementers must ensure that only one thread/CPU may access the protected data once the lock
-/// is owned, that is, between calls to `lock` and `unlock`.
+///   is owned, that is, between calls to [`lock`] and [`unlock`].
-/// - Implementers must also ensure that `relock` uses the same locking method as the original
+/// - Implementers must also ensure that [`relock`] uses the same locking method as the original
-/// lock operation.
+///   lock operation.
+///
+/// [`lock`]: Backend::lock
+/// [`unlock`]: Backend::unlock
+/// [`relock`]: Backend::relock
 pub unsafe trait Backend {
     /// The state required by the lock.
     type State;
-    /// The state required to be kept between lock and unlock.
+    /// The state required to be kept between [`lock`] and [`unlock`].
+    ///
+    /// [`lock`]: Backend::lock
+    /// [`unlock`]: Backend::unlock
     type GuardState;
     /// Initialises the lock.
 unsafe impl<T: Sync + ?Sized, B: Backend> Sync for Guard<'_, T, B> {}
 impl<T: ?Sized, B: Backend> Guard<'_, T, B> {
-    pub(crate) fn do_unlocked(&mut self, cb: impl FnOnce()) {
+    pub(crate) fn do_unlocked<U>(&mut self, cb: impl FnOnce() -> U) -> U {
         // SAFETY: The caller owns the lock, so it is safe to unlock it.
         unsafe { B::unlock(self.lock.state.get(), &self.state) };
         let _relock =
             ScopeGuard::new(|| unsafe { B::relock(self.lock.state.get(), &mut self.state) });
-        cb();
+        cb()
     }
 }

rust/kernel/sync/lock/mutex.rs+2−1

             $inner, $crate::optional_name!($($name)?), $crate::static_lock_class!())
     };
 }
+pub use new_mutex;
 /// A mutual exclusion primitive.
 ///
 /// contains an inner struct (`Inner`) that is protected by a mutex.
 ///
 /// ```
-/// use kernel::{init::InPlaceInit, init::PinInit, new_mutex, pin_init, sync::Mutex};
+/// use kernel::sync::{new_mutex, Mutex};
 ///
 /// struct Inner {
 ///     a: u32,

rust/kernel/sync/lock/spinlock.rs+3−2

             $inner, $crate::optional_name!($($name)?), $crate::static_lock_class!())
     };
 }
+pub use new_spinlock;
 /// A spinlock.
 ///
 /// contains an inner struct (`Inner`) that is protected by a spinlock.
 ///
 /// ```
-/// use kernel::{init::InPlaceInit, init::PinInit, new_spinlock, pin_init, sync::SpinLock};
+/// use kernel::sync::{new_spinlock, SpinLock};
 ///
 /// struct Inner {
 ///     a: u32,
     unsafe fn unlock(ptr: *mut Self::State, _guard_state: &Self::GuardState) {
         // SAFETY: The safety requirements of this function ensure that `ptr` is valid and that the
-        // caller is the owner of the mutex.
+        // caller is the owner of the spinlock.
         unsafe { bindings::spin_unlock(ptr) }
     }
 }

rust/kernel/sync/locked_by.rs+5−2

@@ -9,14 +9,17 @@ use core::{cell::UnsafeCell, mem::size_of, ptr}; · 8 unmodified lines
9	/// Allows access to some data to be serialised by a lock that does not wrap it.	9	/// Allows access to some data to be serialised by a lock that does not wrap it.
10	///	10	///
11	/// In most cases, data protected by a lock is wrapped by the appropriate lock type, e.g.,	11	/// In most cases, data protected by a lock is wrapped by the appropriate lock type, e.g.,
12	/// [`super::Mutex`] or [`super::SpinLock`]. [`LockedBy`] is meant for cases when this is not	12	/// [`Mutex`] or [`SpinLock`]. [`LockedBy`] is meant for cases when this is not possible.
13	/// possible. For example, if a container has a lock and some data in the contained elements needs	13	/// For example, if a container has a lock and some data in the contained elements needs
14	/// to be protected by the same lock.	14	/// to be protected by the same lock.
15	///	15	///
16	/// [`LockedBy`] wraps the data in lieu of another locking primitive, and only allows access to it	16	/// [`LockedBy`] wraps the data in lieu of another locking primitive, and only allows access to it
17	/// when the caller shows evidence that the 'external' lock is locked. It panics if the evidence	17	/// when the caller shows evidence that the 'external' lock is locked. It panics if the evidence
18	/// refers to the wrong instance of the lock.	18	/// refers to the wrong instance of the lock.
19	///	19	///
		20	/// [`Mutex`]: super::Mutex
		21	/// [`SpinLock`]: super::SpinLock
		22	///
20	/// # Examples	23	/// # Examples
21	///	24	///
22	/// The following is an example for illustrative purposes: `InnerDirectory::bytes_used` is an	25	/// The following is an example for illustrative purposes: `InnerDirectory::bytes_used` is an

rust/kernel/task.rs+20−4

 //! C header: [`include/linux/sched.h`](srctree/include/linux/sched.h).
 use crate::{bindings, types::Opaque};
-use core::{marker::PhantomData, ops::Deref, ptr};
+use core::{
+    ffi::{c_int, c_long, c_uint},
+    marker::PhantomData,
+    ops::Deref,
+    ptr,
+};
+/// A sentinel value used for infinite timeouts.
+pub const MAX_SCHEDULE_TIMEOUT: c_long = c_long::MAX;
+/// Bitmask for tasks that are sleeping in an interruptible state.
+pub const TASK_INTERRUPTIBLE: c_int = bindings::TASK_INTERRUPTIBLE as c_int;
+/// Bitmask for tasks that are sleeping in an uninterruptible state.
+pub const TASK_UNINTERRUPTIBLE: c_int = bindings::TASK_UNINTERRUPTIBLE as c_int;
+/// Convenience constant for waking up tasks regardless of whether they are in interruptible or
+/// uninterruptible sleep.
+pub const TASK_NORMAL: c_uint = bindings::TASK_NORMAL as c_uint;
 /// Returns the currently running task.
 #[macro_export]
 ///
 /// All instances are valid tasks created by the C portion of the kernel.
 ///
-/// Instances of this type are always ref-counted, that is, a call to `get_task_struct` ensures
+/// Instances of this type are always refcounted, that is, a call to `get_task_struct` ensures
 /// that the allocation remains valid at least until the matching call to `put_task_struct`.
 ///
 /// # Examples
     /// Returns the group leader of the given task.
     pub fn group_leader(&self) -> &Task {
         // SAFETY: By the type invariant, we know that `self.0` is a valid task. Valid tasks always
-        // have a valid group_leader.
+        // have a valid `group_leader`.
         let ptr = unsafe { *ptr::addr_of!((*self.0.get()).group_leader) };
         // SAFETY: The lifetime of the returned task reference is tied to the lifetime of `self`,
     }
 }
-// SAFETY: The type invariants guarantee that `Task` is always ref-counted.
+// SAFETY: The type invariants guarantee that `Task` is always refcounted.
 unsafe impl crate::types::AlwaysRefCounted for Task {
     fn inc_ref(&self) {
         // SAFETY: The existence of a shared reference means that the refcount is nonzero.

rust/kernel/time.rs+20−0

@@ -0,0 +1,20 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2
	3	//! Time related primitives.
	4	//!
	5	//! This module contains the kernel APIs related to time and timers that
	6	//! have been ported or wrapped for usage by Rust code in the kernel.
	7
	8	/// The time unit of Linux kernel. One jiffy equals (1/HZ) second.
	9	pub type Jiffies = core::ffi::c_ulong;
	10
	11	/// The millisecond time unit.
	12	pub type Msecs = core::ffi::c_uint;
	13
	14	/// Converts milliseconds to jiffies.
	15	#[inline]
	16	pub fn msecs_to_jiffies(msecs: Msecs) -> Jiffies {
	17	// SAFETY: The `__msecs_to_jiffies` function is always safe to call no
	18	// matter what the argument is.
	19	unsafe { bindings::__msecs_to_jiffies(msecs) }
	20	}

rust/kernel/types.rs+22−0

     /// Additionally, all instances (if any) of values returned by [`ForeignOwnable::borrow`] for
     /// this object must have been dropped.
     unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self;
+    /// Tries to convert a foreign-owned object back to a Rust-owned one.
+    ///
+    /// A convenience wrapper over [`ForeignOwnable::from_foreign`] that returns [`None`] if `ptr`
+    /// is null.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must either be null or satisfy the safety requirements for
+    /// [`ForeignOwnable::from_foreign`].
+    unsafe fn try_from_foreign(ptr: *const core::ffi::c_void) -> Option<Self> {
+        if ptr.is_null() {
+            None
+        } else {
+            // SAFETY: Since `ptr` is not null here, then `ptr` satisfies the safety requirements
+            // of `from_foreign` given the safety requirements of this function.
+            unsafe { Some(Self::from_foreign(ptr)) }
+        }
+    }
 }
 impl<T: 'static> ForeignOwnable for Box<T> {
 ///
 /// In the example below, we have multiple exit paths and we want to log regardless of which one is
 /// taken:
+///
 /// ```
 /// # use kernel::types::ScopeGuard;
 /// fn example1(arg: bool) {
 ///
 /// In the example below, we want to log the same message on all early exits but a different one on
 /// the main exit path:
+///
 /// ```
 /// # use kernel::types::ScopeGuard;
 /// fn example2(arg: bool) {
 ///
 /// In the example below, we need a mutable object (the vector) to be accessible within the log
 /// function, so we wrap it in the [`ScopeGuard`]:
+///
 /// ```
 /// # use kernel::types::ScopeGuard;
 /// fn example3(arg: bool) -> Result {

rust/kernel/workqueue.rs+38−40

 //!
 //! # The raw API
 //!
-//! The raw API consists of the `RawWorkItem` trait, where the work item needs to provide an
+//! The raw API consists of the [`RawWorkItem`] trait, where the work item needs to provide an
 //! arbitrary function that knows how to enqueue the work item. It should usually not be used
 //! directly, but if you want to, you can use it without using the pieces from the safe API.
 //!
 //! # The safe API
 //!
-//! The safe API is used via the `Work` struct and `WorkItem` traits. Furthermore, it also includes
+//! The safe API is used via the [`Work`] struct and [`WorkItem`] traits. Furthermore, it also
-//! a trait called `WorkItemPointer`, which is usually not used directly by the user.
+//! includes a trait called [`WorkItemPointer`], which is usually not used directly by the user.
 //!
-//!  * The `Work` struct is the Rust wrapper for the C `work_struct` type.
+//!  * The [`Work`] struct is the Rust wrapper for the C `work_struct` type.
-//!  * The `WorkItem` trait is implemented for structs that can be enqueued to a workqueue.
+//!  * The [`WorkItem`] trait is implemented for structs that can be enqueued to a workqueue.
-//!  * The `WorkItemPointer` trait is implemented for the pointer type that points at a something
+//!  * The [`WorkItemPointer`] trait is implemented for the pointer type that points at a something
-//!    that implements `WorkItem`.
+//!    that implements [`WorkItem`].
 //!
 //! ## Example
 //!
 //! ```
 //! use kernel::prelude::*;
 //! use kernel::sync::Arc;
-//! use kernel::workqueue::{self, Work, WorkItem};
+//! use kernel::workqueue::{self, impl_has_work, new_work, Work, WorkItem};
-//! use kernel::{impl_has_work, new_work};
 //!
 //! #[pin_data]
 //! struct MyStruct {
 //! ```
 //! use kernel::prelude::*;
 //! use kernel::sync::Arc;
-//! use kernel::workqueue::{self, Work, WorkItem};
+//! use kernel::workqueue::{self, impl_has_work, new_work, Work, WorkItem};
-//! use kernel::{impl_has_work, new_work};
 //!
 //! #[pin_data]
 //! struct MyStruct {
         $crate::workqueue::Work::new($crate::optional_name!($($name)?), $crate::static_lock_class!())
     };
 }
+pub use new_work;
 /// A kernel work queue.
 ///
     /// # Safety
     ///
     /// The caller must ensure that the provided raw pointer is not dangling, that it points at a
-    /// valid workqueue, and that it remains valid until the end of 'a.
+    /// valid workqueue, and that it remains valid until the end of `'a`.
     pub unsafe fn from_raw<'a>(ptr: *const bindings::workqueue_struct) -> &'a Queue {
         // SAFETY: The `Queue` type is `#[repr(transparent)]`, so the pointer cast is valid. The
         // caller promises that the pointer is not dangling.
     }
 }
-/// A helper type used in `try_spawn`.
+/// A helper type used in [`try_spawn`].
+///
+/// [`try_spawn`]: Queue::try_spawn
 #[pin_data]
 struct ClosureWork<T> {
     #[pin]
 /// actual value of the id is not important as long as you use different ids for different fields
 /// of the same struct. (Fields of different structs need not use different ids.)
 ///
-/// Note that the id is used only to select the right method to call during compilation. It wont be
+/// Note that the id is used only to select the right method to call during compilation. It won't be
 /// part of the final executable.
 ///
 /// # Safety
 ///
-/// Implementers must ensure that any pointers passed to a `queue_work_on` closure by `__enqueue`
+/// Implementers must ensure that any pointers passed to a `queue_work_on` closure by [`__enqueue`]
 /// remain valid for the duration specified in the guarantees section of the documentation for
-/// `__enqueue`.
+/// [`__enqueue`].
+///
+/// [`__enqueue`]: RawWorkItem::__enqueue
 pub unsafe trait RawWorkItem<const ID: u64> {
     /// The return type of [`Queue::enqueue`].
     type EnqueueOutput;
 /// Defines the method that should be called directly when a work item is executed.
 ///
-/// This trait is implemented by `Pin<Box<T>>` and `Arc<T>`, and is mainly intended to be
+/// This trait is implemented by `Pin<Box<T>>` and [`Arc<T>`], and is mainly intended to be
 /// implemented for smart pointer types. For your own structs, you would implement [`WorkItem`]
-/// instead. The `run` method on this trait will usually just perform the appropriate
+/// instead. The [`run`] method on this trait will usually just perform the appropriate
-/// `container_of` translation and then call into the `run` method from the [`WorkItem`] trait.
+/// `container_of` translation and then call into the [`run`][WorkItem::run] method from the
+/// [`WorkItem`] trait.
 ///
 /// This trait is used when the `work_struct` field is defined using the [`Work`] helper.
 ///
     ///
     /// # Safety
     ///
-    /// The provided `work_struct` pointer must originate from a previous call to `__enqueue` where
+    /// The provided `work_struct` pointer must originate from a previous call to [`__enqueue`]
-    /// the `queue_work_on` closure returned true, and the pointer must still be valid.
+    /// where the `queue_work_on` closure returned true, and the pointer must still be valid.
+    ///
+    /// [`__enqueue`]: RawWorkItem::__enqueue
     unsafe extern "C" fn run(ptr: *mut bindings::work_struct);
 }
 /// Links for a work item.
 ///
-/// This struct contains a function pointer to the `run` function from the [`WorkItemPointer`]
+/// This struct contains a function pointer to the [`run`] function from the [`WorkItemPointer`]
 /// trait, and defines the linked list pointers necessary to enqueue a work item in a workqueue.
 ///
 /// Wraps the kernel's C `struct work_struct`.
 ///
 /// This is a helper type used to associate a `work_struct` with the [`WorkItem`] that uses it.
+///
+/// [`run`]: WorkItemPointer::run
 #[repr(transparent)]
 pub struct Work<T: ?Sized, const ID: u64 = 0> {
     work: Opaque<bindings::work_struct>,
 /// like this:
 ///
 /// ```no_run
-/// use kernel::impl_has_work;
 /// use kernel::prelude::*;
-/// use kernel::workqueue::Work;
+/// use kernel::workqueue::{impl_has_work, Work};
 ///
 /// struct MyWorkItem {
 ///     work_field: Work<MyWorkItem, 1>,
 /// }
 /// ```
 ///
-/// Note that since the `Work` type is annotated with an id, you can have several `work_struct`
+/// Note that since the [`Work`] type is annotated with an id, you can have several `work_struct`
 /// fields by using a different id for each one.
 ///
 /// # Safety
 ///
-/// The [`OFFSET`] constant must be the offset of a field in Self of type [`Work<T, ID>`]. The methods on
+/// The [`OFFSET`] constant must be the offset of a field in `Self` of type [`Work<T, ID>`]. The
-/// this trait must have exactly the behavior that the definitions given below have.
+/// methods on this trait must have exactly the behavior that the definitions given below have.
 ///
-/// [`Work<T, ID>`]: Work
 /// [`impl_has_work!`]: crate::impl_has_work
 /// [`OFFSET`]: HasWork::OFFSET
 pub unsafe trait HasWork<T, const ID: u64 = 0> {
     /// The offset of the [`Work<T, ID>`] field.
-    ///
-    /// [`Work<T, ID>`]: Work
     const OFFSET: usize;
     /// Returns the offset of the [`Work<T, ID>`] field.
     ///
-    /// This method exists because the [`OFFSET`] constant cannot be accessed if the type is not Sized.
+    /// This method exists because the [`OFFSET`] constant cannot be accessed if the type is not
+    /// [`Sized`].
     ///
-    /// [`Work<T, ID>`]: Work
     /// [`OFFSET`]: HasWork::OFFSET
     #[inline]
     fn get_work_offset(&self) -> usize {
     /// # Safety
     ///
     /// The provided pointer must point at a valid struct of type `Self`.
-    ///
-    /// [`Work<T, ID>`]: Work
     #[inline]
     unsafe fn raw_get_work(ptr: *mut Self) -> *mut Work<T, ID> {
         // SAFETY: The caller promises that the pointer is valid.
     /// # Safety
     ///
     /// The pointer must point at a [`Work<T, ID>`] field in a struct of type `Self`.
-    ///
-    /// [`Work<T, ID>`]: Work
     #[inline]
     unsafe fn work_container_of(ptr: *mut Work<T, ID>) -> *mut Self
     where
 /// # Examples
 ///
 /// ```
-/// use kernel::impl_has_work;
 /// use kernel::sync::Arc;
-/// use kernel::workqueue::{self, Work};
+/// use kernel::workqueue::{self, impl_has_work, Work};
 ///
 /// struct MyStruct {
 ///     work_field: Work<MyStruct, 17>,
 ///     impl HasWork<MyStruct, 17> for MyStruct { self.work_field }
 /// }
 /// ```
-///
-/// [`HasWork<T, ID>`]: HasWork
 #[macro_export]
 macro_rules! impl_has_work {
     ($(impl$(<$($implarg:ident),*>)?
         }
     )*};
 }
+pub use impl_has_work;
 impl_has_work! {
     impl<T> HasWork<Self> for ClosureWork<T> { self.work }

rust/macros/module.rs+6−1

             }};
             // Loadable modules need to export the `{{init,cleanup}}_module` identifiers.
+            /// # Safety
+            ///
+            /// This function must not be called after module initialization, because it may be
+            /// freed after that completes.
             #[cfg(MODULE)]
             #[doc(hidden)]
             #[no_mangle]
-            pub extern \"C\" fn init_module() -> core::ffi::c_int {{
+            #[link_section = \".init.text\"]
+            pub unsafe extern \"C\" fn init_module() -> core::ffi::c_int {{
                 __init()
             }}

scripts/Makefile.build+4−4

scripts/Makefile.host+1−1

@@ -156,7 +156,7 @@ quiet_cmd_host-rust = HOSTRUSTC $@ · 155 unmodified lines
156	cmd_host-rust = \	156	cmd_host-rust = \
157	$(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $<	157	$(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $<
158	$(host-rust): $(obj)/%: $(src)/%.rs FORCE	158	$(host-rust): $(obj)/%: $(src)/%.rs FORCE
159	$(call if_changed_dep,host-rust)	159	+$(call if_changed_dep,host-rust)
160		160
161	targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \	161	targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
162	$(host-cxxmulti) $(host-cxxobjs) $(host-rust)	162	$(host-cxxmulti) $(host-cxxobjs) $(host-rust)

scripts/min-tool-version.sh+1−1

@@ -33,7 +33,7 @@ llvm) · 32 unmodified lines
33	fi	33	fi
34	;;	34	;;
35	rustc)	35	rustc)
36	echo 1.74.1	36	echo 1.76.0
37	;;	37	;;
38	bindgen)	38	bindgen)
39	echo 0.65.1	39	echo 0.65.1

@@ -40,7 +40,7 @@ obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.o · 39 unmodified lines
40	ifdef CONFIG_RUST	40	ifdef CONFIG_RUST
41		41
42	# `$(rust_flags)` is passed in case the user added `--sysroot`.	42	# `$(rust_flags)` is passed in case the user added `--sysroot`.
43	rustc_sysroot := $(shell $(RUSTC) $(rust_flags) --print sysroot)	43	rustc_sysroot := $(shell MAKEFLAGS= $(RUSTC) $(rust_flags) --print sysroot)
44	rustc_host_target := $(shell $(RUSTC) --version --verbose \| grep -F 'host: ' \| cut -d' ' -f2)	44	rustc_host_target := $(shell $(RUSTC) --version --verbose \| grep -F 'host: ' \| cut -d' ' -f2)
45	RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library	45	RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library
46		46
@@ -108,14 +108,14 @@ rustdoc-macros: private rustdoc_host = yes · 61 unmodified lines
108	rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \	108	rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \
109	--extern proc_macro	109	--extern proc_macro
110	rustdoc-macros: $(src)/macros/lib.rs FORCE	110	rustdoc-macros: $(src)/macros/lib.rs FORCE
111	$(call if_changed,rustdoc)	111	+$(call if_changed,rustdoc)
112		112
113	rustdoc-core: private rustc_target_flags = $(core-cfgs)	113	rustdoc-core: private rustc_target_flags = $(core-cfgs)
114	rustdoc-core: $(RUST_LIB_SRC)/core/src/lib.rs FORCE	114	rustdoc-core: $(RUST_LIB_SRC)/core/src/lib.rs FORCE
115	$(call if_changed,rustdoc)	115	+$(call if_changed,rustdoc)
116		116
117	rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE	117	rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE
118	$(call if_changed,rustdoc)	118	+$(call if_changed,rustdoc)
119		119
120	# We need to allow `rustdoc::broken_intra_doc_links` because some	120	# We need to allow `rustdoc::broken_intra_doc_links` because some
121	# `no_global_oom_handling` functions refer to non-`no_global_oom_handling`	121	# `no_global_oom_handling` functions refer to non-`no_global_oom_handling`
@@ -124,7 +124,7 @@ rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE · 2 unmodified lines
124	rustdoc-alloc: private rustc_target_flags = $(alloc-cfgs) \	124	rustdoc-alloc: private rustc_target_flags = $(alloc-cfgs) \
125	-Arustdoc::broken_intra_doc_links	125	-Arustdoc::broken_intra_doc_links
126	rustdoc-alloc: $(src)/alloc/lib.rs rustdoc-core rustdoc-compiler_builtins FORCE	126	rustdoc-alloc: $(src)/alloc/lib.rs rustdoc-core rustdoc-compiler_builtins FORCE
127	$(call if_changed,rustdoc)	127	+$(call if_changed,rustdoc)
128		128
129	rustdoc-kernel: private rustc_target_flags = --extern alloc \	129	rustdoc-kernel: private rustc_target_flags = --extern alloc \
130	--extern build_error --extern macros=$(objtree)/$(obj)/libmacros.so \	130	--extern build_error --extern macros=$(objtree)/$(obj)/libmacros.so \
@@ -132,7 +132,7 @@ rustdoc-kernel: private rustc_target_flags = --extern alloc \ · 1 unmodified line
132	rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-macros \	132	rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-macros \
133	rustdoc-compiler_builtins rustdoc-alloc $(obj)/libmacros.so \	133	rustdoc-compiler_builtins rustdoc-alloc $(obj)/libmacros.so \
134	$(obj)/bindings.o FORCE	134	$(obj)/bindings.o FORCE
135	$(call if_changed,rustdoc)	135	+$(call if_changed,rustdoc)
136		136
137	quiet_cmd_rustc_test_library = RUSTC TL $<	137	quiet_cmd_rustc_test_library = RUSTC TL $<
138	cmd_rustc_test_library = \	138	cmd_rustc_test_library = \
@@ -146,18 +146,18 @@ quiet_cmd_rustc_test_library = RUSTC TL $< · 7 unmodified lines
146	--crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<	146	--crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<
147		147
148	rusttestlib-build_error: $(src)/build_error.rs rusttest-prepare FORCE	148	rusttestlib-build_error: $(src)/build_error.rs rusttest-prepare FORCE
149	$(call if_changed,rustc_test_library)	149	+$(call if_changed,rustc_test_library)
150		150
151	rusttestlib-macros: private rustc_target_flags = --extern proc_macro	151	rusttestlib-macros: private rustc_target_flags = --extern proc_macro
152	rusttestlib-macros: private rustc_test_library_proc = yes	152	rusttestlib-macros: private rustc_test_library_proc = yes
153	rusttestlib-macros: $(src)/macros/lib.rs rusttest-prepare FORCE	153	rusttestlib-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
154	$(call if_changed,rustc_test_library)	154	+$(call if_changed,rustc_test_library)
155		155
156	rusttestlib-bindings: $(src)/bindings/lib.rs rusttest-prepare FORCE	156	rusttestlib-bindings: $(src)/bindings/lib.rs rusttest-prepare FORCE
157	$(call if_changed,rustc_test_library)	157	+$(call if_changed,rustc_test_library)
158		158
159	rusttestlib-uapi: $(src)/uapi/lib.rs rusttest-prepare FORCE	159	rusttestlib-uapi: $(src)/uapi/lib.rs rusttest-prepare FORCE
160	$(call if_changed,rustc_test_library)	160	+$(call if_changed,rustc_test_library)
161		161
162	quiet_cmd_rustdoc_test = RUSTDOC T $<	162	quiet_cmd_rustdoc_test = RUSTDOC T $<
163	cmd_rustdoc_test = \	163	cmd_rustdoc_test = \
@@ -189,7 +189,7 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $< · 25 unmodified lines
189	$(src)/kernel/lib.rs $(obj)/kernel.o \	189	$(src)/kernel/lib.rs $(obj)/kernel.o \
190	$(objtree)/scripts/rustdoc_test_builder \	190	$(objtree)/scripts/rustdoc_test_builder \
191	$(objtree)/scripts/rustdoc_test_gen FORCE	191	$(objtree)/scripts/rustdoc_test_gen FORCE
192	$(call if_changed,rustdoc_test_kernel)	192	+$(call if_changed,rustdoc_test_kernel)
193		193
194	# We cannot use `-Zpanic-abort-tests` because some tests are dynamic,	194	# We cannot use `-Zpanic-abort-tests` because some tests are dynamic,
195	# so for the moment we skip `-Cpanic=abort`.	195	# so for the moment we skip `-Cpanic=abort`.
@@ -254,21 +254,21 @@ quiet_cmd_rustsysroot = RUSTSYSROOT · 58 unmodified lines
254	$(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib	254	$(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib
255		255
256	rusttest-prepare: FORCE	256	rusttest-prepare: FORCE
257	$(call if_changed,rustsysroot)	257	+$(call if_changed,rustsysroot)
258		258
259	rusttest-macros: private rustc_target_flags = --extern proc_macro	259	rusttest-macros: private rustc_target_flags = --extern proc_macro
260	rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro	260	rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro
261	rusttest-macros: $(src)/macros/lib.rs rusttest-prepare FORCE	261	rusttest-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
262	$(call if_changed,rustc_test)	262	+$(call if_changed,rustc_test)
263	$(call if_changed,rustdoc_test)	263	+$(call if_changed,rustdoc_test)
264		264
265	rusttest-kernel: private rustc_target_flags = --extern alloc \	265	rusttest-kernel: private rustc_target_flags = --extern alloc \
266	--extern build_error --extern macros --extern bindings --extern uapi	266	--extern build_error --extern macros --extern bindings --extern uapi
267	rusttest-kernel: $(src)/kernel/lib.rs rusttest-prepare \	267	rusttest-kernel: $(src)/kernel/lib.rs rusttest-prepare \
268	rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \	268	rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
269	rusttestlib-uapi FORCE	269	rusttestlib-uapi FORCE
270	$(call if_changed,rustc_test)	270	+$(call if_changed,rustc_test)
271	$(call if_changed,rustc_test_library)	271	+$(call if_changed,rustc_test_library)
272		272
273	ifdef CONFIG_CC_IS_CLANG	273	ifdef CONFIG_CC_IS_CLANG
274	bindgen_c_flags = $(c_flags)	274	bindgen_c_flags = $(c_flags)
@@ -396,7 +396,7 @@ quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ · 121 unmodified lines
396	# Therefore, to get `libmacros.so` automatically recompiled when the compiler	396	# Therefore, to get `libmacros.so` automatically recompiled when the compiler
397	# version changes, we add `core.o` as a dependency (even if it is not needed).	397	# version changes, we add `core.o` as a dependency (even if it is not needed).
398	$(obj)/libmacros.so: $(src)/macros/lib.rs $(obj)/core.o FORCE	398	$(obj)/libmacros.so: $(src)/macros/lib.rs $(obj)/core.o FORCE
399	$(call if_changed_dep,rustc_procmacro)	399	+$(call if_changed_dep,rustc_procmacro)
400		400
401	quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@	401	quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@
402	cmd_rustc_library = \	402	cmd_rustc_library = \
@@ -435,36 +435,36 @@ $(obj)/core.o: private skip_flags = -Dunreachable_pub · 32 unmodified lines
435	$(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))	435	$(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))
436	$(obj)/core.o: private rustc_target_flags = $(core-cfgs)	436	$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
437	$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs scripts/target.json FORCE	437	$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs scripts/target.json FORCE
438	$(call if_changed_dep,rustc_library)	438	+$(call if_changed_dep,rustc_library)
439		439
440	$(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*'	440	$(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*'
441	$(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE	441	$(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE
442	$(call if_changed_dep,rustc_library)	442	+$(call if_changed_dep,rustc_library)
443		443
444	$(obj)/alloc.o: private skip_clippy = 1	444	$(obj)/alloc.o: private skip_clippy = 1
445	$(obj)/alloc.o: private skip_flags = -Dunreachable_pub	445	$(obj)/alloc.o: private skip_flags = -Dunreachable_pub
446	$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)	446	$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)
447	$(obj)/alloc.o: $(src)/alloc/lib.rs $(obj)/compiler_builtins.o FORCE	447	$(obj)/alloc.o: $(src)/alloc/lib.rs $(obj)/compiler_builtins.o FORCE
448	$(call if_changed_dep,rustc_library)	448	+$(call if_changed_dep,rustc_library)
449		449
450	$(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE	450	$(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE
451	$(call if_changed_dep,rustc_library)	451	+$(call if_changed_dep,rustc_library)
452		452
453	$(obj)/bindings.o: $(src)/bindings/lib.rs \	453	$(obj)/bindings.o: $(src)/bindings/lib.rs \
454	$(obj)/compiler_builtins.o \	454	$(obj)/compiler_builtins.o \
455	$(obj)/bindings/bindings_generated.rs \	455	$(obj)/bindings/bindings_generated.rs \
456	$(obj)/bindings/bindings_helpers_generated.rs FORCE	456	$(obj)/bindings/bindings_helpers_generated.rs FORCE
457	$(call if_changed_dep,rustc_library)	457	+$(call if_changed_dep,rustc_library)
458		458
459	$(obj)/uapi.o: $(src)/uapi/lib.rs \	459	$(obj)/uapi.o: $(src)/uapi/lib.rs \
460	$(obj)/compiler_builtins.o \	460	$(obj)/compiler_builtins.o \
461	$(obj)/uapi/uapi_generated.rs FORCE	461	$(obj)/uapi/uapi_generated.rs FORCE
462	$(call if_changed_dep,rustc_library)	462	+$(call if_changed_dep,rustc_library)
463		463
464	$(obj)/kernel.o: private rustc_target_flags = --extern alloc \	464	$(obj)/kernel.o: private rustc_target_flags = --extern alloc \
465	--extern build_error --extern macros --extern bindings --extern uapi	465	--extern build_error --extern macros --extern bindings --extern uapi
466	$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \	466	$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \
467	$(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE	467	$(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE
468	$(call if_changed_dep,rustc_library)	468	+$(call if_changed_dep,rustc_library)
469		469
470	endif # CONFIG_RUST	470	endif # CONFIG_RUST

@@ -290,7 +290,7 @@ quiet_cmd_rustc_o_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ · 289 unmodified lines
290	cmd_rustc_o_rs = $(rust_common_cmd) --emit=obj=$@ $<	290	cmd_rustc_o_rs = $(rust_common_cmd) --emit=obj=$@ $<
291		291
292	$(obj)/%.o: $(src)/%.rs FORCE	292	$(obj)/%.o: $(src)/%.rs FORCE
293	$(call if_changed_dep,rustc_o_rs)	293	+$(call if_changed_dep,rustc_o_rs)
294		294
295	quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@	295	quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
296	cmd_rustc_rsi_rs = \	296	cmd_rustc_rsi_rs = \
@@ -298,19 +298,19 @@ quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ · 1 unmodified line
298	command -v $(RUSTFMT) >/dev/null && $(RUSTFMT) $@	298	command -v $(RUSTFMT) >/dev/null && $(RUSTFMT) $@
299		299
300	$(obj)/%.rsi: $(src)/%.rs FORCE	300	$(obj)/%.rsi: $(src)/%.rs FORCE
301	$(call if_changed_dep,rustc_rsi_rs)	301	+$(call if_changed_dep,rustc_rsi_rs)
302		302
303	quiet_cmd_rustc_s_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@	303	quiet_cmd_rustc_s_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
304	cmd_rustc_s_rs = $(rust_common_cmd) --emit=asm=$@ $<	304	cmd_rustc_s_rs = $(rust_common_cmd) --emit=asm=$@ $<
305		305
306	$(obj)/%.s: $(src)/%.rs FORCE	306	$(obj)/%.s: $(src)/%.rs FORCE
307	$(call if_changed_dep,rustc_s_rs)	307	+$(call if_changed_dep,rustc_s_rs)
308		308
309	quiet_cmd_rustc_ll_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@	309	quiet_cmd_rustc_ll_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
310	cmd_rustc_ll_rs = $(rust_common_cmd) --emit=llvm-ir=$@ $<	310	cmd_rustc_ll_rs = $(rust_common_cmd) --emit=llvm-ir=$@ $<
311		311
312	$(obj)/%.ll: $(src)/%.rs FORCE	312	$(obj)/%.ll: $(src)/%.rs FORCE
313	$(call if_changed_dep,rustc_ll_rs)	313	+$(call if_changed_dep,rustc_ll_rs)
314		314
315	# Compile assembler sources (.S)	315	# Compile assembler sources (.S)
316	# ---------------------------------------------------------------------------	316	# ---------------------------------------------------------------------------