Merge tag 'rust-6.9' of https://github.com/Rust-for-Linux/linux

Pull Rust updates from Miguel Ojeda: "Another routine one in terms of features. We got two version upgrades this time, but in terms of lines, 'alloc' changes are not very large. Toolchain and infrastructure: - Upgrade to Rust 1.76.0 This time around, due to how the kernel and Rust schedules have aligned, there are two upgrades in fact. These allow us to remove two more unstable features ('const_maybe_uninit_zeroed' and 'ptr_metadata') from the list, among other improvements - Mark 'rustc' (and others) invocations as recursive, which fixes a new warning and prepares us for the future in case we eventually take advantage of the Make jobserver 'kernel' crate: - Add the 'container_of!' macro - Stop using the unstable 'ptr_metadata' feature by employing the now stable 'byte_sub' method to implement 'Arc::from_raw()' - Add the 'time' module with a 'msecs_to_jiffies()' conversion function to begin with, to be used by Rust Binder - Add 'notify_sync()' and 'wait_interruptible_timeout()' methods to 'CondVar', to be used by Rust Binder - Update integer types for 'CondVar' - Rename 'wait_list' field to 'wait_queue_head' in 'CondVar' - Implement 'Display' and 'Debug' for 'BStr' - Add the 'try_from_foreign()' method to the 'ForeignOwnable' trait - Add reexports for macros so that they can be used from the right module (in addition to the root) - A series of code documentation improvements, including adding intra-doc links, consistency improvements, typo fixes... 'macros' crate: - Place generated 'init_module()' function in '.init.text' Documentation: - Add documentation on Rust doctests and how they work" * tag 'rust-6.9' of https://github.com/Rust-for-Linux/linux: (29 commits) rust: upgrade to Rust 1.76.0 kbuild: mark `rustc` (and others) invocations as recursive rust: add `container_of!` macro rust: str: implement `Display` and `Debug` for `BStr` rust: module: place generated init_module() function in .init.text rust: types: add `try_from_foreign()` method docs: rust: Add description of Rust documentation test as KUnit ones docs: rust: Move testing to a separate page rust: kernel: stop using ptr_metadata feature rust: kernel: add reexports for macros rust: locked_by: shorten doclink preview rust: kernel: remove unneeded doclink targets rust: kernel: add doclinks rust: kernel: add blank lines in front of code blocks rust: kernel: mark code fragments in docs with backticks rust: kernel: unify spelling of refcount in docs rust: str: move SAFETY comment in front of unsafe block rust: str: use `NUL` instead of 0 in doc comments rust: kernel: add srctree-relative doclinks rust: ioctl: end top-level module docs with full stop ...

Linus Torvalds committed 2 years ago

Commit 8ede8422 parents 5a2a15c 768409c35 files · +809 −252Browse files

Unified · Split

35 files

Documentation/process/changes.rst +1 −1
Documentation/rust/general-information.rst +0 −24
Documentation/rust/index.rst +1 −0
Documentation/rust/testing.rst +135 −0
Makefile +2 −2
rust/Makefile +24 −24
rust/alloc/alloc.rs +11 −1
rust/alloc/boxed.rs +23 −11
rust/alloc/collections/mod.rs +1 −0
rust/alloc/lib.rs +5 −4
rust/alloc/raw_vec.rs +57 −20
rust/alloc/vec/into_iter.rs +11 −5
rust/alloc/vec/mod.rs +61 −20
rust/bindings/bindings_helper.h +3 −2
rust/kernel/allocator.rs +1 −1
rust/kernel/error.rs +3 −7
rust/kernel/init.rs +11 −11
rust/kernel/ioctl.rs +3 −3
rust/kernel/lib.rs +34 −3
rust/kernel/str.rs +182 −11
rust/kernel/sync.rs +3 −2
rust/kernel/sync/arc.rs +14 −16
rust/kernel/sync/condvar.rs +88 −22
rust/kernel/sync/lock.rs +13 −6
rust/kernel/sync/lock/mutex.rs +2 −1
rust/kernel/sync/lock/spinlock.rs +3 −2
rust/kernel/sync/locked_by.rs +5 −2
rust/kernel/task.rs +20 −4
rust/kernel/time.rs +20 −0
rust/kernel/types.rs +22 −0
rust/kernel/workqueue.rs +38 −40
rust/macros/module.rs +6 −1
scripts/Makefile.build +4 −4
scripts/Makefile.host +1 −1
scripts/min-tool-version.sh +1 −1

Documentation/process/changes.rst+1−1

@@ -31,7 +31,7 @@ you probably needn't concern yourself with pcmciautils. · 30 unmodified lines
31	31	====================== =============== ========================================
32	32	GNU C 5.1 gcc --version
33	33	Clang/LLVM (optional) 11.0.0 clang --version
34		−Rust (optional) 1.74.1 rustc --version
	34	+Rust (optional) 1.76.0 rustc --version
35	35	bindgen (optional) 0.65.1 bindgen --version
36	36	GNU make 3.82 make --version
37	37	bash 4.2 bash --version

Documentation/rust/general-information.rst+0−24

@@ -77,27 +77,3 @@ configuration: · 76 unmodified lines
77	77	#[cfg(CONFIG_X="y")] // Enabled as a built-in (`y`)
78	78	#[cfg(CONFIG_X="m")] // Enabled as a module (`m`)
79	79	#[cfg(not(CONFIG_X))] // Disabled
80		−
81		−
82		−Testing
83		−-------
84		−
85		−There are the tests that come from the examples in the Rust documentation
86		−and get transformed into KUnit tests. These can be run via KUnit. For example
87		−via ``kunit_tool`` (``kunit.py``) on the command line::
88		−
89		− ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y
90		−
91		−Alternatively, KUnit can run them as kernel built-in at boot. Refer to
92		−Documentation/dev-tools/kunit/index.rst for the general KUnit documentation
93		−and Documentation/dev-tools/kunit/architecture.rst for the details of kernel
94		−built-in vs. command line testing.
95		−
96		−Additionally, there are the ``#[test]`` tests. These can be run using
97		−the ``rusttest`` Make target::
98		−
99		− make LLVM=1 rusttest
100		−
101		−This requires the kernel ``.config`` and downloads external repositories.
102		−It runs the ``#[test]`` tests on the host (currently) and thus is fairly
103		−limited in what these tests can test.

Documentation/rust/index.rst+1−0

@@ -40,6 +40,7 @@ configurations. · 39 unmodified lines
40	40	general-information
41	41	coding-guidelines
42	42	arch-support
	43	+ testing
43	44
44	45	.. only:: subproject and html
45	46

Documentation/rust/testing.rst+135−0

@@ -0,0 +1,135 @@
	1	+.. SPDX-License-Identifier: GPL-2.0
	2	+
	3	+Testing
	4	+=======
	5	+
	6	+This document contains useful information how to test the Rust code in the
	7	+kernel.
	8	+
	9	+There are two sorts of tests:
	10	+
	11	+- The KUnit tests.
	12	+- The ``#[test]`` tests.
	13	+
	14	+The KUnit tests
	15	+---------------
	16	+
	17	+These are the tests that come from the examples in the Rust documentation. They
	18	+get transformed into KUnit tests.
	19	+
	20	+Usage
	21	+*****
	22	+
	23	+These tests can be run via KUnit. For example via ``kunit_tool`` (``kunit.py``)
	24	+on the command line::
	25	+
	26	+ ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y
	27	+
	28	+Alternatively, KUnit can run them as kernel built-in at boot. Refer to
	29	+Documentation/dev-tools/kunit/index.rst for the general KUnit documentation
	30	+and Documentation/dev-tools/kunit/architecture.rst for the details of kernel
	31	+built-in vs. command line testing.
	32	+
	33	+To use these KUnit doctests, the following must be enabled::
	34	+
	35	+ CONFIG_KUNIT
	36	+ Kernel hacking -> Kernel Testing and Coverage -> KUnit - Enable support for unit tests
	37	+ CONFIG_RUST_KERNEL_DOCTESTS
	38	+ Kernel hacking -> Rust hacking -> Doctests for the `kernel` crate
	39	+
	40	+in the kernel config system.
	41	+
	42	+KUnit tests are documentation tests
	43	+***********************************
	44	+
	45	+These documentation tests are typically examples of usage of any item (e.g.
	46	+function, struct, module...).
	47	+
	48	+They are very convenient because they are just written alongside the
	49	+documentation. For instance:
	50	+
	51	+.. code-block:: rust
	52	+
	53	+ /// Sums two numbers.
	54	+ ///
	55	+ /// ```
	56	+ /// assert_eq!(mymod::f(10, 20), 30);
	57	+ /// ```
	58	+ pub fn f(a: i32, b: i32) -> i32 {
	59	+ a + b
	60	+ }
	61	+
	62	+In userspace, the tests are collected and run via ``rustdoc``. Using the tool
	63	+as-is would be useful already, since it allows verifying that examples compile
	64	+(thus enforcing they are kept in sync with the code they document) and as well
	65	+as running those that do not depend on in-kernel APIs.
	66	+
	67	+For the kernel, however, these tests get transformed into KUnit test suites.
	68	+This means that doctests get compiled as Rust kernel objects, allowing them to
	69	+run against a built kernel.
	70	+
	71	+A benefit of this KUnit integration is that Rust doctests get to reuse existing
	72	+testing facilities. For instance, the kernel log would look like::
	73	+
	74	+ KTAP version 1
	75	+ 1..1
	76	+ KTAP version 1
	77	+ # Subtest: rust_doctests_kernel
	78	+ 1..59
	79	+ # rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
	80	+ ok 1 rust_doctest_kernel_build_assert_rs_0
	81	+ # rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
	82	+ ok 2 rust_doctest_kernel_build_assert_rs_1
	83	+ # rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
	84	+ ok 3 rust_doctest_kernel_init_rs_0
	85	+ ...
	86	+ # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
	87	+ ok 59 rust_doctest_kernel_types_rs_2
	88	+ # rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
	89	+ # Totals: pass:59 fail:0 skip:0 total:59
	90	+ ok 1 rust_doctests_kernel
	91	+
	92	+Tests using the `? <https://doc.rust-lang.org/reference/expressions/operator-expr.html#the-question-mark-operator>`_
	93	+operator are also supported as usual, e.g.:
	94	+
	95	+.. code-block:: rust
	96	+
	97	+ /// ```
	98	+ /// # use kernel::{spawn_work_item, workqueue};
	99	+ /// spawn_work_item!(workqueue::system(), \|\| pr_info!("x"))?;
	100	+ /// # Ok::<(), Error>(())
	101	+ /// ```
	102	+
	103	+The tests are also compiled with Clippy under ``CLIPPY=1``, just like normal
	104	+code, thus also benefitting from extra linting.
	105	+
	106	+In order for developers to easily see which line of doctest code caused a
	107	+failure, a KTAP diagnostic line is printed to the log. This contains the
	108	+location (file and line) of the original test (i.e. instead of the location in
	109	+the generated Rust file)::
	110	+
	111	+ # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
	112	+
	113	+Rust tests appear to assert using the usual ``assert!`` and ``assert_eq!``
	114	+macros from the Rust standard library (``core``). We provide a custom version
	115	+that forwards the call to KUnit instead. Importantly, these macros do not
	116	+require passing context, unlike those for KUnit testing (i.e.
	117	+``struct kunit *``). This makes them easier to use, and readers of the
	118	+documentation do not need to care about which testing framework is used. In
	119	+addition, it may allow us to test third-party code more easily in the future.
	120	+
	121	+A current limitation is that KUnit does not support assertions in other tasks.
	122	+Thus, we presently simply print an error to the kernel log if an assertion
	123	+actually failed. Additionally, doctests are not run for nonpublic functions.
	124	+
	125	+The ``#[test]`` tests
	126	+---------------------
	127	+
	128	+Additionally, there are the ``#[test]`` tests. These can be run using the
	129	+``rusttest`` Make target::
	130	+
	131	+ make LLVM=1 rusttest
	132	+
	133	+This requires the kernel ``.config`` and downloads external repositories. It
	134	+runs the ``#[test]`` tests on the host (currently) and thus is fairly limited in
	135	+what these tests can test.

Makefile+2−2

@@ -1201,7 +1201,7 @@ prepare0: archprepare · 1200 unmodified lines
1201	1201	# All the preparing..
1202	1202	prepare: prepare0
1203	1203	ifdef CONFIG_RUST
1204		− $(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh
	1204	+ +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh
1205	1205	$(Q)$(MAKE) $(build)=rust
1206	1206	endif
1207	1207
@@ -1711,7 +1711,7 @@ $(DOC_TARGETS): · 503 unmodified lines
1711	1711	# "Is Rust available?" target
1712	1712	PHONY += rustavailable
1713	1713	rustavailable:
1714		− $(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh && echo "Rust is available!"
	1714	+ +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh && echo "Rust is available!"
1715	1715
1716	1716	# Documentation target
1717	1717	#

rust/Makefile+24−24

@@ -40,7 +40,7 @@ obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.o · 39 unmodified lines
40	40	ifdef CONFIG_RUST
41	41
42	42	# `$(rust_flags)` is passed in case the user added `--sysroot`.
43		−rustc_sysroot := $(shell $(RUSTC) $(rust_flags) --print sysroot)
	43	+rustc_sysroot := $(shell MAKEFLAGS= $(RUSTC) $(rust_flags) --print sysroot)
44	44	rustc_host_target := $(shell $(RUSTC) --version --verbose \| grep -F 'host: ' \| cut -d' ' -f2)
45	45	RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library
46	46
@@ -108,14 +108,14 @@ rustdoc-macros: private rustdoc_host = yes · 61 unmodified lines
108	108	rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \
109	109	--extern proc_macro
110	110	rustdoc-macros: $(src)/macros/lib.rs FORCE
111		− $(call if_changed,rustdoc)
	111	+ +$(call if_changed,rustdoc)
112	112
113	113	rustdoc-core: private rustc_target_flags = $(core-cfgs)
114	114	rustdoc-core: $(RUST_LIB_SRC)/core/src/lib.rs FORCE
115		− $(call if_changed,rustdoc)
	115	+ +$(call if_changed,rustdoc)
116	116
117	117	rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE
118		− $(call if_changed,rustdoc)
	118	+ +$(call if_changed,rustdoc)
119	119
120	120	# We need to allow `rustdoc::broken_intra_doc_links` because some
121	121	# `no_global_oom_handling` functions refer to non-`no_global_oom_handling`
@@ -124,7 +124,7 @@ rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE · 2 unmodified lines
124	124	rustdoc-alloc: private rustc_target_flags = $(alloc-cfgs) \
125	125	-Arustdoc::broken_intra_doc_links
126	126	rustdoc-alloc: $(src)/alloc/lib.rs rustdoc-core rustdoc-compiler_builtins FORCE
127		− $(call if_changed,rustdoc)
	127	+ +$(call if_changed,rustdoc)
128	128
129	129	rustdoc-kernel: private rustc_target_flags = --extern alloc \
130	130	--extern build_error --extern macros=$(objtree)/$(obj)/libmacros.so \
@@ -132,7 +132,7 @@ rustdoc-kernel: private rustc_target_flags = --extern alloc \ · 1 unmodified line
132	132	rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-macros \
133	133	rustdoc-compiler_builtins rustdoc-alloc $(obj)/libmacros.so \
134	134	$(obj)/bindings.o FORCE
135		− $(call if_changed,rustdoc)
	135	+ +$(call if_changed,rustdoc)
136	136
137	137	quiet_cmd_rustc_test_library = RUSTC TL $<
138	138	cmd_rustc_test_library = \
@@ -146,18 +146,18 @@ quiet_cmd_rustc_test_library = RUSTC TL $< · 7 unmodified lines
146	146	--crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<
147	147
148	148	rusttestlib-build_error: $(src)/build_error.rs rusttest-prepare FORCE
149		− $(call if_changed,rustc_test_library)
	149	+ +$(call if_changed,rustc_test_library)
150	150
151	151	rusttestlib-macros: private rustc_target_flags = --extern proc_macro
152	152	rusttestlib-macros: private rustc_test_library_proc = yes
153	153	rusttestlib-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
154		− $(call if_changed,rustc_test_library)
	154	+ +$(call if_changed,rustc_test_library)
155	155
156	156	rusttestlib-bindings: $(src)/bindings/lib.rs rusttest-prepare FORCE
157		− $(call if_changed,rustc_test_library)
	157	+ +$(call if_changed,rustc_test_library)
158	158
159	159	rusttestlib-uapi: $(src)/uapi/lib.rs rusttest-prepare FORCE
160		− $(call if_changed,rustc_test_library)
	160	+ +$(call if_changed,rustc_test_library)
161	161
162	162	quiet_cmd_rustdoc_test = RUSTDOC T $<
163	163	cmd_rustdoc_test = \
@@ -189,7 +189,7 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $< · 25 unmodified lines
189	189	$(src)/kernel/lib.rs $(obj)/kernel.o \
190	190	$(objtree)/scripts/rustdoc_test_builder \
191	191	$(objtree)/scripts/rustdoc_test_gen FORCE
192		− $(call if_changed,rustdoc_test_kernel)
	192	+ +$(call if_changed,rustdoc_test_kernel)
193	193
194	194	# We cannot use `-Zpanic-abort-tests` because some tests are dynamic,
195	195	# so for the moment we skip `-Cpanic=abort`.
@@ -254,21 +254,21 @@ quiet_cmd_rustsysroot = RUSTSYSROOT · 58 unmodified lines
254	254	$(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib
255	255
256	256	rusttest-prepare: FORCE
257		− $(call if_changed,rustsysroot)
	257	+ +$(call if_changed,rustsysroot)
258	258
259	259	rusttest-macros: private rustc_target_flags = --extern proc_macro
260	260	rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro
261	261	rusttest-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
262		− $(call if_changed,rustc_test)
263		− $(call if_changed,rustdoc_test)
	262	+ +$(call if_changed,rustc_test)
	263	+ +$(call if_changed,rustdoc_test)
264	264
265	265	rusttest-kernel: private rustc_target_flags = --extern alloc \
266	266	--extern build_error --extern macros --extern bindings --extern uapi
267	267	rusttest-kernel: $(src)/kernel/lib.rs rusttest-prepare \
268	268	rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
269	269	rusttestlib-uapi FORCE
270		− $(call if_changed,rustc_test)
271		− $(call if_changed,rustc_test_library)
	270	+ +$(call if_changed,rustc_test)
	271	+ +$(call if_changed,rustc_test_library)
272	272
273	273	ifdef CONFIG_CC_IS_CLANG
274	274	bindgen_c_flags = $(c_flags)
@@ -396,7 +396,7 @@ quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ · 121 unmodified lines
396	396	# Therefore, to get `libmacros.so` automatically recompiled when the compiler
397	397	# version changes, we add `core.o` as a dependency (even if it is not needed).
398	398	$(obj)/libmacros.so: $(src)/macros/lib.rs $(obj)/core.o FORCE
399		− $(call if_changed_dep,rustc_procmacro)
	399	+ +$(call if_changed_dep,rustc_procmacro)
400	400
401	401	quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@
402	402	cmd_rustc_library = \
@@ -435,36 +435,36 @@ $(obj)/core.o: private skip_flags = -Dunreachable_pub · 32 unmodified lines
435	435	$(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))
436	436	$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
437	437	$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs scripts/target.json FORCE
438		− $(call if_changed_dep,rustc_library)
	438	+ +$(call if_changed_dep,rustc_library)
439	439
440	440	$(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*'
441	441	$(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE
442		− $(call if_changed_dep,rustc_library)
	442	+ +$(call if_changed_dep,rustc_library)
443	443
444	444	$(obj)/alloc.o: private skip_clippy = 1
445	445	$(obj)/alloc.o: private skip_flags = -Dunreachable_pub
446	446	$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)
447	447	$(obj)/alloc.o: $(src)/alloc/lib.rs $(obj)/compiler_builtins.o FORCE
448		− $(call if_changed_dep,rustc_library)
	448	+ +$(call if_changed_dep,rustc_library)
449	449
450	450	$(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE
451		− $(call if_changed_dep,rustc_library)
	451	+ +$(call if_changed_dep,rustc_library)
452	452
453	453	$(obj)/bindings.o: $(src)/bindings/lib.rs \
454	454	$(obj)/compiler_builtins.o \
455	455	$(obj)/bindings/bindings_generated.rs \
456	456	$(obj)/bindings/bindings_helpers_generated.rs FORCE
457		− $(call if_changed_dep,rustc_library)
	457	+ +$(call if_changed_dep,rustc_library)
458	458
459	459	$(obj)/uapi.o: $(src)/uapi/lib.rs \
460	460	$(obj)/compiler_builtins.o \
461	461	$(obj)/uapi/uapi_generated.rs FORCE
462		− $(call if_changed_dep,rustc_library)
	462	+ +$(call if_changed_dep,rustc_library)
463	463
464	464	$(obj)/kernel.o: private rustc_target_flags = --extern alloc \
465	465	--extern build_error --extern macros --extern bindings --extern uapi
466	466	$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \
467	467	$(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE
468		− $(call if_changed_dep,rustc_library)
	468	+ +$(call if_changed_dep,rustc_library)
469	469
470	470	endif # CONFIG_RUST

rust/alloc/alloc.rs+11−1

@@ -379,13 +379,20 @@ pub const fn handle_alloc_error(layout: Layout) -> ! { · 378 unmodified lines
379	379	panic!("allocation failed");
380	380	}
381	381
	382	+ #[inline]
382	383	fn rt_error(layout: Layout) -> ! {
383	384	unsafe {
384	385	__rust_alloc_error_handler(layout.size(), layout.align());
385	386	}
386	387	}
387	388
388		− unsafe { core::intrinsics::const_eval_select((layout,), ct_error, rt_error) }
	389	+ #[cfg(not(feature = "panic_immediate_abort"))]
	390	+ unsafe {
	391	+ core::intrinsics::const_eval_select((layout,), ct_error, rt_error)
	392	+ }
	393	+
	394	+ #[cfg(feature = "panic_immediate_abort")]
	395	+ ct_error(layout)
389	396	}
390	397
391	398	// For alloc test `std::alloc::handle_alloc_error` can be used directly.
@@ -418,12 +425,14 @@ pub mod __alloc_error_handler { · 26 unmodified lines
418	425	}
419	426	}
420	427
	428	+#[cfg(not(no_global_oom_handling))]
421	429	/// Specialize clones into pre-allocated, uninitialized memory.
422	430	/// Used by `Box::clone` and `Rc`/`Arc::make_mut`.
423	431	pub(crate) trait WriteCloneIntoRaw: Sized {
424	432	unsafe fn write_clone_into_raw(&self, target: *mut Self);
425	433	}
426	434
	435	+#[cfg(not(no_global_oom_handling))]
427	436	impl<T: Clone> WriteCloneIntoRaw for T {
428	437	#[inline]
429	438	default unsafe fn write_clone_into_raw(&self, target: *mut Self) {
@@ -433,6 +442,7 @@ impl<T: Clone> WriteCloneIntoRaw for T { · 3 unmodified lines
433	442	}
434	443	}
435	444
	445	+#[cfg(not(no_global_oom_handling))]
436	446	impl<T: Copy> WriteCloneIntoRaw for T {
437	447	#[inline]
438	448	unsafe fn write_clone_into_raw(&self, target: *mut Self) {

rust/alloc/boxed.rs+23−11

@@ -161,7 +161,7 @@ use core::marker::Tuple; · 160 unmodified lines
161	161	use core::marker::Unsize;
162	162	use core::mem::{self, SizedTypeProperties};
163	163	use core::ops::{
164		− CoerceUnsized, Deref, DerefMut, DispatchFromDyn, Generator, GeneratorState, Receiver,
	164	+ CoerceUnsized, Coroutine, CoroutineState, Deref, DerefMut, DispatchFromDyn, Receiver,
165	165	};
166	166	use core::pin::Pin;
167	167	use core::ptr::{self, NonNull, Unique};
@@ -211,7 +211,7 @@ impl<T> Box<T> { · 43 unmodified lines
211	211	/// ```
212	212	/// let five = Box::new(5);
213	213	/// ```
214		− #[cfg(all(not(no_global_oom_handling)))]
	214	+ #[cfg(not(no_global_oom_handling))]
215	215	#[inline(always)]
216	216	#[stable(feature = "rust1", since = "1.0.0")]
217	217	#[must_use]
@@ -1042,10 +1042,18 @@ impl<T: ?Sized, A: Allocator> Box<T, A> { · 824 unmodified lines
1042	1042	/// use std::ptr;
1043	1043	///
1044	1044	/// let x = Box::new(String::from("Hello"));
1045		− /// let p = Box::into_raw(x);
	1045	+ /// let ptr = Box::into_raw(x);
1046	1046	/// unsafe {
1047		− /// ptr::drop_in_place(p);
1048		− /// dealloc(p as *mut u8, Layout::new::<String>());
	1047	+ /// ptr::drop_in_place(ptr);
	1048	+ /// dealloc(ptr as *mut u8, Layout::new::<String>());
	1049	+ /// }
	1050	+ /// ```
	1051	+ /// Note: This is equivalent to the following:
	1052	+ /// ```
	1053	+ /// let x = Box::new(String::from("Hello"));
	1054	+ /// let ptr = Box::into_raw(x);
	1055	+ /// unsafe {
	1056	+ /// drop(Box::from_raw(ptr));
1049	1057	/// }
1050	1058	/// ```
1051	1059	///
@@ -2110,28 +2118,28 @@ impl<T: ?Sized, A: Allocator> AsMut<T> for Box<T, A> { · 1058 unmodified lines
2110	2118	#[stable(feature = "pin", since = "1.33.0")]
2111	2119	impl<T: ?Sized, A: Allocator> Unpin for Box<T, A> where A: 'static {}
2112	2120
2113		−#[unstable(feature = "generator_trait", issue = "43122")]
2114		−impl<G: ?Sized + Generator<R> + Unpin, R, A: Allocator> Generator<R> for Box<G, A>
	2121	+#[unstable(feature = "coroutine_trait", issue = "43122")]
	2122	+impl<G: ?Sized + Coroutine<R> + Unpin, R, A: Allocator> Coroutine<R> for Box<G, A>
2115	2123	where
2116	2124	A: 'static,
2117	2125	{
2118	2126	type Yield = G::Yield;
2119	2127	type Return = G::Return;
2120	2128
2121		− fn resume(mut self: Pin<&mut Self>, arg: R) -> GeneratorState<Self::Yield, Self::Return> {
	2129	+ fn resume(mut self: Pin<&mut Self>, arg: R) -> CoroutineState<Self::Yield, Self::Return> {
2122	2130	G::resume(Pin::new(&mut *self), arg)
2123	2131	}
2124	2132	}
2125	2133
2126		−#[unstable(feature = "generator_trait", issue = "43122")]
2127		−impl<G: ?Sized + Generator<R>, R, A: Allocator> Generator<R> for Pin<Box<G, A>>
	2134	+#[unstable(feature = "coroutine_trait", issue = "43122")]
	2135	+impl<G: ?Sized + Coroutine<R>, R, A: Allocator> Coroutine<R> for Pin<Box<G, A>>
2128	2136	where
2129	2137	A: 'static,
2130	2138	{
2131	2139	type Yield = G::Yield;
2132	2140	type Return = G::Return;
2133	2141
2134		− fn resume(mut self: Pin<&mut Self>, arg: R) -> GeneratorState<Self::Yield, Self::Return> {
	2142	+ fn resume(mut self: Pin<&mut Self>, arg: R) -> CoroutineState<Self::Yield, Self::Return> {
2135	2143	G::resume((*self).as_mut(), arg)
2136	2144	}
2137	2145	}
@@ -2448,4 +2456,8 @@ impl<T: core::error::Error> core::error::Error for Box<T> { · 310 unmodified lines
2448	2456	fn source(&self) -> Option<&(dyn core::error::Error + 'static)> {
2449	2457	core::error::Error::source(&**self)
2450	2458	}
	2459	+
	2460	+ fn provide<'b>(&'b self, request: &mut core::error::Request<'b>) {
	2461	+ core::error::Error::provide(&**self, request);
	2462	+ }
2451	2463	}

rust/alloc/collections/mod.rs+1−0

@@ -150,6 +150,7 @@ impl Display for TryReserveError { · 149 unmodified lines
150	150
151	151	/// An intermediate trait for specialization of `Extend`.
152	152	#[doc(hidden)]
	153	+#[cfg(not(no_global_oom_handling))]
153	154	trait SpecExtend<I: IntoIterator> {
154	155	/// Extends `self` with the contents of the given iterator.
155	156	fn spec_extend(&mut self, iter: I);

rust/alloc/lib.rs+5−4

@@ -80,6 +80,8 @@ · 79 unmodified lines
80	80	not(no_sync),
81	81	target_has_atomic = "ptr"
82	82	))]
	83	+#![doc(rust_logo)]
	84	+#![feature(rustdoc_internals)]
83	85	#![no_std]
84	86	#![needs_allocator]
85	87	// Lints:
@@ -115,7 +117,6 @@ · 29 unmodified lines
115	117	#![feature(const_eval_select)]
116	118	#![feature(const_maybe_uninit_as_mut_ptr)]
117	119	#![feature(const_maybe_uninit_write)]
118		−#![feature(const_maybe_uninit_zeroed)]
119	120	#![feature(const_pin)]
120	121	#![feature(const_refs_to_cell)]
121	122	#![feature(const_size_of_val)]
@@ -141,7 +142,6 @@ · 19 unmodified lines
141	142	#![feature(maybe_uninit_uninit_array)]
142	143	#![feature(maybe_uninit_uninit_array_transpose)]
143	144	#![feature(pattern)]
144		−#![feature(pointer_byte_offsets)]
145	145	#![feature(ptr_internals)]
146	146	#![feature(ptr_metadata)]
147	147	#![feature(ptr_sub_ptr)]
@@ -156,6 +156,7 @@ · 8 unmodified lines
156	156	#![feature(std_internals)]
157	157	#![feature(str_internals)]
158	158	#![feature(strict_provenance)]
	159	+#![feature(trusted_fused)]
159	160	#![feature(trusted_len)]
160	161	#![feature(trusted_random_access)]
161	162	#![feature(try_trait_v2)]
@@ -168,7 +169,7 @@ · 6 unmodified lines
168	169	//
169	170	// Language features:
170	171	// tidy-alphabetical-start
171		−#![cfg_attr(not(test), feature(generator_trait))]
	172	+#![cfg_attr(not(test), feature(coroutine_trait))]
172	173	#![cfg_attr(test, feature(panic_update_hook))]
173	174	#![cfg_attr(test, feature(test))]
174	175	#![feature(allocator_internals)]
@@ -276,7 +277,7 @@ pub(crate) mod test_helpers { · 101 unmodified lines
276	277	/// seed not being the same for every RNG invocation too.
277	278	pub(crate) fn test_rng() -> rand_xorshift::XorShiftRng {
278	279	use std::hash::{BuildHasher, Hash, Hasher};
279		− let mut hasher = std::collections::hash_map::RandomState::new().build_hasher();
	280	+ let mut hasher = std::hash::RandomState::new().build_hasher();
280	281	std::panic::Location::caller().hash(&mut hasher);
281	282	let hc64 = hasher.finish();
282	283	let seed_vec =

rust/alloc/raw_vec.rs+57−20

@@ -27,6 +27,16 @@ enum AllocInit { · 26 unmodified lines
27	27	Zeroed,
28	28	}
29	29
	30	+#[repr(transparent)]
	31	+#[cfg_attr(target_pointer_width = "16", rustc_layout_scalar_valid_range_end(0x7fff))]
	32	+#[cfg_attr(target_pointer_width = "32", rustc_layout_scalar_valid_range_end(0x7fff_ffff))]
	33	+#[cfg_attr(target_pointer_width = "64", rustc_layout_scalar_valid_range_end(0x7fff_ffff_ffff_ffff))]
	34	+struct Cap(usize);
	35	+
	36	+impl Cap {
	37	+ const ZERO: Cap = unsafe { Cap(0) };
	38	+}
	39	+
30	40	/// A low-level utility for more ergonomically allocating, reallocating, and deallocating
31	41	/// a buffer of memory on the heap without having to worry about all the corner cases
32	42	/// involved. This type is excellent for building your own data structures like Vec and VecDeque.
@@ -52,7 +62,12 @@ enum AllocInit { · 19 unmodified lines
52	62	#[allow(missing_debug_implementations)]
53	63	pub(crate) struct RawVec<T, A: Allocator = Global> {
54	64	ptr: Unique<T>,
55		− cap: usize,
	65	+ /// Never used for ZSTs; it's `capacity()`'s responsibility to return usize::MAX in that case.
	66	+ ///
	67	+ /// # Safety
	68	+ ///
	69	+ /// `cap` must be in the `0..=isize::MAX` range.
	70	+ cap: Cap,
56	71	alloc: A,
57	72	}
58	73
@@ -121,7 +136,7 @@ impl<T, A: Allocator> RawVec<T, A> { · 62 unmodified lines
121	136	/// the returned `RawVec`.
122	137	pub const fn new_in(alloc: A) -> Self {
123	138	// `cap: 0` means "unallocated". zero-sized types are ignored.
124		− Self { ptr: Unique::dangling(), cap: 0, alloc }
	139	+ Self { ptr: Unique::dangling(), cap: Cap::ZERO, alloc }
125	140	}
126	141
127	142	/// Like `with_capacity`, but parameterized over the choice of
@@ -203,7 +218,7 @@ impl<T, A: Allocator> RawVec<T, A> { · 75 unmodified lines
203	218	// here should change to `ptr.len() / mem::size_of::<T>()`.
204	219	Self {
205	220	ptr: unsafe { Unique::new_unchecked(ptr.cast().as_ptr()) },
206		− cap: capacity,
	221	+ cap: unsafe { Cap(capacity) },
207	222	alloc,
208	223	}
209	224	}
@@ -228,7 +243,7 @@ impl<T, A: Allocator> RawVec<T, A> { · 18 unmodified lines
228	243	// here should change to `ptr.len() / mem::size_of::<T>()`.
229	244	Ok(Self {
230	245	ptr: unsafe { Unique::new_unchecked(ptr.cast().as_ptr()) },
231		− cap: capacity,
	246	+ cap: unsafe { Cap(capacity) },
232	247	alloc,
233	248	})
234	249	}
@@ -240,12 +255,13 @@ impl<T, A: Allocator> RawVec<T, A> { · 5 unmodified lines
240	255	/// The `ptr` must be allocated (via the given allocator `alloc`), and with the given
241	256	/// `capacity`.
242	257	/// The `capacity` cannot exceed `isize::MAX` for sized types. (only a concern on 32-bit
243		− /// systems). ZST vectors may have a capacity up to `usize::MAX`.
	258	+ /// systems). For ZSTs capacity is ignored.
244	259	/// If the `ptr` and `capacity` come from a `RawVec` created via `alloc`, then this is
245	260	/// guaranteed.
246	261	#[inline]
247	262	pub unsafe fn from_raw_parts_in(ptr: *mut T, capacity: usize, alloc: A) -> Self {
248		− Self { ptr: unsafe { Unique::new_unchecked(ptr) }, cap: capacity, alloc }
	263	+ let cap = if T::IS_ZST { Cap::ZERO } else { unsafe { Cap(capacity) } };
	264	+ Self { ptr: unsafe { Unique::new_unchecked(ptr) }, cap, alloc }
249	265	}
250	266
251	267	/// Gets a raw pointer to the start of the allocation. Note that this is
@@ -261,7 +277,7 @@ impl<T, A: Allocator> RawVec<T, A> { · 9 unmodified lines
261	277	/// This will always be `usize::MAX` if `T` is zero-sized.
262	278	#[inline(always)]
263	279	pub fn capacity(&self) -> usize {
264		− if T::IS_ZST { usize::MAX } else { self.cap }
	280	+ if T::IS_ZST { usize::MAX } else { self.cap.0 }
265	281	}
266	282
267	283	/// Returns a shared reference to the allocator backing this `RawVec`.
@@ -270,7 +286,7 @@ impl<T, A: Allocator> RawVec<T, A> { · 2 unmodified lines
270	286	}
271	287
272	288	fn current_memory(&self) -> Option<(NonNull<u8>, Layout)> {
273		− if T::IS_ZST \|\| self.cap == 0 {
	289	+ if T::IS_ZST \|\| self.cap.0 == 0 {
274	290	None
275	291	} else {
276	292	// We could use Layout::array here which ensures the absence of isize and usize overflows
@@ -280,7 +296,7 @@ impl<T, A: Allocator> RawVec<T, A> { · 3 unmodified lines
280	296	let _: () = const { assert!(mem::size_of::<T>() % mem::align_of::<T>() == 0) };
281	297	unsafe {
282	298	let align = mem::align_of::<T>();
283		− let size = mem::size_of::<T>().unchecked_mul(self.cap);
	299	+ let size = mem::size_of::<T>().unchecked_mul(self.cap.0);
284	300	let layout = Layout::from_size_align_unchecked(size, align);
285	301	Some((self.ptr.cast().into(), layout))
286	302	}
@@ -338,10 +354,13 @@ impl<T, A: Allocator> RawVec<T, A> { · 51 unmodified lines
338	354	/// The same as `reserve`, but returns on errors instead of panicking or aborting.
339	355	pub fn try_reserve(&mut self, len: usize, additional: usize) -> Result<(), TryReserveError> {
340	356	if self.needs_to_grow(len, additional) {
341		− self.grow_amortized(len, additional)
342		− } else {
343		− Ok(())
	357	+ self.grow_amortized(len, additional)?;
344	358	}
	359	+ unsafe {
	360	+ // Inform the optimizer that the reservation has succeeded or wasn't needed
	361	+ core::intrinsics::assume(!self.needs_to_grow(len, additional));
	362	+ }
	363	+ Ok(())
345	364	}
346	365
347	366	/// The same as `reserve_for_push`, but returns on errors instead of panicking or aborting.
@@ -378,7 +397,14 @@ impl<T, A: Allocator> RawVec<T, A> { · 30 unmodified lines
378	397	len: usize,
379	398	additional: usize,
380	399	) -> Result<(), TryReserveError> {
381		− if self.needs_to_grow(len, additional) { self.grow_exact(len, additional) } else { Ok(()) }
	400	+ if self.needs_to_grow(len, additional) {
	401	+ self.grow_exact(len, additional)?;
	402	+ }
	403	+ unsafe {
	404	+ // Inform the optimizer that the reservation has succeeded or wasn't needed
	405	+ core::intrinsics::assume(!self.needs_to_grow(len, additional));
	406	+ }
	407	+ Ok(())
382	408	}
383	409
384	410	/// Shrinks the buffer down to the specified capacity. If the given amount
@@ -404,12 +430,15 @@ impl<T, A: Allocator> RawVec<T, A> { · 19 unmodified lines
404	430	additional > self.capacity().wrapping_sub(len)
405	431	}
406	432
407		− fn set_ptr_and_cap(&mut self, ptr: NonNull<[u8]>, cap: usize) {
	433	+ /// # Safety:
	434	+ ///
	435	+ /// `cap` must not exceed `isize::MAX`.
	436	+ unsafe fn set_ptr_and_cap(&mut self, ptr: NonNull<[u8]>, cap: usize) {
408	437	// Allocators currently return a `NonNull<[u8]>` whose length matches
409	438	// the size requested. If that ever changes, the capacity here should
410	439	// change to `ptr.len() / mem::size_of::<T>()`.
411	440	self.ptr = unsafe { Unique::new_unchecked(ptr.cast().as_ptr()) };
412		− self.cap = cap;
	441	+ self.cap = unsafe { Cap(cap) };
413	442	}
414	443
415	444	// This method is usually instantiated many times. So we want it to be as
@@ -434,14 +463,15 @@ impl<T, A: Allocator> RawVec<T, A> { · 18 unmodified lines
434	463
435	464	// This guarantees exponential growth. The doubling cannot overflow
436	465	// because `cap <= isize::MAX` and the type of `cap` is `usize`.
437		− let cap = cmp::max(self.cap * 2, required_cap);
	466	+ let cap = cmp::max(self.cap.0 * 2, required_cap);
438	467	let cap = cmp::max(Self::MIN_NON_ZERO_CAP, cap);
439	468
440	469	let new_layout = Layout::array::<T>(cap);
441	470
442	471	// `finish_grow` is non-generic over `T`.
443	472	let ptr = finish_grow(new_layout, self.current_memory(), &mut self.alloc)?;
444		− self.set_ptr_and_cap(ptr, cap);
	473	+ // SAFETY: finish_grow would have resulted in a capacity overflow if we tried to allocate more than isize::MAX items
	474	+ unsafe { self.set_ptr_and_cap(ptr, cap) };
445	475	Ok(())
446	476	}
447	477
@@ -460,7 +490,10 @@ impl<T, A: Allocator> RawVec<T, A> { · 12 unmodified lines
460	490
461	491	// `finish_grow` is non-generic over `T`.
462	492	let ptr = finish_grow(new_layout, self.current_memory(), &mut self.alloc)?;
463		− self.set_ptr_and_cap(ptr, cap);
	493	+ // SAFETY: finish_grow would have resulted in a capacity overflow if we tried to allocate more than isize::MAX items
	494	+ unsafe {
	495	+ self.set_ptr_and_cap(ptr, cap);
	496	+ }
464	497	Ok(())
465	498	}
466	499
@@ -478,7 +511,7 @@ impl<T, A: Allocator> RawVec<T, A> { · 11 unmodified lines
478	511	if cap == 0 {
479	512	unsafe { self.alloc.deallocate(ptr, layout) };
480	513	self.ptr = Unique::dangling();
481		− self.cap = 0;
	514	+ self.cap = Cap::ZERO;
482	515	} else {
483	516	let ptr = unsafe {
484	517	// `Layout::array` cannot overflow here because it would have
@@ -489,7 +522,10 @@ impl<T, A: Allocator> RawVec<T, A> { · 4 unmodified lines
489	522	.shrink(ptr, layout, new_layout)
490	523	.map_err(\|_\| AllocError { layout: new_layout, non_exhaustive: () })?
491	524	};
492		− self.set_ptr_and_cap(ptr, cap);
	525	+ // SAFETY: if the allocation is valid, then the capacity is too
	526	+ unsafe {
	527	+ self.set_ptr_and_cap(ptr, cap);
	528	+ }
493	529	}
494	530	Ok(())
495	531	}
@@ -569,6 +605,7 @@ fn alloc_guard(alloc_size: usize) -> Result<(), TryReserveError> { · 73 unmodified lines
569	605	// ensure that the code generation related to these panics is minimal as there's
570	606	// only one location which panics rather than a bunch throughout the module.
571	607	#[cfg(not(no_global_oom_handling))]
	608	+#[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
572	609	fn capacity_overflow() -> ! {
573	610	panic!("capacity overflow");
574	611	}

rust/alloc/vec/into_iter.rs+11−5

@@ -9,7 +9,8 @@ use crate::raw_vec::RawVec; · 8 unmodified lines
9	9	use core::array;
10	10	use core::fmt;
11	11	use core::iter::{
12		− FusedIterator, InPlaceIterable, SourceIter, TrustedLen, TrustedRandomAccessNoCoerce,
	12	+ FusedIterator, InPlaceIterable, SourceIter, TrustedFused, TrustedLen,
	13	+ TrustedRandomAccessNoCoerce,
13	14	};
14	15	use core::marker::PhantomData;
15	16	use core::mem::{self, ManuallyDrop, MaybeUninit, SizedTypeProperties};
@@ -287,9 +288,7 @@ impl<T, A: Allocator> Iterator for IntoIter<T, A> { · 271 unmodified lines
287	288	// Also note the implementation of `Self: TrustedRandomAccess` requires
288	289	// that `T: Copy` so reading elements from the buffer doesn't invalidate
289	290	// them for `Drop`.
290		− unsafe {
291		− if T::IS_ZST { mem::zeroed() } else { ptr::read(self.ptr.add(i)) }
292		− }
	291	+ unsafe { if T::IS_ZST { mem::zeroed() } else { ptr::read(self.ptr.add(i)) } }
293	292	}
294	293	}
295	294
@@ -341,6 +340,10 @@ impl<T, A: Allocator> ExactSizeIterator for IntoIter<T, A> { · 45 unmodified lines
341	340	#[stable(feature = "fused", since = "1.26.0")]
342	341	impl<T, A: Allocator> FusedIterator for IntoIter<T, A> {}
343	342
	343	+#[doc(hidden)]
	344	+#[unstable(issue = "none", feature = "trusted_fused")]
	345	+unsafe impl<T, A: Allocator> TrustedFused for IntoIter<T, A> {}
	346	+
344	347	#[unstable(feature = "trusted_len", issue = "37572")]
345	348	unsafe impl<T, A: Allocator> TrustedLen for IntoIter<T, A> {}
346	349
@@ -425,7 +428,10 @@ unsafe impl<#[may_dangle] T, A: Allocator> Drop for IntoIter<T, A> { · 78 unmodified lines
425	428	// also refer to the vec::in_place_collect module documentation to get an overview
426	429	#[unstable(issue = "none", feature = "inplace_iteration")]
427	430	#[doc(hidden)]
428		−unsafe impl<T, A: Allocator> InPlaceIterable for IntoIter<T, A> {}
	431	+unsafe impl<T, A: Allocator> InPlaceIterable for IntoIter<T, A> {
	432	+ const EXPAND_BY: Option<NonZeroUsize> = NonZeroUsize::new(1);
	433	+ const MERGE_BY: Option<NonZeroUsize> = NonZeroUsize::new(1);
	434	+}
429	435
430	436	#[unstable(issue = "none", feature = "inplace_iteration")]
431	437	#[doc(hidden)]

rust/alloc/vec/mod.rs+61−20

@@ -105,6 +105,7 @@ mod into_iter; · 104 unmodified lines
105	105	#[cfg(not(no_global_oom_handling))]
106	106	use self::is_zero::IsZero;
107	107
	108	+#[cfg(not(no_global_oom_handling))]
108	109	mod is_zero;
109	110
110	111	#[cfg(not(no_global_oom_handling))]
@@ -123,7 +124,7 @@ use self::set_len_on_drop::SetLenOnDrop; · 12 unmodified lines
123	124	mod set_len_on_drop;
124	125
125	126	#[cfg(not(no_global_oom_handling))]
126		−use self::in_place_drop::{InPlaceDrop, InPlaceDstBufDrop};
	127	+use self::in_place_drop::{InPlaceDrop, InPlaceDstDataSrcBufDrop};
127	128
128	129	#[cfg(not(no_global_oom_handling))]
129	130	mod in_place_drop;
@@ -1376,7 +1377,7 @@ impl<T, A: Allocator> Vec<T, A> { · 1246 unmodified lines
1376	1377	/// [`as_mut_ptr`]: Vec::as_mut_ptr
1377	1378	/// [`as_ptr`]: Vec::as_ptr
1378	1379	#[stable(feature = "vec_as_ptr", since = "1.37.0")]
1379		− #[cfg_attr(not(bootstrap), rustc_never_returns_null_ptr)]
	1380	+ #[rustc_never_returns_null_ptr]
1380	1381	#[inline]
1381	1382	pub fn as_ptr(&self) -> *const T {
1382	1383	// We shadow the slice method of the same name to avoid going through
@@ -1436,7 +1437,7 @@ impl<T, A: Allocator> Vec<T, A> { · 53 unmodified lines
1436	1437	/// [`as_mut_ptr`]: Vec::as_mut_ptr
1437	1438	/// [`as_ptr`]: Vec::as_ptr
1438	1439	#[stable(feature = "vec_as_ptr", since = "1.37.0")]
1439		− #[cfg_attr(not(bootstrap), rustc_never_returns_null_ptr)]
	1440	+ #[rustc_never_returns_null_ptr]
1440	1441	#[inline]
1441	1442	pub fn as_mut_ptr(&mut self) -> *mut T {
1442	1443	// We shadow the slice method of the same name to avoid going through
@@ -1565,7 +1566,8 @@ impl<T, A: Allocator> Vec<T, A> { · 122 unmodified lines
1565	1566	#[stable(feature = "rust1", since = "1.0.0")]
1566	1567	pub fn swap_remove(&mut self, index: usize) -> T {
1567	1568	#[cold]
1568		− #[inline(never)]
	1569	+ #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
	1570	+ #[track_caller]
1569	1571	fn assert_failed(index: usize, len: usize) -> ! {
1570	1572	panic!("swap_remove index (is {index}) should be < len (is {len})");
1571	1573	}
@@ -1606,7 +1608,8 @@ impl<T, A: Allocator> Vec<T, A> { · 34 unmodified lines
1606	1608	#[stable(feature = "rust1", since = "1.0.0")]
1607	1609	pub fn insert(&mut self, index: usize, element: T) {
1608	1610	#[cold]
1609		− #[inline(never)]
	1611	+ #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
	1612	+ #[track_caller]
1610	1613	fn assert_failed(index: usize, len: usize) -> ! {
1611	1614	panic!("insertion index (is {index}) should be <= len (is {len})");
1612	1615	}
@@ -1667,7 +1670,7 @@ impl<T, A: Allocator> Vec<T, A> { · 54 unmodified lines
1667	1670	#[track_caller]
1668	1671	pub fn remove(&mut self, index: usize) -> T {
1669	1672	#[cold]
1670		− #[inline(never)]
	1673	+ #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
1671	1674	#[track_caller]
1672	1675	fn assert_failed(index: usize, len: usize) -> ! {
1673	1676	panic!("removal index (is {index}) should be < len (is {len})");
@@ -1891,7 +1894,32 @@ impl<T, A: Allocator> Vec<T, A> { · 217 unmodified lines
1891	1894	return;
1892	1895	}
1893	1896
1894		− /* INVARIANT: vec.len() > read >= write > write-1 >= 0 */
	1897	+ // Check if we ever want to remove anything.
	1898	+ // This allows to use copy_non_overlapping in next cycle.
	1899	+ // And avoids any memory writes if we don't need to remove anything.
	1900	+ let mut first_duplicate_idx: usize = 1;
	1901	+ let start = self.as_mut_ptr();
	1902	+ while first_duplicate_idx != len {
	1903	+ let found_duplicate = unsafe {
	1904	+ // SAFETY: first_duplicate always in range [1..len)
	1905	+ // Note that we start iteration from 1 so we never overflow.
	1906	+ let prev = start.add(first_duplicate_idx.wrapping_sub(1));
	1907	+ let current = start.add(first_duplicate_idx);
	1908	+ // We explicitly say in docs that references are reversed.
	1909	+ same_bucket(&mut current, &mut prev)
	1910	+ };
	1911	+ if found_duplicate {
	1912	+ break;
	1913	+ }
	1914	+ first_duplicate_idx += 1;
	1915	+ }
	1916	+ // Don't need to remove anything.
	1917	+ // We cannot get bigger than len.
	1918	+ if first_duplicate_idx == len {
	1919	+ return;
	1920	+ }
	1921	+
	1922	+ /* INVARIANT: vec.len() > read > write > write-1 >= 0 */
1895	1923	struct FillGapOnDrop<'a, T, A: core::alloc::Allocator> {
1896	1924	/* Offset of the element we want to check if it is duplicate */
1897	1925	read: usize,
@@ -1937,31 +1965,39 @@ impl<T, A: Allocator> Vec<T, A> { · 39 unmodified lines
1937	1965	}
1938	1966	}
1939	1967
1940		− let mut gap = FillGapOnDrop { read: 1, write: 1, vec: self };
1941		− let ptr = gap.vec.as_mut_ptr();
1942		−
1943	1968	/* Drop items while going through Vec, it should be more efficient than
1944	1969	* doing slice partition_dedup + truncate */
1945	1970
	1971	+ // Construct gap first and then drop item to avoid memory corruption if `T::drop` panics.
	1972	+ let mut gap =
	1973	+ FillGapOnDrop { read: first_duplicate_idx + 1, write: first_duplicate_idx, vec: self };
	1974	+ unsafe {
	1975	+ // SAFETY: we checked that first_duplicate_idx in bounds before.
	1976	+ // If drop panics, `gap` would remove this item without drop.
	1977	+ ptr::drop_in_place(start.add(first_duplicate_idx));
	1978	+ }
	1979	+
1946	1980	/* SAFETY: Because of the invariant, read_ptr, prev_ptr and write_ptr
1947	1981	* are always in-bounds and read_ptr never aliases prev_ptr */
1948	1982	unsafe {
1949	1983	while gap.read < len {
1950		− let read_ptr = ptr.add(gap.read);
1951		− let prev_ptr = ptr.add(gap.write.wrapping_sub(1));
	1984	+ let read_ptr = start.add(gap.read);
	1985	+ let prev_ptr = start.add(gap.write.wrapping_sub(1));
1952	1986
1953		− if same_bucket(&mut read_ptr, &mut prev_ptr) {
	1987	+ // We explicitly say in docs that references are reversed.
	1988	+ let found_duplicate = same_bucket(&mut read_ptr, &mut prev_ptr);
	1989	+ if found_duplicate {
1954	1990	// Increase `gap.read` now since the drop may panic.
1955	1991	gap.read += 1;
1956	1992	/* We have found duplicate, drop it in-place */
1957	1993	ptr::drop_in_place(read_ptr);
1958	1994	} else {
1959		− let write_ptr = ptr.add(gap.write);
	1995	+ let write_ptr = start.add(gap.write);
1960	1996
1961		− /* Because `read_ptr` can be equal to `write_ptr`, we either
1962		− * have to use `copy` or conditional `copy_nonoverlapping`.
1963		− * Looks like the first option is faster. */
1964		− ptr::copy(read_ptr, write_ptr, 1);
	1997	+ /* read_ptr cannot be equal to write_ptr because at this point
	1998	+ * we guaranteed to skip at least one element (before loop starts).
	1999	+ */
	2000	+ ptr::copy_nonoverlapping(read_ptr, write_ptr, 1);
1965	2001
1966	2002	/* We have filled that place, so go further */
1967	2003	gap.write += 1;
@@ -2097,6 +2133,7 @@ impl<T, A: Allocator> Vec<T, A> { · 129 unmodified lines
2097	2133	} else {
2098	2134	unsafe {
2099	2135	self.len -= 1;
	2136	+ core::intrinsics::assume(self.len < self.capacity());
2100	2137	Some(ptr::read(self.as_ptr().add(self.len())))
2101	2138	}
2102	2139	}
@@ -2299,7 +2336,8 @@ impl<T, A: Allocator> Vec<T, A> { · 196 unmodified lines
2299	2336	A: Clone,
2300	2337	{
2301	2338	#[cold]
2302		− #[inline(never)]
	2339	+ #[cfg_attr(not(feature = "panic_immediate_abort"), inline(never))]
	2340	+ #[track_caller]
2303	2341	fn assert_failed(at: usize, len: usize) -> ! {
2304	2342	panic!("`at` split index (is {at}) should be <= len (is {len})");
2305	2343	}
@@ -2840,6 +2878,7 @@ pub fn from_elem_in<T: Clone, A: Allocator>(elem: T, n: usize, alloc: A) -> Vec< · 534 unmodified lines
2840	2878	<T as SpecFromElem>::from_elem(elem, n, alloc)
2841	2879	}
2842	2880
	2881	+#[cfg(not(no_global_oom_handling))]
2843	2882	trait ExtendFromWithinSpec {
2844	2883	/// # Safety
2845	2884	///
@@ -2848,6 +2887,7 @@ trait ExtendFromWithinSpec { · 2 unmodified lines
2848	2887	unsafe fn spec_extend_from_within(&mut self, src: Range<usize>);
2849	2888	}
2850	2889
	2890	+#[cfg(not(no_global_oom_handling))]
2851	2891	impl<T: Clone, A: Allocator> ExtendFromWithinSpec for Vec<T, A> {
2852	2892	default unsafe fn spec_extend_from_within(&mut self, src: Range<usize>) {
2853	2893	// SAFETY:
@@ -2867,6 +2907,7 @@ impl<T: Clone, A: Allocator> ExtendFromWithinSpec for Vec<T, A> { · 13 unmodified lines
2867	2907	}
2868	2908	}
2869	2909
	2910	+#[cfg(not(no_global_oom_handling))]
2870	2911	impl<T: Copy, A: Allocator> ExtendFromWithinSpec for Vec<T, A> {
2871	2912	unsafe fn spec_extend_from_within(&mut self, src: Range<usize>) {
2872	2913	let count = src.len();
@@ -2947,7 +2988,7 @@ impl<T: Clone, A: Allocator + Clone> Clone for Vec<T, A> { · 74 unmodified lines
2947	2988	/// ```
2948	2989	/// use std::hash::BuildHasher;
2949	2990	///
2950		−/// let b = std::collections::hash_map::RandomState::new();
	2991	+/// let b = std::hash::RandomState::new();
2951	2992	/// let v: Vec<u8> = vec![0xa8, 0x3c, 0x09];
2952	2993	/// let s: &[u8] = &[0xa8, 0x3c, 0x09];
2953	2994	/// assert_eq!(b.hash_one(v), b.hash_one(s));

rust/bindings/bindings_helper.h+3−2

@@ -9,12 +9,13 @@ · 8 unmodified lines
9	9	#include <kunit/test.h>
10	10	#include <linux/errname.h>
11	11	#include <linux/ethtool.h>
	12	+#include <linux/jiffies.h>
12	13	#include <linux/mdio.h>
13	14	#include <linux/phy.h>
14		−#include <linux/slab.h>
15	15	#include <linux/refcount.h>
16		−#include <linux/wait.h>
17	16	#include <linux/sched.h>
	17	+#include <linux/slab.h>
	18	+#include <linux/wait.h>
18	19	#include <linux/workqueue.h>
19	20
20	21	/* `bindgen` gets confused at certain things. */

rust/kernel/allocator.rs+1−1

@@ -35,7 +35,7 @@ unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: bindings::gf · 34 unmodified lines
35	35	// - `ptr` is either null or a pointer returned from a previous `k{re}alloc()` by the
36	36	// function safety requirement.
37	37	// - `size` is greater than 0 since it's either a `layout.size()` (which cannot be zero
38		− // according to the function safety requirement) or a result from `next_power_of_two()`.
	38	+ // according to the function safety requirement) or a result from `next_power_of_two()`.
39	39	unsafe { bindings::krealloc(ptr as const core::ffi::c_void, size, flags) as mut u8 }
40	40	}
41	41

rust/kernel/error.rs+3−7

@@ -264,13 +264,9 @@ pub fn to_result(err: core::ffi::c_int) -> Result { · 263 unmodified lines
264	264	/// pdev: &mut PlatformDevice,
265	265	/// index: u32,
266	266	/// ) -> Result<*mut core::ffi::c_void> {
267		−/// // SAFETY: FFI call.
268		−/// unsafe {
269		−/// from_err_ptr(bindings::devm_platform_ioremap_resource(
270		−/// pdev.to_ptr(),
271		−/// index,
272		−/// ))
273		−/// }
	267	+/// // SAFETY: `pdev` points to a valid platform device. There are no safety requirements
	268	+/// // on `index`.
	269	+/// from_err_ptr(unsafe { bindings::devm_platform_ioremap_resource(pdev.to_ptr(), index) })
274	270	/// }
275	271	/// ```
276	272	// TODO: Remove `dead_code` marker once an in-kernel client is available.

rust/kernel/init.rs+11−11

@@ -36,7 +36,7 @@ · 35 unmodified lines
36	36	//!
37	37	//! ```rust
38	38	//! # #![allow(clippy::disallowed_names)]
39		−//! use kernel::{prelude::*, sync::Mutex, new_mutex};
	39	+//! use kernel::sync::{new_mutex, Mutex};
40	40	//! # use core::pin::Pin;
41	41	//! #[pin_data]
42	42	//! struct Foo {
@@ -56,7 +56,7 @@ · 13 unmodified lines
56	56	//!
57	57	//! ```rust
58	58	//! # #![allow(clippy::disallowed_names)]
59		−//! # use kernel::{prelude::*, sync::Mutex, new_mutex};
	59	+//! # use kernel::sync::{new_mutex, Mutex};
60	60	//! # use core::pin::Pin;
61	61	//! # #[pin_data]
62	62	//! # struct Foo {
@@ -79,7 +79,7 @@ · 16 unmodified lines
79	79	//! above method only works for types where you can access the fields.
80	80	//!
81	81	//! ```rust
82		−//! # use kernel::{new_mutex, sync::{Arc, Mutex}};
	82	+//! # use kernel::sync::{new_mutex, Arc, Mutex};
83	83	//! let mtx: Result<Arc<Mutex<usize>>> = Arc::pin_init(new_mutex!(42, "example::mtx"));
84	84	//! ```
85	85	//!
@@ -751,10 +751,10 @@ macro_rules! try_init { · 665 unmodified lines
751	751	///
752	752	/// # Safety
753	753	///
754		−/// When implementing this type you will need to take great care. Also there are probably very few
	754	+/// When implementing this trait you will need to take great care. Also there are probably very few
755	755	/// cases where a manual implementation is necessary. Use [`pin_init_from_closure`] where possible.
756	756	///
757		−/// The [`PinInit::__pinned_init`] function
	757	+/// The [`PinInit::__pinned_init`] function:
758	758	/// - returns `Ok(())` if it initialized every field of `slot`,
759	759	/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
760	760	/// - `slot` can be deallocated without UB occurring,
@@ -861,10 +861,10 @@ where · 100 unmodified lines
861	861	///
862	862	/// # Safety
863	863	///
864		−/// When implementing this type you will need to take great care. Also there are probably very few
	864	+/// When implementing this trait you will need to take great care. Also there are probably very few
865	865	/// cases where a manual implementation is necessary. Use [`init_from_closure`] where possible.
866	866	///
867		−/// The [`Init::__init`] function
	867	+/// The [`Init::__init`] function:
868	868	/// - returns `Ok(())` if it initialized every field of `slot`,
869	869	/// - returns `Err(err)` if it encountered an error and then cleaned `slot`, this means:
870	870	/// - `slot` can be deallocated without UB occurring,
@@ -1013,7 +1013,7 @@ pub fn uninit<T, E>() -> impl Init<MaybeUninit<T>, E> { · 142 unmodified lines
1013	1013	///
1014	1014	/// ```rust
1015	1015	/// use kernel::{error::Error, init::init_array_from_fn};
1016		−/// let array: Box<[usize; 1_000]>= Box::init::<Error>(init_array_from_fn(\|i\| i)).unwrap();
	1016	+/// let array: Box<[usize; 1_000]> = Box::init::<Error>(init_array_from_fn(\|i\| i)).unwrap();
1017	1017	/// assert_eq!(array.len(), 1_000);
1018	1018	/// ```
1019	1019	pub fn init_array_from_fn<I, const N: usize, T, E>(
@@ -1027,7 +1027,7 @@ where · 7 unmodified lines
1027	1027	// Counts the number of initialized elements and when dropped drops that many elements from
1028	1028	// `slot`.
1029	1029	let mut init_count = ScopeGuard::new_with_data(0, \|i\| {
1030		− // We now free every element that has been initialized before:
	1030	+ // We now free every element that has been initialized before.
1031	1031	// SAFETY: The loop initialized exactly the values from 0..i and since we
1032	1032	// return `Err` below, the caller will consider the memory at `slot` as
1033	1033	// uninitialized.
@@ -1056,7 +1056,7 @@ where · 22 unmodified lines
1056	1056	///
1057	1057	/// ```rust
1058	1058	/// use kernel::{sync::{Arc, Mutex}, init::pin_init_array_from_fn, new_mutex};
1059		−/// let array: Arc<[Mutex<usize>; 1_000]>=
	1059	+/// let array: Arc<[Mutex<usize>; 1_000]> =
1060	1060	/// Arc::pin_init(pin_init_array_from_fn(\|i\| new_mutex!(i))).unwrap();
1061	1061	/// assert_eq!(array.len(), 1_000);
1062	1062	/// ```
@@ -1071,7 +1071,7 @@ where · 8 unmodified lines
1071	1071	// Counts the number of initialized elements and when dropped drops that many elements from
1072	1072	// `slot`.
1073	1073	let mut init_count = ScopeGuard::new_with_data(0, \|i\| {
1074		− // We now free every element that has been initialized before:
	1074	+ // We now free every element that has been initialized before.
1075	1075	// SAFETY: The loop initialized exactly the values from 0..i and since we
1076	1076	// return `Err` below, the caller will consider the memory at `slot` as
1077	1077	// uninitialized.

rust/kernel/ioctl.rs+3−3

@@ -1,6 +1,6 @@
1	1	// SPDX-License-Identifier: GPL-2.0
2	2
3		−//! ioctl() number definitions
	3	+//! `ioctl()` number definitions.
4	4	//!
5	5	//! C header: [`include/asm-generic/ioctl.h`](srctree/include/asm-generic/ioctl.h)
6	6
@@ -28,13 +28,13 @@ pub const fn _IO(ty: u32, nr: u32) -> u32 { · 21 unmodified lines
28	28	_IOC(uapi::_IOC_NONE, ty, nr, 0)
29	29	}
30	30
31		−/// Build an ioctl number for an read-only ioctl.
	31	+/// Build an ioctl number for a read-only ioctl.
32	32	#[inline(always)]
33	33	pub const fn _IOR<T>(ty: u32, nr: u32) -> u32 {
34	34	_IOC(uapi::_IOC_READ, ty, nr, core::mem::size_of::<T>())
35	35	}
36	36
37		−/// Build an ioctl number for an write-only ioctl.
	37	+/// Build an ioctl number for a write-only ioctl.
38	38	#[inline(always)]
39	39	pub const fn _IOW<T>(ty: u32, nr: u32) -> u32 {
40	40	_IOC(uapi::_IOC_WRITE, ty, nr, core::mem::size_of::<T>())

rust/kernel/lib.rs+34−3

@@ -14,11 +14,9 @@ · 13 unmodified lines
14	14	#![no_std]
15	15	#![feature(allocator_api)]
16	16	#![feature(coerce_unsized)]
17		−#![feature(const_maybe_uninit_zeroed)]
18	17	#![feature(dispatch_from_dyn)]
19	18	#![feature(new_uninit)]
20	19	#![feature(offset_of)]
21		−#![feature(ptr_metadata)]
22	20	#![feature(receiver_trait)]
23	21	#![feature(unsize)]
24	22
@@ -49,6 +47,7 @@ pub mod std_vendor; · 24 unmodified lines
49	47	pub mod str;
50	48	pub mod sync;
51	49	pub mod task;
	50	+pub mod time;
52	51	pub mod types;
53	52	pub mod workqueue;
54	53
@@ -78,7 +77,7 @@ pub trait Module: Sized + Sync { · 23 unmodified lines
78	77
79	78	/// Equivalent to `THIS_MODULE` in the C API.
80	79	///
81		−/// C header: `include/linux/export.h`
	80	+/// C header: [`include/linux/export.h`](srctree/include/linux/export.h)
82	81	pub struct ThisModule(*mut bindings::module);
83	82
84	83	// SAFETY: `THIS_MODULE` may be used from all threads within a module.
@@ -102,3 +101,35 @@ fn panic(info: &core::panic::PanicInfo<'_>) -> ! { · 17 unmodified lines
102	101	// SAFETY: FFI call.
103	102	unsafe { bindings::BUG() };
104	103	}
	104	+
	105	+/// Produces a pointer to an object from a pointer to one of its fields.
	106	+///
	107	+/// # Safety
	108	+///
	109	+/// The pointer passed to this macro, and the pointer returned by this macro, must both be in
	110	+/// bounds of the same allocation.
	111	+///
	112	+/// # Examples
	113	+///
	114	+/// ```
	115	+/// # use kernel::container_of;
	116	+/// struct Test {
	117	+/// a: u64,
	118	+/// b: u32,
	119	+/// }
	120	+///
	121	+/// let test = Test { a: 10, b: 20 };
	122	+/// let b_ptr = &test.b;
	123	+/// // SAFETY: The pointer points at the `b` field of a `Test`, so the resulting pointer will be
	124	+/// // in-bounds of the same allocation as `b_ptr`.
	125	+/// let test_alias = unsafe { container_of!(b_ptr, Test, b) };
	126	+/// assert!(core::ptr::eq(&test, test_alias));
	127	+/// ```
	128	+#[macro_export]
	129	+macro_rules! container_of {
	130	+ ($ptr:expr, $type:ty, $($f:tt)*) => {{
	131	+ let ptr = $ptr as const _ as const u8;
	132	+ let offset: usize = ::core::mem::offset_of!($type, $($f)*);
	133	+ ptr.sub(offset) as *const $type
	134	+ }}
	135	+}

rust/kernel/str.rs+182−11

@@ -13,9 +13,102 @@ use crate::{ · 12 unmodified lines
13	13	};
14	14
15	15	/// Byte string without UTF-8 validity guarantee.
16		−///
17		−/// `BStr` is simply an alias to `[u8]`, but has a more evident semantical meaning.
18		−pub type BStr = [u8];
	16	+#[repr(transparent)]
	17	+pub struct BStr([u8]);
	18	+
	19	+impl BStr {
	20	+ /// Returns the length of this string.
	21	+ #[inline]
	22	+ pub const fn len(&self) -> usize {
	23	+ self.0.len()
	24	+ }
	25	+
	26	+ /// Returns `true` if the string is empty.
	27	+ #[inline]
	28	+ pub const fn is_empty(&self) -> bool {
	29	+ self.len() == 0
	30	+ }
	31	+
	32	+ /// Creates a [`BStr`] from a `[u8]`.
	33	+ #[inline]
	34	+ pub const fn from_bytes(bytes: &[u8]) -> &Self {
	35	+ // SAFETY: `BStr` is transparent to `[u8]`.
	36	+ unsafe { &(bytes as const [u8] as *const BStr) }
	37	+ }
	38	+}
	39	+
	40	+impl fmt::Display for BStr {
	41	+ /// Formats printable ASCII characters, escaping the rest.
	42	+ ///
	43	+ /// ```
	44	+ /// # use kernel::{fmt, b_str, str::{BStr, CString}};
	45	+ /// let ascii = b_str!("Hello, BStr!");
	46	+ /// let s = CString::try_from_fmt(fmt!("{}", ascii)).unwrap();
	47	+ /// assert_eq!(s.as_bytes(), "Hello, BStr!".as_bytes());
	48	+ ///
	49	+ /// let non_ascii = b_str!("🦀");
	50	+ /// let s = CString::try_from_fmt(fmt!("{}", non_ascii)).unwrap();
	51	+ /// assert_eq!(s.as_bytes(), "\\xf0\\x9f\\xa6\\x80".as_bytes());
	52	+ /// ```
	53	+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
	54	+ for &b in &self.0 {
	55	+ match b {
	56	+ // Common escape codes.
	57	+ b'\t' => f.write_str("\\t")?,
	58	+ b'\n' => f.write_str("\\n")?,
	59	+ b'\r' => f.write_str("\\r")?,
	60	+ // Printable characters.
	61	+ 0x20..=0x7e => f.write_char(b as char)?,
	62	+ _ => write!(f, "\\x{:02x}", b)?,
	63	+ }
	64	+ }
	65	+ Ok(())
	66	+ }
	67	+}
	68	+
	69	+impl fmt::Debug for BStr {
	70	+ /// Formats printable ASCII characters with a double quote on either end,
	71	+ /// escaping the rest.
	72	+ ///
	73	+ /// ```
	74	+ /// # use kernel::{fmt, b_str, str::{BStr, CString}};
	75	+ /// // Embedded double quotes are escaped.
	76	+ /// let ascii = b_str!("Hello, \"BStr\"!");
	77	+ /// let s = CString::try_from_fmt(fmt!("{:?}", ascii)).unwrap();
	78	+ /// assert_eq!(s.as_bytes(), "\"Hello, \\\"BStr\\\"!\"".as_bytes());
	79	+ ///
	80	+ /// let non_ascii = b_str!("😺");
	81	+ /// let s = CString::try_from_fmt(fmt!("{:?}", non_ascii)).unwrap();
	82	+ /// assert_eq!(s.as_bytes(), "\"\\xf0\\x9f\\x98\\xba\"".as_bytes());
	83	+ /// ```
	84	+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
	85	+ f.write_char('"')?;
	86	+ for &b in &self.0 {
	87	+ match b {
	88	+ // Common escape codes.
	89	+ b'\t' => f.write_str("\\t")?,
	90	+ b'\n' => f.write_str("\\n")?,
	91	+ b'\r' => f.write_str("\\r")?,
	92	+ // String escape characters.
	93	+ b'\"' => f.write_str("\\\"")?,
	94	+ b'\\' => f.write_str("\\\\")?,
	95	+ // Printable characters.
	96	+ 0x20..=0x7e => f.write_char(b as char)?,
	97	+ _ => write!(f, "\\x{:02x}", b)?,
	98	+ }
	99	+ }
	100	+ f.write_char('"')
	101	+ }
	102	+}
	103	+
	104	+impl Deref for BStr {
	105	+ type Target = [u8];
	106	+
	107	+ #[inline]
	108	+ fn deref(&self) -> &Self::Target {
	109	+ &self.0
	110	+ }
	111	+}
19	112
20	113	/// Creates a new [`BStr`] from a string literal.
21	114	///
@@ -33,7 +126,7 @@ pub type BStr = [u8]; · 11 unmodified lines
33	126	macro_rules! b_str {
34	127	($str:literal) => {{
35	128	const S: &'static str = $str;
36		− const C: &'static $crate::str::BStr = S.as_bytes();
	129	+ const C: &'static $crate::str::BStr = $crate::str::BStr::from_bytes(S.as_bytes());
37	130	C
38	131	}};
39	132	}
@@ -149,13 +242,13 @@ impl CStr { · 109 unmodified lines
149	242	self.0.as_ptr() as _
150	243	}
151	244
152		− /// Convert the string to a byte slice without the trailing 0 byte.
	245	+ /// Convert the string to a byte slice without the trailing `NUL` byte.
153	246	#[inline]
154	247	pub fn as_bytes(&self) -> &[u8] {
155	248	&self.0[..self.len()]
156	249	}
157	250
158		− /// Convert the string to a byte slice containing the trailing 0 byte.
	251	+ /// Convert the string to a byte slice containing the trailing `NUL` byte.
159	252	#[inline]
160	253	pub const fn as_bytes_with_nul(&self) -> &[u8] {
161	254	&self.0
@@ -191,9 +284,9 @@ impl CStr { · 29 unmodified lines
191	284	/// ```
192	285	/// # use kernel::c_str;
193	286	/// # use kernel::str::CStr;
	287	+ /// let bar = c_str!("ツ");
194	288	/// // SAFETY: String literals are guaranteed to be valid UTF-8
195	289	/// // by the Rust compiler.
196		− /// let bar = c_str!("ツ");
197	290	/// assert_eq!(unsafe { bar.as_str_unchecked() }, "ツ");
198	291	/// ```
199	292	#[inline]
@@ -271,7 +364,7 @@ impl fmt::Debug for CStr { · 71 unmodified lines
271	364	impl AsRef<BStr> for CStr {
272	365	#[inline]
273	366	fn as_ref(&self) -> &BStr {
274		− self.as_bytes()
	367	+ BStr::from_bytes(self.as_bytes())
275	368	}
276	369	}
277	370
@@ -280,7 +373,7 @@ impl Deref for CStr { · 2 unmodified lines
280	373
281	374	#[inline]
282	375	fn deref(&self) -> &Self::Target {
283		− self.as_bytes()
	376	+ self.as_ref()
284	377	}
285	378	}
286	379
@@ -327,7 +420,7 @@ where · 40 unmodified lines
327	420
328	421	#[inline]
329	422	fn index(&self, index: Idx) -> &Self::Output {
330		− &self.as_bytes()[index]
	423	+ &self.as_ref()[index]
331	424	}
332	425	}
333	426
@@ -357,6 +450,21 @@ macro_rules! c_str { · 23 unmodified lines
357	450	#[cfg(test)]
358	451	mod tests {
359	452	use super::*;
	453	+ use alloc::format;
	454	+
	455	+ const ALL_ASCII_CHARS: &'static str =
	456	+ "\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x09\\x0a\\x0b\\x0c\\x0d\\x0e\\x0f\
	457	+ \\x10\\x11\\x12\\x13\\x14\\x15\\x16\\x17\\x18\\x19\\x1a\\x1b\\x1c\\x1d\\x1e\\x1f \
	458	+ !\"#$%&'()*+,-./0123456789:;<=>?@\
	459	+ ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{\|}~\\x7f\
	460	+ \\x80\\x81\\x82\\x83\\x84\\x85\\x86\\x87\\x88\\x89\\x8a\\x8b\\x8c\\x8d\\x8e\\x8f\
	461	+ \\x90\\x91\\x92\\x93\\x94\\x95\\x96\\x97\\x98\\x99\\x9a\\x9b\\x9c\\x9d\\x9e\\x9f\
	462	+ \\xa0\\xa1\\xa2\\xa3\\xa4\\xa5\\xa6\\xa7\\xa8\\xa9\\xaa\\xab\\xac\\xad\\xae\\xaf\
	463	+ \\xb0\\xb1\\xb2\\xb3\\xb4\\xb5\\xb6\\xb7\\xb8\\xb9\\xba\\xbb\\xbc\\xbd\\xbe\\xbf\
	464	+ \\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\
	465	+ \\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd7\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdf\
	466	+ \\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\
	467	+ \\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf7\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff";
360	468
361	469	#[test]
362	470	fn test_cstr_to_str() {
@@ -381,6 +489,69 @@ mod tests { · 18 unmodified lines
381	489	let unchecked_str = unsafe { checked_cstr.as_str_unchecked() };
382	490	assert_eq!(unchecked_str, "🐧");
383	491	}
	492	+
	493	+ #[test]
	494	+ fn test_cstr_display() {
	495	+ let hello_world = CStr::from_bytes_with_nul(b"hello, world!\0").unwrap();
	496	+ assert_eq!(format!("{}", hello_world), "hello, world!");
	497	+ let non_printables = CStr::from_bytes_with_nul(b"\x01\x09\x0a\0").unwrap();
	498	+ assert_eq!(format!("{}", non_printables), "\\x01\\x09\\x0a");
	499	+ let non_ascii = CStr::from_bytes_with_nul(b"d\xe9j\xe0 vu\0").unwrap();
	500	+ assert_eq!(format!("{}", non_ascii), "d\\xe9j\\xe0 vu");
	501	+ let good_bytes = CStr::from_bytes_with_nul(b"\xf0\x9f\xa6\x80\0").unwrap();
	502	+ assert_eq!(format!("{}", good_bytes), "\\xf0\\x9f\\xa6\\x80");
	503	+ }
	504	+
	505	+ #[test]
	506	+ fn test_cstr_display_all_bytes() {
	507	+ let mut bytes: [u8; 256] = [0; 256];
	508	+ // fill `bytes` with [1..=255] + [0]
	509	+ for i in u8::MIN..=u8::MAX {
	510	+ bytes[i as usize] = i.wrapping_add(1);
	511	+ }
	512	+ let cstr = CStr::from_bytes_with_nul(&bytes).unwrap();
	513	+ assert_eq!(format!("{}", cstr), ALL_ASCII_CHARS);
	514	+ }
	515	+
	516	+ #[test]
	517	+ fn test_cstr_debug() {
	518	+ let hello_world = CStr::from_bytes_with_nul(b"hello, world!\0").unwrap();
	519	+ assert_eq!(format!("{:?}", hello_world), "\"hello, world!\"");
	520	+ let non_printables = CStr::from_bytes_with_nul(b"\x01\x09\x0a\0").unwrap();
	521	+ assert_eq!(format!("{:?}", non_printables), "\"\\x01\\x09\\x0a\"");
	522	+ let non_ascii = CStr::from_bytes_with_nul(b"d\xe9j\xe0 vu\0").unwrap();
	523	+ assert_eq!(format!("{:?}", non_ascii), "\"d\\xe9j\\xe0 vu\"");
	524	+ let good_bytes = CStr::from_bytes_with_nul(b"\xf0\x9f\xa6\x80\0").unwrap();
	525	+ assert_eq!(format!("{:?}", good_bytes), "\"\\xf0\\x9f\\xa6\\x80\"");
	526	+ }
	527	+
	528	+ #[test]
	529	+ fn test_bstr_display() {
	530	+ let hello_world = BStr::from_bytes(b"hello, world!");
	531	+ assert_eq!(format!("{}", hello_world), "hello, world!");
	532	+ let escapes = BStr::from_bytes(b"_\t_\n_\r_\\_\'_\"_");
	533	+ assert_eq!(format!("{}", escapes), "_\\t_\\n_\\r_\\_'_\"_");
	534	+ let others = BStr::from_bytes(b"\x01");
	535	+ assert_eq!(format!("{}", others), "\\x01");
	536	+ let non_ascii = BStr::from_bytes(b"d\xe9j\xe0 vu");
	537	+ assert_eq!(format!("{}", non_ascii), "d\\xe9j\\xe0 vu");
	538	+ let good_bytes = BStr::from_bytes(b"\xf0\x9f\xa6\x80");
	539	+ assert_eq!(format!("{}", good_bytes), "\\xf0\\x9f\\xa6\\x80");
	540	+ }
	541	+
	542	+ #[test]
	543	+ fn test_bstr_debug() {
	544	+ let hello_world = BStr::from_bytes(b"hello, world!");
	545	+ assert_eq!(format!("{:?}", hello_world), "\"hello, world!\"");
	546	+ let escapes = BStr::from_bytes(b"_\t_\n_\r_\\_\'_\"_");
	547	+ assert_eq!(format!("{:?}", escapes), "\"_\\t_\\n_\\r_\\\\_'_\\\"_\"");
	548	+ let others = BStr::from_bytes(b"\x01");
	549	+ assert_eq!(format!("{:?}", others), "\"\\x01\"");
	550	+ let non_ascii = BStr::from_bytes(b"d\xe9j\xe0 vu");
	551	+ assert_eq!(format!("{:?}", non_ascii), "\"d\\xe9j\\xe0 vu\"");
	552	+ let good_bytes = BStr::from_bytes(b"\xf0\x9f\xa6\x80");
	553	+ assert_eq!(format!("{:?}", good_bytes), "\"\\xf0\\x9f\\xa6\\x80\"");
	554	+ }
384	555	}
385	556
386	557	/// Allows formatting of [`fmt::Arguments`] into a raw buffer.
@@ -449,7 +620,7 @@ impl RawFormatter { · 62 unmodified lines
449	620	self.pos as _
450	621	}
451	622
452		− /// Return the number of bytes written to the formatter.
	623	+ /// Returns the number of bytes written to the formatter.
453	624	pub(crate) fn bytes_written(&self) -> usize {
454	625	self.pos - self.beg
455	626	}

rust/kernel/sync.rs+3−2

@@ -13,8 +13,9 @@ pub mod lock; · 12 unmodified lines
13	13	mod locked_by;
14	14
15	15	pub use arc::{Arc, ArcBorrow, UniqueArc};
16		−pub use condvar::CondVar;
17		−pub use lock::{mutex::Mutex, spinlock::SpinLock};
	16	+pub use condvar::{new_condvar, CondVar, CondVarTimeoutResult};
	17	+pub use lock::mutex::{new_mutex, Mutex};
	18	+pub use lock::spinlock::{new_spinlock, SpinLock};
18	19	pub use locked_by::LockedBy;
19	20
20	21	/// Represents a lockdep class. It's a wrapper around C's `lock_class_key`.

rust/kernel/sync/arc.rs+14−16

@@ -30,7 +30,7 @@ use core::{ · 29 unmodified lines
30	30	mem::{ManuallyDrop, MaybeUninit},
31	31	ops::{Deref, DerefMut},
32	32	pin::Pin,
33		− ptr::{NonNull, Pointee},
	33	+ ptr::NonNull,
34	34	};
35	35	use macros::pin_data;
36	36
@@ -56,7 +56,7 @@ mod std_vendor; · 19 unmodified lines
56	56	/// b: u32,
57	57	/// }
58	58	///
59		−/// // Create a ref-counted instance of `Example`.
	59	+/// // Create a refcounted instance of `Example`.
60	60	/// let obj = Arc::try_new(Example { a: 10, b: 20 })?;
61	61	///
62	62	/// // Get a new pointer to `obj` and increment the refcount.
@@ -239,22 +239,20 @@ impl<T: ?Sized> Arc<T> { · 176 unmodified lines
239	239	// binary, so its layout is not so large that it can trigger arithmetic overflow.
240	240	let val_offset = unsafe { refcount_layout.extend(val_layout).unwrap_unchecked().1 };
241	241
242		− let metadata: <T as Pointee>::Metadata = core::ptr::metadata(ptr);
243		− // SAFETY: The metadata of `T` and `ArcInner<T>` is the same because `ArcInner` is a struct
244		− // with `T` as its last field.
	242	+ // Pointer casts leave the metadata unchanged. This is okay because the metadata of `T` and
	243	+ // `ArcInner<T>` is the same since `ArcInner` is a struct with `T` as its last field.
245	244	//
246	245	// This is documented at:
247	246	// <https://doc.rust-lang.org/std/ptr/trait.Pointee.html>.
248		− let metadata: <ArcInner<T> as Pointee>::Metadata =
249		− unsafe { core::mem::transmute_copy(&metadata) };
	247	+ let ptr = ptr as *const ArcInner<T>;
	248	+
250	249	// SAFETY: The pointer is in-bounds of an allocation both before and after offsetting the
251	250	// pointer, since it originates from a previous call to `Arc::into_raw` and is still valid.
252		− let ptr = unsafe { (ptr as mut u8).sub(val_offset) as mut () };
253		− let ptr = core::ptr::from_raw_parts_mut(ptr, metadata);
	251	+ let ptr = unsafe { ptr.byte_sub(val_offset) };
254	252
255	253	// SAFETY: By the safety requirements we know that `ptr` came from `Arc::into_raw`, so the
256	254	// reference count held then will be owned by the new `Arc` object.
257		− unsafe { Self::from_inner(NonNull::new_unchecked(ptr)) }
	255	+ unsafe { Self::from_inner(NonNull::new_unchecked(ptr.cast_mut())) }
258	256	}
259	257
260	258	/// Returns an [`ArcBorrow`] from the given [`Arc`].
@@ -365,12 +363,12 @@ impl<T: ?Sized> From<Pin<UniqueArc<T>>> for Arc<T> { · 104 unmodified lines
365	363	/// A borrowed reference to an [`Arc`] instance.
366	364	///
367	365	/// For cases when one doesn't ever need to increment the refcount on the allocation, it is simpler
368		−/// to use just `&T`, which we can trivially get from an `Arc<T>` instance.
	366	+/// to use just `&T`, which we can trivially get from an [`Arc<T>`] instance.
369	367	///
370	368	/// However, when one may need to increment the refcount, it is preferable to use an `ArcBorrow<T>`
371	369	/// over `&Arc<T>` because the latter results in a double-indirection: a pointer (shared reference)
372		−/// to a pointer (`Arc<T>`) to the object (`T`). An [`ArcBorrow`] eliminates this double
373		−/// indirection while still allowing one to increment the refcount and getting an `Arc<T>` when/if
	370	+/// to a pointer ([`Arc<T>`]) to the object (`T`). An [`ArcBorrow`] eliminates this double
	371	+/// indirection while still allowing one to increment the refcount and getting an [`Arc<T>`] when/if
374	372	/// needed.
375	373	///
376	374	/// # Invariants
@@ -510,7 +508,7 @@ impl<T: ?Sized> Deref for ArcBorrow<'_, T> { · 133 unmodified lines
510	508	/// # test().unwrap();
511	509	/// ```
512	510	///
513		−/// In the following example we first allocate memory for a ref-counted `Example` but we don't
	511	+/// In the following example we first allocate memory for a refcounted `Example` but we don't
514	512	/// initialise it on allocation. We do initialise it later with a call to [`UniqueArc::write`],
515	513	/// followed by a conversion to `Arc<Example>`. This is particularly useful when allocation happens
516	514	/// in one context (e.g., sleepable) and initialisation in another (e.g., atomic):
@@ -560,7 +558,7 @@ impl<T> UniqueArc<T> { · 43 unmodified lines
560	558	/// Tries to allocate a new [`UniqueArc`] instance.
561	559	pub fn try_new(value: T) -> Result<Self, AllocError> {
562	560	Ok(Self {
563		− // INVARIANT: The newly-created object has a ref-count of 1.
	561	+ // INVARIANT: The newly-created object has a refcount of 1.
564	562	inner: Arc::try_new(value)?,
565	563	})
566	564	}
@@ -574,7 +572,7 @@ impl<T> UniqueArc<T> { · 7 unmodified lines
574	572	data <- init::uninit::<T, AllocError>(),
575	573	}? AllocError))?;
576	574	Ok(UniqueArc {
577		− // INVARIANT: The newly-created object has a ref-count of 1.
	575	+ // INVARIANT: The newly-created object has a refcount of 1.
578	576	// SAFETY: The pointer from the `Box` is valid.
579	577	inner: unsafe { Arc::from_inner(Box::leak(inner).into()) },
580	578	})

rust/kernel/sync/condvar.rs+88−22

@@ -6,8 +6,18 @@ · 5 unmodified lines
6	6	//! variable.
7	7
8	8	use super::{lock::Backend, lock::Guard, LockClassKey};
9		−use crate::{bindings, init::PinInit, pin_init, str::CStr, types::Opaque};
	9	+use crate::{
	10	+ bindings,
	11	+ init::PinInit,
	12	+ pin_init,
	13	+ str::CStr,
	14	+ task::{MAX_SCHEDULE_TIMEOUT, TASK_INTERRUPTIBLE, TASK_NORMAL, TASK_UNINTERRUPTIBLE},
	15	+ time::Jiffies,
	16	+ types::Opaque,
	17	+};
	18	+use core::ffi::{c_int, c_long};
10	19	use core::marker::PhantomPinned;
	20	+use core::ptr;
11	21	use macros::pin_data;
12	22
13	23	/// Creates a [`CondVar`] initialiser with the given name and a newly-created lock class.
@@ -17,6 +27,7 @@ macro_rules! new_condvar { · 3 unmodified lines
17	27	$crate::sync::CondVar::new($crate::optional_name!($($name)?), $crate::static_lock_class!())
18	28	};
19	29	}
	30	+pub use new_condvar;
20	31
21	32	/// A conditional variable.
22	33	///
@@ -34,8 +45,7 @@ macro_rules! new_condvar { · 11 unmodified lines
34	45	/// The following is an example of using a condvar with a mutex:
35	46	///
36	47	/// ```
37		−/// use kernel::sync::{CondVar, Mutex};
38		−/// use kernel::{new_condvar, new_mutex};
	48	+/// use kernel::sync::{new_condvar, new_mutex, CondVar, Mutex};
39	49	///
40	50	/// #[pin_data]
41	51	/// pub struct Example {
@@ -73,10 +83,12 @@ macro_rules! new_condvar { · 31 unmodified lines
73	83	#[pin_data]
74	84	pub struct CondVar {
75	85	#[pin]
76		− pub(crate) wait_list: Opaque<bindings::wait_queue_head>,
	86	+ pub(crate) wait_queue_head: Opaque<bindings::wait_queue_head>,
77	87
78	88	/// A condvar needs to be pinned because it contains a [`struct list_head`] that is
79	89	/// self-referential, so it cannot be safely moved once it is initialised.
	90	+ ///
	91	+ /// [`struct list_head`]: srctree/include/linux/types.h
80	92	#[pin]
81	93	_pin: PhantomPinned,
82	94	}
@@ -96,28 +108,35 @@ impl CondVar { · 13 unmodified lines
96	108	_pin: PhantomPinned,
97	109	// SAFETY: `slot` is valid while the closure is called and both `name` and `key` have
98	110	// static lifetimes so they live indefinitely.
99		− wait_list <- Opaque::ffi_init(\|slot\| unsafe {
	111	+ wait_queue_head <- Opaque::ffi_init(\|slot\| unsafe {
100	112	bindings::__init_waitqueue_head(slot, name.as_char_ptr(), key.as_ptr())
101	113	}),
102	114	})
103	115	}
104	116
105		− fn wait_internal<T: ?Sized, B: Backend>(&self, wait_state: u32, guard: &mut Guard<'_, T, B>) {
	117	+ fn wait_internal<T: ?Sized, B: Backend>(
	118	+ &self,
	119	+ wait_state: c_int,
	120	+ guard: &mut Guard<'_, T, B>,
	121	+ timeout_in_jiffies: c_long,
	122	+ ) -> c_long {
106	123	let wait = Opaque::<bindings::wait_queue_entry>::uninit();
107	124
108	125	// SAFETY: `wait` points to valid memory.
109	126	unsafe { bindings::init_wait(wait.get()) };
110	127
111		− // SAFETY: Both `wait` and `wait_list` point to valid memory.
	128	+ // SAFETY: Both `wait` and `wait_queue_head` point to valid memory.
112	129	unsafe {
113		− bindings::prepare_to_wait_exclusive(self.wait_list.get(), wait.get(), wait_state as _)
	130	+ bindings::prepare_to_wait_exclusive(self.wait_queue_head.get(), wait.get(), wait_state)
114	131	};
115	132
116		− // SAFETY: No arguments, switches to another thread.
117		− guard.do_unlocked(\|\| unsafe { bindings::schedule() });
	133	+ // SAFETY: Switches to another thread. The timeout can be any number.
	134	+ let ret = guard.do_unlocked(\|\| unsafe { bindings::schedule_timeout(timeout_in_jiffies) });
	135	+
	136	+ // SAFETY: Both `wait` and `wait_queue_head` point to valid memory.
	137	+ unsafe { bindings::finish_wait(self.wait_queue_head.get(), wait.get()) };
118	138
119		− // SAFETY: Both `wait` and `wait_list` point to valid memory.
120		− unsafe { bindings::finish_wait(self.wait_list.get(), wait.get()) };
	139	+ ret
121	140	}
122	141
123	142	/// Releases the lock and waits for a notification in uninterruptible mode.
@@ -127,7 +146,7 @@ impl CondVar { · 3 unmodified lines
127	146	/// [`CondVar::notify_one`] or [`CondVar::notify_all`]. Note that it may also wake up
128	147	/// spuriously.
129	148	pub fn wait<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) {
130		− self.wait_internal(bindings::TASK_UNINTERRUPTIBLE, guard);
	149	+ self.wait_internal(TASK_UNINTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT);
131	150	}
132	151
133	152	/// Releases the lock and waits for a notification in interruptible mode.
@@ -138,29 +157,60 @@ impl CondVar { · 4 unmodified lines
138	157	/// Returns whether there is a signal pending.
139	158	#[must_use = "wait_interruptible returns if a signal is pending, so the caller must check the return value"]
140	159	pub fn wait_interruptible<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) -> bool {
141		− self.wait_internal(bindings::TASK_INTERRUPTIBLE, guard);
	160	+ self.wait_internal(TASK_INTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT);
142	161	crate::current!().signal_pending()
143	162	}
144	163
145		− /// Calls the kernel function to notify the appropriate number of threads with the given flags.
146		− fn notify(&self, count: i32, flags: u32) {
147		− // SAFETY: `wait_list` points to valid memory.
	164	+ /// Releases the lock and waits for a notification in interruptible mode.
	165	+ ///
	166	+ /// Atomically releases the given lock (whose ownership is proven by the guard) and puts the
	167	+ /// thread to sleep. It wakes up when notified by [`CondVar::notify_one`] or
	168	+ /// [`CondVar::notify_all`], or when a timeout occurs, or when the thread receives a signal.
	169	+ #[must_use = "wait_interruptible_timeout returns if a signal is pending, so the caller must check the return value"]
	170	+ pub fn wait_interruptible_timeout<T: ?Sized, B: Backend>(
	171	+ &self,
	172	+ guard: &mut Guard<'_, T, B>,
	173	+ jiffies: Jiffies,
	174	+ ) -> CondVarTimeoutResult {
	175	+ let jiffies = jiffies.try_into().unwrap_or(MAX_SCHEDULE_TIMEOUT);
	176	+ let res = self.wait_internal(TASK_INTERRUPTIBLE, guard, jiffies);
	177	+
	178	+ match (res as Jiffies, crate::current!().signal_pending()) {
	179	+ (jiffies, true) => CondVarTimeoutResult::Signal { jiffies },
	180	+ (0, false) => CondVarTimeoutResult::Timeout,
	181	+ (jiffies, false) => CondVarTimeoutResult::Woken { jiffies },
	182	+ }
	183	+ }
	184	+
	185	+ /// Calls the kernel function to notify the appropriate number of threads.
	186	+ fn notify(&self, count: c_int) {
	187	+ // SAFETY: `wait_queue_head` points to valid memory.
148	188	unsafe {
149	189	bindings::__wake_up(
150		− self.wait_list.get(),
151		− bindings::TASK_NORMAL,
	190	+ self.wait_queue_head.get(),
	191	+ TASK_NORMAL,
152	192	count,
153		− flags as _,
	193	+ ptr::null_mut(),
154	194	)
155	195	};
156	196	}
157	197
	198	+ /// Calls the kernel function to notify one thread synchronously.
	199	+ ///
	200	+ /// This method behaves like `notify_one`, except that it hints to the scheduler that the
	201	+ /// current thread is about to go to sleep, so it should schedule the target thread on the same
	202	+ /// CPU.
	203	+ pub fn notify_sync(&self) {
	204	+ // SAFETY: `wait_queue_head` points to valid memory.
	205	+ unsafe { bindings::__wake_up_sync(self.wait_queue_head.get(), TASK_NORMAL) };
	206	+ }
	207	+
158	208	/// Wakes a single waiter up, if any.
159	209	///
160	210	/// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
161	211	/// completely (as opposed to automatically waking up the next waiter).
162	212	pub fn notify_one(&self) {
163		− self.notify(1, 0);
	213	+ self.notify(1);
164	214	}
165	215
166	216	/// Wakes all waiters up, if any.
@@ -168,6 +218,22 @@ impl CondVar { · 1 unmodified line
168	218	/// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
169	219	/// completely (as opposed to automatically waking up the next waiter).
170	220	pub fn notify_all(&self) {
171		− self.notify(0, 0);
	221	+ self.notify(0);
172	222	}
173	223	}
	224	+
	225	+/// The return type of `wait_timeout`.
	226	+pub enum CondVarTimeoutResult {
	227	+ /// The timeout was reached.
	228	+ Timeout,
	229	+ /// Somebody woke us up.
	230	+ Woken {
	231	+ /// Remaining sleep duration.
	232	+ jiffies: Jiffies,
	233	+ },
	234	+ /// A signal occurred.
	235	+ Signal {
	236	+ /// Remaining sleep duration.
	237	+ jiffies: Jiffies,
	238	+ },
	239	+}

rust/kernel/sync/lock.rs+13−6

@@ -21,14 +21,21 @@ pub mod spinlock; · 20 unmodified lines
21	21	/// # Safety
22	22	///
23	23	/// - Implementers must ensure that only one thread/CPU may access the protected data once the lock
24		−/// is owned, that is, between calls to `lock` and `unlock`.
25		−/// - Implementers must also ensure that `relock` uses the same locking method as the original
26		−/// lock operation.
	24	+/// is owned, that is, between calls to [`lock`] and [`unlock`].
	25	+/// - Implementers must also ensure that [`relock`] uses the same locking method as the original
	26	+/// lock operation.
	27	+///
	28	+/// [`lock`]: Backend::lock
	29	+/// [`unlock`]: Backend::unlock
	30	+/// [`relock`]: Backend::relock
27	31	pub unsafe trait Backend {
28	32	/// The state required by the lock.
29	33	type State;
30	34
31		− /// The state required to be kept between lock and unlock.
	35	+ /// The state required to be kept between [`lock`] and [`unlock`].
	36	+ ///
	37	+ /// [`lock`]: Backend::lock
	38	+ /// [`unlock`]: Backend::unlock
32	39	type GuardState;
33	40
34	41	/// Initialises the lock.
@@ -139,7 +146,7 @@ pub struct Guard<'a, T: ?Sized, B: Backend> { · 104 unmodified lines
139	146	unsafe impl<T: Sync + ?Sized, B: Backend> Sync for Guard<'_, T, B> {}
140	147
141	148	impl<T: ?Sized, B: Backend> Guard<'_, T, B> {
142		− pub(crate) fn do_unlocked(&mut self, cb: impl FnOnce()) {
	149	+ pub(crate) fn do_unlocked<U>(&mut self, cb: impl FnOnce() -> U) -> U {
143	150	// SAFETY: The caller owns the lock, so it is safe to unlock it.
144	151	unsafe { B::unlock(self.lock.state.get(), &self.state) };
145	152
@@ -147,7 +154,7 @@ impl<T: ?Sized, B: Backend> Guard<'_, T, B> { · 1 unmodified line
147	154	let _relock =
148	155	ScopeGuard::new(\|\| unsafe { B::relock(self.lock.state.get(), &mut self.state) });
149	156
150		− cb();
	157	+ cb()
151	158	}
152	159	}
153	160

rust/kernel/sync/lock/mutex.rs+2−1

@@ -17,6 +17,7 @@ macro_rules! new_mutex { · 16 unmodified lines
17	17	$inner, $crate::optional_name!($($name)?), $crate::static_lock_class!())
18	18	};
19	19	}
	20	+pub use new_mutex;
20	21
21	22	/// A mutual exclusion primitive.
22	23	///
@@ -35,7 +36,7 @@ macro_rules! new_mutex { · 12 unmodified lines
35	36	/// contains an inner struct (`Inner`) that is protected by a mutex.
36	37	///
37	38	/// ```
38		−/// use kernel::{init::InPlaceInit, init::PinInit, new_mutex, pin_init, sync::Mutex};
	39	+/// use kernel::sync::{new_mutex, Mutex};
39	40	///
40	41	/// struct Inner {
41	42	/// a: u32,

rust/kernel/sync/lock/spinlock.rs+3−2

@@ -17,6 +17,7 @@ macro_rules! new_spinlock { · 16 unmodified lines
17	17	$inner, $crate::optional_name!($($name)?), $crate::static_lock_class!())
18	18	};
19	19	}
	20	+pub use new_spinlock;
20	21
21	22	/// A spinlock.
22	23	///
@@ -33,7 +34,7 @@ macro_rules! new_spinlock { · 10 unmodified lines
33	34	/// contains an inner struct (`Inner`) that is protected by a spinlock.
34	35	///
35	36	/// ```
36		−/// use kernel::{init::InPlaceInit, init::PinInit, new_spinlock, pin_init, sync::SpinLock};
	37	+/// use kernel::sync::{new_spinlock, SpinLock};
37	38	///
38	39	/// struct Inner {
39	40	/// a: u32,
@@ -112,7 +113,7 @@ unsafe impl super::Backend for SpinLockBackend { · 72 unmodified lines
112	113
113	114	unsafe fn unlock(ptr: *mut Self::State, _guard_state: &Self::GuardState) {
114	115	// SAFETY: The safety requirements of this function ensure that `ptr` is valid and that the
115		− // caller is the owner of the mutex.
	116	+ // caller is the owner of the spinlock.
116	117	unsafe { bindings::spin_unlock(ptr) }
117	118	}
118	119	}

rust/kernel/sync/locked_by.rs+5−2

@@ -9,14 +9,17 @@ use core::{cell::UnsafeCell, mem::size_of, ptr}; · 8 unmodified lines
9	9	/// Allows access to some data to be serialised by a lock that does not wrap it.
10	10	///
11	11	/// In most cases, data protected by a lock is wrapped by the appropriate lock type, e.g.,
12		−/// [`super::Mutex`] or [`super::SpinLock`]. [`LockedBy`] is meant for cases when this is not
13		−/// possible. For example, if a container has a lock and some data in the contained elements needs
	12	+/// [`Mutex`] or [`SpinLock`]. [`LockedBy`] is meant for cases when this is not possible.
	13	+/// For example, if a container has a lock and some data in the contained elements needs
14	14	/// to be protected by the same lock.
15	15	///
16	16	/// [`LockedBy`] wraps the data in lieu of another locking primitive, and only allows access to it
17	17	/// when the caller shows evidence that the 'external' lock is locked. It panics if the evidence
18	18	/// refers to the wrong instance of the lock.
19	19	///
	20	+/// [`Mutex`]: super::Mutex
	21	+/// [`SpinLock`]: super::SpinLock
	22	+///
20	23	/// # Examples
21	24	///
22	25	/// The following is an example for illustrative purposes: `InnerDirectory::bytes_used` is an

rust/kernel/task.rs+20−4

@@ -5,7 +5,23 @@ · 4 unmodified lines
5	5	//! C header: [`include/linux/sched.h`](srctree/include/linux/sched.h).
6	6
7	7	use crate::{bindings, types::Opaque};
8		−use core::{marker::PhantomData, ops::Deref, ptr};
	8	+use core::{
	9	+ ffi::{c_int, c_long, c_uint},
	10	+ marker::PhantomData,
	11	+ ops::Deref,
	12	+ ptr,
	13	+};
	14	+
	15	+/// A sentinel value used for infinite timeouts.
	16	+pub const MAX_SCHEDULE_TIMEOUT: c_long = c_long::MAX;
	17	+
	18	+/// Bitmask for tasks that are sleeping in an interruptible state.
	19	+pub const TASK_INTERRUPTIBLE: c_int = bindings::TASK_INTERRUPTIBLE as c_int;
	20	+/// Bitmask for tasks that are sleeping in an uninterruptible state.
	21	+pub const TASK_UNINTERRUPTIBLE: c_int = bindings::TASK_UNINTERRUPTIBLE as c_int;
	22	+/// Convenience constant for waking up tasks regardless of whether they are in interruptible or
	23	+/// uninterruptible sleep.
	24	+pub const TASK_NORMAL: c_uint = bindings::TASK_NORMAL as c_uint;
9	25
10	26	/// Returns the currently running task.
11	27	#[macro_export]
@@ -23,7 +39,7 @@ macro_rules! current { · 11 unmodified lines
23	39	///
24	40	/// All instances are valid tasks created by the C portion of the kernel.
25	41	///
26		−/// Instances of this type are always ref-counted, that is, a call to `get_task_struct` ensures
	42	+/// Instances of this type are always refcounted, that is, a call to `get_task_struct` ensures
27	43	/// that the allocation remains valid at least until the matching call to `put_task_struct`.
28	44	///
29	45	/// # Examples
@@ -116,7 +132,7 @@ impl Task { · 86 unmodified lines
116	132	/// Returns the group leader of the given task.
117	133	pub fn group_leader(&self) -> &Task {
118	134	// SAFETY: By the type invariant, we know that `self.0` is a valid task. Valid tasks always
119		− // have a valid group_leader.
	135	+ // have a valid `group_leader`.
120	136	let ptr = unsafe { ptr::addr_of!((self.0.get()).group_leader) };
121	137
122	138	// SAFETY: The lifetime of the returned task reference is tied to the lifetime of `self`,
@@ -147,7 +163,7 @@ impl Task { · 24 unmodified lines
147	163	}
148	164	}
149	165
150		−// SAFETY: The type invariants guarantee that `Task` is always ref-counted.
	166	+// SAFETY: The type invariants guarantee that `Task` is always refcounted.
151	167	unsafe impl crate::types::AlwaysRefCounted for Task {
152	168	fn inc_ref(&self) {
153	169	// SAFETY: The existence of a shared reference means that the refcount is nonzero.

rust/kernel/time.rs+20−0

@@ -0,0 +1,20 @@
	1	+// SPDX-License-Identifier: GPL-2.0
	2	+
	3	+//! Time related primitives.
	4	+//!
	5	+//! This module contains the kernel APIs related to time and timers that
	6	+//! have been ported or wrapped for usage by Rust code in the kernel.
	7	+
	8	+/// The time unit of Linux kernel. One jiffy equals (1/HZ) second.
	9	+pub type Jiffies = core::ffi::c_ulong;
	10	+
	11	+/// The millisecond time unit.
	12	+pub type Msecs = core::ffi::c_uint;
	13	+
	14	+/// Converts milliseconds to jiffies.
	15	+#[inline]
	16	+pub fn msecs_to_jiffies(msecs: Msecs) -> Jiffies {
	17	+ // SAFETY: The `__msecs_to_jiffies` function is always safe to call no
	18	+ // matter what the argument is.
	19	+ unsafe { bindings::__msecs_to_jiffies(msecs) }
	20	+}

rust/kernel/types.rs+22−0

@@ -46,6 +46,25 @@ pub trait ForeignOwnable: Sized { · 45 unmodified lines
46	46	/// Additionally, all instances (if any) of values returned by [`ForeignOwnable::borrow`] for
47	47	/// this object must have been dropped.
48	48	unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self;
	49	+
	50	+ /// Tries to convert a foreign-owned object back to a Rust-owned one.
	51	+ ///
	52	+ /// A convenience wrapper over [`ForeignOwnable::from_foreign`] that returns [`None`] if `ptr`
	53	+ /// is null.
	54	+ ///
	55	+ /// # Safety
	56	+ ///
	57	+ /// `ptr` must either be null or satisfy the safety requirements for
	58	+ /// [`ForeignOwnable::from_foreign`].
	59	+ unsafe fn try_from_foreign(ptr: *const core::ffi::c_void) -> Option<Self> {
	60	+ if ptr.is_null() {
	61	+ None
	62	+ } else {
	63	+ // SAFETY: Since `ptr` is not null here, then `ptr` satisfies the safety requirements
	64	+ // of `from_foreign` given the safety requirements of this function.
	65	+ unsafe { Some(Self::from_foreign(ptr)) }
	66	+ }
	67	+ }
49	68	}
50	69
51	70	impl<T: 'static> ForeignOwnable for Box<T> {
@@ -90,6 +109,7 @@ impl ForeignOwnable for () { · 38 unmodified lines
90	109	///
91	110	/// In the example below, we have multiple exit paths and we want to log regardless of which one is
92	111	/// taken:
	112	+///
93	113	/// ```
94	114	/// # use kernel::types::ScopeGuard;
95	115	/// fn example1(arg: bool) {
@@ -108,6 +128,7 @@ impl ForeignOwnable for () { · 12 unmodified lines
108	128	///
109	129	/// In the example below, we want to log the same message on all early exits but a different one on
110	130	/// the main exit path:
	131	+///
111	132	/// ```
112	133	/// # use kernel::types::ScopeGuard;
113	134	/// fn example2(arg: bool) {
@@ -129,6 +150,7 @@ impl ForeignOwnable for () { · 15 unmodified lines
129	150	///
130	151	/// In the example below, we need a mutable object (the vector) to be accessible within the log
131	152	/// function, so we wrap it in the [`ScopeGuard`]:
	153	+///
132	154	/// ```
133	155	/// # use kernel::types::ScopeGuard;
134	156	/// fn example3(arg: bool) -> Result {

rust/kernel/workqueue.rs+38−40

@@ -12,19 +12,19 @@ · 11 unmodified lines
12	12	//!
13	13	//! # The raw API
14	14	//!
15		−//! The raw API consists of the `RawWorkItem` trait, where the work item needs to provide an
	15	+//! The raw API consists of the [`RawWorkItem`] trait, where the work item needs to provide an
16	16	//! arbitrary function that knows how to enqueue the work item. It should usually not be used
17	17	//! directly, but if you want to, you can use it without using the pieces from the safe API.
18	18	//!
19	19	//! # The safe API
20	20	//!
21		−//! The safe API is used via the `Work` struct and `WorkItem` traits. Furthermore, it also includes
22		−//! a trait called `WorkItemPointer`, which is usually not used directly by the user.
	21	+//! The safe API is used via the [`Work`] struct and [`WorkItem`] traits. Furthermore, it also
	22	+//! includes a trait called [`WorkItemPointer`], which is usually not used directly by the user.
23	23	//!
24		−//! * The `Work` struct is the Rust wrapper for the C `work_struct` type.
25		−//! * The `WorkItem` trait is implemented for structs that can be enqueued to a workqueue.
26		−//! * The `WorkItemPointer` trait is implemented for the pointer type that points at a something
27		−//! that implements `WorkItem`.
	24	+//! * The [`Work`] struct is the Rust wrapper for the C `work_struct` type.
	25	+//! * The [`WorkItem`] trait is implemented for structs that can be enqueued to a workqueue.
	26	+//! * The [`WorkItemPointer`] trait is implemented for the pointer type that points at a something
	27	+//! that implements [`WorkItem`].
28	28	//!
29	29	//! ## Example
30	30	//!
@@ -35,8 +35,7 @@ · 4 unmodified lines
35	35	//! ```
36	36	//! use kernel::prelude::*;
37	37	//! use kernel::sync::Arc;
38		−//! use kernel::workqueue::{self, Work, WorkItem};
39		−//! use kernel::{impl_has_work, new_work};
	38	+//! use kernel::workqueue::{self, impl_has_work, new_work, Work, WorkItem};
40	39	//!
41	40	//! #[pin_data]
42	41	//! struct MyStruct {
@@ -78,8 +77,7 @@ · 35 unmodified lines
78	77	//! ```
79	78	//! use kernel::prelude::*;
80	79	//! use kernel::sync::Arc;
81		−//! use kernel::workqueue::{self, Work, WorkItem};
82		−//! use kernel::{impl_has_work, new_work};
	80	+//! use kernel::workqueue::{self, impl_has_work, new_work, Work, WorkItem};
83	81	//!
84	82	//! #[pin_data]
85	83	//! struct MyStruct {
@@ -147,6 +145,7 @@ macro_rules! new_work { · 61 unmodified lines
147	145	$crate::workqueue::Work::new($crate::optional_name!($($name)?), $crate::static_lock_class!())
148	146	};
149	147	}
	148	+pub use new_work;
150	149
151	150	/// A kernel work queue.
152	151	///
@@ -168,7 +167,7 @@ impl Queue { · 15 unmodified lines
168	167	/// # Safety
169	168	///
170	169	/// The caller must ensure that the provided raw pointer is not dangling, that it points at a
171		− /// valid workqueue, and that it remains valid until the end of 'a.
	170	+ /// valid workqueue, and that it remains valid until the end of `'a`.
172	171	pub unsafe fn from_raw<'a>(ptr: *const bindings::workqueue_struct) -> &'a Queue {
173	172	// SAFETY: The `Queue` type is `#[repr(transparent)]`, so the pointer cast is valid. The
174	173	// caller promises that the pointer is not dangling.
@@ -218,7 +217,9 @@ impl Queue { · 43 unmodified lines
218	217	}
219	218	}
220	219
221		−/// A helper type used in `try_spawn`.
	220	+/// A helper type used in [`try_spawn`].
	221	+///
	222	+/// [`try_spawn`]: Queue::try_spawn
222	223	#[pin_data]
223	224	struct ClosureWork<T> {
224	225	#[pin]
@@ -253,14 +254,16 @@ impl<T: FnOnce()> WorkItem for ClosureWork<T> { · 28 unmodified lines
253	254	/// actual value of the id is not important as long as you use different ids for different fields
254	255	/// of the same struct. (Fields of different structs need not use different ids.)
255	256	///
256		−/// Note that the id is used only to select the right method to call during compilation. It wont be
	257	+/// Note that the id is used only to select the right method to call during compilation. It won't be
257	258	/// part of the final executable.
258	259	///
259	260	/// # Safety
260	261	///
261		−/// Implementers must ensure that any pointers passed to a `queue_work_on` closure by `__enqueue`
	262	+/// Implementers must ensure that any pointers passed to a `queue_work_on` closure by [`__enqueue`]
262	263	/// remain valid for the duration specified in the guarantees section of the documentation for
263		−/// `__enqueue`.
	264	+/// [`__enqueue`].
	265	+///
	266	+/// [`__enqueue`]: RawWorkItem::__enqueue
264	267	pub unsafe trait RawWorkItem<const ID: u64> {
265	268	/// The return type of [`Queue::enqueue`].
266	269	type EnqueueOutput;
@@ -290,10 +293,11 @@ pub unsafe trait RawWorkItem<const ID: u64> { · 23 unmodified lines
290	293
291	294	/// Defines the method that should be called directly when a work item is executed.
292	295	///
293		−/// This trait is implemented by `Pin<Box<T>>` and `Arc<T>`, and is mainly intended to be
	296	+/// This trait is implemented by `Pin<Box<T>>` and [`Arc<T>`], and is mainly intended to be
294	297	/// implemented for smart pointer types. For your own structs, you would implement [`WorkItem`]
295		−/// instead. The `run` method on this trait will usually just perform the appropriate
296		−/// `container_of` translation and then call into the `run` method from the [`WorkItem`] trait.
	298	+/// instead. The [`run`] method on this trait will usually just perform the appropriate
	299	+/// `container_of` translation and then call into the [`run`][WorkItem::run] method from the
	300	+/// [`WorkItem`] trait.
297	301	///
298	302	/// This trait is used when the `work_struct` field is defined using the [`Work`] helper.
299	303	///
@@ -309,8 +313,10 @@ pub unsafe trait WorkItemPointer<const ID: u64>: RawWorkItem<ID> { · 9 unmodified lines
309	313	///
310	314	/// # Safety
311	315	///
312		− /// The provided `work_struct` pointer must originate from a previous call to `__enqueue` where
313		− /// the `queue_work_on` closure returned true, and the pointer must still be valid.
	316	+ /// The provided `work_struct` pointer must originate from a previous call to [`__enqueue`]
	317	+ /// where the `queue_work_on` closure returned true, and the pointer must still be valid.
	318	+ ///
	319	+ /// [`__enqueue`]: RawWorkItem::__enqueue
314	320	unsafe extern "C" fn run(ptr: *mut bindings::work_struct);
315	321	}
316	322
@@ -328,12 +334,14 @@ pub trait WorkItem<const ID: u64 = 0> { · 11 unmodified lines
328	334
329	335	/// Links for a work item.
330	336	///
331		−/// This struct contains a function pointer to the `run` function from the [`WorkItemPointer`]
	337	+/// This struct contains a function pointer to the [`run`] function from the [`WorkItemPointer`]
332	338	/// trait, and defines the linked list pointers necessary to enqueue a work item in a workqueue.
333	339	///
334	340	/// Wraps the kernel's C `struct work_struct`.
335	341	///
336	342	/// This is a helper type used to associate a `work_struct` with the [`WorkItem`] that uses it.
	343	+///
	344	+/// [`run`]: WorkItemPointer::run
337	345	#[repr(transparent)]
338	346	pub struct Work<T: ?Sized, const ID: u64 = 0> {
339	347	work: Opaque<bindings::work_struct>,
@@ -396,9 +404,8 @@ impl<T: ?Sized, const ID: u64> Work<T, ID> { · 56 unmodified lines
396	404	/// like this:
397	405	///
398	406	/// ```no_run
399		−/// use kernel::impl_has_work;
400	407	/// use kernel::prelude::*;
401		−/// use kernel::workqueue::Work;
	408	+/// use kernel::workqueue::{impl_has_work, Work};
402	409	///
403	410	/// struct MyWorkItem {
404	411	/// work_field: Work<MyWorkItem, 1>,
@@ -409,28 +416,25 @@ impl<T: ?Sized, const ID: u64> Work<T, ID> { · 4 unmodified lines
409	416	/// }
410	417	/// ```
411	418	///
412		−/// Note that since the `Work` type is annotated with an id, you can have several `work_struct`
	419	+/// Note that since the [`Work`] type is annotated with an id, you can have several `work_struct`
413	420	/// fields by using a different id for each one.
414	421	///
415	422	/// # Safety
416	423	///
417		−/// The [`OFFSET`] constant must be the offset of a field in Self of type [`Work<T, ID>`]. The methods on
418		−/// this trait must have exactly the behavior that the definitions given below have.
	424	+/// The [`OFFSET`] constant must be the offset of a field in `Self` of type [`Work<T, ID>`]. The
	425	+/// methods on this trait must have exactly the behavior that the definitions given below have.
419	426	///
420		−/// [`Work<T, ID>`]: Work
421	427	/// [`impl_has_work!`]: crate::impl_has_work
422	428	/// [`OFFSET`]: HasWork::OFFSET
423	429	pub unsafe trait HasWork<T, const ID: u64 = 0> {
424	430	/// The offset of the [`Work<T, ID>`] field.
425		− ///
426		− /// [`Work<T, ID>`]: Work
427	431	const OFFSET: usize;
428	432
429	433	/// Returns the offset of the [`Work<T, ID>`] field.
430	434	///
431		− /// This method exists because the [`OFFSET`] constant cannot be accessed if the type is not Sized.
	435	+ /// This method exists because the [`OFFSET`] constant cannot be accessed if the type is not
	436	+ /// [`Sized`].
432	437	///
433		− /// [`Work<T, ID>`]: Work
434	438	/// [`OFFSET`]: HasWork::OFFSET
435	439	#[inline]
436	440	fn get_work_offset(&self) -> usize {
@@ -442,8 +446,6 @@ pub unsafe trait HasWork<T, const ID: u64 = 0> { · 5 unmodified lines
442	446	/// # Safety
443	447	///
444	448	/// The provided pointer must point at a valid struct of type `Self`.
445		− ///
446		− /// [`Work<T, ID>`]: Work
447	449	#[inline]
448	450	unsafe fn raw_get_work(ptr: mut Self) -> mut Work<T, ID> {
449	451	// SAFETY: The caller promises that the pointer is valid.
@@ -455,8 +457,6 @@ pub unsafe trait HasWork<T, const ID: u64 = 0> { · 5 unmodified lines
455	457	/// # Safety
456	458	///
457	459	/// The pointer must point at a [`Work<T, ID>`] field in a struct of type `Self`.
458		− ///
459		− /// [`Work<T, ID>`]: Work
460	460	#[inline]
461	461	unsafe fn work_container_of(ptr: mut Work<T, ID>) -> mut Self
462	462	where
@@ -473,9 +473,8 @@ pub unsafe trait HasWork<T, const ID: u64 = 0> { · 10 unmodified lines
473	473	/// # Examples
474	474	///
475	475	/// ```
476		−/// use kernel::impl_has_work;
477	476	/// use kernel::sync::Arc;
478		−/// use kernel::workqueue::{self, Work};
	477	+/// use kernel::workqueue::{self, impl_has_work, Work};
479	478	///
480	479	/// struct MyStruct {
481	480	/// work_field: Work<MyStruct, 17>,
@@ -485,8 +484,6 @@ pub unsafe trait HasWork<T, const ID: u64 = 0> { · 3 unmodified lines
485	484	/// impl HasWork<MyStruct, 17> for MyStruct { self.work_field }
486	485	/// }
487	486	/// ```
488		−///
489		−/// [`HasWork<T, ID>`]: HasWork
490	487	#[macro_export]
491	488	macro_rules! impl_has_work {
492	489	($(impl$(<$($implarg:ident),*>)?
@@ -509,6 +506,7 @@ macro_rules! impl_has_work { · 16 unmodified lines
509	506	}
510	507	)*};
511	508	}
	509	+pub use impl_has_work;
512	510
513	511	impl_has_work! {
514	512	impl<T> HasWork<Self> for ClosureWork<T> { self.work }

rust/macros/module.rs+6−1

@@ -222,10 +222,15 @@ pub(crate) fn module(ts: TokenStream) -> TokenStream { · 221 unmodified lines
222	222	}};
223	223
224	224	// Loadable modules need to export the `{{init,cleanup}}_module` identifiers.
	225	+ /// # Safety
	226	+ ///
	227	+ /// This function must not be called after module initialization, because it may be
	228	+ /// freed after that completes.
225	229	#[cfg(MODULE)]
226	230	#[doc(hidden)]
227	231	#[no_mangle]
228		− pub extern \"C\" fn init_module() -> core::ffi::c_int {{
	232	+ #[link_section = \".init.text\"]
	233	+ pub unsafe extern \"C\" fn init_module() -> core::ffi::c_int {{
229	234	__init()
230	235	}}
231	236

scripts/Makefile.build+4−4

@@ -290,7 +290,7 @@ quiet_cmd_rustc_o_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ · 289 unmodified lines
290	290	cmd_rustc_o_rs = $(rust_common_cmd) --emit=obj=$@ $<
291	291
292	292	$(obj)/%.o: $(src)/%.rs FORCE
293		− $(call if_changed_dep,rustc_o_rs)
	293	+ +$(call if_changed_dep,rustc_o_rs)
294	294
295	295	quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
296	296	cmd_rustc_rsi_rs = \
@@ -298,19 +298,19 @@ quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ · 1 unmodified line
298	298	command -v $(RUSTFMT) >/dev/null && $(RUSTFMT) $@
299	299
300	300	$(obj)/%.rsi: $(src)/%.rs FORCE
301		− $(call if_changed_dep,rustc_rsi_rs)
	301	+ +$(call if_changed_dep,rustc_rsi_rs)
302	302
303	303	quiet_cmd_rustc_s_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
304	304	cmd_rustc_s_rs = $(rust_common_cmd) --emit=asm=$@ $<
305	305
306	306	$(obj)/%.s: $(src)/%.rs FORCE
307		− $(call if_changed_dep,rustc_s_rs)
	307	+ +$(call if_changed_dep,rustc_s_rs)
308	308
309	309	quiet_cmd_rustc_ll_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
310	310	cmd_rustc_ll_rs = $(rust_common_cmd) --emit=llvm-ir=$@ $<
311	311
312	312	$(obj)/%.ll: $(src)/%.rs FORCE
313		− $(call if_changed_dep,rustc_ll_rs)
	313	+ +$(call if_changed_dep,rustc_ll_rs)
314	314
315	315	# Compile assembler sources (.S)
316	316	# ---------------------------------------------------------------------------

scripts/Makefile.host+1−1

@@ -156,7 +156,7 @@ quiet_cmd_host-rust = HOSTRUSTC $@ · 155 unmodified lines
156	156	cmd_host-rust = \
157	157	$(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $<
158	158	$(host-rust): $(obj)/%: $(src)/%.rs FORCE
159		− $(call if_changed_dep,host-rust)
	159	+ +$(call if_changed_dep,host-rust)
160	160
161	161	targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
162	162	$(host-cxxmulti) $(host-cxxobjs) $(host-rust)

scripts/min-tool-version.sh+1−1

@@ -33,7 +33,7 @@ llvm) · 32 unmodified lines
33	33	fi
34	34	;;
35	35	rustc)
36		− echo 1.74.1
	36	+ echo 1.76.0
37	37	;;
38	38	bindgen)
39	39	echo 0.65.1