History

Merge tag 'rust-6.11' of https://github.com/Rust-for-Linux/linux

Pull Rust updates from Miguel Ojeda: "The highlight is the establishment of a minimum version for the Rust toolchain, including 'rustc' (and bundled tools) and 'bindgen'. The initial minimum will be the pinned version we currently have, i.e. we are just widening the allowed versions. That covers three stable Rust releases: 1.78.0, 1.79.0, 1.80.0 (getting released tomorrow), plus beta, plus nightly. This should already be enough for kernel developers in distributions that provide recent Rust compiler versions routinely, such as Arch Linux, Debian Unstable (outside the freeze period), Fedora Linux, Gentoo Linux (especially the testing channel), Nix (unstable) and openSUSE Slowroll and Tumbleweed. In addition, the kernel is now being built-tested by Rust's pre-merge CI. That is, every change that is attempting to land into the Rust compiler is tested against the kernel, and it is merged only if it passes. Similarly, the bindgen tool has agreed to build the kernel in their CI too. Thus, with the pre-merge CI in place, both projects hope to avoid unintentional changes to Rust that break the kernel. This means that, in general, apart from intentional changes on their side (that we will need to workaround conditionally on our side), the upcoming Rust compiler versions should generally work. In addition, the Rust project has proposed getting the kernel into stable Rust (at least solving the main blockers) as one of its three flagship goals for 2024H2 [1]. I would like to thank Niko, Sid, Emilio et al. for their help promoting the collaboration between Rust and the kernel. Toolchain and infrastructure: - Support several Rust toolchain versions. - Support several bindgen versions. - Remove 'cargo' requirement and simplify 'rusttest', thanks to 'alloc' having been dropped last cycle. - Provide proper error reporting for the 'rust-analyzer' target. 'kernel' crate: - Add 'uaccess' module with a safe userspace pointers abstraction. - Add 'page' module with a 'struct page' abstraction. - Support more complex generics in workqueue's 'impl_has_work!' macro. 'macros' crate: - Add 'firmware' field support to the 'module!' macro. - Improve 'module!' macro documentation. Documentation: - Provide instructions on what packages should be installed to build the kernel in some popular Linux distributions. - Introduce the new kernel.org LLVM+Rust toolchains. - Explain '#[no_std]'. And a few other small bits" Link: https://rust-lang.github.io/rust-project-goals/2024h2/index.html#flagship-goals [1] * tag 'rust-6.11' of https://github.com/Rust-for-Linux/linux: (26 commits) docs: rust: quick-start: add section on Linux distributions rust: warn about `bindgen` versions 0.66.0 and 0.66.1 rust: start supporting several `bindgen` versions rust: work around `bindgen` 0.69.0 issue rust: avoid assuming a particular `bindgen` build rust: start supporting several compiler versions rust: simplify Clippy warning flags set rust: relax most deny-level lints to warnings rust: allow `dead_code` for never constructed bindings rust: init: simplify from `map_err` to `inspect_err` rust: macros: indent list item in `paste!`'s docs rust: add abstraction for `struct page` rust: uaccess: add typed accessors for userspace pointers uaccess: always export _copy_[from|to]_user with CONFIG_RUST rust: uaccess: add userspace pointers kbuild: rust-analyzer: improve comment documentation kbuild: rust-analyzer: better error handling docs: rust: no_std is used rust: alloc: add __GFP_HIGHMEM flag rust: alloc: fix typo in docs for GFP_NOWAIT ...

Linus Torvalds committed 2 years ago

Commit 910bfc22 parents ff30564 b12634125 files · +1058 −236Browse files

Unified · Split

25 files
  1. Documentation/process/changes.rst +1 −8
  2. Documentation/rust/general-information.rst +8 −0
  3. Documentation/rust/quick-start.rst +100 −43
  4. Documentation/rust/testing.rst +2 −3
  5. Makefile +16 −14
  6. include/linux/uaccess.h +32 −14
  7. init/Kconfig +4 −1
  8. lib/usercopy.c +4 −26
  9. rust/Makefile +10 −64
  10. rust/bindings/bindings_helper.h +1 −0
  11. rust/bindings/lib.rs +1 −0
  12. rust/helpers.c +34 −0
  13. rust/kernel/alloc.rs +16 −1
  14. rust/kernel/init.rs +4 −9
  15. rust/kernel/lib.rs +2 −0
  16. rust/kernel/page.rs +250 −0
  17. rust/kernel/types.rs +64 −0
  18. rust/kernel/uaccess.rs +388 −0
  19. rust/kernel/workqueue.rs +9 −7
  20. rust/macros/lib.rs +39 −6
  21. rust/macros/module.rs +16 −2
  22. rust/uapi/lib.rs +1 −0
  23. scripts/rust_is_available.sh +17 −16
  24. scripts/rust_is_available_bindgen_0_66.h +2 −0
  25. scripts/rust_is_available_test.py +37 −22
Documentation/process/changes.rst+1−8
@@ -89,14 +89,7 @@ docs on :ref:`Building Linux with Clang/LLVM <kbuild_llvm>`. · 88 unmodified lines
89Rust (optional)89Rust (optional)
90---------------90---------------
91 91
92A particular version of the Rust toolchain is required. Newer versions may or92A recent version of the Rust compiler is required.
93may not work because the kernel depends on some unstable Rust features, for
94the moment.
95
96Each Rust toolchain comes with several "components", some of which are required
97(like ``rustc``) and some that are optional. The ``rust-src`` component (which
98is optional) needs to be installed to build the kernel. Other components are
99useful for developing.
100 93
101Please see Documentation/rust/quick-start.rst for instructions on how to94Please see Documentation/rust/quick-start.rst for instructions on how to
102satisfy the build requirements of Rust support. In particular, the ``Makefile``95satisfy the build requirements of Rust support. In particular, the ``Makefile``
Documentation/rust/general-information.rst+8−0
@@ -7,6 +7,14 @@ This document contains useful information to know when working with · 6 unmodified lines
7the Rust support in the kernel.7the Rust support in the kernel.
8 8
9 9
10``no_std``
11----------
12
13The Rust support in the kernel can link only `core <https://doc.rust-lang.org/core/>`_,
14but not `std <https://doc.rust-lang.org/std/>`_. Crates for use in the
15kernel must opt into this behavior using the ``#![no_std]`` attribute.
16
17
10Code documentation18Code documentation
11------------------19------------------
12 20
Documentation/rust/quick-start.rst+100−43
@@ -5,17 +5,93 @@ Quick Start · 4 unmodified lines
5 5
6This document describes how to get started with kernel development in Rust.6This document describes how to get started with kernel development in Rust.
7 7
8There are a few ways to install a Rust toolchain needed for kernel development.
9A simple way is to use the packages from your Linux distribution if they are
10suitable -- the first section below explains this approach. An advantage of this
11approach is that, typically, the distribution will match the LLVM used by Rust
12and Clang.
13
14Another way is using the prebuilt stable versions of LLVM+Rust provided on
15`kernel.org <https://kernel.org/pub/tools/llvm/rust/>`_. These are the same slim
16and fast LLVM toolchains from :ref:`Getting LLVM <getting_llvm>` with versions
17of Rust added to them that Rust for Linux supports. Two sets are provided: the
18"latest LLVM" and "matching LLVM" (please see the link for more information).
19
20Alternatively, the next two "Requirements" sections explain each component and
21how to install them through ``rustup``, the standalone installers from Rust
22and/or building them.
23
24The rest of the document explains other aspects on how to get started.
25
26
27Distributions
28-------------
29
30Arch Linux
31**********
32
33Arch Linux provides recent Rust releases and thus it should generally work out
34of the box, e.g.::
35
36 pacman -S rust rust-src rust-bindgen
37
38
39Debian
40******
41
42Debian Unstable (Sid), outside of the freeze period, provides recent Rust
43releases and thus it should generally work out of the box, e.g.::
44
45 apt install rustc rust-src bindgen rustfmt rust-clippy
46
47
48Fedora Linux
49************
50
51Fedora Linux provides recent Rust releases and thus it should generally work out
52of the box, e.g.::
53
54 dnf install rust rust-src bindgen-cli rustfmt clippy
55
56
57Gentoo Linux
58************
59
60Gentoo Linux (and especially the testing branch) provides recent Rust releases
61and thus it should generally work out of the box, e.g.::
62
63 USE='rust-src rustfmt clippy' emerge dev-lang/rust dev-util/bindgen
64
65``LIBCLANG_PATH`` may need to be set.
66
67
68Nix
69***
70
71Nix (unstable channel) provides recent Rust releases and thus it should
72generally work out of the box, e.g.::
73
74 { pkgs ? import <nixpkgs> {} }:
75 pkgs.mkShell {
76 nativeBuildInputs = with pkgs; [ rustc rust-bindgen rustfmt clippy ];
77 RUST_LIB_SRC = "${pkgs.rust.packages.stable.rustPlatform.rustLibSrc}";
78 }
79
80
81openSUSE
82********
83
84openSUSE Slowroll and openSUSE Tumbleweed provide recent Rust releases and thus
85they should generally work out of the box, e.g.::
86
87 zypper install rust rust1.79-src rust-bindgen clang
88
8 89
9Requirements: Building90Requirements: Building
10----------------------91----------------------
11 92
12This section explains how to fetch the tools needed for building.93This section explains how to fetch the tools needed for building.
13 94
14Some of these requirements might be available from Linux distributions
15under names like ``rustc``, ``rust-src``, ``rust-bindgen``, etc. However,
16at the time of writing, they are likely not to be recent enough unless
17the distribution tracks the latest releases.
18
19To easily check whether the requirements are met, the following target95To easily check whether the requirements are met, the following target
20can be used::96can be used::
21 97
@@ -29,16 +105,15 @@ if that is the case. · 7 unmodified lines
29rustc105rustc
30*****106*****
31 107
32A particular version of the Rust compiler is required. Newer versions may or108A recent version of the Rust compiler is required.
33may not work because, for the moment, the kernel depends on some unstable
34Rust features.
35 109
36If ``rustup`` is being used, enter the kernel build directory (or use110If ``rustup`` is being used, enter the kernel build directory (or use
37``--path=<build-dir>`` argument to the ``set`` sub-command) and run::111``--path=<build-dir>`` argument to the ``set`` sub-command) and run,
112for instance::
38 113
39 rustup override set $(scripts/min-tool-version.sh rustc)114 rustup override set stable
40 115
41This will configure your working directory to use the correct version of116This will configure your working directory to use the given version of
42``rustc`` without affecting your default toolchain.117``rustc`` without affecting your default toolchain.
43 118
44Note that the override applies to the current working directory (and its119Note that the override applies to the current working directory (and its
@@ -65,9 +140,9 @@ version later on requires re-adding the component. · 20 unmodified lines
65Otherwise, if a standalone installer is used, the Rust source tree may be140Otherwise, if a standalone installer is used, the Rust source tree may be
66downloaded into the toolchain's installation folder::141downloaded into the toolchain's installation folder::
67 142
68 curl -L "https://static.rust-lang.org/dist/rust-src-$(scripts/min-tool-version.sh rustc).tar.gz" |143 curl -L "https://static.rust-lang.org/dist/rust-src-$(rustc --version | cut -d' ' -f2).tar.gz" |
69 tar -xzf - -C "$(rustc --print sysroot)/lib" \144 tar -xzf - -C "$(rustc --print sysroot)/lib" \
70 "rust-src-$(scripts/min-tool-version.sh rustc)/rust-src/lib/" \145 "rust-src-$(rustc --version | cut -d' ' -f2)/rust-src/lib/" \
71 --strip-components=3146 --strip-components=3
72 147
73In this case, upgrading the Rust compiler version later on requires manually148In this case, upgrading the Rust compiler version later on requires manually
@@ -101,26 +176,22 @@ bindgen · 27 unmodified lines
101*******176*******
102 177
103The bindings to the C side of the kernel are generated at build time using178The bindings to the C side of the kernel are generated at build time using
104the ``bindgen`` tool. A particular version is required.179the ``bindgen`` tool.
105 180
106Install it via (note that this will download and build the tool from source)::181Install it, for instance, via (note that this will download and build the tool
182from source)::
107 183
108 cargo install --locked --version $(scripts/min-tool-version.sh bindgen) bindgen-cli184 cargo install --locked bindgen-cli
109 185
110``bindgen`` needs to find a suitable ``libclang`` in order to work. If it is186``bindgen`` uses the ``clang-sys`` crate to find a suitable ``libclang`` (which
111not found (or a different ``libclang`` than the one found should be used),187may be linked statically, dynamically or loaded at runtime). By default, the
112the process can be tweaked using the environment variables understood by188``cargo`` command above will produce a ``bindgen`` binary that will load
113``clang-sys`` (the Rust bindings crate that ``bindgen`` uses to access189``libclang`` at runtime. If it is not found (or a different ``libclang`` than
114``libclang``):190the one found should be used), the process can be tweaked, e.g. by using the
191``LIBCLANG_PATH`` environment variable. For details, please see ``clang-sys``'s
192documentation at:
115 193
116* ``LLVM_CONFIG_PATH`` can be pointed to an ``llvm-config`` executable.194 https://github.com/KyleMayes/clang-sys#linking
117
118* Or ``LIBCLANG_PATH`` can be pointed to a ``libclang`` shared library
119 or to the directory containing it.
120
121* Or ``CLANG_PATH`` can be pointed to a ``clang`` executable.
122
123For details, please see ``clang-sys``'s documentation at:
124 195
125 https://github.com/KyleMayes/clang-sys#environment-variables196 https://github.com/KyleMayes/clang-sys#environment-variables
126 197
@@ -164,20 +235,6 @@ can be installed manually:: · 37 unmodified lines
164The standalone installers also come with ``clippy``.235The standalone installers also come with ``clippy``.
165 236
166 237
167cargo
168*****
169
170``cargo`` is the Rust native build system. It is currently required to run
171the tests since it is used to build a custom standard library that contains
172the facilities provided by the custom ``alloc`` in the kernel. The tests can
173be run using the ``rusttest`` Make target.
174
175If ``rustup`` is being used, all the profiles already install the tool,
176thus nothing needs to be done.
177
178The standalone installers also come with ``cargo``.
179
180
181rustdoc238rustdoc
182*******239*******
183 240
Documentation/rust/testing.rst+2−3
@@ -131,9 +131,8 @@ Additionally, there are the ``#[test]`` tests. These can be run using the · 130 unmodified lines
131 131
132 make LLVM=1 rusttest132 make LLVM=1 rusttest
133 133
134This requires the kernel ``.config`` and downloads external repositories. It134This requires the kernel ``.config``. It runs the ``#[test]`` tests on the host
135runs the ``#[test]`` tests on the host (currently) and thus is fairly limited in135(currently) and thus is fairly limited in what these tests can test.
136what these tests can test.
137 136
138The Kselftests137The Kselftests
139--------------138--------------
Makefile+16−14
@@ -445,17 +445,17 @@ KBUILD_USERLDFLAGS := $(USERLDFLAGS) · 444 unmodified lines
445# host programs.445# host programs.
446export rust_common_flags := --edition=2021 \446export rust_common_flags := --edition=2021 \
447 -Zbinary_dep_depinfo=y \447 -Zbinary_dep_depinfo=y \
448 -Dunsafe_op_in_unsafe_fn -Drust_2018_idioms \448 -Dunsafe_op_in_unsafe_fn \
449 -Dunreachable_pub -Dnon_ascii_idents \449 -Dnon_ascii_idents \
450 -Wrust_2018_idioms \
451 -Wunreachable_pub \
450 -Wmissing_docs \452 -Wmissing_docs \
451 -Drustdoc::missing_crate_level_docs \453 -Wrustdoc::missing_crate_level_docs \
452 -Dclippy::correctness -Dclippy::style \454 -Wclippy::all \
453 -Dclippy::suspicious -Dclippy::complexity \455 -Wclippy::mut_mut \
454 -Dclippy::perf \456 -Wclippy::needless_bitwise_bool \
455 -Dclippy::let_unit_value -Dclippy::mut_mut \457 -Wclippy::needless_continue \
456 -Dclippy::needless_bitwise_bool \458 -Wclippy::no_mangle_with_rust_abi \
457 -Dclippy::needless_continue \
458 -Dclippy::no_mangle_with_rust_abi \
459 -Wclippy::dbg_macro459 -Wclippy::dbg_macro
460 460
461KBUILD_HOSTCFLAGS := $(KBUILD_USERHOSTCFLAGS) $(HOST_LFS_CFLAGS) \461KBUILD_HOSTCFLAGS := $(KBUILD_USERHOSTCFLAGS) $(HOST_LFS_CFLAGS) \
@@ -493,7 +493,6 @@ RUSTDOC = rustdoc · 31 unmodified lines
493RUSTFMT = rustfmt493RUSTFMT = rustfmt
494CLIPPY_DRIVER = clippy-driver494CLIPPY_DRIVER = clippy-driver
495BINDGEN = bindgen495BINDGEN = bindgen
496CARGO = cargo
497PAHOLE = pahole496PAHOLE = pahole
498RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids497RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids
499LEX = flex498LEX = flex
@@ -559,7 +558,7 @@ KBUILD_RUSTFLAGS := $(rust_common_flags) \ · 59 unmodified lines
559 -Csymbol-mangling-version=v0 \558 -Csymbol-mangling-version=v0 \
560 -Crelocation-model=static \559 -Crelocation-model=static \
561 -Zfunction-sections=n \560 -Zfunction-sections=n \
562 -Dclippy::float_arithmetic561 -Wclippy::float_arithmetic
563 562
564KBUILD_AFLAGS_KERNEL :=563KBUILD_AFLAGS_KERNEL :=
565KBUILD_CFLAGS_KERNEL :=564KBUILD_CFLAGS_KERNEL :=
@@ -587,7 +586,7 @@ endif · 21 unmodified lines
587export RUSTC_BOOTSTRAP := 1586export RUSTC_BOOTSTRAP := 1
588 587
589export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC HOSTPKG_CONFIG588export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC HOSTPKG_CONFIG
590export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN CARGO589export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
591export HOSTRUSTC KBUILD_HOSTRUSTFLAGS590export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
592export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL591export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
593export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX592export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
@@ -1959,9 +1958,12 @@ quiet_cmd_tags = GEN $@ · 1365 unmodified lines
1959tags TAGS cscope gtags: FORCE1958tags TAGS cscope gtags: FORCE
1960 $(call cmd,tags)1959 $(call cmd,tags)
1961 1960
1962# IDE support targets1961# Generate rust-project.json (a file that describes the structure of non-Cargo
1962# Rust projects) for rust-analyzer (an implementation of the Language Server
1963# Protocol).
1963PHONY += rust-analyzer1964PHONY += rust-analyzer
1964rust-analyzer:1965rust-analyzer:
1966 $(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh
1965 $(Q)$(MAKE) $(build)=rust $@1967 $(Q)$(MAKE) $(build)=rust $@
1966 1968
1967# Script to generate missing namespace dependencies1969# Script to generate missing namespace dependencies
include/linux/uaccess.h+32−14
@@ -5,6 +5,7 @@ · 4 unmodified lines
5#include <linux/fault-inject-usercopy.h>5#include <linux/fault-inject-usercopy.h>
6#include <linux/instrumented.h>6#include <linux/instrumented.h>
7#include <linux/minmax.h>7#include <linux/minmax.h>
8#include <linux/nospec.h>
8#include <linux/sched.h>9#include <linux/sched.h>
9#include <linux/thread_info.h>10#include <linux/thread_info.h>
10 11
@@ -138,13 +139,26 @@ __copy_to_user(void __user *to, const void *from, unsigned long n) · 127 unmodified lines
138 return raw_copy_to_user(to, from, n);139 return raw_copy_to_user(to, from, n);
139}140}
140 141
141#ifdef INLINE_COPY_FROM_USER142/*
143 * Architectures that #define INLINE_COPY_TO_USER use this function
144 * directly in the normal copy_to/from_user(), the other ones go
145 * through an extern _copy_to/from_user(), which expands the same code
146 * here.
147 *
148 * Rust code always uses the extern definition.
149 */
142static inline __must_check unsigned long150static inline __must_check unsigned long
143_copy_from_user(void *to, const void __user *from, unsigned long n)151_inline_copy_from_user(void *to, const void __user *from, unsigned long n)
144{152{
145 unsigned long res = n;153 unsigned long res = n;
146 might_fault();154 might_fault();
147 if (!should_fail_usercopy() && likely(access_ok(from, n))) {155 if (!should_fail_usercopy() && likely(access_ok(from, n))) {
156 /*
157 * Ensure that bad access_ok() speculation will not
158 * lead to nasty side effects *after* the copy is
159 * finished:
160 */
161 barrier_nospec();
148 instrument_copy_from_user_before(to, from, n);162 instrument_copy_from_user_before(to, from, n);
149 res = raw_copy_from_user(to, from, n);163 res = raw_copy_from_user(to, from, n);
150 instrument_copy_from_user_after(to, from, n, res);164 instrument_copy_from_user_after(to, from, n, res);
@@ -153,14 +167,11 @@ _copy_from_user(void *to, const void __user *from, unsigned long n) · 2 unmodified lines
153 memset(to + (n - res), 0, res);167 memset(to + (n - res), 0, res);
154 return res;168 return res;
155}169}
156#else
157extern __must_check unsigned long170extern __must_check unsigned long
158_copy_from_user(void *, const void __user *, unsigned long);171_copy_from_user(void *, const void __user *, unsigned long);
159#endif
160 172
161#ifdef INLINE_COPY_TO_USER
162static inline __must_check unsigned long173static inline __must_check unsigned long
163_copy_to_user(void __user *to, const void *from, unsigned long n)174_inline_copy_to_user(void __user *to, const void *from, unsigned long n)
164{175{
165 might_fault();176 might_fault();
166 if (should_fail_usercopy())177 if (should_fail_usercopy())
@@ -171,25 +182,32 @@ _copy_to_user(void __user *to, const void *from, unsigned long n) · 4 unmodified lines
171 }182 }
172 return n;183 return n;
173}184}
174#else
175extern __must_check unsigned long185extern __must_check unsigned long
176_copy_to_user(void __user *, const void *, unsigned long);186_copy_to_user(void __user *, const void *, unsigned long);
177#endif
178 187
179static __always_inline unsigned long __must_check188static __always_inline unsigned long __must_check
180copy_from_user(void *to, const void __user *from, unsigned long n)189copy_from_user(void *to, const void __user *from, unsigned long n)
181{190{
182 if (check_copy_size(to, n, false))191 if (!check_copy_size(to, n, false))
183 n = _copy_from_user(to, from, n);192 return n;
184 return n;193#ifdef INLINE_COPY_FROM_USER
194 return _inline_copy_from_user(to, from, n);
195#else
196 return _copy_from_user(to, from, n);
197#endif
185}198}
186 199
187static __always_inline unsigned long __must_check200static __always_inline unsigned long __must_check
188copy_to_user(void __user *to, const void *from, unsigned long n)201copy_to_user(void __user *to, const void *from, unsigned long n)
189{202{
190 if (check_copy_size(from, n, true))203 if (!check_copy_size(from, n, true))
191 n = _copy_to_user(to, from, n);204 return n;
192 return n;205
206#ifdef INLINE_COPY_TO_USER
207 return _inline_copy_to_user(to, from, n);
208#else
209 return _copy_to_user(to, from, n);
210#endif
193}211}
194 212
195#ifndef copy_mc_to_kernel213#ifndef copy_mc_to_kernel
init/Kconfig+4−1
@@ -1924,7 +1924,10 @@ config RUSTC_VERSION_TEXT · 1923 unmodified lines
1924config BINDGEN_VERSION_TEXT1924config BINDGEN_VERSION_TEXT
1925 string1925 string
1926 depends on RUST1926 depends on RUST
1927 default $(shell,command -v $(BINDGEN) >/dev/null 2>&1 && $(BINDGEN) --version || echo n)1927 # The dummy parameter `workaround-for-0.69.0` is required to support 0.69.0
1928 # (https://github.com/rust-lang/rust-bindgen/pull/2678). It can be removed when
1929 # the minimum version is upgraded past that (0.69.1 already fixed the issue).
1930 default $(shell,command -v $(BINDGEN) >/dev/null 2>&1 && $(BINDGEN) --version workaround-for-0.69.0 || echo n)
1928 1931
1929#1932#
1930# Place an empty function call at each tracepoint site. Can be1933# Place an empty function call at each tracepoint site. Can be
lib/usercopy.c+4−26
@@ -12,40 +12,18 @@ · 11 unmodified lines
12 12
13/* out-of-line parts */13/* out-of-line parts */
14 14
15#ifndef INLINE_COPY_FROM_USER15#if !defined(INLINE_COPY_FROM_USER) || defined(CONFIG_RUST)
16unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)16unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n)
17{17{
18 unsigned long res = n;18 return _inline_copy_from_user(to, from, n);
19 might_fault();
20 if (!should_fail_usercopy() && likely(access_ok(from, n))) {
21 /*
22 * Ensure that bad access_ok() speculation will not
23 * lead to nasty side effects *after* the copy is
24 * finished:
25 */
26 barrier_nospec();
27 instrument_copy_from_user_before(to, from, n);
28 res = raw_copy_from_user(to, from, n);
29 instrument_copy_from_user_after(to, from, n, res);
30 }
31 if (unlikely(res))
32 memset(to + (n - res), 0, res);
33 return res;
34}19}
35EXPORT_SYMBOL(_copy_from_user);20EXPORT_SYMBOL(_copy_from_user);
36#endif21#endif
37 22
38#ifndef INLINE_COPY_TO_USER23#if !defined(INLINE_COPY_TO_USER) || defined(CONFIG_RUST)
39unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)24unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
40{25{
41 might_fault();26 return _inline_copy_to_user(to, from, n);
42 if (should_fail_usercopy())
43 return n;
44 if (likely(access_ok(to, n))) {
45 instrument_copy_to_user(to, from, n);
46 n = raw_copy_to_user(to, from, n);
47 }
48 return n;
49}27}
50EXPORT_SYMBOL(_copy_to_user);28EXPORT_SYMBOL(_copy_to_user);
51#endif29#endif
rust/Makefile+10−64
@@ -44,17 +44,10 @@ rustc_sysroot := $(shell MAKEFLAGS= $(RUSTC) $(rust_flags) --print sysroot) · 43 unmodified lines
44rustc_host_target := $(shell $(RUSTC) --version --verbose | grep -F 'host: ' | cut -d' ' -f2)44rustc_host_target := $(shell $(RUSTC) --version --verbose | grep -F 'host: ' | cut -d' ' -f2)
45RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library45RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library
46 46
47ifeq ($(quiet),silent_)47ifneq ($(quiet),)
48cargo_quiet=-q
49rust_test_quiet=-q48rust_test_quiet=-q
50rustdoc_test_quiet=--test-args -q49rustdoc_test_quiet=--test-args -q
51rustdoc_test_kernel_quiet=>/dev/null50rustdoc_test_kernel_quiet=>/dev/null
52else ifeq ($(quiet),quiet_)
53rust_test_quiet=-q
54rustdoc_test_quiet=--test-args -q
55rustdoc_test_kernel_quiet=>/dev/null
56else
57cargo_quiet=--verbose
58endif51endif
59 52
60core-cfgs = \53core-cfgs = \
@@ -135,22 +128,21 @@ quiet_cmd_rustc_test_library = RUSTC TL $< · 74 unmodified lines
135 @$(objtree)/include/generated/rustc_cfg $(rustc_target_flags) \128 @$(objtree)/include/generated/rustc_cfg $(rustc_target_flags) \
136 --crate-type $(if $(rustc_test_library_proc),proc-macro,rlib) \129 --crate-type $(if $(rustc_test_library_proc),proc-macro,rlib) \
137 --out-dir $(objtree)/$(obj)/test --cfg testlib \130 --out-dir $(objtree)/$(obj)/test --cfg testlib \
138 --sysroot $(objtree)/$(obj)/test/sysroot \
139 -L$(objtree)/$(obj)/test \131 -L$(objtree)/$(obj)/test \
140 --crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<132 --crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<
141 133
142rusttestlib-build_error: $(src)/build_error.rs rusttest-prepare FORCE134rusttestlib-build_error: $(src)/build_error.rs FORCE
143 +$(call if_changed,rustc_test_library)135 +$(call if_changed,rustc_test_library)
144 136
145rusttestlib-macros: private rustc_target_flags = --extern proc_macro137rusttestlib-macros: private rustc_target_flags = --extern proc_macro
146rusttestlib-macros: private rustc_test_library_proc = yes138rusttestlib-macros: private rustc_test_library_proc = yes
147rusttestlib-macros: $(src)/macros/lib.rs rusttest-prepare FORCE139rusttestlib-macros: $(src)/macros/lib.rs FORCE
148 +$(call if_changed,rustc_test_library)140 +$(call if_changed,rustc_test_library)
149 141
150rusttestlib-bindings: $(src)/bindings/lib.rs rusttest-prepare FORCE142rusttestlib-bindings: $(src)/bindings/lib.rs FORCE
151 +$(call if_changed,rustc_test_library)143 +$(call if_changed,rustc_test_library)
152 144
153rusttestlib-uapi: $(src)/uapi/lib.rs rusttest-prepare FORCE145rusttestlib-uapi: $(src)/uapi/lib.rs FORCE
154 +$(call if_changed,rustc_test_library)146 +$(call if_changed,rustc_test_library)
155 147
156quiet_cmd_rustdoc_test = RUSTDOC T $<148quiet_cmd_rustdoc_test = RUSTDOC T $<
@@ -159,7 +151,7 @@ quiet_cmd_rustdoc_test = RUSTDOC T $< · 2 unmodified lines
159 $(RUSTDOC) --test $(rust_common_flags) \151 $(RUSTDOC) --test $(rust_common_flags) \
160 @$(objtree)/include/generated/rustc_cfg \152 @$(objtree)/include/generated/rustc_cfg \
161 $(rustc_target_flags) $(rustdoc_test_target_flags) \153 $(rustc_target_flags) $(rustdoc_test_target_flags) \
162 --sysroot $(objtree)/$(obj)/test/sysroot $(rustdoc_test_quiet) \154 $(rustdoc_test_quiet) \
163 -L$(objtree)/$(obj)/test --output $(rustdoc_output) \155 -L$(objtree)/$(obj)/test --output $(rustdoc_output) \
164 --crate-name $(subst rusttest-,,$@) $<156 --crate-name $(subst rusttest-,,$@) $<
165 157
@@ -192,7 +184,6 @@ quiet_cmd_rustc_test = RUSTC T $< · 26 unmodified lines
192 $(RUSTC) --test $(rust_common_flags) \184 $(RUSTC) --test $(rust_common_flags) \
193 @$(objtree)/include/generated/rustc_cfg \185 @$(objtree)/include/generated/rustc_cfg \
194 $(rustc_target_flags) --out-dir $(objtree)/$(obj)/test \186 $(rustc_target_flags) --out-dir $(objtree)/$(obj)/test \
195 --sysroot $(objtree)/$(obj)/test/sysroot \
196 -L$(objtree)/$(obj)/test \187 -L$(objtree)/$(obj)/test \
197 --crate-name $(subst rusttest-,,$@) $<; \188 --crate-name $(subst rusttest-,,$@) $<; \
198 $(objtree)/$(obj)/test/$(subst rusttest-,,$@) $(rust_test_quiet) \189 $(objtree)/$(obj)/test/$(subst rusttest-,,$@) $(rust_test_quiet) \
@@ -200,60 +191,15 @@ quiet_cmd_rustc_test = RUSTC T $< · 1 unmodified line
200 191
201rusttest: rusttest-macros rusttest-kernel192rusttest: rusttest-macros rusttest-kernel
202 193
203# This prepares a custom sysroot with our custom `alloc` instead of
204# the standard one.
205#
206# This requires several hacks:
207# - Unlike `core` and `alloc`, `std` depends on more than a dozen crates,
208# including third-party crates that need to be downloaded, plus custom
209# `build.rs` steps. Thus hardcoding things here is not maintainable.
210# - `cargo` knows how to build the standard library, but it is an unstable
211# feature so far (`-Zbuild-std`).
212# - `cargo` only considers the use case of building the standard library
213# to use it in a given package. Thus we need to create a dummy package
214# and pick the generated libraries from there.
215# - The usual ways of modifying the dependency graph in `cargo` do not seem
216# to apply for the `-Zbuild-std` steps, thus we have to mislead it
217# by modifying the sources in the sysroot.
218# - To avoid messing with the user's Rust installation, we create a clone
219# of the sysroot. However, `cargo` ignores `RUSTFLAGS` in the `-Zbuild-std`
220# steps, thus we use a wrapper binary passed via `RUSTC` to pass the flag.
221#
222# In the future, we hope to avoid the whole ordeal by either:
223# - Making the `test` crate not depend on `std` (either improving upstream
224# or having our own custom crate).
225# - Making the tests run in kernel space (requires the previous point).
226# - Making `std` and friends be more like a "normal" crate, so that
227# `-Zbuild-std` and related hacks are not needed.
228quiet_cmd_rustsysroot = RUSTSYSROOT
229 cmd_rustsysroot = \
230 rm -rf $(objtree)/$(obj)/test; \
231 mkdir -p $(objtree)/$(obj)/test; \
232 cp -a $(rustc_sysroot) $(objtree)/$(obj)/test/sysroot; \
233 echo '\#!/bin/sh' > $(objtree)/$(obj)/test/rustc_sysroot; \
234 echo "$(RUSTC) --sysroot=$(abspath $(objtree)/$(obj)/test/sysroot) \"\$$@\"" \
235 >> $(objtree)/$(obj)/test/rustc_sysroot; \
236 chmod u+x $(objtree)/$(obj)/test/rustc_sysroot; \
237 $(CARGO) -q new $(objtree)/$(obj)/test/dummy; \
238 RUSTC=$(objtree)/$(obj)/test/rustc_sysroot $(CARGO) $(cargo_quiet) \
239 test -Zbuild-std --target $(rustc_host_target) \
240 --manifest-path $(objtree)/$(obj)/test/dummy/Cargo.toml; \
241 rm $(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib/*; \
242 cp $(objtree)/$(obj)/test/dummy/target/$(rustc_host_target)/debug/deps/* \
243 $(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib
244
245rusttest-prepare: FORCE
246 +$(call if_changed,rustsysroot)
247
248rusttest-macros: private rustc_target_flags = --extern proc_macro194rusttest-macros: private rustc_target_flags = --extern proc_macro
249rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro195rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro
250rusttest-macros: $(src)/macros/lib.rs rusttest-prepare FORCE196rusttest-macros: $(src)/macros/lib.rs FORCE
251 +$(call if_changed,rustc_test)197 +$(call if_changed,rustc_test)
252 +$(call if_changed,rustdoc_test)198 +$(call if_changed,rustdoc_test)
253 199
254rusttest-kernel: private rustc_target_flags = --extern alloc \200rusttest-kernel: private rustc_target_flags = --extern alloc \
255 --extern build_error --extern macros --extern bindings --extern uapi201 --extern build_error --extern macros --extern bindings --extern uapi
256rusttest-kernel: $(src)/kernel/lib.rs rusttest-prepare \202rusttest-kernel: $(src)/kernel/lib.rs \
257 rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \203 rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
258 rusttestlib-uapi FORCE204 rusttestlib-uapi FORCE
259 +$(call if_changed,rustc_test)205 +$(call if_changed,rustc_test)
@@ -421,7 +367,7 @@ ifneq ($(or $(CONFIG_ARM64),$(and $(CONFIG_RISCV),$(CONFIG_64BIT))),) · 161 unmodified lines
421endif367endif
422 368
423$(obj)/core.o: private skip_clippy = 1369$(obj)/core.o: private skip_clippy = 1
424$(obj)/core.o: private skip_flags = -Dunreachable_pub370$(obj)/core.o: private skip_flags = -Wunreachable_pub
425$(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))371$(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))
426$(obj)/core.o: private rustc_target_flags = $(core-cfgs)372$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
427$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs FORCE373$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs FORCE
@@ -435,7 +381,7 @@ $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE · 7 unmodified lines
435 +$(call if_changed_dep,rustc_library)381 +$(call if_changed_dep,rustc_library)
436 382
437$(obj)/alloc.o: private skip_clippy = 1383$(obj)/alloc.o: private skip_clippy = 1
438$(obj)/alloc.o: private skip_flags = -Dunreachable_pub384$(obj)/alloc.o: private skip_flags = -Wunreachable_pub
439$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)385$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)
440$(obj)/alloc.o: $(RUST_LIB_SRC)/alloc/src/lib.rs $(obj)/compiler_builtins.o FORCE386$(obj)/alloc.o: $(RUST_LIB_SRC)/alloc/src/lib.rs $(obj)/compiler_builtins.o FORCE
441 +$(call if_changed_dep,rustc_library)387 +$(call if_changed_dep,rustc_library)
rust/bindings/bindings_helper.h+1−0
@@ -30,4 +30,5 @@ const gfp_t RUST_CONST_HELPER_GFP_KERNEL = GFP_KERNEL; · 29 unmodified lines
30const gfp_t RUST_CONST_HELPER_GFP_KERNEL_ACCOUNT = GFP_KERNEL_ACCOUNT;30const gfp_t RUST_CONST_HELPER_GFP_KERNEL_ACCOUNT = GFP_KERNEL_ACCOUNT;
31const gfp_t RUST_CONST_HELPER_GFP_NOWAIT = GFP_NOWAIT;31const gfp_t RUST_CONST_HELPER_GFP_NOWAIT = GFP_NOWAIT;
32const gfp_t RUST_CONST_HELPER___GFP_ZERO = __GFP_ZERO;32const gfp_t RUST_CONST_HELPER___GFP_ZERO = __GFP_ZERO;
33const gfp_t RUST_CONST_HELPER___GFP_HIGHMEM = ___GFP_HIGHMEM;
33const blk_features_t RUST_CONST_HELPER_BLK_FEAT_ROTATIONAL = BLK_FEAT_ROTATIONAL;34const blk_features_t RUST_CONST_HELPER_BLK_FEAT_ROTATIONAL = BLK_FEAT_ROTATIONAL;
rust/bindings/lib.rs+1−0
@@ -24,6 +24,7 @@ · 23 unmodified lines
24 unsafe_op_in_unsafe_fn24 unsafe_op_in_unsafe_fn
25)]25)]
26 26
27#[allow(dead_code)]
27mod bindings_raw {28mod bindings_raw {
28 // Use glob import here to expose all helpers.29 // Use glob import here to expose all helpers.
29 // Symbols defined within the module will take precedence to the glob import.30 // Symbols defined within the module will take precedence to the glob import.
rust/helpers.c+34−0
@@ -26,6 +26,8 @@ · 25 unmodified lines
26#include <linux/device.h>26#include <linux/device.h>
27#include <linux/err.h>27#include <linux/err.h>
28#include <linux/errname.h>28#include <linux/errname.h>
29#include <linux/gfp.h>
30#include <linux/highmem.h>
29#include <linux/mutex.h>31#include <linux/mutex.h>
30#include <linux/refcount.h>32#include <linux/refcount.h>
31#include <linux/sched/signal.h>33#include <linux/sched/signal.h>
@@ -40,6 +42,20 @@ __noreturn void rust_helper_BUG(void) · 8 unmodified lines
40}42}
41EXPORT_SYMBOL_GPL(rust_helper_BUG);43EXPORT_SYMBOL_GPL(rust_helper_BUG);
42 44
45unsigned long rust_helper_copy_from_user(void *to, const void __user *from,
46 unsigned long n)
47{
48 return copy_from_user(to, from, n);
49}
50EXPORT_SYMBOL_GPL(rust_helper_copy_from_user);
51
52unsigned long rust_helper_copy_to_user(void __user *to, const void *from,
53 unsigned long n)
54{
55 return copy_to_user(to, from, n);
56}
57EXPORT_SYMBOL_GPL(rust_helper_copy_to_user);
58
43void rust_helper_mutex_lock(struct mutex *lock)59void rust_helper_mutex_lock(struct mutex *lock)
44{60{
45 mutex_lock(lock);61 mutex_lock(lock);
@@ -81,6 +97,24 @@ int rust_helper_signal_pending(struct task_struct *t) · 35 unmodified lines
81}97}
82EXPORT_SYMBOL_GPL(rust_helper_signal_pending);98EXPORT_SYMBOL_GPL(rust_helper_signal_pending);
83 99
100struct page *rust_helper_alloc_pages(gfp_t gfp_mask, unsigned int order)
101{
102 return alloc_pages(gfp_mask, order);
103}
104EXPORT_SYMBOL_GPL(rust_helper_alloc_pages);
105
106void *rust_helper_kmap_local_page(struct page *page)
107{
108 return kmap_local_page(page);
109}
110EXPORT_SYMBOL_GPL(rust_helper_kmap_local_page);
111
112void rust_helper_kunmap_local(const void *addr)
113{
114 kunmap_local(addr);
115}
116EXPORT_SYMBOL_GPL(rust_helper_kunmap_local);
117
84refcount_t rust_helper_REFCOUNT_INIT(int n)118refcount_t rust_helper_REFCOUNT_INIT(int n)
85{119{
86 return (refcount_t)REFCOUNT_INIT(n);120 return (refcount_t)REFCOUNT_INIT(n);
rust/kernel/alloc.rs+16−1
@@ -20,6 +20,13 @@ pub struct AllocError; · 19 unmodified lines
20#[derive(Clone, Copy)]20#[derive(Clone, Copy)]
21pub struct Flags(u32);21pub struct Flags(u32);
22 22
23impl Flags {
24 /// Get the raw representation of this flag.
25 pub(crate) fn as_raw(self) -> u32 {
26 self.0
27 }
28}
29
23impl core::ops::BitOr for Flags {30impl core::ops::BitOr for Flags {
24 type Output = Self;31 type Output = Self;
25 fn bitor(self, rhs: Self) -> Self::Output {32 fn bitor(self, rhs: Self) -> Self::Output {
@@ -52,6 +59,14 @@ pub mod flags { · 26 unmodified lines
52 /// This is normally or'd with other flags.59 /// This is normally or'd with other flags.
53 pub const __GFP_ZERO: Flags = Flags(bindings::__GFP_ZERO);60 pub const __GFP_ZERO: Flags = Flags(bindings::__GFP_ZERO);
54 61
62 /// Allow the allocation to be in high memory.
63 ///
64 /// Allocations in high memory may not be mapped into the kernel's address space, so this can't
65 /// be used with `kmalloc` and other similar methods.
66 ///
67 /// This is normally or'd with other flags.
68 pub const __GFP_HIGHMEM: Flags = Flags(bindings::__GFP_HIGHMEM);
69
55 /// Users can not sleep and need the allocation to succeed.70 /// Users can not sleep and need the allocation to succeed.
56 ///71 ///
57 /// A lower watermark is applied to allow access to "atomic reserves". The current72 /// A lower watermark is applied to allow access to "atomic reserves". The current
@@ -66,7 +81,7 @@ pub mod flags { · 8 unmodified lines
66 /// The same as [`GFP_KERNEL`], except the allocation is accounted to kmemcg.81 /// The same as [`GFP_KERNEL`], except the allocation is accounted to kmemcg.
67 pub const GFP_KERNEL_ACCOUNT: Flags = Flags(bindings::GFP_KERNEL_ACCOUNT);82 pub const GFP_KERNEL_ACCOUNT: Flags = Flags(bindings::GFP_KERNEL_ACCOUNT);
68 83
69 /// Ror kernel allocations that should not stall for direct reclaim, start physical IO or84 /// For kernel allocations that should not stall for direct reclaim, start physical IO or
70 /// use any filesystem callback. It is very likely to fail to allocate memory, even for very85 /// use any filesystem callback. It is very likely to fail to allocate memory, even for very
71 /// small allocations.86 /// small allocations.
72 pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);87 pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);
rust/kernel/init.rs+4−9
@@ -843,11 +843,8 @@ where · 842 unmodified lines
843 let val = unsafe { &mut *slot };843 let val = unsafe { &mut *slot };
844 // SAFETY: `slot` is considered pinned.844 // SAFETY: `slot` is considered pinned.
845 let val = unsafe { Pin::new_unchecked(val) };845 let val = unsafe { Pin::new_unchecked(val) };
846 (self.1)(val).map_err(|e| {846 // SAFETY: `slot` was initialized above.
847 // SAFETY: `slot` was initialized above.847 (self.1)(val).inspect_err(|_| unsafe { core::ptr::drop_in_place(slot) })
848 unsafe { core::ptr::drop_in_place(slot) };
849 e
850 })
851 }848 }
852}849}
853 850
@@ -941,11 +938,9 @@ where · 87 unmodified lines
941 // SAFETY: All requirements fulfilled since this function is `__init`.938 // SAFETY: All requirements fulfilled since this function is `__init`.
942 unsafe { self.0.__pinned_init(slot)? };939 unsafe { self.0.__pinned_init(slot)? };
943 // SAFETY: The above call initialized `slot` and we still have unique access.940 // SAFETY: The above call initialized `slot` and we still have unique access.
944 (self.1)(unsafe { &mut *slot }).map_err(|e| {941 (self.1)(unsafe { &mut *slot }).inspect_err(|_|
945 // SAFETY: `slot` was initialized above.942 // SAFETY: `slot` was initialized above.
946 unsafe { core::ptr::drop_in_place(slot) };943 unsafe { core::ptr::drop_in_place(slot) })
947 e
948 })
949 }944 }
950}945}
951 946
rust/kernel/lib.rs+2−0
@@ -40,6 +40,7 @@ pub mod ioctl; · 39 unmodified lines
40pub mod kunit;40pub mod kunit;
41#[cfg(CONFIG_NET)]41#[cfg(CONFIG_NET)]
42pub mod net;42pub mod net;
43pub mod page;
43pub mod prelude;44pub mod prelude;
44pub mod print;45pub mod print;
45mod static_assert;46mod static_assert;
@@ -50,6 +51,7 @@ pub mod sync; · 4 unmodified lines
50pub mod task;51pub mod task;
51pub mod time;52pub mod time;
52pub mod types;53pub mod types;
54pub mod uaccess;
53pub mod workqueue;55pub mod workqueue;
54 56
55#[doc(hidden)]57#[doc(hidden)]
rust/kernel/page.rs+250−0
@@ -0,0 +1,250 @@
1// SPDX-License-Identifier: GPL-2.0
2
3//! Kernel page allocation and management.
4
5use crate::{
6 alloc::{AllocError, Flags},
7 bindings,
8 error::code::*,
9 error::Result,
10 uaccess::UserSliceReader,
11};
12use core::ptr::{self, NonNull};
13
14/// A bitwise shift for the page size.
15pub const PAGE_SHIFT: usize = bindings::PAGE_SHIFT as usize;
16
17/// The number of bytes in a page.
18pub const PAGE_SIZE: usize = bindings::PAGE_SIZE;
19
20/// A bitmask that gives the page containing a given address.
21pub const PAGE_MASK: usize = !(PAGE_SIZE - 1);
22
23/// A pointer to a page that owns the page allocation.
24///
25/// # Invariants
26///
27/// The pointer is valid, and has ownership over the page.
28pub struct Page {
29 page: NonNull<bindings::page>,
30}
31
32// SAFETY: Pages have no logic that relies on them staying on a given thread, so moving them across
33// threads is safe.
34unsafe impl Send for Page {}
35
36// SAFETY: Pages have no logic that relies on them not being accessed concurrently, so accessing
37// them concurrently is safe.
38unsafe impl Sync for Page {}
39
40impl Page {
41 /// Allocates a new page.
42 ///
43 /// # Examples
44 ///
45 /// Allocate memory for a page.
46 ///
47 /// ```
48 /// use kernel::page::Page;
49 ///
50 /// # fn dox() -> Result<(), kernel::alloc::AllocError> {
51 /// let page = Page::alloc_page(GFP_KERNEL)?;
52 /// # Ok(()) }
53 /// ```
54 ///
55 /// Allocate memory for a page and zero its contents.
56 ///
57 /// ```
58 /// use kernel::page::Page;
59 ///
60 /// # fn dox() -> Result<(), kernel::alloc::AllocError> {
61 /// let page = Page::alloc_page(GFP_KERNEL | __GFP_ZERO)?;
62 /// # Ok(()) }
63 /// ```
64 pub fn alloc_page(flags: Flags) -> Result<Self, AllocError> {
65 // SAFETY: Depending on the value of `gfp_flags`, this call may sleep. Other than that, it
66 // is always safe to call this method.
67 let page = unsafe { bindings::alloc_pages(flags.as_raw(), 0) };
68 let page = NonNull::new(page).ok_or(AllocError)?;
69 // INVARIANT: We just successfully allocated a page, so we now have ownership of the newly
70 // allocated page. We transfer that ownership to the new `Page` object.
71 Ok(Self { page })
72 }
73
74 /// Returns a raw pointer to the page.
75 pub fn as_ptr(&self) -> *mut bindings::page {
76 self.page.as_ptr()
77 }
78
79 /// Runs a piece of code with this page mapped to an address.
80 ///
81 /// The page is unmapped when this call returns.
82 ///
83 /// # Using the raw pointer
84 ///
85 /// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for
86 /// `PAGE_SIZE` bytes and for the duration in which the closure is called. The pointer might
87 /// only be mapped on the current thread, and when that is the case, dereferencing it on other
88 /// threads is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't
89 /// cause data races, the memory may be uninitialized, and so on.
90 ///
91 /// If multiple threads map the same page at the same time, then they may reference with
92 /// different addresses. However, even if the addresses are different, the underlying memory is
93 /// still the same for these purposes (e.g., it's still a data race if they both write to the
94 /// same underlying byte at the same time).
95 fn with_page_mapped<T>(&self, f: impl FnOnce(*mut u8) -> T) -> T {
96 // SAFETY: `page` is valid due to the type invariants on `Page`.
97 let mapped_addr = unsafe { bindings::kmap_local_page(self.as_ptr()) };
98
99 let res = f(mapped_addr.cast());
100
101 // This unmaps the page mapped above.
102 //
103 // SAFETY: Since this API takes the user code as a closure, it can only be used in a manner
104 // where the pages are unmapped in reverse order. This is as required by `kunmap_local`.
105 //
106 // In other words, if this call to `kunmap_local` happens when a different page should be
107 // unmapped first, then there must necessarily be a call to `kmap_local_page` other than the
108 // call just above in `with_page_mapped` that made that possible. In this case, it is the
109 // unsafe block that wraps that other call that is incorrect.
110 unsafe { bindings::kunmap_local(mapped_addr) };
111
112 res
113 }
114
115 /// Runs a piece of code with a raw pointer to a slice of this page, with bounds checking.
116 ///
117 /// If `f` is called, then it will be called with a pointer that points at `off` bytes into the
118 /// page, and the pointer will be valid for at least `len` bytes. The pointer is only valid on
119 /// this task, as this method uses a local mapping.
120 ///
121 /// If `off` and `len` refers to a region outside of this page, then this method returns
122 /// [`EINVAL`] and does not call `f`.
123 ///
124 /// # Using the raw pointer
125 ///
126 /// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for
127 /// `len` bytes and for the duration in which the closure is called. The pointer might only be
128 /// mapped on the current thread, and when that is the case, dereferencing it on other threads
129 /// is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't cause
130 /// data races, the memory may be uninitialized, and so on.
131 ///
132 /// If multiple threads map the same page at the same time, then they may reference with
133 /// different addresses. However, even if the addresses are different, the underlying memory is
134 /// still the same for these purposes (e.g., it's still a data race if they both write to the
135 /// same underlying byte at the same time).
136 fn with_pointer_into_page<T>(
137 &self,
138 off: usize,
139 len: usize,
140 f: impl FnOnce(*mut u8) -> Result<T>,
141 ) -> Result<T> {
142 let bounds_ok = off <= PAGE_SIZE && len <= PAGE_SIZE && (off + len) <= PAGE_SIZE;
143
144 if bounds_ok {
145 self.with_page_mapped(move |page_addr| {
146 // SAFETY: The `off` integer is at most `PAGE_SIZE`, so this pointer offset will
147 // result in a pointer that is in bounds or one off the end of the page.
148 f(unsafe { page_addr.add(off) })
149 })
150 } else {
151 Err(EINVAL)
152 }
153 }
154
155 /// Maps the page and reads from it into the given buffer.
156 ///
157 /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
158 /// outside of the page, then this call returns [`EINVAL`].
159 ///
160 /// # Safety
161 ///
162 /// * Callers must ensure that `dst` is valid for writing `len` bytes.
163 /// * Callers must ensure that this call does not race with a write to the same page that
164 /// overlaps with this read.
165 pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result {
166 self.with_pointer_into_page(offset, len, move |src| {
167 // SAFETY: If `with_pointer_into_page` calls into this closure, then
168 // it has performed a bounds check and guarantees that `src` is
169 // valid for `len` bytes.
170 //
171 // There caller guarantees that there is no data race.
172 unsafe { ptr::copy_nonoverlapping(src, dst, len) };
173 Ok(())
174 })
175 }
176
177 /// Maps the page and writes into it from the given buffer.
178 ///
179 /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
180 /// outside of the page, then this call returns [`EINVAL`].
181 ///
182 /// # Safety
183 ///
184 /// * Callers must ensure that `src` is valid for reading `len` bytes.
185 /// * Callers must ensure that this call does not race with a read or write to the same page
186 /// that overlaps with this write.
187 pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Result {
188 self.with_pointer_into_page(offset, len, move |dst| {
189 // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
190 // bounds check and guarantees that `dst` is valid for `len` bytes.
191 //
192 // There caller guarantees that there is no data race.
193 unsafe { ptr::copy_nonoverlapping(src, dst, len) };
194 Ok(())
195 })
196 }
197
198 /// Maps the page and zeroes the given slice.
199 ///
200 /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
201 /// outside of the page, then this call returns [`EINVAL`].
202 ///
203 /// # Safety
204 ///
205 /// Callers must ensure that this call does not race with a read or write to the same page that
206 /// overlaps with this write.
207 pub unsafe fn fill_zero_raw(&self, offset: usize, len: usize) -> Result {
208 self.with_pointer_into_page(offset, len, move |dst| {
209 // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
210 // bounds check and guarantees that `dst` is valid for `len` bytes.
211 //
212 // There caller guarantees that there is no data race.
213 unsafe { ptr::write_bytes(dst, 0u8, len) };
214 Ok(())
215 })
216 }
217
218 /// Copies data from userspace into this page.
219 ///
220 /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
221 /// outside of the page, then this call returns [`EINVAL`].
222 ///
223 /// Like the other `UserSliceReader` methods, data races are allowed on the userspace address.
224 /// However, they are not allowed on the page you are copying into.
225 ///
226 /// # Safety
227 ///
228 /// Callers must ensure that this call does not race with a read or write to the same page that
229 /// overlaps with this write.
230 pub unsafe fn copy_from_user_slice_raw(
231 &self,
232 reader: &mut UserSliceReader,
233 offset: usize,
234 len: usize,
235 ) -> Result {
236 self.with_pointer_into_page(offset, len, move |dst| {
237 // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
238 // bounds check and guarantees that `dst` is valid for `len` bytes. Furthermore, we have
239 // exclusive access to the slice since the caller guarantees that there are no races.
240 reader.read_raw(unsafe { core::slice::from_raw_parts_mut(dst.cast(), len) })
241 })
242 }
243}
244
245impl Drop for Page {
246 fn drop(&mut self) {
247 // SAFETY: By the type invariants, we have ownership of the page and can free it.
248 unsafe { bindings::__free_pages(self.page.as_ptr(), 0) };
249 }
250}
rust/kernel/types.rs+64−0
@@ -409,3 +409,67 @@ pub enum Either<L, R> { · 408 unmodified lines
409 /// Constructs an instance of [`Either`] containing a value of type `R`.409 /// Constructs an instance of [`Either`] containing a value of type `R`.
410 Right(R),410 Right(R),
411}411}
412
413/// Types for which any bit pattern is valid.
414///
415/// Not all types are valid for all values. For example, a `bool` must be either zero or one, so
416/// reading arbitrary bytes into something that contains a `bool` is not okay.
417///
418/// It's okay for the type to have padding, as initializing those bytes has no effect.
419///
420/// # Safety
421///
422/// All bit-patterns must be valid for this type. This type must not have interior mutability.
423pub unsafe trait FromBytes {}
424
425// SAFETY: All bit patterns are acceptable values of the types below.
426unsafe impl FromBytes for u8 {}
427unsafe impl FromBytes for u16 {}
428unsafe impl FromBytes for u32 {}
429unsafe impl FromBytes for u64 {}
430unsafe impl FromBytes for usize {}
431unsafe impl FromBytes for i8 {}
432unsafe impl FromBytes for i16 {}
433unsafe impl FromBytes for i32 {}
434unsafe impl FromBytes for i64 {}
435unsafe impl FromBytes for isize {}
436// SAFETY: If all bit patterns are acceptable for individual values in an array, then all bit
437// patterns are also acceptable for arrays of that type.
438unsafe impl<T: FromBytes> FromBytes for [T] {}
439unsafe impl<T: FromBytes, const N: usize> FromBytes for [T; N] {}
440
441/// Types that can be viewed as an immutable slice of initialized bytes.
442///
443/// If a struct implements this trait, then it is okay to copy it byte-for-byte to userspace. This
444/// means that it should not have any padding, as padding bytes are uninitialized. Reading
445/// uninitialized memory is not just undefined behavior, it may even lead to leaking sensitive
446/// information on the stack to userspace.
447///
448/// The struct should also not hold kernel pointers, as kernel pointer addresses are also considered
449/// sensitive. However, leaking kernel pointers is not considered undefined behavior by Rust, so
450/// this is a correctness requirement, but not a safety requirement.
451///
452/// # Safety
453///
454/// Values of this type may not contain any uninitialized bytes. This type must not have interior
455/// mutability.
456pub unsafe trait AsBytes {}
457
458// SAFETY: Instances of the following types have no uninitialized portions.
459unsafe impl AsBytes for u8 {}
460unsafe impl AsBytes for u16 {}
461unsafe impl AsBytes for u32 {}
462unsafe impl AsBytes for u64 {}
463unsafe impl AsBytes for usize {}
464unsafe impl AsBytes for i8 {}
465unsafe impl AsBytes for i16 {}
466unsafe impl AsBytes for i32 {}
467unsafe impl AsBytes for i64 {}
468unsafe impl AsBytes for isize {}
469unsafe impl AsBytes for bool {}
470unsafe impl AsBytes for char {}
471unsafe impl AsBytes for str {}
472// SAFETY: If individual values in an array have no uninitialized portions, then the array itself
473// does not have any uninitialized portions either.
474unsafe impl<T: AsBytes> AsBytes for [T] {}
475unsafe impl<T: AsBytes, const N: usize> AsBytes for [T; N] {}
rust/kernel/uaccess.rs+388−0
@@ -0,0 +1,388 @@
1// SPDX-License-Identifier: GPL-2.0
2
3//! Slices to user space memory regions.
4//!
5//! C header: [`include/linux/uaccess.h`](srctree/include/linux/uaccess.h)
6
7use crate::{
8 alloc::Flags,
9 bindings,
10 error::Result,
11 prelude::*,
12 types::{AsBytes, FromBytes},
13};
14use alloc::vec::Vec;
15use core::ffi::{c_ulong, c_void};
16use core::mem::{size_of, MaybeUninit};
17
18/// The type used for userspace addresses.
19pub type UserPtr = usize;
20
21/// A pointer to an area in userspace memory, which can be either read-only or read-write.
22///
23/// All methods on this struct are safe: attempting to read or write on bad addresses (either out of
24/// the bound of the slice or unmapped addresses) will return [`EFAULT`]. Concurrent access,
25/// *including data races to/from userspace memory*, is permitted, because fundamentally another
26/// userspace thread/process could always be modifying memory at the same time (in the same way that
27/// userspace Rust's [`std::io`] permits data races with the contents of files on disk). In the
28/// presence of a race, the exact byte values read/written are unspecified but the operation is
29/// well-defined. Kernelspace code should validate its copy of data after completing a read, and not
30/// expect that multiple reads of the same address will return the same value.
31///
32/// These APIs are designed to make it difficult to accidentally write TOCTOU (time-of-check to
33/// time-of-use) bugs. Every time a memory location is read, the reader's position is advanced by
34/// the read length and the next read will start from there. This helps prevent accidentally reading
35/// the same location twice and causing a TOCTOU bug.
36///
37/// Creating a [`UserSliceReader`] and/or [`UserSliceWriter`] consumes the `UserSlice`, helping
38/// ensure that there aren't multiple readers or writers to the same location.
39///
40/// If double-fetching a memory location is necessary for some reason, then that is done by creating
41/// multiple readers to the same memory location, e.g. using [`clone_reader`].
42///
43/// # Examples
44///
45/// Takes a region of userspace memory from the current process, and modify it by adding one to
46/// every byte in the region.
47///
48/// ```no_run
49/// use alloc::vec::Vec;
50/// use core::ffi::c_void;
51/// use kernel::error::Result;
52/// use kernel::uaccess::{UserPtr, UserSlice};
53///
54/// fn bytes_add_one(uptr: UserPtr, len: usize) -> Result<()> {
55/// let (read, mut write) = UserSlice::new(uptr, len).reader_writer();
56///
57/// let mut buf = Vec::new();
58/// read.read_all(&mut buf, GFP_KERNEL)?;
59///
60/// for b in &mut buf {
61/// *b = b.wrapping_add(1);
62/// }
63///
64/// write.write_slice(&buf)?;
65/// Ok(())
66/// }
67/// ```
68///
69/// Example illustrating a TOCTOU (time-of-check to time-of-use) bug.
70///
71/// ```no_run
72/// use alloc::vec::Vec;
73/// use core::ffi::c_void;
74/// use kernel::error::{code::EINVAL, Result};
75/// use kernel::uaccess::{UserPtr, UserSlice};
76///
77/// /// Returns whether the data in this region is valid.
78/// fn is_valid(uptr: UserPtr, len: usize) -> Result<bool> {
79/// let read = UserSlice::new(uptr, len).reader();
80///
81/// let mut buf = Vec::new();
82/// read.read_all(&mut buf, GFP_KERNEL)?;
83///
84/// todo!()
85/// }
86///
87/// /// Returns the bytes behind this user pointer if they are valid.
88/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<Vec<u8>> {
89/// if !is_valid(uptr, len)? {
90/// return Err(EINVAL);
91/// }
92///
93/// let read = UserSlice::new(uptr, len).reader();
94///
95/// let mut buf = Vec::new();
96/// read.read_all(&mut buf, GFP_KERNEL)?;
97///
98/// // THIS IS A BUG! The bytes could have changed since we checked them.
99/// //
100/// // To avoid this kind of bug, don't call `UserSlice::new` multiple
101/// // times with the same address.
102/// Ok(buf)
103/// }
104/// ```
105///
106/// [`std::io`]: https://doc.rust-lang.org/std/io/index.html
107/// [`clone_reader`]: UserSliceReader::clone_reader
108pub struct UserSlice {
109 ptr: UserPtr,
110 length: usize,
111}
112
113impl UserSlice {
114 /// Constructs a user slice from a raw pointer and a length in bytes.
115 ///
116 /// Constructing a [`UserSlice`] performs no checks on the provided address and length, it can
117 /// safely be constructed inside a kernel thread with no current userspace process. Reads and
118 /// writes wrap the kernel APIs `copy_from_user` and `copy_to_user`, which check the memory map
119 /// of the current process and enforce that the address range is within the user range (no
120 /// additional calls to `access_ok` are needed). Validity of the pointer is checked when you
121 /// attempt to read or write, not in the call to `UserSlice::new`.
122 ///
123 /// Callers must be careful to avoid time-of-check-time-of-use (TOCTOU) issues. The simplest way
124 /// is to create a single instance of [`UserSlice`] per user memory block as it reads each byte
125 /// at most once.
126 pub fn new(ptr: UserPtr, length: usize) -> Self {
127 UserSlice { ptr, length }
128 }
129
130 /// Reads the entirety of the user slice, appending it to the end of the provided buffer.
131 ///
132 /// Fails with [`EFAULT`] if the read happens on a bad address.
133 pub fn read_all(self, buf: &mut Vec<u8>, flags: Flags) -> Result {
134 self.reader().read_all(buf, flags)
135 }
136
137 /// Constructs a [`UserSliceReader`].
138 pub fn reader(self) -> UserSliceReader {
139 UserSliceReader {
140 ptr: self.ptr,
141 length: self.length,
142 }
143 }
144
145 /// Constructs a [`UserSliceWriter`].
146 pub fn writer(self) -> UserSliceWriter {
147 UserSliceWriter {
148 ptr: self.ptr,
149 length: self.length,
150 }
151 }
152
153 /// Constructs both a [`UserSliceReader`] and a [`UserSliceWriter`].
154 ///
155 /// Usually when this is used, you will first read the data, and then overwrite it afterwards.
156 pub fn reader_writer(self) -> (UserSliceReader, UserSliceWriter) {
157 (
158 UserSliceReader {
159 ptr: self.ptr,
160 length: self.length,
161 },
162 UserSliceWriter {
163 ptr: self.ptr,
164 length: self.length,
165 },
166 )
167 }
168}
169
170/// A reader for [`UserSlice`].
171///
172/// Used to incrementally read from the user slice.
173pub struct UserSliceReader {
174 ptr: UserPtr,
175 length: usize,
176}
177
178impl UserSliceReader {
179 /// Skip the provided number of bytes.
180 ///
181 /// Returns an error if skipping more than the length of the buffer.
182 pub fn skip(&mut self, num_skip: usize) -> Result {
183 // Update `self.length` first since that's the fallible part of this operation.
184 self.length = self.length.checked_sub(num_skip).ok_or(EFAULT)?;
185 self.ptr = self.ptr.wrapping_add(num_skip);
186 Ok(())
187 }
188
189 /// Create a reader that can access the same range of data.
190 ///
191 /// Reading from the clone does not advance the current reader.
192 ///
193 /// The caller should take care to not introduce TOCTOU issues, as described in the
194 /// documentation for [`UserSlice`].
195 pub fn clone_reader(&self) -> UserSliceReader {
196 UserSliceReader {
197 ptr: self.ptr,
198 length: self.length,
199 }
200 }
201
202 /// Returns the number of bytes left to be read from this reader.
203 ///
204 /// Note that even reading less than this number of bytes may fail.
205 pub fn len(&self) -> usize {
206 self.length
207 }
208
209 /// Returns `true` if no data is available in the io buffer.
210 pub fn is_empty(&self) -> bool {
211 self.length == 0
212 }
213
214 /// Reads raw data from the user slice into a kernel buffer.
215 ///
216 /// For a version that uses `&mut [u8]`, please see [`UserSliceReader::read_slice`].
217 ///
218 /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
219 /// bounds of this [`UserSliceReader`]. This call may modify `out` even if it returns an error.
220 ///
221 /// # Guarantees
222 ///
223 /// After a successful call to this method, all bytes in `out` are initialized.
224 pub fn read_raw(&mut self, out: &mut [MaybeUninit<u8>]) -> Result {
225 let len = out.len();
226 let out_ptr = out.as_mut_ptr().cast::<c_void>();
227 if len > self.length {
228 return Err(EFAULT);
229 }
230 let Ok(len_ulong) = c_ulong::try_from(len) else {
231 return Err(EFAULT);
232 };
233 // SAFETY: `out_ptr` points into a mutable slice of length `len_ulong`, so we may write
234 // that many bytes to it.
235 let res =
236 unsafe { bindings::copy_from_user(out_ptr, self.ptr as *const c_void, len_ulong) };
237 if res != 0 {
238 return Err(EFAULT);
239 }
240 self.ptr = self.ptr.wrapping_add(len);
241 self.length -= len;
242 Ok(())
243 }
244
245 /// Reads raw data from the user slice into a kernel buffer.
246 ///
247 /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
248 /// bounds of this [`UserSliceReader`]. This call may modify `out` even if it returns an error.
249 pub fn read_slice(&mut self, out: &mut [u8]) -> Result {
250 // SAFETY: The types are compatible and `read_raw` doesn't write uninitialized bytes to
251 // `out`.
252 let out = unsafe { &mut *(out as *mut [u8] as *mut [MaybeUninit<u8>]) };
253 self.read_raw(out)
254 }
255
256 /// Reads a value of the specified type.
257 ///
258 /// Fails with [`EFAULT`] if the read happens on a bad address, or if the read goes out of
259 /// bounds of this [`UserSliceReader`].
260 pub fn read<T: FromBytes>(&mut self) -> Result<T> {
261 let len = size_of::<T>();
262 if len > self.length {
263 return Err(EFAULT);
264 }
265 let Ok(len_ulong) = c_ulong::try_from(len) else {
266 return Err(EFAULT);
267 };
268 let mut out: MaybeUninit<T> = MaybeUninit::uninit();
269 // SAFETY: The local variable `out` is valid for writing `size_of::<T>()` bytes.
270 //
271 // By using the _copy_from_user variant, we skip the check_object_size check that verifies
272 // the kernel pointer. This mirrors the logic on the C side that skips the check when the
273 // length is a compile-time constant.
274 let res = unsafe {
275 bindings::_copy_from_user(
276 out.as_mut_ptr().cast::<c_void>(),
277 self.ptr as *const c_void,
278 len_ulong,
279 )
280 };
281 if res != 0 {
282 return Err(EFAULT);
283 }
284 self.ptr = self.ptr.wrapping_add(len);
285 self.length -= len;
286 // SAFETY: The read above has initialized all bytes in `out`, and since `T` implements
287 // `FromBytes`, any bit-pattern is a valid value for this type.
288 Ok(unsafe { out.assume_init() })
289 }
290
291 /// Reads the entirety of the user slice, appending it to the end of the provided buffer.
292 ///
293 /// Fails with [`EFAULT`] if the read happens on a bad address.
294 pub fn read_all(mut self, buf: &mut Vec<u8>, flags: Flags) -> Result {
295 let len = self.length;
296 VecExt::<u8>::reserve(buf, len, flags)?;
297
298 // The call to `try_reserve` was successful, so the spare capacity is at least `len` bytes
299 // long.
300 self.read_raw(&mut buf.spare_capacity_mut()[..len])?;
301
302 // SAFETY: Since the call to `read_raw` was successful, so the next `len` bytes of the
303 // vector have been initialized.
304 unsafe { buf.set_len(buf.len() + len) };
305 Ok(())
306 }
307}
308
309/// A writer for [`UserSlice`].
310///
311/// Used to incrementally write into the user slice.
312pub struct UserSliceWriter {
313 ptr: UserPtr,
314 length: usize,
315}
316
317impl UserSliceWriter {
318 /// Returns the amount of space remaining in this buffer.
319 ///
320 /// Note that even writing less than this number of bytes may fail.
321 pub fn len(&self) -> usize {
322 self.length
323 }
324
325 /// Returns `true` if no more data can be written to this buffer.
326 pub fn is_empty(&self) -> bool {
327 self.length == 0
328 }
329
330 /// Writes raw data to this user pointer from a kernel buffer.
331 ///
332 /// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
333 /// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
334 /// if it returns an error.
335 pub fn write_slice(&mut self, data: &[u8]) -> Result {
336 let len = data.len();
337 let data_ptr = data.as_ptr().cast::<c_void>();
338 if len > self.length {
339 return Err(EFAULT);
340 }
341 let Ok(len_ulong) = c_ulong::try_from(len) else {
342 return Err(EFAULT);
343 };
344 // SAFETY: `data_ptr` points into an immutable slice of length `len_ulong`, so we may read
345 // that many bytes from it.
346 let res = unsafe { bindings::copy_to_user(self.ptr as *mut c_void, data_ptr, len_ulong) };
347 if res != 0 {
348 return Err(EFAULT);
349 }
350 self.ptr = self.ptr.wrapping_add(len);
351 self.length -= len;
352 Ok(())
353 }
354
355 /// Writes the provided Rust value to this userspace pointer.
356 ///
357 /// Fails with [`EFAULT`] if the write happens on a bad address, or if the write goes out of
358 /// bounds of this [`UserSliceWriter`]. This call may modify the associated userspace slice even
359 /// if it returns an error.
360 pub fn write<T: AsBytes>(&mut self, value: &T) -> Result {
361 let len = size_of::<T>();
362 if len > self.length {
363 return Err(EFAULT);
364 }
365 let Ok(len_ulong) = c_ulong::try_from(len) else {
366 return Err(EFAULT);
367 };
368 // SAFETY: The reference points to a value of type `T`, so it is valid for reading
369 // `size_of::<T>()` bytes.
370 //
371 // By using the _copy_to_user variant, we skip the check_object_size check that verifies the
372 // kernel pointer. This mirrors the logic on the C side that skips the check when the length
373 // is a compile-time constant.
374 let res = unsafe {
375 bindings::_copy_to_user(
376 self.ptr as *mut c_void,
377 (value as *const T).cast::<c_void>(),
378 len_ulong,
379 )
380 };
381 if res != 0 {
382 return Err(EFAULT);
383 }
384 self.ptr = self.ptr.wrapping_add(len);
385 self.length -= len;
386 Ok(())
387 }
388}
rust/kernel/workqueue.rs+9−7
@@ -482,24 +482,26 @@ pub unsafe trait HasWork<T, const ID: u64 = 0> { · 481 unmodified lines
482/// use kernel::sync::Arc;482/// use kernel::sync::Arc;
483/// use kernel::workqueue::{self, impl_has_work, Work};483/// use kernel::workqueue::{self, impl_has_work, Work};
484///484///
485/// struct MyStruct {485/// struct MyStruct<'a, T, const N: usize> {
486/// work_field: Work<MyStruct, 17>,486/// work_field: Work<MyStruct<'a, T, N>, 17>,
487/// f: fn(&'a [T; N]),
487/// }488/// }
488///489///
489/// impl_has_work! {490/// impl_has_work! {
490/// impl HasWork<MyStruct, 17> for MyStruct { self.work_field }491/// impl{'a, T, const N: usize} HasWork<MyStruct<'a, T, N>, 17>
492/// for MyStruct<'a, T, N> { self.work_field }
491/// }493/// }
492/// ```494/// ```
493#[macro_export]495#[macro_export]
494macro_rules! impl_has_work {496macro_rules! impl_has_work {
495 ($(impl$(<$($implarg:ident),*>)?497 ($(impl$({$($generics:tt)*})?
496 HasWork<$work_type:ty $(, $id:tt)?>498 HasWork<$work_type:ty $(, $id:tt)?>
497 for $self:ident $(<$($selfarg:ident),*>)?499 for $self:ty
498 { self.$field:ident }500 { self.$field:ident }
499 )*) => {$(501 )*) => {$(
500 // SAFETY: The implementation of `raw_get_work` only compiles if the field has the right502 // SAFETY: The implementation of `raw_get_work` only compiles if the field has the right
501 // type.503 // type.
502 unsafe impl$(<$($implarg),*>)? $crate::workqueue::HasWork<$work_type $(, $id)?> for $self $(<$($selfarg),*>)? {504 unsafe impl$(<$($generics)+>)? $crate::workqueue::HasWork<$work_type $(, $id)?> for $self {
503 const OFFSET: usize = ::core::mem::offset_of!(Self, $field) as usize;505 const OFFSET: usize = ::core::mem::offset_of!(Self, $field) as usize;
504 506
505 #[inline]507 #[inline]
@@ -515,7 +517,7 @@ macro_rules! impl_has_work { · 9 unmodified lines
515pub use impl_has_work;517pub use impl_has_work;
516 518
517impl_has_work! {519impl_has_work! {
518 impl<T> HasWork<Self> for ClosureWork<T> { self.work }520 impl{T} HasWork<Self> for ClosureWork<T> { self.work }
519}521}
520 522
521unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Arc<T>523unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Arc<T>
rust/macros/lib.rs+39−6
@@ -35,6 +35,7 @@ use proc_macro::TokenStream; · 34 unmodified lines
35/// author: "Rust for Linux Contributors",35/// author: "Rust for Linux Contributors",
36/// description: "My very own kernel module!",36/// description: "My very own kernel module!",
37/// license: "GPL",37/// license: "GPL",
38/// alias: ["alternate_module_name"],
38/// }39/// }
39///40///
40/// struct MyModule;41/// struct MyModule;
@@ -55,13 +56,45 @@ use proc_macro::TokenStream; · 14 unmodified lines
55/// }56/// }
56/// ```57/// ```
57///58///
59/// ## Firmware
60///
61/// The following example shows how to declare a kernel module that needs
62/// to load binary firmware files. You need to specify the file names of
63/// the firmware in the `firmware` field. The information is embedded
64/// in the `modinfo` section of the kernel module. For example, a tool to
65/// build an initramfs uses this information to put the firmware files into
66/// the initramfs image.
67///
68/// ```ignore
69/// use kernel::prelude::*;
70///
71/// module!{
72/// type: MyDeviceDriverModule,
73/// name: "my_device_driver_module",
74/// author: "Rust for Linux Contributors",
75/// description: "My device driver requires firmware",
76/// license: "GPL",
77/// firmware: ["my_device_firmware1.bin", "my_device_firmware2.bin"],
78/// }
79///
80/// struct MyDeviceDriverModule;
81///
82/// impl kernel::Module for MyDeviceDriverModule {
83/// fn init() -> Result<Self> {
84/// Ok(Self)
85/// }
86/// }
87/// ```
88///
58/// # Supported argument types89/// # Supported argument types
59/// - `type`: type which implements the [`Module`] trait (required).90/// - `type`: type which implements the [`Module`] trait (required).
60/// - `name`: byte array of the name of the kernel module (required).91/// - `name`: ASCII string literal of the name of the kernel module (required).
61/// - `author`: byte array of the author of the kernel module.92/// - `author`: string literal of the author of the kernel module.
62/// - `description`: byte array of the description of the kernel module.93/// - `description`: string literal of the description of the kernel module.
63/// - `license`: byte array of the license of the kernel module (required).94/// - `license`: ASCII string literal of the license of the kernel module (required).
64/// - `alias`: byte array of alias name of the kernel module.95/// - `alias`: array of ASCII string literals of the alias names of the kernel module.
96/// - `firmware`: array of ASCII string literals of the firmware files of
97/// the kernel module.
65#[proc_macro]98#[proc_macro]
66pub fn module(ts: TokenStream) -> TokenStream {99pub fn module(ts: TokenStream) -> TokenStream {
67 module::module(ts)100 module::module(ts)
@@ -312,7 +345,7 @@ pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream { · 244 unmodified lines
312///345///
313/// Currently supported modifiers are:346/// Currently supported modifiers are:
314/// * `span`: change the span of concatenated identifier to the span of the specified token. By347/// * `span`: change the span of concatenated identifier to the span of the specified token. By
315/// default the span of the `[< >]` group is used.348/// default the span of the `[< >]` group is used.
316/// * `lower`: change the identifier to lower case.349/// * `lower`: change the identifier to lower case.
317/// * `upper`: change the identifier to upper case.350/// * `upper`: change the identifier to upper case.
318///351///
rust/macros/module.rs+16−2
@@ -97,14 +97,22 @@ struct ModuleInfo { · 96 unmodified lines
97 author: Option<String>,97 author: Option<String>,
98 description: Option<String>,98 description: Option<String>,
99 alias: Option<Vec<String>>,99 alias: Option<Vec<String>>,
100 firmware: Option<Vec<String>>,
100}101}
101 102
102impl ModuleInfo {103impl ModuleInfo {
103 fn parse(it: &mut token_stream::IntoIter) -> Self {104 fn parse(it: &mut token_stream::IntoIter) -> Self {
104 let mut info = ModuleInfo::default();105 let mut info = ModuleInfo::default();
105 106
106 const EXPECTED_KEYS: &[&str] =107 const EXPECTED_KEYS: &[&str] = &[
107 &["type", "name", "author", "description", "license", "alias"];108 "type",
109 "name",
110 "author",
111 "description",
112 "license",
113 "alias",
114 "firmware",
115 ];
108 const REQUIRED_KEYS: &[&str] = &["type", "name", "license"];116 const REQUIRED_KEYS: &[&str] = &["type", "name", "license"];
109 let mut seen_keys = Vec::new();117 let mut seen_keys = Vec::new();
110 118
@@ -131,6 +139,7 @@ impl ModuleInfo { · 20 unmodified lines
131 "description" => info.description = Some(expect_string(it)),139 "description" => info.description = Some(expect_string(it)),
132 "license" => info.license = expect_string_ascii(it),140 "license" => info.license = expect_string_ascii(it),
133 "alias" => info.alias = Some(expect_string_array(it)),141 "alias" => info.alias = Some(expect_string_array(it)),
142 "firmware" => info.firmware = Some(expect_string_array(it)),
134 _ => panic!(143 _ => panic!(
135 "Unknown key \"{}\". Valid keys are: {:?}.",144 "Unknown key \"{}\". Valid keys are: {:?}.",
136 key, EXPECTED_KEYS145 key, EXPECTED_KEYS
@@ -186,6 +195,11 @@ pub(crate) fn module(ts: TokenStream) -> TokenStream { · 49 unmodified lines
186 modinfo.emit("alias", &alias);195 modinfo.emit("alias", &alias);
187 }196 }
188 }197 }
198 if let Some(firmware) = info.firmware {
199 for fw in firmware {
200 modinfo.emit("firmware", &fw);
201 }
202 }
189 203
190 // Built-in modules also export the `file` modinfo string.204 // Built-in modules also export the `file` modinfo string.
191 let file =205 let file =
rust/uapi/lib.rs+1−0
@@ -14,6 +14,7 @@ · 13 unmodified lines
14#![cfg_attr(test, allow(unsafe_op_in_unsafe_fn))]14#![cfg_attr(test, allow(unsafe_op_in_unsafe_fn))]
15#![allow(15#![allow(
16 clippy::all,16 clippy::all,
17 dead_code,
17 missing_docs,18 missing_docs,
18 non_camel_case_types,19 non_camel_case_types,
19 non_upper_case_globals,20 non_upper_case_globals,
scripts/rust_is_available.sh+17−16
@@ -117,20 +117,16 @@ if [ "$rust_compiler_cversion" -lt "$rust_compiler_min_cversion" ]; then · 116 unmodified lines
117 echo >&2 "***"117 echo >&2 "***"
118 exit 1118 exit 1
119fi119fi
120if [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
121 echo >&2 "***"
122 echo >&2 "*** Rust compiler '$RUSTC' is too new. This may or may not work."
123 echo >&2 "*** Your version: $rust_compiler_version"
124 echo >&2 "*** Expected version: $rust_compiler_min_version"
125 echo >&2 "***"
126 warning=1
127fi
128 120
129# Check that the Rust bindings generator is suitable.121# Check that the Rust bindings generator is suitable.
130#122#
131# Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.123# Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
124#
125# The dummy parameter `workaround-for-0.69.0` is required to support 0.69.0
126# (https://github.com/rust-lang/rust-bindgen/pull/2678). It can be removed when
127# the minimum version is upgraded past that (0.69.1 already fixed the issue).
132rust_bindings_generator_output=$( \128rust_bindings_generator_output=$( \
133 LC_ALL=C "$BINDGEN" --version 2>/dev/null129 LC_ALL=C "$BINDGEN" --version workaround-for-0.69.0 2>/dev/null
134) || rust_bindings_generator_code=$?130) || rust_bindings_generator_code=$?
135if [ -n "$rust_bindings_generator_code" ]; then131if [ -n "$rust_bindings_generator_code" ]; then
136 echo >&2 "***"132 echo >&2 "***"
@@ -165,13 +161,18 @@ if [ "$rust_bindings_generator_cversion" -lt "$rust_bindings_generator_min_cvers · 28 unmodified lines
165 echo >&2 "***"161 echo >&2 "***"
166 exit 1162 exit 1
167fi163fi
168if [ "$rust_bindings_generator_cversion" -gt "$rust_bindings_generator_min_cversion" ]; then164if [ "$rust_bindings_generator_cversion" -eq 6600 ] ||
169 echo >&2 "***"165 [ "$rust_bindings_generator_cversion" -eq 6601 ]; then
170 echo >&2 "*** Rust bindings generator '$BINDGEN' is too new. This may or may not work."166 # Distributions may have patched the issue (e.g. Debian did).
171 echo >&2 "*** Your version: $rust_bindings_generator_version"167 if ! "$BINDGEN" $(dirname $0)/rust_is_available_bindgen_0_66.h >/dev/null; then
172 echo >&2 "*** Expected version: $rust_bindings_generator_min_version"168 echo >&2 "***"
173 echo >&2 "***"169 echo >&2 "*** Rust bindings generator '$BINDGEN' versions 0.66.0 and 0.66.1 may not"
174 warning=1170 echo >&2 "*** work due to a bug (https://github.com/rust-lang/rust-bindgen/pull/2567),"
171 echo >&2 "*** unless patched (like Debian's)."
172 echo >&2 "*** Your version: $rust_bindings_generator_version"
173 echo >&2 "***"
174 warning=1
175 fi
175fi176fi
176 177
177# Check that the `libclang` used by the Rust bindings generator is suitable.178# Check that the `libclang` used by the Rust bindings generator is suitable.
scripts/rust_is_available_bindgen_0_66.h+2−0
@@ -0,0 +1,2 @@
1/* SPDX-License-Identifier: GPL-2.0 */
2#define A "\0"
scripts/rust_is_available_test.py+37−22
@@ -54,18 +54,34 @@ else: · 53 unmodified lines
54""")54""")
55 55
56 @classmethod56 @classmethod
57 def generate_bindgen(cls, version_stdout, libclang_stderr):57 def generate_bindgen(cls, version_stdout, libclang_stderr, version_0_66_patched=False):
58 if libclang_stderr is None:
59 libclang_case = f"raise SystemExit({cls.bindgen_default_bindgen_libclang_failure_exit_code})"
60 else:
61 libclang_case = f"print({repr(libclang_stderr)}, file=sys.stderr)"
62
63 if version_0_66_patched:
64 version_0_66_case = "pass"
65 else:
66 version_0_66_case = "raise SystemExit(1)"
67
58 return cls.generate_executable(f"""#!/usr/bin/env python368 return cls.generate_executable(f"""#!/usr/bin/env python3
59import sys69import sys
60if "rust_is_available_bindgen_libclang.h" in " ".join(sys.argv):70if "rust_is_available_bindgen_libclang.h" in " ".join(sys.argv):
61 print({repr(libclang_stderr)}, file=sys.stderr)71 {libclang_case}
72elif "rust_is_available_bindgen_0_66.h" in " ".join(sys.argv):
73 {version_0_66_case}
62else:74else:
63 print({repr(version_stdout)})75 print({repr(version_stdout)})
64""")76""")
65 77
66 @classmethod78 @classmethod
67 def generate_bindgen_version(cls, stdout):79 def generate_bindgen_version(cls, stdout, version_0_66_patched=False):
68 return cls.generate_bindgen(stdout, cls.bindgen_default_bindgen_libclang_stderr)80 return cls.generate_bindgen(stdout, cls.bindgen_default_bindgen_libclang_stderr, version_0_66_patched)
81
82 @classmethod
83 def generate_bindgen_libclang_failure(cls):
84 return cls.generate_bindgen(cls.bindgen_default_bindgen_version_stdout, None)
69 85
70 @classmethod86 @classmethod
71 def generate_bindgen_libclang(cls, stderr):87 def generate_bindgen_libclang(cls, stderr):
@@ -89,6 +105,7 @@ else: · 17 unmodified lines
89 cls.rust_default_sysroot = subprocess.check_output(("rustc", "--print", "sysroot")).decode().strip()105 cls.rust_default_sysroot = subprocess.check_output(("rustc", "--print", "sysroot")).decode().strip()
90 106
91 cls.bindgen_default_bindgen_version_stdout = f"bindgen {cls.bindgen_default_version}"107 cls.bindgen_default_bindgen_version_stdout = f"bindgen {cls.bindgen_default_version}"
108 cls.bindgen_default_bindgen_libclang_failure_exit_code = 42
92 cls.bindgen_default_bindgen_libclang_stderr = f"scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {cls.llvm_default_version} [-W#pragma-messages], err: false"109 cls.bindgen_default_bindgen_libclang_stderr = f"scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {cls.llvm_default_version} [-W#pragma-messages], err: false"
93 110
94 cls.default_rustc = cls.generate_rustc(f"rustc {cls.rustc_default_version}")111 cls.default_rustc = cls.generate_rustc(f"rustc {cls.rustc_default_version}")
@@ -193,11 +210,6 @@ else: · 98 unmodified lines
193 result = self.run_script(self.Expected.FAILURE, { "RUSTC": rustc })210 result = self.run_script(self.Expected.FAILURE, { "RUSTC": rustc })
194 self.assertIn(f"Rust compiler '{rustc}' is too old.", result.stderr)211 self.assertIn(f"Rust compiler '{rustc}' is too old.", result.stderr)
195 212
196 def test_rustc_new_version(self):
197 rustc = self.generate_rustc("rustc 1.999.0 (a8314ef7d 2099-06-27)")
198 result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "RUSTC": rustc })
199 self.assertIn(f"Rust compiler '{rustc}' is too new. This may or may not work.", result.stderr)
200
201 def test_bindgen_nonexecutable(self):213 def test_bindgen_nonexecutable(self):
202 result = self.run_script(self.Expected.FAILURE, { "BINDGEN": self.nonexecutable })214 result = self.run_script(self.Expected.FAILURE, { "BINDGEN": self.nonexecutable })
203 self.assertIn(f"Running '{self.nonexecutable}' to check the Rust bindings generator version failed with", result.stderr)215 self.assertIn(f"Running '{self.nonexecutable}' to check the Rust bindings generator version failed with", result.stderr)
@@ -226,21 +238,24 @@ else: · 22 unmodified lines
226 result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })238 result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
227 self.assertIn(f"Rust bindings generator '{bindgen}' is too old.", result.stderr)239 self.assertIn(f"Rust bindings generator '{bindgen}' is too old.", result.stderr)
228 240
229 def test_bindgen_new_version(self):241 def test_bindgen_bad_version_0_66_0_and_0_66_1(self):
230 bindgen = self.generate_bindgen_version("bindgen 0.999.0")242 for version in ("0.66.0", "0.66.1"):
231 result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "BINDGEN": bindgen })243 with self.subTest(version=version):
232 self.assertIn(f"Rust bindings generator '{bindgen}' is too new. This may or may not work.", result.stderr)244 bindgen = self.generate_bindgen_version(f"bindgen {version}")
245 result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "BINDGEN": bindgen })
246 self.assertIn(f"Rust bindings generator '{bindgen}' versions 0.66.0 and 0.66.1 may not", result.stderr)
247
248 def test_bindgen_bad_version_0_66_0_and_0_66_1_patched(self):
249 for version in ("0.66.0", "0.66.1"):
250 with self.subTest(version=version):
251 bindgen = self.generate_bindgen_version(f"bindgen {version}", True)
252 result = self.run_script(self.Expected.SUCCESS, { "BINDGEN": bindgen })
233 253
234 def test_bindgen_libclang_failure(self):254 def test_bindgen_libclang_failure(self):
235 for env in (255 bindgen = self.generate_bindgen_libclang_failure()
236 { "LLVM_CONFIG_PATH": self.missing },256 result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
237 { "LIBCLANG_PATH": self.missing },257 self.assertIn(f"Running '{bindgen}' to check the libclang version (used by the Rust", result.stderr)
238 { "CLANG_PATH": self.missing },258 self.assertIn(f"bindings generator) failed with code {self.bindgen_default_bindgen_libclang_failure_exit_code}. This may be caused by", result.stderr)
239 ):
240 with self.subTest(env=env):
241 result = self.run_script(self.Expected.FAILURE, env | { "PATH": os.environ["PATH"], "BINDGEN": "bindgen" })
242 self.assertIn("Running 'bindgen' to check the libclang version (used by the Rust", result.stderr)
243 self.assertIn("bindings generator) failed with code ", result.stderr)
244 259
245 def test_bindgen_libclang_unexpected_version(self):260 def test_bindgen_libclang_unexpected_version(self):
246 bindgen = self.generate_bindgen_libclang("scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version unexpected [-W#pragma-messages], err: false")261 bindgen = self.generate_bindgen_libclang("scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version unexpected [-W#pragma-messages], err: false")