Merge tag 'modules-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux

Pull modules updates from Luis Chamberlain: - It was time to tidy up kernel/module.c and one way of starting with that effort was to split it up into files. At my request Aaron Tomlin spearheaded that effort with the goal to not introduce any functional at all during that endeavour. The penalty for the split is +1322 bytes total, +112 bytes in data, +1210 bytes in text while bss is unchanged. One of the benefits of this other than helping make the code easier to read and review is summoning more help on review for changes with livepatching so kernel/module/livepatch.c is now pegged as maintained by the live patching folks. The before and after with just the move on a defconfig on x86-64: $ size kernel/module.o text data bss dec hex filename 38434 4540 104 43078 a846 kernel/module.o $ size -t kernel/module/*.o text data bss dec hex filename 4785 120 0 4905 1329 kernel/module/kallsyms.o 28577 4416 104 33097 8149 kernel/module/main.o 1158 8 0 1166 48e kernel/module/procfs.o 902 108 0 1010 3f2 kernel/module/strict_rwx.o 3390 0 0 3390 d3e kernel/module/sysfs.o 832 0 0 832 340 kernel/module/tree_lookup.o 39644 4652 104 44400 ad70 (TOTALS) - Aaron added module unload taint tracking (MODULE_UNLOAD_TAINT_TRACKING), to enable tracking unloaded modules which did taint the kernel. - Christophe Leroy added CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC which lets architectures to request having modules data in vmalloc area instead of module area. There are three reasons why an architecture might want this: a) On some architectures (like book3s/32) it is not possible to protect against execution on a page basis. The exec stuff can be mapped by different arch segment sizes (on book3s/32 that is 256M segments). By default the module area is in an Exec segment while vmalloc area is in a NoExec segment. Using vmalloc lets you muck with module data as NoExec on those architectures whereas before you could not. b) By pushing more module data to vmalloc you also increase the probability of module text to remain within a closer distance from kernel core text and this reduces trampolines, this has been reported on arm first and powerpc folks are following that lead. c) Free'ing module_alloc() (Exec by default) area leaves this exposed as Exec by default, some architectures have some security enhancements to set this as NoExec on free, and splitting module data with text let's future generic special allocators be added to the kernel without having developers try to grok the tribal knowledge per arch. Work like Rick Edgecombe's permission vmalloc interface [0] becomes easier to address over time. [0] https://lore.kernel.org/lkml/20201120202426.18009-1-rick.p.edgecombe@intel.com/#r - Masahiro Yamada's symbol search enhancements * tag 'modules-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux: (33 commits) module: merge check_exported_symbol() into find_exported_symbol_in_section() module: do not binary-search in __ksymtab_gpl if fsa->gplok is false module: do not pass opaque pointer for symbol search module: show disallowed symbol name for inherit_taint() module: fix [e_shstrndx].sh_size=0 OOB access module: Introduce module unload taint tracking module: Move module_assert_mutex_or_preempt() to internal.h module: Make module_flags_taint() accept a module's taints bitmap and usable outside core code module.h: simplify MODULE_IMPORT_NS powerpc: Select ARCH_WANTS_MODULES_DATA_IN_VMALLOC on book3s/32 and 8xx module: Remove module_addr_min and module_addr_max module: Add CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC module: Introduce data_layout module: Prepare for handling several RB trees module: Always have struct mod_tree_root module: Rename debug_align() as strict_align() module: Rework layout alignment to avoid BUG_ON()s module: Move module_enable_x() and frob_text() in strict_rwx.c module: Make module_enable_x() independent of CONFIG_ARCH_HAS_STRICT_MODULE_RWX module: Move version support into a separate file ...

Linus Torvalds committed 4 years ago

Commit ef98f9c2 parents 44d3572 7390b9429 files · +2384 −2052Browse files

Showing the changes to kernel/module (15 of 29 files). Show all

Unified · Split

15 files

kernel/module/Makefile +21 −0
kernel/module/debug_kmemleak.c +30 −0
kernel/module/decompress.c +276 −0
kernel/module/internal.h +302 −0
kernel/module/kallsyms.c +512 −0
kernel/module/kdb.c +62 −0
kernel/module/livepatch.c +74 −0
kernel/module/main.c +3131 −0
kernel/module/procfs.c +146 −0
kernel/module/signing.c +122 −0
kernel/module/strict_rwx.c +143 −0
kernel/module/sysfs.c +436 −0
kernel/module/tracking.c +61 −0
kernel/module/tree_lookup.c +117 −0
kernel/module/version.c +101 −0

kernel/module/Makefile+21−0

@@ -0,0 +1,21 @@
	1	+# SPDX-License-Identifier: GPL-2.0-only
	2	+#
	3	+# Makefile for linux kernel module support
	4	+#
	5	+
	6	+# These are called from save_stack_trace() on slub debug path,
	7	+# and produce insane amounts of uninteresting coverage.
	8	+KCOV_INSTRUMENT_module.o := n
	9	+
	10	+obj-y += main.o strict_rwx.o
	11	+obj-$(CONFIG_MODULE_DECOMPRESS) += decompress.o
	12	+obj-$(CONFIG_MODULE_SIG) += signing.o
	13	+obj-$(CONFIG_LIVEPATCH) += livepatch.o
	14	+obj-$(CONFIG_MODULES_TREE_LOOKUP) += tree_lookup.o
	15	+obj-$(CONFIG_DEBUG_KMEMLEAK) += debug_kmemleak.o
	16	+obj-$(CONFIG_KALLSYMS) += kallsyms.o
	17	+obj-$(CONFIG_PROC_FS) += procfs.o
	18	+obj-$(CONFIG_SYSFS) += sysfs.o
	19	+obj-$(CONFIG_KGDB_KDB) += kdb.o
	20	+obj-$(CONFIG_MODVERSIONS) += version.o
	21	+obj-$(CONFIG_MODULE_UNLOAD_TAINT_TRACKING) += tracking.o

kernel/module/debug_kmemleak.c+30−0

@@ -0,0 +1,30 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module kmemleak support
	4	+ *
	5	+ * Copyright (C) 2009 Catalin Marinas
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/kmemleak.h>
	10	+#include "internal.h"
	11	+
	12	+void kmemleak_load_module(const struct module *mod,
	13	+ const struct load_info *info)
	14	+{
	15	+ unsigned int i;
	16	+
	17	+ /* only scan the sections containing data */
	18	+ kmemleak_scan_area(mod, sizeof(struct module), GFP_KERNEL);
	19	+
	20	+ for (i = 1; i < info->hdr->e_shnum; i++) {
	21	+ /* Scan all writable sections that's not executable */
	22	+ if (!(info->sechdrs[i].sh_flags & SHF_ALLOC) \|\|
	23	+ !(info->sechdrs[i].sh_flags & SHF_WRITE) \|\|
	24	+ (info->sechdrs[i].sh_flags & SHF_EXECINSTR))
	25	+ continue;
	26	+
	27	+ kmemleak_scan_area((void *)info->sechdrs[i].sh_addr,
	28	+ info->sechdrs[i].sh_size, GFP_KERNEL);
	29	+ }
	30	+}

kernel/module/decompress.c+276−0

@@ -0,0 +1,276 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Copyright 2021 Google LLC.
	4	+ */
	5	+
	6	+#include <linux/init.h>
	7	+#include <linux/highmem.h>
	8	+#include <linux/kobject.h>
	9	+#include <linux/mm.h>
	10	+#include <linux/module.h>
	11	+#include <linux/slab.h>
	12	+#include <linux/sysfs.h>
	13	+#include <linux/vmalloc.h>
	14	+
	15	+#include "internal.h"
	16	+
	17	+static int module_extend_max_pages(struct load_info *info, unsigned int extent)
	18	+{
	19	+ struct page **new_pages;
	20	+
	21	+ new_pages = kvmalloc_array(info->max_pages + extent,
	22	+ sizeof(info->pages), GFP_KERNEL);
	23	+ if (!new_pages)
	24	+ return -ENOMEM;
	25	+
	26	+ memcpy(new_pages, info->pages, info->max_pages * sizeof(info->pages));
	27	+ kvfree(info->pages);
	28	+ info->pages = new_pages;
	29	+ info->max_pages += extent;
	30	+
	31	+ return 0;
	32	+}
	33	+
	34	+static struct page module_get_next_page(struct load_info info)
	35	+{
	36	+ struct page *page;
	37	+ int error;
	38	+
	39	+ if (info->max_pages == info->used_pages) {
	40	+ error = module_extend_max_pages(info, info->used_pages);
	41	+ if (error)
	42	+ return ERR_PTR(error);
	43	+ }
	44	+
	45	+ page = alloc_page(GFP_KERNEL \| __GFP_HIGHMEM);
	46	+ if (!page)
	47	+ return ERR_PTR(-ENOMEM);
	48	+
	49	+ info->pages[info->used_pages++] = page;
	50	+ return page;
	51	+}
	52	+
	53	+#ifdef CONFIG_MODULE_COMPRESS_GZIP
	54	+#include <linux/zlib.h>
	55	+#define MODULE_COMPRESSION gzip
	56	+#define MODULE_DECOMPRESS_FN module_gzip_decompress
	57	+
	58	+/*
	59	+ * Calculate length of the header which consists of signature, header
	60	+ * flags, time stamp and operating system ID (10 bytes total), plus
	61	+ * an optional filename.
	62	+ */
	63	+static size_t module_gzip_header_len(const u8 *buf, size_t size)
	64	+{
	65	+ const u8 signature[] = { 0x1f, 0x8b, 0x08 };
	66	+ size_t len = 10;
	67	+
	68	+ if (size < len \|\| memcmp(buf, signature, sizeof(signature)))
	69	+ return 0;
	70	+
	71	+ if (buf[3] & 0x08) {
	72	+ do {
	73	+ /*
	74	+ * If we can't find the end of the file name we must
	75	+ * be dealing with a corrupted file.
	76	+ */
	77	+ if (len == size)
	78	+ return 0;
	79	+ } while (buf[len++] != '\0');
	80	+ }
	81	+
	82	+ return len;
	83	+}
	84	+
	85	+static ssize_t module_gzip_decompress(struct load_info *info,
	86	+ const void *buf, size_t size)
	87	+{
	88	+ struct z_stream_s s = { 0 };
	89	+ size_t new_size = 0;
	90	+ size_t gzip_hdr_len;
	91	+ ssize_t retval;
	92	+ int rc;
	93	+
	94	+ gzip_hdr_len = module_gzip_header_len(buf, size);
	95	+ if (!gzip_hdr_len) {
	96	+ pr_err("not a gzip compressed module\n");
	97	+ return -EINVAL;
	98	+ }
	99	+
	100	+ s.next_in = buf + gzip_hdr_len;
	101	+ s.avail_in = size - gzip_hdr_len;
	102	+
	103	+ s.workspace = kmalloc(zlib_inflate_workspacesize(), GFP_KERNEL);
	104	+ if (!s.workspace)
	105	+ return -ENOMEM;
	106	+
	107	+ rc = zlib_inflateInit2(&s, -MAX_WBITS);
	108	+ if (rc != Z_OK) {
	109	+ pr_err("failed to initialize decompressor: %d\n", rc);
	110	+ retval = -EINVAL;
	111	+ goto out;
	112	+ }
	113	+
	114	+ do {
	115	+ struct page *page = module_get_next_page(info);
	116	+
	117	+ if (!page) {
	118	+ retval = -ENOMEM;
	119	+ goto out_inflate_end;
	120	+ }
	121	+
	122	+ s.next_out = kmap(page);
	123	+ s.avail_out = PAGE_SIZE;
	124	+ rc = zlib_inflate(&s, 0);
	125	+ kunmap(page);
	126	+
	127	+ new_size += PAGE_SIZE - s.avail_out;
	128	+ } while (rc == Z_OK);
	129	+
	130	+ if (rc != Z_STREAM_END) {
	131	+ pr_err("decompression failed with status %d\n", rc);
	132	+ retval = -EINVAL;
	133	+ goto out_inflate_end;
	134	+ }
	135	+
	136	+ retval = new_size;
	137	+
	138	+out_inflate_end:
	139	+ zlib_inflateEnd(&s);
	140	+out:
	141	+ kfree(s.workspace);
	142	+ return retval;
	143	+}
	144	+#elif CONFIG_MODULE_COMPRESS_XZ
	145	+#include <linux/xz.h>
	146	+#define MODULE_COMPRESSION xz
	147	+#define MODULE_DECOMPRESS_FN module_xz_decompress
	148	+
	149	+static ssize_t module_xz_decompress(struct load_info *info,
	150	+ const void *buf, size_t size)
	151	+{
	152	+ static const u8 signature[] = { 0xfd, '7', 'z', 'X', 'Z', 0 };
	153	+ struct xz_dec *xz_dec;
	154	+ struct xz_buf xz_buf;
	155	+ enum xz_ret xz_ret;
	156	+ size_t new_size = 0;
	157	+ ssize_t retval;
	158	+
	159	+ if (size < sizeof(signature) \|\|
	160	+ memcmp(buf, signature, sizeof(signature))) {
	161	+ pr_err("not an xz compressed module\n");
	162	+ return -EINVAL;
	163	+ }
	164	+
	165	+ xz_dec = xz_dec_init(XZ_DYNALLOC, (u32)-1);
	166	+ if (!xz_dec)
	167	+ return -ENOMEM;
	168	+
	169	+ xz_buf.in_size = size;
	170	+ xz_buf.in = buf;
	171	+ xz_buf.in_pos = 0;
	172	+
	173	+ do {
	174	+ struct page *page = module_get_next_page(info);
	175	+
	176	+ if (!page) {
	177	+ retval = -ENOMEM;
	178	+ goto out;
	179	+ }
	180	+
	181	+ xz_buf.out = kmap(page);
	182	+ xz_buf.out_pos = 0;
	183	+ xz_buf.out_size = PAGE_SIZE;
	184	+ xz_ret = xz_dec_run(xz_dec, &xz_buf);
	185	+ kunmap(page);
	186	+
	187	+ new_size += xz_buf.out_pos;
	188	+ } while (xz_buf.out_pos == PAGE_SIZE && xz_ret == XZ_OK);
	189	+
	190	+ if (xz_ret != XZ_STREAM_END) {
	191	+ pr_err("decompression failed with status %d\n", xz_ret);
	192	+ retval = -EINVAL;
	193	+ goto out;
	194	+ }
	195	+
	196	+ retval = new_size;
	197	+
	198	+ out:
	199	+ xz_dec_end(xz_dec);
	200	+ return retval;
	201	+}
	202	+#else
	203	+#error "Unexpected configuration for CONFIG_MODULE_DECOMPRESS"
	204	+#endif
	205	+
	206	+int module_decompress(struct load_info info, const void buf, size_t size)
	207	+{
	208	+ unsigned int n_pages;
	209	+ ssize_t data_size;
	210	+ int error;
	211	+
	212	+ /*
	213	+ * Start with number of pages twice as big as needed for
	214	+ * compressed data.
	215	+ */
	216	+ n_pages = DIV_ROUND_UP(size, PAGE_SIZE) * 2;
	217	+ error = module_extend_max_pages(info, n_pages);
	218	+
	219	+ data_size = MODULE_DECOMPRESS_FN(info, buf, size);
	220	+ if (data_size < 0) {
	221	+ error = data_size;
	222	+ goto err;
	223	+ }
	224	+
	225	+ info->hdr = vmap(info->pages, info->used_pages, VM_MAP, PAGE_KERNEL);
	226	+ if (!info->hdr) {
	227	+ error = -ENOMEM;
	228	+ goto err;
	229	+ }
	230	+
	231	+ info->len = data_size;
	232	+ return 0;
	233	+
	234	+err:
	235	+ module_decompress_cleanup(info);
	236	+ return error;
	237	+}
	238	+
	239	+void module_decompress_cleanup(struct load_info *info)
	240	+{
	241	+ int i;
	242	+
	243	+ if (info->hdr)
	244	+ vunmap(info->hdr);
	245	+
	246	+ for (i = 0; i < info->used_pages; i++)
	247	+ __free_page(info->pages[i]);
	248	+
	249	+ kvfree(info->pages);
	250	+
	251	+ info->pages = NULL;
	252	+ info->max_pages = info->used_pages = 0;
	253	+}
	254	+
	255	+#ifdef CONFIG_SYSFS
	256	+static ssize_t compression_show(struct kobject *kobj,
	257	+ struct kobj_attribute attr, char buf)
	258	+{
	259	+ return sysfs_emit(buf, "%s\n", __stringify(MODULE_COMPRESSION));
	260	+}
	261	+
	262	+static struct kobj_attribute module_compression_attr = __ATTR_RO(compression);
	263	+
	264	+static int __init module_decompress_sysfs_init(void)
	265	+{
	266	+ int error;
	267	+
	268	+ error = sysfs_create_file(&module_kset->kobj,
	269	+ &module_compression_attr.attr);
	270	+ if (error)
	271	+ pr_warn("Failed to create 'compression' attribute");
	272	+
	273	+ return 0;
	274	+}
	275	+late_initcall(module_decompress_sysfs_init);
	276	+#endif

kernel/module/internal.h+302−0

@@ -0,0 +1,302 @@
	1	+/* SPDX-License-Identifier: GPL-2.0-or-later */
	2	+/* Module internals
	3	+ *
	4	+ * Copyright (C) 2012 Red Hat, Inc. All Rights Reserved.
	5	+ * Written by David Howells (dhowells@redhat.com)
	6	+ */
	7	+
	8	+#include <linux/elf.h>
	9	+#include <linux/compiler.h>
	10	+#include <linux/module.h>
	11	+#include <linux/mutex.h>
	12	+#include <linux/rculist.h>
	13	+#include <linux/rcupdate.h>
	14	+
	15	+#ifndef ARCH_SHF_SMALL
	16	+#define ARCH_SHF_SMALL 0
	17	+#endif
	18	+
	19	+/* If this is set, the section belongs in the init part of the module */
	20	+#define INIT_OFFSET_MASK (1UL << (BITS_PER_LONG - 1))
	21	+/* Maximum number of characters written by module_flags() */
	22	+#define MODULE_FLAGS_BUF_SIZE (TAINT_FLAGS_COUNT + 4)
	23	+
	24	+#ifndef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	25	+#define data_layout core_layout
	26	+#endif
	27	+
	28	+/*
	29	+ * Modules' sections will be aligned on page boundaries
	30	+ * to ensure complete separation of code and data, but
	31	+ * only when CONFIG_STRICT_MODULE_RWX=y
	32	+ */
	33	+#ifdef CONFIG_STRICT_MODULE_RWX
	34	+# define strict_align(X) PAGE_ALIGN(X)
	35	+#else
	36	+# define strict_align(X) (X)
	37	+#endif
	38	+
	39	+extern struct mutex module_mutex;
	40	+extern struct list_head modules;
	41	+
	42	+extern struct module_attribute *modinfo_attrs[];
	43	+extern size_t modinfo_attrs_count;
	44	+
	45	+/* Provided by the linker */
	46	+extern const struct kernel_symbol __start___ksymtab[];
	47	+extern const struct kernel_symbol __stop___ksymtab[];
	48	+extern const struct kernel_symbol __start___ksymtab_gpl[];
	49	+extern const struct kernel_symbol __stop___ksymtab_gpl[];
	50	+extern const s32 __start___kcrctab[];
	51	+extern const s32 __start___kcrctab_gpl[];
	52	+
	53	+struct load_info {
	54	+ const char *name;
	55	+ /* pointer to module in temporary copy, freed at end of load_module() */
	56	+ struct module *mod;
	57	+ Elf_Ehdr *hdr;
	58	+ unsigned long len;
	59	+ Elf_Shdr *sechdrs;
	60	+ char secstrings, strtab;
	61	+ unsigned long symoffs, stroffs, init_typeoffs, core_typeoffs;
	62	+ struct _ddebug *debug;
	63	+ unsigned int num_debug;
	64	+ bool sig_ok;
	65	+#ifdef CONFIG_KALLSYMS
	66	+ unsigned long mod_kallsyms_init_off;
	67	+#endif
	68	+#ifdef CONFIG_MODULE_DECOMPRESS
	69	+ struct page **pages;
	70	+ unsigned int max_pages;
	71	+ unsigned int used_pages;
	72	+#endif
	73	+ struct {
	74	+ unsigned int sym, str, mod, vers, info, pcpu;
	75	+ } index;
	76	+};
	77	+
	78	+enum mod_license {
	79	+ NOT_GPL_ONLY,
	80	+ GPL_ONLY,
	81	+};
	82	+
	83	+struct find_symbol_arg {
	84	+ /* Input */
	85	+ const char *name;
	86	+ bool gplok;
	87	+ bool warn;
	88	+
	89	+ /* Output */
	90	+ struct module *owner;
	91	+ const s32 *crc;
	92	+ const struct kernel_symbol *sym;
	93	+ enum mod_license license;
	94	+};
	95	+
	96	+int mod_verify_sig(const void mod, struct load_info info);
	97	+int try_to_force_load(struct module mod, const char reason);
	98	+bool find_symbol(struct find_symbol_arg *fsa);
	99	+struct module find_module_all(const char name, size_t len, bool even_unformed);
	100	+int cmp_name(const void name, const void sym);
	101	+long module_get_offset(struct module mod, unsigned int size, Elf_Shdr *sechdr,
	102	+ unsigned int section);
	103	+char module_flags(struct module mod, char *buf);
	104	+size_t module_flags_taint(unsigned long taints, char *buf);
	105	+
	106	+static inline void module_assert_mutex_or_preempt(void)
	107	+{
	108	+#ifdef CONFIG_LOCKDEP
	109	+ if (unlikely(!debug_locks))
	110	+ return;
	111	+
	112	+ WARN_ON_ONCE(!rcu_read_lock_sched_held() &&
	113	+ !lockdep_is_held(&module_mutex));
	114	+#endif
	115	+}
	116	+
	117	+static inline unsigned long kernel_symbol_value(const struct kernel_symbol *sym)
	118	+{
	119	+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
	120	+ return (unsigned long)offset_to_ptr(&sym->value_offset);
	121	+#else
	122	+ return sym->value;
	123	+#endif
	124	+}
	125	+
	126	+#ifdef CONFIG_LIVEPATCH
	127	+int copy_module_elf(struct module mod, struct load_info info);
	128	+void free_module_elf(struct module *mod);
	129	+#else /* !CONFIG_LIVEPATCH */
	130	+static inline int copy_module_elf(struct module mod, struct load_info info)
	131	+{
	132	+ return 0;
	133	+}
	134	+
	135	+static inline void free_module_elf(struct module *mod) { }
	136	+#endif /* CONFIG_LIVEPATCH */
	137	+
	138	+static inline bool set_livepatch_module(struct module *mod)
	139	+{
	140	+#ifdef CONFIG_LIVEPATCH
	141	+ mod->klp = true;
	142	+ return true;
	143	+#else
	144	+ return false;
	145	+#endif
	146	+}
	147	+
	148	+#ifdef CONFIG_MODULE_UNLOAD_TAINT_TRACKING
	149	+struct mod_unload_taint {
	150	+ struct list_head list;
	151	+ char name[MODULE_NAME_LEN];
	152	+ unsigned long taints;
	153	+ u64 count;
	154	+};
	155	+
	156	+int try_add_tainted_module(struct module *mod);
	157	+void print_unloaded_tainted_modules(void);
	158	+#else /* !CONFIG_MODULE_UNLOAD_TAINT_TRACKING */
	159	+static inline int try_add_tainted_module(struct module *mod)
	160	+{
	161	+ return 0;
	162	+}
	163	+
	164	+static inline void print_unloaded_tainted_modules(void)
	165	+{
	166	+}
	167	+#endif /* CONFIG_MODULE_UNLOAD_TAINT_TRACKING */
	168	+
	169	+#ifdef CONFIG_MODULE_DECOMPRESS
	170	+int module_decompress(struct load_info info, const void buf, size_t size);
	171	+void module_decompress_cleanup(struct load_info *info);
	172	+#else
	173	+static inline int module_decompress(struct load_info *info,
	174	+ const void *buf, size_t size)
	175	+{
	176	+ return -EOPNOTSUPP;
	177	+}
	178	+
	179	+static inline void module_decompress_cleanup(struct load_info *info)
	180	+{
	181	+}
	182	+#endif
	183	+
	184	+struct mod_tree_root {
	185	+#ifdef CONFIG_MODULES_TREE_LOOKUP
	186	+ struct latch_tree_root root;
	187	+#endif
	188	+ unsigned long addr_min;
	189	+ unsigned long addr_max;
	190	+};
	191	+
	192	+extern struct mod_tree_root mod_tree;
	193	+extern struct mod_tree_root mod_data_tree;
	194	+
	195	+#ifdef CONFIG_MODULES_TREE_LOOKUP
	196	+void mod_tree_insert(struct module *mod);
	197	+void mod_tree_remove_init(struct module *mod);
	198	+void mod_tree_remove(struct module *mod);
	199	+struct module mod_find(unsigned long addr, struct mod_tree_root tree);
	200	+#else /* !CONFIG_MODULES_TREE_LOOKUP */
	201	+
	202	+static inline void mod_tree_insert(struct module *mod) { }
	203	+static inline void mod_tree_remove_init(struct module *mod) { }
	204	+static inline void mod_tree_remove(struct module *mod) { }
	205	+static inline struct module mod_find(unsigned long addr, struct mod_tree_root tree)
	206	+{
	207	+ struct module *mod;
	208	+
	209	+ list_for_each_entry_rcu(mod, &modules, list,
	210	+ lockdep_is_held(&module_mutex)) {
	211	+ if (within_module(addr, mod))
	212	+ return mod;
	213	+ }
	214	+
	215	+ return NULL;
	216	+}
	217	+#endif /* CONFIG_MODULES_TREE_LOOKUP */
	218	+
	219	+void module_enable_ro(const struct module *mod, bool after_init);
	220	+void module_enable_nx(const struct module *mod);
	221	+void module_enable_x(const struct module *mod);
	222	+int module_enforce_rwx_sections(Elf_Ehdr hdr, Elf_Shdr sechdrs,
	223	+ char secstrings, struct module mod);
	224	+bool module_check_misalignment(const struct module *mod);
	225	+
	226	+#ifdef CONFIG_MODULE_SIG
	227	+int module_sig_check(struct load_info *info, int flags);
	228	+#else /* !CONFIG_MODULE_SIG */
	229	+static inline int module_sig_check(struct load_info *info, int flags)
	230	+{
	231	+ return 0;
	232	+}
	233	+#endif /* !CONFIG_MODULE_SIG */
	234	+
	235	+#ifdef CONFIG_DEBUG_KMEMLEAK
	236	+void kmemleak_load_module(const struct module mod, const struct load_info info);
	237	+#else /* !CONFIG_DEBUG_KMEMLEAK */
	238	+static inline void kmemleak_load_module(const struct module *mod,
	239	+ const struct load_info *info) { }
	240	+#endif /* CONFIG_DEBUG_KMEMLEAK */
	241	+
	242	+#ifdef CONFIG_KALLSYMS
	243	+void init_build_id(struct module mod, const struct load_info info);
	244	+void layout_symtab(struct module mod, struct load_info info);
	245	+void add_kallsyms(struct module mod, const struct load_info info);
	246	+unsigned long find_kallsyms_symbol_value(struct module mod, const char name);
	247	+
	248	+static inline bool sect_empty(const Elf_Shdr *sect)
	249	+{
	250	+ return !(sect->sh_flags & SHF_ALLOC) \|\| sect->sh_size == 0;
	251	+}
	252	+#else /* !CONFIG_KALLSYMS */
	253	+static inline void init_build_id(struct module mod, const struct load_info info) { }
	254	+static inline void layout_symtab(struct module mod, struct load_info info) { }
	255	+static inline void add_kallsyms(struct module mod, const struct load_info info) { }
	256	+#endif /* CONFIG_KALLSYMS */
	257	+
	258	+#ifdef CONFIG_SYSFS
	259	+int mod_sysfs_setup(struct module mod, const struct load_info info,
	260	+ struct kernel_param *kparam, unsigned int num_params);
	261	+void mod_sysfs_teardown(struct module *mod);
	262	+void init_param_lock(struct module *mod);
	263	+#else /* !CONFIG_SYSFS */
	264	+static inline int mod_sysfs_setup(struct module *mod,
	265	+ const struct load_info *info,
	266	+ struct kernel_param *kparam,
	267	+ unsigned int num_params)
	268	+{
	269	+ return 0;
	270	+}
	271	+
	272	+static inline void mod_sysfs_teardown(struct module *mod) { }
	273	+static inline void init_param_lock(struct module *mod) { }
	274	+#endif /* CONFIG_SYSFS */
	275	+
	276	+#ifdef CONFIG_MODVERSIONS
	277	+int check_version(const struct load_info *info,
	278	+ const char symname, struct module mod, const s32 *crc);
	279	+void module_layout(struct module mod, struct modversion_info ver, struct kernel_param *kp,
	280	+ struct kernel_symbol ks, struct tracepoint const *tp);
	281	+int check_modstruct_version(const struct load_info info, struct module mod);
	282	+int same_magic(const char amagic, const char bmagic, bool has_crcs);
	283	+#else /* !CONFIG_MODVERSIONS */
	284	+static inline int check_version(const struct load_info *info,
	285	+ const char *symname,
	286	+ struct module *mod,
	287	+ const s32 *crc)
	288	+{
	289	+ return 1;
	290	+}
	291	+
	292	+static inline int check_modstruct_version(const struct load_info *info,
	293	+ struct module *mod)
	294	+{
	295	+ return 1;
	296	+}
	297	+
	298	+static inline int same_magic(const char amagic, const char bmagic, bool has_crcs)
	299	+{
	300	+ return strcmp(amagic, bmagic) == 0;
	301	+}
	302	+#endif /* CONFIG_MODVERSIONS */

kernel/module/kallsyms.c+512−0

@@ -0,0 +1,512 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module kallsyms support
	4	+ *
	5	+ * Copyright (C) 2010 Rusty Russell
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/kallsyms.h>
	10	+#include <linux/buildid.h>
	11	+#include <linux/bsearch.h>
	12	+#include "internal.h"
	13	+
	14	+/* Lookup exported symbol in given range of kernel_symbols */
	15	+static const struct kernel_symbol lookup_exported_symbol(const char name,
	16	+ const struct kernel_symbol *start,
	17	+ const struct kernel_symbol *stop)
	18	+{
	19	+ return bsearch(name, start, stop - start,
	20	+ sizeof(struct kernel_symbol), cmp_name);
	21	+}
	22	+
	23	+static int is_exported(const char *name, unsigned long value,
	24	+ const struct module *mod)
	25	+{
	26	+ const struct kernel_symbol *ks;
	27	+
	28	+ if (!mod)
	29	+ ks = lookup_exported_symbol(name, __start___ksymtab, __stop___ksymtab);
	30	+ else
	31	+ ks = lookup_exported_symbol(name, mod->syms, mod->syms + mod->num_syms);
	32	+
	33	+ return ks && kernel_symbol_value(ks) == value;
	34	+}
	35	+
	36	+/* As per nm */
	37	+static char elf_type(const Elf_Sym sym, const struct load_info info)
	38	+{
	39	+ const Elf_Shdr *sechdrs = info->sechdrs;
	40	+
	41	+ if (ELF_ST_BIND(sym->st_info) == STB_WEAK) {
	42	+ if (ELF_ST_TYPE(sym->st_info) == STT_OBJECT)
	43	+ return 'v';
	44	+ else
	45	+ return 'w';
	46	+ }
	47	+ if (sym->st_shndx == SHN_UNDEF)
	48	+ return 'U';
	49	+ if (sym->st_shndx == SHN_ABS \|\| sym->st_shndx == info->index.pcpu)
	50	+ return 'a';
	51	+ if (sym->st_shndx >= SHN_LORESERVE)
	52	+ return '?';
	53	+ if (sechdrs[sym->st_shndx].sh_flags & SHF_EXECINSTR)
	54	+ return 't';
	55	+ if (sechdrs[sym->st_shndx].sh_flags & SHF_ALLOC &&
	56	+ sechdrs[sym->st_shndx].sh_type != SHT_NOBITS) {
	57	+ if (!(sechdrs[sym->st_shndx].sh_flags & SHF_WRITE))
	58	+ return 'r';
	59	+ else if (sechdrs[sym->st_shndx].sh_flags & ARCH_SHF_SMALL)
	60	+ return 'g';
	61	+ else
	62	+ return 'd';
	63	+ }
	64	+ if (sechdrs[sym->st_shndx].sh_type == SHT_NOBITS) {
	65	+ if (sechdrs[sym->st_shndx].sh_flags & ARCH_SHF_SMALL)
	66	+ return 's';
	67	+ else
	68	+ return 'b';
	69	+ }
	70	+ if (strstarts(info->secstrings + sechdrs[sym->st_shndx].sh_name,
	71	+ ".debug")) {
	72	+ return 'n';
	73	+ }
	74	+ return '?';
	75	+}
	76	+
	77	+static bool is_core_symbol(const Elf_Sym src, const Elf_Shdr sechdrs,
	78	+ unsigned int shnum, unsigned int pcpundx)
	79	+{
	80	+ const Elf_Shdr *sec;
	81	+
	82	+ if (src->st_shndx == SHN_UNDEF \|\|
	83	+ src->st_shndx >= shnum \|\|
	84	+ !src->st_name)
	85	+ return false;
	86	+
	87	+#ifdef CONFIG_KALLSYMS_ALL
	88	+ if (src->st_shndx == pcpundx)
	89	+ return true;
	90	+#endif
	91	+
	92	+ sec = sechdrs + src->st_shndx;
	93	+ if (!(sec->sh_flags & SHF_ALLOC)
	94	+#ifndef CONFIG_KALLSYMS_ALL
	95	+ \|\| !(sec->sh_flags & SHF_EXECINSTR)
	96	+#endif
	97	+ \|\| (sec->sh_entsize & INIT_OFFSET_MASK))
	98	+ return false;
	99	+
	100	+ return true;
	101	+}
	102	+
	103	+/*
	104	+ * We only allocate and copy the strings needed by the parts of symtab
	105	+ * we keep. This is simple, but has the effect of making multiple
	106	+ * copies of duplicates. We could be more sophisticated, see
	107	+ * linux-kernel thread starting with
	108	+ * <73defb5e4bca04a6431392cc341112b1@localhost>.
	109	+ */
	110	+void layout_symtab(struct module mod, struct load_info info)
	111	+{
	112	+ Elf_Shdr *symsect = info->sechdrs + info->index.sym;
	113	+ Elf_Shdr *strsect = info->sechdrs + info->index.str;
	114	+ const Elf_Sym *src;
	115	+ unsigned int i, nsrc, ndst, strtab_size = 0;
	116	+
	117	+ /* Put symbol section at end of init part of module. */
	118	+ symsect->sh_flags \|= SHF_ALLOC;
	119	+ symsect->sh_entsize = module_get_offset(mod, &mod->init_layout.size, symsect,
	120	+ info->index.sym) \| INIT_OFFSET_MASK;
	121	+ pr_debug("\t%s\n", info->secstrings + symsect->sh_name);
	122	+
	123	+ src = (void *)info->hdr + symsect->sh_offset;
	124	+ nsrc = symsect->sh_size / sizeof(*src);
	125	+
	126	+ /* Compute total space required for the core symbols' strtab. */
	127	+ for (ndst = i = 0; i < nsrc; i++) {
	128	+ if (i == 0 \|\| is_livepatch_module(mod) \|\|
	129	+ is_core_symbol(src + i, info->sechdrs, info->hdr->e_shnum,
	130	+ info->index.pcpu)) {
	131	+ strtab_size += strlen(&info->strtab[src[i].st_name]) + 1;
	132	+ ndst++;
	133	+ }
	134	+ }
	135	+
	136	+ /* Append room for core symbols at end of core part. */
	137	+ info->symoffs = ALIGN(mod->data_layout.size, symsect->sh_addralign ?: 1);
	138	+ info->stroffs = mod->data_layout.size = info->symoffs + ndst * sizeof(Elf_Sym);
	139	+ mod->data_layout.size += strtab_size;
	140	+ info->core_typeoffs = mod->data_layout.size;
	141	+ mod->data_layout.size += ndst * sizeof(char);
	142	+ mod->data_layout.size = strict_align(mod->data_layout.size);
	143	+
	144	+ /* Put string table section at end of init part of module. */
	145	+ strsect->sh_flags \|= SHF_ALLOC;
	146	+ strsect->sh_entsize = module_get_offset(mod, &mod->init_layout.size, strsect,
	147	+ info->index.str) \| INIT_OFFSET_MASK;
	148	+ pr_debug("\t%s\n", info->secstrings + strsect->sh_name);
	149	+
	150	+ /* We'll tack temporary mod_kallsyms on the end. */
	151	+ mod->init_layout.size = ALIGN(mod->init_layout.size,
	152	+ __alignof__(struct mod_kallsyms));
	153	+ info->mod_kallsyms_init_off = mod->init_layout.size;
	154	+ mod->init_layout.size += sizeof(struct mod_kallsyms);
	155	+ info->init_typeoffs = mod->init_layout.size;
	156	+ mod->init_layout.size += nsrc * sizeof(char);
	157	+ mod->init_layout.size = strict_align(mod->init_layout.size);
	158	+}
	159	+
	160	+/*
	161	+ * We use the full symtab and strtab which layout_symtab arranged to
	162	+ * be appended to the init section. Later we switch to the cut-down
	163	+ * core-only ones.
	164	+ */
	165	+void add_kallsyms(struct module mod, const struct load_info info)
	166	+{
	167	+ unsigned int i, ndst;
	168	+ const Elf_Sym *src;
	169	+ Elf_Sym *dst;
	170	+ char *s;
	171	+ Elf_Shdr *symsec = &info->sechdrs[info->index.sym];
	172	+
	173	+ /* Set up to point into init section. */
	174	+ mod->kallsyms = (void __rcu *)mod->init_layout.base +
	175	+ info->mod_kallsyms_init_off;
	176	+
	177	+ preempt_disable();
	178	+ /* The following is safe since this pointer cannot change */
	179	+ rcu_dereference_sched(mod->kallsyms)->symtab = (void *)symsec->sh_addr;
	180	+ rcu_dereference_sched(mod->kallsyms)->num_symtab = symsec->sh_size / sizeof(Elf_Sym);
	181	+ /* Make sure we get permanent strtab: don't use info->strtab. */
	182	+ rcu_dereference_sched(mod->kallsyms)->strtab =
	183	+ (void *)info->sechdrs[info->index.str].sh_addr;
	184	+ rcu_dereference_sched(mod->kallsyms)->typetab = mod->init_layout.base + info->init_typeoffs;
	185	+
	186	+ /*
	187	+ * Now populate the cut down core kallsyms for after init
	188	+ * and set types up while we still have access to sections.
	189	+ */
	190	+ mod->core_kallsyms.symtab = dst = mod->data_layout.base + info->symoffs;
	191	+ mod->core_kallsyms.strtab = s = mod->data_layout.base + info->stroffs;
	192	+ mod->core_kallsyms.typetab = mod->data_layout.base + info->core_typeoffs;
	193	+ src = rcu_dereference_sched(mod->kallsyms)->symtab;
	194	+ for (ndst = i = 0; i < rcu_dereference_sched(mod->kallsyms)->num_symtab; i++) {
	195	+ rcu_dereference_sched(mod->kallsyms)->typetab[i] = elf_type(src + i, info);
	196	+ if (i == 0 \|\| is_livepatch_module(mod) \|\|
	197	+ is_core_symbol(src + i, info->sechdrs, info->hdr->e_shnum,
	198	+ info->index.pcpu)) {
	199	+ mod->core_kallsyms.typetab[ndst] =
	200	+ rcu_dereference_sched(mod->kallsyms)->typetab[i];
	201	+ dst[ndst] = src[i];
	202	+ dst[ndst++].st_name = s - mod->core_kallsyms.strtab;
	203	+ s += strscpy(s,
	204	+ &rcu_dereference_sched(mod->kallsyms)->strtab[src[i].st_name],
	205	+ KSYM_NAME_LEN) + 1;
	206	+ }
	207	+ }
	208	+ preempt_enable();
	209	+ mod->core_kallsyms.num_symtab = ndst;
	210	+}
	211	+
	212	+#if IS_ENABLED(CONFIG_STACKTRACE_BUILD_ID)
	213	+void init_build_id(struct module mod, const struct load_info info)
	214	+{
	215	+ const Elf_Shdr *sechdr;
	216	+ unsigned int i;
	217	+
	218	+ for (i = 0; i < info->hdr->e_shnum; i++) {
	219	+ sechdr = &info->sechdrs[i];
	220	+ if (!sect_empty(sechdr) && sechdr->sh_type == SHT_NOTE &&
	221	+ !build_id_parse_buf((void *)sechdr->sh_addr, mod->build_id,
	222	+ sechdr->sh_size))
	223	+ break;
	224	+ }
	225	+}
	226	+#else
	227	+void init_build_id(struct module mod, const struct load_info info)
	228	+{
	229	+}
	230	+#endif
	231	+
	232	+/*
	233	+ * This ignores the intensely annoying "mapping symbols" found
	234	+ * in ARM ELF files: $a, $t and $d.
	235	+ */
	236	+static inline int is_arm_mapping_symbol(const char *str)
	237	+{
	238	+ if (str[0] == '.' && str[1] == 'L')
	239	+ return true;
	240	+ return str[0] == '$' && strchr("axtd", str[1]) &&
	241	+ (str[2] == '\0' \|\| str[2] == '.');
	242	+}
	243	+
	244	+static const char kallsyms_symbol_name(struct mod_kallsyms kallsyms, unsigned int symnum)
	245	+{
	246	+ return kallsyms->strtab + kallsyms->symtab[symnum].st_name;
	247	+}
	248	+
	249	+/*
	250	+ * Given a module and address, find the corresponding symbol and return its name
	251	+ * while providing its size and offset if needed.
	252	+ */
	253	+static const char find_kallsyms_symbol(struct module mod,
	254	+ unsigned long addr,
	255	+ unsigned long *size,
	256	+ unsigned long *offset)
	257	+{
	258	+ unsigned int i, best = 0;
	259	+ unsigned long nextval, bestval;
	260	+ struct mod_kallsyms *kallsyms = rcu_dereference_sched(mod->kallsyms);
	261	+
	262	+ /* At worse, next value is at end of module */
	263	+ if (within_module_init(addr, mod))
	264	+ nextval = (unsigned long)mod->init_layout.base + mod->init_layout.text_size;
	265	+ else
	266	+ nextval = (unsigned long)mod->core_layout.base + mod->core_layout.text_size;
	267	+
	268	+ bestval = kallsyms_symbol_value(&kallsyms->symtab[best]);
	269	+
	270	+ /*
	271	+ * Scan for closest preceding symbol, and next symbol. (ELF
	272	+ * starts real symbols at 1).
	273	+ */
	274	+ for (i = 1; i < kallsyms->num_symtab; i++) {
	275	+ const Elf_Sym *sym = &kallsyms->symtab[i];
	276	+ unsigned long thisval = kallsyms_symbol_value(sym);
	277	+
	278	+ if (sym->st_shndx == SHN_UNDEF)
	279	+ continue;
	280	+
	281	+ /*
	282	+ * We ignore unnamed symbols: they're uninformative
	283	+ * and inserted at a whim.
	284	+ */
	285	+ if (*kallsyms_symbol_name(kallsyms, i) == '\0' \|\|
	286	+ is_arm_mapping_symbol(kallsyms_symbol_name(kallsyms, i)))
	287	+ continue;
	288	+
	289	+ if (thisval <= addr && thisval > bestval) {
	290	+ best = i;
	291	+ bestval = thisval;
	292	+ }
	293	+ if (thisval > addr && thisval < nextval)
	294	+ nextval = thisval;
	295	+ }
	296	+
	297	+ if (!best)
	298	+ return NULL;
	299	+
	300	+ if (size)
	301	+ *size = nextval - bestval;
	302	+ if (offset)
	303	+ *offset = addr - bestval;
	304	+
	305	+ return kallsyms_symbol_name(kallsyms, best);
	306	+}
	307	+
	308	+void * __weak dereference_module_function_descriptor(struct module *mod,
	309	+ void *ptr)
	310	+{
	311	+ return ptr;
	312	+}
	313	+
	314	+/*
	315	+ * For kallsyms to ask for address resolution. NULL means not found. Careful
	316	+ * not to lock to avoid deadlock on oopses, simply disable preemption.
	317	+ */
	318	+const char *module_address_lookup(unsigned long addr,
	319	+ unsigned long *size,
	320	+ unsigned long *offset,
	321	+ char **modname,
	322	+ const unsigned char **modbuildid,
	323	+ char *namebuf)
	324	+{
	325	+ const char *ret = NULL;
	326	+ struct module *mod;
	327	+
	328	+ preempt_disable();
	329	+ mod = __module_address(addr);
	330	+ if (mod) {
	331	+ if (modname)
	332	+ *modname = mod->name;
	333	+ if (modbuildid) {
	334	+#if IS_ENABLED(CONFIG_STACKTRACE_BUILD_ID)
	335	+ *modbuildid = mod->build_id;
	336	+#else
	337	+ *modbuildid = NULL;
	338	+#endif
	339	+ }
	340	+
	341	+ ret = find_kallsyms_symbol(mod, addr, size, offset);
	342	+ }
	343	+ /* Make a copy in here where it's safe */
	344	+ if (ret) {
	345	+ strncpy(namebuf, ret, KSYM_NAME_LEN - 1);
	346	+ ret = namebuf;
	347	+ }
	348	+ preempt_enable();
	349	+
	350	+ return ret;
	351	+}
	352	+
	353	+int lookup_module_symbol_name(unsigned long addr, char *symname)
	354	+{
	355	+ struct module *mod;
	356	+
	357	+ preempt_disable();
	358	+ list_for_each_entry_rcu(mod, &modules, list) {
	359	+ if (mod->state == MODULE_STATE_UNFORMED)
	360	+ continue;
	361	+ if (within_module(addr, mod)) {
	362	+ const char *sym;
	363	+
	364	+ sym = find_kallsyms_symbol(mod, addr, NULL, NULL);
	365	+ if (!sym)
	366	+ goto out;
	367	+
	368	+ strscpy(symname, sym, KSYM_NAME_LEN);
	369	+ preempt_enable();
	370	+ return 0;
	371	+ }
	372	+ }
	373	+out:
	374	+ preempt_enable();
	375	+ return -ERANGE;
	376	+}
	377	+
	378	+int lookup_module_symbol_attrs(unsigned long addr, unsigned long *size,
	379	+ unsigned long offset, char modname, char *name)
	380	+{
	381	+ struct module *mod;
	382	+
	383	+ preempt_disable();
	384	+ list_for_each_entry_rcu(mod, &modules, list) {
	385	+ if (mod->state == MODULE_STATE_UNFORMED)
	386	+ continue;
	387	+ if (within_module(addr, mod)) {
	388	+ const char *sym;
	389	+
	390	+ sym = find_kallsyms_symbol(mod, addr, size, offset);
	391	+ if (!sym)
	392	+ goto out;
	393	+ if (modname)
	394	+ strscpy(modname, mod->name, MODULE_NAME_LEN);
	395	+ if (name)
	396	+ strscpy(name, sym, KSYM_NAME_LEN);
	397	+ preempt_enable();
	398	+ return 0;
	399	+ }
	400	+ }
	401	+out:
	402	+ preempt_enable();
	403	+ return -ERANGE;
	404	+}
	405	+
	406	+int module_get_kallsym(unsigned int symnum, unsigned long value, char type,
	407	+ char name, char module_name, int *exported)
	408	+{
	409	+ struct module *mod;
	410	+
	411	+ preempt_disable();
	412	+ list_for_each_entry_rcu(mod, &modules, list) {
	413	+ struct mod_kallsyms *kallsyms;
	414	+
	415	+ if (mod->state == MODULE_STATE_UNFORMED)
	416	+ continue;
	417	+ kallsyms = rcu_dereference_sched(mod->kallsyms);
	418	+ if (symnum < kallsyms->num_symtab) {
	419	+ const Elf_Sym *sym = &kallsyms->symtab[symnum];
	420	+
	421	+ *value = kallsyms_symbol_value(sym);
	422	+ *type = kallsyms->typetab[symnum];
	423	+ strscpy(name, kallsyms_symbol_name(kallsyms, symnum), KSYM_NAME_LEN);
	424	+ strscpy(module_name, mod->name, MODULE_NAME_LEN);
	425	+ exported = is_exported(name, value, mod);
	426	+ preempt_enable();
	427	+ return 0;
	428	+ }
	429	+ symnum -= kallsyms->num_symtab;
	430	+ }
	431	+ preempt_enable();
	432	+ return -ERANGE;
	433	+}
	434	+
	435	+/* Given a module and name of symbol, find and return the symbol's value */
	436	+unsigned long find_kallsyms_symbol_value(struct module mod, const char name)
	437	+{
	438	+ unsigned int i;
	439	+ struct mod_kallsyms *kallsyms = rcu_dereference_sched(mod->kallsyms);
	440	+
	441	+ for (i = 0; i < kallsyms->num_symtab; i++) {
	442	+ const Elf_Sym *sym = &kallsyms->symtab[i];
	443	+
	444	+ if (strcmp(name, kallsyms_symbol_name(kallsyms, i)) == 0 &&
	445	+ sym->st_shndx != SHN_UNDEF)
	446	+ return kallsyms_symbol_value(sym);
	447	+ }
	448	+ return 0;
	449	+}
	450	+
	451	+/* Look for this name: can be of form module:name. */
	452	+unsigned long module_kallsyms_lookup_name(const char *name)
	453	+{
	454	+ struct module *mod;
	455	+ char *colon;
	456	+ unsigned long ret = 0;
	457	+
	458	+ /* Don't lock: we're in enough trouble already. */
	459	+ preempt_disable();
	460	+ if ((colon = strnchr(name, MODULE_NAME_LEN, ':')) != NULL) {
	461	+ if ((mod = find_module_all(name, colon - name, false)) != NULL)
	462	+ ret = find_kallsyms_symbol_value(mod, colon + 1);
	463	+ } else {
	464	+ list_for_each_entry_rcu(mod, &modules, list) {
	465	+ if (mod->state == MODULE_STATE_UNFORMED)
	466	+ continue;
	467	+ if ((ret = find_kallsyms_symbol_value(mod, name)) != 0)
	468	+ break;
	469	+ }
	470	+ }
	471	+ preempt_enable();
	472	+ return ret;
	473	+}
	474	+
	475	+#ifdef CONFIG_LIVEPATCH
	476	+int module_kallsyms_on_each_symbol(int (fn)(void , const char *,
	477	+ struct module *, unsigned long),
	478	+ void *data)
	479	+{
	480	+ struct module *mod;
	481	+ unsigned int i;
	482	+ int ret = 0;
	483	+
	484	+ mutex_lock(&module_mutex);
	485	+ list_for_each_entry(mod, &modules, list) {
	486	+ struct mod_kallsyms *kallsyms;
	487	+
	488	+ if (mod->state == MODULE_STATE_UNFORMED)
	489	+ continue;
	490	+
	491	+ /* Use rcu_dereference_sched() to remain compliant with the sparse tool */
	492	+ preempt_disable();
	493	+ kallsyms = rcu_dereference_sched(mod->kallsyms);
	494	+ preempt_enable();
	495	+
	496	+ for (i = 0; i < kallsyms->num_symtab; i++) {
	497	+ const Elf_Sym *sym = &kallsyms->symtab[i];
	498	+
	499	+ if (sym->st_shndx == SHN_UNDEF)
	500	+ continue;
	501	+
	502	+ ret = fn(data, kallsyms_symbol_name(kallsyms, i),
	503	+ mod, kallsyms_symbol_value(sym));
	504	+ if (ret != 0)
	505	+ goto out;
	506	+ }
	507	+ }
	508	+out:
	509	+ mutex_unlock(&module_mutex);
	510	+ return ret;
	511	+}
	512	+#endif /* CONFIG_LIVEPATCH */

kernel/module/kdb.c+62−0

@@ -0,0 +1,62 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module kdb support
	4	+ *
	5	+ * Copyright (C) 2010 Jason Wessel
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/kdb.h>
	10	+#include "internal.h"
	11	+
	12	+/*
	13	+ * kdb_lsmod - This function implements the 'lsmod' command. Lists
	14	+ * currently loaded kernel modules.
	15	+ * Mostly taken from userland lsmod.
	16	+ */
	17	+int kdb_lsmod(int argc, const char **argv)
	18	+{
	19	+ struct module *mod;
	20	+
	21	+ if (argc != 0)
	22	+ return KDB_ARGCOUNT;
	23	+
	24	+ kdb_printf("Module Size modstruct Used by\n");
	25	+ list_for_each_entry(mod, &modules, list) {
	26	+ if (mod->state == MODULE_STATE_UNFORMED)
	27	+ continue;
	28	+
	29	+ kdb_printf("%-20s%8u", mod->name, mod->core_layout.size);
	30	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	31	+ kdb_printf("/%8u", mod->data_layout.size);
	32	+#endif
	33	+ kdb_printf(" 0x%px ", (void *)mod);
	34	+#ifdef CONFIG_MODULE_UNLOAD
	35	+ kdb_printf("%4d ", module_refcount(mod));
	36	+#endif
	37	+ if (mod->state == MODULE_STATE_GOING)
	38	+ kdb_printf(" (Unloading)");
	39	+ else if (mod->state == MODULE_STATE_COMING)
	40	+ kdb_printf(" (Loading)");
	41	+ else
	42	+ kdb_printf(" (Live)");
	43	+ kdb_printf(" 0x%px", mod->core_layout.base);
	44	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	45	+ kdb_printf("/0x%px", mod->data_layout.base);
	46	+#endif
	47	+
	48	+#ifdef CONFIG_MODULE_UNLOAD
	49	+ {
	50	+ struct module_use *use;
	51	+
	52	+ kdb_printf(" [ ");
	53	+ list_for_each_entry(use, &mod->source_list,
	54	+ source_list)
	55	+ kdb_printf("%s ", use->target->name);
	56	+ kdb_printf("]\n");
	57	+ }
	58	+#endif
	59	+ }
	60	+
	61	+ return 0;
	62	+}

kernel/module/livepatch.c+74−0

@@ -0,0 +1,74 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module livepatch support
	4	+ *
	5	+ * Copyright (C) 2016 Jessica Yu <jeyu@redhat.com>
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/string.h>
	10	+#include <linux/slab.h>
	11	+#include "internal.h"
	12	+
	13	+/*
	14	+ * Persist Elf information about a module. Copy the Elf header,
	15	+ * section header table, section string table, and symtab section
	16	+ * index from info to mod->klp_info.
	17	+ */
	18	+int copy_module_elf(struct module mod, struct load_info info)
	19	+{
	20	+ unsigned int size, symndx;
	21	+ int ret;
	22	+
	23	+ size = sizeof(*mod->klp_info);
	24	+ mod->klp_info = kmalloc(size, GFP_KERNEL);
	25	+ if (!mod->klp_info)
	26	+ return -ENOMEM;
	27	+
	28	+ /* Elf header */
	29	+ size = sizeof(mod->klp_info->hdr);
	30	+ memcpy(&mod->klp_info->hdr, info->hdr, size);
	31	+
	32	+ /* Elf section header table */
	33	+ size = sizeof(info->sechdrs) info->hdr->e_shnum;
	34	+ mod->klp_info->sechdrs = kmemdup(info->sechdrs, size, GFP_KERNEL);
	35	+ if (!mod->klp_info->sechdrs) {
	36	+ ret = -ENOMEM;
	37	+ goto free_info;
	38	+ }
	39	+
	40	+ /* Elf section name string table */
	41	+ size = info->sechdrs[info->hdr->e_shstrndx].sh_size;
	42	+ mod->klp_info->secstrings = kmemdup(info->secstrings, size, GFP_KERNEL);
	43	+ if (!mod->klp_info->secstrings) {
	44	+ ret = -ENOMEM;
	45	+ goto free_sechdrs;
	46	+ }
	47	+
	48	+ /* Elf symbol section index */
	49	+ symndx = info->index.sym;
	50	+ mod->klp_info->symndx = symndx;
	51	+
	52	+ /*
	53	+ * For livepatch modules, core_kallsyms.symtab is a complete
	54	+ * copy of the original symbol table. Adjust sh_addr to point
	55	+ * to core_kallsyms.symtab since the copy of the symtab in module
	56	+ * init memory is freed at the end of do_init_module().
	57	+ */
	58	+ mod->klp_info->sechdrs[symndx].sh_addr = (unsigned long)mod->core_kallsyms.symtab;
	59	+
	60	+ return 0;
	61	+
	62	+free_sechdrs:
	63	+ kfree(mod->klp_info->sechdrs);
	64	+free_info:
	65	+ kfree(mod->klp_info);
	66	+ return ret;
	67	+}
	68	+
	69	+void free_module_elf(struct module *mod)
	70	+{
	71	+ kfree(mod->klp_info->sechdrs);
	72	+ kfree(mod->klp_info->secstrings);
	73	+ kfree(mod->klp_info);
	74	+}

kernel/module/main.c+3131−0

@@ -0,0 +1,3131 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Copyright (C) 2002 Richard Henderson
	4	+ * Copyright (C) 2001 Rusty Russell, 2002, 2010 Rusty Russell IBM.
	5	+ */
	6	+
	7	+#define INCLUDE_VERMAGIC
	8	+
	9	+#include <linux/export.h>
	10	+#include <linux/extable.h>
	11	+#include <linux/moduleloader.h>
	12	+#include <linux/module_signature.h>
	13	+#include <linux/trace_events.h>
	14	+#include <linux/init.h>
	15	+#include <linux/kallsyms.h>
	16	+#include <linux/buildid.h>
	17	+#include <linux/fs.h>
	18	+#include <linux/kernel.h>
	19	+#include <linux/kernel_read_file.h>
	20	+#include <linux/slab.h>
	21	+#include <linux/vmalloc.h>
	22	+#include <linux/elf.h>
	23	+#include <linux/seq_file.h>
	24	+#include <linux/syscalls.h>
	25	+#include <linux/fcntl.h>
	26	+#include <linux/rcupdate.h>
	27	+#include <linux/capability.h>
	28	+#include <linux/cpu.h>
	29	+#include <linux/moduleparam.h>
	30	+#include <linux/errno.h>
	31	+#include <linux/err.h>
	32	+#include <linux/vermagic.h>
	33	+#include <linux/notifier.h>
	34	+#include <linux/sched.h>
	35	+#include <linux/device.h>
	36	+#include <linux/string.h>
	37	+#include <linux/mutex.h>
	38	+#include <linux/rculist.h>
	39	+#include <linux/uaccess.h>
	40	+#include <asm/cacheflush.h>
	41	+#include <linux/set_memory.h>
	42	+#include <asm/mmu_context.h>
	43	+#include <linux/license.h>
	44	+#include <asm/sections.h>
	45	+#include <linux/tracepoint.h>
	46	+#include <linux/ftrace.h>
	47	+#include <linux/livepatch.h>
	48	+#include <linux/async.h>
	49	+#include <linux/percpu.h>
	50	+#include <linux/kmemleak.h>
	51	+#include <linux/jump_label.h>
	52	+#include <linux/pfn.h>
	53	+#include <linux/bsearch.h>
	54	+#include <linux/dynamic_debug.h>
	55	+#include <linux/audit.h>
	56	+#include <uapi/linux/module.h>
	57	+#include "internal.h"
	58	+
	59	+#define CREATE_TRACE_POINTS
	60	+#include <trace/events/module.h>
	61	+
	62	+/*
	63	+ * Mutex protects:
	64	+ * 1) List of modules (also safely readable with preempt_disable),
	65	+ * 2) module_use links,
	66	+ * 3) mod_tree.addr_min/mod_tree.addr_max.
	67	+ * (delete and add uses RCU list operations).
	68	+ */
	69	+DEFINE_MUTEX(module_mutex);
	70	+LIST_HEAD(modules);
	71	+
	72	+/* Work queue for freeing init sections in success case */
	73	+static void do_free_init(struct work_struct *w);
	74	+static DECLARE_WORK(init_free_wq, do_free_init);
	75	+static LLIST_HEAD(init_free_list);
	76	+
	77	+struct mod_tree_root mod_tree __cacheline_aligned = {
	78	+ .addr_min = -1UL,
	79	+};
	80	+
	81	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	82	+struct mod_tree_root mod_data_tree __cacheline_aligned = {
	83	+ .addr_min = -1UL,
	84	+};
	85	+#endif
	86	+
	87	+#define module_addr_min mod_tree.addr_min
	88	+#define module_addr_max mod_tree.addr_max
	89	+
	90	+struct symsearch {
	91	+ const struct kernel_symbol start, stop;
	92	+ const s32 *crcs;
	93	+ enum mod_license license;
	94	+};
	95	+
	96	+/*
	97	+ * Bounds of module text, for speeding up __module_address.
	98	+ * Protected by module_mutex.
	99	+ */
	100	+static void __mod_update_bounds(void base, unsigned int size, struct mod_tree_root tree)
	101	+{
	102	+ unsigned long min = (unsigned long)base;
	103	+ unsigned long max = min + size;
	104	+
	105	+ if (min < tree->addr_min)
	106	+ tree->addr_min = min;
	107	+ if (max > tree->addr_max)
	108	+ tree->addr_max = max;
	109	+}
	110	+
	111	+static void mod_update_bounds(struct module *mod)
	112	+{
	113	+ __mod_update_bounds(mod->core_layout.base, mod->core_layout.size, &mod_tree);
	114	+ if (mod->init_layout.size)
	115	+ __mod_update_bounds(mod->init_layout.base, mod->init_layout.size, &mod_tree);
	116	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	117	+ __mod_update_bounds(mod->data_layout.base, mod->data_layout.size, &mod_data_tree);
	118	+#endif
	119	+}
	120	+
	121	+/* Block module loading/unloading? */
	122	+int modules_disabled = 0;
	123	+core_param(nomodule, modules_disabled, bint, 0);
	124	+
	125	+/* Waiting for a module to finish initializing? */
	126	+static DECLARE_WAIT_QUEUE_HEAD(module_wq);
	127	+
	128	+static BLOCKING_NOTIFIER_HEAD(module_notify_list);
	129	+
	130	+int register_module_notifier(struct notifier_block *nb)
	131	+{
	132	+ return blocking_notifier_chain_register(&module_notify_list, nb);
	133	+}
	134	+EXPORT_SYMBOL(register_module_notifier);
	135	+
	136	+int unregister_module_notifier(struct notifier_block *nb)
	137	+{
	138	+ return blocking_notifier_chain_unregister(&module_notify_list, nb);
	139	+}
	140	+EXPORT_SYMBOL(unregister_module_notifier);
	141	+
	142	+/*
	143	+ * We require a truly strong try_module_get(): 0 means success.
	144	+ * Otherwise an error is returned due to ongoing or failed
	145	+ * initialization etc.
	146	+ */
	147	+static inline int strong_try_module_get(struct module *mod)
	148	+{
	149	+ BUG_ON(mod && mod->state == MODULE_STATE_UNFORMED);
	150	+ if (mod && mod->state == MODULE_STATE_COMING)
	151	+ return -EBUSY;
	152	+ if (try_module_get(mod))
	153	+ return 0;
	154	+ else
	155	+ return -ENOENT;
	156	+}
	157	+
	158	+static inline void add_taint_module(struct module *mod, unsigned flag,
	159	+ enum lockdep_ok lockdep_ok)
	160	+{
	161	+ add_taint(flag, lockdep_ok);
	162	+ set_bit(flag, &mod->taints);
	163	+}
	164	+
	165	+/*
	166	+ * A thread that wants to hold a reference to a module only while it
	167	+ * is running can call this to safely exit.
	168	+ */
	169	+void __noreturn __module_put_and_kthread_exit(struct module *mod, long code)
	170	+{
	171	+ module_put(mod);
	172	+ kthread_exit(code);
	173	+}
	174	+EXPORT_SYMBOL(__module_put_and_kthread_exit);
	175	+
	176	+/* Find a module section: 0 means not found. */
	177	+static unsigned int find_sec(const struct load_info info, const char name)
	178	+{
	179	+ unsigned int i;
	180	+
	181	+ for (i = 1; i < info->hdr->e_shnum; i++) {
	182	+ Elf_Shdr *shdr = &info->sechdrs[i];
	183	+ /* Alloc bit cleared means "ignore it." */
	184	+ if ((shdr->sh_flags & SHF_ALLOC)
	185	+ && strcmp(info->secstrings + shdr->sh_name, name) == 0)
	186	+ return i;
	187	+ }
	188	+ return 0;
	189	+}
	190	+
	191	+/* Find a module section, or NULL. */
	192	+static void section_addr(const struct load_info info, const char *name)
	193	+{
	194	+ /* Section 0 has sh_addr 0. */
	195	+ return (void *)info->sechdrs[find_sec(info, name)].sh_addr;
	196	+}
	197	+
	198	+/* Find a module section, or NULL. Fill in number of "objects" in section. */
	199	+static void section_objs(const struct load_info info,
	200	+ const char *name,
	201	+ size_t object_size,
	202	+ unsigned int *num)
	203	+{
	204	+ unsigned int sec = find_sec(info, name);
	205	+
	206	+ /* Section 0 has sh_addr 0 and sh_size 0. */
	207	+ *num = info->sechdrs[sec].sh_size / object_size;
	208	+ return (void *)info->sechdrs[sec].sh_addr;
	209	+}
	210	+
	211	+/* Find a module section: 0 means not found. Ignores SHF_ALLOC flag. */
	212	+static unsigned int find_any_sec(const struct load_info info, const char name)
	213	+{
	214	+ unsigned int i;
	215	+
	216	+ for (i = 1; i < info->hdr->e_shnum; i++) {
	217	+ Elf_Shdr *shdr = &info->sechdrs[i];
	218	+ if (strcmp(info->secstrings + shdr->sh_name, name) == 0)
	219	+ return i;
	220	+ }
	221	+ return 0;
	222	+}
	223	+
	224	+/*
	225	+ * Find a module section, or NULL. Fill in number of "objects" in section.
	226	+ * Ignores SHF_ALLOC flag.
	227	+ */
	228	+static __maybe_unused void any_section_objs(const struct load_info info,
	229	+ const char *name,
	230	+ size_t object_size,
	231	+ unsigned int *num)
	232	+{
	233	+ unsigned int sec = find_any_sec(info, name);
	234	+
	235	+ /* Section 0 has sh_addr 0 and sh_size 0. */
	236	+ *num = info->sechdrs[sec].sh_size / object_size;
	237	+ return (void *)info->sechdrs[sec].sh_addr;
	238	+}
	239	+
	240	+#ifndef CONFIG_MODVERSIONS
	241	+#define symversion(base, idx) NULL
	242	+#else
	243	+#define symversion(base, idx) ((base != NULL) ? ((base) + (idx)) : NULL)
	244	+#endif
	245	+
	246	+static const char kernel_symbol_name(const struct kernel_symbol sym)
	247	+{
	248	+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
	249	+ return offset_to_ptr(&sym->name_offset);
	250	+#else
	251	+ return sym->name;
	252	+#endif
	253	+}
	254	+
	255	+static const char kernel_symbol_namespace(const struct kernel_symbol sym)
	256	+{
	257	+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
	258	+ if (!sym->namespace_offset)
	259	+ return NULL;
	260	+ return offset_to_ptr(&sym->namespace_offset);
	261	+#else
	262	+ return sym->namespace;
	263	+#endif
	264	+}
	265	+
	266	+int cmp_name(const void name, const void sym)
	267	+{
	268	+ return strcmp(name, kernel_symbol_name(sym));
	269	+}
	270	+
	271	+static bool find_exported_symbol_in_section(const struct symsearch *syms,
	272	+ struct module *owner,
	273	+ struct find_symbol_arg *fsa)
	274	+{
	275	+ struct kernel_symbol *sym;
	276	+
	277	+ if (!fsa->gplok && syms->license == GPL_ONLY)
	278	+ return false;
	279	+
	280	+ sym = bsearch(fsa->name, syms->start, syms->stop - syms->start,
	281	+ sizeof(struct kernel_symbol), cmp_name);
	282	+ if (!sym)
	283	+ return false;
	284	+
	285	+ fsa->owner = owner;
	286	+ fsa->crc = symversion(syms->crcs, sym - syms->start);
	287	+ fsa->sym = sym;
	288	+ fsa->license = syms->license;
	289	+
	290	+ return true;
	291	+}
	292	+
	293	+/*
	294	+ * Find an exported symbol and return it, along with, (optional) crc and
	295	+ * (optional) module which owns it. Needs preempt disabled or module_mutex.
	296	+ */
	297	+bool find_symbol(struct find_symbol_arg *fsa)
	298	+{
	299	+ static const struct symsearch arr[] = {
	300	+ { __start___ksymtab, __stop___ksymtab, __start___kcrctab,
	301	+ NOT_GPL_ONLY },
	302	+ { __start___ksymtab_gpl, __stop___ksymtab_gpl,
	303	+ __start___kcrctab_gpl,
	304	+ GPL_ONLY },
	305	+ };
	306	+ struct module *mod;
	307	+ unsigned int i;
	308	+
	309	+ module_assert_mutex_or_preempt();
	310	+
	311	+ for (i = 0; i < ARRAY_SIZE(arr); i++)
	312	+ if (find_exported_symbol_in_section(&arr[i], NULL, fsa))
	313	+ return true;
	314	+
	315	+ list_for_each_entry_rcu(mod, &modules, list,
	316	+ lockdep_is_held(&module_mutex)) {
	317	+ struct symsearch arr[] = {
	318	+ { mod->syms, mod->syms + mod->num_syms, mod->crcs,
	319	+ NOT_GPL_ONLY },
	320	+ { mod->gpl_syms, mod->gpl_syms + mod->num_gpl_syms,
	321	+ mod->gpl_crcs,
	322	+ GPL_ONLY },
	323	+ };
	324	+
	325	+ if (mod->state == MODULE_STATE_UNFORMED)
	326	+ continue;
	327	+
	328	+ for (i = 0; i < ARRAY_SIZE(arr); i++)
	329	+ if (find_exported_symbol_in_section(&arr[i], mod, fsa))
	330	+ return true;
	331	+ }
	332	+
	333	+ pr_debug("Failed to find symbol %s\n", fsa->name);
	334	+ return false;
	335	+}
	336	+
	337	+/*
	338	+ * Search for module by name: must hold module_mutex (or preempt disabled
	339	+ * for read-only access).
	340	+ */
	341	+struct module find_module_all(const char name, size_t len,
	342	+ bool even_unformed)
	343	+{
	344	+ struct module *mod;
	345	+
	346	+ module_assert_mutex_or_preempt();
	347	+
	348	+ list_for_each_entry_rcu(mod, &modules, list,
	349	+ lockdep_is_held(&module_mutex)) {
	350	+ if (!even_unformed && mod->state == MODULE_STATE_UNFORMED)
	351	+ continue;
	352	+ if (strlen(mod->name) == len && !memcmp(mod->name, name, len))
	353	+ return mod;
	354	+ }
	355	+ return NULL;
	356	+}
	357	+
	358	+struct module find_module(const char name)
	359	+{
	360	+ return find_module_all(name, strlen(name), false);
	361	+}
	362	+
	363	+#ifdef CONFIG_SMP
	364	+
	365	+static inline void __percpu mod_percpu(struct module mod)
	366	+{
	367	+ return mod->percpu;
	368	+}
	369	+
	370	+static int percpu_modalloc(struct module mod, struct load_info info)
	371	+{
	372	+ Elf_Shdr *pcpusec = &info->sechdrs[info->index.pcpu];
	373	+ unsigned long align = pcpusec->sh_addralign;
	374	+
	375	+ if (!pcpusec->sh_size)
	376	+ return 0;
	377	+
	378	+ if (align > PAGE_SIZE) {
	379	+ pr_warn("%s: per-cpu alignment %li > %li\n",
	380	+ mod->name, align, PAGE_SIZE);
	381	+ align = PAGE_SIZE;
	382	+ }
	383	+
	384	+ mod->percpu = __alloc_reserved_percpu(pcpusec->sh_size, align);
	385	+ if (!mod->percpu) {
	386	+ pr_warn("%s: Could not allocate %lu bytes percpu data\n",
	387	+ mod->name, (unsigned long)pcpusec->sh_size);
	388	+ return -ENOMEM;
	389	+ }
	390	+ mod->percpu_size = pcpusec->sh_size;
	391	+ return 0;
	392	+}
	393	+
	394	+static void percpu_modfree(struct module *mod)
	395	+{
	396	+ free_percpu(mod->percpu);
	397	+}
	398	+
	399	+static unsigned int find_pcpusec(struct load_info *info)
	400	+{
	401	+ return find_sec(info, ".data..percpu");
	402	+}
	403	+
	404	+static void percpu_modcopy(struct module *mod,
	405	+ const void *from, unsigned long size)
	406	+{
	407	+ int cpu;
	408	+
	409	+ for_each_possible_cpu(cpu)
	410	+ memcpy(per_cpu_ptr(mod->percpu, cpu), from, size);
	411	+}
	412	+
	413	+bool __is_module_percpu_address(unsigned long addr, unsigned long *can_addr)
	414	+{
	415	+ struct module *mod;
	416	+ unsigned int cpu;
	417	+
	418	+ preempt_disable();
	419	+
	420	+ list_for_each_entry_rcu(mod, &modules, list) {
	421	+ if (mod->state == MODULE_STATE_UNFORMED)
	422	+ continue;
	423	+ if (!mod->percpu_size)
	424	+ continue;
	425	+ for_each_possible_cpu(cpu) {
	426	+ void *start = per_cpu_ptr(mod->percpu, cpu);
	427	+ void va = (void )addr;
	428	+
	429	+ if (va >= start && va < start + mod->percpu_size) {
	430	+ if (can_addr) {
	431	+ *can_addr = (unsigned long) (va - start);
	432	+ *can_addr += (unsigned long)
	433	+ per_cpu_ptr(mod->percpu,
	434	+ get_boot_cpu_id());
	435	+ }
	436	+ preempt_enable();
	437	+ return true;
	438	+ }
	439	+ }
	440	+ }
	441	+
	442	+ preempt_enable();
	443	+ return false;
	444	+}
	445	+
	446	+/**
	447	+ * is_module_percpu_address() - test whether address is from module static percpu
	448	+ * @addr: address to test
	449	+ *
	450	+ * Test whether @addr belongs to module static percpu area.
	451	+ *
	452	+ * Return: %true if @addr is from module static percpu area
	453	+ */
	454	+bool is_module_percpu_address(unsigned long addr)
	455	+{
	456	+ return __is_module_percpu_address(addr, NULL);
	457	+}
	458	+
	459	+#else /* ... !CONFIG_SMP */
	460	+
	461	+static inline void __percpu mod_percpu(struct module mod)
	462	+{
	463	+ return NULL;
	464	+}
	465	+static int percpu_modalloc(struct module mod, struct load_info info)
	466	+{
	467	+ /* UP modules shouldn't have this section: ENOMEM isn't quite right */
	468	+ if (info->sechdrs[info->index.pcpu].sh_size != 0)
	469	+ return -ENOMEM;
	470	+ return 0;
	471	+}
	472	+static inline void percpu_modfree(struct module *mod)
	473	+{
	474	+}
	475	+static unsigned int find_pcpusec(struct load_info *info)
	476	+{
	477	+ return 0;
	478	+}
	479	+static inline void percpu_modcopy(struct module *mod,
	480	+ const void *from, unsigned long size)
	481	+{
	482	+ /* pcpusec should be 0, and size of that section should be 0. */
	483	+ BUG_ON(size != 0);
	484	+}
	485	+bool is_module_percpu_address(unsigned long addr)
	486	+{
	487	+ return false;
	488	+}
	489	+
	490	+bool __is_module_percpu_address(unsigned long addr, unsigned long *can_addr)
	491	+{
	492	+ return false;
	493	+}
	494	+
	495	+#endif /* CONFIG_SMP */
	496	+
	497	+#define MODINFO_ATTR(field) \
	498	+static void setup_modinfo_##field(struct module mod, const char s) \
	499	+{ \
	500	+ mod->field = kstrdup(s, GFP_KERNEL); \
	501	+} \
	502	+static ssize_t show_modinfo_##field(struct module_attribute *mattr, \
	503	+ struct module_kobject mk, char buffer) \
	504	+{ \
	505	+ return scnprintf(buffer, PAGE_SIZE, "%s\n", mk->mod->field); \
	506	+} \
	507	+static int modinfo_##field##_exists(struct module *mod) \
	508	+{ \
	509	+ return mod->field != NULL; \
	510	+} \
	511	+static void free_modinfo_##field(struct module *mod) \
	512	+{ \
	513	+ kfree(mod->field); \
	514	+ mod->field = NULL; \
	515	+} \
	516	+static struct module_attribute modinfo_##field = { \
	517	+ .attr = { .name = __stringify(field), .mode = 0444 }, \
	518	+ .show = show_modinfo_##field, \
	519	+ .setup = setup_modinfo_##field, \
	520	+ .test = modinfo_##field##_exists, \
	521	+ .free = free_modinfo_##field, \
	522	+};
	523	+
	524	+MODINFO_ATTR(version);
	525	+MODINFO_ATTR(srcversion);
	526	+
	527	+static char last_unloaded_module[MODULE_NAME_LEN+1];
	528	+
	529	+#ifdef CONFIG_MODULE_UNLOAD
	530	+
	531	+EXPORT_TRACEPOINT_SYMBOL(module_get);
	532	+
	533	+/* MODULE_REF_BASE is the base reference count by kmodule loader. */
	534	+#define MODULE_REF_BASE 1
	535	+
	536	+/* Init the unload section of the module. */
	537	+static int module_unload_init(struct module *mod)
	538	+{
	539	+ /*
	540	+ * Initialize reference counter to MODULE_REF_BASE.
	541	+ * refcnt == 0 means module is going.
	542	+ */
	543	+ atomic_set(&mod->refcnt, MODULE_REF_BASE);
	544	+
	545	+ INIT_LIST_HEAD(&mod->source_list);
	546	+ INIT_LIST_HEAD(&mod->target_list);
	547	+
	548	+ /* Hold reference count during initialization. */
	549	+ atomic_inc(&mod->refcnt);
	550	+
	551	+ return 0;
	552	+}
	553	+
	554	+/* Does a already use b? */
	555	+static int already_uses(struct module a, struct module b)
	556	+{
	557	+ struct module_use *use;
	558	+
	559	+ list_for_each_entry(use, &b->source_list, source_list) {
	560	+ if (use->source == a) {
	561	+ pr_debug("%s uses %s!\n", a->name, b->name);
	562	+ return 1;
	563	+ }
	564	+ }
	565	+ pr_debug("%s does not use %s!\n", a->name, b->name);
	566	+ return 0;
	567	+}
	568	+
	569	+/*
	570	+ * Module a uses b
	571	+ * - we add 'a' as a "source", 'b' as a "target" of module use
	572	+ * - the module_use is added to the list of 'b' sources (so
	573	+ * 'b' can walk the list to see who sourced them), and of 'a'
	574	+ * targets (so 'a' can see what modules it targets).
	575	+ */
	576	+static int add_module_usage(struct module a, struct module b)
	577	+{
	578	+ struct module_use *use;
	579	+
	580	+ pr_debug("Allocating new usage for %s.\n", a->name);
	581	+ use = kmalloc(sizeof(*use), GFP_ATOMIC);
	582	+ if (!use)
	583	+ return -ENOMEM;
	584	+
	585	+ use->source = a;
	586	+ use->target = b;
	587	+ list_add(&use->source_list, &b->source_list);
	588	+ list_add(&use->target_list, &a->target_list);
	589	+ return 0;
	590	+}
	591	+
	592	+/* Module a uses b: caller needs module_mutex() */
	593	+static int ref_module(struct module a, struct module b)
	594	+{
	595	+ int err;
	596	+
	597	+ if (b == NULL \|\| already_uses(a, b))
	598	+ return 0;
	599	+
	600	+ /* If module isn't available, we fail. */
	601	+ err = strong_try_module_get(b);
	602	+ if (err)
	603	+ return err;
	604	+
	605	+ err = add_module_usage(a, b);
	606	+ if (err) {
	607	+ module_put(b);
	608	+ return err;
	609	+ }
	610	+ return 0;
	611	+}
	612	+
	613	+/* Clear the unload stuff of the module. */
	614	+static void module_unload_free(struct module *mod)
	615	+{
	616	+ struct module_use use, tmp;
	617	+
	618	+ mutex_lock(&module_mutex);
	619	+ list_for_each_entry_safe(use, tmp, &mod->target_list, target_list) {
	620	+ struct module *i = use->target;
	621	+ pr_debug("%s unusing %s\n", mod->name, i->name);
	622	+ module_put(i);
	623	+ list_del(&use->source_list);
	624	+ list_del(&use->target_list);
	625	+ kfree(use);
	626	+ }
	627	+ mutex_unlock(&module_mutex);
	628	+}
	629	+
	630	+#ifdef CONFIG_MODULE_FORCE_UNLOAD
	631	+static inline int try_force_unload(unsigned int flags)
	632	+{
	633	+ int ret = (flags & O_TRUNC);
	634	+ if (ret)
	635	+ add_taint(TAINT_FORCED_RMMOD, LOCKDEP_NOW_UNRELIABLE);
	636	+ return ret;
	637	+}
	638	+#else
	639	+static inline int try_force_unload(unsigned int flags)
	640	+{
	641	+ return 0;
	642	+}
	643	+#endif /* CONFIG_MODULE_FORCE_UNLOAD */
	644	+
	645	+/* Try to release refcount of module, 0 means success. */
	646	+static int try_release_module_ref(struct module *mod)
	647	+{
	648	+ int ret;
	649	+
	650	+ /* Try to decrement refcnt which we set at loading */
	651	+ ret = atomic_sub_return(MODULE_REF_BASE, &mod->refcnt);
	652	+ BUG_ON(ret < 0);
	653	+ if (ret)
	654	+ /* Someone can put this right now, recover with checking */
	655	+ ret = atomic_add_unless(&mod->refcnt, MODULE_REF_BASE, 0);
	656	+
	657	+ return ret;
	658	+}
	659	+
	660	+static int try_stop_module(struct module mod, int flags, int forced)
	661	+{
	662	+ /* If it's not unused, quit unless we're forcing. */
	663	+ if (try_release_module_ref(mod) != 0) {
	664	+ *forced = try_force_unload(flags);
	665	+ if (!(*forced))
	666	+ return -EWOULDBLOCK;
	667	+ }
	668	+
	669	+ /* Mark it as dying. */
	670	+ mod->state = MODULE_STATE_GOING;
	671	+
	672	+ return 0;
	673	+}
	674	+
	675	+/**
	676	+ * module_refcount() - return the refcount or -1 if unloading
	677	+ * @mod: the module we're checking
	678	+ *
	679	+ * Return:
	680	+ * -1 if the module is in the process of unloading
	681	+ * otherwise the number of references in the kernel to the module
	682	+ */
	683	+int module_refcount(struct module *mod)
	684	+{
	685	+ return atomic_read(&mod->refcnt) - MODULE_REF_BASE;
	686	+}
	687	+EXPORT_SYMBOL(module_refcount);
	688	+
	689	+/* This exists whether we can unload or not */
	690	+static void free_module(struct module *mod);
	691	+
	692	+SYSCALL_DEFINE2(delete_module, const char __user *, name_user,
	693	+ unsigned int, flags)
	694	+{
	695	+ struct module *mod;
	696	+ char name[MODULE_NAME_LEN];
	697	+ int ret, forced = 0;
	698	+
	699	+ if (!capable(CAP_SYS_MODULE) \|\| modules_disabled)
	700	+ return -EPERM;
	701	+
	702	+ if (strncpy_from_user(name, name_user, MODULE_NAME_LEN-1) < 0)
	703	+ return -EFAULT;
	704	+ name[MODULE_NAME_LEN-1] = '\0';
	705	+
	706	+ audit_log_kern_module(name);
	707	+
	708	+ if (mutex_lock_interruptible(&module_mutex) != 0)
	709	+ return -EINTR;
	710	+
	711	+ mod = find_module(name);
	712	+ if (!mod) {
	713	+ ret = -ENOENT;
	714	+ goto out;
	715	+ }
	716	+
	717	+ if (!list_empty(&mod->source_list)) {
	718	+ /* Other modules depend on us: get rid of them first. */
	719	+ ret = -EWOULDBLOCK;
	720	+ goto out;
	721	+ }
	722	+
	723	+ /* Doing init or already dying? */
	724	+ if (mod->state != MODULE_STATE_LIVE) {
	725	+ /* FIXME: if (force), slam module count damn the torpedoes */
	726	+ pr_debug("%s already dying\n", mod->name);
	727	+ ret = -EBUSY;
	728	+ goto out;
	729	+ }
	730	+
	731	+ /* If it has an init func, it must have an exit func to unload */
	732	+ if (mod->init && !mod->exit) {
	733	+ forced = try_force_unload(flags);
	734	+ if (!forced) {
	735	+ /* This module can't be removed */
	736	+ ret = -EBUSY;
	737	+ goto out;
	738	+ }
	739	+ }
	740	+
	741	+ ret = try_stop_module(mod, flags, &forced);
	742	+ if (ret != 0)
	743	+ goto out;
	744	+
	745	+ mutex_unlock(&module_mutex);
	746	+ /* Final destruction now no one is using it. */
	747	+ if (mod->exit != NULL)
	748	+ mod->exit();
	749	+ blocking_notifier_call_chain(&module_notify_list,
	750	+ MODULE_STATE_GOING, mod);
	751	+ klp_module_going(mod);
	752	+ ftrace_release_mod(mod);
	753	+
	754	+ async_synchronize_full();
	755	+
	756	+ /* Store the name of the last unloaded module for diagnostic purposes */
	757	+ strlcpy(last_unloaded_module, mod->name, sizeof(last_unloaded_module));
	758	+
	759	+ free_module(mod);
	760	+ /* someone could wait for the module in add_unformed_module() */
	761	+ wake_up_all(&module_wq);
	762	+ return 0;
	763	+out:
	764	+ mutex_unlock(&module_mutex);
	765	+ return ret;
	766	+}
	767	+
	768	+void __symbol_put(const char *symbol)
	769	+{
	770	+ struct find_symbol_arg fsa = {
	771	+ .name = symbol,
	772	+ .gplok = true,
	773	+ };
	774	+
	775	+ preempt_disable();
	776	+ BUG_ON(!find_symbol(&fsa));
	777	+ module_put(fsa.owner);
	778	+ preempt_enable();
	779	+}
	780	+EXPORT_SYMBOL(__symbol_put);
	781	+
	782	+/* Note this assumes addr is a function, which it currently always is. */
	783	+void symbol_put_addr(void *addr)
	784	+{
	785	+ struct module *modaddr;
	786	+ unsigned long a = (unsigned long)dereference_function_descriptor(addr);
	787	+
	788	+ if (core_kernel_text(a))
	789	+ return;
	790	+
	791	+ /*
	792	+ * Even though we hold a reference on the module; we still need to
	793	+ * disable preemption in order to safely traverse the data structure.
	794	+ */
	795	+ preempt_disable();
	796	+ modaddr = __module_text_address(a);
	797	+ BUG_ON(!modaddr);
	798	+ module_put(modaddr);
	799	+ preempt_enable();
	800	+}
	801	+EXPORT_SYMBOL_GPL(symbol_put_addr);
	802	+
	803	+static ssize_t show_refcnt(struct module_attribute *mattr,
	804	+ struct module_kobject mk, char buffer)
	805	+{
	806	+ return sprintf(buffer, "%i\n", module_refcount(mk->mod));
	807	+}
	808	+
	809	+static struct module_attribute modinfo_refcnt =
	810	+ __ATTR(refcnt, 0444, show_refcnt, NULL);
	811	+
	812	+void __module_get(struct module *module)
	813	+{
	814	+ if (module) {
	815	+ preempt_disable();
	816	+ atomic_inc(&module->refcnt);
	817	+ trace_module_get(module, _RET_IP_);
	818	+ preempt_enable();
	819	+ }
	820	+}
	821	+EXPORT_SYMBOL(__module_get);
	822	+
	823	+bool try_module_get(struct module *module)
	824	+{
	825	+ bool ret = true;
	826	+
	827	+ if (module) {
	828	+ preempt_disable();
	829	+ /* Note: here, we can fail to get a reference */
	830	+ if (likely(module_is_live(module) &&
	831	+ atomic_inc_not_zero(&module->refcnt) != 0))
	832	+ trace_module_get(module, _RET_IP_);
	833	+ else
	834	+ ret = false;
	835	+
	836	+ preempt_enable();
	837	+ }
	838	+ return ret;
	839	+}
	840	+EXPORT_SYMBOL(try_module_get);
	841	+
	842	+void module_put(struct module *module)
	843	+{
	844	+ int ret;
	845	+
	846	+ if (module) {
	847	+ preempt_disable();
	848	+ ret = atomic_dec_if_positive(&module->refcnt);
	849	+ WARN_ON(ret < 0); /* Failed to put refcount */
	850	+ trace_module_put(module, _RET_IP_);
	851	+ preempt_enable();
	852	+ }
	853	+}
	854	+EXPORT_SYMBOL(module_put);
	855	+
	856	+#else /* !CONFIG_MODULE_UNLOAD */
	857	+static inline void module_unload_free(struct module *mod)
	858	+{
	859	+}
	860	+
	861	+static int ref_module(struct module a, struct module b)
	862	+{
	863	+ return strong_try_module_get(b);
	864	+}
	865	+
	866	+static inline int module_unload_init(struct module *mod)
	867	+{
	868	+ return 0;
	869	+}
	870	+#endif /* CONFIG_MODULE_UNLOAD */
	871	+
	872	+size_t module_flags_taint(unsigned long taints, char *buf)
	873	+{
	874	+ size_t l = 0;
	875	+ int i;
	876	+
	877	+ for (i = 0; i < TAINT_FLAGS_COUNT; i++) {
	878	+ if (taint_flags[i].module && test_bit(i, &taints))
	879	+ buf[l++] = taint_flags[i].c_true;
	880	+ }
	881	+
	882	+ return l;
	883	+}
	884	+
	885	+static ssize_t show_initstate(struct module_attribute *mattr,
	886	+ struct module_kobject mk, char buffer)
	887	+{
	888	+ const char *state = "unknown";
	889	+
	890	+ switch (mk->mod->state) {
	891	+ case MODULE_STATE_LIVE:
	892	+ state = "live";
	893	+ break;
	894	+ case MODULE_STATE_COMING:
	895	+ state = "coming";
	896	+ break;
	897	+ case MODULE_STATE_GOING:
	898	+ state = "going";
	899	+ break;
	900	+ default:
	901	+ BUG();
	902	+ }
	903	+ return sprintf(buffer, "%s\n", state);
	904	+}
	905	+
	906	+static struct module_attribute modinfo_initstate =
	907	+ __ATTR(initstate, 0444, show_initstate, NULL);
	908	+
	909	+static ssize_t store_uevent(struct module_attribute *mattr,
	910	+ struct module_kobject *mk,
	911	+ const char *buffer, size_t count)
	912	+{
	913	+ int rc;
	914	+
	915	+ rc = kobject_synth_uevent(&mk->kobj, buffer, count);
	916	+ return rc ? rc : count;
	917	+}
	918	+
	919	+struct module_attribute module_uevent =
	920	+ __ATTR(uevent, 0200, NULL, store_uevent);
	921	+
	922	+static ssize_t show_coresize(struct module_attribute *mattr,
	923	+ struct module_kobject mk, char buffer)
	924	+{
	925	+ return sprintf(buffer, "%u\n", mk->mod->core_layout.size);
	926	+}
	927	+
	928	+static struct module_attribute modinfo_coresize =
	929	+ __ATTR(coresize, 0444, show_coresize, NULL);
	930	+
	931	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	932	+static ssize_t show_datasize(struct module_attribute *mattr,
	933	+ struct module_kobject mk, char buffer)
	934	+{
	935	+ return sprintf(buffer, "%u\n", mk->mod->data_layout.size);
	936	+}
	937	+
	938	+static struct module_attribute modinfo_datasize =
	939	+ __ATTR(datasize, 0444, show_datasize, NULL);
	940	+#endif
	941	+
	942	+static ssize_t show_initsize(struct module_attribute *mattr,
	943	+ struct module_kobject mk, char buffer)
	944	+{
	945	+ return sprintf(buffer, "%u\n", mk->mod->init_layout.size);
	946	+}
	947	+
	948	+static struct module_attribute modinfo_initsize =
	949	+ __ATTR(initsize, 0444, show_initsize, NULL);
	950	+
	951	+static ssize_t show_taint(struct module_attribute *mattr,
	952	+ struct module_kobject mk, char buffer)
	953	+{
	954	+ size_t l;
	955	+
	956	+ l = module_flags_taint(mk->mod->taints, buffer);
	957	+ buffer[l++] = '\n';
	958	+ return l;
	959	+}
	960	+
	961	+static struct module_attribute modinfo_taint =
	962	+ __ATTR(taint, 0444, show_taint, NULL);
	963	+
	964	+struct module_attribute *modinfo_attrs[] = {
	965	+ &module_uevent,
	966	+ &modinfo_version,
	967	+ &modinfo_srcversion,
	968	+ &modinfo_initstate,
	969	+ &modinfo_coresize,
	970	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	971	+ &modinfo_datasize,
	972	+#endif
	973	+ &modinfo_initsize,
	974	+ &modinfo_taint,
	975	+#ifdef CONFIG_MODULE_UNLOAD
	976	+ &modinfo_refcnt,
	977	+#endif
	978	+ NULL,
	979	+};
	980	+
	981	+size_t modinfo_attrs_count = ARRAY_SIZE(modinfo_attrs);
	982	+
	983	+static const char vermagic[] = VERMAGIC_STRING;
	984	+
	985	+int try_to_force_load(struct module mod, const char reason)
	986	+{
	987	+#ifdef CONFIG_MODULE_FORCE_LOAD
	988	+ if (!test_taint(TAINT_FORCED_MODULE))
	989	+ pr_warn("%s: %s: kernel tainted.\n", mod->name, reason);
	990	+ add_taint_module(mod, TAINT_FORCED_MODULE, LOCKDEP_NOW_UNRELIABLE);
	991	+ return 0;
	992	+#else
	993	+ return -ENOEXEC;
	994	+#endif
	995	+}
	996	+
	997	+static char get_modinfo(const struct load_info info, const char *tag);
	998	+static char get_next_modinfo(const struct load_info info, const char *tag,
	999	+ char *prev);
	1000	+
	1001	+static int verify_namespace_is_imported(const struct load_info *info,
	1002	+ const struct kernel_symbol *sym,
	1003	+ struct module *mod)
	1004	+{
	1005	+ const char *namespace;
	1006	+ char *imported_namespace;
	1007	+
	1008	+ namespace = kernel_symbol_namespace(sym);
	1009	+ if (namespace && namespace[0]) {
	1010	+ imported_namespace = get_modinfo(info, "import_ns");
	1011	+ while (imported_namespace) {
	1012	+ if (strcmp(namespace, imported_namespace) == 0)
	1013	+ return 0;
	1014	+ imported_namespace = get_next_modinfo(
	1015	+ info, "import_ns", imported_namespace);
	1016	+ }
	1017	+#ifdef CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
	1018	+ pr_warn(
	1019	+#else
	1020	+ pr_err(
	1021	+#endif
	1022	+ "%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
	1023	+ mod->name, kernel_symbol_name(sym), namespace);
	1024	+#ifndef CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
	1025	+ return -EINVAL;
	1026	+#endif
	1027	+ }
	1028	+ return 0;
	1029	+}
	1030	+
	1031	+static bool inherit_taint(struct module mod, struct module owner, const char *name)
	1032	+{
	1033	+ if (!owner \|\| !test_bit(TAINT_PROPRIETARY_MODULE, &owner->taints))
	1034	+ return true;
	1035	+
	1036	+ if (mod->using_gplonly_symbols) {
	1037	+ pr_err("%s: module using GPL-only symbols uses symbols %s from proprietary module %s.\n",
	1038	+ mod->name, name, owner->name);
	1039	+ return false;
	1040	+ }
	1041	+
	1042	+ if (!test_bit(TAINT_PROPRIETARY_MODULE, &mod->taints)) {
	1043	+ pr_warn("%s: module uses symbols %s from proprietary module %s, inheriting taint.\n",
	1044	+ mod->name, name, owner->name);
	1045	+ set_bit(TAINT_PROPRIETARY_MODULE, &mod->taints);
	1046	+ }
	1047	+ return true;
	1048	+}
	1049	+
	1050	+/* Resolve a symbol for this module. I.e. if we find one, record usage. */
	1051	+static const struct kernel_symbol resolve_symbol(struct module mod,
	1052	+ const struct load_info *info,
	1053	+ const char *name,
	1054	+ char ownername[])
	1055	+{
	1056	+ struct find_symbol_arg fsa = {
	1057	+ .name = name,
	1058	+ .gplok = !(mod->taints & (1 << TAINT_PROPRIETARY_MODULE)),
	1059	+ .warn = true,
	1060	+ };
	1061	+ int err;
	1062	+
	1063	+ /*
	1064	+ * The module_mutex should not be a heavily contended lock;
	1065	+ * if we get the occasional sleep here, we'll go an extra iteration
	1066	+ * in the wait_event_interruptible(), which is harmless.
	1067	+ */
	1068	+ sched_annotate_sleep();
	1069	+ mutex_lock(&module_mutex);
	1070	+ if (!find_symbol(&fsa))
	1071	+ goto unlock;
	1072	+
	1073	+ if (fsa.license == GPL_ONLY)
	1074	+ mod->using_gplonly_symbols = true;
	1075	+
	1076	+ if (!inherit_taint(mod, fsa.owner, name)) {
	1077	+ fsa.sym = NULL;
	1078	+ goto getname;
	1079	+ }
	1080	+
	1081	+ if (!check_version(info, name, mod, fsa.crc)) {
	1082	+ fsa.sym = ERR_PTR(-EINVAL);
	1083	+ goto getname;
	1084	+ }
	1085	+
	1086	+ err = verify_namespace_is_imported(info, fsa.sym, mod);
	1087	+ if (err) {
	1088	+ fsa.sym = ERR_PTR(err);
	1089	+ goto getname;
	1090	+ }
	1091	+
	1092	+ err = ref_module(mod, fsa.owner);
	1093	+ if (err) {
	1094	+ fsa.sym = ERR_PTR(err);
	1095	+ goto getname;
	1096	+ }
	1097	+
	1098	+getname:
	1099	+ /* We must make copy under the lock if we failed to get ref. */
	1100	+ strncpy(ownername, module_name(fsa.owner), MODULE_NAME_LEN);
	1101	+unlock:
	1102	+ mutex_unlock(&module_mutex);
	1103	+ return fsa.sym;
	1104	+}
	1105	+
	1106	+static const struct kernel_symbol *
	1107	+resolve_symbol_wait(struct module *mod,
	1108	+ const struct load_info *info,
	1109	+ const char *name)
	1110	+{
	1111	+ const struct kernel_symbol *ksym;
	1112	+ char owner[MODULE_NAME_LEN];
	1113	+
	1114	+ if (wait_event_interruptible_timeout(module_wq,
	1115	+ !IS_ERR(ksym = resolve_symbol(mod, info, name, owner))
	1116	+ \|\| PTR_ERR(ksym) != -EBUSY,
	1117	+ 30 * HZ) <= 0) {
	1118	+ pr_warn("%s: gave up waiting for init of module %s.\n",
	1119	+ mod->name, owner);
	1120	+ }
	1121	+ return ksym;
	1122	+}
	1123	+
	1124	+void __weak module_memfree(void *module_region)
	1125	+{
	1126	+ /*
	1127	+ * This memory may be RO, and freeing RO memory in an interrupt is not
	1128	+ * supported by vmalloc.
	1129	+ */
	1130	+ WARN_ON(in_interrupt());
	1131	+ vfree(module_region);
	1132	+}
	1133	+
	1134	+void __weak module_arch_cleanup(struct module *mod)
	1135	+{
	1136	+}
	1137	+
	1138	+void __weak module_arch_freeing_init(struct module *mod)
	1139	+{
	1140	+}
	1141	+
	1142	+static void cfi_cleanup(struct module *mod);
	1143	+
	1144	+/* Free a module, remove from lists, etc. */
	1145	+static void free_module(struct module *mod)
	1146	+{
	1147	+ trace_module_free(mod);
	1148	+
	1149	+ mod_sysfs_teardown(mod);
	1150	+
	1151	+ /*
	1152	+ * We leave it in list to prevent duplicate loads, but make sure
	1153	+ * that noone uses it while it's being deconstructed.
	1154	+ */
	1155	+ mutex_lock(&module_mutex);
	1156	+ mod->state = MODULE_STATE_UNFORMED;
	1157	+ mutex_unlock(&module_mutex);
	1158	+
	1159	+ /* Remove dynamic debug info */
	1160	+ ddebug_remove_module(mod->name);
	1161	+
	1162	+ /* Arch-specific cleanup. */
	1163	+ module_arch_cleanup(mod);
	1164	+
	1165	+ /* Module unload stuff */
	1166	+ module_unload_free(mod);
	1167	+
	1168	+ /* Free any allocated parameters. */
	1169	+ destroy_params(mod->kp, mod->num_kp);
	1170	+
	1171	+ if (is_livepatch_module(mod))
	1172	+ free_module_elf(mod);
	1173	+
	1174	+ /* Now we can delete it from the lists */
	1175	+ mutex_lock(&module_mutex);
	1176	+ /* Unlink carefully: kallsyms could be walking list. */
	1177	+ list_del_rcu(&mod->list);
	1178	+ mod_tree_remove(mod);
	1179	+ /* Remove this module from bug list, this uses list_del_rcu */
	1180	+ module_bug_cleanup(mod);
	1181	+ /* Wait for RCU-sched synchronizing before releasing mod->list and buglist. */
	1182	+ synchronize_rcu();
	1183	+ if (try_add_tainted_module(mod))
	1184	+ pr_err("%s: adding tainted module to the unloaded tainted modules list failed.\n",
	1185	+ mod->name);
	1186	+ mutex_unlock(&module_mutex);
	1187	+
	1188	+ /* Clean up CFI for the module. */
	1189	+ cfi_cleanup(mod);
	1190	+
	1191	+ /* This may be empty, but that's OK */
	1192	+ module_arch_freeing_init(mod);
	1193	+ module_memfree(mod->init_layout.base);
	1194	+ kfree(mod->args);
	1195	+ percpu_modfree(mod);
	1196	+
	1197	+ /* Free lock-classes; relies on the preceding sync_rcu(). */
	1198	+ lockdep_free_key_range(mod->data_layout.base, mod->data_layout.size);
	1199	+
	1200	+ /* Finally, free the core (containing the module structure) */
	1201	+ module_memfree(mod->core_layout.base);
	1202	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	1203	+ vfree(mod->data_layout.base);
	1204	+#endif
	1205	+}
	1206	+
	1207	+void __symbol_get(const char symbol)
	1208	+{
	1209	+ struct find_symbol_arg fsa = {
	1210	+ .name = symbol,
	1211	+ .gplok = true,
	1212	+ .warn = true,
	1213	+ };
	1214	+
	1215	+ preempt_disable();
	1216	+ if (!find_symbol(&fsa) \|\| strong_try_module_get(fsa.owner)) {
	1217	+ preempt_enable();
	1218	+ return NULL;
	1219	+ }
	1220	+ preempt_enable();
	1221	+ return (void *)kernel_symbol_value(fsa.sym);
	1222	+}
	1223	+EXPORT_SYMBOL_GPL(__symbol_get);
	1224	+
	1225	+/*
	1226	+ * Ensure that an exported symbol [global namespace] does not already exist
	1227	+ * in the kernel or in some other module's exported symbol table.
	1228	+ *
	1229	+ * You must hold the module_mutex.
	1230	+ */
	1231	+static int verify_exported_symbols(struct module *mod)
	1232	+{
	1233	+ unsigned int i;
	1234	+ const struct kernel_symbol *s;
	1235	+ struct {
	1236	+ const struct kernel_symbol *sym;
	1237	+ unsigned int num;
	1238	+ } arr[] = {
	1239	+ { mod->syms, mod->num_syms },
	1240	+ { mod->gpl_syms, mod->num_gpl_syms },
	1241	+ };
	1242	+
	1243	+ for (i = 0; i < ARRAY_SIZE(arr); i++) {
	1244	+ for (s = arr[i].sym; s < arr[i].sym + arr[i].num; s++) {
	1245	+ struct find_symbol_arg fsa = {
	1246	+ .name = kernel_symbol_name(s),
	1247	+ .gplok = true,
	1248	+ };
	1249	+ if (find_symbol(&fsa)) {
	1250	+ pr_err("%s: exports duplicate symbol %s"
	1251	+ " (owned by %s)\n",
	1252	+ mod->name, kernel_symbol_name(s),
	1253	+ module_name(fsa.owner));
	1254	+ return -ENOEXEC;
	1255	+ }
	1256	+ }
	1257	+ }
	1258	+ return 0;
	1259	+}
	1260	+
	1261	+static bool ignore_undef_symbol(Elf_Half emachine, const char *name)
	1262	+{
	1263	+ /*
	1264	+ * On x86, PIC code and Clang non-PIC code may have call foo@PLT. GNU as
	1265	+ * before 2.37 produces an unreferenced _GLOBAL_OFFSET_TABLE_ on x86-64.
	1266	+ * i386 has a similar problem but may not deserve a fix.
	1267	+ *
	1268	+ * If we ever have to ignore many symbols, consider refactoring the code to
	1269	+ * only warn if referenced by a relocation.
	1270	+ */
	1271	+ if (emachine == EM_386 \|\| emachine == EM_X86_64)
	1272	+ return !strcmp(name, "_GLOBAL_OFFSET_TABLE_");
	1273	+ return false;
	1274	+}
	1275	+
	1276	+/* Change all symbols so that st_value encodes the pointer directly. */
	1277	+static int simplify_symbols(struct module mod, const struct load_info info)
	1278	+{
	1279	+ Elf_Shdr *symsec = &info->sechdrs[info->index.sym];
	1280	+ Elf_Sym sym = (void )symsec->sh_addr;
	1281	+ unsigned long secbase;
	1282	+ unsigned int i;
	1283	+ int ret = 0;
	1284	+ const struct kernel_symbol *ksym;
	1285	+
	1286	+ for (i = 1; i < symsec->sh_size / sizeof(Elf_Sym); i++) {
	1287	+ const char *name = info->strtab + sym[i].st_name;
	1288	+
	1289	+ switch (sym[i].st_shndx) {
	1290	+ case SHN_COMMON:
	1291	+ /* Ignore common symbols */
	1292	+ if (!strncmp(name, "__gnu_lto", 9))
	1293	+ break;
	1294	+
	1295	+ /*
	1296	+ * We compiled with -fno-common. These are not
	1297	+ * supposed to happen.
	1298	+ */
	1299	+ pr_debug("Common symbol: %s\n", name);
	1300	+ pr_warn("%s: please compile with -fno-common\n",
	1301	+ mod->name);
	1302	+ ret = -ENOEXEC;
	1303	+ break;
	1304	+
	1305	+ case SHN_ABS:
	1306	+ /* Don't need to do anything */
	1307	+ pr_debug("Absolute symbol: 0x%08lx\n",
	1308	+ (long)sym[i].st_value);
	1309	+ break;
	1310	+
	1311	+ case SHN_LIVEPATCH:
	1312	+ /* Livepatch symbols are resolved by livepatch */
	1313	+ break;
	1314	+
	1315	+ case SHN_UNDEF:
	1316	+ ksym = resolve_symbol_wait(mod, info, name);
	1317	+ /* Ok if resolved. */
	1318	+ if (ksym && !IS_ERR(ksym)) {
	1319	+ sym[i].st_value = kernel_symbol_value(ksym);
	1320	+ break;
	1321	+ }
	1322	+
	1323	+ /* Ok if weak or ignored. */
	1324	+ if (!ksym &&
	1325	+ (ELF_ST_BIND(sym[i].st_info) == STB_WEAK \|\|
	1326	+ ignore_undef_symbol(info->hdr->e_machine, name)))
	1327	+ break;
	1328	+
	1329	+ ret = PTR_ERR(ksym) ?: -ENOENT;
	1330	+ pr_warn("%s: Unknown symbol %s (err %d)\n",
	1331	+ mod->name, name, ret);
	1332	+ break;
	1333	+
	1334	+ default:
	1335	+ /* Divert to percpu allocation if a percpu var. */
	1336	+ if (sym[i].st_shndx == info->index.pcpu)
	1337	+ secbase = (unsigned long)mod_percpu(mod);
	1338	+ else
	1339	+ secbase = info->sechdrs[sym[i].st_shndx].sh_addr;
	1340	+ sym[i].st_value += secbase;
	1341	+ break;
	1342	+ }
	1343	+ }
	1344	+
	1345	+ return ret;
	1346	+}
	1347	+
	1348	+static int apply_relocations(struct module mod, const struct load_info info)
	1349	+{
	1350	+ unsigned int i;
	1351	+ int err = 0;
	1352	+
	1353	+ /* Now do relocations. */
	1354	+ for (i = 1; i < info->hdr->e_shnum; i++) {
	1355	+ unsigned int infosec = info->sechdrs[i].sh_info;
	1356	+
	1357	+ /* Not a valid relocation section? */
	1358	+ if (infosec >= info->hdr->e_shnum)
	1359	+ continue;
	1360	+
	1361	+ /* Don't bother with non-allocated sections */
	1362	+ if (!(info->sechdrs[infosec].sh_flags & SHF_ALLOC))
	1363	+ continue;
	1364	+
	1365	+ if (info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH)
	1366	+ err = klp_apply_section_relocs(mod, info->sechdrs,
	1367	+ info->secstrings,
	1368	+ info->strtab,
	1369	+ info->index.sym, i,
	1370	+ NULL);
	1371	+ else if (info->sechdrs[i].sh_type == SHT_REL)
	1372	+ err = apply_relocate(info->sechdrs, info->strtab,
	1373	+ info->index.sym, i, mod);
	1374	+ else if (info->sechdrs[i].sh_type == SHT_RELA)
	1375	+ err = apply_relocate_add(info->sechdrs, info->strtab,
	1376	+ info->index.sym, i, mod);
	1377	+ if (err < 0)
	1378	+ break;
	1379	+ }
	1380	+ return err;
	1381	+}
	1382	+
	1383	+/* Additional bytes needed by arch in front of individual sections */
	1384	+unsigned int __weak arch_mod_section_prepend(struct module *mod,
	1385	+ unsigned int section)
	1386	+{
	1387	+ /* default implementation just returns zero */
	1388	+ return 0;
	1389	+}
	1390	+
	1391	+/* Update size with this section: return offset. */
	1392	+long module_get_offset(struct module mod, unsigned int size,
	1393	+ Elf_Shdr *sechdr, unsigned int section)
	1394	+{
	1395	+ long ret;
	1396	+
	1397	+ *size += arch_mod_section_prepend(mod, section);
	1398	+ ret = ALIGN(*size, sechdr->sh_addralign ?: 1);
	1399	+ *size = ret + sechdr->sh_size;
	1400	+ return ret;
	1401	+}
	1402	+
	1403	+static bool module_init_layout_section(const char *sname)
	1404	+{
	1405	+#ifndef CONFIG_MODULE_UNLOAD
	1406	+ if (module_exit_section(sname))
	1407	+ return true;
	1408	+#endif
	1409	+ return module_init_section(sname);
	1410	+}
	1411	+
	1412	+/*
	1413	+ * Lay out the SHF_ALLOC sections in a way not dissimilar to how ld
	1414	+ * might -- code, read-only data, read-write data, small data. Tally
	1415	+ * sizes, and place the offsets into sh_entsize fields: high bit means it
	1416	+ * belongs in init.
	1417	+ */
	1418	+static void layout_sections(struct module mod, struct load_info info)
	1419	+{
	1420	+ static unsigned long const masks[][2] = {
	1421	+ /*
	1422	+ * NOTE: all executable code must be the first section
	1423	+ * in this array; otherwise modify the text_size
	1424	+ * finder in the two loops below
	1425	+ */
	1426	+ { SHF_EXECINSTR \| SHF_ALLOC, ARCH_SHF_SMALL },
	1427	+ { SHF_ALLOC, SHF_WRITE \| ARCH_SHF_SMALL },
	1428	+ { SHF_RO_AFTER_INIT \| SHF_ALLOC, ARCH_SHF_SMALL },
	1429	+ { SHF_WRITE \| SHF_ALLOC, ARCH_SHF_SMALL },
	1430	+ { ARCH_SHF_SMALL \| SHF_ALLOC, 0 }
	1431	+ };
	1432	+ unsigned int m, i;
	1433	+
	1434	+ for (i = 0; i < info->hdr->e_shnum; i++)
	1435	+ info->sechdrs[i].sh_entsize = ~0UL;
	1436	+
	1437	+ pr_debug("Core section allocation order:\n");
	1438	+ for (m = 0; m < ARRAY_SIZE(masks); ++m) {
	1439	+ for (i = 0; i < info->hdr->e_shnum; ++i) {
	1440	+ Elf_Shdr *s = &info->sechdrs[i];
	1441	+ const char *sname = info->secstrings + s->sh_name;
	1442	+ unsigned int *sizep;
	1443	+
	1444	+ if ((s->sh_flags & masks[m][0]) != masks[m][0]
	1445	+ \|\| (s->sh_flags & masks[m][1])
	1446	+ \|\| s->sh_entsize != ~0UL
	1447	+ \|\| module_init_layout_section(sname))
	1448	+ continue;
	1449	+ sizep = m ? &mod->data_layout.size : &mod->core_layout.size;
	1450	+ s->sh_entsize = module_get_offset(mod, sizep, s, i);
	1451	+ pr_debug("\t%s\n", sname);
	1452	+ }
	1453	+ switch (m) {
	1454	+ case 0: /* executable */
	1455	+ mod->core_layout.size = strict_align(mod->core_layout.size);
	1456	+ mod->core_layout.text_size = mod->core_layout.size;
	1457	+ break;
	1458	+ case 1: /* RO: text and ro-data */
	1459	+ mod->data_layout.size = strict_align(mod->data_layout.size);
	1460	+ mod->data_layout.ro_size = mod->data_layout.size;
	1461	+ break;
	1462	+ case 2: /* RO after init */
	1463	+ mod->data_layout.size = strict_align(mod->data_layout.size);
	1464	+ mod->data_layout.ro_after_init_size = mod->data_layout.size;
	1465	+ break;
	1466	+ case 4: /* whole core */
	1467	+ mod->data_layout.size = strict_align(mod->data_layout.size);
	1468	+ break;
	1469	+ }
	1470	+ }
	1471	+
	1472	+ pr_debug("Init section allocation order:\n");
	1473	+ for (m = 0; m < ARRAY_SIZE(masks); ++m) {
	1474	+ for (i = 0; i < info->hdr->e_shnum; ++i) {
	1475	+ Elf_Shdr *s = &info->sechdrs[i];
	1476	+ const char *sname = info->secstrings + s->sh_name;
	1477	+
	1478	+ if ((s->sh_flags & masks[m][0]) != masks[m][0]
	1479	+ \|\| (s->sh_flags & masks[m][1])
	1480	+ \|\| s->sh_entsize != ~0UL
	1481	+ \|\| !module_init_layout_section(sname))
	1482	+ continue;
	1483	+ s->sh_entsize = (module_get_offset(mod, &mod->init_layout.size, s, i)
	1484	+ \| INIT_OFFSET_MASK);
	1485	+ pr_debug("\t%s\n", sname);
	1486	+ }
	1487	+ switch (m) {
	1488	+ case 0: /* executable */
	1489	+ mod->init_layout.size = strict_align(mod->init_layout.size);
	1490	+ mod->init_layout.text_size = mod->init_layout.size;
	1491	+ break;
	1492	+ case 1: /* RO: text and ro-data */
	1493	+ mod->init_layout.size = strict_align(mod->init_layout.size);
	1494	+ mod->init_layout.ro_size = mod->init_layout.size;
	1495	+ break;
	1496	+ case 2:
	1497	+ /*
	1498	+ * RO after init doesn't apply to init_layout (only
	1499	+ * core_layout), so it just takes the value of ro_size.
	1500	+ */
	1501	+ mod->init_layout.ro_after_init_size = mod->init_layout.ro_size;
	1502	+ break;
	1503	+ case 4: /* whole init */
	1504	+ mod->init_layout.size = strict_align(mod->init_layout.size);
	1505	+ break;
	1506	+ }
	1507	+ }
	1508	+}
	1509	+
	1510	+static void set_license(struct module mod, const char license)
	1511	+{
	1512	+ if (!license)
	1513	+ license = "unspecified";
	1514	+
	1515	+ if (!license_is_gpl_compatible(license)) {
	1516	+ if (!test_taint(TAINT_PROPRIETARY_MODULE))
	1517	+ pr_warn("%s: module license '%s' taints kernel.\n",
	1518	+ mod->name, license);
	1519	+ add_taint_module(mod, TAINT_PROPRIETARY_MODULE,
	1520	+ LOCKDEP_NOW_UNRELIABLE);
	1521	+ }
	1522	+}
	1523	+
	1524	+/* Parse tag=value strings from .modinfo section */
	1525	+static char next_string(char string, unsigned long *secsize)
	1526	+{
	1527	+ /* Skip non-zero chars */
	1528	+ while (string[0]) {
	1529	+ string++;
	1530	+ if ((*secsize)-- <= 1)
	1531	+ return NULL;
	1532	+ }
	1533	+
	1534	+ /* Skip any zero padding. */
	1535	+ while (!string[0]) {
	1536	+ string++;
	1537	+ if ((*secsize)-- <= 1)
	1538	+ return NULL;
	1539	+ }
	1540	+ return string;
	1541	+}
	1542	+
	1543	+static char get_next_modinfo(const struct load_info info, const char *tag,
	1544	+ char *prev)
	1545	+{
	1546	+ char *p;
	1547	+ unsigned int taglen = strlen(tag);
	1548	+ Elf_Shdr *infosec = &info->sechdrs[info->index.info];
	1549	+ unsigned long size = infosec->sh_size;
	1550	+
	1551	+ /*
	1552	+ * get_modinfo() calls made before rewrite_section_headers()
	1553	+ * must use sh_offset, as sh_addr isn't set!
	1554	+ */
	1555	+ char modinfo = (char )info->hdr + infosec->sh_offset;
	1556	+
	1557	+ if (prev) {
	1558	+ size -= prev - modinfo;
	1559	+ modinfo = next_string(prev, &size);
	1560	+ }
	1561	+
	1562	+ for (p = modinfo; p; p = next_string(p, &size)) {
	1563	+ if (strncmp(p, tag, taglen) == 0 && p[taglen] == '=')
	1564	+ return p + taglen + 1;
	1565	+ }
	1566	+ return NULL;
	1567	+}
	1568	+
	1569	+static char get_modinfo(const struct load_info info, const char *tag)
	1570	+{
	1571	+ return get_next_modinfo(info, tag, NULL);
	1572	+}
	1573	+
	1574	+static void setup_modinfo(struct module mod, struct load_info info)
	1575	+{
	1576	+ struct module_attribute *attr;
	1577	+ int i;
	1578	+
	1579	+ for (i = 0; (attr = modinfo_attrs[i]); i++) {
	1580	+ if (attr->setup)
	1581	+ attr->setup(mod, get_modinfo(info, attr->attr.name));
	1582	+ }
	1583	+}
	1584	+
	1585	+static void free_modinfo(struct module *mod)
	1586	+{
	1587	+ struct module_attribute *attr;
	1588	+ int i;
	1589	+
	1590	+ for (i = 0; (attr = modinfo_attrs[i]); i++) {
	1591	+ if (attr->free)
	1592	+ attr->free(mod);
	1593	+ }
	1594	+}
	1595	+
	1596	+static void dynamic_debug_setup(struct module mod, struct _ddebug debug, unsigned int num)
	1597	+{
	1598	+ if (!debug)
	1599	+ return;
	1600	+ ddebug_add_module(debug, num, mod->name);
	1601	+}
	1602	+
	1603	+static void dynamic_debug_remove(struct module mod, struct _ddebug debug)
	1604	+{
	1605	+ if (debug)
	1606	+ ddebug_remove_module(mod->name);
	1607	+}
	1608	+
	1609	+void * __weak module_alloc(unsigned long size)
	1610	+{
	1611	+ return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END,
	1612	+ GFP_KERNEL, PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS,
	1613	+ NUMA_NO_NODE, __builtin_return_address(0));
	1614	+}
	1615	+
	1616	+bool __weak module_init_section(const char *name)
	1617	+{
	1618	+ return strstarts(name, ".init");
	1619	+}
	1620	+
	1621	+bool __weak module_exit_section(const char *name)
	1622	+{
	1623	+ return strstarts(name, ".exit");
	1624	+}
	1625	+
	1626	+static int validate_section_offset(struct load_info info, Elf_Shdr shdr)
	1627	+{
	1628	+#if defined(CONFIG_64BIT)
	1629	+ unsigned long long secend;
	1630	+#else
	1631	+ unsigned long secend;
	1632	+#endif
	1633	+
	1634	+ /*
	1635	+ * Check for both overflow and offset/size being
	1636	+ * too large.
	1637	+ */
	1638	+ secend = shdr->sh_offset + shdr->sh_size;
	1639	+ if (secend < shdr->sh_offset \|\| secend > info->len)
	1640	+ return -ENOEXEC;
	1641	+
	1642	+ return 0;
	1643	+}
	1644	+
	1645	+/*
	1646	+ * Sanity checks against invalid binaries, wrong arch, weird elf version.
	1647	+ *
	1648	+ * Also do basic validity checks against section offsets and sizes, the
	1649	+ * section name string table, and the indices used for it (sh_name).
	1650	+ */
	1651	+static int elf_validity_check(struct load_info *info)
	1652	+{
	1653	+ unsigned int i;
	1654	+ Elf_Shdr shdr, strhdr;
	1655	+ int err;
	1656	+
	1657	+ if (info->len < sizeof(*(info->hdr))) {
	1658	+ pr_err("Invalid ELF header len %lu\n", info->len);
	1659	+ goto no_exec;
	1660	+ }
	1661	+
	1662	+ if (memcmp(info->hdr->e_ident, ELFMAG, SELFMAG) != 0) {
	1663	+ pr_err("Invalid ELF header magic: != %s\n", ELFMAG);
	1664	+ goto no_exec;
	1665	+ }
	1666	+ if (info->hdr->e_type != ET_REL) {
	1667	+ pr_err("Invalid ELF header type: %u != %u\n",
	1668	+ info->hdr->e_type, ET_REL);
	1669	+ goto no_exec;
	1670	+ }
	1671	+ if (!elf_check_arch(info->hdr)) {
	1672	+ pr_err("Invalid architecture in ELF header: %u\n",
	1673	+ info->hdr->e_machine);
	1674	+ goto no_exec;
	1675	+ }
	1676	+ if (info->hdr->e_shentsize != sizeof(Elf_Shdr)) {
	1677	+ pr_err("Invalid ELF section header size\n");
	1678	+ goto no_exec;
	1679	+ }
	1680	+
	1681	+ /*
	1682	+ * e_shnum is 16 bits, and sizeof(Elf_Shdr) is
	1683	+ * known and small. So e_shnum * sizeof(Elf_Shdr)
	1684	+ * will not overflow unsigned long on any platform.
	1685	+ */
	1686	+ if (info->hdr->e_shoff >= info->len
	1687	+ \|\| (info->hdr->e_shnum * sizeof(Elf_Shdr) >
	1688	+ info->len - info->hdr->e_shoff)) {
	1689	+ pr_err("Invalid ELF section header overflow\n");
	1690	+ goto no_exec;
	1691	+ }
	1692	+
	1693	+ info->sechdrs = (void *)info->hdr + info->hdr->e_shoff;
	1694	+
	1695	+ /*
	1696	+ * Verify if the section name table index is valid.
	1697	+ */
	1698	+ if (info->hdr->e_shstrndx == SHN_UNDEF
	1699	+ \|\| info->hdr->e_shstrndx >= info->hdr->e_shnum) {
	1700	+ pr_err("Invalid ELF section name index: %d \|\| e_shstrndx (%d) >= e_shnum (%d)\n",
	1701	+ info->hdr->e_shstrndx, info->hdr->e_shstrndx,
	1702	+ info->hdr->e_shnum);
	1703	+ goto no_exec;
	1704	+ }
	1705	+
	1706	+ strhdr = &info->sechdrs[info->hdr->e_shstrndx];
	1707	+ err = validate_section_offset(info, strhdr);
	1708	+ if (err < 0) {
	1709	+ pr_err("Invalid ELF section hdr(type %u)\n", strhdr->sh_type);
	1710	+ return err;
	1711	+ }
	1712	+
	1713	+ /*
	1714	+ * The section name table must be NUL-terminated, as required
	1715	+ * by the spec. This makes strcmp and pr_* calls that access
	1716	+ * strings in the section safe.
	1717	+ */
	1718	+ info->secstrings = (void *)info->hdr + strhdr->sh_offset;
	1719	+ if (strhdr->sh_size == 0) {
	1720	+ pr_err("empty section name table\n");
	1721	+ goto no_exec;
	1722	+ }
	1723	+ if (info->secstrings[strhdr->sh_size - 1] != '\0') {
	1724	+ pr_err("ELF Spec violation: section name table isn't null terminated\n");
	1725	+ goto no_exec;
	1726	+ }
	1727	+
	1728	+ /*
	1729	+ * The code assumes that section 0 has a length of zero and
	1730	+ * an addr of zero, so check for it.
	1731	+ */
	1732	+ if (info->sechdrs[0].sh_type != SHT_NULL
	1733	+ \|\| info->sechdrs[0].sh_size != 0
	1734	+ \|\| info->sechdrs[0].sh_addr != 0) {
	1735	+ pr_err("ELF Spec violation: section 0 type(%d)!=SH_NULL or non-zero len or addr\n",
	1736	+ info->sechdrs[0].sh_type);
	1737	+ goto no_exec;
	1738	+ }
	1739	+
	1740	+ for (i = 1; i < info->hdr->e_shnum; i++) {
	1741	+ shdr = &info->sechdrs[i];
	1742	+ switch (shdr->sh_type) {
	1743	+ case SHT_NULL:
	1744	+ case SHT_NOBITS:
	1745	+ continue;
	1746	+ case SHT_SYMTAB:
	1747	+ if (shdr->sh_link == SHN_UNDEF
	1748	+ \|\| shdr->sh_link >= info->hdr->e_shnum) {
	1749	+ pr_err("Invalid ELF sh_link!=SHN_UNDEF(%d) or (sh_link(%d) >= hdr->e_shnum(%d)\n",
	1750	+ shdr->sh_link, shdr->sh_link,
	1751	+ info->hdr->e_shnum);
	1752	+ goto no_exec;
	1753	+ }
	1754	+ fallthrough;
	1755	+ default:
	1756	+ err = validate_section_offset(info, shdr);
	1757	+ if (err < 0) {
	1758	+ pr_err("Invalid ELF section in module (section %u type %u)\n",
	1759	+ i, shdr->sh_type);
	1760	+ return err;
	1761	+ }
	1762	+
	1763	+ if (shdr->sh_flags & SHF_ALLOC) {
	1764	+ if (shdr->sh_name >= strhdr->sh_size) {
	1765	+ pr_err("Invalid ELF section name in module (section %u type %u)\n",
	1766	+ i, shdr->sh_type);
	1767	+ return -ENOEXEC;
	1768	+ }
	1769	+ }
	1770	+ break;
	1771	+ }
	1772	+ }
	1773	+
	1774	+ return 0;
	1775	+
	1776	+no_exec:
	1777	+ return -ENOEXEC;
	1778	+}
	1779	+
	1780	+#define COPY_CHUNK_SIZE (16*PAGE_SIZE)
	1781	+
	1782	+static int copy_chunked_from_user(void dst, const void __user usrc, unsigned long len)
	1783	+{
	1784	+ do {
	1785	+ unsigned long n = min(len, COPY_CHUNK_SIZE);
	1786	+
	1787	+ if (copy_from_user(dst, usrc, n) != 0)
	1788	+ return -EFAULT;
	1789	+ cond_resched();
	1790	+ dst += n;
	1791	+ usrc += n;
	1792	+ len -= n;
	1793	+ } while (len);
	1794	+ return 0;
	1795	+}
	1796	+
	1797	+static int check_modinfo_livepatch(struct module mod, struct load_info info)
	1798	+{
	1799	+ if (!get_modinfo(info, "livepatch"))
	1800	+ /* Nothing more to do */
	1801	+ return 0;
	1802	+
	1803	+ if (set_livepatch_module(mod)) {
	1804	+ add_taint_module(mod, TAINT_LIVEPATCH, LOCKDEP_STILL_OK);
	1805	+ pr_notice_once("%s: tainting kernel with TAINT_LIVEPATCH\n",
	1806	+ mod->name);
	1807	+ return 0;
	1808	+ }
	1809	+
	1810	+ pr_err("%s: module is marked as livepatch module, but livepatch support is disabled",
	1811	+ mod->name);
	1812	+ return -ENOEXEC;
	1813	+}
	1814	+
	1815	+static void check_modinfo_retpoline(struct module mod, struct load_info info)
	1816	+{
	1817	+ if (retpoline_module_ok(get_modinfo(info, "retpoline")))
	1818	+ return;
	1819	+
	1820	+ pr_warn("%s: loading module not compiled with retpoline compiler.\n",
	1821	+ mod->name);
	1822	+}
	1823	+
	1824	+/* Sets info->hdr and info->len. */
	1825	+static int copy_module_from_user(const void __user *umod, unsigned long len,
	1826	+ struct load_info *info)
	1827	+{
	1828	+ int err;
	1829	+
	1830	+ info->len = len;
	1831	+ if (info->len < sizeof(*(info->hdr)))
	1832	+ return -ENOEXEC;
	1833	+
	1834	+ err = security_kernel_load_data(LOADING_MODULE, true);
	1835	+ if (err)
	1836	+ return err;
	1837	+
	1838	+ /* Suck in entire file: we'll want most of it. */
	1839	+ info->hdr = __vmalloc(info->len, GFP_KERNEL \| __GFP_NOWARN);
	1840	+ if (!info->hdr)
	1841	+ return -ENOMEM;
	1842	+
	1843	+ if (copy_chunked_from_user(info->hdr, umod, info->len) != 0) {
	1844	+ err = -EFAULT;
	1845	+ goto out;
	1846	+ }
	1847	+
	1848	+ err = security_kernel_post_load_data((char *)info->hdr, info->len,
	1849	+ LOADING_MODULE, "init_module");
	1850	+out:
	1851	+ if (err)
	1852	+ vfree(info->hdr);
	1853	+
	1854	+ return err;
	1855	+}
	1856	+
	1857	+static void free_copy(struct load_info *info, int flags)
	1858	+{
	1859	+ if (flags & MODULE_INIT_COMPRESSED_FILE)
	1860	+ module_decompress_cleanup(info);
	1861	+ else
	1862	+ vfree(info->hdr);
	1863	+}
	1864	+
	1865	+static int rewrite_section_headers(struct load_info *info, int flags)
	1866	+{
	1867	+ unsigned int i;
	1868	+
	1869	+ /* This should always be true, but let's be sure. */
	1870	+ info->sechdrs[0].sh_addr = 0;
	1871	+
	1872	+ for (i = 1; i < info->hdr->e_shnum; i++) {
	1873	+ Elf_Shdr *shdr = &info->sechdrs[i];
	1874	+
	1875	+ /*
	1876	+ * Mark all sections sh_addr with their address in the
	1877	+ * temporary image.
	1878	+ */
	1879	+ shdr->sh_addr = (size_t)info->hdr + shdr->sh_offset;
	1880	+
	1881	+ }
	1882	+
	1883	+ /* Track but don't keep modinfo and version sections. */
	1884	+ info->sechdrs[info->index.vers].sh_flags &= ~(unsigned long)SHF_ALLOC;
	1885	+ info->sechdrs[info->index.info].sh_flags &= ~(unsigned long)SHF_ALLOC;
	1886	+
	1887	+ return 0;
	1888	+}
	1889	+
	1890	+/*
	1891	+ * Set up our basic convenience variables (pointers to section headers,
	1892	+ * search for module section index etc), and do some basic section
	1893	+ * verification.
	1894	+ *
	1895	+ * Set info->mod to the temporary copy of the module in info->hdr. The final one
	1896	+ * will be allocated in move_module().
	1897	+ */
	1898	+static int setup_load_info(struct load_info *info, int flags)
	1899	+{
	1900	+ unsigned int i;
	1901	+
	1902	+ /* Try to find a name early so we can log errors with a module name */
	1903	+ info->index.info = find_sec(info, ".modinfo");
	1904	+ if (info->index.info)
	1905	+ info->name = get_modinfo(info, "name");
	1906	+
	1907	+ /* Find internal symbols and strings. */
	1908	+ for (i = 1; i < info->hdr->e_shnum; i++) {
	1909	+ if (info->sechdrs[i].sh_type == SHT_SYMTAB) {
	1910	+ info->index.sym = i;
	1911	+ info->index.str = info->sechdrs[i].sh_link;
	1912	+ info->strtab = (char *)info->hdr
	1913	+ + info->sechdrs[info->index.str].sh_offset;
	1914	+ break;
	1915	+ }
	1916	+ }
	1917	+
	1918	+ if (info->index.sym == 0) {
	1919	+ pr_warn("%s: module has no symbols (stripped?)\n",
	1920	+ info->name ?: "(missing .modinfo section or name field)");
	1921	+ return -ENOEXEC;
	1922	+ }
	1923	+
	1924	+ info->index.mod = find_sec(info, ".gnu.linkonce.this_module");
	1925	+ if (!info->index.mod) {
	1926	+ pr_warn("%s: No module found in object\n",
	1927	+ info->name ?: "(missing .modinfo section or name field)");
	1928	+ return -ENOEXEC;
	1929	+ }
	1930	+ /* This is temporary: point mod into copy of data. */
	1931	+ info->mod = (void *)info->hdr + info->sechdrs[info->index.mod].sh_offset;
	1932	+
	1933	+ /*
	1934	+ * If we didn't load the .modinfo 'name' field earlier, fall back to
	1935	+ * on-disk struct mod 'name' field.
	1936	+ */
	1937	+ if (!info->name)
	1938	+ info->name = info->mod->name;
	1939	+
	1940	+ if (flags & MODULE_INIT_IGNORE_MODVERSIONS)
	1941	+ info->index.vers = 0; /* Pretend no __versions section! */
	1942	+ else
	1943	+ info->index.vers = find_sec(info, "__versions");
	1944	+
	1945	+ info->index.pcpu = find_pcpusec(info);
	1946	+
	1947	+ return 0;
	1948	+}
	1949	+
	1950	+static int check_modinfo(struct module mod, struct load_info info, int flags)
	1951	+{
	1952	+ const char *modmagic = get_modinfo(info, "vermagic");
	1953	+ int err;
	1954	+
	1955	+ if (flags & MODULE_INIT_IGNORE_VERMAGIC)
	1956	+ modmagic = NULL;
	1957	+
	1958	+ /* This is allowed: modprobe --force will invalidate it. */
	1959	+ if (!modmagic) {
	1960	+ err = try_to_force_load(mod, "bad vermagic");
	1961	+ if (err)
	1962	+ return err;
	1963	+ } else if (!same_magic(modmagic, vermagic, info->index.vers)) {
	1964	+ pr_err("%s: version magic '%s' should be '%s'\n",
	1965	+ info->name, modmagic, vermagic);
	1966	+ return -ENOEXEC;
	1967	+ }
	1968	+
	1969	+ if (!get_modinfo(info, "intree")) {
	1970	+ if (!test_taint(TAINT_OOT_MODULE))
	1971	+ pr_warn("%s: loading out-of-tree module taints kernel.\n",
	1972	+ mod->name);
	1973	+ add_taint_module(mod, TAINT_OOT_MODULE, LOCKDEP_STILL_OK);
	1974	+ }
	1975	+
	1976	+ check_modinfo_retpoline(mod, info);
	1977	+
	1978	+ if (get_modinfo(info, "staging")) {
	1979	+ add_taint_module(mod, TAINT_CRAP, LOCKDEP_STILL_OK);
	1980	+ pr_warn("%s: module is from the staging directory, the quality "
	1981	+ "is unknown, you have been warned.\n", mod->name);
	1982	+ }
	1983	+
	1984	+ err = check_modinfo_livepatch(mod, info);
	1985	+ if (err)
	1986	+ return err;
	1987	+
	1988	+ /* Set up license info based on the info section */
	1989	+ set_license(mod, get_modinfo(info, "license"));
	1990	+
	1991	+ return 0;
	1992	+}
	1993	+
	1994	+static int find_module_sections(struct module mod, struct load_info info)
	1995	+{
	1996	+ mod->kp = section_objs(info, "__param",
	1997	+ sizeof(*mod->kp), &mod->num_kp);
	1998	+ mod->syms = section_objs(info, "__ksymtab",
	1999	+ sizeof(*mod->syms), &mod->num_syms);
	2000	+ mod->crcs = section_addr(info, "__kcrctab");
	2001	+ mod->gpl_syms = section_objs(info, "__ksymtab_gpl",
	2002	+ sizeof(*mod->gpl_syms),
	2003	+ &mod->num_gpl_syms);
	2004	+ mod->gpl_crcs = section_addr(info, "__kcrctab_gpl");
	2005	+
	2006	+#ifdef CONFIG_CONSTRUCTORS
	2007	+ mod->ctors = section_objs(info, ".ctors",
	2008	+ sizeof(*mod->ctors), &mod->num_ctors);
	2009	+ if (!mod->ctors)
	2010	+ mod->ctors = section_objs(info, ".init_array",
	2011	+ sizeof(*mod->ctors), &mod->num_ctors);
	2012	+ else if (find_sec(info, ".init_array")) {
	2013	+ /*
	2014	+ * This shouldn't happen with same compiler and binutils
	2015	+ * building all parts of the module.
	2016	+ */
	2017	+ pr_warn("%s: has both .ctors and .init_array.\n",
	2018	+ mod->name);
	2019	+ return -EINVAL;
	2020	+ }
	2021	+#endif
	2022	+
	2023	+ mod->noinstr_text_start = section_objs(info, ".noinstr.text", 1,
	2024	+ &mod->noinstr_text_size);
	2025	+
	2026	+#ifdef CONFIG_TRACEPOINTS
	2027	+ mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
	2028	+ sizeof(*mod->tracepoints_ptrs),
	2029	+ &mod->num_tracepoints);
	2030	+#endif
	2031	+#ifdef CONFIG_TREE_SRCU
	2032	+ mod->srcu_struct_ptrs = section_objs(info, "___srcu_struct_ptrs",
	2033	+ sizeof(*mod->srcu_struct_ptrs),
	2034	+ &mod->num_srcu_structs);
	2035	+#endif
	2036	+#ifdef CONFIG_BPF_EVENTS
	2037	+ mod->bpf_raw_events = section_objs(info, "__bpf_raw_tp_map",
	2038	+ sizeof(*mod->bpf_raw_events),
	2039	+ &mod->num_bpf_raw_events);
	2040	+#endif
	2041	+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
	2042	+ mod->btf_data = any_section_objs(info, ".BTF", 1, &mod->btf_data_size);
	2043	+#endif
	2044	+#ifdef CONFIG_JUMP_LABEL
	2045	+ mod->jump_entries = section_objs(info, "__jump_table",
	2046	+ sizeof(*mod->jump_entries),
	2047	+ &mod->num_jump_entries);
	2048	+#endif
	2049	+#ifdef CONFIG_EVENT_TRACING
	2050	+ mod->trace_events = section_objs(info, "_ftrace_events",
	2051	+ sizeof(*mod->trace_events),
	2052	+ &mod->num_trace_events);
	2053	+ mod->trace_evals = section_objs(info, "_ftrace_eval_map",
	2054	+ sizeof(*mod->trace_evals),
	2055	+ &mod->num_trace_evals);
	2056	+#endif
	2057	+#ifdef CONFIG_TRACING
	2058	+ mod->trace_bprintk_fmt_start = section_objs(info, "__trace_printk_fmt",
	2059	+ sizeof(*mod->trace_bprintk_fmt_start),
	2060	+ &mod->num_trace_bprintk_fmt);
	2061	+#endif
	2062	+#ifdef CONFIG_FTRACE_MCOUNT_RECORD
	2063	+ /* sechdrs[0].sh_size is always zero */
	2064	+ mod->ftrace_callsites = section_objs(info, FTRACE_CALLSITE_SECTION,
	2065	+ sizeof(*mod->ftrace_callsites),
	2066	+ &mod->num_ftrace_callsites);
	2067	+#endif
	2068	+#ifdef CONFIG_FUNCTION_ERROR_INJECTION
	2069	+ mod->ei_funcs = section_objs(info, "_error_injection_whitelist",
	2070	+ sizeof(*mod->ei_funcs),
	2071	+ &mod->num_ei_funcs);
	2072	+#endif
	2073	+#ifdef CONFIG_KPROBES
	2074	+ mod->kprobes_text_start = section_objs(info, ".kprobes.text", 1,
	2075	+ &mod->kprobes_text_size);
	2076	+ mod->kprobe_blacklist = section_objs(info, "_kprobe_blacklist",
	2077	+ sizeof(unsigned long),
	2078	+ &mod->num_kprobe_blacklist);
	2079	+#endif
	2080	+#ifdef CONFIG_PRINTK_INDEX
	2081	+ mod->printk_index_start = section_objs(info, ".printk_index",
	2082	+ sizeof(*mod->printk_index_start),
	2083	+ &mod->printk_index_size);
	2084	+#endif
	2085	+#ifdef CONFIG_HAVE_STATIC_CALL_INLINE
	2086	+ mod->static_call_sites = section_objs(info, ".static_call_sites",
	2087	+ sizeof(*mod->static_call_sites),
	2088	+ &mod->num_static_call_sites);
	2089	+#endif
	2090	+ mod->extable = section_objs(info, "__ex_table",
	2091	+ sizeof(*mod->extable), &mod->num_exentries);
	2092	+
	2093	+ if (section_addr(info, "__obsparm"))
	2094	+ pr_warn("%s: Ignoring obsolete parameters\n", mod->name);
	2095	+
	2096	+ info->debug = section_objs(info, "__dyndbg",
	2097	+ sizeof(*info->debug), &info->num_debug);
	2098	+
	2099	+ return 0;
	2100	+}
	2101	+
	2102	+static int move_module(struct module mod, struct load_info info)
	2103	+{
	2104	+ int i;
	2105	+ void *ptr;
	2106	+
	2107	+ /* Do the allocs. */
	2108	+ ptr = module_alloc(mod->core_layout.size);
	2109	+ /*
	2110	+ * The pointer to this block is stored in the module structure
	2111	+ * which is inside the block. Just mark it as not being a
	2112	+ * leak.
	2113	+ */
	2114	+ kmemleak_not_leak(ptr);
	2115	+ if (!ptr)
	2116	+ return -ENOMEM;
	2117	+
	2118	+ memset(ptr, 0, mod->core_layout.size);
	2119	+ mod->core_layout.base = ptr;
	2120	+
	2121	+ if (mod->init_layout.size) {
	2122	+ ptr = module_alloc(mod->init_layout.size);
	2123	+ /*
	2124	+ * The pointer to this block is stored in the module structure
	2125	+ * which is inside the block. This block doesn't need to be
	2126	+ * scanned as it contains data and code that will be freed
	2127	+ * after the module is initialized.
	2128	+ */
	2129	+ kmemleak_ignore(ptr);
	2130	+ if (!ptr) {
	2131	+ module_memfree(mod->core_layout.base);
	2132	+ return -ENOMEM;
	2133	+ }
	2134	+ memset(ptr, 0, mod->init_layout.size);
	2135	+ mod->init_layout.base = ptr;
	2136	+ } else
	2137	+ mod->init_layout.base = NULL;
	2138	+
	2139	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	2140	+ /* Do the allocs. */
	2141	+ ptr = vmalloc(mod->data_layout.size);
	2142	+ /*
	2143	+ * The pointer to this block is stored in the module structure
	2144	+ * which is inside the block. Just mark it as not being a
	2145	+ * leak.
	2146	+ */
	2147	+ kmemleak_not_leak(ptr);
	2148	+ if (!ptr) {
	2149	+ module_memfree(mod->core_layout.base);
	2150	+ module_memfree(mod->init_layout.base);
	2151	+ return -ENOMEM;
	2152	+ }
	2153	+
	2154	+ memset(ptr, 0, mod->data_layout.size);
	2155	+ mod->data_layout.base = ptr;
	2156	+#endif
	2157	+ /* Transfer each section which specifies SHF_ALLOC */
	2158	+ pr_debug("final section addresses:\n");
	2159	+ for (i = 0; i < info->hdr->e_shnum; i++) {
	2160	+ void *dest;
	2161	+ Elf_Shdr *shdr = &info->sechdrs[i];
	2162	+
	2163	+ if (!(shdr->sh_flags & SHF_ALLOC))
	2164	+ continue;
	2165	+
	2166	+ if (shdr->sh_entsize & INIT_OFFSET_MASK)
	2167	+ dest = mod->init_layout.base
	2168	+ + (shdr->sh_entsize & ~INIT_OFFSET_MASK);
	2169	+ else if (!(shdr->sh_flags & SHF_EXECINSTR))
	2170	+ dest = mod->data_layout.base + shdr->sh_entsize;
	2171	+ else
	2172	+ dest = mod->core_layout.base + shdr->sh_entsize;
	2173	+
	2174	+ if (shdr->sh_type != SHT_NOBITS)
	2175	+ memcpy(dest, (void *)shdr->sh_addr, shdr->sh_size);
	2176	+ /* Update sh_addr to point to copy in image. */
	2177	+ shdr->sh_addr = (unsigned long)dest;
	2178	+ pr_debug("\t0x%lx %s\n",
	2179	+ (long)shdr->sh_addr, info->secstrings + shdr->sh_name);
	2180	+ }
	2181	+
	2182	+ return 0;
	2183	+}
	2184	+
	2185	+static int check_module_license_and_versions(struct module *mod)
	2186	+{
	2187	+ int prev_taint = test_taint(TAINT_PROPRIETARY_MODULE);
	2188	+
	2189	+ /*
	2190	+ * ndiswrapper is under GPL by itself, but loads proprietary modules.
	2191	+ * Don't use add_taint_module(), as it would prevent ndiswrapper from
	2192	+ * using GPL-only symbols it needs.
	2193	+ */
	2194	+ if (strcmp(mod->name, "ndiswrapper") == 0)
	2195	+ add_taint(TAINT_PROPRIETARY_MODULE, LOCKDEP_NOW_UNRELIABLE);
	2196	+
	2197	+ /* driverloader was caught wrongly pretending to be under GPL */
	2198	+ if (strcmp(mod->name, "driverloader") == 0)
	2199	+ add_taint_module(mod, TAINT_PROPRIETARY_MODULE,
	2200	+ LOCKDEP_NOW_UNRELIABLE);
	2201	+
	2202	+ /* lve claims to be GPL but upstream won't provide source */
	2203	+ if (strcmp(mod->name, "lve") == 0)
	2204	+ add_taint_module(mod, TAINT_PROPRIETARY_MODULE,
	2205	+ LOCKDEP_NOW_UNRELIABLE);
	2206	+
	2207	+ if (!prev_taint && test_taint(TAINT_PROPRIETARY_MODULE))
	2208	+ pr_warn("%s: module license taints kernel.\n", mod->name);
	2209	+
	2210	+#ifdef CONFIG_MODVERSIONS
	2211	+ if ((mod->num_syms && !mod->crcs) \|\|
	2212	+ (mod->num_gpl_syms && !mod->gpl_crcs)) {
	2213	+ return try_to_force_load(mod,
	2214	+ "no versions for exported symbols");
	2215	+ }
	2216	+#endif
	2217	+ return 0;
	2218	+}
	2219	+
	2220	+static void flush_module_icache(const struct module *mod)
	2221	+{
	2222	+ /*
	2223	+ * Flush the instruction cache, since we've played with text.
	2224	+ * Do it before processing of module parameters, so the module
	2225	+ * can provide parameter accessor functions of its own.
	2226	+ */
	2227	+ if (mod->init_layout.base)
	2228	+ flush_icache_range((unsigned long)mod->init_layout.base,
	2229	+ (unsigned long)mod->init_layout.base
	2230	+ + mod->init_layout.size);
	2231	+ flush_icache_range((unsigned long)mod->core_layout.base,
	2232	+ (unsigned long)mod->core_layout.base + mod->core_layout.size);
	2233	+}
	2234	+
	2235	+int __weak module_frob_arch_sections(Elf_Ehdr *hdr,
	2236	+ Elf_Shdr *sechdrs,
	2237	+ char *secstrings,
	2238	+ struct module *mod)
	2239	+{
	2240	+ return 0;
	2241	+}
	2242	+
	2243	+/* module_blacklist is a comma-separated list of module names */
	2244	+static char *module_blacklist;
	2245	+static bool blacklisted(const char *module_name)
	2246	+{
	2247	+ const char *p;
	2248	+ size_t len;
	2249	+
	2250	+ if (!module_blacklist)
	2251	+ return false;
	2252	+
	2253	+ for (p = module_blacklist; *p; p += len) {
	2254	+ len = strcspn(p, ",");
	2255	+ if (strlen(module_name) == len && !memcmp(module_name, p, len))
	2256	+ return true;
	2257	+ if (p[len] == ',')
	2258	+ len++;
	2259	+ }
	2260	+ return false;
	2261	+}
	2262	+core_param(module_blacklist, module_blacklist, charp, 0400);
	2263	+
	2264	+static struct module layout_and_allocate(struct load_info info, int flags)
	2265	+{
	2266	+ struct module *mod;
	2267	+ unsigned int ndx;
	2268	+ int err;
	2269	+
	2270	+ err = check_modinfo(info->mod, info, flags);
	2271	+ if (err)
	2272	+ return ERR_PTR(err);
	2273	+
	2274	+ /* Allow arches to frob section contents and sizes. */
	2275	+ err = module_frob_arch_sections(info->hdr, info->sechdrs,
	2276	+ info->secstrings, info->mod);
	2277	+ if (err < 0)
	2278	+ return ERR_PTR(err);
	2279	+
	2280	+ err = module_enforce_rwx_sections(info->hdr, info->sechdrs,
	2281	+ info->secstrings, info->mod);
	2282	+ if (err < 0)
	2283	+ return ERR_PTR(err);
	2284	+
	2285	+ /* We will do a special allocation for per-cpu sections later. */
	2286	+ info->sechdrs[info->index.pcpu].sh_flags &= ~(unsigned long)SHF_ALLOC;
	2287	+
	2288	+ /*
	2289	+ * Mark ro_after_init section with SHF_RO_AFTER_INIT so that
	2290	+ * layout_sections() can put it in the right place.
	2291	+ * Note: ro_after_init sections also have SHF_{WRITE,ALLOC} set.
	2292	+ */
	2293	+ ndx = find_sec(info, ".data..ro_after_init");
	2294	+ if (ndx)
	2295	+ info->sechdrs[ndx].sh_flags \|= SHF_RO_AFTER_INIT;
	2296	+ /*
	2297	+ * Mark the __jump_table section as ro_after_init as well: these data
	2298	+ * structures are never modified, with the exception of entries that
	2299	+ * refer to code in the __init section, which are annotated as such
	2300	+ * at module load time.
	2301	+ */
	2302	+ ndx = find_sec(info, "__jump_table");
	2303	+ if (ndx)
	2304	+ info->sechdrs[ndx].sh_flags \|= SHF_RO_AFTER_INIT;
	2305	+
	2306	+ /*
	2307	+ * Determine total sizes, and put offsets in sh_entsize. For now
	2308	+ * this is done generically; there doesn't appear to be any
	2309	+ * special cases for the architectures.
	2310	+ */
	2311	+ layout_sections(info->mod, info);
	2312	+ layout_symtab(info->mod, info);
	2313	+
	2314	+ /* Allocate and move to the final place */
	2315	+ err = move_module(info->mod, info);
	2316	+ if (err)
	2317	+ return ERR_PTR(err);
	2318	+
	2319	+ /* Module has been copied to its final place now: return it. */
	2320	+ mod = (void *)info->sechdrs[info->index.mod].sh_addr;
	2321	+ kmemleak_load_module(mod, info);
	2322	+ return mod;
	2323	+}
	2324	+
	2325	+/* mod is no longer valid after this! */
	2326	+static void module_deallocate(struct module mod, struct load_info info)
	2327	+{
	2328	+ percpu_modfree(mod);
	2329	+ module_arch_freeing_init(mod);
	2330	+ module_memfree(mod->init_layout.base);
	2331	+ module_memfree(mod->core_layout.base);
	2332	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	2333	+ vfree(mod->data_layout.base);
	2334	+#endif
	2335	+}
	2336	+
	2337	+int __weak module_finalize(const Elf_Ehdr *hdr,
	2338	+ const Elf_Shdr *sechdrs,
	2339	+ struct module *me)
	2340	+{
	2341	+ return 0;
	2342	+}
	2343	+
	2344	+static int post_relocation(struct module mod, const struct load_info info)
	2345	+{
	2346	+ /* Sort exception table now relocations are done. */
	2347	+ sort_extable(mod->extable, mod->extable + mod->num_exentries);
	2348	+
	2349	+ /* Copy relocated percpu area over. */
	2350	+ percpu_modcopy(mod, (void *)info->sechdrs[info->index.pcpu].sh_addr,
	2351	+ info->sechdrs[info->index.pcpu].sh_size);
	2352	+
	2353	+ /* Setup kallsyms-specific fields. */
	2354	+ add_kallsyms(mod, info);
	2355	+
	2356	+ /* Arch-specific module finalizing. */
	2357	+ return module_finalize(info->hdr, info->sechdrs, mod);
	2358	+}
	2359	+
	2360	+/* Is this module of this name done loading? No locks held. */
	2361	+static bool finished_loading(const char *name)
	2362	+{
	2363	+ struct module *mod;
	2364	+ bool ret;
	2365	+
	2366	+ /*
	2367	+ * The module_mutex should not be a heavily contended lock;
	2368	+ * if we get the occasional sleep here, we'll go an extra iteration
	2369	+ * in the wait_event_interruptible(), which is harmless.
	2370	+ */
	2371	+ sched_annotate_sleep();
	2372	+ mutex_lock(&module_mutex);
	2373	+ mod = find_module_all(name, strlen(name), true);
	2374	+ ret = !mod \|\| mod->state == MODULE_STATE_LIVE;
	2375	+ mutex_unlock(&module_mutex);
	2376	+
	2377	+ return ret;
	2378	+}
	2379	+
	2380	+/* Call module constructors. */
	2381	+static void do_mod_ctors(struct module *mod)
	2382	+{
	2383	+#ifdef CONFIG_CONSTRUCTORS
	2384	+ unsigned long i;
	2385	+
	2386	+ for (i = 0; i < mod->num_ctors; i++)
	2387	+ mod->ctors[i]();
	2388	+#endif
	2389	+}
	2390	+
	2391	+/* For freeing module_init on success, in case kallsyms traversing */
	2392	+struct mod_initfree {
	2393	+ struct llist_node node;
	2394	+ void *module_init;
	2395	+};
	2396	+
	2397	+static void do_free_init(struct work_struct *w)
	2398	+{
	2399	+ struct llist_node pos, n, *list;
	2400	+ struct mod_initfree *initfree;
	2401	+
	2402	+ list = llist_del_all(&init_free_list);
	2403	+
	2404	+ synchronize_rcu();
	2405	+
	2406	+ llist_for_each_safe(pos, n, list) {
	2407	+ initfree = container_of(pos, struct mod_initfree, node);
	2408	+ module_memfree(initfree->module_init);
	2409	+ kfree(initfree);
	2410	+ }
	2411	+}
	2412	+
	2413	+/*
	2414	+ * This is where the real work happens.
	2415	+ *
	2416	+ * Keep it uninlined to provide a reliable breakpoint target, e.g. for the gdb
	2417	+ * helper command 'lx-symbols'.
	2418	+ */
	2419	+static noinline int do_init_module(struct module *mod)
	2420	+{
	2421	+ int ret = 0;
	2422	+ struct mod_initfree *freeinit;
	2423	+
	2424	+ freeinit = kmalloc(sizeof(*freeinit), GFP_KERNEL);
	2425	+ if (!freeinit) {
	2426	+ ret = -ENOMEM;
	2427	+ goto fail;
	2428	+ }
	2429	+ freeinit->module_init = mod->init_layout.base;
	2430	+
	2431	+ do_mod_ctors(mod);
	2432	+ /* Start the module */
	2433	+ if (mod->init != NULL)
	2434	+ ret = do_one_initcall(mod->init);
	2435	+ if (ret < 0) {
	2436	+ goto fail_free_freeinit;
	2437	+ }
	2438	+ if (ret > 0) {
	2439	+ pr_warn("%s: '%s'->init suspiciously returned %d, it should "
	2440	+ "follow 0/-E convention\n"
	2441	+ "%s: loading module anyway...\n",
	2442	+ __func__, mod->name, ret, __func__);
	2443	+ dump_stack();
	2444	+ }
	2445	+
	2446	+ /* Now it's a first class citizen! */
	2447	+ mod->state = MODULE_STATE_LIVE;
	2448	+ blocking_notifier_call_chain(&module_notify_list,
	2449	+ MODULE_STATE_LIVE, mod);
	2450	+
	2451	+ /* Delay uevent until module has finished its init routine */
	2452	+ kobject_uevent(&mod->mkobj.kobj, KOBJ_ADD);
	2453	+
	2454	+ /*
	2455	+ * We need to finish all async code before the module init sequence
	2456	+ * is done. This has potential to deadlock if synchronous module
	2457	+ * loading is requested from async (which is not allowed!).
	2458	+ *
	2459	+ * See commit 0fdff3ec6d87 ("async, kmod: warn on synchronous
	2460	+ * request_module() from async workers") for more details.
	2461	+ */
	2462	+ if (!mod->async_probe_requested)
	2463	+ async_synchronize_full();
	2464	+
	2465	+ ftrace_free_mem(mod, mod->init_layout.base, mod->init_layout.base +
	2466	+ mod->init_layout.size);
	2467	+ mutex_lock(&module_mutex);
	2468	+ /* Drop initial reference. */
	2469	+ module_put(mod);
	2470	+ trim_init_extable(mod);
	2471	+#ifdef CONFIG_KALLSYMS
	2472	+ /* Switch to core kallsyms now init is done: kallsyms may be walking! */
	2473	+ rcu_assign_pointer(mod->kallsyms, &mod->core_kallsyms);
	2474	+#endif
	2475	+ module_enable_ro(mod, true);
	2476	+ mod_tree_remove_init(mod);
	2477	+ module_arch_freeing_init(mod);
	2478	+ mod->init_layout.base = NULL;
	2479	+ mod->init_layout.size = 0;
	2480	+ mod->init_layout.ro_size = 0;
	2481	+ mod->init_layout.ro_after_init_size = 0;
	2482	+ mod->init_layout.text_size = 0;
	2483	+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
	2484	+ /* .BTF is not SHF_ALLOC and will get removed, so sanitize pointer */
	2485	+ mod->btf_data = NULL;
	2486	+#endif
	2487	+ /*
	2488	+ * We want to free module_init, but be aware that kallsyms may be
	2489	+ * walking this with preempt disabled. In all the failure paths, we
	2490	+ * call synchronize_rcu(), but we don't want to slow down the success
	2491	+ * path. module_memfree() cannot be called in an interrupt, so do the
	2492	+ * work and call synchronize_rcu() in a work queue.
	2493	+ *
	2494	+ * Note that module_alloc() on most architectures creates W+X page
	2495	+ * mappings which won't be cleaned up until do_free_init() runs. Any
	2496	+ * code such as mark_rodata_ro() which depends on those mappings to
	2497	+ * be cleaned up needs to sync with the queued work - ie
	2498	+ * rcu_barrier()
	2499	+ */
	2500	+ if (llist_add(&freeinit->node, &init_free_list))
	2501	+ schedule_work(&init_free_wq);
	2502	+
	2503	+ mutex_unlock(&module_mutex);
	2504	+ wake_up_all(&module_wq);
	2505	+
	2506	+ return 0;
	2507	+
	2508	+fail_free_freeinit:
	2509	+ kfree(freeinit);
	2510	+fail:
	2511	+ /* Try to protect us from buggy refcounters. */
	2512	+ mod->state = MODULE_STATE_GOING;
	2513	+ synchronize_rcu();
	2514	+ module_put(mod);
	2515	+ blocking_notifier_call_chain(&module_notify_list,
	2516	+ MODULE_STATE_GOING, mod);
	2517	+ klp_module_going(mod);
	2518	+ ftrace_release_mod(mod);
	2519	+ free_module(mod);
	2520	+ wake_up_all(&module_wq);
	2521	+ return ret;
	2522	+}
	2523	+
	2524	+static int may_init_module(void)
	2525	+{
	2526	+ if (!capable(CAP_SYS_MODULE) \|\| modules_disabled)
	2527	+ return -EPERM;
	2528	+
	2529	+ return 0;
	2530	+}
	2531	+
	2532	+/*
	2533	+ * We try to place it in the list now to make sure it's unique before
	2534	+ * we dedicate too many resources. In particular, temporary percpu
	2535	+ * memory exhaustion.
	2536	+ */
	2537	+static int add_unformed_module(struct module *mod)
	2538	+{
	2539	+ int err;
	2540	+ struct module *old;
	2541	+
	2542	+ mod->state = MODULE_STATE_UNFORMED;
	2543	+
	2544	+again:
	2545	+ mutex_lock(&module_mutex);
	2546	+ old = find_module_all(mod->name, strlen(mod->name), true);
	2547	+ if (old != NULL) {
	2548	+ if (old->state != MODULE_STATE_LIVE) {
	2549	+ /* Wait in case it fails to load. */
	2550	+ mutex_unlock(&module_mutex);
	2551	+ err = wait_event_interruptible(module_wq,
	2552	+ finished_loading(mod->name));
	2553	+ if (err)
	2554	+ goto out_unlocked;
	2555	+ goto again;
	2556	+ }
	2557	+ err = -EEXIST;
	2558	+ goto out;
	2559	+ }
	2560	+ mod_update_bounds(mod);
	2561	+ list_add_rcu(&mod->list, &modules);
	2562	+ mod_tree_insert(mod);
	2563	+ err = 0;
	2564	+
	2565	+out:
	2566	+ mutex_unlock(&module_mutex);
	2567	+out_unlocked:
	2568	+ return err;
	2569	+}
	2570	+
	2571	+static int complete_formation(struct module mod, struct load_info info)
	2572	+{
	2573	+ int err;
	2574	+
	2575	+ mutex_lock(&module_mutex);
	2576	+
	2577	+ /* Find duplicate symbols (must be called under lock). */
	2578	+ err = verify_exported_symbols(mod);
	2579	+ if (err < 0)
	2580	+ goto out;
	2581	+
	2582	+ /* This relies on module_mutex for list integrity. */
	2583	+ module_bug_finalize(info->hdr, info->sechdrs, mod);
	2584	+
	2585	+ if (module_check_misalignment(mod))
	2586	+ goto out_misaligned;
	2587	+
	2588	+ module_enable_ro(mod, false);
	2589	+ module_enable_nx(mod);
	2590	+ module_enable_x(mod);
	2591	+
	2592	+ /*
	2593	+ * Mark state as coming so strong_try_module_get() ignores us,
	2594	+ * but kallsyms etc. can see us.
	2595	+ */
	2596	+ mod->state = MODULE_STATE_COMING;
	2597	+ mutex_unlock(&module_mutex);
	2598	+
	2599	+ return 0;
	2600	+
	2601	+out_misaligned:
	2602	+ err = -EINVAL;
	2603	+out:
	2604	+ mutex_unlock(&module_mutex);
	2605	+ return err;
	2606	+}
	2607	+
	2608	+static int prepare_coming_module(struct module *mod)
	2609	+{
	2610	+ int err;
	2611	+
	2612	+ ftrace_module_enable(mod);
	2613	+ err = klp_module_coming(mod);
	2614	+ if (err)
	2615	+ return err;
	2616	+
	2617	+ err = blocking_notifier_call_chain_robust(&module_notify_list,
	2618	+ MODULE_STATE_COMING, MODULE_STATE_GOING, mod);
	2619	+ err = notifier_to_errno(err);
	2620	+ if (err)
	2621	+ klp_module_going(mod);
	2622	+
	2623	+ return err;
	2624	+}
	2625	+
	2626	+static int unknown_module_param_cb(char param, char val, const char *modname,
	2627	+ void *arg)
	2628	+{
	2629	+ struct module *mod = arg;
	2630	+ int ret;
	2631	+
	2632	+ if (strcmp(param, "async_probe") == 0) {
	2633	+ mod->async_probe_requested = true;
	2634	+ return 0;
	2635	+ }
	2636	+
	2637	+ /* Check for magic 'dyndbg' arg */
	2638	+ ret = ddebug_dyndbg_module_param_cb(param, val, modname);
	2639	+ if (ret != 0)
	2640	+ pr_warn("%s: unknown parameter '%s' ignored\n", modname, param);
	2641	+ return 0;
	2642	+}
	2643	+
	2644	+static void cfi_init(struct module *mod);
	2645	+
	2646	+/*
	2647	+ * Allocate and load the module: note that size of section 0 is always
	2648	+ * zero, and we rely on this for optional sections.
	2649	+ */
	2650	+static int load_module(struct load_info info, const char __user uargs,
	2651	+ int flags)
	2652	+{
	2653	+ struct module *mod;
	2654	+ long err = 0;
	2655	+ char *after_dashes;
	2656	+
	2657	+ /*
	2658	+ * Do the signature check (if any) first. All that
	2659	+ * the signature check needs is info->len, it does
	2660	+ * not need any of the section info. That can be
	2661	+ * set up later. This will minimize the chances
	2662	+ * of a corrupt module causing problems before
	2663	+ * we even get to the signature check.
	2664	+ *
	2665	+ * The check will also adjust info->len by stripping
	2666	+ * off the sig length at the end of the module, making
	2667	+ * checks against info->len more correct.
	2668	+ */
	2669	+ err = module_sig_check(info, flags);
	2670	+ if (err)
	2671	+ goto free_copy;
	2672	+
	2673	+ /*
	2674	+ * Do basic sanity checks against the ELF header and
	2675	+ * sections.
	2676	+ */
	2677	+ err = elf_validity_check(info);
	2678	+ if (err)
	2679	+ goto free_copy;
	2680	+
	2681	+ /*
	2682	+ * Everything checks out, so set up the section info
	2683	+ * in the info structure.
	2684	+ */
	2685	+ err = setup_load_info(info, flags);
	2686	+ if (err)
	2687	+ goto free_copy;
	2688	+
	2689	+ /*
	2690	+ * Now that we know we have the correct module name, check
	2691	+ * if it's blacklisted.
	2692	+ */
	2693	+ if (blacklisted(info->name)) {
	2694	+ err = -EPERM;
	2695	+ pr_err("Module %s is blacklisted\n", info->name);
	2696	+ goto free_copy;
	2697	+ }
	2698	+
	2699	+ err = rewrite_section_headers(info, flags);
	2700	+ if (err)
	2701	+ goto free_copy;
	2702	+
	2703	+ /* Check module struct version now, before we try to use module. */
	2704	+ if (!check_modstruct_version(info, info->mod)) {
	2705	+ err = -ENOEXEC;
	2706	+ goto free_copy;
	2707	+ }
	2708	+
	2709	+ /* Figure out module layout, and allocate all the memory. */
	2710	+ mod = layout_and_allocate(info, flags);
	2711	+ if (IS_ERR(mod)) {
	2712	+ err = PTR_ERR(mod);
	2713	+ goto free_copy;
	2714	+ }
	2715	+
	2716	+ audit_log_kern_module(mod->name);
	2717	+
	2718	+ /* Reserve our place in the list. */
	2719	+ err = add_unformed_module(mod);
	2720	+ if (err)
	2721	+ goto free_module;
	2722	+
	2723	+#ifdef CONFIG_MODULE_SIG
	2724	+ mod->sig_ok = info->sig_ok;
	2725	+ if (!mod->sig_ok) {
	2726	+ pr_notice_once("%s: module verification failed: signature "
	2727	+ "and/or required key missing - tainting "
	2728	+ "kernel\n", mod->name);
	2729	+ add_taint_module(mod, TAINT_UNSIGNED_MODULE, LOCKDEP_STILL_OK);
	2730	+ }
	2731	+#endif
	2732	+
	2733	+ /* To avoid stressing percpu allocator, do this once we're unique. */
	2734	+ err = percpu_modalloc(mod, info);
	2735	+ if (err)
	2736	+ goto unlink_mod;
	2737	+
	2738	+ /* Now module is in final location, initialize linked lists, etc. */
	2739	+ err = module_unload_init(mod);
	2740	+ if (err)
	2741	+ goto unlink_mod;
	2742	+
	2743	+ init_param_lock(mod);
	2744	+
	2745	+ /*
	2746	+ * Now we've got everything in the final locations, we can
	2747	+ * find optional sections.
	2748	+ */
	2749	+ err = find_module_sections(mod, info);
	2750	+ if (err)
	2751	+ goto free_unload;
	2752	+
	2753	+ err = check_module_license_and_versions(mod);
	2754	+ if (err)
	2755	+ goto free_unload;
	2756	+
	2757	+ /* Set up MODINFO_ATTR fields */
	2758	+ setup_modinfo(mod, info);
	2759	+
	2760	+ /* Fix up syms, so that st_value is a pointer to location. */
	2761	+ err = simplify_symbols(mod, info);
	2762	+ if (err < 0)
	2763	+ goto free_modinfo;
	2764	+
	2765	+ err = apply_relocations(mod, info);
	2766	+ if (err < 0)
	2767	+ goto free_modinfo;
	2768	+
	2769	+ err = post_relocation(mod, info);
	2770	+ if (err < 0)
	2771	+ goto free_modinfo;
	2772	+
	2773	+ flush_module_icache(mod);
	2774	+
	2775	+ /* Setup CFI for the module. */
	2776	+ cfi_init(mod);
	2777	+
	2778	+ /* Now copy in args */
	2779	+ mod->args = strndup_user(uargs, ~0UL >> 1);
	2780	+ if (IS_ERR(mod->args)) {
	2781	+ err = PTR_ERR(mod->args);
	2782	+ goto free_arch_cleanup;
	2783	+ }
	2784	+
	2785	+ init_build_id(mod, info);
	2786	+ dynamic_debug_setup(mod, info->debug, info->num_debug);
	2787	+
	2788	+ /* Ftrace init must be called in the MODULE_STATE_UNFORMED state */
	2789	+ ftrace_module_init(mod);
	2790	+
	2791	+ /* Finally it's fully formed, ready to start executing. */
	2792	+ err = complete_formation(mod, info);
	2793	+ if (err)
	2794	+ goto ddebug_cleanup;
	2795	+
	2796	+ err = prepare_coming_module(mod);
	2797	+ if (err)
	2798	+ goto bug_cleanup;
	2799	+
	2800	+ /* Module is ready to execute: parsing args may do that. */
	2801	+ after_dashes = parse_args(mod->name, mod->args, mod->kp, mod->num_kp,
	2802	+ -32768, 32767, mod,
	2803	+ unknown_module_param_cb);
	2804	+ if (IS_ERR(after_dashes)) {
	2805	+ err = PTR_ERR(after_dashes);
	2806	+ goto coming_cleanup;
	2807	+ } else if (after_dashes) {
	2808	+ pr_warn("%s: parameters '%s' after `--' ignored\n",
	2809	+ mod->name, after_dashes);
	2810	+ }
	2811	+
	2812	+ /* Link in to sysfs. */
	2813	+ err = mod_sysfs_setup(mod, info, mod->kp, mod->num_kp);
	2814	+ if (err < 0)
	2815	+ goto coming_cleanup;
	2816	+
	2817	+ if (is_livepatch_module(mod)) {
	2818	+ err = copy_module_elf(mod, info);
	2819	+ if (err < 0)
	2820	+ goto sysfs_cleanup;
	2821	+ }
	2822	+
	2823	+ /* Get rid of temporary copy. */
	2824	+ free_copy(info, flags);
	2825	+
	2826	+ /* Done! */
	2827	+ trace_module_load(mod);
	2828	+
	2829	+ return do_init_module(mod);
	2830	+
	2831	+ sysfs_cleanup:
	2832	+ mod_sysfs_teardown(mod);
	2833	+ coming_cleanup:
	2834	+ mod->state = MODULE_STATE_GOING;
	2835	+ destroy_params(mod->kp, mod->num_kp);
	2836	+ blocking_notifier_call_chain(&module_notify_list,
	2837	+ MODULE_STATE_GOING, mod);
	2838	+ klp_module_going(mod);
	2839	+ bug_cleanup:
	2840	+ mod->state = MODULE_STATE_GOING;
	2841	+ /* module_bug_cleanup needs module_mutex protection */
	2842	+ mutex_lock(&module_mutex);
	2843	+ module_bug_cleanup(mod);
	2844	+ mutex_unlock(&module_mutex);
	2845	+
	2846	+ ddebug_cleanup:
	2847	+ ftrace_release_mod(mod);
	2848	+ dynamic_debug_remove(mod, info->debug);
	2849	+ synchronize_rcu();
	2850	+ kfree(mod->args);
	2851	+ free_arch_cleanup:
	2852	+ cfi_cleanup(mod);
	2853	+ module_arch_cleanup(mod);
	2854	+ free_modinfo:
	2855	+ free_modinfo(mod);
	2856	+ free_unload:
	2857	+ module_unload_free(mod);
	2858	+ unlink_mod:
	2859	+ mutex_lock(&module_mutex);
	2860	+ /* Unlink carefully: kallsyms could be walking list. */
	2861	+ list_del_rcu(&mod->list);
	2862	+ mod_tree_remove(mod);
	2863	+ wake_up_all(&module_wq);
	2864	+ /* Wait for RCU-sched synchronizing before releasing mod->list. */
	2865	+ synchronize_rcu();
	2866	+ mutex_unlock(&module_mutex);
	2867	+ free_module:
	2868	+ /* Free lock-classes; relies on the preceding sync_rcu() */
	2869	+ lockdep_free_key_range(mod->data_layout.base, mod->data_layout.size);
	2870	+
	2871	+ module_deallocate(mod, info);
	2872	+ free_copy:
	2873	+ free_copy(info, flags);
	2874	+ return err;
	2875	+}
	2876	+
	2877	+SYSCALL_DEFINE3(init_module, void __user *, umod,
	2878	+ unsigned long, len, const char __user *, uargs)
	2879	+{
	2880	+ int err;
	2881	+ struct load_info info = { };
	2882	+
	2883	+ err = may_init_module();
	2884	+ if (err)
	2885	+ return err;
	2886	+
	2887	+ pr_debug("init_module: umod=%p, len=%lu, uargs=%p\n",
	2888	+ umod, len, uargs);
	2889	+
	2890	+ err = copy_module_from_user(umod, len, &info);
	2891	+ if (err)
	2892	+ return err;
	2893	+
	2894	+ return load_module(&info, uargs, 0);
	2895	+}
	2896	+
	2897	+SYSCALL_DEFINE3(finit_module, int, fd, const char __user *, uargs, int, flags)
	2898	+{
	2899	+ struct load_info info = { };
	2900	+ void *buf = NULL;
	2901	+ int len;
	2902	+ int err;
	2903	+
	2904	+ err = may_init_module();
	2905	+ if (err)
	2906	+ return err;
	2907	+
	2908	+ pr_debug("finit_module: fd=%d, uargs=%p, flags=%i\n", fd, uargs, flags);
	2909	+
	2910	+ if (flags & ~(MODULE_INIT_IGNORE_MODVERSIONS
	2911	+ \|MODULE_INIT_IGNORE_VERMAGIC
	2912	+ \|MODULE_INIT_COMPRESSED_FILE))
	2913	+ return -EINVAL;
	2914	+
	2915	+ len = kernel_read_file_from_fd(fd, 0, &buf, INT_MAX, NULL,
	2916	+ READING_MODULE);
	2917	+ if (len < 0)
	2918	+ return len;
	2919	+
	2920	+ if (flags & MODULE_INIT_COMPRESSED_FILE) {
	2921	+ err = module_decompress(&info, buf, len);
	2922	+ vfree(buf); /* compressed data is no longer needed */
	2923	+ if (err)
	2924	+ return err;
	2925	+ } else {
	2926	+ info.hdr = buf;
	2927	+ info.len = len;
	2928	+ }
	2929	+
	2930	+ return load_module(&info, uargs, flags);
	2931	+}
	2932	+
	2933	+static inline int within(unsigned long addr, void *start, unsigned long size)
	2934	+{
	2935	+ return ((void )addr >= start && (void )addr < start + size);
	2936	+}
	2937	+
	2938	+static void cfi_init(struct module *mod)
	2939	+{
	2940	+#ifdef CONFIG_CFI_CLANG
	2941	+ initcall_t *init;
	2942	+ exitcall_t *exit;
	2943	+
	2944	+ rcu_read_lock_sched();
	2945	+ mod->cfi_check = (cfi_check_fn)
	2946	+ find_kallsyms_symbol_value(mod, "__cfi_check");
	2947	+ init = (initcall_t *)
	2948	+ find_kallsyms_symbol_value(mod, "__cfi_jt_init_module");
	2949	+ exit = (exitcall_t *)
	2950	+ find_kallsyms_symbol_value(mod, "__cfi_jt_cleanup_module");
	2951	+ rcu_read_unlock_sched();
	2952	+
	2953	+ /* Fix init/exit functions to point to the CFI jump table */
	2954	+ if (init)
	2955	+ mod->init = *init;
	2956	+#ifdef CONFIG_MODULE_UNLOAD
	2957	+ if (exit)
	2958	+ mod->exit = *exit;
	2959	+#endif
	2960	+
	2961	+ cfi_module_add(mod, mod_tree.addr_min);
	2962	+#endif
	2963	+}
	2964	+
	2965	+static void cfi_cleanup(struct module *mod)
	2966	+{
	2967	+#ifdef CONFIG_CFI_CLANG
	2968	+ cfi_module_remove(mod, mod_tree.addr_min);
	2969	+#endif
	2970	+}
	2971	+
	2972	+/* Keep in sync with MODULE_FLAGS_BUF_SIZE !!! */
	2973	+char module_flags(struct module mod, char *buf)
	2974	+{
	2975	+ int bx = 0;
	2976	+
	2977	+ BUG_ON(mod->state == MODULE_STATE_UNFORMED);
	2978	+ if (mod->taints \|\|
	2979	+ mod->state == MODULE_STATE_GOING \|\|
	2980	+ mod->state == MODULE_STATE_COMING) {
	2981	+ buf[bx++] = '(';
	2982	+ bx += module_flags_taint(mod->taints, buf + bx);
	2983	+ /* Show a - for module-is-being-unloaded */
	2984	+ if (mod->state == MODULE_STATE_GOING)
	2985	+ buf[bx++] = '-';
	2986	+ /* Show a + for module-is-being-loaded */
	2987	+ if (mod->state == MODULE_STATE_COMING)
	2988	+ buf[bx++] = '+';
	2989	+ buf[bx++] = ')';
	2990	+ }
	2991	+ buf[bx] = '\0';
	2992	+
	2993	+ return buf;
	2994	+}
	2995	+
	2996	+/* Given an address, look for it in the module exception tables. */
	2997	+const struct exception_table_entry *search_module_extables(unsigned long addr)
	2998	+{
	2999	+ const struct exception_table_entry *e = NULL;
	3000	+ struct module *mod;
	3001	+
	3002	+ preempt_disable();
	3003	+ mod = __module_address(addr);
	3004	+ if (!mod)
	3005	+ goto out;
	3006	+
	3007	+ if (!mod->num_exentries)
	3008	+ goto out;
	3009	+
	3010	+ e = search_extable(mod->extable,
	3011	+ mod->num_exentries,
	3012	+ addr);
	3013	+out:
	3014	+ preempt_enable();
	3015	+
	3016	+ /*
	3017	+ * Now, if we found one, we are running inside it now, hence
	3018	+ * we cannot unload the module, hence no refcnt needed.
	3019	+ */
	3020	+ return e;
	3021	+}
	3022	+
	3023	+/**
	3024	+ * is_module_address() - is this address inside a module?
	3025	+ * @addr: the address to check.
	3026	+ *
	3027	+ * See is_module_text_address() if you simply want to see if the address
	3028	+ * is code (not data).
	3029	+ */
	3030	+bool is_module_address(unsigned long addr)
	3031	+{
	3032	+ bool ret;
	3033	+
	3034	+ preempt_disable();
	3035	+ ret = __module_address(addr) != NULL;
	3036	+ preempt_enable();
	3037	+
	3038	+ return ret;
	3039	+}
	3040	+
	3041	+/**
	3042	+ * __module_address() - get the module which contains an address.
	3043	+ * @addr: the address.
	3044	+ *
	3045	+ * Must be called with preempt disabled or module mutex held so that
	3046	+ * module doesn't get freed during this.
	3047	+ */
	3048	+struct module *__module_address(unsigned long addr)
	3049	+{
	3050	+ struct module *mod;
	3051	+ struct mod_tree_root *tree;
	3052	+
	3053	+ if (addr >= mod_tree.addr_min && addr <= mod_tree.addr_max)
	3054	+ tree = &mod_tree;
	3055	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	3056	+ else if (addr >= mod_data_tree.addr_min && addr <= mod_data_tree.addr_max)
	3057	+ tree = &mod_data_tree;
	3058	+#endif
	3059	+ else
	3060	+ return NULL;
	3061	+
	3062	+ module_assert_mutex_or_preempt();
	3063	+
	3064	+ mod = mod_find(addr, tree);
	3065	+ if (mod) {
	3066	+ BUG_ON(!within_module(addr, mod));
	3067	+ if (mod->state == MODULE_STATE_UNFORMED)
	3068	+ mod = NULL;
	3069	+ }
	3070	+ return mod;
	3071	+}
	3072	+
	3073	+/**
	3074	+ * is_module_text_address() - is this address inside module code?
	3075	+ * @addr: the address to check.
	3076	+ *
	3077	+ * See is_module_address() if you simply want to see if the address is
	3078	+ * anywhere in a module. See kernel_text_address() for testing if an
	3079	+ * address corresponds to kernel or module code.
	3080	+ */
	3081	+bool is_module_text_address(unsigned long addr)
	3082	+{
	3083	+ bool ret;
	3084	+
	3085	+ preempt_disable();
	3086	+ ret = __module_text_address(addr) != NULL;
	3087	+ preempt_enable();
	3088	+
	3089	+ return ret;
	3090	+}
	3091	+
	3092	+/**
	3093	+ * __module_text_address() - get the module whose code contains an address.
	3094	+ * @addr: the address.
	3095	+ *
	3096	+ * Must be called with preempt disabled or module mutex held so that
	3097	+ * module doesn't get freed during this.
	3098	+ */
	3099	+struct module *__module_text_address(unsigned long addr)
	3100	+{
	3101	+ struct module *mod = __module_address(addr);
	3102	+ if (mod) {
	3103	+ /* Make sure it's within the text section. */
	3104	+ if (!within(addr, mod->init_layout.base, mod->init_layout.text_size)
	3105	+ && !within(addr, mod->core_layout.base, mod->core_layout.text_size))
	3106	+ mod = NULL;
	3107	+ }
	3108	+ return mod;
	3109	+}
	3110	+
	3111	+/* Don't grab lock, we're oopsing. */
	3112	+void print_modules(void)
	3113	+{
	3114	+ struct module *mod;
	3115	+ char buf[MODULE_FLAGS_BUF_SIZE];
	3116	+
	3117	+ printk(KERN_DEFAULT "Modules linked in:");
	3118	+ /* Most callers should already have preempt disabled, but make sure */
	3119	+ preempt_disable();
	3120	+ list_for_each_entry_rcu(mod, &modules, list) {
	3121	+ if (mod->state == MODULE_STATE_UNFORMED)
	3122	+ continue;
	3123	+ pr_cont(" %s%s", mod->name, module_flags(mod, buf));
	3124	+ }
	3125	+
	3126	+ print_unloaded_tainted_modules();
	3127	+ preempt_enable();
	3128	+ if (last_unloaded_module[0])
	3129	+ pr_cont(" [last unloaded: %s]", last_unloaded_module);
	3130	+ pr_cont("\n");
	3131	+}

kernel/module/procfs.c+146−0

@@ -0,0 +1,146 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module proc support
	4	+ *
	5	+ * Copyright (C) 2008 Alexey Dobriyan
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/kallsyms.h>
	10	+#include <linux/mutex.h>
	11	+#include <linux/seq_file.h>
	12	+#include <linux/proc_fs.h>
	13	+#include "internal.h"
	14	+
	15	+#ifdef CONFIG_MODULE_UNLOAD
	16	+static inline void print_unload_info(struct seq_file m, struct module mod)
	17	+{
	18	+ struct module_use *use;
	19	+ int printed_something = 0;
	20	+
	21	+ seq_printf(m, " %i ", module_refcount(mod));
	22	+
	23	+ /*
	24	+ * Always include a trailing , so userspace can differentiate
	25	+ * between this and the old multi-field proc format.
	26	+ */
	27	+ list_for_each_entry(use, &mod->source_list, source_list) {
	28	+ printed_something = 1;
	29	+ seq_printf(m, "%s,", use->source->name);
	30	+ }
	31	+
	32	+ if (mod->init && !mod->exit) {
	33	+ printed_something = 1;
	34	+ seq_puts(m, "[permanent],");
	35	+ }
	36	+
	37	+ if (!printed_something)
	38	+ seq_puts(m, "-");
	39	+}
	40	+#else /* !CONFIG_MODULE_UNLOAD */
	41	+static inline void print_unload_info(struct seq_file m, struct module mod)
	42	+{
	43	+ /* We don't know the usage count, or what modules are using. */
	44	+ seq_puts(m, " - -");
	45	+}
	46	+#endif /* CONFIG_MODULE_UNLOAD */
	47	+
	48	+/* Called by the /proc file system to return a list of modules. */
	49	+static void m_start(struct seq_file m, loff_t *pos)
	50	+{
	51	+ mutex_lock(&module_mutex);
	52	+ return seq_list_start(&modules, *pos);
	53	+}
	54	+
	55	+static void m_next(struct seq_file m, void p, loff_t pos)
	56	+{
	57	+ return seq_list_next(p, &modules, pos);
	58	+}
	59	+
	60	+static void m_stop(struct seq_file m, void p)
	61	+{
	62	+ mutex_unlock(&module_mutex);
	63	+}
	64	+
	65	+static int m_show(struct seq_file m, void p)
	66	+{
	67	+ struct module *mod = list_entry(p, struct module, list);
	68	+ char buf[MODULE_FLAGS_BUF_SIZE];
	69	+ void *value;
	70	+ unsigned int size;
	71	+
	72	+ /* We always ignore unformed modules. */
	73	+ if (mod->state == MODULE_STATE_UNFORMED)
	74	+ return 0;
	75	+
	76	+ size = mod->init_layout.size + mod->core_layout.size;
	77	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	78	+ size += mod->data_layout.size;
	79	+#endif
	80	+ seq_printf(m, "%s %u", mod->name, size);
	81	+ print_unload_info(m, mod);
	82	+
	83	+ /* Informative for users. */
	84	+ seq_printf(m, " %s",
	85	+ mod->state == MODULE_STATE_GOING ? "Unloading" :
	86	+ mod->state == MODULE_STATE_COMING ? "Loading" :
	87	+ "Live");
	88	+ /* Used by oprofile and other similar tools. */
	89	+ value = m->private ? NULL : mod->core_layout.base;
	90	+ seq_printf(m, " 0x%px", value);
	91	+
	92	+ /* Taints info */
	93	+ if (mod->taints)
	94	+ seq_printf(m, " %s", module_flags(mod, buf));
	95	+
	96	+ seq_puts(m, "\n");
	97	+ return 0;
	98	+}
	99	+
	100	+/*
	101	+ * Format: modulename size refcount deps address
	102	+ *
	103	+ * Where refcount is a number or -, and deps is a comma-separated list
	104	+ * of depends or -.
	105	+ */
	106	+static const struct seq_operations modules_op = {
	107	+ .start = m_start,
	108	+ .next = m_next,
	109	+ .stop = m_stop,
	110	+ .show = m_show
	111	+};
	112	+
	113	+/*
	114	+ * This also sets the "private" pointer to non-NULL if the
	115	+ * kernel pointers should be hidden (so you can just test
	116	+ * "m->private" to see if you should keep the values private).
	117	+ *
	118	+ * We use the same logic as for /proc/kallsyms.
	119	+ */
	120	+static int modules_open(struct inode inode, struct file file)
	121	+{
	122	+ int err = seq_open(file, &modules_op);
	123	+
	124	+ if (!err) {
	125	+ struct seq_file *m = file->private_data;
	126	+
	127	+ m->private = kallsyms_show_value(file->f_cred) ? NULL : (void *)8ul;
	128	+ }
	129	+
	130	+ return err;
	131	+}
	132	+
	133	+static const struct proc_ops modules_proc_ops = {
	134	+ .proc_flags = PROC_ENTRY_PERMANENT,
	135	+ .proc_open = modules_open,
	136	+ .proc_read = seq_read,
	137	+ .proc_lseek = seq_lseek,
	138	+ .proc_release = seq_release,
	139	+};
	140	+
	141	+static int __init proc_modules_init(void)
	142	+{
	143	+ proc_create("modules", 0, NULL, &modules_proc_ops);
	144	+ return 0;
	145	+}
	146	+module_init(proc_modules_init);

kernel/module/signing.c+122−0

@@ -0,0 +1,122 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/* Module signature checker
	3	+ *
	4	+ * Copyright (C) 2012 Red Hat, Inc. All Rights Reserved.
	5	+ * Written by David Howells (dhowells@redhat.com)
	6	+ */
	7	+
	8	+#include <linux/kernel.h>
	9	+#include <linux/errno.h>
	10	+#include <linux/module.h>
	11	+#include <linux/module_signature.h>
	12	+#include <linux/string.h>
	13	+#include <linux/verification.h>
	14	+#include <linux/security.h>
	15	+#include <crypto/public_key.h>
	16	+#include <uapi/linux/module.h>
	17	+#include "internal.h"
	18	+
	19	+static bool sig_enforce = IS_ENABLED(CONFIG_MODULE_SIG_FORCE);
	20	+module_param(sig_enforce, bool_enable_only, 0644);
	21	+
	22	+/*
	23	+ * Export sig_enforce kernel cmdline parameter to allow other subsystems rely
	24	+ * on that instead of directly to CONFIG_MODULE_SIG_FORCE config.
	25	+ */
	26	+bool is_module_sig_enforced(void)
	27	+{
	28	+ return sig_enforce;
	29	+}
	30	+EXPORT_SYMBOL(is_module_sig_enforced);
	31	+
	32	+void set_module_sig_enforced(void)
	33	+{
	34	+ sig_enforce = true;
	35	+}
	36	+
	37	+/*
	38	+ * Verify the signature on a module.
	39	+ */
	40	+int mod_verify_sig(const void mod, struct load_info info)
	41	+{
	42	+ struct module_signature ms;
	43	+ size_t sig_len, modlen = info->len;
	44	+ int ret;
	45	+
	46	+ pr_devel("==>%s(,%zu)\n", __func__, modlen);
	47	+
	48	+ if (modlen <= sizeof(ms))
	49	+ return -EBADMSG;
	50	+
	51	+ memcpy(&ms, mod + (modlen - sizeof(ms)), sizeof(ms));
	52	+
	53	+ ret = mod_check_sig(&ms, modlen, "module");
	54	+ if (ret)
	55	+ return ret;
	56	+
	57	+ sig_len = be32_to_cpu(ms.sig_len);
	58	+ modlen -= sig_len + sizeof(ms);
	59	+ info->len = modlen;
	60	+
	61	+ return verify_pkcs7_signature(mod, modlen, mod + modlen, sig_len,
	62	+ VERIFY_USE_SECONDARY_KEYRING,
	63	+ VERIFYING_MODULE_SIGNATURE,
	64	+ NULL, NULL);
	65	+}
	66	+
	67	+int module_sig_check(struct load_info *info, int flags)
	68	+{
	69	+ int err = -ENODATA;
	70	+ const unsigned long markerlen = sizeof(MODULE_SIG_STRING) - 1;
	71	+ const char *reason;
	72	+ const void *mod = info->hdr;
	73	+ bool mangled_module = flags & (MODULE_INIT_IGNORE_MODVERSIONS \|
	74	+ MODULE_INIT_IGNORE_VERMAGIC);
	75	+ /*
	76	+ * Do not allow mangled modules as a module with version information
	77	+ * removed is no longer the module that was signed.
	78	+ */
	79	+ if (!mangled_module &&
	80	+ info->len > markerlen &&
	81	+ memcmp(mod + info->len - markerlen, MODULE_SIG_STRING, markerlen) == 0) {
	82	+ /* We truncate the module to discard the signature */
	83	+ info->len -= markerlen;
	84	+ err = mod_verify_sig(mod, info);
	85	+ if (!err) {
	86	+ info->sig_ok = true;
	87	+ return 0;
	88	+ }
	89	+ }
	90	+
	91	+ /*
	92	+ * We don't permit modules to be loaded into the trusted kernels
	93	+ * without a valid signature on them, but if we're not enforcing,
	94	+ * certain errors are non-fatal.
	95	+ */
	96	+ switch (err) {
	97	+ case -ENODATA:
	98	+ reason = "unsigned module";
	99	+ break;
	100	+ case -ENOPKG:
	101	+ reason = "module with unsupported crypto";
	102	+ break;
	103	+ case -ENOKEY:
	104	+ reason = "module with unavailable key";
	105	+ break;
	106	+
	107	+ default:
	108	+ /*
	109	+ * All other errors are fatal, including lack of memory,
	110	+ * unparseable signatures, and signature check failures --
	111	+ * even if signatures aren't required.
	112	+ */
	113	+ return err;
	114	+ }
	115	+
	116	+ if (is_module_sig_enforced()) {
	117	+ pr_notice("Loading of %s is rejected\n", reason);
	118	+ return -EKEYREJECTED;
	119	+ }
	120	+
	121	+ return security_locked_down(LOCKDOWN_MODULE_SIGNATURE);
	122	+}

kernel/module/strict_rwx.c+143−0

@@ -0,0 +1,143 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module strict rwx
	4	+ *
	5	+ * Copyright (C) 2015 Rusty Russell
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/mm.h>
	10	+#include <linux/vmalloc.h>
	11	+#include <linux/set_memory.h>
	12	+#include "internal.h"
	13	+
	14	+/*
	15	+ * LKM RO/NX protection: protect module's text/ro-data
	16	+ * from modification and any data from execution.
	17	+ *
	18	+ * General layout of module is:
	19	+ * [text] [read-only-data] [ro-after-init] [writable data]
	20	+ * text_size -----^ ^ ^ ^
	21	+ * ro_size ------------------------\| \| \|
	22	+ * ro_after_init_size -----------------------------\| \|
	23	+ * size -----------------------------------------------------------\|
	24	+ *
	25	+ * These values are always page-aligned (as is base) when
	26	+ * CONFIG_STRICT_MODULE_RWX is set.
	27	+ */
	28	+
	29	+/*
	30	+ * Since some arches are moving towards PAGE_KERNEL module allocations instead
	31	+ * of PAGE_KERNEL_EXEC, keep frob_text() and module_enable_x() independent of
	32	+ * CONFIG_STRICT_MODULE_RWX because they are needed regardless of whether we
	33	+ * are strict.
	34	+ */
	35	+static void frob_text(const struct module_layout *layout,
	36	+ int (*set_memory)(unsigned long start, int num_pages))
	37	+{
	38	+ set_memory((unsigned long)layout->base,
	39	+ PAGE_ALIGN(layout->text_size) >> PAGE_SHIFT);
	40	+}
	41	+
	42	+static void frob_rodata(const struct module_layout *layout,
	43	+ int (*set_memory)(unsigned long start, int num_pages))
	44	+{
	45	+ set_memory((unsigned long)layout->base + layout->text_size,
	46	+ (layout->ro_size - layout->text_size) >> PAGE_SHIFT);
	47	+}
	48	+
	49	+static void frob_ro_after_init(const struct module_layout *layout,
	50	+ int (*set_memory)(unsigned long start, int num_pages))
	51	+{
	52	+ set_memory((unsigned long)layout->base + layout->ro_size,
	53	+ (layout->ro_after_init_size - layout->ro_size) >> PAGE_SHIFT);
	54	+}
	55	+
	56	+static void frob_writable_data(const struct module_layout *layout,
	57	+ int (*set_memory)(unsigned long start, int num_pages))
	58	+{
	59	+ set_memory((unsigned long)layout->base + layout->ro_after_init_size,
	60	+ (layout->size - layout->ro_after_init_size) >> PAGE_SHIFT);
	61	+}
	62	+
	63	+static bool layout_check_misalignment(const struct module_layout *layout)
	64	+{
	65	+ return WARN_ON(!PAGE_ALIGNED(layout->base)) \|\|
	66	+ WARN_ON(!PAGE_ALIGNED(layout->text_size)) \|\|
	67	+ WARN_ON(!PAGE_ALIGNED(layout->ro_size)) \|\|
	68	+ WARN_ON(!PAGE_ALIGNED(layout->ro_after_init_size)) \|\|
	69	+ WARN_ON(!PAGE_ALIGNED(layout->size));
	70	+}
	71	+
	72	+bool module_check_misalignment(const struct module *mod)
	73	+{
	74	+ if (!IS_ENABLED(CONFIG_STRICT_MODULE_RWX))
	75	+ return false;
	76	+
	77	+ return layout_check_misalignment(&mod->core_layout) \|\|
	78	+ layout_check_misalignment(&mod->data_layout) \|\|
	79	+ layout_check_misalignment(&mod->init_layout);
	80	+}
	81	+
	82	+void module_enable_x(const struct module *mod)
	83	+{
	84	+ if (!PAGE_ALIGNED(mod->core_layout.base) \|\|
	85	+ !PAGE_ALIGNED(mod->init_layout.base))
	86	+ return;
	87	+
	88	+ frob_text(&mod->core_layout, set_memory_x);
	89	+ frob_text(&mod->init_layout, set_memory_x);
	90	+}
	91	+
	92	+void module_enable_ro(const struct module *mod, bool after_init)
	93	+{
	94	+ if (!IS_ENABLED(CONFIG_STRICT_MODULE_RWX))
	95	+ return;
	96	+#ifdef CONFIG_STRICT_MODULE_RWX
	97	+ if (!rodata_enabled)
	98	+ return;
	99	+#endif
	100	+
	101	+ set_vm_flush_reset_perms(mod->core_layout.base);
	102	+ set_vm_flush_reset_perms(mod->init_layout.base);
	103	+ frob_text(&mod->core_layout, set_memory_ro);
	104	+
	105	+ frob_rodata(&mod->data_layout, set_memory_ro);
	106	+ frob_text(&mod->init_layout, set_memory_ro);
	107	+ frob_rodata(&mod->init_layout, set_memory_ro);
	108	+
	109	+ if (after_init)
	110	+ frob_ro_after_init(&mod->data_layout, set_memory_ro);
	111	+}
	112	+
	113	+void module_enable_nx(const struct module *mod)
	114	+{
	115	+ if (!IS_ENABLED(CONFIG_STRICT_MODULE_RWX))
	116	+ return;
	117	+
	118	+ frob_rodata(&mod->data_layout, set_memory_nx);
	119	+ frob_ro_after_init(&mod->data_layout, set_memory_nx);
	120	+ frob_writable_data(&mod->data_layout, set_memory_nx);
	121	+ frob_rodata(&mod->init_layout, set_memory_nx);
	122	+ frob_writable_data(&mod->init_layout, set_memory_nx);
	123	+}
	124	+
	125	+int module_enforce_rwx_sections(Elf_Ehdr hdr, Elf_Shdr sechdrs,
	126	+ char secstrings, struct module mod)
	127	+{
	128	+ const unsigned long shf_wx = SHF_WRITE \| SHF_EXECINSTR;
	129	+ int i;
	130	+
	131	+ if (!IS_ENABLED(CONFIG_STRICT_MODULE_RWX))
	132	+ return 0;
	133	+
	134	+ for (i = 0; i < hdr->e_shnum; i++) {
	135	+ if ((sechdrs[i].sh_flags & shf_wx) == shf_wx) {
	136	+ pr_err("%s: section %s (index %d) has invalid WRITE\|EXEC flags\n",
	137	+ mod->name, secstrings + sechdrs[i].sh_name, i);
	138	+ return -ENOEXEC;
	139	+ }
	140	+ }
	141	+
	142	+ return 0;
	143	+}

kernel/module/sysfs.c+436−0

@@ -0,0 +1,436 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module sysfs support
	4	+ *
	5	+ * Copyright (C) 2008 Rusty Russell
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/kernel.h>
	10	+#include <linux/fs.h>
	11	+#include <linux/sysfs.h>
	12	+#include <linux/slab.h>
	13	+#include <linux/kallsyms.h>
	14	+#include <linux/mutex.h>
	15	+#include "internal.h"
	16	+
	17	+/*
	18	+ * /sys/module/foo/sections stuff
	19	+ * J. Corbet <corbet@lwn.net>
	20	+ */
	21	+#ifdef CONFIG_KALLSYMS
	22	+struct module_sect_attr {
	23	+ struct bin_attribute battr;
	24	+ unsigned long address;
	25	+};
	26	+
	27	+struct module_sect_attrs {
	28	+ struct attribute_group grp;
	29	+ unsigned int nsections;
	30	+ struct module_sect_attr attrs[];
	31	+};
	32	+
	33	+#define MODULE_SECT_READ_SIZE (3 /* "0x", "\n" */ + (BITS_PER_LONG / 4))
	34	+static ssize_t module_sect_read(struct file file, struct kobject kobj,
	35	+ struct bin_attribute *battr,
	36	+ char *buf, loff_t pos, size_t count)
	37	+{
	38	+ struct module_sect_attr *sattr =
	39	+ container_of(battr, struct module_sect_attr, battr);
	40	+ char bounce[MODULE_SECT_READ_SIZE + 1];
	41	+ size_t wrote;
	42	+
	43	+ if (pos != 0)
	44	+ return -EINVAL;
	45	+
	46	+ /*
	47	+ * Since we're a binary read handler, we must account for the
	48	+ * trailing NUL byte that sprintf will write: if "buf" is
	49	+ * too small to hold the NUL, or the NUL is exactly the last
	50	+ * byte, the read will look like it got truncated by one byte.
	51	+ * Since there is no way to ask sprintf nicely to not write
	52	+ * the NUL, we have to use a bounce buffer.
	53	+ */
	54	+ wrote = scnprintf(bounce, sizeof(bounce), "0x%px\n",
	55	+ kallsyms_show_value(file->f_cred)
	56	+ ? (void *)sattr->address : NULL);
	57	+ count = min(count, wrote);
	58	+ memcpy(buf, bounce, count);
	59	+
	60	+ return count;
	61	+}
	62	+
	63	+static void free_sect_attrs(struct module_sect_attrs *sect_attrs)
	64	+{
	65	+ unsigned int section;
	66	+
	67	+ for (section = 0; section < sect_attrs->nsections; section++)
	68	+ kfree(sect_attrs->attrs[section].battr.attr.name);
	69	+ kfree(sect_attrs);
	70	+}
	71	+
	72	+static void add_sect_attrs(struct module mod, const struct load_info info)
	73	+{
	74	+ unsigned int nloaded = 0, i, size[2];
	75	+ struct module_sect_attrs *sect_attrs;
	76	+ struct module_sect_attr *sattr;
	77	+ struct bin_attribute **gattr;
	78	+
	79	+ /* Count loaded sections and allocate structures */
	80	+ for (i = 0; i < info->hdr->e_shnum; i++)
	81	+ if (!sect_empty(&info->sechdrs[i]))
	82	+ nloaded++;
	83	+ size[0] = ALIGN(struct_size(sect_attrs, attrs, nloaded),
	84	+ sizeof(sect_attrs->grp.bin_attrs[0]));
	85	+ size[1] = (nloaded + 1) * sizeof(sect_attrs->grp.bin_attrs[0]);
	86	+ sect_attrs = kzalloc(size[0] + size[1], GFP_KERNEL);
	87	+ if (!sect_attrs)
	88	+ return;
	89	+
	90	+ /* Setup section attributes. */
	91	+ sect_attrs->grp.name = "sections";
	92	+ sect_attrs->grp.bin_attrs = (void *)sect_attrs + size[0];
	93	+
	94	+ sect_attrs->nsections = 0;
	95	+ sattr = &sect_attrs->attrs[0];
	96	+ gattr = &sect_attrs->grp.bin_attrs[0];
	97	+ for (i = 0; i < info->hdr->e_shnum; i++) {
	98	+ Elf_Shdr *sec = &info->sechdrs[i];
	99	+
	100	+ if (sect_empty(sec))
	101	+ continue;
	102	+ sysfs_bin_attr_init(&sattr->battr);
	103	+ sattr->address = sec->sh_addr;
	104	+ sattr->battr.attr.name =
	105	+ kstrdup(info->secstrings + sec->sh_name, GFP_KERNEL);
	106	+ if (!sattr->battr.attr.name)
	107	+ goto out;
	108	+ sect_attrs->nsections++;
	109	+ sattr->battr.read = module_sect_read;
	110	+ sattr->battr.size = MODULE_SECT_READ_SIZE;
	111	+ sattr->battr.attr.mode = 0400;
	112	+ *(gattr++) = &(sattr++)->battr;
	113	+ }
	114	+ *gattr = NULL;
	115	+
	116	+ if (sysfs_create_group(&mod->mkobj.kobj, &sect_attrs->grp))
	117	+ goto out;
	118	+
	119	+ mod->sect_attrs = sect_attrs;
	120	+ return;
	121	+out:
	122	+ free_sect_attrs(sect_attrs);
	123	+}
	124	+
	125	+static void remove_sect_attrs(struct module *mod)
	126	+{
	127	+ if (mod->sect_attrs) {
	128	+ sysfs_remove_group(&mod->mkobj.kobj,
	129	+ &mod->sect_attrs->grp);
	130	+ /*
	131	+ * We are positive that no one is using any sect attrs
	132	+ * at this point. Deallocate immediately.
	133	+ */
	134	+ free_sect_attrs(mod->sect_attrs);
	135	+ mod->sect_attrs = NULL;
	136	+ }
	137	+}
	138	+
	139	+/*
	140	+ * /sys/module/foo/notes/.section.name gives contents of SHT_NOTE sections.
	141	+ */
	142	+
	143	+struct module_notes_attrs {
	144	+ struct kobject *dir;
	145	+ unsigned int notes;
	146	+ struct bin_attribute attrs[];
	147	+};
	148	+
	149	+static ssize_t module_notes_read(struct file filp, struct kobject kobj,
	150	+ struct bin_attribute *bin_attr,
	151	+ char *buf, loff_t pos, size_t count)
	152	+{
	153	+ /*
	154	+ * The caller checked the pos and count against our size.
	155	+ */
	156	+ memcpy(buf, bin_attr->private + pos, count);
	157	+ return count;
	158	+}
	159	+
	160	+static void free_notes_attrs(struct module_notes_attrs *notes_attrs,
	161	+ unsigned int i)
	162	+{
	163	+ if (notes_attrs->dir) {
	164	+ while (i-- > 0)
	165	+ sysfs_remove_bin_file(notes_attrs->dir,
	166	+ &notes_attrs->attrs[i]);
	167	+ kobject_put(notes_attrs->dir);
	168	+ }
	169	+ kfree(notes_attrs);
	170	+}
	171	+
	172	+static void add_notes_attrs(struct module mod, const struct load_info info)
	173	+{
	174	+ unsigned int notes, loaded, i;
	175	+ struct module_notes_attrs *notes_attrs;
	176	+ struct bin_attribute *nattr;
	177	+
	178	+ /* failed to create section attributes, so can't create notes */
	179	+ if (!mod->sect_attrs)
	180	+ return;
	181	+
	182	+ /* Count notes sections and allocate structures. */
	183	+ notes = 0;
	184	+ for (i = 0; i < info->hdr->e_shnum; i++)
	185	+ if (!sect_empty(&info->sechdrs[i]) &&
	186	+ info->sechdrs[i].sh_type == SHT_NOTE)
	187	+ ++notes;
	188	+
	189	+ if (notes == 0)
	190	+ return;
	191	+
	192	+ notes_attrs = kzalloc(struct_size(notes_attrs, attrs, notes),
	193	+ GFP_KERNEL);
	194	+ if (!notes_attrs)
	195	+ return;
	196	+
	197	+ notes_attrs->notes = notes;
	198	+ nattr = &notes_attrs->attrs[0];
	199	+ for (loaded = i = 0; i < info->hdr->e_shnum; ++i) {
	200	+ if (sect_empty(&info->sechdrs[i]))
	201	+ continue;
	202	+ if (info->sechdrs[i].sh_type == SHT_NOTE) {
	203	+ sysfs_bin_attr_init(nattr);
	204	+ nattr->attr.name = mod->sect_attrs->attrs[loaded].battr.attr.name;
	205	+ nattr->attr.mode = 0444;
	206	+ nattr->size = info->sechdrs[i].sh_size;
	207	+ nattr->private = (void *)info->sechdrs[i].sh_addr;
	208	+ nattr->read = module_notes_read;
	209	+ ++nattr;
	210	+ }
	211	+ ++loaded;
	212	+ }
	213	+
	214	+ notes_attrs->dir = kobject_create_and_add("notes", &mod->mkobj.kobj);
	215	+ if (!notes_attrs->dir)
	216	+ goto out;
	217	+
	218	+ for (i = 0; i < notes; ++i)
	219	+ if (sysfs_create_bin_file(notes_attrs->dir,
	220	+ &notes_attrs->attrs[i]))
	221	+ goto out;
	222	+
	223	+ mod->notes_attrs = notes_attrs;
	224	+ return;
	225	+
	226	+out:
	227	+ free_notes_attrs(notes_attrs, i);
	228	+}
	229	+
	230	+static void remove_notes_attrs(struct module *mod)
	231	+{
	232	+ if (mod->notes_attrs)
	233	+ free_notes_attrs(mod->notes_attrs, mod->notes_attrs->notes);
	234	+}
	235	+
	236	+#else /* !CONFIG_KALLSYMS */
	237	+static inline void add_sect_attrs(struct module mod, const struct load_info info) { }
	238	+static inline void remove_sect_attrs(struct module *mod) { }
	239	+static inline void add_notes_attrs(struct module mod, const struct load_info info) { }
	240	+static inline void remove_notes_attrs(struct module *mod) { }
	241	+#endif /* CONFIG_KALLSYMS */
	242	+
	243	+static void del_usage_links(struct module *mod)
	244	+{
	245	+#ifdef CONFIG_MODULE_UNLOAD
	246	+ struct module_use *use;
	247	+
	248	+ mutex_lock(&module_mutex);
	249	+ list_for_each_entry(use, &mod->target_list, target_list)
	250	+ sysfs_remove_link(use->target->holders_dir, mod->name);
	251	+ mutex_unlock(&module_mutex);
	252	+#endif
	253	+}
	254	+
	255	+static int add_usage_links(struct module *mod)
	256	+{
	257	+ int ret = 0;
	258	+#ifdef CONFIG_MODULE_UNLOAD
	259	+ struct module_use *use;
	260	+
	261	+ mutex_lock(&module_mutex);
	262	+ list_for_each_entry(use, &mod->target_list, target_list) {
	263	+ ret = sysfs_create_link(use->target->holders_dir,
	264	+ &mod->mkobj.kobj, mod->name);
	265	+ if (ret)
	266	+ break;
	267	+ }
	268	+ mutex_unlock(&module_mutex);
	269	+ if (ret)
	270	+ del_usage_links(mod);
	271	+#endif
	272	+ return ret;
	273	+}
	274	+
	275	+static void module_remove_modinfo_attrs(struct module *mod, int end)
	276	+{
	277	+ struct module_attribute *attr;
	278	+ int i;
	279	+
	280	+ for (i = 0; (attr = &mod->modinfo_attrs[i]); i++) {
	281	+ if (end >= 0 && i > end)
	282	+ break;
	283	+ /* pick a field to test for end of list */
	284	+ if (!attr->attr.name)
	285	+ break;
	286	+ sysfs_remove_file(&mod->mkobj.kobj, &attr->attr);
	287	+ if (attr->free)
	288	+ attr->free(mod);
	289	+ }
	290	+ kfree(mod->modinfo_attrs);
	291	+}
	292	+
	293	+static int module_add_modinfo_attrs(struct module *mod)
	294	+{
	295	+ struct module_attribute *attr;
	296	+ struct module_attribute *temp_attr;
	297	+ int error = 0;
	298	+ int i;
	299	+
	300	+ mod->modinfo_attrs = kzalloc((sizeof(struct module_attribute) *
	301	+ (modinfo_attrs_count + 1)),
	302	+ GFP_KERNEL);
	303	+ if (!mod->modinfo_attrs)
	304	+ return -ENOMEM;
	305	+
	306	+ temp_attr = mod->modinfo_attrs;
	307	+ for (i = 0; (attr = modinfo_attrs[i]); i++) {
	308	+ if (!attr->test \|\| attr->test(mod)) {
	309	+ memcpy(temp_attr, attr, sizeof(*temp_attr));
	310	+ sysfs_attr_init(&temp_attr->attr);
	311	+ error = sysfs_create_file(&mod->mkobj.kobj,
	312	+ &temp_attr->attr);
	313	+ if (error)
	314	+ goto error_out;
	315	+ ++temp_attr;
	316	+ }
	317	+ }
	318	+
	319	+ return 0;
	320	+
	321	+error_out:
	322	+ if (i > 0)
	323	+ module_remove_modinfo_attrs(mod, --i);
	324	+ else
	325	+ kfree(mod->modinfo_attrs);
	326	+ return error;
	327	+}
	328	+
	329	+static void mod_kobject_put(struct module *mod)
	330	+{
	331	+ DECLARE_COMPLETION_ONSTACK(c);
	332	+
	333	+ mod->mkobj.kobj_completion = &c;
	334	+ kobject_put(&mod->mkobj.kobj);
	335	+ wait_for_completion(&c);
	336	+}
	337	+
	338	+static int mod_sysfs_init(struct module *mod)
	339	+{
	340	+ int err;
	341	+ struct kobject *kobj;
	342	+
	343	+ if (!module_sysfs_initialized) {
	344	+ pr_err("%s: module sysfs not initialized\n", mod->name);
	345	+ err = -EINVAL;
	346	+ goto out;
	347	+ }
	348	+
	349	+ kobj = kset_find_obj(module_kset, mod->name);
	350	+ if (kobj) {
	351	+ pr_err("%s: module is already loaded\n", mod->name);
	352	+ kobject_put(kobj);
	353	+ err = -EINVAL;
	354	+ goto out;
	355	+ }
	356	+
	357	+ mod->mkobj.mod = mod;
	358	+
	359	+ memset(&mod->mkobj.kobj, 0, sizeof(mod->mkobj.kobj));
	360	+ mod->mkobj.kobj.kset = module_kset;
	361	+ err = kobject_init_and_add(&mod->mkobj.kobj, &module_ktype, NULL,
	362	+ "%s", mod->name);
	363	+ if (err)
	364	+ mod_kobject_put(mod);
	365	+
	366	+out:
	367	+ return err;
	368	+}
	369	+
	370	+int mod_sysfs_setup(struct module *mod,
	371	+ const struct load_info *info,
	372	+ struct kernel_param *kparam,
	373	+ unsigned int num_params)
	374	+{
	375	+ int err;
	376	+
	377	+ err = mod_sysfs_init(mod);
	378	+ if (err)
	379	+ goto out;
	380	+
	381	+ mod->holders_dir = kobject_create_and_add("holders", &mod->mkobj.kobj);
	382	+ if (!mod->holders_dir) {
	383	+ err = -ENOMEM;
	384	+ goto out_unreg;
	385	+ }
	386	+
	387	+ err = module_param_sysfs_setup(mod, kparam, num_params);
	388	+ if (err)
	389	+ goto out_unreg_holders;
	390	+
	391	+ err = module_add_modinfo_attrs(mod);
	392	+ if (err)
	393	+ goto out_unreg_param;
	394	+
	395	+ err = add_usage_links(mod);
	396	+ if (err)
	397	+ goto out_unreg_modinfo_attrs;
	398	+
	399	+ add_sect_attrs(mod, info);
	400	+ add_notes_attrs(mod, info);
	401	+
	402	+ return 0;
	403	+
	404	+out_unreg_modinfo_attrs:
	405	+ module_remove_modinfo_attrs(mod, -1);
	406	+out_unreg_param:
	407	+ module_param_sysfs_remove(mod);
	408	+out_unreg_holders:
	409	+ kobject_put(mod->holders_dir);
	410	+out_unreg:
	411	+ mod_kobject_put(mod);
	412	+out:
	413	+ return err;
	414	+}
	415	+
	416	+static void mod_sysfs_fini(struct module *mod)
	417	+{
	418	+ remove_notes_attrs(mod);
	419	+ remove_sect_attrs(mod);
	420	+ mod_kobject_put(mod);
	421	+}
	422	+
	423	+void mod_sysfs_teardown(struct module *mod)
	424	+{
	425	+ del_usage_links(mod);
	426	+ module_remove_modinfo_attrs(mod, -1);
	427	+ module_param_sysfs_remove(mod);
	428	+ kobject_put(mod->mkobj.drivers_dir);
	429	+ kobject_put(mod->holders_dir);
	430	+ mod_sysfs_fini(mod);
	431	+}
	432	+
	433	+void init_param_lock(struct module *mod)
	434	+{
	435	+ mutex_init(&mod->param_lock);
	436	+}

kernel/module/tracking.c+61−0

@@ -0,0 +1,61 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module taint unload tracking support
	4	+ *
	5	+ * Copyright (C) 2022 Aaron Tomlin
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/string.h>
	10	+#include <linux/printk.h>
	11	+#include <linux/slab.h>
	12	+#include <linux/list.h>
	13	+#include <linux/rculist.h>
	14	+#include "internal.h"
	15	+
	16	+static LIST_HEAD(unloaded_tainted_modules);
	17	+
	18	+int try_add_tainted_module(struct module *mod)
	19	+{
	20	+ struct mod_unload_taint *mod_taint;
	21	+
	22	+ module_assert_mutex_or_preempt();
	23	+
	24	+ list_for_each_entry_rcu(mod_taint, &unloaded_tainted_modules, list,
	25	+ lockdep_is_held(&module_mutex)) {
	26	+ if (!strcmp(mod_taint->name, mod->name) &&
	27	+ mod_taint->taints & mod->taints) {
	28	+ mod_taint->count++;
	29	+ goto out;
	30	+ }
	31	+ }
	32	+
	33	+ mod_taint = kmalloc(sizeof(*mod_taint), GFP_KERNEL);
	34	+ if (unlikely(!mod_taint))
	35	+ return -ENOMEM;
	36	+ strscpy(mod_taint->name, mod->name, MODULE_NAME_LEN);
	37	+ mod_taint->taints = mod->taints;
	38	+ list_add_rcu(&mod_taint->list, &unloaded_tainted_modules);
	39	+ mod_taint->count = 1;
	40	+out:
	41	+ return 0;
	42	+}
	43	+
	44	+void print_unloaded_tainted_modules(void)
	45	+{
	46	+ struct mod_unload_taint *mod_taint;
	47	+ char buf[MODULE_FLAGS_BUF_SIZE];
	48	+
	49	+ if (!list_empty(&unloaded_tainted_modules)) {
	50	+ printk(KERN_DEFAULT "Unloaded tainted modules:");
	51	+ list_for_each_entry_rcu(mod_taint, &unloaded_tainted_modules,
	52	+ list) {
	53	+ size_t l;
	54	+
	55	+ l = module_flags_taint(mod_taint->taints, buf);
	56	+ buf[l++] = '\0';
	57	+ pr_cont(" %s(%s):%llu", mod_taint->name, buf,
	58	+ mod_taint->count);
	59	+ }
	60	+ }
	61	+}

kernel/module/tree_lookup.c+117−0

@@ -0,0 +1,117 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Modules tree lookup
	4	+ *
	5	+ * Copyright (C) 2015 Peter Zijlstra
	6	+ * Copyright (C) 2015 Rusty Russell
	7	+ */
	8	+
	9	+#include <linux/module.h>
	10	+#include <linux/rbtree_latch.h>
	11	+#include "internal.h"
	12	+
	13	+/*
	14	+ * Use a latched RB-tree for __module_address(); this allows us to use
	15	+ * RCU-sched lookups of the address from any context.
	16	+ *
	17	+ * This is conditional on PERF_EVENTS \|\| TRACING because those can really hit
	18	+ * __module_address() hard by doing a lot of stack unwinding; potentially from
	19	+ * NMI context.
	20	+ */
	21	+
	22	+static __always_inline unsigned long __mod_tree_val(struct latch_tree_node *n)
	23	+{
	24	+ struct module_layout *layout = container_of(n, struct module_layout, mtn.node);
	25	+
	26	+ return (unsigned long)layout->base;
	27	+}
	28	+
	29	+static __always_inline unsigned long __mod_tree_size(struct latch_tree_node *n)
	30	+{
	31	+ struct module_layout *layout = container_of(n, struct module_layout, mtn.node);
	32	+
	33	+ return (unsigned long)layout->size;
	34	+}
	35	+
	36	+static __always_inline bool
	37	+mod_tree_less(struct latch_tree_node a, struct latch_tree_node b)
	38	+{
	39	+ return __mod_tree_val(a) < __mod_tree_val(b);
	40	+}
	41	+
	42	+static __always_inline int
	43	+mod_tree_comp(void key, struct latch_tree_node n)
	44	+{
	45	+ unsigned long val = (unsigned long)key;
	46	+ unsigned long start, end;
	47	+
	48	+ start = __mod_tree_val(n);
	49	+ if (val < start)
	50	+ return -1;
	51	+
	52	+ end = start + __mod_tree_size(n);
	53	+ if (val >= end)
	54	+ return 1;
	55	+
	56	+ return 0;
	57	+}
	58	+
	59	+static const struct latch_tree_ops mod_tree_ops = {
	60	+ .less = mod_tree_less,
	61	+ .comp = mod_tree_comp,
	62	+};
	63	+
	64	+static noinline void __mod_tree_insert(struct mod_tree_node node, struct mod_tree_root tree)
	65	+{
	66	+ latch_tree_insert(&node->node, &tree->root, &mod_tree_ops);
	67	+}
	68	+
	69	+static void __mod_tree_remove(struct mod_tree_node node, struct mod_tree_root tree)
	70	+{
	71	+ latch_tree_erase(&node->node, &tree->root, &mod_tree_ops);
	72	+}
	73	+
	74	+/*
	75	+ * These modifications: insert, remove_init and remove; are serialized by the
	76	+ * module_mutex.
	77	+ */
	78	+void mod_tree_insert(struct module *mod)
	79	+{
	80	+ mod->core_layout.mtn.mod = mod;
	81	+ mod->init_layout.mtn.mod = mod;
	82	+
	83	+ __mod_tree_insert(&mod->core_layout.mtn, &mod_tree);
	84	+ if (mod->init_layout.size)
	85	+ __mod_tree_insert(&mod->init_layout.mtn, &mod_tree);
	86	+
	87	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	88	+ mod->data_layout.mtn.mod = mod;
	89	+ __mod_tree_insert(&mod->data_layout.mtn, &mod_data_tree);
	90	+#endif
	91	+}
	92	+
	93	+void mod_tree_remove_init(struct module *mod)
	94	+{
	95	+ if (mod->init_layout.size)
	96	+ __mod_tree_remove(&mod->init_layout.mtn, &mod_tree);
	97	+}
	98	+
	99	+void mod_tree_remove(struct module *mod)
	100	+{
	101	+ __mod_tree_remove(&mod->core_layout.mtn, &mod_tree);
	102	+ mod_tree_remove_init(mod);
	103	+#ifdef CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC
	104	+ __mod_tree_remove(&mod->data_layout.mtn, &mod_data_tree);
	105	+#endif
	106	+}
	107	+
	108	+struct module mod_find(unsigned long addr, struct mod_tree_root tree)
	109	+{
	110	+ struct latch_tree_node *ltn;
	111	+
	112	+ ltn = latch_tree_find((void *)addr, &tree->root, &mod_tree_ops);
	113	+ if (!ltn)
	114	+ return NULL;
	115	+
	116	+ return container_of(ltn, struct mod_tree_node, node)->mod;
	117	+}

kernel/module/version.c+101−0

@@ -0,0 +1,101 @@
	1	+// SPDX-License-Identifier: GPL-2.0-or-later
	2	+/*
	3	+ * Module version support
	4	+ *
	5	+ * Copyright (C) 2008 Rusty Russell
	6	+ */
	7	+
	8	+#include <linux/module.h>
	9	+#include <linux/string.h>
	10	+#include <linux/printk.h>
	11	+#include "internal.h"
	12	+
	13	+int check_version(const struct load_info *info,
	14	+ const char *symname,
	15	+ struct module *mod,
	16	+ const s32 *crc)
	17	+{
	18	+ Elf_Shdr *sechdrs = info->sechdrs;
	19	+ unsigned int versindex = info->index.vers;
	20	+ unsigned int i, num_versions;
	21	+ struct modversion_info *versions;
	22	+
	23	+ /* Exporting module didn't supply crcs? OK, we're already tainted. */
	24	+ if (!crc)
	25	+ return 1;
	26	+
	27	+ /* No versions at all? modprobe --force does this. */
	28	+ if (versindex == 0)
	29	+ return try_to_force_load(mod, symname) == 0;
	30	+
	31	+ versions = (void *)sechdrs[versindex].sh_addr;
	32	+ num_versions = sechdrs[versindex].sh_size
	33	+ / sizeof(struct modversion_info);
	34	+
	35	+ for (i = 0; i < num_versions; i++) {
	36	+ u32 crcval;
	37	+
	38	+ if (strcmp(versions[i].name, symname) != 0)
	39	+ continue;
	40	+
	41	+ crcval = *crc;
	42	+ if (versions[i].crc == crcval)
	43	+ return 1;
	44	+ pr_debug("Found checksum %X vs module %lX\n",
	45	+ crcval, versions[i].crc);
	46	+ goto bad_version;
	47	+ }
	48	+
	49	+ /* Broken toolchain. Warn once, then let it go.. */
	50	+ pr_warn_once("%s: no symbol version for %s\n", info->name, symname);
	51	+ return 1;
	52	+
	53	+bad_version:
	54	+ pr_warn("%s: disagrees about version of symbol %s\n", info->name, symname);
	55	+ return 0;
	56	+}
	57	+
	58	+int check_modstruct_version(const struct load_info *info,
	59	+ struct module *mod)
	60	+{
	61	+ struct find_symbol_arg fsa = {
	62	+ .name = "module_layout",
	63	+ .gplok = true,
	64	+ };
	65	+
	66	+ /*
	67	+ * Since this should be found in kernel (which can't be removed), no
	68	+ * locking is necessary -- use preempt_disable() to placate lockdep.
	69	+ */
	70	+ preempt_disable();
	71	+ if (!find_symbol(&fsa)) {
	72	+ preempt_enable();
	73	+ BUG();
	74	+ }
	75	+ preempt_enable();
	76	+ return check_version(info, "module_layout", mod, fsa.crc);
	77	+}
	78	+
	79	+/* First part is kernel version, which we ignore if module has crcs. */
	80	+int same_magic(const char amagic, const char bmagic,
	81	+ bool has_crcs)
	82	+{
	83	+ if (has_crcs) {
	84	+ amagic += strcspn(amagic, " ");
	85	+ bmagic += strcspn(bmagic, " ");
	86	+ }
	87	+ return strcmp(amagic, bmagic) == 0;
	88	+}
	89	+
	90	+/*
	91	+ * Generate the signature for all relevant module structures here.
	92	+ * If these change, we don't want to try to parse the module.
	93	+ */
	94	+void module_layout(struct module *mod,
	95	+ struct modversion_info *ver,
	96	+ struct kernel_param *kp,
	97	+ struct kernel_symbol *ks,
	98	+ struct tracepoint * const *tp)
	99	+{
	100	+}
	101	+EXPORT_SYMBOL(module_layout);

Merge tag 'modules-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux

Keyboard shortcuts

General

Issues and pull requests

Issues, pull requests and Inbox

Repository code