History

scripts: remove namespace.pl

namespace.pl is intended to help locate symbols which are defined but are not used externally. The goal is to avoid bloat of the namespace in the resulting kernel image. The script relies on object data, and only finds unused symbols for the configuration used to generate that object data. This results in a lot of false positive warnings such as symbols only used by a single architecture, or symbols which are used externally only under certain configurations. Running namespace.pl using allyesconfig, allmodconfig, and x86_64_defconfig yields the following results: * allmodconfig * 11122 unique symbol names with no external reference * 1194 symbols listed as multiply defined * 214 symbols it can't resolve * allyesconfig * 10997 unique symbol names with no external reference * 1194 symbols listed as multiply defined * 214 symbols it can't resolve * x86_64_defconfig * 5757 unique symbol names with no external reference * 528 symbols listed as multiply defined * 154 symbols it can't resolve The script also has no way to easily limit the scope of the checks to a given subset of the kernel, such as only checking for symbols defined within a module or subsystem. Discussion on public mailing lists seems to indicate that many view the tool output as suspect or not very useful (see discussions at [1] and [2] for further context). As described by Masahiro Yamada at [2], namespace.pl provides 3 types of checks: listing multiply defined symbols, resolving external symbols, and warnings about symbols with no reference. The first category of issues is easily caught by the linker as any set of multiply defined symbols should fail to link. The second category of issues is also caught by linking, as undefined symbols would cause issues. Even with modules, these types of issues where a module relies on an external symbol are caught by modpost. The remaining category of issues reported is the list of symbols with no external reference, and is the primary motivation of this script. However, it ought to be clear from the above examples that the output is difficult to sort through. Even allyesconfig has ~10000 entries. The current submit-checklist indicates that patches ought to go through namespacecheck and fix any new issues arising. But that itself presents problems. As described at [1], many cases of reports are due to configuration where a function is used externally by some configuration settings. Prominent maintainers appear to dislike changes modify code such that symbols become static based on CONFIG_* flags ([3], and [4]) One possible solution is to adjust the advice and indicate that we only care about the output of namespacecheck on allyesconfig or allmodconfig builds... However, given the discussion at [2], I suspect that few people are actively using this tool. It doesn't have a maintainer in the MAINTAINERS flie, and it produces so many warnings for unused symbols that it is difficult to use effectively. Thus, I propose we simply remove it. [1] https://lore.kernel.org/netdev/20200708164812.384ae8ea@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com/ [2] https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller@intel.com/ [3] https://lore.kernel.org/netdev/20190828.154744.2058157956381129672.davem@davemloft.net/ [4] https://lore.kernel.org/netdev/20190827210928.576c5fef@cakuba.netronome.com/

Jacob Keller committed 6 years ago

Commit 7dfbea41 parent 76c37663 files · +2 −480Browse files

Showing the changes to scripts (1 of 3 files). Show all

Unified · Split

scripts/namespace.pl+0−473
@@ -1,473 +0,0 @@
1−#!/usr/bin/env perl
2−#
3−# namespace.pl. Mon Aug 30 2004
4−#
5−# Perform a name space analysis on the linux kernel.
6−#
7−# Copyright Keith Owens <kaos@ocs.com.au>. GPL.
8−#
9−# Invoke by changing directory to the top of the kernel object
10−# tree then namespace.pl, no parameters.
11−#
12−# Tuned for 2.1.x kernels with the new module handling, it will
13−# work with 2.0 kernels as well.
14−#
15−# Last change 2.6.9-rc1, adding support for separate source and object
16−# trees.
17−#
18−# The source must be compiled/assembled first, the object files
19−# are the primary input to this script. Incomplete or missing
20−# objects will result in a flawed analysis. Compile both vmlinux
21−# and modules.
22−#
23−# Even with complete objects, treat the result of the analysis
24−# with caution. Some external references are only used by
25−# certain architectures, others with certain combinations of
26−# configuration parameters. Ideally the source should include
27−# something like
28−#
29−# #ifndef CONFIG_...
30−# static
31−# #endif
32−# symbol_definition;
33−#
34−# so the symbols are defined as static unless a particular
35−# CONFIG_... requires it to be external.
36−#
37−# A symbol that is suffixed with '(export only)' has these properties
38−#
39−# * It is global.
40−# * It is marked EXPORT_SYMBOL or EXPORT_SYMBOL_GPL, either in the same
41−# source file or a different source file.
42−# * Given the current .config, nothing uses the symbol.
43−#
44−# The symbol is a candidate for conversion to static, plus removal of the
45−# export. But be careful that a different .config might use the symbol.
46−#
47−#
48−# Name space analysis and cleanup is an iterative process. You cannot
49−# expect to find all the problems in a single pass.
50−#
51−# * Identify possibly unnecessary global declarations, verify that they
52−# really are unnecessary and change them to static.
53−# * Compile and fix up gcc warnings about static, removing dead symbols
54−# as necessary.
55−# * make clean and rebuild with different configs (especially
56−# CONFIG_MODULES=n) to see which symbols are being defined when the
57−# config does not require them. These symbols bloat the kernel object
58−# for no good reason, which is frustrating for embedded systems.
59−# * Wrap config sensitive symbols in #ifdef CONFIG_foo, as long as the
60−# code does not get too ugly.
61−# * Repeat the name space analysis until you can live with with the
62−# result.
63−#
64−
65−use warnings;
66−use strict;
67−use File::Find;
68−use File::Spec;
69−
70−my $nm = ($ENV{'NM'} || "nm") . " -p";
71−my $objdump = ($ENV{'OBJDUMP'} || "objdump") . " -s -j .comment";
72−my $srctree = File::Spec->curdir();
73−my $objtree = File::Spec->curdir();
74−$srctree = File::Spec->rel2abs($ENV{'srctree'}) if (exists($ENV{'srctree'}));
75−$objtree = File::Spec->rel2abs($ENV{'objtree'}) if (exists($ENV{'objtree'}));
76−
77−if ($#ARGV != -1) {
78− print STDERR "usage: $0 takes no parameters\n";
79− die("giving up\n");
80−}
81−
82−my %nmdata = (); # nm data for each object
83−my %def = (); # all definitions for each name
84−my %ksymtab = (); # names that appear in __ksymtab_
85−my %ref = (); # $ref{$name} exists if there is a true external reference to $name
86−my %export = (); # $export{$name} exists if there is an EXPORT_... of $name
87−
88−my %nmexception = (
89− 'fs/ext3/bitmap' => 1,
90− 'fs/ext4/bitmap' => 1,
91− 'arch/x86/lib/thunk_32' => 1,
92− 'arch/x86/lib/cmpxchg' => 1,
93− 'arch/x86/vdso/vdso32/note' => 1,
94− 'lib/irq_regs' => 1,
95− 'usr/initramfs_data' => 1,
96− 'drivers/scsi/aic94xx/aic94xx_dump' => 1,
97− 'drivers/scsi/libsas/sas_dump' => 1,
98− 'lib/dec_and_lock' => 1,
99− 'drivers/ide/ide-probe-mini' => 1,
100− 'usr/initramfs_data' => 1,
101− 'drivers/acpi/acpia/exdump' => 1,
102− 'drivers/acpi/acpia/rsdump' => 1,
103− 'drivers/acpi/acpia/nsdumpdv' => 1,
104− 'drivers/acpi/acpia/nsdump' => 1,
105− 'arch/ia64/sn/kernel/sn2/io' => 1,
106− 'arch/ia64/kernel/gate-data' => 1,
107− 'security/capability' => 1,
108− 'fs/ntfs/sysctl' => 1,
109− 'fs/jfs/jfs_debug' => 1,
110−);
111−
112−my %nameexception = (
113− 'mod_use_count_' => 1,
114− '__initramfs_end' => 1,
115− '__initramfs_start' => 1,
116− '_einittext' => 1,
117− '_sinittext' => 1,
118− 'kallsyms_names' => 1,
119− 'kallsyms_num_syms' => 1,
120− 'kallsyms_addresses'=> 1,
121− 'kallsyms_offsets' => 1,
122− 'kallsyms_relative_base'=> 1,
123− '__this_module' => 1,
124− '_etext' => 1,
125− '_edata' => 1,
126− '_end' => 1,
127− '__bss_start' => 1,
128− '_text' => 1,
129− '_stext' => 1,
130− '__gp' => 1,
131− 'ia64_unw_start' => 1,
132− 'ia64_unw_end' => 1,
133− '__init_begin' => 1,
134− '__init_end' => 1,
135− '__bss_stop' => 1,
136− '__nosave_begin' => 1,
137− '__nosave_end' => 1,
138− 'pg0' => 1,
139− 'vdso_enabled' => 1,
140− '__stack_chk_fail' => 1,
141− 'VDSO32_PRELINK' => 1,
142− 'VDSO32_vsyscall' => 1,
143− 'VDSO32_rt_sigreturn'=>1,
144− 'VDSO32_sigreturn' => 1,
145−);
146−
147−
148−&find(\&linux_objects, '.'); # find the objects and do_nm on them
149−&list_multiply_defined();
150−&resolve_external_references();
151−&list_extra_externals();
152−
153−exit(0);
154−
155−sub linux_objects
156−{
157− # Select objects, ignoring objects which are only created by
158− # merging other objects. Also ignore all of modules, scripts
159− # and compressed. Most conglomerate objects are handled by do_nm,
160− # this list only contains the special cases. These include objects
161− # that are linked from just one other object and objects for which
162− # there is really no permanent source file.
163− my $basename = $_;
164− $_ = $File::Find::name;
165− s:^\./::;
166− if (/.*\.o$/ &&
167− ! (
168− m:/built-in.a$:
169− || m:arch/x86/vdso/:
170− || m:arch/x86/boot/:
171− || m:arch/ia64/ia32/ia32.o$:
172− || m:arch/ia64/kernel/gate-syms.o$:
173− || m:arch/ia64/lib/__divdi3.o$:
174− || m:arch/ia64/lib/__divsi3.o$:
175− || m:arch/ia64/lib/__moddi3.o$:
176− || m:arch/ia64/lib/__modsi3.o$:
177− || m:arch/ia64/lib/__udivdi3.o$:
178− || m:arch/ia64/lib/__udivsi3.o$:
179− || m:arch/ia64/lib/__umoddi3.o$:
180− || m:arch/ia64/lib/__umodsi3.o$:
181− || m:arch/ia64/scripts/check_gas_for_hint.o$:
182− || m:arch/ia64/sn/kernel/xp.o$:
183− || m:boot/bbootsect.o$:
184− || m:boot/bsetup.o$:
185− || m:/bootsect.o$:
186− || m:/boot/setup.o$:
187− || m:/compressed/:
188− || m:drivers/cdrom/driver.o$:
189− || m:drivers/char/drm/tdfx_drv.o$:
190− || m:drivers/ide/ide-detect.o$:
191− || m:drivers/ide/pci/idedriver-pci.o$:
192− || m:drivers/media/media.o$:
193− || m:drivers/scsi/sd_mod.o$:
194− || m:drivers/video/video.o$:
195− || m:fs/devpts/devpts.o$:
196− || m:fs/exportfs/exportfs.o$:
197− || m:fs/hugetlbfs/hugetlbfs.o$:
198− || m:fs/msdos/msdos.o$:
199− || m:fs/nls/nls.o$:
200− || m:fs/ramfs/ramfs.o$:
201− || m:fs/romfs/romfs.o$:
202− || m:fs/vfat/vfat.o$:
203− || m:init/mounts.o$:
204− || m:^modules/:
205− || m:net/netlink/netlink.o$:
206− || m:net/sched/sched.o$:
207− || m:/piggy.o$:
208− || m:^scripts/:
209− || m:sound/.*/snd-:
210− || m:^.*/\.tmp_:
211− || m:^\.tmp_:
212− || m:/vmlinux-obj.o$:
213− || m:^tools/:
214− )
215− ) {
216− do_nm($basename, $_);
217− }
218− $_ = $basename; # File::Find expects $_ untouched (undocumented)
219−}
220−
221−sub do_nm
222−{
223− my ($basename, $fullname) = @_;
224− my ($source, $type, $name);
225− if (! -e $basename) {
226− printf STDERR "$basename does not exist\n";
227− return;
228− }
229− if ($fullname !~ /\.o$/) {
230− printf STDERR "$fullname is not an object file\n";
231− return;
232− }
233− ($source = $basename) =~ s/\.o$//;
234− if (-e "$source.c" || -e "$source.S") {
235− $source = File::Spec->catfile($objtree, $File::Find::dir, $source)
236− } else {
237− $source = File::Spec->catfile($srctree, $File::Find::dir, $source)
238− }
239− if (! -e "$source.c" && ! -e "$source.S") {
240− # No obvious source, exclude the object if it is conglomerate
241− open(my $objdumpdata, "$objdump $basename|")
242− or die "$objdump $fullname failed $!\n";
243−
244− my $comment;
245− while (<$objdumpdata>) {
246− chomp();
247− if (/^In archive/) {
248− # Archives are always conglomerate
249− $comment = "GCC:GCC:";
250− last;
251− }
252− next if (! /^[ 0-9a-f]{5,} /);
253− $comment .= substr($_, 43);
254− }
255− close($objdumpdata);
256−
257− if (!defined($comment) || $comment !~ /GCC\:.*GCC\:/m) {
258− printf STDERR "No source file found for $fullname\n";
259− }
260− return;
261− }
262− open (my $nmdata, "$nm $basename|")
263− or die "$nm $fullname failed $!\n";
264−
265− my @nmdata;
266− while (<$nmdata>) {
267− chop;
268− ($type, $name) = (split(/ +/, $_, 3))[1..2];
269− # Expected types
270− # A absolute symbol
271− # B weak external reference to data that has been resolved
272− # C global variable, uninitialised
273− # D global variable, initialised
274− # G global variable, initialised, small data section
275− # R global array, initialised
276− # S global variable, uninitialised, small bss
277− # T global label/procedure
278− # U external reference
279− # W weak external reference to text that has been resolved
280− # V similar to W, but the value of the weak symbol becomes zero with no error.
281− # a assembler equate
282− # b static variable, uninitialised
283− # d static variable, initialised
284− # g static variable, initialised, small data section
285− # r static array, initialised
286− # s static variable, uninitialised, small bss
287− # t static label/procedures
288− # w weak external reference to text that has not been resolved
289− # v similar to w
290− # ? undefined type, used a lot by modules
291− if ($type !~ /^[ABCDGRSTUWVabdgrstwv?]$/) {
292− printf STDERR "nm output for $fullname contains unknown type '$_'\n";
293− }
294− elsif ($name =~ /\./) {
295− # name with '.' is local static
296− }
297− else {
298− $type = 'R' if ($type eq '?'); # binutils replaced ? with R at one point
299− # binutils keeps changing the type for exported symbols, force it to R
300− $type = 'R' if ($name =~ /^__ksymtab/ || $name =~ /^__kstrtab/);
301− $name =~ s/_R[a-f0-9]{8}$//; # module versions adds this
302− if ($type =~ /[ABCDGRSTWV]/ &&
303− $name ne 'init_module' &&
304− $name ne 'cleanup_module' &&
305− $name ne 'Using_Versions' &&
306− $name !~ /^Version_[0-9]+$/ &&
307− $name !~ /^__parm_/ &&
308− $name !~ /^__kstrtab/ &&
309− $name !~ /^__ksymtab/ &&
310− $name !~ /^__kcrctab_/ &&
311− $name !~ /^__exitcall_/ &&
312− $name !~ /^__initcall_/ &&
313− $name !~ /^__kdb_initcall_/ &&
314− $name !~ /^__kdb_exitcall_/ &&
315− $name !~ /^__module_/ &&
316− $name !~ /^__mod_/ &&
317− $name !~ /^__crc_/ &&
318− $name ne '__this_module' &&
319− $name ne 'kernel_version') {
320− if (!exists($def{$name})) {
321− $def{$name} = [];
322− }
323− push(@{$def{$name}}, $fullname);
324− }
325− push(@nmdata, "$type $name");
326− if ($name =~ /^__ksymtab_/) {
327− $name = substr($name, 10);
328− if (!exists($ksymtab{$name})) {
329− $ksymtab{$name} = [];
330− }
331− push(@{$ksymtab{$name}}, $fullname);
332− }
333− }
334− }
335− close($nmdata);
336−
337− if ($#nmdata < 0) {
338− printf "No nm data for $fullname\n"
339− unless $nmexception{$fullname};
340− return;
341− }
342− $nmdata{$fullname} = \@nmdata;
343−}
344−
345−sub drop_def
346−{
347− my ($object, $name) = @_;
348− my $nmdata = $nmdata{$object};
349− my ($i, $j);
350− for ($i = 0; $i <= $#{$nmdata}; ++$i) {
351− if ($name eq (split(' ', $nmdata->[$i], 2))[1]) {
352− splice(@{$nmdata{$object}}, $i, 1);
353− my $def = $def{$name};
354− for ($j = 0; $j < $#{$def{$name}}; ++$j) {
355− if ($def{$name}[$j] eq $object) {
356− splice(@{$def{$name}}, $j, 1);
357− }
358− }
359− last;
360− }
361− }
362−}
363−
364−sub list_multiply_defined
365−{
366− foreach my $name (keys(%def)) {
367− if ($#{$def{$name}} > 0) {
368− # Special case for cond_syscall
369− if ($#{$def{$name}} == 1 &&
370− ($name =~ /^sys_/ || $name =~ /^compat_sys_/ ||
371− $name =~ /^sys32_/)) {
372− if($def{$name}[0] eq "kernel/sys_ni.o" ||
373− $def{$name}[1] eq "kernel/sys_ni.o") {
374− &drop_def("kernel/sys_ni.o", $name);
375− next;
376− }
377− }
378−
379− printf "$name is multiply defined in :-\n";
380− foreach my $module (@{$def{$name}}) {
381− printf "\t$module\n";
382− }
383− }
384− }
385−}
386−
387−sub resolve_external_references
388−{
389− my ($kstrtab, $ksymtab, $export);
390−
391− printf "\n";
392− foreach my $object (keys(%nmdata)) {
393− my $nmdata = $nmdata{$object};
394− for (my $i = 0; $i <= $#{$nmdata}; ++$i) {
395− my ($type, $name) = split(' ', $nmdata->[$i], 2);
396− if ($type eq "U" || $type eq "w") {
397− if (exists($def{$name}) || exists($ksymtab{$name})) {
398− # add the owning object to the nmdata
399− $nmdata->[$i] = "$type $name $object";
400− # only count as a reference if it is not EXPORT_...
401− $kstrtab = "R __kstrtab_$name";
402− $ksymtab = "R __ksymtab_$name";
403− $export = 0;
404− for (my $j = 0; $j <= $#{$nmdata}; ++$j) {
405− if ($nmdata->[$j] eq $kstrtab ||
406− $nmdata->[$j] eq $ksymtab) {
407− $export = 1;
408− last;
409− }
410− }
411− if ($export) {
412− $export{$name} = "";
413− }
414− else {
415− $ref{$name} = ""
416− }
417− }
418− elsif ( ! $nameexception{$name}
419− && $name !~ /^__sched_text_/
420− && $name !~ /^__start_/
421− && $name !~ /^__end_/
422− && $name !~ /^__stop_/
423− && $name !~ /^__scheduling_functions_.*_here/
424− && $name !~ /^__.*initcall_/
425− && $name !~ /^__.*per_cpu_start/
426− && $name !~ /^__.*per_cpu_end/
427− && $name !~ /^__alt_instructions/
428− && $name !~ /^__setup_/
429− && $name !~ /^__mod_timer/
430− && $name !~ /^__mod_page_state/
431− && $name !~ /^init_module/
432− && $name !~ /^cleanup_module/
433− ) {
434− printf "Cannot resolve ";
435− printf "weak " if ($type eq "w");
436− printf "reference to $name from $object\n";
437− }
438− }
439− }
440− }
441−}
442−
443−sub list_extra_externals
444−{
445− my %noref = ();
446−
447− foreach my $name (keys(%def)) {
448− if (! exists($ref{$name})) {
449− my @module = @{$def{$name}};
450− foreach my $module (@module) {
451− if (! exists($noref{$module})) {
452− $noref{$module} = [];
453− }
454− push(@{$noref{$module}}, $name);
455− }
456− }
457− }
458− if (%noref) {
459− printf "\nExternally defined symbols with no external references\n";
460− foreach my $module (sort(keys(%noref))) {
461− printf " $module\n";
462− foreach (sort(@{$noref{$module}})) {
463− my $export;
464− if (exists($export{$_})) {
465− $export = " (export only)";
466− } else {
467− $export = "";
468− }
469− printf " $_$export\n";
470− }
471− }
472− }
473−}