History

scripts: remove namespace.pl

namespace.pl is intended to help locate symbols which are defined but are not used externally. The goal is to avoid bloat of the namespace in the resulting kernel image. The script relies on object data, and only finds unused symbols for the configuration used to generate that object data. This results in a lot of false positive warnings such as symbols only used by a single architecture, or symbols which are used externally only under certain configurations. Running namespace.pl using allyesconfig, allmodconfig, and x86_64_defconfig yields the following results: * allmodconfig * 11122 unique symbol names with no external reference * 1194 symbols listed as multiply defined * 214 symbols it can't resolve * allyesconfig * 10997 unique symbol names with no external reference * 1194 symbols listed as multiply defined * 214 symbols it can't resolve * x86_64_defconfig * 5757 unique symbol names with no external reference * 528 symbols listed as multiply defined * 154 symbols it can't resolve The script also has no way to easily limit the scope of the checks to a given subset of the kernel, such as only checking for symbols defined within a module or subsystem. Discussion on public mailing lists seems to indicate that many view the tool output as suspect or not very useful (see discussions at [1] and [2] for further context). As described by Masahiro Yamada at [2], namespace.pl provides 3 types of checks: listing multiply defined symbols, resolving external symbols, and warnings about symbols with no reference. The first category of issues is easily caught by the linker as any set of multiply defined symbols should fail to link. The second category of issues is also caught by linking, as undefined symbols would cause issues. Even with modules, these types of issues where a module relies on an external symbol are caught by modpost. The remaining category of issues reported is the list of symbols with no external reference, and is the primary motivation of this script. However, it ought to be clear from the above examples that the output is difficult to sort through. Even allyesconfig has ~10000 entries. The current submit-checklist indicates that patches ought to go through namespacecheck and fix any new issues arising. But that itself presents problems. As described at [1], many cases of reports are due to configuration where a function is used externally by some configuration settings. Prominent maintainers appear to dislike changes modify code such that symbols become static based on CONFIG_* flags ([3], and [4]) One possible solution is to adjust the advice and indicate that we only care about the output of namespacecheck on allyesconfig or allmodconfig builds... However, given the discussion at [2], I suspect that few people are actively using this tool. It doesn't have a maintainer in the MAINTAINERS flie, and it produces so many warnings for unused symbols that it is difficult to use effectively. Thus, I propose we simply remove it. [1] https://lore.kernel.org/netdev/20200708164812.384ae8ea@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com/ [2] https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller@intel.com/ [3] https://lore.kernel.org/netdev/20190828.154744.2058157956381129672.davem@davemloft.net/ [4] https://lore.kernel.org/netdev/20190827210928.576c5fef@cakuba.netronome.com/

Jacob Keller committed 6 years ago

Commit 7dfbea41 parent 76c37663 files · +2 −480Browse files

Showing the changes to scripts (1 of 3 files). Show all

Unified · Split

scripts/namespace.pl+0−473
@@ -1,473 +0,0 @@
1#!/usr/bin/env perl
2#
3# namespace.pl. Mon Aug 30 2004
4#
5# Perform a name space analysis on the linux kernel.
6#
7# Copyright Keith Owens <kaos@ocs.com.au>. GPL.
8#
9# Invoke by changing directory to the top of the kernel object
10# tree then namespace.pl, no parameters.
11#
12# Tuned for 2.1.x kernels with the new module handling, it will
13# work with 2.0 kernels as well.
14#
15# Last change 2.6.9-rc1, adding support for separate source and object
16# trees.
17#
18# The source must be compiled/assembled first, the object files
19# are the primary input to this script. Incomplete or missing
20# objects will result in a flawed analysis. Compile both vmlinux
21# and modules.
22#
23# Even with complete objects, treat the result of the analysis
24# with caution. Some external references are only used by
25# certain architectures, others with certain combinations of
26# configuration parameters. Ideally the source should include
27# something like
28#
29# #ifndef CONFIG_...
30# static
31# #endif
32# symbol_definition;
33#
34# so the symbols are defined as static unless a particular
35# CONFIG_... requires it to be external.
36#
37# A symbol that is suffixed with '(export only)' has these properties
38#
39# * It is global.
40# * It is marked EXPORT_SYMBOL or EXPORT_SYMBOL_GPL, either in the same
41# source file or a different source file.
42# * Given the current .config, nothing uses the symbol.
43#
44# The symbol is a candidate for conversion to static, plus removal of the
45# export. But be careful that a different .config might use the symbol.
46#
47#
48# Name space analysis and cleanup is an iterative process. You cannot
49# expect to find all the problems in a single pass.
50#
51# * Identify possibly unnecessary global declarations, verify that they
52# really are unnecessary and change them to static.
53# * Compile and fix up gcc warnings about static, removing dead symbols
54# as necessary.
55# * make clean and rebuild with different configs (especially
56# CONFIG_MODULES=n) to see which symbols are being defined when the
57# config does not require them. These symbols bloat the kernel object
58# for no good reason, which is frustrating for embedded systems.
59# * Wrap config sensitive symbols in #ifdef CONFIG_foo, as long as the
60# code does not get too ugly.
61# * Repeat the name space analysis until you can live with with the
62# result.
63#
64
65use warnings;
66use strict;
67use File::Find;
68use File::Spec;
69
70my $nm = ($ENV{'NM'} || "nm") . " -p";
71my $objdump = ($ENV{'OBJDUMP'} || "objdump") . " -s -j .comment";
72my $srctree = File::Spec->curdir();
73my $objtree = File::Spec->curdir();
74$srctree = File::Spec->rel2abs($ENV{'srctree'}) if (exists($ENV{'srctree'}));
75$objtree = File::Spec->rel2abs($ENV{'objtree'}) if (exists($ENV{'objtree'}));
76
77if ($#ARGV != -1) {
78 print STDERR "usage: $0 takes no parameters\n";
79 die("giving up\n");
80}
81
82my %nmdata = (); # nm data for each object
83my %def = (); # all definitions for each name
84my %ksymtab = (); # names that appear in __ksymtab_
85my %ref = (); # $ref{$name} exists if there is a true external reference to $name
86my %export = (); # $export{$name} exists if there is an EXPORT_... of $name
87
88my %nmexception = (
89 'fs/ext3/bitmap' => 1,
90 'fs/ext4/bitmap' => 1,
91 'arch/x86/lib/thunk_32' => 1,
92 'arch/x86/lib/cmpxchg' => 1,
93 'arch/x86/vdso/vdso32/note' => 1,
94 'lib/irq_regs' => 1,
95 'usr/initramfs_data' => 1,
96 'drivers/scsi/aic94xx/aic94xx_dump' => 1,
97 'drivers/scsi/libsas/sas_dump' => 1,
98 'lib/dec_and_lock' => 1,
99 'drivers/ide/ide-probe-mini' => 1,
100 'usr/initramfs_data' => 1,
101 'drivers/acpi/acpia/exdump' => 1,
102 'drivers/acpi/acpia/rsdump' => 1,
103 'drivers/acpi/acpia/nsdumpdv' => 1,
104 'drivers/acpi/acpia/nsdump' => 1,
105 'arch/ia64/sn/kernel/sn2/io' => 1,
106 'arch/ia64/kernel/gate-data' => 1,
107 'security/capability' => 1,
108 'fs/ntfs/sysctl' => 1,
109 'fs/jfs/jfs_debug' => 1,
110);
111
112my %nameexception = (
113 'mod_use_count_' => 1,
114 '__initramfs_end' => 1,
115 '__initramfs_start' => 1,
116 '_einittext' => 1,
117 '_sinittext' => 1,
118 'kallsyms_names' => 1,
119 'kallsyms_num_syms' => 1,
120 'kallsyms_addresses'=> 1,
121 'kallsyms_offsets' => 1,
122 'kallsyms_relative_base'=> 1,
123 '__this_module' => 1,
124 '_etext' => 1,
125 '_edata' => 1,
126 '_end' => 1,
127 '__bss_start' => 1,
128 '_text' => 1,
129 '_stext' => 1,
130 '__gp' => 1,
131 'ia64_unw_start' => 1,
132 'ia64_unw_end' => 1,
133 '__init_begin' => 1,
134 '__init_end' => 1,
135 '__bss_stop' => 1,
136 '__nosave_begin' => 1,
137 '__nosave_end' => 1,
138 'pg0' => 1,
139 'vdso_enabled' => 1,
140 '__stack_chk_fail' => 1,
141 'VDSO32_PRELINK' => 1,
142 'VDSO32_vsyscall' => 1,
143 'VDSO32_rt_sigreturn'=>1,
144 'VDSO32_sigreturn' => 1,
145);
146
147
148&find(\&linux_objects, '.'); # find the objects and do_nm on them
149&list_multiply_defined();
150&resolve_external_references();
151&list_extra_externals();
152
153exit(0);
154
155sub linux_objects
156{
157 # Select objects, ignoring objects which are only created by
158 # merging other objects. Also ignore all of modules, scripts
159 # and compressed. Most conglomerate objects are handled by do_nm,
160 # this list only contains the special cases. These include objects
161 # that are linked from just one other object and objects for which
162 # there is really no permanent source file.
163 my $basename = $_;
164 $_ = $File::Find::name;
165 s:^\./::;
166 if (/.*\.o$/ &&
167 ! (
168 m:/built-in.a$:
169 || m:arch/x86/vdso/:
170 || m:arch/x86/boot/:
171 || m:arch/ia64/ia32/ia32.o$:
172 || m:arch/ia64/kernel/gate-syms.o$:
173 || m:arch/ia64/lib/__divdi3.o$:
174 || m:arch/ia64/lib/__divsi3.o$:
175 || m:arch/ia64/lib/__moddi3.o$:
176 || m:arch/ia64/lib/__modsi3.o$:
177 || m:arch/ia64/lib/__udivdi3.o$:
178 || m:arch/ia64/lib/__udivsi3.o$:
179 || m:arch/ia64/lib/__umoddi3.o$:
180 || m:arch/ia64/lib/__umodsi3.o$:
181 || m:arch/ia64/scripts/check_gas_for_hint.o$:
182 || m:arch/ia64/sn/kernel/xp.o$:
183 || m:boot/bbootsect.o$:
184 || m:boot/bsetup.o$:
185 || m:/bootsect.o$:
186 || m:/boot/setup.o$:
187 || m:/compressed/:
188 || m:drivers/cdrom/driver.o$:
189 || m:drivers/char/drm/tdfx_drv.o$:
190 || m:drivers/ide/ide-detect.o$:
191 || m:drivers/ide/pci/idedriver-pci.o$:
192 || m:drivers/media/media.o$:
193 || m:drivers/scsi/sd_mod.o$:
194 || m:drivers/video/video.o$:
195 || m:fs/devpts/devpts.o$:
196 || m:fs/exportfs/exportfs.o$:
197 || m:fs/hugetlbfs/hugetlbfs.o$:
198 || m:fs/msdos/msdos.o$:
199 || m:fs/nls/nls.o$:
200 || m:fs/ramfs/ramfs.o$:
201 || m:fs/romfs/romfs.o$:
202 || m:fs/vfat/vfat.o$:
203 || m:init/mounts.o$:
204 || m:^modules/:
205 || m:net/netlink/netlink.o$:
206 || m:net/sched/sched.o$:
207 || m:/piggy.o$:
208 || m:^scripts/:
209 || m:sound/.*/snd-:
210 || m:^.*/\.tmp_:
211 || m:^\.tmp_:
212 || m:/vmlinux-obj.o$:
213 || m:^tools/:
214 )
215 ) {
216 do_nm($basename, $_);
217 }
218 $_ = $basename; # File::Find expects $_ untouched (undocumented)
219}
220
221sub do_nm
222{
223 my ($basename, $fullname) = @_;
224 my ($source, $type, $name);
225 if (! -e $basename) {
226 printf STDERR "$basename does not exist\n";
227 return;
228 }
229 if ($fullname !~ /\.o$/) {
230 printf STDERR "$fullname is not an object file\n";
231 return;
232 }
233 ($source = $basename) =~ s/\.o$//;
234 if (-e "$source.c" || -e "$source.S") {
235 $source = File::Spec->catfile($objtree, $File::Find::dir, $source)
236 } else {
237 $source = File::Spec->catfile($srctree, $File::Find::dir, $source)
238 }
239 if (! -e "$source.c" && ! -e "$source.S") {
240 # No obvious source, exclude the object if it is conglomerate
241 open(my $objdumpdata, "$objdump $basename|")
242 or die "$objdump $fullname failed $!\n";
243
244 my $comment;
245 while (<$objdumpdata>) {
246 chomp();
247 if (/^In archive/) {
248 # Archives are always conglomerate
249 $comment = "GCC:GCC:";
250 last;
251 }
252 next if (! /^[ 0-9a-f]{5,} /);
253 $comment .= substr($_, 43);
254 }
255 close($objdumpdata);
256
257 if (!defined($comment) || $comment !~ /GCC\:.*GCC\:/m) {
258 printf STDERR "No source file found for $fullname\n";
259 }
260 return;
261 }
262 open (my $nmdata, "$nm $basename|")
263 or die "$nm $fullname failed $!\n";
264
265 my @nmdata;
266 while (<$nmdata>) {
267 chop;
268 ($type, $name) = (split(/ +/, $_, 3))[1..2];
269 # Expected types
270 # A absolute symbol
271 # B weak external reference to data that has been resolved
272 # C global variable, uninitialised
273 # D global variable, initialised
274 # G global variable, initialised, small data section
275 # R global array, initialised
276 # S global variable, uninitialised, small bss
277 # T global label/procedure
278 # U external reference
279 # W weak external reference to text that has been resolved
280 # V similar to W, but the value of the weak symbol becomes zero with no error.
281 # a assembler equate
282 # b static variable, uninitialised
283 # d static variable, initialised
284 # g static variable, initialised, small data section
285 # r static array, initialised
286 # s static variable, uninitialised, small bss
287 # t static label/procedures
288 # w weak external reference to text that has not been resolved
289 # v similar to w
290 # ? undefined type, used a lot by modules
291 if ($type !~ /^[ABCDGRSTUWVabdgrstwv?]$/) {
292 printf STDERR "nm output for $fullname contains unknown type '$_'\n";
293 }
294 elsif ($name =~ /\./) {
295 # name with '.' is local static
296 }
297 else {
298 $type = 'R' if ($type eq '?'); # binutils replaced ? with R at one point
299 # binutils keeps changing the type for exported symbols, force it to R
300 $type = 'R' if ($name =~ /^__ksymtab/ || $name =~ /^__kstrtab/);
301 $name =~ s/_R[a-f0-9]{8}$//; # module versions adds this
302 if ($type =~ /[ABCDGRSTWV]/ &&
303 $name ne 'init_module' &&
304 $name ne 'cleanup_module' &&
305 $name ne 'Using_Versions' &&
306 $name !~ /^Version_[0-9]+$/ &&
307 $name !~ /^__parm_/ &&
308 $name !~ /^__kstrtab/ &&
309 $name !~ /^__ksymtab/ &&
310 $name !~ /^__kcrctab_/ &&
311 $name !~ /^__exitcall_/ &&
312 $name !~ /^__initcall_/ &&
313 $name !~ /^__kdb_initcall_/ &&
314 $name !~ /^__kdb_exitcall_/ &&
315 $name !~ /^__module_/ &&
316 $name !~ /^__mod_/ &&
317 $name !~ /^__crc_/ &&
318 $name ne '__this_module' &&
319 $name ne 'kernel_version') {
320 if (!exists($def{$name})) {
321 $def{$name} = [];
322 }
323 push(@{$def{$name}}, $fullname);
324 }
325 push(@nmdata, "$type $name");
326 if ($name =~ /^__ksymtab_/) {
327 $name = substr($name, 10);
328 if (!exists($ksymtab{$name})) {
329 $ksymtab{$name} = [];
330 }
331 push(@{$ksymtab{$name}}, $fullname);
332 }
333 }
334 }
335 close($nmdata);
336
337 if ($#nmdata < 0) {
338 printf "No nm data for $fullname\n"
339 unless $nmexception{$fullname};
340 return;
341 }
342 $nmdata{$fullname} = \@nmdata;
343}
344
345sub drop_def
346{
347 my ($object, $name) = @_;
348 my $nmdata = $nmdata{$object};
349 my ($i, $j);
350 for ($i = 0; $i <= $#{$nmdata}; ++$i) {
351 if ($name eq (split(' ', $nmdata->[$i], 2))[1]) {
352 splice(@{$nmdata{$object}}, $i, 1);
353 my $def = $def{$name};
354 for ($j = 0; $j < $#{$def{$name}}; ++$j) {
355 if ($def{$name}[$j] eq $object) {
356 splice(@{$def{$name}}, $j, 1);
357 }
358 }
359 last;
360 }
361 }
362}
363
364sub list_multiply_defined
365{
366 foreach my $name (keys(%def)) {
367 if ($#{$def{$name}} > 0) {
368 # Special case for cond_syscall
369 if ($#{$def{$name}} == 1 &&
370 ($name =~ /^sys_/ || $name =~ /^compat_sys_/ ||
371 $name =~ /^sys32_/)) {
372 if($def{$name}[0] eq "kernel/sys_ni.o" ||
373 $def{$name}[1] eq "kernel/sys_ni.o") {
374 &drop_def("kernel/sys_ni.o", $name);
375 next;
376 }
377 }
378
379 printf "$name is multiply defined in :-\n";
380 foreach my $module (@{$def{$name}}) {
381 printf "\t$module\n";
382 }
383 }
384 }
385}
386
387sub resolve_external_references
388{
389 my ($kstrtab, $ksymtab, $export);
390
391 printf "\n";
392 foreach my $object (keys(%nmdata)) {
393 my $nmdata = $nmdata{$object};
394 for (my $i = 0; $i <= $#{$nmdata}; ++$i) {
395 my ($type, $name) = split(' ', $nmdata->[$i], 2);
396 if ($type eq "U" || $type eq "w") {
397 if (exists($def{$name}) || exists($ksymtab{$name})) {
398 # add the owning object to the nmdata
399 $nmdata->[$i] = "$type $name $object";
400 # only count as a reference if it is not EXPORT_...
401 $kstrtab = "R __kstrtab_$name";
402 $ksymtab = "R __ksymtab_$name";
403 $export = 0;
404 for (my $j = 0; $j <= $#{$nmdata}; ++$j) {
405 if ($nmdata->[$j] eq $kstrtab ||
406 $nmdata->[$j] eq $ksymtab) {
407 $export = 1;
408 last;
409 }
410 }
411 if ($export) {
412 $export{$name} = "";
413 }
414 else {
415 $ref{$name} = ""
416 }
417 }
418 elsif ( ! $nameexception{$name}
419 && $name !~ /^__sched_text_/
420 && $name !~ /^__start_/
421 && $name !~ /^__end_/
422 && $name !~ /^__stop_/
423 && $name !~ /^__scheduling_functions_.*_here/
424 && $name !~ /^__.*initcall_/
425 && $name !~ /^__.*per_cpu_start/
426 && $name !~ /^__.*per_cpu_end/
427 && $name !~ /^__alt_instructions/
428 && $name !~ /^__setup_/
429 && $name !~ /^__mod_timer/
430 && $name !~ /^__mod_page_state/
431 && $name !~ /^init_module/
432 && $name !~ /^cleanup_module/
433 ) {
434 printf "Cannot resolve ";
435 printf "weak " if ($type eq "w");
436 printf "reference to $name from $object\n";
437 }
438 }
439 }
440 }
441}
442
443sub list_extra_externals
444{
445 my %noref = ();
446
447 foreach my $name (keys(%def)) {
448 if (! exists($ref{$name})) {
449 my @module = @{$def{$name}};
450 foreach my $module (@module) {
451 if (! exists($noref{$module})) {
452 $noref{$module} = [];
453 }
454 push(@{$noref{$module}}, $name);
455 }
456 }
457 }
458 if (%noref) {
459 printf "\nExternally defined symbols with no external references\n";
460 foreach my $module (sort(keys(%noref))) {
461 printf " $module\n";
462 foreach (sort(@{$noref{$module}})) {
463 my $export;
464 if (exists($export{$_})) {
465 $export = " (export only)";
466 } else {
467 $export = "";
468 }
469 printf " $_$export\n";
470 }
471 }
472 }
473}