History

Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull perf update from Ingo Molnar: "Lots of changes in this cycle as well, with hundreds of commits from over 30 contributors. Most of the activity was on the tooling side. Higher level changes: - New 'perf kvm' analysis tool, from Xiao Guangrong. - New 'perf trace' system-wide tracing tool - uprobes fixes + cleanups from Oleg Nesterov. - Lots of patches to make perf build on Android out of box, from Irina Tirdea - Extend ftrace function tracing utility to be more dynamic for its users. It allows for data passing to the callback functions, as well as reading regs as if a breakpoint were to trigger at function entry. The main goal of this patch series was to allow kprobes to use ftrace as an optimized probe point when a probe is placed on an ftrace nop. With lots of help from Masami Hiramatsu, and going through lots of iterations, we finally came up with a good solution. - Add cpumask for uncore pmu, use it in 'stat', from Yan, Zheng. - Various tracing updates from Steve Rostedt - Clean up and improve 'perf sched' performance by elliminating lots of needless calls to libtraceevent. - Event group parsing support, from Jiri Olsa - UI/gtk refactorings and improvements from Namhyung Kim - Add support for non-tracepoint events in perf script python, from Feng Tang - Add --symbols to 'script', similar to the one in 'report', from Feng Tang. Infrastructure enhancements and fixes: - Convert the trace builtins to use the growing evsel/evlist tracepoint infrastructure, removing several open coded constructs like switch like series of strcmp to dispatch events, etc. Basically what had already been showcased in 'perf sched'. - Add evsel constructor for tracepoints, that uses libtraceevent just to parse the /format events file, use it in a new 'perf test' to make sure the libtraceevent format parsing regressions can be more readily caught. - Some strange errors were happening in some builds, but not on the next, reported by several people, problem was some parser related files, generated during the build, didn't had proper make deps, fix from Eric Sandeen. - Introduce struct and cache information about the environment where a perf.data file was captured, from Namhyung Kim. - Fix handling of unresolved samples when --symbols is used in 'report', from Feng Tang. - Add union member access support to 'probe', from Hyeoncheol Lee. - Fixups to die() removal, from Namhyung Kim. - Render fixes for the TUI, from Namhyung Kim. - Don't enable annotation in non symbolic view, from Namhyung Kim. - Fix pipe mode in 'report', from Namhyung Kim. - Move related stats code from stat to util/, will be used by the 'stat' kvm tool, from Xiao Guangrong. - Remove die()/exit() calls from several tools. - Resolve vdso callchains, from Jiri Olsa - Don't pass const char pointers to basename, so that we can unconditionally use libgen.h and thus avoid ifdef BIONIC lines, from David Ahern - Refactor hist formatting so that it can be reused with the GTK browser, From Namhyung Kim - Fix build for another rbtree.c change, from Adrian Hunter. - Make 'perf diff' command work with evsel hists, from Jiri Olsa. - Use the only field_sep var that is set up: symbol_conf.field_sep, fix from Jiri Olsa. - .gitignore compiled python binaries, from Namhyung Kim. - Get rid of die() in more libtraceevent places, from Namhyung Kim. - Rename libtraceevent 'private' struct member to 'priv' so that it works in C++, from Steven Rostedt - Remove lots of exit()/die() calls from tools so that the main perf exit routine can take place, from David Ahern - Fix x86 build on x86-64, from David Ahern. - {int,str,rb}list fixes from Suzuki K Poulose - perf.data header fixes from Namhyung Kim - Allow user to indicate objdump path, needed in cross environments, from Maciek Borzecki - Fix hardware cache event name generation, fix from Jiri Olsa - Add round trip test for sw, hw and cache event names, catching the problem Jiri fixed, after Jiri's patch, the test passes successfully. - Clean target should do clean for lib/traceevent too, fix from David Ahern - Check the right variable for allocation failure, fix from Namhyung Kim - Set up evsel->tp_format regardless of evsel->name being set already, fix from Namhyung Kim - Oprofile fixes from Robert Richter. - Remove perf_event_attr needless version inflation, from Jiri Olsa - Introduce libtraceevent strerror like error reporting facility, from Namhyung Kim - Add pmu mappings to perf.data header and use event names from cmd line, from Robert Richter - Fix include order for bison/flex-generated C files, from Ben Hutchings - Build fixes and documentation corrections from David Ahern - Assorted cleanups from Robert Richter - Let O= makes handle relative paths, from Steven Rostedt - perf script python fixes, from Feng Tang. - Initial bash completion support, from Frederic Weisbecker - Allow building without libelf, from Namhyung Kim. - Support DWARF CFI based unwind to have callchains when %bp based unwinding is not possible, from Jiri Olsa. - Symbol resolution fixes, while fixing support PPC64 files with an .opt ELF section was the end goal, several fixes for code that handles all architectures and cleanups are included, from Cody Schafer. - Assorted fixes for Documentation and build in 32 bit, from Robert Richter - Cache the libtraceevent event_format associated to each evsel early, so that we avoid relookups, i.e. calling pevent_find_event repeatedly when processing tracepoint events. [ This is to reduce the surface contact with libtraceevents and make clear what is that the perf tools needs from that lib: so far parsing the common and per event fields. ] - Don't stop the build if the audit libraries are not installed, fix from Namhyung Kim. - Fix bfd.h/libbfd detection with recent binutils, from Markus Trippelsdorf. - Improve warning message when libunwind devel packages not present, from Jiri Olsa" * 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (282 commits) perf trace: Add aliases for some syscalls perf probe: Print an enum type variable in "enum variable-name" format when showing accessible variables perf tools: Check libaudit availability for perf-trace builtin perf hists: Add missing period_* fields when collapsing a hist entry perf trace: New tool perf evsel: Export the event_format constructor perf evsel: Introduce rawptr() method perf tools: Use perf_evsel__newtp in the event parser perf evsel: The tracepoint constructor should store sys:name perf evlist: Introduce set_filter() method perf evlist: Renane set_filters method to apply_filters perf test: Add test to check we correctly parse and match syscall open parms perf evsel: Handle endianity in intval method perf evsel: Know if byte swap is needed perf tools: Allow handling a NULL cpu_map as meaning "all cpus" perf evsel: Improve tracepoint constructor setup tools lib traceevent: Fix error path on pevent_parse_event perf test: Fix build failure trace: Move trace event enable from fs_initcall to core_initcall tracing: Add an option for disabling markers ...

Linus Torvalds committed 14 years ago

Commit 7e92daa2 parents 7a68294 1d787d3208 files · +13242 −5240Browse files

Unified · Split

208 files
  1. Makefile +5 −1
  2. arch/Kconfig +13 −0
  3. arch/x86/Kconfig +3 −0
  4. arch/x86/include/asm/ftrace.h +36 −20
  5. arch/x86/include/asm/kprobes.h +1 −0
  6. arch/x86/include/asm/kvm.h +16 −0
  7. arch/x86/include/asm/kvm_host.h +0 −16
  8. arch/x86/include/asm/perf_event.h +2 −0
  9. arch/x86/include/asm/perf_regs.h +33 −0
  10. arch/x86/include/asm/processor.h +2 −0
  11. arch/x86/include/asm/svm.h +130 −75
  12. arch/x86/include/asm/uprobes.h +2 −1
  13. arch/x86/include/asm/vmx.h +84 −43
  14. arch/x86/kernel/Makefile +2 −0
  15. arch/x86/kernel/cpu/perf_event_intel_uncore.c +25 −3
  16. arch/x86/kernel/cpu/perf_event_intel_uncore.h +4 −2
  17. arch/x86/kernel/entry_32.S +70 −4
  18. arch/x86/kernel/entry_64.S +116 −14
  19. arch/x86/kernel/ftrace.c +69 −4
  20. arch/x86/kernel/kprobes.c +67 −0
  21. arch/x86/kernel/perf_regs.c +105 −0
  22. arch/x86/kernel/step.c +33 −20
  23. arch/x86/kernel/uprobes.c +51 −1
  24. arch/x86/kernel/x8664_ksyms_64.c +5 −1
  25. arch/x86/kvm/trace.h +0 −89
  26. drivers/oprofile/cpu_buffer.c +3 −8
  27. include/linux/ftrace.h +148 −10
  28. include/linux/kprobes.h +27 −0
  29. include/linux/perf_event.h +55 −5
  30. include/linux/perf_regs.h +25 −0
  31. include/linux/sched.h +3 −0
  32. include/linux/uprobes.h +9 −6
  33. kernel/Makefile +1 −1
  34. kernel/events/callchain.c +24 −14
  35. kernel/events/core.c +214 −0
  36. kernel/events/internal.h +61 −21
  37. kernel/events/ring_buffer.c +8 −2
  38. kernel/events/uprobes.c +118 −130
  39. kernel/fork.c +1 −5
  40. kernel/kprobes.c +176 −71
  41. kernel/trace/Kconfig +10 −0
  42. kernel/trace/Makefile +3 −5
  43. kernel/trace/ftrace.c +244 −78
  44. kernel/trace/ring_buffer.c +2 −2
  45. kernel/trace/trace.c +8 −4
  46. kernel/trace/trace.h +2 −1
  47. kernel/trace/trace_event_perf.c +2 −1
  48. kernel/trace/trace_events.c +78 −38
  49. kernel/trace/trace_events_filter.c +1 −1
  50. kernel/trace/trace_functions.c +9 −5
  51. kernel/trace/trace_functions_graph.c +4 −1
  52. kernel/trace/trace_irqsoff.c +3 −2
  53. kernel/trace/trace_sched_wakeup.c +3 −2
  54. kernel/trace/trace_selftest.c +283 −21
  55. kernel/trace/trace_stack.c +3 −1
  56. kernel/trace/trace_syscalls.c +1 −1
  57. scripts/recordmcount.h +3 −1
  58. tools/lib/traceevent/Makefile +1 −1
  59. tools/lib/traceevent/event-parse.c +548 −206
  60. tools/lib/traceevent/event-parse.h +40 −6
  61. tools/lib/traceevent/event-utils.h +6 −0
  62. tools/perf/.gitignore +2 −0
  63. tools/perf/Documentation/Makefile +3 −3
  64. tools/perf/Documentation/jit-interface.txt +15 −0
  65. tools/perf/Documentation/perf-annotate.txt +3 −0
  66. tools/perf/Documentation/perf-diff.txt +3 −0
  67. tools/perf/Documentation/perf-kvm.txt +28 −2
  68. tools/perf/Documentation/perf-list.txt +36 −12
  69. tools/perf/Documentation/perf-report.txt +3 −0
  70. tools/perf/Documentation/perf-script-perl.txt +2 −2
  71. tools/perf/Documentation/perf-script-python.txt +5 −5
  72. tools/perf/Documentation/perf-trace.txt +53 −0
  73. tools/perf/MANIFEST +4 −0
  74. tools/perf/Makefile +139 −34
  75. tools/perf/arch/x86/Makefile +3 −0
  76. tools/perf/arch/x86/include/perf_regs.h +80 −0
  77. tools/perf/arch/x86/util/unwind.c +111 −0
  78. tools/perf/bash_completion +26 −0
  79. tools/perf/bench/bench.h +2 −1
  80. tools/perf/bench/mem-memcpy.c +1 −1
  81. tools/perf/bench/mem-memset.c +1 −1
  82. tools/perf/bench/sched-messaging.c +1 −1
  83. tools/perf/bench/sched-pipe.c +5 −5
  84. tools/perf/builtin-annotate.c +3 −1
  85. tools/perf/builtin-bench.c +1 −1
  86. tools/perf/builtin-buildid-cache.c +6 −4
  87. tools/perf/builtin-buildid-list.c +3 −4
  88. tools/perf/builtin-diff.c +63 −33
  89. tools/perf/builtin-evlist.c +1 −1
  90. tools/perf/builtin-help.c +35 −15
  91. tools/perf/builtin-inject.c +17 −12
  92. tools/perf/builtin-kmem.c +120 −114
  93. tools/perf/builtin-kvm.c +834 −4
  94. tools/perf/builtin-list.c +9 −7
  95. tools/perf/builtin-lock.c +203 −211
  96. tools/perf/builtin-probe.c +12 −12
  97. tools/perf/builtin-record.c +237 −68
  98. tools/perf/builtin-report.c +26 −21
  99. tools/perf/builtin-sched.c +683 −839
  100. tools/perf/builtin-script.c +140 −89
  101. tools/perf/builtin-stat.c +54 −82
  102. tools/perf/builtin-test.c +337 −16
  103. tools/perf/builtin-timechart.c +30 −40
  104. tools/perf/builtin-top.c +16 −17
  105. tools/perf/builtin-trace.c +310 −0
  106. tools/perf/builtin.h +2 −0
  107. tools/perf/command-list.txt +2 −1
  108. tools/perf/config/feature-tests.mak +50 −0
  109. tools/perf/perf-archive.sh +3 −3
  110. tools/perf/perf.c +45 −30
  111. tools/perf/perf.h +8 −1
  112. tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py +94 −0
  113. tools/perf/scripts/python/bin/event_analyzing_sample-record +8 −0
  114. tools/perf/scripts/python/bin/event_analyzing_sample-report +3 −0
  115. tools/perf/scripts/python/event_analyzing_sample.py +189 −0
  116. tools/perf/ui/browser.c +4 −3
  117. tools/perf/ui/browsers/annotate.c +4 −2
  118. tools/perf/ui/browsers/hists.c +99 −34
  119. tools/perf/ui/gtk/browser.c +92 −19
  120. tools/perf/ui/gtk/gtk.h +3 −0
  121. tools/perf/ui/gtk/helpline.c +56 −0
  122. tools/perf/ui/gtk/setup.c +5 −1
  123. tools/perf/ui/gtk/util.c +2 −7
  124. tools/perf/ui/helpline.c +19 −37
  125. tools/perf/ui/helpline.h +32 −1
  126. tools/perf/ui/hist.c +390 −0
  127. tools/perf/ui/setup.c +8 −2
  128. tools/perf/ui/stdio/hist.c +498 −0
  129. tools/perf/ui/tui/helpline.c +57 −0
  130. tools/perf/ui/tui/setup.c +6 −4
  131. tools/perf/util/alias.c +2 −1
  132. tools/perf/util/annotate.c +14 −5
  133. tools/perf/util/annotate.h +9 −6
  134. tools/perf/util/build-id.c +6 −5
  135. tools/perf/util/cache.h +3 −3
  136. tools/perf/util/callchain.c +3 −3
  137. tools/perf/util/cgroup.c +2 −2
  138. tools/perf/util/config.c +4 −2
  139. tools/perf/util/cpumap.c +15 −7
  140. tools/perf/util/cpumap.h +12 −1
  141. tools/perf/util/debug.c +3 −1
  142. tools/perf/util/debug.h +6 −11
  143. tools/perf/util/dso-test-data.c +1 −1
  144. tools/perf/util/dwarf-aux.c +2 −0
  145. tools/perf/util/event.c +45 −26
  146. tools/perf/util/event.h +13 −1
  147. tools/perf/util/evlist.c +82 −61
  148. tools/perf/util/evlist.h +25 −10
  149. tools/perf/util/evsel.c +248 −28
  150. tools/perf/util/evsel.h +53 −10
  151. tools/perf/util/generate-cmdlist.sh +15 −0
  152. tools/perf/util/header.c +762 −301
  153. tools/perf/util/header.h +27 −2
  154. tools/perf/util/help.c +3 −1
  155. tools/perf/util/hist.c +26 −695
  156. tools/perf/util/hist.h +58 −17
  157. tools/perf/util/include/linux/bitops.h +4 −0
  158. tools/perf/util/include/linux/compiler.h +8 −1
  159. tools/perf/util/include/linux/kernel.h +15 −2
  160. tools/perf/util/include/linux/magic.h +12 −0
  161. tools/perf/util/include/linux/rbtree.h +1 −0
  162. tools/perf/util/include/linux/string.h +2 −0
  163. tools/perf/util/include/linux/types.h +8 −0
  164. tools/perf/util/intlist.c +4 −4
  165. tools/perf/util/map.c +32 −15
  166. tools/perf/util/map.h +6 −3
  167. tools/perf/util/parse-events-test.c +346 −78
  168. tools/perf/util/parse-events.c +173 −81
  169. tools/perf/util/parse-events.h +11 −7
  170. tools/perf/util/parse-events.l +44 −12
  171. tools/perf/util/parse-events.y +104 −21
  172. tools/perf/util/parse-options.c +2 −1
  173. tools/perf/util/perf_regs.h +14 −0
  174. tools/perf/util/pmu.c +78 −2
  175. tools/perf/util/pmu.h +3 −0
  176. tools/perf/util/pmu.y +3 −3
  177. tools/perf/util/probe-event.c +50 −19
  178. tools/perf/util/probe-finder.c +18 −10
  179. tools/perf/util/python.c +11 −6
  180. tools/perf/util/scripting-engines/trace-event-perl.c +23 −27
  181. tools/perf/util/scripting-engines/trace-event-python.c +98 −15
  182. tools/perf/util/session.c +144 −54
  183. tools/perf/util/session.h +4 −6
  184. tools/perf/util/sort.c +16 −9
  185. tools/perf/util/sort.h +1 −1
  186. tools/perf/util/stat.c +57 −0
  187. tools/perf/util/stat.h +16 −0
  188. tools/perf/util/string.c +17 −1
  189. tools/perf/util/strlist.c +1 −1
  190. tools/perf/util/symbol-elf.c +841 −0
  191. tools/perf/util/symbol-minimal.c +307 −0
  192. tools/perf/util/symbol.c +91 −851
  193. tools/perf/util/symbol.h +63 −4
  194. tools/perf/util/target.c +2 −2
  195. tools/perf/util/thread.h +2 −0
  196. tools/perf/util/top.c +1 −2
  197. tools/perf/util/top.h +1 −0
  198. tools/perf/util/trace-event-parse.c +28 −26
  199. tools/perf/util/trace-event-scripting.c +18 −16
  200. tools/perf/util/trace-event.h +8 −4
  201. tools/perf/util/unwind.c +571 −0
  202. tools/perf/util/unwind.h +35 −0
  203. tools/perf/util/util.c +25 −0
  204. tools/perf/util/util.h +3 −6
  205. tools/perf/util/vdso.c +111 −0
  206. tools/perf/util/vdso.h +18 −0
  207. tools/perf/util/wrapper.c +2 −1
  208. tools/scripts/Makefile.include +4 −2
Makefile+5−1
@@ -609,7 +609,11 @@ KBUILD_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) · 608 unmodified lines
609609 endif
610610
611611 ifdef CONFIG_FUNCTION_TRACER
612−KBUILD_CFLAGS += -pg
612+ifdef CONFIG_HAVE_FENTRY
613+CC_USING_FENTRY := $(call cc-option, -mfentry -DCC_USING_FENTRY)
614+endif
615+KBUILD_CFLAGS += -pg $(CC_USING_FENTRY)
616+KBUILD_AFLAGS += $(CC_USING_FENTRY)
613617 ifdef CONFIG_DYNAMIC_FTRACE
614618 ifdef CONFIG_HAVE_C_RECORDMCOUNT
615619 BUILD_C_RECORDMCOUNT := y
arch/Kconfig+13−0
@@ -222,6 +222,19 @@ config HAVE_PERF_EVENTS_NMI · 221 unmodified lines
222222 subsystem. Also has support for calculating CPU cycle events
223223 to determine how many clock cycles in a given period.
224224
225+config HAVE_PERF_REGS
226+ bool
227+ help
228+ Support selective register dumps for perf events. This includes
229+ bit-mapping of each registers and a unique architecture id.
230+
231+config HAVE_PERF_USER_STACK_DUMP
232+ bool
233+ help
234+ Support user stack dumps for perf event samples. This needs
235+ access to the user stack pointer which is not unified across
236+ architectures.
237+
225238 config HAVE_ARCH_JUMP_LABEL
226239 bool
227240
arch/x86/Kconfig+3−0
@@ -36,6 +36,7 @@ config X86 · 35 unmodified lines
3636 select HAVE_KRETPROBES
3737 select HAVE_OPTPROBES
3838 select HAVE_FTRACE_MCOUNT_RECORD
39+ select HAVE_FENTRY if X86_64
3940 select HAVE_C_RECORDMCOUNT
4041 select HAVE_DYNAMIC_FTRACE
4142 select HAVE_FUNCTION_TRACER
@@ -60,6 +61,8 @@ config X86 · 18 unmodified lines
6061 select HAVE_MIXED_BREAKPOINTS_REGS
6162 select PERF_EVENTS
6263 select HAVE_PERF_EVENTS_NMI
64+ select HAVE_PERF_REGS
65+ select HAVE_PERF_USER_STACK_DUMP
6366 select ANON_INODES
6467 select HAVE_ALIGNED_STRUCT_PAGE if SLUB && !M386
6568 select HAVE_CMPXCHG_LOCAL if !M386
arch/x86/include/asm/ftrace.h+36−20
@@ -3,38 +3,54 @@ · 2 unmodified lines
33
44 #ifdef __ASSEMBLY__
55
6− .macro MCOUNT_SAVE_FRAME
7− /* taken from glibc */
8− subq $0x38, %rsp
9− movq %rax, (%rsp)
10− movq %rcx, 8(%rsp)
11− movq %rdx, 16(%rsp)
12− movq %rsi, 24(%rsp)
13− movq %rdi, 32(%rsp)
14− movq %r8, 40(%rsp)
15− movq %r9, 48(%rsp)
6+ /* skip is set if the stack was already partially adjusted */
7+ .macro MCOUNT_SAVE_FRAME skip=0
8+ /*
9+ * We add enough stack to save all regs.
10+ */
11+ subq $(SS+8-\skip), %rsp
12+ movq %rax, RAX(%rsp)
13+ movq %rcx, RCX(%rsp)
14+ movq %rdx, RDX(%rsp)
15+ movq %rsi, RSI(%rsp)
16+ movq %rdi, RDI(%rsp)
17+ movq %r8, R8(%rsp)
18+ movq %r9, R9(%rsp)
19+ /* Move RIP to its proper location */
20+ movq SS+8(%rsp), %rdx
21+ movq %rdx, RIP(%rsp)
1622 .endm
1723
18− .macro MCOUNT_RESTORE_FRAME
19− movq 48(%rsp), %r9
20− movq 40(%rsp), %r8
21− movq 32(%rsp), %rdi
22− movq 24(%rsp), %rsi
23− movq 16(%rsp), %rdx
24− movq 8(%rsp), %rcx
25− movq (%rsp), %rax
26− addq $0x38, %rsp
24+ .macro MCOUNT_RESTORE_FRAME skip=0
25+ movq R9(%rsp), %r9
26+ movq R8(%rsp), %r8
27+ movq RDI(%rsp), %rdi
28+ movq RSI(%rsp), %rsi
29+ movq RDX(%rsp), %rdx
30+ movq RCX(%rsp), %rcx
31+ movq RAX(%rsp), %rax
32+ addq $(SS+8-\skip), %rsp
2733 .endm
2834
2935 #endif
3036
3137 #ifdef CONFIG_FUNCTION_TRACER
32−#define MCOUNT_ADDR ((long)(mcount))
38+#ifdef CC_USING_FENTRY
39+# define MCOUNT_ADDR ((long)(__fentry__))
40+#else
41+# define MCOUNT_ADDR ((long)(mcount))
42+#endif
3343 #define MCOUNT_INSN_SIZE 5 /* sizeof mcount call */
3444
45+#ifdef CONFIG_DYNAMIC_FTRACE
46+#define ARCH_SUPPORTS_FTRACE_OPS 1
47+#define ARCH_SUPPORTS_FTRACE_SAVE_REGS
48+#endif
49+
3550 #ifndef __ASSEMBLY__
3651 extern void mcount(void);
3752 extern atomic_t modifying_ftrace_code;
53+extern void __fentry__(void);
3854
3955 static inline unsigned long ftrace_call_adjust(unsigned long addr)
4056 {
arch/x86/include/asm/kprobes.h+1−0
@@ -27,6 +27,7 @@ · 26 unmodified lines
2727 #include <asm/insn.h>
2828
2929 #define __ARCH_WANT_KPROBES_INSN_SLOT
30+#define ARCH_SUPPORTS_KPROBES_ON_FTRACE
3031
3132 struct pt_regs;
3233 struct kprobe;
arch/x86/include/asm/kvm.h+16−0
@@ -9,6 +9,22 @@ · 8 unmodified lines
99 #include <linux/types.h>
1010 #include <linux/ioctl.h>
1111
12+#define DE_VECTOR 0
13+#define DB_VECTOR 1
14+#define BP_VECTOR 3
15+#define OF_VECTOR 4
16+#define BR_VECTOR 5
17+#define UD_VECTOR 6
18+#define NM_VECTOR 7
19+#define DF_VECTOR 8
20+#define TS_VECTOR 10
21+#define NP_VECTOR 11
22+#define SS_VECTOR 12
23+#define GP_VECTOR 13
24+#define PF_VECTOR 14
25+#define MF_VECTOR 16
26+#define MC_VECTOR 18
27+
1228 /* Select x86 specific features in <linux/kvm.h> */
1329 #define __KVM_HAVE_PIT
1430 #define __KVM_HAVE_IOAPIC
arch/x86/include/asm/kvm_host.h+0−16
@@ -75,22 +75,6 @@ · 74 unmodified lines
7575 #define KVM_HPAGE_MASK(x) (~(KVM_HPAGE_SIZE(x) - 1))
7676 #define KVM_PAGES_PER_HPAGE(x) (KVM_HPAGE_SIZE(x) / PAGE_SIZE)
7777
78−#define DE_VECTOR 0
79−#define DB_VECTOR 1
80−#define BP_VECTOR 3
81−#define OF_VECTOR 4
82−#define BR_VECTOR 5
83−#define UD_VECTOR 6
84−#define NM_VECTOR 7
85−#define DF_VECTOR 8
86−#define TS_VECTOR 10
87−#define NP_VECTOR 11
88−#define SS_VECTOR 12
89−#define GP_VECTOR 13
90−#define PF_VECTOR 14
91−#define MF_VECTOR 16
92−#define MC_VECTOR 18
93−
9478 #define SELECTOR_TI_MASK (1 << 2)
9579 #define SELECTOR_RPL_MASK 0x03
9680
arch/x86/include/asm/perf_event.h+2−0
@@ -262,4 +262,6 @@ static inline void perf_check_microcode(void) { } · 261 unmodified lines
262262 static inline void amd_pmu_disable_virt(void) { }
263263 #endif
264264
265+#define arch_perf_out_copy_user copy_from_user_nmi
266+
265267 #endif /* _ASM_X86_PERF_EVENT_H */
arch/x86/include/asm/perf_regs.h+33−0
@@ -0,0 +1,33 @@
1+#ifndef _ASM_X86_PERF_REGS_H
2+#define _ASM_X86_PERF_REGS_H
3+
4+enum perf_event_x86_regs {
5+ PERF_REG_X86_AX,
6+ PERF_REG_X86_BX,
7+ PERF_REG_X86_CX,
8+ PERF_REG_X86_DX,
9+ PERF_REG_X86_SI,
10+ PERF_REG_X86_DI,
11+ PERF_REG_X86_BP,
12+ PERF_REG_X86_SP,
13+ PERF_REG_X86_IP,
14+ PERF_REG_X86_FLAGS,
15+ PERF_REG_X86_CS,
16+ PERF_REG_X86_SS,
17+ PERF_REG_X86_DS,
18+ PERF_REG_X86_ES,
19+ PERF_REG_X86_FS,
20+ PERF_REG_X86_GS,
21+ PERF_REG_X86_R8,
22+ PERF_REG_X86_R9,
23+ PERF_REG_X86_R10,
24+ PERF_REG_X86_R11,
25+ PERF_REG_X86_R12,
26+ PERF_REG_X86_R13,
27+ PERF_REG_X86_R14,
28+ PERF_REG_X86_R15,
29+
30+ PERF_REG_X86_32_MAX = PERF_REG_X86_GS + 1,
31+ PERF_REG_X86_64_MAX = PERF_REG_X86_R15 + 1,
32+};
33+#endif /* _ASM_X86_PERF_REGS_H */
arch/x86/include/asm/processor.h+2−0
@@ -759,6 +759,8 @@ static inline void update_debugctlmsr(unsigned long debugctlmsr) · 758 unmodified lines
759759 wrmsrl(MSR_IA32_DEBUGCTLMSR, debugctlmsr);
760760 }
761761
762+extern void set_task_blockstep(struct task_struct *task, bool on);
763+
762764 /*
763765 * from system description table in BIOS. Mostly for MCA use, but
764766 * others may find it useful:
arch/x86/include/asm/svm.h+130−75
@@ -1,6 +1,135 @@
11 #ifndef __SVM_H
22 #define __SVM_H
33
4+#define SVM_EXIT_READ_CR0 0x000
5+#define SVM_EXIT_READ_CR3 0x003
6+#define SVM_EXIT_READ_CR4 0x004
7+#define SVM_EXIT_READ_CR8 0x008
8+#define SVM_EXIT_WRITE_CR0 0x010
9+#define SVM_EXIT_WRITE_CR3 0x013
10+#define SVM_EXIT_WRITE_CR4 0x014
11+#define SVM_EXIT_WRITE_CR8 0x018
12+#define SVM_EXIT_READ_DR0 0x020
13+#define SVM_EXIT_READ_DR1 0x021
14+#define SVM_EXIT_READ_DR2 0x022
15+#define SVM_EXIT_READ_DR3 0x023
16+#define SVM_EXIT_READ_DR4 0x024
17+#define SVM_EXIT_READ_DR5 0x025
18+#define SVM_EXIT_READ_DR6 0x026
19+#define SVM_EXIT_READ_DR7 0x027
20+#define SVM_EXIT_WRITE_DR0 0x030
21+#define SVM_EXIT_WRITE_DR1 0x031
22+#define SVM_EXIT_WRITE_DR2 0x032
23+#define SVM_EXIT_WRITE_DR3 0x033
24+#define SVM_EXIT_WRITE_DR4 0x034
25+#define SVM_EXIT_WRITE_DR5 0x035
26+#define SVM_EXIT_WRITE_DR6 0x036
27+#define SVM_EXIT_WRITE_DR7 0x037
28+#define SVM_EXIT_EXCP_BASE 0x040
29+#define SVM_EXIT_INTR 0x060
30+#define SVM_EXIT_NMI 0x061
31+#define SVM_EXIT_SMI 0x062
32+#define SVM_EXIT_INIT 0x063
33+#define SVM_EXIT_VINTR 0x064
34+#define SVM_EXIT_CR0_SEL_WRITE 0x065
35+#define SVM_EXIT_IDTR_READ 0x066
36+#define SVM_EXIT_GDTR_READ 0x067
37+#define SVM_EXIT_LDTR_READ 0x068
38+#define SVM_EXIT_TR_READ 0x069
39+#define SVM_EXIT_IDTR_WRITE 0x06a
40+#define SVM_EXIT_GDTR_WRITE 0x06b
41+#define SVM_EXIT_LDTR_WRITE 0x06c
42+#define SVM_EXIT_TR_WRITE 0x06d
43+#define SVM_EXIT_RDTSC 0x06e
44+#define SVM_EXIT_RDPMC 0x06f
45+#define SVM_EXIT_PUSHF 0x070
46+#define SVM_EXIT_POPF 0x071
47+#define SVM_EXIT_CPUID 0x072
48+#define SVM_EXIT_RSM 0x073
49+#define SVM_EXIT_IRET 0x074
50+#define SVM_EXIT_SWINT 0x075
51+#define SVM_EXIT_INVD 0x076
52+#define SVM_EXIT_PAUSE 0x077
53+#define SVM_EXIT_HLT 0x078
54+#define SVM_EXIT_INVLPG 0x079
55+#define SVM_EXIT_INVLPGA 0x07a
56+#define SVM_EXIT_IOIO 0x07b
57+#define SVM_EXIT_MSR 0x07c
58+#define SVM_EXIT_TASK_SWITCH 0x07d
59+#define SVM_EXIT_FERR_FREEZE 0x07e
60+#define SVM_EXIT_SHUTDOWN 0x07f
61+#define SVM_EXIT_VMRUN 0x080
62+#define SVM_EXIT_VMMCALL 0x081
63+#define SVM_EXIT_VMLOAD 0x082
64+#define SVM_EXIT_VMSAVE 0x083
65+#define SVM_EXIT_STGI 0x084
66+#define SVM_EXIT_CLGI 0x085
67+#define SVM_EXIT_SKINIT 0x086
68+#define SVM_EXIT_RDTSCP 0x087
69+#define SVM_EXIT_ICEBP 0x088
70+#define SVM_EXIT_WBINVD 0x089
71+#define SVM_EXIT_MONITOR 0x08a
72+#define SVM_EXIT_MWAIT 0x08b
73+#define SVM_EXIT_MWAIT_COND 0x08c
74+#define SVM_EXIT_XSETBV 0x08d
75+#define SVM_EXIT_NPF 0x400
76+
77+#define SVM_EXIT_ERR -1
78+
79+#define SVM_EXIT_REASONS \
80+ { SVM_EXIT_READ_CR0, "read_cr0" }, \
81+ { SVM_EXIT_READ_CR3, "read_cr3" }, \
82+ { SVM_EXIT_READ_CR4, "read_cr4" }, \
83+ { SVM_EXIT_READ_CR8, "read_cr8" }, \
84+ { SVM_EXIT_WRITE_CR0, "write_cr0" }, \
85+ { SVM_EXIT_WRITE_CR3, "write_cr3" }, \
86+ { SVM_EXIT_WRITE_CR4, "write_cr4" }, \
87+ { SVM_EXIT_WRITE_CR8, "write_cr8" }, \
88+ { SVM_EXIT_READ_DR0, "read_dr0" }, \
89+ { SVM_EXIT_READ_DR1, "read_dr1" }, \
90+ { SVM_EXIT_READ_DR2, "read_dr2" }, \
91+ { SVM_EXIT_READ_DR3, "read_dr3" }, \
92+ { SVM_EXIT_WRITE_DR0, "write_dr0" }, \
93+ { SVM_EXIT_WRITE_DR1, "write_dr1" }, \
94+ { SVM_EXIT_WRITE_DR2, "write_dr2" }, \
95+ { SVM_EXIT_WRITE_DR3, "write_dr3" }, \
96+ { SVM_EXIT_WRITE_DR5, "write_dr5" }, \
97+ { SVM_EXIT_WRITE_DR7, "write_dr7" }, \
98+ { SVM_EXIT_EXCP_BASE + DB_VECTOR, "DB excp" }, \
99+ { SVM_EXIT_EXCP_BASE + BP_VECTOR, "BP excp" }, \
100+ { SVM_EXIT_EXCP_BASE + UD_VECTOR, "UD excp" }, \
101+ { SVM_EXIT_EXCP_BASE + PF_VECTOR, "PF excp" }, \
102+ { SVM_EXIT_EXCP_BASE + NM_VECTOR, "NM excp" }, \
103+ { SVM_EXIT_EXCP_BASE + MC_VECTOR, "MC excp" }, \
104+ { SVM_EXIT_INTR, "interrupt" }, \
105+ { SVM_EXIT_NMI, "nmi" }, \
106+ { SVM_EXIT_SMI, "smi" }, \
107+ { SVM_EXIT_INIT, "init" }, \
108+ { SVM_EXIT_VINTR, "vintr" }, \
109+ { SVM_EXIT_CPUID, "cpuid" }, \
110+ { SVM_EXIT_INVD, "invd" }, \
111+ { SVM_EXIT_HLT, "hlt" }, \
112+ { SVM_EXIT_INVLPG, "invlpg" }, \
113+ { SVM_EXIT_INVLPGA, "invlpga" }, \
114+ { SVM_EXIT_IOIO, "io" }, \
115+ { SVM_EXIT_MSR, "msr" }, \
116+ { SVM_EXIT_TASK_SWITCH, "task_switch" }, \
117+ { SVM_EXIT_SHUTDOWN, "shutdown" }, \
118+ { SVM_EXIT_VMRUN, "vmrun" }, \
119+ { SVM_EXIT_VMMCALL, "hypercall" }, \
120+ { SVM_EXIT_VMLOAD, "vmload" }, \
121+ { SVM_EXIT_VMSAVE, "vmsave" }, \
122+ { SVM_EXIT_STGI, "stgi" }, \
123+ { SVM_EXIT_CLGI, "clgi" }, \
124+ { SVM_EXIT_SKINIT, "skinit" }, \
125+ { SVM_EXIT_WBINVD, "wbinvd" }, \
126+ { SVM_EXIT_MONITOR, "monitor" }, \
127+ { SVM_EXIT_MWAIT, "mwait" }, \
128+ { SVM_EXIT_XSETBV, "xsetbv" }, \
129+ { SVM_EXIT_NPF, "npf" }
130+
131+#ifdef __KERNEL__
132+
4133 enum {
5134 INTERCEPT_INTR,
6135 INTERCEPT_NMI,
@@ -264,81 +393,6 @@ struct __attribute__ ((__packed__)) vmcb { · 257 unmodified lines
264393
265394 #define SVM_EXITINFO_REG_MASK 0x0F
266395
267−#define SVM_EXIT_READ_CR0 0x000
268−#define SVM_EXIT_READ_CR3 0x003
269−#define SVM_EXIT_READ_CR4 0x004
270−#define SVM_EXIT_READ_CR8 0x008
271−#define SVM_EXIT_WRITE_CR0 0x010
272−#define SVM_EXIT_WRITE_CR3 0x013
273−#define SVM_EXIT_WRITE_CR4 0x014
274−#define SVM_EXIT_WRITE_CR8 0x018
275−#define SVM_EXIT_READ_DR0 0x020
276−#define SVM_EXIT_READ_DR1 0x021
277−#define SVM_EXIT_READ_DR2 0x022
278−#define SVM_EXIT_READ_DR3 0x023
279−#define SVM_EXIT_READ_DR4 0x024
280−#define SVM_EXIT_READ_DR5 0x025
281−#define SVM_EXIT_READ_DR6 0x026
282−#define SVM_EXIT_READ_DR7 0x027
283−#define SVM_EXIT_WRITE_DR0 0x030
284−#define SVM_EXIT_WRITE_DR1 0x031
285−#define SVM_EXIT_WRITE_DR2 0x032
286−#define SVM_EXIT_WRITE_DR3 0x033
287−#define SVM_EXIT_WRITE_DR4 0x034
288−#define SVM_EXIT_WRITE_DR5 0x035
289−#define SVM_EXIT_WRITE_DR6 0x036
290−#define SVM_EXIT_WRITE_DR7 0x037
291−#define SVM_EXIT_EXCP_BASE 0x040
292−#define SVM_EXIT_INTR 0x060
293−#define SVM_EXIT_NMI 0x061
294−#define SVM_EXIT_SMI 0x062
295−#define SVM_EXIT_INIT 0x063
296−#define SVM_EXIT_VINTR 0x064
297−#define SVM_EXIT_CR0_SEL_WRITE 0x065
298−#define SVM_EXIT_IDTR_READ 0x066
299−#define SVM_EXIT_GDTR_READ 0x067
300−#define SVM_EXIT_LDTR_READ 0x068
301−#define SVM_EXIT_TR_READ 0x069
302−#define SVM_EXIT_IDTR_WRITE 0x06a
303−#define SVM_EXIT_GDTR_WRITE 0x06b
304−#define SVM_EXIT_LDTR_WRITE 0x06c
305−#define SVM_EXIT_TR_WRITE 0x06d
306−#define SVM_EXIT_RDTSC 0x06e
307−#define SVM_EXIT_RDPMC 0x06f
308−#define SVM_EXIT_PUSHF 0x070
309−#define SVM_EXIT_POPF 0x071
310−#define SVM_EXIT_CPUID 0x072
311−#define SVM_EXIT_RSM 0x073
312−#define SVM_EXIT_IRET 0x074
313−#define SVM_EXIT_SWINT 0x075
314−#define SVM_EXIT_INVD 0x076
315−#define SVM_EXIT_PAUSE 0x077
316−#define SVM_EXIT_HLT 0x078
317−#define SVM_EXIT_INVLPG 0x079
318−#define SVM_EXIT_INVLPGA 0x07a
319−#define SVM_EXIT_IOIO 0x07b
320−#define SVM_EXIT_MSR 0x07c
321−#define SVM_EXIT_TASK_SWITCH 0x07d
322−#define SVM_EXIT_FERR_FREEZE 0x07e
323−#define SVM_EXIT_SHUTDOWN 0x07f
324−#define SVM_EXIT_VMRUN 0x080
325−#define SVM_EXIT_VMMCALL 0x081
326−#define SVM_EXIT_VMLOAD 0x082
327−#define SVM_EXIT_VMSAVE 0x083
328−#define SVM_EXIT_STGI 0x084
329−#define SVM_EXIT_CLGI 0x085
330−#define SVM_EXIT_SKINIT 0x086
331−#define SVM_EXIT_RDTSCP 0x087
332−#define SVM_EXIT_ICEBP 0x088
333−#define SVM_EXIT_WBINVD 0x089
334−#define SVM_EXIT_MONITOR 0x08a
335−#define SVM_EXIT_MWAIT 0x08b
336−#define SVM_EXIT_MWAIT_COND 0x08c
337−#define SVM_EXIT_XSETBV 0x08d
338−#define SVM_EXIT_NPF 0x400
339−
340−#define SVM_EXIT_ERR -1
341−
342396 #define SVM_CR0_SELECTIVE_MASK (X86_CR0_TS | X86_CR0_MP)
343397
344398 #define SVM_VMLOAD ".byte 0x0f, 0x01, 0xda"
@@ -350,3 +404,4 @@ struct __attribute__ ((__packed__)) vmcb { · 5 unmodified lines
350404
351405 #endif
352406
407+#endif
arch/x86/include/asm/uprobes.h+2−1
@@ -42,10 +42,11 @@ struct arch_uprobe { · 41 unmodified lines
4242 };
4343
4444 struct arch_uprobe_task {
45− unsigned long saved_trap_nr;
4645 #ifdef CONFIG_X86_64
4746 unsigned long saved_scratch_register;
4847 #endif
48+ unsigned int saved_trap_nr;
49+ unsigned int saved_tf;
4950 };
5051
5152 extern int arch_uprobe_analyze_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long addr);
arch/x86/include/asm/vmx.h+84−43
@@ -25,6 +25,88 @@ · 24 unmodified lines
2525 *
2626 */
2727
28+#define VMX_EXIT_REASONS_FAILED_VMENTRY 0x80000000
29+
30+#define EXIT_REASON_EXCEPTION_NMI 0
31+#define EXIT_REASON_EXTERNAL_INTERRUPT 1
32+#define EXIT_REASON_TRIPLE_FAULT 2
33+
34+#define EXIT_REASON_PENDING_INTERRUPT 7
35+#define EXIT_REASON_NMI_WINDOW 8
36+#define EXIT_REASON_TASK_SWITCH 9
37+#define EXIT_REASON_CPUID 10
38+#define EXIT_REASON_HLT 12
39+#define EXIT_REASON_INVD 13
40+#define EXIT_REASON_INVLPG 14
41+#define EXIT_REASON_RDPMC 15
42+#define EXIT_REASON_RDTSC 16
43+#define EXIT_REASON_VMCALL 18
44+#define EXIT_REASON_VMCLEAR 19
45+#define EXIT_REASON_VMLAUNCH 20
46+#define EXIT_REASON_VMPTRLD 21
47+#define EXIT_REASON_VMPTRST 22
48+#define EXIT_REASON_VMREAD 23
49+#define EXIT_REASON_VMRESUME 24
50+#define EXIT_REASON_VMWRITE 25
51+#define EXIT_REASON_VMOFF 26
52+#define EXIT_REASON_VMON 27
53+#define EXIT_REASON_CR_ACCESS 28
54+#define EXIT_REASON_DR_ACCESS 29
55+#define EXIT_REASON_IO_INSTRUCTION 30
56+#define EXIT_REASON_MSR_READ 31
57+#define EXIT_REASON_MSR_WRITE 32
58+#define EXIT_REASON_INVALID_STATE 33
59+#define EXIT_REASON_MWAIT_INSTRUCTION 36
60+#define EXIT_REASON_MONITOR_INSTRUCTION 39
61+#define EXIT_REASON_PAUSE_INSTRUCTION 40
62+#define EXIT_REASON_MCE_DURING_VMENTRY 41
63+#define EXIT_REASON_TPR_BELOW_THRESHOLD 43
64+#define EXIT_REASON_APIC_ACCESS 44
65+#define EXIT_REASON_EPT_VIOLATION 48
66+#define EXIT_REASON_EPT_MISCONFIG 49
67+#define EXIT_REASON_WBINVD 54
68+#define EXIT_REASON_XSETBV 55
69+#define EXIT_REASON_INVPCID 58
70+
71+#define VMX_EXIT_REASONS \
72+ { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \
73+ { EXIT_REASON_EXTERNAL_INTERRUPT, "EXTERNAL_INTERRUPT" }, \
74+ { EXIT_REASON_TRIPLE_FAULT, "TRIPLE_FAULT" }, \
75+ { EXIT_REASON_PENDING_INTERRUPT, "PENDING_INTERRUPT" }, \
76+ { EXIT_REASON_NMI_WINDOW, "NMI_WINDOW" }, \
77+ { EXIT_REASON_TASK_SWITCH, "TASK_SWITCH" }, \
78+ { EXIT_REASON_CPUID, "CPUID" }, \
79+ { EXIT_REASON_HLT, "HLT" }, \
80+ { EXIT_REASON_INVLPG, "INVLPG" }, \
81+ { EXIT_REASON_RDPMC, "RDPMC" }, \
82+ { EXIT_REASON_RDTSC, "RDTSC" }, \
83+ { EXIT_REASON_VMCALL, "VMCALL" }, \
84+ { EXIT_REASON_VMCLEAR, "VMCLEAR" }, \
85+ { EXIT_REASON_VMLAUNCH, "VMLAUNCH" }, \
86+ { EXIT_REASON_VMPTRLD, "VMPTRLD" }, \
87+ { EXIT_REASON_VMPTRST, "VMPTRST" }, \
88+ { EXIT_REASON_VMREAD, "VMREAD" }, \
89+ { EXIT_REASON_VMRESUME, "VMRESUME" }, \
90+ { EXIT_REASON_VMWRITE, "VMWRITE" }, \
91+ { EXIT_REASON_VMOFF, "VMOFF" }, \
92+ { EXIT_REASON_VMON, "VMON" }, \
93+ { EXIT_REASON_CR_ACCESS, "CR_ACCESS" }, \
94+ { EXIT_REASON_DR_ACCESS, "DR_ACCESS" }, \
95+ { EXIT_REASON_IO_INSTRUCTION, "IO_INSTRUCTION" }, \
96+ { EXIT_REASON_MSR_READ, "MSR_READ" }, \
97+ { EXIT_REASON_MSR_WRITE, "MSR_WRITE" }, \
98+ { EXIT_REASON_MWAIT_INSTRUCTION, "MWAIT_INSTRUCTION" }, \
99+ { EXIT_REASON_MONITOR_INSTRUCTION, "MONITOR_INSTRUCTION" }, \
100+ { EXIT_REASON_PAUSE_INSTRUCTION, "PAUSE_INSTRUCTION" }, \
101+ { EXIT_REASON_MCE_DURING_VMENTRY, "MCE_DURING_VMENTRY" }, \
102+ { EXIT_REASON_TPR_BELOW_THRESHOLD, "TPR_BELOW_THRESHOLD" }, \
103+ { EXIT_REASON_APIC_ACCESS, "APIC_ACCESS" }, \
104+ { EXIT_REASON_EPT_VIOLATION, "EPT_VIOLATION" }, \
105+ { EXIT_REASON_EPT_MISCONFIG, "EPT_MISCONFIG" }, \
106+ { EXIT_REASON_WBINVD, "WBINVD" }
107+
108+#ifdef __KERNEL__
109+
28110 #include <linux/types.h>
29111
30112 /*
@@ -241,49 +323,6 @@ enum vmcs_field { · 210 unmodified lines
241323 HOST_RIP = 0x00006c16,
242324 };
243325
244−#define VMX_EXIT_REASONS_FAILED_VMENTRY 0x80000000
245−
246−#define EXIT_REASON_EXCEPTION_NMI 0
247−#define EXIT_REASON_EXTERNAL_INTERRUPT 1
248−#define EXIT_REASON_TRIPLE_FAULT 2
249−
250−#define EXIT_REASON_PENDING_INTERRUPT 7
251−#define EXIT_REASON_NMI_WINDOW 8
252−#define EXIT_REASON_TASK_SWITCH 9
253−#define EXIT_REASON_CPUID 10
254−#define EXIT_REASON_HLT 12
255−#define EXIT_REASON_INVD 13
256−#define EXIT_REASON_INVLPG 14
257−#define EXIT_REASON_RDPMC 15
258−#define EXIT_REASON_RDTSC 16
259−#define EXIT_REASON_VMCALL 18
260−#define EXIT_REASON_VMCLEAR 19
261−#define EXIT_REASON_VMLAUNCH 20
262−#define EXIT_REASON_VMPTRLD 21
263−#define EXIT_REASON_VMPTRST 22
264−#define EXIT_REASON_VMREAD 23
265−#define EXIT_REASON_VMRESUME 24
266−#define EXIT_REASON_VMWRITE 25
267−#define EXIT_REASON_VMOFF 26
268−#define EXIT_REASON_VMON 27
269−#define EXIT_REASON_CR_ACCESS 28
270−#define EXIT_REASON_DR_ACCESS 29
271−#define EXIT_REASON_IO_INSTRUCTION 30
272−#define EXIT_REASON_MSR_READ 31
273−#define EXIT_REASON_MSR_WRITE 32
274−#define EXIT_REASON_INVALID_STATE 33
275−#define EXIT_REASON_MWAIT_INSTRUCTION 36
276−#define EXIT_REASON_MONITOR_INSTRUCTION 39
277−#define EXIT_REASON_PAUSE_INSTRUCTION 40
278−#define EXIT_REASON_MCE_DURING_VMENTRY 41
279−#define EXIT_REASON_TPR_BELOW_THRESHOLD 43
280−#define EXIT_REASON_APIC_ACCESS 44
281−#define EXIT_REASON_EPT_VIOLATION 48
282−#define EXIT_REASON_EPT_MISCONFIG 49
283−#define EXIT_REASON_WBINVD 54
284−#define EXIT_REASON_XSETBV 55
285−#define EXIT_REASON_INVPCID 58
286−
287326 /*
288327 * Interruption-information format
289328 */
@@ -488,3 +527,5 @@ enum vm_instruction_error_number { · 198 unmodified lines
488527 };
489528
490529 #endif
530+
531+#endif
arch/x86/kernel/Makefile+2−0
@@ -100,6 +100,8 @@ obj-$(CONFIG_SWIOTLB) += pci-swiotlb.o · 99 unmodified lines
100100 obj-$(CONFIG_OF) += devicetree.o
101101 obj-$(CONFIG_UPROBES) += uprobes.o
102102
103+obj-$(CONFIG_PERF_EVENTS) += perf_regs.o
104+
103105 ###
104106 # 64 bit specific files
105107 ifeq ($(CONFIG_X86_64),y)
arch/x86/kernel/cpu/perf_event_intel_uncore.c+25−3
@@ -2347,6 +2347,27 @@ int uncore_pmu_event_init(struct perf_event *event) · 2346 unmodified lines
23472347 return ret;
23482348 }
23492349
2350+static ssize_t uncore_get_attr_cpumask(struct device *dev,
2351+ struct device_attribute *attr, char *buf)
2352+{
2353+ int n = cpulist_scnprintf(buf, PAGE_SIZE - 2, &uncore_cpu_mask);
2354+
2355+ buf[n++] = '\n';
2356+ buf[n] = '\0';
2357+ return n;
2358+}
2359+
2360+static DEVICE_ATTR(cpumask, S_IRUGO, uncore_get_attr_cpumask, NULL);
2361+
2362+static struct attribute *uncore_pmu_attrs[] = {
2363+ &dev_attr_cpumask.attr,
2364+ NULL,
2365+};
2366+
2367+static struct attribute_group uncore_pmu_attr_group = {
2368+ .attrs = uncore_pmu_attrs,
2369+};
2370+
23502371 static int __init uncore_pmu_register(struct intel_uncore_pmu *pmu)
23512372 {
23522373 int ret;
@@ -2384,8 +2405,8 @@ static void __init uncore_type_exit(struct intel_uncore_type *type) · 31 unmodified lines
23842405 free_percpu(type->pmus[i].box);
23852406 kfree(type->pmus);
23862407 type->pmus = NULL;
2387− kfree(type->attr_groups[1]);
2388− type->attr_groups[1] = NULL;
2408+ kfree(type->events_group);
2409+ type->events_group = NULL;
23892410 }
23902411
23912412 static void __init uncore_types_exit(struct intel_uncore_type **types)
@@ -2437,9 +2458,10 @@ static int __init uncore_type_init(struct intel_uncore_type *type) · 45 unmodified lines
24372458 for (j = 0; j < i; j++)
24382459 attrs[j] = &type->event_descs[j].attr.attr;
24392460
2440− type->attr_groups[1] = events_group;
2461+ type->events_group = events_group;
24412462 }
24422463
2464+ type->pmu_group = &uncore_pmu_attr_group;
24432465 type->pmus = pmus;
24442466 return 0;
24452467 fail:
arch/x86/kernel/cpu/perf_event_intel_uncore.h+4−2
@@ -369,10 +369,12 @@ struct intel_uncore_type { · 368 unmodified lines
369369 struct intel_uncore_pmu *pmus;
370370 struct intel_uncore_ops *ops;
371371 struct uncore_event_desc *event_descs;
372− const struct attribute_group *attr_groups[3];
372+ const struct attribute_group *attr_groups[4];
373373 };
374374
375−#define format_group attr_groups[0]
375+#define pmu_group attr_groups[0]
376+#define format_group attr_groups[1]
377+#define events_group attr_groups[2]
376378
377379 struct intel_uncore_ops {
378380 void (*init_box)(struct intel_uncore_box *);
arch/x86/kernel/entry_32.S+70−4
@@ -1109,17 +1109,21 @@ ENTRY(ftrace_caller) · 1108 unmodified lines
11091109 pushl %eax
11101110 pushl %ecx
11111111 pushl %edx
1112− movl 0xc(%esp), %eax
1112+ pushl $0 /* Pass NULL as regs pointer */
1113+ movl 4*4(%esp), %eax
11131114 movl 0x4(%ebp), %edx
1115+ leal function_trace_op, %ecx
11141116 subl $MCOUNT_INSN_SIZE, %eax
11151117
11161118 .globl ftrace_call
11171119 ftrace_call:
11181120 call ftrace_stub
11191121
1122+ addl $4,%esp /* skip NULL pointer */
11201123 popl %edx
11211124 popl %ecx
11221125 popl %eax
1126+ftrace_ret:
11231127 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
11241128 .globl ftrace_graph_call
11251129 ftrace_graph_call:
@@ -1131,6 +1135,71 @@ ftrace_stub: · 5 unmodified lines
11311135 ret
11321136 END(ftrace_caller)
11331137
1138+ENTRY(ftrace_regs_caller)
1139+ pushf /* push flags before compare (in cs location) */
1140+ cmpl $0, function_trace_stop
1141+ jne ftrace_restore_flags
1142+
1143+ /*
1144+ * i386 does not save SS and ESP when coming from kernel.
1145+ * Instead, to get sp, &regs->sp is used (see ptrace.h).
1146+ * Unfortunately, that means eflags must be at the same location
1147+ * as the current return ip is. We move the return ip into the
1148+ * ip location, and move flags into the return ip location.
1149+ */
1150+ pushl 4(%esp) /* save return ip into ip slot */
1151+
1152+ pushl $0 /* Load 0 into orig_ax */
1153+ pushl %gs
1154+ pushl %fs
1155+ pushl %es
1156+ pushl %ds
1157+ pushl %eax
1158+ pushl %ebp
1159+ pushl %edi
1160+ pushl %esi
1161+ pushl %edx
1162+ pushl %ecx
1163+ pushl %ebx
1164+
1165+ movl 13*4(%esp), %eax /* Get the saved flags */
1166+ movl %eax, 14*4(%esp) /* Move saved flags into regs->flags location */
1167+ /* clobbering return ip */
1168+ movl $__KERNEL_CS,13*4(%esp)
1169+
1170+ movl 12*4(%esp), %eax /* Load ip (1st parameter) */
1171+ subl $MCOUNT_INSN_SIZE, %eax /* Adjust ip */
1172+ movl 0x4(%ebp), %edx /* Load parent ip (2nd parameter) */
1173+ leal function_trace_op, %ecx /* Save ftrace_pos in 3rd parameter */
1174+ pushl %esp /* Save pt_regs as 4th parameter */
1175+
1176+GLOBAL(ftrace_regs_call)
1177+ call ftrace_stub
1178+
1179+ addl $4, %esp /* Skip pt_regs */
1180+ movl 14*4(%esp), %eax /* Move flags back into cs */
1181+ movl %eax, 13*4(%esp) /* Needed to keep addl from modifying flags */
1182+ movl 12*4(%esp), %eax /* Get return ip from regs->ip */
1183+ movl %eax, 14*4(%esp) /* Put return ip back for ret */
1184+
1185+ popl %ebx
1186+ popl %ecx
1187+ popl %edx
1188+ popl %esi
1189+ popl %edi
1190+ popl %ebp
1191+ popl %eax
1192+ popl %ds
1193+ popl %es
1194+ popl %fs
1195+ popl %gs
1196+ addl $8, %esp /* Skip orig_ax and ip */
1197+ popf /* Pop flags at end (no addl to corrupt flags) */
1198+ jmp ftrace_ret
1199+
1200+ftrace_restore_flags:
1201+ popf
1202+ jmp ftrace_stub
11341203 #else /* ! CONFIG_DYNAMIC_FTRACE */
11351204
11361205 ENTRY(mcount)
@@ -1171,9 +1240,6 @@ END(mcount) · 34 unmodified lines
11711240
11721241 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
11731242 ENTRY(ftrace_graph_caller)
1174− cmpl $0, function_trace_stop
1175− jne ftrace_stub
1176−
11771243 pushl %eax
11781244 pushl %ecx
11791245 pushl %edx
arch/x86/kernel/entry_64.S+116−14
@@ -69,25 +69,51 @@ · 68 unmodified lines
6969 .section .entry.text, "ax"
7070
7171 #ifdef CONFIG_FUNCTION_TRACER
72+
73+#ifdef CC_USING_FENTRY
74+# define function_hook __fentry__
75+#else
76+# define function_hook mcount
77+#endif
78+
7279 #ifdef CONFIG_DYNAMIC_FTRACE
73−ENTRY(mcount)
80+
81+ENTRY(function_hook)
7482 retq
75−END(mcount)
83+END(function_hook)
84+
85+/* skip is set if stack has been adjusted */
86+.macro ftrace_caller_setup skip=0
87+ MCOUNT_SAVE_FRAME \skip
88+
89+ /* Load the ftrace_ops into the 3rd parameter */
90+ leaq function_trace_op, %rdx
91+
92+ /* Load ip into the first parameter */
93+ movq RIP(%rsp), %rdi
94+ subq $MCOUNT_INSN_SIZE, %rdi
95+ /* Load the parent_ip into the second parameter */
96+#ifdef CC_USING_FENTRY
97+ movq SS+16(%rsp), %rsi
98+#else
99+ movq 8(%rbp), %rsi
100+#endif
101+.endm
76102
77103 ENTRY(ftrace_caller)
104+ /* Check if tracing was disabled (quick check) */
78105 cmpl $0, function_trace_stop
79106 jne ftrace_stub
80107
81− MCOUNT_SAVE_FRAME
82−
83− movq 0x38(%rsp), %rdi
84− movq 8(%rbp), %rsi
85− subq $MCOUNT_INSN_SIZE, %rdi
108+ ftrace_caller_setup
109+ /* regs go into 4th parameter (but make it NULL) */
110+ movq $0, %rcx
86111
87112 GLOBAL(ftrace_call)
88113 call ftrace_stub
89114
90115 MCOUNT_RESTORE_FRAME
116+ftrace_return:
91117
92118 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
93119 GLOBAL(ftrace_graph_call)
@@ -98,8 +124,78 @@ GLOBAL(ftrace_stub) · 4 unmodified lines
98124 retq
99125 END(ftrace_caller)
100126
127+ENTRY(ftrace_regs_caller)
128+ /* Save the current flags before compare (in SS location)*/
129+ pushfq
130+
131+ /* Check if tracing was disabled (quick check) */
132+ cmpl $0, function_trace_stop
133+ jne ftrace_restore_flags
134+
135+ /* skip=8 to skip flags saved in SS */
136+ ftrace_caller_setup 8
137+
138+ /* Save the rest of pt_regs */
139+ movq %r15, R15(%rsp)
140+ movq %r14, R14(%rsp)
141+ movq %r13, R13(%rsp)
142+ movq %r12, R12(%rsp)
143+ movq %r11, R11(%rsp)
144+ movq %r10, R10(%rsp)
145+ movq %rbp, RBP(%rsp)
146+ movq %rbx, RBX(%rsp)
147+ /* Copy saved flags */
148+ movq SS(%rsp), %rcx
149+ movq %rcx, EFLAGS(%rsp)
150+ /* Kernel segments */
151+ movq $__KERNEL_DS, %rcx
152+ movq %rcx, SS(%rsp)
153+ movq $__KERNEL_CS, %rcx
154+ movq %rcx, CS(%rsp)
155+ /* Stack - skipping return address */
156+ leaq SS+16(%rsp), %rcx
157+ movq %rcx, RSP(%rsp)
158+
159+ /* regs go into 4th parameter */
160+ leaq (%rsp), %rcx
161+
162+GLOBAL(ftrace_regs_call)
163+ call ftrace_stub
164+
165+ /* Copy flags back to SS, to restore them */
166+ movq EFLAGS(%rsp), %rax
167+ movq %rax, SS(%rsp)
168+
169+ /* Handlers can change the RIP */
170+ movq RIP(%rsp), %rax
171+ movq %rax, SS+8(%rsp)
172+
173+ /* restore the rest of pt_regs */
174+ movq R15(%rsp), %r15
175+ movq R14(%rsp), %r14
176+ movq R13(%rsp), %r13
177+ movq R12(%rsp), %r12
178+ movq R10(%rsp), %r10
179+ movq RBP(%rsp), %rbp
180+ movq RBX(%rsp), %rbx
181+
182+ /* skip=8 to skip flags saved in SS */
183+ MCOUNT_RESTORE_FRAME 8
184+
185+ /* Restore flags */
186+ popfq
187+
188+ jmp ftrace_return
189+ftrace_restore_flags:
190+ popfq
191+ jmp ftrace_stub
192+
193+END(ftrace_regs_caller)
194+
195+
101196 #else /* ! CONFIG_DYNAMIC_FTRACE */
102−ENTRY(mcount)
197+
198+ENTRY(function_hook)
103199 cmpl $0, function_trace_stop
104200 jne ftrace_stub
105201
@@ -120,8 +216,12 @@ GLOBAL(ftrace_stub) · 14 unmodified lines
120216 trace:
121217 MCOUNT_SAVE_FRAME
122218
123− movq 0x38(%rsp), %rdi
219+ movq RIP(%rsp), %rdi
220+#ifdef CC_USING_FENTRY
221+ movq SS+16(%rsp), %rsi
222+#else
124223 movq 8(%rbp), %rsi
224+#endif
125225 subq $MCOUNT_INSN_SIZE, %rdi
126226
127227 call *ftrace_trace_function
@@ -129,20 +229,22 @@ trace: · 1 unmodified line
129229 MCOUNT_RESTORE_FRAME
130230
131231 jmp ftrace_stub
132−END(mcount)
232+END(function_hook)
133233 #endif /* CONFIG_DYNAMIC_FTRACE */
134234 #endif /* CONFIG_FUNCTION_TRACER */
135235
136236 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
137237 ENTRY(ftrace_graph_caller)
138− cmpl $0, function_trace_stop
139− jne ftrace_stub
140−
141238 MCOUNT_SAVE_FRAME
142239
240+#ifdef CC_USING_FENTRY
241+ leaq SS+16(%rsp), %rdi
242+ movq $0, %rdx /* No framepointers needed */
243+#else
143244 leaq 8(%rbp), %rdi
144− movq 0x38(%rsp), %rsi
145245 movq (%rbp), %rdx
246+#endif
247+ movq RIP(%rsp), %rsi
146248 subq $MCOUNT_INSN_SIZE, %rsi
147249
148250 call prepare_ftrace_return
arch/x86/kernel/ftrace.c+69−4
@@ -206,6 +206,21 @@ static int · 205 unmodified lines
206206 ftrace_modify_code(unsigned long ip, unsigned const char *old_code,
207207 unsigned const char *new_code);
208208
209+/*
210+ * Should never be called:
211+ * As it is only called by __ftrace_replace_code() which is called by
212+ * ftrace_replace_code() that x86 overrides, and by ftrace_update_code()
213+ * which is called to turn mcount into nops or nops into function calls
214+ * but not to convert a function from not using regs to one that uses
215+ * regs, which ftrace_modify_call() is for.
216+ */
217+int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
218+ unsigned long addr)
219+{
220+ WARN_ON(1);
221+ return -EINVAL;
222+}
223+
209224 int ftrace_update_ftrace_func(ftrace_func_t func)
210225 {
211226 unsigned long ip = (unsigned long)(&ftrace_call);
@@ -220,6 +235,14 @@ int ftrace_update_ftrace_func(ftrace_func_t func) · 8 unmodified lines
220235
221236 ret = ftrace_modify_code(ip, old, new);
222237
238+ /* Also update the regs callback function */
239+ if (!ret) {
240+ ip = (unsigned long)(&ftrace_regs_call);
241+ memcpy(old, &ftrace_regs_call, MCOUNT_INSN_SIZE);
242+ new = ftrace_call_replace(ip, (unsigned long)func);
243+ ret = ftrace_modify_code(ip, old, new);
244+ }
245+
223246 atomic_dec(&modifying_ftrace_code);
224247
225248 return ret;
@@ -299,6 +322,32 @@ static int add_brk_on_nop(struct dyn_ftrace *rec) · 73 unmodified lines
299322 return add_break(rec->ip, old);
300323 }
301324
325+/*
326+ * If the record has the FTRACE_FL_REGS set, that means that it
327+ * wants to convert to a callback that saves all regs. If FTRACE_FL_REGS
328+ * is not not set, then it wants to convert to the normal callback.
329+ */
330+static unsigned long get_ftrace_addr(struct dyn_ftrace *rec)
331+{
332+ if (rec->flags & FTRACE_FL_REGS)
333+ return (unsigned long)FTRACE_REGS_ADDR;
334+ else
335+ return (unsigned long)FTRACE_ADDR;
336+}
337+
338+/*
339+ * The FTRACE_FL_REGS_EN is set when the record already points to
340+ * a function that saves all the regs. Basically the '_EN' version
341+ * represents the current state of the function.
342+ */
343+static unsigned long get_ftrace_old_addr(struct dyn_ftrace *rec)
344+{
345+ if (rec->flags & FTRACE_FL_REGS_EN)
346+ return (unsigned long)FTRACE_REGS_ADDR;
347+ else
348+ return (unsigned long)FTRACE_ADDR;
349+}
350+
302351 static int add_breakpoints(struct dyn_ftrace *rec, int enable)
303352 {
304353 unsigned long ftrace_addr;
@@ -306,7 +355,7 @@ static int add_breakpoints(struct dyn_ftrace *rec, int enable) · 1 unmodified line
306355
307356 ret = ftrace_test_record(rec, enable);
308357
309− ftrace_addr = (unsigned long)FTRACE_ADDR;
358+ ftrace_addr = get_ftrace_addr(rec);
310359
311360 switch (ret) {
312361 case FTRACE_UPDATE_IGNORE:
@@ -316,6 +365,10 @@ static int add_breakpoints(struct dyn_ftrace *rec, int enable) · 3 unmodified lines
316365 /* converting nop to call */
317366 return add_brk_on_nop(rec);
318367
368+ case FTRACE_UPDATE_MODIFY_CALL_REGS:
369+ case FTRACE_UPDATE_MODIFY_CALL:
370+ ftrace_addr = get_ftrace_old_addr(rec);
371+ /* fall through */
319372 case FTRACE_UPDATE_MAKE_NOP:
320373 /* converting a call to a nop */
321374 return add_brk_on_call(rec, ftrace_addr);
@@ -360,13 +413,21 @@ static int remove_breakpoint(struct dyn_ftrace *rec) · 38 unmodified lines
360413 * If not, don't touch the breakpoint, we make just create
361414 * a disaster.
362415 */
363− ftrace_addr = (unsigned long)FTRACE_ADDR;
416+ ftrace_addr = get_ftrace_addr(rec);
417+ nop = ftrace_call_replace(ip, ftrace_addr);
418+
419+ if (memcmp(&ins[1], &nop[1], MCOUNT_INSN_SIZE - 1) == 0)
420+ goto update;
421+
422+ /* Check both ftrace_addr and ftrace_old_addr */
423+ ftrace_addr = get_ftrace_old_addr(rec);
364424 nop = ftrace_call_replace(ip, ftrace_addr);
365425
366426 if (memcmp(&ins[1], &nop[1], MCOUNT_INSN_SIZE - 1) != 0)
367427 return -EINVAL;
368428 }
369429
430+ update:
370431 return probe_kernel_write((void *)ip, &nop[0], 1);
371432 }
372433
@@ -405,12 +466,14 @@ static int add_update(struct dyn_ftrace *rec, int enable) · 32 unmodified lines
405466
406467 ret = ftrace_test_record(rec, enable);
407468
408− ftrace_addr = (unsigned long)FTRACE_ADDR;
469+ ftrace_addr = get_ftrace_addr(rec);
409470
410471 switch (ret) {
411472 case FTRACE_UPDATE_IGNORE:
412473 return 0;
413474
475+ case FTRACE_UPDATE_MODIFY_CALL_REGS:
476+ case FTRACE_UPDATE_MODIFY_CALL:
414477 case FTRACE_UPDATE_MAKE_CALL:
415478 /* converting nop to call */
416479 return add_update_call(rec, ftrace_addr);
@@ -455,12 +518,14 @@ static int finish_update(struct dyn_ftrace *rec, int enable) · 38 unmodified lines
455518
456519 ret = ftrace_update_record(rec, enable);
457520
458− ftrace_addr = (unsigned long)FTRACE_ADDR;
521+ ftrace_addr = get_ftrace_addr(rec);
459522
460523 switch (ret) {
461524 case FTRACE_UPDATE_IGNORE:
462525 return 0;
463526
527+ case FTRACE_UPDATE_MODIFY_CALL_REGS:
528+ case FTRACE_UPDATE_MODIFY_CALL:
464529 case FTRACE_UPDATE_MAKE_CALL:
465530 /* converting nop to call */
466531 return finish_update_call(rec, ftrace_addr);
arch/x86/kernel/kprobes.c+67−0
@@ -541,6 +541,23 @@ reenter_kprobe(struct kprobe *p, struct pt_regs *regs, struct kprobe_ctlblk *kcb · 540 unmodified lines
541541 return 1;
542542 }
543543
544+#ifdef KPROBES_CAN_USE_FTRACE
545+static void __kprobes skip_singlestep(struct kprobe *p, struct pt_regs *regs,
546+ struct kprobe_ctlblk *kcb)
547+{
548+ /*
549+ * Emulate singlestep (and also recover regs->ip)
550+ * as if there is a 5byte nop
551+ */
552+ regs->ip = (unsigned long)p->addr + MCOUNT_INSN_SIZE;
553+ if (unlikely(p->post_handler)) {
554+ kcb->kprobe_status = KPROBE_HIT_SSDONE;
555+ p->post_handler(p, regs, 0);
556+ }
557+ __this_cpu_write(current_kprobe, NULL);
558+}
559+#endif
560+
544561 /*
545562 * Interrupts are disabled on entry as trap3 is an interrupt gate and they
546563 * remain disabled throughout this function.
@@ -599,6 +616,12 @@ static int __kprobes kprobe_handler(struct pt_regs *regs) · 52 unmodified lines
599616 } else if (kprobe_running()) {
600617 p = __this_cpu_read(current_kprobe);
601618 if (p->break_handler && p->break_handler(p, regs)) {
619+#ifdef KPROBES_CAN_USE_FTRACE
620+ if (kprobe_ftrace(p)) {
621+ skip_singlestep(p, regs, kcb);
622+ return 1;
623+ }
624+#endif
602625 setup_singlestep(p, regs, kcb, 0);
603626 return 1;
604627 }
@@ -1052,6 +1075,50 @@ int __kprobes longjmp_break_handler(struct kprobe *p, struct pt_regs *regs) · 447 unmodified lines
10521075 return 0;
10531076 }
10541077
1078+#ifdef KPROBES_CAN_USE_FTRACE
1079+/* Ftrace callback handler for kprobes */
1080+void __kprobes kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
1081+ struct ftrace_ops *ops, struct pt_regs *regs)
1082+{
1083+ struct kprobe *p;
1084+ struct kprobe_ctlblk *kcb;
1085+ unsigned long flags;
1086+
1087+ /* Disable irq for emulating a breakpoint and avoiding preempt */
1088+ local_irq_save(flags);
1089+
1090+ p = get_kprobe((kprobe_opcode_t *)ip);
1091+ if (unlikely(!p) || kprobe_disabled(p))
1092+ goto end;
1093+
1094+ kcb = get_kprobe_ctlblk();
1095+ if (kprobe_running()) {
1096+ kprobes_inc_nmissed_count(p);
1097+ } else {
1098+ /* Kprobe handler expects regs->ip = ip + 1 as breakpoint hit */
1099+ regs->ip = ip + sizeof(kprobe_opcode_t);
1100+
1101+ __this_cpu_write(current_kprobe, p);
1102+ kcb->kprobe_status = KPROBE_HIT_ACTIVE;
1103+ if (!p->pre_handler || !p->pre_handler(p, regs))
1104+ skip_singlestep(p, regs, kcb);
1105+ /*
1106+ * If pre_handler returns !0, it sets regs->ip and
1107+ * resets current kprobe.
1108+ */
1109+ }
1110+end:
1111+ local_irq_restore(flags);
1112+}
1113+
1114+int __kprobes arch_prepare_kprobe_ftrace(struct kprobe *p)
1115+{
1116+ p->ainsn.insn = NULL;
1117+ p->ainsn.boostable = -1;
1118+ return 0;
1119+}
1120+#endif
1121+
10551122 int __init arch_init_kprobes(void)
10561123 {
10571124 return arch_init_optprobes();
arch/x86/kernel/perf_regs.c+105−0
@@ -0,0 +1,105 @@
1+#include <linux/errno.h>
2+#include <linux/kernel.h>
3+#include <linux/sched.h>
4+#include <linux/perf_event.h>
5+#include <linux/bug.h>
6+#include <linux/stddef.h>
7+#include <asm/perf_regs.h>
8+#include <asm/ptrace.h>
9+
10+#ifdef CONFIG_X86_32
11+#define PERF_REG_X86_MAX PERF_REG_X86_32_MAX
12+#else
13+#define PERF_REG_X86_MAX PERF_REG_X86_64_MAX
14+#endif
15+
16+#define PT_REGS_OFFSET(id, r) [id] = offsetof(struct pt_regs, r)
17+
18+static unsigned int pt_regs_offset[PERF_REG_X86_MAX] = {
19+ PT_REGS_OFFSET(PERF_REG_X86_AX, ax),
20+ PT_REGS_OFFSET(PERF_REG_X86_BX, bx),
21+ PT_REGS_OFFSET(PERF_REG_X86_CX, cx),
22+ PT_REGS_OFFSET(PERF_REG_X86_DX, dx),
23+ PT_REGS_OFFSET(PERF_REG_X86_SI, si),
24+ PT_REGS_OFFSET(PERF_REG_X86_DI, di),
25+ PT_REGS_OFFSET(PERF_REG_X86_BP, bp),
26+ PT_REGS_OFFSET(PERF_REG_X86_SP, sp),
27+ PT_REGS_OFFSET(PERF_REG_X86_IP, ip),
28+ PT_REGS_OFFSET(PERF_REG_X86_FLAGS, flags),
29+ PT_REGS_OFFSET(PERF_REG_X86_CS, cs),
30+ PT_REGS_OFFSET(PERF_REG_X86_SS, ss),
31+#ifdef CONFIG_X86_32
32+ PT_REGS_OFFSET(PERF_REG_X86_DS, ds),
33+ PT_REGS_OFFSET(PERF_REG_X86_ES, es),
34+ PT_REGS_OFFSET(PERF_REG_X86_FS, fs),
35+ PT_REGS_OFFSET(PERF_REG_X86_GS, gs),
36+#else
37+ /*
38+ * The pt_regs struct does not store
39+ * ds, es, fs, gs in 64 bit mode.
40+ */
41+ (unsigned int) -1,
42+ (unsigned int) -1,
43+ (unsigned int) -1,
44+ (unsigned int) -1,
45+#endif
46+#ifdef CONFIG_X86_64
47+ PT_REGS_OFFSET(PERF_REG_X86_R8, r8),
48+ PT_REGS_OFFSET(PERF_REG_X86_R9, r9),
49+ PT_REGS_OFFSET(PERF_REG_X86_R10, r10),
50+ PT_REGS_OFFSET(PERF_REG_X86_R11, r11),
51+ PT_REGS_OFFSET(PERF_REG_X86_R12, r12),
52+ PT_REGS_OFFSET(PERF_REG_X86_R13, r13),
53+ PT_REGS_OFFSET(PERF_REG_X86_R14, r14),
54+ PT_REGS_OFFSET(PERF_REG_X86_R15, r15),
55+#endif
56+};
57+
58+u64 perf_reg_value(struct pt_regs *regs, int idx)
59+{
60+ if (WARN_ON_ONCE(idx >= ARRAY_SIZE(pt_regs_offset)))
61+ return 0;
62+
63+ return regs_get_register(regs, pt_regs_offset[idx]);
64+}
65+
66+#define REG_RESERVED (~((1ULL << PERF_REG_X86_MAX) - 1ULL))
67+
68+#ifdef CONFIG_X86_32
69+int perf_reg_validate(u64 mask)
70+{
71+ if (!mask || mask & REG_RESERVED)
72+ return -EINVAL;
73+
74+ return 0;
75+}
76+
77+u64 perf_reg_abi(struct task_struct *task)
78+{
79+ return PERF_SAMPLE_REGS_ABI_32;
80+}
81+#else /* CONFIG_X86_64 */
82+#define REG_NOSUPPORT ((1ULL << PERF_REG_X86_DS) | \
83+ (1ULL << PERF_REG_X86_ES) | \
84+ (1ULL << PERF_REG_X86_FS) | \
85+ (1ULL << PERF_REG_X86_GS))
86+
87+int perf_reg_validate(u64 mask)
88+{
89+ if (!mask || mask & REG_RESERVED)
90+ return -EINVAL;
91+
92+ if (mask & REG_NOSUPPORT)
93+ return -EINVAL;
94+
95+ return 0;
96+}
97+
98+u64 perf_reg_abi(struct task_struct *task)
99+{
100+ if (test_tsk_thread_flag(task, TIF_IA32))
101+ return PERF_SAMPLE_REGS_ABI_32;
102+ else
103+ return PERF_SAMPLE_REGS_ABI_64;
104+}
105+#endif /* CONFIG_X86_32 */
arch/x86/kernel/step.c+33−20
@@ -157,6 +157,33 @@ static int enable_single_step(struct task_struct *child) · 156 unmodified lines
157157 return 1;
158158 }
159159
160+void set_task_blockstep(struct task_struct *task, bool on)
161+{
162+ unsigned long debugctl;
163+
164+ /*
165+ * Ensure irq/preemption can't change debugctl in between.
166+ * Note also that both TIF_BLOCKSTEP and debugctl should
167+ * be changed atomically wrt preemption.
168+ * FIXME: this means that set/clear TIF_BLOCKSTEP is simply
169+ * wrong if task != current, SIGKILL can wakeup the stopped
170+ * tracee and set/clear can play with the running task, this
171+ * can confuse the next __switch_to_xtra().
172+ */
173+ local_irq_disable();
174+ debugctl = get_debugctlmsr();
175+ if (on) {
176+ debugctl |= DEBUGCTLMSR_BTF;
177+ set_tsk_thread_flag(task, TIF_BLOCKSTEP);
178+ } else {
179+ debugctl &= ~DEBUGCTLMSR_BTF;
180+ clear_tsk_thread_flag(task, TIF_BLOCKSTEP);
181+ }
182+ if (task == current)
183+ update_debugctlmsr(debugctl);
184+ local_irq_enable();
185+}
186+
160187 /*
161188 * Enable single or block step.
162189 */
@@ -169,19 +196,10 @@ static void enable_step(struct task_struct *child, bool block) · 6 unmodified lines
169196 * So no one should try to use debugger block stepping in a program
170197 * that uses user-mode single stepping itself.
171198 */
172− if (enable_single_step(child) && block) {
173− unsigned long debugctl = get_debugctlmsr();
174−
175− debugctl |= DEBUGCTLMSR_BTF;
176− update_debugctlmsr(debugctl);
177− set_tsk_thread_flag(child, TIF_BLOCKSTEP);
178− } else if (test_tsk_thread_flag(child, TIF_BLOCKSTEP)) {
179− unsigned long debugctl = get_debugctlmsr();
180−
181− debugctl &= ~DEBUGCTLMSR_BTF;
182− update_debugctlmsr(debugctl);
183− clear_tsk_thread_flag(child, TIF_BLOCKSTEP);
184− }
199+ if (enable_single_step(child) && block)
200+ set_task_blockstep(child, true);
201+ else if (test_tsk_thread_flag(child, TIF_BLOCKSTEP))
202+ set_task_blockstep(child, false);
185203 }
186204
187205 void user_enable_single_step(struct task_struct *child)
@@ -199,13 +217,8 @@ void user_disable_single_step(struct task_struct *child) · 11 unmodified lines
199217 /*
200218 * Make sure block stepping (BTF) is disabled.
201219 */
202− if (test_tsk_thread_flag(child, TIF_BLOCKSTEP)) {
203− unsigned long debugctl = get_debugctlmsr();
204−
205− debugctl &= ~DEBUGCTLMSR_BTF;
206− update_debugctlmsr(debugctl);
207− clear_tsk_thread_flag(child, TIF_BLOCKSTEP);
208− }
220+ if (test_tsk_thread_flag(child, TIF_BLOCKSTEP))
221+ set_task_blockstep(child, false);
209222
210223 /* Always clear TIF_SINGLESTEP... */
211224 clear_tsk_thread_flag(child, TIF_SINGLESTEP);
arch/x86/kernel/uprobes.c+51−1
@@ -41,6 +41,9 @@ · 40 unmodified lines
4141 /* Adjust the return address of a call insn */
4242 #define UPROBE_FIX_CALL 0x2
4343
44+/* Instruction will modify TF, don't change it */
45+#define UPROBE_FIX_SETF 0x4
46+
4447 #define UPROBE_FIX_RIP_AX 0x8000
4548 #define UPROBE_FIX_RIP_CX 0x4000
4649
@@ -239,6 +242,10 @@ static void prepare_fixups(struct arch_uprobe *auprobe, struct insn *insn) · 192 unmodified lines
239242 insn_get_opcode(insn); /* should be a nop */
240243
241244 switch (OPCODE1(insn)) {
245+ case 0x9d:
246+ /* popf */
247+ auprobe->fixups |= UPROBE_FIX_SETF;
248+ break;
242249 case 0xc3: /* ret/lret */
243250 case 0xcb:
244251 case 0xc2:
@@ -646,7 +653,7 @@ void arch_uprobe_abort_xol(struct arch_uprobe *auprobe, struct pt_regs *regs) · 401 unmodified lines
646653 * Skip these instructions as per the currently known x86 ISA.
647654 * 0x66* { 0x90 | 0x0f 0x1f | 0x0f 0x19 | 0x87 0xc0 }
648655 */
649−bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
656+static bool __skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
650657 {
651658 int i;
652659
@@ -673,3 +680,46 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs) · 20 unmodified lines
673680 }
674681 return false;
675682 }
683+
684+bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
685+{
686+ bool ret = __skip_sstep(auprobe, regs);
687+ if (ret && (regs->flags & X86_EFLAGS_TF))
688+ send_sig(SIGTRAP, current, 0);
689+ return ret;
690+}
691+
692+void arch_uprobe_enable_step(struct arch_uprobe *auprobe)
693+{
694+ struct task_struct *task = current;
695+ struct arch_uprobe_task *autask = &task->utask->autask;
696+ struct pt_regs *regs = task_pt_regs(task);
697+
698+ autask->saved_tf = !!(regs->flags & X86_EFLAGS_TF);
699+
700+ regs->flags |= X86_EFLAGS_TF;
701+ if (test_tsk_thread_flag(task, TIF_BLOCKSTEP))
702+ set_task_blockstep(task, false);
703+}
704+
705+void arch_uprobe_disable_step(struct arch_uprobe *auprobe)
706+{
707+ struct task_struct *task = current;
708+ struct arch_uprobe_task *autask = &task->utask->autask;
709+ bool trapped = (task->utask->state == UTASK_SSTEP_TRAPPED);
710+ struct pt_regs *regs = task_pt_regs(task);
711+ /*
712+ * The state of TIF_BLOCKSTEP was not saved so we can get an extra
713+ * SIGTRAP if we do not clear TF. We need to examine the opcode to
714+ * make it right.
715+ */
716+ if (unlikely(trapped)) {
717+ if (!autask->saved_tf)
718+ regs->flags &= ~X86_EFLAGS_TF;
719+ } else {
720+ if (autask->saved_tf)
721+ send_sig(SIGTRAP, task, 0);
722+ else if (!(auprobe->fixups & UPROBE_FIX_SETF))
723+ regs->flags &= ~X86_EFLAGS_TF;
724+ }
725+}
arch/x86/kernel/x8664_ksyms_64.c+5−1
@@ -13,9 +13,13 @@ · 12 unmodified lines
1313 #include <asm/ftrace.h>
1414
1515 #ifdef CONFIG_FUNCTION_TRACER
16−/* mcount is defined in assembly */
16+/* mcount and __fentry__ are defined in assembly */
17+#ifdef CC_USING_FENTRY
18+EXPORT_SYMBOL(__fentry__);
19+#else
1720 EXPORT_SYMBOL(mcount);
1821 #endif
22+#endif
1923
2024 EXPORT_SYMBOL(__get_user_1);
2125 EXPORT_SYMBOL(__get_user_2);
arch/x86/kvm/trace.h+0−89
@@ -183,95 +183,6 @@ TRACE_EVENT(kvm_apic, · 182 unmodified lines
183183 #define KVM_ISA_VMX 1
184184 #define KVM_ISA_SVM 2
185185
186−#define VMX_EXIT_REASONS \
187− { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \
188− { EXIT_REASON_EXTERNAL_INTERRUPT, "EXTERNAL_INTERRUPT" }, \
189− { EXIT_REASON_TRIPLE_FAULT, "TRIPLE_FAULT" }, \
190− { EXIT_REASON_PENDING_INTERRUPT, "PENDING_INTERRUPT" }, \
191− { EXIT_REASON_NMI_WINDOW, "NMI_WINDOW" }, \
192− { EXIT_REASON_TASK_SWITCH, "TASK_SWITCH" }, \
193− { EXIT_REASON_CPUID, "CPUID" }, \
194− { EXIT_REASON_HLT, "HLT" }, \
195− { EXIT_REASON_INVLPG, "INVLPG" }, \
196− { EXIT_REASON_RDPMC, "RDPMC" }, \
197− { EXIT_REASON_RDTSC, "RDTSC" }, \
198− { EXIT_REASON_VMCALL, "VMCALL" }, \
199− { EXIT_REASON_VMCLEAR, "VMCLEAR" }, \
200− { EXIT_REASON_VMLAUNCH, "VMLAUNCH" }, \
201− { EXIT_REASON_VMPTRLD, "VMPTRLD" }, \
202− { EXIT_REASON_VMPTRST, "VMPTRST" }, \
203− { EXIT_REASON_VMREAD, "VMREAD" }, \
204− { EXIT_REASON_VMRESUME, "VMRESUME" }, \
205− { EXIT_REASON_VMWRITE, "VMWRITE" }, \
206− { EXIT_REASON_VMOFF, "VMOFF" }, \
207− { EXIT_REASON_VMON, "VMON" }, \
208− { EXIT_REASON_CR_ACCESS, "CR_ACCESS" }, \
209− { EXIT_REASON_DR_ACCESS, "DR_ACCESS" }, \
210− { EXIT_REASON_IO_INSTRUCTION, "IO_INSTRUCTION" }, \
211− { EXIT_REASON_MSR_READ, "MSR_READ" }, \
212− { EXIT_REASON_MSR_WRITE, "MSR_WRITE" }, \
213− { EXIT_REASON_MWAIT_INSTRUCTION, "MWAIT_INSTRUCTION" }, \
214− { EXIT_REASON_MONITOR_INSTRUCTION, "MONITOR_INSTRUCTION" }, \
215− { EXIT_REASON_PAUSE_INSTRUCTION, "PAUSE_INSTRUCTION" }, \
216− { EXIT_REASON_MCE_DURING_VMENTRY, "MCE_DURING_VMENTRY" }, \
217− { EXIT_REASON_TPR_BELOW_THRESHOLD, "TPR_BELOW_THRESHOLD" }, \
218− { EXIT_REASON_APIC_ACCESS, "APIC_ACCESS" }, \
219− { EXIT_REASON_EPT_VIOLATION, "EPT_VIOLATION" }, \
220− { EXIT_REASON_EPT_MISCONFIG, "EPT_MISCONFIG" }, \
221− { EXIT_REASON_WBINVD, "WBINVD" }
222−
223−#define SVM_EXIT_REASONS \
224− { SVM_EXIT_READ_CR0, "read_cr0" }, \
225− { SVM_EXIT_READ_CR3, "read_cr3" }, \
226− { SVM_EXIT_READ_CR4, "read_cr4" }, \
227− { SVM_EXIT_READ_CR8, "read_cr8" }, \
228− { SVM_EXIT_WRITE_CR0, "write_cr0" }, \
229− { SVM_EXIT_WRITE_CR3, "write_cr3" }, \
230− { SVM_EXIT_WRITE_CR4, "write_cr4" }, \
231− { SVM_EXIT_WRITE_CR8, "write_cr8" }, \
232− { SVM_EXIT_READ_DR0, "read_dr0" }, \
233− { SVM_EXIT_READ_DR1, "read_dr1" }, \
234− { SVM_EXIT_READ_DR2, "read_dr2" }, \
235− { SVM_EXIT_READ_DR3, "read_dr3" }, \
236− { SVM_EXIT_WRITE_DR0, "write_dr0" }, \
237− { SVM_EXIT_WRITE_DR1, "write_dr1" }, \
238− { SVM_EXIT_WRITE_DR2, "write_dr2" }, \
239− { SVM_EXIT_WRITE_DR3, "write_dr3" }, \
240− { SVM_EXIT_WRITE_DR5, "write_dr5" }, \
241− { SVM_EXIT_WRITE_DR7, "write_dr7" }, \
242− { SVM_EXIT_EXCP_BASE + DB_VECTOR, "DB excp" }, \
243− { SVM_EXIT_EXCP_BASE + BP_VECTOR, "BP excp" }, \
244− { SVM_EXIT_EXCP_BASE + UD_VECTOR, "UD excp" }, \
245− { SVM_EXIT_EXCP_BASE + PF_VECTOR, "PF excp" }, \
246− { SVM_EXIT_EXCP_BASE + NM_VECTOR, "NM excp" }, \
247− { SVM_EXIT_EXCP_BASE + MC_VECTOR, "MC excp" }, \
248− { SVM_EXIT_INTR, "interrupt" }, \
249− { SVM_EXIT_NMI, "nmi" }, \
250− { SVM_EXIT_SMI, "smi" }, \
251− { SVM_EXIT_INIT, "init" }, \
252− { SVM_EXIT_VINTR, "vintr" }, \
253− { SVM_EXIT_CPUID, "cpuid" }, \
254− { SVM_EXIT_INVD, "invd" }, \
255− { SVM_EXIT_HLT, "hlt" }, \
256− { SVM_EXIT_INVLPG, "invlpg" }, \
257− { SVM_EXIT_INVLPGA, "invlpga" }, \
258− { SVM_EXIT_IOIO, "io" }, \
259− { SVM_EXIT_MSR, "msr" }, \
260− { SVM_EXIT_TASK_SWITCH, "task_switch" }, \
261− { SVM_EXIT_SHUTDOWN, "shutdown" }, \
262− { SVM_EXIT_VMRUN, "vmrun" }, \
263− { SVM_EXIT_VMMCALL, "hypercall" }, \
264− { SVM_EXIT_VMLOAD, "vmload" }, \
265− { SVM_EXIT_VMSAVE, "vmsave" }, \
266− { SVM_EXIT_STGI, "stgi" }, \
267− { SVM_EXIT_CLGI, "clgi" }, \
268− { SVM_EXIT_SKINIT, "skinit" }, \
269− { SVM_EXIT_WBINVD, "wbinvd" }, \
270− { SVM_EXIT_MONITOR, "monitor" }, \
271− { SVM_EXIT_MWAIT, "mwait" }, \
272− { SVM_EXIT_XSETBV, "xsetbv" }, \
273− { SVM_EXIT_NPF, "npf" }
274−
275186 /*
276187 * Tracepoint for kvm guest exit:
277188 */
drivers/oprofile/cpu_buffer.c+3−8
@@ -451,14 +451,9 @@ static void wq_sync_buffer(struct work_struct *work) · 450 unmodified lines
451451 {
452452 struct oprofile_cpu_buffer *b =
453453 container_of(work, struct oprofile_cpu_buffer, work.work);
454− if (b->cpu != smp_processor_id()) {
455− printk(KERN_DEBUG "WQ on CPU%d, prefer CPU%d\n",
456− smp_processor_id(), b->cpu);
457−
458− if (!cpu_online(b->cpu)) {
459− cancel_delayed_work(&b->work);
460− return;
461− }
454+ if (b->cpu != smp_processor_id() && !cpu_online(b->cpu)) {
455+ cancel_delayed_work(&b->work);
456+ return;
462457 }
463458 sync_buffer(b->cpu);
464459
include/linux/ftrace.h+148−10
@@ -10,6 +10,7 @@ · 9 unmodified lines
1010 #include <linux/kallsyms.h>
1111 #include <linux/linkage.h>
1212 #include <linux/bitops.h>
13+#include <linux/ptrace.h>
1314 #include <linux/ktime.h>
1415 #include <linux/sched.h>
1516 #include <linux/types.h>
@@ -18,6 +19,28 @@ · 2 unmodified lines
1819
1920 #include <asm/ftrace.h>
2021
22+/*
23+ * If the arch supports passing the variable contents of
24+ * function_trace_op as the third parameter back from the
25+ * mcount call, then the arch should define this as 1.
26+ */
27+#ifndef ARCH_SUPPORTS_FTRACE_OPS
28+#define ARCH_SUPPORTS_FTRACE_OPS 0
29+#endif
30+
31+/*
32+ * If the arch's mcount caller does not support all of ftrace's
33+ * features, then it must call an indirect function that
34+ * does. Or at least does enough to prevent any unwelcomed side effects.
35+ */
36+#if !defined(CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST) || \
37+ !ARCH_SUPPORTS_FTRACE_OPS
38+# define FTRACE_FORCE_LIST_FUNC 1
39+#else
40+# define FTRACE_FORCE_LIST_FUNC 0
41+#endif
42+
43+
2144 struct module;
2245 struct ftrace_hash;
2346
@@ -29,7 +52,10 @@ ftrace_enable_sysctl(struct ctl_table *table, int write, · 5 unmodified lines
2952 void __user *buffer, size_t *lenp,
3053 loff_t *ppos);
3154
32−typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
55+struct ftrace_ops;
56+
57+typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip,
58+ struct ftrace_ops *op, struct pt_regs *regs);
3359
3460 /*
3561 * FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
@@ -45,12 +71,33 @@ typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip); · 9 unmodified lines
4571 * could be controled by following calls:
4672 * ftrace_function_local_enable
4773 * ftrace_function_local_disable
74+ * SAVE_REGS - The ftrace_ops wants regs saved at each function called
75+ * and passed to the callback. If this flag is set, but the
76+ * architecture does not support passing regs
77+ * (ARCH_SUPPORTS_FTRACE_SAVE_REGS is not defined), then the
78+ * ftrace_ops will fail to register, unless the next flag
79+ * is set.
80+ * SAVE_REGS_IF_SUPPORTED - This is the same as SAVE_REGS, but if the
81+ * handler can handle an arch that does not save regs
82+ * (the handler tests if regs == NULL), then it can set
83+ * this flag instead. It will not fail registering the ftrace_ops
84+ * but, the regs field will be NULL if the arch does not support
85+ * passing regs to the handler.
86+ * Note, if this flag is set, the SAVE_REGS flag will automatically
87+ * get set upon registering the ftrace_ops, if the arch supports it.
88+ * RECURSION_SAFE - The ftrace_ops can set this to tell the ftrace infrastructure
89+ * that the call back has its own recursion protection. If it does
90+ * not set this, then the ftrace infrastructure will add recursion
91+ * protection for the caller.
4892 */
4993 enum {
50− FTRACE_OPS_FL_ENABLED = 1 << 0,
51− FTRACE_OPS_FL_GLOBAL = 1 << 1,
52− FTRACE_OPS_FL_DYNAMIC = 1 << 2,
53− FTRACE_OPS_FL_CONTROL = 1 << 3,
94+ FTRACE_OPS_FL_ENABLED = 1 << 0,
95+ FTRACE_OPS_FL_GLOBAL = 1 << 1,
96+ FTRACE_OPS_FL_DYNAMIC = 1 << 2,
97+ FTRACE_OPS_FL_CONTROL = 1 << 3,
98+ FTRACE_OPS_FL_SAVE_REGS = 1 << 4,
99+ FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED = 1 << 5,
100+ FTRACE_OPS_FL_RECURSION_SAFE = 1 << 6,
54101 };
55102
56103 struct ftrace_ops {
@@ -163,7 +210,8 @@ static inline int ftrace_function_local_disabled(struct ftrace_ops *ops) · 106 unmodified lines
163210 return *this_cpu_ptr(ops->disabled);
164211 }
165212
166−extern void ftrace_stub(unsigned long a0, unsigned long a1);
213+extern void ftrace_stub(unsigned long a0, unsigned long a1,
214+ struct ftrace_ops *op, struct pt_regs *regs);
167215
168216 #else /* !CONFIG_FUNCTION_TRACER */
169217 /*
@@ -172,6 +220,10 @@ extern void ftrace_stub(unsigned long a0, unsigned long a1); · 2 unmodified lines
172220 */
173221 #define register_ftrace_function(ops) ({ 0; })
174222 #define unregister_ftrace_function(ops) ({ 0; })
223+static inline int ftrace_nr_registered_ops(void)
224+{
225+ return 0;
226+}
175227 static inline void clear_ftrace_function(void) { }
176228 static inline void ftrace_kill(void) { }
177229 static inline void ftrace_stop(void) { }
@@ -227,12 +279,33 @@ extern void unregister_ftrace_function_probe_all(char *glob); · 49 unmodified lines
227279
228280 extern int ftrace_text_reserved(void *start, void *end);
229281
282+extern int ftrace_nr_registered_ops(void);
283+
284+/*
285+ * The dyn_ftrace record's flags field is split into two parts.
286+ * the first part which is '0-FTRACE_REF_MAX' is a counter of
287+ * the number of callbacks that have registered the function that
288+ * the dyn_ftrace descriptor represents.
289+ *
290+ * The second part is a mask:
291+ * ENABLED - the function is being traced
292+ * REGS - the record wants the function to save regs
293+ * REGS_EN - the function is set up to save regs.
294+ *
295+ * When a new ftrace_ops is registered and wants a function to save
296+ * pt_regs, the rec->flag REGS is set. When the function has been
297+ * set up to save regs, the REG_EN flag is set. Once a function
298+ * starts saving regs it will do so until all ftrace_ops are removed
299+ * from tracing that function.
300+ */
230301 enum {
231− FTRACE_FL_ENABLED = (1 << 30),
302+ FTRACE_FL_ENABLED = (1UL << 29),
303+ FTRACE_FL_REGS = (1UL << 30),
304+ FTRACE_FL_REGS_EN = (1UL << 31)
232305 };
233306
234−#define FTRACE_FL_MASK (0x3UL << 30)
235−#define FTRACE_REF_MAX ((1 << 30) - 1)
307+#define FTRACE_FL_MASK (0x7UL << 29)
308+#define FTRACE_REF_MAX ((1UL << 29) - 1)
236309
237310 struct dyn_ftrace {
238311 union {
@@ -244,6 +317,8 @@ struct dyn_ftrace { · 5 unmodified lines
244317 };
245318
246319 int ftrace_force_update(void);
320+int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip,
321+ int remove, int reset);
247322 int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
248323 int len, int reset);
249324 int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
@@ -263,9 +338,23 @@ enum { · 13 unmodified lines
263338 FTRACE_STOP_FUNC_RET = (1 << 4),
264339 };
265340
341+/*
342+ * The FTRACE_UPDATE_* enum is used to pass information back
343+ * from the ftrace_update_record() and ftrace_test_record()
344+ * functions. These are called by the code update routines
345+ * to find out what is to be done for a given function.
346+ *
347+ * IGNORE - The function is already what we want it to be
348+ * MAKE_CALL - Start tracing the function
349+ * MODIFY_CALL - Stop saving regs for the function
350+ * MODIFY_CALL_REGS - Start saving regs for the function
351+ * MAKE_NOP - Stop tracing the function
352+ */
266353 enum {
267354 FTRACE_UPDATE_IGNORE,
268355 FTRACE_UPDATE_MAKE_CALL,
356+ FTRACE_UPDATE_MODIFY_CALL,
357+ FTRACE_UPDATE_MODIFY_CALL_REGS,
269358 FTRACE_UPDATE_MAKE_NOP,
270359 };
271360
@@ -317,7 +406,9 @@ extern int ftrace_dyn_arch_init(void *data); · 45 unmodified lines
317406 extern void ftrace_replace_code(int enable);
318407 extern int ftrace_update_ftrace_func(ftrace_func_t func);
319408 extern void ftrace_caller(void);
409+extern void ftrace_regs_caller(void);
320410 extern void ftrace_call(void);
411+extern void ftrace_regs_call(void);
321412 extern void mcount_call(void);
322413
323414 void ftrace_modify_all_code(int command);
@@ -325,6 +416,15 @@ void ftrace_modify_all_code(int command); · 1 unmodified line
325416 #ifndef FTRACE_ADDR
326417 #define FTRACE_ADDR ((unsigned long)ftrace_caller)
327418 #endif
419+
420+#ifndef FTRACE_REGS_ADDR
421+#ifdef ARCH_SUPPORTS_FTRACE_SAVE_REGS
422+# define FTRACE_REGS_ADDR ((unsigned long)ftrace_regs_caller)
423+#else
424+# define FTRACE_REGS_ADDR FTRACE_ADDR
425+#endif
426+#endif
427+
328428 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
329429 extern void ftrace_graph_caller(void);
330430 extern int ftrace_enable_ftrace_graph_caller(void);
@@ -380,6 +480,39 @@ extern int ftrace_make_nop(struct module *mod, · 49 unmodified lines
380480 */
381481 extern int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr);
382482
483+#ifdef ARCH_SUPPORTS_FTRACE_SAVE_REGS
484+/**
485+ * ftrace_modify_call - convert from one addr to another (no nop)
486+ * @rec: the mcount call site record
487+ * @old_addr: the address expected to be currently called to
488+ * @addr: the address to change to
489+ *
490+ * This is a very sensitive operation and great care needs
491+ * to be taken by the arch. The operation should carefully
492+ * read the location, check to see if what is read is indeed
493+ * what we expect it to be, and then on success of the compare,
494+ * it should write to the location.
495+ *
496+ * The code segment at @rec->ip should be a caller to @old_addr
497+ *
498+ * Return must be:
499+ * 0 on success
500+ * -EFAULT on error reading the location
501+ * -EINVAL on a failed compare of the contents
502+ * -EPERM on error writing to the location
503+ * Any other value will be considered a failure.
504+ */
505+extern int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
506+ unsigned long addr);
507+#else
508+/* Should never be called */
509+static inline int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
510+ unsigned long addr)
511+{
512+ return -EINVAL;
513+}
514+#endif
515+
383516 /* May be defined in arch */
384517 extern int ftrace_arch_read_dyn_info(char *buf, int size);
385518
@@ -387,7 +520,7 @@ extern int skip_trace(unsigned long ip); · 1 unmodified line
387520
388521 extern void ftrace_disable_daemon(void);
389522 extern void ftrace_enable_daemon(void);
390−#else
523+#else /* CONFIG_DYNAMIC_FTRACE */
391524 static inline int skip_trace(unsigned long ip) { return 0; }
392525 static inline int ftrace_force_update(void) { return 0; }
393526 static inline void ftrace_disable_daemon(void) { }
@@ -405,6 +538,10 @@ static inline int ftrace_text_reserved(void *start, void *end) · 11 unmodified lines
405538 {
406539 return 0;
407540 }
541+static inline unsigned long ftrace_location(unsigned long ip)
542+{
543+ return 0;
544+}
408545
409546 /*
410547 * Again users of functions that have ftrace_ops may not
@@ -413,6 +550,7 @@ static inline int ftrace_text_reserved(void *start, void *end) · 2 unmodified lines
413550 */
414551 #define ftrace_regex_open(ops, flag, inod, file) ({ -ENODEV; })
415552 #define ftrace_set_early_filter(ops, buf, enable) do { } while (0)
553+#define ftrace_set_filter_ip(ops, ip, remove, reset) ({ -ENODEV; })
416554 #define ftrace_set_filter(ops, buf, len, reset) ({ -ENODEV; })
417555 #define ftrace_set_notrace(ops, buf, len, reset) ({ -ENODEV; })
418556 #define ftrace_free_filter(ops) do { } while (0)
include/linux/kprobes.h+27−0
@@ -38,6 +38,7 @@ · 37 unmodified lines
3838 #include <linux/spinlock.h>
3939 #include <linux/rcupdate.h>
4040 #include <linux/mutex.h>
41+#include <linux/ftrace.h>
4142
4243 #ifdef CONFIG_KPROBES
4344 #include <asm/kprobes.h>
@@ -48,14 +49,26 @@ · 4 unmodified lines
4849 #define KPROBE_REENTER 0x00000004
4950 #define KPROBE_HIT_SSDONE 0x00000008
5051
52+/*
53+ * If function tracer is enabled and the arch supports full
54+ * passing of pt_regs to function tracing, then kprobes can
55+ * optimize on top of function tracing.
56+ */
57+#if defined(CONFIG_FUNCTION_TRACER) && defined(ARCH_SUPPORTS_FTRACE_SAVE_REGS) \
58+ && defined(ARCH_SUPPORTS_KPROBES_ON_FTRACE)
59+# define KPROBES_CAN_USE_FTRACE
60+#endif
61+
5162 /* Attach to insert probes on any functions which should be ignored*/
5263 #define __kprobes __attribute__((__section__(".kprobes.text")))
64+
5365 #else /* CONFIG_KPROBES */
5466 typedef int kprobe_opcode_t;
5567 struct arch_specific_insn {
5668 int dummy;
5769 };
5870 #define __kprobes
71+
5972 #endif /* CONFIG_KPROBES */
6073
6174 struct kprobe;
@@ -128,6 +141,7 @@ struct kprobe { · 66 unmodified lines
128141 * NOTE:
129142 * this flag is only for optimized_kprobe.
130143 */
144+#define KPROBE_FLAG_FTRACE 8 /* probe is using ftrace */
131145
132146 /* Has this kprobe gone ? */
133147 static inline int kprobe_gone(struct kprobe *p)
@@ -146,6 +160,13 @@ static inline int kprobe_optimized(struct kprobe *p) · 12 unmodified lines
146160 {
147161 return p->flags & KPROBE_FLAG_OPTIMIZED;
148162 }
163+
164+/* Is this kprobe uses ftrace ? */
165+static inline int kprobe_ftrace(struct kprobe *p)
166+{
167+ return p->flags & KPROBE_FLAG_FTRACE;
168+}
169+
149170 /*
150171 * Special probe type that uses setjmp-longjmp type tricks to resume
151172 * execution at a specified entry with a matching prototype corresponding
@@ -295,6 +316,12 @@ extern int proc_kprobes_optimization_handler(struct ctl_table *table, · 143 unmodified lines
295316 #endif
296317
297318 #endif /* CONFIG_OPTPROBES */
319+#ifdef KPROBES_CAN_USE_FTRACE
320+extern void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
321+ struct ftrace_ops *ops, struct pt_regs *regs);
322+extern int arch_prepare_kprobe_ftrace(struct kprobe *p);
323+#endif
324+
298325
299326 /* Get the kprobe at this addr (if any) - called with preemption disabled */
300327 struct kprobe *get_kprobe(void *addr);
include/linux/perf_event.h+55−5
@@ -130,8 +130,10 @@ enum perf_event_sample_format { · 129 unmodified lines
130130 PERF_SAMPLE_STREAM_ID = 1U << 9,
131131 PERF_SAMPLE_RAW = 1U << 10,
132132 PERF_SAMPLE_BRANCH_STACK = 1U << 11,
133+ PERF_SAMPLE_REGS_USER = 1U << 12,
134+ PERF_SAMPLE_STACK_USER = 1U << 13,
133135
134− PERF_SAMPLE_MAX = 1U << 12, /* non-ABI */
136+ PERF_SAMPLE_MAX = 1U << 14, /* non-ABI */
135137 };
136138
137139 /*
@@ -162,6 +164,15 @@ enum perf_branch_sample_type { · 24 unmodified lines
162164 PERF_SAMPLE_BRANCH_KERNEL|\
163165 PERF_SAMPLE_BRANCH_HV)
164166
167+/*
168+ * Values to determine ABI of the registers dump.
169+ */
170+enum perf_sample_regs_abi {
171+ PERF_SAMPLE_REGS_ABI_NONE = 0,
172+ PERF_SAMPLE_REGS_ABI_32 = 1,
173+ PERF_SAMPLE_REGS_ABI_64 = 2,
174+};
175+
165176 /*
166177 * The format of the data returned by read() on a perf event fd,
167178 * as specified by attr.read_format:
@@ -194,6 +205,8 @@ enum perf_event_read_format { · 26 unmodified lines
194205 #define PERF_ATTR_SIZE_VER0 64 /* sizeof first published struct */
195206 #define PERF_ATTR_SIZE_VER1 72 /* add: config2 */
196207 #define PERF_ATTR_SIZE_VER2 80 /* add: branch_sample_type */
208+#define PERF_ATTR_SIZE_VER3 96 /* add: sample_regs_user */
209+ /* add: sample_stack_user */
197210
198211 /*
199212 * Hardware event_id to monitor via a performance monitoring event:
@@ -255,7 +268,10 @@ struct perf_event_attr { · 55 unmodified lines
255268 exclude_host : 1, /* don't count in host */
256269 exclude_guest : 1, /* don't count in guest */
257270
258− __reserved_1 : 43;
271+ exclude_callchain_kernel : 1, /* exclude kernel callchains */
272+ exclude_callchain_user : 1, /* exclude user callchains */
273+
274+ __reserved_1 : 41;
259275
260276 union {
261277 __u32 wakeup_events; /* wakeup every n events */
@@ -271,7 +287,21 @@ struct perf_event_attr { · 9 unmodified lines
271287 __u64 bp_len;
272288 __u64 config2; /* extension of config1 */
273289 };
274− __u64 branch_sample_type; /* enum branch_sample_type */
290+ __u64 branch_sample_type; /* enum perf_branch_sample_type */
291+
292+ /*
293+ * Defines set of user regs to dump on samples.
294+ * See asm/perf_regs.h for details.
295+ */
296+ __u64 sample_regs_user;
297+
298+ /*
299+ * Defines size of the user stack to dump on samples.
300+ */
301+ __u32 sample_stack_user;
302+
303+ /* Align to u64. */
304+ __u32 __reserved_2;
275305 };
276306
277307 #define perf_flags(attr) (*(&(attr)->read_format + 1))
@@ -550,6 +580,13 @@ enum perf_event_type { · 272 unmodified lines
550580 * char data[size];}&& PERF_SAMPLE_RAW
551581 *
552582 * { u64 from, to, flags } lbr[nr];} && PERF_SAMPLE_BRANCH_STACK
583+ *
584+ * { u64 abi; # enum perf_sample_regs_abi
585+ * u64 regs[weight(mask)]; } && PERF_SAMPLE_REGS_USER
586+ *
587+ * { u64 size;
588+ * char data[size];
589+ * u64 dyn_size; } && PERF_SAMPLE_STACK_USER
553590 * };
554591 */
555592 PERF_RECORD_SAMPLE = 9,
@@ -611,6 +648,7 @@ struct perf_guest_info_callbacks { · 55 unmodified lines
611648 #include <linux/static_key.h>
612649 #include <linux/atomic.h>
613650 #include <linux/sysfs.h>
651+#include <linux/perf_regs.h>
614652 #include <asm/local.h>
615653
616654 struct perf_callchain_entry {
@@ -656,6 +694,11 @@ struct perf_branch_stack { · 39 unmodified lines
656694 struct perf_branch_entry entries[0];
657695 };
658696
697+struct perf_regs_user {
698+ __u64 abi;
699+ struct pt_regs *regs;
700+};
701+
659702 struct task_struct;
660703
661704 /*
@@ -1135,6 +1178,8 @@ struct perf_sample_data { · 473 unmodified lines
11351178 struct perf_callchain_entry *callchain;
11361179 struct perf_raw_record *raw;
11371180 struct perf_branch_stack *br_stack;
1181+ struct perf_regs_user regs_user;
1182+ u64 stack_user_size;
11381183 };
11391184
11401185 static inline void perf_sample_data_init(struct perf_sample_data *data,
@@ -1144,7 +1189,10 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, · 3 unmodified lines
11441189 data->addr = addr;
11451190 data->raw = NULL;
11461191 data->br_stack = NULL;
1147− data->period = period;
1192+ data->period = period;
1193+ data->regs_user.abi = PERF_SAMPLE_REGS_ABI_NONE;
1194+ data->regs_user.regs = NULL;
1195+ data->stack_user_size = 0;
11481196 }
11491197
11501198 extern void perf_output_sample(struct perf_output_handle *handle,
@@ -1292,8 +1340,10 @@ static inline bool has_branch_stack(struct perf_event *event) · 141 unmodified lines
12921340 extern int perf_output_begin(struct perf_output_handle *handle,
12931341 struct perf_event *event, unsigned int size);
12941342 extern void perf_output_end(struct perf_output_handle *handle);
1295−extern void perf_output_copy(struct perf_output_handle *handle,
1343+extern unsigned int perf_output_copy(struct perf_output_handle *handle,
12961344 const void *buf, unsigned int len);
1345+extern unsigned int perf_output_skip(struct perf_output_handle *handle,
1346+ unsigned int len);
12971347 extern int perf_swevent_get_recursion_context(void);
12981348 extern void perf_swevent_put_recursion_context(int rctx);
12991349 extern void perf_event_enable(struct perf_event *event);
include/linux/perf_regs.h+25−0
@@ -0,0 +1,25 @@
1+#ifndef _LINUX_PERF_REGS_H
2+#define _LINUX_PERF_REGS_H
3+
4+#ifdef CONFIG_HAVE_PERF_REGS
5+#include <asm/perf_regs.h>
6+u64 perf_reg_value(struct pt_regs *regs, int idx);
7+int perf_reg_validate(u64 mask);
8+u64 perf_reg_abi(struct task_struct *task);
9+#else
10+static inline u64 perf_reg_value(struct pt_regs *regs, int idx)
11+{
12+ return 0;
13+}
14+
15+static inline int perf_reg_validate(u64 mask)
16+{
17+ return mask ? -ENOSYS : 0;
18+}
19+
20+static inline u64 perf_reg_abi(struct task_struct *task)
21+{
22+ return PERF_SAMPLE_REGS_ABI_NONE;
23+}
24+#endif /* CONFIG_HAVE_PERF_REGS */
25+#endif /* _LINUX_PERF_REGS_H */
include/linux/sched.h+3−0
@@ -446,6 +446,9 @@ extern int get_dumpable(struct mm_struct *mm); · 445 unmodified lines
446446 #define MMF_VM_HUGEPAGE 17 /* set when VM_HUGEPAGE is set on vma */
447447 #define MMF_EXE_FILE_CHANGED 18 /* see prctl_set_mm_exe_file() */
448448
449+#define MMF_HAS_UPROBES 19 /* has uprobes */
450+#define MMF_RECALC_UPROBES 20 /* MMF_HAS_UPROBES can be wrong */
451+
449452 #define MMF_INIT_MASK (MMF_DUMPABLE_MASK | MMF_DUMP_FILTER_MASK)
450453
451454 struct sighand_struct {
include/linux/uprobes.h+9−6
@@ -99,25 +99,27 @@ struct xol_area { · 98 unmodified lines
9999
100100 struct uprobes_state {
101101 struct xol_area *xol_area;
102− atomic_t count;
103102 };
103+
104104 extern int __weak set_swbp(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
105−extern int __weak set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr, bool verify);
105+extern int __weak set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
106106 extern bool __weak is_swbp_insn(uprobe_opcode_t *insn);
107107 extern int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
108108 extern void uprobe_unregister(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
109109 extern int uprobe_mmap(struct vm_area_struct *vma);
110110 extern void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end);
111+extern void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm);
111112 extern void uprobe_free_utask(struct task_struct *t);
112113 extern void uprobe_copy_process(struct task_struct *t);
113114 extern unsigned long __weak uprobe_get_swbp_addr(struct pt_regs *regs);
115+extern void __weak arch_uprobe_enable_step(struct arch_uprobe *arch);
116+extern void __weak arch_uprobe_disable_step(struct arch_uprobe *arch);
114117 extern int uprobe_post_sstep_notifier(struct pt_regs *regs);
115118 extern int uprobe_pre_sstep_notifier(struct pt_regs *regs);
116119 extern void uprobe_notify_resume(struct pt_regs *regs);
117120 extern bool uprobe_deny_signal(void);
118121 extern bool __weak arch_uprobe_skip_sstep(struct arch_uprobe *aup, struct pt_regs *regs);
119122 extern void uprobe_clear_state(struct mm_struct *mm);
120−extern void uprobe_reset_state(struct mm_struct *mm);
121123 #else /* !CONFIG_UPROBES */
122124 struct uprobes_state {
123125 };
@@ -138,6 +140,10 @@ static inline void · 14 unmodified lines
138140 uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end)
139141 {
140142 }
143+static inline void
144+uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm)
145+{
146+}
141147 static inline void uprobe_notify_resume(struct pt_regs *regs)
142148 {
143149 }
@@ -158,8 +164,5 @@ static inline void uprobe_copy_process(struct task_struct *t) · 14 unmodified lines
158164 static inline void uprobe_clear_state(struct mm_struct *mm)
159165 {
160166 }
161−static inline void uprobe_reset_state(struct mm_struct *mm)
162−{
163−}
164167 #endif /* !CONFIG_UPROBES */
165168 #endif /* _LINUX_UPROBES_H */
kernel/Makefile+1−1
@@ -97,7 +97,7 @@ obj-$(CONFIG_COMPAT_BINFMT_ELF) += elfcore.o · 96 unmodified lines
9797 obj-$(CONFIG_BINFMT_ELF_FDPIC) += elfcore.o
9898 obj-$(CONFIG_FUNCTION_TRACER) += trace/
9999 obj-$(CONFIG_TRACING) += trace/
100−obj-$(CONFIG_X86_DS) += trace/
100+obj-$(CONFIG_TRACE_CLOCK) += trace/
101101 obj-$(CONFIG_RING_BUFFER) += trace/
102102 obj-$(CONFIG_TRACEPOINTS) += trace/
103103 obj-$(CONFIG_IRQ_WORK) += irq_work.o
kernel/events/callchain.c+24−14
@@ -159,6 +159,11 @@ perf_callchain(struct perf_event *event, struct pt_regs *regs) · 158 unmodified lines
159159 int rctx;
160160 struct perf_callchain_entry *entry;
161161
162+ int kernel = !event->attr.exclude_callchain_kernel;
163+ int user = !event->attr.exclude_callchain_user;
164+
165+ if (!kernel && !user)
166+ return NULL;
162167
163168 entry = get_callchain_entry(&rctx);
164169 if (rctx == -1)
@@ -169,24 +174,29 @@ perf_callchain(struct perf_event *event, struct pt_regs *regs) · 4 unmodified lines
169174
170175 entry->nr = 0;
171176
172− if (!user_mode(regs)) {
177+ if (kernel && !user_mode(regs)) {
173178 perf_callchain_store(entry, PERF_CONTEXT_KERNEL);
174179 perf_callchain_kernel(entry, regs);
175− if (current->mm)
176− regs = task_pt_regs(current);
177− else
178− regs = NULL;
179180 }
180181
181− if (regs) {
182− /*
183− * Disallow cross-task user callchains.
184− */
185− if (event->ctx->task && event->ctx->task != current)
186− goto exit_put;
187−
188− perf_callchain_store(entry, PERF_CONTEXT_USER);
189− perf_callchain_user(entry, regs);
182+ if (user) {
183+ if (!user_mode(regs)) {
184+ if (current->mm)
185+ regs = task_pt_regs(current);
186+ else
187+ regs = NULL;
188+ }
189+
190+ if (regs) {
191+ /*
192+ * Disallow cross-task user callchains.
193+ */
194+ if (event->ctx->task && event->ctx->task != current)
195+ goto exit_put;
196+
197+ perf_callchain_store(entry, PERF_CONTEXT_USER);
198+ perf_callchain_user(entry, regs);
199+ }
190200 }
191201
192202 exit_put:
kernel/events/core.c+214−0
@@ -36,6 +36,7 @@ · 35 unmodified lines
3636 #include <linux/perf_event.h>
3737 #include <linux/ftrace_event.h>
3838 #include <linux/hw_breakpoint.h>
39+#include <linux/mm_types.h>
3940
4041 #include "internal.h"
4142
@@ -3764,6 +3765,132 @@ int perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) · 3722 unmodified lines
37643765 }
37653766 EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
37663767
3768+static void
3769+perf_output_sample_regs(struct perf_output_handle *handle,
3770+ struct pt_regs *regs, u64 mask)
3771+{
3772+ int bit;
3773+
3774+ for_each_set_bit(bit, (const unsigned long *) &mask,
3775+ sizeof(mask) * BITS_PER_BYTE) {
3776+ u64 val;
3777+
3778+ val = perf_reg_value(regs, bit);
3779+ perf_output_put(handle, val);
3780+ }
3781+}
3782+
3783+static void perf_sample_regs_user(struct perf_regs_user *regs_user,
3784+ struct pt_regs *regs)
3785+{
3786+ if (!user_mode(regs)) {
3787+ if (current->mm)
3788+ regs = task_pt_regs(current);
3789+ else
3790+ regs = NULL;
3791+ }
3792+
3793+ if (regs) {
3794+ regs_user->regs = regs;
3795+ regs_user->abi = perf_reg_abi(current);
3796+ }
3797+}
3798+
3799+/*
3800+ * Get remaining task size from user stack pointer.
3801+ *
3802+ * It'd be better to take stack vma map and limit this more
3803+ * precisly, but there's no way to get it safely under interrupt,
3804+ * so using TASK_SIZE as limit.
3805+ */
3806+static u64 perf_ustack_task_size(struct pt_regs *regs)
3807+{
3808+ unsigned long addr = perf_user_stack_pointer(regs);
3809+
3810+ if (!addr || addr >= TASK_SIZE)
3811+ return 0;
3812+
3813+ return TASK_SIZE - addr;
3814+}
3815+
3816+static u16
3817+perf_sample_ustack_size(u16 stack_size, u16 header_size,
3818+ struct pt_regs *regs)
3819+{
3820+ u64 task_size;
3821+
3822+ /* No regs, no stack pointer, no dump. */
3823+ if (!regs)
3824+ return 0;
3825+
3826+ /*
3827+ * Check if we fit in with the requested stack size into the:
3828+ * - TASK_SIZE
3829+ * If we don't, we limit the size to the TASK_SIZE.
3830+ *
3831+ * - remaining sample size
3832+ * If we don't, we customize the stack size to
3833+ * fit in to the remaining sample size.
3834+ */
3835+
3836+ task_size = min((u64) USHRT_MAX, perf_ustack_task_size(regs));
3837+ stack_size = min(stack_size, (u16) task_size);
3838+
3839+ /* Current header size plus static size and dynamic size. */
3840+ header_size += 2 * sizeof(u64);
3841+
3842+ /* Do we fit in with the current stack dump size? */
3843+ if ((u16) (header_size + stack_size) < header_size) {
3844+ /*
3845+ * If we overflow the maximum size for the sample,
3846+ * we customize the stack dump size to fit in.
3847+ */
3848+ stack_size = USHRT_MAX - header_size - sizeof(u64);
3849+ stack_size = round_up(stack_size, sizeof(u64));
3850+ }
3851+
3852+ return stack_size;
3853+}
3854+
3855+static void
3856+perf_output_sample_ustack(struct perf_output_handle *handle, u64 dump_size,
3857+ struct pt_regs *regs)
3858+{
3859+ /* Case of a kernel thread, nothing to dump */
3860+ if (!regs) {
3861+ u64 size = 0;
3862+ perf_output_put(handle, size);
3863+ } else {
3864+ unsigned long sp;
3865+ unsigned int rem;
3866+ u64 dyn_size;
3867+
3868+ /*
3869+ * We dump:
3870+ * static size
3871+ * - the size requested by user or the best one we can fit
3872+ * in to the sample max size
3873+ * data
3874+ * - user stack dump data
3875+ * dynamic size
3876+ * - the actual dumped size
3877+ */
3878+
3879+ /* Static size. */
3880+ perf_output_put(handle, dump_size);
3881+
3882+ /* Data. */
3883+ sp = perf_user_stack_pointer(regs);
3884+ rem = __output_copy_user(handle, (void *) sp, dump_size);
3885+ dyn_size = dump_size - rem;
3886+
3887+ perf_output_skip(handle, rem);
3888+
3889+ /* Dynamic size. */
3890+ perf_output_put(handle, dyn_size);
3891+ }
3892+}
3893+
37673894 static void __perf_event_header__init_id(struct perf_event_header *header,
37683895 struct perf_sample_data *data,
37693896 struct perf_event *event)
@@ -4024,6 +4151,28 @@ void perf_output_sample(struct perf_output_handle *handle, · 254 unmodified lines
40244151 perf_output_put(handle, nr);
40254152 }
40264153 }
4154+
4155+ if (sample_type & PERF_SAMPLE_REGS_USER) {
4156+ u64 abi = data->regs_user.abi;
4157+
4158+ /*
4159+ * If there are no regs to dump, notice it through
4160+ * first u64 being zero (PERF_SAMPLE_REGS_ABI_NONE).
4161+ */
4162+ perf_output_put(handle, abi);
4163+
4164+ if (abi) {
4165+ u64 mask = event->attr.sample_regs_user;
4166+ perf_output_sample_regs(handle,
4167+ data->regs_user.regs,
4168+ mask);
4169+ }
4170+ }
4171+
4172+ if (sample_type & PERF_SAMPLE_STACK_USER)
4173+ perf_output_sample_ustack(handle,
4174+ data->stack_user_size,
4175+ data->regs_user.regs);
40274176 }
40284177
40294178 void perf_prepare_sample(struct perf_event_header *header,
@@ -4075,6 +4224,49 @@ void perf_prepare_sample(struct perf_event_header *header, · 45 unmodified lines
40754224 }
40764225 header->size += size;
40774226 }
4227+
4228+ if (sample_type & PERF_SAMPLE_REGS_USER) {
4229+ /* regs dump ABI info */
4230+ int size = sizeof(u64);
4231+
4232+ perf_sample_regs_user(&data->regs_user, regs);
4233+
4234+ if (data->regs_user.regs) {
4235+ u64 mask = event->attr.sample_regs_user;
4236+ size += hweight64(mask) * sizeof(u64);
4237+ }
4238+
4239+ header->size += size;
4240+ }
4241+
4242+ if (sample_type & PERF_SAMPLE_STACK_USER) {
4243+ /*
4244+ * Either we need PERF_SAMPLE_STACK_USER bit to be allways
4245+ * processed as the last one or have additional check added
4246+ * in case new sample type is added, because we could eat
4247+ * up the rest of the sample size.
4248+ */
4249+ struct perf_regs_user *uregs = &data->regs_user;
4250+ u16 stack_size = event->attr.sample_stack_user;
4251+ u16 size = sizeof(u64);
4252+
4253+ if (!uregs->abi)
4254+ perf_sample_regs_user(uregs, regs);
4255+
4256+ stack_size = perf_sample_ustack_size(stack_size, header->size,
4257+ uregs->regs);
4258+
4259+ /*
4260+ * If there is something to dump, add space for the dump
4261+ * itself and for the field that tells the dynamic size,
4262+ * which is how many have been actually dumped.
4263+ */
4264+ if (stack_size)
4265+ size += sizeof(u64) + stack_size;
4266+
4267+ data->stack_user_size = stack_size;
4268+ header->size += size;
4269+ }
40784270 }
40794271
40804272 static void perf_event_output(struct perf_event *event,
@@ -6151,6 +6343,28 @@ static int perf_copy_attr(struct perf_event_attr __user *uattr, · 2070 unmodified lines
61516343 attr->branch_sample_type = mask;
61526344 }
61536345 }
6346+
6347+ if (attr->sample_type & PERF_SAMPLE_REGS_USER) {
6348+ ret = perf_reg_validate(attr->sample_regs_user);
6349+ if (ret)
6350+ return ret;
6351+ }
6352+
6353+ if (attr->sample_type & PERF_SAMPLE_STACK_USER) {
6354+ if (!arch_perf_have_user_stack_dump())
6355+ return -ENOSYS;
6356+
6357+ /*
6358+ * We have __u32 type for the size, but so far
6359+ * we can only use __u16 as maximum due to the
6360+ * __u16 sample size limit.
6361+ */
6362+ if (attr->sample_stack_user >= USHRT_MAX)
6363+ ret = -EINVAL;
6364+ else if (!IS_ALIGNED(attr->sample_stack_user, sizeof(u64)))
6365+ ret = -EINVAL;
6366+ }
6367+
61546368 out:
61556369 return ret;
61566370
kernel/events/internal.h+61−21
@@ -2,6 +2,7 @@ · 1 unmodified line
22 #define _KERNEL_EVENTS_INTERNAL_H
33
44 #include <linux/hardirq.h>
5+#include <linux/uaccess.h>
56
67 /* Buffer handling */
78
@@ -76,30 +77,53 @@ static inline unsigned long perf_data_size(struct ring_buffer *rb) · 68 unmodified lines
7677 return rb->nr_pages << (PAGE_SHIFT + page_order(rb));
7778 }
7879
79−static inline void
80−__output_copy(struct perf_output_handle *handle,
81− const void *buf, unsigned int len)
80+#define DEFINE_OUTPUT_COPY(func_name, memcpy_func) \
81+static inline unsigned int \
82+func_name(struct perf_output_handle *handle, \
83+ const void *buf, unsigned int len) \
84+{ \
85+ unsigned long size, written; \
86+ \
87+ do { \
88+ size = min_t(unsigned long, handle->size, len); \
89+ \
90+ written = memcpy_func(handle->addr, buf, size); \
91+ \
92+ len -= written; \
93+ handle->addr += written; \
94+ buf += written; \
95+ handle->size -= written; \
96+ if (!handle->size) { \
97+ struct ring_buffer *rb = handle->rb; \
98+ \
99+ handle->page++; \
100+ handle->page &= rb->nr_pages - 1; \
101+ handle->addr = rb->data_pages[handle->page]; \
102+ handle->size = PAGE_SIZE << page_order(rb); \
103+ } \
104+ } while (len && written == size); \
105+ \
106+ return len; \
107+}
108+
109+static inline int memcpy_common(void *dst, const void *src, size_t n)
82110 {
83− do {
84− unsigned long size = min_t(unsigned long, handle->size, len);
85−
86− memcpy(handle->addr, buf, size);
87−
88− len -= size;
89− handle->addr += size;
90− buf += size;
91− handle->size -= size;
92− if (!handle->size) {
93− struct ring_buffer *rb = handle->rb;
94−
95− handle->page++;
96− handle->page &= rb->nr_pages - 1;
97− handle->addr = rb->data_pages[handle->page];
98− handle->size = PAGE_SIZE << page_order(rb);
99− }
100− } while (len);
111+ memcpy(dst, src, n);
112+ return n;
101113 }
102114
115+DEFINE_OUTPUT_COPY(__output_copy, memcpy_common)
116+
117+#define MEMCPY_SKIP(dst, src, n) (n)
118+
119+DEFINE_OUTPUT_COPY(__output_skip, MEMCPY_SKIP)
120+
121+#ifndef arch_perf_out_copy_user
122+#define arch_perf_out_copy_user __copy_from_user_inatomic
123+#endif
124+
125+DEFINE_OUTPUT_COPY(__output_copy_user, arch_perf_out_copy_user)
126+
103127 /* Callchain handling */
104128 extern struct perf_callchain_entry *
105129 perf_callchain(struct perf_event *event, struct pt_regs *regs);
@@ -134,4 +158,20 @@ static inline void put_recursion_context(int *recursion, int rctx) · 28 unmodified lines
134158 recursion[rctx]--;
135159 }
136160
161+#ifdef CONFIG_HAVE_PERF_USER_STACK_DUMP
162+static inline bool arch_perf_have_user_stack_dump(void)
163+{
164+ return true;
165+}
166+
167+#define perf_user_stack_pointer(regs) user_stack_pointer(regs)
168+#else
169+static inline bool arch_perf_have_user_stack_dump(void)
170+{
171+ return false;
172+}
173+
174+#define perf_user_stack_pointer(regs) 0
175+#endif /* CONFIG_HAVE_PERF_USER_STACK_DUMP */
176+
137177 #endif /* _KERNEL_EVENTS_INTERNAL_H */
kernel/events/ring_buffer.c+8−2
@@ -182,10 +182,16 @@ out: · 181 unmodified lines
182182 return -ENOSPC;
183183 }
184184
185−void perf_output_copy(struct perf_output_handle *handle,
185+unsigned int perf_output_copy(struct perf_output_handle *handle,
186186 const void *buf, unsigned int len)
187187 {
188− __output_copy(handle, buf, len);
188+ return __output_copy(handle, buf, len);
189+}
190+
191+unsigned int perf_output_skip(struct perf_output_handle *handle,
192+ unsigned int len)
193+{
194+ return __output_skip(handle, NULL, len);
189195 }
190196
191197 void perf_output_end(struct perf_output_handle *handle)
kernel/events/uprobes.c+118−130
@@ -280,12 +280,10 @@ static int read_opcode(struct mm_struct *mm, unsigned long vaddr, uprobe_opcode_ · 279 unmodified lines
280280 if (ret <= 0)
281281 return ret;
282282
283− lock_page(page);
284283 vaddr_new = kmap_atomic(page);
285284 vaddr &= ~PAGE_MASK;
286285 memcpy(opcode, vaddr_new + vaddr, UPROBE_SWBP_INSN_SIZE);
287286 kunmap_atomic(vaddr_new);
288− unlock_page(page);
289287
290288 put_page(page);
291289
@@ -334,7 +332,7 @@ int __weak set_swbp(struct arch_uprobe *auprobe, struct mm_struct *mm, unsigned · 42 unmodified lines
334332 */
335333 result = is_swbp_at_addr(mm, vaddr);
336334 if (result == 1)
337− return -EEXIST;
335+ return 0;
338336
339337 if (result)
340338 return result;
@@ -347,24 +345,22 @@ int __weak set_swbp(struct arch_uprobe *auprobe, struct mm_struct *mm, unsigned · 6 unmodified lines
347345 * @mm: the probed process address space.
348346 * @auprobe: arch specific probepoint information.
349347 * @vaddr: the virtual address to insert the opcode.
350− * @verify: if true, verify existance of breakpoint instruction.
351348 *
352349 * For mm @mm, restore the original opcode (opcode) at @vaddr.
353350 * Return 0 (success) or a negative errno.
354351 */
355352 int __weak
356−set_orig_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, unsigned long vaddr, bool verify)
353+set_orig_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, unsigned long vaddr)
357354 {
358− if (verify) {
359− int result;
355+ int result;
356+
357+ result = is_swbp_at_addr(mm, vaddr);
358+ if (!result)
359+ return -EINVAL;
360360
361− result = is_swbp_at_addr(mm, vaddr);
362− if (!result)
363− return -EINVAL;
361+ if (result != 1)
362+ return result;
364363
365− if (result != 1)
366− return result;
367− }
368364 return write_opcode(auprobe, mm, vaddr, *(uprobe_opcode_t *)auprobe->insn);
369365 }
370366
@@ -415,11 +411,10 @@ static struct uprobe *__find_uprobe(struct inode *inode, loff_t offset) · 44 unmodified lines
415411 static struct uprobe *find_uprobe(struct inode *inode, loff_t offset)
416412 {
417413 struct uprobe *uprobe;
418− unsigned long flags;
419414
420− spin_lock_irqsave(&uprobes_treelock, flags);
415+ spin_lock(&uprobes_treelock);
421416 uprobe = __find_uprobe(inode, offset);
422− spin_unlock_irqrestore(&uprobes_treelock, flags);
417+ spin_unlock(&uprobes_treelock);
423418
424419 return uprobe;
425420 }
@@ -466,12 +461,11 @@ static struct uprobe *__insert_uprobe(struct uprobe *uprobe) · 40 unmodified lines
466461 */
467462 static struct uprobe *insert_uprobe(struct uprobe *uprobe)
468463 {
469− unsigned long flags;
470464 struct uprobe *u;
471465
472− spin_lock_irqsave(&uprobes_treelock, flags);
466+ spin_lock(&uprobes_treelock);
473467 u = __insert_uprobe(uprobe);
474− spin_unlock_irqrestore(&uprobes_treelock, flags);
468+ spin_unlock(&uprobes_treelock);
475469
476470 /* For now assume that the instruction need not be single-stepped */
477471 uprobe->flags |= UPROBE_SKIP_SSTEP;
@@ -649,6 +643,7 @@ static int · 171 unmodified lines
649643 install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm,
650644 struct vm_area_struct *vma, unsigned long vaddr)
651645 {
646+ bool first_uprobe;
652647 int ret;
653648
654649 /*
@@ -659,7 +654,7 @@ install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm, · 4 unmodified lines
659654 * Hence behave as if probe already existed.
660655 */
661656 if (!uprobe->consumers)
662− return -EEXIST;
657+ return 0;
663658
664659 if (!(uprobe->flags & UPROBE_COPY_INSN)) {
665660 ret = copy_insn(uprobe, vma->vm_file);
@@ -681,17 +676,18 @@ install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm, · 15 unmodified lines
681676 }
682677
683678 /*
684− * Ideally, should be updating the probe count after the breakpoint
685− * has been successfully inserted. However a thread could hit the
686− * breakpoint we just inserted even before the probe count is
687− * incremented. If this is the first breakpoint placed, breakpoint
688− * notifier might ignore uprobes and pass the trap to the thread.
689− * Hence increment before and decrement on failure.
679+ * set MMF_HAS_UPROBES in advance for uprobe_pre_sstep_notifier(),
680+ * the task can hit this breakpoint right after __replace_page().
690681 */
691− atomic_inc(&mm->uprobes_state.count);
682+ first_uprobe = !test_bit(MMF_HAS_UPROBES, &mm->flags);
683+ if (first_uprobe)
684+ set_bit(MMF_HAS_UPROBES, &mm->flags);
685+
692686 ret = set_swbp(&uprobe->arch, mm, vaddr);
693− if (ret)
694− atomic_dec(&mm->uprobes_state.count);
687+ if (!ret)
688+ clear_bit(MMF_RECALC_UPROBES, &mm->flags);
689+ else if (first_uprobe)
690+ clear_bit(MMF_HAS_UPROBES, &mm->flags);
695691
696692 return ret;
697693 }
@@ -699,8 +695,12 @@ install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm, · 1 unmodified line
699695 static void
700696 remove_breakpoint(struct uprobe *uprobe, struct mm_struct *mm, unsigned long vaddr)
701697 {
702− if (!set_orig_insn(&uprobe->arch, mm, vaddr, true))
703− atomic_dec(&mm->uprobes_state.count);
698+ /* can happen if uprobe_register() fails */
699+ if (!test_bit(MMF_HAS_UPROBES, &mm->flags))
700+ return;
701+
702+ set_bit(MMF_RECALC_UPROBES, &mm->flags);
703+ set_orig_insn(&uprobe->arch, mm, vaddr);
704704 }
705705
706706 /*
@@ -710,11 +710,9 @@ remove_breakpoint(struct uprobe *uprobe, struct mm_struct *mm, unsigned long vad · 3 unmodified lines
710710 */
711711 static void delete_uprobe(struct uprobe *uprobe)
712712 {
713− unsigned long flags;
714−
715− spin_lock_irqsave(&uprobes_treelock, flags);
713+ spin_lock(&uprobes_treelock);
716714 rb_erase(&uprobe->rb_node, &uprobes_tree);
717− spin_unlock_irqrestore(&uprobes_treelock, flags);
715+ spin_unlock(&uprobes_treelock);
718716 iput(uprobe->inode);
719717 put_uprobe(uprobe);
720718 atomic_dec(&uprobe_events);
@@ -831,17 +829,11 @@ static int register_for_each_vma(struct uprobe *uprobe, bool is_register) · 110 unmodified lines
831829 vaddr_to_offset(vma, info->vaddr) != uprobe->offset)
832830 goto unlock;
833831
834− if (is_register) {
832+ if (is_register)
835833 err = install_breakpoint(uprobe, mm, vma, info->vaddr);
836− /*
837− * We can race against uprobe_mmap(), see the
838− * comment near uprobe_hash().
839− */
840− if (err == -EEXIST)
841− err = 0;
842− } else {
834+ else
843835 remove_breakpoint(uprobe, mm, info->vaddr);
844− }
836+
845837 unlock:
846838 up_write(&mm->mmap_sem);
847839 free:
@@ -908,7 +900,8 @@ int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer * · 60 unmodified lines
908900 }
909901
910902 mutex_unlock(uprobes_hash(inode));
911− put_uprobe(uprobe);
903+ if (uprobe)
904+ put_uprobe(uprobe);
912905
913906 return ret;
914907 }
@@ -978,7 +971,6 @@ static void build_probe_list(struct inode *inode, · 63 unmodified lines
978971 struct list_head *head)
979972 {
980973 loff_t min, max;
981− unsigned long flags;
982974 struct rb_node *n, *t;
983975 struct uprobe *u;
984976
@@ -986,7 +978,7 @@ static void build_probe_list(struct inode *inode, · 1 unmodified line
986978 min = vaddr_to_offset(vma, start);
987979 max = min + (end - start) - 1;
988980
989− spin_lock_irqsave(&uprobes_treelock, flags);
981+ spin_lock(&uprobes_treelock);
990982 n = find_node_in_range(inode, min, max);
991983 if (n) {
992984 for (t = n; t; t = rb_prev(t)) {
@@ -1004,27 +996,20 @@ static void build_probe_list(struct inode *inode, · 11 unmodified lines
1004996 atomic_inc(&u->ref);
1005997 }
1006998 }
1007− spin_unlock_irqrestore(&uprobes_treelock, flags);
999+ spin_unlock(&uprobes_treelock);
10081000 }
10091001
10101002 /*
1011− * Called from mmap_region.
1012− * called with mm->mmap_sem acquired.
1003+ * Called from mmap_region/vma_adjust with mm->mmap_sem acquired.
10131004 *
1014− * Return -ve no if we fail to insert probes and we cannot
1015− * bail-out.
1016− * Return 0 otherwise. i.e:
1017− *
1018− * - successful insertion of probes
1019− * - (or) no possible probes to be inserted.
1020− * - (or) insertion of probes failed but we can bail-out.
1005+ * Currently we ignore all errors and always return 0, the callers
1006+ * can't handle the failure anyway.
10211007 */
10221008 int uprobe_mmap(struct vm_area_struct *vma)
10231009 {
10241010 struct list_head tmp_list;
10251011 struct uprobe *uprobe, *u;
10261012 struct inode *inode;
1027− int ret, count;
10281013
10291014 if (!atomic_read(&uprobe_events) || !valid_vma(vma, true))
10301015 return 0;
@@ -1036,44 +1021,35 @@ int uprobe_mmap(struct vm_area_struct *vma) · 5 unmodified lines
10361021 mutex_lock(uprobes_mmap_hash(inode));
10371022 build_probe_list(inode, vma, vma->vm_start, vma->vm_end, &tmp_list);
10381023
1039− ret = 0;
1040− count = 0;
1041−
10421024 list_for_each_entry_safe(uprobe, u, &tmp_list, pending_list) {
1043− if (!ret) {
1025+ if (!fatal_signal_pending(current)) {
10441026 unsigned long vaddr = offset_to_vaddr(vma, uprobe->offset);
1045−
1046− ret = install_breakpoint(uprobe, vma->vm_mm, vma, vaddr);
1047− /*
1048− * We can race against uprobe_register(), see the
1049− * comment near uprobe_hash().
1050− */
1051− if (ret == -EEXIST) {
1052− ret = 0;
1053−
1054− if (!is_swbp_at_addr(vma->vm_mm, vaddr))
1055− continue;
1056−
1057− /*
1058− * Unable to insert a breakpoint, but
1059− * breakpoint lies underneath. Increment the
1060− * probe count.
1061− */
1062− atomic_inc(&vma->vm_mm->uprobes_state.count);
1063− }
1064−
1065− if (!ret)
1066− count++;
1027+ install_breakpoint(uprobe, vma->vm_mm, vma, vaddr);
10671028 }
10681029 put_uprobe(uprobe);
10691030 }
1070−
10711031 mutex_unlock(uprobes_mmap_hash(inode));
10721032
1073− if (ret)
1074− atomic_sub(count, &vma->vm_mm->uprobes_state.count);
1033+ return 0;
1034+}
10751035
1076− return ret;
1036+static bool
1037+vma_has_uprobes(struct vm_area_struct *vma, unsigned long start, unsigned long end)
1038+{
1039+ loff_t min, max;
1040+ struct inode *inode;
1041+ struct rb_node *n;
1042+
1043+ inode = vma->vm_file->f_mapping->host;
1044+
1045+ min = vaddr_to_offset(vma, start);
1046+ max = min + (end - start) - 1;
1047+
1048+ spin_lock(&uprobes_treelock);
1049+ n = find_node_in_range(inode, min, max);
1050+ spin_unlock(&uprobes_treelock);
1051+
1052+ return !!n;
10771053 }
10781054
10791055 /*
@@ -1081,37 +1057,18 @@ int uprobe_mmap(struct vm_area_struct *vma) · 1 unmodified line
10811057 */
10821058 void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end)
10831059 {
1084− struct list_head tmp_list;
1085− struct uprobe *uprobe, *u;
1086− struct inode *inode;
1087−
10881060 if (!atomic_read(&uprobe_events) || !valid_vma(vma, false))
10891061 return;
10901062
10911063 if (!atomic_read(&vma->vm_mm->mm_users)) /* called by mmput() ? */
10921064 return;
10931065
1094− if (!atomic_read(&vma->vm_mm->uprobes_state.count))
1095− return;
1096−
1097− inode = vma->vm_file->f_mapping->host;
1098− if (!inode)
1066+ if (!test_bit(MMF_HAS_UPROBES, &vma->vm_mm->flags) ||
1067+ test_bit(MMF_RECALC_UPROBES, &vma->vm_mm->flags))
10991068 return;
11001069
1101− mutex_lock(uprobes_mmap_hash(inode));
1102− build_probe_list(inode, vma, start, end, &tmp_list);
1103−
1104− list_for_each_entry_safe(uprobe, u, &tmp_list, pending_list) {
1105− unsigned long vaddr = offset_to_vaddr(vma, uprobe->offset);
1106− /*
1107− * An unregister could have removed the probe before
1108− * unmap. So check before we decrement the count.
1109− */
1110− if (is_swbp_at_addr(vma->vm_mm, vaddr) == 1)
1111− atomic_dec(&vma->vm_mm->uprobes_state.count);
1112− put_uprobe(uprobe);
1113− }
1114− mutex_unlock(uprobes_mmap_hash(inode));
1070+ if (vma_has_uprobes(vma, start, end))
1071+ set_bit(MMF_RECALC_UPROBES, &vma->vm_mm->flags);
11151072 }
11161073
11171074 /* Slot allocation for XOL */
@@ -1213,13 +1170,15 @@ void uprobe_clear_state(struct mm_struct *mm) · 95 unmodified lines
12131170 kfree(area);
12141171 }
12151172
1216−/*
1217− * uprobe_reset_state - Free the area allocated for slots.
1218− */
1219−void uprobe_reset_state(struct mm_struct *mm)
1173+void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm)
12201174 {
1221− mm->uprobes_state.xol_area = NULL;
1222− atomic_set(&mm->uprobes_state.count, 0);
1175+ newmm->uprobes_state.xol_area = NULL;
1176+
1177+ if (test_bit(MMF_HAS_UPROBES, &oldmm->flags)) {
1178+ set_bit(MMF_HAS_UPROBES, &newmm->flags);
1179+ /* unconditionally, dup_mmap() skips VM_DONTCOPY vmas */
1180+ set_bit(MMF_RECALC_UPROBES, &newmm->flags);
1181+ }
12231182 }
12241183
12251184 /*
@@ -1437,6 +1396,25 @@ static bool can_skip_sstep(struct uprobe *uprobe, struct pt_regs *regs) · 211 unmodified lines
14371396 return false;
14381397 }
14391398
1399+static void mmf_recalc_uprobes(struct mm_struct *mm)
1400+{
1401+ struct vm_area_struct *vma;
1402+
1403+ for (vma = mm->mmap; vma; vma = vma->vm_next) {
1404+ if (!valid_vma(vma, false))
1405+ continue;
1406+ /*
1407+ * This is not strictly accurate, we can race with
1408+ * uprobe_unregister() and see the already removed
1409+ * uprobe if delete_uprobe() was not yet called.
1410+ */
1411+ if (vma_has_uprobes(vma, vma->vm_start, vma->vm_end))
1412+ return;
1413+ }
1414+
1415+ clear_bit(MMF_HAS_UPROBES, &mm->flags);
1416+}
1417+
14401418 static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp)
14411419 {
14421420 struct mm_struct *mm = current->mm;
@@ -1458,11 +1436,24 @@ static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp) · 15 unmodified lines
14581436 } else {
14591437 *is_swbp = -EFAULT;
14601438 }
1439+
1440+ if (!uprobe && test_and_clear_bit(MMF_RECALC_UPROBES, &mm->flags))
1441+ mmf_recalc_uprobes(mm);
14611442 up_read(&mm->mmap_sem);
14621443
14631444 return uprobe;
14641445 }
14651446
1447+void __weak arch_uprobe_enable_step(struct arch_uprobe *arch)
1448+{
1449+ user_enable_single_step(current);
1450+}
1451+
1452+void __weak arch_uprobe_disable_step(struct arch_uprobe *arch)
1453+{
1454+ user_disable_single_step(current);
1455+}
1456+
14661457 /*
14671458 * Run handler and ask thread to singlestep.
14681459 * Ensure all non-fatal signals cannot interrupt thread while it singlesteps.
@@ -1509,7 +1500,7 @@ static void handle_swbp(struct pt_regs *regs) · 40 unmodified lines
15091500
15101501 utask->state = UTASK_SSTEP;
15111502 if (!pre_ssout(uprobe, regs, bp_vaddr)) {
1512− user_enable_single_step(current);
1503+ arch_uprobe_enable_step(&uprobe->arch);
15131504 return;
15141505 }
15151506
@@ -1518,17 +1509,15 @@ cleanup_ret: · 2 unmodified lines
15181509 utask->active_uprobe = NULL;
15191510 utask->state = UTASK_RUNNING;
15201511 }
1521− if (uprobe) {
1522− if (!(uprobe->flags & UPROBE_SKIP_SSTEP))
1512+ if (!(uprobe->flags & UPROBE_SKIP_SSTEP))
15231513
1524− /*
1525− * cannot singlestep; cannot skip instruction;
1526− * re-execute the instruction.
1527− */
1528− instruction_pointer_set(regs, bp_vaddr);
1514+ /*
1515+ * cannot singlestep; cannot skip instruction;
1516+ * re-execute the instruction.
1517+ */
1518+ instruction_pointer_set(regs, bp_vaddr);
15291519
1530− put_uprobe(uprobe);
1531− }
1520+ put_uprobe(uprobe);
15321521 }
15331522
15341523 /*
@@ -1547,10 +1536,10 @@ static void handle_singlestep(struct uprobe_task *utask, struct pt_regs *regs) · 12 unmodified lines
15471536 else
15481537 WARN_ON_ONCE(1);
15491538
1539+ arch_uprobe_disable_step(&uprobe->arch);
15501540 put_uprobe(uprobe);
15511541 utask->active_uprobe = NULL;
15521542 utask->state = UTASK_RUNNING;
1553− user_disable_single_step(current);
15541543 xol_free_insn_slot(current);
15551544
15561545 spin_lock_irq(&current->sighand->siglock);
@@ -1589,8 +1578,7 @@ int uprobe_pre_sstep_notifier(struct pt_regs *regs) · 32 unmodified lines
15891578 {
15901579 struct uprobe_task *utask;
15911580
1592− if (!current->mm || !atomic_read(&current->mm->uprobes_state.count))
1593− /* task is currently not uprobed */
1581+ if (!current->mm || !test_bit(MMF_HAS_UPROBES, &current->mm->flags))
15941582 return 0;
15951583
15961584 utask = current->utask;
kernel/fork.c+1−5
@@ -353,6 +353,7 @@ static int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm) · 352 unmodified lines
353353
354354 down_write(&oldmm->mmap_sem);
355355 flush_cache_dup_mm(oldmm);
356+ uprobe_dup_mmap(oldmm, mm);
356357 /*
357358 * Not linked in yet - no deadlock potential:
358359 */
@@ -454,9 +455,6 @@ static int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm) · 95 unmodified lines
454455
455456 if (retval)
456457 goto out;
457−
458− if (file)
459− uprobe_mmap(tmp);
460458 }
461459 /* a new mm has just been created */
462460 arch_dup_mmap(oldmm, mm);
@@ -839,8 +837,6 @@ struct mm_struct *dup_mm(struct task_struct *tsk) · 376 unmodified lines
839837 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
840838 mm->pmd_huge_pte = NULL;
841839 #endif
842− uprobe_reset_state(mm);
843−
844840 if (!mm_init(mm, tsk))
845841 goto fail_nomem;
846842
kernel/kprobes.c+176−71
@@ -561,9 +561,9 @@ static __kprobes void kprobe_optimizer(struct work_struct *work) · 560 unmodified lines
561561 {
562562 LIST_HEAD(free_list);
563563
564+ mutex_lock(&kprobe_mutex);
564565 /* Lock modules while optimizing kprobes */
565566 mutex_lock(&module_mutex);
566− mutex_lock(&kprobe_mutex);
567567
568568 /*
569569 * Step 1: Unoptimize kprobes and collect cleaned (unused and disarmed)
@@ -586,8 +586,8 @@ static __kprobes void kprobe_optimizer(struct work_struct *work) · 16 unmodified lines
586586 /* Step 4: Free cleaned kprobes after quiesence period */
587587 do_free_cleaned_kprobes(&free_list);
588588
589− mutex_unlock(&kprobe_mutex);
590589 mutex_unlock(&module_mutex);
590+ mutex_unlock(&kprobe_mutex);
591591
592592 /* Step 5: Kick optimizer again if needed */
593593 if (!list_empty(&optimizing_list) || !list_empty(&unoptimizing_list))
@@ -759,20 +759,32 @@ static __kprobes void try_to_optimize_kprobe(struct kprobe *p) · 165 unmodified lines
759759 struct kprobe *ap;
760760 struct optimized_kprobe *op;
761761
762+ /* Impossible to optimize ftrace-based kprobe */
763+ if (kprobe_ftrace(p))
764+ return;
765+
766+ /* For preparing optimization, jump_label_text_reserved() is called */
767+ jump_label_lock();
768+ mutex_lock(&text_mutex);
769+
762770 ap = alloc_aggr_kprobe(p);
763771 if (!ap)
764− return;
772+ goto out;
765773
766774 op = container_of(ap, struct optimized_kprobe, kp);
767775 if (!arch_prepared_optinsn(&op->optinsn)) {
768776 /* If failed to setup optimizing, fallback to kprobe */
769777 arch_remove_optimized_kprobe(op);
770778 kfree(op);
771− return;
779+ goto out;
772780 }
773781
774782 init_aggr_kprobe(ap, p);
775− optimize_kprobe(ap);
783+ optimize_kprobe(ap); /* This just kicks optimizer thread */
784+
785+out:
786+ mutex_unlock(&text_mutex);
787+ jump_label_unlock();
776788 }
777789
778790 #ifdef CONFIG_SYSCTL
@@ -907,9 +919,64 @@ static __kprobes struct kprobe *alloc_aggr_kprobe(struct kprobe *p) · 128 unmodified lines
907919 }
908920 #endif /* CONFIG_OPTPROBES */
909921
922+#ifdef KPROBES_CAN_USE_FTRACE
923+static struct ftrace_ops kprobe_ftrace_ops __read_mostly = {
924+ .func = kprobe_ftrace_handler,
925+ .flags = FTRACE_OPS_FL_SAVE_REGS,
926+};
927+static int kprobe_ftrace_enabled;
928+
929+/* Must ensure p->addr is really on ftrace */
930+static int __kprobes prepare_kprobe(struct kprobe *p)
931+{
932+ if (!kprobe_ftrace(p))
933+ return arch_prepare_kprobe(p);
934+
935+ return arch_prepare_kprobe_ftrace(p);
936+}
937+
938+/* Caller must lock kprobe_mutex */
939+static void __kprobes arm_kprobe_ftrace(struct kprobe *p)
940+{
941+ int ret;
942+
943+ ret = ftrace_set_filter_ip(&kprobe_ftrace_ops,
944+ (unsigned long)p->addr, 0, 0);
945+ WARN(ret < 0, "Failed to arm kprobe-ftrace at %p (%d)\n", p->addr, ret);
946+ kprobe_ftrace_enabled++;
947+ if (kprobe_ftrace_enabled == 1) {
948+ ret = register_ftrace_function(&kprobe_ftrace_ops);
949+ WARN(ret < 0, "Failed to init kprobe-ftrace (%d)\n", ret);
950+ }
951+}
952+
953+/* Caller must lock kprobe_mutex */
954+static void __kprobes disarm_kprobe_ftrace(struct kprobe *p)
955+{
956+ int ret;
957+
958+ kprobe_ftrace_enabled--;
959+ if (kprobe_ftrace_enabled == 0) {
960+ ret = unregister_ftrace_function(&kprobe_ftrace_ops);
961+ WARN(ret < 0, "Failed to init kprobe-ftrace (%d)\n", ret);
962+ }
963+ ret = ftrace_set_filter_ip(&kprobe_ftrace_ops,
964+ (unsigned long)p->addr, 1, 0);
965+ WARN(ret < 0, "Failed to disarm kprobe-ftrace at %p (%d)\n", p->addr, ret);
966+}
967+#else /* !KPROBES_CAN_USE_FTRACE */
968+#define prepare_kprobe(p) arch_prepare_kprobe(p)
969+#define arm_kprobe_ftrace(p) do {} while (0)
970+#define disarm_kprobe_ftrace(p) do {} while (0)
971+#endif
972+
910973 /* Arm a kprobe with text_mutex */
911974 static void __kprobes arm_kprobe(struct kprobe *kp)
912975 {
976+ if (unlikely(kprobe_ftrace(kp))) {
977+ arm_kprobe_ftrace(kp);
978+ return;
979+ }
913980 /*
914981 * Here, since __arm_kprobe() doesn't use stop_machine(),
915982 * this doesn't cause deadlock on text_mutex. So, we don't
@@ -921,11 +988,15 @@ static void __kprobes arm_kprobe(struct kprobe *kp) · 5 unmodified lines
921988 }
922989
923990 /* Disarm a kprobe with text_mutex */
924−static void __kprobes disarm_kprobe(struct kprobe *kp)
991+static void __kprobes disarm_kprobe(struct kprobe *kp, bool reopt)
925992 {
993+ if (unlikely(kprobe_ftrace(kp))) {
994+ disarm_kprobe_ftrace(kp);
995+ return;
996+ }
926997 /* Ditto */
927998 mutex_lock(&text_mutex);
928− __disarm_kprobe(kp, true);
999+ __disarm_kprobe(kp, reopt);
9291000 mutex_unlock(&text_mutex);
9301001 }
9311002
@@ -1144,12 +1215,6 @@ static int __kprobes add_new_kprobe(struct kprobe *ap, struct kprobe *p) · 212 unmodified lines
11441215 if (p->post_handler && !ap->post_handler)
11451216 ap->post_handler = aggr_post_handler;
11461217
1147− if (kprobe_disabled(ap) && !kprobe_disabled(p)) {
1148− ap->flags &= ~KPROBE_FLAG_DISABLED;
1149− if (!kprobes_all_disarmed)
1150− /* Arm the breakpoint again. */
1151− __arm_kprobe(ap);
1152− }
11531218 return 0;
11541219 }
11551220
@@ -1189,11 +1254,22 @@ static int __kprobes register_aggr_kprobe(struct kprobe *orig_p, · 33 unmodified lines
11891254 int ret = 0;
11901255 struct kprobe *ap = orig_p;
11911256
1257+ /* For preparing optimization, jump_label_text_reserved() is called */
1258+ jump_label_lock();
1259+ /*
1260+ * Get online CPUs to avoid text_mutex deadlock.with stop machine,
1261+ * which is invoked by unoptimize_kprobe() in add_new_kprobe()
1262+ */
1263+ get_online_cpus();
1264+ mutex_lock(&text_mutex);
1265+
11921266 if (!kprobe_aggrprobe(orig_p)) {
11931267 /* If orig_p is not an aggr_kprobe, create new aggr_kprobe. */
11941268 ap = alloc_aggr_kprobe(orig_p);
1195− if (!ap)
1196− return -ENOMEM;
1269+ if (!ap) {
1270+ ret = -ENOMEM;
1271+ goto out;
1272+ }
11971273 init_aggr_kprobe(ap, orig_p);
11981274 } else if (kprobe_unused(ap))
11991275 /* This probe is going to die. Rescue it */
@@ -1213,7 +1289,7 @@ static int __kprobes register_aggr_kprobe(struct kprobe *orig_p, · 13 unmodified lines
12131289 * free aggr_probe. It will be used next time, or
12141290 * freed by unregister_kprobe.
12151291 */
1216− return ret;
1292+ goto out;
12171293
12181294 /* Prepare optimized instructions if possible. */
12191295 prepare_optimized_kprobe(ap);
@@ -1228,7 +1304,20 @@ static int __kprobes register_aggr_kprobe(struct kprobe *orig_p, · 8 unmodified lines
12281304
12291305 /* Copy ap's insn slot to p */
12301306 copy_kprobe(ap, p);
1231− return add_new_kprobe(ap, p);
1307+ ret = add_new_kprobe(ap, p);
1308+
1309+out:
1310+ mutex_unlock(&text_mutex);
1311+ put_online_cpus();
1312+ jump_label_unlock();
1313+
1314+ if (ret == 0 && kprobe_disabled(ap) && !kprobe_disabled(p)) {
1315+ ap->flags &= ~KPROBE_FLAG_DISABLED;
1316+ if (!kprobes_all_disarmed)
1317+ /* Arm the breakpoint again. */
1318+ arm_kprobe(ap);
1319+ }
1320+ return ret;
12321321 }
12331322
12341323 static int __kprobes in_kprobes_functions(unsigned long addr)
@@ -1313,71 +1402,96 @@ static inline int check_kprobe_rereg(struct kprobe *p) · 78 unmodified lines
13131402 return ret;
13141403 }
13151404
1316−int __kprobes register_kprobe(struct kprobe *p)
1405+static __kprobes int check_kprobe_address_safe(struct kprobe *p,
1406+ struct module **probed_mod)
13171407 {
13181408 int ret = 0;
1319− struct kprobe *old_p;
1320− struct module *probed_mod;
1321− kprobe_opcode_t *addr;
1322−
1323− addr = kprobe_addr(p);
1324− if (IS_ERR(addr))
1325− return PTR_ERR(addr);
1326− p->addr = addr;
1409+ unsigned long ftrace_addr;
13271410
1328− ret = check_kprobe_rereg(p);
1329− if (ret)
1330− return ret;
1411+ /*
1412+ * If the address is located on a ftrace nop, set the
1413+ * breakpoint to the following instruction.
1414+ */
1415+ ftrace_addr = ftrace_location((unsigned long)p->addr);
1416+ if (ftrace_addr) {
1417+#ifdef KPROBES_CAN_USE_FTRACE
1418+ /* Given address is not on the instruction boundary */
1419+ if ((unsigned long)p->addr != ftrace_addr)
1420+ return -EILSEQ;
1421+ p->flags |= KPROBE_FLAG_FTRACE;
1422+#else /* !KPROBES_CAN_USE_FTRACE */
1423+ return -EINVAL;
1424+#endif
1425+ }
13311426
13321427 jump_label_lock();
13331428 preempt_disable();
1429+
1430+ /* Ensure it is not in reserved area nor out of text */
13341431 if (!kernel_text_address((unsigned long) p->addr) ||
13351432 in_kprobes_functions((unsigned long) p->addr) ||
1336− ftrace_text_reserved(p->addr, p->addr) ||
13371433 jump_label_text_reserved(p->addr, p->addr)) {
13381434 ret = -EINVAL;
1339− goto cannot_probe;
1435+ goto out;
13401436 }
13411437
1342− /* User can pass only KPROBE_FLAG_DISABLED to register_kprobe */
1343− p->flags &= KPROBE_FLAG_DISABLED;
1344−
1345− /*
1346− * Check if are we probing a module.
1347− */
1348− probed_mod = __module_text_address((unsigned long) p->addr);
1349− if (probed_mod) {
1350− /* Return -ENOENT if fail. */
1351− ret = -ENOENT;
1438+ /* Check if are we probing a module */
1439+ *probed_mod = __module_text_address((unsigned long) p->addr);
1440+ if (*probed_mod) {
13521441 /*
13531442 * We must hold a refcount of the probed module while updating
13541443 * its code to prohibit unexpected unloading.
13551444 */
1356− if (unlikely(!try_module_get(probed_mod)))
1357− goto cannot_probe;
1445+ if (unlikely(!try_module_get(*probed_mod))) {
1446+ ret = -ENOENT;
1447+ goto out;
1448+ }
13581449
13591450 /*
13601451 * If the module freed .init.text, we couldn't insert
13611452 * kprobes in there.
13621453 */
1363− if (within_module_init((unsigned long)p->addr, probed_mod) &&
1364− probed_mod->state != MODULE_STATE_COMING) {
1365− module_put(probed_mod);
1366− goto cannot_probe;
1454+ if (within_module_init((unsigned long)p->addr, *probed_mod) &&
1455+ (*probed_mod)->state != MODULE_STATE_COMING) {
1456+ module_put(*probed_mod);
1457+ *probed_mod = NULL;
1458+ ret = -ENOENT;
13671459 }
1368− /* ret will be updated by following code */
13691460 }
1461+out:
13701462 preempt_enable();
13711463 jump_label_unlock();
13721464
1465+ return ret;
1466+}
1467+
1468+int __kprobes register_kprobe(struct kprobe *p)
1469+{
1470+ int ret;
1471+ struct kprobe *old_p;
1472+ struct module *probed_mod;
1473+ kprobe_opcode_t *addr;
1474+
1475+ /* Adjust probe address from symbol */
1476+ addr = kprobe_addr(p);
1477+ if (IS_ERR(addr))
1478+ return PTR_ERR(addr);
1479+ p->addr = addr;
1480+
1481+ ret = check_kprobe_rereg(p);
1482+ if (ret)
1483+ return ret;
1484+
1485+ /* User can pass only KPROBE_FLAG_DISABLED to register_kprobe */
1486+ p->flags &= KPROBE_FLAG_DISABLED;
13731487 p->nmissed = 0;
13741488 INIT_LIST_HEAD(&p->list);
1375− mutex_lock(&kprobe_mutex);
13761489
1377− jump_label_lock(); /* needed to call jump_label_text_reserved() */
1490+ ret = check_kprobe_address_safe(p, &probed_mod);
1491+ if (ret)
1492+ return ret;
13781493
1379− get_online_cpus(); /* For avoiding text_mutex deadlock. */
1380− mutex_lock(&text_mutex);
1494+ mutex_lock(&kprobe_mutex);
13811495
13821496 old_p = get_kprobe(p->addr);
13831497 if (old_p) {
@@ -1386,7 +1500,9 @@ int __kprobes register_kprobe(struct kprobe *p) · 2 unmodified lines
13861500 goto out;
13871501 }
13881502
1389− ret = arch_prepare_kprobe(p);
1503+ mutex_lock(&text_mutex); /* Avoiding text modification */
1504+ ret = prepare_kprobe(p);
1505+ mutex_unlock(&text_mutex);
13901506 if (ret)
13911507 goto out;
13921508
@@ -1395,26 +1511,18 @@ int __kprobes register_kprobe(struct kprobe *p) · 2 unmodified lines
13951511 &kprobe_table[hash_ptr(p->addr, KPROBE_HASH_BITS)]);
13961512
13971513 if (!kprobes_all_disarmed && !kprobe_disabled(p))
1398− __arm_kprobe(p);
1514+ arm_kprobe(p);
13991515
14001516 /* Try to optimize kprobe */
14011517 try_to_optimize_kprobe(p);
14021518
14031519 out:
1404− mutex_unlock(&text_mutex);
1405− put_online_cpus();
1406− jump_label_unlock();
14071520 mutex_unlock(&kprobe_mutex);
14081521
14091522 if (probed_mod)
14101523 module_put(probed_mod);
14111524
14121525 return ret;
1413−
1414−cannot_probe:
1415− preempt_enable();
1416− jump_label_unlock();
1417− return ret;
14181526 }
14191527 EXPORT_SYMBOL_GPL(register_kprobe);
14201528
@@ -1451,7 +1559,7 @@ static struct kprobe *__kprobes __disable_kprobe(struct kprobe *p) · 30 unmodified lines
14511559
14521560 /* Try to disarm and disable this/parent probe */
14531561 if (p == orig_p || aggr_kprobe_disabled(orig_p)) {
1454− disarm_kprobe(orig_p);
1562+ disarm_kprobe(orig_p, true);
14551563 orig_p->flags |= KPROBE_FLAG_DISABLED;
14561564 }
14571565 }
@@ -2049,10 +2157,11 @@ static void __kprobes report_probe(struct seq_file *pi, struct kprobe *p, · 591 unmodified lines
20492157
20502158 if (!pp)
20512159 pp = p;
2052− seq_printf(pi, "%s%s%s\n",
2160+ seq_printf(pi, "%s%s%s%s\n",
20532161 (kprobe_gone(p) ? "[GONE]" : ""),
20542162 ((kprobe_disabled(p) && !kprobe_gone(p)) ? "[DISABLED]" : ""),
2055− (kprobe_optimized(pp) ? "[OPTIMIZED]" : ""));
2163+ (kprobe_optimized(pp) ? "[OPTIMIZED]" : ""),
2164+ (kprobe_ftrace(pp) ? "[FTRACE]" : ""));
20562165 }
20572166
20582167 static void __kprobes *kprobe_seq_start(struct seq_file *f, loff_t *pos)
@@ -2131,14 +2240,12 @@ static void __kprobes arm_all_kprobes(void) · 72 unmodified lines
21312240 goto already_enabled;
21322241
21332242 /* Arming kprobes doesn't optimize kprobe itself */
2134− mutex_lock(&text_mutex);
21352243 for (i = 0; i < KPROBE_TABLE_SIZE; i++) {
21362244 head = &kprobe_table[i];
21372245 hlist_for_each_entry_rcu(p, node, head, hlist)
21382246 if (!kprobe_disabled(p))
2139− __arm_kprobe(p);
2247+ arm_kprobe(p);
21402248 }
2141− mutex_unlock(&text_mutex);
21422249
21432250 kprobes_all_disarmed = false;
21442251 printk(KERN_INFO "Kprobes globally enabled\n");
@@ -2166,15 +2273,13 @@ static void __kprobes disarm_all_kprobes(void) · 21 unmodified lines
21662273 kprobes_all_disarmed = true;
21672274 printk(KERN_INFO "Kprobes globally disabled\n");
21682275
2169− mutex_lock(&text_mutex);
21702276 for (i = 0; i < KPROBE_TABLE_SIZE; i++) {
21712277 head = &kprobe_table[i];
21722278 hlist_for_each_entry_rcu(p, node, head, hlist) {
21732279 if (!arch_trampoline_kprobe(p) && !kprobe_disabled(p))
2174− __disarm_kprobe(p, false);
2280+ disarm_kprobe(p, false);
21752281 }
21762282 }
2177− mutex_unlock(&text_mutex);
21782283 mutex_unlock(&kprobe_mutex);
21792284
21802285 /* Wait for disarming all kprobes by optimizer */
kernel/trace/Kconfig+10−0
@@ -49,6 +49,11 @@ config HAVE_SYSCALL_TRACEPOINTS · 48 unmodified lines
4949 help
5050 See Documentation/trace/ftrace-design.txt
5151
52+config HAVE_FENTRY
53+ bool
54+ help
55+ Arch supports the gcc options -pg with -mfentry
56+
5257 config HAVE_C_RECORDMCOUNT
5358 bool
5459 help
@@ -57,8 +62,12 @@ config HAVE_C_RECORDMCOUNT · 2 unmodified lines
5762 config TRACER_MAX_TRACE
5863 bool
5964
65+config TRACE_CLOCK
66+ bool
67+
6068 config RING_BUFFER
6169 bool
70+ select TRACE_CLOCK
6271
6372 config FTRACE_NMI_ENTER
6473 bool
@@ -109,6 +118,7 @@ config TRACING · 44 unmodified lines
109118 select NOP_TRACER
110119 select BINARY_PRINTF
111120 select EVENT_TRACING
121+ select TRACE_CLOCK
112122
113123 config GENERIC_TRACER
114124 bool
kernel/trace/Makefile+3−5
@@ -5,10 +5,12 @@ ifdef CONFIG_FUNCTION_TRACER · 4 unmodified lines
55 ORIG_CFLAGS := $(KBUILD_CFLAGS)
66 KBUILD_CFLAGS = $(subst -pg,,$(ORIG_CFLAGS))
77
8+ifdef CONFIG_FTRACE_SELFTEST
89 # selftest needs instrumentation
910 CFLAGS_trace_selftest_dynamic.o = -pg
1011 obj-y += trace_selftest_dynamic.o
1112 endif
13+endif
1214
1315 # If unlikely tracing is enabled, do not trace these files
1416 ifdef CONFIG_TRACING_BRANCHES
@@ -17,11 +19,7 @@ endif · 2 unmodified lines
1719
1820 CFLAGS_trace_events_filter.o := -I$(src)
1921
20−#
21−# Make the trace clocks available generally: it's infrastructure
22−# relied on by ptrace for example:
23−#
24−obj-y += trace_clock.o
22+obj-$(CONFIG_TRACE_CLOCK) += trace_clock.o
2523
2624 obj-$(CONFIG_FUNCTION_TRACER) += libftrace.o
2725 obj-$(CONFIG_RING_BUFFER) += ring_buffer.o
kernel/trace/ftrace.c+244−78
@@ -64,12 +64,20 @@ · 63 unmodified lines
6464
6565 #define FL_GLOBAL_CONTROL_MASK (FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_CONTROL)
6666
67+static struct ftrace_ops ftrace_list_end __read_mostly = {
68+ .func = ftrace_stub,
69+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
70+};
71+
6772 /* ftrace_enabled is a method to turn ftrace on or off */
6873 int ftrace_enabled __read_mostly;
6974 static int last_ftrace_enabled;
7075
7176 /* Quick disabling of function tracer. */
72−int function_trace_stop;
77+int function_trace_stop __read_mostly;
78+
79+/* Current function tracing op */
80+struct ftrace_ops *function_trace_op __read_mostly = &ftrace_list_end;
7381
7482 /* List for set_ftrace_pid's pids. */
7583 LIST_HEAD(ftrace_pids);
@@ -86,22 +94,43 @@ static int ftrace_disabled __read_mostly; · 10 unmodified lines
8694
8795 static DEFINE_MUTEX(ftrace_lock);
8896
89−static struct ftrace_ops ftrace_list_end __read_mostly = {
90− .func = ftrace_stub,
91−};
92−
9397 static struct ftrace_ops *ftrace_global_list __read_mostly = &ftrace_list_end;
9498 static struct ftrace_ops *ftrace_control_list __read_mostly = &ftrace_list_end;
9599 static struct ftrace_ops *ftrace_ops_list __read_mostly = &ftrace_list_end;
96100 ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub;
97−static ftrace_func_t __ftrace_trace_function_delay __read_mostly = ftrace_stub;
98−ftrace_func_t __ftrace_trace_function __read_mostly = ftrace_stub;
99101 ftrace_func_t ftrace_pid_function __read_mostly = ftrace_stub;
100102 static struct ftrace_ops global_ops;
101103 static struct ftrace_ops control_ops;
102104
103−static void
104−ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip);
105+#if ARCH_SUPPORTS_FTRACE_OPS
106+static void ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
107+ struct ftrace_ops *op, struct pt_regs *regs);
108+#else
109+/* See comment below, where ftrace_ops_list_func is defined */
110+static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip);
111+#define ftrace_ops_list_func ((ftrace_func_t)ftrace_ops_no_ops)
112+#endif
113+
114+/**
115+ * ftrace_nr_registered_ops - return number of ops registered
116+ *
117+ * Returns the number of ftrace_ops registered and tracing functions
118+ */
119+int ftrace_nr_registered_ops(void)
120+{
121+ struct ftrace_ops *ops;
122+ int cnt = 0;
123+
124+ mutex_lock(&ftrace_lock);
125+
126+ for (ops = ftrace_ops_list;
127+ ops != &ftrace_list_end; ops = ops->next)
128+ cnt++;
129+
130+ mutex_unlock(&ftrace_lock);
131+
132+ return cnt;
133+}
105134
106135 /*
107136 * Traverse the ftrace_global_list, invoking all entries. The reason that we
@@ -112,29 +141,29 @@ ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip); · 4 unmodified lines
112141 *
113142 * Silly Alpha and silly pointer-speculation compiler optimizations!
114143 */
115−static void ftrace_global_list_func(unsigned long ip,
116− unsigned long parent_ip)
144+static void
145+ftrace_global_list_func(unsigned long ip, unsigned long parent_ip,
146+ struct ftrace_ops *op, struct pt_regs *regs)
117147 {
118− struct ftrace_ops *op;
119−
120148 if (unlikely(trace_recursion_test(TRACE_GLOBAL_BIT)))
121149 return;
122150
123151 trace_recursion_set(TRACE_GLOBAL_BIT);
124152 op = rcu_dereference_raw(ftrace_global_list); /*see above*/
125153 while (op != &ftrace_list_end) {
126− op->func(ip, parent_ip);
154+ op->func(ip, parent_ip, op, regs);
127155 op = rcu_dereference_raw(op->next); /*see above*/
128156 };
129157 trace_recursion_clear(TRACE_GLOBAL_BIT);
130158 }
131159
132−static void ftrace_pid_func(unsigned long ip, unsigned long parent_ip)
160+static void ftrace_pid_func(unsigned long ip, unsigned long parent_ip,
161+ struct ftrace_ops *op, struct pt_regs *regs)
133162 {
134163 if (!test_tsk_trace_trace(current))
135164 return;
136165
137− ftrace_pid_function(ip, parent_ip);
166+ ftrace_pid_function(ip, parent_ip, op, regs);
138167 }
139168
140169 static void set_ftrace_pid_function(ftrace_func_t func)
@@ -153,25 +182,9 @@ static void set_ftrace_pid_function(ftrace_func_t func) · 12 unmodified lines
153182 void clear_ftrace_function(void)
154183 {
155184 ftrace_trace_function = ftrace_stub;
156− __ftrace_trace_function = ftrace_stub;
157− __ftrace_trace_function_delay = ftrace_stub;
158185 ftrace_pid_function = ftrace_stub;
159186 }
160187
161−#ifndef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
162−/*
163− * For those archs that do not test ftrace_trace_stop in their
164− * mcount call site, we need to do it from C.
165− */
166−static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip)
167−{
168− if (function_trace_stop)
169− return;
170−
171− __ftrace_trace_function(ip, parent_ip);
172−}
173−#endif
174−
175188 static void control_ops_disable_all(struct ftrace_ops *ops)
176189 {
177190 int cpu;
@@ -230,28 +243,27 @@ static void update_ftrace_function(void) · 52 unmodified lines
230243
231244 /*
232245 * If we are at the end of the list and this ops is
233− * not dynamic, then have the mcount trampoline call
234− * the function directly
246+ * recursion safe and not dynamic and the arch supports passing ops,
247+ * then have the mcount trampoline call the function directly.
235248 */
236249 if (ftrace_ops_list == &ftrace_list_end ||
237250 (ftrace_ops_list->next == &ftrace_list_end &&
238− !(ftrace_ops_list->flags & FTRACE_OPS_FL_DYNAMIC)))
251+ !(ftrace_ops_list->flags & FTRACE_OPS_FL_DYNAMIC) &&
252+ (ftrace_ops_list->flags & FTRACE_OPS_FL_RECURSION_SAFE) &&
253+ !FTRACE_FORCE_LIST_FUNC)) {
254+ /* Set the ftrace_ops that the arch callback uses */
255+ if (ftrace_ops_list == &global_ops)
256+ function_trace_op = ftrace_global_list;
257+ else
258+ function_trace_op = ftrace_ops_list;
239259 func = ftrace_ops_list->func;
240− else
260+ } else {
261+ /* Just use the default ftrace_ops */
262+ function_trace_op = &ftrace_list_end;
241263 func = ftrace_ops_list_func;
264+ }
242265
243−#ifdef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
244266 ftrace_trace_function = func;
245−#else
246−#ifdef CONFIG_DYNAMIC_FTRACE
247− /* do not update till all functions have been modified */
248− __ftrace_trace_function_delay = func;
249−#else
250− __ftrace_trace_function = func;
251−#endif
252− ftrace_trace_function =
253− (func == ftrace_stub) ? func : ftrace_test_stop_func;
254−#endif
255267 }
256268
257269 static void add_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
@@ -325,6 +337,20 @@ static int __register_ftrace_function(struct ftrace_ops *ops) · 67 unmodified lines
325337 if ((ops->flags & FL_GLOBAL_CONTROL_MASK) == FL_GLOBAL_CONTROL_MASK)
326338 return -EINVAL;
327339
340+#ifndef ARCH_SUPPORTS_FTRACE_SAVE_REGS
341+ /*
342+ * If the ftrace_ops specifies SAVE_REGS, then it only can be used
343+ * if the arch supports it, or SAVE_REGS_IF_SUPPORTED is also set.
344+ * Setting SAVE_REGS_IF_SUPPORTED makes SAVE_REGS irrelevant.
345+ */
346+ if (ops->flags & FTRACE_OPS_FL_SAVE_REGS &&
347+ !(ops->flags & FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED))
348+ return -EINVAL;
349+
350+ if (ops->flags & FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED)
351+ ops->flags |= FTRACE_OPS_FL_SAVE_REGS;
352+#endif
353+
328354 if (!core_kernel_data((unsigned long)ops))
329355 ops->flags |= FTRACE_OPS_FL_DYNAMIC;
330356
@@ -773,7 +799,8 @@ ftrace_profile_alloc(struct ftrace_profile_stat *stat, unsigned long ip) · 442 unmodified lines
773799 }
774800
775801 static void
776−function_profile_call(unsigned long ip, unsigned long parent_ip)
802+function_profile_call(unsigned long ip, unsigned long parent_ip,
803+ struct ftrace_ops *ops, struct pt_regs *regs)
777804 {
778805 struct ftrace_profile_stat *stat;
779806 struct ftrace_profile *rec;
@@ -803,7 +830,7 @@ function_profile_call(unsigned long ip, unsigned long parent_ip) · 23 unmodified lines
803830 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
804831 static int profile_graph_entry(struct ftrace_graph_ent *trace)
805832 {
806− function_profile_call(trace->func, 0);
833+ function_profile_call(trace->func, 0, NULL, NULL);
807834 return 1;
808835 }
809836
@@ -863,6 +890,7 @@ static void unregister_ftrace_profiler(void) · 53 unmodified lines
863890 #else
864891 static struct ftrace_ops ftrace_profile_ops __read_mostly = {
865892 .func = function_profile_call,
893+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
866894 };
867895
868896 static int register_ftrace_profiler(void)
@@ -1045,6 +1073,7 @@ static struct ftrace_ops global_ops = { · 176 unmodified lines
10451073 .func = ftrace_stub,
10461074 .notrace_hash = EMPTY_HASH,
10471075 .filter_hash = EMPTY_HASH,
1076+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
10481077 };
10491078
10501079 static DEFINE_MUTEX(ftrace_regex_lock);
@@ -1525,6 +1554,12 @@ static void __ftrace_hash_rec_update(struct ftrace_ops *ops, · 474 unmodified lines
15251554 rec->flags++;
15261555 if (FTRACE_WARN_ON((rec->flags & ~FTRACE_FL_MASK) == FTRACE_REF_MAX))
15271556 return;
1557+ /*
1558+ * If any ops wants regs saved for this function
1559+ * then all ops will get saved regs.
1560+ */
1561+ if (ops->flags & FTRACE_OPS_FL_SAVE_REGS)
1562+ rec->flags |= FTRACE_FL_REGS;
15281563 } else {
15291564 if (FTRACE_WARN_ON((rec->flags & ~FTRACE_FL_MASK) == 0))
15301565 return;
@@ -1616,18 +1651,59 @@ static int ftrace_check_record(struct dyn_ftrace *rec, int enable, int update) · 85 unmodified lines
16161651 if (enable && (rec->flags & ~FTRACE_FL_MASK))
16171652 flag = FTRACE_FL_ENABLED;
16181653
1654+ /*
1655+ * If enabling and the REGS flag does not match the REGS_EN, then
1656+ * do not ignore this record. Set flags to fail the compare against
1657+ * ENABLED.
1658+ */
1659+ if (flag &&
1660+ (!(rec->flags & FTRACE_FL_REGS) != !(rec->flags & FTRACE_FL_REGS_EN)))
1661+ flag |= FTRACE_FL_REGS;
1662+
16191663 /* If the state of this record hasn't changed, then do nothing */
16201664 if ((rec->flags & FTRACE_FL_ENABLED) == flag)
16211665 return FTRACE_UPDATE_IGNORE;
16221666
16231667 if (flag) {
1624− if (update)
1668+ /* Save off if rec is being enabled (for return value) */
1669+ flag ^= rec->flags & FTRACE_FL_ENABLED;
1670+
1671+ if (update) {
16251672 rec->flags |= FTRACE_FL_ENABLED;
1626− return FTRACE_UPDATE_MAKE_CALL;
1673+ if (flag & FTRACE_FL_REGS) {
1674+ if (rec->flags & FTRACE_FL_REGS)
1675+ rec->flags |= FTRACE_FL_REGS_EN;
1676+ else
1677+ rec->flags &= ~FTRACE_FL_REGS_EN;
1678+ }
1679+ }
1680+
1681+ /*
1682+ * If this record is being updated from a nop, then
1683+ * return UPDATE_MAKE_CALL.
1684+ * Otherwise, if the EN flag is set, then return
1685+ * UPDATE_MODIFY_CALL_REGS to tell the caller to convert
1686+ * from the non-save regs, to a save regs function.
1687+ * Otherwise,
1688+ * return UPDATE_MODIFY_CALL to tell the caller to convert
1689+ * from the save regs, to a non-save regs function.
1690+ */
1691+ if (flag & FTRACE_FL_ENABLED)
1692+ return FTRACE_UPDATE_MAKE_CALL;
1693+ else if (rec->flags & FTRACE_FL_REGS_EN)
1694+ return FTRACE_UPDATE_MODIFY_CALL_REGS;
1695+ else
1696+ return FTRACE_UPDATE_MODIFY_CALL;
16271697 }
16281698
1629− if (update)
1630− rec->flags &= ~FTRACE_FL_ENABLED;
1699+ if (update) {
1700+ /* If there's no more users, clear all flags */
1701+ if (!(rec->flags & ~FTRACE_FL_MASK))
1702+ rec->flags = 0;
1703+ else
1704+ /* Just disable the record (keep REGS state) */
1705+ rec->flags &= ~FTRACE_FL_ENABLED;
1706+ }
16311707
16321708 return FTRACE_UPDATE_MAKE_NOP;
16331709 }
@@ -1662,13 +1738,17 @@ int ftrace_test_record(struct dyn_ftrace *rec, int enable) · 28 unmodified lines
16621738 static int
16631739 __ftrace_replace_code(struct dyn_ftrace *rec, int enable)
16641740 {
1741+ unsigned long ftrace_old_addr;
16651742 unsigned long ftrace_addr;
16661743 int ret;
16671744
1668− ftrace_addr = (unsigned long)FTRACE_ADDR;
1669−
16701745 ret = ftrace_update_record(rec, enable);
16711746
1747+ if (rec->flags & FTRACE_FL_REGS)
1748+ ftrace_addr = (unsigned long)FTRACE_REGS_ADDR;
1749+ else
1750+ ftrace_addr = (unsigned long)FTRACE_ADDR;
1751+
16721752 switch (ret) {
16731753 case FTRACE_UPDATE_IGNORE:
16741754 return 0;
@@ -1678,6 +1758,15 @@ __ftrace_replace_code(struct dyn_ftrace *rec, int enable) · 3 unmodified lines
16781758
16791759 case FTRACE_UPDATE_MAKE_NOP:
16801760 return ftrace_make_nop(NULL, rec, ftrace_addr);
1761+
1762+ case FTRACE_UPDATE_MODIFY_CALL_REGS:
1763+ case FTRACE_UPDATE_MODIFY_CALL:
1764+ if (rec->flags & FTRACE_FL_REGS)
1765+ ftrace_old_addr = (unsigned long)FTRACE_ADDR;
1766+ else
1767+ ftrace_old_addr = (unsigned long)FTRACE_REGS_ADDR;
1768+
1769+ return ftrace_modify_call(rec, ftrace_old_addr, ftrace_addr);
16811770 }
16821771
16831772 return -1; /* unknow ftrace bug */
@@ -1882,16 +1971,6 @@ static void ftrace_run_update_code(int command) · 198 unmodified lines
18821971 */
18831972 arch_ftrace_update_code(command);
18841973
1885−#ifndef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
1886− /*
1887− * For archs that call ftrace_test_stop_func(), we must
1888− * wait till after we update all the function callers
1889− * before we update the callback. This keeps different
1890− * ops that record different functions from corrupting
1891− * each other.
1892− */
1893− __ftrace_trace_function = __ftrace_trace_function_delay;
1894−#endif
18951974 function_trace_stop--;
18961975
18971976 ret = ftrace_arch_code_modify_post_process();
@@ -2441,8 +2520,9 @@ static int t_show(struct seq_file *m, void *v) · 543 unmodified lines
24412520
24422521 seq_printf(m, "%ps", (void *)rec->ip);
24432522 if (iter->flags & FTRACE_ITER_ENABLED)
2444− seq_printf(m, " (%ld)",
2445− rec->flags & ~FTRACE_FL_MASK);
2523+ seq_printf(m, " (%ld)%s",
2524+ rec->flags & ~FTRACE_FL_MASK,
2525+ rec->flags & FTRACE_FL_REGS ? " R" : "");
24462526 seq_printf(m, "\n");
24472527
24482528 return 0;
@@ -2790,8 +2870,8 @@ static int __init ftrace_mod_cmd_init(void) · 341 unmodified lines
27902870 }
27912871 device_initcall(ftrace_mod_cmd_init);
27922872
2793−static void
2794−function_trace_probe_call(unsigned long ip, unsigned long parent_ip)
2873+static void function_trace_probe_call(unsigned long ip, unsigned long parent_ip,
2874+ struct ftrace_ops *op, struct pt_regs *pt_regs)
27952875 {
27962876 struct ftrace_func_probe *entry;
27972877 struct hlist_head *hhd;
@@ -3162,8 +3242,27 @@ ftrace_notrace_write(struct file *file, const char __user *ubuf, · 364 unmodified lines
31623242 }
31633243
31643244 static int
3165−ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
3166− int reset, int enable)
3245+ftrace_match_addr(struct ftrace_hash *hash, unsigned long ip, int remove)
3246+{
3247+ struct ftrace_func_entry *entry;
3248+
3249+ if (!ftrace_location(ip))
3250+ return -EINVAL;
3251+
3252+ if (remove) {
3253+ entry = ftrace_lookup_ip(hash, ip);
3254+ if (!entry)
3255+ return -ENOENT;
3256+ free_hash_entry(hash, entry);
3257+ return 0;
3258+ }
3259+
3260+ return add_hash_entry(hash, ip);
3261+}
3262+
3263+static int
3264+ftrace_set_hash(struct ftrace_ops *ops, unsigned char *buf, int len,
3265+ unsigned long ip, int remove, int reset, int enable)
31673266 {
31683267 struct ftrace_hash **orig_hash;
31693268 struct ftrace_hash *hash;
@@ -3192,6 +3291,11 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len, · 22 unmodified lines
31923291 ret = -EINVAL;
31933292 goto out_regex_unlock;
31943293 }
3294+ if (ip) {
3295+ ret = ftrace_match_addr(hash, ip, remove);
3296+ if (ret < 0)
3297+ goto out_regex_unlock;
3298+ }
31953299
31963300 mutex_lock(&ftrace_lock);
31973301 ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -3208,6 +3312,37 @@ ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len, · 10 unmodified lines
32083312 return ret;
32093313 }
32103314
3315+static int
3316+ftrace_set_addr(struct ftrace_ops *ops, unsigned long ip, int remove,
3317+ int reset, int enable)
3318+{
3319+ return ftrace_set_hash(ops, 0, 0, ip, remove, reset, enable);
3320+}
3321+
3322+/**
3323+ * ftrace_set_filter_ip - set a function to filter on in ftrace by address
3324+ * @ops - the ops to set the filter with
3325+ * @ip - the address to add to or remove from the filter.
3326+ * @remove - non zero to remove the ip from the filter
3327+ * @reset - non zero to reset all filters before applying this filter.
3328+ *
3329+ * Filters denote which functions should be enabled when tracing is enabled
3330+ * If @ip is NULL, it failes to update filter.
3331+ */
3332+int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip,
3333+ int remove, int reset)
3334+{
3335+ return ftrace_set_addr(ops, ip, remove, reset, 1);
3336+}
3337+EXPORT_SYMBOL_GPL(ftrace_set_filter_ip);
3338+
3339+static int
3340+ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
3341+ int reset, int enable)
3342+{
3343+ return ftrace_set_hash(ops, buf, len, 0, 0, reset, enable);
3344+}
3345+
32113346 /**
32123347 * ftrace_set_filter - set a function to filter on in ftrace
32133348 * @ops - the ops to set the filter with
@@ -3912,6 +4047,7 @@ void __init ftrace_init(void) · 698 unmodified lines
39124047
39134048 static struct ftrace_ops global_ops = {
39144049 .func = ftrace_stub,
4050+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
39154051 };
39164052
39174053 static int __init ftrace_nodyn_init(void)
@@ -3942,10 +4078,9 @@ ftrace_ops_test(struct ftrace_ops *ops, unsigned long ip) · 24 unmodified lines
39424078 #endif /* CONFIG_DYNAMIC_FTRACE */
39434079
39444080 static void
3945−ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip)
4081+ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip,
4082+ struct ftrace_ops *op, struct pt_regs *regs)
39464083 {
3947− struct ftrace_ops *op;
3948−
39494084 if (unlikely(trace_recursion_test(TRACE_CONTROL_BIT)))
39504085 return;
39514086
@@ -3959,7 +4094,7 @@ ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip) · 7 unmodified lines
39594094 while (op != &ftrace_list_end) {
39604095 if (!ftrace_function_local_disabled(op) &&
39614096 ftrace_ops_test(op, ip))
3962− op->func(ip, parent_ip);
4097+ op->func(ip, parent_ip, op, regs);
39634098
39644099 op = rcu_dereference_raw(op->next);
39654100 };
@@ -3969,13 +4104,18 @@ ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip) · 3 unmodified lines
39694104
39704105 static struct ftrace_ops control_ops = {
39714106 .func = ftrace_ops_control_func,
4107+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
39724108 };
39734109
3974−static void
3975−ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip)
4110+static inline void
4111+__ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
4112+ struct ftrace_ops *ignored, struct pt_regs *regs)
39764113 {
39774114 struct ftrace_ops *op;
39784115
4116+ if (function_trace_stop)
4117+ return;
4118+
39794119 if (unlikely(trace_recursion_test(TRACE_INTERNAL_BIT)))
39804120 return;
39814121
@@ -3988,13 +4128,39 @@ ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip) · 6 unmodified lines
39884128 op = rcu_dereference_raw(ftrace_ops_list);
39894129 while (op != &ftrace_list_end) {
39904130 if (ftrace_ops_test(op, ip))
3991− op->func(ip, parent_ip);
4131+ op->func(ip, parent_ip, op, regs);
39924132 op = rcu_dereference_raw(op->next);
39934133 };
39944134 preempt_enable_notrace();
39954135 trace_recursion_clear(TRACE_INTERNAL_BIT);
39964136 }
39974137
4138+/*
4139+ * Some archs only support passing ip and parent_ip. Even though
4140+ * the list function ignores the op parameter, we do not want any
4141+ * C side effects, where a function is called without the caller
4142+ * sending a third parameter.
4143+ * Archs are to support both the regs and ftrace_ops at the same time.
4144+ * If they support ftrace_ops, it is assumed they support regs.
4145+ * If call backs want to use regs, they must either check for regs
4146+ * being NULL, or ARCH_SUPPORTS_FTRACE_SAVE_REGS.
4147+ * Note, ARCH_SUPPORT_SAVE_REGS expects a full regs to be saved.
4148+ * An architecture can pass partial regs with ftrace_ops and still
4149+ * set the ARCH_SUPPORT_FTARCE_OPS.
4150+ */
4151+#if ARCH_SUPPORTS_FTRACE_OPS
4152+static void ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
4153+ struct ftrace_ops *op, struct pt_regs *regs)
4154+{
4155+ __ftrace_ops_list_func(ip, parent_ip, NULL, regs);
4156+}
4157+#else
4158+static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip)
4159+{
4160+ __ftrace_ops_list_func(ip, parent_ip, NULL, NULL);
4161+}
4162+#endif
4163+
39984164 static void clear_ftrace_swapper(void)
39994165 {
40004166 struct task_struct *p;
kernel/trace/ring_buffer.c+2−2
@@ -2816,7 +2816,7 @@ EXPORT_SYMBOL_GPL(ring_buffer_record_enable); · 2815 unmodified lines
28162816 * to the buffer after this will fail and return NULL.
28172817 *
28182818 * This is different than ring_buffer_record_disable() as
2819− * it works like an on/off switch, where as the disable() verison
2819+ * it works like an on/off switch, where as the disable() version
28202820 * must be paired with a enable().
28212821 */
28222822 void ring_buffer_record_off(struct ring_buffer *buffer)
@@ -2839,7 +2839,7 @@ EXPORT_SYMBOL_GPL(ring_buffer_record_off); · 16 unmodified lines
28392839 * ring_buffer_record_off().
28402840 *
28412841 * This is different than ring_buffer_record_enable() as
2842− * it works like an on/off switch, where as the enable() verison
2842+ * it works like an on/off switch, where as the enable() version
28432843 * must be paired with a disable().
28442844 */
28452845 void ring_buffer_record_on(struct ring_buffer *buffer)
kernel/trace/trace.c+8−4
@@ -328,7 +328,7 @@ static DECLARE_WAIT_QUEUE_HEAD(trace_wait); · 327 unmodified lines
328328 unsigned long trace_flags = TRACE_ITER_PRINT_PARENT | TRACE_ITER_PRINTK |
329329 TRACE_ITER_ANNOTATE | TRACE_ITER_CONTEXT_INFO | TRACE_ITER_SLEEP_TIME |
330330 TRACE_ITER_GRAPH_TIME | TRACE_ITER_RECORD_CMD | TRACE_ITER_OVERWRITE |
331− TRACE_ITER_IRQ_INFO;
331+ TRACE_ITER_IRQ_INFO | TRACE_ITER_MARKERS;
332332
333333 static int trace_stop_count;
334334 static DEFINE_RAW_SPINLOCK(tracing_start_lock);
@@ -426,15 +426,15 @@ __setup("trace_buf_size=", set_buf_size); · 91 unmodified lines
426426
427427 static int __init set_tracing_thresh(char *str)
428428 {
429− unsigned long threshhold;
429+ unsigned long threshold;
430430 int ret;
431431
432432 if (!str)
433433 return 0;
434− ret = strict_strtoul(str, 0, &threshhold);
434+ ret = strict_strtoul(str, 0, &threshold);
435435 if (ret < 0)
436436 return 0;
437− tracing_thresh = threshhold * 1000;
437+ tracing_thresh = threshold * 1000;
438438 return 1;
439439 }
440440 __setup("tracing_thresh=", set_tracing_thresh);
@@ -470,6 +470,7 @@ static const char *trace_options[] = { · 29 unmodified lines
470470 "overwrite",
471471 "disable_on_free",
472472 "irq-info",
473+ "markers",
473474 NULL
474475 };
475476
@@ -3886,6 +3887,9 @@ tracing_mark_write(struct file *filp, const char __user *ubuf, · 3410 unmodified lines
38863887 if (tracing_disabled)
38873888 return -EINVAL;
38883889
3890+ if (!(trace_flags & TRACE_ITER_MARKERS))
3891+ return -EINVAL;
3892+
38893893 if (cnt > TRACE_BUF_SIZE)
38903894 cnt = TRACE_BUF_SIZE;
38913895
kernel/trace/trace.h+2−1
@@ -472,11 +472,11 @@ extern void trace_find_cmdline(int pid, char comm[]); · 471 unmodified lines
472472
473473 #ifdef CONFIG_DYNAMIC_FTRACE
474474 extern unsigned long ftrace_update_tot_cnt;
475+#endif
475476 #define DYN_FTRACE_TEST_NAME trace_selftest_dynamic_test_func
476477 extern int DYN_FTRACE_TEST_NAME(void);
477478 #define DYN_FTRACE_TEST_NAME2 trace_selftest_dynamic_test_func2
478479 extern int DYN_FTRACE_TEST_NAME2(void);
479−#endif
480480
481481 extern int ring_buffer_expanded;
482482 extern bool tracing_selftest_disabled;
@@ -680,6 +680,7 @@ enum trace_iterator_flags { · 197 unmodified lines
680680 TRACE_ITER_OVERWRITE = 0x200000,
681681 TRACE_ITER_STOP_ON_FREE = 0x400000,
682682 TRACE_ITER_IRQ_INFO = 0x800000,
683+ TRACE_ITER_MARKERS = 0x1000000,
683684 };
684685
685686 /*
kernel/trace/trace_event_perf.c+2−1
@@ -258,7 +258,8 @@ EXPORT_SYMBOL_GPL(perf_trace_buf_prepare); · 257 unmodified lines
258258
259259 #ifdef CONFIG_FUNCTION_TRACER
260260 static void
261−perf_ftrace_function_call(unsigned long ip, unsigned long parent_ip)
261+perf_ftrace_function_call(unsigned long ip, unsigned long parent_ip,
262+ struct ftrace_ops *ops, struct pt_regs *pt_regs)
262263 {
263264 struct ftrace_entry *entry;
264265 struct hlist_head *head;
kernel/trace/trace_events.c+78−38
@@ -1199,6 +1199,31 @@ event_create_dir(struct ftrace_event_call *call, struct dentry *d_events, · 1198 unmodified lines
11991199 return 0;
12001200 }
12011201
1202+static void event_remove(struct ftrace_event_call *call)
1203+{
1204+ ftrace_event_enable_disable(call, 0);
1205+ if (call->event.funcs)
1206+ __unregister_ftrace_event(&call->event);
1207+ list_del(&call->list);
1208+}
1209+
1210+static int event_init(struct ftrace_event_call *call)
1211+{
1212+ int ret = 0;
1213+
1214+ if (WARN_ON(!call->name))
1215+ return -EINVAL;
1216+
1217+ if (call->class->raw_init) {
1218+ ret = call->class->raw_init(call);
1219+ if (ret < 0 && ret != -ENOSYS)
1220+ pr_warn("Could not initialize trace events/%s\n",
1221+ call->name);
1222+ }
1223+
1224+ return ret;
1225+}
1226+
12021227 static int
12031228 __trace_add_event_call(struct ftrace_event_call *call, struct module *mod,
12041229 const struct file_operations *id,
@@ -1209,19 +1234,9 @@ __trace_add_event_call(struct ftrace_event_call *call, struct module *mod, · 4 unmodified lines
12091234 struct dentry *d_events;
12101235 int ret;
12111236
1212− /* The linker may leave blanks */
1213− if (!call->name)
1214− return -EINVAL;
1215−
1216− if (call->class->raw_init) {
1217− ret = call->class->raw_init(call);
1218− if (ret < 0) {
1219− if (ret != -ENOSYS)
1220− pr_warning("Could not initialize trace events/%s\n",
1221− call->name);
1222− return ret;
1223− }
1224− }
1237+ ret = event_init(call);
1238+ if (ret < 0)
1239+ return ret;
12251240
12261241 d_events = event_trace_events_dir();
12271242 if (!d_events)
@@ -1272,13 +1287,10 @@ static void remove_subsystem_dir(const char *name) · 44 unmodified lines
12721287 */
12731288 static void __trace_remove_event_call(struct ftrace_event_call *call)
12741289 {
1275− ftrace_event_enable_disable(call, 0);
1276− if (call->event.funcs)
1277− __unregister_ftrace_event(&call->event);
1278− debugfs_remove_recursive(call->dir);
1279− list_del(&call->list);
1290+ event_remove(call);
12801291 trace_destroy_fields(call);
12811292 destroy_preds(call);
1293+ debugfs_remove_recursive(call->dir);
12821294 remove_subsystem_dir(call->class->system);
12831295 }
12841296
@@ -1450,15 +1462,43 @@ static __init int setup_trace_event(char *str) · 165 unmodified lines
14501462 }
14511463 __setup("trace_event=", setup_trace_event);
14521464
1465+static __init int event_trace_enable(void)
1466+{
1467+ struct ftrace_event_call **iter, *call;
1468+ char *buf = bootup_event_buf;
1469+ char *token;
1470+ int ret;
1471+
1472+ for_each_event(iter, __start_ftrace_events, __stop_ftrace_events) {
1473+
1474+ call = *iter;
1475+ ret = event_init(call);
1476+ if (!ret)
1477+ list_add(&call->list, &ftrace_events);
1478+ }
1479+
1480+ while (true) {
1481+ token = strsep(&buf, ",");
1482+
1483+ if (!token)
1484+ break;
1485+ if (!*token)
1486+ continue;
1487+
1488+ ret = ftrace_set_clr_event(token, 1);
1489+ if (ret)
1490+ pr_warn("Failed to enable trace event: %s\n", token);
1491+ }
1492+ return 0;
1493+}
1494+
14531495 static __init int event_trace_init(void)
14541496 {
1455− struct ftrace_event_call **call;
1497+ struct ftrace_event_call *call;
14561498 struct dentry *d_tracer;
14571499 struct dentry *entry;
14581500 struct dentry *d_events;
14591501 int ret;
1460− char *buf = bootup_event_buf;
1461− char *token;
14621502
14631503 d_tracer = tracing_init_dentry();
14641504 if (!d_tracer)
@@ -1497,24 +1537,19 @@ static __init int event_trace_init(void) · 32 unmodified lines
14971537 if (trace_define_common_fields())
14981538 pr_warning("tracing: Failed to allocate common fields");
14991539
1500− for_each_event(call, __start_ftrace_events, __stop_ftrace_events) {
1501− __trace_add_event_call(*call, NULL, &ftrace_event_id_fops,
1540+ /*
1541+ * Early initialization already enabled ftrace event.
1542+ * Now it's only necessary to create the event directory.
1543+ */
1544+ list_for_each_entry(call, &ftrace_events, list) {
1545+
1546+ ret = event_create_dir(call, d_events,
1547+ &ftrace_event_id_fops,
15021548 &ftrace_enable_fops,
15031549 &ftrace_event_filter_fops,
15041550 &ftrace_event_format_fops);
1505− }
1506−
1507− while (true) {
1508− token = strsep(&buf, ",");
1509−
1510− if (!token)
1511− break;
1512− if (!*token)
1513− continue;
1514−
1515− ret = ftrace_set_clr_event(token, 1);
1516− if (ret)
1517− pr_warning("Failed to enable trace event: %s\n", token);
1551+ if (ret < 0)
1552+ event_remove(call);
15181553 }
15191554
15201555 ret = register_module_notifier(&trace_module_nb);
@@ -1523,6 +1558,7 @@ static __init int event_trace_init(void) · 2 unmodified lines
15231558
15241559 return 0;
15251560 }
1561+core_initcall(event_trace_enable);
15261562 fs_initcall(event_trace_init);
15271563
15281564 #ifdef CONFIG_FTRACE_STARTUP_TEST
@@ -1646,9 +1682,11 @@ static __init void event_trace_self_tests(void) · 117 unmodified lines
16461682 event_test_stuff();
16471683
16481684 ret = __ftrace_set_clr_event(NULL, system->name, NULL, 0);
1649− if (WARN_ON_ONCE(ret))
1685+ if (WARN_ON_ONCE(ret)) {
16501686 pr_warning("error disabling system %s\n",
16511687 system->name);
1688+ continue;
1689+ }
16521690
16531691 pr_cont("OK\n");
16541692 }
@@ -1681,7 +1719,8 @@ static __init void event_trace_self_tests(void) · 26 unmodified lines
16811719 static DEFINE_PER_CPU(atomic_t, ftrace_test_event_disable);
16821720
16831721 static void
1684−function_test_events_call(unsigned long ip, unsigned long parent_ip)
1722+function_test_events_call(unsigned long ip, unsigned long parent_ip,
1723+ struct ftrace_ops *op, struct pt_regs *pt_regs)
16851724 {
16861725 struct ring_buffer_event *event;
16871726 struct ring_buffer *buffer;
@@ -1720,6 +1759,7 @@ function_test_events_call(unsigned long ip, unsigned long parent_ip) · 32 unmodified lines
17201759 static struct ftrace_ops trace_ops __initdata =
17211760 {
17221761 .func = function_test_events_call,
1762+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
17231763 };
17241764
17251765 static __init void event_trace_self_test_with_function(void)
kernel/trace/trace_events_filter.c+1−1
@@ -2002,7 +2002,7 @@ static int ftrace_function_set_regexp(struct ftrace_ops *ops, int filter, · 2001 unmodified lines
20022002 static int __ftrace_function_set_filter(int filter, char *buf, int len,
20032003 struct function_filter_data *data)
20042004 {
2005− int i, re_cnt, ret;
2005+ int i, re_cnt, ret = -EINVAL;
20062006 int *reset;
20072007 char **re;
20082008
kernel/trace/trace_functions.c+9−5
@@ -49,7 +49,8 @@ static void function_trace_start(struct trace_array *tr) · 48 unmodified lines
4949 }
5050
5151 static void
52−function_trace_call_preempt_only(unsigned long ip, unsigned long parent_ip)
52+function_trace_call_preempt_only(unsigned long ip, unsigned long parent_ip,
53+ struct ftrace_ops *op, struct pt_regs *pt_regs)
5354 {
5455 struct trace_array *tr = func_trace;
5556 struct trace_array_cpu *data;
@@ -84,7 +85,9 @@ enum { · 28 unmodified lines
8485 static struct tracer_flags func_flags;
8586
8687 static void
87−function_trace_call(unsigned long ip, unsigned long parent_ip)
88+function_trace_call(unsigned long ip, unsigned long parent_ip,
89+ struct ftrace_ops *op, struct pt_regs *pt_regs)
90+
8891 {
8992 struct trace_array *tr = func_trace;
9093 struct trace_array_cpu *data;
@@ -121,7 +124,8 @@ function_trace_call(unsigned long ip, unsigned long parent_ip) · 30 unmodified lines
121124 }
122125
123126 static void
124−function_stack_trace_call(unsigned long ip, unsigned long parent_ip)
127+function_stack_trace_call(unsigned long ip, unsigned long parent_ip,
128+ struct ftrace_ops *op, struct pt_regs *pt_regs)
125129 {
126130 struct trace_array *tr = func_trace;
127131 struct trace_array_cpu *data;
@@ -164,13 +168,13 @@ function_stack_trace_call(unsigned long ip, unsigned long parent_ip) · 36 unmodified lines
164168 static struct ftrace_ops trace_ops __read_mostly =
165169 {
166170 .func = function_trace_call,
167− .flags = FTRACE_OPS_FL_GLOBAL,
171+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
168172 };
169173
170174 static struct ftrace_ops trace_stack_ops __read_mostly =
171175 {
172176 .func = function_stack_trace_call,
173− .flags = FTRACE_OPS_FL_GLOBAL,
177+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
174178 };
175179
176180 static struct tracer_opt func_opts[] = {
kernel/trace/trace_functions_graph.c+4−1
@@ -143,7 +143,7 @@ ftrace_pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret, · 142 unmodified lines
143143 return;
144144 }
145145
146−#ifdef CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST
146+#if defined(CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST) && !defined(CC_USING_FENTRY)
147147 /*
148148 * The arch may choose to record the frame pointer used
149149 * and check it here to make sure that it is what we expect it
@@ -154,6 +154,9 @@ ftrace_pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret, · 4 unmodified lines
154154 *
155155 * Currently, x86_32 with optimize for size (-Os) makes the latest
156156 * gcc do the above.
157+ *
158+ * Note, -mfentry does not use frame pointers, and this test
159+ * is not needed if CC_USING_FENTRY is set.
157160 */
158161 if (unlikely(current->ret_stack[index].fp != frame_pointer)) {
159162 ftrace_graph_stop();
kernel/trace/trace_irqsoff.c+3−2
@@ -136,7 +136,8 @@ static int func_prolog_dec(struct trace_array *tr, · 135 unmodified lines
136136 * irqsoff uses its own tracer function to keep the overhead down:
137137 */
138138 static void
139−irqsoff_tracer_call(unsigned long ip, unsigned long parent_ip)
139+irqsoff_tracer_call(unsigned long ip, unsigned long parent_ip,
140+ struct ftrace_ops *op, struct pt_regs *pt_regs)
140141 {
141142 struct trace_array *tr = irqsoff_trace;
142143 struct trace_array_cpu *data;
@@ -153,7 +154,7 @@ irqsoff_tracer_call(unsigned long ip, unsigned long parent_ip) · 10 unmodified lines
153154 static struct ftrace_ops trace_ops __read_mostly =
154155 {
155156 .func = irqsoff_tracer_call,
156− .flags = FTRACE_OPS_FL_GLOBAL,
157+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
157158 };
158159 #endif /* CONFIG_FUNCTION_TRACER */
159160
kernel/trace/trace_sched_wakeup.c+3−2
@@ -108,7 +108,8 @@ out_enable: · 107 unmodified lines
108108 * wakeup uses its own tracer function to keep the overhead down:
109109 */
110110 static void
111−wakeup_tracer_call(unsigned long ip, unsigned long parent_ip)
111+wakeup_tracer_call(unsigned long ip, unsigned long parent_ip,
112+ struct ftrace_ops *op, struct pt_regs *pt_regs)
112113 {
113114 struct trace_array *tr = wakeup_trace;
114115 struct trace_array_cpu *data;
@@ -129,7 +130,7 @@ wakeup_tracer_call(unsigned long ip, unsigned long parent_ip) · 14 unmodified lines
129130 static struct ftrace_ops trace_ops __read_mostly =
130131 {
131132 .func = wakeup_tracer_call,
132− .flags = FTRACE_OPS_FL_GLOBAL,
133+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
133134 };
134135 #endif /* CONFIG_FUNCTION_TRACER */
135136
kernel/trace/trace_selftest.c+283−21
@@ -103,54 +103,67 @@ static inline void warn_failed_init_tracer(struct tracer *trace, int init_ret) · 102 unmodified lines
103103
104104 static int trace_selftest_test_probe1_cnt;
105105 static void trace_selftest_test_probe1_func(unsigned long ip,
106− unsigned long pip)
106+ unsigned long pip,
107+ struct ftrace_ops *op,
108+ struct pt_regs *pt_regs)
107109 {
108110 trace_selftest_test_probe1_cnt++;
109111 }
110112
111113 static int trace_selftest_test_probe2_cnt;
112114 static void trace_selftest_test_probe2_func(unsigned long ip,
113− unsigned long pip)
115+ unsigned long pip,
116+ struct ftrace_ops *op,
117+ struct pt_regs *pt_regs)
114118 {
115119 trace_selftest_test_probe2_cnt++;
116120 }
117121
118122 static int trace_selftest_test_probe3_cnt;
119123 static void trace_selftest_test_probe3_func(unsigned long ip,
120− unsigned long pip)
124+ unsigned long pip,
125+ struct ftrace_ops *op,
126+ struct pt_regs *pt_regs)
121127 {
122128 trace_selftest_test_probe3_cnt++;
123129 }
124130
125131 static int trace_selftest_test_global_cnt;
126132 static void trace_selftest_test_global_func(unsigned long ip,
127− unsigned long pip)
133+ unsigned long pip,
134+ struct ftrace_ops *op,
135+ struct pt_regs *pt_regs)
128136 {
129137 trace_selftest_test_global_cnt++;
130138 }
131139
132140 static int trace_selftest_test_dyn_cnt;
133141 static void trace_selftest_test_dyn_func(unsigned long ip,
134− unsigned long pip)
142+ unsigned long pip,
143+ struct ftrace_ops *op,
144+ struct pt_regs *pt_regs)
135145 {
136146 trace_selftest_test_dyn_cnt++;
137147 }
138148
139149 static struct ftrace_ops test_probe1 = {
140150 .func = trace_selftest_test_probe1_func,
151+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
141152 };
142153
143154 static struct ftrace_ops test_probe2 = {
144155 .func = trace_selftest_test_probe2_func,
156+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
145157 };
146158
147159 static struct ftrace_ops test_probe3 = {
148160 .func = trace_selftest_test_probe3_func,
161+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
149162 };
150163
151164 static struct ftrace_ops test_global = {
152− .func = trace_selftest_test_global_func,
153− .flags = FTRACE_OPS_FL_GLOBAL,
165+ .func = trace_selftest_test_global_func,
166+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
154167 };
155168
156169 static void print_counts(void)
@@ -393,10 +406,253 @@ int trace_selftest_startup_dynamic_tracing(struct tracer *trace, · 236 unmodified lines
393406
394407 return ret;
395408 }
409+
410+static int trace_selftest_recursion_cnt;
411+static void trace_selftest_test_recursion_func(unsigned long ip,
412+ unsigned long pip,
413+ struct ftrace_ops *op,
414+ struct pt_regs *pt_regs)
415+{
416+ /*
417+ * This function is registered without the recursion safe flag.
418+ * The ftrace infrastructure should provide the recursion
419+ * protection. If not, this will crash the kernel!
420+ */
421+ trace_selftest_recursion_cnt++;
422+ DYN_FTRACE_TEST_NAME();
423+}
424+
425+static void trace_selftest_test_recursion_safe_func(unsigned long ip,
426+ unsigned long pip,
427+ struct ftrace_ops *op,
428+ struct pt_regs *pt_regs)
429+{
430+ /*
431+ * We said we would provide our own recursion. By calling
432+ * this function again, we should recurse back into this function
433+ * and count again. But this only happens if the arch supports
434+ * all of ftrace features and nothing else is using the function
435+ * tracing utility.
436+ */
437+ if (trace_selftest_recursion_cnt++)
438+ return;
439+ DYN_FTRACE_TEST_NAME();
440+}
441+
442+static struct ftrace_ops test_rec_probe = {
443+ .func = trace_selftest_test_recursion_func,
444+};
445+
446+static struct ftrace_ops test_recsafe_probe = {
447+ .func = trace_selftest_test_recursion_safe_func,
448+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
449+};
450+
451+static int
452+trace_selftest_function_recursion(void)
453+{
454+ int save_ftrace_enabled = ftrace_enabled;
455+ int save_tracer_enabled = tracer_enabled;
456+ char *func_name;
457+ int len;
458+ int ret;
459+ int cnt;
460+
461+ /* The previous test PASSED */
462+ pr_cont("PASSED\n");
463+ pr_info("Testing ftrace recursion: ");
464+
465+
466+ /* enable tracing, and record the filter function */
467+ ftrace_enabled = 1;
468+ tracer_enabled = 1;
469+
470+ /* Handle PPC64 '.' name */
471+ func_name = "*" __stringify(DYN_FTRACE_TEST_NAME);
472+ len = strlen(func_name);
473+
474+ ret = ftrace_set_filter(&test_rec_probe, func_name, len, 1);
475+ if (ret) {
476+ pr_cont("*Could not set filter* ");
477+ goto out;
478+ }
479+
480+ ret = register_ftrace_function(&test_rec_probe);
481+ if (ret) {
482+ pr_cont("*could not register callback* ");
483+ goto out;
484+ }
485+
486+ DYN_FTRACE_TEST_NAME();
487+
488+ unregister_ftrace_function(&test_rec_probe);
489+
490+ ret = -1;
491+ if (trace_selftest_recursion_cnt != 1) {
492+ pr_cont("*callback not called once (%d)* ",
493+ trace_selftest_recursion_cnt);
494+ goto out;
495+ }
496+
497+ trace_selftest_recursion_cnt = 1;
498+
499+ pr_cont("PASSED\n");
500+ pr_info("Testing ftrace recursion safe: ");
501+
502+ ret = ftrace_set_filter(&test_recsafe_probe, func_name, len, 1);
503+ if (ret) {
504+ pr_cont("*Could not set filter* ");
505+ goto out;
506+ }
507+
508+ ret = register_ftrace_function(&test_recsafe_probe);
509+ if (ret) {
510+ pr_cont("*could not register callback* ");
511+ goto out;
512+ }
513+
514+ DYN_FTRACE_TEST_NAME();
515+
516+ unregister_ftrace_function(&test_recsafe_probe);
517+
518+ /*
519+ * If arch supports all ftrace features, and no other task
520+ * was on the list, we should be fine.
521+ */
522+ if (!ftrace_nr_registered_ops() && !FTRACE_FORCE_LIST_FUNC)
523+ cnt = 2; /* Should have recursed */
524+ else
525+ cnt = 1;
526+
527+ ret = -1;
528+ if (trace_selftest_recursion_cnt != cnt) {
529+ pr_cont("*callback not called expected %d times (%d)* ",
530+ cnt, trace_selftest_recursion_cnt);
531+ goto out;
532+ }
533+
534+ ret = 0;
535+out:
536+ ftrace_enabled = save_ftrace_enabled;
537+ tracer_enabled = save_tracer_enabled;
538+
539+ return ret;
540+}
396541 #else
397542 # define trace_selftest_startup_dynamic_tracing(trace, tr, func) ({ 0; })
543+# define trace_selftest_function_recursion() ({ 0; })
398544 #endif /* CONFIG_DYNAMIC_FTRACE */
399545
546+static enum {
547+ TRACE_SELFTEST_REGS_START,
548+ TRACE_SELFTEST_REGS_FOUND,
549+ TRACE_SELFTEST_REGS_NOT_FOUND,
550+} trace_selftest_regs_stat;
551+
552+static void trace_selftest_test_regs_func(unsigned long ip,
553+ unsigned long pip,
554+ struct ftrace_ops *op,
555+ struct pt_regs *pt_regs)
556+{
557+ if (pt_regs)
558+ trace_selftest_regs_stat = TRACE_SELFTEST_REGS_FOUND;
559+ else
560+ trace_selftest_regs_stat = TRACE_SELFTEST_REGS_NOT_FOUND;
561+}
562+
563+static struct ftrace_ops test_regs_probe = {
564+ .func = trace_selftest_test_regs_func,
565+ .flags = FTRACE_OPS_FL_RECURSION_SAFE | FTRACE_OPS_FL_SAVE_REGS,
566+};
567+
568+static int
569+trace_selftest_function_regs(void)
570+{
571+ int save_ftrace_enabled = ftrace_enabled;
572+ int save_tracer_enabled = tracer_enabled;
573+ char *func_name;
574+ int len;
575+ int ret;
576+ int supported = 0;
577+
578+#ifdef ARCH_SUPPORTS_FTRACE_SAVE_REGS
579+ supported = 1;
580+#endif
581+
582+ /* The previous test PASSED */
583+ pr_cont("PASSED\n");
584+ pr_info("Testing ftrace regs%s: ",
585+ !supported ? "(no arch support)" : "");
586+
587+ /* enable tracing, and record the filter function */
588+ ftrace_enabled = 1;
589+ tracer_enabled = 1;
590+
591+ /* Handle PPC64 '.' name */
592+ func_name = "*" __stringify(DYN_FTRACE_TEST_NAME);
593+ len = strlen(func_name);
594+
595+ ret = ftrace_set_filter(&test_regs_probe, func_name, len, 1);
596+ /*
597+ * If DYNAMIC_FTRACE is not set, then we just trace all functions.
598+ * This test really doesn't care.
599+ */
600+ if (ret && ret != -ENODEV) {
601+ pr_cont("*Could not set filter* ");
602+ goto out;
603+ }
604+
605+ ret = register_ftrace_function(&test_regs_probe);
606+ /*
607+ * Now if the arch does not support passing regs, then this should
608+ * have failed.
609+ */
610+ if (!supported) {
611+ if (!ret) {
612+ pr_cont("*registered save-regs without arch support* ");
613+ goto out;
614+ }
615+ test_regs_probe.flags |= FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED;
616+ ret = register_ftrace_function(&test_regs_probe);
617+ }
618+ if (ret) {
619+ pr_cont("*could not register callback* ");
620+ goto out;
621+ }
622+
623+
624+ DYN_FTRACE_TEST_NAME();
625+
626+ unregister_ftrace_function(&test_regs_probe);
627+
628+ ret = -1;
629+
630+ switch (trace_selftest_regs_stat) {
631+ case TRACE_SELFTEST_REGS_START:
632+ pr_cont("*callback never called* ");
633+ goto out;
634+
635+ case TRACE_SELFTEST_REGS_FOUND:
636+ if (supported)
637+ break;
638+ pr_cont("*callback received regs without arch support* ");
639+ goto out;
640+
641+ case TRACE_SELFTEST_REGS_NOT_FOUND:
642+ if (!supported)
643+ break;
644+ pr_cont("*callback received NULL regs* ");
645+ goto out;
646+ }
647+
648+ ret = 0;
649+out:
650+ ftrace_enabled = save_ftrace_enabled;
651+ tracer_enabled = save_tracer_enabled;
652+
653+ return ret;
654+}
655+
400656 /*
401657 * Simple verification test of ftrace function tracer.
402658 * Enable ftrace, sleep 1/10 second, and then read the trace
@@ -442,7 +698,14 @@ trace_selftest_startup_function(struct tracer *trace, struct trace_array *tr) · 39 unmodified lines
442698
443699 ret = trace_selftest_startup_dynamic_tracing(trace, tr,
444700 DYN_FTRACE_TEST_NAME);
701+ if (ret)
702+ goto out;
445703
704+ ret = trace_selftest_function_recursion();
705+ if (ret)
706+ goto out;
707+
708+ ret = trace_selftest_function_regs();
446709 out:
447710 ftrace_enabled = save_ftrace_enabled;
448711 tracer_enabled = save_tracer_enabled;
@@ -778,6 +1041,8 @@ static int trace_wakeup_test_thread(void *data) · 329 unmodified lines
7781041 set_current_state(TASK_INTERRUPTIBLE);
7791042 schedule();
7801043
1044+ complete(x);
1045+
7811046 /* we are awake, now wait to disappear */
7821047 while (!kthread_should_stop()) {
7831048 /*
@@ -821,24 +1086,21 @@ trace_selftest_startup_wakeup(struct tracer *trace, struct trace_array *tr) · 37 unmodified lines
8211086 /* reset the max latency */
8221087 tracing_max_latency = 0;
8231088
824− /* sleep to let the RT thread sleep too */
825− msleep(100);
1089+ while (p->on_rq) {
1090+ /*
1091+ * Sleep to make sure the RT thread is asleep too.
1092+ * On virtual machines we can't rely on timings,
1093+ * but we want to make sure this test still works.
1094+ */
1095+ msleep(100);
1096+ }
8261097
827− /*
828− * Yes this is slightly racy. It is possible that for some
829− * strange reason that the RT thread we created, did not
830− * call schedule for 100ms after doing the completion,
831− * and we do a wakeup on a task that already is awake.
832− * But that is extremely unlikely, and the worst thing that
833− * happens in such a case, is that we disable tracing.
834− * Honestly, if this race does happen something is horrible
835− * wrong with the system.
836− */
1098+ init_completion(&isrt);
8371099
8381100 wake_up_process(p);
8391101
840− /* give a little time to let the thread wake up */
841− msleep(100);
1102+ /* Wait for the task to wake up */
1103+ wait_for_completion(&isrt);
8421104
8431105 /* stop the tracing. */
8441106 tracing_stop();
kernel/trace/trace_stack.c+3−1
@@ -111,7 +111,8 @@ static inline void check_stack(void) · 110 unmodified lines
111111 }
112112
113113 static void
114−stack_trace_call(unsigned long ip, unsigned long parent_ip)
114+stack_trace_call(unsigned long ip, unsigned long parent_ip,
115+ struct ftrace_ops *op, struct pt_regs *pt_regs)
115116 {
116117 int cpu;
117118
@@ -136,6 +137,7 @@ stack_trace_call(unsigned long ip, unsigned long parent_ip) · 18 unmodified lines
136137 static struct ftrace_ops trace_ops __read_mostly =
137138 {
138139 .func = stack_trace_call,
140+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
139141 };
140142
141143 static ssize_t
kernel/trace/trace_syscalls.c+1−1
@@ -487,7 +487,7 @@ int __init init_ftrace_syscalls(void) · 486 unmodified lines
487487
488488 return 0;
489489 }
490−core_initcall(init_ftrace_syscalls);
490+early_initcall(init_ftrace_syscalls);
491491
492492 #ifdef CONFIG_PERF_EVENTS
493493
scripts/recordmcount.h+3−1
@@ -261,11 +261,13 @@ static unsigned get_mcountsym(Elf_Sym const *const sym0, · 260 unmodified lines
261261 &sym0[Elf_r_sym(relp)];
262262 char const *symname = &str0[w(symp->st_name)];
263263 char const *mcount = gpfx == '_' ? "_mcount" : "mcount";
264+ char const *fentry = "__fentry__";
264265
265266 if (symname[0] == '.')
266267 ++symname; /* ppc64 hack */
267268 if (strcmp(mcount, symname) == 0 ||
268− (altmcount && strcmp(altmcount, symname) == 0))
269+ (altmcount && strcmp(altmcount, symname) == 0) ||
270+ (strcmp(fentry, symname) == 0))
269271 mcountsym = Elf_r_sym(relp);
270272
271273 return mcountsym;
tools/lib/traceevent/Makefile+1−1
@@ -129,7 +129,7 @@ CFLAGS ?= -g -Wall · 128 unmodified lines
129129
130130 # Append required CFLAGS
131131 override CFLAGS += $(CONFIG_FLAGS) $(INCLUDES) $(PLUGIN_DIR_SQ)
132−override CFLAGS += $(udis86-flags)
132+override CFLAGS += $(udis86-flags) -D_GNU_SOURCE
133133
134134 ifeq ($(VERBOSE),1)
135135 Q =
tools/lib/traceevent/event-parse.c+548−206
@@ -24,13 +24,14 @@ · 23 unmodified lines
2424 * Frederic Weisbecker gave his permission to relicense the code to
2525 * the Lesser General Public License.
2626 */
27−#define _GNU_SOURCE
2827 #include <stdio.h>
2928 #include <stdlib.h>
3029 #include <string.h>
3130 #include <stdarg.h>
3231 #include <ctype.h>
3332 #include <errno.h>
33+#include <stdint.h>
34+#include <limits.h>
3435
3536 #include "event-parse.h"
3637 #include "event-utils.h"
@@ -117,14 +118,7 @@ void breakpoint(void) · 80 unmodified lines
117118
118119 struct print_arg *alloc_arg(void)
119120 {
120− struct print_arg *arg;
121−
122− arg = malloc_or_die(sizeof(*arg));
123− if (!arg)
124− return NULL;
125− memset(arg, 0, sizeof(*arg));
126−
127− return arg;
121+ return calloc(1, sizeof(struct print_arg));
128122 }
129123
130124 struct cmdline {
@@ -158,7 +152,9 @@ static int cmdline_init(struct pevent *pevent) · 27 unmodified lines
158152 struct cmdline *cmdlines;
159153 int i;
160154
161− cmdlines = malloc_or_die(sizeof(*cmdlines) * pevent->cmdline_count);
155+ cmdlines = malloc(sizeof(*cmdlines) * pevent->cmdline_count);
156+ if (!cmdlines)
157+ return -1;
162158
163159 i = 0;
164160 while (cmdlist) {
@@ -186,8 +182,8 @@ static char *find_cmdline(struct pevent *pevent, int pid) · 21 unmodified lines
186182 if (!pid)
187183 return "<idle>";
188184
189− if (!pevent->cmdlines)
190− cmdline_init(pevent);
185+ if (!pevent->cmdlines && cmdline_init(pevent))
186+ return "<not enough memory for cmdlines!>";
191187
192188 key.pid = pid;
193189
@@ -215,8 +211,8 @@ int pevent_pid_is_registered(struct pevent *pevent, int pid) · 21 unmodified lines
215211 if (!pid)
216212 return 1;
217213
218− if (!pevent->cmdlines)
219− cmdline_init(pevent);
214+ if (!pevent->cmdlines && cmdline_init(pevent))
215+ return 0;
220216
221217 key.pid = pid;
222218
@@ -258,10 +254,14 @@ static int add_new_comm(struct pevent *pevent, const char *comm, int pid) · 35 unmodified lines
258254 return -1;
259255 }
260256
261− cmdlines[pevent->cmdline_count].pid = pid;
262257 cmdlines[pevent->cmdline_count].comm = strdup(comm);
263− if (!cmdlines[pevent->cmdline_count].comm)
264− die("malloc comm");
258+ if (!cmdlines[pevent->cmdline_count].comm) {
259+ free(cmdlines);
260+ errno = ENOMEM;
261+ return -1;
262+ }
263+
264+ cmdlines[pevent->cmdline_count].pid = pid;
265265
266266 if (cmdlines[pevent->cmdline_count].comm)
267267 pevent->cmdline_count++;
@@ -288,10 +288,15 @@ int pevent_register_comm(struct pevent *pevent, const char *comm, int pid) · 20 unmodified lines
288288 if (pevent->cmdlines)
289289 return add_new_comm(pevent, comm, pid);
290290
291− item = malloc_or_die(sizeof(*item));
291+ item = malloc(sizeof(*item));
292+ if (!item)
293+ return -1;
294+
292295 item->comm = strdup(comm);
293− if (!item->comm)
294− die("malloc comm");
296+ if (!item->comm) {
297+ free(item);
298+ return -1;
299+ }
295300 item->pid = pid;
296301 item->next = pevent->cmdlist;
297302
@@ -355,7 +360,10 @@ static int func_map_init(struct pevent *pevent) · 57 unmodified lines
355360 struct func_map *func_map;
356361 int i;
357362
358− func_map = malloc_or_die(sizeof(*func_map) * (pevent->func_count + 1));
363+ func_map = malloc(sizeof(*func_map) * (pevent->func_count + 1));
364+ if (!func_map)
365+ return -1;
366+
359367 funclist = pevent->funclist;
360368
361369 i = 0;
@@ -455,25 +463,36 @@ pevent_find_function_address(struct pevent *pevent, unsigned long long addr) · 93 unmodified lines
455463 int pevent_register_function(struct pevent *pevent, char *func,
456464 unsigned long long addr, char *mod)
457465 {
458− struct func_list *item;
466+ struct func_list *item = malloc(sizeof(*item));
459467
460− item = malloc_or_die(sizeof(*item));
468+ if (!item)
469+ return -1;
461470
462471 item->next = pevent->funclist;
463472 item->func = strdup(func);
464− if (mod)
473+ if (!item->func)
474+ goto out_free;
475+
476+ if (mod) {
465477 item->mod = strdup(mod);
466− else
478+ if (!item->mod)
479+ goto out_free_func;
480+ } else
467481 item->mod = NULL;
468482 item->addr = addr;
469483
470− if (!item->func || (mod && !item->mod))
471− die("malloc func");
472−
473484 pevent->funclist = item;
474485 pevent->func_count++;
475486
476487 return 0;
488+
489+out_free_func:
490+ free(item->func);
491+ item->func = NULL;
492+out_free:
493+ free(item);
494+ errno = ENOMEM;
495+ return -1;
477496 }
478497
479498 /**
@@ -524,14 +543,16 @@ static int printk_cmp(const void *a, const void *b) · 44 unmodified lines
524543 return 0;
525544 }
526545
527−static void printk_map_init(struct pevent *pevent)
546+static int printk_map_init(struct pevent *pevent)
528547 {
529548 struct printk_list *printklist;
530549 struct printk_list *item;
531550 struct printk_map *printk_map;
532551 int i;
533552
534− printk_map = malloc_or_die(sizeof(*printk_map) * (pevent->printk_count + 1));
553+ printk_map = malloc(sizeof(*printk_map) * (pevent->printk_count + 1));
554+ if (!printk_map)
555+ return -1;
535556
536557 printklist = pevent->printklist;
537558
@@ -549,6 +570,8 @@ static void printk_map_init(struct pevent *pevent) · 11 unmodified lines
549570
550571 pevent->printk_map = printk_map;
551572 pevent->printklist = NULL;
573+
574+ return 0;
552575 }
553576
554577 static struct printk_map *
@@ -557,8 +580,8 @@ find_printk(struct pevent *pevent, unsigned long long addr) · 2 unmodified lines
557580 struct printk_map *printk;
558581 struct printk_map key;
559582
560− if (!pevent->printk_map)
561− printk_map_init(pevent);
583+ if (!pevent->printk_map && printk_map_init(pevent))
584+ return NULL;
562585
563586 key.addr = addr;
564587
@@ -580,21 +603,27 @@ find_printk(struct pevent *pevent, unsigned long long addr) · 15 unmodified lines
580603 int pevent_register_print_string(struct pevent *pevent, char *fmt,
581604 unsigned long long addr)
582605 {
583− struct printk_list *item;
606+ struct printk_list *item = malloc(sizeof(*item));
584607
585− item = malloc_or_die(sizeof(*item));
608+ if (!item)
609+ return -1;
586610
587611 item->next = pevent->printklist;
588− item->printk = strdup(fmt);
589612 item->addr = addr;
590613
614+ item->printk = strdup(fmt);
591615 if (!item->printk)
592− die("malloc fmt");
616+ goto out_free;
593617
594618 pevent->printklist = item;
595619 pevent->printk_count++;
596620
597621 return 0;
622+
623+out_free:
624+ free(item);
625+ errno = ENOMEM;
626+ return -1;
598627 }
599628
600629 /**
@@ -619,24 +648,18 @@ void pevent_print_printk(struct pevent *pevent) · 18 unmodified lines
619648
620649 static struct event_format *alloc_event(void)
621650 {
622− struct event_format *event;
623−
624− event = malloc(sizeof(*event));
625− if (!event)
626− return NULL;
627− memset(event, 0, sizeof(*event));
628−
629− return event;
651+ return calloc(1, sizeof(struct event_format));
630652 }
631653
632−static void add_event(struct pevent *pevent, struct event_format *event)
654+static int add_event(struct pevent *pevent, struct event_format *event)
633655 {
634656 int i;
657+ struct event_format **events = realloc(pevent->events, sizeof(event) *
658+ (pevent->nr_events + 1));
659+ if (!events)
660+ return -1;
635661
636− pevent->events = realloc(pevent->events, sizeof(event) *
637− (pevent->nr_events + 1));
638− if (!pevent->events)
639− die("Can not allocate events");
662+ pevent->events = events;
640663
641664 for (i = 0; i < pevent->nr_events; i++) {
642665 if (pevent->events[i]->id > event->id)
@@ -651,6 +674,8 @@ static void add_event(struct pevent *pevent, struct event_format *event) · 8 unmodified lines
651674 pevent->nr_events++;
652675
653676 event->pevent = pevent;
677+
678+ return 0;
654679 }
655680
656681 static int event_item_type(enum event_type type)
@@ -827,9 +852,9 @@ static enum event_type __read_token(char **tok) · 170 unmodified lines
827852 switch (type) {
828853 case EVENT_NEWLINE:
829854 case EVENT_DELIM:
830− *tok = malloc_or_die(2);
831− (*tok)[0] = ch;
832− (*tok)[1] = 0;
855+ if (asprintf(tok, "%c", ch) < 0)
856+ return EVENT_ERROR;
857+
833858 return type;
834859
835860 case EVENT_OP:
@@ -1240,8 +1265,10 @@ static int event_read_fields(struct event_format *event, struct format_field **f · 404 unmodified lines
12401265
12411266 last_token = token;
12421267
1243− field = malloc_or_die(sizeof(*field));
1244− memset(field, 0, sizeof(*field));
1268+ field = calloc(1, sizeof(*field));
1269+ if (!field)
1270+ goto fail;
1271+
12451272 field->event = event;
12461273
12471274 /* read the rest of the type */
@@ -1282,7 +1309,7 @@ static int event_read_fields(struct event_format *event, struct format_field **f · 34 unmodified lines
12821309 }
12831310
12841311 if (!field->type) {
1285− die("no type found");
1312+ do_warning("%s: no type found", __func__);
12861313 goto fail;
12871314 }
12881315 field->name = last_token;
@@ -1329,7 +1356,7 @@ static int event_read_fields(struct event_format *event, struct format_field **f · 40 unmodified lines
13291356 free_token(token);
13301357 type = read_token(&token);
13311358 if (type == EVENT_NONE) {
1332− die("failed to find token");
1359+ do_warning("failed to find token");
13331360 goto fail;
13341361 }
13351362 }
@@ -1538,6 +1565,14 @@ process_cond(struct event_format *event, struct print_arg *top, char **tok) · 202 unmodified lines
15381565 left = alloc_arg();
15391566 right = alloc_arg();
15401567
1568+ if (!arg || !left || !right) {
1569+ do_warning("%s: not enough memory!", __func__);
1570+ /* arg will be freed at out_free */
1571+ free_arg(left);
1572+ free_arg(right);
1573+ goto out_free;
1574+ }
1575+
15411576 arg->type = PRINT_OP;
15421577 arg->op.left = left;
15431578 arg->op.right = right;
@@ -1580,6 +1615,12 @@ process_array(struct event_format *event, struct print_arg *top, char **tok) · 36 unmodified lines
15801615 char *token = NULL;
15811616
15821617 arg = alloc_arg();
1618+ if (!arg) {
1619+ do_warning("%s: not enough memory!", __func__);
1620+ /* '*tok' is set to top->op.op. No need to free. */
1621+ *tok = NULL;
1622+ return EVENT_ERROR;
1623+ }
15831624
15841625 *tok = NULL;
15851626 type = process_arg(event, arg, &token);
@@ -1595,8 +1636,7 @@ process_array(struct event_format *event, struct print_arg *top, char **tok) · 9 unmodified lines
15951636 return type;
15961637
15971638 out_free:
1598− free_token(*tok);
1599− *tok = NULL;
1639+ free_token(token);
16001640 free_arg(arg);
16011641 return EVENT_ERROR;
16021642 }
@@ -1682,7 +1722,7 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 79 unmodified lines
16821722 if (arg->type == PRINT_OP && !arg->op.left) {
16831723 /* handle single op */
16841724 if (token[1]) {
1685− die("bad op token %s", token);
1725+ do_warning("bad op token %s", token);
16861726 goto out_free;
16871727 }
16881728 switch (token[0]) {
@@ -1699,10 +1739,16 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 10 unmodified lines
16991739
17001740 /* make an empty left */
17011741 left = alloc_arg();
1742+ if (!left)
1743+ goto out_warn_free;
1744+
17021745 left->type = PRINT_NULL;
17031746 arg->op.left = left;
17041747
17051748 right = alloc_arg();
1749+ if (!right)
1750+ goto out_warn_free;
1751+
17061752 arg->op.right = right;
17071753
17081754 /* do not free the token, it belongs to an op */
@@ -1712,6 +1758,9 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 3 unmodified lines
17121758 } else if (strcmp(token, "?") == 0) {
17131759
17141760 left = alloc_arg();
1761+ if (!left)
1762+ goto out_warn_free;
1763+
17151764 /* copy the top arg to the left */
17161765 *left = *arg;
17171766
@@ -1720,6 +1769,7 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 2 unmodified lines
17201769 arg->op.left = left;
17211770 arg->op.prio = 0;
17221771
1772+ /* it will set arg->op.right */
17231773 type = process_cond(event, arg, tok);
17241774
17251775 } else if (strcmp(token, ">>") == 0 ||
@@ -1739,6 +1789,8 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 13 unmodified lines
17391789 strcmp(token, "!=") == 0) {
17401790
17411791 left = alloc_arg();
1792+ if (!left)
1793+ goto out_warn_free;
17421794
17431795 /* copy the top arg to the left */
17441796 *left = *arg;
@@ -1746,6 +1798,7 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 1 unmodified line
17461798 arg->type = PRINT_OP;
17471799 arg->op.op = token;
17481800 arg->op.left = left;
1801+ arg->op.right = NULL;
17491802
17501803 if (set_op_prio(arg) == -1) {
17511804 event->flags |= EVENT_FL_FAILED;
@@ -1762,12 +1815,14 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 10 unmodified lines
17621815 type == EVENT_DELIM && (strcmp(token, ")") == 0)) {
17631816 char *new_atom;
17641817
1765− if (left->type != PRINT_ATOM)
1766− die("bad pointer type");
1818+ if (left->type != PRINT_ATOM) {
1819+ do_warning("bad pointer type");
1820+ goto out_free;
1821+ }
17671822 new_atom = realloc(left->atom.atom,
17681823 strlen(left->atom.atom) + 3);
17691824 if (!new_atom)
1770− goto out_free;
1825+ goto out_warn_free;
17711826
17721827 left->atom.atom = new_atom;
17731828 strcat(left->atom.atom, " *");
@@ -1779,12 +1834,18 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 5 unmodified lines
17791834 }
17801835
17811836 right = alloc_arg();
1837+ if (!right)
1838+ goto out_warn_free;
1839+
17821840 type = process_arg_token(event, right, tok, type);
17831841 arg->op.right = right;
17841842
17851843 } else if (strcmp(token, "[") == 0) {
17861844
17871845 left = alloc_arg();
1846+ if (!left)
1847+ goto out_warn_free;
1848+
17881849 *left = *arg;
17891850
17901851 arg->type = PRINT_OP;
@@ -1793,6 +1854,7 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 2 unmodified lines
17931854
17941855 arg->op.prio = 0;
17951856
1857+ /* it will set arg->op.right */
17961858 type = process_array(event, arg, tok);
17971859
17981860 } else {
@@ -1816,14 +1878,16 @@ process_op(struct event_format *event, struct print_arg *arg, char **tok) · 17 unmodified lines
18161878
18171879 return type;
18181880
1819− out_free:
1881+out_warn_free:
1882+ do_warning("%s: not enough memory!", __func__);
1883+out_free:
18201884 free_token(token);
18211885 *tok = NULL;
18221886 return EVENT_ERROR;
18231887 }
18241888
18251889 static enum event_type
1826−process_entry(struct event_format *event __unused, struct print_arg *arg,
1890+process_entry(struct event_format *event __maybe_unused, struct print_arg *arg,
18271891 char **tok)
18281892 {
18291893 enum event_type type;
@@ -1880,7 +1944,11 @@ eval_type_str(unsigned long long val, const char *type, int pointer) · 50 unmodified lines
18801944 return val;
18811945 }
18821946
1883− ref = malloc_or_die(len);
1947+ ref = malloc(len);
1948+ if (!ref) {
1949+ do_warning("%s: not enough memory!", __func__);
1950+ return val;
1951+ }
18841952 memcpy(ref, type, len);
18851953
18861954 /* chop off the " *" */
@@ -1957,8 +2025,10 @@ eval_type_str(unsigned long long val, const char *type, int pointer) · 70 unmodified lines
19572025 static unsigned long long
19582026 eval_type(unsigned long long val, struct print_arg *arg, int pointer)
19592027 {
1960− if (arg->type != PRINT_TYPE)
1961− die("expected type argument");
2028+ if (arg->type != PRINT_TYPE) {
2029+ do_warning("expected type argument");
2030+ return 0;
2031+ }
19622032
19632033 return eval_type_str(val, arg->typecast.type, pointer);
19642034 }
@@ -2143,7 +2213,7 @@ static char *arg_eval (struct print_arg *arg) · 178 unmodified lines
21432213 case PRINT_STRING:
21442214 case PRINT_BSTRING:
21452215 default:
2146− die("invalid eval type %d", arg->type);
2216+ do_warning("invalid eval type %d", arg->type);
21472217 break;
21482218 }
21492219
@@ -2166,6 +2236,8 @@ process_fields(struct event_format *event, struct print_flag_sym **list, char ** · 16 unmodified lines
21662236 break;
21672237
21682238 arg = alloc_arg();
2239+ if (!arg)
2240+ goto out_free;
21692241
21702242 free_token(token);
21712243 type = process_arg(event, arg, &token);
@@ -2179,30 +2251,33 @@ process_fields(struct event_format *event, struct print_flag_sym **list, char ** · 7 unmodified lines
21792251 if (test_type_token(type, token, EVENT_DELIM, ","))
21802252 goto out_free;
21812253
2182− field = malloc_or_die(sizeof(*field));
2183− memset(field, 0, sizeof(*field));
2254+ field = calloc(1, sizeof(*field));
2255+ if (!field)
2256+ goto out_free;
21842257
21852258 value = arg_eval(arg);
21862259 if (value == NULL)
2187− goto out_free;
2260+ goto out_free_field;
21882261 field->value = strdup(value);
21892262 if (field->value == NULL)
2190− goto out_free;
2263+ goto out_free_field;
21912264
21922265 free_arg(arg);
21932266 arg = alloc_arg();
2267+ if (!arg)
2268+ goto out_free;
21942269
21952270 free_token(token);
21962271 type = process_arg(event, arg, &token);
21972272 if (test_type_token(type, token, EVENT_OP, "}"))
2198− goto out_free;
2273+ goto out_free_field;
21992274
22002275 value = arg_eval(arg);
22012276 if (value == NULL)
2202− goto out_free;
2277+ goto out_free_field;
22032278 field->str = strdup(value);
22042279 if (field->str == NULL)
2205− goto out_free;
2280+ goto out_free_field;
22062281 free_arg(arg);
22072282 arg = NULL;
22082283
@@ -2216,6 +2291,8 @@ process_fields(struct event_format *event, struct print_flag_sym **list, char ** · 7 unmodified lines
22162291 *tok = token;
22172292 return type;
22182293
2294+out_free_field:
2295+ free_flag_sym(field);
22192296 out_free:
22202297 free_arg(arg);
22212298 free_token(token);
@@ -2235,6 +2312,10 @@ process_flags(struct event_format *event, struct print_arg *arg, char **tok) · 13 unmodified lines
22352312 arg->type = PRINT_FLAGS;
22362313
22372314 field = alloc_arg();
2315+ if (!field) {
2316+ do_warning("%s: not enough memory!", __func__);
2317+ goto out_free;
2318+ }
22382319
22392320 type = process_arg(event, field, &token);
22402321
@@ -2243,7 +2324,7 @@ process_flags(struct event_format *event, struct print_arg *arg, char **tok) · 2 unmodified lines
22432324 type = process_op(event, field, &token);
22442325
22452326 if (test_type_token(type, token, EVENT_DELIM, ","))
2246− goto out_free;
2327+ goto out_free_field;
22472328 free_token(token);
22482329
22492330 arg->flags.field = field;
@@ -2265,7 +2346,9 @@ process_flags(struct event_format *event, struct print_arg *arg, char **tok) · 15 unmodified lines
22652346 type = read_token_item(tok);
22662347 return type;
22672348
2268− out_free:
2349+out_free_field:
2350+ free_arg(field);
2351+out_free:
22692352 free_token(token);
22702353 *tok = NULL;
22712354 return EVENT_ERROR;
@@ -2282,10 +2365,14 @@ process_symbols(struct event_format *event, struct print_arg *arg, char **tok) · 10 unmodified lines
22822365 arg->type = PRINT_SYMBOL;
22832366
22842367 field = alloc_arg();
2368+ if (!field) {
2369+ do_warning("%s: not enough memory!", __func__);
2370+ goto out_free;
2371+ }
22852372
22862373 type = process_arg(event, field, &token);
22872374 if (test_type_token(type, token, EVENT_DELIM, ","))
2288− goto out_free;
2375+ goto out_free_field;
22892376
22902377 arg->symbol.field = field;
22912378
@@ -2297,7 +2384,9 @@ process_symbols(struct event_format *event, struct print_arg *arg, char **tok) · 5 unmodified lines
22972384 type = read_token_item(tok);
22982385 return type;
22992386
2300− out_free:
2387+out_free_field:
2388+ free_arg(field);
2389+out_free:
23012390 free_token(token);
23022391 *tok = NULL;
23032392 return EVENT_ERROR;
@@ -2314,6 +2403,11 @@ process_hex(struct event_format *event, struct print_arg *arg, char **tok) · 10 unmodified lines
23142403 arg->type = PRINT_HEX;
23152404
23162405 field = alloc_arg();
2406+ if (!field) {
2407+ do_warning("%s: not enough memory!", __func__);
2408+ goto out_free;
2409+ }
2410+
23172411 type = process_arg(event, field, &token);
23182412
23192413 if (test_type_token(type, token, EVENT_DELIM, ","))
@@ -2324,6 +2418,12 @@ process_hex(struct event_format *event, struct print_arg *arg, char **tok) · 4 unmodified lines
23242418 free_token(token);
23252419
23262420 field = alloc_arg();
2421+ if (!field) {
2422+ do_warning("%s: not enough memory!", __func__);
2423+ *tok = NULL;
2424+ return EVENT_ERROR;
2425+ }
2426+
23272427 type = process_arg(event, field, &token);
23282428
23292429 if (test_type_token(type, token, EVENT_DELIM, ")"))
@@ -2381,6 +2481,12 @@ process_dynamic_array(struct event_format *event, struct print_arg *arg, char ** · 51 unmodified lines
23812481
23822482 free_token(token);
23832483 arg = alloc_arg();
2484+ if (!field) {
2485+ do_warning("%s: not enough memory!", __func__);
2486+ *tok = NULL;
2487+ return EVENT_ERROR;
2488+ }
2489+
23842490 type = process_arg(event, arg, &token);
23852491 if (type == EVENT_ERROR)
23862492 goto out_free_arg;
@@ -2434,10 +2540,16 @@ process_paren(struct event_format *event, struct print_arg *arg, char **tok) · 47 unmodified lines
24342540 /* make this a typecast and contine */
24352541
24362542 /* prevous must be an atom */
2437− if (arg->type != PRINT_ATOM)
2438− die("previous needed to be PRINT_ATOM");
2543+ if (arg->type != PRINT_ATOM) {
2544+ do_warning("previous needed to be PRINT_ATOM");
2545+ goto out_free;
2546+ }
24392547
24402548 item_arg = alloc_arg();
2549+ if (!item_arg) {
2550+ do_warning("%s: not enough memory!", __func__);
2551+ goto out_free;
2552+ }
24412553
24422554 arg->type = PRINT_TYPE;
24432555 arg->typecast.type = arg->atom.atom;
@@ -2457,7 +2569,8 @@ process_paren(struct event_format *event, struct print_arg *arg, char **tok) · 13 unmodified lines
24572569
24582570
24592571 static enum event_type
2460−process_str(struct event_format *event __unused, struct print_arg *arg, char **tok)
2572+process_str(struct event_format *event __maybe_unused, struct print_arg *arg,
2573+ char **tok)
24612574 {
24622575 enum event_type type;
24632576 char *token;
@@ -2532,6 +2645,11 @@ process_func_handler(struct event_format *event, struct pevent_function_handler · 68 unmodified lines
25322645 next_arg = &(arg->func.args);
25332646 for (i = 0; i < func->nr_args; i++) {
25342647 farg = alloc_arg();
2648+ if (!farg) {
2649+ do_warning("%s: not enough memory!", __func__);
2650+ return EVENT_ERROR;
2651+ }
2652+
25352653 type = process_arg(event, farg, &token);
25362654 if (i < (func->nr_args - 1))
25372655 test = ",";
@@ -2676,7 +2794,8 @@ process_arg_token(struct event_format *event, struct print_arg *arg, · 138 unmodified lines
26762794
26772795 case EVENT_ERROR ... EVENT_NEWLINE:
26782796 default:
2679− die("unexpected type %d", type);
2797+ do_warning("unexpected type %d", type);
2798+ return EVENT_ERROR;
26802799 }
26812800 *tok = token;
26822801
@@ -2697,6 +2816,10 @@ static int event_read_print_args(struct event_format *event, struct print_arg ** · 14 unmodified lines
26972816 }
26982817
26992818 arg = alloc_arg();
2819+ if (!arg) {
2820+ do_warning("%s: not enough memory!", __func__);
2821+ return -1;
2822+ }
27002823
27012824 type = process_arg(event, arg, &token);
27022825
@@ -2768,10 +2891,8 @@ static int event_read_print(struct event_format *event) · 65 unmodified lines
27682891 if (type == EVENT_DQUOTE) {
27692892 char *cat;
27702893
2771− cat = malloc_or_die(strlen(event->print_fmt.format) +
2772− strlen(token) + 1);
2773− strcpy(cat, event->print_fmt.format);
2774− strcat(cat, token);
2894+ if (asprintf(&cat, "%s%s", event->print_fmt.format, token) < 0)
2895+ goto fail;
27752896 free_token(token);
27762897 free_token(event->print_fmt.format);
27772898 event->print_fmt.format = NULL;
@@ -2925,8 +3046,10 @@ static int get_common_info(struct pevent *pevent, · 147 unmodified lines
29253046 * All events should have the same common elements.
29263047 * Pick any event to find where the type is;
29273048 */
2928− if (!pevent->events)
2929− die("no event_list!");
3049+ if (!pevent->events) {
3050+ do_warning("no event_list!");
3051+ return -1;
3052+ }
29303053
29313054 event = pevent->events[0];
29323055 field = pevent_find_common_field(event, type);
@@ -3084,7 +3207,8 @@ eval_num_arg(void *data, int size, struct event_format *event, struct print_arg · 151 unmodified lines
30843207 if (!arg->field.field) {
30853208 arg->field.field = pevent_find_any_field(event, arg->field.name);
30863209 if (!arg->field.field)
3087− die("field %s not found", arg->field.name);
3210+ goto out_warning_field;
3211+
30883212 }
30893213 /* must be a number */
30903214 val = pevent_read_number(pevent, data + arg->field.field->offset,
@@ -3145,8 +3269,10 @@ eval_num_arg(void *data, int size, struct event_format *event, struct print_arg · 54 unmodified lines
31453269 if (!larg->field.field) {
31463270 larg->field.field =
31473271 pevent_find_any_field(event, larg->field.name);
3148− if (!larg->field.field)
3149− die("field %s not found", larg->field.name);
3272+ if (!larg->field.field) {
3273+ arg = larg;
3274+ goto out_warning_field;
3275+ }
31503276 }
31513277 field_size = larg->field.field->elementsize;
31523278 offset = larg->field.field->offset +
@@ -3182,7 +3308,7 @@ eval_num_arg(void *data, int size, struct event_format *event, struct print_arg · 29 unmodified lines
31823308 val = left != right;
31833309 break;
31843310 default:
3185− die("unknown op '%s'", arg->op.op);
3311+ goto out_warning_op;
31863312 }
31873313 break;
31883314 case '~':
@@ -3212,7 +3338,7 @@ eval_num_arg(void *data, int size, struct event_format *event, struct print_arg · 23 unmodified lines
32123338 val = left <= right;
32133339 break;
32143340 default:
3215− die("unknown op '%s'", arg->op.op);
3341+ goto out_warning_op;
32163342 }
32173343 break;
32183344 case '>':
@@ -3227,12 +3353,13 @@ eval_num_arg(void *data, int size, struct event_format *event, struct print_arg · 8 unmodified lines
32273353 val = left >= right;
32283354 break;
32293355 default:
3230− die("unknown op '%s'", arg->op.op);
3356+ goto out_warning_op;
32313357 }
32323358 break;
32333359 case '=':
32343360 if (arg->op.op[1] != '=')
3235− die("unknown op '%s'", arg->op.op);
3361+ goto out_warning_op;
3362+
32363363 val = left == right;
32373364 break;
32383365 case '-':
@@ -3248,13 +3375,21 @@ eval_num_arg(void *data, int size, struct event_format *event, struct print_arg · 9 unmodified lines
32483375 val = left * right;
32493376 break;
32503377 default:
3251− die("unknown op '%s'", arg->op.op);
3378+ goto out_warning_op;
32523379 }
32533380 break;
32543381 default: /* not sure what to do there */
32553382 return 0;
32563383 }
32573384 return val;
3385+
3386+out_warning_op:
3387+ do_warning("%s: unknown op '%s'", __func__, arg->op.op);
3388+ return 0;
3389+
3390+out_warning_field:
3391+ do_warning("%s: field %s not found", __func__, arg->field.name);
3392+ return 0;
32583393 }
32593394
32603395 struct flag {
@@ -3331,8 +3466,10 @@ static void print_str_arg(struct trace_seq *s, void *data, int size, · 70 unmodified lines
33313466 field = arg->field.field;
33323467 if (!field) {
33333468 field = pevent_find_any_field(event, arg->field.name);
3334− if (!field)
3335− die("field %s not found", arg->field.name);
3469+ if (!field) {
3470+ str = arg->field.name;
3471+ goto out_warning_field;
3472+ }
33363473 arg->field.field = field;
33373474 }
33383475 /* Zero sized fields, mean the rest of the data */
@@ -3349,7 +3486,11 @@ static void print_str_arg(struct trace_seq *s, void *data, int size, · 10 unmodified lines
33493486 trace_seq_printf(s, "%lx", addr);
33503487 break;
33513488 }
3352− str = malloc_or_die(len + 1);
3489+ str = malloc(len + 1);
3490+ if (!str) {
3491+ do_warning("%s: not enough memory!", __func__);
3492+ return;
3493+ }
33533494 memcpy(str, data + field->offset, len);
33543495 str[len] = 0;
33553496 print_str_to_seq(s, format, len_arg, str);
@@ -3389,7 +3530,7 @@ static void print_str_arg(struct trace_seq *s, void *data, int size, · 33 unmodified lines
33893530 str = arg->hex.field->field.name;
33903531 field = pevent_find_any_field(event, str);
33913532 if (!field)
3392− die("field %s not found", str);
3533+ goto out_warning_field;
33933534 arg->hex.field->field.field = field;
33943535 }
33953536 hex = data + field->offset;
@@ -3441,6 +3582,11 @@ static void print_str_arg(struct trace_seq *s, void *data, int size, · 45 unmodified lines
34413582 /* well... */
34423583 break;
34433584 }
3585+
3586+ return;
3587+
3588+out_warning_field:
3589+ do_warning("%s: field %s not found", __func__, arg->field.name);
34443590 }
34453591
34463592 static unsigned long long
@@ -3467,7 +3613,11 @@ process_defined_func(struct trace_seq *s, void *data, int size, · 20 unmodified lines
34673613 farg = arg->func.args;
34683614 param = func_handle->params;
34693615
3470− args = malloc_or_die(sizeof(*args) * func_handle->nr_args);
3616+ ret = ULLONG_MAX;
3617+ args = malloc(sizeof(*args) * func_handle->nr_args);
3618+ if (!args)
3619+ goto out;
3620+
34713621 for (i = 0; i < func_handle->nr_args; i++) {
34723622 switch (param->type) {
34733623 case PEVENT_FUNC_ARG_INT:
@@ -3479,13 +3629,19 @@ process_defined_func(struct trace_seq *s, void *data, int size, · 5 unmodified lines
34793629 trace_seq_init(&str);
34803630 print_str_arg(&str, data, size, event, "%s", -1, farg);
34813631 trace_seq_terminate(&str);
3482− string = malloc_or_die(sizeof(*string));
3632+ string = malloc(sizeof(*string));
3633+ if (!string) {
3634+ do_warning("%s(%d): malloc str", __func__, __LINE__);
3635+ goto out_free;
3636+ }
34833637 string->next = strings;
34843638 string->str = strdup(str.buffer);
3485− if (!string->str)
3486− die("malloc str");
3487−
3488− args[i] = (unsigned long long)string->str;
3639+ if (!string->str) {
3640+ free(string);
3641+ do_warning("%s(%d): malloc str", __func__, __LINE__);
3642+ goto out_free;
3643+ }
3644+ args[i] = (uintptr_t)string->str;
34893645 strings = string;
34903646 trace_seq_destroy(&str);
34913647 break;
@@ -3494,14 +3650,15 @@ process_defined_func(struct trace_seq *s, void *data, int size, · 2 unmodified lines
34943650 * Something went totally wrong, this is not
34953651 * an input error, something in this code broke.
34963652 */
3497− die("Unexpected end of arguments\n");
3498− break;
3653+ do_warning("Unexpected end of arguments\n");
3654+ goto out_free;
34993655 }
35003656 farg = farg->next;
35013657 param = param->next;
35023658 }
35033659
35043660 ret = (*func_handle->func)(s, args);
3661+out_free:
35053662 free(args);
35063663 while (strings) {
35073664 string = strings;
@@ -3515,6 +3672,18 @@ process_defined_func(struct trace_seq *s, void *data, int size, · 7 unmodified lines
35153672 return ret;
35163673 }
35173674
3675+static void free_args(struct print_arg *args)
3676+{
3677+ struct print_arg *next;
3678+
3679+ while (args) {
3680+ next = args->next;
3681+
3682+ free_arg(args);
3683+ args = next;
3684+ }
3685+}
3686+
35183687 static struct print_arg *make_bprint_args(char *fmt, void *data, int size, struct event_format *event)
35193688 {
35203689 struct pevent *pevent = event->pevent;
@@ -3530,11 +3699,15 @@ static struct print_arg *make_bprint_args(char *fmt, void *data, int size, struc · 9 unmodified lines
35303699
35313700 if (!field) {
35323701 field = pevent_find_field(event, "buf");
3533− if (!field)
3534− die("can't find buffer field for binary printk");
3702+ if (!field) {
3703+ do_warning("can't find buffer field for binary printk");
3704+ return NULL;
3705+ }
35353706 ip_field = pevent_find_field(event, "ip");
3536− if (!ip_field)
3537− die("can't find ip field for binary printk");
3707+ if (!ip_field) {
3708+ do_warning("can't find ip field for binary printk");
3709+ return NULL;
3710+ }
35383711 pevent->bprint_buf_field = field;
35393712 pevent->bprint_ip_field = ip_field;
35403713 }
@@ -3545,13 +3718,18 @@ static struct print_arg *make_bprint_args(char *fmt, void *data, int size, struc · 4 unmodified lines
35453718 * The first arg is the IP pointer.
35463719 */
35473720 args = alloc_arg();
3721+ if (!args) {
3722+ do_warning("%s(%d): not enough memory!", __func__, __LINE__);
3723+ return NULL;
3724+ }
35483725 arg = args;
35493726 arg->next = NULL;
35503727 next = &arg->next;
35513728
35523729 arg->type = PRINT_ATOM;
3553− arg->atom.atom = malloc_or_die(32);
3554− sprintf(arg->atom.atom, "%lld", ip);
3730+
3731+ if (asprintf(&arg->atom.atom, "%lld", ip) < 0)
3732+ goto out_free;
35553733
35563734 /* skip the first "%pf : " */
35573735 for (ptr = fmt + 6, bptr = data + field->offset;
@@ -3606,10 +3784,17 @@ static struct print_arg *make_bprint_args(char *fmt, void *data, int size, struc · 48 unmodified lines
36063784 val = pevent_read_number(pevent, bptr, vsize);
36073785 bptr += vsize;
36083786 arg = alloc_arg();
3787+ if (!arg) {
3788+ do_warning("%s(%d): not enough memory!",
3789+ __func__, __LINE__);
3790+ goto out_free;
3791+ }
36093792 arg->next = NULL;
36103793 arg->type = PRINT_ATOM;
3611− arg->atom.atom = malloc_or_die(32);
3612− sprintf(arg->atom.atom, "%lld", val);
3794+ if (asprintf(&arg->atom.atom, "%lld", val) < 0) {
3795+ free(arg);
3796+ goto out_free;
3797+ }
36133798 *next = arg;
36143799 next = &arg->next;
36153800 /*
@@ -3622,11 +3807,16 @@ static struct print_arg *make_bprint_args(char *fmt, void *data, int size, struc · 6 unmodified lines
36223807 break;
36233808 case 's':
36243809 arg = alloc_arg();
3810+ if (!arg) {
3811+ do_warning("%s(%d): not enough memory!",
3812+ __func__, __LINE__);
3813+ goto out_free;
3814+ }
36253815 arg->next = NULL;
36263816 arg->type = PRINT_BSTRING;
36273817 arg->string.string = strdup(bptr);
36283818 if (!arg->string.string)
3629− break;
3819+ goto out_free;
36303820 bptr += strlen(bptr) + 1;
36313821 *next = arg;
36323822 next = &arg->next;
@@ -3637,22 +3827,15 @@ static struct print_arg *make_bprint_args(char *fmt, void *data, int size, struc · 4 unmodified lines
36373827 }
36383828
36393829 return args;
3640−}
36413830
3642−static void free_args(struct print_arg *args)
3643−{
3644− struct print_arg *next;
3645−
3646− while (args) {
3647− next = args->next;
3648−
3649− free_arg(args);
3650− args = next;
3651− }
3831+out_free:
3832+ free_args(args);
3833+ return NULL;
36523834 }
36533835
36543836 static char *
3655−get_bprint_format(void *data, int size __unused, struct event_format *event)
3837+get_bprint_format(void *data, int size __maybe_unused,
3838+ struct event_format *event)
36563839 {
36573840 struct pevent *pevent = event->pevent;
36583841 unsigned long long addr;
@@ -3665,8 +3848,10 @@ get_bprint_format(void *data, int size __unused, struct event_format *event) · 6 unmodified lines
36653848
36663849 if (!field) {
36673850 field = pevent_find_field(event, "fmt");
3668− if (!field)
3669− die("can't find format field for binary printk");
3851+ if (!field) {
3852+ do_warning("can't find format field for binary printk");
3853+ return NULL;
3854+ }
36703855 pevent->bprint_fmt_field = field;
36713856 }
36723857
@@ -3674,9 +3859,8 @@ get_bprint_format(void *data, int size __unused, struct event_format *event) · 1 unmodified line
36743859
36753860 printk = find_printk(pevent, addr);
36763861 if (!printk) {
3677− format = malloc_or_die(45);
3678− sprintf(format, "%%pf : (NO FORMAT FOUND at %llx)\n",
3679− addr);
3862+ if (asprintf(&format, "%%pf : (NO FORMAT FOUND at %llx)\n", addr) < 0)
3863+ return NULL;
36803864 return format;
36813865 }
36823866
@@ -3684,8 +3868,8 @@ get_bprint_format(void *data, int size __unused, struct event_format *event) · 1 unmodified line
36843868 /* Remove any quotes. */
36853869 if (*p == '"')
36863870 p++;
3687− format = malloc_or_die(strlen(p) + 10);
3688− sprintf(format, "%s : %s", "%pf", p);
3871+ if (asprintf(&format, "%s : %s", "%pf", p) < 0)
3872+ return NULL;
36893873 /* remove ending quotes and new line since we will add one too */
36903874 p = format + strlen(format) - 1;
36913875 if (*p == '"')
@@ -3720,8 +3904,11 @@ static void print_mac_arg(struct trace_seq *s, int mac, void *data, int size, · 28 unmodified lines
37203904 if (!arg->field.field) {
37213905 arg->field.field =
37223906 pevent_find_any_field(event, arg->field.name);
3723− if (!arg->field.field)
3724− die("field %s not found", arg->field.name);
3907+ if (!arg->field.field) {
3908+ do_warning("%s: field %s not found",
3909+ __func__, arg->field.name);
3910+ return;
3911+ }
37253912 }
37263913 if (arg->field.field->size != 6) {
37273914 trace_seq_printf(s, "INVALIDMAC");
@@ -3888,8 +4075,11 @@ static void pretty_print(struct trace_seq *s, void *data, int size, struct event · 160 unmodified lines
38884075 goto cont_process;
38894076 case '*':
38904077 /* The argument is the length. */
3891− if (!arg)
3892− die("no argument match");
4078+ if (!arg) {
4079+ do_warning("no argument match");
4080+ event->flags |= EVENT_FL_FAILED;
4081+ goto out_failed;
4082+ }
38934083 len_arg = eval_num_arg(data, size, event, arg);
38944084 len_as_arg = 1;
38954085 arg = arg->next;
@@ -3922,15 +4112,21 @@ static void pretty_print(struct trace_seq *s, void *data, int size, struct event · 26 unmodified lines
39224112 case 'x':
39234113 case 'X':
39244114 case 'u':
3925− if (!arg)
3926− die("no argument match");
4115+ if (!arg) {
4116+ do_warning("no argument match");
4117+ event->flags |= EVENT_FL_FAILED;
4118+ goto out_failed;
4119+ }
39274120
39284121 len = ((unsigned long)ptr + 1) -
39294122 (unsigned long)saveptr;
39304123
39314124 /* should never happen */
3932− if (len > 31)
3933− die("bad format!");
4125+ if (len > 31) {
4126+ do_warning("bad format!");
4127+ event->flags |= EVENT_FL_FAILED;
4128+ len = 31;
4129+ }
39344130
39354131 memcpy(format, saveptr, len);
39364132 format[len] = 0;
@@ -3994,19 +4190,26 @@ static void pretty_print(struct trace_seq *s, void *data, int size, struct event · 57 unmodified lines
39944190 trace_seq_printf(s, format, (long long)val);
39954191 break;
39964192 default:
3997− die("bad count (%d)", ls);
4193+ do_warning("bad count (%d)", ls);
4194+ event->flags |= EVENT_FL_FAILED;
39984195 }
39994196 break;
40004197 case 's':
4001− if (!arg)
4002− die("no matching argument");
4198+ if (!arg) {
4199+ do_warning("no matching argument");
4200+ event->flags |= EVENT_FL_FAILED;
4201+ goto out_failed;
4202+ }
40034203
40044204 len = ((unsigned long)ptr + 1) -
40054205 (unsigned long)saveptr;
40064206
40074207 /* should never happen */
4008− if (len > 31)
4009− die("bad format!");
4208+ if (len > 31) {
4209+ do_warning("bad format!");
4210+ event->flags |= EVENT_FL_FAILED;
4211+ len = 31;
4212+ }
40104213
40114214 memcpy(format, saveptr, len);
40124215 format[len] = 0;
@@ -4024,6 +4227,11 @@ static void pretty_print(struct trace_seq *s, void *data, int size, struct event · 11 unmodified lines
40244227 trace_seq_putc(s, *ptr);
40254228 }
40264229
4230+ if (event->flags & EVENT_FL_FAILED) {
4231+out_failed:
4232+ trace_seq_printf(s, "[FAILED TO PARSE]");
4233+ }
4234+
40274235 if (args) {
40284236 free_args(args);
40294237 free(bprint_fmt);
@@ -4356,7 +4564,10 @@ get_event_fields(const char *type, const char *name, · 326 unmodified lines
43564564 struct format_field *field;
43574565 int i = 0;
43584566
4359− fields = malloc_or_die(sizeof(*fields) * (count + 1));
4567+ fields = malloc(sizeof(*fields) * (count + 1));
4568+ if (!fields)
4569+ return NULL;
4570+
43604571 for (field = list; field; field = field->next) {
43614572 fields[i++] = field;
43624573 if (i == count + 1) {
@@ -4672,8 +4883,7 @@ static int find_event_handle(struct pevent *pevent, struct event_format *event) · 309 unmodified lines
46724883 }
46734884
46744885 /**
4675− * pevent_parse_event - parse the event format
4676− * @pevent: the handle to the pevent
4886+ * __pevent_parse_format - parse the event format
46774887 * @buf: the buffer storing the event format string
46784888 * @size: the size of @buf
46794889 * @sys: the system the event belongs to
@@ -4685,28 +4895,27 @@ static int find_event_handle(struct pevent *pevent, struct event_format *event) · 5 unmodified lines
46854895 *
46864896 * /sys/kernel/debug/tracing/events/.../.../format
46874897 */
4688−int pevent_parse_event(struct pevent *pevent,
4689− const char *buf, unsigned long size,
4690− const char *sys)
4898+enum pevent_errno __pevent_parse_format(struct event_format **eventp,
4899+ struct pevent *pevent, const char *buf,
4900+ unsigned long size, const char *sys)
46914901 {
46924902 struct event_format *event;
46934903 int ret;
46944904
46954905 init_input_buf(buf, size);
46964906
4697− event = alloc_event();
4907+ *eventp = event = alloc_event();
46984908 if (!event)
4699− return -ENOMEM;
4909+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
47004910
47014911 event->name = event_read_name();
47024912 if (!event->name) {
47034913 /* Bad event? */
4704− free(event);
4705− return -1;
4914+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
4915+ goto event_alloc_failed;
47064916 }
47074917
47084918 if (strcmp(sys, "ftrace") == 0) {
4709−
47104919 event->flags |= EVENT_FL_ISFTRACE;
47114920
47124921 if (strcmp(event->name, "bprint") == 0)
@@ -4714,74 +4923,189 @@ int pevent_parse_event(struct pevent *pevent, · 1 unmodified line
47144923 }
47154924
47164925 event->id = event_read_id();
4717− if (event->id < 0)
4718− die("failed to read event id");
4926+ if (event->id < 0) {
4927+ ret = PEVENT_ERRNO__READ_ID_FAILED;
4928+ /*
4929+ * This isn't an allocation error actually.
4930+ * But as the ID is critical, just bail out.
4931+ */
4932+ goto event_alloc_failed;
4933+ }
47194934
47204935 event->system = strdup(sys);
4721− if (!event->system)
4722− die("failed to allocate system");
4723−
4724− /* Add pevent to event so that it can be referenced */
4725− event->pevent = pevent;
4936+ if (!event->system) {
4937+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
4938+ goto event_alloc_failed;
4939+ }
47264940
47274941 ret = event_read_format(event);
47284942 if (ret < 0) {
4729− do_warning("failed to read event format for %s", event->name);
4730− goto event_failed;
4943+ ret = PEVENT_ERRNO__READ_FORMAT_FAILED;
4944+ goto event_parse_failed;
47314945 }
47324946
47334947 /*
47344948 * If the event has an override, don't print warnings if the event
47354949 * print format fails to parse.
47364950 */
4737− if (find_event_handle(pevent, event))
4951+ if (pevent && find_event_handle(pevent, event))
47384952 show_warning = 0;
47394953
47404954 ret = event_read_print(event);
4741− if (ret < 0) {
4742− do_warning("failed to read event print fmt for %s",
4743− event->name);
4744− show_warning = 1;
4745− goto event_failed;
4746− }
47474955 show_warning = 1;
47484956
4749− add_event(pevent, event);
4957+ if (ret < 0) {
4958+ ret = PEVENT_ERRNO__READ_PRINT_FAILED;
4959+ goto event_parse_failed;
4960+ }
47504961
47514962 if (!ret && (event->flags & EVENT_FL_ISFTRACE)) {
47524963 struct format_field *field;
47534964 struct print_arg *arg, **list;
47544965
47554966 /* old ftrace had no args */
4756−
47574967 list = &event->print_fmt.args;
47584968 for (field = event->format.fields; field; field = field->next) {
47594969 arg = alloc_arg();
4760− *list = arg;
4761− list = &arg->next;
4970+ if (!arg) {
4971+ event->flags |= EVENT_FL_FAILED;
4972+ return PEVENT_ERRNO__OLD_FTRACE_ARG_FAILED;
4973+ }
47624974 arg->type = PRINT_FIELD;
47634975 arg->field.name = strdup(field->name);
47644976 if (!arg->field.name) {
4765− do_warning("failed to allocate field name");
47664977 event->flags |= EVENT_FL_FAILED;
4767− return -1;
4978+ free_arg(arg);
4979+ return PEVENT_ERRNO__OLD_FTRACE_ARG_FAILED;
47684980 }
47694981 arg->field.field = field;
4982+ *list = arg;
4983+ list = &arg->next;
47704984 }
47714985 return 0;
47724986 }
47734987
4988+ return 0;
4989+
4990+ event_parse_failed:
4991+ event->flags |= EVENT_FL_FAILED;
4992+ return ret;
4993+
4994+ event_alloc_failed:
4995+ free(event->system);
4996+ free(event->name);
4997+ free(event);
4998+ *eventp = NULL;
4999+ return ret;
5000+}
5001+
5002+/**
5003+ * pevent_parse_format - parse the event format
5004+ * @buf: the buffer storing the event format string
5005+ * @size: the size of @buf
5006+ * @sys: the system the event belongs to
5007+ *
5008+ * This parses the event format and creates an event structure
5009+ * to quickly parse raw data for a given event.
5010+ *
5011+ * These files currently come from:
5012+ *
5013+ * /sys/kernel/debug/tracing/events/.../.../format
5014+ */
5015+enum pevent_errno pevent_parse_format(struct event_format **eventp, const char *buf,
5016+ unsigned long size, const char *sys)
5017+{
5018+ return __pevent_parse_format(eventp, NULL, buf, size, sys);
5019+}
5020+
5021+/**
5022+ * pevent_parse_event - parse the event format
5023+ * @pevent: the handle to the pevent
5024+ * @buf: the buffer storing the event format string
5025+ * @size: the size of @buf
5026+ * @sys: the system the event belongs to
5027+ *
5028+ * This parses the event format and creates an event structure
5029+ * to quickly parse raw data for a given event.
5030+ *
5031+ * These files currently come from:
5032+ *
5033+ * /sys/kernel/debug/tracing/events/.../.../format
5034+ */
5035+enum pevent_errno pevent_parse_event(struct pevent *pevent, const char *buf,
5036+ unsigned long size, const char *sys)
5037+{
5038+ struct event_format *event = NULL;
5039+ int ret = __pevent_parse_format(&event, pevent, buf, size, sys);
5040+
5041+ if (event == NULL)
5042+ return ret;
5043+
5044+ /* Add pevent to event so that it can be referenced */
5045+ event->pevent = pevent;
5046+
5047+ if (add_event(pevent, event)) {
5048+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
5049+ goto event_add_failed;
5050+ }
5051+
47745052 #define PRINT_ARGS 0
47755053 if (PRINT_ARGS && event->print_fmt.args)
47765054 print_args(event->print_fmt.args);
47775055
47785056 return 0;
47795057
4780− event_failed:
4781− event->flags |= EVENT_FL_FAILED;
4782− /* still add it even if it failed */
4783− add_event(pevent, event);
4784− return -1;
5058+event_add_failed:
5059+ pevent_free_format(event);
5060+ return ret;
5061+}
5062+
5063+#undef _PE
5064+#define _PE(code, str) str
5065+static const char * const pevent_error_str[] = {
5066+ PEVENT_ERRORS
5067+};
5068+#undef _PE
5069+
5070+int pevent_strerror(struct pevent *pevent, enum pevent_errno errnum,
5071+ char *buf, size_t buflen)
5072+{
5073+ int idx;
5074+ const char *msg;
5075+
5076+ if (errnum >= 0) {
5077+ msg = strerror_r(errnum, buf, buflen);
5078+ if (msg != buf) {
5079+ size_t len = strlen(msg);
5080+ memcpy(buf, msg, min(buflen - 1, len));
5081+ *(buf + min(buflen - 1, len)) = '\0';
5082+ }
5083+ return 0;
5084+ }
5085+
5086+ if (errnum <= __PEVENT_ERRNO__START ||
5087+ errnum >= __PEVENT_ERRNO__END)
5088+ return -1;
5089+
5090+ idx = errnum - __PEVENT_ERRNO__START - 1;
5091+ msg = pevent_error_str[idx];
5092+
5093+ switch (errnum) {
5094+ case PEVENT_ERRNO__MEM_ALLOC_FAILED:
5095+ case PEVENT_ERRNO__PARSE_EVENT_FAILED:
5096+ case PEVENT_ERRNO__READ_ID_FAILED:
5097+ case PEVENT_ERRNO__READ_FORMAT_FAILED:
5098+ case PEVENT_ERRNO__READ_PRINT_FAILED:
5099+ case PEVENT_ERRNO__OLD_FTRACE_ARG_FAILED:
5100+ snprintf(buf, buflen, "%s", msg);
5101+ break;
5102+
5103+ default:
5104+ /* cannot reach here */
5105+ break;
5106+ }
5107+
5108+ return 0;
47855109 }
47865110
47875111 int get_field_val(struct trace_seq *s, struct format_field *field,
@@ -5000,6 +5324,7 @@ int pevent_register_print_function(struct pevent *pevent, · 212 unmodified lines
50005324 struct pevent_func_params *param;
50015325 enum pevent_func_arg_type type;
50025326 va_list ap;
5327+ int ret;
50035328
50045329 func_handle = find_func_handler(pevent, name);
50055330 if (func_handle) {
@@ -5012,14 +5337,20 @@ int pevent_register_print_function(struct pevent *pevent, · 6 unmodified lines
50125337 remove_func_handler(pevent, name);
50135338 }
50145339
5015− func_handle = malloc_or_die(sizeof(*func_handle));
5016− memset(func_handle, 0, sizeof(*func_handle));
5340+ func_handle = calloc(1, sizeof(*func_handle));
5341+ if (!func_handle) {
5342+ do_warning("Failed to allocate function handler");
5343+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
5344+ }
50175345
50185346 func_handle->ret_type = ret_type;
50195347 func_handle->name = strdup(name);
50205348 func_handle->func = func;
5021− if (!func_handle->name)
5022− die("Failed to allocate function name");
5349+ if (!func_handle->name) {
5350+ do_warning("Failed to allocate function name");
5351+ free(func_handle);
5352+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
5353+ }
50235354
50245355 next_param = &(func_handle->params);
50255356 va_start(ap, name);
@@ -5029,11 +5360,17 @@ int pevent_register_print_function(struct pevent *pevent, · 3 unmodified lines
50295360 break;
50305361
50315362 if (type < 0 || type >= PEVENT_FUNC_ARG_MAX_TYPES) {
5032− warning("Invalid argument type %d", type);
5363+ do_warning("Invalid argument type %d", type);
5364+ ret = PEVENT_ERRNO__INVALID_ARG_TYPE;
50335365 goto out_free;
50345366 }
50355367
5036− param = malloc_or_die(sizeof(*param));
5368+ param = malloc(sizeof(*param));
5369+ if (!param) {
5370+ do_warning("Failed to allocate function param");
5371+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
5372+ goto out_free;
5373+ }
50375374 param->type = type;
50385375 param->next = NULL;
50395376
@@ -5051,7 +5388,7 @@ int pevent_register_print_function(struct pevent *pevent, · 11 unmodified lines
50515388 out_free:
50525389 va_end(ap);
50535390 free_func_handle(func_handle);
5054− return -1;
5391+ return ret;
50555392 }
50565393
50575394 /**
@@ -5103,8 +5440,12 @@ int pevent_register_event_handler(struct pevent *pevent, · 45 unmodified lines
51035440
51045441 not_found:
51055442 /* Save for later use. */
5106− handle = malloc_or_die(sizeof(*handle));
5107− memset(handle, 0, sizeof(*handle));
5443+ handle = calloc(1, sizeof(*handle));
5444+ if (!handle) {
5445+ do_warning("Failed to allocate event handler");
5446+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
5447+ }
5448+
51085449 handle->id = id;
51095450 if (event_name)
51105451 handle->event_name = strdup(event_name);
@@ -5113,7 +5454,11 @@ int pevent_register_event_handler(struct pevent *pevent, · 2 unmodified lines
51135454
51145455 if ((event_name && !handle->event_name) ||
51155456 (sys_name && !handle->sys_name)) {
5116− die("Failed to allocate event/sys name");
5457+ do_warning("Failed to allocate event/sys name");
5458+ free((void *)handle->event_name);
5459+ free((void *)handle->sys_name);
5460+ free(handle);
5461+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
51175462 }
51185463
51195464 handle->func = func;
@@ -5129,13 +5474,10 @@ int pevent_register_event_handler(struct pevent *pevent, · 9 unmodified lines
51295474 */
51305475 struct pevent *pevent_alloc(void)
51315476 {
5132− struct pevent *pevent;
5477+ struct pevent *pevent = calloc(1, sizeof(*pevent));
51335478
5134− pevent = malloc(sizeof(*pevent));
5135− if (!pevent)
5136− return NULL;
5137− memset(pevent, 0, sizeof(*pevent));
5138− pevent->ref_count = 1;
5479+ if (pevent)
5480+ pevent->ref_count = 1;
51395481
51405482 return pevent;
51415483 }
@@ -5164,7 +5506,7 @@ static void free_formats(struct format *format) · 22 unmodified lines
51645506 free_format_fields(format->fields);
51655507 }
51665508
5167−static void free_event(struct event_format *event)
5509+void pevent_free_format(struct event_format *event)
51685510 {
51695511 free(event->name);
51705512 free(event->system);
@@ -5250,7 +5592,7 @@ void pevent_free(struct pevent *pevent) · 79 unmodified lines
52505592 }
52515593
52525594 for (i = 0; i < pevent->nr_events; i++)
5253− free_event(pevent->events[i]);
5595+ pevent_free_format(pevent->events[i]);
52545596
52555597 while (pevent->handlers) {
52565598 handle = pevent->handlers;
tools/lib/traceevent/event-parse.h+40−6
@@ -24,8 +24,8 @@ · 23 unmodified lines
2424 #include <stdarg.h>
2525 #include <regex.h>
2626
27−#ifndef __unused
28−#define __unused __attribute__ ((unused))
27+#ifndef __maybe_unused
28+#define __maybe_unused __attribute__((unused))
2929 #endif
3030
3131 /* ----------------------- trace_seq ----------------------- */
@@ -49,7 +49,7 @@ struct pevent_record { · 17 unmodified lines
4949 int cpu;
5050 int ref_count;
5151 int locked; /* Do not free, even if ref_count is zero */
52− void *private;
52+ void *priv;
5353 #if DEBUG_RECORD
5454 struct pevent_record *prev;
5555 struct pevent_record *next;
@@ -106,7 +106,7 @@ struct plugin_option { · 50 unmodified lines
106106 char *plugin_alias;
107107 char *description;
108108 char *value;
109− void *private;
109+ void *priv;
110110 int set;
111111 };
112112
@@ -345,6 +345,35 @@ enum pevent_flag { · 232 unmodified lines
345345 PEVENT_NSEC_OUTPUT = 1, /* output in NSECS */
346346 };
347347
348+#define PEVENT_ERRORS \
349+ _PE(MEM_ALLOC_FAILED, "failed to allocate memory"), \
350+ _PE(PARSE_EVENT_FAILED, "failed to parse event"), \
351+ _PE(READ_ID_FAILED, "failed to read event id"), \
352+ _PE(READ_FORMAT_FAILED, "failed to read event format"), \
353+ _PE(READ_PRINT_FAILED, "failed to read event print fmt"), \
354+ _PE(OLD_FTRACE_ARG_FAILED,"failed to allocate field name for ftrace"),\
355+ _PE(INVALID_ARG_TYPE, "invalid argument type")
356+
357+#undef _PE
358+#define _PE(__code, __str) PEVENT_ERRNO__ ## __code
359+enum pevent_errno {
360+ PEVENT_ERRNO__SUCCESS = 0,
361+
362+ /*
363+ * Choose an arbitrary negative big number not to clash with standard
364+ * errno since SUS requires the errno has distinct positive values.
365+ * See 'Issue 6' in the link below.
366+ *
367+ * http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/errno.h.html
368+ */
369+ __PEVENT_ERRNO__START = -100000,
370+
371+ PEVENT_ERRORS,
372+
373+ __PEVENT_ERRNO__END,
374+};
375+#undef _PE
376+
348377 struct cmdline;
349378 struct cmdline_list;
350379 struct func_map;
@@ -509,8 +538,11 @@ void pevent_print_event(struct pevent *pevent, struct trace_seq *s, · 158 unmodified lines
509538 int pevent_parse_header_page(struct pevent *pevent, char *buf, unsigned long size,
510539 int long_size);
511540
512−int pevent_parse_event(struct pevent *pevent, const char *buf,
513− unsigned long size, const char *sys);
541+enum pevent_errno pevent_parse_event(struct pevent *pevent, const char *buf,
542+ unsigned long size, const char *sys);
543+enum pevent_errno pevent_parse_format(struct event_format **eventp, const char *buf,
544+ unsigned long size, const char *sys);
545+void pevent_free_format(struct event_format *event);
514546
515547 void *pevent_get_field_raw(struct trace_seq *s, struct event_format *event,
516548 const char *name, struct pevent_record *record,
@@ -561,6 +593,8 @@ int pevent_data_pid(struct pevent *pevent, struct pevent_record *rec); · 44 unmodified lines
561593 const char *pevent_data_comm_from_pid(struct pevent *pevent, int pid);
562594 void pevent_event_info(struct trace_seq *s, struct event_format *event,
563595 struct pevent_record *record);
596+int pevent_strerror(struct pevent *pevent, enum pevent_errno errnum,
597+ char *buf, size_t buflen);
564598
565599 struct event_format **pevent_list_events(struct pevent *pevent, enum event_sort_type);
566600 struct format_field **pevent_event_common_fields(struct event_format *event);
tools/lib/traceevent/event-utils.h+6−0
@@ -39,6 +39,12 @@ void __vdie(const char *fmt, ...); · 38 unmodified lines
3939 void __vwarning(const char *fmt, ...);
4040 void __vpr_stat(const char *fmt, ...);
4141
42+#define min(x, y) ({ \
43+ typeof(x) _min1 = (x); \
44+ typeof(y) _min2 = (y); \
45+ (void) (&_min1 == &_min2); \
46+ _min1 < _min2 ? _min1 : _min2; })
47+
4248 static inline char *strim(char *string)
4349 {
4450 char *ret;
tools/perf/.gitignore+2−0
@@ -21,3 +21,5 @@ config.mak · 20 unmodified lines
2121 config.mak.autogen
2222 *-bison.*
2323 *-flex.*
24+*.pyc
25+*.pyo
tools/perf/Documentation/Makefile+3−3
@@ -195,10 +195,10 @@ install-pdf: pdf · 194 unmodified lines
195195 #install-html: html
196196 # '$(SHELL_PATH_SQ)' ./install-webdoc.sh $(DESTDIR)$(htmldir)
197197
198−../PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
199− $(QUIET_SUBDIR0)../ $(QUIET_SUBDIR1) PERF-VERSION-FILE
198+$(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
199+ $(QUIET_SUBDIR0)../ $(QUIET_SUBDIR1) $(OUTPUT)PERF-VERSION-FILE
200200
201−-include ../PERF-VERSION-FILE
201+-include $(OUTPUT)PERF-VERSION-FILE
202202
203203 #
204204 # Determine "include::" file references in asciidoc files.
tools/perf/Documentation/jit-interface.txt+15−0
@@ -0,0 +1,15 @@
1+perf supports a simple JIT interface to resolve symbols for dynamic code generated
2+by a JIT.
3+
4+The JIT has to write a /tmp/perf-%d.map (%d = pid of process) file
5+
6+This is a text file.
7+
8+Each line has the following format, fields separated with spaces:
9+
10+START SIZE symbolname
11+
12+START and SIZE are hex numbers without 0x.
13+symbolname is the rest of the line, so it could contain special characters.
14+
15+The ownership of the file has to match the process.
tools/perf/Documentation/perf-annotate.txt+3−0
@@ -85,6 +85,9 @@ OPTIONS · 84 unmodified lines
8585 -M::
8686 --disassembler-style=:: Set disassembler style for objdump.
8787
88+--objdump=<path>::
89+ Path to objdump binary.
90+
8891 SEE ALSO
8992 --------
9093 linkperf:perf-record[1], linkperf:perf-report[1]
tools/perf/Documentation/perf-diff.txt+3−0
@@ -17,6 +17,9 @@ captured via perf record. · 16 unmodified lines
1717
1818 If no parameters are passed it will assume perf.data.old and perf.data.
1919
20+The differential profile is displayed only for events matching both
21+specified perf.data files.
22+
2023 OPTIONS
2124 -------
2225 -M::
tools/perf/Documentation/perf-kvm.txt+28−2
@@ -12,7 +12,7 @@ SYNOPSIS · 11 unmodified lines
1212 [--guestkallsyms=<path> --guestmodules=<path> | --guestvmlinux=<path>]]
1313 {top|record|report|diff|buildid-list}
1414 'perf kvm' [--host] [--guest] [--guestkallsyms=<path> --guestmodules=<path>
15− | --guestvmlinux=<path>] {top|record|report|diff|buildid-list}
15+ | --guestvmlinux=<path>] {top|record|report|diff|buildid-list|stat}
1616
1717 DESCRIPTION
1818 -----------
@@ -38,6 +38,18 @@ There are a couple of variants of perf kvm: · 19 unmodified lines
3838 so that other tools can be used to fetch packages with matching symbol tables
3939 for use by perf report.
4040
41+ 'perf kvm stat <command>' to run a command and gather performance counter
42+ statistics.
43+ Especially, perf 'kvm stat record/report' generates a statistical analysis
44+ of KVM events. Currently, vmexit, mmio and ioport events are supported.
45+ 'perf kvm stat record <command>' records kvm events and the events between
46+ start and end <command>.
47+ And this command produces a file which contains tracing results of kvm
48+ events.
49+
50+ 'perf kvm stat report' reports statistical data which includes events
51+ handled time, samples, and so on.
52+
4153 OPTIONS
4254 -------
4355 -i::
@@ -68,7 +80,21 @@ OPTIONS · 24 unmodified lines
6880 --guestvmlinux=<path>::
6981 Guest os kernel vmlinux.
7082
83+STAT REPORT OPTIONS
84+-------------------
85+--vcpu=<value>::
86+ analyze events which occures on this vcpu. (default: all vcpus)
87+
88+--events=<value>::
89+ events to be analyzed. Possible values: vmexit, mmio, ioport.
90+ (default: vmexit)
91+-k::
92+--key=<value>::
93+ Sorting key. Possible values: sample (default, sort by samples
94+ number), time (sort by average time).
95+
7196 SEE ALSO
7297 --------
7398 linkperf:perf-top[1], linkperf:perf-record[1], linkperf:perf-report[1],
74−linkperf:perf-diff[1], linkperf:perf-buildid-list[1]
99+linkperf:perf-diff[1], linkperf:perf-buildid-list[1],
100+linkperf:perf-stat[1]
tools/perf/Documentation/perf-list.txt+36−12
@@ -15,24 +15,43 @@ DESCRIPTION · 14 unmodified lines
1515 This command displays the symbolic event types which can be selected in the
1616 various perf commands with the -e option.
1717
18+[[EVENT_MODIFIERS]]
1819 EVENT MODIFIERS
1920 ---------------
2021
2122 Events can optionally have a modifer by appending a colon and one or
22−more modifiers. Modifiers allow the user to restrict when events are
23−counted with 'u' for user-space, 'k' for kernel, 'h' for hypervisor.
24−Additional modifiers are 'G' for guest counting (in KVM guests) and 'H'
25−for host counting (not in KVM guests).
23+more modifiers. Modifiers allow the user to restrict the events to be
24+counted. The following modifiers exist:
25+
26+ u - user-space counting
27+ k - kernel counting
28+ h - hypervisor counting
29+ G - guest counting (in KVM guests)
30+ H - host counting (not in KVM guests)
31+ p - precise level
2632
2733 The 'p' modifier can be used for specifying how precise the instruction
28−address should be. The 'p' modifier is currently only implemented for
29−Intel PEBS and can be specified multiple times:
30− 0 - SAMPLE_IP can have arbitrary skid
31− 1 - SAMPLE_IP must have constant skid
32− 2 - SAMPLE_IP requested to have 0 skid
33− 3 - SAMPLE_IP must have 0 skid
34+address should be. The 'p' modifier can be specified multiple times:
35+
36+ 0 - SAMPLE_IP can have arbitrary skid
37+ 1 - SAMPLE_IP must have constant skid
38+ 2 - SAMPLE_IP requested to have 0 skid
39+ 3 - SAMPLE_IP must have 0 skid
40+
41+For Intel systems precise event sampling is implemented with PEBS
42+which supports up to precise-level 2.
3443
35−The PEBS implementation now supports up to 2.
44+On AMD systems it is implemented using IBS (up to precise-level 2).
45+The precise modifier works with event types 0x76 (cpu-cycles, CPU
46+clocks not halted) and 0xC1 (micro-ops retired). Both events map to
47+IBS execution sampling (IBS op) with the IBS Op Counter Control bit
48+(IbsOpCntCtl) set respectively (see AMD64 Architecture Programmer’s
49+Manual Volume 2: System Programming, 13.3 Instruction-Based
50+Sampling). Examples to use IBS:
51+
52+ perf record -a -e cpu-cycles:p ... # use ibs op counting cycles
53+ perf record -a -e r076:p ... # same as -e cpu-cycles:p
54+ perf record -a -e r0C1:p ... # use ibs op counting micro-ops
3655
3756 RAW HARDWARE EVENT DESCRIPTOR
3857 -----------------------------
@@ -44,6 +63,11 @@ layout of IA32_PERFEVTSELx MSRs (see [Intel® 64 and IA-32 Architectures Softwar · 5 unmodified lines
4463 of IA32_PERFEVTSELx MSRs) or AMD's PerfEvtSeln (see [AMD64 Architecture Programmer’s Manual Volume 2: System Programming], Page 344,
4564 Figure 13-7 Performance Event-Select Register (PerfEvtSeln)).
4665
66+Note: Only the following bit fields can be set in x86 counter
67+registers: event, umask, edge, inv, cmask. Esp. guest/host only and
68+OS/user mode flags must be setup using <<EVENT_MODIFIERS, EVENT
69+MODIFIERS>>.
70+
4771 Example:
4872
4973 If the Intel docs for a QM720 Core i7 describe an event as:
@@ -91,4 +115,4 @@ SEE ALSO · 41 unmodified lines
91115 linkperf:perf-stat[1], linkperf:perf-top[1],
92116 linkperf:perf-record[1],
93117 http://www.intel.com/Assets/PDF/manual/253669.pdf[Intel® 64 and IA-32 Architectures Software Developer's Manual Volume 3B: System Programming Guide],
94−http://support.amd.com/us/Processor_TechDocs/24593.pdf[AMD64 Architecture Programmer’s Manual Volume 2: System Programming]
118+http://support.amd.com/us/Processor_TechDocs/24593_APM_v2.pdf[AMD64 Architecture Programmer’s Manual Volume 2: System Programming]
tools/perf/Documentation/perf-report.txt+3−0
@@ -168,6 +168,9 @@ OPTIONS · 167 unmodified lines
168168 branch stacks and it will automatically switch to the branch view mode,
169169 unless --no-branch-stack is used.
170170
171+--objdump=<path>::
172+ Path to objdump binary.
173+
171174 SEE ALSO
172175 --------
173176 linkperf:perf-stat[1], linkperf:perf-annotate[1]
tools/perf/Documentation/perf-script-perl.txt+2−2
@@ -116,8 +116,8 @@ search path and 'use'ing a few support modules (see module · 115 unmodified lines
116116 descriptions below):
117117
118118 ----
119− use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/perf-script-Util/lib";
120− use lib "./perf-script-Util/lib";
119+ use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
120+ use lib "./Perf-Trace-Util/lib";
121121 use Perf::Trace::Core;
122122 use Perf::Trace::Context;
123123 use Perf::Trace::Util;
tools/perf/Documentation/perf-script-python.txt+5−5
@@ -129,7 +129,7 @@ import os · 128 unmodified lines
129129 import sys
130130
131131 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
132− '/scripts/python/perf-script-Util/lib/Perf/Trace')
132+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
133133
134134 from perf_trace_context import *
135135 from Core import *
@@ -216,7 +216,7 @@ import os · 80 unmodified lines
216216 import sys
217217
218218 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
219− '/scripts/python/perf-script-Util/lib/Perf/Trace')
219+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
220220
221221 from perf_trace_context import *
222222 from Core import *
@@ -279,7 +279,7 @@ import os · 56 unmodified lines
279279 import sys
280280
281281 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
282− '/scripts/python/perf-script-Util/lib/Perf/Trace')
282+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
283283
284284 from perf_trace_context import *
285285 from Core import *
@@ -391,7 +391,7 @@ drwxr-xr-x 4 trz trz 4096 2010-01-26 22:30 . · 105 unmodified lines
391391 drwxr-xr-x 4 trz trz 4096 2010-01-26 22:29 ..
392392 drwxr-xr-x 2 trz trz 4096 2010-01-26 22:29 bin
393393 -rw-r--r-- 1 trz trz 2548 2010-01-26 22:29 check-perf-script.py
394−drwxr-xr-x 3 trz trz 4096 2010-01-26 22:49 perf-script-Util
394+drwxr-xr-x 3 trz trz 4096 2010-01-26 22:49 Perf-Trace-Util
395395 -rw-r--r-- 1 trz trz 1462 2010-01-26 22:30 syscall-counts.py
396396 ----
397397
@@ -518,7 +518,7 @@ descriptions below): · 120 unmodified lines
518518 import sys
519519
520520 sys.path.append(os.environ['PERF_EXEC_PATH'] + \
521− '/scripts/python/perf-script-Util/lib/Perf/Trace')
521+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
522522
523523 from perf_trace_context import *
524524 from Core import *
tools/perf/Documentation/perf-trace.txt+53−0
@@ -0,0 +1,53 @@
1+perf-trace(1)
2+=============
3+
4+NAME
5+----
6+perf-trace - strace inspired tool
7+
8+SYNOPSIS
9+--------
10+[verse]
11+'perf trace'
12+
13+DESCRIPTION
14+-----------
15+This command will show the events associated with the target, initially
16+syscalls, but other system events like pagefaults, task lifetime events,
17+scheduling events, etc.
18+
19+Initially this is a live mode only tool, but eventually will work with
20+perf.data files like the other tools, allowing a detached 'record' from
21+analysis phases.
22+
23+OPTIONS
24+-------
25+
26+--all-cpus::
27+ System-wide collection from all CPUs.
28+
29+-p::
30+--pid=::
31+ Record events on existing process ID (comma separated list).
32+
33+--tid=::
34+ Record events on existing thread ID (comma separated list).
35+
36+--uid=::
37+ Record events in threads owned by uid. Name or number.
38+
39+--no-inherit::
40+ Child tasks do not inherit counters.
41+
42+--mmap-pages=::
43+ Number of mmap data pages. Must be a power of two.
44+
45+--cpu::
46+Collect samples only on the list of CPUs provided. Multiple CPUs can be provided as a
47+comma-separated list with no space: 0,1. Ranges of CPUs are specified with -: 0-2.
48+In per-thread mode with inheritance mode on (default), Events are captured only when
49+the thread executes on the designated CPUs. Default is to monitor all CPUs.
50+
51+SEE ALSO
52+--------
53+linkperf:perf-record[1], linkperf:perf-script[1]
tools/perf/MANIFEST+4−0
@@ -10,8 +10,12 @@ include/linux/stringify.h · 9 unmodified lines
1010 lib/rbtree.c
1111 include/linux/swab.h
1212 arch/*/include/asm/unistd*.h
13+arch/*/include/asm/perf_regs.h
1314 arch/*/lib/memcpy*.S
1415 arch/*/lib/memset*.S
1516 include/linux/poison.h
1617 include/linux/magic.h
1718 include/linux/hw_breakpoint.h
19+arch/x86/include/asm/svm.h
20+arch/x86/include/asm/vmx.h
21+arch/x86/include/asm/kvm_host.h
tools/perf/Makefile+139−34
@@ -37,7 +37,14 @@ include config/utilities.mak · 36 unmodified lines
3737 #
3838 # Define NO_NEWT if you do not want TUI support.
3939 #
40+# Define NO_GTK2 if you do not want GTK+ GUI support.
41+#
4042 # Define NO_DEMANGLE if you do not want C++ symbol demangling.
43+#
44+# Define NO_LIBELF if you do not want libelf dependency (e.g. cross-builds)
45+#
46+# Define NO_LIBUNWIND if you do not want libunwind dependency for dwarf
47+# backtrace post unwind.
4148
4249 $(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
4350 @$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
@@ -50,16 +57,19 @@ ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/i386/ -e s/sun4u/sparc64/ \ · 6 unmodified lines
5057 -e s/s390x/s390/ -e s/parisc64/parisc/ \
5158 -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
5259 -e s/sh[234].*/sh/ )
60+NO_PERF_REGS := 1
5361
5462 CC = $(CROSS_COMPILE)gcc
5563 AR = $(CROSS_COMPILE)ar
5664
5765 # Additional ARCH settings for x86
5866 ifeq ($(ARCH),i386)
59− ARCH := x86
67+ override ARCH := x86
68+ NO_PERF_REGS := 0
69+ LIBUNWIND_LIBS = -lunwind -lunwind-x86
6070 endif
6171 ifeq ($(ARCH),x86_64)
62− ARCH := x86
72+ override ARCH := x86
6373 IS_X86_64 := 0
6474 ifeq (, $(findstring m32,$(EXTRA_CFLAGS)))
6575 IS_X86_64 := $(shell echo __x86_64__ | ${CC} -E -xc - | tail -n 1)
@@ -69,6 +79,8 @@ ifeq ($(ARCH),x86_64) · 3 unmodified lines
6979 ARCH_CFLAGS := -DARCH_X86_64
7080 ARCH_INCLUDE = ../../arch/x86/lib/memcpy_64.S ../../arch/x86/lib/memset_64.S
7181 endif
82+ NO_PERF_REGS := 0
83+ LIBUNWIND_LIBS = -lunwind -lunwind-x86_64
7284 endif
7385
7486 # Treat warnings as errors unless directed not to
@@ -89,7 +101,7 @@ ifdef PARSER_DEBUG · 14 unmodified lines
89101 PARSER_DEBUG_CFLAGS := -DPARSER_DEBUG
90102 endif
91103
92−CFLAGS = -fno-omit-frame-pointer -ggdb3 -Wall -Wextra -std=gnu99 $(CFLAGS_WERROR) $(CFLAGS_OPTIMIZE) $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) $(PARSER_DEBUG_CFLAGS)
104+CFLAGS = -fno-omit-frame-pointer -ggdb3 -funwind-tables -Wall -Wextra -std=gnu99 $(CFLAGS_WERROR) $(CFLAGS_OPTIMIZE) $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) $(PARSER_DEBUG_CFLAGS)
93105 EXTLIBS = -lpthread -lrt -lelf -lm
94106 ALL_CFLAGS = $(CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
95107 ALL_LDFLAGS = $(LDFLAGS)
@@ -186,10 +198,10 @@ SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) · 90 unmodified lines
186198
187199 TRACE_EVENT_DIR = ../lib/traceevent/
188200
189−ifeq ("$(origin O)", "command line")
190− TE_PATH=$(OUTPUT)/
201+ifneq ($(OUTPUT),)
202+ TE_PATH=$(OUTPUT)
191203 else
192− TE_PATH=$(TRACE_EVENT_DIR)/
204+ TE_PATH=$(TRACE_EVENT_DIR)
193205 endif
194206
195207 LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
@@ -221,13 +233,13 @@ export PERL_PATH · 25 unmodified lines
221233 FLEX = flex
222234 BISON= bison
223235
224−$(OUTPUT)util/parse-events-flex.c: util/parse-events.l
236+$(OUTPUT)util/parse-events-flex.c: util/parse-events.l $(OUTPUT)util/parse-events-bison.c
225237 $(QUIET_FLEX)$(FLEX) --header-file=$(OUTPUT)util/parse-events-flex.h $(PARSER_DEBUG_FLEX) -t util/parse-events.l > $(OUTPUT)util/parse-events-flex.c
226238
227239 $(OUTPUT)util/parse-events-bison.c: util/parse-events.y
228240 $(QUIET_BISON)$(BISON) -v util/parse-events.y -d $(PARSER_DEBUG_BISON) -o $(OUTPUT)util/parse-events-bison.c
229241
230−$(OUTPUT)util/pmu-flex.c: util/pmu.l
242+$(OUTPUT)util/pmu-flex.c: util/pmu.l $(OUTPUT)util/pmu-bison.c
231243 $(QUIET_FLEX)$(FLEX) --header-file=$(OUTPUT)util/pmu-flex.h -t util/pmu.l > $(OUTPUT)util/pmu-flex.c
232244
233245 $(OUTPUT)util/pmu-bison.c: util/pmu.y
@@ -252,6 +264,7 @@ LIB_H += util/include/linux/ctype.h · 18 unmodified lines
252264 LIB_H += util/include/linux/kernel.h
253265 LIB_H += util/include/linux/list.h
254266 LIB_H += util/include/linux/export.h
267+LIB_H += util/include/linux/magic.h
255268 LIB_H += util/include/linux/poison.h
256269 LIB_H += util/include/linux/prefetch.h
257270 LIB_H += util/include/linux/rbtree.h
@@ -321,6 +334,10 @@ LIB_H += $(TRACE_EVENT_DIR)event-parse.h · 63 unmodified lines
321334 LIB_H += util/target.h
322335 LIB_H += util/rblist.h
323336 LIB_H += util/intlist.h
337+LIB_H += util/perf_regs.h
338+LIB_H += util/unwind.h
339+LIB_H += ui/helpline.h
340+LIB_H += util/vdso.h
324341
325342 LIB_OBJS += $(OUTPUT)util/abspath.o
326343 LIB_OBJS += $(OUTPUT)util/alias.o
@@ -356,6 +373,7 @@ LIB_OBJS += $(OUTPUT)util/usage.o · 29 unmodified lines
356373 LIB_OBJS += $(OUTPUT)util/wrapper.o
357374 LIB_OBJS += $(OUTPUT)util/sigchain.o
358375 LIB_OBJS += $(OUTPUT)util/symbol.o
376+LIB_OBJS += $(OUTPUT)util/symbol-elf.o
359377 LIB_OBJS += $(OUTPUT)util/dso-test-data.o
360378 LIB_OBJS += $(OUTPUT)util/color.o
361379 LIB_OBJS += $(OUTPUT)util/pager.o
@@ -387,11 +405,15 @@ LIB_OBJS += $(OUTPUT)util/cgroup.o · 25 unmodified lines
387405 LIB_OBJS += $(OUTPUT)util/target.o
388406 LIB_OBJS += $(OUTPUT)util/rblist.o
389407 LIB_OBJS += $(OUTPUT)util/intlist.o
408+LIB_OBJS += $(OUTPUT)util/vdso.o
409+LIB_OBJS += $(OUTPUT)util/stat.o
390410
391−BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o
411+LIB_OBJS += $(OUTPUT)ui/helpline.o
412+LIB_OBJS += $(OUTPUT)ui/hist.o
413+LIB_OBJS += $(OUTPUT)ui/stdio/hist.o
392414
415+BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o
393416 BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
394−
395417 # Benchmark modules
396418 BUILTIN_OBJS += $(OUTPUT)bench/sched-messaging.o
397419 BUILTIN_OBJS += $(OUTPUT)bench/sched-pipe.o
@@ -449,34 +471,73 @@ PYRF_OBJS += $(OUTPUT)util/xyarray.o · 51 unmodified lines
449471 -include config.mak.autogen
450472 -include config.mak
451473
452−ifndef NO_DWARF
453−FLAGS_DWARF=$(ALL_CFLAGS) -ldw -lelf $(ALL_LDFLAGS) $(EXTLIBS)
454−ifneq ($(call try-cc,$(SOURCE_DWARF),$(FLAGS_DWARF)),y)
455− msg := $(warning No libdw.h found or old libdw.h found or elfutils is older than 0.138, disables dwarf support. Please install new elfutils-devel/libdw-dev);
474+ifdef NO_LIBELF
456475 NO_DWARF := 1
457−endif # Dwarf support
458−endif # NO_DWARF
459−
460−-include arch/$(ARCH)/Makefile
461−
462−ifneq ($(OUTPUT),)
463− BASIC_CFLAGS += -I$(OUTPUT)
464−endif
465−
476+ NO_DEMANGLE := 1
477+ NO_LIBUNWIND := 1
478+else
466479 FLAGS_LIBELF=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS)
467480 ifneq ($(call try-cc,$(SOURCE_LIBELF),$(FLAGS_LIBELF)),y)
468481 FLAGS_GLIBC=$(ALL_CFLAGS) $(ALL_LDFLAGS)
469482 ifneq ($(call try-cc,$(SOURCE_GLIBC),$(FLAGS_GLIBC)),y)
470483 msg := $(error No gnu/libc-version.h found, please install glibc-dev[el]/glibc-static);
471484 else
472− msg := $(error No libelf.h/libelf found, please install libelf-dev/elfutils-libelf-devel);
485+ NO_LIBELF := 1
486+ NO_DWARF := 1
487+ NO_DEMANGLE := 1
473488 endif
474489 endif
490+endif # NO_LIBELF
491+
492+ifndef NO_LIBUNWIND
493+# for linking with debug library, run like:
494+# make DEBUG=1 LIBUNWIND_DIR=/opt/libunwind/
495+ifdef LIBUNWIND_DIR
496+ LIBUNWIND_CFLAGS := -I$(LIBUNWIND_DIR)/include
497+ LIBUNWIND_LDFLAGS := -L$(LIBUNWIND_DIR)/lib
498+endif
499+
500+FLAGS_UNWIND=$(LIBUNWIND_CFLAGS) $(ALL_CFLAGS) $(LIBUNWIND_LDFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) $(LIBUNWIND_LIBS)
501+ifneq ($(call try-cc,$(SOURCE_LIBUNWIND),$(FLAGS_UNWIND)),y)
502+ msg := $(warning No libunwind found, disabling post unwind support. Please install libunwind-dev[el] >= 0.99);
503+ NO_LIBUNWIND := 1
504+endif # Libunwind support
505+endif # NO_LIBUNWIND
506+
507+-include arch/$(ARCH)/Makefile
508+
509+ifneq ($(OUTPUT),)
510+ BASIC_CFLAGS += -I$(OUTPUT)
511+endif
512+
513+ifdef NO_LIBELF
514+BASIC_CFLAGS += -DNO_LIBELF_SUPPORT
515+
516+EXTLIBS := $(filter-out -lelf,$(EXTLIBS))
517+
518+# Remove ELF/DWARF dependent codes
519+LIB_OBJS := $(filter-out $(OUTPUT)util/symbol-elf.o,$(LIB_OBJS))
520+LIB_OBJS := $(filter-out $(OUTPUT)util/dwarf-aux.o,$(LIB_OBJS))
521+LIB_OBJS := $(filter-out $(OUTPUT)util/probe-event.o,$(LIB_OBJS))
522+LIB_OBJS := $(filter-out $(OUTPUT)util/probe-finder.o,$(LIB_OBJS))
523+
524+BUILTIN_OBJS := $(filter-out $(OUTPUT)builtin-probe.o,$(BUILTIN_OBJS))
525+
526+# Use minimal symbol handling
527+LIB_OBJS += $(OUTPUT)util/symbol-minimal.o
528+
529+else # NO_LIBELF
475530
476531 ifneq ($(call try-cc,$(SOURCE_ELF_MMAP),$(FLAGS_COMMON)),y)
477532 BASIC_CFLAGS += -DLIBELF_NO_MMAP
478533 endif
479534
535+FLAGS_DWARF=$(ALL_CFLAGS) -ldw -lelf $(ALL_LDFLAGS) $(EXTLIBS)
536+ifneq ($(call try-cc,$(SOURCE_DWARF),$(FLAGS_DWARF)),y)
537+ msg := $(warning No libdw.h found or old libdw.h found or elfutils is older than 0.138, disables dwarf support. Please install new elfutils-devel/libdw-dev);
538+ NO_DWARF := 1
539+endif # Dwarf support
540+
480541 ifndef NO_DWARF
481542 ifeq ($(origin PERF_HAVE_DWARF_REGS), undefined)
482543 msg := $(warning DWARF register mappings have not been defined for architecture $(ARCH), DWARF support disabled);
@@ -487,6 +548,29 @@ else · 4 unmodified lines
487548 LIB_OBJS += $(OUTPUT)util/dwarf-aux.o
488549 endif # PERF_HAVE_DWARF_REGS
489550 endif # NO_DWARF
551+endif # NO_LIBELF
552+
553+ifdef NO_LIBUNWIND
554+ BASIC_CFLAGS += -DNO_LIBUNWIND_SUPPORT
555+else
556+ EXTLIBS += $(LIBUNWIND_LIBS)
557+ BASIC_CFLAGS := $(LIBUNWIND_CFLAGS) $(BASIC_CFLAGS)
558+ BASIC_LDFLAGS := $(LIBUNWIND_LDFLAGS) $(BASIC_LDFLAGS)
559+ LIB_OBJS += $(OUTPUT)util/unwind.o
560+endif
561+
562+ifdef NO_LIBAUDIT
563+ BASIC_CFLAGS += -DNO_LIBAUDIT_SUPPORT
564+else
565+ FLAGS_LIBAUDIT = $(ALL_CFLAGS) $(ALL_LDFLAGS) -laudit
566+ ifneq ($(call try-cc,$(SOURCE_LIBAUDIT),$(FLAGS_LIBAUDIT)),y)
567+ msg := $(warning No libaudit.h found, disables 'trace' tool, please install audit-libs-devel or libaudit-dev);
568+ BASIC_CFLAGS += -DNO_LIBAUDIT_SUPPORT
569+ else
570+ BUILTIN_OBJS += $(OUTPUT)builtin-trace.o
571+ EXTLIBS += -laudit
572+ endif
573+endif
490574
491575 ifdef NO_NEWT
492576 BASIC_CFLAGS += -DNO_NEWT_SUPPORT
@@ -504,14 +588,13 @@ else · 11 unmodified lines
504588 LIB_OBJS += $(OUTPUT)ui/browsers/annotate.o
505589 LIB_OBJS += $(OUTPUT)ui/browsers/hists.o
506590 LIB_OBJS += $(OUTPUT)ui/browsers/map.o
507− LIB_OBJS += $(OUTPUT)ui/helpline.o
508591 LIB_OBJS += $(OUTPUT)ui/progress.o
509592 LIB_OBJS += $(OUTPUT)ui/util.o
510593 LIB_OBJS += $(OUTPUT)ui/tui/setup.o
511594 LIB_OBJS += $(OUTPUT)ui/tui/util.o
595+ LIB_OBJS += $(OUTPUT)ui/tui/helpline.o
512596 LIB_H += ui/browser.h
513597 LIB_H += ui/browsers/map.h
514− LIB_H += ui/helpline.h
515598 LIB_H += ui/keysyms.h
516599 LIB_H += ui/libslang.h
517600 LIB_H += ui/progress.h
@@ -523,7 +606,7 @@ endif · 5 unmodified lines
523606 ifdef NO_GTK2
524607 BASIC_CFLAGS += -DNO_GTK2_SUPPORT
525608 else
526− FLAGS_GTK2=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) $(shell pkg-config --libs --cflags gtk+-2.0)
609+ FLAGS_GTK2=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) $(shell pkg-config --libs --cflags gtk+-2.0 2>/dev/null)
527610 ifneq ($(call try-cc,$(SOURCE_GTK2),$(FLAGS_GTK2)),y)
528611 msg := $(warning GTK2 not found, disables GTK2 support. Please install gtk2-devel or libgtk2.0-dev);
529612 BASIC_CFLAGS += -DNO_GTK2_SUPPORT
@@ -531,11 +614,12 @@ else · 1 unmodified line
531614 ifeq ($(call try-cc,$(SOURCE_GTK2_INFOBAR),$(FLAGS_GTK2)),y)
532615 BASIC_CFLAGS += -DHAVE_GTK_INFO_BAR
533616 endif
534− BASIC_CFLAGS += $(shell pkg-config --cflags gtk+-2.0)
535− EXTLIBS += $(shell pkg-config --libs gtk+-2.0)
617+ BASIC_CFLAGS += $(shell pkg-config --cflags gtk+-2.0 2>/dev/null)
618+ EXTLIBS += $(shell pkg-config --libs gtk+-2.0 2>/dev/null)
536619 LIB_OBJS += $(OUTPUT)ui/gtk/browser.o
537620 LIB_OBJS += $(OUTPUT)ui/gtk/setup.o
538621 LIB_OBJS += $(OUTPUT)ui/gtk/util.o
622+ LIB_OBJS += $(OUTPUT)ui/gtk/helpline.o
539623 # Make sure that it'd be included only once.
540624 ifneq ($(findstring -DNO_NEWT_SUPPORT,$(BASIC_CFLAGS)),)
541625 LIB_OBJS += $(OUTPUT)ui/setup.o
@@ -644,7 +728,7 @@ else · 102 unmodified lines
644728 EXTLIBS += -liberty
645729 BASIC_CFLAGS += -DHAVE_CPLUS_DEMANGLE
646730 else
647− FLAGS_BFD=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) -lbfd
731+ FLAGS_BFD=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) -DPACKAGE='perf' -lbfd
648732 has_bfd := $(call try-cc,$(SOURCE_BFD),$(FLAGS_BFD))
649733 ifeq ($(has_bfd),y)
650734 EXTLIBS += -lbfd
@@ -674,6 +758,13 @@ else · 23 unmodified lines
674758 endif
675759 endif
676760
761+ifeq ($(NO_PERF_REGS),0)
762+ ifeq ($(ARCH),x86)
763+ LIB_H += arch/x86/include/perf_regs.h
764+ endif
765+else
766+ BASIC_CFLAGS += -DNO_PERF_REGS
767+endif
677768
678769 ifdef NO_STRLCPY
679770 BASIC_CFLAGS += -DNO_STRLCPY
@@ -683,6 +774,14 @@ else · 3 unmodified lines
683774 endif
684775 endif
685776
777+ifdef NO_BACKTRACE
778+ BASIC_CFLAGS += -DNO_BACKTRACE
779+else
780+ ifneq ($(call try-cc,$(SOURCE_BACKTRACE),),y)
781+ BASIC_CFLAGS += -DNO_BACKTRACE
782+ endif
783+endif
784+
686785 ifdef ASCIIDOC8
687786 export ASCIIDOC8
688787 endif
@@ -700,6 +799,7 @@ perfexecdir_SQ = $(subst ','\'',$(perfexecdir)) · 11 unmodified lines
700799 template_dir_SQ = $(subst ','\'',$(template_dir))
701800 htmldir_SQ = $(subst ','\'',$(htmldir))
702801 prefix_SQ = $(subst ','\'',$(prefix))
802+sysconfdir_SQ = $(subst ','\'',$(sysconfdir))
703803
704804 SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
705805
@@ -767,10 +867,10 @@ $(OUTPUT)perf.o perf.spec \ · 61 unmodified lines
767867 # over the general rule for .o
768868
769869 $(OUTPUT)util/%-flex.o: $(OUTPUT)util/%-flex.c $(OUTPUT)PERF-CFLAGS
770− $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) -Iutil/ -w $<
870+ $(QUIET_CC)$(CC) -o $@ -c -Iutil/ $(ALL_CFLAGS) -w $<
771871
772872 $(OUTPUT)util/%-bison.o: $(OUTPUT)util/%-bison.c $(OUTPUT)PERF-CFLAGS
773− $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) -DYYENABLE_NLS=0 -DYYLTYPE_IS_TRIVIAL=0 -Iutil/ -w $<
873+ $(QUIET_CC)$(CC) -o $@ -c -Iutil/ $(ALL_CFLAGS) -DYYENABLE_NLS=0 -DYYLTYPE_IS_TRIVIAL=0 -w $<
774874
775875 $(OUTPUT)%.o: %.c $(OUTPUT)PERF-CFLAGS
776876 $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
@@ -842,7 +942,10 @@ $(LIB_FILE): $(LIB_OBJS) · 65 unmodified lines
842942
843943 # libtraceevent.a
844944 $(LIBTRACEEVENT):
845− $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(QUIET_SUBDIR1) $(COMMAND_O) libtraceevent.a
945+ $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) libtraceevent.a
946+
947+$(LIBTRACEEVENT)-clean:
948+ $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) clean
846949
847950 help:
848951 @echo 'Perf make targets:'
@@ -951,6 +1054,8 @@ install: all · 102 unmodified lines
9511054 $(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'
9521055 $(INSTALL) scripts/python/*.py -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'
9531056 $(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'
1057+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(sysconfdir_SQ)/bash_completion.d'
1058+ $(INSTALL) bash_completion '$(DESTDIR_SQ)$(sysconfdir_SQ)/bash_completion.d/perf'
9541059
9551060 install-python_ext:
9561061 $(PYTHON_WORD) util/setup.py --quiet install --root='/$(DESTDIR_SQ)'
@@ -981,7 +1086,7 @@ quick-install-html: · 24 unmodified lines
9811086
9821087 ### Cleaning rules
9831088
984−clean:
1089+clean: $(LIBTRACEEVENT)-clean
9851090 $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf.o $(LANG_BINDINGS)
9861091 $(RM) $(ALL_PROGRAMS) perf
9871092 $(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope*
tools/perf/arch/x86/Makefile+3−0
@@ -2,4 +2,7 @@ ifndef NO_DWARF · 1 unmodified line
22 PERF_HAVE_DWARF_REGS := 1
33 LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/dwarf-regs.o
44 endif
5+ifndef NO_LIBUNWIND
6+LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/unwind.o
7+endif
58 LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/header.o
tools/perf/arch/x86/include/perf_regs.h+80−0
@@ -0,0 +1,80 @@
1+#ifndef ARCH_PERF_REGS_H
2+#define ARCH_PERF_REGS_H
3+
4+#include <stdlib.h>
5+#include "../../util/types.h"
6+#include "../../../../../arch/x86/include/asm/perf_regs.h"
7+
8+#ifndef ARCH_X86_64
9+#define PERF_REGS_MASK ((1ULL << PERF_REG_X86_32_MAX) - 1)
10+#else
11+#define REG_NOSUPPORT ((1ULL << PERF_REG_X86_DS) | \
12+ (1ULL << PERF_REG_X86_ES) | \
13+ (1ULL << PERF_REG_X86_FS) | \
14+ (1ULL << PERF_REG_X86_GS))
15+#define PERF_REGS_MASK (((1ULL << PERF_REG_X86_64_MAX) - 1) & ~REG_NOSUPPORT)
16+#endif
17+#define PERF_REG_IP PERF_REG_X86_IP
18+#define PERF_REG_SP PERF_REG_X86_SP
19+
20+static inline const char *perf_reg_name(int id)
21+{
22+ switch (id) {
23+ case PERF_REG_X86_AX:
24+ return "AX";
25+ case PERF_REG_X86_BX:
26+ return "BX";
27+ case PERF_REG_X86_CX:
28+ return "CX";
29+ case PERF_REG_X86_DX:
30+ return "DX";
31+ case PERF_REG_X86_SI:
32+ return "SI";
33+ case PERF_REG_X86_DI:
34+ return "DI";
35+ case PERF_REG_X86_BP:
36+ return "BP";
37+ case PERF_REG_X86_SP:
38+ return "SP";
39+ case PERF_REG_X86_IP:
40+ return "IP";
41+ case PERF_REG_X86_FLAGS:
42+ return "FLAGS";
43+ case PERF_REG_X86_CS:
44+ return "CS";
45+ case PERF_REG_X86_SS:
46+ return "SS";
47+ case PERF_REG_X86_DS:
48+ return "DS";
49+ case PERF_REG_X86_ES:
50+ return "ES";
51+ case PERF_REG_X86_FS:
52+ return "FS";
53+ case PERF_REG_X86_GS:
54+ return "GS";
55+#ifdef ARCH_X86_64
56+ case PERF_REG_X86_R8:
57+ return "R8";
58+ case PERF_REG_X86_R9:
59+ return "R9";
60+ case PERF_REG_X86_R10:
61+ return "R10";
62+ case PERF_REG_X86_R11:
63+ return "R11";
64+ case PERF_REG_X86_R12:
65+ return "R12";
66+ case PERF_REG_X86_R13:
67+ return "R13";
68+ case PERF_REG_X86_R14:
69+ return "R14";
70+ case PERF_REG_X86_R15:
71+ return "R15";
72+#endif /* ARCH_X86_64 */
73+ default:
74+ return NULL;
75+ }
76+
77+ return NULL;
78+}
79+
80+#endif /* ARCH_PERF_REGS_H */
tools/perf/arch/x86/util/unwind.c+111−0
@@ -0,0 +1,111 @@
1+
2+#include <errno.h>
3+#include <libunwind.h>
4+#include "perf_regs.h"
5+#include "../../util/unwind.h"
6+
7+#ifdef ARCH_X86_64
8+int unwind__arch_reg_id(int regnum)
9+{
10+ int id;
11+
12+ switch (regnum) {
13+ case UNW_X86_64_RAX:
14+ id = PERF_REG_X86_AX;
15+ break;
16+ case UNW_X86_64_RDX:
17+ id = PERF_REG_X86_DX;
18+ break;
19+ case UNW_X86_64_RCX:
20+ id = PERF_REG_X86_CX;
21+ break;
22+ case UNW_X86_64_RBX:
23+ id = PERF_REG_X86_BX;
24+ break;
25+ case UNW_X86_64_RSI:
26+ id = PERF_REG_X86_SI;
27+ break;
28+ case UNW_X86_64_RDI:
29+ id = PERF_REG_X86_DI;
30+ break;
31+ case UNW_X86_64_RBP:
32+ id = PERF_REG_X86_BP;
33+ break;
34+ case UNW_X86_64_RSP:
35+ id = PERF_REG_X86_SP;
36+ break;
37+ case UNW_X86_64_R8:
38+ id = PERF_REG_X86_R8;
39+ break;
40+ case UNW_X86_64_R9:
41+ id = PERF_REG_X86_R9;
42+ break;
43+ case UNW_X86_64_R10:
44+ id = PERF_REG_X86_R10;
45+ break;
46+ case UNW_X86_64_R11:
47+ id = PERF_REG_X86_R11;
48+ break;
49+ case UNW_X86_64_R12:
50+ id = PERF_REG_X86_R12;
51+ break;
52+ case UNW_X86_64_R13:
53+ id = PERF_REG_X86_R13;
54+ break;
55+ case UNW_X86_64_R14:
56+ id = PERF_REG_X86_R14;
57+ break;
58+ case UNW_X86_64_R15:
59+ id = PERF_REG_X86_R15;
60+ break;
61+ case UNW_X86_64_RIP:
62+ id = PERF_REG_X86_IP;
63+ break;
64+ default:
65+ pr_err("unwind: invalid reg id %d\n", regnum);
66+ return -EINVAL;
67+ }
68+
69+ return id;
70+}
71+#else
72+int unwind__arch_reg_id(int regnum)
73+{
74+ int id;
75+
76+ switch (regnum) {
77+ case UNW_X86_EAX:
78+ id = PERF_REG_X86_AX;
79+ break;
80+ case UNW_X86_EDX:
81+ id = PERF_REG_X86_DX;
82+ break;
83+ case UNW_X86_ECX:
84+ id = PERF_REG_X86_CX;
85+ break;
86+ case UNW_X86_EBX:
87+ id = PERF_REG_X86_BX;
88+ break;
89+ case UNW_X86_ESI:
90+ id = PERF_REG_X86_SI;
91+ break;
92+ case UNW_X86_EDI:
93+ id = PERF_REG_X86_DI;
94+ break;
95+ case UNW_X86_EBP:
96+ id = PERF_REG_X86_BP;
97+ break;
98+ case UNW_X86_ESP:
99+ id = PERF_REG_X86_SP;
100+ break;
101+ case UNW_X86_EIP:
102+ id = PERF_REG_X86_IP;
103+ break;
104+ default:
105+ pr_err("unwind: invalid reg id %d\n", regnum);
106+ return -EINVAL;
107+ }
108+
109+ return id;
110+}
111+#endif /* ARCH_X86_64 */
tools/perf/bash_completion+26−0
@@ -0,0 +1,26 @@
1+# perf completion
2+
3+have perf &&
4+_perf()
5+{
6+ local cur cmd
7+
8+ COMPREPLY=()
9+ _get_comp_words_by_ref cur prev
10+
11+ cmd=${COMP_WORDS[0]}
12+
13+ # List perf subcommands
14+ if [ $COMP_CWORD -eq 1 ]; then
15+ cmds=$($cmd --list-cmds)
16+ COMPREPLY=( $( compgen -W '$cmds' -- "$cur" ) )
17+ # List possible events for -e option
18+ elif [[ $prev == "-e" && "${COMP_WORDS[1]}" == @(record|stat|top) ]]; then
19+ cmds=$($cmd list --raw-dump)
20+ COMPREPLY=( $( compgen -W '$cmds' -- "$cur" ) )
21+ # Fall down to list regular files
22+ else
23+ _filedir
24+ fi
25+} &&
26+complete -F _perf perf
tools/perf/bench/bench.h+2−1
@@ -3,7 +3,8 @@ · 2 unmodified lines
33
44 extern int bench_sched_messaging(int argc, const char **argv, const char *prefix);
55 extern int bench_sched_pipe(int argc, const char **argv, const char *prefix);
6−extern int bench_mem_memcpy(int argc, const char **argv, const char *prefix __used);
6+extern int bench_mem_memcpy(int argc, const char **argv,
7+ const char *prefix __maybe_unused);
78 extern int bench_mem_memset(int argc, const char **argv, const char *prefix);
89
910 #define BENCH_FORMAT_DEFAULT_STR "default"
tools/perf/bench/mem-memcpy.c+1−1
@@ -177,7 +177,7 @@ static double do_memcpy_gettimeofday(memcpy_t fn, size_t len, bool prefault) · 176 unmodified lines
177177 } while (0)
178178
179179 int bench_mem_memcpy(int argc, const char **argv,
180− const char *prefix __used)
180+ const char *prefix __maybe_unused)
181181 {
182182 int i;
183183 size_t len;
tools/perf/bench/mem-memset.c+1−1
@@ -171,7 +171,7 @@ static double do_memset_gettimeofday(memset_t fn, size_t len, bool prefault) · 170 unmodified lines
171171 } while (0)
172172
173173 int bench_mem_memset(int argc, const char **argv,
174− const char *prefix __used)
174+ const char *prefix __maybe_unused)
175175 {
176176 int i;
177177 size_t len;
tools/perf/bench/sched-messaging.c+1−1
@@ -267,7 +267,7 @@ static const char * const bench_sched_message_usage[] = { · 266 unmodified lines
267267 };
268268
269269 int bench_sched_messaging(int argc, const char **argv,
270− const char *prefix __used)
270+ const char *prefix __maybe_unused)
271271 {
272272 unsigned int i, total_children;
273273 struct timeval start, stop, diff;
tools/perf/bench/sched-pipe.c+5−5
@@ -43,7 +43,7 @@ static const char * const bench_sched_pipe_usage[] = { · 42 unmodified lines
4343 };
4444
4545 int bench_sched_pipe(int argc, const char **argv,
46− const char *prefix __used)
46+ const char *prefix __maybe_unused)
4747 {
4848 int pipe_1[2], pipe_2[2];
4949 int m = 0, i;
@@ -55,14 +55,14 @@ int bench_sched_pipe(int argc, const char **argv, · 5 unmodified lines
5555 * discarding returned value of read(), write()
5656 * causes error in building environment for perf
5757 */
58− int __used ret, wait_stat;
59− pid_t pid, retpid;
58+ int __maybe_unused ret, wait_stat;
59+ pid_t pid, retpid __maybe_unused;
6060
6161 argc = parse_options(argc, argv, options,
6262 bench_sched_pipe_usage, 0);
6363
64− assert(!pipe(pipe_1));
65− assert(!pipe(pipe_2));
64+ BUG_ON(pipe(pipe_1));
65+ BUG_ON(pipe(pipe_2));
6666
6767 pid = fork();
6868 assert(pid >= 0);
tools/perf/builtin-annotate.c+3−1
@@ -239,7 +239,7 @@ static const char * const annotate_usage[] = { · 238 unmodified lines
239239 NULL
240240 };
241241
242−int cmd_annotate(int argc, const char **argv, const char *prefix __used)
242+int cmd_annotate(int argc, const char **argv, const char *prefix __maybe_unused)
243243 {
244244 struct perf_annotate annotate = {
245245 .tool = {
@@ -282,6 +282,8 @@ int cmd_annotate(int argc, const char **argv, const char *prefix __used) · 36 unmodified lines
282282 "Display raw encoding of assembly instructions (default)"),
283283 OPT_STRING('M', "disassembler-style", &disassembler_style, "disassembler style",
284284 "Specify disassembler style (e.g. -M intel for intel syntax)"),
285+ OPT_STRING(0, "objdump", &objdump_path, "path",
286+ "objdump binary to use for disassembly and annotations"),
285287 OPT_END()
286288 };
287289
tools/perf/builtin-bench.c+1−1
@@ -173,7 +173,7 @@ static void all_subsystem(void) · 172 unmodified lines
173173 all_suite(&subsystems[i]);
174174 }
175175
176−int cmd_bench(int argc, const char **argv, const char *prefix __used)
176+int cmd_bench(int argc, const char **argv, const char *prefix __maybe_unused)
177177 {
178178 int i, j, status = 0;
179179
tools/perf/builtin-buildid-cache.c+6−4
@@ -43,15 +43,16 @@ static int build_id_cache__add_file(const char *filename, const char *debugdir) · 42 unmodified lines
4343 }
4444
4545 build_id__sprintf(build_id, sizeof(build_id), sbuild_id);
46− err = build_id_cache__add_s(sbuild_id, debugdir, filename, false);
46+ err = build_id_cache__add_s(sbuild_id, debugdir, filename,
47+ false, false);
4748 if (verbose)
4849 pr_info("Adding %s %s: %s\n", sbuild_id, filename,
4950 err ? "FAIL" : "Ok");
5051 return err;
5152 }
5253
53−static int build_id_cache__remove_file(const char *filename __used,
54− const char *debugdir __used)
54+static int build_id_cache__remove_file(const char *filename __maybe_unused,
55+ const char *debugdir __maybe_unused)
5556 {
5657 u8 build_id[BUILD_ID_SIZE];
5758 char sbuild_id[BUILD_ID_SIZE * 2 + 1];
@@ -119,7 +120,8 @@ static int __cmd_buildid_cache(void) · 61 unmodified lines
119120 return 0;
120121 }
121122
122−int cmd_buildid_cache(int argc, const char **argv, const char *prefix __used)
123+int cmd_buildid_cache(int argc, const char **argv,
124+ const char *prefix __maybe_unused)
123125 {
124126 argc = parse_options(argc, argv, buildid_cache_options,
125127 buildid_cache_usage, 0);
tools/perf/builtin-buildid-list.c+3−4
@@ -16,8 +16,6 @@ · 15 unmodified lines
1616 #include "util/session.h"
1717 #include "util/symbol.h"
1818
19−#include <libelf.h>
20−
2119 static const char *input_name;
2220 static bool force;
2321 static bool show_kernel;
@@ -71,7 +69,7 @@ static int perf_session__list_build_ids(void) · 47 unmodified lines
7169 {
7270 struct perf_session *session;
7371
74− elf_version(EV_CURRENT);
72+ symbol__elf_init();
7573
7674 session = perf_session__new(input_name, O_RDONLY, force, false,
7775 &build_id__mark_dso_hit_ops);
@@ -105,7 +103,8 @@ static int __cmd_buildid_list(void) · 27 unmodified lines
105103 return perf_session__list_build_ids();
106104 }
107105
108−int cmd_buildid_list(int argc, const char **argv, const char *prefix __used)
106+int cmd_buildid_list(int argc, const char **argv,
107+ const char *prefix __maybe_unused)
109108 {
110109 argc = parse_options(argc, argv, options, buildid_list_usage, 0);
111110 setup_pager();
tools/perf/builtin-diff.c+63−33
@@ -10,6 +10,7 @@ · 9 unmodified lines
1010 #include "util/event.h"
1111 #include "util/hist.h"
1212 #include "util/evsel.h"
13+#include "util/evlist.h"
1314 #include "util/session.h"
1415 #include "util/tool.h"
1516 #include "util/sort.h"
@@ -24,11 +25,6 @@ static char diff__default_sort_order[] = "dso,symbol"; · 8 unmodified lines
2425 static bool force;
2526 static bool show_displacement;
2627
27−struct perf_diff {
28− struct perf_tool tool;
29− struct perf_session *session;
30−};
31−
3228 static int hists__add_entry(struct hists *self,
3329 struct addr_location *al, u64 period)
3430 {
@@ -37,14 +33,12 @@ static int hists__add_entry(struct hists *self, · 2 unmodified lines
3733 return -ENOMEM;
3834 }
3935
40−static int diff__process_sample_event(struct perf_tool *tool,
36+static int diff__process_sample_event(struct perf_tool *tool __maybe_unused,
4137 union perf_event *event,
4238 struct perf_sample *sample,
43− struct perf_evsel *evsel __used,
39+ struct perf_evsel *evsel,
4440 struct machine *machine)
4541 {
46− struct perf_diff *_diff = container_of(tool, struct perf_diff, tool);
47− struct perf_session *session = _diff->session;
4842 struct addr_location al;
4943
5044 if (perf_event__preprocess_sample(event, machine, &al, sample, NULL) < 0) {
@@ -56,26 +50,24 @@ static int diff__process_sample_event(struct perf_tool *tool, · 5 unmodified lines
5650 if (al.filtered || al.sym == NULL)
5751 return 0;
5852
59− if (hists__add_entry(&session->hists, &al, sample->period)) {
53+ if (hists__add_entry(&evsel->hists, &al, sample->period)) {
6054 pr_warning("problem incrementing symbol period, skipping event\n");
6155 return -1;
6256 }
6357
64− session->hists.stats.total_period += sample->period;
58+ evsel->hists.stats.total_period += sample->period;
6559 return 0;
6660 }
6761
68−static struct perf_diff diff = {
69− .tool = {
70− .sample = diff__process_sample_event,
71− .mmap = perf_event__process_mmap,
72− .comm = perf_event__process_comm,
73− .exit = perf_event__process_task,
74− .fork = perf_event__process_task,
75− .lost = perf_event__process_lost,
76− .ordered_samples = true,
77− .ordering_requires_timestamps = true,
78− },
62+static struct perf_tool tool = {
63+ .sample = diff__process_sample_event,
64+ .mmap = perf_event__process_mmap,
65+ .comm = perf_event__process_comm,
66+ .exit = perf_event__process_task,
67+ .fork = perf_event__process_task,
68+ .lost = perf_event__process_lost,
69+ .ordered_samples = true,
70+ .ordering_requires_timestamps = true,
7971 };
8072
8173 static void perf_session__insert_hist_entry_by_name(struct rb_root *root,
@@ -146,34 +138,71 @@ static void hists__match(struct hists *older, struct hists *newer) · 64 unmodified lines
146138 }
147139 }
148140
141+static struct perf_evsel *evsel_match(struct perf_evsel *evsel,
142+ struct perf_evlist *evlist)
143+{
144+ struct perf_evsel *e;
145+
146+ list_for_each_entry(e, &evlist->entries, node)
147+ if (perf_evsel__match2(evsel, e))
148+ return e;
149+
150+ return NULL;
151+}
152+
149153 static int __cmd_diff(void)
150154 {
151155 int ret, i;
152156 #define older (session[0])
153157 #define newer (session[1])
154158 struct perf_session *session[2];
159+ struct perf_evlist *evlist_new, *evlist_old;
160+ struct perf_evsel *evsel;
161+ bool first = true;
155162
156163 older = perf_session__new(input_old, O_RDONLY, force, false,
157− &diff.tool);
164+ &tool);
158165 newer = perf_session__new(input_new, O_RDONLY, force, false,
159− &diff.tool);
166+ &tool);
160167 if (session[0] == NULL || session[1] == NULL)
161168 return -ENOMEM;
162169
163170 for (i = 0; i < 2; ++i) {
164− diff.session = session[i];
165− ret = perf_session__process_events(session[i], &diff.tool);
171+ ret = perf_session__process_events(session[i], &tool);
166172 if (ret)
167173 goto out_delete;
168− hists__output_resort(&session[i]->hists);
169174 }
170175
171− if (show_displacement)
172− hists__resort_entries(&older->hists);
176+ evlist_old = older->evlist;
177+ evlist_new = newer->evlist;
178+
179+ list_for_each_entry(evsel, &evlist_new->entries, node)
180+ hists__output_resort(&evsel->hists);
181+
182+ list_for_each_entry(evsel, &evlist_old->entries, node) {
183+ hists__output_resort(&evsel->hists);
184+
185+ if (show_displacement)
186+ hists__resort_entries(&evsel->hists);
187+ }
188+
189+ list_for_each_entry(evsel, &evlist_new->entries, node) {
190+ struct perf_evsel *evsel_old;
191+
192+ evsel_old = evsel_match(evsel, evlist_old);
193+ if (!evsel_old)
194+ continue;
195+
196+ fprintf(stdout, "%s# Event '%s'\n#\n", first ? "" : "\n",
197+ perf_evsel__name(evsel));
198+
199+ first = false;
200+
201+ hists__match(&evsel_old->hists, &evsel->hists);
202+ hists__fprintf(&evsel->hists, &evsel_old->hists,
203+ show_displacement, true, 0, 0, stdout);
204+ }
173205
174− hists__match(&older->hists, &newer->hists);
175− hists__fprintf(&newer->hists, &older->hists,
176− show_displacement, true, 0, 0, stdout);
177206 out_delete:
178207 for (i = 0; i < 2; ++i)
179208 perf_session__delete(session[i]);
@@ -213,7 +242,7 @@ static const struct option options[] = { · 33 unmodified lines
213242 OPT_END()
214243 };
215244
216−int cmd_diff(int argc, const char **argv, const char *prefix __used)
245+int cmd_diff(int argc, const char **argv, const char *prefix __maybe_unused)
217246 {
218247 sort_order = diff__default_sort_order;
219248 argc = parse_options(argc, argv, options, diff_usage, 0);
@@ -235,6 +264,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix __used) · 15 unmodified lines
235264 if (symbol__init() < 0)
236265 return -1;
237266
267+ perf_hpp__init(true, show_displacement);
238268 setup_sorting(diff_usage, options);
239269 setup_pager();
240270
tools/perf/builtin-evlist.c+1−1
@@ -113,7 +113,7 @@ static const char * const evlist_usage[] = { · 112 unmodified lines
113113 NULL
114114 };
115115
116−int cmd_evlist(int argc, const char **argv, const char *prefix __used)
116+int cmd_evlist(int argc, const char **argv, const char *prefix __maybe_unused)
117117 {
118118 struct perf_attr_details details = { .verbose = false, };
119119 const char *input_name = NULL;
tools/perf/builtin-help.c+35−15
@@ -24,13 +24,14 @@ static struct man_viewer_info_list { · 23 unmodified lines
2424 } *man_viewer_info_list;
2525
2626 enum help_format {
27+ HELP_FORMAT_NONE,
2728 HELP_FORMAT_MAN,
2829 HELP_FORMAT_INFO,
2930 HELP_FORMAT_WEB,
3031 };
3132
3233 static bool show_all = false;
33−static enum help_format help_format = HELP_FORMAT_MAN;
34+static enum help_format help_format = HELP_FORMAT_NONE;
3435 static struct option builtin_help_options[] = {
3536 OPT_BOOLEAN('a', "all", &show_all, "print all available commands"),
3637 OPT_SET_UINT('m', "man", &help_format, "show man page", HELP_FORMAT_MAN),
@@ -54,7 +55,9 @@ static enum help_format parse_help_format(const char *format) · 17 unmodified lines
5455 return HELP_FORMAT_INFO;
5556 if (!strcmp(format, "web") || !strcmp(format, "html"))
5657 return HELP_FORMAT_WEB;
57− die("unrecognized help format '%s'", format);
58+
59+ pr_err("unrecognized help format '%s'", format);
60+ return HELP_FORMAT_NONE;
5861 }
5962
6063 static const char *get_man_viewer_info(const char *name)
@@ -259,6 +262,8 @@ static int perf_help_config(const char *var, const char *value, void *cb) · 198 unmodified lines
259262 if (!value)
260263 return config_error_nonbool(var);
261264 help_format = parse_help_format(value);
265+ if (help_format == HELP_FORMAT_NONE)
266+ return -1;
262267 return 0;
263268 }
264269 if (!strcmp(var, "man.viewer")) {
@@ -352,7 +357,7 @@ static void exec_viewer(const char *name, const char *page) · 87 unmodified lines
352357 warning("'%s': unknown man viewer.", name);
353358 }
354359
355−static void show_man_page(const char *perf_cmd)
360+static int show_man_page(const char *perf_cmd)
356361 {
357362 struct man_viewer_list *viewer;
358363 const char *page = cmd_to_page(perf_cmd);
@@ -365,28 +370,35 @@ static void show_man_page(const char *perf_cmd) · 6 unmodified lines
365370 if (fallback)
366371 exec_viewer(fallback, page);
367372 exec_viewer("man", page);
368− die("no man viewer handled the request");
373+
374+ pr_err("no man viewer handled the request");
375+ return -1;
369376 }
370377
371−static void show_info_page(const char *perf_cmd)
378+static int show_info_page(const char *perf_cmd)
372379 {
373380 const char *page = cmd_to_page(perf_cmd);
374381 setenv("INFOPATH", system_path(PERF_INFO_PATH), 1);
375382 execlp("info", "info", "perfman", page, NULL);
383+ return -1;
376384 }
377385
378−static void get_html_page_path(struct strbuf *page_path, const char *page)
386+static int get_html_page_path(struct strbuf *page_path, const char *page)
379387 {
380388 struct stat st;
381389 const char *html_path = system_path(PERF_HTML_PATH);
382390
383391 /* Check that we have a perf documentation directory. */
384392 if (stat(mkpath("%s/perf.html", html_path), &st)
385− || !S_ISREG(st.st_mode))
386− die("'%s': not a documentation directory.", html_path);
393+ || !S_ISREG(st.st_mode)) {
394+ pr_err("'%s': not a documentation directory.", html_path);
395+ return -1;
396+ }
387397
388398 strbuf_init(page_path, 0);
389399 strbuf_addf(page_path, "%s/%s.html", html_path, page);
400+
401+ return 0;
390402 }
391403
392404 /*
@@ -401,19 +413,23 @@ static void open_html(const char *path) · 8 unmodified lines
401413 }
402414 #endif
403415
404−static void show_html_page(const char *perf_cmd)
416+static int show_html_page(const char *perf_cmd)
405417 {
406418 const char *page = cmd_to_page(perf_cmd);
407419 struct strbuf page_path; /* it leaks but we exec bellow */
408420
409− get_html_page_path(&page_path, page);
421+ if (get_html_page_path(&page_path, page) != 0)
422+ return -1;
410423
411424 open_html(page_path.buf);
425+
426+ return 0;
412427 }
413428
414−int cmd_help(int argc, const char **argv, const char *prefix __used)
429+int cmd_help(int argc, const char **argv, const char *prefix __maybe_unused)
415430 {
416431 const char *alias;
432+ int rc = 0;
417433
418434 load_command_list("perf-", &main_cmds, &other_cmds);
419435
@@ -444,16 +460,20 @@ int cmd_help(int argc, const char **argv, const char *prefix __used) · 24 unmodified lines
444460
445461 switch (help_format) {
446462 case HELP_FORMAT_MAN:
447− show_man_page(argv[0]);
463+ rc = show_man_page(argv[0]);
448464 break;
449465 case HELP_FORMAT_INFO:
450− show_info_page(argv[0]);
466+ rc = show_info_page(argv[0]);
451467 break;
452468 case HELP_FORMAT_WEB:
453− show_html_page(argv[0]);
469+ rc = show_html_page(argv[0]);
470+ break;
471+ case HELP_FORMAT_NONE:
472+ /* fall-through */
454473 default:
474+ rc = -1;
455475 break;
456476 }
457477
458− return 0;
478+ return rc;
459479 }
tools/perf/builtin-inject.c+17−12
@@ -17,9 +17,9 @@ · 16 unmodified lines
1717 static char const *input_name = "-";
1818 static bool inject_build_ids;
1919
20−static int perf_event__repipe_synth(struct perf_tool *tool __used,
20+static int perf_event__repipe_synth(struct perf_tool *tool __maybe_unused,
2121 union perf_event *event,
22− struct machine *machine __used)
22+ struct machine *machine __maybe_unused)
2323 {
2424 uint32_t size;
2525 void *buf = event;
@@ -40,7 +40,8 @@ static int perf_event__repipe_synth(struct perf_tool *tool __used, · 14 unmodified lines
4040
4141 static int perf_event__repipe_op2_synth(struct perf_tool *tool,
4242 union perf_event *event,
43− struct perf_session *session __used)
43+ struct perf_session *session
44+ __maybe_unused)
4445 {
4546 return perf_event__repipe_synth(tool, event, NULL);
4647 }
@@ -52,13 +53,14 @@ static int perf_event__repipe_event_type_synth(struct perf_tool *tool, · 5 unmodified lines
5253 }
5354
5455 static int perf_event__repipe_tracing_data_synth(union perf_event *event,
55− struct perf_session *session __used)
56+ struct perf_session *session
57+ __maybe_unused)
5658 {
5759 return perf_event__repipe_synth(NULL, event, NULL);
5860 }
5961
6062 static int perf_event__repipe_attr(union perf_event *event,
61− struct perf_evlist **pevlist __used)
63+ struct perf_evlist **pevlist __maybe_unused)
6264 {
6365 int ret;
6466 ret = perf_event__process_attr(event, pevlist);
@@ -70,7 +72,7 @@ static int perf_event__repipe_attr(union perf_event *event, · 5 unmodified lines
7072
7173 static int perf_event__repipe(struct perf_tool *tool,
7274 union perf_event *event,
73− struct perf_sample *sample __used,
75+ struct perf_sample *sample __maybe_unused,
7476 struct machine *machine)
7577 {
7678 return perf_event__repipe_synth(tool, event, machine);
@@ -78,8 +80,8 @@ static int perf_event__repipe(struct perf_tool *tool, · 1 unmodified line
7880
7981 static int perf_event__repipe_sample(struct perf_tool *tool,
8082 union perf_event *event,
81− struct perf_sample *sample __used,
82− struct perf_evsel *evsel __used,
83+ struct perf_sample *sample __maybe_unused,
84+ struct perf_evsel *evsel __maybe_unused,
8385 struct machine *machine)
8486 {
8587 return perf_event__repipe_synth(tool, event, machine);
@@ -163,7 +165,7 @@ static int dso__inject_build_id(struct dso *self, struct perf_tool *tool, · 77 unmodified lines
163165 static int perf_event__inject_buildid(struct perf_tool *tool,
164166 union perf_event *event,
165167 struct perf_sample *sample,
166− struct perf_evsel *evsel __used,
168+ struct perf_evsel *evsel __maybe_unused,
167169 struct machine *machine)
168170 {
169171 struct addr_location al;
@@ -191,10 +193,13 @@ static int perf_event__inject_buildid(struct perf_tool *tool, · 21 unmodified lines
191193 * If this fails, too bad, let the other side
192194 * account this as unresolved.
193195 */
194− } else
196+ } else {
197+#ifndef NO_LIBELF_SUPPORT
195198 pr_warning("no symbols found in %s, maybe "
196199 "install a debug package?\n",
197200 al.map->dso->long_name);
201+#endif
202+ }
198203 }
199204 }
200205
@@ -221,7 +226,7 @@ struct perf_tool perf_inject = { · 20 unmodified lines
221226
222227 extern volatile int session_done;
223228
224−static void sig_handler(int sig __attribute__((__unused__)))
229+static void sig_handler(int sig __maybe_unused)
225230 {
226231 session_done = 1;
227232 }
@@ -264,7 +269,7 @@ static const struct option options[] = { · 36 unmodified lines
264269 OPT_END()
265270 };
266271
267−int cmd_inject(int argc, const char **argv, const char *prefix __used)
272+int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
268273 {
269274 argc = parse_options(argc, argv, options, report_usage, 0);
270275
tools/perf/builtin-kmem.c+120−114
@@ -1,6 +1,8 @@
11 #include "builtin.h"
22 #include "perf.h"
33
4+#include "util/evlist.h"
5+#include "util/evsel.h"
46 #include "util/util.h"
57 #include "util/cache.h"
68 #include "util/symbol.h"
@@ -57,46 +59,52 @@ static unsigned long nr_allocs, nr_cross_allocs; · 50 unmodified lines
5759
5860 #define PATH_SYS_NODE "/sys/devices/system/node"
5961
60−struct perf_kmem {
61− struct perf_tool tool;
62− struct perf_session *session;
63−};
64−
65−static void init_cpunode_map(void)
62+static int init_cpunode_map(void)
6663 {
6764 FILE *fp;
68− int i;
65+ int i, err = -1;
6966
7067 fp = fopen("/sys/devices/system/cpu/kernel_max", "r");
7168 if (!fp) {
7269 max_cpu_num = 4096;
73− return;
70+ return 0;
71+ }
72+
73+ if (fscanf(fp, "%d", &max_cpu_num) < 1) {
74+ pr_err("Failed to read 'kernel_max' from sysfs");
75+ goto out_close;
7476 }
7577
76− if (fscanf(fp, "%d", &max_cpu_num) < 1)
77− die("Failed to read 'kernel_max' from sysfs");
7878 max_cpu_num++;
7979
8080 cpunode_map = calloc(max_cpu_num, sizeof(int));
81− if (!cpunode_map)
82− die("calloc");
81+ if (!cpunode_map) {
82+ pr_err("%s: calloc failed\n", __func__);
83+ goto out_close;
84+ }
85+
8386 for (i = 0; i < max_cpu_num; i++)
8487 cpunode_map[i] = -1;
88+
89+ err = 0;
90+out_close:
8591 fclose(fp);
92+ return err;
8693 }
8794
88−static void setup_cpunode_map(void)
95+static int setup_cpunode_map(void)
8996 {
9097 struct dirent *dent1, *dent2;
9198 DIR *dir1, *dir2;
9299 unsigned int cpu, mem;
93100 char buf[PATH_MAX];
94101
95− init_cpunode_map();
102+ if (init_cpunode_map())
103+ return -1;
96104
97105 dir1 = opendir(PATH_SYS_NODE);
98106 if (!dir1)
99− return;
107+ return -1;
100108
101109 while ((dent1 = readdir(dir1)) != NULL) {
102110 if (dent1->d_type != DT_DIR ||
@@ -116,10 +124,11 @@ static void setup_cpunode_map(void) · 13 unmodified lines
116124 closedir(dir2);
117125 }
118126 closedir(dir1);
127+ return 0;
119128 }
120129
121−static void insert_alloc_stat(unsigned long call_site, unsigned long ptr,
122− int bytes_req, int bytes_alloc, int cpu)
130+static int insert_alloc_stat(unsigned long call_site, unsigned long ptr,
131+ int bytes_req, int bytes_alloc, int cpu)
123132 {
124133 struct rb_node **node = &root_alloc_stat.rb_node;
125134 struct rb_node *parent = NULL;
@@ -143,8 +152,10 @@ static void insert_alloc_stat(unsigned long call_site, unsigned long ptr, · 17 unmodified lines
143152 data->bytes_alloc += bytes_alloc;
144153 } else {
145154 data = malloc(sizeof(*data));
146− if (!data)
147− die("malloc");
155+ if (!data) {
156+ pr_err("%s: malloc failed\n", __func__);
157+ return -1;
158+ }
148159 data->ptr = ptr;
149160 data->pingpong = 0;
150161 data->hit = 1;
@@ -156,9 +167,10 @@ static void insert_alloc_stat(unsigned long call_site, unsigned long ptr, · 5 unmodified lines
156167 }
157168 data->call_site = call_site;
158169 data->alloc_cpu = cpu;
170+ return 0;
159171 }
160172
161−static void insert_caller_stat(unsigned long call_site,
173+static int insert_caller_stat(unsigned long call_site,
162174 int bytes_req, int bytes_alloc)
163175 {
164176 struct rb_node **node = &root_caller_stat.rb_node;
@@ -183,8 +195,10 @@ static void insert_caller_stat(unsigned long call_site, · 18 unmodified lines
183195 data->bytes_alloc += bytes_alloc;
184196 } else {
185197 data = malloc(sizeof(*data));
186− if (!data)
187− die("malloc");
198+ if (!data) {
199+ pr_err("%s: malloc failed\n", __func__);
200+ return -1;
201+ }
188202 data->call_site = call_site;
189203 data->pingpong = 0;
190204 data->hit = 1;
@@ -194,39 +208,43 @@ static void insert_caller_stat(unsigned long call_site, · 3 unmodified lines
194208 rb_link_node(&data->node, parent, node);
195209 rb_insert_color(&data->node, &root_caller_stat);
196210 }
211+
212+ return 0;
197213 }
198214
199−static void process_alloc_event(void *data,
200− struct event_format *event,
201− int cpu,
202− u64 timestamp __used,
203− struct thread *thread __used,
204− int node)
215+static int perf_evsel__process_alloc_event(struct perf_evsel *evsel,
216+ struct perf_sample *sample)
205217 {
206− unsigned long call_site;
207− unsigned long ptr;
208− int bytes_req;
209− int bytes_alloc;
210− int node1, node2;
211−
212− ptr = raw_field_value(event, "ptr", data);
213− call_site = raw_field_value(event, "call_site", data);
214− bytes_req = raw_field_value(event, "bytes_req", data);
215− bytes_alloc = raw_field_value(event, "bytes_alloc", data);
218+ unsigned long ptr = perf_evsel__intval(evsel, sample, "ptr"),
219+ call_site = perf_evsel__intval(evsel, sample, "call_site");
220+ int bytes_req = perf_evsel__intval(evsel, sample, "bytes_req"),
221+ bytes_alloc = perf_evsel__intval(evsel, sample, "bytes_alloc");
216222
217− insert_alloc_stat(call_site, ptr, bytes_req, bytes_alloc, cpu);
218− insert_caller_stat(call_site, bytes_req, bytes_alloc);
223+ if (insert_alloc_stat(call_site, ptr, bytes_req, bytes_alloc, sample->cpu) ||
224+ insert_caller_stat(call_site, bytes_req, bytes_alloc))
225+ return -1;
219226
220227 total_requested += bytes_req;
221228 total_allocated += bytes_alloc;
222229
223− if (node) {
224− node1 = cpunode_map[cpu];
225− node2 = raw_field_value(event, "node", data);
230+ nr_allocs++;
231+ return 0;
232+}
233+
234+static int perf_evsel__process_alloc_node_event(struct perf_evsel *evsel,
235+ struct perf_sample *sample)
236+{
237+ int ret = perf_evsel__process_alloc_event(evsel, sample);
238+
239+ if (!ret) {
240+ int node1 = cpunode_map[sample->cpu],
241+ node2 = perf_evsel__intval(evsel, sample, "node");
242+
226243 if (node1 != node2)
227244 nr_cross_allocs++;
228245 }
229− nr_allocs++;
246+
247+ return ret;
230248 }
231249
232250 static int ptr_cmp(struct alloc_stat *, struct alloc_stat *);
@@ -257,66 +275,37 @@ static struct alloc_stat *search_alloc_stat(unsigned long ptr, · 24 unmodified lines
257275 return NULL;
258276 }
259277
260−static void process_free_event(void *data,
261− struct event_format *event,
262− int cpu,
263− u64 timestamp __used,
264− struct thread *thread __used)
278+static int perf_evsel__process_free_event(struct perf_evsel *evsel,
279+ struct perf_sample *sample)
265280 {
266− unsigned long ptr;
281+ unsigned long ptr = perf_evsel__intval(evsel, sample, "ptr");
267282 struct alloc_stat *s_alloc, *s_caller;
268283
269− ptr = raw_field_value(event, "ptr", data);
270−
271284 s_alloc = search_alloc_stat(ptr, 0, &root_alloc_stat, ptr_cmp);
272285 if (!s_alloc)
273− return;
286+ return 0;
274287
275− if (cpu != s_alloc->alloc_cpu) {
288+ if ((short)sample->cpu != s_alloc->alloc_cpu) {
276289 s_alloc->pingpong++;
277290
278291 s_caller = search_alloc_stat(0, s_alloc->call_site,
279292 &root_caller_stat, callsite_cmp);
280− assert(s_caller);
293+ if (!s_caller)
294+ return -1;
281295 s_caller->pingpong++;
282296 }
283297 s_alloc->alloc_cpu = -1;
284−}
285298
286−static void process_raw_event(struct perf_tool *tool,
287− union perf_event *raw_event __used, void *data,
288− int cpu, u64 timestamp, struct thread *thread)
289−{
290− struct perf_kmem *kmem = container_of(tool, struct perf_kmem, tool);
291− struct event_format *event;
292− int type;
293−
294− type = trace_parse_common_type(kmem->session->pevent, data);
295− event = pevent_find_event(kmem->session->pevent, type);
296−
297− if (!strcmp(event->name, "kmalloc") ||
298− !strcmp(event->name, "kmem_cache_alloc")) {
299− process_alloc_event(data, event, cpu, timestamp, thread, 0);
300− return;
301− }
302−
303− if (!strcmp(event->name, "kmalloc_node") ||
304− !strcmp(event->name, "kmem_cache_alloc_node")) {
305− process_alloc_event(data, event, cpu, timestamp, thread, 1);
306− return;
307− }
308−
309− if (!strcmp(event->name, "kfree") ||
310− !strcmp(event->name, "kmem_cache_free")) {
311− process_free_event(data, event, cpu, timestamp, thread);
312− return;
313− }
299+ return 0;
314300 }
315301
316−static int process_sample_event(struct perf_tool *tool,
302+typedef int (*tracepoint_handler)(struct perf_evsel *evsel,
303+ struct perf_sample *sample);
304+
305+static int process_sample_event(struct perf_tool *tool __maybe_unused,
317306 union perf_event *event,
318307 struct perf_sample *sample,
319− struct perf_evsel *evsel __used,
308+ struct perf_evsel *evsel,
320309 struct machine *machine)
321310 {
322311 struct thread *thread = machine__findnew_thread(machine, event->ip.pid);
@@ -329,18 +318,18 @@ static int process_sample_event(struct perf_tool *tool, · 6 unmodified lines
329318
330319 dump_printf(" ... thread: %s:%d\n", thread->comm, thread->pid);
331320
332− process_raw_event(tool, event, sample->raw_data, sample->cpu,
333− sample->time, thread);
321+ if (evsel->handler.func != NULL) {
322+ tracepoint_handler f = evsel->handler.func;
323+ return f(evsel, sample);
324+ }
334325
335326 return 0;
336327 }
337328
338−static struct perf_kmem perf_kmem = {
339− .tool = {
340− .sample = process_sample_event,
341− .comm = perf_event__process_comm,
342− .ordered_samples = true,
343− },
329+static struct perf_tool perf_kmem = {
330+ .sample = process_sample_event,
331+ .comm = perf_event__process_comm,
332+ .ordered_samples = true,
344333 };
345334
346335 static double fragmentation(unsigned long n_req, unsigned long n_alloc)
@@ -496,22 +485,32 @@ static int __cmd_kmem(void) · 149 unmodified lines
496485 {
497486 int err = -EINVAL;
498487 struct perf_session *session;
499−
500− session = perf_session__new(input_name, O_RDONLY, 0, false,
501− &perf_kmem.tool);
488+ const struct perf_evsel_str_handler kmem_tracepoints[] = {
489+ { "kmem:kmalloc", perf_evsel__process_alloc_event, },
490+ { "kmem:kmem_cache_alloc", perf_evsel__process_alloc_event, },
491+ { "kmem:kmalloc_node", perf_evsel__process_alloc_node_event, },
492+ { "kmem:kmem_cache_alloc_node", perf_evsel__process_alloc_node_event, },
493+ { "kmem:kfree", perf_evsel__process_free_event, },
494+ { "kmem:kmem_cache_free", perf_evsel__process_free_event, },
495+ };
496+
497+ session = perf_session__new(input_name, O_RDONLY, 0, false, &perf_kmem);
502498 if (session == NULL)
503499 return -ENOMEM;
504500
505− perf_kmem.session = session;
506−
507501 if (perf_session__create_kernel_maps(session) < 0)
508502 goto out_delete;
509503
510504 if (!perf_session__has_traces(session, "kmem record"))
511505 goto out_delete;
512506
507+ if (perf_session__set_tracepoints_handlers(session, kmem_tracepoints)) {
508+ pr_err("Initializing perf session tracepoint handlers failed\n");
509+ return -1;
510+ }
511+
513512 setup_pager();
514− err = perf_session__process_events(session, &perf_kmem.tool);
513+ err = perf_session__process_events(session, &perf_kmem);
515514 if (err != 0)
516515 goto out_delete;
517516 sort_result();
@@ -635,8 +634,10 @@ static int sort_dimension__add(const char *tok, struct list_head *list) · 117 unmodified lines
635634 for (i = 0; i < NUM_AVAIL_SORTS; i++) {
636635 if (!strcmp(avail_sorts[i]->name, tok)) {
637636 sort = malloc(sizeof(*sort));
638− if (!sort)
639− die("malloc");
637+ if (!sort) {
638+ pr_err("%s: malloc failed\n", __func__);
639+ return -1;
640+ }
640641 memcpy(sort, avail_sorts[i], sizeof(*sort));
641642 list_add_tail(&sort->list, list);
642643 return 0;
@@ -651,8 +652,10 @@ static int setup_sorting(struct list_head *sort_list, const char *arg) · 8 unmodified lines
651652 char *tok;
652653 char *str = strdup(arg);
653654
654− if (!str)
655− die("strdup");
655+ if (!str) {
656+ pr_err("%s: strdup failed\n", __func__);
657+ return -1;
658+ }
656659
657660 while (true) {
658661 tok = strsep(&str, ",");
@@ -669,8 +672,8 @@ static int setup_sorting(struct list_head *sort_list, const char *arg) · 10 unmodified lines
669672 return 0;
670673 }
671674
672−static int parse_sort_opt(const struct option *opt __used,
673− const char *arg, int unset __used)
675+static int parse_sort_opt(const struct option *opt __maybe_unused,
676+ const char *arg, int unset __maybe_unused)
674677 {
675678 if (!arg)
676679 return -1;
@@ -683,22 +686,24 @@ static int parse_sort_opt(const struct option *opt __used, · 6 unmodified lines
683686 return 0;
684687 }
685688
686−static int parse_caller_opt(const struct option *opt __used,
687− const char *arg __used, int unset __used)
689+static int parse_caller_opt(const struct option *opt __maybe_unused,
690+ const char *arg __maybe_unused,
691+ int unset __maybe_unused)
688692 {
689693 caller_flag = (alloc_flag + 1);
690694 return 0;
691695 }
692696
693−static int parse_alloc_opt(const struct option *opt __used,
694− const char *arg __used, int unset __used)
697+static int parse_alloc_opt(const struct option *opt __maybe_unused,
698+ const char *arg __maybe_unused,
699+ int unset __maybe_unused)
695700 {
696701 alloc_flag = (caller_flag + 1);
697702 return 0;
698703 }
699704
700−static int parse_line_opt(const struct option *opt __used,
701− const char *arg, int unset __used)
705+static int parse_line_opt(const struct option *opt __maybe_unused,
706+ const char *arg, int unset __maybe_unused)
702707 {
703708 int lines;
704709
@@ -768,7 +773,7 @@ static int __cmd_record(int argc, const char **argv) · 63 unmodified lines
768773 return cmd_record(i, rec_argv, NULL);
769774 }
770775
771−int cmd_kmem(int argc, const char **argv, const char *prefix __used)
776+int cmd_kmem(int argc, const char **argv, const char *prefix __maybe_unused)
772777 {
773778 argc = parse_options(argc, argv, kmem_options, kmem_usage, 0);
774779
@@ -780,7 +785,8 @@ int cmd_kmem(int argc, const char **argv, const char *prefix __used) · 5 unmodified lines
780785 if (!strncmp(argv[0], "rec", 3)) {
781786 return __cmd_record(argc, argv);
782787 } else if (!strcmp(argv[0], "stat")) {
783− setup_cpunode_map();
788+ if (setup_cpunode_map())
789+ return -1;
784790
785791 if (list_empty(&caller_sort))
786792 setup_sorting(&caller_sort, default_sort_order);
tools/perf/builtin-kvm.c+834−4
@@ -1,6 +1,7 @@
11 #include "builtin.h"
22 #include "perf.h"
33
4+#include "util/evsel.h"
45 #include "util/util.h"
56 #include "util/cache.h"
67 #include "util/symbol.h"
@@ -10,8 +11,10 @@ · 3 unmodified lines
1011
1112 #include "util/parse-options.h"
1213 #include "util/trace-event.h"
13−
1414 #include "util/debug.h"
15+#include "util/debugfs.h"
16+#include "util/tool.h"
17+#include "util/stat.h"
1518
1619 #include <sys/prctl.h>
1720
@@ -19,11 +22,836 @@ · 1 unmodified line
1922 #include <pthread.h>
2023 #include <math.h>
2124
22−static const char *file_name;
25+#include "../../arch/x86/include/asm/svm.h"
26+#include "../../arch/x86/include/asm/vmx.h"
27+#include "../../arch/x86/include/asm/kvm.h"
28+
29+struct event_key {
30+ #define INVALID_KEY (~0ULL)
31+ u64 key;
32+ int info;
33+};
34+
35+struct kvm_events_ops {
36+ bool (*is_begin_event)(struct perf_evsel *evsel,
37+ struct perf_sample *sample,
38+ struct event_key *key);
39+ bool (*is_end_event)(struct perf_evsel *evsel,
40+ struct perf_sample *sample, struct event_key *key);
41+ void (*decode_key)(struct event_key *key, char decode[20]);
42+ const char *name;
43+};
44+
45+static void exit_event_get_key(struct perf_evsel *evsel,
46+ struct perf_sample *sample,
47+ struct event_key *key)
48+{
49+ key->info = 0;
50+ key->key = perf_evsel__intval(evsel, sample, "exit_reason");
51+}
52+
53+static bool kvm_exit_event(struct perf_evsel *evsel)
54+{
55+ return !strcmp(evsel->name, "kvm:kvm_exit");
56+}
57+
58+static bool exit_event_begin(struct perf_evsel *evsel,
59+ struct perf_sample *sample, struct event_key *key)
60+{
61+ if (kvm_exit_event(evsel)) {
62+ exit_event_get_key(evsel, sample, key);
63+ return true;
64+ }
65+
66+ return false;
67+}
68+
69+static bool kvm_entry_event(struct perf_evsel *evsel)
70+{
71+ return !strcmp(evsel->name, "kvm:kvm_entry");
72+}
73+
74+static bool exit_event_end(struct perf_evsel *evsel,
75+ struct perf_sample *sample __maybe_unused,
76+ struct event_key *key __maybe_unused)
77+{
78+ return kvm_entry_event(evsel);
79+}
80+
81+struct exit_reasons_table {
82+ unsigned long exit_code;
83+ const char *reason;
84+};
85+
86+struct exit_reasons_table vmx_exit_reasons[] = {
87+ VMX_EXIT_REASONS
88+};
89+
90+struct exit_reasons_table svm_exit_reasons[] = {
91+ SVM_EXIT_REASONS
92+};
93+
94+static int cpu_isa;
95+
96+static const char *get_exit_reason(u64 exit_code)
97+{
98+ int table_size = ARRAY_SIZE(svm_exit_reasons);
99+ struct exit_reasons_table *table = svm_exit_reasons;
100+
101+ if (cpu_isa == 1) {
102+ table = vmx_exit_reasons;
103+ table_size = ARRAY_SIZE(vmx_exit_reasons);
104+ }
105+
106+ while (table_size--) {
107+ if (table->exit_code == exit_code)
108+ return table->reason;
109+ table++;
110+ }
111+
112+ pr_err("unknown kvm exit code:%lld on %s\n",
113+ (unsigned long long)exit_code, cpu_isa ? "VMX" : "SVM");
114+ return "UNKNOWN";
115+}
116+
117+static void exit_event_decode_key(struct event_key *key, char decode[20])
118+{
119+ const char *exit_reason = get_exit_reason(key->key);
120+
121+ scnprintf(decode, 20, "%s", exit_reason);
122+}
123+
124+static struct kvm_events_ops exit_events = {
125+ .is_begin_event = exit_event_begin,
126+ .is_end_event = exit_event_end,
127+ .decode_key = exit_event_decode_key,
128+ .name = "VM-EXIT"
129+};
130+
131+ /*
132+ * For the mmio events, we treat:
133+ * the time of MMIO write: kvm_mmio(KVM_TRACE_MMIO_WRITE...) -> kvm_entry
134+ * the time of MMIO read: kvm_exit -> kvm_mmio(KVM_TRACE_MMIO_READ...).
135+ */
136+static void mmio_event_get_key(struct perf_evsel *evsel, struct perf_sample *sample,
137+ struct event_key *key)
138+{
139+ key->key = perf_evsel__intval(evsel, sample, "gpa");
140+ key->info = perf_evsel__intval(evsel, sample, "type");
141+}
142+
143+#define KVM_TRACE_MMIO_READ_UNSATISFIED 0
144+#define KVM_TRACE_MMIO_READ 1
145+#define KVM_TRACE_MMIO_WRITE 2
146+
147+static bool mmio_event_begin(struct perf_evsel *evsel,
148+ struct perf_sample *sample, struct event_key *key)
149+{
150+ /* MMIO read begin event in kernel. */
151+ if (kvm_exit_event(evsel))
152+ return true;
153+
154+ /* MMIO write begin event in kernel. */
155+ if (!strcmp(evsel->name, "kvm:kvm_mmio") &&
156+ perf_evsel__intval(evsel, sample, "type") == KVM_TRACE_MMIO_WRITE) {
157+ mmio_event_get_key(evsel, sample, key);
158+ return true;
159+ }
160+
161+ return false;
162+}
163+
164+static bool mmio_event_end(struct perf_evsel *evsel, struct perf_sample *sample,
165+ struct event_key *key)
166+{
167+ /* MMIO write end event in kernel. */
168+ if (kvm_entry_event(evsel))
169+ return true;
170+
171+ /* MMIO read end event in kernel.*/
172+ if (!strcmp(evsel->name, "kvm:kvm_mmio") &&
173+ perf_evsel__intval(evsel, sample, "type") == KVM_TRACE_MMIO_READ) {
174+ mmio_event_get_key(evsel, sample, key);
175+ return true;
176+ }
177+
178+ return false;
179+}
180+
181+static void mmio_event_decode_key(struct event_key *key, char decode[20])
182+{
183+ scnprintf(decode, 20, "%#lx:%s", (unsigned long)key->key,
184+ key->info == KVM_TRACE_MMIO_WRITE ? "W" : "R");
185+}
186+
187+static struct kvm_events_ops mmio_events = {
188+ .is_begin_event = mmio_event_begin,
189+ .is_end_event = mmio_event_end,
190+ .decode_key = mmio_event_decode_key,
191+ .name = "MMIO Access"
192+};
193+
194+ /* The time of emulation pio access is from kvm_pio to kvm_entry. */
195+static void ioport_event_get_key(struct perf_evsel *evsel,
196+ struct perf_sample *sample,
197+ struct event_key *key)
198+{
199+ key->key = perf_evsel__intval(evsel, sample, "port");
200+ key->info = perf_evsel__intval(evsel, sample, "rw");
201+}
202+
203+static bool ioport_event_begin(struct perf_evsel *evsel,
204+ struct perf_sample *sample,
205+ struct event_key *key)
206+{
207+ if (!strcmp(evsel->name, "kvm:kvm_pio")) {
208+ ioport_event_get_key(evsel, sample, key);
209+ return true;
210+ }
211+
212+ return false;
213+}
214+
215+static bool ioport_event_end(struct perf_evsel *evsel,
216+ struct perf_sample *sample __maybe_unused,
217+ struct event_key *key __maybe_unused)
218+{
219+ return kvm_entry_event(evsel);
220+}
221+
222+static void ioport_event_decode_key(struct event_key *key, char decode[20])
223+{
224+ scnprintf(decode, 20, "%#llx:%s", (unsigned long long)key->key,
225+ key->info ? "POUT" : "PIN");
226+}
227+
228+static struct kvm_events_ops ioport_events = {
229+ .is_begin_event = ioport_event_begin,
230+ .is_end_event = ioport_event_end,
231+ .decode_key = ioport_event_decode_key,
232+ .name = "IO Port Access"
233+};
234+
235+static const char *report_event = "vmexit";
236+struct kvm_events_ops *events_ops;
237+
238+static bool register_kvm_events_ops(void)
239+{
240+ bool ret = true;
241+
242+ if (!strcmp(report_event, "vmexit"))
243+ events_ops = &exit_events;
244+ else if (!strcmp(report_event, "mmio"))
245+ events_ops = &mmio_events;
246+ else if (!strcmp(report_event, "ioport"))
247+ events_ops = &ioport_events;
248+ else {
249+ pr_err("Unknown report event:%s\n", report_event);
250+ ret = false;
251+ }
252+
253+ return ret;
254+}
255+
256+struct kvm_event_stats {
257+ u64 time;
258+ struct stats stats;
259+};
260+
261+struct kvm_event {
262+ struct list_head hash_entry;
263+ struct rb_node rb;
264+
265+ struct event_key key;
266+
267+ struct kvm_event_stats total;
268+
269+ #define DEFAULT_VCPU_NUM 8
270+ int max_vcpu;
271+ struct kvm_event_stats *vcpu;
272+};
273+
274+struct vcpu_event_record {
275+ int vcpu_id;
276+ u64 start_time;
277+ struct kvm_event *last_event;
278+};
279+
280+#define EVENTS_BITS 12
281+#define EVENTS_CACHE_SIZE (1UL << EVENTS_BITS)
282+
283+static u64 total_time;
284+static u64 total_count;
285+static struct list_head kvm_events_cache[EVENTS_CACHE_SIZE];
286+
287+static void init_kvm_event_record(void)
288+{
289+ int i;
290+
291+ for (i = 0; i < (int)EVENTS_CACHE_SIZE; i++)
292+ INIT_LIST_HEAD(&kvm_events_cache[i]);
293+}
294+
295+static int kvm_events_hash_fn(u64 key)
296+{
297+ return key & (EVENTS_CACHE_SIZE - 1);
298+}
299+
300+static bool kvm_event_expand(struct kvm_event *event, int vcpu_id)
301+{
302+ int old_max_vcpu = event->max_vcpu;
303+
304+ if (vcpu_id < event->max_vcpu)
305+ return true;
306+
307+ while (event->max_vcpu <= vcpu_id)
308+ event->max_vcpu += DEFAULT_VCPU_NUM;
309+
310+ event->vcpu = realloc(event->vcpu,
311+ event->max_vcpu * sizeof(*event->vcpu));
312+ if (!event->vcpu) {
313+ pr_err("Not enough memory\n");
314+ return false;
315+ }
316+
317+ memset(event->vcpu + old_max_vcpu, 0,
318+ (event->max_vcpu - old_max_vcpu) * sizeof(*event->vcpu));
319+ return true;
320+}
321+
322+static struct kvm_event *kvm_alloc_init_event(struct event_key *key)
323+{
324+ struct kvm_event *event;
325+
326+ event = zalloc(sizeof(*event));
327+ if (!event) {
328+ pr_err("Not enough memory\n");
329+ return NULL;
330+ }
331+
332+ event->key = *key;
333+ return event;
334+}
335+
336+static struct kvm_event *find_create_kvm_event(struct event_key *key)
337+{
338+ struct kvm_event *event;
339+ struct list_head *head;
340+
341+ BUG_ON(key->key == INVALID_KEY);
342+
343+ head = &kvm_events_cache[kvm_events_hash_fn(key->key)];
344+ list_for_each_entry(event, head, hash_entry)
345+ if (event->key.key == key->key && event->key.info == key->info)
346+ return event;
347+
348+ event = kvm_alloc_init_event(key);
349+ if (!event)
350+ return NULL;
351+
352+ list_add(&event->hash_entry, head);
353+ return event;
354+}
355+
356+static bool handle_begin_event(struct vcpu_event_record *vcpu_record,
357+ struct event_key *key, u64 timestamp)
358+{
359+ struct kvm_event *event = NULL;
360+
361+ if (key->key != INVALID_KEY)
362+ event = find_create_kvm_event(key);
363+
364+ vcpu_record->last_event = event;
365+ vcpu_record->start_time = timestamp;
366+ return true;
367+}
368+
369+static void
370+kvm_update_event_stats(struct kvm_event_stats *kvm_stats, u64 time_diff)
371+{
372+ kvm_stats->time += time_diff;
373+ update_stats(&kvm_stats->stats, time_diff);
374+}
375+
376+static double kvm_event_rel_stddev(int vcpu_id, struct kvm_event *event)
377+{
378+ struct kvm_event_stats *kvm_stats = &event->total;
379+
380+ if (vcpu_id != -1)
381+ kvm_stats = &event->vcpu[vcpu_id];
382+
383+ return rel_stddev_stats(stddev_stats(&kvm_stats->stats),
384+ avg_stats(&kvm_stats->stats));
385+}
386+
387+static bool update_kvm_event(struct kvm_event *event, int vcpu_id,
388+ u64 time_diff)
389+{
390+ kvm_update_event_stats(&event->total, time_diff);
391+
392+ if (!kvm_event_expand(event, vcpu_id))
393+ return false;
394+
395+ kvm_update_event_stats(&event->vcpu[vcpu_id], time_diff);
396+ return true;
397+}
398+
399+static bool handle_end_event(struct vcpu_event_record *vcpu_record,
400+ struct event_key *key, u64 timestamp)
401+{
402+ struct kvm_event *event;
403+ u64 time_begin, time_diff;
404+
405+ event = vcpu_record->last_event;
406+ time_begin = vcpu_record->start_time;
407+
408+ /* The begin event is not caught. */
409+ if (!time_begin)
410+ return true;
411+
412+ /*
413+ * In some case, the 'begin event' only records the start timestamp,
414+ * the actual event is recognized in the 'end event' (e.g. mmio-event).
415+ */
416+
417+ /* Both begin and end events did not get the key. */
418+ if (!event && key->key == INVALID_KEY)
419+ return true;
420+
421+ if (!event)
422+ event = find_create_kvm_event(key);
423+
424+ if (!event)
425+ return false;
426+
427+ vcpu_record->last_event = NULL;
428+ vcpu_record->start_time = 0;
429+
430+ BUG_ON(timestamp < time_begin);
431+
432+ time_diff = timestamp - time_begin;
433+ return update_kvm_event(event, vcpu_record->vcpu_id, time_diff);
434+}
435+
436+static
437+struct vcpu_event_record *per_vcpu_record(struct thread *thread,
438+ struct perf_evsel *evsel,
439+ struct perf_sample *sample)
440+{
441+ /* Only kvm_entry records vcpu id. */
442+ if (!thread->priv && kvm_entry_event(evsel)) {
443+ struct vcpu_event_record *vcpu_record;
444+
445+ vcpu_record = zalloc(sizeof(*vcpu_record));
446+ if (!vcpu_record) {
447+ pr_err("%s: Not enough memory\n", __func__);
448+ return NULL;
449+ }
450+
451+ vcpu_record->vcpu_id = perf_evsel__intval(evsel, sample, "vcpu_id");
452+ thread->priv = vcpu_record;
453+ }
454+
455+ return thread->priv;
456+}
457+
458+static bool handle_kvm_event(struct thread *thread, struct perf_evsel *evsel,
459+ struct perf_sample *sample)
460+{
461+ struct vcpu_event_record *vcpu_record;
462+ struct event_key key = {.key = INVALID_KEY};
463+
464+ vcpu_record = per_vcpu_record(thread, evsel, sample);
465+ if (!vcpu_record)
466+ return true;
467+
468+ if (events_ops->is_begin_event(evsel, sample, &key))
469+ return handle_begin_event(vcpu_record, &key, sample->time);
470+
471+ if (events_ops->is_end_event(evsel, sample, &key))
472+ return handle_end_event(vcpu_record, &key, sample->time);
473+
474+ return true;
475+}
476+
477+typedef int (*key_cmp_fun)(struct kvm_event*, struct kvm_event*, int);
478+struct kvm_event_key {
479+ const char *name;
480+ key_cmp_fun key;
481+};
482+
483+static int trace_vcpu = -1;
484+#define GET_EVENT_KEY(func, field) \
485+static u64 get_event_ ##func(struct kvm_event *event, int vcpu) \
486+{ \
487+ if (vcpu == -1) \
488+ return event->total.field; \
489+ \
490+ if (vcpu >= event->max_vcpu) \
491+ return 0; \
492+ \
493+ return event->vcpu[vcpu].field; \
494+}
495+
496+#define COMPARE_EVENT_KEY(func, field) \
497+GET_EVENT_KEY(func, field) \
498+static int compare_kvm_event_ ## func(struct kvm_event *one, \
499+ struct kvm_event *two, int vcpu)\
500+{ \
501+ return get_event_ ##func(one, vcpu) > \
502+ get_event_ ##func(two, vcpu); \
503+}
504+
505+GET_EVENT_KEY(time, time);
506+COMPARE_EVENT_KEY(count, stats.n);
507+COMPARE_EVENT_KEY(mean, stats.mean);
508+
509+#define DEF_SORT_NAME_KEY(name, compare_key) \
510+ { #name, compare_kvm_event_ ## compare_key }
511+
512+static struct kvm_event_key keys[] = {
513+ DEF_SORT_NAME_KEY(sample, count),
514+ DEF_SORT_NAME_KEY(time, mean),
515+ { NULL, NULL }
516+};
517+
518+static const char *sort_key = "sample";
519+static key_cmp_fun compare;
520+
521+static bool select_key(void)
522+{
523+ int i;
524+
525+ for (i = 0; keys[i].name; i++) {
526+ if (!strcmp(keys[i].name, sort_key)) {
527+ compare = keys[i].key;
528+ return true;
529+ }
530+ }
531+
532+ pr_err("Unknown compare key:%s\n", sort_key);
533+ return false;
534+}
535+
536+static struct rb_root result;
537+static void insert_to_result(struct kvm_event *event, key_cmp_fun bigger,
538+ int vcpu)
539+{
540+ struct rb_node **rb = &result.rb_node;
541+ struct rb_node *parent = NULL;
542+ struct kvm_event *p;
543+
544+ while (*rb) {
545+ p = container_of(*rb, struct kvm_event, rb);
546+ parent = *rb;
547+
548+ if (bigger(event, p, vcpu))
549+ rb = &(*rb)->rb_left;
550+ else
551+ rb = &(*rb)->rb_right;
552+ }
553+
554+ rb_link_node(&event->rb, parent, rb);
555+ rb_insert_color(&event->rb, &result);
556+}
557+
558+static void update_total_count(struct kvm_event *event, int vcpu)
559+{
560+ total_count += get_event_count(event, vcpu);
561+ total_time += get_event_time(event, vcpu);
562+}
563+
564+static bool event_is_valid(struct kvm_event *event, int vcpu)
565+{
566+ return !!get_event_count(event, vcpu);
567+}
568+
569+static void sort_result(int vcpu)
570+{
571+ unsigned int i;
572+ struct kvm_event *event;
573+
574+ for (i = 0; i < EVENTS_CACHE_SIZE; i++)
575+ list_for_each_entry(event, &kvm_events_cache[i], hash_entry)
576+ if (event_is_valid(event, vcpu)) {
577+ update_total_count(event, vcpu);
578+ insert_to_result(event, compare, vcpu);
579+ }
580+}
581+
582+/* returns left most element of result, and erase it */
583+static struct kvm_event *pop_from_result(void)
584+{
585+ struct rb_node *node = rb_first(&result);
586+
587+ if (!node)
588+ return NULL;
589+
590+ rb_erase(node, &result);
591+ return container_of(node, struct kvm_event, rb);
592+}
593+
594+static void print_vcpu_info(int vcpu)
595+{
596+ pr_info("Analyze events for ");
597+
598+ if (vcpu == -1)
599+ pr_info("all VCPUs:\n\n");
600+ else
601+ pr_info("VCPU %d:\n\n", vcpu);
602+}
603+
604+static void print_result(int vcpu)
605+{
606+ char decode[20];
607+ struct kvm_event *event;
608+
609+ pr_info("\n\n");
610+ print_vcpu_info(vcpu);
611+ pr_info("%20s ", events_ops->name);
612+ pr_info("%10s ", "Samples");
613+ pr_info("%9s ", "Samples%");
614+
615+ pr_info("%9s ", "Time%");
616+ pr_info("%16s ", "Avg time");
617+ pr_info("\n\n");
618+
619+ while ((event = pop_from_result())) {
620+ u64 ecount, etime;
621+
622+ ecount = get_event_count(event, vcpu);
623+ etime = get_event_time(event, vcpu);
624+
625+ events_ops->decode_key(&event->key, decode);
626+ pr_info("%20s ", decode);
627+ pr_info("%10llu ", (unsigned long long)ecount);
628+ pr_info("%8.2f%% ", (double)ecount / total_count * 100);
629+ pr_info("%8.2f%% ", (double)etime / total_time * 100);
630+ pr_info("%9.2fus ( +-%7.2f%% )", (double)etime / ecount/1e3,
631+ kvm_event_rel_stddev(vcpu, event));
632+ pr_info("\n");
633+ }
634+
635+ pr_info("\nTotal Samples:%lld, Total events handled time:%.2fus.\n\n",
636+ (unsigned long long)total_count, total_time / 1e3);
637+}
638+
639+static int process_sample_event(struct perf_tool *tool __maybe_unused,
640+ union perf_event *event,
641+ struct perf_sample *sample,
642+ struct perf_evsel *evsel,
643+ struct machine *machine)
644+{
645+ struct thread *thread = machine__findnew_thread(machine, sample->tid);
646+
647+ if (thread == NULL) {
648+ pr_debug("problem processing %d event, skipping it.\n",
649+ event->header.type);
650+ return -1;
651+ }
652+
653+ if (!handle_kvm_event(thread, evsel, sample))
654+ return -1;
655+
656+ return 0;
657+}
658+
659+static struct perf_tool eops = {
660+ .sample = process_sample_event,
661+ .comm = perf_event__process_comm,
662+ .ordered_samples = true,
663+};
664+
665+static int get_cpu_isa(struct perf_session *session)
666+{
667+ char *cpuid = session->header.env.cpuid;
668+ int isa;
669+
670+ if (strstr(cpuid, "Intel"))
671+ isa = 1;
672+ else if (strstr(cpuid, "AMD"))
673+ isa = 0;
674+ else {
675+ pr_err("CPU %s is not supported.\n", cpuid);
676+ isa = -ENOTSUP;
677+ }
678+
679+ return isa;
680+}
681+
682+static const char *file_name;
683+
684+static int read_events(void)
685+{
686+ struct perf_session *kvm_session;
687+ int ret;
688+
689+ kvm_session = perf_session__new(file_name, O_RDONLY, 0, false, &eops);
690+ if (!kvm_session) {
691+ pr_err("Initializing perf session failed\n");
692+ return -EINVAL;
693+ }
694+
695+ if (!perf_session__has_traces(kvm_session, "kvm record"))
696+ return -EINVAL;
697+
698+ /*
699+ * Do not use 'isa' recorded in kvm_exit tracepoint since it is not
700+ * traced in the old kernel.
701+ */
702+ ret = get_cpu_isa(kvm_session);
703+
704+ if (ret < 0)
705+ return ret;
706+
707+ cpu_isa = ret;
708+
709+ return perf_session__process_events(kvm_session, &eops);
710+}
711+
712+static bool verify_vcpu(int vcpu)
713+{
714+ if (vcpu != -1 && vcpu < 0) {
715+ pr_err("Invalid vcpu:%d.\n", vcpu);
716+ return false;
717+ }
718+
719+ return true;
720+}
721+
722+static int kvm_events_report_vcpu(int vcpu)
723+{
724+ int ret = -EINVAL;
725+
726+ if (!verify_vcpu(vcpu))
727+ goto exit;
728+
729+ if (!select_key())
730+ goto exit;
731+
732+ if (!register_kvm_events_ops())
733+ goto exit;
734+
735+ init_kvm_event_record();
736+ setup_pager();
737+
738+ ret = read_events();
739+ if (ret)
740+ goto exit;
741+
742+ sort_result(vcpu);
743+ print_result(vcpu);
744+exit:
745+ return ret;
746+}
747+
748+static const char * const record_args[] = {
749+ "record",
750+ "-R",
751+ "-f",
752+ "-m", "1024",
753+ "-c", "1",
754+ "-e", "kvm:kvm_entry",
755+ "-e", "kvm:kvm_exit",
756+ "-e", "kvm:kvm_mmio",
757+ "-e", "kvm:kvm_pio",
758+};
759+
760+#define STRDUP_FAIL_EXIT(s) \
761+ ({ char *_p; \
762+ _p = strdup(s); \
763+ if (!_p) \
764+ return -ENOMEM; \
765+ _p; \
766+ })
767+
768+static int kvm_events_record(int argc, const char **argv)
769+{
770+ unsigned int rec_argc, i, j;
771+ const char **rec_argv;
772+
773+ rec_argc = ARRAY_SIZE(record_args) + argc + 2;
774+ rec_argv = calloc(rec_argc + 1, sizeof(char *));
775+
776+ if (rec_argv == NULL)
777+ return -ENOMEM;
778+
779+ for (i = 0; i < ARRAY_SIZE(record_args); i++)
780+ rec_argv[i] = STRDUP_FAIL_EXIT(record_args[i]);
781+
782+ rec_argv[i++] = STRDUP_FAIL_EXIT("-o");
783+ rec_argv[i++] = STRDUP_FAIL_EXIT(file_name);
784+
785+ for (j = 1; j < (unsigned int)argc; j++, i++)
786+ rec_argv[i] = argv[j];
787+
788+ return cmd_record(i, rec_argv, NULL);
789+}
790+
791+static const char * const kvm_events_report_usage[] = {
792+ "perf kvm stat report [<options>]",
793+ NULL
794+};
795+
796+static const struct option kvm_events_report_options[] = {
797+ OPT_STRING(0, "event", &report_event, "report event",
798+ "event for reporting: vmexit, mmio, ioport"),
799+ OPT_INTEGER(0, "vcpu", &trace_vcpu,
800+ "vcpu id to report"),
801+ OPT_STRING('k', "key", &sort_key, "sort-key",
802+ "key for sorting: sample(sort by samples number)"
803+ " time (sort by avg time)"),
804+ OPT_END()
805+};
806+
807+static int kvm_events_report(int argc, const char **argv)
808+{
809+ symbol__init();
810+
811+ if (argc) {
812+ argc = parse_options(argc, argv,
813+ kvm_events_report_options,
814+ kvm_events_report_usage, 0);
815+ if (argc)
816+ usage_with_options(kvm_events_report_usage,
817+ kvm_events_report_options);
818+ }
819+
820+ return kvm_events_report_vcpu(trace_vcpu);
821+}
822+
823+static void print_kvm_stat_usage(void)
824+{
825+ printf("Usage: perf kvm stat <command>\n\n");
826+
827+ printf("# Available commands:\n");
828+ printf("\trecord: record kvm events\n");
829+ printf("\treport: report statistical data of kvm events\n");
830+
831+ printf("\nOtherwise, it is the alias of 'perf stat':\n");
832+}
833+
834+static int kvm_cmd_stat(int argc, const char **argv)
835+{
836+ if (argc == 1) {
837+ print_kvm_stat_usage();
838+ goto perf_stat;
839+ }
840+
841+ if (!strncmp(argv[1], "rec", 3))
842+ return kvm_events_record(argc - 1, argv + 1);
843+
844+ if (!strncmp(argv[1], "rep", 3))
845+ return kvm_events_report(argc - 1 , argv + 1);
846+
847+perf_stat:
848+ return cmd_stat(argc, argv, NULL);
849+}
850+
23851 static char name_buffer[256];
24852
25853 static const char * const kvm_usage[] = {
26− "perf kvm [<options>] {top|record|report|diff|buildid-list}",
854+ "perf kvm [<options>] {top|record|report|diff|buildid-list|stat}",
27855 NULL
28856 };
29857
@@ -102,7 +930,7 @@ static int __cmd_buildid_list(int argc, const char **argv) · 72 unmodified lines
102930 return cmd_buildid_list(i, rec_argv, NULL);
103931 }
104932
105−int cmd_kvm(int argc, const char **argv, const char *prefix __used)
933+int cmd_kvm(int argc, const char **argv, const char *prefix __maybe_unused)
106934 {
107935 perf_host = 0;
108936 perf_guest = 1;
@@ -135,6 +963,8 @@ int cmd_kvm(int argc, const char **argv, const char *prefix __used) · 26 unmodified lines
135963 return cmd_top(argc, argv, NULL);
136964 else if (!strncmp(argv[0], "buildid-list", 12))
137965 return __cmd_buildid_list(argc, argv);
966+ else if (!strncmp(argv[0], "stat", 4))
967+ return kvm_cmd_stat(argc, argv);
138968 else
139969 usage_with_options(kvm_usage, kvm_options);
140970
tools/perf/builtin-list.c+9−7
@@ -14,20 +14,20 @@ · 13 unmodified lines
1414 #include "util/parse-events.h"
1515 #include "util/cache.h"
1616
17−int cmd_list(int argc, const char **argv, const char *prefix __used)
17+int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
1818 {
1919 setup_pager();
2020
2121 if (argc == 1)
22− print_events(NULL);
22+ print_events(NULL, false);
2323 else {
2424 int i;
2525
2626 for (i = 1; i < argc; ++i) {
27− if (i > 1)
27+ if (i > 2)
2828 putchar('\n');
2929 if (strncmp(argv[i], "tracepoint", 10) == 0)
30− print_tracepoint_events(NULL, NULL);
30+ print_tracepoint_events(NULL, NULL, false);
3131 else if (strcmp(argv[i], "hw") == 0 ||
3232 strcmp(argv[i], "hardware") == 0)
3333 print_events_type(PERF_TYPE_HARDWARE);
@@ -36,13 +36,15 @@ int cmd_list(int argc, const char **argv, const char *prefix __used) · 2 unmodified lines
3636 print_events_type(PERF_TYPE_SOFTWARE);
3737 else if (strcmp(argv[i], "cache") == 0 ||
3838 strcmp(argv[i], "hwcache") == 0)
39− print_hwcache_events(NULL);
39+ print_hwcache_events(NULL, false);
40+ else if (strcmp(argv[i], "--raw-dump") == 0)
41+ print_events(NULL, true);
4042 else {
4143 char *sep = strchr(argv[i], ':'), *s;
4244 int sep_idx;
4345
4446 if (sep == NULL) {
45− print_events(argv[i]);
47+ print_events(argv[i], false);
4648 continue;
4749 }
4850 sep_idx = sep - argv[i];
@@ -51,7 +53,7 @@ int cmd_list(int argc, const char **argv, const char *prefix __used) · 2 unmodified lines
5153 return -1;
5254
5355 s[sep_idx] = '\0';
54− print_tracepoint_events(s, s + sep_idx + 1);
56+ print_tracepoint_events(s, s + sep_idx + 1, false);
5557 free(s);
5658 }
5759 }
tools/perf/builtin-lock.c+203−211
@@ -1,6 +1,8 @@
11 #include "builtin.h"
22 #include "perf.h"
33
4+#include "util/evlist.h"
5+#include "util/evsel.h"
46 #include "util/util.h"
57 #include "util/cache.h"
68 #include "util/symbol.h"
@@ -40,7 +42,7 @@ struct lock_stat { · 33 unmodified lines
4042 struct rb_node rb; /* used for sorting */
4143
4244 /*
43− * FIXME: raw_field_value() returns unsigned long long,
45+ * FIXME: perf_evsel__intval() returns u64,
4446 * so address of lockdep_map should be dealed as 64bit.
4547 * Is there more better solution?
4648 */
@@ -160,8 +162,10 @@ static struct thread_stat *thread_stat_findnew_after_first(u32 tid) · 113 unmodified lines
160162 return st;
161163
162164 st = zalloc(sizeof(struct thread_stat));
163− if (!st)
164− die("memory allocation failed\n");
165+ if (!st) {
166+ pr_err("memory allocation failed\n");
167+ return NULL;
168+ }
165169
166170 st->tid = tid;
167171 INIT_LIST_HEAD(&st->seq_list);
@@ -180,8 +184,10 @@ static struct thread_stat *thread_stat_findnew_first(u32 tid) · 12 unmodified lines
180184 struct thread_stat *st;
181185
182186 st = zalloc(sizeof(struct thread_stat));
183− if (!st)
184− die("memory allocation failed\n");
187+ if (!st) {
188+ pr_err("memory allocation failed\n");
189+ return NULL;
190+ }
185191 st->tid = tid;
186192 INIT_LIST_HEAD(&st->seq_list);
187193
@@ -247,18 +253,20 @@ struct lock_key keys[] = { · 59 unmodified lines
247253 { NULL, NULL }
248254 };
249255
250−static void select_key(void)
256+static int select_key(void)
251257 {
252258 int i;
253259
254260 for (i = 0; keys[i].name; i++) {
255261 if (!strcmp(keys[i].name, sort_key)) {
256262 compare = keys[i].key;
257− return;
263+ return 0;
258264 }
259265 }
260266
261− die("Unknown compare key:%s\n", sort_key);
267+ pr_err("Unknown compare key: %s\n", sort_key);
268+
269+ return -1;
262270 }
263271
264272 static void insert_to_result(struct lock_stat *st,
@@ -323,61 +331,24 @@ static struct lock_stat *lock_stat_findnew(void *addr, const char *name) · 58 unmodified lines
323331 return new;
324332
325333 alloc_failed:
326− die("memory allocation failed\n");
334+ pr_err("memory allocation failed\n");
335+ return NULL;
327336 }
328337
329338 static const char *input_name;
330339
331−struct raw_event_sample {
332− u32 size;
333− char data[0];
334−};
335−
336−struct trace_acquire_event {
337− void *addr;
338− const char *name;
339− int flag;
340−};
341−
342−struct trace_acquired_event {
343− void *addr;
344− const char *name;
345−};
340+struct trace_lock_handler {
341+ int (*acquire_event)(struct perf_evsel *evsel,
342+ struct perf_sample *sample);
346343
347−struct trace_contended_event {
348− void *addr;
349− const char *name;
350−};
344+ int (*acquired_event)(struct perf_evsel *evsel,
345+ struct perf_sample *sample);
351346
352−struct trace_release_event {
353− void *addr;
354− const char *name;
355−};
347+ int (*contended_event)(struct perf_evsel *evsel,
348+ struct perf_sample *sample);
356349
357−struct trace_lock_handler {
358− void (*acquire_event)(struct trace_acquire_event *,
359− struct event_format *,
360− int cpu,
361− u64 timestamp,
362− struct thread *thread);
363−
364− void (*acquired_event)(struct trace_acquired_event *,
365− struct event_format *,
366− int cpu,
367− u64 timestamp,
368− struct thread *thread);
369−
370− void (*contended_event)(struct trace_contended_event *,
371− struct event_format *,
372− int cpu,
373− u64 timestamp,
374− struct thread *thread);
375−
376− void (*release_event)(struct trace_release_event *,
377− struct event_format *,
378− int cpu,
379− u64 timestamp,
380− struct thread *thread);
350+ int (*release_event)(struct perf_evsel *evsel,
351+ struct perf_sample *sample);
381352 };
382353
383354 static struct lock_seq_stat *get_seq(struct thread_stat *ts, void *addr)
@@ -390,8 +361,10 @@ static struct lock_seq_stat *get_seq(struct thread_stat *ts, void *addr) · 6 unmodified lines
390361 }
391362
392363 seq = zalloc(sizeof(struct lock_seq_stat));
393− if (!seq)
394− die("Not enough memory\n");
364+ if (!seq) {
365+ pr_err("memory allocation failed\n");
366+ return NULL;
367+ }
395368 seq->state = SEQ_STATE_UNINITIALIZED;
396369 seq->addr = addr;
397370
@@ -414,33 +387,42 @@ enum acquire_flags { · 16 unmodified lines
414387 READ_LOCK = 2,
415388 };
416389
417−static void
418−report_lock_acquire_event(struct trace_acquire_event *acquire_event,
419− struct event_format *__event __used,
420− int cpu __used,
421− u64 timestamp __used,
422− struct thread *thread __used)
390+static int report_lock_acquire_event(struct perf_evsel *evsel,
391+ struct perf_sample *sample)
423392 {
393+ void *addr;
424394 struct lock_stat *ls;
425395 struct thread_stat *ts;
426396 struct lock_seq_stat *seq;
397+ const char *name = perf_evsel__strval(evsel, sample, "name");
398+ u64 tmp = perf_evsel__intval(evsel, sample, "lockdep_addr");
399+ int flag = perf_evsel__intval(evsel, sample, "flag");
400+
401+ memcpy(&addr, &tmp, sizeof(void *));
427402
428− ls = lock_stat_findnew(acquire_event->addr, acquire_event->name);
403+ ls = lock_stat_findnew(addr, name);
404+ if (!ls)
405+ return -1;
429406 if (ls->discard)
430− return;
407+ return 0;
431408
432− ts = thread_stat_findnew(thread->pid);
433− seq = get_seq(ts, acquire_event->addr);
409+ ts = thread_stat_findnew(sample->tid);
410+ if (!ts)
411+ return -1;
412+
413+ seq = get_seq(ts, addr);
414+ if (!seq)
415+ return -1;
434416
435417 switch (seq->state) {
436418 case SEQ_STATE_UNINITIALIZED:
437419 case SEQ_STATE_RELEASED:
438− if (!acquire_event->flag) {
420+ if (!flag) {
439421 seq->state = SEQ_STATE_ACQUIRING;
440422 } else {
441− if (acquire_event->flag & TRY_LOCK)
423+ if (flag & TRY_LOCK)
442424 ls->nr_trylock++;
443− if (acquire_event->flag & READ_LOCK)
425+ if (flag & READ_LOCK)
444426 ls->nr_readlock++;
445427 seq->state = SEQ_STATE_READ_ACQUIRED;
446428 seq->read_count = 1;
@@ -448,7 +430,7 @@ report_lock_acquire_event(struct trace_acquire_event *acquire_event, · 1 unmodified line
448430 }
449431 break;
450432 case SEQ_STATE_READ_ACQUIRED:
451− if (acquire_event->flag & READ_LOCK) {
433+ if (flag & READ_LOCK) {
452434 seq->read_count++;
453435 ls->nr_acquired++;
454436 goto end;
@@ -473,38 +455,46 @@ broken: · 18 unmodified lines
473455 }
474456
475457 ls->nr_acquire++;
476− seq->prev_event_time = timestamp;
458+ seq->prev_event_time = sample->time;
477459 end:
478− return;
460+ return 0;
479461 }
480462
481−static void
482−report_lock_acquired_event(struct trace_acquired_event *acquired_event,
483− struct event_format *__event __used,
484− int cpu __used,
485− u64 timestamp __used,
486− struct thread *thread __used)
463+static int report_lock_acquired_event(struct perf_evsel *evsel,
464+ struct perf_sample *sample)
487465 {
466+ void *addr;
488467 struct lock_stat *ls;
489468 struct thread_stat *ts;
490469 struct lock_seq_stat *seq;
491470 u64 contended_term;
471+ const char *name = perf_evsel__strval(evsel, sample, "name");
472+ u64 tmp = perf_evsel__intval(evsel, sample, "lockdep_addr");
473+
474+ memcpy(&addr, &tmp, sizeof(void *));
492475
493− ls = lock_stat_findnew(acquired_event->addr, acquired_event->name);
476+ ls = lock_stat_findnew(addr, name);
477+ if (!ls)
478+ return -1;
494479 if (ls->discard)
495− return;
480+ return 0;
481+
482+ ts = thread_stat_findnew(sample->tid);
483+ if (!ts)
484+ return -1;
496485
497− ts = thread_stat_findnew(thread->pid);
498− seq = get_seq(ts, acquired_event->addr);
486+ seq = get_seq(ts, addr);
487+ if (!seq)
488+ return -1;
499489
500490 switch (seq->state) {
501491 case SEQ_STATE_UNINITIALIZED:
502492 /* orphan event, do nothing */
503− return;
493+ return 0;
504494 case SEQ_STATE_ACQUIRING:
505495 break;
506496 case SEQ_STATE_CONTENDED:
507− contended_term = timestamp - seq->prev_event_time;
497+ contended_term = sample->time - seq->prev_event_time;
508498 ls->wait_time_total += contended_term;
509499 if (contended_term < ls->wait_time_min)
510500 ls->wait_time_min = contended_term;
@@ -529,33 +519,41 @@ report_lock_acquired_event(struct trace_acquired_event *acquired_event, · 18 unmodified lines
529519
530520 seq->state = SEQ_STATE_ACQUIRED;
531521 ls->nr_acquired++;
532− seq->prev_event_time = timestamp;
522+ seq->prev_event_time = sample->time;
533523 end:
534− return;
524+ return 0;
535525 }
536526
537−static void
538−report_lock_contended_event(struct trace_contended_event *contended_event,
539− struct event_format *__event __used,
540− int cpu __used,
541− u64 timestamp __used,
542− struct thread *thread __used)
527+static int report_lock_contended_event(struct perf_evsel *evsel,
528+ struct perf_sample *sample)
543529 {
530+ void *addr;
544531 struct lock_stat *ls;
545532 struct thread_stat *ts;
546533 struct lock_seq_stat *seq;
534+ const char *name = perf_evsel__strval(evsel, sample, "name");
535+ u64 tmp = perf_evsel__intval(evsel, sample, "lockdep_addr");
547536
548− ls = lock_stat_findnew(contended_event->addr, contended_event->name);
537+ memcpy(&addr, &tmp, sizeof(void *));
538+
539+ ls = lock_stat_findnew(addr, name);
540+ if (!ls)
541+ return -1;
549542 if (ls->discard)
550− return;
543+ return 0;
544+
545+ ts = thread_stat_findnew(sample->tid);
546+ if (!ts)
547+ return -1;
551548
552− ts = thread_stat_findnew(thread->pid);
553− seq = get_seq(ts, contended_event->addr);
549+ seq = get_seq(ts, addr);
550+ if (!seq)
551+ return -1;
554552
555553 switch (seq->state) {
556554 case SEQ_STATE_UNINITIALIZED:
557555 /* orphan event, do nothing */
558− return;
556+ return 0;
559557 case SEQ_STATE_ACQUIRING:
560558 break;
561559 case SEQ_STATE_RELEASED:
@@ -576,28 +574,36 @@ report_lock_contended_event(struct trace_contended_event *contended_event, · 14 unmodified lines
576574
577575 seq->state = SEQ_STATE_CONTENDED;
578576 ls->nr_contended++;
579− seq->prev_event_time = timestamp;
577+ seq->prev_event_time = sample->time;
580578 end:
581− return;
579+ return 0;
582580 }
583581
584−static void
585−report_lock_release_event(struct trace_release_event *release_event,
586− struct event_format *__event __used,
587− int cpu __used,
588− u64 timestamp __used,
589− struct thread *thread __used)
582+static int report_lock_release_event(struct perf_evsel *evsel,
583+ struct perf_sample *sample)
590584 {
585+ void *addr;
591586 struct lock_stat *ls;
592587 struct thread_stat *ts;
593588 struct lock_seq_stat *seq;
589+ const char *name = perf_evsel__strval(evsel, sample, "name");
590+ u64 tmp = perf_evsel__intval(evsel, sample, "lockdep_addr");
591+
592+ memcpy(&addr, &tmp, sizeof(void *));
594593
595− ls = lock_stat_findnew(release_event->addr, release_event->name);
594+ ls = lock_stat_findnew(addr, name);
595+ if (!ls)
596+ return -1;
596597 if (ls->discard)
597− return;
598+ return 0;
599+
600+ ts = thread_stat_findnew(sample->tid);
601+ if (!ts)
602+ return -1;
598603
599− ts = thread_stat_findnew(thread->pid);
600− seq = get_seq(ts, release_event->addr);
604+ seq = get_seq(ts, addr);
605+ if (!seq)
606+ return -1;
601607
602608 switch (seq->state) {
603609 case SEQ_STATE_UNINITIALIZED:
@@ -631,7 +637,7 @@ free_seq: · 27 unmodified lines
631637 list_del(&seq->list);
632638 free(seq);
633639 end:
634− return;
640+ return 0;
635641 }
636642
637643 /* lock oriented handlers */
@@ -645,96 +651,36 @@ static struct trace_lock_handler report_lock_ops = { · 7 unmodified lines
645651
646652 static struct trace_lock_handler *trace_handler;
647653
648−static void
649−process_lock_acquire_event(void *data,
650− struct event_format *event __used,
651− int cpu __used,
652− u64 timestamp __used,
653− struct thread *thread __used)
654−{
655− struct trace_acquire_event acquire_event;
656− u64 tmp; /* this is required for casting... */
657−
658− tmp = raw_field_value(event, "lockdep_addr", data);
659− memcpy(&acquire_event.addr, &tmp, sizeof(void *));
660− acquire_event.name = (char *)raw_field_ptr(event, "name", data);
661− acquire_event.flag = (int)raw_field_value(event, "flag", data);
662−
663− if (trace_handler->acquire_event)
664− trace_handler->acquire_event(&acquire_event, event, cpu, timestamp, thread);
665−}
666−
667−static void
668−process_lock_acquired_event(void *data,
669− struct event_format *event __used,
670− int cpu __used,
671− u64 timestamp __used,
672− struct thread *thread __used)
654+static int perf_evsel__process_lock_acquire(struct perf_evsel *evsel,
655+ struct perf_sample *sample)
673656 {
674− struct trace_acquired_event acquired_event;
675− u64 tmp; /* this is required for casting... */
676−
677− tmp = raw_field_value(event, "lockdep_addr", data);
678− memcpy(&acquired_event.addr, &tmp, sizeof(void *));
679− acquired_event.name = (char *)raw_field_ptr(event, "name", data);
680−
681657 if (trace_handler->acquire_event)
682− trace_handler->acquired_event(&acquired_event, event, cpu, timestamp, thread);
658+ return trace_handler->acquire_event(evsel, sample);
659+ return 0;
683660 }
684661
685−static void
686−process_lock_contended_event(void *data,
687− struct event_format *event __used,
688− int cpu __used,
689− u64 timestamp __used,
690− struct thread *thread __used)
662+static int perf_evsel__process_lock_acquired(struct perf_evsel *evsel,
663+ struct perf_sample *sample)
691664 {
692− struct trace_contended_event contended_event;
693− u64 tmp; /* this is required for casting... */
694−
695− tmp = raw_field_value(event, "lockdep_addr", data);
696− memcpy(&contended_event.addr, &tmp, sizeof(void *));
697− contended_event.name = (char *)raw_field_ptr(event, "name", data);
698−
699− if (trace_handler->acquire_event)
700− trace_handler->contended_event(&contended_event, event, cpu, timestamp, thread);
665+ if (trace_handler->acquired_event)
666+ return trace_handler->acquired_event(evsel, sample);
667+ return 0;
701668 }
702669
703−static void
704−process_lock_release_event(void *data,
705− struct event_format *event __used,
706− int cpu __used,
707− u64 timestamp __used,
708− struct thread *thread __used)
670+static int perf_evsel__process_lock_contended(struct perf_evsel *evsel,
671+ struct perf_sample *sample)
709672 {
710− struct trace_release_event release_event;
711− u64 tmp; /* this is required for casting... */
712−
713− tmp = raw_field_value(event, "lockdep_addr", data);
714− memcpy(&release_event.addr, &tmp, sizeof(void *));
715− release_event.name = (char *)raw_field_ptr(event, "name", data);
716−
717− if (trace_handler->acquire_event)
718− trace_handler->release_event(&release_event, event, cpu, timestamp, thread);
673+ if (trace_handler->contended_event)
674+ return trace_handler->contended_event(evsel, sample);
675+ return 0;
719676 }
720677
721−static void
722−process_raw_event(void *data, int cpu, u64 timestamp, struct thread *thread)
678+static int perf_evsel__process_lock_release(struct perf_evsel *evsel,
679+ struct perf_sample *sample)
723680 {
724− struct event_format *event;
725− int type;
726−
727− type = trace_parse_common_type(session->pevent, data);
728− event = pevent_find_event(session->pevent, type);
729−
730− if (!strcmp(event->name, "lock_acquire"))
731− process_lock_acquire_event(data, event, cpu, timestamp, thread);
732− if (!strcmp(event->name, "lock_acquired"))
733− process_lock_acquired_event(data, event, cpu, timestamp, thread);
734− if (!strcmp(event->name, "lock_contended"))
735− process_lock_contended_event(data, event, cpu, timestamp, thread);
736− if (!strcmp(event->name, "lock_release"))
737− process_lock_release_event(data, event, cpu, timestamp, thread);
681+ if (trace_handler->release_event)
682+ return trace_handler->release_event(evsel, sample);
683+ return 0;
738684 }
739685
740686 static void print_bad_events(int bad, int total)
@@ -836,20 +782,29 @@ static void dump_map(void) · 95 unmodified lines
836782 }
837783 }
838784
839−static void dump_info(void)
785+static int dump_info(void)
840786 {
787+ int rc = 0;
788+
841789 if (info_threads)
842790 dump_threads();
843791 else if (info_map)
844792 dump_map();
845− else
846− die("Unknown type of information\n");
793+ else {
794+ rc = -1;
795+ pr_err("Unknown type of information\n");
796+ }
797+
798+ return rc;
847799 }
848800
849−static int process_sample_event(struct perf_tool *tool __used,
801+typedef int (*tracepoint_handler)(struct perf_evsel *evsel,
802+ struct perf_sample *sample);
803+
804+static int process_sample_event(struct perf_tool *tool __maybe_unused,
850805 union perf_event *event,
851806 struct perf_sample *sample,
852− struct perf_evsel *evsel __used,
807+ struct perf_evsel *evsel,
853808 struct machine *machine)
854809 {
855810 struct thread *thread = machine__findnew_thread(machine, sample->tid);
@@ -860,7 +815,10 @@ static int process_sample_event(struct perf_tool *tool __used, · 4 unmodified lines
860815 return -1;
861816 }
862817
863− process_raw_event(sample->raw_data, sample->cpu, sample->time, thread);
818+ if (evsel->handler.func != NULL) {
819+ tracepoint_handler f = evsel->handler.func;
820+ return f(evsel, sample);
821+ }
864822
865823 return 0;
866824 }
@@ -871,11 +829,25 @@ static struct perf_tool eops = { · 4 unmodified lines
871829 .ordered_samples = true,
872830 };
873831
832+static const struct perf_evsel_str_handler lock_tracepoints[] = {
833+ { "lock:lock_acquire", perf_evsel__process_lock_acquire, }, /* CONFIG_LOCKDEP */
834+ { "lock:lock_acquired", perf_evsel__process_lock_acquired, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
835+ { "lock:lock_contended", perf_evsel__process_lock_contended, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
836+ { "lock:lock_release", perf_evsel__process_lock_release, }, /* CONFIG_LOCKDEP */
837+};
838+
874839 static int read_events(void)
875840 {
876841 session = perf_session__new(input_name, O_RDONLY, 0, false, &eops);
877− if (!session)
878− die("Initializing perf session failed\n");
842+ if (!session) {
843+ pr_err("Initializing perf session failed\n");
844+ return -1;
845+ }
846+
847+ if (perf_session__set_tracepoints_handlers(session, lock_tracepoints)) {
848+ pr_err("Initializing perf session tracepoint handlers failed\n");
849+ return -1;
850+ }
879851
880852 return perf_session__process_events(session, &eops);
881853 }
@@ -892,13 +864,18 @@ static void sort_result(void) · 10 unmodified lines
892864 }
893865 }
894866
895−static void __cmd_report(void)
867+static int __cmd_report(void)
896868 {
897869 setup_pager();
898− select_key();
899− read_events();
870+
871+ if ((select_key() != 0) ||
872+ (read_events() != 0))
873+ return -1;
874+
900875 sort_result();
901876 print_result();
877+
878+ return 0;
902879 }
903880
904881 static const char * const report_usage[] = {
@@ -944,10 +921,6 @@ static const char *record_args[] = { · 39 unmodified lines
944921 "-f",
945922 "-m", "1024",
946923 "-c", "1",
947− "-e", "lock:lock_acquire",
948− "-e", "lock:lock_acquired",
949− "-e", "lock:lock_contended",
950− "-e", "lock:lock_release",
951924 };
952925
953926 static int __cmd_record(int argc, const char **argv)
@@ -955,15 +928,31 @@ static int __cmd_record(int argc, const char **argv) · 1 unmodified line
955928 unsigned int rec_argc, i, j;
956929 const char **rec_argv;
957930
931+ for (i = 0; i < ARRAY_SIZE(lock_tracepoints); i++) {
932+ if (!is_valid_tracepoint(lock_tracepoints[i].name)) {
933+ pr_err("tracepoint %s is not enabled. "
934+ "Are CONFIG_LOCKDEP and CONFIG_LOCK_STAT enabled?\n",
935+ lock_tracepoints[i].name);
936+ return 1;
937+ }
938+ }
939+
958940 rec_argc = ARRAY_SIZE(record_args) + argc - 1;
959− rec_argv = calloc(rec_argc + 1, sizeof(char *));
941+ /* factor of 2 is for -e in front of each tracepoint */
942+ rec_argc += 2 * ARRAY_SIZE(lock_tracepoints);
960943
944+ rec_argv = calloc(rec_argc + 1, sizeof(char *));
961945 if (rec_argv == NULL)
962946 return -ENOMEM;
963947
964948 for (i = 0; i < ARRAY_SIZE(record_args); i++)
965949 rec_argv[i] = strdup(record_args[i]);
966950
951+ for (j = 0; j < ARRAY_SIZE(lock_tracepoints); j++) {
952+ rec_argv[i++] = "-e";
953+ rec_argv[i++] = strdup(lock_tracepoints[j].name);
954+ }
955+
967956 for (j = 1; j < (unsigned int)argc; j++, i++)
968957 rec_argv[i] = argv[j];
969958
@@ -972,9 +961,10 @@ static int __cmd_record(int argc, const char **argv) · 2 unmodified lines
972961 return cmd_record(i, rec_argv, NULL);
973962 }
974963
975−int cmd_lock(int argc, const char **argv, const char *prefix __used)
964+int cmd_lock(int argc, const char **argv, const char *prefix __maybe_unused)
976965 {
977966 unsigned int i;
967+ int rc = 0;
978968
979969 symbol__init();
980970 for (i = 0; i < LOCKHASH_SIZE; i++)
@@ -1009,11 +999,13 @@ int cmd_lock(int argc, const char **argv, const char *prefix __used) · 28 unmodified lines
1009999 /* recycling report_lock_ops */
10101000 trace_handler = &report_lock_ops;
10111001 setup_pager();
1012− read_events();
1013− dump_info();
1002+ if (read_events() != 0)
1003+ rc = -1;
1004+ else
1005+ rc = dump_info();
10141006 } else {
10151007 usage_with_options(lock_usage, lock_options);
10161008 }
10171009
1018− return 0;
1010+ return rc;
10191011 }
tools/perf/builtin-probe.c+12−12
@@ -143,8 +143,8 @@ static int parse_probe_event_argv(int argc, const char **argv) · 142 unmodified lines
143143 return ret;
144144 }
145145
146−static int opt_add_probe_event(const struct option *opt __used,
147− const char *str, int unset __used)
146+static int opt_add_probe_event(const struct option *opt __maybe_unused,
147+ const char *str, int unset __maybe_unused)
148148 {
149149 if (str) {
150150 params.mod_events = true;
@@ -153,8 +153,8 @@ static int opt_add_probe_event(const struct option *opt __used, · 2 unmodified lines
153153 return 0;
154154 }
155155
156−static int opt_del_probe_event(const struct option *opt __used,
157− const char *str, int unset __used)
156+static int opt_del_probe_event(const struct option *opt __maybe_unused,
157+ const char *str, int unset __maybe_unused)
158158 {
159159 if (str) {
160160 params.mod_events = true;
@@ -166,7 +166,7 @@ static int opt_del_probe_event(const struct option *opt __used, · 5 unmodified lines
166166 }
167167
168168 static int opt_set_target(const struct option *opt, const char *str,
169− int unset __used)
169+ int unset __maybe_unused)
170170 {
171171 int ret = -ENOENT;
172172
@@ -188,8 +188,8 @@ static int opt_set_target(const struct option *opt, const char *str, · 15 unmodified lines
188188 }
189189
190190 #ifdef DWARF_SUPPORT
191−static int opt_show_lines(const struct option *opt __used,
192− const char *str, int unset __used)
191+static int opt_show_lines(const struct option *opt __maybe_unused,
192+ const char *str, int unset __maybe_unused)
193193 {
194194 int ret = 0;
195195
@@ -209,8 +209,8 @@ static int opt_show_lines(const struct option *opt __used, · 13 unmodified lines
209209 return ret;
210210 }
211211
212−static int opt_show_vars(const struct option *opt __used,
213− const char *str, int unset __used)
212+static int opt_show_vars(const struct option *opt __maybe_unused,
213+ const char *str, int unset __maybe_unused)
214214 {
215215 struct perf_probe_event *pev = &params.events[params.nevents];
216216 int ret;
@@ -229,8 +229,8 @@ static int opt_show_vars(const struct option *opt __used, · 12 unmodified lines
229229 }
230230 #endif
231231
232−static int opt_set_filter(const struct option *opt __used,
233− const char *str, int unset __used)
232+static int opt_set_filter(const struct option *opt __maybe_unused,
233+ const char *str, int unset __maybe_unused)
234234 {
235235 const char *err;
236236
@@ -327,7 +327,7 @@ static const struct option options[] = { · 90 unmodified lines
327327 OPT_END()
328328 };
329329
330−int cmd_probe(int argc, const char **argv, const char *prefix __used)
330+int cmd_probe(int argc, const char **argv, const char *prefix __maybe_unused)
331331 {
332332 int ret;
333333
tools/perf/builtin-record.c+237−68
@@ -31,6 +31,15 @@ · 30 unmodified lines
3131 #include <sched.h>
3232 #include <sys/mman.h>
3333
34+#define CALLCHAIN_HELP "do call-graph (stack chain/backtrace) recording: "
35+
36+#ifdef NO_LIBUNWIND_SUPPORT
37+static char callchain_help[] = CALLCHAIN_HELP "[fp]";
38+#else
39+static unsigned long default_stack_dump_size = 8192;
40+static char callchain_help[] = CALLCHAIN_HELP "[fp] dwarf";
41+#endif
42+
3443 enum write_mode_t {
3544 WRITE_FORCE,
3645 WRITE_APPEND
@@ -62,32 +71,38 @@ static void advance_output(struct perf_record *rec, size_t size) · 25 unmodified lines
6271 rec->bytes_written += size;
6372 }
6473
65−static void write_output(struct perf_record *rec, void *buf, size_t size)
74+static int write_output(struct perf_record *rec, void *buf, size_t size)
6675 {
6776 while (size) {
6877 int ret = write(rec->output, buf, size);
6978
70− if (ret < 0)
71− die("failed to write");
79+ if (ret < 0) {
80+ pr_err("failed to write\n");
81+ return -1;
82+ }
7283
7384 size -= ret;
7485 buf += ret;
7586
7687 rec->bytes_written += ret;
7788 }
89+
90+ return 0;
7891 }
7992
8093 static int process_synthesized_event(struct perf_tool *tool,
8194 union perf_event *event,
82− struct perf_sample *sample __used,
83− struct machine *machine __used)
95+ struct perf_sample *sample __maybe_unused,
96+ struct machine *machine __maybe_unused)
8497 {
8598 struct perf_record *rec = container_of(tool, struct perf_record, tool);
86− write_output(rec, event, event->header.size);
99+ if (write_output(rec, event, event->header.size) < 0)
100+ return -1;
101+
87102 return 0;
88103 }
89104
90−static void perf_record__mmap_read(struct perf_record *rec,
105+static int perf_record__mmap_read(struct perf_record *rec,
91106 struct perf_mmap *md)
92107 {
93108 unsigned int head = perf_mmap__read_head(md);
@@ -95,9 +110,10 @@ static void perf_record__mmap_read(struct perf_record *rec, · 1 unmodified line
95110 unsigned char *data = md->base + rec->page_size;
96111 unsigned long size;
97112 void *buf;
113+ int rc = 0;
98114
99115 if (old == head)
100− return;
116+ return 0;
101117
102118 rec->samples++;
103119
@@ -108,17 +124,26 @@ static void perf_record__mmap_read(struct perf_record *rec, · 4 unmodified lines
108124 size = md->mask + 1 - (old & md->mask);
109125 old += size;
110126
111− write_output(rec, buf, size);
127+ if (write_output(rec, buf, size) < 0) {
128+ rc = -1;
129+ goto out;
130+ }
112131 }
113132
114133 buf = &data[old & md->mask];
115134 size = head - old;
116135 old += size;
117136
118− write_output(rec, buf, size);
137+ if (write_output(rec, buf, size) < 0) {
138+ rc = -1;
139+ goto out;
140+ }
119141
120142 md->prev = old;
121143 perf_mmap__write_tail(md, old);
144+
145+out:
146+ return rc;
122147 }
123148
124149 static volatile int done = 0;
@@ -134,7 +159,7 @@ static void sig_handler(int sig) · 9 unmodified lines
134159 signr = sig;
135160 }
136161
137−static void perf_record__sig_exit(int exit_status __used, void *arg)
162+static void perf_record__sig_exit(int exit_status __maybe_unused, void *arg)
138163 {
139164 struct perf_record *rec = arg;
140165 int status;
@@ -163,31 +188,32 @@ static bool perf_evlist__equal(struct perf_evlist *evlist, · 22 unmodified lines
163188 if (evlist->nr_entries != other->nr_entries)
164189 return false;
165190
166− pair = list_entry(other->entries.next, struct perf_evsel, node);
191+ pair = perf_evlist__first(other);
167192
168193 list_for_each_entry(pos, &evlist->entries, node) {
169194 if (memcmp(&pos->attr, &pair->attr, sizeof(pos->attr) != 0))
170195 return false;
171− pair = list_entry(pair->node.next, struct perf_evsel, node);
196+ pair = perf_evsel__next(pair);
172197 }
173198
174199 return true;
175200 }
176201
177−static void perf_record__open(struct perf_record *rec)
202+static int perf_record__open(struct perf_record *rec)
178203 {
179− struct perf_evsel *pos, *first;
204+ struct perf_evsel *pos;
180205 struct perf_evlist *evlist = rec->evlist;
181206 struct perf_session *session = rec->session;
182207 struct perf_record_opts *opts = &rec->opts;
183−
184− first = list_entry(evlist->entries.next, struct perf_evsel, node);
208+ int rc = 0;
185209
186210 perf_evlist__config_attrs(evlist, opts);
187211
212+ if (opts->group)
213+ perf_evlist__set_leader(evlist);
214+
188215 list_for_each_entry(pos, &evlist->entries, node) {
189216 struct perf_event_attr *attr = &pos->attr;
190− struct xyarray *group_fd = NULL;
191217 /*
192218 * Check if parse_single_tracepoint_event has already asked for
193219 * PERF_SAMPLE_TIME.
@@ -202,24 +228,24 @@ static void perf_record__open(struct perf_record *rec) · 8 unmodified lines
202228 */
203229 bool time_needed = attr->sample_type & PERF_SAMPLE_TIME;
204230
205− if (opts->group && pos != first)
206− group_fd = first->fd;
207231 fallback_missing_features:
208232 if (opts->exclude_guest_missing)
209233 attr->exclude_guest = attr->exclude_host = 0;
210234 retry_sample_id:
211235 attr->sample_id_all = opts->sample_id_all_missing ? 0 : 1;
212236 try_again:
213− if (perf_evsel__open(pos, evlist->cpus, evlist->threads,
214− opts->group, group_fd) < 0) {
237+ if (perf_evsel__open(pos, evlist->cpus, evlist->threads) < 0) {
215238 int err = errno;
216239
217240 if (err == EPERM || err == EACCES) {
218241 ui__error_paranoid();
219− exit(EXIT_FAILURE);
242+ rc = -err;
243+ goto out;
220244 } else if (err == ENODEV && opts->target.cpu_list) {
221− die("No such device - did you specify"
222− " an out-of-range profile CPU?\n");
245+ pr_err("No such device - did you specify"
246+ " an out-of-range profile CPU?\n");
247+ rc = -err;
248+ goto out;
223249 } else if (err == EINVAL) {
224250 if (!opts->exclude_guest_missing &&
225251 (attr->exclude_guest || attr->exclude_host)) {
@@ -266,42 +292,57 @@ try_again: · 40 unmodified lines
266292 if (err == ENOENT) {
267293 ui__error("The %s event is not supported.\n",
268294 perf_evsel__name(pos));
269− exit(EXIT_FAILURE);
295+ rc = -err;
296+ goto out;
270297 }
271298
272299 printf("\n");
273− error("sys_perf_event_open() syscall returned with %d (%s). /bin/dmesg may provide additional information.\n",
274− err, strerror(err));
300+ error("sys_perf_event_open() syscall returned with %d "
301+ "(%s) for event %s. /bin/dmesg may provide "
302+ "additional information.\n",
303+ err, strerror(err), perf_evsel__name(pos));
275304
276305 #if defined(__i386__) || defined(__x86_64__)
277− if (attr->type == PERF_TYPE_HARDWARE && err == EOPNOTSUPP)
278− die("No hardware sampling interrupt available."
279− " No APIC? If so then you can boot the kernel"
280− " with the \"lapic\" boot parameter to"
281− " force-enable it.\n");
306+ if (attr->type == PERF_TYPE_HARDWARE &&
307+ err == EOPNOTSUPP) {
308+ pr_err("No hardware sampling interrupt available."
309+ " No APIC? If so then you can boot the kernel"
310+ " with the \"lapic\" boot parameter to"
311+ " force-enable it.\n");
312+ rc = -err;
313+ goto out;
314+ }
282315 #endif
283316
284− die("No CONFIG_PERF_EVENTS=y kernel support configured?\n");
317+ pr_err("No CONFIG_PERF_EVENTS=y kernel support configured?\n");
318+ rc = -err;
319+ goto out;
285320 }
286321 }
287322
288− if (perf_evlist__set_filters(evlist)) {
323+ if (perf_evlist__apply_filters(evlist)) {
289324 error("failed to set filter with %d (%s)\n", errno,
290325 strerror(errno));
291− exit(-1);
326+ rc = -1;
327+ goto out;
292328 }
293329
294330 if (perf_evlist__mmap(evlist, opts->mmap_pages, false) < 0) {
295− if (errno == EPERM)
296− die("Permission error mapping pages.\n"
297− "Consider increasing "
298− "/proc/sys/kernel/perf_event_mlock_kb,\n"
299− "or try again with a smaller value of -m/--mmap_pages.\n"
300− "(current value: %d)\n", opts->mmap_pages);
301− else if (!is_power_of_2(opts->mmap_pages))
302− die("--mmap_pages/-m value must be a power of two.");
303−
304− die("failed to mmap with %d (%s)\n", errno, strerror(errno));
331+ if (errno == EPERM) {
332+ pr_err("Permission error mapping pages.\n"
333+ "Consider increasing "
334+ "/proc/sys/kernel/perf_event_mlock_kb,\n"
335+ "or try again with a smaller value of -m/--mmap_pages.\n"
336+ "(current value: %d)\n", opts->mmap_pages);
337+ rc = -errno;
338+ } else if (!is_power_of_2(opts->mmap_pages)) {
339+ pr_err("--mmap_pages/-m value must be a power of two.");
340+ rc = -EINVAL;
341+ } else {
342+ pr_err("failed to mmap with %d (%s)\n", errno, strerror(errno));
343+ rc = -errno;
344+ }
345+ goto out;
305346 }
306347
307348 if (rec->file_new)
@@ -309,11 +350,14 @@ try_again: · 1 unmodified line
309350 else {
310351 if (!perf_evlist__equal(session->evlist, evlist)) {
311352 fprintf(stderr, "incompatible append\n");
312− exit(-1);
353+ rc = -1;
354+ goto out;
313355 }
314356 }
315357
316358 perf_session__set_id_hdr_size(session);
359+out:
360+ return rc;
317361 }
318362
319363 static int process_buildids(struct perf_record *rec)
@@ -329,10 +373,13 @@ static int process_buildids(struct perf_record *rec) · 9 unmodified lines
329373 size, &build_id__mark_dso_hit_ops);
330374 }
331375
332−static void perf_record__exit(int status __used, void *arg)
376+static void perf_record__exit(int status, void *arg)
333377 {
334378 struct perf_record *rec = arg;
335379
380+ if (status != 0)
381+ return;
382+
336383 if (!rec->opts.pipe_output) {
337384 rec->session->header.data_size += rec->bytes_written;
338385
@@ -387,17 +434,26 @@ static struct perf_event_header finished_round_event = { · 48 unmodified lines
387434 .type = PERF_RECORD_FINISHED_ROUND,
388435 };
389436
390−static void perf_record__mmap_read_all(struct perf_record *rec)
437+static int perf_record__mmap_read_all(struct perf_record *rec)
391438 {
392439 int i;
440+ int rc = 0;
393441
394442 for (i = 0; i < rec->evlist->nr_mmaps; i++) {
395− if (rec->evlist->mmap[i].base)
396− perf_record__mmap_read(rec, &rec->evlist->mmap[i]);
443+ if (rec->evlist->mmap[i].base) {
444+ if (perf_record__mmap_read(rec, &rec->evlist->mmap[i]) != 0) {
445+ rc = -1;
446+ goto out;
447+ }
448+ }
397449 }
398450
399451 if (perf_header__has_feat(&rec->session->header, HEADER_TRACING_DATA))
400− write_output(rec, &finished_round_event, sizeof(finished_round_event));
452+ rc = write_output(rec, &finished_round_event,
453+ sizeof(finished_round_event));
454+
455+out:
456+ return rc;
401457 }
402458
403459 static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
@@ -457,7 +513,7 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) · 53 unmodified lines
457513 output = open(output_name, flags, S_IRUSR | S_IWUSR);
458514 if (output < 0) {
459515 perror("failed to create output file");
460− exit(-1);
516+ return -1;
461517 }
462518
463519 rec->output = output;
@@ -497,7 +553,10 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) · 33 unmodified lines
497553 }
498554 }
499555
500− perf_record__open(rec);
556+ if (perf_record__open(rec) != 0) {
557+ err = -1;
558+ goto out_delete_session;
559+ }
501560
502561 /*
503562 * perf_session__delete(session) will be called at perf_record__exit()
@@ -507,19 +566,20 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) · 3 unmodified lines
507566 if (opts->pipe_output) {
508567 err = perf_header__write_pipe(output);
509568 if (err < 0)
510− return err;
569+ goto out_delete_session;
511570 } else if (rec->file_new) {
512571 err = perf_session__write_header(session, evsel_list,
513572 output, false);
514573 if (err < 0)
515− return err;
574+ goto out_delete_session;
516575 }
517576
518577 if (!rec->no_buildid
519578 && !perf_header__has_feat(&session->header, HEADER_BUILD_ID)) {
520579 pr_err("Couldn't generate buildids. "
521580 "Use --no-buildid to profile anyway.\n");
522− return -1;
581+ err = -1;
582+ goto out_delete_session;
523583 }
524584
525585 rec->post_processing_offset = lseek(output, 0, SEEK_CUR);
@@ -527,7 +587,8 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) · 1 unmodified line
527587 machine = perf_session__find_host_machine(session);
528588 if (!machine) {
529589 pr_err("Couldn't find native kernel information.\n");
530− return -1;
590+ err = -1;
591+ goto out_delete_session;
531592 }
532593
533594 if (opts->pipe_output) {
@@ -535,14 +596,14 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) · 1 unmodified line
535596 process_synthesized_event);
536597 if (err < 0) {
537598 pr_err("Couldn't synthesize attrs.\n");
538− return err;
599+ goto out_delete_session;
539600 }
540601
541602 err = perf_event__synthesize_event_types(tool, process_synthesized_event,
542603 machine);
543604 if (err < 0) {
544605 pr_err("Couldn't synthesize event_types.\n");
545− return err;
606+ goto out_delete_session;
546607 }
547608
548609 if (have_tracepoints(&evsel_list->entries)) {
@@ -558,7 +619,7 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) · 9 unmodified lines
558619 process_synthesized_event);
559620 if (err <= 0) {
560621 pr_err("Couldn't record tracing data.\n");
561− return err;
622+ goto out_delete_session;
562623 }
563624 advance_output(rec, err);
564625 }
@@ -586,20 +647,24 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) · 21 unmodified lines
586647 perf_event__synthesize_guest_os);
587648
588649 if (!opts->target.system_wide)
589− perf_event__synthesize_thread_map(tool, evsel_list->threads,
650+ err = perf_event__synthesize_thread_map(tool, evsel_list->threads,
590651 process_synthesized_event,
591652 machine);
592653 else
593− perf_event__synthesize_threads(tool, process_synthesized_event,
654+ err = perf_event__synthesize_threads(tool, process_synthesized_event,
594655 machine);
595656
657+ if (err != 0)
658+ goto out_delete_session;
659+
596660 if (rec->realtime_prio) {
597661 struct sched_param param;
598662
599663 param.sched_priority = rec->realtime_prio;
600664 if (sched_setscheduler(0, SCHED_FIFO, &param)) {
601665 pr_err("Could not set realtime priority.\n");
602− exit(-1);
666+ err = -1;
667+ goto out_delete_session;
603668 }
604669 }
605670
@@ -614,7 +679,10 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) · 8 unmodified lines
614679 for (;;) {
615680 int hits = rec->samples;
616681
617− perf_record__mmap_read_all(rec);
682+ if (perf_record__mmap_read_all(rec) < 0) {
683+ err = -1;
684+ goto out_delete_session;
685+ }
618686
619687 if (hits == rec->samples) {
620688 if (done)
@@ -732,6 +800,106 @@ error: · 111 unmodified lines
732800 return ret;
733801 }
734802
803+#ifndef NO_LIBUNWIND_SUPPORT
804+static int get_stack_size(char *str, unsigned long *_size)
805+{
806+ char *endptr;
807+ unsigned long size;
808+ unsigned long max_size = round_down(USHRT_MAX, sizeof(u64));
809+
810+ size = strtoul(str, &endptr, 0);
811+
812+ do {
813+ if (*endptr)
814+ break;
815+
816+ size = round_up(size, sizeof(u64));
817+ if (!size || size > max_size)
818+ break;
819+
820+ *_size = size;
821+ return 0;
822+
823+ } while (0);
824+
825+ pr_err("callchain: Incorrect stack dump size (max %ld): %s\n",
826+ max_size, str);
827+ return -1;
828+}
829+#endif /* !NO_LIBUNWIND_SUPPORT */
830+
831+static int
832+parse_callchain_opt(const struct option *opt __maybe_unused, const char *arg,
833+ int unset)
834+{
835+ struct perf_record *rec = (struct perf_record *)opt->value;
836+ char *tok, *name, *saveptr = NULL;
837+ char *buf;
838+ int ret = -1;
839+
840+ /* --no-call-graph */
841+ if (unset)
842+ return 0;
843+
844+ /* We specified default option if none is provided. */
845+ BUG_ON(!arg);
846+
847+ /* We need buffer that we know we can write to. */
848+ buf = malloc(strlen(arg) + 1);
849+ if (!buf)
850+ return -ENOMEM;
851+
852+ strcpy(buf, arg);
853+
854+ tok = strtok_r((char *)buf, ",", &saveptr);
855+ name = tok ? : (char *)buf;
856+
857+ do {
858+ /* Framepointer style */
859+ if (!strncmp(name, "fp", sizeof("fp"))) {
860+ if (!strtok_r(NULL, ",", &saveptr)) {
861+ rec->opts.call_graph = CALLCHAIN_FP;
862+ ret = 0;
863+ } else
864+ pr_err("callchain: No more arguments "
865+ "needed for -g fp\n");
866+ break;
867+
868+#ifndef NO_LIBUNWIND_SUPPORT
869+ /* Dwarf style */
870+ } else if (!strncmp(name, "dwarf", sizeof("dwarf"))) {
871+ ret = 0;
872+ rec->opts.call_graph = CALLCHAIN_DWARF;
873+ rec->opts.stack_dump_size = default_stack_dump_size;
874+
875+ tok = strtok_r(NULL, ",", &saveptr);
876+ if (tok) {
877+ unsigned long size = 0;
878+
879+ ret = get_stack_size(tok, &size);
880+ rec->opts.stack_dump_size = size;
881+ }
882+
883+ if (!ret)
884+ pr_debug("callchain: stack dump size %d\n",
885+ rec->opts.stack_dump_size);
886+#endif /* !NO_LIBUNWIND_SUPPORT */
887+ } else {
888+ pr_err("callchain: Unknown -g option "
889+ "value: %s\n", arg);
890+ break;
891+ }
892+
893+ } while (0);
894+
895+ free(buf);
896+
897+ if (!ret)
898+ pr_debug("callchain: type %d\n", rec->opts.call_graph);
899+
900+ return ret;
901+}
902+
735903 static const char * const record_usage[] = {
736904 "perf record [<options>] [<command>]",
737905 "perf record [<options>] -- <command> [<options>]",
@@ -803,8 +971,9 @@ const struct option record_options[] = { · 65 unmodified lines
803971 "number of mmap data pages"),
804972 OPT_BOOLEAN(0, "group", &record.opts.group,
805973 "put the counters into a counter group"),
806− OPT_BOOLEAN('g', "call-graph", &record.opts.call_graph,
807− "do call-graph (stack chain/backtrace) recording"),
974+ OPT_CALLBACK_DEFAULT('g', "call-graph", &record, "mode[,dump_size]",
975+ callchain_help, &parse_callchain_opt,
976+ "fp"),
808977 OPT_INCR('v', "verbose", &verbose,
809978 "be more verbose (show counter open errors, etc)"),
810979 OPT_BOOLEAN('q', "quiet", &quiet, "don't print any message"),
@@ -836,7 +1005,7 @@ const struct option record_options[] = { · 25 unmodified lines
8361005 OPT_END()
8371006 };
8381007
839−int cmd_record(int argc, const char **argv, const char *prefix __used)
1008+int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
8401009 {
8411010 int err = -ENOMEM;
8421011 struct perf_evsel *pos;
tools/perf/builtin-report.c+26−21
@@ -69,8 +69,8 @@ static int perf_report__add_branch_hist_entry(struct perf_tool *tool, · 68 unmodified lines
6969
7070 if ((sort__has_parent || symbol_conf.use_callchain)
7171 && sample->callchain) {
72− err = machine__resolve_callchain(machine, al->thread,
73− sample->callchain, &parent);
72+ err = machine__resolve_callchain(machine, evsel, al->thread,
73+ sample, &parent);
7474 if (err)
7575 return err;
7676 }
@@ -93,7 +93,7 @@ static int perf_report__add_branch_hist_entry(struct perf_tool *tool, · 16 unmodified lines
9393 struct annotation *notes;
9494 err = -ENOMEM;
9595 bx = he->branch_info;
96− if (bx->from.sym && use_browser > 0) {
96+ if (bx->from.sym && use_browser == 1 && sort__has_sym) {
9797 notes = symbol__annotation(bx->from.sym);
9898 if (!notes->src
9999 && symbol__alloc_hist(bx->from.sym) < 0)
@@ -107,7 +107,7 @@ static int perf_report__add_branch_hist_entry(struct perf_tool *tool, · 7 unmodified lines
107107 goto out;
108108 }
109109
110− if (bx->to.sym && use_browser > 0) {
110+ if (bx->to.sym && use_browser == 1 && sort__has_sym) {
111111 notes = symbol__annotation(bx->to.sym);
112112 if (!notes->src
113113 && symbol__alloc_hist(bx->to.sym) < 0)
@@ -140,8 +140,8 @@ static int perf_evsel__add_hist_entry(struct perf_evsel *evsel, · 26 unmodified lines
140140 struct hist_entry *he;
141141
142142 if ((sort__has_parent || symbol_conf.use_callchain) && sample->callchain) {
143− err = machine__resolve_callchain(machine, al->thread,
144− sample->callchain, &parent);
143+ err = machine__resolve_callchain(machine, evsel, al->thread,
144+ sample, &parent);
145145 if (err)
146146 return err;
147147 }
@@ -162,7 +162,7 @@ static int perf_evsel__add_hist_entry(struct perf_evsel *evsel, · 14 unmodified lines
162162 * so we don't allocated the extra space needed because the stdio
163163 * code will not use it.
164164 */
165− if (he->ms.sym != NULL && use_browser > 0) {
165+ if (he->ms.sym != NULL && use_browser == 1 && sort__has_sym) {
166166 struct annotation *notes = symbol__annotation(he->ms.sym);
167167
168168 assert(evsel != NULL);
@@ -223,9 +223,9 @@ static int process_sample_event(struct perf_tool *tool, · 54 unmodified lines
223223
224224 static int process_read_event(struct perf_tool *tool,
225225 union perf_event *event,
226− struct perf_sample *sample __used,
226+ struct perf_sample *sample __maybe_unused,
227227 struct perf_evsel *evsel,
228− struct machine *machine __used)
228+ struct machine *machine __maybe_unused)
229229 {
230230 struct perf_report *rep = container_of(tool, struct perf_report, tool);
231231
@@ -287,7 +287,7 @@ static int perf_report__setup_sample_type(struct perf_report *rep) · 55 unmodified lines
287287
288288 extern volatile int session_done;
289289
290−static void sig_handler(int sig __used)
290+static void sig_handler(int sig __maybe_unused)
291291 {
292292 session_done = 1;
293293 }
@@ -397,17 +397,17 @@ static int __cmd_report(struct perf_report *rep) · 103 unmodified lines
397397 desc);
398398 }
399399
400− if (dump_trace) {
401− perf_session__fprintf_nr_events(session, stdout);
402− goto out_delete;
403− }
404−
405400 if (verbose > 3)
406401 perf_session__fprintf(session, stdout);
407402
408403 if (verbose > 2)
409404 perf_session__fprintf_dsos(session, stdout);
410405
406+ if (dump_trace) {
407+ perf_session__fprintf_nr_events(session, stdout);
408+ goto out_delete;
409+ }
410+
411411 nr_samples = 0;
412412 list_for_each_entry(pos, &session->evlist->entries, node) {
413413 struct hists *hists = &pos->hists;
@@ -533,13 +533,14 @@ setup: · 119 unmodified lines
533533 }
534534
535535 static int
536−parse_branch_mode(const struct option *opt __used, const char *str __used, int unset)
536+parse_branch_mode(const struct option *opt __maybe_unused,
537+ const char *str __maybe_unused, int unset)
537538 {
538539 sort__branch_mode = !unset;
539540 return 0;
540541 }
541542
542−int cmd_report(int argc, const char **argv, const char *prefix __used)
543+int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
543544 {
544545 struct perf_session *session;
545546 struct stat st;
@@ -638,6 +639,8 @@ int cmd_report(int argc, const char **argv, const char *prefix __used) · 92 unmodified lines
638639 "Show a column with the sum of periods"),
639640 OPT_CALLBACK_NOOPT('b', "branch-stack", &sort__branch_mode, "",
640641 "use branch records for histogram filling", parse_branch_mode),
642+ OPT_STRING(0, "objdump", &objdump_path, "path",
643+ "objdump binary to use for disassembly and annotations"),
641644 OPT_END()
642645 };
643646
@@ -686,15 +689,19 @@ int cmd_report(int argc, const char **argv, const char *prefix __used) · 42 unmodified lines
686689
687690 if (strcmp(report.input_name, "-") != 0)
688691 setup_browser(true);
689− else
692+ else {
690693 use_browser = 0;
694+ perf_hpp__init(false, false);
695+ }
696+
697+ setup_sorting(report_usage, options);
691698
692699 /*
693700 * Only in the newt browser we are doing integrated annotation,
694701 * so don't allocate extra space that won't be used in the stdio
695702 * implementation.
696703 */
697− if (use_browser > 0) {
704+ if (use_browser == 1 && sort__has_sym) {
698705 symbol_conf.priv_size = sizeof(struct annotation);
699706 report.annotate_init = symbol__annotate_init;
700707 /*
@@ -717,8 +724,6 @@ int cmd_report(int argc, const char **argv, const char *prefix __used) · 16 unmodified lines
717724 if (symbol__init() < 0)
718725 goto error;
719726
720− setup_sorting(report_usage, options);
721−
722727 if (parent_pattern != default_parent_pattern) {
723728 if (sort_dimension__add("parent") < 0)
724729 goto error;
tools/perf/builtin-sched.c+683−839
@@ -23,31 +23,12 @@ · 22 unmodified lines
2323 #include <pthread.h>
2424 #include <math.h>
2525
26−static const char *input_name;
27−
28−static char default_sort_order[] = "avg, max, switch, runtime";
29−static const char *sort_order = default_sort_order;
30−
31−static int profile_cpu = -1;
32−
3326 #define PR_SET_NAME 15 /* Set process name */
3427 #define MAX_CPUS 4096
35−
36−static u64 run_measurement_overhead;
37−static u64 sleep_measurement_overhead;
38−
3928 #define COMM_LEN 20
4029 #define SYM_LEN 129
41−
4230 #define MAX_PID 65536
4331
44−static unsigned long nr_tasks;
45−
46−struct perf_sched {
47− struct perf_tool tool;
48− struct perf_session *session;
49−};
50−
5132 struct sched_atom;
5233
5334 struct task_desc {
@@ -85,44 +66,6 @@ struct sched_atom { · 31 unmodified lines
8566 struct task_desc *wakee;
8667 };
8768
88−static struct task_desc *pid_to_task[MAX_PID];
89−
90−static struct task_desc **tasks;
91−
92−static pthread_mutex_t start_work_mutex = PTHREAD_MUTEX_INITIALIZER;
93−static u64 start_time;
94−
95−static pthread_mutex_t work_done_wait_mutex = PTHREAD_MUTEX_INITIALIZER;
96−
97−static unsigned long nr_run_events;
98−static unsigned long nr_sleep_events;
99−static unsigned long nr_wakeup_events;
100−
101−static unsigned long nr_sleep_corrections;
102−static unsigned long nr_run_events_optimized;
103−
104−static unsigned long targetless_wakeups;
105−static unsigned long multitarget_wakeups;
106−
107−static u64 cpu_usage;
108−static u64 runavg_cpu_usage;
109−static u64 parent_cpu_usage;
110−static u64 runavg_parent_cpu_usage;
111−
112−static unsigned long nr_runs;
113−static u64 sum_runtime;
114−static u64 sum_fluct;
115−static u64 run_avg;
116−
117−static unsigned int replay_repeat = 10;
118−static unsigned long nr_timestamps;
119−static unsigned long nr_unordered_timestamps;
120−static unsigned long nr_state_machine_bugs;
121−static unsigned long nr_context_switch_bugs;
122−static unsigned long nr_events;
123−static unsigned long nr_lost_chunks;
124−static unsigned long nr_lost_events;
125−
12669 #define TASK_STATE_TO_CHAR_STR "RSDTtZX"
12770
12871 enum thread_state {
@@ -154,11 +97,79 @@ struct work_atoms { · 25 unmodified lines
15497
15598 typedef int (*sort_fn_t)(struct work_atoms *, struct work_atoms *);
15699
157−static struct rb_root atom_root, sorted_atom_root;
100+struct perf_sched;
101+
102+struct trace_sched_handler {
103+ int (*switch_event)(struct perf_sched *sched, struct perf_evsel *evsel,
104+ struct perf_sample *sample, struct machine *machine);
158105
159−static u64 all_runtime;
160−static u64 all_count;
106+ int (*runtime_event)(struct perf_sched *sched, struct perf_evsel *evsel,
107+ struct perf_sample *sample, struct machine *machine);
161108
109+ int (*wakeup_event)(struct perf_sched *sched, struct perf_evsel *evsel,
110+ struct perf_sample *sample, struct machine *machine);
111+
112+ int (*fork_event)(struct perf_sched *sched, struct perf_evsel *evsel,
113+ struct perf_sample *sample);
114+
115+ int (*migrate_task_event)(struct perf_sched *sched,
116+ struct perf_evsel *evsel,
117+ struct perf_sample *sample,
118+ struct machine *machine);
119+};
120+
121+struct perf_sched {
122+ struct perf_tool tool;
123+ const char *input_name;
124+ const char *sort_order;
125+ unsigned long nr_tasks;
126+ struct task_desc *pid_to_task[MAX_PID];
127+ struct task_desc **tasks;
128+ const struct trace_sched_handler *tp_handler;
129+ pthread_mutex_t start_work_mutex;
130+ pthread_mutex_t work_done_wait_mutex;
131+ int profile_cpu;
132+/*
133+ * Track the current task - that way we can know whether there's any
134+ * weird events, such as a task being switched away that is not current.
135+ */
136+ int max_cpu;
137+ u32 curr_pid[MAX_CPUS];
138+ struct thread *curr_thread[MAX_CPUS];
139+ char next_shortname1;
140+ char next_shortname2;
141+ unsigned int replay_repeat;
142+ unsigned long nr_run_events;
143+ unsigned long nr_sleep_events;
144+ unsigned long nr_wakeup_events;
145+ unsigned long nr_sleep_corrections;
146+ unsigned long nr_run_events_optimized;
147+ unsigned long targetless_wakeups;
148+ unsigned long multitarget_wakeups;
149+ unsigned long nr_runs;
150+ unsigned long nr_timestamps;
151+ unsigned long nr_unordered_timestamps;
152+ unsigned long nr_state_machine_bugs;
153+ unsigned long nr_context_switch_bugs;
154+ unsigned long nr_events;
155+ unsigned long nr_lost_chunks;
156+ unsigned long nr_lost_events;
157+ u64 run_measurement_overhead;
158+ u64 sleep_measurement_overhead;
159+ u64 start_time;
160+ u64 cpu_usage;
161+ u64 runavg_cpu_usage;
162+ u64 parent_cpu_usage;
163+ u64 runavg_parent_cpu_usage;
164+ u64 sum_runtime;
165+ u64 sum_fluct;
166+ u64 run_avg;
167+ u64 all_runtime;
168+ u64 all_count;
169+ u64 cpu_last_switched[MAX_CPUS];
170+ struct rb_root atom_root, sorted_atom_root;
171+ struct list_head sort_list, cmp_pid;
172+};
162173
163174 static u64 get_nsecs(void)
164175 {
@@ -169,13 +180,13 @@ static u64 get_nsecs(void) · 4 unmodified lines
169180 return ts.tv_sec * 1000000000ULL + ts.tv_nsec;
170181 }
171182
172−static void burn_nsecs(u64 nsecs)
183+static void burn_nsecs(struct perf_sched *sched, u64 nsecs)
173184 {
174185 u64 T0 = get_nsecs(), T1;
175186
176187 do {
177188 T1 = get_nsecs();
178− } while (T1 + run_measurement_overhead < T0 + nsecs);
189+ } while (T1 + sched->run_measurement_overhead < T0 + nsecs);
179190 }
180191
181192 static void sleep_nsecs(u64 nsecs)
@@ -188,24 +199,24 @@ static void sleep_nsecs(u64 nsecs) · 6 unmodified lines
188199 nanosleep(&ts, NULL);
189200 }
190201
191−static void calibrate_run_measurement_overhead(void)
202+static void calibrate_run_measurement_overhead(struct perf_sched *sched)
192203 {
193204 u64 T0, T1, delta, min_delta = 1000000000ULL;
194205 int i;
195206
196207 for (i = 0; i < 10; i++) {
197208 T0 = get_nsecs();
198− burn_nsecs(0);
209+ burn_nsecs(sched, 0);
199210 T1 = get_nsecs();
200211 delta = T1-T0;
201212 min_delta = min(min_delta, delta);
202213 }
203− run_measurement_overhead = min_delta;
214+ sched->run_measurement_overhead = min_delta;
204215
205216 printf("run measurement overhead: %" PRIu64 " nsecs\n", min_delta);
206217 }
207218
208−static void calibrate_sleep_measurement_overhead(void)
219+static void calibrate_sleep_measurement_overhead(struct perf_sched *sched)
209220 {
210221 u64 T0, T1, delta, min_delta = 1000000000ULL;
211222 int i;
@@ -218,7 +229,7 @@ static void calibrate_sleep_measurement_overhead(void) · 6 unmodified lines
218229 min_delta = min(min_delta, delta);
219230 }
220231 min_delta -= 10000;
221− sleep_measurement_overhead = min_delta;
232+ sched->sleep_measurement_overhead = min_delta;
222233
223234 printf("sleep measurement overhead: %" PRIu64 " nsecs\n", min_delta);
224235 }
@@ -251,8 +262,8 @@ static struct sched_atom *last_event(struct task_desc *task) · 26 unmodified lines
251262 return task->atoms[task->nr_events - 1];
252263 }
253264
254−static void
255−add_sched_event_run(struct task_desc *task, u64 timestamp, u64 duration)
265+static void add_sched_event_run(struct perf_sched *sched, struct task_desc *task,
266+ u64 timestamp, u64 duration)
256267 {
257268 struct sched_atom *event, *curr_event = last_event(task);
258269
@@ -261,7 +272,7 @@ add_sched_event_run(struct task_desc *task, u64 timestamp, u64 duration) · 2 unmodified lines
261272 * to it:
262273 */
263274 if (curr_event && curr_event->type == SCHED_EVENT_RUN) {
264− nr_run_events_optimized++;
275+ sched->nr_run_events_optimized++;
265276 curr_event->duration += duration;
266277 return;
267278 }
@@ -271,12 +282,11 @@ add_sched_event_run(struct task_desc *task, u64 timestamp, u64 duration) · 3 unmodified lines
271282 event->type = SCHED_EVENT_RUN;
272283 event->duration = duration;
273284
274− nr_run_events++;
285+ sched->nr_run_events++;
275286 }
276287
277−static void
278−add_sched_event_wakeup(struct task_desc *task, u64 timestamp,
279− struct task_desc *wakee)
288+static void add_sched_event_wakeup(struct perf_sched *sched, struct task_desc *task,
289+ u64 timestamp, struct task_desc *wakee)
280290 {
281291 struct sched_atom *event, *wakee_event;
282292
@@ -286,11 +296,11 @@ add_sched_event_wakeup(struct task_desc *task, u64 timestamp, · 3 unmodified lines
286296
287297 wakee_event = last_event(wakee);
288298 if (!wakee_event || wakee_event->type != SCHED_EVENT_SLEEP) {
289− targetless_wakeups++;
299+ sched->targetless_wakeups++;
290300 return;
291301 }
292302 if (wakee_event->wait_sem) {
293− multitarget_wakeups++;
303+ sched->multitarget_wakeups++;
294304 return;
295305 }
296306
@@ -299,89 +309,89 @@ add_sched_event_wakeup(struct task_desc *task, u64 timestamp, · 2 unmodified lines
299309 wakee_event->specific_wait = 1;
300310 event->wait_sem = wakee_event->wait_sem;
301311
302− nr_wakeup_events++;
312+ sched->nr_wakeup_events++;
303313 }
304314
305−static void
306−add_sched_event_sleep(struct task_desc *task, u64 timestamp,
307− u64 task_state __used)
315+static void add_sched_event_sleep(struct perf_sched *sched, struct task_desc *task,
316+ u64 timestamp, u64 task_state __maybe_unused)
308317 {
309318 struct sched_atom *event = get_new_event(task, timestamp);
310319
311320 event->type = SCHED_EVENT_SLEEP;
312321
313− nr_sleep_events++;
322+ sched->nr_sleep_events++;
314323 }
315324
316−static struct task_desc *register_pid(unsigned long pid, const char *comm)
325+static struct task_desc *register_pid(struct perf_sched *sched,
326+ unsigned long pid, const char *comm)
317327 {
318328 struct task_desc *task;
319329
320330 BUG_ON(pid >= MAX_PID);
321331
322− task = pid_to_task[pid];
332+ task = sched->pid_to_task[pid];
323333
324334 if (task)
325335 return task;
326336
327337 task = zalloc(sizeof(*task));
328338 task->pid = pid;
329− task->nr = nr_tasks;
339+ task->nr = sched->nr_tasks;
330340 strcpy(task->comm, comm);
331341 /*
332342 * every task starts in sleeping state - this gets ignored
333343 * if there's no wakeup pointing to this sleep state:
334344 */
335− add_sched_event_sleep(task, 0, 0);
345+ add_sched_event_sleep(sched, task, 0, 0);
336346
337− pid_to_task[pid] = task;
338− nr_tasks++;
339− tasks = realloc(tasks, nr_tasks*sizeof(struct task_task *));
340− BUG_ON(!tasks);
341− tasks[task->nr] = task;
347+ sched->pid_to_task[pid] = task;
348+ sched->nr_tasks++;
349+ sched->tasks = realloc(sched->tasks, sched->nr_tasks * sizeof(struct task_task *));
350+ BUG_ON(!sched->tasks);
351+ sched->tasks[task->nr] = task;
342352
343353 if (verbose)
344− printf("registered task #%ld, PID %ld (%s)\n", nr_tasks, pid, comm);
354+ printf("registered task #%ld, PID %ld (%s)\n", sched->nr_tasks, pid, comm);
345355
346356 return task;
347357 }
348358
349359
350−static void print_task_traces(void)
360+static void print_task_traces(struct perf_sched *sched)
351361 {
352362 struct task_desc *task;
353363 unsigned long i;
354364
355− for (i = 0; i < nr_tasks; i++) {
356− task = tasks[i];
365+ for (i = 0; i < sched->nr_tasks; i++) {
366+ task = sched->tasks[i];
357367 printf("task %6ld (%20s:%10ld), nr_events: %ld\n",
358368 task->nr, task->comm, task->pid, task->nr_events);
359369 }
360370 }
361371
362−static void add_cross_task_wakeups(void)
372+static void add_cross_task_wakeups(struct perf_sched *sched)
363373 {
364374 struct task_desc *task1, *task2;
365375 unsigned long i, j;
366376
367− for (i = 0; i < nr_tasks; i++) {
368− task1 = tasks[i];
377+ for (i = 0; i < sched->nr_tasks; i++) {
378+ task1 = sched->tasks[i];
369379 j = i + 1;
370− if (j == nr_tasks)
380+ if (j == sched->nr_tasks)
371381 j = 0;
372− task2 = tasks[j];
373− add_sched_event_wakeup(task1, 0, task2);
382+ task2 = sched->tasks[j];
383+ add_sched_event_wakeup(sched, task1, 0, task2);
374384 }
375385 }
376386
377−static void
378−process_sched_event(struct task_desc *this_task __used, struct sched_atom *atom)
387+static void perf_sched__process_event(struct perf_sched *sched,
388+ struct sched_atom *atom)
379389 {
380390 int ret = 0;
381391
382392 switch (atom->type) {
383393 case SCHED_EVENT_RUN:
384− burn_nsecs(atom->duration);
394+ burn_nsecs(sched, atom->duration);
385395 break;
386396 case SCHED_EVENT_SLEEP:
387397 if (atom->wait_sem)
@@ -428,8 +438,8 @@ static int self_open_counters(void) · 40 unmodified lines
428438 fd = sys_perf_event_open(&attr, 0, -1, -1, 0);
429439
430440 if (fd < 0)
431− die("Error: sys_perf_event_open() syscall returned"
432− "with %d (%s)\n", fd, strerror(errno));
441+ pr_err("Error: sys_perf_event_open() syscall returned "
442+ "with %d (%s)\n", fd, strerror(errno));
433443 return fd;
434444 }
435445
@@ -444,31 +454,41 @@ static u64 get_cpu_usage_nsec_self(int fd) · 8 unmodified lines
444454 return runtime;
445455 }
446456
457+struct sched_thread_parms {
458+ struct task_desc *task;
459+ struct perf_sched *sched;
460+};
461+
447462 static void *thread_func(void *ctx)
448463 {
449− struct task_desc *this_task = ctx;
464+ struct sched_thread_parms *parms = ctx;
465+ struct task_desc *this_task = parms->task;
466+ struct perf_sched *sched = parms->sched;
450467 u64 cpu_usage_0, cpu_usage_1;
451468 unsigned long i, ret;
452469 char comm2[22];
453470 int fd;
454471
472+ free(parms);
473+
455474 sprintf(comm2, ":%s", this_task->comm);
456475 prctl(PR_SET_NAME, comm2);
457476 fd = self_open_counters();
458−
477+ if (fd < 0)
478+ return NULL;
459479 again:
460480 ret = sem_post(&this_task->ready_for_work);
461481 BUG_ON(ret);
462− ret = pthread_mutex_lock(&start_work_mutex);
482+ ret = pthread_mutex_lock(&sched->start_work_mutex);
463483 BUG_ON(ret);
464− ret = pthread_mutex_unlock(&start_work_mutex);
484+ ret = pthread_mutex_unlock(&sched->start_work_mutex);
465485 BUG_ON(ret);
466486
467487 cpu_usage_0 = get_cpu_usage_nsec_self(fd);
468488
469489 for (i = 0; i < this_task->nr_events; i++) {
470490 this_task->curr_event = i;
471− process_sched_event(this_task, this_task->atoms[i]);
491+ perf_sched__process_event(sched, this_task->atoms[i]);
472492 }
473493
474494 cpu_usage_1 = get_cpu_usage_nsec_self(fd);
@@ -476,15 +496,15 @@ again: · 1 unmodified line
476496 ret = sem_post(&this_task->work_done_sem);
477497 BUG_ON(ret);
478498
479− ret = pthread_mutex_lock(&work_done_wait_mutex);
499+ ret = pthread_mutex_lock(&sched->work_done_wait_mutex);
480500 BUG_ON(ret);
481− ret = pthread_mutex_unlock(&work_done_wait_mutex);
501+ ret = pthread_mutex_unlock(&sched->work_done_wait_mutex);
482502 BUG_ON(ret);
483503
484504 goto again;
485505 }
486506
487−static void create_tasks(void)
507+static void create_tasks(struct perf_sched *sched)
488508 {
489509 struct task_desc *task;
490510 pthread_attr_t attr;
@@ -496,128 +516,129 @@ static void create_tasks(void) · 5 unmodified lines
496516 err = pthread_attr_setstacksize(&attr,
497517 (size_t) max(16 * 1024, PTHREAD_STACK_MIN));
498518 BUG_ON(err);
499− err = pthread_mutex_lock(&start_work_mutex);
519+ err = pthread_mutex_lock(&sched->start_work_mutex);
500520 BUG_ON(err);
501− err = pthread_mutex_lock(&work_done_wait_mutex);
521+ err = pthread_mutex_lock(&sched->work_done_wait_mutex);
502522 BUG_ON(err);
503− for (i = 0; i < nr_tasks; i++) {
504− task = tasks[i];
523+ for (i = 0; i < sched->nr_tasks; i++) {
524+ struct sched_thread_parms *parms = malloc(sizeof(*parms));
525+ BUG_ON(parms == NULL);
526+ parms->task = task = sched->tasks[i];
527+ parms->sched = sched;
505528 sem_init(&task->sleep_sem, 0, 0);
506529 sem_init(&task->ready_for_work, 0, 0);
507530 sem_init(&task->work_done_sem, 0, 0);
508531 task->curr_event = 0;
509− err = pthread_create(&task->thread, &attr, thread_func, task);
532+ err = pthread_create(&task->thread, &attr, thread_func, parms);
510533 BUG_ON(err);
511534 }
512535 }
513536
514−static void wait_for_tasks(void)
537+static void wait_for_tasks(struct perf_sched *sched)
515538 {
516539 u64 cpu_usage_0, cpu_usage_1;
517540 struct task_desc *task;
518541 unsigned long i, ret;
519542
520− start_time = get_nsecs();
521− cpu_usage = 0;
522− pthread_mutex_unlock(&work_done_wait_mutex);
543+ sched->start_time = get_nsecs();
544+ sched->cpu_usage = 0;
545+ pthread_mutex_unlock(&sched->work_done_wait_mutex);
523546
524− for (i = 0; i < nr_tasks; i++) {
525− task = tasks[i];
547+ for (i = 0; i < sched->nr_tasks; i++) {
548+ task = sched->tasks[i];
526549 ret = sem_wait(&task->ready_for_work);
527550 BUG_ON(ret);
528551 sem_init(&task->ready_for_work, 0, 0);
529552 }
530− ret = pthread_mutex_lock(&work_done_wait_mutex);
553+ ret = pthread_mutex_lock(&sched->work_done_wait_mutex);
531554 BUG_ON(ret);
532555
533556 cpu_usage_0 = get_cpu_usage_nsec_parent();
534557
535− pthread_mutex_unlock(&start_work_mutex);
558+ pthread_mutex_unlock(&sched->start_work_mutex);
536559
537− for (i = 0; i < nr_tasks; i++) {
538− task = tasks[i];
560+ for (i = 0; i < sched->nr_tasks; i++) {
561+ task = sched->tasks[i];
539562 ret = sem_wait(&task->work_done_sem);
540563 BUG_ON(ret);
541564 sem_init(&task->work_done_sem, 0, 0);
542− cpu_usage += task->cpu_usage;
565+ sched->cpu_usage += task->cpu_usage;
543566 task->cpu_usage = 0;
544567 }
545568
546569 cpu_usage_1 = get_cpu_usage_nsec_parent();
547− if (!runavg_cpu_usage)
548− runavg_cpu_usage = cpu_usage;
549− runavg_cpu_usage = (runavg_cpu_usage*9 + cpu_usage)/10;
570+ if (!sched->runavg_cpu_usage)
571+ sched->runavg_cpu_usage = sched->cpu_usage;
572+ sched->runavg_cpu_usage = (sched->runavg_cpu_usage * 9 + sched->cpu_usage) / 10;
550573
551− parent_cpu_usage = cpu_usage_1 - cpu_usage_0;
552− if (!runavg_parent_cpu_usage)
553− runavg_parent_cpu_usage = parent_cpu_usage;
554− runavg_parent_cpu_usage = (runavg_parent_cpu_usage*9 +
555− parent_cpu_usage)/10;
574+ sched->parent_cpu_usage = cpu_usage_1 - cpu_usage_0;
575+ if (!sched->runavg_parent_cpu_usage)
576+ sched->runavg_parent_cpu_usage = sched->parent_cpu_usage;
577+ sched->runavg_parent_cpu_usage = (sched->runavg_parent_cpu_usage * 9 +
578+ sched->parent_cpu_usage)/10;
556579
557− ret = pthread_mutex_lock(&start_work_mutex);
580+ ret = pthread_mutex_lock(&sched->start_work_mutex);
558581 BUG_ON(ret);
559582
560− for (i = 0; i < nr_tasks; i++) {
561− task = tasks[i];
583+ for (i = 0; i < sched->nr_tasks; i++) {
584+ task = sched->tasks[i];
562585 sem_init(&task->sleep_sem, 0, 0);
563586 task->curr_event = 0;
564587 }
565588 }
566589
567−static void run_one_test(void)
590+static void run_one_test(struct perf_sched *sched)
568591 {
569592 u64 T0, T1, delta, avg_delta, fluct;
570593
571594 T0 = get_nsecs();
572− wait_for_tasks();
595+ wait_for_tasks(sched);
573596 T1 = get_nsecs();
574597
575598 delta = T1 - T0;
576− sum_runtime += delta;
577− nr_runs++;
599+ sched->sum_runtime += delta;
600+ sched->nr_runs++;
578601
579− avg_delta = sum_runtime / nr_runs;
602+ avg_delta = sched->sum_runtime / sched->nr_runs;
580603 if (delta < avg_delta)
581604 fluct = avg_delta - delta;
582605 else
583606 fluct = delta - avg_delta;
584− sum_fluct += fluct;
585− if (!run_avg)
586− run_avg = delta;
587− run_avg = (run_avg*9 + delta)/10;
607+ sched->sum_fluct += fluct;
608+ if (!sched->run_avg)
609+ sched->run_avg = delta;
610+ sched->run_avg = (sched->run_avg * 9 + delta) / 10;
588611
589− printf("#%-3ld: %0.3f, ",
590− nr_runs, (double)delta/1000000.0);
612+ printf("#%-3ld: %0.3f, ", sched->nr_runs, (double)delta / 1000000.0);
591613
592− printf("ravg: %0.2f, ",
593− (double)run_avg/1e6);
614+ printf("ravg: %0.2f, ", (double)sched->run_avg / 1e6);
594615
595616 printf("cpu: %0.2f / %0.2f",
596− (double)cpu_usage/1e6, (double)runavg_cpu_usage/1e6);
617+ (double)sched->cpu_usage / 1e6, (double)sched->runavg_cpu_usage / 1e6);
597618
598619 #if 0
599620 /*
600621 * rusage statistics done by the parent, these are less
601− * accurate than the sum_exec_runtime based statistics:
622+ * accurate than the sched->sum_exec_runtime based statistics:
602623 */
603624 printf(" [%0.2f / %0.2f]",
604− (double)parent_cpu_usage/1e6,
605− (double)runavg_parent_cpu_usage/1e6);
625+ (double)sched->parent_cpu_usage/1e6,
626+ (double)sched->runavg_parent_cpu_usage/1e6);
606627 #endif
607628
608629 printf("\n");
609630
610− if (nr_sleep_corrections)
611− printf(" (%ld sleep corrections)\n", nr_sleep_corrections);
612− nr_sleep_corrections = 0;
631+ if (sched->nr_sleep_corrections)
632+ printf(" (%ld sleep corrections)\n", sched->nr_sleep_corrections);
633+ sched->nr_sleep_corrections = 0;
613634 }
614635
615−static void test_calibrations(void)
636+static void test_calibrations(struct perf_sched *sched)
616637 {
617638 u64 T0, T1;
618639
619640 T0 = get_nsecs();
620− burn_nsecs(1e6);
641+ burn_nsecs(sched, 1e6);
621642 T1 = get_nsecs();
622643
623644 printf("the run test took %" PRIu64 " nsecs\n", T1 - T0);
@@ -629,236 +650,92 @@ static void test_calibrations(void) · 5 unmodified lines
629650 printf("the sleep test took %" PRIu64 " nsecs\n", T1 - T0);
630651 }
631652
632−#define FILL_FIELD(ptr, field, event, data) \
633− ptr.field = (typeof(ptr.field)) raw_field_value(event, #field, data)
634−
635−#define FILL_ARRAY(ptr, array, event, data) \
636−do { \
637− void *__array = raw_field_ptr(event, #array, data); \
638− memcpy(ptr.array, __array, sizeof(ptr.array)); \
639−} while(0)
640−
641−#define FILL_COMMON_FIELDS(ptr, event, data) \
642−do { \
643− FILL_FIELD(ptr, common_type, event, data); \
644− FILL_FIELD(ptr, common_flags, event, data); \
645− FILL_FIELD(ptr, common_preempt_count, event, data); \
646− FILL_FIELD(ptr, common_pid, event, data); \
647− FILL_FIELD(ptr, common_tgid, event, data); \
648−} while (0)
649−
650−
651−
652−struct trace_switch_event {
653− u32 size;
654−
655− u16 common_type;
656− u8 common_flags;
657− u8 common_preempt_count;
658− u32 common_pid;
659− u32 common_tgid;
660−
661− char prev_comm[16];
662− u32 prev_pid;
663− u32 prev_prio;
664− u64 prev_state;
665− char next_comm[16];
666− u32 next_pid;
667− u32 next_prio;
668−};
669−
670−struct trace_runtime_event {
671− u32 size;
672−
673− u16 common_type;
674− u8 common_flags;
675− u8 common_preempt_count;
676− u32 common_pid;
677− u32 common_tgid;
678−
679− char comm[16];
680− u32 pid;
681− u64 runtime;
682− u64 vruntime;
683−};
684−
685−struct trace_wakeup_event {
686− u32 size;
687−
688− u16 common_type;
689− u8 common_flags;
690− u8 common_preempt_count;
691− u32 common_pid;
692− u32 common_tgid;
693−
694− char comm[16];
695− u32 pid;
696−
697− u32 prio;
698− u32 success;
699− u32 cpu;
700−};
701−
702−struct trace_fork_event {
703− u32 size;
704−
705− u16 common_type;
706− u8 common_flags;
707− u8 common_preempt_count;
708− u32 common_pid;
709− u32 common_tgid;
710−
711− char parent_comm[16];
712− u32 parent_pid;
713− char child_comm[16];
714− u32 child_pid;
715−};
716−
717−struct trace_migrate_task_event {
718− u32 size;
719−
720− u16 common_type;
721− u8 common_flags;
722− u8 common_preempt_count;
723− u32 common_pid;
724− u32 common_tgid;
725−
726− char comm[16];
727− u32 pid;
728−
729− u32 prio;
730− u32 cpu;
731−};
732−
733−struct trace_sched_handler {
734− void (*switch_event)(struct trace_switch_event *,
735− struct machine *,
736− struct event_format *,
737− int cpu,
738− u64 timestamp,
739− struct thread *thread);
740−
741− void (*runtime_event)(struct trace_runtime_event *,
742− struct machine *,
743− struct event_format *,
744− int cpu,
745− u64 timestamp,
746− struct thread *thread);
747−
748− void (*wakeup_event)(struct trace_wakeup_event *,
749− struct machine *,
750− struct event_format *,
751− int cpu,
752− u64 timestamp,
753− struct thread *thread);
754−
755− void (*fork_event)(struct trace_fork_event *,
756− struct event_format *,
757− int cpu,
758− u64 timestamp,
759− struct thread *thread);
760−
761− void (*migrate_task_event)(struct trace_migrate_task_event *,
762− struct machine *machine,
763− struct event_format *,
764− int cpu,
765− u64 timestamp,
766− struct thread *thread);
767−};
768−
769−
770−static void
771−replay_wakeup_event(struct trace_wakeup_event *wakeup_event,
772− struct machine *machine __used,
773− struct event_format *event,
774− int cpu __used,
775− u64 timestamp __used,
776− struct thread *thread __used)
653+static int
654+replay_wakeup_event(struct perf_sched *sched,
655+ struct perf_evsel *evsel, struct perf_sample *sample,
656+ struct machine *machine __maybe_unused)
777657 {
658+ const char *comm = perf_evsel__strval(evsel, sample, "comm");
659+ const u32 pid = perf_evsel__intval(evsel, sample, "pid");
778660 struct task_desc *waker, *wakee;
779661
780662 if (verbose) {
781− printf("sched_wakeup event %p\n", event);
663+ printf("sched_wakeup event %p\n", evsel);
782664
783− printf(" ... pid %d woke up %s/%d\n",
784− wakeup_event->common_pid,
785− wakeup_event->comm,
786− wakeup_event->pid);
665+ printf(" ... pid %d woke up %s/%d\n", sample->tid, comm, pid);
787666 }
788667
789− waker = register_pid(wakeup_event->common_pid, "<unknown>");
790− wakee = register_pid(wakeup_event->pid, wakeup_event->comm);
668+ waker = register_pid(sched, sample->tid, "<unknown>");
669+ wakee = register_pid(sched, pid, comm);
791670
792− add_sched_event_wakeup(waker, timestamp, wakee);
671+ add_sched_event_wakeup(sched, waker, sample->time, wakee);
672+ return 0;
793673 }
794674
795−static u64 cpu_last_switched[MAX_CPUS];
796−
797−static void
798−replay_switch_event(struct trace_switch_event *switch_event,
799− struct machine *machine __used,
800− struct event_format *event,
801− int cpu,
802− u64 timestamp,
803− struct thread *thread __used)
675+static int replay_switch_event(struct perf_sched *sched,
676+ struct perf_evsel *evsel,
677+ struct perf_sample *sample,
678+ struct machine *machine __maybe_unused)
804679 {
805− struct task_desc *prev, __used *next;
806− u64 timestamp0;
680+ const char *prev_comm = perf_evsel__strval(evsel, sample, "prev_comm"),
681+ *next_comm = perf_evsel__strval(evsel, sample, "next_comm");
682+ const u32 prev_pid = perf_evsel__intval(evsel, sample, "prev_pid"),
683+ next_pid = perf_evsel__intval(evsel, sample, "next_pid");
684+ const u64 prev_state = perf_evsel__intval(evsel, sample, "prev_state");
685+ struct task_desc *prev, __maybe_unused *next;
686+ u64 timestamp0, timestamp = sample->time;
687+ int cpu = sample->cpu;
807688 s64 delta;
808689
809690 if (verbose)
810− printf("sched_switch event %p\n", event);
691+ printf("sched_switch event %p\n", evsel);
811692
812693 if (cpu >= MAX_CPUS || cpu < 0)
813− return;
694+ return 0;
814695
815− timestamp0 = cpu_last_switched[cpu];
696+ timestamp0 = sched->cpu_last_switched[cpu];
816697 if (timestamp0)
817698 delta = timestamp - timestamp0;
818699 else
819700 delta = 0;
820701
821− if (delta < 0)
822− die("hm, delta: %" PRIu64 " < 0 ?\n", delta);
823−
824− if (verbose) {
825− printf(" ... switch from %s/%d to %s/%d [ran %" PRIu64 " nsecs]\n",
826− switch_event->prev_comm, switch_event->prev_pid,
827− switch_event->next_comm, switch_event->next_pid,
828− delta);
702+ if (delta < 0) {
703+ pr_err("hm, delta: %" PRIu64 " < 0 ?\n", delta);
704+ return -1;
829705 }
830706
831− prev = register_pid(switch_event->prev_pid, switch_event->prev_comm);
832− next = register_pid(switch_event->next_pid, switch_event->next_comm);
707+ pr_debug(" ... switch from %s/%d to %s/%d [ran %" PRIu64 " nsecs]\n",
708+ prev_comm, prev_pid, next_comm, next_pid, delta);
833709
834− cpu_last_switched[cpu] = timestamp;
710+ prev = register_pid(sched, prev_pid, prev_comm);
711+ next = register_pid(sched, next_pid, next_comm);
835712
836− add_sched_event_run(prev, timestamp, delta);
837− add_sched_event_sleep(prev, timestamp, switch_event->prev_state);
838−}
713+ sched->cpu_last_switched[cpu] = timestamp;
839714
715+ add_sched_event_run(sched, prev, timestamp, delta);
716+ add_sched_event_sleep(sched, prev, timestamp, prev_state);
840717
841−static void
842−replay_fork_event(struct trace_fork_event *fork_event,
843− struct event_format *event,
844− int cpu __used,
845− u64 timestamp __used,
846− struct thread *thread __used)
718+ return 0;
719+}
720+
721+static int replay_fork_event(struct perf_sched *sched, struct perf_evsel *evsel,
722+ struct perf_sample *sample)
847723 {
724+ const char *parent_comm = perf_evsel__strval(evsel, sample, "parent_comm"),
725+ *child_comm = perf_evsel__strval(evsel, sample, "child_comm");
726+ const u32 parent_pid = perf_evsel__intval(evsel, sample, "parent_pid"),
727+ child_pid = perf_evsel__intval(evsel, sample, "child_pid");
728+
848729 if (verbose) {
849− printf("sched_fork event %p\n", event);
850− printf("... parent: %s/%d\n", fork_event->parent_comm, fork_event->parent_pid);
851− printf("... child: %s/%d\n", fork_event->child_comm, fork_event->child_pid);
730+ printf("sched_fork event %p\n", evsel);
731+ printf("... parent: %s/%d\n", parent_comm, parent_pid);
732+ printf("... child: %s/%d\n", child_comm, child_pid);
852733 }
853− register_pid(fork_event->parent_pid, fork_event->parent_comm);
854− register_pid(fork_event->child_pid, fork_event->child_comm);
855−}
856734
857−static struct trace_sched_handler replay_ops = {
858− .wakeup_event = replay_wakeup_event,
859− .switch_event = replay_switch_event,
860− .fork_event = replay_fork_event,
861−};
735+ register_pid(sched, parent_pid, parent_comm);
736+ register_pid(sched, child_pid, child_comm);
737+ return 0;
738+}
862739
863740 struct sort_dimension {
864741 const char *name;
@@ -866,8 +743,6 @@ struct sort_dimension { · 1 unmodified line
866743 struct list_head list;
867744 };
868745
869−static LIST_HEAD(cmp_pid);
870−
871746 static int
872747 thread_lat_cmp(struct list_head *list, struct work_atoms *l, struct work_atoms *r)
873748 {
@@ -936,43 +811,45 @@ __thread_latency_insert(struct rb_root *root, struct work_atoms *data, · 62 unmodified lines
936811 rb_insert_color(&data->node, root);
937812 }
938813
939−static void thread_atoms_insert(struct thread *thread)
814+static int thread_atoms_insert(struct perf_sched *sched, struct thread *thread)
940815 {
941816 struct work_atoms *atoms = zalloc(sizeof(*atoms));
942− if (!atoms)
943− die("No memory");
817+ if (!atoms) {
818+ pr_err("No memory at %s\n", __func__);
819+ return -1;
820+ }
944821
945822 atoms->thread = thread;
946823 INIT_LIST_HEAD(&atoms->work_list);
947− __thread_latency_insert(&atom_root, atoms, &cmp_pid);
824+ __thread_latency_insert(&sched->atom_root, atoms, &sched->cmp_pid);
825+ return 0;
948826 }
949827
950−static void
951−latency_fork_event(struct trace_fork_event *fork_event __used,
952− struct event_format *event __used,
953− int cpu __used,
954− u64 timestamp __used,
955− struct thread *thread __used)
828+static int latency_fork_event(struct perf_sched *sched __maybe_unused,
829+ struct perf_evsel *evsel __maybe_unused,
830+ struct perf_sample *sample __maybe_unused)
956831 {
957832 /* should insert the newcomer */
833+ return 0;
958834 }
959835
960−__used
961−static char sched_out_state(struct trace_switch_event *switch_event)
836+static char sched_out_state(u64 prev_state)
962837 {
963838 const char *str = TASK_STATE_TO_CHAR_STR;
964839
965− return str[switch_event->prev_state];
840+ return str[prev_state];
966841 }
967842
968−static void
843+static int
969844 add_sched_out_event(struct work_atoms *atoms,
970845 char run_state,
971846 u64 timestamp)
972847 {
973848 struct work_atom *atom = zalloc(sizeof(*atom));
974− if (!atom)
975− die("Non memory");
849+ if (!atom) {
850+ pr_err("Non memory at %s", __func__);
851+ return -1;
852+ }
976853
977854 atom->sched_out_time = timestamp;
978855
@@ -982,10 +859,12 @@ add_sched_out_event(struct work_atoms *atoms, · 3 unmodified lines
982859 }
983860
984861 list_add_tail(&atom->list, &atoms->work_list);
862+ return 0;
985863 }
986864
987865 static void
988−add_runtime_event(struct work_atoms *atoms, u64 delta, u64 timestamp __used)
866+add_runtime_event(struct work_atoms *atoms, u64 delta,
867+ u64 timestamp __maybe_unused)
989868 {
990869 struct work_atom *atom;
991870
@@ -1028,106 +907,128 @@ add_sched_in_event(struct work_atoms *atoms, u64 timestamp) · 36 unmodified lines
1028907 atoms->nb_atoms++;
1029908 }
1030909
1031−static void
1032−latency_switch_event(struct trace_switch_event *switch_event,
1033− struct machine *machine,
1034− struct event_format *event __used,
1035− int cpu,
1036− u64 timestamp,
1037− struct thread *thread __used)
910+static int latency_switch_event(struct perf_sched *sched,
911+ struct perf_evsel *evsel,
912+ struct perf_sample *sample,
913+ struct machine *machine)
1038914 {
915+ const u32 prev_pid = perf_evsel__intval(evsel, sample, "prev_pid"),
916+ next_pid = perf_evsel__intval(evsel, sample, "next_pid");
917+ const u64 prev_state = perf_evsel__intval(evsel, sample, "prev_state");
1039918 struct work_atoms *out_events, *in_events;
1040919 struct thread *sched_out, *sched_in;
1041− u64 timestamp0;
920+ u64 timestamp0, timestamp = sample->time;
921+ int cpu = sample->cpu;
1042922 s64 delta;
1043923
1044924 BUG_ON(cpu >= MAX_CPUS || cpu < 0);
1045925
1046− timestamp0 = cpu_last_switched[cpu];
1047− cpu_last_switched[cpu] = timestamp;
926+ timestamp0 = sched->cpu_last_switched[cpu];
927+ sched->cpu_last_switched[cpu] = timestamp;
1048928 if (timestamp0)
1049929 delta = timestamp - timestamp0;
1050930 else
1051931 delta = 0;
1052932
1053− if (delta < 0)
1054− die("hm, delta: %" PRIu64 " < 0 ?\n", delta);
1055−
933+ if (delta < 0) {
934+ pr_err("hm, delta: %" PRIu64 " < 0 ?\n", delta);
935+ return -1;
936+ }
1056937
1057− sched_out = machine__findnew_thread(machine, switch_event->prev_pid);
1058− sched_in = machine__findnew_thread(machine, switch_event->next_pid);
938+ sched_out = machine__findnew_thread(machine, prev_pid);
939+ sched_in = machine__findnew_thread(machine, next_pid);
1059940
1060− out_events = thread_atoms_search(&atom_root, sched_out, &cmp_pid);
941+ out_events = thread_atoms_search(&sched->atom_root, sched_out, &sched->cmp_pid);
1061942 if (!out_events) {
1062− thread_atoms_insert(sched_out);
1063− out_events = thread_atoms_search(&atom_root, sched_out, &cmp_pid);
1064− if (!out_events)
1065− die("out-event: Internal tree error");
943+ if (thread_atoms_insert(sched, sched_out))
944+ return -1;
945+ out_events = thread_atoms_search(&sched->atom_root, sched_out, &sched->cmp_pid);
946+ if (!out_events) {
947+ pr_err("out-event: Internal tree error");
948+ return -1;
949+ }
1066950 }
1067− add_sched_out_event(out_events, sched_out_state(switch_event), timestamp);
951+ if (add_sched_out_event(out_events, sched_out_state(prev_state), timestamp))
952+ return -1;
1068953
1069− in_events = thread_atoms_search(&atom_root, sched_in, &cmp_pid);
954+ in_events = thread_atoms_search(&sched->atom_root, sched_in, &sched->cmp_pid);
1070955 if (!in_events) {
1071− thread_atoms_insert(sched_in);
1072− in_events = thread_atoms_search(&atom_root, sched_in, &cmp_pid);
1073− if (!in_events)
1074− die("in-event: Internal tree error");
956+ if (thread_atoms_insert(sched, sched_in))
957+ return -1;
958+ in_events = thread_atoms_search(&sched->atom_root, sched_in, &sched->cmp_pid);
959+ if (!in_events) {
960+ pr_err("in-event: Internal tree error");
961+ return -1;
962+ }
1075963 /*
1076964 * Take came in we have not heard about yet,
1077965 * add in an initial atom in runnable state:
1078966 */
1079− add_sched_out_event(in_events, 'R', timestamp);
967+ if (add_sched_out_event(in_events, 'R', timestamp))
968+ return -1;
1080969 }
1081970 add_sched_in_event(in_events, timestamp);
971+
972+ return 0;
1082973 }
1083974
1084−static void
1085−latency_runtime_event(struct trace_runtime_event *runtime_event,
1086− struct machine *machine,
1087− struct event_format *event __used,
1088− int cpu,
1089− u64 timestamp,
1090− struct thread *this_thread __used)
975+static int latency_runtime_event(struct perf_sched *sched,
976+ struct perf_evsel *evsel,
977+ struct perf_sample *sample,
978+ struct machine *machine)
1091979 {
1092− struct thread *thread = machine__findnew_thread(machine, runtime_event->pid);
1093− struct work_atoms *atoms = thread_atoms_search(&atom_root, thread, &cmp_pid);
980+ const u32 pid = perf_evsel__intval(evsel, sample, "pid");
981+ const u64 runtime = perf_evsel__intval(evsel, sample, "runtime");
982+ struct thread *thread = machine__findnew_thread(machine, pid);
983+ struct work_atoms *atoms = thread_atoms_search(&sched->atom_root, thread, &sched->cmp_pid);
984+ u64 timestamp = sample->time;
985+ int cpu = sample->cpu;
1094986
1095987 BUG_ON(cpu >= MAX_CPUS || cpu < 0);
1096988 if (!atoms) {
1097− thread_atoms_insert(thread);
1098− atoms = thread_atoms_search(&atom_root, thread, &cmp_pid);
1099− if (!atoms)
1100− die("in-event: Internal tree error");
1101− add_sched_out_event(atoms, 'R', timestamp);
989+ if (thread_atoms_insert(sched, thread))
990+ return -1;
991+ atoms = thread_atoms_search(&sched->atom_root, thread, &sched->cmp_pid);
992+ if (!atoms) {
993+ pr_err("in-event: Internal tree error");
994+ return -1;
995+ }
996+ if (add_sched_out_event(atoms, 'R', timestamp))
997+ return -1;
1102998 }
1103999
1104− add_runtime_event(atoms, runtime_event->runtime, timestamp);
1000+ add_runtime_event(atoms, runtime, timestamp);
1001+ return 0;
11051002 }
11061003
1107−static void
1108−latency_wakeup_event(struct trace_wakeup_event *wakeup_event,
1109− struct machine *machine,
1110− struct event_format *__event __used,
1111− int cpu __used,
1112− u64 timestamp,
1113− struct thread *thread __used)
1004+static int latency_wakeup_event(struct perf_sched *sched,
1005+ struct perf_evsel *evsel,
1006+ struct perf_sample *sample,
1007+ struct machine *machine)
11141008 {
1009+ const u32 pid = perf_evsel__intval(evsel, sample, "pid"),
1010+ success = perf_evsel__intval(evsel, sample, "success");
11151011 struct work_atoms *atoms;
11161012 struct work_atom *atom;
11171013 struct thread *wakee;
1014+ u64 timestamp = sample->time;
11181015
11191016 /* Note for later, it may be interesting to observe the failing cases */
1120− if (!wakeup_event->success)
1121− return;
1017+ if (!success)
1018+ return 0;
11221019
1123− wakee = machine__findnew_thread(machine, wakeup_event->pid);
1124− atoms = thread_atoms_search(&atom_root, wakee, &cmp_pid);
1020+ wakee = machine__findnew_thread(machine, pid);
1021+ atoms = thread_atoms_search(&sched->atom_root, wakee, &sched->cmp_pid);
11251022 if (!atoms) {
1126− thread_atoms_insert(wakee);
1127− atoms = thread_atoms_search(&atom_root, wakee, &cmp_pid);
1128− if (!atoms)
1129− die("wakeup-event: Internal tree error");
1130− add_sched_out_event(atoms, 'S', timestamp);
1023+ if (thread_atoms_insert(sched, wakee))
1024+ return -1;
1025+ atoms = thread_atoms_search(&sched->atom_root, wakee, &sched->cmp_pid);
1026+ if (!atoms) {
1027+ pr_err("wakeup-event: Internal tree error");
1028+ return -1;
1029+ }
1030+ if (add_sched_out_event(atoms, 'S', timestamp))
1031+ return -1;
11311032 }
11321033
11331034 BUG_ON(list_empty(&atoms->work_list));
@@ -1139,27 +1040,27 @@ latency_wakeup_event(struct trace_wakeup_event *wakeup_event, · 5 unmodified lines
11391040 * one CPU, or are only looking at only one, so don't
11401041 * make useless noise.
11411042 */
1142− if (profile_cpu == -1 && atom->state != THREAD_SLEEPING)
1143− nr_state_machine_bugs++;
1043+ if (sched->profile_cpu == -1 && atom->state != THREAD_SLEEPING)
1044+ sched->nr_state_machine_bugs++;
11441045
1145− nr_timestamps++;
1046+ sched->nr_timestamps++;
11461047 if (atom->sched_out_time > timestamp) {
1147− nr_unordered_timestamps++;
1148− return;
1048+ sched->nr_unordered_timestamps++;
1049+ return 0;
11491050 }
11501051
11511052 atom->state = THREAD_WAIT_CPU;
11521053 atom->wake_up_time = timestamp;
1054+ return 0;
11531055 }
11541056
1155−static void
1156−latency_migrate_task_event(struct trace_migrate_task_event *migrate_task_event,
1157− struct machine *machine,
1158− struct event_format *__event __used,
1159− int cpu __used,
1160− u64 timestamp,
1161− struct thread *thread __used)
1057+static int latency_migrate_task_event(struct perf_sched *sched,
1058+ struct perf_evsel *evsel,
1059+ struct perf_sample *sample,
1060+ struct machine *machine)
11621061 {
1062+ const u32 pid = perf_evsel__intval(evsel, sample, "pid");
1063+ u64 timestamp = sample->time;
11631064 struct work_atoms *atoms;
11641065 struct work_atom *atom;
11651066 struct thread *migrant;
@@ -1167,18 +1068,22 @@ latency_migrate_task_event(struct trace_migrate_task_event *migrate_task_event, · 1 unmodified line
11671068 /*
11681069 * Only need to worry about migration when profiling one CPU.
11691070 */
1170− if (profile_cpu == -1)
1171− return;
1071+ if (sched->profile_cpu == -1)
1072+ return 0;
11721073
1173− migrant = machine__findnew_thread(machine, migrate_task_event->pid);
1174− atoms = thread_atoms_search(&atom_root, migrant, &cmp_pid);
1074+ migrant = machine__findnew_thread(machine, pid);
1075+ atoms = thread_atoms_search(&sched->atom_root, migrant, &sched->cmp_pid);
11751076 if (!atoms) {
1176− thread_atoms_insert(migrant);
1177− register_pid(migrant->pid, migrant->comm);
1178− atoms = thread_atoms_search(&atom_root, migrant, &cmp_pid);
1179− if (!atoms)
1180− die("migration-event: Internal tree error");
1181− add_sched_out_event(atoms, 'R', timestamp);
1077+ if (thread_atoms_insert(sched, migrant))
1078+ return -1;
1079+ register_pid(sched, migrant->pid, migrant->comm);
1080+ atoms = thread_atoms_search(&sched->atom_root, migrant, &sched->cmp_pid);
1081+ if (!atoms) {
1082+ pr_err("migration-event: Internal tree error");
1083+ return -1;
1084+ }
1085+ if (add_sched_out_event(atoms, 'R', timestamp))
1086+ return -1;
11821087 }
11831088
11841089 BUG_ON(list_empty(&atoms->work_list));
@@ -1186,21 +1091,15 @@ latency_migrate_task_event(struct trace_migrate_task_event *migrate_task_event, · 1 unmodified line
11861091 atom = list_entry(atoms->work_list.prev, struct work_atom, list);
11871092 atom->sched_in_time = atom->sched_out_time = atom->wake_up_time = timestamp;
11881093
1189− nr_timestamps++;
1094+ sched->nr_timestamps++;
11901095
11911096 if (atom->sched_out_time > timestamp)
1192− nr_unordered_timestamps++;
1193−}
1097+ sched->nr_unordered_timestamps++;
11941098
1195−static struct trace_sched_handler lat_ops = {
1196− .wakeup_event = latency_wakeup_event,
1197− .switch_event = latency_switch_event,
1198− .runtime_event = latency_runtime_event,
1199− .fork_event = latency_fork_event,
1200− .migrate_task_event = latency_migrate_task_event,
1201−};
1099+ return 0;
1100+}
12021101
1203−static void output_lat_thread(struct work_atoms *work_list)
1102+static void output_lat_thread(struct perf_sched *sched, struct work_atoms *work_list)
12041103 {
12051104 int i;
12061105 int ret;
@@ -1214,8 +1113,8 @@ static void output_lat_thread(struct work_atoms *work_list) · 7 unmodified lines
12141113 if (!strcmp(work_list->thread->comm, "swapper"))
12151114 return;
12161115
1217− all_runtime += work_list->total_runtime;
1218− all_count += work_list->nb_atoms;
1116+ sched->all_runtime += work_list->total_runtime;
1117+ sched->all_count += work_list->nb_atoms;
12191118
12201119 ret = printf(" %s:%d ", work_list->thread->comm, work_list->thread->pid);
12211120
@@ -1241,11 +1140,6 @@ static int pid_cmp(struct work_atoms *l, struct work_atoms *r) · 19 unmodified lines
12411140 return 0;
12421141 }
12431142
1244−static struct sort_dimension pid_sort_dimension = {
1245− .name = "pid",
1246− .cmp = pid_cmp,
1247−};
1248−
12491143 static int avg_cmp(struct work_atoms *l, struct work_atoms *r)
12501144 {
12511145 u64 avgl, avgr;
@@ -1267,11 +1161,6 @@ static int avg_cmp(struct work_atoms *l, struct work_atoms *r) · 15 unmodified lines
12671161 return 0;
12681162 }
12691163
1270−static struct sort_dimension avg_sort_dimension = {
1271− .name = "avg",
1272− .cmp = avg_cmp,
1273−};
1274−
12751164 static int max_cmp(struct work_atoms *l, struct work_atoms *r)
12761165 {
12771166 if (l->max_lat < r->max_lat)
@@ -1282,11 +1171,6 @@ static int max_cmp(struct work_atoms *l, struct work_atoms *r) · 4 unmodified lines
12821171 return 0;
12831172 }
12841173
1285−static struct sort_dimension max_sort_dimension = {
1286− .name = "max",
1287− .cmp = max_cmp,
1288−};
1289−
12901174 static int switch_cmp(struct work_atoms *l, struct work_atoms *r)
12911175 {
12921176 if (l->nb_atoms < r->nb_atoms)
@@ -1297,11 +1181,6 @@ static int switch_cmp(struct work_atoms *l, struct work_atoms *r) · 4 unmodified lines
12971181 return 0;
12981182 }
12991183
1300−static struct sort_dimension switch_sort_dimension = {
1301− .name = "switch",
1302− .cmp = switch_cmp,
1303−};
1304−
13051184 static int runtime_cmp(struct work_atoms *l, struct work_atoms *r)
13061185 {
13071186 if (l->total_runtime < r->total_runtime)
@@ -1312,28 +1191,38 @@ static int runtime_cmp(struct work_atoms *l, struct work_atoms *r) · 4 unmodified lines
13121191 return 0;
13131192 }
13141193
1315−static struct sort_dimension runtime_sort_dimension = {
1316− .name = "runtime",
1317− .cmp = runtime_cmp,
1318−};
1319−
1320−static struct sort_dimension *available_sorts[] = {
1321− &pid_sort_dimension,
1322− &avg_sort_dimension,
1323− &max_sort_dimension,
1324− &switch_sort_dimension,
1325− &runtime_sort_dimension,
1326−};
1327−
1328−#define NB_AVAILABLE_SORTS (int)(sizeof(available_sorts) / sizeof(struct sort_dimension *))
1329−
1330−static LIST_HEAD(sort_list);
1331−
13321194 static int sort_dimension__add(const char *tok, struct list_head *list)
13331195 {
1334− int i;
1196+ size_t i;
1197+ static struct sort_dimension avg_sort_dimension = {
1198+ .name = "avg",
1199+ .cmp = avg_cmp,
1200+ };
1201+ static struct sort_dimension max_sort_dimension = {
1202+ .name = "max",
1203+ .cmp = max_cmp,
1204+ };
1205+ static struct sort_dimension pid_sort_dimension = {
1206+ .name = "pid",
1207+ .cmp = pid_cmp,
1208+ };
1209+ static struct sort_dimension runtime_sort_dimension = {
1210+ .name = "runtime",
1211+ .cmp = runtime_cmp,
1212+ };
1213+ static struct sort_dimension switch_sort_dimension = {
1214+ .name = "switch",
1215+ .cmp = switch_cmp,
1216+ };
1217+ struct sort_dimension *available_sorts[] = {
1218+ &pid_sort_dimension,
1219+ &avg_sort_dimension,
1220+ &max_sort_dimension,
1221+ &switch_sort_dimension,
1222+ &runtime_sort_dimension,
1223+ };
13351224
1336− for (i = 0; i < NB_AVAILABLE_SORTS; i++) {
1225+ for (i = 0; i < ARRAY_SIZE(available_sorts); i++) {
13371226 if (!strcmp(available_sorts[i]->name, tok)) {
13381227 list_add_tail(&available_sorts[i]->list, list);
13391228
@@ -1344,126 +1233,97 @@ static int sort_dimension__add(const char *tok, struct list_head *list) · 4 unmodified lines
13441233 return -1;
13451234 }
13461235
1347−static void setup_sorting(void);
1348−
1349−static void sort_lat(void)
1236+static void perf_sched__sort_lat(struct perf_sched *sched)
13501237 {
13511238 struct rb_node *node;
13521239
13531240 for (;;) {
13541241 struct work_atoms *data;
1355− node = rb_first(&atom_root);
1242+ node = rb_first(&sched->atom_root);
13561243 if (!node)
13571244 break;
13581245
1359− rb_erase(node, &atom_root);
1246+ rb_erase(node, &sched->atom_root);
13601247 data = rb_entry(node, struct work_atoms, node);
1361− __thread_latency_insert(&sorted_atom_root, data, &sort_list);
1248+ __thread_latency_insert(&sched->sorted_atom_root, data, &sched->sort_list);
13621249 }
13631250 }
13641251
1365−static struct trace_sched_handler *trace_handler;
1366−
1367−static void
1368−process_sched_wakeup_event(struct perf_tool *tool __used,
1369− struct event_format *event,
1370− struct perf_sample *sample,
1371− struct machine *machine,
1372− struct thread *thread)
1252+static int process_sched_wakeup_event(struct perf_tool *tool,
1253+ struct perf_evsel *evsel,
1254+ struct perf_sample *sample,
1255+ struct machine *machine)
13731256 {
1374− void *data = sample->raw_data;
1375− struct trace_wakeup_event wakeup_event;
1376−
1377− FILL_COMMON_FIELDS(wakeup_event, event, data);
1257+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
13781258
1379− FILL_ARRAY(wakeup_event, comm, event, data);
1380− FILL_FIELD(wakeup_event, pid, event, data);
1381− FILL_FIELD(wakeup_event, prio, event, data);
1382− FILL_FIELD(wakeup_event, success, event, data);
1383− FILL_FIELD(wakeup_event, cpu, event, data);
1259+ if (sched->tp_handler->wakeup_event)
1260+ return sched->tp_handler->wakeup_event(sched, evsel, sample, machine);
13841261
1385− if (trace_handler->wakeup_event)
1386− trace_handler->wakeup_event(&wakeup_event, machine, event,
1387− sample->cpu, sample->time, thread);
1262+ return 0;
13881263 }
13891264
1390−/*
1391− * Track the current task - that way we can know whether there's any
1392− * weird events, such as a task being switched away that is not current.
1393− */
1394−static int max_cpu;
1395−
1396−static u32 curr_pid[MAX_CPUS] = { [0 ... MAX_CPUS-1] = -1 };
1397−
1398−static struct thread *curr_thread[MAX_CPUS];
1399−
1400−static char next_shortname1 = 'A';
1401−static char next_shortname2 = '0';
1402−
1403−static void
1404−map_switch_event(struct trace_switch_event *switch_event,
1405− struct machine *machine,
1406− struct event_format *event __used,
1407− int this_cpu,
1408− u64 timestamp,
1409− struct thread *thread __used)
1265+static int map_switch_event(struct perf_sched *sched, struct perf_evsel *evsel,
1266+ struct perf_sample *sample, struct machine *machine)
14101267 {
1411− struct thread *sched_out __used, *sched_in;
1268+ const u32 prev_pid = perf_evsel__intval(evsel, sample, "prev_pid"),
1269+ next_pid = perf_evsel__intval(evsel, sample, "next_pid");
1270+ struct thread *sched_out __maybe_unused, *sched_in;
14121271 int new_shortname;
1413− u64 timestamp0;
1272+ u64 timestamp0, timestamp = sample->time;
14141273 s64 delta;
1415− int cpu;
1274+ int cpu, this_cpu = sample->cpu;
14161275
14171276 BUG_ON(this_cpu >= MAX_CPUS || this_cpu < 0);
14181277
1419− if (this_cpu > max_cpu)
1420− max_cpu = this_cpu;
1278+ if (this_cpu > sched->max_cpu)
1279+ sched->max_cpu = this_cpu;
14211280
1422− timestamp0 = cpu_last_switched[this_cpu];
1423− cpu_last_switched[this_cpu] = timestamp;
1281+ timestamp0 = sched->cpu_last_switched[this_cpu];
1282+ sched->cpu_last_switched[this_cpu] = timestamp;
14241283 if (timestamp0)
14251284 delta = timestamp - timestamp0;
14261285 else
14271286 delta = 0;
14281287
1429− if (delta < 0)
1430− die("hm, delta: %" PRIu64 " < 0 ?\n", delta);
1431−
1288+ if (delta < 0) {
1289+ pr_err("hm, delta: %" PRIu64 " < 0 ?\n", delta);
1290+ return -1;
1291+ }
14321292
1433− sched_out = machine__findnew_thread(machine, switch_event->prev_pid);
1434− sched_in = machine__findnew_thread(machine, switch_event->next_pid);
1293+ sched_out = machine__findnew_thread(machine, prev_pid);
1294+ sched_in = machine__findnew_thread(machine, next_pid);
14351295
1436− curr_thread[this_cpu] = sched_in;
1296+ sched->curr_thread[this_cpu] = sched_in;
14371297
14381298 printf(" ");
14391299
14401300 new_shortname = 0;
14411301 if (!sched_in->shortname[0]) {
1442− sched_in->shortname[0] = next_shortname1;
1443− sched_in->shortname[1] = next_shortname2;
1302+ sched_in->shortname[0] = sched->next_shortname1;
1303+ sched_in->shortname[1] = sched->next_shortname2;
14441304
1445− if (next_shortname1 < 'Z') {
1446− next_shortname1++;
1305+ if (sched->next_shortname1 < 'Z') {
1306+ sched->next_shortname1++;
14471307 } else {
1448− next_shortname1='A';
1449− if (next_shortname2 < '9') {
1450− next_shortname2++;
1308+ sched->next_shortname1='A';
1309+ if (sched->next_shortname2 < '9') {
1310+ sched->next_shortname2++;
14511311 } else {
1452− next_shortname2='0';
1312+ sched->next_shortname2='0';
14531313 }
14541314 }
14551315 new_shortname = 1;
14561316 }
14571317
1458− for (cpu = 0; cpu <= max_cpu; cpu++) {
1318+ for (cpu = 0; cpu <= sched->max_cpu; cpu++) {
14591319 if (cpu != this_cpu)
14601320 printf(" ");
14611321 else
14621322 printf("*");
14631323
1464− if (curr_thread[cpu]) {
1465− if (curr_thread[cpu]->pid)
1466− printf("%2s ", curr_thread[cpu]->shortname);
1324+ if (sched->curr_thread[cpu]) {
1325+ if (sched->curr_thread[cpu]->pid)
1326+ printf("%2s ", sched->curr_thread[cpu]->shortname);
14671327 else
14681328 printf(". ");
14691329 } else
@@ -1477,134 +1337,97 @@ map_switch_event(struct trace_switch_event *switch_event, · 7 unmodified lines
14771337 } else {
14781338 printf("\n");
14791339 }
1340+
1341+ return 0;
14801342 }
14811343
1482−static void
1483−process_sched_switch_event(struct perf_tool *tool __used,
1484− struct event_format *event,
1485− struct perf_sample *sample,
1486− struct machine *machine,
1487− struct thread *thread)
1344+static int process_sched_switch_event(struct perf_tool *tool,
1345+ struct perf_evsel *evsel,
1346+ struct perf_sample *sample,
1347+ struct machine *machine)
14881348 {
1489− int this_cpu = sample->cpu;
1490− void *data = sample->raw_data;
1491− struct trace_switch_event switch_event;
1492−
1493− FILL_COMMON_FIELDS(switch_event, event, data);
1494−
1495− FILL_ARRAY(switch_event, prev_comm, event, data);
1496− FILL_FIELD(switch_event, prev_pid, event, data);
1497− FILL_FIELD(switch_event, prev_prio, event, data);
1498− FILL_FIELD(switch_event, prev_state, event, data);
1499− FILL_ARRAY(switch_event, next_comm, event, data);
1500− FILL_FIELD(switch_event, next_pid, event, data);
1501− FILL_FIELD(switch_event, next_prio, event, data);
1349+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
1350+ int this_cpu = sample->cpu, err = 0;
1351+ u32 prev_pid = perf_evsel__intval(evsel, sample, "prev_pid"),
1352+ next_pid = perf_evsel__intval(evsel, sample, "next_pid");
15021353
1503− if (curr_pid[this_cpu] != (u32)-1) {
1354+ if (sched->curr_pid[this_cpu] != (u32)-1) {
15041355 /*
15051356 * Are we trying to switch away a PID that is
15061357 * not current?
15071358 */
1508− if (curr_pid[this_cpu] != switch_event.prev_pid)
1509− nr_context_switch_bugs++;
1359+ if (sched->curr_pid[this_cpu] != prev_pid)
1360+ sched->nr_context_switch_bugs++;
15101361 }
1511− if (trace_handler->switch_event)
1512− trace_handler->switch_event(&switch_event, machine, event,
1513− this_cpu, sample->time, thread);
15141362
1515− curr_pid[this_cpu] = switch_event.next_pid;
1363+ if (sched->tp_handler->switch_event)
1364+ err = sched->tp_handler->switch_event(sched, evsel, sample, machine);
1365+
1366+ sched->curr_pid[this_cpu] = next_pid;
1367+ return err;
15161368 }
15171369
1518−static void
1519−process_sched_runtime_event(struct perf_tool *tool __used,
1520− struct event_format *event,
1521− struct perf_sample *sample,
1522− struct machine *machine,
1523− struct thread *thread)
1370+static int process_sched_runtime_event(struct perf_tool *tool,
1371+ struct perf_evsel *evsel,
1372+ struct perf_sample *sample,
1373+ struct machine *machine)
15241374 {
1525− void *data = sample->raw_data;
1526− struct trace_runtime_event runtime_event;
1375+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
15271376
1528− FILL_ARRAY(runtime_event, comm, event, data);
1529− FILL_FIELD(runtime_event, pid, event, data);
1530− FILL_FIELD(runtime_event, runtime, event, data);
1531− FILL_FIELD(runtime_event, vruntime, event, data);
1377+ if (sched->tp_handler->runtime_event)
1378+ return sched->tp_handler->runtime_event(sched, evsel, sample, machine);
15321379
1533− if (trace_handler->runtime_event)
1534− trace_handler->runtime_event(&runtime_event, machine, event,
1535− sample->cpu, sample->time, thread);
1380+ return 0;
15361381 }
15371382
1538−static void
1539−process_sched_fork_event(struct perf_tool *tool __used,
1540− struct event_format *event,
1541− struct perf_sample *sample,
1542− struct machine *machine __used,
1543− struct thread *thread)
1383+static int process_sched_fork_event(struct perf_tool *tool,
1384+ struct perf_evsel *evsel,
1385+ struct perf_sample *sample,
1386+ struct machine *machine __maybe_unused)
15441387 {
1545− void *data = sample->raw_data;
1546− struct trace_fork_event fork_event;
1547−
1548− FILL_COMMON_FIELDS(fork_event, event, data);
1388+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
15491389
1550− FILL_ARRAY(fork_event, parent_comm, event, data);
1551− FILL_FIELD(fork_event, parent_pid, event, data);
1552− FILL_ARRAY(fork_event, child_comm, event, data);
1553− FILL_FIELD(fork_event, child_pid, event, data);
1390+ if (sched->tp_handler->fork_event)
1391+ return sched->tp_handler->fork_event(sched, evsel, sample);
15541392
1555− if (trace_handler->fork_event)
1556− trace_handler->fork_event(&fork_event, event,
1557− sample->cpu, sample->time, thread);
1393+ return 0;
15581394 }
15591395
1560−static void
1561−process_sched_exit_event(struct perf_tool *tool __used,
1562− struct event_format *event,
1563− struct perf_sample *sample __used,
1564− struct machine *machine __used,
1565− struct thread *thread __used)
1396+static int process_sched_exit_event(struct perf_tool *tool __maybe_unused,
1397+ struct perf_evsel *evsel,
1398+ struct perf_sample *sample __maybe_unused,
1399+ struct machine *machine __maybe_unused)
15661400 {
1567− if (verbose)
1568− printf("sched_exit event %p\n", event);
1401+ pr_debug("sched_exit event %p\n", evsel);
1402+ return 0;
15691403 }
15701404
1571−static void
1572−process_sched_migrate_task_event(struct perf_tool *tool __used,
1573− struct event_format *event,
1574− struct perf_sample *sample,
1575− struct machine *machine,
1576− struct thread *thread)
1405+static int process_sched_migrate_task_event(struct perf_tool *tool,
1406+ struct perf_evsel *evsel,
1407+ struct perf_sample *sample,
1408+ struct machine *machine)
15771409 {
1578− void *data = sample->raw_data;
1579− struct trace_migrate_task_event migrate_task_event;
1580−
1581− FILL_COMMON_FIELDS(migrate_task_event, event, data);
1410+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
15821411
1583− FILL_ARRAY(migrate_task_event, comm, event, data);
1584− FILL_FIELD(migrate_task_event, pid, event, data);
1585− FILL_FIELD(migrate_task_event, prio, event, data);
1586− FILL_FIELD(migrate_task_event, cpu, event, data);
1412+ if (sched->tp_handler->migrate_task_event)
1413+ return sched->tp_handler->migrate_task_event(sched, evsel, sample, machine);
15871414
1588− if (trace_handler->migrate_task_event)
1589− trace_handler->migrate_task_event(&migrate_task_event, machine,
1590− event, sample->cpu,
1591− sample->time, thread);
1415+ return 0;
15921416 }
15931417
1594−typedef void (*tracepoint_handler)(struct perf_tool *tool, struct event_format *event,
1595− struct perf_sample *sample,
1596− struct machine *machine,
1597− struct thread *thread);
1418+typedef int (*tracepoint_handler)(struct perf_tool *tool,
1419+ struct perf_evsel *evsel,
1420+ struct perf_sample *sample,
1421+ struct machine *machine);
15981422
1599−static int perf_sched__process_tracepoint_sample(struct perf_tool *tool,
1600− union perf_event *event __used,
1423+static int perf_sched__process_tracepoint_sample(struct perf_tool *tool __maybe_unused,
1424+ union perf_event *event __maybe_unused,
16011425 struct perf_sample *sample,
16021426 struct perf_evsel *evsel,
16031427 struct machine *machine)
16041428 {
1605− struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
1606− struct pevent *pevent = sched->session->pevent;
16071429 struct thread *thread = machine__findnew_thread(machine, sample->pid);
1430+ int err = 0;
16081431
16091432 if (thread == NULL) {
16101433 pr_debug("problem processing %s event, skipping it.\n",
@@ -1617,30 +1440,15 @@ static int perf_sched__process_tracepoint_sample(struct perf_tool *tool, · 6 unmodified lines
16171440
16181441 if (evsel->handler.func != NULL) {
16191442 tracepoint_handler f = evsel->handler.func;
1620−
1621− if (evsel->handler.data == NULL)
1622− evsel->handler.data = pevent_find_event(pevent,
1623− evsel->attr.config);
1624−
1625− f(tool, evsel->handler.data, sample, machine, thread);
1443+ err = f(tool, evsel, sample, machine);
16261444 }
16271445
1628− return 0;
1446+ return err;
16291447 }
16301448
1631−static struct perf_sched perf_sched = {
1632− .tool = {
1633− .sample = perf_sched__process_tracepoint_sample,
1634− .comm = perf_event__process_comm,
1635− .lost = perf_event__process_lost,
1636− .fork = perf_event__process_task,
1637− .ordered_samples = true,
1638− },
1639−};
1640−
1641−static void read_events(bool destroy, struct perf_session **psession)
1449+static int perf_sched__read_events(struct perf_sched *sched, bool destroy,
1450+ struct perf_session **psession)
16421451 {
1643− int err = -EINVAL;
16441452 const struct perf_evsel_str_handler handlers[] = {
16451453 { "sched:sched_switch", process_sched_switch_event, },
16461454 { "sched:sched_stat_runtime", process_sched_runtime_event, },
@@ -1652,24 +1460,25 @@ static void read_events(bool destroy, struct perf_session **psession) · 5 unmodified lines
16521460 };
16531461 struct perf_session *session;
16541462
1655− session = perf_session__new(input_name, O_RDONLY, 0, false,
1656− &perf_sched.tool);
1657− if (session == NULL)
1658− die("No Memory");
1659−
1660− perf_sched.session = session;
1463+ session = perf_session__new(sched->input_name, O_RDONLY, 0, false, &sched->tool);
1464+ if (session == NULL) {
1465+ pr_debug("No Memory for session\n");
1466+ return -1;
1467+ }
16611468
1662− err = perf_session__set_tracepoints_handlers(session, handlers);
1663− assert(err == 0);
1469+ if (perf_session__set_tracepoints_handlers(session, handlers))
1470+ goto out_delete;
16641471
16651472 if (perf_session__has_traces(session, "record -R")) {
1666− err = perf_session__process_events(session, &perf_sched.tool);
1667− if (err)
1668− die("Failed to process events, error %d", err);
1473+ int err = perf_session__process_events(session, &sched->tool);
1474+ if (err) {
1475+ pr_err("Failed to process events, error %d", err);
1476+ goto out_delete;
1477+ }
16691478
1670− nr_events = session->hists.stats.nr_events[0];
1671− nr_lost_events = session->hists.stats.total_lost;
1672− nr_lost_chunks = session->hists.stats.nr_events[PERF_RECORD_LOST];
1479+ sched->nr_events = session->hists.stats.nr_events[0];
1480+ sched->nr_lost_events = session->hists.stats.total_lost;
1481+ sched->nr_lost_chunks = session->hists.stats.nr_events[PERF_RECORD_LOST];
16731482 }
16741483
16751484 if (destroy)
@@ -1677,208 +1486,166 @@ static void read_events(bool destroy, struct perf_session **psession) · 1 unmodified line
16771486
16781487 if (psession)
16791488 *psession = session;
1489+
1490+ return 0;
1491+
1492+out_delete:
1493+ perf_session__delete(session);
1494+ return -1;
16801495 }
16811496
1682−static void print_bad_events(void)
1497+static void print_bad_events(struct perf_sched *sched)
16831498 {
1684− if (nr_unordered_timestamps && nr_timestamps) {
1499+ if (sched->nr_unordered_timestamps && sched->nr_timestamps) {
16851500 printf(" INFO: %.3f%% unordered timestamps (%ld out of %ld)\n",
1686− (double)nr_unordered_timestamps/(double)nr_timestamps*100.0,
1687− nr_unordered_timestamps, nr_timestamps);
1501+ (double)sched->nr_unordered_timestamps/(double)sched->nr_timestamps*100.0,
1502+ sched->nr_unordered_timestamps, sched->nr_timestamps);
16881503 }
1689− if (nr_lost_events && nr_events) {
1504+ if (sched->nr_lost_events && sched->nr_events) {
16901505 printf(" INFO: %.3f%% lost events (%ld out of %ld, in %ld chunks)\n",
1691− (double)nr_lost_events/(double)nr_events*100.0,
1692− nr_lost_events, nr_events, nr_lost_chunks);
1506+ (double)sched->nr_lost_events/(double)sched->nr_events * 100.0,
1507+ sched->nr_lost_events, sched->nr_events, sched->nr_lost_chunks);
16931508 }
1694− if (nr_state_machine_bugs && nr_timestamps) {
1509+ if (sched->nr_state_machine_bugs && sched->nr_timestamps) {
16951510 printf(" INFO: %.3f%% state machine bugs (%ld out of %ld)",
1696− (double)nr_state_machine_bugs/(double)nr_timestamps*100.0,
1697− nr_state_machine_bugs, nr_timestamps);
1698− if (nr_lost_events)
1511+ (double)sched->nr_state_machine_bugs/(double)sched->nr_timestamps*100.0,
1512+ sched->nr_state_machine_bugs, sched->nr_timestamps);
1513+ if (sched->nr_lost_events)
16991514 printf(" (due to lost events?)");
17001515 printf("\n");
17011516 }
1702− if (nr_context_switch_bugs && nr_timestamps) {
1517+ if (sched->nr_context_switch_bugs && sched->nr_timestamps) {
17031518 printf(" INFO: %.3f%% context switch bugs (%ld out of %ld)",
1704− (double)nr_context_switch_bugs/(double)nr_timestamps*100.0,
1705− nr_context_switch_bugs, nr_timestamps);
1706− if (nr_lost_events)
1519+ (double)sched->nr_context_switch_bugs/(double)sched->nr_timestamps*100.0,
1520+ sched->nr_context_switch_bugs, sched->nr_timestamps);
1521+ if (sched->nr_lost_events)
17071522 printf(" (due to lost events?)");
17081523 printf("\n");
17091524 }
17101525 }
17111526
1712−static void __cmd_lat(void)
1527+static int perf_sched__lat(struct perf_sched *sched)
17131528 {
17141529 struct rb_node *next;
17151530 struct perf_session *session;
17161531
17171532 setup_pager();
1718− read_events(false, &session);
1719− sort_lat();
1533+ if (perf_sched__read_events(sched, false, &session))
1534+ return -1;
1535+ perf_sched__sort_lat(sched);
17201536
17211537 printf("\n ---------------------------------------------------------------------------------------------------------------\n");
17221538 printf(" Task | Runtime ms | Switches | Average delay ms | Maximum delay ms | Maximum delay at |\n");
17231539 printf(" ---------------------------------------------------------------------------------------------------------------\n");
17241540
1725− next = rb_first(&sorted_atom_root);
1541+ next = rb_first(&sched->sorted_atom_root);
17261542
17271543 while (next) {
17281544 struct work_atoms *work_list;
17291545
17301546 work_list = rb_entry(next, struct work_atoms, node);
1731− output_lat_thread(work_list);
1547+ output_lat_thread(sched, work_list);
17321548 next = rb_next(next);
17331549 }
17341550
17351551 printf(" -----------------------------------------------------------------------------------------\n");
17361552 printf(" TOTAL: |%11.3f ms |%9" PRIu64 " |\n",
1737− (double)all_runtime/1e6, all_count);
1553+ (double)sched->all_runtime / 1e6, sched->all_count);
17381554
17391555 printf(" ---------------------------------------------------\n");
17401556
1741− print_bad_events();
1557+ print_bad_events(sched);
17421558 printf("\n");
17431559
17441560 perf_session__delete(session);
1561+ return 0;
17451562 }
17461563
1747−static struct trace_sched_handler map_ops = {
1748− .wakeup_event = NULL,
1749− .switch_event = map_switch_event,
1750− .runtime_event = NULL,
1751− .fork_event = NULL,
1752−};
1753−
1754−static void __cmd_map(void)
1564+static int perf_sched__map(struct perf_sched *sched)
17551565 {
1756− max_cpu = sysconf(_SC_NPROCESSORS_CONF);
1566+ sched->max_cpu = sysconf(_SC_NPROCESSORS_CONF);
17571567
17581568 setup_pager();
1759− read_events(true, NULL);
1760− print_bad_events();
1569+ if (perf_sched__read_events(sched, true, NULL))
1570+ return -1;
1571+ print_bad_events(sched);
1572+ return 0;
17611573 }
17621574
1763−static void __cmd_replay(void)
1575+static int perf_sched__replay(struct perf_sched *sched)
17641576 {
17651577 unsigned long i;
17661578
1767− calibrate_run_measurement_overhead();
1768− calibrate_sleep_measurement_overhead();
1579+ calibrate_run_measurement_overhead(sched);
1580+ calibrate_sleep_measurement_overhead(sched);
17691581
1770− test_calibrations();
1582+ test_calibrations(sched);
17711583
1772− read_events(true, NULL);
1584+ if (perf_sched__read_events(sched, true, NULL))
1585+ return -1;
17731586
1774− printf("nr_run_events: %ld\n", nr_run_events);
1775− printf("nr_sleep_events: %ld\n", nr_sleep_events);
1776− printf("nr_wakeup_events: %ld\n", nr_wakeup_events);
1587+ printf("nr_run_events: %ld\n", sched->nr_run_events);
1588+ printf("nr_sleep_events: %ld\n", sched->nr_sleep_events);
1589+ printf("nr_wakeup_events: %ld\n", sched->nr_wakeup_events);
17771590
1778− if (targetless_wakeups)
1779− printf("target-less wakeups: %ld\n", targetless_wakeups);
1780− if (multitarget_wakeups)
1781− printf("multi-target wakeups: %ld\n", multitarget_wakeups);
1782− if (nr_run_events_optimized)
1591+ if (sched->targetless_wakeups)
1592+ printf("target-less wakeups: %ld\n", sched->targetless_wakeups);
1593+ if (sched->multitarget_wakeups)
1594+ printf("multi-target wakeups: %ld\n", sched->multitarget_wakeups);
1595+ if (sched->nr_run_events_optimized)
17831596 printf("run atoms optimized: %ld\n",
1784− nr_run_events_optimized);
1597+ sched->nr_run_events_optimized);
17851598
1786− print_task_traces();
1787− add_cross_task_wakeups();
1599+ print_task_traces(sched);
1600+ add_cross_task_wakeups(sched);
17881601
1789− create_tasks();
1602+ create_tasks(sched);
17901603 printf("------------------------------------------------------------\n");
1791− for (i = 0; i < replay_repeat; i++)
1792− run_one_test();
1793−}
1794−
1795−
1796−static const char * const sched_usage[] = {
1797− "perf sched [<options>] {record|latency|map|replay|script}",
1798− NULL
1799−};
1800−
1801−static const struct option sched_options[] = {
1802− OPT_STRING('i', "input", &input_name, "file",
1803− "input file name"),
1804− OPT_INCR('v', "verbose", &verbose,
1805− "be more verbose (show symbol address, etc)"),
1806− OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
1807− "dump raw trace in ASCII"),
1808− OPT_END()
1809−};
1810−
1811−static const char * const latency_usage[] = {
1812− "perf sched latency [<options>]",
1813− NULL
1814−};
1815−
1816−static const struct option latency_options[] = {
1817− OPT_STRING('s', "sort", &sort_order, "key[,key2...]",
1818− "sort by key(s): runtime, switch, avg, max"),
1819− OPT_INCR('v', "verbose", &verbose,
1820− "be more verbose (show symbol address, etc)"),
1821− OPT_INTEGER('C', "CPU", &profile_cpu,
1822− "CPU to profile on"),
1823− OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
1824− "dump raw trace in ASCII"),
1825− OPT_END()
1826−};
1827−
1828−static const char * const replay_usage[] = {
1829− "perf sched replay [<options>]",
1830− NULL
1831−};
1604+ for (i = 0; i < sched->replay_repeat; i++)
1605+ run_one_test(sched);
18321606
1833−static const struct option replay_options[] = {
1834− OPT_UINTEGER('r', "repeat", &replay_repeat,
1835− "repeat the workload replay N times (-1: infinite)"),
1836− OPT_INCR('v', "verbose", &verbose,
1837− "be more verbose (show symbol address, etc)"),
1838− OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
1839− "dump raw trace in ASCII"),
1840− OPT_END()
1841−};
1607+ return 0;
1608+}
18421609
1843−static void setup_sorting(void)
1610+static void setup_sorting(struct perf_sched *sched, const struct option *options,
1611+ const char * const usage_msg[])
18441612 {
1845− char *tmp, *tok, *str = strdup(sort_order);
1613+ char *tmp, *tok, *str = strdup(sched->sort_order);
18461614
18471615 for (tok = strtok_r(str, ", ", &tmp);
18481616 tok; tok = strtok_r(NULL, ", ", &tmp)) {
1849− if (sort_dimension__add(tok, &sort_list) < 0) {
1617+ if (sort_dimension__add(tok, &sched->sort_list) < 0) {
18501618 error("Unknown --sort key: `%s'", tok);
1851− usage_with_options(latency_usage, latency_options);
1619+ usage_with_options(usage_msg, options);
18521620 }
18531621 }
18541622
18551623 free(str);
18561624
1857− sort_dimension__add("pid", &cmp_pid);
1625+ sort_dimension__add("pid", &sched->cmp_pid);
18581626 }
18591627
1860−static const char *record_args[] = {
1861− "record",
1862− "-a",
1863− "-R",
1864− "-f",
1865− "-m", "1024",
1866− "-c", "1",
1867− "-e", "sched:sched_switch",
1868− "-e", "sched:sched_stat_wait",
1869− "-e", "sched:sched_stat_sleep",
1870− "-e", "sched:sched_stat_iowait",
1871− "-e", "sched:sched_stat_runtime",
1872− "-e", "sched:sched_process_exit",
1873− "-e", "sched:sched_process_fork",
1874− "-e", "sched:sched_wakeup",
1875− "-e", "sched:sched_migrate_task",
1876−};
1877−
18781628 static int __cmd_record(int argc, const char **argv)
18791629 {
18801630 unsigned int rec_argc, i, j;
18811631 const char **rec_argv;
1632+ const char * const record_args[] = {
1633+ "record",
1634+ "-a",
1635+ "-R",
1636+ "-f",
1637+ "-m", "1024",
1638+ "-c", "1",
1639+ "-e", "sched:sched_switch",
1640+ "-e", "sched:sched_stat_wait",
1641+ "-e", "sched:sched_stat_sleep",
1642+ "-e", "sched:sched_stat_iowait",
1643+ "-e", "sched:sched_stat_runtime",
1644+ "-e", "sched:sched_process_exit",
1645+ "-e", "sched:sched_process_fork",
1646+ "-e", "sched:sched_wakeup",
1647+ "-e", "sched:sched_migrate_task",
1648+ };
18821649
18831650 rec_argc = ARRAY_SIZE(record_args) + argc - 1;
18841651 rec_argv = calloc(rec_argc + 1, sizeof(char *));
@@ -1897,8 +1664,85 @@ static int __cmd_record(int argc, const char **argv) · 12 unmodified lines
18971664 return cmd_record(i, rec_argv, NULL);
18981665 }
18991666
1900−int cmd_sched(int argc, const char **argv, const char *prefix __used)
1667+int cmd_sched(int argc, const char **argv, const char *prefix __maybe_unused)
19011668 {
1669+ const char default_sort_order[] = "avg, max, switch, runtime";
1670+ struct perf_sched sched = {
1671+ .tool = {
1672+ .sample = perf_sched__process_tracepoint_sample,
1673+ .comm = perf_event__process_comm,
1674+ .lost = perf_event__process_lost,
1675+ .fork = perf_event__process_task,
1676+ .ordered_samples = true,
1677+ },
1678+ .cmp_pid = LIST_HEAD_INIT(sched.cmp_pid),
1679+ .sort_list = LIST_HEAD_INIT(sched.sort_list),
1680+ .start_work_mutex = PTHREAD_MUTEX_INITIALIZER,
1681+ .work_done_wait_mutex = PTHREAD_MUTEX_INITIALIZER,
1682+ .curr_pid = { [0 ... MAX_CPUS - 1] = -1 },
1683+ .sort_order = default_sort_order,
1684+ .replay_repeat = 10,
1685+ .profile_cpu = -1,
1686+ .next_shortname1 = 'A',
1687+ .next_shortname2 = '0',
1688+ };
1689+ const struct option latency_options[] = {
1690+ OPT_STRING('s', "sort", &sched.sort_order, "key[,key2...]",
1691+ "sort by key(s): runtime, switch, avg, max"),
1692+ OPT_INCR('v', "verbose", &verbose,
1693+ "be more verbose (show symbol address, etc)"),
1694+ OPT_INTEGER('C', "CPU", &sched.profile_cpu,
1695+ "CPU to profile on"),
1696+ OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
1697+ "dump raw trace in ASCII"),
1698+ OPT_END()
1699+ };
1700+ const struct option replay_options[] = {
1701+ OPT_UINTEGER('r', "repeat", &sched.replay_repeat,
1702+ "repeat the workload replay N times (-1: infinite)"),
1703+ OPT_INCR('v', "verbose", &verbose,
1704+ "be more verbose (show symbol address, etc)"),
1705+ OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
1706+ "dump raw trace in ASCII"),
1707+ OPT_END()
1708+ };
1709+ const struct option sched_options[] = {
1710+ OPT_STRING('i', "input", &sched.input_name, "file",
1711+ "input file name"),
1712+ OPT_INCR('v', "verbose", &verbose,
1713+ "be more verbose (show symbol address, etc)"),
1714+ OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
1715+ "dump raw trace in ASCII"),
1716+ OPT_END()
1717+ };
1718+ const char * const latency_usage[] = {
1719+ "perf sched latency [<options>]",
1720+ NULL
1721+ };
1722+ const char * const replay_usage[] = {
1723+ "perf sched replay [<options>]",
1724+ NULL
1725+ };
1726+ const char * const sched_usage[] = {
1727+ "perf sched [<options>] {record|latency|map|replay|script}",
1728+ NULL
1729+ };
1730+ struct trace_sched_handler lat_ops = {
1731+ .wakeup_event = latency_wakeup_event,
1732+ .switch_event = latency_switch_event,
1733+ .runtime_event = latency_runtime_event,
1734+ .fork_event = latency_fork_event,
1735+ .migrate_task_event = latency_migrate_task_event,
1736+ };
1737+ struct trace_sched_handler map_ops = {
1738+ .switch_event = map_switch_event,
1739+ };
1740+ struct trace_sched_handler replay_ops = {
1741+ .wakeup_event = replay_wakeup_event,
1742+ .switch_event = replay_switch_event,
1743+ .fork_event = replay_fork_event,
1744+ };
1745+
19021746 argc = parse_options(argc, argv, sched_options, sched_usage,
19031747 PARSE_OPT_STOP_AT_NON_OPTION);
19041748 if (!argc)
@@ -1914,26 +1758,26 @@ int cmd_sched(int argc, const char **argv, const char *prefix __used) · 9 unmodified lines
19141758 if (!strncmp(argv[0], "rec", 3)) {
19151759 return __cmd_record(argc, argv);
19161760 } else if (!strncmp(argv[0], "lat", 3)) {
1917− trace_handler = &lat_ops;
1761+ sched.tp_handler = &lat_ops;
19181762 if (argc > 1) {
19191763 argc = parse_options(argc, argv, latency_options, latency_usage, 0);
19201764 if (argc)
19211765 usage_with_options(latency_usage, latency_options);
19221766 }
1923− setup_sorting();
1924− __cmd_lat();
1767+ setup_sorting(&sched, latency_options, latency_usage);
1768+ return perf_sched__lat(&sched);
19251769 } else if (!strcmp(argv[0], "map")) {
1926− trace_handler = &map_ops;
1927− setup_sorting();
1928− __cmd_map();
1770+ sched.tp_handler = &map_ops;
1771+ setup_sorting(&sched, latency_options, latency_usage);
1772+ return perf_sched__map(&sched);
19291773 } else if (!strncmp(argv[0], "rep", 3)) {
1930− trace_handler = &replay_ops;
1774+ sched.tp_handler = &replay_ops;
19311775 if (argc) {
19321776 argc = parse_options(argc, argv, replay_options, replay_usage, 0);
19331777 if (argc)
19341778 usage_with_options(replay_usage, replay_options);
19351779 }
1936− __cmd_replay();
1780+ return perf_sched__replay(&sched);
19371781 } else {
19381782 usage_with_options(sched_usage, sched_options);
19391783 }
tools/perf/builtin-script.c+140−89
@@ -14,6 +14,7 @@ · 13 unmodified lines
1414 #include "util/util.h"
1515 #include "util/evlist.h"
1616 #include "util/evsel.h"
17+#include "util/sort.h"
1718 #include <linux/bitmap.h>
1819
1920 static char const *script_name;
@@ -28,11 +29,6 @@ static bool system_wide; · 8 unmodified lines
2829 static const char *cpu_list;
2930 static DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
3031
31−struct perf_script {
32− struct perf_tool tool;
33− struct perf_session *session;
34−};
35−
3632 enum perf_output_field {
3733 PERF_OUTPUT_COMM = 1U << 0,
3834 PERF_OUTPUT_TID = 1U << 1,
@@ -262,14 +258,11 @@ static int perf_session__check_output_opt(struct perf_session *session) · 223 unmodified lines
262258 return 0;
263259 }
264260
265−static void print_sample_start(struct pevent *pevent,
266− struct perf_sample *sample,
261+static void print_sample_start(struct perf_sample *sample,
267262 struct thread *thread,
268263 struct perf_evsel *evsel)
269264 {
270− int type;
271265 struct perf_event_attr *attr = &evsel->attr;
272− struct event_format *event;
273266 const char *evname = NULL;
274267 unsigned long secs;
275268 unsigned long usecs;
@@ -307,20 +300,7 @@ static void print_sample_start(struct pevent *pevent, · 31 unmodified lines
307300 }
308301
309302 if (PRINT_FIELD(EVNAME)) {
310− if (attr->type == PERF_TYPE_TRACEPOINT) {
311− /*
312− * XXX Do we really need this here?
313− * perf_evlist__set_tracepoint_names should have done
314− * this already
315− */
316− type = trace_parse_common_type(pevent,
317− sample->raw_data);
318− event = pevent_find_event(pevent, type);
319− if (event)
320− evname = event->name;
321− } else
322− evname = perf_evsel__name(evsel);
323−
303+ evname = perf_evsel__name(evsel);
324304 printf("%s: ", evname ? evname : "[unknown]");
325305 }
326306 }
@@ -401,7 +381,7 @@ static void print_sample_bts(union perf_event *event, · 74 unmodified lines
401381 printf(" ");
402382 else
403383 printf("\n");
404− perf_event__print_ip(event, sample, machine,
384+ perf_evsel__print_ip(evsel, event, sample, machine,
405385 PRINT_FIELD(SYM), PRINT_FIELD(DSO),
406386 PRINT_FIELD(SYMOFFSET));
407387 }
@@ -415,19 +395,17 @@ static void print_sample_bts(union perf_event *event, · 7 unmodified lines
415395 printf("\n");
416396 }
417397
418−static void process_event(union perf_event *event __unused,
419− struct pevent *pevent,
420− struct perf_sample *sample,
421− struct perf_evsel *evsel,
422− struct machine *machine,
423− struct thread *thread)
398+static void process_event(union perf_event *event, struct perf_sample *sample,
399+ struct perf_evsel *evsel, struct machine *machine,
400+ struct addr_location *al)
424401 {
425402 struct perf_event_attr *attr = &evsel->attr;
403+ struct thread *thread = al->thread;
426404
427405 if (output[attr->type].fields == 0)
428406 return;
429407
430− print_sample_start(pevent, sample, thread, evsel);
408+ print_sample_start(sample, thread, evsel);
431409
432410 if (is_bts_event(attr)) {
433411 print_sample_bts(event, sample, evsel, machine, thread);
@@ -435,9 +413,8 @@ static void process_event(union perf_event *event __unused, · 1 unmodified line
435413 }
436414
437415 if (PRINT_FIELD(TRACE))
438− print_trace_event(pevent, sample->cpu, sample->raw_data,
439− sample->raw_size);
440−
416+ event_format__print(evsel->tp_format, sample->cpu,
417+ sample->raw_data, sample->raw_size);
441418 if (PRINT_FIELD(ADDR))
442419 print_sample_addr(event, sample, machine, thread, attr);
443420
@@ -446,7 +423,7 @@ static void process_event(union perf_event *event __unused, · 2 unmodified lines
446423 printf(" ");
447424 else
448425 printf("\n");
449− perf_event__print_ip(event, sample, machine,
426+ perf_evsel__print_ip(evsel, event, sample, machine,
450427 PRINT_FIELD(SYM), PRINT_FIELD(DSO),
451428 PRINT_FIELD(SYMOFFSET));
452429 }
@@ -454,9 +431,9 @@ static void process_event(union perf_event *event __unused, · 1 unmodified line
454431 printf("\n");
455432 }
456433
457−static int default_start_script(const char *script __unused,
458− int argc __unused,
459− const char **argv __unused)
434+static int default_start_script(const char *script __maybe_unused,
435+ int argc __maybe_unused,
436+ const char **argv __maybe_unused)
460437 {
461438 return 0;
462439 }
@@ -466,8 +443,8 @@ static int default_stop_script(void) · 3 unmodified lines
466443 return 0;
467444 }
468445
469−static int default_generate_script(struct pevent *pevent __unused,
470− const char *outfile __unused)
446+static int default_generate_script(struct pevent *pevent __maybe_unused,
447+ const char *outfile __maybe_unused)
471448 {
472449 return 0;
473450 }
@@ -498,14 +475,13 @@ static int cleanup_scripting(void) · 24 unmodified lines
498475
499476 static const char *input_name;
500477
501−static int process_sample_event(struct perf_tool *tool __used,
478+static int process_sample_event(struct perf_tool *tool __maybe_unused,
502479 union perf_event *event,
503480 struct perf_sample *sample,
504481 struct perf_evsel *evsel,
505482 struct machine *machine)
506483 {
507484 struct addr_location al;
508− struct perf_script *scr = container_of(tool, struct perf_script, tool);
509485 struct thread *thread = machine__findnew_thread(machine, event->ip.tid);
510486
511487 if (thread == NULL) {
@@ -537,32 +513,29 @@ static int process_sample_event(struct perf_tool *tool __used, · 25 unmodified lines
537513 if (cpu_list && !test_bit(sample->cpu, cpu_bitmap))
538514 return 0;
539515
540− scripting_ops->process_event(event, scr->session->pevent,
541− sample, evsel, machine, thread);
516+ scripting_ops->process_event(event, sample, evsel, machine, &al);
542517
543518 evsel->hists.stats.total_period += sample->period;
544519 return 0;
545520 }
546521
547−static struct perf_script perf_script = {
548− .tool = {
549− .sample = process_sample_event,
550− .mmap = perf_event__process_mmap,
551− .comm = perf_event__process_comm,
552− .exit = perf_event__process_task,
553− .fork = perf_event__process_task,
554− .attr = perf_event__process_attr,
555− .event_type = perf_event__process_event_type,
556− .tracing_data = perf_event__process_tracing_data,
557− .build_id = perf_event__process_build_id,
558− .ordered_samples = true,
559− .ordering_requires_timestamps = true,
560− },
522+static struct perf_tool perf_script = {
523+ .sample = process_sample_event,
524+ .mmap = perf_event__process_mmap,
525+ .comm = perf_event__process_comm,
526+ .exit = perf_event__process_task,
527+ .fork = perf_event__process_task,
528+ .attr = perf_event__process_attr,
529+ .event_type = perf_event__process_event_type,
530+ .tracing_data = perf_event__process_tracing_data,
531+ .build_id = perf_event__process_build_id,
532+ .ordered_samples = true,
533+ .ordering_requires_timestamps = true,
561534 };
562535
563536 extern volatile int session_done;
564537
565−static void sig_handler(int sig __unused)
538+static void sig_handler(int sig __maybe_unused)
566539 {
567540 session_done = 1;
568541 }
@@ -573,7 +546,7 @@ static int __cmd_script(struct perf_session *session) · 4 unmodified lines
573546
574547 signal(SIGINT, sig_handler);
575548
576− ret = perf_session__process_events(session, &perf_script.tool);
549+ ret = perf_session__process_events(session, &perf_script);
577550
578551 if (debug_mode)
579552 pr_err("Misordered timestamps: %" PRIu64 "\n", nr_unordered);
@@ -672,8 +645,8 @@ static void list_available_languages(void) · 92 unmodified lines
672645 fprintf(stderr, "\n");
673646 }
674647
675−static int parse_scriptname(const struct option *opt __used,
676− const char *str, int unset __used)
648+static int parse_scriptname(const struct option *opt __maybe_unused,
649+ const char *str, int unset __maybe_unused)
677650 {
678651 char spec[PATH_MAX];
679652 const char *script, *ext;
@@ -718,8 +691,8 @@ static int parse_scriptname(const struct option *opt __used, · 38 unmodified lines
718691 return 0;
719692 }
720693
721−static int parse_output_fields(const struct option *opt __used,
722− const char *arg, int unset __used)
694+static int parse_output_fields(const struct option *opt __maybe_unused,
695+ const char *arg, int unset __maybe_unused)
723696 {
724697 char *tok;
725698 int i, imax = sizeof(all_output_options) / sizeof(struct output_option);
@@ -1010,8 +983,9 @@ static char *get_script_root(struct dirent *script_dirent, const char *suffix) · 284 unmodified lines
1010983 return script_root;
1011984 }
1012985
1013−static int list_available_scripts(const struct option *opt __used,
1014− const char *s __used, int unset __used)
986+static int list_available_scripts(const struct option *opt __maybe_unused,
987+ const char *s __maybe_unused,
988+ int unset __maybe_unused)
1015989 {
1016990 struct dirent *script_next, *lang_next, script_dirent, lang_dirent;
1017991 char scripts_path[MAXPATHLEN];
@@ -1058,6 +1032,61 @@ static int list_available_scripts(const struct option *opt __used, · 40 unmodified lines
10581032 exit(0);
10591033 }
10601034
1035+/*
1036+ * Return -1 if none is found, otherwise the actual scripts number.
1037+ *
1038+ * Currently the only user of this function is the script browser, which
1039+ * will list all statically runnable scripts, select one, execute it and
1040+ * show the output in a perf browser.
1041+ */
1042+int find_scripts(char **scripts_array, char **scripts_path_array)
1043+{
1044+ struct dirent *script_next, *lang_next, script_dirent, lang_dirent;
1045+ char scripts_path[MAXPATHLEN];
1046+ DIR *scripts_dir, *lang_dir;
1047+ char lang_path[MAXPATHLEN];
1048+ char *temp;
1049+ int i = 0;
1050+
1051+ snprintf(scripts_path, MAXPATHLEN, "%s/scripts", perf_exec_path());
1052+
1053+ scripts_dir = opendir(scripts_path);
1054+ if (!scripts_dir)
1055+ return -1;
1056+
1057+ for_each_lang(scripts_path, scripts_dir, lang_dirent, lang_next) {
1058+ snprintf(lang_path, MAXPATHLEN, "%s/%s", scripts_path,
1059+ lang_dirent.d_name);
1060+#ifdef NO_LIBPERL
1061+ if (strstr(lang_path, "perl"))
1062+ continue;
1063+#endif
1064+#ifdef NO_LIBPYTHON
1065+ if (strstr(lang_path, "python"))
1066+ continue;
1067+#endif
1068+
1069+ lang_dir = opendir(lang_path);
1070+ if (!lang_dir)
1071+ continue;
1072+
1073+ for_each_script(lang_path, lang_dir, script_dirent, script_next) {
1074+ /* Skip those real time scripts: xxxtop.p[yl] */
1075+ if (strstr(script_dirent.d_name, "top."))
1076+ continue;
1077+ sprintf(scripts_path_array[i], "%s/%s", lang_path,
1078+ script_dirent.d_name);
1079+ temp = strchr(script_dirent.d_name, '.');
1080+ snprintf(scripts_array[i],
1081+ (temp - script_dirent.d_name) + 1,
1082+ "%s", script_dirent.d_name);
1083+ i++;
1084+ }
1085+ }
1086+
1087+ return i;
1088+}
1089+
10611090 static char *get_script_path(const char *script_root, const char *suffix)
10621091 {
10631092 struct dirent *script_next, *lang_next, script_dirent, lang_dirent;
@@ -1170,6 +1199,8 @@ static const struct option options[] = { · 106 unmodified lines
11701199 parse_output_fields),
11711200 OPT_BOOLEAN('a', "all-cpus", &system_wide,
11721201 "system-wide collection from all CPUs"),
1202+ OPT_STRING('S', "symbols", &symbol_conf.sym_list_str, "symbol[,symbol...]",
1203+ "only consider these symbols"),
11731204 OPT_STRING('C', "cpu", &cpu_list, "cpu", "list of cpus to profile"),
11741205 OPT_STRING('c', "comms", &symbol_conf.comm_list_str, "comm[,comm...]",
11751206 "only display events for these comms"),
@@ -1181,21 +1212,26 @@ static const struct option options[] = { · 5 unmodified lines
11811212 OPT_END()
11821213 };
11831214
1184−static bool have_cmd(int argc, const char **argv)
1215+static int have_cmd(int argc, const char **argv)
11851216 {
11861217 char **__argv = malloc(sizeof(const char *) * argc);
11871218
1188− if (!__argv)
1189− die("malloc");
1219+ if (!__argv) {
1220+ pr_err("malloc failed\n");
1221+ return -1;
1222+ }
1223+
11901224 memcpy(__argv, argv, sizeof(const char *) * argc);
11911225 argc = parse_options(argc, (const char **)__argv, record_options,
11921226 NULL, PARSE_OPT_STOP_AT_NON_OPTION);
11931227 free(__argv);
11941228
1195− return argc != 0;
1229+ system_wide = (argc == 0);
1230+
1231+ return 0;
11961232 }
11971233
1198−int cmd_script(int argc, const char **argv, const char *prefix __used)
1234+int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
11991235 {
12001236 char *rec_script_path = NULL;
12011237 char *rep_script_path = NULL;
@@ -1259,13 +1295,13 @@ int cmd_script(int argc, const char **argv, const char *prefix __used) · 57 unmodified lines
12591295
12601296 if (pipe(live_pipe) < 0) {
12611297 perror("failed to create pipe");
1262− exit(-1);
1298+ return -1;
12631299 }
12641300
12651301 pid = fork();
12661302 if (pid < 0) {
12671303 perror("failed to fork");
1268− exit(-1);
1304+ return -1;
12691305 }
12701306
12711307 if (!pid) {
@@ -1277,13 +1313,18 @@ int cmd_script(int argc, const char **argv, const char *prefix __used) · 5 unmodified lines
12771313 if (is_top_script(argv[0])) {
12781314 system_wide = true;
12791315 } else if (!system_wide) {
1280− system_wide = !have_cmd(argc - rep_args,
1281− &argv[rep_args]);
1316+ if (have_cmd(argc - rep_args, &argv[rep_args]) != 0) {
1317+ err = -1;
1318+ goto out;
1319+ }
12821320 }
12831321
12841322 __argv = malloc((argc + 6) * sizeof(const char *));
1285− if (!__argv)
1286− die("malloc");
1323+ if (!__argv) {
1324+ pr_err("malloc failed\n");
1325+ err = -ENOMEM;
1326+ goto out;
1327+ }
12871328
12881329 __argv[j++] = "/bin/sh";
12891330 __argv[j++] = rec_script_path;
@@ -1305,8 +1346,12 @@ int cmd_script(int argc, const char **argv, const char *prefix __used) · 15 unmodified lines
13051346 close(live_pipe[1]);
13061347
13071348 __argv = malloc((argc + 4) * sizeof(const char *));
1308− if (!__argv)
1309− die("malloc");
1349+ if (!__argv) {
1350+ pr_err("malloc failed\n");
1351+ err = -ENOMEM;
1352+ goto out;
1353+ }
1354+
13101355 j = 0;
13111356 __argv[j++] = "/bin/sh";
13121357 __argv[j++] = rep_script_path;
@@ -1331,12 +1376,20 @@ int cmd_script(int argc, const char **argv, const char *prefix __used) · 18 unmodified lines
13311376
13321377 if (!rec_script_path)
13331378 system_wide = false;
1334− else if (!system_wide)
1335− system_wide = !have_cmd(argc - 1, &argv[1]);
1379+ else if (!system_wide) {
1380+ if (have_cmd(argc - 1, &argv[1]) != 0) {
1381+ err = -1;
1382+ goto out;
1383+ }
1384+ }
13361385
13371386 __argv = malloc((argc + 2) * sizeof(const char *));
1338− if (!__argv)
1339− die("malloc");
1387+ if (!__argv) {
1388+ pr_err("malloc failed\n");
1389+ err = -ENOMEM;
1390+ goto out;
1391+ }
1392+
13401393 __argv[j++] = "/bin/sh";
13411394 __argv[j++] = script_path;
13421395 if (system_wide)
@@ -1356,12 +1409,10 @@ int cmd_script(int argc, const char **argv, const char *prefix __used) · 13 unmodified lines
13561409 setup_pager();
13571410
13581411 session = perf_session__new(input_name, O_RDONLY, 0, false,
1359− &perf_script.tool);
1412+ &perf_script);
13601413 if (session == NULL)
13611414 return -ENOMEM;
13621415
1363− perf_script.session = session;
1364−
13651416 if (cpu_list) {
13661417 if (perf_session__cpu_bitmap(session, cpu_list, cpu_bitmap))
13671418 return -1;
@@ -1387,18 +1438,18 @@ int cmd_script(int argc, const char **argv, const char *prefix __used) · 19 unmodified lines
13871438 input = open(session->filename, O_RDONLY); /* input_name */
13881439 if (input < 0) {
13891440 perror("failed to open file");
1390− exit(-1);
1441+ return -1;
13911442 }
13921443
13931444 err = fstat(input, &perf_stat);
13941445 if (err < 0) {
13951446 perror("failed to stat file");
1396− exit(-1);
1447+ return -1;
13971448 }
13981449
13991450 if (!perf_stat.st_size) {
14001451 fprintf(stderr, "zero-sized file, nothing to do!\n");
1401− exit(0);
1452+ return 0;
14021453 }
14031454
14041455 scripting_ops = script_spec__lookup(generate_script_lang);
tools/perf/builtin-stat.c+54−82
@@ -51,13 +51,13 @@ · 50 unmodified lines
5151 #include "util/evsel.h"
5252 #include "util/debug.h"
5353 #include "util/color.h"
54+#include "util/stat.h"
5455 #include "util/header.h"
5556 #include "util/cpumap.h"
5657 #include "util/thread.h"
5758 #include "util/thread_map.h"
5859
5960 #include <sys/prctl.h>
60−#include <math.h>
6161 #include <locale.h>
6262
6363 #define DEFAULT_SEPARATOR " "
@@ -199,11 +199,6 @@ static int output_fd; · 135 unmodified lines
199199
200200 static volatile int done = 0;
201201
202−struct stats
203−{
204− double n, mean, M2;
205−};
206−
207202 struct perf_stat {
208203 struct stats res_stats[3];
209204 };
@@ -220,48 +215,14 @@ static void perf_evsel__free_stat_priv(struct perf_evsel *evsel) · 10 unmodified lines
220215 evsel->priv = NULL;
221216 }
222217
223−static void update_stats(struct stats *stats, u64 val)
224−{
225− double delta;
226−
227− stats->n++;
228− delta = val - stats->mean;
229− stats->mean += delta / stats->n;
230− stats->M2 += delta*(val - stats->mean);
231−}
232−
233−static double avg_stats(struct stats *stats)
218+static inline struct cpu_map *perf_evsel__cpus(struct perf_evsel *evsel)
234219 {
235− return stats->mean;
220+ return (evsel->cpus && !target.cpu_list) ? evsel->cpus : evsel_list->cpus;
236221 }
237222
238−/*
239− * http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
240− *
241− * (\Sum n_i^2) - ((\Sum n_i)^2)/n
242− * s^2 = -------------------------------
243− * n - 1
244− *
245− * http://en.wikipedia.org/wiki/Stddev
246− *
247− * The std dev of the mean is related to the std dev by:
248− *
249− * s
250− * s_mean = -------
251− * sqrt(n)
252− *
253− */
254−static double stddev_stats(struct stats *stats)
223+static inline int perf_evsel__nr_cpus(struct perf_evsel *evsel)
255224 {
256− double variance, variance_mean;
257−
258− if (!stats->n)
259− return 0.0;
260−
261− variance = stats->M2 / (stats->n - 1);
262− variance_mean = variance / stats->n;
263−
264− return sqrt(variance_mean);
225+ return perf_evsel__cpus(evsel)->nr;
265226 }
266227
267228 static struct stats runtime_nsecs_stats[MAX_NR_CPUS];
@@ -281,13 +242,9 @@ static int create_perf_stat_counter(struct perf_evsel *evsel, · 13 unmodified lines
281242 struct perf_evsel *first)
282243 {
283244 struct perf_event_attr *attr = &evsel->attr;
284− struct xyarray *group_fd = NULL;
285245 bool exclude_guest_missing = false;
286246 int ret;
287247
288− if (group && evsel != first)
289− group_fd = first->fd;
290−
291248 if (scale)
292249 attr->read_format = PERF_FORMAT_TOTAL_TIME_ENABLED |
293250 PERF_FORMAT_TOTAL_TIME_RUNNING;
@@ -299,8 +256,7 @@ retry: · 5 unmodified lines
299256 evsel->attr.exclude_guest = evsel->attr.exclude_host = 0;
300257
301258 if (perf_target__has_cpu(&target)) {
302− ret = perf_evsel__open_per_cpu(evsel, evsel_list->cpus,
303− group, group_fd);
259+ ret = perf_evsel__open_per_cpu(evsel, perf_evsel__cpus(evsel));
304260 if (ret)
305261 goto check_ret;
306262 return 0;
@@ -311,8 +267,7 @@ retry: · 4 unmodified lines
311267 attr->enable_on_exec = 1;
312268 }
313269
314− ret = perf_evsel__open_per_thread(evsel, evsel_list->threads,
315− group, group_fd);
270+ ret = perf_evsel__open_per_thread(evsel, evsel_list->threads);
316271 if (!ret)
317272 return 0;
318273 /* fall through */
@@ -382,7 +337,7 @@ static int read_counter_aggr(struct perf_evsel *counter) · 63 unmodified lines
382337 u64 *count = counter->counts->aggr.values;
383338 int i;
384339
385− if (__perf_evsel__read(counter, evsel_list->cpus->nr,
340+ if (__perf_evsel__read(counter, perf_evsel__nr_cpus(counter),
386341 evsel_list->threads->nr, scale) < 0)
387342 return -1;
388343
@@ -411,7 +366,7 @@ static int read_counter(struct perf_evsel *counter) · 22 unmodified lines
411366 u64 *count;
412367 int cpu;
413368
414− for (cpu = 0; cpu < evsel_list->cpus->nr; cpu++) {
369+ for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
415370 if (__perf_evsel__read_on_cpu(counter, cpu, 0, scale) < 0)
416371 return -1;
417372
@@ -423,7 +378,7 @@ static int read_counter(struct perf_evsel *counter) · 5 unmodified lines
423378 return 0;
424379 }
425380
426−static int run_perf_stat(int argc __used, const char **argv)
381+static int run_perf_stat(int argc __maybe_unused, const char **argv)
427382 {
428383 unsigned long long t0, t1;
429384 struct perf_evsel *counter, *first;
@@ -434,7 +389,7 @@ static int run_perf_stat(int argc __used, const char **argv) · 4 unmodified lines
434389
435390 if (forks && (pipe(child_ready_pipe) < 0 || pipe(go_pipe) < 0)) {
436391 perror("failed to create pipes");
437− exit(1);
392+ return -1;
438393 }
439394
440395 if (forks) {
@@ -483,7 +438,10 @@ static int run_perf_stat(int argc __used, const char **argv) · 42 unmodified lines
483438 close(child_ready_pipe[0]);
484439 }
485440
486− first = list_entry(evsel_list->entries.next, struct perf_evsel, node);
441+ if (group)
442+ perf_evlist__set_leader(evsel_list);
443+
444+ first = perf_evlist__first(evsel_list);
487445
488446 list_for_each_entry(counter, &evsel_list->entries, node) {
489447 if (create_perf_stat_counter(counter, first) < 0) {
@@ -513,13 +471,14 @@ static int run_perf_stat(int argc __used, const char **argv) · 23 unmodified lines
513471 }
514472 if (child_pid != -1)
515473 kill(child_pid, SIGTERM);
516− die("Not all events could be opened.\n");
474+
475+ pr_err("Not all events could be opened.\n");
517476 return -1;
518477 }
519478 counter->supported = true;
520479 }
521480
522− if (perf_evlist__set_filters(evsel_list)) {
481+ if (perf_evlist__apply_filters(evsel_list)) {
523482 error("failed to set filter with %d (%s)\n", errno,
524483 strerror(errno));
525484 return -1;
@@ -546,12 +505,12 @@ static int run_perf_stat(int argc __used, const char **argv) · 20 unmodified lines
546505 if (no_aggr) {
547506 list_for_each_entry(counter, &evsel_list->entries, node) {
548507 read_counter(counter);
549− perf_evsel__close_fd(counter, evsel_list->cpus->nr, 1);
508+ perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter), 1);
550509 }
551510 } else {
552511 list_for_each_entry(counter, &evsel_list->entries, node) {
553512 read_counter_aggr(counter);
554− perf_evsel__close_fd(counter, evsel_list->cpus->nr,
513+ perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter),
555514 evsel_list->threads->nr);
556515 }
557516 }
@@ -561,10 +520,7 @@ static int run_perf_stat(int argc __used, const char **argv) · 3 unmodified lines
561520
562521 static void print_noise_pct(double total, double avg)
563522 {
564− double pct = 0.0;
565−
566− if (avg)
567− pct = 100.0*total/avg;
523+ double pct = rel_stddev_stats(total, avg);
568524
569525 if (csv_output)
570526 fprintf(output, "%s%.2f%%", csv_sep, pct);
@@ -592,7 +548,7 @@ static void nsec_printout(int cpu, struct perf_evsel *evsel, double avg) · 21 unmodified lines
592548 if (no_aggr)
593549 sprintf(cpustr, "CPU%*d%s",
594550 csv_output ? 0 : -4,
595− evsel_list->cpus->map[cpu], csv_sep);
551+ perf_evsel__cpus(evsel)->map[cpu], csv_sep);
596552
597553 fprintf(output, fmt, cpustr, msecs, csv_sep, perf_evsel__name(evsel));
598554
@@ -636,7 +592,9 @@ static const char *get_ratio_color(enum grc_type type, double ratio) · 37 unmodified lines
636592 return color;
637593 }
638594
639−static void print_stalled_cycles_frontend(int cpu, struct perf_evsel *evsel __used, double avg)
595+static void print_stalled_cycles_frontend(int cpu,
596+ struct perf_evsel *evsel
597+ __maybe_unused, double avg)
640598 {
641599 double total, ratio = 0.0;
642600 const char *color;
@@ -653,7 +611,9 @@ static void print_stalled_cycles_frontend(int cpu, struct perf_evsel *evsel __us · 10 unmodified lines
653611 fprintf(output, " frontend cycles idle ");
654612 }
655613
656−static void print_stalled_cycles_backend(int cpu, struct perf_evsel *evsel __used, double avg)
614+static void print_stalled_cycles_backend(int cpu,
615+ struct perf_evsel *evsel
616+ __maybe_unused, double avg)
657617 {
658618 double total, ratio = 0.0;
659619 const char *color;
@@ -670,7 +630,9 @@ static void print_stalled_cycles_backend(int cpu, struct perf_evsel *evsel __use · 10 unmodified lines
670630 fprintf(output, " backend cycles idle ");
671631 }
672632
673−static void print_branch_misses(int cpu, struct perf_evsel *evsel __used, double avg)
633+static void print_branch_misses(int cpu,
634+ struct perf_evsel *evsel __maybe_unused,
635+ double avg)
674636 {
675637 double total, ratio = 0.0;
676638 const char *color;
@@ -687,7 +649,9 @@ static void print_branch_misses(int cpu, struct perf_evsel *evsel __used, double · 10 unmodified lines
687649 fprintf(output, " of all branches ");
688650 }
689651
690−static void print_l1_dcache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
652+static void print_l1_dcache_misses(int cpu,
653+ struct perf_evsel *evsel __maybe_unused,
654+ double avg)
691655 {
692656 double total, ratio = 0.0;
693657 const char *color;
@@ -704,7 +668,9 @@ static void print_l1_dcache_misses(int cpu, struct perf_evsel *evsel __used, dou · 10 unmodified lines
704668 fprintf(output, " of all L1-dcache hits ");
705669 }
706670
707−static void print_l1_icache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
671+static void print_l1_icache_misses(int cpu,
672+ struct perf_evsel *evsel __maybe_unused,
673+ double avg)
708674 {
709675 double total, ratio = 0.0;
710676 const char *color;
@@ -721,7 +687,9 @@ static void print_l1_icache_misses(int cpu, struct perf_evsel *evsel __used, dou · 10 unmodified lines
721687 fprintf(output, " of all L1-icache hits ");
722688 }
723689
724−static void print_dtlb_cache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
690+static void print_dtlb_cache_misses(int cpu,
691+ struct perf_evsel *evsel __maybe_unused,
692+ double avg)
725693 {
726694 double total, ratio = 0.0;
727695 const char *color;
@@ -738,7 +706,9 @@ static void print_dtlb_cache_misses(int cpu, struct perf_evsel *evsel __used, do · 10 unmodified lines
738706 fprintf(output, " of all dTLB cache hits ");
739707 }
740708
741−static void print_itlb_cache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
709+static void print_itlb_cache_misses(int cpu,
710+ struct perf_evsel *evsel __maybe_unused,
711+ double avg)
742712 {
743713 double total, ratio = 0.0;
744714 const char *color;
@@ -755,7 +725,9 @@ static void print_itlb_cache_misses(int cpu, struct perf_evsel *evsel __used, do · 10 unmodified lines
755725 fprintf(output, " of all iTLB cache hits ");
756726 }
757727
758−static void print_ll_cache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
728+static void print_ll_cache_misses(int cpu,
729+ struct perf_evsel *evsel __maybe_unused,
730+ double avg)
759731 {
760732 double total, ratio = 0.0;
761733 const char *color;
@@ -788,7 +760,7 @@ static void abs_printout(int cpu, struct perf_evsel *evsel, double avg) · 26 unmodified lines
788760 if (no_aggr)
789761 sprintf(cpustr, "CPU%*d%s",
790762 csv_output ? 0 : -4,
791− evsel_list->cpus->map[cpu], csv_sep);
763+ perf_evsel__cpus(evsel)->map[cpu], csv_sep);
792764 else
793765 cpu = 0;
794766
@@ -949,14 +921,14 @@ static void print_counter(struct perf_evsel *counter) · 154 unmodified lines
949921 u64 ena, run, val;
950922 int cpu;
951923
952− for (cpu = 0; cpu < evsel_list->cpus->nr; cpu++) {
924+ for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
953925 val = counter->counts->cpu[cpu].val;
954926 ena = counter->counts->cpu[cpu].ena;
955927 run = counter->counts->cpu[cpu].run;
956928 if (run == 0 || ena == 0) {
957929 fprintf(output, "CPU%*d%s%*s%s%*s",
958930 csv_output ? 0 : -4,
959− evsel_list->cpus->map[cpu], csv_sep,
931+ perf_evsel__cpus(counter)->map[cpu], csv_sep,
960932 csv_output ? 0 : 18,
961933 counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
962934 csv_sep,
@@ -1061,8 +1033,8 @@ static const char * const stat_usage[] = { · 98 unmodified lines
10611033 NULL
10621034 };
10631035
1064−static int stat__set_big_num(const struct option *opt __used,
1065− const char *s __used, int unset)
1036+static int stat__set_big_num(const struct option *opt __maybe_unused,
1037+ const char *s __maybe_unused, int unset)
10661038 {
10671039 big_num_opt = unset ? 0 : 1;
10681040 return 0;
@@ -1156,7 +1128,7 @@ static int add_default_attributes(void) · 87 unmodified lines
11561128 return perf_evlist__add_default_attrs(evsel_list, very_very_detailed_attrs);
11571129 }
11581130
1159−int cmd_stat(int argc, const char **argv, const char *prefix __used)
1131+int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
11601132 {
11611133 struct perf_evsel *pos;
11621134 int status = -ENOMEM;
@@ -1192,7 +1164,7 @@ int cmd_stat(int argc, const char **argv, const char *prefix __used) · 29 unmodified lines
11921164 output = fopen(output_name, mode);
11931165 if (!output) {
11941166 perror("failed to create output file");
1195− exit(-1);
1167+ return -1;
11961168 }
11971169 clock_gettime(CLOCK_REALTIME, &tm);
11981170 fprintf(output, "# started on %s\n", ctime(&tm.tv_sec));
@@ -1255,7 +1227,7 @@ int cmd_stat(int argc, const char **argv, const char *prefix __used) · 56 unmodified lines
12551227
12561228 list_for_each_entry(pos, &evsel_list->entries, node) {
12571229 if (perf_evsel__alloc_stat_priv(pos) < 0 ||
1258− perf_evsel__alloc_counts(pos, evsel_list->cpus->nr) < 0)
1230+ perf_evsel__alloc_counts(pos, perf_evsel__nr_cpus(pos)) < 0)
12591231 goto out_free_fd;
12601232 }
12611233
tools/perf/builtin-test.c+337−16
@@ -14,11 +14,13 @@ · 13 unmodified lines
1414 #include "util/symbol.h"
1515 #include "util/thread_map.h"
1616 #include "util/pmu.h"
17+#include "event-parse.h"
1718 #include "../../include/linux/hw_breakpoint.h"
1819
1920 #include <sys/mman.h>
2021
21−static int vmlinux_matches_kallsyms_filter(struct map *map __used, struct symbol *sym)
22+static int vmlinux_matches_kallsyms_filter(struct map *map __maybe_unused,
23+ struct symbol *sym)
2224 {
2325 bool *visited = symbol__priv(sym);
2426 *visited = true;
@@ -294,7 +296,7 @@ static int test__open_syscall_event(void) · 269 unmodified lines
294296 goto out_thread_map_delete;
295297 }
296298
297− if (perf_evsel__open_per_thread(evsel, threads, false, NULL) < 0) {
299+ if (perf_evsel__open_per_thread(evsel, threads) < 0) {
298300 pr_debug("failed to open counter: %s, "
299301 "tweak /proc/sys/kernel/perf_event_paranoid?\n",
300302 strerror(errno));
@@ -369,7 +371,7 @@ static int test__open_syscall_event_on_all_cpus(void) · 68 unmodified lines
369371 goto out_thread_map_delete;
370372 }
371373
372− if (perf_evsel__open(evsel, cpus, threads, false, NULL) < 0) {
374+ if (perf_evsel__open(evsel, cpus, threads) < 0) {
373375 pr_debug("failed to open counter: %s, "
374376 "tweak /proc/sys/kernel/perf_event_paranoid?\n",
375377 strerror(errno));
@@ -533,7 +535,7 @@ static int test__basic_mmap(void) · 157 unmodified lines
533535
534536 perf_evlist__add(evlist, evsels[i]);
535537
536− if (perf_evsel__open(evsels[i], cpus, threads, false, NULL) < 0) {
538+ if (perf_evsel__open(evsels[i], cpus, threads) < 0) {
537539 pr_debug("failed to open counter: %s, "
538540 "tweak /proc/sys/kernel/perf_event_paranoid?\n",
539541 strerror(errno));
@@ -562,7 +564,7 @@ static int test__basic_mmap(void) · 22 unmodified lines
562564 goto out_munmap;
563565 }
564566
565− err = perf_evlist__parse_sample(evlist, event, &sample, false);
567+ err = perf_evlist__parse_sample(evlist, event, &sample);
566568 if (err) {
567569 pr_err("Can't parse sample, err = %d\n", err);
568570 goto out_munmap;
@@ -710,7 +712,7 @@ static int test__PERF_RECORD(void) · 141 unmodified lines
710712 /*
711713 * Config the evsels, setting attr->comm on the first one, etc.
712714 */
713− evsel = list_entry(evlist->entries.next, struct perf_evsel, node);
715+ evsel = perf_evlist__first(evlist);
714716 evsel->attr.sample_type |= PERF_SAMPLE_CPU;
715717 evsel->attr.sample_type |= PERF_SAMPLE_TID;
716718 evsel->attr.sample_type |= PERF_SAMPLE_TIME;
@@ -737,7 +739,7 @@ static int test__PERF_RECORD(void) · 20 unmodified lines
737739 * Call sys_perf_event_open on all the fds on all the evsels,
738740 * grouping them if asked to.
739741 */
740− err = perf_evlist__open(evlist, opts.group);
742+ err = perf_evlist__open(evlist);
741743 if (err < 0) {
742744 pr_debug("perf_evlist__open: %s\n", strerror(errno));
743745 goto out_delete_evlist;
@@ -779,7 +781,7 @@ static int test__PERF_RECORD(void) · 35 unmodified lines
779781 if (type < PERF_RECORD_MAX)
780782 nr_events[type]++;
781783
782− err = perf_evlist__parse_sample(evlist, event, &sample, false);
784+ err = perf_evlist__parse_sample(evlist, event, &sample);
783785 if (err < 0) {
784786 if (verbose)
785787 perf_event__fprintf(event, stderr);
@@ -996,7 +998,9 @@ static u64 mmap_read_self(void *addr) · 210 unmodified lines
996998 /*
997999 * If the RDPMC instruction faults then signal this back to the test parent task:
9981000 */
999−static void segfault_handler(int sig __used, siginfo_t *info __used, void *uc __used)
1001+static void segfault_handler(int sig __maybe_unused,
1002+ siginfo_t *info __maybe_unused,
1003+ void *uc __maybe_unused)
10001004 {
10011005 exit(-1);
10021006 }
@@ -1023,14 +1027,16 @@ static int __test__rdpmc(void) · 20 unmodified lines
10231027
10241028 fd = sys_perf_event_open(&attr, 0, -1, -1, 0);
10251029 if (fd < 0) {
1026− die("Error: sys_perf_event_open() syscall returned "
1027− "with %d (%s)\n", fd, strerror(errno));
1030+ pr_err("Error: sys_perf_event_open() syscall returned "
1031+ "with %d (%s)\n", fd, strerror(errno));
1032+ return -1;
10281033 }
10291034
10301035 addr = mmap(NULL, page_size, PROT_READ, MAP_SHARED, fd, 0);
10311036 if (addr == (void *)(-1)) {
1032− die("Error: mmap() syscall returned "
1033− "with (%s)\n", strerror(errno));
1037+ pr_err("Error: mmap() syscall returned with (%s)\n",
1038+ strerror(errno));
1039+ goto out_close;
10341040 }
10351041
10361042 for (n = 0; n < 6; n++) {
@@ -1051,9 +1057,9 @@ static int __test__rdpmc(void) · 14 unmodified lines
10511057 }
10521058
10531059 munmap(addr, page_size);
1054− close(fd);
1055−
10561060 pr_debug(" ");
1061+out_close:
1062+ close(fd);
10571063
10581064 if (!delta_sum)
10591065 return -1;
@@ -1092,6 +1098,309 @@ static int test__perf_pmu(void) · 32 unmodified lines
10921098 return perf_pmu__test();
10931099 }
10941100
1101+static int perf_evsel__roundtrip_cache_name_test(void)
1102+{
1103+ char name[128];
1104+ int type, op, err = 0, ret = 0, i, idx;
1105+ struct perf_evsel *evsel;
1106+ struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
1107+
1108+ if (evlist == NULL)
1109+ return -ENOMEM;
1110+
1111+ for (type = 0; type < PERF_COUNT_HW_CACHE_MAX; type++) {
1112+ for (op = 0; op < PERF_COUNT_HW_CACHE_OP_MAX; op++) {
1113+ /* skip invalid cache type */
1114+ if (!perf_evsel__is_cache_op_valid(type, op))
1115+ continue;
1116+
1117+ for (i = 0; i < PERF_COUNT_HW_CACHE_RESULT_MAX; i++) {
1118+ __perf_evsel__hw_cache_type_op_res_name(type, op, i,
1119+ name, sizeof(name));
1120+ err = parse_events(evlist, name, 0);
1121+ if (err)
1122+ ret = err;
1123+ }
1124+ }
1125+ }
1126+
1127+ idx = 0;
1128+ evsel = perf_evlist__first(evlist);
1129+
1130+ for (type = 0; type < PERF_COUNT_HW_CACHE_MAX; type++) {
1131+ for (op = 0; op < PERF_COUNT_HW_CACHE_OP_MAX; op++) {
1132+ /* skip invalid cache type */
1133+ if (!perf_evsel__is_cache_op_valid(type, op))
1134+ continue;
1135+
1136+ for (i = 0; i < PERF_COUNT_HW_CACHE_RESULT_MAX; i++) {
1137+ __perf_evsel__hw_cache_type_op_res_name(type, op, i,
1138+ name, sizeof(name));
1139+ if (evsel->idx != idx)
1140+ continue;
1141+
1142+ ++idx;
1143+
1144+ if (strcmp(perf_evsel__name(evsel), name)) {
1145+ pr_debug("%s != %s\n", perf_evsel__name(evsel), name);
1146+ ret = -1;
1147+ }
1148+
1149+ evsel = perf_evsel__next(evsel);
1150+ }
1151+ }
1152+ }
1153+
1154+ perf_evlist__delete(evlist);
1155+ return ret;
1156+}
1157+
1158+static int __perf_evsel__name_array_test(const char *names[], int nr_names)
1159+{
1160+ int i, err;
1161+ struct perf_evsel *evsel;
1162+ struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
1163+
1164+ if (evlist == NULL)
1165+ return -ENOMEM;
1166+
1167+ for (i = 0; i < nr_names; ++i) {
1168+ err = parse_events(evlist, names[i], 0);
1169+ if (err) {
1170+ pr_debug("failed to parse event '%s', err %d\n",
1171+ names[i], err);
1172+ goto out_delete_evlist;
1173+ }
1174+ }
1175+
1176+ err = 0;
1177+ list_for_each_entry(evsel, &evlist->entries, node) {
1178+ if (strcmp(perf_evsel__name(evsel), names[evsel->idx])) {
1179+ --err;
1180+ pr_debug("%s != %s\n", perf_evsel__name(evsel), names[evsel->idx]);
1181+ }
1182+ }
1183+
1184+out_delete_evlist:
1185+ perf_evlist__delete(evlist);
1186+ return err;
1187+}
1188+
1189+#define perf_evsel__name_array_test(names) \
1190+ __perf_evsel__name_array_test(names, ARRAY_SIZE(names))
1191+
1192+static int perf_evsel__roundtrip_name_test(void)
1193+{
1194+ int err = 0, ret = 0;
1195+
1196+ err = perf_evsel__name_array_test(perf_evsel__hw_names);
1197+ if (err)
1198+ ret = err;
1199+
1200+ err = perf_evsel__name_array_test(perf_evsel__sw_names);
1201+ if (err)
1202+ ret = err;
1203+
1204+ err = perf_evsel__roundtrip_cache_name_test();
1205+ if (err)
1206+ ret = err;
1207+
1208+ return ret;
1209+}
1210+
1211+static int perf_evsel__test_field(struct perf_evsel *evsel, const char *name,
1212+ int size, bool should_be_signed)
1213+{
1214+ struct format_field *field = perf_evsel__field(evsel, name);
1215+ int is_signed;
1216+ int ret = 0;
1217+
1218+ if (field == NULL) {
1219+ pr_debug("%s: \"%s\" field not found!\n", evsel->name, name);
1220+ return -1;
1221+ }
1222+
1223+ is_signed = !!(field->flags | FIELD_IS_SIGNED);
1224+ if (should_be_signed && !is_signed) {
1225+ pr_debug("%s: \"%s\" signedness(%d) is wrong, should be %d\n",
1226+ evsel->name, name, is_signed, should_be_signed);
1227+ ret = -1;
1228+ }
1229+
1230+ if (field->size != size) {
1231+ pr_debug("%s: \"%s\" size (%d) should be %d!\n",
1232+ evsel->name, name, field->size, size);
1233+ ret = -1;
1234+ }
1235+
1236+ return ret;
1237+}
1238+
1239+static int perf_evsel__tp_sched_test(void)
1240+{
1241+ struct perf_evsel *evsel = perf_evsel__newtp("sched", "sched_switch", 0);
1242+ int ret = 0;
1243+
1244+ if (evsel == NULL) {
1245+ pr_debug("perf_evsel__new\n");
1246+ return -1;
1247+ }
1248+
1249+ if (perf_evsel__test_field(evsel, "prev_comm", 16, true))
1250+ ret = -1;
1251+
1252+ if (perf_evsel__test_field(evsel, "prev_pid", 4, true))
1253+ ret = -1;
1254+
1255+ if (perf_evsel__test_field(evsel, "prev_prio", 4, true))
1256+ ret = -1;
1257+
1258+ if (perf_evsel__test_field(evsel, "prev_state", 8, true))
1259+ ret = -1;
1260+
1261+ if (perf_evsel__test_field(evsel, "next_comm", 16, true))
1262+ ret = -1;
1263+
1264+ if (perf_evsel__test_field(evsel, "next_pid", 4, true))
1265+ ret = -1;
1266+
1267+ if (perf_evsel__test_field(evsel, "next_prio", 4, true))
1268+ ret = -1;
1269+
1270+ perf_evsel__delete(evsel);
1271+
1272+ evsel = perf_evsel__newtp("sched", "sched_wakeup", 0);
1273+
1274+ if (perf_evsel__test_field(evsel, "comm", 16, true))
1275+ ret = -1;
1276+
1277+ if (perf_evsel__test_field(evsel, "pid", 4, true))
1278+ ret = -1;
1279+
1280+ if (perf_evsel__test_field(evsel, "prio", 4, true))
1281+ ret = -1;
1282+
1283+ if (perf_evsel__test_field(evsel, "success", 4, true))
1284+ ret = -1;
1285+
1286+ if (perf_evsel__test_field(evsel, "target_cpu", 4, true))
1287+ ret = -1;
1288+
1289+ return ret;
1290+}
1291+
1292+static int test__syscall_open_tp_fields(void)
1293+{
1294+ struct perf_record_opts opts = {
1295+ .target = {
1296+ .uid = UINT_MAX,
1297+ .uses_mmap = true,
1298+ },
1299+ .no_delay = true,
1300+ .freq = 1,
1301+ .mmap_pages = 256,
1302+ .raw_samples = true,
1303+ };
1304+ const char *filename = "/etc/passwd";
1305+ int flags = O_RDONLY | O_DIRECTORY;
1306+ struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
1307+ struct perf_evsel *evsel;
1308+ int err = -1, i, nr_events = 0, nr_polls = 0;
1309+
1310+ if (evlist == NULL) {
1311+ pr_debug("%s: perf_evlist__new\n", __func__);
1312+ goto out;
1313+ }
1314+
1315+ evsel = perf_evsel__newtp("syscalls", "sys_enter_open", 0);
1316+ if (evsel == NULL) {
1317+ pr_debug("%s: perf_evsel__newtp\n", __func__);
1318+ goto out_delete_evlist;
1319+ }
1320+
1321+ perf_evlist__add(evlist, evsel);
1322+
1323+ err = perf_evlist__create_maps(evlist, &opts.target);
1324+ if (err < 0) {
1325+ pr_debug("%s: perf_evlist__create_maps\n", __func__);
1326+ goto out_delete_evlist;
1327+ }
1328+
1329+ perf_evsel__config(evsel, &opts, evsel);
1330+
1331+ evlist->threads->map[0] = getpid();
1332+
1333+ err = perf_evlist__open(evlist);
1334+ if (err < 0) {
1335+ pr_debug("perf_evlist__open: %s\n", strerror(errno));
1336+ goto out_delete_evlist;
1337+ }
1338+
1339+ err = perf_evlist__mmap(evlist, UINT_MAX, false);
1340+ if (err < 0) {
1341+ pr_debug("perf_evlist__mmap: %s\n", strerror(errno));
1342+ goto out_delete_evlist;
1343+ }
1344+
1345+ perf_evlist__enable(evlist);
1346+
1347+ /*
1348+ * Generate the event:
1349+ */
1350+ open(filename, flags);
1351+
1352+ while (1) {
1353+ int before = nr_events;
1354+
1355+ for (i = 0; i < evlist->nr_mmaps; i++) {
1356+ union perf_event *event;
1357+
1358+ while ((event = perf_evlist__mmap_read(evlist, i)) != NULL) {
1359+ const u32 type = event->header.type;
1360+ int tp_flags;
1361+ struct perf_sample sample;
1362+
1363+ ++nr_events;
1364+
1365+ if (type != PERF_RECORD_SAMPLE)
1366+ continue;
1367+
1368+ err = perf_evsel__parse_sample(evsel, event, &sample);
1369+ if (err) {
1370+ pr_err("Can't parse sample, err = %d\n", err);
1371+ goto out_munmap;
1372+ }
1373+
1374+ tp_flags = perf_evsel__intval(evsel, &sample, "flags");
1375+
1376+ if (flags != tp_flags) {
1377+ pr_debug("%s: Expected flags=%#x, got %#x\n",
1378+ __func__, flags, tp_flags);
1379+ goto out_munmap;
1380+ }
1381+
1382+ goto out_ok;
1383+ }
1384+ }
1385+
1386+ if (nr_events == before)
1387+ poll(evlist->pollfd, evlist->nr_fds, 10);
1388+
1389+ if (++nr_polls > 5) {
1390+ pr_debug("%s: no events!\n", __func__);
1391+ goto out_munmap;
1392+ }
1393+ }
1394+out_ok:
1395+ err = 0;
1396+out_munmap:
1397+ perf_evlist__munmap(evlist);
1398+out_delete_evlist:
1399+ perf_evlist__delete(evlist);
1400+out:
1401+ return err;
1402+}
1403+
10951404 static struct test {
10961405 const char *desc;
10971406 int (*func)(void);
@@ -1134,6 +1443,18 @@ static struct test { · 36 unmodified lines
11341443 .desc = "Test dso data interface",
11351444 .func = dso__test_data,
11361445 },
1446+ {
1447+ .desc = "roundtrip evsel->name check",
1448+ .func = perf_evsel__roundtrip_name_test,
1449+ },
1450+ {
1451+ .desc = "Check parsing of sched tracepoints fields",
1452+ .func = perf_evsel__tp_sched_test,
1453+ },
1454+ {
1455+ .desc = "Generate and check syscalls:sys_enter_open event fields",
1456+ .func = test__syscall_open_tp_fields,
1457+ },
11371458 {
11381459 .func = NULL,
11391460 },
@@ -1199,7 +1520,7 @@ static int perf_test__list(int argc, const char **argv) · 59 unmodified lines
11991520 return 0;
12001521 }
12011522
1202−int cmd_test(int argc, const char **argv, const char *prefix __used)
1523+int cmd_test(int argc, const char **argv, const char *prefix __maybe_unused)
12031524 {
12041525 const char * const test_usage[] = {
12051526 "perf test [<options>] [{list <test-name-fragment>|[<test-name-fragments>|<test-numbers>]}]",
tools/perf/builtin-timechart.c+30−40
@@ -168,9 +168,8 @@ static struct per_pid *find_create_pid(int pid) · 167 unmodified lines
168168 return cursor;
169169 cursor = cursor->next;
170170 }
171− cursor = malloc(sizeof(struct per_pid));
171+ cursor = zalloc(sizeof(*cursor));
172172 assert(cursor != NULL);
173− memset(cursor, 0, sizeof(struct per_pid));
174173 cursor->pid = pid;
175174 cursor->next = all_data;
176175 all_data = cursor;
@@ -195,9 +194,8 @@ static void pid_set_comm(int pid, char *comm) · 18 unmodified lines
195194 }
196195 c = c->next;
197196 }
198− c = malloc(sizeof(struct per_pidcomm));
197+ c = zalloc(sizeof(*c));
199198 assert(c != NULL);
200− memset(c, 0, sizeof(struct per_pidcomm));
201199 c->comm = strdup(comm);
202200 p->current = c;
203201 c->next = p->all;
@@ -239,17 +237,15 @@ pid_put_sample(int pid, int type, unsigned int cpu, u64 start, u64 end) · 35 unmodified lines
239237 p = find_create_pid(pid);
240238 c = p->current;
241239 if (!c) {
242− c = malloc(sizeof(struct per_pidcomm));
240+ c = zalloc(sizeof(*c));
243241 assert(c != NULL);
244− memset(c, 0, sizeof(struct per_pidcomm));
245242 p->current = c;
246243 c->next = p->all;
247244 p->all = c;
248245 }
249246
250− sample = malloc(sizeof(struct cpu_sample));
247+ sample = zalloc(sizeof(*sample));
251248 assert(sample != NULL);
252− memset(sample, 0, sizeof(struct cpu_sample));
253249 sample->start_time = start;
254250 sample->end_time = end;
255251 sample->type = type;
@@ -275,28 +271,28 @@ static int cpus_cstate_state[MAX_CPUS]; · 19 unmodified lines
275271 static u64 cpus_pstate_start_times[MAX_CPUS];
276272 static u64 cpus_pstate_state[MAX_CPUS];
277273
278−static int process_comm_event(struct perf_tool *tool __used,
274+static int process_comm_event(struct perf_tool *tool __maybe_unused,
279275 union perf_event *event,
280− struct perf_sample *sample __used,
281− struct machine *machine __used)
276+ struct perf_sample *sample __maybe_unused,
277+ struct machine *machine __maybe_unused)
282278 {
283279 pid_set_comm(event->comm.tid, event->comm.comm);
284280 return 0;
285281 }
286282
287−static int process_fork_event(struct perf_tool *tool __used,
283+static int process_fork_event(struct perf_tool *tool __maybe_unused,
288284 union perf_event *event,
289− struct perf_sample *sample __used,
290− struct machine *machine __used)
285+ struct perf_sample *sample __maybe_unused,
286+ struct machine *machine __maybe_unused)
291287 {
292288 pid_fork(event->fork.pid, event->fork.ppid, event->fork.time);
293289 return 0;
294290 }
295291
296−static int process_exit_event(struct perf_tool *tool __used,
292+static int process_exit_event(struct perf_tool *tool __maybe_unused,
297293 union perf_event *event,
298− struct perf_sample *sample __used,
299− struct machine *machine __used)
294+ struct perf_sample *sample __maybe_unused,
295+ struct machine *machine __maybe_unused)
300296 {
301297 pid_exit(event->fork.pid, event->fork.time);
302298 return 0;
@@ -373,11 +369,10 @@ static void c_state_start(int cpu, u64 timestamp, int state) · 70 unmodified lines
373369
374370 static void c_state_end(int cpu, u64 timestamp)
375371 {
376− struct power_event *pwr;
377− pwr = malloc(sizeof(struct power_event));
372+ struct power_event *pwr = zalloc(sizeof(*pwr));
373+
378374 if (!pwr)
379375 return;
380− memset(pwr, 0, sizeof(struct power_event));
381376
382377 pwr->state = cpus_cstate_state[cpu];
383378 pwr->start_time = cpus_cstate_start_times[cpu];
@@ -392,14 +387,13 @@ static void c_state_end(int cpu, u64 timestamp) · 8 unmodified lines
392387 static void p_state_change(int cpu, u64 timestamp, u64 new_freq)
393388 {
394389 struct power_event *pwr;
395− pwr = malloc(sizeof(struct power_event));
396390
397391 if (new_freq > 8000000) /* detect invalid data */
398392 return;
399393
394+ pwr = zalloc(sizeof(*pwr));
400395 if (!pwr)
401396 return;
402− memset(pwr, 0, sizeof(struct power_event));
403397
404398 pwr->state = cpus_pstate_state[cpu];
405399 pwr->start_time = cpus_pstate_start_times[cpu];
@@ -429,15 +423,13 @@ static void p_state_change(int cpu, u64 timestamp, u64 new_freq) · 23 unmodified lines
429423 static void
430424 sched_wakeup(int cpu, u64 timestamp, int pid, struct trace_entry *te)
431425 {
432− struct wake_event *we;
433426 struct per_pid *p;
434427 struct wakeup_entry *wake = (void *)te;
428+ struct wake_event *we = zalloc(sizeof(*we));
435429
436− we = malloc(sizeof(struct wake_event));
437430 if (!we)
438431 return;
439432
440− memset(we, 0, sizeof(struct wake_event));
441433 we->time = timestamp;
442434 we->waker = pid;
443435
@@ -491,11 +483,11 @@ static void sched_switch(int cpu, u64 timestamp, struct trace_entry *te) · 47 unmodified lines
491483 }
492484
493485
494−static int process_sample_event(struct perf_tool *tool __used,
495− union perf_event *event __used,
486+static int process_sample_event(struct perf_tool *tool __maybe_unused,
487+ union perf_event *event __maybe_unused,
496488 struct perf_sample *sample,
497489 struct perf_evsel *evsel,
498− struct machine *machine __used)
490+ struct machine *machine __maybe_unused)
499491 {
500492 struct trace_entry *te;
501493
@@ -579,13 +571,12 @@ static void end_sample_processing(void) · 77 unmodified lines
579571 struct power_event *pwr;
580572
581573 for (cpu = 0; cpu <= numcpus; cpu++) {
582− pwr = malloc(sizeof(struct power_event));
574+ /* C state */
575+#if 0
576+ pwr = zalloc(sizeof(*pwr));
583577 if (!pwr)
584578 return;
585− memset(pwr, 0, sizeof(struct power_event));
586579
587− /* C state */
588−#if 0
589580 pwr->state = cpus_cstate_state[cpu];
590581 pwr->start_time = cpus_cstate_start_times[cpu];
591582 pwr->end_time = last_time;
@@ -597,10 +588,9 @@ static void end_sample_processing(void) · 5 unmodified lines
597588 #endif
598589 /* P state */
599590
600− pwr = malloc(sizeof(struct power_event));
591+ pwr = zalloc(sizeof(*pwr));
601592 if (!pwr)
602593 return;
603− memset(pwr, 0, sizeof(struct power_event));
604594
605595 pwr->state = cpus_pstate_state[cpu];
606596 pwr->start_time = cpus_pstate_start_times[cpu];
@@ -830,11 +820,9 @@ static void draw_process_bars(void) · 223 unmodified lines
830820
831821 static void add_process_filter(const char *string)
832822 {
833− struct process_filter *filt;
834− int pid;
823+ int pid = strtoull(string, NULL, 10);
824+ struct process_filter *filt = malloc(sizeof(*filt));
835825
836− pid = strtoull(string, NULL, 10);
837− filt = malloc(sizeof(struct process_filter));
838826 if (!filt)
839827 return;
840828
@@ -1081,7 +1069,8 @@ static int __cmd_record(int argc, const char **argv) · 240 unmodified lines
10811069 }
10821070
10831071 static int
1084−parse_process(const struct option *opt __used, const char *arg, int __used unset)
1072+parse_process(const struct option *opt __maybe_unused, const char *arg,
1073+ int __maybe_unused unset)
10851074 {
10861075 if (arg)
10871076 add_process_filter(arg);
@@ -1106,7 +1095,8 @@ static const struct option options[] = { · 18 unmodified lines
11061095 };
11071096
11081097
1109−int cmd_timechart(int argc, const char **argv, const char *prefix __used)
1098+int cmd_timechart(int argc, const char **argv,
1099+ const char *prefix __maybe_unused)
11101100 {
11111101 argc = parse_options(argc, argv, options, timechart_usage,
11121102 PARSE_OPT_STOP_AT_NON_OPTION);
tools/perf/builtin-top.c+16−17
@@ -95,7 +95,8 @@ static void perf_top__update_print_entries(struct perf_top *top) · 94 unmodified lines
9595 top->print_entries -= 9;
9696 }
9797
98−static void perf_top__sig_winch(int sig __used, siginfo_t *info __used, void *arg)
98+static void perf_top__sig_winch(int sig __maybe_unused,
99+ siginfo_t *info __maybe_unused, void *arg)
99100 {
100101 struct perf_top *top = arg;
101102
@@ -509,7 +510,7 @@ static void perf_top__handle_keypress(struct perf_top *top, int c) · 407 unmodified lines
509510 prompt_integer(&counter, "Enter details event counter");
510511
511512 if (counter >= top->evlist->nr_entries) {
512− top->sym_evsel = list_entry(top->evlist->entries.next, struct perf_evsel, node);
513+ top->sym_evsel = perf_evlist__first(top->evlist);
513514 fprintf(stderr, "Sorry, no such event, using %s.\n", perf_evsel__name(top->sym_evsel));
514515 sleep(1);
515516 break;
@@ -518,7 +519,7 @@ static void perf_top__handle_keypress(struct perf_top *top, int c) · 2 unmodified lines
518519 if (top->sym_evsel->idx == counter)
519520 break;
520521 } else
521− top->sym_evsel = list_entry(top->evlist->entries.next, struct perf_evsel, node);
522+ top->sym_evsel = perf_evlist__first(top->evlist);
522523 break;
523524 case 'f':
524525 prompt_integer(&top->count_filter, "Enter display event count filter");
@@ -663,7 +664,7 @@ static const char *skip_symbols[] = { · 138 unmodified lines
663664 NULL
664665 };
665666
666−static int symbol_filter(struct map *map __used, struct symbol *sym)
667+static int symbol_filter(struct map *map __maybe_unused, struct symbol *sym)
667668 {
668669 const char *name = sym->name;
669670 int i;
@@ -783,8 +784,10 @@ static void perf_event__process_sample(struct perf_tool *tool, · 113 unmodified lines
783784
784785 if ((sort__has_parent || symbol_conf.use_callchain) &&
785786 sample->callchain) {
786− err = machine__resolve_callchain(machine, al.thread,
787− sample->callchain, &parent);
787+ err = machine__resolve_callchain(machine, evsel,
788+ al.thread, sample,
789+ &parent);
790+
788791 if (err)
789792 return;
790793 }
@@ -820,7 +823,7 @@ static void perf_top__mmap_read_idx(struct perf_top *top, int idx) · 29 unmodified lines
820823 int ret;
821824
822825 while ((event = perf_evlist__mmap_read(top->evlist, idx)) != NULL) {
823− ret = perf_evlist__parse_sample(top->evlist, event, &sample, false);
826+ ret = perf_evlist__parse_sample(top->evlist, event, &sample);
824827 if (ret) {
825828 pr_err("Can't parse sample, err = %d\n", ret);
826829 continue;
@@ -884,17 +887,14 @@ static void perf_top__mmap_read(struct perf_top *top) · 57 unmodified lines
884887
885888 static void perf_top__start_counters(struct perf_top *top)
886889 {
887− struct perf_evsel *counter, *first;
890+ struct perf_evsel *counter;
888891 struct perf_evlist *evlist = top->evlist;
889892
890− first = list_entry(evlist->entries.next, struct perf_evsel, node);
893+ if (top->group)
894+ perf_evlist__set_leader(evlist);
891895
892896 list_for_each_entry(counter, &evlist->entries, node) {
893897 struct perf_event_attr *attr = &counter->attr;
894− struct xyarray *group_fd = NULL;
895−
896− if (top->group && counter != first)
897− group_fd = first->fd;
898898
899899 attr->sample_type = PERF_SAMPLE_IP | PERF_SAMPLE_TID;
900900
@@ -925,8 +925,7 @@ retry_sample_id: · 24 unmodified lines
925925 attr->sample_id_all = top->sample_id_all_missing ? 0 : 1;
926926 try_again:
927927 if (perf_evsel__open(counter, top->evlist->cpus,
928− top->evlist->threads, top->group,
929− group_fd) < 0) {
928+ top->evlist->threads) < 0) {
930929 int err = errno;
931930
932931 if (err == EPERM || err == EACCES) {
@@ -1165,7 +1164,7 @@ static const char * const top_usage[] = { · 232 unmodified lines
11651164 NULL
11661165 };
11671166
1168−int cmd_top(int argc, const char **argv, const char *prefix __used)
1167+int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)
11691168 {
11701169 struct perf_evsel *pos;
11711170 int status;
@@ -1328,7 +1327,7 @@ int cmd_top(int argc, const char **argv, const char *prefix __used) · 156 unmodified lines
13281327 pos->attr.sample_period = top.default_interval;
13291328 }
13301329
1331− top.sym_evsel = list_entry(top.evlist->entries.next, struct perf_evsel, node);
1330+ top.sym_evsel = perf_evlist__first(top.evlist);
13321331
13331332 symbol_conf.priv_size = sizeof(struct annotation);
13341333
tools/perf/builtin-trace.c+310−0
@@ -0,0 +1,310 @@
1+#include "builtin.h"
2+#include "util/evlist.h"
3+#include "util/parse-options.h"
4+#include "util/thread_map.h"
5+#include "event-parse.h"
6+
7+#include <libaudit.h>
8+#include <stdlib.h>
9+
10+static struct syscall_fmt {
11+ const char *name;
12+ const char *alias;
13+ bool errmsg;
14+ bool timeout;
15+} syscall_fmts[] = {
16+ { .name = "arch_prctl", .errmsg = true, .alias = "prctl", },
17+ { .name = "fstat", .errmsg = true, .alias = "newfstat", },
18+ { .name = "fstatat", .errmsg = true, .alias = "newfstatat", },
19+ { .name = "futex", .errmsg = true, },
20+ { .name = "poll", .errmsg = true, .timeout = true, },
21+ { .name = "ppoll", .errmsg = true, .timeout = true, },
22+ { .name = "read", .errmsg = true, },
23+ { .name = "recvfrom", .errmsg = true, },
24+ { .name = "select", .errmsg = true, .timeout = true, },
25+ { .name = "stat", .errmsg = true, .alias = "newstat", },
26+};
27+
28+static int syscall_fmt__cmp(const void *name, const void *fmtp)
29+{
30+ const struct syscall_fmt *fmt = fmtp;
31+ return strcmp(name, fmt->name);
32+}
33+
34+static struct syscall_fmt *syscall_fmt__find(const char *name)
35+{
36+ const int nmemb = ARRAY_SIZE(syscall_fmts);
37+ return bsearch(name, syscall_fmts, nmemb, sizeof(struct syscall_fmt), syscall_fmt__cmp);
38+}
39+
40+struct syscall {
41+ struct event_format *tp_format;
42+ const char *name;
43+ struct syscall_fmt *fmt;
44+};
45+
46+struct trace {
47+ int audit_machine;
48+ struct {
49+ int max;
50+ struct syscall *table;
51+ } syscalls;
52+ struct perf_record_opts opts;
53+};
54+
55+static int trace__read_syscall_info(struct trace *trace, int id)
56+{
57+ char tp_name[128];
58+ struct syscall *sc;
59+
60+ if (id > trace->syscalls.max) {
61+ struct syscall *nsyscalls = realloc(trace->syscalls.table, (id + 1) * sizeof(*sc));
62+
63+ if (nsyscalls == NULL)
64+ return -1;
65+
66+ if (trace->syscalls.max != -1) {
67+ memset(nsyscalls + trace->syscalls.max + 1, 0,
68+ (id - trace->syscalls.max) * sizeof(*sc));
69+ } else {
70+ memset(nsyscalls, 0, (id + 1) * sizeof(*sc));
71+ }
72+
73+ trace->syscalls.table = nsyscalls;
74+ trace->syscalls.max = id;
75+ }
76+
77+ sc = trace->syscalls.table + id;
78+ sc->name = audit_syscall_to_name(id, trace->audit_machine);
79+ if (sc->name == NULL)
80+ return -1;
81+
82+ sc->fmt = syscall_fmt__find(sc->name);
83+
84+ snprintf(tp_name, sizeof(tp_name), "sys_enter_%s", sc->name);
85+ sc->tp_format = event_format__new("syscalls", tp_name);
86+
87+ if (sc->tp_format == NULL && sc->fmt && sc->fmt->alias) {
88+ snprintf(tp_name, sizeof(tp_name), "sys_enter_%s", sc->fmt->alias);
89+ sc->tp_format = event_format__new("syscalls", tp_name);
90+ }
91+
92+ return sc->tp_format != NULL ? 0 : -1;
93+}
94+
95+static size_t syscall__fprintf_args(struct syscall *sc, unsigned long *args, FILE *fp)
96+{
97+ int i = 0;
98+ size_t printed = 0;
99+
100+ if (sc->tp_format != NULL) {
101+ struct format_field *field;
102+
103+ for (field = sc->tp_format->format.fields->next; field; field = field->next) {
104+ printed += fprintf(fp, "%s%s: %ld", printed ? ", " : "",
105+ field->name, args[i++]);
106+ }
107+ } else {
108+ while (i < 6) {
109+ printed += fprintf(fp, "%sarg%d: %ld", printed ? ", " : "", i, args[i]);
110+ ++i;
111+ }
112+ }
113+
114+ return printed;
115+}
116+
117+static int trace__run(struct trace *trace)
118+{
119+ struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
120+ struct perf_evsel *evsel, *evsel_enter, *evsel_exit;
121+ int err = -1, i, nr_events = 0, before;
122+
123+ if (evlist == NULL) {
124+ printf("Not enough memory to run!\n");
125+ goto out;
126+ }
127+
128+ evsel_enter = perf_evsel__newtp("raw_syscalls", "sys_enter", 0);
129+ if (evsel_enter == NULL) {
130+ printf("Couldn't read the raw_syscalls:sys_enter tracepoint information!\n");
131+ goto out_delete_evlist;
132+ }
133+
134+ perf_evlist__add(evlist, evsel_enter);
135+
136+ evsel_exit = perf_evsel__newtp("raw_syscalls", "sys_exit", 1);
137+ if (evsel_exit == NULL) {
138+ printf("Couldn't read the raw_syscalls:sys_exit tracepoint information!\n");
139+ goto out_delete_evlist;
140+ }
141+
142+ perf_evlist__add(evlist, evsel_exit);
143+
144+ err = perf_evlist__create_maps(evlist, &trace->opts.target);
145+ if (err < 0) {
146+ printf("Problems parsing the target to trace, check your options!\n");
147+ goto out_delete_evlist;
148+ }
149+
150+ perf_evlist__config_attrs(evlist, &trace->opts);
151+
152+ err = perf_evlist__open(evlist);
153+ if (err < 0) {
154+ printf("Couldn't create the events: %s\n", strerror(errno));
155+ goto out_delete_evlist;
156+ }
157+
158+ err = perf_evlist__mmap(evlist, UINT_MAX, false);
159+ if (err < 0) {
160+ printf("Couldn't mmap the events: %s\n", strerror(errno));
161+ goto out_delete_evlist;
162+ }
163+
164+ perf_evlist__enable(evlist);
165+again:
166+ before = nr_events;
167+
168+ for (i = 0; i < evlist->nr_mmaps; i++) {
169+ union perf_event *event;
170+
171+ while ((event = perf_evlist__mmap_read(evlist, i)) != NULL) {
172+ const u32 type = event->header.type;
173+ struct syscall *sc;
174+ struct perf_sample sample;
175+ int id;
176+
177+ ++nr_events;
178+
179+ switch (type) {
180+ case PERF_RECORD_SAMPLE:
181+ break;
182+ case PERF_RECORD_LOST:
183+ printf("LOST %" PRIu64 " events!\n", event->lost.lost);
184+ continue;
185+ default:
186+ printf("Unexpected %s event, skipping...\n",
187+ perf_event__name(type));
188+ continue;
189+ }
190+
191+ err = perf_evlist__parse_sample(evlist, event, &sample);
192+ if (err) {
193+ printf("Can't parse sample, err = %d, skipping...\n", err);
194+ continue;
195+ }
196+
197+ evsel = perf_evlist__id2evsel(evlist, sample.id);
198+ if (evsel == NULL) {
199+ printf("Unknown tp ID %" PRIu64 ", skipping...\n", sample.id);
200+ continue;
201+ }
202+
203+ id = perf_evsel__intval(evsel, &sample, "id");
204+ if (id < 0) {
205+ printf("Invalid syscall %d id, skipping...\n", id);
206+ continue;
207+ }
208+
209+ if ((id > trace->syscalls.max || trace->syscalls.table[id].name == NULL) &&
210+ trace__read_syscall_info(trace, id))
211+ continue;
212+
213+ if ((id > trace->syscalls.max || trace->syscalls.table[id].name == NULL))
214+ continue;
215+
216+ sc = &trace->syscalls.table[id];
217+
218+ if (evlist->threads->map[0] == -1 || evlist->threads->nr > 1)
219+ printf("%d ", sample.tid);
220+
221+ if (evsel == evsel_enter) {
222+ void *args = perf_evsel__rawptr(evsel, &sample, "args");
223+
224+ printf("%s(", sc->name);
225+ syscall__fprintf_args(sc, args, stdout);
226+ } else if (evsel == evsel_exit) {
227+ int ret = perf_evsel__intval(evsel, &sample, "ret");
228+
229+ if (ret < 0 && sc->fmt && sc->fmt->errmsg) {
230+ char bf[256];
231+ const char *emsg = strerror_r(-ret, bf, sizeof(bf)),
232+ *e = audit_errno_to_name(-ret);
233+
234+ printf(") = -1 %s %s", e, emsg);
235+ } else if (ret == 0 && sc->fmt && sc->fmt->timeout)
236+ printf(") = 0 Timeout");
237+ else
238+ printf(") = %d", ret);
239+
240+ putchar('\n');
241+ }
242+ }
243+ }
244+
245+ if (nr_events == before)
246+ poll(evlist->pollfd, evlist->nr_fds, -1);
247+
248+ goto again;
249+
250+out_delete_evlist:
251+ perf_evlist__delete(evlist);
252+out:
253+ return err;
254+}
255+
256+int cmd_trace(int argc, const char **argv, const char *prefix __maybe_unused)
257+{
258+ const char * const trace_usage[] = {
259+ "perf trace [<options>]",
260+ NULL
261+ };
262+ struct trace trace = {
263+ .audit_machine = audit_detect_machine(),
264+ .syscalls = {
265+ . max = -1,
266+ },
267+ .opts = {
268+ .target = {
269+ .uid = UINT_MAX,
270+ .uses_mmap = true,
271+ },
272+ .user_freq = UINT_MAX,
273+ .user_interval = ULLONG_MAX,
274+ .no_delay = true,
275+ .mmap_pages = 1024,
276+ },
277+ };
278+ const struct option trace_options[] = {
279+ OPT_STRING('p', "pid", &trace.opts.target.pid, "pid",
280+ "trace events on existing process id"),
281+ OPT_STRING(0, "tid", &trace.opts.target.tid, "tid",
282+ "trace events on existing thread id"),
283+ OPT_BOOLEAN(0, "all-cpus", &trace.opts.target.system_wide,
284+ "system-wide collection from all CPUs"),
285+ OPT_STRING(0, "cpu", &trace.opts.target.cpu_list, "cpu",
286+ "list of cpus to monitor"),
287+ OPT_BOOLEAN(0, "no-inherit", &trace.opts.no_inherit,
288+ "child tasks do not inherit counters"),
289+ OPT_UINTEGER(0, "mmap-pages", &trace.opts.mmap_pages,
290+ "number of mmap data pages"),
291+ OPT_STRING(0, "uid", &trace.opts.target.uid_str, "user",
292+ "user to profile"),
293+ OPT_END()
294+ };
295+ int err;
296+
297+ argc = parse_options(argc, argv, trace_options, trace_usage, 0);
298+ if (argc)
299+ usage_with_options(trace_usage, trace_options);
300+
301+ err = perf_target__parse_uid(&trace.opts.target);
302+ if (err) {
303+ char bf[BUFSIZ];
304+ perf_target__strerror(&trace.opts.target, err, bf, sizeof(bf));
305+ printf("%s", bf);
306+ return err;
307+ }
308+
309+ return trace__run(&trace);
310+}
tools/perf/builtin.h+2−0
@@ -34,6 +34,8 @@ extern int cmd_kmem(int argc, const char **argv, const char *prefix); · 33 unmodified lines
3434 extern int cmd_lock(int argc, const char **argv, const char *prefix);
3535 extern int cmd_kvm(int argc, const char **argv, const char *prefix);
3636 extern int cmd_test(int argc, const char **argv, const char *prefix);
37+extern int cmd_trace(int argc, const char **argv, const char *prefix);
3738 extern int cmd_inject(int argc, const char **argv, const char *prefix);
3839
40+extern int find_scripts(char **scripts_array, char **scripts_path_array);
3941 #endif
tools/perf/command-list.txt+2−1
@@ -17,8 +17,9 @@ perf-report mainporcelain common · 16 unmodified lines
1717 perf-stat mainporcelain common
1818 perf-timechart mainporcelain common
1919 perf-top mainporcelain common
20+perf-trace mainporcelain common
2021 perf-script mainporcelain common
21−perf-probe mainporcelain common
22+perf-probe mainporcelain full
2223 perf-kmem mainporcelain common
2324 perf-lock mainporcelain common
2425 perf-kvm mainporcelain common
tools/perf/config/feature-tests.mak+50−0
@@ -154,3 +154,53 @@ int main(void) · 153 unmodified lines
154154 return 0;
155155 }
156156 endef
157+
158+ifndef NO_LIBUNWIND
159+define SOURCE_LIBUNWIND
160+#include <libunwind.h>
161+#include <stdlib.h>
162+
163+extern int UNW_OBJ(dwarf_search_unwind_table) (unw_addr_space_t as,
164+ unw_word_t ip,
165+ unw_dyn_info_t *di,
166+ unw_proc_info_t *pi,
167+ int need_unwind_info, void *arg);
168+
169+
170+#define dwarf_search_unwind_table UNW_OBJ(dwarf_search_unwind_table)
171+
172+int main(void)
173+{
174+ unw_addr_space_t addr_space;
175+ addr_space = unw_create_addr_space(NULL, 0);
176+ unw_init_remote(NULL, addr_space, NULL);
177+ dwarf_search_unwind_table(addr_space, 0, NULL, NULL, 0, NULL);
178+ return 0;
179+}
180+endef
181+endif
182+
183+ifndef NO_BACKTRACE
184+define SOURCE_BACKTRACE
185+#include <execinfo.h>
186+#include <stdio.h>
187+
188+int main(void)
189+{
190+ backtrace(NULL, 0);
191+ backtrace_symbols(NULL, 0);
192+ return 0;
193+}
194+endef
195+endif
196+
197+ifndef NO_LIBAUDIT
198+define SOURCE_LIBAUDIT
199+#include <libaudit.h>
200+
201+int main(void)
202+{
203+ return audit_open();
204+}
205+endef
206+endif
tools/perf/perf-archive.sh+3−3
@@ -24,7 +24,7 @@ NOBUILDID=0000000000000000000000000000000000000000 · 23 unmodified lines
2424 perf buildid-list -i $PERF_DATA --with-hits | grep -v "^$NOBUILDID " > $BUILDIDS
2525 if [ ! -s $BUILDIDS ] ; then
2626 echo "perf archive: no build-ids found"
27− rm -f $BUILDIDS
27+ rm $BUILDIDS || true
2828 exit 1
2929 fi
3030
@@ -39,8 +39,8 @@ while read build_id ; do · 8 unmodified lines
3939 echo ${filename#$PERF_BUILDID_LINKDIR} >> $MANIFEST
4040 done
4141
42−tar cfj $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
43−rm -f $MANIFEST $BUILDIDS
42+tar cjf $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
43+rm $MANIFEST $BUILDIDS || true
4444 echo -e "Now please run:\n"
4545 echo -e "$ tar xvf $PERF_DATA.tar.bz2 -C ~/.debug\n"
4646 echo "wherever you need to run 'perf report' on."
tools/perf/perf.c+45−30
@@ -14,6 +14,7 @@ · 13 unmodified lines
1414 #include "util/run-command.h"
1515 #include "util/parse-events.h"
1616 #include "util/debugfs.h"
17+#include <pthread.h>
1718
1819 const char perf_usage_string[] =
1920 "perf [--version] [--help] COMMAND [ARGS]";
@@ -24,6 +25,42 @@ const char perf_more_info_string[] = · 4 unmodified lines
2425 int use_browser = -1;
2526 static int use_pager = -1;
2627
28+struct cmd_struct {
29+ const char *cmd;
30+ int (*fn)(int, const char **, const char *);
31+ int option;
32+};
33+
34+static struct cmd_struct commands[] = {
35+ { "buildid-cache", cmd_buildid_cache, 0 },
36+ { "buildid-list", cmd_buildid_list, 0 },
37+ { "diff", cmd_diff, 0 },
38+ { "evlist", cmd_evlist, 0 },
39+ { "help", cmd_help, 0 },
40+ { "list", cmd_list, 0 },
41+ { "record", cmd_record, 0 },
42+ { "report", cmd_report, 0 },
43+ { "bench", cmd_bench, 0 },
44+ { "stat", cmd_stat, 0 },
45+ { "timechart", cmd_timechart, 0 },
46+ { "top", cmd_top, 0 },
47+ { "annotate", cmd_annotate, 0 },
48+ { "version", cmd_version, 0 },
49+ { "script", cmd_script, 0 },
50+ { "sched", cmd_sched, 0 },
51+#ifndef NO_LIBELF_SUPPORT
52+ { "probe", cmd_probe, 0 },
53+#endif
54+ { "kmem", cmd_kmem, 0 },
55+ { "lock", cmd_lock, 0 },
56+ { "kvm", cmd_kvm, 0 },
57+ { "test", cmd_test, 0 },
58+#ifndef NO_LIBAUDIT_SUPPORT
59+ { "trace", cmd_trace, 0 },
60+#endif
61+ { "inject", cmd_inject, 0 },
62+};
63+
2764 struct pager_config {
2865 const char *cmd;
2966 int val;
@@ -160,6 +197,14 @@ static int handle_options(const char ***argv, int *argc, int *envchanged) · 130 unmodified lines
160197 fprintf(stderr, "dir: %s\n", debugfs_mountpoint);
161198 if (envchanged)
162199 *envchanged = 1;
200+ } else if (!strcmp(cmd, "--list-cmds")) {
201+ unsigned int i;
202+
203+ for (i = 0; i < ARRAY_SIZE(commands); i++) {
204+ struct cmd_struct *p = commands+i;
205+ printf("%s ", p->cmd);
206+ }
207+ exit(0);
163208 } else {
164209 fprintf(stderr, "Unknown option: %s\n", cmd);
165210 usage(perf_usage_string);
@@ -245,12 +290,6 @@ const char perf_version_string[] = PERF_VERSION; · 79 unmodified lines
245290 */
246291 #define NEED_WORK_TREE (1<<2)
247292
248−struct cmd_struct {
249− const char *cmd;
250− int (*fn)(int, const char **, const char *);
251− int option;
252−};
253−
254293 static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
255294 {
256295 int status;
@@ -296,30 +335,6 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv) · 39 unmodified lines
296335 static void handle_internal_command(int argc, const char **argv)
297336 {
298337 const char *cmd = argv[0];
299− static struct cmd_struct commands[] = {
300− { "buildid-cache", cmd_buildid_cache, 0 },
301− { "buildid-list", cmd_buildid_list, 0 },
302− { "diff", cmd_diff, 0 },
303− { "evlist", cmd_evlist, 0 },
304− { "help", cmd_help, 0 },
305− { "list", cmd_list, 0 },
306− { "record", cmd_record, 0 },
307− { "report", cmd_report, 0 },
308− { "bench", cmd_bench, 0 },
309− { "stat", cmd_stat, 0 },
310− { "timechart", cmd_timechart, 0 },
311− { "top", cmd_top, 0 },
312− { "annotate", cmd_annotate, 0 },
313− { "version", cmd_version, 0 },
314− { "script", cmd_script, 0 },
315− { "sched", cmd_sched, 0 },
316− { "probe", cmd_probe, 0 },
317− { "kmem", cmd_kmem, 0 },
318− { "lock", cmd_lock, 0 },
319− { "kvm", cmd_kvm, 0 },
320− { "test", cmd_test, 0 },
321− { "inject", cmd_inject, 0 },
322− };
323338 unsigned int i;
324339 static const char ext[] = STRIP_EXTENSION;
325340
tools/perf/perf.h+8−1
@@ -209,9 +209,15 @@ void pthread__unblock_sigwinch(void); · 208 unmodified lines
209209
210210 #include "util/target.h"
211211
212+enum perf_call_graph_mode {
213+ CALLCHAIN_NONE,
214+ CALLCHAIN_FP,
215+ CALLCHAIN_DWARF
216+};
217+
212218 struct perf_record_opts {
213219 struct perf_target target;
214− bool call_graph;
220+ int call_graph;
215221 bool group;
216222 bool inherit_stat;
217223 bool no_delay;
@@ -230,6 +236,7 @@ struct perf_record_opts { · 12 unmodified lines
230236 u64 branch_stack;
231237 u64 default_interval;
232238 u64 user_interval;
239+ u16 stack_dump_size;
233240 };
234241
235242 #endif
tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py+94−0
@@ -0,0 +1,94 @@
1+# EventClass.py
2+#
3+# This is a library defining some events types classes, which could
4+# be used by other scripts to analyzing the perf samples.
5+#
6+# Currently there are just a few classes defined for examples,
7+# PerfEvent is the base class for all perf event sample, PebsEvent
8+# is a HW base Intel x86 PEBS event, and user could add more SW/HW
9+# event classes based on requirements.
10+
11+import struct
12+
13+# Event types, user could add more here
14+EVTYPE_GENERIC = 0
15+EVTYPE_PEBS = 1 # Basic PEBS event
16+EVTYPE_PEBS_LL = 2 # PEBS event with load latency info
17+EVTYPE_IBS = 3
18+
19+#
20+# Currently we don't have good way to tell the event type, but by
21+# the size of raw buffer, raw PEBS event with load latency data's
22+# size is 176 bytes, while the pure PEBS event's size is 144 bytes.
23+#
24+def create_event(name, comm, dso, symbol, raw_buf):
25+ if (len(raw_buf) == 144):
26+ event = PebsEvent(name, comm, dso, symbol, raw_buf)
27+ elif (len(raw_buf) == 176):
28+ event = PebsNHM(name, comm, dso, symbol, raw_buf)
29+ else:
30+ event = PerfEvent(name, comm, dso, symbol, raw_buf)
31+
32+ return event
33+
34+class PerfEvent(object):
35+ event_num = 0
36+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_GENERIC):
37+ self.name = name
38+ self.comm = comm
39+ self.dso = dso
40+ self.symbol = symbol
41+ self.raw_buf = raw_buf
42+ self.ev_type = ev_type
43+ PerfEvent.event_num += 1
44+
45+ def show(self):
46+ print "PMU event: name=%12s, symbol=%24s, comm=%8s, dso=%12s" % (self.name, self.symbol, self.comm, self.dso)
47+
48+#
49+# Basic Intel PEBS (Precise Event-based Sampling) event, whose raw buffer
50+# contains the context info when that event happened: the EFLAGS and
51+# linear IP info, as well as all the registers.
52+#
53+class PebsEvent(PerfEvent):
54+ pebs_num = 0
55+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_PEBS):
56+ tmp_buf=raw_buf[0:80]
57+ flags, ip, ax, bx, cx, dx, si, di, bp, sp = struct.unpack('QQQQQQQQQQ', tmp_buf)
58+ self.flags = flags
59+ self.ip = ip
60+ self.ax = ax
61+ self.bx = bx
62+ self.cx = cx
63+ self.dx = dx
64+ self.si = si
65+ self.di = di
66+ self.bp = bp
67+ self.sp = sp
68+
69+ PerfEvent.__init__(self, name, comm, dso, symbol, raw_buf, ev_type)
70+ PebsEvent.pebs_num += 1
71+ del tmp_buf
72+
73+#
74+# Intel Nehalem and Westmere support PEBS plus Load Latency info which lie
75+# in the four 64 bit words write after the PEBS data:
76+# Status: records the IA32_PERF_GLOBAL_STATUS register value
77+# DLA: Data Linear Address (EIP)
78+# DSE: Data Source Encoding, where the latency happens, hit or miss
79+# in L1/L2/L3 or IO operations
80+# LAT: the actual latency in cycles
81+#
82+class PebsNHM(PebsEvent):
83+ pebs_nhm_num = 0
84+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_PEBS_LL):
85+ tmp_buf=raw_buf[144:176]
86+ status, dla, dse, lat = struct.unpack('QQQQ', tmp_buf)
87+ self.status = status
88+ self.dla = dla
89+ self.dse = dse
90+ self.lat = lat
91+
92+ PebsEvent.__init__(self, name, comm, dso, symbol, raw_buf, ev_type)
93+ PebsNHM.pebs_nhm_num += 1
94+ del tmp_buf
tools/perf/scripts/python/bin/event_analyzing_sample-record+8−0
@@ -0,0 +1,8 @@
1+#!/bin/bash
2+
3+#
4+# event_analyzing_sample.py can cover all type of perf samples including
5+# the tracepoints, so no special record requirements, just record what
6+# you want to analyze.
7+#
8+perf record $@
tools/perf/scripts/python/bin/event_analyzing_sample-report+3−0
@@ -0,0 +1,3 @@
1+#!/bin/bash
2+# description: analyze all perf samples
3+perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/event_analyzing_sample.py
tools/perf/scripts/python/event_analyzing_sample.py+189−0
@@ -0,0 +1,189 @@
1+# event_analyzing_sample.py: general event handler in python
2+#
3+# Current perf report is already very powerful with the annotation integrated,
4+# and this script is not trying to be as powerful as perf report, but
5+# providing end user/developer a flexible way to analyze the events other
6+# than trace points.
7+#
8+# The 2 database related functions in this script just show how to gather
9+# the basic information, and users can modify and write their own functions
10+# according to their specific requirement.
11+#
12+# The first function "show_general_events" just does a basic grouping for all
13+# generic events with the help of sqlite, and the 2nd one "show_pebs_ll" is
14+# for a x86 HW PMU event: PEBS with load latency data.
15+#
16+
17+import os
18+import sys
19+import math
20+import struct
21+import sqlite3
22+
23+sys.path.append(os.environ['PERF_EXEC_PATH'] + \
24+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
25+
26+from perf_trace_context import *
27+from EventClass import *
28+
29+#
30+# If the perf.data has a big number of samples, then the insert operation
31+# will be very time consuming (about 10+ minutes for 10000 samples) if the
32+# .db database is on disk. Move the .db file to RAM based FS to speedup
33+# the handling, which will cut the time down to several seconds.
34+#
35+con = sqlite3.connect("/dev/shm/perf.db")
36+con.isolation_level = None
37+
38+def trace_begin():
39+ print "In trace_begin:\n"
40+
41+ #
42+ # Will create several tables at the start, pebs_ll is for PEBS data with
43+ # load latency info, while gen_events is for general event.
44+ #
45+ con.execute("""
46+ create table if not exists gen_events (
47+ name text,
48+ symbol text,
49+ comm text,
50+ dso text
51+ );""")
52+ con.execute("""
53+ create table if not exists pebs_ll (
54+ name text,
55+ symbol text,
56+ comm text,
57+ dso text,
58+ flags integer,
59+ ip integer,
60+ status integer,
61+ dse integer,
62+ dla integer,
63+ lat integer
64+ );""")
65+
66+#
67+# Create and insert event object to a database so that user could
68+# do more analysis with simple database commands.
69+#
70+def process_event(param_dict):
71+ event_attr = param_dict["attr"]
72+ sample = param_dict["sample"]
73+ raw_buf = param_dict["raw_buf"]
74+ comm = param_dict["comm"]
75+ name = param_dict["ev_name"]
76+
77+ # Symbol and dso info are not always resolved
78+ if (param_dict.has_key("dso")):
79+ dso = param_dict["dso"]
80+ else:
81+ dso = "Unknown_dso"
82+
83+ if (param_dict.has_key("symbol")):
84+ symbol = param_dict["symbol"]
85+ else:
86+ symbol = "Unknown_symbol"
87+
88+ # Create the event object and insert it to the right table in database
89+ event = create_event(name, comm, dso, symbol, raw_buf)
90+ insert_db(event)
91+
92+def insert_db(event):
93+ if event.ev_type == EVTYPE_GENERIC:
94+ con.execute("insert into gen_events values(?, ?, ?, ?)",
95+ (event.name, event.symbol, event.comm, event.dso))
96+ elif event.ev_type == EVTYPE_PEBS_LL:
97+ event.ip &= 0x7fffffffffffffff
98+ event.dla &= 0x7fffffffffffffff
99+ con.execute("insert into pebs_ll values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
100+ (event.name, event.symbol, event.comm, event.dso, event.flags,
101+ event.ip, event.status, event.dse, event.dla, event.lat))
102+
103+def trace_end():
104+ print "In trace_end:\n"
105+ # We show the basic info for the 2 type of event classes
106+ show_general_events()
107+ show_pebs_ll()
108+ con.close()
109+
110+#
111+# As the event number may be very big, so we can't use linear way
112+# to show the histogram in real number, but use a log2 algorithm.
113+#
114+
115+def num2sym(num):
116+ # Each number will have at least one '#'
117+ snum = '#' * (int)(math.log(num, 2) + 1)
118+ return snum
119+
120+def show_general_events():
121+
122+ # Check the total record number in the table
123+ count = con.execute("select count(*) from gen_events")
124+ for t in count:
125+ print "There is %d records in gen_events table" % t[0]
126+ if t[0] == 0:
127+ return
128+
129+ print "Statistics about the general events grouped by thread/symbol/dso: \n"
130+
131+ # Group by thread
132+ commq = con.execute("select comm, count(comm) from gen_events group by comm order by -count(comm)")
133+ print "\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42)
134+ for row in commq:
135+ print "%16s %8d %s" % (row[0], row[1], num2sym(row[1]))
136+
137+ # Group by symbol
138+ print "\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58)
139+ symbolq = con.execute("select symbol, count(symbol) from gen_events group by symbol order by -count(symbol)")
140+ for row in symbolq:
141+ print "%32s %8d %s" % (row[0], row[1], num2sym(row[1]))
142+
143+ # Group by dso
144+ print "\n%40s %8s %16s\n%s" % ("dso", "number", "histogram", "="*74)
145+ dsoq = con.execute("select dso, count(dso) from gen_events group by dso order by -count(dso)")
146+ for row in dsoq:
147+ print "%40s %8d %s" % (row[0], row[1], num2sym(row[1]))
148+
149+#
150+# This function just shows the basic info, and we could do more with the
151+# data in the tables, like checking the function parameters when some
152+# big latency events happen.
153+#
154+def show_pebs_ll():
155+
156+ count = con.execute("select count(*) from pebs_ll")
157+ for t in count:
158+ print "There is %d records in pebs_ll table" % t[0]
159+ if t[0] == 0:
160+ return
161+
162+ print "Statistics about the PEBS Load Latency events grouped by thread/symbol/dse/latency: \n"
163+
164+ # Group by thread
165+ commq = con.execute("select comm, count(comm) from pebs_ll group by comm order by -count(comm)")
166+ print "\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42)
167+ for row in commq:
168+ print "%16s %8d %s" % (row[0], row[1], num2sym(row[1]))
169+
170+ # Group by symbol
171+ print "\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58)
172+ symbolq = con.execute("select symbol, count(symbol) from pebs_ll group by symbol order by -count(symbol)")
173+ for row in symbolq:
174+ print "%32s %8d %s" % (row[0], row[1], num2sym(row[1]))
175+
176+ # Group by dse
177+ dseq = con.execute("select dse, count(dse) from pebs_ll group by dse order by -count(dse)")
178+ print "\n%32s %8s %16s\n%s" % ("dse", "number", "histogram", "="*58)
179+ for row in dseq:
180+ print "%32s %8d %s" % (row[0], row[1], num2sym(row[1]))
181+
182+ # Group by latency
183+ latq = con.execute("select lat, count(lat) from pebs_ll group by lat order by lat")
184+ print "\n%32s %8s %16s\n%s" % ("latency", "number", "histogram", "="*58)
185+ for row in latq:
186+ print "%32s %8d %s" % (row[0], row[1], num2sym(row[1]))
187+
188+def trace_unhandled(event_name, context, event_fields_dict):
189+ print ' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())])
tools/perf/ui/browser.c+4−3
@@ -269,7 +269,7 @@ int ui_browser__show(struct ui_browser *browser, const char *title, · 268 unmodified lines
269269 return err ? 0 : -1;
270270 }
271271
272−void ui_browser__hide(struct ui_browser *browser __used)
272+void ui_browser__hide(struct ui_browser *browser __maybe_unused)
273273 {
274274 pthread_mutex_lock(&ui__lock);
275275 ui_helpline__pop();
@@ -518,7 +518,7 @@ static struct ui_browser__colorset { · 242 unmodified lines
518518
519519
520520 static int ui_browser__color_config(const char *var, const char *value,
521− void *data __used)
521+ void *data __maybe_unused)
522522 {
523523 char *fg = NULL, *bg;
524524 int i;
@@ -602,7 +602,8 @@ void __ui_browser__vline(struct ui_browser *browser, unsigned int column, · 77 unmodified lines
602602 SLsmg_set_char_set(0);
603603 }
604604
605−void ui_browser__write_graph(struct ui_browser *browser __used, int graph)
605+void ui_browser__write_graph(struct ui_browser *browser __maybe_unused,
606+ int graph)
606607 {
607608 SLsmg_set_char_set(1);
608609 SLsmg_write_char(graph);
tools/perf/ui/browsers/annotate.c+4−2
@@ -54,7 +54,8 @@ static inline struct browser_disasm_line *disasm_line__browser(struct disasm_lin · 53 unmodified lines
5454 return (struct browser_disasm_line *)(dl + 1);
5555 }
5656
57−static bool disasm_line__filter(struct ui_browser *browser __used, void *entry)
57+static bool disasm_line__filter(struct ui_browser *browser __maybe_unused,
58+ void *entry)
5859 {
5960 if (annotate_browser__opts.hide_src_code) {
6061 struct disasm_line *dl = list_entry(entry, struct disasm_line, node);
@@ -928,7 +929,8 @@ static int annotate_config__cmp(const void *name, const void *cfgp) · 867 unmodified lines
928929 return strcmp(name, cfg->name);
929930 }
930931
931−static int annotate__config(const char *var, const char *value, void *data __used)
932+static int annotate__config(const char *var, const char *value,
933+ void *data __maybe_unused)
932934 {
933935 struct annotate__config *cfg;
934936 const char *name;
tools/perf/ui/browsers/hists.c+99−34
@@ -24,9 +24,12 @@ struct hist_browser { · 23 unmodified lines
2424 struct hist_entry *he_selection;
2525 struct map_symbol *selection;
2626 int print_seq;
27+ bool show_dso;
2728 bool has_symbols;
2829 };
2930
31+extern void hist_browser__init_hpp(void);
32+
3033 static int hists__browser_title(struct hists *hists, char *bf, size_t size,
3134 const char *ev_name);
3235
@@ -376,12 +379,19 @@ out: · 343 unmodified lines
376379 }
377380
378381 static char *callchain_list__sym_name(struct callchain_list *cl,
379− char *bf, size_t bfsize)
382+ char *bf, size_t bfsize, bool show_dso)
380383 {
384+ int printed;
385+
381386 if (cl->ms.sym)
382− return cl->ms.sym->name;
387+ printed = scnprintf(bf, bfsize, "%s", cl->ms.sym->name);
388+ else
389+ printed = scnprintf(bf, bfsize, "%#" PRIx64, cl->ip);
390+
391+ if (show_dso)
392+ scnprintf(bf + printed, bfsize - printed, " %s",
393+ cl->ms.map ? cl->ms.map->dso->short_name : "unknown");
383394
384− snprintf(bf, bfsize, "%#" PRIx64, cl->ip);
385395 return bf;
386396 }
387397
@@ -417,7 +427,7 @@ static int hist_browser__show_callchain_node_rb_tree(struct hist_browser *browse · 29 unmodified lines
417427 remaining -= cumul;
418428
419429 list_for_each_entry(chain, &child->val, list) {
420− char ipstr[BITS_PER_LONG / 4 + 1], *alloc_str;
430+ char bf[1024], *alloc_str;
421431 const char *str;
422432 int color;
423433 bool was_first = first;
@@ -434,7 +444,8 @@ static int hist_browser__show_callchain_node_rb_tree(struct hist_browser *browse · 10 unmodified lines
434444 }
435445
436446 alloc_str = NULL;
437− str = callchain_list__sym_name(chain, ipstr, sizeof(ipstr));
447+ str = callchain_list__sym_name(chain, bf, sizeof(bf),
448+ browser->show_dso);
438449 if (was_first) {
439450 double percent = cumul * 100.0 / new_total;
440451
@@ -493,7 +504,7 @@ static int hist_browser__show_callchain_node(struct hist_browser *browser, · 52 unmodified lines
493504 char folded_sign = ' ';
494505
495506 list_for_each_entry(chain, &node->val, list) {
496− char ipstr[BITS_PER_LONG / 4 + 1], *s;
507+ char bf[1024], *s;
497508 int color;
498509
499510 folded_sign = callchain_list__folded(chain);
@@ -510,7 +521,8 @@ static int hist_browser__show_callchain_node(struct hist_browser *browser, · 10 unmodified lines
510521 *is_current_entry = true;
511522 }
512523
513− s = callchain_list__sym_name(chain, ipstr, sizeof(ipstr));
524+ s = callchain_list__sym_name(chain, bf, sizeof(bf),
525+ browser->show_dso);
514526 ui_browser__gotorc(&browser->b, row, 0);
515527 ui_browser__set_color(&browser->b, color);
516528 slsmg_write_nstring(" ", offset);
@@ -553,14 +565,47 @@ static int hist_browser__show_callchain(struct hist_browser *browser, · 36 unmodified lines
553565 return row - first_row;
554566 }
555567
568+#define HPP__COLOR_FN(_name, _field) \
569+static int hist_browser__hpp_color_ ## _name(struct perf_hpp *hpp, \
570+ struct hist_entry *he) \
571+{ \
572+ double percent = 100.0 * he->_field / hpp->total_period; \
573+ *(double *)hpp->ptr = percent; \
574+ return scnprintf(hpp->buf, hpp->size, "%6.2f%%", percent); \
575+}
576+
577+HPP__COLOR_FN(overhead, period)
578+HPP__COLOR_FN(overhead_sys, period_sys)
579+HPP__COLOR_FN(overhead_us, period_us)
580+HPP__COLOR_FN(overhead_guest_sys, period_guest_sys)
581+HPP__COLOR_FN(overhead_guest_us, period_guest_us)
582+
583+#undef HPP__COLOR_FN
584+
585+void hist_browser__init_hpp(void)
586+{
587+ perf_hpp__init(false, false);
588+
589+ perf_hpp__format[PERF_HPP__OVERHEAD].color =
590+ hist_browser__hpp_color_overhead;
591+ perf_hpp__format[PERF_HPP__OVERHEAD_SYS].color =
592+ hist_browser__hpp_color_overhead_sys;
593+ perf_hpp__format[PERF_HPP__OVERHEAD_US].color =
594+ hist_browser__hpp_color_overhead_us;
595+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_SYS].color =
596+ hist_browser__hpp_color_overhead_guest_sys;
597+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_US].color =
598+ hist_browser__hpp_color_overhead_guest_us;
599+}
600+
556601 static int hist_browser__show_entry(struct hist_browser *browser,
557602 struct hist_entry *entry,
558603 unsigned short row)
559604 {
560605 char s[256];
561606 double percent;
562− int printed = 0;
563− int width = browser->b.width - 6; /* The percentage */
607+ int i, printed = 0;
608+ int width = browser->b.width;
564609 char folded_sign = ' ';
565610 bool current_entry = ui_browser__is_current_entry(&browser->b, row);
566611 off_t row_offset = entry->row_offset;
@@ -576,35 +621,50 @@ static int hist_browser__show_entry(struct hist_browser *browser, · 9 unmodified lines
576621 }
577622
578623 if (row_offset == 0) {
579− hist_entry__snprintf(entry, s, sizeof(s), browser->hists);
580− percent = (entry->period * 100.0) / browser->hists->stats.total_period;
624+ struct perf_hpp hpp = {
625+ .buf = s,
626+ .size = sizeof(s),
627+ .total_period = browser->hists->stats.total_period,
628+ };
581629
582− ui_browser__set_percent_color(&browser->b, percent, current_entry);
583630 ui_browser__gotorc(&browser->b, row, 0);
584− if (symbol_conf.use_callchain) {
585− slsmg_printf("%c ", folded_sign);
586− width -= 2;
587− }
588631
589− slsmg_printf(" %5.2f%%", percent);
632+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
633+ if (!perf_hpp__format[i].cond)
634+ continue;
590635
591− /* The scroll bar isn't being used */
592− if (!browser->b.navkeypressed)
593− width += 1;
636+ if (i) {
637+ slsmg_printf(" ");
638+ width -= 2;
639+ }
594640
595− if (!current_entry || !browser->b.navkeypressed)
596− ui_browser__set_color(&browser->b, HE_COLORSET_NORMAL);
641+ if (perf_hpp__format[i].color) {
642+ hpp.ptr = &percent;
643+ /* It will set percent for us. See HPP__COLOR_FN above. */
644+ width -= perf_hpp__format[i].color(&hpp, entry);
597645
598− if (symbol_conf.show_nr_samples) {
599− slsmg_printf(" %11u", entry->nr_events);
600− width -= 12;
601− }
646+ ui_browser__set_percent_color(&browser->b, percent, current_entry);
647+
648+ if (i == 0 && symbol_conf.use_callchain) {
649+ slsmg_printf("%c ", folded_sign);
650+ width -= 2;
651+ }
652+
653+ slsmg_printf("%s", s);
602654
603− if (symbol_conf.show_total_period) {
604− slsmg_printf(" %12" PRIu64, entry->period);
605− width -= 13;
655+ if (!current_entry || !browser->b.navkeypressed)
656+ ui_browser__set_color(&browser->b, HE_COLORSET_NORMAL);
657+ } else {
658+ width -= perf_hpp__format[i].entry(&hpp, entry);
659+ slsmg_printf("%s", s);
660+ }
606661 }
607662
663+ /* The scroll bar isn't being used */
664+ if (!browser->b.navkeypressed)
665+ width += 1;
666+
667+ hist_entry__sort_snprintf(entry, s, sizeof(s), browser->hists);
608668 slsmg_write_nstring(s, width);
609669 ++row;
610670 ++printed;
@@ -830,7 +890,7 @@ static int hist_browser__fprintf_callchain_node_rb_tree(struct hist_browser *bro · 219 unmodified lines
830890 remaining -= cumul;
831891
832892 list_for_each_entry(chain, &child->val, list) {
833− char ipstr[BITS_PER_LONG / 4 + 1], *alloc_str;
893+ char bf[1024], *alloc_str;
834894 const char *str;
835895 bool was_first = first;
836896
@@ -842,7 +902,8 @@ static int hist_browser__fprintf_callchain_node_rb_tree(struct hist_browser *bro · 5 unmodified lines
842902 folded_sign = callchain_list__folded(chain);
843903
844904 alloc_str = NULL;
845− str = callchain_list__sym_name(chain, ipstr, sizeof(ipstr));
905+ str = callchain_list__sym_name(chain, bf, sizeof(bf),
906+ browser->show_dso);
846907 if (was_first) {
847908 double percent = cumul * 100.0 / new_total;
848909
@@ -880,10 +941,10 @@ static int hist_browser__fprintf_callchain_node(struct hist_browser *browser, · 31 unmodified lines
880941 int printed = 0;
881942
882943 list_for_each_entry(chain, &node->val, list) {
883− char ipstr[BITS_PER_LONG / 4 + 1], *s;
944+ char bf[1024], *s;
884945
885946 folded_sign = callchain_list__folded(chain);
886− s = callchain_list__sym_name(chain, ipstr, sizeof(ipstr));
947+ s = callchain_list__sym_name(chain, bf, sizeof(bf), browser->show_dso);
887948 printed += fprintf(fp, "%*s%c %s\n", offset, " ", folded_sign, s);
888949 }
889950
@@ -920,7 +981,7 @@ static int hist_browser__fprintf_entry(struct hist_browser *browser, · 30 unmodified lines
920981 if (symbol_conf.use_callchain)
921982 folded_sign = hist_entry__folded(he);
922983
923− hist_entry__snprintf(he, s, sizeof(s), browser->hists);
984+ hist_entry__sort_snprintf(he, s, sizeof(s), browser->hists);
924985 percent = (he->period * 100.0) / browser->hists->stats.total_period;
925986
926987 if (symbol_conf.use_callchain)
@@ -1133,6 +1194,9 @@ static int perf_evsel__hists_browse(struct perf_evsel *evsel, int nr_events, · 206 unmodified lines
11331194 continue;
11341195 case 'd':
11351196 goto zoom_dso;
1197+ case 'V':
1198+ browser->show_dso = !browser->show_dso;
1199+ continue;
11361200 case 't':
11371201 goto zoom_thread;
11381202 case '/':
@@ -1164,6 +1228,7 @@ static int perf_evsel__hists_browse(struct perf_evsel *evsel, int nr_events, · 25 unmodified lines
11641228 "d Zoom into current DSO\n"
11651229 "t Zoom into current Thread\n"
11661230 "P Print histograms to perf.hist.N\n"
1231+ "V Verbose (DSO names in callchains, etc)\n"
11671232 "/ Filter symbol by name");
11681233 continue;
11691234 case K_ENTER:
tools/perf/ui/gtk/browser.c+92−19
@@ -3,6 +3,7 @@ · 2 unmodified lines
33 #include "../evsel.h"
44 #include "../sort.h"
55 #include "../hist.h"
6+#include "../helpline.h"
67 #include "gtk.h"
78
89 #include <signal.h>
@@ -35,6 +36,57 @@ static void perf_gtk__resize_window(GtkWidget *window) · 26 unmodified lines
3536 gtk_window_resize(GTK_WINDOW(window), width, height);
3637 }
3738
39+static const char *perf_gtk__get_percent_color(double percent)
40+{
41+ if (percent >= MIN_RED)
42+ return "<span fgcolor='red'>";
43+ if (percent >= MIN_GREEN)
44+ return "<span fgcolor='dark green'>";
45+ return NULL;
46+}
47+
48+#define HPP__COLOR_FN(_name, _field) \
49+static int perf_gtk__hpp_color_ ## _name(struct perf_hpp *hpp, \
50+ struct hist_entry *he) \
51+{ \
52+ double percent = 100.0 * he->_field / hpp->total_period; \
53+ const char *markup; \
54+ int ret = 0; \
55+ \
56+ markup = perf_gtk__get_percent_color(percent); \
57+ if (markup) \
58+ ret += scnprintf(hpp->buf, hpp->size, "%s", markup); \
59+ ret += scnprintf(hpp->buf + ret, hpp->size - ret, "%6.2f%%", percent); \
60+ if (markup) \
61+ ret += scnprintf(hpp->buf + ret, hpp->size - ret, "</span>"); \
62+ \
63+ return ret; \
64+}
65+
66+HPP__COLOR_FN(overhead, period)
67+HPP__COLOR_FN(overhead_sys, period_sys)
68+HPP__COLOR_FN(overhead_us, period_us)
69+HPP__COLOR_FN(overhead_guest_sys, period_guest_sys)
70+HPP__COLOR_FN(overhead_guest_us, period_guest_us)
71+
72+#undef HPP__COLOR_FN
73+
74+void perf_gtk__init_hpp(void)
75+{
76+ perf_hpp__init(false, false);
77+
78+ perf_hpp__format[PERF_HPP__OVERHEAD].color =
79+ perf_gtk__hpp_color_overhead;
80+ perf_hpp__format[PERF_HPP__OVERHEAD_SYS].color =
81+ perf_gtk__hpp_color_overhead_sys;
82+ perf_hpp__format[PERF_HPP__OVERHEAD_US].color =
83+ perf_gtk__hpp_color_overhead_us;
84+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_SYS].color =
85+ perf_gtk__hpp_color_overhead_guest_sys;
86+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_US].color =
87+ perf_gtk__hpp_color_overhead_guest_us;
88+}
89+
3890 static void perf_gtk__show_hists(GtkWidget *window, struct hists *hists)
3991 {
4092 GType col_types[MAX_COLUMNS];
@@ -42,15 +94,25 @@ static void perf_gtk__show_hists(GtkWidget *window, struct hists *hists) · 1 unmodified line
4294 struct sort_entry *se;
4395 GtkListStore *store;
4496 struct rb_node *nd;
45− u64 total_period;
4697 GtkWidget *view;
47− int col_idx;
98+ int i, col_idx;
4899 int nr_cols;
100+ char s[512];
101+
102+ struct perf_hpp hpp = {
103+ .buf = s,
104+ .size = sizeof(s),
105+ .total_period = hists->stats.total_period,
106+ };
49107
50108 nr_cols = 0;
51109
52− /* The percentage column */
53− col_types[nr_cols++] = G_TYPE_STRING;
110+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
111+ if (!perf_hpp__format[i].cond)
112+ continue;
113+
114+ col_types[nr_cols++] = G_TYPE_STRING;
115+ }
54116
55117 list_for_each_entry(se, &hist_entry__sort_list, list) {
56118 if (se->elide)
@@ -67,11 +129,17 @@ static void perf_gtk__show_hists(GtkWidget *window, struct hists *hists) · 10 unmodified lines
67129
68130 col_idx = 0;
69131
70− /* The percentage column */
71− gtk_tree_view_insert_column_with_attributes(GTK_TREE_VIEW(view),
72− -1, "Overhead (%)",
73− renderer, "text",
74− col_idx++, NULL);
132+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
133+ if (!perf_hpp__format[i].cond)
134+ continue;
135+
136+ perf_hpp__format[i].header(&hpp);
137+
138+ gtk_tree_view_insert_column_with_attributes(GTK_TREE_VIEW(view),
139+ -1, s,
140+ renderer, "markup",
141+ col_idx++, NULL);
142+ }
75143
76144 list_for_each_entry(se, &hist_entry__sort_list, list) {
77145 if (se->elide)
@@ -87,13 +155,9 @@ static void perf_gtk__show_hists(GtkWidget *window, struct hists *hists) · 9 unmodified lines
87155
88156 g_object_unref(GTK_TREE_MODEL(store));
89157
90− total_period = hists->stats.total_period;
91−
92158 for (nd = rb_first(&hists->entries); nd; nd = rb_next(nd)) {
93159 struct hist_entry *h = rb_entry(nd, struct hist_entry, rb_node);
94160 GtkTreeIter iter;
95− double percent;
96− char s[512];
97161
98162 if (h->filtered)
99163 continue;
@@ -102,11 +166,17 @@ static void perf_gtk__show_hists(GtkWidget *window, struct hists *hists) · 2 unmodified lines
102166
103167 col_idx = 0;
104168
105− percent = (h->period * 100.0) / total_period;
169+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
170+ if (!perf_hpp__format[i].cond)
171+ continue;
106172
107− snprintf(s, ARRAY_SIZE(s), "%.2f", percent);
173+ if (perf_hpp__format[i].color)
174+ perf_hpp__format[i].color(&hpp, h);
175+ else
176+ perf_hpp__format[i].entry(&hpp, h);
108177
109− gtk_list_store_set(store, &iter, col_idx++, s, -1);
178+ gtk_list_store_set(store, &iter, col_idx++, s, -1);
179+ }
110180
111181 list_for_each_entry(se, &hist_entry__sort_list, list) {
112182 if (se->elide)
@@ -166,9 +236,10 @@ static GtkWidget *perf_gtk__setup_statusbar(void) · 53 unmodified lines
166236 }
167237
168238 int perf_evlist__gtk_browse_hists(struct perf_evlist *evlist,
169− const char *help __used,
170− void (*timer) (void *arg)__used,
171− void *arg __used, int delay_secs __used)
239+ const char *help,
240+ void (*timer) (void *arg)__maybe_unused,
241+ void *arg __maybe_unused,
242+ int delay_secs __maybe_unused)
172243 {
173244 struct perf_evsel *pos;
174245 GtkWidget *vbox;
@@ -233,6 +304,8 @@ int perf_evlist__gtk_browse_hists(struct perf_evlist *evlist, · 58 unmodified lines
233304
234305 gtk_window_set_position(GTK_WINDOW(window), GTK_WIN_POS_CENTER);
235306
307+ ui_helpline__push(help);
308+
236309 gtk_main();
237310
238311 perf_gtk__deactivate_context(&pgctx);
tools/perf/ui/gtk/gtk.h+3−0
@@ -29,6 +29,9 @@ static inline bool perf_gtk__is_active_context(struct perf_gtk_context *ctx) · 28 unmodified lines
2929 struct perf_gtk_context *perf_gtk__activate_context(GtkWidget *window);
3030 int perf_gtk__deactivate_context(struct perf_gtk_context **ctx);
3131
32+void perf_gtk__init_helpline(void);
33+void perf_gtk__init_hpp(void);
34+
3235 #ifndef HAVE_GTK_INFO_BAR
3336 static inline GtkWidget *perf_gtk__setup_info_bar(void)
3437 {
tools/perf/ui/gtk/helpline.c+56−0
@@ -0,0 +1,56 @@
1+#include <stdio.h>
2+#include <string.h>
3+
4+#include "gtk.h"
5+#include "../ui.h"
6+#include "../helpline.h"
7+#include "../../util/debug.h"
8+
9+static void gtk_helpline_pop(void)
10+{
11+ if (!perf_gtk__is_active_context(pgctx))
12+ return;
13+
14+ gtk_statusbar_pop(GTK_STATUSBAR(pgctx->statbar),
15+ pgctx->statbar_ctx_id);
16+}
17+
18+static void gtk_helpline_push(const char *msg)
19+{
20+ if (!perf_gtk__is_active_context(pgctx))
21+ return;
22+
23+ gtk_statusbar_push(GTK_STATUSBAR(pgctx->statbar),
24+ pgctx->statbar_ctx_id, msg);
25+}
26+
27+static struct ui_helpline gtk_helpline_fns = {
28+ .pop = gtk_helpline_pop,
29+ .push = gtk_helpline_push,
30+};
31+
32+void perf_gtk__init_helpline(void)
33+{
34+ helpline_fns = &gtk_helpline_fns;
35+}
36+
37+int perf_gtk__show_helpline(const char *fmt, va_list ap)
38+{
39+ int ret;
40+ char *ptr;
41+ static int backlog;
42+
43+ ret = vscnprintf(ui_helpline__current + backlog,
44+ sizeof(ui_helpline__current) - backlog, fmt, ap);
45+ backlog += ret;
46+
47+ /* only first line can be displayed */
48+ ptr = strchr(ui_helpline__current, '\n');
49+ if (ptr && (ptr - ui_helpline__current) <= backlog) {
50+ *ptr = '\0';
51+ ui_helpline__puts(ui_helpline__current);
52+ backlog = 0;
53+ }
54+
55+ return ret;
56+}
tools/perf/ui/gtk/setup.c+5−1
@@ -7,11 +7,15 @@ extern struct perf_error_ops perf_gtk_eops; · 6 unmodified lines
77 int perf_gtk__init(void)
88 {
99 perf_error__register(&perf_gtk_eops);
10+ perf_gtk__init_helpline();
11+ perf_gtk__init_hpp();
1012 return gtk_init_check(NULL, NULL) ? 0 : -1;
1113 }
1214
13−void perf_gtk__exit(bool wait_for_ok __used)
15+void perf_gtk__exit(bool wait_for_ok __maybe_unused)
1416 {
17+ if (!perf_gtk__is_active_context(pgctx))
18+ return;
1519 perf_error__unregister(&perf_gtk_eops);
1620 gtk_main_quit();
1721 }
tools/perf/ui/gtk/util.c+2−7
@@ -117,13 +117,8 @@ struct perf_error_ops perf_gtk_eops = { · 116 unmodified lines
117117 * For now, just add stubs for NO_NEWT=1 build.
118118 */
119119 #ifdef NO_NEWT_SUPPORT
120−int ui_helpline__show_help(const char *format __used, va_list ap __used)
121−{
122− return 0;
123−}
124−
125−void ui_progress__update(u64 curr __used, u64 total __used,
126− const char *title __used)
120+void ui_progress__update(u64 curr __maybe_unused, u64 total __maybe_unused,
121+ const char *title __maybe_unused)
127122 {
128123 }
129124 #endif
tools/perf/ui/helpline.c+19−37
@@ -5,23 +5,32 @@ · 4 unmodified lines
55 #include "../debug.h"
66 #include "helpline.h"
77 #include "ui.h"
8−#include "libslang.h"
98
10−void ui_helpline__pop(void)
9+char ui_helpline__current[512];
10+
11+static void nop_helpline__pop(void)
1112 {
1213 }
1314
14−char ui_helpline__current[512];
15+static void nop_helpline__push(const char *msg __maybe_unused)
16+{
17+}
1518
16−void ui_helpline__push(const char *msg)
19+static struct ui_helpline default_helpline_fns = {
20+ .pop = nop_helpline__pop,
21+ .push = nop_helpline__push,
22+};
23+
24+struct ui_helpline *helpline_fns = &default_helpline_fns;
25+
26+void ui_helpline__pop(void)
1727 {
18− const size_t sz = sizeof(ui_helpline__current);
28+ helpline_fns->pop();
29+}
1930
20− SLsmg_gotorc(SLtt_Screen_Rows - 1, 0);
21− SLsmg_set_color(0);
22− SLsmg_write_nstring((char *)msg, SLtt_Screen_Cols);
23− SLsmg_refresh();
24− strncpy(ui_helpline__current, msg, sz)[sz - 1] = '\0';
31+void ui_helpline__push(const char *msg)
32+{
33+ helpline_fns->push(msg);
2534 }
2635
2736 void ui_helpline__vpush(const char *fmt, va_list ap)
@@ -50,30 +59,3 @@ void ui_helpline__puts(const char *msg) · 22 unmodified lines
5059 ui_helpline__pop();
5160 ui_helpline__push(msg);
5261 }
53−
54−void ui_helpline__init(void)
55−{
56− ui_helpline__puts(" ");
57−}
58−
59−char ui_helpline__last_msg[1024];
60−
61−int ui_helpline__show_help(const char *format, va_list ap)
62−{
63− int ret;
64− static int backlog;
65−
66− pthread_mutex_lock(&ui__lock);
67− ret = vscnprintf(ui_helpline__last_msg + backlog,
68− sizeof(ui_helpline__last_msg) - backlog, format, ap);
69− backlog += ret;
70−
71− if (ui_helpline__last_msg[backlog - 1] == '\n') {
72− ui_helpline__puts(ui_helpline__last_msg);
73− SLsmg_refresh();
74− backlog = 0;
75− }
76− pthread_mutex_unlock(&ui__lock);
77−
78− return ret;
79−}
tools/perf/ui/helpline.h+32−1
@@ -4,13 +4,44 @@ · 3 unmodified lines
44 #include <stdio.h>
55 #include <stdarg.h>
66
7+#include "../util/cache.h"
8+
9+struct ui_helpline {
10+ void (*pop)(void);
11+ void (*push)(const char *msg);
12+};
13+
14+extern struct ui_helpline *helpline_fns;
15+
716 void ui_helpline__init(void);
17+
818 void ui_helpline__pop(void);
919 void ui_helpline__push(const char *msg);
1020 void ui_helpline__vpush(const char *fmt, va_list ap);
1121 void ui_helpline__fpush(const char *fmt, ...);
1222 void ui_helpline__puts(const char *msg);
1323
14−extern char ui_helpline__current[];
24+extern char ui_helpline__current[512];
25+
26+#ifdef NO_NEWT_SUPPORT
27+static inline int ui_helpline__show_help(const char *format __maybe_unused,
28+ va_list ap __maybe_unused)
29+{
30+ return 0;
31+}
32+#else
33+extern char ui_helpline__last_msg[];
34+int ui_helpline__show_help(const char *format, va_list ap);
35+#endif /* NO_NEWT_SUPPORT */
36+
37+#ifdef NO_GTK2_SUPPORT
38+static inline int perf_gtk__show_helpline(const char *format __maybe_unused,
39+ va_list ap __maybe_unused)
40+{
41+ return 0;
42+}
43+#else
44+int perf_gtk__show_helpline(const char *format, va_list ap);
45+#endif /* NO_GTK2_SUPPORT */
1546
1647 #endif /* _PERF_UI_HELPLINE_H_ */
tools/perf/ui/hist.c+390−0
@@ -0,0 +1,390 @@
1+#include <math.h>
2+
3+#include "../util/hist.h"
4+#include "../util/util.h"
5+#include "../util/sort.h"
6+
7+
8+/* hist period print (hpp) functions */
9+static int hpp__header_overhead(struct perf_hpp *hpp)
10+{
11+ const char *fmt = hpp->ptr ? "Baseline" : "Overhead";
12+
13+ return scnprintf(hpp->buf, hpp->size, fmt);
14+}
15+
16+static int hpp__width_overhead(struct perf_hpp *hpp __maybe_unused)
17+{
18+ return 8;
19+}
20+
21+static int hpp__color_overhead(struct perf_hpp *hpp, struct hist_entry *he)
22+{
23+ double percent = 100.0 * he->period / hpp->total_period;
24+
25+ if (hpp->ptr) {
26+ struct hists *old_hists = hpp->ptr;
27+ u64 total_period = old_hists->stats.total_period;
28+ u64 base_period = he->pair ? he->pair->period : 0;
29+
30+ if (total_period)
31+ percent = 100.0 * base_period / total_period;
32+ else
33+ percent = 0.0;
34+ }
35+
36+ return percent_color_snprintf(hpp->buf, hpp->size, " %6.2f%%", percent);
37+}
38+
39+static int hpp__entry_overhead(struct perf_hpp *hpp, struct hist_entry *he)
40+{
41+ double percent = 100.0 * he->period / hpp->total_period;
42+ const char *fmt = symbol_conf.field_sep ? "%.2f" : " %6.2f%%";
43+
44+ if (hpp->ptr) {
45+ struct hists *old_hists = hpp->ptr;
46+ u64 total_period = old_hists->stats.total_period;
47+ u64 base_period = he->pair ? he->pair->period : 0;
48+
49+ if (total_period)
50+ percent = 100.0 * base_period / total_period;
51+ else
52+ percent = 0.0;
53+ }
54+
55+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
56+}
57+
58+static int hpp__header_overhead_sys(struct perf_hpp *hpp)
59+{
60+ const char *fmt = symbol_conf.field_sep ? "%s" : "%7s";
61+
62+ return scnprintf(hpp->buf, hpp->size, fmt, "sys");
63+}
64+
65+static int hpp__width_overhead_sys(struct perf_hpp *hpp __maybe_unused)
66+{
67+ return 7;
68+}
69+
70+static int hpp__color_overhead_sys(struct perf_hpp *hpp, struct hist_entry *he)
71+{
72+ double percent = 100.0 * he->period_sys / hpp->total_period;
73+ return percent_color_snprintf(hpp->buf, hpp->size, "%6.2f%%", percent);
74+}
75+
76+static int hpp__entry_overhead_sys(struct perf_hpp *hpp, struct hist_entry *he)
77+{
78+ double percent = 100.0 * he->period_sys / hpp->total_period;
79+ const char *fmt = symbol_conf.field_sep ? "%.2f" : "%6.2f%%";
80+
81+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
82+}
83+
84+static int hpp__header_overhead_us(struct perf_hpp *hpp)
85+{
86+ const char *fmt = symbol_conf.field_sep ? "%s" : "%7s";
87+
88+ return scnprintf(hpp->buf, hpp->size, fmt, "user");
89+}
90+
91+static int hpp__width_overhead_us(struct perf_hpp *hpp __maybe_unused)
92+{
93+ return 7;
94+}
95+
96+static int hpp__color_overhead_us(struct perf_hpp *hpp, struct hist_entry *he)
97+{
98+ double percent = 100.0 * he->period_us / hpp->total_period;
99+ return percent_color_snprintf(hpp->buf, hpp->size, "%6.2f%%", percent);
100+}
101+
102+static int hpp__entry_overhead_us(struct perf_hpp *hpp, struct hist_entry *he)
103+{
104+ double percent = 100.0 * he->period_us / hpp->total_period;
105+ const char *fmt = symbol_conf.field_sep ? "%.2f" : "%6.2f%%";
106+
107+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
108+}
109+
110+static int hpp__header_overhead_guest_sys(struct perf_hpp *hpp)
111+{
112+ return scnprintf(hpp->buf, hpp->size, "guest sys");
113+}
114+
115+static int hpp__width_overhead_guest_sys(struct perf_hpp *hpp __maybe_unused)
116+{
117+ return 9;
118+}
119+
120+static int hpp__color_overhead_guest_sys(struct perf_hpp *hpp,
121+ struct hist_entry *he)
122+{
123+ double percent = 100.0 * he->period_guest_sys / hpp->total_period;
124+ return percent_color_snprintf(hpp->buf, hpp->size, " %6.2f%% ", percent);
125+}
126+
127+static int hpp__entry_overhead_guest_sys(struct perf_hpp *hpp,
128+ struct hist_entry *he)
129+{
130+ double percent = 100.0 * he->period_guest_sys / hpp->total_period;
131+ const char *fmt = symbol_conf.field_sep ? "%.2f" : " %6.2f%% ";
132+
133+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
134+}
135+
136+static int hpp__header_overhead_guest_us(struct perf_hpp *hpp)
137+{
138+ return scnprintf(hpp->buf, hpp->size, "guest usr");
139+}
140+
141+static int hpp__width_overhead_guest_us(struct perf_hpp *hpp __maybe_unused)
142+{
143+ return 9;
144+}
145+
146+static int hpp__color_overhead_guest_us(struct perf_hpp *hpp,
147+ struct hist_entry *he)
148+{
149+ double percent = 100.0 * he->period_guest_us / hpp->total_period;
150+ return percent_color_snprintf(hpp->buf, hpp->size, " %6.2f%% ", percent);
151+}
152+
153+static int hpp__entry_overhead_guest_us(struct perf_hpp *hpp,
154+ struct hist_entry *he)
155+{
156+ double percent = 100.0 * he->period_guest_us / hpp->total_period;
157+ const char *fmt = symbol_conf.field_sep ? "%.2f" : " %6.2f%% ";
158+
159+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
160+}
161+
162+static int hpp__header_samples(struct perf_hpp *hpp)
163+{
164+ const char *fmt = symbol_conf.field_sep ? "%s" : "%11s";
165+
166+ return scnprintf(hpp->buf, hpp->size, fmt, "Samples");
167+}
168+
169+static int hpp__width_samples(struct perf_hpp *hpp __maybe_unused)
170+{
171+ return 11;
172+}
173+
174+static int hpp__entry_samples(struct perf_hpp *hpp, struct hist_entry *he)
175+{
176+ const char *fmt = symbol_conf.field_sep ? "%" PRIu64 : "%11" PRIu64;
177+
178+ return scnprintf(hpp->buf, hpp->size, fmt, he->nr_events);
179+}
180+
181+static int hpp__header_period(struct perf_hpp *hpp)
182+{
183+ const char *fmt = symbol_conf.field_sep ? "%s" : "%12s";
184+
185+ return scnprintf(hpp->buf, hpp->size, fmt, "Period");
186+}
187+
188+static int hpp__width_period(struct perf_hpp *hpp __maybe_unused)
189+{
190+ return 12;
191+}
192+
193+static int hpp__entry_period(struct perf_hpp *hpp, struct hist_entry *he)
194+{
195+ const char *fmt = symbol_conf.field_sep ? "%" PRIu64 : "%12" PRIu64;
196+
197+ return scnprintf(hpp->buf, hpp->size, fmt, he->period);
198+}
199+
200+static int hpp__header_delta(struct perf_hpp *hpp)
201+{
202+ const char *fmt = symbol_conf.field_sep ? "%s" : "%7s";
203+
204+ return scnprintf(hpp->buf, hpp->size, fmt, "Delta");
205+}
206+
207+static int hpp__width_delta(struct perf_hpp *hpp __maybe_unused)
208+{
209+ return 7;
210+}
211+
212+static int hpp__entry_delta(struct perf_hpp *hpp, struct hist_entry *he)
213+{
214+ struct hists *pair_hists = hpp->ptr;
215+ u64 old_total, new_total;
216+ double old_percent = 0, new_percent = 0;
217+ double diff;
218+ const char *fmt = symbol_conf.field_sep ? "%s" : "%7.7s";
219+ char buf[32] = " ";
220+
221+ old_total = pair_hists->stats.total_period;
222+ if (old_total > 0 && he->pair)
223+ old_percent = 100.0 * he->pair->period / old_total;
224+
225+ new_total = hpp->total_period;
226+ if (new_total > 0)
227+ new_percent = 100.0 * he->period / new_total;
228+
229+ diff = new_percent - old_percent;
230+ if (fabs(diff) >= 0.01)
231+ scnprintf(buf, sizeof(buf), "%+4.2F%%", diff);
232+
233+ return scnprintf(hpp->buf, hpp->size, fmt, buf);
234+}
235+
236+static int hpp__header_displ(struct perf_hpp *hpp)
237+{
238+ return scnprintf(hpp->buf, hpp->size, "Displ.");
239+}
240+
241+static int hpp__width_displ(struct perf_hpp *hpp __maybe_unused)
242+{
243+ return 6;
244+}
245+
246+static int hpp__entry_displ(struct perf_hpp *hpp,
247+ struct hist_entry *he __maybe_unused)
248+{
249+ const char *fmt = symbol_conf.field_sep ? "%s" : "%6.6s";
250+ char buf[32] = " ";
251+
252+ if (hpp->displacement)
253+ scnprintf(buf, sizeof(buf), "%+4ld", hpp->displacement);
254+
255+ return scnprintf(hpp->buf, hpp->size, fmt, buf);
256+}
257+
258+#define HPP__COLOR_PRINT_FNS(_name) \
259+ .header = hpp__header_ ## _name, \
260+ .width = hpp__width_ ## _name, \
261+ .color = hpp__color_ ## _name, \
262+ .entry = hpp__entry_ ## _name
263+
264+#define HPP__PRINT_FNS(_name) \
265+ .header = hpp__header_ ## _name, \
266+ .width = hpp__width_ ## _name, \
267+ .entry = hpp__entry_ ## _name
268+
269+struct perf_hpp_fmt perf_hpp__format[] = {
270+ { .cond = true, HPP__COLOR_PRINT_FNS(overhead) },
271+ { .cond = false, HPP__COLOR_PRINT_FNS(overhead_sys) },
272+ { .cond = false, HPP__COLOR_PRINT_FNS(overhead_us) },
273+ { .cond = false, HPP__COLOR_PRINT_FNS(overhead_guest_sys) },
274+ { .cond = false, HPP__COLOR_PRINT_FNS(overhead_guest_us) },
275+ { .cond = false, HPP__PRINT_FNS(samples) },
276+ { .cond = false, HPP__PRINT_FNS(period) },
277+ { .cond = false, HPP__PRINT_FNS(delta) },
278+ { .cond = false, HPP__PRINT_FNS(displ) }
279+};
280+
281+#undef HPP__COLOR_PRINT_FNS
282+#undef HPP__PRINT_FNS
283+
284+void perf_hpp__init(bool need_pair, bool show_displacement)
285+{
286+ if (symbol_conf.show_cpu_utilization) {
287+ perf_hpp__format[PERF_HPP__OVERHEAD_SYS].cond = true;
288+ perf_hpp__format[PERF_HPP__OVERHEAD_US].cond = true;
289+
290+ if (perf_guest) {
291+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_SYS].cond = true;
292+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_US].cond = true;
293+ }
294+ }
295+
296+ if (symbol_conf.show_nr_samples)
297+ perf_hpp__format[PERF_HPP__SAMPLES].cond = true;
298+
299+ if (symbol_conf.show_total_period)
300+ perf_hpp__format[PERF_HPP__PERIOD].cond = true;
301+
302+ if (need_pair) {
303+ perf_hpp__format[PERF_HPP__DELTA].cond = true;
304+
305+ if (show_displacement)
306+ perf_hpp__format[PERF_HPP__DISPL].cond = true;
307+ }
308+}
309+
310+static inline void advance_hpp(struct perf_hpp *hpp, int inc)
311+{
312+ hpp->buf += inc;
313+ hpp->size -= inc;
314+}
315+
316+int hist_entry__period_snprintf(struct perf_hpp *hpp, struct hist_entry *he,
317+ bool color)
318+{
319+ const char *sep = symbol_conf.field_sep;
320+ char *start = hpp->buf;
321+ int i, ret;
322+
323+ if (symbol_conf.exclude_other && !he->parent)
324+ return 0;
325+
326+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
327+ if (!perf_hpp__format[i].cond)
328+ continue;
329+
330+ if (!sep || i > 0) {
331+ ret = scnprintf(hpp->buf, hpp->size, "%s", sep ?: " ");
332+ advance_hpp(hpp, ret);
333+ }
334+
335+ if (color && perf_hpp__format[i].color)
336+ ret = perf_hpp__format[i].color(hpp, he);
337+ else
338+ ret = perf_hpp__format[i].entry(hpp, he);
339+
340+ advance_hpp(hpp, ret);
341+ }
342+
343+ return hpp->buf - start;
344+}
345+
346+int hist_entry__sort_snprintf(struct hist_entry *he, char *s, size_t size,
347+ struct hists *hists)
348+{
349+ const char *sep = symbol_conf.field_sep;
350+ struct sort_entry *se;
351+ int ret = 0;
352+
353+ list_for_each_entry(se, &hist_entry__sort_list, list) {
354+ if (se->elide)
355+ continue;
356+
357+ ret += scnprintf(s + ret, size - ret, "%s", sep ?: " ");
358+ ret += se->se_snprintf(he, s + ret, size - ret,
359+ hists__col_len(hists, se->se_width_idx));
360+ }
361+
362+ return ret;
363+}
364+
365+/*
366+ * See hists__fprintf to match the column widths
367+ */
368+unsigned int hists__sort_list_width(struct hists *hists)
369+{
370+ struct sort_entry *se;
371+ int i, ret = 0;
372+
373+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
374+ if (!perf_hpp__format[i].cond)
375+ continue;
376+ if (i)
377+ ret += 2;
378+
379+ ret += perf_hpp__format[i].width(NULL);
380+ }
381+
382+ list_for_each_entry(se, &hist_entry__sort_list, list)
383+ if (!se->elide)
384+ ret += 2 + hists__col_len(hists, se->se_width_idx);
385+
386+ if (verbose) /* Addr + origin */
387+ ret += 3 + BITS_PER_LONG / 4;
388+
389+ return ret;
390+}
tools/perf/ui/setup.c+8−2
@@ -1,6 +1,10 @@
1−#include "../cache.h"
2−#include "../debug.h"
1+#include <pthread.h>
32
3+#include "../util/cache.h"
4+#include "../util/debug.h"
5+#include "../util/hist.h"
6+
7+pthread_mutex_t ui__lock = PTHREAD_MUTEX_INITIALIZER;
48
59 void setup_browser(bool fallback_to_pager)
610 {
@@ -25,6 +29,8 @@ void setup_browser(bool fallback_to_pager) · 18 unmodified lines
2529 use_browser = 0;
2630 if (fallback_to_pager)
2731 setup_pager();
32+
33+ perf_hpp__init(false, false);
2834 break;
2935 }
3036 }
tools/perf/ui/stdio/hist.c+498−0
@@ -0,0 +1,498 @@
1+#include <stdio.h>
2+
3+#include "../../util/util.h"
4+#include "../../util/hist.h"
5+#include "../../util/sort.h"
6+
7+
8+static size_t callchain__fprintf_left_margin(FILE *fp, int left_margin)
9+{
10+ int i;
11+ int ret = fprintf(fp, " ");
12+
13+ for (i = 0; i < left_margin; i++)
14+ ret += fprintf(fp, " ");
15+
16+ return ret;
17+}
18+
19+static size_t ipchain__fprintf_graph_line(FILE *fp, int depth, int depth_mask,
20+ int left_margin)
21+{
22+ int i;
23+ size_t ret = callchain__fprintf_left_margin(fp, left_margin);
24+
25+ for (i = 0; i < depth; i++)
26+ if (depth_mask & (1 << i))
27+ ret += fprintf(fp, "| ");
28+ else
29+ ret += fprintf(fp, " ");
30+
31+ ret += fprintf(fp, "\n");
32+
33+ return ret;
34+}
35+
36+static size_t ipchain__fprintf_graph(FILE *fp, struct callchain_list *chain,
37+ int depth, int depth_mask, int period,
38+ u64 total_samples, u64 hits,
39+ int left_margin)
40+{
41+ int i;
42+ size_t ret = 0;
43+
44+ ret += callchain__fprintf_left_margin(fp, left_margin);
45+ for (i = 0; i < depth; i++) {
46+ if (depth_mask & (1 << i))
47+ ret += fprintf(fp, "|");
48+ else
49+ ret += fprintf(fp, " ");
50+ if (!period && i == depth - 1) {
51+ double percent;
52+
53+ percent = hits * 100.0 / total_samples;
54+ ret += percent_color_fprintf(fp, "--%2.2f%%-- ", percent);
55+ } else
56+ ret += fprintf(fp, "%s", " ");
57+ }
58+ if (chain->ms.sym)
59+ ret += fprintf(fp, "%s\n", chain->ms.sym->name);
60+ else
61+ ret += fprintf(fp, "0x%0" PRIx64 "\n", chain->ip);
62+
63+ return ret;
64+}
65+
66+static struct symbol *rem_sq_bracket;
67+static struct callchain_list rem_hits;
68+
69+static void init_rem_hits(void)
70+{
71+ rem_sq_bracket = malloc(sizeof(*rem_sq_bracket) + 6);
72+ if (!rem_sq_bracket) {
73+ fprintf(stderr, "Not enough memory to display remaining hits\n");
74+ return;
75+ }
76+
77+ strcpy(rem_sq_bracket->name, "[...]");
78+ rem_hits.ms.sym = rem_sq_bracket;
79+}
80+
81+static size_t __callchain__fprintf_graph(FILE *fp, struct rb_root *root,
82+ u64 total_samples, int depth,
83+ int depth_mask, int left_margin)
84+{
85+ struct rb_node *node, *next;
86+ struct callchain_node *child;
87+ struct callchain_list *chain;
88+ int new_depth_mask = depth_mask;
89+ u64 remaining;
90+ size_t ret = 0;
91+ int i;
92+ uint entries_printed = 0;
93+
94+ remaining = total_samples;
95+
96+ node = rb_first(root);
97+ while (node) {
98+ u64 new_total;
99+ u64 cumul;
100+
101+ child = rb_entry(node, struct callchain_node, rb_node);
102+ cumul = callchain_cumul_hits(child);
103+ remaining -= cumul;
104+
105+ /*
106+ * The depth mask manages the output of pipes that show
107+ * the depth. We don't want to keep the pipes of the current
108+ * level for the last child of this depth.
109+ * Except if we have remaining filtered hits. They will
110+ * supersede the last child
111+ */
112+ next = rb_next(node);
113+ if (!next && (callchain_param.mode != CHAIN_GRAPH_REL || !remaining))
114+ new_depth_mask &= ~(1 << (depth - 1));
115+
116+ /*
117+ * But we keep the older depth mask for the line separator
118+ * to keep the level link until we reach the last child
119+ */
120+ ret += ipchain__fprintf_graph_line(fp, depth, depth_mask,
121+ left_margin);
122+ i = 0;
123+ list_for_each_entry(chain, &child->val, list) {
124+ ret += ipchain__fprintf_graph(fp, chain, depth,
125+ new_depth_mask, i++,
126+ total_samples,
127+ cumul,
128+ left_margin);
129+ }
130+
131+ if (callchain_param.mode == CHAIN_GRAPH_REL)
132+ new_total = child->children_hit;
133+ else
134+ new_total = total_samples;
135+
136+ ret += __callchain__fprintf_graph(fp, &child->rb_root, new_total,
137+ depth + 1,
138+ new_depth_mask | (1 << depth),
139+ left_margin);
140+ node = next;
141+ if (++entries_printed == callchain_param.print_limit)
142+ break;
143+ }
144+
145+ if (callchain_param.mode == CHAIN_GRAPH_REL &&
146+ remaining && remaining != total_samples) {
147+
148+ if (!rem_sq_bracket)
149+ return ret;
150+
151+ new_depth_mask &= ~(1 << (depth - 1));
152+ ret += ipchain__fprintf_graph(fp, &rem_hits, depth,
153+ new_depth_mask, 0, total_samples,
154+ remaining, left_margin);
155+ }
156+
157+ return ret;
158+}
159+
160+static size_t callchain__fprintf_graph(FILE *fp, struct rb_root *root,
161+ u64 total_samples, int left_margin)
162+{
163+ struct callchain_node *cnode;
164+ struct callchain_list *chain;
165+ u32 entries_printed = 0;
166+ bool printed = false;
167+ struct rb_node *node;
168+ int i = 0;
169+ int ret = 0;
170+
171+ /*
172+ * If have one single callchain root, don't bother printing
173+ * its percentage (100 % in fractal mode and the same percentage
174+ * than the hist in graph mode). This also avoid one level of column.
175+ */
176+ node = rb_first(root);
177+ if (node && !rb_next(node)) {
178+ cnode = rb_entry(node, struct callchain_node, rb_node);
179+ list_for_each_entry(chain, &cnode->val, list) {
180+ /*
181+ * If we sort by symbol, the first entry is the same than
182+ * the symbol. No need to print it otherwise it appears as
183+ * displayed twice.
184+ */
185+ if (!i++ && sort__first_dimension == SORT_SYM)
186+ continue;
187+ if (!printed) {
188+ ret += callchain__fprintf_left_margin(fp, left_margin);
189+ ret += fprintf(fp, "|\n");
190+ ret += callchain__fprintf_left_margin(fp, left_margin);
191+ ret += fprintf(fp, "---");
192+ left_margin += 3;
193+ printed = true;
194+ } else
195+ ret += callchain__fprintf_left_margin(fp, left_margin);
196+
197+ if (chain->ms.sym)
198+ ret += fprintf(fp, " %s\n", chain->ms.sym->name);
199+ else
200+ ret += fprintf(fp, " %p\n", (void *)(long)chain->ip);
201+
202+ if (++entries_printed == callchain_param.print_limit)
203+ break;
204+ }
205+ root = &cnode->rb_root;
206+ }
207+
208+ ret += __callchain__fprintf_graph(fp, root, total_samples,
209+ 1, 1, left_margin);
210+ ret += fprintf(fp, "\n");
211+
212+ return ret;
213+}
214+
215+static size_t __callchain__fprintf_flat(FILE *fp,
216+ struct callchain_node *self,
217+ u64 total_samples)
218+{
219+ struct callchain_list *chain;
220+ size_t ret = 0;
221+
222+ if (!self)
223+ return 0;
224+
225+ ret += __callchain__fprintf_flat(fp, self->parent, total_samples);
226+
227+
228+ list_for_each_entry(chain, &self->val, list) {
229+ if (chain->ip >= PERF_CONTEXT_MAX)
230+ continue;
231+ if (chain->ms.sym)
232+ ret += fprintf(fp, " %s\n", chain->ms.sym->name);
233+ else
234+ ret += fprintf(fp, " %p\n",
235+ (void *)(long)chain->ip);
236+ }
237+
238+ return ret;
239+}
240+
241+static size_t callchain__fprintf_flat(FILE *fp, struct rb_root *self,
242+ u64 total_samples)
243+{
244+ size_t ret = 0;
245+ u32 entries_printed = 0;
246+ struct rb_node *rb_node;
247+ struct callchain_node *chain;
248+
249+ rb_node = rb_first(self);
250+ while (rb_node) {
251+ double percent;
252+
253+ chain = rb_entry(rb_node, struct callchain_node, rb_node);
254+ percent = chain->hit * 100.0 / total_samples;
255+
256+ ret = percent_color_fprintf(fp, " %6.2f%%\n", percent);
257+ ret += __callchain__fprintf_flat(fp, chain, total_samples);
258+ ret += fprintf(fp, "\n");
259+ if (++entries_printed == callchain_param.print_limit)
260+ break;
261+
262+ rb_node = rb_next(rb_node);
263+ }
264+
265+ return ret;
266+}
267+
268+static size_t hist_entry_callchain__fprintf(struct hist_entry *he,
269+ u64 total_samples, int left_margin,
270+ FILE *fp)
271+{
272+ switch (callchain_param.mode) {
273+ case CHAIN_GRAPH_REL:
274+ return callchain__fprintf_graph(fp, &he->sorted_chain, he->period,
275+ left_margin);
276+ break;
277+ case CHAIN_GRAPH_ABS:
278+ return callchain__fprintf_graph(fp, &he->sorted_chain, total_samples,
279+ left_margin);
280+ break;
281+ case CHAIN_FLAT:
282+ return callchain__fprintf_flat(fp, &he->sorted_chain, total_samples);
283+ break;
284+ case CHAIN_NONE:
285+ break;
286+ default:
287+ pr_err("Bad callchain mode\n");
288+ }
289+
290+ return 0;
291+}
292+
293+static size_t hist_entry__callchain_fprintf(struct hist_entry *he,
294+ struct hists *hists,
295+ u64 total_period, FILE *fp)
296+{
297+ int left_margin = 0;
298+
299+ if (sort__first_dimension == SORT_COMM) {
300+ struct sort_entry *se = list_first_entry(&hist_entry__sort_list,
301+ typeof(*se), list);
302+ left_margin = hists__col_len(hists, se->se_width_idx);
303+ left_margin -= thread__comm_len(he->thread);
304+ }
305+
306+ return hist_entry_callchain__fprintf(he, total_period, left_margin, fp);
307+}
308+
309+static int hist_entry__fprintf(struct hist_entry *he, size_t size,
310+ struct hists *hists, struct hists *pair_hists,
311+ long displacement, u64 total_period, FILE *fp)
312+{
313+ char bf[512];
314+ int ret;
315+ struct perf_hpp hpp = {
316+ .buf = bf,
317+ .size = size,
318+ .total_period = total_period,
319+ .displacement = displacement,
320+ .ptr = pair_hists,
321+ };
322+ bool color = !symbol_conf.field_sep;
323+
324+ if (size == 0 || size > sizeof(bf))
325+ size = hpp.size = sizeof(bf);
326+
327+ ret = hist_entry__period_snprintf(&hpp, he, color);
328+ hist_entry__sort_snprintf(he, bf + ret, size - ret, hists);
329+
330+ ret = fprintf(fp, "%s\n", bf);
331+
332+ if (symbol_conf.use_callchain)
333+ ret += hist_entry__callchain_fprintf(he, hists,
334+ total_period, fp);
335+
336+ return ret;
337+}
338+
339+size_t hists__fprintf(struct hists *hists, struct hists *pair,
340+ bool show_displacement, bool show_header, int max_rows,
341+ int max_cols, FILE *fp)
342+{
343+ struct sort_entry *se;
344+ struct rb_node *nd;
345+ size_t ret = 0;
346+ u64 total_period;
347+ unsigned long position = 1;
348+ long displacement = 0;
349+ unsigned int width;
350+ const char *sep = symbol_conf.field_sep;
351+ const char *col_width = symbol_conf.col_width_list_str;
352+ int idx, nr_rows = 0;
353+ char bf[64];
354+ struct perf_hpp dummy_hpp = {
355+ .buf = bf,
356+ .size = sizeof(bf),
357+ .ptr = pair,
358+ };
359+
360+ init_rem_hits();
361+
362+ if (!show_header)
363+ goto print_entries;
364+
365+ fprintf(fp, "# ");
366+ for (idx = 0; idx < PERF_HPP__MAX_INDEX; idx++) {
367+ if (!perf_hpp__format[idx].cond)
368+ continue;
369+
370+ if (idx)
371+ fprintf(fp, "%s", sep ?: " ");
372+
373+ perf_hpp__format[idx].header(&dummy_hpp);
374+ fprintf(fp, "%s", bf);
375+ }
376+
377+ list_for_each_entry(se, &hist_entry__sort_list, list) {
378+ if (se->elide)
379+ continue;
380+ if (sep) {
381+ fprintf(fp, "%c%s", *sep, se->se_header);
382+ continue;
383+ }
384+ width = strlen(se->se_header);
385+ if (symbol_conf.col_width_list_str) {
386+ if (col_width) {
387+ hists__set_col_len(hists, se->se_width_idx,
388+ atoi(col_width));
389+ col_width = strchr(col_width, ',');
390+ if (col_width)
391+ ++col_width;
392+ }
393+ }
394+ if (!hists__new_col_len(hists, se->se_width_idx, width))
395+ width = hists__col_len(hists, se->se_width_idx);
396+ fprintf(fp, " %*s", width, se->se_header);
397+ }
398+
399+ fprintf(fp, "\n");
400+ if (max_rows && ++nr_rows >= max_rows)
401+ goto out;
402+
403+ if (sep)
404+ goto print_entries;
405+
406+ fprintf(fp, "# ");
407+ for (idx = 0; idx < PERF_HPP__MAX_INDEX; idx++) {
408+ unsigned int i;
409+
410+ if (!perf_hpp__format[idx].cond)
411+ continue;
412+
413+ if (idx)
414+ fprintf(fp, "%s", sep ?: " ");
415+
416+ width = perf_hpp__format[idx].width(&dummy_hpp);
417+ for (i = 0; i < width; i++)
418+ fprintf(fp, ".");
419+ }
420+
421+ list_for_each_entry(se, &hist_entry__sort_list, list) {
422+ unsigned int i;
423+
424+ if (se->elide)
425+ continue;
426+
427+ fprintf(fp, " ");
428+ width = hists__col_len(hists, se->se_width_idx);
429+ if (width == 0)
430+ width = strlen(se->se_header);
431+ for (i = 0; i < width; i++)
432+ fprintf(fp, ".");
433+ }
434+
435+ fprintf(fp, "\n");
436+ if (max_rows && ++nr_rows >= max_rows)
437+ goto out;
438+
439+ fprintf(fp, "#\n");
440+ if (max_rows && ++nr_rows >= max_rows)
441+ goto out;
442+
443+print_entries:
444+ total_period = hists->stats.total_period;
445+
446+ for (nd = rb_first(&hists->entries); nd; nd = rb_next(nd)) {
447+ struct hist_entry *h = rb_entry(nd, struct hist_entry, rb_node);
448+
449+ if (h->filtered)
450+ continue;
451+
452+ if (show_displacement) {
453+ if (h->pair != NULL)
454+ displacement = ((long)h->pair->position -
455+ (long)position);
456+ else
457+ displacement = 0;
458+ ++position;
459+ }
460+ ret += hist_entry__fprintf(h, max_cols, hists, pair, displacement,
461+ total_period, fp);
462+
463+ if (max_rows && ++nr_rows >= max_rows)
464+ goto out;
465+
466+ if (h->ms.map == NULL && verbose > 1) {
467+ __map_groups__fprintf_maps(&h->thread->mg,
468+ MAP__FUNCTION, verbose, fp);
469+ fprintf(fp, "%.10s end\n", graph_dotted_line);
470+ }
471+ }
472+out:
473+ free(rem_sq_bracket);
474+
475+ return ret;
476+}
477+
478+size_t hists__fprintf_nr_events(struct hists *hists, FILE *fp)
479+{
480+ int i;
481+ size_t ret = 0;
482+
483+ for (i = 0; i < PERF_RECORD_HEADER_MAX; ++i) {
484+ const char *name;
485+
486+ if (hists->stats.nr_events[i] == 0)
487+ continue;
488+
489+ name = perf_event__name(i);
490+ if (!strcmp(name, "UNKNOWN"))
491+ continue;
492+
493+ ret += fprintf(fp, "%16s events: %10d\n", name,
494+ hists->stats.nr_events[i]);
495+ }
496+
497+ return ret;
498+}
tools/perf/ui/tui/helpline.c+57−0
@@ -0,0 +1,57 @@
1+#include <stdio.h>
2+#include <stdlib.h>
3+#include <string.h>
4+#include <pthread.h>
5+
6+#include "../../util/debug.h"
7+#include "../helpline.h"
8+#include "../ui.h"
9+#include "../libslang.h"
10+
11+static void tui_helpline__pop(void)
12+{
13+}
14+
15+static void tui_helpline__push(const char *msg)
16+{
17+ const size_t sz = sizeof(ui_helpline__current);
18+
19+ SLsmg_gotorc(SLtt_Screen_Rows - 1, 0);
20+ SLsmg_set_color(0);
21+ SLsmg_write_nstring((char *)msg, SLtt_Screen_Cols);
22+ SLsmg_refresh();
23+ strncpy(ui_helpline__current, msg, sz)[sz - 1] = '\0';
24+}
25+
26+struct ui_helpline tui_helpline_fns = {
27+ .pop = tui_helpline__pop,
28+ .push = tui_helpline__push,
29+};
30+
31+void ui_helpline__init(void)
32+{
33+ helpline_fns = &tui_helpline_fns;
34+ ui_helpline__puts(" ");
35+}
36+
37+char ui_helpline__last_msg[1024];
38+
39+int ui_helpline__show_help(const char *format, va_list ap)
40+{
41+ int ret;
42+ static int backlog;
43+
44+ pthread_mutex_lock(&ui__lock);
45+ ret = vscnprintf(ui_helpline__last_msg + backlog,
46+ sizeof(ui_helpline__last_msg) - backlog, format, ap);
47+ backlog += ret;
48+
49+ if (ui_helpline__last_msg[backlog - 1] == '\n') {
50+ ui_helpline__puts(ui_helpline__last_msg);
51+ SLsmg_refresh();
52+ backlog = 0;
53+ }
54+ pthread_mutex_unlock(&ui__lock);
55+
56+ return ret;
57+}
tools/perf/ui/tui/setup.c+6−4
@@ -11,12 +11,12 @@ · 10 unmodified lines
1111 #include "../libslang.h"
1212 #include "../keysyms.h"
1313
14−pthread_mutex_t ui__lock = PTHREAD_MUTEX_INITIALIZER;
15−
1614 static volatile int ui__need_resize;
1715
1816 extern struct perf_error_ops perf_tui_eops;
1917
18+extern void hist_browser__init_hpp(void);
19+
2020 void ui__refresh_dimensions(bool force)
2121 {
2222 if (force || ui__need_resize) {
@@ -28,7 +28,7 @@ void ui__refresh_dimensions(bool force) · 5 unmodified lines
2828 }
2929 }
3030
31−static void ui__sigwinch(int sig __used)
31+static void ui__sigwinch(int sig __maybe_unused)
3232 {
3333 ui__need_resize = 1;
3434 }
@@ -88,7 +88,7 @@ int ui__getch(int delay_secs) · 53 unmodified lines
8888 return SLkp_getkey();
8989 }
9090
91−static void newt_suspend(void *d __used)
91+static void newt_suspend(void *d __maybe_unused)
9292 {
9393 newtSuspend();
9494 raise(SIGTSTP);
@@ -126,6 +126,8 @@ int ui__init(void) · 31 unmodified lines
126126 signal(SIGTERM, ui__signal);
127127
128128 perf_error__register(&perf_tui_eops);
129+
130+ hist_browser__init_hpp();
129131 out:
130132 return err;
131133 }
tools/perf/util/alias.c+2−1
@@ -3,7 +3,8 @@ · 2 unmodified lines
33 static const char *alias_key;
44 static char *alias_val;
55
6−static int alias_lookup_cb(const char *k, const char *v, void *cb __used)
6+static int alias_lookup_cb(const char *k, const char *v,
7+ void *cb __maybe_unused)
78 {
89 if (!prefixcmp(k, "alias.") && !strcmp(k+6, alias_key)) {
910 if (!v)
tools/perf/util/annotate.c+14−5
@@ -17,6 +17,7 @@ · 16 unmodified lines
1717 #include <pthread.h>
1818
1919 const char *disassembler_style;
20+const char *objdump_path;
2021
2122 static struct ins *ins__find(const char *name);
2223 static int disasm_line__parse(char *line, char **namep, char **rawp);
@@ -312,8 +313,8 @@ static struct ins_ops dec_ops = { · 289 unmodified lines
312313 .scnprintf = dec__scnprintf,
313314 };
314315
315−static int nop__scnprintf(struct ins *ins __used, char *bf, size_t size,
316− struct ins_operands *ops __used)
316+static int nop__scnprintf(struct ins *ins __maybe_unused, char *bf, size_t size,
317+ struct ins_operands *ops __maybe_unused)
317318 {
318319 return scnprintf(bf, size, "%-6.6s", "nop");
319320 }
@@ -415,7 +416,7 @@ static struct ins *ins__find(const char *name) · 95 unmodified lines
415416 return bsearch(name, instructions, nmemb, sizeof(struct ins), ins__cmp);
416417 }
417418
418−int symbol__annotate_init(struct map *map __used, struct symbol *sym)
419+int symbol__annotate_init(struct map *map __maybe_unused, struct symbol *sym)
419420 {
420421 struct annotation *notes = symbol__annotation(sym);
421422 pthread_mutex_init(&notes->lock, NULL);
@@ -820,9 +821,10 @@ fallback: · 398 unmodified lines
820821 dso, dso->long_name, sym, sym->name);
821822
822823 snprintf(command, sizeof(command),
823− "objdump %s%s --start-address=0x%016" PRIx64
824+ "%s %s%s --start-address=0x%016" PRIx64
824825 " --stop-address=0x%016" PRIx64
825826 " -d %s %s -C %s|grep -v %s|expand",
827+ objdump_path ? objdump_path : "objdump",
826828 disassembler_style ? "-M " : "",
827829 disassembler_style ? disassembler_style : "",
828830 map__rip_2objdump(map, sym->start),
@@ -982,7 +984,8 @@ int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx, · 153 unmodified lines
982984 int context)
983985 {
984986 struct dso *dso = map->dso;
985− const char *filename = dso->long_name, *d_filename;
987+ char *filename;
988+ const char *d_filename;
986989 struct annotation *notes = symbol__annotation(sym);
987990 struct disasm_line *pos, *queue = NULL;
988991 u64 start = map__rip_2objdump(map, sym->start);
@@ -990,6 +993,10 @@ int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx, · 1 unmodified line
990993 int more = 0;
991994 u64 len;
992995
996+ filename = strdup(dso->long_name);
997+ if (!filename)
998+ return -ENOMEM;
999+
9931000 if (full_paths)
9941001 d_filename = filename;
9951002 else
@@ -1040,6 +1047,8 @@ int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx, · 44 unmodified lines
10401047 }
10411048 }
10421049
1050+ free(filename);
1051+
10431052 return more;
10441053 }
10451054
tools/perf/util/annotate.h+9−6
@@ -7,6 +7,7 @@ · 6 unmodified lines
77 #include "symbol.h"
88 #include <linux/list.h>
99 #include <linux/rbtree.h>
10+#include <pthread.h>
1011
1112 struct ins;
1213
@@ -125,7 +126,7 @@ int symbol__alloc_hist(struct symbol *sym); · 112 unmodified lines
125126 void symbol__annotate_zero_histograms(struct symbol *sym);
126127
127128 int symbol__annotate(struct symbol *sym, struct map *map, size_t privsize);
128−int symbol__annotate_init(struct map *map __used, struct symbol *sym);
129+int symbol__annotate_init(struct map *map __maybe_unused, struct symbol *sym);
129130 int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx,
130131 bool full_paths, int min_pcnt, int max_lines,
131132 int context);
@@ -138,11 +139,12 @@ int symbol__tty_annotate(struct symbol *sym, struct map *map, int evidx, · 6 unmodified lines
138139 int max_lines);
139140
140141 #ifdef NO_NEWT_SUPPORT
141−static inline int symbol__tui_annotate(struct symbol *sym __used,
142− struct map *map __used,
143− int evidx __used,
144− void(*timer)(void *arg) __used,
145− void *arg __used, int delay_secs __used)
142+static inline int symbol__tui_annotate(struct symbol *sym __maybe_unused,
143+ struct map *map __maybe_unused,
144+ int evidx __maybe_unused,
145+ void(*timer)(void *arg) __maybe_unused,
146+ void *arg __maybe_unused,
147+ int delay_secs __maybe_unused)
146148 {
147149 return 0;
148150 }
@@ -152,5 +154,6 @@ int symbol__tui_annotate(struct symbol *sym, struct map *map, int evidx, · 3 unmodified lines
152154 #endif
153155
154156 extern const char *disassembler_style;
157+extern const char *objdump_path;
155158
156159 #endif /* __PERF_ANNOTATE_H */
tools/perf/util/build-id.c+6−5
@@ -16,10 +16,10 @@ · 15 unmodified lines
1616 #include "session.h"
1717 #include "tool.h"
1818
19−static int build_id__mark_dso_hit(struct perf_tool *tool __used,
19+static int build_id__mark_dso_hit(struct perf_tool *tool __maybe_unused,
2020 union perf_event *event,
21− struct perf_sample *sample __used,
22− struct perf_evsel *evsel __used,
21+ struct perf_sample *sample __maybe_unused,
22+ struct perf_evsel *evsel __maybe_unused,
2323 struct machine *machine)
2424 {
2525 struct addr_location al;
@@ -41,9 +41,10 @@ static int build_id__mark_dso_hit(struct perf_tool *tool __used, · 15 unmodified lines
4141 return 0;
4242 }
4343
44−static int perf_event__exit_del_thread(struct perf_tool *tool __used,
44+static int perf_event__exit_del_thread(struct perf_tool *tool __maybe_unused,
4545 union perf_event *event,
46− struct perf_sample *sample __used,
46+ struct perf_sample *sample
47+ __maybe_unused,
4748 struct machine *machine)
4849 {
4950 struct thread *thread = machine__findnew_thread(machine, event->fork.tid);
tools/perf/util/cache.h+3−3
@@ -39,7 +39,7 @@ static inline void setup_browser(bool fallback_to_pager) · 38 unmodified lines
3939 if (fallback_to_pager)
4040 setup_pager();
4141 }
42−static inline void exit_browser(bool wait_for_ok __used) {}
42+static inline void exit_browser(bool wait_for_ok __maybe_unused) {}
4343 #else
4444 void setup_browser(bool fallback_to_pager);
4545 void exit_browser(bool wait_for_ok);
@@ -49,7 +49,7 @@ static inline int ui__init(void) · 3 unmodified lines
4949 {
5050 return -1;
5151 }
52−static inline void ui__exit(bool wait_for_ok __used) {}
52+static inline void ui__exit(bool wait_for_ok __maybe_unused) {}
5353 #else
5454 int ui__init(void);
5555 void ui__exit(bool wait_for_ok);
@@ -60,7 +60,7 @@ static inline int perf_gtk__init(void) · 4 unmodified lines
6060 {
6161 return -1;
6262 }
63−static inline void perf_gtk__exit(bool wait_for_ok __used) {}
63+static inline void perf_gtk__exit(bool wait_for_ok __maybe_unused) {}
6464 #else
6565 int perf_gtk__init(void);
6666 void perf_gtk__exit(bool wait_for_ok);
tools/perf/util/callchain.c+3−3
@@ -93,7 +93,7 @@ __sort_chain_flat(struct rb_root *rb_root, struct callchain_node *node, · 92 unmodified lines
9393 */
9494 static void
9595 sort_chain_flat(struct rb_root *rb_root, struct callchain_root *root,
96− u64 min_hit, struct callchain_param *param __used)
96+ u64 min_hit, struct callchain_param *param __maybe_unused)
9797 {
9898 __sort_chain_flat(rb_root, &root->node, min_hit);
9999 }
@@ -115,7 +115,7 @@ static void __sort_chain_graph_abs(struct callchain_node *node, · 15 unmodified lines
115115
116116 static void
117117 sort_chain_graph_abs(struct rb_root *rb_root, struct callchain_root *chain_root,
118− u64 min_hit, struct callchain_param *param __used)
118+ u64 min_hit, struct callchain_param *param __maybe_unused)
119119 {
120120 __sort_chain_graph_abs(&chain_root->node, min_hit);
121121 rb_root->rb_node = chain_root->node.rb_root.rb_node;
@@ -140,7 +140,7 @@ static void __sort_chain_graph_rel(struct callchain_node *node, · 18 unmodified lines
140140
141141 static void
142142 sort_chain_graph_rel(struct rb_root *rb_root, struct callchain_root *chain_root,
143− u64 min_hit __used, struct callchain_param *param)
143+ u64 min_hit __maybe_unused, struct callchain_param *param)
144144 {
145145 __sort_chain_graph_rel(&chain_root->node, param->min_percent / 100.0);
146146 rb_root->rb_node = chain_root->node.rb_root.rb_node;
tools/perf/util/cgroup.c+2−2
@@ -138,8 +138,8 @@ void close_cgroup(struct cgroup_sel *cgrp) · 137 unmodified lines
138138 }
139139 }
140140
141−int parse_cgroups(const struct option *opt __used, const char *str,
142− int unset __used)
141+int parse_cgroups(const struct option *opt __maybe_unused, const char *str,
142+ int unset __maybe_unused)
143143 {
144144 struct perf_evlist *evlist = *(struct perf_evlist **)opt->value;
145145 const char *p, *e, *eos = str + strlen(str);
tools/perf/util/config.c+4−2
@@ -342,13 +342,15 @@ const char *perf_config_dirname(const char *name, const char *value) · 341 unmodified lines
342342 return value;
343343 }
344344
345−static int perf_default_core_config(const char *var __used, const char *value __used)
345+static int perf_default_core_config(const char *var __maybe_unused,
346+ const char *value __maybe_unused)
346347 {
347348 /* Add other config variables here. */
348349 return 0;
349350 }
350351
351−int perf_default_config(const char *var, const char *value, void *dummy __used)
352+int perf_default_config(const char *var, const char *value,
353+ void *dummy __maybe_unused)
352354 {
353355 if (!prefixcmp(var, "core."))
354356 return perf_default_core_config(var, value);
tools/perf/util/cpumap.c+15−7
@@ -38,24 +38,19 @@ static struct cpu_map *cpu_map__trim_new(int nr_cpus, int *tmp_cpus) · 37 unmodified lines
3838 return cpus;
3939 }
4040
41−static struct cpu_map *cpu_map__read_all_cpu_map(void)
41+struct cpu_map *cpu_map__read(FILE *file)
4242 {
4343 struct cpu_map *cpus = NULL;
44− FILE *onlnf;
4544 int nr_cpus = 0;
4645 int *tmp_cpus = NULL, *tmp;
4746 int max_entries = 0;
4847 int n, cpu, prev;
4948 char sep;
5049
51− onlnf = fopen("/sys/devices/system/cpu/online", "r");
52− if (!onlnf)
53− return cpu_map__default_new();
54−
5550 sep = 0;
5651 prev = -1;
5752 for (;;) {
58− n = fscanf(onlnf, "%u%c", &cpu, &sep);
53+ n = fscanf(file, "%u%c", &cpu, &sep);
5954 if (n <= 0)
6055 break;
6156 if (prev >= 0) {
@@ -95,6 +90,19 @@ static struct cpu_map *cpu_map__read_all_cpu_map(void) · 33 unmodified lines
9590 cpus = cpu_map__default_new();
9691 out_free_tmp:
9792 free(tmp_cpus);
93+ return cpus;
94+}
95+
96+static struct cpu_map *cpu_map__read_all_cpu_map(void)
97+{
98+ struct cpu_map *cpus = NULL;
99+ FILE *onlnf;
100+
101+ onlnf = fopen("/sys/devices/system/cpu/online", "r");
102+ if (!onlnf)
103+ return cpu_map__default_new();
104+
105+ cpus = cpu_map__read(onlnf);
98106 fclose(onlnf);
99107 return cpus;
100108 }
tools/perf/util/cpumap.h+12−1
@@ -2,6 +2,7 @@ · 1 unmodified line
22 #define __PERF_CPUMAP_H
33
44 #include <stdio.h>
5+#include <stdbool.h>
56
67 struct cpu_map {
78 int nr;
@@ -11,7 +12,17 @@ struct cpu_map { · 3 unmodified lines
1112 struct cpu_map *cpu_map__new(const char *cpu_list);
1213 struct cpu_map *cpu_map__dummy_new(void);
1314 void cpu_map__delete(struct cpu_map *map);
14−
15+struct cpu_map *cpu_map__read(FILE *file);
1516 size_t cpu_map__fprintf(struct cpu_map *map, FILE *fp);
1617
18+static inline int cpu_map__nr(const struct cpu_map *map)
19+{
20+ return map ? map->nr : 1;
21+}
22+
23+static inline bool cpu_map__all(const struct cpu_map *map)
24+{
25+ return map ? map->map[0] == -1 : true;
26+}
27+
1728 #endif /* __PERF_CPUMAP_H */
tools/perf/util/debug.c+3−1
@@ -23,8 +23,10 @@ int eprintf(int level, const char *fmt, ...) · 22 unmodified lines
2323
2424 if (verbose >= level) {
2525 va_start(args, fmt);
26− if (use_browser > 0)
26+ if (use_browser == 1)
2727 ret = ui_helpline__show_help(fmt, args);
28+ else if (use_browser == 2)
29+ ret = perf_gtk__show_helpline(fmt, args);
2830 else
2931 ret = vfprintf(stderr, fmt, args);
3032 va_end(args);
tools/perf/util/debug.h+6−11
@@ -4,6 +4,7 @@ · 3 unmodified lines
44
55 #include <stdbool.h>
66 #include "event.h"
7+#include "../ui/helpline.h"
78
89 extern int verbose;
910 extern bool quiet, dump_trace;
@@ -15,32 +16,26 @@ struct ui_progress; · 5 unmodified lines
1516 struct perf_error_ops;
1617
1718 #if defined(NO_NEWT_SUPPORT) && defined(NO_GTK2_SUPPORT)
18−static inline int ui_helpline__show_help(const char *format __used, va_list ap __used)
19−{
20− return 0;
21−}
22−
23−static inline void ui_progress__update(u64 curr __used, u64 total __used,
24− const char *title __used) {}
19+static inline void ui_progress__update(u64 curr __maybe_unused,
20+ u64 total __maybe_unused,
21+ const char *title __maybe_unused) {}
2522
2623 #define ui__error(format, arg...) ui__warning(format, ##arg)
2724
2825 static inline int
29−perf_error__register(struct perf_error_ops *eops __used)
26+perf_error__register(struct perf_error_ops *eops __maybe_unused)
3027 {
3128 return 0;
3229 }
3330
3431 static inline int
35−perf_error__unregister(struct perf_error_ops *eops __used)
32+perf_error__unregister(struct perf_error_ops *eops __maybe_unused)
3633 {
3734 return 0;
3835 }
3936
4037 #else /* NO_NEWT_SUPPORT && NO_GTK2_SUPPORT */
4138
42−extern char ui_helpline__last_msg[];
43−int ui_helpline__show_help(const char *format, va_list ap);
4439 #include "../ui/progress.h"
4540 int ui__error(const char *format, ...) __attribute__((format(printf, 1, 2)));
4641 #include "../ui/util.h"
tools/perf/util/dso-test-data.c+1−1
@@ -23,7 +23,7 @@ static char *test_file(int size) · 22 unmodified lines
2323 int fd, i;
2424 unsigned char *buf;
2525
26− fd = mkostemp(templ, O_CREAT|O_WRONLY|O_TRUNC);
26+ fd = mkstemp(templ);
2727
2828 buf = malloc(size);
2929 if (!buf) {
tools/perf/util/dwarf-aux.c+2−0
@@ -804,6 +804,8 @@ int die_get_typename(Dwarf_Die *vr_die, char *buf, int len) · 803 unmodified lines
804804 tmp = "union ";
805805 else if (tag == DW_TAG_structure_type)
806806 tmp = "struct ";
807+ else if (tag == DW_TAG_enumeration_type)
808+ tmp = "enum ";
807809 /* Write a base name */
808810 ret = snprintf(buf, len, "%s%s", tmp, dwarf_diename(&type));
809811 return (ret >= len) ? -E2BIG : ret;
tools/perf/util/event.c+45−26
@@ -112,7 +112,7 @@ static pid_t perf_event__synthesize_comm(struct perf_tool *tool, · 111 unmodified lines
112112 event->comm.header.type = PERF_RECORD_COMM;
113113
114114 size = strlen(event->comm.comm) + 1;
115− size = ALIGN(size, sizeof(u64));
115+ size = PERF_ALIGN(size, sizeof(u64));
116116 memset(event->comm.comm + size, 0, machine->id_hdr_size);
117117 event->comm.header.size = (sizeof(event->comm) -
118118 (sizeof(event->comm.comm) - size) +
@@ -120,7 +120,9 @@ static pid_t perf_event__synthesize_comm(struct perf_tool *tool, · 1 unmodified line
120120 if (!full) {
121121 event->comm.tid = pid;
122122
123− process(tool, event, &synth_sample, machine);
123+ if (process(tool, event, &synth_sample, machine) != 0)
124+ return -1;
125+
124126 goto out;
125127 }
126128
@@ -143,7 +145,7 @@ static pid_t perf_event__synthesize_comm(struct perf_tool *tool, · 16 unmodified lines
143145 sizeof(event->comm.comm));
144146
145147 size = strlen(event->comm.comm) + 1;
146− size = ALIGN(size, sizeof(u64));
148+ size = PERF_ALIGN(size, sizeof(u64));
147149 memset(event->comm.comm + size, 0, machine->id_hdr_size);
148150 event->comm.header.size = (sizeof(event->comm) -
149151 (sizeof(event->comm.comm) - size) +
@@ -151,7 +153,10 @@ static pid_t perf_event__synthesize_comm(struct perf_tool *tool, · 1 unmodified line
151153
152154 event->comm.tid = pid;
153155
154− process(tool, event, &synth_sample, machine);
156+ if (process(tool, event, &synth_sample, machine) != 0) {
157+ tgid = -1;
158+ break;
159+ }
155160 }
156161
157162 closedir(tasks);
@@ -167,6 +172,7 @@ static int perf_event__synthesize_mmap_events(struct perf_tool *tool, · 9 unmodified lines
167172 {
168173 char filename[PATH_MAX];
169174 FILE *fp;
175+ int rc = 0;
170176
171177 snprintf(filename, sizeof(filename), "/proc/%d/maps", pid);
172178
@@ -222,7 +228,7 @@ static int perf_event__synthesize_mmap_events(struct perf_tool *tool, · 49 unmodified lines
222228 size = strlen(execname);
223229 execname[size - 1] = '\0'; /* Remove \n */
224230 memcpy(event->mmap.filename, execname, size);
225− size = ALIGN(size, sizeof(u64));
231+ size = PERF_ALIGN(size, sizeof(u64));
226232 event->mmap.len -= event->mmap.start;
227233 event->mmap.header.size = (sizeof(event->mmap) -
228234 (sizeof(event->mmap.filename) - size));
@@ -231,18 +237,22 @@ static int perf_event__synthesize_mmap_events(struct perf_tool *tool, · 2 unmodified lines
231237 event->mmap.pid = tgid;
232238 event->mmap.tid = pid;
233239
234− process(tool, event, &synth_sample, machine);
240+ if (process(tool, event, &synth_sample, machine) != 0) {
241+ rc = -1;
242+ break;
243+ }
235244 }
236245 }
237246
238247 fclose(fp);
239− return 0;
248+ return rc;
240249 }
241250
242251 int perf_event__synthesize_modules(struct perf_tool *tool,
243252 perf_event__handler_t process,
244253 struct machine *machine)
245254 {
255+ int rc = 0;
246256 struct rb_node *nd;
247257 struct map_groups *kmaps = &machine->kmaps;
248258 union perf_event *event = zalloc((sizeof(event->mmap) +
@@ -272,7 +282,7 @@ int perf_event__synthesize_modules(struct perf_tool *tool, · 23 unmodified lines
272282 if (pos->dso->kernel)
273283 continue;
274284
275− size = ALIGN(pos->dso->long_name_len + 1, sizeof(u64));
285+ size = PERF_ALIGN(pos->dso->long_name_len + 1, sizeof(u64));
276286 event->mmap.header.type = PERF_RECORD_MMAP;
277287 event->mmap.header.size = (sizeof(event->mmap) -
278288 (sizeof(event->mmap.filename) - size));
@@ -284,11 +294,14 @@ int perf_event__synthesize_modules(struct perf_tool *tool, · 5 unmodified lines
284294
285295 memcpy(event->mmap.filename, pos->dso->long_name,
286296 pos->dso->long_name_len + 1);
287− process(tool, event, &synth_sample, machine);
297+ if (process(tool, event, &synth_sample, machine) != 0) {
298+ rc = -1;
299+ break;
300+ }
288301 }
289302
290303 free(event);
291− return 0;
304+ return rc;
292305 }
293306
294307 static int __event__synthesize_thread(union perf_event *comm_event,
@@ -392,12 +405,16 @@ int perf_event__synthesize_threads(struct perf_tool *tool, · 97 unmodified lines
392405 if (*end) /* only interested in proper numerical dirents */
393406 continue;
394407
395− __event__synthesize_thread(comm_event, mmap_event, pid, 1,
396− process, tool, machine);
408+ if (__event__synthesize_thread(comm_event, mmap_event, pid, 1,
409+ process, tool, machine) != 0) {
410+ err = -1;
411+ goto out_closedir;
412+ }
397413 }
398414
399− closedir(proc);
400415 err = 0;
416+out_closedir:
417+ closedir(proc);
401418 out_free_mmap:
402419 free(mmap_event);
403420 out_free_comm:
@@ -412,7 +429,7 @@ struct process_symbol_args { · 8 unmodified lines
412429 };
413430
414431 static int find_symbol_cb(void *arg, const char *name, char type,
415− u64 start, u64 end __used)
432+ u64 start)
416433 {
417434 struct process_symbol_args *args = arg;
418435
@@ -477,7 +494,7 @@ int perf_event__synthesize_kernel_mmap(struct perf_tool *tool, · 58 unmodified lines
477494 map = machine->vmlinux_maps[MAP__FUNCTION];
478495 size = snprintf(event->mmap.filename, sizeof(event->mmap.filename),
479496 "%s%s", mmap_name, symbol_name) + 1;
480− size = ALIGN(size, sizeof(u64));
497+ size = PERF_ALIGN(size, sizeof(u64));
481498 event->mmap.header.type = PERF_RECORD_MMAP;
482499 event->mmap.header.size = (sizeof(event->mmap) -
483500 (sizeof(event->mmap.filename) - size) + machine->id_hdr_size);
@@ -497,9 +514,9 @@ size_t perf_event__fprintf_comm(union perf_event *event, FILE *fp) · 13 unmodified lines
497514 return fprintf(fp, ": %s:%d\n", event->comm.comm, event->comm.tid);
498515 }
499516
500−int perf_event__process_comm(struct perf_tool *tool __used,
517+int perf_event__process_comm(struct perf_tool *tool __maybe_unused,
501518 union perf_event *event,
502− struct perf_sample *sample __used,
519+ struct perf_sample *sample __maybe_unused,
503520 struct machine *machine)
504521 {
505522 struct thread *thread = machine__findnew_thread(machine, event->comm.tid);
@@ -515,10 +532,10 @@ int perf_event__process_comm(struct perf_tool *tool __used, · 9 unmodified lines
515532 return 0;
516533 }
517534
518−int perf_event__process_lost(struct perf_tool *tool __used,
535+int perf_event__process_lost(struct perf_tool *tool __maybe_unused,
519536 union perf_event *event,
520− struct perf_sample *sample __used,
521− struct machine *machine __used)
537+ struct perf_sample *sample __maybe_unused,
538+ struct machine *machine __maybe_unused)
522539 {
523540 dump_printf(": id:%" PRIu64 ": lost:%" PRIu64 "\n",
524541 event->lost.id, event->lost.lost);
@@ -538,7 +555,8 @@ static void perf_event__set_kernel_mmap_len(union perf_event *event, · 13 unmodified lines
538555 maps[MAP__FUNCTION]->end = ~0ULL;
539556 }
540557
541−static int perf_event__process_kernel_mmap(struct perf_tool *tool __used,
558+static int perf_event__process_kernel_mmap(struct perf_tool *tool
559+ __maybe_unused,
542560 union perf_event *event,
543561 struct machine *machine)
544562 {
@@ -640,7 +658,7 @@ size_t perf_event__fprintf_mmap(union perf_event *event, FILE *fp) · 95 unmodified lines
640658
641659 int perf_event__process_mmap(struct perf_tool *tool,
642660 union perf_event *event,
643− struct perf_sample *sample __used,
661+ struct perf_sample *sample __maybe_unused,
644662 struct machine *machine)
645663 {
646664 struct thread *thread;
@@ -684,9 +702,9 @@ size_t perf_event__fprintf_task(union perf_event *event, FILE *fp) · 37 unmodified lines
684702 event->fork.ppid, event->fork.ptid);
685703 }
686704
687−int perf_event__process_task(struct perf_tool *tool __used,
705+int perf_event__process_task(struct perf_tool *tool __maybe_unused,
688706 union perf_event *event,
689− struct perf_sample *sample __used,
707+ struct perf_sample *sample __maybe_unused,
690708 struct machine *machine)
691709 {
692710 struct thread *thread = machine__findnew_thread(machine, event->fork.tid);
@@ -886,8 +904,9 @@ int perf_event__preprocess_sample(const union perf_event *event, · 193 unmodified lines
886904 al->sym = map__find_symbol(al->map, al->addr, filter);
887905 }
888906
889− if (symbol_conf.sym_list && al->sym &&
890− !strlist__has_entry(symbol_conf.sym_list, al->sym->name))
907+ if (symbol_conf.sym_list &&
908+ (!al->sym || !strlist__has_entry(symbol_conf.sym_list,
909+ al->sym->name)))
891910 goto out_filtered;
892911
893912 return 0;
tools/perf/util/event.h+13−1
@@ -69,6 +69,16 @@ struct sample_event { · 68 unmodified lines
6969 u64 array[];
7070 };
7171
72+struct regs_dump {
73+ u64 *regs;
74+};
75+
76+struct stack_dump {
77+ u16 offset;
78+ u64 size;
79+ char *data;
80+};
81+
7282 struct perf_sample {
7383 u64 ip;
7484 u32 pid, tid;
@@ -82,6 +92,8 @@ struct perf_sample { · 7 unmodified lines
8292 void *raw_data;
8393 struct ip_callchain *callchain;
8494 struct branch_stack *branch_stack;
95+ struct regs_dump user_regs;
96+ struct stack_dump user_stack;
8597 };
8698
8799 #define BUILD_ID_SIZE 20
@@ -89,7 +101,7 @@ struct perf_sample { · 1 unmodified line
89101 struct build_id_event {
90102 struct perf_event_header header;
91103 pid_t pid;
92− u8 build_id[ALIGN(BUILD_ID_SIZE, sizeof(u64))];
104+ u8 build_id[PERF_ALIGN(BUILD_ID_SIZE, sizeof(u64))];
93105 char filename[];
94106 };
95107
tools/perf/util/evlist.c+82−61
@@ -57,7 +57,7 @@ void perf_evlist__config_attrs(struct perf_evlist *evlist, · 56 unmodified lines
5757 if (evlist->cpus->map[0] < 0)
5858 opts->no_inherit = true;
5959
60− first = list_entry(evlist->entries.next, struct perf_evsel, node);
60+ first = perf_evlist__first(evlist);
6161
6262 list_for_each_entry(evsel, &evlist->entries, node) {
6363 perf_evsel__config(evsel, opts, first);
@@ -108,6 +108,25 @@ void perf_evlist__splice_list_tail(struct perf_evlist *evlist, · 44 unmodified lines
108108 evlist->nr_entries += nr_entries;
109109 }
110110
111+void __perf_evlist__set_leader(struct list_head *list)
112+{
113+ struct perf_evsel *evsel, *leader;
114+
115+ leader = list_entry(list->next, struct perf_evsel, node);
116+ leader->leader = NULL;
117+
118+ list_for_each_entry(evsel, list, node) {
119+ if (evsel != leader)
120+ evsel->leader = leader;
121+ }
122+}
123+
124+void perf_evlist__set_leader(struct perf_evlist *evlist)
125+{
126+ if (evlist->nr_entries)
127+ __perf_evlist__set_leader(&evlist->entries);
128+}
129+
111130 int perf_evlist__add_default(struct perf_evlist *evlist)
112131 {
113132 struct perf_event_attr attr = {
@@ -285,7 +304,7 @@ void perf_evlist__enable(struct perf_evlist *evlist) · 171 unmodified lines
285304 int cpu, thread;
286305 struct perf_evsel *pos;
287306
288− for (cpu = 0; cpu < evlist->cpus->nr; cpu++) {
307+ for (cpu = 0; cpu < cpu_map__nr(evlist->cpus); cpu++) {
289308 list_for_each_entry(pos, &evlist->entries, node) {
290309 for (thread = 0; thread < evlist->threads->nr; thread++)
291310 ioctl(FD(pos, cpu, thread),
@@ -296,7 +315,7 @@ void perf_evlist__enable(struct perf_evlist *evlist) · 4 unmodified lines
296315
297316 static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist)
298317 {
299− int nfds = evlist->cpus->nr * evlist->threads->nr * evlist->nr_entries;
318+ int nfds = cpu_map__nr(evlist->cpus) * evlist->threads->nr * evlist->nr_entries;
300319 evlist->pollfd = malloc(sizeof(struct pollfd) * nfds);
301320 return evlist->pollfd != NULL ? 0 : -ENOMEM;
302321 }
@@ -357,7 +376,7 @@ struct perf_evsel *perf_evlist__id2evsel(struct perf_evlist *evlist, u64 id) · 54 unmodified lines
357376 int hash;
358377
359378 if (evlist->nr_entries == 1)
360− return list_entry(evlist->entries.next, struct perf_evsel, node);
379+ return perf_evlist__first(evlist);
361380
362381 hash = hash_64(id, PERF_EVLIST__HLIST_BITS);
363382 head = &evlist->heads[hash];
@@ -367,7 +386,7 @@ struct perf_evsel *perf_evlist__id2evsel(struct perf_evlist *evlist, u64 id) · 3 unmodified lines
367386 return sid->evsel;
368387
369388 if (!perf_evlist__sample_id_all(evlist))
370− return list_entry(evlist->entries.next, struct perf_evsel, node);
389+ return perf_evlist__first(evlist);
371390
372391 return NULL;
373392 }
@@ -456,8 +475,8 @@ void perf_evlist__munmap(struct perf_evlist *evlist) · 82 unmodified lines
456475
457476 static int perf_evlist__alloc_mmap(struct perf_evlist *evlist)
458477 {
459− evlist->nr_mmaps = evlist->cpus->nr;
460− if (evlist->cpus->map[0] == -1)
478+ evlist->nr_mmaps = cpu_map__nr(evlist->cpus);
479+ if (cpu_map__all(evlist->cpus))
461480 evlist->nr_mmaps = evlist->threads->nr;
462481 evlist->mmap = zalloc(evlist->nr_mmaps * sizeof(struct perf_mmap));
463482 return evlist->mmap != NULL ? 0 : -ENOMEM;
@@ -603,11 +622,11 @@ int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages, · 139 unmodified lines
603622 list_for_each_entry(evsel, &evlist->entries, node) {
604623 if ((evsel->attr.read_format & PERF_FORMAT_ID) &&
605624 evsel->sample_id == NULL &&
606− perf_evsel__alloc_id(evsel, cpus->nr, threads->nr) < 0)
625+ perf_evsel__alloc_id(evsel, cpu_map__nr(cpus), threads->nr) < 0)
607626 return -ENOMEM;
608627 }
609628
610− if (evlist->cpus->map[0] == -1)
629+ if (cpu_map__all(cpus))
611630 return perf_evlist__mmap_per_thread(evlist, prot, mask);
612631
613632 return perf_evlist__mmap_per_cpu(evlist, prot, mask);
@@ -647,39 +666,44 @@ void perf_evlist__delete_maps(struct perf_evlist *evlist) · 33 unmodified lines
647666 evlist->threads = NULL;
648667 }
649668
650−int perf_evlist__set_filters(struct perf_evlist *evlist)
669+int perf_evlist__apply_filters(struct perf_evlist *evlist)
651670 {
652− const struct thread_map *threads = evlist->threads;
653− const struct cpu_map *cpus = evlist->cpus;
654671 struct perf_evsel *evsel;
655− char *filter;
656− int thread;
657− int cpu;
658− int err;
659− int fd;
672+ int err = 0;
673+ const int ncpus = cpu_map__nr(evlist->cpus),
674+ nthreads = evlist->threads->nr;
660675
661676 list_for_each_entry(evsel, &evlist->entries, node) {
662− filter = evsel->filter;
663− if (!filter)
677+ if (evsel->filter == NULL)
664678 continue;
665− for (cpu = 0; cpu < cpus->nr; cpu++) {
666− for (thread = 0; thread < threads->nr; thread++) {
667− fd = FD(evsel, cpu, thread);
668− err = ioctl(fd, PERF_EVENT_IOC_SET_FILTER, filter);
669− if (err)
670− return err;
671− }
672− }
679+
680+ err = perf_evsel__set_filter(evsel, ncpus, nthreads, evsel->filter);
681+ if (err)
682+ break;
673683 }
674684
675− return 0;
685+ return err;
676686 }
677687
678−bool perf_evlist__valid_sample_type(const struct perf_evlist *evlist)
688+int perf_evlist__set_filter(struct perf_evlist *evlist, const char *filter)
679689 {
680− struct perf_evsel *pos, *first;
690+ struct perf_evsel *evsel;
691+ int err = 0;
692+ const int ncpus = cpu_map__nr(evlist->cpus),
693+ nthreads = evlist->threads->nr;
694+
695+ list_for_each_entry(evsel, &evlist->entries, node) {
696+ err = perf_evsel__set_filter(evsel, ncpus, nthreads, filter);
697+ if (err)
698+ break;
699+ }
700+
701+ return err;
702+}
681703
682− pos = first = list_entry(evlist->entries.next, struct perf_evsel, node);
704+bool perf_evlist__valid_sample_type(struct perf_evlist *evlist)
705+{
706+ struct perf_evsel *first = perf_evlist__first(evlist), *pos = first;
683707
684708 list_for_each_entry_continue(pos, &evlist->entries, node) {
685709 if (first->attr.sample_type != pos->attr.sample_type)
@@ -689,23 +713,19 @@ bool perf_evlist__valid_sample_type(const struct perf_evlist *evlist) · 3 unmodified lines
689713 return true;
690714 }
691715
692−u64 perf_evlist__sample_type(const struct perf_evlist *evlist)
716+u64 perf_evlist__sample_type(struct perf_evlist *evlist)
693717 {
694− struct perf_evsel *first;
695−
696− first = list_entry(evlist->entries.next, struct perf_evsel, node);
718+ struct perf_evsel *first = perf_evlist__first(evlist);
697719 return first->attr.sample_type;
698720 }
699721
700−u16 perf_evlist__id_hdr_size(const struct perf_evlist *evlist)
722+u16 perf_evlist__id_hdr_size(struct perf_evlist *evlist)
701723 {
702− struct perf_evsel *first;
724+ struct perf_evsel *first = perf_evlist__first(evlist);
703725 struct perf_sample *data;
704726 u64 sample_type;
705727 u16 size = 0;
706728
707− first = list_entry(evlist->entries.next, struct perf_evsel, node);
708−
709729 if (!first->attr.sample_id_all)
710730 goto out;
711731
@@ -729,11 +749,9 @@ out: · 17 unmodified lines
729749 return size;
730750 }
731751
732−bool perf_evlist__valid_sample_id_all(const struct perf_evlist *evlist)
752+bool perf_evlist__valid_sample_id_all(struct perf_evlist *evlist)
733753 {
734− struct perf_evsel *pos, *first;
735−
736− pos = first = list_entry(evlist->entries.next, struct perf_evsel, node);
754+ struct perf_evsel *first = perf_evlist__first(evlist), *pos = first;
737755
738756 list_for_each_entry_continue(pos, &evlist->entries, node) {
739757 if (first->attr.sample_id_all != pos->attr.sample_id_all)
@@ -743,11 +761,9 @@ bool perf_evlist__valid_sample_id_all(const struct perf_evlist *evlist) · 3 unmodified lines
743761 return true;
744762 }
745763
746−bool perf_evlist__sample_id_all(const struct perf_evlist *evlist)
764+bool perf_evlist__sample_id_all(struct perf_evlist *evlist)
747765 {
748− struct perf_evsel *first;
749−
750− first = list_entry(evlist->entries.next, struct perf_evsel, node);
766+ struct perf_evsel *first = perf_evlist__first(evlist);
751767 return first->attr.sample_id_all;
752768 }
753769
@@ -757,21 +773,13 @@ void perf_evlist__set_selected(struct perf_evlist *evlist, · 3 unmodified lines
757773 evlist->selected = evsel;
758774 }
759775
760−int perf_evlist__open(struct perf_evlist *evlist, bool group)
776+int perf_evlist__open(struct perf_evlist *evlist)
761777 {
762− struct perf_evsel *evsel, *first;
778+ struct perf_evsel *evsel;
763779 int err, ncpus, nthreads;
764780
765− first = list_entry(evlist->entries.next, struct perf_evsel, node);
766−
767781 list_for_each_entry(evsel, &evlist->entries, node) {
768− struct xyarray *group_fd = NULL;
769−
770− if (group && evsel != first)
771− group_fd = first->fd;
772−
773− err = perf_evsel__open(evsel, evlist->cpus, evlist->threads,
774− group, group_fd);
782+ err = perf_evsel__open(evsel, evlist->cpus, evlist->threads);
775783 if (err < 0)
776784 goto out_err;
777785 }
@@ -883,8 +891,21 @@ int perf_evlist__start_workload(struct perf_evlist *evlist) · 105 unmodified lines
883891 }
884892
885893 int perf_evlist__parse_sample(struct perf_evlist *evlist, union perf_event *event,
886− struct perf_sample *sample, bool swapped)
894+ struct perf_sample *sample)
895+{
896+ struct perf_evsel *evsel = perf_evlist__first(evlist);
897+ return perf_evsel__parse_sample(evsel, event, sample);
898+}
899+
900+size_t perf_evlist__fprintf(struct perf_evlist *evlist, FILE *fp)
887901 {
888− struct perf_evsel *e = list_entry(evlist->entries.next, struct perf_evsel, node);
889− return perf_evsel__parse_sample(e, event, sample, swapped);
902+ struct perf_evsel *evsel;
903+ size_t printed = 0;
904+
905+ list_for_each_entry(evsel, &evlist->entries, node) {
906+ printed += fprintf(fp, "%s%s", evsel->idx ? ", " : "",
907+ perf_evsel__name(evsel));
908+ }
909+
910+ return printed + fprintf(fp, "\n");;
890911 }
tools/perf/util/evlist.h+25−10
@@ -5,6 +5,7 @@ · 4 unmodified lines
55 #include <stdio.h>
66 #include "../perf.h"
77 #include "event.h"
8+#include "evsel.h"
89 #include "util.h"
910 #include <unistd.h>
1011
@@ -41,8 +42,6 @@ struct perf_evsel_str_handler { · 30 unmodified lines
4142 void *handler;
4243 };
4344
44−struct perf_evsel;
45−
4645 struct perf_evlist *perf_evlist__new(struct cpu_map *cpus,
4746 struct thread_map *threads);
4847 void perf_evlist__init(struct perf_evlist *evlist, struct cpu_map *cpus,
@@ -73,6 +72,8 @@ int perf_evlist__set_tracepoints_handlers(struct perf_evlist *evlist, · 24 unmodified lines
7372 #define perf_evlist__set_tracepoints_handlers_array(evlist, array) \
7473 perf_evlist__set_tracepoints_handlers(evlist, array, ARRAY_SIZE(array))
7574
75+int perf_evlist__set_filter(struct perf_evlist *evlist, const char *filter);
76+
7677 struct perf_evsel *
7778 perf_evlist__find_tracepoint_by_id(struct perf_evlist *evlist, int id);
7879
@@ -85,7 +86,7 @@ struct perf_evsel *perf_evlist__id2evsel(struct perf_evlist *evlist, u64 id); · 6 unmodified lines
8586
8687 union perf_event *perf_evlist__mmap_read(struct perf_evlist *self, int idx);
8788
88−int perf_evlist__open(struct perf_evlist *evlist, bool group);
89+int perf_evlist__open(struct perf_evlist *evlist);
8990
9091 void perf_evlist__config_attrs(struct perf_evlist *evlist,
9192 struct perf_record_opts *opts);
@@ -116,20 +117,34 @@ static inline void perf_evlist__set_maps(struct perf_evlist *evlist, · 24 unmodified lines
116117 int perf_evlist__create_maps(struct perf_evlist *evlist,
117118 struct perf_target *target);
118119 void perf_evlist__delete_maps(struct perf_evlist *evlist);
119−int perf_evlist__set_filters(struct perf_evlist *evlist);
120+int perf_evlist__apply_filters(struct perf_evlist *evlist);
121+
122+void __perf_evlist__set_leader(struct list_head *list);
123+void perf_evlist__set_leader(struct perf_evlist *evlist);
120124
121−u64 perf_evlist__sample_type(const struct perf_evlist *evlist);
122−bool perf_evlist__sample_id_all(const const struct perf_evlist *evlist);
123−u16 perf_evlist__id_hdr_size(const struct perf_evlist *evlist);
125+u64 perf_evlist__sample_type(struct perf_evlist *evlist);
126+bool perf_evlist__sample_id_all(struct perf_evlist *evlist);
127+u16 perf_evlist__id_hdr_size(struct perf_evlist *evlist);
124128
125129 int perf_evlist__parse_sample(struct perf_evlist *evlist, union perf_event *event,
126− struct perf_sample *sample, bool swapped);
130+ struct perf_sample *sample);
127131
128−bool perf_evlist__valid_sample_type(const struct perf_evlist *evlist);
129−bool perf_evlist__valid_sample_id_all(const struct perf_evlist *evlist);
132+bool perf_evlist__valid_sample_type(struct perf_evlist *evlist);
133+bool perf_evlist__valid_sample_id_all(struct perf_evlist *evlist);
130134
131135 void perf_evlist__splice_list_tail(struct perf_evlist *evlist,
132136 struct list_head *list,
133137 int nr_entries);
134138
139+static inline struct perf_evsel *perf_evlist__first(struct perf_evlist *evlist)
140+{
141+ return list_entry(evlist->entries.next, struct perf_evsel, node);
142+}
143+
144+static inline struct perf_evsel *perf_evlist__last(struct perf_evlist *evlist)
145+{
146+ return list_entry(evlist->entries.prev, struct perf_evsel, node);
147+}
148+
149+size_t perf_evlist__fprintf(struct perf_evlist *evlist, FILE *fp);
135150 #endif /* __PERF_EVLIST_H */
tools/perf/util/evsel.c+248−28
@@ -8,7 +8,10 @@ · 7 unmodified lines
88 */
99
1010 #include <byteswap.h>
11+#include <linux/bitops.h>
1112 #include "asm/bug.h"
13+#include "debugfs.h"
14+#include "event-parse.h"
1215 #include "evsel.h"
1316 #include "evlist.h"
1417 #include "util.h"
@@ -16,9 +19,10 @@ · 1 unmodified line
1619 #include "thread_map.h"
1720 #include "target.h"
1821 #include "../../../include/linux/hw_breakpoint.h"
22+#include "../../include/linux/perf_event.h"
23+#include "perf_regs.h"
1924
2025 #define FD(e, x, y) (*(int *)xyarray__entry(e->fd, x, y))
21−#define GROUP_FD(group_fd, cpu) (*(int *)xyarray__entry(group_fd, cpu, 0))
2226
2327 static int __perf_evsel__sample_size(u64 sample_type)
2428 {
@@ -66,7 +70,80 @@ struct perf_evsel *perf_evsel__new(struct perf_event_attr *attr, int idx) · 41 unmodified lines
6670 return evsel;
6771 }
6872
69−static const char *perf_evsel__hw_names[PERF_COUNT_HW_MAX] = {
73+struct event_format *event_format__new(const char *sys, const char *name)
74+{
75+ int fd, n;
76+ char *filename;
77+ void *bf = NULL, *nbf;
78+ size_t size = 0, alloc_size = 0;
79+ struct event_format *format = NULL;
80+
81+ if (asprintf(&filename, "%s/%s/%s/format", tracing_events_path, sys, name) < 0)
82+ goto out;
83+
84+ fd = open(filename, O_RDONLY);
85+ if (fd < 0)
86+ goto out_free_filename;
87+
88+ do {
89+ if (size == alloc_size) {
90+ alloc_size += BUFSIZ;
91+ nbf = realloc(bf, alloc_size);
92+ if (nbf == NULL)
93+ goto out_free_bf;
94+ bf = nbf;
95+ }
96+
97+ n = read(fd, bf + size, BUFSIZ);
98+ if (n < 0)
99+ goto out_free_bf;
100+ size += n;
101+ } while (n > 0);
102+
103+ pevent_parse_format(&format, bf, size, sys);
104+
105+out_free_bf:
106+ free(bf);
107+ close(fd);
108+out_free_filename:
109+ free(filename);
110+out:
111+ return format;
112+}
113+
114+struct perf_evsel *perf_evsel__newtp(const char *sys, const char *name, int idx)
115+{
116+ struct perf_evsel *evsel = zalloc(sizeof(*evsel));
117+
118+ if (evsel != NULL) {
119+ struct perf_event_attr attr = {
120+ .type = PERF_TYPE_TRACEPOINT,
121+ .sample_type = (PERF_SAMPLE_RAW | PERF_SAMPLE_TIME |
122+ PERF_SAMPLE_CPU | PERF_SAMPLE_PERIOD),
123+ };
124+
125+ if (asprintf(&evsel->name, "%s:%s", sys, name) < 0)
126+ goto out_free;
127+
128+ evsel->tp_format = event_format__new(sys, name);
129+ if (evsel->tp_format == NULL)
130+ goto out_free;
131+
132+ event_attr_init(&attr);
133+ attr.config = evsel->tp_format->id;
134+ attr.sample_period = 1;
135+ perf_evsel__init(evsel, &attr, idx);
136+ }
137+
138+ return evsel;
139+
140+out_free:
141+ free(evsel->name);
142+ free(evsel);
143+ return NULL;
144+}
145+
146+const char *perf_evsel__hw_names[PERF_COUNT_HW_MAX] = {
70147 "cycles",
71148 "instructions",
72149 "cache-references",
@@ -129,12 +206,12 @@ static int perf_evsel__hw_name(struct perf_evsel *evsel, char *bf, size_t size) · 56 unmodified lines
129206 return r + perf_evsel__add_modifiers(evsel, bf + r, size - r);
130207 }
131208
132−static const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX] = {
209+const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX] = {
133210 "cpu-clock",
134211 "task-clock",
135212 "page-faults",
136213 "context-switches",
137− "CPU-migrations",
214+ "cpu-migrations",
138215 "minor-faults",
139216 "major-faults",
140217 "alignment-faults",
@@ -317,7 +394,8 @@ const char *perf_evsel__name(struct perf_evsel *evsel) · 176 unmodified lines
317394 break;
318395
319396 default:
320− scnprintf(bf, sizeof(bf), "%s", "unknown attr type");
397+ scnprintf(bf, sizeof(bf), "unknown attr type: %d",
398+ evsel->attr.type);
321399 break;
322400 }
323401
@@ -367,9 +445,18 @@ void perf_evsel__config(struct perf_evsel *evsel, struct perf_record_opts *opts, · 43 unmodified lines
367445 attr->mmap_data = track;
368446 }
369447
370− if (opts->call_graph)
448+ if (opts->call_graph) {
371449 attr->sample_type |= PERF_SAMPLE_CALLCHAIN;
372450
451+ if (opts->call_graph == CALLCHAIN_DWARF) {
452+ attr->sample_type |= PERF_SAMPLE_REGS_USER |
453+ PERF_SAMPLE_STACK_USER;
454+ attr->sample_regs_user = PERF_REGS_MASK;
455+ attr->sample_stack_user = opts->stack_dump_size;
456+ attr->exclude_callchain_user = 1;
457+ }
458+ }
459+
373460 if (perf_target__has_cpu(&opts->target))
374461 attr->sample_type |= PERF_SAMPLE_CPU;
375462
@@ -421,6 +508,24 @@ int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads) · 45 unmodified lines
421508 return evsel->fd != NULL ? 0 : -ENOMEM;
422509 }
423510
511+int perf_evsel__set_filter(struct perf_evsel *evsel, int ncpus, int nthreads,
512+ const char *filter)
513+{
514+ int cpu, thread;
515+
516+ for (cpu = 0; cpu < ncpus; cpu++) {
517+ for (thread = 0; thread < nthreads; thread++) {
518+ int fd = FD(evsel, cpu, thread),
519+ err = ioctl(fd, PERF_EVENT_IOC_SET_FILTER, filter);
520+
521+ if (err)
522+ return err;
523+ }
524+ }
525+
526+ return 0;
527+}
528+
424529 int perf_evsel__alloc_id(struct perf_evsel *evsel, int ncpus, int nthreads)
425530 {
426531 evsel->sample_id = xyarray__new(ncpus, nthreads, sizeof(struct perf_sample_id));
@@ -481,6 +586,9 @@ void perf_evsel__delete(struct perf_evsel *evsel) · 54 unmodified lines
481586 {
482587 perf_evsel__exit(evsel);
483588 close_cgroup(evsel->cgrp);
589+ free(evsel->group_name);
590+ if (evsel->tp_format)
591+ pevent_free_format(evsel->tp_format);
484592 free(evsel->name);
485593 free(evsel);
486594 }
@@ -556,9 +664,28 @@ int __perf_evsel__read(struct perf_evsel *evsel, · 69 unmodified lines
556664 return 0;
557665 }
558666
667+static int get_group_fd(struct perf_evsel *evsel, int cpu, int thread)
668+{
669+ struct perf_evsel *leader = evsel->leader;
670+ int fd;
671+
672+ if (!leader)
673+ return -1;
674+
675+ /*
676+ * Leader must be already processed/open,
677+ * if not it's a bug.
678+ */
679+ BUG_ON(!leader->fd);
680+
681+ fd = FD(leader, cpu, thread);
682+ BUG_ON(fd == -1);
683+
684+ return fd;
685+}
686+
559687 static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
560− struct thread_map *threads, bool group,
561− struct xyarray *group_fds)
688+ struct thread_map *threads)
562689 {
563690 int cpu, thread;
564691 unsigned long flags = 0;
@@ -574,13 +701,15 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus, · 9 unmodified lines
574701 }
575702
576703 for (cpu = 0; cpu < cpus->nr; cpu++) {
577− int group_fd = group_fds ? GROUP_FD(group_fds, cpu) : -1;
578704
579705 for (thread = 0; thread < threads->nr; thread++) {
706+ int group_fd;
580707
581708 if (!evsel->cgrp)
582709 pid = threads->map[thread];
583710
711+ group_fd = get_group_fd(evsel, cpu, thread);
712+
584713 FD(evsel, cpu, thread) = sys_perf_event_open(&evsel->attr,
585714 pid,
586715 cpus->map[cpu],
@@ -589,9 +718,6 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus, · 2 unmodified lines
589718 err = -errno;
590719 goto out_close;
591720 }
592−
593− if (group && group_fd == -1)
594− group_fd = FD(evsel, cpu, thread);
595721 }
596722 }
597723
@@ -635,8 +761,7 @@ static struct { · 37 unmodified lines
635761 };
636762
637763 int perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
638− struct thread_map *threads, bool group,
639− struct xyarray *group_fd)
764+ struct thread_map *threads)
640765 {
641766 if (cpus == NULL) {
642767 /* Work around old compiler warnings about strict aliasing */
@@ -646,30 +771,28 @@ int perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus, · 3 unmodified lines
646771 if (threads == NULL)
647772 threads = &empty_thread_map.map;
648773
649− return __perf_evsel__open(evsel, cpus, threads, group, group_fd);
774+ return __perf_evsel__open(evsel, cpus, threads);
650775 }
651776
652777 int perf_evsel__open_per_cpu(struct perf_evsel *evsel,
653− struct cpu_map *cpus, bool group,
654− struct xyarray *group_fd)
778+ struct cpu_map *cpus)
655779 {
656− return __perf_evsel__open(evsel, cpus, &empty_thread_map.map, group,
657− group_fd);
780+ return __perf_evsel__open(evsel, cpus, &empty_thread_map.map);
658781 }
659782
660783 int perf_evsel__open_per_thread(struct perf_evsel *evsel,
661− struct thread_map *threads, bool group,
662− struct xyarray *group_fd)
784+ struct thread_map *threads)
663785 {
664− return __perf_evsel__open(evsel, &empty_cpu_map.map, threads, group,
665− group_fd);
786+ return __perf_evsel__open(evsel, &empty_cpu_map.map, threads);
666787 }
667788
668−static int perf_event__parse_id_sample(const union perf_event *event, u64 type,
669− struct perf_sample *sample,
670− bool swapped)
789+static int perf_evsel__parse_id_sample(const struct perf_evsel *evsel,
790+ const union perf_event *event,
791+ struct perf_sample *sample)
671792 {
793+ u64 type = evsel->attr.sample_type;
672794 const u64 *array = event->sample.array;
795+ bool swapped = evsel->needs_swap;
673796 union u64_swap u;
674797
675798 array += ((event->header.size -
@@ -730,9 +853,11 @@ static bool sample_overlap(const union perf_event *event, · 54 unmodified lines
730853 }
731854
732855 int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event,
733− struct perf_sample *data, bool swapped)
856+ struct perf_sample *data)
734857 {
735858 u64 type = evsel->attr.sample_type;
859+ u64 regs_user = evsel->attr.sample_regs_user;
860+ bool swapped = evsel->needs_swap;
736861 const u64 *array;
737862
738863 /*
@@ -749,7 +874,7 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event, · 10 unmodified lines
749874 if (event->header.type != PERF_RECORD_SAMPLE) {
750875 if (!evsel->attr.sample_id_all)
751876 return 0;
752− return perf_event__parse_id_sample(event, type, data, swapped);
877+ return perf_evsel__parse_id_sample(evsel, event, data);
753878 }
754879
755880 array = event->sample.array;
@@ -869,6 +994,32 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event, · 113 unmodified lines
869994 sz /= sizeof(u64);
870995 array += sz;
871996 }
997+
998+ if (type & PERF_SAMPLE_REGS_USER) {
999+ /* First u64 tells us if we have any regs in sample. */
1000+ u64 avail = *array++;
1001+
1002+ if (avail) {
1003+ data->user_regs.regs = (u64 *)array;
1004+ array += hweight_long(regs_user);
1005+ }
1006+ }
1007+
1008+ if (type & PERF_SAMPLE_STACK_USER) {
1009+ u64 size = *array++;
1010+
1011+ data->user_stack.offset = ((char *)(array - 1)
1012+ - (char *) event);
1013+
1014+ if (!size) {
1015+ data->user_stack.size = 0;
1016+ } else {
1017+ data->user_stack.data = (char *)array;
1018+ array += size / sizeof(*array);
1019+ data->user_stack.size = *array;
1020+ }
1021+ }
1022+
8721023 return 0;
8731024 }
8741025
@@ -947,3 +1098,72 @@ int perf_event__synthesize_sample(union perf_event *event, u64 type, · 72 unmodified lines
9471098
9481099 return 0;
9491100 }
1101+
1102+struct format_field *perf_evsel__field(struct perf_evsel *evsel, const char *name)
1103+{
1104+ return pevent_find_field(evsel->tp_format, name);
1105+}
1106+
1107+void *perf_evsel__rawptr(struct perf_evsel *evsel, struct perf_sample *sample,
1108+ const char *name)
1109+{
1110+ struct format_field *field = perf_evsel__field(evsel, name);
1111+ int offset;
1112+
1113+ if (!field)
1114+ return NULL;
1115+
1116+ offset = field->offset;
1117+
1118+ if (field->flags & FIELD_IS_DYNAMIC) {
1119+ offset = *(int *)(sample->raw_data + field->offset);
1120+ offset &= 0xffff;
1121+ }
1122+
1123+ return sample->raw_data + offset;
1124+}
1125+
1126+u64 perf_evsel__intval(struct perf_evsel *evsel, struct perf_sample *sample,
1127+ const char *name)
1128+{
1129+ struct format_field *field = perf_evsel__field(evsel, name);
1130+ void *ptr;
1131+ u64 value;
1132+
1133+ if (!field)
1134+ return 0;
1135+
1136+ ptr = sample->raw_data + field->offset;
1137+
1138+ switch (field->size) {
1139+ case 1:
1140+ return *(u8 *)ptr;
1141+ case 2:
1142+ value = *(u16 *)ptr;
1143+ break;
1144+ case 4:
1145+ value = *(u32 *)ptr;
1146+ break;
1147+ case 8:
1148+ value = *(u64 *)ptr;
1149+ break;
1150+ default:
1151+ return 0;
1152+ }
1153+
1154+ if (!evsel->needs_swap)
1155+ return value;
1156+
1157+ switch (field->size) {
1158+ case 2:
1159+ return bswap_16(value);
1160+ case 4:
1161+ return bswap_32(value);
1162+ case 8:
1163+ return bswap_64(value);
1164+ default:
1165+ return 0;
1166+ }
1167+
1168+ return 0;
1169+}
tools/perf/util/evsel.h+53−10
@@ -53,9 +53,10 @@ struct perf_evsel { · 52 unmodified lines
5353 u64 *id;
5454 struct perf_counts *counts;
5555 int idx;
56− int ids;
56+ u32 ids;
5757 struct hists hists;
5858 char *name;
59+ struct event_format *tp_format;
5960 union {
6061 void *priv;
6162 off_t id_offset;
@@ -65,8 +66,14 @@ struct perf_evsel { · 3 unmodified lines
6566 void *func;
6667 void *data;
6768 } handler;
69+ struct cpu_map *cpus;
6870 unsigned int sample_size;
6971 bool supported;
72+ bool needs_swap;
73+ /* parse modifier helper */
74+ int exclude_GH;
75+ struct perf_evsel *leader;
76+ char *group_name;
7077 };
7178
7279 struct cpu_map;
@@ -75,6 +82,10 @@ struct perf_evlist; · 2 unmodified lines
7582 struct perf_record_opts;
7683
7784 struct perf_evsel *perf_evsel__new(struct perf_event_attr *attr, int idx);
85+struct perf_evsel *perf_evsel__newtp(const char *sys, const char *name, int idx);
86+
87+struct event_format *event_format__new(const char *sys, const char *name);
88+
7889 void perf_evsel__init(struct perf_evsel *evsel,
7990 struct perf_event_attr *attr, int idx);
8091 void perf_evsel__exit(struct perf_evsel *evsel);
@@ -92,8 +103,10 @@ extern const char *perf_evsel__hw_cache[PERF_COUNT_HW_CACHE_MAX] · 11 unmodified lines
92103 [PERF_EVSEL__MAX_ALIASES];
93104 extern const char *perf_evsel__hw_cache_op[PERF_COUNT_HW_CACHE_OP_MAX]
94105 [PERF_EVSEL__MAX_ALIASES];
95−const char *perf_evsel__hw_cache_result[PERF_COUNT_HW_CACHE_RESULT_MAX]
96− [PERF_EVSEL__MAX_ALIASES];
106+extern const char *perf_evsel__hw_cache_result[PERF_COUNT_HW_CACHE_RESULT_MAX]
107+ [PERF_EVSEL__MAX_ALIASES];
108+extern const char *perf_evsel__hw_names[PERF_COUNT_HW_MAX];
109+extern const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX];
97110 int __perf_evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result,
98111 char *bf, size_t size);
99112 const char *perf_evsel__name(struct perf_evsel *evsel);
@@ -105,21 +118,46 @@ void perf_evsel__free_fd(struct perf_evsel *evsel); · 5 unmodified lines
105118 void perf_evsel__free_id(struct perf_evsel *evsel);
106119 void perf_evsel__close_fd(struct perf_evsel *evsel, int ncpus, int nthreads);
107120
121+int perf_evsel__set_filter(struct perf_evsel *evsel, int ncpus, int nthreads,
122+ const char *filter);
123+
108124 int perf_evsel__open_per_cpu(struct perf_evsel *evsel,
109− struct cpu_map *cpus, bool group,
110− struct xyarray *group_fds);
125+ struct cpu_map *cpus);
111126 int perf_evsel__open_per_thread(struct perf_evsel *evsel,
112− struct thread_map *threads, bool group,
113− struct xyarray *group_fds);
127+ struct thread_map *threads);
114128 int perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
115− struct thread_map *threads, bool group,
116− struct xyarray *group_fds);
129+ struct thread_map *threads);
117130 void perf_evsel__close(struct perf_evsel *evsel, int ncpus, int nthreads);
118131
132+struct perf_sample;
133+
134+void *perf_evsel__rawptr(struct perf_evsel *evsel, struct perf_sample *sample,
135+ const char *name);
136+u64 perf_evsel__intval(struct perf_evsel *evsel, struct perf_sample *sample,
137+ const char *name);
138+
139+static inline char *perf_evsel__strval(struct perf_evsel *evsel,
140+ struct perf_sample *sample,
141+ const char *name)
142+{
143+ return perf_evsel__rawptr(evsel, sample, name);
144+}
145+
146+struct format_field;
147+
148+struct format_field *perf_evsel__field(struct perf_evsel *evsel, const char *name);
149+
119150 #define perf_evsel__match(evsel, t, c) \
120151 (evsel->attr.type == PERF_TYPE_##t && \
121152 evsel->attr.config == PERF_COUNT_##c)
122153
154+static inline bool perf_evsel__match2(struct perf_evsel *e1,
155+ struct perf_evsel *e2)
156+{
157+ return (e1->attr.type == e2->attr.type) &&
158+ (e1->attr.config == e2->attr.config);
159+}
160+
123161 int __perf_evsel__read_on_cpu(struct perf_evsel *evsel,
124162 int cpu, int thread, bool scale);
125163
@@ -181,5 +219,10 @@ static inline int perf_evsel__read_scaled(struct perf_evsel *evsel, · 55 unmodified lines
181219 void hists__init(struct hists *hists);
182220
183221 int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event,
184− struct perf_sample *sample, bool swapped);
222+ struct perf_sample *sample);
223+
224+static inline struct perf_evsel *perf_evsel__next(struct perf_evsel *evsel)
225+{
226+ return list_entry(evsel->node.next, struct perf_evsel, node);
227+}
185228 #endif /* __PERF_EVSEL_H */
tools/perf/util/generate-cmdlist.sh+15−0
@@ -21,4 +21,19 @@ do · 20 unmodified lines
2121 p
2222 }' "Documentation/perf-$cmd.txt"
2323 done
24+
25+echo "#ifndef NO_LIBELF_SUPPORT"
26+sed -n -e 's/^perf-\([^ ]*\)[ ].* full.*/\1/p' command-list.txt |
27+sort |
28+while read cmd
29+do
30+ sed -n '
31+ /^NAME/,/perf-'"$cmd"'/H
32+ ${
33+ x
34+ s/.*perf-'"$cmd"' - \(.*\)/ {"'"$cmd"'", "\1"},/
35+ p
36+ }' "Documentation/perf-$cmd.txt"
37+done
38+echo "#endif /* NO_LIBELF_SUPPORT */"
2439 echo "};"
tools/perf/util/header.c+762−301
@@ -20,11 +20,14 @@ · 19 unmodified lines
2020 #include "symbol.h"
2121 #include "debug.h"
2222 #include "cpumap.h"
23+#include "pmu.h"
24+#include "vdso.h"
25+#include "strbuf.h"
2326
2427 static bool no_buildid_cache = false;
2528
26−static int event_count;
27−static struct perf_trace_event_type *events;
29+static int trace_event_count;
30+static struct perf_trace_event_type *trace_events;
2831
2932 static u32 header_argc;
3033 static const char **header_argv;
@@ -36,24 +39,24 @@ int perf_header__push_event(u64 id, const char *name) · 5 unmodified lines
3639 if (strlen(name) > MAX_EVENT_NAME)
3740 pr_warning("Event %s will be truncated\n", name);
3841
39− nevents = realloc(events, (event_count + 1) * sizeof(*events));
42+ nevents = realloc(trace_events, (trace_event_count + 1) * sizeof(*trace_events));
4043 if (nevents == NULL)
4144 return -ENOMEM;
42− events = nevents;
45+ trace_events = nevents;
4346
44− memset(&events[event_count], 0, sizeof(struct perf_trace_event_type));
45− events[event_count].event_id = id;
46− strncpy(events[event_count].name, name, MAX_EVENT_NAME - 1);
47− event_count++;
47+ memset(&trace_events[trace_event_count], 0, sizeof(struct perf_trace_event_type));
48+ trace_events[trace_event_count].event_id = id;
49+ strncpy(trace_events[trace_event_count].name, name, MAX_EVENT_NAME - 1);
50+ trace_event_count++;
4851 return 0;
4952 }
5053
5154 char *perf_header__find_event(u64 id)
5255 {
5356 int i;
54− for (i = 0 ; i < event_count; i++) {
55− if (events[i].event_id == id)
56− return events[i].name;
57+ for (i = 0 ; i < trace_event_count; i++) {
58+ if (trace_events[i].event_id == id)
59+ return trace_events[i].name;
5760 }
5861 return NULL;
5962 }
@@ -128,7 +131,7 @@ static int do_write_string(int fd, const char *str) · 68 unmodified lines
128131 int ret;
129132
130133 olen = strlen(str) + 1;
131− len = ALIGN(olen, NAME_ALIGN);
134+ len = PERF_ALIGN(olen, NAME_ALIGN);
132135
133136 /* write len, incl. \0 */
134137 ret = do_write(fd, &len, sizeof(len));
@@ -206,6 +209,29 @@ perf_header__set_cmdline(int argc, const char **argv) · 71 unmodified lines
206209 continue; \
207210 else
208211
212+static int write_buildid(char *name, size_t name_len, u8 *build_id,
213+ pid_t pid, u16 misc, int fd)
214+{
215+ int err;
216+ struct build_id_event b;
217+ size_t len;
218+
219+ len = name_len + 1;
220+ len = PERF_ALIGN(len, NAME_ALIGN);
221+
222+ memset(&b, 0, sizeof(b));
223+ memcpy(&b.build_id, build_id, BUILD_ID_SIZE);
224+ b.pid = pid;
225+ b.header.misc = misc;
226+ b.header.size = sizeof(b) + len;
227+
228+ err = do_write(fd, &b, sizeof(b));
229+ if (err < 0)
230+ return err;
231+
232+ return write_padded(fd, name, name_len + 1, len);
233+}
234+
209235 static int __dsos__write_buildid_table(struct list_head *head, pid_t pid,
210236 u16 misc, int fd)
211237 {
@@ -213,24 +239,23 @@ static int __dsos__write_buildid_table(struct list_head *head, pid_t pid, · 1 unmodified line
213239
214240 dsos__for_each_with_build_id(pos, head) {
215241 int err;
216− struct build_id_event b;
217− size_t len;
242+ char *name;
243+ size_t name_len;
218244
219245 if (!pos->hit)
220246 continue;
221− len = pos->long_name_len + 1;
222− len = ALIGN(len, NAME_ALIGN);
223− memset(&b, 0, sizeof(b));
224− memcpy(&b.build_id, pos->build_id, sizeof(pos->build_id));
225− b.pid = pid;
226− b.header.misc = misc;
227− b.header.size = sizeof(b) + len;
228− err = do_write(fd, &b, sizeof(b));
229− if (err < 0)
230− return err;
231− err = write_padded(fd, pos->long_name,
232− pos->long_name_len + 1, len);
233− if (err < 0)
247+
248+ if (is_vdso_map(pos->short_name)) {
249+ name = (char *) VDSO__MAP_NAME;
250+ name_len = sizeof(VDSO__MAP_NAME) + 1;
251+ } else {
252+ name = pos->long_name;
253+ name_len = pos->long_name_len + 1;
254+ }
255+
256+ err = write_buildid(name, name_len, pos->build_id,
257+ pid, misc, fd);
258+ if (err)
234259 return err;
235260 }
236261
@@ -276,19 +301,20 @@ static int dsos__write_buildid_table(struct perf_header *header, int fd) · 39 unmodified lines
276301 }
277302
278303 int build_id_cache__add_s(const char *sbuild_id, const char *debugdir,
279− const char *name, bool is_kallsyms)
304+ const char *name, bool is_kallsyms, bool is_vdso)
280305 {
281306 const size_t size = PATH_MAX;
282307 char *realname, *filename = zalloc(size),
283308 *linkname = zalloc(size), *targetname;
284309 int len, err = -1;
310+ bool slash = is_kallsyms || is_vdso;
285311
286312 if (is_kallsyms) {
287313 if (symbol_conf.kptr_restrict) {
288314 pr_debug("Not caching a kptr_restrict'ed /proc/kallsyms\n");
289315 return 0;
290316 }
291− realname = (char *)name;
317+ realname = (char *) name;
292318 } else
293319 realname = realpath(name, NULL);
294320
@@ -296,7 +322,8 @@ int build_id_cache__add_s(const char *sbuild_id, const char *debugdir, · 1 unmodified line
296322 goto out_free;
297323
298324 len = scnprintf(filename, size, "%s%s%s",
299− debugdir, is_kallsyms ? "/" : "", realname);
325+ debugdir, slash ? "/" : "",
326+ is_vdso ? VDSO__MAP_NAME : realname);
300327 if (mkdir_p(filename, 0755))
301328 goto out_free;
302329
@@ -332,13 +359,14 @@ out_free: · 29 unmodified lines
332359
333360 static int build_id_cache__add_b(const u8 *build_id, size_t build_id_size,
334361 const char *name, const char *debugdir,
335− bool is_kallsyms)
362+ bool is_kallsyms, bool is_vdso)
336363 {
337364 char sbuild_id[BUILD_ID_SIZE * 2 + 1];
338365
339366 build_id__sprintf(build_id, build_id_size, sbuild_id);
340367
341− return build_id_cache__add_s(sbuild_id, debugdir, name, is_kallsyms);
368+ return build_id_cache__add_s(sbuild_id, debugdir, name,
369+ is_kallsyms, is_vdso);
342370 }
343371
344372 int build_id_cache__remove_s(const char *sbuild_id, const char *debugdir)
@@ -382,9 +410,11 @@ out_free: · 37 unmodified lines
382410 static int dso__cache_build_id(struct dso *dso, const char *debugdir)
383411 {
384412 bool is_kallsyms = dso->kernel && dso->long_name[0] != '/';
413+ bool is_vdso = is_vdso_map(dso->short_name);
385414
386415 return build_id_cache__add_b(dso->build_id, sizeof(dso->build_id),
387− dso->long_name, debugdir, is_kallsyms);
416+ dso->long_name, debugdir,
417+ is_kallsyms, is_vdso);
388418 }
389419
390420 static int __dsos__cache_build_ids(struct list_head *head, const char *debugdir)
@@ -446,7 +476,7 @@ static bool perf_session__read_build_ids(struct perf_session *session, bool with · 55 unmodified lines
446476 return ret;
447477 }
448478
449−static int write_tracing_data(int fd, struct perf_header *h __used,
479+static int write_tracing_data(int fd, struct perf_header *h __maybe_unused,
450480 struct perf_evlist *evlist)
451481 {
452482 return read_tracing_data(fd, &evlist->entries);
@@ -454,7 +484,7 @@ static int write_tracing_data(int fd, struct perf_header *h __used, · 1 unmodified line
454484
455485
456486 static int write_build_id(int fd, struct perf_header *h,
457− struct perf_evlist *evlist __used)
487+ struct perf_evlist *evlist __maybe_unused)
458488 {
459489 struct perf_session *session;
460490 int err;
@@ -475,8 +505,8 @@ static int write_build_id(int fd, struct perf_header *h, · 14 unmodified lines
475505 return 0;
476506 }
477507
478−static int write_hostname(int fd, struct perf_header *h __used,
479− struct perf_evlist *evlist __used)
508+static int write_hostname(int fd, struct perf_header *h __maybe_unused,
509+ struct perf_evlist *evlist __maybe_unused)
480510 {
481511 struct utsname uts;
482512 int ret;
@@ -488,8 +518,8 @@ static int write_hostname(int fd, struct perf_header *h __used, · 5 unmodified lines
488518 return do_write_string(fd, uts.nodename);
489519 }
490520
491−static int write_osrelease(int fd, struct perf_header *h __used,
492− struct perf_evlist *evlist __used)
521+static int write_osrelease(int fd, struct perf_header *h __maybe_unused,
522+ struct perf_evlist *evlist __maybe_unused)
493523 {
494524 struct utsname uts;
495525 int ret;
@@ -501,8 +531,8 @@ static int write_osrelease(int fd, struct perf_header *h __used, · 5 unmodified lines
501531 return do_write_string(fd, uts.release);
502532 }
503533
504−static int write_arch(int fd, struct perf_header *h __used,
505− struct perf_evlist *evlist __used)
534+static int write_arch(int fd, struct perf_header *h __maybe_unused,
535+ struct perf_evlist *evlist __maybe_unused)
506536 {
507537 struct utsname uts;
508538 int ret;
@@ -514,14 +544,14 @@ static int write_arch(int fd, struct perf_header *h __used, · 5 unmodified lines
514544 return do_write_string(fd, uts.machine);
515545 }
516546
517−static int write_version(int fd, struct perf_header *h __used,
518− struct perf_evlist *evlist __used)
547+static int write_version(int fd, struct perf_header *h __maybe_unused,
548+ struct perf_evlist *evlist __maybe_unused)
519549 {
520550 return do_write_string(fd, perf_version_string);
521551 }
522552
523−static int write_cpudesc(int fd, struct perf_header *h __used,
524− struct perf_evlist *evlist __used)
553+static int write_cpudesc(int fd, struct perf_header *h __maybe_unused,
554+ struct perf_evlist *evlist __maybe_unused)
525555 {
526556 #ifndef CPUINFO_PROC
527557 #define CPUINFO_PROC NULL
@@ -579,8 +609,8 @@ done: · 51 unmodified lines
579609 return ret;
580610 }
581611
582−static int write_nrcpus(int fd, struct perf_header *h __used,
583− struct perf_evlist *evlist __used)
612+static int write_nrcpus(int fd, struct perf_header *h __maybe_unused,
613+ struct perf_evlist *evlist __maybe_unused)
584614 {
585615 long nr;
586616 u32 nrc, nra;
@@ -605,15 +635,14 @@ static int write_nrcpus(int fd, struct perf_header *h __used, · 18 unmodified lines
605635 return do_write(fd, &nra, sizeof(nra));
606636 }
607637
608−static int write_event_desc(int fd, struct perf_header *h __used,
638+static int write_event_desc(int fd, struct perf_header *h __maybe_unused,
609639 struct perf_evlist *evlist)
610640 {
611− struct perf_evsel *attr;
612− u32 nre = 0, nri, sz;
641+ struct perf_evsel *evsel;
642+ u32 nre, nri, sz;
613643 int ret;
614644
615− list_for_each_entry(attr, &evlist->entries, node)
616− nre++;
645+ nre = evlist->nr_entries;
617646
618647 /*
619648 * write number of events
@@ -625,14 +654,14 @@ static int write_event_desc(int fd, struct perf_header *h __used, · 5 unmodified lines
625654 /*
626655 * size of perf_event_attr struct
627656 */
628− sz = (u32)sizeof(attr->attr);
657+ sz = (u32)sizeof(evsel->attr);
629658 ret = do_write(fd, &sz, sizeof(sz));
630659 if (ret < 0)
631660 return ret;
632661
633− list_for_each_entry(attr, &evlist->entries, node) {
662+ list_for_each_entry(evsel, &evlist->entries, node) {
634663
635− ret = do_write(fd, &attr->attr, sz);
664+ ret = do_write(fd, &evsel->attr, sz);
636665 if (ret < 0)
637666 return ret;
638667 /*
@@ -642,7 +671,7 @@ static int write_event_desc(int fd, struct perf_header *h __used, · 3 unmodified lines
642671 * copy into an nri to be independent of the
643672 * type of ids,
644673 */
645− nri = attr->ids;
674+ nri = evsel->ids;
646675 ret = do_write(fd, &nri, sizeof(nri));
647676 if (ret < 0)
648677 return ret;
@@ -650,21 +679,21 @@ static int write_event_desc(int fd, struct perf_header *h __used, · 1 unmodified line
650679 /*
651680 * write event string as passed on cmdline
652681 */
653− ret = do_write_string(fd, perf_evsel__name(attr));
682+ ret = do_write_string(fd, perf_evsel__name(evsel));
654683 if (ret < 0)
655684 return ret;
656685 /*
657686 * write unique ids for this event
658687 */
659− ret = do_write(fd, attr->id, attr->ids * sizeof(u64));
688+ ret = do_write(fd, evsel->id, evsel->ids * sizeof(u64));
660689 if (ret < 0)
661690 return ret;
662691 }
663692 return 0;
664693 }
665694
666−static int write_cmdline(int fd, struct perf_header *h __used,
667− struct perf_evlist *evlist __used)
695+static int write_cmdline(int fd, struct perf_header *h __maybe_unused,
696+ struct perf_evlist *evlist __maybe_unused)
668697 {
669698 char buf[MAXPATHLEN];
670699 char proc[32];
@@ -832,8 +861,8 @@ static struct cpu_topo *build_cpu_topology(void) · 161 unmodified lines
832861 return tp;
833862 }
834863
835−static int write_cpu_topology(int fd, struct perf_header *h __used,
836− struct perf_evlist *evlist __used)
864+static int write_cpu_topology(int fd, struct perf_header *h __maybe_unused,
865+ struct perf_evlist *evlist __maybe_unused)
837866 {
838867 struct cpu_topo *tp;
839868 u32 i;
@@ -868,8 +897,8 @@ done: · 28 unmodified lines
868897
869898
870899
871−static int write_total_mem(int fd, struct perf_header *h __used,
872− struct perf_evlist *evlist __used)
900+static int write_total_mem(int fd, struct perf_header *h __maybe_unused,
901+ struct perf_evlist *evlist __maybe_unused)
873902 {
874903 char *buf = NULL;
875904 FILE *fp;
@@ -954,8 +983,8 @@ done: · 78 unmodified lines
954983 return ret;
955984 }
956985
957−static int write_numa_topology(int fd, struct perf_header *h __used,
958− struct perf_evlist *evlist __used)
986+static int write_numa_topology(int fd, struct perf_header *h __maybe_unused,
987+ struct perf_evlist *evlist __maybe_unused)
959988 {
960989 char *buf = NULL;
961990 size_t len = 0;
@@ -1003,17 +1032,57 @@ done: · 41 unmodified lines
10031032 return ret;
10041033 }
10051034
1035+/*
1036+ * File format:
1037+ *
1038+ * struct pmu_mappings {
1039+ * u32 pmu_num;
1040+ * struct pmu_map {
1041+ * u32 type;
1042+ * char name[];
1043+ * }[pmu_num];
1044+ * };
1045+ */
1046+
1047+static int write_pmu_mappings(int fd, struct perf_header *h __maybe_unused,
1048+ struct perf_evlist *evlist __maybe_unused)
1049+{
1050+ struct perf_pmu *pmu = NULL;
1051+ off_t offset = lseek(fd, 0, SEEK_CUR);
1052+ __u32 pmu_num = 0;
1053+
1054+ /* write real pmu_num later */
1055+ do_write(fd, &pmu_num, sizeof(pmu_num));
1056+
1057+ while ((pmu = perf_pmu__scan(pmu))) {
1058+ if (!pmu->name)
1059+ continue;
1060+ pmu_num++;
1061+ do_write(fd, &pmu->type, sizeof(pmu->type));
1062+ do_write_string(fd, pmu->name);
1063+ }
1064+
1065+ if (pwrite(fd, &pmu_num, sizeof(pmu_num), offset) != sizeof(pmu_num)) {
1066+ /* discard all */
1067+ lseek(fd, offset, SEEK_SET);
1068+ return -1;
1069+ }
1070+
1071+ return 0;
1072+}
1073+
10061074 /*
10071075 * default get_cpuid(): nothing gets recorded
10081076 * actual implementation must be in arch/$(ARCH)/util/header.c
10091077 */
1010−int __attribute__((weak)) get_cpuid(char *buffer __used, size_t sz __used)
1078+int __attribute__ ((weak)) get_cpuid(char *buffer __maybe_unused,
1079+ size_t sz __maybe_unused)
10111080 {
10121081 return -1;
10131082 }
10141083
1015−static int write_cpuid(int fd, struct perf_header *h __used,
1016− struct perf_evlist *evlist __used)
1084+static int write_cpuid(int fd, struct perf_header *h __maybe_unused,
1085+ struct perf_evlist *evlist __maybe_unused)
10171086 {
10181087 char buffer[64];
10191088 int ret;
@@ -1027,133 +1096,113 @@ write_it: · 7 unmodified lines
10271096 return do_write_string(fd, buffer);
10281097 }
10291098
1030−static int write_branch_stack(int fd __used, struct perf_header *h __used,
1031− struct perf_evlist *evlist __used)
1099+static int write_branch_stack(int fd __maybe_unused,
1100+ struct perf_header *h __maybe_unused,
1101+ struct perf_evlist *evlist __maybe_unused)
10321102 {
10331103 return 0;
10341104 }
10351105
1036−static void print_hostname(struct perf_header *ph, int fd, FILE *fp)
1106+static void print_hostname(struct perf_header *ph, int fd __maybe_unused,
1107+ FILE *fp)
10371108 {
1038− char *str = do_read_string(fd, ph);
1039− fprintf(fp, "# hostname : %s\n", str);
1040− free(str);
1109+ fprintf(fp, "# hostname : %s\n", ph->env.hostname);
10411110 }
10421111
1043−static void print_osrelease(struct perf_header *ph, int fd, FILE *fp)
1112+static void print_osrelease(struct perf_header *ph, int fd __maybe_unused,
1113+ FILE *fp)
10441114 {
1045− char *str = do_read_string(fd, ph);
1046− fprintf(fp, "# os release : %s\n", str);
1047− free(str);
1115+ fprintf(fp, "# os release : %s\n", ph->env.os_release);
10481116 }
10491117
1050−static void print_arch(struct perf_header *ph, int fd, FILE *fp)
1118+static void print_arch(struct perf_header *ph, int fd __maybe_unused, FILE *fp)
10511119 {
1052− char *str = do_read_string(fd, ph);
1053− fprintf(fp, "# arch : %s\n", str);
1054− free(str);
1120+ fprintf(fp, "# arch : %s\n", ph->env.arch);
10551121 }
10561122
1057−static void print_cpudesc(struct perf_header *ph, int fd, FILE *fp)
1123+static void print_cpudesc(struct perf_header *ph, int fd __maybe_unused,
1124+ FILE *fp)
10581125 {
1059− char *str = do_read_string(fd, ph);
1060− fprintf(fp, "# cpudesc : %s\n", str);
1061− free(str);
1126+ fprintf(fp, "# cpudesc : %s\n", ph->env.cpu_desc);
10621127 }
10631128
1064−static void print_nrcpus(struct perf_header *ph, int fd, FILE *fp)
1129+static void print_nrcpus(struct perf_header *ph, int fd __maybe_unused,
1130+ FILE *fp)
10651131 {
1066− ssize_t ret;
1067− u32 nr;
1068−
1069− ret = read(fd, &nr, sizeof(nr));
1070− if (ret != (ssize_t)sizeof(nr))
1071− nr = -1; /* interpreted as error */
1072−
1073− if (ph->needs_swap)
1074− nr = bswap_32(nr);
1075−
1076− fprintf(fp, "# nrcpus online : %u\n", nr);
1077−
1078− ret = read(fd, &nr, sizeof(nr));
1079− if (ret != (ssize_t)sizeof(nr))
1080− nr = -1; /* interpreted as error */
1081−
1082− if (ph->needs_swap)
1083− nr = bswap_32(nr);
1084−
1085− fprintf(fp, "# nrcpus avail : %u\n", nr);
1132+ fprintf(fp, "# nrcpus online : %u\n", ph->env.nr_cpus_online);
1133+ fprintf(fp, "# nrcpus avail : %u\n", ph->env.nr_cpus_avail);
10861134 }
10871135
1088−static void print_version(struct perf_header *ph, int fd, FILE *fp)
1136+static void print_version(struct perf_header *ph, int fd __maybe_unused,
1137+ FILE *fp)
10891138 {
1090− char *str = do_read_string(fd, ph);
1091− fprintf(fp, "# perf version : %s\n", str);
1092− free(str);
1139+ fprintf(fp, "# perf version : %s\n", ph->env.version);
10931140 }
10941141
1095−static void print_cmdline(struct perf_header *ph, int fd, FILE *fp)
1142+static void print_cmdline(struct perf_header *ph, int fd __maybe_unused,
1143+ FILE *fp)
10961144 {
1097− ssize_t ret;
1145+ int nr, i;
10981146 char *str;
1099− u32 nr, i;
1100−
1101− ret = read(fd, &nr, sizeof(nr));
1102− if (ret != (ssize_t)sizeof(nr))
1103− return;
11041147
1105− if (ph->needs_swap)
1106− nr = bswap_32(nr);
1148+ nr = ph->env.nr_cmdline;
1149+ str = ph->env.cmdline;
11071150
11081151 fprintf(fp, "# cmdline : ");
11091152
11101153 for (i = 0; i < nr; i++) {
1111− str = do_read_string(fd, ph);
11121154 fprintf(fp, "%s ", str);
1113− free(str);
1155+ str += strlen(str) + 1;
11141156 }
11151157 fputc('\n', fp);
11161158 }
11171159
1118−static void print_cpu_topology(struct perf_header *ph, int fd, FILE *fp)
1160+static void print_cpu_topology(struct perf_header *ph, int fd __maybe_unused,
1161+ FILE *fp)
11191162 {
1120− ssize_t ret;
1121− u32 nr, i;
1163+ int nr, i;
11221164 char *str;
11231165
1124− ret = read(fd, &nr, sizeof(nr));
1125− if (ret != (ssize_t)sizeof(nr))
1126− return;
1127−
1128− if (ph->needs_swap)
1129− nr = bswap_32(nr);
1166+ nr = ph->env.nr_sibling_cores;
1167+ str = ph->env.sibling_cores;
11301168
11311169 for (i = 0; i < nr; i++) {
1132− str = do_read_string(fd, ph);
11331170 fprintf(fp, "# sibling cores : %s\n", str);
1134− free(str);
1171+ str += strlen(str) + 1;
11351172 }
11361173
1137− ret = read(fd, &nr, sizeof(nr));
1138− if (ret != (ssize_t)sizeof(nr))
1139− return;
1140−
1141− if (ph->needs_swap)
1142− nr = bswap_32(nr);
1174+ nr = ph->env.nr_sibling_threads;
1175+ str = ph->env.sibling_threads;
11431176
11441177 for (i = 0; i < nr; i++) {
1145− str = do_read_string(fd, ph);
11461178 fprintf(fp, "# sibling threads : %s\n", str);
1147− free(str);
1179+ str += strlen(str) + 1;
11481180 }
11491181 }
11501182
1151−static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
1183+static void free_event_desc(struct perf_evsel *events)
1184+{
1185+ struct perf_evsel *evsel;
1186+
1187+ if (!events)
1188+ return;
1189+
1190+ for (evsel = events; evsel->attr.size; evsel++) {
1191+ if (evsel->name)
1192+ free(evsel->name);
1193+ if (evsel->id)
1194+ free(evsel->id);
1195+ }
1196+
1197+ free(events);
1198+}
1199+
1200+static struct perf_evsel *
1201+read_event_desc(struct perf_header *ph, int fd)
11521202 {
1153− struct perf_event_attr attr;
1154− uint64_t id;
1203+ struct perf_evsel *evsel, *events = NULL;
1204+ u64 *id;
11551205 void *buf = NULL;
1156− char *str;
11571206 u32 nre, sz, nr, i, j;
11581207 ssize_t ret;
11591208 size_t msz;
@@ -1173,18 +1222,22 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp) · 13 unmodified lines
11731222 if (ph->needs_swap)
11741223 sz = bswap_32(sz);
11751224
1176− memset(&attr, 0, sizeof(attr));
1177−
11781225 /* buffer to hold on file attr struct */
11791226 buf = malloc(sz);
11801227 if (!buf)
11811228 goto error;
11821229
1183− msz = sizeof(attr);
1230+ /* the last event terminates with evsel->attr.size == 0: */
1231+ events = calloc(nre + 1, sizeof(*events));
1232+ if (!events)
1233+ goto error;
1234+
1235+ msz = sizeof(evsel->attr);
11841236 if (sz < msz)
11851237 msz = sz;
11861238
1187− for (i = 0 ; i < nre; i++) {
1239+ for (i = 0, evsel = events; i < nre; evsel++, i++) {
1240+ evsel->idx = i;
11881241
11891242 /*
11901243 * must read entire on-file attr struct to
@@ -1197,146 +1250,188 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp) · 6 unmodified lines
11971250 if (ph->needs_swap)
11981251 perf_event__attr_swap(buf);
11991252
1200− memcpy(&attr, buf, msz);
1253+ memcpy(&evsel->attr, buf, msz);
12011254
12021255 ret = read(fd, &nr, sizeof(nr));
12031256 if (ret != (ssize_t)sizeof(nr))
12041257 goto error;
12051258
1206− if (ph->needs_swap)
1259+ if (ph->needs_swap) {
12071260 nr = bswap_32(nr);
1261+ evsel->needs_swap = true;
1262+ }
12081263
1209− str = do_read_string(fd, ph);
1210− fprintf(fp, "# event : name = %s, ", str);
1211− free(str);
1264+ evsel->name = do_read_string(fd, ph);
1265+
1266+ if (!nr)
1267+ continue;
1268+
1269+ id = calloc(nr, sizeof(*id));
1270+ if (!id)
1271+ goto error;
1272+ evsel->ids = nr;
1273+ evsel->id = id;
1274+
1275+ for (j = 0 ; j < nr; j++) {
1276+ ret = read(fd, id, sizeof(*id));
1277+ if (ret != (ssize_t)sizeof(*id))
1278+ goto error;
1279+ if (ph->needs_swap)
1280+ *id = bswap_64(*id);
1281+ id++;
1282+ }
1283+ }
1284+out:
1285+ if (buf)
1286+ free(buf);
1287+ return events;
1288+error:
1289+ if (events)
1290+ free_event_desc(events);
1291+ events = NULL;
1292+ goto out;
1293+}
1294+
1295+static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
1296+{
1297+ struct perf_evsel *evsel, *events = read_event_desc(ph, fd);
1298+ u32 j;
1299+ u64 *id;
1300+
1301+ if (!events) {
1302+ fprintf(fp, "# event desc: not available or unable to read\n");
1303+ return;
1304+ }
1305+
1306+ for (evsel = events; evsel->attr.size; evsel++) {
1307+ fprintf(fp, "# event : name = %s, ", evsel->name);
12121308
12131309 fprintf(fp, "type = %d, config = 0x%"PRIx64
12141310 ", config1 = 0x%"PRIx64", config2 = 0x%"PRIx64,
1215− attr.type,
1216− (u64)attr.config,
1217− (u64)attr.config1,
1218− (u64)attr.config2);
1311+ evsel->attr.type,
1312+ (u64)evsel->attr.config,
1313+ (u64)evsel->attr.config1,
1314+ (u64)evsel->attr.config2);
12191315
12201316 fprintf(fp, ", excl_usr = %d, excl_kern = %d",
1221− attr.exclude_user,
1222− attr.exclude_kernel);
1317+ evsel->attr.exclude_user,
1318+ evsel->attr.exclude_kernel);
12231319
12241320 fprintf(fp, ", excl_host = %d, excl_guest = %d",
1225− attr.exclude_host,
1226− attr.exclude_guest);
1321+ evsel->attr.exclude_host,
1322+ evsel->attr.exclude_guest);
12271323
1228− fprintf(fp, ", precise_ip = %d", attr.precise_ip);
1324+ fprintf(fp, ", precise_ip = %d", evsel->attr.precise_ip);
12291325
1230− if (nr)
1326+ if (evsel->ids) {
12311327 fprintf(fp, ", id = {");
1232−
1233− for (j = 0 ; j < nr; j++) {
1234− ret = read(fd, &id, sizeof(id));
1235− if (ret != (ssize_t)sizeof(id))
1236− goto error;
1237−
1238− if (ph->needs_swap)
1239− id = bswap_64(id);
1240−
1241− if (j)
1242− fputc(',', fp);
1243−
1244− fprintf(fp, " %"PRIu64, id);
1245− }
1246− if (nr && j == nr)
1328+ for (j = 0, id = evsel->id; j < evsel->ids; j++, id++) {
1329+ if (j)
1330+ fputc(',', fp);
1331+ fprintf(fp, " %"PRIu64, *id);
1332+ }
12471333 fprintf(fp, " }");
1334+ }
1335+
12481336 fputc('\n', fp);
12491337 }
1250− free(buf);
1251− return;
1252−error:
1253− fprintf(fp, "# event desc: not available or unable to read\n");
1338+
1339+ free_event_desc(events);
12541340 }
12551341
1256−static void print_total_mem(struct perf_header *h __used, int fd, FILE *fp)
1342+static void print_total_mem(struct perf_header *ph, int fd __maybe_unused,
1343+ FILE *fp)
12571344 {
1258− uint64_t mem;
1259− ssize_t ret;
1260−
1261− ret = read(fd, &mem, sizeof(mem));
1262− if (ret != sizeof(mem))
1263− goto error;
1264−
1265− if (h->needs_swap)
1266− mem = bswap_64(mem);
1267−
1268− fprintf(fp, "# total memory : %"PRIu64" kB\n", mem);
1269− return;
1270−error:
1271− fprintf(fp, "# total memory : unknown\n");
1345+ fprintf(fp, "# total memory : %Lu kB\n", ph->env.total_mem);
12721346 }
12731347
1274−static void print_numa_topology(struct perf_header *h __used, int fd, FILE *fp)
1348+static void print_numa_topology(struct perf_header *ph, int fd __maybe_unused,
1349+ FILE *fp)
12751350 {
1276− ssize_t ret;
12771351 u32 nr, c, i;
1278− char *str;
1352+ char *str, *tmp;
12791353 uint64_t mem_total, mem_free;
12801354
12811355 /* nr nodes */
1282− ret = read(fd, &nr, sizeof(nr));
1283− if (ret != (ssize_t)sizeof(nr))
1284− goto error;
1285−
1286− if (h->needs_swap)
1287− nr = bswap_32(nr);
1356+ nr = ph->env.nr_numa_nodes;
1357+ str = ph->env.numa_nodes;
12881358
12891359 for (i = 0; i < nr; i++) {
1290−
12911360 /* node number */
1292− ret = read(fd, &c, sizeof(c));
1293− if (ret != (ssize_t)sizeof(c))
1361+ c = strtoul(str, &tmp, 0);
1362+ if (*tmp != ':')
12941363 goto error;
12951364
1296− if (h->needs_swap)
1297− c = bswap_32(c);
1298−
1299− ret = read(fd, &mem_total, sizeof(u64));
1300− if (ret != sizeof(u64))
1365+ str = tmp + 1;
1366+ mem_total = strtoull(str, &tmp, 0);
1367+ if (*tmp != ':')
13011368 goto error;
13021369
1303− ret = read(fd, &mem_free, sizeof(u64));
1304− if (ret != sizeof(u64))
1370+ str = tmp + 1;
1371+ mem_free = strtoull(str, &tmp, 0);
1372+ if (*tmp != ':')
13051373 goto error;
13061374
1307− if (h->needs_swap) {
1308− mem_total = bswap_64(mem_total);
1309− mem_free = bswap_64(mem_free);
1310− }
1311−
13121375 fprintf(fp, "# node%u meminfo : total = %"PRIu64" kB,"
13131376 " free = %"PRIu64" kB\n",
1314− c,
1315− mem_total,
1316− mem_free);
1377+ c, mem_total, mem_free);
13171378
1318− str = do_read_string(fd, h);
1379+ str = tmp + 1;
13191380 fprintf(fp, "# node%u cpu list : %s\n", c, str);
1320− free(str);
13211381 }
13221382 return;
13231383 error:
13241384 fprintf(fp, "# numa topology : not available\n");
13251385 }
13261386
1327−static void print_cpuid(struct perf_header *ph, int fd, FILE *fp)
1387+static void print_cpuid(struct perf_header *ph, int fd __maybe_unused, FILE *fp)
13281388 {
1329− char *str = do_read_string(fd, ph);
1330− fprintf(fp, "# cpuid : %s\n", str);
1331− free(str);
1389+ fprintf(fp, "# cpuid : %s\n", ph->env.cpuid);
13321390 }
13331391
1334−static void print_branch_stack(struct perf_header *ph __used, int fd __used,
1335− FILE *fp)
1392+static void print_branch_stack(struct perf_header *ph __maybe_unused,
1393+ int fd __maybe_unused, FILE *fp)
13361394 {
13371395 fprintf(fp, "# contains samples with branch stack\n");
13381396 }
13391397
1398+static void print_pmu_mappings(struct perf_header *ph, int fd __maybe_unused,
1399+ FILE *fp)
1400+{
1401+ const char *delimiter = "# pmu mappings: ";
1402+ char *str, *tmp;
1403+ u32 pmu_num;
1404+ u32 type;
1405+
1406+ pmu_num = ph->env.nr_pmu_mappings;
1407+ if (!pmu_num) {
1408+ fprintf(fp, "# pmu mappings: not available\n");
1409+ return;
1410+ }
1411+
1412+ str = ph->env.pmu_mappings;
1413+
1414+ while (pmu_num) {
1415+ type = strtoul(str, &tmp, 0);
1416+ if (*tmp != ':')
1417+ goto error;
1418+
1419+ str = tmp + 1;
1420+ fprintf(fp, "%s%s = %" PRIu32, delimiter, str, type);
1421+
1422+ delimiter = ", ";
1423+ str += strlen(str) + 1;
1424+ pmu_num--;
1425+ }
1426+
1427+ fprintf(fp, "\n");
1428+
1429+ if (!pmu_num)
1430+ return;
1431+error:
1432+ fprintf(fp, "# pmu mappings: unable to read\n");
1433+}
1434+
13401435 static int __event_process_build_id(struct build_id_event *bev,
13411436 char *filename,
13421437 struct perf_session *session)
@@ -1398,7 +1493,7 @@ static int perf_header__read_build_ids_abi_quirk(struct perf_header *header, · 55 unmodified lines
13981493 struct perf_session *session = container_of(header, struct perf_session, header);
13991494 struct {
14001495 struct perf_event_header header;
1401− u8 build_id[ALIGN(BUILD_ID_SIZE, sizeof(u64))];
1496+ u8 build_id[PERF_ALIGN(BUILD_ID_SIZE, sizeof(u64))];
14021497 char filename[0];
14031498 } old_bev;
14041499 struct build_id_event bev;
@@ -1487,28 +1582,375 @@ out: · 82 unmodified lines
14871582 return err;
14881583 }
14891584
1490−static int process_tracing_data(struct perf_file_section *section __unused,
1491− struct perf_header *ph __unused,
1492− int feat __unused, int fd, void *data)
1585+static int process_tracing_data(struct perf_file_section *section __maybe_unused,
1586+ struct perf_header *ph __maybe_unused,
1587+ int fd, void *data)
14931588 {
14941589 trace_report(fd, data, false);
14951590 return 0;
14961591 }
14971592
14981593 static int process_build_id(struct perf_file_section *section,
1499− struct perf_header *ph,
1500− int feat __unused, int fd, void *data __used)
1594+ struct perf_header *ph, int fd,
1595+ void *data __maybe_unused)
15011596 {
15021597 if (perf_header__read_build_ids(ph, fd, section->offset, section->size))
15031598 pr_debug("Failed to read buildids, continuing...\n");
15041599 return 0;
15051600 }
15061601
1602+static int process_hostname(struct perf_file_section *section __maybe_unused,
1603+ struct perf_header *ph, int fd,
1604+ void *data __maybe_unused)
1605+{
1606+ ph->env.hostname = do_read_string(fd, ph);
1607+ return ph->env.hostname ? 0 : -ENOMEM;
1608+}
1609+
1610+static int process_osrelease(struct perf_file_section *section __maybe_unused,
1611+ struct perf_header *ph, int fd,
1612+ void *data __maybe_unused)
1613+{
1614+ ph->env.os_release = do_read_string(fd, ph);
1615+ return ph->env.os_release ? 0 : -ENOMEM;
1616+}
1617+
1618+static int process_version(struct perf_file_section *section __maybe_unused,
1619+ struct perf_header *ph, int fd,
1620+ void *data __maybe_unused)
1621+{
1622+ ph->env.version = do_read_string(fd, ph);
1623+ return ph->env.version ? 0 : -ENOMEM;
1624+}
1625+
1626+static int process_arch(struct perf_file_section *section __maybe_unused,
1627+ struct perf_header *ph, int fd,
1628+ void *data __maybe_unused)
1629+{
1630+ ph->env.arch = do_read_string(fd, ph);
1631+ return ph->env.arch ? 0 : -ENOMEM;
1632+}
1633+
1634+static int process_nrcpus(struct perf_file_section *section __maybe_unused,
1635+ struct perf_header *ph, int fd,
1636+ void *data __maybe_unused)
1637+{
1638+ size_t ret;
1639+ u32 nr;
1640+
1641+ ret = read(fd, &nr, sizeof(nr));
1642+ if (ret != sizeof(nr))
1643+ return -1;
1644+
1645+ if (ph->needs_swap)
1646+ nr = bswap_32(nr);
1647+
1648+ ph->env.nr_cpus_online = nr;
1649+
1650+ ret = read(fd, &nr, sizeof(nr));
1651+ if (ret != sizeof(nr))
1652+ return -1;
1653+
1654+ if (ph->needs_swap)
1655+ nr = bswap_32(nr);
1656+
1657+ ph->env.nr_cpus_avail = nr;
1658+ return 0;
1659+}
1660+
1661+static int process_cpudesc(struct perf_file_section *section __maybe_unused,
1662+ struct perf_header *ph, int fd,
1663+ void *data __maybe_unused)
1664+{
1665+ ph->env.cpu_desc = do_read_string(fd, ph);
1666+ return ph->env.cpu_desc ? 0 : -ENOMEM;
1667+}
1668+
1669+static int process_cpuid(struct perf_file_section *section __maybe_unused,
1670+ struct perf_header *ph, int fd,

638 more lines aren’t shown here.

tools/perf/util/header.h+27−2

This diff is large: its 60 lines aren’t shown here.

tools/perf/util/help.c+3−1

This diff is large: its 18 lines aren’t shown here.

tools/perf/util/hist.c+26−695

This diff is large: its 767 lines aren’t shown here.

tools/perf/util/hist.h+58−17

This diff is large: its 105 lines aren’t shown here.

tools/perf/util/include/linux/bitops.h+4−0

This diff is large: its 11 lines aren’t shown here.

tools/perf/util/include/linux/compiler.h+8−1

This diff is large: its 15 lines aren’t shown here.

tools/perf/util/include/linux/kernel.h+15−2

This diff is large: its 34 lines aren’t shown here.

tools/perf/util/include/linux/magic.h+12−0

This diff is large: its 13 lines aren’t shown here.

tools/perf/util/include/linux/rbtree.h+1−0

This diff is large: its 3 lines aren’t shown here.

tools/perf/util/include/linux/string.h+2−0

This diff is large: its 4 lines aren’t shown here.

tools/perf/util/include/linux/types.h+8−0

This diff is large: its 15 lines aren’t shown here.

tools/perf/util/intlist.c+4−4

This diff is large: its 30 lines aren’t shown here.

tools/perf/util/map.c+32−15

This diff is large: its 118 lines aren’t shown here.

tools/perf/util/map.h+6−3

This diff is large: its 39 lines aren’t shown here.

tools/perf/util/parse-events-test.c+346−78

This diff is large: its 727 lines aren’t shown here.

tools/perf/util/parse-events.c+173−81

This diff is large: its 432 lines aren’t shown here.

tools/perf/util/parse-events.h+11−7

This diff is large: its 53 lines aren’t shown here.

tools/perf/util/parse-events.l+44−12

This diff is large: its 115 lines aren’t shown here.

tools/perf/util/parse-events.y+104−21

This diff is large: its 215 lines aren’t shown here.

tools/perf/util/parse-options.c+2−1

This diff is large: its 10 lines aren’t shown here.

tools/perf/util/perf_regs.h+14−0

This diff is large: its 15 lines aren’t shown here.

tools/perf/util/pmu.c+78−2

This diff is large: its 122 lines aren’t shown here.

tools/perf/util/pmu.h+3−0

This diff is large: its 16 lines aren’t shown here.

tools/perf/util/pmu.y+3−3

This diff is large: its 12 lines aren’t shown here.

tools/perf/util/probe-event.c+50−19

This diff is large: its 151 lines aren’t shown here.

tools/perf/util/probe-finder.c+18−10

This diff is large: its 63 lines aren’t shown here.

tools/perf/util/python.c+11−6

This diff is large: its 59 lines aren’t shown here.

tools/perf/util/scripting-engines/trace-event-perl.c+23−27

This diff is large: its 138 lines aren’t shown here.

tools/perf/util/scripting-engines/trace-event-python.c+98−15

This diff is large: its 179 lines aren’t shown here.

tools/perf/util/session.c+144−54

This diff is large: its 408 lines aren’t shown here.

tools/perf/util/session.h+4−6

This diff is large: its 24 lines aren’t shown here.

tools/perf/util/sort.c+16−9

This diff is large: its 88 lines aren’t shown here.

tools/perf/util/sort.h+1−1

This diff is large: its 10 lines aren’t shown here.

tools/perf/util/stat.c+57−0

This diff is large: its 58 lines aren’t shown here.

tools/perf/util/stat.h+16−0

This diff is large: its 17 lines aren’t shown here.

tools/perf/util/string.c+17−1

This diff is large: its 27 lines aren’t shown here.

tools/perf/util/strlist.c+1−1

This diff is large: its 9 lines aren’t shown here.

tools/perf/util/symbol-elf.c+841−0

This diff is large: its 842 lines aren’t shown here.

tools/perf/util/symbol-minimal.c+307−0

This diff is large: its 308 lines aren’t shown here.

tools/perf/util/symbol.c+91−851

This diff is large: its 1153 lines aren’t shown here.

tools/perf/util/symbol.h+63−4

This diff is large: its 134 lines aren’t shown here.

tools/perf/util/target.c+2−2

This diff is large: its 11 lines aren’t shown here.

tools/perf/util/thread.h+2−0

This diff is large: its 9 lines aren’t shown here.

tools/perf/util/top.c+1−2

This diff is large: its 10 lines aren’t shown here.

tools/perf/util/top.h+1−0

This diff is large: its 8 lines aren’t shown here.

tools/perf/util/trace-event-parse.c+28−26

This diff is large: its 109 lines aren’t shown here.

tools/perf/util/trace-event-scripting.c+18−16

This diff is large: its 67 lines aren’t shown here.

tools/perf/util/trace-event.h+8−4

This diff is large: its 48 lines aren’t shown here.

tools/perf/util/unwind.c+571−0

This diff is large: its 572 lines aren’t shown here.

tools/perf/util/unwind.h+35−0

This diff is large: its 36 lines aren’t shown here.

tools/perf/util/util.c+25−0

This diff is large: its 36 lines aren’t shown here.

tools/perf/util/util.h+3−6

This diff is large: its 22 lines aren’t shown here.

tools/perf/util/vdso.c+111−0

This diff is large: its 112 lines aren’t shown here.

tools/perf/util/vdso.h+18−0

This diff is large: its 19 lines aren’t shown here.

tools/perf/util/wrapper.c+2−1

This diff is large: its 10 lines aren’t shown here.

tools/scripts/Makefile.include+4−2

This diff is large: its 11 lines aren’t shown here.