Add support for deferred userspace unwind to perf. Where perf currently relies on in-place stack unwinding; from NMI context and all that. This moves the userspace part of the unwind to right before the return-to-userspace. This has two distinct benefits, the biggest is that it moves the unwind to a faultable context. It becomes possible to fault in debug info (.eh_frame, SFrame etc.) that might not otherwise be readily available. And secondly, it de-duplicates the user callchain where multiple samples happen during the same kernel entry. To facilitate this the perf interface is extended with a new record type: PERF_RECORD_CALLCHAIN_DEFERRED and two new attribute flags: perf_event_attr::defer_callchain - to request the user unwind be deferred perf_event_attr::defer_output - to request PERF_RECORD_CALLCHAIN_DEFERRED records The existing PERF_RECORD_SAMPLE callchain section gets a new context type: PERF_CONTEXT_USER_DEFERRED After which will come a single entry, denoting the 'cookie' of the deferred callchain that should be attached here, matching the 'cookie' field of the above mentioned PERF_RECORD_CALLCHAIN_DEFERRED. The 'defer_callchain' flag is expected on all events with PERF_SAMPLE_CALLCHAIN. The 'defer_output' flag is expect on the event responsible for collecting side-band events (like mmap, comm etc.). Setting 'defer_output' on multiple events will get you duplicated PERF_RECORD_CALLCHAIN_DEFERRED records. Based on earlier patches by Josh and Steven.
Showing the changes to kernel/bpf/stackmap.c (1 of 8 files). Show all
Unified · Split
| @@ -315,7 +315,7 @@ BPF_CALL_3(bpf_get_stackid, struct pt_regs *, regs, struct bpf_map *, map, · 314 unmodified lines | |||
| 315 | max_depth = sysctl_perf_event_max_stack; | 315 | max_depth = sysctl_perf_event_max_stack; |
| 316 | 316 | ||
| 317 | trace = get_perf_callchain(regs, kernel, user, max_depth, | 317 | trace = get_perf_callchain(regs, kernel, user, max_depth, |
| 318 | false, false); | 318 | false, false, 0); |
| 319 | 319 | ||
| 320 | if (unlikely(!trace)) | 320 | if (unlikely(!trace)) |
| 321 | /* couldn't fetch the stack trace */ | 321 | /* couldn't fetch the stack trace */ |
| @@ -452,7 +452,7 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, · 130 unmodified lines | |||
| 452 | trace = get_callchain_entry_for_task(task, max_depth); | 452 | trace = get_callchain_entry_for_task(task, max_depth); |
| 453 | else | 453 | else |
| 454 | trace = get_perf_callchain(regs, kernel, user, max_depth, | 454 | trace = get_perf_callchain(regs, kernel, user, max_depth, |
| 455 | crosstask, false); | 455 | crosstask, false, 0); |
| 456 | 456 | ||
| 457 | if (unlikely(!trace) || trace->nr < skip) { | 457 | if (unlikely(!trace) || trace->nr < skip) { |
| 458 | if (may_fault) | 458 | if (may_fault) |