Allow FENTRY/FEXIT BPF programs to attach to other BPF programs of any type including their subprograms. This feature allows snooping on input and output packets in XDP, TC programs including their return values. In order to do that the verifier needs to track types not only of vmlinux, but types of other BPF programs as well. The verifier also needs to translate uapi/linux/bpf.h types used by networking programs into kernel internal BTF types used by FENTRY/FEXIT BPF programs. In some cases LLVM optimizations can remove arguments from BPF subprograms without adjusting BTF info that LLVM backend knows. When BTF info disagrees with actual types that the verifiers sees the BPF trampoline has to fallback to conservative and treat all arguments as u64. The FENTRY/FEXIT program can still attach to such subprograms, but it won't be able to recognize pointer types like 'struct sk_buff *' and it won't be able to pass them to bpf_skb_output() for dumping packets to user space. The FENTRY/FEXIT program would need to use bpf_probe_read_kernel() instead. The BPF_PROG_LOAD command is extended with attach_prog_fd field. When it's set to zero the attach_btf_id is one vmlinux BTF type ids. When attach_prog_fd points to previously loaded BPF program the attach_btf_id is BTF type id of main function or one of its subprograms.
Showing the changes to include/linux (2 of 8 files). Show all
Unified · Split
| @@ -495,6 +495,7 @@ struct bpf_prog_aux { · 494 unmodified lines | ||
| 495 | 495 | u32 func_cnt; /* used by non-func prog as the number of func progs */ |
| 496 | 496 | u32 func_idx; /* 0 for non-func prog, the index in func array for func prog */ |
| 497 | 497 | u32 attach_btf_id; /* in-kernel BTF type id to attach to */ |
| 498 | + struct bpf_prog *linked_prog; | |
| 498 | 499 | bool verifier_zext; /* Zero extensions has been inserted by verifier. */ |
| 499 | 500 | bool offload_requested; |
| 500 | 501 | bool attach_btf_trace; /* true if attaching to BTF-enabled raw tp */ |
| @@ -88,6 +88,7 @@ static inline bool btf_type_is_func_proto(const struct btf_type *t) · 87 unmodified lines | ||
| 88 | 88 | const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id); |
| 89 | 89 | const char *btf_name_by_offset(const struct btf *btf, u32 offset); |
| 90 | 90 | struct btf *btf_parse_vmlinux(void); |
| 91 | +struct btf *bpf_prog_get_target_btf(const struct bpf_prog *prog); | |
| 91 | 92 | #else |
| 92 | 93 | static inline const struct btf_type *btf_type_by_id(const struct btf *btf, |
| 93 | 94 | u32 type_id) |