Pull bpf updates from Daniel Borkmann: "Major changes: - Redesign the verifier error reporting: failures now carry source and instruction annotations along with the causal event history that led to them, making program rejections far easier to debug and repair (Kumar Kartikeya Dwivedi) - Add arena argument support to kfuncs and struct_ops through the new __arena and __arena__nullable suffixes (Tejun Heo, Puranjay Mohan, Kumar Kartikeya Dwivedi, Ihor Solodrai) - Signed BPF program loader rework to accommodate both BPF and security community needs where the kernel runs the signature verification at BPF_PROG_LOAD time before the LSM admission hook (Daniel Borkmann) - Add a set of ksock kfuncs which let BPF LSM and syscall programs create, connect and send on UDP sockets in order to emit telemetry data (Mahe Tardy) - Unify helper and kfunc call argument verification and classify kfunc arguments purely from BTF into a generated bpf_func_proto which is computed once at add-call time (Amery Hung) Other features and fixes: - Enable EXECMEM_ROX_CACHE for BPF allocations on x86 (Mike Rapoport) - Add bidirectional VLAN support to bpf_fib_lookup() through the new BPF_FIB_LOOKUP_VLAN and BPF_FIB_LOOKUP_VLAN_INPUT flags (Avinash Duduskar) - Infer zext_dst from static register liveness analysis to fix 32-bit zero-extension semantics, and remove the artificial limitations on pointer types eligible for spilling (Eduard Zingerman) - Inline the numeric open-coded iterator kfuncs so that bpf_for() loops no longer pay a kfunc call on every iteration (Puranjay Mohan) - Add an arena-based bitmap data structure to libarena along with serial and parallel selftests (Emil Tsalapatis) - Teach resolve_btfids to discover kfuncs from the kernel's BTF ID sets and to emit kfunc BTF decl tags, reducing the kernel build's dependency on pahole features (Ihor Solodrai) - Add BPF_F_ADJ_ROOM_DECAP_* flags to bpf_skb_adjust_room() so that tunnel decapsulation can update the GSO and encapsulation state of the skb (Nick Hudson) - Fix the ring buffer pending_pos walk and the available-data accounting on 32-bit position wrap (Israel Téllez García) - Add memory usage accounting for arena maps and fix an mmap_lock deadlock on arena lock failure (Jiayuan Chen) - Add tracing_multi link info support to the kernel UAPI and bpftool, and refactor the stack map code to run with preemption disabled (Jiri Olsa) - Support BPF_F_EGRESS in bpf_redirect_peer() to emit the skb in the egress direction of the target's peer device (Jordan Rife) - Add a KF_SPINLOCK_SAFE kfunc flag so that providers, in particular modules, can declare kfuncs safe to call under bpf_spin_lock instead of relying on the verifier's hard-coded allowlist (Kaitao Cheng) - Introduce global percpu data for BPF programs with libbpf probing and bpftool skeleton support, and stop exposing uninitialized kernel heap memory when copying per-CPU map values (Leon Hwang) - Add s390 JIT support for load-acquire and store-release instructions (Maxim Khmelevskii) - Fix a CFI mismatch in the task work callback and an arm64 KASAN false positive after bpf_throw() (Mykyta Yatsenko) - Reject writes through untrusted BTF pointers and bound the rdonly/rdwr_buf_size kfunc arguments (Nicholas Dudar) - Invalidate RCU pointers only after the final spin unlock and account for preempt and IRQ disabled regions as overlapping RCU protection (Ning Ding) - Support mixing bpf2bpf calls and tail calls on RV64, add signed operations and 32-bit atomics to the RV32 JIT, and add timed may_goto support (Pu Lehui, Kuan-Wei Chiu, Feng Jiang) - Fix a use-after-free on mm_struct in bpf_find_vma() for foreign tasks and an mmap_lock leak in the irq_work path (Sanghyun Park) - Populate mmap-able BPF array map memory lazily which makes mmap() O(1) instead of proportional to the map size (Song Liu) - Introduce a jit_required flag and reject programs with inlined helpers when no JIT is available, where the interpreter would otherwise jump into an invalid address (Tiezhu Yang) - Fix the x86 JIT per-CPU address resolution into an extended register where the REX prefix dropped the high destination register bit (Vineet Gupta) - Reject MEM_ALLOC BTF accesses past object bounds, arena frees below the arena base, and mixed arena and ordinary atomic paths (Yiyang Chen) - Fix the trampoline handling of 128-bit arguments and of return values larger than 8 bytes (Yonghong Song) - Ensure that any fault prone load is rewritten with exception table handling, and fix the arena load-acquire and atomic fetch handling in the x86, arm64, riscv and s390 JITs (Daniel Borkmann) - Many more fixes and cleanups across the verifier, arena, trampolines, sockmap, cgroup, ring buffer, x86/arm64/riscv/s390 JITs, libbpf, bpftool, resolve_btfids and selftests" * tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (373 commits) selftests/bpf: Add tests for a store on a fault prone qdisc pointer selftests/bpf: Add tests for fault prone loads out of RCU pointers selftests/bpf: Add tests for pointer type merge at a shared load selftests/bpf: Remove duplicate copies of the arena spinlock qnodes selftests/bpf: Retry stat generation in cgroup_iter_memcg selftests/bpf: Test pseudo-function policy diagnostics bpf: Distinguish function references in policy diagnostics bpf: Preserve source attribution without source text selftests/bpf: Test kfunc argument diagnostics bpf: Correct kfunc argument diagnostics bpf: Use canonical stack argument names in diagnostics bpf: Preserve R0 lineage across helper calls selftests/bpf: Exercise negative optlen in cgroup getsockopt hook bpf: Reject negative optlen in cgroup getsockopt hook selftests/bpf: tc_tunnel - validate decap GSO and encapsulation state bpf: Clear decap state on skb_adjust_room shrink path bpf: Allow new DECAP flags and add guard rails bpf: Add BPF_F_ADJ_ROOM_DECAP_* flags for tunnel decapsulation bpf: Refactor masks for ADJ_ROOM flags and encap validation bpf: Name the enum for BPF_FUNC_skb_adjust_room flags ...
Showing the changes to include/uapi/linux (1 of 272 files). Show all
Unified · Split
| @@ -3038,8 +3038,34 @@ union bpf_attr { · 3037 unmodified lines | ||
| 3038 | 3038 | * |
| 3039 | 3039 | * * **BPF_F_ADJ_ROOM_DECAP_L3_IPV4**, |
| 3040 | 3040 | * **BPF_F_ADJ_ROOM_DECAP_L3_IPV6**: |
| 3041 | − * Indicate the new IP header version after decapsulating the outer | |
| 3042 | − * IP header. Used when the inner and outer IP versions are different. | |
| 3041 | + * Indicate the new IP header version after decapsulating the | |
| 3042 | + * outer IP header. Used when the inner and outer IP versions | |
| 3043 | + * are different. These flags only trigger a protocol change | |
| 3044 | + * without clearing any tunnel-specific GSO flags. | |
| 3045 | + * | |
| 3046 | + * * **BPF_F_ADJ_ROOM_DECAP_L4_GRE**: | |
| 3047 | + * Clear GRE tunnel GSO flags (SKB_GSO_GRE and SKB_GSO_GRE_CSUM) | |
| 3048 | + * when decapsulating a GRE tunnel. | |
| 3049 | + * | |
| 3050 | + * * **BPF_F_ADJ_ROOM_DECAP_L4_UDP**: | |
| 3051 | + * Clear UDP tunnel GSO flags (SKB_GSO_UDP_TUNNEL and | |
| 3052 | + * SKB_GSO_UDP_TUNNEL_CSUM) when decapsulating a UDP tunnel. | |
| 3053 | + * | |
| 3054 | + * * **BPF_F_ADJ_ROOM_DECAP_IPXIP4**: | |
| 3055 | + * Clear IPIP/SIT tunnel GSO flag (SKB_GSO_IPXIP4) when decapsulating | |
| 3056 | + * a tunnel with an outer IPv4 header (IPv4-in-IPv4 or IPv6-in-IPv4). | |
| 3057 | + * | |
| 3058 | + * * **BPF_F_ADJ_ROOM_DECAP_IPXIP6**: | |
| 3059 | + * Clear IPv6 encapsulation tunnel GSO flag (SKB_GSO_IPXIP6) when | |
| 3060 | + * decapsulating a tunnel with an outer IPv6 header (IPv6-in-IPv6 | |
| 3061 | + * or IPv4-in-IPv6). | |
| 3062 | + * | |
| 3063 | + * When using the decapsulation flags above, the skb->encapsulation | |
| 3064 | + * flag is automatically cleared if all tunnel-specific GSO flags | |
| 3065 | + * (SKB_GSO_UDP_TUNNEL, SKB_GSO_UDP_TUNNEL_CSUM, SKB_GSO_GRE, | |
| 3066 | + * SKB_GSO_GRE_CSUM, SKB_GSO_IPXIP4, SKB_GSO_IPXIP6) have been | |
| 3067 | + * removed from the packet. This handles cases where all tunnel | |
| 3068 | + * layers have been decapsulated. | |
| 3043 | 3069 | * |
| 3044 | 3070 | * A call to this helper is susceptible to change the underlying |
| 3045 | 3071 | * packet buffer. Therefore, at load time, all checks on pointers |
| @@ -3532,6 +3558,47 @@ union bpf_attr { · 486 unmodified lines | ||
| 3532 | 3558 | * Use the mark present in *params*->mark for the fib lookup. |
| 3533 | 3559 | * This option should not be used with BPF_FIB_LOOKUP_DIRECT, |
| 3534 | 3560 | * as it only has meaning for full lookups. |
| 3561 | + * **BPF_FIB_LOOKUP_VLAN** | |
| 3562 | + * If the fib lookup resolves to a VLAN device whose | |
| 3563 | + * parent is a real (non-VLAN) device, set | |
| 3564 | + * *params*->h_vlan_proto and *params*->h_vlan_TCI from | |
| 3565 | + * the VLAN device and replace *params*->ifindex with the | |
| 3566 | + * parent's ifindex. *params*->h_vlan_TCI carries the VID | |
| 3567 | + * only, with PCP and DEI bits zero; a consumer wanting to | |
| 3568 | + * set egress priority writes PCP itself. *params*->smac is | |
| 3569 | + * the VLAN device's own address, which can differ from the | |
| 3570 | + * parent's. Only the immediate parent is resolved; if it | |
| 3571 | + * is itself a VLAN device (QinQ) or in another namespace, | |
| 3572 | + * the egress cannot be reduced to a physical device plus | |
| 3573 | + * one tag and the lookup returns | |
| 3574 | + * **BPF_FIB_LKUP_RET_VLAN_FAILURE** with *params*->ifindex | |
| 3575 | + * left at the input. To obtain the VLAN device's own | |
| 3576 | + * ifindex, repeat the lookup without | |
| 3577 | + * **BPF_FIB_LOOKUP_VLAN**, re-initializing *params* | |
| 3578 | + * first: output fields overwrite the inputs they share | |
| 3579 | + * storage with. The swap and the vlan fields | |
| 3580 | + * are written only on success; other output fields keep | |
| 3581 | + * the helper's existing behaviour, so a frag-needed result | |
| 3582 | + * still reports the route mtu in *params*->mtu_result. | |
| 3583 | + * This flag is only valid for XDP programs; tc programs | |
| 3584 | + * receive -EINVAL since they can redirect to the VLAN | |
| 3585 | + * device directly. | |
| 3586 | + * **BPF_FIB_LOOKUP_VLAN_INPUT** | |
| 3587 | + * Treat *params*->h_vlan_proto and *params*->h_vlan_TCI | |
| 3588 | + * as an input VLAN tag and run the lookup as if ingress | |
| 3589 | + * had happened on the VLAN subinterface carrying that tag | |
| 3590 | + * on *params*->ifindex. The VID is the low 12 bits of | |
| 3591 | + * *params*->h_vlan_TCI; *params*->h_vlan_proto must be | |
| 3592 | + * ETH_P_8021Q or ETH_P_8021AD in network byte order, else | |
| 3593 | + * **-EINVAL**. If *params*->ifindex is itself a VLAN | |
| 3594 | + * device, its inner (QinQ) subinterface is matched; for a | |
| 3595 | + * bond or team, pass the master's ifindex. An unmatched | |
| 3596 | + * tag, a down device, or one in another namespace returns | |
| 3597 | + * **BPF_FIB_LKUP_RET_NOT_FWDED**, mirroring real ingress. | |
| 3598 | + * A VID of 0 is looked up literally, so do not set this | |
| 3599 | + * flag for priority-tagged frames. Cannot be combined with | |
| 3600 | + * **BPF_FIB_LOOKUP_TBID** or **BPF_FIB_LOOKUP_OUTPUT** | |
| 3601 | + * (returns **-EINVAL**). | |
| 3535 | 3602 | * |
| 3536 | 3603 | * *ctx* is either **struct xdp_md** for XDP programs or |
| 3537 | 3604 | * **struct sk_buff** tc cls_act programs. |
| @@ -4694,6 +4761,7 @@ union bpf_attr { · 1156 unmodified lines | ||
| 4694 | 4761 | * * **BPF_RB_RING_SIZE**: The size of ring buffer. |
| 4695 | 4762 | * * **BPF_RB_CONS_POS**: Consumer position (can wrap around). |
| 4696 | 4763 | * * **BPF_RB_PROD_POS**: Producer(s) position (can wrap around). |
| 4764 | + * * **BPF_RB_OVERWRITE_POS**: Overwrite position (can wrap around). | |
| 4697 | 4765 | * |
| 4698 | 4766 | * Data returned is just a momentary snapshot of actual values |
| 4699 | 4767 | * and could be inaccurate, so this facility should be used to |
| @@ -5079,17 +5147,19 @@ union bpf_attr { · 379 unmodified lines | ||
| 5079 | 5147 | * Description |
| 5080 | 5148 | * Redirect the packet to another net device of index *ifindex*. |
| 5081 | 5149 | * This helper is somewhat similar to **bpf_redirect**\ (), except |
| 5082 | − * that the redirection happens to the *ifindex*' peer device and | |
| 5083 | − * the netns switch takes place from ingress to ingress without | |
| 5084 | − * going through the CPU's backlog queue. | |
| 5150 | + * that the redirection happens to the *ifindex*' peer device. If | |
| 5151 | + * *flags* is 0, the netns switch takes place from ingress to | |
| 5152 | + * ingress without going through the CPU's backlog queue. If the | |
| 5153 | + * **BPF_F_EGRESS** flag is provided then redirection happens in | |
| 5154 | + * the egress direction of the peer device. | |
| 5085 | 5155 | * |
| 5086 | 5156 | * *skb*\ **->mark** and *skb*\ **->tstamp** are not cleared during |
| 5087 | 5157 | * the netns switch. |
| 5088 | 5158 | * |
| 5089 | − * The *flags* argument is reserved and must be 0. The helper is | |
| 5090 | − * currently only supported for tc BPF program types at the | |
| 5091 | − * ingress hook and for veth and netkit target device types. The | |
| 5092 | − * peer device must reside in a different network namespace. | |
| 5159 | + * If the *flags* argument is 0, the helper is currently only | |
| 5160 | + * supported for tc BPF program types at the ingress hook and for | |
| 5161 | + * veth and netkit target device types. The peer device must reside | |
| 5162 | + * in a different network namespace. | |
| 5093 | 5163 | * Return |
| 5094 | 5164 | * The helper returns **TC_ACT_REDIRECT** on success or |
| 5095 | 5165 | * **TC_ACT_SHOT** on error. |
| @@ -6239,7 +6309,7 @@ enum { · 1143 unmodified lines | ||
| 6239 | 6309 | }; |
| 6240 | 6310 | |
| 6241 | 6311 | /* BPF_FUNC_skb_adjust_room flags. */ |
| 6242 | −enum { | |
| 6312 | +enum bpf_adj_room_flags { | |
| 6243 | 6313 | BPF_F_ADJ_ROOM_FIXED_GSO = (1ULL << 0), |
| 6244 | 6314 | BPF_F_ADJ_ROOM_ENCAP_L3_IPV4 = (1ULL << 1), |
| 6245 | 6315 | BPF_F_ADJ_ROOM_ENCAP_L3_IPV6 = (1ULL << 2), |
| @@ -6249,6 +6319,10 @@ enum { · 3 unmodified lines | ||
| 6249 | 6319 | BPF_F_ADJ_ROOM_ENCAP_L2_ETH = (1ULL << 6), |
| 6250 | 6320 | BPF_F_ADJ_ROOM_DECAP_L3_IPV4 = (1ULL << 7), |
| 6251 | 6321 | BPF_F_ADJ_ROOM_DECAP_L3_IPV6 = (1ULL << 8), |
| 6322 | + BPF_F_ADJ_ROOM_DECAP_L4_GRE = (1ULL << 9), | |
| 6323 | + BPF_F_ADJ_ROOM_DECAP_L4_UDP = (1ULL << 10), | |
| 6324 | + BPF_F_ADJ_ROOM_DECAP_IPXIP4 = (1ULL << 11), | |
| 6325 | + BPF_F_ADJ_ROOM_DECAP_IPXIP6 = (1ULL << 12), | |
| 6252 | 6326 | }; |
| 6253 | 6327 | |
| 6254 | 6328 | enum { |
| @@ -6336,9 +6410,10 @@ enum { · 81 unmodified lines | ||
| 6336 | 6410 | /* Flags for bpf_redirect and bpf_redirect_map helpers */ |
| 6337 | 6411 | enum { |
| 6338 | 6412 | BPF_F_INGRESS = (1ULL << 0), /* used for skb path */ |
| 6413 | + BPF_F_EGRESS = (1ULL << 1), /* used for skb path */ | |
| 6339 | 6414 | BPF_F_BROADCAST = (1ULL << 3), /* used for XDP path */ |
| 6340 | 6415 | BPF_F_EXCLUDE_INGRESS = (1ULL << 4), /* used for XDP path */ |
| 6341 | −#define BPF_F_REDIRECT_FLAGS (BPF_F_INGRESS | BPF_F_BROADCAST | BPF_F_EXCLUDE_INGRESS) | |
| 6416 | +#define BPF_F_REDIRECT_FLAGS (BPF_F_INGRESS | BPF_F_EGRESS | BPF_F_BROADCAST | BPF_F_EXCLUDE_INGRESS) | |
| 6342 | 6417 | }; |
| 6343 | 6418 | |
| 6344 | 6419 | #define __bpf_md_ptr(type, name) \ |
| @@ -6839,6 +6914,15 @@ struct bpf_link_info { · 494 unmodified lines | ||
| 6839 | 6914 | __u32 flags; |
| 6840 | 6915 | __u32 pid; |
| 6841 | 6916 | } uprobe_multi; |
| 6917 | + struct { | |
| 6918 | + __u32 attach_type; | |
| 6919 | + __u32 count; /* in/out: tracing_multi target count */ | |
| 6920 | + __u32 btf_obj_id; | |
| 6921 | + __u32 :32; | |
| 6922 | + __aligned_u64 ids; | |
| 6923 | + __aligned_u64 addrs; | |
| 6924 | + __aligned_u64 cookies; | |
| 6925 | + } tracing_multi; | |
| 6842 | 6926 | struct { |
| 6843 | 6927 | __u32 type; /* enum bpf_perf_event_type */ |
| 6844 | 6928 | __u32 :32; |
| @@ -7327,6 +7411,8 @@ enum { · 482 unmodified lines | ||
| 7327 | 7411 | BPF_FIB_LOOKUP_TBID = (1U << 3), |
| 7328 | 7412 | BPF_FIB_LOOKUP_SRC = (1U << 4), |
| 7329 | 7413 | BPF_FIB_LOOKUP_MARK = (1U << 5), |
| 7414 | + BPF_FIB_LOOKUP_VLAN = (1U << 6), | |
| 7415 | + BPF_FIB_LOOKUP_VLAN_INPUT = (1U << 7), | |
| 7330 | 7416 | }; |
| 7331 | 7417 | |
| 7332 | 7418 | enum { |
| @@ -7340,6 +7426,7 @@ enum { · 7 unmodified lines | ||
| 7340 | 7426 | BPF_FIB_LKUP_RET_NO_NEIGH, /* no neighbor entry for nh */ |
| 7341 | 7427 | BPF_FIB_LKUP_RET_FRAG_NEEDED, /* fragmentation required to fwd */ |
| 7342 | 7428 | BPF_FIB_LKUP_RET_NO_SRC_ADDR, /* failed to derive IP src addr */ |
| 7429 | + BPF_FIB_LKUP_RET_VLAN_FAILURE, /* VLAN egress, parent unresolvable */ | |
| 7343 | 7430 | }; |
| 7344 | 7431 | |
| 7345 | 7432 | struct bpf_fib_lookup { |
| @@ -7393,7 +7480,13 @@ struct bpf_fib_lookup { · 47 unmodified lines | ||
| 7393 | 7480 | |
| 7394 | 7481 | union { |
| 7395 | 7482 | struct { |
| 7396 | − /* output */ | |
| 7483 | + /* | |
| 7484 | + * output with BPF_FIB_LOOKUP_VLAN: set from the | |
| 7485 | + * resolved egress VLAN device (see the flag); zeroed | |
| 7486 | + * on other successful lookups. input with | |
| 7487 | + * BPF_FIB_LOOKUP_VLAN_INPUT: the VLAN tag to scope | |
| 7488 | + * the lookup by. | |
| 7489 | + */ | |
| 7397 | 7490 | __be16 h_vlan_proto; |
| 7398 | 7491 | __be16 h_vlan_TCI; |
| 7399 | 7492 | }; |