Pull networking updates from Jakub Kicinski: - Add redirect_neigh() BPF packet redirect helper, allowing to limit stack traversal in common container configs and improving TCP back-pressure. Daniel reports ~10Gbps => ~15Gbps single stream TCP performance gain. - Expand netlink policy support and improve policy export to user space. (Ge)netlink core performs request validation according to declared policies. Expand the expressiveness of those policies (min/max length and bitmasks). Allow dumping policies for particular commands. This is used for feature discovery by user space (instead of kernel version parsing or trial and error). - Support IGMPv3/MLDv2 multicast listener discovery protocols in bridge. - Allow more than 255 IPv4 multicast interfaces. - Add support for Type of Service (ToS) reflection in SYN/SYN-ACK packets of TCPv6. - In Multi-patch TCP (MPTCP) support concurrent transmission of data on multiple subflows in a load balancing scenario. Enhance advertising addresses via the RM_ADDR/ADD_ADDR options. - Support SMC-Dv2 version of SMC, which enables multi-subnet deployments. - Allow more calls to same peer in RxRPC. - Support two new Controller Area Network (CAN) protocols - CAN-FD and ISO 15765-2:2016. - Add xfrm/IPsec compat layer, solving the 32bit user space on 64bit kernel problem. - Add TC actions for implementing MPLS L2 VPNs. - Improve nexthop code - e.g. handle various corner cases when nexthop objects are removed from groups better, skip unnecessary notifications and make it easier to offload nexthops into HW by converting to a blocking notifier. - Support adding and consuming TCP header options by BPF programs, opening the doors for easy experimental and deployment-specific TCP option use. - Reorganize TCP congestion control (CC) initialization to simplify life of TCP CC implemented in BPF. - Add support for shipping BPF programs with the kernel and loading them early on boot via the User Mode Driver mechanism, hence reusing all the user space infra we have. - Support sleepable BPF programs, initially targeting LSM and tracing. - Add bpf_d_path() helper for returning full path for given 'struct path'. - Make bpf_tail_call compatible with bpf-to-bpf calls. - Allow BPF programs to call map_update_elem on sockmaps. - Add BPF Type Format (BTF) support for type and enum discovery, as well as support for using BTF within the kernel itself (current use is for pretty printing structures). - Support listing and getting information about bpf_links via the bpf syscall. - Enhance kernel interfaces around NIC firmware update. Allow specifying overwrite mask to control if settings etc. are reset during update; report expected max time operation may take to users; support firmware activation without machine reboot incl. limits of how much impact reset may have (e.g. dropping link or not). - Extend ethtool configuration interface to report IEEE-standard counters, to limit the need for per-vendor logic in user space. - Adopt or extend devlink use for debug, monitoring, fw update in many drivers (dsa loop, ice, ionic, sja1105, qed, mlxsw, mv88e6xxx, dpaa2-eth). - In mlxsw expose critical and emergency SFP module temperature alarms. Refactor port buffer handling to make the defaults more suitable and support setting these values explicitly via the DCBNL interface. - Add XDP support for Intel's igb driver. - Support offloading TC flower classification and filtering rules to mscc_ocelot switches. - Add PTP support for Marvell Octeontx2 and PP2.2 hardware, as well as fixed interval period pulse generator and one-step timestamping in dpaa-eth. - Add support for various auth offloads in WiFi APs, e.g. SAE (WPA3) offload. - Add Lynx PHY/PCS MDIO module, and convert various drivers which have this HW to use it. Convert mvpp2 to split PCS. - Support Marvell Prestera 98DX3255 24-port switch ASICs, as well as 7-port Mediatek MT7531 IP. - Add initial support for QCA6390 and IPQ6018 in ath11k WiFi driver, and wcn3680 support in wcn36xx. - Improve performance for packets which don't require much offloads on recent Mellanox NICs by 20% by making multiple packets share a descriptor entry. - Move chelsio inline crypto drivers (for TLS and IPsec) from the crypto subtree to drivers/net. Move MDIO drivers out of the phy directory. - Clean up a lot of W=1 warnings, reportedly the actively developed subsections of networking drivers should now build W=1 warning free. - Make sure drivers don't use in_interrupt() to dynamically adapt their code. Convert tasklets to use new tasklet_setup API (sadly this conversion is not yet complete). * tag 'net-next-5.10' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (2583 commits) Revert "bpfilter: Fix build error with CONFIG_BPFILTER_UMH" net, sockmap: Don't call bpf_prog_put() on NULL pointer bpf, selftest: Fix flaky tcp_hdr_options test when adding addr to lo bpf, sockmap: Add locking annotations to iterator netfilter: nftables: allow re-computing sctp CRC-32C in 'payload' statements net: fix pos incrementment in ipv6_route_seq_next net/smc: fix invalid return code in smcd_new_buf_create() net/smc: fix valid DMBE buffer sizes net/smc: fix use-after-free of delayed events bpfilter: Fix build error with CONFIG_BPFILTER_UMH cxgb4/ch_ipsec: Replace the module name to ch_ipsec from chcr net: sched: Fix suspicious RCU usage while accessing tcf_tunnel_info bpf: Fix register equivalence tracking. rxrpc: Fix loss of final ack on shutdown rxrpc: Fix bundle counting for exclusive connections netfilter: restore NF_INET_NUMHOOKS ibmveth: Identify ingress large send packets. ibmveth: Switch order of ibmveth_helper calls. cxgb4: handle 4-tuple PEDIT to NAT mode translation selftests: Add VRF route leaking tests ...
Showing the changes to tools/include/uapi (1 of 2301 files). Show all
Unified · Split
| @@ -124,6 +124,7 @@ enum bpf_cmd { · 123 unmodified lines | ||
| 124 | 124 | BPF_ENABLE_STATS, |
| 125 | 125 | BPF_ITER_CREATE, |
| 126 | 126 | BPF_LINK_DETACH, |
| 127 | + BPF_PROG_BIND_MAP, | |
| 127 | 128 | }; |
| 128 | 129 | |
| 129 | 130 | enum bpf_map_type { |
| @@ -155,6 +156,7 @@ enum bpf_map_type { · 25 unmodified lines | ||
| 155 | 156 | BPF_MAP_TYPE_DEVMAP_HASH, |
| 156 | 157 | BPF_MAP_TYPE_STRUCT_OPS, |
| 157 | 158 | BPF_MAP_TYPE_RINGBUF, |
| 159 | + BPF_MAP_TYPE_INODE_STORAGE, | |
| 158 | 160 | }; |
| 159 | 161 | |
| 160 | 162 | /* Note that tracing related programs such as |
| @@ -345,19 +347,45 @@ enum bpf_link_type { · 184 unmodified lines | ||
| 345 | 347 | /* The verifier internal test flag. Behavior is undefined */ |
| 346 | 348 | #define BPF_F_TEST_STATE_FREQ (1U << 3) |
| 347 | 349 | |
| 350 | +/* If BPF_F_SLEEPABLE is used in BPF_PROG_LOAD command, the verifier will | |
| 351 | + * restrict map and helper usage for such programs. Sleepable BPF programs can | |
| 352 | + * only be attached to hooks where kernel execution context allows sleeping. | |
| 353 | + * Such programs are allowed to use helpers that may sleep like | |
| 354 | + * bpf_copy_from_user(). | |
| 355 | + */ | |
| 356 | +#define BPF_F_SLEEPABLE (1U << 4) | |
| 357 | + | |
| 348 | 358 | /* When BPF ldimm64's insn[0].src_reg != 0 then this can have |
| 349 | − * two extensions: | |
| 350 | − * | |
| 351 | − * insn[0].src_reg: BPF_PSEUDO_MAP_FD BPF_PSEUDO_MAP_VALUE | |
| 352 | − * insn[0].imm: map fd map fd | |
| 353 | − * insn[1].imm: 0 offset into value | |
| 354 | − * insn[0].off: 0 0 | |
| 355 | − * insn[1].off: 0 0 | |
| 356 | − * ldimm64 rewrite: address of map address of map[0]+offset | |
| 357 | − * verifier type: CONST_PTR_TO_MAP PTR_TO_MAP_VALUE | |
| 359 | + * the following extensions: | |
| 360 | + * | |
| 361 | + * insn[0].src_reg: BPF_PSEUDO_MAP_FD | |
| 362 | + * insn[0].imm: map fd | |
| 363 | + * insn[1].imm: 0 | |
| 364 | + * insn[0].off: 0 | |
| 365 | + * insn[1].off: 0 | |
| 366 | + * ldimm64 rewrite: address of map | |
| 367 | + * verifier type: CONST_PTR_TO_MAP | |
| 358 | 368 | */ |
| 359 | 369 | #define BPF_PSEUDO_MAP_FD 1 |
| 370 | +/* insn[0].src_reg: BPF_PSEUDO_MAP_VALUE | |
| 371 | + * insn[0].imm: map fd | |
| 372 | + * insn[1].imm: offset into value | |
| 373 | + * insn[0].off: 0 | |
| 374 | + * insn[1].off: 0 | |
| 375 | + * ldimm64 rewrite: address of map[0]+offset | |
| 376 | + * verifier type: PTR_TO_MAP_VALUE | |
| 377 | + */ | |
| 360 | 378 | #define BPF_PSEUDO_MAP_VALUE 2 |
| 379 | +/* insn[0].src_reg: BPF_PSEUDO_BTF_ID | |
| 380 | + * insn[0].imm: kernel btd id of VAR | |
| 381 | + * insn[1].imm: 0 | |
| 382 | + * insn[0].off: 0 | |
| 383 | + * insn[1].off: 0 | |
| 384 | + * ldimm64 rewrite: address of the kernel variable | |
| 385 | + * verifier type: PTR_TO_BTF_ID or PTR_TO_MEM, depending on whether the var | |
| 386 | + * is struct/union. | |
| 387 | + */ | |
| 388 | +#define BPF_PSEUDO_BTF_ID 3 | |
| 361 | 389 | |
| 362 | 390 | /* when bpf_call->src_reg == BPF_PSEUDO_CALL, bpf_call->imm == pc-relative |
| 363 | 391 | * offset to another bpf function |
| @@ -404,6 +432,12 @@ enum { · 40 unmodified lines | ||
| 404 | 432 | |
| 405 | 433 | /* Enable memory-mapping BPF map */ |
| 406 | 434 | BPF_F_MMAPABLE = (1U << 10), |
| 435 | + | |
| 436 | +/* Share perf_event among processes */ | |
| 437 | + BPF_F_PRESERVE_ELEMS = (1U << 11), | |
| 438 | + | |
| 439 | +/* Create a map that is suitable to be an inner map with dynamic max entries */ | |
| 440 | + BPF_F_INNER_MAP = (1U << 12), | |
| 407 | 441 | }; |
| 408 | 442 | |
| 409 | 443 | /* Flags for BPF_PROG_QUERY. */ |
| @@ -414,6 +448,11 @@ enum { · 4 unmodified lines | ||
| 414 | 448 | */ |
| 415 | 449 | #define BPF_F_QUERY_EFFECTIVE (1U << 0) |
| 416 | 450 | |
| 451 | +/* Flags for BPF_PROG_TEST_RUN */ | |
| 452 | + | |
| 453 | +/* If set, run the test on the cpu specified by bpf_attr.test.cpu */ | |
| 454 | +#define BPF_F_TEST_RUN_ON_CPU (1U << 0) | |
| 455 | + | |
| 417 | 456 | /* type for BPF_ENABLE_STATS */ |
| 418 | 457 | enum bpf_stats_type { |
| 419 | 458 | /* enabled run_time_ns and run_cnt */ |
| @@ -556,6 +595,8 @@ union bpf_attr { · 136 unmodified lines | ||
| 556 | 595 | */ |
| 557 | 596 | __aligned_u64 ctx_in; |
| 558 | 597 | __aligned_u64 ctx_out; |
| 598 | + __u32 flags; | |
| 599 | + __u32 cpu; | |
| 559 | 600 | } test; |
| 560 | 601 | |
| 561 | 602 | struct { /* anonymous struct used by BPF_*_GET_*_ID */ |
| @@ -622,8 +663,13 @@ union bpf_attr { · 60 unmodified lines | ||
| 622 | 663 | }; |
| 623 | 664 | __u32 attach_type; /* attach type */ |
| 624 | 665 | __u32 flags; /* extra flags */ |
| 625 | − __aligned_u64 iter_info; /* extra bpf_iter_link_info */ | |
| 626 | − __u32 iter_info_len; /* iter_info length */ | |
| 666 | + union { | |
| 667 | + __u32 target_btf_id; /* btf_id of target to attach to */ | |
| 668 | + struct { | |
| 669 | + __aligned_u64 iter_info; /* extra bpf_iter_link_info */ | |
| 670 | + __u32 iter_info_len; /* iter_info length */ | |
| 671 | + }; | |
| 672 | + }; | |
| 627 | 673 | } link_create; |
| 628 | 674 | |
| 629 | 675 | struct { /* struct used by BPF_LINK_UPDATE command */ |
| @@ -649,6 +695,12 @@ union bpf_attr { · 19 unmodified lines | ||
| 649 | 695 | __u32 flags; |
| 650 | 696 | } iter_create; |
| 651 | 697 | |
| 698 | + struct { /* struct used by BPF_PROG_BIND_MAP command */ | |
| 699 | + __u32 prog_fd; | |
| 700 | + __u32 map_fd; | |
| 701 | + __u32 flags; /* extra flags */ | |
| 702 | + } prog_bind_map; | |
| 703 | + | |
| 652 | 704 | } __attribute__((aligned(8))); |
| 653 | 705 | |
| 654 | 706 | /* The description below is an attempt at providing documentation to eBPF |
| @@ -1438,8 +1490,8 @@ union bpf_attr { · 783 unmodified lines | ||
| 1438 | 1490 | * Return |
| 1439 | 1491 | * The return value depends on the result of the test, and can be: |
| 1440 | 1492 | * |
| 1441 | − * * 0, if the *skb* task belongs to the cgroup2. | |
| 1442 | − * * 1, if the *skb* task does not belong to the cgroup2. | |
| 1493 | + * * 0, if current task belongs to the cgroup2. | |
| 1494 | + * * 1, if current task does not belong to the cgroup2. | |
| 1443 | 1495 | * * A negative error code, if an error occurred. |
| 1444 | 1496 | * |
| 1445 | 1497 | * long bpf_skb_change_tail(struct sk_buff *skb, u32 len, u64 flags) |
| @@ -1649,7 +1701,7 @@ union bpf_attr { · 203 unmodified lines | ||
| 1649 | 1701 | * **TCP_CONGESTION**, **TCP_BPF_IW**, |
| 1650 | 1702 | * **TCP_BPF_SNDCWND_CLAMP**, **TCP_SAVE_SYN**, |
| 1651 | 1703 | * **TCP_KEEPIDLE**, **TCP_KEEPINTVL**, **TCP_KEEPCNT**, |
| 1652 | − * **TCP_SYNCNT**, **TCP_USER_TIMEOUT**. | |
| 1704 | + * **TCP_SYNCNT**, **TCP_USER_TIMEOUT**, **TCP_NOTSENT_LOWAT**. | |
| 1653 | 1705 | * * **IPPROTO_IP**, which supports *optname* **IP_TOS**. |
| 1654 | 1706 | * * **IPPROTO_IPV6**, which supports *optname* **IPV6_TCLASS**. |
| 1655 | 1707 | * Return |
| @@ -2204,7 +2256,7 @@ union bpf_attr { · 548 unmodified lines | ||
| 2204 | 2256 | * Description |
| 2205 | 2257 | * This helper is used in programs implementing policies at the |
| 2206 | 2258 | * skb socket level. If the sk_buff *skb* is allowed to pass (i.e. |
| 2207 | − * if the verdeict eBPF program returns **SK_PASS**), redirect it | |
| 2259 | + * if the verdict eBPF program returns **SK_PASS**), redirect it | |
| 2208 | 2260 | * to the socket referenced by *map* (of type |
| 2209 | 2261 | * **BPF_MAP_TYPE_SOCKHASH**) using hash *key*. Both ingress and |
| 2210 | 2262 | * egress interfaces can be used for redirection. The |
| @@ -2496,7 +2548,7 @@ union bpf_attr { · 285 unmodified lines | ||
| 2496 | 2548 | * result is from *reuse*\ **->socks**\ [] using the hash of the |
| 2497 | 2549 | * tuple. |
| 2498 | 2550 | * |
| 2499 | − * long bpf_sk_release(struct bpf_sock *sock) | |
| 2551 | + * long bpf_sk_release(void *sock) | |
| 2500 | 2552 | * Description |
| 2501 | 2553 | * Release the reference held by *sock*. *sock* must be a |
| 2502 | 2554 | * non-**NULL** pointer that was returned from |
| @@ -2676,7 +2728,7 @@ union bpf_attr { · 173 unmodified lines | ||
| 2676 | 2728 | * result is from *reuse*\ **->socks**\ [] using the hash of the |
| 2677 | 2729 | * tuple. |
| 2678 | 2730 | * |
| 2679 | − * long bpf_tcp_check_syncookie(struct bpf_sock *sk, void *iph, u32 iph_len, struct tcphdr *th, u32 th_len) | |
| 2731 | + * long bpf_tcp_check_syncookie(void *sk, void *iph, u32 iph_len, struct tcphdr *th, u32 th_len) | |
| 2680 | 2732 | * Description |
| 2681 | 2733 | * Check whether *iph* and *th* contain a valid SYN cookie ACK for |
| 2682 | 2734 | * the listening socket in *sk*. |
| @@ -2807,7 +2859,7 @@ union bpf_attr { · 124 unmodified lines | ||
| 2807 | 2859 | * |
| 2808 | 2860 | * **-ERANGE** if resulting value was out of range. |
| 2809 | 2861 | * |
| 2810 | − * void *bpf_sk_storage_get(struct bpf_map *map, struct bpf_sock *sk, void *value, u64 flags) | |
| 2862 | + * void *bpf_sk_storage_get(struct bpf_map *map, void *sk, void *value, u64 flags) | |
| 2811 | 2863 | * Description |
| 2812 | 2864 | * Get a bpf-local-storage from a *sk*. |
| 2813 | 2865 | * |
| @@ -2823,6 +2875,9 @@ union bpf_attr { · 9 unmodified lines | ||
| 2823 | 2875 | * "type". The bpf-local-storage "type" (i.e. the *map*) is |
| 2824 | 2876 | * searched against all bpf-local-storages residing at *sk*. |
| 2825 | 2877 | * |
| 2878 | + * *sk* is a kernel **struct sock** pointer for LSM program. | |
| 2879 | + * *sk* is a **struct bpf_sock** pointer for other program types. | |
| 2880 | + * | |
| 2826 | 2881 | * An optional *flags* (**BPF_SK_STORAGE_GET_F_CREATE**) can be |
| 2827 | 2882 | * used such that a new bpf-local-storage will be |
| 2828 | 2883 | * created if one does not exist. *value* can be used |
| @@ -2835,13 +2890,14 @@ union bpf_attr { · 6 unmodified lines | ||
| 2835 | 2890 | * **NULL** if not found or there was an error in adding |
| 2836 | 2891 | * a new bpf-local-storage. |
| 2837 | 2892 | * |
| 2838 | − * long bpf_sk_storage_delete(struct bpf_map *map, struct bpf_sock *sk) | |
| 2893 | + * long bpf_sk_storage_delete(struct bpf_map *map, void *sk) | |
| 2839 | 2894 | * Description |
| 2840 | 2895 | * Delete a bpf-local-storage from a *sk*. |
| 2841 | 2896 | * Return |
| 2842 | 2897 | * 0 on success. |
| 2843 | 2898 | * |
| 2844 | 2899 | * **-ENOENT** if the bpf-local-storage cannot be found. |
| 2900 | + * **-EINVAL** if sk is not a fullsock (e.g. a request_sock). | |
| 2845 | 2901 | * |
| 2846 | 2902 | * long bpf_send_signal(u32 sig) |
| 2847 | 2903 | * Description |
| @@ -2858,7 +2914,7 @@ union bpf_attr { · 10 unmodified lines | ||
| 2858 | 2914 | * |
| 2859 | 2915 | * **-EAGAIN** if bpf program can try again. |
| 2860 | 2916 | * |
| 2861 | − * s64 bpf_tcp_gen_syncookie(struct bpf_sock *sk, void *iph, u32 iph_len, struct tcphdr *th, u32 th_len) | |
| 2917 | + * s64 bpf_tcp_gen_syncookie(void *sk, void *iph, u32 iph_len, struct tcphdr *th, u32 th_len) | |
| 2862 | 2918 | * Description |
| 2863 | 2919 | * Try to issue a SYN cookie for the packet with corresponding |
| 2864 | 2920 | * IP/TCP headers, *iph* and *th*, on the listening socket in *sk*. |
| @@ -3087,7 +3143,7 @@ union bpf_attr { · 222 unmodified lines | ||
| 3087 | 3143 | * Return |
| 3088 | 3144 | * The id is returned or 0 in case the id could not be retrieved. |
| 3089 | 3145 | * |
| 3090 | − * long bpf_sk_assign(struct sk_buff *skb, struct bpf_sock *sk, u64 flags) | |
| 3146 | + * long bpf_sk_assign(struct sk_buff *skb, void *sk, u64 flags) | |
| 3091 | 3147 | * Description |
| 3092 | 3148 | * Helper is overloaded depending on BPF program type. This |
| 3093 | 3149 | * description applies to **BPF_PROG_TYPE_SCHED_CLS** and |
| @@ -3215,11 +3271,11 @@ union bpf_attr { · 121 unmodified lines | ||
| 3215 | 3271 | * |
| 3216 | 3272 | * **-EOVERFLOW** if an overflow happened: The same object will be tried again. |
| 3217 | 3273 | * |
| 3218 | − * u64 bpf_sk_cgroup_id(struct bpf_sock *sk) | |
| 3274 | + * u64 bpf_sk_cgroup_id(void *sk) | |
| 3219 | 3275 | * Description |
| 3220 | 3276 | * Return the cgroup v2 id of the socket *sk*. |
| 3221 | 3277 | * |
| 3222 | − * *sk* must be a non-**NULL** pointer to a full socket, e.g. one | |
| 3278 | + * *sk* must be a non-**NULL** pointer to a socket, e.g. one | |
| 3223 | 3279 | * returned from **bpf_sk_lookup_xxx**\ (), |
| 3224 | 3280 | * **bpf_sk_fullsock**\ (), etc. The format of returned id is |
| 3225 | 3281 | * same as in **bpf_skb_cgroup_id**\ (). |
| @@ -3229,7 +3285,7 @@ union bpf_attr { · 3 unmodified lines | ||
| 3229 | 3285 | * Return |
| 3230 | 3286 | * The id is returned or 0 in case the id could not be retrieved. |
| 3231 | 3287 | * |
| 3232 | − * u64 bpf_sk_ancestor_cgroup_id(struct bpf_sock *sk, int ancestor_level) | |
| 3288 | + * u64 bpf_sk_ancestor_cgroup_id(void *sk, int ancestor_level) | |
| 3233 | 3289 | * Description |
| 3234 | 3290 | * Return id of cgroup v2 that is ancestor of cgroup associated |
| 3235 | 3291 | * with the *sk* at the *ancestor_level*. The root cgroup is at |
| @@ -3337,38 +3393,38 @@ union bpf_attr { · 101 unmodified lines | ||
| 3337 | 3393 | * Description |
| 3338 | 3394 | * Dynamically cast a *sk* pointer to a *tcp6_sock* pointer. |
| 3339 | 3395 | * Return |
| 3340 | − * *sk* if casting is valid, or NULL otherwise. | |
| 3396 | + * *sk* if casting is valid, or **NULL** otherwise. | |
| 3341 | 3397 | * |
| 3342 | 3398 | * struct tcp_sock *bpf_skc_to_tcp_sock(void *sk) |
| 3343 | 3399 | * Description |
| 3344 | 3400 | * Dynamically cast a *sk* pointer to a *tcp_sock* pointer. |
| 3345 | 3401 | * Return |
| 3346 | − * *sk* if casting is valid, or NULL otherwise. | |
| 3402 | + * *sk* if casting is valid, or **NULL** otherwise. | |
| 3347 | 3403 | * |
| 3348 | 3404 | * struct tcp_timewait_sock *bpf_skc_to_tcp_timewait_sock(void *sk) |
| 3349 | 3405 | * Description |
| 3350 | 3406 | * Dynamically cast a *sk* pointer to a *tcp_timewait_sock* pointer. |
| 3351 | 3407 | * Return |
| 3352 | − * *sk* if casting is valid, or NULL otherwise. | |
| 3408 | + * *sk* if casting is valid, or **NULL** otherwise. | |
| 3353 | 3409 | * |
| 3354 | 3410 | * struct tcp_request_sock *bpf_skc_to_tcp_request_sock(void *sk) |
| 3355 | 3411 | * Description |
| 3356 | 3412 | * Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. |
| 3357 | 3413 | * Return |
| 3358 | − * *sk* if casting is valid, or NULL otherwise. | |
| 3414 | + * *sk* if casting is valid, or **NULL** otherwise. | |
| 3359 | 3415 | * |
| 3360 | 3416 | * struct udp6_sock *bpf_skc_to_udp6_sock(void *sk) |
| 3361 | 3417 | * Description |
| 3362 | 3418 | * Dynamically cast a *sk* pointer to a *udp6_sock* pointer. |
| 3363 | 3419 | * Return |
| 3364 | − * *sk* if casting is valid, or NULL otherwise. | |
| 3420 | + * *sk* if casting is valid, or **NULL** otherwise. | |
| 3365 | 3421 | * |
| 3366 | 3422 | * long bpf_get_task_stack(struct task_struct *task, void *buf, u32 size, u64 flags) |
| 3367 | 3423 | * Description |
| 3368 | 3424 | * Return a user or a kernel stack in bpf program provided buffer. |
| 3369 | 3425 | * To achieve this, the helper needs *task*, which is a valid |
| 3370 | − * pointer to struct task_struct. To store the stacktrace, the | |
| 3371 | − * bpf program provides *buf* with a nonnegative *size*. | |
| 3426 | + * pointer to **struct task_struct**. To store the stacktrace, the | |
| 3427 | + * bpf program provides *buf* with a nonnegative *size*. | |
| 3372 | 3428 | * |
| 3373 | 3429 | * The last argument, *flags*, holds the number of stack frames to |
| 3374 | 3430 | * skip (from 0 to 255), masked with |
| @@ -3395,6 +3451,293 @@ union bpf_attr { · 20 unmodified lines | ||
| 3395 | 3451 | * A non-negative value equal to or less than *size* on success, |
| 3396 | 3452 | * or a negative error in case of failure. |
| 3397 | 3453 | * |
| 3454 | + * long bpf_load_hdr_opt(struct bpf_sock_ops *skops, void *searchby_res, u32 len, u64 flags) | |
| 3455 | + * Description | |
| 3456 | + * Load header option. Support reading a particular TCP header | |
| 3457 | + * option for bpf program (**BPF_PROG_TYPE_SOCK_OPS**). | |
| 3458 | + * | |
| 3459 | + * If *flags* is 0, it will search the option from the | |
| 3460 | + * *skops*\ **->skb_data**. The comment in **struct bpf_sock_ops** | |
| 3461 | + * has details on what skb_data contains under different | |
| 3462 | + * *skops*\ **->op**. | |
| 3463 | + * | |
| 3464 | + * The first byte of the *searchby_res* specifies the | |
| 3465 | + * kind that it wants to search. | |
| 3466 | + * | |
| 3467 | + * If the searching kind is an experimental kind | |
| 3468 | + * (i.e. 253 or 254 according to RFC6994). It also | |
| 3469 | + * needs to specify the "magic" which is either | |
| 3470 | + * 2 bytes or 4 bytes. It then also needs to | |
| 3471 | + * specify the size of the magic by using | |
| 3472 | + * the 2nd byte which is "kind-length" of a TCP | |
| 3473 | + * header option and the "kind-length" also | |
| 3474 | + * includes the first 2 bytes "kind" and "kind-length" | |
| 3475 | + * itself as a normal TCP header option also does. | |
| 3476 | + * | |
| 3477 | + * For example, to search experimental kind 254 with | |
| 3478 | + * 2 byte magic 0xeB9F, the searchby_res should be | |
| 3479 | + * [ 254, 4, 0xeB, 0x9F, 0, 0, .... 0 ]. | |
| 3480 | + * | |
| 3481 | + * To search for the standard window scale option (3), | |
| 3482 | + * the *searchby_res* should be [ 3, 0, 0, .... 0 ]. | |
| 3483 | + * Note, kind-length must be 0 for regular option. | |
| 3484 | + * | |
| 3485 | + * Searching for No-Op (0) and End-of-Option-List (1) are | |
| 3486 | + * not supported. | |
| 3487 | + * | |
| 3488 | + * *len* must be at least 2 bytes which is the minimal size | |
| 3489 | + * of a header option. | |
| 3490 | + * | |
| 3491 | + * Supported flags: | |
| 3492 | + * | |
| 3493 | + * * **BPF_LOAD_HDR_OPT_TCP_SYN** to search from the | |
| 3494 | + * saved_syn packet or the just-received syn packet. | |
| 3495 | + * | |
| 3496 | + * Return | |
| 3497 | + * > 0 when found, the header option is copied to *searchby_res*. | |
| 3498 | + * The return value is the total length copied. On failure, a | |
| 3499 | + * negative error code is returned: | |
| 3500 | + * | |
| 3501 | + * **-EINVAL** if a parameter is invalid. | |
| 3502 | + * | |
| 3503 | + * **-ENOMSG** if the option is not found. | |
| 3504 | + * | |
| 3505 | + * **-ENOENT** if no syn packet is available when | |
| 3506 | + * **BPF_LOAD_HDR_OPT_TCP_SYN** is used. | |
| 3507 | + * | |
| 3508 | + * **-ENOSPC** if there is not enough space. Only *len* number of | |
| 3509 | + * bytes are copied. | |
| 3510 | + * | |
| 3511 | + * **-EFAULT** on failure to parse the header options in the | |
| 3512 | + * packet. | |
| 3513 | + * | |
| 3514 | + * **-EPERM** if the helper cannot be used under the current | |
| 3515 | + * *skops*\ **->op**. | |
| 3516 | + * | |
| 3517 | + * long bpf_store_hdr_opt(struct bpf_sock_ops *skops, const void *from, u32 len, u64 flags) | |
| 3518 | + * Description | |
| 3519 | + * Store header option. The data will be copied | |
| 3520 | + * from buffer *from* with length *len* to the TCP header. | |
| 3521 | + * | |
| 3522 | + * The buffer *from* should have the whole option that | |
| 3523 | + * includes the kind, kind-length, and the actual | |
| 3524 | + * option data. The *len* must be at least kind-length | |
| 3525 | + * long. The kind-length does not have to be 4 byte | |
| 3526 | + * aligned. The kernel will take care of the padding | |
| 3527 | + * and setting the 4 bytes aligned value to th->doff. | |
| 3528 | + * | |
| 3529 | + * This helper will check for duplicated option | |
| 3530 | + * by searching the same option in the outgoing skb. | |
| 3531 | + * | |
| 3532 | + * This helper can only be called during | |
| 3533 | + * **BPF_SOCK_OPS_WRITE_HDR_OPT_CB**. | |
| 3534 | + * | |
| 3535 | + * Return | |
| 3536 | + * 0 on success, or negative error in case of failure: | |
| 3537 | + * | |
| 3538 | + * **-EINVAL** If param is invalid. | |
| 3539 | + * | |
| 3540 | + * **-ENOSPC** if there is not enough space in the header. | |
| 3541 | + * Nothing has been written | |
| 3542 | + * | |
| 3543 | + * **-EEXIST** if the option already exists. | |
| 3544 | + * | |
| 3545 | + * **-EFAULT** on failrue to parse the existing header options. | |
| 3546 | + * | |
| 3547 | + * **-EPERM** if the helper cannot be used under the current | |
| 3548 | + * *skops*\ **->op**. | |
| 3549 | + * | |
| 3550 | + * long bpf_reserve_hdr_opt(struct bpf_sock_ops *skops, u32 len, u64 flags) | |
| 3551 | + * Description | |
| 3552 | + * Reserve *len* bytes for the bpf header option. The | |
| 3553 | + * space will be used by **bpf_store_hdr_opt**\ () later in | |
| 3554 | + * **BPF_SOCK_OPS_WRITE_HDR_OPT_CB**. | |
| 3555 | + * | |
| 3556 | + * If **bpf_reserve_hdr_opt**\ () is called multiple times, | |
| 3557 | + * the total number of bytes will be reserved. | |
| 3558 | + * | |
| 3559 | + * This helper can only be called during | |
| 3560 | + * **BPF_SOCK_OPS_HDR_OPT_LEN_CB**. | |
| 3561 | + * | |
| 3562 | + * Return | |
| 3563 | + * 0 on success, or negative error in case of failure: | |
| 3564 | + * | |
| 3565 | + * **-EINVAL** if a parameter is invalid. | |
| 3566 | + * | |
| 3567 | + * **-ENOSPC** if there is not enough space in the header. | |
| 3568 | + * | |
| 3569 | + * **-EPERM** if the helper cannot be used under the current | |
| 3570 | + * *skops*\ **->op**. | |
| 3571 | + * | |
| 3572 | + * void *bpf_inode_storage_get(struct bpf_map *map, void *inode, void *value, u64 flags) | |
| 3573 | + * Description | |
| 3574 | + * Get a bpf_local_storage from an *inode*. | |
| 3575 | + * | |
| 3576 | + * Logically, it could be thought of as getting the value from | |
| 3577 | + * a *map* with *inode* as the **key**. From this | |
| 3578 | + * perspective, the usage is not much different from | |
| 3579 | + * **bpf_map_lookup_elem**\ (*map*, **&**\ *inode*) except this | |
| 3580 | + * helper enforces the key must be an inode and the map must also | |
| 3581 | + * be a **BPF_MAP_TYPE_INODE_STORAGE**. | |
| 3582 | + * | |
| 3583 | + * Underneath, the value is stored locally at *inode* instead of | |
| 3584 | + * the *map*. The *map* is used as the bpf-local-storage | |
| 3585 | + * "type". The bpf-local-storage "type" (i.e. the *map*) is | |
| 3586 | + * searched against all bpf_local_storage residing at *inode*. | |
| 3587 | + * | |
| 3588 | + * An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be | |
| 3589 | + * used such that a new bpf_local_storage will be | |
| 3590 | + * created if one does not exist. *value* can be used | |
| 3591 | + * together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify | |
| 3592 | + * the initial value of a bpf_local_storage. If *value* is | |
| 3593 | + * **NULL**, the new bpf_local_storage will be zero initialized. | |
| 3594 | + * Return | |
| 3595 | + * A bpf_local_storage pointer is returned on success. | |
| 3596 | + * | |
| 3597 | + * **NULL** if not found or there was an error in adding | |
| 3598 | + * a new bpf_local_storage. | |
| 3599 | + * | |
| 3600 | + * int bpf_inode_storage_delete(struct bpf_map *map, void *inode) | |
| 3601 | + * Description | |
| 3602 | + * Delete a bpf_local_storage from an *inode*. | |
| 3603 | + * Return | |
| 3604 | + * 0 on success. | |
| 3605 | + * | |
| 3606 | + * **-ENOENT** if the bpf_local_storage cannot be found. | |
| 3607 | + * | |
| 3608 | + * long bpf_d_path(struct path *path, char *buf, u32 sz) | |
| 3609 | + * Description | |
| 3610 | + * Return full path for given **struct path** object, which | |
| 3611 | + * needs to be the kernel BTF *path* object. The path is | |
| 3612 | + * returned in the provided buffer *buf* of size *sz* and | |
| 3613 | + * is zero terminated. | |
| 3614 | + * | |
| 3615 | + * Return | |
| 3616 | + * On success, the strictly positive length of the string, | |
| 3617 | + * including the trailing NUL character. On error, a negative | |
| 3618 | + * value. | |
| 3619 | + * | |
| 3620 | + * long bpf_copy_from_user(void *dst, u32 size, const void *user_ptr) | |
| 3621 | + * Description | |
| 3622 | + * Read *size* bytes from user space address *user_ptr* and store | |
| 3623 | + * the data in *dst*. This is a wrapper of **copy_from_user**\ (). | |
| 3624 | + * Return | |
| 3625 | + * 0 on success, or a negative error in case of failure. | |
| 3626 | + * | |
| 3627 | + * long bpf_snprintf_btf(char *str, u32 str_size, struct btf_ptr *ptr, u32 btf_ptr_size, u64 flags) | |
| 3628 | + * Description | |
| 3629 | + * Use BTF to store a string representation of *ptr*->ptr in *str*, | |
| 3630 | + * using *ptr*->type_id. This value should specify the type | |
| 3631 | + * that *ptr*->ptr points to. LLVM __builtin_btf_type_id(type, 1) | |
| 3632 | + * can be used to look up vmlinux BTF type ids. Traversing the | |
| 3633 | + * data structure using BTF, the type information and values are | |
| 3634 | + * stored in the first *str_size* - 1 bytes of *str*. Safe copy of | |
| 3635 | + * the pointer data is carried out to avoid kernel crashes during | |
| 3636 | + * operation. Smaller types can use string space on the stack; | |
| 3637 | + * larger programs can use map data to store the string | |
| 3638 | + * representation. | |
| 3639 | + * | |
| 3640 | + * The string can be subsequently shared with userspace via | |
| 3641 | + * bpf_perf_event_output() or ring buffer interfaces. | |
| 3642 | + * bpf_trace_printk() is to be avoided as it places too small | |
| 3643 | + * a limit on string size to be useful. | |
| 3644 | + * | |
| 3645 | + * *flags* is a combination of | |
| 3646 | + * | |
| 3647 | + * **BTF_F_COMPACT** | |
| 3648 | + * no formatting around type information | |
| 3649 | + * **BTF_F_NONAME** | |
| 3650 | + * no struct/union member names/types | |
| 3651 | + * **BTF_F_PTR_RAW** | |
| 3652 | + * show raw (unobfuscated) pointer values; | |
| 3653 | + * equivalent to printk specifier %px. | |
| 3654 | + * **BTF_F_ZERO** | |
| 3655 | + * show zero-valued struct/union members; they | |
| 3656 | + * are not displayed by default | |
| 3657 | + * | |
| 3658 | + * Return | |
| 3659 | + * The number of bytes that were written (or would have been | |
| 3660 | + * written if output had to be truncated due to string size), | |
| 3661 | + * or a negative error in cases of failure. | |
| 3662 | + * | |
| 3663 | + * long bpf_seq_printf_btf(struct seq_file *m, struct btf_ptr *ptr, u32 ptr_size, u64 flags) | |
| 3664 | + * Description | |
| 3665 | + * Use BTF to write to seq_write a string representation of | |
| 3666 | + * *ptr*->ptr, using *ptr*->type_id as per bpf_snprintf_btf(). | |
| 3667 | + * *flags* are identical to those used for bpf_snprintf_btf. | |
| 3668 | + * Return | |
| 3669 | + * 0 on success or a negative error in case of failure. | |
| 3670 | + * | |
| 3671 | + * u64 bpf_skb_cgroup_classid(struct sk_buff *skb) | |
| 3672 | + * Description | |
| 3673 | + * See **bpf_get_cgroup_classid**\ () for the main description. | |
| 3674 | + * This helper differs from **bpf_get_cgroup_classid**\ () in that | |
| 3675 | + * the cgroup v1 net_cls class is retrieved only from the *skb*'s | |
| 3676 | + * associated socket instead of the current process. | |
| 3677 | + * Return | |
| 3678 | + * The id is returned or 0 in case the id could not be retrieved. | |
| 3679 | + * | |
| 3680 | + * long bpf_redirect_neigh(u32 ifindex, u64 flags) | |
| 3681 | + * Description | |
| 3682 | + * Redirect the packet to another net device of index *ifindex* | |
| 3683 | + * and fill in L2 addresses from neighboring subsystem. This helper | |
| 3684 | + * is somewhat similar to **bpf_redirect**\ (), except that it | |
| 3685 | + * populates L2 addresses as well, meaning, internally, the helper | |
| 3686 | + * performs a FIB lookup based on the skb's networking header to | |
| 3687 | + * get the address of the next hop and then relies on the neighbor | |
| 3688 | + * lookup for the L2 address of the nexthop. | |
| 3689 | + * | |
| 3690 | + * The *flags* argument is reserved and must be 0. The helper is | |
| 3691 | + * currently only supported for tc BPF program types, and enabled | |
| 3692 | + * for IPv4 and IPv6 protocols. | |
| 3693 | + * Return | |
| 3694 | + * The helper returns **TC_ACT_REDIRECT** on success or | |
| 3695 | + * **TC_ACT_SHOT** on error. | |
| 3696 | + * | |
| 3697 | + * void *bpf_per_cpu_ptr(const void *percpu_ptr, u32 cpu) | |
| 3698 | + * Description | |
| 3699 | + * Take a pointer to a percpu ksym, *percpu_ptr*, and return a | |
| 3700 | + * pointer to the percpu kernel variable on *cpu*. A ksym is an | |
| 3701 | + * extern variable decorated with '__ksym'. For ksym, there is a | |
| 3702 | + * global var (either static or global) defined of the same name | |
| 3703 | + * in the kernel. The ksym is percpu if the global var is percpu. | |
| 3704 | + * The returned pointer points to the global percpu var on *cpu*. | |
| 3705 | + * | |
| 3706 | + * bpf_per_cpu_ptr() has the same semantic as per_cpu_ptr() in the | |
| 3707 | + * kernel, except that bpf_per_cpu_ptr() may return NULL. This | |
| 3708 | + * happens if *cpu* is larger than nr_cpu_ids. The caller of | |
| 3709 | + * bpf_per_cpu_ptr() must check the returned value. | |
| 3710 | + * Return | |
| 3711 | + * A pointer pointing to the kernel percpu variable on *cpu*, or | |
| 3712 | + * NULL, if *cpu* is invalid. | |
| 3713 | + * | |
| 3714 | + * void *bpf_this_cpu_ptr(const void *percpu_ptr) | |
| 3715 | + * Description | |
| 3716 | + * Take a pointer to a percpu ksym, *percpu_ptr*, and return a | |
| 3717 | + * pointer to the percpu kernel variable on this cpu. See the | |
| 3718 | + * description of 'ksym' in **bpf_per_cpu_ptr**\ (). | |
| 3719 | + * | |
| 3720 | + * bpf_this_cpu_ptr() has the same semantic as this_cpu_ptr() in | |
| 3721 | + * the kernel. Different from **bpf_per_cpu_ptr**\ (), it would | |
| 3722 | + * never return NULL. | |
| 3723 | + * Return | |
| 3724 | + * A pointer pointing to the kernel percpu variable on this cpu. | |
| 3725 | + * | |
| 3726 | + * long bpf_redirect_peer(u32 ifindex, u64 flags) | |
| 3727 | + * Description | |
| 3728 | + * Redirect the packet to another net device of index *ifindex*. | |
| 3729 | + * This helper is somewhat similar to **bpf_redirect**\ (), except | |
| 3730 | + * that the redirection happens to the *ifindex*' peer device and | |
| 3731 | + * the netns switch takes place from ingress to ingress without | |
| 3732 | + * going through the CPU's backlog queue. | |
| 3733 | + * | |
| 3734 | + * The *flags* argument is reserved and must be 0. The helper is | |
| 3735 | + * currently only supported for tc BPF program types at the ingress | |
| 3736 | + * hook and for veth device types. The peer device must reside in a | |
| 3737 | + * different network namespace. | |
| 3738 | + * Return | |
| 3739 | + * The helper returns **TC_ACT_REDIRECT** on success or | |
| 3740 | + * **TC_ACT_SHOT** on error. | |
| 3398 | 3741 | */ |
| 3399 | 3742 | #define __BPF_FUNC_MAPPER(FN) \ |
| 3400 | 3743 | FN(unspec), \ |
| @@ -3539,6 +3882,20 @@ union bpf_attr { · 138 unmodified lines | ||
| 3539 | 3882 | FN(skc_to_tcp_request_sock), \ |
| 3540 | 3883 | FN(skc_to_udp6_sock), \ |
| 3541 | 3884 | FN(get_task_stack), \ |
| 3885 | + FN(load_hdr_opt), \ | |
| 3886 | + FN(store_hdr_opt), \ | |
| 3887 | + FN(reserve_hdr_opt), \ | |
| 3888 | + FN(inode_storage_get), \ | |
| 3889 | + FN(inode_storage_delete), \ | |
| 3890 | + FN(d_path), \ | |
| 3891 | + FN(copy_from_user), \ | |
| 3892 | + FN(snprintf_btf), \ | |
| 3893 | + FN(seq_printf_btf), \ | |
| 3894 | + FN(skb_cgroup_classid), \ | |
| 3895 | + FN(redirect_neigh), \ | |
| 3896 | + FN(bpf_per_cpu_ptr), \ | |
| 3897 | + FN(bpf_this_cpu_ptr), \ | |
| 3898 | + FN(redirect_peer), \ | |
| 3542 | 3899 | /* */ |
| 3543 | 3900 | |
| 3544 | 3901 | /* integer value in 'imm' field of BPF_CALL instruction selects which helper |
| @@ -3648,9 +4005,13 @@ enum { · 103 unmodified lines | ||
| 3648 | 4005 | BPF_F_SYSCTL_BASE_NAME = (1ULL << 0), |
| 3649 | 4006 | }; |
| 3650 | 4007 | |
| 3651 | −/* BPF_FUNC_sk_storage_get flags */ | |
| 4008 | +/* BPF_FUNC_<kernel_obj>_storage_get flags */ | |
| 3652 | 4009 | enum { |
| 3653 | − BPF_SK_STORAGE_GET_F_CREATE = (1ULL << 0), | |
| 4010 | + BPF_LOCAL_STORAGE_GET_F_CREATE = (1ULL << 0), | |
| 4011 | + /* BPF_SK_STORAGE_GET_F_CREATE is only kept for backward compatibility | |
| 4012 | + * and BPF_LOCAL_STORAGE_GET_F_CREATE must be used instead. | |
| 4013 | + */ | |
| 4014 | + BPF_SK_STORAGE_GET_F_CREATE = BPF_LOCAL_STORAGE_GET_F_CREATE, | |
| 3654 | 4015 | }; |
| 3655 | 4016 | |
| 3656 | 4017 | /* BPF_FUNC_read_branch_records flags. */ |
| @@ -4071,6 +4432,15 @@ struct bpf_link_info { · 414 unmodified lines | ||
| 4071 | 4432 | __u64 cgroup_id; |
| 4072 | 4433 | __u32 attach_type; |
| 4073 | 4434 | } cgroup; |
| 4435 | + struct { | |
| 4436 | + __aligned_u64 target_name; /* in/out: target_name buffer ptr */ | |
| 4437 | + __u32 target_name_len; /* in/out: target_name buffer len */ | |
| 4438 | + union { | |
| 4439 | + struct { | |
| 4440 | + __u32 map_id; | |
| 4441 | + } map; | |
| 4442 | + }; | |
| 4443 | + } iter; | |
| 4074 | 4444 | struct { |
| 4075 | 4445 | __u32 netns_ino; |
| 4076 | 4446 | __u32 attach_type; |
| @@ -4158,6 +4528,36 @@ struct bpf_sock_ops { · 81 unmodified lines | ||
| 4158 | 4528 | __u64 bytes_received; |
| 4159 | 4529 | __u64 bytes_acked; |
| 4160 | 4530 | __bpf_md_ptr(struct bpf_sock *, sk); |
| 4531 | + /* [skb_data, skb_data_end) covers the whole TCP header. | |
| 4532 | + * | |
| 4533 | + * BPF_SOCK_OPS_PARSE_HDR_OPT_CB: The packet received | |
| 4534 | + * BPF_SOCK_OPS_HDR_OPT_LEN_CB: Not useful because the | |
| 4535 | + * header has not been written. | |
| 4536 | + * BPF_SOCK_OPS_WRITE_HDR_OPT_CB: The header and options have | |
| 4537 | + * been written so far. | |
| 4538 | + * BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB: The SYNACK that concludes | |
| 4539 | + * the 3WHS. | |
| 4540 | + * BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB: The ACK that concludes | |
| 4541 | + * the 3WHS. | |
| 4542 | + * | |
| 4543 | + * bpf_load_hdr_opt() can also be used to read a particular option. | |
| 4544 | + */ | |
| 4545 | + __bpf_md_ptr(void *, skb_data); | |
| 4546 | + __bpf_md_ptr(void *, skb_data_end); | |
| 4547 | + __u32 skb_len; /* The total length of a packet. | |
| 4548 | + * It includes the header, options, | |
| 4549 | + * and payload. | |
| 4550 | + */ | |
| 4551 | + __u32 skb_tcp_flags; /* tcp_flags of the header. It provides | |
| 4552 | + * an easy way to check for tcp_flags | |
| 4553 | + * without parsing skb_data. | |
| 4554 | + * | |
| 4555 | + * In particular, the skb_tcp_flags | |
| 4556 | + * will still be available in | |
| 4557 | + * BPF_SOCK_OPS_HDR_OPT_LEN even though | |
| 4558 | + * the outgoing header has not | |
| 4559 | + * been written yet. | |
| 4560 | + */ | |
| 4161 | 4561 | }; |
| 4162 | 4562 | |
| 4163 | 4563 | /* Definitions for bpf_sock_ops_cb_flags */ |
| @@ -4166,8 +4566,51 @@ enum { · 2 unmodified lines | ||
| 4166 | 4566 | BPF_SOCK_OPS_RETRANS_CB_FLAG = (1<<1), |
| 4167 | 4567 | BPF_SOCK_OPS_STATE_CB_FLAG = (1<<2), |
| 4168 | 4568 | BPF_SOCK_OPS_RTT_CB_FLAG = (1<<3), |
| 4569 | + /* Call bpf for all received TCP headers. The bpf prog will be | |
| 4570 | + * called under sock_ops->op == BPF_SOCK_OPS_PARSE_HDR_OPT_CB | |
| 4571 | + * | |
| 4572 | + * Please refer to the comment in BPF_SOCK_OPS_PARSE_HDR_OPT_CB | |
| 4573 | + * for the header option related helpers that will be useful | |
| 4574 | + * to the bpf programs. | |
| 4575 | + * | |
| 4576 | + * It could be used at the client/active side (i.e. connect() side) | |
| 4577 | + * when the server told it that the server was in syncookie | |
| 4578 | + * mode and required the active side to resend the bpf-written | |
| 4579 | + * options. The active side can keep writing the bpf-options until | |
| 4580 | + * it received a valid packet from the server side to confirm | |
| 4581 | + * the earlier packet (and options) has been received. The later | |
| 4582 | + * example patch is using it like this at the active side when the | |
| 4583 | + * server is in syncookie mode. | |
| 4584 | + * | |
| 4585 | + * The bpf prog will usually turn this off in the common cases. | |
| 4586 | + */ | |
| 4587 | + BPF_SOCK_OPS_PARSE_ALL_HDR_OPT_CB_FLAG = (1<<4), | |
| 4588 | + /* Call bpf when kernel has received a header option that | |
| 4589 | + * the kernel cannot handle. The bpf prog will be called under | |
| 4590 | + * sock_ops->op == BPF_SOCK_OPS_PARSE_HDR_OPT_CB. | |
| 4591 | + * | |
| 4592 | + * Please refer to the comment in BPF_SOCK_OPS_PARSE_HDR_OPT_CB | |
| 4593 | + * for the header option related helpers that will be useful | |
| 4594 | + * to the bpf programs. | |
| 4595 | + */ | |
| 4596 | + BPF_SOCK_OPS_PARSE_UNKNOWN_HDR_OPT_CB_FLAG = (1<<5), | |
| 4597 | + /* Call bpf when the kernel is writing header options for the | |
| 4598 | + * outgoing packet. The bpf prog will first be called | |
| 4599 | + * to reserve space in a skb under | |
| 4600 | + * sock_ops->op == BPF_SOCK_OPS_HDR_OPT_LEN_CB. Then | |
| 4601 | + * the bpf prog will be called to write the header option(s) | |
| 4602 | + * under sock_ops->op == BPF_SOCK_OPS_WRITE_HDR_OPT_CB. | |
| 4603 | + * | |
| 4604 | + * Please refer to the comment in BPF_SOCK_OPS_HDR_OPT_LEN_CB | |
| 4605 | + * and BPF_SOCK_OPS_WRITE_HDR_OPT_CB for the header option | |
| 4606 | + * related helpers that will be useful to the bpf programs. | |
| 4607 | + * | |
| 4608 | + * The kernel gets its chance to reserve space and write | |
| 4609 | + * options first before the BPF program does. | |
| 4610 | + */ | |
| 4611 | + BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG = (1<<6), | |
| 4169 | 4612 | /* Mask of all currently supported cb flags */ |
| 4170 | − BPF_SOCK_OPS_ALL_CB_FLAGS = 0xF, | |
| 4613 | + BPF_SOCK_OPS_ALL_CB_FLAGS = 0x7F, | |
| 4171 | 4614 | }; |
| 4172 | 4615 | |
| 4173 | 4616 | /* List of known BPF sock_ops operators. |
| @@ -4223,6 +4666,63 @@ enum { · 49 unmodified lines | ||
| 4223 | 4666 | */ |
| 4224 | 4667 | BPF_SOCK_OPS_RTT_CB, /* Called on every RTT. |
| 4225 | 4668 | */ |
| 4669 | + BPF_SOCK_OPS_PARSE_HDR_OPT_CB, /* Parse the header option. | |
| 4670 | + * It will be called to handle | |
| 4671 | + * the packets received at | |
| 4672 | + * an already established | |
| 4673 | + * connection. | |
| 4674 | + * | |
| 4675 | + * sock_ops->skb_data: | |
| 4676 | + * Referring to the received skb. | |
| 4677 | + * It covers the TCP header only. | |
| 4678 | + * | |
| 4679 | + * bpf_load_hdr_opt() can also | |
| 4680 | + * be used to search for a | |
| 4681 | + * particular option. | |
| 4682 | + */ | |
| 4683 | + BPF_SOCK_OPS_HDR_OPT_LEN_CB, /* Reserve space for writing the | |
| 4684 | + * header option later in | |
| 4685 | + * BPF_SOCK_OPS_WRITE_HDR_OPT_CB. | |
| 4686 | + * Arg1: bool want_cookie. (in | |
| 4687 | + * writing SYNACK only) | |
| 4688 | + * | |
| 4689 | + * sock_ops->skb_data: | |
| 4690 | + * Not available because no header has | |
| 4691 | + * been written yet. | |
| 4692 | + * | |
| 4693 | + * sock_ops->skb_tcp_flags: | |
| 4694 | + * The tcp_flags of the | |
| 4695 | + * outgoing skb. (e.g. SYN, ACK, FIN). | |
| 4696 | + * | |
| 4697 | + * bpf_reserve_hdr_opt() should | |
| 4698 | + * be used to reserve space. | |
| 4699 | + */ | |
| 4700 | + BPF_SOCK_OPS_WRITE_HDR_OPT_CB, /* Write the header options | |
| 4701 | + * Arg1: bool want_cookie. (in | |
| 4702 | + * writing SYNACK only) | |
| 4703 | + * | |
| 4704 | + * sock_ops->skb_data: | |
| 4705 | + * Referring to the outgoing skb. | |
| 4706 | + * It covers the TCP header | |
| 4707 | + * that has already been written | |
| 4708 | + * by the kernel and the | |
| 4709 | + * earlier bpf-progs. | |
| 4710 | + * | |
| 4711 | + * sock_ops->skb_tcp_flags: | |
| 4712 | + * The tcp_flags of the outgoing | |
| 4713 | + * skb. (e.g. SYN, ACK, FIN). | |
| 4714 | + * | |
| 4715 | + * bpf_store_hdr_opt() should | |
| 4716 | + * be used to write the | |
| 4717 | + * option. | |
| 4718 | + * | |
| 4719 | + * bpf_load_hdr_opt() can also | |
| 4720 | + * be used to search for a | |
| 4721 | + * particular option that | |
| 4722 | + * has already been written | |
| 4723 | + * by the kernel or the | |
| 4724 | + * earlier bpf-progs. | |
| 4725 | + */ | |
| 4226 | 4726 | }; |
| 4227 | 4727 | |
| 4228 | 4728 | /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect |
| @@ -4250,6 +4750,63 @@ enum { · 21 unmodified lines | ||
| 4250 | 4750 | enum { |
| 4251 | 4751 | TCP_BPF_IW = 1001, /* Set TCP initial congestion window */ |
| 4252 | 4752 | TCP_BPF_SNDCWND_CLAMP = 1002, /* Set sndcwnd_clamp */ |
| 4753 | + TCP_BPF_DELACK_MAX = 1003, /* Max delay ack in usecs */ | |
| 4754 | + TCP_BPF_RTO_MIN = 1004, /* Min delay ack in usecs */ | |
| 4755 | + /* Copy the SYN pkt to optval | |
| 4756 | + * | |
| 4757 | + * BPF_PROG_TYPE_SOCK_OPS only. It is similar to the | |
| 4758 | + * bpf_getsockopt(TCP_SAVED_SYN) but it does not limit | |
| 4759 | + * to only getting from the saved_syn. It can either get the | |
| 4760 | + * syn packet from: | |
| 4761 | + * | |
| 4762 | + * 1. the just-received SYN packet (only available when writing the | |
| 4763 | + * SYNACK). It will be useful when it is not necessary to | |
| 4764 | + * save the SYN packet for latter use. It is also the only way | |
| 4765 | + * to get the SYN during syncookie mode because the syn | |
| 4766 | + * packet cannot be saved during syncookie. | |
| 4767 | + * | |
| 4768 | + * OR | |
| 4769 | + * | |
| 4770 | + * 2. the earlier saved syn which was done by | |
| 4771 | + * bpf_setsockopt(TCP_SAVE_SYN). | |
| 4772 | + * | |
| 4773 | + * The bpf_getsockopt(TCP_BPF_SYN*) option will hide where the | |
| 4774 | + * SYN packet is obtained. | |
| 4775 | + * | |
| 4776 | + * If the bpf-prog does not need the IP[46] header, the | |
| 4777 | + * bpf-prog can avoid parsing the IP header by using | |
| 4778 | + * TCP_BPF_SYN. Otherwise, the bpf-prog can get both | |
| 4779 | + * IP[46] and TCP header by using TCP_BPF_SYN_IP. | |
| 4780 | + * | |
| 4781 | + * >0: Total number of bytes copied | |
| 4782 | + * -ENOSPC: Not enough space in optval. Only optlen number of | |
| 4783 | + * bytes is copied. | |
| 4784 | + * -ENOENT: The SYN skb is not available now and the earlier SYN pkt | |
| 4785 | + * is not saved by setsockopt(TCP_SAVE_SYN). | |
| 4786 | + */ | |
| 4787 | + TCP_BPF_SYN = 1005, /* Copy the TCP header */ | |
| 4788 | + TCP_BPF_SYN_IP = 1006, /* Copy the IP[46] and TCP header */ | |
| 4789 | + TCP_BPF_SYN_MAC = 1007, /* Copy the MAC, IP[46], and TCP header */ | |
| 4790 | +}; | |
| 4791 | + | |
| 4792 | +enum { | |
| 4793 | + BPF_LOAD_HDR_OPT_TCP_SYN = (1ULL << 0), | |
| 4794 | +}; | |
| 4795 | + | |
| 4796 | +/* args[0] value during BPF_SOCK_OPS_HDR_OPT_LEN_CB and | |
| 4797 | + * BPF_SOCK_OPS_WRITE_HDR_OPT_CB. | |
| 4798 | + */ | |
| 4799 | +enum { | |
| 4800 | + BPF_WRITE_HDR_TCP_CURRENT_MSS = 1, /* Kernel is finding the | |
| 4801 | + * total option spaces | |
| 4802 | + * required for an established | |
| 4803 | + * sk in order to calculate the | |
| 4804 | + * MSS. No skb is actually | |
| 4805 | + * sent. | |
| 4806 | + */ | |
| 4807 | + BPF_WRITE_HDR_TCP_SYNACK_COOKIE = 2, /* Kernel is in syncookie mode | |
| 4808 | + * when sending a SYN. | |
| 4809 | + */ | |
| 4253 | 4810 | }; |
| 4254 | 4811 | |
| 4255 | 4812 | struct bpf_perf_event_value { |
| @@ -4447,4 +5004,34 @@ struct bpf_sk_lookup { · 191 unmodified lines | ||
| 4447 | 5004 | __u32 local_port; /* Host byte order */ |
| 4448 | 5005 | }; |
| 4449 | 5006 | |
| 5007 | +/* | |
| 5008 | + * struct btf_ptr is used for typed pointer representation; the | |
| 5009 | + * type id is used to render the pointer data as the appropriate type | |
| 5010 | + * via the bpf_snprintf_btf() helper described above. A flags field - | |
| 5011 | + * potentially to specify additional details about the BTF pointer | |
| 5012 | + * (rather than its mode of display) - is included for future use. | |
| 5013 | + * Display flags - BTF_F_* - are passed to bpf_snprintf_btf separately. | |
| 5014 | + */ | |
| 5015 | +struct btf_ptr { | |
| 5016 | + void *ptr; | |
| 5017 | + __u32 type_id; | |
| 5018 | + __u32 flags; /* BTF ptr flags; unused at present. */ | |
| 5019 | +}; | |
| 5020 | + | |
| 5021 | +/* | |
| 5022 | + * Flags to control bpf_snprintf_btf() behaviour. | |
| 5023 | + * - BTF_F_COMPACT: no formatting around type information | |
| 5024 | + * - BTF_F_NONAME: no struct/union member names/types | |
| 5025 | + * - BTF_F_PTR_RAW: show raw (unobfuscated) pointer values; | |
| 5026 | + * equivalent to %px. | |
| 5027 | + * - BTF_F_ZERO: show zero-valued struct/union members; they | |
| 5028 | + * are not displayed by default | |
| 5029 | + */ | |
| 5030 | +enum { | |
| 5031 | + BTF_F_COMPACT = (1ULL << 0), | |
| 5032 | + BTF_F_NONAME = (1ULL << 1), | |
| 5033 | + BTF_F_PTR_RAW = (1ULL << 2), | |
| 5034 | + BTF_F_ZERO = (1ULL << 3), | |
| 5035 | +}; | |
| 5036 | + | |
| 4450 | 5037 | #endif /* _UAPI__LINUX_BPF_H__ */ |