Pull networking updates from Jakub Kicinski: - Add redirect_neigh() BPF packet redirect helper, allowing to limit stack traversal in common container configs and improving TCP back-pressure. Daniel reports ~10Gbps => ~15Gbps single stream TCP performance gain. - Expand netlink policy support and improve policy export to user space. (Ge)netlink core performs request validation according to declared policies. Expand the expressiveness of those policies (min/max length and bitmasks). Allow dumping policies for particular commands. This is used for feature discovery by user space (instead of kernel version parsing or trial and error). - Support IGMPv3/MLDv2 multicast listener discovery protocols in bridge. - Allow more than 255 IPv4 multicast interfaces. - Add support for Type of Service (ToS) reflection in SYN/SYN-ACK packets of TCPv6. - In Multi-patch TCP (MPTCP) support concurrent transmission of data on multiple subflows in a load balancing scenario. Enhance advertising addresses via the RM_ADDR/ADD_ADDR options. - Support SMC-Dv2 version of SMC, which enables multi-subnet deployments. - Allow more calls to same peer in RxRPC. - Support two new Controller Area Network (CAN) protocols - CAN-FD and ISO 15765-2:2016. - Add xfrm/IPsec compat layer, solving the 32bit user space on 64bit kernel problem. - Add TC actions for implementing MPLS L2 VPNs. - Improve nexthop code - e.g. handle various corner cases when nexthop objects are removed from groups better, skip unnecessary notifications and make it easier to offload nexthops into HW by converting to a blocking notifier. - Support adding and consuming TCP header options by BPF programs, opening the doors for easy experimental and deployment-specific TCP option use. - Reorganize TCP congestion control (CC) initialization to simplify life of TCP CC implemented in BPF. - Add support for shipping BPF programs with the kernel and loading them early on boot via the User Mode Driver mechanism, hence reusing all the user space infra we have. - Support sleepable BPF programs, initially targeting LSM and tracing. - Add bpf_d_path() helper for returning full path for given 'struct path'. - Make bpf_tail_call compatible with bpf-to-bpf calls. - Allow BPF programs to call map_update_elem on sockmaps. - Add BPF Type Format (BTF) support for type and enum discovery, as well as support for using BTF within the kernel itself (current use is for pretty printing structures). - Support listing and getting information about bpf_links via the bpf syscall. - Enhance kernel interfaces around NIC firmware update. Allow specifying overwrite mask to control if settings etc. are reset during update; report expected max time operation may take to users; support firmware activation without machine reboot incl. limits of how much impact reset may have (e.g. dropping link or not). - Extend ethtool configuration interface to report IEEE-standard counters, to limit the need for per-vendor logic in user space. - Adopt or extend devlink use for debug, monitoring, fw update in many drivers (dsa loop, ice, ionic, sja1105, qed, mlxsw, mv88e6xxx, dpaa2-eth). - In mlxsw expose critical and emergency SFP module temperature alarms. Refactor port buffer handling to make the defaults more suitable and support setting these values explicitly via the DCBNL interface. - Add XDP support for Intel's igb driver. - Support offloading TC flower classification and filtering rules to mscc_ocelot switches. - Add PTP support for Marvell Octeontx2 and PP2.2 hardware, as well as fixed interval period pulse generator and one-step timestamping in dpaa-eth. - Add support for various auth offloads in WiFi APs, e.g. SAE (WPA3) offload. - Add Lynx PHY/PCS MDIO module, and convert various drivers which have this HW to use it. Convert mvpp2 to split PCS. - Support Marvell Prestera 98DX3255 24-port switch ASICs, as well as 7-port Mediatek MT7531 IP. - Add initial support for QCA6390 and IPQ6018 in ath11k WiFi driver, and wcn3680 support in wcn36xx. - Improve performance for packets which don't require much offloads on recent Mellanox NICs by 20% by making multiple packets share a descriptor entry. - Move chelsio inline crypto drivers (for TLS and IPsec) from the crypto subtree to drivers/net. Move MDIO drivers out of the phy directory. - Clean up a lot of W=1 warnings, reportedly the actively developed subsections of networking drivers should now build W=1 warning free. - Make sure drivers don't use in_interrupt() to dynamically adapt their code. Convert tasklets to use new tasklet_setup API (sadly this conversion is not yet complete). * tag 'net-next-5.10' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (2583 commits) Revert "bpfilter: Fix build error with CONFIG_BPFILTER_UMH" net, sockmap: Don't call bpf_prog_put() on NULL pointer bpf, selftest: Fix flaky tcp_hdr_options test when adding addr to lo bpf, sockmap: Add locking annotations to iterator netfilter: nftables: allow re-computing sctp CRC-32C in 'payload' statements net: fix pos incrementment in ipv6_route_seq_next net/smc: fix invalid return code in smcd_new_buf_create() net/smc: fix valid DMBE buffer sizes net/smc: fix use-after-free of delayed events bpfilter: Fix build error with CONFIG_BPFILTER_UMH cxgb4/ch_ipsec: Replace the module name to ch_ipsec from chcr net: sched: Fix suspicious RCU usage while accessing tcf_tunnel_info bpf: Fix register equivalence tracking. rxrpc: Fix loss of final ack on shutdown rxrpc: Fix bundle counting for exclusive connections netfilter: restore NF_INET_NUMHOOKS ibmveth: Identify ingress large send packets. ibmveth: Switch order of ibmveth_helper calls. cxgb4: handle 4-tuple PEDIT to NAT mode translation selftests: Add VRF route leaking tests ...
Showing the changes to tools/include/uapi (1 of 2301 files). Show all
Unified · Split
| @@ -124,6 +124,7 @@ enum bpf_cmd { · 123 unmodified lines | |||
| 124 | BPF_ENABLE_STATS, | 124 | BPF_ENABLE_STATS, |
| 125 | BPF_ITER_CREATE, | 125 | BPF_ITER_CREATE, |
| 126 | BPF_LINK_DETACH, | 126 | BPF_LINK_DETACH, |
| 127 | BPF_PROG_BIND_MAP, | ||
| 127 | }; | 128 | }; |
| 128 | 129 | ||
| 129 | enum bpf_map_type { | 130 | enum bpf_map_type { |
| @@ -155,6 +156,7 @@ enum bpf_map_type { · 25 unmodified lines | |||
| 155 | BPF_MAP_TYPE_DEVMAP_HASH, | 156 | BPF_MAP_TYPE_DEVMAP_HASH, |
| 156 | BPF_MAP_TYPE_STRUCT_OPS, | 157 | BPF_MAP_TYPE_STRUCT_OPS, |
| 157 | BPF_MAP_TYPE_RINGBUF, | 158 | BPF_MAP_TYPE_RINGBUF, |
| 159 | BPF_MAP_TYPE_INODE_STORAGE, | ||
| 158 | }; | 160 | }; |
| 159 | 161 | ||
| 160 | /* Note that tracing related programs such as | 162 | /* Note that tracing related programs such as |
| @@ -345,19 +347,45 @@ enum bpf_link_type { · 184 unmodified lines | |||
| 345 | /* The verifier internal test flag. Behavior is undefined */ | 347 | /* The verifier internal test flag. Behavior is undefined */ |
| 346 | #define BPF_F_TEST_STATE_FREQ (1U << 3) | 348 | #define BPF_F_TEST_STATE_FREQ (1U << 3) |
| 347 | 349 | ||
| 350 | /* If BPF_F_SLEEPABLE is used in BPF_PROG_LOAD command, the verifier will | ||
| 351 | * restrict map and helper usage for such programs. Sleepable BPF programs can | ||
| 352 | * only be attached to hooks where kernel execution context allows sleeping. | ||
| 353 | * Such programs are allowed to use helpers that may sleep like | ||
| 354 | * bpf_copy_from_user(). | ||
| 355 | */ | ||
| 356 | #define BPF_F_SLEEPABLE (1U << 4) | ||
| 357 | |||
| 348 | /* When BPF ldimm64's insn[0].src_reg != 0 then this can have | 358 | /* When BPF ldimm64's insn[0].src_reg != 0 then this can have |
| 349 | * two extensions: | 359 | * the following extensions: |
| 350 | * | 360 | * |
| 351 | * insn[0].src_reg: BPF_PSEUDO_MAP_FD BPF_PSEUDO_MAP_VALUE | 361 | * insn[0].src_reg: BPF_PSEUDO_MAP_FD |
| 352 | * insn[0].imm: map fd map fd | 362 | * insn[0].imm: map fd |
| 353 | * insn[1].imm: 0 offset into value | 363 | * insn[1].imm: 0 |
| 354 | * insn[0].off: 0 0 | 364 | * insn[0].off: 0 |
| 355 | * insn[1].off: 0 0 | 365 | * insn[1].off: 0 |
| 356 | * ldimm64 rewrite: address of map address of map[0]+offset | 366 | * ldimm64 rewrite: address of map |
| 357 | * verifier type: CONST_PTR_TO_MAP PTR_TO_MAP_VALUE | 367 | * verifier type: CONST_PTR_TO_MAP |
| 358 | */ | 368 | */ |
| 359 | #define BPF_PSEUDO_MAP_FD 1 | 369 | #define BPF_PSEUDO_MAP_FD 1 |
| 370 | /* insn[0].src_reg: BPF_PSEUDO_MAP_VALUE | ||
| 371 | * insn[0].imm: map fd | ||
| 372 | * insn[1].imm: offset into value | ||
| 373 | * insn[0].off: 0 | ||
| 374 | * insn[1].off: 0 | ||
| 375 | * ldimm64 rewrite: address of map[0]+offset | ||
| 376 | * verifier type: PTR_TO_MAP_VALUE | ||
| 377 | */ | ||
| 360 | #define BPF_PSEUDO_MAP_VALUE 2 | 378 | #define BPF_PSEUDO_MAP_VALUE 2 |
| 379 | /* insn[0].src_reg: BPF_PSEUDO_BTF_ID | ||
| 380 | * insn[0].imm: kernel btd id of VAR | ||
| 381 | * insn[1].imm: 0 | ||
| 382 | * insn[0].off: 0 | ||
| 383 | * insn[1].off: 0 | ||
| 384 | * ldimm64 rewrite: address of the kernel variable | ||
| 385 | * verifier type: PTR_TO_BTF_ID or PTR_TO_MEM, depending on whether the var | ||
| 386 | * is struct/union. | ||
| 387 | */ | ||
| 388 | #define BPF_PSEUDO_BTF_ID 3 | ||
| 361 | 389 | ||
| 362 | /* when bpf_call->src_reg == BPF_PSEUDO_CALL, bpf_call->imm == pc-relative | 390 | /* when bpf_call->src_reg == BPF_PSEUDO_CALL, bpf_call->imm == pc-relative |
| 363 | * offset to another bpf function | 391 | * offset to another bpf function |
| @@ -404,6 +432,12 @@ enum { · 40 unmodified lines | |||
| 404 | 432 | ||
| 405 | /* Enable memory-mapping BPF map */ | 433 | /* Enable memory-mapping BPF map */ |
| 406 | BPF_F_MMAPABLE = (1U << 10), | 434 | BPF_F_MMAPABLE = (1U << 10), |
| 435 | |||
| 436 | /* Share perf_event among processes */ | ||
| 437 | BPF_F_PRESERVE_ELEMS = (1U << 11), | ||
| 438 | |||
| 439 | /* Create a map that is suitable to be an inner map with dynamic max entries */ | ||
| 440 | BPF_F_INNER_MAP = (1U << 12), | ||
| 407 | }; | 441 | }; |
| 408 | 442 | ||
| 409 | /* Flags for BPF_PROG_QUERY. */ | 443 | /* Flags for BPF_PROG_QUERY. */ |
| @@ -414,6 +448,11 @@ enum { · 4 unmodified lines | |||
| 414 | */ | 448 | */ |
| 415 | #define BPF_F_QUERY_EFFECTIVE (1U << 0) | 449 | #define BPF_F_QUERY_EFFECTIVE (1U << 0) |
| 416 | 450 | ||
| 451 | /* Flags for BPF_PROG_TEST_RUN */ | ||
| 452 | |||
| 453 | /* If set, run the test on the cpu specified by bpf_attr.test.cpu */ | ||
| 454 | #define BPF_F_TEST_RUN_ON_CPU (1U << 0) | ||
| 455 | |||
| 417 | /* type for BPF_ENABLE_STATS */ | 456 | /* type for BPF_ENABLE_STATS */ |
| 418 | enum bpf_stats_type { | 457 | enum bpf_stats_type { |
| 419 | /* enabled run_time_ns and run_cnt */ | 458 | /* enabled run_time_ns and run_cnt */ |
| @@ -556,6 +595,8 @@ union bpf_attr { · 136 unmodified lines | |||
| 556 | */ | 595 | */ |
| 557 | __aligned_u64 ctx_in; | 596 | __aligned_u64 ctx_in; |
| 558 | __aligned_u64 ctx_out; | 597 | __aligned_u64 ctx_out; |
| 598 | __u32 flags; | ||
| 599 | __u32 cpu; | ||
| 559 | } test; | 600 | } test; |
| 560 | 601 | ||
| 561 | struct { /* anonymous struct used by BPF_*_GET_*_ID */ | 602 | struct { /* anonymous struct used by BPF_*_GET_*_ID */ |
| @@ -622,8 +663,13 @@ union bpf_attr { · 60 unmodified lines | |||
| 622 | }; | 663 | }; |
| 623 | __u32 attach_type; /* attach type */ | 664 | __u32 attach_type; /* attach type */ |
| 624 | __u32 flags; /* extra flags */ | 665 | __u32 flags; /* extra flags */ |
| 625 | __aligned_u64 iter_info; /* extra bpf_iter_link_info */ | 666 | union { |
| 626 | __u32 iter_info_len; /* iter_info length */ | 667 | __u32 target_btf_id; /* btf_id of target to attach to */ |
| 668 | struct { | ||
| 669 | __aligned_u64 iter_info; /* extra bpf_iter_link_info */ | ||
| 670 | __u32 iter_info_len; /* iter_info length */ | ||
| 671 | }; | ||
| 672 | }; | ||
| 627 | } link_create; | 673 | } link_create; |
| 628 | 674 | ||
| 629 | struct { /* struct used by BPF_LINK_UPDATE command */ | 675 | struct { /* struct used by BPF_LINK_UPDATE command */ |
| @@ -649,6 +695,12 @@ union bpf_attr { · 19 unmodified lines | |||
| 649 | __u32 flags; | 695 | __u32 flags; |
| 650 | } iter_create; | 696 | } iter_create; |
| 651 | 697 | ||
| 698 | struct { /* struct used by BPF_PROG_BIND_MAP command */ | ||
| 699 | __u32 prog_fd; | ||
| 700 | __u32 map_fd; | ||
| 701 | __u32 flags; /* extra flags */ | ||
| 702 | } prog_bind_map; | ||
| 703 | |||
| 652 | } __attribute__((aligned(8))); | 704 | } __attribute__((aligned(8))); |
| 653 | 705 | ||
| 654 | /* The description below is an attempt at providing documentation to eBPF | 706 | /* The description below is an attempt at providing documentation to eBPF |
| @@ -1438,8 +1490,8 @@ union bpf_attr { · 783 unmodified lines | |||
| 1438 | * Return | 1490 | * Return |
| 1439 | * The return value depends on the result of the test, and can be: | 1491 | * The return value depends on the result of the test, and can be: |
| 1440 | * | 1492 | * |
| 1441 | * * 0, if the *skb* task belongs to the cgroup2. | 1493 | * * 0, if current task belongs to the cgroup2. |
| 1442 | * * 1, if the *skb* task does not belong to the cgroup2. | 1494 | * * 1, if current task does not belong to the cgroup2. |
| 1443 | * * A negative error code, if an error occurred. | 1495 | * * A negative error code, if an error occurred. |
| 1444 | * | 1496 | * |
| 1445 | * long bpf_skb_change_tail(struct sk_buff *skb, u32 len, u64 flags) | 1497 | * long bpf_skb_change_tail(struct sk_buff *skb, u32 len, u64 flags) |
| @@ -1649,7 +1701,7 @@ union bpf_attr { · 203 unmodified lines | |||
| 1649 | * **TCP_CONGESTION**, **TCP_BPF_IW**, | 1701 | * **TCP_CONGESTION**, **TCP_BPF_IW**, |
| 1650 | * **TCP_BPF_SNDCWND_CLAMP**, **TCP_SAVE_SYN**, | 1702 | * **TCP_BPF_SNDCWND_CLAMP**, **TCP_SAVE_SYN**, |
| 1651 | * **TCP_KEEPIDLE**, **TCP_KEEPINTVL**, **TCP_KEEPCNT**, | 1703 | * **TCP_KEEPIDLE**, **TCP_KEEPINTVL**, **TCP_KEEPCNT**, |
| 1652 | * **TCP_SYNCNT**, **TCP_USER_TIMEOUT**. | 1704 | * **TCP_SYNCNT**, **TCP_USER_TIMEOUT**, **TCP_NOTSENT_LOWAT**. |
| 1653 | * * **IPPROTO_IP**, which supports *optname* **IP_TOS**. | 1705 | * * **IPPROTO_IP**, which supports *optname* **IP_TOS**. |
| 1654 | * * **IPPROTO_IPV6**, which supports *optname* **IPV6_TCLASS**. | 1706 | * * **IPPROTO_IPV6**, which supports *optname* **IPV6_TCLASS**. |
| 1655 | * Return | 1707 | * Return |
| @@ -2204,7 +2256,7 @@ union bpf_attr { · 548 unmodified lines | |||
| 2204 | * Description | 2256 | * Description |
| 2205 | * This helper is used in programs implementing policies at the | 2257 | * This helper is used in programs implementing policies at the |
| 2206 | * skb socket level. If the sk_buff *skb* is allowed to pass (i.e. | 2258 | * skb socket level. If the sk_buff *skb* is allowed to pass (i.e. |
| 2207 | * if the verdeict eBPF program returns **SK_PASS**), redirect it | 2259 | * if the verdict eBPF program returns **SK_PASS**), redirect it |
| 2208 | * to the socket referenced by *map* (of type | 2260 | * to the socket referenced by *map* (of type |
| 2209 | * **BPF_MAP_TYPE_SOCKHASH**) using hash *key*. Both ingress and | 2261 | * **BPF_MAP_TYPE_SOCKHASH**) using hash *key*. Both ingress and |
| 2210 | * egress interfaces can be used for redirection. The | 2262 | * egress interfaces can be used for redirection. The |
| @@ -2496,7 +2548,7 @@ union bpf_attr { · 285 unmodified lines | |||
| 2496 | * result is from *reuse*\ **->socks**\ [] using the hash of the | 2548 | * result is from *reuse*\ **->socks**\ [] using the hash of the |
| 2497 | * tuple. | 2549 | * tuple. |
| 2498 | * | 2550 | * |
| 2499 | * long bpf_sk_release(struct bpf_sock *sock) | 2551 | * long bpf_sk_release(void *sock) |
| 2500 | * Description | 2552 | * Description |
| 2501 | * Release the reference held by *sock*. *sock* must be a | 2553 | * Release the reference held by *sock*. *sock* must be a |
| 2502 | * non-**NULL** pointer that was returned from | 2554 | * non-**NULL** pointer that was returned from |
| @@ -2676,7 +2728,7 @@ union bpf_attr { · 173 unmodified lines | |||
| 2676 | * result is from *reuse*\ **->socks**\ [] using the hash of the | 2728 | * result is from *reuse*\ **->socks**\ [] using the hash of the |
| 2677 | * tuple. | 2729 | * tuple. |
| 2678 | * | 2730 | * |
| 2679 | * long bpf_tcp_check_syncookie(struct bpf_sock *sk, void *iph, u32 iph_len, struct tcphdr *th, u32 th_len) | 2731 | * long bpf_tcp_check_syncookie(void *sk, void *iph, u32 iph_len, struct tcphdr *th, u32 th_len) |
| 2680 | * Description | 2732 | * Description |
| 2681 | * Check whether *iph* and *th* contain a valid SYN cookie ACK for | 2733 | * Check whether *iph* and *th* contain a valid SYN cookie ACK for |
| 2682 | * the listening socket in *sk*. | 2734 | * the listening socket in *sk*. |
| @@ -2807,7 +2859,7 @@ union bpf_attr { · 124 unmodified lines | |||
| 2807 | * | 2859 | * |
| 2808 | * **-ERANGE** if resulting value was out of range. | 2860 | * **-ERANGE** if resulting value was out of range. |
| 2809 | * | 2861 | * |
| 2810 | * void *bpf_sk_storage_get(struct bpf_map *map, struct bpf_sock *sk, void *value, u64 flags) | 2862 | * void *bpf_sk_storage_get(struct bpf_map *map, void *sk, void *value, u64 flags) |
| 2811 | * Description | 2863 | * Description |
| 2812 | * Get a bpf-local-storage from a *sk*. | 2864 | * Get a bpf-local-storage from a *sk*. |
| 2813 | * | 2865 | * |
| @@ -2823,6 +2875,9 @@ union bpf_attr { · 9 unmodified lines | |||
| 2823 | * "type". The bpf-local-storage "type" (i.e. the *map*) is | 2875 | * "type". The bpf-local-storage "type" (i.e. the *map*) is |
| 2824 | * searched against all bpf-local-storages residing at *sk*. | 2876 | * searched against all bpf-local-storages residing at *sk*. |
| 2825 | * | 2877 | * |
| 2878 | * *sk* is a kernel **struct sock** pointer for LSM program. | ||
| 2879 | * *sk* is a **struct bpf_sock** pointer for other program types. | ||
| 2880 | * | ||
| 2826 | * An optional *flags* (**BPF_SK_STORAGE_GET_F_CREATE**) can be | 2881 | * An optional *flags* (**BPF_SK_STORAGE_GET_F_CREATE**) can be |
| 2827 | * used such that a new bpf-local-storage will be | 2882 | * used such that a new bpf-local-storage will be |
| 2828 | * created if one does not exist. *value* can be used | 2883 | * created if one does not exist. *value* can be used |
| @@ -2835,13 +2890,14 @@ union bpf_attr { · 6 unmodified lines | |||
| 2835 | * **NULL** if not found or there was an error in adding | 2890 | * **NULL** if not found or there was an error in adding |
| 2836 | * a new bpf-local-storage. | 2891 | * a new bpf-local-storage. |
| 2837 | * | 2892 | * |
| 2838 | * long bpf_sk_storage_delete(struct bpf_map *map, struct bpf_sock *sk) | 2893 | * long bpf_sk_storage_delete(struct bpf_map *map, void *sk) |
| 2839 | * Description | 2894 | * Description |
| 2840 | * Delete a bpf-local-storage from a *sk*. | 2895 | * Delete a bpf-local-storage from a *sk*. |
| 2841 | * Return | 2896 | * Return |
| 2842 | * 0 on success. | 2897 | * 0 on success. |
| 2843 | * | 2898 | * |
| 2844 | * **-ENOENT** if the bpf-local-storage cannot be found. | 2899 | * **-ENOENT** if the bpf-local-storage cannot be found. |
| 2900 | * **-EINVAL** if sk is not a fullsock (e.g. a request_sock). | ||
| 2845 | * | 2901 | * |
| 2846 | * long bpf_send_signal(u32 sig) | 2902 | * long bpf_send_signal(u32 sig) |
| 2847 | * Description | 2903 | * Description |
| @@ -2858,7 +2914,7 @@ union bpf_attr { · 10 unmodified lines | |||
| 2858 | * | 2914 | * |
| 2859 | * **-EAGAIN** if bpf program can try again. | 2915 | * **-EAGAIN** if bpf program can try again. |
| 2860 | * | 2916 | * |
| 2861 | * s64 bpf_tcp_gen_syncookie(struct bpf_sock *sk, void *iph, u32 iph_len, struct tcphdr *th, u32 th_len) | 2917 | * s64 bpf_tcp_gen_syncookie(void *sk, void *iph, u32 iph_len, struct tcphdr *th, u32 th_len) |
| 2862 | * Description | 2918 | * Description |
| 2863 | * Try to issue a SYN cookie for the packet with corresponding | 2919 | * Try to issue a SYN cookie for the packet with corresponding |
| 2864 | * IP/TCP headers, *iph* and *th*, on the listening socket in *sk*. | 2920 | * IP/TCP headers, *iph* and *th*, on the listening socket in *sk*. |
| @@ -3087,7 +3143,7 @@ union bpf_attr { · 222 unmodified lines | |||
| 3087 | * Return | 3143 | * Return |
| 3088 | * The id is returned or 0 in case the id could not be retrieved. | 3144 | * The id is returned or 0 in case the id could not be retrieved. |
| 3089 | * | 3145 | * |
| 3090 | * long bpf_sk_assign(struct sk_buff *skb, struct bpf_sock *sk, u64 flags) | 3146 | * long bpf_sk_assign(struct sk_buff *skb, void *sk, u64 flags) |
| 3091 | * Description | 3147 | * Description |
| 3092 | * Helper is overloaded depending on BPF program type. This | 3148 | * Helper is overloaded depending on BPF program type. This |
| 3093 | * description applies to **BPF_PROG_TYPE_SCHED_CLS** and | 3149 | * description applies to **BPF_PROG_TYPE_SCHED_CLS** and |
| @@ -3215,11 +3271,11 @@ union bpf_attr { · 121 unmodified lines | |||
| 3215 | * | 3271 | * |
| 3216 | * **-EOVERFLOW** if an overflow happened: The same object will be tried again. | 3272 | * **-EOVERFLOW** if an overflow happened: The same object will be tried again. |
| 3217 | * | 3273 | * |
| 3218 | * u64 bpf_sk_cgroup_id(struct bpf_sock *sk) | 3274 | * u64 bpf_sk_cgroup_id(void *sk) |
| 3219 | * Description | 3275 | * Description |
| 3220 | * Return the cgroup v2 id of the socket *sk*. | 3276 | * Return the cgroup v2 id of the socket *sk*. |
| 3221 | * | 3277 | * |
| 3222 | * *sk* must be a non-**NULL** pointer to a full socket, e.g. one | 3278 | * *sk* must be a non-**NULL** pointer to a socket, e.g. one |
| 3223 | * returned from **bpf_sk_lookup_xxx**\ (), | 3279 | * returned from **bpf_sk_lookup_xxx**\ (), |
| 3224 | * **bpf_sk_fullsock**\ (), etc. The format of returned id is | 3280 | * **bpf_sk_fullsock**\ (), etc. The format of returned id is |
| 3225 | * same as in **bpf_skb_cgroup_id**\ (). | 3281 | * same as in **bpf_skb_cgroup_id**\ (). |
| @@ -3229,7 +3285,7 @@ union bpf_attr { · 3 unmodified lines | |||
| 3229 | * Return | 3285 | * Return |
| 3230 | * The id is returned or 0 in case the id could not be retrieved. | 3286 | * The id is returned or 0 in case the id could not be retrieved. |
| 3231 | * | 3287 | * |
| 3232 | * u64 bpf_sk_ancestor_cgroup_id(struct bpf_sock *sk, int ancestor_level) | 3288 | * u64 bpf_sk_ancestor_cgroup_id(void *sk, int ancestor_level) |
| 3233 | * Description | 3289 | * Description |
| 3234 | * Return id of cgroup v2 that is ancestor of cgroup associated | 3290 | * Return id of cgroup v2 that is ancestor of cgroup associated |
| 3235 | * with the *sk* at the *ancestor_level*. The root cgroup is at | 3291 | * with the *sk* at the *ancestor_level*. The root cgroup is at |
| @@ -3337,38 +3393,38 @@ union bpf_attr { · 101 unmodified lines | |||
| 3337 | * Description | 3393 | * Description |
| 3338 | * Dynamically cast a *sk* pointer to a *tcp6_sock* pointer. | 3394 | * Dynamically cast a *sk* pointer to a *tcp6_sock* pointer. |
| 3339 | * Return | 3395 | * Return |
| 3340 | * *sk* if casting is valid, or NULL otherwise. | 3396 | * *sk* if casting is valid, or **NULL** otherwise. |
| 3341 | * | 3397 | * |
| 3342 | * struct tcp_sock *bpf_skc_to_tcp_sock(void *sk) | 3398 | * struct tcp_sock *bpf_skc_to_tcp_sock(void *sk) |
| 3343 | * Description | 3399 | * Description |
| 3344 | * Dynamically cast a *sk* pointer to a *tcp_sock* pointer. | 3400 | * Dynamically cast a *sk* pointer to a *tcp_sock* pointer. |
| 3345 | * Return | 3401 | * Return |
| 3346 | * *sk* if casting is valid, or NULL otherwise. | 3402 | * *sk* if casting is valid, or **NULL** otherwise. |
| 3347 | * | 3403 | * |
| 3348 | * struct tcp_timewait_sock *bpf_skc_to_tcp_timewait_sock(void *sk) | 3404 | * struct tcp_timewait_sock *bpf_skc_to_tcp_timewait_sock(void *sk) |
| 3349 | * Description | 3405 | * Description |
| 3350 | * Dynamically cast a *sk* pointer to a *tcp_timewait_sock* pointer. | 3406 | * Dynamically cast a *sk* pointer to a *tcp_timewait_sock* pointer. |
| 3351 | * Return | 3407 | * Return |
| 3352 | * *sk* if casting is valid, or NULL otherwise. | 3408 | * *sk* if casting is valid, or **NULL** otherwise. |
| 3353 | * | 3409 | * |
| 3354 | * struct tcp_request_sock *bpf_skc_to_tcp_request_sock(void *sk) | 3410 | * struct tcp_request_sock *bpf_skc_to_tcp_request_sock(void *sk) |
| 3355 | * Description | 3411 | * Description |
| 3356 | * Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. | 3412 | * Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. |
| 3357 | * Return | 3413 | * Return |
| 3358 | * *sk* if casting is valid, or NULL otherwise. | 3414 | * *sk* if casting is valid, or **NULL** otherwise. |
| 3359 | * | 3415 | * |
| 3360 | * struct udp6_sock *bpf_skc_to_udp6_sock(void *sk) | 3416 | * struct udp6_sock *bpf_skc_to_udp6_sock(void *sk) |
| 3361 | * Description | 3417 | * Description |
| 3362 | * Dynamically cast a *sk* pointer to a *udp6_sock* pointer. | 3418 | * Dynamically cast a *sk* pointer to a *udp6_sock* pointer. |
| 3363 | * Return | 3419 | * Return |
| 3364 | * *sk* if casting is valid, or NULL otherwise. | 3420 | * *sk* if casting is valid, or **NULL** otherwise. |
| 3365 | * | 3421 | * |
| 3366 | * long bpf_get_task_stack(struct task_struct *task, void *buf, u32 size, u64 flags) | 3422 | * long bpf_get_task_stack(struct task_struct *task, void *buf, u32 size, u64 flags) |
| 3367 | * Description | 3423 | * Description |
| 3368 | * Return a user or a kernel stack in bpf program provided buffer. | 3424 | * Return a user or a kernel stack in bpf program provided buffer. |
| 3369 | * To achieve this, the helper needs *task*, which is a valid | 3425 | * To achieve this, the helper needs *task*, which is a valid |
| 3370 | * pointer to struct task_struct. To store the stacktrace, the | 3426 | * pointer to **struct task_struct**. To store the stacktrace, the |
| 3371 | * bpf program provides *buf* with a nonnegative *size*. | 3427 | * bpf program provides *buf* with a nonnegative *size*. |
| 3372 | * | 3428 | * |
| 3373 | * The last argument, *flags*, holds the number of stack frames to | 3429 | * The last argument, *flags*, holds the number of stack frames to |
| 3374 | * skip (from 0 to 255), masked with | 3430 | * skip (from 0 to 255), masked with |
| @@ -3395,6 +3451,293 @@ union bpf_attr { · 20 unmodified lines | |||
| 3395 | * A non-negative value equal to or less than *size* on success, | 3451 | * A non-negative value equal to or less than *size* on success, |
| 3396 | * or a negative error in case of failure. | 3452 | * or a negative error in case of failure. |
| 3397 | * | 3453 | * |
| 3454 | * long bpf_load_hdr_opt(struct bpf_sock_ops *skops, void *searchby_res, u32 len, u64 flags) | ||
| 3455 | * Description | ||
| 3456 | * Load header option. Support reading a particular TCP header | ||
| 3457 | * option for bpf program (**BPF_PROG_TYPE_SOCK_OPS**). | ||
| 3458 | * | ||
| 3459 | * If *flags* is 0, it will search the option from the | ||
| 3460 | * *skops*\ **->skb_data**. The comment in **struct bpf_sock_ops** | ||
| 3461 | * has details on what skb_data contains under different | ||
| 3462 | * *skops*\ **->op**. | ||
| 3463 | * | ||
| 3464 | * The first byte of the *searchby_res* specifies the | ||
| 3465 | * kind that it wants to search. | ||
| 3466 | * | ||
| 3467 | * If the searching kind is an experimental kind | ||
| 3468 | * (i.e. 253 or 254 according to RFC6994). It also | ||
| 3469 | * needs to specify the "magic" which is either | ||
| 3470 | * 2 bytes or 4 bytes. It then also needs to | ||
| 3471 | * specify the size of the magic by using | ||
| 3472 | * the 2nd byte which is "kind-length" of a TCP | ||
| 3473 | * header option and the "kind-length" also | ||
| 3474 | * includes the first 2 bytes "kind" and "kind-length" | ||
| 3475 | * itself as a normal TCP header option also does. | ||
| 3476 | * | ||
| 3477 | * For example, to search experimental kind 254 with | ||
| 3478 | * 2 byte magic 0xeB9F, the searchby_res should be | ||
| 3479 | * [ 254, 4, 0xeB, 0x9F, 0, 0, .... 0 ]. | ||
| 3480 | * | ||
| 3481 | * To search for the standard window scale option (3), | ||
| 3482 | * the *searchby_res* should be [ 3, 0, 0, .... 0 ]. | ||
| 3483 | * Note, kind-length must be 0 for regular option. | ||
| 3484 | * | ||
| 3485 | * Searching for No-Op (0) and End-of-Option-List (1) are | ||
| 3486 | * not supported. | ||
| 3487 | * | ||
| 3488 | * *len* must be at least 2 bytes which is the minimal size | ||
| 3489 | * of a header option. | ||
| 3490 | * | ||
| 3491 | * Supported flags: | ||
| 3492 | * | ||
| 3493 | * * **BPF_LOAD_HDR_OPT_TCP_SYN** to search from the | ||
| 3494 | * saved_syn packet or the just-received syn packet. | ||
| 3495 | * | ||
| 3496 | * Return | ||
| 3497 | * > 0 when found, the header option is copied to *searchby_res*. | ||
| 3498 | * The return value is the total length copied. On failure, a | ||
| 3499 | * negative error code is returned: | ||
| 3500 | * | ||
| 3501 | * **-EINVAL** if a parameter is invalid. | ||
| 3502 | * | ||
| 3503 | * **-ENOMSG** if the option is not found. | ||
| 3504 | * | ||
| 3505 | * **-ENOENT** if no syn packet is available when | ||
| 3506 | * **BPF_LOAD_HDR_OPT_TCP_SYN** is used. | ||
| 3507 | * | ||
| 3508 | * **-ENOSPC** if there is not enough space. Only *len* number of | ||
| 3509 | * bytes are copied. | ||
| 3510 | * | ||
| 3511 | * **-EFAULT** on failure to parse the header options in the | ||
| 3512 | * packet. | ||
| 3513 | * | ||
| 3514 | * **-EPERM** if the helper cannot be used under the current | ||
| 3515 | * *skops*\ **->op**. | ||
| 3516 | * | ||
| 3517 | * long bpf_store_hdr_opt(struct bpf_sock_ops *skops, const void *from, u32 len, u64 flags) | ||
| 3518 | * Description | ||
| 3519 | * Store header option. The data will be copied | ||
| 3520 | * from buffer *from* with length *len* to the TCP header. | ||
| 3521 | * | ||
| 3522 | * The buffer *from* should have the whole option that | ||
| 3523 | * includes the kind, kind-length, and the actual | ||
| 3524 | * option data. The *len* must be at least kind-length | ||
| 3525 | * long. The kind-length does not have to be 4 byte | ||
| 3526 | * aligned. The kernel will take care of the padding | ||
| 3527 | * and setting the 4 bytes aligned value to th->doff. | ||
| 3528 | * | ||
| 3529 | * This helper will check for duplicated option | ||
| 3530 | * by searching the same option in the outgoing skb. | ||
| 3531 | * | ||
| 3532 | * This helper can only be called during | ||
| 3533 | * **BPF_SOCK_OPS_WRITE_HDR_OPT_CB**. | ||
| 3534 | * | ||
| 3535 | * Return | ||
| 3536 | * 0 on success, or negative error in case of failure: | ||
| 3537 | * | ||
| 3538 | * **-EINVAL** If param is invalid. | ||
| 3539 | * | ||
| 3540 | * **-ENOSPC** if there is not enough space in the header. | ||
| 3541 | * Nothing has been written | ||
| 3542 | * | ||
| 3543 | * **-EEXIST** if the option already exists. | ||
| 3544 | * | ||
| 3545 | * **-EFAULT** on failrue to parse the existing header options. | ||
| 3546 | * | ||
| 3547 | * **-EPERM** if the helper cannot be used under the current | ||
| 3548 | * *skops*\ **->op**. | ||
| 3549 | * | ||
| 3550 | * long bpf_reserve_hdr_opt(struct bpf_sock_ops *skops, u32 len, u64 flags) | ||
| 3551 | * Description | ||
| 3552 | * Reserve *len* bytes for the bpf header option. The | ||
| 3553 | * space will be used by **bpf_store_hdr_opt**\ () later in | ||
| 3554 | * **BPF_SOCK_OPS_WRITE_HDR_OPT_CB**. | ||
| 3555 | * | ||
| 3556 | * If **bpf_reserve_hdr_opt**\ () is called multiple times, | ||
| 3557 | * the total number of bytes will be reserved. | ||
| 3558 | * | ||
| 3559 | * This helper can only be called during | ||
| 3560 | * **BPF_SOCK_OPS_HDR_OPT_LEN_CB**. | ||
| 3561 | * | ||
| 3562 | * Return | ||
| 3563 | * 0 on success, or negative error in case of failure: | ||
| 3564 | * | ||
| 3565 | * **-EINVAL** if a parameter is invalid. | ||
| 3566 | * | ||
| 3567 | * **-ENOSPC** if there is not enough space in the header. | ||
| 3568 | * | ||
| 3569 | * **-EPERM** if the helper cannot be used under the current | ||
| 3570 | * *skops*\ **->op**. | ||
| 3571 | * | ||
| 3572 | * void *bpf_inode_storage_get(struct bpf_map *map, void *inode, void *value, u64 flags) | ||
| 3573 | * Description | ||
| 3574 | * Get a bpf_local_storage from an *inode*. | ||
| 3575 | * | ||
| 3576 | * Logically, it could be thought of as getting the value from | ||
| 3577 | * a *map* with *inode* as the **key**. From this | ||
| 3578 | * perspective, the usage is not much different from | ||
| 3579 | * **bpf_map_lookup_elem**\ (*map*, **&**\ *inode*) except this | ||
| 3580 | * helper enforces the key must be an inode and the map must also | ||
| 3581 | * be a **BPF_MAP_TYPE_INODE_STORAGE**. | ||
| 3582 | * | ||
| 3583 | * Underneath, the value is stored locally at *inode* instead of | ||
| 3584 | * the *map*. The *map* is used as the bpf-local-storage | ||
| 3585 | * "type". The bpf-local-storage "type" (i.e. the *map*) is | ||
| 3586 | * searched against all bpf_local_storage residing at *inode*. | ||
| 3587 | * | ||
| 3588 | * An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be | ||
| 3589 | * used such that a new bpf_local_storage will be | ||
| 3590 | * created if one does not exist. *value* can be used | ||
| 3591 | * together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify | ||
| 3592 | * the initial value of a bpf_local_storage. If *value* is | ||
| 3593 | * **NULL**, the new bpf_local_storage will be zero initialized. | ||
| 3594 | * Return | ||
| 3595 | * A bpf_local_storage pointer is returned on success. | ||
| 3596 | * | ||
| 3597 | * **NULL** if not found or there was an error in adding | ||
| 3598 | * a new bpf_local_storage. | ||
| 3599 | * | ||
| 3600 | * int bpf_inode_storage_delete(struct bpf_map *map, void *inode) | ||
| 3601 | * Description | ||
| 3602 | * Delete a bpf_local_storage from an *inode*. | ||
| 3603 | * Return | ||
| 3604 | * 0 on success. | ||
| 3605 | * | ||
| 3606 | * **-ENOENT** if the bpf_local_storage cannot be found. | ||
| 3607 | * | ||
| 3608 | * long bpf_d_path(struct path *path, char *buf, u32 sz) | ||
| 3609 | * Description | ||
| 3610 | * Return full path for given **struct path** object, which | ||
| 3611 | * needs to be the kernel BTF *path* object. The path is | ||
| 3612 | * returned in the provided buffer *buf* of size *sz* and | ||
| 3613 | * is zero terminated. | ||
| 3614 | * | ||
| 3615 | * Return | ||
| 3616 | * On success, the strictly positive length of the string, | ||
| 3617 | * including the trailing NUL character. On error, a negative | ||
| 3618 | * value. | ||
| 3619 | * | ||
| 3620 | * long bpf_copy_from_user(void *dst, u32 size, const void *user_ptr) | ||
| 3621 | * Description | ||
| 3622 | * Read *size* bytes from user space address *user_ptr* and store | ||
| 3623 | * the data in *dst*. This is a wrapper of **copy_from_user**\ (). | ||
| 3624 | * Return | ||
| 3625 | * 0 on success, or a negative error in case of failure. | ||
| 3626 | * | ||
| 3627 | * long bpf_snprintf_btf(char *str, u32 str_size, struct btf_ptr *ptr, u32 btf_ptr_size, u64 flags) | ||
| 3628 | * Description | ||
| 3629 | * Use BTF to store a string representation of *ptr*->ptr in *str*, | ||
| 3630 | * using *ptr*->type_id. This value should specify the type | ||
| 3631 | * that *ptr*->ptr points to. LLVM __builtin_btf_type_id(type, 1) | ||
| 3632 | * can be used to look up vmlinux BTF type ids. Traversing the | ||
| 3633 | * data structure using BTF, the type information and values are | ||
| 3634 | * stored in the first *str_size* - 1 bytes of *str*. Safe copy of | ||
| 3635 | * the pointer data is carried out to avoid kernel crashes during | ||
| 3636 | * operation. Smaller types can use string space on the stack; | ||
| 3637 | * larger programs can use map data to store the string | ||
| 3638 | * representation. | ||
| 3639 | * | ||
| 3640 | * The string can be subsequently shared with userspace via | ||
| 3641 | * bpf_perf_event_output() or ring buffer interfaces. | ||
| 3642 | * bpf_trace_printk() is to be avoided as it places too small | ||
| 3643 | * a limit on string size to be useful. | ||
| 3644 | * | ||
| 3645 | * *flags* is a combination of | ||
| 3646 | * | ||
| 3647 | * **BTF_F_COMPACT** | ||
| 3648 | * no formatting around type information | ||
| 3649 | * **BTF_F_NONAME** | ||
| 3650 | * no struct/union member names/types | ||
| 3651 | * **BTF_F_PTR_RAW** | ||
| 3652 | * show raw (unobfuscated) pointer values; | ||
| 3653 | * equivalent to printk specifier %px. | ||
| 3654 | * **BTF_F_ZERO** | ||
| 3655 | * show zero-valued struct/union members; they | ||
| 3656 | * are not displayed by default | ||
| 3657 | * | ||
| 3658 | * Return | ||
| 3659 | * The number of bytes that were written (or would have been | ||
| 3660 | * written if output had to be truncated due to string size), | ||
| 3661 | * or a negative error in cases of failure. | ||
| 3662 | * | ||
| 3663 | * long bpf_seq_printf_btf(struct seq_file *m, struct btf_ptr *ptr, u32 ptr_size, u64 flags) | ||
| 3664 | * Description | ||
| 3665 | * Use BTF to write to seq_write a string representation of | ||
| 3666 | * *ptr*->ptr, using *ptr*->type_id as per bpf_snprintf_btf(). | ||
| 3667 | * *flags* are identical to those used for bpf_snprintf_btf. | ||
| 3668 | * Return | ||
| 3669 | * 0 on success or a negative error in case of failure. | ||
| 3670 | * | ||
| 3671 | * u64 bpf_skb_cgroup_classid(struct sk_buff *skb) | ||
| 3672 | * Description | ||
| 3673 | * See **bpf_get_cgroup_classid**\ () for the main description. | ||
| 3674 | * This helper differs from **bpf_get_cgroup_classid**\ () in that | ||
| 3675 | * the cgroup v1 net_cls class is retrieved only from the *skb*'s | ||
| 3676 | * associated socket instead of the current process. | ||
| 3677 | * Return | ||
| 3678 | * The id is returned or 0 in case the id could not be retrieved. | ||
| 3679 | * | ||
| 3680 | * long bpf_redirect_neigh(u32 ifindex, u64 flags) | ||
| 3681 | * Description | ||
| 3682 | * Redirect the packet to another net device of index *ifindex* | ||
| 3683 | * and fill in L2 addresses from neighboring subsystem. This helper | ||
| 3684 | * is somewhat similar to **bpf_redirect**\ (), except that it | ||
| 3685 | * populates L2 addresses as well, meaning, internally, the helper | ||
| 3686 | * performs a FIB lookup based on the skb's networking header to | ||
| 3687 | * get the address of the next hop and then relies on the neighbor | ||
| 3688 | * lookup for the L2 address of the nexthop. | ||
| 3689 | * | ||
| 3690 | * The *flags* argument is reserved and must be 0. The helper is | ||
| 3691 | * currently only supported for tc BPF program types, and enabled | ||
| 3692 | * for IPv4 and IPv6 protocols. | ||
| 3693 | * Return | ||
| 3694 | * The helper returns **TC_ACT_REDIRECT** on success or | ||
| 3695 | * **TC_ACT_SHOT** on error. | ||
| 3696 | * | ||
| 3697 | * void *bpf_per_cpu_ptr(const void *percpu_ptr, u32 cpu) | ||
| 3698 | * Description | ||
| 3699 | * Take a pointer to a percpu ksym, *percpu_ptr*, and return a | ||
| 3700 | * pointer to the percpu kernel variable on *cpu*. A ksym is an | ||
| 3701 | * extern variable decorated with '__ksym'. For ksym, there is a | ||
| 3702 | * global var (either static or global) defined of the same name | ||
| 3703 | * in the kernel. The ksym is percpu if the global var is percpu. | ||
| 3704 | * The returned pointer points to the global percpu var on *cpu*. | ||
| 3705 | * | ||
| 3706 | * bpf_per_cpu_ptr() has the same semantic as per_cpu_ptr() in the | ||
| 3707 | * kernel, except that bpf_per_cpu_ptr() may return NULL. This | ||
| 3708 | * happens if *cpu* is larger than nr_cpu_ids. The caller of | ||
| 3709 | * bpf_per_cpu_ptr() must check the returned value. | ||
| 3710 | * Return | ||
| 3711 | * A pointer pointing to the kernel percpu variable on *cpu*, or | ||
| 3712 | * NULL, if *cpu* is invalid. | ||
| 3713 | * | ||
| 3714 | * void *bpf_this_cpu_ptr(const void *percpu_ptr) | ||
| 3715 | * Description | ||
| 3716 | * Take a pointer to a percpu ksym, *percpu_ptr*, and return a | ||
| 3717 | * pointer to the percpu kernel variable on this cpu. See the | ||
| 3718 | * description of 'ksym' in **bpf_per_cpu_ptr**\ (). | ||
| 3719 | * | ||
| 3720 | * bpf_this_cpu_ptr() has the same semantic as this_cpu_ptr() in | ||
| 3721 | * the kernel. Different from **bpf_per_cpu_ptr**\ (), it would | ||
| 3722 | * never return NULL. | ||
| 3723 | * Return | ||
| 3724 | * A pointer pointing to the kernel percpu variable on this cpu. | ||
| 3725 | * | ||
| 3726 | * long bpf_redirect_peer(u32 ifindex, u64 flags) | ||
| 3727 | * Description | ||
| 3728 | * Redirect the packet to another net device of index *ifindex*. | ||
| 3729 | * This helper is somewhat similar to **bpf_redirect**\ (), except | ||
| 3730 | * that the redirection happens to the *ifindex*' peer device and | ||
| 3731 | * the netns switch takes place from ingress to ingress without | ||
| 3732 | * going through the CPU's backlog queue. | ||
| 3733 | * | ||
| 3734 | * The *flags* argument is reserved and must be 0. The helper is | ||
| 3735 | * currently only supported for tc BPF program types at the ingress | ||
| 3736 | * hook and for veth device types. The peer device must reside in a | ||
| 3737 | * different network namespace. | ||
| 3738 | * Return | ||
| 3739 | * The helper returns **TC_ACT_REDIRECT** on success or | ||
| 3740 | * **TC_ACT_SHOT** on error. | ||
| 3398 | */ | 3741 | */ |
| 3399 | #define __BPF_FUNC_MAPPER(FN) \ | 3742 | #define __BPF_FUNC_MAPPER(FN) \ |
| 3400 | FN(unspec), \ | 3743 | FN(unspec), \ |
| @@ -3539,6 +3882,20 @@ union bpf_attr { · 138 unmodified lines | |||
| 3539 | FN(skc_to_tcp_request_sock), \ | 3882 | FN(skc_to_tcp_request_sock), \ |
| 3540 | FN(skc_to_udp6_sock), \ | 3883 | FN(skc_to_udp6_sock), \ |
| 3541 | FN(get_task_stack), \ | 3884 | FN(get_task_stack), \ |
| 3885 | FN(load_hdr_opt), \ | ||
| 3886 | FN(store_hdr_opt), \ | ||
| 3887 | FN(reserve_hdr_opt), \ | ||
| 3888 | FN(inode_storage_get), \ | ||
| 3889 | FN(inode_storage_delete), \ | ||
| 3890 | FN(d_path), \ | ||
| 3891 | FN(copy_from_user), \ | ||
| 3892 | FN(snprintf_btf), \ | ||
| 3893 | FN(seq_printf_btf), \ | ||
| 3894 | FN(skb_cgroup_classid), \ | ||
| 3895 | FN(redirect_neigh), \ | ||
| 3896 | FN(bpf_per_cpu_ptr), \ | ||
| 3897 | FN(bpf_this_cpu_ptr), \ | ||
| 3898 | FN(redirect_peer), \ | ||
| 3542 | /* */ | 3899 | /* */ |
| 3543 | 3900 | ||
| 3544 | /* integer value in 'imm' field of BPF_CALL instruction selects which helper | 3901 | /* integer value in 'imm' field of BPF_CALL instruction selects which helper |
| @@ -3648,9 +4005,13 @@ enum { · 103 unmodified lines | |||
| 3648 | BPF_F_SYSCTL_BASE_NAME = (1ULL << 0), | 4005 | BPF_F_SYSCTL_BASE_NAME = (1ULL << 0), |
| 3649 | }; | 4006 | }; |
| 3650 | 4007 | ||
| 3651 | /* BPF_FUNC_sk_storage_get flags */ | 4008 | /* BPF_FUNC_<kernel_obj>_storage_get flags */ |
| 3652 | enum { | 4009 | enum { |
| 3653 | BPF_SK_STORAGE_GET_F_CREATE = (1ULL << 0), | 4010 | BPF_LOCAL_STORAGE_GET_F_CREATE = (1ULL << 0), |
| 4011 | /* BPF_SK_STORAGE_GET_F_CREATE is only kept for backward compatibility | ||
| 4012 | * and BPF_LOCAL_STORAGE_GET_F_CREATE must be used instead. | ||
| 4013 | */ | ||
| 4014 | BPF_SK_STORAGE_GET_F_CREATE = BPF_LOCAL_STORAGE_GET_F_CREATE, | ||
| 3654 | }; | 4015 | }; |
| 3655 | 4016 | ||
| 3656 | /* BPF_FUNC_read_branch_records flags. */ | 4017 | /* BPF_FUNC_read_branch_records flags. */ |
| @@ -4071,6 +4432,15 @@ struct bpf_link_info { · 414 unmodified lines | |||
| 4071 | __u64 cgroup_id; | 4432 | __u64 cgroup_id; |
| 4072 | __u32 attach_type; | 4433 | __u32 attach_type; |
| 4073 | } cgroup; | 4434 | } cgroup; |
| 4435 | struct { | ||
| 4436 | __aligned_u64 target_name; /* in/out: target_name buffer ptr */ | ||
| 4437 | __u32 target_name_len; /* in/out: target_name buffer len */ | ||
| 4438 | union { | ||
| 4439 | struct { | ||
| 4440 | __u32 map_id; | ||
| 4441 | } map; | ||
| 4442 | }; | ||
| 4443 | } iter; | ||
| 4074 | struct { | 4444 | struct { |
| 4075 | __u32 netns_ino; | 4445 | __u32 netns_ino; |
| 4076 | __u32 attach_type; | 4446 | __u32 attach_type; |
| @@ -4158,6 +4528,36 @@ struct bpf_sock_ops { · 81 unmodified lines | |||
| 4158 | __u64 bytes_received; | 4528 | __u64 bytes_received; |
| 4159 | __u64 bytes_acked; | 4529 | __u64 bytes_acked; |
| 4160 | __bpf_md_ptr(struct bpf_sock *, sk); | 4530 | __bpf_md_ptr(struct bpf_sock *, sk); |
| 4531 | /* [skb_data, skb_data_end) covers the whole TCP header. | ||
| 4532 | * | ||
| 4533 | * BPF_SOCK_OPS_PARSE_HDR_OPT_CB: The packet received | ||
| 4534 | * BPF_SOCK_OPS_HDR_OPT_LEN_CB: Not useful because the | ||
| 4535 | * header has not been written. | ||
| 4536 | * BPF_SOCK_OPS_WRITE_HDR_OPT_CB: The header and options have | ||
| 4537 | * been written so far. | ||
| 4538 | * BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB: The SYNACK that concludes | ||
| 4539 | * the 3WHS. | ||
| 4540 | * BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB: The ACK that concludes | ||
| 4541 | * the 3WHS. | ||
| 4542 | * | ||
| 4543 | * bpf_load_hdr_opt() can also be used to read a particular option. | ||
| 4544 | */ | ||
| 4545 | __bpf_md_ptr(void *, skb_data); | ||
| 4546 | __bpf_md_ptr(void *, skb_data_end); | ||
| 4547 | __u32 skb_len; /* The total length of a packet. | ||
| 4548 | * It includes the header, options, | ||
| 4549 | * and payload. | ||
| 4550 | */ | ||
| 4551 | __u32 skb_tcp_flags; /* tcp_flags of the header. It provides | ||
| 4552 | * an easy way to check for tcp_flags | ||
| 4553 | * without parsing skb_data. | ||
| 4554 | * | ||
| 4555 | * In particular, the skb_tcp_flags | ||
| 4556 | * will still be available in | ||
| 4557 | * BPF_SOCK_OPS_HDR_OPT_LEN even though | ||
| 4558 | * the outgoing header has not | ||
| 4559 | * been written yet. | ||
| 4560 | */ | ||
| 4161 | }; | 4561 | }; |
| 4162 | 4562 | ||
| 4163 | /* Definitions for bpf_sock_ops_cb_flags */ | 4563 | /* Definitions for bpf_sock_ops_cb_flags */ |
| @@ -4166,8 +4566,51 @@ enum { · 2 unmodified lines | |||
| 4166 | BPF_SOCK_OPS_RETRANS_CB_FLAG = (1<<1), | 4566 | BPF_SOCK_OPS_RETRANS_CB_FLAG = (1<<1), |
| 4167 | BPF_SOCK_OPS_STATE_CB_FLAG = (1<<2), | 4567 | BPF_SOCK_OPS_STATE_CB_FLAG = (1<<2), |
| 4168 | BPF_SOCK_OPS_RTT_CB_FLAG = (1<<3), | 4568 | BPF_SOCK_OPS_RTT_CB_FLAG = (1<<3), |
| 4569 | /* Call bpf for all received TCP headers. The bpf prog will be | ||
| 4570 | * called under sock_ops->op == BPF_SOCK_OPS_PARSE_HDR_OPT_CB | ||
| 4571 | * | ||
| 4572 | * Please refer to the comment in BPF_SOCK_OPS_PARSE_HDR_OPT_CB | ||
| 4573 | * for the header option related helpers that will be useful | ||
| 4574 | * to the bpf programs. | ||
| 4575 | * | ||
| 4576 | * It could be used at the client/active side (i.e. connect() side) | ||
| 4577 | * when the server told it that the server was in syncookie | ||
| 4578 | * mode and required the active side to resend the bpf-written | ||
| 4579 | * options. The active side can keep writing the bpf-options until | ||
| 4580 | * it received a valid packet from the server side to confirm | ||
| 4581 | * the earlier packet (and options) has been received. The later | ||
| 4582 | * example patch is using it like this at the active side when the | ||
| 4583 | * server is in syncookie mode. | ||
| 4584 | * | ||
| 4585 | * The bpf prog will usually turn this off in the common cases. | ||
| 4586 | */ | ||
| 4587 | BPF_SOCK_OPS_PARSE_ALL_HDR_OPT_CB_FLAG = (1<<4), | ||
| 4588 | /* Call bpf when kernel has received a header option that | ||
| 4589 | * the kernel cannot handle. The bpf prog will be called under | ||
| 4590 | * sock_ops->op == BPF_SOCK_OPS_PARSE_HDR_OPT_CB. | ||
| 4591 | * | ||
| 4592 | * Please refer to the comment in BPF_SOCK_OPS_PARSE_HDR_OPT_CB | ||
| 4593 | * for the header option related helpers that will be useful | ||
| 4594 | * to the bpf programs. | ||
| 4595 | */ | ||
| 4596 | BPF_SOCK_OPS_PARSE_UNKNOWN_HDR_OPT_CB_FLAG = (1<<5), | ||
| 4597 | /* Call bpf when the kernel is writing header options for the | ||
| 4598 | * outgoing packet. The bpf prog will first be called | ||
| 4599 | * to reserve space in a skb under | ||
| 4600 | * sock_ops->op == BPF_SOCK_OPS_HDR_OPT_LEN_CB. Then | ||
| 4601 | * the bpf prog will be called to write the header option(s) | ||
| 4602 | * under sock_ops->op == BPF_SOCK_OPS_WRITE_HDR_OPT_CB. | ||
| 4603 | * | ||
| 4604 | * Please refer to the comment in BPF_SOCK_OPS_HDR_OPT_LEN_CB | ||
| 4605 | * and BPF_SOCK_OPS_WRITE_HDR_OPT_CB for the header option | ||
| 4606 | * related helpers that will be useful to the bpf programs. | ||
| 4607 | * | ||
| 4608 | * The kernel gets its chance to reserve space and write | ||
| 4609 | * options first before the BPF program does. | ||
| 4610 | */ | ||
| 4611 | BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG = (1<<6), | ||
| 4169 | /* Mask of all currently supported cb flags */ | 4612 | /* Mask of all currently supported cb flags */ |
| 4170 | BPF_SOCK_OPS_ALL_CB_FLAGS = 0xF, | 4613 | BPF_SOCK_OPS_ALL_CB_FLAGS = 0x7F, |
| 4171 | }; | 4614 | }; |
| 4172 | 4615 | ||
| 4173 | /* List of known BPF sock_ops operators. | 4616 | /* List of known BPF sock_ops operators. |
| @@ -4223,6 +4666,63 @@ enum { · 49 unmodified lines | |||
| 4223 | */ | 4666 | */ |
| 4224 | BPF_SOCK_OPS_RTT_CB, /* Called on every RTT. | 4667 | BPF_SOCK_OPS_RTT_CB, /* Called on every RTT. |
| 4225 | */ | 4668 | */ |
| 4669 | BPF_SOCK_OPS_PARSE_HDR_OPT_CB, /* Parse the header option. | ||
| 4670 | * It will be called to handle | ||
| 4671 | * the packets received at | ||
| 4672 | * an already established | ||
| 4673 | * connection. | ||
| 4674 | * | ||
| 4675 | * sock_ops->skb_data: | ||
| 4676 | * Referring to the received skb. | ||
| 4677 | * It covers the TCP header only. | ||
| 4678 | * | ||
| 4679 | * bpf_load_hdr_opt() can also | ||
| 4680 | * be used to search for a | ||
| 4681 | * particular option. | ||
| 4682 | */ | ||
| 4683 | BPF_SOCK_OPS_HDR_OPT_LEN_CB, /* Reserve space for writing the | ||
| 4684 | * header option later in | ||
| 4685 | * BPF_SOCK_OPS_WRITE_HDR_OPT_CB. | ||
| 4686 | * Arg1: bool want_cookie. (in | ||
| 4687 | * writing SYNACK only) | ||
| 4688 | * | ||
| 4689 | * sock_ops->skb_data: | ||
| 4690 | * Not available because no header has | ||
| 4691 | * been written yet. | ||
| 4692 | * | ||
| 4693 | * sock_ops->skb_tcp_flags: | ||
| 4694 | * The tcp_flags of the | ||
| 4695 | * outgoing skb. (e.g. SYN, ACK, FIN). | ||
| 4696 | * | ||
| 4697 | * bpf_reserve_hdr_opt() should | ||
| 4698 | * be used to reserve space. | ||
| 4699 | */ | ||
| 4700 | BPF_SOCK_OPS_WRITE_HDR_OPT_CB, /* Write the header options | ||
| 4701 | * Arg1: bool want_cookie. (in | ||
| 4702 | * writing SYNACK only) | ||
| 4703 | * | ||
| 4704 | * sock_ops->skb_data: | ||
| 4705 | * Referring to the outgoing skb. | ||
| 4706 | * It covers the TCP header | ||
| 4707 | * that has already been written | ||
| 4708 | * by the kernel and the | ||
| 4709 | * earlier bpf-progs. | ||
| 4710 | * | ||
| 4711 | * sock_ops->skb_tcp_flags: | ||
| 4712 | * The tcp_flags of the outgoing | ||
| 4713 | * skb. (e.g. SYN, ACK, FIN). | ||
| 4714 | * | ||
| 4715 | * bpf_store_hdr_opt() should | ||
| 4716 | * be used to write the | ||
| 4717 | * option. | ||
| 4718 | * | ||
| 4719 | * bpf_load_hdr_opt() can also | ||
| 4720 | * be used to search for a | ||
| 4721 | * particular option that | ||
| 4722 | * has already been written | ||
| 4723 | * by the kernel or the | ||
| 4724 | * earlier bpf-progs. | ||
| 4725 | */ | ||
| 4226 | }; | 4726 | }; |
| 4227 | 4727 | ||
| 4228 | /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect | 4728 | /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect |
| @@ -4250,6 +4750,63 @@ enum { · 21 unmodified lines | |||
| 4250 | enum { | 4750 | enum { |
| 4251 | TCP_BPF_IW = 1001, /* Set TCP initial congestion window */ | 4751 | TCP_BPF_IW = 1001, /* Set TCP initial congestion window */ |
| 4252 | TCP_BPF_SNDCWND_CLAMP = 1002, /* Set sndcwnd_clamp */ | 4752 | TCP_BPF_SNDCWND_CLAMP = 1002, /* Set sndcwnd_clamp */ |
| 4753 | TCP_BPF_DELACK_MAX = 1003, /* Max delay ack in usecs */ | ||
| 4754 | TCP_BPF_RTO_MIN = 1004, /* Min delay ack in usecs */ | ||
| 4755 | /* Copy the SYN pkt to optval | ||
| 4756 | * | ||
| 4757 | * BPF_PROG_TYPE_SOCK_OPS only. It is similar to the | ||
| 4758 | * bpf_getsockopt(TCP_SAVED_SYN) but it does not limit | ||
| 4759 | * to only getting from the saved_syn. It can either get the | ||
| 4760 | * syn packet from: | ||
| 4761 | * | ||
| 4762 | * 1. the just-received SYN packet (only available when writing the | ||
| 4763 | * SYNACK). It will be useful when it is not necessary to | ||
| 4764 | * save the SYN packet for latter use. It is also the only way | ||
| 4765 | * to get the SYN during syncookie mode because the syn | ||
| 4766 | * packet cannot be saved during syncookie. | ||
| 4767 | * | ||
| 4768 | * OR | ||
| 4769 | * | ||
| 4770 | * 2. the earlier saved syn which was done by | ||
| 4771 | * bpf_setsockopt(TCP_SAVE_SYN). | ||
| 4772 | * | ||
| 4773 | * The bpf_getsockopt(TCP_BPF_SYN*) option will hide where the | ||
| 4774 | * SYN packet is obtained. | ||
| 4775 | * | ||
| 4776 | * If the bpf-prog does not need the IP[46] header, the | ||
| 4777 | * bpf-prog can avoid parsing the IP header by using | ||
| 4778 | * TCP_BPF_SYN. Otherwise, the bpf-prog can get both | ||
| 4779 | * IP[46] and TCP header by using TCP_BPF_SYN_IP. | ||
| 4780 | * | ||
| 4781 | * >0: Total number of bytes copied | ||
| 4782 | * -ENOSPC: Not enough space in optval. Only optlen number of | ||
| 4783 | * bytes is copied. | ||
| 4784 | * -ENOENT: The SYN skb is not available now and the earlier SYN pkt | ||
| 4785 | * is not saved by setsockopt(TCP_SAVE_SYN). | ||
| 4786 | */ | ||
| 4787 | TCP_BPF_SYN = 1005, /* Copy the TCP header */ | ||
| 4788 | TCP_BPF_SYN_IP = 1006, /* Copy the IP[46] and TCP header */ | ||
| 4789 | TCP_BPF_SYN_MAC = 1007, /* Copy the MAC, IP[46], and TCP header */ | ||
| 4790 | }; | ||
| 4791 | |||
| 4792 | enum { | ||
| 4793 | BPF_LOAD_HDR_OPT_TCP_SYN = (1ULL << 0), | ||
| 4794 | }; | ||
| 4795 | |||
| 4796 | /* args[0] value during BPF_SOCK_OPS_HDR_OPT_LEN_CB and | ||
| 4797 | * BPF_SOCK_OPS_WRITE_HDR_OPT_CB. | ||
| 4798 | */ | ||
| 4799 | enum { | ||
| 4800 | BPF_WRITE_HDR_TCP_CURRENT_MSS = 1, /* Kernel is finding the | ||
| 4801 | * total option spaces | ||
| 4802 | * required for an established | ||
| 4803 | * sk in order to calculate the | ||
| 4804 | * MSS. No skb is actually | ||
| 4805 | * sent. | ||
| 4806 | */ | ||
| 4807 | BPF_WRITE_HDR_TCP_SYNACK_COOKIE = 2, /* Kernel is in syncookie mode | ||
| 4808 | * when sending a SYN. | ||
| 4809 | */ | ||
| 4253 | }; | 4810 | }; |
| 4254 | 4811 | ||
| 4255 | struct bpf_perf_event_value { | 4812 | struct bpf_perf_event_value { |
| @@ -4447,4 +5004,34 @@ struct bpf_sk_lookup { · 191 unmodified lines | |||
| 4447 | __u32 local_port; /* Host byte order */ | 5004 | __u32 local_port; /* Host byte order */ |
| 4448 | }; | 5005 | }; |
| 4449 | 5006 | ||
| 5007 | /* | ||
| 5008 | * struct btf_ptr is used for typed pointer representation; the | ||
| 5009 | * type id is used to render the pointer data as the appropriate type | ||
| 5010 | * via the bpf_snprintf_btf() helper described above. A flags field - | ||
| 5011 | * potentially to specify additional details about the BTF pointer | ||
| 5012 | * (rather than its mode of display) - is included for future use. | ||
| 5013 | * Display flags - BTF_F_* - are passed to bpf_snprintf_btf separately. | ||
| 5014 | */ | ||
| 5015 | struct btf_ptr { | ||
| 5016 | void *ptr; | ||
| 5017 | __u32 type_id; | ||
| 5018 | __u32 flags; /* BTF ptr flags; unused at present. */ | ||
| 5019 | }; | ||
| 5020 | |||
| 5021 | /* | ||
| 5022 | * Flags to control bpf_snprintf_btf() behaviour. | ||
| 5023 | * - BTF_F_COMPACT: no formatting around type information | ||
| 5024 | * - BTF_F_NONAME: no struct/union member names/types | ||
| 5025 | * - BTF_F_PTR_RAW: show raw (unobfuscated) pointer values; | ||
| 5026 | * equivalent to %px. | ||
| 5027 | * - BTF_F_ZERO: show zero-valued struct/union members; they | ||
| 5028 | * are not displayed by default | ||
| 5029 | */ | ||
| 5030 | enum { | ||
| 5031 | BTF_F_COMPACT = (1ULL << 0), | ||
| 5032 | BTF_F_NONAME = (1ULL << 1), | ||
| 5033 | BTF_F_PTR_RAW = (1ULL << 2), | ||
| 5034 | BTF_F_ZERO = (1ULL << 3), | ||
| 5035 | }; | ||
| 5036 | |||
| 4450 | #endif /* _UAPI__LINUX_BPF_H__ */ | 5037 | #endif /* _UAPI__LINUX_BPF_H__ */ |