Daniel Borkmann says: ==================== pull-request: bpf-next 2021-02-16 The following pull-request contains BPF updates for your *net-next* tree. There's a small merge conflict between 7eeba1706eba ("tcp: Add receive timestamp support for receive zerocopy.") from net-next tree and 9cacf81f8161 ("bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE") from bpf-next tree. Resolve as follows: [...] lock_sock(sk); err = tcp_zerocopy_receive(sk, &zc, &tss); err = BPF_CGROUP_RUN_PROG_GETSOCKOPT_KERN(sk, level, optname, &zc, &len, err); release_sock(sk); [...] We've added 116 non-merge commits during the last 27 day(s) which contain a total of 156 files changed, 5662 insertions(+), 1489 deletions(-). The main changes are: 1) Adds support of pointers to types with known size among global function args to overcome the limit on max # of allowed args, from Dmitrii Banshchikov. 2) Add bpf_iter for task_vma which can be used to generate information similar to /proc/pid/maps, from Song Liu. 3) Enable bpf_{g,s}etsockopt() from all sock_addr related program hooks. Allow rewriting bind user ports from BPF side below the ip_unprivileged_port_start range, both from Stanislav Fomichev. 4) Prevent recursion on fentry/fexit & sleepable programs and allow map-in-map as well as per-cpu maps for the latter, from Alexei Starovoitov. 5) Add selftest script to run BPF CI locally. Also enable BPF ringbuffer for sleepable programs, both from KP Singh. 6) Extend verifier to enable variable offset read/write access to the BPF program stack, from Andrei Matei. 7) Improve tc & XDP MTU handling and add a new bpf_check_mtu() helper to query device MTU from programs, from Jesper Dangaard Brouer. 8) Allow bpf_get_socket_cookie() helper also be called from [sleepable] BPF tracing programs, from Florent Revest. 9) Extend x86 JIT to pad JMPs with NOPs for helping image to converge when otherwise too many passes are required, from Gary Lin. 10) Verifier fixes on atomics with BPF_FETCH as well as function-by-function verification both related to zero-extension handling, from Ilya Leoshkevich. 11) Better kernel build integration of resolve_btfids tool, from Jiri Olsa. 12) Batch of AF_XDP selftest cleanups and small performance improvement for libbpf's xsk map redirect for newer kernels, from Björn Töpel. 13) Follow-up BPF doc and verifier improvements around atomics with BPF_FETCH, from Brendan Jackman. 14) Permit zero-sized data sections e.g. if ELF .rodata section contains read-only data from local variables, from Yonghong Song. 15) veth driver skb bulk-allocation for ndo_xdp_xmit, from Lorenzo Bianconi. ====================
Showing the changes to include/uapi (1 of 156 files). Show all
Unified · Split
| @@ -1656,22 +1656,30 @@ union bpf_attr { · 1655 unmodified lines | |||
| 1656 | * networking traffic statistics as it provides a global socket | 1656 | * networking traffic statistics as it provides a global socket |
| 1657 | * identifier that can be assumed unique. | 1657 | * identifier that can be assumed unique. |
| 1658 | * Return | 1658 | * Return |
| 1659 | * A 8-byte long non-decreasing number on success, or 0 if the | 1659 | * A 8-byte long unique number on success, or 0 if the socket |
| 1660 | * socket field is missing inside *skb*. | 1660 | * field is missing inside *skb*. |
| 1661 | * | 1661 | * |
| 1662 | * u64 bpf_get_socket_cookie(struct bpf_sock_addr *ctx) | 1662 | * u64 bpf_get_socket_cookie(struct bpf_sock_addr *ctx) |
| 1663 | * Description | 1663 | * Description |
| 1664 | * Equivalent to bpf_get_socket_cookie() helper that accepts | 1664 | * Equivalent to bpf_get_socket_cookie() helper that accepts |
| 1665 | * *skb*, but gets socket from **struct bpf_sock_addr** context. | 1665 | * *skb*, but gets socket from **struct bpf_sock_addr** context. |
| 1666 | * Return | 1666 | * Return |
| 1667 | * A 8-byte long non-decreasing number. | 1667 | * A 8-byte long unique number. |
| 1668 | * | 1668 | * |
| 1669 | * u64 bpf_get_socket_cookie(struct bpf_sock_ops *ctx) | 1669 | * u64 bpf_get_socket_cookie(struct bpf_sock_ops *ctx) |
| 1670 | * Description | 1670 | * Description |
| 1671 | * Equivalent to **bpf_get_socket_cookie**\ () helper that accepts | 1671 | * Equivalent to **bpf_get_socket_cookie**\ () helper that accepts |
| 1672 | * *skb*, but gets socket from **struct bpf_sock_ops** context. | 1672 | * *skb*, but gets socket from **struct bpf_sock_ops** context. |
| 1673 | * Return | 1673 | * Return |
| 1674 | * A 8-byte long non-decreasing number. | 1674 | * A 8-byte long unique number. |
| 1675 | * | ||
| 1676 | * u64 bpf_get_socket_cookie(struct sock *sk) | ||
| 1677 | * Description | ||
| 1678 | * Equivalent to **bpf_get_socket_cookie**\ () helper that accepts | ||
| 1679 | * *sk*, but gets socket from a BTF **struct sock**. This helper | ||
| 1680 | * also works for sleepable programs. | ||
| 1681 | * Return | ||
| 1682 | * A 8-byte long unique number or 0 if *sk* is NULL. | ||
| 1675 | * | 1683 | * |
| 1676 | * u32 bpf_get_socket_uid(struct sk_buff *skb) | 1684 | * u32 bpf_get_socket_uid(struct sk_buff *skb) |
| 1677 | * Return | 1685 | * Return |
| @@ -2231,6 +2239,9 @@ union bpf_attr { · 553 unmodified lines | |||
| 2231 | * * > 0 one of **BPF_FIB_LKUP_RET_** codes explaining why the | 2239 | * * > 0 one of **BPF_FIB_LKUP_RET_** codes explaining why the |
| 2232 | * packet is not forwarded or needs assist from full stack | 2240 | * packet is not forwarded or needs assist from full stack |
| 2233 | * | 2241 | * |
| 2242 | * If lookup fails with BPF_FIB_LKUP_RET_FRAG_NEEDED, then the MTU | ||
| 2243 | * was exceeded and output params->mtu_result contains the MTU. | ||
| 2244 | * | ||
| 2234 | * long bpf_sock_hash_update(struct bpf_sock_ops *skops, struct bpf_map *map, void *key, u64 flags) | 2245 | * long bpf_sock_hash_update(struct bpf_sock_ops *skops, struct bpf_map *map, void *key, u64 flags) |
| 2235 | * Description | 2246 | * Description |
| 2236 | * Add an entry to, or update a sockhash *map* referencing sockets. | 2247 | * Add an entry to, or update a sockhash *map* referencing sockets. |
| @@ -3836,6 +3847,69 @@ union bpf_attr { · 1599 unmodified lines | |||
| 3836 | * Return | 3847 | * Return |
| 3837 | * A pointer to a struct socket on success or NULL if the file is | 3848 | * A pointer to a struct socket on success or NULL if the file is |
| 3838 | * not a socket. | 3849 | * not a socket. |
| 3850 | * | ||
| 3851 | * long bpf_check_mtu(void *ctx, u32 ifindex, u32 *mtu_len, s32 len_diff, u64 flags) | ||
| 3852 | * Description | ||
| 3853 | |||
| 3854 | * Check ctx packet size against exceeding MTU of net device (based | ||
| 3855 | * on *ifindex*). This helper will likely be used in combination | ||
| 3856 | * with helpers that adjust/change the packet size. | ||
| 3857 | * | ||
| 3858 | * The argument *len_diff* can be used for querying with a planned | ||
| 3859 | * size change. This allows to check MTU prior to changing packet | ||
| 3860 | * ctx. Providing an *len_diff* adjustment that is larger than the | ||
| 3861 | * actual packet size (resulting in negative packet size) will in | ||
| 3862 | * principle not exceed the MTU, why it is not considered a | ||
| 3863 | * failure. Other BPF-helpers are needed for performing the | ||
| 3864 | * planned size change, why the responsability for catch a negative | ||
| 3865 | * packet size belong in those helpers. | ||
| 3866 | * | ||
| 3867 | * Specifying *ifindex* zero means the MTU check is performed | ||
| 3868 | * against the current net device. This is practical if this isn't | ||
| 3869 | * used prior to redirect. | ||
| 3870 | * | ||
| 3871 | * The Linux kernel route table can configure MTUs on a more | ||
| 3872 | * specific per route level, which is not provided by this helper. | ||
| 3873 | * For route level MTU checks use the **bpf_fib_lookup**\ () | ||
| 3874 | * helper. | ||
| 3875 | * | ||
| 3876 | * *ctx* is either **struct xdp_md** for XDP programs or | ||
| 3877 | * **struct sk_buff** for tc cls_act programs. | ||
| 3878 | * | ||
| 3879 | * The *flags* argument can be a combination of one or more of the | ||
| 3880 | * following values: | ||
| 3881 | * | ||
| 3882 | * **BPF_MTU_CHK_SEGS** | ||
| 3883 | * This flag will only works for *ctx* **struct sk_buff**. | ||
| 3884 | * If packet context contains extra packet segment buffers | ||
| 3885 | * (often knows as GSO skb), then MTU check is harder to | ||
| 3886 | * check at this point, because in transmit path it is | ||
| 3887 | * possible for the skb packet to get re-segmented | ||
| 3888 | * (depending on net device features). This could still be | ||
| 3889 | * a MTU violation, so this flag enables performing MTU | ||
| 3890 | * check against segments, with a different violation | ||
| 3891 | * return code to tell it apart. Check cannot use len_diff. | ||
| 3892 | * | ||
| 3893 | * On return *mtu_len* pointer contains the MTU value of the net | ||
| 3894 | * device. Remember the net device configured MTU is the L3 size, | ||
| 3895 | * which is returned here and XDP and TX length operate at L2. | ||
| 3896 | * Helper take this into account for you, but remember when using | ||
| 3897 | * MTU value in your BPF-code. On input *mtu_len* must be a valid | ||
| 3898 | * pointer and be initialized (to zero), else verifier will reject | ||
| 3899 | * BPF program. | ||
| 3900 | * | ||
| 3901 | * Return | ||
| 3902 | * * 0 on success, and populate MTU value in *mtu_len* pointer. | ||
| 3903 | * | ||
| 3904 | * * < 0 if any input argument is invalid (*mtu_len* not updated) | ||
| 3905 | * | ||
| 3906 | * MTU violations return positive values, but also populate MTU | ||
| 3907 | * value in *mtu_len* pointer, as this can be needed for | ||
| 3908 | * implementing PMTU handing: | ||
| 3909 | * | ||
| 3910 | * * **BPF_MTU_CHK_RET_FRAG_NEEDED** | ||
| 3911 | * * **BPF_MTU_CHK_RET_SEGS_TOOBIG** | ||
| 3912 | * | ||
| 3839 | */ | 3913 | */ |
| 3840 | #define __BPF_FUNC_MAPPER(FN) \ | 3914 | #define __BPF_FUNC_MAPPER(FN) \ |
| 3841 | FN(unspec), \ | 3915 | FN(unspec), \ |
| @@ -4001,6 +4075,7 @@ union bpf_attr { · 159 unmodified lines | |||
| 4001 | FN(ktime_get_coarse_ns), \ | 4075 | FN(ktime_get_coarse_ns), \ |
| 4002 | FN(ima_inode_hash), \ | 4076 | FN(ima_inode_hash), \ |
| 4003 | FN(sock_from_file), \ | 4077 | FN(sock_from_file), \ |
| 4078 | FN(check_mtu), \ | ||
| 4004 | /* */ | 4079 | /* */ |
| 4005 | 4080 | ||
| 4006 | /* integer value in 'imm' field of BPF_CALL instruction selects which helper | 4081 | /* integer value in 'imm' field of BPF_CALL instruction selects which helper |
| @@ -4501,6 +4576,7 @@ struct bpf_prog_info { · 494 unmodified lines | |||
| 4501 | __aligned_u64 prog_tags; | 4576 | __aligned_u64 prog_tags; |
| 4502 | __u64 run_time_ns; | 4577 | __u64 run_time_ns; |
| 4503 | __u64 run_cnt; | 4578 | __u64 run_cnt; |
| 4579 | __u64 recursion_misses; | ||
| 4504 | } __attribute__((aligned(8))); | 4580 | } __attribute__((aligned(8))); |
| 4505 | 4581 | ||
| 4506 | struct bpf_map_info { | 4582 | struct bpf_map_info { |
| @@ -4981,9 +5057,13 @@ struct bpf_fib_lookup { · 474 unmodified lines | |||
| 4981 | __be16 sport; | 5057 | __be16 sport; |
| 4982 | __be16 dport; | 5058 | __be16 dport; |
| 4983 | 5059 | ||
| 4984 | /* total length of packet from network header - used for MTU check */ | 5060 | union { /* used for MTU check */ |
| 4985 | __u16 tot_len; | 5061 | /* input to lookup */ |
| 5062 | __u16 tot_len; /* L3 length from network hdr (iph->tot_len) */ | ||
| 4986 | 5063 | ||
| 5064 | /* output: MTU value */ | ||
| 5065 | __u16 mtu_result; | ||
| 5066 | }; | ||
| 4987 | /* input: L3 device index for lookup | 5067 | /* input: L3 device index for lookup |
| 4988 | * output: device index from FIB lookup | 5068 | * output: device index from FIB lookup |
| 4989 | */ | 5069 | */ |
| @@ -5029,6 +5109,17 @@ struct bpf_redir_neigh { · 39 unmodified lines | |||
| 5029 | }; | 5109 | }; |
| 5030 | }; | 5110 | }; |
| 5031 | 5111 | ||
| 5112 | /* bpf_check_mtu flags*/ | ||
| 5113 | enum bpf_check_mtu_flags { | ||
| 5114 | BPF_MTU_CHK_SEGS = (1U << 0), | ||
| 5115 | }; | ||
| 5116 | |||
| 5117 | enum bpf_check_mtu_ret { | ||
| 5118 | BPF_MTU_CHK_RET_SUCCESS, /* check and lookup successful */ | ||
| 5119 | BPF_MTU_CHK_RET_FRAG_NEEDED, /* fragmentation required to fwd */ | ||
| 5120 | BPF_MTU_CHK_RET_SEGS_TOOBIG, /* GSO re-segmentation needed to fwd */ | ||
| 5121 | }; | ||
| 5122 | |||
| 5032 | enum bpf_task_fd_type { | 5123 | enum bpf_task_fd_type { |
| 5033 | BPF_FD_TYPE_RAW_TRACEPOINT, /* tp name */ | 5124 | BPF_FD_TYPE_RAW_TRACEPOINT, /* tp name */ |
| 5034 | BPF_FD_TYPE_TRACEPOINT, /* tp name */ | 5125 | BPF_FD_TYPE_TRACEPOINT, /* tp name */ |