History

Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Daniel Borkmann says: ==================== pull-request: bpf-next 2022-09-05 The following pull-request contains BPF updates for your *net-next* tree. We've added 106 non-merge commits during the last 18 day(s) which contain a total of 159 files changed, 5225 insertions(+), 1358 deletions(-). There are two small merge conflicts, resolve them as follows: 1) tools/testing/selftests/bpf/DENYLIST.s390x Commit 27e23836ce22 ("selftests/bpf: Add lru_bug to s390x deny list") in bpf tree was needed to get BPF CI green on s390x, but it conflicted with newly added tests on bpf-next. Resolve by adding both hunks, result: [...] lru_bug # prog 'printk': failed to auto-attach: -524 setget_sockopt # attach unexpected error: -524 (trampoline) cb_refs # expected error message unexpected error: -524 (trampoline) cgroup_hierarchical_stats # JIT does not support calling kernel function (kfunc) htab_update # failed to attach: ERROR: strerror_r(-524)=22 (trampoline) [...] 2) net/core/filter.c Commit 1227c1771dd2 ("net: Fix data-races around sysctl_[rw]mem_(max|default).") from net tree conflicts with commit 29003875bd5b ("bpf: Change bpf_setsockopt(SOL_SOCKET) to reuse sk_setsockopt()") from bpf-next tree. Take the code as it is from bpf-next tree, result: [...] if (getopt) { if (optname == SO_BINDTODEVICE) return -EINVAL; return sk_getsockopt(sk, SOL_SOCKET, optname, KERNEL_SOCKPTR(optval), KERNEL_SOCKPTR(optlen)); } return sk_setsockopt(sk, SOL_SOCKET, optname, KERNEL_SOCKPTR(optval), *optlen); [...] The main changes are: 1) Add any-context BPF specific memory allocator which is useful in particular for BPF tracing with bonus of performance equal to full prealloc, from Alexei Starovoitov. 2) Big batch to remove duplicated code from bpf_{get,set}sockopt() helpers as an effort to reuse the existing core socket code as much as possible, from Martin KaFai Lau. 3) Extend BPF flow dissector for BPF programs to just augment the in-kernel dissector with custom logic. In other words, allow for partial replacement, from Shmulik Ladkani. 4) Add a new cgroup iterator to BPF with different traversal options, from Hao Luo. 5) Support for BPF to collect hierarchical cgroup statistics efficiently through BPF integration with the rstat framework, from Yosry Ahmed. 6) Support bpf_{g,s}et_retval() under more BPF cgroup hooks, from Stanislav Fomichev. 7) BPF hash table and local storages fixes under fully preemptible kernel, from Hou Tao. 8) Add various improvements to BPF selftests and libbpf for compilation with gcc BPF backend, from James Hilliard. 9) Fix verifier helper permissions and reference state management for synchronous callbacks, from Kumar Kartikeya Dwivedi. 10) Add support for BPF selftest's xskxceiver to also be used against real devices that support MAC loopback, from Maciej Fijalkowski. 11) Various fixes to the bpf-helpers(7) man page generation script, from Quentin Monnet. 12) Document BPF verifier's tnum_in(tnum_range(), ...) gotchas, from Shung-Hsi Yu. 13) Various minor misc improvements all over the place. * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (106 commits) bpf: Optimize rcu_barrier usage between hash map and bpf_mem_alloc. bpf: Remove usage of kmem_cache from bpf_mem_cache. bpf: Remove prealloc-only restriction for sleepable bpf programs. bpf: Prepare bpf_mem_alloc to be used by sleepable bpf programs. bpf: Remove tracing program restriction on map types bpf: Convert percpu hash map to per-cpu bpf_mem_alloc. bpf: Add percpu allocation support to bpf_mem_alloc. bpf: Batch call_rcu callbacks instead of SLAB_TYPESAFE_BY_RCU. bpf: Adjust low/high watermarks in bpf_mem_cache bpf: Optimize call_rcu in non-preallocated hash map. bpf: Optimize element count in non-preallocated hash map. bpf: Relax the requirement to use preallocated hash maps in tracing progs. samples/bpf: Reduce syscall overhead in map_perf_test. selftests/bpf: Improve test coverage of test_maps bpf: Convert hash map to bpf_mem_alloc. bpf: Introduce any context BPF specific memory allocator. selftest/bpf: Add test for bpf_getsockopt() bpf: Change bpf_getsockopt(SOL_IPV6) to reuse do_ipv6_getsockopt() bpf: Change bpf_getsockopt(SOL_IP) to reuse do_ip_getsockopt() bpf: Change bpf_getsockopt(SOL_TCP) to reuse do_tcp_getsockopt() ... ====================

Paolo Abeni committed 4 years ago

Commit 2786bcf2 parents 03fdb11 274052a159 files · +5225 −1358Browse files

Showing the changes to scripts (1 of 159 files). Show all

Unified · Split

scripts/bpf_doc.py+71−7
@@ -10,6 +10,9 @@ from __future__ import print_function · 9 unmodified lines
1010 import argparse
1111 import re
1212 import sys, os
13+import subprocess
14+
15+helpersDocStart = 'Start of BPF helper function descriptions:'
1316
1417 class NoHelperFound(BaseException):
1518 pass
@@ -47,6 +50,10 @@ class Helper(APIElement): · 31 unmodified lines
4750 @desc: textual description of the helper function
4851 @ret: description of the return value of the helper function
4952 """
53+ def __init__(self, *args, **kwargs):
54+ super().__init__(*args, **kwargs)
55+ self.enum_val = None
56+
5057 def proto_break_down(self):
5158 """
5259 Break down helper function protocol into smaller chunks: return type,
@@ -89,6 +96,7 @@ class HeaderParser(object): · 36 unmodified lines
8996 self.commands = []
9097 self.desc_unique_helpers = set()
9198 self.define_unique_helpers = []
99+ self.helper_enum_vals = {}
92100 self.desc_syscalls = []
93101 self.enum_syscalls = []
94102
@@ -233,7 +241,7 @@ class HeaderParser(object): · 138 unmodified lines
233241 self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str)
234242
235243 def parse_desc_helpers(self):
236− self.seek_to('* Start of BPF helper function descriptions:',
244+ self.seek_to(helpersDocStart,
237245 'Could not find start of eBPF helper descriptions list')
238246 while True:
239247 try:
@@ -245,30 +253,54 @@ class HeaderParser(object): · 5 unmodified lines
245253 break
246254
247255 def parse_define_helpers(self):
248− # Parse the number of FN(...) in #define __BPF_FUNC_MAPPER to compare
249− # later with the number of unique function names present in description.
256+ # Parse FN(...) in #define __BPF_FUNC_MAPPER to compare later with the
257+ # number of unique function names present in description and use the
258+ # correct enumeration value.
250259 # Note: seek_to(..) discards the first line below the target search text,
251260 # resulting in FN(unspec) being skipped and not added to self.define_unique_helpers.
252261 self.seek_to('#define __BPF_FUNC_MAPPER(FN)',
253262 'Could not find start of eBPF helper definition list')
254− # Searches for either one or more FN(\w+) defines or a backslash for newline
255− p = re.compile('\s*(FN\(\w+\))+|\\\\')
263+ # Searches for one FN(\w+) define or a backslash for newline
264+ p = re.compile('\s*FN\((\w+)\)|\\\\')
256265 fn_defines_str = ''
266+ i = 1 # 'unspec' is skipped as mentioned above
257267 while True:
258268 capture = p.match(self.line)
259269 if capture:
260270 fn_defines_str += self.line
271+ self.helper_enum_vals[capture.expand(r'bpf_\1')] = i
272+ i += 1
261273 else:
262274 break
263275 self.line = self.reader.readline()
264276 # Find the number of occurences of FN(\w+)
265277 self.define_unique_helpers = re.findall('FN\(\w+\)', fn_defines_str)
266278
279+ def assign_helper_values(self):
280+ seen_helpers = set()
281+ for helper in self.helpers:
282+ proto = helper.proto_break_down()
283+ name = proto['name']
284+ try:
285+ enum_val = self.helper_enum_vals[name]
286+ except KeyError:
287+ raise Exception("Helper %s is missing from enum bpf_func_id" % name)
288+
289+ # Enforce current practice of having the descriptions ordered
290+ # by enum value.
291+ seen_helpers.add(name)
292+ desc_val = len(seen_helpers)
293+ if desc_val != enum_val:
294+ raise Exception("Helper %s comment order (#%d) must be aligned with its position (#%d) in enum bpf_func_id" % (name, desc_val, enum_val))
295+
296+ helper.enum_val = enum_val
297+
267298 def run(self):
268299 self.parse_desc_syscall()
269300 self.parse_enum_syscall()
270301 self.parse_desc_helpers()
271302 self.parse_define_helpers()
303+ self.assign_helper_values()
272304 self.reader.close()
273305
274306 ###############################################################################
@@ -357,6 +389,31 @@ class PrinterRST(Printer): · 82 unmodified lines
357389
358390 print('')
359391
392+ def get_kernel_version(self):
393+ try:
394+ version = subprocess.run(['git', 'describe'], cwd=linuxRoot,
395+ capture_output=True, check=True)
396+ version = version.stdout.decode().rstrip()
397+ except:
398+ try:
399+ version = subprocess.run(['make', 'kernelversion'], cwd=linuxRoot,
400+ capture_output=True, check=True)
401+ version = version.stdout.decode().rstrip()
402+ except:
403+ return 'Linux'
404+ return 'Linux {version}'.format(version=version)
405+
406+ def get_last_doc_update(self, delimiter):
407+ try:
408+ cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch',
409+ '-L',
410+ '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
411+ date = subprocess.run(cmd, cwd=linuxRoot,
412+ capture_output=True, check=True)
413+ return date.stdout.decode().rstrip()
414+ except:
415+ return ''
416+
360417 class PrinterHelpersRST(PrinterRST):
361418 """
362419 A printer for dumping collected information about helpers as a ReStructured
@@ -378,6 +435,8 @@ list of eBPF helper functions · 15 unmodified lines
378435 -------------------------------------------------------------------------------
379436
380437 :Manual section: 7
438+:Version: {version}
439+{date_field}{date}
381440
382441 DESCRIPTION
383442 ===========
@@ -410,8 +469,13 @@ kernel at the top). · 26 unmodified lines
410469 HELPERS
411470 =======
412471 '''
472+ kernelVersion = self.get_kernel_version()
473+ lastUpdate = self.get_last_doc_update(helpersDocStart)
474+
413475 PrinterRST.print_license(self)
414− print(header)
476+ print(header.format(version=kernelVersion,
477+ date_field = ':Date: ' if lastUpdate else '',
478+ date=lastUpdate))
415479
416480 def print_footer(self):
417481 footer = '''
@@ -761,7 +825,7 @@ class PrinterHelpers(Printer): · 343 unmodified lines
761825 comma = ', '
762826 print(one_arg, end='')
763827
764− print(') = (void *) %d;' % len(self.seen_helpers))
828+ print(') = (void *) %d;' % helper.enum_val)
765829 print('')
766830
767831 ###############################################################################

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t