History

bpf: Allow sleepable programs to use LPM trie maps directly

The previous change relaxed the rcu_dereference annotations in lpm_trie.c so the trie walks no longer trip lockdep when reached from a sleepable BPF program holding only rcu_read_lock_trace(). By itself that only helps tries reached as the inner map of a map-of-maps, or from the classic-RCU syscall path: a sleepable program that references an LPM trie directly is still rejected at load time by check_map_prog_compatibility(), whose sleepable whitelist omits BPF_MAP_TYPE_LPM_TRIE: Sleepable programs can only use array, hash, ringbuf and local storage maps LPM trie nodes are allocated from a bpf_mem_alloc (trie->ma) and freed with bpf_mem_cache_free_rcu(), which chains a regular RCU grace period into a Tasks Trace grace period before the node -- and the value embedded in it that trie_lookup_elem() returns to the program -- is released. That is the same reclaim discipline BPF_MAP_TYPE_HASH relies on for sleepable access, so a value handed to a sleepable reader cannot be freed while the program is still running under rcu_read_lock_trace(). The writer paths take trie->lock across the walk and never relied on the RCU read-side lock to keep nodes alive. Add BPF_MAP_TYPE_LPM_TRIE to the sleepable map whitelist so these programs can use LPM tries directly.

Vlad Poenaru committed 5 months ago

Commit a3d76e21 parent 2f884d31 file · +1 −0Browse files

Showing the changes to kernel (1 of 1 file). Show all

Unified · Split

kernel/bpf/verifier.c+1−0
@@ -17737,6 +17737,7 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env, · 17736 unmodified lines
17737 case BPF_MAP_TYPE_PERCPU_HASH:17737 case BPF_MAP_TYPE_PERCPU_HASH:
17738 case BPF_MAP_TYPE_PERCPU_ARRAY:17738 case BPF_MAP_TYPE_PERCPU_ARRAY:
17739 case BPF_MAP_TYPE_LRU_PERCPU_HASH:17739 case BPF_MAP_TYPE_LRU_PERCPU_HASH:
17740 case BPF_MAP_TYPE_LPM_TRIE:
17740 case BPF_MAP_TYPE_ARRAY_OF_MAPS:17741 case BPF_MAP_TYPE_ARRAY_OF_MAPS:
17741 case BPF_MAP_TYPE_HASH_OF_MAPS:17742 case BPF_MAP_TYPE_HASH_OF_MAPS:
17742 case BPF_MAP_TYPE_RINGBUF:17743 case BPF_MAP_TYPE_RINGBUF: