History

bpf: Register dtor for freeing special fields

There is a race window where BPF hash map elements can leak special fields if the program with access to the map value recreates these special fields between the check_and_free_fields done on the map value and its eventual return to the memory allocator. Several ways were explored prior to this patch, most notably [0] tried to use a poison value to reject attempts to recreate special fields for map values that have been logically deleted but still accessible to BPF programs (either while sitting in the free list or when reused). While this approach works well for task work, timers, wq, etc., it is harder to apply the idea to kptrs, which have a similar race and failure mode. Instead, we change bpf_mem_alloc to allow registering destructor for allocated elements, such that when they are returned to the allocator, any special fields created while they were accessible to programs in the mean time will be freed. If these values get reused, we do not free the fields again before handing the element back. The special fields thus may remain initialized while the map value sits in a free list. When bpf_mem_alloc is retired in the future, a similar concept can be introduced to kmalloc_nolock-backed kmem_cache, paired with the existing idea of a constructor. Note that the destructor registration happens in map_check_btf, after the BTF record is populated and (at that point) avaiable for inspection and duplication. Duplication is necessary since the freeing of embedded bpf_mem_alloc can be decoupled from actual map lifetime due to logic introduced to reduce the cost of rcu_barrier()s in mem alloc free path in 9f2c6e96c65e ("bpf: Optimize rcu_barrier usage between hash map and bpf_mem_alloc."). As such, once all callbacks are done, we must also free the duplicated record. To remove dependency on the bpf_map itself, also stash the key size of the map to obtain value from htab_elem long after the map is gone. [0]: https://lore.kernel.org/bpf/20260216131341.1285427-1-mykyta.yatsenko5@gmail.com

Kumar Kartikeya Dwivedi committed 8 months ago

Commit 1df97a71 parent 6881af23 files · +140 −11Browse files

Showing the changes to include (1 of 3 files). Show all

Unified · Split

include/linux/bpf_mem_alloc.h+6−0
@@ -14,6 +14,8 @@ struct bpf_mem_alloc { · 13 unmodified lines
1414 struct obj_cgroup *objcg;
1515 bool percpu;
1616 struct work_struct work;
17+ void (*dtor_ctx_free)(void *ctx);
18+ void *dtor_ctx;
1719 };
1820
1921 /* 'size != 0' is for bpf_mem_alloc which manages fixed-size objects.
@@ -32,6 +34,10 @@ int bpf_mem_alloc_percpu_init(struct bpf_mem_alloc *ma, struct obj_cgroup *objcg · 12 unmodified lines
3234 /* The percpu allocation with a specific unit size. */
3335 int bpf_mem_alloc_percpu_unit_init(struct bpf_mem_alloc *ma, int size);
3436 void bpf_mem_alloc_destroy(struct bpf_mem_alloc *ma);
37+void bpf_mem_alloc_set_dtor(struct bpf_mem_alloc *ma,
38+ void (*dtor)(void *obj, void *ctx),
39+ void (*dtor_ctx_free)(void *ctx),
40+ void *ctx);
3541
3642 /* Check the allocation size for kmalloc equivalent allocator */
3743 int bpf_mem_alloc_check_size(bool percpu, size_t size);

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t