History

bpf: Fix available-data accounting on 32-bit wrap in overwrite mode

In overwrite mode ringbuf_avail_data_sz() picks the newer of the consumer and overwrite positions before measuring how much data is available: return prod_pos - max(cons_pos, over_pos); max() is an ordering comparison, and consumer_pos, producer_pos and overwrite_pos are unsigned long, i.e. 32-bit on 32-bit architectures, where Documentation/bpf/ringbuf.rst allows them to wrap. Once one of the two positions has wrapped and the other has not, max() returns the older one: the result is then a modular difference close to 2^32, so the function reports far more available data than the ring can hold. Pollers using BPF_RB_AVAIL_DATA get a bogus figure, and epoll consumers can be woken with nothing to read. Compare distances rather than positions. prod_pos - X is the amount of data produced since X for either position, wrap or no wrap, so the newer position is simply the one with the smaller distance, which is also the value the function wants to return. 64-bit hosts are unaffected in practice: their counters would need 16 EiB to wrap. Found by review of the same class of bug fixed in "bpf: Fix pending_pos walk on 32-bit ring position wrap".

Israel Téllez García committed 3 months ago

Commit 3f611e91 parent 6ff5b561 file · +1 −1Browse files

Unified · Split

kernel/bpf/ringbuf.c+1−1
@@ -321,7 +321,7 @@ static unsigned long ringbuf_avail_data_sz(struct bpf_ringbuf *rb) · 320 unmodified lines
321 if (unlikely(rb->overwrite_mode)) {321 if (unlikely(rb->overwrite_mode)) {
322 over_pos = smp_load_acquire(&rb->overwrite_pos);322 over_pos = smp_load_acquire(&rb->overwrite_pos);
323 prod_pos = smp_load_acquire(&rb->producer_pos);323 prod_pos = smp_load_acquire(&rb->producer_pos);
324 return prod_pos - max(cons_pos, over_pos);324 return min(prod_pos - cons_pos, prod_pos - over_pos);
325 } else {325 } else {
326 prod_pos = smp_load_acquire(&rb->producer_pos);326 prod_pos = smp_load_acquire(&rb->producer_pos);
327 return prod_pos - cons_pos;327 return prod_pos - cons_pos;

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t