History

bpf: Avoid deadlock when using queue and stack maps from NMI

Sysbot discovered that the queue and stack maps can deadlock if they are being used from a BPF program that can be called from NMI context (such as one that is attached to a perf HW counter event). To fix this, add an in_nmi() check and use raw_spin_trylock() in NMI context, erroring out if grabbing the lock fails.

Toke Høiland-Jørgensen committed 3 years ago

Commit a34a9f11 parent b772b701 file · +18 −3Browse files

Showing the changes to kernel/bpf/queue_stack_maps.c (1 of 1 file). Show all

Unified · Split

kernel/bpf/queue_stack_maps.c+18−3
@@ -98,7 +98,12 @@ static long __queue_map_get(struct bpf_map *map, void *value, bool delete) · 97 unmodified lines
9898 int err = 0;
9999 void *ptr;
100100
101− raw_spin_lock_irqsave(&qs->lock, flags);
101+ if (in_nmi()) {
102+ if (!raw_spin_trylock_irqsave(&qs->lock, flags))
103+ return -EBUSY;
104+ } else {
105+ raw_spin_lock_irqsave(&qs->lock, flags);
106+ }
102107
103108 if (queue_stack_map_is_empty(qs)) {
104109 memset(value, 0, qs->map.value_size);
@@ -128,7 +133,12 @@ static long __stack_map_get(struct bpf_map *map, void *value, bool delete) · 23 unmodified lines
128133 void *ptr;
129134 u32 index;
130135
131− raw_spin_lock_irqsave(&qs->lock, flags);
136+ if (in_nmi()) {
137+ if (!raw_spin_trylock_irqsave(&qs->lock, flags))
138+ return -EBUSY;
139+ } else {
140+ raw_spin_lock_irqsave(&qs->lock, flags);
141+ }
132142
133143 if (queue_stack_map_is_empty(qs)) {
134144 memset(value, 0, qs->map.value_size);
@@ -193,7 +203,12 @@ static long queue_stack_map_push_elem(struct bpf_map *map, void *value, · 58 unmodified lines
193203 if (flags & BPF_NOEXIST || flags > BPF_EXIST)
194204 return -EINVAL;
195205
196− raw_spin_lock_irqsave(&qs->lock, irq_flags);
206+ if (in_nmi()) {
207+ if (!raw_spin_trylock_irqsave(&qs->lock, irq_flags))
208+ return -EBUSY;
209+ } else {
210+ raw_spin_lock_irqsave(&qs->lock, irq_flags);
211+ }
197212
198213 if (queue_stack_map_is_full(qs)) {
199214 if (!replace) {

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t