sched_ext: Use dsq->first_task instead of list_empty() in dispatch_enqueue() FIFO-tail

dispatch_enqueue()'s FIFO-tail path used list_empty(&dsq->list) to decide whether to set dsq->first_task on enqueue. dsq->list can contain parked BPF iterator cursors (SCX_DSQ_LNODE_ITER_CURSOR), so list_empty() is not a reliable "no real task" check. If the last real task is unlinked while a cursor is parked, first_task becomes NULL; the next FIFO-tail enqueue then sees list_empty() == false and skips the first_task update, leaving scx_bpf_dsq_peek() returning NULL for a non-empty DSQ. Test dsq->first_task directly, which already tracks only real tasks and is maintained under dsq->lock.

Tejun Heo committed 6 months ago

Commit 2f2ea771 parent cc2a3871 file · +6 −4Browse files

Showing the changes to kernel (1 of 1 file). Show all

Unified · Split

kernel/sched/ext.c+6−4

@@ -1495,11 +1495,13 @@ static void dispatch_enqueue(struct scx_sched sch, struct rq rq, · 1494 unmodified lines
1495	1495	if (!(dsq->id & SCX_DSQ_FLAG_BUILTIN))
1496	1496	rcu_assign_pointer(dsq->first_task, p);
1497	1497	} else {
1498		− bool was_empty;
1499		−
1500		− was_empty = list_empty(&dsq->list);
	1498	+ /*
	1499	+ * dsq->list can contain parked BPF iterator cursors, so
	1500	+ * list_empty() here isn't a reliable proxy for "no real
	1501	+ * task in the DSQ". Test dsq->first_task directly.
	1502	+ */
1501	1503	list_add_tail(&p->scx.dsq_list.node, &dsq->list);
1502		− if (was_empty && !(dsq->id & SCX_DSQ_FLAG_BUILTIN))
	1504	+ if (!dsq->first_task && !(dsq->id & SCX_DSQ_FLAG_BUILTIN))
1503	1505	rcu_assign_pointer(dsq->first_task, p);
1504	1506	}
1505	1507	}