sched_ext: Use dsq->first_task instead of list_empty() in dispatch_enqueue() FIFO-tail

dispatch_enqueue()'s FIFO-tail path used list_empty(&dsq->list) to decide whether to set dsq->first_task on enqueue. dsq->list can contain parked BPF iterator cursors (SCX_DSQ_LNODE_ITER_CURSOR), so list_empty() is not a reliable "no real task" check. If the last real task is unlinked while a cursor is parked, first_task becomes NULL; the next FIFO-tail enqueue then sees list_empty() == false and skips the first_task update, leaving scx_bpf_dsq_peek() returning NULL for a non-empty DSQ. Test dsq->first_task directly, which already tracks only real tasks and is maintained under dsq->lock.

Tejun Heo committed 6 months ago

Commit 2f2ea771 parent cc2a3871 file · +6 −4Browse files

Showing the changes to kernel (1 of 1 file). Show all

Unified · Split

kernel/sched/ext.c+6−4

@@ -1495,11 +1495,13 @@ static void dispatch_enqueue(struct scx_sched sch, struct rq rq, · 1494 unmodified lines
1495	if (!(dsq->id & SCX_DSQ_FLAG_BUILTIN))	1495	if (!(dsq->id & SCX_DSQ_FLAG_BUILTIN))
1496	rcu_assign_pointer(dsq->first_task, p);	1496	rcu_assign_pointer(dsq->first_task, p);
1497	} else {	1497	} else {
1498	bool was_empty;	1498	/*
1499		1499	* dsq->list can contain parked BPF iterator cursors, so
1500	was_empty = list_empty(&dsq->list);	1500	* list_empty() here isn't a reliable proxy for "no real
		1501	* task in the DSQ". Test dsq->first_task directly.
		1502	*/
1501	list_add_tail(&p->scx.dsq_list.node, &dsq->list);	1503	list_add_tail(&p->scx.dsq_list.node, &dsq->list);
1502	if (was_empty && !(dsq->id & SCX_DSQ_FLAG_BUILTIN))	1504	if (!dsq->first_task && !(dsq->id & SCX_DSQ_FLAG_BUILTIN))
1503	rcu_assign_pointer(dsq->first_task, p);	1505	rcu_assign_pointer(dsq->first_task, p);
1504	}	1506	}
1505	}	1507	}