History

sched/core: Fix inter-class wakeup_preempt()

The way wakeup_preempt() works since commit 704069649b5b ("sched/core: Rework sched_class::wakeup_preempt() and rq_modified_*()") is that it will call rq->next_class->wakeup_preempt(rq, p) when p is of an equal or higher class, and raise ->next_class when higher. This means that: running idle task wakeup fair-A (next_class == idle) if (sched_class_above(fair, idle)) { wakeup_preempt_idle(fair-A); resched_curr(rq); next_class = fair; } wakeup fair-B (next_class == fair) if (fair == fair) wakeup_preempt_fair(fair-B); (but current is idle) All wakeup_preempt_$class() methods, except for wakeup_preempt_scx() (for whoem this was build) ignore cross-class wakeups by testing if @p is of the right class, but per the above case, it also should check current. This is mostly harmless in the current form, but will lead to trouble with later patches.

Peter Zijlstra committed 4 months ago

Commit fa02b281 parent dc59e4f3 files · +8 −4Browse files

Showing the changes to kernel (3 of 3 files). Show all

Unified · Split

3 files
  1. kernel/sched/deadline.c +4 −2
  2. kernel/sched/fair.c +2 −1
  3. kernel/sched/rt.c +2 −1
kernel/sched/deadline.c+4−2
@@ -2733,15 +2733,17 @@ static int balance_dl(struct rq *rq, struct rq_flags *rf) · 2732 unmodified lines
27332733 */
27342734 static void wakeup_preempt_dl(struct rq *rq, struct task_struct *p, int flags)
27352735 {
2736+ struct task_struct *donor = rq->donor;
27362737 /*
27372738 * Can only get preempted by stop-class, and those should be
27382739 * few and short lived, doesn't really make sense to push
27392740 * anything away for that.
27402741 */
2741− if (p->sched_class != &dl_sched_class)
2742+ if (p->sched_class != &dl_sched_class ||
2743+ donor->sched_class != &dl_sched_class)
27422744 return;
27432745
2744− if (dl_entity_preempt(&p->dl, &rq->donor->dl)) {
2746+ if (dl_entity_preempt(&p->dl, &donor->dl)) {
27452747 resched_curr(rq);
27462748 return;
27472749 }
kernel/sched/fair.c+2−1
@@ -9778,7 +9778,8 @@ static void wakeup_preempt_fair(struct rq *rq, struct task_struct *p, int wake_f · 9777 unmodified lines
97789778 /*
97799779 * XXX Getting preempted by higher class, try and find idle CPU?
97809780 */
9781− if (p->sched_class != &fair_sched_class)
9781+ if (p->sched_class != &fair_sched_class ||
9782+ donor->sched_class != &fair_sched_class)
97829783 return;
97839784
97849785 if (unlikely(se == pse))
kernel/sched/rt.c+2−1
@@ -1629,7 +1629,8 @@ static void wakeup_preempt_rt(struct rq *rq, struct task_struct *p, int flags) · 1628 unmodified lines
16291629 /*
16301630 * XXX If we're preempted by DL, queue a push?
16311631 */
1632− if (p->sched_class != &rt_sched_class)
1632+ if (p->sched_class != &rt_sched_class ||
1633+ donor->sched_class != &rt_sched_class)
16331634 return;
16341635
16351636 if (p->prio < donor->prio) {