Entering RCU idle mode may cause a deferred wake up of an RCU NOCB_GP kthread (rcuog) to be serviced. Usually a local wake up happening while running the idle task is handled in one of the need_resched() checks carefully placed within the idle loop that can break to the scheduler. Unfortunately the call to rcu_idle_enter() is already beyond the last generic need_resched() check and we may halt the CPU with a resched request unhandled, leaving the task hanging. Fix this with splitting the rcuog wakeup handling from rcu_idle_enter() and place it before the last generic need_resched() check in the idle loop. It is then assumed that no call to call_rcu() will be performed after that in the idle loop until the CPU is put in low power mode.
Showing the changes to kernel (3 of 4 files). Show all
Unified · Split
| @@ -671,10 +671,7 @@ static noinstr void rcu_eqs_enter(bool user) · 670 unmodified lines | |||
| 671 | */ | 671 | */ |
| 672 | void rcu_idle_enter(void) | 672 | void rcu_idle_enter(void) |
| 673 | { | 673 | { |
| 674 | struct rcu_data *rdp = this_cpu_ptr(&rcu_data); | ||
| 675 | |||
| 676 | lockdep_assert_irqs_disabled(); | 674 | lockdep_assert_irqs_disabled(); |
| 677 | do_nocb_deferred_wakeup(rdp); | ||
| 678 | rcu_eqs_enter(false); | 675 | rcu_eqs_enter(false); |
| 679 | } | 676 | } |
| 680 | EXPORT_SYMBOL_GPL(rcu_idle_enter); | 677 | EXPORT_SYMBOL_GPL(rcu_idle_enter); |
| @@ -2187,6 +2187,11 @@ static void do_nocb_deferred_wakeup(struct rcu_data *rdp) · 2186 unmodified lines | |||
| 2187 | do_nocb_deferred_wakeup_common(rdp); | 2187 | do_nocb_deferred_wakeup_common(rdp); |
| 2188 | } | 2188 | } |
| 2189 | 2189 | ||
| 2190 | void rcu_nocb_flush_deferred_wakeup(void) | ||
| 2191 | { | ||
| 2192 | do_nocb_deferred_wakeup(this_cpu_ptr(&rcu_data)); | ||
| 2193 | } | ||
| 2194 | |||
| 2190 | void __init rcu_init_nohz(void) | 2195 | void __init rcu_init_nohz(void) |
| 2191 | { | 2196 | { |
| 2192 | int cpu; | 2197 | int cpu; |
| @@ -285,6 +285,7 @@ static void do_idle(void) · 284 unmodified lines | |||
| 285 | } | 285 | } |
| 286 | 286 | ||
| 287 | arch_cpu_idle_enter(); | 287 | arch_cpu_idle_enter(); |
| 288 | rcu_nocb_flush_deferred_wakeup(); | ||
| 288 | 289 | ||
| 289 | /* | 290 | /* |
| 290 | * In poll mode we reenable interrupts and spin. Also if we | 291 | * In poll mode we reenable interrupts and spin. Also if we |