The commit 6d25be5782e4 ("sched/core, workqueues: Distangle worker accounting from rq lock") changed the schedule callbacks for workqueue and moved the schedule callback from the wakeup code to at end of schedule() in the worker's process context. It means that the callback wq_worker_running() is guaranteed that it sees the %WORKER_UNBOUND flag after scheduled since unbind_workers() is running on the same CPU that all the pool's workers bound to.
Showing the changes to kernel/workqueue.c (1 of 1 file). Show all
Unified · Split
| @@ -4979,6 +4979,9 @@ static void unbind_workers(int cpu) · 4978 unmodified lines | |||
| 4979 | * We've blocked all attach/detach operations. Make all workers | 4979 | * We've blocked all attach/detach operations. Make all workers |
| 4980 | * unbound and set DISASSOCIATED. Before this, all workers | 4980 | * unbound and set DISASSOCIATED. Before this, all workers |
| 4981 | * must be on the cpu. After this, they may become diasporas. | 4981 | * must be on the cpu. After this, they may become diasporas. |
| 4982 | * And the preemption disabled section in their sched callbacks | ||
| 4983 | * are guaranteed to see WORKER_UNBOUND since the code here | ||
| 4984 | * is on the same cpu. | ||
| 4982 | */ | 4985 | */ |
| 4983 | for_each_pool_worker(worker, pool) | 4986 | for_each_pool_worker(worker, pool) |
| 4984 | worker->flags |= WORKER_UNBOUND; | 4987 | worker->flags |= WORKER_UNBOUND; |
| @@ -4994,14 +4997,6 @@ static void unbind_workers(int cpu) · 9 unmodified lines | |||
| 4994 | 4997 | ||
| 4995 | mutex_unlock(&wq_pool_attach_mutex); | 4998 | mutex_unlock(&wq_pool_attach_mutex); |
| 4996 | 4999 | ||
| 4997 | /* | ||
| 4998 | * Call schedule() so that we cross rq->lock and thus can | ||
| 4999 | * guarantee sched callbacks see the %WORKER_UNBOUND flag. | ||
| 5000 | * This is necessary as scheduler callbacks may be invoked | ||
| 5001 | * from other cpus. | ||
| 5002 | */ | ||
| 5003 | schedule(); | ||
| 5004 | |||
| 5005 | /* | 5000 | /* |
| 5006 | * Sched callbacks are disabled now. Zap nr_running. | 5001 | * Sched callbacks are disabled now. Zap nr_running. |
| 5007 | * After this, nr_running stays zero and need_more_worker() | 5002 | * After this, nr_running stays zero and need_more_worker() |