smp_call_function_single() disables preemption while it validates the target CPU, prepares the call single data, queues the callback and sends the IPI. For the !wait case, preemption protects the per-CPU csd_data from concurrent modification by another task on the same CPU. For the wait case, the CSD is stack allocated and no other task can reuse it. CPU pinning is still required until the callback has been queued and the IPI has been sent, to ensure that the target CPU cannot be offlined after the online check but before dispatch. After generic_exec_single() has queued the callback, the synchronous csd_lock_wait() invocation at the end of the execution does not require the caller to remain pinned to the current CPU. Enable preemption before csd_lock_wait() to shorten the preemption-disabled section.
Showing the changes to kernel (1 of 1 file). Show all
Unified · Split
| @@ -712,11 +712,16 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info, · 711 unmodified lines | ||
| 712 | 712 | |
| 713 | 713 | err = generic_exec_single(cpu, csd); |
| 714 | 714 | |
| 715 | + /* | |
| 716 | + * @csd is stack-allocated when @wait is true. No concurrent access | |
| 717 | + * except from the IPI completion path, so we can re-enable preemption | |
| 718 | + * early to reduce latency. | |
| 719 | + */ | |
| 720 | + put_cpu(); | |
| 721 | + | |
| 715 | 722 | if (wait) |
| 716 | 723 | csd_lock_wait(csd); |
| 717 | 724 | |
| 718 | − put_cpu(); | |
| 719 | − | |
| 720 | 725 | return err; |
| 721 | 726 | } |
| 722 | 727 | EXPORT_SYMBOL(smp_call_function_single); |