The NMI entry code that switches to the normal kernel stack needs to be very careful not to clobber any extra stack slots on the NMI stack. The code is fine under the assumption that SWAPGS is just a normal instruction, but that assumption isn't really true. Use SWAPGS_UNSAFE_STACK instead. This is part of a fix for some random crashes that Sasha saw.
Unified · Split
| @@ -1190,9 +1190,12 @@ ENTRY(nmi) · 1189 unmodified lines | |||
| 1190 | * we don't want to enable interrupts, because then we'll end | 1190 | * we don't want to enable interrupts, because then we'll end |
| 1191 | * up in an awkward situation in which IRQs are on but NMIs | 1191 | * up in an awkward situation in which IRQs are on but NMIs |
| 1192 | * are off. | 1192 | * are off. |
| 1193 | * | ||
| 1194 | * We also must not push anything to the stack before switching | ||
| 1195 | * stacks lest we corrupt the "NMI executing" variable. | ||
| 1193 | */ | 1196 | */ |
| 1194 | 1197 | ||
| 1195 | SWAPGS | 1198 | SWAPGS_UNSAFE_STACK |
| 1196 | cld | 1199 | cld |
| 1197 | movq %rsp, %rdx | 1200 | movq %rsp, %rdx |
| 1198 | movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp | 1201 | movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp |