The commit c75890700455 ("x86/entry/64: Remove unneeded kernel CR3 switching") removed a CR3 write in the faulting path of load_gs_index(). But the path's FENCE_SWAPGS_USER_ENTRY has no fence operation if PTI is enabled, see spectre_v1_select_mitigation(). Rather, it depended on the serializing CR3 write of SWITCH_TO_KERNEL_CR3 and since it got removed, add a FENCE_SWAPGS_KERNEL_ENTRY call to make sure speculation is blocked. [ bp: Massage commit message and comment. ]
Unified · Split
| @@ -987,11 +987,6 @@ SYM_CODE_START_LOCAL(error_entry) · 986 unmodified lines | |||
| 987 | pushq %r12 | 987 | pushq %r12 |
| 988 | ret | 988 | ret |
| 989 | 989 | ||
| 990 | .Lerror_entry_done_lfence: | ||
| 991 | FENCE_SWAPGS_KERNEL_ENTRY | ||
| 992 | .Lerror_entry_done: | ||
| 993 | ret | ||
| 994 | |||
| 995 | /* | 990 | /* |
| 996 | * There are two places in the kernel that can potentially fault with | 991 | * There are two places in the kernel that can potentially fault with |
| 997 | * usergs. Handle them here. B stepping K8s sometimes report a | 992 | * usergs. Handle them here. B stepping K8s sometimes report a |
| @@ -1014,8 +1009,14 @@ SYM_CODE_START_LOCAL(error_entry) · 16 unmodified lines | |||
| 1014 | * .Lgs_change's error handler with kernel gsbase. | 1009 | * .Lgs_change's error handler with kernel gsbase. |
| 1015 | */ | 1010 | */ |
| 1016 | SWAPGS | 1011 | SWAPGS |
| 1017 | FENCE_SWAPGS_USER_ENTRY | 1012 | |
| 1018 | jmp .Lerror_entry_done | 1013 | /* |
| 1014 | * Issue an LFENCE to prevent GS speculation, regardless of whether it is a | ||
| 1015 | * kernel or user gsbase. | ||
| 1016 | */ | ||
| 1017 | .Lerror_entry_done_lfence: | ||
| 1018 | FENCE_SWAPGS_KERNEL_ENTRY | ||
| 1019 | ret | ||
| 1019 | 1020 | ||
| 1020 | .Lbstep_iret: | 1021 | .Lbstep_iret: |
| 1021 | /* Fix truncated RIP */ | 1022 | /* Fix truncated RIP */ |