As with shmem uffd-wp special ptes, only drop the uffd-wp special swap pte if unmapping an entire vma or synchronized such that faults can not race with the unmap operation. This requires passing zap_flags all the way to the lowest level hugetlb unmap routine: __unmap_hugepage_range. In general, unmap calls originated in hugetlbfs code will pass the ZAP_FLAG_DROP_MARKER flag as synchronization is in place to prevent faults. The exception is hole punch which will first unmap without any synchronization. Later when hole punch actually removes the page from the file, it will check to see if there was a subsequent fault and if so take the hugetlb fault mutex while unmapping again. This second unmap will pass in ZAP_FLAG_DROP_MARKER. The justification of "whether to apply ZAP_FLAG_DROP_MARKER flag when unmap a hugetlb range" is (IMHO): we should never reach a state when a page fault could errornously fault in a page-cache page that was wr-protected to be writable, even in an extremely short period. That could happen if e.g. we pass ZAP_FLAG_DROP_MARKER when hugetlbfs_punch_hole() calls hugetlb_vmdelete_list(), because if a page faults after that call and before remove_inode_hugepages() is executed, the page cache can be mapped writable again in the small racy window, that can cause unexpected data overwritten. [peterx@redhat.com: fix sparse warning] Link: https://lkml.kernel.org/r/Ylcdw8I1L5iAoWhb@xz-m1.local [akpm@linux-foundation.org: move zap_flags_t from mm.h to mm_types.h to fix build issues] Link: https://lkml.kernel.org/r/20220405014915.14873-1-peterx@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com> Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: David Hildenbrand <david@redhat.com> Cc: Hugh Dickins <hughd@google.com> Cc: Jerome Glisse <jglisse@redhat.com> Cc: "Kirill A . Shutemov" <kirill@shutemov.name> Cc: Matthew Wilcox <willy@infradead.org> Cc: Mike Rapoport <rppt@linux.vnet.ibm.com> Cc: Nadav Amit <nadav.amit@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Showing the changes to include (3 of 6 files). Show all
Unified · Split
| @@ -143,11 +143,12 @@ long follow_hugetlb_page(struct mm_struct *, struct vm_area_struct *, · 142 unmodified lines | ||
| 143 | 143 | unsigned long *, unsigned long *, long, unsigned int, |
| 144 | 144 | int *); |
| 145 | 145 | void unmap_hugepage_range(struct vm_area_struct *, |
| 146 | − unsigned long, unsigned long, struct page *); | |
| 146 | + unsigned long, unsigned long, struct page *, | |
| 147 | + zap_flags_t); | |
| 147 | 148 | void __unmap_hugepage_range_final(struct mmu_gather *tlb, |
| 148 | 149 | struct vm_area_struct *vma, |
| 149 | 150 | unsigned long start, unsigned long end, |
| 150 | − struct page *ref_page); | |
| 151 | + struct page *ref_page, zap_flags_t zap_flags); | |
| 151 | 152 | void hugetlb_report_meminfo(struct seq_file *); |
| 152 | 153 | int hugetlb_report_node_meminfo(char *buf, int len, int nid); |
| 153 | 154 | void hugetlb_show_meminfo(void); |
| @@ -406,7 +407,8 @@ static inline unsigned long hugetlb_change_protection( · 252 unmodified lines | ||
| 406 | 407 | |
| 407 | 408 | static inline void __unmap_hugepage_range_final(struct mmu_gather *tlb, |
| 408 | 409 | struct vm_area_struct *vma, unsigned long start, |
| 409 | − unsigned long end, struct page *ref_page) | |
| 410 | + unsigned long end, struct page *ref_page, | |
| 411 | + zap_flags_t zap_flags) | |
| 410 | 412 | { |
| 411 | 413 | BUG(); |
| 412 | 414 | } |
| @@ -3428,8 +3428,6 @@ madvise_set_anon_name(struct mm_struct *mm, unsigned long start, · 3427 unmodified lines | ||
| 3428 | 3428 | } |
| 3429 | 3429 | #endif |
| 3430 | 3430 | |
| 3431 | −typedef unsigned int __bitwise zap_flags_t; | |
| 3432 | − | |
| 3433 | 3431 | /* |
| 3434 | 3432 | * Whether to drop the pte markers, for example, the uffd-wp information for |
| 3435 | 3433 | * file-backed memory. This should only be specified when we will completely |
| @@ -863,4 +863,6 @@ enum fault_flag { · 862 unmodified lines | ||
| 863 | 863 | FAULT_FLAG_ORIG_PTE_VALID = 1 << 11, |
| 864 | 864 | }; |
| 865 | 865 | |
| 866 | +typedef unsigned int __bitwise zap_flags_t; | |
| 867 | + | |
| 866 | 868 | #endif /* _LINUX_MM_TYPES_H */ |