History

hugetlbfs: remove call to huge_pte_alloc without i_mmap_rwsem

Commit c0d0381ade79 ("hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization") requires callers of huge_pte_alloc to hold i_mmap_rwsem in at least read mode. This is because the explicit locking in huge_pmd_share (called by huge_pte_alloc) was removed. When restructuring the code, the call to huge_pte_alloc in the else block at the beginning of hugetlb_fault was missed. Unfortunately, that else clause is exercised when there is no page table entry. This will likely lead to a call to huge_pmd_share. If huge_pmd_share thinks pmd sharing is possible, it will traverse the mapping tree (i_mmap) without holding i_mmap_rwsem. If someone else is modifying the tree, bad things such as addressing exceptions or worse could happen. Simply remove the else clause. It should have been removed previously. The code following the else will call huge_pte_alloc with the appropriate locking. To prevent this type of issue in the future, add routines to assert that i_mmap_rwsem is held, and call these routines in huge pmd sharing routines.

Mike Kravetz committed 6 years ago

Commit 34ae2041 parent 15568294 files · +23 −12Browse files

Showing the changes to include/linux (2 of 4 files). Show all

Unified · Split

2 files
  1. include/linux/fs.h +10 −0
  2. include/linux/hugetlb.h +5 −3
include/linux/fs.h+10−0
@@ -518,6 +518,16 @@ static inline void i_mmap_unlock_read(struct address_space *mapping) · 517 unmodified lines
518518 up_read(&mapping->i_mmap_rwsem);
519519 }
520520
521+static inline void i_mmap_assert_locked(struct address_space *mapping)
522+{
523+ lockdep_assert_held(&mapping->i_mmap_rwsem);
524+}
525+
526+static inline void i_mmap_assert_write_locked(struct address_space *mapping)
527+{
528+ lockdep_assert_held_write(&mapping->i_mmap_rwsem);
529+}
530+
521531 /*
522532 * Might pages of this file be mapped into userspace?
523533 */
include/linux/hugetlb.h+5−3
@@ -164,7 +164,8 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, · 163 unmodified lines
164164 unsigned long addr, unsigned long sz);
165165 pte_t *huge_pte_offset(struct mm_struct *mm,
166166 unsigned long addr, unsigned long sz);
167−int huge_pmd_unshare(struct mm_struct *mm, unsigned long *addr, pte_t *ptep);
167+int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma,
168+ unsigned long *addr, pte_t *ptep);
168169 void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,
169170 unsigned long *start, unsigned long *end);
170171 struct page *follow_huge_addr(struct mm_struct *mm, unsigned long address,
@@ -203,8 +204,9 @@ static inline struct address_space *hugetlb_page_mapping_lock_write( · 32 unmodified lines
203204 return NULL;
204205 }
205206
206−static inline int huge_pmd_unshare(struct mm_struct *mm, unsigned long *addr,
207− pte_t *ptep)
207+static inline int huge_pmd_unshare(struct mm_struct *mm,
208+ struct vm_area_struct *vma,
209+ unsigned long *addr, pte_t *ptep)
208210 {
209211 return 0;
210212 }