History

mm: do page fault accounting in handle_mm_fault

Patch series "mm: Page fault accounting cleanups", v5. This is v5 of the pf accounting cleanup series. It originates from Gerald Schaefer's report on an issue a week ago regarding to incorrect page fault accountings for retried page fault after commit 4064b9827063 ("mm: allow VM_FAULT_RETRY for multiple times"): https://lore.kernel.org/lkml/20200610174811.44b94525@thinkpad/ What this series did: - Correct page fault accounting: we do accounting for a page fault (no matter whether it's from #PF handling, or gup, or anything else) only with the one that completed the fault. For example, page fault retries should not be counted in page fault counters. Same to the perf events. - Unify definition of PERF_COUNT_SW_PAGE_FAULTS: currently this perf event is used in an adhoc way across different archs. Case (1): for many archs it's done at the entry of a page fault handler, so that it will also cover e.g. errornous faults. Case (2): for some other archs, it is only accounted when the page fault is resolved successfully. Case (3): there're still quite some archs that have not enabled this perf event. Since this series will touch merely all the archs, we unify this perf event to always follow case (1), which is the one that makes most sense. And since we moved the accounting into handle_mm_fault, the other two MAJ/MIN perf events are well taken care of naturally. - Unify definition of "major faults": the definition of "major fault" is slightly changed when used in accounting (not VM_FAULT_MAJOR). More information in patch 1. - Always account the page fault onto the one that triggered the page fault. This does not matter much for #PF handlings, but mostly for gup. More information on this in patch 25. Patchset layout: Patch 1: Introduced the accounting in handle_mm_fault(), not enabled. Patch 2-23: Enable the new accounting for arch #PF handlers one by one. Patch 24: Enable the new accounting for the rest outliers (gup, iommu, etc.) Patch 25: Cleanup GUP task_struct pointer since it's not needed any more This patch (of 25): This is a preparation patch to move page fault accountings into the general code in handle_mm_fault(). This includes both the per task flt_maj/flt_min counters, and the major/minor page fault perf events. To do this, the pt_regs pointer is passed into handle_mm_fault(). PERF_COUNT_SW_PAGE_FAULTS should still be kept in per-arch page fault handlers. So far, all the pt_regs pointer that passed into handle_mm_fault() is NULL, which means this patch should have no intented functional change.

Peter Xu committed 6 years ago

Commit bce617e1 parent ed03d9231 files · +103 −34Browse files

Showing the changes to include/linux (1 of 31 files). Show all

Unified · Split

include/linux/mm.h+5−2
@@ -38,6 +38,7 @@ struct file_ra_state; · 37 unmodified lines
38struct user_struct;38struct user_struct;
39struct writeback_control;39struct writeback_control;
40struct bdi_writeback;40struct bdi_writeback;
41struct pt_regs;
41 42
42void init_mm_internals(void);43void init_mm_internals(void);
43 44
@@ -1658,7 +1659,8 @@ int invalidate_inode_page(struct page *page); · 1614 unmodified lines
1658 1659
1659#ifdef CONFIG_MMU1660#ifdef CONFIG_MMU
1660extern vm_fault_t handle_mm_fault(struct vm_area_struct *vma,1661extern vm_fault_t handle_mm_fault(struct vm_area_struct *vma,
1661 unsigned long address, unsigned int flags);1662 unsigned long address, unsigned int flags,
1663 struct pt_regs *regs);
1662extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,1664extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
1663 unsigned long address, unsigned int fault_flags,1665 unsigned long address, unsigned int fault_flags,
1664 bool *unlocked);1666 bool *unlocked);
@@ -1668,7 +1670,8 @@ void unmap_mapping_range(struct address_space *mapping, · 3 unmodified lines
1668 loff_t const holebegin, loff_t const holelen, int even_cows);1670 loff_t const holebegin, loff_t const holelen, int even_cows);
1669#else1671#else
1670static inline vm_fault_t handle_mm_fault(struct vm_area_struct *vma,1672static inline vm_fault_t handle_mm_fault(struct vm_area_struct *vma,
1671 unsigned long address, unsigned int flags)1673 unsigned long address, unsigned int flags,
1674 struct pt_regs *regs)
1672{1675{
1673 /* should never happen if there's no MMU */1676 /* should never happen if there's no MMU */
1674 BUG();1677 BUG();