History

mm/userfaultfd: add RWP fault delivery and expose UFFDIO_REGISTER_MODE_RWP

Wire the fault side of read-write protection tracking and turn the userspace interface on. An RWP-protected PTE is PAGE_NONE with the uffd bit set. The PROT_NONE triggers a fault on any access; the uffd bit distinguishes it from plain mprotect(PROT_NONE) or NUMA hinting. Fault dispatch, per level: PTE handle_pte_fault() -> do_uffd_rwp() PMD __handle_mm_fault() -> do_huge_pmd_uffd_rwp() hugetlb hugetlb_fault() -> hugetlb_handle_userfault() The RWP branches gate on userfaultfd_pte_rwp() / userfaultfd_huge_pmd_rwp() (VM_UFFD_RWP plus the uffd bit) and fall through to do_numa_page() / do_huge_pmd_numa_page() otherwise. Each delivers a UFFD_PAGEFAULT_FLAG_RWP message through handle_userfault(); the handler resolves it with UFFDIO_RWPROTECT clearing MODE_RWP. userfaultfd_must_wait() and userfaultfd_huge_must_wait() add matching protnone+uffd waiters so sync-mode fault handlers block correctly. Expose the UAPI: UFFDIO_REGISTER_MODE_RWP -> UFFD_API_REGISTER_MODES UFFD_FEATURE_RWP -> UFFD_API_FEATURES _UFFDIO_RWPROTECT -> UFFD_API_RANGE_IOCTLS UFFD_API_RANGE_IOCTLS_BASIC UFFD_FEATURE_RWP is masked out at UFFDIO_API time when PROT_NONE is not available or VM_UFFD_RWP aliases VM_NONE (32-bit), so userspace never sees an advertised-but-broken feature. Works on anonymous, shmem, and hugetlb memory.

Kiryl Shutsemau (Meta) committed 4 months ago

Commit 2d427bb1 parent 6eab8f27 files · +114 −8Browse files

Showing the changes to include/uapi (1 of 7 files). Show all

Unified · Split

include/uapi/linux/userfaultfd.h+8−4
@@ -25,7 +25,8 @@ · 24 unmodified lines
2525 #define UFFD_API ((__u64)0xAA)
2626 #define UFFD_API_REGISTER_MODES (UFFDIO_REGISTER_MODE_MISSING | \
2727 UFFDIO_REGISTER_MODE_WP | \
28− UFFDIO_REGISTER_MODE_MINOR)
28+ UFFDIO_REGISTER_MODE_MINOR | \
29+ UFFDIO_REGISTER_MODE_RWP)
2930 #define UFFD_API_FEATURES (UFFD_FEATURE_PAGEFAULT_FLAG_WP | \
3031 UFFD_FEATURE_EVENT_FORK | \
3132 UFFD_FEATURE_EVENT_REMAP | \
@@ -42,7 +43,8 @@ · 10 unmodified lines
4243 UFFD_FEATURE_WP_UNPOPULATED | \
4344 UFFD_FEATURE_POISON | \
4445 UFFD_FEATURE_WP_ASYNC | \
45− UFFD_FEATURE_MOVE)
46+ UFFD_FEATURE_MOVE | \
47+ UFFD_FEATURE_RWP)
4648 #define UFFD_API_IOCTLS \
4749 ((__u64)1 << _UFFDIO_REGISTER | \
4850 (__u64)1 << _UFFDIO_UNREGISTER | \
@@ -54,13 +56,15 @@ · 5 unmodified lines
5456 (__u64)1 << _UFFDIO_MOVE | \
5557 (__u64)1 << _UFFDIO_WRITEPROTECT | \
5658 (__u64)1 << _UFFDIO_CONTINUE | \
57− (__u64)1 << _UFFDIO_POISON)
59+ (__u64)1 << _UFFDIO_POISON | \
60+ (__u64)1 << _UFFDIO_RWPROTECT)
5861 #define UFFD_API_RANGE_IOCTLS_BASIC \
5962 ((__u64)1 << _UFFDIO_WAKE | \
6063 (__u64)1 << _UFFDIO_COPY | \
6164 (__u64)1 << _UFFDIO_WRITEPROTECT | \
6265 (__u64)1 << _UFFDIO_CONTINUE | \
63− (__u64)1 << _UFFDIO_POISON)
66+ (__u64)1 << _UFFDIO_POISON | \
67+ (__u64)1 << _UFFDIO_RWPROTECT)
6468
6569 /*
6670 * Valid ioctl command number range with this API is from 0x00 to