Merge tag 'mm-stable-2026-06-18-09-26' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull MM updates from Andrew Morton: - "selftests/mm: clean up build output and verbosity" (Li Wang) Remove some noise from the MM selftests build - "mm: Free contiguous order-0 pages efficiently" (Ryan Roberts) Speed up the freeing of a batch of 0-order pages by first scanning them for coalescing opportunities. This is applicable to vfree() and to the releasing of frozen pages - "mm/damon: introduce DAMOS failed region quota charge ratio" (SeongJae Park) Address a DAMOS usability issue: The DAMOS quota often exhausts prematurely because it charges for all memory attempted, causing slow and inconsistent performance when actions fail on unreclaimable memory. To fix this, a new feature lets users set a smaller, flexible quota charge ratio (via a numerator and denominator) for failed regions. Since failed actions cause less overhead, reducing their quota cost ensures more predictable and efficient DAMOS processing - "selftests/cgroup: improve zswap tests robustness and support large page sizes" (Li Wang) Fix various spurious failures and improves the overall robustness of the cgroup zswap selftests - "fix MAP_DROPPABLE not supported errno" (Anthony Yznaga) Fix an issue in the mlock selftests on arm32 - "mm: huge_memory: clean up defrag sysfs with shared" (Breno Leitao) Some maintenance work in the huge_memory code - "treewide: fixup gfp_t printks" (Brendan Jackman) Use the special vprintf() gfp_t conversion in various places - "mm: Fix vmemmap optimization accounting and initialization" (Muchun Song) Fix several bugs in the vmemmap optimization, mainly around incorrect page accounting and memmap initialization in the DAX and memory hotplug paths. It also fixes pageblock migratetype initialization and struct page initialization for ZONE_DEVICE compound pages - "mm/damon: repost non-hotfix reviewed patches in damon/next tree" A sprinkle of unrelated minor bugfixes for DAMON - "mm: remove page_mapped()" (David Hildenbrand) Remove this function from the tree, replacing it with folio_mapped() - "mm/damon: let DAMON be paused and resumed" (SeongJae Park) Allow DAMON to be paused and resumed without losing its current state - "kasan: hw_tags: Disable tagging for stack and page-tables" (Muhammad Usama Anjum) Simplify and speed up kasan by removing its ineffective tagging of stacks and page tables - "mm/damon/reclaim,lru_sort: monitor all system rams by default" (SeongJae Park) Simplify deployment on diverse hardware like NUMA systems by updating DAMON_RECLAIM and DAMON_LRU_SORT to automatically monitor the physical address range covering all System RAM areas by default, replacing the overly restrictive behavior that only targeted the single largest memory block to save on negligible overhead - "mm/damon/sysfs: document filters/ directory as deprecated" (SeongJae Park) Update some DAMON docs - "mm: use spinlock guards for zone lock" (Dmitry Ilvokhin) Switch zone->lock handling over to using the guard() mechanisms - "mm/filemap: tighten mmap_miss hit accounting" (fujunjie) Fix a flaw where the mmap_miss counter over-credited page cache hits during fault-arounds and page-fault retries. This results in significant reduction of redundant synchronous mmap readahead I/O, drastically cutting down execution time and gigabytes read for sparse random or strided memory access workloads - "selftests/cgroup: Fix false positive failures in test_percpu_basic" (Li Wang) Fix a couple of false-positives in the cgroup kmem selftests - "mm/damon/reclaim: support monitoring intervals auto-tuning" (SeongJae Park) Add a new parameter to DAMON permitting DAMON_RECLAIM to automatically tune DAMON's sampling and aggregation intervals - "mm/damon/stat: add kdamond_pid parameter" (SeongJae Park) Change DAMON_STAT to provide the pid of its kdamond - "mm/kmemleak: dedupe verbose scan output" (Breno Leitao) Remove large amounts of duplicated backtraces from the verbose-mode kmemleak output - "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 1)" (David Hildenbrand) Reduce our use of CONFIG_HAVE_BOOTMEM_INFO_NODE, with a view to removing it entirely in a later series - "mm/damon: validate min_region_size to be power of 2" (Liew Rui Yan) Prevent users from passing a non-power-of-2 value of `addr_unit', as this later results in undesirable behavior - "mm: document read_pages and simplify usage" (Frederick Mayle) - "tools/mm/page-types: Fix misc bugs" (Ye Liu) Fix three issues in tools/mm/page-types.c - "mm: misc cleanups from __GFP_UNMAPPED series" (Brendan Jackman) Implement several cleanups in the page allocator and related code - "mm, swap: swap table phase IV: unify allocation" (Kairui Song) Unify the allocation and charging of anon and shmem swap in folios, provides better synchronization, consolidates the metadata management, hence dropping the static array and map, and improves performance - "mm/damon: introduce data attributes monitoring" (SeongJae Park( Extend DAMON to monitor general data attributes other than accesses - "mm/vmalloc: free unused pages on vrealloc() shrink" (Shivam Kalra) Implement the TODO in vrealloc() to unmap and free unused pages when shrinking across a page boundary - "mm/damon: documentation and comment fixes" (niecheng) - "remove mmap_action success, error hooks" (Lorenzo Stoakes) Eliminate custom hooks from mmap_action by removing the problematic success_hook which allowed drivers to improperly access uninitialized VMAs. It replaces the error_hook with a simple error-code field and updates the memory char driver accordingly - "mm/damon: minor improvements for code readability and tests" (SeongJae Park) - "mm/damon: fix macro arguments and clarify quota goals doc" (Maksym Shcherba) - "userfaultfd: merge fs/userfaultfd.c into mm/userfaultfd.c" (Mike Rapoport) - "mm/mglru: improve reclaim loop and dirty folio" (Kairui Song and others) Clean up and slightly improves MGLRU's reclaim loop and dirty writeback handling. Large performance improvements are measured - "use vma locks for proc/pid/{smaps|numa_maps} reads" (Suren Baghdasaryan) Use per-vma locks when reading /proc/pid/smaps and numa_maps similar to reduce contention on central mmap_lock - "refactors thpsize_shmem_enabled_store() and thpsize_shmem_enabled_show()" (Ran Xiaokai) Some cleanup work in the THP code - "selftests/memfd: fix compilation warnings" (Konstantin Khorenko) Fix a few build glitches in the memfd selftest code. - "memcg: shrink obj_stock_pcp and cache multiple objcgs" (Shakeel Butt) Resolve a 68% performance regression caused by NUMA-node cache thrashing around struct obj_stock_pcp by shrinking its existing fields and expanding it into a multi-slot array that caches up to five obj_cgroup pointers per CPU, allowing per-node variants of the same memcg to coexist within a single 64-byte cache line. - "zram: writeback fixes" (Sergey Senozhatsky) address a couple of unrelated zram writeback issues - "mm: switch THP shrinker to list_lru" (Johannes Weiner) Resolve NUMA-awareness issues and streamlines callsite interaction by refactoring and extending the list_lru API to completely replace the complex, open-coded deferred split queue for Transparent Huge Pages - "mm: improve large folio readahead for exec memory" (Usama Arif) Improve large-folio readahead on systems like 64K-page arm64 by preventing the mmap_miss check from permanently disabling target-oriented VM_EXEC readahead, and by generalizing the force_thp_readahead gate to support mappings with any usefully large maximum folio order under the cache cap. - "userfaultfd/pagemap: pre-existing fixes" (Kiryl Shutsemau) Fix a bunch of minor issues in the userfaultfd/pagemap, all of which were flagged by Sashiko review of proposed new material - "mm/sparse-vmemmap: Provide generic vmemmap_set_pmd() and vmemmap_check_pmd()" (Muchun Song) Provide generic versions of these two functions so the four arch-specific implementations can be removed. - "mm/swap, PM: hibernate: fix swapoff race in uswsusp by pinning swap device" (Youngjun Park) Address a uswsusp-vs-swapoff race and reduces the swap device reference taking/releasing frequency. - "mm/hmm: A fix and a selftest" (Dev Jain) * tag 'mm-stable-2026-06-18-09-26' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (321 commits) selftests/mm/hmm-tests: test pagemap reads of PMD device-private entries fs/proc/task_mmu: do not warn on seeing non-migration pmd entry lib/test_hmm: check alloc_page_vma() return value and handle OOM mm/compaction: cap compact_gap() at COMPACT_CLUSTER_MAX mm/swap: remove redundant swap device reference in alloc/free mm/swap, PM: hibernate: fix swapoff race in uswsusp by pinning swap device mm/filemap: use folio_next_index() for start vmalloc: fix NULL pointer dereference in is_vm_area_hugepages() sparc/mm: drop vmemmap_check_pmd helper and use generic code loongarch/mm: drop vmemmap_check_pmd helper and use generic code riscv/mm: drop vmemmap_pmd helpers and use generic code arm64/mm: drop vmemmap_pmd helpers and use generic code mm/sparse-vmemmap: provide generic vmemmap_set_pmd() and vmemmap_check_pmd() rust: page: mark Page::nid as inline userfaultfd: build __VMA_UFFD_FLAGS from config-gated masks userfaultfd: gate must_wait writability check on pte_present() mm/huge_memory: preserve pmd_swp_uffd_wp on device-private PMD downgrade fs/proc/task_mmu: fix hugetlb self-deadlock in pagemap_scan_pte_hole() fs/proc/task_mmu: use huge_page_size() in pagemap_scan_hugetlb_entry() fs/proc/task_mmu: fix make_uffd_wp_huge_pte() prot-update race ...

Linus Torvalds committed 4 months ago

Commit a552c812 parents c98d767 e3d8707169 files · +9215 −5618Browse files

Showing the changes to include (29 of 169 files). Show all

Unified · Split

29 files

include/asm-generic/pgalloc.h +1 −1
include/linux/bootmem_info.h +0 −1
include/linux/compaction.h +5 −5
include/linux/damon.h +111 −25
include/linux/gfp.h +4 −0
include/linux/gfp_types.h +3 −3
include/linux/highmem-internal.h +1 −1
include/linux/huge_mm.h +42 −5
include/linux/list_lru.h +70 −0
include/linux/memcontrol.h +17 −14
include/linux/memory.h +5 −2
include/linux/memory_hotplug.h +5 −3
include/linux/mm.h +44 −18
include/linux/mm_inline.h +1 −1
include/linux/mm_types.h +3 −16
include/linux/mmu_notifier.h +2 −2
include/linux/mmzone.h +6 −17
include/linux/nodemask.h +9 −9
include/linux/page_ref.h +18 −0
include/linux/pageblock-flags.h +3 −3
include/linux/pagemap.h +2 −0
include/linux/swap.h +12 −12
include/linux/swap_cgroup.h +0 −47
include/linux/thread_info.h +1 −1
include/linux/userfaultfd_k.h +2 −38
include/linux/vmalloc.h +3 −1
include/linux/vmpressure.h +5 −4
include/trace/events/damon.h +38 −0
include/trace/events/vmscan.h +52 −0

include/asm-generic/pgalloc.h+1−1

@@ -4,7 +4,7 @@ · 3 unmodified lines
4	4
5	5	#ifdef CONFIG_MMU
6	6
7		−#define GFP_PGTABLE_KERNEL (GFP_KERNEL \| __GFP_ZERO)
	7	+#define GFP_PGTABLE_KERNEL (GFP_KERNEL \| __GFP_ZERO \| __GFP_SKIP_KASAN)
8	8	#define GFP_PGTABLE_USER (GFP_PGTABLE_KERNEL \| __GFP_ACCOUNT)
9	9
10	10	/**

include/linux/bootmem_info.h+0−1

@@ -82,7 +82,6 @@ static inline void get_page_bootmem(unsigned long info, struct page *page, · 81 unmodified lines
82	82
83	83	static inline void free_bootmem_page(struct page *page)
84	84	{
85		− kmemleak_free_part_phys(PFN_PHYS(page_to_pfn(page)), PAGE_SIZE);
86	85	free_reserved_page(page);
87	86	}
88	87	#endif

include/linux/compaction.h+5−5

@@ -2,6 +2,8 @@ · 1 unmodified line
2	2	#ifndef _LINUX_COMPACTION_H
3	3	#define _LINUX_COMPACTION_H
4	4
	5	+#include <linux/swap.h>
	6	+
5	7	/*
6	8	* Determines how hard direct compaction should try to succeed.
7	9	* Lower value means higher priority, analogically to reclaim priority.
@@ -73,11 +75,9 @@ static inline unsigned long compact_gap(unsigned int order) · 65 unmodified lines
73	75	* effectively limited by COMPACT_CLUSTER_MAX, as that's the maximum
74	76	* that the migrate scanner can have isolated on migrate list, and free
75	77	* scanner is only invoked when the number of isolated free pages is
76		− * lower than that. But it's not worth to complicate the formula here
77		− * as a bigger gap for higher orders than strictly necessary can also
78		− * improve chances of compaction success.
	78	+ * lower than that.
79	79	*/
80		− return 2UL << order;
	80	+ return min(2UL << order, COMPACT_CLUSTER_MAX);
81	81	}
82	82
83	83	static inline int current_is_kcompactd(void)
@@ -101,7 +101,7 @@ extern void compaction_defer_reset(struct zone *zone, int order, · 17 unmodified lines
101	101	bool alloc_success);
102	102
103	103	bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
104		− int alloc_flags);
	104	+ int alloc_flags, gfp_t gfp_mask);
105	105
106	106	extern void __meminit kcompactd_run(int nid);
107	107	extern void __meminit kcompactd_stop(int nid);

include/linux/damon.h+111−25

@@ -8,23 +8,20 @@ · 7 unmodified lines
8	8	#ifndef _DAMON_H_
9	9	#define _DAMON_H_
10	10
	11	+#include <linux/math64.h>
11	12	#include <linux/memcontrol.h>
12	13	#include <linux/mutex.h>
	14	+#include <linux/prandom.h>
13	15	#include <linux/time64.h>
14	16	#include <linux/types.h>
15		−#include <linux/random.h>
16	17
17	18	/* Minimal region size. Every damon_region is aligned by this. */
18	19	#define DAMON_MIN_REGION_SZ PAGE_SIZE
	20	+/* Maximum number of monitoring probes. */
	21	+#define DAMON_MAX_PROBES (4)
19	22	/* Max priority score for DAMON-based operation schemes */
20	23	#define DAMOS_MAX_SCORE (99)
21	24
22		−/* Get a random number in [l, r) */
23		−static inline unsigned long damon_rand(unsigned long l, unsigned long r)
24		−{
25		− return l + get_random_u32_below(r - l);
26		−}
27		−
28	25	/**
29	26	* struct damon_addr_range - Represents an address region of [@start, @end).
30	27	* @start: Start address of the region (inclusive).
@@ -52,6 +49,7 @@ struct damon_size_range { · 21 unmodified lines
52	49	* @nr_accesses: Access frequency of this region.
53	50	* @nr_accesses_bp: @nr_accesses in basis point (0.01%) that updated for
54	51	* each sampling interval.
	52	+ * @probe_hits: Number of probe-positive region samples.
55	53	* @list: List head for siblings.
56	54	* @age: Age of this region.
57	55	*
@@ -80,6 +78,7 @@ struct damon_region { · 22 unmodified lines
80	78	unsigned long sampling_addr;
81	79	unsigned int nr_accesses;
82	80	unsigned int nr_accesses_bp;
	81	+ unsigned char probe_hits[DAMON_MAX_PROBES];
83	82	struct list_head list;
84	83
85	84	unsigned int age;
@@ -121,6 +120,7 @@ struct damon_target { · 35 unmodified lines
121	120	* @DAMOS_PAGEOUT: Reclaim the region.
122	121	* @DAMOS_HUGEPAGE: Call ``madvise()`` for the region with MADV_HUGEPAGE.
123	122	* @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPAGE.
	123	+ * @DAMOS_COLLAPSE: Call ``madvise()`` for the region with MADV_COLLAPSE.
124	124	* @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists.
125	125	* @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists.
126	126	* @DAMOS_MIGRATE_HOT: Migrate the regions prioritizing warmer regions.
@@ -140,6 +140,7 @@ enum damos_action { · 13 unmodified lines
140	140	DAMOS_PAGEOUT,
141	141	DAMOS_HUGEPAGE,
142	142	DAMOS_NOHUGEPAGE,
	143	+ DAMOS_COLLAPSE,
143	144	DAMOS_LRU_PRIO,
144	145	DAMOS_LRU_DEPRIO,
145	146	DAMOS_MIGRATE_HOT,
@@ -159,6 +160,8 @@ enum damos_action { · 13 unmodified lines
159	160	* @DAMOS_QUOTA_NODE_MEMCG_FREE_BP: MemFree ratio of a node for a cgroup.
160	161	* @DAMOS_QUOTA_ACTIVE_MEM_BP: Active to total LRU memory ratio.
161	162	* @DAMOS_QUOTA_INACTIVE_MEM_BP: Inactive to total LRU memory ratio.
	163	+ * @DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: Scheme-eligible memory ratio of a
	164	+ * node in basis points (0-10000).
162	165	* @NR_DAMOS_QUOTA_GOAL_METRICS: Number of DAMOS quota goal metrics.
163	166	*
164	167	* Metrics equal to larger than @NR_DAMOS_QUOTA_GOAL_METRICS are unsupported.
@@ -172,6 +175,7 @@ enum damos_quota_goal_metric { · 7 unmodified lines
172	175	DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
173	176	DAMOS_QUOTA_ACTIVE_MEM_BP,
174	177	DAMOS_QUOTA_INACTIVE_MEM_BP,
	178	+ DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
175	179	NR_DAMOS_QUOTA_GOAL_METRICS,
176	180	};
177	181
@@ -233,6 +237,8 @@ enum damos_quota_goal_tuner { · 55 unmodified lines
233	237	* @goals: Head of quota tuning goals (&damos_quota_goal) list.
234	238	* @goal_tuner: Goal-based @esz tuning algorithm to use.
235	239	* @esz: Effective size quota in bytes.
	240	+ * @fail_charge_num: Failed regions charge rate numerator.
	241	+ * @fail_charge_denom: Failed regions charge rate denominator.
236	242	*
237	243	* @weight_sz: Weight of the region's size for prioritization.
238	244	* @weight_nr_accesses: Weight of the region's nr_accesses for prioritization.
@@ -262,6 +268,10 @@ enum damos_quota_goal_tuner { · 23 unmodified lines
262	268	*
263	269	* The resulting effective size quota in bytes is set to @esz.
264	270	*
	271	+ * For DAMOS action applying failed amount of regions, charging those same to
	272	+ * those that the action has successfully applied may be unfair. For the
	273	+ * reason, 'the size * @fail_charge_num / @fail_charge_denom' is charged.
	274	+ *
265	275	* For selecting regions within the quota, DAMON prioritizes current scheme's
266	276	* target memory regions using the &struct damon_operations->get_scheme_score.
267	277	* You could customize the prioritization logic by setting &weight_sz,
@@ -276,6 +286,9 @@ struct damos_quota { · 8 unmodified lines
276	286	enum damos_quota_goal_tuner goal_tuner;
277	287	unsigned long esz;
278	288
	289	+ unsigned int fail_charge_num;
	290	+ unsigned int fail_charge_denom;
	291	+
279	292	unsigned int weight_sz;
280	293	unsigned int weight_nr_accesses;
281	294	unsigned int weight_age;
@@ -617,6 +630,7 @@ enum damon_ops_id { · 335 unmodified lines
617	630	* @update: Update operations-related data structures.
618	631	* @prepare_access_checks: Prepare next access check of target regions.
619	632	* @check_accesses: Check the accesses to target regions.
	633	+ * @apply_probes: Apply probes for each region.
620	634	* @get_scheme_score: Get the score of a region for a scheme.
621	635	* @apply_scheme: Apply a DAMON-based operation scheme.
622	636	* @target_valid: Determine if the target is valid.
@@ -643,6 +657,8 @@ enum damon_ops_id { · 20 unmodified lines
643	657	* last preparation and update the number of observed accesses of each region.
644	658	* It should also return max number of observed accesses that made as a result
645	659	* of its update. The value will be used for regions adjustment threshold.
	660	+ * @apply_probes should apply the data attribute probes to each region and
	661	+ * accordingly update the probe hits counter of the region.
646	662	* @get_scheme_score should return the priority score of a region for a scheme
647	663	* as an integer in [0, &DAMOS_MAX_SCORE].
648	664	* @apply_scheme is called from @kdamond when a region for user provided
@@ -660,6 +676,7 @@ struct damon_operations { · 11 unmodified lines
660	676	void (update)(struct damon_ctx context);
661	677	void (prepare_access_checks)(struct damon_ctx context);
662	678	unsigned int (check_accesses)(struct damon_ctx context);
	679	+ void (apply_probes)(struct damon_ctx context);
663	680	int (get_scheme_score)(struct damon_ctx context,
664	681	struct damon_region r, struct damos scheme);
665	682	unsigned long (apply_scheme)(struct damon_ctx context,
@@ -721,6 +738,47 @@ struct damon_intervals_goal { · 55 unmodified lines
721	738	unsigned long max_sample_us;
722	739	};
723	740
	741	+/**
	742	+ * enum damon_filter_type - Type of &struct damon_filter
	743	+ *
	744	+ * @DAMON_FILTER_TYPE_ANON: Anonymous pages.
	745	+ * @DAMON_FILTER_TYPE_MEMCG: Specific memcg's pages.
	746	+ */
	747	+enum damon_filter_type {
	748	+ DAMON_FILTER_TYPE_ANON,
	749	+ DAMON_FILTER_TYPE_MEMCG,
	750	+};
	751	+
	752	+/**
	753	+ * struct damon_filter - DAMON region filter for &struct damon_probe.
	754	+ *
	755	+ * @type: Type of the region.
	756	+ * @matching: Whether this filter is for the type-matching ones.
	757	+ * @allow: Whether the @type-@matching ones should pass this filter.
	758	+ * @memcg_id: Memcg id of the question if @type is DAMON_FILTER_MEMCG.
	759	+ * @list: Siblings list.
	760	+ */
	761	+struct damon_filter {
	762	+ enum damon_filter_type type;
	763	+ bool matching;
	764	+ bool allow;
	765	+ union {
	766	+ u64 memcg_id;
	767	+ };
	768	+ struct list_head list;
	769	+};
	770	+
	771	+/**
	772	+ * struct damon_probe - Data region attribute probe.
	773	+ *
	774	+ * @filters: Filters for assessing if a given region is for this probe.
	775	+ * @list: Siblings list.
	776	+ */
	777	+struct damon_probe {
	778	+ struct list_head filters;
	779	+ struct list_head list;
	780	+};
	781	+
724	782	/**
725	783	* struct damon_attrs - Monitoring attributes for accuracy/overhead control.
726	784	*
@@ -787,6 +845,7 @@ struct damon_attrs { · 60 unmodified lines
787	845	* @ops: Set of monitoring operations for given use cases.
788	846	* @addr_unit: Scale factor for core to ops address conversion.
789	847	* @min_region_sz: Minimum region size.
	848	+ * @pause: Pause kdamond main loop.
790	849	* @adaptive_targets: Head of monitoring targets (&damon_target) list.
791	850	* @schemes: Head of schemes (&damos) list.
792	851	*/
@@ -838,13 +897,34 @@ struct damon_ctx { · 45 unmodified lines
838	897
839	898	/* public: */
840	899	struct damon_operations ops;
	900	+ struct list_head probes;
841	901	unsigned long addr_unit;
842	902	unsigned long min_region_sz;
	903	+ bool pause;
843	904
844	905	struct list_head adaptive_targets;
845	906	struct list_head schemes;
	907	+
	908	+ /* Per-ctx PRNG state for damon_rand(); kdamond is the sole consumer. */
	909	+ struct rnd_state rnd_state;
846	910	};
847	911
	912	+/* Get a random number in [@l, @r) using @ctx's lockless PRNG. */
	913	+static inline unsigned long damon_rand(struct damon_ctx *ctx,
	914	+ unsigned long l, unsigned long r)
	915	+{
	916	+ unsigned long span = r - l;
	917	+ u64 rnd;
	918	+
	919	+ if (span <= U32_MAX) {
	920	+ rnd = prandom_u32_state(&ctx->rnd_state);
	921	+ return l + (unsigned long)((rnd * span) >> 32);
	922	+ }
	923	+ rnd = ((u64)prandom_u32_state(&ctx->rnd_state) << 32) \|
	924	+ prandom_u32_state(&ctx->rnd_state);
	925	+ return l + mul_u64_u64_shr(rnd, span, 64);
	926	+}
	927	+
848	928	static inline struct damon_region damon_next_region(struct damon_region r)
849	929	{
850	930	return container_of(r->list.next, struct damon_region, list);
@@ -870,15 +950,26 @@ static inline unsigned long damon_sz_region(struct damon_region *r) · 19 unmodified lines
870	950	return r->ar.end - r->ar.start;
871	951	}
872	952
	953	+#define damon_for_each_filter(f, p) \
	954	+ list_for_each_entry(f, &(p)->filters, list)
	955	+
	956	+#define damon_for_each_filter_safe(f, next, p) \
	957	+ list_for_each_entry_safe(f, next, &(p)->filters, list)
	958	+
	959	+#define damon_for_each_probe(p, ctx) \
	960	+ list_for_each_entry(p, &(ctx)->probes, list)
	961	+
	962	+#define damon_for_each_probe_safe(p, next, ctx) \
	963	+ list_for_each_entry_safe(p, next, &(ctx)->probes, list)
873	964
874	965	#define damon_for_each_region(r, t) \
875		− list_for_each_entry(r, &t->regions_list, list)
	966	+ list_for_each_entry(r, &(t)->regions_list, list)
876	967
877	968	#define damon_for_each_region_from(r, t) \
878		− list_for_each_entry_from(r, &t->regions_list, list)
	969	+ list_for_each_entry_from(r, &(t)->regions_list, list)
879	970
880	971	#define damon_for_each_region_safe(r, next, t) \
881		− list_for_each_entry_safe(r, next, &t->regions_list, list)
	972	+ list_for_each_entry_safe(r, next, &(t)->regions_list, list)
882	973
883	974	#define damon_for_each_target(t, ctx) \
884	975	list_for_each_entry(t, &(ctx)->adaptive_targets, list)
@@ -893,7 +984,7 @@ static inline unsigned long damon_sz_region(struct damon_region *r) · 8 unmodified lines
893	984	list_for_each_entry_safe(s, next, &(ctx)->schemes, list)
894	985
895	986	#define damos_for_each_quota_goal(goal, quota) \
896		− list_for_each_entry(goal, &quota->goals, list)
	987	+ list_for_each_entry(goal, &(quota)->goals, list)
897	988
898	989	#define damos_for_each_quota_goal_safe(goal, next, quota) \
899	990	list_for_each_entry_safe(goal, next, &(quota)->goals, list)
@@ -912,21 +1003,16 @@ static inline unsigned long damon_sz_region(struct damon_region *r) · 12 unmodified lines
912	1003
913	1004	#ifdef CONFIG_DAMON
914	1005
915		−struct damon_region *damon_new_region(unsigned long start, unsigned long end);
	1006	+struct damon_filter *damon_new_filter(enum damon_filter_type type,
	1007	+ bool matching, bool allow);
	1008	+void damon_add_filter(struct damon_probe probe, struct damon_filter f);
	1009	+void damon_destroy_filter(struct damon_filter *f);
916	1010
917		−/*
918		− * Add a region between two other regions
919		− */
920		−static inline void damon_insert_region(struct damon_region *r,
921		− struct damon_region prev, struct damon_region next,
922		− struct damon_target *t)
923		−{
924		− __list_add(&r->list, &prev->list, &next->list);
925		− t->nr_regions++;
926		−}
	1011	+struct damon_probe *damon_new_probe(void);
	1012	+void damon_add_probe(struct damon_ctx ctx, struct damon_probe probe);
	1013	+
	1014	+struct damon_region *damon_new_region(unsigned long start, unsigned long end);
927	1015
928		−void damon_add_region(struct damon_region r, struct damon_target t);
929		−void damon_destroy_region(struct damon_region r, struct damon_target t);
930	1016	int damon_set_regions(struct damon_target t, struct damon_addr_range ranges,
931	1017	unsigned int nr_ranges, unsigned long min_region_sz);
932	1018	void damon_update_region_access_rate(struct damon_region *r, bool accessed,
@@ -994,7 +1080,7 @@ int damon_kdamond_pid(struct damon_ctx *ctx); · 61 unmodified lines
994	1080	int damon_call(struct damon_ctx ctx, struct damon_call_control control);
995	1081	int damos_walk(struct damon_ctx ctx, struct damos_walk_control control);
996	1082
997		−int damon_set_region_biggest_system_ram_default(struct damon_target *t,
	1083	+int damon_set_region_system_rams_default(struct damon_target *t,
998	1084	unsigned long start, unsigned long end,
999	1085	unsigned long addr_unit,
1000	1086	unsigned long min_region_sz);

include/linux/gfp.h+4−0

@@ -239,6 +239,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid, · 238 unmodified lines
239	239	struct page **page_array);
240	240	#define __alloc_pages_bulk(...) alloc_hooks(alloc_pages_bulk_noprof(__VA_ARGS__))
241	241
	242	+void free_pages_bulk(struct page **page_array, unsigned long nr_pages);
	243	+
242	244	unsigned long alloc_pages_bulk_mempolicy_noprof(gfp_t gfp,
243	245	unsigned long nr_pages,
244	246	struct page **page_array);
@@ -467,6 +469,8 @@ void free_contig_frozen_range(unsigned long pfn, unsigned long nr_pages); · 222 unmodified lines
467	469	void free_contig_range(unsigned long pfn, unsigned long nr_pages);
468	470	#endif
469	471
	472	+void __free_contig_range(unsigned long pfn, unsigned long nr_pages);
	473	+
470	474	DEFINE_FREE(free_page, void *, free_page((unsigned long)_T))
471	475
472	476	#endif /* __LINUX_GFP_H */

include/linux/gfp_types.h+3−3

@@ -281,9 +281,9 @@ enum { · 280 unmodified lines
281	281	*
282	282	* %__GFP_SKIP_KASAN makes KASAN skip unpoisoning on page allocation.
283	283	* Used for userspace and vmalloc pages; the latter are unpoisoned by
284		− * kasan_unpoison_vmalloc instead. For userspace pages, results in
285		− * poisoning being skipped as well, see should_skip_kasan_poison for
286		− * details. Only effective in HW_TAGS mode.
	284	+ * kasan_unpoison_vmalloc instead. If passed to vmalloc, kasan_unpoison_vmalloc
	285	+ * is skipped too. For userspace pages, results in poisoning being skipped as
	286	+ * well, see should_skip_kasan_poison for details. Only effective in HW_TAGS mode.
287	287	*/
288	288	#define __GFP_NOWARN ((__force gfp_t)___GFP_NOWARN)
289	289	#define __GFP_COMP ((__force gfp_t)___GFP_COMP)

include/linux/highmem-internal.h+1−1

@@ -262,7 +262,7 @@ static inline bool is_kmap_addr(const void *x) · 261 unmodified lines
262	262	* @__addr: Virtual address to be unmapped
263	263	*
264	264	* Unmaps an address previously mapped by kmap_atomic() and re-enables
265		− * pagefaults. Depending on PREEMP_RT configuration, re-enables also
	265	+ * pagefaults. Depending on PREEMPT_RT configuration, re-enables also
266	266	* migration and preemption. Users should not count on these side effects.
267	267	*
268	268	* Mappings should be unmapped in the reverse order that they were mapped.

include/linux/huge_mm.h+42−5

@@ -237,6 +237,31 @@ static inline bool thp_vma_suitable_order(struct vm_area_struct *vma, · 236 unmodified lines
237	237	return true;
238	238	}
239	239
	240	+/*
	241	+ * Make sure huge_gfp is always more limited than limit_gfp.
	242	+ * Some shmem users want THP allocation to be done less aggressively
	243	+ * and only in certain zone.
	244	+ */
	245	+static inline gfp_t thp_shmem_limit_gfp_mask(gfp_t huge_gfp, gfp_t limit_gfp)
	246	+{
	247	+ gfp_t allowflags = __GFP_IO \| __GFP_FS \| __GFP_RECLAIM;
	248	+ gfp_t denyflags = __GFP_NOWARN \| __GFP_NORETRY;
	249	+ gfp_t zoneflags = limit_gfp & GFP_ZONEMASK;
	250	+ gfp_t result = huge_gfp & ~(allowflags \| GFP_ZONEMASK);
	251	+
	252	+ /* Allow allocations only from the originally specified zones. */
	253	+ result \|= zoneflags;
	254	+
	255	+ /*
	256	+ * Minimize the result gfp by taking the union with the deny flags,
	257	+ * and the intersection of the allow flags.
	258	+ */
	259	+ result \|= (limit_gfp & denyflags);
	260	+ result \|= (huge_gfp & limit_gfp) & allowflags;
	261	+
	262	+ return result;
	263	+}
	264	+
240	265	/*
241	266	* Filter the bitfield of input orders to the ones suitable for use in the vma.
242	267	* See thp_vma_suitable_order().
@@ -414,10 +439,10 @@ static inline int split_huge_page(struct page *page) · 171 unmodified lines
414	439	{
415	440	return split_huge_page_to_list_to_order(page, NULL, 0);
416	441	}
	442	+
	443	+int folio_memcg_alloc_deferred(struct folio *folio);
	444	+
417	445	void deferred_split_folio(struct folio *folio, bool partially_mapped);
418		−#ifdef CONFIG_MEMCG
419		−void reparent_deferred_split_queue(struct mem_cgroup *memcg);
420		−#endif
421	446
422	447	void __split_huge_pmd(struct vm_area_struct vma, pmd_t pmd,
423	448	unsigned long address, bool freeze);
@@ -581,6 +606,11 @@ static inline bool thp_vma_suitable_order(struct vm_area_struct *vma, · 157 unmodified lines
581	606	return false;
582	607	}
583	608
	609	+static inline gfp_t thp_shmem_limit_gfp_mask(gfp_t huge_gfp, gfp_t limit_gfp)
	610	+{
	611	+ return huge_gfp;
	612	+}
	613	+
584	614	static inline unsigned long thp_vma_suitable_orders(struct vm_area_struct *vma,
585	615	unsigned long addr, unsigned long orders)
586	616	{
@@ -649,8 +679,15 @@ static inline int try_folio_split_to_order(struct folio *folio, · 62 unmodified lines
649	679	return -EINVAL;
650	680	}
651	681
652		−static inline void deferred_split_folio(struct folio *folio, bool partially_mapped) {}
653		−static inline void reparent_deferred_split_queue(struct mem_cgroup *memcg) {}
	682	+static inline int folio_memcg_alloc_deferred(struct folio *folio)
	683	+{
	684	+ return 0;
	685	+}
	686	+
	687	+static inline void deferred_split_folio(struct folio *folio, bool partially_mapped)
	688	+{
	689	+}
	690	+
654	691	#define split_huge_pmd(__vma, __pmd, __address) \
655	692	do { } while (0)
656	693

include/linux/list_lru.h+70−0

@@ -81,8 +81,75 @@ static inline int list_lru_init_memcg_key(struct list_lru *lru, struct shrinker · 80 unmodified lines
81	81
82	82	int memcg_list_lru_alloc(struct mem_cgroup memcg, struct list_lru lru,
83	83	gfp_t gfp);
	84	+
	85	+#ifdef CONFIG_MEMCG
	86	+/**
	87	+ * folio_memcg_list_lru_alloc - allocate list_lru heads for shrinkable folio
	88	+ * @folio: the newly allocated & charged folio
	89	+ * @lru: the list_lru this might be queued on
	90	+ * @gfp: gfp mask
	91	+ *
	92	+ * Allocate list_lru heads (per-memcg, per-node) needed to queue this
	93	+ * particular folio down the line.
	94	+ *
	95	+ * This does memcg_list_lru_alloc(), but on the memcg that @folio is
	96	+ * associated with. Handles folio_memcg() access rules in the fast
	97	+ * path (list_lru heads allocated) and the allocation slowpath.
	98	+ *
	99	+ * Returns 0 on success, a negative error value otherwise.
	100	+ */
	101	+int folio_memcg_list_lru_alloc(struct folio folio, struct list_lru lru,
	102	+ gfp_t gfp);
	103	+#else
	104	+static inline int folio_memcg_list_lru_alloc(struct folio *folio,
	105	+ struct list_lru *lru, gfp_t gfp)
	106	+{
	107	+ return 0;
	108	+}
	109	+#endif
	110	+
84	111	void memcg_reparent_list_lrus(struct mem_cgroup memcg, struct mem_cgroup parent);
85	112
	113	+/**
	114	+ * list_lru_lock: lock the sublist for the given node and memcg
	115	+ * @lru: the lru pointer
	116	+ * @nid: the node id of the sublist to lock.
	117	+ * @memcg: pointer to the cgroup of the sublist to lock. On return,
	118	+ * updated to the cgroup whose sublist was actually locked,
	119	+ * which may be an ancestor if the original memcg was dying.
	120	+ *
	121	+ * Returns the locked list_lru_one sublist. The caller must call
	122	+ * list_lru_unlock() when done.
	123	+ *
	124	+ * You must ensure that the memcg is not freed during this call (e.g., with
	125	+ * rcu or by taking a css refcnt).
	126	+ *
	127	+ * Return: the locked list_lru_one, or NULL on failure
	128	+ */
	129	+struct list_lru_one list_lru_lock(struct list_lru lru, int nid,
	130	+ struct mem_cgroup **memcg);
	131	+
	132	+/**
	133	+ * list_lru_unlock: unlock a sublist locked by list_lru_lock()
	134	+ * @l: the list_lru_one to unlock
	135	+ */
	136	+void list_lru_unlock(struct list_lru_one *l);
	137	+
	138	+struct list_lru_one list_lru_lock_irq(struct list_lru lru, int nid,
	139	+ struct mem_cgroup **memcg);
	140	+void list_lru_unlock_irq(struct list_lru_one *l);
	141	+
	142	+struct list_lru_one list_lru_lock_irqsave(struct list_lru lru, int nid,
	143	+ struct mem_cgroup *memcg, unsigned long irq_flags);
	144	+void list_lru_unlock_irqrestore(struct list_lru_one *l,
	145	+ unsigned long *irq_flags);
	146	+
	147	+/* Caller-locked variants, see list_lru_add() etc for documentation */
	148	+bool __list_lru_add(struct list_lru lru, struct list_lru_one l,
	149	+ struct list_head item, int nid, struct mem_cgroup memcg);
	150	+bool __list_lru_del(struct list_lru lru, struct list_lru_one l,
	151	+ struct list_head *item, int nid);
	152	+
86	153	/**
87	154	* list_lru_add: add an element to the lru list's tail
88	155	* @lru: the lru pointer
@@ -115,6 +182,9 @@ void memcg_reparent_list_lrus(struct mem_cgroup memcg, struct mem_cgroup paren · 26 unmodified lines
115	182	bool list_lru_add(struct list_lru lru, struct list_head item, int nid,
116	183	struct mem_cgroup *memcg);
117	184
	185	+bool list_lru_add_irq(struct list_lru lru, struct list_head item, int nid,
	186	+ struct mem_cgroup *memcg);
	187	+
118	188	/**
119	189	* list_lru_add_obj: add an element to the lru list's tail
120	190	* @lru: the lru pointer

include/linux/memcontrol.h+17−14

@@ -29,6 +29,7 @@ struct obj_cgroup; · 28 unmodified lines
29	29	struct page;
30	30	struct mm_struct;
31	31	struct kmem_cache;
	32	+struct swap_cluster_info;
32	33
33	34	/* Cgroup-specific page state, on top of universal node page state */
34	35	enum memcg_stat_item {
@@ -277,10 +278,6 @@ struct mem_cgroup { · 242 unmodified lines
277	278	struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT];
278	279	#endif
279	280
280		−#ifdef CONFIG_TRANSPARENT_HUGEPAGE
281		− struct deferred_split deferred_split_queue;
282		−#endif
283		−
284	281	#ifdef CONFIG_LRU_GEN_WALKS_MMU
285	282	/* per-memcg mm_struct list */
286	283	struct lru_gen_mm_list mm_list;
@@ -646,8 +643,8 @@ static inline int mem_cgroup_charge(struct folio folio, struct mm_struct mm, · 359 unmodified lines
646	643
647	644	int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp);
648	645
649		−int mem_cgroup_swapin_charge_folio(struct folio folio, struct mm_struct mm,
650		− gfp_t gfp, swp_entry_t entry);
	646	+int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id,
	647	+ struct mm_struct *mm, gfp_t gfp);
651	648
652	649	void __mem_cgroup_uncharge(struct folio *folio);
653	650
@@ -1137,7 +1134,7 @@ static inline int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp) · 483 unmodified lines
1137	1134	}
1138	1135
1139	1136	static inline int mem_cgroup_swapin_charge_folio(struct folio *folio,
1140		− struct mm_struct *mm, gfp_t gfp, swp_entry_t entry)
	1137	+ unsigned short id, struct mm_struct *mm, gfp_t gfp)
1141	1138	{
1142	1139	return 0;
1143	1140	}
@@ -1899,9 +1896,6 @@ static inline void mem_cgroup_exit_user_fault(void) · 755 unmodified lines
1899	1896	current->in_user_fault = 0;
1900	1897	}
1901	1898
1902		−void memcg1_swapout(struct folio *folio, swp_entry_t entry);
1903		−void memcg1_swapin(swp_entry_t entry, unsigned int nr_pages);
1904		−
1905	1899	#else /* CONFIG_MEMCG_V1 */
1906	1900	static inline
1907	1901	unsigned long memcg1_soft_limit_reclaim(pg_data_t *pgdat, int order,
@@ -1929,14 +1923,23 @@ static inline void mem_cgroup_exit_user_fault(void) · 21 unmodified lines
1929	1923	{
1930	1924	}
1931	1925
1932		−static inline void memcg1_swapout(struct folio *folio, swp_entry_t entry)
	1926	+#endif /* CONFIG_MEMCG_V1 */
	1927	+
	1928	+#if defined(CONFIG_MEMCG_V1) && defined(CONFIG_SWAP)
	1929	+
	1930	+void __memcg1_swapout(struct folio folio, struct swap_cluster_info ci);
	1931	+void memcg1_swapin(struct folio *folio);
	1932	+
	1933	+#else
	1934	+
	1935	+static inline void __memcg1_swapout(struct folio *folio,
	1936	+ struct swap_cluster_info *ci)
1933	1937	{
1934	1938	}
1935	1939
1936		−static inline void memcg1_swapin(swp_entry_t entry, unsigned int nr_pages)
	1940	+static inline void memcg1_swapin(struct folio *folio)
1937	1941	{
1938	1942	}
1939		−
1940		−#endif /* CONFIG_MEMCG_V1 */
	1943	+#endif
1941	1944
1942	1945	#endif /* _LINUX_MEMCONTROL_H */

include/linux/memory.h+5−2

@@ -158,7 +158,11 @@ int create_memory_block_devices(unsigned long start, unsigned long size, · 157 unmodified lines
158	158	void remove_memory_block_devices(unsigned long start, unsigned long size);
159	159	extern void memory_dev_init(void);
160	160	extern int memory_notify(enum memory_block_state state, void *v);
161		−extern struct memory_block *find_memory_block(unsigned long section_nr);
	161	+struct memory_block *memory_block_get(unsigned long block_id);
	162	+static inline void memory_block_put(struct memory_block *mem)
	163	+{
	164	+ put_device(&mem->dev);
	165	+}
162	166	typedef int (walk_memory_blocks_func_t)(struct memory_block , void *);
163	167	extern int walk_memory_blocks(unsigned long start, unsigned long size,
164	168	void *arg, walk_memory_blocks_func_t func);
@@ -171,7 +175,6 @@ struct memory_group *memory_group_find_by_id(int mgid); · 6 unmodified lines
171	175	typedef int (walk_memory_groups_func_t)(struct memory_group , void *);
172	176	int walk_dynamic_memory_groups(int nid, walk_memory_groups_func_t func,
173	177	struct memory_group excluded, void arg);
174		−struct memory_block *find_memory_block_by_id(unsigned long block_id);
175	178	#define hotplug_memory_notifier(fn, pri) ({ \
176	179	static __meminitdata struct notifier_block fn##_mem_nb =\
177	180	{ .notifier_call = fn, .priority = pri };\

include/linux/memory_hotplug.h+5−3

@@ -135,9 +135,10 @@ static inline bool movable_node_is_enabled(void) · 134 unmodified lines
135	135	return movable_node_enabled;
136	136	}
137	137
138		−extern void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap);
	138	+extern void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap,
	139	+ struct dev_pagemap *pgmap);
139	140	extern void __remove_pages(unsigned long start_pfn, unsigned long nr_pages,
140		− struct vmem_altmap *altmap);
	141	+ struct vmem_altmap altmap, struct dev_pagemap pgmap);
141	142
142	143	/* reasonably generic interface to expand the physical pages */
143	144	extern int __add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages,
@@ -307,7 +308,8 @@ extern int sparse_add_section(int nid, unsigned long pfn, · 163 unmodified lines
307	308	unsigned long nr_pages, struct vmem_altmap *altmap,
308	309	struct dev_pagemap *pgmap);
309	310	extern void sparse_remove_section(unsigned long pfn, unsigned long nr_pages,
310		− struct vmem_altmap *altmap);
	311	+ struct vmem_altmap *altmap,
	312	+ struct dev_pagemap *pgmap);
311	313	extern struct zone *zone_for_pfn_range(enum mmop online_type,
312	314	int nid, struct memory_group *group, unsigned long start_pfn,
313	315	unsigned long nr_pages);

include/linux/mm.h+44−18

@@ -496,6 +496,21 @@ enum { · 495 unmodified lines
496	496	#else
497	497	#define VM_UFFD_MINOR VM_NONE
498	498	#endif
	499	+
	500	+/*
	501	+ * vma_flags_t masks for the userfaultfd VMA flags. VMA_UFFD_MINOR is gated on
	502	+ * the same config as VM_UFFD_MINOR -- which implies 64BIT, where the bit fits
	503	+ * -- so an out-of-range bit is never fed to mk_vma_flags() on a build whose
	504	+ * bitmap cannot hold it.
	505	+ */
	506	+#define VMA_UFFD_MISSING mk_vma_flags(VMA_UFFD_MISSING_BIT)
	507	+#define VMA_UFFD_WP mk_vma_flags(VMA_UFFD_WP_BIT)
	508	+#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_MINOR
	509	+#define VMA_UFFD_MINOR mk_vma_flags(VMA_UFFD_MINOR_BIT)
	510	+#else
	511	+#define VMA_UFFD_MINOR EMPTY_VMA_FLAGS
	512	+#endif
	513	+
499	514	#ifdef CONFIG_64BIT
500	515	#define VM_ALLOW_ANY_UNCACHED INIT_VM_FLAG(ALLOW_ANY_UNCACHED)
501	516	#define VM_SEALED INIT_VM_FLAG(SEALED)
@@ -1238,6 +1253,30 @@ static __always_inline void vma_flags_set_mask(vma_flags_t *flags, · 736 unmodified lines
1238	1253	#define vma_flags_set(flags, ...) \
1239	1254	vma_flags_set_mask(flags, mk_vma_flags(__VA_ARGS__))
1240	1255
	1256	+static __always_inline vma_flags_t __mk_vma_flags_from_masks(size_t count,
	1257	+ const vma_flags_t *masks)
	1258	+{
	1259	+ vma_flags_t flags = EMPTY_VMA_FLAGS;
	1260	+ size_t i;
	1261	+
	1262	+ for (i = 0; i < count; i++)
	1263	+ vma_flags_set_mask(&flags, masks[i]);
	1264	+ return flags;
	1265	+}
	1266	+
	1267	+/*
	1268	+ * Combine pre-computed vma_flags_t masks into one value, e.g.:
	1269	+ *
	1270	+ * vma_flags_t flags = mk_vma_flags_from_masks(VMA_UFFD_WP, VMA_UFFD_MINOR);
	1271	+ *
	1272	+ * Unlike mk_vma_flags(), which takes bit numbers, this takes whole masks --
	1273	+ * each of which may be EMPTY_VMA_FLAGS when its feature is unavailable -- so a
	1274	+ * bit that does not exist on the current build is never materialised.
	1275	+ */
	1276	+#define mk_vma_flags_from_masks(...) \
	1277	+ __mk_vma_flags_from_masks(COUNT_ARGS(__VA_ARGS__), \
	1278	+ (const vma_flags_t []){__VA_ARGS__})
	1279	+
1241	1280	/* Clear all of the to-clear flags in flags, non-atomically. */
1242	1281	static __always_inline void vma_flags_clear_mask(vma_flags_t *flags,
1243	1282	vma_flags_t to_clear)
@@ -1489,6 +1528,11 @@ static inline void vma_set_anonymous(struct vm_area_struct *vma) · 245 unmodified lines
1489	1528	vma->vm_ops = NULL;
1490	1529	}
1491	1530
	1531	+static inline void vma_desc_set_anonymous(struct vm_area_desc *desc)
	1532	+{
	1533	+ desc->vm_ops = NULL;
	1534	+}
	1535	+
1492	1536	static inline bool vma_is_anonymous(struct vm_area_struct *vma)
1493	1537	{
1494	1538	return !vma->vm_ops;
@@ -1888,16 +1932,6 @@ static inline bool folio_mapped(const struct folio *folio) · 393 unmodified lines
1888	1932	return folio_mapcount(folio) >= 1;
1889	1933	}
1890	1934
1891		−/*
1892		− * Return true if this page is mapped into pagetables.
1893		− * For compound page it returns true if any sub-page of compound page is mapped,
1894		− * even if this particular sub-page is not itself mapped by any PTE or PMD.
1895		− */
1896		−static inline bool page_mapped(const struct page *page)
1897		−{
1898		− return folio_mapped(page_folio(page));
1899		−}
1900		−
1901	1935	static inline struct page virt_to_head_page(const void x)
1902	1936	{
1903	1937	struct page *page = virt_to_page(x);
@@ -4855,18 +4889,10 @@ static inline void print_vma_addr(char *prefix, unsigned long rip) · 2951 unmodified lines
4855	4889	}
4856	4890	#endif
4857	4891
4858		−void *sparse_buffer_alloc(unsigned long size);
4859	4892	unsigned long section_map_size(void);
4860	4893	struct page * __populate_section_memmap(unsigned long pfn,
4861	4894	unsigned long nr_pages, int nid, struct vmem_altmap *altmap,
4862	4895	struct dev_pagemap *pgmap);
4863		−pgd_t *vmemmap_pgd_populate(unsigned long addr, int node);
4864		−p4d_t vmemmap_p4d_populate(pgd_t pgd, unsigned long addr, int node);
4865		−pud_t vmemmap_pud_populate(p4d_t p4d, unsigned long addr, int node);
4866		−pmd_t vmemmap_pmd_populate(pud_t pud, unsigned long addr, int node);
4867		−pte_t vmemmap_pte_populate(pmd_t pmd, unsigned long addr, int node,
4868		− struct vmem_altmap *altmap, unsigned long ptpfn,
4869		− unsigned long flags);
4870	4896	void *vmemmap_alloc_block(unsigned long size, int node);
4871	4897	struct vmem_altmap;
4872	4898	void *vmemmap_alloc_block_buf(unsigned long size, int node,

include/linux/mm_inline.h+1−1

@@ -247,7 +247,7 @@ static inline unsigned long lru_gen_folio_seq(const struct lruvec *lruvec, · 246 unmodified lines
247	247	(folio_test_dirty(folio) \|\| folio_test_writeback(folio))))
248	248	gen = MIN_NR_GENS;
249	249	else
250		− gen = MAX_NR_GENS - folio_test_workingset(folio);
	250	+ gen = MAX_NR_GENS - (folio_test_workingset(folio) \|\| folio_test_referenced(folio));
251	251
252	252	return max(READ_ONCE(lrugen->max_seq) - gen + 1, READ_ONCE(lrugen->min_seq[type]));
253	253	}

include/linux/mm_types.h+3−16

@@ -845,23 +845,10 @@ struct mmap_action { · 844 unmodified lines
845	845	enum mmap_action_type type;
846	846
847	847	/*
848		− * If specified, this hook is invoked after the selected action has been
849		− * successfully completed. Note that the VMA write lock still held.
850		− *
851		− * The absolute minimum ought to be done here.
852		− *
853		− * Returns 0 on success, or an error code.
854		− */
855		− int (success_hook)(const struct vm_area_struct vma);
856		−
857		− /*
858		− * If specified, this hook is invoked when an error occurred when
859		− * attempting the selected action.
860		− *
861		− * The hook can return an error code in order to filter the error, but
862		− * it is not valid to clear the error here.
	848	+ * If non-zero, replace errors that arise from mmap actions with this
	849	+ * value instead. Only valid error codes may be specified.
863	850	*/
864		− int (*error_hook)(int err);
	851	+ int error_override;
865	852
866	853	/*
867	854	* This should be set in rare instances where the operation required

include/linux/mmu_notifier.h+2−2

@@ -134,8 +134,8 @@ struct mmu_notifier_ops { · 133 unmodified lines
134	134	* Invalidation of multiple concurrent ranges may be
135	135	* optionally permitted by the driver. Either way the
136	136	* establishment of sptes is forbidden in the range passed to
137		− * invalidate_range_begin/end for the whole duration of the
138		− * invalidate_range_begin/end critical section.
	137	+ * invalidate_range_start/end for the whole duration of the
	138	+ * invalidate_range_start/end critical section.
139	139	*
140	140	* invalidate_range_start() is called when all pages in the
141	141	* range are still mapped and have at least a refcount of one.

include/linux/mmzone.h+6−17

@@ -177,9 +177,12 @@ static inline bool migratetype_is_mergeable(int mt) · 176 unmodified lines
177	177	return mt < MIGRATE_PCPTYPES;
178	178	}
179	179
180		−#define for_each_migratetype_order(order, type) \
181		− for (order = 0; order < NR_PAGE_ORDERS; order++) \
182		− for (type = 0; type < MIGRATE_TYPES; type++)
	180	+#define for_each_free_list(list, zone, order) \
	181	+ for (order = 0; order < NR_PAGE_ORDERS; order++) \
	182	+ for (unsigned int __type = 0; \
	183	+ __type < MIGRATE_TYPES && \
	184	+ (list = &(zone)->free_area[order].free_list[__type], 1); \
	185	+ __type++)
183	186
184	187	extern int page_group_by_mobility_disabled;
185	188
@@ -211,7 +214,6 @@ enum numa_stat_item { · 25 unmodified lines
211	214	#endif
212	215
213	216	enum zone_stat_item {
214		− /* First 128 byte cacheline (assuming 64 bit words) */
215	217	NR_FREE_PAGES,
216	218	NR_FREE_PAGES_BLOCKS,
217	219	NR_ZONE_LRU_BASE, /* Used only for compaction and reclaim retry */
@@ -222,7 +224,6 @@ enum zone_stat_item { · 4 unmodified lines
222	224	NR_ZONE_UNEVICTABLE,
223	225	NR_ZONE_WRITE_PENDING, /* Count of dirty, writeback and unstable pages */
224	226	NR_MLOCK, /* mlock()ed pages found and moved off LRU */
225		− /* Second 128 byte cacheline */
226	227	#if IS_ENABLED(CONFIG_ZSMALLOC)
227	228	NR_ZSPAGES, /* allocated in zsmalloc */
228	229	#endif
@@ -1428,14 +1429,6 @@ struct zonelist { · 1199 unmodified lines
1428	1429	*/
1429	1430	extern struct page *mem_map;
1430	1431
1431		−#ifdef CONFIG_TRANSPARENT_HUGEPAGE
1432		−struct deferred_split {
1433		− spinlock_t split_queue_lock;
1434		− struct list_head split_queue;
1435		− unsigned long split_queue_len;
1436		−};
1437		−#endif
1438		−
1439	1432	#ifdef CONFIG_MEMORY_FAILURE
1440	1433	/*
1441	1434	* Per NUMA node memory failure handling statistics.
@@ -1561,10 +1554,6 @@ typedef struct pglist_data { · 119 unmodified lines
1561	1554	unsigned long first_deferred_pfn;
1562	1555	#endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */
1563	1556
1564		−#ifdef CONFIG_TRANSPARENT_HUGEPAGE
1565		− struct deferred_split deferred_split_queue;
1566		−#endif
1567		−
1568	1557	#ifdef CONFIG_NUMA_BALANCING
1569	1558	/* start time in ms of current promote rate limit period */
1570	1559	unsigned int nbp_rl_start;

include/linux/nodemask.h+9−9

@@ -24,23 +24,23 @@ · 23 unmodified lines
24	24	* void nodes_setall(mask) set all bits
25	25	* void nodes_clear(mask) clear all bits
26	26	* int node_isset(node, mask) true iff bit 'node' set in mask
27		− * int node_test_and_set(node, mask) test and set bit 'node' in mask
	27	+ * bool node_test_and_set(node, mask) test and set bit 'node' in mask
28	28	*
29		− * void nodes_and(dst, src1, src2) dst = src1 & src2 [intersection]
	29	+ * bool nodes_and(dst, src1, src2) dst = src1 & src2 [intersection]
30	30	* void nodes_or(dst, src1, src2) dst = src1 \| src2 [union]
31	31	* void nodes_xor(dst, src1, src2) dst = src1 ^ src2
32		− * void nodes_andnot(dst, src1, src2) dst = src1 & ~src2
	32	+ * bool nodes_andnot(dst, src1, src2) dst = src1 & ~src2
33	33	* void nodes_complement(dst, src) dst = ~src
34	34	*
35		− * int nodes_equal(mask1, mask2) Does mask1 == mask2?
36		− * int nodes_intersects(mask1, mask2) Do mask1 and mask2 intersect?
37		− * int nodes_subset(mask1, mask2) Is mask1 a subset of mask2?
38		− * int nodes_empty(mask) Is mask empty (no bits sets)?
39		− * int nodes_full(mask) Is mask full (all bits sets)?
	35	+ * bool nodes_equal(mask1, mask2) Does mask1 == mask2?
	36	+ * bool nodes_intersects(mask1, mask2) Do mask1 and mask2 intersect?
	37	+ * bool nodes_subset(mask1, mask2) Is mask1 a subset of mask2?
	38	+ * bool nodes_empty(mask) Is mask empty (no bits sets)?
	39	+ * bool nodes_full(mask) Is mask full (all bits sets)?
40	40	* int nodes_weight(mask) Hamming weight - number of set bits
41	41	*
42	42	* unsigned int first_node(mask) Number lowest set bit, or MAX_NUMNODES
43		− * unsigend int next_node(node, mask) Next node past 'node', or MAX_NUMNODES
	43	+ * unsigned int next_node(node, mask) Next node past 'node', or MAX_NUMNODES
44	44	* unsigned int next_node_in(node, mask) Next node past 'node', or wrap to first,
45	45	* or MAX_NUMNODES
46	46	* unsigned int first_unset_node(mask) First node not set in mask, or

include/linux/page_ref.h+18−0

@@ -71,6 +71,12 @@ static inline int page_ref_count(const struct page *page) · 70 unmodified lines
71	71	* folio_ref_count - The reference count on this folio.
72	72	* @folio: The folio.
73	73	*
	74	+ * Folios contain a reference count. When that reference count reaches
	75	+ * zero, the folio is referred to as frozen. At this point, it will
	76	+ * usually be returned to the memory allocator, but some parts of the
	77	+ * kernel freeze folios in order to perform unusual operations on them
	78	+ * such as splitting or migration.
	79	+ *
74	80	* The refcount is usually incremented by calls to folio_get() and
75	81	* decremented by calls to folio_put(). Some typical users of the
76	82	* folio refcount:
@@ -82,6 +88,18 @@ static inline int page_ref_count(const struct page *page) · 5 unmodified lines
82	88	* - Pipes
83	89	* - Direct IO which references this page in the process address space
84	90	*
	91	+ * The reference count has three components: expected, temporary and
	92	+ * spurious. The expected reference count of a folio is that which
	93	+ * we would logically expect it to be from just reading the code.
	94	+ * Temporary refcounts are gained by threads which need a temporary
	95	+ * reference to make sure the folio isn't reallocated while they use it.
	96	+ * Spurious refcounts are gained by threads which, thanks to RCU walks
	97	+ * of the page tables or file cache, find a stale pointer to a folio.
	98	+ * These threads will drop the refcount after discoveering the pointer
	99	+ * is stale, but it can surprise other users to see the spurious refcount
	100	+ * on a freshly allocated folio (eg they may see a refcount of 2 instead
	101	+ * of 1).
	102	+ *
85	103	* Return: The number of references to this folio.
86	104	*/
87	105	static inline int folio_ref_count(const struct folio *folio)

include/linux/pageblock-flags.h+3−3

@@ -36,12 +36,12 @@ enum pageblock_bits { · 35 unmodified lines
36	36
37	37	#define NR_PAGEBLOCK_BITS (roundup_pow_of_two(__NR_PAGEBLOCK_BITS))
38	38
39		−#define MIGRATETYPE_MASK (BIT(PB_migrate_0)\|BIT(PB_migrate_1)\|BIT(PB_migrate_2))
	39	+#define PAGEBLOCK_MIGRATETYPE_MASK (BIT(PB_migrate_0)\|BIT(PB_migrate_1)\|BIT(PB_migrate_2))
40	40
41	41	#ifdef CONFIG_MEMORY_ISOLATION
42		−#define MIGRATETYPE_AND_ISO_MASK (MIGRATETYPE_MASK \| BIT(PB_migrate_isolate))
	42	+#define PAGEBLOCK_ISO_MASK BIT(PB_migrate_isolate)
43	43	#else
44		−#define MIGRATETYPE_AND_ISO_MASK MIGRATETYPE_MASK
	44	+#define PAGEBLOCK_ISO_MASK 0
45	45	#endif
46	46
47	47	#if defined(CONFIG_HUGETLB_PAGE)

include/linux/pagemap.h+2−0

@@ -1350,6 +1350,7 @@ struct readahead_control { · 1349 unmodified lines
1350	1350	struct file_ra_state *ra;
1351	1351	/* private: use the readahead_* accessors instead */
1352	1352	pgoff_t _index;
	1353	+ pgoff_t _max_index; /* limit readahead to _max_index, inclusive */
1353	1354	unsigned int _nr_pages;
1354	1355	unsigned int _batch_count;
1355	1356	bool dropbehind;
@@ -1363,6 +1364,7 @@ struct readahead_control { · 7 unmodified lines
1363	1364	.mapping = m, \
1364	1365	.ra = r, \
1365	1366	._index = i, \
	1367	+ ._max_index = ULONG_MAX, \
1366	1368	}
1367	1369
1368	1370	#define VM_READAHEAD_PAGES (SZ_128K / PAGE_SIZE)

include/linux/swap.h+12−12

@@ -213,6 +213,7 @@ enum { · 212 unmodified lines
213	213	SWP_PAGE_DISCARD = (1 << 10), /* freed swap page-cluster discards */
214	214	SWP_STABLE_WRITES = (1 << 11), /* no overwrite PG_writeback pages */
215	215	SWP_SYNCHRONOUS_IO = (1 << 12), /* synchronous IO is efficient */
	216	+ SWP_HIBERNATION = (1 << 13), /* pinned for hibernation */
216	217	/* add others here before... */
217	218	};
218	219
@@ -252,7 +253,6 @@ struct swap_info_struct { · 33 unmodified lines
252	253	struct plist_node list; /* entry in swap_active_head */
253	254	signed char type; /* strange name for an index */
254	255	unsigned int max; /* size of this swap device */
255		− unsigned long zeromap; / kvmalloc'ed bitmap to track zero pages */
256	256	struct swap_cluster_info cluster_info; / cluster info. Only for SSD */
257	257	struct list_head free_clusters; /* free clusters list */
258	258	struct list_head full_clusters; /* full clusters list */
@@ -433,7 +433,9 @@ static inline long get_nr_swap_pages(void) · 174 unmodified lines
433	433	}
434	434
435	435	extern void si_swapinfo(struct sysinfo *);
436		−int swap_type_of(dev_t device, sector_t offset);
	436	+extern int pin_hibernation_swap_type(dev_t device, sector_t offset);
	437	+extern void unpin_hibernation_swap_type(int type);
	438	+extern int find_hibernation_swap_type(dev_t device, sector_t offset);
437	439	int find_first_swap(dev_t *device);
438	440	extern unsigned int count_swap_pages(int, int);
439	441	extern sector_t swapdev_block(int, pgoff_t);
@@ -571,33 +573,31 @@ static inline void folio_throttle_swaprate(struct folio *folio, gfp_t gfp) · 131 unmodified lines
571	573	#endif
572	574
573	575	#if defined(CONFIG_MEMCG) && defined(CONFIG_SWAP)
574		−int __mem_cgroup_try_charge_swap(struct folio *folio, swp_entry_t entry);
575		−static inline int mem_cgroup_try_charge_swap(struct folio *folio,
576		− swp_entry_t entry)
	576	+int __mem_cgroup_try_charge_swap(struct folio *folio);
	577	+static inline int mem_cgroup_try_charge_swap(struct folio *folio)
577	578	{
578	579	if (mem_cgroup_disabled())
579	580	return 0;
580		− return __mem_cgroup_try_charge_swap(folio, entry);
	581	+ return __mem_cgroup_try_charge_swap(folio);
581	582	}
582	583
583		−extern void __mem_cgroup_uncharge_swap(swp_entry_t entry, unsigned int nr_pages);
584		−static inline void mem_cgroup_uncharge_swap(swp_entry_t entry, unsigned int nr_pages)
	584	+extern void __mem_cgroup_uncharge_swap(unsigned short id, unsigned int nr_pages);
	585	+static inline void mem_cgroup_uncharge_swap(unsigned short id, unsigned int nr_pages)
585	586	{
586	587	if (mem_cgroup_disabled())
587	588	return;
588		− __mem_cgroup_uncharge_swap(entry, nr_pages);
	589	+ __mem_cgroup_uncharge_swap(id, nr_pages);
589	590	}
590	591
591	592	extern long mem_cgroup_get_nr_swap_pages(struct mem_cgroup *memcg);
592	593	extern bool mem_cgroup_swap_full(struct folio *folio);
593	594	#else
594		−static inline int mem_cgroup_try_charge_swap(struct folio *folio,
595		− swp_entry_t entry)
	595	+static inline int mem_cgroup_try_charge_swap(struct folio *folio)
596	596	{
597	597	return 0;
598	598	}
599	599
600		−static inline void mem_cgroup_uncharge_swap(swp_entry_t entry,
	600	+static inline void mem_cgroup_uncharge_swap(unsigned short id,
601	601	unsigned int nr_pages)
602	602	{
603	603	}

include/linux/swap_cgroup.h+0−47

@@ -1,47 +0,0 @@
1		−/* SPDX-License-Identifier: GPL-2.0 */
2		−#ifndef __LINUX_SWAP_CGROUP_H
3		−#define __LINUX_SWAP_CGROUP_H
4		−
5		−#include <linux/swap.h>
6		−
7		−#if defined(CONFIG_MEMCG) && defined(CONFIG_SWAP)
8		−
9		−extern void swap_cgroup_record(struct folio *folio, unsigned short id, swp_entry_t ent);
10		−extern unsigned short swap_cgroup_clear(swp_entry_t ent, unsigned int nr_ents);
11		−extern unsigned short lookup_swap_cgroup_id(swp_entry_t ent);
12		−extern int swap_cgroup_swapon(int type, unsigned long max_pages);
13		−extern void swap_cgroup_swapoff(int type);
14		−
15		−#else
16		−
17		−static inline
18		−void swap_cgroup_record(struct folio *folio, unsigned short id, swp_entry_t ent)
19		−{
20		−}
21		−
22		−static inline
23		−unsigned short swap_cgroup_clear(swp_entry_t ent, unsigned int nr_ents)
24		−{
25		− return 0;
26		−}
27		−
28		−static inline
29		−unsigned short lookup_swap_cgroup_id(swp_entry_t ent)
30		−{
31		− return 0;
32		−}
33		−
34		−static inline int
35		−swap_cgroup_swapon(int type, unsigned long max_pages)
36		−{
37		− return 0;
38		−}
39		−
40		−static inline void swap_cgroup_swapoff(int type)
41		−{
42		− return;
43		−}
44		−
45		−#endif
46		−
47		−#endif /* __LINUX_SWAP_CGROUP_H */

include/linux/thread_info.h+1−1

@@ -92,7 +92,7 @@ static inline long set_restart_fn(struct restart_block *restart, · 91 unmodified lines
92	92	#define THREAD_ALIGN THREAD_SIZE
93	93	#endif
94	94
95		−#define THREADINFO_GFP (GFP_KERNEL_ACCOUNT \| __GFP_ZERO)
	95	+#define THREADINFO_GFP (GFP_KERNEL_ACCOUNT \| __GFP_ZERO \| __GFP_SKIP_KASAN)
96	96
97	97	/*
98	98	* flag set/clear/test wrappers

include/linux/userfaultfd_k.h+2−38

@@ -23,8 +23,8 @@ · 22 unmodified lines
23	23	/* The set of all possible UFFD-related VM flags. */
24	24	#define __VM_UFFD_FLAGS (VM_UFFD_MISSING \| VM_UFFD_WP \| VM_UFFD_MINOR)
25	25
26		−#define __VMA_UFFD_FLAGS mk_vma_flags(VMA_UFFD_MISSING_BIT, VMA_UFFD_WP_BIT, \
27		− VMA_UFFD_MINOR_BIT)
	26	+#define __VMA_UFFD_FLAGS mk_vma_flags_from_masks(VMA_UFFD_MISSING, VMA_UFFD_WP, \
	27	+ VMA_UFFD_MINOR)
28	28
29	29	/*
30	30	* CAREFUL: Check include/uapi/asm-generic/fcntl.h when defining
@@ -147,26 +147,12 @@ static inline uffd_flags_t uffd_flags_set_mode(uffd_flags_t flags, enum mfill_at · 116 unmodified lines
147	147	/* Flags controlling behavior. These behavior changes are mode-independent. */
148	148	#define MFILL_ATOMIC_WP MFILL_ATOMIC_FLAG(0)
149	149
150		−extern ssize_t mfill_atomic_copy(struct userfaultfd_ctx *ctx, unsigned long dst_start,
151		− unsigned long src_start, unsigned long len,
152		− uffd_flags_t flags);
153		−extern ssize_t mfill_atomic_zeropage(struct userfaultfd_ctx *ctx,
154		− unsigned long dst_start,
155		− unsigned long len);
156		−extern ssize_t mfill_atomic_continue(struct userfaultfd_ctx *ctx, unsigned long dst_start,
157		− unsigned long len, uffd_flags_t flags);
158		−extern ssize_t mfill_atomic_poison(struct userfaultfd_ctx *ctx, unsigned long start,
159		− unsigned long len, uffd_flags_t flags);
160		−extern int mwriteprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
161		− unsigned long len, bool enable_wp);
162	150	extern long uffd_wp_range(struct vm_area_struct *vma,
163	151	unsigned long start, unsigned long len, bool enable_wp);
164	152
165	153	/* move_pages */
166	154	void double_pt_lock(spinlock_t ptl1, spinlock_t ptl2);
167	155	void double_pt_unlock(spinlock_t ptl1, spinlock_t ptl2);
168		−ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start,
169		− unsigned long src_start, unsigned long len, __u64 flags);
170	156	int move_pages_huge_pmd(struct mm_struct mm, pmd_t dst_pmd, pmd_t *src_pmd, pmd_t dst_pmdval,
171	157	struct vm_area_struct *dst_vma,
172	158	struct vm_area_struct *src_vma,
@@ -239,9 +225,6 @@ static inline bool userfaultfd_armed(struct vm_area_struct *vma) · 66 unmodified lines
239	225	return vma->vm_flags & __VM_UFFD_FLAGS;
240	226	}
241	227
242		−bool vma_can_userfault(struct vm_area_struct *vma, vm_flags_t vm_flags,
243		− bool wp_async);
244		−
245	228	static inline bool vma_has_uffd_without_event_remap(struct vm_area_struct *vma)
246	229	{
247	230	struct userfaultfd_ctx *uffd_ctx = vma->vm_userfaultfd_ctx.ctx;
@@ -271,25 +254,6 @@ extern void userfaultfd_unmap_complete(struct mm_struct *mm, · 23 unmodified lines
271	254	extern bool userfaultfd_wp_unpopulated(struct vm_area_struct *vma);
272	255	extern bool userfaultfd_wp_async(struct vm_area_struct *vma);
273	256
274		−void userfaultfd_reset_ctx(struct vm_area_struct *vma);
275		−
276		−struct vm_area_struct userfaultfd_clear_vma(struct vma_iterator vmi,
277		− struct vm_area_struct *prev,
278		− struct vm_area_struct *vma,
279		− unsigned long start,
280		− unsigned long end);
281		−
282		−int userfaultfd_register_range(struct userfaultfd_ctx *ctx,
283		− struct vm_area_struct *vma,
284		− vm_flags_t vm_flags,
285		− unsigned long start, unsigned long end,
286		− bool wp_async);
287		−
288		−void userfaultfd_release_new(struct userfaultfd_ctx *ctx);
289		−
290		−void userfaultfd_release_all(struct mm_struct *mm,
291		− struct userfaultfd_ctx *ctx);
292		−
293	257	static inline bool userfaultfd_wp_use_markers(struct vm_area_struct *vma)
294	258	{
295	259	/* Only wr-protect mode uses pte markers */

include/linux/vmalloc.h+3−1

@@ -265,7 +265,9 @@ static inline bool is_vm_area_hugepages(const void *addr) · 264 unmodified lines
265	265	* allocated in the vmalloc layer.
266	266	*/
267	267	#ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC
268		− return find_vm_area(addr)->page_order > 0;
	268	+ struct vm_struct *area = find_vm_area(addr);
	269	+
	270	+ return area && area->page_order > 0;
269	271	#else
270	272	return false;
271	273	#endif

include/linux/vmpressure.h+5−4

@@ -30,8 +30,8 @@ struct vmpressure { · 29 unmodified lines
30	30	struct mem_cgroup;
31	31
32	32	#ifdef CONFIG_MEMCG
33		−extern void vmpressure(gfp_t gfp, struct mem_cgroup *memcg, bool tree,
34		− unsigned long scanned, unsigned long reclaimed);
	33	+void vmpressure(gfp_t gfp, int order, struct mem_cgroup *memcg, bool tree,
	34	+ unsigned long scanned, unsigned long reclaimed);
35	35	extern void vmpressure_prio(gfp_t gfp, struct mem_cgroup *memcg, int prio);
36	36
37	37	extern void vmpressure_init(struct vmpressure *vmpr);
@@ -44,8 +44,9 @@ extern int vmpressure_register_event(struct mem_cgroup *memcg, · 6 unmodified lines
44	44	extern void vmpressure_unregister_event(struct mem_cgroup *memcg,
45	45	struct eventfd_ctx *eventfd);
46	46	#else
47		−static inline void vmpressure(gfp_t gfp, struct mem_cgroup *memcg, bool tree,
48		− unsigned long scanned, unsigned long reclaimed) {}
	47	+static inline void vmpressure(gfp_t gfp, int order, struct mem_cgroup *memcg,
	48	+ bool tree, unsigned long scanned,
	49	+ unsigned long reclaimed) {}
49	50	static inline void vmpressure_prio(gfp_t gfp, struct mem_cgroup *memcg,
50	51	int prio) {}
51	52	#endif /* CONFIG_MEMCG */

include/trace/events/damon.h+38−0

@@ -130,6 +130,44 @@ TRACE_EVENT(damon_monitor_intervals_tune, · 129 unmodified lines
130	130	TP_printk("sample_us=%lu", __entry->sample_us)
131	131	);
132	132
	133	+TRACE_EVENT_CONDITION(damon_region_aggregated,
	134	+
	135	+ TP_PROTO(unsigned int target_id, struct damon_region *r,
	136	+ unsigned int nr_regions, unsigned int nr_probes),
	137	+
	138	+ TP_ARGS(target_id, r, nr_regions, nr_probes),
	139	+
	140	+ TP_CONDITION(nr_probes > 0),
	141	+
	142	+ TP_STRUCT__entry(
	143	+ __field(unsigned long, target_id)
	144	+ __field(unsigned long, start)
	145	+ __field(unsigned long, end)
	146	+ __field(unsigned int, nr_regions)
	147	+ __field(unsigned int, nr_accesses)
	148	+ __field(unsigned int, age)
	149	+ __dynamic_array(unsigned char, probe_hits, nr_probes)
	150	+ ),
	151	+
	152	+ TP_fast_assign(
	153	+ __entry->target_id = target_id;
	154	+ __entry->start = r->ar.start;
	155	+ __entry->end = r->ar.end;
	156	+ __entry->nr_regions = nr_regions;
	157	+ __entry->nr_accesses = r->nr_accesses;
	158	+ __entry->age = r->age;
	159	+ memcpy(__get_dynamic_array(probe_hits), r->probe_hits,
	160	+ sizeof(r->probe_hits) nr_probes);
	161	+ ),
	162	+
	163	+ TP_printk("target_id=%lu nr_regions=%u %lu-%lu: %u %u probe_hits=%s",
	164	+ __entry->target_id, __entry->nr_regions,
	165	+ __entry->start, __entry->end,
	166	+ __entry->nr_accesses, __entry->age,
	167	+ __print_hex(__get_dynamic_array(probe_hits),
	168	+ __get_dynamic_array_len(probe_hits)))
	169	+);
	170	+
133	171	TRACE_EVENT(damon_aggregated,
134	172
135	173	TP_PROTO(unsigned int target_id, struct damon_region *r,

include/trace/events/vmscan.h+52−0

@@ -96,6 +96,58 @@ TRACE_EVENT(mm_vmscan_kswapd_wake, · 95 unmodified lines
96	96	__entry->order)
97	97	);
98	98
	99	+TRACE_EVENT(mm_vmscan_balance_pgdat_begin,
	100	+
	101	+ TP_PROTO(int nid, int order, int highest_zoneidx),
	102	+
	103	+ TP_ARGS(nid, order, highest_zoneidx),
	104	+
	105	+ TP_STRUCT__entry(
	106	+ __field(int, nid)
	107	+ __field(int, order)
	108	+ __field(int, highest_zoneidx)
	109	+ ),
	110	+
	111	+ TP_fast_assign(
	112	+ __entry->nid = nid;
	113	+ __entry->order = order;
	114	+ __entry->highest_zoneidx = highest_zoneidx;
	115	+ ),
	116	+
	117	+ TP_printk("nid=%d order=%d highest_zoneidx=%-8s",
	118	+ __entry->nid,
	119	+ __entry->order,
	120	+ __print_symbolic(__entry->highest_zoneidx, ZONE_TYPE))
	121	+);
	122	+
	123	+TRACE_EVENT(mm_vmscan_balance_pgdat_end,
	124	+
	125	+ TP_PROTO(int nid, int order, int highest_zoneidx,
	126	+ unsigned long nr_reclaimed),
	127	+
	128	+ TP_ARGS(nid, order, highest_zoneidx, nr_reclaimed),
	129	+
	130	+ TP_STRUCT__entry(
	131	+ __field(int, nid)
	132	+ __field(int, order)
	133	+ __field(int, highest_zoneidx)
	134	+ __field(unsigned long, nr_reclaimed)
	135	+ ),
	136	+
	137	+ TP_fast_assign(
	138	+ __entry->nid = nid;
	139	+ __entry->order = order;
	140	+ __entry->highest_zoneidx = highest_zoneidx;
	141	+ __entry->nr_reclaimed = nr_reclaimed;
	142	+ ),
	143	+
	144	+ TP_printk("nid=%d order=%d highest_zoneidx=%-8s nr_reclaimed=%lu",
	145	+ __entry->nid,
	146	+ __entry->order,
	147	+ __print_symbolic(__entry->highest_zoneidx, ZONE_TYPE),
	148	+ __entry->nr_reclaimed)
	149	+);
	150	+
99	151	TRACE_EVENT(mm_vmscan_wakeup_kswapd,
100	152
101	153	TP_PROTO(int nid, int zid, int order, gfp_t gfp_flags),

Merge tag 'mm-stable-2026-06-18-09-26' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Keyboard shortcuts

General

Issues and pull requests

Issues, pull requests and Inbox

Repository code