Merge tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull more MM updates from Andrew Morton: - "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff" (Lorenzo Stoakes) Index MAP_PRIVATE file-backed folios by their anonymous page offset to resolve confusion around reverse mapping for zeroed and CoW'd file-backed memory. Use this new VMA anonymous page offset tracking to eliminate index conflicts and lay the foundation for scalable CoW performance improvements. - "promote mapped executable folios after first usage for MGLRU" (Baolin Wang) Make MGLRU's protection of mapped executable file folios more reliable. Follow the classical LRU's logic, promoting mapped executable file folios after their first usage to give executable code a better chance to stay in memory and improve workload performance. - "mm: vmscan: fix node reclaim ignoring swappiness parameter" (Ridong Chen) Fix per-node proactive reclaim interface's ignoring the swappiness parameter when CONFIG_MEMCG is disabled by consolidating sc_swappiness() into a single function that checks proactive_swappiness regardless of kernel configuration. - "mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost" (Usama Arif) Reduce lru_lock contention in the reclaim path by deriving scan-balance costs from vmstat counters rather than lock-acquired producer updates. Read and decay these cost signals on the reclaim side under a dedicated per-lruvec lock, reducing total LRU lock wait time by over 60% without impacting scan throughput. - "zram: fix zram issues reported by sashiko" (Sergey Senozhatsky) Fix two low-risk zram bugs which Sashiko spotted in drive-by review. - "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's memcg" (Zi Yan) Fix xas_split_alloc() by enabling target folio memcg charging during splits and adding the missing __GFP_ACCOUNT flag for proper XArray node memory accounting. - "selftests/mm: use pattern matching in .gitignore" (Pratyush Mallick) Replace hardcoded binary names in selftests/mm/.gitignore with a generic pattern-matching rule to automatically ignore generated test files and avoid manual updates when adding new tests. - "mm/page_ext: remove pgdat_page_ext_init()" (Sang-Heon Jeon) Make the incompatibility between FLATMEM and NUMA explicit in mm/Kconfig and remove the unused pgdat_page_ext_init() function. - "zram: fix zstd error paths and add parameter validation" (Haoqin Huang) Clean up zram compression backends by removing redundant error cleanup, adding parameter and dictionary validation, auto-prefixing algorithm error logs, and resetting parameters prior to reinitialization. - "zram: fix stale scan bounds after reinitialization" (Longlong Xia) Prevent out-of-bounds slot accesses during concurrent zram resets by moving table scan bound calculations under dev_lock in writeback_store() and read_block_state(). - "add anon mTHP collapse test cases" (Baolin Wang) Extend selftests helper functions to support arbitrary page orders and add new test cases and options for mTHP collapse in khugepaged. - "selftests/mm: Handle unsupported and transient test conditions" (Muhammad Usama Anjum) Update MM selftests to report a SKIP status instead of a failure when required kernel or filesystem features are unsupported, while adding retry logic for transient page migration errors. - "mm/zswap: Fixes and improves the zswap shrink" (Hao Jia) Fix the missing zswap global shrinker when CONFIG_MEMCG is disabled and extend shrink_memcg() to support batch writeback for improved writeback efficiency. - "alloc_tag: introduce IOCTL-based filtering for MAP" (Suren Baghdasaryan) Introduce an IOCTL-based binary interface for memory allocation profiling that enables kernel-side filtering before per-CPU counter aggregation. This eliminates the text-parsing overhead of /proc/allocinfo and provides up to a 20x speedup by transferring only filtered allocation data to userspace. - "better block swap batching and a different take on swap_ops v5" (Christoph Hellwig) Refactor block swap I/O to use swap_iocb for batching instead of single-bio requests and rebase the swap_ops interface, achieving faster swap throughput during kernel builds. - "mm: kmemleak: reduce transient false positives by confirming leaks" (Catalin Marinas) Reduce false-positive kmemleak reports by combining two kmemleak enhancements that add a second confirmation scan and a configurable minimum unreferenced scan count module parameter. - "mm: kmemleak: default min_unref_scans to 2 for verbose kernels" (Breno Leitao) Auto-scanning kernels can generate false-positive memory leak reports on single scans, so this patch defaults min_unref_scans to 2 when CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled to require a second confirming scan. - "swap_ops updates" (Christoph Hellwig) Batching I/O for synchronous swap devices causes performance regressions and filesystem-based swap suffers from double-indirection overhead. This series resolves both issues by reintroducing per-folio writes for synchronous swap and allowing filesystems to directly export their own swap_ops. - "mm/khugepaged: several cleanups" (Nico Pache) khugepaged accumulated redundant state-checking patterns and outdated comments following mTHP integration. Introduce dedicated helpers for PTE validation and event counting while refreshing the internal documentation. - "maple_tree: lock checking and clean ups" (Liam Howlett) Syzbot reports incorrectly blame memory management exit paths for locking bugs, maple tree erase operations risk allocation failures without gfp flags and internal documentation lacks clarity. Improve lock error detection, update docs, fix race and allocation edge cases and optimize erase allocations using a fallback to GFP_KERNEL | GFP_NOFAIL. * tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (172 commits) selftests/proc: make proc-maps-race work with READ_IMPLIES_EXEC memcg: move LRU size accounting on reparenting instead of copying it mm/vmscan: fix comment logic in balance_pgdat maple_tree: add helper mas_make_walkable() maple_tree: avoid extra gap calculation maple_tree: fix argument name in header maple_tree: change two GFP flags in tests maple_tree: document erase and allocations better maple_tree: avoid mas_erase() and mtree_erase() failures maple_tree: document that erase may use GFP_KERNEL for allocations maple_tree: catch race in mas_alloc_cyclic() maple_tree: add bulk parent set helper maple_tree: micro optimisation of mas_wr_store_type() maple_tree: optimise mas_wr_node_store() when not in rcu mode maple_tree: use prefetched value in mas_wr_store_type() maple_tree: clarify comments on mas_nomem() maple_tree: drop MAPLE_ALLOC_SLOTS maple_tree: drop dead code from mas_extend_spanning_null() maple_tree: documentation fix maple_tree: add write lock checking with lockdep sequence numbers ...

Linus Torvalds committed 2 months ago

Commit 18fbf512 parents 5e6ff28 0685630168 files · +4561 −1770Browse files

Showing the changes to include (25 of 168 files). Show all

Unified · Split

25 files

include/linux/codetag.h +2 −0
include/linux/execmem.h +1 −1
include/linux/fs.h +0 −1
include/linux/lockdep.h +3 −0
include/linux/lockdep_types.h +2 −1
include/linux/maple_tree.h +9 −7
include/linux/memcontrol.h +17 −4
include/linux/mm.h +125 −7
include/linux/mm_types.h +26 −14
include/linux/mmzone.h +13 −3
include/linux/nfs_fs.h +0 −1
include/linux/page_ext.h +0 −5
include/linux/pagemap.h +37 −3
include/linux/percpu-defs.h +1 −6
include/linux/rmap.h +6 −5
include/linux/sched.h +1 −0
include/linux/shmem_fs.h +1 −4
include/linux/swap.h +9 −21
include/linux/swap_ops.h +44 −0
include/linux/swapops.h +4 −3
include/linux/vm_event_item.h +4 −0
include/linux/vmalloc.h +1 −1
include/linux/vmstat.h +16 −1
include/rv/da_monitor.h +1 −1
include/uapi/linux/alloc_tag.h +99 −0

include/linux/codetag.h+2−0

@@ -77,6 +77,8 @@ struct codetag_iterator { · 76 unmodified lines
77	77	void codetag_lock_module_list(struct codetag_type *cttype);
78	78	bool codetag_trylock_module_list(struct codetag_type *cttype);
79	79	void codetag_unlock_module_list(struct codetag_type *cttype);
	80	+unsigned long codetag_get_content_id(struct codetag_type *cttype);
	81	+unsigned int codetag_get_count(struct codetag_type *cttype);
80	82	struct codetag_iterator codetag_get_ct_iter(struct codetag_type *cttype);
81	83	struct codetag codetag_next_ct(struct codetag_iterator iter);
82	84

include/linux/execmem.h+1−1

@@ -89,7 +89,7 @@ static inline int execmem_restore_rox(void *ptr, size_t size) { return 0; } · 88 unmodified lines
89	89	* @end: address space end (inclusive)
90	90	* @fallback_start: start of the secondary address space range for fallback
91	91	* allocations on architectures that require it
92		− * @fallback_end: start of the secondary address space (inclusive)
	92	+ * @fallback_end: end of the secondary address space (inclusive)
93	93	* @pgprot: permissions for memory in this address space
94	94	* @alignment: alignment required for text allocations
95	95	* @flags: options for memory allocations for this range

include/linux/fs.h+0−1

@@ -438,7 +438,6 @@ struct address_space_operations { · 437 unmodified lines
438	438	int (swap_activate)(struct swap_info_struct sis, struct file *file,
439	439	sector_t *span);
440	440	void (swap_deactivate)(struct file file);
441		− int (swap_rw)(struct kiocb iocb, struct iov_iter *iter);
442	441	};
443	442
444	443	extern const struct address_space_operations empty_aops;

include/linux/lockdep.h+3−0

@@ -273,6 +273,9 @@ extern struct pin_cookie lock_pin_lock(struct lockdep_map *lock); · 272 unmodified lines
273	273	extern void lock_repin_lock(struct lockdep_map *lock, struct pin_cookie);
274	274	extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie);
275	275
	276	+extern u32 lock_sequence(struct lockdep_map *lock);
	277	+#define lockdep_sequence(lock) lock_sequence(&(lock)->dep_map)
	278	+
276	279	#define lockdep_depth(tsk) (debug_locks ? (tsk)->lockdep_depth : 0)
277	280
278	281	#define lockdep_assert(cond) \

include/linux/lockdep_types.h+2−1

@@ -253,7 +253,8 @@ struct held_lock { · 252 unmodified lines
253	253	unsigned int hardirqs_off:1;
254	254	unsigned int sync:1;
255	255	unsigned int references:11; /* 32 bits */
256		− unsigned int pin_count;
	256	+ unsigned int pin_count:24;
	257	+ unsigned int seq_count:8;
257	258	};
258	259
259	260	#else /* !CONFIG_LOCKDEP */

include/linux/maple_tree.h+9−7

@@ -11,7 +11,6 @@ · 10 unmodified lines
11	11	#include <linux/kernel.h>
12	12	#include <linux/rcupdate.h>
13	13	#include <linux/spinlock.h>
14		−/* #define CONFIG_MAPLE_RCU_DISABLED */
15	14
16	15	/*
17	16	* Allocated nodes are mutable until they have been inserted into the tree,
@@ -29,13 +28,11 @@ · 11 unmodified lines
29	28	#define MAPLE_NODE_SLOTS 31 /* 256 bytes including ->parent */
30	29	#define MAPLE_RANGE64_SLOTS 16 /* 256 bytes */
31	30	#define MAPLE_ARANGE64_SLOTS 10 /* 240 bytes */
32		−#define MAPLE_ALLOC_SLOTS (MAPLE_NODE_SLOTS - 1)
33	31	#else
34	32	/* 32bit sizes */
35	33	#define MAPLE_NODE_SLOTS 63 /* 256 bytes including ->parent */
36	34	#define MAPLE_RANGE64_SLOTS 32 /* 256 bytes */
37	35	#define MAPLE_ARANGE64_SLOTS 21 /* 240 bytes */
38		−#define MAPLE_ALLOC_SLOTS (MAPLE_NODE_SLOTS - 2)
39	36	#endif /* defined(CONFIG_64BIT) \|\| defined(BUILD_VDSO32_64) */
40	37
41	38	#define MAPLE_NODE_MASK 255UL
@@ -485,6 +482,12 @@ struct ma_state { · 443 unmodified lines
485	482	unsigned char mas_flags;
486	483	unsigned char end; /* The end of the node */
487	484	enum store_type store_type; /* The type of store needed for this operation */
	485	+#ifdef CONFIG_LOCKDEP
	486	+ u32 ld_seq;
	487	+#ifdef CONFIG_RCU_STRICT_GRACE_PERIOD
	488	+ unsigned long rcu_gp;
	489	+#endif /* CONFIG_RCU_STRICT_GRACE_PERIOD */
	490	+#endif /* CONFIG_LOCKDEP */
488	491	};
489	492
490	493	struct ma_wr_state {
@@ -567,12 +570,14 @@ int mas_alloc_cyclic(struct ma_state mas, unsigned long startp, · 76 unmodified lines
567	570	unsigned long *next, gfp_t gfp);
568	571
569	572	bool mas_nomem(struct ma_state *mas, gfp_t gfp);
	573	+bool mas_nomem_nofail(struct ma_state *mas, unsigned long index,
	574	+ unsigned long last);
570	575	void mas_pause(struct ma_state *mas);
571	576	void maple_tree_init(void);
572	577	void mas_destroy(struct ma_state *mas);
573	578
574	579	void mas_prev(struct ma_state mas, unsigned long min);
575		−void mas_prev_range(struct ma_state mas, unsigned long max);
	580	+void mas_prev_range(struct ma_state mas, unsigned long min);
576	581	void mas_next(struct ma_state mas, unsigned long max);
577	582	void mas_next_range(struct ma_state mas, unsigned long max);
578	583
@@ -864,9 +869,6 @@ static inline void mt_init(struct maple_tree *mt) · 285 unmodified lines
864	869
865	870	static inline bool mt_in_rcu(struct maple_tree *mt)
866	871	{
867		−#ifdef CONFIG_MAPLE_RCU_DISABLED
868		− return false;
869		−#endif
870	872	return mt->ma_flags & MT_FLAGS_USE_RCU;
871	873	}
872	874

include/linux/memcontrol.h+17−4

@@ -239,8 +239,6 @@ struct mem_cgroup { · 238 unmodified lines
239	239	*/
240	240	bool oom_group;
241	241
242		− int swappiness;
243		−
244	242	/* memory.events and memory.events.local */
245	243	struct cgroup_file events_file;
246	244	struct cgroup_file events_local_file;
@@ -270,10 +268,15 @@ struct mem_cgroup { · 23 unmodified lines
270	268	#endif
271	269	int kmemcg_id;
272	270
273		− struct memcg_vmstats_percpu __percpu *vmstats_percpu;
274		−
275	271	#ifdef CONFIG_CGROUP_WRITEBACK
276	272	struct list_head cgwb_list;
	273	+#endif
	274	+
	275	+ /* Keep the hot per-CPU stats pointer away from memory event counters. */
	276	+ struct memcg_vmstats_percpu __percpu *vmstats_percpu
	277	+ ____cacheline_aligned_in_smp;
	278	+
	279	+#ifdef CONFIG_CGROUP_WRITEBACK
277	280	struct wb_domain cgwb_domain;
278	281	struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT];
279	282	#endif
@@ -318,6 +321,8 @@ struct mem_cgroup { · 38 unmodified lines
318	321	/* List of events which userspace want to receive */
319	322	struct list_head event_list;
320	323	spinlock_t event_list_lock;
	324	+
	325	+ int swappiness;
321	326	#endif /* CONFIG_MEMCG_V1 */
322	327
323	328	struct mem_cgroup_per_node *nodeinfo[];
@@ -947,6 +952,8 @@ unsigned long memcg_page_state_output(struct mem_cgroup *memcg, int item); · 623 unmodified lines
947	952	bool memcg_stat_item_valid(int idx);
948	953	bool memcg_vm_event_item_valid(enum vm_event_item idx);
949	954	unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx);
	955	+unsigned long lruvec_page_state_monotonic(struct lruvec *lruvec,
	956	+ enum node_stat_item idx);
950	957	unsigned long lruvec_page_state_local(struct lruvec *lruvec,
951	958	enum node_stat_item idx);
952	959
@@ -1399,6 +1406,12 @@ static inline unsigned long lruvec_page_state(struct lruvec *lruvec, · 446 unmodified lines
1399	1406	return node_page_state(lruvec_pgdat(lruvec), idx);
1400	1407	}
1401	1408
	1409	+static inline unsigned long lruvec_page_state_monotonic(struct lruvec *lruvec,
	1410	+ enum node_stat_item idx)
	1411	+{
	1412	+ return node_page_state_monotonic(lruvec_pgdat(lruvec), idx);
	1413	+}
	1414	+
1402	1415	static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec,
1403	1416	enum node_stat_item idx)
1404	1417	{

include/linux/mm.h+125−7

@@ -1556,7 +1556,7 @@ static inline void vma_desc_set_anonymous(struct vm_area_desc *desc) · 1555 unmodified lines
1556	1556	desc->vm_ops = NULL;
1557	1557	}
1558	1558
1559		−static inline bool vma_is_anonymous(struct vm_area_struct *vma)
	1559	+static inline bool vma_is_anonymous(const struct vm_area_struct *vma)
1560	1560	{
1561	1561	return !vma->vm_ops;
1562	1562	}
@@ -2271,17 +2271,76 @@ void unpin_user_pages(struct page **pages, unsigned long npages); · 708 unmodified lines
2271	2271	void unpin_user_folio(struct folio *folio, unsigned long npages);
2272	2272	void unpin_folios(struct folio **folios, unsigned long nfolios);
2273	2273
2274		−static inline bool is_cow_mapping(vm_flags_t flags)
	2274	+/**
	2275	+ * vma_flags_is_cow_mapping() - Do these VMA flags imply a CoW mapping?
	2276	+ * @flags: The VMA flags to check.
	2277	+ *
	2278	+ * Mappings which could be CoW'd (subject to Copy-On-Write faults) are
	2279	+ * described as CoW mappings.
	2280	+ *
	2281	+ * All mappings backed by anonymous folios (all anonymous mappings and most
	2282	+ * MAP_PRIVATE-file backed ranges) are CoW mappings.
	2283	+ *
	2284	+ * All other mappings (including all MAP_SHARED mappings) are non-CoW.
	2285	+ *
	2286	+ * The criteria are !VMA_SHARED_BIT, VMA_MAYWRITE_BIT.
	2287	+ *
	2288	+ * VMA_MAYWRITE_BIT is checked instead of VMA_WRITE_BIT to account for both
	2289	+ * future mprotect() calls which can render a read-only mapping writable, and
	2290	+ * GUP with FOLL_FORCE (e.g. ptrace) which can CoW a read-only mapping.
	2291	+ *
	2292	+ * - No anonymous mapping can ever clear VMA_MAYWRITE_BIT.
	2293	+ *
	2294	+ * - Writes to anonymous mappings do not immediately result in CoW faults but
	2295	+ * may do so after the process is forked or if a read is followed by a
	2296	+ * write.
	2297	+ *
	2298	+ * - Writes to MAP_PRIVATE file-backed mappings result in CoW faults and may
	2299	+ * do so again after fork.
	2300	+ *
	2301	+ * - MAP_SHARED mappings of a file opened read-only are transformed into
	2302	+ * VMA_MAYSHARE_BIT, !VMA_SHARED_BIT, !VMA_MAYWRITE_BIT mappings, so remain
	2303	+ * non-CoW.
	2304	+ *
	2305	+ * - Drivers may clear VMA_MAYWRITE_BIT but do so at mmap() time and cannot
	2306	+ * mark themselves anonymous. Having cleared this flag it is not valid for
	2307	+ * them to leave the VMA_WRITE_BIT flag set.
	2308	+ *
	2309	+ * As a consequence, the anonymous reverse mapping only tracks CoW mappings.
	2310	+ *
	2311	+ * Returns: true if the flags indicate a CoW mapping, otherwise false.
	2312	+ */
	2313	+static inline bool vma_flags_is_cow_mapping(const vma_flags_t *flags)
2275	2314	{
2276		− return (flags & (VM_SHARED \| VM_MAYWRITE)) == VM_MAYWRITE;
	2315	+ return vma_flags_test(flags, VMA_MAYWRITE_BIT) &&
	2316	+ !vma_flags_test(flags, VMA_SHARED_BIT);
2277	2317	}
2278	2318
2279		−static inline bool vma_desc_is_cow_mapping(struct vm_area_desc *desc)
	2319	+/**
	2320	+ * vma_is_cow_mapping() - Is this VMA a CoW mapping?
	2321	+ * @vma: The VMA to check.
	2322	+ *
	2323	+ * See vma_flags_is_cow_mapping() for details.
	2324	+ *
	2325	+ * Returns: true if the VMA is a CoW mapping, otherwise false.
	2326	+ */
	2327	+static inline bool vma_is_cow_mapping(const struct vm_area_struct *vma)
2280	2328	{
2281		− const vma_flags_t *flags = &desc->vma_flags;
	2329	+ return vma_flags_is_cow_mapping(&vma->flags);
	2330	+}
2282	2331
2283		− return vma_flags_test(flags, VMA_MAYWRITE_BIT) &&
2284		− !vma_flags_test(flags, VMA_SHARED_BIT);
	2332	+/**
	2333	+ * vma_desc_is_cow_mapping() - Is this VMA descriptor a CoW mapping?
	2334	+ * @desc: The VMA descriptor to check.
	2335	+ *
	2336	+ * See vma_flags_is_cow_mapping() for details.
	2337	+ *
	2338	+ * Returns: true if the VMA descriptor describes a CoW mapping, otherwise
	2339	+ * false.
	2340	+ */
	2341	+static inline bool vma_desc_is_cow_mapping(struct vm_area_desc *desc)
	2342	+{
	2343	+ return vma_flags_is_cow_mapping(&desc->vma_flags);
2285	2344	}
2286	2345
2287	2346	#ifndef CONFIG_MMU
@@ -4393,6 +4452,65 @@ static inline pgoff_t vma_last_pgoff(const struct vm_area_struct *vma) · 2105 unmodified lines
4393	4452	return vma_end_pgoff(vma) - 1;
4394	4453	}
4395	4454
	4455	+/**
	4456	+ * vma_start_anon_pgoff() - Get the anonymous page offset of the start of @vma
	4457	+ * @vma: The VMA whose anonymous page offset is required.
	4458	+ *
	4459	+ * If unfaulted, then this is vma->vm_start >> PAGE_SHIFT, if faulted then the
	4460	+ * anonymous page offset at the time of first fault.
	4461	+ *
	4462	+ * If the VMA is anonymous, this returns the same value as vma_start_pgoff().
	4463	+ *
	4464	+ * This value is used for tracking MAP_PRIVATE file-backed mappings by their
	4465	+ * anonymous page offset.
	4466	+ *
	4467	+ * Returns: The anonymous page offset of the start of @vma.
	4468	+ */
	4469	+static inline pgoff_t vma_start_anon_pgoff(const struct vm_area_struct *vma)
	4470	+{
	4471	+ pgoff_t pgoff = 0;
	4472	+
	4473	+#ifdef CONFIG_64BIT
	4474	+ pgoff += vma->__vm_anon_pgoff_hi;
	4475	+ pgoff <<= 32;
	4476	+#endif
	4477	+ pgoff += vma->__vm_anon_pgoff_lo;
	4478	+ return pgoff;
	4479	+}
	4480	+
	4481	+/**
	4482	+ * vma_end_anon_pgoff() - Get the anonymous page offset of the exclusive end of
	4483	+ * @vma.
	4484	+ * @vma: The VMA whose end anonymous page offset is required.
	4485	+ *
	4486	+ * This returns the anonymous exclusive end page offset of @vma, which is useful
	4487	+ * for expressing page offset ranges.
	4488	+ *
	4489	+ * See the description of vma_start_anon_pgoff() for a description of VMA
	4490	+ * anonymous page offsets.
	4491	+ *
	4492	+ * Returns: The exclusive end anonymous page offset of @vma.
	4493	+ */
	4494	+static inline pgoff_t vma_end_anon_pgoff(const struct vm_area_struct *vma)
	4495	+{
	4496	+ return vma_start_anon_pgoff(vma) + vma_pages(vma);
	4497	+}
	4498	+
	4499	+/**
	4500	+ * vma_last_anon_pgoff() - Get the anonymous page offset of the last page in
	4501	+ * @vma.
	4502	+ * @vma: The VMA whose last anonymous page offset is required.
	4503	+ *
	4504	+ * See the description of vma_start_anon_pgoff() for a description of VMA
	4505	+ * anonymous page offsets.
	4506	+ *
	4507	+ * Returns: The last anonymous page offset of @vma.
	4508	+ */
	4509	+static inline pgoff_t vma_last_anon_pgoff(const struct vm_area_struct *vma)
	4510	+{
	4511	+ return vma_end_anon_pgoff(vma) - 1;
	4512	+}
	4513	+
4396	4514	static inline unsigned long vma_desc_size(const struct vm_area_desc *desc)
4397	4515	{
4398	4516	return desc->end - desc->start;

include/linux/mm_types.h+26−14

@@ -967,6 +967,11 @@ struct vm_area_struct { · 966 unmodified lines
967	967	*/
968	968	unsigned int vm_lock_seq;
969	969	#endif
	970	+ /*
	971	+ * Low 32-bits of anonymous page offset.
	972	+ * See vma_start_anon_pgoff() comment for details.
	973	+ */
	974	+ unsigned int __vm_anon_pgoff_lo;
970	975	/*
971	976	* A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma
972	977	* list, after a COW of one of the file pages. A MAP_SHARED vma
@@ -1041,6 +1046,13 @@ struct vm_area_struct { · 68 unmodified lines
1041	1046	#ifdef CONFIG_DEBUG_LOCK_ALLOC
1042	1047	struct lockdep_map vmlock_dep_map;
1043	1048	#endif
	1049	+#endif
	1050	+#ifdef CONFIG_64BIT
	1051	+ /*
	1052	+ * High 32-bits of anonymous page offset.
	1053	+ * See vma_start_anon_pgoff() comment for details.
	1054	+ */
	1055	+ unsigned int __vm_anon_pgoff_hi;
1044	1056	#endif
1045	1057	/*
1046	1058	* For areas with an address space and backing store,
@@ -1706,20 +1718,20 @@ enum vm_fault_reason { · 659 unmodified lines
1706	1718	VM_FAULT_SIGSEGV \| VM_FAULT_HWPOISON \| \
1707	1719	VM_FAULT_HWPOISON_LARGE \| VM_FAULT_FALLBACK)
1708	1720
1709		−#define VM_FAULT_RESULT_TRACE \
1710		− { VM_FAULT_OOM, "OOM" }, \
1711		− { VM_FAULT_SIGBUS, "SIGBUS" }, \
1712		− { VM_FAULT_MAJOR, "MAJOR" }, \
1713		− { VM_FAULT_HWPOISON, "HWPOISON" }, \
1714		− { VM_FAULT_HWPOISON_LARGE, "HWPOISON_LARGE" }, \
1715		− { VM_FAULT_SIGSEGV, "SIGSEGV" }, \
1716		− { VM_FAULT_NOPAGE, "NOPAGE" }, \
1717		− { VM_FAULT_LOCKED, "LOCKED" }, \
1718		− { VM_FAULT_RETRY, "RETRY" }, \
1719		− { VM_FAULT_FALLBACK, "FALLBACK" }, \
1720		− { VM_FAULT_DONE_COW, "DONE_COW" }, \
1721		− { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" }, \
1722		− { VM_FAULT_COMPLETED, "COMPLETED" }
	1721	+#define VM_FAULT_RESULT_TRACE \
	1722	+ { (__force u32)VM_FAULT_OOM, "OOM" }, \
	1723	+ { (__force u32)VM_FAULT_SIGBUS, "SIGBUS" }, \
	1724	+ { (__force u32)VM_FAULT_MAJOR, "MAJOR" }, \
	1725	+ { (__force u32)VM_FAULT_HWPOISON, "HWPOISON" }, \
	1726	+ { (__force u32)VM_FAULT_HWPOISON_LARGE, "HWPOISON_LARGE" }, \
	1727	+ { (__force u32)VM_FAULT_SIGSEGV, "SIGSEGV" }, \
	1728	+ { (__force u32)VM_FAULT_NOPAGE, "NOPAGE" }, \
	1729	+ { (__force u32)VM_FAULT_LOCKED, "LOCKED" }, \
	1730	+ { (__force u32)VM_FAULT_RETRY, "RETRY" }, \
	1731	+ { (__force u32)VM_FAULT_FALLBACK, "FALLBACK" }, \
	1732	+ { (__force u32)VM_FAULT_DONE_COW, "DONE_COW" }, \
	1733	+ { (__force u32)VM_FAULT_NEEDDSYNC, "NEEDDSYNC" }, \
	1734	+ { (__force u32)VM_FAULT_COMPLETED, "COMPLETED" }
1723	1735
1724	1736	struct vm_special_mapping {
1725	1737	const char name; / The name, e.g. "[vdso]". */

include/linux/mmzone.h+13−3

@@ -323,6 +323,8 @@ enum node_stat_item { · 322 unmodified lines
323	323	PGSCAN_PROACTIVE,
324	324	PGSCAN_ANON,
325	325	PGSCAN_FILE,
	326	+ PGROTATE_ANON,
	327	+ PGROTATE_FILE,
326	328	PGREFILL,
327	329	#ifdef CONFIG_HUGETLB_PAGE
328	330	NR_HUGETLB,
@@ -755,6 +757,12 @@ void lru_gen_reparent_memcg(struct mem_cgroup memcg, struct mem_cgroup parent, · 426 unmodified lines
755	757
756	758	#endif /* CONFIG_LRU_GEN */
757	759
	760	+struct lru_cost {
	761	+ unsigned long count;
	762	+ unsigned long last_rotated;
	763	+ unsigned long last_io;
	764	+};
	765	+
758	766	struct lruvec {
759	767	struct list_head lists[NR_LRU_LISTS];
760	768	/* per lruvec lru_lock for memcg */
@@ -763,9 +771,12 @@ struct lruvec { · 2 unmodified lines
763	771	* These track the cost of reclaiming one LRU - file or anon -
764	772	* over the other. As the observed cost of reclaiming one LRU
765	773	* increases, the reclaim scan balance tips toward the other.
	774	+ * Updated and decayed at prepare_scan_control() time; cost_lock
	775	+ * serialises that update.
766	776	*/
767		− unsigned long anon_cost;
768		− unsigned long file_cost;
	777	+ struct lru_cost cost[ANON_AND_FILE];
	778	+ /* Protects cost[]. */
	779	+ spinlock_t cost_lock;
769	780	/* Non-resident age, driven by LRU movement */
770	781	atomic_long_t nonresident_age;
771	782	/* Refaults at the time of last reclaim cycle */
@@ -2058,7 +2069,6 @@ static inline struct mem_section *__nr_to_section(unsigned long nr) · 1286 unmodified lines
2058	2069	#endif
2059	2070	return &mem_section[root][nr & SECTION_ROOT_MASK];
2060	2071	}
2061		−extern size_t mem_section_usage_size(void);
2062	2072
2063	2073	/*
2064	2074	* We use the lower bits of the mem_map pointer to store a little bit of

include/linux/nfs_fs.h+0−1

@@ -552,7 +552,6 @@ static inline const struct cred nfs_file_cred(struct file file) · 551 unmodified lines
552	552	/*
553	553	* linux/fs/nfs/direct.c
554	554	*/
555		−int nfs_swap_rw(struct kiocb iocb, struct iov_iter iter);
556	555	ssize_t nfs_file_direct_read(struct kiocb *iocb,
557	556	struct iov_iter *iter, bool swap);
558	557	ssize_t nfs_file_direct_write(struct kiocb *iocb,

include/linux/page_ext.h+0−5

@@ -55,7 +55,6 @@ struct page_ext { · 54 unmodified lines
55	55
56	56	extern bool early_page_ext;
57	57	extern unsigned long page_ext_size;
58		−extern void pgdat_page_ext_init(struct pglist_data *pgdat);
59	58
60	59	static inline bool early_page_ext_enabled(void)
61	60	{
@@ -202,10 +201,6 @@ static inline bool early_page_ext_enabled(void) · 140 unmodified lines
202	201	return false;
203	202	}
204	203
205		−static inline void pgdat_page_ext_init(struct pglist_data *pgdat)
206		−{
207		−}
208		−
209	204	static inline void page_ext_init(void)
210	205	{
211	206	}

include/linux/pagemap.h+37−3

@@ -1094,10 +1094,44 @@ static inline pgoff_t linear_page_delta(const struct vm_area_struct *vma, · 1093 unmodified lines
1094	1094	static inline pgoff_t linear_page_index(const struct vm_area_struct *vma,
1095	1095	const unsigned long address)
1096	1096	{
1097		− pgoff_t pgoff;
	1097	+ return linear_page_delta(vma, address) + vma_start_pgoff(vma);
	1098	+}
	1099	+
	1100	+static inline pgoff_t __linear_anon_page_index(const struct vm_area_struct *vma,
	1101	+ const unsigned long address)
	1102	+{
	1103	+ return linear_page_delta(vma, address) + vma_start_anon_pgoff(vma);
	1104	+}
	1105	+
	1106	+/**
	1107	+ * linear_anon_page_index() - Determine the absolute anonymous page offset of
	1108	+ * @address within @vma.
	1109	+ * @vma: An anonymous or MAP_PRIVATE file-backed VMA in which @address resides.
	1110	+ * @address: The address whose absolute page offset is required.
	1111	+ *
	1112	+ * This returns the anonymous page offset of @address, which is the page offset
	1113	+ * the address possessed at the time the VMA was first faulted.
	1114	+ *
	1115	+ * For anonymous mappings, this returns the same value as linear_page_index().
	1116	+ *
	1117	+ * For MAP_PRIVATE file-backed mappings, this returns the anonymous page offset
	1118	+ * of @address, which is the page offset the address possessed at the time the
	1119	+ * VMA was first faulted.
	1120	+ *
	1121	+ * It is not valid to call this function for shared file-backed mappings.
	1122	+ *
	1123	+ * Returns: The absolute anonymous page offset of @address within @vma.
	1124	+ */
	1125	+static inline pgoff_t linear_anon_page_index(const struct vm_area_struct *vma,
	1126	+ const unsigned long address)
	1127	+{
	1128	+ const pgoff_t pgoff = __linear_anon_page_index(vma, address);
	1129	+
	1130	+ VM_WARN_ON_ONCE(!vma_is_cow_mapping(vma));
	1131	+ /* Account for MAP_PRIVATE-/dev/zero which is only semi-anonymous. */
	1132	+ if (vma_is_anonymous(vma) && !vma->vm_file)
	1133	+ VM_WARN_ON_ONCE(pgoff != linear_page_index(vma, address));
1098	1134
1099		− pgoff = linear_page_delta(vma, address);
1100		− pgoff += vma_start_pgoff(vma);
1101	1135	return pgoff;
1102	1136	}
1103	1137

include/linux/percpu-defs.h+1−6

@@ -65,13 +65,8 @@ · 64 unmodified lines
65	65	*
66	66	* Archs which need weak percpu definitions should set
67	67	* CONFIG_ARCH_MODULE_NEEDS_WEAK_PER_CPU when necessary.
68		− *
69		− * To ensure that the generic code observes the above two
70		− * restrictions, if CONFIG_DEBUG_FORCE_WEAK_PER_CPU is set weak
71		− * definition is used for all cases.
72	68	*/
73		−#if (defined(CONFIG_ARCH_MODULE_NEEDS_WEAK_PER_CPU) && defined(MODULE)) \|\| \
74		− defined(CONFIG_DEBUG_FORCE_WEAK_PER_CPU)
	69	+#if defined(CONFIG_ARCH_MODULE_NEEDS_WEAK_PER_CPU) && defined(MODULE)
75	70	/*
76	71	* __pcpu_scope_* dummy variable is used to enforce scope. It
77	72	* receives the static modifier when it's used in front of

include/linux/rmap.h+6−5

@@ -843,7 +843,7 @@ static inline int folio_try_share_anon_rmap_pmd(struct folio *folio, · 842 unmodified lines
843	843	* Called from mm/vmscan.c to handle paging out
844	844	*/
845	845	int folio_referenced(struct folio *, int is_locked,
846		− struct mem_cgroup memcg, vm_flags_t vm_flags);
	846	+ struct mem_cgroup memcg, vma_flags_t vma_flags);
847	847
848	848	void try_to_migrate(struct folio *folio, enum ttu_flags flags);
849	849	void try_to_unmap(struct folio *, enum ttu_flags flags);
@@ -864,13 +864,14 @@ struct page make_device_exclusive(struct mm_struct mm, unsigned long addr, · 14 unmodified lines
864	864	struct page_vma_mapped_walk {
865	865	unsigned long pfn;
866	866	unsigned long nr_pages;
867		− pgoff_t pgoff;
	867	+ pgoff_t pgoff; /* Only meaningful if nr_pages > 1 and not a KSM walk */
868	868	struct vm_area_struct *vma;
869	869	unsigned long address;
870	870	pmd_t *pmd;
871	871	pte_t *pte;
872	872	spinlock_t *ptl;
873	873	unsigned int flags;
	874	+ bool pgoff_is_anon : 1;
874	875	};
875	876
876	877	#define DEFINE_FOLIO_VMA_WALK(name, _folio, _vma, _address, _flags) \
@@ -881,6 +882,7 @@ struct page_vma_mapped_walk { · 4 unmodified lines
881	882	.vma = _vma, \
882	883	.address = _address, \
883	884	.flags = _flags, \
	885	+ .pgoff_is_anon = folio_test_anon(_folio), \
884	886	}
885	887
886	888	static inline void page_vma_mapped_walk_done(struct page_vma_mapped_walk *pvmw)
@@ -975,10 +977,9 @@ struct anon_vma folio_lock_anon_vma_read(const struct folio folio, · 88 unmodified lines
975	977	#define anon_vma_prepare(vma) (0)
976	978
977	979	static inline int folio_referenced(struct folio *folio, int is_locked,
978		− struct mem_cgroup *memcg,
979		− vm_flags_t *vm_flags)
	980	+ struct mem_cgroup memcg, vma_flags_t vma_flags)
980	981	{
981		− *vm_flags = 0;
	982	+ vma_flags_clear_all(vma_flags);
982	983	return 0;
983	984	}
984	985

include/linux/sched.h+1−0

@@ -1298,6 +1298,7 @@ struct task_struct { · 1297 unmodified lines
1298	1298	u64 curr_chain_key;
1299	1299	int lockdep_depth;
1300	1300	unsigned int lockdep_recursion;
	1301	+ unsigned int lockdep_seq;
1301	1302	struct held_lock held_locks[MAX_LOCK_DEPTH];
1302	1303	#endif
1303	1304

include/linux/shmem_fs.h+1−4

@@ -12,8 +12,6 @@ · 11 unmodified lines
12	12	#include <linux/userfaultfd_k.h>
13	13	#include <linux/bits.h>
14	14
15		−struct swap_iocb;
16		−
17	15	/* inode in-kernel data */
18	16
19	17	#ifdef CONFIG_TMPFS_QUOTA
@@ -123,8 +121,7 @@ static inline bool shmem_mapping(const struct address_space *mapping) · 103 unmodified lines
123	121	void shmem_unlock_mapping(struct address_space *mapping);
124	122	struct page shmem_read_mapping_page_gfp(struct address_space mapping,
125	123	pgoff_t index, gfp_t gfp_mask);
126		−int shmem_writeout(struct folio folio, struct swap_iocb *plug,
127		− struct list_head *folio_list);
	124	+int shmem_write_folio(struct folio *folio);
128	125	void shmem_truncate_range(struct inode *inode, loff_t start, uoff_t end);
129	126	int shmem_unuse(unsigned int type);
130	127

include/linux/swap.h+9−21

@@ -202,7 +202,6 @@ enum { · 201 unmodified lines
202	202	SWP_SOLIDSTATE = (1 << 4), /* blkdev seeks are cheap */
203	203	SWP_BLKDEV = (1 << 6), /* its a block device */
204	204	SWP_ACTIVATED = (1 << 7), /* set after swap_activate success */
205		− SWP_FS_OPS = (1 << 8), /* swapfile operations go through fs */
206	205	SWP_AREA_DISCARD = (1 << 9), /* single-time swap area discards */
207	206	SWP_PAGE_DISCARD = (1 << 10), /* freed swap page-cluster discards */
208	207	SWP_STABLE_WRITES = (1 << 11), /* no overwrite PG_writeback pages */
@@ -276,6 +275,7 @@ struct swap_info_struct { · 67 unmodified lines
276	275	struct work_struct reclaim_work; /* reclaim worker */
277	276	struct list_head discard_clusters; /* discard clusters list */
278	277	struct plist_node avail_list; /* entry in swap_avail_head */
	278	+ const struct swap_ops *ops;
279	279	};
280	280
281	281	static inline swp_entry_t page_swap_entry(struct page *page)
@@ -298,6 +298,14 @@ void folio_add_lru(struct folio *folio); · 16 unmodified lines
298	298	void folio_mark_accessed(struct folio *folio);
299	299	void lru_add_drain_all(void);
300	300
	301	+enum lru_cache_drained {
	302	+ LRU_CACHE_NOT_DRAINED,
	303	+ LRU_CACHE_DRAINED,
	304	+ LRU_CACHE_DRAINED_ALL,
	305	+};
	306	+void lru_cache_drain_for_folio(const struct folio *folio,
	307	+ unsigned int extra_refs, enum lru_cache_drained *drained);
	308	+
301	309	/* linux/mm/folio-compat.c */
302	310	void mark_page_accessed(struct page *page);
303	311
@@ -309,7 +317,6 @@ static inline bool lru_cache_disabled(void) · 5 unmodified lines
309	317	}
310	318
311	319	extern unsigned long shrink_all_memory(unsigned long nr_pages);
312		−extern int vm_swappiness;
313	320	long remove_mapping(struct address_space mapping, struct folio folio);
314	321
315	322	#if defined(CONFIG_SYSFS) && defined(CONFIG_NUMA)
@@ -334,7 +341,6 @@ extern void __meminit kswapd_run(int nid); · 18 unmodified lines
334	341	extern void __meminit kswapd_stop(int nid);
335	342
336	343	#ifdef CONFIG_SWAP
337		−
338	344	int add_swap_extent(struct swap_info_struct *sis, unsigned long start_page,
339	345	unsigned long nr_pages, sector_t start_block);
340	346	int generic_swapfile_activate(struct swap_info_struct , struct file ,
@@ -468,25 +474,7 @@ static inline int add_swap_extent(struct swap_info_struct *sis, · 127 unmodified lines
468	474	}
469	475	#endif /* CONFIG_SWAP */
470	476	#ifdef CONFIG_MEMCG
471		−static inline int mem_cgroup_swappiness(struct mem_cgroup *memcg)
472		−{
473		− /* Cgroup2 doesn't have per-cgroup swappiness */
474		− if (cgroup_subsys_on_dfl(memory_cgrp_subsys))
475		− return READ_ONCE(vm_swappiness);
476		−
477		− /* root ? */
478		− if (mem_cgroup_disabled() \|\| mem_cgroup_is_root(memcg))
479		− return READ_ONCE(vm_swappiness);
480		−
481		− return READ_ONCE(memcg->swappiness);
482		−}
483		−
484	477	void lru_reparent_memcg(struct mem_cgroup memcg, struct mem_cgroup parent, int nid);
485		−#else
486		−static inline int mem_cgroup_swappiness(struct mem_cgroup *memcg)
487		−{
488		− return READ_ONCE(vm_swappiness);
489		−}
490	478	#endif
491	479
492	480	#if defined(CONFIG_SWAP) && defined(CONFIG_MEMCG) && defined(CONFIG_BLK_CGROUP)

include/linux/swap_ops.h+44−0

@@ -0,0 +1,44 @@
	1	+/* SPDX-License-Identifier: GPL-2.0 */
	2	+#ifndef _MM_SWAP_OPS_H
	3	+#define _MM_SWAP_OPS_H
	4	+
	5	+#include <linux/swap.h> /* for SWAP_CLUSTER_MAX */
	6	+
	7	+struct swap_iocb {
	8	+ union {
	9	+ struct kiocb iocb;
	10	+ struct bio bio;
	11	+ };
	12	+ struct bio_vec bvecs[SWAP_CLUSTER_MAX];
	13	+ int nr_bvecs;
	14	+ int len;
	15	+};
	16	+
	17	+struct swap_io_ctx {
	18	+ struct swap_iocb *sio;
	19	+ struct swap_info_struct *sis;
	20	+};
	21	+
	22	+/*
	23	+ * SWAP_OPS_F_REQUIRE_NOFS:
	24	+ * When set, all reclaim operations must operated as GFS_NOFS and not
	25	+ * just GFP_NOIO, as GFP_NOIO allocations could recourse into the
	26	+ * file system backing this swap file.
	27	+ */
	28	+#define SWAP_OPS_F_REQUIRE_NOFS (1U << 0)
	29	+
	30	+struct swap_ops {
	31	+ unsigned int flags;
	32	+
	33	+ bool (can_merge)(struct folio folio, struct folio *prev_folio,
	34	+ size_t prev_folio_size, int rw);
	35	+ void (submit_write)(struct swap_io_ctx ctx);
	36	+ void (submit_read)(struct swap_io_ctx ctx);
	37	+};
	38	+
	39	+void swap_fs_prepare_rw(struct swap_io_ctx ctx, int rw, struct iov_iter iter);
	40	+bool swap_fs_can_merge(struct folio folio, struct folio prev_folio,
	41	+ size_t prev_folio_size, int rw);
	42	+int swap_fs_activate(struct swap_info_struct sis, const struct swap_ops ops);
	43	+
	44	+#endif /* _MM_SWAP_OPS_H */

include/linux/swapops.h+4−3

@@ -5,6 +5,7 @@ · 4 unmodified lines
5	5	#include <linux/radix-tree.h>
6	6	#include <linux/bug.h>
7	7	#include <linux/mm_types.h>
	8	+#include <linux/swap.h>
8	9
9	10	#ifdef CONFIG_MMU
10	11
@@ -325,8 +326,8 @@ struct page_vma_mapped_walk; · 314 unmodified lines
325	326	extern int set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw,
326	327	struct page *page);
327	328
328		−extern void remove_migration_pmd(struct page_vma_mapped_walk *pvmw,
329		− struct page *new);
	329	+void remove_migration_pmd(struct page_vma_mapped_walk *pvmw,
	330	+ struct folio *folio);
330	331
331	332	extern void pmd_migration_entry_wait(struct mm_struct mm, pmd_t pmd);
332	333
@@ -346,7 +347,7 @@ static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw, · 13 unmodified lines
346	347	}
347	348
348	349	static inline void remove_migration_pmd(struct page_vma_mapped_walk *pvmw,
349		− struct page *new)
	350	+ struct folio *folio)
350	351	{
351	352	BUILD_BUG();
352	353	}

include/linux/vm_event_item.h+4−0

@@ -175,6 +175,10 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, · 174 unmodified lines
175	175	KSTACK_REST,
176	176	#endif
177	177	#endif /* CONFIG_DEBUG_STACK_USAGE */
	178	+#ifdef CONFIG_SWAP
	179	+ NRSWPIN,
	180	+ NRSWPOUT,
	181	+#endif /* CONFIG_SWAP */
178	182	NR_VM_EVENT_ITEMS
179	183	};
180	184

include/linux/vmalloc.h+1−1

@@ -62,7 +62,7 @@ struct vm_struct { · 61 unmodified lines
62	62	#ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC
63	63	unsigned int page_order;
64	64	#endif
65		− unsigned int nr_pages;
	65	+ unsigned long nr_pages;
66	66	phys_addr_t phys_addr;
67	67	const void *caller;
68	68	unsigned long requested_size;

include/linux/vmstat.h+16−1

@@ -20,7 +20,6 @@ struct reclaim_stat { · 19 unmodified lines
20	20	unsigned nr_congested;
21	21	unsigned nr_writeback;
22	22	unsigned nr_immediate;
23		− unsigned nr_pageout;
24	23	unsigned nr_activate[ANON_AND_FILE];
25	24	unsigned nr_ref_keep;
26	25	unsigned nr_unmap_fail;
@@ -194,6 +193,19 @@ unsigned long global_node_page_state_pages(enum node_stat_item item) · 167 unmodified lines
194	193	return x;
195	194	}
196	195
	196	+/*
	197	+ * Non-clamping variant of global_node_page_state() intended for callers that
	198	+ * snapshot a monotonically-incremented counter and subtract two samples.
	199	+ * Returns the raw wrapping value so that unsigned modular subtraction stays
	200	+ * correct across a signed-long overflow (a real hazard on 32-bit) that the
	201	+ * clamp in global_node_page_state() would otherwise turn into a huge spurious
	202	+ * delta. Do NOT use for non-monotonic page-count reads.
	203	+ */
	204	+static inline unsigned long global_node_page_state_monotonic(enum node_stat_item item)
	205	+{
	206	+ return (unsigned long)atomic_long_read(&vm_node_stat[item]);
	207	+}
	208	+
197	209	static inline unsigned long global_node_page_state(enum node_stat_item item)
198	210	{
199	211	VM_WARN_ON_ONCE(vmstat_item_in_bytes(item));
@@ -259,11 +271,14 @@ extern unsigned long node_page_state(struct pglist_data *pgdat, · 59 unmodified lines
259	271	enum node_stat_item item);
260	272	extern unsigned long node_page_state_pages(struct pglist_data *pgdat,
261	273	enum node_stat_item item);
	274	+extern unsigned long node_page_state_monotonic(struct pglist_data *pgdat,
	275	+ enum node_stat_item item);
262	276	extern void fold_vm_numa_events(void);
263	277	#else
264	278	#define sum_zone_node_page_state(node, item) global_zone_page_state(item)
265	279	#define node_page_state(node, item) global_node_page_state(item)
266	280	#define node_page_state_pages(node, item) global_node_page_state_pages(item)
	281	+#define node_page_state_monotonic(node, item) global_node_page_state_monotonic(item)
267	282	static inline void fold_vm_numa_events(void)
268	283	{
269	284	}

include/rv/da_monitor.h+1−1

@@ -25,7 +25,7 @@ · 24 unmodified lines
25	25
26	26	/*
27	27	* Per-cpu variables require a unique name although static in some
28		− * configurations (e.g. CONFIG_DEBUG_FORCE_WEAK_PER_CPU or alpha modules).
	28	+ * configurations (e.g. alpha modules).
29	29	*/
30	30	#define DA_MON_NAME CONCATENATE(da_mon_, MONITOR_NAME)
31	31

include/uapi/linux/alloc_tag.h+99−0

@@ -0,0 +1,99 @@
	1	+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
	2	+/*
	3	+ * alloc_tag IOCTL API definition
	4	+ *
	5	+ * Copyright (C) 2026 Google, LLC. All rights reserved.
	6	+ *
	7	+ * This program is free software; you can redistribute it and/or modify
	8	+ * it under the terms of the GNU General Public License version 2 as
	9	+ * published by the Free Software Foundation.
	10	+ */
	11	+
	12	+#ifndef _UAPI_ALLOC_TAG_H
	13	+#define _UAPI_ALLOC_TAG_H
	14	+
	15	+#include <linux/types.h>
	16	+
	17	+/*
	18	+ * Function, file and module names often have the same prefixes, therefore
	19	+ * when filtering by these criteria, we compare the last 64 characters to
	20	+ * minimize the chances of name collisions
	21	+ */
	22	+#define ALLOCINFO_STR_SIZE 64
	23	+
	24	+struct allocinfo_content_id {
	25	+ __u64 id;
	26	+};
	27	+
	28	+struct allocinfo_tag {
	29	+ /* Longer names are trimmed */
	30	+ char modname[ALLOCINFO_STR_SIZE];
	31	+ char function[ALLOCINFO_STR_SIZE];
	32	+ char filename[ALLOCINFO_STR_SIZE];
	33	+ __u64 lineno;
	34	+};
	35	+
	36	+/* The alignment ensures 32-bit compatible interfaces are not broken */
	37	+struct allocinfo_counter {
	38	+ __u64 bytes;
	39	+ __u64 calls;
	40	+ __u8 accurate;
	41	+} __attribute__((aligned(8)));
	42	+
	43	+struct allocinfo_tag_data {
	44	+ struct allocinfo_tag tag;
	45	+ struct allocinfo_counter counter;
	46	+};
	47	+
	48	+enum {
	49	+ ALLOCINFO_FILTER_MODNAME,
	50	+ ALLOCINFO_FILTER_FUNCTION,
	51	+ ALLOCINFO_FILTER_FILENAME,
	52	+ ALLOCINFO_FILTER_LINENO,
	53	+ ALLOCINFO_FILTER_INACCURATE,
	54	+ ALLOCINFO_FILTER_MIN_SIZE,
	55	+ ALLOCINFO_FILTER_MAX_SIZE,
	56	+ __ALLOCINFO_FILTER_LAST = ALLOCINFO_FILTER_MAX_SIZE
	57	+};
	58	+
	59	+#define ALLOCINFO_FILTER_MASK_MODNAME (1 << ALLOCINFO_FILTER_MODNAME)
	60	+#define ALLOCINFO_FILTER_MASK_FUNCTION (1 << ALLOCINFO_FILTER_FUNCTION)
	61	+#define ALLOCINFO_FILTER_MASK_FILENAME (1 << ALLOCINFO_FILTER_FILENAME)
	62	+#define ALLOCINFO_FILTER_MASK_LINENO (1 << ALLOCINFO_FILTER_LINENO)
	63	+#define ALLOCINFO_FILTER_MASK_INACCURATE (1 << ALLOCINFO_FILTER_INACCURATE)
	64	+#define ALLOCINFO_FILTER_MASK_MIN_SIZE (1 << ALLOCINFO_FILTER_MIN_SIZE)
	65	+#define ALLOCINFO_FILTER_MASK_MAX_SIZE (1 << ALLOCINFO_FILTER_MAX_SIZE)
	66	+
	67	+#define ALLOCINFO_FILTER_MASKS \
	68	+ ((1 << (__ALLOCINFO_FILTER_LAST + 1)) - 1)
	69	+
	70	+struct allocinfo_filter {
	71	+ __u64 mask; /* bitmask of the filter fields used */
	72	+ struct allocinfo_tag fields;
	73	+ __u64 min_size;
	74	+ __u64 max_size;
	75	+ /* filter criteria only; see allocinfo_counter.accurate for actual accuracy */
	76	+ __u64 inaccurate;
	77	+};
	78	+
	79	+struct allocinfo_get_at {
	80	+ /* inputs */
	81	+ __u64 pos;
	82	+ struct allocinfo_filter filter;
	83	+ /* output */
	84	+ struct allocinfo_tag_data data;
	85	+};
	86	+
	87	+#define _ALLOCINFO_IOC_CONTENT_ID 0
	88	+#define _ALLOCINFO_IOC_GET_AT 1
	89	+#define _ALLOCINFO_IOC_GET_NEXT 2
	90	+
	91	+#define ALLOCINFO_IOC_BASE 0xA6
	92	+#define ALLOCINFO_IOC_CONTENT_ID _IOR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_CONTENT_ID, \
	93	+ struct allocinfo_content_id)
	94	+#define ALLOCINFO_IOC_GET_AT _IOWR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_GET_AT, \
	95	+ struct allocinfo_get_at)
	96	+#define ALLOCINFO_IOC_GET_NEXT _IOR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_GET_NEXT, \
	97	+ struct allocinfo_tag_data)
	98	+
	99	+#endif /* _UAPI_ALLOC_TAG_H */