mm: folio_may_be_lru_cached() unless folio_test_large()

mm/swap.c and mm/mlock.c agree to drain any per-CPU batch as soon as a large folio is added: so collect_longterm_unpinnable_folios() just wastes effort when calling lru_add_drain[_all]() on a large folio. But although there is good reason not to batch up PMD-sized folios, we might well benefit from batching a small number of low-order mTHPs (though unclear how that "small number" limitation will be implemented). So ask if folio_may_be_lru_cached() rather than !folio_test_large(), to insulate those particular checks from future change. Name preferred to "folio_is_batchable" because large folios can well be put on a batch: it's just the per-CPU LRU caches, drained much later, which need care. Marked for stable, to counter the increase in lru_add_drain_all()s from "mm/gup: check ref_count instead of lru before migration".

Hugh Dickins committed 1 year ago

Commit 2da6de31 parent 8d79ed34 files · +16 −6Browse files

Showing the changes to include (1 of 4 files). Show all

Unified · Split

include/linux/swap.h+10−0

@@ -385,6 +385,16 @@ void folio_add_lru_vma(struct folio , struct vm_area_struct ); · 384 unmodified lines
385	385	void mark_page_accessed(struct page *);
386	386	void folio_mark_accessed(struct folio *);
387	387
	388	+static inline bool folio_may_be_lru_cached(struct folio *folio)
	389	+{
	390	+ /*
	391	+ * Holding PMD-sized folios in per-CPU LRU cache unbalances accounting.
	392	+ * Holding small numbers of low-order mTHP folios in per-CPU LRU cache
	393	+ * will be sensible, but nobody has implemented and tested that yet.
	394	+ */
	395	+ return !folio_test_large(folio);
	396	+}
	397	+
388	398	extern atomic_t lru_disable_count;
389	399
390	400	static inline bool lru_cache_disabled(void)