Merge tag 'docs-4.18' of git://git.lwn.net/linux

Pull documentation updates from Jonathan Corbet: "There's been a fair amount of work in the docs tree this time around, including: - Extensive RST conversions and organizational work in the memory-management docs thanks to Mike Rapoport. - An update of Documentation/features from Andrea Parri and a script to keep it updated. - Various LICENSES updates from Thomas, along with a script to check SPDX tags. - Work to fix dangling references to documentation files; this involved a fair number of one-liner comment changes outside of Documentation/ ... and the usual list of documentation improvements, typo fixes, etc" * tag 'docs-4.18' of git://git.lwn.net/linux: (103 commits) Documentation: document hung_task_panic kernel parameter docs/admin-guide/mm: add high level concepts overview docs/vm: move ksm and transhuge from "user" to "internals" section. docs: Use the kerneldoc comments for memalloc_no*() doc: document scope NOFS, NOIO APIs docs: update kernel versions and dates in tables docs/vm: transhuge: split userspace bits to admin-guide/mm/transhuge docs/vm: transhuge: minor updates docs/vm: transhuge: change sections order Documentation: arm: clean up Marvell Berlin family info Documentation: gpio: driver: Fix a typo and some odd grammar docs: ranoops.rst: fix location of ramoops.txt scripts/documentation-file-ref-check: rewrite it in perl with auto-fix mode docs: uio-howto.rst: use a code block to solve a warning mm, THP, doc: Add document for thp_swpout/thp_swpout_fallback w1: w1_io.c: fix a kernel-doc warning Documentation/process/posting: wrap text at 80 cols docs: admin-guide: add cgroup-v2 documentation Revert "Documentation/features/vm: Remove arch support status file for 'pte_special'" Documentation: refcount-vs-atomic: Update reference to LKMM doc. ...

Linus Torvalds committed 8 years ago

Commit eeee3142 parents c5e7a7e a49d9c0166 files · +5535 −2920Browse files

Unified · Split

166 files

Documentation/00-INDEX +0 −10
Documentation/ABI/stable/sysfs-devices-node +1 −1
Documentation/ABI/testing/sysfs-kernel-mm-hugepages +1 −1
Documentation/ABI/testing/sysfs-kernel-mm-ksm +1 −1
Documentation/ABI/testing/sysfs-kernel-slab +2 −2
Documentation/admin-guide/bcache.rst +0 −0
Documentation/admin-guide/cgroup-v2.rst +0 −0
Documentation/admin-guide/index.rst +3 −0
Documentation/admin-guide/kernel-parameters.txt +85 −75
Documentation/admin-guide/mm/concepts.rst +222 −0
Documentation/admin-guide/mm/hugetlbpage.rst +147 −116
Documentation/admin-guide/mm/idle_page_tracking.rst +37 −19
Documentation/admin-guide/mm/index.rst +36 −0
Documentation/admin-guide/mm/ksm.rst +189 −0
Documentation/admin-guide/mm/numa_memory_policy.rst +495 −0
Documentation/admin-guide/mm/pagemap.rst +101 −83
Documentation/admin-guide/mm/soft-dirty.rst +12 −8
Documentation/admin-guide/mm/transhuge.rst +418 −0
Documentation/admin-guide/mm/userfaultfd.rst +39 −27
Documentation/admin-guide/ramoops.rst +1 −1
Documentation/arm/Marvell/README +7 −8
Documentation/block/cmdline-partition.txt +14 −7
Documentation/core-api/cachetlb.rst +0 −0
Documentation/core-api/circular-buffers.rst +0 −0
Documentation/core-api/gfp_mask-from-fs-io.rst +66 −0
Documentation/core-api/index.rst +3 −0
Documentation/core-api/kernel-api.rst +30 −30
Documentation/core-api/refcount-vs-atomic.rst +1 −1
Documentation/crypto/index.rst +1 −0
Documentation/dev-tools/kasan.rst +1 −1
Documentation/dev-tools/kselftest.rst +5 −0
Documentation/driver-api/clk.rst +0 −0
Documentation/driver-api/device_connection.rst +1 −1
Documentation/driver-api/gpio/driver.rst +3 −3
Documentation/driver-api/index.rst +2 −0
Documentation/driver-api/uio-howto.rst +2 −1
Documentation/features/core/cBPF-JIT/arch-support.txt +8 −6
Documentation/features/core/eBPF-JIT/arch-support.txt +5 −3
Documentation/features/core/generic-idle-thread/arch-support.txt +3 −1
Documentation/features/core/jump-labels/arch-support.txt +2 −0
Documentation/features/core/tracehook/arch-support.txt +2 −0
Documentation/features/debug/KASAN/arch-support.txt +3 −1
Documentation/features/debug/gcov-profile-all/arch-support.txt +2 −0
Documentation/features/debug/kgdb/arch-support.txt +3 −1
Documentation/features/debug/kprobes-on-ftrace/arch-support.txt +2 −0
Documentation/features/debug/kprobes/arch-support.txt +3 −1
Documentation/features/debug/kretprobes/arch-support.txt +3 −1
Documentation/features/debug/optprobes/arch-support.txt +3 −1
Documentation/features/debug/stackprotector/arch-support.txt +2 −0
Documentation/features/debug/uprobes/arch-support.txt +4 −2
Documentation/features/debug/user-ret-profiler/arch-support.txt +2 −0
Documentation/features/io/dma-contiguous/arch-support.txt +3 −1
Documentation/features/io/sg-chain/arch-support.txt +2 −0
Documentation/features/locking/cmpxchg-local/arch-support.txt +3 −1
Documentation/features/locking/lockdep/arch-support.txt +3 −1
Documentation/features/locking/queued-rwlocks/arch-support.txt +6 −4
Documentation/features/locking/queued-spinlocks/arch-support.txt +5 −3
Documentation/features/locking/rwsem-optimized/arch-support.txt +6 −4
Documentation/features/perf/kprobes-event/arch-support.txt +4 −2
Documentation/features/perf/perf-regs/arch-support.txt +3 −1
Documentation/features/perf/perf-stackdump/arch-support.txt +3 −1
Documentation/features/sched/membarrier-sync-core/arch-support.txt +2 −0
Documentation/features/sched/numa-balancing/arch-support.txt +4 −2
Documentation/features/scripts/features-refresh.sh +98 −0
Documentation/features/seccomp/seccomp-filter/arch-support.txt +4 −2
Documentation/features/time/arch-tick-broadcast/arch-support.txt +3 −1
Documentation/features/time/clockevents/arch-support.txt +3 −1
Documentation/features/time/context-tracking/arch-support.txt +2 −0
Documentation/features/time/irq-time-acct/arch-support.txt +3 −1
Documentation/features/time/modern-timekeeping/arch-support.txt +2 −0
Documentation/features/time/virt-cpuacct/arch-support.txt +2 −0
Documentation/features/vm/ELF-ASLR/arch-support.txt +3 −1
Documentation/features/vm/PG_uncached/arch-support.txt +2 −0
Documentation/features/vm/THP/arch-support.txt +2 −0
Documentation/features/vm/TLB/arch-support.txt +2 −0
Documentation/features/vm/huge-vmap/arch-support.txt +2 −0
Documentation/features/vm/ioremap_prot/arch-support.txt +2 −0
Documentation/features/vm/numa-memblock/arch-support.txt +3 −1
Documentation/features/vm/pte_special/arch-support.txt +2 −0
Documentation/filesystems/proc.txt +5 −3
Documentation/filesystems/tmpfs.txt +3 −2
Documentation/index.rst +2 −1
Documentation/ioctl/botching-up-ioctls.txt +3 −1
Documentation/memory-barriers.txt +2 −2
Documentation/process/2.Process.rst +38 −34
Documentation/process/5.Posting.rst +8 −8
Documentation/process/index.rst +1 −0
Documentation/process/maintainer-pgp-guide.rst +37 −2
Documentation/process/submitting-patches.rst +1 −1
Documentation/security/index.rst +2 −0
Documentation/sound/alsa-configuration.rst +2 −2
Documentation/sound/soc/codec.rst +1 −1
Documentation/sound/soc/platform.rst +1 −1
Documentation/sysctl/vm.txt +3 −3
Documentation/trace/coresight.txt +75 −28
Documentation/trace/ftrace-uses.rst +2 −2
Documentation/trace/ftrace.rst +6 −1
Documentation/translations/ko_KR/memory-barriers.txt +2 −2
Documentation/vfio.txt +2 −3
Documentation/vm/00-INDEX +23 −35
Documentation/vm/active_mm.rst +91 −0
Documentation/vm/active_mm.txt +0 −83
Documentation/vm/balance.rst +11 −4
Documentation/vm/cleancache.rst +62 −43
Documentation/vm/conf.py +10 −0
Documentation/vm/frontswap.rst +37 −22
Documentation/vm/highmem.rst +36 −51
Documentation/vm/hmm.rst +34 −44
Documentation/vm/hugetlbfs_reserv.rst +143 −77
Documentation/vm/hwpoison.rst +70 −71
Documentation/vm/index.rst +50 −0
Documentation/vm/ksm.rst +87 −0
Documentation/vm/ksm.txt +0 −178
Documentation/vm/mmu_notifier.rst +99 −0
Documentation/vm/mmu_notifier.txt +0 −93
Documentation/vm/numa.rst +5 −1
Documentation/vm/numa_memory_policy.txt +0 −452
Documentation/vm/overcommit-accounting +0 −80
Documentation/vm/overcommit-accounting.rst +87 −0
Documentation/vm/page_frags.rst +4 −1
Documentation/vm/page_migration.rst +77 −72
Documentation/vm/page_owner.rst +21 −13
Documentation/vm/remap_file_pages.rst +6 −0
Documentation/vm/slub.rst +361 −0
Documentation/vm/slub.txt +0 −342
Documentation/vm/split_page_table_lock.rst +9 −3
Documentation/vm/swap_numa.rst +33 −22
Documentation/vm/transhuge.rst +197 −0
Documentation/vm/transhuge.txt +0 −527
Documentation/vm/unevictable-lru.rst +49 −68
Documentation/vm/z3fold.rst +5 −1
Documentation/vm/zsmalloc.rst +36 −24
Documentation/vm/zswap.rst +42 −29
LICENSES/exceptions/Linux-syscall-note +1 −1
LICENSES/other/Apache-2.0 +183 −0
LICENSES/other/CC-BY-SA-4.0 +397 −0
LICENSES/other/CDDL-1.0 +364 −0
LICENSES/other/Linux-OpenIB +26 −0
LICENSES/other/X11 +37 −0
LICENSES/preferred/GPL-2.0 +6 −0
MAINTAINERS +1 −1
arch/alpha/Kconfig +1 −1
arch/ia64/Kconfig +1 −1
arch/mips/Kconfig +1 −1
arch/powerpc/Kconfig +1 −1
drivers/w1/w1_io.c +1 −0
fs/Kconfig +1 −1
fs/dax.c +1 −1
fs/proc/task_mmu.c +2 −2
include/linux/hmm.h +1 −1
include/linux/memremap.h +2 −2
include/linux/mmu_notifier.h +1 −1
include/linux/sched/mm.h +40 −2
include/linux/swap.h +1 −1
mm/Kconfig +4 −3
mm/cleancache.c +1 −1
mm/frontswap.c +1 −1
mm/hmm.c +1 −1
mm/huge_memory.c +2 −2
mm/hugetlb.c +2 −2
mm/ksm.c +20 −3
mm/mmap.c +1 −1
mm/rmap.c +3 −3
mm/util.c +1 −1
scripts/documentation-file-ref-check +113 −12
scripts/spdxcheck.py +284 −0

Documentation/00-INDEX+0−10

@@ -64,8 +64,6 @@ auxdisplay/ · 63 unmodified lines
64	64	- misc. LCD driver documentation (cfag12864b, ks0108).
65	65	backlight/
66	66	- directory with info on controlling backlights in flat panel displays
67		−bcache.txt
68		− - Block-layer cache on fast SSDs to improve slow (raid) I/O performance.
69	67	block/
70	68	- info on the Block I/O (BIO) layer.
71	69	blockdev/
@@ -78,18 +76,10 @@ bus-devices/ · 6 unmodified lines
78	76	- directory with info on TI GPMC (General Purpose Memory Controller)
79	77	bus-virt-phys-mapping.txt
80	78	- how to access I/O mapped memory from within device drivers.
81		−cachetlb.txt
82		− - describes the cache/TLB flushing interfaces Linux uses.
83	79	cdrom/
84	80	- directory with information on the CD-ROM drivers that Linux has.
85	81	cgroup-v1/
86	82	- cgroups v1 features, including cpusets and memory controller.
87		−cgroup-v2.txt
88		− - cgroups v2 features, including cpusets and memory controller.
89		−circular-buffers.txt
90		− - how to make use of the existing circular buffer infrastructure
91		−clk.txt
92		− - info on the common clock framework
93	83	cma/
94	84	- Continuous Memory Area (CMA) debugfs interface.
95	85	conf.py

Documentation/ABI/stable/sysfs-devices-node+1−1

@@ -90,4 +90,4 @@ Date: December 2009 · 89 unmodified lines
90	90	Contact: Lee Schermerhorn <lee.schermerhorn@hp.com>
91	91	Description:
92	92	The node's huge page size control/query attributes.
93		− See Documentation/vm/hugetlbpage.txt
	93	+ See Documentation/admin-guide/mm/hugetlbpage.rst

Documentation/ABI/testing/sysfs-kernel-mm-hugepages+1−1

@@ -12,4 +12,4 @@ Description: · 11 unmodified lines
12	12	free_hugepages
13	13	surplus_hugepages
14	14	resv_hugepages
15		− See Documentation/vm/hugetlbpage.txt for details.
	15	+ See Documentation/admin-guide/mm/hugetlbpage.rst for details.

Documentation/ABI/testing/sysfs-kernel-mm-ksm+1−1

@@ -40,7 +40,7 @@ Description: Kernel Samepage Merging daemon sysfs interface · 39 unmodified lines
40	40	sleep_millisecs: how many milliseconds ksm should sleep between
41	41	scans.
42	42
43		− See Documentation/vm/ksm.txt for more information.
	43	+ See Documentation/vm/ksm.rst for more information.
44	44
45	45	What: /sys/kernel/mm/ksm/merge_across_nodes
46	46	Date: January 2013

Documentation/ABI/testing/sysfs-kernel-slab+2−2

@@ -37,7 +37,7 @@ Description: · 36 unmodified lines
37	37	The alloc_calls file is read-only and lists the kernel code
38	38	locations from which allocations for this cache were performed.
39	39	The alloc_calls file only contains information if debugging is
40		− enabled for that cache (see Documentation/vm/slub.txt).
	40	+ enabled for that cache (see Documentation/vm/slub.rst).
41	41
42	42	What: /sys/kernel/slab/cache/alloc_fastpath
43	43	Date: February 2008
@@ -219,7 +219,7 @@ Contact: Pekka Enberg <penberg@cs.helsinki.fi>, · 175 unmodified lines
219	219	Description:
220	220	The free_calls file is read-only and lists the locations of
221	221	object frees if slab debugging is enabled (see
222		− Documentation/vm/slub.txt).
	222	+ Documentation/vm/slub.rst).
223	223
224	224	What: /sys/kernel/slab/cache/free_fastpath
225	225	Date: February 2008

Documentation/admin-guide/bcache.rstrenamed from Documentation/bcache.txt+0−0

File renamed without changes.

Documentation/admin-guide/cgroup-v2.rstrenamed from Documentation/cgroup-v2.txt+0−0

File renamed without changes.

Documentation/admin-guide/index.rst+3−0

@@ -48,6 +48,7 @@ configure specific aspects of kernel behavior to your liking. · 47 unmodified lines
48	48	:maxdepth: 1
49	49
50	50	initrd
	51	+ cgroup-v2
51	52	serial-console
52	53	braille-console
53	54	parport
@@ -60,9 +61,11 @@ configure specific aspects of kernel behavior to your liking. · 6 unmodified lines
60	61	mono
61	62	java
62	63	ras
	64	+ bcache
63	65	pm/index
64	66	thunderbolt
65	67	LSM/index
	68	+ mm/index
66	69
67	70	.. only:: subproject and html
68	71

Documentation/admin-guide/kernel-parameters.txt+85−75

@@ -106,11 +106,11 @@ · 105 unmodified lines
106	106	use by PCI
107	107	Format: <irq>,<irq>...
108	108
109		− acpi_mask_gpe= [HW,ACPI]
	109	+ acpi_mask_gpe= [HW,ACPI]
110	110	Due to the existence of _Lxx/_Exx, some GPEs triggered
111	111	by unsupported hardware/firmware features can result in
112		− GPE floodings that cannot be automatically disabled by
113		− the GPE dispatcher.
	112	+ GPE floodings that cannot be automatically disabled by
	113	+ the GPE dispatcher.
114	114	This facility can be used to prevent such uncontrolled
115	115	GPE floodings.
116	116	Format: <int>
@@ -472,10 +472,10 @@ · 355 unmodified lines
472	472	for platform specific values (SB1, Loongson3 and
473	473	others).
474	474
475		− ccw_timeout_log [S390]
	475	+ ccw_timeout_log [S390]
476	476	See Documentation/s390/CommonIO for details.
477	477
478		− cgroup_disable= [KNL] Disable a particular controller
	478	+ cgroup_disable= [KNL] Disable a particular controller
479	479	Format: {name of the controller(s) to disable}
480	480	The effects of cgroup_disable=foo are:
481	481	- foo isn't auto-mounted if you mount all cgroups in
@@ -518,7 +518,7 @@ · 36 unmodified lines
518	518	those clocks in any way. This parameter is useful for
519	519	debug and development, but should not be needed on a
520	520	platform with proper driver support. For more
521		− information, see Documentation/clk.txt.
	521	+ information, see Documentation/driver-api/clk.rst.
522	522
523	523	clock= [BUGS=X86-32, HW] gettimeofday clocksource override.
524	524	[Deprecated]
@@ -641,8 +641,8 @@ · 116 unmodified lines
641	641	hvc<n> Use the hypervisor console device <n>. This is for
642	642	both Xen and PowerPC hypervisors.
643	643
644		− If the device connected to the port is not a TTY but a braille
645		− device, prepend "brl," before the device type, for instance
	644	+ If the device connected to the port is not a TTY but a braille
	645	+ device, prepend "brl," before the device type, for instance
646	646	console=brl,ttyS0
647	647	For now, only VisioBraille is supported.
648	648
@@ -662,7 +662,7 @@ · 13 unmodified lines
662	662
663	663	consoleblank= [KNL] The console blank (screen saver) timeout in
664	664	seconds. A value of 0 disables the blank timer.
665		− Defaults to 0.
	665	+ Defaults to 0.
666	666
667	667	coredump_filter=
668	668	[KNL] Change the default value for
@@ -730,7 +730,7 @@ · 61 unmodified lines
730	730	or memory reserved is below 4G.
731	731
732	732	cryptomgr.notests
733		− [KNL] Disable crypto self-tests
	733	+ [KNL] Disable crypto self-tests
734	734
735	735	cs89x0_dma= [HW,NET]
736	736	Format: <dma>
@@ -746,7 +746,7 @@ · 9 unmodified lines
746	746	Format: <port#>,<type>
747	747	See also Documentation/input/devices/joystick-parport.rst
748	748
749		− ddebug_query= [KNL,DYNAMIC_DEBUG] Enable debug messages at early boot
	749	+ ddebug_query= [KNL,DYNAMIC_DEBUG] Enable debug messages at early boot
750	750	time. See
751	751	Documentation/admin-guide/dynamic-debug-howto.rst for
752	752	details. Deprecated, see dyndbg.
@@ -833,7 +833,7 @@ · 80 unmodified lines
833	833	causing system reset or hang due to sending
834	834	INIT from AP to BSP.
835	835
836		− disable_ddw [PPC/PSERIES]
	836	+ disable_ddw [PPC/PSERIES]
837	837	Disable Dynamic DMA Window support. Use this if
838	838	to workaround buggy firmware.
839	839
@@ -1188,7 +1188,7 @@ · 348 unmodified lines
1188	1188	parameter will force ia64_sal_cache_flush to call
1189	1189	ia64_pal_cache_flush instead of SAL_CACHE_FLUSH.
1190	1190
1191		− forcepae [X86-32]
	1191	+ forcepae [X86-32]
1192	1192	Forcefully enable Physical Address Extension (PAE).
1193	1193	Many Pentium M systems disable PAE but may have a
1194	1194	functionally usable PAE implementation.
@@ -1247,7 +1247,7 @@ · 52 unmodified lines
1247	1247
1248	1248	gamma= [HW,DRM]
1249	1249
1250		− gart_fix_e820= [X86_64] disable the fix e820 for K8 GART
	1250	+ gart_fix_e820= [X86_64] disable the fix e820 for K8 GART
1251	1251	Format: off \| on
1252	1252	default: on
1253	1253
@@ -1341,23 +1341,32 @@ · 87 unmodified lines
1341	1341	x86-64 are 2M (when the CPU supports "pse") and 1G
1342	1342	(when the CPU supports the "pdpe1gb" cpuinfo flag).
1343	1343
1344		− hvc_iucv= [S390] Number of z/VM IUCV hypervisor console (HVC)
1345		− terminal devices. Valid values: 0..8
1346		− hvc_iucv_allow= [S390] Comma-separated list of z/VM user IDs.
1347		− If specified, z/VM IUCV HVC accepts connections
1348		− from listed z/VM user IDs only.
	1344	+ hung_task_panic=
	1345	+ [KNL] Should the hung task detector generate panics.
	1346	+ Format: <integer>
1349	1347
	1348	+ A nonzero value instructs the kernel to panic when a
	1349	+ hung task is detected. The default value is controlled
	1350	+ by the CONFIG_BOOTPARAM_HUNG_TASK_PANIC build-time
	1351	+ option. The value selected by this boot parameter can
	1352	+ be changed later by the kernel.hung_task_panic sysctl.
	1353	+
	1354	+ hvc_iucv= [S390] Number of z/VM IUCV hypervisor console (HVC)
	1355	+ terminal devices. Valid values: 0..8
	1356	+ hvc_iucv_allow= [S390] Comma-separated list of z/VM user IDs.
	1357	+ If specified, z/VM IUCV HVC accepts connections
	1358	+ from listed z/VM user IDs only.
1350	1359	keep_bootcon [KNL]
1351	1360	Do not unregister boot console at start. This is only
1352	1361	useful for debugging when something happens in the window
1353	1362	between unregistering the boot console and initializing
1354	1363	the real console.
1355	1364
1356		− i2c_bus= [HW] Override the default board specific I2C bus speed
1357		− or register an additional I2C bus that is not
1358		− registered from board initialization code.
1359		− Format:
1360		− <bus_id>,<clkrate>
	1365	+ i2c_bus= [HW] Override the default board specific I2C bus speed
	1366	+ or register an additional I2C bus that is not
	1367	+ registered from board initialization code.
	1368	+ Format:
	1369	+ <bus_id>,<clkrate>
1361	1370
1362	1371	i8042.debug [HW] Toggle i8042 debug mode
1363	1372	i8042.unmask_kbd_data
@@ -1386,7 +1395,7 @@ · 22 unmodified lines
1386	1395	Default: only on s2r transitions on x86; most other
1387	1396	architectures force reset to be always executed
1388	1397	i8042.unlock [HW] Unlock (ignore) the keylock
1389		− i8042.kbdreset [HW] Reset device connected to KBD port
	1398	+ i8042.kbdreset [HW] Reset device connected to KBD port
1390	1399
1391	1400	i810= [HW,DRM]
1392	1401
@@ -1548,13 +1557,13 @@ · 155 unmodified lines
1548	1557	programs exec'd, files mmap'd for exec, and all files
1549	1558	opened for read by uid=0.
1550	1559
1551		− ima_template= [IMA]
	1560	+ ima_template= [IMA]
1552	1561	Select one of defined IMA measurements template formats.
1553	1562	Formats: { "ima" \| "ima-ng" \| "ima-sig" }
1554	1563	Default: "ima-ng"
1555	1564
1556	1565	ima_template_fmt=
1557		− [IMA] Define a custom template format.
	1566	+ [IMA] Define a custom template format.
1558	1567	Format: { "field1\|...\|fieldN" }
1559	1568
1560	1569	ima.ahash_minsize= [IMA] Minimum file size for asynchronous hash usage
@@ -1597,7 +1606,7 @@ · 36 unmodified lines
1597	1606	inport.irq= [HW] Inport (ATI XL and Microsoft) busmouse driver
1598	1607	Format: <irq>
1599	1608
1600		− int_pln_enable [x86] Enable power limit notification interrupt
	1609	+ int_pln_enable [x86] Enable power limit notification interrupt
1601	1610
1602	1611	integrity_audit=[IMA]
1603	1612	Format: { "0" \| "1" }
@@ -1650,39 +1659,39 @@ · 46 unmodified lines
1650	1659	0 disables intel_idle and fall back on acpi_idle.
1651	1660	1 to 9 specify maximum depth of C-state.
1652	1661
1653		− intel_pstate= [X86]
1654		− disable
1655		− Do not enable intel_pstate as the default
1656		− scaling driver for the supported processors
1657		− passive
1658		− Use intel_pstate as a scaling driver, but configure it
1659		− to work with generic cpufreq governors (instead of
1660		− enabling its internal governor). This mode cannot be
1661		− used along with the hardware-managed P-states (HWP)
1662		− feature.
1663		− force
1664		− Enable intel_pstate on systems that prohibit it by default
1665		− in favor of acpi-cpufreq. Forcing the intel_pstate driver
1666		− instead of acpi-cpufreq may disable platform features, such
1667		− as thermal controls and power capping, that rely on ACPI
1668		− P-States information being indicated to OSPM and therefore
1669		− should be used with caution. This option does not work with
1670		− processors that aren't supported by the intel_pstate driver
1671		− or on platforms that use pcc-cpufreq instead of acpi-cpufreq.
1672		− no_hwp
1673		− Do not enable hardware P state control (HWP)
1674		− if available.
1675		− hwp_only
1676		− Only load intel_pstate on systems which support
1677		− hardware P state control (HWP) if available.
1678		− support_acpi_ppc
1679		− Enforce ACPI _PPC performance limits. If the Fixed ACPI
1680		− Description Table, specifies preferred power management
1681		− profile as "Enterprise Server" or "Performance Server",
1682		− then this feature is turned on by default.
1683		− per_cpu_perf_limits
1684		− Allow per-logical-CPU P-State performance control limits using
1685		− cpufreq sysfs interface
	1662	+ intel_pstate= [X86]
	1663	+ disable
	1664	+ Do not enable intel_pstate as the default
	1665	+ scaling driver for the supported processors
	1666	+ passive
	1667	+ Use intel_pstate as a scaling driver, but configure it
	1668	+ to work with generic cpufreq governors (instead of
	1669	+ enabling its internal governor). This mode cannot be
	1670	+ used along with the hardware-managed P-states (HWP)
	1671	+ feature.
	1672	+ force
	1673	+ Enable intel_pstate on systems that prohibit it by default
	1674	+ in favor of acpi-cpufreq. Forcing the intel_pstate driver
	1675	+ instead of acpi-cpufreq may disable platform features, such
	1676	+ as thermal controls and power capping, that rely on ACPI
	1677	+ P-States information being indicated to OSPM and therefore
	1678	+ should be used with caution. This option does not work with
	1679	+ processors that aren't supported by the intel_pstate driver
	1680	+ or on platforms that use pcc-cpufreq instead of acpi-cpufreq.
	1681	+ no_hwp
	1682	+ Do not enable hardware P state control (HWP)
	1683	+ if available.
	1684	+ hwp_only
	1685	+ Only load intel_pstate on systems which support
	1686	+ hardware P state control (HWP) if available.
	1687	+ support_acpi_ppc
	1688	+ Enforce ACPI _PPC performance limits. If the Fixed ACPI
	1689	+ Description Table, specifies preferred power management
	1690	+ profile as "Enterprise Server" or "Performance Server",
	1691	+ then this feature is turned on by default.
	1692	+ per_cpu_perf_limits
	1693	+ Allow per-logical-CPU P-State performance control limits using
	1694	+ cpufreq sysfs interface
1686	1695
1687	1696	intremap= [X86-64, Intel-IOMMU]
1688	1697	on enable Interrupt Remapping (default)
@@ -2026,7 +2035,7 @@ · 337 unmodified lines
2026	2035	* [no]ncqtrim: Turn off queued DSM TRIM.
2027	2036
2028	2037	* nohrst, nosrst, norst: suppress hard, soft
2029		− and both resets.
	2038	+ and both resets.
2030	2039
2031	2040	* rstonce: only attempt one reset during
2032	2041	hot-unplug link recovery
@@ -2214,7 +2223,7 @@ · 181 unmodified lines
2214	2223	[KNL,SH] Allow user to override the default size for
2215	2224	per-device physically contiguous DMA buffers.
2216	2225
2217		− memhp_default_state=online/offline
	2226	+ memhp_default_state=online/offline
2218	2227	[KNL] Set the initial state for the memory hotplug
2219	2228	onlining policy. If not specified, the default value is
2220	2229	set according to the
@@ -2764,7 +2773,7 @@ · 543 unmodified lines
2764	2773	[X86,PV_OPS] Disable paravirtualized VMware scheduler
2765	2774	clock and use the default one.
2766	2775
2767		− no-steal-acc [X86,KVM] Disable paravirtualized steal time accounting.
	2776	+ no-steal-acc [X86,KVM] Disable paravirtualized steal time accounting.
2768	2777	steal time is computed, but won't influence scheduler
2769	2778	behaviour
2770	2779
@@ -2825,7 +2834,7 @@ · 54 unmodified lines
2825	2834	notsc [BUGS=X86-32] Disable Time Stamp Counter
2826	2835
2827	2836	nowatchdog [KNL] Disable both lockup detectors, i.e.
2828		− soft-lockup and NMI watchdog (hard-lockup).
	2837	+ soft-lockup and NMI watchdog (hard-lockup).
2829	2838
2830	2839	nowb [ARM]
2831	2840
@@ -2845,7 +2854,7 @@ · 13 unmodified lines
2845	2854	If the dependencies are under your control, you can
2846	2855	turn on cpu0_hotplug.
2847	2856
2848		− nps_mtm_hs_ctr= [KNL,ARC]
	2857	+ nps_mtm_hs_ctr= [KNL,ARC]
2849	2858	This parameter sets the maximum duration, in
2850	2859	cycles, each HW thread of the CTOP can run
2851	2860	without interruptions, before HW switches it.
@@ -2986,7 +2995,7 @@ · 134 unmodified lines
2986	2995
2987	2996	pci=option[,option...] [PCI] various PCI subsystem options:
2988	2997	earlydump [X86] dump PCI config space before the kernel
2989		− changes anything
	2998	+ changes anything
2990	2999	off [X86] don't probe for the PCI bus
2991	3000	bios [X86-32] force use of PCI BIOS, don't access
2992	3001	the hardware directly. Use this if your machine
@@ -3074,7 +3083,7 @@ · 81 unmodified lines
3074	3083	is enabled by default. If you need to use this,
3075	3084	please report a bug.
3076	3085	nocrs [X86] Ignore PCI host bridge windows from ACPI.
3077		− If you need to use this, please report a bug.
	3086	+ If you need to use this, please report a bug.
3078	3087	routeirq Do IRQ routing for all PCI devices.
3079	3088	This is normally done in pci_enable_device(),
3080	3089	so this option is a temporary workaround
@@ -3917,7 +3926,7 @@ · 836 unmodified lines
3917	3926	cache (risks via metadata attacks are mostly
3918	3927	unchanged). Debug options disable merging on their
3919	3928	own.
3920		− For more information see Documentation/vm/slub.txt.
	3929	+ For more information see Documentation/vm/slub.rst.
3921	3930
3922	3931	slab_max_order= [MM, SLAB]
3923	3932	Determines the maximum allowed order for slabs.
@@ -3931,7 +3940,7 @@ · 7 unmodified lines
3931	3940	slub_debug can create guard zones around objects and
3932	3941	may poison objects when not in use. Also tracks the
3933	3942	last alloc / free. For more information see
3934		− Documentation/vm/slub.txt.
	3943	+ Documentation/vm/slub.rst.
3935	3944
3936	3945	slub_memcg_sysfs= [MM, SLUB]
3937	3946	Determines whether to enable sysfs directories for
@@ -3945,7 +3954,7 @@ · 7 unmodified lines
3945	3954	Determines the maximum allowed order for slabs.
3946	3955	A high setting may cause OOMs due to memory
3947	3956	fragmentation. For more information see
3948		− Documentation/vm/slub.txt.
	3957	+ Documentation/vm/slub.rst.
3949	3958
3950	3959	slub_min_objects= [MM, SLUB]
3951	3960	The minimum number of objects per slab. SLUB will
@@ -3954,12 +3963,12 @@ · 2 unmodified lines
3954	3963	the number of objects indicated. The higher the number
3955	3964	of objects the smaller the overhead of tracking slabs
3956	3965	and the less frequently locks need to be acquired.
3957		− For more information see Documentation/vm/slub.txt.
	3966	+ For more information see Documentation/vm/slub.rst.
3958	3967
3959	3968	slub_min_order= [MM, SLUB]
3960	3969	Determines the minimum page order for slabs. Must be
3961	3970	lower than slub_max_order.
3962		− For more information see Documentation/vm/slub.txt.
	3971	+ For more information see Documentation/vm/slub.rst.
3963	3972
3964	3973	slub_nomerge [MM, SLUB]
3965	3974	Same with slab_nomerge. This is supported for legacy.
@@ -4357,7 +4366,8 @@ · 391 unmodified lines
4357	4366	Format: [always\|madvise\|never]
4358	4367	Can be used to control the default behavior of the system
4359	4368	with respect to transparent hugepages.
4360		− See Documentation/vm/transhuge.txt for more details.
	4369	+ See Documentation/admin-guide/mm/transhuge.rst
	4370	+ for more details.
4361	4371
4362	4372	tsc= Disable clocksource stability checks for TSC.
4363	4373	Format: <string>
@@ -4435,7 +4445,7 @@ · 71 unmodified lines
4435	4445
4436	4446	usbcore.initial_descriptor_timeout=
4437	4447	[USB] Specifies timeout for the initial 64-byte
4438		− USB_REQ_GET_DESCRIPTOR request in milliseconds
	4448	+ USB_REQ_GET_DESCRIPTOR request in milliseconds
4439	4449	(default 5000 = 5.0 seconds).
4440	4450
4441	4451	usbcore.nousb [USB] Disable the USB subsystem

Documentation/admin-guide/mm/concepts.rst+222−0

@@ -0,0 +1,222 @@
	1	+.. _mm_concepts:
	2	+
	3	+=================
	4	+Concepts overview
	5	+=================
	6	+
	7	+The memory management in Linux is complex system that evolved over the
	8	+years and included more and more functionality to support variety of
	9	+systems from MMU-less microcontrollers to supercomputers. The memory
	10	+management for systems without MMU is called ``nommu`` and it
	11	+definitely deserves a dedicated document, which hopefully will be
	12	+eventually written. Yet, although some of the concepts are the same,
	13	+here we assume that MMU is available and CPU can translate a virtual
	14	+address to a physical address.
	15	+
	16	+.. contents:: :local:
	17	+
	18	+Virtual Memory Primer
	19	+=====================
	20	+
	21	+The physical memory in a computer system is a limited resource and
	22	+even for systems that support memory hotplug there is a hard limit on
	23	+the amount of memory that can be installed. The physical memory is not
	24	+necessary contiguous, it might be accessible as a set of distinct
	25	+address ranges. Besides, different CPU architectures, and even
	26	+different implementations of the same architecture have different view
	27	+how these address ranges defined.
	28	+
	29	+All this makes dealing directly with physical memory quite complex and
	30	+to avoid this complexity a concept of virtual memory was developed.
	31	+
	32	+The virtual memory abstracts the details of physical memory from the
	33	+application software, allows to keep only needed information in the
	34	+physical memory (demand paging) and provides a mechanism for the
	35	+protection and controlled sharing of data between processes.
	36	+
	37	+With virtual memory, each and every memory access uses a virtual
	38	+address. When the CPU decodes the an instruction that reads (or
	39	+writes) from (or to) the system memory, it translates the `virtual`
	40	+address encoded in that instruction to a `physical` address that the
	41	+memory controller can understand.
	42	+
	43	+The physical system memory is divided into page frames, or pages. The
	44	+size of each page is architecture specific. Some architectures allow
	45	+selection of the page size from several supported values; this
	46	+selection is performed at the kernel build time by setting an
	47	+appropriate kernel configuration option.
	48	+
	49	+Each physical memory page can be mapped as one or more virtual
	50	+pages. These mappings are described by page tables that allow
	51	+translation from virtual address used by programs to real address in
	52	+the physical memory. The page tables organized hierarchically.
	53	+
	54	+The tables at the lowest level of the hierarchy contain physical
	55	+addresses of actual pages used by the software. The tables at higher
	56	+levels contain physical addresses of the pages belonging to the lower
	57	+levels. The pointer to the top level page table resides in a
	58	+register. When the CPU performs the address translation, it uses this
	59	+register to access the top level page table. The high bits of the
	60	+virtual address are used to index an entry in the top level page
	61	+table. That entry is then used to access the next level in the
	62	+hierarchy with the next bits of the virtual address as the index to
	63	+that level page table. The lowest bits in the virtual address define
	64	+the offset inside the actual page.
	65	+
	66	+Huge Pages
	67	+==========
	68	+
	69	+The address translation requires several memory accesses and memory
	70	+accesses are slow relatively to CPU speed. To avoid spending precious
	71	+processor cycles on the address translation, CPUs maintain a cache of
	72	+such translations called Translation Lookaside Buffer (or
	73	+TLB). Usually TLB is pretty scarce resource and applications with
	74	+large memory working set will experience performance hit because of
	75	+TLB misses.
	76	+
	77	+Many modern CPU architectures allow mapping of the memory pages
	78	+directly by the higher levels in the page table. For instance, on x86,
	79	+it is possible to map 2M and even 1G pages using entries in the second
	80	+and the third level page tables. In Linux such pages are called
	81	+`huge`. Usage of huge pages significantly reduces pressure on TLB,
	82	+improves TLB hit-rate and thus improves overall system performance.
	83	+
	84	+There are two mechanisms in Linux that enable mapping of the physical
	85	+memory with the huge pages. The first one is `HugeTLB filesystem`, or
	86	+hugetlbfs. It is a pseudo filesystem that uses RAM as its backing
	87	+store. For the files created in this filesystem the data resides in
	88	+the memory and mapped using huge pages. The hugetlbfs is described at
	89	+:ref:`Documentation/admin-guide/mm/hugetlbpage.rst <hugetlbpage>`.
	90	+
	91	+Another, more recent, mechanism that enables use of the huge pages is
	92	+called `Transparent HugePages`, or THP. Unlike the hugetlbfs that
	93	+requires users and/or system administrators to configure what parts of
	94	+the system memory should and can be mapped by the huge pages, THP
	95	+manages such mappings transparently to the user and hence the
	96	+name. See
	97	+:ref:`Documentation/admin-guide/mm/transhuge.rst <admin_guide_transhuge>`
	98	+for more details about THP.
	99	+
	100	+Zones
	101	+=====
	102	+
	103	+Often hardware poses restrictions on how different physical memory
	104	+ranges can be accessed. In some cases, devices cannot perform DMA to
	105	+all the addressable memory. In other cases, the size of the physical
	106	+memory exceeds the maximal addressable size of virtual memory and
	107	+special actions are required to access portions of the memory. Linux
	108	+groups memory pages into `zones` according to their possible
	109	+usage. For example, ZONE_DMA will contain memory that can be used by
	110	+devices for DMA, ZONE_HIGHMEM will contain memory that is not
	111	+permanently mapped into kernel's address space and ZONE_NORMAL will
	112	+contain normally addressed pages.
	113	+
	114	+The actual layout of the memory zones is hardware dependent as not all
	115	+architectures define all zones, and requirements for DMA are different
	116	+for different platforms.
	117	+
	118	+Nodes
	119	+=====
	120	+
	121	+Many multi-processor machines are NUMA - Non-Uniform Memory Access -
	122	+systems. In such systems the memory is arranged into banks that have
	123	+different access latency depending on the "distance" from the
	124	+processor. Each bank is referred as `node` and for each node Linux
	125	+constructs an independent memory management subsystem. A node has it's
	126	+own set of zones, lists of free and used pages and various statistics
	127	+counters. You can find more details about NUMA in
	128	+:ref:`Documentation/vm/numa.rst <numa>` and in
	129	+:ref:`Documentation/admin-guide/mm/numa_memory_policy.rst <numa_memory_policy>`.
	130	+
	131	+Page cache
	132	+==========
	133	+
	134	+The physical memory is volatile and the common case for getting data
	135	+into the memory is to read it from files. Whenever a file is read, the
	136	+data is put into the `page cache` to avoid expensive disk access on
	137	+the subsequent reads. Similarly, when one writes to a file, the data
	138	+is placed in the page cache and eventually gets into the backing
	139	+storage device. The written pages are marked as `dirty` and when Linux
	140	+decides to reuse them for other purposes, it makes sure to synchronize
	141	+the file contents on the device with the updated data.
	142	+
	143	+Anonymous Memory
	144	+================
	145	+
	146	+The `anonymous memory` or `anonymous mappings` represent memory that
	147	+is not backed by a filesystem. Such mappings are implicitly created
	148	+for program's stack and heap or by explicit calls to mmap(2) system
	149	+call. Usually, the anonymous mappings only define virtual memory areas
	150	+that the program is allowed to access. The read accesses will result
	151	+in creation of a page table entry that references a special physical
	152	+page filled with zeroes. When the program performs a write, regular
	153	+physical page will be allocated to hold the written data. The page
	154	+will be marked dirty and if the kernel will decide to repurpose it,
	155	+the dirty page will be swapped out.
	156	+
	157	+Reclaim
	158	+=======
	159	+
	160	+Throughout the system lifetime, a physical page can be used for storing
	161	+different types of data. It can be kernel internal data structures,
	162	+DMA'able buffers for device drivers use, data read from a filesystem,
	163	+memory allocated by user space processes etc.
	164	+
	165	+Depending on the page usage it is treated differently by the Linux
	166	+memory management. The pages that can be freed at any time, either
	167	+because they cache the data available elsewhere, for instance, on a
	168	+hard disk, or because they can be swapped out, again, to the hard
	169	+disk, are called `reclaimable`. The most notable categories of the
	170	+reclaimable pages are page cache and anonymous memory.
	171	+
	172	+In most cases, the pages holding internal kernel data and used as DMA
	173	+buffers cannot be repurposed, and they remain pinned until freed by
	174	+their user. Such pages are called `unreclaimable`. However, in certain
	175	+circumstances, even pages occupied with kernel data structures can be
	176	+reclaimed. For instance, in-memory caches of filesystem metadata can
	177	+be re-read from the storage device and therefore it is possible to
	178	+discard them from the main memory when system is under memory
	179	+pressure.
	180	+
	181	+The process of freeing the reclaimable physical memory pages and
	182	+repurposing them is called (surprise!) `reclaim`. Linux can reclaim
	183	+pages either asynchronously or synchronously, depending on the state
	184	+of the system. When system is not loaded, most of the memory is free
	185	+and allocation request will be satisfied immediately from the free
	186	+pages supply. As the load increases, the amount of the free pages goes
	187	+down and when it reaches a certain threshold (high watermark), an
	188	+allocation request will awaken the ``kswapd`` daemon. It will
	189	+asynchronously scan memory pages and either just free them if the data
	190	+they contain is available elsewhere, or evict to the backing storage
	191	+device (remember those dirty pages?). As memory usage increases even
	192	+more and reaches another threshold - min watermark - an allocation
	193	+will trigger the `direct reclaim`. In this case allocation is stalled
	194	+until enough memory pages are reclaimed to satisfy the request.
	195	+
	196	+Compaction
	197	+==========
	198	+
	199	+As the system runs, tasks allocate and free the memory and it becomes
	200	+fragmented. Although with virtual memory it is possible to present
	201	+scattered physical pages as virtually contiguous range, sometimes it is
	202	+necessary to allocate large physically contiguous memory areas. Such
	203	+need may arise, for instance, when a device driver requires large
	204	+buffer for DMA, or when THP allocates a huge page. Memory `compaction`
	205	+addresses the fragmentation issue. This mechanism moves occupied pages
	206	+from the lower part of a memory zone to free pages in the upper part
	207	+of the zone. When a compaction scan is finished free pages are grouped
	208	+together at the beginning of the zone and allocations of large
	209	+physically contiguous areas become possible.
	210	+
	211	+Like reclaim, the compaction may happen asynchronously in ``kcompactd``
	212	+daemon or synchronously as a result of memory allocation request.
	213	+
	214	+OOM killer
	215	+==========
	216	+
	217	+It may happen, that on a loaded machine memory will be exhausted. When
	218	+the kernel detects that the system runs out of memory (OOM) it invokes
	219	+`OOM killer`. Its mission is simple: all it has to do is to select a
	220	+task to sacrifice for the sake of the overall system health. The
	221	+selected task is killed in a hope that after it exits enough memory
	222	+will be freed to continue normal operation.

Documentation/admin-guide/mm/hugetlbpage.rstrenamed from Documentation/vm/hugetlbpage.txt+147−116

@@ -1,3 +1,11 @@
	1	+.. _hugetlbpage:
	2	+
	3	+=============
	4	+HugeTLB Pages
	5	+=============
	6	+
	7	+Overview
	8	+========
1	9
2	10	The intent of this file is to give a brief summary of hugetlbpage support in
3	11	the Linux kernel. This support is built on top of multiple page size support
@@ -18,53 +26,59 @@ First the Linux kernel needs to be built with the CONFIG_HUGETLBFS · 14 unmodified lines
18	26	automatically when CONFIG_HUGETLBFS is selected) configuration
19	27	options.
20	28
21		−The /proc/meminfo file provides information about the total number of
	29	+The ``/proc/meminfo`` file provides information about the total number of
22	30	persistent hugetlb pages in the kernel's huge page pool. It also displays
23	31	default huge page size and information about the number of free, reserved
24	32	and surplus huge pages in the pool of huge pages of default size.
25	33	The huge page size is needed for generating the proper alignment and
26	34	size of the arguments to system calls that map huge page regions.
27	35
28		−The output of "cat /proc/meminfo" will include lines like:
	36	+The output of ``cat /proc/meminfo`` will include lines like::
29	37
30		−.....
31		−HugePages_Total: uuu
32		−HugePages_Free: vvv
33		−HugePages_Rsvd: www
34		−HugePages_Surp: xxx
35		−Hugepagesize: yyy kB
36		−Hugetlb: zzz kB
	38	+ HugePages_Total: uuu
	39	+ HugePages_Free: vvv
	40	+ HugePages_Rsvd: www
	41	+ HugePages_Surp: xxx
	42	+ Hugepagesize: yyy kB
	43	+ Hugetlb: zzz kB
37	44
38	45	where:
39		−HugePages_Total is the size of the pool of huge pages.
40		−HugePages_Free is the number of huge pages in the pool that are not yet
41		− allocated.
42		−HugePages_Rsvd is short for "reserved," and is the number of huge pages for
43		− which a commitment to allocate from the pool has been made,
44		− but no allocation has yet been made. Reserved huge pages
45		− guarantee that an application will be able to allocate a
46		− huge page from the pool of huge pages at fault time.
47		−HugePages_Surp is short for "surplus," and is the number of huge pages in
48		− the pool above the value in /proc/sys/vm/nr_hugepages. The
49		− maximum number of surplus huge pages is controlled by
50		− /proc/sys/vm/nr_overcommit_hugepages.
51		−Hugepagesize is the default hugepage size (in Kb).
52		−Hugetlb is the total amount of memory (in kB), consumed by huge
53		− pages of all sizes.
54		− If huge pages of different sizes are in use, this number
55		− will exceed HugePages_Total * Hugepagesize. To get more
56		− detailed information, please, refer to
57		− /sys/kernel/mm/hugepages (described below).
58		−
59		−
60		−/proc/filesystems should also show a filesystem of type "hugetlbfs" configured
61		−in the kernel.
62		−
63		−/proc/sys/vm/nr_hugepages indicates the current number of "persistent" huge
	46	+
	47	+HugePages_Total
	48	+ is the size of the pool of huge pages.
	49	+HugePages_Free
	50	+ is the number of huge pages in the pool that are not yet
	51	+ allocated.
	52	+HugePages_Rsvd
	53	+ is short for "reserved," and is the number of huge pages for
	54	+ which a commitment to allocate from the pool has been made,
	55	+ but no allocation has yet been made. Reserved huge pages
	56	+ guarantee that an application will be able to allocate a
	57	+ huge page from the pool of huge pages at fault time.
	58	+HugePages_Surp
	59	+ is short for "surplus," and is the number of huge pages in
	60	+ the pool above the value in ``/proc/sys/vm/nr_hugepages``. The
	61	+ maximum number of surplus huge pages is controlled by
	62	+ ``/proc/sys/vm/nr_overcommit_hugepages``.
	63	+Hugepagesize
	64	+ is the default hugepage size (in Kb).
	65	+Hugetlb
	66	+ is the total amount of memory (in kB), consumed by huge
	67	+ pages of all sizes.
	68	+ If huge pages of different sizes are in use, this number
	69	+ will exceed HugePages_Total \* Hugepagesize. To get more
	70	+ detailed information, please, refer to
	71	+ ``/sys/kernel/mm/hugepages`` (described below).
	72	+
	73	+
	74	+``/proc/filesystems`` should also show a filesystem of type "hugetlbfs"
	75	+configured in the kernel.
	76	+
	77	+``/proc/sys/vm/nr_hugepages`` indicates the current number of "persistent" huge
64	78	pages in the kernel's huge page pool. "Persistent" huge pages will be
65	79	returned to the huge page pool when freed by a task. A user with root
66	80	privileges can dynamically allocate more or free some persistent huge pages
67		−by increasing or decreasing the value of 'nr_hugepages'.
	81	+by increasing or decreasing the value of ``nr_hugepages``.
68	82
69	83	Pages that are used as huge pages are reserved inside the kernel and cannot
70	84	be used for other purposes. Huge pages cannot be swapped out under
@@ -73,7 +87,7 @@ memory pressure. · 2 unmodified lines
73	87	Once a number of huge pages have been pre-allocated to the kernel huge page
74	88	pool, a user with appropriate privilege can use either the mmap system call
75	89	or shared memory system calls to use the huge pages. See the discussion of
76		−Using Huge Pages, below.
	90	+:ref:`Using Huge Pages <using_huge_pages>`, below.
77	91
78	92	The administrator can allocate persistent huge pages on the kernel boot
79	93	command line by specifying the "hugepages=N" parameter, where 'N' = the
@@ -86,10 +100,10 @@ with a huge page size selection parameter "hugepagesz=<size>". <size> must · 6 unmodified lines
86	100	be specified in bytes with optional scale suffix [kKmMgG]. The default huge
87	101	page size may be selected with the "default_hugepagesz=<size>" boot parameter.
88	102
89		−When multiple huge page sizes are supported, /proc/sys/vm/nr_hugepages
	103	+When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages``
90	104	indicates the current number of pre-allocated huge pages of the default size.
91	105	Thus, one can use the following command to dynamically allocate/deallocate
92		−default sized persistent huge pages:
	106	+default sized persistent huge pages::
93	107
94	108	echo 20 > /proc/sys/vm/nr_hugepages
95	109
@@ -98,11 +112,12 @@ huge page pool to 20, allocating or freeing huge pages, as required. · 2 unmodified lines
98	112
99	113	On a NUMA platform, the kernel will attempt to distribute the huge page pool
100	114	over all the set of allowed nodes specified by the NUMA memory policy of the
101		−task that modifies nr_hugepages. The default for the allowed nodes--when the
	115	+task that modifies ``nr_hugepages``. The default for the allowed nodes--when the
102	116	task has default memory policy--is all on-line nodes with memory. Allowed
103	117	nodes with insufficient available, contiguous memory for a huge page will be
104		−silently skipped when allocating persistent huge pages. See the discussion
105		−below of the interaction of task memory policy, cpusets and per node attributes
	118	+silently skipped when allocating persistent huge pages. See the
	119	+:ref:`discussion below <mem_policy_and_hp_alloc>`
	120	+of the interaction of task memory policy, cpusets and per node attributes
106	121	with the allocation and freeing of persistent huge pages.
107	122
108	123	The success or failure of huge page allocation depends on the amount of
@@ -117,51 +132,52 @@ init files. This will enable the kernel to allocate huge pages early in · 8 unmodified lines
117	132	the boot process when the possibility of getting physical contiguous pages
118	133	is still very high. Administrators can verify the number of huge pages
119	134	actually allocated by checking the sysctl or meminfo. To check the per node
120		−distribution of huge pages in a NUMA system, use:
	135	+distribution of huge pages in a NUMA system, use::
121	136
122	137	cat /sys/devices/system/node/node*/meminfo \| fgrep Huge
123	138
124		−/proc/sys/vm/nr_overcommit_hugepages specifies how large the pool of
125		−huge pages can grow, if more huge pages than /proc/sys/vm/nr_hugepages are
	139	+``/proc/sys/vm/nr_overcommit_hugepages`` specifies how large the pool of
	140	+huge pages can grow, if more huge pages than ``/proc/sys/vm/nr_hugepages`` are
126	141	requested by applications. Writing any non-zero value into this file
127	142	indicates that the hugetlb subsystem is allowed to try to obtain that
128	143	number of "surplus" huge pages from the kernel's normal page pool, when the
129	144	persistent huge page pool is exhausted. As these surplus huge pages become
130	145	unused, they are freed back to the kernel's normal page pool.
131	146
132		−When increasing the huge page pool size via nr_hugepages, any existing surplus
133		−pages will first be promoted to persistent huge pages. Then, additional
	147	+When increasing the huge page pool size via ``nr_hugepages``, any existing
	148	+surplus pages will first be promoted to persistent huge pages. Then, additional
134	149	huge pages will be allocated, if necessary and if possible, to fulfill
135	150	the new persistent huge page pool size.
136	151
137	152	The administrator may shrink the pool of persistent huge pages for
138		−the default huge page size by setting the nr_hugepages sysctl to a
	153	+the default huge page size by setting the ``nr_hugepages`` sysctl to a
139	154	smaller value. The kernel will attempt to balance the freeing of huge pages
140		−across all nodes in the memory policy of the task modifying nr_hugepages.
	155	+across all nodes in the memory policy of the task modifying ``nr_hugepages``.
141	156	Any free huge pages on the selected nodes will be freed back to the kernel's
142	157	normal page pool.
143	158
144		−Caveat: Shrinking the persistent huge page pool via nr_hugepages such that
	159	+Caveat: Shrinking the persistent huge page pool via ``nr_hugepages`` such that
145	160	it becomes less than the number of huge pages in use will convert the balance
146	161	of the in-use huge pages to surplus huge pages. This will occur even if
147		−the number of surplus pages it would exceed the overcommit value. As long as
148		−this condition holds--that is, until nr_hugepages+nr_overcommit_hugepages is
	162	+the number of surplus pages would exceed the overcommit value. As long as
	163	+this condition holds--that is, until ``nr_hugepages+nr_overcommit_hugepages`` is
149	164	increased sufficiently, or the surplus huge pages go out of use and are freed--
150	165	no more surplus huge pages will be allowed to be allocated.
151	166
152	167	With support for multiple huge page pools at run-time available, much of
153		−the huge page userspace interface in /proc/sys/vm has been duplicated in sysfs.
154		−The /proc interfaces discussed above have been retained for backwards
155		−compatibility. The root huge page control directory in sysfs is:
	168	+the huge page userspace interface in ``/proc/sys/vm`` has been duplicated in
	169	+sysfs.
	170	+The ``/proc`` interfaces discussed above have been retained for backwards
	171	+compatibility. The root huge page control directory in sysfs is::
156	172
157	173	/sys/kernel/mm/hugepages
158	174
159	175	For each huge page size supported by the running kernel, a subdirectory
160		−will exist, of the form:
	176	+will exist, of the form::
161	177
162	178	hugepages-${size}kB
163	179
164		−Inside each of these directories, the same set of files will exist:
	180	+Inside each of these directories, the same set of files will exist::
165	181
166	182	nr_hugepages
167	183	nr_hugepages_mempolicy
@@ -172,37 +188,39 @@ Inside each of these directories, the same set of files will exist: · 4 unmodified lines
172	188
173	189	which function as described above for the default huge page-sized case.
174	190
	191	+.. _mem_policy_and_hp_alloc:
175	192
176	193	Interaction of Task Memory Policy with Huge Page Allocation/Freeing
177	194	===================================================================
178	195
179		−Whether huge pages are allocated and freed via the /proc interface or
180		−the /sysfs interface using the nr_hugepages_mempolicy attribute, the NUMA
181		−nodes from which huge pages are allocated or freed are controlled by the
182		−NUMA memory policy of the task that modifies the nr_hugepages_mempolicy
183		−sysctl or attribute. When the nr_hugepages attribute is used, mempolicy
	196	+Whether huge pages are allocated and freed via the ``/proc`` interface or
	197	+the ``/sysfs`` interface using the ``nr_hugepages_mempolicy`` attribute, the
	198	+NUMA nodes from which huge pages are allocated or freed are controlled by the
	199	+NUMA memory policy of the task that modifies the ``nr_hugepages_mempolicy``
	200	+sysctl or attribute. When the ``nr_hugepages`` attribute is used, mempolicy
184	201	is ignored.
185	202
186	203	The recommended method to allocate or free huge pages to/from the kernel
187		−huge page pool, using the nr_hugepages example above, is:
	204	+huge page pool, using the ``nr_hugepages`` example above, is::
188	205
189	206	numactl --interleave <node-list> echo 20 \
190	207	>/proc/sys/vm/nr_hugepages_mempolicy
191	208
192		−or, more succinctly:
	209	+or, more succinctly::
193	210
194	211	numactl -m <node-list> echo 20 >/proc/sys/vm/nr_hugepages_mempolicy
195	212
196		−This will allocate or free abs(20 - nr_hugepages) to or from the nodes
	213	+This will allocate or free ``abs(20 - nr_hugepages)`` to or from the nodes
197	214	specified in <node-list>, depending on whether number of persistent huge pages
198	215	is initially less than or greater than 20, respectively. No huge pages will be
199	216	allocated nor freed on any node not included in the specified <node-list>.
200	217
201		−When adjusting the persistent hugepage count via nr_hugepages_mempolicy, any
	218	+When adjusting the persistent hugepage count via ``nr_hugepages_mempolicy``, any
202	219	memory policy mode--bind, preferred, local or interleave--may be used. The
203	220	resulting effect on persistent huge page allocation is as follows:
204	221
205		−1) Regardless of mempolicy mode [see Documentation/vm/numa_memory_policy.txt],
	222	+#. Regardless of mempolicy mode [see
	223	+ :ref:`Documentation/admin-guide/mm/numa_memory_policy.rst <numa_memory_policy>`],
206	224	persistent huge pages will be distributed across the node or nodes
207	225	specified in the mempolicy as if "interleave" had been specified.
208	226	However, if a node in the policy does not contain sufficient contiguous
@@ -212,7 +230,7 @@ resulting effect on persistent huge page allocation is as follows: · 3 unmodified lines
212	230	possibly, allocation of persistent huge pages on nodes not allowed by
213	231	the task's memory policy.
214	232
215		−2) One or more nodes may be specified with the bind or interleave policy.
	233	+#. One or more nodes may be specified with the bind or interleave policy.
216	234	If more than one node is specified with the preferred policy, only the
217	235	lowest numeric id will be used. Local policy will select the node where
218	236	the task is running at the time the nodes_allowed mask is constructed.
@@ -222,20 +240,20 @@ resulting effect on persistent huge page allocation is as follows: · 3 unmodified lines
222	240	indeterminate. Thus, local policy is not very useful for this purpose.
223	241	Any of the other mempolicy modes may be used to specify a single node.
224	242
225		−3) The nodes allowed mask will be derived from any non-default task mempolicy,
	243	+#. The nodes allowed mask will be derived from any non-default task mempolicy,
226	244	whether this policy was set explicitly by the task itself or one of its
227	245	ancestors, such as numactl. This means that if the task is invoked from a
228	246	shell with non-default policy, that policy will be used. One can specify a
229	247	node list of "all" with numactl --interleave or --membind [-m] to achieve
230	248	interleaving over all nodes in the system or cpuset.
231	249
232		−4) Any task mempolicy specified--e.g., using numactl--will be constrained by
	250	+#. Any task mempolicy specified--e.g., using numactl--will be constrained by
233	251	the resource limits of any cpuset in which the task runs. Thus, there will
234	252	be no way for a task with non-default policy running in a cpuset with a
235	253	subset of the system nodes to allocate huge pages outside the cpuset
236	254	without first moving to a cpuset that contains all of the desired nodes.
237	255
238		−5) Boot-time huge page allocation attempts to distribute the requested number
	256	+#. Boot-time huge page allocation attempts to distribute the requested number
239	257	of huge pages over all on-lines nodes with memory.
240	258
241	259	Per Node Hugepages Attributes
@@ -243,22 +261,22 @@ Per Node Hugepages Attributes · 1 unmodified line
243	261
244	262	A subset of the contents of the root huge page control directory in sysfs,
245	263	described above, will be replicated under each the system device of each
246		−NUMA node with memory in:
	264	+NUMA node with memory in::
247	265
248	266	/sys/devices/system/node/node[0-9]*/hugepages/
249	267
250	268	Under this directory, the subdirectory for each supported huge page size
251		−contains the following attribute files:
	269	+contains the following attribute files::
252	270
253	271	nr_hugepages
254	272	free_hugepages
255	273	surplus_hugepages
256	274
257		−The free_' and surplus_' attribute files are read-only. They return the number
	275	+The free\_' and surplus\_' attribute files are read-only. They return the number
258	276	of free and surplus [overcommitted] huge pages, respectively, on the parent
259	277	node.
260	278
261		−The nr_hugepages attribute returns the total number of huge pages on the
	279	+The ``nr_hugepages`` attribute returns the total number of huge pages on the
262	280	specified node. When this attribute is written, the number of persistent huge
263	281	pages on the parent node will be adjusted to the specified value, if sufficient
264	282	resources exist, regardless of the task's mempolicy or cpuset constraints.
@@ -267,43 +285,58 @@ Note that the number of overcommit and reserve pages remain global quantities, · 2 unmodified lines
267	285	as we don't know until fault time, when the faulting task's mempolicy is
268	286	applied, from which node the huge page allocation will be attempted.
269	287
	288	+.. _using_huge_pages:
270	289
271	290	Using Huge Pages
272	291	================
273	292
274	293	If the user applications are going to request huge pages using mmap system
275	294	call, then it is required that system administrator mount a file system of
276		−type hugetlbfs:
	295	+type hugetlbfs::
277	296
278	297	mount -t hugetlbfs \
279	298	-o uid=<value>,gid=<value>,mode=<value>,pagesize=<value>,size=<value>,\
280	299	min_size=<value>,nr_inodes=<value> none /mnt/huge
281	300
282	301	This command mounts a (pseudo) filesystem of type hugetlbfs on the directory
283		−/mnt/huge. Any files created on /mnt/huge uses huge pages. The uid and gid
284		−options sets the owner and group of the root of the file system. By default
285		−the uid and gid of the current process are taken. The mode option sets the
286		−mode of root of file system to value & 01777. This value is given in octal.
287		−By default the value 0755 is picked. If the platform supports multiple huge
288		−page sizes, the pagesize option can be used to specify the huge page size and
289		−associated pool. pagesize is specified in bytes. If pagesize is not specified
290		−the platform's default huge page size and associated pool will be used. The
291		−size option sets the maximum value of memory (huge pages) allowed for that
292		−filesystem (/mnt/huge). The size option can be specified in bytes, or as a
293		−percentage of the specified huge page pool (nr_hugepages). The size is
294		−rounded down to HPAGE_SIZE boundary. The min_size option sets the minimum
295		−value of memory (huge pages) allowed for the filesystem. min_size can be
296		−specified in the same way as size, either bytes or a percentage of the
297		−huge page pool. At mount time, the number of huge pages specified by
298		−min_size are reserved for use by the filesystem. If there are not enough
299		−free huge pages available, the mount will fail. As huge pages are allocated
300		−to the filesystem and freed, the reserve count is adjusted so that the sum
301		−of allocated and reserved huge pages is always at least min_size. The option
302		−nr_inodes sets the maximum number of inodes that /mnt/huge can use. If the
303		−size, min_size or nr_inodes option is not provided on command line then
304		−no limits are set. For pagesize, size, min_size and nr_inodes options, you
305		−can use [G\|g]/[M\|m]/[K\|k] to represent giga/mega/kilo. For example, size=2K
306		−has the same meaning as size=2048.
	302	+``/mnt/huge``. Any file created on ``/mnt/huge`` uses huge pages.
	303	+
	304	+The ``uid`` and ``gid`` options sets the owner and group of the root of the
	305	+file system. By default the ``uid`` and ``gid`` of the current process
	306	+are taken.
	307	+
	308	+The ``mode`` option sets the mode of root of file system to value & 01777.
	309	+This value is given in octal. By default the value 0755 is picked.
	310	+
	311	+If the platform supports multiple huge page sizes, the ``pagesize`` option can
	312	+be used to specify the huge page size and associated pool. ``pagesize``
	313	+is specified in bytes. If ``pagesize`` is not specified the platform's
	314	+default huge page size and associated pool will be used.
	315	+
	316	+The ``size`` option sets the maximum value of memory (huge pages) allowed
	317	+for that filesystem (``/mnt/huge``). The ``size`` option can be specified
	318	+in bytes, or as a percentage of the specified huge page pool (``nr_hugepages``).
	319	+The size is rounded down to HPAGE_SIZE boundary.
	320	+
	321	+The ``min_size`` option sets the minimum value of memory (huge pages) allowed
	322	+for the filesystem. ``min_size`` can be specified in the same way as ``size``,
	323	+either bytes or a percentage of the huge page pool.
	324	+At mount time, the number of huge pages specified by ``min_size`` are reserved
	325	+for use by the filesystem.
	326	+If there are not enough free huge pages available, the mount will fail.
	327	+As huge pages are allocated to the filesystem and freed, the reserve count
	328	+is adjusted so that the sum of allocated and reserved huge pages is always
	329	+at least ``min_size``.
	330	+
	331	+The option ``nr_inodes`` sets the maximum number of inodes that ``/mnt/huge``
	332	+can use.
	333	+
	334	+If the ``size``, ``min_size`` or ``nr_inodes`` option is not provided on
	335	+command line then no limits are set.
	336	+
	337	+For ``pagesize``, ``size``, ``min_size`` and ``nr_inodes`` options, you can
	338	+use [G\|g]/[M\|m]/[K\|k] to represent giga/mega/kilo.
	339	+For example, size=2K has the same meaning as size=2048.
307	340
308	341	While read system calls are supported on files that reside on hugetlb
309	342	file systems, write system calls are not.
@@ -313,12 +346,12 @@ used to change the file attributes on hugetlbfs. · 3 unmodified lines
313	346
314	347	Also, it is important to note that no such mount command is required if
315	348	applications are going to use only shmat/shmget system calls or mmap with
316		−MAP_HUGETLB. For an example of how to use mmap with MAP_HUGETLB see map_hugetlb
317		−below.
	349	+MAP_HUGETLB. For an example of how to use mmap with MAP_HUGETLB see
	350	+:ref:`map_hugetlb <map_hugetlb>` below.
318	351
319		−Users who wish to use hugetlb memory via shared memory segment should be a
320		−member of a supplementary group and system admin needs to configure that gid
321		−into /proc/sys/vm/hugetlb_shm_group. It is possible for same or different
	352	+Users who wish to use hugetlb memory via shared memory segment should be
	353	+members of a supplementary group and system admin needs to configure that gid
	354	+into ``/proc/sys/vm/hugetlb_shm_group``. It is possible for same or different
322	355	applications to use any combination of mmaps and shm* calls, though the mount of
323	356	filesystem will be required for using mmap calls without MAP_HUGETLB.
324	357
@@ -332,20 +365,18 @@ a hugetlb page and the length is smaller than the hugepage size. · 7 unmodified lines
332	365	Examples
333	366	========
334	367
335		−1) map_hugetlb: see tools/testing/selftests/vm/map_hugetlb.c
	368	+.. _map_hugetlb:
336	369
337		−2) hugepage-shm: see tools/testing/selftests/vm/hugepage-shm.c
	370	+``map_hugetlb``
	371	+ see tools/testing/selftests/vm/map_hugetlb.c
338	372
339		−3) hugepage-mmap: see tools/testing/selftests/vm/hugepage-mmap.c
	373	+``hugepage-shm``
	374	+ see tools/testing/selftests/vm/hugepage-shm.c
340	375
341		−4) The libhugetlbfs (https://github.com/libhugetlbfs/libhugetlbfs) library
342		− provides a wide range of userspace tools to help with huge page usability,
343		− environment setup, and control.
	376	+``hugepage-mmap``
	377	+ see tools/testing/selftests/vm/hugepage-mmap.c
344	378
345		−Kernel development regression testing
346		−=====================================
	379	+The `libhugetlbfs`_ library provides a wide range of userspace tools
	380	+to help with huge page usability, environment setup, and control.
347	381
348		−The most complete set of hugetlb tests are in the libhugetlbfs repository.
349		−If you modify any hugetlb related code, use the libhugetlbfs test suite
350		−to check for regressions. In addition, if you add any new hugetlb
351		−functionality, please add appropriate tests to libhugetlbfs.
	382	+.. _libhugetlbfs: https://github.com/libhugetlbfs/libhugetlbfs

Documentation/admin-guide/mm/idle_page_tracking.rstrenamed from Documentation/vm/idle_page_tracking.txt+37−19

@@ -1,4 +1,11 @@
1		−MOTIVATION
	1	+.. _idle_page_tracking:
	2	+
	3	+==================
	4	+Idle Page Tracking
	5	+==================
	6	+
	7	+Motivation
	8	+==========
2	9
3	10	The idle page tracking feature allows to track which memory pages are being
4	11	accessed by a workload and which are idle. This information can be useful for
@@ -8,10 +15,14 @@ or deciding where to place the workload within a compute cluster. · 3 unmodified lines
8	15
9	16	It is enabled by CONFIG_IDLE_PAGE_TRACKING=y.
10	17
11		−USER API
	18	+.. _user_api:
12	19
13		−The idle page tracking API is located at /sys/kernel/mm/page_idle. Currently,
14		−it consists of the only read-write file, /sys/kernel/mm/page_idle/bitmap.
	20	+User API
	21	+========
	22	+
	23	+The idle page tracking API is located at ``/sys/kernel/mm/page_idle``.
	24	+Currently, it consists of the only read-write file,
	25	+``/sys/kernel/mm/page_idle/bitmap``.
15	26
16	27	The file implements a bitmap where each bit corresponds to a memory page. The
17	28	bitmap is represented by an array of 8-byte integers, and the page at PFN #i is
@@ -19,8 +30,9 @@ mapped to bit #i%64 of array element #i/64, byte order is native. When a bit is · 1 unmodified line
19	30	set, the corresponding page is idle.
20	31
21	32	A page is considered idle if it has not been accessed since it was marked idle
22		−(for more details on what "accessed" actually means see the IMPLEMENTATION
23		−DETAILS section). To mark a page idle one has to set the bit corresponding to
	33	+(for more details on what "accessed" actually means see the :ref:`Implementation
	34	+Details <impl_details>` section).
	35	+To mark a page idle one has to set the bit corresponding to
24	36	the page by writing to the file. A value written to the file is OR-ed with the
25	37	current bitmap value.
26	38
@@ -30,9 +42,9 @@ page types (e.g. SLAB pages) an attempt to mark a page idle is silently ignored, · 3 unmodified lines
30	42	and hence such pages are never reported idle.
31	43
32	44	For huge pages the idle flag is set only on the head page, so one has to read
33		−/proc/kpageflags in order to correctly count idle huge pages.
	45	+``/proc/kpageflags`` in order to correctly count idle huge pages.
34	46
35		−Reading from or writing to /sys/kernel/mm/page_idle/bitmap will return
	47	+Reading from or writing to ``/sys/kernel/mm/page_idle/bitmap`` will return
36	48	-EINVAL if you are not starting the read/write on an 8-byte boundary, or
37	49	if the size of the read/write is not a multiple of 8 bytes. Writing to
38	50	this file beyond max PFN will return -ENXIO.
@@ -41,21 +53,26 @@ That said, in order to estimate the amount of pages that are not used by a · 2 unmodified lines
41	53	workload one should:
42	54
43	55	1. Mark all the workload's pages as idle by setting corresponding bits in
44		− /sys/kernel/mm/page_idle/bitmap. The pages can be found by reading
45		− /proc/pid/pagemap if the workload is represented by a process, or by
46		− filtering out alien pages using /proc/kpagecgroup in case the workload is
47		− placed in a memory cgroup.
	56	+ ``/sys/kernel/mm/page_idle/bitmap``. The pages can be found by reading
	57	+ ``/proc/pid/pagemap`` if the workload is represented by a process, or by
	58	+ filtering out alien pages using ``/proc/kpagecgroup`` in case the workload
	59	+ is placed in a memory cgroup.
48	60
49	61	2. Wait until the workload accesses its working set.
50	62
51		− 3. Read /sys/kernel/mm/page_idle/bitmap and count the number of bits set. If
52		− one wants to ignore certain types of pages, e.g. mlocked pages since they
53		− are not reclaimable, he or she can filter them out using /proc/kpageflags.
	63	+ 3. Read ``/sys/kernel/mm/page_idle/bitmap`` and count the number of bits set.
	64	+ If one wants to ignore certain types of pages, e.g. mlocked pages since they
	65	+ are not reclaimable, he or she can filter them out using
	66	+ ``/proc/kpageflags``.
	67	+
	68	+See :ref:`Documentation/admin-guide/mm/pagemap.rst <pagemap>` for more
	69	+information about ``/proc/pid/pagemap``, ``/proc/kpageflags``, and
	70	+``/proc/kpagecgroup``.
54	71
55		−See Documentation/vm/pagemap.txt for more information about /proc/pid/pagemap,
56		−/proc/kpageflags, and /proc/kpagecgroup.
	72	+.. _impl_details:
57	73
58		−IMPLEMENTATION DETAILS
	74	+Implementation Details
	75	+======================
59	76
60	77	The kernel internally keeps track of accesses to user memory pages in order to
61	78	reclaim unreferenced pages first on memory shortage conditions. A page is
@@ -77,7 +94,8 @@ When a dirty page is written to swap or disk as a result of memory reclaim or · 15 unmodified lines
77	94	exceeding the dirty memory limit, it is not marked referenced.
78	95
79	96	The idle memory tracking feature adds a new page flag, the Idle flag. This flag
80		−is set manually, by writing to /sys/kernel/mm/page_idle/bitmap (see the USER API
	97	+is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` (see the
	98	+:ref:`User API <user_api>`
81	99	section), and cleared automatically whenever a page is referenced as defined
82	100	above.
83	101

Documentation/admin-guide/mm/index.rst+36−0

@@ -0,0 +1,36 @@
	1	+=================
	2	+Memory Management
	3	+=================
	4	+
	5	+Linux memory management subsystem is responsible, as the name implies,
	6	+for managing the memory in the system. This includes implemnetation of
	7	+virtual memory and demand paging, memory allocation both for kernel
	8	+internal structures and user space programms, mapping of files into
	9	+processes address space and many other cool things.
	10	+
	11	+Linux memory management is a complex system with many configurable
	12	+settings. Most of these settings are available via ``/proc``
	13	+filesystem and can be quired and adjusted using ``sysctl``. These APIs
	14	+are described in Documentation/sysctl/vm.txt and in `man 5 proc`_.
	15	+
	16	+.. _man 5 proc: http://man7.org/linux/man-pages/man5/proc.5.html
	17	+
	18	+Linux memory management has its own jargon and if you are not yet
	19	+familiar with it, consider reading
	20	+:ref:`Documentation/admin-guide/mm/concepts.rst <mm_concepts>`.
	21	+
	22	+Here we document in detail how to interact with various mechanisms in
	23	+the Linux memory management.
	24	+
	25	+.. toctree::
	26	+ :maxdepth: 1
	27	+
	28	+ concepts
	29	+ hugetlbpage
	30	+ idle_page_tracking
	31	+ ksm
	32	+ numa_memory_policy
	33	+ pagemap
	34	+ soft-dirty
	35	+ transhuge
	36	+ userfaultfd

Documentation/admin-guide/mm/ksm.rst+189−0

@@ -0,0 +1,189 @@
	1	+.. _admin_guide_ksm:
	2	+
	3	+=======================
	4	+Kernel Samepage Merging
	5	+=======================
	6	+
	7	+Overview
	8	+========
	9	+
	10	+KSM is a memory-saving de-duplication feature, enabled by CONFIG_KSM=y,
	11	+added to the Linux kernel in 2.6.32. See ``mm/ksm.c`` for its implementation,
	12	+and http://lwn.net/Articles/306704/ and http://lwn.net/Articles/330589/
	13	+
	14	+KSM was originally developed for use with KVM (where it was known as
	15	+Kernel Shared Memory), to fit more virtual machines into physical memory,
	16	+by sharing the data common between them. But it can be useful to any
	17	+application which generates many instances of the same data.
	18	+
	19	+The KSM daemon ksmd periodically scans those areas of user memory
	20	+which have been registered with it, looking for pages of identical
	21	+content which can be replaced by a single write-protected page (which
	22	+is automatically copied if a process later wants to update its
	23	+content). The amount of pages that KSM daemon scans in a single pass
	24	+and the time between the passes are configured using :ref:`sysfs
	25	+intraface <ksm_sysfs>`
	26	+
	27	+KSM only merges anonymous (private) pages, never pagecache (file) pages.
	28	+KSM's merged pages were originally locked into kernel memory, but can now
	29	+be swapped out just like other user pages (but sharing is broken when they
	30	+are swapped back in: ksmd must rediscover their identity and merge again).
	31	+
	32	+Controlling KSM with madvise
	33	+============================
	34	+
	35	+KSM only operates on those areas of address space which an application
	36	+has advised to be likely candidates for merging, by using the madvise(2)
	37	+system call::
	38	+
	39	+ int madvise(addr, length, MADV_MERGEABLE)
	40	+
	41	+The app may call
	42	+
	43	+::
	44	+
	45	+ int madvise(addr, length, MADV_UNMERGEABLE)
	46	+
	47	+to cancel that advice and restore unshared pages: whereupon KSM
	48	+unmerges whatever it merged in that range. Note: this unmerging call
	49	+may suddenly require more memory than is available - possibly failing
	50	+with EAGAIN, but more probably arousing the Out-Of-Memory killer.
	51	+
	52	+If KSM is not configured into the running kernel, madvise MADV_MERGEABLE
	53	+and MADV_UNMERGEABLE simply fail with EINVAL. If the running kernel was
	54	+built with CONFIG_KSM=y, those calls will normally succeed: even if the
	55	+the KSM daemon is not currently running, MADV_MERGEABLE still registers
	56	+the range for whenever the KSM daemon is started; even if the range
	57	+cannot contain any pages which KSM could actually merge; even if
	58	+MADV_UNMERGEABLE is applied to a range which was never MADV_MERGEABLE.
	59	+
	60	+If a region of memory must be split into at least one new MADV_MERGEABLE
	61	+or MADV_UNMERGEABLE region, the madvise may return ENOMEM if the process
	62	+will exceed ``vm.max_map_count`` (see Documentation/sysctl/vm.txt).
	63	+
	64	+Like other madvise calls, they are intended for use on mapped areas of
	65	+the user address space: they will report ENOMEM if the specified range
	66	+includes unmapped gaps (though working on the intervening mapped areas),
	67	+and might fail with EAGAIN if not enough memory for internal structures.
	68	+
	69	+Applications should be considerate in their use of MADV_MERGEABLE,
	70	+restricting its use to areas likely to benefit. KSM's scans may use a lot
	71	+of processing power: some installations will disable KSM for that reason.
	72	+
	73	+.. _ksm_sysfs:
	74	+
	75	+KSM daemon sysfs interface
	76	+==========================
	77	+
	78	+The KSM daemon is controlled by sysfs files in ``/sys/kernel/mm/ksm/``,
	79	+readable by all but writable only by root:
	80	+
	81	+pages_to_scan
	82	+ how many pages to scan before ksmd goes to sleep
	83	+ e.g. ``echo 100 > /sys/kernel/mm/ksm/pages_to_scan``.
	84	+
	85	+ Default: 100 (chosen for demonstration purposes)
	86	+
	87	+sleep_millisecs
	88	+ how many milliseconds ksmd should sleep before next scan
	89	+ e.g. ``echo 20 > /sys/kernel/mm/ksm/sleep_millisecs``
	90	+
	91	+ Default: 20 (chosen for demonstration purposes)
	92	+
	93	+merge_across_nodes
	94	+ specifies if pages from different NUMA nodes can be merged.
	95	+ When set to 0, ksm merges only pages which physically reside
	96	+ in the memory area of same NUMA node. That brings lower
	97	+ latency to access of shared pages. Systems with more nodes, at
	98	+ significant NUMA distances, are likely to benefit from the
	99	+ lower latency of setting 0. Smaller systems, which need to
	100	+ minimize memory usage, are likely to benefit from the greater
	101	+ sharing of setting 1 (default). You may wish to compare how
	102	+ your system performs under each setting, before deciding on
	103	+ which to use. ``merge_across_nodes`` setting can be changed only
	104	+ when there are no ksm shared pages in the system: set run 2 to
	105	+ unmerge pages first, then to 1 after changing
	106	+ ``merge_across_nodes``, to remerge according to the new setting.
	107	+
	108	+ Default: 1 (merging across nodes as in earlier releases)
	109	+
	110	+run
	111	+ * set to 0 to stop ksmd from running but keep merged pages,
	112	+ * set to 1 to run ksmd e.g. ``echo 1 > /sys/kernel/mm/ksm/run``,
	113	+ * set to 2 to stop ksmd and unmerge all pages currently merged, but
	114	+ leave mergeable areas registered for next run.
	115	+
	116	+ Default: 0 (must be changed to 1 to activate KSM, except if
	117	+ CONFIG_SYSFS is disabled)
	118	+
	119	+use_zero_pages
	120	+ specifies whether empty pages (i.e. allocated pages that only
	121	+ contain zeroes) should be treated specially. When set to 1,
	122	+ empty pages are merged with the kernel zero page(s) instead of
	123	+ with each other as it would happen normally. This can improve
	124	+ the performance on architectures with coloured zero pages,
	125	+ depending on the workload. Care should be taken when enabling
	126	+ this setting, as it can potentially degrade the performance of
	127	+ KSM for some workloads, for example if the checksums of pages
	128	+ candidate for merging match the checksum of an empty
	129	+ page. This setting can be changed at any time, it is only
	130	+ effective for pages merged after the change.
	131	+
	132	+ Default: 0 (normal KSM behaviour as in earlier releases)
	133	+
	134	+max_page_sharing
	135	+ Maximum sharing allowed for each KSM page. This enforces a
	136	+ deduplication limit to avoid high latency for virtual memory
	137	+ operations that involve traversal of the virtual mappings that
	138	+ share the KSM page. The minimum value is 2 as a newly created
	139	+ KSM page will have at least two sharers. The higher this value
	140	+ the faster KSM will merge the memory and the higher the
	141	+ deduplication factor will be, but the slower the worst case
	142	+ virtual mappings traversal could be for any given KSM
	143	+ page. Slowing down this traversal means there will be higher
	144	+ latency for certain virtual memory operations happening during
	145	+ swapping, compaction, NUMA balancing and page migration, in
	146	+ turn decreasing responsiveness for the caller of those virtual
	147	+ memory operations. The scheduler latency of other tasks not
	148	+ involved with the VM operations doing the virtual mappings
	149	+ traversal is not affected by this parameter as these
	150	+ traversals are always schedule friendly themselves.
	151	+
	152	+stable_node_chains_prune_millisecs
	153	+ specifies how frequently KSM checks the metadata of the pages
	154	+ that hit the deduplication limit for stale information.
	155	+ Smaller milllisecs values will free up the KSM metadata with
	156	+ lower latency, but they will make ksmd use more CPU during the
	157	+ scan. It's a noop if not a single KSM page hit the
	158	+ ``max_page_sharing`` yet.
	159	+
	160	+The effectiveness of KSM and MADV_MERGEABLE is shown in ``/sys/kernel/mm/ksm/``:
	161	+
	162	+pages_shared
	163	+ how many shared pages are being used
	164	+pages_sharing
	165	+ how many more sites are sharing them i.e. how much saved
	166	+pages_unshared
	167	+ how many pages unique but repeatedly checked for merging
	168	+pages_volatile
	169	+ how many pages changing too fast to be placed in a tree
	170	+full_scans
	171	+ how many times all mergeable areas have been scanned
	172	+stable_node_chains
	173	+ the number of KSM pages that hit the ``max_page_sharing`` limit
	174	+stable_node_dups
	175	+ number of duplicated KSM pages
	176	+
	177	+A high ratio of ``pages_sharing`` to ``pages_shared`` indicates good
	178	+sharing, but a high ratio of ``pages_unshared`` to ``pages_sharing``
	179	+indicates wasted effort. ``pages_volatile`` embraces several
	180	+different kinds of activity, but a high proportion there would also
	181	+indicate poor use of madvise MADV_MERGEABLE.
	182	+
	183	+The maximum possible ``pages_sharing/pages_shared`` ratio is limited by the
	184	+``max_page_sharing`` tunable. To increase the ratio ``max_page_sharing`` must
	185	+be increased accordingly.
	186	+
	187	+--
	188	+Izik Eidus,
	189	+Hugh Dickins, 17 Nov 2009

Documentation/admin-guide/mm/numa_memory_policy.rst+495−0

@@ -0,0 +1,495 @@
	1	+.. _numa_memory_policy:
	2	+
	3	+==================
	4	+NUMA Memory Policy
	5	+==================
	6	+
	7	+What is NUMA Memory Policy?
	8	+============================
	9	+
	10	+In the Linux kernel, "memory policy" determines from which node the kernel will
	11	+allocate memory in a NUMA system or in an emulated NUMA system. Linux has
	12	+supported platforms with Non-Uniform Memory Access architectures since 2.4.?.
	13	+The current memory policy support was added to Linux 2.6 around May 2004. This
	14	+document attempts to describe the concepts and APIs of the 2.6 memory policy
	15	+support.
	16	+
	17	+Memory policies should not be confused with cpusets
	18	+(``Documentation/cgroup-v1/cpusets.txt``)
	19	+which is an administrative mechanism for restricting the nodes from which
	20	+memory may be allocated by a set of processes. Memory policies are a
	21	+programming interface that a NUMA-aware application can take advantage of. When
	22	+both cpusets and policies are applied to a task, the restrictions of the cpuset
	23	+takes priority. See :ref:`Memory Policies and cpusets <mem_pol_and_cpusets>`
	24	+below for more details.
	25	+
	26	+Memory Policy Concepts
	27	+======================
	28	+
	29	+Scope of Memory Policies
	30	+------------------------
	31	+
	32	+The Linux kernel supports _scopes_ of memory policy, described here from
	33	+most general to most specific:
	34	+
	35	+System Default Policy
	36	+ this policy is "hard coded" into the kernel. It is the policy
	37	+ that governs all page allocations that aren't controlled by
	38	+ one of the more specific policy scopes discussed below. When
	39	+ the system is "up and running", the system default policy will
	40	+ use "local allocation" described below. However, during boot
	41	+ up, the system default policy will be set to interleave
	42	+ allocations across all nodes with "sufficient" memory, so as
	43	+ not to overload the initial boot node with boot-time
	44	+ allocations.
	45	+
	46	+Task/Process Policy
	47	+ this is an optional, per-task policy. When defined for a
	48	+ specific task, this policy controls all page allocations made
	49	+ by or on behalf of the task that aren't controlled by a more
	50	+ specific scope. If a task does not define a task policy, then
	51	+ all page allocations that would have been controlled by the
	52	+ task policy "fall back" to the System Default Policy.
	53	+
	54	+ The task policy applies to the entire address space of a task. Thus,
	55	+ it is inheritable, and indeed is inherited, across both fork()
	56	+ [clone() w/o the CLONE_VM flag] and exec*(). This allows a parent task
	57	+ to establish the task policy for a child task exec()'d from an
	58	+ executable image that has no awareness of memory policy. See the
	59	+ :ref:`Memory Policy APIs <memory_policy_apis>` section,
	60	+ below, for an overview of the system call
	61	+ that a task may use to set/change its task/process policy.
	62	+
	63	+ In a multi-threaded task, task policies apply only to the thread
	64	+ [Linux kernel task] that installs the policy and any threads
	65	+ subsequently created by that thread. Any sibling threads existing
	66	+ at the time a new task policy is installed retain their current
	67	+ policy.
	68	+
	69	+ A task policy applies only to pages allocated after the policy is
	70	+ installed. Any pages already faulted in by the task when the task
	71	+ changes its task policy remain where they were allocated based on
	72	+ the policy at the time they were allocated.
	73	+
	74	+.. _vma_policy:
	75	+
	76	+VMA Policy
	77	+ A "VMA" or "Virtual Memory Area" refers to a range of a task's
	78	+ virtual address space. A task may define a specific policy for a range
	79	+ of its virtual address space. See the
	80	+ :ref:`Memory Policy APIs <memory_policy_apis>` section,
	81	+ below, for an overview of the mbind() system call used to set a VMA
	82	+ policy.
	83	+
	84	+ A VMA policy will govern the allocation of pages that back
	85	+ this region of the address space. Any regions of the task's
	86	+ address space that don't have an explicit VMA policy will fall
	87	+ back to the task policy, which may itself fall back to the
	88	+ System Default Policy.
	89	+
	90	+ VMA policies have a few complicating details:
	91	+
	92	+ * VMA policy applies ONLY to anonymous pages. These include
	93	+ pages allocated for anonymous segments, such as the task
	94	+ stack and heap, and any regions of the address space
	95	+ mmap()ed with the MAP_ANONYMOUS flag. If a VMA policy is
	96	+ applied to a file mapping, it will be ignored if the mapping
	97	+ used the MAP_SHARED flag. If the file mapping used the
	98	+ MAP_PRIVATE flag, the VMA policy will only be applied when
	99	+ an anonymous page is allocated on an attempt to write to the
	100	+ mapping-- i.e., at Copy-On-Write.
	101	+
	102	+ * VMA policies are shared between all tasks that share a
	103	+ virtual address space--a.k.a. threads--independent of when
	104	+ the policy is installed; and they are inherited across
	105	+ fork(). However, because VMA policies refer to a specific
	106	+ region of a task's address space, and because the address
	107	+ space is discarded and recreated on exec*(), VMA policies
	108	+ are NOT inheritable across exec(). Thus, only NUMA-aware
	109	+ applications may use VMA policies.
	110	+
	111	+ * A task may install a new VMA policy on a sub-range of a
	112	+ previously mmap()ed region. When this happens, Linux splits
	113	+ the existing virtual memory area into 2 or 3 VMAs, each with
	114	+ it's own policy.
	115	+
	116	+ * By default, VMA policy applies only to pages allocated after
	117	+ the policy is installed. Any pages already faulted into the
	118	+ VMA range remain where they were allocated based on the
	119	+ policy at the time they were allocated. However, since
	120	+ 2.6.16, Linux supports page migration via the mbind() system
	121	+ call, so that page contents can be moved to match a newly
	122	+ installed policy.
	123	+
	124	+Shared Policy
	125	+ Conceptually, shared policies apply to "memory objects" mapped
	126	+ shared into one or more tasks' distinct address spaces. An
	127	+ application installs shared policies the same way as VMA
	128	+ policies--using the mbind() system call specifying a range of
	129	+ virtual addresses that map the shared object. However, unlike
	130	+ VMA policies, which can be considered to be an attribute of a
	131	+ range of a task's address space, shared policies apply
	132	+ directly to the shared object. Thus, all tasks that attach to
	133	+ the object share the policy, and all pages allocated for the
	134	+ shared object, by any task, will obey the shared policy.
	135	+
	136	+ As of 2.6.22, only shared memory segments, created by shmget() or
	137	+ mmap(MAP_ANONYMOUS\|MAP_SHARED), support shared policy. When shared
	138	+ policy support was added to Linux, the associated data structures were
	139	+ added to hugetlbfs shmem segments. At the time, hugetlbfs did not
	140	+ support allocation at fault time--a.k.a lazy allocation--so hugetlbfs
	141	+ shmem segments were never "hooked up" to the shared policy support.
	142	+ Although hugetlbfs segments now support lazy allocation, their support
	143	+ for shared policy has not been completed.
	144	+
	145	+ As mentioned above in :ref:`VMA policies <vma_policy>` section,
	146	+ allocations of page cache pages for regular files mmap()ed
	147	+ with MAP_SHARED ignore any VMA policy installed on the virtual
	148	+ address range backed by the shared file mapping. Rather,
	149	+ shared page cache pages, including pages backing private
	150	+ mappings that have not yet been written by the task, follow
	151	+ task policy, if any, else System Default Policy.
	152	+
	153	+ The shared policy infrastructure supports different policies on subset
	154	+ ranges of the shared object. However, Linux still splits the VMA of
	155	+ the task that installs the policy for each range of distinct policy.
	156	+ Thus, different tasks that attach to a shared memory segment can have
	157	+ different VMA configurations mapping that one shared object. This
	158	+ can be seen by examining the /proc/<pid>/numa_maps of tasks sharing
	159	+ a shared memory region, when one task has installed shared policy on
	160	+ one or more ranges of the region.
	161	+
	162	+Components of Memory Policies
	163	+-----------------------------
	164	+
	165	+A NUMA memory policy consists of a "mode", optional mode flags, and
	166	+an optional set of nodes. The mode determines the behavior of the
	167	+policy, the optional mode flags determine the behavior of the mode,
	168	+and the optional set of nodes can be viewed as the arguments to the
	169	+policy behavior.
	170	+
	171	+Internally, memory policies are implemented by a reference counted
	172	+structure, struct mempolicy. Details of this structure will be
	173	+discussed in context, below, as required to explain the behavior.
	174	+
	175	+NUMA memory policy supports the following 4 behavioral modes:
	176	+
	177	+Default Mode--MPOL_DEFAULT
	178	+ This mode is only used in the memory policy APIs. Internally,
	179	+ MPOL_DEFAULT is converted to the NULL memory policy in all
	180	+ policy scopes. Any existing non-default policy will simply be
	181	+ removed when MPOL_DEFAULT is specified. As a result,
	182	+ MPOL_DEFAULT means "fall back to the next most specific policy
	183	+ scope."
	184	+
	185	+ For example, a NULL or default task policy will fall back to the
	186	+ system default policy. A NULL or default vma policy will fall
	187	+ back to the task policy.
	188	+
	189	+ When specified in one of the memory policy APIs, the Default mode
	190	+ does not use the optional set of nodes.
	191	+
	192	+ It is an error for the set of nodes specified for this policy to
	193	+ be non-empty.
	194	+
	195	+MPOL_BIND
	196	+ This mode specifies that memory must come from the set of
	197	+ nodes specified by the policy. Memory will be allocated from
	198	+ the node in the set with sufficient free memory that is
	199	+ closest to the node where the allocation takes place.
	200	+
	201	+MPOL_PREFERRED
	202	+ This mode specifies that the allocation should be attempted
	203	+ from the single node specified in the policy. If that
	204	+ allocation fails, the kernel will search other nodes, in order
	205	+ of increasing distance from the preferred node based on
	206	+ information provided by the platform firmware.
	207	+
	208	+ Internally, the Preferred policy uses a single node--the
	209	+ preferred_node member of struct mempolicy. When the internal
	210	+ mode flag MPOL_F_LOCAL is set, the preferred_node is ignored
	211	+ and the policy is interpreted as local allocation. "Local"
	212	+ allocation policy can be viewed as a Preferred policy that
	213	+ starts at the node containing the cpu where the allocation
	214	+ takes place.
	215	+
	216	+ It is possible for the user to specify that local allocation
	217	+ is always preferred by passing an empty nodemask with this
	218	+ mode. If an empty nodemask is passed, the policy cannot use
	219	+ the MPOL_F_STATIC_NODES or MPOL_F_RELATIVE_NODES flags
	220	+ described below.
	221	+
	222	+MPOL_INTERLEAVED
	223	+ This mode specifies that page allocations be interleaved, on a
	224	+ page granularity, across the nodes specified in the policy.
	225	+ This mode also behaves slightly differently, based on the
	226	+ context where it is used:
	227	+
	228	+ For allocation of anonymous pages and shared memory pages,
	229	+ Interleave mode indexes the set of nodes specified by the
	230	+ policy using the page offset of the faulting address into the
	231	+ segment [VMA] containing the address modulo the number of
	232	+ nodes specified by the policy. It then attempts to allocate a
	233	+ page, starting at the selected node, as if the node had been
	234	+ specified by a Preferred policy or had been selected by a
	235	+ local allocation. That is, allocation will follow the per
	236	+ node zonelist.
	237	+
	238	+ For allocation of page cache pages, Interleave mode indexes
	239	+ the set of nodes specified by the policy using a node counter
	240	+ maintained per task. This counter wraps around to the lowest
	241	+ specified node after it reaches the highest specified node.
	242	+ This will tend to spread the pages out over the nodes
	243	+ specified by the policy based on the order in which they are
	244	+ allocated, rather than based on any page offset into an
	245	+ address range or file. During system boot up, the temporary
	246	+ interleaved system default policy works in this mode.
	247	+
	248	+NUMA memory policy supports the following optional mode flags:
	249	+
	250	+MPOL_F_STATIC_NODES
	251	+ This flag specifies that the nodemask passed by
	252	+ the user should not be remapped if the task or VMA's set of allowed
	253	+ nodes changes after the memory policy has been defined.
	254	+
	255	+ Without this flag, any time a mempolicy is rebound because of a
	256	+ change in the set of allowed nodes, the node (Preferred) or
	257	+ nodemask (Bind, Interleave) is remapped to the new set of
	258	+ allowed nodes. This may result in nodes being used that were
	259	+ previously undesired.
	260	+
	261	+ With this flag, if the user-specified nodes overlap with the
	262	+ nodes allowed by the task's cpuset, then the memory policy is
	263	+ applied to their intersection. If the two sets of nodes do not
	264	+ overlap, the Default policy is used.
	265	+
	266	+ For example, consider a task that is attached to a cpuset with
	267	+ mems 1-3 that sets an Interleave policy over the same set. If
	268	+ the cpuset's mems change to 3-5, the Interleave will now occur
	269	+ over nodes 3, 4, and 5. With this flag, however, since only node
	270	+ 3 is allowed from the user's nodemask, the "interleave" only
	271	+ occurs over that node. If no nodes from the user's nodemask are
	272	+ now allowed, the Default behavior is used.
	273	+
	274	+ MPOL_F_STATIC_NODES cannot be combined with the
	275	+ MPOL_F_RELATIVE_NODES flag. It also cannot be used for
	276	+ MPOL_PREFERRED policies that were created with an empty nodemask
	277	+ (local allocation).
	278	+
	279	+MPOL_F_RELATIVE_NODES
	280	+ This flag specifies that the nodemask passed
	281	+ by the user will be mapped relative to the set of the task or VMA's
	282	+ set of allowed nodes. The kernel stores the user-passed nodemask,
	283	+ and if the allowed nodes changes, then that original nodemask will
	284	+ be remapped relative to the new set of allowed nodes.
	285	+
	286	+ Without this flag (and without MPOL_F_STATIC_NODES), anytime a
	287	+ mempolicy is rebound because of a change in the set of allowed
	288	+ nodes, the node (Preferred) or nodemask (Bind, Interleave) is
	289	+ remapped to the new set of allowed nodes. That remap may not
	290	+ preserve the relative nature of the user's passed nodemask to its
	291	+ set of allowed nodes upon successive rebinds: a nodemask of
	292	+ 1,3,5 may be remapped to 7-9 and then to 1-3 if the set of
	293	+ allowed nodes is restored to its original state.
	294	+
	295	+ With this flag, the remap is done so that the node numbers from
	296	+ the user's passed nodemask are relative to the set of allowed
	297	+ nodes. In other words, if nodes 0, 2, and 4 are set in the user's
	298	+ nodemask, the policy will be effected over the first (and in the
	299	+ Bind or Interleave case, the third and fifth) nodes in the set of
	300	+ allowed nodes. The nodemask passed by the user represents nodes
	301	+ relative to task or VMA's set of allowed nodes.
	302	+
	303	+ If the user's nodemask includes nodes that are outside the range
	304	+ of the new set of allowed nodes (for example, node 5 is set in
	305	+ the user's nodemask when the set of allowed nodes is only 0-3),
	306	+ then the remap wraps around to the beginning of the nodemask and,
	307	+ if not already set, sets the node in the mempolicy nodemask.
	308	+
	309	+ For example, consider a task that is attached to a cpuset with
	310	+ mems 2-5 that sets an Interleave policy over the same set with
	311	+ MPOL_F_RELATIVE_NODES. If the cpuset's mems change to 3-7, the
	312	+ interleave now occurs over nodes 3,5-7. If the cpuset's mems
	313	+ then change to 0,2-3,5, then the interleave occurs over nodes
	314	+ 0,2-3,5.
	315	+
	316	+ Thanks to the consistent remapping, applications preparing
	317	+ nodemasks to specify memory policies using this flag should
	318	+ disregard their current, actual cpuset imposed memory placement
	319	+ and prepare the nodemask as if they were always located on
	320	+ memory nodes 0 to N-1, where N is the number of memory nodes the
	321	+ policy is intended to manage. Let the kernel then remap to the
	322	+ set of memory nodes allowed by the task's cpuset, as that may
	323	+ change over time.
	324	+
	325	+ MPOL_F_RELATIVE_NODES cannot be combined with the
	326	+ MPOL_F_STATIC_NODES flag. It also cannot be used for
	327	+ MPOL_PREFERRED policies that were created with an empty nodemask
	328	+ (local allocation).
	329	+
	330	+Memory Policy Reference Counting
	331	+================================
	332	+
	333	+To resolve use/free races, struct mempolicy contains an atomic reference
	334	+count field. Internal interfaces, mpol_get()/mpol_put() increment and
	335	+decrement this reference count, respectively. mpol_put() will only free
	336	+the structure back to the mempolicy kmem cache when the reference count
	337	+goes to zero.
	338	+
	339	+When a new memory policy is allocated, its reference count is initialized
	340	+to '1', representing the reference held by the task that is installing the
	341	+new policy. When a pointer to a memory policy structure is stored in another
	342	+structure, another reference is added, as the task's reference will be dropped
	343	+on completion of the policy installation.
	344	+
	345	+During run-time "usage" of the policy, we attempt to minimize atomic operations
	346	+on the reference count, as this can lead to cache lines bouncing between cpus
	347	+and NUMA nodes. "Usage" here means one of the following:
	348	+
	349	+1) querying of the policy, either by the task itself [using the get_mempolicy()
	350	+ API discussed below] or by another task using the /proc/<pid>/numa_maps
	351	+ interface.
	352	+
	353	+2) examination of the policy to determine the policy mode and associated node
	354	+ or node lists, if any, for page allocation. This is considered a "hot
	355	+ path". Note that for MPOL_BIND, the "usage" extends across the entire
	356	+ allocation process, which may sleep during page reclaimation, because the
	357	+ BIND policy nodemask is used, by reference, to filter ineligible nodes.
	358	+
	359	+We can avoid taking an extra reference during the usages listed above as
	360	+follows:
	361	+
	362	+1) we never need to get/free the system default policy as this is never
	363	+ changed nor freed, once the system is up and running.
	364	+
	365	+2) for querying the policy, we do not need to take an extra reference on the
	366	+ target task's task policy nor vma policies because we always acquire the
	367	+ task's mm's mmap_sem for read during the query. The set_mempolicy() and
	368	+ mbind() APIs [see below] always acquire the mmap_sem for write when
	369	+ installing or replacing task or vma policies. Thus, there is no possibility
	370	+ of a task or thread freeing a policy while another task or thread is
	371	+ querying it.
	372	+
	373	+3) Page allocation usage of task or vma policy occurs in the fault path where
	374	+ we hold them mmap_sem for read. Again, because replacing the task or vma
	375	+ policy requires that the mmap_sem be held for write, the policy can't be
	376	+ freed out from under us while we're using it for page allocation.
	377	+
	378	+4) Shared policies require special consideration. One task can replace a
	379	+ shared memory policy while another task, with a distinct mmap_sem, is
	380	+ querying or allocating a page based on the policy. To resolve this
	381	+ potential race, the shared policy infrastructure adds an extra reference
	382	+ to the shared policy during lookup while holding a spin lock on the shared
	383	+ policy management structure. This requires that we drop this extra
	384	+ reference when we're finished "using" the policy. We must drop the
	385	+ extra reference on shared policies in the same query/allocation paths
	386	+ used for non-shared policies. For this reason, shared policies are marked
	387	+ as such, and the extra reference is dropped "conditionally"--i.e., only
	388	+ for shared policies.
	389	+
	390	+ Because of this extra reference counting, and because we must lookup
	391	+ shared policies in a tree structure under spinlock, shared policies are
	392	+ more expensive to use in the page allocation path. This is especially
	393	+ true for shared policies on shared memory regions shared by tasks running
	394	+ on different NUMA nodes. This extra overhead can be avoided by always
	395	+ falling back to task or system default policy for shared memory regions,
	396	+ or by prefaulting the entire shared memory region into memory and locking
	397	+ it down. However, this might not be appropriate for all applications.
	398	+
	399	+.. _memory_policy_apis:
	400	+
	401	+Memory Policy APIs
	402	+==================
	403	+
	404	+Linux supports 3 system calls for controlling memory policy. These APIS
	405	+always affect only the calling task, the calling task's address space, or
	406	+some shared object mapped into the calling task's address space.
	407	+
	408	+.. note::
	409	+ the headers that define these APIs and the parameter data types for
	410	+ user space applications reside in a package that is not part of the
	411	+ Linux kernel. The kernel system call interfaces, with the 'sys\_'
	412	+ prefix, are defined in <linux/syscalls.h>; the mode and flag
	413	+ definitions are defined in <linux/mempolicy.h>.
	414	+
	415	+Set [Task] Memory Policy::
	416	+
	417	+ long set_mempolicy(int mode, const unsigned long *nmask,
	418	+ unsigned long maxnode);
	419	+
	420	+Set's the calling task's "task/process memory policy" to mode
	421	+specified by the 'mode' argument and the set of nodes defined by
	422	+'nmask'. 'nmask' points to a bit mask of node ids containing at least
	423	+'maxnode' ids. Optional mode flags may be passed by combining the
	424	+'mode' argument with the flag (for example: MPOL_INTERLEAVE \|
	425	+MPOL_F_STATIC_NODES).
	426	+
	427	+See the set_mempolicy(2) man page for more details
	428	+
	429	+
	430	+Get [Task] Memory Policy or Related Information::
	431	+
	432	+ long get_mempolicy(int *mode,
	433	+ const unsigned long *nmask, unsigned long maxnode,
	434	+ void *addr, int flags);
	435	+
	436	+Queries the "task/process memory policy" of the calling task, or the
	437	+policy or location of a specified virtual address, depending on the
	438	+'flags' argument.
	439	+
	440	+See the get_mempolicy(2) man page for more details
	441	+
	442	+
	443	+Install VMA/Shared Policy for a Range of Task's Address Space::
	444	+
	445	+ long mbind(void *start, unsigned long len, int mode,
	446	+ const unsigned long *nmask, unsigned long maxnode,
	447	+ unsigned flags);
	448	+
	449	+mbind() installs the policy specified by (mode, nmask, maxnodes) as a
	450	+VMA policy for the range of the calling task's address space specified
	451	+by the 'start' and 'len' arguments. Additional actions may be
	452	+requested via the 'flags' argument.
	453	+
	454	+See the mbind(2) man page for more details.
	455	+
	456	+Memory Policy Command Line Interface
	457	+====================================
	458	+
	459	+Although not strictly part of the Linux implementation of memory policy,
	460	+a command line tool, numactl(8), exists that allows one to:
	461	+
	462	++ set the task policy for a specified program via set_mempolicy(2), fork(2) and
	463	+ exec(2)
	464	+
	465	++ set the shared policy for a shared memory segment via mbind(2)
	466	+
	467	+The numactl(8) tool is packaged with the run-time version of the library
	468	+containing the memory policy system call wrappers. Some distributions
	469	+package the headers and compile-time libraries in a separate development
	470	+package.
	471	+
	472	+.. _mem_pol_and_cpusets:
	473	+
	474	+Memory Policies and cpusets
	475	+===========================
	476	+
	477	+Memory policies work within cpusets as described above. For memory policies
	478	+that require a node or set of nodes, the nodes are restricted to the set of
	479	+nodes whose memories are allowed by the cpuset constraints. If the nodemask
	480	+specified for the policy contains nodes that are not allowed by the cpuset and
	481	+MPOL_F_RELATIVE_NODES is not used, the intersection of the set of nodes
	482	+specified for the policy and the set of nodes with memory is used. If the
	483	+result is the empty set, the policy is considered invalid and cannot be
	484	+installed. If MPOL_F_RELATIVE_NODES is used, the policy's nodes are mapped
	485	+onto and folded into the task's set of allowed nodes as previously described.
	486	+
	487	+The interaction of memory policies and cpusets can be problematic when tasks
	488	+in two cpusets share access to a memory region, such as shared memory segments
	489	+created by shmget() of mmap() with the MAP_ANONYMOUS and MAP_SHARED flags, and
	490	+any of the tasks install shared policy on the region, only nodes whose
	491	+memories are allowed in both cpusets may be used in the policies. Obtaining
	492	+this information requires "stepping outside" the memory policy APIs to use the
	493	+cpuset information and requires that one know in what cpusets other task might
	494	+be attaching to the shared region. Furthermore, if the cpusets' allowed
	495	+memory sets are disjoint, "local" allocation is the only valid policy.

Documentation/admin-guide/mm/pagemap.rstrenamed from Documentation/vm/pagemap.txt+101−83

@@ -1,21 +1,25 @@
1		−pagemap, from the userspace perspective
2		−---------------------------------------
	1	+.. _pagemap:
	2	+
	3	+=============================
	4	+Examining Process Page Tables
	5	+=============================
3	6
4	7	pagemap is a new (as of 2.6.25) set of interfaces in the kernel that allow
5	8	userspace programs to examine the page tables and related information by
6		−reading files in /proc.
	9	+reading files in ``/proc``.
7	10
8	11	There are four components to pagemap:
9	12
10		− * /proc/pid/pagemap. This file lets a userspace process find out which
	13	+ * ``/proc/pid/pagemap``. This file lets a userspace process find out which
11	14	physical frame each virtual page is mapped to. It contains one 64-bit
12	15	value for each virtual page, containing the following data (from
13		− fs/proc/task_mmu.c, above pagemap_read):
	16	+ ``fs/proc/task_mmu.c``, above pagemap_read):
14	17
15	18	* Bits 0-54 page frame number (PFN) if present
16	19	* Bits 0-4 swap type if swapped
17	20	* Bits 5-54 swap offset if swapped
18		− * Bit 55 pte is soft-dirty (see Documentation/vm/soft-dirty.txt)
	21	+ * Bit 55 pte is soft-dirty (see
	22	+ :ref:`Documentation/admin-guide/mm/soft-dirty.rst <soft_dirty>`)
19	23	* Bit 56 page exclusively mapped (since 4.2)
20	24	* Bits 57-60 zero
21	25	* Bit 61 page is file-page or shared-anon (since 3.5)
@@ -33,28 +37,28 @@ There are four components to pagemap: · 11 unmodified lines
33	37	precisely which pages are mapped (or in swap) and comparing mapped
34	38	pages between processes.
35	39
36		− Efficient users of this interface will use /proc/pid/maps to
	40	+ Efficient users of this interface will use ``/proc/pid/maps`` to
37	41	determine which areas of memory are actually mapped and llseek to
38	42	skip over unmapped regions.
39	43
40		− * /proc/kpagecount. This file contains a 64-bit count of the number of
	44	+ * ``/proc/kpagecount``. This file contains a 64-bit count of the number of
41	45	times each page is mapped, indexed by PFN.
42	46
43		− * /proc/kpageflags. This file contains a 64-bit set of flags for each
	47	+ * ``/proc/kpageflags``. This file contains a 64-bit set of flags for each
44	48	page, indexed by PFN.
45	49
46		− The flags are (from fs/proc/page.c, above kpageflags_read):
47		−
48		− 0. LOCKED
49		− 1. ERROR
50		− 2. REFERENCED
51		− 3. UPTODATE
52		− 4. DIRTY
53		− 5. LRU
54		− 6. ACTIVE
55		− 7. SLAB
56		− 8. WRITEBACK
57		− 9. RECLAIM
	50	+ The flags are (from ``fs/proc/page.c``, above kpageflags_read):
	51	+
	52	+ 0. LOCKED
	53	+ 1. ERROR
	54	+ 2. REFERENCED
	55	+ 3. UPTODATE
	56	+ 4. DIRTY
	57	+ 5. LRU
	58	+ 6. ACTIVE
	59	+ 7. SLAB
	60	+ 8. WRITEBACK
	61	+ 9. RECLAIM
58	62	10. BUDDY
59	63	11. MMAP
60	64	12. ANON
@@ -72,98 +76,111 @@ There are four components to pagemap: · 11 unmodified lines
72	76	24. ZERO_PAGE
73	77	25. IDLE
74	78
75		− * /proc/kpagecgroup. This file contains a 64-bit inode number of the
	79	+ * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the
76	80	memory cgroup each page is charged to, indexed by PFN. Only available when
77	81	CONFIG_MEMCG is set.
78	82
79		−Short descriptions to the page flags:
80		−
81		− 0. LOCKED
82		− page is being locked for exclusive access, eg. by undergoing read/write IO
83		−
84		− 7. SLAB
85		− page is managed by the SLAB/SLOB/SLUB/SLQB kernel memory allocator
86		− When compound page is used, SLUB/SLQB will only set this flag on the head
87		− page; SLOB will not flag it at all.
	83	+Short descriptions to the page flags
	84	+====================================
88	85
89		−10. BUDDY
	86	+0 - LOCKED
	87	+ page is being locked for exclusive access, e.g. by undergoing read/write IO
	88	+7 - SLAB
	89	+ page is managed by the SLAB/SLOB/SLUB/SLQB kernel memory allocator
	90	+ When compound page is used, SLUB/SLQB will only set this flag on the head
	91	+ page; SLOB will not flag it at all.
	92	+10 - BUDDY
90	93	a free memory block managed by the buddy system allocator
91	94	The buddy system organizes free memory in blocks of various orders.
92	95	An order N block has 2^N physically contiguous pages, with the BUDDY flag
93	96	set for and _only_ for the first page.
94		−
95		−15. COMPOUND_HEAD
96		−16. COMPOUND_TAIL
	97	+15 - COMPOUND_HEAD
97	98	A compound page with order N consists of 2^N physically contiguous pages.
98	99	A compound page with order 2 takes the form of "HTTT", where H donates its
99	100	head page and T donates its tail page(s). The major consumers of compound
100		− pages are hugeTLB pages (Documentation/vm/hugetlbpage.txt), the SLUB etc.
101		− memory allocators and various device drivers. However in this interface,
102		− only huge/giga pages are made visible to end users.
103		−17. HUGE
	101	+ pages are hugeTLB pages
	102	+ (:ref:`Documentation/admin-guide/mm/hugetlbpage.rst <hugetlbpage>`),
	103	+ the SLUB etc. memory allocators and various device drivers.
	104	+ However in this interface, only huge/giga pages are made visible
	105	+ to end users.
	106	+16 - COMPOUND_TAIL
	107	+ A compound page tail (see description above).
	108	+17 - HUGE
104	109	this is an integral part of a HugeTLB page
105		−
106		−19. HWPOISON
	110	+19 - HWPOISON
107	111	hardware detected memory corruption on this page: don't touch the data!
108		−
109		−20. NOPAGE
	112	+20 - NOPAGE
110	113	no page frame exists at the requested address
111		−
112		−21. KSM
	114	+21 - KSM
113	115	identical memory pages dynamically shared between one or more processes
114		−
115		−22. THP
	116	+22 - THP
116	117	contiguous pages which construct transparent hugepages
117		−
118		−23. BALLOON
	118	+23 - BALLOON
119	119	balloon compaction page
120		−
121		−24. ZERO_PAGE
	120	+24 - ZERO_PAGE
122	121	zero page for pfn_zero or huge_zero page
123		−
124		−25. IDLE
	122	+25 - IDLE
125	123	page has not been accessed since it was marked idle (see
126		− Documentation/vm/idle_page_tracking.txt). Note that this flag may be
127		− stale in case the page was accessed via a PTE. To make sure the flag
128		− is up-to-date one has to read /sys/kernel/mm/page_idle/bitmap first.
129		−
130		− [IO related page flags]
131		− 1. ERROR IO error occurred
132		− 3. UPTODATE page has up-to-date data
133		− ie. for file backed page: (in-memory data revision >= on-disk one)
134		− 4. DIRTY page has been written to, hence contains new data
135		− ie. for file backed page: (in-memory data revision > on-disk one)
136		− 8. WRITEBACK page is being synced to disk
137		−
138		− [LRU related page flags]
139		− 5. LRU page is in one of the LRU lists
140		− 6. ACTIVE page is in the active LRU list
141		−18. UNEVICTABLE page is in the unevictable (non-)LRU list
142		− It is somehow pinned and not a candidate for LRU page reclaims,
143		− eg. ramfs pages, shmctl(SHM_LOCK) and mlock() memory segments
144		− 2. REFERENCED page has been referenced since last LRU list enqueue/requeue
145		− 9. RECLAIM page will be reclaimed soon after its pageout IO completed
146		−11. MMAP a memory mapped page
147		−12. ANON a memory mapped page that is not part of a file
148		−13. SWAPCACHE page is mapped to swap space, ie. has an associated swap entry
149		−14. SWAPBACKED page is backed by swap/RAM
	124	+ :ref:`Documentation/admin-guide/mm/idle_page_tracking.rst <idle_page_tracking>`).
	125	+ Note that this flag may be stale in case the page was accessed via
	126	+ a PTE. To make sure the flag is up-to-date one has to read
	127	+ ``/sys/kernel/mm/page_idle/bitmap`` first.
	128	+
	129	+IO related page flags
	130	+---------------------
	131	+
	132	+1 - ERROR
	133	+ IO error occurred
	134	+3 - UPTODATE
	135	+ page has up-to-date data
	136	+ ie. for file backed page: (in-memory data revision >= on-disk one)
	137	+4 - DIRTY
	138	+ page has been written to, hence contains new data
	139	+ i.e. for file backed page: (in-memory data revision > on-disk one)
	140	+8 - WRITEBACK
	141	+ page is being synced to disk
	142	+
	143	+LRU related page flags
	144	+----------------------
	145	+
	146	+5 - LRU
	147	+ page is in one of the LRU lists
	148	+6 - ACTIVE
	149	+ page is in the active LRU list
	150	+18 - UNEVICTABLE
	151	+ page is in the unevictable (non-)LRU list It is somehow pinned and
	152	+ not a candidate for LRU page reclaims, e.g. ramfs pages,
	153	+ shmctl(SHM_LOCK) and mlock() memory segments
	154	+2 - REFERENCED
	155	+ page has been referenced since last LRU list enqueue/requeue
	156	+9 - RECLAIM
	157	+ page will be reclaimed soon after its pageout IO completed
	158	+11 - MMAP
	159	+ a memory mapped page
	160	+12 - ANON
	161	+ a memory mapped page that is not part of a file
	162	+13 - SWAPCACHE
	163	+ page is mapped to swap space, i.e. has an associated swap entry
	164	+14 - SWAPBACKED
	165	+ page is backed by swap/RAM
150	166
151	167	The page-types tool in the tools/vm directory can be used to query the
152	168	above flags.
153	169
154		−Using pagemap to do something useful:
	170	+Using pagemap to do something useful
	171	+====================================
155	172
156	173	The general procedure for using pagemap to find out about a process' memory
157	174	usage goes like this:
158	175
159		− 1. Read /proc/pid/maps to determine which parts of the memory space are
	176	+ 1. Read ``/proc/pid/maps`` to determine which parts of the memory space are
160	177	mapped to what.
161	178	2. Select the maps you are interested in -- all of them, or a particular
162	179	library, or the stack or the heap, etc.
163		− 3. Open /proc/pid/pagemap and seek to the pages you would like to examine.
	180	+ 3. Open ``/proc/pid/pagemap`` and seek to the pages you would like to examine.
164	181	4. Read a u64 for each page from pagemap.
165		− 5. Open /proc/kpagecount and/or /proc/kpageflags. For each PFN you just
166		− read, seek to that entry in the file, and read the data you want.
	182	+ 5. Open ``/proc/kpagecount`` and/or ``/proc/kpageflags``. For each PFN you
	183	+ just read, seek to that entry in the file, and read the data you want.
167	184
168	185	For example, to find the "unique set size" (USS), which is the amount of
169	186	memory that a process is using that is not shared with any other process,
@@ -171,7 +188,8 @@ you can go through every map in the process, find the PFNs, look those up · 1 unmodified line
171	188	in kpagecount, and tally up the number of pages that are only referenced
172	189	once.
173	190
174		−Other notes:
	191	+Other notes
	192	+===========
175	193
176	194	Reading from any of the files will return -EINVAL if you are not starting
177	195	the read on an 8-byte boundary (e.g., if you sought an odd number of bytes

Documentation/admin-guide/mm/soft-dirty.rstrenamed from Documentation/vm/soft-dirty.txt+12−8

@@ -1,34 +1,38 @@
1		− SOFT-DIRTY PTEs
	1	+.. _soft_dirty:
2	2
3		− The soft-dirty is a bit on a PTE which helps to track which pages a task
	3	+===============
	4	+Soft-Dirty PTEs
	5	+===============
	6	+
	7	+The soft-dirty is a bit on a PTE which helps to track which pages a task
4	8	writes to. In order to do this tracking one should
5	9
6	10	1. Clear soft-dirty bits from the task's PTEs.
7	11
8		− This is done by writing "4" into the /proc/PID/clear_refs file of the
	12	+ This is done by writing "4" into the ``/proc/PID/clear_refs`` file of the
9	13	task in question.
10	14
11	15	2. Wait some time.
12	16
13	17	3. Read soft-dirty bits from the PTEs.
14	18
15		− This is done by reading from the /proc/PID/pagemap. The bit 55 of the
	19	+ This is done by reading from the ``/proc/PID/pagemap``. The bit 55 of the
16	20	64-bit qword is the soft-dirty one. If set, the respective PTE was
17	21	written to since step 1.
18	22
19	23
20		− Internally, to do this tracking, the writable bit is cleared from PTEs
	24	+Internally, to do this tracking, the writable bit is cleared from PTEs
21	25	when the soft-dirty bit is cleared. So, after this, when the task tries to
22	26	modify a page at some virtual address the #PF occurs and the kernel sets
23	27	the soft-dirty bit on the respective PTE.
24	28
25		− Note, that although all the task's address space is marked as r/o after the
	29	+Note, that although all the task's address space is marked as r/o after the
26	30	soft-dirty bits clear, the #PF-s that occur after that are processed fast.
27	31	This is so, since the pages are still mapped to physical memory, and thus all
28	32	the kernel does is finds this fact out and puts both writable and soft-dirty
29	33	bits on the PTE.
30	34
31		− While in most cases tracking memory changes by #PF-s is more than enough
	35	+While in most cases tracking memory changes by #PF-s is more than enough
32	36	there is still a scenario when we can lose soft dirty bits -- a task
33	37	unmaps a previously mapped memory region and then maps a new one at exactly
34	38	the same place. When unmap is called, the kernel internally clears PTE values
@@ -36,7 +40,7 @@ including soft dirty bits. To notify user space application about such · 1 unmodified line
36	40	memory region renewal the kernel always marks new memory regions (and
37	41	expanded regions) as soft dirty.
38	42
39		− This feature is actively used by the checkpoint-restore project. You
	43	+This feature is actively used by the checkpoint-restore project. You
40	44	can find more details about it on http://criu.org
41	45
42	46

Documentation/admin-guide/mm/transhuge.rst+418−0

@@ -0,0 +1,418 @@
	1	+.. _admin_guide_transhuge:
	2	+
	3	+============================
	4	+Transparent Hugepage Support
	5	+============================
	6	+
	7	+Objective
	8	+=========
	9	+
	10	+Performance critical computing applications dealing with large memory
	11	+working sets are already running on top of libhugetlbfs and in turn
	12	+hugetlbfs. Transparent HugePage Support (THP) is an alternative mean of
	13	+using huge pages for the backing of virtual memory with huge pages
	14	+that supports the automatic promotion and demotion of page sizes and
	15	+without the shortcomings of hugetlbfs.
	16	+
	17	+Currently THP only works for anonymous memory mappings and tmpfs/shmem.
	18	+But in the future it can expand to other filesystems.
	19	+
	20	+.. note::
	21	+ in the examples below we presume that the basic page size is 4K and
	22	+ the huge page size is 2M, although the actual numbers may vary
	23	+ depending on the CPU architecture.
	24	+
	25	+The reason applications are running faster is because of two
	26	+factors. The first factor is almost completely irrelevant and it's not
	27	+of significant interest because it'll also have the downside of
	28	+requiring larger clear-page copy-page in page faults which is a
	29	+potentially negative effect. The first factor consists in taking a
	30	+single page fault for each 2M virtual region touched by userland (so
	31	+reducing the enter/exit kernel frequency by a 512 times factor). This
	32	+only matters the first time the memory is accessed for the lifetime of
	33	+a memory mapping. The second long lasting and much more important
	34	+factor will affect all subsequent accesses to the memory for the whole
	35	+runtime of the application. The second factor consist of two
	36	+components:
	37	+
	38	+1) the TLB miss will run faster (especially with virtualization using
	39	+ nested pagetables but almost always also on bare metal without
	40	+ virtualization)
	41	+
	42	+2) a single TLB entry will be mapping a much larger amount of virtual
	43	+ memory in turn reducing the number of TLB misses. With
	44	+ virtualization and nested pagetables the TLB can be mapped of
	45	+ larger size only if both KVM and the Linux guest are using
	46	+ hugepages but a significant speedup already happens if only one of
	47	+ the two is using hugepages just because of the fact the TLB miss is
	48	+ going to run faster.
	49	+
	50	+THP can be enabled system wide or restricted to certain tasks or even
	51	+memory ranges inside task's address space. Unless THP is completely
	52	+disabled, there is ``khugepaged`` daemon that scans memory and
	53	+collapses sequences of basic pages into huge pages.
	54	+
	55	+The THP behaviour is controlled via :ref:`sysfs <thp_sysfs>`
	56	+interface and using madivse(2) and prctl(2) system calls.
	57	+
	58	+Transparent Hugepage Support maximizes the usefulness of free memory
	59	+if compared to the reservation approach of hugetlbfs by allowing all
	60	+unused memory to be used as cache or other movable (or even unmovable
	61	+entities). It doesn't require reservation to prevent hugepage
	62	+allocation failures to be noticeable from userland. It allows paging
	63	+and all other advanced VM features to be available on the
	64	+hugepages. It requires no modifications for applications to take
	65	+advantage of it.
	66	+
	67	+Applications however can be further optimized to take advantage of
	68	+this feature, like for example they've been optimized before to avoid
	69	+a flood of mmap system calls for every malloc(4k). Optimizing userland
	70	+is by far not mandatory and khugepaged already can take care of long
	71	+lived page allocations even for hugepage unaware applications that
	72	+deals with large amounts of memory.
	73	+
	74	+In certain cases when hugepages are enabled system wide, application
	75	+may end up allocating more memory resources. An application may mmap a
	76	+large region but only touch 1 byte of it, in that case a 2M page might
	77	+be allocated instead of a 4k page for no good. This is why it's
	78	+possible to disable hugepages system-wide and to only have them inside
	79	+MADV_HUGEPAGE madvise regions.
	80	+
	81	+Embedded systems should enable hugepages only inside madvise regions
	82	+to eliminate any risk of wasting any precious byte of memory and to
	83	+only run faster.
	84	+
	85	+Applications that gets a lot of benefit from hugepages and that don't
	86	+risk to lose memory by using hugepages, should use
	87	+madvise(MADV_HUGEPAGE) on their critical mmapped regions.
	88	+
	89	+.. _thp_sysfs:
	90	+
	91	+sysfs
	92	+=====
	93	+
	94	+Global THP controls
	95	+-------------------
	96	+
	97	+Transparent Hugepage Support for anonymous memory can be entirely disabled
	98	+(mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE
	99	+regions (to avoid the risk of consuming more memory resources) or enabled
	100	+system wide. This can be achieved with one of::
	101	+
	102	+ echo always >/sys/kernel/mm/transparent_hugepage/enabled
	103	+ echo madvise >/sys/kernel/mm/transparent_hugepage/enabled
	104	+ echo never >/sys/kernel/mm/transparent_hugepage/enabled
	105	+
	106	+It's also possible to limit defrag efforts in the VM to generate
	107	+anonymous hugepages in case they're not immediately free to madvise
	108	+regions or to never try to defrag memory and simply fallback to regular
	109	+pages unless hugepages are immediately available. Clearly if we spend CPU
	110	+time to defrag memory, we would expect to gain even more by the fact we
	111	+use hugepages later instead of regular pages. This isn't always
	112	+guaranteed, but it may be more likely in case the allocation is for a
	113	+MADV_HUGEPAGE region.
	114	+
	115	+::
	116	+
	117	+ echo always >/sys/kernel/mm/transparent_hugepage/defrag
	118	+ echo defer >/sys/kernel/mm/transparent_hugepage/defrag
	119	+ echo defer+madvise >/sys/kernel/mm/transparent_hugepage/defrag
	120	+ echo madvise >/sys/kernel/mm/transparent_hugepage/defrag
	121	+ echo never >/sys/kernel/mm/transparent_hugepage/defrag
	122	+
	123	+always
	124	+ means that an application requesting THP will stall on
	125	+ allocation failure and directly reclaim pages and compact
	126	+ memory in an effort to allocate a THP immediately. This may be
	127	+ desirable for virtual machines that benefit heavily from THP
	128	+ use and are willing to delay the VM start to utilise them.
	129	+
	130	+defer
	131	+ means that an application will wake kswapd in the background
	132	+ to reclaim pages and wake kcompactd to compact memory so that
	133	+ THP is available in the near future. It's the responsibility
	134	+ of khugepaged to then install the THP pages later.
	135	+
	136	+defer+madvise
	137	+ will enter direct reclaim and compaction like ``always``, but
	138	+ only for regions that have used madvise(MADV_HUGEPAGE); all
	139	+ other regions will wake kswapd in the background to reclaim
	140	+ pages and wake kcompactd to compact memory so that THP is
	141	+ available in the near future.
	142	+
	143	+madvise
	144	+ will enter direct reclaim like ``always`` but only for regions
	145	+ that are have used madvise(MADV_HUGEPAGE). This is the default
	146	+ behaviour.
	147	+
	148	+never
	149	+ should be self-explanatory.
	150	+
	151	+By default kernel tries to use huge zero page on read page fault to
	152	+anonymous mapping. It's possible to disable huge zero page by writing 0
	153	+or enable it back by writing 1::
	154	+
	155	+ echo 0 >/sys/kernel/mm/transparent_hugepage/use_zero_page
	156	+ echo 1 >/sys/kernel/mm/transparent_hugepage/use_zero_page
	157	+
	158	+Some userspace (such as a test program, or an optimized memory allocation
	159	+library) may want to know the size (in bytes) of a transparent hugepage::
	160	+
	161	+ cat /sys/kernel/mm/transparent_hugepage/hpage_pmd_size
	162	+
	163	+khugepaged will be automatically started when
	164	+transparent_hugepage/enabled is set to "always" or "madvise, and it'll
	165	+be automatically shutdown if it's set to "never".
	166	+
	167	+Khugepaged controls
	168	+-------------------
	169	+
	170	+khugepaged runs usually at low frequency so while one may not want to
	171	+invoke defrag algorithms synchronously during the page faults, it
	172	+should be worth invoking defrag at least in khugepaged. However it's
	173	+also possible to disable defrag in khugepaged by writing 0 or enable
	174	+defrag in khugepaged by writing 1::
	175	+
	176	+ echo 0 >/sys/kernel/mm/transparent_hugepage/khugepaged/defrag
	177	+ echo 1 >/sys/kernel/mm/transparent_hugepage/khugepaged/defrag
	178	+
	179	+You can also control how many pages khugepaged should scan at each
	180	+pass::
	181	+
	182	+ /sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan
	183	+
	184	+and how many milliseconds to wait in khugepaged between each pass (you
	185	+can set this to 0 to run khugepaged at 100% utilization of one core)::
	186	+
	187	+ /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs
	188	+
	189	+and how many milliseconds to wait in khugepaged if there's an hugepage
	190	+allocation failure to throttle the next allocation attempt::
	191	+
	192	+ /sys/kernel/mm/transparent_hugepage/khugepaged/alloc_sleep_millisecs
	193	+
	194	+The khugepaged progress can be seen in the number of pages collapsed::
	195	+
	196	+ /sys/kernel/mm/transparent_hugepage/khugepaged/pages_collapsed
	197	+
	198	+for each pass::
	199	+
	200	+ /sys/kernel/mm/transparent_hugepage/khugepaged/full_scans
	201	+
	202	+``max_ptes_none`` specifies how many extra small pages (that are
	203	+not already mapped) can be allocated when collapsing a group
	204	+of small pages into one large page::
	205	+
	206	+ /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none
	207	+
	208	+A higher value leads to use additional memory for programs.
	209	+A lower value leads to gain less thp performance. Value of
	210	+max_ptes_none can waste cpu time very little, you can
	211	+ignore it.
	212	+
	213	+``max_ptes_swap`` specifies how many pages can be brought in from
	214	+swap when collapsing a group of pages into a transparent huge page::
	215	+
	216	+ /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_swap
	217	+
	218	+A higher value can cause excessive swap IO and waste
	219	+memory. A lower value can prevent THPs from being
	220	+collapsed, resulting fewer pages being collapsed into
	221	+THPs, and lower memory access performance.
	222	+
	223	+Boot parameter
	224	+==============
	225	+
	226	+You can change the sysfs boot time defaults of Transparent Hugepage
	227	+Support by passing the parameter ``transparent_hugepage=always`` or
	228	+``transparent_hugepage=madvise`` or ``transparent_hugepage=never``
	229	+to the kernel command line.
	230	+
	231	+Hugepages in tmpfs/shmem
	232	+========================
	233	+
	234	+You can control hugepage allocation policy in tmpfs with mount option
	235	+``huge=``. It can have following values:
	236	+
	237	+always
	238	+ Attempt to allocate huge pages every time we need a new page;
	239	+
	240	+never
	241	+ Do not allocate huge pages;
	242	+
	243	+within_size
	244	+ Only allocate huge page if it will be fully within i_size.
	245	+ Also respect fadvise()/madvise() hints;
	246	+
	247	+advise
	248	+ Only allocate huge pages if requested with fadvise()/madvise();
	249	+
	250	+The default policy is ``never``.
	251	+
	252	+``mount -o remount,huge= /mountpoint`` works fine after mount: remounting
	253	+``huge=never`` will not attempt to break up huge pages at all, just stop more
	254	+from being allocated.
	255	+
	256	+There's also sysfs knob to control hugepage allocation policy for internal
	257	+shmem mount: /sys/kernel/mm/transparent_hugepage/shmem_enabled. The mount
	258	+is used for SysV SHM, memfds, shared anonymous mmaps (of /dev/zero or
	259	+MAP_ANONYMOUS), GPU drivers' DRM objects, Ashmem.
	260	+
	261	+In addition to policies listed above, shmem_enabled allows two further
	262	+values:
	263	+
	264	+deny
	265	+ For use in emergencies, to force the huge option off from
	266	+ all mounts;
	267	+force
	268	+ Force the huge option on for all - very useful for testing;
	269	+
	270	+Need of application restart
	271	+===========================
	272	+
	273	+The transparent_hugepage/enabled values and tmpfs mount option only affect
	274	+future behavior. So to make them effective you need to restart any
	275	+application that could have been using hugepages. This also applies to the
	276	+regions registered in khugepaged.
	277	+
	278	+Monitoring usage
	279	+================
	280	+
	281	+The number of anonymous transparent huge pages currently used by the
	282	+system is available by reading the AnonHugePages field in ``/proc/meminfo``.
	283	+To identify what applications are using anonymous transparent huge pages,
	284	+it is necessary to read ``/proc/PID/smaps`` and count the AnonHugePages fields
	285	+for each mapping.
	286	+
	287	+The number of file transparent huge pages mapped to userspace is available
	288	+by reading ShmemPmdMapped and ShmemHugePages fields in ``/proc/meminfo``.
	289	+To identify what applications are mapping file transparent huge pages, it
	290	+is necessary to read ``/proc/PID/smaps`` and count the FileHugeMapped fields
	291	+for each mapping.
	292	+
	293	+Note that reading the smaps file is expensive and reading it
	294	+frequently will incur overhead.
	295	+
	296	+There are a number of counters in ``/proc/vmstat`` that may be used to
	297	+monitor how successfully the system is providing huge pages for use.
	298	+
	299	+thp_fault_alloc
	300	+ is incremented every time a huge page is successfully
	301	+ allocated to handle a page fault. This applies to both the
	302	+ first time a page is faulted and for COW faults.
	303	+
	304	+thp_collapse_alloc
	305	+ is incremented by khugepaged when it has found
	306	+ a range of pages to collapse into one huge page and has
	307	+ successfully allocated a new huge page to store the data.
	308	+
	309	+thp_fault_fallback
	310	+ is incremented if a page fault fails to allocate
	311	+ a huge page and instead falls back to using small pages.
	312	+
	313	+thp_collapse_alloc_failed
	314	+ is incremented if khugepaged found a range
	315	+ of pages that should be collapsed into one huge page but failed
	316	+ the allocation.
	317	+
	318	+thp_file_alloc
	319	+ is incremented every time a file huge page is successfully
	320	+ allocated.
	321	+
	322	+thp_file_mapped
	323	+ is incremented every time a file huge page is mapped into
	324	+ user address space.
	325	+
	326	+thp_split_page
	327	+ is incremented every time a huge page is split into base
	328	+ pages. This can happen for a variety of reasons but a common
	329	+ reason is that a huge page is old and is being reclaimed.
	330	+ This action implies splitting all PMD the page mapped with.
	331	+
	332	+thp_split_page_failed
	333	+ is incremented if kernel fails to split huge
	334	+ page. This can happen if the page was pinned by somebody.
	335	+
	336	+thp_deferred_split_page
	337	+ is incremented when a huge page is put onto split
	338	+ queue. This happens when a huge page is partially unmapped and
	339	+ splitting it would free up some memory. Pages on split queue are
	340	+ going to be split under memory pressure.
	341	+
	342	+thp_split_pmd
	343	+ is incremented every time a PMD split into table of PTEs.
	344	+ This can happen, for instance, when application calls mprotect() or
	345	+ munmap() on part of huge page. It doesn't split huge page, only
	346	+ page table entry.
	347	+
	348	+thp_zero_page_alloc
	349	+ is incremented every time a huge zero page is
	350	+ successfully allocated. It includes allocations which where
	351	+ dropped due race with other allocation. Note, it doesn't count
	352	+ every map of the huge zero page, only its allocation.
	353	+
	354	+thp_zero_page_alloc_failed
	355	+ is incremented if kernel fails to allocate
	356	+ huge zero page and falls back to using small pages.
	357	+
	358	+thp_swpout
	359	+ is incremented every time a huge page is swapout in one
	360	+ piece without splitting.
	361	+
	362	+thp_swpout_fallback
	363	+ is incremented if a huge page has to be split before swapout.
	364	+ Usually because failed to allocate some continuous swap space
	365	+ for the huge page.
	366	+
	367	+As the system ages, allocating huge pages may be expensive as the
	368	+system uses memory compaction to copy data around memory to free a
	369	+huge page for use. There are some counters in ``/proc/vmstat`` to help
	370	+monitor this overhead.
	371	+
	372	+compact_stall
	373	+ is incremented every time a process stalls to run
	374	+ memory compaction so that a huge page is free for use.
	375	+
	376	+compact_success
	377	+ is incremented if the system compacted memory and
	378	+ freed a huge page for use.
	379	+
	380	+compact_fail
	381	+ is incremented if the system tries to compact memory
	382	+ but failed.
	383	+
	384	+compact_pages_moved
	385	+ is incremented each time a page is moved. If
	386	+ this value is increasing rapidly, it implies that the system
	387	+ is copying a lot of data to satisfy the huge page allocation.
	388	+ It is possible that the cost of copying exceeds any savings
	389	+ from reduced TLB misses.
	390	+
	391	+compact_pagemigrate_failed
	392	+ is incremented when the underlying mechanism
	393	+ for moving a page failed.
	394	+
	395	+compact_blocks_moved
	396	+ is incremented each time memory compaction examines
	397	+ a huge page aligned range of pages.
	398	+
	399	+It is possible to establish how long the stalls were using the function
	400	+tracer to record how long was spent in __alloc_pages_nodemask and
	401	+using the mm_page_alloc tracepoint to identify which allocations were
	402	+for huge pages.
	403	+
	404	+Optimizing the applications
	405	+===========================
	406	+
	407	+To be guaranteed that the kernel will map a 2M page immediately in any
	408	+memory region, the mmap region has to be hugepage naturally
	409	+aligned. posix_memalign() can provide that guarantee.
	410	+
	411	+Hugetlbfs
	412	+=========
	413	+
	414	+You can use hugetlbfs on a kernel that has transparent hugepage
	415	+support enabled just fine as always. No difference can be noted in
	416	+hugetlbfs other than there will be less overall fragmentation. All
	417	+usual features belonging to hugetlbfs are preserved and
	418	+unaffected. libhugetlbfs will also work fine as usual.

Documentation/admin-guide/mm/userfaultfd.rstrenamed from Documentation/vm/userfaultfd.txt+39−27

@@ -1,6 +1,11 @@
1		−= Userfaultfd =
	1	+.. _userfaultfd:
2	2
3		−== Objective ==
	3	+===========
	4	+Userfaultfd
	5	+===========
	6	+
	7	+Objective
	8	+=========
4	9
5	10	Userfaults allow the implementation of on-demand paging from userland
6	11	and more generally they allow userland to take control of various
@@ -9,7 +14,8 @@ memory page faults, something otherwise only the kernel code could do. · 2 unmodified lines
9	14	For example userfaults allows a proper and more optimal implementation
10	15	of the PROT_NONE+SIGSEGV trick.
11	16
12		−== Design ==
	17	+Design
	18	+======
13	19
14	20	Userfaults are delivered and resolved through the userfaultfd syscall.
15	21
@@ -41,7 +47,8 @@ different processes without them being aware about what is going on · 25 unmodified lines
41	47	themselves on the same region the manager is already tracking, which
42	48	is a corner case that would currently return -EBUSY).
43	49
44		−== API ==
	50	+API
	51	+===
45	52
46	53	When first opened the userfaultfd must be enabled invoking the
47	54	UFFDIO_API ioctl specifying a uffdio_api.api value set to UFFD_API (or
@@ -101,7 +108,8 @@ UFFDIO_COPY. They're atomic as in guaranteeing that nothing can see an · 53 unmodified lines
101	108	half copied page since it'll keep userfaulting until the copy has
102	109	finished.
103	110
104		−== QEMU/KVM ==
	111	+QEMU/KVM
	112	+========
105	113
106	114	QEMU/KVM is using the userfaultfd syscall to implement postcopy live
107	115	migration. Postcopy live migration is one form of memory
@@ -163,7 +171,8 @@ sending the same page twice (in case the userfault is read by the · 55 unmodified lines
163	171	postcopy thread just before UFFDIO_COPY\|ZEROPAGE runs in the migration
164	172	thread).
165	173
166		−== Non-cooperative userfaultfd ==
	174	+Non-cooperative userfaultfd
	175	+===========================
167	176
168	177	When the userfaultfd is monitored by an external manager, the manager
169	178	must be able to track changes in the process virtual memory
@@ -172,27 +181,30 @@ the same read(2) protocol as for the page fault notifications. The · 2 unmodified lines
172	181	manager has to explicitly enable these events by setting appropriate
173	182	bits in uffdio_api.features passed to UFFDIO_API ioctl:
174	183
175		−UFFD_FEATURE_EVENT_FORK - enable userfaultfd hooks for fork(). When
176		−this feature is enabled, the userfaultfd context of the parent process
177		−is duplicated into the newly created process. The manager receives
178		−UFFD_EVENT_FORK with file descriptor of the new userfaultfd context in
179		−the uffd_msg.fork.
180		−
181		−UFFD_FEATURE_EVENT_REMAP - enable notifications about mremap()
182		−calls. When the non-cooperative process moves a virtual memory area to
183		−a different location, the manager will receive UFFD_EVENT_REMAP. The
184		−uffd_msg.remap will contain the old and new addresses of the area and
185		−its original length.
186		−
187		−UFFD_FEATURE_EVENT_REMOVE - enable notifications about
188		−madvise(MADV_REMOVE) and madvise(MADV_DONTNEED) calls. The event
189		−UFFD_EVENT_REMOVE will be generated upon these calls to madvise. The
190		−uffd_msg.remove will contain start and end addresses of the removed
191		−area.
192		−
193		−UFFD_FEATURE_EVENT_UNMAP - enable notifications about memory
194		−unmapping. The manager will get UFFD_EVENT_UNMAP with uffd_msg.remove
195		−containing start and end addresses of the unmapped area.
	184	+UFFD_FEATURE_EVENT_FORK
	185	+ enable userfaultfd hooks for fork(). When this feature is
	186	+ enabled, the userfaultfd context of the parent process is
	187	+ duplicated into the newly created process. The manager
	188	+ receives UFFD_EVENT_FORK with file descriptor of the new
	189	+ userfaultfd context in the uffd_msg.fork.
	190	+
	191	+UFFD_FEATURE_EVENT_REMAP
	192	+ enable notifications about mremap() calls. When the
	193	+ non-cooperative process moves a virtual memory area to a
	194	+ different location, the manager will receive
	195	+ UFFD_EVENT_REMAP. The uffd_msg.remap will contain the old and
	196	+ new addresses of the area and its original length.
	197	+
	198	+UFFD_FEATURE_EVENT_REMOVE
	199	+ enable notifications about madvise(MADV_REMOVE) and
	200	+ madvise(MADV_DONTNEED) calls. The event UFFD_EVENT_REMOVE will
	201	+ be generated upon these calls to madvise. The uffd_msg.remove
	202	+ will contain start and end addresses of the removed area.
	203	+
	204	+UFFD_FEATURE_EVENT_UNMAP
	205	+ enable notifications about memory unmapping. The manager will
	206	+ get UFFD_EVENT_UNMAP with uffd_msg.remove containing start and
	207	+ end addresses of the unmapped area.
196	208
197	209	Although the UFFD_FEATURE_EVENT_REMOVE and UFFD_FEATURE_EVENT_UNMAP
198	210	are pretty similar, they quite differ in the action expected from the

Documentation/admin-guide/ramoops.rst+1−1

@@ -61,7 +61,7 @@ Setting the ramoops parameters can be done in several different manners: · 60 unmodified lines
61	61	mem=128M ramoops.mem_address=0x8000000 ramoops.ecc=1
62	62
63	63	B. Use Device Tree bindings, as described in
64		− ``Documentation/device-tree/bindings/reserved-memory/admin-guide/ramoops.rst``.
	64	+ ``Documentation/devicetree/bindings/reserved-memory/ramoops.txt``.
65	65	For example::
66	66
67	67	reserved-memory {

Documentation/arm/Marvell/README+7−8

@@ -302,19 +302,15 @@ Berlin family (Multimedia Solutions) · 301 unmodified lines
302	302	88DE3010, Armada 1000 (no Linux support)
303	303	Core: Marvell PJ1 (ARMv5TE), Dual-core
304	304	Product Brief: http://www.marvell.com.cn/digital-entertainment/assets/armada_1000_pb.pdf
305		− 88DE3005, Armada 1500-mini
306	305	88DE3005, Armada 1500 Mini
307	306	Design name: BG2CD
308	307	Core: ARM Cortex-A9, PL310 L2CC
309		− Homepage: http://www.marvell.com/multimedia-solutions/armada-1500-mini/
310		− 88DE3006, Armada 1500 Mini Plus
311		− Design name: BG2CDP
312		− Core: Dual Core ARM Cortex-A7
313		− Homepage: http://www.marvell.com/multimedia-solutions/armada-1500-mini-plus/
	308	+ 88DE3006, Armada 1500 Mini Plus
	309	+ Design name: BG2CDP
	310	+ Core: Dual Core ARM Cortex-A7
314	311	88DE3100, Armada 1500
315	312	Design name: BG2
316	313	Core: Marvell PJ4B-MP (ARMv7), Tauros3 L2CC
317		− Product Brief: http://www.marvell.com/digital-entertainment/armada-1500/assets/Marvell-ARMADA-1500-Product-Brief.pdf
318	314	88DE3114, Armada 1500 Pro
319	315	Design name: BG2Q
320	316	Core: Quad Core ARM Cortex-A9, PL310 L2CC
@@ -324,13 +320,16 @@ Berlin family (Multimedia Solutions) · 3 unmodified lines
324	320	88DE3218, ARMADA 1500 Ultra
325	321	Core: ARM Cortex-A53
326	322
327		− Homepage: http://www.marvell.com/multimedia-solutions/
	323	+ Homepage: https://www.synaptics.com/products/multimedia-solutions
328	324	Directory: arch/arm/mach-berlin
329	325
330	326	Comments:
	327	+
331	328	* This line of SoCs is based on Marvell Sheeva or ARM Cortex CPUs
332	329	with Synopsys DesignWare (IRQ, GPIO, Timers, ...) and PXA IP (SDHCI, USB, ETH, ...).
333	330
	331	+ * The Berlin family was acquired by Synaptics from Marvell in 2017.
	332	+
334	333	CPU Cores
335	334	---------
336	335

Documentation/block/cmdline-partition.txt+14−7

@@ -1,7 +1,9 @@
1	1	Embedded device command line partition parsing
2	2	=====================================================================
3	3
4		−Support for reading the block device partition table from the command line.
	4	+The "blkdevparts" command line option adds support for reading the
	5	+block device partition table from the kernel command line.
	6	+
5	7	It is typically used for fixed block (eMMC) embedded devices.
6	8	It has no MBR, so saves storage space. Bootloader can be easily accessed
7	9	by absolute address of data on the block device.
@@ -14,22 +16,27 @@ blkdevparts=<blkdev-def>[;<blkdev-def>] · 6 unmodified lines
14	16	<partdef> := <size>[@<offset>](part-name)
15	17
16	18	<blkdev-id>
17		− block device disk name, embedded device used fixed block device,
18		− it's disk name also fixed. such as: mmcblk0, mmcblk1, mmcblk0boot0.
	19	+ block device disk name. Embedded device uses fixed block device.
	20	+ Its disk name is also fixed, such as: mmcblk0, mmcblk1, mmcblk0boot0.
19	21
20	22	<size>
21	23	partition size, in bytes, such as: 512, 1m, 1G.
	24	+ size may contain an optional suffix of (upper or lower case):
	25	+ K, M, G, T, P, E.
	26	+ "-" is used to denote all remaining space.
22	27
23	28	<offset>
24	29	partition start address, in bytes.
	30	+ offset may contain an optional suffix of (upper or lower case):
	31	+ K, M, G, T, P, E.
25	32
26	33	(part-name)
27		− partition name, kernel send uevent with "PARTNAME". application can create
28		− a link to block device partition with the name "PARTNAME".
29		− user space application can access partition by partition name.
	34	+ partition name. Kernel sends uevent with "PARTNAME". Application can
	35	+ create a link to block device partition with the name "PARTNAME".
	36	+ User space application can access partition by partition name.
30	37
31	38	Example:
32		− eMMC disk name is "mmcblk0" and "mmcblk0boot0"
	39	+ eMMC disk names are "mmcblk0" and "mmcblk0boot0".
33	40
34	41	bootargs:
35	42	'blkdevparts=mmcblk0:1G(data0),1G(data1),-;mmcblk0boot0:1m(boot),-(kernel)'

Documentation/core-api/cachetlb.rstrenamed from Documentation/cachetlb.txt+0−0

File renamed without changes.

Documentation/core-api/circular-buffers.rstrenamed from Documentation/circular-buffers.txt+0−0

File renamed without changes.

Documentation/core-api/gfp_mask-from-fs-io.rst+66−0

@@ -0,0 +1,66 @@
	1	+=================================
	2	+GFP masks used from FS/IO context
	3	+=================================
	4	+
	5	+:Date: May, 2018
	6	+:Author: Michal Hocko <mhocko@kernel.org>
	7	+
	8	+Introduction
	9	+============
	10	+
	11	+Code paths in the filesystem and IO stacks must be careful when
	12	+allocating memory to prevent recursion deadlocks caused by direct
	13	+memory reclaim calling back into the FS or IO paths and blocking on
	14	+already held resources (e.g. locks - most commonly those used for the
	15	+transaction context).
	16	+
	17	+The traditional way to avoid this deadlock problem is to clear __GFP_FS
	18	+respectively __GFP_IO (note the latter implies clearing the first as well) in
	19	+the gfp mask when calling an allocator. GFP_NOFS respectively GFP_NOIO can be
	20	+used as shortcut. It turned out though that above approach has led to
	21	+abuses when the restricted gfp mask is used "just in case" without a
	22	+deeper consideration which leads to problems because an excessive use
	23	+of GFP_NOFS/GFP_NOIO can lead to memory over-reclaim or other memory
	24	+reclaim issues.
	25	+
	26	+New API
	27	+========
	28	+
	29	+Since 4.12 we do have a generic scope API for both NOFS and NOIO context
	30	+``memalloc_nofs_save``, ``memalloc_nofs_restore`` respectively ``memalloc_noio_save``,
	31	+``memalloc_noio_restore`` which allow to mark a scope to be a critical
	32	+section from a filesystem or I/O point of view. Any allocation from that
	33	+scope will inherently drop __GFP_FS respectively __GFP_IO from the given
	34	+mask so no memory allocation can recurse back in the FS/IO.
	35	+
	36	+.. kernel-doc:: include/linux/sched/mm.h
	37	+ :functions: memalloc_nofs_save memalloc_nofs_restore
	38	+.. kernel-doc:: include/linux/sched/mm.h
	39	+ :functions: memalloc_noio_save memalloc_noio_restore
	40	+
	41	+FS/IO code then simply calls the appropriate save function before
	42	+any critical section with respect to the reclaim is started - e.g.
	43	+lock shared with the reclaim context or when a transaction context
	44	+nesting would be possible via reclaim. The restore function should be
	45	+called when the critical section ends. All that ideally along with an
	46	+explanation what is the reclaim context for easier maintenance.
	47	+
	48	+Please note that the proper pairing of save/restore functions
	49	+allows nesting so it is safe to call ``memalloc_noio_save`` or
	50	+``memalloc_noio_restore`` respectively from an existing NOIO or NOFS
	51	+scope.
	52	+
	53	+What about __vmalloc(GFP_NOFS)
	54	+==============================
	55	+
	56	+vmalloc doesn't support GFP_NOFS semantic because there are hardcoded
	57	+GFP_KERNEL allocations deep inside the allocator which are quite non-trivial
	58	+to fix up. That means that calling ``vmalloc`` with GFP_NOFS/GFP_NOIO is
	59	+almost always a bug. The good news is that the NOFS/NOIO semantic can be
	60	+achieved by the scope API.
	61	+
	62	+In the ideal world, upper layers should already mark dangerous contexts
	63	+and so no special care is required and vmalloc should be called without
	64	+any problems. Sometimes if the context is not really clear or there are
	65	+layering violations then the recommended way around that is to wrap ``vmalloc``
	66	+by the scope API with a comment explaining the problem.

Documentation/core-api/index.rst+3−0

@@ -14,6 +14,7 @@ Core utilities · 13 unmodified lines
14	14	kernel-api
15	15	assoc_array
16	16	atomic_ops
	17	+ cachetlb
17	18	refcount-vs-atomic
18	19	cpu_hotplug
19	20	idr
@@ -25,6 +26,8 @@ Core utilities · 5 unmodified lines
25	26	genalloc
26	27	errseq
27	28	printk-formats
	29	+ circular-buffers
	30	+ gfp_mask-from-fs-io
28	31
29	32	Interfaces for kernel debugging
30	33	===============================

Documentation/core-api/kernel-api.rst+30−30

@@ -39,17 +39,17 @@ String Manipulation · 38 unmodified lines
39	39	.. kernel-doc:: lib/string.c
40	40	:export:
41	41
	42	+Basic Kernel Library Functions
	43	+==============================
	44	+
	45	+The Linux kernel provides more basic utility functions.
	46	+
42	47	Bit Operations
43	48	--------------
44	49
45	50	.. kernel-doc:: arch/x86/include/asm/bitops.h
46	51	:internal:
47	52
48		−Basic Kernel Library Functions
49		−==============================
50		−
51		−The Linux kernel provides more basic utility functions.
52		−
53	53	Bitmap Operations
54	54	-----------------
55	55
@@ -80,6 +80,31 @@ Command-line Parsing · 24 unmodified lines
80	80	.. kernel-doc:: lib/cmdline.c
81	81	:export:
82	82
	83	+Sorting
	84	+-------
	85	+
	86	+.. kernel-doc:: lib/sort.c
	87	+ :export:
	88	+
	89	+.. kernel-doc:: lib/list_sort.c
	90	+ :export:
	91	+
	92	+Text Searching
	93	+--------------
	94	+
	95	+.. kernel-doc:: lib/textsearch.c
	96	+ :doc: ts_intro
	97	+
	98	+.. kernel-doc:: lib/textsearch.c
	99	+ :export:
	100	+
	101	+.. kernel-doc:: include/linux/textsearch.h
	102	+ :functions: textsearch_find textsearch_next \
	103	+ textsearch_get_pattern textsearch_get_pattern_len
	104	+
	105	+CRC and Math Functions in Linux
	106	+===============================
	107	+
83	108	CRC Functions
84	109	-------------
85	110
@@ -103,9 +128,6 @@ CRC Functions · 17 unmodified lines
103	128	.. kernel-doc:: lib/crc-itu-t.c
104	129	:export:
105	130
106		−Math Functions in Linux
107		−=======================
108		−
109	131	Base 2 log and power Functions
110	132	------------------------------
111	133
@@ -127,28 +149,6 @@ Division Functions · 15 unmodified lines
127	149	.. kernel-doc:: lib/gcd.c
128	150	:export:
129	151
130		−Sorting
131		−-------
132		−
133		−.. kernel-doc:: lib/sort.c
134		− :export:
135		−
136		−.. kernel-doc:: lib/list_sort.c
137		− :export:
138		−
139		−Text Searching
140		−--------------
141		−
142		−.. kernel-doc:: lib/textsearch.c
143		− :doc: ts_intro
144		−
145		−.. kernel-doc:: lib/textsearch.c
146		− :export:
147		−
148		−.. kernel-doc:: include/linux/textsearch.h
149		− :functions: textsearch_find textsearch_next \
150		− textsearch_get_pattern textsearch_get_pattern_len
151		−
152	152	UUID/GUID
153	153	---------
154	154

Documentation/core-api/refcount-vs-atomic.rst+1−1

@@ -17,7 +17,7 @@ in order to help maintainers validate their code against the change in · 16 unmodified lines
17	17	these memory ordering guarantees.
18	18
19	19	The terms used through this document try to follow the formal LKMM defined in
20		−github.com/aparri/memory-model/blob/master/Documentation/explanation.txt
	20	+tools/memory-model/Documentation/explanation.txt.
21	21
22	22	memory-barriers.txt and atomic_t.txt provide more background to the
23	23	memory ordering in general and for atomic operations specifically.

Documentation/crypto/index.rst+1−0

@@ -20,5 +20,6 @@ for cryptographic use cases, as well as programming examples. · 19 unmodified lines
20	20	architecture
21	21	devel-algos
22	22	userspace-if
	23	+ crypto_engine
23	24	api
24	25	api-samples

Documentation/dev-tools/kasan.rst+1−1

@@ -120,7 +120,7 @@ A typical out of bounds access report looks like this:: · 119 unmodified lines
120	120
121	121	The header of the report discribe what kind of bug happened and what kind of
122	122	access caused it. It's followed by the description of the accessed slub object
123		−(see 'SLUB Debug output' section in Documentation/vm/slub.txt for details) and
	123	+(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and
124	124	the description of the accessed memory page.
125	125
126	126	In the last section the report shows memory state around the accessed address.

Documentation/dev-tools/kselftest.rst+5−0

@@ -151,6 +151,11 @@ Contributing new tests (details) · 150 unmodified lines
151	151	TEST_FILES, TEST_GEN_FILES mean it is the file which is used by
152	152	test.
153	153
	154	+ * First use the headers inside the kernel source and/or git repo, and then the
	155	+ system headers. Headers for the kernel release as opposed to headers
	156	+ installed by the distro on the system should be the primary focus to be able
	157	+ to find regressions.
	158	+
154	159	Test Harness
155	160	============
156	161

Documentation/driver-api/clk.rstrenamed from Documentation/clk.txt+0−0

File renamed without changes.

Documentation/driver-api/device_connection.rst+1−1

@@ -40,4 +40,4 @@ API · 39 unmodified lines
40	40	---
41	41
42	42	.. kernel-doc:: drivers/base/devcon.c
43		− : functions: device_connection_find_match device_connection_find device_connection_add device_connection_remove
	43	+ :functions: device_connection_find_match device_connection_find device_connection_add device_connection_remove

Documentation/driver-api/gpio/driver.rst+3−3

@@ -44,7 +44,7 @@ common to each controller of that type: · 43 unmodified lines
44	44
45	45	- methods to establish GPIO line direction
46	46	- methods used to access GPIO line values
47		− - method to set electrical configuration to a a given GPIO line
	47	+ - method to set electrical configuration for a given GPIO line
48	48	- method to return the IRQ number associated to a given GPIO line
49	49	- flag saying whether calls to its methods may sleep
50	50	- optional line names array to identify lines
@@ -143,7 +143,7 @@ resistor will make the line tend to high level unless one of the transistors on · 92 unmodified lines
143	143	the rail actively pulls it down.
144	144
145	145	The level on the line will go as high as the VDD on the pull-up resistor, which
146		−may be higher than the level supported by the transistor, achieveing a
	146	+may be higher than the level supported by the transistor, achieving a
147	147	level-shift to the higher VDD.
148	148
149	149	Integrated electronics often have an output driver stage in the form of a CMOS
@@ -382,7 +382,7 @@ Real-Time compliance for GPIO IRQ chips · 232 unmodified lines
382	382
383	383	Any provider of irqchips needs to be carefully tailored to support Real Time
384	384	preemption. It is desirable that all irqchips in the GPIO subsystem keep this
385		−in mind and does the proper testing to assure they are real time-enabled.
	385	+in mind and do the proper testing to assure they are real time-enabled.
386	386	So, pay attention on above " RT_FULL:" notes, please.
387	387	The following is a checklist to follow when preparing a driver for real
388	388	time-compliance:

Documentation/driver-api/index.rst+2−0

@@ -17,7 +17,9 @@ available subsections can be seen below. · 16 unmodified lines
17	17	basics
18	18	infrastructure
19	19	pm/index
	20	+ clk
20	21	device-io
	22	+ device_connection
21	23	dma-buf
22	24	device_link
23	25	message-based

Documentation/driver-api/uio-howto.rst+2−1

@@ -711,7 +711,8 @@ The vmbus device regions are mapped into uio device resources: · 710 unmodified lines
711	711
712	712	If a subchannel is created by a request to host, then the uio_hv_generic
713	713	device driver will create a sysfs binary file for the per-channel ring buffer.
714		−For example:
	714	+For example::
	715	+
715	716	/sys/bus/vmbus/devices/3811fe4d-0fa0-4b62-981a-74fc1084c757/channels/21/ring
716	717
717	718	Further information

Documentation/features/core/cBPF-JIT/arch-support.txtrenamed from Documentation/features/lib/strncasecmp/arch-support.txt+8−6

@@ -1,7 +1,7 @@
1	1	#
2		−# Feature name: strncasecmp
3		−# Kconfig: __HAVE_ARCH_STRNCASECMP
4		−# description: arch provides an optimized strncasecmp() function
	2	+# Feature name: cBPF-JIT
	3	+# Kconfig: HAVE_CBPF_JIT
	4	+# description: arch supports cBPF JIT optimizations
5	5	#
6	6	-----------------------
7	7	\| arch \|status\|
@@ -16,14 +16,16 @@ · 8 unmodified lines
16	16	\| ia64: \| TODO \|
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19		− \| mips: \| TODO \|
	19	+ \| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23		− \| powerpc: \| TODO \|
	24	+ \| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26		− \| sparc: \| TODO \|
	28	+ \| sparc: \| ok \|
27	29	\| um: \| TODO \|
28	30	\| unicore32: \| TODO \|
29	31	\| x86: \| TODO \|

Documentation/features/core/eBPF-JIT/arch-support.txtrenamed from Documentation/features/core/BPF-JIT/arch-support.txt+5−3

@@ -1,7 +1,7 @@
1	1	#
2		−# Feature name: BPF-JIT
3		−# Kconfig: HAVE_BPF_JIT
4		−# description: arch supports BPF JIT optimizations
	2	+# Feature name: eBPF-JIT
	3	+# Kconfig: HAVE_EBPF_JIT
	4	+# description: arch supports eBPF JIT optimizations
5	5	#
6	6	-----------------------
7	7	\| arch \|status\|
@@ -17,10 +17,12 @@ · 9 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| ok \|

Documentation/features/core/generic-idle-thread/arch-support.txt+3−1

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21		− \| openrisc: \| TODO \|
	22	+ \| openrisc: \| ok \|
22	23	\| parisc: \| ok \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| ok \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/core/jump-labels/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| ok \|

Documentation/features/core/tracehook/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| ok \|
20	21	\| nios2: \| ok \|
21	22	\| openrisc: \| ok \|
22	23	\| parisc: \| ok \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| ok \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/debug/KASAN/arch-support.txt+3−1

@@ -17,15 +17,17 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|
27	29	\| um: \| TODO \|
28	30	\| unicore32: \| TODO \|
29		− \| x86: \| ok \| 64-bit only
	31	+ \| x86: \| ok \|
30	32	\| xtensa: \| ok \|
31	33	-----------------------

Documentation/features/debug/gcov-profile-all/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| ok \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| TODO \|

Documentation/features/debug/kgdb/arch-support.txt+3−1

@@ -11,16 +11,18 @@ · 10 unmodified lines
11	11	\| arm: \| ok \|
12	12	\| arm64: \| ok \|
13	13	\| c6x: \| TODO \|
14		− \| h8300: \| TODO \|
	14	+ \| h8300: \| ok \|
15	15	\| hexagon: \| ok \|
16	16	\| ia64: \| TODO \|
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| ok \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| ok \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/debug/kprobes-on-ftrace/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/debug/kprobes/arch-support.txt+3−1

@@ -9,7 +9,7 @@ · 8 unmodified lines
9	9	\| alpha: \| TODO \|
10	10	\| arc: \| ok \|
11	11	\| arm: \| ok \|
12		− \| arm64: \| TODO \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| TODO \|
14	14	\| h8300: \| TODO \|
15	15	\| hexagon: \| TODO \|
@@ -17,10 +17,12 @@ · 1 unmodified line
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| ok \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/debug/kretprobes/arch-support.txt+3−1

@@ -9,7 +9,7 @@ · 8 unmodified lines
9	9	\| alpha: \| TODO \|
10	10	\| arc: \| ok \|
11	11	\| arm: \| ok \|
12		− \| arm64: \| TODO \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| TODO \|
14	14	\| h8300: \| TODO \|
15	15	\| hexagon: \| TODO \|
@@ -17,10 +17,12 @@ · 1 unmodified line
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/debug/optprobes/arch-support.txt+3−1

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23		− \| powerpc: \| TODO \|
	24	+ \| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/debug/stackprotector/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| TODO \|

Documentation/features/debug/uprobes/arch-support.txt+4−2

@@ -9,7 +9,7 @@ · 8 unmodified lines
9	9	\| alpha: \| TODO \|
10	10	\| arc: \| TODO \|
11	11	\| arm: \| ok \|
12		− \| arm64: \| TODO \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| TODO \|
14	14	\| h8300: \| TODO \|
15	15	\| hexagon: \| TODO \|
@@ -17,13 +17,15 @@ · 1 unmodified line
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| TODO \|
26		− \| sparc: \| TODO \|
	28	+ \| sparc: \| ok \|
27	29	\| um: \| TODO \|
28	30	\| unicore32: \| TODO \|
29	31	\| x86: \| ok \|

Documentation/features/debug/user-ret-profiler/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/io/dma-contiguous/arch-support.txt+3−1

@@ -17,11 +17,13 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
24		− \| s390: \| TODO \|
	25	+ \| riscv: \| ok \|
	26	+ \| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|
27	29	\| um: \| TODO \|

Documentation/features/io/sg-chain/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| ok \|

Documentation/features/locking/cmpxchg-local/arch-support.txt+3−1

@@ -9,7 +9,7 @@ · 8 unmodified lines
9	9	\| alpha: \| TODO \|
10	10	\| arc: \| TODO \|
11	11	\| arm: \| TODO \|
12		− \| arm64: \| TODO \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| TODO \|
14	14	\| h8300: \| TODO \|
15	15	\| hexagon: \| TODO \|
@@ -17,10 +17,12 @@ · 1 unmodified line
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/locking/lockdep/arch-support.txt+3−1

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| ok \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| ok \|
20	21	\| nios2: \| TODO \|
21		− \| openrisc: \| TODO \|
	22	+ \| openrisc: \| ok \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/locking/queued-rwlocks/arch-support.txt+6−4

@@ -9,21 +9,23 @@ · 8 unmodified lines
9	9	\| alpha: \| TODO \|
10	10	\| arc: \| TODO \|
11	11	\| arm: \| TODO \|
12		− \| arm64: \| TODO \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| TODO \|
14	14	\| h8300: \| TODO \|
15	15	\| hexagon: \| TODO \|
16	16	\| ia64: \| TODO \|
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19		− \| mips: \| TODO \|
	19	+ \| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21		− \| openrisc: \| TODO \|
	22	+ \| openrisc: \| ok \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26		− \| sparc: \| TODO \|
	28	+ \| sparc: \| ok \|
27	29	\| um: \| TODO \|
28	30	\| unicore32: \| TODO \|
29	31	\| x86: \| ok \|

Documentation/features/locking/queued-spinlocks/arch-support.txt+5−3

@@ -16,14 +16,16 @@ · 15 unmodified lines
16	16	\| ia64: \| TODO \|
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19		− \| mips: \| TODO \|
	19	+ \| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21		− \| openrisc: \| TODO \|
	22	+ \| openrisc: \| ok \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26		− \| sparc: \| TODO \|
	28	+ \| sparc: \| ok \|
27	29	\| um: \| TODO \|
28	30	\| unicore32: \| TODO \|
29	31	\| x86: \| ok \|

Documentation/features/locking/rwsem-optimized/arch-support.txt+6−4

@@ -1,6 +1,6 @@
1	1	#
2	2	# Feature name: rwsem-optimized
3		−# Kconfig: Optimized asm/rwsem.h
	3	+# Kconfig: !RWSEM_GENERIC_SPINLOCK
4	4	# description: arch provides optimized rwsem APIs
5	5	#
6	6	-----------------------
@@ -8,8 +8,8 @@ · 1 unmodified line
8	8	-----------------------
9	9	\| alpha: \| ok \|
10	10	\| arc: \| TODO \|
11		− \| arm: \| TODO \|
12		− \| arm64: \| TODO \|
	11	+ \| arm: \| ok \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| TODO \|
14	14	\| h8300: \| TODO \|
15	15	\| hexagon: \| TODO \|
@@ -17,14 +17,16 @@ · 1 unmodified line
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|
27		− \| um: \| TODO \|
	29	+ \| um: \| ok \|
28	30	\| unicore32: \| TODO \|
29	31	\| x86: \| ok \|
30	32	\| xtensa: \| ok \|

Documentation/features/perf/kprobes-event/arch-support.txt+4−2

@@ -9,7 +9,7 @@ · 8 unmodified lines
9	9	\| alpha: \| TODO \|
10	10	\| arc: \| TODO \|
11	11	\| arm: \| ok \|
12		− \| arm64: \| TODO \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| TODO \|
14	14	\| h8300: \| TODO \|
15	15	\| hexagon: \| ok \|
@@ -17,13 +17,15 @@ · 1 unmodified line
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| ok \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26		− \| sparc: \| TODO \|
	28	+ \| sparc: \| ok \|
27	29	\| um: \| TODO \|
28	30	\| unicore32: \| TODO \|
29	31	\| x86: \| ok \|

Documentation/features/perf/perf-regs/arch-support.txt+3−1

@@ -17,11 +17,13 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
24		− \| s390: \| TODO \|
	25	+ \| riscv: \| TODO \|
	26	+ \| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|
27	29	\| um: \| TODO \|

Documentation/features/perf/perf-stackdump/arch-support.txt+3−1

@@ -17,11 +17,13 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
24		− \| s390: \| TODO \|
	25	+ \| riscv: \| TODO \|
	26	+ \| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|
27	29	\| um: \| TODO \|

Documentation/features/sched/membarrier-sync-core/arch-support.txt+2−0

@@ -40,10 +40,12 @@ · 39 unmodified lines
40	40	\| m68k: \| TODO \|
41	41	\| microblaze: \| TODO \|
42	42	\| mips: \| TODO \|
	43	+ \| nds32: \| TODO \|
43	44	\| nios2: \| TODO \|
44	45	\| openrisc: \| TODO \|
45	46	\| parisc: \| TODO \|
46	47	\| powerpc: \| TODO \|
	48	+ \| riscv: \| TODO \|
47	49	\| s390: \| TODO \|
48	50	\| sh: \| TODO \|
49	51	\| sparc: \| TODO \|

Documentation/features/sched/numa-balancing/arch-support.txt+4−2

@@ -9,7 +9,7 @@ · 8 unmodified lines
9	9	\| alpha: \| TODO \|
10	10	\| arc: \| .. \|
11	11	\| arm: \| .. \|
12		− \| arm64: \| .. \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| .. \|
14	14	\| h8300: \| .. \|
15	15	\| hexagon: \| .. \|
@@ -17,11 +17,13 @@ · 1 unmodified line
17	17	\| m68k: \| .. \|
18	18	\| microblaze: \| .. \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| .. \|
21	22	\| openrisc: \| .. \|
22	23	\| parisc: \| .. \|
23	24	\| powerpc: \| ok \|
24		− \| s390: \| .. \|
	25	+ \| riscv: \| TODO \|
	26	+ \| s390: \| ok \|
25	27	\| sh: \| .. \|
26	28	\| sparc: \| TODO \|
27	29	\| um: \| .. \|

Documentation/features/scripts/features-refresh.sh+98−0

@@ -0,0 +1,98 @@
	1	+#
	2	+# Small script that refreshes the kernel feature support status in place.
	3	+#
	4	+
	5	+for F_FILE in Documentation/features///arch-support.txt; do
	6	+ F=$(grep "^# Kconfig:" "$F_FILE" \| cut -c26-)
	7	+
	8	+ #
	9	+ # Each feature F is identified by a pair (O, K), where 'O' can
	10	+ # be either the empty string (for 'nop') or "not" (the logical
	11	+ # negation operator '!'); other operators are not supported.
	12	+ #
	13	+ O=""
	14	+ K=$F
	15	+ if [[ "$F" == !* ]]; then
	16	+ O="not"
	17	+ K=$(echo $F \| sed -e 's/^!//g')
	18	+ fi
	19	+
	20	+ #
	21	+ # F := (O, K) is 'valid' iff there is a Kconfig file (for some
	22	+ # arch) which contains K.
	23	+ #
	24	+ # Notice that this definition entails an 'asymmetry' between
	25	+ # the case 'O = ""' and the case 'O = "not"'. E.g., F may be
	26	+ # _invalid_ if:
	27	+ #
	28	+ # [case 'O = ""']
	29	+ # 1) no arch provides support for F,
	30	+ # 2) K does not exist (e.g., it was renamed/mis-typed);
	31	+ #
	32	+ # [case 'O = "not"']
	33	+ # 3) all archs provide support for F,
	34	+ # 4) as in (2).
	35	+ #
	36	+ # The rationale for adopting this definition (and, thus, for
	37	+ # keeping the asymmetry) is:
	38	+ #
	39	+ # We want to be able to 'detect' (2) (or (4)).
	40	+ #
	41	+ # (1) and (3) may further warn the developers about the fact
	42	+ # that K can be removed.
	43	+ #
	44	+ F_VALID="false"
	45	+ for ARCH_DIR in arch/*/; do
	46	+ K_FILES=$(find $ARCH_DIR -name "Kconfig*")
	47	+ K_GREP=$(grep "$K" $K_FILES)
	48	+ if [ ! -z "$K_GREP" ]; then
	49	+ F_VALID="true"
	50	+ break
	51	+ fi
	52	+ done
	53	+ if [ "$F_VALID" = "false" ]; then
	54	+ printf "WARNING: '%s' is not a valid Kconfig\n" "$F"
	55	+ fi
	56	+
	57	+ T_FILE="$F_FILE.tmp"
	58	+ grep "^#" $F_FILE > $T_FILE
	59	+ echo " -----------------------" >> $T_FILE
	60	+ echo " \| arch \|status\|" >> $T_FILE
	61	+ echo " -----------------------" >> $T_FILE
	62	+ for ARCH_DIR in arch/*/; do
	63	+ ARCH=$(echo $ARCH_DIR \| sed -e 's/arch//g' \| sed -e 's/\///g')
	64	+ K_FILES=$(find $ARCH_DIR -name "Kconfig*")
	65	+ K_GREP=$(grep "$K" $K_FILES)
	66	+ #
	67	+ # Arch support status values for (O, K) are updated according
	68	+ # to the following rules.
	69	+ #
	70	+ # - ("", K) is 'supported by a given arch', if there is a
	71	+ # Kconfig file for that arch which contains K;
	72	+ #
	73	+ # - ("not", K) is 'supported by a given arch', if there is
	74	+ # no Kconfig file for that arch which contains K;
	75	+ #
	76	+ # - otherwise: preserve the previous status value (if any),
	77	+ # default to 'not yet supported'.
	78	+ #
	79	+ # Notice that, according these rules, invalid features may be
	80	+ # updated/modified.
	81	+ #
	82	+ if [ "$O" = "" ] && [ ! -z "$K_GREP" ]; then
	83	+ printf " \|%12s: \| ok \|\n" "$ARCH" >> $T_FILE
	84	+ elif [ "$O" = "not" ] && [ -z "$K_GREP" ]; then
	85	+ printf " \|%12s: \| ok \|\n" "$ARCH" >> $T_FILE
	86	+ else
	87	+ S=$(grep -v "^#" "$F_FILE" \| grep " $ARCH:")
	88	+ if [ ! -z "$S" ]; then
	89	+ echo "$S" >> $T_FILE
	90	+ else
	91	+ printf " \|%12s: \| TODO \|\n" "$ARCH" \
	92	+ >> $T_FILE
	93	+ fi
	94	+ fi
	95	+ done
	96	+ echo " -----------------------" >> $T_FILE
	97	+ mv $T_FILE $F_FILE
	98	+done

Documentation/features/seccomp/seccomp-filter/arch-support.txt+4−2

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22		− \| parisc: \| TODO \|
23		− \| powerpc: \| TODO \|
	23	+ \| parisc: \| ok \|
	24	+ \| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/time/arch-tick-broadcast/arch-support.txt+3−1

@@ -17,12 +17,14 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25		− \| sh: \| TODO \|
	27	+ \| sh: \| ok \|
26	28	\| sparc: \| TODO \|
27	29	\| um: \| TODO \|
28	30	\| unicore32: \| TODO \|

Documentation/features/time/clockevents/arch-support.txt+3−1

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| ok \|
18	18	\| microblaze: \| ok \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| ok \|
20	21	\| nios2: \| ok \|
21	22	\| openrisc: \| ok \|
22		− \| parisc: \| TODO \|
	23	+ \| parisc: \| ok \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| ok \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/time/context-tracking/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| ok \|

Documentation/features/time/irq-time-acct/arch-support.txt+3−1

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| .. \|
23		− \| powerpc: \| .. \|
	24	+ \| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| .. \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| .. \|

Documentation/features/time/modern-timekeeping/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| ok \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| ok \|
20	21	\| nios2: \| ok \|
21	22	\| openrisc: \| ok \|
22	23	\| parisc: \| ok \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| ok \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/time/virt-cpuacct/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| ok \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| ok \|

Documentation/features/vm/ELF-ASLR/arch-support.txt+3−1

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22		− \| parisc: \| TODO \|
	23	+ \| parisc: \| ok \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/vm/PG_uncached/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/vm/THP/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| .. \|
18	18	\| microblaze: \| .. \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| .. \|
21	22	\| openrisc: \| .. \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| .. \|
26	28	\| sparc: \| ok \|

Documentation/features/vm/TLB/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| .. \|
18	18	\| microblaze: \| .. \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| .. \|
21	22	\| openrisc: \| .. \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/vm/huge-vmap/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| TODO \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| TODO \|
26	28	\| sparc: \| TODO \|

Documentation/features/vm/ioremap_prot/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| TODO \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| TODO \|

Documentation/features/vm/numa-memblock/arch-support.txt+3−1

@@ -9,7 +9,7 @@ · 8 unmodified lines
9	9	\| alpha: \| TODO \|
10	10	\| arc: \| .. \|
11	11	\| arm: \| .. \|
12		− \| arm64: \| .. \|
	12	+ \| arm64: \| ok \|
13	13	\| c6x: \| .. \|
14	14	\| h8300: \| .. \|
15	15	\| hexagon: \| .. \|
@@ -17,10 +17,12 @@ · 1 unmodified line
17	17	\| m68k: \| .. \|
18	18	\| microblaze: \| ok \|
19	19	\| mips: \| ok \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| .. \|
21	22	\| openrisc: \| .. \|
22	23	\| parisc: \| .. \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| ok \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/features/vm/pte_special/arch-support.txt+2−0

@@ -17,10 +17,12 @@ · 16 unmodified lines
17	17	\| m68k: \| TODO \|
18	18	\| microblaze: \| TODO \|
19	19	\| mips: \| TODO \|
	20	+ \| nds32: \| TODO \|
20	21	\| nios2: \| TODO \|
21	22	\| openrisc: \| TODO \|
22	23	\| parisc: \| TODO \|
23	24	\| powerpc: \| ok \|
	25	+ \| riscv: \| TODO \|
24	26	\| s390: \| ok \|
25	27	\| sh: \| ok \|
26	28	\| sparc: \| ok \|

Documentation/filesystems/proc.txt+5−3

@@ -515,7 +515,8 @@ guarantees: · 514 unmodified lines
515	515
516	516	The /proc/PID/clear_refs is used to reset the PG_Referenced and ACCESSED/YOUNG
517	517	bits on both physical and virtual pages associated with a process, and the
518		−soft-dirty bit on pte (see Documentation/vm/soft-dirty.txt for details).
	518	+soft-dirty bit on pte (see Documentation/admin-guide/mm/soft-dirty.rst
	519	+for details).
519	520	To clear the bits for all the pages associated with the process
520	521	> echo 1 > /proc/PID/clear_refs
521	522
@@ -536,7 +537,8 @@ Any other value written to /proc/PID/clear_refs will have no effect. · 14 unmodified lines
536	537
537	538	The /proc/pid/pagemap gives the PFN, which can be used to find the pageflags
538	539	using /proc/kpageflags and number of times a page is mapped using
539		−/proc/kpagecount. For detailed explanation, see Documentation/vm/pagemap.txt.
	540	+/proc/kpagecount. For detailed explanation, see
	541	+Documentation/admin-guide/mm/pagemap.rst.
540	542
541	543	The /proc/pid/numa_maps is an extension based on maps, showing the memory
542	544	locality and binding policy, as well as the memory usage (in pages) of
@@ -564,7 +566,7 @@ address policy mapping details · 21 unmodified lines
564	566
565	567	Where:
566	568	"address" is the starting address for the mapping;
567		−"policy" reports the NUMA memory policy set for the mapping (see vm/numa_memory_policy.txt);
	569	+"policy" reports the NUMA memory policy set for the mapping (see Documentation/admin-guide/mm/numa_memory_policy.rst);
568	570	"mapping details" summarizes mapping data such as mapping type, page usage counters,
569	571	node locality page counters (N0 == node0, N1 == node1, ...) and the kernel page
570	572	size, in KB, that is backing the mapping up.

Documentation/filesystems/tmpfs.txt+3−2

@@ -105,8 +105,9 @@ policy for the file will revert to "default" policy. · 104 unmodified lines
105	105	NUMA memory allocation policies have optional flags that can be used in
106	106	conjunction with their modes. These optional flags can be specified
107	107	when tmpfs is mounted by appending them to the mode before the NodeList.
108		−See Documentation/vm/numa_memory_policy.txt for a list of all available
109		−memory allocation policy mode flags and their effect on memory policy.
	108	+See Documentation/admin-guide/mm/numa_memory_policy.rst for a list of
	109	+all available memory allocation policy mode flags and their effect on
	110	+memory policy.
110	111
111	112	=static is equivalent to MPOL_F_STATIC_NODES
112	113	=relative is equivalent to MPOL_F_RELATIVE_NODES

Documentation/index.rst+2−1

@@ -45,7 +45,7 @@ the kernel interface as seen by application developers. · 44 unmodified lines
45	45	.. toctree::
46	46	:maxdepth: 2
47	47
48		− userspace-api/index
	48	+ userspace-api/index
49	49
50	50
51	51	Introduction to kernel development
@@ -89,6 +89,7 @@ needed). · 37 unmodified lines
89	89	sound/index
90	90	crypto/index
91	91	filesystems/index
	92	+ vm/index
92	93
93	94	Architecture-specific documentation
94	95	-----------------------------------

Documentation/ioctl/botching-up-ioctls.txt+3−1

@@ -73,7 +73,9 @@ will have a second iteration or at least an extension for any given interface. · 72 unmodified lines
73	73	future extensions is going right down the gutters since someone will submit
74	74	an ioctl struct with random stack garbage in the yet unused parts. Which
75	75	then bakes in the ABI that those fields can never be used for anything else
76		− but garbage.
	76	+ but garbage. This is also the reason why you must explicitly pad all
	77	+ structures, even if you never use them in an array - the padding the compiler
	78	+ might insert could contain garbage.
77	79
78	80	* Have simple testcases for all of the above.
79	81

Documentation/memory-barriers.txt+2−2

@@ -2903,7 +2903,7 @@ is discarded from the CPU's cache and reloaded. To deal with this, the · 2902 unmodified lines
2903	2903	appropriate part of the kernel must invalidate the overlapping bits of the
2904	2904	cache on each CPU.
2905	2905
2906		−See Documentation/cachetlb.txt for more information on cache management.
	2906	+See Documentation/core-api/cachetlb.rst for more information on cache management.
2907	2907
2908	2908
2909	2909	CACHE COHERENCY VS MMIO
@@ -3083,7 +3083,7 @@ CIRCULAR BUFFERS · 173 unmodified lines
3083	3083	Memory barriers can be used to implement circular buffering without the need
3084	3084	of a lock to serialise the producer with the consumer. See:
3085	3085
3086		− Documentation/circular-buffers.txt
	3086	+ Documentation/core-api/circular-buffers.rst
3087	3087
3088	3088	for details.
3089	3089

Documentation/process/2.Process.rst+38−34

@@ -18,17 +18,17 @@ major kernel release happening every two or three months. The recent · 17 unmodified lines
18	18	release history looks like this:
19	19
20	20	====== =================
21		− 2.6.38 March 14, 2011
22		− 2.6.37 January 4, 2011
23		− 2.6.36 October 20, 2010
24		− 2.6.35 August 1, 2010
25		− 2.6.34 May 15, 2010
26		− 2.6.33 February 24, 2010
	21	+ 4.11 April 30, 2017
	22	+ 4.12 July 2, 2017
	23	+ 4.13 September 3, 2017
	24	+ 4.14 November 12, 2017
	25	+ 4.15 January 28, 2018
	26	+ 4.16 April 1, 2018
27	27	====== =================
28	28
29		−Every 2.6.x release is a major kernel release with new features, internal
30		−API changes, and more. A typical 2.6 release can contain nearly 10,000
31		−changesets with changes to several hundred thousand lines of code. 2.6 is
	29	+Every 4.x release is a major kernel release with new features, internal
	30	+API changes, and more. A typical 4.x release contain about 13,000
	31	+changesets with changes to several hundred thousand lines of code. 4.x is
32	32	thus the leading edge of Linux kernel development; the kernel uses a
33	33	rolling development model which is continually integrating major changes.
34	34
@@ -70,20 +70,19 @@ will get up to somewhere between -rc6 and -rc9 before the kernel is · 35 unmodified lines
70	70	considered to be sufficiently stable and the final 2.6.x release is made.
71	71	At that point the whole process starts over again.
72	72
73		−As an example, here is how the 2.6.38 development cycle went (all dates in
74		−2011):
	73	+As an example, here is how the 4.16 development cycle went (all dates in
	74	+2018):
75	75
76	76	============== ===============================
77		− January 4 2.6.37 stable release
78		− January 18 2.6.38-rc1, merge window closes
79		− January 21 2.6.38-rc2
80		− February 1 2.6.38-rc3
81		− February 7 2.6.38-rc4
82		− February 15 2.6.38-rc5
83		− February 21 2.6.38-rc6
84		− March 1 2.6.38-rc7
85		− March 7 2.6.38-rc8
86		− March 14 2.6.38 stable release
	77	+ January 28 4.15 stable release
	78	+ February 11 4.16-rc1, merge window closes
	79	+ February 18 4.16-rc2
	80	+ February 25 4.16-rc3
	81	+ March 4 4.16-rc4
	82	+ March 11 4.16-rc5
	83	+ March 18 4.16-rc6
	84	+ March 25 4.16-rc7
	85	+ April 1 4.17 stable release
87	86	============== ===============================
88	87
89	88	How do the developers decide when to close the development cycle and create
@@ -99,37 +98,42 @@ release is made. In the real world, this kind of perfection is hard to · 9 unmodified lines
99	98	achieve; there are just too many variables in a project of this size.
100	99	There comes a point where delaying the final release just makes the problem
101	100	worse; the pile of changes waiting for the next merge window will grow
102		−larger, creating even more regressions the next time around. So most 2.6.x
	101	+larger, creating even more regressions the next time around. So most 4.x
103	102	kernels go out with a handful of known regressions though, hopefully, none
104	103	of them are serious.
105	104
106	105	Once a stable release is made, its ongoing maintenance is passed off to the
107	106	"stable team," currently consisting of Greg Kroah-Hartman. The stable team
108		−will release occasional updates to the stable release using the 2.6.x.y
	107	+will release occasional updates to the stable release using the 4.x.y
109	108	numbering scheme. To be considered for an update release, a patch must (1)
110	109	fix a significant bug, and (2) already be merged into the mainline for the
111	110	next development kernel. Kernels will typically receive stable updates for
112	111	a little more than one development cycle past their initial release. So,
113		−for example, the 2.6.36 kernel's history looked like:
	112	+for example, the 4.13 kernel's history looked like:
114	113
115	114	============== ===============================
116		− October 10 2.6.36 stable release
117		− November 22 2.6.36.1
118		− December 9 2.6.36.2
119		− January 7 2.6.36.3
120		− February 17 2.6.36.4
	115	+ September 3 4.13 stable release
	116	+ September 13 4.13.1
	117	+ September 20 4.13.2
	118	+ September 27 4.13.3
	119	+ October 5 4.13.4
	120	+ October 12 4.13.5
	121	+ ... ...
	122	+ November 24 4.13.16
121	123	============== ===============================
122	124
123		−2.6.36.4 was the final stable update for the 2.6.36 release.
	125	+4.13.16 was the final stable update of the 4.13 release.
124	126
125	127	Some kernels are designated "long term" kernels; they will receive support
126	128	for a longer period. As of this writing, the current long term kernels
127	129	and their maintainers are:
128	130
129		− ====== ====================== ===========================
130		− 2.6.27 Willy Tarreau (Deep-frozen stable kernel)
131		− 2.6.32 Greg Kroah-Hartman
132		− 2.6.35 Andi Kleen (Embedded flag kernel)
	131	+ ====== ====================== ==============================
	132	+ 3.16 Ben Hutchings (very long-term stable kernel)
	133	+ 4.1 Sasha Levin
	134	+ 4.4 Greg Kroah-Hartman (very long-term stable kernel)
	135	+ 4.9 Greg Kroah-Hartman
	136	+ 4.14 Greg Kroah-Hartman
133	137	====== ====================== ===========================
134	138
135	139	The selection of a kernel for long-term support is purely a matter of a

Documentation/process/5.Posting.rst+8−8

@@ -10,8 +10,8 @@ of conventions and procedures which are used in the posting of patches; · 9 unmodified lines
10	10	following them will make life much easier for everybody involved. This
11	11	document will attempt to cover these expectations in reasonable detail;
12	12	more information can also be found in the files process/submitting-patches.rst,
13		−process/submitting-drivers.rst, and process/submit-checklist.rst in the kernel documentation
14		−directory.
	13	+process/submitting-drivers.rst, and process/submit-checklist.rst in the kernel
	14	+documentation directory.
15	15
16	16
17	17	When to post
@@ -198,8 +198,8 @@ pass it to diff with the "-X" option. · 180 unmodified lines
198	198
199	199	The tags mentioned above are used to describe how various developers have
200	200	been associated with the development of this patch. They are described in
201		−detail in the process/submitting-patches.rst document; what follows here is a brief
202		−summary. Each of these lines has the format:
	201	+detail in the process/submitting-patches.rst document; what follows here is a
	202	+brief summary. Each of these lines has the format:
203	203
204	204	::
205	205
@@ -210,8 +210,8 @@ The tags in common use are: · 4 unmodified lines
210	210	- Signed-off-by: this is a developer's certification that he or she has
211	211	the right to submit the patch for inclusion into the kernel. It is an
212	212	agreement to the Developer's Certificate of Origin, the full text of
213		− which can be found in Documentation/process/submitting-patches.rst. Code without a
214		− proper signoff cannot be merged into the mainline.
	213	+ which can be found in Documentation/process/submitting-patches.rst. Code
	214	+ without a proper signoff cannot be merged into the mainline.
215	215
216	216	- Co-developed-by: states that the patch was also created by another developer
217	217	along with the original author. This is useful at times when multiple
@@ -226,8 +226,8 @@ The tags in common use are: · 8 unmodified lines
226	226	it to work.
227	227
228	228	- Reviewed-by: the named developer has reviewed the patch for correctness;
229		− see the reviewer's statement in Documentation/process/submitting-patches.rst for more
230		− detail.
	229	+ see the reviewer's statement in Documentation/process/submitting-patches.rst
	230	+ for more detail.
231	231
232	232	- Reported-by: names a user who reported a problem which is fixed by this
233	233	patch; this tag is used to give credit to the (often underappreciated)

Documentation/process/index.rst+1−0

@@ -52,6 +52,7 @@ lack of a better place. · 51 unmodified lines
52	52	adding-syscalls
53	53	magic-number
54	54	volatile-considered-harmful
	55	+ clang-format
55	56
56	57	.. only:: subproject and html
57	58

Documentation/process/maintainer-pgp-guide.rst+37−2

@@ -219,7 +219,7 @@ Our goal is to protect your master key by moving it to offline media, so · 218 unmodified lines
219	219	if you only have a combined [SC] key, then you should create a separate
220	220	signing subkey::
221	221
222		− $ gpg --quick-add-key [fpr] ed25519 sign
	222	+ $ gpg --quick-addkey [fpr] ed25519 sign
223	223
224	224	Remember to tell the keyservers about this change, so others can pull down
225	225	your new subkey::
@@ -450,11 +450,18 @@ functionality. There are several options available: · 224 unmodified lines
450	450	others. If you want to use ECC keys, your best bet among commercially
451	451	available devices is the Nitrokey Start.
452	452
	453	+.. note::
	454	+
	455	+ If you are listed in MAINTAINERS or have an account at kernel.org,
	456	+ you `qualify for a free Nitrokey Start`_ courtesy of The Linux
	457	+ Foundation.
	458	+
453	459	.. _`Nitrokey Start`: https://shop.nitrokey.com/shop/product/nitrokey-start-6
454	460	.. _`Nitrokey Pro`: https://shop.nitrokey.com/shop/product/nitrokey-pro-3
455	461	.. _`Yubikey 4`: https://www.yubico.com/product/yubikey-4-series/
456	462	.. _Gnuk: http://www.fsij.org/doc-gnuk/
457	463	.. _`LWN has a good review`: https://lwn.net/Articles/736231/
	464	+.. _`qualify for a free Nitrokey Start`: https://www.kernel.org/nitrokey-digital-tokens-for-kernel-developers.html
458	465
459	466	Configure your smartcard device
460	467	-------------------------------
@@ -482,7 +489,7 @@ there are no convenient command-line switches:: · 21 unmodified lines
482	489	You should set the user PIN (1), Admin PIN (3), and the Reset Code (4).
483	490	Please make sure to record and store these in a safe place -- especially
484	491	the Admin PIN and the Reset Code (which allows you to completely wipe
485		−the smartcard). You so rarely need to use the Admin PIN, that you will
	492	+the smartcard). You so rarely need to use the Admin PIN, that you will
486	493	inevitably forget what it is if you do not record it.
487	494
488	495	Getting back to the main card menu, you can also set other values (such
@@ -494,6 +501,12 @@ additionally leak information about your smartcard should you lose it. · 5 unmodified lines
494	501	Despite having the name "PIN", neither the user PIN nor the admin
495	502	PIN on the card need to be numbers.
496	503
	504	+.. warning::
	505	+
	506	+ Some devices may require that you move the subkeys onto the device
	507	+ before you can change the passphrase. Please check the documentation
	508	+ provided by the device manufacturer.
	509	+
497	510	Move the subkeys to your smartcard
498	511	----------------------------------
499	512
@@ -655,6 +668,20 @@ want to import these changes back into your regular working directory:: · 155 unmodified lines
655	668	$ gpg --export \| gpg --homedir ~/.gnupg --import
656	669	$ unset GNUPGHOME
657	670
	671	+Using gpg-agent over ssh
	672	+~~~~~~~~~~~~~~~~~~~~~~~~
	673	+
	674	+You can forward your gpg-agent over ssh if you need to sign tags or
	675	+commits on a remote system. Please refer to the instructions provided
	676	+on the GnuPG wiki:
	677	+
	678	+- `Agent Forwarding over SSH`_
	679	+
	680	+It works more smoothly if you can modify the sshd server settings on the
	681	+remote end.
	682	+
	683	+.. _`Agent Forwarding over SSH`: https://wiki.gnupg.org/AgentForwarding
	684	+
658	685
659	686	Using PGP with Git
660	687	==================
@@ -692,6 +719,7 @@ should be used (``[fpr]`` is the fingerprint of your key):: · 31 unmodified lines
692	719	tell git to always use it instead of the legacy ``gpg`` from version 1::
693	720
694	721	$ git config --global gpg.program gpg2
	722	+ $ git config --global gpgv.program gpgv2
695	723
696	724	How to work with signed tags
697	725	----------------------------
@@ -731,6 +759,13 @@ If you are verifying someone else's git tag, then you will need to · 33 unmodified lines
731	759	import their PGP key. Please refer to the
732	760	":ref:`verify_identities`" section below.
733	761
	762	+.. note::
	763	+
	764	+ If you get "``gpg: Can't check signature: unknown pubkey
	765	+ algorithm``" error, you need to tell git to use gpgv2 for
	766	+ verification, so it properly processes signatures made by ECC keys.
	767	+ See instructions at the start of this section.
	768	+
734	769	Configure git to always sign annotated tags
735	770	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
736	771

Documentation/process/submitting-patches.rst+1−1

@@ -761,7 +761,7 @@ requests, especially from new, unknown developers. If in doubt you can use · 760 unmodified lines
761	761	the pull request as the cover letter for a normal posting of the patch
762	762	series, giving the maintainer the option of using either.
763	763
764		−A pull request should have [GIT] or [PULL] in the subject line. The
	764	+A pull request should have [GIT PULL] in the subject line. The
765	765	request itself should include the repository name and the branch of
766	766	interest on a single line; it should look something like::
767	767

Documentation/security/index.rst+2−0

@@ -9,5 +9,7 @@ Security Documentation · 8 unmodified lines
9	9	IMA-templates
10	10	keys/index
11	11	LSM
	12	+ LSM-sctp
	13	+ SELinux-sctp
12	14	self-protection
13	15	tpm/index

Documentation/sound/alsa-configuration.rst+2−2

@@ -1062,7 +1062,7 @@ output (with ``--no-upload`` option) to kernel bugzilla or alsa-devel · 1061 unmodified lines
1062	1062	ML (see the section `Links and Addresses`_).
1063	1063
1064	1064	``power_save`` and ``power_save_controller`` options are for power-saving
1065		−mode. See powersave.txt for details.
	1065	+mode. See powersave.rst for details.
1066	1066
1067	1067	Note 2: If you get click noises on output, try the module option
1068	1068	``position_fix=1`` or ``2``. ``position_fix=1`` will use the SD_LPIB
@@ -1133,7 +1133,7 @@ line_outs_monitor · 64 unmodified lines
1133	1133	enable_monitor
1134	1134	Enable Analog Out on Channel 63/64 by default.
1135	1135
1136		−See hdspm.txt for details.
	1136	+See hdspm.rst for details.
1137	1137
1138	1138	Module snd-ice1712
1139	1139	------------------

Documentation/sound/soc/codec.rst+1−1

@@ -139,7 +139,7 @@ DAPM description · 138 unmodified lines
139	139	----------------
140	140	The Dynamic Audio Power Management description describes the codec power
141	141	components and their relationships and registers to the ASoC core.
142		−Please read dapm.txt for details of building the description.
	142	+Please read dapm.rst for details of building the description.
143	143
144	144	Please also see the examples in other codec drivers.
145	145

Documentation/sound/soc/platform.rst+1−1

@@ -66,7 +66,7 @@ Each SoC DAI driver must provide the following features:- · 65 unmodified lines
66	66	4. SYSCLK configuration
67	67	5. Suspend and resume (optional)
68	68
69		−Please see codec.txt for a description of items 1 - 4.
	69	+Please see codec.rst for a description of items 1 - 4.
70	70
71	71
72	72	SoC DSP Drivers

Documentation/sysctl/vm.txt+3−3

@@ -515,7 +515,7 @@ nr_hugepages · 514 unmodified lines
515	515
516	516	Change the minimum size of the hugepage pool.
517	517
518		−See Documentation/vm/hugetlbpage.txt
	518	+See Documentation/admin-guide/mm/hugetlbpage.rst
519	519
520	520	==============================================================
521	521
@@ -524,7 +524,7 @@ nr_overcommit_hugepages · 2 unmodified lines
524	524	Change the maximum size of the hugepage pool. The maximum is
525	525	nr_hugepages + nr_overcommit_hugepages.
526	526
527		−See Documentation/vm/hugetlbpage.txt
	527	+See Documentation/admin-guide/mm/hugetlbpage.rst
528	528
529	529	==============================================================
530	530
@@ -667,7 +667,7 @@ and don't use much of it. · 136 unmodified lines
667	667
668	668	The default value is 0.
669	669
670		−See Documentation/vm/overcommit-accounting and
	670	+See Documentation/vm/overcommit-accounting.rst and
671	671	mm/mmap.c::__vm_enough_memory() for more information.
672	672
673	673	==============================================================

Documentation/trace/coresight.txt+75−28

@@ -187,13 +187,19 @@ that can be performed on them (see "struct coresight_ops"). The · 186 unmodified lines
187	187	specific to that component only. "Implementation defined" customisations are
188	188	expected to be accessed and controlled using those entries.
189	189
190		−Last but not least, "struct module *owner" is expected to be set to reflect
191		−the information carried in "THIS_MODULE".
192	190
193	191	How to use the tracer modules
194	192	-----------------------------
195	193
196		−Before trace collection can start, a coresight sink needs to be identify.
	194	+There are two ways to use the Coresight framework: 1) using the perf cmd line
	195	+tools and 2) interacting directly with the Coresight devices using the sysFS
	196	+interface. Preference is given to the former as using the sysFS interface
	197	+requires a deep understanding of the Coresight HW. The following sections
	198	+provide details on using both methods.
	199	+
	200	+1) Using the sysFS interface:
	201	+
	202	+Before trace collection can start, a coresight sink needs to be identified.
197	203	There is no limit on the amount of sinks (nor sources) that can be enabled at
198	204	any given moment. As a generic operation, all device pertaining to the sink
199	205	class will have an "active" entry in sysfs:
@@ -298,42 +304,48 @@ Instruction 13570831 0x8026B584 E28DD00C false ADD · 98 unmodified lines
298	304	Instruction 0 0x8026B588 E8BD8000 true LDM sp!,{pc}
299	305	Timestamp Timestamp: 17107041535
300	306
301		−How to use the STM module
302		−-------------------------
	307	+2) Using perf framework:
303	308
304		−Using the System Trace Macrocell module is the same as the tracers - the only
305		−difference is that clients are driving the trace capture rather
306		−than the program flow through the code.
	309	+Coresight tracers are represented using the Perf framework's Performance
	310	+Monitoring Unit (PMU) abstraction. As such the perf framework takes charge of
	311	+controlling when tracing gets enabled based on when the process of interest is
	312	+scheduled. When configured in a system, Coresight PMUs will be listed when
	313	+queried by the perf command line tool:
307	314
308		−As with any other CoreSight component, specifics about the STM tracer can be
309		−found in sysfs with more information on each entry being found in [1]:
	315	+ linaro@linaro-nano:~$ ./perf list pmu
310	316
311		−root@genericarmv8:~# ls /sys/bus/coresight/devices/20100000.stm
312		−enable_source hwevent_select port_enable subsystem uevent
313		−hwevent_enable mgmt port_select traceid
314		−root@genericarmv8:~#
	317	+ List of pre-defined events (to be used in -e):
315	318
316		−Like any other source a sink needs to be identified and the STM enabled before
317		−being used:
	319	+ cs_etm// [Kernel PMU event]
318	320
319		−root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/20010000.etf/enable_sink
320		−root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/20100000.stm/enable_source
	321	+ linaro@linaro-nano:~$
321	322
322		−From there user space applications can request and use channels using the devfs
323		−interface provided for that purpose by the generic STM API:
	323	+Regardless of the number of tracers available in a system (usually equal to the
	324	+amount of processor cores), the "cs_etm" PMU will be listed only once.
324	325
325		−root@genericarmv8:~# ls -l /dev/20100000.stm
326		−crw------- 1 root root 10, 61 Jan 3 18:11 /dev/20100000.stm
327		−root@genericarmv8:~#
	326	+A Coresight PMU works the same way as any other PMU, i.e the name of the PMU is
	327	+listed along with configuration options within forward slashes '/'. Since a
	328	+Coresight system will typically have more than one sink, the name of the sink to
	329	+work with needs to be specified as an event option. Names for sink to choose
	330	+from are listed in sysFS under ($SYSFS)/bus/coresight/devices:
328	331
329		−Details on how to use the generic STM API can be found here [2].
	332	+ root@linaro-nano:~# ls /sys/bus/coresight/devices/
	333	+ 20010000.etf 20040000.funnel 20100000.stm 22040000.etm
	334	+ 22140000.etm 230c0000.funnel 23240000.etm 20030000.tpiu
	335	+ 20070000.etr 20120000.replicator 220c0000.funnel
	336	+ 23040000.etm 23140000.etm 23340000.etm
330	337
331		−[1]. Documentation/ABI/testing/sysfs-bus-coresight-devices-stm
332		−[2]. Documentation/trace/stm.txt
	338	+ root@linaro-nano:~# perf record -e cs_etm/@20070000.etr/u --per-thread program
333	339
	340	+The syntax within the forward slashes '/' is important. The '@' character
	341	+tells the parser that a sink is about to be specified and that this is the sink
	342	+to use for the trace session.
334	343
335		−Using perf tools
336		−----------------
	344	+More information on the above and other example on how to use Coresight with
	345	+the perf tools can be found in the "HOWTO.md" file of the openCSD gitHub
	346	+repository [3].
	347	+
	348	+2.1) AutoFDO analysis using the perf tools:
337	349
338	350	perf can be used to record and analyze trace of programs.
339	351
@@ -381,3 +393,38 @@ sort example is from the AutoFDO tutorial (https://gcc.gnu.org/wiki/AutoFDO/Tuto · 41 unmodified lines
381	393	$ taskset -c 2 ./sort_autofdo
382	394	Bubble sorting array of 30000 elements
383	395	5806 ms
	396	+
	397	+
	398	+How to use the STM module
	399	+-------------------------
	400	+
	401	+Using the System Trace Macrocell module is the same as the tracers - the only
	402	+difference is that clients are driving the trace capture rather
	403	+than the program flow through the code.
	404	+
	405	+As with any other CoreSight component, specifics about the STM tracer can be
	406	+found in sysfs with more information on each entry being found in [1]:
	407	+
	408	+root@genericarmv8:~# ls /sys/bus/coresight/devices/20100000.stm
	409	+enable_source hwevent_select port_enable subsystem uevent
	410	+hwevent_enable mgmt port_select traceid
	411	+root@genericarmv8:~#
	412	+
	413	+Like any other source a sink needs to be identified and the STM enabled before
	414	+being used:
	415	+
	416	+root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/20010000.etf/enable_sink
	417	+root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/20100000.stm/enable_source
	418	+
	419	+From there user space applications can request and use channels using the devfs
	420	+interface provided for that purpose by the generic STM API:
	421	+
	422	+root@genericarmv8:~# ls -l /dev/20100000.stm
	423	+crw------- 1 root root 10, 61 Jan 3 18:11 /dev/20100000.stm
	424	+root@genericarmv8:~#
	425	+
	426	+Details on how to use the generic STM API can be found here [2].
	427	+
	428	+[1]. Documentation/ABI/testing/sysfs-bus-coresight-devices-stm
	429	+[2]. Documentation/trace/stm.txt
	430	+[3]. https://github.com/Linaro/perf-opencsd

Documentation/trace/ftrace-uses.rst+2−2

@@ -12,7 +12,7 @@ Written for: 4.14 · 11 unmodified lines
12	12	Introduction
13	13	============
14	14
15		−The ftrace infrastructure was originially created to attach callbacks to the
	15	+The ftrace infrastructure was originally created to attach callbacks to the
16	16	beginning of functions in order to record and trace the flow of the kernel.
17	17	But callbacks to the start of a function can have other use cases. Either
18	18	for live kernel patching, or for security monitoring. This document describes
@@ -30,7 +30,7 @@ The ftrace context · 11 unmodified lines
30	30	This requires extra care to what can be done inside a callback. A callback
31	31	can be called outside the protective scope of RCU.
32	32
33		−The ftrace infrastructure has some protections agains recursions and RCU
	33	+The ftrace infrastructure has some protections against recursions and RCU
34	34	but one must still be very careful how they use the callbacks.
35	35
36	36

Documentation/trace/ftrace.rst+6−1

@@ -224,6 +224,8 @@ of ftrace. Here is a list of some of the key files: · 223 unmodified lines
224	224	has a side effect of enabling or disabling specific functions
225	225	to be traced. Echoing names of functions into this file
226	226	will limit the trace to only those functions.
	227	+ This influences the tracers "function" and "function_graph"
	228	+ and thus also function profiling (see "function_profile_enabled").
227	229
228	230	The functions listed in "available_filter_functions" are what
229	231	can be written into this file.
@@ -265,6 +267,8 @@ of ftrace. Here is a list of some of the key files: · 35 unmodified lines
265	267	Functions listed in this file will cause the function graph
266	268	tracer to only trace these functions and the functions that
267	269	they call. (See the section "dynamic ftrace" for more details).
	270	+ Note, set_ftrace_filter and set_ftrace_notrace still affects
	271	+ what functions are being traced.
268	272
269	273	set_graph_notrace:
270	274
@@ -277,7 +281,8 @@ of ftrace. Here is a list of some of the key files: · 6 unmodified lines
277	281
278	282	This lists the functions that ftrace has processed and can trace.
279	283	These are the function names that you can pass to
280		− "set_ftrace_filter" or "set_ftrace_notrace".
	284	+ "set_ftrace_filter", "set_ftrace_notrace",
	285	+ "set_graph_function", or "set_graph_notrace".
281	286	(See the section "dynamic ftrace" below for more details.)
282	287
283	288	dyn_ftrace_total_info:

Documentation/translations/ko_KR/memory-barriers.txt+2−2

@@ -2846,7 +2846,7 @@ CPU 의 캐시에서 RAM 으로 쓰여지는 더티 캐시 라인에 의해 덮 · 2845 unmodified lines
2846	2846	문제를 해결하기 위해선, 커널의 적절한 부분에서 각 CPU 의 캐시 안의 문제가 되는
2847	2847	비트들을 무효화 시켜야 합니다.
2848	2848
2849		−캐시 관리에 대한 더 많은 정보를 위해선 Documentation/cachetlb.txt 를
	2849	+캐시 관리에 대한 더 많은 정보를 위해선 Documentation/core-api/cachetlb.rst 를
2850	2850	참고하세요.
2851	2851
2852	2852
@@ -3023,7 +3023,7 @@ smp_mb() 가 아니라 virt_mb() 를 사용해야 합니다. · 170 unmodified lines
3023	3023	동기화에 락을 사용하지 않고 구현하는데에 사용될 수 있습니다. 더 자세한 내용을
3024	3024	위해선 다음을 참고하세요:
3025	3025
3026		− Documentation/circular-buffers.txt
	3026	+ Documentation/core-api/circular-buffers.rst
3027	3027
3028	3028
3029	3029	=========

Documentation/vfio.txt+2−3

@@ -252,15 +252,14 @@ into VFIO core. When devices are bound and unbound to the driver, · 251 unmodified lines
252	252	the driver should call vfio_add_group_dev() and vfio_del_group_dev()
253	253	respectively::
254	254
255		− extern int vfio_add_group_dev(struct iommu_group *iommu_group,
256		− struct device *dev,
	255	+ extern int vfio_add_group_dev(struct device *dev,
257	256	const struct vfio_device_ops *ops,
258	257	void *device_data);
259	258
260	259	extern void vfio_del_group_dev(struct device dev);
261	260
262	261	vfio_add_group_dev() indicates to the core to begin tracking the
263		−specified iommu_group and register the specified dev as owned by
	262	+iommu_group of the specified dev and register the dev as owned by
264	263	a VFIO bus driver. The driver provides an ops structure for callbacks
265	264	similar to a file operations structure::
266	265

Documentation/vm/00-INDEX+23−35

@@ -1,62 +1,50 @@
1	1	00-INDEX
2	2	- this file.
3		−active_mm.txt
	3	+active_mm.rst
4	4	- An explanation from Linus about tsk->active_mm vs tsk->mm.
5		−balance
	5	+balance.rst
6	6	- various information on memory balancing.
7		−cleancache.txt
	7	+cleancache.rst
8	8	- Intro to cleancache and page-granularity victim cache.
9		−frontswap.txt
	9	+frontswap.rst
10	10	- Outline frontswap, part of the transcendent memory frontend.
11		−highmem.txt
	11	+highmem.rst
12	12	- Outline of highmem and common issues.
13		−hmm.txt
	13	+hmm.rst
14	14	- Documentation of heterogeneous memory management
15		−hugetlbpage.txt
16		− - a brief summary of hugetlbpage support in the Linux kernel.
17		−hugetlbfs_reserv.txt
	15	+hugetlbfs_reserv.rst
18	16	- A brief overview of hugetlbfs reservation design/implementation.
19		−hwpoison.txt
	17	+hwpoison.rst
20	18	- explains what hwpoison is
21		−idle_page_tracking.txt
22		− - description of the idle page tracking feature.
23		−ksm.txt
	19	+ksm.rst
24	20	- how to use the Kernel Samepage Merging feature.
25		−mmu_notifier.txt
	21	+mmu_notifier.rst
26	22	- a note about clearing pte/pmd and mmu notifications
27		−numa
	23	+numa.rst
28	24	- information about NUMA specific code in the Linux vm.
29		−numa_memory_policy.txt
30		− - documentation of concepts and APIs of the 2.6 memory policy support.
31		−overcommit-accounting
	25	+overcommit-accounting.rst
32	26	- description of the Linux kernels overcommit handling modes.
33		−page_frags
	27	+page_frags.rst
34	28	- description of page fragments allocator
35		−page_migration
	29	+page_migration.rst
36	30	- description of page migration in NUMA systems.
37		−pagemap.txt
38		− - pagemap, from the userspace perspective
39		−page_owner.txt
	31	+page_owner.rst
40	32	- tracking about who allocated each page
41		−remap_file_pages.txt
	33	+remap_file_pages.rst
42	34	- a note about remap_file_pages() system call
43		−slub.txt
	35	+slub.rst
44	36	- a short users guide for SLUB.
45		−soft-dirty.txt
46		− - short explanation for soft-dirty PTEs
47		−split_page_table_lock
	37	+split_page_table_lock.rst
48	38	- Separate per-table lock to improve scalability of the old page_table_lock.
49		−swap_numa.txt
	39	+swap_numa.rst
50	40	- automatic binding of swap device to numa node
51		−transhuge.txt
	41	+transhuge.rst
52	42	- Transparent Hugepage Support, alternative way of using hugepages.
53		−unevictable-lru.txt
	43	+unevictable-lru.rst
54	44	- Unevictable LRU infrastructure
55		−userfaultfd.txt
56		− - description of userfaultfd system call
57	45	z3fold.txt
58	46	- outline of z3fold allocator for storing compressed pages
59		−zsmalloc.txt
	47	+zsmalloc.rst
60	48	- outline of zsmalloc allocator for storing compressed pages
61		−zswap.txt
	49	+zswap.rst
62	50	- Intro to compressed cache for swap pages

Documentation/vm/active_mm.rst+91−0

@@ -0,0 +1,91 @@
	1	+.. _active_mm:
	2	+
	3	+=========
	4	+Active MM
	5	+=========
	6	+
	7	+::
	8	+
	9	+ List: linux-kernel
	10	+ Subject: Re: active_mm
	11	+ From: Linus Torvalds <torvalds () transmeta ! com>
	12	+ Date: 1999-07-30 21:36:24
	13	+
	14	+ Cc'd to linux-kernel, because I don't write explanations all that often,
	15	+ and when I do I feel better about more people reading them.
	16	+
	17	+ On Fri, 30 Jul 1999, David Mosberger wrote:
	18	+ >
	19	+ > Is there a brief description someplace on how "mm" vs. "active_mm" in
	20	+ > the task_struct are supposed to be used? (My apologies if this was
	21	+ > discussed on the mailing lists---I just returned from vacation and
	22	+ > wasn't able to follow linux-kernel for a while).
	23	+
	24	+ Basically, the new setup is:
	25	+
	26	+ - we have "real address spaces" and "anonymous address spaces". The
	27	+ difference is that an anonymous address space doesn't care about the
	28	+ user-level page tables at all, so when we do a context switch into an
	29	+ anonymous address space we just leave the previous address space
	30	+ active.
	31	+
	32	+ The obvious use for a "anonymous address space" is any thread that
	33	+ doesn't need any user mappings - all kernel threads basically fall into
	34	+ this category, but even "real" threads can temporarily say that for
	35	+ some amount of time they are not going to be interested in user space,
	36	+ and that the scheduler might as well try to avoid wasting time on
	37	+ switching the VM state around. Currently only the old-style bdflush
	38	+ sync does that.
	39	+
	40	+ - "tsk->mm" points to the "real address space". For an anonymous process,
	41	+ tsk->mm will be NULL, for the logical reason that an anonymous process
	42	+ really doesn't _have_ a real address space at all.
	43	+
	44	+ - however, we obviously need to keep track of which address space we
	45	+ "stole" for such an anonymous user. For that, we have "tsk->active_mm",
	46	+ which shows what the currently active address space is.
	47	+
	48	+ The rule is that for a process with a real address space (ie tsk->mm is
	49	+ non-NULL) the active_mm obviously always has to be the same as the real
	50	+ one.
	51	+
	52	+ For a anonymous process, tsk->mm == NULL, and tsk->active_mm is the
	53	+ "borrowed" mm while the anonymous process is running. When the
	54	+ anonymous process gets scheduled away, the borrowed address space is
	55	+ returned and cleared.
	56	+
	57	+ To support all that, the "struct mm_struct" now has two counters: a
	58	+ "mm_users" counter that is how many "real address space users" there are,
	59	+ and a "mm_count" counter that is the number of "lazy" users (ie anonymous
	60	+ users) plus one if there are any real users.
	61	+
	62	+ Usually there is at least one real user, but it could be that the real
	63	+ user exited on another CPU while a lazy user was still active, so you do
	64	+ actually get cases where you have a address space that is _only_ used by
	65	+ lazy users. That is often a short-lived state, because once that thread
	66	+ gets scheduled away in favour of a real thread, the "zombie" mm gets
	67	+ released because "mm_users" becomes zero.
	68	+
	69	+ Also, a new rule is that _nobody_ ever has "init_mm" as a real MM any
	70	+ more. "init_mm" should be considered just a "lazy context when no other
	71	+ context is available", and in fact it is mainly used just at bootup when
	72	+ no real VM has yet been created. So code that used to check
	73	+
	74	+ if (current->mm == &init_mm)
	75	+
	76	+ should generally just do
	77	+
	78	+ if (!current->mm)
	79	+
	80	+ instead (which makes more sense anyway - the test is basically one of "do
	81	+ we have a user context", and is generally done by the page fault handler
	82	+ and things like that).
	83	+
	84	+ Anyway, I put a pre-patch-2.3.13-1 on ftp.kernel.org just a moment ago,
	85	+ because it slightly changes the interfaces to accommodate the alpha (who
	86	+ would have thought it, but the alpha actually ends up having one of the
	87	+ ugliest context switch codes - unlike the other architectures where the MM
	88	+ and register state is separate, the alpha PALcode joins the two, and you
	89	+ need to switch both together).
	90	+
	91	+ (From http://marc.info/?l=linux-kernel&m=93337278602211&w=2)

Documentation/vm/active_mm.txt+0−83

@@ -1,83 +0,0 @@
1		−List: linux-kernel
2		−Subject: Re: active_mm
3		−From: Linus Torvalds <torvalds () transmeta ! com>
4		−Date: 1999-07-30 21:36:24
5		−
6		−Cc'd to linux-kernel, because I don't write explanations all that often,
7		−and when I do I feel better about more people reading them.
8		−
9		−On Fri, 30 Jul 1999, David Mosberger wrote:
10		−>
11		−> Is there a brief description someplace on how "mm" vs. "active_mm" in
12		−> the task_struct are supposed to be used? (My apologies if this was
13		−> discussed on the mailing lists---I just returned from vacation and
14		−> wasn't able to follow linux-kernel for a while).
15		−
16		−Basically, the new setup is:
17		−
18		− - we have "real address spaces" and "anonymous address spaces". The
19		− difference is that an anonymous address space doesn't care about the
20		− user-level page tables at all, so when we do a context switch into an
21		− anonymous address space we just leave the previous address space
22		− active.
23		−
24		− The obvious use for a "anonymous address space" is any thread that
25		− doesn't need any user mappings - all kernel threads basically fall into
26		− this category, but even "real" threads can temporarily say that for
27		− some amount of time they are not going to be interested in user space,
28		− and that the scheduler might as well try to avoid wasting time on
29		− switching the VM state around. Currently only the old-style bdflush
30		− sync does that.
31		−
32		− - "tsk->mm" points to the "real address space". For an anonymous process,
33		− tsk->mm will be NULL, for the logical reason that an anonymous process
34		− really doesn't _have_ a real address space at all.
35		−
36		− - however, we obviously need to keep track of which address space we
37		− "stole" for such an anonymous user. For that, we have "tsk->active_mm",
38		− which shows what the currently active address space is.
39		−
40		− The rule is that for a process with a real address space (ie tsk->mm is
41		− non-NULL) the active_mm obviously always has to be the same as the real
42		− one.
43		−
44		− For a anonymous process, tsk->mm == NULL, and tsk->active_mm is the
45		− "borrowed" mm while the anonymous process is running. When the
46		− anonymous process gets scheduled away, the borrowed address space is
47		− returned and cleared.
48		−
49		−To support all that, the "struct mm_struct" now has two counters: a
50		−"mm_users" counter that is how many "real address space users" there are,
51		−and a "mm_count" counter that is the number of "lazy" users (ie anonymous
52		−users) plus one if there are any real users.
53		−
54		−Usually there is at least one real user, but it could be that the real
55		−user exited on another CPU while a lazy user was still active, so you do
56		−actually get cases where you have a address space that is _only_ used by
57		−lazy users. That is often a short-lived state, because once that thread
58		−gets scheduled away in favour of a real thread, the "zombie" mm gets
59		−released because "mm_users" becomes zero.
60		−
61		−Also, a new rule is that _nobody_ ever has "init_mm" as a real MM any
62		−more. "init_mm" should be considered just a "lazy context when no other
63		−context is available", and in fact it is mainly used just at bootup when
64		−no real VM has yet been created. So code that used to check
65		−
66		− if (current->mm == &init_mm)
67		−
68		−should generally just do
69		−
70		− if (!current->mm)
71		−
72		−instead (which makes more sense anyway - the test is basically one of "do
73		−we have a user context", and is generally done by the page fault handler
74		−and things like that).
75		−
76		−Anyway, I put a pre-patch-2.3.13-1 on ftp.kernel.org just a moment ago,
77		−because it slightly changes the interfaces to accommodate the alpha (who
78		−would have thought it, but the alpha actually ends up having one of the
79		−ugliest context switch codes - unlike the other architectures where the MM
80		−and register state is separate, the alpha PALcode joins the two, and you
81		−need to switch both together).
82		−
83		−(From http://marc.info/?l=linux-kernel&m=93337278602211&w=2)

Documentation/vm/balance.rstrenamed from Documentation/vm/balance+11−4

@@ -1,3 +1,9 @@
	1	+.. _balance:
	2	+
	3	+================
	4	+Memory Balancing
	5	+================
	6	+
1	7	Started Jan 2000 by Kanoj Sarcar <kanoj@sgi.com>
2	8
3	9	Memory balancing is needed for !__GFP_ATOMIC and !__GFP_KSWAPD_RECLAIM as
@@ -62,11 +68,11 @@ for non-sleepable allocations. Second, the HIGHMEM zone is also balanced, · 58 unmodified lines
62	68	so as to give a fighting chance for replace_with_highmem() to get a
63	69	HIGHMEM page, as well as to ensure that HIGHMEM allocations do not
64	70	fall back into regular zone. This also makes sure that HIGHMEM pages
65		−are not leaked (for example, in situations where a HIGHMEM page is in
	71	+are not leaked (for example, in situations where a HIGHMEM page is in
66	72	the swapcache but is not being used by anyone)
67	73
68	74	kswapd also needs to know about the zones it should balance. kswapd is
69		−primarily needed in a situation where balancing can not be done,
	75	+primarily needed in a situation where balancing can not be done,
70	76	probably because all allocation requests are coming from intr context
71	77	and all process contexts are sleeping. For 2.3, kswapd does not really
72	78	need to balance the highmem zone, since intr context does not request
@@ -89,7 +95,8 @@ pages is below watermark[WMARK_LOW]; in which case zone_wake_kswapd is also set. · 16 unmodified lines
89	95
90	96
91	97	(Good) Ideas that I have heard:
	98	+
92	99	1. Dynamic experience should influence balancing: number of failed requests
93		−for a zone can be tracked and fed into the balancing scheme (jalvo@mbay.net)
	100	+ for a zone can be tracked and fed into the balancing scheme (jalvo@mbay.net)
94	101	2. Implement a replace_with_highmem()-like replace_with_regular() to preserve
95		−dma pages. (lkd@tantalophile.demon.co.uk)
	102	+ dma pages. (lkd@tantalophile.demon.co.uk)

Documentation/vm/cleancache.rstrenamed from Documentation/vm/cleancache.txt+62−43

@@ -1,4 +1,11 @@
1		−MOTIVATION
	1	+.. _cleancache:
	2	+
	3	+==========
	4	+Cleancache
	5	+==========
	6	+
	7	+Motivation
	8	+==========
2	9
3	10	Cleancache is a new optional feature provided by the VFS layer that
4	11	potentially dramatically increases page cache effectiveness for
@@ -21,9 +28,10 @@ Transcendent memory "drivers" for cleancache are currently implemented · 16 unmodified lines
21	28	in Xen (using hypervisor memory) and zcache (using in-kernel compressed
22	29	memory) and other implementations are in development.
23	30
24		−FAQs are included below.
	31	+:ref:`FAQs <faq>` are included below.
25	32
26		−IMPLEMENTATION OVERVIEW
	33	+Implementation Overview
	34	+=======================
27	35
28	36	A cleancache "backend" that provides transcendent memory registers itself
29	37	to the kernel's cleancache "frontend" by calling cleancache_register_ops,
@@ -80,22 +88,33 @@ different Linux threads are simultaneously putting and invalidating a page · 50 unmodified lines
80	88	with the same handle, the results are indeterminate. Callers must
81	89	lock the page to ensure serial behavior.
82	90
83		−CLEANCACHE PERFORMANCE METRICS
	91	+Cleancache Performance Metrics
	92	+==============================
84	93
85	94	If properly configured, monitoring of cleancache is done via debugfs in
86		−the /sys/kernel/debug/cleancache directory. The effectiveness of cleancache
	95	+the `/sys/kernel/debug/cleancache` directory. The effectiveness of cleancache
87	96	can be measured (across all filesystems) with:
88	97
89		−succ_gets - number of gets that were successful
90		−failed_gets - number of gets that failed
91		−puts - number of puts attempted (all "succeed")
92		−invalidates - number of invalidates attempted
	98	+``succ_gets``
	99	+ number of gets that were successful
	100	+
	101	+``failed_gets``
	102	+ number of gets that failed
	103	+
	104	+``puts``
	105	+ number of puts attempted (all "succeed")
	106	+
	107	+``invalidates``
	108	+ number of invalidates attempted
93	109
94	110	A backend implementation may provide additional metrics.
95	111
	112	+.. _faq:
	113	+
96	114	FAQ
	115	+===
97	116
98		−1) Where's the value? (Andrew Morton)
	117	+* Where's the value? (Andrew Morton)
99	118
100	119	Cleancache provides a significant performance benefit to many workloads
101	120	in many environments with negligible overhead by improving the
@@ -137,8 +156,8 @@ device that stores pages of data in a compressed state. And · 35 unmodified lines
137	156	the proposed "RAMster" driver shares RAM across multiple physical
138	157	systems.
139	158
140		−2) Why does cleancache have its sticky fingers so deep inside the
141		− filesystems and VFS? (Andrew Morton and Christoph Hellwig)
	159	+* Why does cleancache have its sticky fingers so deep inside the
	160	+ filesystems and VFS? (Andrew Morton and Christoph Hellwig)
142	161
143	162	The core hooks for cleancache in VFS are in most cases a single line
144	163	and the minimum set are placed precisely where needed to maintain
@@ -168,9 +187,9 @@ filesystems in the future. · 23 unmodified lines
168	187	The total impact of the hooks to existing fs and mm files is only
169	188	about 40 lines added (not counting comments and blank lines).
170	189
171		−3) Why not make cleancache asynchronous and batched so it can
172		− more easily interface with real devices with DMA instead
173		− of copying each individual page? (Minchan Kim)
	190	+* Why not make cleancache asynchronous and batched so it can more
	191	+ easily interface with real devices with DMA instead of copying each
	192	+ individual page? (Minchan Kim)
174	193
175	194	The one-page-at-a-time copy semantics simplifies the implementation
176	195	on both the frontend and backend and also allows the backend to
@@ -182,8 +201,8 @@ are avoided. While the interface seems odd for a "real device" · 5 unmodified lines
182	201	or for real kernel-addressable RAM, it makes perfect sense for
183	202	transcendent memory.
184	203
185		−4) Why is non-shared cleancache "exclusive"? And where is the
186		− page "invalidated" after a "get"? (Minchan Kim)
	204	+* Why is non-shared cleancache "exclusive"? And where is the
	205	+ page "invalidated" after a "get"? (Minchan Kim)
187	206
188	207	The main reason is to free up space in transcendent memory and
189	208	to avoid unnecessary cleancache_invalidate calls. If you want inclusive,
@@ -193,7 +212,7 @@ be easily extended to add a "get_no_invalidate" call. · 3 unmodified lines
193	212
194	213	The invalidate is done by the cleancache backend implementation.
195	214
196		−5) What's the performance impact?
	215	+* What's the performance impact?
197	216
198	217	Performance analysis has been presented at OLS'09 and LCA'10.
199	218	Briefly, performance gains can be significant on most workloads,
@@ -206,7 +225,7 @@ single-core systems with slow memory-copy speeds, cleancache · 6 unmodified lines
206	225	has little value, but in newer multicore machines, especially
207	226	consolidated/virtualized machines, it has great value.
208	227
209		−6) How do I add cleancache support for filesystem X? (Boaz Harrash)
	228	+* How do I add cleancache support for filesystem X? (Boaz Harrash)
210	229
211	230	Filesystems that are well-behaved and conform to certain
212	231	restrictions can utilize cleancache simply by making a call to
@@ -217,26 +236,26 @@ not enable the optional cleancache. · 4 unmodified lines
217	236
218	237	Some points for a filesystem to consider:
219	238
220		−- The FS should be block-device-based (e.g. a ram-based FS such
221		− as tmpfs should not enable cleancache)
222		−- To ensure coherency/correctness, the FS must ensure that all
223		− file removal or truncation operations either go through VFS or
224		− add hooks to do the equivalent cleancache "invalidate" operations
225		−- To ensure coherency/correctness, either inode numbers must
226		− be unique across the lifetime of the on-disk file OR the
227		− FS must provide an "encode_fh" function.
228		−- The FS must call the VFS superblock alloc and deactivate routines
229		− or add hooks to do the equivalent cleancache calls done there.
230		−- To maximize performance, all pages fetched from the FS should
231		− go through the do_mpag_readpage routine or the FS should add
232		− hooks to do the equivalent (cf. btrfs)
233		−- Currently, the FS blocksize must be the same as PAGESIZE. This
234		− is not an architectural restriction, but no backends currently
235		− support anything different.
236		−- A clustered FS should invoke the "shared_init_fs" cleancache
237		− hook to get best performance for some backends.
238		−
239		−7) Why not use the KVA of the inode as the key? (Christoph Hellwig)
	239	+ - The FS should be block-device-based (e.g. a ram-based FS such
	240	+ as tmpfs should not enable cleancache)
	241	+ - To ensure coherency/correctness, the FS must ensure that all
	242	+ file removal or truncation operations either go through VFS or
	243	+ add hooks to do the equivalent cleancache "invalidate" operations
	244	+ - To ensure coherency/correctness, either inode numbers must
	245	+ be unique across the lifetime of the on-disk file OR the
	246	+ FS must provide an "encode_fh" function.
	247	+ - The FS must call the VFS superblock alloc and deactivate routines
	248	+ or add hooks to do the equivalent cleancache calls done there.
	249	+ - To maximize performance, all pages fetched from the FS should
	250	+ go through the do_mpag_readpage routine or the FS should add
	251	+ hooks to do the equivalent (cf. btrfs)
	252	+ - Currently, the FS blocksize must be the same as PAGESIZE. This
	253	+ is not an architectural restriction, but no backends currently
	254	+ support anything different.
	255	+ - A clustered FS should invoke the "shared_init_fs" cleancache
	256	+ hook to get best performance for some backends.
	257	+
	258	+* Why not use the KVA of the inode as the key? (Christoph Hellwig)
240	259
241	260	If cleancache would use the inode virtual address instead of
242	261	inode/filehandle, the pool id could be eliminated. But, this
@@ -251,7 +270,7 @@ of cleancache would be lost because the cache of pages in cleanache · 8 unmodified lines
251	270	is potentially much larger than the kernel pagecache and is most
252	271	useful if the pages survive inode cache removal.
253	272
254		−8) Why is a global variable required?
	273	+* Why is a global variable required?
255	274
256	275	The cleancache_enabled flag is checked in all of the frequently-used
257	276	cleancache hooks. The alternative is a function call to check a static
@@ -262,14 +281,14 @@ global variable allows cleancache to be enabled by default at compile · 4 unmodified lines
262	281	time, but have insignificant performance impact when cleancache remains
263	282	disabled at runtime.
264	283
265		−9) Does cleanache work with KVM?
	284	+* Does cleanache work with KVM?
266	285
267	286	The memory model of KVM is sufficiently different that a cleancache
268	287	backend may have less value for KVM. This remains to be tested,
269	288	especially in an overcommitted system.
270	289
271		−10) Does cleancache work in userspace? It sounds useful for
272		− memory hungry caches like web browsers. (Jamie Lokier)
	290	+* Does cleancache work in userspace? It sounds useful for
	291	+ memory hungry caches like web browsers. (Jamie Lokier)
273	292
274	293	No plans yet, though we agree it sounds useful, at least for
275	294	apps that bypass the page cache (e.g. O_DIRECT).

Documentation/vm/conf.py+10−0

@@ -0,0 +1,10 @@
	1	+# -- coding: utf-8; mode: python --
	2	+
	3	+project = "Linux Memory Management Documentation"
	4	+
	5	+tags.add("subproject")
	6	+
	7	+latex_documents = [
	8	+ ('index', 'memory-management.tex', project,
	9	+ 'The kernel development community', 'manual'),
	10	+]

Documentation/vm/frontswap.rstrenamed from Documentation/vm/frontswap.txt+37−22

@@ -1,13 +1,20 @@
	1	+.. _frontswap:
	2	+
	3	+=========
	4	+Frontswap
	5	+=========
	6	+
1	7	Frontswap provides a "transcendent memory" interface for swap pages.
2	8	In some environments, dramatic performance savings may be obtained because
3	9	swapped pages are saved in RAM (or a RAM-like device) instead of a swap disk.
4	10
5		−(Note, frontswap -- and cleancache (merged at 3.0) -- are the "frontends"
	11	+(Note, frontswap -- and :ref:`cleancache` (merged at 3.0) -- are the "frontends"
6	12	and the only necessary changes to the core kernel for transcendent memory;
7	13	all other supporting code -- the "backends" -- is implemented as drivers.
8		−See the LWN.net article "Transcendent memory in a nutshell" for a detailed
9		−overview of frontswap and related kernel parts:
10		−https://lwn.net/Articles/454795/ )
	14	+See the LWN.net article `Transcendent memory in a nutshell`_
	15	+for a detailed overview of frontswap and related kernel parts)
	16	+
	17	+.. _Transcendent memory in a nutshell: https://lwn.net/Articles/454795/
11	18
12	19	Frontswap is so named because it can be thought of as the opposite of
13	20	a "backing" store for a swap device. The storage is assumed to be
@@ -50,19 +57,27 @@ or the store fails AND the page is invalidated. This ensures stale data may · 36 unmodified lines
50	57	never be obtained from frontswap.
51	58
52	59	If properly configured, monitoring of frontswap is done via debugfs in
53		−the /sys/kernel/debug/frontswap directory. The effectiveness of
	60	+the `/sys/kernel/debug/frontswap` directory. The effectiveness of
54	61	frontswap can be measured (across all swap devices) with:
55	62
56		−failed_stores - how many store attempts have failed
57		−loads - how many loads were attempted (all should succeed)
58		−succ_stores - how many store attempts have succeeded
59		−invalidates - how many invalidates were attempted
	63	+``failed_stores``
	64	+ how many store attempts have failed
	65	+
	66	+``loads``
	67	+ how many loads were attempted (all should succeed)
	68	+
	69	+``succ_stores``
	70	+ how many store attempts have succeeded
	71	+
	72	+``invalidates``
	73	+ how many invalidates were attempted
60	74
61	75	A backend implementation may provide additional metrics.
62	76
63	77	FAQ
	78	+===
64	79
65		−1) Where's the value?
	80	+* Where's the value?
66	81
67	82	When a workload starts swapping, performance falls through the floor.
68	83	Frontswap significantly increases performance in many such workloads by
@@ -117,8 +132,8 @@ A KVM implementation is underway and has been RFC'ed to lkml. And, · 48 unmodified lines
117	132	using frontswap, investigation is also underway on the use of NVM as
118	133	a memory extension technology.
119	134
120		−2) Sure there may be performance advantages in some situations, but
121		− what's the space/time overhead of frontswap?
	135	+* Sure there may be performance advantages in some situations, but
	136	+ what's the space/time overhead of frontswap?
122	137
123	138	If CONFIG_FRONTSWAP is disabled, every frontswap hook compiles into
124	139	nothingness and the only overhead is a few extra bytes per swapon'ed
@@ -148,8 +163,8 @@ pressure that can potentially outweigh the other advantages. A · 23 unmodified lines
148	163	backend, such as zcache, must implement policies to carefully (but
149	164	dynamically) manage memory limits to ensure this doesn't happen.
150	165
151		−3) OK, how about a quick overview of what this frontswap patch does
152		− in terms that a kernel hacker can grok?
	166	+* OK, how about a quick overview of what this frontswap patch does
	167	+ in terms that a kernel hacker can grok?
153	168
154	169	Let's assume that a frontswap "backend" has registered during
155	170	kernel initialization; this registration indicates that this
@@ -188,9 +203,9 @@ and (potentially) a swap device write are replaced by a "frontswap backend · 32 unmodified lines
188	203	store" and (possibly) a "frontswap backend loads", which are presumably much
189	204	faster.
190	205
191		−4) Can't frontswap be configured as a "special" swap device that is
192		− just higher priority than any real swap device (e.g. like zswap,
193		− or maybe swap-over-nbd/NFS)?
	206	+* Can't frontswap be configured as a "special" swap device that is
	207	+ just higher priority than any real swap device (e.g. like zswap,
	208	+ or maybe swap-over-nbd/NFS)?
194	209
195	210	No. First, the existing swap subsystem doesn't allow for any kind of
196	211	swap hierarchy. Perhaps it could be rewritten to accommodate a hierarchy,
@@ -240,9 +255,9 @@ installation, frontswap is useless. Swapless portable devices · 43 unmodified lines
240	255	can still use frontswap but a backend for such devices must configure
241	256	some kind of "ghost" swap device and ensure that it is never used.
242	257
243		−5) Why this weird definition about "duplicate stores"? If a page
244		− has been previously successfully stored, can't it always be
245		− successfully overwritten?
	258	+* Why this weird definition about "duplicate stores"? If a page
	259	+ has been previously successfully stored, can't it always be
	260	+ successfully overwritten?
246	261
247	262	Nearly always it can, but no, sometimes it cannot. Consider an example
248	263	where data is compressed and the original 4K page has been compressed
@@ -254,7 +269,7 @@ the old data and ensure that it is no longer accessible. Since the · 5 unmodified lines
254	269	swap subsystem then writes the new data to the read swap device,
255	270	this is the correct course of action to ensure coherency.
256	271
257		−6) What is frontswap_shrink for?
	272	+* What is frontswap_shrink for?
258	273
259	274	When the (non-frontswap) swap subsystem swaps out a page to a real
260	275	swap device, that page is only taking up low-value pre-allocated disk
@@ -267,7 +282,7 @@ to "repatriate" pages sent to a remote machine back to the local machine; · 6 unmodified lines
267	282	this is driven using the frontswap_shrink mechanism when memory pressure
268	283	subsides.
269	284
270		−7) Why does the frontswap patch create the new include file swapfile.h?
	285	+* Why does the frontswap patch create the new include file swapfile.h?
271	286
272	287	The frontswap code depends on some swap-subsystem-internal data
273	288	structures that have, over the years, moved back and forth between

Documentation/vm/highmem.rstrenamed from Documentation/vm/highmem.txt+36−51

@@ -1,25 +1,14 @@
	1	+.. _highmem:
1	2
2		− ====================
3		− HIGH MEMORY HANDLING
4		− ====================
	3	+====================
	4	+High Memory Handling
	5	+====================
5	6
6	7	By: Peter Zijlstra <a.p.zijlstra@chello.nl>
7	8
8		−Contents:
9		−
10		− (*) What is high memory?
11		−
12		− (*) Temporary virtual mappings.
13		−
14		− (*) Using kmap_atomic.
15		−
16		− (*) Cost of temporary mappings.
17		−
18		− (*) i386 PAE.
	9	+.. contents:: :local:
19	10
20		−
21		−====================
22		−WHAT IS HIGH MEMORY?
	11	+What Is High Memory?
23	12	====================
24	13
25	14	High memory (highmem) is used when the size of physical memory approaches or
@@ -38,7 +27,7 @@ kernel entry/exit. This means the available virtual memory space (4GiB on · 12 unmodified lines
38	27	i386) has to be divided between user and kernel space.
39	28
40	29	The traditional split for architectures using this approach is 3:1, 3GiB for
41		−userspace and the top 1GiB for kernel space:
	30	+userspace and the top 1GiB for kernel space::
42	31
43	32	+--------+ 0xffffffff
44	33	\| Kernel \|
@@ -58,40 +47,38 @@ and user maps. Some hardware (like some ARMs), however, have limited virtual · 13 unmodified lines
58	47	space when they use mm context tags.
59	48
60	49
61		−==========================
62		−TEMPORARY VIRTUAL MAPPINGS
	50	+Temporary Virtual Mappings
63	51	==========================
64	52
65	53	The kernel contains several ways of creating temporary mappings:
66	54
67		− (*) vmap(). This can be used to make a long duration mapping of multiple
68		− physical pages into a contiguous virtual space. It needs global
69		− synchronization to unmap.
	55	+* vmap(). This can be used to make a long duration mapping of multiple
	56	+ physical pages into a contiguous virtual space. It needs global
	57	+ synchronization to unmap.
70	58
71		− (*) kmap(). This permits a short duration mapping of a single page. It needs
72		− global synchronization, but is amortized somewhat. It is also prone to
73		− deadlocks when using in a nested fashion, and so it is not recommended for
74		− new code.
	59	+* kmap(). This permits a short duration mapping of a single page. It needs
	60	+ global synchronization, but is amortized somewhat. It is also prone to
	61	+ deadlocks when using in a nested fashion, and so it is not recommended for
	62	+ new code.
75	63
76		− (*) kmap_atomic(). This permits a very short duration mapping of a single
77		− page. Since the mapping is restricted to the CPU that issued it, it
78		− performs well, but the issuing task is therefore required to stay on that
79		− CPU until it has finished, lest some other task displace its mappings.
	64	+* kmap_atomic(). This permits a very short duration mapping of a single
	65	+ page. Since the mapping is restricted to the CPU that issued it, it
	66	+ performs well, but the issuing task is therefore required to stay on that
	67	+ CPU until it has finished, lest some other task displace its mappings.
80	68
81		− kmap_atomic() may also be used by interrupt contexts, since it is does not
82		− sleep and the caller may not sleep until after kunmap_atomic() is called.
	69	+ kmap_atomic() may also be used by interrupt contexts, since it is does not
	70	+ sleep and the caller may not sleep until after kunmap_atomic() is called.
83	71
84		− It may be assumed that k[un]map_atomic() won't fail.
	72	+ It may be assumed that k[un]map_atomic() won't fail.
85	73
86	74
87		−=================
88		−USING KMAP_ATOMIC
	75	+Using kmap_atomic
89	76	=================
90	77
91	78	When and where to use kmap_atomic() is straightforward. It is used when code
92	79	wants to access the contents of a page that might be allocated from high memory
93	80	(see __GFP_HIGHMEM), for example a page in the pagecache. The API has two
94		−functions, and they can be used in a manner similar to the following:
	81	+functions, and they can be used in a manner similar to the following::
95	82
96	83	/* Find the page of interest. */
97	84	struct page *page = find_get_page(mapping, offset);
@@ -109,7 +96,7 @@ Note that the kunmap_atomic() call takes the result of the kmap_atomic() call · 11 unmodified lines
109	96	not the argument.
110	97
111	98	If you need to map two pages because you want to copy from one page to
112		−another you need to keep the kmap_atomic calls strictly nested, like:
	99	+another you need to keep the kmap_atomic calls strictly nested, like::
113	100
114	101	vaddr1 = kmap_atomic(page1);
115	102	vaddr2 = kmap_atomic(page2);
@@ -120,8 +107,7 @@ another you need to keep the kmap_atomic calls strictly nested, like: · 4 unmodified lines
120	107	kunmap_atomic(vaddr1);
121	108
122	109
123		−==========================
124		−COST OF TEMPORARY MAPPINGS
	110	+Cost of Temporary Mappings
125	111	==========================
126	112
127	113	The cost of creating temporary mappings can be quite high. The arch has to
@@ -136,25 +122,24 @@ If CONFIG_MMU is not set, then there can be no temporary mappings and no · 8 unmodified lines
136	122	highmem. In such a case, the arithmetic approach will also be used.
137	123
138	124
139		−========
140	125	i386 PAE
141	126	========
142	127
143	128	The i386 arch, under some circumstances, will permit you to stick up to 64GiB
144	129	of RAM into your 32-bit machine. This has a number of consequences:
145	130
146		− (*) Linux needs a page-frame structure for each page in the system and the
147		− pageframes need to live in the permanent mapping, which means:
	131	+* Linux needs a page-frame structure for each page in the system and the
	132	+ pageframes need to live in the permanent mapping, which means:
148	133
149		− (*) you can have 896M/sizeof(struct page) page-frames at most; with struct
150		− page being 32-bytes that would end up being something in the order of 112G
151		− worth of pages; the kernel, however, needs to store more than just
152		− page-frames in that memory...
	134	+* you can have 896M/sizeof(struct page) page-frames at most; with struct
	135	+ page being 32-bytes that would end up being something in the order of 112G
	136	+ worth of pages; the kernel, however, needs to store more than just
	137	+ page-frames in that memory...
153	138
154		− (*) PAE makes your page tables larger - which slows the system down as more
155		− data has to be accessed to traverse in TLB fills and the like. One
156		− advantage is that PAE has more PTE bits and can provide advanced features
157		− like NX and PAT.
	139	+* PAE makes your page tables larger - which slows the system down as more
	140	+ data has to be accessed to traverse in TLB fills and the like. One
	141	+ advantage is that PAE has more PTE bits and can provide advanced features
	142	+ like NX and PAT.
158	143
159	144	The general recommendation is that you don't use more than 8GiB on a 32-bit
160	145	machine - although more might work for you and your workload, you're pretty

Documentation/vm/hmm.rstrenamed from Documentation/vm/hmm.txt+34−44

@@ -1,4 +1,8 @@
	1	+.. hmm:
	2	+
	3	+=====================================
1	4	Heterogeneous Memory Management (HMM)
	5	+=====================================
2	6
3	7	Provide infrastructure and helpers to integrate non-conventional memory (device
4	8	memory like GPU on board memory) into regular kernel path, with the cornerstone
@@ -6,10 +10,10 @@ of this being specialized struct page for such memory (see sections 5 to 7 of · 1 unmodified line
6	10	this document).
7	11
8	12	HMM also provides optional helpers for SVM (Share Virtual Memory), i.e.,
9		−allowing a device to transparently access program address coherently with the
10		−CPU meaning that any valid pointer on the CPU is also a valid pointer for the
11		−device. This is becoming mandatory to simplify the use of advanced hetero-
12		−geneous computing where GPU, DSP, or FPGA are used to perform various
	13	+allowing a device to transparently access program address coherently with
	14	+the CPU meaning that any valid pointer on the CPU is also a valid pointer
	15	+for the device. This is becoming mandatory to simplify the use of advanced
	16	+heterogeneous computing where GPU, DSP, or FPGA are used to perform various
13	17	computations on behalf of a process.
14	18
15	19	This document is divided as follows: in the first section I expose the problems
@@ -21,19 +25,10 @@ fifth section deals with how device memory is represented inside the kernel. · 5 unmodified lines
21	25	Finally, the last section presents a new migration helper that allows lever-
22	26	aging the device DMA engine.
23	27
	28	+.. contents:: :local:
24	29
25		−1) Problems of using a device specific memory allocator:
26		−2) I/O bus, device memory characteristics
27		−3) Shared address space and migration
28		−4) Address space mirroring implementation and API
29		−5) Represent and manage device memory from core kernel point of view
30		−6) Migration to and from device memory
31		−7) Memory cgroup (memcg) and rss accounting
32		−
33		−
34		−-------------------------------------------------------------------------------
35		−
36		−1) Problems of using a device specific memory allocator:
	30	+Problems of using a device specific memory allocator
	31	+====================================================
37	32
38	33	Devices with a large amount of on board memory (several gigabytes) like GPUs
39	34	have historically managed their memory through dedicated driver specific APIs.
@@ -77,9 +72,8 @@ are only do-able with a shared address space. It is also more reasonable to use · 37 unmodified lines
77	72	a shared address space for all other patterns.
78	73
79	74
80		−-------------------------------------------------------------------------------
81		−
82		−2) I/O bus, device memory characteristics
	75	+I/O bus, device memory characteristics
	76	+======================================
83	77
84	78	I/O buses cripple shared address spaces due to a few limitations. Most I/O
85	79	buses only allow basic memory access from device to main memory; even cache
@@ -109,9 +103,8 @@ access any memory but we must also permit any memory to be migrated to device · 23 unmodified lines
109	103	memory while device is using it (blocking CPU access while it happens).
110	104
111	105
112		−-------------------------------------------------------------------------------
113		−
114		−3) Shared address space and migration
	106	+Shared address space and migration
	107	+==================================
115	108
116	109	HMM intends to provide two main features. First one is to share the address
117	110	space by duplicating the CPU page table in the device page table so the same
@@ -148,23 +141,23 @@ ages device memory by migrating the part of the data set that is actively being · 30 unmodified lines
148	141	used by the device.
149	142
150	143
151		−-------------------------------------------------------------------------------
152		−
153		−4) Address space mirroring implementation and API
	144	+Address space mirroring implementation and API
	145	+==============================================
154	146
155	147	Address space mirroring's main objective is to allow duplication of a range of
156	148	CPU page table into a device page table; HMM helps keep both synchronized. A
157	149	device driver that wants to mirror a process address space must start with the
158		−registration of an hmm_mirror struct:
	150	+registration of an hmm_mirror struct::
159	151
160	152	int hmm_mirror_register(struct hmm_mirror *mirror,
161	153	struct mm_struct *mm);
162	154	int hmm_mirror_register_locked(struct hmm_mirror *mirror,
163	155	struct mm_struct *mm);
164	156
	157	+
165	158	The locked variant is to be used when the driver is already holding mmap_sem
166	159	of the mm in write mode. The mirror struct has a set of callbacks that are used
167		−to propagate CPU page tables:
	160	+to propagate CPU page tables::
168	161
169	162	struct hmm_mirror_ops {
170	163	/* sync_cpu_device_pagetables() - synchronize page tables
@@ -193,10 +186,10 @@ The device driver must perform the update action to the range (mark range · 22 unmodified lines
193	186	read only, or fully unmap, ...). The device must be done with the update before
194	187	the driver callback returns.
195	188
196		−
197	189	When the device driver wants to populate a range of virtual addresses, it can
198		−use either:
199		− int hmm_vma_get_pfns(struct vm_area_struct *vma,
	190	+use either::
	191	+
	192	+ int hmm_vma_get_pfns(struct vm_area_struct *vma,
200	193	struct hmm_range *range,
201	194	unsigned long start,
202	195	unsigned long end,
@@ -221,7 +214,7 @@ provides a set of flags to help the driver identify special CPU page table · 18 unmodified lines
221	214	entries.
222	215
223	216	Locking with the update() callback is the most important aspect the driver must
224		−respect in order to keep things properly synchronized. The usage pattern is:
	217	+respect in order to keep things properly synchronized. The usage pattern is::
225	218
226	219	int driver_populate_range(...)
227	220	{
@@ -262,9 +255,8 @@ report commands as executed is serialized (there is no point in doing this · 34 unmodified lines
262	255	concurrently).
263	256
264	257
265		−-------------------------------------------------------------------------------
266		−
267		−5) Represent and manage device memory from core kernel point of view
	258	+Represent and manage device memory from core kernel point of view
	259	+=================================================================
268	260
269	261	Several different designs were tried to support device memory. First one used
270	262	a device specific data structure to keep information about migrated memory and
@@ -280,14 +272,14 @@ unaware of the difference. We only need to make sure that no one ever tries to · 9 unmodified lines
280	272	map those pages from the CPU side.
281	273
282	274	HMM provides a set of helpers to register and hotplug device memory as a new
283		−region needing a struct page. This is offered through a very simple API:
	275	+region needing a struct page. This is offered through a very simple API::
284	276
285	277	struct hmm_devmem hmm_devmem_add(const struct hmm_devmem_ops ops,
286	278	struct device *device,
287	279	unsigned long size);
288	280	void hmm_devmem_remove(struct hmm_devmem *devmem);
289	281
290		−The hmm_devmem_ops is where most of the important things are:
	282	+The hmm_devmem_ops is where most of the important things are::
291	283
292	284	struct hmm_devmem_ops {
293	285	void (free)(struct hmm_devmem devmem, struct page *page);
@@ -306,13 +298,12 @@ which it cannot do. This second callback must trigger a migration back to · 12 unmodified lines
306	298	system memory.
307	299
308	300
309		−-------------------------------------------------------------------------------
310		−
311		−6) Migration to and from device memory
	301	+Migration to and from device memory
	302	+===================================
312	303
313	304	Because the CPU cannot access device memory, migration must use the device DMA
314	305	engine to perform copy from and to device memory. For this we need a new
315		−migration helper:
	306	+migration helper::
316	307
317	308	int migrate_vma(const struct migrate_vma_ops *ops,
318	309	struct vm_area_struct *vma,
@@ -331,7 +322,7 @@ migration might be for a range of addresses the device is actively accessing. · 12 unmodified lines
331	322
332	323	The migrate_vma_ops struct defines two callbacks. First one (alloc_and_copy())
333	324	controls destination memory allocation and copy operation. Second one is there
334		−to allow the device driver to perform cleanup operations after migration.
	325	+to allow the device driver to perform cleanup operations after migration::
335	326
336	327	struct migrate_vma_ops {
337	328	void (alloc_and_copy)(struct vm_area_struct vma,
@@ -365,9 +356,8 @@ bandwidth but this is considered as a rare event and a price that we are · 27 unmodified lines
365	356	willing to pay to keep all the code simpler.
366	357
367	358
368		−-------------------------------------------------------------------------------
369		−
370		−7) Memory cgroup (memcg) and rss accounting
	359	+Memory cgroup (memcg) and rss accounting
	360	+========================================
371	361
372	362	For now device memory is accounted as any regular page in rss counters (either
373	363	anonymous if device page is used for anonymous, file if device page is used for

Documentation/vm/hugetlbfs_reserv.rstrenamed from Documentation/vm/hugetlbfs_reserv.txt+143−77

@@ -1,6 +1,13 @@
1		−Hugetlbfs Reservation Overview
2		−------------------------------
3		−Huge pages as described at 'Documentation/vm/hugetlbpage.txt' are typically
	1	+.. _hugetlbfs_reserve:
	2	+
	3	+=====================
	4	+Hugetlbfs Reservation
	5	+=====================
	6	+
	7	+Overview
	8	+========
	9	+
	10	+Huge pages as described at :ref:`hugetlbpage` are typically
4	11	preallocated for application use. These huge pages are instantiated in a
5	12	task's address space at page fault time if the VMA indicates huge pages are
6	13	to be used. If no huge page exists at page fault time, the task is sent
@@ -17,47 +24,55 @@ describe how huge page reserve processing is done in the v4.10 kernel. · 10 unmodified lines
17	24
18	25
19	26	Audience
20		−--------
	27	+========
21	28	This description is primarily targeted at kernel developers who are modifying
22	29	hugetlbfs code.
23	30
24	31
25	32	The Data Structures
26		−-------------------
	33	+===================
	34	+
27	35	resv_huge_pages
28	36	This is a global (per-hstate) count of reserved huge pages. Reserved
29	37	huge pages are only available to the task which reserved them.
30	38	Therefore, the number of huge pages generally available is computed
31		− as (free_huge_pages - resv_huge_pages).
	39	+ as (``free_huge_pages - resv_huge_pages``).
32	40	Reserve Map
33		− A reserve map is described by the structure:
34		− struct resv_map {
35		− struct kref refs;
36		− spinlock_t lock;
37		− struct list_head regions;
38		− long adds_in_progress;
39		− struct list_head region_cache;
40		− long region_cache_count;
41		− };
	41	+ A reserve map is described by the structure::
	42	+
	43	+ struct resv_map {
	44	+ struct kref refs;
	45	+ spinlock_t lock;
	46	+ struct list_head regions;
	47	+ long adds_in_progress;
	48	+ struct list_head region_cache;
	49	+ long region_cache_count;
	50	+ };
	51	+
42	52	There is one reserve map for each huge page mapping in the system.
43	53	The regions list within the resv_map describes the regions within
44		− the mapping. A region is described as:
45		− struct file_region {
46		− struct list_head link;
47		− long from;
48		− long to;
49		− };
	54	+ the mapping. A region is described as::
	55	+
	56	+ struct file_region {
	57	+ struct list_head link;
	58	+ long from;
	59	+ long to;
	60	+ };
	61	+
50	62	The 'from' and 'to' fields of the file region structure are huge page
51	63	indices into the mapping. Depending on the type of mapping, a
52	64	region in the reserv_map may indicate reservations exist for the
53	65	range, or reservations do not exist.
54	66	Flags for MAP_PRIVATE Reservations
55	67	These are stored in the bottom bits of the reservation map pointer.
56		− #define HPAGE_RESV_OWNER (1UL << 0) Indicates this task is the
57		− owner of the reservations associated with the mapping.
58		− #define HPAGE_RESV_UNMAPPED (1UL << 1) Indicates task originally
59		− mapping this range (and creating reserves) has unmapped a
60		− page from this task (the child) due to a failed COW.
	68	+
	69	+ ``#define HPAGE_RESV_OWNER (1UL << 0)``
	70	+ Indicates this task is the owner of the reservations
	71	+ associated with the mapping.
	72	+ ``#define HPAGE_RESV_UNMAPPED (1UL << 1)``
	73	+ Indicates task originally mapping this range (and creating
	74	+ reserves) has unmapped a page from this task (the child)
	75	+ due to a failed COW.
61	76	Page Flags
62	77	The PagePrivate page flag is used to indicate that a huge page
63	78	reservation must be restored when the huge page is freed. More
@@ -65,12 +80,14 @@ Page Flags · 1 unmodified line
65	80
66	81
67	82	Reservation Map Location (Private or Shared)
68		−--------------------------------------------
	83	+============================================
	84	+
69	85	A huge page mapping or segment is either private or shared. If private,
70	86	it is typically only available to a single address space (task). If shared,
71	87	it can be mapped into multiple address spaces (tasks). The location and
72	88	semantics of the reservation map is significantly different for two types
73	89	of mappings. Location differences are:
	90	+
74	91	- For private mappings, the reservation map hangs off the the VMA structure.
75	92	Specifically, vma->vm_private_data. This reserve map is created at the
76	93	time the mapping (mmap(MAP_PRIVATE)) is created.
@@ -82,15 +99,15 @@ of mappings. Location differences are: · 5 unmodified lines
82	99
83	100
84	101	Creating Reservations
85		−---------------------
	102	+=====================
86	103	Reservations are created when a huge page backed shared memory segment is
87	104	created (shmget(SHM_HUGETLB)) or a mapping is created via mmap(MAP_HUGETLB).
88		−These operations result in a call to the routine hugetlb_reserve_pages()
	105	+These operations result in a call to the routine hugetlb_reserve_pages()::
89	106
90		−int hugetlb_reserve_pages(struct inode *inode,
91		− long from, long to,
92		− struct vm_area_struct *vma,
93		− vm_flags_t vm_flags)
	107	+ int hugetlb_reserve_pages(struct inode *inode,
	108	+ long from, long to,
	109	+ struct vm_area_struct *vma,
	110	+ vm_flags_t vm_flags)
94	111
95	112	The first thing hugetlb_reserve_pages() does is check for the NORESERVE
96	113	flag was specified in either the shmget() or mmap() call. If NORESERVE
@@ -105,6 +122,7 @@ the 'from' and 'to' arguments have been adjusted by this offset. · 8 unmodified lines
105	122
106	123	One of the big differences between PRIVATE and SHARED mappings is the way
107	124	in which reservations are represented in the reservation map.
	125	+
108	126	- For shared mappings, an entry in the reservation map indicates a reservation
109	127	exists or did exist for the corresponding page. As reservations are
110	128	consumed, the reservation map is not modified.
@@ -121,12 +139,13 @@ to indicate this VMA owns the reservations. · 10 unmodified lines
121	139	The reservation map is consulted to determine how many huge page reservations
122	140	are needed for the current mapping/segment. For private mappings, this is
123	141	always the value (to - from). However, for shared mappings it is possible that some reservations may already exist within the range (to - from). See the
124		−section "Reservation Map Modifications" for details on how this is accomplished.
	142	+section :ref:`Reservation Map Modifications <resv_map_modifications>`
	143	+for details on how this is accomplished.
125	144
126	145	The mapping may be associated with a subpool. If so, the subpool is consulted
127	146	to ensure there is sufficient space for the mapping. It is possible that the
128	147	subpool has set aside reservations that can be used for the mapping. See the
129		−section "Subpool Reservations" for more details.
	148	+section :ref:`Subpool Reservations <sub_pool_resv>` for more details.
130	149
131	150	After consulting the reservation map and subpool, the number of needed new
132	151	reservations is known. The routine hugetlb_acct_memory() is called to check
@@ -135,9 +154,11 @@ calls into routines that potentially allocate and adjust surplus page counts. · 2 unmodified lines
135	154	However, within those routines the code is simply checking to ensure there
136	155	are enough free huge pages to accommodate the reservation. If there are,
137	156	the global reservation count resv_huge_pages is adjusted something like the
138		−following.
	157	+following::
	158	+
139	159	if (resv_needed <= (resv_huge_pages - free_huge_pages))
140	160	resv_huge_pages += resv_needed;
	161	+
141	162	Note that the global lock hugetlb_lock is held when checking and adjusting
142	163	these counters.
143	164
@@ -152,14 +173,18 @@ If hugetlb_reserve_pages() was successful, the global reservation count and · 8 unmodified lines
152	173	reservation map associated with the mapping will be modified as required to
153	174	ensure reservations exist for the range 'from' - 'to'.
154	175
	176	+.. _consume_resv:
155	177
156	178	Consuming Reservations/Allocating a Huge Page
157		−---------------------------------------------
	179	+=============================================
	180	+
158	181	Reservations are consumed when huge pages associated with the reservations
159	182	are allocated and instantiated in the corresponding mapping. The allocation
160		−is performed within the routine alloc_huge_page().
161		−struct page alloc_huge_page(struct vm_area_struct vma,
162		− unsigned long addr, int avoid_reserve)
	183	+is performed within the routine alloc_huge_page()::
	184	+
	185	+ struct page alloc_huge_page(struct vm_area_struct vma,
	186	+ unsigned long addr, int avoid_reserve)
	187	+
163	188	alloc_huge_page is passed a VMA pointer and a virtual address, so it can
164	189	consult the reservation map to determine if a reservation exists. In addition,
165	190	alloc_huge_page takes the argument avoid_reserve which indicates reserves
@@ -170,8 +195,9 @@ page are being allocated. · 4 unmodified lines
170	195
171	196	The helper routine vma_needs_reservation() is called to determine if a
172	197	reservation exists for the address within the mapping(vma). See the section
173		−"Reservation Map Helper Routines" for detailed information on what this
174		−routine does. The value returned from vma_needs_reservation() is generally
	198	+:ref:`Reservation Map Helper Routines <resv_map_helpers>` for detailed
	199	+information on what this routine does.
	200	+The value returned from vma_needs_reservation() is generally
175	201	0 or 1. 0 if a reservation exists for the address, 1 if no reservation exists.
176	202	If a reservation does not exist, and there is a subpool associated with the
177	203	mapping the subpool is consulted to determine if it contains reservations.
@@ -180,21 +206,25 @@ However, in every case the avoid_reserve argument overrides the use of · 2 unmodified lines
180	206	a reservation for the allocation. After determining whether a reservation
181	207	exists and can be used for the allocation, the routine dequeue_huge_page_vma()
182	208	is called. This routine takes two arguments related to reservations:
	209	+
183	210	- avoid_reserve, this is the same value/argument passed to alloc_huge_page()
184	211	- chg, even though this argument is of type long only the values 0 or 1 are
185	212	passed to dequeue_huge_page_vma. If the value is 0, it indicates a
186	213	reservation exists (see the section "Memory Policy and Reservations" for
187	214	possible issues). If the value is 1, it indicates a reservation does not
188	215	exist and the page must be taken from the global free pool if possible.
	216	+
189	217	The free lists associated with the memory policy of the VMA are searched for
190	218	a free page. If a page is found, the value free_huge_pages is decremented
191	219	when the page is removed from the free list. If there was a reservation
192		−associated with the page, the following adjustments are made:
	220	+associated with the page, the following adjustments are made::
	221	+
193	222	SetPagePrivate(page); /* Indicates allocating this page consumed
194	223	* a reservation, and if an error is
195	224	* encountered such that the page must be
196	225	* freed, the reservation will be restored. */
197	226	resv_huge_pages--; /* Decrement the global reservation count */
	227	+
198	228	Note, if no huge page can be found that satisfies the VMA's memory policy
199	229	an attempt will be made to allocate one using the buddy allocator. This
200	230	brings up the issue of surplus huge pages and overcommit which is beyond
@@ -222,12 +252,14 @@ mapping. In such cases, the reservation count and subpool free page count · 21 unmodified lines
222	252	will be off by one. This rare condition can be identified by comparing the
223	253	return value from vma_needs_reservation and vma_commit_reservation. If such
224	254	a race is detected, the subpool and global reserve counts are adjusted to
225		−compensate. See the section "Reservation Map Helper Routines" for more
	255	+compensate. See the section
	256	+:ref:`Reservation Map Helper Routines <resv_map_helpers>` for more
226	257	information on these routines.
227	258
228	259
229	260	Instantiate Huge Pages
230		−----------------------
	261	+======================
	262	+
231	263	After huge page allocation, the page is typically added to the page tables
232	264	of the allocating task. Before this, pages in a shared mapping are added
233	265	to the page cache and pages in private mappings are added to an anonymous
@@ -237,7 +269,8 @@ to the global reservation count (resv_huge_pages). · 3 unmodified lines
237	269
238	270
239	271	Freeing Huge Pages
240		−------------------
	272	+==================
	273	+
241	274	Huge page freeing is performed by the routine free_huge_page(). This routine
242	275	is the destructor for hugetlbfs compound pages. As a result, it is only
243	276	passed a pointer to the page struct. When a huge page is freed, reservation
@@ -247,7 +280,8 @@ on an error path where a global reserve count must be restored. · 3 unmodified lines
247	280
248	281	The page->private field points to any subpool associated with the page.
249	282	If the PagePrivate flag is set, it indicates the global reserve count should
250		−be adjusted (see the section "Consuming Reservations/Allocating a Huge Page"
	283	+be adjusted (see the section
	284	+:ref:`Consuming Reservations/Allocating a Huge Page <consume_resv>`
251	285	for information on how these are set).
252	286
253	287	The routine first calls hugepage_subpool_put_pages() for the page. If this
@@ -259,9 +293,11 @@ Therefore, the global resv_huge_pages counter is incremented in this case. · 5 unmodified lines
259	293	If the PagePrivate flag was set in the page, the global resv_huge_pages counter
260	294	will always be incremented.
261	295
	296	+.. _sub_pool_resv:
262	297
263	298	Subpool Reservations
264		−--------------------
	299	+====================
	300	+
265	301	There is a struct hstate associated with each huge page size. The hstate
266	302	tracks all huge pages of the specified size. A subpool represents a subset
267	303	of pages within a hstate that is associated with a mounted hugetlbfs
@@ -295,7 +331,8 @@ the global pools. · 27 unmodified lines
295	331
296	332
297	333	COW and Reservations
298		−--------------------
	334	+====================
	335	+
299	336	Since shared mappings all point to and use the same underlying pages, the
300	337	biggest reservation concern for COW is private mappings. In this case,
301	338	two tasks can be pointing at the same previously allocated page. One task
@@ -326,30 +363,36 @@ faults on a non-present page. But, the original owner of the · 24 unmodified lines
326	363	mapping/reservation will behave as expected.
327	364
328	365
	366	+.. _resv_map_modifications:
	367	+
329	368	Reservation Map Modifications
330		−-----------------------------
	369	+=============================
	370	+
331	371	The following low level routines are used to make modifications to a
332	372	reservation map. Typically, these routines are not called directly. Rather,
333	373	a reservation map helper routine is called which calls one of these low level
334	374	routines. These low level routines are fairly well documented in the source
335		−code (mm/hugetlb.c). These routines are:
336		−long region_chg(struct resv_map *resv, long f, long t);
337		−long region_add(struct resv_map *resv, long f, long t);
338		−void region_abort(struct resv_map *resv, long f, long t);
339		−long region_count(struct resv_map *resv, long f, long t);
	375	+code (mm/hugetlb.c). These routines are::
	376	+
	377	+ long region_chg(struct resv_map *resv, long f, long t);
	378	+ long region_add(struct resv_map *resv, long f, long t);
	379	+ void region_abort(struct resv_map *resv, long f, long t);
	380	+ long region_count(struct resv_map *resv, long f, long t);
340	381
341	382	Operations on the reservation map typically involve two operations:
	383	+
342	384	1) region_chg() is called to examine the reserve map and determine how
343	385	many pages in the specified range [f, t) are NOT currently represented.
344	386
345	387	The calling code performs global checks and allocations to determine if
346	388	there are enough huge pages for the operation to succeed.
347	389
348		−2a) If the operation can succeed, region_add() is called to actually modify
349		− the reservation map for the same range [f, t) previously passed to
350		− region_chg().
351		−2b) If the operation can not succeed, region_abort is called for the same range
352		− [f, t) to abort the operation.
	390	+2)
	391	+ a) If the operation can succeed, region_add() is called to actually modify
	392	+ the reservation map for the same range [f, t) previously passed to
	393	+ region_chg().
	394	+ b) If the operation can not succeed, region_abort is called for the same
	395	+ range [f, t) to abort the operation.
353	396
354	397	Note that this is a two step process where region_add() and region_abort()
355	398	are guaranteed to succeed after a prior call to region_chg() for the same
@@ -371,6 +414,7 @@ and make the appropriate adjustments. · 15 unmodified lines
371	414
372	415	The routine region_del() is called to remove regions from a reservation map.
373	416	It is typically called in the following situations:
	417	+
374	418	- When a file in the hugetlbfs filesystem is being removed, the inode will
375	419	be released and the reservation map freed. Before freeing the reservation
376	420	map, all the individual file_region structures must be freed. In this case
@@ -384,6 +428,7 @@ It is typically called in the following situations: · 7 unmodified lines
384	428	removed, region_del() is called to remove the corresponding entry from the
385	429	reservation map. In this case, region_del is passed the range
386	430	[page_idx, page_idx + 1).
	431	+
387	432	In every case, region_del() will return the number of pages removed from the
388	433	reservation map. In VERY rare cases, region_del() can fail. This can only
389	434	happen in the hole punch case where it has to split an existing file_region
@@ -403,9 +448,11 @@ outstanding (outstanding = (end - start) - region_count(resv, start, end)). · 13 unmodified lines
403	448	Since the mapping is going away, the subpool and global reservation counts
404	449	are decremented by the number of outstanding reservations.
405	450
	451	+.. _resv_map_helpers:
406	452
407	453	Reservation Map Helper Routines
408		−-------------------------------
	454	+===============================
	455	+
409	456	Several helper routines exist to query and modify the reservation maps.
410	457	These routines are only interested with reservations for a specific huge
411	458	page, so they just pass in an address instead of a range. In addition,
@@ -414,32 +461,40 @@ or shared) and the location of the reservation map (inode or VMA) can be · 2 unmodified lines
414	461	determined. These routines simply call the underlying routines described
415	462	in the section "Reservation Map Modifications". However, they do take into
416	463	account the 'opposite' meaning of reservation map entries for private and
417		−shared mappings and hide this detail from the caller.
	464	+shared mappings and hide this detail from the caller::
	465	+
	466	+ long vma_needs_reservation(struct hstate *h,
	467	+ struct vm_area_struct *vma,
	468	+ unsigned long addr)
418	469
419		−long vma_needs_reservation(struct hstate *h,
420		− struct vm_area_struct *vma, unsigned long addr)
421	470	This routine calls region_chg() for the specified page. If no reservation
422		−exists, 1 is returned. If a reservation exists, 0 is returned.
	471	+exists, 1 is returned. If a reservation exists, 0 is returned::
	472	+
	473	+ long vma_commit_reservation(struct hstate *h,
	474	+ struct vm_area_struct *vma,
	475	+ unsigned long addr)
423	476
424		−long vma_commit_reservation(struct hstate *h,
425		− struct vm_area_struct *vma, unsigned long addr)
426	477	This calls region_add() for the specified page. As in the case of region_chg
427	478	and region_add, this routine is to be called after a previous call to
428	479	vma_needs_reservation. It will add a reservation entry for the page. It
429	480	returns 1 if the reservation was added and 0 if not. The return value should
430	481	be compared with the return value of the previous call to
431	482	vma_needs_reservation. An unexpected difference indicates the reservation
432		−map was modified between calls.
	483	+map was modified between calls::
	484	+
	485	+ void vma_end_reservation(struct hstate *h,
	486	+ struct vm_area_struct *vma,
	487	+ unsigned long addr)
433	488
434		−void vma_end_reservation(struct hstate *h,
435		− struct vm_area_struct *vma, unsigned long addr)
436	489	This calls region_abort() for the specified page. As in the case of region_chg
437	490	and region_abort, this routine is to be called after a previous call to
438	491	vma_needs_reservation. It will abort/end the in progress reservation add
439		−operation.
	492	+operation::
	493	+
	494	+ long vma_add_reservation(struct hstate *h,
	495	+ struct vm_area_struct *vma,
	496	+ unsigned long addr)
440	497
441		−long vma_add_reservation(struct hstate *h,
442		− struct vm_area_struct *vma, unsigned long addr)
443	498	This is a special wrapper routine to help facilitate reservation cleanup
444	499	on error paths. It is only called from the routine restore_reserve_on_error().
445	500	This routine is used in conjunction with vma_needs_reservation in an attempt
@@ -453,8 +508,10 @@ be done on error paths. · 7 unmodified lines
453	508
454	509
455	510	Reservation Cleanup in Error Paths
456		−----------------------------------
457		−As mentioned in the section "Reservation Map Helper Routines", reservation
	511	+==================================
	512	+
	513	+As mentioned in the section
	514	+:ref:`Reservation Map Helper Routines <resv_map_helpers>`, reservation
458	515	map modifications are performed in two steps. First vma_needs_reservation
459	516	is called before a page is allocated. If the allocation is successful,
460	517	then vma_commit_reservation is called. If not, vma_end_reservation is called.
@@ -494,13 +551,14 @@ so that a reservation will not be leaked when the huge page is freed. · 33 unmodified lines
494	551
495	552
496	553	Reservations and Memory Policy
497		−------------------------------
	554	+==============================
498	555	Per-node huge page lists existed in struct hstate when git was first used
499	556	to manage Linux code. The concept of reservations was added some time later.
500	557	When reservations were added, no attempt was made to take memory policy
501	558	into account. While cpusets are not exactly the same as memory policy, this
502	559	comment in hugetlb_acct_memory sums up the interaction between reservations
503		−and cpusets/memory policy.
	560	+and cpusets/memory policy::
	561	+
504	562	/*
505	563	* When cpuset is configured, it breaks the strict hugetlb page
506	564	* reservation as the accounting is done on a global variable. Such
@@ -525,5 +583,13 @@ of cpusets or memory policy there is no guarantee that huge pages will be · 18 unmodified lines
525	583	available on the required nodes. This is true even if there are a sufficient
526	584	number of global reservations.
527	585
	586	+Hugetlbfs regression testing
	587	+============================
	588	+
	589	+The most complete set of hugetlb tests are in the libhugetlbfs repository.
	590	+If you modify any hugetlb related code, use the libhugetlbfs test suite
	591	+to check for regressions. In addition, if you add any new hugetlb
	592	+functionality, please add appropriate tests to libhugetlbfs.
528	593
	594	+--
529	595	Mike Kravetz, 7 April 2017

Documentation/vm/hwpoison.rstrenamed from Documentation/vm/hwpoison.txt+70−71

@@ -1,7 +1,14 @@
	1	+.. hwpoison:
	2	+
	3	+========
	4	+hwpoison
	5	+========
	6	+
1	7	What is hwpoison?
	8	+=================
2	9
3	10	Upcoming Intel CPUs have support for recovering from some memory errors
4		−(``MCA recovery''). This requires the OS to declare a page "poisoned",
	11	+(``MCA recovery``). This requires the OS to declare a page "poisoned",
5	12	kill the processes associated with it and avoid using it in the future.
6	13
7	14	This patchkit implements the necessary infrastructure in the VM.
@@ -46,9 +53,10 @@ address. This in theory allows other applications to handle · 38 unmodified lines
46	53	memory failures too. The expection is that near all applications
47	54	won't do that, but some very specialized ones might.
48	55
49		−---
	56	+Failure recovery modes
	57	+======================
50	58
51		−There are two (actually three) modi memory failure recovery can be in:
	59	+There are two (actually three) modes memory failure recovery can be in:
52	60
53	61	vm.memory_failure_recovery sysctl set to zero:
54	62	All memory failures cause a panic. Do not attempt recovery.
@@ -67,9 +75,8 @@ late kill · 12 unmodified lines
67	75	This is best for memory error unaware applications and default
68	76	Note some pages are always handled as late kill.
69	77
70		−---
71		−
72		−User control:
	78	+User control
	79	+============
73	80
74	81	vm.memory_failure_recovery
75	82	See sysctl.txt
@@ -79,11 +86,19 @@ vm.memory_failure_early_kill · 3 unmodified lines
79	86
80	87	PR_MCE_KILL
81	88	Set early/late kill mode/revert to system default
82		− arg1: PR_MCE_KILL_CLEAR: Revert to system default
83		− arg1: PR_MCE_KILL_SET: arg2 defines thread specific mode
84		− PR_MCE_KILL_EARLY: Early kill
85		− PR_MCE_KILL_LATE: Late kill
86		− PR_MCE_KILL_DEFAULT: Use system global default
	89	+
	90	+ arg1: PR_MCE_KILL_CLEAR:
	91	+ Revert to system default
	92	+ arg1: PR_MCE_KILL_SET:
	93	+ arg2 defines thread specific mode
	94	+
	95	+ PR_MCE_KILL_EARLY:
	96	+ Early kill
	97	+ PR_MCE_KILL_LATE:
	98	+ Late kill
	99	+ PR_MCE_KILL_DEFAULT
	100	+ Use system global default
	101	+
87	102	Note that if you want to have a dedicated thread which handles
88	103	the SIGBUS(BUS_MCEERR_AO) on behalf of the process, you should
89	104	call prctl(PR_MCE_KILL_EARLY) on the designated thread. Otherwise,
@@ -92,77 +107,64 @@ PR_MCE_KILL · 2 unmodified lines
92	107	PR_MCE_KILL_GET
93	108	return current mode
94	109
	110	+Testing
	111	+=======
95	112
96		−---
97		−
98		−Testing:
99		−
100		−madvise(MADV_HWPOISON, ....)
101		− (as root)
102		− Poison a page in the process for testing
103		−
	113	+* madvise(MADV_HWPOISON, ....) (as root) - Poison a page in the
	114	+ process for testing
104	115
105		−hwpoison-inject module through debugfs
	116	+* hwpoison-inject module through debugfs ``/sys/kernel/debug/hwpoison/``
106	117
107		−/sys/kernel/debug/hwpoison/
	118	+ corrupt-pfn
	119	+ Inject hwpoison fault at PFN echoed into this file. This does
	120	+ some early filtering to avoid corrupted unintended pages in test suites.
108	121
109		−corrupt-pfn
	122	+ unpoison-pfn
	123	+ Software-unpoison page at PFN echoed into this file. This way
	124	+ a page can be reused again. This only works for Linux
	125	+ injected failures, not for real memory failures.
110	126
111		−Inject hwpoison fault at PFN echoed into this file. This does
112		−some early filtering to avoid corrupted unintended pages in test suites.
	127	+ Note these injection interfaces are not stable and might change between
	128	+ kernel versions
113	129
114		−unpoison-pfn
	130	+ corrupt-filter-dev-major, corrupt-filter-dev-minor
	131	+ Only handle memory failures to pages associated with the file
	132	+ system defined by block device major/minor. -1U is the
	133	+ wildcard value. This should be only used for testing with
	134	+ artificial injection.
115	135
116		−Software-unpoison page at PFN echoed into this file. This
117		−way a page can be reused again.
118		−This only works for Linux injected failures, not for real
119		−memory failures.
	136	+ corrupt-filter-memcg
	137	+ Limit injection to pages owned by memgroup. Specified by inode
	138	+ number of the memcg.
120	139
121		−Note these injection interfaces are not stable and might change between
122		−kernel versions
	140	+ Example::
123	141
124		−corrupt-filter-dev-major
125		−corrupt-filter-dev-minor
	142	+ mkdir /sys/fs/cgroup/mem/hwpoison
126	143
127		−Only handle memory failures to pages associated with the file system defined
128		−by block device major/minor. -1U is the wildcard value.
129		−This should be only used for testing with artificial injection.
	144	+ usemem -m 100 -s 1000 &
	145	+ echo `jobs -p` > /sys/fs/cgroup/mem/hwpoison/tasks
130	146
131		−corrupt-filter-memcg
	147	+ memcg_ino=$(ls -id /sys/fs/cgroup/mem/hwpoison \| cut -f1 -d' ')
	148	+ echo $memcg_ino > /debug/hwpoison/corrupt-filter-memcg
132	149
133		−Limit injection to pages owned by memgroup. Specified by inode number
134		−of the memcg.
	150	+ page-types -p `pidof init` --hwpoison # shall do nothing
	151	+ page-types -p `pidof usemem` --hwpoison # poison its pages
135	152
136		−Example:
137		− mkdir /sys/fs/cgroup/mem/hwpoison
	153	+ corrupt-filter-flags-mask, corrupt-filter-flags-value
	154	+ When specified, only poison pages if ((page_flags & mask) ==
	155	+ value). This allows stress testing of many kinds of
	156	+ pages. The page_flags are the same as in /proc/kpageflags. The
	157	+ flag bits are defined in include/linux/kernel-page-flags.h and
	158	+ documented in Documentation/admin-guide/mm/pagemap.rst
138	159
139		− usemem -m 100 -s 1000 &
140		− echo `jobs -p` > /sys/fs/cgroup/mem/hwpoison/tasks
	160	+* Architecture specific MCE injector
141	161
142		− memcg_ino=$(ls -id /sys/fs/cgroup/mem/hwpoison \| cut -f1 -d' ')
143		− echo $memcg_ino > /debug/hwpoison/corrupt-filter-memcg
	162	+ x86 has mce-inject, mce-test
144	163
145		− page-types -p `pidof init` --hwpoison # shall do nothing
146		− page-types -p `pidof usemem` --hwpoison # poison its pages
	164	+ Some portable hwpoison test programs in mce-test, see below.
147	165
148		−corrupt-filter-flags-mask
149		−corrupt-filter-flags-value
150		−
151		−When specified, only poison pages if ((page_flags & mask) == value).
152		−This allows stress testing of many kinds of pages. The page_flags
153		−are the same as in /proc/kpageflags. The flag bits are defined in
154		−include/linux/kernel-page-flags.h and documented in
155		−Documentation/vm/pagemap.txt
156		−
157		−Architecture specific MCE injector
158		−
159		−x86 has mce-inject, mce-test
160		−
161		−Some portable hwpoison test programs in mce-test, see blow.
162		−
163		−---
164		−
165		−References:
	166	+References
	167	+==========
166	168
167	169	http://halobates.de/mce-lc09-2.pdf
168	170	Overview presentation from LinuxCon 09
@@ -174,14 +176,11 @@ git://git.kernel.org/pub/scm/utils/cpu/mce/mce-inject.git · 5 unmodified lines
174	176	x86 specific injector
175	177
176	178
177		−---
178		−
179		−Limitations:
180		−
	179	+Limitations
	180	+===========
181	181	- Not all page types are supported and never will. Most kernel internal
182		−objects cannot be recovered, only LRU pages for now.
	182	+ objects cannot be recovered, only LRU pages for now.
183	183	- Right now hugepage support is missing.
184	184
185	185	---
186	186	Andi Kleen, Oct 2009
187		−

Documentation/vm/index.rst+50−0

@@ -0,0 +1,50 @@
	1	+=====================================
	2	+Linux Memory Management Documentation
	3	+=====================================
	4	+
	5	+This is a collection of documents about Linux memory management (mm) subsystem.
	6	+
	7	+User guides for MM features
	8	+===========================
	9	+
	10	+The following documents provide guides for controlling and tuning
	11	+various features of the Linux memory management
	12	+
	13	+.. toctree::
	14	+ :maxdepth: 1
	15	+
	16	+ swap_numa
	17	+ zswap
	18	+
	19	+Kernel developers MM documentation
	20	+==================================
	21	+
	22	+The below documents describe MM internals with different level of
	23	+details ranging from notes and mailing list responses to elaborate
	24	+descriptions of data structures and algorithms.
	25	+
	26	+.. toctree::
	27	+ :maxdepth: 1
	28	+
	29	+ active_mm
	30	+ balance
	31	+ cleancache
	32	+ frontswap
	33	+ highmem
	34	+ hmm
	35	+ hwpoison
	36	+ hugetlbfs_reserv
	37	+ ksm
	38	+ mmu_notifier
	39	+ numa
	40	+ overcommit-accounting
	41	+ page_migration
	42	+ page_frags
	43	+ page_owner
	44	+ remap_file_pages
	45	+ slub
	46	+ split_page_table_lock
	47	+ transhuge
	48	+ unevictable-lru
	49	+ z3fold
	50	+ zsmalloc

Documentation/vm/ksm.rst+87−0

@@ -0,0 +1,87 @@
	1	+.. _ksm:
	2	+
	3	+=======================
	4	+Kernel Samepage Merging
	5	+=======================
	6	+
	7	+KSM is a memory-saving de-duplication feature, enabled by CONFIG_KSM=y,
	8	+added to the Linux kernel in 2.6.32. See ``mm/ksm.c`` for its implementation,
	9	+and http://lwn.net/Articles/306704/ and http://lwn.net/Articles/330589/
	10	+
	11	+The userspace interface of KSM is described in :ref:`Documentation/admin-guide/mm/ksm.rst <admin_guide_ksm>`
	12	+
	13	+Design
	14	+======
	15	+
	16	+Overview
	17	+--------
	18	+
	19	+.. kernel-doc:: mm/ksm.c
	20	+ :DOC: Overview
	21	+
	22	+Reverse mapping
	23	+---------------
	24	+KSM maintains reverse mapping information for KSM pages in the stable
	25	+tree.
	26	+
	27	+If a KSM page is shared between less than ``max_page_sharing`` VMAs,
	28	+the node of the stable tree that represents such KSM page points to a
	29	+list of :c:type:`struct rmap_item` and the ``page->mapping`` of the
	30	+KSM page points to the stable tree node.
	31	+
	32	+When the sharing passes this threshold, KSM adds a second dimension to
	33	+the stable tree. The tree node becomes a "chain" that links one or
	34	+more "dups". Each "dup" keeps reverse mapping information for a KSM
	35	+page with ``page->mapping`` pointing to that "dup".
	36	+
	37	+Every "chain" and all "dups" linked into a "chain" enforce the
	38	+invariant that they represent the same write protected memory content,
	39	+even if each "dup" will be pointed by a different KSM page copy of
	40	+that content.
	41	+
	42	+This way the stable tree lookup computational complexity is unaffected
	43	+if compared to an unlimited list of reverse mappings. It is still
	44	+enforced that there cannot be KSM page content duplicates in the
	45	+stable tree itself.
	46	+
	47	+The deduplication limit enforced by ``max_page_sharing`` is required
	48	+to avoid the virtual memory rmap lists to grow too large. The rmap
	49	+walk has O(N) complexity where N is the number of rmap_items
	50	+(i.e. virtual mappings) that are sharing the page, which is in turn
	51	+capped by ``max_page_sharing``. So this effectively spreads the linear
	52	+O(N) computational complexity from rmap walk context over different
	53	+KSM pages. The ksmd walk over the stable_node "chains" is also O(N),
	54	+but N is the number of stable_node "dups", not the number of
	55	+rmap_items, so it has not a significant impact on ksmd performance. In
	56	+practice the best stable_node "dup" candidate will be kept and found
	57	+at the head of the "dups" list.
	58	+
	59	+High values of ``max_page_sharing`` result in faster memory merging
	60	+(because there will be fewer stable_node dups queued into the
	61	+stable_node chain->hlist to check for pruning) and higher
	62	+deduplication factor at the expense of slower worst case for rmap
	63	+walks for any KSM page which can happen during swapping, compaction,
	64	+NUMA balancing and page migration.
	65	+
	66	+The ``stable_node_dups/stable_node_chains`` ratio is also affected by the
	67	+``max_page_sharing`` tunable, and an high ratio may indicate fragmentation
	68	+in the stable_node dups, which could be solved by introducing
	69	+fragmentation algorithms in ksmd which would refile rmap_items from
	70	+one stable_node dup to another stable_node dup, in order to free up
	71	+stable_node "dups" with few rmap_items in them, but that may increase
	72	+the ksmd CPU usage and possibly slowdown the readonly computations on
	73	+the KSM pages of the applications.
	74	+
	75	+The whole list of stable_node "dups" linked in the stable_node
	76	+"chains" is scanned periodically in order to prune stale stable_nodes.
	77	+The frequency of such scans is defined by
	78	+``stable_node_chains_prune_millisecs`` sysfs tunable.
	79	+
	80	+Reference
	81	+---------
	82	+.. kernel-doc:: mm/ksm.c
	83	+ :functions: mm_slot ksm_scan stable_node rmap_item
	84	+
	85	+--
	86	+Izik Eidus,
	87	+Hugh Dickins, 17 Nov 2009

Documentation/vm/ksm.txt+0−178

@@ -1,178 +0,0 @@
1		−How to use the Kernel Samepage Merging feature
2		−----------------------------------------------
3		−
4		−KSM is a memory-saving de-duplication feature, enabled by CONFIG_KSM=y,
5		−added to the Linux kernel in 2.6.32. See mm/ksm.c for its implementation,
6		−and http://lwn.net/Articles/306704/ and http://lwn.net/Articles/330589/
7		−
8		−The KSM daemon ksmd periodically scans those areas of user memory which
9		−have been registered with it, looking for pages of identical content which
10		−can be replaced by a single write-protected page (which is automatically
11		−copied if a process later wants to update its content).
12		−
13		−KSM was originally developed for use with KVM (where it was known as
14		−Kernel Shared Memory), to fit more virtual machines into physical memory,
15		−by sharing the data common between them. But it can be useful to any
16		−application which generates many instances of the same data.
17		−
18		−KSM only merges anonymous (private) pages, never pagecache (file) pages.
19		−KSM's merged pages were originally locked into kernel memory, but can now
20		−be swapped out just like other user pages (but sharing is broken when they
21		−are swapped back in: ksmd must rediscover their identity and merge again).
22		−
23		−KSM only operates on those areas of address space which an application
24		−has advised to be likely candidates for merging, by using the madvise(2)
25		−system call: int madvise(addr, length, MADV_MERGEABLE).
26		−
27		−The app may call int madvise(addr, length, MADV_UNMERGEABLE) to cancel
28		−that advice and restore unshared pages: whereupon KSM unmerges whatever
29		−it merged in that range. Note: this unmerging call may suddenly require
30		−more memory than is available - possibly failing with EAGAIN, but more
31		−probably arousing the Out-Of-Memory killer.
32		−
33		−If KSM is not configured into the running kernel, madvise MADV_MERGEABLE
34		−and MADV_UNMERGEABLE simply fail with EINVAL. If the running kernel was
35		−built with CONFIG_KSM=y, those calls will normally succeed: even if the
36		−the KSM daemon is not currently running, MADV_MERGEABLE still registers
37		−the range for whenever the KSM daemon is started; even if the range
38		−cannot contain any pages which KSM could actually merge; even if
39		−MADV_UNMERGEABLE is applied to a range which was never MADV_MERGEABLE.
40		−
41		−If a region of memory must be split into at least one new MADV_MERGEABLE
42		−or MADV_UNMERGEABLE region, the madvise may return ENOMEM if the process
43		−will exceed vm.max_map_count (see Documentation/sysctl/vm.txt).
44		−
45		−Like other madvise calls, they are intended for use on mapped areas of
46		−the user address space: they will report ENOMEM if the specified range
47		−includes unmapped gaps (though working on the intervening mapped areas),
48		−and might fail with EAGAIN if not enough memory for internal structures.
49		−
50		−Applications should be considerate in their use of MADV_MERGEABLE,
51		−restricting its use to areas likely to benefit. KSM's scans may use a lot
52		−of processing power: some installations will disable KSM for that reason.
53		−
54		−The KSM daemon is controlled by sysfs files in /sys/kernel/mm/ksm/,
55		−readable by all but writable only by root:
56		−
57		−pages_to_scan - how many present pages to scan before ksmd goes to sleep
58		− e.g. "echo 100 > /sys/kernel/mm/ksm/pages_to_scan"
59		− Default: 100 (chosen for demonstration purposes)
60		−
61		−sleep_millisecs - how many milliseconds ksmd should sleep before next scan
62		− e.g. "echo 20 > /sys/kernel/mm/ksm/sleep_millisecs"
63		− Default: 20 (chosen for demonstration purposes)
64		−
65		−merge_across_nodes - specifies if pages from different numa nodes can be merged.
66		− When set to 0, ksm merges only pages which physically
67		− reside in the memory area of same NUMA node. That brings
68		− lower latency to access of shared pages. Systems with more
69		− nodes, at significant NUMA distances, are likely to benefit
70		− from the lower latency of setting 0. Smaller systems, which
71		− need to minimize memory usage, are likely to benefit from
72		− the greater sharing of setting 1 (default). You may wish to
73		− compare how your system performs under each setting, before
74		− deciding on which to use. merge_across_nodes setting can be
75		− changed only when there are no ksm shared pages in system:
76		− set run 2 to unmerge pages first, then to 1 after changing
77		− merge_across_nodes, to remerge according to the new setting.
78		− Default: 1 (merging across nodes as in earlier releases)
79		−
80		−run - set 0 to stop ksmd from running but keep merged pages,
81		− set 1 to run ksmd e.g. "echo 1 > /sys/kernel/mm/ksm/run",
82		− set 2 to stop ksmd and unmerge all pages currently merged,
83		− but leave mergeable areas registered for next run
84		− Default: 0 (must be changed to 1 to activate KSM,
85		− except if CONFIG_SYSFS is disabled)
86		−
87		−use_zero_pages - specifies whether empty pages (i.e. allocated pages
88		− that only contain zeroes) should be treated specially.
89		− When set to 1, empty pages are merged with the kernel
90		− zero page(s) instead of with each other as it would
91		− happen normally. This can improve the performance on
92		− architectures with coloured zero pages, depending on
93		− the workload. Care should be taken when enabling this
94		− setting, as it can potentially degrade the performance
95		− of KSM for some workloads, for example if the checksums
96		− of pages candidate for merging match the checksum of
97		− an empty page. This setting can be changed at any time,
98		− it is only effective for pages merged after the change.
99		− Default: 0 (normal KSM behaviour as in earlier releases)
100		−
101		−max_page_sharing - Maximum sharing allowed for each KSM page. This
102		− enforces a deduplication limit to avoid the virtual
103		− memory rmap lists to grow too large. The minimum
104		− value is 2 as a newly created KSM page will have at
105		− least two sharers. The rmap walk has O(N)
106		− complexity where N is the number of rmap_items
107		− (i.e. virtual mappings) that are sharing the page,
108		− which is in turn capped by max_page_sharing. So
109		− this effectively spread the the linear O(N)
110		− computational complexity from rmap walk context
111		− over different KSM pages. The ksmd walk over the
112		− stable_node "chains" is also O(N), but N is the
113		− number of stable_node "dups", not the number of
114		− rmap_items, so it has not a significant impact on
115		− ksmd performance. In practice the best stable_node
116		− "dup" candidate will be kept and found at the head
117		− of the "dups" list. The higher this value the
118		− faster KSM will merge the memory (because there
119		− will be fewer stable_node dups queued into the
120		− stable_node chain->hlist to check for pruning) and
121		− the higher the deduplication factor will be, but
122		− the slowest the worst case rmap walk could be for
123		− any given KSM page. Slowing down the rmap_walk
124		− means there will be higher latency for certain
125		− virtual memory operations happening during
126		− swapping, compaction, NUMA balancing and page
127		− migration, in turn decreasing responsiveness for
128		− the caller of those virtual memory operations. The
129		− scheduler latency of other tasks not involved with
130		− the VM operations doing the rmap walk is not
131		− affected by this parameter as the rmap walks are
132		− always schedule friendly themselves.
133		−
134		−stable_node_chains_prune_millisecs - How frequently to walk the whole
135		− list of stable_node "dups" linked in the
136		− stable_node "chains" in order to prune stale
137		− stable_nodes. Smaller milllisecs values will free
138		− up the KSM metadata with lower latency, but they
139		− will make ksmd use more CPU during the scan. This
140		− only applies to the stable_node chains so it's a
141		− noop if not a single KSM page hit the
142		− max_page_sharing yet (there would be no stable_node
143		− chains in such case).
144		−
145		−The effectiveness of KSM and MADV_MERGEABLE is shown in /sys/kernel/mm/ksm/:
146		−
147		−pages_shared - how many shared pages are being used
148		−pages_sharing - how many more sites are sharing them i.e. how much saved
149		−pages_unshared - how many pages unique but repeatedly checked for merging
150		−pages_volatile - how many pages changing too fast to be placed in a tree
151		−full_scans - how many times all mergeable areas have been scanned
152		−
153		−stable_node_chains - number of stable node chains allocated, this is
154		− effectively the number of KSM pages that hit the
155		− max_page_sharing limit
156		−stable_node_dups - number of stable node dups queued into the
157		− stable_node chains
158		−
159		−A high ratio of pages_sharing to pages_shared indicates good sharing, but
160		−a high ratio of pages_unshared to pages_sharing indicates wasted effort.
161		−pages_volatile embraces several different kinds of activity, but a high
162		−proportion there would also indicate poor use of madvise MADV_MERGEABLE.
163		−
164		−The maximum possible page_sharing/page_shared ratio is limited by the
165		−max_page_sharing tunable. To increase the ratio max_page_sharing must
166		−be increased accordingly.
167		−
168		−The stable_node_dups/stable_node_chains ratio is also affected by the
169		−max_page_sharing tunable, and an high ratio may indicate fragmentation
170		−in the stable_node dups, which could be solved by introducing
171		−fragmentation algorithms in ksmd which would refile rmap_items from
172		−one stable_node dup to another stable_node dup, in order to freeup
173		−stable_node "dups" with few rmap_items in them, but that may increase
174		−the ksmd CPU usage and possibly slowdown the readonly computations on
175		−the KSM pages of the applications.
176		−
177		−Izik Eidus,
178		−Hugh Dickins, 17 Nov 2009

Documentation/vm/mmu_notifier.rst+99−0

@@ -0,0 +1,99 @@
	1	+.. _mmu_notifier:
	2	+
	3	+When do you need to notify inside page table lock ?
	4	+===================================================
	5	+
	6	+When clearing a pte/pmd we are given a choice to notify the event through
	7	+(notify version of \*_clear_flush call mmu_notifier_invalidate_range) under
	8	+the page table lock. But that notification is not necessary in all cases.
	9	+
	10	+For secondary TLB (non CPU TLB) like IOMMU TLB or device TLB (when device use
	11	+thing like ATS/PASID to get the IOMMU to walk the CPU page table to access a
	12	+process virtual address space). There is only 2 cases when you need to notify
	13	+those secondary TLB while holding page table lock when clearing a pte/pmd:
	14	+
	15	+ A) page backing address is free before mmu_notifier_invalidate_range_end()
	16	+ B) a page table entry is updated to point to a new page (COW, write fault
	17	+ on zero page, __replace_page(), ...)
	18	+
	19	+Case A is obvious you do not want to take the risk for the device to write to
	20	+a page that might now be used by some completely different task.
	21	+
	22	+Case B is more subtle. For correctness it requires the following sequence to
	23	+happen:
	24	+
	25	+ - take page table lock
	26	+ - clear page table entry and notify ([pmd/pte]p_huge_clear_flush_notify())
	27	+ - set page table entry to point to new page
	28	+
	29	+If clearing the page table entry is not followed by a notify before setting
	30	+the new pte/pmd value then you can break memory model like C11 or C++11 for
	31	+the device.
	32	+
	33	+Consider the following scenario (device use a feature similar to ATS/PASID):
	34	+
	35	+Two address addrA and addrB such that \\|addrA - addrB\\| >= PAGE_SIZE we assume
	36	+they are write protected for COW (other case of B apply too).
	37	+
	38	+::
	39	+
	40	+ [Time N] --------------------------------------------------------------------
	41	+ CPU-thread-0 {try to write to addrA}
	42	+ CPU-thread-1 {try to write to addrB}
	43	+ CPU-thread-2 {}
	44	+ CPU-thread-3 {}
	45	+ DEV-thread-0 {read addrA and populate device TLB}
	46	+ DEV-thread-2 {read addrB and populate device TLB}
	47	+ [Time N+1] ------------------------------------------------------------------
	48	+ CPU-thread-0 {COW_step0: {mmu_notifier_invalidate_range_start(addrA)}}
	49	+ CPU-thread-1 {COW_step0: {mmu_notifier_invalidate_range_start(addrB)}}
	50	+ CPU-thread-2 {}
	51	+ CPU-thread-3 {}
	52	+ DEV-thread-0 {}
	53	+ DEV-thread-2 {}
	54	+ [Time N+2] ------------------------------------------------------------------
	55	+ CPU-thread-0 {COW_step1: {update page table to point to new page for addrA}}
	56	+ CPU-thread-1 {COW_step1: {update page table to point to new page for addrB}}
	57	+ CPU-thread-2 {}
	58	+ CPU-thread-3 {}
	59	+ DEV-thread-0 {}
	60	+ DEV-thread-2 {}
	61	+ [Time N+3] ------------------------------------------------------------------
	62	+ CPU-thread-0 {preempted}
	63	+ CPU-thread-1 {preempted}
	64	+ CPU-thread-2 {write to addrA which is a write to new page}
	65	+ CPU-thread-3 {}
	66	+ DEV-thread-0 {}
	67	+ DEV-thread-2 {}
	68	+ [Time N+3] ------------------------------------------------------------------
	69	+ CPU-thread-0 {preempted}
	70	+ CPU-thread-1 {preempted}
	71	+ CPU-thread-2 {}
	72	+ CPU-thread-3 {write to addrB which is a write to new page}
	73	+ DEV-thread-0 {}
	74	+ DEV-thread-2 {}
	75	+ [Time N+4] ------------------------------------------------------------------
	76	+ CPU-thread-0 {preempted}
	77	+ CPU-thread-1 {COW_step3: {mmu_notifier_invalidate_range_end(addrB)}}
	78	+ CPU-thread-2 {}
	79	+ CPU-thread-3 {}
	80	+ DEV-thread-0 {}
	81	+ DEV-thread-2 {}
	82	+ [Time N+5] ------------------------------------------------------------------
	83	+ CPU-thread-0 {preempted}
	84	+ CPU-thread-1 {}
	85	+ CPU-thread-2 {}
	86	+ CPU-thread-3 {}
	87	+ DEV-thread-0 {read addrA from old page}
	88	+ DEV-thread-2 {read addrB from new page}
	89	+
	90	+So here because at time N+2 the clear page table entry was not pair with a
	91	+notification to invalidate the secondary TLB, the device see the new value for
	92	+addrB before seing the new value for addrA. This break total memory ordering
	93	+for the device.
	94	+
	95	+When changing a pte to write protect or to point to a new write protected page
	96	+with same content (KSM) it is fine to delay the mmu_notifier_invalidate_range
	97	+call to mmu_notifier_invalidate_range_end() outside the page table lock. This
	98	+is true even if the thread doing the page table update is preempted right after
	99	+releasing page table lock but before call mmu_notifier_invalidate_range_end().

Documentation/vm/mmu_notifier.txt+0−93

@@ -1,93 +0,0 @@
1		−When do you need to notify inside page table lock ?
2		−
3		−When clearing a pte/pmd we are given a choice to notify the event through
4		−(notify version of *_clear_flush call mmu_notifier_invalidate_range) under
5		−the page table lock. But that notification is not necessary in all cases.
6		−
7		−For secondary TLB (non CPU TLB) like IOMMU TLB or device TLB (when device use
8		−thing like ATS/PASID to get the IOMMU to walk the CPU page table to access a
9		−process virtual address space). There is only 2 cases when you need to notify
10		−those secondary TLB while holding page table lock when clearing a pte/pmd:
11		−
12		− A) page backing address is free before mmu_notifier_invalidate_range_end()
13		− B) a page table entry is updated to point to a new page (COW, write fault
14		− on zero page, __replace_page(), ...)
15		−
16		−Case A is obvious you do not want to take the risk for the device to write to
17		−a page that might now be used by some completely different task.
18		−
19		−Case B is more subtle. For correctness it requires the following sequence to
20		−happen:
21		− - take page table lock
22		− - clear page table entry and notify ([pmd/pte]p_huge_clear_flush_notify())
23		− - set page table entry to point to new page
24		−
25		−If clearing the page table entry is not followed by a notify before setting
26		−the new pte/pmd value then you can break memory model like C11 or C++11 for
27		−the device.
28		−
29		−Consider the following scenario (device use a feature similar to ATS/PASID):
30		−
31		−Two address addrA and addrB such that \|addrA - addrB\| >= PAGE_SIZE we assume
32		−they are write protected for COW (other case of B apply too).
33		−
34		−[Time N] --------------------------------------------------------------------
35		−CPU-thread-0 {try to write to addrA}
36		−CPU-thread-1 {try to write to addrB}
37		−CPU-thread-2 {}
38		−CPU-thread-3 {}
39		−DEV-thread-0 {read addrA and populate device TLB}
40		−DEV-thread-2 {read addrB and populate device TLB}
41		−[Time N+1] ------------------------------------------------------------------
42		−CPU-thread-0 {COW_step0: {mmu_notifier_invalidate_range_start(addrA)}}
43		−CPU-thread-1 {COW_step0: {mmu_notifier_invalidate_range_start(addrB)}}
44		−CPU-thread-2 {}
45		−CPU-thread-3 {}
46		−DEV-thread-0 {}
47		−DEV-thread-2 {}
48		−[Time N+2] ------------------------------------------------------------------
49		−CPU-thread-0 {COW_step1: {update page table to point to new page for addrA}}
50		−CPU-thread-1 {COW_step1: {update page table to point to new page for addrB}}
51		−CPU-thread-2 {}
52		−CPU-thread-3 {}
53		−DEV-thread-0 {}
54		−DEV-thread-2 {}
55		−[Time N+3] ------------------------------------------------------------------
56		−CPU-thread-0 {preempted}
57		−CPU-thread-1 {preempted}
58		−CPU-thread-2 {write to addrA which is a write to new page}
59		−CPU-thread-3 {}
60		−DEV-thread-0 {}
61		−DEV-thread-2 {}
62		−[Time N+3] ------------------------------------------------------------------
63		−CPU-thread-0 {preempted}
64		−CPU-thread-1 {preempted}
65		−CPU-thread-2 {}
66		−CPU-thread-3 {write to addrB which is a write to new page}
67		−DEV-thread-0 {}
68		−DEV-thread-2 {}
69		−[Time N+4] ------------------------------------------------------------------
70		−CPU-thread-0 {preempted}
71		−CPU-thread-1 {COW_step3: {mmu_notifier_invalidate_range_end(addrB)}}
72		−CPU-thread-2 {}
73		−CPU-thread-3 {}
74		−DEV-thread-0 {}
75		−DEV-thread-2 {}
76		−[Time N+5] ------------------------------------------------------------------
77		−CPU-thread-0 {preempted}
78		−CPU-thread-1 {}
79		−CPU-thread-2 {}
80		−CPU-thread-3 {}
81		−DEV-thread-0 {read addrA from old page}
82		−DEV-thread-2 {read addrB from new page}
83		−
84		−So here because at time N+2 the clear page table entry was not pair with a
85		−notification to invalidate the secondary TLB, the device see the new value for
86		−addrB before seing the new value for addrA. This break total memory ordering
87		−for the device.
88		−
89		−When changing a pte to write protect or to point to a new write protected page
90		−with same content (KSM) it is fine to delay the mmu_notifier_invalidate_range
91		−call to mmu_notifier_invalidate_range_end() outside the page table lock. This
92		−is true even if the thread doing the page table update is preempted right after
93		−releasing page table lock but before call mmu_notifier_invalidate_range_end().

Documentation/vm/numa.rstrenamed from Documentation/vm/numa+5−1

@@ -1,6 +1,10 @@
	1	+.. _numa:
	2	+
1	3	Started Nov 1999 by Kanoj Sarcar <kanoj@sgi.com>
2	4
	5	+=============
3	6	What is NUMA?
	7	+=============
4	8
5	9	This question can be answered from a couple of perspectives: the
6	10	hardware view and the Linux software view.
@@ -106,7 +110,7 @@ to improve NUMA locality using various CPU affinity command line interfaces, · 99 unmodified lines
106	110	such as taskset(1) and numactl(1), and program interfaces such as
107	111	sched_setaffinity(2). Further, one can modify the kernel's default local
108	112	allocation behavior using Linux NUMA memory policy.
109		−[see Documentation/vm/numa_memory_policy.txt.]
	113	+[see Documentation/admin-guide/mm/numa_memory_policy.rst.]
110	114
111	115	System administrators can restrict the CPUs and nodes' memories that a non-
112	116	privileged user can specify in the scheduling or NUMA commands and functions

Documentation/vm/numa_memory_policy.txt+0−452

@@ -1,452 +0,0 @@
1		−
2		−What is Linux Memory Policy?
3		−
4		−In the Linux kernel, "memory policy" determines from which node the kernel will
5		−allocate memory in a NUMA system or in an emulated NUMA system. Linux has
6		−supported platforms with Non-Uniform Memory Access architectures since 2.4.?.
7		−The current memory policy support was added to Linux 2.6 around May 2004. This
8		−document attempts to describe the concepts and APIs of the 2.6 memory policy
9		−support.
10		−
11		−Memory policies should not be confused with cpusets
12		−(Documentation/cgroup-v1/cpusets.txt)
13		−which is an administrative mechanism for restricting the nodes from which
14		−memory may be allocated by a set of processes. Memory policies are a
15		−programming interface that a NUMA-aware application can take advantage of. When
16		−both cpusets and policies are applied to a task, the restrictions of the cpuset
17		−takes priority. See "MEMORY POLICIES AND CPUSETS" below for more details.
18		−
19		−MEMORY POLICY CONCEPTS
20		−
21		−Scope of Memory Policies
22		−
23		−The Linux kernel supports _scopes_ of memory policy, described here from
24		−most general to most specific:
25		−
26		− System Default Policy: this policy is "hard coded" into the kernel. It
27		− is the policy that governs all page allocations that aren't controlled
28		− by one of the more specific policy scopes discussed below. When the
29		− system is "up and running", the system default policy will use "local
30		− allocation" described below. However, during boot up, the system
31		− default policy will be set to interleave allocations across all nodes
32		− with "sufficient" memory, so as not to overload the initial boot node
33		− with boot-time allocations.
34		−
35		− Task/Process Policy: this is an optional, per-task policy. When defined
36		− for a specific task, this policy controls all page allocations made by or
37		− on behalf of the task that aren't controlled by a more specific scope.
38		− If a task does not define a task policy, then all page allocations that
39		− would have been controlled by the task policy "fall back" to the System
40		− Default Policy.
41		−
42		− The task policy applies to the entire address space of a task. Thus,
43		− it is inheritable, and indeed is inherited, across both fork()
44		− [clone() w/o the CLONE_VM flag] and exec*(). This allows a parent task
45		− to establish the task policy for a child task exec()'d from an
46		− executable image that has no awareness of memory policy. See the
47		− MEMORY POLICY APIS section, below, for an overview of the system call
48		− that a task may use to set/change its task/process policy.
49		−
50		− In a multi-threaded task, task policies apply only to the thread
51		− [Linux kernel task] that installs the policy and any threads
52		− subsequently created by that thread. Any sibling threads existing
53		− at the time a new task policy is installed retain their current
54		− policy.
55		−
56		− A task policy applies only to pages allocated after the policy is
57		− installed. Any pages already faulted in by the task when the task
58		− changes its task policy remain where they were allocated based on
59		− the policy at the time they were allocated.
60		−
61		− VMA Policy: A "VMA" or "Virtual Memory Area" refers to a range of a task's
62		− virtual address space. A task may define a specific policy for a range
63		− of its virtual address space. See the MEMORY POLICIES APIS section,
64		− below, for an overview of the mbind() system call used to set a VMA
65		− policy.
66		−
67		− A VMA policy will govern the allocation of pages that back this region of
68		− the address space. Any regions of the task's address space that don't
69		− have an explicit VMA policy will fall back to the task policy, which may
70		− itself fall back to the System Default Policy.
71		−
72		− VMA policies have a few complicating details:
73		−
74		− VMA policy applies ONLY to anonymous pages. These include pages
75		− allocated for anonymous segments, such as the task stack and heap, and
76		− any regions of the address space mmap()ed with the MAP_ANONYMOUS flag.
77		− If a VMA policy is applied to a file mapping, it will be ignored if
78		− the mapping used the MAP_SHARED flag. If the file mapping used the
79		− MAP_PRIVATE flag, the VMA policy will only be applied when an
80		− anonymous page is allocated on an attempt to write to the mapping--
81		− i.e., at Copy-On-Write.
82		−
83		− VMA policies are shared between all tasks that share a virtual address
84		− space--a.k.a. threads--independent of when the policy is installed; and
85		− they are inherited across fork(). However, because VMA policies refer
86		− to a specific region of a task's address space, and because the address
87		− space is discarded and recreated on exec*(), VMA policies are NOT
88		− inheritable across exec(). Thus, only NUMA-aware applications may
89		− use VMA policies.
90		−
91		− A task may install a new VMA policy on a sub-range of a previously
92		− mmap()ed region. When this happens, Linux splits the existing virtual
93		− memory area into 2 or 3 VMAs, each with it's own policy.
94		−
95		− By default, VMA policy applies only to pages allocated after the policy
96		− is installed. Any pages already faulted into the VMA range remain
97		− where they were allocated based on the policy at the time they were
98		− allocated. However, since 2.6.16, Linux supports page migration via
99		− the mbind() system call, so that page contents can be moved to match
100		− a newly installed policy.
101		−
102		− Shared Policy: Conceptually, shared policies apply to "memory objects"
103		− mapped shared into one or more tasks' distinct address spaces. An
104		− application installs a shared policies the same way as VMA policies--using
105		− the mbind() system call specifying a range of virtual addresses that map
106		− the shared object. However, unlike VMA policies, which can be considered
107		− to be an attribute of a range of a task's address space, shared policies
108		− apply directly to the shared object. Thus, all tasks that attach to the
109		− object share the policy, and all pages allocated for the shared object,
110		− by any task, will obey the shared policy.
111		−
112		− As of 2.6.22, only shared memory segments, created by shmget() or
113		− mmap(MAP_ANONYMOUS\|MAP_SHARED), support shared policy. When shared
114		− policy support was added to Linux, the associated data structures were
115		− added to hugetlbfs shmem segments. At the time, hugetlbfs did not
116		− support allocation at fault time--a.k.a lazy allocation--so hugetlbfs
117		− shmem segments were never "hooked up" to the shared policy support.
118		− Although hugetlbfs segments now support lazy allocation, their support
119		− for shared policy has not been completed.
120		−
121		− As mentioned above [re: VMA policies], allocations of page cache
122		− pages for regular files mmap()ed with MAP_SHARED ignore any VMA
123		− policy installed on the virtual address range backed by the shared
124		− file mapping. Rather, shared page cache pages, including pages backing
125		− private mappings that have not yet been written by the task, follow
126		− task policy, if any, else System Default Policy.
127		−
128		− The shared policy infrastructure supports different policies on subset
129		− ranges of the shared object. However, Linux still splits the VMA of
130		− the task that installs the policy for each range of distinct policy.
131		− Thus, different tasks that attach to a shared memory segment can have
132		− different VMA configurations mapping that one shared object. This
133		− can be seen by examining the /proc/<pid>/numa_maps of tasks sharing
134		− a shared memory region, when one task has installed shared policy on
135		− one or more ranges of the region.
136		−
137		−Components of Memory Policies
138		−
139		− A Linux memory policy consists of a "mode", optional mode flags, and an
140		− optional set of nodes. The mode determines the behavior of the policy,
141		− the optional mode flags determine the behavior of the mode, and the
142		− optional set of nodes can be viewed as the arguments to the policy
143		− behavior.
144		−
145		− Internally, memory policies are implemented by a reference counted
146		− structure, struct mempolicy. Details of this structure will be discussed
147		− in context, below, as required to explain the behavior.
148		−
149		− Linux memory policy supports the following 4 behavioral modes:
150		−
151		− Default Mode--MPOL_DEFAULT: This mode is only used in the memory
152		− policy APIs. Internally, MPOL_DEFAULT is converted to the NULL
153		− memory policy in all policy scopes. Any existing non-default policy
154		− will simply be removed when MPOL_DEFAULT is specified. As a result,
155		− MPOL_DEFAULT means "fall back to the next most specific policy scope."
156		−
157		− For example, a NULL or default task policy will fall back to the
158		− system default policy. A NULL or default vma policy will fall
159		− back to the task policy.
160		−
161		− When specified in one of the memory policy APIs, the Default mode
162		− does not use the optional set of nodes.
163		−
164		− It is an error for the set of nodes specified for this policy to
165		− be non-empty.
166		−
167		− MPOL_BIND: This mode specifies that memory must come from the
168		− set of nodes specified by the policy. Memory will be allocated from
169		− the node in the set with sufficient free memory that is closest to
170		− the node where the allocation takes place.
171		−
172		− MPOL_PREFERRED: This mode specifies that the allocation should be
173		− attempted from the single node specified in the policy. If that
174		− allocation fails, the kernel will search other nodes, in order of
175		− increasing distance from the preferred node based on information
176		− provided by the platform firmware.
177		−
178		− Internally, the Preferred policy uses a single node--the
179		− preferred_node member of struct mempolicy. When the internal
180		− mode flag MPOL_F_LOCAL is set, the preferred_node is ignored and
181		− the policy is interpreted as local allocation. "Local" allocation
182		− policy can be viewed as a Preferred policy that starts at the node
183		− containing the cpu where the allocation takes place.
184		−
185		− It is possible for the user to specify that local allocation is
186		− always preferred by passing an empty nodemask with this mode.
187		− If an empty nodemask is passed, the policy cannot use the
188		− MPOL_F_STATIC_NODES or MPOL_F_RELATIVE_NODES flags described
189		− below.
190		−
191		− MPOL_INTERLEAVED: This mode specifies that page allocations be
192		− interleaved, on a page granularity, across the nodes specified in
193		− the policy. This mode also behaves slightly differently, based on
194		− the context where it is used:
195		−
196		− For allocation of anonymous pages and shared memory pages,
197		− Interleave mode indexes the set of nodes specified by the policy
198		− using the page offset of the faulting address into the segment
199		− [VMA] containing the address modulo the number of nodes specified
200		− by the policy. It then attempts to allocate a page, starting at
201		− the selected node, as if the node had been specified by a Preferred
202		− policy or had been selected by a local allocation. That is,
203		− allocation will follow the per node zonelist.
204		−
205		− For allocation of page cache pages, Interleave mode indexes the set
206		− of nodes specified by the policy using a node counter maintained
207		− per task. This counter wraps around to the lowest specified node
208		− after it reaches the highest specified node. This will tend to
209		− spread the pages out over the nodes specified by the policy based
210		− on the order in which they are allocated, rather than based on any
211		− page offset into an address range or file. During system boot up,
212		− the temporary interleaved system default policy works in this
213		− mode.
214		−
215		− Linux memory policy supports the following optional mode flags:
216		−
217		− MPOL_F_STATIC_NODES: This flag specifies that the nodemask passed by
218		− the user should not be remapped if the task or VMA's set of allowed
219		− nodes changes after the memory policy has been defined.
220		−
221		− Without this flag, anytime a mempolicy is rebound because of a
222		− change in the set of allowed nodes, the node (Preferred) or
223		− nodemask (Bind, Interleave) is remapped to the new set of
224		− allowed nodes. This may result in nodes being used that were
225		− previously undesired.
226		−
227		− With this flag, if the user-specified nodes overlap with the
228		− nodes allowed by the task's cpuset, then the memory policy is
229		− applied to their intersection. If the two sets of nodes do not
230		− overlap, the Default policy is used.
231		−
232		− For example, consider a task that is attached to a cpuset with
233		− mems 1-3 that sets an Interleave policy over the same set. If
234		− the cpuset's mems change to 3-5, the Interleave will now occur
235		− over nodes 3, 4, and 5. With this flag, however, since only node
236		− 3 is allowed from the user's nodemask, the "interleave" only
237		− occurs over that node. If no nodes from the user's nodemask are
238		− now allowed, the Default behavior is used.
239		−
240		− MPOL_F_STATIC_NODES cannot be combined with the
241		− MPOL_F_RELATIVE_NODES flag. It also cannot be used for
242		− MPOL_PREFERRED policies that were created with an empty nodemask
243		− (local allocation).
244		−
245		− MPOL_F_RELATIVE_NODES: This flag specifies that the nodemask passed
246		− by the user will be mapped relative to the set of the task or VMA's
247		− set of allowed nodes. The kernel stores the user-passed nodemask,
248		− and if the allowed nodes changes, then that original nodemask will
249		− be remapped relative to the new set of allowed nodes.
250		−
251		− Without this flag (and without MPOL_F_STATIC_NODES), anytime a
252		− mempolicy is rebound because of a change in the set of allowed
253		− nodes, the node (Preferred) or nodemask (Bind, Interleave) is
254		− remapped to the new set of allowed nodes. That remap may not
255		− preserve the relative nature of the user's passed nodemask to its
256		− set of allowed nodes upon successive rebinds: a nodemask of
257		− 1,3,5 may be remapped to 7-9 and then to 1-3 if the set of
258		− allowed nodes is restored to its original state.
259		−
260		− With this flag, the remap is done so that the node numbers from
261		− the user's passed nodemask are relative to the set of allowed
262		− nodes. In other words, if nodes 0, 2, and 4 are set in the user's
263		− nodemask, the policy will be effected over the first (and in the
264		− Bind or Interleave case, the third and fifth) nodes in the set of
265		− allowed nodes. The nodemask passed by the user represents nodes
266		− relative to task or VMA's set of allowed nodes.
267		−
268		− If the user's nodemask includes nodes that are outside the range
269		− of the new set of allowed nodes (for example, node 5 is set in
270		− the user's nodemask when the set of allowed nodes is only 0-3),
271		− then the remap wraps around to the beginning of the nodemask and,
272		− if not already set, sets the node in the mempolicy nodemask.
273		−
274		− For example, consider a task that is attached to a cpuset with
275		− mems 2-5 that sets an Interleave policy over the same set with
276		− MPOL_F_RELATIVE_NODES. If the cpuset's mems change to 3-7, the
277		− interleave now occurs over nodes 3,5-7. If the cpuset's mems
278		− then change to 0,2-3,5, then the interleave occurs over nodes
279		− 0,2-3,5.
280		−
281		− Thanks to the consistent remapping, applications preparing
282		− nodemasks to specify memory policies using this flag should
283		− disregard their current, actual cpuset imposed memory placement
284		− and prepare the nodemask as if they were always located on
285		− memory nodes 0 to N-1, where N is the number of memory nodes the
286		− policy is intended to manage. Let the kernel then remap to the
287		− set of memory nodes allowed by the task's cpuset, as that may
288		− change over time.
289		−
290		− MPOL_F_RELATIVE_NODES cannot be combined with the
291		− MPOL_F_STATIC_NODES flag. It also cannot be used for
292		− MPOL_PREFERRED policies that were created with an empty nodemask
293		− (local allocation).
294		−
295		−MEMORY POLICY REFERENCE COUNTING
296		−
297		−To resolve use/free races, struct mempolicy contains an atomic reference
298		−count field. Internal interfaces, mpol_get()/mpol_put() increment and
299		−decrement this reference count, respectively. mpol_put() will only free
300		−the structure back to the mempolicy kmem cache when the reference count
301		−goes to zero.
302		−
303		−When a new memory policy is allocated, its reference count is initialized
304		−to '1', representing the reference held by the task that is installing the
305		−new policy. When a pointer to a memory policy structure is stored in another
306		−structure, another reference is added, as the task's reference will be dropped
307		−on completion of the policy installation.
308		−
309		−During run-time "usage" of the policy, we attempt to minimize atomic operations
310		−on the reference count, as this can lead to cache lines bouncing between cpus
311		−and NUMA nodes. "Usage" here means one of the following:
312		−
313		−1) querying of the policy, either by the task itself [using the get_mempolicy()
314		− API discussed below] or by another task using the /proc/<pid>/numa_maps
315		− interface.
316		−
317		−2) examination of the policy to determine the policy mode and associated node
318		− or node lists, if any, for page allocation. This is considered a "hot
319		− path". Note that for MPOL_BIND, the "usage" extends across the entire
320		− allocation process, which may sleep during page reclaimation, because the
321		− BIND policy nodemask is used, by reference, to filter ineligible nodes.
322		−
323		−We can avoid taking an extra reference during the usages listed above as
324		−follows:
325		−
326		−1) we never need to get/free the system default policy as this is never
327		− changed nor freed, once the system is up and running.
328		−
329		−2) for querying the policy, we do not need to take an extra reference on the
330		− target task's task policy nor vma policies because we always acquire the
331		− task's mm's mmap_sem for read during the query. The set_mempolicy() and
332		− mbind() APIs [see below] always acquire the mmap_sem for write when
333		− installing or replacing task or vma policies. Thus, there is no possibility
334		− of a task or thread freeing a policy while another task or thread is
335		− querying it.
336		−
337		−3) Page allocation usage of task or vma policy occurs in the fault path where
338		− we hold them mmap_sem for read. Again, because replacing the task or vma
339		− policy requires that the mmap_sem be held for write, the policy can't be
340		− freed out from under us while we're using it for page allocation.
341		−
342		−4) Shared policies require special consideration. One task can replace a
343		− shared memory policy while another task, with a distinct mmap_sem, is
344		− querying or allocating a page based on the policy. To resolve this
345		− potential race, the shared policy infrastructure adds an extra reference
346		− to the shared policy during lookup while holding a spin lock on the shared
347		− policy management structure. This requires that we drop this extra
348		− reference when we're finished "using" the policy. We must drop the
349		− extra reference on shared policies in the same query/allocation paths
350		− used for non-shared policies. For this reason, shared policies are marked
351		− as such, and the extra reference is dropped "conditionally"--i.e., only
352		− for shared policies.
353		−
354		− Because of this extra reference counting, and because we must lookup
355		− shared policies in a tree structure under spinlock, shared policies are
356		− more expensive to use in the page allocation path. This is especially
357		− true for shared policies on shared memory regions shared by tasks running
358		− on different NUMA nodes. This extra overhead can be avoided by always
359		− falling back to task or system default policy for shared memory regions,
360		− or by prefaulting the entire shared memory region into memory and locking
361		− it down. However, this might not be appropriate for all applications.
362		−
363		−MEMORY POLICY APIs
364		−
365		−Linux supports 3 system calls for controlling memory policy. These APIS
366		−always affect only the calling task, the calling task's address space, or
367		−some shared object mapped into the calling task's address space.
368		−
369		− Note: the headers that define these APIs and the parameter data types
370		− for user space applications reside in a package that is not part of
371		− the Linux kernel. The kernel system call interfaces, with the 'sys_'
372		− prefix, are defined in <linux/syscalls.h>; the mode and flag
373		− definitions are defined in <linux/mempolicy.h>.
374		−
375		−Set [Task] Memory Policy:
376		−
377		− long set_mempolicy(int mode, const unsigned long *nmask,
378		− unsigned long maxnode);
379		−
380		− Set's the calling task's "task/process memory policy" to mode
381		− specified by the 'mode' argument and the set of nodes defined
382		− by 'nmask'. 'nmask' points to a bit mask of node ids containing
383		− at least 'maxnode' ids. Optional mode flags may be passed by
384		− combining the 'mode' argument with the flag (for example:
385		− MPOL_INTERLEAVE \| MPOL_F_STATIC_NODES).
386		−
387		− See the set_mempolicy(2) man page for more details
388		−
389		−
390		−Get [Task] Memory Policy or Related Information
391		−
392		− long get_mempolicy(int *mode,
393		− const unsigned long *nmask, unsigned long maxnode,
394		− void *addr, int flags);
395		−
396		− Queries the "task/process memory policy" of the calling task, or
397		− the policy or location of a specified virtual address, depending
398		− on the 'flags' argument.
399		−
400		− See the get_mempolicy(2) man page for more details
401		−
402		−
403		−Install VMA/Shared Policy for a Range of Task's Address Space
404		−
405		− long mbind(void *start, unsigned long len, int mode,
406		− const unsigned long *nmask, unsigned long maxnode,
407		− unsigned flags);
408		−
409		− mbind() installs the policy specified by (mode, nmask, maxnodes) as
410		− a VMA policy for the range of the calling task's address space
411		− specified by the 'start' and 'len' arguments. Additional actions
412		− may be requested via the 'flags' argument.
413		−
414		− See the mbind(2) man page for more details.
415		−
416		−MEMORY POLICY COMMAND LINE INTERFACE
417		−
418		−Although not strictly part of the Linux implementation of memory policy,
419		−a command line tool, numactl(8), exists that allows one to:
420		−
421		−+ set the task policy for a specified program via set_mempolicy(2), fork(2) and
422		− exec(2)
423		−
424		−+ set the shared policy for a shared memory segment via mbind(2)
425		−
426		−The numactl(8) tool is packaged with the run-time version of the library
427		−containing the memory policy system call wrappers. Some distributions
428		−package the headers and compile-time libraries in a separate development
429		−package.
430		−
431		−
432		−MEMORY POLICIES AND CPUSETS
433		−
434		−Memory policies work within cpusets as described above. For memory policies
435		−that require a node or set of nodes, the nodes are restricted to the set of
436		−nodes whose memories are allowed by the cpuset constraints. If the nodemask
437		−specified for the policy contains nodes that are not allowed by the cpuset and
438		−MPOL_F_RELATIVE_NODES is not used, the intersection of the set of nodes
439		−specified for the policy and the set of nodes with memory is used. If the
440		−result is the empty set, the policy is considered invalid and cannot be
441		−installed. If MPOL_F_RELATIVE_NODES is used, the policy's nodes are mapped
442		−onto and folded into the task's set of allowed nodes as previously described.
443		−
444		−The interaction of memory policies and cpusets can be problematic when tasks
445		−in two cpusets share access to a memory region, such as shared memory segments
446		−created by shmget() of mmap() with the MAP_ANONYMOUS and MAP_SHARED flags, and
447		−any of the tasks install shared policy on the region, only nodes whose
448		−memories are allowed in both cpusets may be used in the policies. Obtaining
449		−this information requires "stepping outside" the memory policy APIs to use the
450		−cpuset information and requires that one know in what cpusets other task might
451		−be attaching to the shared region. Furthermore, if the cpusets' allowed
452		−memory sets are disjoint, "local" allocation is the only valid policy.

Documentation/vm/overcommit-accounting+0−80

@@ -1,80 +0,0 @@
1		−The Linux kernel supports the following overcommit handling modes
2		−
3		−0 - Heuristic overcommit handling. Obvious overcommits of
4		− address space are refused. Used for a typical system. It
5		− ensures a seriously wild allocation fails while allowing
6		− overcommit to reduce swap usage. root is allowed to
7		− allocate slightly more memory in this mode. This is the
8		− default.
9		−
10		−1 - Always overcommit. Appropriate for some scientific
11		− applications. Classic example is code using sparse arrays
12		− and just relying on the virtual memory consisting almost
13		− entirely of zero pages.
14		−
15		−2 - Don't overcommit. The total address space commit
16		− for the system is not permitted to exceed swap + a
17		− configurable amount (default is 50%) of physical RAM.
18		− Depending on the amount you use, in most situations
19		− this means a process will not be killed while accessing
20		− pages but will receive errors on memory allocation as
21		− appropriate.
22		−
23		− Useful for applications that want to guarantee their
24		− memory allocations will be available in the future
25		− without having to initialize every page.
26		−
27		−The overcommit policy is set via the sysctl `vm.overcommit_memory'.
28		−
29		−The overcommit amount can be set via `vm.overcommit_ratio' (percentage)
30		−or `vm.overcommit_kbytes' (absolute value).
31		−
32		−The current overcommit limit and amount committed are viewable in
33		−/proc/meminfo as CommitLimit and Committed_AS respectively.
34		−
35		−Gotchas
36		−-------
37		−
38		−The C language stack growth does an implicit mremap. If you want absolute
39		−guarantees and run close to the edge you MUST mmap your stack for the
40		−largest size you think you will need. For typical stack usage this does
41		−not matter much but it's a corner case if you really really care
42		−
43		−In mode 2 the MAP_NORESERVE flag is ignored.
44		−
45		−
46		−How It Works
47		−------------
48		−
49		−The overcommit is based on the following rules
50		−
51		−For a file backed map
52		− SHARED or READ-only - 0 cost (the file is the map not swap)
53		− PRIVATE WRITABLE - size of mapping per instance
54		−
55		−For an anonymous or /dev/zero map
56		− SHARED - size of mapping
57		− PRIVATE READ-only - 0 cost (but of little use)
58		− PRIVATE WRITABLE - size of mapping per instance
59		−
60		−Additional accounting
61		− Pages made writable copies by mmap
62		− shmfs memory drawn from the same pool
63		−
64		−Status
65		−------
66		−
67		−o We account mmap memory mappings
68		−o We account mprotect changes in commit
69		−o We account mremap changes in size
70		−o We account brk
71		−o We account munmap
72		−o We report the commit status in /proc
73		−o Account and check on fork
74		−o Review stack handling/building on exec
75		−o SHMfs accounting
76		−o Implement actual limit enforcement
77		−
78		−To Do
79		−-----
80		−o Account ptrace pages (this is hard)

Documentation/vm/overcommit-accounting.rst+87−0

@@ -0,0 +1,87 @@
	1	+.. _overcommit_accounting:
	2	+
	3	+=====================
	4	+Overcommit Accounting
	5	+=====================
	6	+
	7	+The Linux kernel supports the following overcommit handling modes
	8	+
	9	+0
	10	+ Heuristic overcommit handling. Obvious overcommits of address
	11	+ space are refused. Used for a typical system. It ensures a
	12	+ seriously wild allocation fails while allowing overcommit to
	13	+ reduce swap usage. root is allowed to allocate slightly more
	14	+ memory in this mode. This is the default.
	15	+
	16	+1
	17	+ Always overcommit. Appropriate for some scientific
	18	+ applications. Classic example is code using sparse arrays and
	19	+ just relying on the virtual memory consisting almost entirely
	20	+ of zero pages.
	21	+
	22	+2
	23	+ Don't overcommit. The total address space commit for the
	24	+ system is not permitted to exceed swap + a configurable amount
	25	+ (default is 50%) of physical RAM. Depending on the amount you
	26	+ use, in most situations this means a process will not be
	27	+ killed while accessing pages but will receive errors on memory
	28	+ allocation as appropriate.
	29	+
	30	+ Useful for applications that want to guarantee their memory
	31	+ allocations will be available in the future without having to
	32	+ initialize every page.
	33	+
	34	+The overcommit policy is set via the sysctl ``vm.overcommit_memory``.
	35	+
	36	+The overcommit amount can be set via ``vm.overcommit_ratio`` (percentage)
	37	+or ``vm.overcommit_kbytes`` (absolute value).
	38	+
	39	+The current overcommit limit and amount committed are viewable in
	40	+``/proc/meminfo`` as CommitLimit and Committed_AS respectively.
	41	+
	42	+Gotchas
	43	+=======
	44	+
	45	+The C language stack growth does an implicit mremap. If you want absolute
	46	+guarantees and run close to the edge you MUST mmap your stack for the
	47	+largest size you think you will need. For typical stack usage this does
	48	+not matter much but it's a corner case if you really really care
	49	+
	50	+In mode 2 the MAP_NORESERVE flag is ignored.
	51	+
	52	+
	53	+How It Works
	54	+============
	55	+
	56	+The overcommit is based on the following rules
	57	+
	58	+For a file backed map
	59	+ \| SHARED or READ-only - 0 cost (the file is the map not swap)
	60	+ \| PRIVATE WRITABLE - size of mapping per instance
	61	+
	62	+For an anonymous or ``/dev/zero`` map
	63	+ \| SHARED - size of mapping
	64	+ \| PRIVATE READ-only - 0 cost (but of little use)
	65	+ \| PRIVATE WRITABLE - size of mapping per instance
	66	+
	67	+Additional accounting
	68	+ \| Pages made writable copies by mmap
	69	+ \| shmfs memory drawn from the same pool
	70	+
	71	+Status
	72	+======
	73	+
	74	+* We account mmap memory mappings
	75	+* We account mprotect changes in commit
	76	+* We account mremap changes in size
	77	+* We account brk
	78	+* We account munmap
	79	+* We report the commit status in /proc
	80	+* Account and check on fork
	81	+* Review stack handling/building on exec
	82	+* SHMfs accounting
	83	+* Implement actual limit enforcement
	84	+
	85	+To Do
	86	+=====
	87	+* Account ptrace pages (this is hard)

Documentation/vm/page_frags.rstrenamed from Documentation/vm/page_frags+4−1

@@ -1,5 +1,8 @@
	1	+.. _page_frags:
	2	+
	3	+==============
1	4	Page fragments
2		−--------------
	5	+==============
3	6
4	7	A page fragment is an arbitrary-length arbitrary-offset area of memory
5	8	which resides within a 0 or higher order compound page. Multiple

Documentation/vm/page_migration.rstrenamed from Documentation/vm/page_migration+77−72

@@ -1,5 +1,8 @@
	1	+.. _page_migration:
	2	+
	3	+==============
1	4	Page migration
2		−--------------
	5	+==============
3	6
4	7	Page migration allows the moving of the physical location of pages between
5	8	nodes in a numa system while the process is running. This means that the
@@ -20,7 +23,7 @@ Page migration functions are provided by the numactl package by Andi Kleen · 14 unmodified lines
20	23	(a version later than 0.9.3 is required. Get it from
21	24	ftp://oss.sgi.com/www/projects/libnuma/download/). numactl provides libnuma
22	25	which provides an interface similar to other numa functionality for page
23		−migration. cat /proc/<pid>/numa_maps allows an easy review of where the
	26	+migration. cat ``/proc/<pid>/numa_maps`` allows an easy review of where the
24	27	pages of a process are located. See also the numa_maps documentation in the
25	28	proc(5) man page.
26	29
@@ -56,8 +59,8 @@ description for those trying to use migrate_pages() from the kernel · 29 unmodified lines
56	59	(for userspace usage see the Andi Kleen's numactl package mentioned above)
57	60	and then a low level description of how the low level details work.
58	61
59		−A. In kernel use of migrate_pages()
60		−-----------------------------------
	62	+In kernel use of migrate_pages()
	63	+================================
61	64
62	65	1. Remove pages from the LRU.
63	66
@@ -78,8 +81,8 @@ A. In kernel use of migrate_pages() · 14 unmodified lines
78	81	the new page for each page that is considered for
79	82	moving.
80	83
81		−B. How migrate_pages() works
82		−----------------------------
	84	+How migrate_pages() works
	85	+=========================
83	86
84	87	migrate_pages() does several passes over its list of pages. A page is moved
85	88	if all references to a page are removable at the time. The page has
@@ -142,8 +145,8 @@ Steps: · 56 unmodified lines
142	145	20. The new page is moved to the LRU and can be scanned by the swapper
143	146	etc again.
144	147
145		−C. Non-LRU page migration
146		−-------------------------
	148	+Non-LRU page migration
	149	+======================
147	150
148	151	Although original migration aimed for reducing the latency of memory access
149	152	for NUMA, compaction who want to create high-order page is also main customer.
@@ -164,89 +167,91 @@ migration path. · 14 unmodified lines
164	167	If a driver want to make own pages movable, it should define three functions
165	168	which are function pointers of struct address_space_operations.
166	169
167		−1. bool (isolate_page) (struct page page, isolate_mode_t mode);
	170	+1. ``bool (isolate_page) (struct page page, isolate_mode_t mode);``
168	171
169		−What VM expects on isolate_page function of driver is to return true
170		−if driver isolates page successfully. On returing true, VM marks the page
171		−as PG_isolated so concurrent isolation in several CPUs skip the page
172		−for isolation. If a driver cannot isolate the page, it should return false.
	172	+ What VM expects on isolate_page function of driver is to return true
	173	+ if driver isolates page successfully. On returing true, VM marks the page
	174	+ as PG_isolated so concurrent isolation in several CPUs skip the page
	175	+ for isolation. If a driver cannot isolate the page, it should return false.
173	176
174		−Once page is successfully isolated, VM uses page.lru fields so driver
175		−shouldn't expect to preserve values in that fields.
	177	+ Once page is successfully isolated, VM uses page.lru fields so driver
	178	+ shouldn't expect to preserve values in that fields.
176	179
177		−2. int (migratepage) (struct address_space mapping,
178		− struct page newpage, struct page oldpage, enum migrate_mode);
	180	+2. ``int (migratepage) (struct address_space mapping,``
	181	+\| ``struct page newpage, struct page oldpage, enum migrate_mode);``
179	182
180		−After isolation, VM calls migratepage of driver with isolated page.
181		−The function of migratepage is to move content of the old page to new page
182		−and set up fields of struct page newpage. Keep in mind that you should
183		−indicate to the VM the oldpage is no longer movable via __ClearPageMovable()
184		−under page_lock if you migrated the oldpage successfully and returns
185		−MIGRATEPAGE_SUCCESS. If driver cannot migrate the page at the moment, driver
186		−can return -EAGAIN. On -EAGAIN, VM will retry page migration in a short time
187		−because VM interprets -EAGAIN as "temporal migration failure". On returning
188		−any error except -EAGAIN, VM will give up the page migration without retrying
189		−in this time.
	183	+ After isolation, VM calls migratepage of driver with isolated page.
	184	+ The function of migratepage is to move content of the old page to new page
	185	+ and set up fields of struct page newpage. Keep in mind that you should
	186	+ indicate to the VM the oldpage is no longer movable via __ClearPageMovable()
	187	+ under page_lock if you migrated the oldpage successfully and returns
	188	+ MIGRATEPAGE_SUCCESS. If driver cannot migrate the page at the moment, driver
	189	+ can return -EAGAIN. On -EAGAIN, VM will retry page migration in a short time
	190	+ because VM interprets -EAGAIN as "temporal migration failure". On returning
	191	+ any error except -EAGAIN, VM will give up the page migration without retrying
	192	+ in this time.
190	193
191		−Driver shouldn't touch page.lru field VM using in the functions.
	194	+ Driver shouldn't touch page.lru field VM using in the functions.
192	195
193		−3. void (putback_page)(struct page );
	196	+3. ``void (putback_page)(struct page );``
194	197
195		−If migration fails on isolated page, VM should return the isolated page
196		−to the driver so VM calls driver's putback_page with migration failed page.
197		−In this function, driver should put the isolated page back to the own data
198		−structure.
	198	+ If migration fails on isolated page, VM should return the isolated page
	199	+ to the driver so VM calls driver's putback_page with migration failed page.
	200	+ In this function, driver should put the isolated page back to the own data
	201	+ structure.
199	202
200	203	4. non-lru movable page flags
201	204
202		−There are two page flags for supporting non-lru movable page.
	205	+ There are two page flags for supporting non-lru movable page.
203	206
204		−* PG_movable
	207	+ * PG_movable
205	208
206		−Driver should use the below function to make page movable under page_lock.
	209	+ Driver should use the below function to make page movable under page_lock::
207	210
208	211	void __SetPageMovable(struct page page, struct address_space mapping)
209	212
210		−It needs argument of address_space for registering migration family functions
211		−which will be called by VM. Exactly speaking, PG_movable is not a real flag of
212		−struct page. Rather than, VM reuses page->mapping's lower bits to represent it.
	213	+ It needs argument of address_space for registering migration
	214	+ family functions which will be called by VM. Exactly speaking,
	215	+ PG_movable is not a real flag of struct page. Rather than, VM
	216	+ reuses page->mapping's lower bits to represent it.
213	217
	218	+::
214	219	#define PAGE_MAPPING_MOVABLE 0x2
215	220	page->mapping = page->mapping \| PAGE_MAPPING_MOVABLE;
216	221
217		−so driver shouldn't access page->mapping directly. Instead, driver should
218		−use page_mapping which mask off the low two bits of page->mapping under
219		−page lock so it can get right struct address_space.
220		−
221		−For testing of non-lru movable page, VM supports __PageMovable function.
222		−However, it doesn't guarantee to identify non-lru movable page because
223		−page->mapping field is unified with other variables in struct page.
224		−As well, if driver releases the page after isolation by VM, page->mapping
225		−doesn't have stable value although it has PAGE_MAPPING_MOVABLE
226		−(Look at __ClearPageMovable). But __PageMovable is cheap to catch whether
227		−page is LRU or non-lru movable once the page has been isolated. Because
228		−LRU pages never can have PAGE_MAPPING_MOVABLE in page->mapping. It is also
229		−good for just peeking to test non-lru movable pages before more expensive
230		−checking with lock_page in pfn scanning to select victim.
231		−
232		−For guaranteeing non-lru movable page, VM provides PageMovable function.
233		−Unlike __PageMovable, PageMovable functions validates page->mapping and
234		−mapping->a_ops->isolate_page under lock_page. The lock_page prevents sudden
235		−destroying of page->mapping.
236		−
237		−Driver using __SetPageMovable should clear the flag via __ClearMovablePage
238		−under page_lock before the releasing the page.
239		−
240		−* PG_isolated
241		−
242		−To prevent concurrent isolation among several CPUs, VM marks isolated page
243		−as PG_isolated under lock_page. So if a CPU encounters PG_isolated non-lru
244		−movable page, it can skip it. Driver doesn't need to manipulate the flag
245		−because VM will set/clear it automatically. Keep in mind that if driver
246		−sees PG_isolated page, it means the page have been isolated by VM so it
247		−shouldn't touch page.lru field.
248		−PG_isolated is alias with PG_reclaim flag so driver shouldn't use the flag
249		−for own purpose.
	222	+ so driver shouldn't access page->mapping directly. Instead, driver should
	223	+ use page_mapping which mask off the low two bits of page->mapping under
	224	+ page lock so it can get right struct address_space.
	225	+
	226	+ For testing of non-lru movable page, VM supports __PageMovable function.
	227	+ However, it doesn't guarantee to identify non-lru movable page because
	228	+ page->mapping field is unified with other variables in struct page.
	229	+ As well, if driver releases the page after isolation by VM, page->mapping
	230	+ doesn't have stable value although it has PAGE_MAPPING_MOVABLE
	231	+ (Look at __ClearPageMovable). But __PageMovable is cheap to catch whether
	232	+ page is LRU or non-lru movable once the page has been isolated. Because
	233	+ LRU pages never can have PAGE_MAPPING_MOVABLE in page->mapping. It is also
	234	+ good for just peeking to test non-lru movable pages before more expensive
	235	+ checking with lock_page in pfn scanning to select victim.
	236	+
	237	+ For guaranteeing non-lru movable page, VM provides PageMovable function.
	238	+ Unlike __PageMovable, PageMovable functions validates page->mapping and
	239	+ mapping->a_ops->isolate_page under lock_page. The lock_page prevents sudden
	240	+ destroying of page->mapping.
	241	+
	242	+ Driver using __SetPageMovable should clear the flag via __ClearMovablePage
	243	+ under page_lock before the releasing the page.
	244	+
	245	+ * PG_isolated
	246	+
	247	+ To prevent concurrent isolation among several CPUs, VM marks isolated page
	248	+ as PG_isolated under lock_page. So if a CPU encounters PG_isolated non-lru
	249	+ movable page, it can skip it. Driver doesn't need to manipulate the flag
	250	+ because VM will set/clear it automatically. Keep in mind that if driver
	251	+ sees PG_isolated page, it means the page have been isolated by VM so it
	252	+ shouldn't touch page.lru field.
	253	+ PG_isolated is alias with PG_reclaim flag so driver shouldn't use the flag
	254	+ for own purpose.
250	255
251	256	Christoph Lameter, May 8, 2006.
252	257	Minchan Kim, Mar 28, 2016.

Documentation/vm/page_owner.rstrenamed from Documentation/vm/page_owner.txt+21−13

@@ -1,7 +1,11 @@
	1	+.. _page_owner:
	2	+
	3	+==================================================
1	4	page owner: Tracking about who allocated each page
2		−-----------------------------------------------------------
	5	+==================================================
3	6
4		−* Introduction
	7	+Introduction
	8	+============
5	9
6	10	page owner is for the tracking about who allocated each page.
7	11	It can be used to debug memory leak or to find a memory hogger.
@@ -34,13 +38,15 @@ not affect to allocation performance, especially if the static keys jump · 26 unmodified lines
34	38	label patching functionality is available. Following is the kernel's code
35	39	size change due to this facility.
36	40
37		−- Without page owner
	41	+- Without page owner::
	42	+
38	43	text data bss dec hex filename
39		− 40662 1493 644 42799 a72f mm/page_alloc.o
	44	+ 40662 1493 644 42799 a72f mm/page_alloc.o
	45	+
	46	+- With page owner::
40	47
41		−- With page owner
42	48	text data bss dec hex filename
43		− 40892 1493 644 43029 a815 mm/page_alloc.o
	49	+ 40892 1493 644 43029 a815 mm/page_alloc.o
44	50	1427 24 8 1459 5b3 mm/page_ext.o
45	51	2722 50 0 2772 ad4 mm/page_owner.o
46	52
@@ -62,21 +68,23 @@ are catched and marked, although they are mostly allocated from struct · 15 unmodified lines
62	68	page extension feature. Anyway, after that, no page is left in
63	69	un-tracking state.
64	70
65		−* Usage
	71	+Usage
	72	+=====
	73	+
	74	+1) Build user-space helper::
66	75
67		−1) Build user-space helper
68	76	cd tools/vm
69	77	make page_owner_sort
70	78
71		−2) Enable page owner
72		− Add "page_owner=on" to boot cmdline.
	79	+2) Enable page owner: add "page_owner=on" to boot cmdline.
73	80
74	81	3) Do the job what you want to debug
75	82
76		−4) Analyze information from page owner
	83	+4) Analyze information from page owner::
	84	+
77	85	cat /sys/kernel/debug/page_owner > page_owner_full.txt
78	86	grep -v ^PFN page_owner_full.txt > page_owner.txt
79	87	./page_owner_sort page_owner.txt sorted_page_owner.txt
80	88
81		− See the result about who allocated each page
82		− in the sorted_page_owner.txt.
	89	+ See the result about who allocated each page
	90	+ in the ``sorted_page_owner.txt``.

Documentation/vm/remap_file_pages.rstrenamed from Documentation/vm/remap_file_pages.txt+6−0

@@ -1,3 +1,9 @@
	1	+.. _remap_file_pages:
	2	+
	3	+==============================
	4	+remap_file_pages() system call
	5	+==============================
	6	+
1	7	The remap_file_pages() system call is used to create a nonlinear mapping,
2	8	that is, a mapping in which the pages of the file are mapped into a
3	9	nonsequential order in memory. The advantage of using remap_file_pages()

Documentation/vm/slub.rst+361−0

@@ -0,0 +1,361 @@
	1	+.. _slub:
	2	+
	3	+==========================
	4	+Short users guide for SLUB
	5	+==========================
	6	+
	7	+The basic philosophy of SLUB is very different from SLAB. SLAB
	8	+requires rebuilding the kernel to activate debug options for all
	9	+slab caches. SLUB always includes full debugging but it is off by default.
	10	+SLUB can enable debugging only for selected slabs in order to avoid
	11	+an impact on overall system performance which may make a bug more
	12	+difficult to find.
	13	+
	14	+In order to switch debugging on one can add an option ``slub_debug``
	15	+to the kernel command line. That will enable full debugging for
	16	+all slabs.
	17	+
	18	+Typically one would then use the ``slabinfo`` command to get statistical
	19	+data and perform operation on the slabs. By default ``slabinfo`` only lists
	20	+slabs that have data in them. See "slabinfo -h" for more options when
	21	+running the command. ``slabinfo`` can be compiled with
	22	+::
	23	+
	24	+ gcc -o slabinfo tools/vm/slabinfo.c
	25	+
	26	+Some of the modes of operation of ``slabinfo`` require that slub debugging
	27	+be enabled on the command line. F.e. no tracking information will be
	28	+available without debugging on and validation can only partially
	29	+be performed if debugging was not switched on.
	30	+
	31	+Some more sophisticated uses of slub_debug:
	32	+-------------------------------------------
	33	+
	34	+Parameters may be given to ``slub_debug``. If none is specified then full
	35	+debugging is enabled. Format:
	36	+
	37	+slub_debug=<Debug-Options>
	38	+ Enable options for all slabs
	39	+slub_debug=<Debug-Options>,<slab name>
	40	+ Enable options only for select slabs
	41	+
	42	+
	43	+Possible debug options are::
	44	+
	45	+ F Sanity checks on (enables SLAB_DEBUG_CONSISTENCY_CHECKS
	46	+ Sorry SLAB legacy issues)
	47	+ Z Red zoning
	48	+ P Poisoning (object and padding)
	49	+ U User tracking (free and alloc)
	50	+ T Trace (please only use on single slabs)
	51	+ A Toggle failslab filter mark for the cache
	52	+ O Switch debugging off for caches that would have
	53	+ caused higher minimum slab orders
	54	+ - Switch all debugging off (useful if the kernel is
	55	+ configured with CONFIG_SLUB_DEBUG_ON)
	56	+
	57	+F.e. in order to boot just with sanity checks and red zoning one would specify::
	58	+
	59	+ slub_debug=FZ
	60	+
	61	+Trying to find an issue in the dentry cache? Try::
	62	+
	63	+ slub_debug=,dentry
	64	+
	65	+to only enable debugging on the dentry cache.
	66	+
	67	+Red zoning and tracking may realign the slab. We can just apply sanity checks
	68	+to the dentry cache with::
	69	+
	70	+ slub_debug=F,dentry
	71	+
	72	+Debugging options may require the minimum possible slab order to increase as
	73	+a result of storing the metadata (for example, caches with PAGE_SIZE object
	74	+sizes). This has a higher liklihood of resulting in slab allocation errors
	75	+in low memory situations or if there's high fragmentation of memory. To
	76	+switch off debugging for such caches by default, use::
	77	+
	78	+ slub_debug=O
	79	+
	80	+In case you forgot to enable debugging on the kernel command line: It is
	81	+possible to enable debugging manually when the kernel is up. Look at the
	82	+contents of::
	83	+
	84	+ /sys/kernel/slab/<slab name>/
	85	+
	86	+Look at the writable files. Writing 1 to them will enable the
	87	+corresponding debug option. All options can be set on a slab that does
	88	+not contain objects. If the slab already contains objects then sanity checks
	89	+and tracing may only be enabled. The other options may cause the realignment
	90	+of objects.
	91	+
	92	+Careful with tracing: It may spew out lots of information and never stop if
	93	+used on the wrong slab.
	94	+
	95	+Slab merging
	96	+============
	97	+
	98	+If no debug options are specified then SLUB may merge similar slabs together
	99	+in order to reduce overhead and increase cache hotness of objects.
	100	+``slabinfo -a`` displays which slabs were merged together.
	101	+
	102	+Slab validation
	103	+===============
	104	+
	105	+SLUB can validate all object if the kernel was booted with slub_debug. In
	106	+order to do so you must have the ``slabinfo`` tool. Then you can do
	107	+::
	108	+
	109	+ slabinfo -v
	110	+
	111	+which will test all objects. Output will be generated to the syslog.
	112	+
	113	+This also works in a more limited way if boot was without slab debug.
	114	+In that case ``slabinfo -v`` simply tests all reachable objects. Usually
	115	+these are in the cpu slabs and the partial slabs. Full slabs are not
	116	+tracked by SLUB in a non debug situation.
	117	+
	118	+Getting more performance
	119	+========================
	120	+
	121	+To some degree SLUB's performance is limited by the need to take the
	122	+list_lock once in a while to deal with partial slabs. That overhead is
	123	+governed by the order of the allocation for each slab. The allocations
	124	+can be influenced by kernel parameters:
	125	+
	126	+.. slub_min_objects=x (default 4)
	127	+.. slub_min_order=x (default 0)
	128	+.. slub_max_order=x (default 3 (PAGE_ALLOC_COSTLY_ORDER))
	129	+
	130	+``slub_min_objects``
	131	+ allows to specify how many objects must at least fit into one
	132	+ slab in order for the allocation order to be acceptable. In
	133	+ general slub will be able to perform this number of
	134	+ allocations on a slab without consulting centralized resources
	135	+ (list_lock) where contention may occur.
	136	+
	137	+``slub_min_order``
	138	+ specifies a minim order of slabs. A similar effect like
	139	+ ``slub_min_objects``.
	140	+
	141	+``slub_max_order``
	142	+ specified the order at which ``slub_min_objects`` should no
	143	+ longer be checked. This is useful to avoid SLUB trying to
	144	+ generate super large order pages to fit ``slub_min_objects``
	145	+ of a slab cache with large object sizes into one high order
	146	+ page. Setting command line parameter
	147	+ ``debug_guardpage_minorder=N`` (N > 0), forces setting
	148	+ ``slub_max_order`` to 0, what cause minimum possible order of
	149	+ slabs allocation.
	150	+
	151	+SLUB Debug output
	152	+=================
	153	+
	154	+Here is a sample of slub debug output::
	155	+
	156	+ ====================================================================
	157	+ BUG kmalloc-8: Redzone overwritten
	158	+ --------------------------------------------------------------------
	159	+
	160	+ INFO: 0xc90f6d28-0xc90f6d2b. First byte 0x00 instead of 0xcc
	161	+ INFO: Slab 0xc528c530 flags=0x400000c3 inuse=61 fp=0xc90f6d58
	162	+ INFO: Object 0xc90f6d20 @offset=3360 fp=0xc90f6d58
	163	+ INFO: Allocated in get_modalias+0x61/0xf5 age=53 cpu=1 pid=554
	164	+
	165	+ Bytes b4 0xc90f6d10: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ
	166	+ Object 0xc90f6d20: 31 30 31 39 2e 30 30 35 1019.005
	167	+ Redzone 0xc90f6d28: 00 cc cc cc .
	168	+ Padding 0xc90f6d50: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
	169	+
	170	+ [<c010523d>] dump_trace+0x63/0x1eb
	171	+ [<c01053df>] show_trace_log_lvl+0x1a/0x2f
	172	+ [<c010601d>] show_trace+0x12/0x14
	173	+ [<c0106035>] dump_stack+0x16/0x18
	174	+ [<c017e0fa>] object_err+0x143/0x14b
	175	+ [<c017e2cc>] check_object+0x66/0x234
	176	+ [<c017eb43>] __slab_free+0x239/0x384
	177	+ [<c017f446>] kfree+0xa6/0xc6
	178	+ [<c02e2335>] get_modalias+0xb9/0xf5
	179	+ [<c02e23b7>] dmi_dev_uevent+0x27/0x3c
	180	+ [<c027866a>] dev_uevent+0x1ad/0x1da
	181	+ [<c0205024>] kobject_uevent_env+0x20a/0x45b
	182	+ [<c020527f>] kobject_uevent+0xa/0xf
	183	+ [<c02779f1>] store_uevent+0x4f/0x58
	184	+ [<c027758e>] dev_attr_store+0x29/0x2f
	185	+ [<c01bec4f>] sysfs_write_file+0x16e/0x19c
	186	+ [<c0183ba7>] vfs_write+0xd1/0x15a
	187	+ [<c01841d7>] sys_write+0x3d/0x72
	188	+ [<c0104112>] sysenter_past_esp+0x5f/0x99
	189	+ [<b7f7b410>] 0xb7f7b410
	190	+ =======================
	191	+
	192	+ FIX kmalloc-8: Restoring Redzone 0xc90f6d28-0xc90f6d2b=0xcc
	193	+
	194	+If SLUB encounters a corrupted object (full detection requires the kernel
	195	+to be booted with slub_debug) then the following output will be dumped
	196	+into the syslog:
	197	+
	198	+1. Description of the problem encountered
	199	+
	200	+ This will be a message in the system log starting with::
	201	+
	202	+ ===============================================
	203	+ BUG <slab cache affected>: <What went wrong>
	204	+ -----------------------------------------------
	205	+
	206	+ INFO: <corruption start>-<corruption_end> <more info>
	207	+ INFO: Slab <address> <slab information>
	208	+ INFO: Object <address> <object information>
	209	+ INFO: Allocated in <kernel function> age=<jiffies since alloc> cpu=<allocated by
	210	+ cpu> pid=<pid of the process>
	211	+ INFO: Freed in <kernel function> age=<jiffies since free> cpu=<freed by cpu>
	212	+ pid=<pid of the process>
	213	+
	214	+ (Object allocation / free information is only available if SLAB_STORE_USER is
	215	+ set for the slab. slub_debug sets that option)
	216	+
	217	+2. The object contents if an object was involved.
	218	+
	219	+ Various types of lines can follow the BUG SLUB line:
	220	+
	221	+ Bytes b4 <address> : <bytes>
	222	+ Shows a few bytes before the object where the problem was detected.
	223	+ Can be useful if the corruption does not stop with the start of the
	224	+ object.
	225	+
	226	+ Object <address> : <bytes>
	227	+ The bytes of the object. If the object is inactive then the bytes
	228	+ typically contain poison values. Any non-poison value shows a
	229	+ corruption by a write after free.
	230	+
	231	+ Redzone <address> : <bytes>
	232	+ The Redzone following the object. The Redzone is used to detect
	233	+ writes after the object. All bytes should always have the same
	234	+ value. If there is any deviation then it is due to a write after
	235	+ the object boundary.
	236	+
	237	+ (Redzone information is only available if SLAB_RED_ZONE is set.
	238	+ slub_debug sets that option)
	239	+
	240	+ Padding <address> : <bytes>
	241	+ Unused data to fill up the space in order to get the next object
	242	+ properly aligned. In the debug case we make sure that there are
	243	+ at least 4 bytes of padding. This allows the detection of writes
	244	+ before the object.
	245	+
	246	+3. A stackdump
	247	+
	248	+ The stackdump describes the location where the error was detected. The cause
	249	+ of the corruption is may be more likely found by looking at the function that
	250	+ allocated or freed the object.
	251	+
	252	+4. Report on how the problem was dealt with in order to ensure the continued
	253	+ operation of the system.
	254	+
	255	+ These are messages in the system log beginning with::
	256	+
	257	+ FIX <slab cache affected>: <corrective action taken>
	258	+
	259	+ In the above sample SLUB found that the Redzone of an active object has
	260	+ been overwritten. Here a string of 8 characters was written into a slab that
	261	+ has the length of 8 characters. However, a 8 character string needs a
	262	+ terminating 0. That zero has overwritten the first byte of the Redzone field.
	263	+ After reporting the details of the issue encountered the FIX SLUB message
	264	+ tells us that SLUB has restored the Redzone to its proper value and then
	265	+ system operations continue.
	266	+
	267	+Emergency operations
	268	+====================
	269	+
	270	+Minimal debugging (sanity checks alone) can be enabled by booting with::
	271	+
	272	+ slub_debug=F
	273	+
	274	+This will be generally be enough to enable the resiliency features of slub
	275	+which will keep the system running even if a bad kernel component will
	276	+keep corrupting objects. This may be important for production systems.
	277	+Performance will be impacted by the sanity checks and there will be a
	278	+continual stream of error messages to the syslog but no additional memory
	279	+will be used (unlike full debugging).
	280	+
	281	+No guarantees. The kernel component still needs to be fixed. Performance
	282	+may be optimized further by locating the slab that experiences corruption
	283	+and enabling debugging only for that cache
	284	+
	285	+I.e.::
	286	+
	287	+ slub_debug=F,dentry
	288	+
	289	+If the corruption occurs by writing after the end of the object then it
	290	+may be advisable to enable a Redzone to avoid corrupting the beginning
	291	+of other objects::
	292	+
	293	+ slub_debug=FZ,dentry
	294	+
	295	+Extended slabinfo mode and plotting
	296	+===================================
	297	+
	298	+The ``slabinfo`` tool has a special 'extended' ('-X') mode that includes:
	299	+ - Slabcache Totals
	300	+ - Slabs sorted by size (up to -N <num> slabs, default 1)
	301	+ - Slabs sorted by loss (up to -N <num> slabs, default 1)
	302	+
	303	+Additionally, in this mode ``slabinfo`` does not dynamically scale
	304	+sizes (G/M/K) and reports everything in bytes (this functionality is
	305	+also available to other slabinfo modes via '-B' option) which makes
	306	+reporting more precise and accurate. Moreover, in some sense the `-X'
	307	+mode also simplifies the analysis of slabs' behaviour, because its
	308	+output can be plotted using the ``slabinfo-gnuplot.sh`` script. So it
	309	+pushes the analysis from looking through the numbers (tons of numbers)
	310	+to something easier -- visual analysis.
	311	+
	312	+To generate plots:
	313	+
	314	+a) collect slabinfo extended records, for example::
	315	+
	316	+ while [ 1 ]; do slabinfo -X >> FOO_STATS; sleep 1; done
	317	+
	318	+b) pass stats file(-s) to ``slabinfo-gnuplot.sh`` script::
	319	+
	320	+ slabinfo-gnuplot.sh FOO_STATS [FOO_STATS2 .. FOO_STATSN]
	321	+
	322	+ The ``slabinfo-gnuplot.sh`` script will pre-processes the collected records
	323	+ and generates 3 png files (and 3 pre-processing cache files) per STATS
	324	+ file:
	325	+ - Slabcache Totals: FOO_STATS-totals.png
	326	+ - Slabs sorted by size: FOO_STATS-slabs-by-size.png
	327	+ - Slabs sorted by loss: FOO_STATS-slabs-by-loss.png
	328	+
	329	+Another use case, when ``slabinfo-gnuplot.sh`` can be useful, is when you
	330	+need to compare slabs' behaviour "prior to" and "after" some code
	331	+modification. To help you out there, ``slabinfo-gnuplot.sh`` script
	332	+can 'merge' the `Slabcache Totals` sections from different
	333	+measurements. To visually compare N plots:
	334	+
	335	+a) Collect as many STATS1, STATS2, .. STATSN files as you need::
	336	+
	337	+ while [ 1 ]; do slabinfo -X >> STATS<X>; sleep 1; done
	338	+
	339	+b) Pre-process those STATS files::
	340	+
	341	+ slabinfo-gnuplot.sh STATS1 STATS2 .. STATSN
	342	+
	343	+c) Execute ``slabinfo-gnuplot.sh`` in '-t' mode, passing all of the
	344	+ generated pre-processed \*-totals::
	345	+
	346	+ slabinfo-gnuplot.sh -t STATS1-totals STATS2-totals .. STATSN-totals
	347	+
	348	+ This will produce a single plot (png file).
	349	+
	350	+ Plots, expectedly, can be large so some fluctuations or small spikes
	351	+ can go unnoticed. To deal with that, ``slabinfo-gnuplot.sh`` has two
	352	+ options to 'zoom-in'/'zoom-out':
	353	+
	354	+ a) ``-s %d,%d`` -- overwrites the default image width and heigh
	355	+ b) ``-r %d,%d`` -- specifies a range of samples to use (for example,
	356	+ in ``slabinfo -X >> FOO_STATS; sleep 1;`` case, using a ``-r
	357	+ 40,60`` range will plot only samples collected between 40th and
	358	+ 60th seconds).
	359	+
	360	+Christoph Lameter, May 30, 2007
	361	+Sergey Senozhatsky, October 23, 2015

Documentation/vm/slub.txt+0−342

@@ -1,342 +0,0 @@
1		−Short users guide for SLUB
2		−--------------------------
3		−
4		−The basic philosophy of SLUB is very different from SLAB. SLAB
5		−requires rebuilding the kernel to activate debug options for all
6		−slab caches. SLUB always includes full debugging but it is off by default.
7		−SLUB can enable debugging only for selected slabs in order to avoid
8		−an impact on overall system performance which may make a bug more
9		−difficult to find.
10		−
11		−In order to switch debugging on one can add an option "slub_debug"
12		−to the kernel command line. That will enable full debugging for
13		−all slabs.
14		−
15		−Typically one would then use the "slabinfo" command to get statistical
16		−data and perform operation on the slabs. By default slabinfo only lists
17		−slabs that have data in them. See "slabinfo -h" for more options when
18		−running the command. slabinfo can be compiled with
19		−
20		−gcc -o slabinfo tools/vm/slabinfo.c
21		−
22		−Some of the modes of operation of slabinfo require that slub debugging
23		−be enabled on the command line. F.e. no tracking information will be
24		−available without debugging on and validation can only partially
25		−be performed if debugging was not switched on.
26		−
27		−Some more sophisticated uses of slub_debug:
28		−-------------------------------------------
29		−
30		−Parameters may be given to slub_debug. If none is specified then full
31		−debugging is enabled. Format:
32		−
33		−slub_debug=<Debug-Options> Enable options for all slabs
34		−slub_debug=<Debug-Options>,<slab name>
35		− Enable options only for select slabs
36		−
37		−Possible debug options are
38		− F Sanity checks on (enables SLAB_DEBUG_CONSISTENCY_CHECKS
39		− Sorry SLAB legacy issues)
40		− Z Red zoning
41		− P Poisoning (object and padding)
42		− U User tracking (free and alloc)
43		− T Trace (please only use on single slabs)
44		− A Toggle failslab filter mark for the cache
45		− O Switch debugging off for caches that would have
46		− caused higher minimum slab orders
47		− - Switch all debugging off (useful if the kernel is
48		− configured with CONFIG_SLUB_DEBUG_ON)
49		−
50		−F.e. in order to boot just with sanity checks and red zoning one would specify:
51		−
52		− slub_debug=FZ
53		−
54		−Trying to find an issue in the dentry cache? Try
55		−
56		− slub_debug=,dentry
57		−
58		−to only enable debugging on the dentry cache.
59		−
60		−Red zoning and tracking may realign the slab. We can just apply sanity checks
61		−to the dentry cache with
62		−
63		− slub_debug=F,dentry
64		−
65		−Debugging options may require the minimum possible slab order to increase as
66		−a result of storing the metadata (for example, caches with PAGE_SIZE object
67		−sizes). This has a higher liklihood of resulting in slab allocation errors
68		−in low memory situations or if there's high fragmentation of memory. To
69		−switch off debugging for such caches by default, use
70		−
71		− slub_debug=O
72		−
73		−In case you forgot to enable debugging on the kernel command line: It is
74		−possible to enable debugging manually when the kernel is up. Look at the
75		−contents of:
76		−
77		−/sys/kernel/slab/<slab name>/
78		−
79		−Look at the writable files. Writing 1 to them will enable the
80		−corresponding debug option. All options can be set on a slab that does
81		−not contain objects. If the slab already contains objects then sanity checks
82		−and tracing may only be enabled. The other options may cause the realignment
83		−of objects.
84		−
85		−Careful with tracing: It may spew out lots of information and never stop if
86		−used on the wrong slab.
87		−
88		−Slab merging
89		−------------
90		−
91		−If no debug options are specified then SLUB may merge similar slabs together
92		−in order to reduce overhead and increase cache hotness of objects.
93		−slabinfo -a displays which slabs were merged together.
94		−
95		−Slab validation
96		−---------------
97		−
98		−SLUB can validate all object if the kernel was booted with slub_debug. In
99		−order to do so you must have the slabinfo tool. Then you can do
100		−
101		−slabinfo -v
102		−
103		−which will test all objects. Output will be generated to the syslog.
104		−
105		−This also works in a more limited way if boot was without slab debug.
106		−In that case slabinfo -v simply tests all reachable objects. Usually
107		−these are in the cpu slabs and the partial slabs. Full slabs are not
108		−tracked by SLUB in a non debug situation.
109		−
110		−Getting more performance
111		−------------------------
112		−
113		−To some degree SLUB's performance is limited by the need to take the
114		−list_lock once in a while to deal with partial slabs. That overhead is
115		−governed by the order of the allocation for each slab. The allocations
116		−can be influenced by kernel parameters:
117		−
118		−slub_min_objects=x (default 4)
119		−slub_min_order=x (default 0)
120		−slub_max_order=x (default 3 (PAGE_ALLOC_COSTLY_ORDER))
121		−
122		−slub_min_objects allows to specify how many objects must at least fit
123		−into one slab in order for the allocation order to be acceptable.
124		−In general slub will be able to perform this number of allocations
125		−on a slab without consulting centralized resources (list_lock) where
126		−contention may occur.
127		−
128		−slub_min_order specifies a minim order of slabs. A similar effect like
129		−slub_min_objects.
130		−
131		−slub_max_order specified the order at which slub_min_objects should no
132		−longer be checked. This is useful to avoid SLUB trying to generate
133		−super large order pages to fit slub_min_objects of a slab cache with
134		−large object sizes into one high order page. Setting command line
135		−parameter debug_guardpage_minorder=N (N > 0), forces setting
136		−slub_max_order to 0, what cause minimum possible order of slabs
137		−allocation.
138		−
139		−SLUB Debug output
140		−-----------------
141		−
142		−Here is a sample of slub debug output:
143		−
144		−====================================================================
145		−BUG kmalloc-8: Redzone overwritten
146		−--------------------------------------------------------------------
147		−
148		−INFO: 0xc90f6d28-0xc90f6d2b. First byte 0x00 instead of 0xcc
149		−INFO: Slab 0xc528c530 flags=0x400000c3 inuse=61 fp=0xc90f6d58
150		−INFO: Object 0xc90f6d20 @offset=3360 fp=0xc90f6d58
151		−INFO: Allocated in get_modalias+0x61/0xf5 age=53 cpu=1 pid=554
152		−
153		−Bytes b4 0xc90f6d10: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ
154		− Object 0xc90f6d20: 31 30 31 39 2e 30 30 35 1019.005
155		− Redzone 0xc90f6d28: 00 cc cc cc .
156		− Padding 0xc90f6d50: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
157		−
158		− [<c010523d>] dump_trace+0x63/0x1eb
159		− [<c01053df>] show_trace_log_lvl+0x1a/0x2f
160		− [<c010601d>] show_trace+0x12/0x14
161		− [<c0106035>] dump_stack+0x16/0x18
162		− [<c017e0fa>] object_err+0x143/0x14b
163		− [<c017e2cc>] check_object+0x66/0x234
164		− [<c017eb43>] __slab_free+0x239/0x384
165		− [<c017f446>] kfree+0xa6/0xc6
166		− [<c02e2335>] get_modalias+0xb9/0xf5
167		− [<c02e23b7>] dmi_dev_uevent+0x27/0x3c
168		− [<c027866a>] dev_uevent+0x1ad/0x1da
169		− [<c0205024>] kobject_uevent_env+0x20a/0x45b
170		− [<c020527f>] kobject_uevent+0xa/0xf
171		− [<c02779f1>] store_uevent+0x4f/0x58
172		− [<c027758e>] dev_attr_store+0x29/0x2f
173		− [<c01bec4f>] sysfs_write_file+0x16e/0x19c
174		− [<c0183ba7>] vfs_write+0xd1/0x15a
175		− [<c01841d7>] sys_write+0x3d/0x72
176		− [<c0104112>] sysenter_past_esp+0x5f/0x99
177		− [<b7f7b410>] 0xb7f7b410
178		− =======================
179		−
180		−FIX kmalloc-8: Restoring Redzone 0xc90f6d28-0xc90f6d2b=0xcc
181		−
182		−If SLUB encounters a corrupted object (full detection requires the kernel
183		−to be booted with slub_debug) then the following output will be dumped
184		−into the syslog:
185		−
186		−1. Description of the problem encountered
187		−
188		−This will be a message in the system log starting with
189		−
190		−===============================================
191		−BUG <slab cache affected>: <What went wrong>
192		−-----------------------------------------------
193		−
194		−INFO: <corruption start>-<corruption_end> <more info>
195		−INFO: Slab <address> <slab information>
196		−INFO: Object <address> <object information>
197		−INFO: Allocated in <kernel function> age=<jiffies since alloc> cpu=<allocated by
198		− cpu> pid=<pid of the process>
199		−INFO: Freed in <kernel function> age=<jiffies since free> cpu=<freed by cpu>
200		− pid=<pid of the process>
201		−
202		−(Object allocation / free information is only available if SLAB_STORE_USER is
203		−set for the slab. slub_debug sets that option)
204		−
205		−2. The object contents if an object was involved.
206		−
207		−Various types of lines can follow the BUG SLUB line:
208		−
209		−Bytes b4 <address> : <bytes>
210		− Shows a few bytes before the object where the problem was detected.
211		− Can be useful if the corruption does not stop with the start of the
212		− object.
213		−
214		−Object <address> : <bytes>
215		− The bytes of the object. If the object is inactive then the bytes
216		− typically contain poison values. Any non-poison value shows a
217		− corruption by a write after free.
218		−
219		−Redzone <address> : <bytes>
220		− The Redzone following the object. The Redzone is used to detect
221		− writes after the object. All bytes should always have the same
222		− value. If there is any deviation then it is due to a write after
223		− the object boundary.
224		−
225		− (Redzone information is only available if SLAB_RED_ZONE is set.
226		− slub_debug sets that option)
227		−
228		−Padding <address> : <bytes>
229		− Unused data to fill up the space in order to get the next object
230		− properly aligned. In the debug case we make sure that there are
231		− at least 4 bytes of padding. This allows the detection of writes
232		− before the object.
233		−
234		−3. A stackdump
235		−
236		−The stackdump describes the location where the error was detected. The cause
237		−of the corruption is may be more likely found by looking at the function that
238		−allocated or freed the object.
239		−
240		−4. Report on how the problem was dealt with in order to ensure the continued
241		−operation of the system.
242		−
243		−These are messages in the system log beginning with
244		−
245		−FIX <slab cache affected>: <corrective action taken>
246		−
247		−In the above sample SLUB found that the Redzone of an active object has
248		−been overwritten. Here a string of 8 characters was written into a slab that
249		−has the length of 8 characters. However, a 8 character string needs a
250		−terminating 0. That zero has overwritten the first byte of the Redzone field.
251		−After reporting the details of the issue encountered the FIX SLUB message
252		−tells us that SLUB has restored the Redzone to its proper value and then
253		−system operations continue.
254		−
255		−Emergency operations:
256		−---------------------
257		−
258		−Minimal debugging (sanity checks alone) can be enabled by booting with
259		−
260		− slub_debug=F
261		−
262		−This will be generally be enough to enable the resiliency features of slub
263		−which will keep the system running even if a bad kernel component will
264		−keep corrupting objects. This may be important for production systems.
265		−Performance will be impacted by the sanity checks and there will be a
266		−continual stream of error messages to the syslog but no additional memory
267		−will be used (unlike full debugging).
268		−
269		−No guarantees. The kernel component still needs to be fixed. Performance
270		−may be optimized further by locating the slab that experiences corruption
271		−and enabling debugging only for that cache
272		−
273		−I.e.
274		−
275		− slub_debug=F,dentry
276		−
277		−If the corruption occurs by writing after the end of the object then it
278		−may be advisable to enable a Redzone to avoid corrupting the beginning
279		−of other objects.
280		−
281		− slub_debug=FZ,dentry
282		−
283		−Extended slabinfo mode and plotting
284		−-----------------------------------
285		−
286		−The slabinfo tool has a special 'extended' ('-X') mode that includes:
287		− - Slabcache Totals
288		− - Slabs sorted by size (up to -N <num> slabs, default 1)
289		− - Slabs sorted by loss (up to -N <num> slabs, default 1)
290		−
291		−Additionally, in this mode slabinfo does not dynamically scale sizes (G/M/K)
292		−and reports everything in bytes (this functionality is also available to
293		−other slabinfo modes via '-B' option) which makes reporting more precise and
294		−accurate. Moreover, in some sense the `-X' mode also simplifies the analysis
295		−of slabs' behaviour, because its output can be plotted using the
296		−slabinfo-gnuplot.sh script. So it pushes the analysis from looking through
297		−the numbers (tons of numbers) to something easier -- visual analysis.
298		−
299		−To generate plots:
300		−a) collect slabinfo extended records, for example:
301		−
302		− while [ 1 ]; do slabinfo -X >> FOO_STATS; sleep 1; done
303		−
304		−b) pass stats file(-s) to slabinfo-gnuplot.sh script:
305		− slabinfo-gnuplot.sh FOO_STATS [FOO_STATS2 .. FOO_STATSN]
306		−
307		−The slabinfo-gnuplot.sh script will pre-processes the collected records
308		−and generates 3 png files (and 3 pre-processing cache files) per STATS
309		−file:
310		− - Slabcache Totals: FOO_STATS-totals.png
311		− - Slabs sorted by size: FOO_STATS-slabs-by-size.png
312		− - Slabs sorted by loss: FOO_STATS-slabs-by-loss.png
313		−
314		−Another use case, when slabinfo-gnuplot can be useful, is when you need
315		−to compare slabs' behaviour "prior to" and "after" some code modification.
316		−To help you out there, slabinfo-gnuplot.sh script can 'merge' the
317		−`Slabcache Totals` sections from different measurements. To visually
318		−compare N plots:
319		−
320		−a) Collect as many STATS1, STATS2, .. STATSN files as you need
321		− while [ 1 ]; do slabinfo -X >> STATS<X>; sleep 1; done
322		−
323		−b) Pre-process those STATS files
324		− slabinfo-gnuplot.sh STATS1 STATS2 .. STATSN
325		−
326		−c) Execute slabinfo-gnuplot.sh in '-t' mode, passing all of the
327		−generated pre-processed *-totals
328		− slabinfo-gnuplot.sh -t STATS1-totals STATS2-totals .. STATSN-totals
329		−
330		−This will produce a single plot (png file).
331		−
332		−Plots, expectedly, can be large so some fluctuations or small spikes
333		−can go unnoticed. To deal with that, `slabinfo-gnuplot.sh' has two
334		−options to 'zoom-in'/'zoom-out':
335		− a) -s %d,%d overwrites the default image width and heigh
336		− b) -r %d,%d specifies a range of samples to use (for example,
337		− in `slabinfo -X >> FOO_STATS; sleep 1;' case, using
338		− a "-r 40,60" range will plot only samples collected
339		− between 40th and 60th seconds).
340		−
341		−Christoph Lameter, May 30, 2007
342		−Sergey Senozhatsky, October 23, 2015

Documentation/vm/split_page_table_lock.rstrenamed from Documentation/vm/split_page_table_lock+9−3

@@ -1,3 +1,6 @@
	1	+.. _split_page_table_lock:
	2	+
	3	+=====================
1	4	Split page table lock
2	5	=====================
3	6
@@ -11,6 +14,7 @@ access to the table. At the moment we use split lock for PTE and PMD · 7 unmodified lines
11	14	tables. Access to higher level tables protected by mm->page_table_lock.
12	15
13	16	There are helpers to lock/unlock a table and other accessor functions:
	17	+
14	18	- pte_offset_map_lock()
15	19	maps pte and takes PTE table lock, returns pointer to the taken
16	20	lock;
@@ -34,12 +38,13 @@ Split page table lock for PMD tables is enabled, if it's enabled for PTE · 17 unmodified lines
34	38	tables and the architecture supports it (see below).
35	39
36	40	Hugetlb and split page table lock
37		−---------------------------------
	41	+=================================
38	42
39	43	Hugetlb can support several page sizes. We use split lock only for PMD
40	44	level, but not for PUD.
41	45
42	46	Hugetlb-specific helpers:
	47	+
43	48	- huge_pte_lock()
44	49	takes pmd split lock for PMD_SIZE page, mm->page_table_lock
45	50	otherwise;
@@ -47,7 +52,7 @@ Hugetlb-specific helpers: · 1 unmodified line
47	52	returns pointer to table lock;
48	53
49	54	Support of split page table lock by an architecture
50		−---------------------------------------------------
	55	+===================================================
51	56
52	57	There's no need in special enabling of PTE split page table lock:
53	58	everything required is done by pgtable_page_ctor() and pgtable_page_dtor(),
@@ -73,7 +78,7 @@ NOTE: pgtable_page_ctor() and pgtable_pmd_page_ctor() can fail -- it must · 19 unmodified lines
73	78	be handled properly.
74	79
75	80	page->ptl
76		−---------
	81	+=========
77	82
78	83	page->ptl is used to access split page table lock, where 'page' is struct
79	84	page of page containing the table. It shares storage with page->private
@@ -81,6 +86,7 @@ page of page containing the table. It shares storage with page->private · 1 unmodified line
81	86
82	87	To avoid increasing size of struct page and have best performance, we use a
83	88	trick:
	89	+
84	90	- if spinlock_t fits into long, we use page->ptr as spinlock, so we
85	91	can avoid indirect access and save a cache line.
86	92	- if size of spinlock_t is bigger then size of long, we use page->ptl as

Documentation/vm/swap_numa.rstrenamed from Documentation/vm/swap_numa.txt+33−22

@@ -1,5 +1,8 @@
	1	+.. _swap_numa:
	2	+
	3	+===========================================
1	4	Automatically bind swap device to numa node
2		−-------------------------------------------
	5	+===========================================
3	6
4	7	If the system has more than one swap device and swap device has the node
5	8	information, we can make use of this information to decide which swap
@@ -7,15 +10,16 @@ device to use in get_swap_pages() to get better performance. · 1 unmodified line
7	10
8	11
9	12	How to use this feature
10		−-----------------------
	13	+=======================
11	14
12	15	Swap device has priority and that decides the order of it to be used. To make
13	16	use of automatically binding, there is no need to manipulate priority settings
14	17	for swap devices. e.g. on a 2 node machine, assume 2 swap devices swapA and
15	18	swapB, with swapA attached to node 0 and swapB attached to node 1, are going
16		−to be swapped on. Simply swapping them on by doing:
17		−# swapon /dev/swapA
18		−# swapon /dev/swapB
	19	+to be swapped on. Simply swapping them on by doing::
	20	+
	21	+ # swapon /dev/swapA
	22	+ # swapon /dev/swapB
19	23
20	24	Then node 0 will use the two swap devices in the order of swapA then swapB and
21	25	node 1 will use the two swap devices in the order of swapB then swapA. Note
@@ -24,32 +28,39 @@ that the order of them being swapped on doesn't matter. · 2 unmodified lines
24	28	A more complex example on a 4 node machine. Assume 6 swap devices are going to
25	29	be swapped on: swapA and swapB are attached to node 0, swapC is attached to
26	30	node 1, swapD and swapE are attached to node 2 and swapF is attached to node3.
27		−The way to swap them on is the same as above:
28		−# swapon /dev/swapA
29		−# swapon /dev/swapB
30		−# swapon /dev/swapC
31		−# swapon /dev/swapD
32		−# swapon /dev/swapE
33		−# swapon /dev/swapF
34		−
35		−Then node 0 will use them in the order of:
36		−swapA/swapB -> swapC -> swapD -> swapE -> swapF
	31	+The way to swap them on is the same as above::
	32	+
	33	+ # swapon /dev/swapA
	34	+ # swapon /dev/swapB
	35	+ # swapon /dev/swapC
	36	+ # swapon /dev/swapD
	37	+ # swapon /dev/swapE
	38	+ # swapon /dev/swapF
	39	+
	40	+Then node 0 will use them in the order of::
	41	+
	42	+ swapA/swapB -> swapC -> swapD -> swapE -> swapF
	43	+
37	44	swapA and swapB will be used in a round robin mode before any other swap device.
38	45
39		−node 1 will use them in the order of:
40		−swapC -> swapA -> swapB -> swapD -> swapE -> swapF
	46	+node 1 will use them in the order of::
	47	+
	48	+ swapC -> swapA -> swapB -> swapD -> swapE -> swapF
	49	+
	50	+node 2 will use them in the order of::
	51	+
	52	+ swapD/swapE -> swapA -> swapB -> swapC -> swapF
41	53
42		−node 2 will use them in the order of:
43		−swapD/swapE -> swapA -> swapB -> swapC -> swapF
44	54	Similaly, swapD and swapE will be used in a round robin mode before any
45	55	other swap devices.
46	56
47		−node 3 will use them in the order of:
48		−swapF -> swapA -> swapB -> swapC -> swapD -> swapE
	57	+node 3 will use them in the order of::
	58	+
	59	+ swapF -> swapA -> swapB -> swapC -> swapD -> swapE
49	60
50	61
51	62	Implementation details
52		−----------------------
	63	+======================
53	64
54	65	The current code uses a priority based list, swap_avail_list, to decide
55	66	which swap device to use and if multiple swap devices share the same

Documentation/vm/transhuge.rst+197−0

@@ -0,0 +1,197 @@
	1	+.. _transhuge:
	2	+
	3	+============================
	4	+Transparent Hugepage Support
	5	+============================
	6	+
	7	+This document describes design principles Transparent Hugepage (THP)
	8	+Support and its interaction with other parts of the memory management.
	9	+
	10	+Design principles
	11	+=================
	12	+
	13	+- "graceful fallback": mm components which don't have transparent hugepage
	14	+ knowledge fall back to breaking huge pmd mapping into table of ptes and,
	15	+ if necessary, split a transparent hugepage. Therefore these components
	16	+ can continue working on the regular pages or regular pte mappings.
	17	+
	18	+- if a hugepage allocation fails because of memory fragmentation,
	19	+ regular pages should be gracefully allocated instead and mixed in
	20	+ the same vma without any failure or significant delay and without
	21	+ userland noticing
	22	+
	23	+- if some task quits and more hugepages become available (either
	24	+ immediately in the buddy or through the VM), guest physical memory
	25	+ backed by regular pages should be relocated on hugepages
	26	+ automatically (with khugepaged)
	27	+
	28	+- it doesn't require memory reservation and in turn it uses hugepages
	29	+ whenever possible (the only possible reservation here is kernelcore=
	30	+ to avoid unmovable pages to fragment all the memory but such a tweak
	31	+ is not specific to transparent hugepage support and it's a generic
	32	+ feature that applies to all dynamic high order allocations in the
	33	+ kernel)
	34	+
	35	+get_user_pages and follow_page
	36	+==============================
	37	+
	38	+get_user_pages and follow_page if run on a hugepage, will return the
	39	+head or tail pages as usual (exactly as they would do on
	40	+hugetlbfs). Most gup users will only care about the actual physical
	41	+address of the page and its temporary pinning to release after the I/O
	42	+is complete, so they won't ever notice the fact the page is huge. But
	43	+if any driver is going to mangle over the page structure of the tail
	44	+page (like for checking page->mapping or other bits that are relevant
	45	+for the head page and not the tail page), it should be updated to jump
	46	+to check head page instead. Taking reference on any head/tail page would
	47	+prevent page from being split by anyone.
	48	+
	49	+.. note::
	50	+ these aren't new constraints to the GUP API, and they match the
	51	+ same constrains that applies to hugetlbfs too, so any driver capable
	52	+ of handling GUP on hugetlbfs will also work fine on transparent
	53	+ hugepage backed mappings.
	54	+
	55	+In case you can't handle compound pages if they're returned by
	56	+follow_page, the FOLL_SPLIT bit can be specified as parameter to
	57	+follow_page, so that it will split the hugepages before returning
	58	+them. Migration for example passes FOLL_SPLIT as parameter to
	59	+follow_page because it's not hugepage aware and in fact it can't work
	60	+at all on hugetlbfs (but it instead works fine on transparent
	61	+hugepages thanks to FOLL_SPLIT). migration simply can't deal with
	62	+hugepages being returned (as it's not only checking the pfn of the
	63	+page and pinning it during the copy but it pretends to migrate the
	64	+memory in regular page sizes and with regular pte/pmd mappings).
	65	+
	66	+Graceful fallback
	67	+=================
	68	+
	69	+Code walking pagetables but unaware about huge pmds can simply call
	70	+split_huge_pmd(vma, pmd, addr) where the pmd is the one returned by
	71	+pmd_offset. It's trivial to make the code transparent hugepage aware
	72	+by just grepping for "pmd_offset" and adding split_huge_pmd where
	73	+missing after pmd_offset returns the pmd. Thanks to the graceful
	74	+fallback design, with a one liner change, you can avoid to write
	75	+hundred if not thousand of lines of complex code to make your code
	76	+hugepage aware.
	77	+
	78	+If you're not walking pagetables but you run into a physical hugepage
	79	+but you can't handle it natively in your code, you can split it by
	80	+calling split_huge_page(page). This is what the Linux VM does before
	81	+it tries to swapout the hugepage for example. split_huge_page() can fail
	82	+if the page is pinned and you must handle this correctly.
	83	+
	84	+Example to make mremap.c transparent hugepage aware with a one liner
	85	+change::
	86	+
	87	+ diff --git a/mm/mremap.c b/mm/mremap.c
	88	+ --- a/mm/mremap.c
	89	+ +++ b/mm/mremap.c
	90	+ @@ -41,6 +41,7 @@ static pmd_t *get_old_pmd(struct mm_stru
	91	+ return NULL;
	92	+
	93	+ pmd = pmd_offset(pud, addr);
	94	+ + split_huge_pmd(vma, pmd, addr);
	95	+ if (pmd_none_or_clear_bad(pmd))
	96	+ return NULL;
	97	+
	98	+Locking in hugepage aware code
	99	+==============================
	100	+
	101	+We want as much code as possible hugepage aware, as calling
	102	+split_huge_page() or split_huge_pmd() has a cost.
	103	+
	104	+To make pagetable walks huge pmd aware, all you need to do is to call
	105	+pmd_trans_huge() on the pmd returned by pmd_offset. You must hold the
	106	+mmap_sem in read (or write) mode to be sure an huge pmd cannot be
	107	+created from under you by khugepaged (khugepaged collapse_huge_page
	108	+takes the mmap_sem in write mode in addition to the anon_vma lock). If
	109	+pmd_trans_huge returns false, you just fallback in the old code
	110	+paths. If instead pmd_trans_huge returns true, you have to take the
	111	+page table lock (pmd_lock()) and re-run pmd_trans_huge. Taking the
	112	+page table lock will prevent the huge pmd to be converted into a
	113	+regular pmd from under you (split_huge_pmd can run in parallel to the
	114	+pagetable walk). If the second pmd_trans_huge returns false, you
	115	+should just drop the page table lock and fallback to the old code as
	116	+before. Otherwise you can proceed to process the huge pmd and the
	117	+hugepage natively. Once finished you can drop the page table lock.
	118	+
	119	+Refcounts and transparent huge pages
	120	+====================================
	121	+
	122	+Refcounting on THP is mostly consistent with refcounting on other compound
	123	+pages:
	124	+
	125	+ - get_page()/put_page() and GUP operate in head page's ->_refcount.
	126	+
	127	+ - ->_refcount in tail pages is always zero: get_page_unless_zero() never
	128	+ succeed on tail pages.
	129	+
	130	+ - map/unmap of the pages with PTE entry increment/decrement ->_mapcount
	131	+ on relevant sub-page of the compound page.
	132	+
	133	+ - map/unmap of the whole compound page accounted in compound_mapcount
	134	+ (stored in first tail page). For file huge pages, we also increment
	135	+ ->_mapcount of all sub-pages in order to have race-free detection of
	136	+ last unmap of subpages.
	137	+
	138	+PageDoubleMap() indicates that the page is possibly mapped with PTEs.
	139	+
	140	+For anonymous pages PageDoubleMap() also indicates ->_mapcount in all
	141	+subpages is offset up by one. This additional reference is required to
	142	+get race-free detection of unmap of subpages when we have them mapped with
	143	+both PMDs and PTEs.
	144	+
	145	+This is optimization required to lower overhead of per-subpage mapcount
	146	+tracking. The alternative is alter ->_mapcount in all subpages on each
	147	+map/unmap of the whole compound page.
	148	+
	149	+For anonymous pages, we set PG_double_map when a PMD of the page got split
	150	+for the first time, but still have PMD mapping. The additional references
	151	+go away with last compound_mapcount.
	152	+
	153	+File pages get PG_double_map set on first map of the page with PTE and
	154	+goes away when the page gets evicted from page cache.
	155	+
	156	+split_huge_page internally has to distribute the refcounts in the head
	157	+page to the tail pages before clearing all PG_head/tail bits from the page
	158	+structures. It can be done easily for refcounts taken by page table
	159	+entries. But we don't have enough information on how to distribute any
	160	+additional pins (i.e. from get_user_pages). split_huge_page() fails any
	161	+requests to split pinned huge page: it expects page count to be equal to
	162	+sum of mapcount of all sub-pages plus one (split_huge_page caller must
	163	+have reference for head page).
	164	+
	165	+split_huge_page uses migration entries to stabilize page->_refcount and
	166	+page->_mapcount of anonymous pages. File pages just got unmapped.
	167	+
	168	+We safe against physical memory scanners too: the only legitimate way
	169	+scanner can get reference to a page is get_page_unless_zero().
	170	+
	171	+All tail pages have zero ->_refcount until atomic_add(). This prevents the
	172	+scanner from getting a reference to the tail page up to that point. After the
	173	+atomic_add() we don't care about the ->_refcount value. We already known how
	174	+many references should be uncharged from the head page.
	175	+
	176	+For head page get_page_unless_zero() will succeed and we don't mind. It's
	177	+clear where reference should go after split: it will stay on head page.
	178	+
	179	+Note that split_huge_pmd() doesn't have any limitation on refcounting:
	180	+pmd can be split at any point and never fails.
	181	+
	182	+Partial unmap and deferred_split_huge_page()
	183	+============================================
	184	+
	185	+Unmapping part of THP (with munmap() or other way) is not going to free
	186	+memory immediately. Instead, we detect that a subpage of THP is not in use
	187	+in page_remove_rmap() and queue the THP for splitting if memory pressure
	188	+comes. Splitting will free up unused subpages.
	189	+
	190	+Splitting the page right away is not an option due to locking context in
	191	+the place where we can detect partial unmap. It's also might be
	192	+counterproductive since in many cases partial unmap happens during exit(2) if
	193	+a THP crosses a VMA boundary.
	194	+
	195	+Function deferred_split_huge_page() is used to queue page for splitting.
	196	+The splitting itself will happen when we get memory pressure via shrinker
	197	+interface.

Documentation/vm/transhuge.txt+0−527

@@ -1,527 +0,0 @@
1		−= Transparent Hugepage Support =
2		−
3		−== Objective ==
4		−
5		−Performance critical computing applications dealing with large memory
6		−working sets are already running on top of libhugetlbfs and in turn
7		−hugetlbfs. Transparent Hugepage Support is an alternative means of
8		−using huge pages for the backing of virtual memory with huge pages
9		−that supports the automatic promotion and demotion of page sizes and
10		−without the shortcomings of hugetlbfs.
11		−
12		−Currently it only works for anonymous memory mappings and tmpfs/shmem.
13		−But in the future it can expand to other filesystems.
14		−
15		−The reason applications are running faster is because of two
16		−factors. The first factor is almost completely irrelevant and it's not
17		−of significant interest because it'll also have the downside of
18		−requiring larger clear-page copy-page in page faults which is a
19		−potentially negative effect. The first factor consists in taking a
20		−single page fault for each 2M virtual region touched by userland (so
21		−reducing the enter/exit kernel frequency by a 512 times factor). This
22		−only matters the first time the memory is accessed for the lifetime of
23		−a memory mapping. The second long lasting and much more important
24		−factor will affect all subsequent accesses to the memory for the whole
25		−runtime of the application. The second factor consist of two
26		−components: 1) the TLB miss will run faster (especially with
27		−virtualization using nested pagetables but almost always also on bare
28		−metal without virtualization) and 2) a single TLB entry will be
29		−mapping a much larger amount of virtual memory in turn reducing the
30		−number of TLB misses. With virtualization and nested pagetables the
31		−TLB can be mapped of larger size only if both KVM and the Linux guest
32		−are using hugepages but a significant speedup already happens if only
33		−one of the two is using hugepages just because of the fact the TLB
34		−miss is going to run faster.
35		−
36		−== Design ==
37		−
38		−- "graceful fallback": mm components which don't have transparent hugepage
39		− knowledge fall back to breaking huge pmd mapping into table of ptes and,
40		− if necessary, split a transparent hugepage. Therefore these components
41		− can continue working on the regular pages or regular pte mappings.
42		−
43		−- if a hugepage allocation fails because of memory fragmentation,
44		− regular pages should be gracefully allocated instead and mixed in
45		− the same vma without any failure or significant delay and without
46		− userland noticing
47		−
48		−- if some task quits and more hugepages become available (either
49		− immediately in the buddy or through the VM), guest physical memory
50		− backed by regular pages should be relocated on hugepages
51		− automatically (with khugepaged)
52		−
53		−- it doesn't require memory reservation and in turn it uses hugepages
54		− whenever possible (the only possible reservation here is kernelcore=
55		− to avoid unmovable pages to fragment all the memory but such a tweak
56		− is not specific to transparent hugepage support and it's a generic
57		− feature that applies to all dynamic high order allocations in the
58		− kernel)
59		−
60		−Transparent Hugepage Support maximizes the usefulness of free memory
61		−if compared to the reservation approach of hugetlbfs by allowing all
62		−unused memory to be used as cache or other movable (or even unmovable
63		−entities). It doesn't require reservation to prevent hugepage
64		−allocation failures to be noticeable from userland. It allows paging
65		−and all other advanced VM features to be available on the
66		−hugepages. It requires no modifications for applications to take
67		−advantage of it.
68		−
69		−Applications however can be further optimized to take advantage of
70		−this feature, like for example they've been optimized before to avoid
71		−a flood of mmap system calls for every malloc(4k). Optimizing userland
72		−is by far not mandatory and khugepaged already can take care of long
73		−lived page allocations even for hugepage unaware applications that
74		−deals with large amounts of memory.
75		−
76		−In certain cases when hugepages are enabled system wide, application
77		−may end up allocating more memory resources. An application may mmap a
78		−large region but only touch 1 byte of it, in that case a 2M page might
79		−be allocated instead of a 4k page for no good. This is why it's
80		−possible to disable hugepages system-wide and to only have them inside
81		−MADV_HUGEPAGE madvise regions.
82		−
83		−Embedded systems should enable hugepages only inside madvise regions
84		−to eliminate any risk of wasting any precious byte of memory and to
85		−only run faster.
86		−
87		−Applications that gets a lot of benefit from hugepages and that don't
88		−risk to lose memory by using hugepages, should use
89		−madvise(MADV_HUGEPAGE) on their critical mmapped regions.
90		−
91		−== sysfs ==
92		−
93		−Transparent Hugepage Support for anonymous memory can be entirely disabled
94		−(mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE
95		−regions (to avoid the risk of consuming more memory resources) or enabled
96		−system wide. This can be achieved with one of:
97		−
98		−echo always >/sys/kernel/mm/transparent_hugepage/enabled
99		−echo madvise >/sys/kernel/mm/transparent_hugepage/enabled
100		−echo never >/sys/kernel/mm/transparent_hugepage/enabled
101		−
102		−It's also possible to limit defrag efforts in the VM to generate
103		−anonymous hugepages in case they're not immediately free to madvise
104		−regions or to never try to defrag memory and simply fallback to regular
105		−pages unless hugepages are immediately available. Clearly if we spend CPU
106		−time to defrag memory, we would expect to gain even more by the fact we
107		−use hugepages later instead of regular pages. This isn't always
108		−guaranteed, but it may be more likely in case the allocation is for a
109		−MADV_HUGEPAGE region.
110		−
111		−echo always >/sys/kernel/mm/transparent_hugepage/defrag
112		−echo defer >/sys/kernel/mm/transparent_hugepage/defrag
113		−echo defer+madvise >/sys/kernel/mm/transparent_hugepage/defrag
114		−echo madvise >/sys/kernel/mm/transparent_hugepage/defrag
115		−echo never >/sys/kernel/mm/transparent_hugepage/defrag
116		−
117		−"always" means that an application requesting THP will stall on allocation
118		−failure and directly reclaim pages and compact memory in an effort to
119		−allocate a THP immediately. This may be desirable for virtual machines
120		−that benefit heavily from THP use and are willing to delay the VM start
121		−to utilise them.
122		−
123		−"defer" means that an application will wake kswapd in the background
124		−to reclaim pages and wake kcompactd to compact memory so that THP is
125		−available in the near future. It's the responsibility of khugepaged
126		−to then install the THP pages later.
127		−
128		−"defer+madvise" will enter direct reclaim and compaction like "always", but
129		−only for regions that have used madvise(MADV_HUGEPAGE); all other regions
130		−will wake kswapd in the background to reclaim pages and wake kcompactd to
131		−compact memory so that THP is available in the near future.
132		−
133		−"madvise" will enter direct reclaim like "always" but only for regions
134		−that are have used madvise(MADV_HUGEPAGE). This is the default behaviour.
135		−
136		−"never" should be self-explanatory.
137		−
138		−By default kernel tries to use huge zero page on read page fault to
139		−anonymous mapping. It's possible to disable huge zero page by writing 0
140		−or enable it back by writing 1:
141		−
142		−echo 0 >/sys/kernel/mm/transparent_hugepage/use_zero_page
143		−echo 1 >/sys/kernel/mm/transparent_hugepage/use_zero_page
144		−
145		−Some userspace (such as a test program, or an optimized memory allocation
146		−library) may want to know the size (in bytes) of a transparent hugepage:
147		−
148		−cat /sys/kernel/mm/transparent_hugepage/hpage_pmd_size
149		−
150		−khugepaged will be automatically started when
151		−transparent_hugepage/enabled is set to "always" or "madvise, and it'll
152		−be automatically shutdown if it's set to "never".
153		−
154		−khugepaged runs usually at low frequency so while one may not want to
155		−invoke defrag algorithms synchronously during the page faults, it
156		−should be worth invoking defrag at least in khugepaged. However it's
157		−also possible to disable defrag in khugepaged by writing 0 or enable
158		−defrag in khugepaged by writing 1:
159		−
160		−echo 0 >/sys/kernel/mm/transparent_hugepage/khugepaged/defrag
161		−echo 1 >/sys/kernel/mm/transparent_hugepage/khugepaged/defrag
162		−
163		−You can also control how many pages khugepaged should scan at each
164		−pass:
165		−
166		−/sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan
167		−
168		−and how many milliseconds to wait in khugepaged between each pass (you
169		−can set this to 0 to run khugepaged at 100% utilization of one core):
170		−
171		−/sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs
172		−
173		−and how many milliseconds to wait in khugepaged if there's an hugepage
174		−allocation failure to throttle the next allocation attempt.
175		−
176		−/sys/kernel/mm/transparent_hugepage/khugepaged/alloc_sleep_millisecs
177		−
178		−The khugepaged progress can be seen in the number of pages collapsed:
179		−
180		−/sys/kernel/mm/transparent_hugepage/khugepaged/pages_collapsed
181		−
182		−for each pass:
183		−
184		−/sys/kernel/mm/transparent_hugepage/khugepaged/full_scans
185		−
186		−max_ptes_none specifies how many extra small pages (that are
187		−not already mapped) can be allocated when collapsing a group
188		−of small pages into one large page.
189		−
190		−/sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none
191		−
192		−A higher value leads to use additional memory for programs.
193		−A lower value leads to gain less thp performance. Value of
194		−max_ptes_none can waste cpu time very little, you can
195		−ignore it.
196		−
197		−max_ptes_swap specifies how many pages can be brought in from
198		−swap when collapsing a group of pages into a transparent huge page.
199		−
200		−/sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_swap
201		−
202		−A higher value can cause excessive swap IO and waste
203		−memory. A lower value can prevent THPs from being
204		−collapsed, resulting fewer pages being collapsed into
205		−THPs, and lower memory access performance.
206		−
207		−== Boot parameter ==
208		−
209		−You can change the sysfs boot time defaults of Transparent Hugepage
210		−Support by passing the parameter "transparent_hugepage=always" or
211		−"transparent_hugepage=madvise" or "transparent_hugepage=never"
212		−(without "") to the kernel command line.
213		−
214		−== Hugepages in tmpfs/shmem ==
215		−
216		−You can control hugepage allocation policy in tmpfs with mount option
217		−"huge=". It can have following values:
218		−
219		− - "always":
220		− Attempt to allocate huge pages every time we need a new page;
221		−
222		− - "never":
223		− Do not allocate huge pages;
224		−
225		− - "within_size":
226		− Only allocate huge page if it will be fully within i_size.
227		− Also respect fadvise()/madvise() hints;
228		−
229		− - "advise:
230		− Only allocate huge pages if requested with fadvise()/madvise();
231		−
232		−The default policy is "never".
233		−
234		−"mount -o remount,huge= /mountpoint" works fine after mount: remounting
235		−huge=never will not attempt to break up huge pages at all, just stop more
236		−from being allocated.
237		−
238		−There's also sysfs knob to control hugepage allocation policy for internal
239		−shmem mount: /sys/kernel/mm/transparent_hugepage/shmem_enabled. The mount
240		−is used for SysV SHM, memfds, shared anonymous mmaps (of /dev/zero or
241		−MAP_ANONYMOUS), GPU drivers' DRM objects, Ashmem.
242		−
243		−In addition to policies listed above, shmem_enabled allows two further
244		−values:
245		−
246		− - "deny":
247		− For use in emergencies, to force the huge option off from
248		− all mounts;
249		− - "force":
250		− Force the huge option on for all - very useful for testing;
251		−
252		−== Need of application restart ==
253		−
254		−The transparent_hugepage/enabled values and tmpfs mount option only affect
255		−future behavior. So to make them effective you need to restart any
256		−application that could have been using hugepages. This also applies to the
257		−regions registered in khugepaged.
258		−
259		−== Monitoring usage ==
260		−
261		−The number of anonymous transparent huge pages currently used by the
262		−system is available by reading the AnonHugePages field in /proc/meminfo.
263		−To identify what applications are using anonymous transparent huge pages,
264		−it is necessary to read /proc/PID/smaps and count the AnonHugePages fields
265		−for each mapping.
266		−
267		−The number of file transparent huge pages mapped to userspace is available
268		−by reading ShmemPmdMapped and ShmemHugePages fields in /proc/meminfo.
269		−To identify what applications are mapping file transparent huge pages, it
270		−is necessary to read /proc/PID/smaps and count the FileHugeMapped fields
271		−for each mapping.
272		−
273		−Note that reading the smaps file is expensive and reading it
274		−frequently will incur overhead.
275		−
276		−There are a number of counters in /proc/vmstat that may be used to
277		−monitor how successfully the system is providing huge pages for use.
278		−
279		−thp_fault_alloc is incremented every time a huge page is successfully
280		− allocated to handle a page fault. This applies to both the
281		− first time a page is faulted and for COW faults.
282		−
283		−thp_collapse_alloc is incremented by khugepaged when it has found
284		− a range of pages to collapse into one huge page and has
285		− successfully allocated a new huge page to store the data.
286		−
287		−thp_fault_fallback is incremented if a page fault fails to allocate
288		− a huge page and instead falls back to using small pages.
289		−
290		−thp_collapse_alloc_failed is incremented if khugepaged found a range
291		− of pages that should be collapsed into one huge page but failed
292		− the allocation.
293		−
294		−thp_file_alloc is incremented every time a file huge page is successfully
295		− allocated.
296		−
297		−thp_file_mapped is incremented every time a file huge page is mapped into
298		− user address space.
299		−
300		−thp_split_page is incremented every time a huge page is split into base
301		− pages. This can happen for a variety of reasons but a common
302		− reason is that a huge page is old and is being reclaimed.
303		− This action implies splitting all PMD the page mapped with.
304		−
305		−thp_split_page_failed is incremented if kernel fails to split huge
306		− page. This can happen if the page was pinned by somebody.
307		−
308		−thp_deferred_split_page is incremented when a huge page is put onto split
309		− queue. This happens when a huge page is partially unmapped and
310		− splitting it would free up some memory. Pages on split queue are
311		− going to be split under memory pressure.
312		−
313		−thp_split_pmd is incremented every time a PMD split into table of PTEs.
314		− This can happen, for instance, when application calls mprotect() or
315		− munmap() on part of huge page. It doesn't split huge page, only
316		− page table entry.
317		−
318		−thp_zero_page_alloc is incremented every time a huge zero page is
319		− successfully allocated. It includes allocations which where
320		− dropped due race with other allocation. Note, it doesn't count
321		− every map of the huge zero page, only its allocation.
322		−
323		−thp_zero_page_alloc_failed is incremented if kernel fails to allocate
324		− huge zero page and falls back to using small pages.
325		−
326		−As the system ages, allocating huge pages may be expensive as the
327		−system uses memory compaction to copy data around memory to free a
328		−huge page for use. There are some counters in /proc/vmstat to help
329		−monitor this overhead.
330		−
331		−compact_stall is incremented every time a process stalls to run
332		− memory compaction so that a huge page is free for use.
333		−
334		−compact_success is incremented if the system compacted memory and
335		− freed a huge page for use.
336		−
337		−compact_fail is incremented if the system tries to compact memory
338		− but failed.
339		−
340		−compact_pages_moved is incremented each time a page is moved. If
341		− this value is increasing rapidly, it implies that the system
342		− is copying a lot of data to satisfy the huge page allocation.
343		− It is possible that the cost of copying exceeds any savings
344		− from reduced TLB misses.
345		−
346		−compact_pagemigrate_failed is incremented when the underlying mechanism
347		− for moving a page failed.
348		−
349		−compact_blocks_moved is incremented each time memory compaction examines
350		− a huge page aligned range of pages.
351		−
352		−It is possible to establish how long the stalls were using the function
353		−tracer to record how long was spent in __alloc_pages_nodemask and
354		−using the mm_page_alloc tracepoint to identify which allocations were
355		−for huge pages.
356		−
357		−== get_user_pages and follow_page ==
358		−
359		−get_user_pages and follow_page if run on a hugepage, will return the
360		−head or tail pages as usual (exactly as they would do on
361		−hugetlbfs). Most gup users will only care about the actual physical
362		−address of the page and its temporary pinning to release after the I/O
363		−is complete, so they won't ever notice the fact the page is huge. But
364		−if any driver is going to mangle over the page structure of the tail
365		−page (like for checking page->mapping or other bits that are relevant
366		−for the head page and not the tail page), it should be updated to jump
367		−to check head page instead. Taking reference on any head/tail page would
368		−prevent page from being split by anyone.
369		−
370		−NOTE: these aren't new constraints to the GUP API, and they match the
371		−same constrains that applies to hugetlbfs too, so any driver capable
372		−of handling GUP on hugetlbfs will also work fine on transparent
373		−hugepage backed mappings.
374		−
375		−In case you can't handle compound pages if they're returned by
376		−follow_page, the FOLL_SPLIT bit can be specified as parameter to
377		−follow_page, so that it will split the hugepages before returning
378		−them. Migration for example passes FOLL_SPLIT as parameter to
379		−follow_page because it's not hugepage aware and in fact it can't work
380		−at all on hugetlbfs (but it instead works fine on transparent
381		−hugepages thanks to FOLL_SPLIT). migration simply can't deal with
382		−hugepages being returned (as it's not only checking the pfn of the
383		−page and pinning it during the copy but it pretends to migrate the
384		−memory in regular page sizes and with regular pte/pmd mappings).
385		−
386		−== Optimizing the applications ==
387		−
388		−To be guaranteed that the kernel will map a 2M page immediately in any
389		−memory region, the mmap region has to be hugepage naturally
390		−aligned. posix_memalign() can provide that guarantee.
391		−
392		−== Hugetlbfs ==
393		−
394		−You can use hugetlbfs on a kernel that has transparent hugepage
395		−support enabled just fine as always. No difference can be noted in
396		−hugetlbfs other than there will be less overall fragmentation. All
397		−usual features belonging to hugetlbfs are preserved and
398		−unaffected. libhugetlbfs will also work fine as usual.
399		−
400		−== Graceful fallback ==
401		−
402		−Code walking pagetables but unaware about huge pmds can simply call
403		−split_huge_pmd(vma, pmd, addr) where the pmd is the one returned by
404		−pmd_offset. It's trivial to make the code transparent hugepage aware
405		−by just grepping for "pmd_offset" and adding split_huge_pmd where
406		−missing after pmd_offset returns the pmd. Thanks to the graceful
407		−fallback design, with a one liner change, you can avoid to write
408		−hundred if not thousand of lines of complex code to make your code
409		−hugepage aware.
410		−
411		−If you're not walking pagetables but you run into a physical hugepage
412		−but you can't handle it natively in your code, you can split it by
413		−calling split_huge_page(page). This is what the Linux VM does before
414		−it tries to swapout the hugepage for example. split_huge_page() can fail
415		−if the page is pinned and you must handle this correctly.
416		−
417		−Example to make mremap.c transparent hugepage aware with a one liner
418		−change:
419		−
420		−diff --git a/mm/mremap.c b/mm/mremap.c
421		−--- a/mm/mremap.c
422		−+++ b/mm/mremap.c
423		−@@ -41,6 +41,7 @@ static pmd_t *get_old_pmd(struct mm_stru
424		− return NULL;
425		−
426		− pmd = pmd_offset(pud, addr);
427		−+ split_huge_pmd(vma, pmd, addr);
428		− if (pmd_none_or_clear_bad(pmd))
429		− return NULL;
430		−
431		−== Locking in hugepage aware code ==
432		−
433		−We want as much code as possible hugepage aware, as calling
434		−split_huge_page() or split_huge_pmd() has a cost.
435		−
436		−To make pagetable walks huge pmd aware, all you need to do is to call
437		−pmd_trans_huge() on the pmd returned by pmd_offset. You must hold the
438		−mmap_sem in read (or write) mode to be sure an huge pmd cannot be
439		−created from under you by khugepaged (khugepaged collapse_huge_page
440		−takes the mmap_sem in write mode in addition to the anon_vma lock). If
441		−pmd_trans_huge returns false, you just fallback in the old code
442		−paths. If instead pmd_trans_huge returns true, you have to take the
443		−page table lock (pmd_lock()) and re-run pmd_trans_huge. Taking the
444		−page table lock will prevent the huge pmd to be converted into a
445		−regular pmd from under you (split_huge_pmd can run in parallel to the
446		−pagetable walk). If the second pmd_trans_huge returns false, you
447		−should just drop the page table lock and fallback to the old code as
448		−before. Otherwise you can proceed to process the huge pmd and the
449		−hugepage natively. Once finished you can drop the page table lock.
450		−
451		−== Refcounts and transparent huge pages ==
452		−
453		−Refcounting on THP is mostly consistent with refcounting on other compound
454		−pages:
455		−
456		− - get_page()/put_page() and GUP operate in head page's ->_refcount.
457		−
458		− - ->_refcount in tail pages is always zero: get_page_unless_zero() never
459		− succeed on tail pages.
460		−
461		− - map/unmap of the pages with PTE entry increment/decrement ->_mapcount
462		− on relevant sub-page of the compound page.
463		−
464		− - map/unmap of the whole compound page accounted in compound_mapcount
465		− (stored in first tail page). For file huge pages, we also increment
466		− ->_mapcount of all sub-pages in order to have race-free detection of
467		− last unmap of subpages.
468		−
469		−PageDoubleMap() indicates that the page is possibly mapped with PTEs.
470		−
471		−For anonymous pages PageDoubleMap() also indicates ->_mapcount in all
472		−subpages is offset up by one. This additional reference is required to
473		−get race-free detection of unmap of subpages when we have them mapped with
474		−both PMDs and PTEs.
475		−
476		−This is optimization required to lower overhead of per-subpage mapcount
477		−tracking. The alternative is alter ->_mapcount in all subpages on each
478		−map/unmap of the whole compound page.
479		−
480		−For anonymous pages, we set PG_double_map when a PMD of the page got split
481		−for the first time, but still have PMD mapping. The additional references
482		−go away with last compound_mapcount.
483		−
484		−File pages get PG_double_map set on first map of the page with PTE and
485		−goes away when the page gets evicted from page cache.
486		−
487		−split_huge_page internally has to distribute the refcounts in the head
488		−page to the tail pages before clearing all PG_head/tail bits from the page
489		−structures. It can be done easily for refcounts taken by page table
490		−entries. But we don't have enough information on how to distribute any
491		−additional pins (i.e. from get_user_pages). split_huge_page() fails any
492		−requests to split pinned huge page: it expects page count to be equal to
493		−sum of mapcount of all sub-pages plus one (split_huge_page caller must
494		−have reference for head page).
495		−
496		−split_huge_page uses migration entries to stabilize page->_refcount and
497		−page->_mapcount of anonymous pages. File pages just got unmapped.
498		−
499		−We safe against physical memory scanners too: the only legitimate way
500		−scanner can get reference to a page is get_page_unless_zero().
501		−
502		−All tail pages have zero ->_refcount until atomic_add(). This prevents the
503		−scanner from getting a reference to the tail page up to that point. After the
504		−atomic_add() we don't care about the ->_refcount value. We already known how
505		−many references should be uncharged from the head page.
506		−
507		−For head page get_page_unless_zero() will succeed and we don't mind. It's
508		−clear where reference should go after split: it will stay on head page.
509		−
510		−Note that split_huge_pmd() doesn't have any limitation on refcounting:
511		−pmd can be split at any point and never fails.
512		−
513		−== Partial unmap and deferred_split_huge_page() ==
514		−
515		−Unmapping part of THP (with munmap() or other way) is not going to free
516		−memory immediately. Instead, we detect that a subpage of THP is not in use
517		−in page_remove_rmap() and queue the THP for splitting if memory pressure
518		−comes. Splitting will free up unused subpages.
519		−
520		−Splitting the page right away is not an option due to locking context in
521		−the place where we can detect partial unmap. It's also might be
522		−counterproductive since in many cases partial unmap happens during exit(2) if
523		−a THP crosses a VMA boundary.
524		−
525		−Function deferred_split_huge_page() is used to queue page for splitting.
526		−The splitting itself will happen when we get memory pressure via shrinker
527		−interface.

Documentation/vm/unevictable-lru.rstrenamed from Documentation/vm/unevictable-lru.txt+49−68

@@ -1,37 +1,13 @@
1		− ==============================
2		− UNEVICTABLE LRU INFRASTRUCTURE
3		− ==============================
4		−
5		−========
6		−CONTENTS
7		−========
8		−
9		− (*) The Unevictable LRU
10		−
11		− - The unevictable page list.
12		− - Memory control group interaction.
13		− - Marking address spaces unevictable.
14		− - Detecting Unevictable Pages.
15		− - vmscan's handling of unevictable pages.
16		−
17		− (*) mlock()'d pages.
18		−
19		− - History.
20		− - Basic management.
21		− - mlock()/mlockall() system call handling.
22		− - Filtering special vmas.
23		− - munlock()/munlockall() system call handling.
24		− - Migrating mlocked pages.
25		− - Compacting mlocked pages.
26		− - mmap(MAP_LOCKED) system call handling.
27		− - munmap()/exit()/exec() system call handling.
28		− - try_to_unmap().
29		− - try_to_munlock() reverse map scan.
30		− - Page reclaim in shrink_*_list().
	1	+.. _unevictable_lru:
31	2
	3	+==============================
	4	+Unevictable LRU Infrastructure
	5	+==============================
32	6
33		−============
34		−INTRODUCTION
	7	+.. contents:: :local:
	8	+
	9	+
	10	+Introduction
35	11	============
36	12
37	13	This document describes the Linux memory manager's "Unevictable LRU"
@@ -46,8 +22,8 @@ details - the "what does it do?" - by reading the code. One hopes that the · 8 unmodified lines
46	22	descriptions below add value by provide the answer to "why does it do that?".
47	23
48	24
49		−===================
50		−THE UNEVICTABLE LRU
	25	+
	26	+The Unevictable LRU
51	27	===================
52	28
53	29	The Unevictable LRU facility adds an additional LRU list to track unevictable
@@ -66,17 +42,17 @@ completely unresponsive. · 12 unmodified lines
66	42
67	43	The unevictable list addresses the following classes of unevictable pages:
68	44
69		− (*) Those owned by ramfs.
	45	+ * Those owned by ramfs.
70	46
71		− (*) Those mapped into SHM_LOCK'd shared memory regions.
	47	+ * Those mapped into SHM_LOCK'd shared memory regions.
72	48
73		− (*) Those mapped into VM_LOCKED [mlock()ed] VMAs.
	49	+ * Those mapped into VM_LOCKED [mlock()ed] VMAs.
74	50
75	51	The infrastructure may also be able to handle other conditions that make pages
76	52	unevictable, either by definition or by circumstance, in the future.
77	53
78	54
79		−THE UNEVICTABLE PAGE LIST
	55	+The Unevictable Page List
80	56	-------------------------
81	57
82	58	The Unevictable LRU infrastructure consists of an additional, per-zone, LRU list
@@ -118,7 +94,7 @@ the unevictable list when one task has the page isolated from the LRU and other · 35 unmodified lines
118	94	tasks are changing the "evictability" state of the page.
119	95
120	96
121		−MEMORY CONTROL GROUP INTERACTION
	97	+Memory Control Group Interaction
122	98	--------------------------------
123	99
124	100	The unevictable LRU facility interacts with the memory control group [aka
@@ -144,7 +120,9 @@ effects: · 19 unmodified lines
144	120	the control group to thrash or to OOM-kill tasks.
145	121
146	122
147		−MARKING ADDRESS SPACES UNEVICTABLE
	123	+.. _mark_addr_space_unevict:
	124	+
	125	+Marking Address Spaces Unevictable
148	126	----------------------------------
149	127
150	128	For facilities such as ramfs none of the pages attached to the address space
@@ -152,15 +130,15 @@ may be evicted. To prevent eviction of any such pages, the AS_UNEVICTABLE · 1 unmodified line
152	130	address space flag is provided, and this can be manipulated by a filesystem
153	131	using a number of wrapper functions:
154	132
155		− () void mapping_set_unevictable(struct address_space mapping);
	133	+ * ``void mapping_set_unevictable(struct address_space *mapping);``
156	134
157	135	Mark the address space as being completely unevictable.
158	136
159		− () void mapping_clear_unevictable(struct address_space mapping);
	137	+ * ``void mapping_clear_unevictable(struct address_space *mapping);``
160	138
161	139	Mark the address space as being evictable.
162	140
163		− () int mapping_unevictable(struct address_space mapping);
	141	+ * ``int mapping_unevictable(struct address_space *mapping);``
164	142
165	143	Query the address space, and return true if it is completely
166	144	unevictable.
@@ -177,12 +155,13 @@ These are currently used in two places in the kernel: · 10 unmodified lines
177	155	ensure they're in memory.
178	156
179	157
180		−DETECTING UNEVICTABLE PAGES
	158	+Detecting Unevictable Pages
181	159	---------------------------
182	160
183	161	The function page_evictable() in vmscan.c determines whether a page is
184		−evictable or not using the query function outlined above [see section "Marking
185		−address spaces unevictable"] to check the AS_UNEVICTABLE flag.
	162	+evictable or not using the query function outlined above [see section
	163	+:ref:`Marking address spaces unevictable <mark_addr_space_unevict>`]
	164	+to check the AS_UNEVICTABLE flag.
186	165
187	166	For address spaces that are so marked after being populated (as SHM regions
188	167	might be), the lock action (eg: SHM_LOCK) can be lazy, and need not populate
@@ -202,7 +181,7 @@ flag, PG_mlocked (as wrapped by PageMlocked()), which is set when a page is · 13 unmodified lines
202	181	faulted into a VM_LOCKED vma, or found in a vma being VM_LOCKED.
203	182
204	183
205		−VMSCAN'S HANDLING OF UNEVICTABLE PAGES
	184	+Vmscan's Handling of Unevictable Pages
206	185	--------------------------------------
207	186
208	187	If unevictable pages are culled in the fault path, or moved to the unevictable
@@ -233,8 +212,7 @@ extra evictabilty checks should not occur in the majority of calls to · 24 unmodified lines
233	212	putback_lru_page().
234	213
235	214
236		−=============
237		−MLOCKED PAGES
	215	+MLOCKED Pages
238	216	=============
239	217
240	218	The unevictable page list is also useful for mlock(), in addition to ramfs and
@@ -242,7 +220,7 @@ SYSV SHM. Note that mlock() is only available in CONFIG_MMU=y situations; in · 1 unmodified line
242	220	NOMMU situations, all mappings are effectively mlocked.
243	221
244	222
245		−HISTORY
	223	+History
246	224	-------
247	225
248	226	The "Unevictable mlocked Pages" infrastructure is based on work originally
@@ -263,7 +241,7 @@ replaced by walking the reverse map to determine whether any VM_LOCKED VMAs · 14 unmodified lines
263	241	mapped the page. More on this below.
264	242
265	243
266		−BASIC MANAGEMENT
	244	+Basic Management
267	245	----------------
268	246
269	247	mlocked pages - pages mapped into a VM_LOCKED VMA - are a class of unevictable
@@ -304,10 +282,10 @@ mlocked pages become unlocked and rescued from the unevictable list when: · 34 unmodified lines
304	282	(4) before a page is COW'd in a VM_LOCKED VMA.
305	283
306	284
307		−mlock()/mlockall() SYSTEM CALL HANDLING
	285	+mlock()/mlockall() System Call Handling
308	286	---------------------------------------
309	287
310		−Both [do_]mlock() and [do_]mlockall() system call handlers call mlock_fixup()
	288	+Both [do\_]mlock() and [do\_]mlockall() system call handlers call mlock_fixup()
311	289	for each VMA in the range specified by the call. In the case of mlockall(),
312	290	this is the entire active address space of the task. Note that mlock_fixup()
313	291	is used for both mlocking and munlocking a range of memory. A call to mlock()
@@ -351,7 +329,7 @@ mlock_vma_page() is unable to isolate the page from the LRU, vmscan will handle · 37 unmodified lines
351	329	it later if and when it attempts to reclaim the page.
352	330
353	331
354		−FILTERING SPECIAL VMAS
	332	+Filtering Special VMAs
355	333	----------------------
356	334
357	335	mlock_fixup() filters several classes of "special" VMAs:
@@ -379,8 +357,9 @@ VM_LOCKED flag. Therefore, we won't have to deal with them later during · 21 unmodified lines
379	357	munlock(), munmap() or task exit. Neither does mlock_fixup() account these
380	358	VMAs against the task's "locked_vm".
381	359
	360	+.. _munlock_munlockall_handling:
382	361
383		−munlock()/munlockall() SYSTEM CALL HANDLING
	362	+munlock()/munlockall() System Call Handling
384	363	-------------------------------------------
385	364
386	365	The munlock() and munlockall() system calls are handled by the same functions -
@@ -426,7 +405,7 @@ This is fine, because we'll catch it later if and if vmscan tries to reclaim · 39 unmodified lines
426	405	the page. This should be relatively rare.
427	406
428	407
429		−MIGRATING MLOCKED PAGES
	408	+Migrating MLOCKED Pages
430	409	-----------------------
431	410
432	411	A page that is being migrated has been isolated from the LRU lists and is held
@@ -451,7 +430,7 @@ list because of a race between munlock and migration, page migration uses the · 18 unmodified lines
451	430	putback_lru_page() function to add migrated pages back to the LRU.
452	431
453	432
454		−COMPACTING MLOCKED PAGES
	433	+Compacting MLOCKED Pages
455	434	------------------------
456	435
457	436	The unevictable LRU can be scanned for compactable regions and the default
@@ -461,7 +440,7 @@ unevictable LRU is enabled, the work of compaction is mostly handled by · 3 unmodified lines
461	440	the page migration code and the same work flow as described in MIGRATING
462	441	MLOCKED PAGES will apply.
463	442
464		−MLOCKING TRANSPARENT HUGE PAGES
	443	+MLOCKING Transparent Huge Pages
465	444	-------------------------------
466	445
467	446	A transparent huge page is represented by a single entry on an LRU list.
@@ -483,7 +462,7 @@ to unevictable LRU and the rest can be reclaimed. · 15 unmodified lines
483	462
484	463	See also comment in follow_trans_huge_pmd().
485	464
486		−mmap(MAP_LOCKED) SYSTEM CALL HANDLING
	465	+mmap(MAP_LOCKED) System Call Handling
487	466	-------------------------------------
488	467
489	468	In addition the mlock()/mlockall() system calls, an application can request
@@ -514,7 +493,7 @@ memory range accounted as locked_vm, as the protections could be changed later · 24 unmodified lines
514	493	and pages allocated into that region.
515	494
516	495
517		−munmap()/exit()/exec() SYSTEM CALL HANDLING
	496	+munmap()/exit()/exec() System Call Handling
518	497	-------------------------------------------
519	498
520	499	When unmapping an mlocked region of memory, whether by an explicit call to
@@ -568,16 +547,18 @@ munlock or munmap system calls, mm teardown (munlock_vma_pages_all), reclaim, · 47 unmodified lines
568	547	holepunching, and truncation of file pages and their anonymous COWed pages.
569	548
570	549
571		−try_to_munlock() REVERSE MAP SCAN
	550	+try_to_munlock() Reverse Map Scan
572	551	---------------------------------
573	552
574		− [!] TODO/FIXME: a better name might be page_mlocked() - analogous to the
575		− page_referenced() reverse map walker.
	553	+.. warning::
	554	+ [!] TODO/FIXME: a better name might be page_mlocked() - analogous to the
	555	+ page_referenced() reverse map walker.
576	556
577		−When munlock_vma_page() [see section "munlock()/munlockall() System Call
578		−Handling" above] tries to munlock a page, it needs to determine whether or not
579		−the page is mapped by any VM_LOCKED VMA without actually attempting to unmap
580		−all PTEs from the page. For this purpose, the unevictable/mlock infrastructure
	557	+When munlock_vma_page() [see section :ref:`munlock()/munlockall() System Call
	558	+Handling <munlock_munlockall_handling>` above] tries to munlock a
	559	+page, it needs to determine whether or not the page is mapped by any
	560	+VM_LOCKED VMA without actually attempting to unmap all PTEs from the
	561	+page. For this purpose, the unevictable/mlock infrastructure
581	562	introduced a variant of try_to_unmap() called try_to_munlock().
582	563
583	564	try_to_munlock() calls the same functions as try_to_unmap() for anonymous and
@@ -595,7 +576,7 @@ large region or tearing down a large address space that has been mlocked via · 11 unmodified lines
595	576	mlockall(), overall this is a fairly rare event.
596	577
597	578
598		−PAGE RECLAIM IN shrink_*_list()
	579	+Page Reclaim in shrink_*_list()
599	580	-------------------------------
600	581
601	582	shrink_active_list() culls any obviously unevictable pages - i.e.

Documentation/vm/z3fold.rstrenamed from Documentation/vm/z3fold.txt+5−1

@@ -1,5 +1,8 @@
	1	+.. _z3fold:
	2	+
	3	+======
1	4	z3fold
2		−------
	5	+======
3	6
4	7	z3fold is a special purpose allocator for storing compressed pages.
5	8	It is designed to store up to three compressed pages per physical page.
@@ -7,6 +10,7 @@ It is a zbud derivative which allows for higher compression · 1 unmodified line
7	10	ratio keeping the simplicity and determinism of its predecessor.
8	11
9	12	The main differences between z3fold and zbud are:
	13	+
10	14	* unlike zbud, z3fold allows for up to PAGE_SIZE allocations
11	15	* z3fold can hold up to 3 compressed pages in its page
12	16	* z3fold doesn't export any API itself and is thus intended to be used

Documentation/vm/zsmalloc.rstrenamed from Documentation/vm/zsmalloc.txt+36−24

@@ -1,5 +1,8 @@
	1	+.. _zsmalloc:
	2	+
	3	+========
1	4	zsmalloc
2		−--------
	5	+========
3	6
4	7	This allocator is designed for use with zram. Thus, the allocator is
5	8	supposed to work well under low memory conditions. In particular, it
@@ -31,40 +34,49 @@ be mapped using zs_map_object() to get a usable pointer and subsequently · 25 unmodified lines
31	34	unmapped using zs_unmap_object().
32	35
33	36	stat
34		−----
	37	+====
35	38
36	39	With CONFIG_ZSMALLOC_STAT, we could see zsmalloc internal information via
37		−/sys/kernel/debug/zsmalloc/<user name>. Here is a sample of stat output:
	40	+``/sys/kernel/debug/zsmalloc/<user name>``. Here is a sample of stat output::
38	41
39		−# cat /sys/kernel/debug/zsmalloc/zram0/classes
	42	+ # cat /sys/kernel/debug/zsmalloc/zram0/classes
40	43
41	44	class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage
42		− ..
43		− ..
	45	+ ...
	46	+ ...
44	47	9 176 0 1 186 129 8 4
45	48	10 192 1 0 2880 2872 135 3
46	49	11 208 0 1 819 795 42 2
47	50	12 224 0 1 219 159 12 4
48		− ..
49		− ..
	51	+ ...
	52	+ ...
	53	+
50	54
	55	+class
	56	+ index
	57	+size
	58	+ object size zspage stores
	59	+almost_empty
	60	+ the number of ZS_ALMOST_EMPTY zspages(see below)
	61	+almost_full
	62	+ the number of ZS_ALMOST_FULL zspages(see below)
	63	+obj_allocated
	64	+ the number of objects allocated
	65	+obj_used
	66	+ the number of objects allocated to the user
	67	+pages_used
	68	+ the number of pages allocated for the class
	69	+pages_per_zspage
	70	+ the number of 0-order pages to make a zspage
51	71
52		−class: index
53		−size: object size zspage stores
54		−almost_empty: the number of ZS_ALMOST_EMPTY zspages(see below)
55		−almost_full: the number of ZS_ALMOST_FULL zspages(see below)
56		−obj_allocated: the number of objects allocated
57		−obj_used: the number of objects allocated to the user
58		−pages_used: the number of pages allocated for the class
59		−pages_per_zspage: the number of 0-order pages to make a zspage
	72	+We assign a zspage to ZS_ALMOST_EMPTY fullness group when n <= N / f, where
60	73
61		−We assign a zspage to ZS_ALMOST_EMPTY fullness group when:
62		− n <= N / f, where
63		−n = number of allocated objects
64		−N = total number of objects zspage can store
65		−f = fullness_threshold_frac(ie, 4 at the moment)
	74	+* n = number of allocated objects
	75	+* N = total number of objects zspage can store
	76	+* f = fullness_threshold_frac(ie, 4 at the moment)
66	77
67	78	Similarly, we assign zspage to:
68		− ZS_ALMOST_FULL when n > N / f
69		− ZS_EMPTY when n == 0
70		− ZS_FULL when n == N
	79	+
	80	+* ZS_ALMOST_FULL when n > N / f
	81	+* ZS_EMPTY when n == 0
	82	+* ZS_FULL when n == N

Documentation/vm/zswap.rstrenamed from Documentation/vm/zswap.txt+42−29

@@ -1,4 +1,11 @@
1		−Overview:
	1	+.. _zswap:
	2	+
	3	+=====
	4	+zswap
	5	+=====
	6	+
	7	+Overview
	8	+========
2	9
3	10	Zswap is a lightweight compressed cache for swap pages. It takes pages that are
4	11	in the process of being swapped out and attempts to compress them into a
@@ -7,32 +14,34 @@ for potentially reduced swap I/O. This trade-off can also result in a · 2 unmodified lines
7	14	significant performance improvement if reads from the compressed cache are
8	15	faster than reads from a swap device.
9	16
10		−NOTE: Zswap is a new feature as of v3.11 and interacts heavily with memory
11		−reclaim. This interaction has not been fully explored on the large set of
12		−potential configurations and workloads that exist. For this reason, zswap
13		−is a work in progress and should be considered experimental.
	17	+.. note::
	18	+ Zswap is a new feature as of v3.11 and interacts heavily with memory
	19	+ reclaim. This interaction has not been fully explored on the large set of
	20	+ potential configurations and workloads that exist. For this reason, zswap
	21	+ is a work in progress and should be considered experimental.
	22	+
	23	+ Some potential benefits:
14	24
15		−Some potential benefits:
16	25	* Desktop/laptop users with limited RAM capacities can mitigate the
17		− performance impact of swapping.
	26	+ performance impact of swapping.
18	27	* Overcommitted guests that share a common I/O resource can
19		− dramatically reduce their swap I/O pressure, avoiding heavy handed I/O
20		− throttling by the hypervisor. This allows more work to get done with less
21		− impact to the guest workload and guests sharing the I/O subsystem
	28	+ dramatically reduce their swap I/O pressure, avoiding heavy handed I/O
	29	+ throttling by the hypervisor. This allows more work to get done with less
	30	+ impact to the guest workload and guests sharing the I/O subsystem
22	31	* Users with SSDs as swap devices can extend the life of the device by
23		− drastically reducing life-shortening writes.
	32	+ drastically reducing life-shortening writes.
24	33
25	34	Zswap evicts pages from compressed cache on an LRU basis to the backing swap
26	35	device when the compressed pool reaches its size limit. This requirement had
27	36	been identified in prior community discussions.
28	37
29	38	Zswap is disabled by default but can be enabled at boot time by setting
30		−the "enabled" attribute to 1 at boot time. ie: zswap.enabled=1. Zswap
	39	+the ``enabled`` attribute to 1 at boot time. ie: ``zswap.enabled=1``. Zswap
31	40	can also be enabled and disabled at runtime using the sysfs interface.
32	41	An example command to enable zswap at runtime, assuming sysfs is mounted
33		−at /sys, is:
	42	+at ``/sys``, is::
34	43
35		−echo 1 > /sys/module/zswap/parameters/enabled
	44	+ echo 1 > /sys/module/zswap/parameters/enabled
36	45
37	46	When zswap is disabled at runtime it will stop storing pages that are
38	47	being swapped out. However, it will _not_ immediately write out or fault
@@ -43,7 +52,8 @@ pages out of the compressed pool, a swapoff on the swap device(s) will · 4 unmodified lines
43	52	fault back into memory all swapped out pages, including those in the
44	53	compressed pool.
45	54
46		−Design:
	55	+Design
	56	+======
47	57
48	58	Zswap receives pages for compression through the Frontswap API and is able to
49	59	evict pages from its own compressed pool on an LRU basis and write them back to
@@ -53,12 +63,12 @@ Zswap makes use of zpool for the managing the compressed memory pool. Each · 3 unmodified lines
53	63	allocation in zpool is not directly accessible by address. Rather, a handle is
54	64	returned by the allocation routine and that handle must be mapped before being
55	65	accessed. The compressed memory pool grows on demand and shrinks as compressed
56		−pages are freed. The pool is not preallocated. By default, a zpool of type
57		−zbud is created, but it can be selected at boot time by setting the "zpool"
58		−attribute, e.g. zswap.zpool=zbud. It can also be changed at runtime using the
59		−sysfs "zpool" attribute, e.g.
	66	+pages are freed. The pool is not preallocated. By default, a zpool
	67	+of type zbud is created, but it can be selected at boot time by
	68	+setting the ``zpool`` attribute, e.g. ``zswap.zpool=zbud``. It can
	69	+also be changed at runtime using the sysfs ``zpool`` attribute, e.g.::
60	70
61		−echo zbud > /sys/module/zswap/parameters/zpool
	71	+ echo zbud > /sys/module/zswap/parameters/zpool
62	72
63	73	The zbud type zpool allocates exactly 1 page to store 2 compressed pages, which
64	74	means the compression ratio will always be 2:1 or worse (because of half-full
@@ -83,14 +93,16 @@ via frontswap, to free the compressed entry. · 18 unmodified lines
83	93
84	94	Zswap seeks to be simple in its policies. Sysfs attributes allow for one user
85	95	controlled policy:
	96	+
86	97	* max_pool_percent - The maximum percentage of memory that the compressed
87		− pool can occupy.
	98	+ pool can occupy.
88	99
89		−The default compressor is lzo, but it can be selected at boot time by setting
90		−the “compressor” attribute, e.g. zswap.compressor=lzo. It can also be changed
91		−at runtime using the sysfs "compressor" attribute, e.g.
	100	+The default compressor is lzo, but it can be selected at boot time by
	101	+setting the ``compressor`` attribute, e.g. ``zswap.compressor=lzo``.
	102	+It can also be changed at runtime using the sysfs "compressor"
	103	+attribute, e.g.::
92	104
93		−echo lzo > /sys/module/zswap/parameters/compressor
	105	+ echo lzo > /sys/module/zswap/parameters/compressor
94	106
95	107	When the zpool and/or compressor parameter is changed at runtime, any existing
96	108	compressed pages are not modified; they are left in their own zpool. When a
@@ -106,11 +118,12 @@ compressed length of the page is set to zero and the pattern or same-filled · 9 unmodified lines
106	118	value is stored.
107	119
108	120	Same-value filled pages identification feature is enabled by default and can be
109		−disabled at boot time by setting the "same_filled_pages_enabled" attribute to 0,
110		−e.g. zswap.same_filled_pages_enabled=0. It can also be enabled and disabled at
111		−runtime using the sysfs "same_filled_pages_enabled" attribute, e.g.
	121	+disabled at boot time by setting the ``same_filled_pages_enabled`` attribute
	122	+to 0, e.g. ``zswap.same_filled_pages_enabled=0``. It can also be enabled and
	123	+disabled at runtime using the sysfs ``same_filled_pages_enabled``
	124	+attribute, e.g.::
112	125
113		−echo 1 > /sys/module/zswap/parameters/same_filled_pages_enabled
	126	+ echo 1 > /sys/module/zswap/parameters/same_filled_pages_enabled
114	127
115	128	When zswap same-filled page identification is disabled at runtime, it will stop
116	129	checking for the same-value filled pages during store operation. However, the

LICENSES/exceptions/Linux-syscall-note+1−1

@@ -1,6 +1,6 @@
1	1	SPDX-Exception-Identifier: Linux-syscall-note
2	2	SPDX-URL: https://spdx.org/licenses/Linux-syscall-note.html
3		−SPDX-Licenses: GPL-2.0, GPL-2.0+, GPL-1.0+, LGPL-2.0, LGPL-2.0+, LGPL-2.1, LGPL-2.1+
	3	+SPDX-Licenses: GPL-2.0, GPL-2.0+, GPL-1.0+, LGPL-2.0, LGPL-2.0+, LGPL-2.1, LGPL-2.1+, GPL-2.0-only, GPL-2.0-or-later
4	4	Usage-Guide:
5	5	This exception is used together with one of the above SPDX-Licenses
6	6	to mark user space API (uapi) header files so they can be included

LICENSES/other/Apache-2.0+183−0

@@ -0,0 +1,183 @@
	1	+Valid-License-Identifier: Apache-2.0
	2	+SPDX-URL: https://spdx.org/licenses/Apache-2.0.html
	3	+Usage-Guide:
	4	+ To use the Apache License version 2.0 put the following SPDX tag/value
	5	+ pair into a comment according to the placement guidelines in the
	6	+ licensing rules documentation:
	7	+ SPDX-License-Identifier: Apache-2.0
	8	+License-Text:
	9	+
	10	+Apache License
	11	+
	12	+Version 2.0, January 2004
	13	+
	14	+http://www.apache.org/licenses/
	15	+
	16	+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
	17	+
	18	+1. Definitions.
	19	+
	20	+"License" shall mean the terms and conditions for use, reproduction, and
	21	+distribution as defined by Sections 1 through 9 of this document.
	22	+
	23	+"Licensor" shall mean the copyright owner or entity authorized by the
	24	+copyright owner that is granting the License.
	25	+
	26	+"Legal Entity" shall mean the union of the acting entity and all other
	27	+entities that control, are controlled by, or are under common control with
	28	+that entity. For the purposes of this definition, "control" means (i) the
	29	+power, direct or indirect, to cause the direction or management of such
	30	+entity, whether by contract or otherwise, or (ii) ownership of fifty
	31	+percent (50%) or more of the outstanding shares, or (iii) beneficial
	32	+ownership of such entity.
	33	+
	34	+"You" (or "Your") shall mean an individual or Legal Entity exercising
	35	+permissions granted by this License.
	36	+
	37	+"Source" form shall mean the preferred form for making modifications,
	38	+including but not limited to software source code, documentation source,
	39	+and configuration files.
	40	+
	41	+"Object" form shall mean any form resulting from mechanical transformation
	42	+or translation of a Source form, including but not limited to compiled
	43	+object code, generated documentation, and conversions to other media types.
	44	+
	45	+"Work" shall mean the work of authorship, whether in Source or Object form,
	46	+made available under the License, as indicated by a copyright notice that
	47	+is included in or attached to the work (an example is provided in the
	48	+Appendix below).
	49	+
	50	+"Derivative Works" shall mean any work, whether in Source or Object form,
	51	+that is based on (or derived from) the Work and for which the editorial
	52	+revisions, annotations, elaborations, or other modifications represent, as
	53	+a whole, an original work of authorship. For the purposes of this License,
	54	+Derivative Works shall not include works that remain separable from, or
	55	+merely link (or bind by name) to the interfaces of, the Work and Derivative
	56	+Works thereof.
	57	+
	58	+"Contribution" shall mean any work of authorship, including the original
	59	+version of the Work and any modifications or additions to that Work or
	60	+Derivative Works thereof, that is intentionally submitted to Licensor for
	61	+inclusion in the Work by the copyright owner or by an individual or Legal
	62	+Entity authorized to submit on behalf of the copyright owner. For the
	63	+purposes of this definition, "submitted" means any form of electronic,
	64	+verbal, or written communication sent to the Licensor or its
	65	+representatives, including but not limited to communication on electronic
	66	+mailing lists, source code control systems, and issue tracking systems that
	67	+are managed by, or on behalf of, the Licensor for the purpose of discussing
	68	+and improving the Work, but excluding communication that is conspicuously
	69	+marked or otherwise designated in writing by the copyright owner as "Not a
	70	+Contribution."
	71	+
	72	+"Contributor" shall mean Licensor and any individual or Legal Entity on
	73	+behalf of whom a Contribution has been received by Licensor and
	74	+subsequently incorporated within the Work.
	75	+
	76	+2. Grant of Copyright License. Subject to the terms and conditions of this
	77	+ License, each Contributor hereby grants to You a perpetual, worldwide,
	78	+ non-exclusive, no-charge, royalty-free, irrevocable copyright license to
	79	+ reproduce, prepare Derivative Works of, publicly display, publicly
	80	+ perform, sublicense, and distribute the Work and such Derivative Works
	81	+ in Source or Object form.
	82	+
	83	+3. Grant of Patent License. Subject to the terms and conditions of this
	84	+ License, each Contributor hereby grants to You a perpetual, worldwide,
	85	+ non-exclusive, no-charge, royalty-free, irrevocable (except as stated in
	86	+ this section) patent license to make, have made, use, offer to sell,
	87	+ sell, import, and otherwise transfer the Work, where such license
	88	+ applies only to those patent claims licensable by such Contributor that
	89	+ are necessarily infringed by their Contribution(s) alone or by
	90	+ combination of their Contribution(s) with the Work to which such
	91	+ Contribution(s) was submitted. If You institute patent litigation
	92	+ against any entity (including a cross-claim or counterclaim in a
	93	+ lawsuit) alleging that the Work or a Contribution incorporated within
	94	+ the Work constitutes direct or contributory patent infringement, then
	95	+ any patent licenses granted to You under this License for that Work
	96	+ shall terminate as of the date such litigation is filed.
	97	+
	98	+4. Redistribution. You may reproduce and distribute copies of the Work or
	99	+ Derivative Works thereof in any medium, with or without modifications,
	100	+ and in Source or Object form, provided that You meet the following
	101	+ conditions:
	102	+
	103	+ a. You must give any other recipients of the Work or Derivative Works a
	104	+ copy of this License; and
	105	+
	106	+ b. You must cause any modified files to carry prominent notices stating
	107	+ that You changed the files; and
	108	+
	109	+ c. You must retain, in the Source form of any Derivative Works that You
	110	+ distribute, all copyright, patent, trademark, and attribution notices
	111	+ from the Source form of the Work, excluding those notices that do not
	112	+ pertain to any part of the Derivative Works; and
	113	+
	114	+ d. If the Work includes a "NOTICE" text file as part of its
	115	+ distribution, then any Derivative Works that You distribute must
	116	+ include a readable copy of the attribution notices contained within
	117	+ such NOTICE file, excluding those notices that do not pertain to any
	118	+ part of the Derivative Works, in at least one of the following
	119	+ places: within a NOTICE text file distributed as part of the
	120	+ Derivative Works; within the Source form or documentation, if
	121	+ provided along with the Derivative Works; or, within a display
	122	+ generated by the Derivative Works, if and wherever such third-party
	123	+ notices normally appear. The contents of the NOTICE file are for
	124	+ informational purposes only and do not modify the License. You may
	125	+ add Your own attribution notices within Derivative Works that You
	126	+ distribute, alongside or as an addendum to the NOTICE text from the
	127	+ Work, provided that such additional attribution notices cannot be
	128	+ construed as modifying the License.
	129	+
	130	+ You may add Your own copyright statement to Your modifications and may
	131	+ provide additional or different license terms and conditions for use,
	132	+ reproduction, or distribution of Your modifications, or for any such
	133	+ Derivative Works as a whole, provided Your use, reproduction, and
	134	+ distribution of the Work otherwise complies with the conditions stated
	135	+ in this License.
	136	+
	137	+5. Submission of Contributions. Unless You explicitly state otherwise, any
	138	+ Contribution intentionally submitted for inclusion in the Work by You to
	139	+ the Licensor shall be under the terms and conditions of this License,
	140	+ without any additional terms or conditions. Notwithstanding the above,
	141	+ nothing herein shall supersede or modify the terms of any separate
	142	+ license agreement you may have executed with Licensor regarding such
	143	+ Contributions.
	144	+
	145	+6. Trademarks. This License does not grant permission to use the trade
	146	+ names, trademarks, service marks, or product names of the Licensor,
	147	+ except as required for reasonable and customary use in describing the
	148	+ origin of the Work and reproducing the content of the NOTICE file.
	149	+
	150	+7. Disclaimer of Warranty. Unless required by applicable law or agreed to
	151	+ in writing, Licensor provides the Work (and each Contributor provides
	152	+ its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
	153	+ OF ANY KIND, either express or implied, including, without limitation,
	154	+ any warranties or conditions of TITLE, NON-INFRINGEMENT,
	155	+ MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely
	156	+ responsible for determining the appropriateness of using or
	157	+ redistributing the Work and assume any risks associated with Your
	158	+ exercise of permissions under this License.
	159	+
	160	+8. Limitation of Liability. In no event and under no legal theory, whether
	161	+ in tort (including negligence), contract, or otherwise, unless required
	162	+ by applicable law (such as deliberate and grossly negligent acts) or
	163	+ agreed to in writing, shall any Contributor be liable to You for
	164	+ damages, including any direct, indirect, special, incidental, or
	165	+ consequential damages of any character arising as a result of this
	166	+ License or out of the use or inability to use the Work (including but
	167	+ not limited to damages for loss of goodwill, work stoppage, computer
	168	+ failure or malfunction, or any and all other commercial damages or
	169	+ losses), even if such Contributor has been advised of the possibility of
	170	+ such damages.
	171	+
	172	+9. Accepting Warranty or Additional Liability. While redistributing the
	173	+ Work or Derivative Works thereof, You may choose to offer, and charge a
	174	+ fee for, acceptance of support, warranty, indemnity, or other liability
	175	+ obligations and/or rights consistent with this License. However, in
	176	+ accepting such obligations, You may act only on Your own behalf and on
	177	+ Your sole responsibility, not on behalf of any other Contributor, and
	178	+ only if You agree to indemnify, defend, and hold each Contributor
	179	+ harmless for any liability incurred by, or claims asserted against, such
	180	+ Contributor by reason of your accepting any such warranty or additional
	181	+ liability.
	182	+
	183	+END OF TERMS AND CONDITIONS

LICENSES/other/CC-BY-SA-4.0+397−0

@@ -0,0 +1,397 @@
	1	+Valid-License-Identifier: CC-BY-SA-4.0
	2	+SPDX-URL: https://spdx.org/licenses/CC-BY-SA-4.0
	3	+Usage-Guide:
	4	+ To use the Creative Commons Attribution Share Alike 4.0 International
	5	+ license put the following SPDX tag/value pair into a comment according to
	6	+ the placement guidelines in the licensing rules documentation:
	7	+ SPDX-License-Identifier: CC-BY-SA-4.0
	8	+License-Text:
	9	+
	10	+Creative Commons Attribution-ShareAlike 4.0 International
	11	+
	12	+Creative Commons Corporation ("Creative Commons") is not a law firm and
	13	+does not provide legal services or legal advice. Distribution of Creative
	14	+Commons public licenses does not create a lawyer-client or other
	15	+relationship. Creative Commons makes its licenses and related information
	16	+available on an "as-is" basis. Creative Commons gives no warranties
	17	+regarding its licenses, any material licensed under their terms and
	18	+conditions, or any related information. Creative Commons disclaims all
	19	+liability for damages resulting from their use to the fullest extent
	20	+possible.
	21	+
	22	+Using Creative Commons Public Licenses
	23	+
	24	+Creative Commons public licenses provide a standard set of terms and
	25	+conditions that creators and other rights holders may use to share original
	26	+works of authorship and other material subject to copyright and certain
	27	+other rights specified in the public license below. The following
	28	+considerations are for informational purposes only, are not exhaustive, and
	29	+do not form part of our licenses.
	30	+
	31	+Considerations for licensors: Our public licenses are intended for use by
	32	+those authorized to give the public permission to use material in ways
	33	+otherwise restricted by copyright and certain other rights. Our licenses
	34	+are irrevocable. Licensors should read and understand the terms and
	35	+conditions of the license they choose before applying it. Licensors should
	36	+also secure all rights necessary before applying our licenses so that the
	37	+public can reuse the material as expected. Licensors should clearly mark
	38	+any material not subject to the license. This includes other CC-licensed
	39	+material, or material used under an exception or limitation to
	40	+copyright. More considerations for licensors :
	41	+wiki.creativecommons.org/Considerations_for_licensors
	42	+
	43	+Considerations for the public: By using one of our public licenses, a
	44	+licensor grants the public permission to use the licensed material under
	45	+specified terms and conditions. If the licensor's permission is not
	46	+necessary for any reason - for example, because of any applicable exception
	47	+or limitation to copyright - then that use is not regulated by the
	48	+license. Our licenses grant only permissions under copyright and certain
	49	+other rights that a licensor has authority to grant. Use of the licensed
	50	+material may still be restricted for other reasons, including because
	51	+others have copyright or other rights in the material. A licensor may make
	52	+special requests, such as asking that all changes be marked or described.
	53	+
	54	+Although not required by our licenses, you are encouraged to respect those
	55	+requests where reasonable. More considerations for the public :
	56	+wiki.creativecommons.org/Considerations_for_licensees
	57	+
	58	+Creative Commons Attribution-ShareAlike 4.0 International Public License
	59	+
	60	+By exercising the Licensed Rights (defined below), You accept and agree to
	61	+be bound by the terms and conditions of this Creative Commons
	62	+Attribution-ShareAlike 4.0 International Public License ("Public
	63	+License"). To the extent this Public License may be interpreted as a
	64	+contract, You are granted the Licensed Rights in consideration of Your
	65	+acceptance of these terms and conditions, and the Licensor grants You such
	66	+rights in consideration of benefits the Licensor receives from making the
	67	+Licensed Material available under these terms and conditions.
	68	+
	69	+Section 1 - Definitions.
	70	+
	71	+ a. Adapted Material means material subject to Copyright and Similar
	72	+ Rights that is derived from or based upon the Licensed Material and
	73	+ in which the Licensed Material is translated, altered, arranged,
	74	+ transformed, or otherwise modified in a manner requiring permission
	75	+ under the Copyright and Similar Rights held by the Licensor. For
	76	+ purposes of this Public License, where the Licensed Material is a
	77	+ musical work, performance, or sound recording, Adapted Material is
	78	+ always produced where the Licensed Material is synched in timed
	79	+ relation with a moving image.
	80	+
	81	+ b. Adapter's License means the license You apply to Your Copyright and
	82	+ Similar Rights in Your contributions to Adapted Material in
	83	+ accordance with the terms and conditions of this Public License.
	84	+
	85	+ c. BY-SA Compatible License means a license listed at
	86	+ creativecommons.org/compatiblelicenses, approved by Creative Commons
	87	+ as essentially the equivalent of this Public License.
	88	+
	89	+ d. Copyright and Similar Rights means copyright and/or similar rights
	90	+ closely related to copyright including, without limitation,
	91	+ performance, broadcast, sound recording, and Sui Generis Database
	92	+ Rights, without regard to how the rights are labeled or
	93	+ categorized. For purposes of this Public License, the rights
	94	+ specified in Section 2(b)(1)-(2) are not Copyright and Similar
	95	+ Rights.
	96	+
	97	+ e. Effective Technological Measures means those measures that, in the
	98	+ absence of proper authority, may not be circumvented under laws
	99	+ fulfilling obligations under Article 11 of the WIPO Copyright Treaty
	100	+ adopted on December 20, 1996, and/or similar international
	101	+ agreements.
	102	+
	103	+ f. Exceptions and Limitations means fair use, fair dealing, and/or any
	104	+ other exception or limitation to Copyright and Similar Rights that
	105	+ applies to Your use of the Licensed Material.
	106	+
	107	+ g. License Elements means the license attributes listed in the name of
	108	+ a Creative Commons Public License. The License Elements of this
	109	+ Public License are Attribution and ShareAlike.
	110	+
	111	+ h. Licensed Material means the artistic or literary work, database, or
	112	+ other material to which the Licensor applied this Public License.
	113	+
	114	+ i. Licensed Rights means the rights granted to You subject to the terms
	115	+ and conditions of this Public License, which are limited to all
	116	+ Copyright and Similar Rights that apply to Your use of the Licensed
	117	+ Material and that the Licensor has authority to license.
	118	+
	119	+ j. Licensor means the individual(s) or entity(ies) granting rights
	120	+ under this Public License.
	121	+
	122	+ k. Share means to provide material to the public by any means or
	123	+ process that requires permission under the Licensed Rights, such as
	124	+ reproduction, public display, public performance, distribution,
	125	+ dissemination, communication, or importation, and to make material
	126	+ available to the public including in ways that members of the public
	127	+ may access the material from a place and at a time individually
	128	+ chosen by them.
	129	+
	130	+ l. Sui Generis Database Rights means rights other than copyright
	131	+ resulting from Directive 96/9/EC of the European Parliament and of
	132	+ the Council of 11 March 1996 on the legal protection of databases,
	133	+ as amended and/or succeeded, as well as other essentially equivalent
	134	+ rights anywhere in the world. m. You means the individual or entity
	135	+ exercising the Licensed Rights under this Public License. Your has a
	136	+ corresponding meaning.
	137	+
	138	+Section 2 - Scope.
	139	+
	140	+ a. License grant.
	141	+
	142	+ 1. Subject to the terms and conditions of this Public License, the
	143	+ Licensor hereby grants You a worldwide, royalty-free,
	144	+ non-sublicensable, non-exclusive, irrevocable license to
	145	+ exercise the Licensed Rights in the Licensed Material to:
	146	+
	147	+ A. reproduce and Share the Licensed Material, in whole or in part; and
	148	+
	149	+ B. produce, reproduce, and Share Adapted Material.
	150	+
	151	+ 2. Exceptions and Limitations. For the avoidance of doubt, where
	152	+ Exceptions and Limitations apply to Your use, this Public
	153	+ License does not apply, and You do not need to comply with its
	154	+ terms and conditions.
	155	+
	156	+ 3. Term. The term of this Public License is specified in Section 6(a).
	157	+
	158	+ 4. Media and formats; technical modifications allowed. The Licensor
	159	+ authorizes You to exercise the Licensed Rights in all media and
	160	+ formats whether now known or hereafter created, and to make
	161	+ technical modifications necessary to do so. The Licensor waives
	162	+ and/or agrees not to assert any right or authority to forbid You
	163	+ from making technical modifications necessary to exercise the
	164	+ Licensed Rights, including technical modifications necessary to
	165	+ circumvent Effective Technological Measures. For purposes of
	166	+ this Public License, simply making modifications authorized by
	167	+ this Section 2(a)(4) never produces Adapted Material.
	168	+
	169	+ 5. Downstream recipients.
	170	+
	171	+ A. Offer from the Licensor - Licensed Material. Every recipient
	172	+ of the Licensed Material automatically receives an offer
	173	+ from the Licensor to exercise the Licensed Rights under the
	174	+ terms and conditions of this Public License.
	175	+
	176	+ B. Additional offer from the Licensor - Adapted Material. Every
	177	+ recipient of Adapted Material from You automatically
	178	+ receives an offer from the Licensor to exercise the Licensed
	179	+ Rights in the Adapted Material under the conditions of the
	180	+ Adapter's License You apply.
	181	+
	182	+ C. No downstream restrictions. You may not offer or impose any
	183	+ additional or different terms or conditions on, or apply any
	184	+ Effective Technological Measures to, the Licensed Material
	185	+ if doing so restricts exercise of the Licensed Rights by any
	186	+ recipient of the Licensed Material.
	187	+
	188	+ 6. No endorsement. Nothing in this Public License constitutes or
	189	+ may be construed as permission to assert or imply that You are,
	190	+ or that Your use of the Licensed Material is, connected with, or
	191	+ sponsored, endorsed, or granted official status by, the Licensor
	192	+ or others designated to receive attribution as provided in
	193	+ Section 3(a)(1)(A)(i).
	194	+
	195	+ b. Other rights.
	196	+
	197	+ 1. Moral rights, such as the right of integrity, are not licensed
	198	+ under this Public License, nor are publicity, privacy, and/or
	199	+ other similar personality rights; however, to the extent
	200	+ possible, the Licensor waives and/or agrees not to assert any
	201	+ such rights held by the Licensor to the limited extent necessary
	202	+ to allow You to exercise the Licensed Rights, but not otherwise.
	203	+
	204	+ 2. Patent and trademark rights are not licensed under this Public
	205	+ License.
	206	+
	207	+ 3. To the extent possible, the Licensor waives any right to collect
	208	+ royalties from You for the exercise of the Licensed Rights,
	209	+ whether directly or through a collecting society under any
	210	+ voluntary or waivable statutory or compulsory licensing
	211	+ scheme. In all other cases the Licensor expressly reserves any
	212	+ right to collect such royalties.
	213	+
	214	+Section 3 - License Conditions.
	215	+
	216	+Your exercise of the Licensed Rights is expressly made subject to the
	217	+following conditions.
	218	+
	219	+ a. Attribution.
	220	+
	221	+ 1. If You Share the Licensed Material (including in modified form),
	222	+ You must:
	223	+
	224	+ A. retain the following if it is supplied by the Licensor with
	225	+ the Licensed Material:
	226	+
	227	+ i. identification of the creator(s) of the Licensed
	228	+ Material and any others designated to receive
	229	+ attribution, in any reasonable manner requested by the
	230	+ Licensor (including by pseudonym if designated);
	231	+
	232	+ ii. a copyright notice;
	233	+
	234	+ iii. a notice that refers to this Public License;
	235	+
	236	+ iv. a notice that refers to the disclaimer of warranties;
	237	+
	238	+ v. a URI or hyperlink to the Licensed Material to the extent reasonably practicable;
	239	+
	240	+ B. indicate if You modified the Licensed Material and retain an
	241	+ indication of any previous modifications; and
	242	+
	243	+ C. indicate the Licensed Material is licensed under this Public
	244	+ License, and include the text of, or the URI or hyperlink to,
	245	+ this Public License.
	246	+
	247	+ 2. You may satisfy the conditions in Section 3(a)(1) in any
	248	+ reasonable manner based on the medium, means, and context in
	249	+ which You Share the Licensed Material. For example, it may be
	250	+ reasonable to satisfy the conditions by providing a URI or
	251	+ hyperlink to a resource that includes the required information.
	252	+
	253	+ 3. If requested by the Licensor, You must remove any of the
	254	+ information required by Section 3(a)(1)(A) to the extent
	255	+ reasonably practicable. b. ShareAlike.In addition to the
	256	+ conditions in Section 3(a), if You Share Adapted Material You
	257	+ produce, the following conditions also apply.
	258	+
	259	+ 1. The Adapter's License You apply must be a Creative Commons
	260	+ license with the same License Elements, this version or
	261	+ later, or a BY-SA Compatible License.
	262	+
	263	+ 2. You must include the text of, or the URI or hyperlink to, the
	264	+ Adapter's License You apply. You may satisfy this condition
	265	+ in any reasonable manner based on the medium, means, and
	266	+ context in which You Share Adapted Material.
	267	+
	268	+ 3. You may not offer or impose any additional or different terms
	269	+ or conditions on, or apply any Effective Technological
	270	+ Measures to, Adapted Material that restrict exercise of the
	271	+ rights granted under the Adapter's License You apply.
	272	+
	273	+Section 4 - Sui Generis Database Rights.
	274	+
	275	+Where the Licensed Rights include Sui Generis Database Rights that apply to
	276	+Your use of the Licensed Material:
	277	+
	278	+ a. for the avoidance of doubt, Section 2(a)(1) grants You the right to
	279	+ extract, reuse, reproduce, and Share all or a substantial portion of
	280	+ the contents of the database;
	281	+
	282	+ b. if You include all or a substantial portion of the database contents
	283	+ in a database in which You have Sui Generis Database Rights, then
	284	+ the database in which You have Sui Generis Database Rights (but not
	285	+ its individual contents) is Adapted Material, including for purposes
	286	+ of Section 3(b); and
	287	+
	288	+ c. You must comply with the conditions in Section 3(a) if You Share all
	289	+ or a substantial portion of the contents of the database.
	290	+
	291	+ For the avoidance of doubt, this Section 4 supplements and does not
	292	+ replace Your obligations under this Public License where the Licensed
	293	+ Rights include other Copyright and Similar Rights.
	294	+
	295	+Section 5 - Disclaimer of Warranties and Limitation of Liability.
	296	+
	297	+ a. Unless otherwise separately undertaken by the Licensor, to the
	298	+ extent possible, the Licensor offers the Licensed Material as-is and
	299	+ as-available, and makes no representations or warranties of any kind
	300	+ concerning the Licensed Material, whether express, implied,
	301	+ statutory, or other. This includes, without limitation, warranties
	302	+ of title, merchantability, fitness for a particular purpose,
	303	+ non-infringement, absence of latent or other defects, accuracy, or
	304	+ the presence or absence of errors, whether or not known or
	305	+ discoverable. Where disclaimers of warranties are not allowed in
	306	+ full or in part, this disclaimer may not apply to You.
	307	+
	308	+ b. To the extent possible, in no event will the Licensor be liable to
	309	+ You on any legal theory (including, without limitation, negligence)
	310	+ or otherwise for any direct, special, indirect, incidental,
	311	+ consequential, punitive, exemplary, or other losses, costs,
	312	+ expenses, or damages arising out of this Public License or use of
	313	+ the Licensed Material, even if the Licensor has been advised of the
	314	+ possibility of such losses, costs, expenses, or damages. Where a
	315	+ limitation of liability is not allowed in full or in part, this
	316	+ limitation may not apply to You.
	317	+
	318	+ c. The disclaimer of warranties and limitation of liability provided
	319	+ above shall be interpreted in a manner that, to the extent possible,
	320	+ most closely approximates an absolute disclaimer and waiver of all
	321	+ liability.
	322	+
	323	+Section 6 - Term and Termination.
	324	+
	325	+ a. This Public License applies for the term of the Copyright and
	326	+ Similar Rights licensed here. However, if You fail to comply with
	327	+ this Public License, then Your rights under this Public License
	328	+ terminate automatically.
	329	+
	330	+ b. Where Your right to use the Licensed Material has terminated under
	331	+ Section 6(a), it reinstates:
	332	+
	333	+ 1. automatically as of the date the violation is cured, provided it
	334	+ is cured within 30 days of Your discovery of the violation; or
	335	+
	336	+ 2. upon express reinstatement by the Licensor.
	337	+
	338	+ c. For the avoidance of doubt, this Section 6(b) does not affect any
	339	+ right the Licensor may have to seek remedies for Your violations of
	340	+ this Public License.
	341	+
	342	+ d. For the avoidance of doubt, the Licensor may also offer the Licensed
	343	+ Material under separate terms or conditions or stop distributing the
	344	+ Licensed Material at any time; however, doing so will not terminate
	345	+ this Public License.
	346	+
	347	+ e. Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
	348	+
	349	+Section 7 - Other Terms and Conditions.
	350	+
	351	+ a. The Licensor shall not be bound by any additional or different terms
	352	+ or conditions communicated by You unless expressly agreed.
	353	+
	354	+ b. Any arrangements, understandings, or agreements regarding the
	355	+ Licensed Material not stated herein are separate from and
	356	+ independent of the terms and conditions of this Public License.
	357	+
	358	+Section 8 - Interpretation.
	359	+
	360	+ a. For the avoidance of doubt, this Public License does not, and shall
	361	+ not be interpreted to, reduce, limit, restrict, or impose conditions
	362	+ on any use of the Licensed Material that could lawfully be made
	363	+ without permission under this Public License.
	364	+
	365	+ b. To the extent possible, if any provision of this Public License is
	366	+ deemed unenforceable, it shall be automatically reformed to the
	367	+ minimum extent necessary to make it enforceable. If the provision
	368	+ cannot be reformed, it shall be severed from this Public License
	369	+ without affecting the enforceability of the remaining terms and
	370	+ conditions.
	371	+
	372	+ c. No term or condition of this Public License will be waived and no
	373	+ failure to comply consented to unless expressly agreed to by the
	374	+ Licensor.
	375	+
	376	+ d. Nothing in this Public License constitutes or may be interpreted as
	377	+ a limitation upon, or waiver of, any privileges and immunities that
	378	+ apply to the Licensor or You, including from the legal processes of
	379	+ any jurisdiction or authority.
	380	+
	381	+Creative Commons is not a party to its public licenses. Notwithstanding,
	382	+Creative Commons may elect to apply one of its public licenses to material
	383	+it publishes and in those instances will be considered the "Licensor." The
	384	+text of the Creative Commons public licenses is dedicated to the public
	385	+domain under the CC0 Public Domain Dedication. Except for the limited
	386	+purpose of indicating that material is shared under a Creative Commons
	387	+public license or as otherwise permitted by the Creative Commons policies
	388	+published at creativecommons.org/policies, Creative Commons does not
	389	+authorize the use of the trademark "Creative Commons" or any other
	390	+trademark or logo of Creative Commons without its prior written consent
	391	+including, without limitation, in connection with any unauthorized
	392	+modifications to any of its public licenses or any other arrangements,
	393	+understandings, or agreements concerning use of licensed material. For the
	394	+avoidance of doubt, this paragraph does not form part of the public
	395	+licenses.
	396	+
	397	+Creative Commons may be contacted at creativecommons.org.

LICENSES/other/CDDL-1.0+364−0

@@ -0,0 +1,364 @@
	1	+Valid-License-Identifier: CDDL-1.0
	2	+SPDX-URL: https://spdx.org/licenses/CDDL-1.0.html
	3	+Usage-Guide:
	4	+ To use the Common Development and Distribution License 1.0 put the
	5	+ following SPDX tag/value pair into a comment according to the placement
	6	+ guidelines in the licensing rules documentation:
	7	+ SPDX-License-Identifier: CDDL-1.0
	8	+
	9	+License-Text:
	10	+
	11	+COMMON DEVELOPMENT AND DISTRIBUTION LICENSE (CDDL)
	12	+Version 1.0
	13	+
	14	+ 1. Definitions.
	15	+
	16	+ 1.1. "Contributor" means each individual or entity that creates or
	17	+ contributes to the creation of Modifications.
	18	+
	19	+ 1.2. "Contributor Version" means the combination of the Original
	20	+ Software, prior Modifications used by a Contributor (if any),
	21	+ and the Modifications made by that particular Contributor.
	22	+
	23	+ 1.3. "Covered Software" means (a) the Original Software, or (b)
	24	+ Modifications, or (c) the combination of files containing
	25	+ Original Software with files containing Modifications, in each
	26	+ case including portions thereof.
	27	+
	28	+ 1.4. "Executable" means the Covered Software in any form other than
	29	+ Source Code.
	30	+
	31	+ 1.5. "Initial Developer" means the individual or entity that first
	32	+ makes Original Software available under this License.
	33	+
	34	+ 1.6. "Larger Work" means a work which combines Covered Software or
	35	+ portions thereof with code not governed by the terms of this
	36	+ License.
	37	+
	38	+ 1.7. "License" means this document.
	39	+
	40	+ 1.8. "Licensable" means having the right to grant, to the maximum
	41	+ extent possible, whether at the time of the initial grant or
	42	+ subsequently acquired, any and all of the rights conveyed herein.
	43	+
	44	+ 1.9. "Modifications" means the Source Code and Executable form of
	45	+ any of the following:
	46	+
	47	+ A. Any file that results from an addition to, deletion from or
	48	+ modification of the contents of a file containing Original
	49	+ Software or previous Modifications;
	50	+
	51	+ B. Any new file that contains any part of the Original Software
	52	+ or previous Modification; or
	53	+
	54	+ C. Any new file that is contributed or otherwise made available
	55	+ under the terms of this License.
	56	+
	57	+ 1.10. "Original Software" means the Source Code and Executable form
	58	+ of computer software code that is originally released under
	59	+ this License.
	60	+
	61	+ 1.11. "Patent Claims" means any patent claim(s), now owned or
	62	+ hereafter acquired, including without limitation, method,
	63	+ process, and apparatus claims, in any patent Licensable by
	64	+ grantor.
	65	+
	66	+ 1.12. "Source Code" means (a) the common form of computer software
	67	+ code in which modifications are made and (b) associated
	68	+ documentation included in or with such code.
	69	+
	70	+ 1.13. "You" (or "Your") means an individual or a legal entity
	71	+ exercising rights under, and complying with all of the terms
	72	+ of, this License. For legal entities, "You" includes any
	73	+ entity which controls, is controlled by, or is under common
	74	+ control with You. For purposes of this definition, "control"
	75	+ means (a) the power, direct or indirect, to cause the
	76	+ direction or management of such entity, whether by contract
	77	+ or otherwise, or (b) ownership of more than fifty percent
	78	+ (50%) of the outstanding shares or beneficial ownership of
	79	+ such entity.
	80	+
	81	+ 2. License Grants.
	82	+ 2.1. The Initial Developer Grant.
	83	+
	84	+ Conditioned upon Your compliance with Section 3.1 below and subject
	85	+ to third party intellectual property claims, the Initial Developer
	86	+ hereby grants You a world-wide, royalty-free, non-exclusive
	87	+ license:
	88	+
	89	+ (a) under intellectual property rights (other than patent or
	90	+ trademark) Licensable by Initial Developer, to use,
	91	+ reproduce, modify, display, perform, sublicense and
	92	+ distribute the Original Software (or portions thereof),
	93	+ with or without Modifications, and/or as part of a Larger
	94	+ Work; and
	95	+
	96	+ (b) under Patent Claims infringed by the making, using or
	97	+ selling of Original Software, to make, have made, use,
	98	+ practice, sell, and offer for sale, and/or otherwise
	99	+ dispose of the Original Software (or portions thereof).
	100	+
	101	+ (c) The licenses granted in Sections 2.1(a) and (b) are
	102	+ effective on the date Initial Developer first distributes
	103	+ or otherwise makes the Original Software available to a
	104	+ third party under the terms of this License.
	105	+
	106	+ (d) Notwithstanding Section 2.1(b) above, no patent license is
	107	+ granted: (1) for code that You delete from the Original
	108	+ Software, or (2) for infringements caused by: (i) the
	109	+ modification of the Original Software, or (ii) the
	110	+ combination of the Original Software with other software or
	111	+ devices.
	112	+
	113	+ 2.2. Contributor Grant.
	114	+
	115	+ Conditioned upon Your compliance with Section 3.1 below and subject
	116	+ to third party intellectual property claims, each Contributor
	117	+ hereby grants You a world-wide, royalty-free, non-exclusive
	118	+ license:
	119	+
	120	+ (a) under intellectual property rights (other than patent or
	121	+ trademark) Licensable by Contributor to use, reproduce,
	122	+ modify, display, perform, sublicense and distribute the
	123	+ Modifications created by such Contributor (or portions
	124	+ thereof), either on an unmodified basis, with other
	125	+ Modifications, as Covered Software and/or as part of a
	126	+ Larger Work; and
	127	+
	128	+ (b) under Patent Claims infringed by the making, using, or
	129	+ selling of Modifications made by that Contributor either
	130	+ alone and/or in combination with its Contributor Version
	131	+ (or portions of such combination), to make, use, sell,
	132	+ offer for sale, have made, and/or otherwise dispose of: (1)
	133	+ Modifications made by that Contributor (or portions
	134	+ thereof); and (2) the combination of Modifications made by
	135	+ that Contributor with its Contributor Version (or portions
	136	+ of such combination).
	137	+
	138	+ (c) The licenses granted in Sections 2.2(a) and 2.2(b) are
	139	+ effective on the date Contributor first distributes or
	140	+ otherwise makes the Modifications available to a third
	141	+ party.
	142	+
	143	+ (d) Notwithstanding Section 2.2(b) above, no patent license is
	144	+ granted: (1) for any code that Contributor has deleted from
	145	+ the Contributor Version; (2) for infringements caused by:
	146	+ (i) third party modifications of Contributor Version, or
	147	+ (ii) the combination of Modifications made by that
	148	+ Contributor with other software (except as part of the
	149	+ Contributor Version) or other devices; or (3) under Patent
	150	+ Claims infringed by Covered Software in the absence of
	151	+ Modifications made by that Contributor.
	152	+
	153	+ 3. Distribution Obligations.
	154	+ 3.1. Availability of Source Code.
	155	+
	156	+ Any Covered Software that You distribute or otherwise make
	157	+ available in Executable form must also be made available in Source
	158	+ Code form and that Source Code form must be distributed only under
	159	+ the terms of this License. You must include a copy of this License
	160	+ with every copy of the Source Code form of the Covered Software You
	161	+ distribute or otherwise make available. You must inform recipients
	162	+ of any such Covered Software in Executable form as to how they can
	163	+ obtain such Covered Software in Source Code form in a reasonable
	164	+ manner on or through a medium customarily used for software
	165	+ exchange.
	166	+
	167	+ 3.2. Modifications.
	168	+
	169	+ The Modifications that You create or to which You contribute are
	170	+ governed by the terms of this License. You represent that You
	171	+ believe Your Modifications are Your original creation(s) and/or You
	172	+ have sufficient rights to grant the rights conveyed by this
	173	+ License.
	174	+
	175	+ 3.3. Required Notices.
	176	+
	177	+ You must include a notice in each of Your Modifications that
	178	+ identifies You as the Contributor of the Modification. You may not
	179	+ remove or alter any copyright, patent or trademark notices
	180	+ contained within the Covered Software, or any notices of licensing
	181	+ or any descriptive text giving attribution to any Contributor or
	182	+ the Initial Developer.
	183	+
	184	+ 3.4. Application of Additional Terms.
	185	+
	186	+ You may not offer or impose any terms on any Covered Software in
	187	+ Source Code form that alters or restricts the applicable version of
	188	+ this License or the recipients' rights hereunder. You may choose to
	189	+ offer, and to charge a fee for, warranty, support, indemnity or
	190	+ liability obligations to one or more recipients of Covered
	191	+ Software. However, you may do so only on Your own behalf, and not
	192	+ on behalf of the Initial Developer or any Contributor. You must
	193	+ make it absolutely clear that any such warranty, support, indemnity
	194	+ or liability obligation is offered by You alone, and You hereby
	195	+ agree to indemnify the Initial Developer and every Contributor for
	196	+ any liability incurred by the Initial Developer or such Contributor
	197	+ as a result of warranty, support, indemnity or liability terms You
	198	+ offer.
	199	+
	200	+ 3.5. Distribution of Executable Versions.
	201	+
	202	+ You may distribute the Executable form of the Covered Software
	203	+ under the terms of this License or under the terms of a license of
	204	+ Your choice, which may contain terms different from this License,
	205	+ provided that You are in compliance with the terms of this License
	206	+ and that the license for the Executable form does not attempt to
	207	+ limit or alter the recipient's rights in the Source Code form from
	208	+ the rights set forth in this License. If You distribute the Covered
	209	+ Software in Executable form under a different license, You must
	210	+ make it absolutely clear that any terms which differ from this
	211	+ License are offered by You alone, not by the Initial Developer or
	212	+ Contributor. You hereby agree to indemnify the Initial Developer
	213	+ and every Contributor for any liability incurred by the Initial
	214	+ Developer or such Contributor as a result of any such terms You
	215	+ offer.
	216	+
	217	+ 3.6. Larger Works.
	218	+
	219	+ You may create a Larger Work by combining Covered Software with
	220	+ other code not governed by the terms of this License and distribute
	221	+ the Larger Work as a single product. In such a case, You must make
	222	+ sure the requirements of this License are fulfilled for the Covered
	223	+ Software.
	224	+
	225	+ 4. Versions of the License.
	226	+ 4.1. New Versions.
	227	+
	228	+ Sun Microsystems, Inc. is the initial license steward and may
	229	+ publish revised and/or new versions of this License from time to
	230	+ time. Each version will be given a distinguishing version
	231	+ number. Except as provided in Section 4.3, no one other than the
	232	+ license steward has the right to modify this License.
	233	+
	234	+ 4.2. Effect of New Versions.
	235	+
	236	+ You may always continue to use, distribute or otherwise make the
	237	+ Covered Software available under the terms of the version of the
	238	+ License under which You originally received the Covered
	239	+ Software. If the Initial Developer includes a notice in the
	240	+ Original Software prohibiting it from being distributed or
	241	+ otherwise made available under any subsequent version of the
	242	+ License, You must distribute and make the Covered Software
	243	+ available under the terms of the version of the License under which
	244	+ You originally received the Covered Software. Otherwise, You may
	245	+ also choose to use, distribute or otherwise make the Covered
	246	+ Software available under the terms of any subsequent version of the
	247	+ License published by the license steward.
	248	+
	249	+ 4.3. Modified Versions.
	250	+
	251	+ When You are an Initial Developer and You want to create a new
	252	+ license for Your Original Software, You may create and use a
	253	+ modified version of this License if You: (a) rename the license and
	254	+ remove any references to the name of the license steward (except to
	255	+ note that the license differs from this License); and (b) otherwise
	256	+ make it clear that the license contains terms which differ from
	257	+ this License.
	258	+
	259	+ 5. DISCLAIMER OF WARRANTY.
	260	+
	261	+ COVERED SOFTWARE IS PROVIDED UNDER THIS LICENSE ON AN "AS IS" BASIS,
	262	+ WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING,
	263	+ WITHOUT LIMITATION, WARRANTIES THAT THE COVERED SOFTWARE IS FREE OF
	264	+ DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE OR
	265	+ NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF
	266	+ THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY COVERED SOFTWARE PROVE
	267	+ DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER
	268	+ CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY SERVICING, REPAIR OR
	269	+ CORRECTION. THIS DISCLAIMER OF WARRANTY CONSTITUTES AN ESSENTIAL PART
	270	+ OF THIS LICENSE. NO USE OF ANY COVERED SOFTWARE IS AUTHORIZED HEREUNDER
	271	+ EXCEPT UNDER THIS DISCLAIMER.
	272	+
	273	+ 6. TERMINATION.
	274	+
	275	+ 6.1. This License and the rights granted hereunder will terminate
	276	+ automatically if You fail to comply with terms herein and fail to
	277	+ cure such breach within 30 days of becoming aware of the
	278	+ breach. Provisions which, by their nature, must remain in effect
	279	+ beyond the termination of this License shall survive.
	280	+
	281	+ 6.2. If You assert a patent infringement claim (excluding
	282	+ declaratory judgment actions) against Initial Developer or a
	283	+ Contributor (the Initial Developer or Contributor against whom You
	284	+ assert such claim is referred to as "Participant") alleging that
	285	+ the Participant Software (meaning the Contributor Version where the
	286	+ Participant is a Contributor or the Original Software where the
	287	+ Participant is the Initial Developer) directly or indirectly
	288	+ infringes any patent, then any and all rights granted directly or
	289	+ indirectly to You by such Participant, the Initial Developer (if
	290	+ the Initial Developer is not the Participant) and all Contributors
	291	+ under Sections 2.1 and/or 2.2 of this License shall, upon 60 days
	292	+ notice from Participant terminate prospectively and automatically
	293	+ at the expiration of such 60 day notice period, unless if within
	294	+ such 60 day period You withdraw Your claim with respect to the
	295	+ Participant Software against such Participant either unilaterally
	296	+ or pursuant to a written agreement with Participant.
	297	+
	298	+ 6.3. In the event of termination under Sections 6.1 or 6.2 above,
	299	+ all end user licenses that have been validly granted by You or any
	300	+ distributor hereunder prior to termination (excluding licenses
	301	+ granted to You by any distributor) shall survive termination.
	302	+
	303	+ 7. LIMITATION OF LIABILITY.
	304	+
	305	+ UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER TORT
	306	+ (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL YOU, THE INITIAL
	307	+ DEVELOPER, ANY OTHER CONTRIBUTOR, OR ANY DISTRIBUTOR OF COVERED
	308	+ SOFTWARE, OR ANY SUPPLIER OF ANY OF SUCH PARTIES, BE LIABLE TO ANY
	309	+ PERSON FOR ANY INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES
	310	+ OF ANY CHARACTER INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOST
	311	+ PROFITS, LOSS OF GOODWILL, WORK STOPPAGE, COMPUTER FAILURE OR
	312	+ MALFUNCTION, OR ANY AND ALL OTHER COMMERCIAL DAMAGES OR LOSSES, EVEN IF
	313	+ SUCH PARTY SHALL HAVE BEEN INFORMED OF THE POSSIBILITY OF SUCH
	314	+ DAMAGES. THIS LIMITATION OF LIABILITY SHALL NOT APPLY TO LIABILITY FOR
	315	+ DEATH OR PERSONAL INJURY RESULTING FROM SUCH PARTY'S NEGLIGENCE TO THE
	316	+ EXTENT APPLICABLE LAW PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO
	317	+ NOT ALLOW THE EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL
	318	+ DAMAGES, SO THIS EXCLUSION AND LIMITATION MAY NOT APPLY TO YOU.
	319	+
	320	+ 8. U.S. GOVERNMENT END USERS.
	321	+
	322	+ The Covered Software is a "commercial item," as that term is defined in
	323	+ 48 C.F.R. 2.101 (Oct. 1995), consisting of "commercial computer
	324	+ software" (as that term is defined at 48 C.F.R. $ 252.227-7014(a)(1))
	325	+ and "commercial computer software documentation" as such terms are used
	326	+ in 48 C.F.R. 12.212 (Sept. 1995). Consistent with 48 C.F.R. 12.212 and
	327	+ 48 C.F.R. 227.7202-1 through 227.7202-4 (June 1995), all
	328	+ U.S. Government End Users acquire Covered Software with only those
	329	+ rights set forth herein. This U.S. Government Rights clause is in lieu
	330	+ of, and supersedes, any other FAR, DFAR, or other clause or provision
	331	+ that addresses Government rights in computer software under this
	332	+ License.
	333	+
	334	+ 9. MISCELLANEOUS.
	335	+
	336	+ This License represents the complete agreement concerning subject
	337	+ matter hereof. If any provision of this License is held to be
	338	+ unenforceable, such provision shall be reformed only to the extent
	339	+ necessary to make it enforceable. This License shall be governed by the
	340	+ law of the jurisdiction specified in a notice contained within the
	341	+ Original Software (except to the extent applicable law, if any,
	342	+ provides otherwise), excluding such jurisdiction's conflict-of-law
	343	+ provisions. Any litigation relating to this License shall be subject to
	344	+ the jurisdiction of the courts located in the jurisdiction and venue
	345	+ specified in a notice contained within the Original Software, with the
	346	+ losing party responsible for costs, including, without limitation,
	347	+ court costs and reasonable attorneys' fees and expenses. The
	348	+ application of the United Nations Convention on Contracts for the
	349	+ International Sale of Goods is expressly excluded. Any law or
	350	+ regulation which provides that the language of a contract shall be
	351	+ construed against the drafter shall not apply to this License. You
	352	+ agree that You alone are responsible for compliance with the United
	353	+ States export administration regulations (and the export control laws
	354	+ and regulation of any other countries) when You use, distribute or
	355	+ otherwise make available any Covered Software.
	356	+
	357	+ 10. RESPONSIBILITY FOR CLAIMS.
	358	+
	359	+ As between Initial Developer and the Contributors, each party is
	360	+ responsible for claims and damages arising, directly or indirectly, out
	361	+ of its utilization of rights under this License and You agree to work
	362	+ with Initial Developer and Contributors to distribute such
	363	+ responsibility on an equitable basis. Nothing herein is intended or
	364	+ shall be deemed to constitute any admission of liability.

LICENSES/other/Linux-OpenIB+26−0

@@ -0,0 +1,26 @@
	1	+Valid-License-Identifier: Linux-OpenIB
	2	+SPDX-URL: https://spdx.org/licenses/Linux-OpenIB.html
	3	+Usage-Guide:
	4	+ To use the Linux Kernel Variant of OpenIB.org license put the following
	5	+ SPDX tag/value pair into a comment according to the placement guidelines
	6	+ in the licensing rules documentation:
	7	+ SPDX-License-Identifier: Linux-OpenIB
	8	+License-Text:
	9	+
	10	+Redistribution and use in source and binary forms, with or without
	11	+modification, are permitted provided that the following conditions are met:
	12	+
	13	+ - Redistributions of source code must retain the above copyright
	14	+ notice, this list of conditions and the following disclaimer.
	15	+
	16	+ - Redistributions in binary form must reproduce the above copyright
	17	+ notice, this list of conditions and the following disclaimer in the
	18	+ documentation and/or other materials provided with the distribution.
	19	+
	20	+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	21	+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	22	+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
	23	+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
	24	+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	25	+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
	26	+DEALINGS IN THE SOFTWARE.

LICENSES/other/X11+37−0

@@ -0,0 +1,37 @@
	1	+Valid-License-Identifier: X11
	2	+SPDX-URL: https://spdx.org/licenses/X11.html
	3	+Usage-Guide:
	4	+ To use the X11 put the following SPDX tag/value pair into a comment
	5	+ according to the placement guidelines in the licensing rules
	6	+ documentation:
	7	+ SPDX-License-Identifier: X11
	8	+License-Text:
	9	+
	10	+
	11	+X11 License
	12	+
	13	+Copyright (C) 1996 X Consortium
	14	+
	15	+Permission is hereby granted, free of charge, to any person obtaining a
	16	+copy of this software and associated documentation files (the "Software"),
	17	+to deal in the Software without restriction, including without limitation
	18	+the rights to use, copy, modify, merge, publish, distribute, sublicense,
	19	+and/or sell copies of the Software, and to permit persons to whom the
	20	+Software is furnished to do so, subject to the following conditions:
	21	+
	22	+The above copyright notice and this permission notice shall be included in
	23	+all copies or substantial portions of the Software.
	24	+
	25	+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	26	+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	27	+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
	28	+X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
	29	+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
	30	+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
	31	+
	32	+Except as contained in this notice, the name of the X Consortium shall not
	33	+be used in advertising or otherwise to promote the sale, use or other
	34	+dealings in this Software without prior written authorization from the X
	35	+Consortium.
	36	+
	37	+X Window System is a trademark of X Consortium, Inc.

LICENSES/preferred/GPL-2.0+6−0

@@ -1,5 +1,7 @@
1	1	Valid-License-Identifier: GPL-2.0
	2	+Valid-License-Identifier: GPL-2.0-only
2	3	Valid-License-Identifier: GPL-2.0+
	4	+Valid-License-Identifier: GPL-2.0-or-later
3	5	SPDX-URL: https://spdx.org/licenses/GPL-2.0.html
4	6	Usage-Guide:
5	7	To use this license in source code, put one of the following SPDX
@@ -7,8 +9,12 @@ Usage-Guide: · 1 unmodified line
7	9	guidelines in the licensing rules documentation.
8	10	For 'GNU General Public License (GPL) version 2 only' use:
9	11	SPDX-License-Identifier: GPL-2.0
	12	+ or
	13	+ SPDX-License-Identifier: GPL-2.0-only
10	14	For 'GNU General Public License (GPL) version 2 or any later version' use:
11	15	SPDX-License-Identifier: GPL-2.0+
	16	+ or
	17	+ SPDX-License-Identifier: GPL-2.0-or-later
12	18	License-Text:
13	19
14	20	GNU GENERAL PUBLIC LICENSE

MAINTAINERS+1−1

@@ -15642,7 +15642,7 @@ L: linux-mm@kvack.org · 15641 unmodified lines
15642	15642	S: Maintained
15643	15643	F: mm/zsmalloc.c
15644	15644	F: include/linux/zsmalloc.h
15645		−F: Documentation/vm/zsmalloc.txt
	15645	+F: Documentation/vm/zsmalloc.rst
15646	15646
15647	15647	ZSWAP COMPRESSED SWAP CACHING
15648	15648	M: Seth Jennings <sjenning@redhat.com>

arch/alpha/Kconfig+1−1

@@ -576,7 +576,7 @@ config ARCH_DISCONTIGMEM_ENABLE · 575 unmodified lines
576	576	Say Y to support efficient handling of discontiguous physical memory,
577	577	for architectures which are either NUMA (Non-Uniform Memory Access)
578	578	or have huge holes in the physical address space for other reasons.
579		− See <file:Documentation/vm/numa> for more.
	579	+ See <file:Documentation/vm/numa.rst> for more.
580	580
581	581	source "mm/Kconfig"
582	582

arch/ia64/Kconfig+1−1

@@ -381,7 +381,7 @@ config ARCH_DISCONTIGMEM_ENABLE · 380 unmodified lines
381	381	Say Y to support efficient handling of discontiguous physical memory,
382	382	for architectures which are either NUMA (Non-Uniform Memory Access)
383	383	or have huge holes in the physical address space for other reasons.
384		− See <file:Documentation/vm/numa> for more.
	384	+ See <file:Documentation/vm/numa.rst> for more.
385	385
386	386	config ARCH_FLATMEM_ENABLE
387	387	def_bool y

arch/mips/Kconfig+1−1

@@ -2548,7 +2548,7 @@ config ARCH_DISCONTIGMEM_ENABLE · 2547 unmodified lines
2548	2548	Say Y to support efficient handling of discontiguous physical memory,
2549	2549	for architectures which are either NUMA (Non-Uniform Memory Access)
2550	2550	or have huge holes in the physical address space for other reasons.
2551		− See <file:Documentation/vm/numa> for more.
	2551	+ See <file:Documentation/vm/numa.rst> for more.
2552	2552
2553	2553	config ARCH_SPARSEMEM_ENABLE
2554	2554	bool

arch/powerpc/Kconfig+1−1

@@ -865,7 +865,7 @@ config PPC_MEM_KEYS · 864 unmodified lines
865	865	page-based protections, but without requiring modification of the
866	866	page tables when an application changes protection domains.
867	867
868		− For details, see Documentation/vm/protection-keys.txt
	868	+ For details, see Documentation/vm/protection-keys.rst
869	869
870	870	If unsure, say y.
871	871

drivers/w1/w1_io.c+1−0

@@ -194,6 +194,7 @@ static u8 w1_read_bit(struct w1_master *dev) · 193 unmodified lines
194	194	* bit 0 = id_bit
195	195	* bit 1 = comp_bit
196	196	* bit 2 = dir_taken
	197	+ *
197	198	* If both bits 0 & 1 are set, the search should be restarted.
198	199	*
199	200	* Return: bit fields - see above

fs/Kconfig+1−1

@@ -196,7 +196,7 @@ config HUGETLBFS · 195 unmodified lines
196	196	help
197	197	hugetlbfs is a filesystem backing for HugeTLB pages, based on
198	198	ramfs. For architectures that support it, say Y here and read
199		− <file:Documentation/vm/hugetlbpage.txt> for details.
	199	+ <file:Documentation/admin-guide/mm/hugetlbpage.rst> for details.
200	200
201	201	If unsure, say N.
202	202

fs/dax.c+1−1

@@ -677,7 +677,7 @@ static void dax_mapping_entry_mkclean(struct address_space *mapping, · 676 unmodified lines
677	677	* downgrading page table protection not changing it to point
678	678	* to a new page.
679	679	*
680		− * See Documentation/vm/mmu_notifier.txt
	680	+ * See Documentation/vm/mmu_notifier.rst
681	681	*/
682	682	if (pmdp) {
683	683	#ifdef CONFIG_FS_DAX_PMD

fs/proc/task_mmu.c+2−2

@@ -937,7 +937,7 @@ static inline void clear_soft_dirty(struct vm_area_struct *vma, · 936 unmodified lines
937	937	/*
938	938	* The soft-dirty tracker uses #PF-s to catch writes
939	939	* to pages, so write-protect the pte as well. See the
940		− * Documentation/vm/soft-dirty.txt for full description
	940	+ * Documentation/admin-guide/mm/soft-dirty.rst for full description
941	941	* of how soft-dirty works.
942	942	*/
943	943	pte_t ptent = *pte;
@@ -1421,7 +1421,7 @@ static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask, · 477 unmodified lines
1421	1421	* Bits 0-54 page frame number (PFN) if present
1422	1422	* Bits 0-4 swap type if swapped
1423	1423	* Bits 5-54 swap offset if swapped
1424		− * Bit 55 pte is soft-dirty (see Documentation/vm/soft-dirty.txt)
	1424	+ * Bit 55 pte is soft-dirty (see Documentation/admin-guide/mm/soft-dirty.rst)
1425	1425	* Bit 56 page exclusively mapped
1426	1426	* Bits 57-60 zero
1427	1427	* Bit 61 page is file-page or shared-anon

include/linux/hmm.h+1−1

@@ -16,7 +16,7 @@ · 15 unmodified lines
16	16	/*
17	17	* Heterogeneous Memory Management (HMM)
18	18	*
19		− * See Documentation/vm/hmm.txt for reasons and overview of what HMM is and it
	19	+ * See Documentation/vm/hmm.rst for reasons and overview of what HMM is and it
20	20	* is for. Here we focus on the HMM API description, with some explanation of
21	21	* the underlying implementation.
22	22	*

include/linux/memremap.h+2−2

@@ -45,7 +45,7 @@ struct vmem_altmap { · 44 unmodified lines
45	45	* must be treated as an opaque object, rather than a "normal" struct page.
46	46	*
47	47	* A more complete discussion of unaddressable memory may be found in
48		− * include/linux/hmm.h and Documentation/vm/hmm.txt.
	48	+ * include/linux/hmm.h and Documentation/vm/hmm.rst.
49	49	*
50	50	* MEMORY_DEVICE_PUBLIC:
51	51	* Device memory that is cache coherent from device and CPU point of view. This
@@ -67,7 +67,7 @@ enum memory_type { · 15 unmodified lines
67	67	* page_free()
68	68	*
69	69	* Additional notes about MEMORY_DEVICE_PRIVATE may be found in
70		− * include/linux/hmm.h and Documentation/vm/hmm.txt. There is also a brief
	70	+ * include/linux/hmm.h and Documentation/vm/hmm.rst. There is also a brief
71	71	* explanation in include/linux/memory_hotplug.h.
72	72	*
73	73	* The page_fault() callback must migrate page back, from device memory to

include/linux/mmu_notifier.h+1−1

@@ -174,7 +174,7 @@ struct mmu_notifier_ops { · 173 unmodified lines
174	174	* invalidate_range_start()/end() notifiers, as
175	175	* invalidate_range() alread catches the points in time when an
176	176	* external TLB range needs to be flushed. For more in depth
177		− * discussion on this see Documentation/vm/mmu_notifier.txt
	177	+ * discussion on this see Documentation/vm/mmu_notifier.rst
178	178	*
179	179	* Note that this function might be called with just a sub-range
180	180	* of what was passed to invalidate_range_start()/end(), if

include/linux/sched/mm.h+40−2

@@ -28,7 +28,7 @@ extern struct mm_struct *mm_alloc(void); · 27 unmodified lines
28	28	*
29	29	* Use mmdrop() to release the reference acquired by mmgrab().
30	30	*
31		− * See also <Documentation/vm/active_mm.txt> for an in-depth explanation
	31	+ * See also <Documentation/vm/active_mm.rst> for an in-depth explanation
32	32	* of &mm_struct.mm_count vs &mm_struct.mm_users.
33	33	*/
34	34	static inline void mmgrab(struct mm_struct *mm)
@@ -62,7 +62,7 @@ static inline void mmdrop(struct mm_struct *mm) · 27 unmodified lines
62	62	*
63	63	* Use mmput() to release the reference acquired by mmget().
64	64	*
65		− * See also <Documentation/vm/active_mm.txt> for an in-depth explanation
	65	+ * See also <Documentation/vm/active_mm.rst> for an in-depth explanation
66	66	* of &mm_struct.mm_count vs &mm_struct.mm_users.
67	67	*/
68	68	static inline void mmget(struct mm_struct *mm)
@@ -170,6 +170,17 @@ static inline void fs_reclaim_acquire(gfp_t gfp_mask) { } · 101 unmodified lines
170	170	static inline void fs_reclaim_release(gfp_t gfp_mask) { }
171	171	#endif
172	172
	173	+/**
	174	+ * memalloc_noio_save - Marks implicit GFP_NOIO allocation scope.
	175	+ *
	176	+ * This functions marks the beginning of the GFP_NOIO allocation scope.
	177	+ * All further allocations will implicitly drop __GFP_IO flag and so
	178	+ * they are safe for the IO critical section from the allocation recursion
	179	+ * point of view. Use memalloc_noio_restore to end the scope with flags
	180	+ * returned by this function.
	181	+ *
	182	+ * This function is safe to be used from any context.
	183	+ */
173	184	static inline unsigned int memalloc_noio_save(void)
174	185	{
175	186	unsigned int flags = current->flags & PF_MEMALLOC_NOIO;
@@ -177,11 +188,30 @@ static inline unsigned int memalloc_noio_save(void) · 1 unmodified line
177	188	return flags;
178	189	}
179	190
	191	+/**
	192	+ * memalloc_noio_restore - Ends the implicit GFP_NOIO scope.
	193	+ * @flags: Flags to restore.
	194	+ *
	195	+ * Ends the implicit GFP_NOIO scope started by memalloc_noio_save function.
	196	+ * Always make sure that that the given flags is the return value from the
	197	+ * pairing memalloc_noio_save call.
	198	+ */
180	199	static inline void memalloc_noio_restore(unsigned int flags)
181	200	{
182	201	current->flags = (current->flags & ~PF_MEMALLOC_NOIO) \| flags;
183	202	}
184	203
	204	+/**
	205	+ * memalloc_nofs_save - Marks implicit GFP_NOFS allocation scope.
	206	+ *
	207	+ * This functions marks the beginning of the GFP_NOFS allocation scope.
	208	+ * All further allocations will implicitly drop __GFP_FS flag and so
	209	+ * they are safe for the FS critical section from the allocation recursion
	210	+ * point of view. Use memalloc_nofs_restore to end the scope with flags
	211	+ * returned by this function.
	212	+ *
	213	+ * This function is safe to be used from any context.
	214	+ */
185	215	static inline unsigned int memalloc_nofs_save(void)
186	216	{
187	217	unsigned int flags = current->flags & PF_MEMALLOC_NOFS;
@@ -189,6 +219,14 @@ static inline unsigned int memalloc_nofs_save(void) · 1 unmodified line
189	219	return flags;
190	220	}
191	221
	222	+/**
	223	+ * memalloc_nofs_restore - Ends the implicit GFP_NOFS scope.
	224	+ * @flags: Flags to restore.
	225	+ *
	226	+ * Ends the implicit GFP_NOFS scope started by memalloc_nofs_save function.
	227	+ * Always make sure that that the given flags is the return value from the
	228	+ * pairing memalloc_nofs_save call.
	229	+ */
192	230	static inline void memalloc_nofs_restore(unsigned int flags)
193	231	{
194	232	current->flags = (current->flags & ~PF_MEMALLOC_NOFS) \| flags;

include/linux/swap.h+1−1

@@ -53,7 +53,7 @@ static inline int current_is_kswapd(void) · 52 unmodified lines
53	53
54	54	/*
55	55	* Unaddressable device memory support. See include/linux/hmm.h and
56		− * Documentation/vm/hmm.txt. Short description is we need struct pages for
	56	+ * Documentation/vm/hmm.rst. Short description is we need struct pages for
57	57	* device memory that is unaddressable (inaccessible) by CPU, so that we can
58	58	* migrate part of a process memory to device memory.
59	59	*

mm/Kconfig+4−3

@@ -305,7 +305,7 @@ config KSM · 304 unmodified lines
305	305	the many instances by a single page with that content, so
306	306	saving memory until one or another app needs to modify the content.
307	307	Recommended for use with KVM, or with other duplicative applications.
308		− See Documentation/vm/ksm.txt for more information: KSM is inactive
	308	+ See Documentation/vm/ksm.rst for more information: KSM is inactive
309	309	until a program has madvised that an area is MADV_MERGEABLE, and
310	310	root has set /sys/kernel/mm/ksm/run to 1 (if CONFIG_SYSFS is set).
311	311
@@ -530,7 +530,7 @@ config MEM_SOFT_DIRTY · 218 unmodified lines
530	530	into a page just as regular dirty bit, but unlike the latter
531	531	it can be cleared by hands.
532	532
533		− See Documentation/vm/soft-dirty.txt for more details.
	533	+ See Documentation/admin-guide/mm/soft-dirty.rst for more details.
534	534
535	535	config ZSWAP
536	536	bool "Compressed cache for swap pages (EXPERIMENTAL)"
@@ -657,7 +657,8 @@ config IDLE_PAGE_TRACKING · 120 unmodified lines
657	657	be useful to tune memory cgroup limits and/or for job placement
658	658	within a compute cluster.
659	659
660		− See Documentation/vm/idle_page_tracking.txt for more details.
	660	+ See Documentation/admin-guide/mm/idle_page_tracking.rst for
	661	+ more details.
661	662
662	663	# arch_add_memory() comprehends device memory
663	664	config ARCH_HAS_ZONE_DEVICE

mm/cleancache.c+1−1

@@ -3,7 +3,7 @@ · 2 unmodified lines
3	3	*
4	4	* This code provides the generic "frontend" layer to call a matching
5	5	* "backend" driver implementation of cleancache. See
6		− * Documentation/vm/cleancache.txt for more information.
	6	+ * Documentation/vm/cleancache.rst for more information.
7	7	*
8	8	* Copyright (C) 2009-2010 Oracle Corp. All rights reserved.
9	9	* Author: Dan Magenheimer

mm/frontswap.c+1−1

@@ -3,7 +3,7 @@ · 2 unmodified lines
3	3	*
4	4	* This code provides the generic "frontend" layer to call a matching
5	5	* "backend" driver implementation of frontswap. See
6		− * Documentation/vm/frontswap.txt for more information.
	6	+ * Documentation/vm/frontswap.rst for more information.
7	7	*
8	8	* Copyright (C) 2009-2012 Oracle Corp. All rights reserved.
9	9	* Author: Dan Magenheimer

mm/hmm.c+1−1

@@ -37,7 +37,7 @@ · 36 unmodified lines
37	37
38	38	#if defined(CONFIG_DEVICE_PRIVATE) \|\| defined(CONFIG_DEVICE_PUBLIC)
39	39	/*
40		− * Device private memory see HMM (Documentation/vm/hmm.txt) or hmm.h
	40	+ * Device private memory see HMM (Documentation/vm/hmm.rst) or hmm.h
41	41	*/
42	42	DEFINE_STATIC_KEY_FALSE(device_private_key);
43	43	EXPORT_SYMBOL(device_private_key);

mm/huge_memory.c+2−2

@@ -1185,7 +1185,7 @@ static int do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, pmd_t orig_pmd, · 1184 unmodified lines
1185	1185	* mmu_notifier_invalidate_range_end() happens which can lead to a
1186	1186	* device seeing memory write in different order than CPU.
1187	1187	*
1188		− * See Documentation/vm/mmu_notifier.txt
	1188	+ * See Documentation/vm/mmu_notifier.rst
1189	1189	*/
1190	1190	pmdp_huge_clear_flush_notify(vma, haddr, vmf->pmd);
1191	1191
@@ -2037,7 +2037,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, · 845 unmodified lines
2037	2037	* replacing a zero pmd write protected page with a zero pte write
2038	2038	* protected page.
2039	2039	*
2040		− * See Documentation/vm/mmu_notifier.txt
	2040	+ * See Documentation/vm/mmu_notifier.rst
2041	2041	*/
2042	2042	pmdp_huge_clear_flush(vma, haddr, pmd);
2043	2043

mm/hugetlb.c+2−2

@@ -3291,7 +3291,7 @@ int copy_hugetlb_page_range(struct mm_struct dst, struct mm_struct src, · 3290 unmodified lines
3291	3291	* table protection not changing it to point
3292	3292	* to a new page.
3293	3293	*
3294		− * See Documentation/vm/mmu_notifier.txt
	3294	+ * See Documentation/vm/mmu_notifier.rst
3295	3295	*/
3296	3296	huge_ptep_set_wrprotect(src, addr, src_pte);
3297	3297	}
@@ -4357,7 +4357,7 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, · 1059 unmodified lines
4357	4357	* No need to call mmu_notifier_invalidate_range() we are downgrading
4358	4358	* page table protection not changing it to point to a new page.
4359	4359	*
4360		− * See Documentation/vm/mmu_notifier.txt
	4360	+ * See Documentation/vm/mmu_notifier.rst
4361	4361	*/
4362	4362	i_mmap_unlock_write(vma->vm_file->f_mapping);
4363	4363	mmu_notifier_invalidate_range_end(mm, start, end);

mm/ksm.c+20−3

@@ -51,7 +51,9 @@ · 50 unmodified lines
51	51	#define DO_NUMA(x) do { } while (0)
52	52	#endif
53	53
54		−/*
	54	+/**
	55	+ * DOC: Overview
	56	+ *
55	57	* A few notes about the KSM scanning process,
56	58	* to make it easier to understand the data structures below:
57	59	*
@@ -67,6 +69,21 @@ · 9 unmodified lines
67	69	* this tree is fully assured to be working (except when pages are unmapped),
68	70	* and therefore this tree is called the stable tree.
69	71	*
	72	+ * The stable tree node includes information required for reverse
	73	+ * mapping from a KSM page to virtual addresses that map this page.
	74	+ *
	75	+ * In order to avoid large latencies of the rmap walks on KSM pages,
	76	+ * KSM maintains two types of nodes in the stable tree:
	77	+ *
	78	+ * * the regular nodes that keep the reverse mapping structures in a
	79	+ * linked list
	80	+ * * the "chains" that link nodes ("dups") that represent the same
	81	+ * write protected memory content, but each "dup" corresponds to a
	82	+ * different KSM page copy of that content
	83	+ *
	84	+ * Internally, the regular nodes, "dups" and "chains" are represented
	85	+ * using the same :c:type:`struct stable_node` structure.
	86	+ *
70	87	* In addition to the stable tree, KSM uses a second data structure called the
71	88	* unstable tree: this tree holds pointers to pages which have been found to
72	89	* be "unchanged for a period of time". The unstable tree sorts these pages
@@ -1049,7 +1066,7 @@ static int write_protect_page(struct vm_area_struct vma, struct page page, · 976 unmodified lines
1049	1066	* No need to notify as we are downgrading page table to read
1050	1067	* only not changing it to point to a new page.
1051	1068	*
1052		− * See Documentation/vm/mmu_notifier.txt
	1069	+ * See Documentation/vm/mmu_notifier.rst
1053	1070	*/
1054	1071	entry = ptep_clear_flush(vma, pvmw.address, pvmw.pte);
1055	1072	/*
@@ -1145,7 +1162,7 @@ static int replace_page(struct vm_area_struct vma, struct page page, · 89 unmodified lines
1145	1162	* No need to notify as we are replacing a read only page with another
1146	1163	* read only page with the same content.
1147	1164	*
1148		− * See Documentation/vm/mmu_notifier.txt
	1165	+ * See Documentation/vm/mmu_notifier.rst
1149	1166	*/
1150	1167	ptep_clear_flush(vma, addr, ptep);
1151	1168	set_pte_at_notify(mm, addr, ptep, newpte);

mm/mmap.c+1−1

@@ -2828,7 +2828,7 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size, · 2827 unmodified lines
2828	2828	unsigned long ret = -EINVAL;
2829	2829	struct file *file;
2830	2830
2831		− pr_warn_once("%s (%d) uses deprecated remap_file_pages() syscall. See Documentation/vm/remap_file_pages.txt.\n",
	2831	+ pr_warn_once("%s (%d) uses deprecated remap_file_pages() syscall. See Documentation/vm/remap_file_pages.rst.\n",
2832	2832	current->comm, current->pid);
2833	2833
2834	2834	if (prot)

mm/rmap.c+3−3

@@ -942,7 +942,7 @@ static bool page_mkclean_one(struct page page, struct vm_area_struct vma, · 941 unmodified lines
942	942	* downgrading page table protection not changing it to point
943	943	* to a new page.
944	944	*
945		− * See Documentation/vm/mmu_notifier.txt
	945	+ * See Documentation/vm/mmu_notifier.rst
946	946	*/
947	947	if (ret)
948	948	(*cleaned)++;
@@ -1599,7 +1599,7 @@ static bool try_to_unmap_one(struct page page, struct vm_area_struct vma, · 650 unmodified lines
1599	1599	* point at new page while a device still is using this
1600	1600	* page.
1601	1601	*
1602		− * See Documentation/vm/mmu_notifier.txt
	1602	+ * See Documentation/vm/mmu_notifier.rst
1603	1603	*/
1604	1604	dec_mm_counter(mm, mm_counter_file(page));
1605	1605	}
@@ -1609,7 +1609,7 @@ discard: · 3 unmodified lines
1609	1609	* done above for all cases requiring it to happen under page
1610	1610	* table lock before mmu_notifier_invalidate_range_end()
1611	1611	*
1612		− * See Documentation/vm/mmu_notifier.txt
	1612	+ * See Documentation/vm/mmu_notifier.rst
1613	1613	*/
1614	1614	page_remove_rmap(subpage, PageHuge(page));
1615	1615	put_page(page);

mm/util.c+1−1

@@ -621,7 +621,7 @@ EXPORT_SYMBOL_GPL(vm_memory_committed); · 620 unmodified lines
621	621	* succeed and -ENOMEM implies there is not.
622	622	*
623	623	* We currently support three overcommit policies, which are set via the
624		− * vm.overcommit_memory sysctl. See Documentation/vm/overcommit-accounting
	624	+ * vm.overcommit_memory sysctl. See Documentation/vm/overcommit-accounting.rst
625	625	*
626	626	* Strict overcommit modes added 2002 Feb 26 by Alan Cox.
627	627	* Additional code 2002 Jul 20 by Robert Love.

scripts/documentation-file-ref-check+113−12

@@ -1,15 +1,116 @@
1		−#!/bin/sh
	1	+#!/usr/bin/env perl
	2	+# SPDX-License-Identifier: GPL-2.0
	3	+#
2	4	# Treewide grep for references to files under Documentation, and report
3	5	# non-existing files in stderr.
4	6
5		−for f in $(git ls-files); do
6		− for ref in $(grep -ho "Documentation/[A-Za-z0-9_.,~/+-]" "$f"); do
7		− # presume trailing . and , are not part of the name
8		− ref=${ref%%[.,]}
9		−
10		− # use ls to handle wildcards
11		− if ! ls $ref >/dev/null 2>&1; then
12		− echo "$f: $ref" >&2
13		− fi
14		− done
15		−done
	7	+use warnings;
	8	+use strict;
	9	+use Getopt::Long qw(:config no_auto_abbrev);
	10	+
	11	+my $scriptname = $0;
	12	+$scriptname =~ s,.*/([^/]+/),$1,;
	13	+
	14	+# Parse arguments
	15	+my $help = 0;
	16	+my $fix = 0;
	17	+
	18	+GetOptions(
	19	+ 'fix' => \$fix,
	20	+ 'h\|help\|usage' => \$help,
	21	+);
	22	+
	23	+if ($help != 0) {
	24	+ print "$scriptname [--help] [--fix-rst]\n";
	25	+ exit -1;
	26	+}
	27	+
	28	+# Step 1: find broken references
	29	+print "Finding broken references. This may take a while... " if ($fix);
	30	+
	31	+my %broken_ref;
	32	+
	33	+open IN, "git grep 'Documentation/'\|"
	34	+ or die "Failed to run git grep";
	35	+while (<IN>) {
	36	+ next if (!m/^([^:]+):(.*)/);
	37	+
	38	+ my $f = $1;
	39	+ my $ln = $2;
	40	+
	41	+ # Makefiles contain nasty expressions to parse docs
	42	+ next if ($f =~ m/Makefile/);
	43	+ # Skip this script
	44	+ next if ($f eq $scriptname);
	45	+
	46	+ if ($ln =~ m,\b(\S)(Documentation/[A-Za-z0-9\_\.\,\~/\+-]*),) {
	47	+ my $prefix = $1;
	48	+ my $ref = $2;
	49	+ my $base = $2;
	50	+
	51	+ $ref =~ s/[\,\.]+$//;
	52	+
	53	+ my $fulref = "$prefix$ref";
	54	+
	55	+ $fulref =~ s/^(\<file\|ref)://;
	56	+ $fulref =~ s/^[\'\`]+//;
	57	+ $fulref =~ s,^\$$.*$/,,;
	58	+ $base =~ s,.*/,,;
	59	+
	60	+ # Remove URL false-positives
	61	+ next if ($fulref =~ m/^http/);
	62	+
	63	+ # Check if exists, evaluating wildcards
	64	+ next if (grep -e, glob("$ref $fulref"));
	65	+
	66	+ if ($fix) {
	67	+ if (!($ref =~ m/(devicetree\|scripts\|Kconfig\|Kbuild)/)) {
	68	+ $broken_ref{$ref}++;
	69	+ }
	70	+ } else {
	71	+ print STDERR "$f: $fulref\n";
	72	+ }
	73	+ }
	74	+}
	75	+
	76	+exit 0 if (!$fix);
	77	+
	78	+# Step 2: Seek for file name alternatives
	79	+print "Auto-fixing broken references. Please double-check the results\n";
	80	+
	81	+foreach my $ref (keys %broken_ref) {
	82	+ my $new =$ref;
	83	+
	84	+ # get just the basename
	85	+ $new =~ s,.*/,,;
	86	+
	87	+ # Seek for the same name on another place, as it may have been moved
	88	+ my $f="";
	89	+
	90	+ $f = qx(find . -iname $new) if ($new);
	91	+
	92	+ # usual reason for breakage: file renamed to .rst
	93	+ if (!$f) {
	94	+ $new =~ s/\.txt$/.rst/;
	95	+ $f=qx(find . -iname $new) if ($new);
	96	+ }
	97	+
	98	+ my @find = split /\s+/, $f;
	99	+
	100	+ if (!$f) {
	101	+ print STDERR "ERROR: Didn't find a replacement for $ref\n";
	102	+ } elsif (scalar(@find) > 1) {
	103	+ print STDERR "WARNING: Won't auto-replace, as found multiple files close to $ref:\n";
	104	+ foreach my $j (@find) {
	105	+ $j =~ s,^./,,;
	106	+ print STDERR " $j\n";
	107	+ }
	108	+ } else {
	109	+ $f = $find[0];
	110	+ $f =~ s,^./,,;
	111	+ print "INFO: Replacing $ref to $f\n";
	112	+ foreach my $j (qx(git grep -l $ref)) {
	113	+ qx(sed "s\@$ref\@$f\@g" -i $j);
	114	+ }
	115	+ }
	116	+}

scripts/spdxcheck.py+284−0

@@ -0,0 +1,284 @@
	1	+#!/usr/bin/env python
	2	+# SPDX-License-Identifier: GPL-2.0
	3	+# Copyright Thomas Gleixner <tglx@linutronix.de>
	4	+
	5	+from argparse import ArgumentParser
	6	+from ply import lex, yacc
	7	+import traceback
	8	+import sys
	9	+import git
	10	+import re
	11	+import os
	12	+
	13	+class ParserException(Exception):
	14	+ def __init__(self, tok, txt):
	15	+ self.tok = tok
	16	+ self.txt = txt
	17	+
	18	+class SPDXException(Exception):
	19	+ def __init__(self, el, txt):
	20	+ self.el = el
	21	+ self.txt = txt
	22	+
	23	+class SPDXdata(object):
	24	+ def __init__(self):
	25	+ self.license_files = 0
	26	+ self.exception_files = 0
	27	+ self.licenses = [ ]
	28	+ self.exceptions = { }
	29	+
	30	+# Read the spdx data from the LICENSES directory
	31	+def read_spdxdata(repo):
	32	+
	33	+ # The subdirectories of LICENSES in the kernel source
	34	+ license_dirs = [ "preferred", "other", "exceptions" ]
	35	+ lictree = repo.heads.master.commit.tree['LICENSES']
	36	+
	37	+ spdx = SPDXdata()
	38	+
	39	+ for d in license_dirs:
	40	+ for el in lictree[d].traverse():
	41	+ if not os.path.isfile(el.path):
	42	+ continue
	43	+
	44	+ exception = None
	45	+ for l in open(el.path).readlines():
	46	+ if l.startswith('Valid-License-Identifier:'):
	47	+ lid = l.split(':')[1].strip().upper()
	48	+ if lid in spdx.licenses:
	49	+ raise SPDXException(el, 'Duplicate License Identifier: %s' %lid)
	50	+ else:
	51	+ spdx.licenses.append(lid)
	52	+
	53	+ elif l.startswith('SPDX-Exception-Identifier:'):
	54	+ exception = l.split(':')[1].strip().upper()
	55	+ spdx.exceptions[exception] = []
	56	+
	57	+ elif l.startswith('SPDX-Licenses:'):
	58	+ for lic in l.split(':')[1].upper().strip().replace(' ', '').replace('\t', '').split(','):
	59	+ if not lic in spdx.licenses:
	60	+ raise SPDXException(None, 'Exception %s missing license %s' %(ex, lic))
	61	+ spdx.exceptions[exception].append(lic)
	62	+
	63	+ elif l.startswith("License-Text:"):
	64	+ if exception:
	65	+ if not len(spdx.exceptions[exception]):
	66	+ raise SPDXException(el, 'Exception %s is missing SPDX-Licenses' %excid)
	67	+ spdx.exception_files += 1
	68	+ else:
	69	+ spdx.license_files += 1
	70	+ break
	71	+ return spdx
	72	+
	73	+class id_parser(object):
	74	+
	75	+ reserved = [ 'AND', 'OR', 'WITH' ]
	76	+ tokens = [ 'LPAR', 'RPAR', 'ID', 'EXC' ] + reserved
	77	+
	78	+ precedence = ( ('nonassoc', 'AND', 'OR'), )
	79	+
	80	+ t_ignore = ' \t'
	81	+
	82	+ def __init__(self, spdx):
	83	+ self.spdx = spdx
	84	+ self.lasttok = None
	85	+ self.lastid = None
	86	+ self.lexer = lex.lex(module = self, reflags = re.UNICODE)
	87	+ # Initialize the parser. No debug file and no parser rules stored on disk
	88	+ # The rules are small enough to be generated on the fly
	89	+ self.parser = yacc.yacc(module = self, write_tables = False, debug = False)
	90	+ self.lines_checked = 0
	91	+ self.checked = 0
	92	+ self.spdx_valid = 0
	93	+ self.spdx_errors = 0
	94	+ self.curline = 0
	95	+ self.deepest = 0
	96	+
	97	+ # Validate License and Exception IDs
	98	+ def validate(self, tok):
	99	+ id = tok.value.upper()
	100	+ if tok.type == 'ID':
	101	+ if not id in self.spdx.licenses:
	102	+ raise ParserException(tok, 'Invalid License ID')
	103	+ self.lastid = id
	104	+ elif tok.type == 'EXC':
	105	+ if not self.spdx.exceptions.has_key(id):
	106	+ raise ParserException(tok, 'Invalid Exception ID')
	107	+ if self.lastid not in self.spdx.exceptions[id]:
	108	+ raise ParserException(tok, 'Exception not valid for license %s' %self.lastid)
	109	+ self.lastid = None
	110	+ elif tok.type != 'WITH':
	111	+ self.lastid = None
	112	+
	113	+ # Lexer functions
	114	+ def t_RPAR(self, tok):
	115	+ r'\)'
	116	+ self.lasttok = tok.type
	117	+ return tok
	118	+
	119	+ def t_LPAR(self, tok):
	120	+ r'\('
	121	+ self.lasttok = tok.type
	122	+ return tok
	123	+
	124	+ def t_ID(self, tok):
	125	+ r'[A-Za-z.0-9\-+]+'
	126	+
	127	+ if self.lasttok == 'EXC':
	128	+ print(tok)
	129	+ raise ParserException(tok, 'Missing parentheses')
	130	+
	131	+ tok.value = tok.value.strip()
	132	+ val = tok.value.upper()
	133	+
	134	+ if val in self.reserved:
	135	+ tok.type = val
	136	+ elif self.lasttok == 'WITH':
	137	+ tok.type = 'EXC'
	138	+
	139	+ self.lasttok = tok.type
	140	+ self.validate(tok)
	141	+ return tok
	142	+
	143	+ def t_error(self, tok):
	144	+ raise ParserException(tok, 'Invalid token')
	145	+
	146	+ def p_expr(self, p):
	147	+ '''expr : ID
	148	+ \| ID WITH EXC
	149	+ \| expr AND expr
	150	+ \| expr OR expr
	151	+ \| LPAR expr RPAR'''
	152	+ pass
	153	+
	154	+ def p_error(self, p):
	155	+ if not p:
	156	+ raise ParserException(None, 'Unfinished license expression')
	157	+ else:
	158	+ raise ParserException(p, 'Syntax error')
	159	+
	160	+ def parse(self, expr):
	161	+ self.lasttok = None
	162	+ self.lastid = None
	163	+ self.parser.parse(expr, lexer = self.lexer)
	164	+
	165	+ def parse_lines(self, fd, maxlines, fname):
	166	+ self.checked += 1
	167	+ self.curline = 0
	168	+ try:
	169	+ for line in fd:
	170	+ self.curline += 1
	171	+ if self.curline > maxlines:
	172	+ break
	173	+ self.lines_checked += 1
	174	+ if line.find("SPDX-License-Identifier:") < 0:
	175	+ continue
	176	+ expr = line.split(':')[1].replace('*/', '').strip()
	177	+ self.parse(expr)
	178	+ self.spdx_valid += 1
	179	+ #
	180	+ # Should we check for more SPDX ids in the same file and
	181	+ # complain if there are any?
	182	+ #
	183	+ break
	184	+
	185	+ except ParserException as pe:
	186	+ if pe.tok:
	187	+ col = line.find(expr) + pe.tok.lexpos
	188	+ tok = pe.tok.value
	189	+ sys.stdout.write('%s: %d:%d %s: %s\n' %(fname, self.curline, col, pe.txt, tok))
	190	+ else:
	191	+ sys.stdout.write('%s: %d:0 %s\n' %(fname, self.curline, col, pe.txt))
	192	+ self.spdx_errors += 1
	193	+
	194	+def scan_git_tree(tree):
	195	+ for el in tree.traverse():
	196	+ # Exclude stuff which would make pointless noise
	197	+ # FIXME: Put this somewhere more sensible
	198	+ if el.path.startswith("LICENSES"):
	199	+ continue
	200	+ if el.path.find("license-rules.rst") >= 0:
	201	+ continue
	202	+ if el.path == 'scripts/checkpatch.pl':
	203	+ continue
	204	+ if not os.path.isfile(el.path):
	205	+ continue
	206	+ parser.parse_lines(open(el.path), args.maxlines, el.path)
	207	+
	208	+def scan_git_subtree(tree, path):
	209	+ for p in path.strip('/').split('/'):
	210	+ tree = tree[p]
	211	+ scan_git_tree(tree)
	212	+
	213	+if __name__ == '__main__':
	214	+
	215	+ ap = ArgumentParser(description='SPDX expression checker')
	216	+ ap.add_argument('path', nargs='*', help='Check path or file. If not given full git tree scan. For stdin use "-"')
	217	+ ap.add_argument('-m', '--maxlines', type=int, default=15,
	218	+ help='Maximum number of lines to scan in a file. Default 15')
	219	+ ap.add_argument('-v', '--verbose', action='store_true', help='Verbose statistics output')
	220	+ args = ap.parse_args()
	221	+
	222	+ # Sanity check path arguments
	223	+ if '-' in args.path and len(args.path) > 1:
	224	+ sys.stderr.write('stdin input "-" must be the only path argument\n')
	225	+ sys.exit(1)
	226	+
	227	+ try:
	228	+ # Use git to get the valid license expressions
	229	+ repo = git.Repo(os.getcwd())
	230	+ assert not repo.bare
	231	+
	232	+ # Initialize SPDX data
	233	+ spdx = read_spdxdata(repo)
	234	+
	235	+ # Initilize the parser
	236	+ parser = id_parser(spdx)
	237	+
	238	+ except SPDXException as se:
	239	+ if se.el:
	240	+ sys.stderr.write('%s: %s\n' %(se.el.path, se.txt))
	241	+ else:
	242	+ sys.stderr.write('%s\n' %se.txt)
	243	+ sys.exit(1)
	244	+
	245	+ except Exception as ex:
	246	+ sys.stderr.write('FAIL: %s\n' %ex)
	247	+ sys.stderr.write('%s\n' %traceback.format_exc())
	248	+ sys.exit(1)
	249	+
	250	+ try:
	251	+ if len(args.path) and args.path[0] == '-':
	252	+ parser.parse_lines(sys.stdin, args.maxlines, '-')
	253	+ else:
	254	+ if args.path:
	255	+ for p in args.path:
	256	+ if os.path.isfile(p):
	257	+ parser.parse_lines(open(p), args.maxlines, p)
	258	+ elif os.path.isdir(p):
	259	+ scan_git_subtree(repo.head.reference.commit.tree, p)
	260	+ else:
	261	+ sys.stderr.write('path %s does not exist\n' %p)
	262	+ sys.exit(1)
	263	+ else:
	264	+ # Full git tree scan
	265	+ scan_git_tree(repo.head.commit.tree)
	266	+
	267	+ if args.verbose:
	268	+ sys.stderr.write('\n')
	269	+ sys.stderr.write('License files: %12d\n' %spdx.license_files)
	270	+ sys.stderr.write('Exception files: %12d\n' %spdx.exception_files)
	271	+ sys.stderr.write('License IDs %12d\n' %len(spdx.licenses))
	272	+ sys.stderr.write('Exception IDs %12d\n' %len(spdx.exceptions))
	273	+ sys.stderr.write('\n')
	274	+ sys.stderr.write('Files checked: %12d\n' %parser.checked)
	275	+ sys.stderr.write('Lines checked: %12d\n' %parser.lines_checked)
	276	+ sys.stderr.write('Files with SPDX: %12d\n' %parser.spdx_valid)
	277	+ sys.stderr.write('Files with errors: %12d\n' %parser.spdx_errors)
	278	+
	279	+ sys.exit(0)
	280	+
	281	+ except Exception as ex:
	282	+ sys.stderr.write('FAIL: %s\n' %ex)
	283	+ sys.stderr.write('%s\n' %traceback.format_exc())
	284	+ sys.exit(1)

Merge tag 'docs-4.18' of git://git.lwn.net/linux

Keyboard shortcuts

General

Issues and pull requests

Issues, pull requests and Inbox

Repository code