Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

Pull arm64 and cross-arch updates from Catalin Marinas: "Here's a slightly wider-spread set of updates for 5.9. Going outside the usual arch/arm64/ area is the removal of read_barrier_depends() series from Will and the MSI/IOMMU ID translation series from Lorenzo. The notable arm64 updates include ARMv8.4 TLBI range operations and translation level hint, time namespace support, and perf. Summary: - Removal of the tremendously unpopular read_barrier_depends() barrier, which is a NOP on all architectures apart from Alpha, in favour of allowing architectures to override READ_ONCE() and do whatever dance they need to do to ensure address dependencies provide LOAD -> LOAD/STORE ordering. This work also offers a potential solution if compilers are shown to convert LOAD -> LOAD address dependencies into control dependencies (e.g. under LTO), as weakly ordered architectures will effectively be able to upgrade READ_ONCE() to smp_load_acquire(). The latter case is not used yet, but will be discussed further at LPC. - Make the MSI/IOMMU input/output ID translation PCI agnostic, augment the MSI/IOMMU ACPI/OF ID mapping APIs to accept an input ID bus-specific parameter and apply the resulting changes to the device ID space provided by the Freescale FSL bus. - arm64 support for TLBI range operations and translation table level hints (part of the ARMv8.4 architecture version). - Time namespace support for arm64. - Export the virtual and physical address sizes in vmcoreinfo for makedumpfile and crash utilities. - CPU feature handling cleanups and checks for programmer errors (overlapping bit-fields). - ACPI updates for arm64: disallow AML accesses to EFI code regions and kernel memory. - perf updates for arm64. - Miscellaneous fixes and cleanups, most notably PLT counting optimisation for module loading, recordmcount fix to ignore relocations other than R_AARCH64_CALL26, CMA areas reserved for gigantic pages on 16K and 64K configurations. - Trivial typos, duplicate words" Link: http://lkml.kernel.org/r/20200710165203.31284-1-will@kernel.org Link: http://lkml.kernel.org/r/20200619082013.13661-1-lorenzo.pieralisi@arm.com * tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (82 commits) arm64: use IRQ_STACK_SIZE instead of THREAD_SIZE for irq stack arm64/mm: save memory access in check_and_switch_context() fast switch path arm64: sigcontext.h: delete duplicated word arm64: ptrace.h: delete duplicated word arm64: pgtable-hwdef.h: delete duplicated words bus: fsl-mc: Add ACPI support for fsl-mc bus/fsl-mc: Refactor the MSI domain creation in the DPRC driver of/irq: Make of_msi_map_rid() PCI bus agnostic of/irq: make of_msi_map_get_device_domain() bus agnostic dt-bindings: arm: fsl: Add msi-map device-tree binding for fsl-mc bus of/device: Add input id to of_dma_configure() of/iommu: Make of_map_rid() PCI agnostic ACPI/IORT: Add an input ID to acpi_dma_configure() ACPI/IORT: Remove useless PCI bus walk ACPI/IORT: Make iort_msi_map_rid() PCI agnostic ACPI/IORT: Make iort_get_device_domain IRQ domain agnostic ACPI/IORT: Make iort_match_node_callback walk the ACPI namespace for NC arm64: enable time namespace support arm64/vdso: Restrict splitting VVAR VMA arm64/vdso: Handle faults on timens page ...

Linus Torvalds committed 6 years ago

Commit 145ff1e2 parents 8c4e1c0 0e4cd9f98 files · +1892 −1091Browse files

Unified · Split

98 files

Documentation/RCU/Design/Requirements/Requirements.rst +1 −1
Documentation/admin-guide/kdump/vmcoreinfo.rst +16 −0
Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt +44 −6
Documentation/memory-barriers.txt +8 −148
Documentation/translations/ko_KR/memory-barriers.txt +3 −143
arch/alpha/include/asm/atomic.h +8 −8
arch/alpha/include/asm/barrier.h +5 −54
arch/alpha/include/asm/pgtable.h +5 −5
arch/alpha/include/asm/rwonce.h +35 −0
arch/arm/include/asm/vdso/gettimeofday.h +1 −0
arch/arm64/Kconfig +21 −2
arch/arm64/Makefile +7 −0
arch/arm64/configs/defconfig +1 −0
arch/arm64/include/asm/acpi.h +1 −14
arch/arm64/include/asm/cpucaps.h +3 −1
arch/arm64/include/asm/cpufeature.h +7 −0
arch/arm64/include/asm/hugetlb.h +2 −0
arch/arm64/include/asm/hwcap.h +1 −0
arch/arm64/include/asm/kernel-pgtable.h +1 −1
arch/arm64/include/asm/memory.h +7 −5
arch/arm64/include/asm/mmu_context.h +2 −4
arch/arm64/include/asm/perf_event.h +27 −0
arch/arm64/include/asm/pgtable-hwdef.h +13 −10
arch/arm64/include/asm/pgtable.h +10 −0
arch/arm64/include/asm/ptrace.h +1 −1
arch/arm64/include/asm/stage2_pgtable.h +9 −0
arch/arm64/include/asm/sysreg.h +47 −2
arch/arm64/include/asm/tlb.h +28 −1
arch/arm64/include/asm/tlbflush.h +162 −15
arch/arm64/include/asm/uaccess.h +1 −0
arch/arm64/include/asm/vdso.h +2 −0
arch/arm64/include/asm/vdso/compat_gettimeofday.h +13 −0
arch/arm64/include/asm/vdso/gettimeofday.h +9 −0
arch/arm64/include/uapi/asm/hwcap.h +1 −0
arch/arm64/include/uapi/asm/sigcontext.h +1 −1
arch/arm64/kernel/acpi.c +75 −0
arch/arm64/kernel/cpufeature.c +106 −43
arch/arm64/kernel/cpuinfo.c +1 −0
arch/arm64/kernel/crash_core.c +10 −0
arch/arm64/kernel/entry.S +48 −48
arch/arm64/kernel/module-plts.c +43 −3
arch/arm64/kernel/perf_event.c +70 −19
arch/arm64/kernel/setup.c +14 −10
arch/arm64/kernel/stacktrace.c +1 −1
arch/arm64/kernel/traps.c +1 −1
arch/arm64/kernel/vdso.c +123 −13
arch/arm64/kernel/vdso/vdso.lds.S +4 −1
arch/arm64/kernel/vdso32/vdso.lds.S +4 −1
arch/arm64/kernel/vmlinux.lds.S +0 −1
arch/arm64/kvm/hyp-init.S +1 −0
arch/arm64/kvm/sys_regs.c +34 −34
arch/arm64/mm/context.c +6 −4
arch/arm64/mm/hugetlbpage.c +40 −2
arch/arm64/mm/init.c +3 −19
arch/riscv/include/asm/vdso/gettimeofday.h +1 −0
drivers/acpi/arm64/iort.c +75 −33
drivers/acpi/scan.c +5 −3
drivers/bus/fsl-mc/dprc-driver.c +8 −23
drivers/bus/fsl-mc/fsl-mc-bus.c +59 −20
drivers/bus/fsl-mc/fsl-mc-msi.c +25 −11
drivers/bus/fsl-mc/fsl-mc-private.h +4 −2
drivers/iommu/of_iommu.c +44 −37
drivers/irqchip/irq-gic-v3-its-fsl-mc-msi.c +86 −19
drivers/of/base.c +21 −21
drivers/of/device.c +5 −3
drivers/of/irq.c +18 −16
drivers/pci/msi.c +5 −4
drivers/perf/arm_smmuv3_pmu.c +1 −2
drivers/vhost/vhost.c +0 −5
include/acpi/acpi_bus.h +7 −2
include/asm-generic/Kbuild +1 −0
include/asm-generic/barrier.h +1 −18
include/asm-generic/rwonce.h +90 −0
include/asm-generic/tlb.h +40 −15
include/linux/acpi.h +7 −0
include/linux/acpi_iort.h +11 −9
include/linux/arm-smccc.h +22 −22
include/linux/compiler.h +2 −132
include/linux/compiler_types.h +41 −0
include/linux/nospec.h +2 −0
include/linux/of.h +2 −2
include/linux/of_device.h +14 −2
include/linux/of_iommu.h +4 −2
include/linux/of_irq.h +7 −6
include/linux/percpu-refcount.h +1 −1
include/linux/ptr_ring.h +1 −1
include/linux/sched_clock.h +28 −0
include/uapi/linux/perf_event.h +20 −3
include/vdso/datapage.h +1 −0
kernel/crash_core.c +1 −0
kernel/time/sched_clock.c +13 −28
mm/memory.c +1 −1
scripts/checkpatch.pl +1 −8
scripts/recordmcount.c +6 −0
tools/bpf/Makefile +2 −1
tools/include/uapi/linux/filter.h +90 −0
tools/include/uapi/linux/perf_event.h +20 −3
tools/memory-model/Documentation/explanation.txt +12 −14

Documentation/RCU/Design/Requirements/Requirements.rst+1−1

@@ -463,7 +463,7 @@ again without disrupting RCU readers. · 462 unmodified lines
463	463	This guarantee was only partially premeditated. DYNIX/ptx used an
464	464	explicit memory barrier for publication, but had nothing resembling
465	465	``rcu_dereference()`` for subscription, nor did it have anything
466		−resembling the ``smp_read_barrier_depends()`` that was later subsumed
	466	+resembling the dependency-ordering barrier that was later subsumed
467	467	into ``rcu_dereference()`` and later still into ``READ_ONCE()``. The
468	468	need for these operations made itself known quite suddenly at a
469	469	late-1990s meeting with the DEC Alpha architects, back in the days when

Documentation/admin-guide/kdump/vmcoreinfo.rst+16−0

@@ -93,6 +93,11 @@ It exists in the sparse memory mapping model, and it is also somewhat · 92 unmodified lines
93	93	similar to the mem_map variable, both of them are used to translate an
94	94	address.
95	95
	96	+MAX_PHYSMEM_BITS
	97	+----------------
	98	+
	99	+Defines the maximum supported physical address space memory.
	100	+
96	101	page
97	102	----
98	103
@@ -399,6 +404,17 @@ KERNELPACMASK · 300 unmodified lines
399	404	The mask to extract the Pointer Authentication Code from a kernel virtual
400	405	address.
401	406
	407	+TCR_EL1.T1SZ
	408	+------------
	409	+
	410	+Indicates the size offset of the memory region addressed by TTBR1_EL1.
	411	+The region size is 2^(64-T1SZ) bytes.
	412	+
	413	+TTBR1_EL1 is the table base address register specified by ARMv8-A
	414	+architecture which is used to lookup the page-tables for the Virtual
	415	+addresses in the higher VA range (refer to ARMv8 ARM document for
	416	+more details).
	417	+
402	418	arm
403	419	===
404	420

Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt+44−6

@@ -28,6 +28,16 @@ Documentation/devicetree/bindings/iommu/iommu.txt. · 27 unmodified lines
28	28	For arm-smmu binding, see:
29	29	Documentation/devicetree/bindings/iommu/arm,smmu.yaml.
30	30
	31	+The MSI writes are accompanied by sideband data which is derived from the ICID.
	32	+The msi-map property is used to associate the devices with both the ITS
	33	+controller and the sideband data which accompanies the writes.
	34	+
	35	+For generic MSI bindings, see
	36	+Documentation/devicetree/bindings/interrupt-controller/msi.txt.
	37	+
	38	+For GICv3 and GIC ITS bindings, see:
	39	+Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml.
	40	+
31	41	Required properties:
32	42
33	43	- compatible
@@ -49,11 +59,6 @@ Required properties: · 15 unmodified lines
49	59	region may not be present in some scenarios, such
50	60	as in the device tree presented to a virtual machine.
51	61
52		− - msi-parent
53		− Value type: <phandle>
54		− Definition: Must be present and point to the MSI controller node
55		− handling message interrupts for the MC.
56		−
57	62	- ranges
58	63	Value type: <prop-encoded-array>
59	64	Definition: A standard property. Defines the mapping between the child
@@ -119,6 +124,28 @@ Optional properties: · 59 unmodified lines
119	124	associated with the listed IOMMU, with the iommu-specifier
120	125	(i - icid-base + iommu-base).
121	126
	127	+- msi-map: Maps an ICID to a GIC ITS and associated msi-specifier
	128	+ data.
	129	+
	130	+ The property is an arbitrary number of tuples of
	131	+ (icid-base,gic-its,msi-base,length).
	132	+
	133	+ Any ICID in the interval [icid-base, icid-base + length) is
	134	+ associated with the listed GIC ITS, with the msi-specifier
	135	+ (i - icid-base + msi-base).
	136	+
	137	+Deprecated properties:
	138	+
	139	+ - msi-parent
	140	+ Value type: <phandle>
	141	+ Definition: Describes the MSI controller node handling message
	142	+ interrupts for the MC. When there is no translation
	143	+ between the ICID and deviceID this property can be used
	144	+ to describe the MSI controller used by the devices on the
	145	+ mc-bus.
	146	+ The use of this property for mc-bus is deprecated. Please
	147	+ use msi-map.
	148	+
122	149	Example:
123	150
124	151	smmu: iommu@5000000 {
@@ -128,13 +155,24 @@ Example: · 3 unmodified lines
128	155	...
129	156	};
130	157
	158	+ gic: interrupt-controller@6000000 {
	159	+ compatible = "arm,gic-v3";
	160	+ ...
	161	+ }
	162	+ its: gic-its@6020000 {
	163	+ compatible = "arm,gic-v3-its";
	164	+ msi-controller;
	165	+ ...
	166	+ };
	167	+
131	168	fsl_mc: fsl-mc@80c000000 {
132	169	compatible = "fsl,qoriq-mc";
133	170	reg = <0x00000008 0x0c000000 0 0x40>, /* MC portal base */
134	171	<0x00000000 0x08340000 0 0x40000>; /* MC control reg */
135		− msi-parent = <&its>;
136	172	/* define map for ICIDs 23-64 */
137	173	iommu-map = <23 &smmu 23 41>;
	174	+ /* define msi map for ICIDs 23-64 */
	175	+ msi-map = <23 &its 23 41>;
138	176	#address-cells = <3>;
139	177	#size-cells = <1>;
140	178

Documentation/memory-barriers.txt+8−148

@@ -553,12 +553,12 @@ There are certain things that the Linux kernel memory barriers do not guarantee: · 552 unmodified lines
553	553	DATA DEPENDENCY BARRIERS (HISTORICAL)
554	554	-------------------------------------
555	555
556		−As of v4.15 of the Linux kernel, an smp_read_barrier_depends() was
557		−added to READ_ONCE(), which means that about the only people who
558		−need to pay attention to this section are those working on DEC Alpha
559		−architecture-specific code and those working on READ_ONCE() itself.
560		−For those who need it, and for those who are interested in the history,
561		−here is the story of data-dependency barriers.
	556	+As of v4.15 of the Linux kernel, an smp_mb() was added to READ_ONCE() for
	557	+DEC Alpha, which means that about the only people who need to pay attention
	558	+to this section are those working on DEC Alpha architecture-specific code
	559	+and those working on READ_ONCE() itself. For those who need it, and for
	560	+those who are interested in the history, here is the story of
	561	+data-dependency barriers.
562	562
563	563	The usage requirements of data dependency barriers are a little subtle, and
564	564	it's not always obvious that they're needed. To illustrate, consider the
@@ -2708,144 +2708,6 @@ the properties of the memory window through which devices are accessed and/or · 2143 unmodified lines
2708	2708	the use of any special device communication instructions the CPU may have.
2709	2709
2710	2710
2711		−CACHE COHERENCY
2712		−---------------
2713		−
2714		−Life isn't quite as simple as it may appear above, however: for while the
2715		−caches are expected to be coherent, there's no guarantee that that coherency
2716		−will be ordered. This means that while changes made on one CPU will
2717		−eventually become visible on all CPUs, there's no guarantee that they will
2718		−become apparent in the same order on those other CPUs.
2719		−
2720		−
2721		−Consider dealing with a system that has a pair of CPUs (1 & 2), each of which
2722		−has a pair of parallel data caches (CPU 1 has A/B, and CPU 2 has C/D):
2723		−
2724		− :
2725		− : +--------+
2726		− : +---------+ \| \|
2727		− +--------+ : +--->\| Cache A \|<------->\| \|
2728		− \| \| : \| +---------+ \| \|
2729		− \| CPU 1 \|<---+ \| \|
2730		− \| \| : \| +---------+ \| \|
2731		− +--------+ : +--->\| Cache B \|<------->\| \|
2732		− : +---------+ \| \|
2733		− : \| Memory \|
2734		− : +---------+ \| System \|
2735		− +--------+ : +--->\| Cache C \|<------->\| \|
2736		− \| \| : \| +---------+ \| \|
2737		− \| CPU 2 \|<---+ \| \|
2738		− \| \| : \| +---------+ \| \|
2739		− +--------+ : +--->\| Cache D \|<------->\| \|
2740		− : +---------+ \| \|
2741		− : +--------+
2742		− :
2743		−
2744		−Imagine the system has the following properties:
2745		−
2746		− (*) an odd-numbered cache line may be in cache A, cache C or it may still be
2747		− resident in memory;
2748		−
2749		− (*) an even-numbered cache line may be in cache B, cache D or it may still be
2750		− resident in memory;
2751		−
2752		− (*) while the CPU core is interrogating one cache, the other cache may be
2753		− making use of the bus to access the rest of the system - perhaps to
2754		− displace a dirty cacheline or to do a speculative load;
2755		−
2756		− (*) each cache has a queue of operations that need to be applied to that cache
2757		− to maintain coherency with the rest of the system;
2758		−
2759		− (*) the coherency queue is not flushed by normal loads to lines already
2760		− present in the cache, even though the contents of the queue may
2761		− potentially affect those loads.
2762		−
2763		−Imagine, then, that two writes are made on the first CPU, with a write barrier
2764		−between them to guarantee that they will appear to reach that CPU's caches in
2765		−the requisite order:
2766		−
2767		− CPU 1 CPU 2 COMMENT
2768		− =============== =============== =======================================
2769		− u == 0, v == 1 and p == &u, q == &u
2770		− v = 2;
2771		− smp_wmb(); Make sure change to v is visible before
2772		− change to p
2773		− <A:modify v=2> v is now in cache A exclusively
2774		− p = &v;
2775		− <B:modify p=&v> p is now in cache B exclusively
2776		−
2777		−The write memory barrier forces the other CPUs in the system to perceive that
2778		−the local CPU's caches have apparently been updated in the correct order. But
2779		−now imagine that the second CPU wants to read those values:
2780		−
2781		− CPU 1 CPU 2 COMMENT
2782		− =============== =============== =======================================
2783		− ...
2784		− q = p;
2785		− x = *q;
2786		−
2787		−The above pair of reads may then fail to happen in the expected order, as the
2788		−cacheline holding p may get updated in one of the second CPU's caches while
2789		−the update to the cacheline holding v is delayed in the other of the second
2790		−CPU's caches by some other cache event:
2791		−
2792		− CPU 1 CPU 2 COMMENT
2793		− =============== =============== =======================================
2794		− u == 0, v == 1 and p == &u, q == &u
2795		− v = 2;
2796		− smp_wmb();
2797		− <A:modify v=2> <C:busy>
2798		− <C:queue v=2>
2799		− p = &v; q = p;
2800		− <D:request p>
2801		− <B:modify p=&v> <D:commit p=&v>
2802		− <D:read p>
2803		− x = *q;
2804		− <C:read *q> Reads from v before v updated in cache
2805		− <C:unbusy>
2806		− <C:commit v=2>
2807		−
2808		−Basically, while both cachelines will be updated on CPU 2 eventually, there's
2809		−no guarantee that, without intervention, the order of update will be the same
2810		−as that committed on CPU 1.
2811		−
2812		−
2813		−To intervene, we need to interpolate a data dependency barrier or a read
2814		−barrier between the loads (which as of v4.15 is supplied unconditionally
2815		−by the READ_ONCE() macro). This will force the cache to commit its
2816		−coherency queue before processing any further requests:
2817		−
2818		− CPU 1 CPU 2 COMMENT
2819		− =============== =============== =======================================
2820		− u == 0, v == 1 and p == &u, q == &u
2821		− v = 2;
2822		− smp_wmb();
2823		− <A:modify v=2> <C:busy>
2824		− <C:queue v=2>
2825		− p = &v; q = p;
2826		− <D:request p>
2827		− <B:modify p=&v> <D:commit p=&v>
2828		− <D:read p>
2829		− smp_read_barrier_depends()
2830		− <C:unbusy>
2831		− <C:commit v=2>
2832		− x = *q;
2833		− <C:read *q> Reads from v after v updated in cache
2834		−
2835		−
2836		−This sort of problem can be encountered on DEC Alpha processors as they have a
2837		−split cache that improves performance by making better use of the data bus.
2838		−While most CPUs do imply a data dependency barrier on the read when a memory
2839		−access depends on a read, not all do, so it may not be relied on.
2840		−
2841		−Other CPUs may also have split caches, but must coordinate between the various
2842		−cachelets for normal memory accesses. The semantics of the Alpha removes the
2843		−need for hardware coordination in the absence of memory barriers, which
2844		−permitted Alpha to sport higher CPU clock rates back in the day. However,
2845		−please note that (again, as of v4.15) smp_read_barrier_depends() should not
2846		−be used except in Alpha arch-specific code and within the READ_ONCE() macro.
2847		−
2848		−
2849	2711	CACHE COHERENCY VS DMA
2850	2712	----------------------
2851	2713
@@ -3009,10 +2871,8 @@ caches with the memory coherence system, thus making it seem like pointer · 157 unmodified lines
3009	2871	changes vs new data occur in the right order.
3010	2872
3011	2873	The Alpha defines the Linux kernel's memory model, although as of v4.15
3012		−the Linux kernel's addition of smp_read_barrier_depends() to READ_ONCE()
3013		−greatly reduced Alpha's impact on the memory model.
3014		−
3015		−See the subsection on "Cache Coherency" above.
	2874	+the Linux kernel's addition of smp_mb() to READ_ONCE() on Alpha greatly
	2875	+reduced its impact on the memory model.
3016	2876
3017	2877
3018	2878	VIRTUAL MACHINE GUESTS

Documentation/translations/ko_KR/memory-barriers.txt+3−143

@@ -577,7 +577,7 @@ ACQUIRE 는 해당 오퍼레이션의 로드 부분에만 적용되고 RELEASE · 576 unmodified lines
577	577	데이터 의존성 배리어 (역사적)
578	578	-----------------------------
579	579
580		−리눅스 커널 v4.15 기준으로, smp_read_barrier_depends() 가 READ_ONCE() 에
	580	+리눅스 커널 v4.15 기준으로, smp_mb() 가 DEC Alpha 용 READ_ONCE() 코드에
581	581	추가되었는데, 이는 이 섹션에 주의를 기울여야 하는 사람들은 DEC Alpha 아키텍쳐
582	582	전용 코드를 만드는 사람들과 READ_ONCE() 자체를 만드는 사람들 뿐임을 의미합니다.
583	583	그런 분들을 위해, 그리고 역사에 관심 있는 분들을 위해, 여기 데이터 의존성
@@ -2664,144 +2664,6 @@ CPU 코어는 프로그램의 인과성이 유지된다고만 여겨진다면 · 2080 unmodified lines
2664	2664	수도 있습니다.
2665	2665
2666	2666
2667		−캐시 일관성
2668		−-----------
2669		−
2670		−하지만 삶은 앞에서 이야기한 것처럼 단순하지 않습니다: 캐시들은 일관적일 것으로
2671		−기대되지만, 그 일관성이 순서에도 적용될 거라는 보장은 없습니다. 한 CPU 에서
2672		−만들어진 변경 사항은 최종적으로는 시스템의 모든 CPU 에게 보여지게 되지만, 다른
2673		−CPU 들에게도 같은 순서로 보이게 될 거라는 보장은 없다는 뜻입니다.
2674		−
2675		−
2676		−두개의 CPU (1 & 2) 가 달려 있고, 각 CPU 에 두개의 데이터 캐시(CPU 1 은 A/B 를,
2677		−CPU 2 는 C/D 를 갖습니다)가 병렬로 연결되어 있는 시스템을 다룬다고 생각해
2678		−봅시다:
2679		−
2680		− :
2681		− : +--------+
2682		− : +---------+ \| \|
2683		− +--------+ : +--->\| Cache A \|<------->\| \|
2684		− \| \| : \| +---------+ \| \|
2685		− \| CPU 1 \|<---+ \| \|
2686		− \| \| : \| +---------+ \| \|
2687		− +--------+ : +--->\| Cache B \|<------->\| \|
2688		− : +---------+ \| \|
2689		− : \| Memory \|
2690		− : +---------+ \| System \|
2691		− +--------+ : +--->\| Cache C \|<------->\| \|
2692		− \| \| : \| +---------+ \| \|
2693		− \| CPU 2 \|<---+ \| \|
2694		− \| \| : \| +---------+ \| \|
2695		− +--------+ : +--->\| Cache D \|<------->\| \|
2696		− : +---------+ \| \|
2697		− : +--------+
2698		− :
2699		−
2700		−이 시스템이 다음과 같은 특성을 갖는다 생각해 봅시다:
2701		−
2702		− (*) 홀수번 캐시라인은 캐시 A, 캐시 C 또는 메모리에 위치할 수 있음;
2703		−
2704		− (*) 짝수번 캐시라인은 캐시 B, 캐시 D 또는 메모리에 위치할 수 있음;
2705		−
2706		− (*) CPU 코어가 한개의 캐시에 접근하는 동안, 다른 캐시는 - 더티 캐시라인을
2707		− 메모리에 내리거나 추측성 로드를 하거나 하기 위해 - 시스템의 다른 부분에
2708		− 액세스 하기 위해 버스를 사용할 수 있음;
2709		−
2710		− (*) 각 캐시는 시스템의 나머지 부분들과 일관성을 맞추기 위해 해당 캐시에
2711		− 적용되어야 할 오퍼레이션들의 큐를 가짐;
2712		−
2713		− (*) 이 일관성 큐는 캐시에 이미 존재하는 라인에 가해지는 평범한 로드에 의해서는
2714		− 비워지지 않는데, 큐의 오퍼레이션들이 이 로드의 결과에 영향을 끼칠 수 있다
2715		− 할지라도 그러함.
2716		−
2717		−이제, 첫번째 CPU 에서 두개의 쓰기 오퍼레이션을 만드는데, 해당 CPU 의 캐시에
2718		−요청된 순서로 오퍼레이션이 도달됨을 보장하기 위해 두 오퍼레이션 사이에 쓰기
2719		−배리어를 사용하는 상황을 상상해 봅시다:
2720		−
2721		− CPU 1 CPU 2 COMMENT
2722		− =============== =============== =======================================
2723		− u == 0, v == 1 and p == &u, q == &u
2724		− v = 2;
2725		− smp_wmb(); v 의 변경이 p 의 변경 전에 보일 것을
2726		− 분명히 함
2727		− <A:modify v=2> v 는 이제 캐시 A 에 독점적으로 존재함
2728		− p = &v;
2729		− <B:modify p=&v> p 는 이제 캐시 B 에 독점적으로 존재함
2730		−
2731		−여기서의 쓰기 메모리 배리어는 CPU 1 의 캐시가 올바른 순서로 업데이트 된 것으로
2732		−시스템의 다른 CPU 들이 인지하게 만듭니다. 하지만, 이제 두번째 CPU 가 그 값들을
2733		−읽으려 하는 상황을 생각해 봅시다:
2734		−
2735		− CPU 1 CPU 2 COMMENT
2736		− =============== =============== =======================================
2737		− ...
2738		− q = p;
2739		− x = *q;
2740		−
2741		−위의 두개의 읽기 오퍼레이션은 예상된 순서로 일어나지 못할 수 있는데, 두번째 CPU
2742		−의 한 캐시에 다른 캐시 이벤트가 발생해 v 를 담고 있는 캐시라인의 해당 캐시에의
2743		−업데이트가 지연되는 사이, p 를 담고 있는 캐시라인은 두번째 CPU 의 다른 캐시에
2744		−업데이트 되어버렸을 수 있기 때문입니다.
2745		−
2746		− CPU 1 CPU 2 COMMENT
2747		− =============== =============== =======================================
2748		− u == 0, v == 1 and p == &u, q == &u
2749		− v = 2;
2750		− smp_wmb();
2751		− <A:modify v=2> <C:busy>
2752		− <C:queue v=2>
2753		− p = &v; q = p;
2754		− <D:request p>
2755		− <B:modify p=&v> <D:commit p=&v>
2756		− <D:read p>
2757		− x = *q;
2758		− <C:read *q> 캐시에 업데이트 되기 전의 v 를 읽음
2759		− <C:unbusy>
2760		− <C:commit v=2>
2761		−
2762		−기본적으로, 두개의 캐시라인 모두 CPU 2 에 최종적으로는 업데이트 될 것이지만,
2763		−별도의 개입 없이는, 업데이트의 순서가 CPU 1 에서 만들어진 순서와 동일할
2764		−것이라는 보장이 없습니다.
2765		−
2766		−
2767		−여기에 개입하기 위해선, 데이터 의존성 배리어나 읽기 배리어를 로드 오퍼레이션들
2768		−사이에 넣어야 합니다 (v4.15 부터는 READ_ONCE() 매크로에 의해 무조건적으로
2769		−그렇게 됩니다). 이렇게 함으로써 캐시가 다음 요청을 처리하기 전에 일관성 큐를
2770		−처리하도록 강제하게 됩니다.
2771		−
2772		− CPU 1 CPU 2 COMMENT
2773		− =============== =============== =======================================
2774		− u == 0, v == 1 and p == &u, q == &u
2775		− v = 2;
2776		− smp_wmb();
2777		− <A:modify v=2> <C:busy>
2778		− <C:queue v=2>
2779		− p = &v; q = p;
2780		− <D:request p>
2781		− <B:modify p=&v> <D:commit p=&v>
2782		− <D:read p>
2783		− smp_read_barrier_depends()
2784		− <C:unbusy>
2785		− <C:commit v=2>
2786		− x = *q;
2787		− <C:read *q> 캐시에 업데이트 된 v 를 읽음
2788		−
2789		−
2790		−이런 부류의 문제는 DEC Alpha 계열 프로세서들에서 발견될 수 있는데, 이들은
2791		−데이터 버스를 좀 더 잘 사용해 성능을 개선할 수 있는, 분할된 캐시를 가지고 있기
2792		−때문입니다. 대부분의 CPU 는 하나의 읽기 오퍼레이션의 메모리 액세스가 다른 읽기
2793		−오퍼레이션에 의존적이라면 데이터 의존성 배리어를 내포시킵니다만, 모두가 그런건
2794		−아니기 때문에 이점에 의존해선 안됩니다.
2795		−
2796		−다른 CPU 들도 분할된 캐시를 가지고 있을 수 있지만, 그런 CPU 들은 평범한 메모리
2797		−액세스를 위해서도 이 분할된 캐시들 사이의 조정을 해야만 합니다. Alpha 는 가장
2798		−약한 메모리 순서 시맨틱 (semantic) 을 선택함으로써 메모리 배리어가 명시적으로
2799		−사용되지 않았을 때에는 그런 조정이 필요하지 않게 했으며, 이는 Alpha 가 당시에
2800		−더 높은 CPU 클락 속도를 가질 수 있게 했습니다. 하지만, (다시 말하건대, v4.15
2801		−이후부터는) Alpha 아키텍쳐 전용 코드와 READ_ONCE() 매크로 내부에서를 제외하고는
2802		−smp_read_barrier_depends() 가 사용되지 않아야 함을 알아두시기 바랍니다.
2803		−
2804		−
2805	2667	캐시 일관성 VS DMA
2806	2668	------------------
2807	2669
@@ -2962,10 +2824,8 @@ Alpha CPU 의 일부 버전은 분할된 데이터 캐시를 가지고 있어서 · 154 unmodified lines
2962	2824	데이터의 발견을 올바른 순서로 일어나게 하기 때문입니다.
2963	2825
2964	2826	리눅스 커널의 메모리 배리어 모델은 Alpha 에 기초해서 정의되었습니다만, v4.15
2965		−부터는 리눅스 커널이 READ_ONCE() 내에 smp_read_barrier_depends() 를 추가해서
2966		−Alpha 의 메모리 모델로의 영향력이 크게 줄어들긴 했습니다.
2967		−
2968		−위의 "캐시 일관성" 서브섹션을 참고하세요.
	2827	+부터는 Alpha 용 READ_ONCE() 코드 내에 smp_mb() 가 추가되어서 메모리 모델로의
	2828	+Alpha 의 영향력이 크게 줄어들었습니다.
2969	2829
2970	2830
2971	2831	가상 머신 게스트

arch/alpha/include/asm/atomic.h+8−8

@@ -16,10 +16,10 @@ · 15 unmodified lines
16	16
17	17	/*
18	18	* To ensure dependency ordering is preserved for the _relaxed and
19		− * _release atomics, an smp_read_barrier_depends() is unconditionally
20		− * inserted into the _relaxed variants, which are used to build the
21		− * barriered versions. Avoid redundant back-to-back fences in the
22		− * _acquire and _fence versions.
	19	+ * _release atomics, an smp_mb() is unconditionally inserted into the
	20	+ * _relaxed variants, which are used to build the barriered versions.
	21	+ * Avoid redundant back-to-back fences in the _acquire and _fence
	22	+ * versions.
23	23	*/
24	24	#define __atomic_acquire_fence()
25	25	#define __atomic_post_full_fence()
@@ -70,7 +70,7 @@ static inline int atomic_##op##_return_relaxed(int i, atomic_t *v) \ · 44 unmodified lines
70	70	".previous" \
71	71	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \
72	72	:"Ir" (i), "m" (v->counter) : "memory"); \
73		− smp_read_barrier_depends(); \
	73	+ smp_mb(); \
74	74	return result; \
75	75	}
76	76
@@ -88,7 +88,7 @@ static inline int atomic_fetch_##op##_relaxed(int i, atomic_t *v) \ · 11 unmodified lines
88	88	".previous" \
89	89	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \
90	90	:"Ir" (i), "m" (v->counter) : "memory"); \
91		− smp_read_barrier_depends(); \
	91	+ smp_mb(); \
92	92	return result; \
93	93	}
94	94
@@ -123,7 +123,7 @@ static __inline__ s64 atomic64_##op##_return_relaxed(s64 i, atomic64_t * v) \ · 28 unmodified lines
123	123	".previous" \
124	124	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \
125	125	:"Ir" (i), "m" (v->counter) : "memory"); \
126		− smp_read_barrier_depends(); \
	126	+ smp_mb(); \
127	127	return result; \
128	128	}
129	129
@@ -141,7 +141,7 @@ static __inline__ s64 atomic64_fetch_##op##_relaxed(s64 i, atomic64_t * v) \ · 11 unmodified lines
141	141	".previous" \
142	142	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \
143	143	:"Ir" (i), "m" (v->counter) : "memory"); \
144		− smp_read_barrier_depends(); \
	144	+ smp_mb(); \
145	145	return result; \
146	146	}
147	147

arch/alpha/include/asm/barrier.h+5−54

@@ -2,64 +2,15 @@ · 1 unmodified line
2	2	#ifndef __BARRIER_H
3	3	#define __BARRIER_H
4	4
5		−#include <asm/compiler.h>
6		−
7	5	#define mb() __asm__ __volatile__("mb": : :"memory")
8	6	#define rmb() __asm__ __volatile__("mb": : :"memory")
9	7	#define wmb() __asm__ __volatile__("wmb": : :"memory")
10	8
11		−/**
12		− * read_barrier_depends - Flush all pending reads that subsequents reads
13		− * depend on.
14		− *
15		− * No data-dependent reads from memory-like regions are ever reordered
16		− * over this barrier. All reads preceding this primitive are guaranteed
17		− * to access memory (but not necessarily other CPUs' caches) before any
18		− * reads following this primitive that depend on the data return by
19		− * any of the preceding reads. This primitive is much lighter weight than
20		− * rmb() on most CPUs, and is never heavier weight than is
21		− * rmb().
22		− *
23		− * These ordering constraints are respected by both the local CPU
24		− * and the compiler.
25		− *
26		− * Ordering is not guaranteed by anything other than these primitives,
27		− * not even by data dependencies. See the documentation for
28		− * memory_barrier() for examples and URLs to more information.
29		− *
30		− * For example, the following code would force ordering (the initial
31		− * value of "a" is zero, "b" is one, and "p" is "&a"):
32		− *
33		− * <programlisting>
34		− * CPU 0 CPU 1
35		− *
36		− * b = 2;
37		− * memory_barrier();
38		− * p = &b; q = p;
39		− * read_barrier_depends();
40		− * d = *q;
41		− * </programlisting>
42		− *
43		− * because the read of "*q" depends on the read of "p" and these
44		− * two reads are separated by a read_barrier_depends(). However,
45		− * the following code, with the same initial values for "a" and "b":
46		− *
47		− * <programlisting>
48		− * CPU 0 CPU 1
49		− *
50		− * a = 2;
51		− * memory_barrier();
52		− * b = 3; y = b;
53		− * read_barrier_depends();
54		− * x = a;
55		− * </programlisting>
56		− *
57		− * does not enforce ordering, since there is no data dependency between
58		− * the read of "a" and the read of "b". Therefore, on some CPUs, such
59		− * as Alpha, "y" could be set to 3 and "x" to 0. Use rmb()
60		− * in cases like this where there are no data dependencies.
61		− */
62		−#define read_barrier_depends() __asm__ __volatile__("mb": : :"memory")
	9	+#define __smp_load_acquire(p) \
	10	+({ \
	11	+ compiletime_assert_atomic_type(*p); \
	12	+ __READ_ONCE(*p); \
	13	+})
63	14
64	15	#ifdef CONFIG_SMP
65	16	#define __ASM_SMP_MB "\tmb\n"

arch/alpha/include/asm/pgtable.h+5−5

@@ -277,9 +277,9 @@ extern inline pte_t pte_mkdirty(pte_t pte) { pte_val(pte) \|= __DIRTY_BITS; retur · 276 unmodified lines
277	277	extern inline pte_t pte_mkyoung(pte_t pte) { pte_val(pte) \|= __ACCESS_BITS; return pte; }
278	278
279	279	/*
280		− * The smp_read_barrier_depends() in the following functions are required to
281		− * order the load of *dir (the pointer in the top level page table) with any
282		− * subsequent load of the returned pmd_t ret (ret is data dependent on dir).
	280	+ * The smp_rmb() in the following functions are required to order the load of
	281	+ * *dir (the pointer in the top level page table) with any subsequent load of
	282	+ * the returned pmd_t ret (ret is data dependent on dir).
283	283	*
284	284	* If this ordering is not enforced, the CPU might load an older value of
285	285	* *ret, which may be uninitialized data. See mm/memory.c:__pte_alloc for
@@ -293,7 +293,7 @@ extern inline pte_t pte_mkyoung(pte_t pte) { pte_val(pte) \|= __ACCESS_BITS; retu · 7 unmodified lines
293	293	extern inline pmd_t * pmd_offset(pud_t * dir, unsigned long address)
294	294	{
295	295	pmd_t ret = (pmd_t ) pud_page_vaddr(*dir) + ((address >> PMD_SHIFT) & (PTRS_PER_PAGE - 1));
296		− smp_read_barrier_depends(); /* see above */
	296	+ smp_rmb(); /* see above */
297	297	return ret;
298	298	}
299	299	#define pmd_offset pmd_offset
@@ -303,7 +303,7 @@ extern inline pte_t * pte_offset_kernel(pmd_t * dir, unsigned long address) · 3 unmodified lines
303	303	{
304	304	pte_t ret = (pte_t ) pmd_page_vaddr(*dir)
305	305	+ ((address >> PAGE_SHIFT) & (PTRS_PER_PAGE - 1));
306		− smp_read_barrier_depends(); /* see above */
	306	+ smp_rmb(); /* see above */
307	307	return ret;
308	308	}
309	309	#define pte_offset_kernel pte_offset_kernel

arch/alpha/include/asm/rwonce.h+35−0

@@ -0,0 +1,35 @@
	1	+/* SPDX-License-Identifier: GPL-2.0 */
	2	+/*
	3	+ * Copyright (C) 2019 Google LLC.
	4	+ */
	5	+#ifndef __ASM_RWONCE_H
	6	+#define __ASM_RWONCE_H
	7	+
	8	+#ifdef CONFIG_SMP
	9	+
	10	+#include <asm/barrier.h>
	11	+
	12	+/*
	13	+ * Alpha is apparently daft enough to reorder address-dependent loads
	14	+ * on some CPU implementations. Knock some common sense into it with
	15	+ * a memory barrier in READ_ONCE().
	16	+ *
	17	+ * For the curious, more information about this unusual reordering is
	18	+ * available in chapter 15 of the "perfbook":
	19	+ *
	20	+ * https://kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html
	21	+ *
	22	+ */
	23	+#define __READ_ONCE(x) \
	24	+({ \
	25	+ __unqual_scalar_typeof(x) __x = \
	26	+ ((volatile typeof(__x) )(&(x))); \
	27	+ mb(); \
	28	+ (typeof(x))__x; \
	29	+})
	30	+
	31	+#endif /* CONFIG_SMP */
	32	+
	33	+#include <asm-generic/rwonce.h>
	34	+
	35	+#endif /* __ASM_RWONCE_H */

arch/arm/include/asm/vdso/gettimeofday.h+1−0

@@ -7,6 +7,7 @@ · 6 unmodified lines
7	7
8	8	#ifndef __ASSEMBLY__
9	9
	10	+#include <asm/barrier.h>
10	11	#include <asm/errno.h>
11	12	#include <asm/unistd.h>
12	13	#include <asm/vdso/cp15.h>

arch/arm64/Kconfig+21−2

@@ -118,6 +118,7 @@ config ARM64 · 117 unmodified lines
118	118	select GENERIC_STRNLEN_USER
119	119	select GENERIC_TIME_VSYSCALL
120	120	select GENERIC_GETTIMEOFDAY
	121	+ select GENERIC_VDSO_TIME_NS
121	122	select HANDLE_DOMAIN_IRQ
122	123	select HARDIRQS_SW_RESEND
123	124	select HAVE_PCI
@@ -1327,6 +1328,8 @@ config SWP_EMULATION · 1203 unmodified lines
1327	1328	ARMv8 obsoletes the use of A32 SWP/SWPB instructions such that
1328	1329	they are always undefined. Say Y here to enable software
1329	1330	emulation of these instructions for userspace using LDXR/STXR.
	1331	+ This feature can be controlled at runtime with the abi.swp
	1332	+ sysctl which is disabled by default.
1330	1333
1331	1334	In some older versions of glibc [<=2.8] SWP is used during futex
1332	1335	trylock() operations with the assumption that the code will not
@@ -1353,7 +1356,8 @@ config CP15_BARRIER_EMULATION · 20 unmodified lines
1353	1356	Say Y here to enable software emulation of these
1354	1357	instructions for AArch32 userspace code. When this option is
1355	1358	enabled, CP15 barrier usage is traced which can help
1356		− identify software that needs updating.
	1359	+ identify software that needs updating. This feature can be
	1360	+ controlled at runtime with the abi.cp15_barrier sysctl.
1357	1361
1358	1362	If unsure, say Y
1359	1363
@@ -1364,7 +1368,8 @@ config SETEND_EMULATION · 4 unmodified lines
1364	1368	AArch32 EL0, and is deprecated in ARMv8.
1365	1369
1366	1370	Say Y here to enable software emulation of the instruction
1367		− for AArch32 userspace code.
	1371	+ for AArch32 userspace code. This feature can be controlled
	1372	+ at runtime with the abi.setend sysctl.
1368	1373
1369	1374	Note: All the cpus on the system must have mixed endian support at EL0
1370	1375	for this feature to be enabled. If a new CPU - which doesn't support mixed
@@ -1596,6 +1601,20 @@ config ARM64_AMU_EXTN · 225 unmodified lines
1596	1601	correctly reflect reality. Most commonly, the value read will be 0,
1597	1602	indicating that the counter is not enabled.
1598	1603
	1604	+config AS_HAS_ARMV8_4
	1605	+ def_bool $(cc-option,-Wa$(comma)-march=armv8.4-a)
	1606	+
	1607	+config ARM64_TLB_RANGE
	1608	+ bool "Enable support for tlbi range feature"
	1609	+ default y
	1610	+ depends on AS_HAS_ARMV8_4
	1611	+ help
	1612	+ ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a
	1613	+ range of input addresses.
	1614	+
	1615	+ The feature introduces new assembly instructions, and they were
	1616	+ support when binutils >= 2.30.
	1617	+
1599	1618	endmenu
1600	1619
1601	1620	menu "ARMv8.5 architectural features"

arch/arm64/Makefile+7−0

@@ -82,11 +82,18 @@ endif · 81 unmodified lines
82	82	# compiler to generate them and consequently to break the single image contract
83	83	# we pass it only to the assembler. This option is utilized only in case of non
84	84	# integrated assemblers.
	85	+ifneq ($(CONFIG_AS_HAS_ARMV8_4), y)
85	86	branch-prot-flags-$(CONFIG_AS_HAS_PAC) += -Wa,-march=armv8.3-a
86	87	endif
	88	+endif
87	89
88	90	KBUILD_CFLAGS += $(branch-prot-flags-y)
89	91
	92	+ifeq ($(CONFIG_AS_HAS_ARMV8_4), y)
	93	+# make sure to pass the newest target architecture to -march.
	94	+KBUILD_CFLAGS += -Wa,-march=armv8.4-a
	95	+endif
	96	+
90	97	ifeq ($(CONFIG_SHADOW_CALL_STACK), y)
91	98	KBUILD_CFLAGS += -ffixed-x18
92	99	endif

arch/arm64/configs/defconfig+1−0

@@ -66,6 +66,7 @@ CONFIG_SCHED_SMT=y · 65 unmodified lines
66	66	CONFIG_NUMA=y
67	67	CONFIG_SECCOMP=y
68	68	CONFIG_KEXEC=y
	69	+CONFIG_KEXEC_FILE=y
69	70	CONFIG_CRASH_DUMP=y
70	71	CONFIG_XEN=y
71	72	CONFIG_COMPAT=y

arch/arm64/include/asm/acpi.h+1−14

@@ -47,20 +47,7 @@ · 46 unmodified lines
47	47	pgprot_t __acpi_get_mem_attribute(phys_addr_t addr);
48	48
49	49	/* ACPI table mapping after acpi_permanent_mmap is set */
50		−static inline void __iomem *acpi_os_ioremap(acpi_physical_address phys,
51		− acpi_size size)
52		−{
53		− /* For normal memory we already have a cacheable mapping. */
54		− if (memblock_is_map_memory(phys))
55		− return (void __iomem *)__phys_to_virt(phys);
56		−
57		− /*
58		− * We should still honor the memory's attribute here because
59		− * crash dump kernel possibly excludes some ACPI (reclaim)
60		− * regions from memblock list.
61		− */
62		− return __ioremap(phys, size, __acpi_get_mem_attribute(phys));
63		−}
	50	+void __iomem *acpi_os_ioremap(acpi_physical_address phys, acpi_size size);
64	51	#define acpi_os_ioremap acpi_os_ioremap
65	52
66	53	typedef u64 phys_cpuid_t;

arch/arm64/include/asm/cpucaps.h+3−1

@@ -62,7 +62,9 @@ · 61 unmodified lines
62	62	#define ARM64_HAS_GENERIC_AUTH 52
63	63	#define ARM64_HAS_32BIT_EL1 53
64	64	#define ARM64_BTI 54
	65	+#define ARM64_HAS_ARMv8_4_TTL 55
	66	+#define ARM64_HAS_TLB_RANGE 56
65	67
66		−#define ARM64_NCAPS 55
	68	+#define ARM64_NCAPS 57
67	69
68	70	#endif /* __ASM_CPUCAPS_H */

arch/arm64/include/asm/cpufeature.h+7−0

@@ -692,6 +692,12 @@ static inline bool system_supports_bti(void) · 691 unmodified lines
692	692	return IS_ENABLED(CONFIG_ARM64_BTI) && cpus_have_const_cap(ARM64_BTI);
693	693	}
694	694
	695	+static inline bool system_supports_tlb_range(void)
	696	+{
	697	+ return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
	698	+ cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
	699	+}
	700	+
695	701	#define ARM64_BP_HARDEN_UNKNOWN -1
696	702	#define ARM64_BP_HARDEN_WA_NEEDED 0
697	703	#define ARM64_BP_HARDEN_NOT_REQUIRED 1
@@ -774,6 +780,7 @@ static inline unsigned int get_vmid_bits(u64 mmfr1) · 76 unmodified lines
774	780	}
775	781
776	782	u32 get_kvm_ipa_limit(void);
	783	+void dump_cpu_features(void);
777	784
778	785	#endif /* __ASSEMBLY__ */
779	786

arch/arm64/include/asm/hugetlb.h+2−0

@@ -49,6 +49,8 @@ extern void set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr, · 48 unmodified lines
49	49	pte_t *ptep, pte_t pte, unsigned long sz);
50	50	#define set_huge_swap_pte_at set_huge_swap_pte_at
51	51
	52	+void __init arm64_hugetlb_cma_reserve(void);
	53	+
52	54	#include <asm-generic/hugetlb.h>
53	55
54	56	#endif /* __ASM_HUGETLB_H */

arch/arm64/include/asm/hwcap.h+1−0

@@ -95,6 +95,7 @@ · 94 unmodified lines
95	95	#define KERNEL_HWCAP_DGH __khwcap2_feature(DGH)
96	96	#define KERNEL_HWCAP_RNG __khwcap2_feature(RNG)
97	97	#define KERNEL_HWCAP_BTI __khwcap2_feature(BTI)
	98	+/* reserved for KERNEL_HWCAP_MTE __khwcap2_feature(MTE) */
98	99
99	100	/*
100	101	* This yields a mask that user programs can use to figure out what

arch/arm64/include/asm/kernel-pgtable.h+1−1

@@ -8,7 +8,7 @@ · 7 unmodified lines
8	8	#ifndef __ASM_KERNEL_PGTABLE_H
9	9	#define __ASM_KERNEL_PGTABLE_H
10	10
11		−#include <linux/pgtable.h>
	11	+#include <asm/pgtable-hwdef.h>
12	12	#include <asm/sparsemem.h>
13	13
14	14	/*

arch/arm64/include/asm/memory.h+7−5

@@ -10,11 +10,8 @@ · 9 unmodified lines
10	10	#ifndef __ASM_MEMORY_H
11	11	#define __ASM_MEMORY_H
12	12
13		−#include <linux/compiler.h>
14	13	#include <linux/const.h>
15	14	#include <linux/sizes.h>
16		−#include <linux/types.h>
17		−#include <asm/bug.h>
18	15	#include <asm/page-def.h>
19	16
20	17	/*
@@ -157,11 +154,15 @@ · 136 unmodified lines
157	154	#endif
158	155
159	156	#ifndef __ASSEMBLY__
160		−extern u64 vabits_actual;
161		−#define PAGE_END (_PAGE_END(vabits_actual))
162	157
163	158	#include <linux/bitops.h>
	159	+#include <linux/compiler.h>
164	160	#include <linux/mmdebug.h>
	161	+#include <linux/types.h>
	162	+#include <asm/bug.h>
	163	+
	164	+extern u64 vabits_actual;
	165	+#define PAGE_END (_PAGE_END(vabits_actual))
165	166
166	167	extern s64 physvirt_offset;
167	168	extern s64 memstart_addr;
@@ -322,6 +323,7 @@ static inline void *phys_to_virt(phys_addr_t x) · 154 unmodified lines
322	323	__is_lm_address(__addr) && pfn_valid(virt_to_pfn(__addr)); \
323	324	})
324	325
	326	+void dump_mem_limit(void);
325	327	#endif /* !ASSEMBLY */
326	328
327	329	/*

arch/arm64/include/asm/mmu_context.h+2−4

@@ -175,7 +175,7 @@ static inline void cpu_replace_ttbr1(pgd_t *pgdp) · 174 unmodified lines
175	175	* take CPU migration into account.
176	176	*/
177	177	#define destroy_context(mm) do { } while(0)
178		−void check_and_switch_context(struct mm_struct *mm, unsigned int cpu);
	178	+void check_and_switch_context(struct mm_struct *mm);
179	179
180	180	#define init_new_context(tsk,mm) ({ atomic64_set(&(mm)->context.id, 0); 0; })
181	181
@@ -214,8 +214,6 @@ enter_lazy_tlb(struct mm_struct mm, struct task_struct tsk) · 32 unmodified lines
214	214
215	215	static inline void __switch_mm(struct mm_struct *next)
216	216	{
217		− unsigned int cpu = smp_processor_id();
218		−
219	217	/*
220	218	* init_mm.pgd does not contain any user mappings and it is always
221	219	* active for kernel addresses in TTBR1. Just set the reserved TTBR0.
@@ -225,7 +223,7 @@ static inline void __switch_mm(struct mm_struct *next) · 3 unmodified lines
225	223	return;
226	224	}
227	225
228		− check_and_switch_context(next, cpu);
	226	+ check_and_switch_context(next);
229	227	}
230	228
231	229	static inline void

arch/arm64/include/asm/perf_event.h+27−0

@@ -72,6 +72,13 @@ · 71 unmodified lines
72	72	#define ARMV8_PMUV3_PERFCTR_LL_CACHE_RD 0x36
73	73	#define ARMV8_PMUV3_PERFCTR_LL_CACHE_MISS_RD 0x37
74	74	#define ARMV8_PMUV3_PERFCTR_REMOTE_ACCESS_RD 0x38
	75	+#define ARMV8_PMUV3_PERFCTR_L1D_CACHE_LMISS_RD 0x39
	76	+#define ARMV8_PMUV3_PERFCTR_OP_RETIRED 0x3A
	77	+#define ARMV8_PMUV3_PERFCTR_OP_SPEC 0x3B
	78	+#define ARMV8_PMUV3_PERFCTR_STALL 0x3C
	79	+#define ARMV8_PMUV3_PERFCTR_STALL_SLOT_BACKEND 0x3D
	80	+#define ARMV8_PMUV3_PERFCTR_STALL_SLOT_FRONTEND 0x3E
	81	+#define ARMV8_PMUV3_PERFCTR_STALL_SLOT 0x3F
75	82
76	83	/* Statistical profiling extension microarchitectural events */
77	84	#define ARMV8_SPE_PERFCTR_SAMPLE_POP 0x4000
@@ -79,6 +86,26 @@ · 1 unmodified line
79	86	#define ARMV8_SPE_PERFCTR_SAMPLE_FILTRATE 0x4002
80	87	#define ARMV8_SPE_PERFCTR_SAMPLE_COLLISION 0x4003
81	88
	89	+/* AMUv1 architecture events */
	90	+#define ARMV8_AMU_PERFCTR_CNT_CYCLES 0x4004
	91	+#define ARMV8_AMU_PERFCTR_STALL_BACKEND_MEM 0x4005
	92	+
	93	+/* long-latency read miss events */
	94	+#define ARMV8_PMUV3_PERFCTR_L1I_CACHE_LMISS 0x4006
	95	+#define ARMV8_PMUV3_PERFCTR_L2D_CACHE_LMISS_RD 0x4009
	96	+#define ARMV8_PMUV3_PERFCTR_L2I_CACHE_LMISS 0x400A
	97	+#define ARMV8_PMUV3_PERFCTR_L3D_CACHE_LMISS_RD 0x400B
	98	+
	99	+/* additional latency from alignment events */
	100	+#define ARMV8_PMUV3_PERFCTR_LDST_ALIGN_LAT 0x4020
	101	+#define ARMV8_PMUV3_PERFCTR_LD_ALIGN_LAT 0x4021
	102	+#define ARMV8_PMUV3_PERFCTR_ST_ALIGN_LAT 0x4022
	103	+
	104	+/* Armv8.5 Memory Tagging Extension events */
	105	+#define ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED 0x4024
	106	+#define ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED_RD 0x4025
	107	+#define ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED_WR 0x4026
	108	+
82	109	/* ARMv8 recommended implementation defined event types */
83	110	#define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_RD 0x40
84	111	#define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR 0x41

arch/arm64/include/asm/pgtable-hwdef.h+13−10

@@ -29,7 +29,7 @@ · 28 unmodified lines
29	29	* Size mapped by an entry at level n ( 0 <= n <= 3)
30	30	* We map (PAGE_SHIFT - 3) at all translation levels and PAGE_SHIFT bits
31	31	* in the final page. The maximum number of translation levels supported by
32		− * the architecture is 4. Hence, starting at at level n, we have further
	32	+ * the architecture is 4. Hence, starting at level n, we have further
33	33	* ((4 - n) - 1) levels of translation excluding the offset within the page.
34	34	* So, the total number of bits mapped by an entry at level n is :
35	35	*
@@ -82,23 +82,23 @@ · 46 unmodified lines
82	82	* Contiguous page definitions.
83	83	*/
84	84	#ifdef CONFIG_ARM64_64K_PAGES
85		−#define CONT_PTE_SHIFT 5
86		−#define CONT_PMD_SHIFT 5
	85	+#define CONT_PTE_SHIFT (5 + PAGE_SHIFT)
	86	+#define CONT_PMD_SHIFT (5 + PMD_SHIFT)
87	87	#elif defined(CONFIG_ARM64_16K_PAGES)
88		−#define CONT_PTE_SHIFT 7
89		−#define CONT_PMD_SHIFT 5
	88	+#define CONT_PTE_SHIFT (7 + PAGE_SHIFT)
	89	+#define CONT_PMD_SHIFT (5 + PMD_SHIFT)
90	90	#else
91		−#define CONT_PTE_SHIFT 4
92		−#define CONT_PMD_SHIFT 4
	91	+#define CONT_PTE_SHIFT (4 + PAGE_SHIFT)
	92	+#define CONT_PMD_SHIFT (4 + PMD_SHIFT)
93	93	#endif
94	94
95		−#define CONT_PTES (1 << CONT_PTE_SHIFT)
	95	+#define CONT_PTES (1 << (CONT_PTE_SHIFT - PAGE_SHIFT))
96	96	#define CONT_PTE_SIZE (CONT_PTES * PAGE_SIZE)
97	97	#define CONT_PTE_MASK (~(CONT_PTE_SIZE - 1))
98		−#define CONT_PMDS (1 << CONT_PMD_SHIFT)
	98	+#define CONT_PMDS (1 << (CONT_PMD_SHIFT - PMD_SHIFT))
99	99	#define CONT_PMD_SIZE (CONT_PMDS * PMD_SIZE)
100	100	#define CONT_PMD_MASK (~(CONT_PMD_SIZE - 1))
101		−/* the the numerical offset of the PTE within a range of CONT_PTES */
	101	+/* the numerical offset of the PTE within a range of CONT_PTES */
102	102	#define CONT_RANGE_OFFSET(addr) (((addr)>>PAGE_SHIFT)&(CONT_PTES-1))
103	103
104	104	/*
@@ -178,10 +178,12 @@ · 73 unmodified lines
178	178	#define PTE_S2_RDONLY (_AT(pteval_t, 1) << 6) /* HAP[2:1] */
179	179	#define PTE_S2_RDWR (_AT(pteval_t, 3) << 6) /* HAP[2:1] */
180	180	#define PTE_S2_XN (_AT(pteval_t, 2) << 53) /* XN[1:0] */
	181	+#define PTE_S2_SW_RESVD (_AT(pteval_t, 15) << 55) /* Reserved for SW */
181	182
182	183	#define PMD_S2_RDONLY (_AT(pmdval_t, 1) << 6) /* HAP[2:1] */
183	184	#define PMD_S2_RDWR (_AT(pmdval_t, 3) << 6) /* HAP[2:1] */
184	185	#define PMD_S2_XN (_AT(pmdval_t, 2) << 53) /* XN[1:0] */
	186	+#define PMD_S2_SW_RESVD (_AT(pmdval_t, 15) << 55) /* Reserved for SW */
185	187
186	188	#define PUD_S2_RDONLY (_AT(pudval_t, 1) << 6) /* HAP[2:1] */
187	189	#define PUD_S2_RDWR (_AT(pudval_t, 3) << 6) /* HAP[2:1] */
@@ -216,6 +218,7 @@ · 28 unmodified lines
216	218	#define TCR_TxSZ(x) (TCR_T0SZ(x) \| TCR_T1SZ(x))
217	219	#define TCR_TxSZ_WIDTH 6
218	220	#define TCR_T0SZ_MASK (((UL(1) << TCR_TxSZ_WIDTH) - 1) << TCR_T0SZ_OFFSET)
	221	+#define TCR_T1SZ_MASK (((UL(1) << TCR_TxSZ_WIDTH) - 1) << TCR_T1SZ_OFFSET)
219	222
220	223	#define TCR_EPD0_SHIFT 7
221	224	#define TCR_EPD0_MASK (UL(1) << TCR_EPD0_SHIFT)

arch/arm64/include/asm/pgtable.h+10−0

@@ -40,6 +40,16 @@ extern void __pmd_error(const char *file, int line, unsigned long val); · 39 unmodified lines
40	40	extern void __pud_error(const char *file, int line, unsigned long val);
41	41	extern void __pgd_error(const char *file, int line, unsigned long val);
42	42
	43	+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
	44	+#define __HAVE_ARCH_FLUSH_PMD_TLB_RANGE
	45	+
	46	+/* Set stride and tlb_level in flush__tlb_range /
	47	+#define flush_pmd_tlb_range(vma, addr, end) \
	48	+ __flush_tlb_range(vma, addr, end, PMD_SIZE, false, 2)
	49	+#define flush_pud_tlb_range(vma, addr, end) \
	50	+ __flush_tlb_range(vma, addr, end, PUD_SIZE, false, 1)
	51	+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
	52	+
43	53	/*
44	54	* ZERO_PAGE is a global shared page that is always zero: used
45	55	* for zero-mapped memory areas etc..

arch/arm64/include/asm/ptrace.h+1−1

@@ -27,7 +27,7 @@ · 26 unmodified lines
27	27	*
28	28	* Some code sections either automatically switch back to PSR.I or explicitly
29	29	* require to not use priority masking. If bit GIC_PRIO_PSR_I_SET is included
30		− * in the the priority mask, it indicates that PSR.I should be set and
	30	+ * in the priority mask, it indicates that PSR.I should be set and
31	31	* interrupt disabling temporarily does not rely on IRQ priorities.
32	32	*/
33	33	#define GIC_PRIO_IRQON 0xe0

arch/arm64/include/asm/stage2_pgtable.h+9−0

@@ -256,4 +256,13 @@ stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end) · 255 unmodified lines
256	256	return (boundary - 1 < end - 1) ? boundary : end;
257	257	}
258	258
	259	+/*
	260	+ * Level values for the ARMv8.4-TTL extension, mapping PUD/PMD/PTE and
	261	+ * the architectural page-table level.
	262	+ */
	263	+#define S2_NO_LEVEL_HINT 0
	264	+#define S2_PUD_LEVEL 1
	265	+#define S2_PMD_LEVEL 2
	266	+#define S2_PTE_LEVEL 3
	267	+
259	268	#endif /* __ARM64_S2_PGTABLE_H_ */

arch/arm64/include/asm/sysreg.h+47−2

@@ -421,9 +421,9 @@ · 420 unmodified lines
421	421	*/
422	422
423	423	#define SYS_AMEVCNTR0_EL0(n) SYS_AM_EL0(4 + ((n) >> 3), (n) & 7)
424		−#define SYS_AMEVTYPE0_EL0(n) SYS_AM_EL0(6 + ((n) >> 3), (n) & 7)
	424	+#define SYS_AMEVTYPER0_EL0(n) SYS_AM_EL0(6 + ((n) >> 3), (n) & 7)
425	425	#define SYS_AMEVCNTR1_EL0(n) SYS_AM_EL0(12 + ((n) >> 3), (n) & 7)
426		−#define SYS_AMEVTYPE1_EL0(n) SYS_AM_EL0(14 + ((n) >> 3), (n) & 7)
	426	+#define SYS_AMEVTYPER1_EL0(n) SYS_AM_EL0(14 + ((n) >> 3), (n) & 7)
427	427
428	428	/* AMU v1: Fixed (architecturally defined) activity monitors */
429	429	#define SYS_AMEVCNTR0_CORE_EL0 SYS_AMEVCNTR0_EL0(0)
@@ -617,6 +617,9 @@ · 187 unmodified lines
617	617	#define ID_AA64ISAR0_SHA1_SHIFT 8
618	618	#define ID_AA64ISAR0_AES_SHIFT 4
619	619
	620	+#define ID_AA64ISAR0_TLB_RANGE_NI 0x0
	621	+#define ID_AA64ISAR0_TLB_RANGE 0x2
	622	+
620	623	/* id_aa64isar1 */
621	624	#define ID_AA64ISAR1_I8MM_SHIFT 52
622	625	#define ID_AA64ISAR1_DGH_SHIFT 48
@@ -706,6 +709,9 @@ · 83 unmodified lines
706	709	#define ID_AA64ZFR0_SVEVER_SVE2 0x1
707	710
708	711	/* id_aa64mmfr0 */
	712	+#define ID_AA64MMFR0_ECV_SHIFT 60
	713	+#define ID_AA64MMFR0_FGT_SHIFT 56
	714	+#define ID_AA64MMFR0_EXS_SHIFT 44
709	715	#define ID_AA64MMFR0_TGRAN4_2_SHIFT 40
710	716	#define ID_AA64MMFR0_TGRAN64_2_SHIFT 36
711	717	#define ID_AA64MMFR0_TGRAN16_2_SHIFT 32
@@ -734,6 +740,10 @@ · 22 unmodified lines
734	740	#endif
735	741
736	742	/* id_aa64mmfr1 */
	743	+#define ID_AA64MMFR1_ETS_SHIFT 36
	744	+#define ID_AA64MMFR1_TWED_SHIFT 32
	745	+#define ID_AA64MMFR1_XNX_SHIFT 28
	746	+#define ID_AA64MMFR1_SPECSEI_SHIFT 24
737	747	#define ID_AA64MMFR1_PAN_SHIFT 20
738	748	#define ID_AA64MMFR1_LOR_SHIFT 16
739	749	#define ID_AA64MMFR1_HPD_SHIFT 12
@@ -746,8 +756,15 @@ · 6 unmodified lines
746	756
747	757	/* id_aa64mmfr2 */
748	758	#define ID_AA64MMFR2_E0PD_SHIFT 60
	759	+#define ID_AA64MMFR2_EVT_SHIFT 56
	760	+#define ID_AA64MMFR2_BBM_SHIFT 52
	761	+#define ID_AA64MMFR2_TTL_SHIFT 48
749	762	#define ID_AA64MMFR2_FWB_SHIFT 40
	763	+#define ID_AA64MMFR2_IDS_SHIFT 36
750	764	#define ID_AA64MMFR2_AT_SHIFT 32
	765	+#define ID_AA64MMFR2_ST_SHIFT 28
	766	+#define ID_AA64MMFR2_NV_SHIFT 24
	767	+#define ID_AA64MMFR2_CCIDX_SHIFT 20
751	768	#define ID_AA64MMFR2_LVA_SHIFT 16
752	769	#define ID_AA64MMFR2_IESB_SHIFT 12
753	770	#define ID_AA64MMFR2_LSM_SHIFT 8
@@ -755,6 +772,7 @@ · 1 unmodified line
755	772	#define ID_AA64MMFR2_CNP_SHIFT 0
756	773
757	774	/* id_aa64dfr0 */
	775	+#define ID_AA64DFR0_DOUBLELOCK_SHIFT 36
758	776	#define ID_AA64DFR0_PMSVER_SHIFT 32
759	777	#define ID_AA64DFR0_CTX_CMPS_SHIFT 28
760	778	#define ID_AA64DFR0_WRPS_SHIFT 20
@@ -807,18 +825,40 @@ · 46 unmodified lines
807	825	#define ID_ISAR6_DP_SHIFT 4
808	826	#define ID_ISAR6_JSCVT_SHIFT 0
809	827
	828	+#define ID_MMFR0_INNERSHR_SHIFT 28
	829	+#define ID_MMFR0_FCSE_SHIFT 24
	830	+#define ID_MMFR0_AUXREG_SHIFT 20
	831	+#define ID_MMFR0_TCM_SHIFT 16
	832	+#define ID_MMFR0_SHARELVL_SHIFT 12
	833	+#define ID_MMFR0_OUTERSHR_SHIFT 8
	834	+#define ID_MMFR0_PMSA_SHIFT 4
	835	+#define ID_MMFR0_VMSA_SHIFT 0
	836	+
810	837	#define ID_MMFR4_EVT_SHIFT 28
811	838	#define ID_MMFR4_CCIDX_SHIFT 24
812	839	#define ID_MMFR4_LSM_SHIFT 20
813	840	#define ID_MMFR4_HPDS_SHIFT 16
814	841	#define ID_MMFR4_CNP_SHIFT 12
815	842	#define ID_MMFR4_XNX_SHIFT 8
	843	+#define ID_MMFR4_AC2_SHIFT 4
816	844	#define ID_MMFR4_SPECSEI_SHIFT 0
817	845
818	846	#define ID_MMFR5_ETS_SHIFT 0
819	847
820	848	#define ID_PFR0_DIT_SHIFT 24
821	849	#define ID_PFR0_CSV2_SHIFT 16
	850	+#define ID_PFR0_STATE3_SHIFT 12
	851	+#define ID_PFR0_STATE2_SHIFT 8
	852	+#define ID_PFR0_STATE1_SHIFT 4
	853	+#define ID_PFR0_STATE0_SHIFT 0
	854	+
	855	+#define ID_DFR0_PERFMON_SHIFT 24
	856	+#define ID_DFR0_MPROFDBG_SHIFT 20
	857	+#define ID_DFR0_MMAPTRC_SHIFT 16
	858	+#define ID_DFR0_COPTRC_SHIFT 12
	859	+#define ID_DFR0_MMAPDBG_SHIFT 8
	860	+#define ID_DFR0_COPSDBG_SHIFT 4
	861	+#define ID_DFR0_COPDBG_SHIFT 0
822	862
823	863	#define ID_PFR2_SSBS_SHIFT 4
824	864	#define ID_PFR2_CSV3_SHIFT 0
@@ -861,6 +901,11 @@ · 36 unmodified lines
861	901	#define ID_AA64MMFR0_TGRAN_SUPPORTED ID_AA64MMFR0_TGRAN64_SUPPORTED
862	902	#endif
863	903
	904	+#define MVFR2_FPMISC_SHIFT 4
	905	+#define MVFR2_SIMDMISC_SHIFT 0
	906	+
	907	+#define DCZID_DZP_SHIFT 4
	908	+#define DCZID_BS_SHIFT 0
864	909
865	910	/*
866	911	* The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which

arch/arm64/include/asm/tlb.h+28−1

@@ -21,11 +21,37 @@ static void tlb_flush(struct mmu_gather *tlb); · 20 unmodified lines
21	21
22	22	#include <asm-generic/tlb.h>
23	23
	24	+/*
	25	+ * get the tlbi levels in arm64. Default value is 0 if more than one
	26	+ * of cleared_* is set or neither is set.
	27	+ * Arm64 doesn't support p4ds now.
	28	+ */
	29	+static inline int tlb_get_level(struct mmu_gather *tlb)
	30	+{
	31	+ if (tlb->cleared_ptes && !(tlb->cleared_pmds \|\|
	32	+ tlb->cleared_puds \|\|
	33	+ tlb->cleared_p4ds))
	34	+ return 3;
	35	+
	36	+ if (tlb->cleared_pmds && !(tlb->cleared_ptes \|\|
	37	+ tlb->cleared_puds \|\|
	38	+ tlb->cleared_p4ds))
	39	+ return 2;
	40	+
	41	+ if (tlb->cleared_puds && !(tlb->cleared_ptes \|\|
	42	+ tlb->cleared_pmds \|\|
	43	+ tlb->cleared_p4ds))
	44	+ return 1;
	45	+
	46	+ return 0;
	47	+}
	48	+
24	49	static inline void tlb_flush(struct mmu_gather *tlb)
25	50	{
26	51	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
27	52	bool last_level = !tlb->freed_tables;
28	53	unsigned long stride = tlb_get_unmap_size(tlb);
	54	+ int tlb_level = tlb_get_level(tlb);
29	55
30	56	/*
31	57	* If we're tearing down the address space then we only care about
@@ -38,7 +64,8 @@ static inline void tlb_flush(struct mmu_gather *tlb) · 6 unmodified lines
38	64	return;
39	65	}
40	66
41		− __flush_tlb_range(&vma, tlb->start, tlb->end, stride, last_level);
	67	+ __flush_tlb_range(&vma, tlb->start, tlb->end, stride,
	68	+ last_level, tlb_level);
42	69	}
43	70
44	71	static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,

arch/arm64/include/asm/tlbflush.h+162−15

@@ -10,6 +10,7 @@ · 9 unmodified lines
10	10
11	11	#ifndef __ASSEMBLY__
12	12
	13	+#include <linux/bitfield.h>
13	14	#include <linux/mm_types.h>
14	15	#include <linux/sched.h>
15	16	#include <asm/cputype.h>
@@ -59,6 +60,102 @@ · 43 unmodified lines
59	60	__ta; \
60	61	})
61	62
	63	+/*
	64	+ * Get translation granule of the system, which is decided by
	65	+ * PAGE_SIZE. Used by TTL.
	66	+ * - 4KB : 1
	67	+ * - 16KB : 2
	68	+ * - 64KB : 3
	69	+ */
	70	+#define TLBI_TTL_TG_4K 1
	71	+#define TLBI_TTL_TG_16K 2
	72	+#define TLBI_TTL_TG_64K 3
	73	+
	74	+static inline unsigned long get_trans_granule(void)
	75	+{
	76	+ switch (PAGE_SIZE) {
	77	+ case SZ_4K:
	78	+ return TLBI_TTL_TG_4K;
	79	+ case SZ_16K:
	80	+ return TLBI_TTL_TG_16K;
	81	+ case SZ_64K:
	82	+ return TLBI_TTL_TG_64K;
	83	+ default:
	84	+ return 0;
	85	+ }
	86	+}
	87	+
	88	+/*
	89	+ * Level-based TLBI operations.
	90	+ *
	91	+ * When ARMv8.4-TTL exists, TLBI operations take an additional hint for
	92	+ * the level at which the invalidation must take place. If the level is
	93	+ * wrong, no invalidation may take place. In the case where the level
	94	+ * cannot be easily determined, a 0 value for the level parameter will
	95	+ * perform a non-hinted invalidation.
	96	+ *
	97	+ * For Stage-2 invalidation, use the level values provided to that effect
	98	+ * in asm/stage2_pgtable.h.
	99	+ */
	100	+#define TLBI_TTL_MASK GENMASK_ULL(47, 44)
	101	+
	102	+#define __tlbi_level(op, addr, level) do { \
	103	+ u64 arg = addr; \
	104	+ \
	105	+ if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL) && \
	106	+ level) { \
	107	+ u64 ttl = level & 3; \
	108	+ ttl \|= get_trans_granule() << 2; \
	109	+ arg &= ~TLBI_TTL_MASK; \
	110	+ arg \|= FIELD_PREP(TLBI_TTL_MASK, ttl); \
	111	+ } \
	112	+ \
	113	+ __tlbi(op, arg); \
	114	+} while(0)
	115	+
	116	+#define __tlbi_user_level(op, arg, level) do { \
	117	+ if (arm64_kernel_unmapped_at_el0()) \
	118	+ __tlbi_level(op, (arg \| USER_ASID_FLAG), level); \
	119	+} while (0)
	120	+
	121	+/*
	122	+ * This macro creates a properly formatted VA operand for the TLB RANGE.
	123	+ * The value bit assignments are:
	124	+ *
	125	+ * +----------+------+-------+-------+-------+----------------------+
	126	+ * \| ASID \| TG \| SCALE \| NUM \| TTL \| BADDR \|
	127	+ * +-----------------+-------+-------+-------+----------------------+
	128	+ * \|63 48\|47 46\|45 44\|43 39\|38 37\|36 0\|
	129	+ *
	130	+ * The address range is determined by below formula:
	131	+ * [BADDR, BADDR + (NUM + 1) * 2^(5SCALE + 1) PAGESIZE)
	132	+ *
	133	+ */
	134	+#define __TLBI_VADDR_RANGE(addr, asid, scale, num, ttl) \
	135	+ ({ \
	136	+ unsigned long __ta = (addr) >> PAGE_SHIFT; \
	137	+ __ta &= GENMASK_ULL(36, 0); \
	138	+ __ta \|= (unsigned long)(ttl) << 37; \
	139	+ __ta \|= (unsigned long)(num) << 39; \
	140	+ __ta \|= (unsigned long)(scale) << 44; \
	141	+ __ta \|= get_trans_granule() << 46; \
	142	+ __ta \|= (unsigned long)(asid) << 48; \
	143	+ __ta; \
	144	+ })
	145	+
	146	+/* These macros are used by the TLBI RANGE feature. */
	147	+#define __TLBI_RANGE_PAGES(num, scale) \
	148	+ ((unsigned long)((num) + 1) << (5 * (scale) + 1))
	149	+#define MAX_TLBI_RANGE_PAGES __TLBI_RANGE_PAGES(31, 3)
	150	+
	151	+/*
	152	+ * Generate 'num' values from -1 to 30 with -1 rejected by the
	153	+ * __flush_tlb_range() loop below.
	154	+ */
	155	+#define TLBI_RANGE_MASK GENMASK_ULL(4, 0)
	156	+#define __TLBI_RANGE_NUM(pages, scale) \
	157	+ ((((pages) >> (5 * (scale) + 1)) & TLBI_RANGE_MASK) - 1)
	158	+
62	159	/*
63	160	* TLB Invalidation
64	161	* ================
@@ -179,34 +276,83 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, · 114 unmodified lines
179	276
180	277	static inline void __flush_tlb_range(struct vm_area_struct *vma,
181	278	unsigned long start, unsigned long end,
182		− unsigned long stride, bool last_level)
	279	+ unsigned long stride, bool last_level,
	280	+ int tlb_level)
183	281	{
	282	+ int num = 0;
	283	+ int scale = 0;
184	284	unsigned long asid = ASID(vma->vm_mm);
185	285	unsigned long addr;
	286	+ unsigned long pages;
186	287
187	288	start = round_down(start, stride);
188	289	end = round_up(end, stride);
	290	+ pages = (end - start) >> PAGE_SHIFT;
189	291
190		− if ((end - start) >= (MAX_TLBI_OPS * stride)) {
	292	+ /*
	293	+ * When not uses TLB range ops, we can handle up to
	294	+ * (MAX_TLBI_OPS - 1) pages;
	295	+ * When uses TLB range ops, we can handle up to
	296	+ * (MAX_TLBI_RANGE_PAGES - 1) pages.
	297	+ */
	298	+ if ((!system_supports_tlb_range() &&
	299	+ (end - start) >= (MAX_TLBI_OPS * stride)) \|\|
	300	+ pages >= MAX_TLBI_RANGE_PAGES) {
191	301	flush_tlb_mm(vma->vm_mm);
192	302	return;
193	303	}
194	304
195		− /* Convert the stride into units of 4k */
196		− stride >>= 12;
	305	+ dsb(ishst);
197	306
198		− start = __TLBI_VADDR(start, asid);
199		− end = __TLBI_VADDR(end, asid);
	307	+ /*
	308	+ * When the CPU does not support TLB range operations, flush the TLB
	309	+ * entries one by one at the granularity of 'stride'. If the the TLB
	310	+ * range ops are supported, then:
	311	+ *
	312	+ * 1. If 'pages' is odd, flush the first page through non-range
	313	+ * operations;
	314	+ *
	315	+ * 2. For remaining pages: the minimum range granularity is decided
	316	+ * by 'scale', so multiple range TLBI operations may be required.
	317	+ * Start from scale = 0, flush the corresponding number of pages
	318	+ * ((num+1)2^(5scale+1) starting from 'addr'), then increase it
	319	+ * until no pages left.
	320	+ *
	321	+ * Note that certain ranges can be represented by either num = 31 and
	322	+ * scale or num = 0 and scale + 1. The loop below favours the latter
	323	+ * since num is limited to 30 by the __TLBI_RANGE_NUM() macro.
	324	+ */
	325	+ while (pages > 0) {
	326	+ if (!system_supports_tlb_range() \|\|
	327	+ pages % 2 == 1) {
	328	+ addr = __TLBI_VADDR(start, asid);
	329	+ if (last_level) {
	330	+ __tlbi_level(vale1is, addr, tlb_level);
	331	+ __tlbi_user_level(vale1is, addr, tlb_level);
	332	+ } else {
	333	+ __tlbi_level(vae1is, addr, tlb_level);
	334	+ __tlbi_user_level(vae1is, addr, tlb_level);
	335	+ }
	336	+ start += stride;
	337	+ pages -= stride >> PAGE_SHIFT;
	338	+ continue;
	339	+ }
200	340
201		− dsb(ishst);
202		− for (addr = start; addr < end; addr += stride) {
203		− if (last_level) {
204		− __tlbi(vale1is, addr);
205		− __tlbi_user(vale1is, addr);
206		− } else {
207		− __tlbi(vae1is, addr);
208		− __tlbi_user(vae1is, addr);
	341	+ num = __TLBI_RANGE_NUM(pages, scale);
	342	+ if (num >= 0) {
	343	+ addr = __TLBI_VADDR_RANGE(start, asid, scale,
	344	+ num, tlb_level);
	345	+ if (last_level) {
	346	+ __tlbi(rvale1is, addr);
	347	+ __tlbi_user(rvale1is, addr);
	348	+ } else {
	349	+ __tlbi(rvae1is, addr);
	350	+ __tlbi_user(rvae1is, addr);
	351	+ }
	352	+ start += __TLBI_RANGE_PAGES(num, scale) << PAGE_SHIFT;
	353	+ pages -= __TLBI_RANGE_PAGES(num, scale);
209	354	}
	355	+ scale++;
210	356	}
211	357	dsb(ish);
212	358	}
@@ -217,8 +363,9 @@ static inline void flush_tlb_range(struct vm_area_struct *vma, · 4 unmodified lines
217	363	/*
218	364	* We cannot use leaf-only invalidation here, since we may be invalidating
219	365	* table entries as part of collapsing hugepages or moving page tables.
	366	+ * Set the tlb_level to 0 because we can not get enough information here.
220	367	*/
221		− __flush_tlb_range(vma, start, end, PAGE_SIZE, false);
	368	+ __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0);
222	369	}
223	370
224	371	static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end)

arch/arm64/include/asm/uaccess.h+1−0

@@ -19,6 +19,7 @@ · 18 unmodified lines
19	19	#include <linux/string.h>
20	20
21	21	#include <asm/cpufeature.h>
	22	+#include <asm/mmu.h>
22	23	#include <asm/ptrace.h>
23	24	#include <asm/memory.h>
24	25	#include <asm/extable.h>

arch/arm64/include/asm/vdso.h+2−0

@@ -12,6 +12,8 @@ · 11 unmodified lines
12	12	*/
13	13	#define VDSO_LBASE 0x0
14	14
	15	+#define __VVAR_PAGES 2
	16	+
15	17	#ifndef __ASSEMBLY__
16	18
17	19	#include <generated/vdso-offsets.h>

arch/arm64/include/asm/vdso/compat_gettimeofday.h+13−0

@@ -7,6 +7,7 @@ · 6 unmodified lines
7	7
8	8	#ifndef __ASSEMBLY__
9	9
	10	+#include <asm/barrier.h>
10	11	#include <asm/unistd.h>
11	12	#include <asm/errno.h>
12	13
@@ -152,6 +153,18 @@ static __always_inline const struct vdso_data *__arch_get_vdso_data(void) · 139 unmodified lines
152	153	return ret;
153	154	}
154	155
	156	+#ifdef CONFIG_TIME_NS
	157	+static __always_inline const struct vdso_data *__arch_get_timens_vdso_data(void)
	158	+{
	159	+ const struct vdso_data *ret;
	160	+
	161	+ /* See __arch_get_vdso_data(). */
	162	+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(_timens_data));
	163	+
	164	+ return ret;
	165	+}
	166	+#endif
	167	+
155	168	static inline bool vdso_clocksource_ok(const struct vdso_data *vd)
156	169	{
157	170	return vd->clock_mode == VDSO_CLOCKMODE_ARCHTIMER;

arch/arm64/include/asm/vdso/gettimeofday.h+9−0

@@ -7,6 +7,7 @@ · 6 unmodified lines
7	7
8	8	#ifndef __ASSEMBLY__
9	9
	10	+#include <asm/barrier.h>
10	11	#include <asm/unistd.h>
11	12
12	13	#define VDSO_HAS_CLOCK_GETRES 1
@@ -96,6 +97,14 @@ const struct vdso_data *__arch_get_vdso_data(void) · 83 unmodified lines
96	97	return _vdso_data;
97	98	}
98	99
	100	+#ifdef CONFIG_TIME_NS
	101	+static __always_inline
	102	+const struct vdso_data *__arch_get_timens_vdso_data(void)
	103	+{
	104	+ return _timens_data;
	105	+}
	106	+#endif
	107	+
99	108	#endif /* !__ASSEMBLY__ */
100	109
101	110	#endif /* __ASM_VDSO_GETTIMEOFDAY_H */

arch/arm64/include/uapi/asm/hwcap.h+1−0

@@ -74,5 +74,6 @@ · 73 unmodified lines
74	74	#define HWCAP2_DGH (1 << 15)
75	75	#define HWCAP2_RNG (1 << 16)
76	76	#define HWCAP2_BTI (1 << 17)
	77	+/* reserved for HWCAP2_MTE (1 << 18) */
77	78
78	79	#endif /* _UAPI__ASM_HWCAP_H */

arch/arm64/include/uapi/asm/sigcontext.h+1−1

@@ -179,7 +179,7 @@ struct sve_context { · 178 unmodified lines
179	179	* The same convention applies when returning from a signal: a caller
180	180	* will need to remove or resize the sve_context block if it wants to
181	181	* make the SVE registers live when they were previously non-live or
182		− * vice-versa. This may require the the caller to allocate fresh
	182	+ * vice-versa. This may require the caller to allocate fresh
183	183	* memory and/or move other context blocks in the signal frame.
184	184	*
185	185	* Changing the vector length during signal return is not permitted:

arch/arm64/kernel/acpi.c+75−0

@@ -261,6 +261,81 @@ pgprot_t __acpi_get_mem_attribute(phys_addr_t addr) · 260 unmodified lines
261	261	return __pgprot(PROT_DEVICE_nGnRnE);
262	262	}
263	263
	264	+void __iomem *acpi_os_ioremap(acpi_physical_address phys, acpi_size size)
	265	+{
	266	+ efi_memory_desc_t md, region = NULL;
	267	+ pgprot_t prot;
	268	+
	269	+ if (WARN_ON_ONCE(!efi_enabled(EFI_MEMMAP)))
	270	+ return NULL;
	271	+
	272	+ for_each_efi_memory_desc(md) {
	273	+ u64 end = md->phys_addr + (md->num_pages << EFI_PAGE_SHIFT);
	274	+
	275	+ if (phys < md->phys_addr \|\| phys >= end)
	276	+ continue;
	277	+
	278	+ if (phys + size > end) {
	279	+ pr_warn(FW_BUG "requested region covers multiple EFI memory regions\n");
	280	+ return NULL;
	281	+ }
	282	+ region = md;
	283	+ break;
	284	+ }
	285	+
	286	+ /*
	287	+ * It is fine for AML to remap regions that are not represented in the
	288	+ * EFI memory map at all, as it only describes normal memory, and MMIO
	289	+ * regions that require a virtual mapping to make them accessible to
	290	+ * the EFI runtime services.
	291	+ */
	292	+ prot = __pgprot(PROT_DEVICE_nGnRnE);
	293	+ if (region) {
	294	+ switch (region->type) {
	295	+ case EFI_LOADER_CODE:
	296	+ case EFI_LOADER_DATA:
	297	+ case EFI_BOOT_SERVICES_CODE:
	298	+ case EFI_BOOT_SERVICES_DATA:
	299	+ case EFI_CONVENTIONAL_MEMORY:
	300	+ case EFI_PERSISTENT_MEMORY:
	301	+ pr_warn(FW_BUG "requested region covers kernel memory @ %pa\n", &phys);
	302	+ return NULL;
	303	+
	304	+ case EFI_RUNTIME_SERVICES_CODE:
	305	+ /*
	306	+ * This would be unusual, but not problematic per se,
	307	+ * as long as we take care not to create a writable
	308	+ * mapping for executable code.
	309	+ */
	310	+ prot = PAGE_KERNEL_RO;
	311	+ break;
	312	+
	313	+ case EFI_ACPI_RECLAIM_MEMORY:
	314	+ /*
	315	+ * ACPI reclaim memory is used to pass firmware tables
	316	+ * and other data that is intended for consumption by
	317	+ * the OS only, which may decide it wants to reclaim
	318	+ * that memory and use it for something else. We never
	319	+ * do that, but we usually add it to the linear map
	320	+ * anyway, in which case we should use the existing
	321	+ * mapping.
	322	+ */
	323	+ if (memblock_is_map_memory(phys))
	324	+ return (void __iomem *)__phys_to_virt(phys);
	325	+ /* fall through */
	326	+
	327	+ default:
	328	+ if (region->attribute & EFI_MEMORY_WB)
	329	+ prot = PAGE_KERNEL;
	330	+ else if (region->attribute & EFI_MEMORY_WT)
	331	+ prot = __pgprot(PROT_NORMAL_WT);
	332	+ else if (region->attribute & EFI_MEMORY_WC)
	333	+ prot = __pgprot(PROT_NORMAL_NC);
	334	+ }
	335	+ }
	336	+ return __ioremap(phys, size, prot);
	337	+}
	338	+
264	339	/*
265	340	* Claim Synchronous External Aborts as a firmware first notification.
266	341	*

arch/arm64/kernel/cpufeature.c+106−43

@@ -119,25 +119,12 @@ static inline void finalize_system_capabilities(void) · 118 unmodified lines
119	119	static_branch_enable(&arm64_const_caps_ready);
120	120	}
121	121
122		−static int dump_cpu_hwcaps(struct notifier_block self, unsigned long v, void p)
	122	+void dump_cpu_features(void)
123	123	{
124	124	/* file-wide pr_fmt adds "CPU features: " prefix */
125	125	pr_emerg("0x%*pb\n", ARM64_NCAPS, &cpu_hwcaps);
126		− return 0;
127	126	}
128	127
129		−static struct notifier_block cpu_hwcaps_notifier = {
130		− .notifier_call = dump_cpu_hwcaps
131		−};
132		−
133		−static int __init register_cpu_hwcaps_dumper(void)
134		−{
135		− atomic_notifier_chain_register(&panic_notifier_list,
136		− &cpu_hwcaps_notifier);
137		− return 0;
138		−}
139		−__initcall(register_cpu_hwcaps_dumper);
140		−
141	128	DEFINE_STATIC_KEY_ARRAY_FALSE(cpu_hwcap_keys, ARM64_NCAPS);
142	129	EXPORT_SYMBOL(cpu_hwcap_keys);
143	130
@@ -269,6 +256,9 @@ static const struct arm64_ftr_bits ftr_id_aa64zfr0[] = { · 125 unmodified lines
269	256	};
270	257
271	258	static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
	259	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_SHIFT, 4, 0),
	260	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_FGT_SHIFT, 4, 0),
	261	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_EXS_SHIFT, 4, 0),
272	262	/*
273	263	* Page size not being supported at Stage-2 is not fatal. You
274	264	* just give up KVM if PAGE_SIZE isn't supported there. Go fix
@@ -312,6 +302,10 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = { · 37 unmodified lines
312	302	};
313	303
314	304	static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
	305	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_SHIFT, 4, 0),
	306	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_SHIFT, 4, 0),
	307	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_SHIFT, 4, 0),
	308	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64MMFR1_SPECSEI_SHIFT, 4, 0),
315	309	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_PAN_SHIFT, 4, 0),
316	310	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_LOR_SHIFT, 4, 0),
317	311	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HPD_SHIFT, 4, 0),
@@ -323,8 +317,15 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = { · 5 unmodified lines
323	317
324	318	static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
325	319	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_E0PD_SHIFT, 4, 0),
	320	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_EVT_SHIFT, 4, 0),
	321	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_BBM_SHIFT, 4, 0),
	322	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_TTL_SHIFT, 4, 0),
326	323	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_FWB_SHIFT, 4, 0),
	324	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IDS_SHIFT, 4, 0),
327	325	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_AT_SHIFT, 4, 0),
	326	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_ST_SHIFT, 4, 0),
	327	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_NV_SHIFT, 4, 0),
	328	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CCIDX_SHIFT, 4, 0),
328	329	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LVA_SHIFT, 4, 0),
329	330	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IESB_SHIFT, 4, 0),
330	331	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LSM_SHIFT, 4, 0),
@@ -345,7 +346,7 @@ static const struct arm64_ftr_bits ftr_ctr[] = { · 14 unmodified lines
345	346	* make use of *minLine.
346	347	* If we have differing I-cache policies, report it as the weakest - VIPT.
347	348	*/
348		− ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, 14, 2, ICACHE_POLICY_VIPT), /* L1Ip */
	349	+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, CTR_L1IP_SHIFT, 2, ICACHE_POLICY_VIPT), /* L1Ip */
349	350	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
350	351	ARM64_FTR_END,
351	352	};
@@ -356,19 +357,19 @@ struct arm64_ftr_reg arm64_ftr_reg_ctrel0 = { · 4 unmodified lines
356	357	};
357	358
358	359	static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
359		− S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 28, 4, 0xf), /* InnerShr */
360		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0), /* FCSE */
361		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, 20, 4, 0), /* AuxReg */
362		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0), /* TCM */
363		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0), /* ShareLvl */
364		− S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0xf), /* OuterShr */
365		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0), /* PMSA */
366		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0), /* VMSA */
	360	+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
	361	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_FCSE_SHIFT, 4, 0),
	362	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_MMFR0_AUXREG_SHIFT, 4, 0),
	363	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_TCM_SHIFT, 4, 0),
	364	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_SHARELVL_SHIFT, 4, 0),
	365	+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_OUTERSHR_SHIFT, 4, 0xf),
	366	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_PMSA_SHIFT, 4, 0),
	367	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_VMSA_SHIFT, 4, 0),
367	368	ARM64_FTR_END,
368	369	};
369	370
370	371	static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
371		− S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 36, 4, 0),
	372	+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_DOUBLELOCK_SHIFT, 4, 0),
372	373	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_PMSVER_SHIFT, 4, 0),
373	374	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_CTX_CMPS_SHIFT, 4, 0),
374	375	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_WRPS_SHIFT, 4, 0),
@@ -384,14 +385,14 @@ static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = { · 9 unmodified lines
384	385	};
385	386
386	387	static const struct arm64_ftr_bits ftr_mvfr2[] = {
387		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0), /* FPMisc */
388		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0), /* SIMDMisc */
	388	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_FPMISC_SHIFT, 4, 0),
	389	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_SIMDMISC_SHIFT, 4, 0),
389	390	ARM64_FTR_END,
390	391	};
391	392
392	393	static const struct arm64_ftr_bits ftr_dczid[] = {
393		− ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 4, 1, 1), /* DZP */
394		− ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0), /* BS */
	394	+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
	395	+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
395	396	ARM64_FTR_END,
396	397	};
397	398
@@ -423,7 +424,8 @@ static const struct arm64_ftr_bits ftr_id_mmfr4[] = { · 25 unmodified lines
423	424	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_HPDS_SHIFT, 4, 0),
424	425	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CNP_SHIFT, 4, 0),
425	426	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_XNX_SHIFT, 4, 0),
426		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0), /* ac2 */
	427	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_AC2_SHIFT, 4, 0),
	428	+
427	429	/*
428	430	* SpecSEI = 1 indicates that the PE might generate an SError on an
429	431	* external abort on speculative read. It is safe to assume that an
@@ -465,10 +467,10 @@ static const struct arm64_ftr_bits ftr_id_isar6[] = { · 35 unmodified lines
465	467	static const struct arm64_ftr_bits ftr_id_pfr0[] = {
466	468	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_DIT_SHIFT, 4, 0),
467	469	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR0_CSV2_SHIFT, 4, 0),
468		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0), /* State3 */
469		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0), /* State2 */
470		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0), /* State1 */
471		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0), /* State0 */
	470	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE3_SHIFT, 4, 0),
	471	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE2_SHIFT, 4, 0),
	472	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE1_SHIFT, 4, 0),
	473	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE0_SHIFT, 4, 0),
472	474	ARM64_FTR_END,
473	475	};
474	476
@@ -492,13 +494,13 @@ static const struct arm64_ftr_bits ftr_id_pfr2[] = { · 17 unmodified lines
492	494
493	495	static const struct arm64_ftr_bits ftr_id_dfr0[] = {
494	496	/* [31:28] TraceFilt */
495		− S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0xf), /* PerfMon */
496		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 20, 4, 0),
497		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0),
498		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),
499		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0),
500		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),
501		− ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),
	497	+ S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_PERFMON_SHIFT, 4, 0xf),
	498	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MPROFDBG_SHIFT, 4, 0),
	499	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPTRC_SHIFT, 4, 0),
	500	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPTRC_SHIFT, 4, 0),
	501	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPDBG_SHIFT, 4, 0),
	502	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPSDBG_SHIFT, 4, 0),
	503	+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPDBG_SHIFT, 4, 0),
502	504	ARM64_FTR_END,
503	505	};
504	506
@@ -697,11 +699,52 @@ static s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new, · 192 unmodified lines
697	699
698	700	static void __init sort_ftr_regs(void)
699	701	{
700		− int i;
	702	+ unsigned int i;
	703	+
	704	+ for (i = 0; i < ARRAY_SIZE(arm64_ftr_regs); i++) {
	705	+ const struct arm64_ftr_reg *ftr_reg = arm64_ftr_regs[i].reg;
	706	+ const struct arm64_ftr_bits *ftr_bits = ftr_reg->ftr_bits;
	707	+ unsigned int j = 0;
	708	+
	709	+ /*
	710	+ * Features here must be sorted in descending order with respect
	711	+ * to their shift values and should not overlap with each other.
	712	+ */
	713	+ for (; ftr_bits->width != 0; ftr_bits++, j++) {
	714	+ unsigned int width = ftr_reg->ftr_bits[j].width;
	715	+ unsigned int shift = ftr_reg->ftr_bits[j].shift;
	716	+ unsigned int prev_shift;
	717	+
	718	+ WARN((shift + width) > 64,
	719	+ "%s has invalid feature at shift %d\n",
	720	+ ftr_reg->name, shift);
	721	+
	722	+ /*
	723	+ * Skip the first feature. There is nothing to
	724	+ * compare against for now.
	725	+ */
	726	+ if (j == 0)
	727	+ continue;
	728	+
	729	+ prev_shift = ftr_reg->ftr_bits[j - 1].shift;
	730	+ WARN((shift + width) > prev_shift,
	731	+ "%s has feature overlap at shift %d\n",
	732	+ ftr_reg->name, shift);
	733	+ }
701	734
702		− /* Check that the array is sorted so that we can do the binary search */
703		− for (i = 1; i < ARRAY_SIZE(arm64_ftr_regs); i++)
	735	+ /*
	736	+ * Skip the first register. There is nothing to
	737	+ * compare against for now.
	738	+ */
	739	+ if (i == 0)
	740	+ continue;
	741	+ /*
	742	+ * Registers here must be sorted in ascending order with respect
	743	+ * to sys_id for subsequent binary search in get_arm64_ftr_reg()
	744	+ * to work correctly.
	745	+ */
704	746	BUG_ON(arm64_ftr_regs[i].sys_id < arm64_ftr_regs[i - 1].sys_id);
	747	+ }
705	748	}
706	749
707	750	/*
@@ -1884,6 +1927,26 @@ static const struct arm64_cpu_capabilities arm64_features[] = { · 1176 unmodified lines
1884	1927	.matches = has_cpuid_feature,
1885	1928	.cpu_enable = cpu_has_fwb,
1886	1929	},
	1930	+ {
	1931	+ .desc = "ARMv8.4 Translation Table Level",
	1932	+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
	1933	+ .capability = ARM64_HAS_ARMv8_4_TTL,
	1934	+ .sys_reg = SYS_ID_AA64MMFR2_EL1,
	1935	+ .sign = FTR_UNSIGNED,
	1936	+ .field_pos = ID_AA64MMFR2_TTL_SHIFT,
	1937	+ .min_field_value = 1,
	1938	+ .matches = has_cpuid_feature,
	1939	+ },
	1940	+ {
	1941	+ .desc = "TLB range maintenance instructions",
	1942	+ .capability = ARM64_HAS_TLB_RANGE,
	1943	+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
	1944	+ .matches = has_cpuid_feature,
	1945	+ .sys_reg = SYS_ID_AA64ISAR0_EL1,
	1946	+ .field_pos = ID_AA64ISAR0_TLB_SHIFT,
	1947	+ .sign = FTR_UNSIGNED,
	1948	+ .min_field_value = ID_AA64ISAR0_TLB_RANGE,
	1949	+ },
1887	1950	#ifdef CONFIG_ARM64_HW_AFDBM
1888	1951	{
1889	1952	/*

arch/arm64/kernel/cpuinfo.c+1−0

@@ -93,6 +93,7 @@ static const char *const hwcap_str[] = { · 92 unmodified lines
93	93	"dgh",
94	94	"rng",
95	95	"bti",
	96	+ /* reserved for "mte" */
96	97	NULL
97	98	};
98	99

arch/arm64/kernel/crash_core.c+10−0

@@ -7,6 +7,14 @@ · 6 unmodified lines
7	7	#include <linux/crash_core.h>
8	8	#include <asm/cpufeature.h>
9	9	#include <asm/memory.h>
	10	+#include <asm/pgtable-hwdef.h>
	11	+
	12	+static inline u64 get_tcr_el1_t1sz(void);
	13	+
	14	+static inline u64 get_tcr_el1_t1sz(void)
	15	+{
	16	+ return (read_sysreg(tcr_el1) & TCR_T1SZ_MASK) >> TCR_T1SZ_OFFSET;
	17	+}
10	18
11	19	void arch_crash_save_vmcoreinfo(void)
12	20	{
@@ -16,6 +24,8 @@ void arch_crash_save_vmcoreinfo(void) · 3 unmodified lines
16	24	kimage_voffset);
17	25	vmcoreinfo_append_str("NUMBER(PHYS_OFFSET)=0x%llx\n",
18	26	PHYS_OFFSET);
	27	+ vmcoreinfo_append_str("NUMBER(TCR_EL1_T1SZ)=0x%llx\n",
	28	+ get_tcr_el1_t1sz());
19	29	vmcoreinfo_append_str("KERNELOFFSET=%lx\n", kaslr_offset());
20	30	vmcoreinfo_append_str("NUMBER(KERNELPACMASK)=0x%llx\n",
21	31	system_supports_address_auth() ?

arch/arm64/kernel/entry.S+48−48

@@ -15,6 +15,7 @@ · 14 unmodified lines
15	15	#include <asm/assembler.h>
16	16	#include <asm/asm-offsets.h>
17	17	#include <asm/asm_pointer_auth.h>
	18	+#include <asm/bug.h>
18	19	#include <asm/cpufeature.h>
19	20	#include <asm/errno.h>
20	21	#include <asm/esr.h>
@@ -226,28 +227,9 @@ alternative_else_nop_endif · 205 unmodified lines
226	227	add x29, sp, #S_STACKFRAME
227	228
228	229	#ifdef CONFIG_ARM64_SW_TTBR0_PAN
229		− /*
230		− * Set the TTBR0 PAN bit in SPSR. When the exception is taken from
231		− * EL0, there is no need to check the state of TTBR0_EL1 since
232		− * accesses are always enabled.
233		− * Note that the meaning of this bit differs from the ARMv8.1 PAN
234		− * feature as all TTBR0_EL1 accesses are disabled, not just those to
235		− * user mappings.
236		− */
237		−alternative_if ARM64_HAS_PAN
238		− b 1f // skip TTBR0 PAN
	230	+alternative_if_not ARM64_HAS_PAN
	231	+ bl __swpan_entry_el\el
239	232	alternative_else_nop_endif
240		−
241		− .if \el != 0
242		− mrs x21, ttbr0_el1
243		− tst x21, #TTBR_ASID_MASK // Check for the reserved ASID
244		− orr x23, x23, #PSR_PAN_BIT // Set the emulated PAN in the saved SPSR
245		− b.eq 1f // TTBR0 access already disabled
246		− and x23, x23, #~PSR_PAN_BIT // Clear the emulated PAN in the saved SPSR
247		− .endif
248		−
249		− __uaccess_ttbr0_disable x21
250		−1:
251	233	#endif
252	234
253	235	stp x22, x23, [sp, #S_PC]
@@ -301,34 +283,9 @@ alternative_else_nop_endif · 47 unmodified lines
301	283	.endif
302	284
303	285	#ifdef CONFIG_ARM64_SW_TTBR0_PAN
304		− /*
305		− * Restore access to TTBR0_EL1. If returning to EL0, no need for SPSR
306		− * PAN bit checking.
307		− */
308		−alternative_if ARM64_HAS_PAN
309		− b 2f // skip TTBR0 PAN
	286	+alternative_if_not ARM64_HAS_PAN
	287	+ bl __swpan_exit_el\el
310	288	alternative_else_nop_endif
311		−
312		− .if \el != 0
313		− tbnz x22, #22, 1f // Skip re-enabling TTBR0 access if the PSR_PAN_BIT is set
314		− .endif
315		−
316		− __uaccess_ttbr0_enable x0, x1
317		−
318		− .if \el == 0
319		− /*
320		− * Enable errata workarounds only if returning to user. The only
321		− * workaround currently required for TTBR0_EL1 changes are for the
322		− * Cavium erratum 27456 (broadcast TLBI instructions may cause I-cache
323		− * corruption).
324		− */
325		− bl post_ttbr_update_workaround
326		− .endif
327		−1:
328		− .if \el != 0
329		− and x22, x22, #~PSR_PAN_BIT // ARMv8.0 CPUs do not understand this bit
330		− .endif
331		−2:
332	289	#endif
333	290
334	291	.if \el == 0
@@ -401,6 +358,49 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 · 66 unmodified lines
401	358	sb
402	359	.endm
403	360
	361	+#ifdef CONFIG_ARM64_SW_TTBR0_PAN
	362	+ /*
	363	+ * Set the TTBR0 PAN bit in SPSR. When the exception is taken from
	364	+ * EL0, there is no need to check the state of TTBR0_EL1 since
	365	+ * accesses are always enabled.
	366	+ * Note that the meaning of this bit differs from the ARMv8.1 PAN
	367	+ * feature as all TTBR0_EL1 accesses are disabled, not just those to
	368	+ * user mappings.
	369	+ */
	370	+SYM_CODE_START_LOCAL(__swpan_entry_el1)
	371	+ mrs x21, ttbr0_el1
	372	+ tst x21, #TTBR_ASID_MASK // Check for the reserved ASID
	373	+ orr x23, x23, #PSR_PAN_BIT // Set the emulated PAN in the saved SPSR
	374	+ b.eq 1f // TTBR0 access already disabled
	375	+ and x23, x23, #~PSR_PAN_BIT // Clear the emulated PAN in the saved SPSR
	376	+SYM_INNER_LABEL(__swpan_entry_el0, SYM_L_LOCAL)
	377	+ __uaccess_ttbr0_disable x21
	378	+1: ret
	379	+SYM_CODE_END(__swpan_entry_el1)
	380	+
	381	+ /*
	382	+ * Restore access to TTBR0_EL1. If returning to EL0, no need for SPSR
	383	+ * PAN bit checking.
	384	+ */
	385	+SYM_CODE_START_LOCAL(__swpan_exit_el1)
	386	+ tbnz x22, #22, 1f // Skip re-enabling TTBR0 access if the PSR_PAN_BIT is set
	387	+ __uaccess_ttbr0_enable x0, x1
	388	+1: and x22, x22, #~PSR_PAN_BIT // ARMv8.0 CPUs do not understand this bit
	389	+ ret
	390	+SYM_CODE_END(__swpan_exit_el1)
	391	+
	392	+SYM_CODE_START_LOCAL(__swpan_exit_el0)
	393	+ __uaccess_ttbr0_enable x0, x1
	394	+ /*
	395	+ * Enable errata workarounds only if returning to user. The only
	396	+ * workaround currently required for TTBR0_EL1 changes are for the
	397	+ * Cavium erratum 27456 (broadcast TLBI instructions may cause I-cache
	398	+ * corruption).
	399	+ */
	400	+ b post_ttbr_update_workaround
	401	+SYM_CODE_END(__swpan_exit_el0)
	402	+#endif
	403	+
404	404	.macro irq_stack_entry
405	405	mov x19, sp // preserve the original sp
406	406	#ifdef CONFIG_SHADOW_CALL_STACK

arch/arm64/kernel/module-plts.c+43−3

@@ -253,6 +253,40 @@ static unsigned int count_plts(Elf64_Sym syms, Elf64_Rela rela, int num, · 252 unmodified lines
253	253	return ret;
254	254	}
255	255
	256	+static bool branch_rela_needs_plt(Elf64_Sym syms, Elf64_Rela rela,
	257	+ Elf64_Word dstidx)
	258	+{
	259	+
	260	+ Elf64_Sym *s = syms + ELF64_R_SYM(rela->r_info);
	261	+
	262	+ if (s->st_shndx == dstidx)
	263	+ return false;
	264	+
	265	+ return ELF64_R_TYPE(rela->r_info) == R_AARCH64_JUMP26 \|\|
	266	+ ELF64_R_TYPE(rela->r_info) == R_AARCH64_CALL26;
	267	+}
	268	+
	269	+/* Group branch PLT relas at the front end of the array. */
	270	+static int partition_branch_plt_relas(Elf64_Sym syms, Elf64_Rela rela,
	271	+ int numrels, Elf64_Word dstidx)
	272	+{
	273	+ int i = 0, j = numrels - 1;
	274	+
	275	+ if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE))
	276	+ return 0;
	277	+
	278	+ while (i < j) {
	279	+ if (branch_rela_needs_plt(syms, &rela[i], dstidx))
	280	+ i++;
	281	+ else if (branch_rela_needs_plt(syms, &rela[j], dstidx))
	282	+ swap(rela[i], rela[j]);
	283	+ else
	284	+ j--;
	285	+ }
	286	+
	287	+ return i;
	288	+}
	289	+
256	290	int module_frob_arch_sections(Elf_Ehdr ehdr, Elf_Shdr sechdrs,
257	291	char secstrings, struct module mod)
258	292	{
@@ -290,7 +324,7 @@ int module_frob_arch_sections(Elf_Ehdr ehdr, Elf_Shdr sechdrs, · 31 unmodified lines
290	324
291	325	for (i = 0; i < ehdr->e_shnum; i++) {
292	326	Elf64_Rela rels = (void )ehdr + sechdrs[i].sh_offset;
293		− int numrels = sechdrs[i].sh_size / sizeof(Elf64_Rela);
	327	+ int nents, numrels = sechdrs[i].sh_size / sizeof(Elf64_Rela);
294	328	Elf64_Shdr *dstsec = sechdrs + sechdrs[i].sh_info;
295	329
296	330	if (sechdrs[i].sh_type != SHT_RELA)
@@ -300,8 +334,14 @@ int module_frob_arch_sections(Elf_Ehdr ehdr, Elf_Shdr sechdrs, · 3 unmodified lines
300	334	if (!(dstsec->sh_flags & SHF_EXECINSTR))
301	335	continue;
302	336
303		− /* sort by type, symbol index and addend */
304		− sort(rels, numrels, sizeof(Elf64_Rela), cmp_rela, NULL);
	337	+ /*
	338	+ * sort branch relocations requiring a PLT by type, symbol index
	339	+ * and addend
	340	+ */
	341	+ nents = partition_branch_plt_relas(syms, rels, numrels,
	342	+ sechdrs[i].sh_info);
	343	+ if (nents)
	344	+ sort(rels, nents, sizeof(Elf64_Rela), cmp_rela, NULL);
305	345
306	346	if (!str_has_prefix(secstrings + dstsec->sh_name, ".init"))
307	347	core_plts += count_plts(syms, rels, numrels,

arch/arm64/kernel/perf_event.c+70−19

@@ -13,12 +13,15 @@ · 12 unmodified lines
13	13	#include <asm/sysreg.h>
14	14	#include <asm/virt.h>
15	15
	16	+#include <clocksource/arm_arch_timer.h>
	17	+
16	18	#include <linux/acpi.h>
17	19	#include <linux/clocksource.h>
18	20	#include <linux/kvm_host.h>
19	21	#include <linux/of.h>
20	22	#include <linux/perf/arm_pmu.h>
21	23	#include <linux/platform_device.h>
	24	+#include <linux/sched_clock.h>
22	25	#include <linux/smp.h>
23	26
24	27	/* ARMv8 Cortex-A53 specific event types. */
@@ -155,7 +158,7 @@ armv8pmu_events_sysfs_show(struct device *dev, · 130 unmodified lines
155	158
156	159	pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr);
157	160
158		− return sprintf(page, "event=0x%03llx\n", pmu_attr->id);
	161	+ return sprintf(page, "event=0x%04llx\n", pmu_attr->id);
159	162	}
160	163
161	164	#define ARMV8_EVENT_ATTR(name, config) \
@@ -222,10 +225,29 @@ static struct attribute *armv8_pmuv3_event_attrs[] = { · 60 unmodified lines
222	225	ARMV8_EVENT_ATTR(ll_cache_rd, ARMV8_PMUV3_PERFCTR_LL_CACHE_RD),
223	226	ARMV8_EVENT_ATTR(ll_cache_miss_rd, ARMV8_PMUV3_PERFCTR_LL_CACHE_MISS_RD),
224	227	ARMV8_EVENT_ATTR(remote_access_rd, ARMV8_PMUV3_PERFCTR_REMOTE_ACCESS_RD),
	228	+ ARMV8_EVENT_ATTR(l1d_cache_lmiss_rd, ARMV8_PMUV3_PERFCTR_L1D_CACHE_LMISS_RD),
	229	+ ARMV8_EVENT_ATTR(op_retired, ARMV8_PMUV3_PERFCTR_OP_RETIRED),
	230	+ ARMV8_EVENT_ATTR(op_spec, ARMV8_PMUV3_PERFCTR_OP_SPEC),
	231	+ ARMV8_EVENT_ATTR(stall, ARMV8_PMUV3_PERFCTR_STALL),
	232	+ ARMV8_EVENT_ATTR(stall_slot_backend, ARMV8_PMUV3_PERFCTR_STALL_SLOT_BACKEND),
	233	+ ARMV8_EVENT_ATTR(stall_slot_frontend, ARMV8_PMUV3_PERFCTR_STALL_SLOT_FRONTEND),
	234	+ ARMV8_EVENT_ATTR(stall_slot, ARMV8_PMUV3_PERFCTR_STALL_SLOT),
225	235	ARMV8_EVENT_ATTR(sample_pop, ARMV8_SPE_PERFCTR_SAMPLE_POP),
226	236	ARMV8_EVENT_ATTR(sample_feed, ARMV8_SPE_PERFCTR_SAMPLE_FEED),
227	237	ARMV8_EVENT_ATTR(sample_filtrate, ARMV8_SPE_PERFCTR_SAMPLE_FILTRATE),
228	238	ARMV8_EVENT_ATTR(sample_collision, ARMV8_SPE_PERFCTR_SAMPLE_COLLISION),
	239	+ ARMV8_EVENT_ATTR(cnt_cycles, ARMV8_AMU_PERFCTR_CNT_CYCLES),
	240	+ ARMV8_EVENT_ATTR(stall_backend_mem, ARMV8_AMU_PERFCTR_STALL_BACKEND_MEM),
	241	+ ARMV8_EVENT_ATTR(l1i_cache_lmiss, ARMV8_PMUV3_PERFCTR_L1I_CACHE_LMISS),
	242	+ ARMV8_EVENT_ATTR(l2d_cache_lmiss_rd, ARMV8_PMUV3_PERFCTR_L2D_CACHE_LMISS_RD),
	243	+ ARMV8_EVENT_ATTR(l2i_cache_lmiss, ARMV8_PMUV3_PERFCTR_L2I_CACHE_LMISS),
	244	+ ARMV8_EVENT_ATTR(l3d_cache_lmiss_rd, ARMV8_PMUV3_PERFCTR_L3D_CACHE_LMISS_RD),
	245	+ ARMV8_EVENT_ATTR(ldst_align_lat, ARMV8_PMUV3_PERFCTR_LDST_ALIGN_LAT),
	246	+ ARMV8_EVENT_ATTR(ld_align_lat, ARMV8_PMUV3_PERFCTR_LD_ALIGN_LAT),
	247	+ ARMV8_EVENT_ATTR(st_align_lat, ARMV8_PMUV3_PERFCTR_ST_ALIGN_LAT),
	248	+ ARMV8_EVENT_ATTR(mem_access_checked, ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED),
	249	+ ARMV8_EVENT_ATTR(mem_access_checked_rd, ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED_RD),
	250	+ ARMV8_EVENT_ATTR(mem_access_checked_wr, ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED_WR),
229	251	NULL,
230	252	};
231	253
@@ -244,10 +266,13 @@ armv8pmu_event_attr_is_visible(struct kobject *kobj, · 12 unmodified lines
244	266	test_bit(pmu_attr->id, cpu_pmu->pmceid_bitmap))
245	267	return attr->mode;
246	268
247		− pmu_attr->id -= ARMV8_PMUV3_EXT_COMMON_EVENT_BASE;
248		− if (pmu_attr->id < ARMV8_PMUV3_MAX_COMMON_EVENTS &&
249		− test_bit(pmu_attr->id, cpu_pmu->pmceid_ext_bitmap))
250		− return attr->mode;
	269	+ if (pmu_attr->id >= ARMV8_PMUV3_EXT_COMMON_EVENT_BASE) {
	270	+ u64 id = pmu_attr->id - ARMV8_PMUV3_EXT_COMMON_EVENT_BASE;
	271	+
	272	+ if (id < ARMV8_PMUV3_MAX_COMMON_EVENTS &&
	273	+ test_bit(id, cpu_pmu->pmceid_ext_bitmap))
	274	+ return attr->mode;
	275	+ }
251	276
252	277	return 0;
253	278	}
@@ -1165,28 +1190,54 @@ device_initcall(armv8_pmu_driver_init) · 911 unmodified lines
1165	1190	void arch_perf_update_userpage(struct perf_event *event,
1166	1191	struct perf_event_mmap_page *userpg, u64 now)
1167	1192	{
1168		− u32 freq;
1169		− u32 shift;
	1193	+ struct clock_read_data *rd;
	1194	+ unsigned int seq;
	1195	+ u64 ns;
1170	1196
1171		− /*
1172		− * Internal timekeeping for enabled/running/stopped times
1173		− * is always computed with the sched_clock.
1174		− */
1175		− freq = arch_timer_get_rate();
1176		− userpg->cap_user_time = 1;
	1197	+ userpg->cap_user_time = 0;
	1198	+ userpg->cap_user_time_zero = 0;
	1199	+ userpg->cap_user_time_short = 0;
	1200	+
	1201	+ do {
	1202	+ rd = sched_clock_read_begin(&seq);
	1203	+
	1204	+ if (rd->read_sched_clock != arch_timer_read_counter)
	1205	+ return;
	1206	+
	1207	+ userpg->time_mult = rd->mult;
	1208	+ userpg->time_shift = rd->shift;
	1209	+ userpg->time_zero = rd->epoch_ns;
	1210	+ userpg->time_cycles = rd->epoch_cyc;
	1211	+ userpg->time_mask = rd->sched_clock_mask;
	1212	+
	1213	+ /*
	1214	+ * Subtract the cycle base, such that software that
	1215	+ * doesn't know about cap_user_time_short still 'works'
	1216	+ * assuming no wraps.
	1217	+ */
	1218	+ ns = mul_u64_u32_shr(rd->epoch_cyc, rd->mult, rd->shift);
	1219	+ userpg->time_zero -= ns;
	1220	+
	1221	+ } while (sched_clock_read_retry(seq));
	1222	+
	1223	+ userpg->time_offset = userpg->time_zero - now;
1177	1224
1178		− clocks_calc_mult_shift(&userpg->time_mult, &shift, freq,
1179		− NSEC_PER_SEC, 0);
1180	1225	/*
1181	1226	* time_shift is not expected to be greater than 31 due to
1182	1227	* the original published conversion algorithm shifting a
1183	1228	* 32-bit value (now specifies a 64-bit value) - refer
1184	1229	* perf_event_mmap_page documentation in perf_event.h.
1185	1230	*/
1186		− if (shift == 32) {
1187		− shift = 31;
	1231	+ if (userpg->time_shift == 32) {
	1232	+ userpg->time_shift = 31;
1188	1233	userpg->time_mult >>= 1;
1189	1234	}
1190		− userpg->time_shift = (u16)shift;
1191		− userpg->time_offset = -now;
	1235	+
	1236	+ /*
	1237	+ * Internal timekeeping for enabled/running/stopped times
	1238	+ * is always computed with the sched_clock.
	1239	+ */
	1240	+ userpg->cap_user_time = 1;
	1241	+ userpg->cap_user_time_zero = 1;
	1242	+ userpg->cap_user_time_short = 1;
1192	1243	}

arch/arm64/kernel/setup.c+14−10

@@ -400,11 +400,7 @@ static int __init topology_init(void) · 399 unmodified lines
400	400	}
401	401	subsys_initcall(topology_init);
402	402
403		−/*
404		− * Dump out kernel offset information on panic.
405		− */
406		−static int dump_kernel_offset(struct notifier_block *self, unsigned long v,
407		− void *p)
	403	+static void dump_kernel_offset(void)
408	404	{
409	405	const unsigned long offset = kaslr_offset();
410	406
@@ -415,17 +411,25 @@ static int dump_kernel_offset(struct notifier_block *self, unsigned long v, · 4 unmodified lines
415	411	} else {
416	412	pr_emerg("Kernel Offset: disabled\n");
417	413	}
	414	+}
	415	+
	416	+static int arm64_panic_block_dump(struct notifier_block *self,
	417	+ unsigned long v, void *p)
	418	+{
	419	+ dump_kernel_offset();
	420	+ dump_cpu_features();
	421	+ dump_mem_limit();
418	422	return 0;
419	423	}
420	424
421		−static struct notifier_block kernel_offset_notifier = {
422		− .notifier_call = dump_kernel_offset
	425	+static struct notifier_block arm64_panic_block = {
	426	+ .notifier_call = arm64_panic_block_dump
423	427	};
424	428
425		−static int __init register_kernel_offset_dumper(void)
	429	+static int __init register_arm64_panic_block(void)
426	430	{
427	431	atomic_notifier_chain_register(&panic_notifier_list,
428		− &kernel_offset_notifier);
	432	+ &arm64_panic_block);
429	433	return 0;
430	434	}
431		−__initcall(register_kernel_offset_dumper);
	435	+device_initcall(register_arm64_panic_block);

arch/arm64/kernel/stacktrace.c+1−1

@@ -199,12 +199,12 @@ static noinline void __save_stack_trace(struct task_struct *tsk, · 198 unmodified lines
199	199
200	200	put_task_stack(tsk);
201	201	}
202		−EXPORT_SYMBOL_GPL(save_stack_trace_tsk);
203	202
204	203	void save_stack_trace_tsk(struct task_struct tsk, struct stack_trace trace)
205	204	{
206	205	__save_stack_trace(tsk, trace, 1);
207	206	}
	207	+EXPORT_SYMBOL_GPL(save_stack_trace_tsk);
208	208
209	209	void save_stack_trace(struct stack_trace *trace)
210	210	{

arch/arm64/kernel/traps.c+1−1

@@ -855,7 +855,7 @@ asmlinkage void handle_bad_stack(struct pt_regs *regs) · 854 unmodified lines
855	855	pr_emerg("Task stack: [0x%016lx..0x%016lx]\n",
856	856	tsk_stk, tsk_stk + THREAD_SIZE);
857	857	pr_emerg("IRQ stack: [0x%016lx..0x%016lx]\n",
858		− irq_stk, irq_stk + THREAD_SIZE);
	858	+ irq_stk, irq_stk + IRQ_STACK_SIZE);
859	859	pr_emerg("Overflow stack: [0x%016lx..0x%016lx]\n",
860	860	ovf_stk, ovf_stk + OVERFLOW_STACK_SIZE);
861	861

arch/arm64/kernel/vdso.c+123−13

@@ -18,6 +18,7 @@ · 17 unmodified lines
18	18	#include <linux/sched.h>
19	19	#include <linux/signal.h>
20	20	#include <linux/slab.h>
	21	+#include <linux/time_namespace.h>
21	22	#include <linux/timekeeper_internal.h>
22	23	#include <linux/vmalloc.h>
23	24	#include <vdso/datapage.h>
@@ -40,6 +41,12 @@ enum vdso_abi { · 16 unmodified lines
40	41	#endif /* CONFIG_COMPAT_VDSO */
41	42	};
42	43
	44	+enum vvar_pages {
	45	+ VVAR_DATA_PAGE_OFFSET,
	46	+ VVAR_TIMENS_PAGE_OFFSET,
	47	+ VVAR_NR_PAGES,
	48	+};
	49	+
43	50	struct vdso_abi_info {
44	51	const char *name;
45	52	const char *vdso_code_start;
@@ -107,25 +114,122 @@ static int __vdso_init(enum vdso_abi abi) · 61 unmodified lines
107	114	vdso_info[abi].vdso_code_start) >>
108	115	PAGE_SHIFT;
109	116
110		− /* Allocate the vDSO pagelist, plus a page for the data. */
111		− vdso_pagelist = kcalloc(vdso_info[abi].vdso_pages + 1,
	117	+ vdso_pagelist = kcalloc(vdso_info[abi].vdso_pages,
112	118	sizeof(struct page *),
113	119	GFP_KERNEL);
114	120	if (vdso_pagelist == NULL)
115	121	return -ENOMEM;
116	122
117		− /* Grab the vDSO data page. */
118		− vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
119		−
120		−
121	123	/* Grab the vDSO code pages. */
122	124	pfn = sym_to_pfn(vdso_info[abi].vdso_code_start);
123	125
124	126	for (i = 0; i < vdso_info[abi].vdso_pages; i++)
125		− vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
	127	+ vdso_pagelist[i] = pfn_to_page(pfn + i);
	128	+
	129	+ vdso_info[abi].cm->pages = vdso_pagelist;
	130	+
	131	+ return 0;
	132	+}
	133	+
	134	+#ifdef CONFIG_TIME_NS
	135	+struct vdso_data arch_get_vdso_data(void vvar_page)
	136	+{
	137	+ return (struct vdso_data *)(vvar_page);
	138	+}
	139	+
	140	+/*
	141	+ * The vvar mapping contains data for a specific time namespace, so when a task
	142	+ * changes namespace we must unmap its vvar data for the old namespace.
	143	+ * Subsequent faults will map in data for the new namespace.
	144	+ *
	145	+ * For more details see timens_setup_vdso_data().
	146	+ */
	147	+int vdso_join_timens(struct task_struct task, struct time_namespace ns)
	148	+{
	149	+ struct mm_struct *mm = task->mm;
	150	+ struct vm_area_struct *vma;
	151	+
	152	+ mmap_read_lock(mm);
	153	+
	154	+ for (vma = mm->mmap; vma; vma = vma->vm_next) {
	155	+ unsigned long size = vma->vm_end - vma->vm_start;
126	156
127		− vdso_info[abi].dm->pages = &vdso_pagelist[0];
128		− vdso_info[abi].cm->pages = &vdso_pagelist[1];
	157	+ if (vma_is_special_mapping(vma, vdso_info[VDSO_ABI_AA64].dm))
	158	+ zap_page_range(vma, vma->vm_start, size);
	159	+#ifdef CONFIG_COMPAT_VDSO
	160	+ if (vma_is_special_mapping(vma, vdso_info[VDSO_ABI_AA32].dm))
	161	+ zap_page_range(vma, vma->vm_start, size);
	162	+#endif
	163	+ }
	164	+
	165	+ mmap_read_unlock(mm);
	166	+ return 0;
	167	+}
	168	+
	169	+static struct page find_timens_vvar_page(struct vm_area_struct vma)
	170	+{
	171	+ if (likely(vma->vm_mm == current->mm))
	172	+ return current->nsproxy->time_ns->vvar_page;
	173	+
	174	+ /*
	175	+ * VM_PFNMAP \| VM_IO protect .fault() handler from being called
	176	+ * through interfaces like /proc/$pid/mem or
	177	+ * process_vm_{readv,writev}() as long as there's no .access()
	178	+ * in special_mapping_vmops.
	179	+ * For more details check_vma_flags() and __access_remote_vm()
	180	+ */
	181	+ WARN(1, "vvar_page accessed remotely");
	182	+
	183	+ return NULL;
	184	+}
	185	+#else
	186	+static struct page find_timens_vvar_page(struct vm_area_struct vma)
	187	+{
	188	+ return NULL;
	189	+}
	190	+#endif
	191	+
	192	+static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
	193	+ struct vm_area_struct vma, struct vm_fault vmf)
	194	+{
	195	+ struct page *timens_page = find_timens_vvar_page(vma);
	196	+ unsigned long pfn;
	197	+
	198	+ switch (vmf->pgoff) {
	199	+ case VVAR_DATA_PAGE_OFFSET:
	200	+ if (timens_page)
	201	+ pfn = page_to_pfn(timens_page);
	202	+ else
	203	+ pfn = sym_to_pfn(vdso_data);
	204	+ break;
	205	+#ifdef CONFIG_TIME_NS
	206	+ case VVAR_TIMENS_PAGE_OFFSET:
	207	+ /*
	208	+ * If a task belongs to a time namespace then a namespace
	209	+ * specific VVAR is mapped with the VVAR_DATA_PAGE_OFFSET and
	210	+ * the real VVAR page is mapped with the VVAR_TIMENS_PAGE_OFFSET
	211	+ * offset.
	212	+ * See also the comment near timens_setup_vdso_data().
	213	+ */
	214	+ if (!timens_page)
	215	+ return VM_FAULT_SIGBUS;
	216	+ pfn = sym_to_pfn(vdso_data);
	217	+ break;
	218	+#endif /* CONFIG_TIME_NS */
	219	+ default:
	220	+ return VM_FAULT_SIGBUS;
	221	+ }
	222	+
	223	+ return vmf_insert_pfn(vma, vmf->address, pfn);
	224	+}
	225	+
	226	+static int vvar_mremap(const struct vm_special_mapping *sm,
	227	+ struct vm_area_struct *new_vma)
	228	+{
	229	+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
	230	+
	231	+ if (new_size != VVAR_NR_PAGES * PAGE_SIZE)
	232	+ return -EINVAL;
129	233
130	234	return 0;
131	235	}
@@ -139,9 +243,11 @@ static int __setup_additional_pages(enum vdso_abi abi, · 7 unmodified lines
139	243	unsigned long gp_flags = 0;
140	244	void *ret;
141	245
	246	+ BUILD_BUG_ON(VVAR_NR_PAGES != __VVAR_PAGES);
	247	+
142	248	vdso_text_len = vdso_info[abi].vdso_pages << PAGE_SHIFT;
143	249	/* Be sure to map the data page */
144		− vdso_mapping_len = vdso_text_len + PAGE_SIZE;
	250	+ vdso_mapping_len = vdso_text_len + VVAR_NR_PAGES * PAGE_SIZE;
145	251
146	252	vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
147	253	if (IS_ERR_VALUE(vdso_base)) {
@@ -149,8 +255,8 @@ static int __setup_additional_pages(enum vdso_abi abi, · 1 unmodified line
149	255	goto up_fail;
150	256	}
151	257
152		− ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
153		− VM_READ\|VM_MAYREAD,
	258	+ ret = _install_special_mapping(mm, vdso_base, VVAR_NR_PAGES * PAGE_SIZE,
	259	+ VM_READ\|VM_MAYREAD\|VM_PFNMAP,
154	260	vdso_info[abi].dm);
155	261	if (IS_ERR(ret))
156	262	goto up_fail;
@@ -158,7 +264,7 @@ static int __setup_additional_pages(enum vdso_abi abi, · 1 unmodified line
158	264	if (IS_ENABLED(CONFIG_ARM64_BTI_KERNEL) && system_supports_bti())
159	265	gp_flags = VM_ARM64_BTI;
160	266
161		− vdso_base += PAGE_SIZE;
	267	+ vdso_base += VVAR_NR_PAGES * PAGE_SIZE;
162	268	mm->context.vdso = (void *)vdso_base;
163	269	ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
164	270	VM_READ\|VM_EXEC\|gp_flags\|
@@ -206,6 +312,8 @@ static struct vm_special_mapping aarch32_vdso_maps[] = { · 41 unmodified lines
206	312	#ifdef CONFIG_COMPAT_VDSO
207	313	[AA32_MAP_VVAR] = {
208	314	.name = "[vvar]",
	315	+ .fault = vvar_fault,
	316	+ .mremap = vvar_mremap,
209	317	},
210	318	[AA32_MAP_VDSO] = {
211	319	.name = "[vdso]",
@@ -371,6 +479,8 @@ enum aarch64_map { · 159 unmodified lines
371	479	static struct vm_special_mapping aarch64_vdso_maps[] __ro_after_init = {
372	480	[AA64_MAP_VVAR] = {
373	481	.name = "[vvar]",
	482	+ .fault = vvar_fault,
	483	+ .mremap = vvar_mremap,
374	484	},
375	485	[AA64_MAP_VDSO] = {
376	486	.name = "[vdso]",

arch/arm64/kernel/vdso/vdso.lds.S+4−1

@@ -17,7 +17,10 @@ OUTPUT_ARCH(aarch64) · 16 unmodified lines
17	17
18	18	SECTIONS
19	19	{
20		− PROVIDE(_vdso_data = . - PAGE_SIZE);
	20	+ PROVIDE(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE);
	21	+#ifdef CONFIG_TIME_NS
	22	+ PROVIDE(_timens_data = _vdso_data + PAGE_SIZE);
	23	+#endif
21	24	. = VDSO_LBASE + SIZEOF_HEADERS;
22	25
23	26	.hash : { *(.hash) } :text

arch/arm64/kernel/vdso32/vdso.lds.S+4−1

@@ -17,7 +17,10 @@ OUTPUT_ARCH(arm) · 16 unmodified lines
17	17
18	18	SECTIONS
19	19	{
20		− PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
	20	+ PROVIDE_HIDDEN(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE);
	21	+#ifdef CONFIG_TIME_NS
	22	+ PROVIDE_HIDDEN(_timens_data = _vdso_data + PAGE_SIZE);
	23	+#endif
21	24	. = VDSO_LBASE + SIZEOF_HEADERS;
22	25
23	26	.hash : { *(.hash) } :text

arch/arm64/kernel/vmlinux.lds.S+0−1

@@ -10,7 +10,6 @@ · 9 unmodified lines
10	10	#include <asm-generic/vmlinux.lds.h>
11	11	#include <asm/cache.h>
12	12	#include <asm/kernel-pgtable.h>
13		−#include <asm/thread_info.h>
14	13	#include <asm/memory.h>
15	14	#include <asm/page.h>
16	15

arch/arm64/kvm/hyp-init.S+1−0

@@ -6,6 +6,7 @@ · 5 unmodified lines
6	6
7	7	#include <linux/linkage.h>
8	8
	9	+#include <asm/alternative.h>
9	10	#include <asm/assembler.h>
10	11	#include <asm/kvm_arm.h>
11	12	#include <asm/kvm_mmu.h>

arch/arm64/kvm/sys_regs.c+34−34

@@ -1024,9 +1024,9 @@ static bool access_amu(struct kvm_vcpu vcpu, struct sys_reg_params p, · 1023 unmodified lines
1024	1024
1025	1025	/* Macro to expand the AMU counter and type registers*/
1026	1026	#define AMU_AMEVCNTR0_EL0(n) { SYS_DESC(SYS_AMEVCNTR0_EL0(n)), access_amu }
1027		−#define AMU_AMEVTYPE0_EL0(n) { SYS_DESC(SYS_AMEVTYPE0_EL0(n)), access_amu }
	1027	+#define AMU_AMEVTYPER0_EL0(n) { SYS_DESC(SYS_AMEVTYPER0_EL0(n)), access_amu }
1028	1028	#define AMU_AMEVCNTR1_EL0(n) { SYS_DESC(SYS_AMEVCNTR1_EL0(n)), access_amu }
1029		−#define AMU_AMEVTYPE1_EL0(n) { SYS_DESC(SYS_AMEVTYPE1_EL0(n)), access_amu }
	1029	+#define AMU_AMEVTYPER1_EL0(n) { SYS_DESC(SYS_AMEVTYPER1_EL0(n)), access_amu }
1030	1030
1031	1031	static bool trap_ptrauth(struct kvm_vcpu *vcpu,
1032	1032	struct sys_reg_params *p,
@@ -1629,22 +1629,22 @@ static const struct sys_reg_desc sys_reg_descs[] = { · 596 unmodified lines
1629	1629	AMU_AMEVCNTR0_EL0(13),
1630	1630	AMU_AMEVCNTR0_EL0(14),
1631	1631	AMU_AMEVCNTR0_EL0(15),
1632		− AMU_AMEVTYPE0_EL0(0),
1633		− AMU_AMEVTYPE0_EL0(1),
1634		− AMU_AMEVTYPE0_EL0(2),
1635		− AMU_AMEVTYPE0_EL0(3),
1636		− AMU_AMEVTYPE0_EL0(4),
1637		− AMU_AMEVTYPE0_EL0(5),
1638		− AMU_AMEVTYPE0_EL0(6),
1639		− AMU_AMEVTYPE0_EL0(7),
1640		− AMU_AMEVTYPE0_EL0(8),
1641		− AMU_AMEVTYPE0_EL0(9),
1642		− AMU_AMEVTYPE0_EL0(10),
1643		− AMU_AMEVTYPE0_EL0(11),
1644		− AMU_AMEVTYPE0_EL0(12),
1645		− AMU_AMEVTYPE0_EL0(13),
1646		− AMU_AMEVTYPE0_EL0(14),
1647		− AMU_AMEVTYPE0_EL0(15),
	1632	+ AMU_AMEVTYPER0_EL0(0),
	1633	+ AMU_AMEVTYPER0_EL0(1),
	1634	+ AMU_AMEVTYPER0_EL0(2),
	1635	+ AMU_AMEVTYPER0_EL0(3),
	1636	+ AMU_AMEVTYPER0_EL0(4),
	1637	+ AMU_AMEVTYPER0_EL0(5),
	1638	+ AMU_AMEVTYPER0_EL0(6),
	1639	+ AMU_AMEVTYPER0_EL0(7),
	1640	+ AMU_AMEVTYPER0_EL0(8),
	1641	+ AMU_AMEVTYPER0_EL0(9),
	1642	+ AMU_AMEVTYPER0_EL0(10),
	1643	+ AMU_AMEVTYPER0_EL0(11),
	1644	+ AMU_AMEVTYPER0_EL0(12),
	1645	+ AMU_AMEVTYPER0_EL0(13),
	1646	+ AMU_AMEVTYPER0_EL0(14),
	1647	+ AMU_AMEVTYPER0_EL0(15),
1648	1648	AMU_AMEVCNTR1_EL0(0),
1649	1649	AMU_AMEVCNTR1_EL0(1),
1650	1650	AMU_AMEVCNTR1_EL0(2),
@@ -1661,22 +1661,22 @@ static const struct sys_reg_desc sys_reg_descs[] = { · 10 unmodified lines
1661	1661	AMU_AMEVCNTR1_EL0(13),
1662	1662	AMU_AMEVCNTR1_EL0(14),
1663	1663	AMU_AMEVCNTR1_EL0(15),
1664		− AMU_AMEVTYPE1_EL0(0),
1665		− AMU_AMEVTYPE1_EL0(1),
1666		− AMU_AMEVTYPE1_EL0(2),
1667		− AMU_AMEVTYPE1_EL0(3),
1668		− AMU_AMEVTYPE1_EL0(4),
1669		− AMU_AMEVTYPE1_EL0(5),
1670		− AMU_AMEVTYPE1_EL0(6),
1671		− AMU_AMEVTYPE1_EL0(7),
1672		− AMU_AMEVTYPE1_EL0(8),
1673		− AMU_AMEVTYPE1_EL0(9),
1674		− AMU_AMEVTYPE1_EL0(10),
1675		− AMU_AMEVTYPE1_EL0(11),
1676		− AMU_AMEVTYPE1_EL0(12),
1677		− AMU_AMEVTYPE1_EL0(13),
1678		− AMU_AMEVTYPE1_EL0(14),
1679		− AMU_AMEVTYPE1_EL0(15),
	1664	+ AMU_AMEVTYPER1_EL0(0),
	1665	+ AMU_AMEVTYPER1_EL0(1),
	1666	+ AMU_AMEVTYPER1_EL0(2),
	1667	+ AMU_AMEVTYPER1_EL0(3),
	1668	+ AMU_AMEVTYPER1_EL0(4),
	1669	+ AMU_AMEVTYPER1_EL0(5),
	1670	+ AMU_AMEVTYPER1_EL0(6),
	1671	+ AMU_AMEVTYPER1_EL0(7),
	1672	+ AMU_AMEVTYPER1_EL0(8),
	1673	+ AMU_AMEVTYPER1_EL0(9),
	1674	+ AMU_AMEVTYPER1_EL0(10),
	1675	+ AMU_AMEVTYPER1_EL0(11),
	1676	+ AMU_AMEVTYPER1_EL0(12),
	1677	+ AMU_AMEVTYPER1_EL0(13),
	1678	+ AMU_AMEVTYPER1_EL0(14),
	1679	+ AMU_AMEVTYPER1_EL0(15),
1680	1680
1681	1681	{ SYS_DESC(SYS_CNTP_TVAL_EL0), access_arch_timer },
1682	1682	{ SYS_DESC(SYS_CNTP_CTL_EL0), access_arch_timer },

arch/arm64/mm/context.c+6−4

@@ -198,9 +198,10 @@ set_asid: · 197 unmodified lines
198	198	return idx2asid(asid) \| generation;
199	199	}
200	200
201		−void check_and_switch_context(struct mm_struct *mm, unsigned int cpu)
	201	+void check_and_switch_context(struct mm_struct *mm)
202	202	{
203	203	unsigned long flags;
	204	+ unsigned int cpu;
204	205	u64 asid, old_active_asid;
205	206
206	207	if (system_supports_cnp())
@@ -222,9 +223,9 @@ void check_and_switch_context(struct mm_struct *mm, unsigned int cpu) · 15 unmodified lines
222	223	* relaxed xchg in flush_context will treat us as reserved
223	224	* because atomic RmWs are totally ordered for a given location.
224	225	*/
225		− old_active_asid = atomic64_read(&per_cpu(active_asids, cpu));
	226	+ old_active_asid = atomic64_read(this_cpu_ptr(&active_asids));
226	227	if (old_active_asid && asid_gen_match(asid) &&
227		− atomic64_cmpxchg_relaxed(&per_cpu(active_asids, cpu),
	228	+ atomic64_cmpxchg_relaxed(this_cpu_ptr(&active_asids),
228	229	old_active_asid, asid))
229	230	goto switch_mm_fastpath;
230	231
@@ -236,10 +237,11 @@ void check_and_switch_context(struct mm_struct *mm, unsigned int cpu) · 5 unmodified lines
236	237	atomic64_set(&mm->context.id, asid);
237	238	}
238	239
	240	+ cpu = smp_processor_id();
239	241	if (cpumask_test_and_clear_cpu(cpu, &tlb_flush_pending))
240	242	local_flush_tlb_all();
241	243
242		− atomic64_set(&per_cpu(active_asids, cpu), asid);
	244	+ atomic64_set(this_cpu_ptr(&active_asids), asid);
243	245	raw_spin_unlock_irqrestore(&cpu_asid_lock, flags);
244	246
245	247	switch_mm_fastpath:

arch/arm64/mm/hugetlbpage.c+40−2

@@ -19,6 +19,44 @@ · 18 unmodified lines
19	19	#include <asm/tlbflush.h>
20	20	#include <asm/pgalloc.h>
21	21
	22	+/*
	23	+ * HugeTLB Support Matrix
	24	+ *
	25	+ * ---------------------------------------------------
	26	+ * \| Page Size \| CONT PTE \| PMD \| CONT PMD \| PUD \|
	27	+ * ---------------------------------------------------
	28	+ * \| 4K \| 64K \| 2M \| 32M \| 1G \|
	29	+ * \| 16K \| 2M \| 32M \| 1G \| \|
	30	+ * \| 64K \| 2M \| 512M \| 16G \| \|
	31	+ * ---------------------------------------------------
	32	+ */
	33	+
	34	+/*
	35	+ * Reserve CMA areas for the largest supported gigantic
	36	+ * huge page when requested. Any other smaller gigantic
	37	+ * huge pages could still be served from those areas.
	38	+ */
	39	+#ifdef CONFIG_CMA
	40	+void __init arm64_hugetlb_cma_reserve(void)
	41	+{
	42	+ int order;
	43	+
	44	+#ifdef CONFIG_ARM64_4K_PAGES
	45	+ order = PUD_SHIFT - PAGE_SHIFT;
	46	+#else
	47	+ order = CONT_PMD_SHIFT + PMD_SHIFT - PAGE_SHIFT;
	48	+#endif
	49	+ /*
	50	+ * HugeTLB CMA reservation is required for gigantic
	51	+ * huge pages which could not be allocated via the
	52	+ * page allocator. Just warn if there is any change
	53	+ * breaking this assumption.
	54	+ */
	55	+ WARN_ON(order <= MAX_ORDER);
	56	+ hugetlb_cma_reserve(order);
	57	+}
	58	+#endif /* CONFIG_CMA */
	59	+
22	60	#ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION
23	61	bool arch_hugetlb_migration_supported(struct hstate *h)
24	62	{
@@ -457,9 +495,9 @@ static int __init hugetlbpage_init(void) · 432 unmodified lines
457	495	#ifdef CONFIG_ARM64_4K_PAGES
458	496	hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
459	497	#endif
460		− hugetlb_add_hstate((CONT_PMD_SHIFT + PMD_SHIFT) - PAGE_SHIFT);
	498	+ hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
461	499	hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
462		− hugetlb_add_hstate((CONT_PTE_SHIFT + PAGE_SHIFT) - PAGE_SHIFT);
	500	+ hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
463	501
464	502	return 0;
465	503	}

arch/arm64/mm/init.c+3−19

@@ -425,8 +425,8 @@ void __init bootmem_init(void) · 424 unmodified lines
425	425	* initialize node_online_map that gets used in hugetlb_cma_reserve()
426	426	* while allocating required CMA size across online nodes.
427	427	*/
428		−#ifdef CONFIG_ARM64_4K_PAGES
429		− hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT);
	428	+#if defined(CONFIG_HUGETLB_PAGE) && defined(CONFIG_CMA)
	429	+ arm64_hugetlb_cma_reserve();
430	430	#endif
431	431
432	432	/*
@@ -563,27 +563,11 @@ void free_initmem(void) · 130 unmodified lines
563	563	unmap_kernel_range((u64)__init_begin, (u64)(__init_end - __init_begin));
564	564	}
565	565
566		−/*
567		− * Dump out memory limit information on panic.
568		− */
569		−static int dump_mem_limit(struct notifier_block self, unsigned long v, void p)
	566	+void dump_mem_limit(void)
570	567	{
571	568	if (memory_limit != PHYS_ADDR_MAX) {
572	569	pr_emerg("Memory Limit: %llu MB\n", memory_limit >> 20);
573	570	} else {
574	571	pr_emerg("Memory Limit: none\n");
575	572	}
576		− return 0;
577		−}
578		−
579		−static struct notifier_block mem_limit_notifier = {
580		− .notifier_call = dump_mem_limit,
581		−};
582		−
583		−static int __init register_mem_limit_dumper(void)
584		−{
585		− atomic_notifier_chain_register(&panic_notifier_list,
586		− &mem_limit_notifier);
587		− return 0;
588	573	}
589		−__initcall(register_mem_limit_dumper);

arch/riscv/include/asm/vdso/gettimeofday.h+1−0

@@ -4,6 +4,7 @@ · 3 unmodified lines
4	4
5	5	#ifndef __ASSEMBLY__
6	6
	7	+#include <asm/barrier.h>
7	8	#include <asm/unistd.h>
8	9	#include <asm/csr.h>
9	10	#include <uapi/linux/time.h>

drivers/acpi/arm64/iort.c+75−33

@@ -264,15 +264,31 @@ static acpi_status iort_match_node_callback(struct acpi_iort_node *node, · 263 unmodified lines
264	264
265	265	if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT) {
266	266	struct acpi_buffer buf = { ACPI_ALLOCATE_BUFFER, NULL };
267		− struct acpi_device *adev = to_acpi_device_node(dev->fwnode);
	267	+ struct acpi_device *adev;
268	268	struct acpi_iort_named_component *ncomp;
	269	+ struct device *nc_dev = dev;
	270	+
	271	+ /*
	272	+ * Walk the device tree to find a device with an
	273	+ * ACPI companion; there is no point in scanning
	274	+ * IORT for a device matching a named component if
	275	+ * the device does not have an ACPI companion to
	276	+ * start with.
	277	+ */
	278	+ do {
	279	+ adev = ACPI_COMPANION(nc_dev);
	280	+ if (adev)
	281	+ break;
	282	+
	283	+ nc_dev = nc_dev->parent;
	284	+ } while (nc_dev);
269	285
270	286	if (!adev)
271	287	goto out;
272	288
273	289	status = acpi_get_name(adev->handle, ACPI_FULL_PATHNAME, &buf);
274	290	if (ACPI_FAILURE(status)) {
275		− dev_warn(dev, "Can't get device full path name\n");
	291	+ dev_warn(nc_dev, "Can't get device full path name\n");
276	292	goto out;
277	293	}
278	294
@@ -534,7 +550,6 @@ static struct acpi_iort_node iort_find_dev_node(struct device dev) · 255 unmodified lines
534	550	node = iort_get_iort_node(dev->fwnode);
535	551	if (node)
536	552	return node;
537		−
538	553	/*
539	554	* if not, then it should be a platform device defined in
540	555	* DSDT/SSDT (with Named Component node in IORT)
@@ -543,32 +558,29 @@ static struct acpi_iort_node iort_find_dev_node(struct device dev) · 2 unmodified lines
543	558	iort_match_node_callback, dev);
544	559	}
545	560
546		− /* Find a PCI root bus */
547	561	pbus = to_pci_dev(dev)->bus;
548		− while (!pci_is_root_bus(pbus))
549		− pbus = pbus->parent;
550	562
551	563	return iort_scan_node(ACPI_IORT_NODE_PCI_ROOT_COMPLEX,
552	564	iort_match_node_callback, &pbus->dev);
553	565	}
554	566
555	567	/**
556		− * iort_msi_map_rid() - Map a MSI requester ID for a device
	568	+ * iort_msi_map_id() - Map a MSI input ID for a device
557	569	* @dev: The device for which the mapping is to be done.
558		− * @req_id: The device requester ID.
	570	+ * @input_id: The device input ID.
559	571	*
560		− * Returns: mapped MSI RID on success, input requester ID otherwise
	572	+ * Returns: mapped MSI ID on success, input ID otherwise
561	573	*/
562		−u32 iort_msi_map_rid(struct device *dev, u32 req_id)
	574	+u32 iort_msi_map_id(struct device *dev, u32 input_id)
563	575	{
564	576	struct acpi_iort_node *node;
565	577	u32 dev_id;
566	578
567	579	node = iort_find_dev_node(dev);
568	580	if (!node)
569		− return req_id;
	581	+ return input_id;
570	582
571		− iort_node_map_id(node, req_id, &dev_id, IORT_MSI_TYPE);
	583	+ iort_node_map_id(node, input_id, &dev_id, IORT_MSI_TYPE);
572	584	return dev_id;
573	585	}
574	586
@@ -625,13 +637,13 @@ static int __maybe_unused iort_find_its_base(u32 its_id, phys_addr_t *base) · 50 unmodified lines
625	637	/**
626	638	* iort_dev_find_its_id() - Find the ITS identifier for a device
627	639	* @dev: The device.
628		− * @req_id: Device's requester ID
	640	+ * @id: Device's ID
629	641	* @idx: Index of the ITS identifier list.
630	642	* @its_id: ITS identifier.
631	643	*
632	644	* Returns: 0 on success, appropriate error value otherwise
633	645	*/
634		−static int iort_dev_find_its_id(struct device *dev, u32 req_id,
	646	+static int iort_dev_find_its_id(struct device *dev, u32 id,
635	647	unsigned int idx, int *its_id)
636	648	{
637	649	struct acpi_iort_its_group *its;
@@ -641,7 +653,7 @@ static int iort_dev_find_its_id(struct device *dev, u32 req_id, · 3 unmodified lines
641	653	if (!node)
642	654	return -ENXIO;
643	655
644		− node = iort_node_map_id(node, req_id, NULL, IORT_MSI_TYPE);
	656	+ node = iort_node_map_id(node, id, NULL, IORT_MSI_TYPE);
645	657	if (!node)
646	658	return -ENXIO;
647	659
@@ -664,19 +676,20 @@ static int iort_dev_find_its_id(struct device *dev, u32 req_id, · 16 unmodified lines
664	676	*
665	677	* Returns: the MSI domain for this device, NULL otherwise
666	678	*/
667		−struct irq_domain iort_get_device_domain(struct device dev, u32 req_id)
	679	+struct irq_domain iort_get_device_domain(struct device dev, u32 id,
	680	+ enum irq_domain_bus_token bus_token)
668	681	{
669	682	struct fwnode_handle *handle;
670	683	int its_id;
671	684
672		− if (iort_dev_find_its_id(dev, req_id, 0, &its_id))
	685	+ if (iort_dev_find_its_id(dev, id, 0, &its_id))
673	686	return NULL;
674	687
675	688	handle = iort_find_domain_token(its_id);
676	689	if (!handle)
677	690	return NULL;
678	691
679		− return irq_find_matching_fwnode(handle, DOMAIN_BUS_PCI_MSI);
	692	+ return irq_find_matching_fwnode(handle, bus_token);
680	693	}
681	694
682	695	static void iort_set_device_domain(struct device *dev,
@@ -965,19 +978,54 @@ static void iort_named_component_init(struct device *dev, · 282 unmodified lines
965	978	nc->node_flags);
966	979	}
967	980
	981	+static int iort_nc_iommu_map(struct device dev, struct acpi_iort_node node)
	982	+{
	983	+ struct acpi_iort_node *parent;
	984	+ int err = -ENODEV, i = 0;
	985	+ u32 streamid = 0;
	986	+
	987	+ do {
	988	+
	989	+ parent = iort_node_map_platform_id(node, &streamid,
	990	+ IORT_IOMMU_TYPE,
	991	+ i++);
	992	+
	993	+ if (parent)
	994	+ err = iort_iommu_xlate(dev, parent, streamid);
	995	+ } while (parent && !err);
	996	+
	997	+ return err;
	998	+}
	999	+
	1000	+static int iort_nc_iommu_map_id(struct device *dev,
	1001	+ struct acpi_iort_node *node,
	1002	+ const u32 *in_id)
	1003	+{
	1004	+ struct acpi_iort_node *parent;
	1005	+ u32 streamid;
	1006	+
	1007	+ parent = iort_node_map_id(node, *in_id, &streamid, IORT_IOMMU_TYPE);
	1008	+ if (parent)
	1009	+ return iort_iommu_xlate(dev, parent, streamid);
	1010	+
	1011	+ return -ENODEV;
	1012	+}
	1013	+
	1014	+
968	1015	/**
969		− * iort_iommu_configure - Set-up IOMMU configuration for a device.
	1016	+ * iort_iommu_configure_id - Set-up IOMMU configuration for a device.
970	1017	*
971	1018	* @dev: device to configure
	1019	+ * @id_in: optional input id const value pointer
972	1020	*
973	1021	* Returns: iommu_ops pointer on configuration success
974	1022	* NULL on configuration failure
975	1023	*/
976		−const struct iommu_ops iort_iommu_configure(struct device dev)
	1024	+const struct iommu_ops iort_iommu_configure_id(struct device dev,
	1025	+ const u32 *id_in)
977	1026	{
978		− struct acpi_iort_node node, parent;
	1027	+ struct acpi_iort_node *node;
979	1028	const struct iommu_ops *ops;
980		− u32 streamid = 0;
981	1029	int err = -ENODEV;
982	1030
983	1031	/*
@@ -1006,21 +1054,13 @@ const struct iommu_ops iort_iommu_configure(struct device dev) · 22 unmodified lines
1006	1054	if (fwspec && iort_pci_rc_supports_ats(node))
1007	1055	fwspec->flags \|= IOMMU_FWSPEC_PCI_RC_ATS;
1008	1056	} else {
1009		− int i = 0;
1010		−
1011	1057	node = iort_scan_node(ACPI_IORT_NODE_NAMED_COMPONENT,
1012	1058	iort_match_node_callback, dev);
1013	1059	if (!node)
1014	1060	return NULL;
1015	1061
1016		− do {
1017		− parent = iort_node_map_platform_id(node, &streamid,
1018		− IORT_IOMMU_TYPE,
1019		− i++);
1020		−
1021		− if (parent)
1022		− err = iort_iommu_xlate(dev, parent, streamid);
1023		− } while (parent && !err);
	1062	+ err = id_in ? iort_nc_iommu_map_id(dev, node, id_in) :
	1063	+ iort_nc_iommu_map(dev, node);
1024	1064
1025	1065	if (!err)
1026	1066	iort_named_component_init(dev, node);
@@ -1045,6 +1085,7 @@ const struct iommu_ops iort_iommu_configure(struct device dev) · 18 unmodified lines
1045	1085
1046	1086	return ops;
1047	1087	}
	1088	+
1048	1089	#else
1049	1090	static inline const struct iommu_ops iort_fwspec_iommu_ops(struct device dev)
1050	1091	{ return NULL; }
@@ -1053,7 +1094,8 @@ static inline int iort_add_device_replay(const struct iommu_ops *ops, · 2 unmodified lines
1053	1094	{ return 0; }
1054	1095	int iort_iommu_msi_get_resv_regions(struct device dev, struct list_head head)
1055	1096	{ return 0; }
1056		−const struct iommu_ops iort_iommu_configure(struct device dev)
	1097	+const struct iommu_ops iort_iommu_configure_id(struct device dev,
	1098	+ const u32 *input_id)
1057	1099	{ return NULL; }
1058	1100	#endif
1059	1101

drivers/acpi/scan.c+5−3

@@ -1457,8 +1457,10 @@ int acpi_dma_get_range(struct device dev, u64 dma_addr, u64 *offset, · 1456 unmodified lines
1457	1457	* acpi_dma_configure - Set-up DMA configuration for the device.
1458	1458	* @dev: The pointer to the device
1459	1459	* @attr: device dma attributes
	1460	+ * @input_id: input device id const value pointer
1460	1461	*/
1461		−int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr)
	1462	+int acpi_dma_configure_id(struct device *dev, enum dev_dma_attr attr,
	1463	+ const u32 *input_id)
1462	1464	{
1463	1465	const struct iommu_ops *iommu;
1464	1466	u64 dma_addr = 0, size = 0;
@@ -1470,7 +1472,7 @@ int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr) · 5 unmodified lines
1470	1472
1471	1473	iort_dma_setup(dev, &dma_addr, &size);
1472	1474
1473		− iommu = iort_iommu_configure(dev);
	1475	+ iommu = iort_iommu_configure_id(dev, input_id);
1474	1476	if (PTR_ERR(iommu) == -EPROBE_DEFER)
1475	1477	return -EPROBE_DEFER;
1476	1478
@@ -1479,7 +1481,7 @@ int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr) · 2 unmodified lines
1479	1481
1480	1482	return 0;
1481	1483	}
1482		−EXPORT_SYMBOL_GPL(acpi_dma_configure);
	1484	+EXPORT_SYMBOL_GPL(acpi_dma_configure_id);
1483	1485
1484	1486	static void acpi_init_coherency(struct acpi_device *adev)
1485	1487	{

drivers/bus/fsl-mc/dprc-driver.c+8−23

@@ -592,6 +592,7 @@ static int dprc_probe(struct fsl_mc_device *mc_dev) · 591 unmodified lines
592	592	bool mc_io_created = false;
593	593	bool msi_domain_set = false;
594	594	u16 major_ver, minor_ver;
	595	+ struct irq_domain *mc_msi_domain;
595	596
596	597	if (!is_fsl_mc_bus_dprc(mc_dev))
597	598	return -EINVAL;
@@ -621,31 +622,15 @@ static int dprc_probe(struct fsl_mc_device *mc_dev) · 23 unmodified lines
621	622	return error;
622	623
623	624	mc_io_created = true;
	625	+ }
624	626
625		− /*
626		− * Inherit parent MSI domain:
627		− */
628		− dev_set_msi_domain(&mc_dev->dev,
629		− dev_get_msi_domain(parent_dev));
630		− msi_domain_set = true;
	627	+ mc_msi_domain = fsl_mc_find_msi_domain(&mc_dev->dev);
	628	+ if (!mc_msi_domain) {
	629	+ dev_warn(&mc_dev->dev,
	630	+ "WARNING: MC bus without interrupt support\n");
631	631	} else {
632		− /*
633		− * This is a root DPRC
634		− */
635		− struct irq_domain *mc_msi_domain;
636		−
637		− if (dev_is_fsl_mc(parent_dev))
638		− return -EINVAL;
639		−
640		− error = fsl_mc_find_msi_domain(parent_dev,
641		− &mc_msi_domain);
642		− if (error < 0) {
643		− dev_warn(&mc_dev->dev,
644		− "WARNING: MC bus without interrupt support\n");
645		− } else {
646		− dev_set_msi_domain(&mc_dev->dev, mc_msi_domain);
647		− msi_domain_set = true;
648		− }
	632	+ dev_set_msi_domain(&mc_dev->dev, mc_msi_domain);
	633	+ msi_domain_set = true;
649	634	}
650	635
651	636	error = dprc_open(mc_dev->mc_io, 0, mc_dev->obj_desc.id,

drivers/bus/fsl-mc/fsl-mc-bus.c+59−20

@@ -18,6 +18,8 @@ · 17 unmodified lines
18	18	#include <linux/bitops.h>
19	19	#include <linux/msi.h>
20	20	#include <linux/dma-mapping.h>
	21	+#include <linux/acpi.h>
	22	+#include <linux/iommu.h>
21	23
22	24	#include "fsl-mc-private.h"
23	25
@@ -38,6 +40,7 @@ struct fsl_mc { · 14 unmodified lines
38	40	struct fsl_mc_device *root_mc_bus_dev;
39	41	u8 num_translation_ranges;
40	42	struct fsl_mc_addr_translation_range *translation_ranges;
	43	+ void *fsl_mc_regs;
41	44	};
42	45
43	46	/**
@@ -56,6 +59,10 @@ struct fsl_mc_addr_translation_range { · 12 unmodified lines
56	59	phys_addr_t start_phys_addr;
57	60	};
58	61
	62	+#define FSL_MC_FAPR 0x28
	63	+#define MC_FAPR_PL BIT(18)
	64	+#define MC_FAPR_BMT BIT(17)
	65	+
59	66	/**
60	67	* fsl_mc_bus_match - device to driver matching callback
61	68	* @dev: the fsl-mc device to match against
@@ -118,11 +125,16 @@ static int fsl_mc_bus_uevent(struct device dev, struct kobj_uevent_env env) · 56 unmodified lines
118	125	static int fsl_mc_dma_configure(struct device *dev)
119	126	{
120	127	struct device *dma_dev = dev;
	128	+ struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
	129	+ u32 input_id = mc_dev->icid;
121	130
122	131	while (dev_is_fsl_mc(dma_dev))
123	132	dma_dev = dma_dev->parent;
124	133
125		− return of_dma_configure(dev, dma_dev->of_node, 0);
	134	+ if (dev_of_node(dma_dev))
	135	+ return of_dma_configure_id(dev, dma_dev->of_node, 0, &input_id);
	136	+
	137	+ return acpi_dma_configure_id(dev, DEV_DMA_COHERENT, &input_id);
126	138	}
127	139
128	140	static ssize_t modalias_show(struct device dev, struct device_attribute attr,
@@ -368,8 +380,8 @@ EXPORT_SYMBOL_GPL(fsl_mc_get_version); · 239 unmodified lines
368	380	/**
369	381	* fsl_mc_get_root_dprc - function to traverse to the root dprc
370	382	*/
371		−static void fsl_mc_get_root_dprc(struct device *dev,
372		− struct device **root_dprc_dev)
	383	+void fsl_mc_get_root_dprc(struct device *dev,
	384	+ struct device **root_dprc_dev)
373	385	{
374	386	if (!dev) {
375	387	*root_dprc_dev = NULL;
@@ -863,8 +875,11 @@ static int fsl_mc_bus_probe(struct platform_device *pdev) · 487 unmodified lines
863	875	struct fsl_mc_io *mc_io = NULL;
864	876	int container_id;
865	877	phys_addr_t mc_portal_phys_addr;
866		− u32 mc_portal_size;
867		− struct resource res;
	878	+ u32 mc_portal_size, mc_stream_id;
	879	+ struct resource *plat_res;
	880	+
	881	+ if (!iommu_present(&fsl_mc_bus_type))
	882	+ return -EPROBE_DEFER;
868	883
869	884	mc = devm_kzalloc(&pdev->dev, sizeof(*mc), GFP_KERNEL);
870	885	if (!mc)
@@ -872,19 +887,33 @@ static int fsl_mc_bus_probe(struct platform_device *pdev) · 1 unmodified line
872	887
873	888	platform_set_drvdata(pdev, mc);
874	889
	890	+ plat_res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
	891	+ mc->fsl_mc_regs = devm_ioremap_resource(&pdev->dev, plat_res);
	892	+ if (IS_ERR(mc->fsl_mc_regs))
	893	+ return PTR_ERR(mc->fsl_mc_regs);
	894	+
	895	+ if (IS_ENABLED(CONFIG_ACPI) && !dev_of_node(&pdev->dev)) {
	896	+ mc_stream_id = readl(mc->fsl_mc_regs + FSL_MC_FAPR);
	897	+ /*
	898	+ * HW ORs the PL and BMT bit, places the result in bit 15 of
	899	+ * the StreamID and ORs in the ICID. Calculate it accordingly.
	900	+ */
	901	+ mc_stream_id = (mc_stream_id & 0xffff) \|
	902	+ ((mc_stream_id & (MC_FAPR_PL \| MC_FAPR_BMT)) ?
	903	+ 0x4000 : 0);
	904	+ error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
	905	+ &mc_stream_id);
	906	+ if (error)
	907	+ dev_warn(&pdev->dev, "failed to configure dma: %d.\n",
	908	+ error);
	909	+ }
	910	+
875	911	/*
876	912	* Get physical address of MC portal for the root DPRC:
877	913	*/
878		− error = of_address_to_resource(pdev->dev.of_node, 0, &res);
879		− if (error < 0) {
880		− dev_err(&pdev->dev,
881		− "of_address_to_resource() failed for %pOF\n",
882		− pdev->dev.of_node);
883		− return error;
884		− }
885		−
886		− mc_portal_phys_addr = res.start;
887		− mc_portal_size = resource_size(&res);
	914	+ plat_res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
	915	+ mc_portal_phys_addr = plat_res->start;
	916	+ mc_portal_size = resource_size(plat_res);
888	917	error = fsl_create_mc_io(&pdev->dev, mc_portal_phys_addr,
889	918	mc_portal_size, NULL,
890	919	FSL_MC_IO_ATOMIC_CONTEXT_PORTAL, &mc_io);
@@ -901,11 +930,13 @@ static int fsl_mc_bus_probe(struct platform_device *pdev) · 10 unmodified lines
901	930	dev_info(&pdev->dev, "MC firmware version: %u.%u.%u\n",
902	931	mc_version.major, mc_version.minor, mc_version.revision);
903	932
904		− error = get_mc_addr_translation_ranges(&pdev->dev,
905		− &mc->translation_ranges,
906		− &mc->num_translation_ranges);
907		− if (error < 0)
908		− goto error_cleanup_mc_io;
	933	+ if (dev_of_node(&pdev->dev)) {
	934	+ error = get_mc_addr_translation_ranges(&pdev->dev,
	935	+ &mc->translation_ranges,
	936	+ &mc->num_translation_ranges);
	937	+ if (error < 0)
	938	+ goto error_cleanup_mc_io;
	939	+ }
909	940
910	941	error = dprc_get_container_id(mc_io, 0, &container_id);
911	942	if (error < 0) {
@@ -932,6 +963,7 @@ static int fsl_mc_bus_probe(struct platform_device *pdev) · 20 unmodified lines
932	963	goto error_cleanup_mc_io;
933	964
934	965	mc->root_mc_bus_dev = mc_bus_dev;
	966	+ mc_bus_dev->dev.fwnode = pdev->dev.fwnode;
935	967	return 0;
936	968
937	969	error_cleanup_mc_io:
@@ -965,11 +997,18 @@ static const struct of_device_id fsl_mc_bus_match_table[] = { · 27 unmodified lines
965	997
966	998	MODULE_DEVICE_TABLE(of, fsl_mc_bus_match_table);
967	999
	1000	+static const struct acpi_device_id fsl_mc_bus_acpi_match_table[] = {
	1001	+ {"NXP0008", 0 },
	1002	+ { }
	1003	+};
	1004	+MODULE_DEVICE_TABLE(acpi, fsl_mc_bus_acpi_match_table);
	1005	+
968	1006	static struct platform_driver fsl_mc_bus_driver = {
969	1007	.driver = {
970	1008	.name = "fsl_mc_bus",
971	1009	.pm = NULL,
972	1010	.of_match_table = fsl_mc_bus_match_table,
	1011	+ .acpi_match_table = fsl_mc_bus_acpi_match_table,
973	1012	},
974	1013	.probe = fsl_mc_bus_probe,
975	1014	.remove = fsl_mc_bus_remove,

drivers/bus/fsl-mc/fsl-mc-msi.c+25−11

@@ -13,6 +13,7 @@ · 12 unmodified lines
13	13	#include <linux/irq.h>
14	14	#include <linux/irqdomain.h>
15	15	#include <linux/msi.h>
	16	+#include <linux/acpi_iort.h>
16	17
17	18	#include "fsl-mc-private.h"
18	19
@@ -177,23 +178,36 @@ struct irq_domain fsl_mc_msi_create_irq_domain(struct fwnode_handle fwnode, · 158 unmodified lines
177	178	return domain;
178	179	}
179	180
180		−int fsl_mc_find_msi_domain(struct device *mc_platform_dev,
181		− struct irq_domain **mc_msi_domain)
	181	+struct irq_domain fsl_mc_find_msi_domain(struct device dev)
182	182	{
	183	+ struct device *root_dprc_dev;
	184	+ struct device *bus_dev;
183	185	struct irq_domain *msi_domain;
184		− struct device_node *mc_of_node = mc_platform_dev->of_node;
	186	+ struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
185	187
186		− msi_domain = of_msi_get_domain(mc_platform_dev, mc_of_node,
187		− DOMAIN_BUS_FSL_MC_MSI);
188		− if (!msi_domain) {
189		− pr_err("Unable to find fsl-mc MSI domain for %pOF\n",
190		− mc_of_node);
	188	+ fsl_mc_get_root_dprc(dev, &root_dprc_dev);
	189	+ bus_dev = root_dprc_dev->parent;
	190	+
	191	+ if (bus_dev->of_node) {
	192	+ msi_domain = of_msi_map_get_device_domain(dev,
	193	+ mc_dev->icid,
	194	+ DOMAIN_BUS_FSL_MC_MSI);
191	195
192		− return -ENOENT;
	196	+ /*
	197	+ * if the msi-map property is missing assume that all the
	198	+ * child containers inherit the domain from the parent
	199	+ */
	200	+ if (!msi_domain)
	201	+
	202	+ msi_domain = of_msi_get_domain(bus_dev,
	203	+ bus_dev->of_node,
	204	+ DOMAIN_BUS_FSL_MC_MSI);
	205	+ } else {
	206	+ msi_domain = iort_get_device_domain(dev, mc_dev->icid,
	207	+ DOMAIN_BUS_FSL_MC_MSI);
193	208	}
194	209
195		− *mc_msi_domain = msi_domain;
196		− return 0;
	210	+ return msi_domain;
197	211	}
198	212
199	213	static void fsl_mc_msi_free_descs(struct device *dev)

drivers/bus/fsl-mc/fsl-mc-private.h+4−2

@@ -595,8 +595,7 @@ int fsl_mc_msi_domain_alloc_irqs(struct device *dev, · 594 unmodified lines
595	595
596	596	void fsl_mc_msi_domain_free_irqs(struct device *dev);
597	597
598		−int fsl_mc_find_msi_domain(struct device *mc_platform_dev,
599		− struct irq_domain **mc_msi_domain);
	598	+struct irq_domain fsl_mc_find_msi_domain(struct device dev);
600	599
601	600	int fsl_mc_populate_irq_pool(struct fsl_mc_bus *mc_bus,
602	601	unsigned int irq_count);
@@ -613,6 +612,9 @@ void fsl_destroy_mc_io(struct fsl_mc_io *mc_io); · 10 unmodified lines
613	612
614	613	bool fsl_mc_is_root_dprc(struct device *dev);
615	614
	615	+void fsl_mc_get_root_dprc(struct device *dev,
	616	+ struct device **root_dprc_dev);
	617	+
616	618	struct fsl_mc_device fsl_mc_device_lookup(struct fsl_mc_obj_desc obj_desc,
617	619	struct fsl_mc_device *mc_bus_dev);
618	620

drivers/iommu/of_iommu.c+44−37

@@ -118,46 +118,66 @@ static int of_iommu_xlate(struct device *dev, · 117 unmodified lines
118	118	return ret;
119	119	}
120	120
121		−struct of_pci_iommu_alias_info {
122		− struct device *dev;
123		− struct device_node *np;
124		−};
125		−
126		−static int of_pci_iommu_init(struct pci_dev pdev, u16 alias, void data)
	121	+static int of_iommu_configure_dev_id(struct device_node *master_np,
	122	+ struct device *dev,
	123	+ const u32 *id)
127	124	{
128		− struct of_pci_iommu_alias_info *info = data;
129	125	struct of_phandle_args iommu_spec = { .args_count = 1 };
130	126	int err;
131	127
132		− err = of_map_rid(info->np, alias, "iommu-map", "iommu-map-mask",
133		− &iommu_spec.np, iommu_spec.args);
	128	+ err = of_map_id(master_np, *id, "iommu-map",
	129	+ "iommu-map-mask", &iommu_spec.np,
	130	+ iommu_spec.args);
134	131	if (err)
135	132	return err == -ENODEV ? NO_IOMMU : err;
136	133
137		− err = of_iommu_xlate(info->dev, &iommu_spec);
	134	+ err = of_iommu_xlate(dev, &iommu_spec);
138	135	of_node_put(iommu_spec.np);
139	136	return err;
140	137	}
141	138
142		−static int of_fsl_mc_iommu_init(struct fsl_mc_device *mc_dev,
143		− struct device_node *master_np)
	139	+static int of_iommu_configure_dev(struct device_node *master_np,
	140	+ struct device *dev)
144	141	{
145		− struct of_phandle_args iommu_spec = { .args_count = 1 };
146		− int err;
147		−
148		− err = of_map_rid(master_np, mc_dev->icid, "iommu-map",
149		− "iommu-map-mask", &iommu_spec.np,
150		− iommu_spec.args);
151		− if (err)
152		− return err == -ENODEV ? NO_IOMMU : err;
	142	+ struct of_phandle_args iommu_spec;
	143	+ int err = NO_IOMMU, idx = 0;
	144	+
	145	+ while (!of_parse_phandle_with_args(master_np, "iommus",
	146	+ "#iommu-cells",
	147	+ idx, &iommu_spec)) {
	148	+ err = of_iommu_xlate(dev, &iommu_spec);
	149	+ of_node_put(iommu_spec.np);
	150	+ idx++;
	151	+ if (err)
	152	+ break;
	153	+ }
153	154
154		− err = of_iommu_xlate(&mc_dev->dev, &iommu_spec);
155		− of_node_put(iommu_spec.np);
156	155	return err;
157	156	}
158	157
	158	+struct of_pci_iommu_alias_info {
	159	+ struct device *dev;
	160	+ struct device_node *np;
	161	+};
	162	+
	163	+static int of_pci_iommu_init(struct pci_dev pdev, u16 alias, void data)
	164	+{
	165	+ struct of_pci_iommu_alias_info *info = data;
	166	+ u32 input_id = alias;
	167	+
	168	+ return of_iommu_configure_dev_id(info->np, info->dev, &input_id);
	169	+}
	170	+
	171	+static int of_iommu_configure_device(struct device_node *master_np,
	172	+ struct device dev, const u32 id)
	173	+{
	174	+ return (id) ? of_iommu_configure_dev_id(master_np, dev, id) :
	175	+ of_iommu_configure_dev(master_np, dev);
	176	+}
	177	+
159	178	const struct iommu_ops of_iommu_configure(struct device dev,
160		− struct device_node *master_np)
	179	+ struct device_node *master_np,
	180	+ const u32 *id)
161	181	{
162	182	const struct iommu_ops *ops = NULL;
163	183	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
@@ -188,21 +208,8 @@ const struct iommu_ops of_iommu_configure(struct device dev, · 24 unmodified lines
188	208	pci_request_acs();
189	209	err = pci_for_each_dma_alias(to_pci_dev(dev),
190	210	of_pci_iommu_init, &info);
191		− } else if (dev_is_fsl_mc(dev)) {
192		− err = of_fsl_mc_iommu_init(to_fsl_mc_device(dev), master_np);
193	211	} else {
194		− struct of_phandle_args iommu_spec;
195		− int idx = 0;
196		−
197		− while (!of_parse_phandle_with_args(master_np, "iommus",
198		− "#iommu-cells",
199		− idx, &iommu_spec)) {
200		− err = of_iommu_xlate(dev, &iommu_spec);
201		− of_node_put(iommu_spec.np);
202		− idx++;
203		− if (err)
204		− break;
205		− }
	212	+ err = of_iommu_configure_device(master_np, dev, id);
206	213
207	214	fwspec = dev_iommu_fwspec_get(dev);
208	215	if (!err && fwspec)

drivers/irqchip/irq-gic-v3-its-fsl-mc-msi.c+86−19

@@ -7,6 +7,8 @@ · 6 unmodified lines
7	7	*
8	8	*/
9	9
	10	+#include <linux/acpi.h>
	11	+#include <linux/acpi_iort.h>
10	12	#include <linux/of_device.h>
11	13	#include <linux/of_address.h>
12	14	#include <linux/irq.h>
@@ -23,6 +25,19 @@ static struct irq_chip its_msi_irq_chip = { · 10 unmodified lines
23	25	.irq_set_affinity = msi_domain_set_affinity
24	26	};
25	27
	28	+static u32 fsl_mc_msi_domain_get_msi_id(struct irq_domain *domain,
	29	+ struct fsl_mc_device *mc_dev)
	30	+{
	31	+ struct device_node *of_node;
	32	+ u32 out_id;
	33	+
	34	+ of_node = irq_domain_get_of_node(domain);
	35	+ out_id = of_node ? of_msi_map_id(&mc_dev->dev, of_node, mc_dev->icid) :
	36	+ iort_msi_map_id(&mc_dev->dev, mc_dev->icid);
	37	+
	38	+ return out_id;
	39	+}
	40	+
26	41	static int its_fsl_mc_msi_prepare(struct irq_domain *msi_domain,
27	42	struct device *dev,
28	43	int nvec, msi_alloc_info_t *info)
@@ -43,7 +58,8 @@ static int its_fsl_mc_msi_prepare(struct irq_domain *msi_domain, · 14 unmodified lines
43	58	* NOTE: This device id corresponds to the IOMMU stream ID
44	59	* associated with the DPRC object (ICID).
45	60	*/
46		− info->scratchpad[0].ul = mc_bus_dev->icid;
	61	+ info->scratchpad[0].ul = fsl_mc_msi_domain_get_msi_id(msi_domain,
	62	+ mc_bus_dev);
47	63	msi_info = msi_get_domain_info(msi_domain->parent);
48	64
49	65	/* Allocate at least 32 MSIs, and always as a power of 2 */
@@ -66,12 +82,71 @@ static const struct of_device_id its_device_id[] = { · 16 unmodified lines
66	82	{},
67	83	};
68	84
69		−static int __init its_fsl_mc_msi_init(void)
	85	+static void __init its_fsl_mc_msi_init_one(struct fwnode_handle *handle,
	86	+ const char *name)
70	87	{
71		− struct device_node *np;
72	88	struct irq_domain *parent;
73	89	struct irq_domain *mc_msi_domain;
74	90
	91	+ parent = irq_find_matching_fwnode(handle, DOMAIN_BUS_NEXUS);
	92	+ if (!parent \|\| !msi_get_domain_info(parent)) {
	93	+ pr_err("%s: unable to locate ITS domain\n", name);
	94	+ return;
	95	+ }
	96	+
	97	+ mc_msi_domain = fsl_mc_msi_create_irq_domain(handle,
	98	+ &its_fsl_mc_msi_domain_info,
	99	+ parent);
	100	+ if (!mc_msi_domain) {
	101	+ pr_err("%s: unable to create fsl-mc domain\n", name);
	102	+ return;
	103	+ }
	104	+
	105	+ pr_info("fsl-mc MSI: %s domain created\n", name);
	106	+}
	107	+
	108	+#ifdef CONFIG_ACPI
	109	+static int __init
	110	+its_fsl_mc_msi_parse_madt(union acpi_subtable_headers *header,
	111	+ const unsigned long end)
	112	+{
	113	+ struct acpi_madt_generic_translator *its_entry;
	114	+ struct fwnode_handle *dom_handle;
	115	+ const char *node_name;
	116	+ int err = 0;
	117	+
	118	+ its_entry = (struct acpi_madt_generic_translator *)header;
	119	+ node_name = kasprintf(GFP_KERNEL, "ITS@0x%lx",
	120	+ (long)its_entry->base_address);
	121	+
	122	+ dom_handle = iort_find_domain_token(its_entry->translation_id);
	123	+ if (!dom_handle) {
	124	+ pr_err("%s: Unable to locate ITS domain handle\n", node_name);
	125	+ err = -ENXIO;
	126	+ goto out;
	127	+ }
	128	+
	129	+ its_fsl_mc_msi_init_one(dom_handle, node_name);
	130	+
	131	+out:
	132	+ kfree(node_name);
	133	+ return err;
	134	+}
	135	+
	136	+
	137	+static void __init its_fsl_mc_acpi_msi_init(void)
	138	+{
	139	+ acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_TRANSLATOR,
	140	+ its_fsl_mc_msi_parse_madt, 0);
	141	+}
	142	+#else
	143	+static inline void its_fsl_mc_acpi_msi_init(void) { }
	144	+#endif
	145	+
	146	+static void __init its_fsl_mc_of_msi_init(void)
	147	+{
	148	+ struct device_node *np;
	149	+
75	150	for (np = of_find_matching_node(NULL, its_device_id); np;
76	151	np = of_find_matching_node(np, its_device_id)) {
77	152	if (!of_device_is_available(np))
@@ -79,23 +154,15 @@ static int __init its_fsl_mc_msi_init(void) · 1 unmodified line
79	154	if (!of_property_read_bool(np, "msi-controller"))
80	155	continue;
81	156
82		− parent = irq_find_matching_host(np, DOMAIN_BUS_NEXUS);
83		− if (!parent \|\| !msi_get_domain_info(parent)) {
84		− pr_err("%pOF: unable to locate ITS domain\n", np);
85		− continue;
86		− }
87		−
88		− mc_msi_domain = fsl_mc_msi_create_irq_domain(
89		− of_node_to_fwnode(np),
90		− &its_fsl_mc_msi_domain_info,
91		− parent);
92		− if (!mc_msi_domain) {
93		− pr_err("%pOF: unable to create fsl-mc domain\n", np);
94		− continue;
95		− }
96		−
97		− pr_info("fsl-mc MSI: %pOF domain created\n", np);
	157	+ its_fsl_mc_msi_init_one(of_node_to_fwnode(np),
	158	+ np->full_name);
98	159	}
	160	+}
	161	+
	162	+static int __init its_fsl_mc_msi_init(void)
	163	+{
	164	+ its_fsl_mc_of_msi_init();
	165	+ its_fsl_mc_acpi_msi_init();
99	166
100	167	return 0;
101	168	}

drivers/of/base.c+21−21

@@ -2201,15 +2201,15 @@ int of_find_last_cache_level(unsigned int cpu) · 2200 unmodified lines
2201	2201	}
2202	2202
2203	2203	/**
2204		− * of_map_rid - Translate a requester ID through a downstream mapping.
	2204	+ * of_map_id - Translate an ID through a downstream mapping.
2205	2205	* @np: root complex device node.
2206		− * @rid: device requester ID to map.
	2206	+ * @id: device ID to map.
2207	2207	* @map_name: property name of the map to use.
2208	2208	* @map_mask_name: optional property name of the mask to use.
2209	2209	* @target: optional pointer to a target device node.
2210	2210	* @id_out: optional pointer to receive the translated ID.
2211	2211	*
2212		− * Given a device requester ID, look up the appropriate implementation-defined
	2212	+ * Given a device ID, look up the appropriate implementation-defined
2213	2213	* platform ID and/or the target device which receives transactions on that
2214	2214	* ID, as per the "iommu-map" and "msi-map" bindings. Either of @target or
2215	2215	* @id_out may be NULL if only the other is required. If @target points to
@@ -2219,11 +2219,11 @@ int of_find_last_cache_level(unsigned int cpu) · 3 unmodified lines
2219	2219	*
2220	2220	* Return: 0 on success or a standard error code on failure.
2221	2221	*/
2222		−int of_map_rid(struct device_node *np, u32 rid,
	2222	+int of_map_id(struct device_node *np, u32 id,
2223	2223	const char map_name, const char map_mask_name,
2224	2224	struct device_node *target, u32 id_out)
2225	2225	{
2226		− u32 map_mask, masked_rid;
	2226	+ u32 map_mask, masked_id;
2227	2227	int map_len;
2228	2228	const __be32 *map = NULL;
2229	2229
@@ -2235,7 +2235,7 @@ int of_map_rid(struct device_node *np, u32 rid, · 5 unmodified lines
2235	2235	if (target)
2236	2236	return -ENODEV;
2237	2237	/* Otherwise, no map implies no translation */
2238		− *id_out = rid;
	2238	+ *id_out = id;
2239	2239	return 0;
2240	2240	}
2241	2241
@@ -2255,22 +2255,22 @@ int of_map_rid(struct device_node *np, u32 rid, · 13 unmodified lines
2255	2255	if (map_mask_name)
2256	2256	of_property_read_u32(np, map_mask_name, &map_mask);
2257	2257
2258		− masked_rid = map_mask & rid;
	2258	+ masked_id = map_mask & id;
2259	2259	for ( ; map_len > 0; map_len -= 4 * sizeof(*map), map += 4) {
2260	2260	struct device_node *phandle_node;
2261		− u32 rid_base = be32_to_cpup(map + 0);
	2261	+ u32 id_base = be32_to_cpup(map + 0);
2262	2262	u32 phandle = be32_to_cpup(map + 1);
2263	2263	u32 out_base = be32_to_cpup(map + 2);
2264		− u32 rid_len = be32_to_cpup(map + 3);
	2264	+ u32 id_len = be32_to_cpup(map + 3);
2265	2265
2266		− if (rid_base & ~map_mask) {
2267		− pr_err("%pOF: Invalid %s translation - %s-mask (0x%x) ignores rid-base (0x%x)\n",
	2266	+ if (id_base & ~map_mask) {
	2267	+ pr_err("%pOF: Invalid %s translation - %s-mask (0x%x) ignores id-base (0x%x)\n",
2268	2268	np, map_name, map_name,
2269		− map_mask, rid_base);
	2269	+ map_mask, id_base);
2270	2270	return -EFAULT;
2271	2271	}
2272	2272
2273		− if (masked_rid < rid_base \|\| masked_rid >= rid_base + rid_len)
	2273	+ if (masked_id < id_base \|\| masked_id >= id_base + id_len)
2274	2274	continue;
2275	2275
2276	2276	phandle_node = of_find_node_by_phandle(phandle);
@@ -2288,20 +2288,20 @@ int of_map_rid(struct device_node *np, u32 rid, · 11 unmodified lines
2288	2288	}
2289	2289
2290	2290	if (id_out)
2291		− *id_out = masked_rid - rid_base + out_base;
	2291	+ *id_out = masked_id - id_base + out_base;
2292	2292
2293		− pr_debug("%pOF: %s, using mask %08x, rid-base: %08x, out-base: %08x, length: %08x, rid: %08x -> %08x\n",
2294		− np, map_name, map_mask, rid_base, out_base,
2295		− rid_len, rid, masked_rid - rid_base + out_base);
	2293	+ pr_debug("%pOF: %s, using mask %08x, id-base: %08x, out-base: %08x, length: %08x, id: %08x -> %08x\n",
	2294	+ np, map_name, map_mask, id_base, out_base,
	2295	+ id_len, id, masked_id - id_base + out_base);
2296	2296	return 0;
2297	2297	}
2298	2298
2299		− pr_info("%pOF: no %s translation for rid 0x%x on %pOF\n", np, map_name,
2300		− rid, target && target ? target : NULL);
	2299	+ pr_info("%pOF: no %s translation for id 0x%x on %pOF\n", np, map_name,
	2300	+ id, target && target ? target : NULL);
2301	2301
2302	2302	/* Bypasses translation */
2303	2303	if (id_out)
2304		− *id_out = rid;
	2304	+ *id_out = id;
2305	2305	return 0;
2306	2306	}
2307		−EXPORT_SYMBOL_GPL(of_map_rid);
	2307	+EXPORT_SYMBOL_GPL(of_map_id);

drivers/of/device.c+5−3

@@ -78,6 +78,7 @@ int of_device_add(struct platform_device *ofdev) · 77 unmodified lines
78	78	* @np: Pointer to OF node having DMA configuration
79	79	* @force_dma: Whether device is to be set up by of_dma_configure() even if
80	80	* DMA capability is not explicitly described by firmware.
	81	+ * @id: Optional const pointer value input id
81	82	*
82	83	* Try to get devices's DMA configuration from DT and update it
83	84	* accordingly.
@@ -86,7 +87,8 @@ int of_device_add(struct platform_device *ofdev) · 2 unmodified lines
86	87	* can use a platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE events
87	88	* to fix up DMA configuration.
88	89	*/
89		−int of_dma_configure(struct device dev, struct device_node np, bool force_dma)
	90	+int of_dma_configure_id(struct device dev, struct device_node np,
	91	+ bool force_dma, const u32 *id)
90	92	{
91	93	u64 dma_addr, paddr, size = 0;
92	94	int ret;
@@ -160,7 +162,7 @@ int of_dma_configure(struct device dev, struct device_node np, bool force_dma) · 67 unmodified lines
160	162	dev_dbg(dev, "device is%sdma coherent\n",
161	163	coherent ? " " : " not ");
162	164
163		− iommu = of_iommu_configure(dev, np);
	165	+ iommu = of_iommu_configure(dev, np, id);
164	166	if (PTR_ERR(iommu) == -EPROBE_DEFER)
165	167	return -EPROBE_DEFER;
166	168
@@ -171,7 +173,7 @@ int of_dma_configure(struct device dev, struct device_node np, bool force_dma) · 4 unmodified lines
171	173
172	174	return 0;
173	175	}
174		−EXPORT_SYMBOL_GPL(of_dma_configure);
	176	+EXPORT_SYMBOL_GPL(of_dma_configure_id);
175	177
176	178	int of_device_register(struct platform_device *pdev)
177	179	{

drivers/of/irq.c+18−16

@@ -576,55 +576,57 @@ err: · 575 unmodified lines
576	576	}
577	577	}
578	578
579		−static u32 __of_msi_map_rid(struct device dev, struct device_node *np,
580		− u32 rid_in)
	579	+static u32 __of_msi_map_id(struct device dev, struct device_node *np,
	580	+ u32 id_in)
581	581	{
582	582	struct device *parent_dev;
583		− u32 rid_out = rid_in;
	583	+ u32 id_out = id_in;
584	584
585	585	/*
586	586	* Walk up the device parent links looking for one with a
587	587	* "msi-map" property.
588	588	*/
589	589	for (parent_dev = dev; parent_dev; parent_dev = parent_dev->parent)
590		− if (!of_map_rid(parent_dev->of_node, rid_in, "msi-map",
591		− "msi-map-mask", np, &rid_out))
	590	+ if (!of_map_id(parent_dev->of_node, id_in, "msi-map",
	591	+ "msi-map-mask", np, &id_out))
592	592	break;
593		− return rid_out;
	593	+ return id_out;
594	594	}
595	595
596	596	/**
597		− * of_msi_map_rid - Map a MSI requester ID for a device.
	597	+ * of_msi_map_id - Map a MSI ID for a device.
598	598	* @dev: device for which the mapping is to be done.
599	599	* @msi_np: device node of the expected msi controller.
600		− * @rid_in: unmapped MSI requester ID for the device.
	600	+ * @id_in: unmapped MSI ID for the device.
601	601	*
602	602	* Walk up the device hierarchy looking for devices with a "msi-map"
603		− * property. If found, apply the mapping to @rid_in.
	603	+ * property. If found, apply the mapping to @id_in.
604	604	*
605		− * Returns the mapped MSI requester ID.
	605	+ * Returns the mapped MSI ID.
606	606	*/
607		−u32 of_msi_map_rid(struct device dev, struct device_node msi_np, u32 rid_in)
	607	+u32 of_msi_map_id(struct device dev, struct device_node msi_np, u32 id_in)
608	608	{
609		− return __of_msi_map_rid(dev, &msi_np, rid_in);
	609	+ return __of_msi_map_id(dev, &msi_np, id_in);
610	610	}
611	611
612	612	/**
613	613	* of_msi_map_get_device_domain - Use msi-map to find the relevant MSI domain
614	614	* @dev: device for which the mapping is to be done.
615		− * @rid: Requester ID for the device.
	615	+ * @id: Device ID.
	616	+ * @bus_token: Bus token
616	617	*
617	618	* Walk up the device hierarchy looking for devices with a "msi-map"
618	619	* property.
619	620	*
620	621	* Returns: the MSI domain for this device (or NULL on failure)
621	622	*/
622		−struct irq_domain of_msi_map_get_device_domain(struct device dev, u32 rid)
	623	+struct irq_domain of_msi_map_get_device_domain(struct device dev, u32 id,
	624	+ u32 bus_token)
623	625	{
624	626	struct device_node *np = NULL;
625	627
626		− __of_msi_map_rid(dev, &np, rid);
627		− return irq_find_matching_host(np, DOMAIN_BUS_PCI_MSI);
	628	+ __of_msi_map_id(dev, &np, id);
	629	+ return irq_find_matching_host(np, bus_token);
628	630	}
629	631
630	632	/**

drivers/pci/msi.c+5−4

@@ -1535,8 +1535,8 @@ u32 pci_msi_domain_get_msi_rid(struct irq_domain domain, struct pci_dev pdev) · 1534 unmodified lines
1535	1535	pci_for_each_dma_alias(pdev, get_msi_id_cb, &rid);
1536	1536
1537	1537	of_node = irq_domain_get_of_node(domain);
1538		− rid = of_node ? of_msi_map_rid(&pdev->dev, of_node, rid) :
1539		− iort_msi_map_rid(&pdev->dev, rid);
	1538	+ rid = of_node ? of_msi_map_id(&pdev->dev, of_node, rid) :
	1539	+ iort_msi_map_id(&pdev->dev, rid);
1540	1540
1541	1541	return rid;
1542	1542	}
@@ -1556,9 +1556,10 @@ struct irq_domain pci_msi_get_device_domain(struct pci_dev pdev) · 13 unmodified lines
1556	1556	u32 rid = pci_dev_id(pdev);
1557	1557
1558	1558	pci_for_each_dma_alias(pdev, get_msi_id_cb, &rid);
1559		− dom = of_msi_map_get_device_domain(&pdev->dev, rid);
	1559	+ dom = of_msi_map_get_device_domain(&pdev->dev, rid, DOMAIN_BUS_PCI_MSI);
1560	1560	if (!dom)
1561		− dom = iort_get_device_domain(&pdev->dev, rid);
	1561	+ dom = iort_get_device_domain(&pdev->dev, rid,
	1562	+ DOMAIN_BUS_PCI_MSI);
1562	1563	return dom;
1563	1564	}
1564	1565	#endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */

drivers/perf/arm_smmuv3_pmu.c+1−2

@@ -756,8 +756,7 @@ static int smmu_pmu_probe(struct platform_device *pdev) · 755 unmodified lines
756	756	.capabilities = PERF_PMU_CAP_NO_EXCLUDE,
757	757	};
758	758
759		− res_0 = platform_get_resource(pdev, IORESOURCE_MEM, 0);
760		− smmu_pmu->reg_base = devm_ioremap_resource(dev, res_0);
	759	+ smmu_pmu->reg_base = devm_platform_get_and_ioremap_resource(pdev, 0, &res_0);
761	760	if (IS_ERR(smmu_pmu->reg_base))
762	761	return PTR_ERR(smmu_pmu->reg_base);
763	762

drivers/vhost/vhost.c+0−5

@@ -2092,11 +2092,6 @@ static int get_indirect(struct vhost_virtqueue *vq, · 2091 unmodified lines
2092	2092	return ret;
2093	2093	}
2094	2094	iov_iter_init(&from, READ, vq->indirect, ret, len);
2095		−
2096		− /* We will use the result as an address to read from, so most
2097		− * architectures only need a compiler barrier here. */
2098		− read_barrier_depends();
2099		−
2100	2095	count = len / sizeof desc;
2101	2096	/* Buffers are chained via a 16 bit next field, so
2102	2097	* we can have at most 2^16 of these. */

include/acpi/acpi_bus.h+7−2

@@ -588,8 +588,13 @@ bool acpi_dma_supported(struct acpi_device *adev); · 587 unmodified lines
588	588	enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev);
589	589	int acpi_dma_get_range(struct device dev, u64 dma_addr, u64 *offset,
590	590	u64 *size);
591		−int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr);
592		−
	591	+int acpi_dma_configure_id(struct device *dev, enum dev_dma_attr attr,
	592	+ const u32 *input_id);
	593	+static inline int acpi_dma_configure(struct device *dev,
	594	+ enum dev_dma_attr attr)
	595	+{
	596	+ return acpi_dma_configure_id(dev, attr, NULL);
	597	+}
593	598	struct acpi_device acpi_find_child_device(struct acpi_device parent,
594	599	u64 address, bool check_children);
595	600	int acpi_is_root_bridge(acpi_handle);

include/asm-generic/Kbuild+1−0

@@ -45,6 +45,7 @@ mandatory-y += pci.h · 44 unmodified lines
45	45	mandatory-y += percpu.h
46	46	mandatory-y += pgalloc.h
47	47	mandatory-y += preempt.h
	48	+mandatory-y += rwonce.h
48	49	mandatory-y += sections.h
49	50	mandatory-y += serial.h
50	51	mandatory-y += shmparam.h

include/asm-generic/barrier.h+1−18

@@ -13,7 +13,7 @@ · 12 unmodified lines
13	13
14	14	#ifndef __ASSEMBLY__
15	15
16		−#include <linux/compiler.h>
	16	+#include <asm/rwonce.h>
17	17
18	18	#ifndef nop
19	19	#define nop() asm volatile ("nop")
@@ -46,10 +46,6 @@ · 26 unmodified lines
46	46	#define dma_wmb() wmb()
47	47	#endif
48	48
49		−#ifndef read_barrier_depends
50		−#define read_barrier_depends() do { } while (0)
51		−#endif
52		−
53	49	#ifndef __smp_mb
54	50	#define __smp_mb() mb()
55	51	#endif
@@ -62,10 +58,6 @@ · 6 unmodified lines
62	58	#define __smp_wmb() wmb()
63	59	#endif
64	60
65		−#ifndef __smp_read_barrier_depends
66		−#define __smp_read_barrier_depends() read_barrier_depends()
67		−#endif
68		−
69	61	#ifdef CONFIG_SMP
70	62
71	63	#ifndef smp_mb
@@ -80,10 +72,6 @@ · 8 unmodified lines
80	72	#define smp_wmb() __smp_wmb()
81	73	#endif
82	74
83		−#ifndef smp_read_barrier_depends
84		−#define smp_read_barrier_depends() __smp_read_barrier_depends()
85		−#endif
86		−
87	75	#else /* !CONFIG_SMP */
88	76
89	77	#ifndef smp_mb
@@ -98,10 +86,6 @@ · 8 unmodified lines
98	86	#define smp_wmb() barrier()
99	87	#endif
100	88
101		−#ifndef smp_read_barrier_depends
102		−#define smp_read_barrier_depends() do { } while (0)
103		−#endif
104		−
105	89	#endif /* CONFIG_SMP */
106	90
107	91	#ifndef __smp_store_mb
@@ -196,7 +180,6 @@ do { \ · 88 unmodified lines
196	180	#define virt_mb() __smp_mb()
197	181	#define virt_rmb() __smp_rmb()
198	182	#define virt_wmb() __smp_wmb()
199		−#define virt_read_barrier_depends() __smp_read_barrier_depends()
200	183	#define virt_store_mb(var, value) __smp_store_mb(var, value)
201	184	#define virt_mb__before_atomic() __smp_mb__before_atomic()
202	185	#define virt_mb__after_atomic() __smp_mb__after_atomic()

include/asm-generic/rwonce.h+90−0

@@ -0,0 +1,90 @@
	1	+/* SPDX-License-Identifier: GPL-2.0 */
	2	+/*
	3	+ * Prevent the compiler from merging or refetching reads or writes. The
	4	+ * compiler is also forbidden from reordering successive instances of
	5	+ * READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
	6	+ * particular ordering. One way to make the compiler aware of ordering is to
	7	+ * put the two invocations of READ_ONCE or WRITE_ONCE in different C
	8	+ * statements.
	9	+ *
	10	+ * These two macros will also work on aggregate data types like structs or
	11	+ * unions.
	12	+ *
	13	+ * Their two major use cases are: (1) Mediating communication between
	14	+ * process-level code and irq/NMI handlers, all running on the same CPU,
	15	+ * and (2) Ensuring that the compiler does not fold, spindle, or otherwise
	16	+ * mutilate accesses that either do not require ordering or that interact
	17	+ * with an explicit memory barrier or atomic instruction that provides the
	18	+ * required ordering.
	19	+ */
	20	+#ifndef __ASM_GENERIC_RWONCE_H
	21	+#define __ASM_GENERIC_RWONCE_H
	22	+
	23	+#ifndef __ASSEMBLY__
	24	+
	25	+#include <linux/compiler_types.h>
	26	+#include <linux/kasan-checks.h>
	27	+#include <linux/kcsan-checks.h>
	28	+
	29	+/*
	30	+ * Yes, this permits 64-bit accesses on 32-bit architectures. These will
	31	+ * actually be atomic in some cases (namely Armv7 + LPAE), but for others we
	32	+ * rely on the access being split into 2x32-bit accesses for a 32-bit quantity
	33	+ * (e.g. a virtual address) and a strong prevailing wind.
	34	+ */
	35	+#define compiletime_assert_rwonce_type(t) \
	36	+ compiletime_assert(__native_word(t) \|\| sizeof(t) == sizeof(long long), \
	37	+ "Unsupported access size for {READ,WRITE}_ONCE().")
	38	+
	39	+/*
	40	+ * Use __READ_ONCE() instead of READ_ONCE() if you do not require any
	41	+ * atomicity. Note that this may result in tears!
	42	+ */
	43	+#ifndef __READ_ONCE
	44	+#define __READ_ONCE(x) ((const volatile __unqual_scalar_typeof(x) )&(x))
	45	+#endif
	46	+
	47	+#define READ_ONCE(x) \
	48	+({ \
	49	+ compiletime_assert_rwonce_type(x); \
	50	+ __READ_ONCE(x); \
	51	+})
	52	+
	53	+#define __WRITE_ONCE(x, val) \
	54	+do { \
	55	+ (volatile typeof(x) )&(x) = (val); \
	56	+} while (0)
	57	+
	58	+#define WRITE_ONCE(x, val) \
	59	+do { \
	60	+ compiletime_assert_rwonce_type(x); \
	61	+ __WRITE_ONCE(x, val); \
	62	+} while (0)
	63	+
	64	+static __no_sanitize_or_inline
	65	+unsigned long __read_once_word_nocheck(const void *addr)
	66	+{
	67	+ return __READ_ONCE((unsigned long )addr);
	68	+}
	69	+
	70	+/*
	71	+ * Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need to load a
	72	+ * word from memory atomically but without telling KASAN/KCSAN. This is
	73	+ * usually used by unwinding code when walking the stack of a running process.
	74	+ */
	75	+#define READ_ONCE_NOCHECK(x) \
	76	+({ \
	77	+ compiletime_assert(sizeof(x) == sizeof(unsigned long), \
	78	+ "Unsupported access size for READ_ONCE_NOCHECK()."); \
	79	+ (typeof(x))__read_once_word_nocheck(&(x)); \
	80	+})
	81	+
	82	+static __no_kasan_or_inline
	83	+unsigned long read_word_at_a_time(const void *addr)
	84	+{
	85	+ kasan_check_read(addr, 1);
	86	+ return (unsigned long )addr;
	87	+}
	88	+
	89	+#endif /* __ASSEMBLY__ */
	90	+#endif /* __ASM_GENERIC_RWONCE_H */

include/asm-generic/tlb.h+40−15

@@ -512,6 +512,38 @@ static inline void tlb_end_vma(struct mmu_gather tlb, struct vm_area_struct vm · 511 unmodified lines
512	512	}
513	513	#endif
514	514
	515	+/*
	516	+ * tlb_flush_{pte\|pmd\|pud\|p4d}_range() adjust the tlb->start and tlb->end,
	517	+ * and set corresponding cleared_*.
	518	+ */
	519	+static inline void tlb_flush_pte_range(struct mmu_gather *tlb,
	520	+ unsigned long address, unsigned long size)
	521	+{
	522	+ __tlb_adjust_range(tlb, address, size);
	523	+ tlb->cleared_ptes = 1;
	524	+}
	525	+
	526	+static inline void tlb_flush_pmd_range(struct mmu_gather *tlb,
	527	+ unsigned long address, unsigned long size)
	528	+{
	529	+ __tlb_adjust_range(tlb, address, size);
	530	+ tlb->cleared_pmds = 1;
	531	+}
	532	+
	533	+static inline void tlb_flush_pud_range(struct mmu_gather *tlb,
	534	+ unsigned long address, unsigned long size)
	535	+{
	536	+ __tlb_adjust_range(tlb, address, size);
	537	+ tlb->cleared_puds = 1;
	538	+}
	539	+
	540	+static inline void tlb_flush_p4d_range(struct mmu_gather *tlb,
	541	+ unsigned long address, unsigned long size)
	542	+{
	543	+ __tlb_adjust_range(tlb, address, size);
	544	+ tlb->cleared_p4ds = 1;
	545	+}
	546	+
515	547	#ifndef __tlb_remove_tlb_entry
516	548	#define __tlb_remove_tlb_entry(tlb, ptep, address) do { } while (0)
517	549	#endif
@@ -525,19 +557,17 @@ static inline void tlb_end_vma(struct mmu_gather tlb, struct vm_area_struct vm · 7 unmodified lines
525	557	*/
526	558	#define tlb_remove_tlb_entry(tlb, ptep, address) \
527	559	do { \
528		− __tlb_adjust_range(tlb, address, PAGE_SIZE); \
529		− tlb->cleared_ptes = 1; \
	560	+ tlb_flush_pte_range(tlb, address, PAGE_SIZE); \
530	561	__tlb_remove_tlb_entry(tlb, ptep, address); \
531	562	} while (0)
532	563
533	564	#define tlb_remove_huge_tlb_entry(h, tlb, ptep, address) \
534	565	do { \
535	566	unsigned long _sz = huge_page_size(h); \
536		− __tlb_adjust_range(tlb, address, _sz); \
537	567	if (_sz == PMD_SIZE) \
538		− tlb->cleared_pmds = 1; \
	568	+ tlb_flush_pmd_range(tlb, address, _sz); \
539	569	else if (_sz == PUD_SIZE) \
540		− tlb->cleared_puds = 1; \
	570	+ tlb_flush_pud_range(tlb, address, _sz); \
541	571	__tlb_remove_tlb_entry(tlb, ptep, address); \
542	572	} while (0)
543	573
@@ -551,8 +581,7 @@ static inline void tlb_end_vma(struct mmu_gather tlb, struct vm_area_struct vm · 7 unmodified lines
551	581
552	582	#define tlb_remove_pmd_tlb_entry(tlb, pmdp, address) \
553	583	do { \
554		− __tlb_adjust_range(tlb, address, HPAGE_PMD_SIZE); \
555		− tlb->cleared_pmds = 1; \
	584	+ tlb_flush_pmd_range(tlb, address, HPAGE_PMD_SIZE); \
556	585	__tlb_remove_pmd_tlb_entry(tlb, pmdp, address); \
557	586	} while (0)
558	587
@@ -566,8 +595,7 @@ static inline void tlb_end_vma(struct mmu_gather tlb, struct vm_area_struct vm · 7 unmodified lines
566	595
567	596	#define tlb_remove_pud_tlb_entry(tlb, pudp, address) \
568	597	do { \
569		− __tlb_adjust_range(tlb, address, HPAGE_PUD_SIZE); \
570		− tlb->cleared_puds = 1; \
	598	+ tlb_flush_pud_range(tlb, address, HPAGE_PUD_SIZE); \
571	599	__tlb_remove_pud_tlb_entry(tlb, pudp, address); \
572	600	} while (0)
573	601
@@ -592,9 +620,8 @@ static inline void tlb_end_vma(struct mmu_gather tlb, struct vm_area_struct vm · 18 unmodified lines
592	620	#ifndef pte_free_tlb
593	621	#define pte_free_tlb(tlb, ptep, address) \
594	622	do { \
595		− __tlb_adjust_range(tlb, address, PAGE_SIZE); \
	623	+ tlb_flush_pmd_range(tlb, address, PAGE_SIZE); \
596	624	tlb->freed_tables = 1; \
597		− tlb->cleared_pmds = 1; \
598	625	__pte_free_tlb(tlb, ptep, address); \
599	626	} while (0)
600	627	#endif
@@ -602,9 +629,8 @@ static inline void tlb_end_vma(struct mmu_gather tlb, struct vm_area_struct vm · 1 unmodified line
602	629	#ifndef pmd_free_tlb
603	630	#define pmd_free_tlb(tlb, pmdp, address) \
604	631	do { \
605		− __tlb_adjust_range(tlb, address, PAGE_SIZE); \
	632	+ tlb_flush_pud_range(tlb, address, PAGE_SIZE); \
606	633	tlb->freed_tables = 1; \
607		− tlb->cleared_puds = 1; \
608	634	__pmd_free_tlb(tlb, pmdp, address); \
609	635	} while (0)
610	636	#endif
@@ -612,9 +638,8 @@ static inline void tlb_end_vma(struct mmu_gather tlb, struct vm_area_struct vm · 1 unmodified line
612	638	#ifndef pud_free_tlb
613	639	#define pud_free_tlb(tlb, pudp, address) \
614	640	do { \
615		− __tlb_adjust_range(tlb, address, PAGE_SIZE); \
	641	+ tlb_flush_p4d_range(tlb, address, PAGE_SIZE); \
616	642	tlb->freed_tables = 1; \
617		− tlb->cleared_p4ds = 1; \
618	643	__pud_free_tlb(tlb, pudp, address); \
619	644	} while (0)
620	645	#endif

include/linux/acpi.h+7−0

@@ -905,6 +905,13 @@ static inline int acpi_dma_configure(struct device *dev, · 904 unmodified lines
905	905	return 0;
906	906	}
907	907
	908	+static inline int acpi_dma_configure_id(struct device *dev,
	909	+ enum dev_dma_attr attr,
	910	+ const u32 *input_id)
	911	+{
	912	+ return 0;
	913	+}
	914	+
908	915	#define ACPI_PTR(_ptr) (NULL)
909	916
910	917	static inline void acpi_device_set_enumerated(struct acpi_device *adev)

include/linux/acpi_iort.h+11−9

@@ -28,27 +28,29 @@ void iort_deregister_domain_token(int trans_id); · 27 unmodified lines
28	28	struct fwnode_handle *iort_find_domain_token(int trans_id);
29	29	#ifdef CONFIG_ACPI_IORT
30	30	void acpi_iort_init(void);
31		−u32 iort_msi_map_rid(struct device *dev, u32 req_id);
32		−struct irq_domain iort_get_device_domain(struct device dev, u32 req_id);
	31	+u32 iort_msi_map_id(struct device *dev, u32 id);
	32	+struct irq_domain iort_get_device_domain(struct device dev, u32 id,
	33	+ enum irq_domain_bus_token bus_token);
33	34	void acpi_configure_pmsi_domain(struct device *dev);
34	35	int iort_pmsi_get_dev_id(struct device dev, u32 dev_id);
35	36	/* IOMMU interface */
36	37	void iort_dma_setup(struct device dev, u64 dma_addr, u64 *size);
37		−const struct iommu_ops iort_iommu_configure(struct device dev);
	38	+const struct iommu_ops iort_iommu_configure_id(struct device dev,
	39	+ const u32 *id_in);
38	40	int iort_iommu_msi_get_resv_regions(struct device dev, struct list_head head);
39	41	#else
40	42	static inline void acpi_iort_init(void) { }
41		−static inline u32 iort_msi_map_rid(struct device *dev, u32 req_id)
42		−{ return req_id; }
43		−static inline struct irq_domain iort_get_device_domain(struct device dev,
44		− u32 req_id)
	43	+static inline u32 iort_msi_map_id(struct device *dev, u32 id)
	44	+{ return id; }
	45	+static inline struct irq_domain *iort_get_device_domain(
	46	+ struct device *dev, u32 id, enum irq_domain_bus_token bus_token)
45	47	{ return NULL; }
46	48	static inline void acpi_configure_pmsi_domain(struct device *dev) { }
47	49	/* IOMMU interface */
48	50	static inline void iort_dma_setup(struct device dev, u64 dma_addr,
49	51	u64 *size) { }
50		−static inline const struct iommu_ops *iort_iommu_configure(
51		− struct device *dev)
	52	+static inline const struct iommu_ops *iort_iommu_configure_id(
	53	+ struct device dev, const u32 id_in)
52	54	{ return NULL; }
53	55	static inline
54	56	int iort_iommu_msi_get_resv_regions(struct device dev, struct list_head head)

include/linux/arm-smccc.h+22−22

@@ -81,6 +81,28 @@ · 80 unmodified lines
81	81	ARM_SMCCC_SMC_32, \
82	82	0, 0x7fff)
83	83
	84	+/* Paravirtualised time calls (defined by ARM DEN0057A) */
	85	+#define ARM_SMCCC_HV_PV_TIME_FEATURES \
	86	+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
	87	+ ARM_SMCCC_SMC_64, \
	88	+ ARM_SMCCC_OWNER_STANDARD_HYP, \
	89	+ 0x20)
	90	+
	91	+#define ARM_SMCCC_HV_PV_TIME_ST \
	92	+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
	93	+ ARM_SMCCC_SMC_64, \
	94	+ ARM_SMCCC_OWNER_STANDARD_HYP, \
	95	+ 0x21)
	96	+
	97	+/*
	98	+ * Return codes defined in ARM DEN 0070A
	99	+ * ARM DEN 0070A is now merged/consolidated into ARM DEN 0028 C
	100	+ */
	101	+#define SMCCC_RET_SUCCESS 0
	102	+#define SMCCC_RET_NOT_SUPPORTED -1
	103	+#define SMCCC_RET_NOT_REQUIRED -2
	104	+#define SMCCC_RET_INVALID_PARAMETER -3
	105	+
84	106	#ifndef __ASSEMBLY__
85	107
86	108	#include <linux/linkage.h>
@@ -331,15 +353,6 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, · 244 unmodified lines
331	353	*/
332	354	#define arm_smccc_1_1_hvc(...) __arm_smccc_1_1(SMCCC_HVC_INST, __VA_ARGS__)
333	355
334		−/*
335		− * Return codes defined in ARM DEN 0070A
336		− * ARM DEN 0070A is now merged/consolidated into ARM DEN 0028 C
337		− */
338		−#define SMCCC_RET_SUCCESS 0
339		−#define SMCCC_RET_NOT_SUPPORTED -1
340		−#define SMCCC_RET_NOT_REQUIRED -2
341		−#define SMCCC_RET_INVALID_PARAMETER -3
342		−
343	356	/*
344	357	* Like arm_smccc_1_1* but always returns SMCCC_RET_NOT_SUPPORTED.
345	358	* Used when the SMCCC conduit is not defined. The empty asm statement
@@ -385,18 +398,5 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, · 39 unmodified lines
385	398	method; \
386	399	})
387	400
388		−/* Paravirtualised time calls (defined by ARM DEN0057A) */
389		−#define ARM_SMCCC_HV_PV_TIME_FEATURES \
390		− ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
391		− ARM_SMCCC_SMC_64, \
392		− ARM_SMCCC_OWNER_STANDARD_HYP, \
393		− 0x20)
394		−
395		−#define ARM_SMCCC_HV_PV_TIME_ST \
396		− ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
397		− ARM_SMCCC_SMC_64, \
398		− ARM_SMCCC_OWNER_STANDARD_HYP, \
399		− 0x21)
400		−
401	401	#endif /__ASSEMBLY__/
402	402	#endif /__LINUX_ARM_SMCCC_H/

include/linux/compiler.h+2−132

@@ -230,28 +230,6 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, · 229 unmodified lines
230	230	# define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __LINE__)
231	231	#endif
232	232
233		−/*
234		− * Prevent the compiler from merging or refetching reads or writes. The
235		− * compiler is also forbidden from reordering successive instances of
236		− * READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
237		− * particular ordering. One way to make the compiler aware of ordering is to
238		− * put the two invocations of READ_ONCE or WRITE_ONCE in different C
239		− * statements.
240		− *
241		− * These two macros will also work on aggregate data types like structs or
242		− * unions.
243		− *
244		− * Their two major use cases are: (1) Mediating communication between
245		− * process-level code and irq/NMI handlers, all running on the same CPU,
246		− * and (2) Ensuring that the compiler does not fold, spindle, or otherwise
247		− * mutilate accesses that either do not require ordering or that interact
248		− * with an explicit memory barrier or atomic instruction that provides the
249		− * required ordering.
250		− */
251		−#include <asm/barrier.h>
252		−#include <linux/kasan-checks.h>
253		−#include <linux/kcsan-checks.h>
254		−
255	233	/**
256	234	* data_race - mark an expression as containing intentional data races
257	235	*
@@ -272,65 +250,6 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, · 14 unmodified lines
272	250	__v; \
273	251	})
274	252
275		−/*
276		− * Use __READ_ONCE() instead of READ_ONCE() if you do not require any
277		− * atomicity or dependency ordering guarantees. Note that this may result
278		− * in tears!
279		− */
280		−#define __READ_ONCE(x) ((const volatile __unqual_scalar_typeof(x) )&(x))
281		−
282		−#define __READ_ONCE_SCALAR(x) \
283		−({ \
284		− __unqual_scalar_typeof(x) __x = __READ_ONCE(x); \
285		− smp_read_barrier_depends(); \
286		− (typeof(x))__x; \
287		−})
288		−
289		−#define READ_ONCE(x) \
290		−({ \
291		− compiletime_assert_rwonce_type(x); \
292		− __READ_ONCE_SCALAR(x); \
293		−})
294		−
295		−#define __WRITE_ONCE(x, val) \
296		−do { \
297		− (volatile typeof(x) )&(x) = (val); \
298		−} while (0)
299		−
300		−#define WRITE_ONCE(x, val) \
301		−do { \
302		− compiletime_assert_rwonce_type(x); \
303		− __WRITE_ONCE(x, val); \
304		−} while (0)
305		−
306		−static __no_sanitize_or_inline
307		−unsigned long __read_once_word_nocheck(const void *addr)
308		−{
309		− return __READ_ONCE((unsigned long )addr);
310		−}
311		−
312		−/*
313		− * Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need to load a
314		− * word from memory atomically but without telling KASAN/KCSAN. This is
315		− * usually used by unwinding code when walking the stack of a running process.
316		− */
317		−#define READ_ONCE_NOCHECK(x) \
318		−({ \
319		− unsigned long __x; \
320		− compiletime_assert(sizeof(x) == sizeof(__x), \
321		− "Unsupported access size for READ_ONCE_NOCHECK()."); \
322		− __x = __read_once_word_nocheck(&(x)); \
323		− smp_read_barrier_depends(); \
324		− (typeof(x))__x; \
325		−})
326		−
327		−static __no_kasan_or_inline
328		−unsigned long read_word_at_a_time(const void *addr)
329		−{
330		− kasan_check_read(addr, 1);
331		− return (unsigned long )addr;
332		−}
333		−
334	253	#endif /* __KERNEL__ */
335	254
336	255	/*
@@ -354,57 +273,6 @@ static inline void offset_to_ptr(const int off) · 17 unmodified lines
354	273
355	274	#endif /* __ASSEMBLY__ */
356	275
357		−/* Compile time object size, -1 for unknown */
358		−#ifndef __compiletime_object_size
359		−# define __compiletime_object_size(obj) -1
360		−#endif
361		−#ifndef __compiletime_warning
362		−# define __compiletime_warning(message)
363		−#endif
364		−#ifndef __compiletime_error
365		−# define __compiletime_error(message)
366		−#endif
367		−
368		−#ifdef __OPTIMIZE__
369		−# define __compiletime_assert(condition, msg, prefix, suffix) \
370		− do { \
371		− extern void prefix ## suffix(void) __compiletime_error(msg); \
372		− if (!(condition)) \
373		− prefix ## suffix(); \
374		− } while (0)
375		−#else
376		−# define __compiletime_assert(condition, msg, prefix, suffix) do { } while (0)
377		−#endif
378		−
379		−#define _compiletime_assert(condition, msg, prefix, suffix) \
380		− __compiletime_assert(condition, msg, prefix, suffix)
381		−
382		−/**
383		− * compiletime_assert - break build and emit msg if condition is false
384		− * @condition: a compile-time constant condition to check
385		− * @msg: a message to emit if condition is false
386		− *
387		− * In tradition of POSIX assert, this macro will break the build if the
388		− * supplied condition is false, emitting the supplied error message if the
389		− * compiler has support to do so.
390		− */
391		−#define compiletime_assert(condition, msg) \
392		− _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
393		−
394		−#define compiletime_assert_atomic_type(t) \
395		− compiletime_assert(__native_word(t), \
396		− "Need native word sized stores/loads for atomicity.")
397		−
398		−/*
399		− * Yes, this permits 64-bit accesses on 32-bit architectures. These will
400		− * actually be atomic in some cases (namely Armv7 + LPAE), but for others we
401		− * rely on the access being split into 2x32-bit accesses for a 32-bit quantity
402		− * (e.g. a virtual address) and a strong prevailing wind.
403		− */
404		−#define compiletime_assert_rwonce_type(t) \
405		− compiletime_assert(__native_word(t) \|\| sizeof(t) == sizeof(long long), \
406		− "Unsupported access size for {READ,WRITE}_ONCE().")
407		−
408	276	/* &a[0] degrades to a pointer: a different type from an array */
409	277	#define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))
410	278
@@ -414,4 +282,6 @@ static inline void offset_to_ptr(const int off) · 3 unmodified lines
414	282	*/
415	283	#define prevent_tail_call_optimization() mb()
416	284
	285	+#include <asm/rwonce.h>
	286	+
417	287	#endif /* __LINUX_COMPILER_H */

include/linux/compiler_types.h+41−0

@@ -275,6 +275,47 @@ struct ftrace_likely_data { · 274 unmodified lines
275	275	(sizeof(t) == sizeof(char) \|\| sizeof(t) == sizeof(short) \|\| \
276	276	sizeof(t) == sizeof(int) \|\| sizeof(t) == sizeof(long))
277	277
	278	+/* Compile time object size, -1 for unknown */
	279	+#ifndef __compiletime_object_size
	280	+# define __compiletime_object_size(obj) -1
	281	+#endif
	282	+#ifndef __compiletime_warning
	283	+# define __compiletime_warning(message)
	284	+#endif
	285	+#ifndef __compiletime_error
	286	+# define __compiletime_error(message)
	287	+#endif
	288	+
	289	+#ifdef __OPTIMIZE__
	290	+# define __compiletime_assert(condition, msg, prefix, suffix) \
	291	+ do { \
	292	+ extern void prefix ## suffix(void) __compiletime_error(msg); \
	293	+ if (!(condition)) \
	294	+ prefix ## suffix(); \
	295	+ } while (0)
	296	+#else
	297	+# define __compiletime_assert(condition, msg, prefix, suffix) do { } while (0)
	298	+#endif
	299	+
	300	+#define _compiletime_assert(condition, msg, prefix, suffix) \
	301	+ __compiletime_assert(condition, msg, prefix, suffix)
	302	+
	303	+/**
	304	+ * compiletime_assert - break build and emit msg if condition is false
	305	+ * @condition: a compile-time constant condition to check
	306	+ * @msg: a message to emit if condition is false
	307	+ *
	308	+ * In tradition of POSIX assert, this macro will break the build if the
	309	+ * supplied condition is false, emitting the supplied error message if the
	310	+ * compiler has support to do so.
	311	+ */
	312	+#define compiletime_assert(condition, msg) \
	313	+ _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
	314	+
	315	+#define compiletime_assert_atomic_type(t) \
	316	+ compiletime_assert(__native_word(t), \
	317	+ "Need native word sized stores/loads for atomicity.")
	318	+
278	319	/* Helpers for emitting diagnostics in pragmas. */
279	320	#ifndef __diag
280	321	#define __diag(string)

include/linux/nospec.h+2−0

@@ -5,6 +5,8 @@ · 4 unmodified lines
5	5
6	6	#ifndef _LINUX_NOSPEC_H
7	7	#define _LINUX_NOSPEC_H
	8	+
	9	+#include <linux/compiler.h>
8	10	#include <asm/barrier.h>
9	11
10	12	struct task_struct;

include/linux/of.h+2−2

@@ -554,7 +554,7 @@ bool of_console_check(struct device_node dn, char name, int index); · 553 unmodified lines
554	554
555	555	extern int of_cpu_node_to_id(struct device_node *np);
556	556
557		−int of_map_rid(struct device_node *np, u32 rid,
	557	+int of_map_id(struct device_node *np, u32 id,
558	558	const char map_name, const char map_mask_name,
559	559	struct device_node *target, u32 id_out);
560	560
@@ -978,7 +978,7 @@ static inline int of_cpu_node_to_id(struct device_node *np) · 417 unmodified lines
978	978	return -ENODEV;
979	979	}
980	980
981		−static inline int of_map_rid(struct device_node *np, u32 rid,
	981	+static inline int of_map_id(struct device_node *np, u32 id,
982	982	const char map_name, const char map_mask_name,
983	983	struct device_node *target, u32 id_out)
984	984	{

include/linux/of_device.h+14−2

@@ -55,9 +55,15 @@ static inline struct device_node *of_cpu_device_node_get(int cpu) · 54 unmodified lines
55	55	return of_node_get(cpu_dev->of_node);
56	56	}
57	57
58		−int of_dma_configure(struct device *dev,
	58	+int of_dma_configure_id(struct device *dev,
59	59	struct device_node *np,
60		− bool force_dma);
	60	+ bool force_dma, const u32 *id);
	61	+static inline int of_dma_configure(struct device *dev,
	62	+ struct device_node *np,
	63	+ bool force_dma)
	64	+{
	65	+ return of_dma_configure_id(dev, np, force_dma, NULL);
	66	+}
61	67	#else /* CONFIG_OF */
62	68
63	69	static inline int of_driver_match_device(struct device *dev,
@@ -106,6 +112,12 @@ static inline struct device_node *of_cpu_device_node_get(int cpu) · 42 unmodified lines
106	112	return NULL;
107	113	}
108	114
	115	+static inline int of_dma_configure_id(struct device *dev,
	116	+ struct device_node *np,
	117	+ bool force_dma)
	118	+{
	119	+ return 0;
	120	+}
109	121	static inline int of_dma_configure(struct device *dev,
110	122	struct device_node *np,
111	123	bool force_dma)

include/linux/of_iommu.h+4−2

@@ -13,7 +13,8 @@ extern int of_get_dma_window(struct device_node dn, const char prefix, · 12 unmodified lines
13	13	size_t *size);
14	14
15	15	extern const struct iommu_ops of_iommu_configure(struct device dev,
16		− struct device_node *master_np);
	16	+ struct device_node *master_np,
	17	+ const u32 *id);
17	18
18	19	#else
19	20
@@ -25,7 +26,8 @@ static inline int of_get_dma_window(struct device_node dn, const char prefix, · 5 unmodified lines
25	26	}
26	27
27	28	static inline const struct iommu_ops of_iommu_configure(struct device dev,
28		− struct device_node *master_np)
	29	+ struct device_node *master_np,
	30	+ const u32 *id)
29	31	{
30	32	return NULL;
31	33	}

include/linux/of_irq.h+7−6

@@ -52,9 +52,10 @@ extern struct irq_domain of_msi_get_domain(struct device dev, · 51 unmodified lines
52	52	struct device_node *np,
53	53	enum irq_domain_bus_token token);
54	54	extern struct irq_domain of_msi_map_get_device_domain(struct device dev,
55		− u32 rid);
	55	+ u32 id,
	56	+ u32 bus_token);
56	57	extern void of_msi_configure(struct device dev, struct device_node np);
57		−u32 of_msi_map_rid(struct device dev, struct device_node msi_np, u32 rid_in);
	58	+u32 of_msi_map_id(struct device dev, struct device_node msi_np, u32 id_in);
58	59	#else
59	60	static inline int of_irq_count(struct device_node *dev)
60	61	{
@@ -85,17 +86,17 @@ static inline struct irq_domain of_msi_get_domain(struct device dev, · 24 unmodified lines
85	86	return NULL;
86	87	}
87	88	static inline struct irq_domain of_msi_map_get_device_domain(struct device dev,
88		− u32 rid)
	89	+ u32 id, u32 bus_token)
89	90	{
90	91	return NULL;
91	92	}
92	93	static inline void of_msi_configure(struct device dev, struct device_node np)
93	94	{
94	95	}
95		−static inline u32 of_msi_map_rid(struct device *dev,
96		− struct device_node *msi_np, u32 rid_in)
	96	+static inline u32 of_msi_map_id(struct device *dev,
	97	+ struct device_node *msi_np, u32 id_in)
97	98	{
98		− return rid_in;
	99	+ return id_in;
99	100	}
100	101	#endif
101	102

include/linux/percpu-refcount.h+1−1

@@ -155,7 +155,7 @@ static inline bool __ref_is_percpu(struct percpu_ref *ref, · 154 unmodified lines
155	155	* between contaminating the pointer value, meaning that
156	156	* READ_ONCE() is required when fetching it.
157	157	*
158		− * The smp_read_barrier_depends() implied by READ_ONCE() pairs
	158	+ * The dependency ordering from the READ_ONCE() pairs
159	159	* with smp_store_release() in __percpu_ref_switch_to_percpu().
160	160	*/
161	161	percpu_ptr = READ_ONCE(ref->percpu_count_ptr);

include/linux/ptr_ring.h+1−1

@@ -107,7 +107,7 @@ static inline int __ptr_ring_produce(struct ptr_ring r, void ptr) · 106 unmodified lines
107	107	return -ENOSPC;
108	108
109	109	/* Make sure the pointer we are storing points to a valid data. */
110		− /* Pairs with smp_read_barrier_depends in __ptr_ring_consume. */
	110	+ /* Pairs with the dependency ordering in __ptr_ring_consume. */
111	111	smp_wmb();
112	112
113	113	WRITE_ONCE(r->queue[r->producer++], ptr);

include/linux/sched_clock.h+28−0

@@ -6,6 +6,34 @@ · 5 unmodified lines
6	6	#define LINUX_SCHED_CLOCK
7	7
8	8	#ifdef CONFIG_GENERIC_SCHED_CLOCK
	9	+/**
	10	+ * struct clock_read_data - data required to read from sched_clock()
	11	+ *
	12	+ * @epoch_ns: sched_clock() value at last update
	13	+ * @epoch_cyc: Clock cycle value at last update.
	14	+ * @sched_clock_mask: Bitmask for two's complement subtraction of non 64bit
	15	+ * clocks.
	16	+ * @read_sched_clock: Current clock source (or dummy source when suspended).
	17	+ * @mult: Multipler for scaled math conversion.
	18	+ * @shift: Shift value for scaled math conversion.
	19	+ *
	20	+ * Care must be taken when updating this structure; it is read by
	21	+ * some very hot code paths. It occupies <=40 bytes and, when combined
	22	+ * with the seqcount used to synchronize access, comfortably fits into
	23	+ * a 64 byte cache line.
	24	+ */
	25	+struct clock_read_data {
	26	+ u64 epoch_ns;
	27	+ u64 epoch_cyc;
	28	+ u64 sched_clock_mask;
	29	+ u64 (*read_sched_clock)(void);
	30	+ u32 mult;
	31	+ u32 shift;
	32	+};
	33	+
	34	+extern struct clock_read_data sched_clock_read_begin(unsigned int seq);
	35	+extern int sched_clock_read_retry(unsigned int seq);
	36	+
9	37	extern void generic_sched_clock_init(void);
10	38
11	39	extern void sched_clock_register(u64 (*read)(void), int bits,

include/uapi/linux/perf_event.h+20−3

@@ -532,9 +532,10 @@ struct perf_event_mmap_page { · 531 unmodified lines
532	532	cap_bit0_is_deprecated : 1, /* Always 1, signals that bit 0 is zero */
533	533
534	534	cap_user_rdpmc : 1, /* The RDPMC instruction can be used to read counts */
535		− cap_user_time : 1, /* The time_* fields are used */
	535	+ cap_user_time : 1, /* The time_{shift,mult,offset} fields are used */
536	536	cap_user_time_zero : 1, /* The time_zero field is used */
537		− cap_____res : 59;
	537	+ cap_user_time_short : 1, /* the time_{cycle,mask} fields are used */
	538	+ cap_____res : 58;
538	539	};
539	540	};
540	541
@@ -593,13 +594,29 @@ struct perf_event_mmap_page { · 52 unmodified lines
593	594	* ((rem * time_mult) >> time_shift);
594	595	*/
595	596	__u64 time_zero;
	597	+
596	598	__u32 size; /* Header size up to __reserved[] fields. */
	599	+ __u32 __reserved_1;
	600	+
	601	+ /*
	602	+ * If cap_usr_time_short, the hardware clock is less than 64bit wide
	603	+ * and we must compute the 'cyc' value, as used by cap_usr_time, as:
	604	+ *
	605	+ * cyc = time_cycles + ((cyc - time_cycles) & time_mask)
	606	+ *
	607	+ * NOTE: this form is explicitly chosen such that cap_usr_time_short
	608	+ * is a correction on top of cap_usr_time, and code that doesn't
	609	+ * know about cap_usr_time_short still works under the assumption
	610	+ * the counter doesn't wrap.
	611	+ */
	612	+ __u64 time_cycles;
	613	+ __u64 time_mask;
597	614
598	615	/*
599	616	* Hole for extension of the self monitor capabilities
600	617	*/
601	618
602		− __u8 __reserved[1188+4]; / align to 1k. */
	619	+ __u8 __reserved[1168]; / align to 1k. */
603	620
604	621	/*
605	622	* Control data for the mmap() data buffer.

include/vdso/datapage.h+1−0

@@ -109,6 +109,7 @@ struct vdso_data { · 108 unmodified lines
109	109	* relocation, and this is what we need.
110	110	*/
111	111	extern struct vdso_data _vdso_data[CS_BASES] __attribute__((visibility("hidden")));
	112	+extern struct vdso_data _timens_data[CS_BASES] __attribute__((visibility("hidden")));
112	113
113	114	/*
114	115	* The generic vDSO implementation requires that gettimeofday.h

kernel/crash_core.c+1−0

@@ -413,6 +413,7 @@ static int __init crash_save_vmcoreinfo_init(void) · 412 unmodified lines
413	413	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
414	414	VMCOREINFO_STRUCT_SIZE(mem_section);
415	415	VMCOREINFO_OFFSET(mem_section, section_mem_map);
	416	+ VMCOREINFO_NUMBER(MAX_PHYSMEM_BITS);
416	417	#endif
417	418	VMCOREINFO_STRUCT_SIZE(page);
418	419	VMCOREINFO_STRUCT_SIZE(pglist_data);

kernel/time/sched_clock.c+13−28

@@ -19,31 +19,6 @@ · 18 unmodified lines
19	19
20	20	#include "timekeeping.h"
21	21
22		−/**
23		− * struct clock_read_data - data required to read from sched_clock()
24		− *
25		− * @epoch_ns: sched_clock() value at last update
26		− * @epoch_cyc: Clock cycle value at last update.
27		− * @sched_clock_mask: Bitmask for two's complement subtraction of non 64bit
28		− * clocks.
29		− * @read_sched_clock: Current clock source (or dummy source when suspended).
30		− * @mult: Multipler for scaled math conversion.
31		− * @shift: Shift value for scaled math conversion.
32		− *
33		− * Care must be taken when updating this structure; it is read by
34		− * some very hot code paths. It occupies <=40 bytes and, when combined
35		− * with the seqcount used to synchronize access, comfortably fits into
36		− * a 64 byte cache line.
37		− */
38		−struct clock_read_data {
39		− u64 epoch_ns;
40		− u64 epoch_cyc;
41		− u64 sched_clock_mask;
42		− u64 (*read_sched_clock)(void);
43		− u32 mult;
44		− u32 shift;
45		−};
46		−
47	22	/**
48	23	* struct clock_data - all data needed for sched_clock() (including
49	24	* registration of a new clock source)
@@ -93,6 +68,17 @@ static inline u64 notrace cyc_to_ns(u64 cyc, u32 mult, u32 shift) · 43 unmodified lines
93	68	return (cyc * mult) >> shift;
94	69	}
95	70
	71	+struct clock_read_data sched_clock_read_begin(unsigned int seq)
	72	+{
	73	+ *seq = raw_read_seqcount_latch(&cd.seq);
	74	+ return cd.read_data + (*seq & 1);
	75	+}
	76	+
	77	+int sched_clock_read_retry(unsigned int seq)
	78	+{
	79	+ return read_seqcount_retry(&cd.seq, seq);
	80	+}
	81	+
96	82	unsigned long long notrace sched_clock(void)
97	83	{
98	84	u64 cyc, res;
@@ -100,13 +86,12 @@ unsigned long long notrace sched_clock(void) · 1 unmodified line
100	86	struct clock_read_data *rd;
101	87
102	88	do {
103		− seq = raw_read_seqcount(&cd.seq);
104		− rd = cd.read_data + (seq & 1);
	89	+ rd = sched_clock_read_begin(&seq);
105	90
106	91	cyc = (rd->read_sched_clock() - rd->epoch_cyc) &
107	92	rd->sched_clock_mask;
108	93	res = rd->epoch_ns + cyc_to_ns(cyc, rd->mult, rd->shift);
109		− } while (read_seqcount_retry(&cd.seq, seq));
	94	+ } while (sched_clock_read_retry(seq));
110	95
111	96	return res;
112	97	}

mm/memory.c+1−1

@@ -437,7 +437,7 @@ int __pte_alloc(struct mm_struct mm, pmd_t pmd) · 436 unmodified lines
437	437	* of a chain of data-dependent loads, meaning most CPUs (alpha
438	438	* being the notable exception) will already guarantee loads are
439	439	* seen in-order. See the alpha page table accessors for the
440		− * smp_read_barrier_depends() barriers in page table walking code.
	440	+ * smp_rmb() barriers in page table walking code.
441	441	*/
442	442	smp_wmb(); /* Could be smp_wmb__xxx(before\|after)_spin_lock */
443	443

scripts/checkpatch.pl+1−8

@@ -5903,8 +5903,7 @@ sub process { · 5902 unmodified lines
5903	5903	my $barriers = qr{
5904	5904	mb\|
5905	5905	rmb\|
5906		− wmb\|
5907		− read_barrier_depends
	5906	+ wmb
5908	5907	}x;
5909	5908	my $barrier_stems = qr{
5910	5909	mb__before_atomic\|
@@ -5953,12 +5952,6 @@ sub process { · 42 unmodified lines
5953	5952	}
5954	5953	}
5955	5954
5956		−# check for smp_read_barrier_depends and read_barrier_depends
5957		− if (!$file && $line =~ /\b(smp_\|)read_barrier_depends\s*\(/) {
5958		− WARN("READ_BARRIER_DEPENDS",
5959		− "$1read_barrier_depends should only be used in READ_ONCE or DEC Alpha code\n" . $herecurr);
5960		− }
5961		−
5962	5955	# check of hardware specific defines
5963	5956	if ($line =~ m@^.\s\#\sif.*\b(__i386__\|__powerpc64__\|__sun__\|__s390x__)\b@ && $realfile !~ m@include/asm-@) {
5964	5957	CHK("ARCH_DEFINES",

scripts/recordmcount.c+6−0

@@ -434,6 +434,11 @@ static int arm_is_fake_mcount(Elf32_Rel const *rp) · 433 unmodified lines
434	434	return 1;
435	435	}
436	436
	437	+static int arm64_is_fake_mcount(Elf64_Rel const *rp)
	438	+{
	439	+ return ELF64_R_TYPE(w(rp->r_info)) != R_AARCH64_CALL26;
	440	+}
	441	+
437	442	/* 64-bit EM_MIPS has weird ELF64_Rela.r_info.
438	443	* http://techpubs.sgi.com/library/manuals/4000/007-4658-001/pdf/007-4658-001.pdf
439	444	* We interpret Table 29 Relocation Operation (Elf64_Rel, Elf64_Rela) [p.40]
@@ -547,6 +552,7 @@ static int do_file(char const *const fname) · 107 unmodified lines
547	552	make_nop = make_nop_arm64;
548	553	rel_type_nop = R_AARCH64_NONE;
549	554	ideal_nop = ideal_nop4_arm64;
	555	+ is_fake_mcount64 = arm64_is_fake_mcount;
550	556	break;
551	557	case EM_IA_64: reltype = R_IA64_IMM64; break;
552	558	case EM_MIPS: /* reltype: e_class */ break;

tools/bpf/Makefile+2−1

@@ -9,7 +9,8 @@ MAKE = make · 8 unmodified lines
9	9	INSTALL ?= install
10	10
11	11	CFLAGS += -Wall -O2
12		−CFLAGS += -D__EXPORTED_HEADERS__ -I$(srctree)/include/uapi -I$(srctree)/include
	12	+CFLAGS += -D__EXPORTED_HEADERS__ -I$(srctree)/tools/include/uapi \
	13	+ -I$(srctree)/tools/include
13	14
14	15	# This will work when bpf is built in tools env. where srctree
15	16	# isn't set and when invoked from selftests build, where srctree

tools/include/uapi/linux/filter.h+90−0

@@ -0,0 +1,90 @@
	1	+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
	2	+/*
	3	+ * Linux Socket Filter Data Structures
	4	+ */
	5	+
	6	+#ifndef __LINUX_FILTER_H__
	7	+#define __LINUX_FILTER_H__
	8	+
	9	+
	10	+#include <linux/types.h>
	11	+#include <linux/bpf_common.h>
	12	+
	13	+/*
	14	+ * Current version of the filter code architecture.
	15	+ */
	16	+#define BPF_MAJOR_VERSION 1
	17	+#define BPF_MINOR_VERSION 1
	18	+
	19	+/*
	20	+ * Try and keep these values and structures similar to BSD, especially
	21	+ * the BPF code definitions which need to match so you can share filters
	22	+ */
	23	+
	24	+struct sock_filter { /* Filter block */
	25	+ __u16 code; /* Actual filter code */
	26	+ __u8 jt; /* Jump true */
	27	+ __u8 jf; /* Jump false */
	28	+ __u32 k; /* Generic multiuse field */
	29	+};
	30	+
	31	+struct sock_fprog { /* Required for SO_ATTACH_FILTER. */
	32	+ unsigned short len; /* Number of filter blocks */
	33	+ struct sock_filter *filter;
	34	+};
	35	+
	36	+/* ret - BPF_K and BPF_X also apply */
	37	+#define BPF_RVAL(code) ((code) & 0x18)
	38	+#define BPF_A 0x10
	39	+
	40	+/* misc */
	41	+#define BPF_MISCOP(code) ((code) & 0xf8)
	42	+#define BPF_TAX 0x00
	43	+#define BPF_TXA 0x80
	44	+
	45	+/*
	46	+ * Macros for filter block array initializers.
	47	+ */
	48	+#ifndef BPF_STMT
	49	+#define BPF_STMT(code, k) { (unsigned short)(code), 0, 0, k }
	50	+#endif
	51	+#ifndef BPF_JUMP
	52	+#define BPF_JUMP(code, k, jt, jf) { (unsigned short)(code), jt, jf, k }
	53	+#endif
	54	+
	55	+/*
	56	+ * Number of scratch memory words for: BPF_ST and BPF_STX
	57	+ */
	58	+#define BPF_MEMWORDS 16
	59	+
	60	+/* RATIONALE. Negative offsets are invalid in BPF.
	61	+ We use them to reference ancillary data.
	62	+ Unlike introduction new instructions, it does not break
	63	+ existing compilers/optimizers.
	64	+ */
	65	+#define SKF_AD_OFF (-0x1000)
	66	+#define SKF_AD_PROTOCOL 0
	67	+#define SKF_AD_PKTTYPE 4
	68	+#define SKF_AD_IFINDEX 8
	69	+#define SKF_AD_NLATTR 12
	70	+#define SKF_AD_NLATTR_NEST 16
	71	+#define SKF_AD_MARK 20
	72	+#define SKF_AD_QUEUE 24
	73	+#define SKF_AD_HATYPE 28
	74	+#define SKF_AD_RXHASH 32
	75	+#define SKF_AD_CPU 36
	76	+#define SKF_AD_ALU_XOR_X 40
	77	+#define SKF_AD_VLAN_TAG 44
	78	+#define SKF_AD_VLAN_TAG_PRESENT 48
	79	+#define SKF_AD_PAY_OFFSET 52
	80	+#define SKF_AD_RANDOM 56
	81	+#define SKF_AD_VLAN_TPID 60
	82	+#define SKF_AD_MAX 64
	83	+
	84	+#define SKF_NET_OFF (-0x100000)
	85	+#define SKF_LL_OFF (-0x200000)
	86	+
	87	+#define BPF_NET_OFF SKF_NET_OFF
	88	+#define BPF_LL_OFF SKF_LL_OFF
	89	+
	90	+#endif /* __LINUX_FILTER_H__ */

tools/include/uapi/linux/perf_event.h+20−3

@@ -532,9 +532,10 @@ struct perf_event_mmap_page { · 531 unmodified lines
532	532	cap_bit0_is_deprecated : 1, /* Always 1, signals that bit 0 is zero */
533	533
534	534	cap_user_rdpmc : 1, /* The RDPMC instruction can be used to read counts */
535		− cap_user_time : 1, /* The time_* fields are used */
	535	+ cap_user_time : 1, /* The time_{shift,mult,offset} fields are used */
536	536	cap_user_time_zero : 1, /* The time_zero field is used */
537		− cap_____res : 59;
	537	+ cap_user_time_short : 1, /* the time_{cycle,mask} fields are used */
	538	+ cap_____res : 58;
538	539	};
539	540	};
540	541
@@ -593,13 +594,29 @@ struct perf_event_mmap_page { · 52 unmodified lines
593	594	* ((rem * time_mult) >> time_shift);
594	595	*/
595	596	__u64 time_zero;
	597	+
596	598	__u32 size; /* Header size up to __reserved[] fields. */
	599	+ __u32 __reserved_1;
	600	+
	601	+ /*
	602	+ * If cap_usr_time_short, the hardware clock is less than 64bit wide
	603	+ * and we must compute the 'cyc' value, as used by cap_usr_time, as:
	604	+ *
	605	+ * cyc = time_cycles + ((cyc - time_cycles) & time_mask)
	606	+ *
	607	+ * NOTE: this form is explicitly chosen such that cap_usr_time_short
	608	+ * is a correction on top of cap_usr_time, and code that doesn't
	609	+ * know about cap_usr_time_short still works under the assumption
	610	+ * the counter doesn't wrap.
	611	+ */
	612	+ __u64 time_cycles;
	613	+ __u64 time_mask;
597	614
598	615	/*
599	616	* Hole for extension of the self monitor capabilities
600	617	*/
601	618
602		− __u8 __reserved[1188+4]; / align to 1k. */
	619	+ __u8 __reserved[1168]; / align to 1k. */
603	620
604	621	/*
605	622	* Control data for the mmap() data buffer.

tools/memory-model/Documentation/explanation.txt+12−14

@@ -1122,12 +1122,10 @@ maintain at least the appearance of FIFO order. · 1121 unmodified lines
1122	1122	In practice, this difficulty is solved by inserting a special fence
1123	1123	between P1's two loads when the kernel is compiled for the Alpha
1124	1124	architecture. In fact, as of version 4.15, the kernel automatically
1125		−adds this fence (called smp_read_barrier_depends() and defined as
1126		−nothing at all on non-Alpha builds) after every READ_ONCE() and atomic
1127		−load. The effect of the fence is to cause the CPU not to execute any
1128		−po-later instructions until after the local cache has finished
1129		−processing all the stores it has already received. Thus, if the code
1130		−was changed to:
	1125	+adds this fence after every READ_ONCE() and atomic load on Alpha. The
	1126	+effect of the fence is to cause the CPU not to execute any po-later
	1127	+instructions until after the local cache has finished processing all
	1128	+the stores it has already received. Thus, if the code was changed to:
1131	1129
1132	1130	P1()
1133	1131	{
@@ -1146,14 +1144,14 @@ READ_ONCE() or another synchronization primitive rather than accessed · 12 unmodified lines
1146	1144	directly.
1147	1145
1148	1146	The LKMM requires that smp_rmb(), acquire fences, and strong fences
1149		−share this property with smp_read_barrier_depends(): They do not allow
1150		−the CPU to execute any po-later instructions (or po-later loads in the
1151		−case of smp_rmb()) until all outstanding stores have been processed by
1152		−the local cache. In the case of a strong fence, the CPU first has to
1153		−wait for all of its po-earlier stores to propagate to every other CPU
1154		−in the system; then it has to wait for the local cache to process all
1155		−the stores received as of that time -- not just the stores received
1156		−when the strong fence began.
	1147	+share this property: They do not allow the CPU to execute any po-later
	1148	+instructions (or po-later loads in the case of smp_rmb()) until all
	1149	+outstanding stores have been processed by the local cache. In the
	1150	+case of a strong fence, the CPU first has to wait for all of its
	1151	+po-earlier stores to propagate to every other CPU in the system; then
	1152	+it has to wait for the local cache to process all the stores received
	1153	+as of that time -- not just the stores received when the strong fence
	1154	+began.
1157	1155
1158	1156	And of course, none of this matters for any architecture other than
1159	1157	Alpha.