lael-labs/linux

Documentation/RCU/Design/Requirements/Requirements.rst+1−1

@@ -463,7 +463,7 @@ again without disrupting RCU readers. · 462 unmodified lines
463	This guarantee was only partially premeditated. DYNIX/ptx used an	463	This guarantee was only partially premeditated. DYNIX/ptx used an
464	explicit memory barrier for publication, but had nothing resembling	464	explicit memory barrier for publication, but had nothing resembling
465	``rcu_dereference()`` for subscription, nor did it have anything	465	``rcu_dereference()`` for subscription, nor did it have anything
466	resembling the ``smp_read_barrier_depends()`` that was later subsumed	466	resembling the dependency-ordering barrier that was later subsumed
467	into ``rcu_dereference()`` and later still into ``READ_ONCE()``. The	467	into ``rcu_dereference()`` and later still into ``READ_ONCE()``. The
468	need for these operations made itself known quite suddenly at a	468	need for these operations made itself known quite suddenly at a
469	late-1990s meeting with the DEC Alpha architects, back in the days when	469	late-1990s meeting with the DEC Alpha architects, back in the days when

Documentation/admin-guide/kdump/vmcoreinfo.rst+16−0

@@ -93,6 +93,11 @@ It exists in the sparse memory mapping model, and it is also somewhat · 92 unmodified lines
93	similar to the mem_map variable, both of them are used to translate an	93	similar to the mem_map variable, both of them are used to translate an
94	address.	94	address.
95		95
		96	MAX_PHYSMEM_BITS
		97	----------------
		98
		99	Defines the maximum supported physical address space memory.
		100
96	page	101	page
97	----	102	----
98		103
@@ -399,6 +404,17 @@ KERNELPACMASK · 300 unmodified lines
399	The mask to extract the Pointer Authentication Code from a kernel virtual	404	The mask to extract the Pointer Authentication Code from a kernel virtual
400	address.	405	address.
401		406
		407	TCR_EL1.T1SZ
		408	------------
		409
		410	Indicates the size offset of the memory region addressed by TTBR1_EL1.
		411	The region size is 2^(64-T1SZ) bytes.
		412
		413	TTBR1_EL1 is the table base address register specified by ARMv8-A
		414	architecture which is used to lookup the page-tables for the Virtual
		415	addresses in the higher VA range (refer to ARMv8 ARM document for
		416	more details).
		417
402	arm	418	arm
403	===	419	===
404		420

Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt+44−6

@@ -28,6 +28,16 @@ Documentation/devicetree/bindings/iommu/iommu.txt. · 27 unmodified lines
28	For arm-smmu binding, see:	28	For arm-smmu binding, see:
29	Documentation/devicetree/bindings/iommu/arm,smmu.yaml.	29	Documentation/devicetree/bindings/iommu/arm,smmu.yaml.
30		30
		31	The MSI writes are accompanied by sideband data which is derived from the ICID.
		32	The msi-map property is used to associate the devices with both the ITS
		33	controller and the sideband data which accompanies the writes.
		34
		35	For generic MSI bindings, see
		36	Documentation/devicetree/bindings/interrupt-controller/msi.txt.
		37
		38	For GICv3 and GIC ITS bindings, see:
		39	Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml.
		40
31	Required properties:	41	Required properties:
32		42
33	- compatible	43	- compatible
@@ -49,11 +59,6 @@ Required properties: · 15 unmodified lines
49	region may not be present in some scenarios, such	59	region may not be present in some scenarios, such
50	as in the device tree presented to a virtual machine.	60	as in the device tree presented to a virtual machine.
51		61
52	- msi-parent
53	Value type: <phandle>
54	Definition: Must be present and point to the MSI controller node
55	handling message interrupts for the MC.
56
57	- ranges	62	- ranges
58	Value type: <prop-encoded-array>	63	Value type: <prop-encoded-array>
59	Definition: A standard property. Defines the mapping between the child	64	Definition: A standard property. Defines the mapping between the child
@@ -119,6 +124,28 @@ Optional properties: · 59 unmodified lines
119	associated with the listed IOMMU, with the iommu-specifier	124	associated with the listed IOMMU, with the iommu-specifier
120	(i - icid-base + iommu-base).	125	(i - icid-base + iommu-base).
121		126
		127	- msi-map: Maps an ICID to a GIC ITS and associated msi-specifier
		128	data.
		129
		130	The property is an arbitrary number of tuples of
		131	(icid-base,gic-its,msi-base,length).
		132
		133	Any ICID in the interval [icid-base, icid-base + length) is
		134	associated with the listed GIC ITS, with the msi-specifier
		135	(i - icid-base + msi-base).
		136
		137	Deprecated properties:
		138
		139	- msi-parent
		140	Value type: <phandle>
		141	Definition: Describes the MSI controller node handling message
		142	interrupts for the MC. When there is no translation
		143	between the ICID and deviceID this property can be used
		144	to describe the MSI controller used by the devices on the
		145	mc-bus.
		146	The use of this property for mc-bus is deprecated. Please
		147	use msi-map.
		148
122	Example:	149	Example:
123		150
124	smmu: iommu@5000000 {	151	smmu: iommu@5000000 {
@@ -128,13 +155,24 @@ Example: · 3 unmodified lines
128	...	155	...
129	};	156	};
130		157
		158	gic: interrupt-controller@6000000 {
		159	compatible = "arm,gic-v3";
		160	...
		161	}
		162	its: gic-its@6020000 {
		163	compatible = "arm,gic-v3-its";
		164	msi-controller;
		165	...
		166	};
		167
131	fsl_mc: fsl-mc@80c000000 {	168	fsl_mc: fsl-mc@80c000000 {
132	compatible = "fsl,qoriq-mc";	169	compatible = "fsl,qoriq-mc";
133	reg = <0x00000008 0x0c000000 0 0x40>, /* MC portal base */	170	reg = <0x00000008 0x0c000000 0 0x40>, /* MC portal base */
134	<0x00000000 0x08340000 0 0x40000>; /* MC control reg */	171	<0x00000000 0x08340000 0 0x40000>; /* MC control reg */
135	msi-parent = <&its>;
136	/* define map for ICIDs 23-64 */	172	/* define map for ICIDs 23-64 */
137	iommu-map = <23 &smmu 23 41>;	173	iommu-map = <23 &smmu 23 41>;
		174	/* define msi map for ICIDs 23-64 */
		175	msi-map = <23 &its 23 41>;
138	#address-cells = <3>;	176	#address-cells = <3>;
139	#size-cells = <1>;	177	#size-cells = <1>;
140		178

Documentation/memory-barriers.txt+8−148

@@ -553,12 +553,12 @@ There are certain things that the Linux kernel memory barriers do not guarantee: · 552 unmodified lines
553	DATA DEPENDENCY BARRIERS (HISTORICAL)	553	DATA DEPENDENCY BARRIERS (HISTORICAL)
554	-------------------------------------	554	-------------------------------------
555		555
556	As of v4.15 of the Linux kernel, an smp_read_barrier_depends() was	556	As of v4.15 of the Linux kernel, an smp_mb() was added to READ_ONCE() for
557	added to READ_ONCE(), which means that about the only people who	557	DEC Alpha, which means that about the only people who need to pay attention
558	need to pay attention to this section are those working on DEC Alpha	558	to this section are those working on DEC Alpha architecture-specific code
559	architecture-specific code and those working on READ_ONCE() itself.	559	and those working on READ_ONCE() itself. For those who need it, and for
560	For those who need it, and for those who are interested in the history,	560	those who are interested in the history, here is the story of
561	here is the story of data-dependency barriers.	561	data-dependency barriers.
562		562
563	The usage requirements of data dependency barriers are a little subtle, and	563	The usage requirements of data dependency barriers are a little subtle, and
564	it's not always obvious that they're needed. To illustrate, consider the	564	it's not always obvious that they're needed. To illustrate, consider the
@@ -2708,144 +2708,6 @@ the properties of the memory window through which devices are accessed and/or · 2143 unmodified lines
2708	the use of any special device communication instructions the CPU may have.	2708	the use of any special device communication instructions the CPU may have.
2709		2709
2710		2710
2711	CACHE COHERENCY
2712	---------------
2713
2714	Life isn't quite as simple as it may appear above, however: for while the
2715	caches are expected to be coherent, there's no guarantee that that coherency
2716	will be ordered. This means that while changes made on one CPU will
2717	eventually become visible on all CPUs, there's no guarantee that they will
2718	become apparent in the same order on those other CPUs.
2719
2720
2721	Consider dealing with a system that has a pair of CPUs (1 & 2), each of which
2722	has a pair of parallel data caches (CPU 1 has A/B, and CPU 2 has C/D):
2723
2724	:
2725	: +--------+
2726	: +---------+ \| \|
2727	+--------+ : +--->\| Cache A \|<------->\| \|
2728	\| \| : \| +---------+ \| \|
2729	\| CPU 1 \|<---+ \| \|
2730	\| \| : \| +---------+ \| \|
2731	+--------+ : +--->\| Cache B \|<------->\| \|
2732	: +---------+ \| \|
2733	: \| Memory \|
2734	: +---------+ \| System \|
2735	+--------+ : +--->\| Cache C \|<------->\| \|
2736	\| \| : \| +---------+ \| \|
2737	\| CPU 2 \|<---+ \| \|
2738	\| \| : \| +---------+ \| \|
2739	+--------+ : +--->\| Cache D \|<------->\| \|
2740	: +---------+ \| \|
2741	: +--------+
2742	:
2743
2744	Imagine the system has the following properties:
2745
2746	(*) an odd-numbered cache line may be in cache A, cache C or it may still be
2747	resident in memory;
2748
2749	(*) an even-numbered cache line may be in cache B, cache D or it may still be
2750	resident in memory;
2751
2752	(*) while the CPU core is interrogating one cache, the other cache may be
2753	making use of the bus to access the rest of the system - perhaps to
2754	displace a dirty cacheline or to do a speculative load;
2755
2756	(*) each cache has a queue of operations that need to be applied to that cache
2757	to maintain coherency with the rest of the system;
2758
2759	(*) the coherency queue is not flushed by normal loads to lines already
2760	present in the cache, even though the contents of the queue may
2761	potentially affect those loads.
2762
2763	Imagine, then, that two writes are made on the first CPU, with a write barrier
2764	between them to guarantee that they will appear to reach that CPU's caches in
2765	the requisite order:
2766
2767	CPU 1 CPU 2 COMMENT
2768	=============== =============== =======================================
2769	u == 0, v == 1 and p == &u, q == &u
2770	v = 2;
2771	smp_wmb(); Make sure change to v is visible before
2772	change to p
2773	<A:modify v=2> v is now in cache A exclusively
2774	p = &v;
2775	<B:modify p=&v> p is now in cache B exclusively
2776
2777	The write memory barrier forces the other CPUs in the system to perceive that
2778	the local CPU's caches have apparently been updated in the correct order. But
2779	now imagine that the second CPU wants to read those values:
2780
2781	CPU 1 CPU 2 COMMENT
2782	=============== =============== =======================================
2783	...
2784	q = p;
2785	x = *q;
2786
2787	The above pair of reads may then fail to happen in the expected order, as the
2788	cacheline holding p may get updated in one of the second CPU's caches while
2789	the update to the cacheline holding v is delayed in the other of the second
2790	CPU's caches by some other cache event:
2791
2792	CPU 1 CPU 2 COMMENT
2793	=============== =============== =======================================
2794	u == 0, v == 1 and p == &u, q == &u
2795	v = 2;
2796	smp_wmb();
2797	<A:modify v=2> <C:busy>
2798	<C:queue v=2>
2799	p = &v; q = p;
2800	<D:request p>
2801	<B:modify p=&v> <D:commit p=&v>
2802	<D:read p>
2803	x = *q;
2804	<C:read *q> Reads from v before v updated in cache
2805	<C:unbusy>
2806	<C:commit v=2>
2807
2808	Basically, while both cachelines will be updated on CPU 2 eventually, there's
2809	no guarantee that, without intervention, the order of update will be the same
2810	as that committed on CPU 1.
2811
2812
2813	To intervene, we need to interpolate a data dependency barrier or a read
2814	barrier between the loads (which as of v4.15 is supplied unconditionally
2815	by the READ_ONCE() macro). This will force the cache to commit its
2816	coherency queue before processing any further requests:
2817
2818	CPU 1 CPU 2 COMMENT
2819	=============== =============== =======================================
2820	u == 0, v == 1 and p == &u, q == &u
2821	v = 2;
2822	smp_wmb();
2823	<A:modify v=2> <C:busy>
2824	<C:queue v=2>
2825	p = &v; q = p;
2826	<D:request p>
2827	<B:modify p=&v> <D:commit p=&v>
2828	<D:read p>
2829	smp_read_barrier_depends()
2830	<C:unbusy>
2831	<C:commit v=2>
2832	x = *q;
2833	<C:read *q> Reads from v after v updated in cache
2834
2835
2836	This sort of problem can be encountered on DEC Alpha processors as they have a
2837	split cache that improves performance by making better use of the data bus.
2838	While most CPUs do imply a data dependency barrier on the read when a memory
2839	access depends on a read, not all do, so it may not be relied on.
2840
2841	Other CPUs may also have split caches, but must coordinate between the various
2842	cachelets for normal memory accesses. The semantics of the Alpha removes the
2843	need for hardware coordination in the absence of memory barriers, which
2844	permitted Alpha to sport higher CPU clock rates back in the day. However,
2845	please note that (again, as of v4.15) smp_read_barrier_depends() should not
2846	be used except in Alpha arch-specific code and within the READ_ONCE() macro.
2847
2848
2849	CACHE COHERENCY VS DMA	2711	CACHE COHERENCY VS DMA
2850	----------------------	2712	----------------------
2851		2713
@@ -3009,10 +2871,8 @@ caches with the memory coherence system, thus making it seem like pointer · 157 unmodified lines
3009	changes vs new data occur in the right order.	2871	changes vs new data occur in the right order.
3010		2872
3011	The Alpha defines the Linux kernel's memory model, although as of v4.15	2873	The Alpha defines the Linux kernel's memory model, although as of v4.15
3012	the Linux kernel's addition of smp_read_barrier_depends() to READ_ONCE()	2874	the Linux kernel's addition of smp_mb() to READ_ONCE() on Alpha greatly
3013	greatly reduced Alpha's impact on the memory model.	2875	reduced its impact on the memory model.
3014
3015	See the subsection on "Cache Coherency" above.
3016		2876
3017		2877
3018	VIRTUAL MACHINE GUESTS	2878	VIRTUAL MACHINE GUESTS

Documentation/translations/ko_KR/memory-barriers.txt+3−143

@@ -577,7 +577,7 @@ ACQUIRE 는 해당 오퍼레이션의 로드 부분에만 적용되고 RELEASE · 576 unmodified lines
577	데이터 의존성 배리어 (역사적)	577	데이터 의존성 배리어 (역사적)
578	-----------------------------	578	-----------------------------
579		579
580	리눅스 커널 v4.15 기준으로, smp_read_barrier_depends() 가 READ_ONCE() 에	580	리눅스 커널 v4.15 기준으로, smp_mb() 가 DEC Alpha 용 READ_ONCE() 코드에
581	추가되었는데, 이는 이 섹션에 주의를 기울여야 하는 사람들은 DEC Alpha 아키텍쳐	581	추가되었는데, 이는 이 섹션에 주의를 기울여야 하는 사람들은 DEC Alpha 아키텍쳐
582	전용 코드를 만드는 사람들과 READ_ONCE() 자체를 만드는 사람들 뿐임을 의미합니다.	582	전용 코드를 만드는 사람들과 READ_ONCE() 자체를 만드는 사람들 뿐임을 의미합니다.
583	그런 분들을 위해, 그리고 역사에 관심 있는 분들을 위해, 여기 데이터 의존성	583	그런 분들을 위해, 그리고 역사에 관심 있는 분들을 위해, 여기 데이터 의존성
@@ -2664,144 +2664,6 @@ CPU 코어는 프로그램의 인과성이 유지된다고만 여겨진다면 · 2080 unmodified lines
2664	수도 있습니다.	2664	수도 있습니다.
2665		2665
2666		2666
2667	캐시 일관성
2668	-----------
2669
2670	하지만 삶은 앞에서 이야기한 것처럼 단순하지 않습니다: 캐시들은 일관적일 것으로
2671	기대되지만, 그 일관성이 순서에도 적용될 거라는 보장은 없습니다. 한 CPU 에서
2672	만들어진 변경 사항은 최종적으로는 시스템의 모든 CPU 에게 보여지게 되지만, 다른
2673	CPU 들에게도 같은 순서로 보이게 될 거라는 보장은 없다는 뜻입니다.
2674
2675
2676	두개의 CPU (1 & 2) 가 달려 있고, 각 CPU 에 두개의 데이터 캐시(CPU 1 은 A/B 를,
2677	CPU 2 는 C/D 를 갖습니다)가 병렬로 연결되어 있는 시스템을 다룬다고 생각해
2678	봅시다:
2679
2680	:
2681	: +--------+
2682	: +---------+ \| \|
2683	+--------+ : +--->\| Cache A \|<------->\| \|
2684	\| \| : \| +---------+ \| \|
2685	\| CPU 1 \|<---+ \| \|
2686	\| \| : \| +---------+ \| \|
2687	+--------+ : +--->\| Cache B \|<------->\| \|
2688	: +---------+ \| \|
2689	: \| Memory \|
2690	: +---------+ \| System \|
2691	+--------+ : +--->\| Cache C \|<------->\| \|
2692	\| \| : \| +---------+ \| \|
2693	\| CPU 2 \|<---+ \| \|
2694	\| \| : \| +---------+ \| \|
2695	+--------+ : +--->\| Cache D \|<------->\| \|
2696	: +---------+ \| \|
2697	: +--------+
2698	:
2699
2700	이 시스템이 다음과 같은 특성을 갖는다 생각해 봅시다:
2701
2702	(*) 홀수번 캐시라인은 캐시 A, 캐시 C 또는 메모리에 위치할 수 있음;
2703
2704	(*) 짝수번 캐시라인은 캐시 B, 캐시 D 또는 메모리에 위치할 수 있음;
2705
2706	(*) CPU 코어가 한개의 캐시에 접근하는 동안, 다른 캐시는 - 더티 캐시라인을
2707	메모리에 내리거나 추측성 로드를 하거나 하기 위해 - 시스템의 다른 부분에
2708	액세스 하기 위해 버스를 사용할 수 있음;
2709
2710	(*) 각 캐시는 시스템의 나머지 부분들과 일관성을 맞추기 위해 해당 캐시에
2711	적용되어야 할 오퍼레이션들의 큐를 가짐;
2712
2713	(*) 이 일관성 큐는 캐시에 이미 존재하는 라인에 가해지는 평범한 로드에 의해서는
2714	비워지지 않는데, 큐의 오퍼레이션들이 이 로드의 결과에 영향을 끼칠 수 있다
2715	할지라도 그러함.
2716
2717	이제, 첫번째 CPU 에서 두개의 쓰기 오퍼레이션을 만드는데, 해당 CPU 의 캐시에
2718	요청된 순서로 오퍼레이션이 도달됨을 보장하기 위해 두 오퍼레이션 사이에 쓰기
2719	배리어를 사용하는 상황을 상상해 봅시다:
2720
2721	CPU 1 CPU 2 COMMENT
2722	=============== =============== =======================================
2723	u == 0, v == 1 and p == &u, q == &u
2724	v = 2;
2725	smp_wmb(); v 의 변경이 p 의 변경 전에 보일 것을
2726	분명히 함
2727	<A:modify v=2> v 는 이제 캐시 A 에 독점적으로 존재함
2728	p = &v;
2729	<B:modify p=&v> p 는 이제 캐시 B 에 독점적으로 존재함
2730
2731	여기서의 쓰기 메모리 배리어는 CPU 1 의 캐시가 올바른 순서로 업데이트 된 것으로
2732	시스템의 다른 CPU 들이 인지하게 만듭니다. 하지만, 이제 두번째 CPU 가 그 값들을
2733	읽으려 하는 상황을 생각해 봅시다:
2734
2735	CPU 1 CPU 2 COMMENT
2736	=============== =============== =======================================
2737	...
2738	q = p;
2739	x = *q;
2740
2741	위의 두개의 읽기 오퍼레이션은 예상된 순서로 일어나지 못할 수 있는데, 두번째 CPU
2742	의 한 캐시에 다른 캐시 이벤트가 발생해 v 를 담고 있는 캐시라인의 해당 캐시에의
2743	업데이트가 지연되는 사이, p 를 담고 있는 캐시라인은 두번째 CPU 의 다른 캐시에
2744	업데이트 되어버렸을 수 있기 때문입니다.
2745
2746	CPU 1 CPU 2 COMMENT
2747	=============== =============== =======================================
2748	u == 0, v == 1 and p == &u, q == &u
2749	v = 2;
2750	smp_wmb();
2751	<A:modify v=2> <C:busy>
2752	<C:queue v=2>
2753	p = &v; q = p;
2754	<D:request p>
2755	<B:modify p=&v> <D:commit p=&v>
2756	<D:read p>
2757	x = *q;
2758	<C:read *q> 캐시에 업데이트 되기 전의 v 를 읽음
2759	<C:unbusy>
2760	<C:commit v=2>
2761
2762	기본적으로, 두개의 캐시라인 모두 CPU 2 에 최종적으로는 업데이트 될 것이지만,
2763	별도의 개입 없이는, 업데이트의 순서가 CPU 1 에서 만들어진 순서와 동일할
2764	것이라는 보장이 없습니다.
2765
2766
2767	여기에 개입하기 위해선, 데이터 의존성 배리어나 읽기 배리어를 로드 오퍼레이션들
2768	사이에 넣어야 합니다 (v4.15 부터는 READ_ONCE() 매크로에 의해 무조건적으로
2769	그렇게 됩니다). 이렇게 함으로써 캐시가 다음 요청을 처리하기 전에 일관성 큐를
2770	처리하도록 강제하게 됩니다.
2771
2772	CPU 1 CPU 2 COMMENT
2773	=============== =============== =======================================
2774	u == 0, v == 1 and p == &u, q == &u
2775	v = 2;
2776	smp_wmb();
2777	<A:modify v=2> <C:busy>
2778	<C:queue v=2>
2779	p = &v; q = p;
2780	<D:request p>
2781	<B:modify p=&v> <D:commit p=&v>
2782	<D:read p>
2783	smp_read_barrier_depends()
2784	<C:unbusy>
2785	<C:commit v=2>
2786	x = *q;
2787	<C:read *q> 캐시에 업데이트 된 v 를 읽음
2788
2789
2790	이런 부류의 문제는 DEC Alpha 계열 프로세서들에서 발견될 수 있는데, 이들은
2791	데이터 버스를 좀 더 잘 사용해 성능을 개선할 수 있는, 분할된 캐시를 가지고 있기
2792	때문입니다. 대부분의 CPU 는 하나의 읽기 오퍼레이션의 메모리 액세스가 다른 읽기
2793	오퍼레이션에 의존적이라면 데이터 의존성 배리어를 내포시킵니다만, 모두가 그런건
2794	아니기 때문에 이점에 의존해선 안됩니다.
2795
2796	다른 CPU 들도 분할된 캐시를 가지고 있을 수 있지만, 그런 CPU 들은 평범한 메모리
2797	액세스를 위해서도 이 분할된 캐시들 사이의 조정을 해야만 합니다. Alpha 는 가장
2798	약한 메모리 순서 시맨틱 (semantic) 을 선택함으로써 메모리 배리어가 명시적으로
2799	사용되지 않았을 때에는 그런 조정이 필요하지 않게 했으며, 이는 Alpha 가 당시에
2800	더 높은 CPU 클락 속도를 가질 수 있게 했습니다. 하지만, (다시 말하건대, v4.15
2801	이후부터는) Alpha 아키텍쳐 전용 코드와 READ_ONCE() 매크로 내부에서를 제외하고는
2802	smp_read_barrier_depends() 가 사용되지 않아야 함을 알아두시기 바랍니다.
2803
2804
2805	캐시 일관성 VS DMA	2667	캐시 일관성 VS DMA
2806	------------------	2668	------------------
2807		2669
@@ -2962,10 +2824,8 @@ Alpha CPU 의 일부 버전은 분할된 데이터 캐시를 가지고 있어서 · 154 unmodified lines
2962	데이터의 발견을 올바른 순서로 일어나게 하기 때문입니다.	2824	데이터의 발견을 올바른 순서로 일어나게 하기 때문입니다.
2963		2825
2964	리눅스 커널의 메모리 배리어 모델은 Alpha 에 기초해서 정의되었습니다만, v4.15	2826	리눅스 커널의 메모리 배리어 모델은 Alpha 에 기초해서 정의되었습니다만, v4.15
2965	부터는 리눅스 커널이 READ_ONCE() 내에 smp_read_barrier_depends() 를 추가해서	2827	부터는 Alpha 용 READ_ONCE() 코드 내에 smp_mb() 가 추가되어서 메모리 모델로의
2966	Alpha 의 메모리 모델로의 영향력이 크게 줄어들긴 했습니다.	2828	Alpha 의 영향력이 크게 줄어들었습니다.
2967
2968	위의 "캐시 일관성" 서브섹션을 참고하세요.
2969		2829
2970		2830
2971	가상 머신 게스트	2831	가상 머신 게스트

arch/alpha/include/asm/atomic.h+8−8

@@ -16,10 +16,10 @@ · 15 unmodified lines
16		16
17	/*	17	/*
18	* To ensure dependency ordering is preserved for the _relaxed and	18	* To ensure dependency ordering is preserved for the _relaxed and
19	* _release atomics, an smp_read_barrier_depends() is unconditionally	19	* _release atomics, an smp_mb() is unconditionally inserted into the
20	* inserted into the _relaxed variants, which are used to build the	20	* _relaxed variants, which are used to build the barriered versions.
21	* barriered versions. Avoid redundant back-to-back fences in the	21	* Avoid redundant back-to-back fences in the _acquire and _fence
22	* _acquire and _fence versions.	22	* versions.
23	*/	23	*/
24	#define __atomic_acquire_fence()	24	#define __atomic_acquire_fence()
25	#define __atomic_post_full_fence()	25	#define __atomic_post_full_fence()
@@ -70,7 +70,7 @@ static inline int atomic_##op##_return_relaxed(int i, atomic_t *v) \ · 44 unmodified lines
70	".previous" \	70	".previous" \
71	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \	71	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \
72	:"Ir" (i), "m" (v->counter) : "memory"); \	72	:"Ir" (i), "m" (v->counter) : "memory"); \
73	smp_read_barrier_depends(); \	73	smp_mb(); \
74	return result; \	74	return result; \
75	}	75	}
76		76
@@ -88,7 +88,7 @@ static inline int atomic_fetch_##op##_relaxed(int i, atomic_t *v) \ · 11 unmodified lines
88	".previous" \	88	".previous" \
89	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \	89	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \
90	:"Ir" (i), "m" (v->counter) : "memory"); \	90	:"Ir" (i), "m" (v->counter) : "memory"); \
91	smp_read_barrier_depends(); \	91	smp_mb(); \
92	return result; \	92	return result; \
93	}	93	}
94		94
@@ -123,7 +123,7 @@ static __inline__ s64 atomic64_##op##_return_relaxed(s64 i, atomic64_t * v) \ · 28 unmodified lines
123	".previous" \	123	".previous" \
124	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \	124	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \
125	:"Ir" (i), "m" (v->counter) : "memory"); \	125	:"Ir" (i), "m" (v->counter) : "memory"); \
126	smp_read_barrier_depends(); \	126	smp_mb(); \
127	return result; \	127	return result; \
128	}	128	}
129		129
@@ -141,7 +141,7 @@ static __inline__ s64 atomic64_fetch_##op##_relaxed(s64 i, atomic64_t * v) \ · 11 unmodified lines
141	".previous" \	141	".previous" \
142	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \	142	:"=&r" (temp), "=m" (v->counter), "=&r" (result) \
143	:"Ir" (i), "m" (v->counter) : "memory"); \	143	:"Ir" (i), "m" (v->counter) : "memory"); \
144	smp_read_barrier_depends(); \	144	smp_mb(); \
145	return result; \	145	return result; \
146	}	146	}
147		147

arch/alpha/include/asm/barrier.h+5−54

@@ -2,64 +2,15 @@ · 1 unmodified line
2	#ifndef __BARRIER_H	2	#ifndef __BARRIER_H
3	#define __BARRIER_H	3	#define __BARRIER_H
4		4
5	#include <asm/compiler.h>
6
7	#define mb() __asm__ __volatile__("mb": : :"memory")	5	#define mb() __asm__ __volatile__("mb": : :"memory")
8	#define rmb() __asm__ __volatile__("mb": : :"memory")	6	#define rmb() __asm__ __volatile__("mb": : :"memory")
9	#define wmb() __asm__ __volatile__("wmb": : :"memory")	7	#define wmb() __asm__ __volatile__("wmb": : :"memory")
10		8
11	/**	9	#define __smp_load_acquire(p) \
12	* read_barrier_depends - Flush all pending reads that subsequents reads	10	({ \
13	* depend on.	11	compiletime_assert_atomic_type(*p); \
14	*	12	__READ_ONCE(*p); \
15	* No data-dependent reads from memory-like regions are ever reordered	13	})
16	* over this barrier. All reads preceding this primitive are guaranteed
17	* to access memory (but not necessarily other CPUs' caches) before any
18	* reads following this primitive that depend on the data return by
19	* any of the preceding reads. This primitive is much lighter weight than
20	* rmb() on most CPUs, and is never heavier weight than is
21	* rmb().
22	*
23	* These ordering constraints are respected by both the local CPU
24	* and the compiler.
25	*
26	* Ordering is not guaranteed by anything other than these primitives,
27	* not even by data dependencies. See the documentation for
28	* memory_barrier() for examples and URLs to more information.
29	*
30	* For example, the following code would force ordering (the initial
31	* value of "a" is zero, "b" is one, and "p" is "&a"):
32	*
33	* <programlisting>
34	* CPU 0 CPU 1
35	*
36	* b = 2;
37	* memory_barrier();
38	* p = &b; q = p;
39	* read_barrier_depends();
40	* d = *q;
41	* </programlisting>
42	*
43	* because the read of "*q" depends on the read of "p" and these
44	* two reads are separated by a read_barrier_depends(). However,
45	* the following code, with the same initial values for "a" and "b":
46	*
47	* <programlisting>
48	* CPU 0 CPU 1
49	*
50	* a = 2;
51	* memory_barrier();
52	* b = 3; y = b;
53	* read_barrier_depends();
54	* x = a;
55	* </programlisting>
56	*
57	* does not enforce ordering, since there is no data dependency between
58	* the read of "a" and the read of "b". Therefore, on some CPUs, such
59	* as Alpha, "y" could be set to 3 and "x" to 0. Use rmb()
60	* in cases like this where there are no data dependencies.
61	*/
62	#define read_barrier_depends() __asm__ __volatile__("mb": : :"memory")
63		14
64	#ifdef CONFIG_SMP	15	#ifdef CONFIG_SMP
65	#define __ASM_SMP_MB "\tmb\n"	16	#define __ASM_SMP_MB "\tmb\n"

arch/alpha/include/asm/pgtable.h+5−5

@@ -277,9 +277,9 @@ extern inline pte_t pte_mkdirty(pte_t pte) { pte_val(pte) \|= __DIRTY_BITS; retur · 276 unmodified lines
277	extern inline pte_t pte_mkyoung(pte_t pte) { pte_val(pte) \|= __ACCESS_BITS; return pte; }	277	extern inline pte_t pte_mkyoung(pte_t pte) { pte_val(pte) \|= __ACCESS_BITS; return pte; }
278		278
279	/*	279	/*
280	* The smp_read_barrier_depends() in the following functions are required to	280	* The smp_rmb() in the following functions are required to order the load of
281	* order the load of *dir (the pointer in the top level page table) with any	281	* *dir (the pointer in the top level page table) with any subsequent load of
282	* subsequent load of the returned pmd_t ret (ret is data dependent on dir).	282	* the returned pmd_t ret (ret is data dependent on dir).
283	*	283	*
284	* If this ordering is not enforced, the CPU might load an older value of	284	* If this ordering is not enforced, the CPU might load an older value of
285	* *ret, which may be uninitialized data. See mm/memory.c:__pte_alloc for	285	* *ret, which may be uninitialized data. See mm/memory.c:__pte_alloc for
@@ -293,7 +293,7 @@ extern inline pte_t pte_mkyoung(pte_t pte) { pte_val(pte) \|= __ACCESS_BITS; retu · 7 unmodified lines
293	extern inline pmd_t * pmd_offset(pud_t * dir, unsigned long address)	293	extern inline pmd_t * pmd_offset(pud_t * dir, unsigned long address)
294	{	294	{
295	pmd_t ret = (pmd_t ) pud_page_vaddr(*dir) + ((address >> PMD_SHIFT) & (PTRS_PER_PAGE - 1));	295	pmd_t ret = (pmd_t ) pud_page_vaddr(*dir) + ((address >> PMD_SHIFT) & (PTRS_PER_PAGE - 1));
296	smp_read_barrier_depends(); /* see above */	296	smp_rmb(); /* see above */
297	return ret;	297	return ret;
298	}	298	}
299	#define pmd_offset pmd_offset	299	#define pmd_offset pmd_offset
@@ -303,7 +303,7 @@ extern inline pte_t * pte_offset_kernel(pmd_t * dir, unsigned long address) · 3 unmodified lines
303	{	303	{
304	pte_t ret = (pte_t ) pmd_page_vaddr(*dir)	304	pte_t ret = (pte_t ) pmd_page_vaddr(*dir)
305	+ ((address >> PAGE_SHIFT) & (PTRS_PER_PAGE - 1));	305	+ ((address >> PAGE_SHIFT) & (PTRS_PER_PAGE - 1));
306	smp_read_barrier_depends(); /* see above */	306	smp_rmb(); /* see above */
307	return ret;	307	return ret;
308	}	308	}
309	#define pte_offset_kernel pte_offset_kernel	309	#define pte_offset_kernel pte_offset_kernel

arch/alpha/include/asm/rwonce.h+35−0

@@ -0,0 +1,35 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2019 Google LLC.
	4	*/
	5	#ifndef __ASM_RWONCE_H
	6	#define __ASM_RWONCE_H
	7
	8	#ifdef CONFIG_SMP
	9
	10	#include <asm/barrier.h>
	11
	12	/*
	13	* Alpha is apparently daft enough to reorder address-dependent loads
	14	* on some CPU implementations. Knock some common sense into it with
	15	* a memory barrier in READ_ONCE().
	16	*
	17	* For the curious, more information about this unusual reordering is
	18	* available in chapter 15 of the "perfbook":
	19	*
	20	* https://kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html
	21	*
	22	*/
	23	#define __READ_ONCE(x) \
	24	({ \
	25	__unqual_scalar_typeof(x) __x = \
	26	((volatile typeof(__x) )(&(x))); \
	27	mb(); \
	28	(typeof(x))__x; \
	29	})
	30
	31	#endif /* CONFIG_SMP */
	32
	33	#include <asm-generic/rwonce.h>
	34
	35	#endif /* __ASM_RWONCE_H */

arch/arm/include/asm/vdso/gettimeofday.h+1−0

@@ -7,6 +7,7 @@ · 6 unmodified lines
7		7
8	#ifndef __ASSEMBLY__	8	#ifndef __ASSEMBLY__
9		9
		10	#include <asm/barrier.h>
10	#include <asm/errno.h>	11	#include <asm/errno.h>
11	#include <asm/unistd.h>	12	#include <asm/unistd.h>
12	#include <asm/vdso/cp15.h>	13	#include <asm/vdso/cp15.h>

arch/arm64/Kconfig+21−2

 	select GENERIC_STRNLEN_USER
 	select GENERIC_TIME_VSYSCALL
 	select GENERIC_GETTIMEOFDAY
+	select GENERIC_VDSO_TIME_NS
 	select HANDLE_DOMAIN_IRQ
 	select HARDIRQS_SW_RESEND
 	select HAVE_PCI
 	  ARMv8 obsoletes the use of A32 SWP/SWPB instructions such that
 	  they are always undefined. Say Y here to enable software
 	  emulation of these instructions for userspace using LDXR/STXR.
+	  This feature can be controlled at runtime with the abi.swp
+	  sysctl which is disabled by default.
 	  In some older versions of glibc [<=2.8] SWP is used during futex
 	  trylock() operations with the assumption that the code will not
 	  Say Y here to enable software emulation of these
 	  instructions for AArch32 userspace code. When this option is
 	  enabled, CP15 barrier usage is traced which can help
-	  identify software that needs updating.
+	  identify software that needs updating. This feature can be
+	  controlled at runtime with the abi.cp15_barrier sysctl.
 	  If unsure, say Y
 	  AArch32 EL0, and is deprecated in ARMv8.
 	  Say Y here to enable software emulation of the instruction
-	  for AArch32 userspace code.
+	  for AArch32 userspace code. This feature can be controlled
+	  at runtime with the abi.setend sysctl.
 	  Note: All the cpus on the system must have mixed endian support at EL0
 	  for this feature to be enabled. If a new CPU - which doesn't support mixed
 	  correctly reflect reality. Most commonly, the value read will be 0,
 	  indicating that the counter is not enabled.
+config AS_HAS_ARMV8_4
+	def_bool $(cc-option,-Wa$(comma)-march=armv8.4-a)
+config ARM64_TLB_RANGE
+	bool "Enable support for tlbi range feature"
+	default y
+	depends on AS_HAS_ARMV8_4
+	help
+	  ARMv8.4-TLBI provides TLBI invalidation instruction that apply to a
+	  range of input addresses.
+	  The feature introduces new assembly instructions, and they were
+	  support when binutils >= 2.30.
 endmenu
 menu "ARMv8.5 architectural features"

arch/arm64/Makefile+7−0

 # compiler to generate them and consequently to break the single image contract
 # we pass it only to the assembler. This option is utilized only in case of non
 # integrated assemblers.
+ifneq ($(CONFIG_AS_HAS_ARMV8_4), y)
 branch-prot-flags-$(CONFIG_AS_HAS_PAC) += -Wa,-march=armv8.3-a
 endif
+endif
 KBUILD_CFLAGS += $(branch-prot-flags-y)
+ifeq ($(CONFIG_AS_HAS_ARMV8_4), y)
+# make sure to pass the newest target architecture to -march.
+KBUILD_CFLAGS	+= -Wa,-march=armv8.4-a
+endif
 ifeq ($(CONFIG_SHADOW_CALL_STACK), y)
 KBUILD_CFLAGS	+= -ffixed-x18
 endif

arch/arm64/configs/defconfig+1−0

@@ -66,6 +66,7 @@ CONFIG_SCHED_SMT=y · 65 unmodified lines
66	CONFIG_NUMA=y	66	CONFIG_NUMA=y
67	CONFIG_SECCOMP=y	67	CONFIG_SECCOMP=y
68	CONFIG_KEXEC=y	68	CONFIG_KEXEC=y
		69	CONFIG_KEXEC_FILE=y
69	CONFIG_CRASH_DUMP=y	70	CONFIG_CRASH_DUMP=y
70	CONFIG_XEN=y	71	CONFIG_XEN=y
71	CONFIG_COMPAT=y	72	CONFIG_COMPAT=y

arch/arm64/include/asm/acpi.h+1−14

@@ -47,20 +47,7 @@ · 46 unmodified lines
47	pgprot_t __acpi_get_mem_attribute(phys_addr_t addr);	47	pgprot_t __acpi_get_mem_attribute(phys_addr_t addr);
48		48
49	/* ACPI table mapping after acpi_permanent_mmap is set */	49	/* ACPI table mapping after acpi_permanent_mmap is set */
50	static inline void __iomem *acpi_os_ioremap(acpi_physical_address phys,	50	void __iomem *acpi_os_ioremap(acpi_physical_address phys, acpi_size size);
51	acpi_size size)
52	{
53	/* For normal memory we already have a cacheable mapping. */
54	if (memblock_is_map_memory(phys))
55	return (void __iomem *)__phys_to_virt(phys);
56
57	/*
58	* We should still honor the memory's attribute here because
59	* crash dump kernel possibly excludes some ACPI (reclaim)
60	* regions from memblock list.
61	*/
62	return __ioremap(phys, size, __acpi_get_mem_attribute(phys));
63	}
64	#define acpi_os_ioremap acpi_os_ioremap	51	#define acpi_os_ioremap acpi_os_ioremap
65		52
66	typedef u64 phys_cpuid_t;	53	typedef u64 phys_cpuid_t;

arch/arm64/include/asm/cpucaps.h+3−1

@@ -62,7 +62,9 @@ · 61 unmodified lines
62	#define ARM64_HAS_GENERIC_AUTH 52	62	#define ARM64_HAS_GENERIC_AUTH 52
63	#define ARM64_HAS_32BIT_EL1 53	63	#define ARM64_HAS_32BIT_EL1 53
64	#define ARM64_BTI 54	64	#define ARM64_BTI 54
		65	#define ARM64_HAS_ARMv8_4_TTL 55
		66	#define ARM64_HAS_TLB_RANGE 56
65		67
66	#define ARM64_NCAPS 55	68	#define ARM64_NCAPS 57
67		69
68	#endif /* __ASM_CPUCAPS_H */	70	#endif /* __ASM_CPUCAPS_H */

arch/arm64/include/asm/cpufeature.h+7−0

 	return IS_ENABLED(CONFIG_ARM64_BTI) && cpus_have_const_cap(ARM64_BTI);
 }
+static inline bool system_supports_tlb_range(void)
+{
+	return IS_ENABLED(CONFIG_ARM64_TLB_RANGE) &&
+		cpus_have_const_cap(ARM64_HAS_TLB_RANGE);
+}
 #define ARM64_BP_HARDEN_UNKNOWN		-1
 #define ARM64_BP_HARDEN_WA_NEEDED	0
 #define ARM64_BP_HARDEN_NOT_REQUIRED	1
 }
 u32 get_kvm_ipa_limit(void);
+void dump_cpu_features(void);
 #endif /* __ASSEMBLY__ */

arch/arm64/include/asm/hugetlb.h+2−0

@@ -49,6 +49,8 @@ extern void set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr, · 48 unmodified lines
49	pte_t *ptep, pte_t pte, unsigned long sz);	49	pte_t *ptep, pte_t pte, unsigned long sz);
50	#define set_huge_swap_pte_at set_huge_swap_pte_at	50	#define set_huge_swap_pte_at set_huge_swap_pte_at
51		51
		52	void __init arm64_hugetlb_cma_reserve(void);
		53
52	#include <asm-generic/hugetlb.h>	54	#include <asm-generic/hugetlb.h>
53		55
54	#endif /* __ASM_HUGETLB_H */	56	#endif /* __ASM_HUGETLB_H */

arch/arm64/include/asm/hwcap.h+1−0

@@ -95,6 +95,7 @@ · 94 unmodified lines
95	#define KERNEL_HWCAP_DGH __khwcap2_feature(DGH)	95	#define KERNEL_HWCAP_DGH __khwcap2_feature(DGH)
96	#define KERNEL_HWCAP_RNG __khwcap2_feature(RNG)	96	#define KERNEL_HWCAP_RNG __khwcap2_feature(RNG)
97	#define KERNEL_HWCAP_BTI __khwcap2_feature(BTI)	97	#define KERNEL_HWCAP_BTI __khwcap2_feature(BTI)
		98	/* reserved for KERNEL_HWCAP_MTE __khwcap2_feature(MTE) */
98		99
99	/*	100	/*
100	* This yields a mask that user programs can use to figure out what	101	* This yields a mask that user programs can use to figure out what

arch/arm64/include/asm/kernel-pgtable.h+1−1

@@ -8,7 +8,7 @@ · 7 unmodified lines
8	#ifndef __ASM_KERNEL_PGTABLE_H	8	#ifndef __ASM_KERNEL_PGTABLE_H
9	#define __ASM_KERNEL_PGTABLE_H	9	#define __ASM_KERNEL_PGTABLE_H
10		10
11	#include <linux/pgtable.h>	11	#include <asm/pgtable-hwdef.h>
12	#include <asm/sparsemem.h>	12	#include <asm/sparsemem.h>
13		13
14	/*	14	/*

arch/arm64/include/asm/memory.h+7−5

 #ifndef __ASM_MEMORY_H
 #define __ASM_MEMORY_H
-#include <linux/compiler.h>
 #include <linux/const.h>
 #include <linux/sizes.h>
-#include <linux/types.h>
-#include <asm/bug.h>
 #include <asm/page-def.h>
 /*
 #endif
 #ifndef __ASSEMBLY__
-extern u64			vabits_actual;
-#define PAGE_END		(_PAGE_END(vabits_actual))
 #include <linux/bitops.h>
+#include <linux/compiler.h>
 #include <linux/mmdebug.h>
+#include <linux/types.h>
+#include <asm/bug.h>
+extern u64			vabits_actual;
+#define PAGE_END		(_PAGE_END(vabits_actual))
 extern s64			physvirt_offset;
 extern s64			memstart_addr;
 	__is_lm_address(__addr) && pfn_valid(virt_to_pfn(__addr));	\
 })
+void dump_mem_limit(void);
 #endif /* !ASSEMBLY */
 /*

arch/arm64/include/asm/mmu_context.h+2−4

  * take CPU migration into account.
  */
 #define destroy_context(mm)		do { } while(0)
-void check_and_switch_context(struct mm_struct *mm, unsigned int cpu);
+void check_and_switch_context(struct mm_struct *mm);
 #define init_new_context(tsk,mm)	({ atomic64_set(&(mm)->context.id, 0); 0; })
 static inline void __switch_mm(struct mm_struct *next)
 {
-	unsigned int cpu = smp_processor_id();
 	/*
 	 * init_mm.pgd does not contain any user mappings and it is always
 	 * active for kernel addresses in TTBR1. Just set the reserved TTBR0.
 		return;
 	}
-	check_and_switch_context(next, cpu);
+	check_and_switch_context(next);
 }
 static inline void

arch/arm64/include/asm/perf_event.h+27−0

@@ -72,6 +72,13 @@ · 71 unmodified lines
72	#define ARMV8_PMUV3_PERFCTR_LL_CACHE_RD 0x36	72	#define ARMV8_PMUV3_PERFCTR_LL_CACHE_RD 0x36
73	#define ARMV8_PMUV3_PERFCTR_LL_CACHE_MISS_RD 0x37	73	#define ARMV8_PMUV3_PERFCTR_LL_CACHE_MISS_RD 0x37
74	#define ARMV8_PMUV3_PERFCTR_REMOTE_ACCESS_RD 0x38	74	#define ARMV8_PMUV3_PERFCTR_REMOTE_ACCESS_RD 0x38
		75	#define ARMV8_PMUV3_PERFCTR_L1D_CACHE_LMISS_RD 0x39
		76	#define ARMV8_PMUV3_PERFCTR_OP_RETIRED 0x3A
		77	#define ARMV8_PMUV3_PERFCTR_OP_SPEC 0x3B
		78	#define ARMV8_PMUV3_PERFCTR_STALL 0x3C
		79	#define ARMV8_PMUV3_PERFCTR_STALL_SLOT_BACKEND 0x3D
		80	#define ARMV8_PMUV3_PERFCTR_STALL_SLOT_FRONTEND 0x3E
		81	#define ARMV8_PMUV3_PERFCTR_STALL_SLOT 0x3F
75		82
76	/* Statistical profiling extension microarchitectural events */	83	/* Statistical profiling extension microarchitectural events */
77	#define ARMV8_SPE_PERFCTR_SAMPLE_POP 0x4000	84	#define ARMV8_SPE_PERFCTR_SAMPLE_POP 0x4000
@@ -79,6 +86,26 @@ · 1 unmodified line
79	#define ARMV8_SPE_PERFCTR_SAMPLE_FILTRATE 0x4002	86	#define ARMV8_SPE_PERFCTR_SAMPLE_FILTRATE 0x4002
80	#define ARMV8_SPE_PERFCTR_SAMPLE_COLLISION 0x4003	87	#define ARMV8_SPE_PERFCTR_SAMPLE_COLLISION 0x4003
81		88
		89	/* AMUv1 architecture events */
		90	#define ARMV8_AMU_PERFCTR_CNT_CYCLES 0x4004
		91	#define ARMV8_AMU_PERFCTR_STALL_BACKEND_MEM 0x4005
		92
		93	/* long-latency read miss events */
		94	#define ARMV8_PMUV3_PERFCTR_L1I_CACHE_LMISS 0x4006
		95	#define ARMV8_PMUV3_PERFCTR_L2D_CACHE_LMISS_RD 0x4009
		96	#define ARMV8_PMUV3_PERFCTR_L2I_CACHE_LMISS 0x400A
		97	#define ARMV8_PMUV3_PERFCTR_L3D_CACHE_LMISS_RD 0x400B
		98
		99	/* additional latency from alignment events */
		100	#define ARMV8_PMUV3_PERFCTR_LDST_ALIGN_LAT 0x4020
		101	#define ARMV8_PMUV3_PERFCTR_LD_ALIGN_LAT 0x4021
		102	#define ARMV8_PMUV3_PERFCTR_ST_ALIGN_LAT 0x4022
		103
		104	/* Armv8.5 Memory Tagging Extension events */
		105	#define ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED 0x4024
		106	#define ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED_RD 0x4025
		107	#define ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED_WR 0x4026
		108
82	/* ARMv8 recommended implementation defined event types */	109	/* ARMv8 recommended implementation defined event types */
83	#define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_RD 0x40	110	#define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_RD 0x40
84	#define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR 0x41	111	#define ARMV8_IMPDEF_PERFCTR_L1D_CACHE_WR 0x41

arch/arm64/include/asm/pgtable-hwdef.h+13−10

  * Size mapped by an entry at level n ( 0 <= n <= 3)
  * We map (PAGE_SHIFT - 3) at all translation levels and PAGE_SHIFT bits
  * in the final page. The maximum number of translation levels supported by
- * the architecture is 4. Hence, starting at at level n, we have further
+ * the architecture is 4. Hence, starting at level n, we have further
  * ((4 - n) - 1) levels of translation excluding the offset within the page.
  * So, the total number of bits mapped by an entry at level n is :
  *
  * Contiguous page definitions.
  */
 #ifdef CONFIG_ARM64_64K_PAGES
-#define CONT_PTE_SHIFT		5
+#define CONT_PTE_SHIFT		(5 + PAGE_SHIFT)
-#define CONT_PMD_SHIFT		5
+#define CONT_PMD_SHIFT		(5 + PMD_SHIFT)
 #elif defined(CONFIG_ARM64_16K_PAGES)
-#define CONT_PTE_SHIFT		7
+#define CONT_PTE_SHIFT		(7 + PAGE_SHIFT)
-#define CONT_PMD_SHIFT		5
+#define CONT_PMD_SHIFT		(5 + PMD_SHIFT)
 #else
-#define CONT_PTE_SHIFT		4
+#define CONT_PTE_SHIFT		(4 + PAGE_SHIFT)
-#define CONT_PMD_SHIFT		4
+#define CONT_PMD_SHIFT		(4 + PMD_SHIFT)
 #endif
-#define CONT_PTES		(1 << CONT_PTE_SHIFT)
+#define CONT_PTES		(1 << (CONT_PTE_SHIFT - PAGE_SHIFT))
 #define CONT_PTE_SIZE		(CONT_PTES * PAGE_SIZE)
 #define CONT_PTE_MASK		(~(CONT_PTE_SIZE - 1))
-#define CONT_PMDS		(1 << CONT_PMD_SHIFT)
+#define CONT_PMDS		(1 << (CONT_PMD_SHIFT - PMD_SHIFT))
 #define CONT_PMD_SIZE		(CONT_PMDS * PMD_SIZE)
 #define CONT_PMD_MASK		(~(CONT_PMD_SIZE - 1))
-/* the the numerical offset of the PTE within a range of CONT_PTES */
+/* the numerical offset of the PTE within a range of CONT_PTES */
 #define CONT_RANGE_OFFSET(addr) (((addr)>>PAGE_SHIFT)&(CONT_PTES-1))
 /*
 #define PTE_S2_RDONLY		(_AT(pteval_t, 1) << 6)   /* HAP[2:1] */
 #define PTE_S2_RDWR		(_AT(pteval_t, 3) << 6)   /* HAP[2:1] */
 #define PTE_S2_XN		(_AT(pteval_t, 2) << 53)  /* XN[1:0] */
+#define PTE_S2_SW_RESVD		(_AT(pteval_t, 15) << 55) /* Reserved for SW */
 #define PMD_S2_RDONLY		(_AT(pmdval_t, 1) << 6)   /* HAP[2:1] */
 #define PMD_S2_RDWR		(_AT(pmdval_t, 3) << 6)   /* HAP[2:1] */
 #define PMD_S2_XN		(_AT(pmdval_t, 2) << 53)  /* XN[1:0] */
+#define PMD_S2_SW_RESVD		(_AT(pmdval_t, 15) << 55) /* Reserved for SW */
 #define PUD_S2_RDONLY		(_AT(pudval_t, 1) << 6)   /* HAP[2:1] */
 #define PUD_S2_RDWR		(_AT(pudval_t, 3) << 6)   /* HAP[2:1] */
 #define TCR_TxSZ(x)		(TCR_T0SZ(x) | TCR_T1SZ(x))
 #define TCR_TxSZ_WIDTH		6
 #define TCR_T0SZ_MASK		(((UL(1) << TCR_TxSZ_WIDTH) - 1) << TCR_T0SZ_OFFSET)
+#define TCR_T1SZ_MASK		(((UL(1) << TCR_TxSZ_WIDTH) - 1) << TCR_T1SZ_OFFSET)
 #define TCR_EPD0_SHIFT		7
 #define TCR_EPD0_MASK		(UL(1) << TCR_EPD0_SHIFT)

arch/arm64/include/asm/pgtable.h+10−0

@@ -40,6 +40,16 @@ extern void __pmd_error(const char *file, int line, unsigned long val); · 39 unmodified lines
40	extern void __pud_error(const char *file, int line, unsigned long val);	40	extern void __pud_error(const char *file, int line, unsigned long val);
41	extern void __pgd_error(const char *file, int line, unsigned long val);	41	extern void __pgd_error(const char *file, int line, unsigned long val);
42		42
		43	#ifdef CONFIG_TRANSPARENT_HUGEPAGE
		44	#define __HAVE_ARCH_FLUSH_PMD_TLB_RANGE
		45
		46	/* Set stride and tlb_level in flush__tlb_range /
		47	#define flush_pmd_tlb_range(vma, addr, end) \
		48	__flush_tlb_range(vma, addr, end, PMD_SIZE, false, 2)
		49	#define flush_pud_tlb_range(vma, addr, end) \
		50	__flush_tlb_range(vma, addr, end, PUD_SIZE, false, 1)
		51	#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
		52
43	/*	53	/*
44	* ZERO_PAGE is a global shared page that is always zero: used	54	* ZERO_PAGE is a global shared page that is always zero: used
45	* for zero-mapped memory areas etc..	55	* for zero-mapped memory areas etc..

arch/arm64/include/asm/ptrace.h+1−1

@@ -27,7 +27,7 @@ · 26 unmodified lines
27	*	27	*
28	* Some code sections either automatically switch back to PSR.I or explicitly	28	* Some code sections either automatically switch back to PSR.I or explicitly
29	* require to not use priority masking. If bit GIC_PRIO_PSR_I_SET is included	29	* require to not use priority masking. If bit GIC_PRIO_PSR_I_SET is included
30	* in the the priority mask, it indicates that PSR.I should be set and	30	* in the priority mask, it indicates that PSR.I should be set and
31	* interrupt disabling temporarily does not rely on IRQ priorities.	31	* interrupt disabling temporarily does not rely on IRQ priorities.
32	*/	32	*/
33	#define GIC_PRIO_IRQON 0xe0	33	#define GIC_PRIO_IRQON 0xe0

arch/arm64/include/asm/stage2_pgtable.h+9−0

@@ -256,4 +256,13 @@ stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end) · 255 unmodified lines
256	return (boundary - 1 < end - 1) ? boundary : end;	256	return (boundary - 1 < end - 1) ? boundary : end;
257	}	257	}
258		258
		259	/*
		260	* Level values for the ARMv8.4-TTL extension, mapping PUD/PMD/PTE and
		261	* the architectural page-table level.
		262	*/
		263	#define S2_NO_LEVEL_HINT 0
		264	#define S2_PUD_LEVEL 1
		265	#define S2_PMD_LEVEL 2
		266	#define S2_PTE_LEVEL 3
		267
259	#endif /* __ARM64_S2_PGTABLE_H_ */	268	#endif /* __ARM64_S2_PGTABLE_H_ */

arch/arm64/include/asm/sysreg.h+47−2

  */
 #define SYS_AMEVCNTR0_EL0(n)		SYS_AM_EL0(4 + ((n) >> 3), (n) & 7)
-#define SYS_AMEVTYPE0_EL0(n)		SYS_AM_EL0(6 + ((n) >> 3), (n) & 7)
+#define SYS_AMEVTYPER0_EL0(n)		SYS_AM_EL0(6 + ((n) >> 3), (n) & 7)
 #define SYS_AMEVCNTR1_EL0(n)		SYS_AM_EL0(12 + ((n) >> 3), (n) & 7)
-#define SYS_AMEVTYPE1_EL0(n)		SYS_AM_EL0(14 + ((n) >> 3), (n) & 7)
+#define SYS_AMEVTYPER1_EL0(n)		SYS_AM_EL0(14 + ((n) >> 3), (n) & 7)
 /* AMU v1: Fixed (architecturally defined) activity monitors */
 #define SYS_AMEVCNTR0_CORE_EL0		SYS_AMEVCNTR0_EL0(0)
 #define ID_AA64ISAR0_SHA1_SHIFT		8
 #define ID_AA64ISAR0_AES_SHIFT		4
+#define ID_AA64ISAR0_TLB_RANGE_NI	0x0
+#define ID_AA64ISAR0_TLB_RANGE		0x2
 /* id_aa64isar1 */
 #define ID_AA64ISAR1_I8MM_SHIFT		52
 #define ID_AA64ISAR1_DGH_SHIFT		48
 #define ID_AA64ZFR0_SVEVER_SVE2		0x1
 /* id_aa64mmfr0 */
+#define ID_AA64MMFR0_ECV_SHIFT		60
+#define ID_AA64MMFR0_FGT_SHIFT		56
+#define ID_AA64MMFR0_EXS_SHIFT		44
 #define ID_AA64MMFR0_TGRAN4_2_SHIFT	40
 #define ID_AA64MMFR0_TGRAN64_2_SHIFT	36
 #define ID_AA64MMFR0_TGRAN16_2_SHIFT	32
 #endif
 /* id_aa64mmfr1 */
+#define ID_AA64MMFR1_ETS_SHIFT		36
+#define ID_AA64MMFR1_TWED_SHIFT		32
+#define ID_AA64MMFR1_XNX_SHIFT		28
+#define ID_AA64MMFR1_SPECSEI_SHIFT	24
 #define ID_AA64MMFR1_PAN_SHIFT		20
 #define ID_AA64MMFR1_LOR_SHIFT		16
 #define ID_AA64MMFR1_HPD_SHIFT		12
 /* id_aa64mmfr2 */
 #define ID_AA64MMFR2_E0PD_SHIFT		60
+#define ID_AA64MMFR2_EVT_SHIFT		56
+#define ID_AA64MMFR2_BBM_SHIFT		52
+#define ID_AA64MMFR2_TTL_SHIFT		48
 #define ID_AA64MMFR2_FWB_SHIFT		40
+#define ID_AA64MMFR2_IDS_SHIFT		36
 #define ID_AA64MMFR2_AT_SHIFT		32
+#define ID_AA64MMFR2_ST_SHIFT		28
+#define ID_AA64MMFR2_NV_SHIFT		24
+#define ID_AA64MMFR2_CCIDX_SHIFT	20
 #define ID_AA64MMFR2_LVA_SHIFT		16
 #define ID_AA64MMFR2_IESB_SHIFT		12
 #define ID_AA64MMFR2_LSM_SHIFT		8
 #define ID_AA64MMFR2_CNP_SHIFT		0
 /* id_aa64dfr0 */
+#define ID_AA64DFR0_DOUBLELOCK_SHIFT	36
 #define ID_AA64DFR0_PMSVER_SHIFT	32
 #define ID_AA64DFR0_CTX_CMPS_SHIFT	28
 #define ID_AA64DFR0_WRPS_SHIFT		20
 #define ID_ISAR6_DP_SHIFT		4
 #define ID_ISAR6_JSCVT_SHIFT		0
+#define ID_MMFR0_INNERSHR_SHIFT		28
+#define ID_MMFR0_FCSE_SHIFT		24
+#define ID_MMFR0_AUXREG_SHIFT		20
+#define ID_MMFR0_TCM_SHIFT		16
+#define ID_MMFR0_SHARELVL_SHIFT		12
+#define ID_MMFR0_OUTERSHR_SHIFT		8
+#define ID_MMFR0_PMSA_SHIFT		4
+#define ID_MMFR0_VMSA_SHIFT		0
 #define ID_MMFR4_EVT_SHIFT		28
 #define ID_MMFR4_CCIDX_SHIFT		24
 #define ID_MMFR4_LSM_SHIFT		20
 #define ID_MMFR4_HPDS_SHIFT		16
 #define ID_MMFR4_CNP_SHIFT		12
 #define ID_MMFR4_XNX_SHIFT		8
+#define ID_MMFR4_AC2_SHIFT		4
 #define ID_MMFR4_SPECSEI_SHIFT		0
 #define ID_MMFR5_ETS_SHIFT		0
 #define ID_PFR0_DIT_SHIFT		24
 #define ID_PFR0_CSV2_SHIFT		16
+#define ID_PFR0_STATE3_SHIFT		12
+#define ID_PFR0_STATE2_SHIFT		8
+#define ID_PFR0_STATE1_SHIFT		4
+#define ID_PFR0_STATE0_SHIFT		0
+#define ID_DFR0_PERFMON_SHIFT		24
+#define ID_DFR0_MPROFDBG_SHIFT		20
+#define ID_DFR0_MMAPTRC_SHIFT		16
+#define ID_DFR0_COPTRC_SHIFT		12
+#define ID_DFR0_MMAPDBG_SHIFT		8
+#define ID_DFR0_COPSDBG_SHIFT		4
+#define ID_DFR0_COPDBG_SHIFT		0
 #define ID_PFR2_SSBS_SHIFT		4
 #define ID_PFR2_CSV3_SHIFT		0
 #define ID_AA64MMFR0_TGRAN_SUPPORTED	ID_AA64MMFR0_TGRAN64_SUPPORTED
 #endif
+#define MVFR2_FPMISC_SHIFT		4
+#define MVFR2_SIMDMISC_SHIFT		0
+#define DCZID_DZP_SHIFT			4
+#define DCZID_BS_SHIFT			0
 /*
  * The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which

arch/arm64/include/asm/tlb.h+28−1

@@ -21,11 +21,37 @@ static void tlb_flush(struct mmu_gather *tlb); · 20 unmodified lines
21		21
22	#include <asm-generic/tlb.h>	22	#include <asm-generic/tlb.h>
23		23
		24	/*
		25	* get the tlbi levels in arm64. Default value is 0 if more than one
		26	* of cleared_* is set or neither is set.
		27	* Arm64 doesn't support p4ds now.
		28	*/
		29	static inline int tlb_get_level(struct mmu_gather *tlb)
		30	{
		31	if (tlb->cleared_ptes && !(tlb->cleared_pmds \|\|
		32	tlb->cleared_puds \|\|
		33	tlb->cleared_p4ds))
		34	return 3;
		35
		36	if (tlb->cleared_pmds && !(tlb->cleared_ptes \|\|
		37	tlb->cleared_puds \|\|
		38	tlb->cleared_p4ds))
		39	return 2;
		40
		41	if (tlb->cleared_puds && !(tlb->cleared_ptes \|\|
		42	tlb->cleared_pmds \|\|
		43	tlb->cleared_p4ds))
		44	return 1;
		45
		46	return 0;
		47	}
		48
24	static inline void tlb_flush(struct mmu_gather *tlb)	49	static inline void tlb_flush(struct mmu_gather *tlb)
25	{	50	{
26	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);	51	struct vm_area_struct vma = TLB_FLUSH_VMA(tlb->mm, 0);
27	bool last_level = !tlb->freed_tables;	52	bool last_level = !tlb->freed_tables;
28	unsigned long stride = tlb_get_unmap_size(tlb);	53	unsigned long stride = tlb_get_unmap_size(tlb);
		54	int tlb_level = tlb_get_level(tlb);
29		55
30	/*	56	/*
31	* If we're tearing down the address space then we only care about	57	* If we're tearing down the address space then we only care about
@@ -38,7 +64,8 @@ static inline void tlb_flush(struct mmu_gather *tlb) · 6 unmodified lines
38	return;	64	return;
39	}	65	}
40		66
41	__flush_tlb_range(&vma, tlb->start, tlb->end, stride, last_level);	67	__flush_tlb_range(&vma, tlb->start, tlb->end, stride,
		68	last_level, tlb_level);
42	}	69	}
43		70
44	static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,	71	static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,

arch/arm64/include/asm/tlbflush.h+162−15

@@ -10,6 +10,7 @@ · 9 unmodified lines
10		10
11	#ifndef __ASSEMBLY__	11	#ifndef __ASSEMBLY__
12		12
		13	#include <linux/bitfield.h>
13	#include <linux/mm_types.h>	14	#include <linux/mm_types.h>
14	#include <linux/sched.h>	15	#include <linux/sched.h>
15	#include <asm/cputype.h>	16	#include <asm/cputype.h>
@@ -59,6 +60,102 @@ · 43 unmodified lines
59	__ta; \	60	__ta; \
60	})	61	})
61		62
		63	/*
		64	* Get translation granule of the system, which is decided by
		65	* PAGE_SIZE. Used by TTL.
		66	* - 4KB : 1
		67	* - 16KB : 2
		68	* - 64KB : 3
		69	*/
		70	#define TLBI_TTL_TG_4K 1
		71	#define TLBI_TTL_TG_16K 2
		72	#define TLBI_TTL_TG_64K 3
		73
		74	static inline unsigned long get_trans_granule(void)
		75	{
		76	switch (PAGE_SIZE) {
		77	case SZ_4K:
		78	return TLBI_TTL_TG_4K;
		79	case SZ_16K:
		80	return TLBI_TTL_TG_16K;
		81	case SZ_64K:
		82	return TLBI_TTL_TG_64K;
		83	default:
		84	return 0;
		85	}
		86	}
		87
		88	/*
		89	* Level-based TLBI operations.
		90	*
		91	* When ARMv8.4-TTL exists, TLBI operations take an additional hint for
		92	* the level at which the invalidation must take place. If the level is
		93	* wrong, no invalidation may take place. In the case where the level
		94	* cannot be easily determined, a 0 value for the level parameter will
		95	* perform a non-hinted invalidation.
		96	*
		97	* For Stage-2 invalidation, use the level values provided to that effect
		98	* in asm/stage2_pgtable.h.
		99	*/
		100	#define TLBI_TTL_MASK GENMASK_ULL(47, 44)
		101
		102	#define __tlbi_level(op, addr, level) do { \
		103	u64 arg = addr; \
		104	\
		105	if (cpus_have_const_cap(ARM64_HAS_ARMv8_4_TTL) && \
		106	level) { \
		107	u64 ttl = level & 3; \
		108	ttl \|= get_trans_granule() << 2; \
		109	arg &= ~TLBI_TTL_MASK; \
		110	arg \|= FIELD_PREP(TLBI_TTL_MASK, ttl); \
		111	} \
		112	\
		113	__tlbi(op, arg); \
		114	} while(0)
		115
		116	#define __tlbi_user_level(op, arg, level) do { \
		117	if (arm64_kernel_unmapped_at_el0()) \
		118	__tlbi_level(op, (arg \| USER_ASID_FLAG), level); \
		119	} while (0)
		120
		121	/*
		122	* This macro creates a properly formatted VA operand for the TLB RANGE.
		123	* The value bit assignments are:
		124	*
		125	* +----------+------+-------+-------+-------+----------------------+
		126	* \| ASID \| TG \| SCALE \| NUM \| TTL \| BADDR \|
		127	* +-----------------+-------+-------+-------+----------------------+
		128	* \|63 48\|47 46\|45 44\|43 39\|38 37\|36 0\|
		129	*
		130	* The address range is determined by below formula:
		131	* [BADDR, BADDR + (NUM + 1) * 2^(5SCALE + 1) PAGESIZE)
		132	*
		133	*/
		134	#define __TLBI_VADDR_RANGE(addr, asid, scale, num, ttl) \
		135	({ \
		136	unsigned long __ta = (addr) >> PAGE_SHIFT; \
		137	__ta &= GENMASK_ULL(36, 0); \
		138	__ta \|= (unsigned long)(ttl) << 37; \
		139	__ta \|= (unsigned long)(num) << 39; \
		140	__ta \|= (unsigned long)(scale) << 44; \
		141	__ta \|= get_trans_granule() << 46; \
		142	__ta \|= (unsigned long)(asid) << 48; \
		143	__ta; \
		144	})
		145
		146	/* These macros are used by the TLBI RANGE feature. */
		147	#define __TLBI_RANGE_PAGES(num, scale) \
		148	((unsigned long)((num) + 1) << (5 * (scale) + 1))
		149	#define MAX_TLBI_RANGE_PAGES __TLBI_RANGE_PAGES(31, 3)
		150
		151	/*
		152	* Generate 'num' values from -1 to 30 with -1 rejected by the
		153	* __flush_tlb_range() loop below.
		154	*/
		155	#define TLBI_RANGE_MASK GENMASK_ULL(4, 0)
		156	#define __TLBI_RANGE_NUM(pages, scale) \
		157	((((pages) >> (5 * (scale) + 1)) & TLBI_RANGE_MASK) - 1)
		158
62	/*	159	/*
63	* TLB Invalidation	160	* TLB Invalidation
64	* ================	161	* ================
@@ -179,34 +276,83 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, · 114 unmodified lines
179		276
180	static inline void __flush_tlb_range(struct vm_area_struct *vma,	277	static inline void __flush_tlb_range(struct vm_area_struct *vma,
181	unsigned long start, unsigned long end,	278	unsigned long start, unsigned long end,
182	unsigned long stride, bool last_level)	279	unsigned long stride, bool last_level,
		280	int tlb_level)
183	{	281	{
		282	int num = 0;
		283	int scale = 0;
184	unsigned long asid = ASID(vma->vm_mm);	284	unsigned long asid = ASID(vma->vm_mm);
185	unsigned long addr;	285	unsigned long addr;
		286	unsigned long pages;
186		287
187	start = round_down(start, stride);	288	start = round_down(start, stride);
188	end = round_up(end, stride);	289	end = round_up(end, stride);
		290	pages = (end - start) >> PAGE_SHIFT;
189		291
190	if ((end - start) >= (MAX_TLBI_OPS * stride)) {	292	/*
		293	* When not uses TLB range ops, we can handle up to
		294	* (MAX_TLBI_OPS - 1) pages;
		295	* When uses TLB range ops, we can handle up to
		296	* (MAX_TLBI_RANGE_PAGES - 1) pages.
		297	*/
		298	if ((!system_supports_tlb_range() &&
		299	(end - start) >= (MAX_TLBI_OPS * stride)) \|\|
		300	pages >= MAX_TLBI_RANGE_PAGES) {
191	flush_tlb_mm(vma->vm_mm);	301	flush_tlb_mm(vma->vm_mm);
192	return;	302	return;
193	}	303	}
194		304
195	/* Convert the stride into units of 4k */	305	dsb(ishst);
196	stride >>= 12;
197		306
198	start = __TLBI_VADDR(start, asid);	307	/*
199	end = __TLBI_VADDR(end, asid);	308	* When the CPU does not support TLB range operations, flush the TLB
		309	* entries one by one at the granularity of 'stride'. If the the TLB
		310	* range ops are supported, then:
		311	*
		312	* 1. If 'pages' is odd, flush the first page through non-range
		313	* operations;
		314	*
		315	* 2. For remaining pages: the minimum range granularity is decided
		316	* by 'scale', so multiple range TLBI operations may be required.
		317	* Start from scale = 0, flush the corresponding number of pages
		318	* ((num+1)2^(5scale+1) starting from 'addr'), then increase it
		319	* until no pages left.
		320	*
		321	* Note that certain ranges can be represented by either num = 31 and
		322	* scale or num = 0 and scale + 1. The loop below favours the latter
		323	* since num is limited to 30 by the __TLBI_RANGE_NUM() macro.
		324	*/
		325	while (pages > 0) {
		326	if (!system_supports_tlb_range() \|\|
		327	pages % 2 == 1) {
		328	addr = __TLBI_VADDR(start, asid);
		329	if (last_level) {
		330	__tlbi_level(vale1is, addr, tlb_level);
		331	__tlbi_user_level(vale1is, addr, tlb_level);
		332	} else {
		333	__tlbi_level(vae1is, addr, tlb_level);
		334	__tlbi_user_level(vae1is, addr, tlb_level);
		335	}
		336	start += stride;
		337	pages -= stride >> PAGE_SHIFT;
		338	continue;
		339	}
200		340
201	dsb(ishst);	341	num = __TLBI_RANGE_NUM(pages, scale);
202	for (addr = start; addr < end; addr += stride) {	342	if (num >= 0) {
203	if (last_level) {	343	addr = __TLBI_VADDR_RANGE(start, asid, scale,
204	__tlbi(vale1is, addr);	344	num, tlb_level);
205	__tlbi_user(vale1is, addr);	345	if (last_level) {
206	} else {	346	__tlbi(rvale1is, addr);
207	__tlbi(vae1is, addr);	347	__tlbi_user(rvale1is, addr);
208	__tlbi_user(vae1is, addr);	348	} else {
		349	__tlbi(rvae1is, addr);
		350	__tlbi_user(rvae1is, addr);
		351	}
		352	start += __TLBI_RANGE_PAGES(num, scale) << PAGE_SHIFT;
		353	pages -= __TLBI_RANGE_PAGES(num, scale);
209	}	354	}
		355	scale++;
210	}	356	}
211	dsb(ish);	357	dsb(ish);
212	}	358	}
@@ -217,8 +363,9 @@ static inline void flush_tlb_range(struct vm_area_struct *vma, · 4 unmodified lines
217	/*	363	/*
218	* We cannot use leaf-only invalidation here, since we may be invalidating	364	* We cannot use leaf-only invalidation here, since we may be invalidating
219	* table entries as part of collapsing hugepages or moving page tables.	365	* table entries as part of collapsing hugepages or moving page tables.
		366	* Set the tlb_level to 0 because we can not get enough information here.
220	*/	367	*/
221	__flush_tlb_range(vma, start, end, PAGE_SIZE, false);	368	__flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0);
222	}	369	}
223		370
224	static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end)	371	static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end)

arch/arm64/include/asm/uaccess.h+1−0

@@ -19,6 +19,7 @@ · 18 unmodified lines
19	#include <linux/string.h>	19	#include <linux/string.h>
20		20
21	#include <asm/cpufeature.h>	21	#include <asm/cpufeature.h>
		22	#include <asm/mmu.h>
22	#include <asm/ptrace.h>	23	#include <asm/ptrace.h>
23	#include <asm/memory.h>	24	#include <asm/memory.h>
24	#include <asm/extable.h>	25	#include <asm/extable.h>

arch/arm64/include/asm/vdso.h+2−0

@@ -12,6 +12,8 @@ · 11 unmodified lines
12	*/	12	*/
13	#define VDSO_LBASE 0x0	13	#define VDSO_LBASE 0x0
14		14
		15	#define __VVAR_PAGES 2
		16
15	#ifndef __ASSEMBLY__	17	#ifndef __ASSEMBLY__
16		18
17	#include <generated/vdso-offsets.h>	19	#include <generated/vdso-offsets.h>

arch/arm64/include/asm/vdso/compat_gettimeofday.h+13−0

@@ -7,6 +7,7 @@ · 6 unmodified lines
7		7
8	#ifndef __ASSEMBLY__	8	#ifndef __ASSEMBLY__
9		9
		10	#include <asm/barrier.h>
10	#include <asm/unistd.h>	11	#include <asm/unistd.h>
11	#include <asm/errno.h>	12	#include <asm/errno.h>
12		13
@@ -152,6 +153,18 @@ static __always_inline const struct vdso_data *__arch_get_vdso_data(void) · 139 unmodified lines
152	return ret;	153	return ret;
153	}	154	}
154		155
		156	#ifdef CONFIG_TIME_NS
		157	static __always_inline const struct vdso_data *__arch_get_timens_vdso_data(void)
		158	{
		159	const struct vdso_data *ret;
		160
		161	/* See __arch_get_vdso_data(). */
		162	asm volatile("mov %0, %1" : "=r"(ret) : "r"(_timens_data));
		163
		164	return ret;
		165	}
		166	#endif
		167
155	static inline bool vdso_clocksource_ok(const struct vdso_data *vd)	168	static inline bool vdso_clocksource_ok(const struct vdso_data *vd)
156	{	169	{
157	return vd->clock_mode == VDSO_CLOCKMODE_ARCHTIMER;	170	return vd->clock_mode == VDSO_CLOCKMODE_ARCHTIMER;

arch/arm64/include/asm/vdso/gettimeofday.h+9−0

 #ifndef __ASSEMBLY__
+#include <asm/barrier.h>
 #include <asm/unistd.h>
 #define VDSO_HAS_CLOCK_GETRES		1
 	return _vdso_data;
 }
+#ifdef CONFIG_TIME_NS
+static __always_inline
+const struct vdso_data *__arch_get_timens_vdso_data(void)
+{
+	return _timens_data;
+}
+#endif
 #endif /* !__ASSEMBLY__ */
 #endif /* __ASM_VDSO_GETTIMEOFDAY_H */

arch/arm64/include/uapi/asm/hwcap.h+1−0

@@ -74,5 +74,6 @@ · 73 unmodified lines
74	#define HWCAP2_DGH (1 << 15)	74	#define HWCAP2_DGH (1 << 15)
75	#define HWCAP2_RNG (1 << 16)	75	#define HWCAP2_RNG (1 << 16)
76	#define HWCAP2_BTI (1 << 17)	76	#define HWCAP2_BTI (1 << 17)
		77	/* reserved for HWCAP2_MTE (1 << 18) */
77		78
78	#endif /* _UAPI__ASM_HWCAP_H */	79	#endif /* _UAPI__ASM_HWCAP_H */

arch/arm64/include/uapi/asm/sigcontext.h+1−1

@@ -179,7 +179,7 @@ struct sve_context { · 178 unmodified lines
179	* The same convention applies when returning from a signal: a caller	179	* The same convention applies when returning from a signal: a caller
180	* will need to remove or resize the sve_context block if it wants to	180	* will need to remove or resize the sve_context block if it wants to
181	* make the SVE registers live when they were previously non-live or	181	* make the SVE registers live when they were previously non-live or
182	* vice-versa. This may require the the caller to allocate fresh	182	* vice-versa. This may require the caller to allocate fresh
183	* memory and/or move other context blocks in the signal frame.	183	* memory and/or move other context blocks in the signal frame.
184	*	184	*
185	* Changing the vector length during signal return is not permitted:	185	* Changing the vector length during signal return is not permitted:

arch/arm64/kernel/acpi.c+75−0

@@ -261,6 +261,81 @@ pgprot_t __acpi_get_mem_attribute(phys_addr_t addr) · 260 unmodified lines
261	return __pgprot(PROT_DEVICE_nGnRnE);	261	return __pgprot(PROT_DEVICE_nGnRnE);
262	}	262	}
263		263
		264	void __iomem *acpi_os_ioremap(acpi_physical_address phys, acpi_size size)
		265	{
		266	efi_memory_desc_t md, region = NULL;
		267	pgprot_t prot;
		268
		269	if (WARN_ON_ONCE(!efi_enabled(EFI_MEMMAP)))
		270	return NULL;
		271
		272	for_each_efi_memory_desc(md) {
		273	u64 end = md->phys_addr + (md->num_pages << EFI_PAGE_SHIFT);
		274
		275	if (phys < md->phys_addr \|\| phys >= end)
		276	continue;
		277
		278	if (phys + size > end) {
		279	pr_warn(FW_BUG "requested region covers multiple EFI memory regions\n");
		280	return NULL;
		281	}
		282	region = md;
		283	break;
		284	}
		285
		286	/*
		287	* It is fine for AML to remap regions that are not represented in the
		288	* EFI memory map at all, as it only describes normal memory, and MMIO
		289	* regions that require a virtual mapping to make them accessible to
		290	* the EFI runtime services.
		291	*/
		292	prot = __pgprot(PROT_DEVICE_nGnRnE);
		293	if (region) {
		294	switch (region->type) {
		295	case EFI_LOADER_CODE:
		296	case EFI_LOADER_DATA:
		297	case EFI_BOOT_SERVICES_CODE:
		298	case EFI_BOOT_SERVICES_DATA:
		299	case EFI_CONVENTIONAL_MEMORY:
		300	case EFI_PERSISTENT_MEMORY:
		301	pr_warn(FW_BUG "requested region covers kernel memory @ %pa\n", &phys);
		302	return NULL;
		303
		304	case EFI_RUNTIME_SERVICES_CODE:
		305	/*
		306	* This would be unusual, but not problematic per se,
		307	* as long as we take care not to create a writable
		308	* mapping for executable code.
		309	*/
		310	prot = PAGE_KERNEL_RO;
		311	break;
		312
		313	case EFI_ACPI_RECLAIM_MEMORY:
		314	/*
		315	* ACPI reclaim memory is used to pass firmware tables
		316	* and other data that is intended for consumption by
		317	* the OS only, which may decide it wants to reclaim
		318	* that memory and use it for something else. We never
		319	* do that, but we usually add it to the linear map
		320	* anyway, in which case we should use the existing
		321	* mapping.
		322	*/
		323	if (memblock_is_map_memory(phys))
		324	return (void __iomem *)__phys_to_virt(phys);
		325	/* fall through */
		326
		327	default:
		328	if (region->attribute & EFI_MEMORY_WB)
		329	prot = PAGE_KERNEL;
		330	else if (region->attribute & EFI_MEMORY_WT)
		331	prot = __pgprot(PROT_NORMAL_WT);
		332	else if (region->attribute & EFI_MEMORY_WC)
		333	prot = __pgprot(PROT_NORMAL_NC);
		334	}
		335	}
		336	return __ioremap(phys, size, prot);
		337	}
		338
264	/*	339	/*
265	* Claim Synchronous External Aborts as a firmware first notification.	340	* Claim Synchronous External Aborts as a firmware first notification.
266	*	341	*

arch/arm64/kernel/cpufeature.c+106−43

 	static_branch_enable(&arm64_const_caps_ready);
 }
-static int dump_cpu_hwcaps(struct notifier_block *self, unsigned long v, void *p)
+void dump_cpu_features(void)
 {
 	/* file-wide pr_fmt adds "CPU features: " prefix */
 	pr_emerg("0x%*pb\n", ARM64_NCAPS, &cpu_hwcaps);
-	return 0;
 }
-static struct notifier_block cpu_hwcaps_notifier = {
-	.notifier_call = dump_cpu_hwcaps
-};
-static int __init register_cpu_hwcaps_dumper(void)
-{
-	atomic_notifier_chain_register(&panic_notifier_list,
-				       &cpu_hwcaps_notifier);
-	return 0;
-}
-__initcall(register_cpu_hwcaps_dumper);
 DEFINE_STATIC_KEY_ARRAY_FALSE(cpu_hwcap_keys, ARM64_NCAPS);
 EXPORT_SYMBOL(cpu_hwcap_keys);
 };
 static const struct arm64_ftr_bits ftr_id_aa64mmfr0[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_ECV_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_FGT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR0_EXS_SHIFT, 4, 0),
 	/*
 	 * Page size not being supported at Stage-2 is not fatal. You
 	 * just give up KVM if PAGE_SIZE isn't supported there. Go fix
 };
 static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_ETS_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_TWED_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_XNX_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_HIGHER_SAFE, ID_AA64MMFR1_SPECSEI_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_PAN_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_LOR_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR1_HPD_SHIFT, 4, 0),
 static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_E0PD_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_EVT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_BBM_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_TTL_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_FWB_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IDS_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_AT_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_ST_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_NV_SHIFT, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_CCIDX_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LVA_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_IESB_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LSM_SHIFT, 4, 0),
 	 * make use of *minLine.
 	 * If we have differing I-cache policies, report it as the weakest - VIPT.
 	 */
-	ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, 14, 2, ICACHE_POLICY_VIPT),	/* L1Ip */
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, CTR_L1IP_SHIFT, 2, ICACHE_POLICY_VIPT),	/* L1Ip */
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
 };
 static const struct arm64_ftr_bits ftr_id_mmfr0[] = {
-	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 28, 4, 0xf),	/* InnerShr */
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_INNERSHR_SHIFT, 4, 0xf),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0),	/* FCSE */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_FCSE_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, 20, 4, 0),	/* AuxReg */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_MMFR0_AUXREG_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0),	/* TCM */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_TCM_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),	/* ShareLvl */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_SHARELVL_SHIFT, 4, 0),
-	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0xf),	/* OuterShr */
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_OUTERSHR_SHIFT, 4, 0xf),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),	/* PMSA */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_PMSA_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),	/* VMSA */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR0_VMSA_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
 static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
-	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 36, 4, 0),
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_DOUBLELOCK_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64DFR0_PMSVER_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_CTX_CMPS_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64DFR0_WRPS_SHIFT, 4, 0),
 };
 static const struct arm64_ftr_bits ftr_mvfr2[] = {
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),		/* FPMisc */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_FPMISC_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),		/* SIMDMisc */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, MVFR2_SIMDMISC_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
 static const struct arm64_ftr_bits ftr_dczid[] = {
-	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, 4, 1, 1),		/* DZP */
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_EXACT, DCZID_DZP_SHIFT, 1, 1),
-	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),	/* BS */
+	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, DCZID_BS_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_HPDS_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_CNP_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_XNX_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),	/* ac2 */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_MMFR4_AC2_SHIFT, 4, 0),
 	/*
 	 * SpecSEI = 1 indicates that the PE might generate an SError on an
 	 * external abort on speculative read. It is safe to assume that an
 static const struct arm64_ftr_bits ftr_id_pfr0[] = {
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_DIT_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_PFR0_CSV2_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),		/* State3 */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE3_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0),		/* State2 */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE2_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),		/* State1 */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE1_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),		/* State0 */
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_PFR0_STATE0_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
 static const struct arm64_ftr_bits ftr_id_dfr0[] = {
 	/* [31:28] TraceFilt */
-	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 24, 4, 0xf),	/* PerfMon */
+	S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_PERFMON_SHIFT, 4, 0xf),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 20, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MPROFDBG_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 16, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPTRC_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 12, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPTRC_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 8, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_MMAPDBG_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 4, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPSDBG_SHIFT, 4, 0),
-	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0),
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_COPDBG_SHIFT, 4, 0),
 	ARM64_FTR_END,
 };
 static void __init sort_ftr_regs(void)
 {
-	int i;
+	unsigned int i;
+	for (i = 0; i < ARRAY_SIZE(arm64_ftr_regs); i++) {
+		const struct arm64_ftr_reg *ftr_reg = arm64_ftr_regs[i].reg;
+		const struct arm64_ftr_bits *ftr_bits = ftr_reg->ftr_bits;
+		unsigned int j = 0;
+		/*
+		 * Features here must be sorted in descending order with respect
+		 * to their shift values and should not overlap with each other.
+		 */
+		for (; ftr_bits->width != 0; ftr_bits++, j++) {
+			unsigned int width = ftr_reg->ftr_bits[j].width;
+			unsigned int shift = ftr_reg->ftr_bits[j].shift;
+			unsigned int prev_shift;
+			WARN((shift  + width) > 64,
+				"%s has invalid feature at shift %d\n",
+				ftr_reg->name, shift);
+			/*
+			 * Skip the first feature. There is nothing to
+			 * compare against for now.
+			 */
+			if (j == 0)
+				continue;
+			prev_shift = ftr_reg->ftr_bits[j - 1].shift;
+			WARN((shift + width) > prev_shift,
+				"%s has feature overlap at shift %d\n",
+				ftr_reg->name, shift);
+		}
-	/* Check that the array is sorted so that we can do the binary search */
+		/*
-	for (i = 1; i < ARRAY_SIZE(arm64_ftr_regs); i++)
+		 * Skip the first register. There is nothing to
+		 * compare against for now.
+		 */
+		if (i == 0)
+			continue;
+		/*
+		 * Registers here must be sorted in ascending order with respect
+		 * to sys_id for subsequent binary search in get_arm64_ftr_reg()
+		 * to work correctly.
+		 */
 		BUG_ON(arm64_ftr_regs[i].sys_id < arm64_ftr_regs[i - 1].sys_id);
+	}
 }
 /*
 		.matches = has_cpuid_feature,
 		.cpu_enable = cpu_has_fwb,
 	},
+	{
+		.desc = "ARMv8.4 Translation Table Level",
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.capability = ARM64_HAS_ARMv8_4_TTL,
+		.sys_reg = SYS_ID_AA64MMFR2_EL1,
+		.sign = FTR_UNSIGNED,
+		.field_pos = ID_AA64MMFR2_TTL_SHIFT,
+		.min_field_value = 1,
+		.matches = has_cpuid_feature,
+	},
+	{
+		.desc = "TLB range maintenance instructions",
+		.capability = ARM64_HAS_TLB_RANGE,
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.matches = has_cpuid_feature,
+		.sys_reg = SYS_ID_AA64ISAR0_EL1,
+		.field_pos = ID_AA64ISAR0_TLB_SHIFT,
+		.sign = FTR_UNSIGNED,
+		.min_field_value = ID_AA64ISAR0_TLB_RANGE,
+	},
 #ifdef CONFIG_ARM64_HW_AFDBM
 	{
 		/*

arch/arm64/kernel/cpuinfo.c+1−0

@@ -93,6 +93,7 @@ static const char *const hwcap_str[] = { · 92 unmodified lines
93	"dgh",	93	"dgh",
94	"rng",	94	"rng",
95	"bti",	95	"bti",
		96	/* reserved for "mte" */
96	NULL	97	NULL
97	};	98	};
98		99

arch/arm64/kernel/crash_core.c+10−0

 #include <linux/crash_core.h>
 #include <asm/cpufeature.h>
 #include <asm/memory.h>
+#include <asm/pgtable-hwdef.h>
+static inline u64 get_tcr_el1_t1sz(void);
+static inline u64 get_tcr_el1_t1sz(void)
+{
+	return (read_sysreg(tcr_el1) & TCR_T1SZ_MASK) >> TCR_T1SZ_OFFSET;
+}
 void arch_crash_save_vmcoreinfo(void)
 {
 						kimage_voffset);
 	vmcoreinfo_append_str("NUMBER(PHYS_OFFSET)=0x%llx\n",
 						PHYS_OFFSET);
+	vmcoreinfo_append_str("NUMBER(TCR_EL1_T1SZ)=0x%llx\n",
+						get_tcr_el1_t1sz());
 	vmcoreinfo_append_str("KERNELOFFSET=%lx\n", kaslr_offset());
 	vmcoreinfo_append_str("NUMBER(KERNELPACMASK)=0x%llx\n",
 						system_supports_address_auth() ?

arch/arm64/kernel/entry.S+48−48

 #include <asm/assembler.h>
 #include <asm/asm-offsets.h>
 #include <asm/asm_pointer_auth.h>
+#include <asm/bug.h>
 #include <asm/cpufeature.h>
 #include <asm/errno.h>
 #include <asm/esr.h>
 	add	x29, sp, #S_STACKFRAME
 #ifdef CONFIG_ARM64_SW_TTBR0_PAN
-	/*
+alternative_if_not ARM64_HAS_PAN
-	 * Set the TTBR0 PAN bit in SPSR. When the exception is taken from
+	bl	__swpan_entry_el\el
-	 * EL0, there is no need to check the state of TTBR0_EL1 since
-	 * accesses are always enabled.
-	 * Note that the meaning of this bit differs from the ARMv8.1 PAN
-	 * feature as all TTBR0_EL1 accesses are disabled, not just those to
-	 * user mappings.
-	 */
-alternative_if ARM64_HAS_PAN
-	b	1f				// skip TTBR0 PAN
 alternative_else_nop_endif
-	.if	\el != 0
-	mrs	x21, ttbr0_el1
-	tst	x21, #TTBR_ASID_MASK		// Check for the reserved ASID
-	orr	x23, x23, #PSR_PAN_BIT		// Set the emulated PAN in the saved SPSR
-	b.eq	1f				// TTBR0 access already disabled
-	and	x23, x23, #~PSR_PAN_BIT		// Clear the emulated PAN in the saved SPSR
-	.endif
-	__uaccess_ttbr0_disable x21
-1:
 #endif
 	stp	x22, x23, [sp, #S_PC]
 	.endif
 #ifdef CONFIG_ARM64_SW_TTBR0_PAN
-	/*
+alternative_if_not ARM64_HAS_PAN
-	 * Restore access to TTBR0_EL1. If returning to EL0, no need for SPSR
+	bl	__swpan_exit_el\el
-	 * PAN bit checking.
-	 */
-alternative_if ARM64_HAS_PAN
-	b	2f				// skip TTBR0 PAN
 alternative_else_nop_endif
-	.if	\el != 0
-	tbnz	x22, #22, 1f			// Skip re-enabling TTBR0 access if the PSR_PAN_BIT is set
-	.endif
-	__uaccess_ttbr0_enable x0, x1
-	.if	\el == 0
-	/*
-	 * Enable errata workarounds only if returning to user. The only
-	 * workaround currently required for TTBR0_EL1 changes are for the
-	 * Cavium erratum 27456 (broadcast TLBI instructions may cause I-cache
-	 * corruption).
-	 */
-	bl	post_ttbr_update_workaround
-	.endif
-1:
-	.if	\el != 0
-	and	x22, x22, #~PSR_PAN_BIT		// ARMv8.0 CPUs do not understand this bit
-	.endif
-2:
 #endif
 	.if	\el == 0
 	sb
 	.endm
+#ifdef CONFIG_ARM64_SW_TTBR0_PAN
+	/*
+	 * Set the TTBR0 PAN bit in SPSR. When the exception is taken from
+	 * EL0, there is no need to check the state of TTBR0_EL1 since
+	 * accesses are always enabled.
+	 * Note that the meaning of this bit differs from the ARMv8.1 PAN
+	 * feature as all TTBR0_EL1 accesses are disabled, not just those to
+	 * user mappings.
+	 */
+SYM_CODE_START_LOCAL(__swpan_entry_el1)
+	mrs	x21, ttbr0_el1
+	tst	x21, #TTBR_ASID_MASK		// Check for the reserved ASID
+	orr	x23, x23, #PSR_PAN_BIT		// Set the emulated PAN in the saved SPSR
+	b.eq	1f				// TTBR0 access already disabled
+	and	x23, x23, #~PSR_PAN_BIT		// Clear the emulated PAN in the saved SPSR
+SYM_INNER_LABEL(__swpan_entry_el0, SYM_L_LOCAL)
+	__uaccess_ttbr0_disable x21
+1:	ret
+SYM_CODE_END(__swpan_entry_el1)
+	/*
+	 * Restore access to TTBR0_EL1. If returning to EL0, no need for SPSR
+	 * PAN bit checking.
+	 */
+SYM_CODE_START_LOCAL(__swpan_exit_el1)
+	tbnz	x22, #22, 1f			// Skip re-enabling TTBR0 access if the PSR_PAN_BIT is set
+	__uaccess_ttbr0_enable x0, x1
+1:	and	x22, x22, #~PSR_PAN_BIT		// ARMv8.0 CPUs do not understand this bit
+	ret
+SYM_CODE_END(__swpan_exit_el1)
+SYM_CODE_START_LOCAL(__swpan_exit_el0)
+	__uaccess_ttbr0_enable x0, x1
+	/*
+	 * Enable errata workarounds only if returning to user. The only
+	 * workaround currently required for TTBR0_EL1 changes are for the
+	 * Cavium erratum 27456 (broadcast TLBI instructions may cause I-cache
+	 * corruption).
+	 */
+	b	post_ttbr_update_workaround
+SYM_CODE_END(__swpan_exit_el0)
+#endif
 	.macro	irq_stack_entry
 	mov	x19, sp			// preserve the original sp
 #ifdef CONFIG_SHADOW_CALL_STACK

arch/arm64/kernel/module-plts.c+43−3

@@ -253,6 +253,40 @@ static unsigned int count_plts(Elf64_Sym syms, Elf64_Rela rela, int num, · 252 unmodified lines
253	return ret;	253	return ret;
254	}	254	}
255		255
		256	static bool branch_rela_needs_plt(Elf64_Sym syms, Elf64_Rela rela,
		257	Elf64_Word dstidx)
		258	{
		259
		260	Elf64_Sym *s = syms + ELF64_R_SYM(rela->r_info);
		261
		262	if (s->st_shndx == dstidx)
		263	return false;
		264
		265	return ELF64_R_TYPE(rela->r_info) == R_AARCH64_JUMP26 \|\|
		266	ELF64_R_TYPE(rela->r_info) == R_AARCH64_CALL26;
		267	}
		268
		269	/* Group branch PLT relas at the front end of the array. */
		270	static int partition_branch_plt_relas(Elf64_Sym syms, Elf64_Rela rela,
		271	int numrels, Elf64_Word dstidx)
		272	{
		273	int i = 0, j = numrels - 1;
		274
		275	if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE))
		276	return 0;
		277
		278	while (i < j) {
		279	if (branch_rela_needs_plt(syms, &rela[i], dstidx))
		280	i++;
		281	else if (branch_rela_needs_plt(syms, &rela[j], dstidx))
		282	swap(rela[i], rela[j]);
		283	else
		284	j--;
		285	}
		286
		287	return i;
		288	}
		289
256	int module_frob_arch_sections(Elf_Ehdr ehdr, Elf_Shdr sechdrs,	290	int module_frob_arch_sections(Elf_Ehdr ehdr, Elf_Shdr sechdrs,
257	char secstrings, struct module mod)	291	char secstrings, struct module mod)
258	{	292	{
@@ -290,7 +324,7 @@ int module_frob_arch_sections(Elf_Ehdr ehdr, Elf_Shdr sechdrs, · 31 unmodified lines
290		324
291	for (i = 0; i < ehdr->e_shnum; i++) {	325	for (i = 0; i < ehdr->e_shnum; i++) {
292	Elf64_Rela rels = (void )ehdr + sechdrs[i].sh_offset;	326	Elf64_Rela rels = (void )ehdr + sechdrs[i].sh_offset;
293	int numrels = sechdrs[i].sh_size / sizeof(Elf64_Rela);	327	int nents, numrels = sechdrs[i].sh_size / sizeof(Elf64_Rela);
294	Elf64_Shdr *dstsec = sechdrs + sechdrs[i].sh_info;	328	Elf64_Shdr *dstsec = sechdrs + sechdrs[i].sh_info;
295		329
296	if (sechdrs[i].sh_type != SHT_RELA)	330	if (sechdrs[i].sh_type != SHT_RELA)
@@ -300,8 +334,14 @@ int module_frob_arch_sections(Elf_Ehdr ehdr, Elf_Shdr sechdrs, · 3 unmodified lines
300	if (!(dstsec->sh_flags & SHF_EXECINSTR))	334	if (!(dstsec->sh_flags & SHF_EXECINSTR))
301	continue;	335	continue;
302		336
303	/* sort by type, symbol index and addend */	337	/*
304	sort(rels, numrels, sizeof(Elf64_Rela), cmp_rela, NULL);	338	* sort branch relocations requiring a PLT by type, symbol index
		339	* and addend
		340	*/
		341	nents = partition_branch_plt_relas(syms, rels, numrels,
		342	sechdrs[i].sh_info);
		343	if (nents)
		344	sort(rels, nents, sizeof(Elf64_Rela), cmp_rela, NULL);
305		345
306	if (!str_has_prefix(secstrings + dstsec->sh_name, ".init"))	346	if (!str_has_prefix(secstrings + dstsec->sh_name, ".init"))
307	core_plts += count_plts(syms, rels, numrels,	347	core_plts += count_plts(syms, rels, numrels,

arch/arm64/kernel/perf_event.c+70−19

@@ -13,12 +13,15 @@ · 12 unmodified lines
13	#include <asm/sysreg.h>	13	#include <asm/sysreg.h>
14	#include <asm/virt.h>	14	#include <asm/virt.h>
15		15
		16	#include <clocksource/arm_arch_timer.h>
		17
16	#include <linux/acpi.h>	18	#include <linux/acpi.h>
17	#include <linux/clocksource.h>	19	#include <linux/clocksource.h>
18	#include <linux/kvm_host.h>	20	#include <linux/kvm_host.h>
19	#include <linux/of.h>	21	#include <linux/of.h>
20	#include <linux/perf/arm_pmu.h>	22	#include <linux/perf/arm_pmu.h>
21	#include <linux/platform_device.h>	23	#include <linux/platform_device.h>
		24	#include <linux/sched_clock.h>
22	#include <linux/smp.h>	25	#include <linux/smp.h>
23		26
24	/* ARMv8 Cortex-A53 specific event types. */	27	/* ARMv8 Cortex-A53 specific event types. */
@@ -155,7 +158,7 @@ armv8pmu_events_sysfs_show(struct device *dev, · 130 unmodified lines
155		158
156	pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr);	159	pmu_attr = container_of(attr, struct perf_pmu_events_attr, attr);
157		160
158	return sprintf(page, "event=0x%03llx\n", pmu_attr->id);	161	return sprintf(page, "event=0x%04llx\n", pmu_attr->id);
159	}	162	}
160		163
161	#define ARMV8_EVENT_ATTR(name, config) \	164	#define ARMV8_EVENT_ATTR(name, config) \
@@ -222,10 +225,29 @@ static struct attribute *armv8_pmuv3_event_attrs[] = { · 60 unmodified lines
222	ARMV8_EVENT_ATTR(ll_cache_rd, ARMV8_PMUV3_PERFCTR_LL_CACHE_RD),	225	ARMV8_EVENT_ATTR(ll_cache_rd, ARMV8_PMUV3_PERFCTR_LL_CACHE_RD),
223	ARMV8_EVENT_ATTR(ll_cache_miss_rd, ARMV8_PMUV3_PERFCTR_LL_CACHE_MISS_RD),	226	ARMV8_EVENT_ATTR(ll_cache_miss_rd, ARMV8_PMUV3_PERFCTR_LL_CACHE_MISS_RD),
224	ARMV8_EVENT_ATTR(remote_access_rd, ARMV8_PMUV3_PERFCTR_REMOTE_ACCESS_RD),	227	ARMV8_EVENT_ATTR(remote_access_rd, ARMV8_PMUV3_PERFCTR_REMOTE_ACCESS_RD),
		228	ARMV8_EVENT_ATTR(l1d_cache_lmiss_rd, ARMV8_PMUV3_PERFCTR_L1D_CACHE_LMISS_RD),
		229	ARMV8_EVENT_ATTR(op_retired, ARMV8_PMUV3_PERFCTR_OP_RETIRED),
		230	ARMV8_EVENT_ATTR(op_spec, ARMV8_PMUV3_PERFCTR_OP_SPEC),
		231	ARMV8_EVENT_ATTR(stall, ARMV8_PMUV3_PERFCTR_STALL),
		232	ARMV8_EVENT_ATTR(stall_slot_backend, ARMV8_PMUV3_PERFCTR_STALL_SLOT_BACKEND),
		233	ARMV8_EVENT_ATTR(stall_slot_frontend, ARMV8_PMUV3_PERFCTR_STALL_SLOT_FRONTEND),
		234	ARMV8_EVENT_ATTR(stall_slot, ARMV8_PMUV3_PERFCTR_STALL_SLOT),
225	ARMV8_EVENT_ATTR(sample_pop, ARMV8_SPE_PERFCTR_SAMPLE_POP),	235	ARMV8_EVENT_ATTR(sample_pop, ARMV8_SPE_PERFCTR_SAMPLE_POP),
226	ARMV8_EVENT_ATTR(sample_feed, ARMV8_SPE_PERFCTR_SAMPLE_FEED),	236	ARMV8_EVENT_ATTR(sample_feed, ARMV8_SPE_PERFCTR_SAMPLE_FEED),
227	ARMV8_EVENT_ATTR(sample_filtrate, ARMV8_SPE_PERFCTR_SAMPLE_FILTRATE),	237	ARMV8_EVENT_ATTR(sample_filtrate, ARMV8_SPE_PERFCTR_SAMPLE_FILTRATE),
228	ARMV8_EVENT_ATTR(sample_collision, ARMV8_SPE_PERFCTR_SAMPLE_COLLISION),	238	ARMV8_EVENT_ATTR(sample_collision, ARMV8_SPE_PERFCTR_SAMPLE_COLLISION),
		239	ARMV8_EVENT_ATTR(cnt_cycles, ARMV8_AMU_PERFCTR_CNT_CYCLES),
		240	ARMV8_EVENT_ATTR(stall_backend_mem, ARMV8_AMU_PERFCTR_STALL_BACKEND_MEM),
		241	ARMV8_EVENT_ATTR(l1i_cache_lmiss, ARMV8_PMUV3_PERFCTR_L1I_CACHE_LMISS),
		242	ARMV8_EVENT_ATTR(l2d_cache_lmiss_rd, ARMV8_PMUV3_PERFCTR_L2D_CACHE_LMISS_RD),
		243	ARMV8_EVENT_ATTR(l2i_cache_lmiss, ARMV8_PMUV3_PERFCTR_L2I_CACHE_LMISS),
		244	ARMV8_EVENT_ATTR(l3d_cache_lmiss_rd, ARMV8_PMUV3_PERFCTR_L3D_CACHE_LMISS_RD),
		245	ARMV8_EVENT_ATTR(ldst_align_lat, ARMV8_PMUV3_PERFCTR_LDST_ALIGN_LAT),
		246	ARMV8_EVENT_ATTR(ld_align_lat, ARMV8_PMUV3_PERFCTR_LD_ALIGN_LAT),
		247	ARMV8_EVENT_ATTR(st_align_lat, ARMV8_PMUV3_PERFCTR_ST_ALIGN_LAT),
		248	ARMV8_EVENT_ATTR(mem_access_checked, ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED),
		249	ARMV8_EVENT_ATTR(mem_access_checked_rd, ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED_RD),
		250	ARMV8_EVENT_ATTR(mem_access_checked_wr, ARMV8_MTE_PERFCTR_MEM_ACCESS_CHECKED_WR),
229	NULL,	251	NULL,
230	};	252	};
231		253
@@ -244,10 +266,13 @@ armv8pmu_event_attr_is_visible(struct kobject *kobj, · 12 unmodified lines
244	test_bit(pmu_attr->id, cpu_pmu->pmceid_bitmap))	266	test_bit(pmu_attr->id, cpu_pmu->pmceid_bitmap))
245	return attr->mode;	267	return attr->mode;
246		268
247	pmu_attr->id -= ARMV8_PMUV3_EXT_COMMON_EVENT_BASE;	269	if (pmu_attr->id >= ARMV8_PMUV3_EXT_COMMON_EVENT_BASE) {
248	if (pmu_attr->id < ARMV8_PMUV3_MAX_COMMON_EVENTS &&	270	u64 id = pmu_attr->id - ARMV8_PMUV3_EXT_COMMON_EVENT_BASE;
249	test_bit(pmu_attr->id, cpu_pmu->pmceid_ext_bitmap))	271
250	return attr->mode;	272	if (id < ARMV8_PMUV3_MAX_COMMON_EVENTS &&
		273	test_bit(id, cpu_pmu->pmceid_ext_bitmap))
		274	return attr->mode;
		275	}
251		276
252	return 0;	277	return 0;
253	}	278	}
@@ -1165,28 +1190,54 @@ device_initcall(armv8_pmu_driver_init) · 911 unmodified lines
1165	void arch_perf_update_userpage(struct perf_event *event,	1190	void arch_perf_update_userpage(struct perf_event *event,
1166	struct perf_event_mmap_page *userpg, u64 now)	1191	struct perf_event_mmap_page *userpg, u64 now)
1167	{	1192	{
1168	u32 freq;	1193	struct clock_read_data *rd;
1169	u32 shift;	1194	unsigned int seq;
		1195	u64 ns;
1170		1196
1171	/*	1197	userpg->cap_user_time = 0;
1172	* Internal timekeeping for enabled/running/stopped times	1198	userpg->cap_user_time_zero = 0;
1173	* is always computed with the sched_clock.	1199	userpg->cap_user_time_short = 0;
1174	*/	1200
1175	freq = arch_timer_get_rate();	1201	do {
1176	userpg->cap_user_time = 1;	1202	rd = sched_clock_read_begin(&seq);
		1203
		1204	if (rd->read_sched_clock != arch_timer_read_counter)
		1205	return;
		1206
		1207	userpg->time_mult = rd->mult;
		1208	userpg->time_shift = rd->shift;
		1209	userpg->time_zero = rd->epoch_ns;
		1210	userpg->time_cycles = rd->epoch_cyc;
		1211	userpg->time_mask = rd->sched_clock_mask;
		1212
		1213	/*
		1214	* Subtract the cycle base, such that software that
		1215	* doesn't know about cap_user_time_short still 'works'
		1216	* assuming no wraps.
		1217	*/
		1218	ns = mul_u64_u32_shr(rd->epoch_cyc, rd->mult, rd->shift);
		1219	userpg->time_zero -= ns;
		1220
		1221	} while (sched_clock_read_retry(seq));
		1222
		1223	userpg->time_offset = userpg->time_zero - now;
1177		1224
1178	clocks_calc_mult_shift(&userpg->time_mult, &shift, freq,
1179	NSEC_PER_SEC, 0);
1180	/*	1225	/*
1181	* time_shift is not expected to be greater than 31 due to	1226	* time_shift is not expected to be greater than 31 due to
1182	* the original published conversion algorithm shifting a	1227	* the original published conversion algorithm shifting a
1183	* 32-bit value (now specifies a 64-bit value) - refer	1228	* 32-bit value (now specifies a 64-bit value) - refer
1184	* perf_event_mmap_page documentation in perf_event.h.	1229	* perf_event_mmap_page documentation in perf_event.h.
1185	*/	1230	*/
1186	if (shift == 32) {	1231	if (userpg->time_shift == 32) {
1187	shift = 31;	1232	userpg->time_shift = 31;
1188	userpg->time_mult >>= 1;	1233	userpg->time_mult >>= 1;
1189	}	1234	}
1190	userpg->time_shift = (u16)shift;	1235
1191	userpg->time_offset = -now;	1236	/*
		1237	* Internal timekeeping for enabled/running/stopped times
		1238	* is always computed with the sched_clock.
		1239	*/
		1240	userpg->cap_user_time = 1;
		1241	userpg->cap_user_time_zero = 1;
		1242	userpg->cap_user_time_short = 1;
1192	}	1243	}

arch/arm64/kernel/setup.c+14−10

 }
 subsys_initcall(topology_init);
-/*
+static void dump_kernel_offset(void)
- * Dump out kernel offset information on panic.
- */
-static int dump_kernel_offset(struct notifier_block *self, unsigned long v,
-			      void *p)
 {
 	const unsigned long offset = kaslr_offset();
 	} else {
 		pr_emerg("Kernel Offset: disabled\n");
 	}
+}
+static int arm64_panic_block_dump(struct notifier_block *self,
+				  unsigned long v, void *p)
+{
+	dump_kernel_offset();
+	dump_cpu_features();
+	dump_mem_limit();
 	return 0;
 }
-static struct notifier_block kernel_offset_notifier = {
+static struct notifier_block arm64_panic_block = {
-	.notifier_call = dump_kernel_offset
+	.notifier_call = arm64_panic_block_dump
 };
-static int __init register_kernel_offset_dumper(void)
+static int __init register_arm64_panic_block(void)
 {
 	atomic_notifier_chain_register(&panic_notifier_list,
-				       &kernel_offset_notifier);
+				       &arm64_panic_block);
 	return 0;
 }
-__initcall(register_kernel_offset_dumper);
+device_initcall(register_arm64_panic_block);

arch/arm64/kernel/stacktrace.c+1−1

 	put_task_stack(tsk);
 }
-EXPORT_SYMBOL_GPL(save_stack_trace_tsk);
 void save_stack_trace_tsk(struct task_struct *tsk, struct stack_trace *trace)
 {
 	__save_stack_trace(tsk, trace, 1);
 }
+EXPORT_SYMBOL_GPL(save_stack_trace_tsk);
 void save_stack_trace(struct stack_trace *trace)
 {

arch/arm64/kernel/traps.c+1−1

@@ -855,7 +855,7 @@ asmlinkage void handle_bad_stack(struct pt_regs *regs) · 854 unmodified lines
855	pr_emerg("Task stack: [0x%016lx..0x%016lx]\n",	855	pr_emerg("Task stack: [0x%016lx..0x%016lx]\n",
856	tsk_stk, tsk_stk + THREAD_SIZE);	856	tsk_stk, tsk_stk + THREAD_SIZE);
857	pr_emerg("IRQ stack: [0x%016lx..0x%016lx]\n",	857	pr_emerg("IRQ stack: [0x%016lx..0x%016lx]\n",
858	irq_stk, irq_stk + THREAD_SIZE);	858	irq_stk, irq_stk + IRQ_STACK_SIZE);
859	pr_emerg("Overflow stack: [0x%016lx..0x%016lx]\n",	859	pr_emerg("Overflow stack: [0x%016lx..0x%016lx]\n",
860	ovf_stk, ovf_stk + OVERFLOW_STACK_SIZE);	860	ovf_stk, ovf_stk + OVERFLOW_STACK_SIZE);
861		861

arch/arm64/kernel/vdso.c+123−13

@@ -18,6 +18,7 @@ · 17 unmodified lines
18	#include <linux/sched.h>	18	#include <linux/sched.h>
19	#include <linux/signal.h>	19	#include <linux/signal.h>
20	#include <linux/slab.h>	20	#include <linux/slab.h>
		21	#include <linux/time_namespace.h>
21	#include <linux/timekeeper_internal.h>	22	#include <linux/timekeeper_internal.h>
22	#include <linux/vmalloc.h>	23	#include <linux/vmalloc.h>
23	#include <vdso/datapage.h>	24	#include <vdso/datapage.h>
@@ -40,6 +41,12 @@ enum vdso_abi { · 16 unmodified lines
40	#endif /* CONFIG_COMPAT_VDSO */	41	#endif /* CONFIG_COMPAT_VDSO */
41	};	42	};
42		43
		44	enum vvar_pages {
		45	VVAR_DATA_PAGE_OFFSET,
		46	VVAR_TIMENS_PAGE_OFFSET,
		47	VVAR_NR_PAGES,
		48	};
		49
43	struct vdso_abi_info {	50	struct vdso_abi_info {
44	const char *name;	51	const char *name;
45	const char *vdso_code_start;	52	const char *vdso_code_start;
@@ -107,25 +114,122 @@ static int __vdso_init(enum vdso_abi abi) · 61 unmodified lines
107	vdso_info[abi].vdso_code_start) >>	114	vdso_info[abi].vdso_code_start) >>
108	PAGE_SHIFT;	115	PAGE_SHIFT;
109		116
110	/* Allocate the vDSO pagelist, plus a page for the data. */	117	vdso_pagelist = kcalloc(vdso_info[abi].vdso_pages,
111	vdso_pagelist = kcalloc(vdso_info[abi].vdso_pages + 1,
112	sizeof(struct page *),	118	sizeof(struct page *),
113	GFP_KERNEL);	119	GFP_KERNEL);
114	if (vdso_pagelist == NULL)	120	if (vdso_pagelist == NULL)
115	return -ENOMEM;	121	return -ENOMEM;
116		122
117	/* Grab the vDSO data page. */
118	vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
119
120
121	/* Grab the vDSO code pages. */	123	/* Grab the vDSO code pages. */
122	pfn = sym_to_pfn(vdso_info[abi].vdso_code_start);	124	pfn = sym_to_pfn(vdso_info[abi].vdso_code_start);
123		125
124	for (i = 0; i < vdso_info[abi].vdso_pages; i++)	126	for (i = 0; i < vdso_info[abi].vdso_pages; i++)
125	vdso_pagelist[i + 1] = pfn_to_page(pfn + i);	127	vdso_pagelist[i] = pfn_to_page(pfn + i);
		128
		129	vdso_info[abi].cm->pages = vdso_pagelist;
		130
		131	return 0;
		132	}
		133
		134	#ifdef CONFIG_TIME_NS
		135	struct vdso_data arch_get_vdso_data(void vvar_page)
		136	{
		137	return (struct vdso_data *)(vvar_page);
		138	}
		139
		140	/*
		141	* The vvar mapping contains data for a specific time namespace, so when a task
		142	* changes namespace we must unmap its vvar data for the old namespace.
		143	* Subsequent faults will map in data for the new namespace.
		144	*
		145	* For more details see timens_setup_vdso_data().
		146	*/
		147	int vdso_join_timens(struct task_struct task, struct time_namespace ns)
		148	{
		149	struct mm_struct *mm = task->mm;
		150	struct vm_area_struct *vma;
		151
		152	mmap_read_lock(mm);
		153
		154	for (vma = mm->mmap; vma; vma = vma->vm_next) {
		155	unsigned long size = vma->vm_end - vma->vm_start;
126		156
127	vdso_info[abi].dm->pages = &vdso_pagelist[0];	157	if (vma_is_special_mapping(vma, vdso_info[VDSO_ABI_AA64].dm))
128	vdso_info[abi].cm->pages = &vdso_pagelist[1];	158	zap_page_range(vma, vma->vm_start, size);
		159	#ifdef CONFIG_COMPAT_VDSO
		160	if (vma_is_special_mapping(vma, vdso_info[VDSO_ABI_AA32].dm))
		161	zap_page_range(vma, vma->vm_start, size);
		162	#endif
		163	}
		164
		165	mmap_read_unlock(mm);
		166	return 0;
		167	}
		168
		169	static struct page find_timens_vvar_page(struct vm_area_struct vma)
		170	{
		171	if (likely(vma->vm_mm == current->mm))
		172	return current->nsproxy->time_ns->vvar_page;
		173
		174	/*
		175	* VM_PFNMAP \| VM_IO protect .fault() handler from being called
		176	* through interfaces like /proc/$pid/mem or
		177	* process_vm_{readv,writev}() as long as there's no .access()
		178	* in special_mapping_vmops.
		179	* For more details check_vma_flags() and __access_remote_vm()
		180	*/
		181	WARN(1, "vvar_page accessed remotely");
		182
		183	return NULL;
		184	}
		185	#else
		186	static struct page find_timens_vvar_page(struct vm_area_struct vma)
		187	{
		188	return NULL;
		189	}
		190	#endif
		191
		192	static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
		193	struct vm_area_struct vma, struct vm_fault vmf)
		194	{
		195	struct page *timens_page = find_timens_vvar_page(vma);
		196	unsigned long pfn;
		197
		198	switch (vmf->pgoff) {
		199	case VVAR_DATA_PAGE_OFFSET:
		200	if (timens_page)
		201	pfn = page_to_pfn(timens_page);
		202	else
		203	pfn = sym_to_pfn(vdso_data);
		204	break;
		205	#ifdef CONFIG_TIME_NS
		206	case VVAR_TIMENS_PAGE_OFFSET:
		207	/*
		208	* If a task belongs to a time namespace then a namespace
		209	* specific VVAR is mapped with the VVAR_DATA_PAGE_OFFSET and
		210	* the real VVAR page is mapped with the VVAR_TIMENS_PAGE_OFFSET
		211	* offset.
		212	* See also the comment near timens_setup_vdso_data().
		213	*/
		214	if (!timens_page)
		215	return VM_FAULT_SIGBUS;
		216	pfn = sym_to_pfn(vdso_data);
		217	break;
		218	#endif /* CONFIG_TIME_NS */
		219	default:
		220	return VM_FAULT_SIGBUS;
		221	}
		222
		223	return vmf_insert_pfn(vma, vmf->address, pfn);
		224	}
		225
		226	static int vvar_mremap(const struct vm_special_mapping *sm,
		227	struct vm_area_struct *new_vma)
		228	{
		229	unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
		230
		231	if (new_size != VVAR_NR_PAGES * PAGE_SIZE)
		232	return -EINVAL;
129		233
130	return 0;	234	return 0;
131	}	235	}
@@ -139,9 +243,11 @@ static int __setup_additional_pages(enum vdso_abi abi, · 7 unmodified lines
139	unsigned long gp_flags = 0;	243	unsigned long gp_flags = 0;
140	void *ret;	244	void *ret;
141		245
		246	BUILD_BUG_ON(VVAR_NR_PAGES != __VVAR_PAGES);
		247
142	vdso_text_len = vdso_info[abi].vdso_pages << PAGE_SHIFT;	248	vdso_text_len = vdso_info[abi].vdso_pages << PAGE_SHIFT;
143	/* Be sure to map the data page */	249	/* Be sure to map the data page */
144	vdso_mapping_len = vdso_text_len + PAGE_SIZE;	250	vdso_mapping_len = vdso_text_len + VVAR_NR_PAGES * PAGE_SIZE;
145		251
146	vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);	252	vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
147	if (IS_ERR_VALUE(vdso_base)) {	253	if (IS_ERR_VALUE(vdso_base)) {
@@ -149,8 +255,8 @@ static int __setup_additional_pages(enum vdso_abi abi, · 1 unmodified line
149	goto up_fail;	255	goto up_fail;
150	}	256	}
151		257
152	ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,	258	ret = _install_special_mapping(mm, vdso_base, VVAR_NR_PAGES * PAGE_SIZE,
153	VM_READ\|VM_MAYREAD,	259	VM_READ\|VM_MAYREAD\|VM_PFNMAP,
154	vdso_info[abi].dm);	260	vdso_info[abi].dm);
155	if (IS_ERR(ret))	261	if (IS_ERR(ret))
156	goto up_fail;	262	goto up_fail;
@@ -158,7 +264,7 @@ static int __setup_additional_pages(enum vdso_abi abi, · 1 unmodified line
158	if (IS_ENABLED(CONFIG_ARM64_BTI_KERNEL) && system_supports_bti())	264	if (IS_ENABLED(CONFIG_ARM64_BTI_KERNEL) && system_supports_bti())
159	gp_flags = VM_ARM64_BTI;	265	gp_flags = VM_ARM64_BTI;
160		266
161	vdso_base += PAGE_SIZE;	267	vdso_base += VVAR_NR_PAGES * PAGE_SIZE;
162	mm->context.vdso = (void *)vdso_base;	268	mm->context.vdso = (void *)vdso_base;
163	ret = _install_special_mapping(mm, vdso_base, vdso_text_len,	269	ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
164	VM_READ\|VM_EXEC\|gp_flags\|	270	VM_READ\|VM_EXEC\|gp_flags\|
@@ -206,6 +312,8 @@ static struct vm_special_mapping aarch32_vdso_maps[] = { · 41 unmodified lines
206	#ifdef CONFIG_COMPAT_VDSO	312	#ifdef CONFIG_COMPAT_VDSO
207	[AA32_MAP_VVAR] = {	313	[AA32_MAP_VVAR] = {
208	.name = "[vvar]",	314	.name = "[vvar]",
		315	.fault = vvar_fault,
		316	.mremap = vvar_mremap,
209	},	317	},
210	[AA32_MAP_VDSO] = {	318	[AA32_MAP_VDSO] = {
211	.name = "[vdso]",	319	.name = "[vdso]",
@@ -371,6 +479,8 @@ enum aarch64_map { · 159 unmodified lines
371	static struct vm_special_mapping aarch64_vdso_maps[] __ro_after_init = {	479	static struct vm_special_mapping aarch64_vdso_maps[] __ro_after_init = {
372	[AA64_MAP_VVAR] = {	480	[AA64_MAP_VVAR] = {
373	.name = "[vvar]",	481	.name = "[vvar]",
		482	.fault = vvar_fault,
		483	.mremap = vvar_mremap,
374	},	484	},
375	[AA64_MAP_VDSO] = {	485	[AA64_MAP_VDSO] = {
376	.name = "[vdso]",	486	.name = "[vdso]",

arch/arm64/kernel/vdso/vdso.lds.S+4−1

 SECTIONS
 {
-	PROVIDE(_vdso_data = . - PAGE_SIZE);
+	PROVIDE(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE);
+#ifdef CONFIG_TIME_NS
+	PROVIDE(_timens_data = _vdso_data + PAGE_SIZE);
+#endif
 	. = VDSO_LBASE + SIZEOF_HEADERS;
 	.hash		: { *(.hash) }			:text

arch/arm64/kernel/vdso32/vdso.lds.S+4−1

 SECTIONS
 {
-	PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+	PROVIDE_HIDDEN(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE);
+#ifdef CONFIG_TIME_NS
+	PROVIDE_HIDDEN(_timens_data = _vdso_data + PAGE_SIZE);
+#endif
 	. = VDSO_LBASE + SIZEOF_HEADERS;
 	.hash		: { *(.hash) }			:text

arch/arm64/kernel/vmlinux.lds.S+0−1

@@ -10,7 +10,6 @@ · 9 unmodified lines
10	#include <asm-generic/vmlinux.lds.h>	10	#include <asm-generic/vmlinux.lds.h>
11	#include <asm/cache.h>	11	#include <asm/cache.h>
12	#include <asm/kernel-pgtable.h>	12	#include <asm/kernel-pgtable.h>
13	#include <asm/thread_info.h>
14	#include <asm/memory.h>	13	#include <asm/memory.h>
15	#include <asm/page.h>	14	#include <asm/page.h>
16		15

arch/arm64/kvm/hyp-init.S+1−0

@@ -6,6 +6,7 @@ · 5 unmodified lines
6		6
7	#include <linux/linkage.h>	7	#include <linux/linkage.h>
8		8
		9	#include <asm/alternative.h>
9	#include <asm/assembler.h>	10	#include <asm/assembler.h>
10	#include <asm/kvm_arm.h>	11	#include <asm/kvm_arm.h>
11	#include <asm/kvm_mmu.h>	12	#include <asm/kvm_mmu.h>

arch/arm64/kvm/sys_regs.c+34−34

@@ -1024,9 +1024,9 @@ static bool access_amu(struct kvm_vcpu vcpu, struct sys_reg_params p, · 1023 unmodified lines
1024		1024
1025	/* Macro to expand the AMU counter and type registers*/	1025	/* Macro to expand the AMU counter and type registers*/
1026	#define AMU_AMEVCNTR0_EL0(n) { SYS_DESC(SYS_AMEVCNTR0_EL0(n)), access_amu }	1026	#define AMU_AMEVCNTR0_EL0(n) { SYS_DESC(SYS_AMEVCNTR0_EL0(n)), access_amu }
1027	#define AMU_AMEVTYPE0_EL0(n) { SYS_DESC(SYS_AMEVTYPE0_EL0(n)), access_amu }	1027	#define AMU_AMEVTYPER0_EL0(n) { SYS_DESC(SYS_AMEVTYPER0_EL0(n)), access_amu }
1028	#define AMU_AMEVCNTR1_EL0(n) { SYS_DESC(SYS_AMEVCNTR1_EL0(n)), access_amu }	1028	#define AMU_AMEVCNTR1_EL0(n) { SYS_DESC(SYS_AMEVCNTR1_EL0(n)), access_amu }
1029	#define AMU_AMEVTYPE1_EL0(n) { SYS_DESC(SYS_AMEVTYPE1_EL0(n)), access_amu }	1029	#define AMU_AMEVTYPER1_EL0(n) { SYS_DESC(SYS_AMEVTYPER1_EL0(n)), access_amu }
1030		1030
1031	static bool trap_ptrauth(struct kvm_vcpu *vcpu,	1031	static bool trap_ptrauth(struct kvm_vcpu *vcpu,
1032	struct sys_reg_params *p,	1032	struct sys_reg_params *p,
@@ -1629,22 +1629,22 @@ static const struct sys_reg_desc sys_reg_descs[] = { · 596 unmodified lines
1629	AMU_AMEVCNTR0_EL0(13),	1629	AMU_AMEVCNTR0_EL0(13),
1630	AMU_AMEVCNTR0_EL0(14),	1630	AMU_AMEVCNTR0_EL0(14),
1631	AMU_AMEVCNTR0_EL0(15),	1631	AMU_AMEVCNTR0_EL0(15),
1632	AMU_AMEVTYPE0_EL0(0),	1632	AMU_AMEVTYPER0_EL0(0),
1633	AMU_AMEVTYPE0_EL0(1),	1633	AMU_AMEVTYPER0_EL0(1),
1634	AMU_AMEVTYPE0_EL0(2),	1634	AMU_AMEVTYPER0_EL0(2),
1635	AMU_AMEVTYPE0_EL0(3),	1635	AMU_AMEVTYPER0_EL0(3),
1636	AMU_AMEVTYPE0_EL0(4),	1636	AMU_AMEVTYPER0_EL0(4),
1637	AMU_AMEVTYPE0_EL0(5),	1637	AMU_AMEVTYPER0_EL0(5),
1638	AMU_AMEVTYPE0_EL0(6),	1638	AMU_AMEVTYPER0_EL0(6),
1639	AMU_AMEVTYPE0_EL0(7),	1639	AMU_AMEVTYPER0_EL0(7),
1640	AMU_AMEVTYPE0_EL0(8),	1640	AMU_AMEVTYPER0_EL0(8),
1641	AMU_AMEVTYPE0_EL0(9),	1641	AMU_AMEVTYPER0_EL0(9),
1642	AMU_AMEVTYPE0_EL0(10),	1642	AMU_AMEVTYPER0_EL0(10),
1643	AMU_AMEVTYPE0_EL0(11),	1643	AMU_AMEVTYPER0_EL0(11),
1644	AMU_AMEVTYPE0_EL0(12),	1644	AMU_AMEVTYPER0_EL0(12),
1645	AMU_AMEVTYPE0_EL0(13),	1645	AMU_AMEVTYPER0_EL0(13),
1646	AMU_AMEVTYPE0_EL0(14),	1646	AMU_AMEVTYPER0_EL0(14),
1647	AMU_AMEVTYPE0_EL0(15),	1647	AMU_AMEVTYPER0_EL0(15),
1648	AMU_AMEVCNTR1_EL0(0),	1648	AMU_AMEVCNTR1_EL0(0),
1649	AMU_AMEVCNTR1_EL0(1),	1649	AMU_AMEVCNTR1_EL0(1),
1650	AMU_AMEVCNTR1_EL0(2),	1650	AMU_AMEVCNTR1_EL0(2),
@@ -1661,22 +1661,22 @@ static const struct sys_reg_desc sys_reg_descs[] = { · 10 unmodified lines
1661	AMU_AMEVCNTR1_EL0(13),	1661	AMU_AMEVCNTR1_EL0(13),
1662	AMU_AMEVCNTR1_EL0(14),	1662	AMU_AMEVCNTR1_EL0(14),
1663	AMU_AMEVCNTR1_EL0(15),	1663	AMU_AMEVCNTR1_EL0(15),
1664	AMU_AMEVTYPE1_EL0(0),	1664	AMU_AMEVTYPER1_EL0(0),
1665	AMU_AMEVTYPE1_EL0(1),	1665	AMU_AMEVTYPER1_EL0(1),
1666	AMU_AMEVTYPE1_EL0(2),	1666	AMU_AMEVTYPER1_EL0(2),
1667	AMU_AMEVTYPE1_EL0(3),	1667	AMU_AMEVTYPER1_EL0(3),
1668	AMU_AMEVTYPE1_EL0(4),	1668	AMU_AMEVTYPER1_EL0(4),
1669	AMU_AMEVTYPE1_EL0(5),	1669	AMU_AMEVTYPER1_EL0(5),
1670	AMU_AMEVTYPE1_EL0(6),	1670	AMU_AMEVTYPER1_EL0(6),
1671	AMU_AMEVTYPE1_EL0(7),	1671	AMU_AMEVTYPER1_EL0(7),
1672	AMU_AMEVTYPE1_EL0(8),	1672	AMU_AMEVTYPER1_EL0(8),
1673	AMU_AMEVTYPE1_EL0(9),	1673	AMU_AMEVTYPER1_EL0(9),
1674	AMU_AMEVTYPE1_EL0(10),	1674	AMU_AMEVTYPER1_EL0(10),
1675	AMU_AMEVTYPE1_EL0(11),	1675	AMU_AMEVTYPER1_EL0(11),
1676	AMU_AMEVTYPE1_EL0(12),	1676	AMU_AMEVTYPER1_EL0(12),
1677	AMU_AMEVTYPE1_EL0(13),	1677	AMU_AMEVTYPER1_EL0(13),
1678	AMU_AMEVTYPE1_EL0(14),	1678	AMU_AMEVTYPER1_EL0(14),
1679	AMU_AMEVTYPE1_EL0(15),	1679	AMU_AMEVTYPER1_EL0(15),
1680		1680
1681	{ SYS_DESC(SYS_CNTP_TVAL_EL0), access_arch_timer },	1681	{ SYS_DESC(SYS_CNTP_TVAL_EL0), access_arch_timer },
1682	{ SYS_DESC(SYS_CNTP_CTL_EL0), access_arch_timer },	1682	{ SYS_DESC(SYS_CNTP_CTL_EL0), access_arch_timer },

arch/arm64/mm/context.c+6−4

 	return idx2asid(asid) | generation;
 }
-void check_and_switch_context(struct mm_struct *mm, unsigned int cpu)
+void check_and_switch_context(struct mm_struct *mm)
 {
 	unsigned long flags;
+	unsigned int cpu;
 	u64 asid, old_active_asid;
 	if (system_supports_cnp())
 	 *   relaxed xchg in flush_context will treat us as reserved
 	 *   because atomic RmWs are totally ordered for a given location.
 	 */
-	old_active_asid = atomic64_read(&per_cpu(active_asids, cpu));
+	old_active_asid = atomic64_read(this_cpu_ptr(&active_asids));
 	if (old_active_asid && asid_gen_match(asid) &&
-	    atomic64_cmpxchg_relaxed(&per_cpu(active_asids, cpu),
+	    atomic64_cmpxchg_relaxed(this_cpu_ptr(&active_asids),
 				     old_active_asid, asid))
 		goto switch_mm_fastpath;
 		atomic64_set(&mm->context.id, asid);
 	}
+	cpu = smp_processor_id();
 	if (cpumask_test_and_clear_cpu(cpu, &tlb_flush_pending))
 		local_flush_tlb_all();
-	atomic64_set(&per_cpu(active_asids, cpu), asid);
+	atomic64_set(this_cpu_ptr(&active_asids), asid);
 	raw_spin_unlock_irqrestore(&cpu_asid_lock, flags);
 switch_mm_fastpath:

arch/arm64/mm/hugetlbpage.c+40−2

@@ -19,6 +19,44 @@ · 18 unmodified lines
19	#include <asm/tlbflush.h>	19	#include <asm/tlbflush.h>
20	#include <asm/pgalloc.h>	20	#include <asm/pgalloc.h>
21		21
		22	/*
		23	* HugeTLB Support Matrix
		24	*
		25	* ---------------------------------------------------
		26	* \| Page Size \| CONT PTE \| PMD \| CONT PMD \| PUD \|
		27	* ---------------------------------------------------
		28	* \| 4K \| 64K \| 2M \| 32M \| 1G \|
		29	* \| 16K \| 2M \| 32M \| 1G \| \|
		30	* \| 64K \| 2M \| 512M \| 16G \| \|
		31	* ---------------------------------------------------
		32	*/
		33
		34	/*
		35	* Reserve CMA areas for the largest supported gigantic
		36	* huge page when requested. Any other smaller gigantic
		37	* huge pages could still be served from those areas.
		38	*/
		39	#ifdef CONFIG_CMA
		40	void __init arm64_hugetlb_cma_reserve(void)
		41	{
		42	int order;
		43
		44	#ifdef CONFIG_ARM64_4K_PAGES
		45	order = PUD_SHIFT - PAGE_SHIFT;
		46	#else
		47	order = CONT_PMD_SHIFT + PMD_SHIFT - PAGE_SHIFT;
		48	#endif
		49	/*
		50	* HugeTLB CMA reservation is required for gigantic
		51	* huge pages which could not be allocated via the
		52	* page allocator. Just warn if there is any change
		53	* breaking this assumption.
		54	*/
		55	WARN_ON(order <= MAX_ORDER);
		56	hugetlb_cma_reserve(order);
		57	}
		58	#endif /* CONFIG_CMA */
		59
22	#ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION	60	#ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION
23	bool arch_hugetlb_migration_supported(struct hstate *h)	61	bool arch_hugetlb_migration_supported(struct hstate *h)
24	{	62	{
@@ -457,9 +495,9 @@ static int __init hugetlbpage_init(void) · 432 unmodified lines
457	#ifdef CONFIG_ARM64_4K_PAGES	495	#ifdef CONFIG_ARM64_4K_PAGES
458	hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);	496	hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
459	#endif	497	#endif
460	hugetlb_add_hstate((CONT_PMD_SHIFT + PMD_SHIFT) - PAGE_SHIFT);	498	hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
461	hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);	499	hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
462	hugetlb_add_hstate((CONT_PTE_SHIFT + PAGE_SHIFT) - PAGE_SHIFT);	500	hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
463		501
464	return 0;	502	return 0;
465	}	503	}

arch/arm64/mm/init.c+3−19

 	 * initialize node_online_map that gets used in hugetlb_cma_reserve()
 	 * while allocating required CMA size across online nodes.
 	 */
-#ifdef CONFIG_ARM64_4K_PAGES
+#if defined(CONFIG_HUGETLB_PAGE) && defined(CONFIG_CMA)
-	hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT);
+	arm64_hugetlb_cma_reserve();
 #endif
 	/*
 	unmap_kernel_range((u64)__init_begin, (u64)(__init_end - __init_begin));
 }
-/*
+void dump_mem_limit(void)
- * Dump out memory limit information on panic.
- */
-static int dump_mem_limit(struct notifier_block *self, unsigned long v, void *p)
 {
 	if (memory_limit != PHYS_ADDR_MAX) {
 		pr_emerg("Memory Limit: %llu MB\n", memory_limit >> 20);
 	} else {
 		pr_emerg("Memory Limit: none\n");
 	}
-	return 0;
-}
-static struct notifier_block mem_limit_notifier = {
-	.notifier_call = dump_mem_limit,
-};
-static int __init register_mem_limit_dumper(void)
-{
-	atomic_notifier_chain_register(&panic_notifier_list,
-				       &mem_limit_notifier);
-	return 0;
 }
-__initcall(register_mem_limit_dumper);

arch/riscv/include/asm/vdso/gettimeofday.h+1−0

@@ -4,6 +4,7 @@ · 3 unmodified lines
4		4
5	#ifndef __ASSEMBLY__	5	#ifndef __ASSEMBLY__
6		6
		7	#include <asm/barrier.h>
7	#include <asm/unistd.h>	8	#include <asm/unistd.h>
8	#include <asm/csr.h>	9	#include <asm/csr.h>
9	#include <uapi/linux/time.h>	10	#include <uapi/linux/time.h>

drivers/acpi/arm64/iort.c+75−33

 	if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT) {
 		struct acpi_buffer buf = { ACPI_ALLOCATE_BUFFER, NULL };
-		struct acpi_device *adev = to_acpi_device_node(dev->fwnode);
+		struct acpi_device *adev;
 		struct acpi_iort_named_component *ncomp;
+		struct device *nc_dev = dev;
+		/*
+		 * Walk the device tree to find a device with an
+		 * ACPI companion; there is no point in scanning
+		 * IORT for a device matching a named component if
+		 * the device does not have an ACPI companion to
+		 * start with.
+		 */
+		do {
+			adev = ACPI_COMPANION(nc_dev);
+			if (adev)
+				break;
+			nc_dev = nc_dev->parent;
+		} while (nc_dev);
 		if (!adev)
 			goto out;
 		status = acpi_get_name(adev->handle, ACPI_FULL_PATHNAME, &buf);
 		if (ACPI_FAILURE(status)) {
-			dev_warn(dev, "Can't get device full path name\n");
+			dev_warn(nc_dev, "Can't get device full path name\n");
 			goto out;
 		}
 		node = iort_get_iort_node(dev->fwnode);
 		if (node)
 			return node;
 		/*
 		 * if not, then it should be a platform device defined in
 		 * DSDT/SSDT (with Named Component node in IORT)
 				      iort_match_node_callback, dev);
 	}
-	/* Find a PCI root bus */
 	pbus = to_pci_dev(dev)->bus;
-	while (!pci_is_root_bus(pbus))
-		pbus = pbus->parent;
 	return iort_scan_node(ACPI_IORT_NODE_PCI_ROOT_COMPLEX,
 			      iort_match_node_callback, &pbus->dev);
 }
 /**
- * iort_msi_map_rid() - Map a MSI requester ID for a device
+ * iort_msi_map_id() - Map a MSI input ID for a device
  * @dev: The device for which the mapping is to be done.
- * @req_id: The device requester ID.
+ * @input_id: The device input ID.
  *
- * Returns: mapped MSI RID on success, input requester ID otherwise
+ * Returns: mapped MSI ID on success, input ID otherwise
  */
-u32 iort_msi_map_rid(struct device *dev, u32 req_id)
+u32 iort_msi_map_id(struct device *dev, u32 input_id)
 {
 	struct acpi_iort_node *node;
 	u32 dev_id;
 	node = iort_find_dev_node(dev);
 	if (!node)
-		return req_id;
+		return input_id;
-	iort_node_map_id(node, req_id, &dev_id, IORT_MSI_TYPE);
+	iort_node_map_id(node, input_id, &dev_id, IORT_MSI_TYPE);
 	return dev_id;
 }
 /**
  * iort_dev_find_its_id() - Find the ITS identifier for a device
  * @dev: The device.
- * @req_id: Device's requester ID
+ * @id: Device's ID
  * @idx: Index of the ITS identifier list.
  * @its_id: ITS identifier.
  *
  * Returns: 0 on success, appropriate error value otherwise
  */
-static int iort_dev_find_its_id(struct device *dev, u32 req_id,
+static int iort_dev_find_its_id(struct device *dev, u32 id,
 				unsigned int idx, int *its_id)
 {
 	struct acpi_iort_its_group *its;
 	if (!node)
 		return -ENXIO;
-	node = iort_node_map_id(node, req_id, NULL, IORT_MSI_TYPE);
+	node = iort_node_map_id(node, id, NULL, IORT_MSI_TYPE);
 	if (!node)
 		return -ENXIO;
  *
  * Returns: the MSI domain for this device, NULL otherwise
  */
-struct irq_domain *iort_get_device_domain(struct device *dev, u32 req_id)
+struct irq_domain *iort_get_device_domain(struct device *dev, u32 id,
+					  enum irq_domain_bus_token bus_token)
 {
 	struct fwnode_handle *handle;
 	int its_id;
-	if (iort_dev_find_its_id(dev, req_id, 0, &its_id))
+	if (iort_dev_find_its_id(dev, id, 0, &its_id))
 		return NULL;
 	handle = iort_find_domain_token(its_id);
 	if (!handle)
 		return NULL;
-	return irq_find_matching_fwnode(handle, DOMAIN_BUS_PCI_MSI);
+	return irq_find_matching_fwnode(handle, bus_token);
 }
 static void iort_set_device_domain(struct device *dev,
 					   nc->node_flags);
 }
+static int iort_nc_iommu_map(struct device *dev, struct acpi_iort_node *node)
+{
+	struct acpi_iort_node *parent;
+	int err = -ENODEV, i = 0;
+	u32 streamid = 0;
+	do {
+		parent = iort_node_map_platform_id(node, &streamid,
+						   IORT_IOMMU_TYPE,
+						   i++);
+		if (parent)
+			err = iort_iommu_xlate(dev, parent, streamid);
+	} while (parent && !err);
+	return err;
+}
+static int iort_nc_iommu_map_id(struct device *dev,
+				struct acpi_iort_node *node,
+				const u32 *in_id)
+{
+	struct acpi_iort_node *parent;
+	u32 streamid;
+	parent = iort_node_map_id(node, *in_id, &streamid, IORT_IOMMU_TYPE);
+	if (parent)
+		return iort_iommu_xlate(dev, parent, streamid);
+	return -ENODEV;
+}
 /**
- * iort_iommu_configure - Set-up IOMMU configuration for a device.
+ * iort_iommu_configure_id - Set-up IOMMU configuration for a device.
  *
  * @dev: device to configure
+ * @id_in: optional input id const value pointer
  *
  * Returns: iommu_ops pointer on configuration success
  *          NULL on configuration failure
  */
-const struct iommu_ops *iort_iommu_configure(struct device *dev)
+const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
+						const u32 *id_in)
 {
-	struct acpi_iort_node *node, *parent;
+	struct acpi_iort_node *node;
 	const struct iommu_ops *ops;
-	u32 streamid = 0;
 	int err = -ENODEV;
 	/*
 		if (fwspec && iort_pci_rc_supports_ats(node))
 			fwspec->flags |= IOMMU_FWSPEC_PCI_RC_ATS;
 	} else {
-		int i = 0;
 		node = iort_scan_node(ACPI_IORT_NODE_NAMED_COMPONENT,
 				      iort_match_node_callback, dev);
 		if (!node)
 			return NULL;
-		do {
+		err = id_in ? iort_nc_iommu_map_id(dev, node, id_in) :
-			parent = iort_node_map_platform_id(node, &streamid,
+			      iort_nc_iommu_map(dev, node);
-							   IORT_IOMMU_TYPE,
-							   i++);
-			if (parent)
-				err = iort_iommu_xlate(dev, parent, streamid);
-		} while (parent && !err);
 		if (!err)
 			iort_named_component_init(dev, node);
 	return ops;
 }
 #else
 static inline const struct iommu_ops *iort_fwspec_iommu_ops(struct device *dev)
 { return NULL; }
 { return 0; }
 int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 { return 0; }
-const struct iommu_ops *iort_iommu_configure(struct device *dev)
+const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
+						const u32 *input_id)
 { return NULL; }
 #endif

drivers/acpi/scan.c+5−3

  * acpi_dma_configure - Set-up DMA configuration for the device.
  * @dev: The pointer to the device
  * @attr: device dma attributes
+ * @input_id: input device id const value pointer
  */
-int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr)
+int acpi_dma_configure_id(struct device *dev, enum dev_dma_attr attr,
+			  const u32 *input_id)
 {
 	const struct iommu_ops *iommu;
 	u64 dma_addr = 0, size = 0;
 	iort_dma_setup(dev, &dma_addr, &size);
-	iommu = iort_iommu_configure(dev);
+	iommu = iort_iommu_configure_id(dev, input_id);
 	if (PTR_ERR(iommu) == -EPROBE_DEFER)
 		return -EPROBE_DEFER;
 	return 0;
 }
-EXPORT_SYMBOL_GPL(acpi_dma_configure);
+EXPORT_SYMBOL_GPL(acpi_dma_configure_id);
 static void acpi_init_coherency(struct acpi_device *adev)
 {

drivers/bus/fsl-mc/dprc-driver.c+8−23

 	bool mc_io_created = false;
 	bool msi_domain_set = false;
 	u16 major_ver, minor_ver;
+	struct irq_domain *mc_msi_domain;
 	if (!is_fsl_mc_bus_dprc(mc_dev))
 		return -EINVAL;
 			return error;
 		mc_io_created = true;
+	}
-		/*
+	mc_msi_domain = fsl_mc_find_msi_domain(&mc_dev->dev);
-		 * Inherit parent MSI domain:
+	if (!mc_msi_domain) {
-		 */
+		dev_warn(&mc_dev->dev,
-		dev_set_msi_domain(&mc_dev->dev,
+			 "WARNING: MC bus without interrupt support\n");
-				   dev_get_msi_domain(parent_dev));
-		msi_domain_set = true;
 	} else {
-		/*
+		dev_set_msi_domain(&mc_dev->dev, mc_msi_domain);
-		 * This is a root DPRC
+		msi_domain_set = true;
-		 */
-		struct irq_domain *mc_msi_domain;
-		if (dev_is_fsl_mc(parent_dev))
-			return -EINVAL;
-		error = fsl_mc_find_msi_domain(parent_dev,
-					       &mc_msi_domain);
-		if (error < 0) {
-			dev_warn(&mc_dev->dev,
-				 "WARNING: MC bus without interrupt support\n");
-		} else {
-			dev_set_msi_domain(&mc_dev->dev, mc_msi_domain);
-			msi_domain_set = true;
-		}
 	}
 	error = dprc_open(mc_dev->mc_io, 0, mc_dev->obj_desc.id,

drivers/bus/fsl-mc/fsl-mc-bus.c+59−20

 #include <linux/bitops.h>
 #include <linux/msi.h>
 #include <linux/dma-mapping.h>
+#include <linux/acpi.h>
+#include <linux/iommu.h>
 #include "fsl-mc-private.h"
 	struct fsl_mc_device *root_mc_bus_dev;
 	u8 num_translation_ranges;
 	struct fsl_mc_addr_translation_range *translation_ranges;
+	void *fsl_mc_regs;
 };
 /**
 	phys_addr_t start_phys_addr;
 };
+#define FSL_MC_FAPR	0x28
+#define MC_FAPR_PL	BIT(18)
+#define MC_FAPR_BMT	BIT(17)
 /**
  * fsl_mc_bus_match - device to driver matching callback
  * @dev: the fsl-mc device to match against
 static int fsl_mc_dma_configure(struct device *dev)
 {
 	struct device *dma_dev = dev;
+	struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
+	u32 input_id = mc_dev->icid;
 	while (dev_is_fsl_mc(dma_dev))
 		dma_dev = dma_dev->parent;
-	return of_dma_configure(dev, dma_dev->of_node, 0);
+	if (dev_of_node(dma_dev))
+		return of_dma_configure_id(dev, dma_dev->of_node, 0, &input_id);
+	return acpi_dma_configure_id(dev, DEV_DMA_COHERENT, &input_id);
 }
 static ssize_t modalias_show(struct device *dev, struct device_attribute *attr,
 /**
  * fsl_mc_get_root_dprc - function to traverse to the root dprc
  */
-static void fsl_mc_get_root_dprc(struct device *dev,
+void fsl_mc_get_root_dprc(struct device *dev,
-				 struct device **root_dprc_dev)
+			 struct device **root_dprc_dev)
 {
 	if (!dev) {
 		*root_dprc_dev = NULL;
 	struct fsl_mc_io *mc_io = NULL;
 	int container_id;
 	phys_addr_t mc_portal_phys_addr;
-	u32 mc_portal_size;
+	u32 mc_portal_size, mc_stream_id;
-	struct resource res;
+	struct resource *plat_res;
+	if (!iommu_present(&fsl_mc_bus_type))
+		return -EPROBE_DEFER;
 	mc = devm_kzalloc(&pdev->dev, sizeof(*mc), GFP_KERNEL);
 	if (!mc)
 	platform_set_drvdata(pdev, mc);
+	plat_res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+	mc->fsl_mc_regs = devm_ioremap_resource(&pdev->dev, plat_res);
+	if (IS_ERR(mc->fsl_mc_regs))
+		return PTR_ERR(mc->fsl_mc_regs);
+	if (IS_ENABLED(CONFIG_ACPI) && !dev_of_node(&pdev->dev)) {
+		mc_stream_id = readl(mc->fsl_mc_regs + FSL_MC_FAPR);
+		/*
+		 * HW ORs the PL and BMT bit, places the result in bit 15 of
+		 * the StreamID and ORs in the ICID. Calculate it accordingly.
+		 */
+		mc_stream_id = (mc_stream_id & 0xffff) |
+				((mc_stream_id & (MC_FAPR_PL | MC_FAPR_BMT)) ?
+					0x4000 : 0);
+		error = acpi_dma_configure_id(&pdev->dev, DEV_DMA_COHERENT,
+					      &mc_stream_id);
+		if (error)
+			dev_warn(&pdev->dev, "failed to configure dma: %d.\n",
+				 error);
+	}
 	/*
 	 * Get physical address of MC portal for the root DPRC:
 	 */
-	error = of_address_to_resource(pdev->dev.of_node, 0, &res);
+	plat_res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-	if (error < 0) {
+	mc_portal_phys_addr = plat_res->start;
-		dev_err(&pdev->dev,
+	mc_portal_size = resource_size(plat_res);
-			"of_address_to_resource() failed for %pOF\n",
-			pdev->dev.of_node);
-		return error;
-	}
-	mc_portal_phys_addr = res.start;
-	mc_portal_size = resource_size(&res);
 	error = fsl_create_mc_io(&pdev->dev, mc_portal_phys_addr,
 				 mc_portal_size, NULL,
 				 FSL_MC_IO_ATOMIC_CONTEXT_PORTAL, &mc_io);
 	dev_info(&pdev->dev, "MC firmware version: %u.%u.%u\n",
 		 mc_version.major, mc_version.minor, mc_version.revision);
-	error = get_mc_addr_translation_ranges(&pdev->dev,
+	if (dev_of_node(&pdev->dev)) {
-					       &mc->translation_ranges,
+		error = get_mc_addr_translation_ranges(&pdev->dev,
-					       &mc->num_translation_ranges);
+						&mc->translation_ranges,
-	if (error < 0)
+						&mc->num_translation_ranges);
-		goto error_cleanup_mc_io;
+		if (error < 0)
+			goto error_cleanup_mc_io;
+	}
 	error = dprc_get_container_id(mc_io, 0, &container_id);
 	if (error < 0) {
 		goto error_cleanup_mc_io;
 	mc->root_mc_bus_dev = mc_bus_dev;
+	mc_bus_dev->dev.fwnode = pdev->dev.fwnode;
 	return 0;
 error_cleanup_mc_io:
 MODULE_DEVICE_TABLE(of, fsl_mc_bus_match_table);
+static const struct acpi_device_id fsl_mc_bus_acpi_match_table[] = {
+	{"NXP0008", 0 },
+	{ }
+};
+MODULE_DEVICE_TABLE(acpi, fsl_mc_bus_acpi_match_table);
 static struct platform_driver fsl_mc_bus_driver = {
 	.driver = {
 		   .name = "fsl_mc_bus",
 		   .pm = NULL,
 		   .of_match_table = fsl_mc_bus_match_table,
+		   .acpi_match_table = fsl_mc_bus_acpi_match_table,
 		   },
 	.probe = fsl_mc_bus_probe,
 	.remove = fsl_mc_bus_remove,

drivers/bus/fsl-mc/fsl-mc-msi.c+25−11

@@ -13,6 +13,7 @@ · 12 unmodified lines
13	#include <linux/irq.h>	13	#include <linux/irq.h>
14	#include <linux/irqdomain.h>	14	#include <linux/irqdomain.h>
15	#include <linux/msi.h>	15	#include <linux/msi.h>
		16	#include <linux/acpi_iort.h>
16		17
17	#include "fsl-mc-private.h"	18	#include "fsl-mc-private.h"
18		19
@@ -177,23 +178,36 @@ struct irq_domain fsl_mc_msi_create_irq_domain(struct fwnode_handle fwnode, · 158 unmodified lines
177	return domain;	178	return domain;
178	}	179	}
179		180
180	int fsl_mc_find_msi_domain(struct device *mc_platform_dev,	181	struct irq_domain fsl_mc_find_msi_domain(struct device dev)
181	struct irq_domain **mc_msi_domain)
182	{	182	{
		183	struct device *root_dprc_dev;
		184	struct device *bus_dev;
183	struct irq_domain *msi_domain;	185	struct irq_domain *msi_domain;
184	struct device_node *mc_of_node = mc_platform_dev->of_node;	186	struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
185		187
186	msi_domain = of_msi_get_domain(mc_platform_dev, mc_of_node,	188	fsl_mc_get_root_dprc(dev, &root_dprc_dev);
187	DOMAIN_BUS_FSL_MC_MSI);	189	bus_dev = root_dprc_dev->parent;
188	if (!msi_domain) {	190
189	pr_err("Unable to find fsl-mc MSI domain for %pOF\n",	191	if (bus_dev->of_node) {
190	mc_of_node);	192	msi_domain = of_msi_map_get_device_domain(dev,
		193	mc_dev->icid,
		194	DOMAIN_BUS_FSL_MC_MSI);
191		195
192	return -ENOENT;	196	/*
		197	* if the msi-map property is missing assume that all the
		198	* child containers inherit the domain from the parent
		199	*/
		200	if (!msi_domain)
		201
		202	msi_domain = of_msi_get_domain(bus_dev,
		203	bus_dev->of_node,
		204	DOMAIN_BUS_FSL_MC_MSI);
		205	} else {
		206	msi_domain = iort_get_device_domain(dev, mc_dev->icid,
		207	DOMAIN_BUS_FSL_MC_MSI);
193	}	208	}
194		209
195	*mc_msi_domain = msi_domain;	210	return msi_domain;
196	return 0;
197	}	211	}
198		212
199	static void fsl_mc_msi_free_descs(struct device *dev)	213	static void fsl_mc_msi_free_descs(struct device *dev)

drivers/bus/fsl-mc/fsl-mc-private.h+4−2

 void fsl_mc_msi_domain_free_irqs(struct device *dev);
-int fsl_mc_find_msi_domain(struct device *mc_platform_dev,
+struct irq_domain *fsl_mc_find_msi_domain(struct device *dev);
-			   struct irq_domain **mc_msi_domain);
 int fsl_mc_populate_irq_pool(struct fsl_mc_bus *mc_bus,
 			     unsigned int irq_count);
 bool fsl_mc_is_root_dprc(struct device *dev);
+void fsl_mc_get_root_dprc(struct device *dev,
+			 struct device **root_dprc_dev);
 struct fsl_mc_device *fsl_mc_device_lookup(struct fsl_mc_obj_desc *obj_desc,
 					   struct fsl_mc_device *mc_bus_dev);

drivers/iommu/of_iommu.c+44−37

 	return ret;
 }
-struct of_pci_iommu_alias_info {
+static int of_iommu_configure_dev_id(struct device_node *master_np,
-	struct device *dev;
+				     struct device *dev,
-	struct device_node *np;
+				     const u32 *id)
-};
-static int of_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data)
 {
-	struct of_pci_iommu_alias_info *info = data;
 	struct of_phandle_args iommu_spec = { .args_count = 1 };
 	int err;
-	err = of_map_rid(info->np, alias, "iommu-map", "iommu-map-mask",
+	err = of_map_id(master_np, *id, "iommu-map",
-			 &iommu_spec.np, iommu_spec.args);
+			 "iommu-map-mask", &iommu_spec.np,
+			 iommu_spec.args);
 	if (err)
 		return err == -ENODEV ? NO_IOMMU : err;
-	err = of_iommu_xlate(info->dev, &iommu_spec);
+	err = of_iommu_xlate(dev, &iommu_spec);
 	of_node_put(iommu_spec.np);
 	return err;
 }
-static int of_fsl_mc_iommu_init(struct fsl_mc_device *mc_dev,
+static int of_iommu_configure_dev(struct device_node *master_np,
-				struct device_node *master_np)
+				  struct device *dev)
 {
-	struct of_phandle_args iommu_spec = { .args_count = 1 };
+	struct of_phandle_args iommu_spec;
-	int err;
+	int err = NO_IOMMU, idx = 0;
-	err = of_map_rid(master_np, mc_dev->icid, "iommu-map",
+	while (!of_parse_phandle_with_args(master_np, "iommus",
-			 "iommu-map-mask", &iommu_spec.np,
+					   "#iommu-cells",
-			 iommu_spec.args);
+					   idx, &iommu_spec)) {
-	if (err)
+		err = of_iommu_xlate(dev, &iommu_spec);
-		return err == -ENODEV ? NO_IOMMU : err;
+		of_node_put(iommu_spec.np);
+		idx++;
+		if (err)
+			break;
+	}
-	err = of_iommu_xlate(&mc_dev->dev, &iommu_spec);
-	of_node_put(iommu_spec.np);
 	return err;
 }
+struct of_pci_iommu_alias_info {
+	struct device *dev;
+	struct device_node *np;
+};
+static int of_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data)
+{
+	struct of_pci_iommu_alias_info *info = data;
+	u32 input_id = alias;
+	return of_iommu_configure_dev_id(info->np, info->dev, &input_id);
+}
+static int of_iommu_configure_device(struct device_node *master_np,
+				     struct device *dev, const u32 *id)
+{
+	return (id) ? of_iommu_configure_dev_id(master_np, dev, id) :
+		      of_iommu_configure_dev(master_np, dev);
+}
 const struct iommu_ops *of_iommu_configure(struct device *dev,
-					   struct device_node *master_np)
+					   struct device_node *master_np,
+					   const u32 *id)
 {
 	const struct iommu_ops *ops = NULL;
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 		pci_request_acs();
 		err = pci_for_each_dma_alias(to_pci_dev(dev),
 					     of_pci_iommu_init, &info);
-	} else if (dev_is_fsl_mc(dev)) {
-		err = of_fsl_mc_iommu_init(to_fsl_mc_device(dev), master_np);
 	} else {
-		struct of_phandle_args iommu_spec;
+		err = of_iommu_configure_device(master_np, dev, id);
-		int idx = 0;
-		while (!of_parse_phandle_with_args(master_np, "iommus",
-						   "#iommu-cells",
-						   idx, &iommu_spec)) {
-			err = of_iommu_xlate(dev, &iommu_spec);
-			of_node_put(iommu_spec.np);
-			idx++;
-			if (err)
-				break;
-		}
 		fwspec = dev_iommu_fwspec_get(dev);
 		if (!err && fwspec)

drivers/irqchip/irq-gic-v3-its-fsl-mc-msi.c+86−19

@@ -7,6 +7,8 @@ · 6 unmodified lines
7	*	7	*
8	*/	8	*/
9		9
		10	#include <linux/acpi.h>
		11	#include <linux/acpi_iort.h>
10	#include <linux/of_device.h>	12	#include <linux/of_device.h>
11	#include <linux/of_address.h>	13	#include <linux/of_address.h>
12	#include <linux/irq.h>	14	#include <linux/irq.h>
@@ -23,6 +25,19 @@ static struct irq_chip its_msi_irq_chip = { · 10 unmodified lines
23	.irq_set_affinity = msi_domain_set_affinity	25	.irq_set_affinity = msi_domain_set_affinity
24	};	26	};
25		27
		28	static u32 fsl_mc_msi_domain_get_msi_id(struct irq_domain *domain,
		29	struct fsl_mc_device *mc_dev)
		30	{
		31	struct device_node *of_node;
		32	u32 out_id;
		33
		34	of_node = irq_domain_get_of_node(domain);
		35	out_id = of_node ? of_msi_map_id(&mc_dev->dev, of_node, mc_dev->icid) :
		36	iort_msi_map_id(&mc_dev->dev, mc_dev->icid);
		37
		38	return out_id;
		39	}
		40
26	static int its_fsl_mc_msi_prepare(struct irq_domain *msi_domain,	41	static int its_fsl_mc_msi_prepare(struct irq_domain *msi_domain,
27	struct device *dev,	42	struct device *dev,
28	int nvec, msi_alloc_info_t *info)	43	int nvec, msi_alloc_info_t *info)
@@ -43,7 +58,8 @@ static int its_fsl_mc_msi_prepare(struct irq_domain *msi_domain, · 14 unmodified lines
43	* NOTE: This device id corresponds to the IOMMU stream ID	58	* NOTE: This device id corresponds to the IOMMU stream ID
44	* associated with the DPRC object (ICID).	59	* associated with the DPRC object (ICID).
45	*/	60	*/
46	info->scratchpad[0].ul = mc_bus_dev->icid;	61	info->scratchpad[0].ul = fsl_mc_msi_domain_get_msi_id(msi_domain,
		62	mc_bus_dev);
47	msi_info = msi_get_domain_info(msi_domain->parent);	63	msi_info = msi_get_domain_info(msi_domain->parent);
48		64
49	/* Allocate at least 32 MSIs, and always as a power of 2 */	65	/* Allocate at least 32 MSIs, and always as a power of 2 */
@@ -66,12 +82,71 @@ static const struct of_device_id its_device_id[] = { · 16 unmodified lines
66	{},	82	{},
67	};	83	};
68		84
69	static int __init its_fsl_mc_msi_init(void)	85	static void __init its_fsl_mc_msi_init_one(struct fwnode_handle *handle,
		86	const char *name)
70	{	87	{
71	struct device_node *np;
72	struct irq_domain *parent;	88	struct irq_domain *parent;
73	struct irq_domain *mc_msi_domain;	89	struct irq_domain *mc_msi_domain;
74		90
		91	parent = irq_find_matching_fwnode(handle, DOMAIN_BUS_NEXUS);
		92	if (!parent \|\| !msi_get_domain_info(parent)) {
		93	pr_err("%s: unable to locate ITS domain\n", name);
		94	return;
		95	}
		96
		97	mc_msi_domain = fsl_mc_msi_create_irq_domain(handle,
		98	&its_fsl_mc_msi_domain_info,
		99	parent);
		100	if (!mc_msi_domain) {
		101	pr_err("%s: unable to create fsl-mc domain\n", name);
		102	return;
		103	}
		104
		105	pr_info("fsl-mc MSI: %s domain created\n", name);
		106	}
		107
		108	#ifdef CONFIG_ACPI
		109	static int __init
		110	its_fsl_mc_msi_parse_madt(union acpi_subtable_headers *header,
		111	const unsigned long end)
		112	{
		113	struct acpi_madt_generic_translator *its_entry;
		114	struct fwnode_handle *dom_handle;
		115	const char *node_name;
		116	int err = 0;
		117
		118	its_entry = (struct acpi_madt_generic_translator *)header;
		119	node_name = kasprintf(GFP_KERNEL, "ITS@0x%lx",
		120	(long)its_entry->base_address);
		121
		122	dom_handle = iort_find_domain_token(its_entry->translation_id);
		123	if (!dom_handle) {
		124	pr_err("%s: Unable to locate ITS domain handle\n", node_name);
		125	err = -ENXIO;
		126	goto out;
		127	}
		128
		129	its_fsl_mc_msi_init_one(dom_handle, node_name);
		130
		131	out:
		132	kfree(node_name);
		133	return err;
		134	}
		135
		136
		137	static void __init its_fsl_mc_acpi_msi_init(void)
		138	{
		139	acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_TRANSLATOR,
		140	its_fsl_mc_msi_parse_madt, 0);
		141	}
		142	#else
		143	static inline void its_fsl_mc_acpi_msi_init(void) { }
		144	#endif
		145
		146	static void __init its_fsl_mc_of_msi_init(void)
		147	{
		148	struct device_node *np;
		149
75	for (np = of_find_matching_node(NULL, its_device_id); np;	150	for (np = of_find_matching_node(NULL, its_device_id); np;
76	np = of_find_matching_node(np, its_device_id)) {	151	np = of_find_matching_node(np, its_device_id)) {
77	if (!of_device_is_available(np))	152	if (!of_device_is_available(np))
@@ -79,23 +154,15 @@ static int __init its_fsl_mc_msi_init(void) · 1 unmodified line
79	if (!of_property_read_bool(np, "msi-controller"))	154	if (!of_property_read_bool(np, "msi-controller"))
80	continue;	155	continue;
81		156
82	parent = irq_find_matching_host(np, DOMAIN_BUS_NEXUS);	157	its_fsl_mc_msi_init_one(of_node_to_fwnode(np),
83	if (!parent \|\| !msi_get_domain_info(parent)) {	158	np->full_name);
84	pr_err("%pOF: unable to locate ITS domain\n", np);
85	continue;
86	}
87
88	mc_msi_domain = fsl_mc_msi_create_irq_domain(
89	of_node_to_fwnode(np),
90	&its_fsl_mc_msi_domain_info,
91	parent);
92	if (!mc_msi_domain) {
93	pr_err("%pOF: unable to create fsl-mc domain\n", np);
94	continue;
95	}
96
97	pr_info("fsl-mc MSI: %pOF domain created\n", np);
98	}	159	}
		160	}
		161
		162	static int __init its_fsl_mc_msi_init(void)
		163	{
		164	its_fsl_mc_of_msi_init();
		165	its_fsl_mc_acpi_msi_init();
99		166
100	return 0;	167	return 0;
101	}	168	}

drivers/of/base.c+21−21

@@ -2201,15 +2201,15 @@ int of_find_last_cache_level(unsigned int cpu) · 2200 unmodified lines
2201	}	2201	}
2202		2202
2203	/**	2203	/**
2204	* of_map_rid - Translate a requester ID through a downstream mapping.	2204	* of_map_id - Translate an ID through a downstream mapping.
2205	* @np: root complex device node.	2205	* @np: root complex device node.
2206	* @rid: device requester ID to map.	2206	* @id: device ID to map.
2207	* @map_name: property name of the map to use.	2207	* @map_name: property name of the map to use.
2208	* @map_mask_name: optional property name of the mask to use.	2208	* @map_mask_name: optional property name of the mask to use.
2209	* @target: optional pointer to a target device node.	2209	* @target: optional pointer to a target device node.
2210	* @id_out: optional pointer to receive the translated ID.	2210	* @id_out: optional pointer to receive the translated ID.
2211	*	2211	*
2212	* Given a device requester ID, look up the appropriate implementation-defined	2212	* Given a device ID, look up the appropriate implementation-defined
2213	* platform ID and/or the target device which receives transactions on that	2213	* platform ID and/or the target device which receives transactions on that
2214	* ID, as per the "iommu-map" and "msi-map" bindings. Either of @target or	2214	* ID, as per the "iommu-map" and "msi-map" bindings. Either of @target or
2215	* @id_out may be NULL if only the other is required. If @target points to	2215	* @id_out may be NULL if only the other is required. If @target points to
@@ -2219,11 +2219,11 @@ int of_find_last_cache_level(unsigned int cpu) · 3 unmodified lines
2219	*	2219	*
2220	* Return: 0 on success or a standard error code on failure.	2220	* Return: 0 on success or a standard error code on failure.
2221	*/	2221	*/
2222	int of_map_rid(struct device_node *np, u32 rid,	2222	int of_map_id(struct device_node *np, u32 id,
2223	const char map_name, const char map_mask_name,	2223	const char map_name, const char map_mask_name,
2224	struct device_node *target, u32 id_out)	2224	struct device_node *target, u32 id_out)
2225	{	2225	{
2226	u32 map_mask, masked_rid;	2226	u32 map_mask, masked_id;
2227	int map_len;	2227	int map_len;
2228	const __be32 *map = NULL;	2228	const __be32 *map = NULL;
2229		2229
@@ -2235,7 +2235,7 @@ int of_map_rid(struct device_node *np, u32 rid, · 5 unmodified lines
2235	if (target)	2235	if (target)
2236	return -ENODEV;	2236	return -ENODEV;
2237	/* Otherwise, no map implies no translation */	2237	/* Otherwise, no map implies no translation */
2238	*id_out = rid;	2238	*id_out = id;
2239	return 0;	2239	return 0;
2240	}	2240	}
2241		2241
@@ -2255,22 +2255,22 @@ int of_map_rid(struct device_node *np, u32 rid, · 13 unmodified lines
2255	if (map_mask_name)	2255	if (map_mask_name)
2256	of_property_read_u32(np, map_mask_name, &map_mask);	2256	of_property_read_u32(np, map_mask_name, &map_mask);
2257		2257
2258	masked_rid = map_mask & rid;	2258	masked_id = map_mask & id;
2259	for ( ; map_len > 0; map_len -= 4 * sizeof(*map), map += 4) {	2259	for ( ; map_len > 0; map_len -= 4 * sizeof(*map), map += 4) {
2260	struct device_node *phandle_node;	2260	struct device_node *phandle_node;
2261	u32 rid_base = be32_to_cpup(map + 0);	2261	u32 id_base = be32_to_cpup(map + 0);
2262	u32 phandle = be32_to_cpup(map + 1);	2262	u32 phandle = be32_to_cpup(map + 1);
2263	u32 out_base = be32_to_cpup(map + 2);	2263	u32 out_base = be32_to_cpup(map + 2);
2264	u32 rid_len = be32_to_cpup(map + 3);	2264	u32 id_len = be32_to_cpup(map + 3);
2265		2265
2266	if (rid_base & ~map_mask) {	2266	if (id_base & ~map_mask) {
2267	pr_err("%pOF: Invalid %s translation - %s-mask (0x%x) ignores rid-base (0x%x)\n",	2267	pr_err("%pOF: Invalid %s translation - %s-mask (0x%x) ignores id-base (0x%x)\n",
2268	np, map_name, map_name,	2268	np, map_name, map_name,
2269	map_mask, rid_base);	2269	map_mask, id_base);
2270	return -EFAULT;	2270	return -EFAULT;
2271	}	2271	}
2272		2272
2273	if (masked_rid < rid_base \|\| masked_rid >= rid_base + rid_len)	2273	if (masked_id < id_base \|\| masked_id >= id_base + id_len)
2274	continue;	2274	continue;
2275		2275
2276	phandle_node = of_find_node_by_phandle(phandle);	2276	phandle_node = of_find_node_by_phandle(phandle);
@@ -2288,20 +2288,20 @@ int of_map_rid(struct device_node *np, u32 rid, · 11 unmodified lines
2288	}	2288	}
2289		2289
2290	if (id_out)	2290	if (id_out)
2291	*id_out = masked_rid - rid_base + out_base;	2291	*id_out = masked_id - id_base + out_base;
2292		2292
2293	pr_debug("%pOF: %s, using mask %08x, rid-base: %08x, out-base: %08x, length: %08x, rid: %08x -> %08x\n",	2293	pr_debug("%pOF: %s, using mask %08x, id-base: %08x, out-base: %08x, length: %08x, id: %08x -> %08x\n",
2294	np, map_name, map_mask, rid_base, out_base,	2294	np, map_name, map_mask, id_base, out_base,
2295	rid_len, rid, masked_rid - rid_base + out_base);	2295	id_len, id, masked_id - id_base + out_base);
2296	return 0;	2296	return 0;
2297	}	2297	}
2298		2298
2299	pr_info("%pOF: no %s translation for rid 0x%x on %pOF\n", np, map_name,	2299	pr_info("%pOF: no %s translation for id 0x%x on %pOF\n", np, map_name,
2300	rid, target && target ? target : NULL);	2300	id, target && target ? target : NULL);
2301		2301
2302	/* Bypasses translation */	2302	/* Bypasses translation */
2303	if (id_out)	2303	if (id_out)
2304	*id_out = rid;	2304	*id_out = id;
2305	return 0;	2305	return 0;
2306	}	2306	}
2307	EXPORT_SYMBOL_GPL(of_map_rid);	2307	EXPORT_SYMBOL_GPL(of_map_id);

drivers/of/device.c+5−3

  * @np:		Pointer to OF node having DMA configuration
  * @force_dma:  Whether device is to be set up by of_dma_configure() even if
  *		DMA capability is not explicitly described by firmware.
+ * @id:		Optional const pointer value input id
  *
  * Try to get devices's DMA configuration from DT and update it
  * accordingly.
  * can use a platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE events
  * to fix up DMA configuration.
  */
-int of_dma_configure(struct device *dev, struct device_node *np, bool force_dma)
+int of_dma_configure_id(struct device *dev, struct device_node *np,
+			bool force_dma, const u32 *id)
 {
 	u64 dma_addr, paddr, size = 0;
 	int ret;
 	dev_dbg(dev, "device is%sdma coherent\n",
 		coherent ? " " : " not ");
-	iommu = of_iommu_configure(dev, np);
+	iommu = of_iommu_configure(dev, np, id);
 	if (PTR_ERR(iommu) == -EPROBE_DEFER)
 		return -EPROBE_DEFER;
 	return 0;
 }
-EXPORT_SYMBOL_GPL(of_dma_configure);
+EXPORT_SYMBOL_GPL(of_dma_configure_id);
 int of_device_register(struct platform_device *pdev)
 {

drivers/of/irq.c+18−16

 	}
 }
-static u32 __of_msi_map_rid(struct device *dev, struct device_node **np,
+static u32 __of_msi_map_id(struct device *dev, struct device_node **np,
-			    u32 rid_in)
+			    u32 id_in)
 {
 	struct device *parent_dev;
-	u32 rid_out = rid_in;
+	u32 id_out = id_in;
 	/*
 	 * Walk up the device parent links looking for one with a
 	 * "msi-map" property.
 	 */
 	for (parent_dev = dev; parent_dev; parent_dev = parent_dev->parent)
-		if (!of_map_rid(parent_dev->of_node, rid_in, "msi-map",
+		if (!of_map_id(parent_dev->of_node, id_in, "msi-map",
-				"msi-map-mask", np, &rid_out))
+				"msi-map-mask", np, &id_out))
 			break;
-	return rid_out;
+	return id_out;
 }
 /**
- * of_msi_map_rid - Map a MSI requester ID for a device.
+ * of_msi_map_id - Map a MSI ID for a device.
  * @dev: device for which the mapping is to be done.
  * @msi_np: device node of the expected msi controller.
- * @rid_in: unmapped MSI requester ID for the device.
+ * @id_in: unmapped MSI ID for the device.
  *
  * Walk up the device hierarchy looking for devices with a "msi-map"
- * property.  If found, apply the mapping to @rid_in.
+ * property.  If found, apply the mapping to @id_in.
  *
- * Returns the mapped MSI requester ID.
+ * Returns the mapped MSI ID.
  */
-u32 of_msi_map_rid(struct device *dev, struct device_node *msi_np, u32 rid_in)
+u32 of_msi_map_id(struct device *dev, struct device_node *msi_np, u32 id_in)
 {
-	return __of_msi_map_rid(dev, &msi_np, rid_in);
+	return __of_msi_map_id(dev, &msi_np, id_in);
 }
 /**
  * of_msi_map_get_device_domain - Use msi-map to find the relevant MSI domain
  * @dev: device for which the mapping is to be done.
- * @rid: Requester ID for the device.
+ * @id: Device ID.
+ * @bus_token: Bus token
  *
  * Walk up the device hierarchy looking for devices with a "msi-map"
  * property.
  *
  * Returns: the MSI domain for this device (or NULL on failure)
  */
-struct irq_domain *of_msi_map_get_device_domain(struct device *dev, u32 rid)
+struct irq_domain *of_msi_map_get_device_domain(struct device *dev, u32 id,
+						u32 bus_token)
 {
 	struct device_node *np = NULL;
-	__of_msi_map_rid(dev, &np, rid);
+	__of_msi_map_id(dev, &np, id);
-	return irq_find_matching_host(np, DOMAIN_BUS_PCI_MSI);
+	return irq_find_matching_host(np, bus_token);
 }
 /**

drivers/pci/msi.c+5−4

 	pci_for_each_dma_alias(pdev, get_msi_id_cb, &rid);
 	of_node = irq_domain_get_of_node(domain);
-	rid = of_node ? of_msi_map_rid(&pdev->dev, of_node, rid) :
+	rid = of_node ? of_msi_map_id(&pdev->dev, of_node, rid) :
-			iort_msi_map_rid(&pdev->dev, rid);
+			iort_msi_map_id(&pdev->dev, rid);
 	return rid;
 }
 	u32 rid = pci_dev_id(pdev);
 	pci_for_each_dma_alias(pdev, get_msi_id_cb, &rid);
-	dom = of_msi_map_get_device_domain(&pdev->dev, rid);
+	dom = of_msi_map_get_device_domain(&pdev->dev, rid, DOMAIN_BUS_PCI_MSI);
 	if (!dom)
-		dom = iort_get_device_domain(&pdev->dev, rid);
+		dom = iort_get_device_domain(&pdev->dev, rid,
+					     DOMAIN_BUS_PCI_MSI);
 	return dom;
 }
 #endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */

drivers/perf/arm_smmuv3_pmu.c+1−2

@@ -756,8 +756,7 @@ static int smmu_pmu_probe(struct platform_device *pdev) · 755 unmodified lines
756	.capabilities = PERF_PMU_CAP_NO_EXCLUDE,	756	.capabilities = PERF_PMU_CAP_NO_EXCLUDE,
757	};	757	};
758		758
759	res_0 = platform_get_resource(pdev, IORESOURCE_MEM, 0);	759	smmu_pmu->reg_base = devm_platform_get_and_ioremap_resource(pdev, 0, &res_0);
760	smmu_pmu->reg_base = devm_ioremap_resource(dev, res_0);
761	if (IS_ERR(smmu_pmu->reg_base))	760	if (IS_ERR(smmu_pmu->reg_base))
762	return PTR_ERR(smmu_pmu->reg_base);	761	return PTR_ERR(smmu_pmu->reg_base);
763		762

drivers/vhost/vhost.c+0−5

 		return ret;
 	}
 	iov_iter_init(&from, READ, vq->indirect, ret, len);
-	/* We will use the result as an address to read from, so most
-	 * architectures only need a compiler barrier here. */
-	read_barrier_depends();
 	count = len / sizeof desc;
 	/* Buffers are chained via a 16 bit next field, so
 	 * we can have at most 2^16 of these. */

include/acpi/acpi_bus.h+7−2

@@ -588,8 +588,13 @@ bool acpi_dma_supported(struct acpi_device *adev); · 587 unmodified lines
588	enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev);	588	enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev);
589	int acpi_dma_get_range(struct device dev, u64 dma_addr, u64 *offset,	589	int acpi_dma_get_range(struct device dev, u64 dma_addr, u64 *offset,
590	u64 *size);	590	u64 *size);
591	int acpi_dma_configure(struct device *dev, enum dev_dma_attr attr);	591	int acpi_dma_configure_id(struct device *dev, enum dev_dma_attr attr,
592		592	const u32 *input_id);
		593	static inline int acpi_dma_configure(struct device *dev,
		594	enum dev_dma_attr attr)
		595	{
		596	return acpi_dma_configure_id(dev, attr, NULL);
		597	}
593	struct acpi_device acpi_find_child_device(struct acpi_device parent,	598	struct acpi_device acpi_find_child_device(struct acpi_device parent,
594	u64 address, bool check_children);	599	u64 address, bool check_children);
595	int acpi_is_root_bridge(acpi_handle);	600	int acpi_is_root_bridge(acpi_handle);

include/asm-generic/Kbuild+1−0

@@ -45,6 +45,7 @@ mandatory-y += pci.h · 44 unmodified lines
45	mandatory-y += percpu.h	45	mandatory-y += percpu.h
46	mandatory-y += pgalloc.h	46	mandatory-y += pgalloc.h
47	mandatory-y += preempt.h	47	mandatory-y += preempt.h
		48	mandatory-y += rwonce.h
48	mandatory-y += sections.h	49	mandatory-y += sections.h
49	mandatory-y += serial.h	50	mandatory-y += serial.h
50	mandatory-y += shmparam.h	51	mandatory-y += shmparam.h

include/asm-generic/barrier.h+1−18

 #ifndef __ASSEMBLY__
-#include <linux/compiler.h>
+#include <asm/rwonce.h>
 #ifndef nop
 #define nop()	asm volatile ("nop")
 #define dma_wmb()	wmb()
 #endif
-#ifndef read_barrier_depends
-#define read_barrier_depends()		do { } while (0)
-#endif
 #ifndef __smp_mb
 #define __smp_mb()	mb()
 #endif
 #define __smp_wmb()	wmb()
 #endif
-#ifndef __smp_read_barrier_depends
-#define __smp_read_barrier_depends()	read_barrier_depends()
-#endif
 #ifdef CONFIG_SMP
 #ifndef smp_mb
 #define smp_wmb()	__smp_wmb()
 #endif
-#ifndef smp_read_barrier_depends
-#define smp_read_barrier_depends()	__smp_read_barrier_depends()
-#endif
 #else	/* !CONFIG_SMP */
 #ifndef smp_mb
 #define smp_wmb()	barrier()
 #endif
-#ifndef smp_read_barrier_depends
-#define smp_read_barrier_depends()	do { } while (0)
-#endif
 #endif	/* CONFIG_SMP */
 #ifndef __smp_store_mb
 #define virt_mb() __smp_mb()
 #define virt_rmb() __smp_rmb()
 #define virt_wmb() __smp_wmb()
-#define virt_read_barrier_depends() __smp_read_barrier_depends()
 #define virt_store_mb(var, value) __smp_store_mb(var, value)
 #define virt_mb__before_atomic() __smp_mb__before_atomic()
 #define virt_mb__after_atomic()	__smp_mb__after_atomic()

include/asm-generic/rwonce.h+90−0

@@ -0,0 +1,90 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Prevent the compiler from merging or refetching reads or writes. The
	4	* compiler is also forbidden from reordering successive instances of
	5	* READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
	6	* particular ordering. One way to make the compiler aware of ordering is to
	7	* put the two invocations of READ_ONCE or WRITE_ONCE in different C
	8	* statements.
	9	*
	10	* These two macros will also work on aggregate data types like structs or
	11	* unions.
	12	*
	13	* Their two major use cases are: (1) Mediating communication between
	14	* process-level code and irq/NMI handlers, all running on the same CPU,
	15	* and (2) Ensuring that the compiler does not fold, spindle, or otherwise
	16	* mutilate accesses that either do not require ordering or that interact
	17	* with an explicit memory barrier or atomic instruction that provides the
	18	* required ordering.
	19	*/
	20	#ifndef __ASM_GENERIC_RWONCE_H
	21	#define __ASM_GENERIC_RWONCE_H
	22
	23	#ifndef __ASSEMBLY__
	24
	25	#include <linux/compiler_types.h>
	26	#include <linux/kasan-checks.h>
	27	#include <linux/kcsan-checks.h>
	28
	29	/*
	30	* Yes, this permits 64-bit accesses on 32-bit architectures. These will
	31	* actually be atomic in some cases (namely Armv7 + LPAE), but for others we
	32	* rely on the access being split into 2x32-bit accesses for a 32-bit quantity
	33	* (e.g. a virtual address) and a strong prevailing wind.
	34	*/
	35	#define compiletime_assert_rwonce_type(t) \
	36	compiletime_assert(__native_word(t) \|\| sizeof(t) == sizeof(long long), \
	37	"Unsupported access size for {READ,WRITE}_ONCE().")
	38
	39	/*
	40	* Use __READ_ONCE() instead of READ_ONCE() if you do not require any
	41	* atomicity. Note that this may result in tears!
	42	*/
	43	#ifndef __READ_ONCE
	44	#define __READ_ONCE(x) ((const volatile __unqual_scalar_typeof(x) )&(x))
	45	#endif
	46
	47	#define READ_ONCE(x) \
	48	({ \
	49	compiletime_assert_rwonce_type(x); \
	50	__READ_ONCE(x); \
	51	})
	52
	53	#define __WRITE_ONCE(x, val) \
	54	do { \
	55	(volatile typeof(x) )&(x) = (val); \
	56	} while (0)
	57
	58	#define WRITE_ONCE(x, val) \
	59	do { \
	60	compiletime_assert_rwonce_type(x); \
	61	__WRITE_ONCE(x, val); \
	62	} while (0)
	63
	64	static __no_sanitize_or_inline
	65	unsigned long __read_once_word_nocheck(const void *addr)
	66	{
	67	return __READ_ONCE((unsigned long )addr);
	68	}
	69
	70	/*
	71	* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need to load a
	72	* word from memory atomically but without telling KASAN/KCSAN. This is
	73	* usually used by unwinding code when walking the stack of a running process.
	74	*/
	75	#define READ_ONCE_NOCHECK(x) \
	76	({ \
	77	compiletime_assert(sizeof(x) == sizeof(unsigned long), \
	78	"Unsupported access size for READ_ONCE_NOCHECK()."); \
	79	(typeof(x))__read_once_word_nocheck(&(x)); \
	80	})
	81
	82	static __no_kasan_or_inline
	83	unsigned long read_word_at_a_time(const void *addr)
	84	{
	85	kasan_check_read(addr, 1);
	86	return (unsigned long )addr;
	87	}
	88
	89	#endif /* __ASSEMBLY__ */
	90	#endif /* __ASM_GENERIC_RWONCE_H */

include/asm-generic/tlb.h+40−15

 }
 #endif
+/*
+ * tlb_flush_{pte|pmd|pud|p4d}_range() adjust the tlb->start and tlb->end,
+ * and set corresponding cleared_*.
+ */
+static inline void tlb_flush_pte_range(struct mmu_gather *tlb,
+				     unsigned long address, unsigned long size)
+{
+	__tlb_adjust_range(tlb, address, size);
+	tlb->cleared_ptes = 1;
+}
+static inline void tlb_flush_pmd_range(struct mmu_gather *tlb,
+				     unsigned long address, unsigned long size)
+{
+	__tlb_adjust_range(tlb, address, size);
+	tlb->cleared_pmds = 1;
+}
+static inline void tlb_flush_pud_range(struct mmu_gather *tlb,
+				     unsigned long address, unsigned long size)
+{
+	__tlb_adjust_range(tlb, address, size);
+	tlb->cleared_puds = 1;
+}
+static inline void tlb_flush_p4d_range(struct mmu_gather *tlb,
+				     unsigned long address, unsigned long size)
+{
+	__tlb_adjust_range(tlb, address, size);
+	tlb->cleared_p4ds = 1;
+}
 #ifndef __tlb_remove_tlb_entry
 #define __tlb_remove_tlb_entry(tlb, ptep, address) do { } while (0)
 #endif
  */
 #define tlb_remove_tlb_entry(tlb, ptep, address)		\
 	do {							\
-		__tlb_adjust_range(tlb, address, PAGE_SIZE);	\
+		tlb_flush_pte_range(tlb, address, PAGE_SIZE);	\
-		tlb->cleared_ptes = 1;				\
 		__tlb_remove_tlb_entry(tlb, ptep, address);	\
 	} while (0)
 #define tlb_remove_huge_tlb_entry(h, tlb, ptep, address)	\
 	do {							\
 		unsigned long _sz = huge_page_size(h);		\
-		__tlb_adjust_range(tlb, address, _sz);		\
 		if (_sz == PMD_SIZE)				\
-			tlb->cleared_pmds = 1;			\
+			tlb_flush_pmd_range(tlb, address, _sz);	\
 		else if (_sz == PUD_SIZE)			\
-			tlb->cleared_puds = 1;			\
+			tlb_flush_pud_range(tlb, address, _sz);	\
 		__tlb_remove_tlb_entry(tlb, ptep, address);	\
 	} while (0)
 #define tlb_remove_pmd_tlb_entry(tlb, pmdp, address)			\
 	do {								\
-		__tlb_adjust_range(tlb, address, HPAGE_PMD_SIZE);	\
+		tlb_flush_pmd_range(tlb, address, HPAGE_PMD_SIZE);	\
-		tlb->cleared_pmds = 1;					\
 		__tlb_remove_pmd_tlb_entry(tlb, pmdp, address);		\
 	} while (0)
 #define tlb_remove_pud_tlb_entry(tlb, pudp, address)			\
 	do {								\
-		__tlb_adjust_range(tlb, address, HPAGE_PUD_SIZE);	\
+		tlb_flush_pud_range(tlb, address, HPAGE_PUD_SIZE);	\
-		tlb->cleared_puds = 1;					\
 		__tlb_remove_pud_tlb_entry(tlb, pudp, address);		\
 	} while (0)
 #ifndef pte_free_tlb
 #define pte_free_tlb(tlb, ptep, address)			\
 	do {							\
-		__tlb_adjust_range(tlb, address, PAGE_SIZE);	\
+		tlb_flush_pmd_range(tlb, address, PAGE_SIZE);	\
 		tlb->freed_tables = 1;				\
-		tlb->cleared_pmds = 1;				\
 		__pte_free_tlb(tlb, ptep, address);		\
 	} while (0)
 #endif
 #ifndef pmd_free_tlb
 #define pmd_free_tlb(tlb, pmdp, address)			\
 	do {							\
-		__tlb_adjust_range(tlb, address, PAGE_SIZE);	\
+		tlb_flush_pud_range(tlb, address, PAGE_SIZE);	\
 		tlb->freed_tables = 1;				\
-		tlb->cleared_puds = 1;				\
 		__pmd_free_tlb(tlb, pmdp, address);		\
 	} while (0)
 #endif
 #ifndef pud_free_tlb
 #define pud_free_tlb(tlb, pudp, address)			\
 	do {							\
-		__tlb_adjust_range(tlb, address, PAGE_SIZE);	\
+		tlb_flush_p4d_range(tlb, address, PAGE_SIZE);	\
 		tlb->freed_tables = 1;				\
-		tlb->cleared_p4ds = 1;				\
 		__pud_free_tlb(tlb, pudp, address);		\
 	} while (0)
 #endif

include/linux/acpi.h+7−0

@@ -905,6 +905,13 @@ static inline int acpi_dma_configure(struct device *dev, · 904 unmodified lines
905	return 0;	905	return 0;
906	}	906	}
907		907
		908	static inline int acpi_dma_configure_id(struct device *dev,
		909	enum dev_dma_attr attr,
		910	const u32 *input_id)
		911	{
		912	return 0;
		913	}
		914
908	#define ACPI_PTR(_ptr) (NULL)	915	#define ACPI_PTR(_ptr) (NULL)
909		916
910	static inline void acpi_device_set_enumerated(struct acpi_device *adev)	917	static inline void acpi_device_set_enumerated(struct acpi_device *adev)

include/linux/acpi_iort.h+11−9

 struct fwnode_handle *iort_find_domain_token(int trans_id);
 #ifdef CONFIG_ACPI_IORT
 void acpi_iort_init(void);
-u32 iort_msi_map_rid(struct device *dev, u32 req_id);
+u32 iort_msi_map_id(struct device *dev, u32 id);
-struct irq_domain *iort_get_device_domain(struct device *dev, u32 req_id);
+struct irq_domain *iort_get_device_domain(struct device *dev, u32 id,
+					  enum irq_domain_bus_token bus_token);
 void acpi_configure_pmsi_domain(struct device *dev);
 int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id);
 /* IOMMU interface */
 void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *size);
-const struct iommu_ops *iort_iommu_configure(struct device *dev);
+const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
+						const u32 *id_in);
 int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head);
 #else
 static inline void acpi_iort_init(void) { }
-static inline u32 iort_msi_map_rid(struct device *dev, u32 req_id)
+static inline u32 iort_msi_map_id(struct device *dev, u32 id)
-{ return req_id; }
+{ return id; }
-static inline struct irq_domain *iort_get_device_domain(struct device *dev,
+static inline struct irq_domain *iort_get_device_domain(
-							u32 req_id)
+	struct device *dev, u32 id, enum irq_domain_bus_token bus_token)
 { return NULL; }
 static inline void acpi_configure_pmsi_domain(struct device *dev) { }
 /* IOMMU interface */
 static inline void iort_dma_setup(struct device *dev, u64 *dma_addr,
 				  u64 *size) { }
-static inline const struct iommu_ops *iort_iommu_configure(
+static inline const struct iommu_ops *iort_iommu_configure_id(
-				      struct device *dev)
+				      struct device *dev, const u32 *id_in)
 { return NULL; }
 static inline
 int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)

include/linux/arm-smccc.h+22−22

 			   ARM_SMCCC_SMC_32,				\
 			   0, 0x7fff)
+/* Paravirtualised time calls (defined by ARM DEN0057A) */
+#define ARM_SMCCC_HV_PV_TIME_FEATURES				\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,			\
+			   ARM_SMCCC_SMC_64,			\
+			   ARM_SMCCC_OWNER_STANDARD_HYP,	\
+			   0x20)
+#define ARM_SMCCC_HV_PV_TIME_ST					\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,			\
+			   ARM_SMCCC_SMC_64,			\
+			   ARM_SMCCC_OWNER_STANDARD_HYP,	\
+			   0x21)
+/*
+ * Return codes defined in ARM DEN 0070A
+ * ARM DEN 0070A is now merged/consolidated into ARM DEN 0028 C
+ */
+#define SMCCC_RET_SUCCESS			0
+#define SMCCC_RET_NOT_SUPPORTED			-1
+#define SMCCC_RET_NOT_REQUIRED			-2
+#define SMCCC_RET_INVALID_PARAMETER		-3
 #ifndef __ASSEMBLY__
 #include <linux/linkage.h>
  */
 #define arm_smccc_1_1_hvc(...)	__arm_smccc_1_1(SMCCC_HVC_INST, __VA_ARGS__)
-/*
- * Return codes defined in ARM DEN 0070A
- * ARM DEN 0070A is now merged/consolidated into ARM DEN 0028 C
- */
-#define SMCCC_RET_SUCCESS			0
-#define SMCCC_RET_NOT_SUPPORTED			-1
-#define SMCCC_RET_NOT_REQUIRED			-2
-#define SMCCC_RET_INVALID_PARAMETER		-3
 /*
  * Like arm_smccc_1_1* but always returns SMCCC_RET_NOT_SUPPORTED.
  * Used when the SMCCC conduit is not defined. The empty asm statement
 		method;							\
 	})
-/* Paravirtualised time calls (defined by ARM DEN0057A) */
-#define ARM_SMCCC_HV_PV_TIME_FEATURES				\
-	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,			\
-			   ARM_SMCCC_SMC_64,			\
-			   ARM_SMCCC_OWNER_STANDARD_HYP,	\
-			   0x20)
-#define ARM_SMCCC_HV_PV_TIME_ST					\
-	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,			\
-			   ARM_SMCCC_SMC_64,			\
-			   ARM_SMCCC_OWNER_STANDARD_HYP,	\
-			   0x21)
 #endif /*__ASSEMBLY__*/
 #endif /*__LINUX_ARM_SMCCC_H*/

include/linux/compiler.h+2−132

@@ -230,28 +230,6 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, · 229 unmodified lines
230	# define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __LINE__)	230	# define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __LINE__)
231	#endif	231	#endif
232		232
233	/*
234	* Prevent the compiler from merging or refetching reads or writes. The
235	* compiler is also forbidden from reordering successive instances of
236	* READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
237	* particular ordering. One way to make the compiler aware of ordering is to
238	* put the two invocations of READ_ONCE or WRITE_ONCE in different C
239	* statements.
240	*
241	* These two macros will also work on aggregate data types like structs or
242	* unions.
243	*
244	* Their two major use cases are: (1) Mediating communication between
245	* process-level code and irq/NMI handlers, all running on the same CPU,
246	* and (2) Ensuring that the compiler does not fold, spindle, or otherwise
247	* mutilate accesses that either do not require ordering or that interact
248	* with an explicit memory barrier or atomic instruction that provides the
249	* required ordering.
250	*/
251	#include <asm/barrier.h>
252	#include <linux/kasan-checks.h>
253	#include <linux/kcsan-checks.h>
254
255	/**	233	/**
256	* data_race - mark an expression as containing intentional data races	234	* data_race - mark an expression as containing intentional data races
257	*	235	*
@@ -272,65 +250,6 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, · 14 unmodified lines
272	__v; \	250	__v; \
273	})	251	})
274		252
275	/*
276	* Use __READ_ONCE() instead of READ_ONCE() if you do not require any
277	* atomicity or dependency ordering guarantees. Note that this may result
278	* in tears!
279	*/
280	#define __READ_ONCE(x) ((const volatile __unqual_scalar_typeof(x) )&(x))
281
282	#define __READ_ONCE_SCALAR(x) \
283	({ \
284	__unqual_scalar_typeof(x) __x = __READ_ONCE(x); \
285	smp_read_barrier_depends(); \
286	(typeof(x))__x; \
287	})
288
289	#define READ_ONCE(x) \
290	({ \
291	compiletime_assert_rwonce_type(x); \
292	__READ_ONCE_SCALAR(x); \
293	})
294
295	#define __WRITE_ONCE(x, val) \
296	do { \
297	(volatile typeof(x) )&(x) = (val); \
298	} while (0)
299
300	#define WRITE_ONCE(x, val) \
301	do { \
302	compiletime_assert_rwonce_type(x); \
303	__WRITE_ONCE(x, val); \
304	} while (0)
305
306	static __no_sanitize_or_inline
307	unsigned long __read_once_word_nocheck(const void *addr)
308	{
309	return __READ_ONCE((unsigned long )addr);
310	}
311
312	/*
313	* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need to load a
314	* word from memory atomically but without telling KASAN/KCSAN. This is
315	* usually used by unwinding code when walking the stack of a running process.
316	*/
317	#define READ_ONCE_NOCHECK(x) \
318	({ \
319	unsigned long __x; \
320	compiletime_assert(sizeof(x) == sizeof(__x), \
321	"Unsupported access size for READ_ONCE_NOCHECK()."); \
322	__x = __read_once_word_nocheck(&(x)); \
323	smp_read_barrier_depends(); \
324	(typeof(x))__x; \
325	})
326
327	static __no_kasan_or_inline
328	unsigned long read_word_at_a_time(const void *addr)
329	{
330	kasan_check_read(addr, 1);
331	return (unsigned long )addr;
332	}
333
334	#endif /* __KERNEL__ */	253	#endif /* __KERNEL__ */
335		254
336	/*	255	/*
@@ -354,57 +273,6 @@ static inline void offset_to_ptr(const int off) · 17 unmodified lines
354		273
355	#endif /* __ASSEMBLY__ */	274	#endif /* __ASSEMBLY__ */
356		275
357	/* Compile time object size, -1 for unknown */
358	#ifndef __compiletime_object_size
359	# define __compiletime_object_size(obj) -1
360	#endif
361	#ifndef __compiletime_warning
362	# define __compiletime_warning(message)
363	#endif
364	#ifndef __compiletime_error
365	# define __compiletime_error(message)
366	#endif
367
368	#ifdef __OPTIMIZE__
369	# define __compiletime_assert(condition, msg, prefix, suffix) \
370	do { \
371	extern void prefix ## suffix(void) __compiletime_error(msg); \
372	if (!(condition)) \
373	prefix ## suffix(); \
374	} while (0)
375	#else
376	# define __compiletime_assert(condition, msg, prefix, suffix) do { } while (0)
377	#endif
378
379	#define _compiletime_assert(condition, msg, prefix, suffix) \
380	__compiletime_assert(condition, msg, prefix, suffix)
381
382	/**
383	* compiletime_assert - break build and emit msg if condition is false
384	* @condition: a compile-time constant condition to check
385	* @msg: a message to emit if condition is false
386	*
387	* In tradition of POSIX assert, this macro will break the build if the
388	* supplied condition is false, emitting the supplied error message if the
389	* compiler has support to do so.
390	*/
391	#define compiletime_assert(condition, msg) \
392	_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
393
394	#define compiletime_assert_atomic_type(t) \
395	compiletime_assert(__native_word(t), \
396	"Need native word sized stores/loads for atomicity.")
397
398	/*
399	* Yes, this permits 64-bit accesses on 32-bit architectures. These will
400	* actually be atomic in some cases (namely Armv7 + LPAE), but for others we
401	* rely on the access being split into 2x32-bit accesses for a 32-bit quantity
402	* (e.g. a virtual address) and a strong prevailing wind.
403	*/
404	#define compiletime_assert_rwonce_type(t) \
405	compiletime_assert(__native_word(t) \|\| sizeof(t) == sizeof(long long), \
406	"Unsupported access size for {READ,WRITE}_ONCE().")
407
408	/* &a[0] degrades to a pointer: a different type from an array */	276	/* &a[0] degrades to a pointer: a different type from an array */
409	#define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))	277	#define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))
410		278
@@ -414,4 +282,6 @@ static inline void offset_to_ptr(const int off) · 3 unmodified lines
414	*/	282	*/
415	#define prevent_tail_call_optimization() mb()	283	#define prevent_tail_call_optimization() mb()
416		284
		285	#include <asm/rwonce.h>
		286
417	#endif /* __LINUX_COMPILER_H */	287	#endif /* __LINUX_COMPILER_H */

include/linux/compiler_types.h+41−0

@@ -275,6 +275,47 @@ struct ftrace_likely_data { · 274 unmodified lines
275	(sizeof(t) == sizeof(char) \|\| sizeof(t) == sizeof(short) \|\| \	275	(sizeof(t) == sizeof(char) \|\| sizeof(t) == sizeof(short) \|\| \
276	sizeof(t) == sizeof(int) \|\| sizeof(t) == sizeof(long))	276	sizeof(t) == sizeof(int) \|\| sizeof(t) == sizeof(long))
277		277
		278	/* Compile time object size, -1 for unknown */
		279	#ifndef __compiletime_object_size
		280	# define __compiletime_object_size(obj) -1
		281	#endif
		282	#ifndef __compiletime_warning
		283	# define __compiletime_warning(message)
		284	#endif
		285	#ifndef __compiletime_error
		286	# define __compiletime_error(message)
		287	#endif
		288
		289	#ifdef __OPTIMIZE__
		290	# define __compiletime_assert(condition, msg, prefix, suffix) \
		291	do { \
		292	extern void prefix ## suffix(void) __compiletime_error(msg); \
		293	if (!(condition)) \
		294	prefix ## suffix(); \
		295	} while (0)
		296	#else
		297	# define __compiletime_assert(condition, msg, prefix, suffix) do { } while (0)
		298	#endif
		299
		300	#define _compiletime_assert(condition, msg, prefix, suffix) \
		301	__compiletime_assert(condition, msg, prefix, suffix)
		302
		303	/**
		304	* compiletime_assert - break build and emit msg if condition is false
		305	* @condition: a compile-time constant condition to check
		306	* @msg: a message to emit if condition is false
		307	*
		308	* In tradition of POSIX assert, this macro will break the build if the
		309	* supplied condition is false, emitting the supplied error message if the
		310	* compiler has support to do so.
		311	*/
		312	#define compiletime_assert(condition, msg) \
		313	_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
		314
		315	#define compiletime_assert_atomic_type(t) \
		316	compiletime_assert(__native_word(t), \
		317	"Need native word sized stores/loads for atomicity.")
		318
278	/* Helpers for emitting diagnostics in pragmas. */	319	/* Helpers for emitting diagnostics in pragmas. */
279	#ifndef __diag	320	#ifndef __diag
280	#define __diag(string)	321	#define __diag(string)

include/linux/nospec.h+2−0

@@ -5,6 +5,8 @@ · 4 unmodified lines
5		5
6	#ifndef _LINUX_NOSPEC_H	6	#ifndef _LINUX_NOSPEC_H
7	#define _LINUX_NOSPEC_H	7	#define _LINUX_NOSPEC_H
		8
		9	#include <linux/compiler.h>
8	#include <asm/barrier.h>	10	#include <asm/barrier.h>
9		11
10	struct task_struct;	12	struct task_struct;

include/linux/of.h+2−2

@@ -554,7 +554,7 @@ bool of_console_check(struct device_node dn, char name, int index); · 553 unmodified lines
554		554
555	extern int of_cpu_node_to_id(struct device_node *np);	555	extern int of_cpu_node_to_id(struct device_node *np);
556		556
557	int of_map_rid(struct device_node *np, u32 rid,	557	int of_map_id(struct device_node *np, u32 id,
558	const char map_name, const char map_mask_name,	558	const char map_name, const char map_mask_name,
559	struct device_node *target, u32 id_out);	559	struct device_node *target, u32 id_out);
560		560
@@ -978,7 +978,7 @@ static inline int of_cpu_node_to_id(struct device_node *np) · 417 unmodified lines
978	return -ENODEV;	978	return -ENODEV;
979	}	979	}
980		980
981	static inline int of_map_rid(struct device_node *np, u32 rid,	981	static inline int of_map_id(struct device_node *np, u32 id,
982	const char map_name, const char map_mask_name,	982	const char map_name, const char map_mask_name,
983	struct device_node *target, u32 id_out)	983	struct device_node *target, u32 id_out)
984	{	984	{

include/linux/of_device.h+14−2

 	return of_node_get(cpu_dev->of_node);
 }
-int of_dma_configure(struct device *dev,
+int of_dma_configure_id(struct device *dev,
 		     struct device_node *np,
-		     bool force_dma);
+		     bool force_dma, const u32 *id);
+static inline int of_dma_configure(struct device *dev,
+				   struct device_node *np,
+				   bool force_dma)
+{
+	return of_dma_configure_id(dev, np, force_dma, NULL);
+}
 #else /* CONFIG_OF */
 static inline int of_driver_match_device(struct device *dev,
 	return NULL;
 }
+static inline int of_dma_configure_id(struct device *dev,
+				   struct device_node *np,
+				   bool force_dma)
+{
+	return 0;
+}
 static inline int of_dma_configure(struct device *dev,
 				   struct device_node *np,
 				   bool force_dma)

include/linux/of_iommu.h+4−2

 			     size_t *size);
 extern const struct iommu_ops *of_iommu_configure(struct device *dev,
-					struct device_node *master_np);
+					struct device_node *master_np,
+					const u32 *id);
 #else
 }
 static inline const struct iommu_ops *of_iommu_configure(struct device *dev,
-					 struct device_node *master_np)
+					 struct device_node *master_np,
+					 const u32 *id)
 {
 	return NULL;
 }

include/linux/of_irq.h+7−6

 					    struct device_node *np,
 					    enum irq_domain_bus_token token);
 extern struct irq_domain *of_msi_map_get_device_domain(struct device *dev,
-						       u32 rid);
+							u32 id,
+							u32 bus_token);
 extern void of_msi_configure(struct device *dev, struct device_node *np);
-u32 of_msi_map_rid(struct device *dev, struct device_node *msi_np, u32 rid_in);
+u32 of_msi_map_id(struct device *dev, struct device_node *msi_np, u32 id_in);
 #else
 static inline int of_irq_count(struct device_node *dev)
 {
 	return NULL;
 }
 static inline struct irq_domain *of_msi_map_get_device_domain(struct device *dev,
-							      u32 rid)
+						u32 id, u32 bus_token)
 {
 	return NULL;
 }
 static inline void of_msi_configure(struct device *dev, struct device_node *np)
 {
 }
-static inline u32 of_msi_map_rid(struct device *dev,
+static inline u32 of_msi_map_id(struct device *dev,
-				 struct device_node *msi_np, u32 rid_in)
+				 struct device_node *msi_np, u32 id_in)
 {
-	return rid_in;
+	return id_in;
 }
 #endif

include/linux/percpu-refcount.h+1−1

@@ -155,7 +155,7 @@ static inline bool __ref_is_percpu(struct percpu_ref *ref, · 154 unmodified lines
155	* between contaminating the pointer value, meaning that	155	* between contaminating the pointer value, meaning that
156	* READ_ONCE() is required when fetching it.	156	* READ_ONCE() is required when fetching it.
157	*	157	*
158	* The smp_read_barrier_depends() implied by READ_ONCE() pairs	158	* The dependency ordering from the READ_ONCE() pairs
159	* with smp_store_release() in __percpu_ref_switch_to_percpu().	159	* with smp_store_release() in __percpu_ref_switch_to_percpu().
160	*/	160	*/
161	percpu_ptr = READ_ONCE(ref->percpu_count_ptr);	161	percpu_ptr = READ_ONCE(ref->percpu_count_ptr);

include/linux/ptr_ring.h+1−1

@@ -107,7 +107,7 @@ static inline int __ptr_ring_produce(struct ptr_ring r, void ptr) · 106 unmodified lines
107	return -ENOSPC;	107	return -ENOSPC;
108		108
109	/* Make sure the pointer we are storing points to a valid data. */	109	/* Make sure the pointer we are storing points to a valid data. */
110	/* Pairs with smp_read_barrier_depends in __ptr_ring_consume. */	110	/* Pairs with the dependency ordering in __ptr_ring_consume. */
111	smp_wmb();	111	smp_wmb();
112		112
113	WRITE_ONCE(r->queue[r->producer++], ptr);	113	WRITE_ONCE(r->queue[r->producer++], ptr);

include/linux/sched_clock.h+28−0

@@ -6,6 +6,34 @@ · 5 unmodified lines
6	#define LINUX_SCHED_CLOCK	6	#define LINUX_SCHED_CLOCK
7		7
8	#ifdef CONFIG_GENERIC_SCHED_CLOCK	8	#ifdef CONFIG_GENERIC_SCHED_CLOCK
		9	/**
		10	* struct clock_read_data - data required to read from sched_clock()
		11	*
		12	* @epoch_ns: sched_clock() value at last update
		13	* @epoch_cyc: Clock cycle value at last update.
		14	* @sched_clock_mask: Bitmask for two's complement subtraction of non 64bit
		15	* clocks.
		16	* @read_sched_clock: Current clock source (or dummy source when suspended).
		17	* @mult: Multipler for scaled math conversion.
		18	* @shift: Shift value for scaled math conversion.
		19	*
		20	* Care must be taken when updating this structure; it is read by
		21	* some very hot code paths. It occupies <=40 bytes and, when combined
		22	* with the seqcount used to synchronize access, comfortably fits into
		23	* a 64 byte cache line.
		24	*/
		25	struct clock_read_data {
		26	u64 epoch_ns;
		27	u64 epoch_cyc;
		28	u64 sched_clock_mask;
		29	u64 (*read_sched_clock)(void);
		30	u32 mult;
		31	u32 shift;
		32	};
		33
		34	extern struct clock_read_data sched_clock_read_begin(unsigned int seq);
		35	extern int sched_clock_read_retry(unsigned int seq);
		36
9	extern void generic_sched_clock_init(void);	37	extern void generic_sched_clock_init(void);
10		38
11	extern void sched_clock_register(u64 (*read)(void), int bits,	39	extern void sched_clock_register(u64 (*read)(void), int bits,

include/uapi/linux/perf_event.h+20−3

 				cap_bit0_is_deprecated	: 1, /* Always 1, signals that bit 0 is zero */
 				cap_user_rdpmc		: 1, /* The RDPMC instruction can be used to read counts */
-				cap_user_time		: 1, /* The time_* fields are used */
+				cap_user_time		: 1, /* The time_{shift,mult,offset} fields are used */
 				cap_user_time_zero	: 1, /* The time_zero field is used */
-				cap_____res		: 59;
+				cap_user_time_short	: 1, /* the time_{cycle,mask} fields are used */
+				cap_____res		: 58;
 		};
 	};
 	 *               ((rem * time_mult) >> time_shift);
 	 */
 	__u64	time_zero;
 	__u32	size;			/* Header size up to __reserved[] fields. */
+	__u32	__reserved_1;
+	/*
+	 * If cap_usr_time_short, the hardware clock is less than 64bit wide
+	 * and we must compute the 'cyc' value, as used by cap_usr_time, as:
+	 *
+	 *   cyc = time_cycles + ((cyc - time_cycles) & time_mask)
+	 *
+	 * NOTE: this form is explicitly chosen such that cap_usr_time_short
+	 *       is a correction on top of cap_usr_time, and code that doesn't
+	 *       know about cap_usr_time_short still works under the assumption
+	 *       the counter doesn't wrap.
+	 */
+	__u64	time_cycles;
+	__u64	time_mask;
 		/*
 		 * Hole for extension of the self monitor capabilities
 		 */
-	__u8	__reserved[118*8+4];	/* align to 1k. */
+	__u8	__reserved[116*8];	/* align to 1k. */
 	/*
 	 * Control data for the mmap() data buffer.

include/vdso/datapage.h+1−0

@@ -109,6 +109,7 @@ struct vdso_data { · 108 unmodified lines
109	* relocation, and this is what we need.	109	* relocation, and this is what we need.
110	*/	110	*/
111	extern struct vdso_data _vdso_data[CS_BASES] __attribute__((visibility("hidden")));	111	extern struct vdso_data _vdso_data[CS_BASES] __attribute__((visibility("hidden")));
		112	extern struct vdso_data _timens_data[CS_BASES] __attribute__((visibility("hidden")));
112		113
113	/*	114	/*
114	* The generic vDSO implementation requires that gettimeofday.h	115	* The generic vDSO implementation requires that gettimeofday.h

kernel/crash_core.c+1−0

@@ -413,6 +413,7 @@ static int __init crash_save_vmcoreinfo_init(void) · 412 unmodified lines
413	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);	413	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
414	VMCOREINFO_STRUCT_SIZE(mem_section);	414	VMCOREINFO_STRUCT_SIZE(mem_section);
415	VMCOREINFO_OFFSET(mem_section, section_mem_map);	415	VMCOREINFO_OFFSET(mem_section, section_mem_map);
		416	VMCOREINFO_NUMBER(MAX_PHYSMEM_BITS);
416	#endif	417	#endif
417	VMCOREINFO_STRUCT_SIZE(page);	418	VMCOREINFO_STRUCT_SIZE(page);
418	VMCOREINFO_STRUCT_SIZE(pglist_data);	419	VMCOREINFO_STRUCT_SIZE(pglist_data);

kernel/time/sched_clock.c+13−28

 #include "timekeeping.h"
-/**
- * struct clock_read_data - data required to read from sched_clock()
- *
- * @epoch_ns:		sched_clock() value at last update
- * @epoch_cyc:		Clock cycle value at last update.
- * @sched_clock_mask:   Bitmask for two's complement subtraction of non 64bit
- *			clocks.
- * @read_sched_clock:	Current clock source (or dummy source when suspended).
- * @mult:		Multipler for scaled math conversion.
- * @shift:		Shift value for scaled math conversion.
- *
- * Care must be taken when updating this structure; it is read by
- * some very hot code paths. It occupies <=40 bytes and, when combined
- * with the seqcount used to synchronize access, comfortably fits into
- * a 64 byte cache line.
- */
-struct clock_read_data {
-	u64 epoch_ns;
-	u64 epoch_cyc;
-	u64 sched_clock_mask;
-	u64 (*read_sched_clock)(void);
-	u32 mult;
-	u32 shift;
-};
 /**
  * struct clock_data - all data needed for sched_clock() (including
  *                     registration of a new clock source)
 	return (cyc * mult) >> shift;
 }
+struct clock_read_data *sched_clock_read_begin(unsigned int *seq)
+{
+	*seq = raw_read_seqcount_latch(&cd.seq);
+	return cd.read_data + (*seq & 1);
+}
+int sched_clock_read_retry(unsigned int seq)
+{
+	return read_seqcount_retry(&cd.seq, seq);
+}
 unsigned long long notrace sched_clock(void)
 {
 	u64 cyc, res;
 	struct clock_read_data *rd;
 	do {
-		seq = raw_read_seqcount(&cd.seq);
+		rd = sched_clock_read_begin(&seq);
-		rd = cd.read_data + (seq & 1);
 		cyc = (rd->read_sched_clock() - rd->epoch_cyc) &
 		      rd->sched_clock_mask;
 		res = rd->epoch_ns + cyc_to_ns(cyc, rd->mult, rd->shift);
-	} while (read_seqcount_retry(&cd.seq, seq));
+	} while (sched_clock_read_retry(seq));
 	return res;
 }

mm/memory.c+1−1

@@ -437,7 +437,7 @@ int __pte_alloc(struct mm_struct mm, pmd_t pmd) · 436 unmodified lines
437	* of a chain of data-dependent loads, meaning most CPUs (alpha	437	* of a chain of data-dependent loads, meaning most CPUs (alpha
438	* being the notable exception) will already guarantee loads are	438	* being the notable exception) will already guarantee loads are
439	* seen in-order. See the alpha page table accessors for the	439	* seen in-order. See the alpha page table accessors for the
440	* smp_read_barrier_depends() barriers in page table walking code.	440	* smp_rmb() barriers in page table walking code.
441	*/	441	*/
442	smp_wmb(); /* Could be smp_wmb__xxx(before\|after)_spin_lock */	442	smp_wmb(); /* Could be smp_wmb__xxx(before\|after)_spin_lock */
443		443

scripts/checkpatch.pl+1−8

 		my $barriers = qr{
 			mb|
 			rmb|
-			wmb|
+			wmb
-			read_barrier_depends
 		}x;
 		my $barrier_stems = qr{
 			mb__before_atomic|
 			}
 		}
-# check for smp_read_barrier_depends and read_barrier_depends
-		if (!$file && $line =~ /\b(smp_|)read_barrier_depends\s*\(/) {
-			WARN("READ_BARRIER_DEPENDS",
-			     "$1read_barrier_depends should only be used in READ_ONCE or DEC Alpha code\n" . $herecurr);
-		}
 # check of hardware specific defines
 		if ($line =~ m@^.\s*\#\s*if.*\b(__i386__|__powerpc64__|__sun__|__s390x__)\b@ && $realfile !~ m@include/asm-@) {
 			CHK("ARCH_DEFINES",

scripts/recordmcount.c+6−0

 	return 1;
 }
+static int arm64_is_fake_mcount(Elf64_Rel const *rp)
+{
+	return ELF64_R_TYPE(w(rp->r_info)) != R_AARCH64_CALL26;
+}
 /* 64-bit EM_MIPS has weird ELF64_Rela.r_info.
  * http://techpubs.sgi.com/library/manuals/4000/007-4658-001/pdf/007-4658-001.pdf
  * We interpret Table 29 Relocation Operation (Elf64_Rel, Elf64_Rela) [p.40]
 		make_nop = make_nop_arm64;
 		rel_type_nop = R_AARCH64_NONE;
 		ideal_nop = ideal_nop4_arm64;
+		is_fake_mcount64 = arm64_is_fake_mcount;
 		break;
 	case EM_IA_64:	reltype = R_IA64_IMM64; break;
 	case EM_MIPS:	/* reltype: e_class    */ break;

tools/bpf/Makefile+2−1

@@ -9,7 +9,8 @@ MAKE = make · 8 unmodified lines
9	INSTALL ?= install	9	INSTALL ?= install
10		10
11	CFLAGS += -Wall -O2	11	CFLAGS += -Wall -O2
12	CFLAGS += -D__EXPORTED_HEADERS__ -I$(srctree)/include/uapi -I$(srctree)/include	12	CFLAGS += -D__EXPORTED_HEADERS__ -I$(srctree)/tools/include/uapi \
		13	-I$(srctree)/tools/include
13		14
14	# This will work when bpf is built in tools env. where srctree	15	# This will work when bpf is built in tools env. where srctree
15	# isn't set and when invoked from selftests build, where srctree	16	# isn't set and when invoked from selftests build, where srctree

tools/include/uapi/linux/filter.h+90−0

@@ -0,0 +1,90 @@
	1	/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
	2	/*
	3	* Linux Socket Filter Data Structures
	4	*/
	5
	6	#ifndef __LINUX_FILTER_H__
	7	#define __LINUX_FILTER_H__
	8
	9
	10	#include <linux/types.h>
	11	#include <linux/bpf_common.h>
	12
	13	/*
	14	* Current version of the filter code architecture.
	15	*/
	16	#define BPF_MAJOR_VERSION 1
	17	#define BPF_MINOR_VERSION 1
	18
	19	/*
	20	* Try and keep these values and structures similar to BSD, especially
	21	* the BPF code definitions which need to match so you can share filters
	22	*/
	23
	24	struct sock_filter { /* Filter block */
	25	__u16 code; /* Actual filter code */
	26	__u8 jt; /* Jump true */
	27	__u8 jf; /* Jump false */
	28	__u32 k; /* Generic multiuse field */
	29	};
	30
	31	struct sock_fprog { /* Required for SO_ATTACH_FILTER. */
	32	unsigned short len; /* Number of filter blocks */
	33	struct sock_filter *filter;
	34	};
	35
	36	/* ret - BPF_K and BPF_X also apply */
	37	#define BPF_RVAL(code) ((code) & 0x18)
	38	#define BPF_A 0x10
	39
	40	/* misc */
	41	#define BPF_MISCOP(code) ((code) & 0xf8)
	42	#define BPF_TAX 0x00
	43	#define BPF_TXA 0x80
	44
	45	/*
	46	* Macros for filter block array initializers.
	47	*/
	48	#ifndef BPF_STMT
	49	#define BPF_STMT(code, k) { (unsigned short)(code), 0, 0, k }
	50	#endif
	51	#ifndef BPF_JUMP
	52	#define BPF_JUMP(code, k, jt, jf) { (unsigned short)(code), jt, jf, k }
	53	#endif
	54
	55	/*
	56	* Number of scratch memory words for: BPF_ST and BPF_STX
	57	*/
	58	#define BPF_MEMWORDS 16
	59
	60	/* RATIONALE. Negative offsets are invalid in BPF.
	61	We use them to reference ancillary data.
	62	Unlike introduction new instructions, it does not break
	63	existing compilers/optimizers.
	64	*/
	65	#define SKF_AD_OFF (-0x1000)
	66	#define SKF_AD_PROTOCOL 0
	67	#define SKF_AD_PKTTYPE 4
	68	#define SKF_AD_IFINDEX 8
	69	#define SKF_AD_NLATTR 12
	70	#define SKF_AD_NLATTR_NEST 16
	71	#define SKF_AD_MARK 20
	72	#define SKF_AD_QUEUE 24
	73	#define SKF_AD_HATYPE 28
	74	#define SKF_AD_RXHASH 32
	75	#define SKF_AD_CPU 36
	76	#define SKF_AD_ALU_XOR_X 40
	77	#define SKF_AD_VLAN_TAG 44
	78	#define SKF_AD_VLAN_TAG_PRESENT 48
	79	#define SKF_AD_PAY_OFFSET 52
	80	#define SKF_AD_RANDOM 56
	81	#define SKF_AD_VLAN_TPID 60
	82	#define SKF_AD_MAX 64
	83
	84	#define SKF_NET_OFF (-0x100000)
	85	#define SKF_LL_OFF (-0x200000)
	86
	87	#define BPF_NET_OFF SKF_NET_OFF
	88	#define BPF_LL_OFF SKF_LL_OFF
	89
	90	#endif /* __LINUX_FILTER_H__ */

tools/include/uapi/linux/perf_event.h+20−3

 				cap_bit0_is_deprecated	: 1, /* Always 1, signals that bit 0 is zero */
 				cap_user_rdpmc		: 1, /* The RDPMC instruction can be used to read counts */
-				cap_user_time		: 1, /* The time_* fields are used */
+				cap_user_time		: 1, /* The time_{shift,mult,offset} fields are used */
 				cap_user_time_zero	: 1, /* The time_zero field is used */
-				cap_____res		: 59;
+				cap_user_time_short	: 1, /* the time_{cycle,mask} fields are used */
+				cap_____res		: 58;
 		};
 	};
 	 *               ((rem * time_mult) >> time_shift);
 	 */
 	__u64	time_zero;
 	__u32	size;			/* Header size up to __reserved[] fields. */
+	__u32	__reserved_1;
+	/*
+	 * If cap_usr_time_short, the hardware clock is less than 64bit wide
+	 * and we must compute the 'cyc' value, as used by cap_usr_time, as:
+	 *
+	 *   cyc = time_cycles + ((cyc - time_cycles) & time_mask)
+	 *
+	 * NOTE: this form is explicitly chosen such that cap_usr_time_short
+	 *       is a correction on top of cap_usr_time, and code that doesn't
+	 *       know about cap_usr_time_short still works under the assumption
+	 *       the counter doesn't wrap.
+	 */
+	__u64	time_cycles;
+	__u64	time_mask;
 		/*
 		 * Hole for extension of the self monitor capabilities
 		 */
-	__u8	__reserved[118*8+4];	/* align to 1k. */
+	__u8	__reserved[116*8];	/* align to 1k. */
 	/*
 	 * Control data for the mmap() data buffer.

tools/memory-model/Documentation/explanation.txt+12−14

@@ -1122,12 +1122,10 @@ maintain at least the appearance of FIFO order. · 1121 unmodified lines
1122	In practice, this difficulty is solved by inserting a special fence	1122	In practice, this difficulty is solved by inserting a special fence
1123	between P1's two loads when the kernel is compiled for the Alpha	1123	between P1's two loads when the kernel is compiled for the Alpha
1124	architecture. In fact, as of version 4.15, the kernel automatically	1124	architecture. In fact, as of version 4.15, the kernel automatically
1125	adds this fence (called smp_read_barrier_depends() and defined as	1125	adds this fence after every READ_ONCE() and atomic load on Alpha. The
1126	nothing at all on non-Alpha builds) after every READ_ONCE() and atomic	1126	effect of the fence is to cause the CPU not to execute any po-later
1127	load. The effect of the fence is to cause the CPU not to execute any	1127	instructions until after the local cache has finished processing all
1128	po-later instructions until after the local cache has finished	1128	the stores it has already received. Thus, if the code was changed to:
1129	processing all the stores it has already received. Thus, if the code
1130	was changed to:
1131		1129
1132	P1()	1130	P1()
1133	{	1131	{
@@ -1146,14 +1144,14 @@ READ_ONCE() or another synchronization primitive rather than accessed · 12 unmodified lines
1146	directly.	1144	directly.
1147		1145
1148	The LKMM requires that smp_rmb(), acquire fences, and strong fences	1146	The LKMM requires that smp_rmb(), acquire fences, and strong fences
1149	share this property with smp_read_barrier_depends(): They do not allow	1147	share this property: They do not allow the CPU to execute any po-later
1150	the CPU to execute any po-later instructions (or po-later loads in the	1148	instructions (or po-later loads in the case of smp_rmb()) until all
1151	case of smp_rmb()) until all outstanding stores have been processed by	1149	outstanding stores have been processed by the local cache. In the
1152	the local cache. In the case of a strong fence, the CPU first has to	1150	case of a strong fence, the CPU first has to wait for all of its
1153	wait for all of its po-earlier stores to propagate to every other CPU	1151	po-earlier stores to propagate to every other CPU in the system; then
1154	in the system; then it has to wait for the local cache to process all	1152	it has to wait for the local cache to process all the stores received
1155	the stores received as of that time -- not just the stores received	1153	as of that time -- not just the stores received when the strong fence
1156	when the strong fence began.	1154	began.
1157		1155
1158	And of course, none of this matters for any architecture other than	1156	And of course, none of this matters for any architecture other than
1159	Alpha.	1157	Alpha.