lael-labs/linux

.mailmap+3−0

 Christian Borntraeger <borntraeger@linux.ibm.com> <borntraeger@de.ibm.com>
 Christian Borntraeger <borntraeger@linux.ibm.com> <cborntra@de.ibm.com>
 Christian Borntraeger <borntraeger@linux.ibm.com> <borntrae@de.ibm.com>
+Christian Brauner <brauner@kernel.org> <christian@brauner.io>
+Christian Brauner <brauner@kernel.org> <christian.brauner@canonical.com>
+Christian Brauner <brauner@kernel.org> <christian.brauner@ubuntu.com>
 Christophe Ricard <christophe.ricard@gmail.com>
 Christoph Hellwig <hch@lst.de>
 Colin Ian King <colin.king@intel.com> <colin.king@canonical.com>

Documentation/ABI/testing/sysfs-driver-aspeed-uart-routing+3−3

@@ -1,4 +1,4 @@
1	What: /sys/bus/platform/drivers/aspeed-uart-routing//uart	1	What: /sys/bus/platform/drivers/aspeed-uart-routing/\/uart\
2	Date: September 2021	2	Date: September 2021
3	Contact: Oskar Senft <osk@google.com>	3	Contact: Oskar Senft <osk@google.com>
4	Chia-Wei Wang <chiawei_wang@aspeedtech.com>	4	Chia-Wei Wang <chiawei_wang@aspeedtech.com>
@@ -9,7 +9,7 @@ Description: Selects the RX source of the UARTx device. · 4 unmodified lines
9	depends on the selected file.	9	depends on the selected file.
10		10
11	e.g.	11	e.g.
12	cat /sys/bus/platform/drivers/aspeed-uart-routing/*.uart_routing/uart1	12	cat /sys/bus/platform/drivers/aspeed-uart-routing/\*.uart_routing/uart1
13	[io1] io2 io3 io4 uart2 uart3 uart4 io6	13	[io1] io2 io3 io4 uart2 uart3 uart4 io6
14		14
15	In this case, UART1 gets its input from IO1 (physical serial port 1).	15	In this case, UART1 gets its input from IO1 (physical serial port 1).
@@ -17,7 +17,7 @@ Description: Selects the RX source of the UARTx device. · 1 unmodified line
17	Users: OpenBMC. Proposed changes should be mailed to	17	Users: OpenBMC. Proposed changes should be mailed to
18	openbmc@lists.ozlabs.org	18	openbmc@lists.ozlabs.org
19		19
20	What: /sys/bus/platform/drivers/aspeed-uart-routing//io	20	What: /sys/bus/platform/drivers/aspeed-uart-routing/\/io\
21	Date: September 2021	21	Date: September 2021
22	Contact: Oskar Senft <osk@google.com>	22	Contact: Oskar Senft <osk@google.com>
23	Chia-Wei Wang <chiawei_wang@aspeedtech.com>	23	Chia-Wei Wang <chiawei_wang@aspeedtech.com>

Documentation/arm64/silicon-errata.rst+2−0

 +----------------+-----------------+-----------------+-----------------------------+
 | ARM            | Cortex-A510     | #2051678        | ARM64_ERRATUM_2051678       |
 +----------------+-----------------+-----------------+-----------------------------+
+| ARM            | Cortex-A510     | #2077057        | ARM64_ERRATUM_2077057       |
++----------------+-----------------+-----------------+-----------------------------+
 | ARM            | Cortex-A710     | #2119858        | ARM64_ERRATUM_2119858       |
 +----------------+-----------------+-----------------+-----------------------------+
 | ARM            | Cortex-A710     | #2054223        | ARM64_ERRATUM_2054223       |

Documentation/dev-tools/kselftest.rst+8−0

@@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code · 6 unmodified lines
7	paths in the kernel. Tests are intended to be run after building, installing	7	paths in the kernel. Tests are intended to be run after building, installing
8	and booting a kernel.	8	and booting a kernel.
9		9
		10	Kselftest from mainline can be run on older stable kernels. Running tests
		11	from mainline offers the best coverage. Several test rings run mainline
		12	kselftest suite on stable releases. The reason is that when a new test
		13	gets added to test existing code to regression test a bug, we should be
		14	able to run that test on an older kernel. Hence, it is important to keep
		15	code that can still test an older kernel and make sure it skips the test
		16	gracefully on newer releases.
		17
10	You can find additional information on Kselftest framework, how to	18	You can find additional information on Kselftest framework, how to
11	write new tests using the framework on Kselftest wiki:	19	write new tests using the framework on Kselftest wiki:
12		20

Documentation/dev-tools/kunit/usage.rst+1−1

@@ -242,7 +242,7 @@ example: · 241 unmodified lines
242		242
243	int rectangle_area(struct shape *this)	243	int rectangle_area(struct shape *this)
244	{	244	{
245	struct rectangle *self = container_of(this, struct shape, parent);	245	struct rectangle *self = container_of(this, struct rectangle, parent);
246		246
247	return self->length * self->width;	247	return self->length * self->width;
248	};	248	};

Documentation/devicetree/bindings/arm/omap/omap.txt+3−0

@@ -119,6 +119,9 @@ Boards (incomplete list of examples): · 118 unmodified lines
119	- OMAP3 BeagleBoard : Low cost community board	119	- OMAP3 BeagleBoard : Low cost community board
120	compatible = "ti,omap3-beagle", "ti,omap3430", "ti,omap3"	120	compatible = "ti,omap3-beagle", "ti,omap3430", "ti,omap3"
121		121
		122	- OMAP3 BeagleBoard A to B4 : Early BeagleBoard revisions A to B4 with a timer quirk
		123	compatible = "ti,omap3-beagle-ab4", "ti,omap3-beagle", "ti,omap3430", "ti,omap3"
		124
122	- OMAP3 Tobi with Overo : Commercial expansion board with daughter board	125	- OMAP3 Tobi with Overo : Commercial expansion board with daughter board
123	compatible = "gumstix,omap3-overo-tobi", "gumstix,omap3-overo", "ti,omap3430", "ti,omap3"	126	compatible = "gumstix,omap3-overo-tobi", "gumstix,omap3-overo", "ti,omap3430", "ti,omap3"
124		127

Documentation/devicetree/bindings/display/bridge/analogix,anx7625.yaml+17−0

@@ -83,6 +83,9 @@ properties: · 82 unmodified lines
83	type: boolean	83	type: boolean
84	description: let the driver enable audio HDMI codec function or not.	84	description: let the driver enable audio HDMI codec function or not.
85		85
		86	aux-bus:
		87	$ref: /schemas/display/dp-aux-bus.yaml#
		88
86	ports:	89	ports:
87	$ref: /schemas/graph.yaml#/properties/ports	90	$ref: /schemas/graph.yaml#/properties/ports
88		91
@@ -167,5 +170,19 @@ examples: · 78 unmodified lines
167	};	170	};
168	};	171	};
169	};	172	};
		173
		174	aux-bus {
		175	panel {
		176	compatible = "innolux,n125hce-gn1";
		177	power-supply = <&pp3300_disp_x>;
		178	backlight = <&backlight_lcd0>;
		179
		180	port {
		181	panel_in: endpoint {
		182	remote-endpoint = <&anx7625_out>;
		183	};
		184	};
		185	};
		186	};
170	};	187	};
171	};	188	};

Documentation/devicetree/bindings/display/bridge/ingenic,jz4780-hdmi.yaml+82−0

@@ -0,0 +1,82 @@
	1	# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
	2	%YAML 1.2
	3	---
	4	$id: http://devicetree.org/schemas/display/bridge/ingenic,jz4780-hdmi.yaml#
	5	$schema: http://devicetree.org/meta-schemas/core.yaml#
	6
	7	title: Bindings for Ingenic JZ4780 HDMI Transmitter
	8
	9	maintainers:
	10	- H. Nikolaus Schaller <hns@goldelico.com>
	11
	12	description: \|
	13	The HDMI Transmitter in the Ingenic JZ4780 is a Synopsys DesignWare HDMI 1.4
	14	TX controller IP with accompanying PHY IP.
	15
	16	allOf:
	17	- $ref: synopsys,dw-hdmi.yaml#
	18
	19	properties:
	20	compatible:
	21	const: ingenic,jz4780-dw-hdmi
	22
	23	reg-io-width:
	24	const: 4
	25
	26	clocks:
	27	maxItems: 2
	28
	29	ports:
	30	$ref: /schemas/graph.yaml#/properties/ports
	31
	32	properties:
	33	port@0:
	34	$ref: /schemas/graph.yaml#/properties/port
	35	description: Input from LCD controller output.
	36
	37	port@1:
	38	$ref: /schemas/graph.yaml#/properties/port
	39	description: Link to the HDMI connector.
	40
	41	required:
	42	- compatible
	43	- clocks
	44	- clock-names
	45	- ports
	46	- reg-io-width
	47
	48	unevaluatedProperties: false
	49
	50	examples:
	51	- \|
	52	#include <dt-bindings/clock/ingenic,jz4780-cgu.h>
	53
	54	hdmi: hdmi@10180000 {
	55	compatible = "ingenic,jz4780-dw-hdmi";
	56	reg = <0x10180000 0x8000>;
	57	reg-io-width = <4>;
	58	ddc-i2c-bus = <&i2c4>;
	59	interrupt-parent = <&intc>;
	60	interrupts = <3>;
	61	clocks = <&cgu JZ4780_CLK_AHB0>, <&cgu JZ4780_CLK_HDMI>;
	62	clock-names = "iahb", "isfr";
	63
	64	ports {
	65	#address-cells = <1>;
	66	#size-cells = <0>;
	67	hdmi_in: port@0 {
	68	reg = <0>;
	69	dw_hdmi_in: endpoint {
	70	remote-endpoint = <&jz4780_lcd_out>;
	71	};
	72	};
	73	hdmi_out: port@1 {
	74	reg = <1>;
	75	dw_hdmi_out: endpoint {
	76	remote-endpoint = <&hdmi_con>;
	77	};
	78	};
	79	};
	80	};
	81
	82	...

Documentation/devicetree/bindings/display/bridge/lvds-codec.yaml+1−1

@@ -68,7 +68,7 @@ properties: · 67 unmodified lines
68	- vesa-24	68	- vesa-24
69	description: \|	69	description: \|
70	The color signals mapping order. See details in	70	The color signals mapping order. See details in
71	Documentation/devicetree/bindings/display/panel/lvds.yaml	71	Documentation/devicetree/bindings/display/lvds.yaml
72		72
73	port@1:	73	port@1:
74	$ref: /schemas/graph.yaml#/properties/port	74	$ref: /schemas/graph.yaml#/properties/port

Documentation/devicetree/bindings/display/lvds.yamlrenamed from Documentation/devicetree/bindings/display/panel/lvds.yaml+4−31

 # SPDX-License-Identifier: GPL-2.0
 %YAML 1.2
 ---
-$id: http://devicetree.org/schemas/display/panel/lvds.yaml#
+$id: http://devicetree.org/schemas/display/lvds.yaml#
 $schema: http://devicetree.org/meta-schemas/core.yaml#
-title: LVDS Display Panel
+title: LVDS Display Common Properties
 maintainers:
   - Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
 description: |+
   LVDS is a physical layer specification defined in ANSI/TIA/EIA-644-A. Multiple
   incompatible data link layers have been used over time to transmit image data
-  to LVDS panels. This bindings supports display panels compatible with the
+  to LVDS devices. This bindings supports devices compatible with the following
-  following specifications.
+  specifications.
   [JEIDA] "Digital Interface Standards for Monitor", JEIDA-59-1999, February
   1999 (Version 1.0), Japan Electronic Industry Development Association (JEIDA)
   Device compatible with those specifications have been marketed under the
   FPD-Link and FlatLink brands.
-allOf:
-  - $ref: panel-common.yaml#
 properties:
-  compatible:
-    contains:
-      const: panel-lvds
-    description:
-      Shall contain "panel-lvds" in addition to a mandatory panel-specific
-      compatible string defined in individual panel bindings. The "panel-lvds"
-      value shall never be used on its own.
   data-mapping:
     enum:
       - jeida-18
       If set, reverse the bit order described in the data mappings below on all
       data lanes, transmitting bits for slots 6 to 0 instead of 0 to 6.
-  port: true
-  ports: true
-required:
-  - compatible
-  - data-mapping
-  - width-mm
-  - height-mm
-  - panel-timing
-oneOf:
-  - required:
-      - port
-  - required:
-      - ports
 additionalProperties: true
 ...

Documentation/devicetree/bindings/display/panel/advantech,idk-1110wr.yaml+17−2

   - Thierry Reding <thierry.reding@gmail.com>
 allOf:
-  - $ref: lvds.yaml#
+  - $ref: panel-common.yaml#
+  - $ref: /schemas/display/lvds.yaml/#
+select:
+  properties:
+    compatible:
+      contains:
+        const: advantech,idk-1110wr
+  required:
+    - compatible
 properties:
   compatible:
     items:
       - const: advantech,idk-1110wr
-      - {} # panel-lvds, but not listed here to avoid false select
+      - const: panel-lvds
   data-mapping:
     const: jeida-24
 required:
   - compatible
+  - data-mapping
+  - width-mm
+  - height-mm
+  - panel-timing
+  - port
 examples:
   - |+

Documentation/devicetree/bindings/display/panel/innolux,ee101ia-01d.yaml+21−2

   - Thierry Reding <thierry.reding@gmail.com>
 allOf:
-  - $ref: lvds.yaml#
+  - $ref: panel-common.yaml#
+  - $ref: /schemas/display/lvds.yaml/#
+select:
+  properties:
+    compatible:
+      contains:
+        const: innolux,ee101ia-01d
+  required:
+    - compatible
 properties:
   compatible:
     items:
       - const: innolux,ee101ia-01d
-      - {} # panel-lvds, but not listed here to avoid false select
+      - const: panel-lvds
   backlight: true
+  data-mapping: true
   enable-gpios: true
   power-supply: true
   width-mm: true
   panel-timing: true
   port: true
+required:
+  - compatible
+  - data-mapping
+  - width-mm
+  - height-mm
+  - panel-timing
+  - port
 additionalProperties: false
 ...

Documentation/devicetree/bindings/display/panel/mitsubishi,aa104xd12.yaml+17−2

   - Thierry Reding <thierry.reding@gmail.com>
 allOf:
-  - $ref: lvds.yaml#
+  - $ref: panel-common.yaml#
+  - $ref: /schemas/display/lvds.yaml/#
+select:
+  properties:
+    compatible:
+      contains:
+        const: mitsubishi,aa104xd12
+  required:
+    - compatible
 properties:
   compatible:
     items:
       - const: mitsubishi,aa104xd12
-      - {} # panel-lvds, but not listed here to avoid false select
+      - const: panel-lvds
   vcc-supply:
     description: Reference to the regulator powering the panel VCC pins.
 required:
   - compatible
   - vcc-supply
+  - data-mapping
+  - width-mm
+  - height-mm
+  - panel-timing
+  - port
 examples:
   - |+

Documentation/devicetree/bindings/display/panel/mitsubishi,aa121td01.yaml+17−2

   - Thierry Reding <thierry.reding@gmail.com>
 allOf:
-  - $ref: lvds.yaml#
+  - $ref: panel-common.yaml#
+  - $ref: /schemas/display/lvds.yaml/#
+select:
+  properties:
+    compatible:
+      contains:
+        const: mitsubishi,aa121td01
+  required:
+    - compatible
 properties:
   compatible:
     items:
       - const: mitsubishi,aa121td01
-      - {} # panel-lvds, but not listed here to avoid false select
+      - const: panel-lvds
   vcc-supply:
     description: Reference to the regulator powering the panel VCC pins.
 required:
   - compatible
   - vcc-supply
+  - data-mapping
+  - width-mm
+  - height-mm
+  - panel-timing
+  - port
 examples:
   - |+

Documentation/devicetree/bindings/display/panel/panel-lvds.yaml+57−0

@@ -0,0 +1,57 @@
	1	# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
	2	%YAML 1.2
	3	---
	4	$id: http://devicetree.org/schemas/display/panel/panel-lvds.yaml#
	5	$schema: http://devicetree.org/meta-schemas/core.yaml#
	6
	7	title: Generic LVDS Display Panel Device Tree Bindings
	8
	9	maintainers:
	10	- Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
	11	- Thierry Reding <thierry.reding@gmail.com>
	12
	13	allOf:
	14	- $ref: panel-common.yaml#
	15	- $ref: /schemas/display/lvds.yaml/#
	16
	17	select:
	18	properties:
	19	compatible:
	20	contains:
	21	const: panel-lvds
	22
	23	not:
	24	properties:
	25	compatible:
	26	contains:
	27	enum:
	28	- advantech,idk-1110wr
	29	- advantech,idk-2121wr
	30	- innolux,ee101ia-01d
	31	- mitsubishi,aa104xd12
	32	- mitsubishi,aa121td01
	33	- sgd,gktw70sdae4se
	34
	35	required:
	36	- compatible
	37
	38	properties:
	39	compatible:
	40	items:
	41	- enum:
	42	- auo,b101ew05
	43	- tbs,a711-panel
	44
	45	- const: panel-lvds
	46
	47	unevaluatedProperties: false
	48
	49	required:
	50	- compatible
	51	- data-mapping
	52	- width-mm
	53	- height-mm
	54	- panel-timing
	55	- port
	56
	57	...

Documentation/devicetree/bindings/display/panel/panel-simple.yaml+2−0

       - sharp,lq101k1ly04
         # Sharp 12.3" (2400x1600 pixels) TFT LCD panel
       - sharp,lq123p1jx31
+        # Sharp 14" (1920x1080 pixels) TFT LCD panel
+      - sharp,lq140m1jw46
         # Sharp LS020B1DD01D 2.0" HQVGA TFT LCD panel
       - sharp,ls020b1dd01d
         # Shelly SCA07010-BFN-LNN 7.0" WVGA TFT LCD panel

Documentation/devicetree/bindings/display/panel/sgd,gktw70sdae4se.yaml+17−2

   - Thierry Reding <thierry.reding@gmail.com>
 allOf:
-  - $ref: lvds.yaml#
+  - $ref: panel-common.yaml#
+  - $ref: /schemas/display/lvds.yaml/#
+select:
+  properties:
+    compatible:
+      contains:
+        const: sgd,gktw70sdae4se
+  required:
+    - compatible
 properties:
   compatible:
     items:
       - const: sgd,gktw70sdae4se
-      - {} # panel-lvds, but not listed here to avoid false select
+      - const: panel-lvds
   data-mapping:
     const: jeida-18
 required:
   - compatible
+  - port
+  - data-mapping
+  - width-mm
+  - height-mm
+  - panel-timing
 examples:
   - |+

Documentation/devicetree/bindings/display/solomon,ssd1307fb.yaml+1−0

@@ -8,6 +8,7 @@ title: Solomon SSD1307 OLED Controller Framebuffer · 7 unmodified lines
8		8
9	maintainers:	9	maintainers:
10	- Maxime Ripard <mripard@kernel.org>	10	- Maxime Ripard <mripard@kernel.org>
		11	- Javier Martinez Canillas <javierm@redhat.com>
11		12
12	properties:	13	properties:
13	compatible:	14	compatible:

Documentation/devicetree/bindings/gpu/arm,mali-bifrost.yaml+15−0

@@ -159,6 +159,21 @@ allOf: · 158 unmodified lines
159	power-domains:	159	power-domains:
160	maxItems: 1	160	maxItems: 1
161	sram-supply: false	161	sram-supply: false
		162	- if:
		163	properties:
		164	compatible:
		165	contains:
		166	const: rockchip,rk3568-mali
		167	then:
		168	properties:
		169	clocks:
		170	minItems: 2
		171	clock-names:
		172	items:
		173	- const: gpu
		174	- const: bus
		175	required:
		176	- clock-names
162		177
163	examples:	178	examples:
164	- \|	179	- \|

Documentation/devicetree/bindings/interrupt-controller/sifive,plic-1.0.0.yaml+15−6

   contains a specific memory layout, which is documented in chapter 8 of the
   SiFive U5 Coreplex Series Manual <https://static.dev.sifive.com/U54-MC-RVCoreIP.pdf>.
+  The thead,c900-plic is different from sifive,plic-1.0.0 in opensbi, the
+  T-HEAD PLIC implementation requires setting a delegation bit to allow access
+  from S-mode. So add thead,c900-plic to distinguish them.
 maintainers:
   - Sagar Kadam <sagar.kadam@sifive.com>
   - Paul Walmsley  <paul.walmsley@sifive.com>
 properties:
   compatible:
-    items:
+    oneOf:
-      - enum:
+      - items:
-          - sifive,fu540-c000-plic
+          - enum:
-          - starfive,jh7100-plic
+              - sifive,fu540-c000-plic
-          - canaan,k210-plic
+              - starfive,jh7100-plic
-      - const: sifive,plic-1.0.0
+              - canaan,k210-plic
+          - const: sifive,plic-1.0.0
+      - items:
+          - enum:
+              - allwinner,sun20i-d1-plic
+          - const: thead,c900-plic
   reg:
     maxItems: 1

Documentation/devicetree/bindings/net/qcom,ipa.yaml+6−0

           - const: imem
           - const: config
+  qcom,qmp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: phandle to the AOSS side-channel message RAM
   qcom,smem-states:
     $ref: /schemas/types.yaml#/definitions/phandle-array
     description: State bits used in by the AP to signal the modem.
                                      "imem",
                                      "config";
+                qcom,qmp = <&aoss_qmp>;
                 qcom,smem-states = <&ipa_smp2p_out 0>,
                                    <&ipa_smp2p_out 1>;
                 qcom,smem-state-names = "ipa-clock-enabled-valid",

Documentation/devicetree/bindings/spi/spi-peripheral-props.yaml+3−2

     minItems: 1
     maxItems: 256
     items:
-      minimum: 0
+      items:
-      maximum: 256
+        - minimum: 0
+          maximum: 256
     description:
       Chip select used by the device.

Documentation/driver-api/device-io.rst+9−0

@@ -502,6 +502,15 @@ pcim_iomap() · 501 unmodified lines
502	Not using these wrappers may make drivers unusable on certain platforms with	502	Not using these wrappers may make drivers unusable on certain platforms with
503	stricter rules for mapping I/O memory.	503	stricter rules for mapping I/O memory.
504		504
		505	Generalizing Access to System and I/O Memory
		506	============================================
		507
		508	.. kernel-doc:: include/linux/iosys-map.h
		509	:doc: overview
		510
		511	.. kernel-doc:: include/linux/iosys-map.h
		512	:internal:
		513
505	Public Functions Provided	514	Public Functions Provided
506	=========================	515	=========================
507		516

Documentation/driver-api/dma-buf.rst+0−9

@@ -128,15 +128,6 @@ Kernel Functions and Structures Reference · 127 unmodified lines
128	.. kernel-doc:: include/linux/dma-buf.h	128	.. kernel-doc:: include/linux/dma-buf.h
129	:internal:	129	:internal:
130		130
131	Buffer Mapping Helpers
132	~~~~~~~~~~~~~~~~~~~~~~
133
134	.. kernel-doc:: include/linux/dma-buf-map.h
135	:doc: overview
136
137	.. kernel-doc:: include/linux/dma-buf-map.h
138	:internal:
139
140	Reservation Objects	131	Reservation Objects
141	-------------------	132	-------------------
142		133

Documentation/filesystems/netfs_library.rst+16−0

@@ -462,6 +462,10 @@ operation table looks like the following:: · 461 unmodified lines
462	struct iov_iter *iter,	462	struct iov_iter *iter,
463	netfs_io_terminated_t term_func,	463	netfs_io_terminated_t term_func,
464	void *term_func_priv);	464	void *term_func_priv);
		465
		466	int (query_occupancy)(struct netfs_cache_resources cres,
		467	loff_t start, size_t len, size_t granularity,
		468	loff_t _data_start, size_t _data_len);
465	};	469	};
466		470
467	With a termination handler function pointer::	471	With a termination handler function pointer::
@@ -536,6 +540,18 @@ The methods defined in the table are: · 68 unmodified lines
536	indicating whether the termination is definitely happening in the caller's	540	indicating whether the termination is definitely happening in the caller's
537	context.	541	context.
538		542
		543	* ``query_occupancy()``
		544
		545	[Required] Called to find out where the next piece of data is within a
		546	particular region of the cache. The start and length of the region to be
		547	queried are passed in, along with the granularity to which the answer needs
		548	to be aligned. The function passes back the start and length of the data,
		549	if any, available within that region. Note that there may be a hole at the
		550	front.
		551
		552	It returns 0 if some data was found, -ENODATA if there was no usable data
		553	within the region or -ENOBUFS if there is no caching on this file.
		554
539	Note that these methods are passed a pointer to the cache resource structure,	555	Note that these methods are passed a pointer to the cache resource structure,
540	not the read request structure as they could be used in other situations where	556	not the read request structure as they could be used in other situations where
541	there isn't a read request structure as well, such as writing dirty data to the	557	there isn't a read request structure as well, such as writing dirty data to the

Documentation/gpu/todo.rst+10−34

 Most drivers can use drm_fbdev_generic_setup(). Driver have to implement
 atomic modesetting and GEM vmap support. Historically, generic fbdev emulation
 expected the framebuffer in system memory or system-like memory. By employing
-struct dma_buf_map, drivers with frambuffers in I/O memory can be supported
+struct iosys_map, drivers with frambuffers in I/O memory can be supported
 as well.
 Contact: Maintainer of the driver you plan to convert
 A number of callback functions in drm_fbdev_fb_ops could benefit from
 being rewritten without dependencies on the fbdev module. Some of the
-helpers could further benefit from using struct dma_buf_map instead of
+helpers could further benefit from using struct iosys_map instead of
 raw pointers.
 Contact: Thomas Zimmermann <tzimmermann@suse.de>, Daniel Vetter
 Level: Advanced
-Garbage collect fbdev scrolling acceleration
---------------------------------------------
-Scroll acceleration has been disabled in fbcon. Now it works as the old
-SCROLL_REDRAW mode. A ton of code was removed in fbcon.c and the hook bmove was
-removed from fbcon_ops.
-Remaining tasks:
-- a bunch of the hooks in fbcon_ops could be removed or simplified by calling
-  directly instead of the function table (with a switch on p->rotate)
-- fb_copyarea is unused after this, and can be deleted from all drivers
-- after that, fb_copyarea can be deleted from fb_ops in include/linux/fb.h as
-  well as cfb_copyarea
-Note that not all acceleration code can be deleted, since clearing and cursor
-support is still accelerated, which might be good candidates for further
-deletion projects.
-Contact: Daniel Vetter
-Level: Intermediate
 idr_init_base()
 ---------------
 Level: Intermediate
-Use struct dma_buf_map throughout codebase
+Use struct iosys_map throughout codebase
-------------------------------------------
+----------------------------------------
-Pointers to shared device memory are stored in struct dma_buf_map. Each
+Pointers to shared device memory are stored in struct iosys_map. Each
 instance knows whether it refers to system or I/O memory. Most of the DRM-wide
-interface have been converted to use struct dma_buf_map, but implementations
+interface have been converted to use struct iosys_map, but implementations
 often still use raw pointers.
-The task is to use struct dma_buf_map where it makes sense.
+The task is to use struct iosys_map where it makes sense.
-* Memory managers should use struct dma_buf_map for dma-buf-imported buffers.
+* Memory managers should use struct iosys_map for dma-buf-imported buffers.
-* TTM might benefit from using struct dma_buf_map internally.
+* TTM might benefit from using struct iosys_map internally.
-* Framebuffer copying and blitting helpers should operate on struct dma_buf_map.
+* Framebuffer copying and blitting helpers should operate on struct iosys_map.
 Contact: Thomas Zimmermann <tzimmermann@suse.de>, Christian König, Daniel Vetter

Documentation/userspace-api/ioctl/ioctl-number.rst+3−0

MAINTAINERS+89−20

@@ -1620,6 +1620,7 @@ M: Olof Johansson <olof@lixom.net> · 1619 unmodified lines
1620	M: soc@kernel.org	1620	M: soc@kernel.org
1621	L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)	1621	L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
1622	S: Maintained	1622	S: Maintained
		1623	C: irc://irc.libera.chat/armlinux
1623	T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git	1624	T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git
1624	F: arch/arm/boot/dts/Makefile	1625	F: arch/arm/boot/dts/Makefile
1625	F: arch/arm64/boot/dts/Makefile	1626	F: arch/arm64/boot/dts/Makefile
@@ -1627,6 +1628,7 @@ F: arch/arm64/boot/dts/Makefile · 1 unmodified line
1627	ARM SUB-ARCHITECTURES	1628	ARM SUB-ARCHITECTURES
1628	L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)	1629	L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
1629	S: Maintained	1630	S: Maintained
		1631	C: irc://irc.libera.chat/armlinux
1630	T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git	1632	T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git
1631	F: arch/arm/mach-*/	1633	F: arch/arm/mach-*/
1632	F: arch/arm/plat-*/	1634	F: arch/arm/plat-*/
@@ -1780,6 +1782,7 @@ F: drivers/irqchip/irq-apple-aic.c · 147 unmodified lines
1780	F: drivers/mailbox/apple-mailbox.c	1782	F: drivers/mailbox/apple-mailbox.c
1781	F: drivers/pinctrl/pinctrl-apple-gpio.c	1783	F: drivers/pinctrl/pinctrl-apple-gpio.c
1782	F: drivers/soc/apple/*	1784	F: drivers/soc/apple/*
		1785	F: drivers/watchdog/apple_wdt.c
1783	F: include/dt-bindings/interrupt-controller/apple-aic.h	1786	F: include/dt-bindings/interrupt-controller/apple-aic.h
1784	F: include/dt-bindings/pinctrl/apple.h	1787	F: include/dt-bindings/pinctrl/apple.h
1785	F: include/linux/apple-mailbox.h	1788	F: include/linux/apple-mailbox.h
@@ -2570,10 +2573,13 @@ N: rockchip · 784 unmodified lines
2570		2573
2571	ARM/SAMSUNG S3C, S5P AND EXYNOS ARM ARCHITECTURES	2574	ARM/SAMSUNG S3C, S5P AND EXYNOS ARM ARCHITECTURES
2572	M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>	2575	M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
		2576	R: Alim Akhtar <alim.akhtar@samsung.com>
2573	L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)	2577	L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
2574	L: linux-samsung-soc@vger.kernel.org	2578	L: linux-samsung-soc@vger.kernel.org
2575	S: Maintained	2579	S: Maintained
		2580	C: irc://irc.libera.chat/linux-exynos
2576	Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/	2581	Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/
		2582	T: git git://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git
2577	F: Documentation/arm/samsung/	2583	F: Documentation/arm/samsung/
2578	F: Documentation/devicetree/bindings/arm/samsung/	2584	F: Documentation/devicetree/bindings/arm/samsung/
2579	F: Documentation/devicetree/bindings/power/pd-samsung.yaml	2585	F: Documentation/devicetree/bindings/power/pd-samsung.yaml
@@ -4157,9 +4163,8 @@ N: csky · 1577 unmodified lines
4157	K: csky	4163	K: csky
4158		4164
4159	CA8210 IEEE-802.15.4 RADIO DRIVER	4165	CA8210 IEEE-802.15.4 RADIO DRIVER
4160	M: Harry Morris <h.morris@cascoda.com>
4161	L: linux-wpan@vger.kernel.org	4166	L: linux-wpan@vger.kernel.org
4162	S: Maintained	4167	S: Orphan
4163	W: https://github.com/Cascoda/ca8210-linux.git	4168	W: https://github.com/Cascoda/ca8210-linux.git
4164	F: Documentation/devicetree/bindings/net/ieee802154/ca8210.txt	4169	F: Documentation/devicetree/bindings/net/ieee802154/ca8210.txt
4165	F: drivers/net/ieee802154/ca8210.c	4170	F: drivers/net/ieee802154/ca8210.c
@@ -5734,7 +5739,7 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 1568 unmodified lines
5734	F: Documentation/driver-api/dma-buf.rst	5739	F: Documentation/driver-api/dma-buf.rst
5735	F: drivers/dma-buf/	5740	F: drivers/dma-buf/
5736	F: include/linux/*fence.h	5741	F: include/linux/*fence.h
5737	F: include/linux/dma-buf*	5742	F: include/linux/dma-buf.h
5738	F: include/linux/dma-resv.h	5743	F: include/linux/dma-resv.h
5739	K: \bdma_(?:buf\|fence\|resv)\b	5744	K: \bdma_(?:buf\|fence\|resv)\b
5740		5745
@@ -5773,7 +5778,7 @@ F: tools/testing/selftests/dma/ · 32 unmodified lines
5773		5778
5774	DMA-BUF HEAPS FRAMEWORK	5779	DMA-BUF HEAPS FRAMEWORK
5775	M: Sumit Semwal <sumit.semwal@linaro.org>	5780	M: Sumit Semwal <sumit.semwal@linaro.org>
5776	R: Benjamin Gaignard <benjamin.gaignard@linaro.org>	5781	R: Benjamin Gaignard <benjamin.gaignard@collabora.com>
5777	R: Liam Mark <lmark@codeaurora.org>	5782	R: Liam Mark <lmark@codeaurora.org>
5778	R: Laura Abbott <labbott@redhat.com>	5783	R: Laura Abbott <labbott@redhat.com>
5779	R: Brian Starkey <Brian.Starkey@arm.com>	5784	R: Brian Starkey <Brian.Starkey@arm.com>
@@ -6077,7 +6082,8 @@ L: dri-devel@lists.freedesktop.org · 297 unmodified lines
6077	T: git git://anongit.freedesktop.org/drm/drm-misc	6082	T: git git://anongit.freedesktop.org/drm/drm-misc
6078	S: Maintained	6083	S: Maintained
6079	F: drivers/gpu/drm/panel/panel-lvds.c	6084	F: drivers/gpu/drm/panel/panel-lvds.c
6080	F: Documentation/devicetree/bindings/display/panel/lvds.yaml	6085	F: Documentation/devicetree/bindings/display/lvds.yaml
		6086	F: Documentation/devicetree/bindings/display/panel/panel-lvds.yaml
6081		6087
6082	DRM DRIVER FOR MANTIX MLAF057WE51 PANELS	6088	DRM DRIVER FOR MANTIX MLAF057WE51 PANELS
6083	M: Guido Günther <agx@sigxcpu.org>	6089	M: Guido Günther <agx@sigxcpu.org>
@@ -6126,6 +6132,13 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 42 unmodified lines
6126	F: Documentation/devicetree/bindings/display/panel/novatek,nt35510.yaml	6132	F: Documentation/devicetree/bindings/display/panel/novatek,nt35510.yaml
6127	F: drivers/gpu/drm/panel/panel-novatek-nt35510.c	6133	F: drivers/gpu/drm/panel/panel-novatek-nt35510.c
6128		6134
		6135	DRM DRIVER FOR NOVATEK NT35560 PANELS
		6136	M: Linus Walleij <linus.walleij@linaro.org>
		6137	S: Maintained
		6138	T: git git://anongit.freedesktop.org/drm/drm-misc
		6139	F: Documentation/devicetree/bindings/display/panel/sony,acx424akp.yaml
		6140	F: drivers/gpu/drm/panel/panel-novatek-nt35560.c
		6141
6129	DRM DRIVER FOR NOVATEK NT36672A PANELS	6142	DRM DRIVER FOR NOVATEK NT36672A PANELS
6130	M: Sumit Semwal <sumit.semwal@linaro.org>	6143	M: Sumit Semwal <sumit.semwal@linaro.org>
6131	S: Maintained	6144	S: Maintained
@@ -6162,6 +6175,13 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 30 unmodified lines
6162	F: Documentation/devicetree/bindings/display/repaper.txt	6175	F: Documentation/devicetree/bindings/display/repaper.txt
6163	F: drivers/gpu/drm/tiny/repaper.c	6176	F: drivers/gpu/drm/tiny/repaper.c
6164		6177
		6178	DRM DRIVER FOR SOLOMON SSD130X OLED DISPLAYS
		6179	M: Javier Martinez Canillas <javierm@redhat.com>
		6180	S: Maintained
		6181	T: git git://anongit.freedesktop.org/drm/drm-misc
		6182	F: Documentation/devicetree/bindings/display/solomon,ssd1307fb.yaml
		6183	F: drivers/gpu/drm/solomon/ssd130x*
		6184
6165	DRM DRIVER FOR QEMU'S CIRRUS DEVICE	6185	DRM DRIVER FOR QEMU'S CIRRUS DEVICE
6166	M: Dave Airlie <airlied@redhat.com>	6186	M: Dave Airlie <airlied@redhat.com>
6167	M: Gerd Hoffmann <kraxel@redhat.com>	6187	M: Gerd Hoffmann <kraxel@redhat.com>
@@ -6250,12 +6270,6 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 82 unmodified lines
6250	F: Documentation/devicetree/bindings/display/sitronix,st7735r.yaml	6270	F: Documentation/devicetree/bindings/display/sitronix,st7735r.yaml
6251	F: drivers/gpu/drm/tiny/st7735r.c	6271	F: drivers/gpu/drm/tiny/st7735r.c
6252		6272
6253	DRM DRIVER FOR SONY ACX424AKP PANELS
6254	M: Linus Walleij <linus.walleij@linaro.org>
6255	S: Maintained
6256	T: git git://anongit.freedesktop.org/drm/drm-misc
6257	F: drivers/gpu/drm/panel/panel-sony-acx424akp.c
6258
6259	DRM DRIVER FOR ST-ERICSSON MCDE	6273	DRM DRIVER FOR ST-ERICSSON MCDE
6260	M: Linus Walleij <linus.walleij@linaro.org>	6274	M: Linus Walleij <linus.walleij@linaro.org>
6261	S: Maintained	6275	S: Maintained
@@ -6503,7 +6517,7 @@ F: Documentation/devicetree/bindings/display/rockchip/ · 241 unmodified lines
6503	F: drivers/gpu/drm/rockchip/	6517	F: drivers/gpu/drm/rockchip/
6504		6518
6505	DRM DRIVERS FOR STI	6519	DRM DRIVERS FOR STI
6506	M: Benjamin Gaignard <benjamin.gaignard@linaro.org>	6520	M: Alain Volmat <alain.volmat@foss.st.com>
6507	L: dri-devel@lists.freedesktop.org	6521	L: dri-devel@lists.freedesktop.org
6508	S: Maintained	6522	S: Maintained
6509	T: git git://anongit.freedesktop.org/drm/drm-misc	6523	T: git git://anongit.freedesktop.org/drm/drm-misc
@@ -6512,8 +6526,8 @@ F: drivers/gpu/drm/sti · 2 unmodified lines
6512		6526
6513	DRM DRIVERS FOR STM	6527	DRM DRIVERS FOR STM
6514	M: Yannick Fertre <yannick.fertre@foss.st.com>	6528	M: Yannick Fertre <yannick.fertre@foss.st.com>
		6529	M: Raphael Gallais-Pou <raphael.gallais-pou@foss.st.com>
6515	M: Philippe Cornu <philippe.cornu@foss.st.com>	6530	M: Philippe Cornu <philippe.cornu@foss.st.com>
6516	M: Benjamin Gaignard <benjamin.gaignard@linaro.org>
6517	L: dri-devel@lists.freedesktop.org	6531	L: dri-devel@lists.freedesktop.org
6518	S: Maintained	6532	S: Maintained
6519	T: git git://anongit.freedesktop.org/drm/drm-misc	6533	T: git git://anongit.freedesktop.org/drm/drm-misc
@@ -7576,6 +7590,12 @@ S: Maintained · 1056 unmodified lines
7576	W: http://floatingpoint.sourceforge.net/emulator/index.html	7590	W: http://floatingpoint.sourceforge.net/emulator/index.html
7577	F: arch/x86/math-emu/	7591	F: arch/x86/math-emu/
7578		7592
		7593	FRAMEBUFFER CORE
		7594	M: Daniel Vetter <daniel@ffwll.ch>
		7595	F: drivers/video/fbdev/core/
		7596	S: Odd Fixes
		7597	T: git git://anongit.freedesktop.org/drm/drm-misc
		7598
7579	FRAMEBUFFER LAYER	7599	FRAMEBUFFER LAYER
7580	M: Helge Deller <deller@gmx.de>	7600	M: Helge Deller <deller@gmx.de>
7581	L: linux-fbdev@vger.kernel.org	7601	L: linux-fbdev@vger.kernel.org
@@ -10050,6 +10070,13 @@ F: include/linux/iova.h · 2468 unmodified lines
10050	F: include/linux/of_iommu.h	10070	F: include/linux/of_iommu.h
10051	F: include/uapi/linux/iommu.h	10071	F: include/uapi/linux/iommu.h
10052		10072
		10073	IOSYS-MAP HELPERS
		10074	M: Thomas Zimmermann <tzimmermann@suse.de>
		10075	L: dri-devel@lists.freedesktop.org
		10076	S: Maintained
		10077	T: git git://anongit.freedesktop.org/drm/drm-misc
		10078	F: include/linux/iosys-map.h
		10079
10053	IO_URING	10080	IO_URING
10054	M: Jens Axboe <axboe@kernel.dk>	10081	M: Jens Axboe <axboe@kernel.dk>
10055	R: Pavel Begunkov <asml.silence@gmail.com>	10082	R: Pavel Begunkov <asml.silence@gmail.com>
@@ -10880,6 +10907,12 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git · 824 unmodified lines
10880	F: drivers/ata/pata_arasan_cf.c	10907	F: drivers/ata/pata_arasan_cf.c
10881	F: include/linux/pata_arasan_cf_data.h	10908	F: include/linux/pata_arasan_cf_data.h
10882		10909
		10910	LIBATA PATA DRIVERS
		10911	R: Sergey Shtylyov <s.shtylyov@omp.ru>
		10912	L: linux-ide@vger.kernel.org
		10913	F: drivers/ata/ata_*.c
		10914	F: drivers/ata/pata_*.c
		10915
10883	LIBATA PATA FARADAY FTIDE010 AND GEMINI SATA BRIDGE DRIVERS	10916	LIBATA PATA FARADAY FTIDE010 AND GEMINI SATA BRIDGE DRIVERS
10884	M: Linus Walleij <linus.walleij@linaro.org>	10917	M: Linus Walleij <linus.walleij@linaro.org>
10885	L: linux-ide@vger.kernel.org	10918	L: linux-ide@vger.kernel.org
@@ -12400,7 +12433,7 @@ F: include/uapi/linux/membarrier.h · 1514 unmodified lines
12400	F: kernel/sched/membarrier.c	12433	F: kernel/sched/membarrier.c
12401		12434
12402	MEMBLOCK	12435	MEMBLOCK
12403	M: Mike Rapoport <rppt@linux.ibm.com>	12436	M: Mike Rapoport <rppt@kernel.org>
12404	L: linux-mm@kvack.org	12437	L: linux-mm@kvack.org
12405	S: Maintained	12438	S: Maintained
12406	F: Documentation/core-api/boot-time-mm.rst	12439	F: Documentation/core-api/boot-time-mm.rst
@@ -13298,8 +13331,8 @@ W: http://www.iptables.org/ · 891 unmodified lines
13298	W: http://www.nftables.org/	13331	W: http://www.nftables.org/
13299	Q: http://patchwork.ozlabs.org/project/netfilter-devel/list/	13332	Q: http://patchwork.ozlabs.org/project/netfilter-devel/list/
13300	C: irc://irc.libera.chat/netfilter	13333	C: irc://irc.libera.chat/netfilter
13301	T: git git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf.git	13334	T: git git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf.git
13302	T: git git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next.git	13335	T: git git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf-next.git
13303	F: include/linux/netfilter*	13336	F: include/linux/netfilter*
13304	F: include/linux/netfilter/	13337	F: include/linux/netfilter/
13305	F: include/net/netfilter/	13338	F: include/net/netfilter/
@@ -13566,7 +13599,7 @@ F: tools/testing/selftests/nci/ · 260 unmodified lines
13566		13599
13567	NFS, SUNRPC, AND LOCKD CLIENTS	13600	NFS, SUNRPC, AND LOCKD CLIENTS
13568	M: Trond Myklebust <trond.myklebust@hammerspace.com>	13601	M: Trond Myklebust <trond.myklebust@hammerspace.com>
13569	M: Anna Schumaker <anna.schumaker@netapp.com>	13602	M: Anna Schumaker <anna@kernel.org>
13570	L: linux-nfs@vger.kernel.org	13603	L: linux-nfs@vger.kernel.org
13571	S: Maintained	13604	S: Maintained
13572	W: http://client.linux-nfs.org	13605	W: http://client.linux-nfs.org
@@ -14384,6 +14417,7 @@ M: Rob Herring <robh+dt@kernel.org> · 811 unmodified lines
14384	M: Frank Rowand <frowand.list@gmail.com>	14417	M: Frank Rowand <frowand.list@gmail.com>
14385	L: devicetree@vger.kernel.org	14418	L: devicetree@vger.kernel.org
14386	S: Maintained	14419	S: Maintained
		14420	C: irc://irc.libera.chat/devicetree
14387	W: http://www.devicetree.org/	14421	W: http://www.devicetree.org/
14388	T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git	14422	T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git
14389	F: Documentation/ABI/testing/sysfs-firmware-ofw	14423	F: Documentation/ABI/testing/sysfs-firmware-ofw
@@ -14395,6 +14429,7 @@ OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS · 5 unmodified lines
14395	M: Rob Herring <robh+dt@kernel.org>	14429	M: Rob Herring <robh+dt@kernel.org>
14396	L: devicetree@vger.kernel.org	14430	L: devicetree@vger.kernel.org
14397	S: Maintained	14431	S: Maintained
		14432	C: irc://irc.libera.chat/devicetree
14398	Q: http://patchwork.ozlabs.org/project/devicetree-bindings/list/	14433	Q: http://patchwork.ozlabs.org/project/devicetree-bindings/list/
14399	T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git	14434	T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git
14400	F: Documentation/devicetree/	14435	F: Documentation/devicetree/
@@ -15285,9 +15320,11 @@ PIN CONTROLLER - SAMSUNG · 884 unmodified lines
15285	M: Tomasz Figa <tomasz.figa@gmail.com>	15320	M: Tomasz Figa <tomasz.figa@gmail.com>
15286	M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>	15321	M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
15287	M: Sylwester Nawrocki <s.nawrocki@samsung.com>	15322	M: Sylwester Nawrocki <s.nawrocki@samsung.com>
		15323	R: Alim Akhtar <alim.akhtar@samsung.com>
15288	L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)	15324	L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
15289	L: linux-samsung-soc@vger.kernel.org	15325	L: linux-samsung-soc@vger.kernel.org
15290	S: Maintained	15326	S: Maintained
		15327	C: irc://irc.libera.chat/linux-exynos
15291	Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/	15328	Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/
15292	T: git git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git	15329	T: git git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git
15293	F: Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt	15330	F: Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt
@@ -16469,6 +16506,14 @@ F: Documentation/devicetree/bindings/i2c/renesas,rmobile-iic.yaml · 1175 unmodified lines
16469	F: drivers/i2c/busses/i2c-rcar.c	16506	F: drivers/i2c/busses/i2c-rcar.c
16470	F: drivers/i2c/busses/i2c-sh_mobile.c	16507	F: drivers/i2c/busses/i2c-sh_mobile.c
16471		16508
		16509	RENESAS R-CAR SATA DRIVER
		16510	R: Sergey Shtylyov <s.shtylyov@omp.ru>
		16511	S: Supported
		16512	L: linux-ide@vger.kernel.org
		16513	L: linux-renesas-soc@vger.kernel.org
		16514	F: Documentation/devicetree/bindings/ata/renesas,rcar-sata.yaml
		16515	F: drivers/ata/sata_rcar.c
		16516
16472	RENESAS R-CAR THERMAL DRIVERS	16517	RENESAS R-CAR THERMAL DRIVERS
16473	M: Niklas Söderlund <niklas.soderlund@ragnatech.se>	16518	M: Niklas Söderlund <niklas.soderlund@ragnatech.se>
16474	L: linux-renesas-soc@vger.kernel.org	16519	L: linux-renesas-soc@vger.kernel.org
@@ -16804,8 +16849,8 @@ F: drivers/video/fbdev/savage/ · 329 unmodified lines
16804	S390	16849	S390
16805	M: Heiko Carstens <hca@linux.ibm.com>	16850	M: Heiko Carstens <hca@linux.ibm.com>
16806	M: Vasily Gorbik <gor@linux.ibm.com>	16851	M: Vasily Gorbik <gor@linux.ibm.com>
16807	M: Christian Borntraeger <borntraeger@linux.ibm.com>	16852	M: Alexander Gordeev <agordeev@linux.ibm.com>
16808	R: Alexander Gordeev <agordeev@linux.ibm.com>	16853	R: Christian Borntraeger <borntraeger@linux.ibm.com>
16809	R: Sven Schnelle <svens@linux.ibm.com>	16854	R: Sven Schnelle <svens@linux.ibm.com>
16810	L: linux-s390@vger.kernel.org	16855	L: linux-s390@vger.kernel.org
16811	S: Supported	16856	S: Supported
@@ -17076,6 +17121,7 @@ SAMSUNG SOC CLOCK DRIVERS · 264 unmodified lines
17076	M: Sylwester Nawrocki <s.nawrocki@samsung.com>	17121	M: Sylwester Nawrocki <s.nawrocki@samsung.com>
17077	M: Tomasz Figa <tomasz.figa@gmail.com>	17122	M: Tomasz Figa <tomasz.figa@gmail.com>
17078	M: Chanwoo Choi <cw00.choi@samsung.com>	17123	M: Chanwoo Choi <cw00.choi@samsung.com>
		17124	R: Alim Akhtar <alim.akhtar@samsung.com>
17079	L: linux-samsung-soc@vger.kernel.org	17125	L: linux-samsung-soc@vger.kernel.org
17080	S: Supported	17126	S: Supported
17081	T: git git://git.kernel.org/pub/scm/linux/kernel/git/snawrocki/clk.git	17127	T: git git://git.kernel.org/pub/scm/linux/kernel/git/snawrocki/clk.git
@@ -17712,6 +17758,21 @@ S: Maintained · 630 unmodified lines
17712	W: http://www.winischhofer.at/linuxsisusbvga.shtml	17758	W: http://www.winischhofer.at/linuxsisusbvga.shtml
17713	F: drivers/usb/misc/sisusbvga/	17759	F: drivers/usb/misc/sisusbvga/
17714		17760
		17761	SL28 CPLD MFD DRIVER
		17762	M: Michael Walle <michael@walle.cc>
		17763	S: Maintained
		17764	F: Documentation/devicetree/bindings/gpio/kontron,sl28cpld-gpio.yaml
		17765	F: Documentation/devicetree/bindings/hwmon/kontron,sl28cpld-hwmon.yaml
		17766	F: Documentation/devicetree/bindings/interrupt-controller/kontron,sl28cpld-intc.yaml
		17767	F: Documentation/devicetree/bindings/mfd/kontron,sl28cpld.yaml
		17768	F: Documentation/devicetree/bindings/pwm/kontron,sl28cpld-pwm.yaml
		17769	F: Documentation/devicetree/bindings/watchdog/kontron,sl28cpld-wdt.yaml
		17770	F: drivers/gpio/gpio-sl28cpld.c
		17771	F: drivers/hwmon/sl28cpld-hwmon.c
		17772	F: drivers/irqchip/irq-sl28cpld.c
		17773	F: drivers/pwm/pwm-sl28cpld.c
		17774	F: drivers/watchdog/sl28cpld_wdt.c
		17775
17715	SLAB ALLOCATOR	17776	SLAB ALLOCATOR
17716	M: Christoph Lameter <cl@linux.com>	17777	M: Christoph Lameter <cl@linux.com>
17717	M: Pekka Enberg <penberg@kernel.org>	17778	M: Pekka Enberg <penberg@kernel.org>
@@ -18428,7 +18489,7 @@ F: Documentation/devicetree/bindings/sound/st,sti-asoc-card.txt · 710 unmodified lines
18428	F: sound/soc/sti/	18489	F: sound/soc/sti/
18429		18490
18430	STI CEC DRIVER	18491	STI CEC DRIVER
18431	M: Benjamin Gaignard <benjamin.gaignard@linaro.org>	18492	M: Alain Volmat <alain.volmat@foss.st.com>
18432	S: Maintained	18493	S: Maintained
18433	F: Documentation/devicetree/bindings/media/stih-cec.txt	18494	F: Documentation/devicetree/bindings/media/stih-cec.txt
18434	F: drivers/media/cec/platform/sti/	18495	F: drivers/media/cec/platform/sti/
@@ -19582,6 +19643,14 @@ F: Documentation/trace/timerlat-tracer.rst · 1147 unmodified lines
19582	F: Documentation/trace/hwlat_detector.rst	19643	F: Documentation/trace/hwlat_detector.rst
19583	F: arch/*/kernel/trace.c	19644	F: arch/*/kernel/trace.c
19584		19645
		19646	Real-time Linux Analysis (RTLA) tools
		19647	M: Daniel Bristot de Oliveira <bristot@kernel.org>
		19648	M: Steven Rostedt <rostedt@goodmis.org>
		19649	L: linux-trace-devel@vger.kernel.org
		19650	S: Maintained
		19651	F: Documentation/tools/rtla/
		19652	F: tools/tracing/rtla/
		19653
19585	TRADITIONAL CHINESE DOCUMENTATION	19654	TRADITIONAL CHINESE DOCUMENTATION
19586	M: Hu Haowen <src.res@email.cn>	19655	M: Hu Haowen <src.res@email.cn>
19587	L: linux-doc-tw-discuss@lists.sourceforge.net	19656	L: linux-doc-tw-discuss@lists.sourceforge.net

Makefile+2−2

@@ -2,8 +2,8 @@ · 1 unmodified line
2	VERSION = 5	2	VERSION = 5
3	PATCHLEVEL = 17	3	PATCHLEVEL = 17
4	SUBLEVEL = 0	4	SUBLEVEL = 0
5	EXTRAVERSION = -rc2	5	EXTRAVERSION = -rc4
6	NAME = Gobble Gobble	6	NAME = Superb Owl
7		7
8	# DOCUMENTATION	8	# DOCUMENTATION
9	# To see a list of typical targets execute "make help"	9	# To see a list of typical targets execute "make help"

arch/arm/boot/dts/Makefile+1−0

@@ -806,6 +806,7 @@ dtb-$(CONFIG_ARCH_OMAP3) += \ · 805 unmodified lines
806	logicpd-som-lv-37xx-devkit.dtb \	806	logicpd-som-lv-37xx-devkit.dtb \
807	omap3430-sdp.dtb \	807	omap3430-sdp.dtb \
808	omap3-beagle.dtb \	808	omap3-beagle.dtb \
		809	omap3-beagle-ab4.dtb \
809	omap3-beagle-xm.dtb \	810	omap3-beagle-xm.dtb \
810	omap3-beagle-xm-ab.dtb \	811	omap3-beagle-xm-ab.dtb \
811	omap3-cm-t3517.dtb \	812	omap3-cm-t3517.dtb \

arch/arm/boot/dts/am335x-wega.dtsi+1−1

@@ -55,7 +55,7 @@ · 54 unmodified lines
55	2 1 0 0 /* # 0: INACTIVE, 1: TX, 2: RX */	55	2 1 0 0 /* # 0: INACTIVE, 1: TX, 2: RX */
56	>;	56	>;
57	tx-num-evt = <16>;	57	tx-num-evt = <16>;
58	rt-num-evt = <16>;	58	rx-num-evt = <16>;
59	status = "okay";	59	status = "okay";
60	};	60	};
61		61

arch/arm/boot/dts/dra7.dtsi+10−10

@@ -160,7 +160,7 @@ · 159 unmodified lines
160	target-module@48210000 {	160	target-module@48210000 {
161	compatible = "ti,sysc-omap4-simple", "ti,sysc";	161	compatible = "ti,sysc-omap4-simple", "ti,sysc";
162	power-domains = <&prm_mpu>;	162	power-domains = <&prm_mpu>;
163	clocks = <&mpu_clkctrl DRA7_MPU_CLKCTRL 0>;	163	clocks = <&mpu_clkctrl DRA7_MPU_MPU_CLKCTRL 0>;
164	clock-names = "fck";	164	clock-names = "fck";
165	#address-cells = <1>;	165	#address-cells = <1>;
166	#size-cells = <1>;	166	#size-cells = <1>;
@@ -875,10 +875,10 @@ · 708 unmodified lines
875	<0x58000014 4>;	875	<0x58000014 4>;
876	reg-names = "rev", "syss";	876	reg-names = "rev", "syss";
877	ti,syss-mask = <1>;	877	ti,syss-mask = <1>;
878	clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 0>,	878	clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 0>,
879	<&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 9>,	879	<&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 9>,
880	<&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 10>,	880	<&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 10>,
881	<&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 11>;	881	<&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 11>;
882	clock-names = "fck", "hdmi_clk", "sys_clk", "tv_clk";	882	clock-names = "fck", "hdmi_clk", "sys_clk", "tv_clk";
883	#address-cells = <1>;	883	#address-cells = <1>;
884	#size-cells = <1>;	884	#size-cells = <1>;
@@ -912,7 +912,7 @@ · 27 unmodified lines
912	SYSC_OMAP2_SOFTRESET \|	912	SYSC_OMAP2_SOFTRESET \|
913	SYSC_OMAP2_AUTOIDLE)>;	913	SYSC_OMAP2_AUTOIDLE)>;
914	ti,syss-mask = <1>;	914	ti,syss-mask = <1>;
915	clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 8>;	915	clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 8>;
916	clock-names = "fck";	916	clock-names = "fck";
917	#address-cells = <1>;	917	#address-cells = <1>;
918	#size-cells = <1>;	918	#size-cells = <1>;
@@ -939,8 +939,8 @@ · 20 unmodified lines
939	<SYSC_IDLE_SMART>,	939	<SYSC_IDLE_SMART>,
940	<SYSC_IDLE_SMART_WKUP>;	940	<SYSC_IDLE_SMART_WKUP>;
941	ti,sysc-mask = <(SYSC_OMAP4_SOFTRESET)>;	941	ti,sysc-mask = <(SYSC_OMAP4_SOFTRESET)>;
942	clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 9>,	942	clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 9>,
943	<&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 8>;	943	<&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 8>;
944	clock-names = "fck", "dss_clk";	944	clock-names = "fck", "dss_clk";
945	#address-cells = <1>;	945	#address-cells = <1>;
946	#size-cells = <1>;	946	#size-cells = <1>;
@@ -979,7 +979,7 @@ · 32 unmodified lines
979	compatible = "vivante,gc";	979	compatible = "vivante,gc";
980	reg = <0x0 0x700>;	980	reg = <0x0 0x700>;
981	interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>;	981	interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>;
982	clocks = <&dss_clkctrl DRA7_BB2D_CLKCTRL 0>;	982	clocks = <&dss_clkctrl DRA7_DSS_BB2D_CLKCTRL 0>;
983	clock-names = "core";	983	clock-names = "core";
984	};	984	};
985	};	985	};
@@ -1333,7 +1333,7 @@ · 347 unmodified lines
1333	ti,no-reset-on-init;	1333	ti,no-reset-on-init;
1334	ti,no-idle;	1334	ti,no-idle;
1335	timer@0 {	1335	timer@0 {
1336	assigned-clocks = <&wkupaon_clkctrl DRA7_TIMER1_CLKCTRL 24>;	1336	assigned-clocks = <&wkupaon_clkctrl DRA7_WKUPAON_TIMER1_CLKCTRL 24>;
1337	assigned-clock-parents = <&sys_32k_ck>;	1337	assigned-clock-parents = <&sys_32k_ck>;
1338	};	1338	};
1339	};	1339	};

arch/arm/boot/dts/imx23-evk.dts+0−1

@@ -79,7 +79,6 @@ · 78 unmodified lines
79	MX23_PAD_LCD_RESET__GPIO_1_18	79	MX23_PAD_LCD_RESET__GPIO_1_18
80	MX23_PAD_PWM3__GPIO_1_29	80	MX23_PAD_PWM3__GPIO_1_29
81	MX23_PAD_PWM4__GPIO_1_30	81	MX23_PAD_PWM4__GPIO_1_30
82	MX23_PAD_SSP1_DETECT__SSP1_DETECT
83	>;	82	>;
84	fsl,drive-strength = <MXS_DRIVE_4mA>;	83	fsl,drive-strength = <MXS_DRIVE_4mA>;
85	fsl,voltage = <MXS_VOLTAGE_HIGH>;	84	fsl,voltage = <MXS_VOLTAGE_HIGH>;

arch/arm/boot/dts/imx6qdl-udoo.dtsi+4−1

  * Author: Fabio Estevam <fabio.estevam@freescale.com>
  */
+#include <dt-bindings/gpio/gpio.h>
 / {
 	aliases {
 		backlight = &backlight;
 				MX6QDL_PAD_SD3_DAT1__SD3_DATA1		0x17059
 				MX6QDL_PAD_SD3_DAT2__SD3_DATA2		0x17059
 				MX6QDL_PAD_SD3_DAT3__SD3_DATA3		0x17059
+				MX6QDL_PAD_SD3_DAT5__GPIO7_IO00		0x1b0b0
 			>;
 		};
 &usdhc3 {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_usdhc3>;
-	non-removable;
+	cd-gpios = <&gpio7 0 GPIO_ACTIVE_LOW>;
 	status = "okay";
 };

arch/arm/boot/dts/imx7ulp.dtsi+1−1

@@ -259,7 +259,7 @@ · 258 unmodified lines
259	interrupts = <GIC_SPI 55 IRQ_TYPE_LEVEL_HIGH>;	259	interrupts = <GIC_SPI 55 IRQ_TYPE_LEVEL_HIGH>;
260	clocks = <&pcc2 IMX7ULP_CLK_WDG1>;	260	clocks = <&pcc2 IMX7ULP_CLK_WDG1>;
261	assigned-clocks = <&pcc2 IMX7ULP_CLK_WDG1>;	261	assigned-clocks = <&pcc2 IMX7ULP_CLK_WDG1>;
262	assigned-clocks-parents = <&scg1 IMX7ULP_CLK_FIRC_BUS_CLK>;	262	assigned-clock-parents = <&scg1 IMX7ULP_CLK_FIRC_BUS_CLK>;
263	timeout-sec = <40>;	263	timeout-sec = <40>;
264	};	264	};
265		265

arch/arm/boot/dts/meson.dtsi+4−4

@@ -59,7 +59,7 @@ · 58 unmodified lines
59	};	59	};
60		60
61	uart_A: serial@84c0 {	61	uart_A: serial@84c0 {
62	compatible = "amlogic,meson6-uart", "amlogic,meson-uart";	62	compatible = "amlogic,meson6-uart";
63	reg = <0x84c0 0x18>;	63	reg = <0x84c0 0x18>;
64	interrupts = <GIC_SPI 26 IRQ_TYPE_EDGE_RISING>;	64	interrupts = <GIC_SPI 26 IRQ_TYPE_EDGE_RISING>;
65	fifo-size = <128>;	65	fifo-size = <128>;
@@ -67,7 +67,7 @@ · 1 unmodified line
67	};	67	};
68		68
69	uart_B: serial@84dc {	69	uart_B: serial@84dc {
70	compatible = "amlogic,meson6-uart", "amlogic,meson-uart";	70	compatible = "amlogic,meson6-uart";
71	reg = <0x84dc 0x18>;	71	reg = <0x84dc 0x18>;
72	interrupts = <GIC_SPI 75 IRQ_TYPE_EDGE_RISING>;	72	interrupts = <GIC_SPI 75 IRQ_TYPE_EDGE_RISING>;
73	status = "disabled";	73	status = "disabled";
@@ -105,7 +105,7 @@ · 31 unmodified lines
105	};	105	};
106		106
107	uart_C: serial@8700 {	107	uart_C: serial@8700 {
108	compatible = "amlogic,meson6-uart", "amlogic,meson-uart";	108	compatible = "amlogic,meson6-uart";
109	reg = <0x8700 0x18>;	109	reg = <0x8700 0x18>;
110	interrupts = <GIC_SPI 93 IRQ_TYPE_EDGE_RISING>;	110	interrupts = <GIC_SPI 93 IRQ_TYPE_EDGE_RISING>;
111	status = "disabled";	111	status = "disabled";
@@ -228,7 +228,7 @@ · 116 unmodified lines
228	};	228	};
229		229
230	uart_AO: serial@4c0 {	230	uart_AO: serial@4c0 {
231	compatible = "amlogic,meson6-uart", "amlogic,meson-ao-uart", "amlogic,meson-uart";	231	compatible = "amlogic,meson6-uart", "amlogic,meson-ao-uart";
232	reg = <0x4c0 0x18>;	232	reg = <0x4c0 0x18>;
233	interrupts = <GIC_SPI 90 IRQ_TYPE_EDGE_RISING>;	233	interrupts = <GIC_SPI 90 IRQ_TYPE_EDGE_RISING>;
234	status = "disabled";	234	status = "disabled";

arch/arm/boot/dts/meson8.dtsi+12−12

@@ -736,27 +736,27 @@ · 735 unmodified lines
736	};	736	};
737		737
738	&uart_AO {	738	&uart_AO {
739	compatible = "amlogic,meson8-uart", "amlogic,meson-uart";	739	compatible = "amlogic,meson8-uart", "amlogic,meson-ao-uart";
740	clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_CLK81>;	740	clocks = <&xtal>, <&clkc CLKID_CLK81>, <&clkc CLKID_CLK81>;
741	clock-names = "baud", "xtal", "pclk";	741	clock-names = "xtal", "pclk", "baud";
742	};	742	};
743		743
744	&uart_A {	744	&uart_A {
745	compatible = "amlogic,meson8-uart", "amlogic,meson-uart";	745	compatible = "amlogic,meson8-uart";
746	clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART0>;	746	clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>;
747	clock-names = "baud", "xtal", "pclk";	747	clock-names = "xtal", "pclk", "baud";
748	};	748	};
749		749
750	&uart_B {	750	&uart_B {
751	compatible = "amlogic,meson8-uart", "amlogic,meson-uart";	751	compatible = "amlogic,meson8-uart";
752	clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART1>;	752	clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>;
753	clock-names = "baud", "xtal", "pclk";	753	clock-names = "xtal", "pclk", "baud";
754	};	754	};
755		755
756	&uart_C {	756	&uart_C {
757	compatible = "amlogic,meson8-uart", "amlogic,meson-uart";	757	compatible = "amlogic,meson8-uart";
758	clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART2>;	758	clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>;
759	clock-names = "baud", "xtal", "pclk";	759	clock-names = "xtal", "pclk", "baud";
760	};	760	};
761		761
762	&usb0 {	762	&usb0 {

arch/arm/boot/dts/meson8b.dtsi+12−12

@@ -724,27 +724,27 @@ · 723 unmodified lines
724	};	724	};
725		725
726	&uart_AO {	726	&uart_AO {
727	compatible = "amlogic,meson8b-uart", "amlogic,meson-uart";	727	compatible = "amlogic,meson8b-uart", "amlogic,meson-ao-uart";
728	clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_CLK81>;	728	clocks = <&xtal>, <&clkc CLKID_CLK81>, <&clkc CLKID_CLK81>;
729	clock-names = "baud", "xtal", "pclk";	729	clock-names = "xtal", "pclk", "baud";
730	};	730	};
731		731
732	&uart_A {	732	&uart_A {
733	compatible = "amlogic,meson8b-uart", "amlogic,meson-uart";	733	compatible = "amlogic,meson8b-uart";
734	clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART0>;	734	clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>;
735	clock-names = "baud", "xtal", "pclk";	735	clock-names = "xtal", "pclk", "baud";
736	};	736	};
737		737
738	&uart_B {	738	&uart_B {
739	compatible = "amlogic,meson8b-uart", "amlogic,meson-uart";	739	compatible = "amlogic,meson8b-uart";
740	clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART1>;	740	clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>;
741	clock-names = "baud", "xtal", "pclk";	741	clock-names = "xtal", "pclk", "baud";
742	};	742	};
743		743
744	&uart_C {	744	&uart_C {
745	compatible = "amlogic,meson8b-uart", "amlogic,meson-uart";	745	compatible = "amlogic,meson8b-uart";
746	clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART2>;	746	clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>;
747	clock-names = "baud", "xtal", "pclk";	747	clock-names = "xtal", "pclk", "baud";
748	};	748	};
749		749
750	&usb0 {	750	&usb0 {

arch/arm/boot/dts/omap3-beagle-ab4.dts+47−0

@@ -0,0 +1,47 @@
	1	// SPDX-License-Identifier: GPL-2.0-only
	2	/dts-v1/;
	3
	4	#include "omap3-beagle.dts"
	5
	6	/ {
	7	model = "TI OMAP3 BeagleBoard A to B4";
	8	compatible = "ti,omap3-beagle-ab4", "ti,omap3-beagle", "ti,omap3430", "ti,omap3";
	9	};
	10
	11	/*
	12	* Workaround for capacitor C70 issue, see "Boards revision A and < B5"
	13	* section at https://elinux.org/BeagleBoard_Community
	14	*/
	15
	16	/* Unusable as clocksource because of unreliable oscillator */
	17	&counter32k {
	18	status = "disabled";
	19	};
	20
	21	/* Unusable as clockevent because of unreliable oscillator, allow to idle */
	22	&timer1_target {
	23	/delete-property/ti,no-reset-on-init;
	24	/delete-property/ti,no-idle;
	25	timer@0 {
	26	/delete-property/ti,timer-alwon;
	27	};
	28	};
	29
	30	/* Preferred always-on timer for clocksource */
	31	&timer12_target {
	32	ti,no-reset-on-init;
	33	ti,no-idle;
	34	timer@0 {
	35	/* Always clocked by secure_32k_fck */
	36	};
	37	};
	38
	39	/* Preferred timer for clockevent */
	40	&timer2_target {
	41	ti,no-reset-on-init;
	42	ti,no-idle;
	43	timer@0 {
	44	assigned-clocks = <&gpt2_fck>;
	45	assigned-clock-parents = <&sys_ck>;
	46	};
	47	};

arch/arm/boot/dts/omap3-beagle.dts+0−33

@@ -304,39 +304,6 @@ · 303 unmodified lines
304	phys = <0 &hsusb2_phy>;	304	phys = <0 &hsusb2_phy>;
305	};	305	};
306		306
307	/* Unusable as clocksource because of unreliable oscillator */
308	&counter32k {
309	status = "disabled";
310	};
311
312	/* Unusable as clockevent because if unreliable oscillator, allow to idle */
313	&timer1_target {
314	/delete-property/ti,no-reset-on-init;
315	/delete-property/ti,no-idle;
316	timer@0 {
317	/delete-property/ti,timer-alwon;
318	};
319	};
320
321	/* Preferred always-on timer for clocksource */
322	&timer12_target {
323	ti,no-reset-on-init;
324	ti,no-idle;
325	timer@0 {
326	/* Always clocked by secure_32k_fck */
327	};
328	};
329
330	/* Preferred timer for clockevent */
331	&timer2_target {
332	ti,no-reset-on-init;
333	ti,no-idle;
334	timer@0 {
335	assigned-clocks = <&gpt2_fck>;
336	assigned-clock-parents = <&sys_ck>;
337	};
338	};
339
340	&twl_gpio {	307	&twl_gpio {
341	ti,use-leds;	308	ti,use-leds;
342	/* pullups: BIT(1) */	309	/* pullups: BIT(1) */

arch/arm/boot/dts/spear320-hmi.dts+0−1

@@ -235,7 +235,6 @@ · 234 unmodified lines
235	#address-cells = <1>;	235	#address-cells = <1>;
236	#size-cells = <0>;	236	#size-cells = <0>;
237	reg = <0x41>;	237	reg = <0x41>;
238	irq-over-gpio;
239	irq-gpios = <&gpiopinctrl 29 0x4>;	238	irq-gpios = <&gpiopinctrl 29 0x4>;
240	id = <0>;	239	id = <0>;
241	blocks = <0x5>;	240	blocks = <0x5>;

arch/arm/boot/dts/ste-ux500-samsung-skomer.dts+0−4

 			cap-sd-highspeed;
 			cap-mmc-highspeed;
 			/* All direction control is used */
-			st,sig-dir-cmd;
-			st,sig-dir-dat0;
-			st,sig-dir-dat2;
-			st,sig-dir-dat31;
 			st,sig-pin-fbclk;
 			full-pwr-cycle;
 			vmmc-supply = <&ab8500_ldo_aux3_reg>;

arch/arm/crypto/blake2s-shash.c+2−2

@@ -13,12 +13,12 @@ · 12 unmodified lines
13	static int crypto_blake2s_update_arm(struct shash_desc *desc,	13	static int crypto_blake2s_update_arm(struct shash_desc *desc,
14	const u8 *in, unsigned int inlen)	14	const u8 *in, unsigned int inlen)
15	{	15	{
16	return crypto_blake2s_update(desc, in, inlen, blake2s_compress);	16	return crypto_blake2s_update(desc, in, inlen, false);
17	}	17	}
18		18
19	static int crypto_blake2s_final_arm(struct shash_desc desc, u8 out)	19	static int crypto_blake2s_final_arm(struct shash_desc desc, u8 out)
20	{	20	{
21	return crypto_blake2s_final(desc, out, blake2s_compress);	21	return crypto_blake2s_final(desc, out, false);
22	}	22	}
23		23
24	#define BLAKE2S_ALG(name, driver_name, digest_size) \	24	#define BLAKE2S_ALG(name, driver_name, digest_size) \

arch/arm/mach-omap2/display.c+1−1

 	}
 	r = of_platform_populate(node, NULL, NULL, &pdev->dev);
+	put_device(&pdev->dev);
 	if (r) {
 		pr_err("Unable to populate DSS submodule devices\n");
-		put_device(&pdev->dev);
 		return r;
 	}

arch/arm/mach-omap2/omap_hwmod.c+3−1

 	for_each_matching_node(np, ti_clkctrl_match_table) {
 		ret = _setup_clkctrl_provider(np);
-		if (ret)
+		if (ret) {
+			of_node_put(np);
 			break;
+		}
 	}
 	return ret;

arch/arm/mach-socfpga/Kconfig+2−0

 menuconfig ARCH_INTEL_SOCFPGA
 	bool "Altera SOCFPGA family"
 	depends on ARCH_MULTI_V7
+	select ARCH_HAS_RESET_CONTROLLER
 	select ARCH_SUPPORTS_BIG_ENDIAN
 	select ARM_AMBA
 	select ARM_GIC
 	select PL310_ERRATA_727915
 	select PL310_ERRATA_753970 if PL310
 	select PL310_ERRATA_769419
+	select RESET_CONTROLLER
 if ARCH_INTEL_SOCFPGA
 config SOCFPGA_SUSPEND

arch/arm64/Kconfig+17−0

@@ -672,6 +672,7 @@ config ARM64_WORKAROUND_TRBE_OVERWRITE_FILL_MODE · 671 unmodified lines
672		672
673	config ARM64_ERRATUM_2051678	673	config ARM64_ERRATUM_2051678
674	bool "Cortex-A510: 2051678: disable Hardware Update of the page table dirty bit"	674	bool "Cortex-A510: 2051678: disable Hardware Update of the page table dirty bit"
		675	default y
675	help	676	help
676	This options adds the workaround for ARM Cortex-A510 erratum ARM64_ERRATUM_2051678.	677	This options adds the workaround for ARM Cortex-A510 erratum ARM64_ERRATUM_2051678.
677	Affected Coretex-A510 might not respect the ordering rules for	678	Affected Coretex-A510 might not respect the ordering rules for
@@ -680,6 +681,22 @@ config ARM64_ERRATUM_2051678 · 2 unmodified lines
680		681
681	If unsure, say Y.	682	If unsure, say Y.
682		683
		684	config ARM64_ERRATUM_2077057
		685	bool "Cortex-A510: 2077057: workaround software-step corrupting SPSR_EL2"
		686	help
		687	This option adds the workaround for ARM Cortex-A510 erratum 2077057.
		688	Affected Cortex-A510 may corrupt SPSR_EL2 when the a step exception is
		689	expected, but a Pointer Authentication trap is taken instead. The
		690	erratum causes SPSR_EL1 to be copied to SPSR_EL2, which could allow
		691	EL1 to cause a return to EL2 with a guest controlled ELR_EL2.
		692
		693	This can only happen when EL2 is stepping EL1.
		694
		695	When these conditions occur, the SPSR_EL2 value is unchanged from the
		696	previous guest entry, and can be restored from the in-memory copy.
		697
		698	If unsure, say Y.
		699
683	config ARM64_ERRATUM_2119858	700	config ARM64_ERRATUM_2119858
684	bool "Cortex-A710/X2: 2119858: workaround TRBE overwriting trace data in FILL mode"	701	bool "Cortex-A710/X2: 2119858: workaround TRBE overwriting trace data in FILL mode"
685	default y	702	default y

arch/arm64/Kconfig.platforms+0−3

@@ -309,9 +309,6 @@ config ARCH_VISCONTI · 308 unmodified lines
309	help	309	help
310	This enables support for Toshiba Visconti SoCs Family.	310	This enables support for Toshiba Visconti SoCs Family.
311		311
312	config ARCH_VULCAN
313	def_bool n
314
315	config ARCH_XGENE	312	config ARCH_XGENE
316	bool "AppliedMicro X-Gene SOC Family"	313	bool "AppliedMicro X-Gene SOC Family"
317	help	314	help

arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi+6−0

 			no-map;
 		};
+		/* 32 MiB reserved for ARM Trusted Firmware (BL32) */
+		secmon_reserved_bl32: secmon@5300000 {
+			reg = <0x0 0x05300000 0x0 0x2000000>;
+			no-map;
+		};
 		linux,cma {
 			compatible = "shared-dma-pool";
 			reusable;

arch/arm64/boot/dts/amlogic/meson-g12a-sei510.dts+0−8

@@ -157,14 +157,6 @@ · 156 unmodified lines
157	regulator-always-on;	157	regulator-always-on;
158	};	158	};
159		159
160	reserved-memory {
161	/* TEE Reserved Memory */
162	bl32_reserved: bl32@5000000 {
163	reg = <0x0 0x05300000 0x0 0x2000000>;
164	no-map;
165	};
166	};
167
168	sdio_pwrseq: sdio-pwrseq {	160	sdio_pwrseq: sdio-pwrseq {
169	compatible = "mmc-pwrseq-simple";	161	compatible = "mmc-pwrseq-simple";
170	reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>;	162	reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>;

arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dtsi+2−2

@@ -17,7 +17,7 @@ · 16 unmodified lines
17	rtc1 = &vrtc;	17	rtc1 = &vrtc;
18	};	18	};
19		19
20	dioo2133: audio-amplifier-0 {	20	dio2133: audio-amplifier-0 {
21	compatible = "simple-audio-amplifier";	21	compatible = "simple-audio-amplifier";
22	enable-gpios = <&gpio_ao GPIOAO_2 GPIO_ACTIVE_HIGH>;	22	enable-gpios = <&gpio_ao GPIOAO_2 GPIO_ACTIVE_HIGH>;
23	VCC-supply = <&vcc_5v>;	23	VCC-supply = <&vcc_5v>;
@@ -219,7 +219,7 @@ · 195 unmodified lines
219	audio-widgets = "Line", "Lineout";	219	audio-widgets = "Line", "Lineout";
220	audio-aux-devs = <&tdmout_b>, <&tdmout_c>, <&tdmin_a>,	220	audio-aux-devs = <&tdmout_b>, <&tdmout_c>, <&tdmin_a>,
221	<&tdmin_b>, <&tdmin_c>, <&tdmin_lb>,	221	<&tdmin_b>, <&tdmin_c>, <&tdmin_lb>,
222	<&dioo2133>;	222	<&dio2133>;
223	audio-routing = "TDMOUT_B IN 0", "FRDDR_A OUT 1",	223	audio-routing = "TDMOUT_B IN 0", "FRDDR_A OUT 1",
224	"TDMOUT_B IN 1", "FRDDR_B OUT 1",	224	"TDMOUT_B IN 1", "FRDDR_B OUT 1",
225	"TDMOUT_B IN 2", "FRDDR_C OUT 1",	225	"TDMOUT_B IN 2", "FRDDR_C OUT 1",

arch/arm64/boot/dts/amlogic/meson-gx.dtsi+6−0

 			no-map;
 		};
+		/* 32 MiB reserved for ARM Trusted Firmware (BL32) */
+		secmon_reserved_bl32: secmon@5300000 {
+			reg = <0x0 0x05300000 0x0 0x2000000>;
+			no-map;
+		};
 		linux,cma {
 			compatible = "shared-dma-pool";
 			reusable;

arch/arm64/boot/dts/amlogic/meson-sm1-bananapi-m5.dts+1−1

@@ -123,7 +123,7 @@ · 122 unmodified lines
123	regulator-min-microvolt = <1800000>;	123	regulator-min-microvolt = <1800000>;
124	regulator-max-microvolt = <3300000>;	124	regulator-max-microvolt = <3300000>;
125		125
126	enable-gpio = <&gpio GPIOE_2 GPIO_ACTIVE_HIGH>;	126	enable-gpio = <&gpio_ao GPIOE_2 GPIO_ACTIVE_HIGH>;
127	enable-active-high;	127	enable-active-high;
128	regulator-always-on;	128	regulator-always-on;
129		129

arch/arm64/boot/dts/amlogic/meson-sm1-odroid.dtsi+1−1

@@ -48,7 +48,7 @@ · 47 unmodified lines
48	regulator-max-microvolt = <3300000>;	48	regulator-max-microvolt = <3300000>;
49	vin-supply = <&vcc_5v>;	49	vin-supply = <&vcc_5v>;
50		50
51	enable-gpio = <&gpio GPIOE_2 GPIO_ACTIVE_HIGH>;	51	enable-gpio = <&gpio_ao GPIOE_2 GPIO_OPEN_DRAIN>;
52	enable-active-high;	52	enable-active-high;
53	regulator-always-on;	53	regulator-always-on;
54		54

arch/arm64/boot/dts/amlogic/meson-sm1-sei610.dts+0−8

@@ -203,14 +203,6 @@ · 202 unmodified lines
203	regulator-always-on;	203	regulator-always-on;
204	};	204	};
205		205
206	reserved-memory {
207	/* TEE Reserved Memory */
208	bl32_reserved: bl32@5000000 {
209	reg = <0x0 0x05300000 0x0 0x2000000>;
210	no-map;
211	};
212	};
213
214	sdio_pwrseq: sdio-pwrseq {	206	sdio_pwrseq: sdio-pwrseq {
215	compatible = "mmc-pwrseq-simple";	207	compatible = "mmc-pwrseq-simple";
216	reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>;	208	reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>;

arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28.dts+4−0

 	};
 };
+&ftm_alarm0 {
+	status = "okay";
+};
 &gpio1 {
 	gpio-line-names =
 		"", "", "", "", "", "", "", "",

arch/arm64/boot/dts/freescale/imx8mq-librem5.dtsi+2−2

@@ -1115,8 +1115,8 @@ · 1114 unmodified lines
1115	status = "okay";	1115	status = "okay";
1116		1116
1117	ports {	1117	ports {
1118	port@1 {	1118	port@0 {
1119	reg = <1>;	1119	reg = <0>;
1120		1120
1121	mipi1_sensor_ep: endpoint {	1121	mipi1_sensor_ep: endpoint {
1122	remote-endpoint = <&camera1_ep>;	1122	remote-endpoint = <&camera1_ep>;

arch/arm64/boot/dts/freescale/imx8mq.dtsi+5−5

@@ -554,7 +554,7 @@ · 553 unmodified lines
554	assigned-clock-rates = <0>, <0>, <0>, <594000000>;	554	assigned-clock-rates = <0>, <0>, <0>, <594000000>;
555	status = "disabled";	555	status = "disabled";
556		556
557	port@0 {	557	port {
558	lcdif_mipi_dsi: endpoint {	558	lcdif_mipi_dsi: endpoint {
559	remote-endpoint = <&mipi_dsi_lcdif_in>;	559	remote-endpoint = <&mipi_dsi_lcdif_in>;
560	};	560	};
@@ -1151,8 +1151,8 @@ · 590 unmodified lines
1151	#address-cells = <1>;	1151	#address-cells = <1>;
1152	#size-cells = <0>;	1152	#size-cells = <0>;
1153		1153
1154	port@0 {	1154	port@1 {
1155	reg = <0>;	1155	reg = <1>;
1156		1156
1157	csi1_mipi_ep: endpoint {	1157	csi1_mipi_ep: endpoint {
1158	remote-endpoint = <&csi1_ep>;	1158	remote-endpoint = <&csi1_ep>;
@@ -1203,8 +1203,8 @@ · 44 unmodified lines
1203	#address-cells = <1>;	1203	#address-cells = <1>;
1204	#size-cells = <0>;	1204	#size-cells = <0>;
1205		1205
1206	port@0 {	1206	port@1 {
1207	reg = <0>;	1207	reg = <1>;
1208		1208
1209	csi2_mipi_ep: endpoint {	1209	csi2_mipi_ep: endpoint {
1210	remote-endpoint = <&csi2_ep>;	1210	remote-endpoint = <&csi2_ep>;

arch/arm64/boot/dts/freescale/mba8mx.dtsi+1−1

@@ -91,7 +91,7 @@ · 90 unmodified lines
91		91
92	sound {	92	sound {
93	compatible = "fsl,imx-audio-tlv320aic32x4";	93	compatible = "fsl,imx-audio-tlv320aic32x4";
94	model = "tqm-tlv320aic32";	94	model = "imx-audio-tlv320aic32x4";
95	ssi-controller = <&sai3>;	95	ssi-controller = <&sai3>;
96	audio-codec = <&tlv320aic3x04>;	96	audio-codec = <&tlv320aic3x04>;
97	};	97	};

arch/arm64/boot/dts/ti/k3-j721s2-common-proc-board.dts+12−2

@@ -15,8 +15,18 @@ · 14 unmodified lines
15	model = "Texas Instruments J721S2 EVM";	15	model = "Texas Instruments J721S2 EVM";
16		16
17	chosen {	17	chosen {
18	stdout-path = "serial10:115200n8";	18	stdout-path = "serial2:115200n8";
19	bootargs = "console=ttyS10,115200n8 earlycon=ns16550a,mmio32,2880000";	19	bootargs = "console=ttyS2,115200n8 earlycon=ns16550a,mmio32,2880000";
		20	};
		21
		22	aliases {
		23	serial1 = &mcu_uart0;
		24	serial2 = &main_uart8;
		25	mmc0 = &main_sdhci0;
		26	mmc1 = &main_sdhci1;
		27	can0 = &main_mcan16;
		28	can1 = &mcu_mcan0;
		29	can2 = &mcu_mcan1;
20	};	30	};
21		31
22	evm_12v0: fixedregulator-evm12v0 {	32	evm_12v0: fixedregulator-evm12v0 {

arch/arm64/boot/dts/ti/k3-j721s2.dtsi+0−22

@@ -21,28 +21,6 @@ · 20 unmodified lines
21	#address-cells = <2>;	21	#address-cells = <2>;
22	#size-cells = <2>;	22	#size-cells = <2>;
23		23
24	aliases {
25	serial0 = &wkup_uart0;
26	serial1 = &mcu_uart0;
27	serial2 = &main_uart0;
28	serial3 = &main_uart1;
29	serial4 = &main_uart2;
30	serial5 = &main_uart3;
31	serial6 = &main_uart4;
32	serial7 = &main_uart5;
33	serial8 = &main_uart6;
34	serial9 = &main_uart7;
35	serial10 = &main_uart8;
36	serial11 = &main_uart9;
37	mmc0 = &main_sdhci0;
38	mmc1 = &main_sdhci1;
39	can0 = &main_mcan16;
40	can1 = &mcu_mcan0;
41	can2 = &mcu_mcan1;
42	can3 = &main_mcan3;
43	can4 = &main_mcan5;
44	};
45
46	chosen { };	24	chosen { };
47		25
48	cpus {	26	cpus {

arch/arm64/kernel/cpu_errata.c+8−0

@@ -600,6 +600,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = { · 599 unmodified lines
600	CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus),	600	CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus),
601	},	601	},
602	#endif	602	#endif
		603	#ifdef CONFIG_ARM64_ERRATUM_2077057
		604	{
		605	.desc = "ARM erratum 2077057",
		606	.capability = ARM64_WORKAROUND_2077057,
		607	.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
		608	ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A510, 0, 0, 2),
		609	},
		610	#endif
603	#ifdef CONFIG_ARM64_ERRATUM_2064142	611	#ifdef CONFIG_ARM64_ERRATUM_2064142
604	{	612	{
605	.desc = "ARM erratum 2064142",	613	.desc = "ARM erratum 2064142",

arch/arm64/kvm/arm.c+33−18

 			xfer_to_guest_mode_work_pending();
 }
+/*
+ * Actually run the vCPU, entering an RCU extended quiescent state (EQS) while
+ * the vCPU is running.
+ *
+ * This must be noinstr as instrumentation may make use of RCU, and this is not
+ * safe during the EQS.
+ */
+static int noinstr kvm_arm_vcpu_enter_exit(struct kvm_vcpu *vcpu)
+{
+	int ret;
+	guest_state_enter_irqoff();
+	ret = kvm_call_hyp_ret(__kvm_vcpu_run, vcpu);
+	guest_state_exit_irqoff();
+	return ret;
+}
 /**
  * kvm_arch_vcpu_ioctl_run - the main VCPU run function to execute guest code
  * @vcpu:	The VCPU pointer
 		 * Enter the guest
 		 */
 		trace_kvm_entry(*vcpu_pc(vcpu));
-		guest_enter_irqoff();
+		guest_timing_enter_irqoff();
-		ret = kvm_call_hyp_ret(__kvm_vcpu_run, vcpu);
+		ret = kvm_arm_vcpu_enter_exit(vcpu);
 		vcpu->mode = OUTSIDE_GUEST_MODE;
 		vcpu->stat.exits++;
 		kvm_arch_vcpu_ctxsync_fp(vcpu);
 		/*
-		 * We may have taken a host interrupt in HYP mode (ie
+		 * We must ensure that any pending interrupts are taken before
-		 * while executing the guest). This interrupt is still
+		 * we exit guest timing so that timer ticks are accounted as
-		 * pending, as we haven't serviced it yet!
+		 * guest time. Transiently unmask interrupts so that any
+		 * pending interrupts are taken.
 		 *
-		 * We're now back in SVC mode, with interrupts
+		 * Per ARM DDI 0487G.b section D1.13.4, an ISB (or other
-		 * disabled.  Enabling the interrupts now will have
+		 * context synchronization event) is necessary to ensure that
-		 * the effect of taking the interrupt again, in SVC
+		 * pending interrupts are taken.
-		 * mode this time.
 		 */
 		local_irq_enable();
+		isb();
+		local_irq_disable();
+		guest_timing_exit_irqoff();
+		local_irq_enable();
-		/*
-		 * We do local_irq_enable() before calling guest_exit() so
-		 * that if a timer interrupt hits while running the guest we
-		 * account that tick as being spent in the guest.  We enable
-		 * preemption after calling guest_exit() so that if we get
-		 * preempted we make sure ticks after that is not counted as
-		 * guest time.
-		 */
-		guest_exit();
 		trace_kvm_exit(ret, kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu));
 		/* Exit types that need handling before we can be preempted */

arch/arm64/kvm/handle_exit.c+8−0

@@ -228,6 +228,14 @@ int handle_exit(struct kvm_vcpu *vcpu, int exception_index) · 227 unmodified lines
228	{	228	{
229	struct kvm_run *run = vcpu->run;	229	struct kvm_run *run = vcpu->run;
230		230
		231	if (ARM_SERROR_PENDING(exception_index)) {
		232	/*
		233	* The SError is handled by handle_exit_early(). If the guest
		234	* survives it will re-execute the original instruction.
		235	*/
		236	return 1;
		237	}
		238
231	exception_index = ARM_EXCEPTION_CODE(exception_index);	239	exception_index = ARM_EXCEPTION_CODE(exception_index);
232		240
233	switch (exception_index) {	241	switch (exception_index) {

arch/arm64/kvm/hyp/include/hyp/switch.h+21−2

 	return false;
 }
+static inline void synchronize_vcpu_pstate(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
+	/*
+	 * Check for the conditions of Cortex-A510's #2077057. When these occur
+	 * SPSR_EL2 can't be trusted, but isn't needed either as it is
+	 * unchanged from the value in vcpu_gp_regs(vcpu)->pstate.
+	 * Are we single-stepping the guest, and took a PAC exception from the
+	 * active-not-pending state?
+	 */
+	if (cpus_have_final_cap(ARM64_WORKAROUND_2077057)		&&
+	    vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP			&&
+	    *vcpu_cpsr(vcpu) & DBG_SPSR_SS				&&
+	    ESR_ELx_EC(read_sysreg_el2(SYS_ESR)) == ESR_ELx_EC_PAC)
+		write_sysreg_el2(*vcpu_cpsr(vcpu), SYS_SPSR);
+	vcpu->arch.ctxt.regs.pstate = read_sysreg_el2(SYS_SPSR);
+}
 /*
  * Return true when we were able to fixup the guest exit and should return to
  * the guest, false when we should restore the host state and return to the
 	 * Save PSTATE early so that we can evaluate the vcpu mode
 	 * early on.
 	 */
-	vcpu->arch.ctxt.regs.pstate = read_sysreg_el2(SYS_SPSR);
+	synchronize_vcpu_pstate(vcpu, exit_code);
 	/*
 	 * Check whether we want to repaint the state one way or
 	if (ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ)
 		vcpu->arch.fault.esr_el2 = read_sysreg_el2(SYS_ESR);
-	if (ARM_SERROR_PENDING(*exit_code)) {
+	if (ARM_SERROR_PENDING(*exit_code) &&
+	    ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ) {
 		u8 esr_ec = kvm_vcpu_trap_get_class(vcpu);
 		/*

arch/arm64/tools/cpucaps+3−2

 WORKAROUND_1463225
 WORKAROUND_1508412
 WORKAROUND_1542419
-WORKAROUND_2064142
-WORKAROUND_2038923
 WORKAROUND_1902691
+WORKAROUND_2038923
+WORKAROUND_2064142
+WORKAROUND_2077057
 WORKAROUND_TRBE_OVERWRITE_FILL_MODE
 WORKAROUND_TSB_FLUSH_FAILURE
 WORKAROUND_TRBE_WRITE_OUT_OF_RANGE

arch/mips/boot/dts/ingenic/ci20.dts+2−13

 		label = "HDMI OUT";
 		type = "a";
+		ddc-en-gpios = <&gpa 25 GPIO_ACTIVE_HIGH>;
 		port {
 			hdmi_con: endpoint {
 				remote-endpoint = <&dw_hdmi_out>;
 		gpio = <&gpf 14 GPIO_ACTIVE_LOW>;
 		enable-active-high;
 	};
-	hdmi_power: fixedregulator@3 {
-		compatible = "regulator-fixed";
-		regulator-name = "hdmi_power";
-		regulator-min-microvolt = <5000000>;
-		regulator-max-microvolt = <5000000>;
-		gpio = <&gpa 25 0>;
-		enable-active-high;
-	};
 };
 &ext {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pins_hdmi_ddc>;
-	hdmi-5v-supply = <&hdmi_power>;
 	ports {
 		#address-cells = <1>;
 		#size-cells = <0>;

arch/mips/cavium-octeon/octeon-memcpy.S+1−1

@@ -74,7 +74,7 @@ · 73 unmodified lines
74	#define EXC(inst_reg,addr,handler) \	74	#define EXC(inst_reg,addr,handler) \
75	9: inst_reg, addr; \	75	9: inst_reg, addr; \
76	.section __ex_table,"a"; \	76	.section __ex_table,"a"; \
77	PTR 9b, handler; \	77	PTR_WD 9b, handler; \
78	.previous	78	.previous
79		79
80	/*	80	/*

arch/mips/kvm/mips.c+46−4

@@ -414,6 +414,24 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu, · 413 unmodified lines
414	return -ENOIOCTLCMD;	414	return -ENOIOCTLCMD;
415	}	415	}
416		416
		417	/*
		418	* Actually run the vCPU, entering an RCU extended quiescent state (EQS) while
		419	* the vCPU is running.
		420	*
		421	* This must be noinstr as instrumentation may make use of RCU, and this is not
		422	* safe during the EQS.
		423	*/
		424	static int noinstr kvm_mips_vcpu_enter_exit(struct kvm_vcpu *vcpu)
		425	{
		426	int ret;
		427
		428	guest_state_enter_irqoff();
		429	ret = kvm_mips_callbacks->vcpu_run(vcpu);
		430	guest_state_exit_irqoff();
		431
		432	return ret;
		433	}
		434
417	int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)	435	int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
418	{	436	{
419	int r = -EINTR;	437	int r = -EINTR;
@@ -434,7 +452,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 14 unmodified lines
434	lose_fpu(1);	452	lose_fpu(1);
435		453
436	local_irq_disable();	454	local_irq_disable();
437	guest_enter_irqoff();	455	guest_timing_enter_irqoff();
438	trace_kvm_enter(vcpu);	456	trace_kvm_enter(vcpu);
439		457
440	/*	458	/*
@@ -445,10 +463,23 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 4 unmodified lines
445	*/	463	*/
446	smp_store_mb(vcpu->mode, IN_GUEST_MODE);	464	smp_store_mb(vcpu->mode, IN_GUEST_MODE);
447		465
448	r = kvm_mips_callbacks->vcpu_run(vcpu);	466	r = kvm_mips_vcpu_enter_exit(vcpu);
		467
		468	/*
		469	* We must ensure that any pending interrupts are taken before
		470	* we exit guest timing so that timer ticks are accounted as
		471	* guest time. Transiently unmask interrupts so that any
		472	* pending interrupts are taken.
		473	*
		474	* TODO: is there a barrier which ensures that pending interrupts are
		475	* recognised? Currently this just hopes that the CPU takes any pending
		476	* interrupts between the enable and disable.
		477	*/
		478	local_irq_enable();
		479	local_irq_disable();
449		480
450	trace_kvm_out(vcpu);	481	trace_kvm_out(vcpu);
451	guest_exit_irqoff();	482	guest_timing_exit_irqoff();
452	local_irq_enable();	483	local_irq_enable();
453		484
454	out:	485	out:
@@ -1168,7 +1199,7 @@ static void kvm_mips_set_c0_status(void) · 713 unmodified lines
1168	/*	1199	/*
1169	* Return value is in the form (errcode<<2 \| RESUME_FLAG_HOST \| RESUME_FLAG_NV)	1200	* Return value is in the form (errcode<<2 \| RESUME_FLAG_HOST \| RESUME_FLAG_NV)
1170	*/	1201	*/
1171	int kvm_mips_handle_exit(struct kvm_vcpu *vcpu)	1202	static int __kvm_mips_handle_exit(struct kvm_vcpu *vcpu)
1172	{	1203	{
1173	struct kvm_run *run = vcpu->run;	1204	struct kvm_run *run = vcpu->run;
1174	u32 cause = vcpu->arch.host_cp0_cause;	1205	u32 cause = vcpu->arch.host_cp0_cause;
@@ -1357,6 +1388,17 @@ int kvm_mips_handle_exit(struct kvm_vcpu *vcpu) · 182 unmodified lines
1357	return ret;	1388	return ret;
1358	}	1389	}
1359		1390
		1391	int noinstr kvm_mips_handle_exit(struct kvm_vcpu *vcpu)
		1392	{
		1393	int ret;
		1394
		1395	guest_state_exit_irqoff();
		1396	ret = __kvm_mips_handle_exit(vcpu);
		1397	guest_state_enter_irqoff();
		1398
		1399	return ret;
		1400	}
		1401
1360	/* Enable FPU for guest and restore context */	1402	/* Enable FPU for guest and restore context */
1361	void kvm_own_fpu(struct kvm_vcpu *vcpu)	1403	void kvm_own_fpu(struct kvm_vcpu *vcpu)
1362	{	1404	{

arch/mips/kvm/vz.c+9−3

 /**
  * _kvm_vz_save_htimer() - Switch to software emulation of guest timer.
  * @vcpu:	Virtual CPU.
- * @compare:	Pointer to write compare value to.
+ * @out_compare: Pointer to write compare value to.
- * @cause:	Pointer to write cause value to.
+ * @out_cause:	Pointer to write cause value to.
  *
  * Save VZ guest timer state and switch to software emulation of guest CP0
  * timer. The hard timer must already be in use, so preemption should be
 }
 /**
- * kvm_trap_vz_handle_cop_unusuable() - Guest used unusable coprocessor.
+ * kvm_trap_vz_handle_cop_unusable() - Guest used unusable coprocessor.
  * @vcpu:	Virtual CPU context.
  *
  * Handle when the guest attempts to use a coprocessor which hasn't been allowed
  * by the root context.
+ *
+ * Return: value indicating whether to resume the host or the guest
+ * 	   (RESUME_HOST or RESUME_GUEST)
  */
 static int kvm_trap_vz_handle_cop_unusable(struct kvm_vcpu *vcpu)
 {
  *
  * Handle when the guest attempts to use MSA when it is disabled in the root
  * context.
+ *
+ * Return: value indicating whether to resume the host or the guest
+ * 	   (RESUME_HOST or RESUME_GUEST)
  */
 static int kvm_trap_vz_handle_msa_disabled(struct kvm_vcpu *vcpu)
 {

arch/riscv/Makefile+6−0

 riscv-march-$(CONFIG_ARCH_RV64I)	:= rv64ima
 riscv-march-$(CONFIG_FPU)		:= $(riscv-march-y)fd
 riscv-march-$(CONFIG_RISCV_ISA_C)	:= $(riscv-march-y)c
+# Newer binutils versions default to ISA spec version 20191213 which moves some
+# instructions from the I extension to the Zicsr and Zifencei extensions.
+toolchain-need-zicsr-zifencei := $(call cc-option-yn, -march=$(riscv-march-y)_zicsr_zifencei)
+riscv-march-$(toolchain-need-zicsr-zifencei) := $(riscv-march-y)_zicsr_zifencei
 KBUILD_CFLAGS += -march=$(subst fd,,$(riscv-march-y))
 KBUILD_AFLAGS += -march=$(riscv-march-y)

arch/riscv/kernel/cpu-hotplug.c+2−0

 #include <linux/sched/hotplug.h>
 #include <asm/irq.h>
 #include <asm/cpu_ops.h>
+#include <asm/numa.h>
 #include <asm/sbi.h>
 bool cpu_has_hotplug(unsigned int cpu)
 		return ret;
 	remove_cpu_topology(cpu);
+	numa_remove_cpu(cpu);
 	set_cpu_online(cpu, false);
 	irq_migrate_all_off_this_cpu();

arch/riscv/kernel/head.S+5−6

 	add \reg, \reg, t0
 .endm
 .macro XIP_FIXUP_FLASH_OFFSET reg
-	la t1, __data_loc
+	la t0, __data_loc
-	li t0, XIP_OFFSET_MASK
+	REG_L t1, _xip_phys_offset
-	and t1, t1, t0
+	sub \reg, \reg, t1
-	li t1, XIP_OFFSET
+	add \reg, \reg, t0
-	sub t0, t0, t1
-	sub \reg, \reg, t0
 .endm
 _xip_fixup: .dword CONFIG_PHYS_RAM_BASE - CONFIG_XIP_PHYS_ADDR - XIP_OFFSET
+_xip_phys_offset: .dword CONFIG_XIP_PHYS_ADDR + XIP_OFFSET
 #else
 .macro XIP_FIXUP_OFFSET reg
 .endm

arch/riscv/kernel/stacktrace.c+5−4

 			     bool (*fn)(void *, unsigned long), void *arg)
 {
 	unsigned long fp, sp, pc;
+	int level = 0;
 	if (regs) {
 		fp = frame_pointer(regs);
 		sp = user_stack_pointer(regs);
 		pc = instruction_pointer(regs);
 	} else if (task == NULL || task == current) {
-		fp = (unsigned long)__builtin_frame_address(1);
+		fp = (unsigned long)__builtin_frame_address(0);
-		sp = (unsigned long)__builtin_frame_address(0);
+		sp = sp_in_global;
-		pc = (unsigned long)__builtin_return_address(0);
+		pc = (unsigned long)walk_stackframe;
 	} else {
 		/* task blocked in __switch_to */
 		fp = task->thread.s[0];
 		unsigned long low, high;
 		struct stackframe *frame;
-		if (unlikely(!__kernel_text_address(pc) || !fn(arg, pc)))
+		if (unlikely(!__kernel_text_address(pc) || (level++ >= 1 && !fn(arg, pc))))
 			break;
 		/* Validate frame pointer */

arch/riscv/kvm/vcpu.c+31−17

@@ -90,6 +90,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) · 89 unmodified lines
90	int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)	90	int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
91	{	91	{
92	struct kvm_cpu_context *cntx;	92	struct kvm_cpu_context *cntx;
		93	struct kvm_vcpu_csr *reset_csr = &vcpu->arch.guest_reset_csr;
93		94
94	/* Mark this VCPU never ran */	95	/* Mark this VCPU never ran */
95	vcpu->arch.ran_atleast_once = false;	96	vcpu->arch.ran_atleast_once = false;
@@ -106,6 +107,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · 10 unmodified lines
106	cntx->hstatus \|= HSTATUS_SPVP;	107	cntx->hstatus \|= HSTATUS_SPVP;
107	cntx->hstatus \|= HSTATUS_SPV;	108	cntx->hstatus \|= HSTATUS_SPV;
108		109
		110	/* By default, make CY, TM, and IR counters accessible in VU mode */
		111	reset_csr->scounteren = 0x7;
		112
109	/* Setup VCPU timer */	113	/* Setup VCPU timer */
110	kvm_riscv_vcpu_timer_init(vcpu);	114	kvm_riscv_vcpu_timer_init(vcpu);
111		115
@@ -699,6 +703,20 @@ static void kvm_riscv_update_hvip(struct kvm_vcpu *vcpu) · 587 unmodified lines
699	csr_write(CSR_HVIP, csr->hvip);	703	csr_write(CSR_HVIP, csr->hvip);
700	}	704	}
701		705
		706	/*
		707	* Actually run the vCPU, entering an RCU extended quiescent state (EQS) while
		708	* the vCPU is running.
		709	*
		710	* This must be noinstr as instrumentation may make use of RCU, and this is not
		711	* safe during the EQS.
		712	*/
		713	static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu)
		714	{
		715	guest_state_enter_irqoff();
		716	__kvm_riscv_switch_to(&vcpu->arch);
		717	guest_state_exit_irqoff();
		718	}
		719
702	int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)	720	int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
703	{	721	{
704	int ret;	722	int ret;
@@ -790,9 +808,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 85 unmodified lines
790	continue;	808	continue;
791	}	809	}
792		810
793	guest_enter_irqoff();	811	guest_timing_enter_irqoff();
794		812
795	__kvm_riscv_switch_to(&vcpu->arch);	813	kvm_riscv_vcpu_enter_exit(vcpu);
796		814
797	vcpu->mode = OUTSIDE_GUEST_MODE;	815	vcpu->mode = OUTSIDE_GUEST_MODE;
798	vcpu->stat.exits++;	816	vcpu->stat.exits++;
@@ -812,25 +830,21 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 13 unmodified lines
812	kvm_riscv_vcpu_sync_interrupts(vcpu);	830	kvm_riscv_vcpu_sync_interrupts(vcpu);
813		831
814	/*	832	/*
815	* We may have taken a host interrupt in VS/VU-mode (i.e.	833	* We must ensure that any pending interrupts are taken before
816	* while executing the guest). This interrupt is still	834	* we exit guest timing so that timer ticks are accounted as
817	* pending, as we haven't serviced it yet!	835	* guest time. Transiently unmask interrupts so that any
		836	* pending interrupts are taken.
818	*	837	*
819	* We're now back in HS-mode with interrupts disabled	838	* There's no barrier which ensures that pending interrupts are
820	* so enabling the interrupts now will have the effect	839	* recognised, so we just hope that the CPU takes any pending
821	* of taking the interrupt again, in HS-mode this time.	840	* interrupts between the enable and disable.
822	*/	841	*/
823	local_irq_enable();	842	local_irq_enable();
		843	local_irq_disable();
824		844
825	/*	845	guest_timing_exit_irqoff();
826	* We do local_irq_enable() before calling guest_exit() so	846
827	* that if a timer interrupt hits while running the guest	847	local_irq_enable();
828	* we account that tick as being spent in the guest. We
829	* enable preemption after calling guest_exit() so that if
830	* we get preempted we make sure ticks after that is not
831	* counted as guest time.
832	*/
833	guest_exit();
834		848
835	preempt_enable();	849	preempt_enable();
836		850

arch/riscv/kvm/vcpu_sbi_base.c+2−1

 #include <linux/errno.h>
 #include <linux/err.h>
 #include <linux/kvm_host.h>
+#include <linux/version.h>
 #include <asm/csr.h>
 #include <asm/sbi.h>
 #include <asm/kvm_vcpu_timer.h>
 		*out_val = KVM_SBI_IMPID;
 		break;
 	case SBI_EXT_BASE_GET_IMP_VERSION:
-		*out_val = 0;
+		*out_val = LINUX_VERSION_CODE;
 		break;
 	case SBI_EXT_BASE_PROBE_EXT:
 		if ((cp->a0 >= SBI_EXT_EXPERIMENTAL_START &&

arch/riscv/mm/extable.c+3−3

@@ -33,7 +33,7 @@ static inline void regs_set_gpr(struct pt_regs *regs, unsigned int offset, · 32 unmodified lines
33	if (unlikely(offset > MAX_REG_OFFSET))	33	if (unlikely(offset > MAX_REG_OFFSET))
34	return;	34	return;
35		35
36	if (!offset)	36	if (offset)
37	(unsigned long )((unsigned long)regs + offset) = val;	37	(unsigned long )((unsigned long)regs + offset) = val;
38	}	38	}
39		39
@@ -43,8 +43,8 @@ static bool ex_handler_uaccess_err_zero(const struct exception_table_entry *ex, · 3 unmodified lines
43	int reg_err = FIELD_GET(EX_DATA_REG_ERR, ex->data);	43	int reg_err = FIELD_GET(EX_DATA_REG_ERR, ex->data);
44	int reg_zero = FIELD_GET(EX_DATA_REG_ZERO, ex->data);	44	int reg_zero = FIELD_GET(EX_DATA_REG_ZERO, ex->data);
45		45
46	regs_set_gpr(regs, reg_err, -EFAULT);	46	regs_set_gpr(regs, reg_err * sizeof(unsigned long), -EFAULT);
47	regs_set_gpr(regs, reg_zero, 0);	47	regs_set_gpr(regs, reg_zero * sizeof(unsigned long), 0);
48		48
49	regs->epc = get_ex_fixup(ex);	49	regs->epc = get_ex_fixup(ex);
50	return true;	50	return true;

arch/riscv/mm/init.c+2−0

 #ifdef CONFIG_XIP_KERNEL
 #define pt_ops			(*(struct pt_alloc_ops *)XIP_FIXUP(&pt_ops))
+#define riscv_pfn_base         (*(unsigned long  *)XIP_FIXUP(&riscv_pfn_base))
 #define trampoline_pg_dir      ((pgd_t *)XIP_FIXUP(trampoline_pg_dir))
 #define fixmap_pte             ((pte_t *)XIP_FIXUP(fixmap_pte))
 #define early_pg_dir           ((pgd_t *)XIP_FIXUP(early_pg_dir))
 }
 #ifdef CONFIG_XIP_KERNEL
+#define phys_ram_base  (*(phys_addr_t *)XIP_FIXUP(&phys_ram_base))
 extern char _xiprom[], _exiprom[], __data_loc;
 /* called from head.S with MMU off */

arch/s390/kvm/kvm-s390.c+2−0

@@ -4667,6 +4667,8 @@ static long kvm_s390_guest_sida_op(struct kvm_vcpu *vcpu, · 4666 unmodified lines
4667	return -EINVAL;	4667	return -EINVAL;
4668	if (mop->size + mop->sida_offset > sida_size(vcpu->arch.sie_block))	4668	if (mop->size + mop->sida_offset > sida_size(vcpu->arch.sie_block))
4669	return -E2BIG;	4669	return -E2BIG;
		4670	if (!kvm_s390_pv_cpu_is_protected(vcpu))
		4671	return -EINVAL;
4670		4672
4671	switch (mop->op) {	4673	switch (mop->op) {
4672	case KVM_S390_MEMOP_SIDA_READ:	4674	case KVM_S390_MEMOP_SIDA_READ:

arch/s390/lib/test_modules.c+0−3

@@ -5,9 +5,6 @@ · 4 unmodified lines
5		5
6	#include "test_modules.h"	6	#include "test_modules.h"
7		7
8	#define DECLARE_RETURN(i) int test_modules_return_ ## i(void)
9	REPEAT_10000(DECLARE_RETURN);
10
11	/*	8	/*
12	* Test that modules with many relocations are loaded properly.	9	* Test that modules with many relocations are loaded properly.
13	*/	10	*/

arch/s390/lib/test_modules.h+3−0

@@ -47,4 +47,7 @@ · 46 unmodified lines
47	__REPEAT_10000_1(f, 8); \	47	__REPEAT_10000_1(f, 8); \
48	__REPEAT_10000_1(f, 9)	48	__REPEAT_10000_1(f, 9)
49		49
		50	#define DECLARE_RETURN(i) int test_modules_return_ ## i(void)
		51	REPEAT_10000(DECLARE_RETURN);
		52
50	#endif	53	#endif

arch/x86/crypto/blake2s-shash.c+2−2

@@ -18,12 +18,12 @@ · 17 unmodified lines
18	static int crypto_blake2s_update_x86(struct shash_desc *desc,	18	static int crypto_blake2s_update_x86(struct shash_desc *desc,
19	const u8 *in, unsigned int inlen)	19	const u8 *in, unsigned int inlen)
20	{	20	{
21	return crypto_blake2s_update(desc, in, inlen, blake2s_compress);	21	return crypto_blake2s_update(desc, in, inlen, false);
22	}	22	}
23		23
24	static int crypto_blake2s_final_x86(struct shash_desc desc, u8 out)	24	static int crypto_blake2s_final_x86(struct shash_desc desc, u8 out)
25	{	25	{
26	return crypto_blake2s_final(desc, out, blake2s_compress);	26	return crypto_blake2s_final(desc, out, false);
27	}	27	}
28		28
29	#define BLAKE2S_ALG(name, driver_name, digest_size) \	29	#define BLAKE2S_ALG(name, driver_name, digest_size) \

arch/x86/events/intel/core.c+13−0

@@ -4703,6 +4703,19 @@ static __initconst const struct x86_pmu intel_pmu = { · 4702 unmodified lines
4703	.lbr_read = intel_pmu_lbr_read_64,	4703	.lbr_read = intel_pmu_lbr_read_64,
4704	.lbr_save = intel_pmu_lbr_save,	4704	.lbr_save = intel_pmu_lbr_save,
4705	.lbr_restore = intel_pmu_lbr_restore,	4705	.lbr_restore = intel_pmu_lbr_restore,
		4706
		4707	/*
		4708	* SMM has access to all 4 rings and while traditionally SMM code only
		4709	* ran in CPL0, 2021-era firmware is starting to make use of CPL3 in SMM.
		4710	*
		4711	* Since the EVENTSEL.{USR,OS} CPL filtering makes no distinction
		4712	* between SMM or not, this results in what should be pure userspace
		4713	* counters including SMM data.
		4714	*
		4715	* This is a clear privilege issue, therefore globally disable
		4716	* counting SMM by default.
		4717	*/
		4718	.attr_freeze_on_smi = 1,
4706	};	4719	};
4707		4720
4708	static __init void intel_clovertown_quirk(void)	4721	static __init void intel_clovertown_quirk(void)

arch/x86/events/intel/pt.c+3−2

 		 * means we are already losing data; need to let the decoder
 		 * know.
 		 */
-		if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries) ||
+		if (!buf->single &&
-		    buf->output_off == pt_buffer_region_size(buf)) {
+		    (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries) ||
+		     buf->output_off == pt_buffer_region_size(buf))) {
 			perf_aux_output_flag(&pt->handle,
 			                     PERF_AUX_FLAG_TRUNCATED);
 			advance++;

arch/x86/include/asm/bug.h+11−9

 #ifdef CONFIG_DEBUG_BUGVERBOSE
-#define _BUG_FLAGS(ins, flags)						\
+#define _BUG_FLAGS(ins, flags, extra)					\
 do {									\
 	asm_inline volatile("1:\t" ins "\n"				\
 		     ".pushsection __bug_table,\"aw\"\n"		\
 		     "\t.word %c1"        "\t# bug_entry::line\n"	\
 		     "\t.word %c2"        "\t# bug_entry::flags\n"	\
 		     "\t.org 2b+%c3\n"					\
-		     ".popsection"					\
+		     ".popsection\n"					\
+		     extra						\
 		     : : "i" (__FILE__), "i" (__LINE__),		\
 			 "i" (flags),					\
 			 "i" (sizeof(struct bug_entry)));		\
 #else /* !CONFIG_DEBUG_BUGVERBOSE */
-#define _BUG_FLAGS(ins, flags)						\
+#define _BUG_FLAGS(ins, flags, extra)					\
 do {									\
 	asm_inline volatile("1:\t" ins "\n"				\
 		     ".pushsection __bug_table,\"aw\"\n"		\
 		     "2:\t" __BUG_REL(1b) "\t# bug_entry::bug_addr\n"	\
 		     "\t.word %c0"        "\t# bug_entry::flags\n"	\
 		     "\t.org 2b+%c1\n"					\
-		     ".popsection"					\
+		     ".popsection\n"					\
+		     extra						\
 		     : : "i" (flags),					\
 			 "i" (sizeof(struct bug_entry)));		\
 } while (0)
 #else
-#define _BUG_FLAGS(ins, flags)  asm volatile(ins)
+#define _BUG_FLAGS(ins, flags, extra)  asm volatile(ins)
 #endif /* CONFIG_GENERIC_BUG */
 #define BUG()							\
 do {								\
 	instrumentation_begin();				\
-	_BUG_FLAGS(ASM_UD2, 0);					\
+	_BUG_FLAGS(ASM_UD2, 0, "");				\
-	unreachable();						\
+	__builtin_unreachable();				\
 } while (0)
 /*
  */
 #define __WARN_FLAGS(flags)					\
 do {								\
+	__auto_type f = BUGFLAG_WARNING|(flags);		\
 	instrumentation_begin();				\
-	_BUG_FLAGS(ASM_UD2, BUGFLAG_WARNING|(flags));		\
+	_BUG_FLAGS(ASM_UD2, f, ASM_REACHABLE);			\
-	annotate_reachable();					\
 	instrumentation_end();					\
 } while (0)

arch/x86/include/asm/kvm-x86-ops.h+1−1

@@ -82,7 +82,7 @@ KVM_X86_OP_NULL(guest_apic_has_interrupt) · 81 unmodified lines
82	KVM_X86_OP(load_eoi_exitmap)	82	KVM_X86_OP(load_eoi_exitmap)
83	KVM_X86_OP(set_virtual_apic_mode)	83	KVM_X86_OP(set_virtual_apic_mode)
84	KVM_X86_OP_NULL(set_apic_access_page_addr)	84	KVM_X86_OP_NULL(set_apic_access_page_addr)
85	KVM_X86_OP(deliver_posted_interrupt)	85	KVM_X86_OP(deliver_interrupt)
86	KVM_X86_OP_NULL(sync_pir_to_irr)	86	KVM_X86_OP_NULL(sync_pir_to_irr)
87	KVM_X86_OP(set_tss_addr)	87	KVM_X86_OP(set_tss_addr)
88	KVM_X86_OP(set_identity_map_addr)	88	KVM_X86_OP(set_identity_map_addr)

arch/x86/include/asm/kvm_host.h+2−1

 	void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap);
 	void (*set_virtual_apic_mode)(struct kvm_vcpu *vcpu);
 	void (*set_apic_access_page_addr)(struct kvm_vcpu *vcpu);
-	int (*deliver_posted_interrupt)(struct kvm_vcpu *vcpu, int vector);
+	void (*deliver_interrupt)(struct kvm_lapic *apic, int delivery_mode,
+				  int trig_mode, int vector);
 	int (*sync_pir_to_irr)(struct kvm_vcpu *vcpu);
 	int (*set_tss_addr)(struct kvm *kvm, unsigned int addr);
 	int (*set_identity_map_addr)(struct kvm *kvm, u64 ident_addr);

arch/x86/include/asm/xen/cpuid.h+7−0

@@ -100,6 +100,13 @@ · 99 unmodified lines
100	/* Memory mapped from other domains has valid IOMMU entries */	100	/* Memory mapped from other domains has valid IOMMU entries */
101	#define XEN_HVM_CPUID_IOMMU_MAPPINGS (1u << 2)	101	#define XEN_HVM_CPUID_IOMMU_MAPPINGS (1u << 2)
102	#define XEN_HVM_CPUID_VCPU_ID_PRESENT (1u << 3) /* vcpu id is present in EBX */	102	#define XEN_HVM_CPUID_VCPU_ID_PRESENT (1u << 3) /* vcpu id is present in EBX */
		103	#define XEN_HVM_CPUID_DOMID_PRESENT (1u << 4) /* domid is present in ECX */
		104	/*
		105	* Bits 55:49 from the IO-APIC RTE and bits 11:5 from the MSI address can be
		106	* used to store high bits for the Destination ID. This expands the Destination
		107	* ID field from 8 to 15 bits, allowing to target APIC IDs up 32768.
		108	*/
		109	#define XEN_HVM_CPUID_EXT_DEST_ID (1u << 5)
103		110
104	/*	111	/*
105	* Leaf 6 (0x40000x05)	112	* Leaf 6 (0x40000x05)

arch/x86/include/asm/xen/hypervisor.h+0−14

@@ -43,20 +43,6 @@ static inline uint32_t xen_cpuid_base(void) · 42 unmodified lines
43	return hypervisor_cpuid_base("XenVMMXenVMM", 2);	43	return hypervisor_cpuid_base("XenVMMXenVMM", 2);
44	}	44	}
45		45
46	#ifdef CONFIG_XEN
47	extern bool __init xen_hvm_need_lapic(void);
48
49	static inline bool __init xen_x2apic_para_available(void)
50	{
51	return xen_hvm_need_lapic();
52	}
53	#else
54	static inline bool __init xen_x2apic_para_available(void)
55	{
56	return (xen_cpuid_base() != 0);
57	}
58	#endif
59
60	struct pci_dev;	46	struct pci_dev;
61		47
62	#ifdef CONFIG_XEN_PV_DOM0	48	#ifdef CONFIG_XEN_PV_DOM0

arch/x86/kernel/cpu/sgx/encl.c+2−0

@@ -410,6 +410,8 @@ void sgx_encl_release(struct kref *ref) · 409 unmodified lines
410	}	410	}
411		411
412	kfree(entry);	412	kfree(entry);
		413	/* Invoke scheduler to prevent soft lockups. */
		414	cond_resched();
413	}	415	}
414		416
415	xa_destroy(&encl->page_array);	417	xa_destroy(&encl->page_array);

arch/x86/kernel/early-quirks.c+1−0

@@ -555,6 +555,7 @@ static const struct pci_device_id intel_early_ids[] __initconst = { · 554 unmodified lines
555	INTEL_RKL_IDS(&gen11_early_ops),	555	INTEL_RKL_IDS(&gen11_early_ops),
556	INTEL_ADLS_IDS(&gen11_early_ops),	556	INTEL_ADLS_IDS(&gen11_early_ops),
557	INTEL_ADLP_IDS(&gen11_early_ops),	557	INTEL_ADLP_IDS(&gen11_early_ops),
		558	INTEL_ADLN_IDS(&gen11_early_ops),
558	INTEL_RPLS_IDS(&gen11_early_ops),	559	INTEL_RPLS_IDS(&gen11_early_ops),
559	};	560	};
560		561

arch/x86/kernel/resource.c+1−22

@@ -1,5 +1,4 @@
1	// SPDX-License-Identifier: GPL-2.0	1	// SPDX-License-Identifier: GPL-2.0
2	#include <linux/dmi.h>
3	#include <linux/ioport.h>	2	#include <linux/ioport.h>
4	#include <asm/e820/api.h>	3	#include <asm/e820/api.h>
5		4
@@ -24,31 +23,11 @@ static void resource_clip(struct resource *res, resource_size_t start, · 18 unmodified lines
24	res->start = end + 1;	23	res->start = end + 1;
25	}	24	}
26		25
27	/*
28	* Some BIOS-es contain a bug where they add addresses which map to
29	* system RAM in the PCI host bridge window returned by the ACPI _CRS
30	* method, see commit 4dc2287c1805 ("x86: avoid E820 regions when
31	* allocating address space"). To avoid this Linux by default excludes
32	* E820 reservations when allocating addresses since 2010.
33	* In 2019 some systems have shown-up with E820 reservations which cover
34	* the entire _CRS returned PCI host bridge window, causing all attempts
35	* to assign memory to PCI BARs to fail if Linux uses E820 reservations.
36	*
37	* Ideally Linux would fully stop using E820 reservations, but then
38	* the old systems this was added for will regress.
39	* Instead keep the old behavior for old systems, while ignoring the
40	* E820 reservations for any systems from now on.
41	*/
42	static void remove_e820_regions(struct resource *avail)	26	static void remove_e820_regions(struct resource *avail)
43	{	27	{
44	int i, year = dmi_get_bios_year();	28	int i;
45	struct e820_entry *entry;	29	struct e820_entry *entry;
46		30
47	if (year >= 2018)
48	return;
49
50	pr_info_once("PCI: Removing E820 reservations from host bridge windows\n");
51
52	for (i = 0; i < e820_table->nr_entries; i++) {	31	for (i = 0; i < e820_table->nr_entries; i++) {
53	entry = &e820_table->entries[i];	32	entry = &e820_table->entries[i];
54		33

arch/x86/kvm/cpuid.c+7−6

@@ -554,12 +554,13 @@ void kvm_set_cpu_caps(void) · 553 unmodified lines
554	);	554	);
555		555
556	kvm_cpu_cap_mask(CPUID_7_0_EBX,	556	kvm_cpu_cap_mask(CPUID_7_0_EBX,
557	F(FSGSBASE) \| F(SGX) \| F(BMI1) \| F(HLE) \| F(AVX2) \| F(SMEP) \|	557	F(FSGSBASE) \| F(SGX) \| F(BMI1) \| F(HLE) \| F(AVX2) \|
558	F(BMI2) \| F(ERMS) \| F(INVPCID) \| F(RTM) \| 0 /MPX/ \| F(RDSEED) \|	558	F(FDP_EXCPTN_ONLY) \| F(SMEP) \| F(BMI2) \| F(ERMS) \| F(INVPCID) \|
559	F(ADX) \| F(SMAP) \| F(AVX512IFMA) \| F(AVX512F) \| F(AVX512PF) \|	559	F(RTM) \| F(ZERO_FCS_FDS) \| 0 /MPX/ \| F(AVX512F) \|
560	F(AVX512ER) \| F(AVX512CD) \| F(CLFLUSHOPT) \| F(CLWB) \| F(AVX512DQ) \|	560	F(AVX512DQ) \| F(RDSEED) \| F(ADX) \| F(SMAP) \| F(AVX512IFMA) \|
561	F(SHA_NI) \| F(AVX512BW) \| F(AVX512VL) \| 0 /INTEL_PT/	561	F(CLFLUSHOPT) \| F(CLWB) \| 0 /INTEL_PT/ \| F(AVX512PF) \|
562	);	562	F(AVX512ER) \| F(AVX512CD) \| F(SHA_NI) \| F(AVX512BW) \|
		563	F(AVX512VL));
563		564
564	kvm_cpu_cap_mask(CPUID_7_ECX,	565	kvm_cpu_cap_mask(CPUID_7_ECX,
565	F(AVX512VBMI) \| F(LA57) \| F(PKU) \| 0 /OSPKE/ \| F(RDPID) \|	566	F(AVX512VBMI) \| F(LA57) \| F(PKU) \| 0 /OSPKE/ \| F(RDPID) \|

arch/x86/kvm/lapic.c+2−8

 						       apic->regs + APIC_TMR);
 		}
-		if (static_call(kvm_x86_deliver_posted_interrupt)(vcpu, vector)) {
+		static_call(kvm_x86_deliver_interrupt)(apic, delivery_mode,
-			kvm_lapic_set_irr(vector, apic);
+						       trig_mode, vector);
-			kvm_make_request(KVM_REQ_EVENT, vcpu);
-			kvm_vcpu_kick(vcpu);
-		} else {
-			trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode,
-						   trig_mode, vector);
-		}
 		break;
 	case APIC_DM_REMRD:

arch/x86/kvm/svm/svm.c+18−3

 		SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR;
 }
+static void svm_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
+				  int trig_mode, int vector)
+{
+	struct kvm_vcpu *vcpu = apic->vcpu;
+	if (svm_deliver_avic_intr(vcpu, vector)) {
+		kvm_lapic_set_irr(vector, apic);
+		kvm_make_request(KVM_REQ_EVENT, vcpu);
+		kvm_vcpu_kick(vcpu);
+	} else {
+		trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode,
+					   trig_mode, vector);
+	}
+}
 static void svm_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 	struct vcpu_svm *svm = to_svm(vcpu);
 	unsigned long vmcb_pa = svm->current_vmcb->pa;
-	kvm_guest_enter_irqoff();
+	guest_state_enter_irqoff();
 	if (sev_es_guest(vcpu->kvm)) {
 		__svm_sev_es_vcpu_run(vmcb_pa);
 		vmload(__sme_page_pa(sd->save_area));
 	}
-	kvm_guest_exit_irqoff();
+	guest_state_exit_irqoff();
 }
 static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)
 	.pmu_ops = &amd_pmu_ops,
 	.nested_ops = &svm_nested_ops,
-	.deliver_posted_interrupt = svm_deliver_avic_intr,
+	.deliver_interrupt = svm_deliver_interrupt,
 	.dy_apicv_has_pending_interrupt = svm_dy_apicv_has_pending_interrupt,
 	.update_pi_irte = svm_update_pi_irte,
 	.setup_mce = svm_setup_mce,

arch/x86/kvm/vmx/vmx.c+18−3

 	return 0;
 }
+static void vmx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
+				  int trig_mode, int vector)
+{
+	struct kvm_vcpu *vcpu = apic->vcpu;
+	if (vmx_deliver_posted_interrupt(vcpu, vector)) {
+		kvm_lapic_set_irr(vector, apic);
+		kvm_make_request(KVM_REQ_EVENT, vcpu);
+		kvm_vcpu_kick(vcpu);
+	} else {
+		trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode,
+					   trig_mode, vector);
+	}
+}
 /*
  * Set up the vmcs's constant host-state fields, i.e., host-state fields that
  * will not change in the lifetime of the guest.
 static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,
 					struct vcpu_vmx *vmx)
 {
-	kvm_guest_enter_irqoff();
+	guest_state_enter_irqoff();
 	/* L1D Flush includes CPU buffer clear to mitigate MDS */
 	if (static_branch_unlikely(&vmx_l1d_should_flush))
 	vcpu->arch.cr2 = native_read_cr2();
-	kvm_guest_exit_irqoff();
+	guest_state_exit_irqoff();
 }
 static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)
 	.hwapic_isr_update = vmx_hwapic_isr_update,
 	.guest_apic_has_interrupt = vmx_guest_apic_has_interrupt,
 	.sync_pir_to_irr = vmx_sync_pir_to_irr,
-	.deliver_posted_interrupt = vmx_deliver_posted_interrupt,
+	.deliver_interrupt = vmx_deliver_interrupt,
 	.dy_apicv_has_pending_interrupt = pi_has_pending_interrupt,
 	.set_tss_addr = vmx_set_tss_addr,

arch/x86/kvm/x86.c+6−4

 u64 __read_mostly kvm_mce_cap_supported = MCG_CTL_P | MCG_SER_P;
 EXPORT_SYMBOL_GPL(kvm_mce_cap_supported);
+#define  ERR_PTR_USR(e)  ((void __user *)ERR_PTR(e))
 #define emul_to_vcpu(ctxt) \
 	((struct kvm_vcpu *)(ctxt)->vcpu)
 	void __user *uaddr = (void __user*)(unsigned long)attr->addr;
 	if ((u64)(unsigned long)uaddr != attr->addr)
-		return ERR_PTR(-EFAULT);
+		return ERR_PTR_USR(-EFAULT);
 	return uaddr;
 }
 		set_debugreg(0, 7);
 	}
+	guest_timing_enter_irqoff();
 	for (;;) {
 		/*
 		 * Assert that vCPU vs. VM APICv state is consistent.  An APICv
 	 * of accounting via context tracking, but the loss of accuracy is
 	 * acceptable for all known use cases.
 	 */
-	vtime_account_guest_exit();
+	guest_timing_exit_irqoff();
 	if (lapic_in_kernel(vcpu)) {
 		s64 delta = vcpu->arch.apic->lapic_timer.advance_expire_delta;
 	kvm_free_pit(kvm);
 }
-#define  ERR_PTR_USR(e)  ((void __user *)ERR_PTR(e))
 /**
  * __x86_set_memory_region: Setup KVM internal memory slot
  *

arch/x86/kvm/x86.h+0−45

@@ -10,51 +10,6 @@ · 9 unmodified lines
10		10
11	void kvm_spurious_fault(void);	11	void kvm_spurious_fault(void);
12		12
13	static __always_inline void kvm_guest_enter_irqoff(void)
14	{
15	/*
16	* VMENTER enables interrupts (host state), but the kernel state is
17	* interrupts disabled when this is invoked. Also tell RCU about
18	* it. This is the same logic as for exit_to_user_mode().
19	*
20	* This ensures that e.g. latency analysis on the host observes
21	* guest mode as interrupt enabled.
22	*
23	* guest_enter_irqoff() informs context tracking about the
24	* transition to guest mode and if enabled adjusts RCU state
25	* accordingly.
26	*/
27	instrumentation_begin();
28	trace_hardirqs_on_prepare();
29	lockdep_hardirqs_on_prepare(CALLER_ADDR0);
30	instrumentation_end();
31
32	guest_enter_irqoff();
33	lockdep_hardirqs_on(CALLER_ADDR0);
34	}
35
36	static __always_inline void kvm_guest_exit_irqoff(void)
37	{
38	/*
39	* VMEXIT disables interrupts (host state), but tracing and lockdep
40	* have them in state 'on' as recorded before entering guest mode.
41	* Same as enter_from_user_mode().
42	*
43	* context_tracking_guest_exit() restores host context and reinstates
44	* RCU if enabled and required.
45	*
46	* This needs to be done immediately after VM-Exit, before any code
47	* that might contain tracepoints or call out to the greater world,
48	* e.g. before x86_spec_ctrl_restore_host().
49	*/
50	lockdep_hardirqs_off(CALLER_ADDR0);
51	context_tracking_guest_exit();
52
53	instrumentation_begin();
54	trace_hardirqs_off_finish();
55	instrumentation_end();
56	}
57
58	#define KVM_NESTED_VMENTER_CONSISTENCY_CHECK(consistency_check) \	13	#define KVM_NESTED_VMENTER_CONSISTENCY_CHECK(consistency_check) \
59	({ \	14	({ \
60	bool failed = (consistency_check); \	15	bool failed = (consistency_check); \

arch/x86/xen/enlighten_hvm.c+11−11

@@ -9,6 +9,7 @@ · 8 unmodified lines
9	#include <xen/events.h>	9	#include <xen/events.h>
10	#include <xen/interface/memory.h>	10	#include <xen/interface/memory.h>
11		11
		12	#include <asm/apic.h>
12	#include <asm/cpu.h>	13	#include <asm/cpu.h>
13	#include <asm/smp.h>	14	#include <asm/smp.h>
14	#include <asm/io_apic.h>	15	#include <asm/io_apic.h>
@@ -184,8 +185,7 @@ static int xen_cpu_dead_hvm(unsigned int cpu) · 169 unmodified lines
184		185
185	if (xen_have_vector_callback && xen_feature(XENFEAT_hvm_safe_pvclock))	186	if (xen_have_vector_callback && xen_feature(XENFEAT_hvm_safe_pvclock))
186	xen_teardown_timer(cpu);	187	xen_teardown_timer(cpu);
187		188	return 0;
188	return 0;
189	}	189	}
190		190
191	static bool no_vector_callback __initdata;	191	static bool no_vector_callback __initdata;
@@ -242,15 +242,14 @@ static __init int xen_parse_no_vector_callback(char *arg) · 50 unmodified lines
242	}	242	}
243	early_param("xen_no_vector_callback", xen_parse_no_vector_callback);	243	early_param("xen_no_vector_callback", xen_parse_no_vector_callback);
244		244
245	bool __init xen_hvm_need_lapic(void)	245	static __init bool xen_x2apic_available(void)
246	{	246	{
247	if (xen_pv_domain())	247	return x2apic_supported();
248	return false;	248	}
249	if (!xen_hvm_domain())	249
250	return false;	250	static bool __init msi_ext_dest_id(void)
251	if (xen_feature(XENFEAT_hvm_pirqs) && xen_have_vector_callback)	251	{
252	return false;	252	return cpuid_eax(xen_cpuid_base() + 4) & XEN_HVM_CPUID_EXT_DEST_ID;
253	return true;
254	}	253	}
255		254
256	static __init void xen_hvm_guest_late_init(void)	255	static __init void xen_hvm_guest_late_init(void)
@@ -312,9 +311,10 @@ struct hypervisor_x86 x86_hyper_xen_hvm __initdata = { · 55 unmodified lines
312	.detect = xen_platform_hvm,	311	.detect = xen_platform_hvm,
313	.type = X86_HYPER_XEN_HVM,	312	.type = X86_HYPER_XEN_HVM,
314	.init.init_platform = xen_hvm_guest_init,	313	.init.init_platform = xen_hvm_guest_init,
315	.init.x2apic_available = xen_x2apic_para_available,	314	.init.x2apic_available = xen_x2apic_available,
316	.init.init_mem_mapping = xen_hvm_init_mem_mapping,	315	.init.init_mem_mapping = xen_hvm_init_mem_mapping,
317	.init.guest_late_init = xen_hvm_guest_late_init,	316	.init.guest_late_init = xen_hvm_guest_late_init,
		317	.init.msi_ext_dest_id = msi_ext_dest_id,
318	.runtime.pin_vcpu = xen_pin_vcpu,	318	.runtime.pin_vcpu = xen_pin_vcpu,
319	.ignore_nopv = true,	319	.ignore_nopv = true,
320	};	320	};

arch/x86/xen/enlighten_pv.c+0−4

@@ -1341,10 +1341,6 @@ asmlinkage __visible void __init xen_start_kernel(void) · 1340 unmodified lines
1341		1341
1342	xen_acpi_sleep_register();	1342	xen_acpi_sleep_register();
1343		1343
1344	/* Avoid searching for BIOS MP tables */
1345	x86_init.mpparse.find_smp_config = x86_init_noop;
1346	x86_init.mpparse.get_smp_config = x86_init_uint_noop;
1347
1348	xen_boot_params_init_edd();	1344	xen_boot_params_init_edd();
1349		1345
1350	#ifdef CONFIG_ACPI	1346	#ifdef CONFIG_ACPI

arch/x86/xen/smp_pv.c+6−20

@@ -148,28 +148,12 @@ int xen_smp_intr_init_pv(unsigned int cpu) · 147 unmodified lines
148	return rc;	148	return rc;
149	}	149	}
150		150
151	static void __init xen_fill_possible_map(void)	151	static void __init _get_smp_config(unsigned int early)
152	{
153	int i, rc;
154
155	if (xen_initial_domain())
156	return;
157
158	for (i = 0; i < nr_cpu_ids; i++) {
159	rc = HYPERVISOR_vcpu_op(VCPUOP_is_up, i, NULL);
160	if (rc >= 0) {
161	num_processors++;
162	set_cpu_possible(i, true);
163	}
164	}
165	}
166
167	static void __init xen_filter_cpu_maps(void)
168	{	152	{
169	int i, rc;	153	int i, rc;
170	unsigned int subtract = 0;	154	unsigned int subtract = 0;
171		155
172	if (!xen_initial_domain())	156	if (early)
173	return;	157	return;
174		158
175	num_processors = 0;	159	num_processors = 0;
@@ -210,7 +194,6 @@ static void __init xen_pv_smp_prepare_boot_cpu(void) · 34 unmodified lines
210	* sure the old memory can be recycled. */	194	* sure the old memory can be recycled. */
211	make_lowmem_page_readwrite(xen_initial_gdt);	195	make_lowmem_page_readwrite(xen_initial_gdt);
212		196
213	xen_filter_cpu_maps();
214	xen_setup_vcpu_info_placement();	197	xen_setup_vcpu_info_placement();
215		198
216	/*	199	/*
@@ -476,5 +459,8 @@ static const struct smp_ops xen_smp_ops __initconst = { · 259 unmodified lines
476	void __init xen_smp_init(void)	459	void __init xen_smp_init(void)
477	{	460	{
478	smp_ops = xen_smp_ops;	461	smp_ops = xen_smp_ops;
479	xen_fill_possible_map();	462
		463	/* Avoid searching for BIOS MP tables */
		464	x86_init.mpparse.find_smp_config = x86_init_noop;
		465	x86_init.mpparse.get_smp_config = _get_smp_config;
480	}	466	}

arch/x86/xen/vga.c+8−8

 		screen_info->rsvd_size = info->u.vesa_lfb.rsvd_size;
 		screen_info->rsvd_pos = info->u.vesa_lfb.rsvd_pos;
+		if (size >= offsetof(struct dom0_vga_console_info,
+				     u.vesa_lfb.ext_lfb_base)
+		    + sizeof(info->u.vesa_lfb.ext_lfb_base)
+		    && info->u.vesa_lfb.ext_lfb_base) {
+			screen_info->ext_lfb_base = info->u.vesa_lfb.ext_lfb_base;
+			screen_info->capabilities |= VIDEO_CAPABILITY_64BIT_BASE;
+		}
 		if (info->video_type == XEN_VGATYPE_EFI_LFB) {
 			screen_info->orig_video_isVGA = VIDEO_TYPE_EFI;
 			break;
 				     u.vesa_lfb.mode_attrs)
 		    + sizeof(info->u.vesa_lfb.mode_attrs))
 			screen_info->vesa_attributes = info->u.vesa_lfb.mode_attrs;
-		if (size >= offsetof(struct dom0_vga_console_info,
-				     u.vesa_lfb.ext_lfb_base)
-		    + sizeof(info->u.vesa_lfb.ext_lfb_base)
-		    && info->u.vesa_lfb.ext_lfb_base) {
-			screen_info->ext_lfb_base = info->u.vesa_lfb.ext_lfb_base;
-			screen_info->capabilities |= VIDEO_CAPABILITY_64BIT_BASE;
-		}
 		break;
 	}
 }

block/bio-integrity.c+1−1

@@ -373,7 +373,7 @@ void bio_integrity_advance(struct bio *bio, unsigned int bytes_done) · 372 unmodified lines
373	struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk);	373	struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk);
374	unsigned bytes = bio_integrity_bytes(bi, bytes_done >> 9);	374	unsigned bytes = bio_integrity_bytes(bi, bytes_done >> 9);
375		375
376	bip->bip_iter.bi_sector += bytes_done >> 9;	376	bip->bip_iter.bi_sector += bio_integrity_intervals(bi, bytes_done >> 9);
377	bvec_iter_advance(bip->bip_vec, &bip->bip_iter, bytes);	377	bvec_iter_advance(bip->bip_vec, &bip->bip_iter, bytes);
378	}	378	}
379		379

block/fops.c+19−14

 {
 	struct block_device *bdev = iocb->ki_filp->private_data;
 	loff_t size = bdev_nr_bytes(bdev);
-	size_t count = iov_iter_count(to);
 	loff_t pos = iocb->ki_pos;
 	size_t shorted = 0;
 	ssize_t ret = 0;
+	size_t count;
-	if (unlikely(pos + count > size)) {
+	if (unlikely(pos + iov_iter_count(to) > size)) {
 		if (pos >= size)
 			return 0;
 		size -= pos;
-		if (count > size) {
+		shorted = iov_iter_count(to) - size;
-			shorted = count - size;
+		iov_iter_truncate(to, size);
-			iov_iter_truncate(to, size);
-		}
 	}
+	count = iov_iter_count(to);
+	if (!count)
+		goto reexpand; /* skip atime */
 	if (iocb->ki_flags & IOCB_DIRECT) {
 		struct address_space *mapping = iocb->ki_filp->f_mapping;
 		if (iocb->ki_flags & IOCB_NOWAIT) {
-			if (filemap_range_needs_writeback(mapping, iocb->ki_pos,
+			if (filemap_range_needs_writeback(mapping, pos,
-						iocb->ki_pos + count - 1))
+							  pos + count - 1)) {
-				return -EAGAIN;
+				ret = -EAGAIN;
+				goto reexpand;
+			}
 		} else {
-			ret = filemap_write_and_wait_range(mapping,
+			ret = filemap_write_and_wait_range(mapping, pos,
-						iocb->ki_pos,
+							   pos + count - 1);
-					        iocb->ki_pos + count - 1);
 			if (ret < 0)
-				return ret;
+				goto reexpand;
 		}
 		file_accessed(iocb->ki_filp);
 			iocb->ki_pos += ret;
 			count -= ret;
 		}
+		iov_iter_revert(to, count - iov_iter_count(to));
 		if (ret < 0 || !count)
-			return ret;
+			goto reexpand;
 	}
 	ret = filemap_read(iocb, to, ret);
+reexpand:
 	if (unlikely(shorted))
 		iov_iter_reexpand(to, iov_iter_count(to) + shorted);
 	return ret;

crypto/algapi.c+1−0

@@ -1324,3 +1324,4 @@ module_exit(crypto_algapi_exit); · 1323 unmodified lines
1324		1324
1325	MODULE_LICENSE("GPL");	1325	MODULE_LICENSE("GPL");
1326	MODULE_DESCRIPTION("Cryptographic algorithms API");	1326	MODULE_DESCRIPTION("Cryptographic algorithms API");
		1327	MODULE_SOFTDEP("pre: cryptomgr");

crypto/api.c+0−1

@@ -643,4 +643,3 @@ EXPORT_SYMBOL_GPL(crypto_req_done); · 642 unmodified lines
643		643
644	MODULE_DESCRIPTION("Cryptographic core API");	644	MODULE_DESCRIPTION("Cryptographic core API");
645	MODULE_LICENSE("GPL");	645	MODULE_LICENSE("GPL");
646	MODULE_SOFTDEP("pre: cryptomgr");

crypto/blake2s_generic.c+2−2

@@ -15,12 +15,12 @@ · 14 unmodified lines
15	static int crypto_blake2s_update_generic(struct shash_desc *desc,	15	static int crypto_blake2s_update_generic(struct shash_desc *desc,
16	const u8 *in, unsigned int inlen)	16	const u8 *in, unsigned int inlen)
17	{	17	{
18	return crypto_blake2s_update(desc, in, inlen, blake2s_compress_generic);	18	return crypto_blake2s_update(desc, in, inlen, true);
19	}	19	}
20		20
21	static int crypto_blake2s_final_generic(struct shash_desc desc, u8 out)	21	static int crypto_blake2s_final_generic(struct shash_desc desc, u8 out)
22	{	22	{
23	return crypto_blake2s_final(desc, out, blake2s_compress_generic);	23	return crypto_blake2s_final(desc, out, true);
24	}	24	}
25		25
26	#define BLAKE2S_ALG(name, driver_name, digest_size) \	26	#define BLAKE2S_ALG(name, driver_name, digest_size) \

drivers/accessibility/speakup/speakup_dectlk.c+1−0

@@ -44,6 +44,7 @@ static struct var_t vars[] = { · 43 unmodified lines
44	{ CAPS_START, .u.s = {"[:dv ap 160] " } },	44	{ CAPS_START, .u.s = {"[:dv ap 160] " } },
45	{ CAPS_STOP, .u.s = {"[:dv ap 100 ] " } },	45	{ CAPS_STOP, .u.s = {"[:dv ap 100 ] " } },
46	{ RATE, .u.n = {"[:ra %d] ", 180, 75, 650, 0, 0, NULL } },	46	{ RATE, .u.n = {"[:ra %d] ", 180, 75, 650, 0, 0, NULL } },
		47	{ PITCH, .u.n = {"[:dv ap %d] ", 122, 50, 350, 0, 0, NULL } },
47	{ INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } },	48	{ INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } },
48	{ VOL, .u.n = {"[:dv g5 %d] ", 86, 60, 86, 0, 0, NULL } },	49	{ VOL, .u.n = {"[:dv g5 %d] ", 86, 60, 86, 0, 0, NULL } },
49	{ PUNCT, .u.n = {"[:pu %c] ", 0, 0, 2, 0, 0, "nsa" } },	50	{ PUNCT, .u.n = {"[:pu %c] ", 0, 0, 2, 0, 0, "nsa" } },

drivers/acpi/Kconfig+1−0

@@ -11,6 +11,7 @@ menuconfig ACPI · 10 unmodified lines
11	depends on ARCH_SUPPORTS_ACPI	11	depends on ARCH_SUPPORTS_ACPI
12	select PNP	12	select PNP
13	select NLS	13	select NLS
		14	select CRC32
14	default y if X86	15	default y if X86
15	help	16	help
16	Advanced Configuration and Power Interface (ACPI) support for	17	Advanced Configuration and Power Interface (ACPI) support for

drivers/acpi/arm64/iort.c+11−3

 	res[0].start = pmcg->page0_base_address;
 	res[0].end = pmcg->page0_base_address + SZ_4K - 1;
 	res[0].flags = IORESOURCE_MEM;
-	res[1].start = pmcg->page1_base_address;
+	/*
-	res[1].end = pmcg->page1_base_address + SZ_4K - 1;
+	 * The initial version in DEN0049C lacked a way to describe register
-	res[1].flags = IORESOURCE_MEM;
+	 * page 1, which makes it broken for most PMCG implementations; in
+	 * that case, just let the driver fail gracefully if it expects to
+	 * find a second memory resource.
+	 */
+	if (node->revision > 0) {
+		res[1].start = pmcg->page1_base_address;
+		res[1].end = pmcg->page1_base_address + SZ_4K - 1;
+		res[1].flags = IORESOURCE_MEM;
+	}
 	if (pmcg->overflow_gsiv)
 		acpi_iort_register_irq(pmcg->overflow_gsiv, "overflow",

drivers/acpi/ec.c+10−0

@@ -2065,6 +2065,16 @@ bool acpi_ec_dispatch_gpe(void) · 2064 unmodified lines
2065	if (acpi_any_gpe_status_set(first_ec->gpe))	2065	if (acpi_any_gpe_status_set(first_ec->gpe))
2066	return true;	2066	return true;
2067		2067
		2068	/*
		2069	* Cancel the SCI wakeup and process all pending events in case there
		2070	* are any wakeup ones in there.
		2071	*
		2072	* Note that if any non-EC GPEs are active at this point, the SCI will
		2073	* retrigger after the rearming in acpi_s2idle_wake(), so no events
		2074	* should be missed by canceling the wakeup here.
		2075	*/
		2076	pm_system_cancel_wakeup();
		2077
2068	/*	2078	/*
2069	* Dispatch the EC GPE in-band, but do not report wakeup in any case	2079	* Dispatch the EC GPE in-band, but do not report wakeup in any case
2070	* to allow the caller to process events properly after that.	2080	* to allow the caller to process events properly after that.

drivers/acpi/sleep.c+5−10

 			return true;
 		}
-		/* Check non-EC GPE wakeups and dispatch the EC GPE. */
+		/*
+		 * Check non-EC GPE wakeups and if there are none, cancel the
+		 * SCI-related wakeup and dispatch the EC GPE.
+		 */
 		if (acpi_ec_dispatch_gpe()) {
 			pm_pr_dbg("ACPI non-EC GPE wakeup\n");
 			return true;
 		}
-		/*
-		 * Cancel the SCI wakeup and process all pending events in case
-		 * there are any wakeup ones in there.
-		 *
-		 * Note that if any non-EC GPEs are active at this point, the
-		 * SCI will retrigger after the rearming below, so no events
-		 * should be missed by canceling the wakeup here.
-		 */
-		pm_system_cancel_wakeup();
 		acpi_os_wait_events_complete();
 		/*
 			return true;
 		}
+		pm_wakeup_clear(acpi_sci_irq);
 		rearm_wake_irq(acpi_sci_irq);
 	}

drivers/acpi/x86/s2idle.c+4−8

 		mem_sleep_current = PM_SUSPEND_TO_IDLE;
 	/*
-	 * Some Intel based LPS0 systems, like ASUS Zenbook UX430UNR/i7-8550U don't
+	 * Some LPS0 systems, like ASUS Zenbook UX430UNR/i7-8550U, require the
-	 * use intel-hid or intel-vbtn but require the EC GPE to be enabled while
+	 * EC GPE to be enabled while suspended for certain wakeup devices to
-	 * suspended for certain wakeup devices to work, so mark it as wakeup-capable.
+	 * work, so mark it as wakeup-capable.
-	 *
-	 * Only enable on !AMD as enabling this universally causes problems for a number
-	 * of AMD based systems.
 	 */
-	if (!acpi_s2idle_vendor_amd())
+	acpi_ec_mark_gpe_for_wake();
-		acpi_ec_mark_gpe_for_wake();
 	return 0;
 }

drivers/ata/libata-core.c+17−8

 {
 	struct ata_port *ap = dev->link->ap;
+	if (dev->horkage & ATA_HORKAGE_NO_LOG_DIR)
+		return false;
 	if (ata_read_log_page(dev, ATA_LOG_DIRECTORY, 0, ap->sector_buf, 1))
 		return false;
 	return get_unaligned_le16(&ap->sector_buf[log * 2]) ? true : false;
 	struct ata_cpr_log *cpr_log = NULL;
 	u8 *desc, *buf = NULL;
-	if (!ata_identify_page_supported(dev,
+	if (ata_id_major_version(dev->id) < 11 ||
-				 ATA_LOG_CONCURRENT_POSITIONING_RANGES))
+	    !ata_log_supported(dev, ATA_LOG_CONCURRENT_POSITIONING_RANGES))
 		goto out;
 	/*
-	 * Read IDENTIFY DEVICE data log, page 0x47
+	 * Read the concurrent positioning ranges log (0x47). We can have at
-	 * (concurrent positioning ranges). We can have at most 255 32B range
+	 * most 255 32B range descriptors plus a 64B header.
-	 * descriptors plus a 64B header.
 	 */
 	buf_len = (64 + 255 * 32 + 511) & ~511;
 	buf = kzalloc(buf_len, GFP_KERNEL);
 	if (!buf)
 		goto out;
-	err_mask = ata_read_log_page(dev, ATA_LOG_IDENTIFY_DEVICE,
+	err_mask = ata_read_log_page(dev, ATA_LOG_CONCURRENT_POSITIONING_RANGES,
-				     ATA_LOG_CONCURRENT_POSITIONING_RANGES,
+				     0, buf, buf_len >> 9);
-				     buf, buf_len >> 9);
 	if (err_mask)
 		goto out;
 	/* devices that don't properly handle TRIM commands */
 	{ "SuperSSpeed S238*",		NULL,	ATA_HORKAGE_NOTRIM, },
+	{ "M88V29*",			NULL,	ATA_HORKAGE_NOTRIM, },
 	/*
 	 * As defined, the DRAT (Deterministic Read After Trim) and RZAT
 	{ "WDC WD3000JD-*",		NULL,	ATA_HORKAGE_WD_BROKEN_LPM },
 	{ "WDC WD3200JD-*",		NULL,	ATA_HORKAGE_WD_BROKEN_LPM },
+	/*
+	 * This sata dom device goes on a walkabout when the ATA_LOG_DIRECTORY
+	 * log page is accessed. Ensure we never ask for this log page with
+	 * these devices.
+	 */
+	{ "SATADOM-ML 3ME",		NULL,	ATA_HORKAGE_NO_LOG_DIR },
 	/* End Marker */
 	{ }
 };

drivers/ata/sata_fsl.c+7−9

 static ssize_t fsl_sata_intr_coalescing_show(struct device *dev,
 		struct device_attribute *attr, char *buf)
 {
-	return sysfs_emit(buf, "%d	%d\n",
+	return sysfs_emit(buf, "%u	%u\n",
 			intr_coalescing_count, intr_coalescing_ticks);
 }
 {
 	unsigned int coalescing_count,	coalescing_ticks;
-	if (sscanf(buf, "%d%d",
+	if (sscanf(buf, "%u%u", &coalescing_count, &coalescing_ticks) != 2) {
-				&coalescing_count,
+		dev_err(dev, "fsl-sata: wrong parameter format.\n");
-				&coalescing_ticks) != 2) {
-		printk(KERN_ERR "fsl-sata: wrong parameter format.\n");
 		return -EINVAL;
 	}
 	rx_watermark &= 0x1f;
 	spin_unlock_irqrestore(&host->lock, flags);
-	return sysfs_emit(buf, "%d\n", rx_watermark);
+	return sysfs_emit(buf, "%u\n", rx_watermark);
 }
 static ssize_t fsl_sata_rx_watermark_store(struct device *dev,
 	void __iomem *csr_base = host_priv->csr_base;
 	u32 temp;
-	if (sscanf(buf, "%d", &rx_watermark) != 1) {
+	if (kstrtouint(buf, 10, &rx_watermark) < 0) {
-		printk(KERN_ERR "fsl-sata: wrong parameter format.\n");
+		dev_err(dev, "fsl-sata: wrong parameter format.\n");
 		return -EINVAL;
 	}
 	temp = ioread32(csr_base + TRANSCFG);
 	temp &= 0xffffffe0;
 	iowrite32(temp | rx_watermark, csr_base + TRANSCFG);
 	spin_unlock_irqrestore(&host->lock, flags);
 	return strlen(buf);
 }

drivers/base/power/wakeup.c+34−7

@@ -34,7 +34,8 @@ suspend_state_t pm_suspend_target_state; · 33 unmodified lines
34	bool events_check_enabled __read_mostly;	34	bool events_check_enabled __read_mostly;
35		35
36	/* First wakeup IRQ seen by the kernel in the last cycle. */	36	/* First wakeup IRQ seen by the kernel in the last cycle. */
37	unsigned int pm_wakeup_irq __read_mostly;	37	static unsigned int wakeup_irq[2] __read_mostly;
		38	static DEFINE_RAW_SPINLOCK(wakeup_irq_lock);
38		39
39	/* If greater than 0 and the system is suspending, terminate the suspend. */	40	/* If greater than 0 and the system is suspending, terminate the suspend. */
40	static atomic_t pm_abort_suspend __read_mostly;	41	static atomic_t pm_abort_suspend __read_mostly;
@@ -942,19 +943,45 @@ void pm_system_cancel_wakeup(void) · 901 unmodified lines
942	atomic_dec_if_positive(&pm_abort_suspend);	943	atomic_dec_if_positive(&pm_abort_suspend);
943	}	944	}
944		945
945	void pm_wakeup_clear(bool reset)	946	void pm_wakeup_clear(unsigned int irq_number)
946	{	947	{
947	pm_wakeup_irq = 0;	948	raw_spin_lock_irq(&wakeup_irq_lock);
948	if (reset)	949
		950	if (irq_number && wakeup_irq[0] == irq_number)
		951	wakeup_irq[0] = wakeup_irq[1];
		952	else
		953	wakeup_irq[0] = 0;
		954
		955	wakeup_irq[1] = 0;
		956
		957	raw_spin_unlock_irq(&wakeup_irq_lock);
		958
		959	if (!irq_number)
949	atomic_set(&pm_abort_suspend, 0);	960	atomic_set(&pm_abort_suspend, 0);
950	}	961	}
951		962
952	void pm_system_irq_wakeup(unsigned int irq_number)	963	void pm_system_irq_wakeup(unsigned int irq_number)
953	{	964	{
954	if (pm_wakeup_irq == 0) {	965	unsigned long flags;
955	pm_wakeup_irq = irq_number;	966
		967	raw_spin_lock_irqsave(&wakeup_irq_lock, flags);
		968
		969	if (wakeup_irq[0] == 0)
		970	wakeup_irq[0] = irq_number;
		971	else if (wakeup_irq[1] == 0)
		972	wakeup_irq[1] = irq_number;
		973	else
		974	irq_number = 0;
		975
		976	raw_spin_unlock_irqrestore(&wakeup_irq_lock, flags);
		977
		978	if (irq_number)
956	pm_system_wakeup();	979	pm_system_wakeup();
957	}	980	}
		981
		982	unsigned int pm_wakeup_irq(void)
		983	{
		984	return wakeup_irq[0];
958	}	985	}
959		986
960	/**	987	/**

drivers/block/loop.c+29−36

@@ -1082,7 +1082,7 @@ out_putf: · 1081 unmodified lines
1082	return error;	1082	return error;
1083	}	1083	}
1084		1084
1085	static void __loop_clr_fd(struct loop_device *lo)	1085	static void __loop_clr_fd(struct loop_device *lo, bool release)
1086	{	1086	{
1087	struct file *filp;	1087	struct file *filp;
1088	gfp_t gfp = lo->old_gfp_mask;	1088	gfp_t gfp = lo->old_gfp_mask;
@@ -1144,6 +1144,8 @@ static void __loop_clr_fd(struct loop_device *lo) · 55 unmodified lines
1144	/* let user-space know about this change */	1144	/* let user-space know about this change */
1145	kobject_uevent(&disk_to_dev(lo->lo_disk)->kobj, KOBJ_CHANGE);	1145	kobject_uevent(&disk_to_dev(lo->lo_disk)->kobj, KOBJ_CHANGE);
1146	mapping_set_gfp_mask(filp->f_mapping, gfp);	1146	mapping_set_gfp_mask(filp->f_mapping, gfp);
		1147	/* This is safe: open() is still holding a reference. */
		1148	module_put(THIS_MODULE);
1147	blk_mq_unfreeze_queue(lo->lo_queue);	1149	blk_mq_unfreeze_queue(lo->lo_queue);
1148		1150
1149	disk_force_media_change(lo->lo_disk, DISK_EVENT_MEDIA_CHANGE);	1151	disk_force_media_change(lo->lo_disk, DISK_EVENT_MEDIA_CHANGE);
@@ -1151,52 +1153,44 @@ static void __loop_clr_fd(struct loop_device *lo) · 1 unmodified line
1151	if (lo->lo_flags & LO_FLAGS_PARTSCAN) {	1153	if (lo->lo_flags & LO_FLAGS_PARTSCAN) {
1152	int err;	1154	int err;
1153		1155
1154	mutex_lock(&lo->lo_disk->open_mutex);	1156	/*
		1157	* open_mutex has been held already in release path, so don't
		1158	* acquire it if this function is called in such case.
		1159	*
		1160	* If the reread partition isn't from release path, lo_refcnt
		1161	* must be at least one and it can only become zero when the
		1162	* current holder is released.
		1163	*/
		1164	if (!release)
		1165	mutex_lock(&lo->lo_disk->open_mutex);
1155	err = bdev_disk_changed(lo->lo_disk, false);	1166	err = bdev_disk_changed(lo->lo_disk, false);
1156	mutex_unlock(&lo->lo_disk->open_mutex);	1167	if (!release)
		1168	mutex_unlock(&lo->lo_disk->open_mutex);
1157	if (err)	1169	if (err)
1158	pr_warn("%s: partition scan of loop%d failed (rc=%d)\n",	1170	pr_warn("%s: partition scan of loop%d failed (rc=%d)\n",
1159	__func__, lo->lo_number, err);	1171	__func__, lo->lo_number, err);
1160	/* Device is gone, no point in returning error */	1172	/* Device is gone, no point in returning error */
1161	}	1173	}
1162		1174
		1175	/*
		1176	* lo->lo_state is set to Lo_unbound here after above partscan has
		1177	* finished. There cannot be anybody else entering __loop_clr_fd() as
		1178	* Lo_rundown state protects us from all the other places trying to
		1179	* change the 'lo' device.
		1180	*/
1163	lo->lo_flags = 0;	1181	lo->lo_flags = 0;
1164	if (!part_shift)	1182	if (!part_shift)
1165	lo->lo_disk->flags \|= GENHD_FL_NO_PART;	1183	lo->lo_disk->flags \|= GENHD_FL_NO_PART;
1166
1167	fput(filp);
1168	}
1169
1170	static void loop_rundown_completed(struct loop_device *lo)
1171	{
1172	mutex_lock(&lo->lo_mutex);	1184	mutex_lock(&lo->lo_mutex);
1173	lo->lo_state = Lo_unbound;	1185	lo->lo_state = Lo_unbound;
1174	mutex_unlock(&lo->lo_mutex);	1186	mutex_unlock(&lo->lo_mutex);
1175	module_put(THIS_MODULE);
1176	}
1177
1178	static void loop_rundown_workfn(struct work_struct *work)
1179	{
1180	struct loop_device *lo = container_of(work, struct loop_device,
1181	rundown_work);
1182	struct block_device *bdev = lo->lo_device;
1183	struct gendisk *disk = lo->lo_disk;
1184
1185	__loop_clr_fd(lo);
1186	kobject_put(&bdev->bd_device.kobj);
1187	module_put(disk->fops->owner);
1188	loop_rundown_completed(lo);
1189	}
1190		1187
1191	static void loop_schedule_rundown(struct loop_device *lo)	1188	/*
1192	{	1189	* Need not hold lo_mutex to fput backing file. Calling fput holding
1193	struct block_device *bdev = lo->lo_device;	1190	* lo_mutex triggers a circular lock dependency possibility warning as
1194	struct gendisk *disk = lo->lo_disk;	1191	* fput can take open_mutex which is usually taken before lo_mutex.
1195		1192	*/
1196	__module_get(disk->fops->owner);	1193	fput(filp);
1197	kobject_get(&bdev->bd_device.kobj);
1198	INIT_WORK(&lo->rundown_work, loop_rundown_workfn);
1199	queue_work(system_long_wq, &lo->rundown_work);
1200	}	1194	}
1201		1195
1202	static int loop_clr_fd(struct loop_device *lo)	1196	static int loop_clr_fd(struct loop_device *lo)
@@ -1228,8 +1222,7 @@ static int loop_clr_fd(struct loop_device *lo) · 25 unmodified lines
1228	lo->lo_state = Lo_rundown;	1222	lo->lo_state = Lo_rundown;
1229	mutex_unlock(&lo->lo_mutex);	1223	mutex_unlock(&lo->lo_mutex);
1230		1224
1231	__loop_clr_fd(lo);	1225	__loop_clr_fd(lo, false);
1232	loop_rundown_completed(lo);
1233	return 0;	1226	return 0;
1234	}	1227	}
1235		1228
@@ -1754,7 +1747,7 @@ static void lo_release(struct gendisk *disk, fmode_t mode) · 518 unmodified lines
1754	* In autoclear mode, stop the loop thread	1747	* In autoclear mode, stop the loop thread
1755	* and remove configuration after last close.	1748	* and remove configuration after last close.
1756	*/	1749	*/
1757	loop_schedule_rundown(lo);	1750	__loop_clr_fd(lo, true);
1758	return;	1751	return;
1759	} else if (lo->lo_state == Lo_bound) {	1752	} else if (lo->lo_state == Lo_bound) {
1760	/*	1753	/*

drivers/block/loop.h+0−1

@@ -56,7 +56,6 @@ struct loop_device { · 55 unmodified lines
56	struct gendisk *lo_disk;	56	struct gendisk *lo_disk;
57	struct mutex lo_mutex;	57	struct mutex lo_mutex;
58	bool idr_visible;	58	bool idr_visible;
59	struct work_struct rundown_work;
60	};	59	};
61		60
62	struct loop_cmd {	61	struct loop_cmd {

drivers/bus/mhi/pci_generic.c+2−0

 	.config = &modem_foxconn_sdx55_config,
 	.bar_num = MHI_PCI_DEFAULT_BAR_NUM,
 	.dma_data_width = 32,
+	.mru_default = 32768,
 	.sideband_wake = false,
 };
 	.config = &modem_mv31_config,
 	.bar_num = MHI_PCI_DEFAULT_BAR_NUM,
 	.dma_data_width = 32,
+	.mru_default = 32768,
 };
 static const struct mhi_channel_config mhi_sierra_em919x_channels[] = {

drivers/char/random.c+22−17

 	return arch_init;
 }
-static bool __init crng_init_try_arch_early(struct crng_state *crng)
+static bool __init crng_init_try_arch_early(void)
 {
 	int i;
 	bool arch_init = true;
 			rv = random_get_entropy();
 			arch_init = false;
 		}
-		crng->state[i] ^= rv;
+		primary_crng.state[i] ^= rv;
 	}
 	return arch_init;
 	crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
 }
-static void __init crng_initialize_primary(struct crng_state *crng)
+static void __init crng_initialize_primary(void)
 {
-	_extract_entropy(&crng->state[4], sizeof(u32) * 12);
+	_extract_entropy(&primary_crng.state[4], sizeof(u32) * 12);
-	if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) {
+	if (crng_init_try_arch_early() && trust_cpu && crng_init < 2) {
 		invalidate_batched_entropy();
 		numa_crng_init();
 		crng_init = 2;
 		pr_notice("crng init done (trusting CPU's manufacturer)\n");
 	}
-	crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
+	primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
 }
-static void crng_finalize_init(struct crng_state *crng)
+static void crng_finalize_init(void)
 {
-	if (crng != &primary_crng || crng_init >= 2)
-		return;
 	if (!system_wq) {
 		/* We can't call numa_crng_init until we have workqueues,
 		 * so mark this for processing later. */
 	invalidate_batched_entropy();
 	numa_crng_init();
 	crng_init = 2;
+	crng_need_final_init = false;
 	process_random_ready_list();
 	wake_up_interruptible(&crng_init_wait);
 	kill_fasync(&fasync, SIGIO, POLL_IN);
 	memzero_explicit(&buf, sizeof(buf));
 	WRITE_ONCE(crng->init_time, jiffies);
 	spin_unlock_irqrestore(&crng->lock, flags);
-	crng_finalize_init(crng);
+	if (crng == &primary_crng && crng_init < 2)
+		crng_finalize_init();
 }
 static void _extract_crng(struct crng_state *crng, u8 out[CHACHA_BLOCK_SIZE])
 {
 	init_std_data();
 	if (crng_need_final_init)
-		crng_finalize_init(&primary_crng);
+		crng_finalize_init();
-	crng_initialize_primary(&primary_crng);
+	crng_initialize_primary();
 	crng_global_init_time = jiffies;
 	if (ratelimit_disable) {
 		urandom_warning.interval = 0;
 		 */
 		if (!capable(CAP_SYS_ADMIN))
 			return -EPERM;
-		input_pool.entropy_count = 0;
+		if (xchg(&input_pool.entropy_count, 0) && random_write_wakeup_bits) {
+			wake_up_interruptible(&random_write_wait);
+			kill_fasync(&fasync, SIGIO, POLL_OUT);
+		}
 		return 0;
 	case RNDRESEEDCRNG:
 		if (!capable(CAP_SYS_ADMIN))
 			return;
 	}
-	/* Suspend writing if we're above the trickle threshold.
+	/* Throttle writing if we're above the trickle threshold.
 	 * We'll be woken up again once below random_write_wakeup_thresh,
-	 * or when the calling thread is about to terminate.
+	 * when the calling thread is about to terminate, or once
+	 * CRNG_RESEED_INTERVAL has lapsed.
 	 */
-	wait_event_interruptible(random_write_wait,
+	wait_event_interruptible_timeout(random_write_wait,
 			!system_wq || kthread_should_stop() ||
-			POOL_ENTROPY_BITS() <= random_write_wakeup_bits);
+			POOL_ENTROPY_BITS() <= random_write_wakeup_bits,
+			CRNG_RESEED_INTERVAL);
 	mix_pool_bytes(buffer, count);
 	credit_entropy_bits(entropy);
 }

drivers/clocksource/timer-ti-dm-systimer.c+1−1

@@ -241,7 +241,7 @@ static void __init dmtimer_systimer_assign_alwon(void) · 240 unmodified lines
241	bool quirk_unreliable_oscillator = false;	241	bool quirk_unreliable_oscillator = false;
242		242
243	/* Quirk unreliable 32 KiHz oscillator with incomplete dts */	243	/* Quirk unreliable 32 KiHz oscillator with incomplete dts */
244	if (of_machine_is_compatible("ti,omap3-beagle") \|\|	244	if (of_machine_is_compatible("ti,omap3-beagle-ab4") \|\|
245	of_machine_is_compatible("timll,omap3-devkit8000")) {	245	of_machine_is_compatible("timll,omap3-devkit8000")) {
246	quirk_unreliable_oscillator = true;	246	quirk_unreliable_oscillator = true;
247	counter_32k = -ENODEV;	247	counter_32k = -ENODEV;

drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c+2−1

 	char engs_info[2 * OTX2_CPT_NAME_LENGTH];
 	struct otx2_cpt_eng_grp_info *grp;
 	struct otx2_cpt_engs_rsvd *engs;
-	u32 mask[4];
 	int i, j;
 	pr_debug("Engine groups global info");
 		for (j = 0; j < OTX2_CPT_MAX_ETYPES_PER_GRP; j++) {
 			engs = &grp->engs[j];
 			if (engs->type) {
+				u32 mask[5] = { };
 				get_engs_info(grp, engs_info,
 					      2 * OTX2_CPT_NAME_LENGTH, j);
 				pr_debug("Slot%d: %s", j, engs_info);

drivers/dma-buf/dma-buf.c+11−11

@@ -1047,8 +1047,8 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_move_notify, DMA_BUF); · 1046 unmodified lines
1047	*	1047	*
1048	* Interfaces::	1048	* Interfaces::
1049	*	1049	*
1050	* void \dma_buf_vmap(struct dma_buf \dmabuf, struct dma_buf_map \*map)	1050	* void \dma_buf_vmap(struct dma_buf \dmabuf, struct iosys_map \*map)
1051	* void dma_buf_vunmap(struct dma_buf \dmabuf, struct dma_buf_map \map)	1051	* void dma_buf_vunmap(struct dma_buf \dmabuf, struct iosys_map \map)
1052	*	1052	*
1053	* The vmap call can fail if there is no vmap support in the exporter, or if	1053	* The vmap call can fail if there is no vmap support in the exporter, or if
1054	* it runs out of vmalloc space. Note that the dma-buf layer keeps a reference	1054	* it runs out of vmalloc space. Note that the dma-buf layer keeps a reference
@@ -1260,12 +1260,12 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF); · 205 unmodified lines
1260	*	1260	*
1261	* Returns 0 on success, or a negative errno code otherwise.	1261	* Returns 0 on success, or a negative errno code otherwise.
1262	*/	1262	*/
1263	int dma_buf_vmap(struct dma_buf dmabuf, struct dma_buf_map map)	1263	int dma_buf_vmap(struct dma_buf dmabuf, struct iosys_map map)
1264	{	1264	{
1265	struct dma_buf_map ptr;	1265	struct iosys_map ptr;
1266	int ret = 0;	1266	int ret = 0;
1267		1267
1268	dma_buf_map_clear(map);	1268	iosys_map_clear(map);
1269		1269
1270	if (WARN_ON(!dmabuf))	1270	if (WARN_ON(!dmabuf))
1271	return -EINVAL;	1271	return -EINVAL;
@@ -1276,12 +1276,12 @@ int dma_buf_vmap(struct dma_buf dmabuf, struct dma_buf_map map) · 4 unmodified lines
1276	mutex_lock(&dmabuf->lock);	1276	mutex_lock(&dmabuf->lock);
1277	if (dmabuf->vmapping_counter) {	1277	if (dmabuf->vmapping_counter) {
1278	dmabuf->vmapping_counter++;	1278	dmabuf->vmapping_counter++;
1279	BUG_ON(dma_buf_map_is_null(&dmabuf->vmap_ptr));	1279	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
1280	*map = dmabuf->vmap_ptr;	1280	*map = dmabuf->vmap_ptr;
1281	goto out_unlock;	1281	goto out_unlock;
1282	}	1282	}
1283		1283
1284	BUG_ON(dma_buf_map_is_set(&dmabuf->vmap_ptr));	1284	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
1285		1285
1286	ret = dmabuf->ops->vmap(dmabuf, &ptr);	1286	ret = dmabuf->ops->vmap(dmabuf, &ptr);
1287	if (WARN_ON_ONCE(ret))	1287	if (WARN_ON_ONCE(ret))
@@ -1303,20 +1303,20 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF); · 15 unmodified lines
1303	* @dmabuf: [in] buffer to vunmap	1303	* @dmabuf: [in] buffer to vunmap
1304	* @map: [in] vmap pointer to vunmap	1304	* @map: [in] vmap pointer to vunmap
1305	*/	1305	*/
1306	void dma_buf_vunmap(struct dma_buf dmabuf, struct dma_buf_map map)	1306	void dma_buf_vunmap(struct dma_buf dmabuf, struct iosys_map map)
1307	{	1307	{
1308	if (WARN_ON(!dmabuf))	1308	if (WARN_ON(!dmabuf))
1309	return;	1309	return;
1310		1310
1311	BUG_ON(dma_buf_map_is_null(&dmabuf->vmap_ptr));	1311	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
1312	BUG_ON(dmabuf->vmapping_counter == 0);	1312	BUG_ON(dmabuf->vmapping_counter == 0);
1313	BUG_ON(!dma_buf_map_is_equal(&dmabuf->vmap_ptr, map));	1313	BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
1314		1314
1315	mutex_lock(&dmabuf->lock);	1315	mutex_lock(&dmabuf->lock);
1316	if (--dmabuf->vmapping_counter == 0) {	1316	if (--dmabuf->vmapping_counter == 0) {
1317	if (dmabuf->ops->vunmap)	1317	if (dmabuf->ops->vunmap)
1318	dmabuf->ops->vunmap(dmabuf, map);	1318	dmabuf->ops->vunmap(dmabuf, map);
1319	dma_buf_map_clear(&dmabuf->vmap_ptr);	1319	iosys_map_clear(&dmabuf->vmap_ptr);
1320	}	1320	}
1321	mutex_unlock(&dmabuf->lock);	1321	mutex_unlock(&dmabuf->lock);
1322	}	1322	}

drivers/dma-buf/dma-fence-array.c+14−0

@@ -176,6 +176,20 @@ struct dma_fence_array *dma_fence_array_create(int num_fences, · 175 unmodified lines
176		176
177	array->base.error = PENDING_ERROR;	177	array->base.error = PENDING_ERROR;
178		178
		179	/*
		180	* dma_fence_array objects should never contain any other fence
		181	* containers or otherwise we run into recursion and potential kernel
		182	* stack overflow on operations on the dma_fence_array.
		183	*
		184	* The correct way of handling this is to flatten out the array by the
		185	* caller instead.
		186	*
		187	* Enforce this here by checking that we don't create a dma_fence_array
		188	* with any container inside.
		189	*/
		190	while (num_fences--)
		191	WARN_ON(dma_fence_is_container(fences[num_fences]));
		192
179	return array;	193	return array;
180	}	194	}
181	EXPORT_SYMBOL(dma_fence_array_create);	195	EXPORT_SYMBOL(dma_fence_array_create);

drivers/dma-buf/dma-fence-chain.c+11−4

 	dma_fence_get(&head->base);
 	dma_fence_chain_for_each(fence, &head->base) {
-		struct dma_fence_chain *chain = to_dma_fence_chain(fence);
+		struct dma_fence *f = dma_fence_chain_contained(fence);
-		struct dma_fence *f = chain ? chain->fence : fence;
 		dma_fence_get(f);
 		if (!dma_fence_add_callback(f, &head->cb, dma_fence_chain_cb)) {
 static bool dma_fence_chain_signaled(struct dma_fence *fence)
 {
 	dma_fence_chain_for_each(fence, fence) {
-		struct dma_fence_chain *chain = to_dma_fence_chain(fence);
+		struct dma_fence *f = dma_fence_chain_contained(fence);
-		struct dma_fence *f = chain ? chain->fence : fence;
 		if (!dma_fence_is_signaled(f)) {
 			dma_fence_put(fence);
 	dma_fence_init(&chain->base, &dma_fence_chain_ops,
 		       &chain->lock, context, seqno);
+	/*
+	 * Chaining dma_fence_chain container together is only allowed through
+	 * the prev fence and not through the contained fence.
+	 *
+	 * The correct way of handling this is to flatten out the fence
+	 * structure into a dma_fence_array by the caller instead.
+	 */
+	WARN_ON(dma_fence_is_chain(fence));
 }
 EXPORT_SYMBOL(dma_fence_chain_init);

drivers/dma-buf/dma-heap.c+2−0

 #include <linux/xarray.h>
 #include <linux/list.h>
 #include <linux/slab.h>
+#include <linux/nospec.h>
 #include <linux/uaccess.h>
 #include <linux/syscalls.h>
 #include <linux/dma-heap.h>
 	if (nr >= ARRAY_SIZE(dma_heap_ioctl_cmds))
 		return -EINVAL;
+	nr = array_index_nospec(nr, ARRAY_SIZE(dma_heap_ioctl_cmds));
 	/* Get the kernel ioctl cmd that matches */
 	kcmd = dma_heap_ioctl_cmds[nr];

drivers/dma-buf/dma-resv.c+20−14

 	dma_resv_assert_held(obj);
+	/* Drivers should not add containers here, instead add each fence
+	 * individually.
+	 */
+	WARN_ON(dma_fence_is_container(fence));
 	fobj = dma_resv_shared_list(obj);
 	count = fobj->shared_count;
 }
 EXPORT_SYMBOL(dma_resv_add_excl_fence);
-/**
+/* Restart the iterator by initializing all the necessary fields, but not the
- * dma_resv_iter_restart_unlocked - restart the unlocked iterator
+ * relation to the dma_resv object. */
- * @cursor: The dma_resv_iter object to restart
- *
- * Restart the unlocked iteration by initializing the cursor object.
- */
 static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor)
 {
 	cursor->seq = read_seqcount_begin(&cursor->obj->seq);
 	cursor->is_restarted = true;
 }
-/**
+/* Walk to the next not signaled fence and grab a reference to it */
- * dma_resv_iter_walk_unlocked - walk over fences in a dma_resv obj
- * @cursor: cursor to record the current position
- *
- * Return all the fences in the dma_resv object which are not yet signaled.
- * The returned fence has an extra local reference so will stay alive.
- * If a concurrent modify is detected the whole iteration is started over again.
- */
 static void dma_resv_iter_walk_unlocked(struct dma_resv_iter *cursor)
 {
 	struct dma_resv *obj = cursor->obj;
  * dma_resv_iter_first_unlocked - first fence in an unlocked dma_resv obj.
  * @cursor: the cursor with the current position
  *
+ * Subsequent fences are iterated with dma_resv_iter_next_unlocked().
+ *
+ * Beware that the iterator can be restarted.  Code which accumulates statistics
+ * or similar needs to check for this with dma_resv_iter_is_restarted(). For
+ * this reason prefer the locked dma_resv_iter_first() whenver possible.
+ *
  * Returns the first fence from an unlocked dma_resv obj.
  */
 struct dma_fence *dma_resv_iter_first_unlocked(struct dma_resv_iter *cursor)
  * dma_resv_iter_next_unlocked - next fence in an unlocked dma_resv obj.
  * @cursor: the cursor with the current position
  *
+ * Beware that the iterator can be restarted.  Code which accumulates statistics
+ * or similar needs to check for this with dma_resv_iter_is_restarted(). For
+ * this reason prefer the locked dma_resv_iter_next() whenver possible.
+ *
  * Returns the next fence from an unlocked dma_resv obj.
  */
 struct dma_fence *dma_resv_iter_next_unlocked(struct dma_resv_iter *cursor)
  * dma_resv_iter_first - first fence from a locked dma_resv object
  * @cursor: cursor to record the current position
  *
+ * Subsequent fences are iterated with dma_resv_iter_next_unlocked().
+ *
  * Return the first fence in the dma_resv object while holding the
  * &dma_resv.lock.
  */

drivers/dma-buf/heaps/cma_heap.c+5−5

@@ -202,7 +202,7 @@ static void cma_heap_do_vmap(struct cma_heap_buffer buffer) · 201 unmodified lines
202	return vaddr;	202	return vaddr;
203	}	203	}
204		204
205	static int cma_heap_vmap(struct dma_buf dmabuf, struct dma_buf_map map)	205	static int cma_heap_vmap(struct dma_buf dmabuf, struct iosys_map map)
206	{	206	{
207	struct cma_heap_buffer *buffer = dmabuf->priv;	207	struct cma_heap_buffer *buffer = dmabuf->priv;
208	void *vaddr;	208	void *vaddr;
@@ -211,7 +211,7 @@ static int cma_heap_vmap(struct dma_buf dmabuf, struct dma_buf_map map) · 2 unmodified lines
211	mutex_lock(&buffer->lock);	211	mutex_lock(&buffer->lock);
212	if (buffer->vmap_cnt) {	212	if (buffer->vmap_cnt) {
213	buffer->vmap_cnt++;	213	buffer->vmap_cnt++;
214	dma_buf_map_set_vaddr(map, buffer->vaddr);	214	iosys_map_set_vaddr(map, buffer->vaddr);
215	goto out;	215	goto out;
216	}	216	}
217		217
@@ -222,14 +222,14 @@ static int cma_heap_vmap(struct dma_buf dmabuf, struct dma_buf_map map) · 4 unmodified lines
222	}	222	}
223	buffer->vaddr = vaddr;	223	buffer->vaddr = vaddr;
224	buffer->vmap_cnt++;	224	buffer->vmap_cnt++;
225	dma_buf_map_set_vaddr(map, buffer->vaddr);	225	iosys_map_set_vaddr(map, buffer->vaddr);
226	out:	226	out:
227	mutex_unlock(&buffer->lock);	227	mutex_unlock(&buffer->lock);
228		228
229	return ret;	229	return ret;
230	}	230	}
231		231
232	static void cma_heap_vunmap(struct dma_buf dmabuf, struct dma_buf_map map)	232	static void cma_heap_vunmap(struct dma_buf dmabuf, struct iosys_map map)
233	{	233	{
234	struct cma_heap_buffer *buffer = dmabuf->priv;	234	struct cma_heap_buffer *buffer = dmabuf->priv;
235		235
@@ -239,7 +239,7 @@ static void cma_heap_vunmap(struct dma_buf dmabuf, struct dma_buf_map map) · 3 unmodified lines
239	buffer->vaddr = NULL;	239	buffer->vaddr = NULL;
240	}	240	}
241	mutex_unlock(&buffer->lock);	241	mutex_unlock(&buffer->lock);
242	dma_buf_map_clear(map);	242	iosys_map_clear(map);
243	}	243	}
244		244
245	static void cma_heap_dma_buf_release(struct dma_buf *dmabuf)	245	static void cma_heap_dma_buf_release(struct dma_buf *dmabuf)

drivers/dma-buf/heaps/system_heap.c+5−5

@@ -241,7 +241,7 @@ static void system_heap_do_vmap(struct system_heap_buffer buffer) · 240 unmodified lines
241	return vaddr;	241	return vaddr;
242	}	242	}
243		243
244	static int system_heap_vmap(struct dma_buf dmabuf, struct dma_buf_map map)	244	static int system_heap_vmap(struct dma_buf dmabuf, struct iosys_map map)
245	{	245	{
246	struct system_heap_buffer *buffer = dmabuf->priv;	246	struct system_heap_buffer *buffer = dmabuf->priv;
247	void *vaddr;	247	void *vaddr;
@@ -250,7 +250,7 @@ static int system_heap_vmap(struct dma_buf dmabuf, struct dma_buf_map map) · 2 unmodified lines
250	mutex_lock(&buffer->lock);	250	mutex_lock(&buffer->lock);
251	if (buffer->vmap_cnt) {	251	if (buffer->vmap_cnt) {
252	buffer->vmap_cnt++;	252	buffer->vmap_cnt++;
253	dma_buf_map_set_vaddr(map, buffer->vaddr);	253	iosys_map_set_vaddr(map, buffer->vaddr);
254	goto out;	254	goto out;
255	}	255	}
256		256
@@ -262,14 +262,14 @@ static int system_heap_vmap(struct dma_buf dmabuf, struct dma_buf_map map) · 5 unmodified lines
262		262
263	buffer->vaddr = vaddr;	263	buffer->vaddr = vaddr;
264	buffer->vmap_cnt++;	264	buffer->vmap_cnt++;
265	dma_buf_map_set_vaddr(map, buffer->vaddr);	265	iosys_map_set_vaddr(map, buffer->vaddr);
266	out:	266	out:
267	mutex_unlock(&buffer->lock);	267	mutex_unlock(&buffer->lock);
268		268
269	return ret;	269	return ret;
270	}	270	}
271		271
272	static void system_heap_vunmap(struct dma_buf dmabuf, struct dma_buf_map map)	272	static void system_heap_vunmap(struct dma_buf dmabuf, struct iosys_map map)
273	{	273	{
274	struct system_heap_buffer *buffer = dmabuf->priv;	274	struct system_heap_buffer *buffer = dmabuf->priv;
275		275
@@ -279,7 +279,7 @@ static void system_heap_vunmap(struct dma_buf dmabuf, struct dma_buf_map map) · 3 unmodified lines
279	buffer->vaddr = NULL;	279	buffer->vaddr = NULL;
280	}	280	}
281	mutex_unlock(&buffer->lock);	281	mutex_unlock(&buffer->lock);
282	dma_buf_map_clear(map);	282	iosys_map_clear(map);
283	}	283	}
284		284
285	static void system_heap_dma_buf_release(struct dma_buf *dmabuf)	285	static void system_heap_dma_buf_release(struct dma_buf *dmabuf)

drivers/edac/altera_edac.c+1−1

@@ -350,7 +350,7 @@ static int altr_sdram_probe(struct platform_device *pdev) · 349 unmodified lines
350	if (irq < 0) {	350	if (irq < 0) {
351	edac_printk(KERN_ERR, EDAC_MC,	351	edac_printk(KERN_ERR, EDAC_MC,
352	"No irq %d in DT\n", irq);	352	"No irq %d in DT\n", irq);
353	return -ENODEV;	353	return irq;
354	}	354	}
355		355
356	/* Arria10 has a 2nd IRQ */	356	/* Arria10 has a 2nd IRQ */

drivers/edac/xgene_edac.c+1−1

@@ -1919,7 +1919,7 @@ static int xgene_edac_probe(struct platform_device *pdev) · 1918 unmodified lines
1919	irq = platform_get_irq_optional(pdev, i);	1919	irq = platform_get_irq_optional(pdev, i);
1920	if (irq < 0) {	1920	if (irq < 0) {
1921	dev_err(&pdev->dev, "No IRQ resource\n");	1921	dev_err(&pdev->dev, "No IRQ resource\n");
1922	rc = -EINVAL;	1922	rc = irq;
1923	goto out_err;	1923	goto out_err;
1924	}	1924	}
1925	rc = devm_request_irq(&pdev->dev, irq,	1925	rc = devm_request_irq(&pdev->dev, irq,

drivers/gpio/gpio-aggregator.c+14−4

 {
 	struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
-	return gpiod_get_value(fwd->descs[offset]);
+	return chip->can_sleep ? gpiod_get_value_cansleep(fwd->descs[offset])
+			       : gpiod_get_value(fwd->descs[offset]);
 }
 static int gpio_fwd_get_multiple(struct gpiochip_fwd *fwd, unsigned long *mask,
 	for_each_set_bit(i, mask, fwd->chip.ngpio)
 		descs[j++] = fwd->descs[i];
-	error = gpiod_get_array_value(j, descs, NULL, values);
+	if (fwd->chip.can_sleep)
+		error = gpiod_get_array_value_cansleep(j, descs, NULL, values);
+	else
+		error = gpiod_get_array_value(j, descs, NULL, values);
 	if (error)
 		return error;
 {
 	struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
-	gpiod_set_value(fwd->descs[offset], value);
+	if (chip->can_sleep)
+		gpiod_set_value_cansleep(fwd->descs[offset], value);
+	else
+		gpiod_set_value(fwd->descs[offset], value);
 }
 static void gpio_fwd_set_multiple(struct gpiochip_fwd *fwd, unsigned long *mask,
 		descs[j++] = fwd->descs[i];
 	}
-	gpiod_set_array_value(j, descs, NULL, values);
+	if (fwd->chip.can_sleep)
+		gpiod_set_array_value_cansleep(j, descs, NULL, values);
+	else
+		gpiod_set_array_value(j, descs, NULL, values);
 }
 static void gpio_fwd_set_multiple_locked(struct gpio_chip *chip,

drivers/gpio/gpio-sifive.c+1−1

@@ -223,7 +223,7 @@ static int sifive_gpio_probe(struct platform_device *pdev) · 222 unmodified lines
223	NULL,	223	NULL,
224	chip->base + SIFIVE_GPIO_OUTPUT_EN,	224	chip->base + SIFIVE_GPIO_OUTPUT_EN,
225	chip->base + SIFIVE_GPIO_INPUT_EN,	225	chip->base + SIFIVE_GPIO_INPUT_EN,
226	0);	226	BGPIOF_READ_OUTPUT_REG_SET);
227	if (ret) {	227	if (ret) {
228	dev_err(dev, "unable to init generic GPIO\n");	228	dev_err(dev, "unable to init generic GPIO\n");
229	return ret;	229	return ret;

drivers/gpio/gpio-sim.c+15−4

 	return container_of(group, struct gpio_sim_bank, group);
 }
+static bool gpio_sim_bank_has_label(struct gpio_sim_bank *bank)
+{
+	return bank->label && *bank->label;
+}
 static struct gpio_sim_device *
 gpio_sim_bank_get_device(struct gpio_sim_bank *bank)
 {
 			 * point the device doesn't exist yet and so dev_name()
 			 * is not available.
 			 */
-			hog->chip_label = kasprintf(GFP_KERNEL,
+			if (gpio_sim_bank_has_label(bank))
-						    "gpio-sim.%u-%s", dev->id,
+				hog->chip_label = kstrdup(bank->label,
-						    fwnode_get_name(bank->swnode));
+							  GFP_KERNEL);
+			else
+				hog->chip_label = kasprintf(GFP_KERNEL,
+							"gpio-sim.%u-%s",
+							dev->id,
+							fwnode_get_name(
+								bank->swnode));
 			if (!hog->chip_label) {
 				gpio_sim_remove_hogs(dev);
 				return -ENOMEM;
 	properties[prop_idx++] = PROPERTY_ENTRY_U32("ngpios", bank->num_lines);
-	if (bank->label && (strlen(bank->label) > 0))
+	if (gpio_sim_bank_has_label(bank))
 		properties[prop_idx++] = PROPERTY_ENTRY_STRING("gpio-sim,label",
 							       bank->label);

drivers/gpio/gpiolib-cdev.c+3−3

@@ -330,7 +330,7 @@ static int linehandle_create(struct gpio_device gdev, void __user ip) · 329 unmodified lines
330	goto out_free_lh;	330	goto out_free_lh;
331	}	331	}
332		332
333	ret = gpiod_request(desc, lh->label);	333	ret = gpiod_request_user(desc, lh->label);
334	if (ret)	334	if (ret)
335	goto out_free_lh;	335	goto out_free_lh;
336	lh->descs[i] = desc;	336	lh->descs[i] = desc;
@@ -1378,7 +1378,7 @@ static int linereq_create(struct gpio_device gdev, void __user ip) · 1041 unmodified lines
1378	goto out_free_linereq;	1378	goto out_free_linereq;
1379	}	1379	}
1380		1380
1381	ret = gpiod_request(desc, lr->label);	1381	ret = gpiod_request_user(desc, lr->label);
1382	if (ret)	1382	if (ret)
1383	goto out_free_linereq;	1383	goto out_free_linereq;
1384		1384
@@ -1764,7 +1764,7 @@ static int lineevent_create(struct gpio_device gdev, void __user ip) · 379 unmodified lines
1764	}	1764	}
1765	}	1765	}
1766		1766
1767	ret = gpiod_request(desc, le->label);	1767	ret = gpiod_request_user(desc, le->label);
1768	if (ret)	1768	if (ret)
1769	goto out_free_le;	1769	goto out_free_le;
1770	le->desc = desc;	1770	le->desc = desc;

drivers/gpio/gpiolib-sysfs.c+2−5

 	 * they may be undone on its behalf too.
 	 */
-	status = gpiod_request(desc, "sysfs");
+	status = gpiod_request_user(desc, "sysfs");
-	if (status) {
+	if (status)
-		if (status == -EPROBE_DEFER)
-			status = -ENODEV;
 		goto done;
-	}
 	status = gpiod_set_transitory(desc, false);
 	if (!status) {

drivers/gpio/gpiolib.h+12−0

@@ -135,6 +135,18 @@ struct gpio_desc { · 134 unmodified lines
135		135
136	int gpiod_request(struct gpio_desc desc, const char label);	136	int gpiod_request(struct gpio_desc desc, const char label);
137	void gpiod_free(struct gpio_desc *desc);	137	void gpiod_free(struct gpio_desc *desc);
		138
		139	static inline int gpiod_request_user(struct gpio_desc desc, const char label)
		140	{
		141	int ret;
		142
		143	ret = gpiod_request(desc, label);
		144	if (ret == -EPROBE_DEFER)
		145	ret = -ENODEV;
		146
		147	return ret;
		148	}
		149
138	int gpiod_configure_flags(struct gpio_desc desc, const char con_id,	150	int gpiod_configure_flags(struct gpio_desc desc, const char con_id,
139	unsigned long lflags, enum gpiod_flags dflags);	151	unsigned long lflags, enum gpiod_flags dflags);
140	int gpio_set_debounce_timeout(struct gpio_desc *desc, unsigned int debounce);	152	int gpio_set_debounce_timeout(struct gpio_desc *desc, unsigned int debounce);

drivers/gpu/drm/Kconfig+3−0

 	select DRM_DP_HELPER
 	select DRM_LIB_RANDOM
 	select DRM_KMS_HELPER
+	select DRM_BUDDY
 	select DRM_EXPORT_FOR_TESTS if m
 	default n
 	help
 source "drivers/gpu/drm/gud/Kconfig"
+source "drivers/gpu/drm/solomon/Kconfig"
 source "drivers/gpu/drm/sprd/Kconfig"
 config DRM_HYPERV

drivers/gpu/drm/Makefile+1−0

@@ -132,4 +132,5 @@ obj-$(CONFIG_DRM_TIDSS) += tidss/ · 131 unmodified lines
132	obj-y += xlnx/	132	obj-y += xlnx/
133	obj-y += gud/	133	obj-y += gud/
134	obj-$(CONFIG_DRM_HYPERV) += hyperv/	134	obj-$(CONFIG_DRM_HYPERV) += hyperv/
		135	obj-y += solomon/
135	obj-$(CONFIG_DRM_SPRD) += sprd/	136	obj-$(CONFIG_DRM_SPRD) += sprd/

drivers/gpu/drm/amd/amdgpu/aldebaran.c+1−1

@@ -260,7 +260,7 @@ static int aldebaran_mode2_restore_ip(struct amdgpu_device *adev) · 259 unmodified lines
260	adev->gfx.rlc.funcs->resume(adev);	260	adev->gfx.rlc.funcs->resume(adev);
261		261
262	/* Wait for FW reset event complete */	262	/* Wait for FW reset event complete */
263	r = smu_wait_for_event(adev, SMU_EVENT_RESET_COMPLETE, 0);	263	r = amdgpu_dpm_wait_for_event(adev, SMU_EVENT_RESET_COMPLETE, 0);
264	if (r) {	264	if (r) {
265	dev_err(adev->dev,	265	dev_err(adev->dev,
266	"Failed to get response from firmware after reset\n");	266	"Failed to get response from firmware after reset\n");

drivers/gpu/drm/amd/amdgpu/amdgpu.h+39−17

@@ -99,7 +99,6 @@ · 98 unmodified lines
99	#include "amdgpu_gem.h"	99	#include "amdgpu_gem.h"
100	#include "amdgpu_doorbell.h"	100	#include "amdgpu_doorbell.h"
101	#include "amdgpu_amdkfd.h"	101	#include "amdgpu_amdkfd.h"
102	#include "amdgpu_smu.h"
103	#include "amdgpu_discovery.h"	102	#include "amdgpu_discovery.h"
104	#include "amdgpu_mes.h"	103	#include "amdgpu_mes.h"
105	#include "amdgpu_umc.h"	104	#include "amdgpu_umc.h"
@@ -109,6 +108,7 @@ · 3 unmodified lines
109	#include "amdgpu_smuio.h"	108	#include "amdgpu_smuio.h"
110	#include "amdgpu_fdinfo.h"	109	#include "amdgpu_fdinfo.h"
111	#include "amdgpu_mca.h"	110	#include "amdgpu_mca.h"
		111	#include "amdgpu_ras.h"
112		112
113	#define MAX_GPU_INSTANCE 16	113	#define MAX_GPU_INSTANCE 16
114		114
@@ -197,7 +197,6 @@ extern int amdgpu_emu_mode; · 82 unmodified lines
197	extern uint amdgpu_smu_memory_pool_size;	197	extern uint amdgpu_smu_memory_pool_size;
198	extern int amdgpu_smu_pptable_id;	198	extern int amdgpu_smu_pptable_id;
199	extern uint amdgpu_dc_feature_mask;	199	extern uint amdgpu_dc_feature_mask;
200	extern uint amdgpu_freesync_vid_mode;
201	extern uint amdgpu_dc_debug_mask;	200	extern uint amdgpu_dc_debug_mask;
202	extern uint amdgpu_dm_abm_level;	201	extern uint amdgpu_dm_abm_level;
203	extern int amdgpu_backlight;	202	extern int amdgpu_backlight;
@@ -373,7 +372,8 @@ int amdgpu_device_ip_block_add(struct amdgpu_device *adev, · 169 unmodified lines
373	*/	372	*/
374	bool amdgpu_get_bios(struct amdgpu_device *adev);	373	bool amdgpu_get_bios(struct amdgpu_device *adev);
375	bool amdgpu_read_bios(struct amdgpu_device *adev);	374	bool amdgpu_read_bios(struct amdgpu_device *adev);
376		375	bool amdgpu_soc15_read_bios_from_rom(struct amdgpu_device *adev,
		376	u8 *bios, u32 length_bytes);
377	/*	377	/*
378	* Clocks	378	* Clocks
379	*/	379	*/
@@ -771,6 +771,8 @@ struct amd_powerplay { · 391 unmodified lines
771	const struct amd_pm_funcs *pp_funcs;	771	const struct amd_pm_funcs *pp_funcs;
772	};	772	};
773		773
		774	struct ip_discovery_top;
		775
774	/* polaris10 kickers */	776	/* polaris10 kickers */
775	#define ASICID_IS_P20(did, rid) (((did == 0x67DF) && \	777	#define ASICID_IS_P20(did, rid) (((did == 0x67DF) && \
776	((rid == 0xE3) \|\| \	778	((rid == 0xE3) \|\| \
@@ -813,6 +815,8 @@ struct amd_powerplay { · 36 unmodified lines
813	#define AMDGPU_RESET_MAGIC_NUM 64	815	#define AMDGPU_RESET_MAGIC_NUM 64
814	#define AMDGPU_MAX_DF_PERFMONS 4	816	#define AMDGPU_MAX_DF_PERFMONS 4
815	#define AMDGPU_PRODUCT_NAME_LEN 64	817	#define AMDGPU_PRODUCT_NAME_LEN 64
		818	struct amdgpu_reset_domain;
		819
816	struct amdgpu_device {	820	struct amdgpu_device {
817	struct device *dev;	821	struct device *dev;
818	struct pci_dev *pdev;	822	struct pci_dev *pdev;
@@ -950,12 +954,6 @@ struct amdgpu_device { · 131 unmodified lines
950		954
951	/* powerplay */	955	/* powerplay */
952	struct amd_powerplay powerplay;	956	struct amd_powerplay powerplay;
953	bool pp_force_state_enabled;
954
955	/* smu */
956	struct smu_context smu;
957
958	/* dpm */
959	struct amdgpu_pm pm;	957	struct amdgpu_pm pm;
960	u32 cg_flags;	958	u32 cg_flags;
961	u32 pg_flags;	959	u32 pg_flags;
@@ -1054,9 +1052,7 @@ struct amdgpu_device { · 92 unmodified lines
1054	bool in_s4;	1052	bool in_s4;
1055	bool in_s0ix;	1053	bool in_s0ix;
1056		1054
1057	atomic_t in_gpu_reset;
1058	enum pp_mp1_state mp1_state;	1055	enum pp_mp1_state mp1_state;
1059	struct rw_semaphore reset_sem;
1060	struct amdgpu_doorbell_index doorbell_index;	1056	struct amdgpu_doorbell_index doorbell_index;
1061		1057
1062	struct mutex notifier_lock;	1058	struct mutex notifier_lock;
@@ -1100,6 +1096,12 @@ struct amdgpu_device { · 37 unmodified lines
1100	uint32_t ip_versions[MAX_HWIP][HWIP_MAX_INSTANCE];	1096	uint32_t ip_versions[MAX_HWIP][HWIP_MAX_INSTANCE];
1101		1097
1102	bool ram_is_direct_mapped;	1098	bool ram_is_direct_mapped;
		1099
		1100	struct list_head ras_list;
		1101
		1102	struct ip_discovery_top *ip_top;
		1103
		1104	struct amdgpu_reset_domain *reset_domain;
1103	};	1105	};
1104		1106
1105	static inline struct amdgpu_device drm_to_adev(struct drm_device ddev)	1107	static inline struct amdgpu_device drm_to_adev(struct drm_device ddev)
@@ -1293,9 +1295,12 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev); · 187 unmodified lines
1293	bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev);	1295	bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev);
1294	int amdgpu_device_gpu_recover(struct amdgpu_device *adev,	1296	int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
1295	struct amdgpu_job* job);	1297	struct amdgpu_job* job);
		1298	int amdgpu_device_gpu_recover_imp(struct amdgpu_device *adev,
		1299	struct amdgpu_job *job);
1296	void amdgpu_device_pci_config_reset(struct amdgpu_device *adev);	1300	void amdgpu_device_pci_config_reset(struct amdgpu_device *adev);
1297	int amdgpu_device_pci_reset(struct amdgpu_device *adev);	1301	int amdgpu_device_pci_reset(struct amdgpu_device *adev);
1298	bool amdgpu_device_need_post(struct amdgpu_device *adev);	1302	bool amdgpu_device_need_post(struct amdgpu_device *adev);
		1303	bool amdgpu_device_should_use_aspm(struct amdgpu_device *adev);
1299		1304
1300	void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,	1305	void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
1301	u64 num_vis_bytes);	1306	u64 num_vis_bytes);
@@ -1321,6 +1326,10 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, · 19 unmodified lines
1321	struct amdgpu_ring *ring);	1326	struct amdgpu_ring *ring);
1322		1327
1323	void amdgpu_device_halt(struct amdgpu_device *adev);	1328	void amdgpu_device_halt(struct amdgpu_device *adev);
		1329	u32 amdgpu_device_pcie_port_rreg(struct amdgpu_device *adev,
		1330	u32 reg);
		1331	void amdgpu_device_pcie_port_wreg(struct amdgpu_device *adev,
		1332	u32 reg, u32 v);
1324		1333
1325	/* atpx handler */	1334	/* atpx handler */
1326	#if defined(CONFIG_VGA_SWITCHEROO)	1335	#if defined(CONFIG_VGA_SWITCHEROO)
@@ -1408,12 +1417,10 @@ int amdgpu_acpi_smart_shift_update(struct drm_device *dev, enum amdgpu_ss ss_sta · 81 unmodified lines
1408	int amdgpu_acpi_pcie_notify_device_ready(struct amdgpu_device *adev);	1417	int amdgpu_acpi_pcie_notify_device_ready(struct amdgpu_device *adev);
1409		1418
1410	void amdgpu_acpi_get_backlight_caps(struct amdgpu_dm_backlight_caps *caps);	1419	void amdgpu_acpi_get_backlight_caps(struct amdgpu_dm_backlight_caps *caps);
1411	bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev);
1412	void amdgpu_acpi_detect(void);	1420	void amdgpu_acpi_detect(void);
1413	#else	1421	#else
1414	static inline int amdgpu_acpi_init(struct amdgpu_device *adev) { return 0; }	1422	static inline int amdgpu_acpi_init(struct amdgpu_device *adev) { return 0; }
1415	static inline void amdgpu_acpi_fini(struct amdgpu_device *adev) { }	1423	static inline void amdgpu_acpi_fini(struct amdgpu_device *adev) { }
1416	static inline bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) { return false; }
1417	static inline void amdgpu_acpi_detect(void) { }	1424	static inline void amdgpu_acpi_detect(void) { }
1418	static inline bool amdgpu_acpi_is_power_shift_control_supported(void) { return false; }	1425	static inline bool amdgpu_acpi_is_power_shift_control_supported(void) { return false; }
1419	static inline int amdgpu_acpi_power_shift_control(struct amdgpu_device *adev,	1426	static inline int amdgpu_acpi_power_shift_control(struct amdgpu_device *adev,
@@ -1422,6 +1429,14 @@ static inline int amdgpu_acpi_smart_shift_update(struct drm_device *dev, · 2 unmodified lines
1422	enum amdgpu_ss ss_state) { return 0; }	1429	enum amdgpu_ss ss_state) { return 0; }
1423	#endif	1430	#endif
1424		1431
		1432	#if defined(CONFIG_ACPI) && defined(CONFIG_SUSPEND)
		1433	bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev);
		1434	bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev);
		1435	#else
		1436	static inline bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) { return false; }
		1437	static inline bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev) { return false; }
		1438	#endif
		1439
1425	int amdgpu_cs_find_mapping(struct amdgpu_cs_parser *parser,	1440	int amdgpu_cs_find_mapping(struct amdgpu_cs_parser *parser,
1426	uint64_t addr, struct amdgpu_bo **bo,	1441	uint64_t addr, struct amdgpu_bo **bo,
1427	struct amdgpu_bo_va_mapping **mapping);	1442	struct amdgpu_bo_va_mapping **mapping);
@@ -1452,6 +1467,15 @@ int amdgpu_device_set_cg_state(struct amdgpu_device *adev, · 24 unmodified lines
1452	int amdgpu_device_set_pg_state(struct amdgpu_device *adev,	1467	int amdgpu_device_set_pg_state(struct amdgpu_device *adev,
1453	enum amd_powergating_state state);	1468	enum amd_powergating_state state);
1454		1469
		1470	static inline bool amdgpu_device_has_timeouts_enabled(struct amdgpu_device *adev)
		1471	{
		1472	return amdgpu_gpu_recovery != 0 &&
		1473	adev->gfx_timeout != MAX_SCHEDULE_TIMEOUT &&
		1474	adev->compute_timeout != MAX_SCHEDULE_TIMEOUT &&
		1475	adev->sdma_timeout != MAX_SCHEDULE_TIMEOUT &&
		1476	adev->video_timeout != MAX_SCHEDULE_TIMEOUT;
		1477	}
		1478
1455	#include "amdgpu_object.h"	1479	#include "amdgpu_object.h"
1456		1480
1457	static inline bool amdgpu_is_tmz(struct amdgpu_device *adev)	1481	static inline bool amdgpu_is_tmz(struct amdgpu_device *adev)
@@ -1459,8 +1483,6 @@ static inline bool amdgpu_is_tmz(struct amdgpu_device *adev) · 1 unmodified line
1459	return adev->gmc.tmz_enabled;	1483	return adev->gmc.tmz_enabled;
1460	}	1484	}
1461		1485
1462	static inline int amdgpu_in_reset(struct amdgpu_device *adev)	1486	int amdgpu_in_reset(struct amdgpu_device *adev);
1463	{	1487
1464	return atomic_read(&adev->in_gpu_reset);
1465	}
1466	#endif	1488	#endif

drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c+32−5

@@ -1031,6 +1031,20 @@ void amdgpu_acpi_detect(void) · 1030 unmodified lines
1031	}	1031	}
1032	}	1032	}
1033		1033
		1034	#if IS_ENABLED(CONFIG_SUSPEND)
		1035	/**
		1036	* amdgpu_acpi_is_s3_active
		1037	*
		1038	* @adev: amdgpu_device_pointer
		1039	*
		1040	* returns true if supported, false if not.
		1041	*/
		1042	bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev)
		1043	{
		1044	return !(adev->flags & AMD_IS_APU) \|\|
		1045	(pm_suspend_target_state == PM_SUSPEND_MEM);
		1046	}
		1047
1034	/**	1048	/**
1035	* amdgpu_acpi_is_s0ix_active	1049	* amdgpu_acpi_is_s0ix_active
1036	*	1050	*
@@ -1040,11 +1054,24 @@ void amdgpu_acpi_detect(void) · 3 unmodified lines
1040	*/	1054	*/
1041	bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev)	1055	bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev)
1042	{	1056	{
1043	#if IS_ENABLED(CONFIG_AMD_PMC) && IS_ENABLED(CONFIG_SUSPEND)	1057	if (!(adev->flags & AMD_IS_APU) \|\|
1044	if (acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0) {	1058	(pm_suspend_target_state != PM_SUSPEND_TO_IDLE))
1045	if (adev->flags & AMD_IS_APU)	1059	return false;
1046	return pm_suspend_target_state == PM_SUSPEND_TO_IDLE;	1060
		1061	if (!(acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0)) {
		1062	dev_warn_once(adev->dev,
		1063	"Power consumption will be higher as BIOS has not been configured for suspend-to-idle.\n"
		1064	"To use suspend-to-idle change the sleep mode in BIOS setup.\n");
		1065	return false;
1047	}	1066	}
1048	#endif	1067
		1068	#if !IS_ENABLED(CONFIG_AMD_PMC)
		1069	dev_warn_once(adev->dev,
		1070	"Power consumption will be higher as the kernel has not been compiled with CONFIG_AMD_PMC.\n");
1049	return false;	1071	return false;
		1072	#else
		1073	return true;
		1074	#endif /* CONFIG_AMD_PMC */
1050	}	1075	}
		1076
		1077	#endif /* CONFIG_SUSPEND */

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h+6−1

 	atomic_t evicted_bos;
 	struct delayed_work restore_userptr_work;
 	struct pid *pid;
+	bool block_mmu_notifications;
 };
 int amdgpu_amdkfd_init(void);
 int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
 		struct amdgpu_device *adev, uint64_t va, uint64_t size,
 		void *drm_priv, struct kgd_mem **mem,
-		uint64_t *offset, uint32_t flags);
+		uint64_t *offset, uint32_t flags, bool criu_resume);
 int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
 		struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv,
 		uint64_t *size);
 				struct tile_config *config);
 void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev,
 				bool reset);
+bool amdgpu_amdkfd_bo_mapped_to_dev(struct amdgpu_device *adev, struct kgd_mem *mem);
+void amdgpu_amdkfd_block_mmu_notifications(void *p);
+int amdgpu_amdkfd_criu_resume(void *p);
 #if IS_ENABLED(CONFIG_HSA_AMD)
 void amdgpu_amdkfd_gpuvm_init_mem_limits(void);
 void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev,

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_aldebaran.c+0−3

 	.hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied,
 	.hqd_destroy = kgd_gfx_v9_hqd_destroy,
 	.hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy,
-	.address_watch_disable = kgd_gfx_v9_address_watch_disable,
-	.address_watch_execute = kgd_gfx_v9_address_watch_execute,
 	.wave_control_execute = kgd_gfx_v9_wave_control_execute,
-	.address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset,
 	.get_atc_vmid_pasid_mapping_info =
 				kgd_gfx_v9_get_atc_vmid_pasid_mapping_info,
 	.set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base,

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c+0−3

 	.hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied,
 	.hqd_destroy = kgd_gfx_v9_hqd_destroy,
 	.hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy,
-	.address_watch_disable = kgd_gfx_v9_address_watch_disable,
-	.address_watch_execute = kgd_gfx_v9_address_watch_execute,
 	.wave_control_execute = kgd_gfx_v9_wave_control_execute,
-	.address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset,
 	.get_atc_vmid_pasid_mapping_info =
 				kgd_gfx_v9_get_atc_vmid_pasid_mapping_info,
 	.set_vm_context_page_table_base =

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c+0−24

@@ -671,20 +671,6 @@ static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 670 unmodified lines
671	return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);	671	return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);
672	}	672	}
673		673
674	static int kgd_address_watch_disable(struct amdgpu_device *adev)
675	{
676	return 0;
677	}
678
679	static int kgd_address_watch_execute(struct amdgpu_device *adev,
680	unsigned int watch_point_id,
681	uint32_t cntl_val,
682	uint32_t addr_hi,
683	uint32_t addr_lo)
684	{
685	return 0;
686	}
687
688	static int kgd_wave_control_execute(struct amdgpu_device *adev,	674	static int kgd_wave_control_execute(struct amdgpu_device *adev,
689	uint32_t gfx_index_val,	675	uint32_t gfx_index_val,
690	uint32_t sq_cmd)	676	uint32_t sq_cmd)
@@ -709,13 +695,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines
709	return 0;	695	return 0;
710	}	696	}
711		697
712	static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev,
713	unsigned int watch_point_id,
714	unsigned int reg_offset)
715	{
716	return 0;
717	}
718
719	static void set_vm_context_page_table_base(struct amdgpu_device *adev,	698	static void set_vm_context_page_table_base(struct amdgpu_device *adev,
720	uint32_t vmid, uint64_t page_table_base)	699	uint32_t vmid, uint64_t page_table_base)
721	{	700	{
@@ -767,10 +746,7 @@ const struct kfd2kgd_calls gfx_v10_kfd2kgd = { · 45 unmodified lines
767	.hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied,	746	.hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied,
768	.hqd_destroy = kgd_hqd_destroy,	747	.hqd_destroy = kgd_hqd_destroy,
769	.hqd_sdma_destroy = kgd_hqd_sdma_destroy,	748	.hqd_sdma_destroy = kgd_hqd_sdma_destroy,
770	.address_watch_disable = kgd_address_watch_disable,
771	.address_watch_execute = kgd_address_watch_execute,
772	.wave_control_execute = kgd_wave_control_execute,	749	.wave_control_execute = kgd_wave_control_execute,
773	.address_watch_get_offset = kgd_address_watch_get_offset,
774	.get_atc_vmid_pasid_mapping_info =	750	.get_atc_vmid_pasid_mapping_info =
775	get_atc_vmid_pasid_mapping_info,	751	get_atc_vmid_pasid_mapping_info,
776	.set_vm_context_page_table_base = set_vm_context_page_table_base,	752	.set_vm_context_page_table_base = set_vm_context_page_table_base,

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10_3.c+0−25

@@ -582,21 +582,6 @@ static int hqd_sdma_destroy_v10_3(struct amdgpu_device adev, void mqd, · 581 unmodified lines
582	return 0;	582	return 0;
583	}	583	}
584		584
585
586	static int address_watch_disable_v10_3(struct amdgpu_device *adev)
587	{
588	return 0;
589	}
590
591	static int address_watch_execute_v10_3(struct amdgpu_device *adev,
592	unsigned int watch_point_id,
593	uint32_t cntl_val,
594	uint32_t addr_hi,
595	uint32_t addr_lo)
596	{
597	return 0;
598	}
599
600	static int wave_control_execute_v10_3(struct amdgpu_device *adev,	585	static int wave_control_execute_v10_3(struct amdgpu_device *adev,
601	uint32_t gfx_index_val,	586	uint32_t gfx_index_val,
602	uint32_t sq_cmd)	587	uint32_t sq_cmd)
@@ -621,13 +606,6 @@ static int wave_control_execute_v10_3(struct amdgpu_device *adev, · 18 unmodified lines
621	return 0;	606	return 0;
622	}	607	}
623		608
624	static uint32_t address_watch_get_offset_v10_3(struct amdgpu_device *adev,
625	unsigned int watch_point_id,
626	unsigned int reg_offset)
627	{
628	return 0;
629	}
630
631	static void set_vm_context_page_table_base_v10_3(struct amdgpu_device *adev,	609	static void set_vm_context_page_table_base_v10_3(struct amdgpu_device *adev,
632	uint32_t vmid, uint64_t page_table_base)	610	uint32_t vmid, uint64_t page_table_base)
633	{	611	{
@@ -809,10 +787,7 @@ const struct kfd2kgd_calls gfx_v10_3_kfd2kgd = { · 175 unmodified lines
809	.hqd_sdma_is_occupied = hqd_sdma_is_occupied_v10_3,	787	.hqd_sdma_is_occupied = hqd_sdma_is_occupied_v10_3,
810	.hqd_destroy = hqd_destroy_v10_3,	788	.hqd_destroy = hqd_destroy_v10_3,
811	.hqd_sdma_destroy = hqd_sdma_destroy_v10_3,	789	.hqd_sdma_destroy = hqd_sdma_destroy_v10_3,
812	.address_watch_disable = address_watch_disable_v10_3,
813	.address_watch_execute = address_watch_execute_v10_3,
814	.wave_control_execute = wave_control_execute_v10_3,	790	.wave_control_execute = wave_control_execute_v10_3,
815	.address_watch_get_offset = address_watch_get_offset_v10_3,
816	.get_atc_vmid_pasid_mapping_info = NULL,	791	.get_atc_vmid_pasid_mapping_info = NULL,
817	.set_vm_context_page_table_base = set_vm_context_page_table_base_v10_3,	792	.set_vm_context_page_table_base = set_vm_context_page_table_base_v10_3,
818	.program_trap_handler_settings = program_trap_handler_settings_v10_3,	793	.program_trap_handler_settings = program_trap_handler_settings_v10_3,

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c+0−96

@@ -45,43 +45,6 @@ enum { · 44 unmodified lines
45	MAX_WATCH_ADDRESSES = 4	45	MAX_WATCH_ADDRESSES = 4
46	};	46	};
47		47
48	enum {
49	ADDRESS_WATCH_REG_ADDR_HI = 0,
50	ADDRESS_WATCH_REG_ADDR_LO,
51	ADDRESS_WATCH_REG_CNTL,
52	ADDRESS_WATCH_REG_MAX
53	};
54
55	/* not defined in the CI/KV reg file */
56	enum {
57	ADDRESS_WATCH_REG_CNTL_ATC_BIT = 0x10000000UL,
58	ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK = 0x00FFFFFF,
59	ADDRESS_WATCH_REG_ADDLOW_MASK_EXTENSION = 0x03000000,
60	/* extend the mask to 26 bits to match the low address field */
61	ADDRESS_WATCH_REG_ADDLOW_SHIFT = 6,
62	ADDRESS_WATCH_REG_ADDHIGH_MASK = 0xFFFF
63	};
64
65	static const uint32_t watchRegs[MAX_WATCH_ADDRESSES * ADDRESS_WATCH_REG_MAX] = {
66	mmTCP_WATCH0_ADDR_H, mmTCP_WATCH0_ADDR_L, mmTCP_WATCH0_CNTL,
67	mmTCP_WATCH1_ADDR_H, mmTCP_WATCH1_ADDR_L, mmTCP_WATCH1_CNTL,
68	mmTCP_WATCH2_ADDR_H, mmTCP_WATCH2_ADDR_L, mmTCP_WATCH2_CNTL,
69	mmTCP_WATCH3_ADDR_H, mmTCP_WATCH3_ADDR_L, mmTCP_WATCH3_CNTL
70	};
71
72	union TCP_WATCH_CNTL_BITS {
73	struct {
74	uint32_t mask:24;
75	uint32_t vmid:4;
76	uint32_t atc:1;
77	uint32_t mode:2;
78	uint32_t valid:1;
79	} bitfields, bits;
80	uint32_t u32All;
81	signed int i32All;
82	float f32All;
83	};
84
85	static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe,	48	static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe,
86	uint32_t queue, uint32_t vmid)	49	uint32_t queue, uint32_t vmid)
87	{	50	{
@@ -529,55 +492,6 @@ static int kgd_hqd_sdma_destroy(struct amdgpu_device adev, void mqd, · 441 unmodified lines
529	return 0;	492	return 0;
530	}	493	}
531		494
532	static int kgd_address_watch_disable(struct amdgpu_device *adev)
533	{
534	union TCP_WATCH_CNTL_BITS cntl;
535	unsigned int i;
536
537	cntl.u32All = 0;
538
539	cntl.bitfields.valid = 0;
540	cntl.bitfields.mask = ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK;
541	cntl.bitfields.atc = 1;
542
543	/* Turning off this address until we set all the registers */
544	for (i = 0; i < MAX_WATCH_ADDRESSES; i++)
545	WREG32(watchRegs[i * ADDRESS_WATCH_REG_MAX +
546	ADDRESS_WATCH_REG_CNTL], cntl.u32All);
547
548	return 0;
549	}
550
551	static int kgd_address_watch_execute(struct amdgpu_device *adev,
552	unsigned int watch_point_id,
553	uint32_t cntl_val,
554	uint32_t addr_hi,
555	uint32_t addr_lo)
556	{
557	union TCP_WATCH_CNTL_BITS cntl;
558
559	cntl.u32All = cntl_val;
560
561	/* Turning off this watch point until we set all the registers */
562	cntl.bitfields.valid = 0;
563	WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX +
564	ADDRESS_WATCH_REG_CNTL], cntl.u32All);
565
566	WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX +
567	ADDRESS_WATCH_REG_ADDR_HI], addr_hi);
568
569	WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX +
570	ADDRESS_WATCH_REG_ADDR_LO], addr_lo);
571
572	/* Enable the watch point */
573	cntl.bitfields.valid = 1;
574
575	WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX +
576	ADDRESS_WATCH_REG_CNTL], cntl.u32All);
577
578	return 0;
579	}
580
581	static int kgd_wave_control_execute(struct amdgpu_device *adev,	495	static int kgd_wave_control_execute(struct amdgpu_device *adev,
582	uint32_t gfx_index_val,	496	uint32_t gfx_index_val,
583	uint32_t sq_cmd)	497	uint32_t sq_cmd)
@@ -602,13 +516,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines
602	return 0;	516	return 0;
603	}	517	}
604		518
605	static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev,
606	unsigned int watch_point_id,
607	unsigned int reg_offset)
608	{
609	return watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + reg_offset];
610	}
611
612	static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,	519	static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,
613	uint8_t vmid, uint16_t *p_pasid)	520	uint8_t vmid, uint16_t *p_pasid)
614	{	521	{
@@ -665,10 +572,7 @@ const struct kfd2kgd_calls gfx_v7_kfd2kgd = { · 50 unmodified lines
665	.hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied,	572	.hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied,
666	.hqd_destroy = kgd_hqd_destroy,	573	.hqd_destroy = kgd_hqd_destroy,
667	.hqd_sdma_destroy = kgd_hqd_sdma_destroy,	574	.hqd_sdma_destroy = kgd_hqd_sdma_destroy,
668	.address_watch_disable = kgd_address_watch_disable,
669	.address_watch_execute = kgd_address_watch_execute,
670	.wave_control_execute = kgd_wave_control_execute,	575	.wave_control_execute = kgd_wave_control_execute,
671	.address_watch_get_offset = kgd_address_watch_get_offset,
672	.get_atc_vmid_pasid_mapping_info = get_atc_vmid_pasid_mapping_info,	576	.get_atc_vmid_pasid_mapping_info = get_atc_vmid_pasid_mapping_info,
673	.set_scratch_backing_va = set_scratch_backing_va,	577	.set_scratch_backing_va = set_scratch_backing_va,
674	.set_vm_context_page_table_base = set_vm_context_page_table_base,	578	.set_vm_context_page_table_base = set_vm_context_page_table_base,

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c+0−24

@@ -538,20 +538,6 @@ static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 537 unmodified lines
538	return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);	538	return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);
539	}	539	}
540		540
541	static int kgd_address_watch_disable(struct amdgpu_device *adev)
542	{
543	return 0;
544	}
545
546	static int kgd_address_watch_execute(struct amdgpu_device *adev,
547	unsigned int watch_point_id,
548	uint32_t cntl_val,
549	uint32_t addr_hi,
550	uint32_t addr_lo)
551	{
552	return 0;
553	}
554
555	static int kgd_wave_control_execute(struct amdgpu_device *adev,	541	static int kgd_wave_control_execute(struct amdgpu_device *adev,
556	uint32_t gfx_index_val,	542	uint32_t gfx_index_val,
557	uint32_t sq_cmd)	543	uint32_t sq_cmd)
@@ -576,13 +562,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines
576	return 0;	562	return 0;
577	}	563	}
578		564
579	static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev,
580	unsigned int watch_point_id,
581	unsigned int reg_offset)
582	{
583	return 0;
584	}
585
586	static void set_scratch_backing_va(struct amdgpu_device *adev,	565	static void set_scratch_backing_va(struct amdgpu_device *adev,
587	uint64_t va, uint32_t vmid)	566	uint64_t va, uint32_t vmid)
588	{	567	{
@@ -614,10 +593,7 @@ const struct kfd2kgd_calls gfx_v8_kfd2kgd = { · 25 unmodified lines
614	.hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied,	593	.hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied,
615	.hqd_destroy = kgd_hqd_destroy,	594	.hqd_destroy = kgd_hqd_destroy,
616	.hqd_sdma_destroy = kgd_hqd_sdma_destroy,	595	.hqd_sdma_destroy = kgd_hqd_sdma_destroy,
617	.address_watch_disable = kgd_address_watch_disable,
618	.address_watch_execute = kgd_address_watch_execute,
619	.wave_control_execute = kgd_wave_control_execute,	596	.wave_control_execute = kgd_wave_control_execute,
620	.address_watch_get_offset = kgd_address_watch_get_offset,
621	.get_atc_vmid_pasid_mapping_info =	597	.get_atc_vmid_pasid_mapping_info =
622	get_atc_vmid_pasid_mapping_info,	598	get_atc_vmid_pasid_mapping_info,
623	.set_scratch_backing_va = set_scratch_backing_va,	599	.set_scratch_backing_va = set_scratch_backing_va,

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c+2−26

 	return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK);
 }
-int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev)
-{
-	return 0;
-}
-int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev,
-					unsigned int watch_point_id,
-					uint32_t cntl_val,
-					uint32_t addr_hi,
-					uint32_t addr_lo)
-{
-	return 0;
-}
 int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev,
 					uint32_t gfx_index_val,
 					uint32_t sq_cmd)
 	return 0;
 }
-uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev,
-					unsigned int watch_point_id,
-					unsigned int reg_offset)
-{
-	return 0;
-}
 void kgd_gfx_v9_set_vm_context_page_table_base(struct amdgpu_device *adev,
 			uint32_t vmid, uint64_t page_table_base)
 {
  * process whose pasid is provided as a parameter. The process could have ZERO
  * or more queues running and submitting waves to compute units.
  *
- * @kgd: Handle of device from which to get number of waves in flight
+ * @adev: Handle of device from which to get number of waves in flight
  * @pasid: Identifies the process for which this query call is invoked
  * @pasid_wave_cnt: Output parameter updated with number of waves in flight that
  * belong to process with given pasid
  *
  * Note: It's possible that the device has too many queues (oversubscription)
  * in which case a VMID could be remapped to a different PASID. This could lead
- * to an iaccurate wave count. Following is a high-level sequence:
+ * to an inaccurate wave count. Following is a high-level sequence:
  *    Time T1: vmid = getVmid(); vmid is associated with Pasid P1
  *    Time T2: passId = getPasId(vmid); vmid is associated with Pasid P2
  * In the sequence above wave count obtained from time T1 will be incorrectly
 	.hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied,
 	.hqd_destroy = kgd_gfx_v9_hqd_destroy,
 	.hqd_sdma_destroy = kgd_hqd_sdma_destroy,
-	.address_watch_disable = kgd_gfx_v9_address_watch_disable,
-	.address_watch_execute = kgd_gfx_v9_address_watch_execute,
 	.wave_control_execute = kgd_gfx_v9_wave_control_execute,
-	.address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset,
 	.get_atc_vmid_pasid_mapping_info =
 			kgd_gfx_v9_get_atc_vmid_pasid_mapping_info,
 	.set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base,

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.h+0−10

@@ -46,19 +46,9 @@ int kgd_gfx_v9_hqd_destroy(struct amdgpu_device adev, void mqd, · 45 unmodified lines
46	enum kfd_preempt_type reset_type,	46	enum kfd_preempt_type reset_type,
47	unsigned int utimeout, uint32_t pipe_id,	47	unsigned int utimeout, uint32_t pipe_id,
48	uint32_t queue_id);	48	uint32_t queue_id);
49	int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev);
50	int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev,
51	unsigned int watch_point_id,
52	uint32_t cntl_val,
53	uint32_t addr_hi,
54	uint32_t addr_lo);
55	int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev,	49	int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev,
56	uint32_t gfx_index_val,	50	uint32_t gfx_index_val,
57	uint32_t sq_cmd);	51	uint32_t sq_cmd);
58	uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev,
59	unsigned int watch_point_id,
60	unsigned int reg_offset);
61
62	bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,	52	bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev,
63	uint8_t vmid, uint16_t *p_pasid);	53	uint8_t vmid, uint16_t *p_pasid);
64		54

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c+64−12

 }
 /**
- * @amdgpu_amdkfd_reserve_mem_limit() - Decrease available memory by size
+ * amdgpu_amdkfd_reserve_mem_limit() - Decrease available memory by size
  * of buffer including any reserved for control structures
  *
  * @adev: Device to which allocated BO belongs to
 			continue;
 		if (attachment[i]->bo_va) {
 			amdgpu_bo_reserve(bo[i], true);
-			amdgpu_vm_bo_rmv(adev, attachment[i]->bo_va);
+			amdgpu_vm_bo_del(adev, attachment[i]->bo_va);
 			amdgpu_bo_unreserve(bo[i]);
 			list_del(&attachment[i]->list);
 		}
 	pr_debug("\t remove VA 0x%llx in entry %p\n",
 			attachment->va, attachment);
-	amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va);
+	amdgpu_vm_bo_del(attachment->adev, attachment->bo_va);
 	drm_gem_object_put(&bo->tbo.base);
 	list_del(&attachment->list);
 	kfree(attachment);
  *
  * Returns 0 for success, negative errno for errors.
  */
-static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr)
+static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr,
+			   bool criu_resume)
 {
 	struct amdkfd_process_info *process_info = mem->process_info;
 	struct amdgpu_bo *bo = mem->bo;
 		goto out;
 	}
+	if (criu_resume) {
+		/*
+		 * During a CRIU restore operation, the userptr buffer objects
+		 * will be validated in the restore_userptr_work worker at a
+		 * later stage when it is scheduled by another ioctl called by
+		 * CRIU master process for the target pid for restore.
+		 */
+		atomic_inc(&mem->invalid);
+		mutex_unlock(&process_info->lock);
+		return 0;
+	}
 	ret = amdgpu_ttm_tt_get_user_pages(bo, bo->tbo.ttm->pages);
 	if (ret) {
 		pr_err("%s: Failed to get user pages: %d\n", __func__, ret);
 	return avm->pd_phys_addr;
 }
+void amdgpu_amdkfd_block_mmu_notifications(void *p)
+{
+	struct amdkfd_process_info *pinfo = (struct amdkfd_process_info *)p;
+	mutex_lock(&pinfo->lock);
+	WRITE_ONCE(pinfo->block_mmu_notifications, true);
+	mutex_unlock(&pinfo->lock);
+}
+int amdgpu_amdkfd_criu_resume(void *p)
+{
+	int ret = 0;
+	struct amdkfd_process_info *pinfo = (struct amdkfd_process_info *)p;
+	mutex_lock(&pinfo->lock);
+	pr_debug("scheduling work\n");
+	atomic_inc(&pinfo->evicted_bos);
+	if (!READ_ONCE(pinfo->block_mmu_notifications)) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+	WRITE_ONCE(pinfo->block_mmu_notifications, false);
+	schedule_delayed_work(&pinfo->restore_userptr_work, 0);
+out_unlock:
+	mutex_unlock(&pinfo->lock);
+	return ret;
+}
 int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
 		struct amdgpu_device *adev, uint64_t va, uint64_t size,
 		void *drm_priv, struct kgd_mem **mem,
-		uint64_t *offset, uint32_t flags)
+		uint64_t *offset, uint32_t flags, bool criu_resume)
 {
 	struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv);
 	enum ttm_bo_type bo_type = ttm_bo_type_device;
 	add_kgd_mem_to_kfd_bo_list(*mem, avm->process_info, user_addr);
 	if (user_addr) {
-		ret = init_user_pages(*mem, user_addr);
+		pr_debug("creating userptr BO for user_addr = %llu\n", user_addr);
+		ret = init_user_pages(*mem, user_addr, criu_resume);
 		if (ret)
 			goto allocate_init_user_pages_failed;
 	} else  if (flags & (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL |
 				true);
 	ret = unreserve_bo_and_vms(&ctx, false, false);
-	/* Only apply no TLB flush on Aldebaran to
-	 * workaround regressions on other Asics.
-	 */
-	if (table_freed && (adev->asic_type != CHIP_ALDEBARAN))
-		*table_freed = true;
 	goto out;
 out_unreserve:
 	int evicted_bos;
 	int r = 0;
+	/* Do not process MMU notifications until stage-4 IOCTL is received */
+	if (READ_ONCE(process_info->block_mmu_notifications))
+		return 0;
 	atomic_inc(&mem->invalid);
 	evicted_bos = atomic_inc_return(&process_info->evicted_bos);
 	if (evicted_bos == 1) {
 	return 0;
 }
+bool amdgpu_amdkfd_bo_mapped_to_dev(struct amdgpu_device *adev, struct kgd_mem *mem)
+{
+	struct kfd_mem_attachment *entry;
+	list_for_each_entry(entry, &mem->attachments, list) {
+		if (entry->is_mapped && entry->adev == adev)
+			return true;
+	}
+	return false;
+}

drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c+2−0

 	return 0;
 }
+#ifdef CONFIG_DRM_AMDGPU_SI
 int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev,
 					    u32 clock,
 					    bool strobe_mode,
 	}
 	return -EINVAL;
 }
+#endif
 bool amdgpu_atombios_has_gpu_virtualization_table(struct amdgpu_device *adev)
 {

drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h+12−10

 				       bool strobe_mode,
 				       struct atom_clock_dividers *dividers);
+#ifdef CONFIG_DRM_AMDGPU_SI
 int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev,
 					    u32 clock,
 					    bool strobe_mode,
 int amdgpu_atombios_init_mc_reg_table(struct amdgpu_device *adev,
 				      u8 module_index,
 				      struct atom_mc_reg_table *reg_table);
+int amdgpu_atombios_get_max_vddc(struct amdgpu_device *adev, u8 voltage_type,
+			     u16 voltage_id, u16 *voltage);
+int amdgpu_atombios_get_leakage_vddc_based_on_leakage_idx(struct amdgpu_device *adev,
+						      u16 *voltage,
+						      u16 leakage_idx);
+void amdgpu_atombios_get_default_voltages(struct amdgpu_device *adev,
+					  u16 *vddc, u16 *vddci, u16 *mvdd);
+int amdgpu_atombios_get_svi2_info(struct amdgpu_device *adev,
+			      u8 voltage_type,
+			      u8 *svd_gpio_id, u8 *svc_gpio_id);
+#endif
 bool amdgpu_atombios_has_gpu_virtualization_table(struct amdgpu_device *adev);
 bool amdgpu_atombios_scratch_need_asic_init(struct amdgpu_device *adev);
 void amdgpu_atombios_copy_swap(u8 *dst, u8 *src, u8 num_bytes, bool to_le);
-int amdgpu_atombios_get_max_vddc(struct amdgpu_device *adev, u8 voltage_type,
-			     u16 voltage_id, u16 *voltage);
-int amdgpu_atombios_get_leakage_vddc_based_on_leakage_idx(struct amdgpu_device *adev,
-						      u16 *voltage,
-						      u16 leakage_idx);
-void amdgpu_atombios_get_default_voltages(struct amdgpu_device *adev,
-					  u16 *vddc, u16 *vddci, u16 *mvdd);
 int amdgpu_atombios_get_clock_dividers(struct amdgpu_device *adev,
 				       u8 clock_type,
 				       u32 clock,
 				       bool strobe_mode,
 				       struct atom_clock_dividers *dividers);
-int amdgpu_atombios_get_svi2_info(struct amdgpu_device *adev,
-			      u8 voltage_type,
-			      u8 *svd_gpio_id, u8 *svc_gpio_id);
 int amdgpu_atombios_get_data_table(struct amdgpu_device *adev,
 				   uint32_t table,

drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c+38−0

@@ -464,3 +464,41 @@ success: · 463 unmodified lines
464	adev->is_atom_fw = (adev->asic_type >= CHIP_VEGA10) ? true : false;	464	adev->is_atom_fw = (adev->asic_type >= CHIP_VEGA10) ? true : false;
465	return true;	465	return true;
466	}	466	}
		467
		468	/* helper function for soc15 and onwards to read bios from rom */
		469	bool amdgpu_soc15_read_bios_from_rom(struct amdgpu_device *adev,
		470	u8 *bios, u32 length_bytes)
		471	{
		472	u32 *dw_ptr;
		473	u32 i, length_dw;
		474	u32 rom_index_offset;
		475	u32 rom_data_offset;
		476
		477	if (bios == NULL)
		478	return false;
		479	if (length_bytes == 0)
		480	return false;
		481	/* APU vbios image is part of sbios image */
		482	if (adev->flags & AMD_IS_APU)
		483	return false;
		484	if (!adev->smuio.funcs \|\|
		485	!adev->smuio.funcs->get_rom_index_offset \|\|
		486	!adev->smuio.funcs->get_rom_data_offset)
		487	return false;
		488
		489	dw_ptr = (u32 *)bios;
		490	length_dw = ALIGN(length_bytes, 4) / 4;
		491
		492	rom_index_offset =
		493	adev->smuio.funcs->get_rom_index_offset(adev);
		494	rom_data_offset =
		495	adev->smuio.funcs->get_rom_data_offset(adev);
		496
		497	/* set rom index to 0 */
		498	WREG32(rom_index_offset, 0);
		499	/* read out the rom data */
		500	for (i = 0; i < length_dw; i++)
		501	dw_ptr[i] = RREG32(rom_data_offset);
		502
		503	return true;
		504	}

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c+13−6

 		goto free_chunk;
 	}
-	mutex_lock(&p->ctx->lock);
 	/* skip guilty context job */
 	if (atomic_read(&p->ctx->guilty) == 1) {
 		ret = -ECANCELED;
 	}
 	total_vram = adev->gmc.real_vram_size - atomic64_read(&adev->vram_pin_size);
-	used_vram = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr);
+	used_vram = ttm_resource_manager_usage(&adev->mman.vram_mgr.manager);
 	free_vram = used_vram >= total_vram ? 0 : total_vram - used_vram;
 	spin_lock(&adev->mm_stats.lock);
 		}
 	}
+	/* Move fence waiting after getting reservation lock of
+	 * PD root. Then there is no need on a ctx mutex lock.
+	 */
+	r = amdgpu_ctx_wait_prev_fence(p->ctx, p->entity);
+	if (unlikely(r != 0)) {
+		if (r != -ERESTARTSYS)
+			DRM_ERROR("amdgpu_ctx_wait_prev_fence failed.\n");
+		goto error_validate;
+	}
 	amdgpu_cs_get_threshold_for_moves(p->adev, &p->bytes_moved_threshold,
 					  &p->bytes_moved_vis_threshold);
 	p->bytes_moved = 0;
 	dma_fence_put(parser->fence);
 	if (parser->ctx) {
-		mutex_unlock(&parser->ctx->lock);
 		amdgpu_ctx_put(parser->ctx);
 	}
 	if (parser->bo_list)
 	if (parser->job->uf_addr && ring->funcs->no_user_fence)
 		return -EINVAL;
-	return amdgpu_ctx_wait_prev_fence(parser->ctx, parser->entity);
+	return 0;
 }
 static int amdgpu_cs_process_fence_dep(struct amdgpu_cs_parser *p,
 		goto out;
 	r = amdgpu_cs_submit(&parser, cs);
 out:
 	amdgpu_cs_parser_fini(&parser, r, reserved_buffers);
 		return 0;
 	default:
+		dma_fence_put(fence);
 		return -EINVAL;
 	}
 }

drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c+1−1

@@ -98,7 +98,7 @@ int amdgpu_map_static_csa(struct amdgpu_device adev, struct amdgpu_vm vm, · 97 unmodified lines
98		98
99	if (r) {	99	if (r) {
100	DRM_ERROR("failed to do bo_map on static CSA, err=%d\n", r);	100	DRM_ERROR("failed to do bo_map on static CSA, err=%d\n", r);
101	amdgpu_vm_bo_rmv(adev, *bo_va);	101	amdgpu_vm_bo_del(adev, *bo_va);
102	ttm_eu_backoff_reservation(&ticket, &list);	102	ttm_eu_backoff_reservation(&ticket, &list);
103	return r;	103	return r;
104	}	104	}

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c+128−4

@@ -230,13 +230,13 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, · 229 unmodified lines
230		230
231	kref_init(&ctx->refcount);	231	kref_init(&ctx->refcount);
232	spin_lock_init(&ctx->ring_lock);	232	spin_lock_init(&ctx->ring_lock);
233	mutex_init(&ctx->lock);
234		233
235	ctx->reset_counter = atomic_read(&adev->gpu_reset_counter);	234	ctx->reset_counter = atomic_read(&adev->gpu_reset_counter);
236	ctx->reset_counter_query = ctx->reset_counter;	235	ctx->reset_counter_query = ctx->reset_counter;
237	ctx->vram_lost_counter = atomic_read(&adev->vram_lost_counter);	236	ctx->vram_lost_counter = atomic_read(&adev->vram_lost_counter);
238	ctx->init_priority = priority;	237	ctx->init_priority = priority;
239	ctx->override_priority = AMDGPU_CTX_PRIORITY_UNSET;	238	ctx->override_priority = AMDGPU_CTX_PRIORITY_UNSET;
		239	ctx->stable_pstate = AMDGPU_CTX_STABLE_PSTATE_NONE;
240		240
241	return 0;	241	return 0;
242	}	242	}
@@ -255,6 +255,86 @@ static void amdgpu_ctx_fini_entity(struct amdgpu_ctx_entity *entity) · 12 unmodified lines
255	kfree(entity);	255	kfree(entity);
256	}	256	}
257		257
		258	static int amdgpu_ctx_get_stable_pstate(struct amdgpu_ctx *ctx,
		259	u32 *stable_pstate)
		260	{
		261	struct amdgpu_device *adev = ctx->adev;
		262	enum amd_dpm_forced_level current_level;
		263
		264	if (!ctx)
		265	return -EINVAL;
		266
		267	current_level = amdgpu_dpm_get_performance_level(adev);
		268
		269	switch (current_level) {
		270	case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD:
		271	*stable_pstate = AMDGPU_CTX_STABLE_PSTATE_STANDARD;
		272	break;
		273	case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK:
		274	*stable_pstate = AMDGPU_CTX_STABLE_PSTATE_MIN_SCLK;
		275	break;
		276	case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
		277	*stable_pstate = AMDGPU_CTX_STABLE_PSTATE_MIN_MCLK;
		278	break;
		279	case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
		280	*stable_pstate = AMDGPU_CTX_STABLE_PSTATE_PEAK;
		281	break;
		282	default:
		283	*stable_pstate = AMDGPU_CTX_STABLE_PSTATE_NONE;
		284	break;
		285	}
		286	return 0;
		287	}
		288
		289	static int amdgpu_ctx_set_stable_pstate(struct amdgpu_ctx *ctx,
		290	u32 stable_pstate)
		291	{
		292	struct amdgpu_device *adev = ctx->adev;
		293	enum amd_dpm_forced_level level;
		294	int r;
		295
		296	if (!ctx)
		297	return -EINVAL;
		298
		299	mutex_lock(&adev->pm.stable_pstate_ctx_lock);
		300	if (adev->pm.stable_pstate_ctx && adev->pm.stable_pstate_ctx != ctx) {
		301	r = -EBUSY;
		302	goto done;
		303	}
		304
		305	switch (stable_pstate) {
		306	case AMDGPU_CTX_STABLE_PSTATE_NONE:
		307	level = AMD_DPM_FORCED_LEVEL_AUTO;
		308	break;
		309	case AMDGPU_CTX_STABLE_PSTATE_STANDARD:
		310	level = AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD;
		311	break;
		312	case AMDGPU_CTX_STABLE_PSTATE_MIN_SCLK:
		313	level = AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK;
		314	break;
		315	case AMDGPU_CTX_STABLE_PSTATE_MIN_MCLK:
		316	level = AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK;
		317	break;
		318	case AMDGPU_CTX_STABLE_PSTATE_PEAK:
		319	level = AMD_DPM_FORCED_LEVEL_PROFILE_PEAK;
		320	break;
		321	default:
		322	r = -EINVAL;
		323	goto done;
		324	}
		325
		326	r = amdgpu_dpm_force_performance_level(adev, level);
		327
		328	if (level == AMD_DPM_FORCED_LEVEL_AUTO)
		329	adev->pm.stable_pstate_ctx = NULL;
		330	else
		331	adev->pm.stable_pstate_ctx = ctx;
		332	done:
		333	mutex_unlock(&adev->pm.stable_pstate_ctx_lock);
		334
		335	return r;
		336	}
		337
258	static void amdgpu_ctx_fini(struct kref *ref)	338	static void amdgpu_ctx_fini(struct kref *ref)
259	{	339	{
260	struct amdgpu_ctx *ctx = container_of(ref, struct amdgpu_ctx, refcount);	340	struct amdgpu_ctx *ctx = container_of(ref, struct amdgpu_ctx, refcount);
@@ -270,8 +350,7 @@ static void amdgpu_ctx_fini(struct kref *ref) · 9 unmodified lines
270	ctx->entities[i][j] = NULL;	350	ctx->entities[i][j] = NULL;
271	}	351	}
272	}	352	}
273		353	amdgpu_ctx_set_stable_pstate(ctx, AMDGPU_CTX_STABLE_PSTATE_NONE);
274	mutex_destroy(&ctx->lock);
275	kfree(ctx);	354	kfree(ctx);
276	}	355	}
277		356
@@ -467,11 +546,41 @@ static int amdgpu_ctx_query2(struct amdgpu_device *adev, · 189 unmodified lines
467	return 0;	546	return 0;
468	}	547	}
469		548
		549
		550
		551	static int amdgpu_ctx_stable_pstate(struct amdgpu_device *adev,
		552	struct amdgpu_fpriv *fpriv, uint32_t id,
		553	bool set, u32 *stable_pstate)
		554	{
		555	struct amdgpu_ctx *ctx;
		556	struct amdgpu_ctx_mgr *mgr;
		557	int r;
		558
		559	if (!fpriv)
		560	return -EINVAL;
		561
		562	mgr = &fpriv->ctx_mgr;
		563	mutex_lock(&mgr->lock);
		564	ctx = idr_find(&mgr->ctx_handles, id);
		565	if (!ctx) {
		566	mutex_unlock(&mgr->lock);
		567	return -EINVAL;
		568	}
		569
		570	if (set)
		571	r = amdgpu_ctx_set_stable_pstate(ctx, *stable_pstate);
		572	else
		573	r = amdgpu_ctx_get_stable_pstate(ctx, stable_pstate);
		574
		575	mutex_unlock(&mgr->lock);
		576	return r;
		577	}
		578
470	int amdgpu_ctx_ioctl(struct drm_device dev, void data,	579	int amdgpu_ctx_ioctl(struct drm_device dev, void data,
471	struct drm_file *filp)	580	struct drm_file *filp)
472	{	581	{
473	int r;	582	int r;
474	uint32_t id;	583	uint32_t id, stable_pstate;
475	int32_t priority;	584	int32_t priority;
476		585
477	union drm_amdgpu_ctx *args = data;	586	union drm_amdgpu_ctx *args = data;
@@ -500,6 +609,21 @@ int amdgpu_ctx_ioctl(struct drm_device dev, void data, · 22 unmodified lines
500	case AMDGPU_CTX_OP_QUERY_STATE2:	609	case AMDGPU_CTX_OP_QUERY_STATE2:
501	r = amdgpu_ctx_query2(adev, fpriv, id, &args->out);	610	r = amdgpu_ctx_query2(adev, fpriv, id, &args->out);
502	break;	611	break;
		612	case AMDGPU_CTX_OP_GET_STABLE_PSTATE:
		613	if (args->in.flags)
		614	return -EINVAL;
		615	r = amdgpu_ctx_stable_pstate(adev, fpriv, id, false, &stable_pstate);
		616	if (!r)
		617	args->out.pstate.flags = stable_pstate;
		618	break;
		619	case AMDGPU_CTX_OP_SET_STABLE_PSTATE:
		620	if (args->in.flags & ~AMDGPU_CTX_STABLE_PSTATE_FLAGS_MASK)
		621	return -EINVAL;
		622	stable_pstate = args->in.flags & AMDGPU_CTX_STABLE_PSTATE_FLAGS_MASK;
		623	if (stable_pstate > AMDGPU_CTX_STABLE_PSTATE_PEAK)
		624	return -EINVAL;
		625	r = amdgpu_ctx_stable_pstate(adev, fpriv, id, true, &stable_pstate);
		626	break;
503	default:	627	default:
504	return -EINVAL;	628	return -EINVAL;
505	}	629	}

drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.h+1−1

 	bool				preamble_presented;
 	int32_t				init_priority;
 	int32_t				override_priority;
-	struct mutex			lock;
 	atomic_t			guilty;
 	unsigned long			ras_counter_ce;
 	unsigned long			ras_counter_ue;
+	uint32_t			stable_pstate;
 };
 struct amdgpu_ctx_mgr {

drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c+27−17

 #include "amdgpu_fw_attestation.h"
 #include "amdgpu_umr.h"
+#include "amdgpu_reset.h"
 #if defined(CONFIG_DEBUG_FS)
 /**
 		return -ENOMEM;
 	/* version, increment each time something is added */
-	config[no_regs++] = 3;
+	config[no_regs++] = 4;
 	config[no_regs++] = adev->gfx.config.max_shader_engines;
 	config[no_regs++] = adev->gfx.config.max_tile_pipes;
 	config[no_regs++] = adev->gfx.config.max_cu_per_sh;
 	config[no_regs++] = adev->pdev->subsystem_device;
 	config[no_regs++] = adev->pdev->subsystem_vendor;
+	/* rev==4 APU flag */
+	config[no_regs++] = adev->flags & AMD_IS_APU ? 1 : 0;
 	while (size && (*pos < no_regs * 4)) {
 		uint32_t value;
 		return -EINVAL;
 	r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
-	if (r < 0)
+	if (r < 0) {
+		pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
 		return r;
+	}
 	while (size) {
 		uint32_t value;
 	}
 	/* Avoid accidently unparking the sched thread during GPU reset */
-	r = down_write_killable(&adev->reset_sem);
+	r = down_write_killable(&adev->reset_domain->sem);
 	if (r)
 		return r;
 		kthread_unpark(ring->sched.thread);
 	}
-	up_write(&adev->reset_sem);
+	up_write(&adev->reset_domain->sem);
 	pm_runtime_mark_last_busy(dev->dev);
 	pm_runtime_put_autosuspend(dev->dev);
 		return -ENOMEM;
 	/* Avoid accidently unparking the sched thread during GPU reset */
-	r = down_read_killable(&adev->reset_sem);
+	r = down_read_killable(&adev->reset_domain->sem);
 	if (r)
 		goto pro_end;
 	/* restart the scheduler */
 	kthread_unpark(ring->sched.thread);
-	up_read(&adev->reset_sem);
+	up_read(&adev->reset_domain->sem);
 	ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, resched);
 		return ret;
 	}
-	if (is_support_sw_smu(adev)) {
+	ret = amdgpu_dpm_get_dpm_freq_range(adev, PP_SCLK, &min_freq, &max_freq);
-		ret = smu_get_dpm_freq_range(&adev->smu, SMU_SCLK, &min_freq, &max_freq);
+	if (ret == -EOPNOTSUPP) {
-		if (ret || val > max_freq || val < min_freq)
+		ret = 0;
-			return -EINVAL;
+		goto out;
-		ret = smu_set_soft_freq_range(&adev->smu, SMU_SCLK, (uint32_t)val, (uint32_t)val);
+	}
-	} else {
+	if (ret || val > max_freq || val < min_freq) {
-		return 0;
+		ret = -EINVAL;
+		goto out;
 	}
+	ret = amdgpu_dpm_set_soft_freq_range(adev, PP_SCLK, (uint32_t)val, (uint32_t)val);
+	if (ret)
+		ret = -EINVAL;
+out:
 	pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
 	pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
-	if (ret)
+	return ret;
-		return -EINVAL;
-	return 0;
 }
 DEFINE_DEBUGFS_ATTRIBUTE(fops_ib_preempt, NULL,

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c+262−143

@@ -31,6 +31,7 @@ · 30 unmodified lines
31	#include <linux/console.h>	31	#include <linux/console.h>
32	#include <linux/slab.h>	32	#include <linux/slab.h>
33	#include <linux/iommu.h>	33	#include <linux/iommu.h>
		34	#include <linux/pci.h>
34		35
35	#include <drm/drm_atomic_helper.h>	36	#include <drm/drm_atomic_helper.h>
36	#include <drm/drm_probe_helper.h>	37	#include <drm/drm_probe_helper.h>
@@ -55,7 +56,6 @@ · 18 unmodified lines
55	#include "soc15.h"	56	#include "soc15.h"
56	#include "nv.h"	57	#include "nv.h"
57	#include "bif/bif_4_1_d.h"	58	#include "bif/bif_4_1_d.h"
58	#include <linux/pci.h>
59	#include <linux/firmware.h>	59	#include <linux/firmware.h>
60	#include "amdgpu_vf_error.h"	60	#include "amdgpu_vf_error.h"
61		61
@@ -88,6 +88,8 @@ MODULE_FIRMWARE("amdgpu/vangogh_gpu_info.bin"); · 26 unmodified lines
88	MODULE_FIRMWARE("amdgpu/yellow_carp_gpu_info.bin");	88	MODULE_FIRMWARE("amdgpu/yellow_carp_gpu_info.bin");
89		89
90	#define AMDGPU_RESUME_MS 2000	90	#define AMDGPU_RESUME_MS 2000
		91	#define AMDGPU_MAX_RETRY_LIMIT 2
		92	#define AMDGPU_RETRY_SRIOV_RESET(r) ((r) == -EBUSY \|\| (r) == -ETIMEDOUT \|\| (r) == -EINVAL)
91		93
92	const char *amdgpu_asic_name[] = {	94	const char *amdgpu_asic_name[] = {
93	"TAHITI",	95	"TAHITI",
@@ -424,10 +426,10 @@ bool amdgpu_device_skip_hw_access(struct amdgpu_device *adev) · 330 unmodified lines
424	* the lock.	426	* the lock.
425	*/	427	*/
426	if (in_task()) {	428	if (in_task()) {
427	if (down_read_trylock(&adev->reset_sem))	429	if (down_read_trylock(&adev->reset_domain->sem))
428	up_read(&adev->reset_sem);	430	up_read(&adev->reset_domain->sem);
429	else	431	else
430	lockdep_assert_held(&adev->reset_sem);	432	lockdep_assert_held(&adev->reset_domain->sem);
431	}	433	}
432	#endif	434	#endif
433	return false;	435	return false;
@@ -453,9 +455,9 @@ uint32_t amdgpu_device_rreg(struct amdgpu_device *adev, · 19 unmodified lines
453	if ((reg * 4) < adev->rmmio_size) {	455	if ((reg * 4) < adev->rmmio_size) {
454	if (!(acc_flags & AMDGPU_REGS_NO_KIQ) &&	456	if (!(acc_flags & AMDGPU_REGS_NO_KIQ) &&
455	amdgpu_sriov_runtime(adev) &&	457	amdgpu_sriov_runtime(adev) &&
456	down_read_trylock(&adev->reset_sem)) {	458	down_read_trylock(&adev->reset_domain->sem)) {
457	ret = amdgpu_kiq_rreg(adev, reg);	459	ret = amdgpu_kiq_rreg(adev, reg);
458	up_read(&adev->reset_sem);	460	up_read(&adev->reset_domain->sem);
459	} else {	461	} else {
460	ret = readl(((void __iomem )adev->rmmio) + (reg 4));	462	ret = readl(((void __iomem )adev->rmmio) + (reg 4));
461	}	463	}
@@ -538,9 +540,9 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, · 76 unmodified lines
538	if ((reg * 4) < adev->rmmio_size) {	540	if ((reg * 4) < adev->rmmio_size) {
539	if (!(acc_flags & AMDGPU_REGS_NO_KIQ) &&	541	if (!(acc_flags & AMDGPU_REGS_NO_KIQ) &&
540	amdgpu_sriov_runtime(adev) &&	542	amdgpu_sriov_runtime(adev) &&
541	down_read_trylock(&adev->reset_sem)) {	543	down_read_trylock(&adev->reset_domain->sem)) {
542	amdgpu_kiq_wreg(adev, reg, v);	544	amdgpu_kiq_wreg(adev, reg, v);
543	up_read(&adev->reset_sem);	545	up_read(&adev->reset_domain->sem);
544	} else {	546	} else {
545	writel(v, ((void __iomem )adev->rmmio) + (reg 4));	547	writel(v, ((void __iomem )adev->rmmio) + (reg 4));
546	}	548	}
@@ -554,7 +556,11 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, · 7 unmodified lines
554	/**	556	/**
555	* amdgpu_mm_wreg_mmio_rlc - write register either with direct/indirect mmio or with RLC path if in range	557	* amdgpu_mm_wreg_mmio_rlc - write register either with direct/indirect mmio or with RLC path if in range
556	*	558	*
557	* this function is invoked only the debugfs register access	559	* @adev: amdgpu_device pointer
		560	* @reg: mmio/rlc register
		561	* @v: value to write
		562	*
		563	* this function is invoked only for the debugfs register access
558	*/	564	*/
559	void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,	565	void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
560	uint32_t reg, uint32_t v)	566	uint32_t reg, uint32_t v)
@@ -566,7 +572,7 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, · 5 unmodified lines
566	adev->gfx.rlc.funcs &&	572	adev->gfx.rlc.funcs &&
567	adev->gfx.rlc.funcs->is_rlcg_access_range) {	573	adev->gfx.rlc.funcs->is_rlcg_access_range) {
568	if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))	574	if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
569	return adev->gfx.rlc.funcs->sriov_wreg(adev, reg, v, 0, 0);	575	return amdgpu_sriov_wreg(adev, reg, v, 0, 0);
570	} else if ((reg * 4) >= adev->rmmio_size) {	576	} else if ((reg * 4) >= adev->rmmio_size) {
571	adev->pcie_wreg(adev, reg * 4, v);	577	adev->pcie_wreg(adev, reg * 4, v);
572	} else {	578	} else {
@@ -1312,6 +1318,31 @@ bool amdgpu_device_need_post(struct amdgpu_device *adev) · 739 unmodified lines
1312	return true;	1318	return true;
1313	}	1319	}
1314		1320
		1321	/**
		1322	* amdgpu_device_should_use_aspm - check if the device should program ASPM
		1323	*
		1324	* @adev: amdgpu_device pointer
		1325	*
		1326	* Confirm whether the module parameter and pcie bridge agree that ASPM should
		1327	* be set for this device.
		1328	*
		1329	* Returns true if it should be used or false if not.
		1330	*/
		1331	bool amdgpu_device_should_use_aspm(struct amdgpu_device *adev)
		1332	{
		1333	switch (amdgpu_aspm) {
		1334	case -1:
		1335	break;
		1336	case 0:
		1337	return false;
		1338	case 1:
		1339	return true;
		1340	default:
		1341	return false;
		1342	}
		1343	return pcie_aspm_enabled(adev->pdev);
		1344	}
		1345
1315	/* if we get transitioned to only one device, take VGA back */	1346	/* if we get transitioned to only one device, take VGA back */
1316	/**	1347	/**
1317	* amdgpu_device_vga_set_decode - enable/disable vga decode	1348	* amdgpu_device_vga_set_decode - enable/disable vga decode
@@ -1446,7 +1477,8 @@ static int amdgpu_device_init_apu_flags(struct amdgpu_device *adev) · 128 unmodified lines
1446	case CHIP_YELLOW_CARP:	1477	case CHIP_YELLOW_CARP:
1447	break;	1478	break;
1448	case CHIP_CYAN_SKILLFISH:	1479	case CHIP_CYAN_SKILLFISH:
1449	if (adev->pdev->device == 0x13FE)	1480	if ((adev->pdev->device == 0x13FE) \|\|
		1481	(adev->pdev->device == 0x143F))
1450	adev->apu_flags \|= AMD_APU_IS_CYAN_SKILLFISH2;	1482	adev->apu_flags \|= AMD_APU_IS_CYAN_SKILLFISH2;
1451	break;	1483	break;
1452	default:	1484	default:
@@ -2073,6 +2105,8 @@ out: · 620 unmodified lines
2073	*/	2105	*/
2074	static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)	2106	static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)
2075	{	2107	{
		2108	struct drm_device *dev = adev_to_drm(adev);
		2109	struct pci_dev *parent;
2076	int i, r;	2110	int i, r;
2077		2111
2078	amdgpu_device_enable_virtual_display(adev);	2112	amdgpu_device_enable_virtual_display(adev);
@@ -2137,6 +2171,16 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) · 58 unmodified lines
2137	break;	2171	break;
2138	}	2172	}
2139		2173
		2174	if (amdgpu_has_atpx() &&
		2175	(amdgpu_is_atpx_hybrid() \|\|
		2176	amdgpu_has_atpx_dgpu_power_cntl()) &&
		2177	((adev->flags & AMD_IS_APU) == 0) &&
		2178	!pci_is_thunderbolt_attached(to_pci_dev(dev->dev)))
		2179	adev->flags \|= AMD_IS_PX;
		2180
		2181	parent = pci_upstream_bridge(adev->pdev);
		2182	adev->has_pr3 = parent ? pci_pr3_present(parent) : false;
		2183
2140	amdgpu_amdkfd_device_probe(adev);	2184	amdgpu_amdkfd_device_probe(adev);
2141		2185
2142	adev->pm.pp_feature = amdgpu_pp_feature_mask;	2186	adev->pm.pp_feature = amdgpu_pp_feature_mask;
@@ -2287,6 +2331,49 @@ static int amdgpu_device_fw_loading(struct amdgpu_device *adev) · 144 unmodified lines
2287	return r;	2331	return r;
2288	}	2332	}
2289		2333
		2334	static int amdgpu_device_init_schedulers(struct amdgpu_device *adev)
		2335	{
		2336	long timeout;
		2337	int r, i;
		2338
		2339	for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
		2340	struct amdgpu_ring *ring = adev->rings[i];
		2341
		2342	/* No need to setup the GPU scheduler for rings that don't need it */
		2343	if (!ring \|\| ring->no_scheduler)
		2344	continue;
		2345
		2346	switch (ring->funcs->type) {
		2347	case AMDGPU_RING_TYPE_GFX:
		2348	timeout = adev->gfx_timeout;
		2349	break;
		2350	case AMDGPU_RING_TYPE_COMPUTE:
		2351	timeout = adev->compute_timeout;
		2352	break;
		2353	case AMDGPU_RING_TYPE_SDMA:
		2354	timeout = adev->sdma_timeout;
		2355	break;
		2356	default:
		2357	timeout = adev->video_timeout;
		2358	break;
		2359	}
		2360
		2361	r = drm_sched_init(&ring->sched, &amdgpu_sched_ops,
		2362	ring->num_hw_submission, amdgpu_job_hang_limit,
		2363	timeout, adev->reset_domain->wq,
		2364	ring->sched_score, ring->name,
		2365	adev->dev);
		2366	if (r) {
		2367	DRM_ERROR("Failed to create scheduler on ring %s.\n",
		2368	ring->name);
		2369	return r;
		2370	}
		2371	}
		2372
		2373	return 0;
		2374	}
		2375
		2376
2290	/**	2377	/**
2291	* amdgpu_device_ip_init - run init for hardware IPs	2378	* amdgpu_device_ip_init - run init for hardware IPs
2292	*	2379	*
@@ -2398,8 +2485,28 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev) · 105 unmodified lines
2398	if (r)	2485	if (r)
2399	goto init_failed;	2486	goto init_failed;
2400		2487
2401	if (adev->gmc.xgmi.num_physical_nodes > 1)	2488	/**
2402	amdgpu_xgmi_add_device(adev);	2489	* In case of XGMI grab extra reference for reset domain for this device
		2490	*/
		2491	if (adev->gmc.xgmi.num_physical_nodes > 1) {
		2492	if (amdgpu_xgmi_add_device(adev) == 0) {
		2493	struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev);
		2494
		2495	if (!hive->reset_domain \|\|
		2496	!amdgpu_reset_get_reset_domain(hive->reset_domain)) {
		2497	r = -ENOENT;
		2498	goto init_failed;
		2499	}
		2500
		2501	/* Drop the early temporary reset domain we created for device */
		2502	amdgpu_reset_put_reset_domain(adev->reset_domain);
		2503	adev->reset_domain = hive->reset_domain;
		2504	}
		2505	}
		2506
		2507	r = amdgpu_device_init_schedulers(adev);
		2508	if (r)
		2509	goto init_failed;
2403		2510
2404	/* Don't init kfd if whole hive need to be reset during init */	2511	/* Don't init kfd if whole hive need to be reset during init */
2405	if (!adev->gmc.xgmi.pending_reset)	2512	if (!adev->gmc.xgmi.pending_reset)
@@ -2610,6 +2717,12 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 204 unmodified lines
2610	adev->ip_blocks[i].status.late_initialized = true;	2717	adev->ip_blocks[i].status.late_initialized = true;
2611	}	2718	}
2612		2719
		2720	r = amdgpu_ras_late_init(adev);
		2721	if (r) {
		2722	DRM_ERROR("amdgpu_ras_late_init failed %d", r);
		2723	return r;
		2724	}
		2725
2613	amdgpu_ras_set_error_query_ready(adev, true);	2726	amdgpu_ras_set_error_query_ready(adev, true);
2614		2727
2615	amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);	2728	amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);
@@ -2624,7 +2737,7 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 8 unmodified lines
2624	/* For passthrough configuration on arcturus and aldebaran, enable special handling SBR */	2737	/* For passthrough configuration on arcturus and aldebaran, enable special handling SBR */
2625	if (amdgpu_passthrough(adev) && ((adev->asic_type == CHIP_ARCTURUS && adev->gmc.xgmi.num_physical_nodes > 1)\|\|	2738	if (amdgpu_passthrough(adev) && ((adev->asic_type == CHIP_ARCTURUS && adev->gmc.xgmi.num_physical_nodes > 1)\|\|
2626	adev->asic_type == CHIP_ALDEBARAN ))	2739	adev->asic_type == CHIP_ALDEBARAN ))
2627	smu_handle_passthrough_sbr(&adev->smu, true);	2740	amdgpu_dpm_handle_passthrough_sbr(adev, true);
2628		2741
2629	if (adev->gmc.xgmi.num_physical_nodes > 1) {	2742	if (adev->gmc.xgmi.num_physical_nodes > 1) {
2630	mutex_lock(&mgpu_info.mutex);	2743	mutex_lock(&mgpu_info.mutex);
@@ -2708,11 +2821,11 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 77 unmodified lines
2708	}	2821	}
2709	}	2822	}
2710		2823
2711	amdgpu_amdkfd_suspend(adev, false);
2712
2713	amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);	2824	amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
2714	amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);	2825	amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
2715		2826
		2827	amdgpu_amdkfd_suspend(adev, false);
		2828
2716	/* Workaroud for ASICs need to disable SMC first */	2829	/* Workaroud for ASICs need to disable SMC first */
2717	amdgpu_device_smu_fini_early(adev);	2830	amdgpu_device_smu_fini_early(adev);
2718		2831
@@ -2881,7 +2994,7 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) · 162 unmodified lines
2881	int i, r;	2994	int i, r;
2882		2995
2883	if (adev->in_s0ix)	2996	if (adev->in_s0ix)
2884	amdgpu_gfx_state_change_set(adev, sGpuChangeState_D3Entry);	2997	amdgpu_dpm_gfx_state_change(adev, sGpuChangeState_D3Entry);
2885		2998
2886	for (i = adev->num_ip_blocks - 1; i >= 0; i--) {	2999	for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
2887	if (!adev->ip_blocks[i].status.valid)	3000	if (!adev->ip_blocks[i].status.valid)
@@ -3307,9 +3420,9 @@ static void amdgpu_device_xgmi_reset_func(struct work_struct *__work) · 419 unmodified lines
3307	if (adev->asic_reset_res)	3420	if (adev->asic_reset_res)
3308	goto fail;	3421	goto fail;
3309		3422
3310	if (adev->mmhub.ras_funcs &&	3423	if (adev->mmhub.ras && adev->mmhub.ras->ras_block.hw_ops &&
3311	adev->mmhub.ras_funcs->reset_ras_error_count)	3424	adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count)
3312	adev->mmhub.ras_funcs->reset_ras_error_count(adev);	3425	adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(adev);
3313	} else {	3426	} else {
3314		3427
3315	task_barrier_full(&hive->tb);	3428	task_barrier_full(&hive->tb);
@@ -3493,12 +3606,11 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 177 unmodified lines
3493	mutex_init(&adev->mn_lock);	3606	mutex_init(&adev->mn_lock);
3494	mutex_init(&adev->virt.vf_errors.lock);	3607	mutex_init(&adev->virt.vf_errors.lock);
3495	hash_init(adev->mn_hash);	3608	hash_init(adev->mn_hash);
3496	atomic_set(&adev->in_gpu_reset, 0);
3497	init_rwsem(&adev->reset_sem);
3498	mutex_init(&adev->psp.mutex);	3609	mutex_init(&adev->psp.mutex);
3499	mutex_init(&adev->notifier_lock);	3610	mutex_init(&adev->notifier_lock);
		3611	mutex_init(&adev->pm.stable_pstate_ctx_lock);
3500		3612
3501	amdgpu_device_init_apu_flags(adev);	3613	amdgpu_device_init_apu_flags(adev);
3502		3614
3503	r = amdgpu_device_check_arguments(adev);	3615	r = amdgpu_device_check_arguments(adev);
3504	if (r)	3616	if (r)
@@ -3519,6 +3631,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 14 unmodified lines
3519		3631
3520	INIT_LIST_HEAD(&adev->reset_list);	3632	INIT_LIST_HEAD(&adev->reset_list);
3521		3633
		3634	INIT_LIST_HEAD(&adev->ras_list);
		3635
3522	INIT_DELAYED_WORK(&adev->delayed_init_work,	3636	INIT_DELAYED_WORK(&adev->delayed_init_work,
3523	amdgpu_device_delayed_init_work_handler);	3637	amdgpu_device_delayed_init_work_handler);
3524	INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work,	3638	INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work,
@@ -3577,6 +3691,15 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 52 unmodified lines
3577	return r;	3691	return r;
3578	}	3692	}
3579		3693
		3694	/*
		3695	* Reset domain needs to be present early, before XGMI hive discovered
		3696	* (if any) and intitialized to use reset sem and in_gpu reset flag
		3697	* early on during init.
		3698	*/
		3699	adev->reset_domain = amdgpu_reset_create_reset_domain(SINGLE_DEVICE ,"amdgpu-reset-dev");
		3700	if (!adev->reset_domain)
		3701	return -ENOMEM;
		3702
3580	/* early init functions */	3703	/* early init functions */
3581	r = amdgpu_device_ip_early_init(adev);	3704	r = amdgpu_device_ip_early_init(adev);
3582	if (r)	3705	if (r)
@@ -3953,6 +4076,9 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) · 370 unmodified lines
3953	if (adev->mman.discovery_bin)	4076	if (adev->mman.discovery_bin)
3954	amdgpu_discovery_fini(adev);	4077	amdgpu_discovery_fini(adev);
3955		4078
		4079	amdgpu_reset_put_reset_domain(adev->reset_domain);
		4080	adev->reset_domain = NULL;
		4081
3956	kfree(adev->pci_state);	4082	kfree(adev->pci_state);
3957		4083
3958	}	4084	}
@@ -4044,7 +4170,7 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon) · 85 unmodified lines
4044	return 0;	4170	return 0;
4045		4171
4046	if (adev->in_s0ix)	4172	if (adev->in_s0ix)
4047	amdgpu_gfx_state_change_set(adev, sGpuChangeState_D0Entry);	4173	amdgpu_dpm_gfx_state_change(adev, sGpuChangeState_D0Entry);
4048		4174
4049	/* post card */	4175	/* post card */
4050	if (amdgpu_device_need_post(adev)) {	4176	if (amdgpu_device_need_post(adev)) {
@@ -4347,7 +4473,9 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, · 296 unmodified lines
4347	{	4473	{
4348	int r;	4474	int r;
4349	struct amdgpu_hive_info *hive = NULL;	4475	struct amdgpu_hive_info *hive = NULL;
		4476	int retry_limit = 0;
4350		4477
		4478	retry:
4351	amdgpu_amdkfd_pre_reset(adev);	4479	amdgpu_amdkfd_pre_reset(adev);
4352		4480
4353	amdgpu_amdkfd_pre_reset(adev);	4481	amdgpu_amdkfd_pre_reset(adev);
@@ -4396,6 +4524,14 @@ error: · 42 unmodified lines
4396	}	4524	}
4397	amdgpu_virt_release_full_gpu(adev, true);	4525	amdgpu_virt_release_full_gpu(adev, true);
4398		4526
		4527	if (AMDGPU_RETRY_SRIOV_RESET(r)) {
		4528	if (retry_limit < AMDGPU_MAX_RETRY_LIMIT) {
		4529	retry_limit++;
		4530	goto retry;
		4531	} else
		4532	DRM_ERROR("GPU reset retry is beyond the retry limit\n");
		4533	}
		4534
4399	return r;	4535	return r;
4400	}	4536	}
4401		4537
@@ -4645,9 +4781,9 @@ int amdgpu_do_asic_reset(struct list_head *device_list_handle, · 243 unmodified lines
4645		4781
4646	if (!r && amdgpu_ras_intr_triggered()) {	4782	if (!r && amdgpu_ras_intr_triggered()) {
4647	list_for_each_entry(tmp_adev, device_list_handle, reset_list) {	4783	list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
4648	if (tmp_adev->mmhub.ras_funcs &&	4784	if (tmp_adev->mmhub.ras && tmp_adev->mmhub.ras->ras_block.hw_ops &&
4649	tmp_adev->mmhub.ras_funcs->reset_ras_error_count)	4785	tmp_adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count)
4650	tmp_adev->mmhub.ras_funcs->reset_ras_error_count(tmp_adev);	4786	tmp_adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(tmp_adev);
4651	}	4787	}
4652		4788
4653	amdgpu_ras_intr_cleared();	4789	amdgpu_ras_intr_cleared();
@@ -4754,17 +4890,8 @@ end: · 100 unmodified lines
4754	return r;	4890	return r;
4755	}	4891	}
4756		4892
4757	static bool amdgpu_device_lock_adev(struct amdgpu_device *adev,	4893	static void amdgpu_device_set_mp1_state(struct amdgpu_device *adev)
4758	struct amdgpu_hive_info *hive)
4759	{	4894	{
4760	if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0)
4761	return false;
4762
4763	if (hive) {
4764	down_write_nest_lock(&adev->reset_sem, &hive->hive_lock);
4765	} else {
4766	down_write(&adev->reset_sem);
4767	}
4768		4895
4769	switch (amdgpu_asic_reset_method(adev)) {	4896	switch (amdgpu_asic_reset_method(adev)) {
4770	case AMD_RESET_METHOD_MODE1:	4897	case AMD_RESET_METHOD_MODE1:
@@ -4777,56 +4904,12 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, · 6 unmodified lines
4777	adev->mp1_state = PP_MP1_STATE_NONE;	4904	adev->mp1_state = PP_MP1_STATE_NONE;
4778	break;	4905	break;
4779	}	4906	}
4780
4781	return true;
4782	}	4907	}
4783		4908
4784	static void amdgpu_device_unlock_adev(struct amdgpu_device *adev)	4909	static void amdgpu_device_unset_mp1_state(struct amdgpu_device *adev)
4785	{	4910	{
4786	amdgpu_vf_error_trans_all(adev);	4911	amdgpu_vf_error_trans_all(adev);
4787	adev->mp1_state = PP_MP1_STATE_NONE;	4912	adev->mp1_state = PP_MP1_STATE_NONE;
4788	atomic_set(&adev->in_gpu_reset, 0);
4789	up_write(&adev->reset_sem);
4790	}
4791
4792	/*
4793	* to lockup a list of amdgpu devices in a hive safely, if not a hive
4794	* with multiple nodes, it will be similar as amdgpu_device_lock_adev.
4795	*
4796	* unlock won't require roll back.
4797	*/
4798	static int amdgpu_device_lock_hive_adev(struct amdgpu_device adev, struct amdgpu_hive_info hive)
4799	{
4800	struct amdgpu_device *tmp_adev = NULL;
4801
4802	if (!amdgpu_sriov_vf(adev) && (adev->gmc.xgmi.num_physical_nodes > 1)) {
4803	if (!hive) {
4804	dev_err(adev->dev, "Hive is NULL while device has multiple xgmi nodes");
4805	return -ENODEV;
4806	}
4807	list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) {
4808	if (!amdgpu_device_lock_adev(tmp_adev, hive))
4809	goto roll_back;
4810	}
4811	} else if (!amdgpu_device_lock_adev(adev, hive))
4812	return -EAGAIN;
4813
4814	return 0;
4815	roll_back:
4816	if (!list_is_first(&tmp_adev->gmc.xgmi.head, &hive->device_list)) {
4817	/*
4818	* if the lockup iteration break in the middle of a hive,
4819	* it may means there may has a race issue,
4820	* or a hive device locked up independently.
4821	* we may be in trouble and may not, so will try to roll back
4822	* the lock and give out a warnning.
4823	*/
4824	dev_warn(tmp_adev->dev, "Hive lock iteration broke in the middle. Rolling back to unlock");
4825	list_for_each_entry_continue_reverse(tmp_adev, &hive->device_list, gmc.xgmi.head) {
4826	amdgpu_device_unlock_adev(tmp_adev);
4827	}
4828	}
4829	return -EAGAIN;
4830	}	4913	}
4831		4914
4832	static void amdgpu_device_resume_display_audio(struct amdgpu_device *adev)	4915	static void amdgpu_device_resume_display_audio(struct amdgpu_device *adev)
@@ -4960,7 +5043,7 @@ retry: · 127 unmodified lines
4960	}	5043	}
4961		5044
4962	/**	5045	/**
4963	* amdgpu_device_gpu_recover - reset the asic and recover scheduler	5046	* amdgpu_device_gpu_recover_imp - reset the asic and recover scheduler
4964	*	5047	*
4965	* @adev: amdgpu_device pointer	5048	* @adev: amdgpu_device pointer
4966	* @job: which job trigger hang	5049	* @job: which job trigger hang
@@ -4970,7 +5053,7 @@ retry: · 3 unmodified lines
4970	* Returns 0 for success or an error on failure.	5053	* Returns 0 for success or an error on failure.
4971	*/	5054	*/
4972		5055
4973	int amdgpu_device_gpu_recover(struct amdgpu_device *adev,	5056	int amdgpu_device_gpu_recover_imp(struct amdgpu_device *adev,
4974	struct amdgpu_job *job)	5057	struct amdgpu_job *job)
4975	{	5058	{
4976	struct list_head device_list, *device_list_handle = NULL;	5059	struct list_head device_list, *device_list_handle = NULL;
@@ -5004,26 +5087,10 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 27 unmodified lines
5004	dev_info(adev->dev, "GPU %s begin!\n",	5087	dev_info(adev->dev, "GPU %s begin!\n",
5005	need_emergency_restart ? "jobs stop":"reset");	5088	need_emergency_restart ? "jobs stop":"reset");
5006		5089
5007	/*
5008	* Here we trylock to avoid chain of resets executing from
5009	* either trigger by jobs on different adevs in XGMI hive or jobs on
5010	* different schedulers for same device while this TO handler is running.
5011	* We always reset all schedulers for device and all devices for XGMI
5012	* hive so that should take care of them too.
5013	*/
5014	if (!amdgpu_sriov_vf(adev))	5090	if (!amdgpu_sriov_vf(adev))
5015	hive = amdgpu_get_xgmi_hive(adev);	5091	hive = amdgpu_get_xgmi_hive(adev);
5016	if (hive) {	5092	if (hive)
5017	if (atomic_cmpxchg(&hive->in_reset, 0, 1) != 0) {
5018	DRM_INFO("Bailing on TDR for s_job:%llx, hive: %llx as another already in progress",
5019	job ? job->base.id : -1, hive->hive_id);
5020	amdgpu_put_xgmi_hive(hive);
5021	if (job && job->vm)
5022	drm_sched_increase_karma(&job->base);
5023	return 0;
5024	}
5025	mutex_lock(&hive->hive_lock);	5093	mutex_lock(&hive->hive_lock);
5026	}
5027		5094
5028	reset_context.method = AMD_RESET_METHOD_NONE;	5095	reset_context.method = AMD_RESET_METHOD_NONE;
5029	reset_context.reset_req_dev = adev;	5096	reset_context.reset_req_dev = adev;
@@ -5031,22 +5098,6 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 1 unmodified line
5031	reset_context.hive = hive;	5098	reset_context.hive = hive;
5032	clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);	5099	clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
5033		5100
5034	/*
5035	* lock the device before we try to operate the linked list
5036	* if didn't get the device lock, don't touch the linked list since
5037	* others may iterating it.
5038	*/
5039	r = amdgpu_device_lock_hive_adev(adev, hive);
5040	if (r) {
5041	dev_info(adev->dev, "Bailing on TDR for s_job:%llx, as another already in progress",
5042	job ? job->base.id : -1);
5043
5044	/* even we skipped this reset, still need to set the job to guilty */
5045	if (job && job->vm)
5046	drm_sched_increase_karma(&job->base);
5047	goto skip_recovery;
5048	}
5049
5050	/*	5101	/*
5051	* Build list of devices to reset.	5102	* Build list of devices to reset.
5052	* In case we are in XGMI hive mode, resort the device list	5103	* In case we are in XGMI hive mode, resort the device list
@@ -5064,8 +5115,16 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 11 unmodified lines
5064	device_list_handle = &device_list;	5115	device_list_handle = &device_list;
5065	}	5116	}
5066		5117
		5118	/* We need to lock reset domain only once both for XGMI and single device */
		5119	tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device,
		5120	reset_list);
		5121	amdgpu_device_lock_reset_domain(tmp_adev->reset_domain);
		5122
5067	/* block all schedulers and reset given job's ring */	5123	/* block all schedulers and reset given job's ring */
5068	list_for_each_entry(tmp_adev, device_list_handle, reset_list) {	5124	list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
		5125
		5126	amdgpu_device_set_mp1_state(tmp_adev);
		5127
5069	/*	5128	/*
5070	* Try to put the audio codec into suspend state	5129	* Try to put the audio codec into suspend state
5071	* before gpu reset started.	5130	* before gpu reset started.
@@ -5187,6 +5246,9 @@ skip_hw_reset: · 115 unmodified lines
5187	drm_helper_resume_force_mode(adev_to_drm(tmp_adev));	5246	drm_helper_resume_force_mode(adev_to_drm(tmp_adev));
5188	}	5247	}
5189		5248
		5249	if (tmp_adev->asic_reset_res)
		5250	r = tmp_adev->asic_reset_res;
		5251
5190	tmp_adev->asic_reset_res = 0;	5252	tmp_adev->asic_reset_res = 0;
5191		5253
5192	if (r) {	5254	if (r) {
@@ -5214,21 +5276,55 @@ skip_sched_resume: · 21 unmodified lines
5214		5276
5215	if (audio_suspended)	5277	if (audio_suspended)
5216	amdgpu_device_resume_display_audio(tmp_adev);	5278	amdgpu_device_resume_display_audio(tmp_adev);
5217	amdgpu_device_unlock_adev(tmp_adev);	5279
		5280	amdgpu_device_unset_mp1_state(tmp_adev);
5218	}	5281	}
5219		5282
5220	skip_recovery:	5283	tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device,
		5284	reset_list);
		5285	amdgpu_device_unlock_reset_domain(tmp_adev->reset_domain);
		5286
5221	if (hive) {	5287	if (hive) {
5222	atomic_set(&hive->in_reset, 0);
5223	mutex_unlock(&hive->hive_lock);	5288	mutex_unlock(&hive->hive_lock);
5224	amdgpu_put_xgmi_hive(hive);	5289	amdgpu_put_xgmi_hive(hive);
5225	}	5290	}
5226		5291
5227	if (r && r != -EAGAIN)	5292	if (r)
5228	dev_info(adev->dev, "GPU reset end with ret = %d\n", r);	5293	dev_info(adev->dev, "GPU reset end with ret = %d\n", r);
5229	return r;	5294	return r;
5230	}	5295	}
5231		5296
		5297	struct amdgpu_recover_work_struct {
		5298	struct work_struct base;
		5299	struct amdgpu_device *adev;
		5300	struct amdgpu_job *job;
		5301	int ret;
		5302	};
		5303
		5304	static void amdgpu_device_queue_gpu_recover_work(struct work_struct *work)
		5305	{
		5306	struct amdgpu_recover_work_struct *recover_work = container_of(work, struct amdgpu_recover_work_struct, base);
		5307
		5308	recover_work->ret = amdgpu_device_gpu_recover_imp(recover_work->adev, recover_work->job);
		5309	}
		5310	/*
		5311	* Serialize gpu recover into reset domain single threaded wq
		5312	*/
		5313	int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
		5314	struct amdgpu_job *job)
		5315	{
		5316	struct amdgpu_recover_work_struct work = {.adev = adev, .job = job};
		5317
		5318	INIT_WORK(&work.base, amdgpu_device_queue_gpu_recover_work);
		5319
		5320	if (!amdgpu_reset_domain_schedule(adev->reset_domain, &work.base))
		5321	return -EAGAIN;
		5322
		5323	flush_work(&work.base);
		5324
		5325	return work.ret;
		5326	}
		5327
5232	/**	5328	/**
5233	* amdgpu_device_get_pcie_info - fence pcie info about the PCIE slot	5329	* amdgpu_device_get_pcie_info - fence pcie info about the PCIE slot
5234	*	5330	*
@@ -5416,20 +5512,6 @@ int amdgpu_device_baco_exit(struct drm_device *dev) · 181 unmodified lines
5416	return 0;	5512	return 0;
5417	}	5513	}
5418		5514
5419	static void amdgpu_cancel_all_tdr(struct amdgpu_device *adev)
5420	{
5421	int i;
5422
5423	for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
5424	struct amdgpu_ring *ring = adev->rings[i];
5425
5426	if (!ring \|\| !ring->sched.thread)
5427	continue;
5428
5429	cancel_delayed_work_sync(&ring->sched.work_tdr);
5430	}
5431	}
5432
5433	/**	5515	/**
5434	* amdgpu_pci_error_detected - Called when a PCI error is detected.	5516	* amdgpu_pci_error_detected - Called when a PCI error is detected.
5435	* @pdev: PCI device struct	5517	* @pdev: PCI device struct
@@ -5460,14 +5542,11 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_sta · 24 unmodified lines
5460	/* Fatal error, prepare for slot reset */	5542	/* Fatal error, prepare for slot reset */
5461	case pci_channel_io_frozen:	5543	case pci_channel_io_frozen:
5462	/*	5544	/*
5463	* Cancel and wait for all TDRs in progress if failing to	5545	* Locking adev->reset_domain->sem will prevent any external access
5464	* set adev->in_gpu_reset in amdgpu_device_lock_adev
5465	*
5466	* Locking adev->reset_sem will prevent any external access
5467	* to GPU during PCI error recovery	5546	* to GPU during PCI error recovery
5468	*/	5547	*/
5469	while (!amdgpu_device_lock_adev(adev, NULL))	5548	amdgpu_device_lock_reset_domain(adev->reset_domain);
5470	amdgpu_cancel_all_tdr(adev);	5549	amdgpu_device_set_mp1_state(adev);
5471		5550
5472	/*	5551	/*
5473	* Block any work scheduling as we do for regular GPU reset	5552	* Block any work scheduling as we do for regular GPU reset
@@ -5574,7 +5653,8 @@ out: · 100 unmodified lines
5574	DRM_INFO("PCIe error recovery succeeded\n");	5653	DRM_INFO("PCIe error recovery succeeded\n");
5575	} else {	5654	} else {
5576	DRM_ERROR("PCIe error recovery failed, err:%d", r);	5655	DRM_ERROR("PCIe error recovery failed, err:%d", r);
5577	amdgpu_device_unlock_adev(adev);	5656	amdgpu_device_unset_mp1_state(adev);
		5657	amdgpu_device_unlock_reset_domain(adev->reset_domain);
5578	}	5658	}
5579		5659
5580	return r ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED;	5660	return r ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED;
@@ -5611,7 +5691,8 @@ void amdgpu_pci_resume(struct pci_dev *pdev) · 30 unmodified lines
5611	drm_sched_start(&ring->sched, true);	5691	drm_sched_start(&ring->sched, true);
5612	}	5692	}
5613		5693
5614	amdgpu_device_unlock_adev(adev);	5694	amdgpu_device_unset_mp1_state(adev);
		5695	amdgpu_device_unlock_reset_domain(adev->reset_domain);
5615	}	5696	}
5616		5697
5617	bool amdgpu_device_cache_pci_state(struct pci_dev *pdev)	5698	bool amdgpu_device_cache_pci_state(struct pci_dev *pdev)
@@ -5688,6 +5769,11 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, · 70 unmodified lines
5688	amdgpu_asic_invalidate_hdp(adev, ring);	5769	amdgpu_asic_invalidate_hdp(adev, ring);
5689	}	5770	}
5690		5771
		5772	int amdgpu_in_reset(struct amdgpu_device *adev)
		5773	{
		5774	return atomic_read(&adev->reset_domain->in_gpu_reset);
		5775	}
		5776
5691	/**	5777	/**
5692	* amdgpu_device_halt() - bring hardware to some kind of halt state	5778	* amdgpu_device_halt() - bring hardware to some kind of halt state
5693	*	5779	*
@@ -5726,3 +5812,36 @@ void amdgpu_device_halt(struct amdgpu_device *adev) · 32 unmodified lines
5726	pci_disable_device(pdev);	5812	pci_disable_device(pdev);
5727	pci_wait_for_pending_transaction(pdev);	5813	pci_wait_for_pending_transaction(pdev);
5728	}	5814	}
		5815
		5816	u32 amdgpu_device_pcie_port_rreg(struct amdgpu_device *adev,
		5817	u32 reg)
		5818	{
		5819	unsigned long flags, address, data;
		5820	u32 r;
		5821
		5822	address = adev->nbio.funcs->get_pcie_port_index_offset(adev);
		5823	data = adev->nbio.funcs->get_pcie_port_data_offset(adev);
		5824
		5825	spin_lock_irqsave(&adev->pcie_idx_lock, flags);
		5826	WREG32(address, reg * 4);
		5827	(void)RREG32(address);
		5828	r = RREG32(data);
		5829	spin_unlock_irqrestore(&adev->pcie_idx_lock, flags);
		5830	return r;
		5831	}
		5832
		5833	void amdgpu_device_pcie_port_wreg(struct amdgpu_device *adev,
		5834	u32 reg, u32 v)
		5835	{
		5836	unsigned long flags, address, data;
		5837
		5838	address = adev->nbio.funcs->get_pcie_port_index_offset(adev);
		5839	data = adev->nbio.funcs->get_pcie_port_data_offset(adev);
		5840
		5841	spin_lock_irqsave(&adev->pcie_idx_lock, flags);
		5842	WREG32(address, reg * 4);
		5843	(void)RREG32(address);
		5844	WREG32(data, v);
		5845	(void)RREG32(data);
		5846	spin_unlock_irqrestore(&adev->pcie_idx_lock, flags);
		5847	}

drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c+544−7

@@ -360,8 +360,11 @@ out: · 359 unmodified lines
360	return r;	360	return r;
361	}	361	}
362		362
		363	static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev);
		364
363	void amdgpu_discovery_fini(struct amdgpu_device *adev)	365	void amdgpu_discovery_fini(struct amdgpu_device *adev)
364	{	366	{
		367	amdgpu_discovery_sysfs_fini(adev);
365	kfree(adev->mman.discovery_bin);	368	kfree(adev->mman.discovery_bin);
366	adev->mman.discovery_bin = NULL;	369	adev->mman.discovery_bin = NULL;
367	}	370	}
@@ -382,6 +385,497 @@ static int amdgpu_discovery_validate_ip(const struct ip *ip) · 14 unmodified lines
382	return 0;	385	return 0;
383	}	386	}
384		387
		388	/* ================================================== */
		389
		390	struct ip_hw_instance {
		391	struct kobject kobj; /* ip_discovery/die/#die/#hw_id/#instance/<attrs...> */
		392
		393	int hw_id;
		394	u8 num_instance;
		395	u8 major, minor, revision;
		396	u8 harvest;
		397
		398	int num_base_addresses;
		399	u32 base_addr[];
		400	};
		401
		402	struct ip_hw_id {
		403	struct kset hw_id_kset; /* ip_discovery/die/#die/#hw_id/, contains ip_hw_instance */
		404	int hw_id;
		405	};
		406
		407	struct ip_die_entry {
		408	struct kset ip_kset; /* ip_discovery/die/#die/, contains ip_hw_id */
		409	u16 num_ips;
		410	};
		411
		412	/* -------------------------------------------------- */
		413
		414	struct ip_hw_instance_attr {
		415	struct attribute attr;
		416	ssize_t (show)(struct ip_hw_instance ip_hw_instance, char *buf);
		417	};
		418
		419	static ssize_t hw_id_show(struct ip_hw_instance ip_hw_instance, char buf)
		420	{
		421	return sysfs_emit(buf, "%d\n", ip_hw_instance->hw_id);
		422	}
		423
		424	static ssize_t num_instance_show(struct ip_hw_instance ip_hw_instance, char buf)
		425	{
		426	return sysfs_emit(buf, "%d\n", ip_hw_instance->num_instance);
		427	}
		428
		429	static ssize_t major_show(struct ip_hw_instance ip_hw_instance, char buf)
		430	{
		431	return sysfs_emit(buf, "%d\n", ip_hw_instance->major);
		432	}
		433
		434	static ssize_t minor_show(struct ip_hw_instance ip_hw_instance, char buf)
		435	{
		436	return sysfs_emit(buf, "%d\n", ip_hw_instance->minor);
		437	}
		438
		439	static ssize_t revision_show(struct ip_hw_instance ip_hw_instance, char buf)
		440	{
		441	return sysfs_emit(buf, "%d\n", ip_hw_instance->revision);
		442	}
		443
		444	static ssize_t harvest_show(struct ip_hw_instance ip_hw_instance, char buf)
		445	{
		446	return sysfs_emit(buf, "0x%01X\n", ip_hw_instance->harvest);
		447	}
		448
		449	static ssize_t num_base_addresses_show(struct ip_hw_instance ip_hw_instance, char buf)
		450	{
		451	return sysfs_emit(buf, "%d\n", ip_hw_instance->num_base_addresses);
		452	}
		453
		454	static ssize_t base_addr_show(struct ip_hw_instance ip_hw_instance, char buf)
		455	{
		456	ssize_t res, at;
		457	int ii;
		458
		459	for (res = at = ii = 0; ii < ip_hw_instance->num_base_addresses; ii++) {
		460	/* Here we satisfy the condition that, at + size <= PAGE_SIZE.
		461	*/
		462	if (at + 12 > PAGE_SIZE)
		463	break;
		464	res = sysfs_emit_at(buf, at, "0x%08X\n",
		465	ip_hw_instance->base_addr[ii]);
		466	if (res <= 0)
		467	break;
		468	at += res;
		469	}
		470
		471	return res < 0 ? res : at;
		472	}
		473
		474	static struct ip_hw_instance_attr ip_hw_attr[] = {
		475	__ATTR_RO(hw_id),
		476	__ATTR_RO(num_instance),
		477	__ATTR_RO(major),
		478	__ATTR_RO(minor),
		479	__ATTR_RO(revision),
		480	__ATTR_RO(harvest),
		481	__ATTR_RO(num_base_addresses),
		482	__ATTR_RO(base_addr),
		483	};
		484
		485	static struct attribute *ip_hw_instance_attrs[ARRAY_SIZE(ip_hw_attr) + 1];
		486	ATTRIBUTE_GROUPS(ip_hw_instance);
		487
		488	#define to_ip_hw_instance(x) container_of(x, struct ip_hw_instance, kobj)
		489	#define to_ip_hw_instance_attr(x) container_of(x, struct ip_hw_instance_attr, attr)
		490
		491	static ssize_t ip_hw_instance_attr_show(struct kobject *kobj,
		492	struct attribute *attr,
		493	char *buf)
		494	{
		495	struct ip_hw_instance *ip_hw_instance = to_ip_hw_instance(kobj);
		496	struct ip_hw_instance_attr *ip_hw_attr = to_ip_hw_instance_attr(attr);
		497
		498	if (!ip_hw_attr->show)
		499	return -EIO;
		500
		501	return ip_hw_attr->show(ip_hw_instance, buf);
		502	}
		503
		504	static const struct sysfs_ops ip_hw_instance_sysfs_ops = {
		505	.show = ip_hw_instance_attr_show,
		506	};
		507
		508	static void ip_hw_instance_release(struct kobject *kobj)
		509	{
		510	struct ip_hw_instance *ip_hw_instance = to_ip_hw_instance(kobj);
		511
		512	kfree(ip_hw_instance);
		513	}
		514
		515	static struct kobj_type ip_hw_instance_ktype = {
		516	.release = ip_hw_instance_release,
		517	.sysfs_ops = &ip_hw_instance_sysfs_ops,
		518	.default_groups = ip_hw_instance_groups,
		519	};
		520
		521	/* -------------------------------------------------- */
		522
		523	#define to_ip_hw_id(x) container_of(to_kset(x), struct ip_hw_id, hw_id_kset)
		524
		525	static void ip_hw_id_release(struct kobject *kobj)
		526	{
		527	struct ip_hw_id *ip_hw_id = to_ip_hw_id(kobj);
		528
		529	if (!list_empty(&ip_hw_id->hw_id_kset.list))
		530	DRM_ERROR("ip_hw_id->hw_id_kset is not empty");
		531	kfree(ip_hw_id);
		532	}
		533
		534	static struct kobj_type ip_hw_id_ktype = {
		535	.release = ip_hw_id_release,
		536	.sysfs_ops = &kobj_sysfs_ops,
		537	};
		538
		539	/* -------------------------------------------------- */
		540
		541	static void die_kobj_release(struct kobject *kobj);
		542	static void ip_disc_release(struct kobject *kobj);
		543
		544	struct ip_die_entry_attribute {
		545	struct attribute attr;
		546	ssize_t (show)(struct ip_die_entry ip_die_entry, char *buf);
		547	};
		548
		549	#define to_ip_die_entry_attr(x) container_of(x, struct ip_die_entry_attribute, attr)
		550
		551	static ssize_t num_ips_show(struct ip_die_entry ip_die_entry, char buf)
		552	{
		553	return sysfs_emit(buf, "%d\n", ip_die_entry->num_ips);
		554	}
		555
		556	/* If there are more ip_die_entry attrs, other than the number of IPs,
		557	* we can make this intro an array of attrs, and then initialize
		558	* ip_die_entry_attrs in a loop.
		559	*/
		560	static struct ip_die_entry_attribute num_ips_attr =
		561	__ATTR_RO(num_ips);
		562
		563	static struct attribute *ip_die_entry_attrs[] = {
		564	&num_ips_attr.attr,
		565	NULL,
		566	};
		567	ATTRIBUTE_GROUPS(ip_die_entry); /* ip_die_entry_groups */
		568
		569	#define to_ip_die_entry(x) container_of(to_kset(x), struct ip_die_entry, ip_kset)
		570
		571	static ssize_t ip_die_entry_attr_show(struct kobject *kobj,
		572	struct attribute *attr,
		573	char *buf)
		574	{
		575	struct ip_die_entry_attribute *ip_die_entry_attr = to_ip_die_entry_attr(attr);
		576	struct ip_die_entry *ip_die_entry = to_ip_die_entry(kobj);
		577
		578	if (!ip_die_entry_attr->show)
		579	return -EIO;
		580
		581	return ip_die_entry_attr->show(ip_die_entry, buf);
		582	}
		583
		584	static void ip_die_entry_release(struct kobject *kobj)
		585	{
		586	struct ip_die_entry *ip_die_entry = to_ip_die_entry(kobj);
		587
		588	if (!list_empty(&ip_die_entry->ip_kset.list))
		589	DRM_ERROR("ip_die_entry->ip_kset is not empty");
		590	kfree(ip_die_entry);
		591	}
		592
		593	static const struct sysfs_ops ip_die_entry_sysfs_ops = {
		594	.show = ip_die_entry_attr_show,
		595	};
		596
		597	static struct kobj_type ip_die_entry_ktype = {
		598	.release = ip_die_entry_release,
		599	.sysfs_ops = &ip_die_entry_sysfs_ops,
		600	.default_groups = ip_die_entry_groups,
		601	};
		602
		603	static struct kobj_type die_kobj_ktype = {
		604	.release = die_kobj_release,
		605	.sysfs_ops = &kobj_sysfs_ops,
		606	};
		607
		608	static struct kobj_type ip_discovery_ktype = {
		609	.release = ip_disc_release,
		610	.sysfs_ops = &kobj_sysfs_ops,
		611	};
		612
		613	struct ip_discovery_top {
		614	struct kobject kobj; /* ip_discovery/ */
		615	struct kset die_kset; /* ip_discovery/die/, contains ip_die_entry */
		616	struct amdgpu_device *adev;
		617	};
		618
		619	static void die_kobj_release(struct kobject *kobj)
		620	{
		621	struct ip_discovery_top *ip_top = container_of(to_kset(kobj),
		622	struct ip_discovery_top,
		623	die_kset);
		624	if (!list_empty(&ip_top->die_kset.list))
		625	DRM_ERROR("ip_top->die_kset is not empty");
		626	}
		627
		628	static void ip_disc_release(struct kobject *kobj)
		629	{
		630	struct ip_discovery_top *ip_top = container_of(kobj, struct ip_discovery_top,
		631	kobj);
		632	struct amdgpu_device *adev = ip_top->adev;
		633
		634	adev->ip_top = NULL;
		635	kfree(ip_top);
		636	}
		637
		638	static int amdgpu_discovery_sysfs_ips(struct amdgpu_device *adev,
		639	struct ip_die_entry *ip_die_entry,
		640	const size_t _ip_offset, const int num_ips)
		641	{
		642	int ii, jj, kk, res;
		643
		644	DRM_DEBUG("num_ips:%d", num_ips);
		645
		646	/* Find all IPs of a given HW ID, and add their instance to
		647	* #die/#hw_id/#instance/<attributes>
		648	*/
		649	for (ii = 0; ii < HW_ID_MAX; ii++) {
		650	struct ip_hw_id *ip_hw_id = NULL;
		651	size_t ip_offset = _ip_offset;
		652
		653	for (jj = 0; jj < num_ips; jj++) {
		654	struct ip *ip;
		655	struct ip_hw_instance *ip_hw_instance;
		656
		657	ip = (struct ip *)(adev->mman.discovery_bin + ip_offset);
		658	if (amdgpu_discovery_validate_ip(ip) \|\|
		659	le16_to_cpu(ip->hw_id) != ii)
		660	goto next_ip;
		661
		662	DRM_DEBUG("match:%d @ ip_offset:%zu", ii, ip_offset);
		663
		664	/* We have a hw_id match; register the hw
		665	* block if not yet registered.
		666	*/
		667	if (!ip_hw_id) {
		668	ip_hw_id = kzalloc(sizeof(*ip_hw_id), GFP_KERNEL);
		669	if (!ip_hw_id)
		670	return -ENOMEM;
		671	ip_hw_id->hw_id = ii;
		672
		673	kobject_set_name(&ip_hw_id->hw_id_kset.kobj, "%d", ii);
		674	ip_hw_id->hw_id_kset.kobj.kset = &ip_die_entry->ip_kset;
		675	ip_hw_id->hw_id_kset.kobj.ktype = &ip_hw_id_ktype;
		676	res = kset_register(&ip_hw_id->hw_id_kset);
		677	if (res) {
		678	DRM_ERROR("Couldn't register ip_hw_id kset");
		679	kfree(ip_hw_id);
		680	return res;
		681	}
		682	if (hw_id_names[ii]) {
		683	res = sysfs_create_link(&ip_die_entry->ip_kset.kobj,
		684	&ip_hw_id->hw_id_kset.kobj,
		685	hw_id_names[ii]);
		686	if (res) {
		687	DRM_ERROR("Couldn't create IP link %s in IP Die:%s\n",
		688	hw_id_names[ii],
		689	kobject_name(&ip_die_entry->ip_kset.kobj));
		690	}
		691	}
		692	}
		693
		694	/* Now register its instance.
		695	*/
		696	ip_hw_instance = kzalloc(struct_size(ip_hw_instance,
		697	base_addr,
		698	ip->num_base_address),
		699	GFP_KERNEL);
		700	if (!ip_hw_instance) {
		701	DRM_ERROR("no memory for ip_hw_instance");
		702	return -ENOMEM;
		703	}
		704	ip_hw_instance->hw_id = le16_to_cpu(ip->hw_id); /* == ii */
		705	ip_hw_instance->num_instance = ip->number_instance;
		706	ip_hw_instance->major = ip->major;
		707	ip_hw_instance->minor = ip->minor;
		708	ip_hw_instance->revision = ip->revision;
		709	ip_hw_instance->harvest = ip->harvest;
		710	ip_hw_instance->num_base_addresses = ip->num_base_address;
		711
		712	for (kk = 0; kk < ip_hw_instance->num_base_addresses; kk++)
		713	ip_hw_instance->base_addr[kk] = ip->base_address[kk];
		714
		715	kobject_init(&ip_hw_instance->kobj, &ip_hw_instance_ktype);
		716	ip_hw_instance->kobj.kset = &ip_hw_id->hw_id_kset;
		717	res = kobject_add(&ip_hw_instance->kobj, NULL,
		718	"%d", ip_hw_instance->num_instance);
		719	next_ip:
		720	ip_offset += sizeof(ip) + 4 (ip->num_base_address - 1);
		721	}
		722	}
		723
		724	return 0;
		725	}
		726
		727	static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev)
		728	{
		729	struct binary_header *bhdr;
		730	struct ip_discovery_header *ihdr;
		731	struct die_header *dhdr;
		732	struct kset *die_kset = &adev->ip_top->die_kset;
		733	u16 num_dies, die_offset, num_ips;
		734	size_t ip_offset;
		735	int ii, res;
		736
		737	bhdr = (struct binary_header *)adev->mman.discovery_bin;
		738	ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin +
		739	le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset));
		740	num_dies = le16_to_cpu(ihdr->num_dies);
		741
		742	DRM_DEBUG("number of dies: %d\n", num_dies);
		743
		744	for (ii = 0; ii < num_dies; ii++) {
		745	struct ip_die_entry *ip_die_entry;
		746
		747	die_offset = le16_to_cpu(ihdr->die_info[ii].die_offset);
		748	dhdr = (struct die_header *)(adev->mman.discovery_bin + die_offset);
		749	num_ips = le16_to_cpu(dhdr->num_ips);
		750	ip_offset = die_offset + sizeof(*dhdr);
		751
		752	/* Add the die to the kset.
		753	*
		754	* dhdr->die_id == ii, which was checked in
		755	* amdgpu_discovery_reg_base_init().
		756	*/
		757
		758	ip_die_entry = kzalloc(sizeof(*ip_die_entry), GFP_KERNEL);
		759	if (!ip_die_entry)
		760	return -ENOMEM;
		761
		762	ip_die_entry->num_ips = num_ips;
		763
		764	kobject_set_name(&ip_die_entry->ip_kset.kobj, "%d", le16_to_cpu(dhdr->die_id));
		765	ip_die_entry->ip_kset.kobj.kset = die_kset;
		766	ip_die_entry->ip_kset.kobj.ktype = &ip_die_entry_ktype;
		767	res = kset_register(&ip_die_entry->ip_kset);
		768	if (res) {
		769	DRM_ERROR("Couldn't register ip_die_entry kset");
		770	kfree(ip_die_entry);
		771	return res;
		772	}
		773
		774	amdgpu_discovery_sysfs_ips(adev, ip_die_entry, ip_offset, num_ips);
		775	}
		776
		777	return 0;
		778	}
		779
		780	static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)
		781	{
		782	struct kset *die_kset;
		783	int res, ii;
		784
		785	adev->ip_top = kzalloc(sizeof(*adev->ip_top), GFP_KERNEL);
		786	if (!adev->ip_top)
		787	return -ENOMEM;
		788
		789	adev->ip_top->adev = adev;
		790
		791	res = kobject_init_and_add(&adev->ip_top->kobj, &ip_discovery_ktype,
		792	&adev->dev->kobj, "ip_discovery");
		793	if (res) {
		794	DRM_ERROR("Couldn't init and add ip_discovery/");
		795	goto Err;
		796	}
		797
		798	die_kset = &adev->ip_top->die_kset;
		799	kobject_set_name(&die_kset->kobj, "%s", "die");
		800	die_kset->kobj.parent = &adev->ip_top->kobj;
		801	die_kset->kobj.ktype = &die_kobj_ktype;
		802	res = kset_register(&adev->ip_top->die_kset);
		803	if (res) {
		804	DRM_ERROR("Couldn't register die_kset");
		805	goto Err;
		806	}
		807
		808	for (ii = 0; ii < ARRAY_SIZE(ip_hw_attr); ii++)
		809	ip_hw_instance_attrs[ii] = &ip_hw_attr[ii].attr;
		810	ip_hw_instance_attrs[ii] = NULL;
		811
		812	res = amdgpu_discovery_sysfs_recurse(adev);
		813
		814	return res;
		815	Err:
		816	kobject_put(&adev->ip_top->kobj);
		817	return res;
		818	}
		819
		820	/* -------------------------------------------------- */
		821
		822	#define list_to_kobj(el) container_of(el, struct kobject, entry)
		823
		824	static void amdgpu_discovery_sysfs_ip_hw_free(struct ip_hw_id *ip_hw_id)
		825	{
		826	struct list_head el, tmp;
		827	struct kset *hw_id_kset;
		828
		829	hw_id_kset = &ip_hw_id->hw_id_kset;
		830	spin_lock(&hw_id_kset->list_lock);
		831	list_for_each_prev_safe(el, tmp, &hw_id_kset->list) {
		832	list_del_init(el);
		833	spin_unlock(&hw_id_kset->list_lock);
		834	/* kobject is embedded in ip_hw_instance */
		835	kobject_put(list_to_kobj(el));
		836	spin_lock(&hw_id_kset->list_lock);
		837	}
		838	spin_unlock(&hw_id_kset->list_lock);
		839	kobject_put(&ip_hw_id->hw_id_kset.kobj);
		840	}
		841
		842	static void amdgpu_discovery_sysfs_die_free(struct ip_die_entry *ip_die_entry)
		843	{
		844	struct list_head el, tmp;
		845	struct kset *ip_kset;
		846
		847	ip_kset = &ip_die_entry->ip_kset;
		848	spin_lock(&ip_kset->list_lock);
		849	list_for_each_prev_safe(el, tmp, &ip_kset->list) {
		850	list_del_init(el);
		851	spin_unlock(&ip_kset->list_lock);
		852	amdgpu_discovery_sysfs_ip_hw_free(to_ip_hw_id(list_to_kobj(el)));
		853	spin_lock(&ip_kset->list_lock);
		854	}
		855	spin_unlock(&ip_kset->list_lock);
		856	kobject_put(&ip_die_entry->ip_kset.kobj);
		857	}
		858
		859	static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev)
		860	{
		861	struct list_head el, tmp;
		862	struct kset *die_kset;
		863
		864	die_kset = &adev->ip_top->die_kset;
		865	spin_lock(&die_kset->list_lock);
		866	list_for_each_prev_safe(el, tmp, &die_kset->list) {
		867	list_del_init(el);
		868	spin_unlock(&die_kset->list_lock);
		869	amdgpu_discovery_sysfs_die_free(to_ip_die_entry(list_to_kobj(el)));
		870	spin_lock(&die_kset->list_lock);
		871	}
		872	spin_unlock(&die_kset->list_lock);
		873	kobject_put(&adev->ip_top->die_kset.kobj);
		874	kobject_put(&adev->ip_top->kobj);
		875	}
		876
		877	/* ================================================== */
		878
385	int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev)	879	int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev)
386	{	880	{
387	struct binary_header *bhdr;	881	struct binary_header *bhdr;
@@ -492,6 +986,8 @@ next_ip: · 104 unmodified lines
492	}	986	}
493	}	987	}
494		988
		989	amdgpu_discovery_sysfs_init(adev);
		990
495	return 0;	991	return 0;
496	}	992	}
497		993
@@ -674,12 +1170,15 @@ static int amdgpu_discovery_set_common_ip_blocks(struct amdgpu_device *adev) · 176 unmodified lines
674	case IP_VERSION(10, 1, 1):	1170	case IP_VERSION(10, 1, 1):
675	case IP_VERSION(10, 1, 2):	1171	case IP_VERSION(10, 1, 2):
676	case IP_VERSION(10, 1, 3):	1172	case IP_VERSION(10, 1, 3):
		1173	case IP_VERSION(10, 1, 4):
677	case IP_VERSION(10, 3, 0):	1174	case IP_VERSION(10, 3, 0):
678	case IP_VERSION(10, 3, 1):	1175	case IP_VERSION(10, 3, 1):
679	case IP_VERSION(10, 3, 2):	1176	case IP_VERSION(10, 3, 2):
680	case IP_VERSION(10, 3, 3):	1177	case IP_VERSION(10, 3, 3):
681	case IP_VERSION(10, 3, 4):	1178	case IP_VERSION(10, 3, 4):
682	case IP_VERSION(10, 3, 5):	1179	case IP_VERSION(10, 3, 5):
		1180	case IP_VERSION(10, 3, 6):
		1181	case IP_VERSION(10, 3, 7):
683	amdgpu_device_ip_block_add(adev, &nv_common_ip_block);	1182	amdgpu_device_ip_block_add(adev, &nv_common_ip_block);
684	break;	1183	break;
685	default:	1184	default:
@@ -709,12 +1208,15 @@ static int amdgpu_discovery_set_gmc_ip_blocks(struct amdgpu_device *adev) · 23 unmodified lines
709	case IP_VERSION(10, 1, 1):	1208	case IP_VERSION(10, 1, 1):
710	case IP_VERSION(10, 1, 2):	1209	case IP_VERSION(10, 1, 2):
711	case IP_VERSION(10, 1, 3):	1210	case IP_VERSION(10, 1, 3):
		1211	case IP_VERSION(10, 1, 4):
712	case IP_VERSION(10, 3, 0):	1212	case IP_VERSION(10, 3, 0):
713	case IP_VERSION(10, 3, 1):	1213	case IP_VERSION(10, 3, 1):
714	case IP_VERSION(10, 3, 2):	1214	case IP_VERSION(10, 3, 2):
715	case IP_VERSION(10, 3, 3):	1215	case IP_VERSION(10, 3, 3):
716	case IP_VERSION(10, 3, 4):	1216	case IP_VERSION(10, 3, 4):
717	case IP_VERSION(10, 3, 5):	1217	case IP_VERSION(10, 3, 5):
		1218	case IP_VERSION(10, 3, 6):
		1219	case IP_VERSION(10, 3, 7):
718	amdgpu_device_ip_block_add(adev, &gmc_v10_0_ip_block);	1220	amdgpu_device_ip_block_add(adev, &gmc_v10_0_ip_block);
719	break;	1221	break;
720	default:	1222	default:
@@ -790,6 +1292,7 @@ static int amdgpu_discovery_set_psp_ip_blocks(struct amdgpu_device *adev) · 69 unmodified lines
790	case IP_VERSION(13, 0, 1):	1292	case IP_VERSION(13, 0, 1):
791	case IP_VERSION(13, 0, 2):	1293	case IP_VERSION(13, 0, 2):
792	case IP_VERSION(13, 0, 3):	1294	case IP_VERSION(13, 0, 3):
		1295	case IP_VERSION(13, 0, 8):
793	amdgpu_device_ip_block_add(adev, &psp_v13_0_ip_block);	1296	amdgpu_device_ip_block_add(adev, &psp_v13_0_ip_block);
794	break;	1297	break;
795	default:	1298	default:
@@ -831,6 +1334,7 @@ static int amdgpu_discovery_set_smu_ip_blocks(struct amdgpu_device *adev) · 35 unmodified lines
831	case IP_VERSION(13, 0, 1):	1334	case IP_VERSION(13, 0, 1):
832	case IP_VERSION(13, 0, 2):	1335	case IP_VERSION(13, 0, 2):
833	case IP_VERSION(13, 0, 3):	1336	case IP_VERSION(13, 0, 3):
		1337	case IP_VERSION(13, 0, 8):
834	amdgpu_device_ip_block_add(adev, &smu_v13_0_ip_block);	1338	amdgpu_device_ip_block_add(adev, &smu_v13_0_ip_block);
835	break;	1339	break;
836	default:	1340	default:
@@ -846,8 +1350,14 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 9 unmodified lines
846	{	1350	{
847	if (adev->enable_virtual_display \|\| amdgpu_sriov_vf(adev)) {	1351	if (adev->enable_virtual_display \|\| amdgpu_sriov_vf(adev)) {
848	amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block);	1352	amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block);
		1353	return 0;
		1354	}
		1355
		1356	if (!amdgpu_device_has_dc_support(adev))
		1357	return 0;
		1358
849	#if defined(CONFIG_DRM_AMD_DC)	1359	#if defined(CONFIG_DRM_AMD_DC)
850	} else if (adev->ip_versions[DCE_HWIP][0]) {	1360	if (adev->ip_versions[DCE_HWIP][0]) {
851	switch (adev->ip_versions[DCE_HWIP][0]) {	1361	switch (adev->ip_versions[DCE_HWIP][0]) {
852	case IP_VERSION(1, 0, 0):	1362	case IP_VERSION(1, 0, 0):
853	case IP_VERSION(1, 0, 1):	1363	case IP_VERSION(1, 0, 1):
@@ -861,6 +1371,7 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 7 unmodified lines
861	case IP_VERSION(3, 0, 1):	1371	case IP_VERSION(3, 0, 1):
862	case IP_VERSION(3, 1, 2):	1372	case IP_VERSION(3, 1, 2):
863	case IP_VERSION(3, 1, 3):	1373	case IP_VERSION(3, 1, 3):
		1374	case IP_VERSION(3, 1, 6):
864	amdgpu_device_ip_block_add(adev, &dm_ip_block);	1375	amdgpu_device_ip_block_add(adev, &dm_ip_block);
865	break;	1376	break;
866	default:	1377	default:
@@ -882,8 +1393,8 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 15 unmodified lines
882	adev->ip_versions[DCI_HWIP][0]);	1393	adev->ip_versions[DCI_HWIP][0]);
883	return -EINVAL;	1394	return -EINVAL;
884	}	1395	}
885	#endif
886	}	1396	}
		1397	#endif
887	return 0;	1398	return 0;
888	}	1399	}
889		1400
@@ -904,12 +1415,15 @@ static int amdgpu_discovery_set_gc_ip_blocks(struct amdgpu_device *adev) · 14 unmodified lines
904	case IP_VERSION(10, 1, 2):	1415	case IP_VERSION(10, 1, 2):
905	case IP_VERSION(10, 1, 1):	1416	case IP_VERSION(10, 1, 1):
906	case IP_VERSION(10, 1, 3):	1417	case IP_VERSION(10, 1, 3):
		1418	case IP_VERSION(10, 1, 4):
907	case IP_VERSION(10, 3, 0):	1419	case IP_VERSION(10, 3, 0):
908	case IP_VERSION(10, 3, 2):	1420	case IP_VERSION(10, 3, 2):
909	case IP_VERSION(10, 3, 1):	1421	case IP_VERSION(10, 3, 1):
910	case IP_VERSION(10, 3, 4):	1422	case IP_VERSION(10, 3, 4):
911	case IP_VERSION(10, 3, 5):	1423	case IP_VERSION(10, 3, 5):
		1424	case IP_VERSION(10, 3, 6):
912	case IP_VERSION(10, 3, 3):	1425	case IP_VERSION(10, 3, 3):
		1426	case IP_VERSION(10, 3, 7):
913	amdgpu_device_ip_block_add(adev, &gfx_v10_0_ip_block);	1427	amdgpu_device_ip_block_add(adev, &gfx_v10_0_ip_block);
914	break;	1428	break;
915	default:	1429	default:
@@ -946,6 +1460,7 @@ static int amdgpu_discovery_set_sdma_ip_blocks(struct amdgpu_device *adev) · 30 unmodified lines
946	case IP_VERSION(5, 2, 5):	1460	case IP_VERSION(5, 2, 5):
947	case IP_VERSION(5, 2, 3):	1461	case IP_VERSION(5, 2, 3):
948	case IP_VERSION(5, 2, 1):	1462	case IP_VERSION(5, 2, 1):
		1463	case IP_VERSION(5, 2, 7):
949	amdgpu_device_ip_block_add(adev, &sdma_v5_2_ip_block);	1464	amdgpu_device_ip_block_add(adev, &sdma_v5_2_ip_block);
950	break;	1465	break;
951	default:	1466	default:
@@ -1038,12 +1553,14 @@ static int amdgpu_discovery_set_mes_ip_blocks(struct amdgpu_device *adev) · 86 unmodified lines
1038	case IP_VERSION(10, 1, 1):	1553	case IP_VERSION(10, 1, 1):
1039	case IP_VERSION(10, 1, 2):	1554	case IP_VERSION(10, 1, 2):
1040	case IP_VERSION(10, 1, 3):	1555	case IP_VERSION(10, 1, 3):
		1556	case IP_VERSION(10, 1, 4):
1041	case IP_VERSION(10, 3, 0):	1557	case IP_VERSION(10, 3, 0):
1042	case IP_VERSION(10, 3, 1):	1558	case IP_VERSION(10, 3, 1):
1043	case IP_VERSION(10, 3, 2):	1559	case IP_VERSION(10, 3, 2):
1044	case IP_VERSION(10, 3, 3):	1560	case IP_VERSION(10, 3, 3):
1045	case IP_VERSION(10, 3, 4):	1561	case IP_VERSION(10, 3, 4):
1046	case IP_VERSION(10, 3, 5):	1562	case IP_VERSION(10, 3, 5):
		1563	case IP_VERSION(10, 3, 6):
1047	amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block);	1564	amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block);
1048	break;	1565	break;
1049	default:	1566	default:
@@ -1217,11 +1734,6 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 167 unmodified lines
1217	return -EINVAL;	1734	return -EINVAL;
1218		1735
1219	amdgpu_discovery_harvest_ip(adev);	1736	amdgpu_discovery_harvest_ip(adev);
1220
1221	if (!adev->mman.discovery_bin) {
1222	DRM_ERROR("ip discovery uninitialized\n");
1223	return -EINVAL;
1224	}
1225	break;	1737	break;
1226	}	1738	}
1227		1739
@@ -1242,6 +1754,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 14 unmodified lines
1242	case IP_VERSION(10, 1, 1):	1754	case IP_VERSION(10, 1, 1):
1243	case IP_VERSION(10, 1, 2):	1755	case IP_VERSION(10, 1, 2):
1244	case IP_VERSION(10, 1, 3):	1756	case IP_VERSION(10, 1, 3):
		1757	case IP_VERSION(10, 1, 4):
1245	case IP_VERSION(10, 3, 0):	1758	case IP_VERSION(10, 3, 0):
1246	case IP_VERSION(10, 3, 2):	1759	case IP_VERSION(10, 3, 2):
1247	case IP_VERSION(10, 3, 4):	1760	case IP_VERSION(10, 3, 4):
@@ -1254,10 +1767,32 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 6 unmodified lines
1254	case IP_VERSION(10, 3, 3):	1767	case IP_VERSION(10, 3, 3):
1255	adev->family = AMDGPU_FAMILY_YC;	1768	adev->family = AMDGPU_FAMILY_YC;
1256	break;	1769	break;
		1770	case IP_VERSION(10, 3, 6):
		1771	adev->family = AMDGPU_FAMILY_GC_10_3_6;
		1772	break;
		1773	case IP_VERSION(10, 3, 7):
		1774	adev->family = AMDGPU_FAMILY_GC_10_3_7;
		1775	break;
1257	default:	1776	default:
1258	return -EINVAL;	1777	return -EINVAL;
1259	}	1778	}
1260		1779
		1780	switch (adev->ip_versions[GC_HWIP][0]) {
		1781	case IP_VERSION(9, 1, 0):
		1782	case IP_VERSION(9, 2, 2):
		1783	case IP_VERSION(9, 3, 0):
		1784	case IP_VERSION(10, 1, 3):
		1785	case IP_VERSION(10, 1, 4):
		1786	case IP_VERSION(10, 3, 1):
		1787	case IP_VERSION(10, 3, 3):
		1788	case IP_VERSION(10, 3, 6):
		1789	case IP_VERSION(10, 3, 7):
		1790	adev->flags \|= AMD_IS_APU;
		1791	break;
		1792	default:
		1793	break;
		1794	}
		1795
1261	if (adev->ip_versions[XGMI_HWIP][0] == IP_VERSION(4, 8, 0))	1796	if (adev->ip_versions[XGMI_HWIP][0] == IP_VERSION(4, 8, 0))
1262	adev->gmc.xgmi.supported = true;	1797	adev->gmc.xgmi.supported = true;
1263		1798
@@ -1286,6 +1821,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 22 unmodified lines
1286	case IP_VERSION(7, 2, 0):	1821	case IP_VERSION(7, 2, 0):
1287	case IP_VERSION(7, 2, 1):	1822	case IP_VERSION(7, 2, 1):
1288	case IP_VERSION(7, 5, 0):	1823	case IP_VERSION(7, 5, 0):
		1824	case IP_VERSION(7, 5, 1):
1289	adev->nbio.funcs = &nbio_v7_2_funcs;	1825	adev->nbio.funcs = &nbio_v7_2_funcs;
1290	adev->nbio.hdp_flush_reg = &nbio_v7_2_hdp_flush_reg;	1826	adev->nbio.hdp_flush_reg = &nbio_v7_2_hdp_flush_reg;
1291	break;	1827	break;
@@ -1368,6 +1904,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 76 unmodified lines
1368	case IP_VERSION(11, 0, 11):	1904	case IP_VERSION(11, 0, 11):
1369	case IP_VERSION(11, 5, 0):	1905	case IP_VERSION(11, 5, 0):
1370	case IP_VERSION(13, 0, 1):	1906	case IP_VERSION(13, 0, 1):
		1907	case IP_VERSION(13, 0, 9):
1371	adev->smuio.funcs = &smuio_v11_0_6_funcs;	1908	adev->smuio.funcs = &smuio_v11_0_6_funcs;
1372	break;	1909	break;
1373	case IP_VERSION(13, 0, 2):	1910	case IP_VERSION(13, 0, 2):

drivers/gpu/drm/amd/amdgpu/amdgpu_display.c+20−15

 		case CHIP_STONEY:
 			domain |= AMDGPU_GEM_DOMAIN_GTT;
 			break;
-		case CHIP_RAVEN:
-			/* enable S/G on PCO and RV2 */
-			if ((adev->apu_flags & AMD_APU_IS_RAVEN2) ||
-			    (adev->apu_flags & AMD_APU_IS_PICASSO))
-				domain |= AMDGPU_GEM_DOMAIN_GTT;
-			break;
-		case CHIP_RENOIR:
-		case CHIP_VANGOGH:
-		case CHIP_YELLOW_CARP:
-			domain |= AMDGPU_GEM_DOMAIN_GTT;
-			break;
 		default:
+			switch (adev->ip_versions[DCE_HWIP][0]) {
+			case IP_VERSION(1, 0, 0):
+			case IP_VERSION(1, 0, 1):
+				/* enable S/G on PCO and RV2 */
+				if ((adev->apu_flags & AMD_APU_IS_RAVEN2) ||
+				    (adev->apu_flags & AMD_APU_IS_PICASSO))
+					domain |= AMDGPU_GEM_DOMAIN_GTT;
+				break;
+			case IP_VERSION(2, 1, 0):
+			case IP_VERSION(3, 0, 1):
+			case IP_VERSION(3, 1, 2):
+			case IP_VERSION(3, 1, 3):
+				domain |= AMDGPU_GEM_DOMAIN_GTT;
+				break;
+			default:
+				break;
+			}
 			break;
 		}
 	}
 	int ret;
 	unsigned int i, block_width, block_height, block_size_log2;
-	if (!rfb->base.dev->mode_config.allow_fb_modifiers)
+	if (rfb->base.dev->mode_config.fb_modifiers_not_supported)
 		return 0;
 	for (i = 0; i < format_info->num_planes; ++i) {
 	if (ret)
 		return ret;
-	if (!dev->mode_config.allow_fb_modifiers) {
+	if (dev->mode_config.fb_modifiers_not_supported) {
 		drm_WARN_ONCE(dev, adev->family >= AMDGPU_FAMILY_AI,
 			      "GFX9+ requires FB check based on format modifier\n");
 		ret = check_tiling_flags_gfx6(rfb);
 			return ret;
 	}
-	if (dev->mode_config.allow_fb_modifiers &&
+	if (!dev->mode_config.fb_modifiers_not_supported &&
 	    !(rfb->base.flags & DRM_MODE_FB_MODIFIERS)) {
 		ret = convert_tiling_flags_to_modifier(rfb);
 		if (ret) {

drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c+87−34

@@ -99,9 +99,10 @@ · 98 unmodified lines
99	* - 3.42.0 - Add 16bpc fixed point display support	99	* - 3.42.0 - Add 16bpc fixed point display support
100	* - 3.43.0 - Add device hot plug/unplug support	100	* - 3.43.0 - Add device hot plug/unplug support
101	* - 3.44.0 - DCN3 supports DCC independent block settings: !64B && 128B, 64B && 128B	101	* - 3.44.0 - DCN3 supports DCC independent block settings: !64B && 128B, 64B && 128B
		102	* - 3.45.0 - Add context ioctl stable pstate interface
102	*/	103	*/
103	#define KMS_DRIVER_MAJOR 3	104	#define KMS_DRIVER_MAJOR 3
104	#define KMS_DRIVER_MINOR 44	105	#define KMS_DRIVER_MINOR 45
105	#define KMS_DRIVER_PATCHLEVEL 0	106	#define KMS_DRIVER_PATCHLEVEL 0
106		107
107	int amdgpu_vram_limit;	108	int amdgpu_vram_limit;
@@ -174,7 +175,6 @@ int amdgpu_mes; · 66 unmodified lines
174	int amdgpu_noretry = -1;	175	int amdgpu_noretry = -1;
175	int amdgpu_force_asic_type = -1;	176	int amdgpu_force_asic_type = -1;
176	int amdgpu_tmz = -1; /* auto */	177	int amdgpu_tmz = -1; /* auto */
177	uint amdgpu_freesync_vid_mode;
178	int amdgpu_reset_method = -1; /* auto */	178	int amdgpu_reset_method = -1; /* auto */
179	int amdgpu_num_kcq = -1;	179	int amdgpu_num_kcq = -1;
180	int amdgpu_smartshift_bias;	180	int amdgpu_smartshift_bias;
@@ -843,32 +843,6 @@ module_param_named(backlight, amdgpu_backlight, bint, 0444); · 662 unmodified lines
843	MODULE_PARM_DESC(tmz, "Enable TMZ feature (-1 = auto (default), 0 = off, 1 = on)");	843	MODULE_PARM_DESC(tmz, "Enable TMZ feature (-1 = auto (default), 0 = off, 1 = on)");
844	module_param_named(tmz, amdgpu_tmz, int, 0444);	844	module_param_named(tmz, amdgpu_tmz, int, 0444);
845		845
846	/**
847	* DOC: freesync_video (uint)
848	* Enable the optimization to adjust front porch timing to achieve seamless
849	* mode change experience when setting a freesync supported mode for which full
850	* modeset is not needed.
851	*
852	* The Display Core will add a set of modes derived from the base FreeSync
853	* video mode into the corresponding connector's mode list based on commonly
854	* used refresh rates and VRR range of the connected display, when users enable
855	* this feature. From the userspace perspective, they can see a seamless mode
856	* change experience when the change between different refresh rates under the
857	* same resolution. Additionally, userspace applications such as Video playback
858	* can read this modeset list and change the refresh rate based on the video
859	* frame rate. Finally, the userspace can also derive an appropriate mode for a
860	* particular refresh rate based on the FreeSync Mode and add it to the
861	* connector's mode list.
862	*
863	* Note: This is an experimental feature.
864	*
865	* The default value: 0 (off).
866	*/
867	MODULE_PARM_DESC(
868	freesync_video,
869	"Enable freesync modesetting optimization feature (0 = off (default), 1 = on)");
870	module_param_named(freesync_video, amdgpu_freesync_vid_mode, uint, 0444);
871
872	/**	846	/**
873	* DOC: reset_method (int)	847	* DOC: reset_method (int)
874	* GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco, 5 = pci)	848	* GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco, 5 = pci)
@@ -1942,13 +1916,14 @@ static const struct pci_device_id pciidlist[] = { · 1067 unmodified lines
1942	{0x1002, 0x73FF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_DIMGREY_CAVEFISH},	1916	{0x1002, 0x73FF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_DIMGREY_CAVEFISH},
1943		1917
1944	/* Aldebaran */	1918	/* Aldebaran */
1945	{0x1002, 0x7408, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN\|AMD_EXP_HW_SUPPORT},	1919	{0x1002, 0x7408, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN},
1946	{0x1002, 0x740C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN\|AMD_EXP_HW_SUPPORT},	1920	{0x1002, 0x740C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN},
1947	{0x1002, 0x740F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN\|AMD_EXP_HW_SUPPORT},	1921	{0x1002, 0x740F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN},
1948	{0x1002, 0x7410, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN\|AMD_EXP_HW_SUPPORT},	1922	{0x1002, 0x7410, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN},
1949		1923
1950	/* CYAN_SKILLFISH */	1924	/* CYAN_SKILLFISH */
1951	{0x1002, 0x13FE, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CYAN_SKILLFISH\|AMD_IS_APU},	1925	{0x1002, 0x13FE, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CYAN_SKILLFISH\|AMD_IS_APU},
		1926	{0x1002, 0x143F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CYAN_SKILLFISH\|AMD_IS_APU},
1952		1927
1953	/* BEIGE_GOBY */	1928	/* BEIGE_GOBY */
1954	{0x1002, 0x7420, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_BEIGE_GOBY},	1929	{0x1002, 0x7420, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_BEIGE_GOBY},
@@ -1994,6 +1969,28 @@ static bool amdgpu_is_fw_framebuffer(resource_size_t base, · 39 unmodified lines
1994	return found;	1969	return found;
1995	}	1970	}
1996		1971
		1972	static void amdgpu_get_secondary_funcs(struct amdgpu_device *adev)
		1973	{
		1974	struct pci_dev *p = NULL;
		1975	int i;
		1976
		1977	/* 0 - GPU
		1978	* 1 - audio
		1979	* 2 - USB
		1980	* 3 - UCSI
		1981	*/
		1982	for (i = 1; i < 4; i++) {
		1983	p = pci_get_domain_bus_and_slot(pci_domain_nr(adev->pdev->bus),
		1984	adev->pdev->bus->number, i);
		1985	if (p) {
		1986	pm_runtime_get_sync(&p->dev);
		1987	pm_runtime_mark_last_busy(&p->dev);
		1988	pm_runtime_put_autosuspend(&p->dev);
		1989	pci_dev_put(p);
		1990	}
		1991	}
		1992	}
		1993
1997	static int amdgpu_pci_probe(struct pci_dev *pdev,	1994	static int amdgpu_pci_probe(struct pci_dev *pdev,
1998	const struct pci_device_id *ent)	1995	const struct pci_device_id *ent)
1999	{	1996	{
@@ -2126,6 +2123,48 @@ retry_init: · 126 unmodified lines
2126	if (ret)	2123	if (ret)
2127	DRM_ERROR("Creating debugfs files failed (%d).\n", ret);	2124	DRM_ERROR("Creating debugfs files failed (%d).\n", ret);
2128		2125
		2126	if (adev->runpm) {
		2127	/* only need to skip on ATPX */
		2128	if (amdgpu_device_supports_px(ddev))
		2129	dev_pm_set_driver_flags(ddev->dev, DPM_FLAG_NO_DIRECT_COMPLETE);
		2130	/* we want direct complete for BOCO */
		2131	if (amdgpu_device_supports_boco(ddev))
		2132	dev_pm_set_driver_flags(ddev->dev, DPM_FLAG_SMART_PREPARE \|
		2133	DPM_FLAG_SMART_SUSPEND \|
		2134	DPM_FLAG_MAY_SKIP_RESUME);
		2135	pm_runtime_use_autosuspend(ddev->dev);
		2136	pm_runtime_set_autosuspend_delay(ddev->dev, 5000);
		2137
		2138	pm_runtime_allow(ddev->dev);
		2139
		2140	pm_runtime_mark_last_busy(ddev->dev);
		2141	pm_runtime_put_autosuspend(ddev->dev);
		2142
		2143	/*
		2144	* For runpm implemented via BACO, PMFW will handle the
		2145	* timing for BACO in and out:
		2146	* - put ASIC into BACO state only when both video and
		2147	* audio functions are in D3 state.
		2148	* - pull ASIC out of BACO state when either video or
		2149	* audio function is in D0 state.
		2150	* Also, at startup, PMFW assumes both functions are in
		2151	* D0 state.
		2152	*
		2153	* So if snd driver was loaded prior to amdgpu driver
		2154	* and audio function was put into D3 state, there will
		2155	* be no PMFW-aware D-state transition(D0->D3) on runpm
		2156	* suspend. Thus the BACO will be not correctly kicked in.
		2157	*
		2158	* Via amdgpu_get_secondary_funcs(), the audio dev is put
		2159	* into D0 state. Then there will be a PMFW-aware D-state
		2160	* transition(D0->D3) on runpm suspend.
		2161	*/
		2162	if (amdgpu_device_supports_baco(ddev) &&
		2163	!(adev->flags & AMD_IS_APU) &&
		2164	(adev->asic_type >= CHIP_NAVI10))
		2165	amdgpu_get_secondary_funcs(adev);
		2166	}
		2167
2129	return 0;	2168	return 0;
2130		2169
2131	err_pci:	2170	err_pci:
@@ -2137,8 +2176,15 @@ static void · 5 unmodified lines
2137	amdgpu_pci_remove(struct pci_dev *pdev)	2176	amdgpu_pci_remove(struct pci_dev *pdev)
2138	{	2177	{
2139	struct drm_device *dev = pci_get_drvdata(pdev);	2178	struct drm_device *dev = pci_get_drvdata(pdev);
		2179	struct amdgpu_device *adev = drm_to_adev(dev);
2140		2180
2141	drm_dev_unplug(dev);	2181	drm_dev_unplug(dev);
		2182
		2183	if (adev->runpm) {
		2184	pm_runtime_get_sync(dev->dev);
		2185	pm_runtime_forbid(dev->dev);
		2186	}
		2187
2142	amdgpu_driver_unload_kms(dev);	2188	amdgpu_driver_unload_kms(dev);
2143		2189
2144	/*	2190	/*
@@ -2246,13 +2292,20 @@ static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work) · 101 unmodified lines
2246	static int amdgpu_pmops_prepare(struct device *dev)	2292	static int amdgpu_pmops_prepare(struct device *dev)
2247	{	2293	{
2248	struct drm_device *drm_dev = dev_get_drvdata(dev);	2294	struct drm_device *drm_dev = dev_get_drvdata(dev);
		2295	struct amdgpu_device *adev = drm_to_adev(drm_dev);
2249		2296
2250	/* Return a positive number here so	2297	/* Return a positive number here so
2251	* DPM_FLAG_SMART_SUSPEND works properly	2298	* DPM_FLAG_SMART_SUSPEND works properly
2252	*/	2299	*/
2253	if (amdgpu_device_supports_boco(drm_dev))	2300	if (amdgpu_device_supports_boco(drm_dev))
2254	return pm_runtime_suspended(dev) &&	2301	return pm_runtime_suspended(dev);
2255	pm_suspend_via_firmware();	2302
		2303	/* if we will not support s3 or s2i for the device
		2304	* then skip suspend
		2305	*/
		2306	if (!amdgpu_acpi_is_s0ix_active(adev) &&
		2307	!amdgpu_acpi_is_s3_active(adev))
		2308	return 1;
2256		2309
2257	return 0;	2310	return 0;
2258	}	2311	}

drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c+5−38

@@ -446,24 +446,18 @@ int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring, · 445 unmodified lines
446	* for the requested ring.	446	* for the requested ring.
447	*	447	*
448	* @ring: ring to init the fence driver on	448	* @ring: ring to init the fence driver on
449	* @num_hw_submission: number of entries on the hardware queue
450	* @sched_score: optional score atomic shared with other schedulers
451	*	449	*
452	* Init the fence driver for the requested ring (all asics).	450	* Init the fence driver for the requested ring (all asics).
453	* Helper function for amdgpu_fence_driver_init().	451	* Helper function for amdgpu_fence_driver_init().
454	*/	452	*/
455	int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,	453	int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring)
456	unsigned num_hw_submission,
457	atomic_t *sched_score)
458	{	454	{
459	struct amdgpu_device *adev = ring->adev;	455	struct amdgpu_device *adev = ring->adev;
460	long timeout;
461	int r;
462		456
463	if (!adev)	457	if (!adev)
464	return -EINVAL;	458	return -EINVAL;
465		459
466	if (!is_power_of_2(num_hw_submission))	460	if (!is_power_of_2(ring->num_hw_submission))
467	return -EINVAL;	461	return -EINVAL;
468		462
469	ring->fence_drv.cpu_addr = NULL;	463	ring->fence_drv.cpu_addr = NULL;
@@ -474,41 +468,14 @@ int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, · 4 unmodified lines
474		468
475	timer_setup(&ring->fence_drv.fallback_timer, amdgpu_fence_fallback, 0);	469	timer_setup(&ring->fence_drv.fallback_timer, amdgpu_fence_fallback, 0);
476		470
477	ring->fence_drv.num_fences_mask = num_hw_submission * 2 - 1;	471	ring->fence_drv.num_fences_mask = ring->num_hw_submission * 2 - 1;
478	spin_lock_init(&ring->fence_drv.lock);	472	spin_lock_init(&ring->fence_drv.lock);
479	ring->fence_drv.fences = kcalloc(num_hw_submission * 2, sizeof(void *),	473	ring->fence_drv.fences = kcalloc(ring->num_hw_submission * 2, sizeof(void *),
480	GFP_KERNEL);	474	GFP_KERNEL);
		475
481	if (!ring->fence_drv.fences)	476	if (!ring->fence_drv.fences)
482	return -ENOMEM;	477	return -ENOMEM;
483		478
484	/* No need to setup the GPU scheduler for rings that don't need it */
485	if (ring->no_scheduler)
486	return 0;
487
488	switch (ring->funcs->type) {
489	case AMDGPU_RING_TYPE_GFX:
490	timeout = adev->gfx_timeout;
491	break;
492	case AMDGPU_RING_TYPE_COMPUTE:
493	timeout = adev->compute_timeout;
494	break;
495	case AMDGPU_RING_TYPE_SDMA:
496	timeout = adev->sdma_timeout;
497	break;
498	default:
499	timeout = adev->video_timeout;
500	break;
501	}
502
503	r = drm_sched_init(&ring->sched, &amdgpu_sched_ops,
504	num_hw_submission, amdgpu_job_hang_limit,
505	timeout, NULL, sched_score, ring->name);
506	if (r) {
507	DRM_ERROR("Failed to create scheduler on ring %s.\n",
508	ring->name);
509	return r;
510	}
511
512	return 0;	479	return 0;
513	}	480	}
514		481

drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.c+31−22

 #include "amdgpu_eeprom.h"
 #define FRU_EEPROM_MADDR        0x60000
-#define I2C_PRODUCT_INFO_OFFSET 0xC0
 static bool is_fru_eeprom_supported(struct amdgpu_device *adev)
 {
 	 */
 	struct atom_context *atom_ctx = adev->mode_info.atom_context;
-	/* VBIOS is of the format ###-DXXXYY-##. For SKU identification,
+	/* The i2c access is blocked on VF
+	 * TODO: Need other way to get the info
+	 */
+	if (amdgpu_sriov_vf(adev))
+		return false;
+	/* VBIOS is of the format ###-DXXXYYYY-##. For SKU identification,
 	 * we can use just the "DXXX" portion. If there were more models, we
 	 * could convert the 3 characters to a hex integer and use a switch
 	 * for ease/speed/readability. For now, 2 string comparisons are
 	case CHIP_ALDEBARAN:
 		/* All Aldebaran SKUs have the FRU */
 		return true;
+	case CHIP_SIENNA_CICHLID:
+		if (strnstr(atom_ctx->vbios_version, "D603",
+			    sizeof(atom_ctx->vbios_version)))
+			return true;
+		else
+			return false;
 	default:
 		return false;
 	}
 }
 static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr,
-				  unsigned char *buff)
+				  unsigned char *buf, size_t buf_size)
 {
-	int ret, size;
+	int ret;
+	u8 size;
-	ret = amdgpu_eeprom_read(&adev->pm.smu_i2c, addrptr, buff, 1);
+	ret = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addrptr, buf, 1);
 	if (ret < 1) {
 		DRM_WARN("FRU: Failed to get size field");
 		return ret;
 	/* The size returned by the i2c requires subtraction of 0xC0 since the
 	 * size apparently always reports as 0xC0+actual size.
 	 */
-	size = buff[0] - I2C_PRODUCT_INFO_OFFSET;
+	size = buf[0] & 0x3F;
+	size = min_t(size_t, size, buf_size);
-	ret = amdgpu_eeprom_read(&adev->pm.smu_i2c, addrptr + 1, buff, size);
+	ret = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addrptr + 1,
+				 buf, size);
 	if (ret < 1) {
 		DRM_WARN("FRU: Failed to get data field");
 		return ret;
 int amdgpu_fru_get_product_info(struct amdgpu_device *adev)
 {
-	unsigned char buff[AMDGPU_PRODUCT_NAME_LEN+2];
+	unsigned char buf[AMDGPU_PRODUCT_NAME_LEN];
 	u32 addrptr;
 	int size, len;
-	int offset = 2;
 	if (!is_fru_eeprom_supported(adev))
 		return 0;
-	if (adev->asic_type == CHIP_ALDEBARAN)
-		offset = 0;
 	/* If algo exists, it means that the i2c_adapter's initialized */
-	if (!adev->pm.smu_i2c.algo) {
+	if (!adev->pm.fru_eeprom_i2c_bus || !adev->pm.fru_eeprom_i2c_bus->algo) {
 		DRM_WARN("Cannot access FRU, EEPROM accessor not initialized");
 		return -ENODEV;
 	}
 	 * and the language field, so just start from 0xb, manufacturer size
 	 */
 	addrptr = FRU_EEPROM_MADDR + 0xb;
-	size = amdgpu_fru_read_eeprom(adev, addrptr, buff);
+	size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
 	if (size < 1) {
 		DRM_ERROR("Failed to read FRU Manufacturer, ret:%d", size);
 		return -EINVAL;
 	 * size field being 1 byte. This pattern continues below.
 	 */
 	addrptr += size + 1;
-	size = amdgpu_fru_read_eeprom(adev, addrptr, buff);
+	size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
 	if (size < 1) {
 		DRM_ERROR("Failed to read FRU product name, ret:%d", size);
 		return -EINVAL;
 				AMDGPU_PRODUCT_NAME_LEN);
 		len = AMDGPU_PRODUCT_NAME_LEN - 1;
 	}
-	/* Start at 2 due to buff using fields 0 and 1 for the address */
+	memcpy(adev->product_name, buf, len);
-	memcpy(adev->product_name, &buff[offset], len);
 	adev->product_name[len] = '\0';
 	addrptr += size + 1;
-	size = amdgpu_fru_read_eeprom(adev, addrptr, buff);
+	size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
 	if (size < 1) {
 		DRM_ERROR("Failed to read FRU product number, ret:%d", size);
 		return -EINVAL;
 		DRM_WARN("FRU Product Number is larger than 16 characters. This is likely a mistake");
 		len = sizeof(adev->product_number) - 1;
 	}
-	memcpy(adev->product_number, &buff[offset], len);
+	memcpy(adev->product_number, buf, len);
 	adev->product_number[len] = '\0';
 	addrptr += size + 1;
-	size = amdgpu_fru_read_eeprom(adev, addrptr, buff);
+	size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
 	if (size < 1) {
 		DRM_ERROR("Failed to read FRU product version, ret:%d", size);
 	}
 	addrptr += size + 1;
-	size = amdgpu_fru_read_eeprom(adev, addrptr, buff);
+	size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf));
 	if (size < 1) {
 		DRM_ERROR("Failed to read FRU serial number, ret:%d", size);
 		DRM_WARN("FRU Serial Number is larger than 16 characters. This is likely a mistake");
 		len = sizeof(adev->serial) - 1;
 	}
-	memcpy(adev->serial, &buff[offset], len);
+	memcpy(adev->serial, buf, len);
 	adev->serial[len] = '\0';
 	return 0;

drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c+12−24

  * replaces them with the dummy page (all asics).
  * Returns 0 for success, -EINVAL for failure.
  */
-int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
+void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
 			int pages)
 {
 	unsigned t;
 	uint64_t flags = 0;
 	int idx;
-	if (!adev->gart.ready) {
+	if (!adev->gart.ptr)
-		WARN(1, "trying to unbind memory from uninitialized GART !\n");
+		return;
-		return -EINVAL;
-	}
 	if (!drm_dev_enter(adev_to_drm(adev), &idx))
-		return 0;
+		return;
 	t = offset / AMDGPU_GPU_PAGE_SIZE;
 	p = t / AMDGPU_GPU_PAGES_IN_CPU_PAGE;
 		amdgpu_gmc_flush_gpu_tlb(adev, 0, i, 0);
 	drm_dev_exit(idx);
-	return 0;
 }
 /**
  * Map the dma_addresses into GART entries (all asics).
  * Returns 0 for success, -EINVAL for failure.
  */
-int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
+void amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
 		    int pages, dma_addr_t *dma_addr, uint64_t flags,
 		    void *dst)
 {
 	unsigned i, j, t;
 	int idx;
-	if (!adev->gart.ready) {
-		WARN(1, "trying to bind memory to uninitialized GART !\n");
-		return -EINVAL;
-	}
 	if (!drm_dev_enter(adev_to_drm(adev), &idx))
-		return 0;
+		return;
 	t = offset / AMDGPU_GPU_PAGE_SIZE;
 		}
 	}
 	drm_dev_exit(idx);
-	return 0;
 }
 /**
  * (all asics).
  * Returns 0 for success, -EINVAL for failure.
  */
-int amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset,
+void amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset,
 		     int pages, dma_addr_t *dma_addr,
 		     uint64_t flags)
 {
-	if (!adev->gart.ready) {
-		WARN(1, "trying to bind memory to uninitialized GART !\n");
-		return -EINVAL;
-	}
 	if (!adev->gart.ptr)
-		return 0;
+		return;
-	return amdgpu_gart_map(adev, offset, pages, dma_addr, flags,
+	amdgpu_gart_map(adev, offset, pages, dma_addr, flags, adev->gart.ptr);
-			       adev->gart.ptr);
 }
 /**
 {
 	int i;
+	if (!adev->gart.ptr)
+		return;
 	mb();
 	amdgpu_device_flush_hdp(adev, NULL);
 	for (i = 0; i < adev->num_vmhubs; i++)

drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h+7−8

 	unsigned			num_gpu_pages;
 	unsigned			num_cpu_pages;
 	unsigned			table_size;
-	bool				ready;
 	/* Asic default pte flags */
 	uint64_t			gart_pte_flags;
 void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
 int amdgpu_gart_init(struct amdgpu_device *adev);
 void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
-int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
+void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
-		       int pages);
+			int pages);
-int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
+void amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
-		    int pages, dma_addr_t *dma_addr, uint64_t flags,
+		     int pages, dma_addr_t *dma_addr, uint64_t flags,
-		    void *dst);
+		     void *dst);
-int amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset,
+void amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset,
-		     int pages, dma_addr_t *dma_addr, uint64_t flags);
+		      int pages, dma_addr_t *dma_addr, uint64_t flags);
 void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev);
 #endif

drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c+1−1

@@ -222,7 +222,7 @@ static void amdgpu_gem_object_close(struct drm_gem_object *obj, · 221 unmodified lines
222	if (!bo_va \|\| --bo_va->ref_count)	222	if (!bo_va \|\| --bo_va->ref_count)
223	goto out_unlock;	223	goto out_unlock;
224		224
225	amdgpu_vm_bo_rmv(adev, bo_va);	225	amdgpu_vm_bo_del(adev, bo_va);
226	if (!amdgpu_vm_ready(vm))	226	if (!amdgpu_vm_ready(vm))
227	goto out_unlock;	227	goto out_unlock;
228		228

drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c+11−59

 	mutex_lock(&adev->gfx.gfx_off_mutex);
-	r = smu_get_status_gfxoff(adev, value);
+	r = amdgpu_dpm_get_status_gfxoff(adev, value);
 	mutex_unlock(&adev->gfx.gfx_off_mutex);
 	return r;
 }
-int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev)
+int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block)
 {
 	int r;
-	struct ras_fs_if fs_info = {
+	r = amdgpu_ras_block_late_init(adev, ras_block);
-		.sysfs_name = "gfx_err_count",
-	};
-	struct ras_ih_if ih_info = {
-		.cb = amdgpu_gfx_process_ras_data_cb,
-	};
-	if (!adev->gfx.ras_if) {
-		adev->gfx.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
-		if (!adev->gfx.ras_if)
-			return -ENOMEM;
-		adev->gfx.ras_if->block = AMDGPU_RAS_BLOCK__GFX;
-		adev->gfx.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
-		adev->gfx.ras_if->sub_block_index = 0;
-	}
-	fs_info.head = ih_info.head = *adev->gfx.ras_if;
-	r = amdgpu_ras_late_init(adev, adev->gfx.ras_if,
-				 &fs_info, &ih_info);
 	if (r)
-		goto free;
+		return r;
-	if (amdgpu_ras_is_supported(adev, adev->gfx.ras_if->block)) {
+	if (amdgpu_ras_is_supported(adev, ras_block->block)) {
 		if (!amdgpu_persistent_edc_harvesting_supported(adev))
 			amdgpu_ras_reset_error_status(adev, AMDGPU_RAS_BLOCK__GFX);
 		r = amdgpu_irq_get(adev, &adev->gfx.cp_ecc_error_irq, 0);
 		if (r)
 			goto late_fini;
-	} else {
-		/* free gfx ras_if if ras is not supported */
-		r = 0;
-		goto free;
 	}
 	return 0;
 late_fini:
-	amdgpu_ras_late_fini(adev, adev->gfx.ras_if, &ih_info);
+	amdgpu_ras_block_late_fini(adev, ras_block);
-free:
-	kfree(adev->gfx.ras_if);
-	adev->gfx.ras_if = NULL;
 	return r;
 }
 void amdgpu_gfx_ras_fini(struct amdgpu_device *adev)
 {
 	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX) &&
-			adev->gfx.ras_if) {
+			adev->gfx.ras_if)
-		struct ras_common_if *ras_if = adev->gfx.ras_if;
+		amdgpu_ras_block_late_fini(adev, adev->gfx.ras_if);
-		struct ras_ih_if ih_info = {
-			.head = *ras_if,
-			.cb = amdgpu_gfx_process_ras_data_cb,
-		};
-		amdgpu_ras_late_fini(adev, ras_if, &ih_info);
-		kfree(ras_if);
-	}
 }
 int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev,
 	 */
 	if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) {
 		kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
-		if (adev->gfx.ras_funcs &&
+		if (adev->gfx.ras && adev->gfx.ras->ras_block.hw_ops &&
-		    adev->gfx.ras_funcs->query_ras_error_count)
+		    adev->gfx.ras->ras_block.hw_ops->query_ras_error_count)
-			adev->gfx.ras_funcs->query_ras_error_count(adev, err_data);
+			adev->gfx.ras->ras_block.hw_ops->query_ras_error_count(adev, err_data);
 		amdgpu_ras_reset_gpu(adev);
 	}
 	return AMDGPU_RAS_SUCCESS;
 	}
 	return amdgpu_num_kcq;
 }
-/* amdgpu_gfx_state_change_set - Handle gfx power state change set
- * @adev: amdgpu_device pointer
- * @state: gfx power state(1 -sGpuChangeState_D0Entry and 2 -sGpuChangeState_D3Entry)
- *
- */
-void amdgpu_gfx_state_change_set(struct amdgpu_device *adev, enum gfx_change_state state)
-{
-	mutex_lock(&adev->pm.mutex);
-	if (adev->powerplay.pp_funcs &&
-	    adev->powerplay.pp_funcs->gfx_state_change_set)
-		((adev)->powerplay.pp_funcs->gfx_state_change_set(
-			(adev)->powerplay.pp_handle, state));
-	mutex_unlock(&adev->pm.mutex);
-}

drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h+5−19

 #include "amdgpu_ring.h"
 #include "amdgpu_rlc.h"
 #include "soc15.h"
+#include "amdgpu_ras.h"
 /* GFX current status */
 #define AMDGPU_GFX_NORMAL_MODE			0x00000000L
 	AMDGPU_GFX_PIPE_PRIO_HIGH = AMDGPU_RING_PRIO_2
 };
-/* Argument for PPSMC_MSG_GpuChangeState */
-enum gfx_change_state {
-	sGpuChangeState_D0Entry = 1,
-	sGpuChangeState_D3Entry,
-};
 #define AMDGPU_GFX_QUEUE_PRIORITY_MINIMUM  0
 #define AMDGPU_GFX_QUEUE_PRIORITY_MAXIMUM  15
 	uint32_t bitmap[4][4];
 };
-struct amdgpu_gfx_ras_funcs {
+struct amdgpu_gfx_ras {
-	int (*ras_late_init)(struct amdgpu_device *adev);
+	struct amdgpu_ras_block_object  ras_block;
-	void (*ras_fini)(struct amdgpu_device *adev);
-	int (*ras_error_inject)(struct amdgpu_device *adev,
-				void *inject_if);
-	int (*query_ras_error_count)(struct amdgpu_device *adev,
-				     void *ras_error_status);
-	void (*reset_ras_error_count)(struct amdgpu_device *adev);
-	void (*query_ras_error_status)(struct amdgpu_device *adev);
-	void (*reset_ras_error_status)(struct amdgpu_device *adev);
 	void (*enable_watchdog_timer)(struct amdgpu_device *adev);
 };
 	/*ras */
 	struct ras_common_if			*ras_if;
-	const struct amdgpu_gfx_ras_funcs	*ras_funcs;
+	struct amdgpu_gfx_ras	*ras;
 };
 #define amdgpu_gfx_get_gpu_clock_counter(adev) (adev)->gfx.funcs->get_gpu_clock_counter((adev))
 				    int pipe, int queue);
 void amdgpu_gfx_off_ctrl(struct amdgpu_device *adev, bool enable);
 int amdgpu_get_gfx_off_status(struct amdgpu_device *adev, uint32_t *value);
-int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev);
+int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block);
 void amdgpu_gfx_ras_fini(struct amdgpu_device *adev);
 int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev,
 		void *err_data,
 uint32_t amdgpu_kiq_rreg(struct amdgpu_device *adev, uint32_t reg);
 void amdgpu_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v);
 int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev);
-void amdgpu_gfx_state_change_set(struct amdgpu_device *adev, enum gfx_change_state state);
 #endif

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c+63−64

@@ -436,82 +436,35 @@ void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr, · 435 unmodified lines
436	} while (fault->timestamp < tmp);	436	} while (fault->timestamp < tmp);
437	}	437	}
438		438
439	int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev)	439	int amdgpu_gmc_ras_early_init(struct amdgpu_device *adev)
440	{	440	{
441	int r;	441	if (!adev->gmc.xgmi.connected_to_cpu) {
442		442	adev->gmc.xgmi.ras = &xgmi_ras;
443	if (adev->umc.ras_funcs &&	443	amdgpu_ras_register_ras_block(adev, &adev->gmc.xgmi.ras->ras_block);
444	adev->umc.ras_funcs->ras_late_init) {	444	adev->gmc.xgmi.ras_if = &adev->gmc.xgmi.ras->ras_block.ras_comm;
445	r = adev->umc.ras_funcs->ras_late_init(adev);
446	if (r)
447	return r;
448	}
449
450	if (adev->mmhub.ras_funcs &&
451	adev->mmhub.ras_funcs->ras_late_init) {
452	r = adev->mmhub.ras_funcs->ras_late_init(adev);
453	if (r)
454	return r;
455	}
456
457	if (!adev->gmc.xgmi.connected_to_cpu)
458	adev->gmc.xgmi.ras_funcs = &xgmi_ras_funcs;
459
460	if (adev->gmc.xgmi.ras_funcs &&
461	adev->gmc.xgmi.ras_funcs->ras_late_init) {
462	r = adev->gmc.xgmi.ras_funcs->ras_late_init(adev);
463	if (r)
464	return r;
465	}
466
467	if (adev->hdp.ras_funcs &&
468	adev->hdp.ras_funcs->ras_late_init) {
469	r = adev->hdp.ras_funcs->ras_late_init(adev);
470	if (r)
471	return r;
472	}
473
474	if (adev->mca.mp0.ras_funcs &&
475	adev->mca.mp0.ras_funcs->ras_late_init) {
476	r = adev->mca.mp0.ras_funcs->ras_late_init(adev);
477	if (r)
478	return r;
479	}
480
481	if (adev->mca.mp1.ras_funcs &&
482	adev->mca.mp1.ras_funcs->ras_late_init) {
483	r = adev->mca.mp1.ras_funcs->ras_late_init(adev);
484	if (r)
485	return r;
486	}	445	}
487		446
488	if (adev->mca.mpio.ras_funcs &&	447	return 0;
489	adev->mca.mpio.ras_funcs->ras_late_init) {	448	}
490	r = adev->mca.mpio.ras_funcs->ras_late_init(adev);
491	if (r)
492	return r;
493	}
494		449
		450	int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev)
		451	{
495	return 0;	452	return 0;
496	}	453	}
497		454
498	void amdgpu_gmc_ras_fini(struct amdgpu_device *adev)	455	void amdgpu_gmc_ras_fini(struct amdgpu_device *adev)
499	{	456	{
500	if (adev->umc.ras_funcs &&	457	if (adev->umc.ras && adev->umc.ras->ras_block.ras_fini)
501	adev->umc.ras_funcs->ras_fini)	458	adev->umc.ras->ras_block.ras_fini(adev);
502	adev->umc.ras_funcs->ras_fini(adev);
503		459
504	if (adev->mmhub.ras_funcs &&	460	if (adev->mmhub.ras && adev->mmhub.ras->ras_block.ras_fini)
505	adev->mmhub.ras_funcs->ras_fini)	461	adev->mmhub.ras->ras_block.ras_fini(adev);
506	adev->mmhub.ras_funcs->ras_fini(adev);
507		462
508	if (adev->gmc.xgmi.ras_funcs &&	463	if (adev->gmc.xgmi.ras && adev->gmc.xgmi.ras->ras_block.ras_fini)
509	adev->gmc.xgmi.ras_funcs->ras_fini)	464	adev->gmc.xgmi.ras->ras_block.ras_fini(adev);
510	adev->gmc.xgmi.ras_funcs->ras_fini(adev);
511		465
512	if (adev->hdp.ras_funcs &&	466	if (adev->hdp.ras && adev->hdp.ras->ras_block.ras_fini)
513	adev->hdp.ras_funcs->ras_fini)	467	adev->hdp.ras->ras_block.ras_fini(adev);
514	adev->hdp.ras_funcs->ras_fini(adev);
515	}	468	}
516		469
517	/*	470	/*
@@ -831,3 +784,49 @@ void amdgpu_gmc_get_reserved_allocation(struct amdgpu_device *adev) · 313 unmodified lines
831	break;	784	break;
832	}	785	}
833	}	786	}
		787
		788	int amdgpu_gmc_vram_checking(struct amdgpu_device *adev)
		789	{
		790	struct amdgpu_bo *vram_bo = NULL;
		791	uint64_t vram_gpu = 0;
		792	void *vram_ptr = NULL;
		793
		794	int ret, size = 0x100000;
		795	uint8_t cptr[10];
		796
		797	ret = amdgpu_bo_create_kernel(adev, size, PAGE_SIZE,
		798	AMDGPU_GEM_DOMAIN_VRAM,
		799	&vram_bo,
		800	&vram_gpu,
		801	&vram_ptr);
		802	if (ret)
		803	return ret;
		804
		805	memset(vram_ptr, 0x86, size);
		806	memset(cptr, 0x86, 10);
		807
		808	/**
		809	* Check the start, the mid, and the end of the memory if the content of
		810	* each byte is the pattern "0x86". If yes, we suppose the vram bo is
		811	* workable.
		812	*
		813	* Note: If check the each byte of whole 1M bo, it will cost too many
		814	* seconds, so here, we just pick up three parts for emulation.
		815	*/
		816	ret = memcmp(vram_ptr, cptr, 10);
		817	if (ret)
		818	return ret;
		819
		820	ret = memcmp(vram_ptr + (size / 2), cptr, 10);
		821	if (ret)
		822	return ret;
		823
		824	ret = memcmp(vram_ptr + size - 10, cptr, 10);
		825	if (ret)
		826	return ret;
		827
		828	amdgpu_bo_free_kernel(&vram_bo, &vram_gpu,
		829	&vram_ptr);
		830
		831	return 0;
		832	}

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h+6−7

 #include <linux/types.h>
 #include "amdgpu_irq.h"
+#include "amdgpu_ras.h"
 /* VA hole for 48bit addresses on Vega10 */
 #define AMDGPU_GMC_HOLE_START	0x0000800000000000ULL
 	unsigned int (*get_vbios_fb_size)(struct amdgpu_device *adev);
 };
-struct amdgpu_xgmi_ras_funcs {
+struct amdgpu_xgmi_ras {
-	int (*ras_late_init)(struct amdgpu_device *adev);
+	struct amdgpu_ras_block_object ras_block;
-	void (*ras_fini)(struct amdgpu_device *adev);
-	int (*query_ras_error_count)(struct amdgpu_device *adev,
-				     void *ras_error_status);
-	void (*reset_ras_error_count)(struct amdgpu_device *adev);
 };
 struct amdgpu_xgmi {
 	struct ras_common_if *ras_if;
 	bool connected_to_cpu;
 	bool pending_reset;
-	const struct amdgpu_xgmi_ras_funcs *ras_funcs;
+	struct amdgpu_xgmi_ras *ras;
 };
 struct amdgpu_gmc {
 			      uint16_t pasid, uint64_t timestamp);
 void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr,
 				     uint16_t pasid);
+int amdgpu_gmc_ras_early_init(struct amdgpu_device *adev);
 int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev);
 void amdgpu_gmc_ras_fini(struct amdgpu_device *adev);
 int amdgpu_gmc_allocate_vm_inv_eng(struct amdgpu_device *adev);
 uint64_t amdgpu_gmc_vram_mc2pa(struct amdgpu_device *adev, uint64_t mc_addr);
 uint64_t amdgpu_gmc_vram_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo);
 uint64_t amdgpu_gmc_vram_cpu_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo);
+int amdgpu_gmc_vram_checking(struct amdgpu_device *adev);
 #endif

drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c+13−45

 	struct ttm_resource_manager *man;
 	man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT);
-	return sysfs_emit(buf, "%llu\n", man->size * PAGE_SIZE);
+	return sysfs_emit(buf, "%llu\n", man->size);
 }
 /**
 {
 	struct drm_device *ddev = dev_get_drvdata(dev);
 	struct amdgpu_device *adev = drm_to_adev(ddev);
+	struct ttm_resource_manager *man = &adev->mman.gtt_mgr.manager;
-	return sysfs_emit(buf, "%llu\n", amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr));
+	return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man));
 }
 static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO,
 	struct amdgpu_gtt_node *node;
 	int r;
-	if (!(place->flags & TTM_PL_FLAG_TEMPORARY) &&
-	    atomic64_add_return(num_pages, &mgr->used) >  man->size) {
-		atomic64_sub(num_pages, &mgr->used);
-		return -ENOSPC;
-	}
 	node = kzalloc(struct_size(node, base.mm_nodes, 1), GFP_KERNEL);
-	if (!node) {
+	if (!node)
-		r = -ENOMEM;
+		return -ENOMEM;
-		goto err_out;
-	}
 	node->tbo = tbo;
 	ttm_resource_init(tbo, place, &node->base.base);
+	if (!(place->flags & TTM_PL_FLAG_TEMPORARY) &&
+	    ttm_resource_manager_usage(man) > man->size) {
+		r = -ENOSPC;
+		goto err_free;
+	}
 	if (place->lpfn) {
 		spin_lock(&mgr->lock);
 err_free:
 	ttm_resource_fini(man, &node->base.base);
 	kfree(node);
-err_out:
-	if (!(place->flags & TTM_PL_FLAG_TEMPORARY))
-		atomic64_sub(num_pages, &mgr->used);
 	return r;
 }
 		drm_mm_remove_node(&node->base.mm_nodes[0]);
 	spin_unlock(&mgr->lock);
-	if (!(res->placement & TTM_PL_FLAG_TEMPORARY))
-		atomic64_sub(res->num_pages, &mgr->used);
 	ttm_resource_fini(man, res);
 	kfree(node);
 }
-/**
- * amdgpu_gtt_mgr_usage - return usage of GTT domain
- *
- * @mgr: amdgpu_gtt_mgr pointer
- *
- * Return how many bytes are used in the GTT domain
- */
-uint64_t amdgpu_gtt_mgr_usage(struct amdgpu_gtt_mgr *mgr)
-{
-	return atomic64_read(&mgr->used) * PAGE_SIZE;
-}
 /**
  * amdgpu_gtt_mgr_recover - re-init gart
  *
  *
  * Re-init the gart for each known BO in the GTT.
  */
-int amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr)
+void amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr)
 {
 	struct amdgpu_gtt_node *node;
 	struct drm_mm_node *mm_node;
 	struct amdgpu_device *adev;
-	int r = 0;
 	adev = container_of(mgr, typeof(*adev), mman.gtt_mgr);
 	spin_lock(&mgr->lock);
 	drm_mm_for_each_node(mm_node, &mgr->mm) {
 		node = container_of(mm_node, typeof(*node), base.mm_nodes[0]);
-		r = amdgpu_ttm_recover_gart(node->tbo);
+		amdgpu_ttm_recover_gart(node->tbo);
-		if (r)
-			break;
 	}
 	spin_unlock(&mgr->lock);
 	amdgpu_gart_invalidate_tlb(adev);
-	return r;
 }
 /**
 	spin_lock(&mgr->lock);
 	drm_mm_print(&mgr->mm, printer);
 	spin_unlock(&mgr->lock);
-	drm_printf(printer, "man size:%llu pages,  gtt used:%llu pages\n",
-		   man->size, atomic64_read(&mgr->used));
 }
 static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func = {
 	man->use_tt = true;
 	man->func = &amdgpu_gtt_mgr_func;
-	ttm_resource_manager_init(man, &adev->mman.bdev,
+	ttm_resource_manager_init(man, &adev->mman.bdev, gtt_size);
-				  gtt_size >> PAGE_SHIFT);
 	start = AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS;
 	size = (adev->gmc.gart_size >> PAGE_SHIFT) - start;
 	drm_mm_init(&mgr->mm, start, size);
 	spin_lock_init(&mgr->lock);
-	atomic64_set(&mgr->used, 0);
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager);
 	ttm_resource_manager_set_used(man, true);

drivers/gpu/drm/amd/amdgpu/amdgpu_hdp.c+2−38

@@ -24,45 +24,9 @@ · 23 unmodified lines
24	#include "amdgpu.h"	24	#include "amdgpu.h"
25	#include "amdgpu_ras.h"	25	#include "amdgpu_ras.h"
26		26
27	int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev)
28	{
29	int r;
30	struct ras_ih_if ih_info = {
31	.cb = NULL,
32	};
33	struct ras_fs_if fs_info = {
34	.sysfs_name = "hdp_err_count",
35	};
36
37	if (!adev->hdp.ras_if) {
38	adev->hdp.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
39	if (!adev->hdp.ras_if)
40	return -ENOMEM;
41	adev->hdp.ras_if->block = AMDGPU_RAS_BLOCK__HDP;
42	adev->hdp.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
43	adev->hdp.ras_if->sub_block_index = 0;
44	}
45	ih_info.head = fs_info.head = *adev->hdp.ras_if;
46	r = amdgpu_ras_late_init(adev, adev->hdp.ras_if,
47	&fs_info, &ih_info);
48	if (r \|\| !amdgpu_ras_is_supported(adev, adev->hdp.ras_if->block)) {
49	kfree(adev->hdp.ras_if);
50	adev->hdp.ras_if = NULL;
51	}
52
53	return r;
54	}
55
56	void amdgpu_hdp_ras_fini(struct amdgpu_device *adev)	27	void amdgpu_hdp_ras_fini(struct amdgpu_device *adev)
57	{	28	{
58	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__HDP) &&	29	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__HDP) &&
59	adev->hdp.ras_if) {	30	adev->hdp.ras_if)
60	struct ras_common_if *ras_if = adev->hdp.ras_if;	31	amdgpu_ras_block_late_fini(adev, adev->hdp.ras_if);
61	struct ras_ih_if ih_info = {
62	.cb = NULL,
63	};
64
65	amdgpu_ras_late_fini(adev, ras_if, &ih_info);
66	kfree(ras_if);
67	}
68	}	32	}

drivers/gpu/drm/amd/amdgpu/amdgpu_hdp.h+5−8

  */
 #ifndef __AMDGPU_HDP_H__
 #define __AMDGPU_HDP_H__
+#include "amdgpu_ras.h"
-struct amdgpu_hdp_ras_funcs {
+struct amdgpu_hdp_ras {
-	int (*ras_late_init)(struct amdgpu_device *adev);
+	struct amdgpu_ras_block_object ras_block;
-	void (*ras_fini)(struct amdgpu_device *adev);
-	void (*query_ras_error_count)(struct amdgpu_device *adev,
-				      void *ras_error_status);
-	void (*reset_ras_error_count)(struct amdgpu_device *adev);
 };
 struct amdgpu_hdp_funcs {
 struct amdgpu_hdp {
 	struct ras_common_if			*ras_if;
 	const struct amdgpu_hdp_funcs		*funcs;
-	const struct amdgpu_hdp_ras_funcs	*ras_funcs;
+	struct amdgpu_hdp_ras	*ras;
 };
-int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev);
+int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block);
 void amdgpu_hdp_ras_fini(struct amdgpu_device *adev);
 #endif /* __AMDGPU_HDP_H__ */

drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c+6−6

@@ -199,13 +199,13 @@ static irqreturn_t amdgpu_irq_handler(int irq, void *arg) · 198 unmodified lines
199	* ack the interrupt if it is there	199	* ack the interrupt if it is there
200	*/	200	*/
201	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF)) {	201	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF)) {
202	if (adev->nbio.ras_funcs &&	202	if (adev->nbio.ras &&
203	adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring)	203	adev->nbio.ras->handle_ras_controller_intr_no_bifring)
204	adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring(adev);	204	adev->nbio.ras->handle_ras_controller_intr_no_bifring(adev);
205		205
206	if (adev->nbio.ras_funcs &&	206	if (adev->nbio.ras &&
207	adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring)	207	adev->nbio.ras->handle_ras_err_event_athub_intr_no_bifring)
208	adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring(adev);	208	adev->nbio.ras->handle_ras_err_event_athub_intr_no_bifring(adev);
209	}	209	}
210		210
211	return ret;	211	return ret;

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c+4−1

 	struct amdgpu_task_info ti;
 	struct amdgpu_device *adev = ring->adev;
 	int idx;
+	int r;
 	if (!drm_dev_enter(adev_to_drm(adev), &idx)) {
 		DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s",
 		  ti.process_name, ti.tgid, ti.task_name, ti.pid);
 	if (amdgpu_device_should_recover_gpu(ring->adev)) {
-		amdgpu_device_gpu_recover(ring->adev, job);
+		r = amdgpu_device_gpu_recover_imp(ring->adev, job);
+		if (r)
+			DRM_ERROR("GPU Recovery Failed: %d\n", r);
 	} else {
 		drm_sched_suspend_timeout(&ring->sched);
 		if (amdgpu_sriov_vf(adev))

drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c+33−93

 	if (adev->rmmio == NULL)
 		return;
-	if (adev->runpm) {
-		pm_runtime_get_sync(dev->dev);
-		pm_runtime_forbid(dev->dev);
-	}
 	if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DRV_UNLOAD))
 		DRM_WARN("smart shift update failed\n");
 	mutex_unlock(&mgpu_info.mutex);
 }
-static void amdgpu_get_audio_func(struct amdgpu_device *adev)
-{
-	struct pci_dev *p = NULL;
-	p = pci_get_domain_bus_and_slot(pci_domain_nr(adev->pdev->bus),
-			adev->pdev->bus->number, 1);
-	if (p) {
-		pm_runtime_get_sync(&p->dev);
-		pm_runtime_mark_last_busy(&p->dev);
-		pm_runtime_put_autosuspend(&p->dev);
-		pci_dev_put(p);
-	}
-}
 /**
  * amdgpu_driver_load_kms - Main load function for KMS.
  *
 int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags)
 {
 	struct drm_device *dev;
-	struct pci_dev *parent;
 	int r, acpi_status;
 	dev = adev_to_drm(adev);
-	if (amdgpu_has_atpx() &&
-	    (amdgpu_is_atpx_hybrid() ||
-	     amdgpu_has_atpx_dgpu_power_cntl()) &&
-	    ((flags & AMD_IS_APU) == 0) &&
-	    !pci_is_thunderbolt_attached(to_pci_dev(dev->dev)))
-		flags |= AMD_IS_PX;
-	parent = pci_upstream_bridge(adev->pdev);
-	adev->has_pr3 = parent ? pci_pr3_present(parent) : false;
 	/* amdgpu_device_init should report only fatal error
 	 * like memory allocation failure or iomapping failure,
 	 * or memory manager initialization failure, it must
 	if (acpi_status)
 		dev_dbg(dev->dev, "Error during ACPI methods call\n");
-	if (adev->runpm) {
-		/* only need to skip on ATPX */
-		if (amdgpu_device_supports_px(dev))
-			dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NO_DIRECT_COMPLETE);
-		/* we want direct complete for BOCO */
-		if (amdgpu_device_supports_boco(dev))
-			dev_pm_set_driver_flags(dev->dev, DPM_FLAG_SMART_PREPARE |
-						DPM_FLAG_SMART_SUSPEND |
-						DPM_FLAG_MAY_SKIP_RESUME);
-		pm_runtime_use_autosuspend(dev->dev);
-		pm_runtime_set_autosuspend_delay(dev->dev, 5000);
-		pm_runtime_allow(dev->dev);
-		pm_runtime_mark_last_busy(dev->dev);
-		pm_runtime_put_autosuspend(dev->dev);
-		/*
-		 * For runpm implemented via BACO, PMFW will handle the
-		 * timing for BACO in and out:
-		 *   - put ASIC into BACO state only when both video and
-		 *     audio functions are in D3 state.
-		 *   - pull ASIC out of BACO state when either video or
-		 *     audio function is in D0 state.
-		 * Also, at startup, PMFW assumes both functions are in
-		 * D0 state.
-		 *
-		 * So if snd driver was loaded prior to amdgpu driver
-		 * and audio function was put into D3 state, there will
-		 * be no PMFW-aware D-state transition(D0->D3) on runpm
-		 * suspend. Thus the BACO will be not correctly kicked in.
-		 *
-		 * Via amdgpu_get_audio_func(), the audio dev is put
-		 * into D0 state. Then there will be a PMFW-aware D-state
-		 * transition(D0->D3) on runpm suspend.
-		 */
-		if (amdgpu_device_supports_baco(dev) &&
-		    !(adev->flags & AMD_IS_APU) &&
-		    (adev->asic_type >= CHIP_NAVI10))
-			amdgpu_get_audio_func(adev);
-	}
 	if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DRV_LOAD))
 		DRM_WARN("smart shift update failed\n");
 out:
-	if (r) {
+	if (r)
-		/* balance pm_runtime_get_sync in amdgpu_driver_unload_kms */
-		if (adev->rmmio && adev->runpm)
-			pm_runtime_put_noidle(dev->dev);
 		amdgpu_driver_unload_kms(dev);
-	}
 	return r;
 }
 		fw_info->ver = adev->psp.toc.fw_version;
 		fw_info->feature = adev->psp.toc.feature_version;
 		break;
+	case AMDGPU_INFO_FW_CAP:
+		fw_info->ver = adev->psp.cap_fw_version;
+		fw_info->feature = adev->psp.cap_feature_version;
+		break;
 	default:
 		return -EINVAL;
 	}
 		ui64 = atomic64_read(&adev->num_vram_cpu_page_faults);
 		return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0;
 	case AMDGPU_INFO_VRAM_USAGE:
-		ui64 = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr);
+		ui64 = ttm_resource_manager_usage(&adev->mman.vram_mgr.manager);
 		return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0;
 	case AMDGPU_INFO_VIS_VRAM_USAGE:
 		ui64 = amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr);
 		return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0;
 	case AMDGPU_INFO_GTT_USAGE:
-		ui64 = amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr);
+		ui64 = ttm_resource_manager_usage(&adev->mman.gtt_mgr.manager);
 		return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0;
 	case AMDGPU_INFO_GDS_CONFIG: {
 		struct drm_amdgpu_info_gds gds_info;
 	case AMDGPU_INFO_MEMORY: {
 		struct drm_amdgpu_memory_info mem;
 		struct ttm_resource_manager *gtt_man =
-			ttm_manager_type(&adev->mman.bdev, TTM_PL_TT);
+			&adev->mman.gtt_mgr.manager;
+		struct ttm_resource_manager *vram_man =
+			&adev->mman.vram_mgr.manager;
 		memset(&mem, 0, sizeof(mem));
 		mem.vram.total_heap_size = adev->gmc.real_vram_size;
 		mem.vram.usable_heap_size = adev->gmc.real_vram_size -
 			atomic64_read(&adev->vram_pin_size) -
 			AMDGPU_VM_RESERVED_VRAM;
 		mem.vram.heap_usage =
-			amdgpu_vram_mgr_usage(&adev->mman.vram_mgr);
+			ttm_resource_manager_usage(vram_man);
 		mem.vram.max_allocation = mem.vram.usable_heap_size * 3 / 4;
 		mem.cpu_accessible_vram.total_heap_size =
 		mem.gtt.total_heap_size *= PAGE_SIZE;
 		mem.gtt.usable_heap_size = mem.gtt.total_heap_size -
 			atomic64_read(&adev->gart_pin_size);
-		mem.gtt.heap_usage =
+		mem.gtt.heap_usage = ttm_resource_manager_usage(gtt_man);
-			amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr);
 		mem.gtt.max_allocation = mem.gtt.usable_heap_size * 3 / 4;
 		return copy_to_user(out, &mem,
 	if (amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_VCE) != NULL)
 		amdgpu_vce_free_handles(adev, file_priv);
-	amdgpu_vm_bo_rmv(adev, fpriv->prt_va);
 	if (amdgpu_mcbp || amdgpu_sriov_vf(adev)) {
 		/* TODO: how to handle reserve failure */
 		BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, true));
-		amdgpu_vm_bo_rmv(adev, fpriv->csa_va);
+		amdgpu_vm_bo_del(adev, fpriv->csa_va);
 		fpriv->csa_va = NULL;
 		amdgpu_bo_unreserve(adev->virt.csa_obj);
 	}
 	pasid = fpriv->vm.pasid;
 	pd = amdgpu_bo_ref(fpriv->vm.root.bo);
+	if (!WARN_ON(amdgpu_bo_reserve(pd, true))) {
+		amdgpu_vm_bo_del(adev, fpriv->prt_va);
+		amdgpu_bo_unreserve(pd);
+	}
 	amdgpu_ctx_mgr_fini(&fpriv->ctx_mgr);
 	amdgpu_vm_fini(adev, &fpriv->vm);
 	struct drm_amdgpu_info_firmware fw_info;
 	struct drm_amdgpu_query_fw query_fw;
 	struct atom_context *ctx = adev->mode_info.atom_context;
-	uint8_t smu_minor, smu_debug;
+	uint8_t smu_program, smu_major, smu_minor, smu_debug;
-	uint16_t smu_major;
 	int ret, i;
 	static const char *ta_fw_name[TA_FW_TYPE_MAX_INDEX] = {
 	ret = amdgpu_firmware_info(&fw_info, &query_fw, adev);
 	if (ret)
 		return ret;
-	smu_major = (fw_info.ver >> 16) & 0xffff;
+	smu_program = (fw_info.ver >> 24) & 0xff;
+	smu_major = (fw_info.ver >> 16) & 0xff;
 	smu_minor = (fw_info.ver >> 8) & 0xff;
 	smu_debug = (fw_info.ver >> 0) & 0xff;
-	seq_printf(m, "SMC feature version: %u, firmware version: 0x%08x (%d.%d.%d)\n",
+	seq_printf(m, "SMC feature version: %u, program: %d, firmware version: 0x%08x (%d.%d.%d)\n",
-		   fw_info.feature, fw_info.ver, smu_major, smu_minor, smu_debug);
+		   fw_info.feature, smu_program, fw_info.ver, smu_major, smu_minor, smu_debug);
 	/* SDMA */
 	query_fw.fw_type = AMDGPU_INFO_FW_SDMA;
 	seq_printf(m, "TOC feature version: %u, firmware version: 0x%08x\n",
 		   fw_info.feature, fw_info.ver);
+	/* CAP */
+	if (adev->psp.cap_fw) {
+		query_fw.fw_type = AMDGPU_INFO_FW_CAP;
+		ret = amdgpu_firmware_info(&fw_info, &query_fw, adev);
+		if (ret)
+			return ret;
+		seq_printf(m, "CAP feature version: %u, firmware version: 0x%08x\n",
+				fw_info.feature, fw_info.ver);
+	}
 	seq_printf(m, "VBIOS version: %s\n", ctx->vbios_version);
 	return 0;

drivers/gpu/drm/amd/amdgpu/amdgpu_mca.c+1−40

@@ -71,47 +71,8 @@ void amdgpu_mca_query_ras_error_count(struct amdgpu_device *adev, · 70 unmodified lines
71	amdgpu_mca_reset_error_count(adev, mc_status_addr);	71	amdgpu_mca_reset_error_count(adev, mc_status_addr);
72	}	72	}
73		73
74	int amdgpu_mca_ras_late_init(struct amdgpu_device *adev,
75	struct amdgpu_mca_ras *mca_dev)
76	{
77	int r;
78	struct ras_ih_if ih_info = {
79	.cb = NULL,
80	};
81	struct ras_fs_if fs_info = {
82	.sysfs_name = mca_dev->ras_funcs->sysfs_name,
83	};
84
85	if (!mca_dev->ras_if) {
86	mca_dev->ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
87	if (!mca_dev->ras_if)
88	return -ENOMEM;
89	mca_dev->ras_if->block = mca_dev->ras_funcs->ras_block;
90	mca_dev->ras_if->sub_block_index = mca_dev->ras_funcs->ras_sub_block;
91	mca_dev->ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
92	}
93	ih_info.head = fs_info.head = *mca_dev->ras_if;
94	r = amdgpu_ras_late_init(adev, mca_dev->ras_if,
95	&fs_info, &ih_info);
96	if (r \|\| !amdgpu_ras_is_supported(adev, mca_dev->ras_if->block)) {
97	kfree(mca_dev->ras_if);
98	mca_dev->ras_if = NULL;
99	}
100
101	return r;
102	}
103
104	void amdgpu_mca_ras_fini(struct amdgpu_device *adev,	74	void amdgpu_mca_ras_fini(struct amdgpu_device *adev,
105	struct amdgpu_mca_ras *mca_dev)	75	struct amdgpu_mca_ras *mca_dev)
106	{	76	{
107	struct ras_ih_if ih_info = {	77	amdgpu_ras_block_late_fini(adev, mca_dev->ras_if);
108	.cb = NULL,
109	};
110
111	if (!mca_dev->ras_if)
112	return;
113
114	amdgpu_ras_late_fini(adev, mca_dev->ras_if, &ih_info);
115	kfree(mca_dev->ras_if);
116	mca_dev->ras_if = NULL;
117	}	78	}

drivers/gpu/drm/amd/amdgpu/amdgpu_mca.h+3−14

 #ifndef __AMDGPU_MCA_H__
 #define __AMDGPU_MCA_H__
-struct amdgpu_mca_ras_funcs {
+struct amdgpu_mca_ras_block {
-	int (*ras_late_init)(struct amdgpu_device *adev);
+	struct amdgpu_ras_block_object ras_block;
-	void (*ras_fini)(struct amdgpu_device *adev);
-	void (*query_ras_error_count)(struct amdgpu_device *adev,
-				      void *ras_error_status);
-	void (*query_ras_error_address)(struct amdgpu_device *adev,
-					void *ras_error_status);
-	uint32_t ras_block;
-	uint32_t ras_sub_block;
-	const char* sysfs_name;
 };
 struct amdgpu_mca_ras {
 	struct ras_common_if *ras_if;
-	const struct amdgpu_mca_ras_funcs *ras_funcs;
+	struct amdgpu_mca_ras_block *ras;
 };
 struct amdgpu_mca_funcs {
 				      uint64_t mc_status_addr,
 				      void *ras_error_status);
-int amdgpu_mca_ras_late_init(struct amdgpu_device *adev,
-			     struct amdgpu_mca_ras *mca_dev);
 void amdgpu_mca_ras_fini(struct amdgpu_device *adev,
 			 struct amdgpu_mca_ras *mca_dev);

drivers/gpu/drm/amd/amdgpu/amdgpu_mmhub.c+2−38

@@ -24,45 +24,9 @@ · 23 unmodified lines
24	#include "amdgpu.h"	24	#include "amdgpu.h"
25	#include "amdgpu_ras.h"	25	#include "amdgpu_ras.h"
26		26
27	int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev)
28	{
29	int r;
30	struct ras_ih_if ih_info = {
31	.cb = NULL,
32	};
33	struct ras_fs_if fs_info = {
34	.sysfs_name = "mmhub_err_count",
35	};
36
37	if (!adev->mmhub.ras_if) {
38	adev->mmhub.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
39	if (!adev->mmhub.ras_if)
40	return -ENOMEM;
41	adev->mmhub.ras_if->block = AMDGPU_RAS_BLOCK__MMHUB;
42	adev->mmhub.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
43	adev->mmhub.ras_if->sub_block_index = 0;
44	}
45	ih_info.head = fs_info.head = *adev->mmhub.ras_if;
46	r = amdgpu_ras_late_init(adev, adev->mmhub.ras_if,
47	&fs_info, &ih_info);
48	if (r \|\| !amdgpu_ras_is_supported(adev, adev->mmhub.ras_if->block)) {
49	kfree(adev->mmhub.ras_if);
50	adev->mmhub.ras_if = NULL;
51	}
52
53	return r;
54	}
55
56	void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev)	27	void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev)
57	{	28	{
58	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__MMHUB) &&	29	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__MMHUB) &&
59	adev->mmhub.ras_if) {	30	adev->mmhub.ras_if)
60	struct ras_common_if *ras_if = adev->mmhub.ras_if;	31	amdgpu_ras_block_late_fini(adev, adev->mmhub.ras_if);
61	struct ras_ih_if ih_info = {
62	.cb = NULL,
63	};
64
65	amdgpu_ras_late_fini(adev, ras_if, &ih_info);
66	kfree(ras_if);
67	}
68	}	32	}

drivers/gpu/drm/amd/amdgpu/amdgpu_mmhub.h+3−10

 #ifndef __AMDGPU_MMHUB_H__
 #define __AMDGPU_MMHUB_H__
-struct amdgpu_mmhub_ras_funcs {
+struct amdgpu_mmhub_ras {
-	int (*ras_late_init)(struct amdgpu_device *adev);
+	struct amdgpu_ras_block_object ras_block;
-	void (*ras_fini)(struct amdgpu_device *adev);
-	void (*query_ras_error_count)(struct amdgpu_device *adev,
-				      void *ras_error_status);
-	void (*query_ras_error_status)(struct amdgpu_device *adev);
-	void (*reset_ras_error_count)(struct amdgpu_device *adev);
-	void (*reset_ras_error_status)(struct amdgpu_device *adev);
 };
 struct amdgpu_mmhub_funcs {
 struct amdgpu_mmhub {
 	struct ras_common_if *ras_if;
 	const struct amdgpu_mmhub_funcs *funcs;
-	const struct amdgpu_mmhub_ras_funcs *ras_funcs;
+	struct amdgpu_mmhub_ras  *ras;
 };
-int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev);
 void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev);
 #endif

drivers/gpu/drm/amd/amdgpu/amdgpu_nbio.c+7−37

 #include "amdgpu.h"
 #include "amdgpu_ras.h"
-int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev)
+int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block)
 {
 	int r;
-	struct ras_ih_if ih_info = {
+	r = amdgpu_ras_block_late_init(adev, ras_block);
-		.cb = NULL,
-	};
-	struct ras_fs_if fs_info = {
-		.sysfs_name = "pcie_bif_err_count",
-	};
-	if (!adev->nbio.ras_if) {
-		adev->nbio.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
-		if (!adev->nbio.ras_if)
-			return -ENOMEM;
-		adev->nbio.ras_if->block = AMDGPU_RAS_BLOCK__PCIE_BIF;
-		adev->nbio.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
-		adev->nbio.ras_if->sub_block_index = 0;
-	}
-	ih_info.head = fs_info.head = *adev->nbio.ras_if;
-	r = amdgpu_ras_late_init(adev, adev->nbio.ras_if,
-				 &fs_info, &ih_info);
 	if (r)
-		goto free;
+		return r;
-	if (amdgpu_ras_is_supported(adev, adev->nbio.ras_if->block)) {
+	if (amdgpu_ras_is_supported(adev, ras_block->block)) {
 		r = amdgpu_irq_get(adev, &adev->nbio.ras_controller_irq, 0);
 		if (r)
 			goto late_fini;
 		r = amdgpu_irq_get(adev, &adev->nbio.ras_err_event_athub_irq, 0);
 		if (r)
 			goto late_fini;
-	} else {
-		r = 0;
-		goto free;
 	}
 	return 0;
 late_fini:
-	amdgpu_ras_late_fini(adev, adev->nbio.ras_if, &ih_info);
+	amdgpu_ras_block_late_fini(adev, ras_block);
-free:
-	kfree(adev->nbio.ras_if);
-	adev->nbio.ras_if = NULL;
 	return r;
 }
 void amdgpu_nbio_ras_fini(struct amdgpu_device *adev)
 {
 	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF) &&
-			adev->nbio.ras_if) {
+			adev->nbio.ras_if)
-		struct ras_common_if *ras_if = adev->nbio.ras_if;
+		amdgpu_ras_block_late_fini(adev, adev->nbio.ras_if);
-		struct ras_ih_if ih_info = {
-			.cb = NULL,
-		};
-		amdgpu_ras_late_fini(adev, ras_if, &ih_info);
-		kfree(ras_if);
-	}
 }

drivers/gpu/drm/amd/amdgpu/amdgpu_nbio.h+4−7

 	u32 ref_and_mask_sdma7;
 };
-struct amdgpu_nbio_ras_funcs {
+struct amdgpu_nbio_ras {
+	struct amdgpu_ras_block_object ras_block;
 	void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev);
 	void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev);
 	int (*init_ras_controller_interrupt)(struct amdgpu_device *adev);
 	int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev);
-	void (*query_ras_error_count)(struct amdgpu_device *adev,
-				      void *ras_error_status);
-	int (*ras_late_init)(struct amdgpu_device *adev);
-	void (*ras_fini)(struct amdgpu_device *adev);
 };
 struct amdgpu_nbio_funcs {
 	struct amdgpu_irq_src ras_err_event_athub_irq;
 	struct ras_common_if *ras_if;
 	const struct amdgpu_nbio_funcs *funcs;
-	const struct amdgpu_nbio_ras_funcs *ras_funcs;
+	struct amdgpu_nbio_ras  *ras;
 };
-int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev);
+int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block);
 void amdgpu_nbio_ras_fini(struct amdgpu_device *adev);
 #endif

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c+7−3

 	if (domain & AMDGPU_GEM_DOMAIN_GTT) {
 		man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT);
-		if (size < (man->size << PAGE_SHIFT))
+		if (size < man->size)
 			return true;
 		else
 			goto fail;
 	if (domain & AMDGPU_GEM_DOMAIN_VRAM) {
 		man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
-		if (size < (man->size << PAGE_SHIFT))
+		if (size < man->size)
 			return true;
 		else
 			goto fail;
 	if (!amdgpu_bo_support_uswc(bo->flags))
 		bo->flags &= ~AMDGPU_GEM_CREATE_CPU_GTT_USWC;
+	if (adev->ras_enabled)
+		bo->flags |= AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE;
 	bo->tbo.bdev = &adev->mman.bdev;
 	if (bp->domain & (AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA |
 			  AMDGPU_GEM_DOMAIN_GDS))
 	    !(abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE))
 		return;
-	dma_resv_lock(bo->base.resv, NULL);
+	if (WARN_ON_ONCE(!dma_resv_trylock(bo->base.resv)))
+		return;
 	r = amdgpu_fill_buffer(abo, AMDGPU_POISON, bo->base.resv, &fence);
 	if (!WARN_ON(r)) {

drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c+5−57

 #include "amdgpu.h"
-static inline struct amdgpu_preempt_mgr *
-to_preempt_mgr(struct ttm_resource_manager *man)
-{
-	return container_of(man, struct amdgpu_preempt_mgr, manager);
-}
 /**
  * DOC: mem_info_preempt_used
  *
 {
 	struct drm_device *ddev = dev_get_drvdata(dev);
 	struct amdgpu_device *adev = drm_to_adev(ddev);
-	struct ttm_resource_manager *man;
+	struct ttm_resource_manager *man = &adev->mman.preempt_mgr;
-	man = ttm_manager_type(&adev->mman.bdev, AMDGPU_PL_PREEMPT);
+	return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man));
-	return sysfs_emit(buf, "%llu\n", amdgpu_preempt_mgr_usage(man));
 }
 static DEVICE_ATTR_RO(mem_info_preempt_used);
 				  const struct ttm_place *place,
 				  struct ttm_resource **res)
 {
-	struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man);
 	*res = kzalloc(sizeof(**res), GFP_KERNEL);
 	if (!*res)
 		return -ENOMEM;
 	ttm_resource_init(tbo, place, *res);
 	(*res)->start = AMDGPU_BO_INVALID_OFFSET;
-	atomic64_add((*res)->num_pages, &mgr->used);
 	return 0;
 }
 static void amdgpu_preempt_mgr_del(struct ttm_resource_manager *man,
 				   struct ttm_resource *res)
 {
-	struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man);
-	atomic64_sub(res->num_pages, &mgr->used);
 	ttm_resource_fini(man, res);
 	kfree(res);
 }
-/**
- * amdgpu_preempt_mgr_usage - return usage of PREEMPT domain
- *
- * @man: TTM memory type manager
- *
- * Return how many bytes are used in the GTT domain
- */
-uint64_t amdgpu_preempt_mgr_usage(struct ttm_resource_manager *man)
-{
-	struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man);
-	s64 result = atomic64_read(&mgr->used);
-	return (result > 0 ? result : 0) * PAGE_SIZE;
-}
-/**
- * amdgpu_preempt_mgr_debug - dump VRAM table
- *
- * @man: TTM memory type manager
- * @printer: DRM printer to use
- *
- * Dump the table content using printk.
- */
-static void amdgpu_preempt_mgr_debug(struct ttm_resource_manager *man,
-				     struct drm_printer *printer)
-{
-	struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man);
-	drm_printf(printer, "man size:%llu pages, preempt used:%lld pages\n",
-		   man->size, (u64)atomic64_read(&mgr->used));
-}
 static const struct ttm_resource_manager_func amdgpu_preempt_mgr_func = {
 	.alloc = amdgpu_preempt_mgr_new,
 	.free = amdgpu_preempt_mgr_del,
-	.debug = amdgpu_preempt_mgr_debug
 };
 /**
  */
 int amdgpu_preempt_mgr_init(struct amdgpu_device *adev)
 {
-	struct amdgpu_preempt_mgr *mgr = &adev->mman.preempt_mgr;
+	struct ttm_resource_manager *man = &adev->mman.preempt_mgr;
-	struct ttm_resource_manager *man = &mgr->manager;
 	int ret;
 	man->use_tt = true;
 	ttm_resource_manager_init(man, &adev->mman.bdev, (1 << 30));
-	atomic64_set(&mgr->used, 0);
 	ret = device_create_file(adev->dev, &dev_attr_mem_info_preempt_used);
 	if (ret) {
 		DRM_ERROR("Failed to create device file mem_info_preempt_used\n");
 		return ret;
 	}
-	ttm_set_driver_manager(&adev->mman.bdev, AMDGPU_PL_PREEMPT,
+	ttm_set_driver_manager(&adev->mman.bdev, AMDGPU_PL_PREEMPT, man);
-			       &mgr->manager);
 	ttm_resource_manager_set_used(man, true);
 	return 0;
 }
  */
 void amdgpu_preempt_mgr_fini(struct amdgpu_device *adev)
 {
-	struct amdgpu_preempt_mgr *mgr = &adev->mman.preempt_mgr;
+	struct ttm_resource_manager *man = &adev->mman.preempt_mgr;
-	struct ttm_resource_manager *man = &mgr->manager;
 	int ret;
 	ttm_resource_manager_set_used(man, false);

drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c+105−19

@@ -133,6 +133,7 @@ static int psp_early_init(void *handle) · 132 unmodified lines
133	break;	133	break;
134	case IP_VERSION(13, 0, 1):	134	case IP_VERSION(13, 0, 1):
135	case IP_VERSION(13, 0, 3):	135	case IP_VERSION(13, 0, 3):
		136	case IP_VERSION(13, 0, 8):
136	psp_v13_0_set_psp_funcs(psp);	137	psp_v13_0_set_psp_funcs(psp);
137	psp->autoload_supported = true;	138	psp->autoload_supported = true;
138	break;	139	break;
@@ -259,6 +260,32 @@ static bool psp_get_runtime_db_entry(struct amdgpu_device *adev, · 120 unmodified lines
259	return ret;	260	return ret;
260	}	261	}
261		262
		263	static int psp_init_sriov_microcode(struct psp_context *psp)
		264	{
		265	struct amdgpu_device *adev = psp->adev;
		266	int ret = 0;
		267
		268	switch (adev->ip_versions[MP0_HWIP][0]) {
		269	case IP_VERSION(9, 0, 0):
		270	ret = psp_init_cap_microcode(psp, "vega10");
		271	break;
		272	case IP_VERSION(11, 0, 9):
		273	ret = psp_init_cap_microcode(psp, "navi12");
		274	break;
		275	case IP_VERSION(11, 0, 7):
		276	ret = psp_init_cap_microcode(psp, "sienna_cichlid");
		277	break;
		278	case IP_VERSION(13, 0, 2):
		279	ret = psp_init_ta_microcode(psp, "aldebaran");
		280	break;
		281	default:
		282	BUG();
		283	break;
		284	}
		285
		286	return ret;
		287	}
		288
262	static int psp_sw_init(void *handle)	289	static int psp_sw_init(void *handle)
263	{	290	{
264	struct amdgpu_device adev = (struct amdgpu_device )handle;	291	struct amdgpu_device adev = (struct amdgpu_device )handle;
@@ -273,19 +300,13 @@ static int psp_sw_init(void *handle) · 8 unmodified lines
273	ret = -ENOMEM;	300	ret = -ENOMEM;
274	}	301	}
275		302
276	if (!amdgpu_sriov_vf(adev)) {	303	if (amdgpu_sriov_vf(adev))
		304	ret = psp_init_sriov_microcode(psp);
		305	else
277	ret = psp_init_microcode(psp);	306	ret = psp_init_microcode(psp);
278	if (ret) {	307	if (ret) {
279	DRM_ERROR("Failed to load psp firmware!\n");	308	DRM_ERROR("Failed to load psp firmware!\n");
280	return ret;	309	return ret;
281	}
282	} else if (amdgpu_sriov_vf(adev) &&
283	adev->ip_versions[MP0_HWIP][0] == IP_VERSION(13, 0, 2)) {
284	ret = psp_init_ta_microcode(psp, "aldebaran");
285	if (ret) {
286	DRM_ERROR("Failed to initialize ta microcode!\n");
287	return ret;
288	}
289	}	310	}
290		311
291	memset(&boot_cfg_entry, 0, sizeof(boot_cfg_entry));	312	memset(&boot_cfg_entry, 0, sizeof(boot_cfg_entry));
@@ -353,6 +374,10 @@ static int psp_sw_fini(void *handle) · 61 unmodified lines
353	release_firmware(psp->ta_fw);	374	release_firmware(psp->ta_fw);
354	psp->ta_fw = NULL;	375	psp->ta_fw = NULL;
355	}	376	}
		377	if (adev->psp.cap_fw) {
		378	release_firmware(psp->cap_fw);
		379	psp->cap_fw = NULL;
		380	}
356		381
357	if (adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 0) \|\|	382	if (adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 0) \|\|
358	adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 7))	383	adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 7))
@@ -491,7 +516,10 @@ psp_cmd_submit_buf(struct psp_context *psp, · 132 unmodified lines
491	DRM_WARN("psp gfx command %s(0x%X) failed and response status is (0x%X)\n",	516	DRM_WARN("psp gfx command %s(0x%X) failed and response status is (0x%X)\n",
492	psp_gfx_cmd_name(psp->cmd_buf_mem->cmd_id), psp->cmd_buf_mem->cmd_id,	517	psp_gfx_cmd_name(psp->cmd_buf_mem->cmd_id), psp->cmd_buf_mem->cmd_id,
493	psp->cmd_buf_mem->resp.status);	518	psp->cmd_buf_mem->resp.status);
494	if (!timeout) {	519	/* If we load CAP FW, PSP must return 0 under SRIOV
		520	* also return failure in case of timeout
		521	*/
		522	if ((ucode && (ucode->ucode_id == AMDGPU_UCODE_ID_CAP)) \|\| !timeout) {
495	ret = -EINVAL;	523	ret = -EINVAL;
496	goto exit;	524	goto exit;
497	}	525	}
@@ -914,19 +942,15 @@ static void psp_prep_ta_load_cmd_buf(struct psp_gfx_cmd_resp *cmd, · 416 unmodified lines
914	static int psp_ta_init_shared_buf(struct psp_context *psp,	942	static int psp_ta_init_shared_buf(struct psp_context *psp,
915	struct ta_mem_context *mem_ctx)	943	struct ta_mem_context *mem_ctx)
916	{	944	{
917	int ret;
918
919	/*	945	/*
920	* Allocate 16k memory aligned to 4k from Frame Buffer (local	946	* Allocate 16k memory aligned to 4k from Frame Buffer (local
921	* physical) for ta to host memory	947	* physical) for ta to host memory
922	*/	948	*/
923	ret = amdgpu_bo_create_kernel(psp->adev, mem_ctx->shared_mem_size,	949	return amdgpu_bo_create_kernel(psp->adev, mem_ctx->shared_mem_size,
924	PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM,	950	PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM,
925	&mem_ctx->shared_bo,	951	&mem_ctx->shared_bo,
926	&mem_ctx->shared_mc_addr,	952	&mem_ctx->shared_mc_addr,
927	&mem_ctx->shared_buf);	953	&mem_ctx->shared_buf);
928
929	return ret;
930	}	954	}
931		955
932	static void psp_ta_free_shared_buf(struct ta_mem_context *mem_ctx)	956	static void psp_ta_free_shared_buf(struct ta_mem_context *mem_ctx)
@@ -1308,6 +1332,11 @@ static void psp_ras_ta_check_status(struct psp_context *psp) · 375 unmodified lines
1308	break;	1332	break;
1309	case TA_RAS_STATUS__SUCCESS:	1333	case TA_RAS_STATUS__SUCCESS:
1310	break;	1334	break;
		1335	case TA_RAS_STATUS__TEE_ERROR_ACCESS_DENIED:
		1336	if (ras_cmd->cmd_id == TA_RAS_COMMAND__TRIGGER_ERROR)
		1337	dev_warn(psp->adev->dev,
		1338	"RAS WARNING: Inject error to critical region is not allowed\n");
		1339	break;
1311	default:	1340	default:
1312	dev_warn(psp->adev->dev,	1341	dev_warn(psp->adev->dev,
1313	"RAS WARNING: ras status = 0x%X\n", ras_cmd->ras_status);	1342	"RAS WARNING: ras status = 0x%X\n", ras_cmd->ras_status);
@@ -1520,7 +1549,9 @@ int psp_ras_trigger_error(struct psp_context *psp, · 206 unmodified lines
1520	if (amdgpu_ras_intr_triggered())	1549	if (amdgpu_ras_intr_triggered())
1521	return 0;	1550	return 0;
1522		1551
1523	if (ras_cmd->ras_status)	1552	if (ras_cmd->ras_status == TA_RAS_STATUS__TEE_ERROR_ACCESS_DENIED)
		1553	return -EACCES;
		1554	else if (ras_cmd->ras_status)
1524	return -EINVAL;	1555	return -EINVAL;
1525		1556
1526	return 0;	1557	return 0;
@@ -2051,6 +2082,9 @@ static int psp_get_fw_type(struct amdgpu_firmware_info *ucode, · 524 unmodified lines
2051	enum psp_gfx_fw_type *type)	2082	enum psp_gfx_fw_type *type)
2052	{	2083	{
2053	switch (ucode->ucode_id) {	2084	switch (ucode->ucode_id) {
		2085	case AMDGPU_UCODE_ID_CAP:
		2086	*type = GFX_FW_TYPE_CAP;
		2087	break;
2054	case AMDGPU_UCODE_ID_SDMA0:	2088	case AMDGPU_UCODE_ID_SDMA0:
2055	*type = GFX_FW_TYPE_SDMA0;	2089	*type = GFX_FW_TYPE_SDMA0;
2056	break;	2090	break;
@@ -3217,6 +3251,58 @@ out: · 1160 unmodified lines
3217	return err;	3251	return err;
3218	}	3252	}
3219		3253
		3254	int psp_init_cap_microcode(struct psp_context *psp,
		3255	const char *chip_name)
		3256	{
		3257	struct amdgpu_device *adev = psp->adev;
		3258	char fw_name[PSP_FW_NAME_LEN];
		3259	const struct psp_firmware_header_v1_0 *cap_hdr_v1_0;
		3260	struct amdgpu_firmware_info *info = NULL;
		3261	int err = 0;
		3262
		3263	if (!chip_name) {
		3264	dev_err(adev->dev, "invalid chip name for cap microcode\n");
		3265	return -EINVAL;
		3266	}
		3267
		3268	if (!amdgpu_sriov_vf(adev)) {
		3269	dev_err(adev->dev, "cap microcode should only be loaded under SRIOV\n");
		3270	return -EINVAL;
		3271	}
		3272
		3273	snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_cap.bin", chip_name);
		3274	err = request_firmware(&adev->psp.cap_fw, fw_name, adev->dev);
		3275	if (err) {
		3276	dev_warn(adev->dev, "cap microcode does not exist, skip\n");
		3277	err = 0;
		3278	goto out;
		3279	}
		3280
		3281	err = amdgpu_ucode_validate(adev->psp.cap_fw);
		3282	if (err) {
		3283	dev_err(adev->dev, "fail to initialize cap microcode\n");
		3284	goto out;
		3285	}
		3286
		3287	info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CAP];
		3288	info->ucode_id = AMDGPU_UCODE_ID_CAP;
		3289	info->fw = adev->psp.cap_fw;
		3290	cap_hdr_v1_0 = (const struct psp_firmware_header_v1_0 *)
		3291	adev->psp.cap_fw->data;
		3292	adev->firmware.fw_size += ALIGN(
		3293	le32_to_cpu(cap_hdr_v1_0->header.ucode_size_bytes), PAGE_SIZE);
		3294	adev->psp.cap_fw_version = le32_to_cpu(cap_hdr_v1_0->header.ucode_version);
		3295	adev->psp.cap_feature_version = le32_to_cpu(cap_hdr_v1_0->sos.fw_version);
		3296	adev->psp.cap_ucode_size = le32_to_cpu(cap_hdr_v1_0->header.ucode_size_bytes);
		3297
		3298	return 0;
		3299
		3300	out:
		3301	release_firmware(adev->psp.cap_fw);
		3302	adev->psp.cap_fw = NULL;
		3303	return err;
		3304	}
		3305
3220	static int psp_set_clockgating_state(void *handle,	3306	static int psp_set_clockgating_state(void *handle,
3221	enum amd_clockgating_state state)	3307	enum amd_clockgating_state state)
3222	{	3308	{

drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h+9−0

 	/* toc firmware */
 	const struct firmware		*toc_fw;
+	/* cap firmware */
+	const struct firmware		*cap_fw;
 	/* fence buffer */
 	struct amdgpu_bo		*fence_buf_bo;
 	uint64_t			fence_buf_mc_addr;
 	const struct firmware		*ta_fw;
 	uint32_t			ta_fw_version;
+	uint32_t			cap_fw_version;
+	uint32_t			cap_feature_version;
+	uint32_t			cap_ucode_size;
 	struct ta_context		asd_context;
 	struct psp_xgmi_context		xgmi_context;
 	struct psp_ras_context		ras_context;
 			   const char *chip_name);
 int psp_init_ta_microcode(struct psp_context *psp,
 			  const char *chip_name);
+int psp_init_cap_microcode(struct psp_context *psp,
+			  const char *chip_name);
 int psp_get_fw_attestation_records_addr(struct psp_context *psp,
 					uint64_t *output_ptr);

drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c+285−240

@@ -75,6 +75,13 @@ const char *ras_mca_block_string[] = { · 74 unmodified lines
75	"mca_iohc",	75	"mca_iohc",
76	};	76	};
77		77
		78	struct amdgpu_ras_block_list {
		79	/* ras block link */
		80	struct list_head node;
		81
		82	struct amdgpu_ras_block_object *ras_obj;
		83	};
		84
78	const char get_ras_block_str(struct ras_common_if ras_block)	85	const char get_ras_block_str(struct ras_common_if ras_block)
79	{	86	{
80	if (!ras_block)	87	if (!ras_block)
@@ -89,6 +96,9 @@ const char get_ras_block_str(struct ras_common_if ras_block) · 8 unmodified lines
89	return ras_block_string[ras_block->block];	96	return ras_block_string[ras_block->block];
90	}	97	}
91		98
		99	#define ras_block_str(_BLOCK_) \
		100	(((_BLOCK_) < ARRAY_SIZE(ras_block_string)) ? ras_block_string[_BLOCK_] : "Out Of Range")
		101
92	#define ras_err_str(i) (ras_error_string[ffs(i)])	102	#define ras_err_str(i) (ras_error_string[ffs(i)])
93		103
94	#define RAS_DEFAULT_FLAGS (AMDGPU_RAS_FLAG_INIT_BY_VBIOS)	104	#define RAS_DEFAULT_FLAGS (AMDGPU_RAS_FLAG_INIT_BY_VBIOS)
@@ -155,14 +165,9 @@ static int amdgpu_reserve_page_direct(struct amdgpu_device *adev, uint64_t addre · 60 unmodified lines
155	}	165	}
156		166
157	memset(&err_rec, 0x0, sizeof(struct eeprom_table_record));	167	memset(&err_rec, 0x0, sizeof(struct eeprom_table_record));
158
159	err_rec.address = address;
160	err_rec.retired_page = address >> AMDGPU_GPU_PAGE_SHIFT;
161	err_rec.ts = (uint64_t)ktime_get_real_seconds();
162	err_rec.err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
163
164	err_data.err_addr = &err_rec;	168	err_data.err_addr = &err_rec;
165	err_data.err_addr_cnt = 1;	169	amdgpu_umc_fill_error_record(&err_data, address,
		170	(address >> AMDGPU_GPU_PAGE_SHIFT), 0, 0);
166		171
167	if (amdgpu_bad_page_threshold != 0) {	172	if (amdgpu_bad_page_threshold != 0) {
168	amdgpu_ras_add_bad_pages(adev, err_data.err_addr,	173	amdgpu_ras_add_bad_pages(adev, err_data.err_addr,
@@ -452,7 +457,7 @@ static ssize_t amdgpu_ras_debugfs_ctrl_write(struct file *f, · 283 unmodified lines
452	}	457	}
453		458
454	if (ret)	459	if (ret)
455	return -EINVAL;	460	return ret;
456		461
457	return size;	462	return size;
458	}	463	}
@@ -866,30 +871,47 @@ static int amdgpu_ras_enable_all_features(struct amdgpu_device *adev, · 407 unmodified lines
866	}	871	}
867	/* feature ctl end */	872	/* feature ctl end */
868		873
		874	static int amdgpu_ras_block_match_default(struct amdgpu_ras_block_object *block_obj,
		875	enum amdgpu_ras_block block)
		876	{
		877	if (!block_obj)
		878	return -EINVAL;
		879
		880	if (block_obj->ras_comm.block == block)
		881	return 0;
		882
		883	return -EINVAL;
		884	}
869		885
870	static void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev,	886	static struct amdgpu_ras_block_object amdgpu_ras_get_ras_block(struct amdgpu_device adev,
871	struct ras_common_if *ras_block,	887	enum amdgpu_ras_block block, uint32_t sub_block_index)
872	struct ras_err_data *err_data)
873	{	888	{
874	switch (ras_block->sub_block_index) {	889	struct amdgpu_ras_block_list node, tmp;
875	case AMDGPU_RAS_MCA_BLOCK__MP0:	890	struct amdgpu_ras_block_object *obj;
876	if (adev->mca.mp0.ras_funcs &&	891
877	adev->mca.mp0.ras_funcs->query_ras_error_count)	892	if (block >= AMDGPU_RAS_BLOCK__LAST)
878	adev->mca.mp0.ras_funcs->query_ras_error_count(adev, &err_data);	893	return NULL;
879	break;	894
880	case AMDGPU_RAS_MCA_BLOCK__MP1:	895	if (!amdgpu_ras_is_supported(adev, block))
881	if (adev->mca.mp1.ras_funcs &&	896	return NULL;
882	adev->mca.mp1.ras_funcs->query_ras_error_count)	897
883	adev->mca.mp1.ras_funcs->query_ras_error_count(adev, &err_data);	898	list_for_each_entry_safe(node, tmp, &adev->ras_list, node) {
884	break;	899	if (!node->ras_obj) {
885	case AMDGPU_RAS_MCA_BLOCK__MPIO:	900	dev_warn(adev->dev, "Warning: abnormal ras list node.\n");
886	if (adev->mca.mpio.ras_funcs &&	901	continue;
887	adev->mca.mpio.ras_funcs->query_ras_error_count)	902	}
888	adev->mca.mpio.ras_funcs->query_ras_error_count(adev, &err_data);	903
889	break;	904	obj = node->ras_obj;
890	default:	905	if (obj->ras_block_match) {
891	break;	906	if (obj->ras_block_match(obj, block, sub_block_index) == 0)
		907	return obj;
		908	} else {
		909	if (amdgpu_ras_block_match_default(obj, block) == 0)
		910	return obj;
		911	}
892	}	912	}
		913
		914	return NULL;
893	}	915	}
894		916
895	static void amdgpu_ras_get_ecc_info(struct amdgpu_device adev, struct ras_err_data err_data)	917	static void amdgpu_ras_get_ecc_info(struct amdgpu_device adev, struct ras_err_data err_data)
@@ -901,26 +923,26 @@ static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_d · 5 unmodified lines
901	* choosing right query method according to	923	* choosing right query method according to
902	* whether smu support query error information	924	* whether smu support query error information
903	*/	925	*/
904	ret = smu_get_ecc_info(&adev->smu, (void *)&(ras->umc_ecc));	926	ret = amdgpu_dpm_get_ecc_info(adev, (void *)&(ras->umc_ecc));
905	if (ret == -EOPNOTSUPP) {	927	if (ret == -EOPNOTSUPP) {
906	if (adev->umc.ras_funcs &&	928	if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops &&
907	adev->umc.ras_funcs->query_ras_error_count)	929	adev->umc.ras->ras_block.hw_ops->query_ras_error_count)
908	adev->umc.ras_funcs->query_ras_error_count(adev, err_data);	930	adev->umc.ras->ras_block.hw_ops->query_ras_error_count(adev, err_data);
909		931
910	/* umc query_ras_error_address is also responsible for clearing	932	/* umc query_ras_error_address is also responsible for clearing
911	* error status	933	* error status
912	*/	934	*/
913	if (adev->umc.ras_funcs &&	935	if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops &&
914	adev->umc.ras_funcs->query_ras_error_address)	936	adev->umc.ras->ras_block.hw_ops->query_ras_error_address)
915	adev->umc.ras_funcs->query_ras_error_address(adev, err_data);	937	adev->umc.ras->ras_block.hw_ops->query_ras_error_address(adev, err_data);
916	} else if (!ret) {	938	} else if (!ret) {
917	if (adev->umc.ras_funcs &&	939	if (adev->umc.ras &&
918	adev->umc.ras_funcs->ecc_info_query_ras_error_count)	940	adev->umc.ras->ecc_info_query_ras_error_count)
919	adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, err_data);	941	adev->umc.ras->ecc_info_query_ras_error_count(adev, err_data);
920		942
921	if (adev->umc.ras_funcs &&	943	if (adev->umc.ras &&
922	adev->umc.ras_funcs->ecc_info_query_ras_error_address)	944	adev->umc.ras->ecc_info_query_ras_error_address)
923	adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, err_data);	945	adev->umc.ras->ecc_info_query_ras_error_address(adev, err_data);
924	}	946	}
925	}	947	}
926		948
@@ -928,62 +950,32 @@ static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_d · 1 unmodified line
928	int amdgpu_ras_query_error_status(struct amdgpu_device *adev,	950	int amdgpu_ras_query_error_status(struct amdgpu_device *adev,
929	struct ras_query_if *info)	951	struct ras_query_if *info)
930	{	952	{
		953	struct amdgpu_ras_block_object *block_obj = NULL;
931	struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head);	954	struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head);
932	struct ras_err_data err_data = {0, 0, 0, NULL};	955	struct ras_err_data err_data = {0, 0, 0, NULL};
933	int i;
934		956
935	if (!obj)	957	if (!obj)
936	return -EINVAL;	958	return -EINVAL;
937		959
938	switch (info->head.block) {	960	if (info->head.block == AMDGPU_RAS_BLOCK__UMC) {
939	case AMDGPU_RAS_BLOCK__UMC:
940	amdgpu_ras_get_ecc_info(adev, &err_data);	961	amdgpu_ras_get_ecc_info(adev, &err_data);
941	break;	962	} else {
942	case AMDGPU_RAS_BLOCK__SDMA:	963	block_obj = amdgpu_ras_get_ras_block(adev, info->head.block, 0);
943	if (adev->sdma.funcs->query_ras_error_count) {	964	if (!block_obj \|\| !block_obj->hw_ops) {
944	for (i = 0; i < adev->sdma.num_instances; i++)	965	dev_dbg_once(adev->dev, "%s doesn't config RAS function\n",
945	adev->sdma.funcs->query_ras_error_count(adev, i,	966	get_ras_block_str(&info->head));
946	&err_data);	967	return -EINVAL;
947	}	968	}
948	break;	969
949	case AMDGPU_RAS_BLOCK__GFX:	970	if (block_obj->hw_ops->query_ras_error_count)
950	if (adev->gfx.ras_funcs &&	971	block_obj->hw_ops->query_ras_error_count(adev, &err_data);
951	adev->gfx.ras_funcs->query_ras_error_count)	972
952	adev->gfx.ras_funcs->query_ras_error_count(adev, &err_data);	973	if ((info->head.block == AMDGPU_RAS_BLOCK__SDMA) \|\|
953		974	(info->head.block == AMDGPU_RAS_BLOCK__GFX) \|\|
954	if (adev->gfx.ras_funcs &&	975	(info->head.block == AMDGPU_RAS_BLOCK__MMHUB)) {
955	adev->gfx.ras_funcs->query_ras_error_status)	976	if (block_obj->hw_ops->query_ras_error_status)
956	adev->gfx.ras_funcs->query_ras_error_status(adev);	977	block_obj->hw_ops->query_ras_error_status(adev);
957	break;	978	}
958	case AMDGPU_RAS_BLOCK__MMHUB:
959	if (adev->mmhub.ras_funcs &&
960	adev->mmhub.ras_funcs->query_ras_error_count)
961	adev->mmhub.ras_funcs->query_ras_error_count(adev, &err_data);
962
963	if (adev->mmhub.ras_funcs &&
964	adev->mmhub.ras_funcs->query_ras_error_status)
965	adev->mmhub.ras_funcs->query_ras_error_status(adev);
966	break;
967	case AMDGPU_RAS_BLOCK__PCIE_BIF:
968	if (adev->nbio.ras_funcs &&
969	adev->nbio.ras_funcs->query_ras_error_count)
970	adev->nbio.ras_funcs->query_ras_error_count(adev, &err_data);
971	break;
972	case AMDGPU_RAS_BLOCK__XGMI_WAFL:
973	if (adev->gmc.xgmi.ras_funcs &&
974	adev->gmc.xgmi.ras_funcs->query_ras_error_count)
975	adev->gmc.xgmi.ras_funcs->query_ras_error_count(adev, &err_data);
976	break;
977	case AMDGPU_RAS_BLOCK__HDP:
978	if (adev->hdp.ras_funcs &&
979	adev->hdp.ras_funcs->query_ras_error_count)
980	adev->hdp.ras_funcs->query_ras_error_count(adev, &err_data);
981	break;
982	case AMDGPU_RAS_BLOCK__MCA:
983	amdgpu_ras_mca_query_error_status(adev, &info->head, &err_data);
984	break;
985	default:
986	break;
987	}	979	}
988		980
989	obj->err_data.ue_count += err_data.ue_count;	981	obj->err_data.ue_count += err_data.ue_count;
@@ -1040,68 +1032,27 @@ int amdgpu_ras_query_error_status(struct amdgpu_device *adev, · 50 unmodified lines
1040	int amdgpu_ras_reset_error_status(struct amdgpu_device *adev,	1032	int amdgpu_ras_reset_error_status(struct amdgpu_device *adev,
1041	enum amdgpu_ras_block block)	1033	enum amdgpu_ras_block block)
1042	{	1034	{
		1035	struct amdgpu_ras_block_object *block_obj = amdgpu_ras_get_ras_block(adev, block, 0);
		1036
1043	if (!amdgpu_ras_is_supported(adev, block))	1037	if (!amdgpu_ras_is_supported(adev, block))
1044	return -EINVAL;	1038	return -EINVAL;
1045		1039
1046	switch (block) {	1040	if (!block_obj \|\| !block_obj->hw_ops) {
1047	case AMDGPU_RAS_BLOCK__GFX:	1041	dev_dbg_once(adev->dev, "%s doesn't config RAS function\n",
1048	if (adev->gfx.ras_funcs &&	1042	ras_block_str(block));
1049	adev->gfx.ras_funcs->reset_ras_error_count)	1043	return -EINVAL;
1050	adev->gfx.ras_funcs->reset_ras_error_count(adev);
1051
1052	if (adev->gfx.ras_funcs &&
1053	adev->gfx.ras_funcs->reset_ras_error_status)
1054	adev->gfx.ras_funcs->reset_ras_error_status(adev);
1055	break;
1056	case AMDGPU_RAS_BLOCK__MMHUB:
1057	if (adev->mmhub.ras_funcs &&
1058	adev->mmhub.ras_funcs->reset_ras_error_count)
1059	adev->mmhub.ras_funcs->reset_ras_error_count(adev);
1060
1061	if (adev->mmhub.ras_funcs &&
1062	adev->mmhub.ras_funcs->reset_ras_error_status)
1063	adev->mmhub.ras_funcs->reset_ras_error_status(adev);
1064	break;
1065	case AMDGPU_RAS_BLOCK__SDMA:
1066	if (adev->sdma.funcs->reset_ras_error_count)
1067	adev->sdma.funcs->reset_ras_error_count(adev);
1068	break;
1069	case AMDGPU_RAS_BLOCK__HDP:
1070	if (adev->hdp.ras_funcs &&
1071	adev->hdp.ras_funcs->reset_ras_error_count)
1072	adev->hdp.ras_funcs->reset_ras_error_count(adev);
1073	break;
1074	default:
1075	break;
1076	}	1044	}
1077		1045
1078	return 0;	1046	if (block_obj->hw_ops->reset_ras_error_count)
1079	}	1047	block_obj->hw_ops->reset_ras_error_count(adev);
1080
1081	/* Trigger XGMI/WAFL error */
1082	static int amdgpu_ras_error_inject_xgmi(struct amdgpu_device *adev,
1083	struct ta_ras_trigger_error_input *block_info)
1084	{
1085	int ret;
1086		1048
1087	if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_DISALLOW))	1049	if ((block == AMDGPU_RAS_BLOCK__GFX) \|\|
1088	dev_warn(adev->dev, "Failed to disallow df cstate");	1050	(block == AMDGPU_RAS_BLOCK__MMHUB)) {
1089		1051	if (block_obj->hw_ops->reset_ras_error_status)
1090	if (amdgpu_dpm_allow_xgmi_power_down(adev, false))	1052	block_obj->hw_ops->reset_ras_error_status(adev);
1091	dev_warn(adev->dev, "Failed to disallow XGMI power down");	1053	}
1092
1093	ret = psp_ras_trigger_error(&adev->psp, block_info);
1094
1095	if (amdgpu_ras_intr_triggered())
1096	return ret;
1097
1098	if (amdgpu_dpm_allow_xgmi_power_down(adev, true))
1099	dev_warn(adev->dev, "Failed to allow XGMI power down");
1100
1101	if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_ALLOW))
1102	dev_warn(adev->dev, "Failed to allow df cstate");
1103		1054
1104	return ret;	1055	return 0;
1105	}	1056	}
1106		1057
1107	/* wrapper of psp_ras_trigger_error */	1058	/* wrapper of psp_ras_trigger_error */
@@ -1116,11 +1067,20 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 8 unmodified lines
1116	.address = info->address,	1067	.address = info->address,
1117	.value = info->value,	1068	.value = info->value,
1118	};	1069	};
1119	int ret = 0;	1070	int ret = -EINVAL;
		1071	struct amdgpu_ras_block_object *block_obj = amdgpu_ras_get_ras_block(adev,
		1072	info->head.block,
		1073	info->head.sub_block_index);
1120		1074
1121	if (!obj)	1075	if (!obj)
1122	return -EINVAL;	1076	return -EINVAL;
1123		1077
		1078	if (!block_obj \|\| !block_obj->hw_ops) {
		1079	dev_dbg_once(adev->dev, "%s doesn't config RAS function\n",
		1080	get_ras_block_str(&info->head));
		1081	return -EINVAL;
		1082	}
		1083
1124	/* Calculate XGMI relative offset */	1084	/* Calculate XGMI relative offset */
1125	if (adev->gmc.xgmi.num_physical_nodes > 1) {	1085	if (adev->gmc.xgmi.num_physical_nodes > 1) {
1126	block_info.address =	1086	block_info.address =
@@ -1128,28 +1088,15 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 1 unmodified line
1128	block_info.address);	1088	block_info.address);
1129	}	1089	}
1130		1090
1131	switch (info->head.block) {	1091	if (info->head.block == AMDGPU_RAS_BLOCK__GFX) {
1132	case AMDGPU_RAS_BLOCK__GFX:	1092	if (block_obj->hw_ops->ras_error_inject)
1133	if (adev->gfx.ras_funcs &&	1093	ret = block_obj->hw_ops->ras_error_inject(adev, info);
1134	adev->gfx.ras_funcs->ras_error_inject)	1094	} else {
1135	ret = adev->gfx.ras_funcs->ras_error_inject(adev, info);	1095	/* If defined special ras_error_inject(e.g: xgmi), implement special ras_error_inject */
1136	else	1096	if (block_obj->hw_ops->ras_error_inject)
1137	ret = -EINVAL;	1097	ret = block_obj->hw_ops->ras_error_inject(adev, &block_info);
1138	break;	1098	else /If not defined .ras_error_inject, use default ras_error_inject/
1139	case AMDGPU_RAS_BLOCK__UMC:	1099	ret = psp_ras_trigger_error(&adev->psp, &block_info);
1140	case AMDGPU_RAS_BLOCK__SDMA:
1141	case AMDGPU_RAS_BLOCK__MMHUB:
1142	case AMDGPU_RAS_BLOCK__PCIE_BIF:
1143	case AMDGPU_RAS_BLOCK__MCA:
1144	ret = psp_ras_trigger_error(&adev->psp, &block_info);
1145	break;
1146	case AMDGPU_RAS_BLOCK__XGMI_WAFL:
1147	ret = amdgpu_ras_error_inject_xgmi(adev, &block_info);
1148	break;
1149	default:
1150	dev_info(adev->dev, "%s error injection is not supported yet\n",
1151	get_ras_block_str(&info->head));
1152	ret = -EINVAL;
1153	}	1100	}
1154		1101
1155	if (ret)	1102	if (ret)
@@ -1329,18 +1276,17 @@ static int amdgpu_ras_sysfs_remove_feature_node(struct amdgpu_device *adev) · 173 unmodified lines
1329	}	1276	}
1330		1277
1331	int amdgpu_ras_sysfs_create(struct amdgpu_device *adev,	1278	int amdgpu_ras_sysfs_create(struct amdgpu_device *adev,
1332	struct ras_fs_if *head)	1279	struct ras_common_if *head)
1333	{	1280	{
1334	struct ras_manager *obj = amdgpu_ras_find_obj(adev, &head->head);	1281	struct ras_manager *obj = amdgpu_ras_find_obj(adev, head);
1335		1282
1336	if (!obj \|\| obj->attr_inuse)	1283	if (!obj \|\| obj->attr_inuse)
1337	return -EINVAL;	1284	return -EINVAL;
1338		1285
1339	get_obj(obj);	1286	get_obj(obj);
1340		1287
1341	memcpy(obj->fs_data.sysfs_name,	1288	snprintf(obj->fs_data.sysfs_name, sizeof(obj->fs_data.sysfs_name),
1342	head->sysfs_name,	1289	"%s_err_count", head->name);
1343	sizeof(obj->fs_data.sysfs_name));
1344		1290
1345	obj->sysfs_attr = (struct device_attribute){	1291	obj->sysfs_attr = (struct device_attribute){
1346	.attr = {	1292	.attr = {
@@ -1647,9 +1593,9 @@ int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, · 300 unmodified lines
1647	}	1593	}
1648		1594
1649	int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev,	1595	int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev,
1650	struct ras_ih_if *info)	1596	struct ras_common_if *head)
1651	{	1597	{
1652	struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head);	1598	struct ras_manager *obj = amdgpu_ras_find_obj(adev, head);
1653	struct ras_ih_data *data;	1599	struct ras_ih_data *data;
1654		1600
1655	if (!obj)	1601	if (!obj)
@@ -1669,24 +1615,27 @@ int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, · 13 unmodified lines
1669	}	1615	}
1670		1616
1671	int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev,	1617	int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev,
1672	struct ras_ih_if *info)	1618	struct ras_common_if *head)
1673	{	1619	{
1674	struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head);	1620	struct ras_manager *obj = amdgpu_ras_find_obj(adev, head);
1675	struct ras_ih_data *data;	1621	struct ras_ih_data *data;
		1622	struct amdgpu_ras_block_object *ras_obj;
1676		1623
1677	if (!obj) {	1624	if (!obj) {
1678	/* in case we registe the IH before enable ras feature */	1625	/* in case we registe the IH before enable ras feature */
1679	obj = amdgpu_ras_create_obj(adev, &info->head);	1626	obj = amdgpu_ras_create_obj(adev, head);
1680	if (!obj)	1627	if (!obj)
1681	return -EINVAL;	1628	return -EINVAL;
1682	} else	1629	} else
1683	get_obj(obj);	1630	get_obj(obj);
1684		1631
		1632	ras_obj = container_of(head, struct amdgpu_ras_block_object, ras_comm);
		1633
1685	data = &obj->ih_data;	1634	data = &obj->ih_data;
1686	/* add the callback.etc */	1635	/* add the callback.etc */
1687	*data = (struct ras_ih_data) {	1636	*data = (struct ras_ih_data) {
1688	.inuse = 0,	1637	.inuse = 0,
1689	.cb = info->cb,	1638	.cb = ras_obj->ras_cb,
1690	.element_size = sizeof(struct amdgpu_iv_entry),	1639	.element_size = sizeof(struct amdgpu_iv_entry),
1691	.rptr = 0,	1640	.rptr = 0,
1692	.wptr = 0,	1641	.wptr = 0,
@@ -1715,10 +1664,7 @@ static int amdgpu_ras_interrupt_remove_all(struct amdgpu_device *adev) · 22 unmodified lines
1715	struct ras_manager obj, tmp;	1664	struct ras_manager obj, tmp;
1716		1665
1717	list_for_each_entry_safe(obj, tmp, &con->head, node) {	1666	list_for_each_entry_safe(obj, tmp, &con->head, node) {
1718	struct ras_ih_if info = {	1667	amdgpu_ras_interrupt_remove_handler(adev, &obj->head);
1719	.head = obj->head,
1720	};
1721	amdgpu_ras_interrupt_remove_handler(adev, &info);
1722	}	1668	}
1723		1669
1724	return 0;	1670	return 0;
@@ -1766,24 +1712,28 @@ static void amdgpu_ras_log_on_err_counter(struct amdgpu_device *adev) · 41 unmodified lines
1766	static void amdgpu_ras_error_status_query(struct amdgpu_device *adev,	1712	static void amdgpu_ras_error_status_query(struct amdgpu_device *adev,
1767	struct ras_query_if *info)	1713	struct ras_query_if *info)
1768	{	1714	{
		1715	struct amdgpu_ras_block_object *block_obj;
1769	/*	1716	/*
1770	* Only two block need to query read/write	1717	* Only two block need to query read/write
1771	* RspStatus at current state	1718	* RspStatus at current state
1772	*/	1719	*/
1773	switch (info->head.block) {	1720	if ((info->head.block != AMDGPU_RAS_BLOCK__GFX) &&
1774	case AMDGPU_RAS_BLOCK__GFX:	1721	(info->head.block != AMDGPU_RAS_BLOCK__MMHUB))
1775	if (adev->gfx.ras_funcs &&	1722	return;
1776	adev->gfx.ras_funcs->query_ras_error_status)	1723
1777	adev->gfx.ras_funcs->query_ras_error_status(adev);	1724	block_obj = amdgpu_ras_get_ras_block(adev,
1778	break;	1725	info->head.block,
1779	case AMDGPU_RAS_BLOCK__MMHUB:	1726	info->head.sub_block_index);
1780	if (adev->mmhub.ras_funcs &&	1727
1781	adev->mmhub.ras_funcs->query_ras_error_status)	1728	if (!block_obj \|\| !block_obj->hw_ops) {
1782	adev->mmhub.ras_funcs->query_ras_error_status(adev);	1729	dev_dbg_once(adev->dev, "%s doesn't config RAS function\n",
1783	break;	1730	get_ras_block_str(&info->head));
1784	default:	1731	return;
1785	break;
1786	}	1732	}
		1733
		1734	if (block_obj->hw_ops->query_ras_error_status)
		1735	block_obj->hw_ops->query_ras_error_status(adev);
		1736
1787	}	1737	}
1788		1738
1789	static void amdgpu_ras_query_err_status(struct amdgpu_device *adev)	1739	static void amdgpu_ras_query_err_status(struct amdgpu_device *adev)
@@ -2141,8 +2091,7 @@ int amdgpu_ras_recovery_init(struct amdgpu_device *adev) · 351 unmodified lines
2141	if (ret)	2091	if (ret)
2142	goto free;	2092	goto free;
2143		2093
2144	if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num)	2094	amdgpu_dpm_send_hbm_bad_pages_num(adev, con->eeprom_control.ras_num_recs);
2145	adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs);
2146	}	2095	}
2147		2096
2148	#ifdef CONFIG_X86_MCE_AMD	2097	#ifdef CONFIG_X86_MCE_AMD
@@ -2348,24 +2297,27 @@ int amdgpu_ras_init(struct amdgpu_device *adev) · 199 unmodified lines
2348	case CHIP_VEGA20:	2297	case CHIP_VEGA20:
2349	case CHIP_ARCTURUS:	2298	case CHIP_ARCTURUS:
2350	case CHIP_ALDEBARAN:	2299	case CHIP_ALDEBARAN:
2351	if (!adev->gmc.xgmi.connected_to_cpu)	2300	if (!adev->gmc.xgmi.connected_to_cpu) {
2352	adev->nbio.ras_funcs = &nbio_v7_4_ras_funcs;	2301	adev->nbio.ras = &nbio_v7_4_ras;
		2302	amdgpu_ras_register_ras_block(adev, &adev->nbio.ras->ras_block);
		2303	adev->nbio.ras_if = &adev->nbio.ras->ras_block.ras_comm;
		2304	}
2353	break;	2305	break;
2354	default:	2306	default:
2355	/* nbio ras is not available */	2307	/* nbio ras is not available */
2356	break;	2308	break;
2357	}	2309	}
2358		2310
2359	if (adev->nbio.ras_funcs &&	2311	if (adev->nbio.ras &&
2360	adev->nbio.ras_funcs->init_ras_controller_interrupt) {	2312	adev->nbio.ras->init_ras_controller_interrupt) {
2361	r = adev->nbio.ras_funcs->init_ras_controller_interrupt(adev);	2313	r = adev->nbio.ras->init_ras_controller_interrupt(adev);
2362	if (r)	2314	if (r)
2363	goto release_con;	2315	goto release_con;
2364	}	2316	}
2365		2317
2366	if (adev->nbio.ras_funcs &&	2318	if (adev->nbio.ras &&
2367	adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt) {	2319	adev->nbio.ras->init_ras_err_event_athub_interrupt) {
2368	r = adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt(adev);	2320	r = adev->nbio.ras->init_ras_err_event_athub_interrupt(adev);
2369	if (r)	2321	if (r)
2370	goto release_con;	2322	goto release_con;
2371	}	2323	}
@@ -2377,12 +2329,12 @@ int amdgpu_ras_init(struct amdgpu_device *adev) · 5 unmodified lines
2377	}	2329	}
2378	else if (adev->df.funcs &&	2330	else if (adev->df.funcs &&
2379	adev->df.funcs->query_ras_poison_mode &&	2331	adev->df.funcs->query_ras_poison_mode &&
2380	adev->umc.ras_funcs &&	2332	adev->umc.ras &&
2381	adev->umc.ras_funcs->query_ras_poison_mode) {	2333	adev->umc.ras->query_ras_poison_mode) {
2382	df_poison =	2334	df_poison =
2383	adev->df.funcs->query_ras_poison_mode(adev);	2335	adev->df.funcs->query_ras_poison_mode(adev);
2384	umc_poison =	2336	umc_poison =
2385	adev->umc.ras_funcs->query_ras_poison_mode(adev);	2337	adev->umc.ras->query_ras_poison_mode(adev);
2386	/* Only poison is set in both DF and UMC, we can support it */	2338	/* Only poison is set in both DF and UMC, we can support it */
2387	if (df_poison && umc_poison)	2339	if (df_poison && umc_poison)
2388	con->poison_supported = true;	2340	con->poison_supported = true;
@@ -2445,11 +2397,10 @@ bool amdgpu_ras_is_poison_mode_supported(struct amdgpu_device *adev) · 56 unmodified lines
2445	}	2397	}
2446		2398
2447	/* helper function to handle common stuff in ip late init phase */	2399	/* helper function to handle common stuff in ip late init phase */
2448	int amdgpu_ras_late_init(struct amdgpu_device *adev,	2400	int amdgpu_ras_block_late_init(struct amdgpu_device *adev,
2449	struct ras_common_if *ras_block,	2401	struct ras_common_if *ras_block)
2450	struct ras_fs_if *fs_info,
2451	struct ras_ih_if *ih_info)
2452	{	2402	{
		2403	struct amdgpu_ras_block_object *ras_obj;
2453	struct amdgpu_ras *con = amdgpu_ras_get_context(adev);	2404	struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
2454	unsigned long ue_count, ce_count;	2405	unsigned long ue_count, ce_count;
2455	int r;	2406	int r;
@@ -2477,15 +2428,16 @@ int amdgpu_ras_late_init(struct amdgpu_device *adev, · 21 unmodified lines
2477	if (adev->in_suspend \|\| amdgpu_in_reset(adev))	2428	if (adev->in_suspend \|\| amdgpu_in_reset(adev))
2478	return 0;	2429	return 0;
2479		2430
2480	if (ih_info->cb) {	2431	ras_obj = container_of(ras_block, struct amdgpu_ras_block_object, ras_comm);
2481	r = amdgpu_ras_interrupt_add_handler(adev, ih_info);	2432	if (ras_obj->ras_cb) {
		2433	r = amdgpu_ras_interrupt_add_handler(adev, ras_block);
2482	if (r)	2434	if (r)
2483	goto interrupt;	2435	goto cleanup;
2484	}	2436	}
2485		2437
2486	r = amdgpu_ras_sysfs_create(adev, fs_info);	2438	r = amdgpu_ras_sysfs_create(adev, ras_block);
2487	if (r)	2439	if (r)
2488	goto sysfs;	2440	goto interrupt;
2489		2441
2490	/* Those are the cached values at init.	2442	/* Those are the cached values at init.
2491	*/	2443	*/
@@ -2495,27 +2447,34 @@ int amdgpu_ras_late_init(struct amdgpu_device *adev, · 3 unmodified lines
2495	}	2447	}
2496		2448
2497	return 0;	2449	return 0;
2498	cleanup:	2450
2499	amdgpu_ras_sysfs_remove(adev, ras_block);
2500	sysfs:
2501	if (ih_info->cb)
2502	amdgpu_ras_interrupt_remove_handler(adev, ih_info);
2503	interrupt:	2451	interrupt:
		2452	if (ras_obj->ras_cb)
		2453	amdgpu_ras_interrupt_remove_handler(adev, ras_block);
		2454	cleanup:
2504	amdgpu_ras_feature_enable(adev, ras_block, 0);	2455	amdgpu_ras_feature_enable(adev, ras_block, 0);
2505	return r;	2456	return r;
2506	}	2457	}
2507		2458
		2459	int amdgpu_ras_block_late_init_default(struct amdgpu_device *adev,
		2460	struct ras_common_if *ras_block)
		2461	{
		2462	return amdgpu_ras_block_late_init(adev, ras_block);
		2463	}
		2464
2508	/* helper function to remove ras fs node and interrupt handler */	2465	/* helper function to remove ras fs node and interrupt handler */
2509	void amdgpu_ras_late_fini(struct amdgpu_device *adev,	2466	void amdgpu_ras_block_late_fini(struct amdgpu_device *adev,
2510	struct ras_common_if *ras_block,	2467	struct ras_common_if *ras_block)
2511	struct ras_ih_if *ih_info)
2512	{	2468	{
2513	if (!ras_block \|\| !ih_info)	2469	struct amdgpu_ras_block_object *ras_obj;
		2470	if (!ras_block)
2514	return;	2471	return;
2515		2472
2516	amdgpu_ras_sysfs_remove(adev, ras_block);	2473	amdgpu_ras_sysfs_remove(adev, ras_block);
2517	if (ih_info->cb)	2474
2518	amdgpu_ras_interrupt_remove_handler(adev, ih_info);	2475	ras_obj = container_of(ras_block, struct amdgpu_ras_block_object, ras_comm);
		2476	if (ras_obj->ras_cb)
		2477	amdgpu_ras_interrupt_remove_handler(adev, ras_block);
2519	}	2478	}
2520		2479
2521	/* do some init work after IP late init as dependence.	2480	/* do some init work after IP late init as dependence.
@@ -2568,6 +2527,33 @@ void amdgpu_ras_suspend(struct amdgpu_device *adev) · 46 unmodified lines
2568	amdgpu_ras_disable_all_features(adev, 1);	2527	amdgpu_ras_disable_all_features(adev, 1);
2569	}	2528	}
2570		2529
		2530	int amdgpu_ras_late_init(struct amdgpu_device *adev)
		2531	{
		2532	struct amdgpu_ras_block_list node, tmp;
		2533	struct amdgpu_ras_block_object *obj;
		2534	int r;
		2535
		2536	list_for_each_entry_safe(node, tmp, &adev->ras_list, node) {
		2537	if (!node->ras_obj) {
		2538	dev_warn(adev->dev, "Warning: abnormal ras list node.\n");
		2539	continue;
		2540	}
		2541
		2542	obj = node->ras_obj;
		2543	if (obj->ras_late_init) {
		2544	r = obj->ras_late_init(adev, &obj->ras_comm);
		2545	if (r) {
		2546	dev_err(adev->dev, "%s failed to execute ras_late_init! ret:%d\n",
		2547	obj->ras_comm.name, r);
		2548	return r;
		2549	}
		2550	} else
		2551	amdgpu_ras_block_late_init_default(adev, &obj->ras_comm);
		2552	}
		2553
		2554	return 0;
		2555	}
		2556
2571	/* do some fini work before IP fini as dependence */	2557	/* do some fini work before IP fini as dependence */
2572	int amdgpu_ras_pre_fini(struct amdgpu_device *adev)	2558	int amdgpu_ras_pre_fini(struct amdgpu_device *adev)
2573	{	2559	{
@@ -2585,6 +2571,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) · 11 unmodified lines
2585		2571
2586	int amdgpu_ras_fini(struct amdgpu_device *adev)	2572	int amdgpu_ras_fini(struct amdgpu_device *adev)
2587	{	2573	{
		2574	struct amdgpu_ras_block_list ras_node, tmp;
2588	struct amdgpu_ras *con = amdgpu_ras_get_context(adev);	2575	struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
2589		2576
2590	if (!adev->ras_enabled \|\| !con)	2577	if (!adev->ras_enabled \|\| !con)
@@ -2603,6 +2590,12 @@ int amdgpu_ras_fini(struct amdgpu_device *adev) · 12 unmodified lines
2603	amdgpu_ras_set_context(adev, NULL);	2590	amdgpu_ras_set_context(adev, NULL);
2604	kfree(con);	2591	kfree(con);
2605		2592
		2593	/* Clear ras blocks from ras_list and free ras block list node */
		2594	list_for_each_entry_safe(ras_node, tmp, &adev->ras_list, node) {
		2595	list_del(&ras_node->node);
		2596	kfree(ras_node);
		2597	}
		2598
2606	return 0;	2599	return 0;
2607	}	2600	}
2608		2601
@@ -2717,8 +2710,6 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb, · 108 unmodified lines
2717	dev_info(adev->dev, "Uncorrectable error detected in UMC inst: %d, chan_idx: %d",	2710	dev_info(adev->dev, "Uncorrectable error detected in UMC inst: %d, chan_idx: %d",
2718	umc_inst, ch_inst);	2711	umc_inst, ch_inst);
2719		2712
2720	memset(&err_rec, 0x0, sizeof(struct eeprom_table_record));
2721
2722	/*	2713	/*
2723	* Translate UMC channel address to Physical address	2714	* Translate UMC channel address to Physical address
2724	*/	2715	*/
@@ -2730,16 +2721,10 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb, · 5 unmodified lines
2730	ADDR_OF_256B_BLOCK(channel_index) \|	2721	ADDR_OF_256B_BLOCK(channel_index) \|
2731	OFFSET_IN_256B_BLOCK(m->addr);	2722	OFFSET_IN_256B_BLOCK(m->addr);
2732		2723
2733	err_rec.address = m->addr;	2724	memset(&err_rec, 0x0, sizeof(struct eeprom_table_record));
2734	err_rec.retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;
2735	err_rec.ts = (uint64_t)ktime_get_real_seconds();
2736	err_rec.err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
2737	err_rec.cu = 0;
2738	err_rec.mem_channel = channel_index;
2739	err_rec.mcumc_id = umc_inst;
2740
2741	err_data.err_addr = &err_rec;	2725	err_data.err_addr = &err_rec;
2742	err_data.err_addr_cnt = 1;	2726	amdgpu_umc_fill_error_record(&err_data, m->addr,
		2727	retired_page, channel_index, umc_inst);
2743		2728
2744	if (amdgpu_bad_page_threshold != 0) {	2729	if (amdgpu_bad_page_threshold != 0) {
2745	amdgpu_ras_add_bad_pages(adev, err_data.err_addr,	2730	amdgpu_ras_add_bad_pages(adev, err_data.err_addr,
@@ -2777,3 +2762,63 @@ static void amdgpu_register_bad_pages_mca_notifier(struct amdgpu_device *adev) · 31 unmodified lines
2777	}	2762	}
2778	}	2763	}
2779	#endif	2764	#endif
		2765
		2766	struct amdgpu_ras amdgpu_ras_get_context(struct amdgpu_device adev)
		2767	{
		2768	if (!adev)
		2769	return NULL;
		2770
		2771	return adev->psp.ras_context.ras;
		2772	}
		2773
		2774	int amdgpu_ras_set_context(struct amdgpu_device adev, struct amdgpu_ras ras_con)
		2775	{
		2776	if (!adev)
		2777	return -EINVAL;
		2778
		2779	adev->psp.ras_context.ras = ras_con;
		2780	return 0;
		2781	}
		2782
		2783	/* check if ras is supported on block, say, sdma, gfx */
		2784	int amdgpu_ras_is_supported(struct amdgpu_device *adev,
		2785	unsigned int block)
		2786	{
		2787	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
		2788
		2789	if (block >= AMDGPU_RAS_BLOCK_COUNT)
		2790	return 0;
		2791	return ras && (adev->ras_enabled & (1 << block));
		2792	}
		2793
		2794	int amdgpu_ras_reset_gpu(struct amdgpu_device *adev)
		2795	{
		2796	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
		2797
		2798	if (atomic_cmpxchg(&ras->in_recovery, 0, 1) == 0)
		2799	schedule_work(&ras->recovery_work);
		2800	return 0;
		2801	}
		2802
		2803
		2804	/* Register each ip ras block into amdgpu ras */
		2805	int amdgpu_ras_register_ras_block(struct amdgpu_device *adev,
		2806	struct amdgpu_ras_block_object *ras_block_obj)
		2807	{
		2808	struct amdgpu_ras_block_list *ras_node;
		2809	if (!adev \|\| !ras_block_obj)
		2810	return -EINVAL;
		2811
		2812	if (!amdgpu_ras_asic_supported(adev))
		2813	return 0;
		2814
		2815	ras_node = kzalloc(sizeof(*ras_node), GFP_KERNEL);
		2816	if (!ras_node)
		2817	return -ENOMEM;
		2818
		2819	INIT_LIST_HEAD(&ras_node->node);
		2820	ras_node->ras_obj = ras_block_obj;
		2821	list_add_tail(&ras_node->node, &adev->ras_list);
		2822
		2823	return 0;
		2824	}

drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h+43−34

@@ -26,11 +26,11 @@ · 25 unmodified lines
26		26
27	#include <linux/debugfs.h>	27	#include <linux/debugfs.h>
28	#include <linux/list.h>	28	#include <linux/list.h>
29	#include "amdgpu.h"
30	#include "amdgpu_psp.h"
31	#include "ta_ras_if.h"	29	#include "ta_ras_if.h"
32	#include "amdgpu_ras_eeprom.h"	30	#include "amdgpu_ras_eeprom.h"
33		31
		32	struct amdgpu_iv_entry;
		33
34	#define AMDGPU_RAS_FLAG_INIT_BY_VBIOS (0x1 << 0)	34	#define AMDGPU_RAS_FLAG_INIT_BY_VBIOS (0x1 << 0)
35		35
36	enum amdgpu_ras_block {	36	enum amdgpu_ras_block {
@@ -484,6 +484,27 @@ struct ras_debug_if { · 447 unmodified lines
484	};	484	};
485	int op;	485	int op;
486	};	486	};
		487
		488	struct amdgpu_ras_block_object {
		489	struct ras_common_if ras_comm;
		490
		491	int (ras_block_match)(struct amdgpu_ras_block_object block_obj,
		492	enum amdgpu_ras_block block, uint32_t sub_block_index);
		493	int (ras_late_init)(struct amdgpu_device adev, struct ras_common_if *ras_block);
		494	void (ras_fini)(struct amdgpu_device adev);
		495	ras_ih_cb ras_cb;
		496	const struct amdgpu_ras_block_hw_ops *hw_ops;
		497	};
		498
		499	struct amdgpu_ras_block_hw_ops {
		500	int (ras_error_inject)(struct amdgpu_device adev, void *inject_if);
		501	void (query_ras_error_count)(struct amdgpu_device adev, void *ras_error_status);
		502	void (query_ras_error_status)(struct amdgpu_device adev);
		503	void (query_ras_error_address)(struct amdgpu_device adev, void *ras_error_status);
		504	void (reset_ras_error_count)(struct amdgpu_device adev);
		505	void (reset_ras_error_status)(struct amdgpu_device adev);
		506	};
		507
487	/* work flow	508	/* work flow
488	* vbios	509	* vbios
489	* 1: ras feature enable (enabled by default)	510	* 1: ras feature enable (enabled by default)
@@ -498,19 +519,6 @@ struct ras_debug_if { · 8 unmodified lines
498	* 8: feature disable	519	* 8: feature disable
499	*/	520	*/
500		521
501	#define amdgpu_ras_get_context(adev) ((adev)->psp.ras_context.ras)
502	#define amdgpu_ras_set_context(adev, ras_con) ((adev)->psp.ras_context.ras = (ras_con))
503
504	/* check if ras is supported on block, say, sdma, gfx */
505	static inline int amdgpu_ras_is_supported(struct amdgpu_device *adev,
506	unsigned int block)
507	{
508	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
509
510	if (block >= AMDGPU_RAS_BLOCK_COUNT)
511	return 0;
512	return ras && (adev->ras_enabled & (1 << block));
513	}
514		522
515	int amdgpu_ras_recovery_init(struct amdgpu_device *adev);	523	int amdgpu_ras_recovery_init(struct amdgpu_device *adev);
516		524
@@ -527,15 +535,6 @@ int amdgpu_ras_add_bad_pages(struct amdgpu_device *adev, · 10 unmodified lines
527		535
528	int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev);	536	int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev);
529		537
530	static inline int amdgpu_ras_reset_gpu(struct amdgpu_device *adev)
531	{
532	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
533
534	if (atomic_cmpxchg(&ras->in_recovery, 0, 1) == 0)
535	schedule_work(&ras->recovery_work);
536	return 0;
537	}
538
539	static inline enum ta_ras_block	538	static inline enum ta_ras_block
540	amdgpu_ras_block_to_ta(enum amdgpu_ras_block block) {	539	amdgpu_ras_block_to_ta(enum amdgpu_ras_block block) {
541	switch (block) {	540	switch (block) {
@@ -596,15 +595,15 @@ amdgpu_ras_error_to_ta(enum amdgpu_ras_error_type error) { · 54 unmodified lines
596		595
597	/* called in ip_init and ip_fini */	596	/* called in ip_init and ip_fini */
598	int amdgpu_ras_init(struct amdgpu_device *adev);	597	int amdgpu_ras_init(struct amdgpu_device *adev);
		598	int amdgpu_ras_late_init(struct amdgpu_device *adev);
599	int amdgpu_ras_fini(struct amdgpu_device *adev);	599	int amdgpu_ras_fini(struct amdgpu_device *adev);
600	int amdgpu_ras_pre_fini(struct amdgpu_device *adev);	600	int amdgpu_ras_pre_fini(struct amdgpu_device *adev);
601	int amdgpu_ras_late_init(struct amdgpu_device *adev,	601
602	struct ras_common_if *ras_block,	602	int amdgpu_ras_block_late_init(struct amdgpu_device *adev,
603	struct ras_fs_if *fs_info,	603	struct ras_common_if *ras_block);
604	struct ras_ih_if *ih_info);	604
605	void amdgpu_ras_late_fini(struct amdgpu_device *adev,	605	void amdgpu_ras_block_late_fini(struct amdgpu_device *adev,
606	struct ras_common_if *ras_block,	606	struct ras_common_if *ras_block);
607	struct ras_ih_if *ih_info);
608		607
609	int amdgpu_ras_feature_enable(struct amdgpu_device *adev,	608	int amdgpu_ras_feature_enable(struct amdgpu_device *adev,
610	struct ras_common_if *head, bool enable);	609	struct ras_common_if *head, bool enable);
@@ -613,7 +612,7 @@ int amdgpu_ras_feature_enable_on_boot(struct amdgpu_device *adev, · 2 unmodified lines
613	struct ras_common_if *head, bool enable);	612	struct ras_common_if *head, bool enable);
614		613
615	int amdgpu_ras_sysfs_create(struct amdgpu_device *adev,	614	int amdgpu_ras_sysfs_create(struct amdgpu_device *adev,
616	struct ras_fs_if *head);	615	struct ras_common_if *head);
617		616
618	int amdgpu_ras_sysfs_remove(struct amdgpu_device *adev,	617	int amdgpu_ras_sysfs_remove(struct amdgpu_device *adev,
619	struct ras_common_if *head);	618	struct ras_common_if *head);
@@ -630,10 +629,10 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 10 unmodified lines
630	struct ras_inject_if *info);	629	struct ras_inject_if *info);
631		630
632	int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev,	631	int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev,
633	struct ras_ih_if *info);	632	struct ras_common_if *head);
634		633
635	int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev,	634	int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev,
636	struct ras_ih_if *info);	635	struct ras_common_if *head);
637		636
638	int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev,	637	int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev,
639	struct ras_dispatch_if *info);	638	struct ras_dispatch_if *info);
@@ -667,4 +666,14 @@ const char get_ras_block_str(struct ras_common_if ras_block); · 27 unmodified lines
667		666
668	bool amdgpu_ras_is_poison_mode_supported(struct amdgpu_device *adev);	667	bool amdgpu_ras_is_poison_mode_supported(struct amdgpu_device *adev);
669		668
		669	int amdgpu_ras_is_supported(struct amdgpu_device *adev, unsigned int block);
		670
		671	int amdgpu_ras_reset_gpu(struct amdgpu_device *adev);
		672
		673	struct amdgpu_ras* amdgpu_ras_get_context(struct amdgpu_device *adev);
		674
		675	int amdgpu_ras_set_context(struct amdgpu_device adev, struct amdgpu_ras ras_con);
		676
		677	int amdgpu_ras_register_ras_block(struct amdgpu_device *adev,
		678	struct amdgpu_ras_block_object *ras_block_obj);
670	#endif	679	#endif

drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c+20−15

 #include <linux/debugfs.h>
 #include <linux/uaccess.h>
+#include "amdgpu_reset.h"
 #define EEPROM_I2C_MADDR_VEGA20         0x0
 #define EEPROM_I2C_MADDR_ARCTURUS       0x40000
 #define EEPROM_I2C_MADDR_ARCTURUS_D342  0x0
 	__encode_table_header_to_buf(&control->tbl_hdr, buf);
 	/* i2c may be unstable in gpu reset */
-	down_read(&adev->reset_sem);
+	down_read(&adev->reset_domain->sem);
-	res = amdgpu_eeprom_write(&adev->pm.smu_i2c,
+	res = amdgpu_eeprom_write(adev->pm.ras_eeprom_i2c_bus,
 				  control->i2c_address +
 				  control->ras_header_offset,
 				  buf, RAS_TABLE_HEADER_SIZE);
-	up_read(&adev->reset_sem);
+	up_read(&adev->reset_domain->sem);
 	if (res < 0) {
 		DRM_ERROR("Failed to write EEPROM table header:%d", res);
  */
 int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control)
 {
+	struct amdgpu_device *adev = to_amdgpu_device(control);
 	struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr;
 	u8 csum;
 	int res;
 	control->ras_num_recs = 0;
 	control->ras_fri = 0;
+	amdgpu_dpm_send_hbm_bad_pages_num(adev, control->ras_num_recs);
 	amdgpu_ras_debugfs_set_ret_size(control);
 	mutex_unlock(&control->ras_tbl_mutex);
 	int res;
 	/* i2c may be unstable in gpu reset */
-	down_read(&adev->reset_sem);
+	down_read(&adev->reset_domain->sem);
 	buf_size = num * RAS_TABLE_RECORD_SIZE;
-	res = amdgpu_eeprom_write(&adev->pm.smu_i2c,
+	res = amdgpu_eeprom_write(adev->pm.ras_eeprom_i2c_bus,
 				  control->i2c_address +
 				  RAS_INDEX_TO_OFFSET(control, fri),
 				  buf, buf_size);
-	up_read(&adev->reset_sem);
+	up_read(&adev->reset_domain->sem);
 	if (res < 0) {
 		DRM_ERROR("Writing %d EEPROM table records error:%d",
 			  num, res);
 		goto Out;
 	}
-	down_read(&adev->reset_sem);
+	down_read(&adev->reset_domain->sem);
-	res = amdgpu_eeprom_read(&adev->pm.smu_i2c,
+	res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus,
 				 control->i2c_address +
 				 control->ras_record_offset,
 				 buf, buf_size);
-	up_read(&adev->reset_sem);
+	up_read(&adev->reset_domain->sem);
 	if (res < 0) {
 		DRM_ERROR("EEPROM failed reading records:%d\n",
 			  res);
 	int res;
 	/* i2c may be unstable in gpu reset */
-	down_read(&adev->reset_sem);
+	down_read(&adev->reset_domain->sem);
 	buf_size = num * RAS_TABLE_RECORD_SIZE;
-	res = amdgpu_eeprom_read(&adev->pm.smu_i2c,
+	res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus,
 				 control->i2c_address +
 				 RAS_INDEX_TO_OFFSET(control, fri),
 				 buf, buf_size);
-	up_read(&adev->reset_sem);
+	up_read(&adev->reset_domain->sem);
 	if (res < 0) {
 		DRM_ERROR("Reading %d EEPROM table records error:%d",
 			  num, res);
 		return -ENOMEM;
 	}
-	res = amdgpu_eeprom_read(&adev->pm.smu_i2c,
+	res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus,
 				 control->i2c_address +
 				 control->ras_header_offset,
 				 buf, buf_size);
 		return 0;
 	/* Verify i2c adapter is initialized */
-	if (!adev->pm.smu_i2c.algo)
+	if (!adev->pm.ras_eeprom_i2c_bus || !adev->pm.ras_eeprom_i2c_bus->algo)
 		return -ENOENT;
 	if (!__get_eeprom_i2c_addr(adev, control))
 	mutex_init(&control->ras_tbl_mutex);
 	/* Read the table header from EEPROM address */
-	res = amdgpu_eeprom_read(&adev->pm.smu_i2c,
+	res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus,
 				 control->i2c_address + control->ras_header_offset,
 				 buf, RAS_TABLE_HEADER_SIZE);
 	if (res < RAS_TABLE_HEADER_SIZE) {

drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c+56−0

@@ -96,3 +96,59 @@ int amdgpu_reset_perform_reset(struct amdgpu_device *adev, · 95 unmodified lines
96	return reset_handler->restore_hwcontext(adev->reset_cntl,	96	return reset_handler->restore_hwcontext(adev->reset_cntl,
97	reset_context);	97	reset_context);
98	}	98	}
		99
		100
		101	void amdgpu_reset_destroy_reset_domain(struct kref *ref)
		102	{
		103	struct amdgpu_reset_domain *reset_domain = container_of(ref,
		104	struct amdgpu_reset_domain,
		105	refcount);
		106	if (reset_domain->wq)
		107	destroy_workqueue(reset_domain->wq);
		108
		109	kvfree(reset_domain);
		110	}
		111
		112	struct amdgpu_reset_domain *amdgpu_reset_create_reset_domain(enum amdgpu_reset_domain_type type,
		113	char *wq_name)
		114	{
		115	struct amdgpu_reset_domain *reset_domain;
		116
		117	reset_domain = kvzalloc(sizeof(struct amdgpu_reset_domain), GFP_KERNEL);
		118	if (!reset_domain) {
		119	DRM_ERROR("Failed to allocate amdgpu_reset_domain!");
		120	return NULL;
		121	}
		122
		123	reset_domain->type = type;
		124	kref_init(&reset_domain->refcount);
		125
		126	reset_domain->wq = create_singlethread_workqueue(wq_name);
		127	if (!reset_domain->wq) {
		128	DRM_ERROR("Failed to allocate wq for amdgpu_reset_domain!");
		129	amdgpu_reset_put_reset_domain(reset_domain);
		130	return NULL;
		131
		132	}
		133
		134	atomic_set(&reset_domain->in_gpu_reset, 0);
		135	init_rwsem(&reset_domain->sem);
		136
		137	return reset_domain;
		138	}
		139
		140	void amdgpu_device_lock_reset_domain(struct amdgpu_reset_domain *reset_domain)
		141	{
		142	atomic_set(&reset_domain->in_gpu_reset, 1);
		143	down_write(&reset_domain->sem);
		144	}
		145
		146
		147	void amdgpu_device_unlock_reset_domain(struct amdgpu_reset_domain *reset_domain)
		148	{
		149	atomic_set(&reset_domain->in_gpu_reset, 0);
		150	up_write(&reset_domain->sem);
		151	}
		152
		153
		154

drivers/gpu/drm/amd/amdgpu/amdgpu_reset.h+40−0

@@ -70,6 +70,21 @@ struct amdgpu_reset_control { · 69 unmodified lines
70	void (async_reset)(struct work_struct work);	70	void (async_reset)(struct work_struct work);
71	};	71	};
72		72
		73
		74	enum amdgpu_reset_domain_type {
		75	SINGLE_DEVICE,
		76	XGMI_HIVE
		77	};
		78
		79	struct amdgpu_reset_domain {
		80	struct kref refcount;
		81	struct workqueue_struct *wq;
		82	enum amdgpu_reset_domain_type type;
		83	struct rw_semaphore sem;
		84	atomic_t in_gpu_reset;
		85	};
		86
		87
73	int amdgpu_reset_init(struct amdgpu_device *adev);	88	int amdgpu_reset_init(struct amdgpu_device *adev);
74	int amdgpu_reset_fini(struct amdgpu_device *adev);	89	int amdgpu_reset_fini(struct amdgpu_device *adev);
75		90
@@ -82,4 +97,29 @@ int amdgpu_reset_perform_reset(struct amdgpu_device *adev, · 6 unmodified lines
82	int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl,	97	int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl,
83	struct amdgpu_reset_handler *handler);	98	struct amdgpu_reset_handler *handler);
84		99
		100	struct amdgpu_reset_domain *amdgpu_reset_create_reset_domain(enum amdgpu_reset_domain_type type,
		101	char *wq_name);
		102
		103	void amdgpu_reset_destroy_reset_domain(struct kref *ref);
		104
		105	static inline bool amdgpu_reset_get_reset_domain(struct amdgpu_reset_domain *domain)
		106	{
		107	return kref_get_unless_zero(&domain->refcount) != 0;
		108	}
		109
		110	static inline void amdgpu_reset_put_reset_domain(struct amdgpu_reset_domain *domain)
		111	{
		112	kref_put(&domain->refcount, amdgpu_reset_destroy_reset_domain);
		113	}
		114
		115	static inline bool amdgpu_reset_domain_schedule(struct amdgpu_reset_domain *domain,
		116	struct work_struct *work)
		117	{
		118	return queue_work(domain->wq, work);
		119	}
		120
		121	void amdgpu_device_lock_reset_domain(struct amdgpu_reset_domain *reset_domain);
		122
		123	void amdgpu_device_unlock_reset_domain(struct amdgpu_reset_domain *reset_domain);
		124
85	#endif	125	#endif

drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c+3−2

 		ring->adev = adev;
 		ring->idx = adev->num_rings++;
 		adev->rings[ring->idx] = ring;
-		r = amdgpu_fence_driver_init_ring(ring, sched_hw_submission,
+		ring->num_hw_submission = sched_hw_submission;
-						  sched_score);
+		ring->sched_score = sched_score;
+		r = amdgpu_fence_driver_init_ring(ring);
 		if (r)
 			return r;
 	}

drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h+3−3

 void amdgpu_fence_driver_clear_job_fences(struct amdgpu_ring *ring);
 void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
-int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
+int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring);
-				  unsigned num_hw_submission,
-				  atomic_t *sched_score);
 int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring,
 				   struct amdgpu_irq_src *irq_src,
 				   unsigned irq_type);
 	bool			has_compute_vm_bug;
 	bool			no_scheduler;
 	int			hw_prio;
+	unsigned 		num_hw_submission;
+	atomic_t		*sched_score;
 };
 #define amdgpu_ring_parse_cs(r, p, ib) ((r)->funcs->parse_cs((p), (ib)))

drivers/gpu/drm/amd/amdgpu/amdgpu_rlc.h+14−2

 	void (*reset)(struct amdgpu_device *adev);
 	void (*start)(struct amdgpu_device *adev);
 	void (*update_spm_vmid)(struct amdgpu_device *adev, unsigned vmid);
-	void (*sriov_wreg)(struct amdgpu_device *adev, u32 offset, u32 v, u32 acc_flags, u32 hwip);
-	u32 (*sriov_rreg)(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip);
 	bool (*is_rlcg_access_range)(struct amdgpu_device *adev, uint32_t reg);
 };
+struct amdgpu_rlcg_reg_access_ctrl {
+	uint32_t scratch_reg0;
+	uint32_t scratch_reg1;
+	uint32_t scratch_reg2;
+	uint32_t scratch_reg3;
+	uint32_t grbm_cntl;
+	uint32_t grbm_idx;
+	uint32_t spare_int;
+};
 struct amdgpu_rlc {
 	/* for power gating */
 	struct amdgpu_bo        *save_restore_obj;
 	struct amdgpu_bo	*rlc_toc_bo;
 	uint64_t		rlc_toc_gpu_addr;
 	void			*rlc_toc_buf;
+	bool rlcg_reg_access_supported;
+	/* registers for rlcg indirect reg access */
+	struct amdgpu_rlcg_reg_access_ctrl reg_access_ctrl;
 };
 void amdgpu_gfx_rlc_enter_safe_mode(struct amdgpu_device *adev);

drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.c+7−43

@@ -87,71 +87,35 @@ uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, · 86 unmodified lines
87	}	87	}
88		88
89	int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev,	89	int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev,
90	void *ras_ih_info)	90	struct ras_common_if *ras_block)
91	{	91	{
92	int r, i;	92	int r, i;
93	struct ras_ih_if ih_info = (struct ras_ih_if )ras_ih_info;
94	struct ras_fs_if fs_info = {
95	.sysfs_name = "sdma_err_count",
96	};
97
98	if (!ih_info)
99	return -EINVAL;
100		93
101	if (!adev->sdma.ras_if) {	94	r = amdgpu_ras_block_late_init(adev, ras_block);
102	adev->sdma.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
103	if (!adev->sdma.ras_if)
104	return -ENOMEM;
105	adev->sdma.ras_if->block = AMDGPU_RAS_BLOCK__SDMA;
106	adev->sdma.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
107	adev->sdma.ras_if->sub_block_index = 0;
108	}
109	fs_info.head = ih_info->head = *adev->sdma.ras_if;
110
111	r = amdgpu_ras_late_init(adev, adev->sdma.ras_if,
112	&fs_info, ih_info);
113	if (r)	95	if (r)
114	goto free;	96	return r;
115		97
116	if (amdgpu_ras_is_supported(adev, adev->sdma.ras_if->block)) {	98	if (amdgpu_ras_is_supported(adev, ras_block->block)) {
117	for (i = 0; i < adev->sdma.num_instances; i++) {	99	for (i = 0; i < adev->sdma.num_instances; i++) {
118	r = amdgpu_irq_get(adev, &adev->sdma.ecc_irq,	100	r = amdgpu_irq_get(adev, &adev->sdma.ecc_irq,
119	AMDGPU_SDMA_IRQ_INSTANCE0 + i);	101	AMDGPU_SDMA_IRQ_INSTANCE0 + i);
120	if (r)	102	if (r)
121	goto late_fini;	103	goto late_fini;
122	}	104	}
123	} else {
124	r = 0;
125	goto free;
126	}	105	}
127		106
128	return 0;	107	return 0;
129		108
130	late_fini:	109	late_fini:
131	amdgpu_ras_late_fini(adev, adev->sdma.ras_if, ih_info);	110	amdgpu_ras_block_late_fini(adev, ras_block);
132	free:
133	kfree(adev->sdma.ras_if);
134	adev->sdma.ras_if = NULL;
135	return r;	111	return r;
136	}	112	}
137		113
138	void amdgpu_sdma_ras_fini(struct amdgpu_device *adev)	114	void amdgpu_sdma_ras_fini(struct amdgpu_device *adev)
139	{	115	{
140	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__SDMA) &&	116	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__SDMA) &&
141	adev->sdma.ras_if) {	117	adev->sdma.ras_if)
142	struct ras_common_if *ras_if = adev->sdma.ras_if;	118	amdgpu_ras_block_late_fini(adev, adev->sdma.ras_if);
143	struct ras_ih_if ih_info = {
144	.head = *ras_if,
145	/* the cb member will not be used by
146	* amdgpu_ras_interrupt_remove_handler, init it only
147	* to cheat the check in ras_late_fini
148	*/
149	.cb = amdgpu_sdma_process_ras_data_cb,
150	};
151
152	amdgpu_ras_late_fini(adev, ras_if, &ih_info);
153	kfree(ras_if);
154	}
155	}	119	}
156		120
157	int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev,	121	int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev,

drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h+5−9

 #ifndef __AMDGPU_SDMA_H__
 #define __AMDGPU_SDMA_H__
+#include "amdgpu_ras.h"
 /* max number of IP instances */
 #define AMDGPU_MAX_SDMA_INSTANCES		8
 	bool			burst_nop;
 };
-struct amdgpu_sdma_ras_funcs {
+struct amdgpu_sdma_ras {
-	int (*ras_late_init)(struct amdgpu_device *adev,
+	struct amdgpu_ras_block_object ras_block;
-			void *ras_ih_info);
-	void (*ras_fini)(struct amdgpu_device *adev);
-	int (*query_ras_error_count)(struct amdgpu_device *adev,
-			uint32_t instance, void *ras_error_status);
-	void (*reset_ras_error_count)(struct amdgpu_device *adev);
 };
 struct amdgpu_sdma {
 	uint32_t                    srbm_soft_reset;
 	bool			has_page_queue;
 	struct ras_common_if	*ras_if;
-	const struct amdgpu_sdma_ras_funcs	*funcs;
+	struct amdgpu_sdma_ras	*ras;
 };
 /*
 int amdgpu_sdma_get_index_from_ring(struct amdgpu_ring *ring, uint32_t *index);
 uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, unsigned vmid);
 int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev,
-			      void *ras_ih_info);
+			      struct ras_common_if *ras_block);
 void amdgpu_sdma_ras_fini(struct amdgpu_device *adev);
 int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev,
 		void *err_data,

drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c+2−3

 	dma_resv_for_each_fence(&cursor, resv, true, f) {
 		dma_fence_chain_for_each(f, f) {
-			struct dma_fence_chain *chain = to_dma_fence_chain(f);
+			struct dma_fence *tmp = dma_fence_chain_contained(f);
-			if (amdgpu_sync_test_fence(adev, mode, owner, chain ?
+			if (amdgpu_sync_test_fence(adev, mode, owner, tmp)) {
-						   chain->fence : f)) {
 				r = amdgpu_sync_fence(sync, f);
 				dma_fence_put(f);
 				if (r)

drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h+2−3

 			}
 	),
 	TP_printk("pid:%u vm_ctx:0x%llx start:0x%010llx end:0x%010llx,"
-		  " flags:0x%llx, incr:%llu, dst:\n%s%s", __entry->pid,
+		  " flags:0x%llx, incr:%llu, dst:\n%s", __entry->pid,
 		  __entry->vm_ctx, __entry->start, __entry->end,
 		  __entry->flags, __entry->incr,  __print_array(
-		  __get_dynamic_array(dst), min(__entry->nptes, 32u), 8),
+		  __get_dynamic_array(dst), __entry->nptes, 8))
-		  __entry->nptes > 32 ? "..." : "")
 );
 TRACE_EVENT(amdgpu_vm_set_ptes,

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c+246−165

@@ -50,6 +50,7 @@ · 49 unmodified lines
50	#include <drm/ttm/ttm_range_manager.h>	50	#include <drm/ttm/ttm_range_manager.h>
51		51
52	#include <drm/amdgpu_drm.h>	52	#include <drm/amdgpu_drm.h>
		53	#include <drm/drm_drv.h>
53		54
54	#include "amdgpu.h"	55	#include "amdgpu.h"
55	#include "amdgpu_object.h"	56	#include "amdgpu_object.h"
@@ -170,10 +171,10 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · 114 unmodified lines
170	* @bo: buffer object to map	171	* @bo: buffer object to map
171	* @mem: memory object to map	172	* @mem: memory object to map
172	* @mm_cur: range to map	173	* @mm_cur: range to map
173	* @num_pages: number of pages to map
174	* @window: which GART window to use	174	* @window: which GART window to use
175	* @ring: DMA ring to use for the copy	175	* @ring: DMA ring to use for the copy
176	* @tmz: if we should setup a TMZ enabled mapping	176	* @tmz: if we should setup a TMZ enabled mapping
		177	* @size: in number of bytes to map, out number of bytes mapped
177	* @addr: resulting address inside the MC address space	178	* @addr: resulting address inside the MC address space
178	*	179	*
179	* Setup one of the GART windows to access a specific piece of memory or return	180	* Setup one of the GART windows to access a specific piece of memory or return
@@ -182,15 +183,14 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · 2 unmodified lines
182	static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo,	183	static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo,
183	struct ttm_resource *mem,	184	struct ttm_resource *mem,
184	struct amdgpu_res_cursor *mm_cur,	185	struct amdgpu_res_cursor *mm_cur,
185	unsigned num_pages, unsigned window,	186	unsigned window, struct amdgpu_ring *ring,
186	struct amdgpu_ring *ring, bool tmz,	187	bool tmz, uint64_t size, uint64_t addr)
187	uint64_t *addr)
188	{	188	{
189	struct amdgpu_device *adev = ring->adev;	189	struct amdgpu_device *adev = ring->adev;
190	struct amdgpu_job *job;	190	unsigned offset, num_pages, num_dw, num_bytes;
191	unsigned num_dw, num_bytes;
192	struct dma_fence *fence;
193	uint64_t src_addr, dst_addr;	191	uint64_t src_addr, dst_addr;
		192	struct dma_fence *fence;
		193	struct amdgpu_job *job;
194	void *cpu_addr;	194	void *cpu_addr;
195	uint64_t flags;	195	uint64_t flags;
196	unsigned int i;	196	unsigned int i;
@@ -198,7 +198,9 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 1 unmodified line
198		198
199	BUG_ON(adev->mman.buffer_funcs->copy_max_bytes <	199	BUG_ON(adev->mman.buffer_funcs->copy_max_bytes <
200	AMDGPU_GTT_MAX_TRANSFER_SIZE * 8);	200	AMDGPU_GTT_MAX_TRANSFER_SIZE * 8);
201	BUG_ON(mem->mem_type == AMDGPU_PL_PREEMPT);	201
		202	if (WARN_ON(mem->mem_type == AMDGPU_PL_PREEMPT))
		203	return -EINVAL;
202		204
203	/* Map only what can't be accessed directly */	205	/* Map only what can't be accessed directly */
204	if (!tmz && mem->start != AMDGPU_BO_INVALID_OFFSET) {	206	if (!tmz && mem->start != AMDGPU_BO_INVALID_OFFSET) {
@@ -207,10 +209,22 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 2 unmodified lines
207	return 0;	209	return 0;
208	}	210	}
209		211
		212
		213	/*
		214	* If start begins at an offset inside the page, then adjust the size
		215	* and addr accordingly
		216	*/
		217	offset = mm_cur->start & ~PAGE_MASK;
		218
		219	num_pages = PFN_UP(*size + offset);
		220	num_pages = min_t(uint32_t, num_pages, AMDGPU_GTT_MAX_TRANSFER_SIZE);
		221
		222	size = min(size, (uint64_t)num_pages * PAGE_SIZE - offset);
		223
210	*addr = adev->gmc.gart_start;	224	*addr = adev->gmc.gart_start;
211	addr += (u64)window AMDGPU_GTT_MAX_TRANSFER_SIZE *	225	addr += (u64)window AMDGPU_GTT_MAX_TRANSFER_SIZE *
212	AMDGPU_GPU_PAGE_SIZE;	226	AMDGPU_GPU_PAGE_SIZE;
213	*addr += mm_cur->start & ~PAGE_MASK;	227	*addr += offset;
214		228
215	num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);	229	num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
216	num_bytes = num_pages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE;	230	num_bytes = num_pages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE;
@@ -241,10 +255,7 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 24 unmodified lines
241	dma_addr_t *dma_addr;	255	dma_addr_t *dma_addr;
242		256
243	dma_addr = &bo->ttm->dma_address[mm_cur->start >> PAGE_SHIFT];	257	dma_addr = &bo->ttm->dma_address[mm_cur->start >> PAGE_SHIFT];
244	r = amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags,	258	amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags, cpu_addr);
245	cpu_addr);
246	if (r)
247	goto error_free;
248	} else {	259	} else {
249	dma_addr_t dma_address;	260	dma_addr_t dma_address;
250		261
@@ -252,11 +263,8 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 1 unmodified line
252	dma_address += adev->vm_manager.vram_base_offset;	263	dma_address += adev->vm_manager.vram_base_offset;
253		264
254	for (i = 0; i < num_pages; ++i) {	265	for (i = 0; i < num_pages; ++i) {
255	r = amdgpu_gart_map(adev, i << PAGE_SHIFT, 1,	266	amdgpu_gart_map(adev, i << PAGE_SHIFT, 1, &dma_address,
256	&dma_address, flags, cpu_addr);	267	flags, cpu_addr);
257	if (r)
258	goto error_free;
259
260	dma_address += PAGE_SIZE;	268	dma_address += PAGE_SIZE;
261	}	269	}
262	}	270	}
@@ -297,9 +305,6 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev, · 34 unmodified lines
297	struct dma_resv *resv,	305	struct dma_resv *resv,
298	struct dma_fence **f)	306	struct dma_fence **f)
299	{	307	{
300	const uint32_t GTT_MAX_BYTES = (AMDGPU_GTT_MAX_TRANSFER_SIZE *
301	AMDGPU_GPU_PAGE_SIZE);
302
303	struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;	308	struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
304	struct amdgpu_res_cursor src_mm, dst_mm;	309	struct amdgpu_res_cursor src_mm, dst_mm;
305	struct dma_fence *fence = NULL;	310	struct dma_fence *fence = NULL;
@@ -315,29 +320,20 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev, · 9 unmodified lines
315		320
316	mutex_lock(&adev->mman.gtt_window_lock);	321	mutex_lock(&adev->mman.gtt_window_lock);
317	while (src_mm.remaining) {	322	while (src_mm.remaining) {
318	uint32_t src_page_offset = src_mm.start & ~PAGE_MASK;	323	uint64_t from, to, cur_size;
319	uint32_t dst_page_offset = dst_mm.start & ~PAGE_MASK;
320	struct dma_fence *next;	324	struct dma_fence *next;
321	uint32_t cur_size;
322	uint64_t from, to;
323		325
324	/* Copy size cannot exceed GTT_MAX_BYTES. So if src or dst	326	/* Never copy more than 256MiB at once to avoid a timeout */
325	* begins at an offset, then adjust the size accordingly	327	cur_size = min3(src_mm.size, dst_mm.size, 256ULL << 20);
326	*/
327	cur_size = max(src_page_offset, dst_page_offset);
328	cur_size = min(min3(src_mm.size, dst_mm.size, size),
329	(uint64_t)(GTT_MAX_BYTES - cur_size));
330		328
331	/* Map src to window 0 and dst to window 1. */	329	/* Map src to window 0 and dst to window 1. */
332	r = amdgpu_ttm_map_buffer(src->bo, src->mem, &src_mm,	330	r = amdgpu_ttm_map_buffer(src->bo, src->mem, &src_mm,
333	PFN_UP(cur_size + src_page_offset),	331	0, ring, tmz, &cur_size, &from);
334	0, ring, tmz, &from);
335	if (r)	332	if (r)
336	goto error;	333	goto error;
337		334
338	r = amdgpu_ttm_map_buffer(dst->bo, dst->mem, &dst_mm,	335	r = amdgpu_ttm_map_buffer(dst->bo, dst->mem, &dst_mm,
339	PFN_UP(cur_size + dst_page_offset),	336	1, ring, tmz, &cur_size, &to);
340	1, ring, tmz, &to);
341	if (r)	337	if (r)
342	goto error;	338	goto error;
343		339
@@ -396,8 +392,7 @@ static int amdgpu_move_blit(struct ttm_buffer_object *bo, · 52 unmodified lines
396	(abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE)) {	392	(abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE)) {
397	struct dma_fence *wipe_fence = NULL;	393	struct dma_fence *wipe_fence = NULL;
398		394
399	r = amdgpu_fill_buffer(ttm_to_amdgpu_bo(bo), AMDGPU_POISON,	395	r = amdgpu_fill_buffer(abo, AMDGPU_POISON, NULL, &wipe_fence);
400	NULL, &wipe_fence);
401	if (r) {	396	if (r) {
402	goto error;	397	goto error;
403	} else if (wipe_fence) {	398	} else if (wipe_fence) {
@@ -821,14 +816,13 @@ static void amdgpu_ttm_tt_unpin_userptr(struct ttm_device *bdev, · 417 unmodified lines
821	#endif	816	#endif
822	}	817	}
823		818
824	static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev,	819	static void amdgpu_ttm_gart_bind(struct amdgpu_device *adev,
825	struct ttm_buffer_object *tbo,	820	struct ttm_buffer_object *tbo,
826	uint64_t flags)	821	uint64_t flags)
827	{	822	{
828	struct amdgpu_bo *abo = ttm_to_amdgpu_bo(tbo);	823	struct amdgpu_bo *abo = ttm_to_amdgpu_bo(tbo);
829	struct ttm_tt *ttm = tbo->ttm;	824	struct ttm_tt *ttm = tbo->ttm;
830	struct amdgpu_ttm_tt gtt = (void )ttm;	825	struct amdgpu_ttm_tt gtt = (void )ttm;
831	int r;
832		826
833	if (amdgpu_bo_encrypted(abo))	827	if (amdgpu_bo_encrypted(abo))
834	flags \|= AMDGPU_PTE_TMZ;	828	flags \|= AMDGPU_PTE_TMZ;
@@ -836,10 +830,8 @@ static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev, · 1 unmodified line
836	if (abo->flags & AMDGPU_GEM_CREATE_CP_MQD_GFX9) {	830	if (abo->flags & AMDGPU_GEM_CREATE_CP_MQD_GFX9) {
837	uint64_t page_idx = 1;	831	uint64_t page_idx = 1;
838		832
839	r = amdgpu_gart_bind(adev, gtt->offset, page_idx,	833	amdgpu_gart_bind(adev, gtt->offset, page_idx,
840	gtt->ttm.dma_address, flags);	834	gtt->ttm.dma_address, flags);
841	if (r)
842	goto gart_bind_fail;
843		835
844	/* The memory type of the first page defaults to UC. Now	836	/* The memory type of the first page defaults to UC. Now
845	* modify the memory type to NC from the second page of	837	* modify the memory type to NC from the second page of
@@ -848,21 +840,13 @@ static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev, · 2 unmodified lines
848	flags &= ~AMDGPU_PTE_MTYPE_VG10_MASK;	840	flags &= ~AMDGPU_PTE_MTYPE_VG10_MASK;
849	flags \|= AMDGPU_PTE_MTYPE_VG10(AMDGPU_MTYPE_NC);	841	flags \|= AMDGPU_PTE_MTYPE_VG10(AMDGPU_MTYPE_NC);
850		842
851	r = amdgpu_gart_bind(adev,	843	amdgpu_gart_bind(adev, gtt->offset + (page_idx << PAGE_SHIFT),
852	gtt->offset + (page_idx << PAGE_SHIFT),	844	ttm->num_pages - page_idx,
853	ttm->num_pages - page_idx,	845	&(gtt->ttm.dma_address[page_idx]), flags);
854	&(gtt->ttm.dma_address[page_idx]), flags);
855	} else {	846	} else {
856	r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages,	847	amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages,
857	gtt->ttm.dma_address, flags);	848	gtt->ttm.dma_address, flags);
858	}	849	}
859
860	gart_bind_fail:
861	if (r)
862	DRM_ERROR("failed to bind %u pages at 0x%08llX\n",
863	ttm->num_pages, gtt->offset);
864
865	return r;
866	}	850	}
867		851
868	/*	852	/*
@@ -878,7 +862,7 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev, · 9 unmodified lines
878	struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);	862	struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
879	struct amdgpu_ttm_tt gtt = (void)ttm;	863	struct amdgpu_ttm_tt gtt = (void)ttm;
880	uint64_t flags;	864	uint64_t flags;
881	int r = 0;	865	int r;
882		866
883	if (!bo_mem)	867	if (!bo_mem)
884	return -EINVAL;	868	return -EINVAL;
@@ -925,14 +909,10 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev, · 40 unmodified lines
925		909
926	/* bind pages into GART page tables */	910	/* bind pages into GART page tables */
927	gtt->offset = (u64)bo_mem->start << PAGE_SHIFT;	911	gtt->offset = (u64)bo_mem->start << PAGE_SHIFT;
928	r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages,	912	amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages,
929	gtt->ttm.dma_address, flags);	913	gtt->ttm.dma_address, flags);
930
931	if (r)
932	DRM_ERROR("failed to bind %u pages at 0x%08llX\n",
933	ttm->num_pages, gtt->offset);
934	gtt->bound = true;	914	gtt->bound = true;
935	return r;	915	return 0;
936	}	916	}
937		917
938	/*	918	/*
@@ -982,12 +962,7 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · 43 unmodified lines
982		962
983	/* Bind pages */	963	/* Bind pages */
984	gtt->offset = (u64)tmp->start << PAGE_SHIFT;	964	gtt->offset = (u64)tmp->start << PAGE_SHIFT;
985	r = amdgpu_ttm_gart_bind(adev, bo, flags);	965	amdgpu_ttm_gart_bind(adev, bo, flags);
986	if (unlikely(r)) {
987	ttm_resource_free(bo, &tmp);
988	return r;
989	}
990
991	amdgpu_gart_invalidate_tlb(adev);	966	amdgpu_gart_invalidate_tlb(adev);
992	ttm_resource_free(bo, &bo->resource);	967	ttm_resource_free(bo, &bo->resource);
993	ttm_bo_assign_mem(bo, tmp);	968	ttm_bo_assign_mem(bo, tmp);
@@ -1001,19 +976,16 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · 7 unmodified lines
1001	* Called by amdgpu_gtt_mgr_recover() from amdgpu_device_reset() to	976	* Called by amdgpu_gtt_mgr_recover() from amdgpu_device_reset() to
1002	* rebind GTT pages during a GPU reset.	977	* rebind GTT pages during a GPU reset.
1003	*/	978	*/
1004	int amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo)	979	void amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo)
1005	{	980	{
1006	struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev);	981	struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev);
1007	uint64_t flags;	982	uint64_t flags;
1008	int r;
1009		983
1010	if (!tbo->ttm)	984	if (!tbo->ttm)
1011	return 0;	985	return;
1012		986
1013	flags = amdgpu_ttm_tt_pte_flags(adev, tbo->ttm, tbo->resource);	987	flags = amdgpu_ttm_tt_pte_flags(adev, tbo->ttm, tbo->resource);
1014	r = amdgpu_ttm_gart_bind(adev, tbo, flags);	988	amdgpu_ttm_gart_bind(adev, tbo, flags);
1015
1016	return r;
1017	}	989	}
1018		990
1019	/*	991	/*
@@ -1027,7 +999,6 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev, · 7 unmodified lines
1027	{	999	{
1028	struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);	1000	struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
1029	struct amdgpu_ttm_tt gtt = (void )ttm;	1001	struct amdgpu_ttm_tt gtt = (void )ttm;
1030	int r;
1031		1002
1032	/* if the pages have userptr pinning then clear that first */	1003	/* if the pages have userptr pinning then clear that first */
1033	if (gtt->userptr) {	1004	if (gtt->userptr) {
@@ -1047,10 +1018,7 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev, · 13 unmodified lines
1047	return;	1018	return;
1048		1019
1049	/* unbind shouldn't be done for GDS/GWS/OA in ttm_bo_clean_mm */	1020	/* unbind shouldn't be done for GDS/GWS/OA in ttm_bo_clean_mm */
1050	r = amdgpu_gart_unbind(adev, gtt->offset, ttm->num_pages);	1021	amdgpu_gart_unbind(adev, gtt->offset, ttm->num_pages);
1051	if (r)
1052	DRM_ERROR("failed to unbind %u pages at 0x%08llX\n",
1053	gtt->ttm.num_pages, gtt->offset);
1054	gtt->bound = false;	1022	gtt->bound = false;
1055	}	1023	}
1056		1024
@@ -1168,6 +1136,26 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, · 111 unmodified lines
1168	return ttm_pool_free(&adev->mman.bdev.pool, ttm);	1136	return ttm_pool_free(&adev->mman.bdev.pool, ttm);
1169	}	1137	}
1170		1138
		1139	/**
		1140	* amdgpu_ttm_tt_get_userptr - Return the userptr GTT ttm_tt for the current
		1141	* task
		1142	*
		1143	* @tbo: The ttm_buffer_object that contains the userptr
		1144	* @user_addr: The returned value
		1145	*/
		1146	int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo,
		1147	uint64_t *user_addr)
		1148	{
		1149	struct amdgpu_ttm_tt *gtt;
		1150
		1151	if (!tbo->ttm)
		1152	return -EINVAL;
		1153
		1154	gtt = (void *)tbo->ttm;
		1155	*user_addr = gtt->userptr;
		1156	return 0;
		1157	}
		1158
1171	/**	1159	/**
1172	* amdgpu_ttm_tt_set_userptr - Initialize userptr GTT ttm_tt for the current	1160	* amdgpu_ttm_tt_set_userptr - Initialize userptr GTT ttm_tt for the current
1173	* task	1161	* task
@@ -1433,6 +1421,63 @@ static void amdgpu_ttm_vram_mm_access(struct amdgpu_device *adev, loff_t pos, · 259 unmodified lines
1433	}	1421	}
1434	}	1422	}
1435		1423
		1424	static int amdgpu_ttm_access_memory_sdma(struct ttm_buffer_object *bo,
		1425	unsigned long offset, void *buf, int len, int write)
		1426	{
		1427	struct amdgpu_bo *abo = ttm_to_amdgpu_bo(bo);
		1428	struct amdgpu_device *adev = amdgpu_ttm_adev(abo->tbo.bdev);
		1429	struct amdgpu_res_cursor src_mm;
		1430	struct amdgpu_job *job;
		1431	struct dma_fence *fence;
		1432	uint64_t src_addr, dst_addr;
		1433	unsigned int num_dw;
		1434	int r, idx;
		1435
		1436	if (len != PAGE_SIZE)
		1437	return -EINVAL;
		1438
		1439	if (!adev->mman.sdma_access_ptr)
		1440	return -EACCES;
		1441
		1442	if (!drm_dev_enter(adev_to_drm(adev), &idx))
		1443	return -ENODEV;
		1444
		1445	if (write)
		1446	memcpy(adev->mman.sdma_access_ptr, buf, len);
		1447
		1448	num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
		1449	r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, AMDGPU_IB_POOL_DELAYED, &job);
		1450	if (r)
		1451	goto out;
		1452
		1453	amdgpu_res_first(abo->tbo.resource, offset, len, &src_mm);
		1454	src_addr = amdgpu_ttm_domain_start(adev, bo->resource->mem_type) + src_mm.start;
		1455	dst_addr = amdgpu_bo_gpu_offset(adev->mman.sdma_access_bo);
		1456	if (write)
		1457	swap(src_addr, dst_addr);
		1458
		1459	amdgpu_emit_copy_buffer(adev, &job->ibs[0], src_addr, dst_addr, PAGE_SIZE, false);
		1460
		1461	amdgpu_ring_pad_ib(adev->mman.buffer_funcs_ring, &job->ibs[0]);
		1462	WARN_ON(job->ibs[0].length_dw > num_dw);
		1463
		1464	r = amdgpu_job_submit(job, &adev->mman.entity, AMDGPU_FENCE_OWNER_UNDEFINED, &fence);
		1465	if (r) {
		1466	amdgpu_job_free(job);
		1467	goto out;
		1468	}
		1469
		1470	if (!dma_fence_wait_timeout(fence, false, adev->sdma_timeout))
		1471	r = -ETIMEDOUT;
		1472	dma_fence_put(fence);
		1473
		1474	if (!(r \|\| write))
		1475	memcpy(buf, adev->mman.sdma_access_ptr, len);
		1476	out:
		1477	drm_dev_exit(idx);
		1478	return r;
		1479	}
		1480
1436	/**	1481	/**
1437	* amdgpu_ttm_access_memory - Read or Write memory that backs a buffer object.	1482	* amdgpu_ttm_access_memory - Read or Write memory that backs a buffer object.
1438	*	1483	*
@@ -1457,6 +1502,10 @@ static int amdgpu_ttm_access_memory(struct ttm_buffer_object *bo, · 18 unmodified lines
1457	if (bo->resource->mem_type != TTM_PL_VRAM)	1502	if (bo->resource->mem_type != TTM_PL_VRAM)
1458	return -EIO;	1503	return -EIO;
1459		1504
		1505	if (amdgpu_device_has_timeouts_enabled(adev) &&
		1506	!amdgpu_ttm_access_memory_sdma(bo, offset, buf, len, write))
		1507	return len;
		1508
1460	amdgpu_res_first(bo->resource, offset, len, &cursor);	1509	amdgpu_res_first(bo->resource, offset, len, &cursor);
1461	while (cursor.remaining) {	1510	while (cursor.remaining) {
1462	size_t count, size = cursor.size;	1511	size_t count, size = cursor.size;
@@ -1797,6 +1846,12 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) · 334 unmodified lines
1797	return r;	1846	return r;
1798	}	1847	}
1799		1848
		1849	if (amdgpu_bo_create_kernel(adev, PAGE_SIZE, PAGE_SIZE,
		1850	AMDGPU_GEM_DOMAIN_GTT,
		1851	&adev->mman.sdma_access_bo, NULL,
		1852	&adev->mman.sdma_access_ptr))
		1853	DRM_WARN("Debug VRAM access will use slowpath MM access\n");
		1854
1800	return 0;	1855	return 0;
1801	}	1856	}
1802		1857
@@ -1818,6 +1873,8 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) · 15 unmodified lines
1818	if (adev->mman.stolen_reserved_size)	1873	if (adev->mman.stolen_reserved_size)
1819	amdgpu_bo_free_kernel(&adev->mman.stolen_reserved_memory,	1874	amdgpu_bo_free_kernel(&adev->mman.stolen_reserved_memory,
1820	NULL, NULL);	1875	NULL, NULL);
		1876	amdgpu_bo_free_kernel(&adev->mman.sdma_access_bo, NULL,
		1877	&adev->mman.sdma_access_ptr);
1821	amdgpu_ttm_fw_reserve_vram_fini(adev);	1878	amdgpu_ttm_fw_reserve_vram_fini(adev);
1822		1879
1823	if (drm_dev_enter(adev_to_drm(adev), &idx)) {	1880	if (drm_dev_enter(adev_to_drm(adev), &idx)) {
@@ -1884,27 +1941,59 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) · 60 unmodified lines
1884	size = adev->gmc.real_vram_size;	1941	size = adev->gmc.real_vram_size;
1885	else	1942	else
1886	size = adev->gmc.visible_vram_size;	1943	size = adev->gmc.visible_vram_size;
1887	man->size = size >> PAGE_SHIFT;	1944	man->size = size;
1888	adev->mman.buffer_funcs_enabled = enable;	1945	adev->mman.buffer_funcs_enabled = enable;
1889	}	1946	}
1890		1947
		1948	static int amdgpu_ttm_prepare_job(struct amdgpu_device *adev,
		1949	bool direct_submit,
		1950	unsigned int num_dw,
		1951	struct dma_resv *resv,
		1952	bool vm_needs_flush,
		1953	struct amdgpu_job **job)
		1954	{
		1955	enum amdgpu_ib_pool_type pool = direct_submit ?
		1956	AMDGPU_IB_POOL_DIRECT :
		1957	AMDGPU_IB_POOL_DELAYED;
		1958	int r;
		1959
		1960	r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, pool, job);
		1961	if (r)
		1962	return r;
		1963
		1964	if (vm_needs_flush) {
		1965	(*job)->vm_pd_addr = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo ?
		1966	adev->gmc.pdb0_bo :
		1967	adev->gart.bo);
		1968	(*job)->vm_needs_flush = true;
		1969	}
		1970	if (resv) {
		1971	r = amdgpu_sync_resv(adev, &(*job)->sync, resv,
		1972	AMDGPU_SYNC_ALWAYS,
		1973	AMDGPU_FENCE_OWNER_UNDEFINED);
		1974	if (r) {
		1975	DRM_ERROR("sync failed (%d).\n", r);
		1976	amdgpu_job_free(*job);
		1977	return r;
		1978	}
		1979	}
		1980	return 0;
		1981	}
		1982
1891	int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset,	1983	int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset,
1892	uint64_t dst_offset, uint32_t byte_count,	1984	uint64_t dst_offset, uint32_t byte_count,
1893	struct dma_resv *resv,	1985	struct dma_resv *resv,
1894	struct dma_fence **fence, bool direct_submit,	1986	struct dma_fence **fence, bool direct_submit,
1895	bool vm_needs_flush, bool tmz)	1987	bool vm_needs_flush, bool tmz)
1896	{	1988	{
1897	enum amdgpu_ib_pool_type pool = direct_submit ? AMDGPU_IB_POOL_DIRECT :
1898	AMDGPU_IB_POOL_DELAYED;
1899	struct amdgpu_device *adev = ring->adev;	1989	struct amdgpu_device *adev = ring->adev;
		1990	unsigned num_loops, num_dw;
1900	struct amdgpu_job *job;	1991	struct amdgpu_job *job;
1901
1902	uint32_t max_bytes;	1992	uint32_t max_bytes;
1903	unsigned num_loops, num_dw;
1904	unsigned i;	1993	unsigned i;
1905	int r;	1994	int r;
1906		1995
1907	if (direct_submit && !ring->sched.ready) {	1996	if (!direct_submit && !ring->sched.ready) {
1908	DRM_ERROR("Trying to move memory with ring turned off.\n");	1997	DRM_ERROR("Trying to move memory with ring turned off.\n");
1909	return -EINVAL;	1998	return -EINVAL;
1910	}	1999	}
@@ -1912,26 +2001,11 @@ int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset, · 1 unmodified line
1912	max_bytes = adev->mman.buffer_funcs->copy_max_bytes;	2001	max_bytes = adev->mman.buffer_funcs->copy_max_bytes;
1913	num_loops = DIV_ROUND_UP(byte_count, max_bytes);	2002	num_loops = DIV_ROUND_UP(byte_count, max_bytes);
1914	num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->copy_num_dw, 8);	2003	num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->copy_num_dw, 8);
1915		2004	r = amdgpu_ttm_prepare_job(adev, direct_submit, num_dw,
1916	r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, pool, &job);	2005	resv, vm_needs_flush, &job);
1917	if (r)	2006	if (r)
1918	return r;	2007	return r;
1919		2008
1920	if (vm_needs_flush) {
1921	job->vm_pd_addr = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo ?
1922	adev->gmc.pdb0_bo : adev->gart.bo);
1923	job->vm_needs_flush = true;
1924	}
1925	if (resv) {
1926	r = amdgpu_sync_resv(adev, &job->sync, resv,
1927	AMDGPU_SYNC_ALWAYS,
1928	AMDGPU_FENCE_OWNER_UNDEFINED);
1929	if (r) {
1930	DRM_ERROR("sync failed (%d).\n", r);
1931	goto error_free;
1932	}
1933	}
1934
1935	for (i = 0; i < num_loops; i++) {	2009	for (i = 0; i < num_loops; i++) {
1936	uint32_t cur_size_in_bytes = min(byte_count, max_bytes);	2010	uint32_t cur_size_in_bytes = min(byte_count, max_bytes);
1937		2011
@@ -1961,77 +2035,35 @@ error_free: · 23 unmodified lines
1961	return r;	2035	return r;
1962	}	2036	}
1963		2037
1964	int amdgpu_fill_buffer(struct amdgpu_bo *bo,	2038	static int amdgpu_ttm_fill_mem(struct amdgpu_ring *ring, uint32_t src_data,
1965	uint32_t src_data,	2039	uint64_t dst_addr, uint32_t byte_count,
1966	struct dma_resv *resv,	2040	struct dma_resv *resv,
1967	struct dma_fence **fence)	2041	struct dma_fence **fence,
		2042	bool vm_needs_flush)
1968	{	2043	{
1969	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);	2044	struct amdgpu_device *adev = ring->adev;
1970	uint32_t max_bytes = adev->mman.buffer_funcs->fill_max_bytes;
1971	struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
1972
1973	struct amdgpu_res_cursor cursor;
1974	unsigned int num_loops, num_dw;	2045	unsigned int num_loops, num_dw;
1975	uint64_t num_bytes;
1976
1977	struct amdgpu_job *job;	2046	struct amdgpu_job *job;
		2047	uint32_t max_bytes;
		2048	unsigned int i;
1978	int r;	2049	int r;
1979		2050
1980	if (!adev->mman.buffer_funcs_enabled) {	2051	max_bytes = adev->mman.buffer_funcs->fill_max_bytes;
1981	DRM_ERROR("Trying to clear memory with ring turned off.\n");	2052	num_loops = DIV_ROUND_UP_ULL(byte_count, max_bytes);
1982	return -EINVAL;	2053	num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->fill_num_dw, 8);
1983	}	2054	r = amdgpu_ttm_prepare_job(adev, false, num_dw, resv, vm_needs_flush,
1984		2055	&job);
1985	if (bo->tbo.resource->mem_type == AMDGPU_PL_PREEMPT) {
1986	DRM_ERROR("Trying to clear preemptible memory.\n");
1987	return -EINVAL;
1988	}
1989
1990	if (bo->tbo.resource->mem_type == TTM_PL_TT) {
1991	r = amdgpu_ttm_alloc_gart(&bo->tbo);
1992	if (r)
1993	return r;
1994	}
1995
1996	num_bytes = bo->tbo.resource->num_pages << PAGE_SHIFT;
1997	num_loops = 0;
1998
1999	amdgpu_res_first(bo->tbo.resource, 0, num_bytes, &cursor);
2000	while (cursor.remaining) {
2001	num_loops += DIV_ROUND_UP_ULL(cursor.size, max_bytes);
2002	amdgpu_res_next(&cursor, cursor.size);
2003	}
2004	num_dw = num_loops * adev->mman.buffer_funcs->fill_num_dw;
2005
2006	/* for IB padding */
2007	num_dw += 64;
2008
2009	r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, AMDGPU_IB_POOL_DELAYED,
2010	&job);
2011	if (r)	2056	if (r)
2012	return r;	2057	return r;
2013		2058
2014	if (resv) {	2059	for (i = 0; i < num_loops; i++) {
2015	r = amdgpu_sync_resv(adev, &job->sync, resv,	2060	uint32_t cur_size = min(byte_count, max_bytes);
2016	AMDGPU_SYNC_ALWAYS,
2017	AMDGPU_FENCE_OWNER_UNDEFINED);
2018	if (r) {
2019	DRM_ERROR("sync failed (%d).\n", r);
2020	goto error_free;
2021	}
2022	}
2023
2024	amdgpu_res_first(bo->tbo.resource, 0, num_bytes, &cursor);
2025	while (cursor.remaining) {
2026	uint32_t cur_size = min_t(uint64_t, cursor.size, max_bytes);
2027	uint64_t dst_addr = cursor.start;
2028		2061
2029	dst_addr += amdgpu_ttm_domain_start(adev,
2030	bo->tbo.resource->mem_type);
2031	amdgpu_emit_fill_buffer(adev, &job->ibs[0], src_data, dst_addr,	2062	amdgpu_emit_fill_buffer(adev, &job->ibs[0], src_data, dst_addr,
2032	cur_size);	2063	cur_size);
2033		2064
2034	amdgpu_res_next(&cursor, cur_size);	2065	dst_addr += cur_size;
		2066	byte_count -= cur_size;
2035	}	2067	}
2036		2068
2037	amdgpu_ring_pad_ib(ring, &job->ibs[0]);	2069	amdgpu_ring_pad_ib(ring, &job->ibs[0]);
@@ -2048,6 +2080,55 @@ error_free: · 10 unmodified lines
2048	return r;	2080	return r;
2049	}	2081	}
2050		2082
		2083	int amdgpu_fill_buffer(struct amdgpu_bo *bo,
		2084	uint32_t src_data,
		2085	struct dma_resv *resv,
		2086	struct dma_fence **f)
		2087	{
		2088	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
		2089	struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
		2090	struct dma_fence *fence = NULL;
		2091	struct amdgpu_res_cursor dst;
		2092	int r;
		2093
		2094	if (!adev->mman.buffer_funcs_enabled) {
		2095	DRM_ERROR("Trying to clear memory with ring turned off.\n");
		2096	return -EINVAL;
		2097	}
		2098
		2099	amdgpu_res_first(bo->tbo.resource, 0, amdgpu_bo_size(bo), &dst);
		2100
		2101	mutex_lock(&adev->mman.gtt_window_lock);
		2102	while (dst.remaining) {
		2103	struct dma_fence *next;
		2104	uint64_t cur_size, to;
		2105
		2106	/* Never fill more than 256MiB at once to avoid timeouts */
		2107	cur_size = min(dst.size, 256ULL << 20);
		2108
		2109	r = amdgpu_ttm_map_buffer(&bo->tbo, bo->tbo.resource, &dst,
		2110	1, ring, false, &cur_size, &to);
		2111	if (r)
		2112	goto error;
		2113
		2114	r = amdgpu_ttm_fill_mem(ring, src_data, to, cur_size, resv,
		2115	&next, true);
		2116	if (r)
		2117	goto error;
		2118
		2119	dma_fence_put(fence);
		2120	fence = next;
		2121
		2122	amdgpu_res_next(&dst, cur_size);
		2123	}
		2124	error:
		2125	mutex_unlock(&adev->mman.gtt_window_lock);
		2126	if (f)
		2127	*f = dma_fence_get(fence);
		2128	dma_fence_put(fence);
		2129	return r;
		2130	}
		2131
2051	/**	2132	/**
2052	* amdgpu_ttm_evict_resources - evict memory buffers	2133	* amdgpu_ttm_evict_resources - evict memory buffers
2053	* @adev: amdgpu device object	2134	* @adev: amdgpu device object

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h+9−12

 	spinlock_t lock;
 	struct list_head reservations_pending;
 	struct list_head reserved_pages;
-	atomic64_t usage;
 	atomic64_t vis_usage;
 };
 	struct ttm_resource_manager manager;
 	struct drm_mm mm;
 	spinlock_t lock;
-	atomic64_t used;
-};
-struct amdgpu_preempt_mgr {
-	struct ttm_resource_manager manager;
-	atomic64_t used;
 };
 struct amdgpu_mman {
 	struct amdgpu_vram_mgr vram_mgr;
 	struct amdgpu_gtt_mgr gtt_mgr;
-	struct amdgpu_preempt_mgr preempt_mgr;
+	struct ttm_resource_manager preempt_mgr;
 	uint64_t		stolen_vga_size;
 	struct amdgpu_bo	*stolen_vga_memory;
 	u64		fw_vram_usage_size;
 	struct amdgpu_bo	*fw_vram_usage_reserved_bo;
 	void		*fw_vram_usage_va;
+	/* PAGE_SIZE'd BO for process memory r/w over SDMA. */
+	struct amdgpu_bo	*sdma_access_bo;
+	void			*sdma_access_ptr;
 };
 struct amdgpu_copy_mem {
 void amdgpu_vram_mgr_fini(struct amdgpu_device *adev);
 bool amdgpu_gtt_mgr_has_gart_addr(struct ttm_resource *mem);
-uint64_t amdgpu_gtt_mgr_usage(struct amdgpu_gtt_mgr *mgr);
+void amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr);
-int amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr);
 uint64_t amdgpu_preempt_mgr_usage(struct ttm_resource_manager *man);
 void amdgpu_vram_mgr_free_sgt(struct device *dev,
 			      enum dma_data_direction dir,
 			      struct sg_table *sgt);
-uint64_t amdgpu_vram_mgr_usage(struct amdgpu_vram_mgr *mgr);
 uint64_t amdgpu_vram_mgr_vis_usage(struct amdgpu_vram_mgr *mgr);
 int amdgpu_vram_mgr_reserve_range(struct amdgpu_vram_mgr *mgr,
 				  uint64_t start, uint64_t size);
 			struct dma_fence **fence);
 int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo);
-int amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo);
+void amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo);
 uint64_t amdgpu_ttm_domain_start(struct amdgpu_device *adev, uint32_t type);
 #if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
 #endif
 void amdgpu_ttm_tt_set_user_pages(struct ttm_tt *ttm, struct page **pages);
+int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo,
+			      uint64_t *user_addr);
 int amdgpu_ttm_tt_set_userptr(struct ttm_buffer_object *bo,
 			      uint64_t addr, uint32_t flags);
 bool amdgpu_ttm_tt_has_userptr(struct ttm_tt *ttm);

drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h+1−0

@@ -378,6 +378,7 @@ enum AMDGPU_UCODE_ID { · 377 unmodified lines
378	AMDGPU_UCODE_ID_VCN0_RAM,	378	AMDGPU_UCODE_ID_VCN0_RAM,
379	AMDGPU_UCODE_ID_VCN1_RAM,	379	AMDGPU_UCODE_ID_VCN1_RAM,
380	AMDGPU_UCODE_ID_DMCUB,	380	AMDGPU_UCODE_ID_DMCUB,
		381	AMDGPU_UCODE_ID_CAP,
381	AMDGPU_UCODE_ID_MAXIMUM,	382	AMDGPU_UCODE_ID_MAXIMUM,
382	};	383	};
383		384

drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c+47−59

  *
  */
-#include "amdgpu_ras.h"
+#include "amdgpu.h"
 static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev,
 		void *ras_error_status,
 	int ret = 0;
 	kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
-	ret = smu_get_ecc_info(&adev->smu, (void *)&(con->umc_ecc));
+	ret = amdgpu_dpm_get_ecc_info(adev, (void *)&(con->umc_ecc));
 	if (ret == -EOPNOTSUPP) {
-		if (adev->umc.ras_funcs &&
+		if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops &&
-		    adev->umc.ras_funcs->query_ras_error_count)
+		    adev->umc.ras->ras_block.hw_ops->query_ras_error_count)
-		    adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status);
+		    adev->umc.ras->ras_block.hw_ops->query_ras_error_count(adev, ras_error_status);
-		if (adev->umc.ras_funcs &&
+		if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops &&
-		    adev->umc.ras_funcs->query_ras_error_address &&
+		    adev->umc.ras->ras_block.hw_ops->query_ras_error_address &&
 		    adev->umc.max_ras_err_cnt_per_query) {
 			err_data->err_addr =
 				kcalloc(adev->umc.max_ras_err_cnt_per_query,
 			/* umc query_ras_error_address is also responsible for clearing
 			 * error status
 			 */
-			adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status);
+			adev->umc.ras->ras_block.hw_ops->query_ras_error_address(adev, ras_error_status);
 		}
 	} else if (!ret) {
-		if (adev->umc.ras_funcs &&
+		if (adev->umc.ras &&
-		    adev->umc.ras_funcs->ecc_info_query_ras_error_count)
+		    adev->umc.ras->ecc_info_query_ras_error_count)
-		    adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, ras_error_status);
+		    adev->umc.ras->ecc_info_query_ras_error_count(adev, ras_error_status);
-		if (adev->umc.ras_funcs &&
+		if (adev->umc.ras &&
-		    adev->umc.ras_funcs->ecc_info_query_ras_error_address &&
+		    adev->umc.ras->ecc_info_query_ras_error_address &&
 		    adev->umc.max_ras_err_cnt_per_query) {
 			err_data->err_addr =
 				kcalloc(adev->umc.max_ras_err_cnt_per_query,
 			/* umc query_ras_error_address is also responsible for clearing
 			 * error status
 			 */
-			adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, ras_error_status);
+			adev->umc.ras->ecc_info_query_ras_error_address(adev, ras_error_status);
 		}
 	}
 						err_data->err_addr_cnt);
 			amdgpu_ras_save_bad_pages(adev);
-			if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num)
+			amdgpu_dpm_send_hbm_bad_pages_num(adev, con->eeprom_control.ras_num_recs);
-				adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs);
 		}
 		if (reset)
 	return ret;
 }
-static int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
+int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
 		void *ras_error_status,
 		struct amdgpu_iv_entry *entry)
 {
 	return amdgpu_umc_do_page_retirement(adev, ras_error_status, entry, true);
 }
-int amdgpu_umc_ras_late_init(struct amdgpu_device *adev)
+int amdgpu_umc_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block)
 {
 	int r;
-	struct ras_fs_if fs_info = {
-		.sysfs_name = "umc_err_count",
-	};
-	struct ras_ih_if ih_info = {
-		.cb = amdgpu_umc_process_ras_data_cb,
-	};
-	if (!adev->umc.ras_if) {
-		adev->umc.ras_if =
-			kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
-		if (!adev->umc.ras_if)
-			return -ENOMEM;
-		adev->umc.ras_if->block = AMDGPU_RAS_BLOCK__UMC;
-		adev->umc.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
-		adev->umc.ras_if->sub_block_index = 0;
-	}
-	ih_info.head = fs_info.head = *adev->umc.ras_if;
-	r = amdgpu_ras_late_init(adev, adev->umc.ras_if,
+	r = amdgpu_ras_block_late_init(adev, ras_block);
-				 &fs_info, &ih_info);
 	if (r)
-		goto free;
+		return r;
-	if (amdgpu_ras_is_supported(adev, adev->umc.ras_if->block)) {
+	if (amdgpu_ras_is_supported(adev, ras_block->block)) {
 		r = amdgpu_irq_get(adev, &adev->gmc.ecc_irq, 0);
 		if (r)
 			goto late_fini;
-	} else {
-		r = 0;
-		goto free;
 	}
 	/* ras init of specific umc version */
-	if (adev->umc.ras_funcs &&
+	if (adev->umc.ras &&
-	    adev->umc.ras_funcs->err_cnt_init)
+	    adev->umc.ras->err_cnt_init)
-		adev->umc.ras_funcs->err_cnt_init(adev);
+		adev->umc.ras->err_cnt_init(adev);
 	return 0;
 late_fini:
-	amdgpu_ras_late_fini(adev, adev->umc.ras_if, &ih_info);
+	amdgpu_ras_block_late_fini(adev, ras_block);
-free:
-	kfree(adev->umc.ras_if);
-	adev->umc.ras_if = NULL;
 	return r;
 }
 void amdgpu_umc_ras_fini(struct amdgpu_device *adev)
 {
 	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__UMC) &&
-			adev->umc.ras_if) {
+			adev->umc.ras_if)
-		struct ras_common_if *ras_if = adev->umc.ras_if;
+		amdgpu_ras_block_late_fini(adev, adev->umc.ras_if);
-		struct ras_ih_if ih_info = {
-			.head = *ras_if,
-			.cb = amdgpu_umc_process_ras_data_cb,
-		};
-		amdgpu_ras_late_fini(adev, ras_if, &ih_info);
-		kfree(ras_if);
-	}
 }
 int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev,
 	amdgpu_ras_interrupt_dispatch(adev, &ih_data);
 	return 0;
 }
+void amdgpu_umc_fill_error_record(struct ras_err_data *err_data,
+		uint64_t err_addr,
+		uint64_t retired_page,
+		uint32_t channel_index,
+		uint32_t umc_inst)
+{
+	struct eeprom_table_record *err_rec =
+		&err_data->err_addr[err_data->err_addr_cnt];
+	err_rec->address = err_addr;
+	/* page frame address is saved */
+	err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;
+	err_rec->ts = (uint64_t)ktime_get_real_seconds();
+	err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
+	err_rec->cu = 0;
+	err_rec->mem_channel = channel_index;
+	err_rec->mcumc_id = umc_inst;
+	err_data->err_addr_cnt++;
+}

drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h+14−9

  */
 #ifndef __AMDGPU_UMC_H__
 #define __AMDGPU_UMC_H__
+#include "amdgpu_ras.h"
 /*
  * (addr / 256) * 4096, the higher 26 bits in ErrorAddr
 #define LOOP_UMC_CH_INST(ch_inst) for ((ch_inst) = 0; (ch_inst) < adev->umc.channel_inst_num; (ch_inst)++)
 #define LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) LOOP_UMC_INST((umc_inst)) LOOP_UMC_CH_INST((ch_inst))
-struct amdgpu_umc_ras_funcs {
+struct amdgpu_umc_ras {
+	struct amdgpu_ras_block_object ras_block;
 	void (*err_cnt_init)(struct amdgpu_device *adev);
-	int (*ras_late_init)(struct amdgpu_device *adev);
-	void (*ras_fini)(struct amdgpu_device *adev);
-	void (*query_ras_error_count)(struct amdgpu_device *adev,
-				      void *ras_error_status);
-	void (*query_ras_error_address)(struct amdgpu_device *adev,
-					void *ras_error_status);
 	bool (*query_ras_poison_mode)(struct amdgpu_device *adev);
 	void (*ecc_info_query_ras_error_count)(struct amdgpu_device *adev,
 				      void *ras_error_status);
 	struct ras_common_if *ras_if;
 	const struct amdgpu_umc_funcs *funcs;
-	const struct amdgpu_umc_ras_funcs *ras_funcs;
+	struct amdgpu_umc_ras *ras;
 };
-int amdgpu_umc_ras_late_init(struct amdgpu_device *adev);
+int amdgpu_umc_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block);
 void amdgpu_umc_ras_fini(struct amdgpu_device *adev);
 int amdgpu_umc_poison_handler(struct amdgpu_device *adev,
 		void *ras_error_status,
 int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev,
 		struct amdgpu_irq_src *source,
 		struct amdgpu_iv_entry *entry);
+void amdgpu_umc_fill_error_record(struct ras_err_data *err_data,
+		uint64_t err_addr,
+		uint64_t retired_page,
+		uint32_t channel_index,
+		uint32_t umc_inst);
+int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
+		void *ras_error_status,
+		struct amdgpu_iv_entry *entry);
 #endif

drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c+2−0

 	handle = msg[2];
 	if (handle == 0) {
+		amdgpu_bo_kunmap(bo);
 		DRM_ERROR("Invalid UVD handle!\n");
 		return -EINVAL;
 	}
 		DRM_ERROR("Illegal UVD message type (%d)!\n", msg_type);
 	}
+	amdgpu_bo_kunmap(bo);
 	return -EINVAL;
 }

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c+149−2

@@ -575,8 +575,10 @@ static int amdgpu_virt_write_vf2pf_data(struct amdgpu_device *adev) · 574 unmodified lines
575	vf2pf_info->driver_cert = 0;	575	vf2pf_info->driver_cert = 0;
576	vf2pf_info->os_info.all = 0;	576	vf2pf_info->os_info.all = 0;
577		577
578	vf2pf_info->fb_usage = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr) >> 20;	578	vf2pf_info->fb_usage =
579	vf2pf_info->fb_vis_usage = amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr) >> 20;	579	ttm_resource_manager_usage(&adev->mman.vram_mgr.manager) >> 20;
		580	vf2pf_info->fb_vis_usage =
		581	amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr) >> 20;
580	vf2pf_info->fb_size = adev->gmc.real_vram_size >> 20;	582	vf2pf_info->fb_size = adev->gmc.real_vram_size >> 20;
581	vf2pf_info->fb_vis_size = adev->gmc.visible_vram_size >> 20;	583	vf2pf_info->fb_vis_size = adev->gmc.visible_vram_size >> 20;
582		584
@@ -820,3 +822,148 @@ void amdgpu_virt_update_sriov_video_codec(struct amdgpu_device *adev, · 237 unmodified lines
820	}	822	}
821	}	823	}
822	}	824	}
		825
		826	static bool amdgpu_virt_get_rlcg_reg_access_flag(struct amdgpu_device *adev,
		827	u32 acc_flags, u32 hwip,
		828	bool write, u32 *rlcg_flag)
		829	{
		830	bool ret = false;
		831
		832	switch (hwip) {
		833	case GC_HWIP:
		834	if (amdgpu_sriov_reg_indirect_gc(adev)) {
		835	*rlcg_flag =
		836	write ? AMDGPU_RLCG_GC_WRITE : AMDGPU_RLCG_GC_READ;
		837	ret = true;
		838	/* only in new version, AMDGPU_REGS_NO_KIQ and
		839	* AMDGPU_REGS_RLC are enabled simultaneously */
		840	} else if ((acc_flags & AMDGPU_REGS_RLC) &&
		841	!(acc_flags & AMDGPU_REGS_NO_KIQ) && write) {
		842	*rlcg_flag = AMDGPU_RLCG_GC_WRITE_LEGACY;
		843	ret = true;
		844	}
		845	break;
		846	case MMHUB_HWIP:
		847	if (amdgpu_sriov_reg_indirect_mmhub(adev) &&
		848	(acc_flags & AMDGPU_REGS_RLC) && write) {
		849	*rlcg_flag = AMDGPU_RLCG_MMHUB_WRITE;
		850	ret = true;
		851	}
		852	break;
		853	default:
		854	break;
		855	}
		856	return ret;
		857	}
		858
		859	static u32 amdgpu_virt_rlcg_reg_rw(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag)
		860	{
		861	struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl;
		862	uint32_t timeout = 50000;
		863	uint32_t i, tmp;
		864	uint32_t ret = 0;
		865	static void *scratch_reg0;
		866	static void *scratch_reg1;
		867	static void *scratch_reg2;
		868	static void *scratch_reg3;
		869	static void *spare_int;
		870
		871	if (!adev->gfx.rlc.rlcg_reg_access_supported) {
		872	dev_err(adev->dev,
		873	"indirect registers access through rlcg is not available\n");
		874	return 0;
		875	}
		876
		877	reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl;
		878	scratch_reg0 = (void __iomem )adev->rmmio + 4 reg_access_ctrl->scratch_reg0;
		879	scratch_reg1 = (void __iomem )adev->rmmio + 4 reg_access_ctrl->scratch_reg1;
		880	scratch_reg2 = (void __iomem )adev->rmmio + 4 reg_access_ctrl->scratch_reg2;
		881	scratch_reg3 = (void __iomem )adev->rmmio + 4 reg_access_ctrl->scratch_reg3;
		882	if (reg_access_ctrl->spare_int)
		883	spare_int = (void __iomem )adev->rmmio + 4 reg_access_ctrl->spare_int;
		884
		885	if (offset == reg_access_ctrl->grbm_cntl) {
		886	/* if the target reg offset is grbm_cntl, write to scratch_reg2 */
		887	writel(v, scratch_reg2);
		888	writel(v, ((void __iomem )adev->rmmio) + (offset 4));
		889	} else if (offset == reg_access_ctrl->grbm_idx) {
		890	/* if the target reg offset is grbm_idx, write to scratch_reg3 */
		891	writel(v, scratch_reg3);
		892	writel(v, ((void __iomem )adev->rmmio) + (offset 4));
		893	} else {
		894	/*
		895	* SCRATCH_REG0 = read/write value
		896	* SCRATCH_REG1[30:28] = command
		897	* SCRATCH_REG1[19:0] = address in dword
		898	* SCRATCH_REG1[26:24] = Error reporting
		899	*/
		900	writel(v, scratch_reg0);
		901	writel((offset \| flag), scratch_reg1);
		902	if (reg_access_ctrl->spare_int)
		903	writel(1, spare_int);
		904
		905	for (i = 0; i < timeout; i++) {
		906	tmp = readl(scratch_reg1);
		907	if (!(tmp & AMDGPU_RLCG_SCRATCH1_ADDRESS_MASK))
		908	break;
		909	udelay(10);
		910	}
		911
		912	if (i >= timeout) {
		913	if (amdgpu_sriov_rlcg_error_report_enabled(adev)) {
		914	if (tmp & AMDGPU_RLCG_VFGATE_DISABLED) {
		915	dev_err(adev->dev,
		916	"vfgate is disabled, rlcg failed to program reg: 0x%05x\n", offset);
		917	} else if (tmp & AMDGPU_RLCG_WRONG_OPERATION_TYPE) {
		918	dev_err(adev->dev,
		919	"wrong operation type, rlcg failed to program reg: 0x%05x\n", offset);
		920	} else if (tmp & AMDGPU_RLCG_REG_NOT_IN_RANGE) {
		921	dev_err(adev->dev,
		922	"regiser is not in range, rlcg failed to program reg: 0x%05x\n", offset);
		923	} else {
		924	dev_err(adev->dev,
		925	"unknown error type, rlcg failed to program reg: 0x%05x\n", offset);
		926	}
		927	} else {
		928	dev_err(adev->dev,
		929	"timeout: rlcg faled to program reg: 0x%05x\n", offset);
		930	}
		931	}
		932	}
		933
		934	ret = readl(scratch_reg0);
		935	return ret;
		936	}
		937
		938	void amdgpu_sriov_wreg(struct amdgpu_device *adev,
		939	u32 offset, u32 value,
		940	u32 acc_flags, u32 hwip)
		941	{
		942	u32 rlcg_flag;
		943
		944	if (!amdgpu_sriov_runtime(adev) &&
		945	amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, true, &rlcg_flag)) {
		946	amdgpu_virt_rlcg_reg_rw(adev, offset, value, rlcg_flag);
		947	return;
		948	}
		949
		950	if (acc_flags & AMDGPU_REGS_NO_KIQ)
		951	WREG32_NO_KIQ(offset, value);
		952	else
		953	WREG32(offset, value);
		954	}
		955
		956	u32 amdgpu_sriov_rreg(struct amdgpu_device *adev,
		957	u32 offset, u32 acc_flags, u32 hwip)
		958	{
		959	u32 rlcg_flag;
		960
		961	if (!amdgpu_sriov_runtime(adev) &&
		962	amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag))
		963	return amdgpu_virt_rlcg_reg_rw(adev, offset, 0, rlcg_flag);
		964
		965	if (acc_flags & AMDGPU_REGS_NO_KIQ)
		966	return RREG32_NO_KIQ(offset);
		967	else
		968	return RREG32(offset);
		969	}

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h+24−2

 #define AMDGPU_PASSTHROUGH_MODE        (1 << 3) /* thw whole GPU is pass through for VM */
 #define AMDGPU_SRIOV_CAPS_RUNTIME      (1 << 4) /* is out of full access mode */
+/* flags for indirect register access path supported by rlcg for sriov */
+#define AMDGPU_RLCG_GC_WRITE_LEGACY    (0x8 << 28)
+#define AMDGPU_RLCG_GC_WRITE           (0x0 << 28)
+#define AMDGPU_RLCG_GC_READ            (0x1 << 28)
+#define AMDGPU_RLCG_MMHUB_WRITE        (0x2 << 28)
+/* error code for indirect register access path supported by rlcg for sriov */
+#define AMDGPU_RLCG_VFGATE_DISABLED		0x4000000
+#define AMDGPU_RLCG_WRONG_OPERATION_TYPE	0x2000000
+#define AMDGPU_RLCG_REG_NOT_IN_RANGE		0x1000000
+#define AMDGPU_RLCG_SCRATCH1_ADDRESS_MASK	0xFFFFF
 /* all asic after AI use this offset */
 #define mmRCC_IOV_FUNC_IDENTIFIER 0xDE5
 /* tonga/fiji use this offset */
 (amdgpu_sriov_vf((adev)) && \
 	((adev)->virt.reg_access & (AMDGIM_FEATURE_GC_REG_RLC_EN)))
+#define amdgpu_sriov_rlcg_error_report_enabled(adev) \
+        (amdgpu_sriov_reg_indirect_mmhub(adev) || amdgpu_sriov_reg_indirect_gc(adev))
 #define amdgpu_passthrough(adev) \
 ((adev)->virt.caps & AMDGPU_PASSTHROUGH_MODE)
 static inline bool is_virtual_machine(void)
 {
-#ifdef CONFIG_X86
+#if defined(CONFIG_X86)
 	return boot_cpu_has(X86_FEATURE_HYPERVISOR);
+#elif defined(CONFIG_ARM64)
+	return !is_kernel_in_hyp_mode();
 #else
 	return false;
 #endif
 	((!amdgpu_in_reset(adev)) && adev->virt.tdr_debug)
 #define amdgpu_sriov_is_normal(adev) \
 	((!amdgpu_in_reset(adev)) && (!adev->virt.tdr_debug))
 bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev);
 void amdgpu_virt_init_setting(struct amdgpu_device *adev);
 void amdgpu_virt_kiq_reg_write_reg_wait(struct amdgpu_device *adev,
 void amdgpu_virt_update_sriov_video_codec(struct amdgpu_device *adev,
 			struct amdgpu_video_codec_info *encode, uint32_t encode_array_size,
 			struct amdgpu_video_codec_info *decode, uint32_t decode_array_size);
+void amdgpu_sriov_wreg(struct amdgpu_device *adev,
+		       u32 offset, u32 value,
+		       u32 acc_flags, u32 hwip);
+u32 amdgpu_sriov_rreg(struct amdgpu_device *adev,
+		      u32 offset, u32 acc_flags, u32 hwip);
 #endif

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c+9−3

 	if (bo->tbo.base.resv != vm->root.bo->tbo.base.resv)
 		return;
+	dma_resv_assert_held(vm->root.bo->tbo.base.resv);
 	vm->bulk_moveable = false;
 	if (bo->tbo.type == ttm_bo_type_kernel && bo->parent)
 		amdgpu_vm_bo_relocated(base);
 			nptes = max(nptes, 1u);
 			trace_amdgpu_vm_update_ptes(params, frag_start, upd_end,
-						    nptes, dst, incr, upd_flags,
+						    min(nptes, 32u), dst, incr, upd_flags,
 						    vm->task_info.pid,
 						    vm->immediate.fence_context);
 			amdgpu_vm_update_flags(params, to_amdgpu_bo_vm(pt),
 	if (!bo)
 		return bo_va;
+	dma_resv_assert_held(bo->tbo.base.resv);
 	if (amdgpu_dmabuf_is_xgmi_accessible(adev, bo)) {
 		bo_va->is_xgmi = true;
 		/* Power up XGMI if it can be potentially used */
 }
 /**
- * amdgpu_vm_bo_rmv - remove a bo to a specific vm
+ * amdgpu_vm_bo_del - remove a bo from a specific vm
  *
  * @adev: amdgpu_device pointer
  * @bo_va: requested bo_va
  *
  * Object have to be reserved!
  */
-void amdgpu_vm_bo_rmv(struct amdgpu_device *adev,
+void amdgpu_vm_bo_del(struct amdgpu_device *adev,
 		      struct amdgpu_bo_va *bo_va)
 {
 	struct amdgpu_bo_va_mapping *mapping, *next;
 	struct amdgpu_vm *vm = bo_va->base.vm;
 	struct amdgpu_vm_bo_base **base;
+	dma_resv_assert_held(vm->root.bo->tbo.base.resv);
 	if (bo) {
+		dma_resv_assert_held(bo->tbo.base.resv);
 		if (bo->tbo.base.resv == vm->root.bo->tbo.base.resv)
 			vm->bulk_moveable = false;

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h+1−1

@@ -435,7 +435,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, · 434 unmodified lines
435	struct amdgpu_bo_va_mapping amdgpu_vm_bo_lookup_mapping(struct amdgpu_vm vm,	435	struct amdgpu_bo_va_mapping amdgpu_vm_bo_lookup_mapping(struct amdgpu_vm vm,
436	uint64_t addr);	436	uint64_t addr);
437	void amdgpu_vm_bo_trace_cs(struct amdgpu_vm vm, struct ww_acquire_ctx ticket);	437	void amdgpu_vm_bo_trace_cs(struct amdgpu_vm vm, struct ww_acquire_ctx ticket);
438	void amdgpu_vm_bo_rmv(struct amdgpu_device *adev,	438	void amdgpu_vm_bo_del(struct amdgpu_device *adev,
439	struct amdgpu_bo_va *bo_va);	439	struct amdgpu_bo_va *bo_va);
440	void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint32_t min_vm_size,	440	void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint32_t min_vm_size,
441	uint32_t fragment_size_default, unsigned max_level,	441	uint32_t fragment_size_default, unsigned max_level,

drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c+22−38

 {
 	struct drm_device *ddev = dev_get_drvdata(dev);
 	struct amdgpu_device *adev = drm_to_adev(ddev);
+	struct ttm_resource_manager *man = &adev->mman.vram_mgr.manager;
-	return sysfs_emit(buf, "%llu\n",
+	return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man));
-			  amdgpu_vram_mgr_usage(&adev->mman.vram_mgr));
 }
 /**
 		vis_usage = amdgpu_vram_mgr_vis_size(adev, &rsv->mm_node);
 		atomic64_add(vis_usage, &mgr->vis_usage);
-		atomic64_add(rsv->mm_node.size << PAGE_SHIFT, &mgr->usage);
+		spin_lock(&man->bdev->lru_lock);
+		man->usage += rsv->mm_node.size << PAGE_SHIFT;
+		spin_unlock(&man->bdev->lru_lock);
 		list_move(&rsv->node, &mgr->reserved_pages);
 	}
 }
 	rsv->mm_node.size = size >> PAGE_SHIFT;
 	spin_lock(&mgr->lock);
-	list_add_tail(&mgr->reservations_pending, &rsv->node);
+	list_add_tail(&rsv->node, &mgr->reservations_pending);
 	amdgpu_vram_mgr_do_reserve(&mgr->manager);
 	spin_unlock(&mgr->lock);
 	lpfn = place->lpfn;
 	if (!lpfn)
-		lpfn = man->size;
+		lpfn = man->size >> PAGE_SHIFT;
 	max_bytes = adev->gmc.mc_vram_size;
 	if (tbo->type != ttm_bo_type_kernel)
 		max_bytes -= AMDGPU_VM_RESERVED_VRAM;
-	/* bail out quickly if there's likely not enough VRAM for this BO */
 	mem_bytes = tbo->base.size;
-	if (atomic64_add_return(mem_bytes, &mgr->usage) > max_bytes) {
-		r = -ENOSPC;
-		goto error_sub;
-	}
 	if (place->flags & TTM_PL_FLAG_CONTIGUOUS) {
 		pages_per_node = ~0ul;
 		num_nodes = 1;
 	node = kvmalloc(struct_size(node, mm_nodes, num_nodes),
 			GFP_KERNEL | __GFP_ZERO);
-	if (!node) {
+	if (!node)
-		r = -ENOMEM;
+		return -ENOMEM;
-		goto error_sub;
-	}
 	ttm_resource_init(tbo, place, &node->base);
+	/* bail out quickly if there's likely not enough VRAM for this BO */
+	if (ttm_resource_manager_usage(man) > max_bytes) {
+		r = -ENOSPC;
+		goto error_fini;
+	}
 	mode = DRM_MM_INSERT_BEST;
 	if (place->flags & TTM_PL_FLAG_TOPDOWN)
 		mode = DRM_MM_INSERT_HIGH;
 	while (i--)
 		drm_mm_remove_node(&node->mm_nodes[i]);
 	spin_unlock(&mgr->lock);
+error_fini:
 	ttm_resource_fini(man, &node->base);
 	kvfree(node);
-error_sub:
-	atomic64_sub(mem_bytes, &mgr->usage);
 	return r;
 }
 	struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res);
 	struct amdgpu_vram_mgr *mgr = to_vram_mgr(man);
 	struct amdgpu_device *adev = to_amdgpu_device(mgr);
-	uint64_t usage = 0, vis_usage = 0;
+	uint64_t vis_usage = 0;
 	unsigned i, pages;
 	spin_lock(&mgr->lock);
 		struct drm_mm_node *mm = &node->mm_nodes[i];
 		drm_mm_remove_node(mm);
-		usage += mm->size << PAGE_SHIFT;
 		vis_usage += amdgpu_vram_mgr_vis_size(adev, mm);
 	}
 	amdgpu_vram_mgr_do_reserve(man);
 	spin_unlock(&mgr->lock);
-	atomic64_sub(usage, &mgr->usage);
 	atomic64_sub(vis_usage, &mgr->vis_usage);
 	ttm_resource_fini(man, res);
 	kfree(sgt);
 }
-/**
- * amdgpu_vram_mgr_usage - how many bytes are used in this domain
- *
- * @mgr: amdgpu_vram_mgr pointer
- *
- * Returns how many bytes are used in this domain.
- */
-uint64_t amdgpu_vram_mgr_usage(struct amdgpu_vram_mgr *mgr)
-{
-	return atomic64_read(&mgr->usage);
-}
 /**
  * amdgpu_vram_mgr_vis_usage - how many bytes are used in the visible part
  *
 {
 	struct amdgpu_vram_mgr *mgr = to_vram_mgr(man);
+	drm_printf(printer, "  vis usage:%llu\n",
+		   amdgpu_vram_mgr_vis_usage(mgr));
 	spin_lock(&mgr->lock);
 	drm_mm_print(&mgr->mm, printer);
 	spin_unlock(&mgr->lock);
-	drm_printf(printer, "man size:%llu pages, ram usage:%lluMB, vis usage:%lluMB\n",
-		   man->size, amdgpu_vram_mgr_usage(mgr) >> 20,
-		   amdgpu_vram_mgr_vis_usage(mgr) >> 20);
 }
 static const struct ttm_resource_manager_func amdgpu_vram_mgr_func = {
 	struct ttm_resource_manager *man = &mgr->manager;
 	ttm_resource_manager_init(man, &adev->mman.bdev,
-				  adev->gmc.real_vram_size >> PAGE_SHIFT);
+				  adev->gmc.real_vram_size);
 	man->func = &amdgpu_vram_mgr_func;
-	drm_mm_init(&mgr->mm, 0, man->size);
+	drm_mm_init(&mgr->mm, 0, man->size >> PAGE_SHIFT);
 	spin_lock_init(&mgr->lock);
 	INIT_LIST_HEAD(&mgr->reservations_pending);
 	INIT_LIST_HEAD(&mgr->reserved_pages);

drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c+75−44

 #include "wafl/wafl2_4_0_0_smn.h"
 #include "wafl/wafl2_4_0_0_sh_mask.h"
+#include "amdgpu_reset.h"
 #define smnPCS_XGMI23_PCS_ERROR_STATUS   0x11a01210
 #define smnPCS_XGMI3X16_PCS_ERROR_STATUS 0x11a0020c
 #define smnPCS_GOPX1_PCS_ERROR_STATUS    0x12200210
 	struct amdgpu_hive_info *hive = container_of(
 		kobj, struct amdgpu_hive_info, kobj);
+	amdgpu_reset_put_reset_domain(hive->reset_domain);
+	hive->reset_domain = NULL;
 	mutex_destroy(&hive->hive_lock);
 	kfree(hive);
 }
 		goto pro_end;
 	}
+	/**
+	 * Avoid recreating reset domain when hive is reconstructed for the case
+	 * of reset the devices in the XGMI hive during probe for SRIOV
+	 * See https://www.spinics.net/lists/amd-gfx/msg58836.html
+	 */
+	if (adev->reset_domain->type != XGMI_HIVE) {
+		hive->reset_domain = amdgpu_reset_create_reset_domain(XGMI_HIVE, "amdgpu-reset-hive");
+			if (!hive->reset_domain) {
+				dev_err(adev->dev, "XGMI: failed initializing reset domain for xgmi hive\n");
+				ret = -ENOMEM;
+				kobject_put(&hive->kobj);
+				kfree(hive);
+				hive = NULL;
+				goto pro_end;
+			}
+	} else {
+		amdgpu_reset_get_reset_domain(adev->reset_domain);
+		hive->reset_domain = adev->reset_domain;
+	}
 	hive->hive_id = adev->gmc.xgmi.hive_id;
 	INIT_LIST_HEAD(&hive->device_list);
 	INIT_LIST_HEAD(&hive->node);
 	mutex_init(&hive->hive_lock);
-	atomic_set(&hive->in_reset, 0);
 	atomic_set(&hive->number_devices, 0);
 	task_barrier_init(&hive->tb);
 	hive->pstate = AMDGPU_XGMI_PSTATE_UNKNOWN;
 	hive->hi_req_gpu = NULL;
 	/*
 	 * hive pstate on boot is high in vega20 so we have to go to low
 	 * pstate on after boot.
 	return psp_xgmi_terminate(&adev->psp);
 }
-static int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev)
+static int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block)
 {
-	int r;
-	struct ras_ih_if ih_info = {
-		.cb = NULL,
-	};
-	struct ras_fs_if fs_info = {
-		.sysfs_name = "xgmi_wafl_err_count",
-	};
 	if (!adev->gmc.xgmi.supported ||
 	    adev->gmc.xgmi.num_physical_nodes == 0)
 		return 0;
-	adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev);
+	adev->gmc.xgmi.ras->ras_block.hw_ops->reset_ras_error_count(adev);
-	if (!adev->gmc.xgmi.ras_if) {
-		adev->gmc.xgmi.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
-		if (!adev->gmc.xgmi.ras_if)
-			return -ENOMEM;
-		adev->gmc.xgmi.ras_if->block = AMDGPU_RAS_BLOCK__XGMI_WAFL;
-		adev->gmc.xgmi.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
-		adev->gmc.xgmi.ras_if->sub_block_index = 0;
-	}
-	ih_info.head = fs_info.head = *adev->gmc.xgmi.ras_if;
-	r = amdgpu_ras_late_init(adev, adev->gmc.xgmi.ras_if,
-				 &fs_info, &ih_info);
-	if (r || !amdgpu_ras_is_supported(adev, adev->gmc.xgmi.ras_if->block)) {
-		kfree(adev->gmc.xgmi.ras_if);
-		adev->gmc.xgmi.ras_if = NULL;
-	}
-	return r;
+	return amdgpu_ras_block_late_init(adev, ras_block);
 }
 static void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev)
 {
 	if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL) &&
-			adev->gmc.xgmi.ras_if) {
+			adev->gmc.xgmi.ras_if)
-		struct ras_common_if *ras_if = adev->gmc.xgmi.ras_if;
+		amdgpu_ras_block_late_fini(adev, adev->gmc.xgmi.ras_if);
-		struct ras_ih_if ih_info = {
-			.cb = NULL,
-		};
-		amdgpu_ras_late_fini(adev, ras_if, &ih_info);
-		kfree(ras_if);
-	}
 }
 uint64_t amdgpu_xgmi_get_relative_phy_addr(struct amdgpu_device *adev,
 	return 0;
 }
-static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
+static void amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
 					     void *ras_error_status)
 {
 	struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
 	uint32_t ue_cnt = 0, ce_cnt = 0;
 	if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL))
-		return -EINVAL;
+		return ;
 	err_data->ue_count = 0;
 	err_data->ce_count = 0;
 		break;
 	}
-	adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev);
+	adev->gmc.xgmi.ras->ras_block.hw_ops->reset_ras_error_count(adev);
 	err_data->ue_count += ue_cnt;
 	err_data->ce_count += ce_cnt;
+}
-	return 0;
+/* Trigger XGMI/WAFL error */
+static int amdgpu_ras_error_inject_xgmi(struct amdgpu_device *adev,  void *inject_if)
+{
+	int ret = 0;
+	struct ta_ras_trigger_error_input *block_info =
+				(struct ta_ras_trigger_error_input *)inject_if;
+	if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_DISALLOW))
+		dev_warn(adev->dev, "Failed to disallow df cstate");
+	if (amdgpu_dpm_allow_xgmi_power_down(adev, false))
+		dev_warn(adev->dev, "Failed to disallow XGMI power down");
+	ret = psp_ras_trigger_error(&adev->psp, block_info);
+	if (amdgpu_ras_intr_triggered())
+		return ret;
+	if (amdgpu_dpm_allow_xgmi_power_down(adev, true))
+		dev_warn(adev->dev, "Failed to allow XGMI power down");
+	if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_ALLOW))
+		dev_warn(adev->dev, "Failed to allow df cstate");
+	return ret;
 }
-const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs = {
+struct amdgpu_ras_block_hw_ops  xgmi_ras_hw_ops = {
-	.ras_late_init = amdgpu_xgmi_ras_late_init,
-	.ras_fini = amdgpu_xgmi_ras_fini,
 	.query_ras_error_count = amdgpu_xgmi_query_ras_error_count,
 	.reset_ras_error_count = amdgpu_xgmi_reset_ras_error_count,
+	.ras_error_inject = amdgpu_ras_error_inject_xgmi,
+};
+struct amdgpu_xgmi_ras xgmi_ras = {
+	.ras_block = {
+		.ras_comm = {
+			.name = "xgmi_wafl",
+			.block = AMDGPU_RAS_BLOCK__XGMI_WAFL,
+			.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE,
+		},
+		.hw_ops = &xgmi_ras_hw_ops,
+		.ras_late_init = amdgpu_xgmi_ras_late_init,
+		.ras_fini = amdgpu_xgmi_ras_fini,
+	},
 };

drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.h+4−3

@@ -24,7 +24,7 @@ · 23 unmodified lines
24		24
25	#include <drm/task_barrier.h>	25	#include <drm/task_barrier.h>
26	#include "amdgpu_psp.h"	26	#include "amdgpu_psp.h"
27		27	#include "amdgpu_ras.h"
28		28
29	struct amdgpu_hive_info {	29	struct amdgpu_hive_info {
30	struct kobject kobj;	30	struct kobject kobj;
@@ -33,7 +33,6 @@ struct amdgpu_hive_info { · 2 unmodified lines
33	struct list_head node;	33	struct list_head node;
34	atomic_t number_devices;	34	atomic_t number_devices;
35	struct mutex hive_lock;	35	struct mutex hive_lock;
36	atomic_t in_reset;
37	int hi_req_count;	36	int hi_req_count;
38	struct amdgpu_device *hi_req_gpu;	37	struct amdgpu_device *hi_req_gpu;
39	struct task_barrier tb;	38	struct task_barrier tb;
@@ -42,6 +41,8 @@ struct amdgpu_hive_info { · 2 unmodified lines
42	AMDGPU_XGMI_PSTATE_MAX_VEGA20,	41	AMDGPU_XGMI_PSTATE_MAX_VEGA20,
43	AMDGPU_XGMI_PSTATE_UNKNOWN	42	AMDGPU_XGMI_PSTATE_UNKNOWN
44	} pstate;	43	} pstate;
		44
		45	struct amdgpu_reset_domain *reset_domain;
45	};	46	};
46		47
47	struct amdgpu_pcs_ras_field {	48	struct amdgpu_pcs_ras_field {
@@ -50,7 +51,7 @@ struct amdgpu_pcs_ras_field { · 2 unmodified lines
50	uint32_t pcs_err_shift;	51	uint32_t pcs_err_shift;
51	};	52	};
52		53
53	extern const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs;	54	extern struct amdgpu_xgmi_ras xgmi_ras;
54	struct amdgpu_hive_info amdgpu_get_xgmi_hive(struct amdgpu_device adev);	55	struct amdgpu_hive_info amdgpu_get_xgmi_hive(struct amdgpu_device adev);
55	void amdgpu_put_xgmi_hive(struct amdgpu_hive_info *hive);	56	void amdgpu_put_xgmi_hive(struct amdgpu_hive_info *hive);
56	int amdgpu_xgmi_update_topology(struct amdgpu_hive_info hive, struct amdgpu_device adev);	57	int amdgpu_xgmi_update_topology(struct amdgpu_hive_info hive, struct amdgpu_device adev);

drivers/gpu/drm/amd/amdgpu/athub_v1_0.c+7−6

@@ -68,12 +68,13 @@ int athub_v1_0_set_clockgating(struct amdgpu_device *adev, · 67 unmodified lines
68	if (amdgpu_sriov_vf(adev))	68	if (amdgpu_sriov_vf(adev))
69	return 0;	69	return 0;
70		70
71	switch (adev->asic_type) {	71	switch (adev->ip_versions[ATHUB_HWIP][0]) {
72	case CHIP_VEGA10:	72	case IP_VERSION(9, 0, 0):
73	case CHIP_VEGA12:	73	case IP_VERSION(9, 1, 0):
74	case CHIP_VEGA20:	74	case IP_VERSION(9, 2, 0):
75	case CHIP_RAVEN:	75	case IP_VERSION(9, 3, 0):
76	case CHIP_RENOIR:	76	case IP_VERSION(9, 4, 0):
		77	case IP_VERSION(1, 5, 0):
77	athub_update_medium_grain_clock_gating(adev,	78	athub_update_medium_grain_clock_gating(adev,
78	state == AMD_CG_STATE_GATE);	79	state == AMD_CG_STATE_GATE);
79	athub_update_medium_grain_light_sleep(adev,	80	athub_update_medium_grain_light_sleep(adev,

drivers/gpu/drm/amd/amdgpu/athub_v2_0.c+1−0

@@ -78,6 +78,7 @@ int athub_v2_0_set_clockgating(struct amdgpu_device *adev, · 77 unmodified lines
78	return 0;	78	return 0;
79		79
80	switch (adev->ip_versions[ATHUB_HWIP][0]) {	80	switch (adev->ip_versions[ATHUB_HWIP][0]) {
		81	case IP_VERSION(1, 3, 1):
81	case IP_VERSION(2, 0, 0):	82	case IP_VERSION(2, 0, 0):
82	case IP_VERSION(2, 0, 2):	83	case IP_VERSION(2, 0, 2):
83	athub_v2_0_update_medium_grain_clock_gating(adev,	84	athub_v2_0_update_medium_grain_clock_gating(adev,

drivers/gpu/drm/amd/amdgpu/athub_v2_1.c+1−0

@@ -74,6 +74,7 @@ int athub_v2_1_set_clockgating(struct amdgpu_device *adev, · 73 unmodified lines
74	case IP_VERSION(2, 1, 0):	74	case IP_VERSION(2, 1, 0):
75	case IP_VERSION(2, 1, 1):	75	case IP_VERSION(2, 1, 1):
76	case IP_VERSION(2, 1, 2):	76	case IP_VERSION(2, 1, 2):
		77	case IP_VERSION(2, 4, 0):
77	athub_v2_1_update_medium_grain_clock_gating(adev, state == AMD_CG_STATE_GATE);	78	athub_v2_1_update_medium_grain_clock_gating(adev, state == AMD_CG_STATE_GATE);
78	athub_v2_1_update_medium_grain_light_sleep(adev, state == AMD_CG_STATE_GATE);	79	athub_v2_1_update_medium_grain_light_sleep(adev, state == AMD_CG_STATE_GATE);
79	break;	80	break;

drivers/gpu/drm/amd/amdgpu/atom.c+3−1

 #include <linux/module.h>
 #include <linux/sched.h>
 #include <linux/slab.h>
+#include <linux/string_helpers.h>
 #include <asm/unaligned.h>
 #include <drm/drm_util.h>
 		break;
 	}
 	if (arg != ATOM_COND_ALWAYS)
-		SDEBUG("   taken: %s\n", execute ? "yes" : "no");
+		SDEBUG("   taken: %s\n", str_yes_no(execute));
 	SDEBUG("   target: 0x%04X\n", target);
 	if (execute) {
 		if (ctx->last_jump == (ctx->start + target)) {

drivers/gpu/drm/amd/amdgpu/cik.c+1−1

@@ -1719,7 +1719,7 @@ static void cik_program_aspm(struct amdgpu_device *adev) · 1718 unmodified lines
1719	bool disable_l0s = false, disable_l1 = false, disable_plloff_in_l1 = false;	1719	bool disable_l0s = false, disable_l1 = false, disable_plloff_in_l1 = false;
1720	bool disable_clkreq = false;	1720	bool disable_clkreq = false;
1721		1721
1722	if (amdgpu_aspm == 0)	1722	if (!amdgpu_device_should_use_aspm(adev))
1723	return;	1723	return;
1724		1724
1725	if (pci_is_root_bus(adev->pdev->bus))	1725	if (pci_is_root_bus(adev->pdev->bus))

drivers/gpu/drm/amd/amdgpu/dce_v10_0.c+3−1

 		break;
 	}
 	/* adjust pm to dpms */
-	amdgpu_pm_compute_clocks(adev);
+	amdgpu_dpm_compute_clocks(adev);
 }
 static void dce_v10_0_crtc_prepare(struct drm_crtc *crtc)
 	adev_to_drm(adev)->mode_config.preferred_depth = 24;
 	adev_to_drm(adev)->mode_config.prefer_shadow = 1;
+	adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
 	adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
 	r = amdgpu_display_modeset_create_props(adev);

drivers/gpu/drm/amd/amdgpu/dce_v11_0.c+3−1

 		break;
 	}
 	/* adjust pm to dpms */
-	amdgpu_pm_compute_clocks(adev);
+	amdgpu_dpm_compute_clocks(adev);
 }
 static void dce_v11_0_crtc_prepare(struct drm_crtc *crtc)
 	adev_to_drm(adev)->mode_config.preferred_depth = 24;
 	adev_to_drm(adev)->mode_config.prefer_shadow = 1;
+	adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
 	adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
 	r = amdgpu_display_modeset_create_props(adev);

drivers/gpu/drm/amd/amdgpu/dce_v6_0.c+2−1

 		break;
 	}
 	/* adjust pm to dpms */
-	amdgpu_pm_compute_clocks(adev);
+	amdgpu_dpm_compute_clocks(adev);
 }
 static void dce_v6_0_crtc_prepare(struct drm_crtc *crtc)
 	adev_to_drm(adev)->mode_config.max_height = 16384;
 	adev_to_drm(adev)->mode_config.preferred_depth = 24;
 	adev_to_drm(adev)->mode_config.prefer_shadow = 1;
+	adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
 	adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
 	r = amdgpu_display_modeset_create_props(adev);

drivers/gpu/drm/amd/amdgpu/dce_v8_0.c+3−1

 		break;
 	}
 	/* adjust pm to dpms */
-	amdgpu_pm_compute_clocks(adev);
+	amdgpu_dpm_compute_clocks(adev);
 }
 static void dce_v8_0_crtc_prepare(struct drm_crtc *crtc)
 	adev_to_drm(adev)->mode_config.preferred_depth = 24;
 	adev_to_drm(adev)->mode_config.prefer_shadow = 1;
+	adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true;
 	adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base;
 	r = amdgpu_display_modeset_create_props(adev);

drivers/gpu/drm/amd/amdgpu/df_v3_6.c+3−3

@@ -458,7 +458,7 @@ static int df_v3_6_pmc_add_cntr(struct amdgpu_device *adev, · 457 unmodified lines
458		458
459	#define DEFERRED_ARM_MASK (1 << 31)	459	#define DEFERRED_ARM_MASK (1 << 31)
460	static int df_v3_6_pmc_set_deferred(struct amdgpu_device *adev,	460	static int df_v3_6_pmc_set_deferred(struct amdgpu_device *adev,
461	int counter_idx, uint64_t config,	461	uint64_t config, int counter_idx,
462	bool is_deferred)	462	bool is_deferred)
463	{	463	{
464		464
@@ -476,8 +476,8 @@ static int df_v3_6_pmc_set_deferred(struct amdgpu_device *adev, · 11 unmodified lines
476	}	476	}
477		477
478	static bool df_v3_6_pmc_is_deferred(struct amdgpu_device *adev,	478	static bool df_v3_6_pmc_is_deferred(struct amdgpu_device *adev,
479	int counter_idx,	479	uint64_t config,
480	uint64_t config)	480	int counter_idx)
481	{	481	{
482	return (df_v3_6_pmc_has_counter(adev, config, counter_idx) &&	482	return (df_v3_6_pmc_has_counter(adev, config, counter_idx) &&
483	(adev->df_perfmon_config_assign_mask[counter_idx]	483	(adev->df_perfmon_config_assign_mask[counter_idx]

drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c+177−157

 #define GFX10_NUM_GFX_RINGS_Sienna_Cichlid	1
 #define GFX10_MEC_HPD_SIZE	2048
-#define RLCG_VFGATE_DISABLED	0x4000000
-#define RLCG_WRONG_OPERATION_TYPE	0x2000000
-#define RLCG_NOT_IN_RANGE	0x1000000
 #define F32_CE_PROGRAM_RAM_SIZE		65536
 #define RLCG_UCODE_LOADING_START_ADDRESS	0x00002000L
 #define mmGOLDEN_TSC_COUNT_UPPER_Vangogh_BASE_IDX       1
 #define mmGOLDEN_TSC_COUNT_LOWER_Vangogh                0x0026
 #define mmGOLDEN_TSC_COUNT_LOWER_Vangogh_BASE_IDX       1
+#define mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6                0x002d
+#define mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6_BASE_IDX       1
+#define mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6                0x002e
+#define mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6_BASE_IDX       1
 #define mmSPI_CONFIG_CNTL_1_Vangogh		 0x2441
 #define mmSPI_CONFIG_CNTL_1_Vangogh_BASE_IDX	 1
 #define mmVGT_TF_MEMORY_BASE_HI_Vangogh          0x2261
 #define mmRLC_SPARE_INT_0_Sienna_Cichlid               0x4ca5
 #define mmRLC_SPARE_INT_0_Sienna_Cichlid_BASE_IDX      1
-#define GFX_RLCG_GC_WRITE_OLD	(0x8 << 28)
-#define GFX_RLCG_GC_WRITE	(0x0 << 28)
-#define GFX_RLCG_GC_READ	(0x1 << 28)
-#define GFX_RLCG_MMHUB_WRITE	(0x2 << 28)
-#define RLCG_ERROR_REPORT_ENABLED(adev) \
-	(amdgpu_sriov_reg_indirect_mmhub(adev) || amdgpu_sriov_reg_indirect_gc(adev))
 MODULE_FIRMWARE("amdgpu/navi10_ce.bin");
 MODULE_FIRMWARE("amdgpu/navi10_pfp.bin");
 MODULE_FIRMWARE("amdgpu/navi10_me.bin");
 MODULE_FIRMWARE("amdgpu/cyan_skillfish2_mec2.bin");
 MODULE_FIRMWARE("amdgpu/cyan_skillfish2_rlc.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_6_ce.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_6_pfp.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_6_me.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_6_mec.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_6_mec2.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_6_rlc.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_7_ce.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_7_pfp.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_7_me.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_7_mec.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_7_mec2.bin");
+MODULE_FIRMWARE("amdgpu/gc_10_3_7_rlc.bin");
 static const struct soc15_reg_golden golden_settings_gc_10_1[] =
 {
 	SOC15_REG_GOLDEN_VALUE(GC, 0, mmCB_HW_CONTROL_4, 0xffffffff, 0x00400014),
 	SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00c00000)
 };
-static bool gfx_v10_get_rlcg_flag(struct amdgpu_device *adev, u32 acc_flags, u32 hwip,
-				 int write, u32 *rlcg_flag)
-{
-	switch (hwip) {
-	case GC_HWIP:
-		if (amdgpu_sriov_reg_indirect_gc(adev)) {
-			*rlcg_flag = write ? GFX_RLCG_GC_WRITE : GFX_RLCG_GC_READ;
-			return true;
-		/* only in new version, AMDGPU_REGS_NO_KIQ and AMDGPU_REGS_RLC enabled simultaneously */
-		} else if ((acc_flags & AMDGPU_REGS_RLC) && !(acc_flags & AMDGPU_REGS_NO_KIQ)) {
-			*rlcg_flag = GFX_RLCG_GC_WRITE_OLD;
-			return true;
-		}
-		break;
-	case MMHUB_HWIP:
-		if (amdgpu_sriov_reg_indirect_mmhub(adev) &&
-		    (acc_flags & AMDGPU_REGS_RLC) && write) {
-			*rlcg_flag = GFX_RLCG_MMHUB_WRITE;
-			return true;
-		}
-		break;
-	default:
-		DRM_DEBUG("Not program register by RLCG\n");
-	}
-	return false;
-}
-static u32 gfx_v10_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag)
-{
-	static void *scratch_reg0;
-	static void *scratch_reg1;
-	static void *scratch_reg2;
-	static void *scratch_reg3;
-	static void *spare_int;
-	static uint32_t grbm_cntl;
-	static uint32_t grbm_idx;
-	uint32_t i = 0;
-	uint32_t retries = 50000;
-	u32 ret = 0;
-	u32 tmp;
-	scratch_reg0 = adev->rmmio +
-		       (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0) * 4;
-	scratch_reg1 = adev->rmmio +
-		       (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1) * 4;
-	scratch_reg2 = adev->rmmio +
-		       (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG2) * 4;
-	scratch_reg3 = adev->rmmio +
-		       (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3) * 4;
-	if (adev->ip_versions[GC_HWIP][0] >= IP_VERSION(10, 3, 0)) {
-		spare_int = adev->rmmio +
-			    (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_0_Sienna_Cichlid_BASE_IDX]
-			     + mmRLC_SPARE_INT_0_Sienna_Cichlid) * 4;
-	} else {
-		spare_int = adev->rmmio +
-			    (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT) * 4;
-	}
-	grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL;
-	grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX;
-	if (offset == grbm_cntl || offset == grbm_idx) {
-		if (offset  == grbm_cntl)
-			writel(v, scratch_reg2);
-		else if (offset == grbm_idx)
-			writel(v, scratch_reg3);
-		writel(v, ((void __iomem *)adev->rmmio) + (offset * 4));
-	} else {
-		writel(v, scratch_reg0);
-		writel(offset | flag, scratch_reg1);
-		writel(1, spare_int);
-		for (i = 0; i < retries; i++) {
-			tmp = readl(scratch_reg1);
-			if (!(tmp & flag))
-				break;
-			udelay(10);
-		}
-		if (i >= retries) {
-			if (RLCG_ERROR_REPORT_ENABLED(adev)) {
-				if (tmp & RLCG_VFGATE_DISABLED)
-					pr_err("The vfgate is disabled, program reg:0x%05x failed!\n", offset);
-				else if (tmp & RLCG_WRONG_OPERATION_TYPE)
-					pr_err("Wrong operation type, program reg:0x%05x failed!\n", offset);
-				else if (tmp & RLCG_NOT_IN_RANGE)
-					pr_err("The register is not in range, program reg:0x%05x failed!\n", offset);
-				else
-					pr_err("Unknown error type, program reg:0x%05x failed!\n", offset);
-			} else
-				pr_err("timeout: rlcg program reg:0x%05x failed!\n", offset);
-		}
-	}
-	ret = readl(scratch_reg0);
-	return ret;
-}
-static void gfx_v10_sriov_wreg(struct amdgpu_device *adev, u32 offset, u32 value, u32 acc_flags, u32 hwip)
-{
-	u32 rlcg_flag;
-	if (!amdgpu_sriov_runtime(adev) &&
-	    gfx_v10_get_rlcg_flag(adev, acc_flags, hwip, 1, &rlcg_flag)) {
-		gfx_v10_rlcg_rw(adev, offset, value, rlcg_flag);
-		return;
-	}
-	if (acc_flags & AMDGPU_REGS_NO_KIQ)
-		WREG32_NO_KIQ(offset, value);
-	else
-		WREG32(offset, value);
-}
-static u32 gfx_v10_sriov_rreg(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip)
-{
-	u32 rlcg_flag;
-	if (!amdgpu_sriov_runtime(adev) &&
-	    gfx_v10_get_rlcg_flag(adev, acc_flags, hwip, 0, &rlcg_flag))
-		return gfx_v10_rlcg_rw(adev, offset, 0, rlcg_flag);
-	if (acc_flags & AMDGPU_REGS_NO_KIQ)
-		return RREG32_NO_KIQ(offset);
-	else
-		return RREG32(offset);
-}
 static const struct soc15_reg_golden golden_settings_gc_10_1_nv14[] =
 {
 	/* Pending on emulation bring up */
 	SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0x00800000, 0x00800000)
 };
+static const struct soc15_reg_golden golden_settings_gc_10_3_6[] =
+{
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmCGTT_SPI_CS_CLK_CTRL, 0xff7f0fff, 0x78000100),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmCH_PIPE_STEER, 0x000000ff, 0x00000044),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmCPF_GCR_CNTL, 0x0007ffff, 0x0000c200),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0xffffffff, 0x00000280),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG4, 0xffffffff, 0x00800000),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGB_ADDR_CONFIG, 0x0c1807ff, 0x00000042),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL, 0x1ff1ffff, 0x00000500),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL1_PIPE_STEER, 0x000000ff, 0x00000044),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_0, 0x77777777, 0x32103210),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_1, 0x77777777, 0x32103210),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2A_ADDR_MATCH_MASK, 0xffffffff, 0xfffffff3),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_ADDR_MATCH_MASK, 0xffffffff, 0xfffffff3),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CM_CTRL1, 0xff8fff0f, 0x580f1008),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CTRL3, 0xf7ffffff, 0x00f80988),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmLDS_CONFIG, 0x000001ff, 0x00000020),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_CL_ENHANCE, 0xf17fffff, 0x01200007),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_BINNER_TIMEOUT_COUNTER, 0xffffffff, 0x00000800),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_ENHANCE_2, 0xffffffbf, 0x00000820),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmSQG_CONFIG, 0x000017ff, 0x00001000),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmSX_DEBUG_1, 0xffffff7f, 0x00010020),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmTA_CNTL_AUX, 0xfff7ffff, 0x01030000),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00100000)
+};
+static const struct soc15_reg_golden golden_settings_gc_10_3_7[] = {
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmCGTT_SPI_CS_CLK_CTRL, 0xff7f0fff, 0x78000100),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmCH_PIPE_STEER, 0x000000ff, 0x000000e4),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmCPF_GCR_CNTL, 0x0007ffff, 0x0000c200),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0xffffffff, 0x00000280),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG4, 0xffffffff, 0x00800000),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGB_ADDR_CONFIG, 0x0c1807ff, 0x00000041),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL, 0x1ff1ffff, 0x00000500),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL1_PIPE_STEER, 0x000000ff, 0x000000e4),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_0, 0x77777777, 0x32103210),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_1, 0x77777777, 0x32103210),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2A_ADDR_MATCH_MASK, 0xffffffff, 0xffffffff),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_ADDR_MATCH_MASK, 0xffffffff, 0xffffffff),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CM_CTRL1, 0xff8fff0f, 0x580f1008),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CTRL3, 0xf7ffffff, 0x00f80988),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmLDS_CONFIG, 0x000001ff, 0x00000020),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_CL_ENHANCE, 0xf000003f, 0x01200007),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_BINNER_TIMEOUT_COUNTER, 0xffffffff, 0x00000800),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_ENHANCE_2, 0xffffffbf, 0x00000820),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmSQG_CONFIG, 0x000017ff, 0x00001000),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmSX_DEBUG_1, 0xffffff7f, 0x00010020),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmTA_CNTL_AUX, 0xfff7ffff, 0x01030000),
+	SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00100000)
+};
 #define DEFAULT_SH_MEM_CONFIG \
 	((SH_MEM_ADDRESS_MODE_64 << SH_MEM_CONFIG__ADDRESS_MODE__SHIFT) | \
 	 (SH_MEM_ALIGNMENT_MODE_UNALIGNED << SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT) | \
 						(const u32)ARRAY_SIZE(golden_settings_gc_10_3_5));
 		break;
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 		soc15_program_register_sequence(adev,
 						golden_settings_gc_10_0_cyan_skillfish,
 						(const u32)ARRAY_SIZE(golden_settings_gc_10_0_cyan_skillfish));
 		break;
+	case IP_VERSION(10, 3, 6):
+		soc15_program_register_sequence(adev,
+						golden_settings_gc_10_3_6,
+						(const u32)ARRAY_SIZE(golden_settings_gc_10_3_6));
+		break;
+	case IP_VERSION(10, 3, 7):
+		soc15_program_register_sequence(adev,
+						golden_settings_gc_10_3_7,
+						(const u32)ARRAY_SIZE(golden_settings_gc_10_3_7));
+		break;
 	default:
 		break;
 	}
 	case IP_VERSION(10, 1, 2):
 	case IP_VERSION(10, 1, 1):
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 		if ((adev->gfx.me_fw_version >= 0x00000046) &&
 		    (adev->gfx.me_feature_version >= 27) &&
 		    (adev->gfx.pfp_fw_version >= 0x00000068) &&
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		adev->gfx.cp_fw_write_wait = true;
 		break;
 	default:
 	case IP_VERSION(10, 3, 3):
 		chip_name = "yellow_carp";
 		break;
+	case IP_VERSION(10, 3, 6):
+		chip_name = "gc_10_3_6";
+		break;
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 		if (adev->apu_flags & AMD_APU_IS_CYAN_SKILLFISH2)
 			chip_name = "cyan_skillfish2";
 		else
 			chip_name = "cyan_skillfish";
 		break;
+	case IP_VERSION(10, 3, 7):
+		chip_name = "gc_10_3_7";
+		break;
 	default:
 		BUG();
 	}
 			(void **)&adev->gfx.rlc.cp_table_ptr);
 }
+static void gfx_v10_0_init_rlcg_reg_access_ctrl(struct amdgpu_device *adev)
+{
+	struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl;
+	reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl;
+	reg_access_ctrl->scratch_reg0 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG0);
+	reg_access_ctrl->scratch_reg1 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG1);
+	reg_access_ctrl->scratch_reg2 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG2);
+	reg_access_ctrl->scratch_reg3 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG3);
+	reg_access_ctrl->grbm_cntl = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL);
+	reg_access_ctrl->grbm_idx = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX);
+	switch (adev->ip_versions[GC_HWIP][0]) {
+		case IP_VERSION(10, 3, 0):
+			reg_access_ctrl->spare_int =
+				SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT_0_Sienna_Cichlid);
+			break;
+		default:
+			reg_access_ctrl->spare_int =
+				SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT);
+			break;
+	}
+	adev->gfx.rlc.rlcg_reg_access_supported = true;
+}
 static int gfx_v10_0_rlc_init(struct amdgpu_device *adev)
 {
 	const struct cs_section_def *cs_data;
 	if (adev->gfx.rlc.funcs->update_spm_vmid)
 		adev->gfx.rlc.funcs->update_spm_vmid(adev, 0xf);
 	return 0;
 }
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		adev->gfx.config.max_hw_contexts = 8;
 		adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
 		adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
 			1 << REG_GET_FIELD(gb_addr_config, GB_ADDR_CONFIG, NUM_PKRS);
 		break;
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 		adev->gfx.config.max_hw_contexts = 8;
 		adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
 		adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
 	case IP_VERSION(10, 1, 1):
 	case IP_VERSION(10, 1, 2):
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 		adev->gfx.me.num_me = 1;
 		adev->gfx.me.num_pipe_per_me = 1;
 		adev->gfx.me.num_queue_per_pipe = 1;
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		adev->gfx.me.num_me = 1;
 		adev->gfx.me.num_pipe_per_me = 1;
 		adev->gfx.me.num_queue_per_pipe = 1;
 	if (r)
 		return r;
-	r = gfx_v10_0_rlc_init(adev);
+	if (adev->gfx.rlc.funcs) {
-	if (r) {
+		if (adev->gfx.rlc.funcs->init) {
-		DRM_ERROR("Failed to init rlc BOs!\n");
+			r = adev->gfx.rlc.funcs->init(adev);
-		return r;
+			if (r) {
+				dev_err(adev->dev, "Failed to init rlc BOs!\n");
+				return r;
+			}
+		}
 	}
 	r = gfx_v10_0_mec_init(adev);
 		for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) {
 			bitmap = i * adev->gfx.config.max_sh_per_se + j;
 			if (((adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 0)) ||
-				(adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3))) &&
+				(adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) ||
+				(adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 6))) &&
 			    ((gfx_v10_3_get_disabled_sa(adev) >> bitmap) & 1))
 				continue;
 			gfx_v10_0_select_se_sh(adev, i, j, 0xffffffff);
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		tmp = REG_SET_FIELD(0, CP_RB_DOORBELL_RANGE_LOWER,
 				    DOORBELL_RANGE_LOWER_Sienna_Cichlid, ring->doorbell_index);
 		WREG32_SOC15(GC, 0, mmCP_RB_DOORBELL_RANGE_LOWER, tmp);
 		case IP_VERSION(10, 3, 1):
 		case IP_VERSION(10, 3, 4):
 		case IP_VERSION(10, 3, 5):
+		case IP_VERSION(10, 3, 6):
 		case IP_VERSION(10, 3, 3):
+		case IP_VERSION(10, 3, 7):
 			WREG32_SOC15(GC, 0, mmCP_MEC_CNTL_Sienna_Cichlid, 0);
 			break;
 		default:
 		case IP_VERSION(10, 3, 1):
 		case IP_VERSION(10, 3, 4):
 		case IP_VERSION(10, 3, 5):
+		case IP_VERSION(10, 3, 6):
 		case IP_VERSION(10, 3, 3):
+		case IP_VERSION(10, 3, 7):
 			WREG32_SOC15(GC, 0, mmCP_MEC_CNTL_Sienna_Cichlid,
 				     (CP_MEC_CNTL__MEC_ME1_HALT_MASK |
 				      CP_MEC_CNTL__MEC_ME2_HALT_MASK));
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
 		tmp = RREG32_SOC15(GC, 0, mmRLC_CP_SCHEDULERS_Sienna_Cichlid);
 		tmp &= 0xffffff00;
 		break;
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 6):
+	case IP_VERSION(10, 3, 7):
 		return true;
 	default:
 		data = RREG32_SOC15(GC, 0, mmVGT_ESGS_RING_SIZE);
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		/* mmVGT_TF_RING_SIZE_UMD -> mmVGT_TF_RING_SIZE */
 		data = (SOC15_REG_OFFSET(GC, 0, mmVGT_TF_RING_SIZE_UMD) <<
 			GRBM_CAM_DATA__CAM_ADDR__SHIFT) |
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
 		if (REG_GET_FIELD(tmp, GRBM_STATUS2, RLC_BUSY_Sienna_Cichlid))
 			grbm_soft_reset = REG_SET_FIELD(grbm_soft_reset,
 		preempt_enable();
 		clock = clock_lo | (clock_hi << 32ULL);
 		break;
+	case IP_VERSION(10, 3, 6):
+		preempt_disable();
+		clock_hi = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6);
+		clock_lo = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6);
+		hi_check = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6);
+		/* The SMUIO TSC clock frequency is 100MHz, which sets 32-bit carry over
+		 * roughly every 42 seconds.
+		 */
+		if (hi_check != clock_hi) {
+			clock_lo = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6);
+			clock_hi = hi_check;
+		}
+		preempt_enable();
+		clock = clock_lo | (clock_hi << 32ULL);
+		break;
 	default:
 		preempt_disable();
 		clock_hi = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER);
 	case IP_VERSION(10, 1, 1):
 	case IP_VERSION(10, 1, 2):
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 		adev->gfx.num_gfx_rings = GFX10_NUM_GFX_RINGS_NV1X;
 		break;
 	case IP_VERSION(10, 3, 0):
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		adev->gfx.num_gfx_rings = GFX10_NUM_GFX_RINGS_Sienna_Cichlid;
 		break;
 	default:
 	gfx_v10_0_set_gds_init(adev);
 	gfx_v10_0_set_rlc_funcs(adev);
+	/* init rlcg reg access ctrl */
+	gfx_v10_0_init_rlcg_reg_access_ctrl(adev);
 	return 0;
 }
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
 		WREG32_SOC15(GC, 0, mmRLC_SAFE_MODE_Sienna_Cichlid, data);
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
 		WREG32_SOC15(GC, 0, mmRLC_SAFE_MODE_Sienna_Cichlid, data);
 		break;
 		switch (adev->ip_versions[GC_HWIP][0]) {
 		case IP_VERSION(10, 3, 1):
 		case IP_VERSION(10, 3, 3):
+		case IP_VERSION(10, 3, 6):
 			data = 0x4E20 & RLC_PG_DELAY_3__CGCG_ACTIVE_BEFORE_CGPG_MASK_Vangogh;
 			WREG32_SOC15(GC, 0, mmRLC_PG_DELAY_3, data);
 			break;
 	.reset = gfx_v10_0_rlc_reset,
 	.start = gfx_v10_0_rlc_start,
 	.update_spm_vmid = gfx_v10_0_update_spm_vmid,
-	.sriov_wreg = gfx_v10_sriov_wreg,
-	.sriov_rreg = gfx_v10_sriov_rreg,
 	.is_rlcg_access_range = gfx_v10_0_is_rlcg_access_range,
 };
 		break;
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 6):
 		gfx_v10_cntl_pg(adev, enable);
 		amdgpu_gfx_off_ctrl(adev, enable);
 		break;
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
 		gfx_v10_0_update_gfx_clock_gating(adev,
 						 state == AMD_CG_STATE_GATE);
 	case IP_VERSION(10, 1, 10):
 	case IP_VERSION(10, 1, 1):
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 	case IP_VERSION(10, 3, 2):
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		adev->gfx.rlc.funcs = &gfx_v10_0_rlc_funcs;
 		break;
 	case IP_VERSION(10, 1, 2):
 		for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) {
 			bitmap = i * adev->gfx.config.max_sh_per_se + j;
 			if (((adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 0)) ||
-				(adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3))) &&
+			     (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) ||
+			     (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 6)) ||
+			     (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 7))) &&
 			    ((gfx_v10_3_get_disabled_sa(adev) >> bitmap) & 1))
 				continue;
 			mask = 1;

drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c+72−162

@@ -63,13 +63,6 @@ · 62 unmodified lines
63	#define mmGCEA_PROBE_MAP 0x070c	63	#define mmGCEA_PROBE_MAP 0x070c
64	#define mmGCEA_PROBE_MAP_BASE_IDX 0	64	#define mmGCEA_PROBE_MAP_BASE_IDX 0
65		65
66	#define GFX9_RLCG_GC_WRITE_OLD (0x8 << 28)
67	#define GFX9_RLCG_GC_WRITE (0x0 << 28)
68	#define GFX9_RLCG_GC_READ (0x1 << 28)
69	#define GFX9_RLCG_VFGATE_DISABLED 0x4000000
70	#define GFX9_RLCG_WRONG_OPERATION_TYPE 0x2000000
71	#define GFX9_RLCG_NOT_IN_RANGE 0x1000000
72
73	MODULE_FIRMWARE("amdgpu/vega10_ce.bin");	66	MODULE_FIRMWARE("amdgpu/vega10_ce.bin");
74	MODULE_FIRMWARE("amdgpu/vega10_pfp.bin");	67	MODULE_FIRMWARE("amdgpu/vega10_pfp.bin");
75	MODULE_FIRMWARE("amdgpu/vega10_me.bin");	68	MODULE_FIRMWARE("amdgpu/vega10_me.bin");
@@ -746,128 +739,6 @@ static const u32 GFX_RLC_SRM_INDEX_CNTL_DATA_OFFSETS[] = · 670 unmodified lines
746	mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0,	739	mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0,
747	};	740	};
748		741
749	static u32 gfx_v9_0_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag)
750	{
751	static void *scratch_reg0;
752	static void *scratch_reg1;
753	static void *scratch_reg2;
754	static void *scratch_reg3;
755	static void *spare_int;
756	static uint32_t grbm_cntl;
757	static uint32_t grbm_idx;
758	uint32_t i = 0;
759	uint32_t retries = 50000;
760	u32 ret = 0;
761	u32 tmp;
762
763	scratch_reg0 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0)*4;
764	scratch_reg1 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1)*4;
765	scratch_reg2 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG2_BASE_IDX] + mmSCRATCH_REG2)*4;
766	scratch_reg3 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG3_BASE_IDX] + mmSCRATCH_REG3)*4;
767	spare_int = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT)*4;
768
769	grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL;
770	grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX;
771
772	if (offset == grbm_cntl \|\| offset == grbm_idx) {
773	if (offset == grbm_cntl)
774	writel(v, scratch_reg2);
775	else if (offset == grbm_idx)
776	writel(v, scratch_reg3);
777
778	writel(v, ((void __iomem )adev->rmmio) + (offset 4));
779	} else {
780	/*
781	* SCRATCH_REG0 = read/write value
782	* SCRATCH_REG1[30:28] = command
783	* SCRATCH_REG1[19:0] = address in dword
784	* SCRATCH_REG1[26:24] = Error reporting
785	*/
786	writel(v, scratch_reg0);
787	writel(offset \| flag, scratch_reg1);
788	writel(1, spare_int);
789
790	for (i = 0; i < retries; i++) {
791	tmp = readl(scratch_reg1);
792	if (!(tmp & flag))
793	break;
794
795	udelay(10);
796	}
797
798	if (i >= retries) {
799	if (amdgpu_sriov_reg_indirect_gc(adev)) {
800	if (tmp & GFX9_RLCG_VFGATE_DISABLED)
801	pr_err("The vfgate is disabled, program reg:0x%05x failed!\n", offset);
802	else if (tmp & GFX9_RLCG_WRONG_OPERATION_TYPE)
803	pr_err("Wrong operation type, program reg:0x%05x failed!\n", offset);
804	else if (tmp & GFX9_RLCG_NOT_IN_RANGE)
805	pr_err("The register is not in range, program reg:0x%05x failed!\n", offset);
806	else
807	pr_err("Unknown error type, program reg:0x%05x failed!\n", offset);
808	} else
809	pr_err("timeout: rlcg program reg:0x%05x failed!\n", offset);
810	}
811	}
812
813	ret = readl(scratch_reg0);
814
815	return ret;
816	}
817
818	static bool gfx_v9_0_get_rlcg_flag(struct amdgpu_device *adev, u32 acc_flags, u32 hwip,
819	int write, u32 *rlcg_flag)
820	{
821
822	switch (hwip) {
823	case GC_HWIP:
824	if (amdgpu_sriov_reg_indirect_gc(adev)) {
825	*rlcg_flag = write ? GFX9_RLCG_GC_WRITE : GFX9_RLCG_GC_READ;
826
827	return true;
828	/* only in new version, AMDGPU_REGS_NO_KIQ and AMDGPU_REGS_RLC enabled simultaneously */
829	} else if ((acc_flags & AMDGPU_REGS_RLC) && !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) {
830	*rlcg_flag = GFX9_RLCG_GC_WRITE_OLD;
831	return true;
832	}
833
834	break;
835	default:
836	return false;
837	}
838
839	return false;
840	}
841
842	static u32 gfx_v9_0_sriov_rreg(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip)
843	{
844	u32 rlcg_flag;
845
846	if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 0, &rlcg_flag))
847	return gfx_v9_0_rlcg_rw(adev, offset, 0, rlcg_flag);
848
849	if (acc_flags & AMDGPU_REGS_NO_KIQ)
850	return RREG32_NO_KIQ(offset);
851	else
852	return RREG32(offset);
853	}
854
855	static void gfx_v9_0_sriov_wreg(struct amdgpu_device *adev, u32 offset,
856	u32 value, u32 acc_flags, u32 hwip)
857	{
858	u32 rlcg_flag;
859
860	if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 1, &rlcg_flag)) {
861	gfx_v9_0_rlcg_rw(adev, offset, value, rlcg_flag);
862	return;
863	}
864
865	if (acc_flags & AMDGPU_REGS_NO_KIQ)
866	WREG32_NO_KIQ(offset, value);
867	else
868	WREG32(offset, value);
869	}
870
871	#define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042	742	#define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042
872	#define VEGA12_GB_ADDR_CONFIG_GOLDEN 0x24104041	743	#define VEGA12_GB_ADDR_CONFIG_GOLDEN 0x24104041
873	#define RAVEN_GB_ADDR_CONFIG_GOLDEN 0x24000042	744	#define RAVEN_GB_ADDR_CONFIG_GOLDEN 0x24000042
@@ -882,7 +753,7 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev, · 8 unmodified lines
882	static uint64_t gfx_v9_0_get_gpu_clock_counter(struct amdgpu_device *adev);	753	static uint64_t gfx_v9_0_get_gpu_clock_counter(struct amdgpu_device *adev);
883	static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring);	754	static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring);
884	static u64 gfx_v9_0_ring_get_rptr_compute(struct amdgpu_ring *ring);	755	static u64 gfx_v9_0_ring_get_rptr_compute(struct amdgpu_ring *ring);
885	static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev,	756	static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev,
886	void *ras_error_status);	757	void *ras_error_status);
887	static int gfx_v9_0_ras_error_inject(struct amdgpu_device *adev,	758	static int gfx_v9_0_ras_error_inject(struct amdgpu_device *adev,
888	void *inject_if);	759	void *inject_if);
@@ -2008,6 +1879,21 @@ static int gfx_v9_0_cp_jump_table_num(struct amdgpu_device *adev) · 1119 unmodified lines
2008	return 4;	1879	return 4;
2009	}	1880	}
2010		1881
		1882	static void gfx_v9_0_init_rlcg_reg_access_ctrl(struct amdgpu_device *adev)
		1883	{
		1884	struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl;
		1885
		1886	reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl;
		1887	reg_access_ctrl->scratch_reg0 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG0);
		1888	reg_access_ctrl->scratch_reg1 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG1);
		1889	reg_access_ctrl->scratch_reg2 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG2);
		1890	reg_access_ctrl->scratch_reg3 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG3);
		1891	reg_access_ctrl->grbm_cntl = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL);
		1892	reg_access_ctrl->grbm_idx = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX);
		1893	reg_access_ctrl->spare_int = SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT);
		1894	adev->gfx.rlc.rlcg_reg_access_supported = true;
		1895	}
		1896
2011	static int gfx_v9_0_rlc_init(struct amdgpu_device *adev)	1897	static int gfx_v9_0_rlc_init(struct amdgpu_device *adev)
2012	{	1898	{
2013	const struct cs_section_def *cs_data;	1899	const struct cs_section_def *cs_data;
@@ -2197,12 +2083,16 @@ static const struct amdgpu_gfx_funcs gfx_v9_0_gfx_funcs = { · 183 unmodified lines
2197	.select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,	2083	.select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,
2198	};	2084	};
2199		2085
2200	static const struct amdgpu_gfx_ras_funcs gfx_v9_0_ras_funcs = {	2086	const struct amdgpu_ras_block_hw_ops gfx_v9_0_ras_ops = {
2201	.ras_late_init = amdgpu_gfx_ras_late_init,	2087	.ras_error_inject = &gfx_v9_0_ras_error_inject,
2202	.ras_fini = amdgpu_gfx_ras_fini,	2088	.query_ras_error_count = &gfx_v9_0_query_ras_error_count,
2203	.ras_error_inject = &gfx_v9_0_ras_error_inject,	2089	.reset_ras_error_count = &gfx_v9_0_reset_ras_error_count,
2204	.query_ras_error_count = &gfx_v9_0_query_ras_error_count,	2090	};
2205	.reset_ras_error_count = &gfx_v9_0_reset_ras_error_count,	2091
		2092	static struct amdgpu_gfx_ras gfx_v9_0_ras = {
		2093	.ras_block = {
		2094	.hw_ops = &gfx_v9_0_ras_ops,
		2095	},
2206	};	2096	};
2207		2097
2208	static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev)	2098	static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev)
@@ -2231,7 +2121,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 22 unmodified lines
2231	DRM_INFO("fix gfx.config for vega12\n");	2121	DRM_INFO("fix gfx.config for vega12\n");
2232	break;	2122	break;
2233	case IP_VERSION(9, 4, 0):	2123	case IP_VERSION(9, 4, 0):
2234	adev->gfx.ras_funcs = &gfx_v9_0_ras_funcs;	2124	adev->gfx.ras = &gfx_v9_0_ras;
2235	adev->gfx.config.max_hw_contexts = 8;	2125	adev->gfx.config.max_hw_contexts = 8;
2236	adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;	2126	adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
2237	adev->gfx.config.sc_prim_fifo_size_backend = 0x100;	2127	adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
@@ -2258,7 +2148,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 20 unmodified lines
2258	gb_addr_config = RAVEN_GB_ADDR_CONFIG_GOLDEN;	2148	gb_addr_config = RAVEN_GB_ADDR_CONFIG_GOLDEN;
2259	break;	2149	break;
2260	case IP_VERSION(9, 4, 1):	2150	case IP_VERSION(9, 4, 1):
2261	adev->gfx.ras_funcs = &gfx_v9_4_ras_funcs;	2151	adev->gfx.ras = &gfx_v9_4_ras;
2262	adev->gfx.config.max_hw_contexts = 8;	2152	adev->gfx.config.max_hw_contexts = 8;
2263	adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;	2153	adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
2264	adev->gfx.config.sc_prim_fifo_size_backend = 0x100;	2154	adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
@@ -2279,7 +2169,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 14 unmodified lines
2279	gb_addr_config \|= 0x22010042;	2169	gb_addr_config \|= 0x22010042;
2280	break;	2170	break;
2281	case IP_VERSION(9, 4, 2):	2171	case IP_VERSION(9, 4, 2):
2282	adev->gfx.ras_funcs = &gfx_v9_4_2_ras_funcs;	2172	adev->gfx.ras = &gfx_v9_4_2_ras;
2283	adev->gfx.config.max_hw_contexts = 8;	2173	adev->gfx.config.max_hw_contexts = 8;
2284	adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;	2174	adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
2285	adev->gfx.config.sc_prim_fifo_size_backend = 0x100;	2175	adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
@@ -2298,6 +2188,31 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 12 unmodified lines
2298	break;	2188	break;
2299	}	2189	}
2300		2190
		2191	if (adev->gfx.ras) {
		2192	err = amdgpu_ras_register_ras_block(adev, &adev->gfx.ras->ras_block);
		2193	if (err) {
		2194	DRM_ERROR("Failed to register gfx ras block!\n");
		2195	return err;
		2196	}
		2197
		2198	strcpy(adev->gfx.ras->ras_block.ras_comm.name, "gfx");
		2199	adev->gfx.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__GFX;
		2200	adev->gfx.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
		2201	adev->gfx.ras_if = &adev->gfx.ras->ras_block.ras_comm;
		2202
		2203	/* If not define special ras_late_init function, use gfx default ras_late_init */
		2204	if (!adev->gfx.ras->ras_block.ras_late_init)
		2205	adev->gfx.ras->ras_block.ras_late_init = amdgpu_gfx_ras_late_init;
		2206
		2207	/* If not define special ras_fini function, use gfx default ras_fini */
		2208	if (!adev->gfx.ras->ras_block.ras_fini)
		2209	adev->gfx.ras->ras_block.ras_fini = amdgpu_gfx_ras_fini;
		2210
		2211	/* If not defined special ras_cb function, use default ras_cb */
		2212	if (!adev->gfx.ras->ras_block.ras_cb)
		2213	adev->gfx.ras->ras_block.ras_cb = amdgpu_gfx_process_ras_data_cb;
		2214	}
		2215
2301	adev->gfx.config.gb_addr_config = gb_addr_config;	2216	adev->gfx.config.gb_addr_config = gb_addr_config;
2302		2217
2303	adev->gfx.config.gb_addr_config_fields.num_pipes = 1 <<	2218	adev->gfx.config.gb_addr_config_fields.num_pipes = 1 <<
@@ -2434,10 +2349,14 @@ static int gfx_v9_0_sw_init(void *handle) · 130 unmodified lines
2434	return r;	2349	return r;
2435	}	2350	}
2436		2351
2437	r = adev->gfx.rlc.funcs->init(adev);	2352	if (adev->gfx.rlc.funcs) {
2438	if (r) {	2353	if (adev->gfx.rlc.funcs->init) {
2439	DRM_ERROR("Failed to init rlc BOs!\n");	2354	r = adev->gfx.rlc.funcs->init(adev);
2440	return r;	2355	if (r) {
		2356	dev_err(adev->dev, "Failed to init rlc BOs!\n");
		2357	return r;
		2358	}
		2359	}
2441	}	2360	}
2442		2361
2443	r = gfx_v9_0_mec_init(adev);	2362	r = gfx_v9_0_mec_init(adev);
@@ -2513,9 +2432,8 @@ static int gfx_v9_0_sw_fini(void *handle) · 69 unmodified lines
2513	int i;	2432	int i;
2514	struct amdgpu_device adev = (struct amdgpu_device )handle;	2433	struct amdgpu_device adev = (struct amdgpu_device )handle;
2515		2434
2516	if (adev->gfx.ras_funcs &&	2435	if (adev->gfx.ras && adev->gfx.ras->ras_block.ras_fini)
2517	adev->gfx.ras_funcs->ras_fini)	2436	adev->gfx.ras->ras_block.ras_fini(adev);
2518	adev->gfx.ras_funcs->ras_fini(adev);
2519		2437
2520	for (i = 0; i < adev->gfx.num_gfx_rings; i++)	2438	for (i = 0; i < adev->gfx.num_gfx_rings; i++)
2521	amdgpu_ring_fini(&adev->gfx.gfx_ring[i]);	2439	amdgpu_ring_fini(&adev->gfx.gfx_ring[i]);
@@ -4840,6 +4758,9 @@ static int gfx_v9_0_early_init(void *handle) · 2318 unmodified lines
4840	gfx_v9_0_set_gds_init(adev);	4758	gfx_v9_0_set_gds_init(adev);
4841	gfx_v9_0_set_rlc_funcs(adev);	4759	gfx_v9_0_set_rlc_funcs(adev);
4842		4760
		4761	/* init rlcg reg access ctrl */
		4762	gfx_v9_0_init_rlcg_reg_access_ctrl(adev);
		4763
4843	return 0;	4764	return 0;
4844	}	4765	}
4845		4766
@@ -4870,16 +4791,9 @@ static int gfx_v9_0_ecc_late_init(void *handle) · 24 unmodified lines
4870	if (r)	4791	if (r)
4871	return r;	4792	return r;
4872		4793
4873	if (adev->gfx.ras_funcs &&	4794	if (adev->gfx.ras &&
4874	adev->gfx.ras_funcs->ras_late_init) {	4795	adev->gfx.ras->enable_watchdog_timer)
4875	r = adev->gfx.ras_funcs->ras_late_init(adev);	4796	adev->gfx.ras->enable_watchdog_timer(adev);
4876	if (r)
4877	return r;
4878	}
4879
4880	if (adev->gfx.ras_funcs &&
4881	adev->gfx.ras_funcs->enable_watchdog_timer)
4882	adev->gfx.ras_funcs->enable_watchdog_timer(adev);
4883		4797
4884	return 0;	4798	return 0;
4885	}	4799	}
@@ -5250,8 +5164,6 @@ static const struct amdgpu_rlc_funcs gfx_v9_0_rlc_funcs = { · 364 unmodified lines
5250	.reset = gfx_v9_0_rlc_reset,	5164	.reset = gfx_v9_0_rlc_reset,
5251	.start = gfx_v9_0_rlc_start,	5165	.start = gfx_v9_0_rlc_start,
5252	.update_spm_vmid = gfx_v9_0_update_spm_vmid,	5166	.update_spm_vmid = gfx_v9_0_update_spm_vmid,
5253	.sriov_wreg = gfx_v9_0_sriov_wreg,
5254	.sriov_rreg = gfx_v9_0_sriov_rreg,
5255	.is_rlcg_access_range = gfx_v9_0_is_rlcg_access_range,	5167	.is_rlcg_access_range = gfx_v9_0_is_rlcg_access_range,
5256	};	5168	};
5257		5169
@@ -6819,7 +6731,7 @@ static void gfx_v9_0_reset_ras_error_count(struct amdgpu_device *adev) · 1561 unmodified lines
6819	WREG32_SOC15(GC, 0, mmATC_L2_CACHE_4K_EDC_INDEX, 255);	6731	WREG32_SOC15(GC, 0, mmATC_L2_CACHE_4K_EDC_INDEX, 255);
6820	}	6732	}
6821		6733
6822	static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev,	6734	static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev,
6823	void *ras_error_status)	6735	void *ras_error_status)
6824	{	6736	{
6825	struct ras_err_data err_data = (struct ras_err_data )ras_error_status;	6737	struct ras_err_data err_data = (struct ras_err_data )ras_error_status;
@@ -6828,7 +6740,7 @@ static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, · 2 unmodified lines
6828	uint32_t reg_value;	6740	uint32_t reg_value;
6829		6741
6830	if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))	6742	if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
6831	return -EINVAL;	6743	return;
6832		6744
6833	err_data->ue_count = 0;	6745	err_data->ue_count = 0;
6834	err_data->ce_count = 0;	6746	err_data->ce_count = 0;
@@ -6857,8 +6769,6 @@ static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, · 22 unmodified lines
6857	mutex_unlock(&adev->grbm_idx_mutex);	6769	mutex_unlock(&adev->grbm_idx_mutex);
6858		6770
6859	gfx_v9_0_query_utc_edc_status(adev, err_data);	6771	gfx_v9_0_query_utc_edc_status(adev, err_data);
6860
6861	return 0;
6862	}	6772	}
6863		6773
6864	static void gfx_v9_0_emit_mem_sync(struct amdgpu_ring *ring)	6774	static void gfx_v9_0_emit_mem_sync(struct amdgpu_ring *ring)

drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c+14−10

@@ -863,7 +863,7 @@ static int gfx_v9_4_ras_error_count(struct amdgpu_device *adev, · 862 unmodified lines
863	return 0;	863	return 0;
864	}	864	}
865		865
866	static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,	866	static void gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,
867	void *ras_error_status)	867	void *ras_error_status)
868	{	868	{
869	struct ras_err_data err_data = (struct ras_err_data )ras_error_status;	869	struct ras_err_data err_data = (struct ras_err_data )ras_error_status;
@@ -872,7 +872,7 @@ static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, · 2 unmodified lines
872	uint32_t reg_value;	872	uint32_t reg_value;
873		873
874	if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))	874	if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
875	return -EINVAL;	875	return;
876		876
877	err_data->ue_count = 0;	877	err_data->ue_count = 0;
878	err_data->ce_count = 0;	878	err_data->ce_count = 0;
@@ -903,7 +903,6 @@ static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, · 24 unmodified lines
903		903
904	gfx_v9_4_query_utc_edc_status(adev, err_data);	904	gfx_v9_4_query_utc_edc_status(adev, err_data);
905		905
906	return 0;
907	}	906	}
908		907
909	static void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev)	908	static void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev)
@@ -1029,11 +1028,16 @@ static void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev) · 119 unmodified lines
1029	mutex_unlock(&adev->grbm_idx_mutex);	1028	mutex_unlock(&adev->grbm_idx_mutex);
1030	}	1029	}
1031		1030
1032	const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs = {	1031
1033	.ras_late_init = amdgpu_gfx_ras_late_init,	1032	const struct amdgpu_ras_block_hw_ops gfx_v9_4_ras_ops = {
1034	.ras_fini = amdgpu_gfx_ras_fini,	1033	.ras_error_inject = &gfx_v9_4_ras_error_inject,
1035	.ras_error_inject = &gfx_v9_4_ras_error_inject,	1034	.query_ras_error_count = &gfx_v9_4_query_ras_error_count,
1036	.query_ras_error_count = &gfx_v9_4_query_ras_error_count,	1035	.reset_ras_error_count = &gfx_v9_4_reset_ras_error_count,
1037	.reset_ras_error_count = &gfx_v9_4_reset_ras_error_count,	1036	.query_ras_error_status = &gfx_v9_4_query_ras_error_status,
1038	.query_ras_error_status = &gfx_v9_4_query_ras_error_status,	1037	};
		1038
		1039	struct amdgpu_gfx_ras gfx_v9_4_ras = {
		1040	.ras_block = {
		1041	.hw_ops = &gfx_v9_4_ras_ops,
		1042	},
1039	};	1043	};

drivers/gpu/drm/amd/amdgpu/gfx_v9_4.h+1−1

@@ -24,6 +24,6 @@ · 23 unmodified lines
24	#ifndef __GFX_V9_4_H__	24	#ifndef __GFX_V9_4_H__
25	#define __GFX_V9_4_H__	25	#define __GFX_V9_4_H__
26		26
27	extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs;	27	extern struct amdgpu_gfx_ras gfx_v9_4_ras;
28		28
29	#endif /* __GFX_V9_4_H__ */	29	#endif /* __GFX_V9_4_H__ */

drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c+14−11

@@ -1641,14 +1641,14 @@ static int gfx_v9_4_2_query_utc_edc_count(struct amdgpu_device *adev, · 1640 unmodified lines
1641	return 0;	1641	return 0;
1642	}	1642	}
1643		1643
1644	static int gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev,	1644	static void gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev,
1645	void *ras_error_status)	1645	void *ras_error_status)
1646	{	1646	{
1647	struct ras_err_data err_data = (struct ras_err_data )ras_error_status;	1647	struct ras_err_data err_data = (struct ras_err_data )ras_error_status;
1648	uint32_t sec_count = 0, ded_count = 0;	1648	uint32_t sec_count = 0, ded_count = 0;
1649		1649
1650	if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))	1650	if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
1651	return -EINVAL;	1651	return;
1652		1652
1653	err_data->ue_count = 0;	1653	err_data->ue_count = 0;
1654	err_data->ce_count = 0;	1654	err_data->ce_count = 0;
@@ -1661,7 +1661,6 @@ static int gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev, · 6 unmodified lines
1661	err_data->ce_count += sec_count;	1661	err_data->ce_count += sec_count;
1662	err_data->ue_count += ded_count;	1662	err_data->ue_count += ded_count;
1663		1663
1664	return 0;
1665	}	1664	}
1666		1665
1667	static void gfx_v9_4_2_reset_utc_err_status(struct amdgpu_device *adev)	1666	static void gfx_v9_4_2_reset_utc_err_status(struct amdgpu_device *adev)
@@ -1931,13 +1930,17 @@ static void gfx_v9_4_2_reset_sq_timeout_status(struct amdgpu_device *adev) · 263 unmodified lines
1931	mutex_unlock(&adev->grbm_idx_mutex);	1930	mutex_unlock(&adev->grbm_idx_mutex);
1932	}	1931	}
1933		1932
1934	const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs = {	1933	struct amdgpu_ras_block_hw_ops gfx_v9_4_2_ras_ops = {
1935	.ras_late_init = amdgpu_gfx_ras_late_init,	1934	.ras_error_inject = &gfx_v9_4_2_ras_error_inject,
1936	.ras_fini = amdgpu_gfx_ras_fini,	1935	.query_ras_error_count = &gfx_v9_4_2_query_ras_error_count,
1937	.ras_error_inject = &gfx_v9_4_2_ras_error_inject,	1936	.reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count,
1938	.query_ras_error_count = &gfx_v9_4_2_query_ras_error_count,	1937	.query_ras_error_status = &gfx_v9_4_2_query_ras_error_status,
1939	.reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count,	1938	.reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status,
1940	.query_ras_error_status = &gfx_v9_4_2_query_ras_error_status,	1939	};
1941	.reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status,	1940
		1941	struct amdgpu_gfx_ras gfx_v9_4_2_ras = {
		1942	.ras_block = {
		1943	.hw_ops = &gfx_v9_4_2_ras_ops,
		1944	},
1942	.enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer,	1945	.enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer,
1943	};	1946	};

drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.h+1−1

@@ -31,6 +31,6 @@ void gfx_v9_4_2_init_golden_registers(struct amdgpu_device *adev, · 30 unmodified lines
31	void gfx_v9_4_2_set_power_brake_sequence(struct amdgpu_device *adev);	31	void gfx_v9_4_2_set_power_brake_sequence(struct amdgpu_device *adev);
32	int gfx_v9_4_2_do_edc_gpr_workarounds(struct amdgpu_device *adev);	32	int gfx_v9_4_2_do_edc_gpr_workarounds(struct amdgpu_device *adev);
33		33
34	extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs;	34	extern struct amdgpu_gfx_ras gfx_v9_4_2_ras;
35		35
36	#endif /* __GFX_V9_4_2_H__ */	36	#endif /* __GFX_V9_4_2_H__ */

drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c+6−1

 		adev->gfx.config.max_sh_per_se *
 		adev->gfx.config.max_shader_engines);
-	if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) {
+	switch (adev->ip_versions[GC_HWIP][0]) {
+	case IP_VERSION(10, 3, 1):
+	case IP_VERSION(10, 3, 3):
 		/* Get SA disabled bitmap from eFuse setting */
 		efuse_setting = RREG32_SOC15(GC, 0, mmCC_GC_SA_UNIT_DISABLE);
 		efuse_setting &= CC_GC_SA_UNIT_DISABLE__SA_DISABLE_MASK;
 		disabled_sa = tmp;
 		WREG32_SOC15(GC, 0, mmGCUTCL2_HARVEST_BYPASS_GROUPS_YELLOW_CARP, disabled_sa);
+		break;
+	default:
+		break;
 	}
 }

drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c+47−13

 #include "athub_v2_0.h"
 #include "athub_v2_1.h"
+#include "amdgpu_reset.h"
 #if 0
 static const struct soc15_reg_golden golden_settings_navi10_hdp[] =
 {
 	 */
 	if (adev->gfx.kiq.ring.sched.ready &&
 	    (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) &&
-	    down_read_trylock(&adev->reset_sem)) {
+	    down_read_trylock(&adev->reset_domain->sem)) {
 		struct amdgpu_vmhub *hub = &adev->vmhub[vmhub];
 		const unsigned eng = 17;
 		u32 inv_req = hub->vmhub_funcs->get_invalidate_req(vmid, flush_type);
 		amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req,
 				1 << vmid);
-		up_read(&adev->reset_sem);
+		up_read(&adev->reset_domain->sem);
 		return;
 	}
 		adev->umc.umc_inst_num = UMC_V8_7_UMC_INSTANCE_NUM;
 		adev->umc.channel_offs = UMC_V8_7_PER_CHANNEL_OFFSET_SIENNA;
 		adev->umc.channel_idx_tbl = &umc_v8_7_channel_idx_tbl[0][0];
-		adev->umc.ras_funcs = &umc_v8_7_ras_funcs;
+		adev->umc.ras = &umc_v8_7_ras;
 		break;
 	default:
 		break;
 	}
+	if (adev->umc.ras) {
+		amdgpu_ras_register_ras_block(adev, &adev->umc.ras->ras_block);
+		strcpy(adev->umc.ras->ras_block.ras_comm.name, "umc");
+		adev->umc.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__UMC;
+		adev->umc.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
+		adev->umc.ras_if = &adev->umc.ras->ras_block.ras_comm;
+		/* If don't define special ras_late_init function, use default ras_late_init */
+		if (!adev->umc.ras->ras_block.ras_late_init)
+				adev->umc.ras->ras_block.ras_late_init = amdgpu_umc_ras_late_init;
+		/* If don't define special ras_fini function, use default ras_fini */
+		if (!adev->umc.ras->ras_block.ras_fini)
+				adev->umc.ras->ras_block.ras_fini = amdgpu_umc_ras_fini;
+		/* If not defined special ras_cb function, use default ras_cb */
+		if (!adev->umc.ras->ras_block.ras_cb)
+			adev->umc.ras->ras_block.ras_cb = amdgpu_umc_process_ras_data_cb;
+	}
 }
 	switch (adev->ip_versions[MMHUB_HWIP][0]) {
 	case IP_VERSION(2, 3, 0):
 	case IP_VERSION(2, 4, 0):
+	case IP_VERSION(2, 4, 1):
 		adev->mmhub.funcs = &mmhub_v2_3_funcs;
 		break;
 	default:
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		adev->gfxhub.funcs = &gfxhub_v2_1_funcs;
 		break;
 	default:
 static int gmc_v10_0_early_init(void *handle)
 {
+	int r;
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 	gmc_v10_0_set_mmhub_funcs(adev);
 	adev->gmc.private_aperture_end =
 		adev->gmc.private_aperture_start + (4ULL << 30) - 1;
+	r = amdgpu_gmc_ras_early_init(adev);
+	if (r)
+		return r;
 	return 0;
 }
 	case IP_VERSION(10, 1, 1):
 	case IP_VERSION(10, 1, 2):
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 	case IP_VERSION(10, 3, 0):
 	case IP_VERSION(10, 3, 2):
 	case IP_VERSION(10, 3, 1):
 	case IP_VERSION(10, 3, 4):
 	case IP_VERSION(10, 3, 5):
+	case IP_VERSION(10, 3, 6):
 	case IP_VERSION(10, 3, 3):
+	case IP_VERSION(10, 3, 7):
 		adev->num_vmhubs = 2;
 		/*
 		 * To fulfill 4-level page support,
 		return -EINVAL;
 	}
-	if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev))
+	amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
-		goto skip_pin_bo;
-	r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
-	if (r)
-		return r;
-skip_pin_bo:
 	r = adev->gfxhub.funcs->gart_enable(adev);
 	if (r)
 		return r;
 		 (unsigned)(adev->gmc.gart_size >> 20),
 		 (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo));
-	adev->gart.ready = true;
 	return 0;
 }
 	if (r)
 		return r;
+	if (amdgpu_emu_mode == 1) {
+		r = amdgpu_gmc_vram_checking(adev);
+		if (r)
+			return r;
+	}
 	if (adev->umc.funcs && adev->umc.funcs->init_registers)
 		adev->umc.funcs->init_registers(adev);
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+	if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 1, 3) ||
+	    adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 1, 4))
+		return;
 	adev->mmhub.funcs->get_clockgating(adev, flags);
 	if (adev->ip_versions[ATHUB_HWIP][0] >= IP_VERSION(2, 1, 0))

drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c+6−6

 static int gmc_v6_0_gart_enable(struct amdgpu_device *adev)
 {
 	uint64_t table_addr;
-	int r, i;
 	u32 field;
+	int i;
 	if (adev->gart.bo == NULL) {
 		dev_err(adev->dev, "No VRAM object for PCIE GART.\n");
 		return -EINVAL;
 	}
-	r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
+	amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
-	if (r)
-		return r;
 	table_addr = amdgpu_bo_gpu_offset(adev->gart.bo);
 	dev_info(adev->dev, "PCIE GART of %uM enabled (table at 0x%016llX).\n",
 		 (unsigned)(adev->gmc.gart_size >> 20),
 		 (unsigned long long)table_addr);
-	adev->gart.ready = true;
 	return 0;
 }
 	if (r)
 		return r;
-	return r;
+	if (amdgpu_emu_mode == 1)
+		return amdgpu_gmc_vram_checking(adev);
+	else
+		return r;
 }
 static int gmc_v6_0_hw_fini(void *handle)

drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c+6−7

 static int gmc_v7_0_gart_enable(struct amdgpu_device *adev)
 {
 	uint64_t table_addr;
-	int r, i;
 	u32 tmp, field;
+	int i;
 	if (adev->gart.bo == NULL) {
 		dev_err(adev->dev, "No VRAM object for PCIE GART.\n");
 		return -EINVAL;
 	}
-	r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
+	amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
-	if (r)
-		return r;
 	table_addr = amdgpu_bo_gpu_offset(adev->gart.bo);
 	/* Setup TLB control */
 	DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n",
 		 (unsigned)(adev->gmc.gart_size >> 20),
 		 (unsigned long long)table_addr);
-	adev->gart.ready = true;
 	return 0;
 }
 	if (r)
 		return r;
-	return r;
+	if (amdgpu_emu_mode == 1)
+		return amdgpu_gmc_vram_checking(adev);
+	else
+		return r;
 }
 static int gmc_v7_0_hw_fini(void *handle)

drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c+6−7

 static int gmc_v8_0_gart_enable(struct amdgpu_device *adev)
 {
 	uint64_t table_addr;
-	int r, i;
 	u32 tmp, field;
+	int i;
 	if (adev->gart.bo == NULL) {
 		dev_err(adev->dev, "No VRAM object for PCIE GART.\n");
 		return -EINVAL;
 	}
-	r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
+	amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
-	if (r)
-		return r;
 	table_addr = amdgpu_bo_gpu_offset(adev->gart.bo);
 	/* Setup TLB control */
 	DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n",
 		 (unsigned)(adev->gmc.gart_size >> 20),
 		 (unsigned long long)table_addr);
-	adev->gart.ready = true;
 	return 0;
 }
 	if (r)
 		return r;
-	return r;
+	if (amdgpu_emu_mode == 1)
+		return amdgpu_gmc_vram_checking(adev);
+	else
+		return r;
 }
 static int gmc_v8_0_hw_fini(void *handle)

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c+74−31

 #include "amdgpu_ras.h"
 #include "amdgpu_xgmi.h"
+#include "amdgpu_reset.h"
 /* add these here since we already include dce12 headers and these are for DCN */
 #define mmHUBP0_DCSURF_PRI_VIEWPORT_DIMENSION                                                          0x055d
 #define mmHUBP0_DCSURF_PRI_VIEWPORT_DIMENSION_BASE_IDX                                                 2
 	 */
 	if (adev->gfx.kiq.ring.sched.ready &&
 	    (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) &&
-	    down_read_trylock(&adev->reset_sem)) {
+	    down_read_trylock(&adev->reset_domain->sem)) {
 		uint32_t req = hub->vm_inv_eng0_req + hub->eng_distance * eng;
 		uint32_t ack = hub->vm_inv_eng0_ack + hub->eng_distance * eng;
 		amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req,
 						   1 << vmid);
-		up_read(&adev->reset_sem);
+		up_read(&adev->reset_domain->sem);
 		return;
 	}
 	if (amdgpu_in_reset(adev))
 		return -EIO;
-	if (ring->sched.ready && down_read_trylock(&adev->reset_sem)) {
+	if (ring->sched.ready && down_read_trylock(&adev->reset_domain->sem)) {
 		/* Vega20+XGMI caches PTEs in TC and TLB. Add a
 		 * heavy-weight TLB flush (type 2), which flushes
 		 * both. Due to a race condition with concurrent
 		if (r) {
 			amdgpu_ring_undo(ring);
 			spin_unlock(&adev->gfx.kiq.ring_lock);
-			up_read(&adev->reset_sem);
+			up_read(&adev->reset_domain->sem);
 			return -ETIME;
 		}
 		r = amdgpu_fence_wait_polling(ring, seq, adev->usec_timeout);
 		if (r < 1) {
 			dev_err(adev->dev, "wait for kiq fence error: %ld.\n", r);
-			up_read(&adev->reset_sem);
+			up_read(&adev->reset_domain->sem);
 			return -ETIME;
 		}
-		up_read(&adev->reset_sem);
+		up_read(&adev->reset_domain->sem);
 		return 0;
 	}
 		adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM;
 		adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_VG20;
 		adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0];
-		adev->umc.ras_funcs = &umc_v6_1_ras_funcs;
+		adev->umc.ras = &umc_v6_1_ras;
 		break;
 	case IP_VERSION(6, 1, 2):
 		adev->umc.max_ras_err_cnt_per_query = UMC_V6_1_TOTAL_CHANNEL_NUM;
 		adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM;
 		adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_ARCT;
 		adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0];
-		adev->umc.ras_funcs = &umc_v6_1_ras_funcs;
+		adev->umc.ras = &umc_v6_1_ras;
 		break;
 	case IP_VERSION(6, 7, 0):
-		adev->umc.max_ras_err_cnt_per_query = UMC_V6_7_TOTAL_CHANNEL_NUM;
+		adev->umc.max_ras_err_cnt_per_query =
+			UMC_V6_7_TOTAL_CHANNEL_NUM * UMC_V6_7_BAD_PAGE_NUM_PER_CHANNEL;
 		adev->umc.channel_inst_num = UMC_V6_7_CHANNEL_INSTANCE_NUM;
 		adev->umc.umc_inst_num = UMC_V6_7_UMC_INSTANCE_NUM;
 		adev->umc.channel_offs = UMC_V6_7_PER_CHANNEL_OFFSET;
 		if (!adev->gmc.xgmi.connected_to_cpu)
-			adev->umc.ras_funcs = &umc_v6_7_ras_funcs;
+			adev->umc.ras = &umc_v6_7_ras;
 		if (1 & adev->smuio.funcs->get_die_id(adev))
 			adev->umc.channel_idx_tbl = &umc_v6_7_channel_idx_tbl_first[0][0];
 		else
 	default:
 		break;
 	}
+	if (adev->umc.ras) {
+		amdgpu_ras_register_ras_block(adev, &adev->umc.ras->ras_block);
+		strcpy(adev->umc.ras->ras_block.ras_comm.name, "umc");
+		adev->umc.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__UMC;
+		adev->umc.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
+		adev->umc.ras_if = &adev->umc.ras->ras_block.ras_comm;
+		/* If don't define special ras_late_init function, use default ras_late_init */
+		if (!adev->umc.ras->ras_block.ras_late_init)
+				adev->umc.ras->ras_block.ras_late_init = amdgpu_umc_ras_late_init;
+		/* If don't define special ras_fini function, use default ras_fini */
+		if (!adev->umc.ras->ras_block.ras_fini)
+				adev->umc.ras->ras_block.ras_fini = amdgpu_umc_ras_fini;
+		/* If not defined special ras_cb function, use default ras_cb */
+		if (!adev->umc.ras->ras_block.ras_cb)
+			adev->umc.ras->ras_block.ras_cb = amdgpu_umc_process_ras_data_cb;
+	}
 }
 static void gmc_v9_0_set_mmhub_funcs(struct amdgpu_device *adev)
 {
 	switch (adev->ip_versions[MMHUB_HWIP][0]) {
 	case IP_VERSION(9, 4, 0):
-		adev->mmhub.ras_funcs = &mmhub_v1_0_ras_funcs;
+		adev->mmhub.ras = &mmhub_v1_0_ras;
 		break;
 	case IP_VERSION(9, 4, 1):
-		adev->mmhub.ras_funcs = &mmhub_v9_4_ras_funcs;
+		adev->mmhub.ras = &mmhub_v9_4_ras;
 		break;
 	case IP_VERSION(9, 4, 2):
-		adev->mmhub.ras_funcs = &mmhub_v1_7_ras_funcs;
+		adev->mmhub.ras = &mmhub_v1_7_ras;
 		break;
 	default:
 		/* mmhub ras is not available */
 		break;
 	}
+	if (adev->mmhub.ras) {
+		amdgpu_ras_register_ras_block(adev, &adev->mmhub.ras->ras_block);
+		strcpy(adev->mmhub.ras->ras_block.ras_comm.name, "mmhub");
+		adev->mmhub.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__MMHUB;
+		adev->mmhub.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
+		adev->mmhub.ras_if = &adev->mmhub.ras->ras_block.ras_comm;
+		/* If don't define special ras_fini function, use default ras_fini */
+		if (!adev->mmhub.ras->ras_block.ras_fini)
+			adev->mmhub.ras->ras_block.ras_fini = amdgpu_mmhub_ras_fini;
+	}
 }
 static void gmc_v9_0_set_gfxhub_funcs(struct amdgpu_device *adev)
 static void gmc_v9_0_set_hdp_ras_funcs(struct amdgpu_device *adev)
 {
-	adev->hdp.ras_funcs = &hdp_v4_0_ras_funcs;
+	adev->hdp.ras = &hdp_v4_0_ras;
+	amdgpu_ras_register_ras_block(adev, &adev->hdp.ras->ras_block);
+	adev->hdp.ras_if = &adev->hdp.ras->ras_block.ras_comm;
 }
 static void gmc_v9_0_set_mca_funcs(struct amdgpu_device *adev)
 static int gmc_v9_0_early_init(void *handle)
 {
+	int r;
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 	/* ARCT and VEGA20 don't have XGMI defined in their IP discovery tables */
 	adev->gmc.private_aperture_end =
 		adev->gmc.private_aperture_start + (4ULL << 30) - 1;
+	r = amdgpu_gmc_ras_early_init(adev);
+	if (r)
+		return r;
 	return 0;
 }
 	}
 	if (!amdgpu_persistent_edc_harvesting_supported(adev)) {
-		if (adev->mmhub.ras_funcs &&
+		if (adev->mmhub.ras && adev->mmhub.ras->ras_block.hw_ops &&
-		    adev->mmhub.ras_funcs->reset_ras_error_count)
+		    adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count)
-			adev->mmhub.ras_funcs->reset_ras_error_count(adev);
+			adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(adev);
-		if (adev->hdp.ras_funcs &&
+		if (adev->hdp.ras && adev->hdp.ras->ras_block.hw_ops &&
-		    adev->hdp.ras_funcs->reset_ras_error_count)
+		    adev->hdp.ras->ras_block.hw_ops->reset_ras_error_count)
-			adev->hdp.ras_funcs->reset_ras_error_count(adev);
+			adev->hdp.ras->ras_block.hw_ops->reset_ras_error_count(adev);
 	}
 	r = amdgpu_gmc_ras_late_init(adev);
 		return -EINVAL;
 	}
-	if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev))
+	amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
-		goto skip_pin_bo;
-	r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr);
-	if (r)
-		return r;
-skip_pin_bo:
 	r = adev->gfxhub.funcs->gart_enable(adev);
 	if (r)
 		return r;
 	DRM_INFO("PTB located at 0x%016llX\n",
 			(unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo));
-	adev->gart.ready = true;
 	return 0;
 }
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 	bool value;
-	int i;
+	int i, r;
 	/* The sequence of these two function calls matters.*/
 	gmc_v9_0_init_golden_registers(adev);
 	if (adev->umc.funcs && adev->umc.funcs->init_registers)
 		adev->umc.funcs->init_registers(adev);
-	return gmc_v9_0_gart_enable(adev);
+	r = gmc_v9_0_gart_enable(adev);
+	if (r)
+		return r;
+	if (amdgpu_emu_mode == 1)
+		return amdgpu_gmc_vram_checking(adev);
+	else
+		return r;
 }
 /**

drivers/gpu/drm/amd/amdgpu/hdp_v4_0.c+13−3

 	WREG32_SOC15(HDP, 0, mmHDP_NONSURFACE_BASE_HI, (adev->gmc.vram_start >> 40));
 }
-const struct amdgpu_hdp_ras_funcs hdp_v4_0_ras_funcs = {
+struct amdgpu_ras_block_hw_ops hdp_v4_0_ras_hw_ops = {
-	.ras_late_init = amdgpu_hdp_ras_late_init,
-	.ras_fini = amdgpu_hdp_ras_fini,
 	.query_ras_error_count = hdp_v4_0_query_ras_error_count,
 	.reset_ras_error_count = hdp_v4_0_reset_ras_error_count,
 };
+struct amdgpu_hdp_ras hdp_v4_0_ras = {
+	.ras_block = {
+		.ras_comm = {
+			.name = "hdp",
+			.block = AMDGPU_RAS_BLOCK__HDP,
+			.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE,
+		},
+		.hw_ops = &hdp_v4_0_ras_hw_ops,
+		.ras_fini = amdgpu_hdp_ras_fini,
+	},
+};
 const struct amdgpu_hdp_funcs hdp_v4_0_funcs = {
 	.flush_hdp = hdp_v4_0_flush_hdp,
 	.invalidate_hdp = hdp_v4_0_invalidate_hdp,

drivers/gpu/drm/amd/amdgpu/hdp_v4_0.h+1−1

@@ -27,6 +27,6 @@ · 26 unmodified lines
27	#include "soc15_common.h"	27	#include "soc15_common.h"
28		28
29	extern const struct amdgpu_hdp_funcs hdp_v4_0_funcs;	29	extern const struct amdgpu_hdp_funcs hdp_v4_0_funcs;
30	extern const struct amdgpu_hdp_ras_funcs hdp_v4_0_ras_funcs;	30	extern struct amdgpu_hdp_ras hdp_v4_0_ras;
31		31
32	#endif	32	#endif

drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c+7−2

 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-	if (adev->asic_type != CHIP_YELLOW_CARP) {
+	u32 harvest;
-		u32 harvest = RREG32_SOC15(JPEG, 0, mmCC_UVD_HARVESTING);
+	switch (adev->ip_versions[UVD_HWIP][0]) {
+	case IP_VERSION(3, 1, 1):
+		break;
+	default:
+		harvest = RREG32_SOC15(JPEG, 0, mmCC_UVD_HARVESTING);
 		if (harvest & CC_UVD_HARVESTING__UVD_DISABLE_MASK)
 			return -ENOENT;
+		break;
 	}
 	adev->jpeg.num_jpeg_inst = 1;

drivers/gpu/drm/amd/amdgpu/mca_v3_0.c+67−35

@@ -37,24 +37,42 @@ static void mca_v3_0_mp0_query_ras_error_count(struct amdgpu_device *adev, · 36 unmodified lines
37	ras_error_status);	37	ras_error_status);
38	}	38	}
39		39
40	static int mca_v3_0_mp0_ras_late_init(struct amdgpu_device *adev)	40	static void mca_v3_0_mp0_ras_fini(struct amdgpu_device *adev)
41	{	41	{
42	return amdgpu_mca_ras_late_init(adev, &adev->mca.mp0);	42	amdgpu_mca_ras_fini(adev, &adev->mca.mp0);
43	}	43	}
44		44
45	static void mca_v3_0_mp0_ras_fini(struct amdgpu_device *adev)	45	static int mca_v3_0_ras_block_match(struct amdgpu_ras_block_object *block_obj,
		46	enum amdgpu_ras_block block, uint32_t sub_block_index)
46	{	47	{
47	amdgpu_mca_ras_fini(adev, &adev->mca.mp0);	48	if (!block_obj)
		49	return -EINVAL;
		50
		51	if ((block_obj->ras_comm.block == block) &&
		52	(block_obj->ras_comm.sub_block_index == sub_block_index)) {
		53	return 0;
		54	}
		55
		56	return -EINVAL;
48	}	57	}
49		58
50	const struct amdgpu_mca_ras_funcs mca_v3_0_mp0_ras_funcs = {	59	const struct amdgpu_ras_block_hw_ops mca_v3_0_mp0_hw_ops = {
51	.ras_late_init = mca_v3_0_mp0_ras_late_init,
52	.ras_fini = mca_v3_0_mp0_ras_fini,
53	.query_ras_error_count = mca_v3_0_mp0_query_ras_error_count,	60	.query_ras_error_count = mca_v3_0_mp0_query_ras_error_count,
54	.query_ras_error_address = NULL,	61	.query_ras_error_address = NULL,
55	.ras_block = AMDGPU_RAS_BLOCK__MCA,	62	};
56	.ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MP0,	63
57	.sysfs_name = "mp0_err_count",	64	struct amdgpu_mca_ras_block mca_v3_0_mp0_ras = {
		65	.ras_block = {
		66	.ras_comm = {
		67	.block = AMDGPU_RAS_BLOCK__MCA,
		68	.sub_block_index = AMDGPU_RAS_MCA_BLOCK__MP0,
		69	.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE,
		70	.name = "mp0",
		71	},
		72	.hw_ops = &mca_v3_0_mp0_hw_ops,
		73	.ras_block_match = mca_v3_0_ras_block_match,
		74	.ras_fini = mca_v3_0_mp0_ras_fini,
		75	},
58	};	76	};
59		77
60	static void mca_v3_0_mp1_query_ras_error_count(struct amdgpu_device *adev,	78	static void mca_v3_0_mp1_query_ras_error_count(struct amdgpu_device *adev,
@@ -65,24 +83,28 @@ static void mca_v3_0_mp1_query_ras_error_count(struct amdgpu_device *adev, · 4 unmodified lines
65	ras_error_status);	83	ras_error_status);
66	}	84	}
67		85
68	static int mca_v3_0_mp1_ras_late_init(struct amdgpu_device *adev)
69	{
70	return amdgpu_mca_ras_late_init(adev, &adev->mca.mp1);
71	}
72
73	static void mca_v3_0_mp1_ras_fini(struct amdgpu_device *adev)	86	static void mca_v3_0_mp1_ras_fini(struct amdgpu_device *adev)
74	{	87	{
75	amdgpu_mca_ras_fini(adev, &adev->mca.mp1);	88	amdgpu_mca_ras_fini(adev, &adev->mca.mp1);
76	}	89	}
77		90
78	const struct amdgpu_mca_ras_funcs mca_v3_0_mp1_ras_funcs = {	91	const struct amdgpu_ras_block_hw_ops mca_v3_0_mp1_hw_ops = {
79	.ras_late_init = mca_v3_0_mp1_ras_late_init,
80	.ras_fini = mca_v3_0_mp1_ras_fini,
81	.query_ras_error_count = mca_v3_0_mp1_query_ras_error_count,	92	.query_ras_error_count = mca_v3_0_mp1_query_ras_error_count,
82	.query_ras_error_address = NULL,	93	.query_ras_error_address = NULL,
83	.ras_block = AMDGPU_RAS_BLOCK__MCA,	94	};
84	.ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MP1,	95
85	.sysfs_name = "mp1_err_count",	96	struct amdgpu_mca_ras_block mca_v3_0_mp1_ras = {
		97	.ras_block = {
		98	.ras_comm = {
		99	.block = AMDGPU_RAS_BLOCK__MCA,
		100	.sub_block_index = AMDGPU_RAS_MCA_BLOCK__MP1,
		101	.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE,
		102	.name = "mp1",
		103	},
		104	.hw_ops = &mca_v3_0_mp1_hw_ops,
		105	.ras_block_match = mca_v3_0_ras_block_match,
		106	.ras_fini = mca_v3_0_mp1_ras_fini,
		107	},
86	};	108	};
87		109
88	static void mca_v3_0_mpio_query_ras_error_count(struct amdgpu_device *adev,	110	static void mca_v3_0_mpio_query_ras_error_count(struct amdgpu_device *adev,
@@ -93,24 +115,28 @@ static void mca_v3_0_mpio_query_ras_error_count(struct amdgpu_device *adev, · 4 unmodified lines
93	ras_error_status);	115	ras_error_status);
94	}	116	}
95		117
96	static int mca_v3_0_mpio_ras_late_init(struct amdgpu_device *adev)
97	{
98	return amdgpu_mca_ras_late_init(adev, &adev->mca.mpio);
99	}
100
101	static void mca_v3_0_mpio_ras_fini(struct amdgpu_device *adev)	118	static void mca_v3_0_mpio_ras_fini(struct amdgpu_device *adev)
102	{	119	{
103	amdgpu_mca_ras_fini(adev, &adev->mca.mpio);	120	amdgpu_mca_ras_fini(adev, &adev->mca.mpio);
104	}	121	}
105		122
106	const struct amdgpu_mca_ras_funcs mca_v3_0_mpio_ras_funcs = {	123	const struct amdgpu_ras_block_hw_ops mca_v3_0_mpio_hw_ops = {
107	.ras_late_init = mca_v3_0_mpio_ras_late_init,
108	.ras_fini = mca_v3_0_mpio_ras_fini,
109	.query_ras_error_count = mca_v3_0_mpio_query_ras_error_count,	124	.query_ras_error_count = mca_v3_0_mpio_query_ras_error_count,
110	.query_ras_error_address = NULL,	125	.query_ras_error_address = NULL,
111	.ras_block = AMDGPU_RAS_BLOCK__MCA,	126	};
112	.ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MPIO,	127
113	.sysfs_name = "mpio_err_count",	128	struct amdgpu_mca_ras_block mca_v3_0_mpio_ras = {
		129	.ras_block = {
		130	.ras_comm = {
		131	.block = AMDGPU_RAS_BLOCK__MCA,
		132	.sub_block_index = AMDGPU_RAS_MCA_BLOCK__MPIO,
		133	.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE,
		134	.name = "mpio",
		135	},
		136	.hw_ops = &mca_v3_0_mpio_hw_ops,
		137	.ras_block_match = mca_v3_0_ras_block_match,
		138	.ras_fini = mca_v3_0_mpio_ras_fini,
		139	},
114	};	140	};
115		141
116		142
@@ -118,9 +144,15 @@ static void mca_v3_0_init(struct amdgpu_device *adev) · 1 unmodified line
118	{	144	{
119	struct amdgpu_mca *mca = &adev->mca;	145	struct amdgpu_mca *mca = &adev->mca;
120		146
121	mca->mp0.ras_funcs = &mca_v3_0_mp0_ras_funcs;	147	mca->mp0.ras = &mca_v3_0_mp0_ras;
122	mca->mp1.ras_funcs = &mca_v3_0_mp1_ras_funcs;	148	mca->mp1.ras = &mca_v3_0_mp1_ras;
123	mca->mpio.ras_funcs = &mca_v3_0_mpio_ras_funcs;	149	mca->mpio.ras = &mca_v3_0_mpio_ras;
		150	amdgpu_ras_register_ras_block(adev, &mca->mp0.ras->ras_block);
		151	amdgpu_ras_register_ras_block(adev, &mca->mp1.ras->ras_block);
		152	amdgpu_ras_register_ras_block(adev, &mca->mpio.ras->ras_block);
		153	mca->mp0.ras_if = &mca->mp0.ras->ras_block.ras_comm;
		154	mca->mp1.ras_if = &mca->mp1.ras->ras_block.ras_comm;
		155	mca->mpio.ras_if = &mca->mpio.ras->ras_block.ras_comm;
124	}	156	}
125		157
126	const struct amdgpu_mca_funcs mca_v3_0_funcs = {	158	const struct amdgpu_mca_funcs mca_v3_0_funcs = {

drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c+7−3

 	}
 }
-const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs = {
+struct amdgpu_ras_block_hw_ops mmhub_v1_0_ras_hw_ops = {
-	.ras_late_init = amdgpu_mmhub_ras_late_init,
-	.ras_fini = amdgpu_mmhub_ras_fini,
 	.query_ras_error_count = mmhub_v1_0_query_ras_error_count,
 	.reset_ras_error_count = mmhub_v1_0_reset_ras_error_count,
 };
+struct amdgpu_mmhub_ras mmhub_v1_0_ras = {
+	.ras_block = {
+		.hw_ops = &mmhub_v1_0_ras_hw_ops,
+	},
+};
 const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs = {
 	.get_fb_location = mmhub_v1_0_get_fb_location,
 	.init = mmhub_v1_0_init,

drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.h+1−1

@@ -24,6 +24,6 @@ · 23 unmodified lines
24	#define __MMHUB_V1_0_H__	24	#define __MMHUB_V1_0_H__
25		25
26	extern const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs;	26	extern const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs;
27	extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs;	27	extern struct amdgpu_mmhub_ras mmhub_v1_0_ras;
28		28
29	#endif	29	#endif

drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.c+7−3

 	}
 }
-const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs = {
+struct amdgpu_ras_block_hw_ops mmhub_v1_7_ras_hw_ops = {
-	.ras_late_init = amdgpu_mmhub_ras_late_init,
-	.ras_fini = amdgpu_mmhub_ras_fini,
 	.query_ras_error_count = mmhub_v1_7_query_ras_error_count,
 	.reset_ras_error_count = mmhub_v1_7_reset_ras_error_count,
 	.query_ras_error_status = mmhub_v1_7_query_ras_error_status,
 	.reset_ras_error_status = mmhub_v1_7_reset_ras_error_status,
 };
+struct amdgpu_mmhub_ras mmhub_v1_7_ras = {
+	.ras_block = {
+		.hw_ops = &mmhub_v1_7_ras_hw_ops,
+	},
+};
 const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs = {
 	.get_fb_location = mmhub_v1_7_get_fb_location,
 	.init = mmhub_v1_7_init,

drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.h+1−1

@@ -24,6 +24,6 @@ · 23 unmodified lines
24	#define __MMHUB_V1_7_H__	24	#define __MMHUB_V1_7_H__
25		25
26	extern const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs;	26	extern const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs;
27	extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs;	27	extern struct amdgpu_mmhub_ras mmhub_v1_7_ras;
28		28
29	#endif	29	#endif

drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c+1−0

@@ -93,6 +93,7 @@ mmhub_v2_3_print_l2_protection_fault_status(struct amdgpu_device *adev, · 92 unmodified lines
93	switch (adev->ip_versions[MMHUB_HWIP][0]) {	93	switch (adev->ip_versions[MMHUB_HWIP][0]) {
94	case IP_VERSION(2, 3, 0):	94	case IP_VERSION(2, 3, 0):
95	case IP_VERSION(2, 4, 0):	95	case IP_VERSION(2, 4, 0):
		96	case IP_VERSION(2, 4, 1):
96	mmhub_cid = mmhub_client_ids_vangogh[cid][rw];	97	mmhub_cid = mmhub_client_ids_vangogh[cid][rw];
97	break;	98	break;
98	default:	99	default:

drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c+7−3

 	}
 }
-const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs = {
+const struct amdgpu_ras_block_hw_ops mmhub_v9_4_ras_hw_ops = {
-	.ras_late_init = amdgpu_mmhub_ras_late_init,
-	.ras_fini = amdgpu_mmhub_ras_fini,
 	.query_ras_error_count = mmhub_v9_4_query_ras_error_count,
 	.reset_ras_error_count = mmhub_v9_4_reset_ras_error_count,
 	.query_ras_error_status = mmhub_v9_4_query_ras_error_status,
 };
+struct amdgpu_mmhub_ras mmhub_v9_4_ras = {
+	.ras_block = {
+		.hw_ops = &mmhub_v9_4_ras_hw_ops,
+	},
+};
 const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = {
 	.get_fb_location = mmhub_v9_4_get_fb_location,
 	.init = mmhub_v9_4_init,

drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.h+1−1

@@ -24,6 +24,6 @@ · 23 unmodified lines
24	#define __MMHUB_V9_4_H__	24	#define __MMHUB_V9_4_H__
25		25
26	extern const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs;	26	extern const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs;
27	extern const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs;	27	extern struct amdgpu_mmhub_ras mmhub_v9_4_ras;
28		28
29	#endif	29	#endif

drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c+12−7

 #include "soc15_common.h"
 #include "mxgpu_ai.h"
+#include "amdgpu_reset.h"
 static void xgpu_ai_mailbox_send_ack(struct amdgpu_device *adev)
 {
 	WREG8(AI_MAIBOX_CONTROL_RCV_OFFSET_BYTE, 2);
 	 * otherwise the mailbox msg will be ruined/reseted by
 	 * the VF FLR.
 	 */
-	if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0)
+	if (atomic_cmpxchg(&adev->reset_domain->in_gpu_reset, 0, 1) != 0)
 		return;
-	down_write(&adev->reset_sem);
+	down_write(&adev->reset_domain->sem);
 	amdgpu_virt_fini_data_exchange(adev);
 	} while (timeout > 1);
 flr_done:
-	atomic_set(&adev->in_gpu_reset, 0);
+	atomic_set(&adev->reset_domain->in_gpu_reset, 0);
-	up_write(&adev->reset_sem);
+	up_write(&adev->reset_domain->sem);
 	/* Trigger recovery for world switch failure if no TDR */
 	if (amdgpu_device_should_recover_gpu(adev)
 		&& (!amdgpu_device_has_job_running(adev) ||
 		adev->sdma_timeout == MAX_SCHEDULE_TIMEOUT))
-		amdgpu_device_gpu_recover(adev, NULL);
+		amdgpu_device_gpu_recover_imp(adev, NULL);
 }
 static int xgpu_ai_set_mailbox_rcv_irq(struct amdgpu_device *adev,
 	switch (event) {
 		case IDH_FLR_NOTIFICATION:
-		if (amdgpu_sriov_runtime(adev))
+		if (amdgpu_sriov_runtime(adev) && !amdgpu_in_reset(adev))
-			schedule_work(&adev->virt.flr_work);
+			WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain,
+								&adev->virt.flr_work),
+				  "Failed to queue work! at %s",
+				  __func__);
 		break;
 		case IDH_QUERY_ALIVE:
 			xgpu_ai_mailbox_send_ack(adev);

drivers/gpu/drm/amd/amdgpu/mxgpu_nv.c+12−7

 #include "soc15_common.h"
 #include "mxgpu_nv.h"
+#include "amdgpu_reset.h"
 static void xgpu_nv_mailbox_send_ack(struct amdgpu_device *adev)
 {
 	WREG8(NV_MAIBOX_CONTROL_RCV_OFFSET_BYTE, 2);
 	 * otherwise the mailbox msg will be ruined/reseted by
 	 * the VF FLR.
 	 */
-	if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0)
+	if (atomic_cmpxchg(&adev->reset_domain->in_gpu_reset, 0, 1) != 0)
 		return;
-	down_write(&adev->reset_sem);
+	down_write(&adev->reset_domain->sem);
 	amdgpu_virt_fini_data_exchange(adev);
 	} while (timeout > 1);
 flr_done:
-	atomic_set(&adev->in_gpu_reset, 0);
+	atomic_set(&adev->reset_domain->in_gpu_reset, 0);
-	up_write(&adev->reset_sem);
+	up_write(&adev->reset_domain->sem);
 	/* Trigger recovery for world switch failure if no TDR */
 	if (amdgpu_device_should_recover_gpu(adev)
 		adev->gfx_timeout == MAX_SCHEDULE_TIMEOUT ||
 		adev->compute_timeout == MAX_SCHEDULE_TIMEOUT ||
 		adev->video_timeout == MAX_SCHEDULE_TIMEOUT))
-		amdgpu_device_gpu_recover(adev, NULL);
+		amdgpu_device_gpu_recover_imp(adev, NULL);
 }
 static int xgpu_nv_set_mailbox_rcv_irq(struct amdgpu_device *adev,
 	switch (event) {
 	case IDH_FLR_NOTIFICATION:
-		if (amdgpu_sriov_runtime(adev))
+		if (amdgpu_sriov_runtime(adev) && !amdgpu_in_reset(adev))
-			schedule_work(&adev->virt.flr_work);
+			WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain,
+				   &adev->virt.flr_work),
+				  "Failed to queue work! at %s",
+				  __func__);
 		break;
 		/* READY_TO_ACCESS_GPU is fetched by kernel polling, IRQ can ignore
 		 * it byfar since that polling thread will handle it,

drivers/gpu/drm/amd/amdgpu/mxgpu_vi.c+9−4

 #include "smu/smu_7_1_3_d.h"
 #include "mxgpu_vi.h"
+#include "amdgpu_reset.h"
 /* VI golden setting */
 static const u32 xgpu_fiji_mgcg_cgcg_init[] = {
 	mmRLC_CGTT_MGCG_OVERRIDE, 0xffffffff, 0xffffffff,
 	/* Trigger recovery due to world switch failure */
 	if (amdgpu_device_should_recover_gpu(adev))
-		amdgpu_device_gpu_recover(adev, NULL);
+		amdgpu_device_gpu_recover_imp(adev, NULL);
 }
 static int xgpu_vi_set_mailbox_rcv_irq(struct amdgpu_device *adev,
 {
 	int r;
-	/* trigger gpu-reset by hypervisor only if TDR disbaled */
+	/* trigger gpu-reset by hypervisor only if TDR disabled */
 	if (!amdgpu_gpu_recovery) {
 		/* see what event we get */
 		r = xgpu_vi_mailbox_rcv_msg(adev, IDH_FLR_NOTIFICATION);
 		/* only handle FLR_NOTIFY now */
-		if (!r)
+		if (!r && !amdgpu_in_reset(adev))
-			schedule_work(&adev->virt.flr_work);
+			WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain,
+								&adev->virt.flr_work),
+				  "Failed to queue work! at %s",
+				  __func__);
 	}
 	return 0;

drivers/gpu/drm/amd/amdgpu/nbio_v7_2.c+31−13

 {
 	u32 tmp;
-	if (adev->asic_type == CHIP_YELLOW_CARP)
+	switch (adev->ip_versions[NBIO_HWIP][0]) {
+	case IP_VERSION(7, 2, 1):
+	case IP_VERSION(7, 5, 0):
 		tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_YC);
-	else
+		break;
+	default:
 		tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0);
+		break;
+	}
 	tmp &= RCC_STRAP0_RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0_MASK;
 	tmp >>= RCC_STRAP0_RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0__SHIFT;
 static void nbio_v7_2_mc_access_enable(struct amdgpu_device *adev, bool enable)
 {
-	if (enable)
+	switch (adev->ip_versions[NBIO_HWIP][0]) {
-		if (adev->asic_type == CHIP_YELLOW_CARP)
+	case IP_VERSION(7, 2, 1):
+	case IP_VERSION(7, 5, 0):
+		if (enable)
 			WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC,
 				BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK |
 				BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK);
 		else
+			WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, 0);
+	break;
+	default:
+		if (enable)
 			WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN,
 				BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK |
 				BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK);
-	else
-		if (adev->asic_type == CHIP_YELLOW_CARP)
-			WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, 0);
 		else
 			WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN, 0);
+		break;
+	}
 }
 static u32 nbio_v7_2_get_memsize(struct amdgpu_device *adev)
 {
 	uint32_t def, data;
-	if (adev->asic_type == CHIP_YELLOW_CARP) {
+	switch (adev->ip_versions[NBIO_HWIP][0]) {
+	case IP_VERSION(7, 2, 1):
+	case IP_VERSION(7, 5, 0):
 		def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2));
 		if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS))
 			data |= PCIE_CNTL2__SLV_MEM_LS_EN_MASK;
 		if (def != data)
 			WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2), data);
-		data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_TX_POWER_CTRL_1));
+		def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0,
-		def = data;
+			regBIF1_PCIE_TX_POWER_CTRL_1));
 		if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS))
 			data |= (BIF1_PCIE_TX_POWER_CTRL_1__MST_MEM_LS_EN_MASK |
 				BIF1_PCIE_TX_POWER_CTRL_1__REPLAY_MEM_LS_EN_MASK);
 		if (def != data)
 			WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_TX_POWER_CTRL_1),
 				data);
-	} else {
+		break;
+	default:
 		def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2));
 		if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS))
 			data |= (PCIE_CNTL2__SLV_MEM_LS_EN_MASK |
 		if (def != data)
 			WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2), data);
+		break;
 	}
 }
 static void nbio_v7_2_init_registers(struct amdgpu_device *adev)
 {
 	uint32_t def, data;
-	if (adev->asic_type == CHIP_YELLOW_CARP) {
+	switch (adev->ip_versions[NBIO_HWIP][0]) {
+	case IP_VERSION(7, 2, 1):
+	case IP_VERSION(7, 5, 0):
 		def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_MST_CTRL_3));
 		data = REG_SET_FIELD(data, BIF1_PCIE_MST_CTRL_3,
 			CI_SWUS_MAX_READ_REQUEST_SIZE_MODE, 1);
 		if (def != data)
 			WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_MST_CTRL_3), data);
-	} else {
+		break;
+	default:
 		def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CONFIG_CNTL));
 		data = REG_SET_FIELD(data, PCIE_CONFIG_CNTL,
 			CI_SWUS_MAX_READ_REQUEST_SIZE_MODE, 1);
 		if (def != data)
 			WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CONFIG_CNTL), data);
+		break;
 	}
 	if (amdgpu_sriov_vf(adev))

drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c+16−4

 		       DOORBELL_INTERRUPT_DISABLE, enable ? 0 : 1);
 }
-const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs = {
+const struct amdgpu_ras_block_hw_ops nbio_v7_4_ras_hw_ops = {
+	.query_ras_error_count = nbio_v7_4_query_ras_error_count,
+};
+struct amdgpu_nbio_ras nbio_v7_4_ras = {
+	.ras_block = {
+		.ras_comm = {
+			.name = "pcie_bif",
+			.block = AMDGPU_RAS_BLOCK__PCIE_BIF,
+			.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE,
+		},
+		.hw_ops = &nbio_v7_4_ras_hw_ops,
+		.ras_late_init = amdgpu_nbio_ras_late_init,
+		.ras_fini = amdgpu_nbio_ras_fini,
+	},
 	.handle_ras_controller_intr_no_bifring = nbio_v7_4_handle_ras_controller_intr_no_bifring,
 	.handle_ras_err_event_athub_intr_no_bifring = nbio_v7_4_handle_ras_err_event_athub_intr_no_bifring,
 	.init_ras_controller_interrupt = nbio_v7_4_init_ras_controller_interrupt,
 	.init_ras_err_event_athub_interrupt = nbio_v7_4_init_ras_err_event_athub_interrupt,
-	.query_ras_error_count = nbio_v7_4_query_ras_error_count,
-	.ras_late_init = amdgpu_nbio_ras_late_init,
-	.ras_fini = amdgpu_nbio_ras_fini,
 };
 static void nbio_v7_4_program_ltr(struct amdgpu_device *adev)
 {
 	uint32_t def, data;

drivers/gpu/drm/amd/amdgpu/nbio_v7_4.h+1−1

@@ -29,6 +29,6 @@ · 28 unmodified lines
29	extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg;	29	extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg;
30	extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg_ald;	30	extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg_ald;
31	extern const struct amdgpu_nbio_funcs nbio_v7_4_funcs;	31	extern const struct amdgpu_nbio_funcs nbio_v7_4_funcs;
32	extern const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs;	32	extern struct amdgpu_nbio_ras nbio_v7_4_ras;
33		33
34	#endif	34	#endif

drivers/gpu/drm/amd/amdgpu/nv.c+35−67

 	return amdgpu_device_indirect_rreg64(adev, address, data, reg);
 }
-static u32 nv_pcie_port_rreg(struct amdgpu_device *adev, u32 reg)
-{
-	unsigned long flags, address, data;
-	u32 r;
-	address = adev->nbio.funcs->get_pcie_port_index_offset(adev);
-	data = adev->nbio.funcs->get_pcie_port_data_offset(adev);
-	spin_lock_irqsave(&adev->pcie_idx_lock, flags);
-	WREG32(address, reg * 4);
-	(void)RREG32(address);
-	r = RREG32(data);
-	spin_unlock_irqrestore(&adev->pcie_idx_lock, flags);
-	return r;
-}
 static void nv_pcie_wreg64(struct amdgpu_device *adev, u32 reg, u64 v)
 {
 	unsigned long address, data;
 	amdgpu_device_indirect_wreg64(adev, address, data, reg, v);
 }
-static void nv_pcie_port_wreg(struct amdgpu_device *adev, u32 reg, u32 v)
-{
-	unsigned long flags, address, data;
-	address = adev->nbio.funcs->get_pcie_port_index_offset(adev);
-	data = adev->nbio.funcs->get_pcie_port_data_offset(adev);
-	spin_lock_irqsave(&adev->pcie_idx_lock, flags);
-	WREG32(address, reg * 4);
-	(void)RREG32(address);
-	WREG32(data, v);
-	(void)RREG32(data);
-	spin_unlock_irqrestore(&adev->pcie_idx_lock, flags);
-}
 static u32 nv_didt_rreg(struct amdgpu_device *adev, u32 reg)
 {
 	unsigned long flags, address, data;
 	return false;
 }
-static bool nv_read_bios_from_rom(struct amdgpu_device *adev,
-				  u8 *bios, u32 length_bytes)
-{
-	u32 *dw_ptr;
-	u32 i, length_dw;
-	u32 rom_index_offset, rom_data_offset;
-	if (bios == NULL)
-		return false;
-	if (length_bytes == 0)
-		return false;
-	/* APU vbios image is part of sbios image */
-	if (adev->flags & AMD_IS_APU)
-		return false;
-	dw_ptr = (u32 *)bios;
-	length_dw = ALIGN(length_bytes, 4) / 4;
-	rom_index_offset =
-		adev->smuio.funcs->get_rom_index_offset(adev);
-	rom_data_offset =
-		adev->smuio.funcs->get_rom_data_offset(adev);
-	/* set rom index to 0 */
-	WREG32(rom_index_offset, 0);
-	/* read out the rom data */
-	for (i = 0; i < length_dw; i++)
-		dw_ptr[i] = RREG32(rom_data_offset);
-	return true;
-}
 static struct soc15_allowed_register_entry nv_allowed_read_registers[] = {
 	{ SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS)},
 	{ SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS2)},
 static void nv_program_aspm(struct amdgpu_device *adev)
 {
-	if (!amdgpu_aspm)
+	if (!amdgpu_device_should_use_aspm(adev))
 		return;
 	if (!(adev->flags & AMD_IS_APU) &&
 		adev->gfx.funcs->update_perfmon_mgcg(adev, !enter);
 	if (!(adev->flags & AMD_IS_APU) &&
-	    (adev->nbio.funcs->enable_aspm))
+	    (adev->nbio.funcs->enable_aspm) &&
+	     amdgpu_device_should_use_aspm(adev))
 		adev->nbio.funcs->enable_aspm(adev, !enter);
 	return 0;
 static const struct amdgpu_asic_funcs nv_asic_funcs =
 {
 	.read_disabled_bios = &nv_read_disabled_bios,
-	.read_bios_from_rom = &nv_read_bios_from_rom,
+	.read_bios_from_rom = &amdgpu_soc15_read_bios_from_rom,
 	.read_register = &nv_read_register,
 	.reset = &nv_asic_reset,
 	.reset_method = &nv_asic_reset_method,
 	adev->pcie_wreg = &nv_pcie_wreg;
 	adev->pcie_rreg64 = &nv_pcie_rreg64;
 	adev->pcie_wreg64 = &nv_pcie_wreg64;
-	adev->pciep_rreg = &nv_pcie_port_rreg;
+	adev->pciep_rreg = amdgpu_device_pcie_port_rreg;
-	adev->pciep_wreg = &nv_pcie_port_wreg;
+	adev->pciep_wreg = amdgpu_device_pcie_port_wreg;
 	/* TODO: will add them during VCN v2 implementation */
 	adev->uvd_ctx_rreg = NULL;
 			adev->external_rev_id = adev->rev_id + 0x01;
 		break;
 	case IP_VERSION(10, 1, 3):
+	case IP_VERSION(10, 1, 4):
 		adev->cg_flags = 0;
 		adev->pg_flags = 0;
 		adev->external_rev_id = adev->rev_id + 0x82;
 		break;
+	case IP_VERSION(10, 3, 6):
+		adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG |
+			AMD_CG_SUPPORT_GFX_MGLS |
+			AMD_CG_SUPPORT_GFX_CGCG |
+			AMD_CG_SUPPORT_GFX_CGLS |
+			AMD_CG_SUPPORT_GFX_3D_CGCG |
+			AMD_CG_SUPPORT_GFX_3D_CGLS |
+			AMD_CG_SUPPORT_GFX_RLC_LS |
+			AMD_CG_SUPPORT_GFX_CP_LS |
+			AMD_CG_SUPPORT_GFX_FGCG |
+			AMD_CG_SUPPORT_MC_MGCG |
+			AMD_CG_SUPPORT_MC_LS |
+			AMD_CG_SUPPORT_SDMA_LS |
+			AMD_CG_SUPPORT_HDP_MGCG |
+			AMD_CG_SUPPORT_HDP_LS |
+			AMD_CG_SUPPORT_ATHUB_MGCG |
+			AMD_CG_SUPPORT_ATHUB_LS |
+			AMD_CG_SUPPORT_IH_CG;
+		adev->pg_flags = AMD_PG_SUPPORT_GFX_PG;
+		adev->external_rev_id = adev->rev_id + 0x01;
+		break;
+	case IP_VERSION(10, 3, 7):
+		adev->cg_flags = 0;
+		adev->pg_flags = AMD_PG_SUPPORT_VCN |
+			AMD_PG_SUPPORT_VCN_DPG |
+			AMD_PG_SUPPORT_JPEG;
+		adev->external_rev_id = adev->rev_id + 0x01;
+		break;
 	default:
 		/* FIXME: not supported yet */
 		return -EINVAL;

drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h+1−0

@@ -258,6 +258,7 @@ enum psp_gfx_fw_type { · 257 unmodified lines
258	GFX_FW_TYPE_SDMA6 = 56, /* SDMA6 MI */	258	GFX_FW_TYPE_SDMA6 = 56, /* SDMA6 MI */
259	GFX_FW_TYPE_SDMA7 = 57, /* SDMA7 MI */	259	GFX_FW_TYPE_SDMA7 = 57, /* SDMA7 MI */
260	GFX_FW_TYPE_VCN1 = 58, /* VCN1 MI */	260	GFX_FW_TYPE_VCN1 = 58, /* VCN1 MI */
		261	GFX_FW_TYPE_CAP = 62, /* CAP_FW */
261	GFX_FW_TYPE_REG_LIST = 67, /* REG_LIST MI */	262	GFX_FW_TYPE_REG_LIST = 67, /* REG_LIST MI */
262	GFX_FW_TYPE_MAX	263	GFX_FW_TYPE_MAX
263	};	264	};

drivers/gpu/drm/amd/amdgpu/psp_v11_0.c+2−2

 MODULE_FIRMWARE("amdgpu/navi12_sos.bin");
 MODULE_FIRMWARE("amdgpu/navi12_asd.bin");
 MODULE_FIRMWARE("amdgpu/navi12_ta.bin");
+MODULE_FIRMWARE("amdgpu/navi12_cap.bin");
 MODULE_FIRMWARE("amdgpu/arcturus_sos.bin");
 MODULE_FIRMWARE("amdgpu/arcturus_asd.bin");
 MODULE_FIRMWARE("amdgpu/arcturus_ta.bin");
 MODULE_FIRMWARE("amdgpu/sienna_cichlid_sos.bin");
 MODULE_FIRMWARE("amdgpu/sienna_cichlid_ta.bin");
+MODULE_FIRMWARE("amdgpu/sienna_cichlid_cap.bin");
 MODULE_FIRMWARE("amdgpu/navy_flounder_sos.bin");
 MODULE_FIRMWARE("amdgpu/navy_flounder_ta.bin");
 MODULE_FIRMWARE("amdgpu/vangogh_asd.bin");
 		err = psp_init_asd_microcode(psp, chip_name);
 		if (err)
 			return err;
-		if (amdgpu_sriov_vf(adev))
-			break;
 		snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_ta.bin", chip_name);
 		err = request_firmware(&adev->psp.ta_fw, fw_name, adev->dev);
 		if (err) {

drivers/gpu/drm/amd/amdgpu/psp_v13_0.c+7−0

 MODULE_FIRMWARE("amdgpu/yellow_carp_asd.bin");
 MODULE_FIRMWARE("amdgpu/yellow_carp_toc.bin");
 MODULE_FIRMWARE("amdgpu/yellow_carp_ta.bin");
+MODULE_FIRMWARE("amdgpu/psp_13_0_8_asd.bin");
+MODULE_FIRMWARE("amdgpu/psp_13_0_8_toc.bin");
+MODULE_FIRMWARE("amdgpu/psp_13_0_8_ta.bin");
 /* For large FW files the time to complete can be very long */
 #define USBC_PD_POLLING_LIMIT_S 240
 	case IP_VERSION(13, 0, 3):
 		chip_name = "yellow_carp";
 		break;
+	case IP_VERSION(13, 0, 8):
+		chip_name = "psp_13_0_8";
+		break;
 	default:
 		BUG();
 	}
 		break;
 	case IP_VERSION(13, 0, 1):
 	case IP_VERSION(13, 0, 3):
+	case IP_VERSION(13, 0, 8):
 		err = psp_init_asd_microcode(psp, chip_name);
 		if (err)
 			return err;

drivers/gpu/drm/amd/amdgpu/psp_v3_1.c+1−0

@@ -44,6 +44,7 @@ · 43 unmodified lines
44		44
45	MODULE_FIRMWARE("amdgpu/vega10_sos.bin");	45	MODULE_FIRMWARE("amdgpu/vega10_sos.bin");
46	MODULE_FIRMWARE("amdgpu/vega10_asd.bin");	46	MODULE_FIRMWARE("amdgpu/vega10_asd.bin");
		47	MODULE_FIRMWARE("amdgpu/vega10_cap.bin");
47	MODULE_FIRMWARE("amdgpu/vega12_sos.bin");	48	MODULE_FIRMWARE("amdgpu/vega12_sos.bin");
48	MODULE_FIRMWARE("amdgpu/vega12_asd.bin");	49	MODULE_FIRMWARE("amdgpu/vega12_asd.bin");
49		50

drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c+59−18

 static int sdma_v4_0_late_init(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-	struct ras_ih_if ih_info = {
-		.cb = sdma_v4_0_process_ras_data_cb,
-	};
 	sdma_v4_0_setup_ulv(adev);
 	if (!amdgpu_persistent_edc_harvesting_supported(adev)) {
-		if (adev->sdma.funcs &&
+		if (adev->sdma.ras && adev->sdma.ras->ras_block.hw_ops &&
-		    adev->sdma.funcs->reset_ras_error_count)
+		    adev->sdma.ras->ras_block.hw_ops->reset_ras_error_count)
-			adev->sdma.funcs->reset_ras_error_count(adev);
+			adev->sdma.ras->ras_block.hw_ops->reset_ras_error_count(adev);
 	}
-	if (adev->sdma.funcs && adev->sdma.funcs->ras_late_init)
+	return 0;
-		return adev->sdma.funcs->ras_late_init(adev, &ih_info);
-	else
-		return 0;
 }
 static int sdma_v4_0_sw_init(void *handle)
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 	int i;
-	if (adev->sdma.funcs && adev->sdma.funcs->ras_fini)
+	if (adev->sdma.ras && adev->sdma.ras->ras_block.hw_ops &&
-		adev->sdma.funcs->ras_fini(adev);
+		adev->sdma.ras->ras_block.ras_fini)
+		adev->sdma.ras->ras_block.ras_fini(adev);
 	for (i = 0; i < adev->sdma.num_instances; i++) {
 		amdgpu_ring_fini(&adev->sdma.instance[i].ring);
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+	/* SMU saves SDMA state for us */
+	if (adev->in_s0ix)
+		return 0;
 	return sdma_v4_0_hw_fini(adev);
 }
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+	/* SMU restores SDMA state for us */
+	if (adev->in_s0ix)
+		return 0;
 	return sdma_v4_0_hw_init(adev);
 }
 	}
 }
-static int sdma_v4_0_query_ras_error_count(struct amdgpu_device *adev,
+static int sdma_v4_0_query_ras_error_count_by_instance(struct amdgpu_device *adev,
 			uint32_t instance, void *ras_error_status)
 {
 	struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
 	return 0;
 };
+static void sdma_v4_0_query_ras_error_count(struct amdgpu_device *adev,  void *ras_error_status)
+{
+	int i = 0;
+	for (i = 0; i < adev->sdma.num_instances; i++) {
+		if (sdma_v4_0_query_ras_error_count_by_instance(adev, i, ras_error_status)) {
+			dev_err(adev->dev, "Query ras error count failed in SDMA%d\n", i);
+			return;
+		}
+	}
+}
 static void sdma_v4_0_reset_ras_error_count(struct amdgpu_device *adev)
 {
 	int i;
 	}
 }
-static const struct amdgpu_sdma_ras_funcs sdma_v4_0_ras_funcs = {
+const struct amdgpu_ras_block_hw_ops sdma_v4_0_ras_hw_ops = {
-	.ras_late_init = amdgpu_sdma_ras_late_init,
-	.ras_fini = amdgpu_sdma_ras_fini,
 	.query_ras_error_count = sdma_v4_0_query_ras_error_count,
 	.reset_ras_error_count = sdma_v4_0_reset_ras_error_count,
 };
+static struct amdgpu_sdma_ras sdma_v4_0_ras = {
+	.ras_block = {
+		.hw_ops = &sdma_v4_0_ras_hw_ops,
+		.ras_cb = sdma_v4_0_process_ras_data_cb,
+	},
+};
 static void sdma_v4_0_set_ras_funcs(struct amdgpu_device *adev)
 {
 	switch (adev->ip_versions[SDMA0_HWIP][0]) {
 	case IP_VERSION(4, 2, 0):
 	case IP_VERSION(4, 2, 2):
-		adev->sdma.funcs = &sdma_v4_0_ras_funcs;
+		adev->sdma.ras = &sdma_v4_0_ras;
 		break;
 	case IP_VERSION(4, 4, 0):
-		adev->sdma.funcs = &sdma_v4_4_ras_funcs;
+		adev->sdma.ras = &sdma_v4_4_ras;
 		break;
 	default:
 		break;
 	}
+	if (adev->sdma.ras) {
+		amdgpu_ras_register_ras_block(adev, &adev->sdma.ras->ras_block);
+		strcpy(adev->sdma.ras->ras_block.ras_comm.name, "sdma");
+		adev->sdma.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__SDMA;
+		adev->sdma.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
+		adev->sdma.ras_if = &adev->sdma.ras->ras_block.ras_comm;
+		/* If don't define special ras_late_init function, use default ras_late_init */
+		if (!adev->sdma.ras->ras_block.ras_late_init)
+			adev->sdma.ras->ras_block.ras_late_init = amdgpu_sdma_ras_late_init;
+		/* If don't define special ras_fini function, use default ras_fini */
+		if (!adev->sdma.ras->ras_block.ras_fini)
+			adev->sdma.ras->ras_block.ras_fini = amdgpu_sdma_ras_fini;
+		/* If not defined special ras_cb function, use default ras_cb */
+		if (!adev->sdma.ras->ras_block.ras_cb)
+			adev->sdma.ras->ras_block.ras_cb = amdgpu_sdma_process_ras_data_cb;
+	}
 }
 const struct amdgpu_ip_block_version sdma_v4_0_ip_block = {

drivers/gpu/drm/amd/amdgpu/sdma_v4_4.c+21−4

@@ -188,7 +188,7 @@ static void sdma_v4_4_get_ras_error_count(struct amdgpu_device *adev, · 187 unmodified lines
188	}	188	}
189	}	189	}
190		190
191	static int sdma_v4_4_query_ras_error_count(struct amdgpu_device *adev,	191	static int sdma_v4_4_query_ras_error_count_by_instance(struct amdgpu_device *adev,
192	uint32_t instance,	192	uint32_t instance,
193	void *ras_error_status)	193	void *ras_error_status)
194	{	194	{
@@ -245,9 +245,26 @@ static void sdma_v4_4_reset_ras_error_count(struct amdgpu_device *adev) · 50 unmodified lines
245	}	245	}
246	}	246	}
247		247
248	const struct amdgpu_sdma_ras_funcs sdma_v4_4_ras_funcs = {	248	static void sdma_v4_4_query_ras_error_count(struct amdgpu_device adev, void ras_error_status)
249	.ras_late_init = amdgpu_sdma_ras_late_init,	249	{
250	.ras_fini = amdgpu_sdma_ras_fini,	250	int i = 0;
		251
		252	for (i = 0; i < adev->sdma.num_instances; i++) {
		253	if (sdma_v4_4_query_ras_error_count_by_instance(adev, i, ras_error_status)) {
		254	dev_err(adev->dev, "Query ras error count failed in SDMA%d\n", i);
		255	return;
		256	}
		257	}
		258
		259	}
		260
		261	const struct amdgpu_ras_block_hw_ops sdma_v4_4_ras_hw_ops = {
251	.query_ras_error_count = sdma_v4_4_query_ras_error_count,	262	.query_ras_error_count = sdma_v4_4_query_ras_error_count,
252	.reset_ras_error_count = sdma_v4_4_reset_ras_error_count,	263	.reset_ras_error_count = sdma_v4_4_reset_ras_error_count,
253	};	264	};
		265
		266	struct amdgpu_sdma_ras sdma_v4_4_ras = {
		267	.ras_block = {
		268	.hw_ops = &sdma_v4_4_ras_hw_ops,
		269	},
		270	};

drivers/gpu/drm/amd/amdgpu/sdma_v4_4.h+1−1

@@ -23,6 +23,6 @@ · 22 unmodified lines
23	#ifndef __SDMA_V4_4_H__	23	#ifndef __SDMA_V4_4_H__
24	#define __SDMA_V4_4_H__	24	#define __SDMA_V4_4_H__
25		25
26	extern const struct amdgpu_sdma_ras_funcs sdma_v4_4_ras_funcs;	26	extern struct amdgpu_sdma_ras sdma_v4_4_ras;
27		27
28	#endif	28	#endif

drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c+12−7

 MODULE_FIRMWARE("amdgpu/vangogh_sdma.bin");
 MODULE_FIRMWARE("amdgpu/yellow_carp_sdma.bin");
+MODULE_FIRMWARE("amdgpu/sdma_5_2_7.bin");
 #define SDMA1_REG_OFFSET 0x600
 #define SDMA3_REG_OFFSET 0x400
 	switch (adev->ip_versions[SDMA0_HWIP][0]) {
 	case IP_VERSION(5, 2, 0):
-		chip_name = "sienna_cichlid";
+		chip_name = "sienna_cichlid_sdma";
 		break;
 	case IP_VERSION(5, 2, 2):
-		chip_name = "navy_flounder";
+		chip_name = "navy_flounder_sdma";
 		break;
 	case IP_VERSION(5, 2, 1):
-		chip_name = "vangogh";
+		chip_name = "vangogh_sdma";
 		break;
 	case IP_VERSION(5, 2, 4):
-		chip_name = "dimgrey_cavefish";
+		chip_name = "dimgrey_cavefish_sdma";
 		break;
 	case IP_VERSION(5, 2, 5):
-		chip_name = "beige_goby";
+		chip_name = "beige_goby_sdma";
 		break;
 	case IP_VERSION(5, 2, 3):
-		chip_name = "yellow_carp";
+		chip_name = "yellow_carp_sdma";
 		break;
+	case IP_VERSION(5, 2, 7):
+		chip_name = "sdma_5_2_7";
+		break;
 	default:
 		BUG();
 	}
-	snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_sdma.bin", chip_name);
+	snprintf(fw_name, sizeof(fw_name), "amdgpu/%s.bin", chip_name);
 	err = request_firmware(&adev->sdma.instance[0].fw, fw_name, adev->dev);
 	if (err)

drivers/gpu/drm/amd/amdgpu/si.c+1−1

@@ -2453,7 +2453,7 @@ static void si_program_aspm(struct amdgpu_device *adev) · 2452 unmodified lines
2453	bool disable_l0s = false, disable_l1 = false, disable_plloff_in_l1 = false;	2453	bool disable_l0s = false, disable_l1 = false, disable_plloff_in_l1 = false;
2454	bool disable_clkreq = false;	2454	bool disable_clkreq = false;
2455		2455
2456	if (amdgpu_aspm == 0)	2456	if (!amdgpu_device_should_use_aspm(adev))
2457	return;	2457	return;
2458		2458
2459	if (adev->flags & AMD_IS_APU)	2459	if (adev->flags & AMD_IS_APU)

drivers/gpu/drm/amd/amdgpu/smu_v11_0_i2c.c+54−27

 #include "smu_v11_0_i2c.h"
 #include "amdgpu.h"
+#include "amdgpu_dpm.h"
 #include "soc15_common.h"
 #include <drm/drm_fixed.h>
 #include <drm/drm_drv.h>
 #define I2C_X_RESTART         BIT(31)
-#define to_amdgpu_device(x) (container_of(x, struct amdgpu_device, pm.smu_i2c))
 static void smu_v11_0_i2c_set_clock_gating(struct i2c_adapter *control, bool en)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	uint32_t reg = RREG32_SOC15(SMUIO, 0, mmSMUIO_PWRMGT);
 	reg = REG_SET_FIELD(reg, SMUIO_PWRMGT, i2c_clk_gate_en, en ? 1 : 0);
 static int smu_v11_0_i2c_enable(struct i2c_adapter *control, bool enable)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	WREG32_SOC15(SMUIO, 0, mmCKSVII2C_IC_ENABLE, enable ? 1 : 0);
 static void smu_v11_0_i2c_clear_status(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	/* do */
 	{
 		RREG32_SOC15(SMUIO, 0, mmCKSVII2C_IC_CLR_INTR);
 static void smu_v11_0_i2c_configure(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	uint32_t reg = 0;
 	reg = REG_SET_FIELD(reg, CKSVII2C_IC_CON, IC_SLAVE_DISABLE, 1);
 static void smu_v11_0_i2c_set_clock(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	/*
 	 * Standard mode speed, These values are taken from SMUIO MAS,
 static void smu_v11_0_i2c_set_address(struct i2c_adapter *control, u16 address)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	/* The IC_TAR::IC_TAR field is 10-bits wide.
 	 * It takes a 7-bit or 10-bit addresses as an address,
 static uint32_t smu_v11_0_i2c_poll_tx_status(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	uint32_t ret = I2C_OK;
 	uint32_t reg, reg_c_tx_abrt_source;
 static uint32_t smu_v11_0_i2c_poll_rx_status(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	uint32_t ret = I2C_OK;
 	uint32_t reg_ic_status, reg_c_tx_abrt_source;
 				       u16 address, u8 *data,
 				       u32 numbytes, u32 i2c_flag)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	u32 bytes_sent, reg, ret = I2C_OK;
 	unsigned long  timeout_counter;
 				      u16 address, u8 *data,
 				      u32 numbytes, u32 i2c_flag)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	uint32_t bytes_received, ret = I2C_OK;
 	bytes_received = 0;
 static void smu_v11_0_i2c_abort(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	uint32_t reg = 0;
 	/* Enable I2C engine; */
 static bool smu_v11_0_i2c_activity_done(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	const uint32_t IDLE_TIMEOUT = 1024;
 	uint32_t timeout_count = 0;
 static void smu_v11_0_i2c_fini(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	u32 status, enable, en_stat;
 	int res;
 static bool smu_v11_0_i2c_bus_lock(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	/* Send  PPSMC_MSG_RequestI2CBus */
 	if (!amdgpu_dpm_smu_i2c_bus_access(adev, true))
 static bool smu_v11_0_i2c_bus_unlock(struct i2c_adapter *control)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(control);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	/* Send  PPSMC_MSG_ReleaseI2CBus */
 	if (!amdgpu_dpm_smu_i2c_bus_access(adev, false))
 	if (ret != I2C_OK)
 		DRM_ERROR("WriteI2CData() - I2C error occurred :%x", ret);
 	return ret;
 }
 static void lock_bus(struct i2c_adapter *i2c, unsigned int flags)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(i2c);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(i2c);
+	struct amdgpu_device *adev = smu_i2c->adev;
-	mutex_lock(&adev->pm.smu_i2c_mutex);
+	mutex_lock(&smu_i2c->mutex);
 	if (!smu_v11_0_i2c_bus_lock(i2c))
 		DRM_ERROR("Failed to lock the bus from SMU");
 	else
 static void unlock_bus(struct i2c_adapter *i2c, unsigned int flags)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(i2c);
+	struct amdgpu_smu_i2c_bus *smu_i2c = i2c_get_adapdata(i2c);
+	struct amdgpu_device *adev = smu_i2c->adev;
 	if (!smu_v11_0_i2c_bus_unlock(i2c))
 		DRM_ERROR("Failed to unlock the bus from SMU");
 	else
 		adev->pm.bus_locked = false;
-	mutex_unlock(&adev->pm.smu_i2c_mutex);
+	mutex_unlock(&smu_i2c->mutex);
 }
 static const struct i2c_lock_operations smu_v11_0_i2c_i2c_lock_ops = {
 	.flags = I2C_AQ_NO_ZERO_LEN,
 };
-int smu_v11_0_i2c_control_init(struct i2c_adapter *control)
+int smu_v11_0_i2c_control_init(struct amdgpu_device *adev)
 {
-	struct amdgpu_device *adev = to_amdgpu_device(control);
+	struct amdgpu_smu_i2c_bus *smu_i2c = &adev->pm.smu_i2c[0];
+	struct i2c_adapter *control = &smu_i2c->adapter;
 	int res;
-	mutex_init(&adev->pm.smu_i2c_mutex);
+	smu_i2c->adev = adev;
+	smu_i2c->port = 0;
+	mutex_init(&smu_i2c->mutex);
 	control->owner = THIS_MODULE;
 	control->class = I2C_CLASS_HWMON;
 	control->dev.parent = &adev->pdev->dev;
 	control->algo = &smu_v11_0_i2c_algo;
-	snprintf(control->name, sizeof(control->name), "AMDGPU SMU");
+	snprintf(control->name, sizeof(control->name), "AMDGPU SMU 0");
 	control->lock_ops = &smu_v11_0_i2c_i2c_lock_ops;
 	control->quirks = &smu_v11_0_i2c_control_quirks;
+	i2c_set_adapdata(control, smu_i2c);
+	adev->pm.ras_eeprom_i2c_bus = &adev->pm.smu_i2c[0].adapter;
+	adev->pm.fru_eeprom_i2c_bus = &adev->pm.smu_i2c[0].adapter;
 	res = i2c_add_adapter(control);
 	if (res)
 	return res;
 }
-void smu_v11_0_i2c_control_fini(struct i2c_adapter *control)
+void smu_v11_0_i2c_control_fini(struct amdgpu_device *adev)
 {
+	struct i2c_adapter *control = adev->pm.ras_eeprom_i2c_bus;
 	i2c_del_adapter(control);
+	adev->pm.ras_eeprom_i2c_bus = NULL;
+	adev->pm.fru_eeprom_i2c_bus = NULL;
 }
 /*

drivers/gpu/drm/amd/amdgpu/smu_v11_0_i2c.h+3−3

@@ -26,9 +26,9 @@ · 25 unmodified lines
26		26
27	#include <linux/types.h>	27	#include <linux/types.h>
28		28
29	struct i2c_adapter;	29	struct amdgpu_device;
30		30
31	int smu_v11_0_i2c_control_init(struct i2c_adapter *control);	31	int smu_v11_0_i2c_control_init(struct amdgpu_device *adev);
32	void smu_v11_0_i2c_control_fini(struct i2c_adapter *control);	32	void smu_v11_0_i2c_control_fini(struct amdgpu_device *adev);
33		33
34	#endif	34	#endif

drivers/gpu/drm/amd/amdgpu/soc15.c+14−50

 	return false;
 }
-static bool soc15_read_bios_from_rom(struct amdgpu_device *adev,
-				     u8 *bios, u32 length_bytes)
-{
-	u32 *dw_ptr;
-	u32 i, length_dw;
-	uint32_t rom_index_offset;
-	uint32_t rom_data_offset;
-	if (bios == NULL)
-		return false;
-	if (length_bytes == 0)
-		return false;
-	/* APU vbios image is part of sbios image */
-	if (adev->flags & AMD_IS_APU)
-		return false;
-	dw_ptr = (u32 *)bios;
-	length_dw = ALIGN(length_bytes, 4) / 4;
-	rom_index_offset =
-		adev->smuio.funcs->get_rom_index_offset(adev);
-	rom_data_offset =
-		adev->smuio.funcs->get_rom_data_offset(adev);
-	/* set rom index to 0 */
-	WREG32(rom_index_offset, 0);
-	/* read out the rom data */
-	for (i = 0; i < length_dw; i++)
-		dw_ptr[i] = RREG32(rom_data_offset);
-	return true;
-}
 static struct soc15_allowed_register_entry soc15_allowed_read_registers[] = {
 	{ SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS)},
 	{ SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS2)},
 static void soc15_program_aspm(struct amdgpu_device *adev)
 {
-	if (!amdgpu_aspm)
+	if (!amdgpu_device_should_use_aspm(adev))
 		return;
 	if (!(adev->flags & AMD_IS_APU) &&
 static const struct amdgpu_asic_funcs soc15_asic_funcs =
 {
 	.read_disabled_bios = &soc15_read_disabled_bios,
-	.read_bios_from_rom = &soc15_read_bios_from_rom,
+	.read_bios_from_rom = &amdgpu_soc15_read_bios_from_rom,
 	.read_register = &soc15_read_register,
 	.reset = &soc15_asic_reset,
 	.reset_method = &soc15_asic_reset_method,
 static const struct amdgpu_asic_funcs vega20_asic_funcs =
 {
 	.read_disabled_bios = &soc15_read_disabled_bios,
-	.read_bios_from_rom = &soc15_read_bios_from_rom,
+	.read_bios_from_rom = &amdgpu_soc15_read_bios_from_rom,
 	.read_register = &soc15_read_register,
 	.reset = &soc15_asic_reset,
 	.reset_method = &soc15_asic_reset_method,
 				AMD_CG_SUPPORT_SDMA_LS |
 				AMD_CG_SUPPORT_VCN_MGCG;
+			/*
+			 * MMHUB PG needs to be disabled for Picasso for
+			 * stability reasons.
+			 */
 			adev->pg_flags = AMD_PG_SUPPORT_SDMA |
-				AMD_PG_SUPPORT_MMHUB |
 				AMD_PG_SUPPORT_VCN;
 		} else {
 			adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG |
 static int soc15_common_late_init(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-	int r = 0;
 	if (amdgpu_sriov_vf(adev))
 		xgpu_ai_mailbox_get_irq(adev);
-	if (adev->nbio.ras_funcs &&
+	return 0;
-	    adev->nbio.ras_funcs->ras_late_init)
-		r = adev->nbio.ras_funcs->ras_late_init(adev);
-	return r;
 }
 static int soc15_common_sw_init(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-	if (adev->nbio.ras_funcs &&
+	if (adev->nbio.ras && adev->nbio.ras->ras_block.ras_fini)
-	    adev->nbio.ras_funcs->ras_fini)
+		adev->nbio.ras->ras_block.ras_fini(adev);
-		adev->nbio.ras_funcs->ras_fini(adev);
 	if (adev->df.funcs &&
 	    adev->df.funcs->sw_fini)
 	if (adev->nbio.ras_if &&
 	    amdgpu_ras_is_supported(adev, adev->nbio.ras_if->block)) {
-		if (adev->nbio.ras_funcs &&
+		if (adev->nbio.ras &&
-		    adev->nbio.ras_funcs->init_ras_controller_interrupt)
+		    adev->nbio.ras->init_ras_controller_interrupt)
 			amdgpu_irq_put(adev, &adev->nbio.ras_controller_irq, 0);
-		if (adev->nbio.ras_funcs &&
+		if (adev->nbio.ras &&
-		    adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt)
+		    adev->nbio.ras->init_ras_err_event_athub_interrupt)
 			amdgpu_irq_put(adev, &adev->nbio.ras_err_event_athub_irq, 0);
 	}

drivers/gpu/drm/amd/amdgpu/soc15_common.h+4−4

@@ -28,13 +28,13 @@ · 27 unmodified lines
28	#define SOC15_REG_OFFSET(ip, inst, reg) (adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg)	28	#define SOC15_REG_OFFSET(ip, inst, reg) (adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg)
29		29
30	#define __WREG32_SOC15_RLC__(reg, value, flag, hwip) \	30	#define __WREG32_SOC15_RLC__(reg, value, flag, hwip) \
31	((amdgpu_sriov_vf(adev) && adev->gfx.rlc.funcs && adev->gfx.rlc.funcs->sriov_wreg) ? \	31	((amdgpu_sriov_vf(adev) && adev->gfx.rlc.funcs && adev->gfx.rlc.rlcg_reg_access_supported) ? \
32	adev->gfx.rlc.funcs->sriov_wreg(adev, reg, value, flag, hwip) : \	32	amdgpu_sriov_wreg(adev, reg, value, flag, hwip) : \
33	WREG32(reg, value))	33	WREG32(reg, value))
34		34
35	#define __RREG32_SOC15_RLC__(reg, flag, hwip) \	35	#define __RREG32_SOC15_RLC__(reg, flag, hwip) \
36	((amdgpu_sriov_vf(adev) && adev->gfx.rlc.funcs && adev->gfx.rlc.funcs->sriov_rreg) ? \	36	((amdgpu_sriov_vf(adev) && adev->gfx.rlc.funcs && adev->gfx.rlc.rlcg_reg_access_supported) ? \
37	adev->gfx.rlc.funcs->sriov_rreg(adev, reg, flag, hwip) : \	37	amdgpu_sriov_rreg(adev, reg, flag, hwip) : \
38	RREG32(reg))	38	RREG32(reg))
39		39
40	#define WREG32_FIELD15(ip, idx, reg, field, val) \	40	#define WREG32_FIELD15(ip, idx, reg, field, val) \

drivers/gpu/drm/amd/amdgpu/ta_ras_if.h+2−1

@@ -64,7 +64,8 @@ enum ta_ras_status { · 63 unmodified lines
64	TA_RAS_STATUS__ERROR_PCS_STATE_ERROR = 0xA016,	64	TA_RAS_STATUS__ERROR_PCS_STATE_ERROR = 0xA016,
65	TA_RAS_STATUS__ERROR_PCS_STATE_HANG = 0xA017,	65	TA_RAS_STATUS__ERROR_PCS_STATE_HANG = 0xA017,
66	TA_RAS_STATUS__ERROR_PCS_STATE_UNKNOWN = 0xA018,	66	TA_RAS_STATUS__ERROR_PCS_STATE_UNKNOWN = 0xA018,
67	TA_RAS_STATUS__ERROR_UNSUPPORTED_ERROR_INJ = 0xA019	67	TA_RAS_STATUS__ERROR_UNSUPPORTED_ERROR_INJ = 0xA019,
		68	TA_RAS_STATUS__TEE_ERROR_ACCESS_DENIED = 0xA01A
68	};	69	};
69		70
70	enum ta_ras_block {	71	enum ta_ras_block {

drivers/gpu/drm/amd/amdgpu/umc_v6_1.c+11−19

 {
 	uint32_t lsb, mc_umc_status_addr;
 	uint64_t mc_umc_status, err_addr, retired_page, mc_umc_addrt0;
-	struct eeprom_table_record *err_rec;
 	uint32_t channel_index = adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
 	if (adev->asic_type == CHIP_ARCTURUS) {
 		return;
 	}
-	err_rec = &err_data->err_addr[err_data->err_addr_cnt];
 	/* calculate error address if ue/ce error is detected */
 	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
 	    (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 ||
 		/* we only save ue error information currently, ce is skipped */
 		if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)
-				== 1) {
+				== 1)
-			err_rec->address = err_addr;
+			amdgpu_umc_fill_error_record(err_data, err_addr,
-			/* page frame address is saved */
+					retired_page, channel_index, umc_inst);
-			err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;
-			err_rec->ts = (uint64_t)ktime_get_real_seconds();
-			err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
-			err_rec->cu = 0;
-			err_rec->mem_channel = channel_index;
-			err_rec->mcumc_id = umc_inst;
-			err_data->err_addr_cnt++;
-		}
 	}
 	/* clear umc status */
 		umc_v6_1_enable_umc_index_mode(adev);
 }
-const struct amdgpu_umc_ras_funcs umc_v6_1_ras_funcs = {
+const struct amdgpu_ras_block_hw_ops umc_v6_1_ras_hw_ops = {
-	.err_cnt_init = umc_v6_1_err_cnt_init,
-	.ras_late_init = amdgpu_umc_ras_late_init,
-	.ras_fini = amdgpu_umc_ras_fini,
 	.query_ras_error_count = umc_v6_1_query_ras_error_count,
 	.query_ras_error_address = umc_v6_1_query_ras_error_address,
 };
+struct amdgpu_umc_ras umc_v6_1_ras = {
+	.ras_block = {
+		.hw_ops = &umc_v6_1_ras_hw_ops,
+	},
+	.err_cnt_init = umc_v6_1_err_cnt_init,
+};

drivers/gpu/drm/amd/amdgpu/umc_v6_1.h+1−1

@@ -45,7 +45,7 @@ · 44 unmodified lines
45	/* umc ce count initial value */	45	/* umc ce count initial value */
46	#define UMC_V6_1_CE_CNT_INIT (UMC_V6_1_CE_CNT_MAX - UMC_V6_1_CE_INT_THRESHOLD)	46	#define UMC_V6_1_CE_CNT_INIT (UMC_V6_1_CE_CNT_MAX - UMC_V6_1_CE_INT_THRESHOLD)
47		47
48	extern const struct amdgpu_umc_ras_funcs umc_v6_1_ras_funcs;	48	extern struct amdgpu_umc_ras umc_v6_1_ras;
49	extern const uint32_t	49	extern const uint32_t
50	umc_v6_1_channel_idx_tbl[UMC_V6_1_UMC_INSTANCE_NUM][UMC_V6_1_CHANNEL_INSTANCE_NUM];	50	umc_v6_1_channel_idx_tbl[UMC_V6_1_UMC_INSTANCE_NUM][UMC_V6_1_CHANNEL_INSTANCE_NUM];
51		51

drivers/gpu/drm/amd/amdgpu/umc_v6_7.c+142−83

@@ -47,6 +47,13 @@ static inline uint32_t get_umc_v6_7_reg_offset(struct amdgpu_device *adev, · 46 unmodified lines
47	uint32_t umc_inst,	47	uint32_t umc_inst,
48	uint32_t ch_inst)	48	uint32_t ch_inst)
49	{	49	{
		50	uint32_t index = umc_inst * adev->umc.channel_inst_num + ch_inst;
		51
		52	/* adjust umc and channel index offset,
		53	* the register address is not linear on each umc instace */
		54	umc_inst = index / 4;
		55	ch_inst = index % 4;
		56
50	return adev->umc.channel_offs * ch_inst + UMC_V6_7_INST_DIST * umc_inst;	57	return adev->umc.channel_offs * ch_inst + UMC_V6_7_INST_DIST * umc_inst;
51	}	58	}
52		59
@@ -58,49 +65,74 @@ static inline uint32_t get_umc_v6_7_channel_index(struct amdgpu_device *adev, · 5 unmodified lines
58	}	65	}
59		66
60	static void umc_v6_7_ecc_info_query_correctable_error_count(struct amdgpu_device *adev,	67	static void umc_v6_7_ecc_info_query_correctable_error_count(struct amdgpu_device *adev,
61	uint32_t channel_index,	68	uint32_t umc_inst, uint32_t ch_inst,
62	unsigned long *error_count)	69	unsigned long *error_count)
63	{	70	{
64	uint32_t ecc_err_cnt;
65	uint64_t mc_umc_status;	71	uint64_t mc_umc_status;
		72	uint32_t eccinfo_table_idx;
66	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);	73	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
67		74
68	/*	75	eccinfo_table_idx = umc_inst * adev->umc.channel_inst_num + ch_inst;
69	* select the lower chip and check the error count
70	* skip add error count, calc error counter only from mca_umc_status
71	*/
72	ecc_err_cnt = ras->umc_ecc.ecc[channel_index].ce_count_lo_chip;
73
74	/*
75	* select the higher chip and check the err counter
76	* skip add error count, calc error counter only from mca_umc_status
77	*/
78	ecc_err_cnt = ras->umc_ecc.ecc[channel_index].ce_count_hi_chip;
79
80	/* check for SRAM correctable error	76	/* check for SRAM correctable error
81	MCUMC_STATUS is a 64 bit register */	77	MCUMC_STATUS is a 64 bit register */
82	mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status;	78	mc_umc_status = ras->umc_ecc.ecc[eccinfo_table_idx].mca_umc_status;
83	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&	79	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
84	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)	80	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)
85	*error_count += 1;	81	*error_count += 1;
86	}	82	}
87		83
88	static void umc_v6_7_ecc_info_querry_uncorrectable_error_count(struct amdgpu_device *adev,	84	static void umc_v6_7_ecc_info_querry_uncorrectable_error_count(struct amdgpu_device *adev,
89	uint32_t channel_index,	85	uint32_t umc_inst, uint32_t ch_inst,
90	unsigned long *error_count)	86	unsigned long *error_count)
91	{	87	{
92	uint64_t mc_umc_status;	88	uint64_t mc_umc_status;
		89	uint32_t eccinfo_table_idx;
		90	uint32_t umc_reg_offset;
		91	uint32_t mc_umc_addr;
		92	uint64_t reg_value;
93	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);	93	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
94		94
		95	umc_reg_offset = get_umc_v6_7_reg_offset(adev,
		96	umc_inst, ch_inst);
		97
		98	eccinfo_table_idx = umc_inst * adev->umc.channel_inst_num + ch_inst;
95	/* check the MCUMC_STATUS */	99	/* check the MCUMC_STATUS */
96	mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status;	100	mc_umc_status = ras->umc_ecc.ecc[eccinfo_table_idx].mca_umc_status;
97	if ((REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1) &&	101	if ((REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1) &&
98	(REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1 \|\|	102	(REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1 \|\|
99	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|	103	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|
100	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 \|\|	104	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 \|\|
101	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 \|\|	105	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 \|\|
102	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1))	106	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1)) {
103	*error_count += 1;	107	*error_count += 1;
		108
		109	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1)
		110	dev_info(adev->dev, "Deferred error, no user action is needed.\n");
		111
		112	if (mc_umc_status)
		113	dev_info(adev->dev, "MCA STATUS 0x%llx, umc_reg_offset 0x%x\n", mc_umc_status, umc_reg_offset);
		114
		115	/* print IPID registers value */
		116	mc_umc_addr =
		117	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_IPIDT0);
		118	reg_value = RREG64_PCIE((mc_umc_addr + umc_reg_offset) * 4);
		119	if (reg_value)
		120	dev_info(adev->dev, "MCA IPID 0x%llx, umc_reg_offset 0x%x\n", reg_value, umc_reg_offset);
		121
		122	/* print SYND registers value */
		123	mc_umc_addr =
		124	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_SYNDT0);
		125	reg_value = RREG64_PCIE((mc_umc_addr + umc_reg_offset) * 4);
		126	if (reg_value)
		127	dev_info(adev->dev, "MCA SYND 0x%llx, umc_reg_offset 0x%x\n", reg_value, umc_reg_offset);
		128
		129	/* print MISC0 registers value */
		130	mc_umc_addr =
		131	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_MISC0T0);
		132	reg_value = RREG64_PCIE((mc_umc_addr + umc_reg_offset) * 4);
		133	if (reg_value)
		134	dev_info(adev->dev, "MCA MISC0 0x%llx, umc_reg_offset 0x%x\n", reg_value, umc_reg_offset);
		135	}
104	}	136	}
105		137
106	static void umc_v6_7_ecc_info_query_ras_error_count(struct amdgpu_device *adev,	138	static void umc_v6_7_ecc_info_query_ras_error_count(struct amdgpu_device *adev,
@@ -110,42 +142,34 @@ static void umc_v6_7_ecc_info_query_ras_error_count(struct amdgpu_device *adev, · 3 unmodified lines
110		142
111	uint32_t umc_inst = 0;	143	uint32_t umc_inst = 0;
112	uint32_t ch_inst = 0;	144	uint32_t ch_inst = 0;
113	uint32_t umc_reg_offset = 0;
114	uint32_t channel_index = 0;
115		145
116	/*TODO: driver needs to toggle DF Cstate to ensure	146	/*TODO: driver needs to toggle DF Cstate to ensure
117	* safe access of UMC registers. Will add the protection */	147	* safe access of UMC registers. Will add the protection */
118	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {	148	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
119	umc_reg_offset = get_umc_v6_7_reg_offset(adev,
120	umc_inst,
121	ch_inst);
122	channel_index = get_umc_v6_7_channel_index(adev,
123	umc_inst,
124	ch_inst);
125	umc_v6_7_ecc_info_query_correctable_error_count(adev,	149	umc_v6_7_ecc_info_query_correctable_error_count(adev,
126	channel_index,	150	umc_inst, ch_inst,
127	&(err_data->ce_count));	151	&(err_data->ce_count));
128	umc_v6_7_ecc_info_querry_uncorrectable_error_count(adev,	152	umc_v6_7_ecc_info_querry_uncorrectable_error_count(adev,
129	channel_index,	153	umc_inst, ch_inst,
130	&(err_data->ue_count));	154	&(err_data->ue_count));
131	}	155	}
132	}	156	}
133		157
134	static void umc_v6_7_ecc_info_query_error_address(struct amdgpu_device *adev,	158	static void umc_v6_7_ecc_info_query_error_address(struct amdgpu_device *adev,
135	struct ras_err_data *err_data,	159	struct ras_err_data *err_data,
136	uint32_t umc_reg_offset,
137	uint32_t ch_inst,	160	uint32_t ch_inst,
138	uint32_t umc_inst)	161	uint32_t umc_inst)
139	{	162	{
140	uint64_t mc_umc_status, err_addr, retired_page;	163	uint64_t mc_umc_status, err_addr, soc_pa, retired_page, column;
141	struct eeprom_table_record *err_rec;
142	uint32_t channel_index;	164	uint32_t channel_index;
		165	uint32_t eccinfo_table_idx;
143	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);	166	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
144		167
		168	eccinfo_table_idx = umc_inst * adev->umc.channel_inst_num + ch_inst;
145	channel_index =	169	channel_index =
146	adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];	170	adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
147		171
148	mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status;	172	mc_umc_status = ras->umc_ecc.ecc[eccinfo_table_idx].mca_umc_status;
149		173
150	if (mc_umc_status == 0)	174	if (mc_umc_status == 0)
151	return;	175	return;
@@ -153,34 +177,41 @@ static void umc_v6_7_ecc_info_query_error_address(struct amdgpu_device *adev, · 1 unmodified line
153	if (!err_data->err_addr)	177	if (!err_data->err_addr)
154	return;	178	return;
155		179
156	err_rec = &err_data->err_addr[err_data->err_addr_cnt];
157
158	/* calculate error address if ue/ce error is detected */	180	/* calculate error address if ue/ce error is detected */
159	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&	181	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
160	(REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|	182	(REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|
161	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)) {	183	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)) {
162		184
163	err_addr = ras->umc_ecc.ecc[channel_index].mca_umc_addr;	185	err_addr = ras->umc_ecc.ecc[eccinfo_table_idx].mca_umc_addr;
164	err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr);	186	err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr);
165		187
166	/* translate umc channel address to soc pa, 3 parts are included */	188	/* translate umc channel address to soc pa, 3 parts are included */
167	retired_page = ADDR_OF_8KB_BLOCK(err_addr) \|	189	soc_pa = ADDR_OF_8KB_BLOCK(err_addr) \|
168	ADDR_OF_256B_BLOCK(channel_index) \|	190	ADDR_OF_256B_BLOCK(channel_index) \|
169	OFFSET_IN_256B_BLOCK(err_addr);	191	OFFSET_IN_256B_BLOCK(err_addr);
170		192
		193	/* The umc channel bits are not original values, they are hashed */
		194	SET_CHANNEL_HASH(channel_index, soc_pa);
		195
		196	/* clear [C4 C3 C2] in soc physical address */
		197	soc_pa &= ~(0x7ULL << UMC_V6_7_PA_C2_BIT);
		198
171	/* we only save ue error information currently, ce is skipped */	199	/* we only save ue error information currently, ce is skipped */
172	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)	200	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)
173	== 1) {	201	== 1) {
174	err_rec->address = err_addr;	202	/* loop for all possibilities of [C4 C3 C2] */
175	/* page frame address is saved */	203	for (column = 0; column < UMC_V6_7_NA_MAP_PA_NUM; column++) {
176	err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;	204	retired_page = soc_pa \| (column << UMC_V6_7_PA_C2_BIT);
177	err_rec->ts = (uint64_t)ktime_get_real_seconds();	205	dev_info(adev->dev, "Error Address(PA): 0x%llx\n", retired_page);
178	err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;	206	amdgpu_umc_fill_error_record(err_data, err_addr,
179	err_rec->cu = 0;	207	retired_page, channel_index, umc_inst);
180	err_rec->mem_channel = channel_index;	208
181	err_rec->mcumc_id = umc_inst;	209	/* shift R14 bit */
182		210	retired_page ^= (0x1ULL << UMC_V6_7_PA_R14_BIT);
183	err_data->err_addr_cnt++;	211	dev_info(adev->dev, "Error Address(PA): 0x%llx\n", retired_page);
		212	amdgpu_umc_fill_error_record(err_data, err_addr,
		213	retired_page, channel_index, umc_inst);
		214	}
184	}	215	}
185	}	216	}
186	}	217	}
@@ -192,18 +223,13 @@ static void umc_v6_7_ecc_info_query_ras_error_address(struct amdgpu_device *adev · 5 unmodified lines
192		223
193	uint32_t umc_inst = 0;	224	uint32_t umc_inst = 0;
194	uint32_t ch_inst = 0;	225	uint32_t ch_inst = 0;
195	uint32_t umc_reg_offset = 0;
196		226
197	/*TODO: driver needs to toggle DF Cstate to ensure	227	/*TODO: driver needs to toggle DF Cstate to ensure
198	* safe access of UMC resgisters. Will add the protection	228	* safe access of UMC resgisters. Will add the protection
199	* when firmware interface is ready */	229	* when firmware interface is ready */
200	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {	230	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
201	umc_reg_offset = get_umc_v6_7_reg_offset(adev,
202	umc_inst,
203	ch_inst);
204	umc_v6_7_ecc_info_query_error_address(adev,	231	umc_v6_7_ecc_info_query_error_address(adev,
205	err_data,	232	err_data,
206	umc_reg_offset,
207	ch_inst,	233	ch_inst,
208	umc_inst);	234	umc_inst);
209	}	235	}
@@ -261,6 +287,8 @@ static void umc_v6_7_querry_uncorrectable_error_count(struct amdgpu_device *adev · 51 unmodified lines
261	{	287	{
262	uint64_t mc_umc_status;	288	uint64_t mc_umc_status;
263	uint32_t mc_umc_status_addr;	289	uint32_t mc_umc_status_addr;
		290	uint32_t mc_umc_addr;
		291	uint64_t reg_value;
264		292
265	mc_umc_status_addr =	293	mc_umc_status_addr =
266	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);	294	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);
@@ -272,8 +300,36 @@ static void umc_v6_7_querry_uncorrectable_error_count(struct amdgpu_device *adev · 5 unmodified lines
272	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|	300	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|
273	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 \|\|	301	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 \|\|
274	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 \|\|	302	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 \|\|
275	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1))	303	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1)) {
276	*error_count += 1;	304	*error_count += 1;
		305
		306	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1)
		307	dev_info(adev->dev, "Deferred error, no user action is needed.\n");
		308
		309	if (mc_umc_status)
		310	dev_info(adev->dev, "MCA STATUS 0x%llx, umc_reg_offset 0x%x\n", mc_umc_status, umc_reg_offset);
		311
		312	/* print IPID registers value */
		313	mc_umc_addr =
		314	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_IPIDT0);
		315	reg_value = RREG64_PCIE((mc_umc_addr + umc_reg_offset) * 4);
		316	if (reg_value)
		317	dev_info(adev->dev, "MCA IPID 0x%llx, umc_reg_offset 0x%x\n", reg_value, umc_reg_offset);
		318
		319	/* print SYND registers value */
		320	mc_umc_addr =
		321	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_SYNDT0);
		322	reg_value = RREG64_PCIE((mc_umc_addr + umc_reg_offset) * 4);
		323	if (reg_value)
		324	dev_info(adev->dev, "MCA SYND 0x%llx, umc_reg_offset 0x%x\n", reg_value, umc_reg_offset);
		325
		326	/* print MISC0 registers value */
		327	mc_umc_addr =
		328	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_MISC0T0);
		329	reg_value = RREG64_PCIE((mc_umc_addr + umc_reg_offset) * 4);
		330	if (reg_value)
		331	dev_info(adev->dev, "MCA MISC0 0x%llx, umc_reg_offset 0x%x\n", reg_value, umc_reg_offset);
		332	}
277	}	333	}
278		334
279	static void umc_v6_7_reset_error_count_per_channel(struct amdgpu_device *adev,	335	static void umc_v6_7_reset_error_count_per_channel(struct amdgpu_device *adev,
@@ -365,9 +421,9 @@ static void umc_v6_7_query_error_address(struct amdgpu_device *adev, · 85 unmodified lines
365	uint32_t umc_inst)	421	uint32_t umc_inst)
366	{	422	{
367	uint32_t mc_umc_status_addr;	423	uint32_t mc_umc_status_addr;
368	uint64_t mc_umc_status, err_addr, retired_page, mc_umc_addrt0;
369	struct eeprom_table_record *err_rec;
370	uint32_t channel_index;	424	uint32_t channel_index;
		425	uint64_t mc_umc_status, mc_umc_addrt0;
		426	uint64_t err_addr, soc_pa, retired_page, column;
371		427
372	mc_umc_status_addr =	428	mc_umc_status_addr =
373	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);	429	SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);
@@ -385,8 +441,6 @@ static void umc_v6_7_query_error_address(struct amdgpu_device *adev, · 11 unmodified lines
385	return;	441	return;
386	}	442	}
387		443
388	err_rec = &err_data->err_addr[err_data->err_addr_cnt];
389
390	channel_index =	444	channel_index =
391	adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];	445	adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
392		446
@@ -399,23 +453,32 @@ static void umc_v6_7_query_error_address(struct amdgpu_device *adev, · 6 unmodified lines
399	err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr);	453	err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr);
400		454
401	/* translate umc channel address to soc pa, 3 parts are included */	455	/* translate umc channel address to soc pa, 3 parts are included */
402	retired_page = ADDR_OF_8KB_BLOCK(err_addr) \|	456	soc_pa = ADDR_OF_8KB_BLOCK(err_addr) \|
403	ADDR_OF_256B_BLOCK(channel_index) \|	457	ADDR_OF_256B_BLOCK(channel_index) \|
404	OFFSET_IN_256B_BLOCK(err_addr);	458	OFFSET_IN_256B_BLOCK(err_addr);
405		459
		460	/* The umc channel bits are not original values, they are hashed */
		461	SET_CHANNEL_HASH(channel_index, soc_pa);
		462
		463	/* clear [C4 C3 C2] in soc physical address */
		464	soc_pa &= ~(0x7ULL << UMC_V6_7_PA_C2_BIT);
		465
406	/* we only save ue error information currently, ce is skipped */	466	/* we only save ue error information currently, ce is skipped */
407	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)	467	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)
408	== 1) {	468	== 1) {
409	err_rec->address = err_addr;	469	/* loop for all possibilities of [C4 C3 C2] */
410	/* page frame address is saved */	470	for (column = 0; column < UMC_V6_7_NA_MAP_PA_NUM; column++) {
411	err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;	471	retired_page = soc_pa \| (column << UMC_V6_7_PA_C2_BIT);
412	err_rec->ts = (uint64_t)ktime_get_real_seconds();	472	dev_info(adev->dev, "Error Address(PA): 0x%llx\n", retired_page);
413	err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;	473	amdgpu_umc_fill_error_record(err_data, err_addr,
414	err_rec->cu = 0;	474	retired_page, channel_index, umc_inst);
415	err_rec->mem_channel = channel_index;	475
416	err_rec->mcumc_id = umc_inst;	476	/* shift R14 bit */
417		477	retired_page ^= (0x1ULL << UMC_V6_7_PA_R14_BIT);
418	err_data->err_addr_cnt++;	478	dev_info(adev->dev, "Error Address(PA): 0x%llx\n", retired_page);
		479	amdgpu_umc_fill_error_record(err_data, err_addr,
		480	retired_page, channel_index, umc_inst);
		481	}
419	}	482	}
420	}	483	}
421		484
@@ -463,28 +526,24 @@ static uint32_t umc_v6_7_query_ras_poison_mode_per_channel( · 41 unmodified lines
463		526
464	static bool umc_v6_7_query_ras_poison_mode(struct amdgpu_device *adev)	527	static bool umc_v6_7_query_ras_poison_mode(struct amdgpu_device *adev)
465	{	528	{
466	uint32_t umc_inst = 0;
467	uint32_t ch_inst = 0;
468	uint32_t umc_reg_offset = 0;	529	uint32_t umc_reg_offset = 0;
469		530
470	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {	531	/* Enabling fatal error in umc instance0 channel0 will be
471	umc_reg_offset = get_umc_v6_7_reg_offset(adev,	532	* considered as fatal error mode
472	umc_inst,	533	*/
473	ch_inst);	534	umc_reg_offset = get_umc_v6_7_reg_offset(adev, 0, 0);
474	/* Enabling fatal error in one channel will be considered	535	return !umc_v6_7_query_ras_poison_mode_per_channel(adev, umc_reg_offset);
475	as fatal error mode */
476	if (umc_v6_7_query_ras_poison_mode_per_channel(adev, umc_reg_offset))
477	return false;
478	}
479
480	return true;
481	}	536	}
482		537
483	const struct amdgpu_umc_ras_funcs umc_v6_7_ras_funcs = {	538	const struct amdgpu_ras_block_hw_ops umc_v6_7_ras_hw_ops = {
484	.ras_late_init = amdgpu_umc_ras_late_init,
485	.ras_fini = amdgpu_umc_ras_fini,
486	.query_ras_error_count = umc_v6_7_query_ras_error_count,	539	.query_ras_error_count = umc_v6_7_query_ras_error_count,
487	.query_ras_error_address = umc_v6_7_query_ras_error_address,	540	.query_ras_error_address = umc_v6_7_query_ras_error_address,
		541	};
		542
		543	struct amdgpu_umc_ras umc_v6_7_ras = {
		544	.ras_block = {
		545	.hw_ops = &umc_v6_7_ras_hw_ops,
		546	},
488	.query_ras_poison_mode = umc_v6_7_query_ras_poison_mode,	547	.query_ras_poison_mode = umc_v6_7_query_ras_poison_mode,
489	.ecc_info_query_ras_error_count = umc_v6_7_ecc_info_query_ras_error_count,	548	.ecc_info_query_ras_error_count = umc_v6_7_ecc_info_query_ras_error_count,
490	.ecc_info_query_ras_error_address = umc_v6_7_ecc_info_query_ras_error_address,	549	.ecc_info_query_ras_error_address = umc_v6_7_ecc_info_query_ras_error_address,

drivers/gpu/drm/amd/amdgpu/umc_v6_7.h+24−1

@@ -41,9 +41,32 @@ · 40 unmodified lines
41	#define UMC_V6_7_CHANNEL_INSTANCE_NUM 8	41	#define UMC_V6_7_CHANNEL_INSTANCE_NUM 8
42	/* total channel instances in one umc block */	42	/* total channel instances in one umc block */
43	#define UMC_V6_7_TOTAL_CHANNEL_NUM (UMC_V6_7_CHANNEL_INSTANCE_NUM * UMC_V6_7_UMC_INSTANCE_NUM)	43	#define UMC_V6_7_TOTAL_CHANNEL_NUM (UMC_V6_7_CHANNEL_INSTANCE_NUM * UMC_V6_7_UMC_INSTANCE_NUM)
		44	/* one piece of normalizing address is mapped to 8 pieces of physical address */
		45	#define UMC_V6_7_NA_MAP_PA_NUM 8
		46	/* R14 bit shift should be considered, double the number */
		47	#define UMC_V6_7_BAD_PAGE_NUM_PER_CHANNEL (UMC_V6_7_NA_MAP_PA_NUM * 2)
		48	/* The CH4 bit in SOC physical address */
		49	#define UMC_V6_7_PA_CH4_BIT 12
		50	/* The C2 bit in SOC physical address */
		51	#define UMC_V6_7_PA_C2_BIT 17
		52	/* The R14 bit in SOC physical address */
		53	#define UMC_V6_7_PA_R14_BIT 34
44	/* UMC regiser per channel offset */	54	/* UMC regiser per channel offset */
45	#define UMC_V6_7_PER_CHANNEL_OFFSET 0x400	55	#define UMC_V6_7_PER_CHANNEL_OFFSET 0x400
46	extern const struct amdgpu_umc_ras_funcs umc_v6_7_ras_funcs;	56
		57	/* XOR bit 20, 25, 34 of PA into CH4 bit (bit 12 of PA),
		58	* hash bit is only effective when related setting is enabled
		59	*/
		60	#define CHANNEL_HASH(channel_idx, pa) (((channel_idx) >> 4) ^ \
		61	(((pa) >> 20) & 0x1ULL & adev->df.hash_status.hash_64k) ^ \
		62	(((pa) >> 25) & 0x1ULL & adev->df.hash_status.hash_2m) ^ \
		63	(((pa) >> 34) & 0x1ULL & adev->df.hash_status.hash_1g))
		64	#define SET_CHANNEL_HASH(channel_idx, pa) do { \
		65	(pa) &= ~(0x1ULL << UMC_V6_7_PA_CH4_BIT); \
		66	(pa) \|= (CHANNEL_HASH(channel_idx, pa) << UMC_V6_7_PA_CH4_BIT); \
		67	} while (0)
		68
		69	extern struct amdgpu_umc_ras umc_v6_7_ras;
47	extern const uint32_t	70	extern const uint32_t
48	umc_v6_7_channel_idx_tbl_second[UMC_V6_7_UMC_INSTANCE_NUM][UMC_V6_7_CHANNEL_INSTANCE_NUM];	71	umc_v6_7_channel_idx_tbl_second[UMC_V6_7_UMC_INSTANCE_NUM][UMC_V6_7_CHANNEL_INSTANCE_NUM];
49	extern const uint32_t	72	extern const uint32_t

drivers/gpu/drm/amd/amdgpu/umc_v8_7.c+149−24

@@ -40,13 +40,144 @@ const uint32_t · 39 unmodified lines
40	{9, 0}, {15, 6}	40	{9, 0}, {15, 6}
41	};	41	};
42		42
43	static inline uint32_t get_umc_8_reg_offset(struct amdgpu_device *adev,	43	static inline uint32_t get_umc_v8_7_reg_offset(struct amdgpu_device *adev,
44	uint32_t umc_inst,	44	uint32_t umc_inst,
45	uint32_t ch_inst)	45	uint32_t ch_inst)
46	{	46	{
47	return adev->umc.channel_offsch_inst + UMC_8_INST_DISTumc_inst;	47	return adev->umc.channel_offsch_inst + UMC_8_INST_DISTumc_inst;
48	}	48	}
49		49
		50	static inline uint32_t get_umc_v8_7_channel_index(struct amdgpu_device *adev,
		51	uint32_t umc_inst,
		52	uint32_t ch_inst)
		53	{
		54	return adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
		55	}
		56
		57	static void umc_v8_7_ecc_info_query_correctable_error_count(struct amdgpu_device *adev,
		58	uint32_t umc_inst, uint32_t ch_inst,
		59	unsigned long *error_count)
		60	{
		61	uint64_t mc_umc_status;
		62	uint32_t eccinfo_table_idx;
		63	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
		64
		65	eccinfo_table_idx = umc_inst * adev->umc.channel_inst_num + ch_inst;
		66
		67	/* check for SRAM correctable error
		68	* MCUMC_STATUS is a 64 bit register
		69	*/
		70	mc_umc_status = ras->umc_ecc.ecc[eccinfo_table_idx].mca_umc_status;
		71	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
		72	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)
		73	*error_count += 1;
		74	}
		75
		76	static void umc_v8_7_ecc_info_querry_uncorrectable_error_count(struct amdgpu_device *adev,
		77	uint32_t umc_inst, uint32_t ch_inst,
		78	unsigned long *error_count)
		79	{
		80	uint64_t mc_umc_status;
		81	uint32_t eccinfo_table_idx;
		82	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
		83
		84	eccinfo_table_idx = umc_inst * adev->umc.channel_inst_num + ch_inst;
		85
		86	/* check the MCUMC_STATUS */
		87	mc_umc_status = ras->umc_ecc.ecc[eccinfo_table_idx].mca_umc_status;
		88	if ((REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1) &&
		89	(REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1 \|\|
		90	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|
		91	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 \|\|
		92	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 \|\|
		93	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1))
		94	*error_count += 1;
		95	}
		96
		97	static void umc_v8_7_ecc_info_query_ras_error_count(struct amdgpu_device *adev,
		98	void *ras_error_status)
		99	{
		100	struct ras_err_data err_data = (struct ras_err_data )ras_error_status;
		101
		102	uint32_t umc_inst = 0;
		103	uint32_t ch_inst = 0;
		104
		105	/* TODO: driver needs to toggle DF Cstate to ensure
		106	* safe access of UMC registers. Will add the protection
		107	*/
		108	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
		109	umc_v8_7_ecc_info_query_correctable_error_count(adev,
		110	umc_inst, ch_inst,
		111	&(err_data->ce_count));
		112	umc_v8_7_ecc_info_querry_uncorrectable_error_count(adev,
		113	umc_inst, ch_inst,
		114	&(err_data->ue_count));
		115	}
		116	}
		117
		118	static void umc_v8_7_ecc_info_query_error_address(struct amdgpu_device *adev,
		119	struct ras_err_data *err_data,
		120	uint32_t ch_inst,
		121	uint32_t umc_inst)
		122	{
		123	uint64_t mc_umc_status, err_addr, retired_page;
		124	uint32_t channel_index;
		125	uint32_t eccinfo_table_idx;
		126	struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
		127
		128	eccinfo_table_idx = umc_inst * adev->umc.channel_inst_num + ch_inst;
		129	channel_index =
		130	adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
		131
		132	mc_umc_status = ras->umc_ecc.ecc[eccinfo_table_idx].mca_umc_status;
		133
		134	if (mc_umc_status == 0)
		135	return;
		136
		137	if (!err_data->err_addr)
		138	return;
		139
		140	/* calculate error address if ue/ce error is detected */
		141	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
		142	(REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|
		143	REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)) {
		144
		145	err_addr = ras->umc_ecc.ecc[eccinfo_table_idx].mca_umc_addr;
		146	err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr);
		147
		148	/* translate umc channel address to soc pa, 3 parts are included */
		149	retired_page = ADDR_OF_4KB_BLOCK(err_addr) \|
		150	ADDR_OF_256B_BLOCK(channel_index) \|
		151	OFFSET_IN_256B_BLOCK(err_addr);
		152
		153	/* we only save ue error information currently, ce is skipped */
		154	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)
		155	== 1)
		156	amdgpu_umc_fill_error_record(err_data, err_addr,
		157	retired_page, channel_index, umc_inst);
		158	}
		159	}
		160
		161	static void umc_v8_7_ecc_info_query_ras_error_address(struct amdgpu_device *adev,
		162	void *ras_error_status)
		163	{
		164	struct ras_err_data err_data = (struct ras_err_data )ras_error_status;
		165
		166	uint32_t umc_inst = 0;
		167	uint32_t ch_inst = 0;
		168
		169	/* TODO: driver needs to toggle DF Cstate to ensure
		170	* safe access of UMC resgisters. Will add the protection
		171	* when firmware interface is ready
		172	*/
		173	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
		174	umc_v8_7_ecc_info_query_error_address(adev,
		175	err_data,
		176	ch_inst,
		177	umc_inst);
		178	}
		179	}
		180
50	static void umc_v8_7_clear_error_count_per_channel(struct amdgpu_device *adev,	181	static void umc_v8_7_clear_error_count_per_channel(struct amdgpu_device *adev,
51	uint32_t umc_reg_offset)	182	uint32_t umc_reg_offset)
52	{	183	{
@@ -92,7 +223,7 @@ static void umc_v8_7_clear_error_count(struct amdgpu_device *adev) · 39 unmodified lines
92	uint32_t umc_reg_offset = 0;	223	uint32_t umc_reg_offset = 0;
93		224
94	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {	225	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
95	umc_reg_offset = get_umc_8_reg_offset(adev,	226	umc_reg_offset = get_umc_v8_7_reg_offset(adev,
96	umc_inst,	227	umc_inst,
97	ch_inst);	228	ch_inst);
98		229
@@ -178,7 +309,7 @@ static void umc_v8_7_query_ras_error_count(struct amdgpu_device *adev, · 79 unmodified lines
178	uint32_t umc_reg_offset = 0;	309	uint32_t umc_reg_offset = 0;
179		310
180	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {	311	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
181	umc_reg_offset = get_umc_8_reg_offset(adev,	312	umc_reg_offset = get_umc_v8_7_reg_offset(adev,
182	umc_inst,	313	umc_inst,
183	ch_inst);	314	ch_inst);
184		315
@@ -201,7 +332,6 @@ static void umc_v8_7_query_error_address(struct amdgpu_device *adev, · 16 unmodified lines
201	{	332	{
202	uint32_t lsb, mc_umc_status_addr;	333	uint32_t lsb, mc_umc_status_addr;
203	uint64_t mc_umc_status, err_addr, retired_page, mc_umc_addrt0;	334	uint64_t mc_umc_status, err_addr, retired_page, mc_umc_addrt0;
204	struct eeprom_table_record *err_rec;
205	uint32_t channel_index = adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];	335	uint32_t channel_index = adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
206		336
207	mc_umc_status_addr =	337	mc_umc_status_addr =
@@ -220,8 +350,6 @@ static void umc_v8_7_query_error_address(struct amdgpu_device *adev, · 12 unmodified lines
220	return;	350	return;
221	}	351	}
222		352
223	err_rec = &err_data->err_addr[err_data->err_addr_cnt];
224
225	/* calculate error address if ue/ce error is detected */	353	/* calculate error address if ue/ce error is detected */
226	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&	354	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
227	(REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|	355	(REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 \|\|
@@ -240,18 +368,9 @@ static void umc_v8_7_query_error_address(struct amdgpu_device *adev, · 12 unmodified lines
240		368
241	/* we only save ue error information currently, ce is skipped */	369	/* we only save ue error information currently, ce is skipped */
242	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)	370	if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)
243	== 1) {	371	== 1)
244	err_rec->address = err_addr;	372	amdgpu_umc_fill_error_record(err_data, err_addr,
245	/* page frame address is saved */	373	retired_page, channel_index, umc_inst);
246	err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;
247	err_rec->ts = (uint64_t)ktime_get_real_seconds();
248	err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
249	err_rec->cu = 0;
250	err_rec->mem_channel = channel_index;
251	err_rec->mcumc_id = umc_inst;
252
253	err_data->err_addr_cnt++;
254	}
255	}	374	}
256		375
257	/* clear umc status */	376	/* clear umc status */
@@ -268,7 +387,7 @@ static void umc_v8_7_query_ras_error_address(struct amdgpu_device *adev, · 10 unmodified lines
268	uint32_t umc_reg_offset = 0;	387	uint32_t umc_reg_offset = 0;
269		388
270	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {	389	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
271	umc_reg_offset = get_umc_8_reg_offset(adev,	390	umc_reg_offset = get_umc_v8_7_reg_offset(adev,
272	umc_inst,	391	umc_inst,
273	ch_inst);	392	ch_inst);
274		393
@@ -316,7 +435,7 @@ static void umc_v8_7_err_cnt_init(struct amdgpu_device *adev) · 41 unmodified lines
316	uint32_t umc_reg_offset = 0;	435	uint32_t umc_reg_offset = 0;
317		436
318	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {	437	LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
319	umc_reg_offset = get_umc_8_reg_offset(adev,	438	umc_reg_offset = get_umc_v8_7_reg_offset(adev,
320	umc_inst,	439	umc_inst,
321	ch_inst);	440	ch_inst);
322		441
@@ -324,10 +443,16 @@ static void umc_v8_7_err_cnt_init(struct amdgpu_device *adev) · 1 unmodified line
324	}	443	}
325	}	444	}
326		445
327	const struct amdgpu_umc_ras_funcs umc_v8_7_ras_funcs = {	446	const struct amdgpu_ras_block_hw_ops umc_v8_7_ras_hw_ops = {
328	.err_cnt_init = umc_v8_7_err_cnt_init,
329	.ras_late_init = amdgpu_umc_ras_late_init,
330	.ras_fini = amdgpu_umc_ras_fini,
331	.query_ras_error_count = umc_v8_7_query_ras_error_count,	447	.query_ras_error_count = umc_v8_7_query_ras_error_count,
332	.query_ras_error_address = umc_v8_7_query_ras_error_address,	448	.query_ras_error_address = umc_v8_7_query_ras_error_address,
333	};	449	};
		450
		451	struct amdgpu_umc_ras umc_v8_7_ras = {
		452	.ras_block = {
		453	.hw_ops = &umc_v8_7_ras_hw_ops,
		454	},
		455	.err_cnt_init = umc_v8_7_err_cnt_init,
		456	.ecc_info_query_ras_error_count = umc_v8_7_ecc_info_query_ras_error_count,
		457	.ecc_info_query_ras_error_address = umc_v8_7_ecc_info_query_ras_error_address,
		458	};

drivers/gpu/drm/amd/amdgpu/umc_v8_7.h+1−1

@@ -44,7 +44,7 @@ · 43 unmodified lines
44	/* umc ce count initial value */	44	/* umc ce count initial value */
45	#define UMC_V8_7_CE_CNT_INIT (UMC_V8_7_CE_CNT_MAX - UMC_V8_7_CE_INT_THRESHOLD)	45	#define UMC_V8_7_CE_CNT_INIT (UMC_V8_7_CE_CNT_MAX - UMC_V8_7_CE_INT_THRESHOLD)
46		46
47	extern const struct amdgpu_umc_ras_funcs umc_v8_7_ras_funcs;	47	extern struct amdgpu_umc_ras umc_v8_7_ras;
48	extern const uint32_t	48	extern const uint32_t
49	umc_v8_7_channel_idx_tbl[UMC_V8_7_UMC_INSTANCE_NUM][UMC_V8_7_CHANNEL_INSTANCE_NUM];	49	umc_v8_7_channel_idx_tbl[UMC_V8_7_UMC_INSTANCE_NUM][UMC_V8_7_CHANNEL_INSTANCE_NUM];
50		50

drivers/gpu/drm/amd/amdgpu/vi.c+1−1

@@ -1140,7 +1140,7 @@ static void vi_program_aspm(struct amdgpu_device *adev) · 1139 unmodified lines
1140	bool bL1SS = false;	1140	bool bL1SS = false;
1141	bool bClkReqSupport = true;	1141	bool bClkReqSupport = true;
1142		1142
1143	if (!amdgpu_aspm)	1143	if (!amdgpu_device_should_use_aspm(adev))
1144	return;	1144	return;
1145		1145
1146	if (adev->flags & AMD_IS_APU \|\|	1146	if (adev->flags & AMD_IS_APU \|\|

drivers/gpu/drm/amd/amdkfd/Makefile+0−2

@@ -51,8 +51,6 @@ AMDKFD_FILES := $(AMDKFD_PATH)/kfd_module.o \ · 50 unmodified lines
51	$(AMDKFD_PATH)/kfd_events.o \	51	$(AMDKFD_PATH)/kfd_events.o \
52	$(AMDKFD_PATH)/cik_event_interrupt.o \	52	$(AMDKFD_PATH)/cik_event_interrupt.o \
53	$(AMDKFD_PATH)/kfd_int_process_v9.o \	53	$(AMDKFD_PATH)/kfd_int_process_v9.o \
54	$(AMDKFD_PATH)/kfd_dbgdev.o \
55	$(AMDKFD_PATH)/kfd_dbgmgr.o \
56	$(AMDKFD_PATH)/kfd_smi_events.o \	54	$(AMDKFD_PATH)/kfd_smi_events.o \
57	$(AMDKFD_PATH)/kfd_crat.o	55	$(AMDKFD_PATH)/kfd_crat.o
58		56

drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c+1−1

@@ -110,7 +110,7 @@ static void cik_event_interrupt_wq(struct kfd_dev *dev, · 109 unmodified lines
110	struct kfd_vm_fault_info info;	110	struct kfd_vm_fault_info info;
111		111
112	kfd_smi_event_update_vmfault(dev, pasid);	112	kfd_smi_event_update_vmfault(dev, pasid);
113	kfd_process_vm_fault(dev->dqm, pasid);	113	kfd_dqm_evict_pasid(dev->dqm, pasid);
114		114
115	memset(&info, 0, sizeof(info));	115	memset(&info, 0, sizeof(info));
116	amdgpu_amdkfd_gpuvm_get_vm_fault_info(dev->adev, &info);	116	amdgpu_amdkfd_gpuvm_get_vm_fault_info(dev->adev, &info);

drivers/gpu/drm/amd/amdkfd/kfd_chardev.c+1182−428

@@ -1,5 +1,6 @@
		1	// SPDX-License-Identifier: GPL-2.0 OR MIT
1	/*	2	/*
2	* Copyright 2014 Advanced Micro Devices, Inc.	3	* Copyright 2014-2022 Advanced Micro Devices, Inc.
3	*	4	*
4	* Permission is hereby granted, free of charge, to any person obtaining a	5	* Permission is hereby granted, free of charge, to any person obtaining a
5	* copy of this software and associated documentation files (the "Software"),	6	* copy of this software and associated documentation files (the "Software"),
@@ -33,14 +34,16 @@ · 27 unmodified lines
33	#include <linux/time.h>	34	#include <linux/time.h>
34	#include <linux/mm.h>	35	#include <linux/mm.h>
35	#include <linux/mman.h>	36	#include <linux/mman.h>
		37	#include <linux/ptrace.h>
36	#include <linux/dma-buf.h>	38	#include <linux/dma-buf.h>
37	#include <asm/processor.h>	39	#include <linux/fdtable.h>
		40	#include <linux/processor.h>
38	#include "kfd_priv.h"	41	#include "kfd_priv.h"
39	#include "kfd_device_queue_manager.h"	42	#include "kfd_device_queue_manager.h"
40	#include "kfd_dbgmgr.h"
41	#include "kfd_svm.h"	43	#include "kfd_svm.h"
42	#include "amdgpu_amdkfd.h"	44	#include "amdgpu_amdkfd.h"
43	#include "kfd_smi_events.h"	45	#include "kfd_smi_events.h"
		46	#include "amdgpu_dma_buf.h"
44		47
45	static long kfd_ioctl(struct file *, unsigned int, unsigned long);	48	static long kfd_ioctl(struct file *, unsigned int, unsigned long);
46	static int kfd_open(struct inode , struct file );	49	static int kfd_open(struct inode , struct file );
@@ -292,14 +295,17 @@ static int kfd_ioctl_create_queue(struct file filep, struct kfd_process p, · 245 unmodified lines
292	return err;	295	return err;
293		296
294	pr_debug("Looking for gpu id 0x%x\n", args->gpu_id);	297	pr_debug("Looking for gpu id 0x%x\n", args->gpu_id);
295	dev = kfd_device_by_id(args->gpu_id);
296	if (!dev) {
297	pr_debug("Could not find gpu id 0x%x\n", args->gpu_id);
298	return -EINVAL;
299	}
300		298
301	mutex_lock(&p->mutex);	299	mutex_lock(&p->mutex);
302		300
		301	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
		302	if (!pdd) {
		303	pr_debug("Could not find gpu id 0x%x\n", args->gpu_id);
		304	err = -EINVAL;
		305	goto err_pdd;
		306	}
		307	dev = pdd->dev;
		308
303	pdd = kfd_bind_process_to_device(dev, p);	309	pdd = kfd_bind_process_to_device(dev, p);
304	if (IS_ERR(pdd)) {	310	if (IS_ERR(pdd)) {
305	err = -ESRCH;	311	err = -ESRCH;
@@ -310,7 +316,7 @@ static int kfd_ioctl_create_queue(struct file filep, struct kfd_process p, · 4 unmodified lines
310	p->pasid,	316	p->pasid,
311	dev->id);	317	dev->id);
312		318
313	err = pqm_create_queue(&p->pqm, dev, filep, &q_properties, &queue_id,	319	err = pqm_create_queue(&p->pqm, dev, filep, &q_properties, &queue_id, NULL, NULL, NULL,
314	&doorbell_offset_in_process);	320	&doorbell_offset_in_process);
315	if (err != 0)	321	if (err != 0)
316	goto err_create_queue;	322	goto err_create_queue;
@@ -344,6 +350,7 @@ static int kfd_ioctl_create_queue(struct file filep, struct kfd_process p, · 27 unmodified lines
344		350
345	err_create_queue:	351	err_create_queue:
346	err_bind_process:	352	err_bind_process:
		353	err_pdd:
347	mutex_unlock(&p->mutex);	354	mutex_unlock(&p->mutex);
348	return err;	355	return err;
349	}	356	}
@@ -490,7 +497,6 @@ static int kfd_ioctl_set_memory_policy(struct file *filep, · 140 unmodified lines
490	struct kfd_process p, void data)	497	struct kfd_process p, void data)
491	{	498	{
492	struct kfd_ioctl_set_memory_policy_args *args = data;	499	struct kfd_ioctl_set_memory_policy_args *args = data;
493	struct kfd_dev *dev;
494	int err = 0;	500	int err = 0;
495	struct kfd_process_device *pdd;	501	struct kfd_process_device *pdd;
496	enum cache_policy default_policy, alternate_policy;	502	enum cache_policy default_policy, alternate_policy;
@@ -505,13 +511,15 @@ static int kfd_ioctl_set_memory_policy(struct file *filep, · 8 unmodified lines
505	return -EINVAL;	511	return -EINVAL;
506	}	512	}
507		513
508	dev = kfd_device_by_id(args->gpu_id);
509	if (!dev)
510	return -EINVAL;
511
512	mutex_lock(&p->mutex);	514	mutex_lock(&p->mutex);
		515	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
		516	if (!pdd) {
		517	pr_debug("Could not find gpu id 0x%x\n", args->gpu_id);
		518	err = -EINVAL;
		519	goto err_pdd;
		520	}
513		521
514	pdd = kfd_bind_process_to_device(dev, p);	522	pdd = kfd_bind_process_to_device(pdd->dev, p);
515	if (IS_ERR(pdd)) {	523	if (IS_ERR(pdd)) {
516	err = -ESRCH;	524	err = -ESRCH;
517	goto out;	525	goto out;
@@ -524,7 +532,7 @@ static int kfd_ioctl_set_memory_policy(struct file *filep, · 6 unmodified lines
524	(args->alternate_policy == KFD_IOC_CACHE_POLICY_COHERENT)	532	(args->alternate_policy == KFD_IOC_CACHE_POLICY_COHERENT)
525	? cache_policy_coherent : cache_policy_noncoherent;	533	? cache_policy_coherent : cache_policy_noncoherent;
526		534
527	if (!dev->dqm->ops.set_cache_memory_policy(dev->dqm,	535	if (!pdd->dev->dqm->ops.set_cache_memory_policy(pdd->dev->dqm,
528	&pdd->qpd,	536	&pdd->qpd,
529	default_policy,	537	default_policy,
530	alternate_policy,	538	alternate_policy,
@@ -533,6 +541,7 @@ static int kfd_ioctl_set_memory_policy(struct file *filep, · 2 unmodified lines
533	err = -EINVAL;	541	err = -EINVAL;
534		542
535	out:	543	out:
		544	err_pdd:
536	mutex_unlock(&p->mutex);	545	mutex_unlock(&p->mutex);
537		546
538	return err;	547	return err;
@@ -542,17 +551,18 @@ static int kfd_ioctl_set_trap_handler(struct file *filep, · 3 unmodified lines
542	struct kfd_process p, void data)	551	struct kfd_process p, void data)
543	{	552	{
544	struct kfd_ioctl_set_trap_handler_args *args = data;	553	struct kfd_ioctl_set_trap_handler_args *args = data;
545	struct kfd_dev *dev;
546	int err = 0;	554	int err = 0;
547	struct kfd_process_device *pdd;	555	struct kfd_process_device *pdd;
548		556
549	dev = kfd_device_by_id(args->gpu_id);
550	if (!dev)
551	return -EINVAL;
552
553	mutex_lock(&p->mutex);	557	mutex_lock(&p->mutex);
554		558
555	pdd = kfd_bind_process_to_device(dev, p);	559	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
		560	if (!pdd) {
		561	err = -EINVAL;
		562	goto err_pdd;
		563	}
		564
		565	pdd = kfd_bind_process_to_device(pdd->dev, p);
556	if (IS_ERR(pdd)) {	566	if (IS_ERR(pdd)) {
557	err = -ESRCH;	567	err = -ESRCH;
558	goto out;	568	goto out;
@@ -561,6 +571,7 @@ static int kfd_ioctl_set_trap_handler(struct file *filep, · 2 unmodified lines
561	kfd_process_set_trap_handler(&pdd->qpd, args->tba_addr, args->tma_addr);	571	kfd_process_set_trap_handler(&pdd->qpd, args->tba_addr, args->tma_addr);
562		572
563	out:	573	out:
		574	err_pdd:
564	mutex_unlock(&p->mutex);	575	mutex_unlock(&p->mutex);
565		576
566	return err;	577	return err;
@@ -569,289 +580,40 @@ out: · 2 unmodified lines
569	static int kfd_ioctl_dbg_register(struct file *filep,	580	static int kfd_ioctl_dbg_register(struct file *filep,
570	struct kfd_process p, void data)	581	struct kfd_process p, void data)
571	{	582	{
572	struct kfd_ioctl_dbg_register_args *args = data;	583	return -EPERM;
573	struct kfd_dev *dev;
574	struct kfd_dbgmgr *dbgmgr_ptr;
575	struct kfd_process_device *pdd;
576	bool create_ok;
577	long status = 0;
578
579	dev = kfd_device_by_id(args->gpu_id);
580	if (!dev)
581	return -EINVAL;
582
583	if (dev->adev->asic_type == CHIP_CARRIZO) {
584	pr_debug("kfd_ioctl_dbg_register not supported on CZ\n");
585	return -EINVAL;
586	}
587
588	mutex_lock(&p->mutex);
589	mutex_lock(kfd_get_dbgmgr_mutex());
590
591	/*
592	* make sure that we have pdd, if this the first queue created for
593	* this process
594	*/
595	pdd = kfd_bind_process_to_device(dev, p);
596	if (IS_ERR(pdd)) {
597	status = PTR_ERR(pdd);
598	goto out;
599	}
600
601	if (!dev->dbgmgr) {
602	/* In case of a legal call, we have no dbgmgr yet */
603	create_ok = kfd_dbgmgr_create(&dbgmgr_ptr, dev);
604	if (create_ok) {
605	status = kfd_dbgmgr_register(dbgmgr_ptr, p);
606	if (status != 0)
607	kfd_dbgmgr_destroy(dbgmgr_ptr);
608	else
609	dev->dbgmgr = dbgmgr_ptr;
610	}
611	} else {
612	pr_debug("debugger already registered\n");
613	status = -EINVAL;
614	}
615
616	out:
617	mutex_unlock(kfd_get_dbgmgr_mutex());
618	mutex_unlock(&p->mutex);
619
620	return status;
621	}	584	}
622		585
623	static int kfd_ioctl_dbg_unregister(struct file *filep,	586	static int kfd_ioctl_dbg_unregister(struct file *filep,
624	struct kfd_process p, void data)	587	struct kfd_process p, void data)
625	{	588	{
626	struct kfd_ioctl_dbg_unregister_args *args = data;	589	return -EPERM;
627	struct kfd_dev *dev;
628	long status;
629
630	dev = kfd_device_by_id(args->gpu_id);
631	if (!dev \|\| !dev->dbgmgr)
632	return -EINVAL;
633
634	if (dev->adev->asic_type == CHIP_CARRIZO) {
635	pr_debug("kfd_ioctl_dbg_unregister not supported on CZ\n");
636	return -EINVAL;
637	}
638
639	mutex_lock(kfd_get_dbgmgr_mutex());
640
641	status = kfd_dbgmgr_unregister(dev->dbgmgr, p);
642	if (!status) {
643	kfd_dbgmgr_destroy(dev->dbgmgr);
644	dev->dbgmgr = NULL;
645	}
646
647	mutex_unlock(kfd_get_dbgmgr_mutex());
648
649	return status;
650	}	590	}
651		591
652	/*
653	* Parse and generate variable size data structure for address watch.
654	* Total size of the buffer and # watch points is limited in order
655	* to prevent kernel abuse. (no bearing to the much smaller HW limitation
656	* which is enforced by dbgdev module)
657	* please also note that the watch address itself are not "copied from user",
658	* since it be set into the HW in user mode values.
659	*
660	*/
661	static int kfd_ioctl_dbg_address_watch(struct file *filep,	592	static int kfd_ioctl_dbg_address_watch(struct file *filep,
662	struct kfd_process p, void data)	593	struct kfd_process p, void data)
663	{	594	{
664	struct kfd_ioctl_dbg_address_watch_args *args = data;	595	return -EPERM;
665	struct kfd_dev *dev;
666	struct dbg_address_watch_info aw_info;
667	unsigned char *args_buff;
668	long status;
669	void __user *cmd_from_user;
670	uint64_t watch_mask_value = 0;
671	unsigned int args_idx = 0;
672
673	memset((void *) &aw_info, 0, sizeof(struct dbg_address_watch_info));
674
675	dev = kfd_device_by_id(args->gpu_id);
676	if (!dev)
677	return -EINVAL;
678
679	if (dev->adev->asic_type == CHIP_CARRIZO) {
680	pr_debug("kfd_ioctl_dbg_wave_control not supported on CZ\n");
681	return -EINVAL;
682	}
683
684	cmd_from_user = (void __user *) args->content_ptr;
685
686	/* Validate arguments */
687
688	if ((args->buf_size_in_bytes > MAX_ALLOWED_AW_BUFF_SIZE) \|\|
689	(args->buf_size_in_bytes <= sizeof(args) + sizeof(int) 2) \|\|
690	(cmd_from_user == NULL))
691	return -EINVAL;
692
693	/* this is the actual buffer to work with */
694	args_buff = memdup_user(cmd_from_user,
695	args->buf_size_in_bytes - sizeof(*args));
696	if (IS_ERR(args_buff))
697	return PTR_ERR(args_buff);
698
699	aw_info.process = p;
700
701	aw_info.num_watch_points = ((uint32_t )(&args_buff[args_idx]));
702	args_idx += sizeof(aw_info.num_watch_points);
703
704	aw_info.watch_mode = (enum HSA_DBG_WATCH_MODE *) &args_buff[args_idx];
705	args_idx += sizeof(enum HSA_DBG_WATCH_MODE) * aw_info.num_watch_points;
706
707	/*
708	* set watch address base pointer to point on the array base
709	* within args_buff
710	*/
711	aw_info.watch_address = (uint64_t *) &args_buff[args_idx];
712
713	/* skip over the addresses buffer */
714	args_idx += sizeof(aw_info.watch_address) * aw_info.num_watch_points;
715
716	if (args_idx >= args->buf_size_in_bytes - sizeof(*args)) {
717	status = -EINVAL;
718	goto out;
719	}
720
721	watch_mask_value = (uint64_t) args_buff[args_idx];
722
723	if (watch_mask_value > 0) {
724	/*
725	* There is an array of masks.
726	* set watch mask base pointer to point on the array base
727	* within args_buff
728	*/
729	aw_info.watch_mask = (uint64_t *) &args_buff[args_idx];
730
731	/* skip over the masks buffer */
732	args_idx += sizeof(aw_info.watch_mask) *
733	aw_info.num_watch_points;
734	} else {
735	/* just the NULL mask, set to NULL and skip over it */
736	aw_info.watch_mask = NULL;
737	args_idx += sizeof(aw_info.watch_mask);
738	}
739
740	if (args_idx >= args->buf_size_in_bytes - sizeof(args)) {
741	status = -EINVAL;
742	goto out;
743	}
744
745	/* Currently HSA Event is not supported for DBG */
746	aw_info.watch_event = NULL;
747
748	mutex_lock(kfd_get_dbgmgr_mutex());
749
750	status = kfd_dbgmgr_address_watch(dev->dbgmgr, &aw_info);
751
752	mutex_unlock(kfd_get_dbgmgr_mutex());
753
754	out:
755	kfree(args_buff);
756
757	return status;
758	}	596	}
759		597
760	/* Parse and generate fixed size data structure for wave control */	598	/* Parse and generate fixed size data structure for wave control */
761	static int kfd_ioctl_dbg_wave_control(struct file *filep,	599	static int kfd_ioctl_dbg_wave_control(struct file *filep,
762	struct kfd_process p, void data)	600	struct kfd_process p, void data)
763	{	601	{
764	struct kfd_ioctl_dbg_wave_control_args *args = data;	602	return -EPERM;
765	struct kfd_dev *dev;
766	struct dbg_wave_control_info wac_info;
767	unsigned char *args_buff;
768	uint32_t computed_buff_size;
769	long status;
770	void __user *cmd_from_user;
771	unsigned int args_idx = 0;
772
773	memset((void *) &wac_info, 0, sizeof(struct dbg_wave_control_info));
774
775	/* we use compact form, independent of the packing attribute value */
776	computed_buff_size = sizeof(*args) +
777	sizeof(wac_info.mode) +
778	sizeof(wac_info.operand) +
779	sizeof(wac_info.dbgWave_msg.DbgWaveMsg) +
780	sizeof(wac_info.dbgWave_msg.MemoryVA) +
781	sizeof(wac_info.trapId);
782
783	dev = kfd_device_by_id(args->gpu_id);
784	if (!dev)
785	return -EINVAL;
786
787	if (dev->adev->asic_type == CHIP_CARRIZO) {
788	pr_debug("kfd_ioctl_dbg_wave_control not supported on CZ\n");
789	return -EINVAL;
790	}
791
792	/* input size must match the computed "compact" size */
793	if (args->buf_size_in_bytes != computed_buff_size) {
794	pr_debug("size mismatch, computed : actual %u : %u\n",
795	args->buf_size_in_bytes, computed_buff_size);
796	return -EINVAL;
797	}
798
799	cmd_from_user = (void __user *) args->content_ptr;
800
801	if (cmd_from_user == NULL)
802	return -EINVAL;
803
804	/* copy the entire buffer from user */
805
806	args_buff = memdup_user(cmd_from_user,
807	args->buf_size_in_bytes - sizeof(*args));
808	if (IS_ERR(args_buff))
809	return PTR_ERR(args_buff);
810
811	/* move ptr to the start of the "pay-load" area */
812	wac_info.process = p;
813
814	wac_info.operand = ((enum HSA_DBG_WAVEOP )(&args_buff[args_idx]));
815	args_idx += sizeof(wac_info.operand);
816
817	wac_info.mode = ((enum HSA_DBG_WAVEMODE )(&args_buff[args_idx]));
818	args_idx += sizeof(wac_info.mode);
819
820	wac_info.trapId = ((uint32_t )(&args_buff[args_idx]));
821	args_idx += sizeof(wac_info.trapId);
822
823	wac_info.dbgWave_msg.DbgWaveMsg.WaveMsgInfoGen2.Value =
824	((uint32_t )(&args_buff[args_idx]));
825	wac_info.dbgWave_msg.MemoryVA = NULL;
826
827	mutex_lock(kfd_get_dbgmgr_mutex());
828
829	pr_debug("Calling dbg manager process %p, operand %u, mode %u, trapId %u, message %u\n",
830	wac_info.process, wac_info.operand,
831	wac_info.mode, wac_info.trapId,
832	wac_info.dbgWave_msg.DbgWaveMsg.WaveMsgInfoGen2.Value);
833
834	status = kfd_dbgmgr_wave_control(dev->dbgmgr, &wac_info);
835
836	pr_debug("Returned status of dbg manager is %ld\n", status);
837
838	mutex_unlock(kfd_get_dbgmgr_mutex());
839
840	kfree(args_buff);
841
842	return status;
843	}	603	}
844		604
845	static int kfd_ioctl_get_clock_counters(struct file *filep,	605	static int kfd_ioctl_get_clock_counters(struct file *filep,
846	struct kfd_process p, void data)	606	struct kfd_process p, void data)
847	{	607	{
848	struct kfd_ioctl_get_clock_counters_args *args = data;	608	struct kfd_ioctl_get_clock_counters_args *args = data;
849	struct kfd_dev *dev;	609	struct kfd_process_device *pdd;
850		610
851	dev = kfd_device_by_id(args->gpu_id);	611	mutex_lock(&p->mutex);
852	if (dev)	612	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
		613	mutex_unlock(&p->mutex);
		614	if (pdd)
853	/* Reading GPU clock counter from KGD */	615	/* Reading GPU clock counter from KGD */
854	args->gpu_clock_counter = amdgpu_amdkfd_get_gpu_clock_counter(dev->adev);	616	args->gpu_clock_counter = amdgpu_amdkfd_get_gpu_clock_counter(pdd->dev->adev);
855	else	617	else
856	/* Node without GPU resource */	618	/* Node without GPU resource */
857	args->gpu_clock_counter = 0;	619	args->gpu_clock_counter = 0;
@@ -1007,57 +769,11 @@ static int kfd_ioctl_create_event(struct file filp, struct kfd_process p, · 149 unmodified lines
1007	* through the event_page_offset field.	769	* through the event_page_offset field.
1008	*/	770	*/
1009	if (args->event_page_offset) {	771	if (args->event_page_offset) {
1010	struct kfd_dev *kfd;
1011	struct kfd_process_device *pdd;
1012	void mem, kern_addr;
1013	uint64_t size;
1014
1015	kfd = kfd_device_by_id(GET_GPU_ID(args->event_page_offset));
1016	if (!kfd) {
1017	pr_err("Getting device by id failed in %s\n", __func__);
1018	return -EINVAL;
1019	}
1020
1021	mutex_lock(&p->mutex);	772	mutex_lock(&p->mutex);
1022		773	err = kfd_kmap_event_page(p, args->event_page_offset);
1023	if (p->signal_page) {
1024	pr_err("Event page is already set\n");
1025	err = -EINVAL;
1026	goto out_unlock;
1027	}
1028
1029	pdd = kfd_bind_process_to_device(kfd, p);
1030	if (IS_ERR(pdd)) {
1031	err = PTR_ERR(pdd);
1032	goto out_unlock;
1033	}
1034
1035	mem = kfd_process_device_translate_handle(pdd,
1036	GET_IDR_HANDLE(args->event_page_offset));
1037	if (!mem) {
1038	pr_err("Can't find BO, offset is 0x%llx\n",
1039	args->event_page_offset);
1040	err = -EINVAL;
1041	goto out_unlock;
1042	}
1043
1044	err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kfd->adev,
1045	mem, &kern_addr, &size);
1046	if (err) {
1047	pr_err("Failed to map event page to kernel\n");
1048	goto out_unlock;
1049	}
1050
1051	err = kfd_event_page_set(p, kern_addr, size);
1052	if (err) {
1053	pr_err("Failed to set event page\n");
1054	amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kfd->adev, mem);
1055	goto out_unlock;
1056	}
1057
1058	p->signal_handle = args->event_page_offset;
1059
1060	mutex_unlock(&p->mutex);	774	mutex_unlock(&p->mutex);
		775	if (err)
		776	return err;
1061	}	777	}
1062		778
1063	err = kfd_event_create(filp, p, args->event_type,	779	err = kfd_event_create(filp, p, args->event_type,
@@ -1066,10 +782,7 @@ static int kfd_ioctl_create_event(struct file filp, struct kfd_process p, · 2 unmodified lines
1066	&args->event_page_offset,	782	&args->event_page_offset,
1067	&args->event_slot_index);	783	&args->event_slot_index);
1068		784
1069	return err;	785	pr_debug("Created event (id:0x%08x) (%s)\n", args->event_id, __func__);
1070
1071	out_unlock:
1072	mutex_unlock(&p->mutex);
1073	return err;	786	return err;
1074	}	787	}
1075		788
@@ -1118,11 +831,13 @@ static int kfd_ioctl_set_scratch_backing_va(struct file *filep, · 42 unmodified lines
1118	struct kfd_dev *dev;	831	struct kfd_dev *dev;
1119	long err;	832	long err;
1120		833
1121	dev = kfd_device_by_id(args->gpu_id);
1122	if (!dev)
1123	return -EINVAL;
1124
1125	mutex_lock(&p->mutex);	834	mutex_lock(&p->mutex);
		835	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
		836	if (!pdd) {
		837	err = -EINVAL;
		838	goto err_pdd;
		839	}
		840	dev = pdd->dev;
1126		841
1127	pdd = kfd_bind_process_to_device(dev, p);	842	pdd = kfd_bind_process_to_device(dev, p);
1128	if (IS_ERR(pdd)) {	843	if (IS_ERR(pdd)) {
@@ -1142,6 +857,7 @@ static int kfd_ioctl_set_scratch_backing_va(struct file *filep, · 13 unmodified lines
1142	return 0;	857	return 0;
1143		858
1144	bind_process_to_device_fail:	859	bind_process_to_device_fail:
		860	err_pdd:
1145	mutex_unlock(&p->mutex);	861	mutex_unlock(&p->mutex);
1146	return err;	862	return err;
1147	}	863	}
@@ -1150,15 +866,17 @@ static int kfd_ioctl_get_tile_config(struct file *filep, · 2 unmodified lines
1150	struct kfd_process p, void data)	866	struct kfd_process p, void data)
1151	{	867	{
1152	struct kfd_ioctl_get_tile_config_args *args = data;	868	struct kfd_ioctl_get_tile_config_args *args = data;
1153	struct kfd_dev *dev;	869	struct kfd_process_device *pdd;
1154	struct tile_config config;	870	struct tile_config config;
1155	int err = 0;	871	int err = 0;
1156		872
1157	dev = kfd_device_by_id(args->gpu_id);	873	mutex_lock(&p->mutex);
1158	if (!dev)	874	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
		875	mutex_unlock(&p->mutex);
		876	if (!pdd)
1159	return -EINVAL;	877	return -EINVAL;
1160		878
1161	amdgpu_amdkfd_get_tile_config(dev->adev, &config);	879	amdgpu_amdkfd_get_tile_config(pdd->dev->adev, &config);
1162		880
1163	args->gb_addr_config = config.gb_addr_config;	881	args->gb_addr_config = config.gb_addr_config;
1164	args->num_banks = config.num_banks;	882	args->num_banks = config.num_banks;
@@ -1193,40 +911,37 @@ static int kfd_ioctl_acquire_vm(struct file filep, struct kfd_process p, · 28 unmodified lines
1193	{	911	{
1194	struct kfd_ioctl_acquire_vm_args *args = data;	912	struct kfd_ioctl_acquire_vm_args *args = data;
1195	struct kfd_process_device *pdd;	913	struct kfd_process_device *pdd;
1196	struct kfd_dev *dev;
1197	struct file *drm_file;	914	struct file *drm_file;
1198	int ret;	915	int ret;
1199		916
1200	dev = kfd_device_by_id(args->gpu_id);
1201	if (!dev)
1202	return -EINVAL;
1203
1204	drm_file = fget(args->drm_fd);	917	drm_file = fget(args->drm_fd);
1205	if (!drm_file)	918	if (!drm_file)
1206	return -EINVAL;	919	return -EINVAL;
1207		920
1208	mutex_lock(&p->mutex);	921	mutex_lock(&p->mutex);
1209		922	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
1210	pdd = kfd_get_process_device_data(dev, p);
1211	if (!pdd) {	923	if (!pdd) {
1212	ret = -EINVAL;	924	ret = -EINVAL;
1213	goto err_unlock;	925	goto err_pdd;
1214	}	926	}
1215		927
1216	if (pdd->drm_file) {	928	if (pdd->drm_file) {
1217	ret = pdd->drm_file == drm_file ? 0 : -EBUSY;	929	ret = pdd->drm_file == drm_file ? 0 : -EBUSY;
1218	goto err_unlock;	930	goto err_drm_file;
1219	}	931	}
1220		932
1221	ret = kfd_process_device_init_vm(pdd, drm_file);	933	ret = kfd_process_device_init_vm(pdd, drm_file);
1222	if (ret)	934	if (ret)
1223	goto err_unlock;	935	goto err_unlock;
		936
1224	/* On success, the PDD keeps the drm_file reference */	937	/* On success, the PDD keeps the drm_file reference */
1225	mutex_unlock(&p->mutex);	938	mutex_unlock(&p->mutex);
1226		939
1227	return 0;	940	return 0;
1228		941
1229	err_unlock:	942	err_unlock:
		943	err_pdd:
		944	err_drm_file:
1230	mutex_unlock(&p->mutex);	945	mutex_unlock(&p->mutex);
1231	fput(drm_file);	946	fput(drm_file);
1232	return ret;	947	return ret;
@@ -1283,19 +998,23 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep, · 50 unmodified lines
1283	}	998	}
1284	mutex_unlock(&p->svms.lock);	999	mutex_unlock(&p->svms.lock);
1285	#endif	1000	#endif
1286	dev = kfd_device_by_id(args->gpu_id);	1001	mutex_lock(&p->mutex);
1287	if (!dev)	1002	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
1288	return -EINVAL;	1003	if (!pdd) {
		1004	err = -EINVAL;
		1005	goto err_pdd;
		1006	}
		1007
		1008	dev = pdd->dev;
1289		1009
1290	if ((flags & KFD_IOC_ALLOC_MEM_FLAGS_PUBLIC) &&	1010	if ((flags & KFD_IOC_ALLOC_MEM_FLAGS_PUBLIC) &&
1291	(flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) &&	1011	(flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) &&
1292	!kfd_dev_is_large_bar(dev)) {	1012	!kfd_dev_is_large_bar(dev)) {
1293	pr_err("Alloc host visible vram on small bar is not allowed\n");	1013	pr_err("Alloc host visible vram on small bar is not allowed\n");
1294	return -EINVAL;	1014	err = -EINVAL;
		1015	goto err_large_bar;
1295	}	1016	}
1296		1017
1297	mutex_lock(&p->mutex);
1298
1299	pdd = kfd_bind_process_to_device(dev, p);	1018	pdd = kfd_bind_process_to_device(dev, p);
1300	if (IS_ERR(pdd)) {	1019	if (IS_ERR(pdd)) {
1301	err = PTR_ERR(pdd);	1020	err = PTR_ERR(pdd);
@@ -1323,7 +1042,7 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep, · 21 unmodified lines
1323	err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(	1042	err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
1324	dev->adev, args->va_addr, args->size,	1043	dev->adev, args->va_addr, args->size,
1325	pdd->drm_priv, (struct kgd_mem **) &mem, &offset,	1044	pdd->drm_priv, (struct kgd_mem **) &mem, &offset,
1326	flags);	1045	flags, false);
1327		1046
1328	if (err)	1047	if (err)
1329	goto err_unlock;	1048	goto err_unlock;
@@ -1356,6 +1075,8 @@ err_free: · 26 unmodified lines
1356	amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, (struct kgd_mem *)mem,	1075	amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, (struct kgd_mem *)mem,
1357	pdd->drm_priv, NULL);	1076	pdd->drm_priv, NULL);
1358	err_unlock:	1077	err_unlock:
		1078	err_pdd:
		1079	err_large_bar:
1359	mutex_unlock(&p->mutex);	1080	mutex_unlock(&p->mutex);
1360	return err;	1081	return err;
1361	}	1082	}
@@ -1366,14 +1087,9 @@ static int kfd_ioctl_free_memory_of_gpu(struct file *filep, · 4 unmodified lines
1366	struct kfd_ioctl_free_memory_of_gpu_args *args = data;	1087	struct kfd_ioctl_free_memory_of_gpu_args *args = data;
1367	struct kfd_process_device *pdd;	1088	struct kfd_process_device *pdd;
1368	void *mem;	1089	void *mem;
1369	struct kfd_dev *dev;
1370	int ret;	1090	int ret;
1371	uint64_t size = 0;	1091	uint64_t size = 0;
1372		1092
1373	dev = kfd_device_by_id(GET_GPU_ID(args->handle));
1374	if (!dev)
1375	return -EINVAL;
1376
1377	mutex_lock(&p->mutex);	1093	mutex_lock(&p->mutex);
1378	/*	1094	/*
1379	* Safeguard to prevent user space from freeing signal BO.	1095	* Safeguard to prevent user space from freeing signal BO.
@@ -1385,11 +1101,11 @@ static int kfd_ioctl_free_memory_of_gpu(struct file *filep, · 5 unmodified lines
1385	goto err_unlock;	1101	goto err_unlock;
1386	}	1102	}
1387		1103
1388	pdd = kfd_get_process_device_data(dev, p);	1104	pdd = kfd_process_device_data_by_id(p, GET_GPU_ID(args->handle));
1389	if (!pdd) {	1105	if (!pdd) {
1390	pr_err("Process device data doesn't exist\n");	1106	pr_err("Process device data doesn't exist\n");
1391	ret = -EINVAL;	1107	ret = -EINVAL;
1392	goto err_unlock;	1108	goto err_pdd;
1393	}	1109	}
1394		1110
1395	mem = kfd_process_device_translate_handle(	1111	mem = kfd_process_device_translate_handle(
@@ -1399,7 +1115,7 @@ static int kfd_ioctl_free_memory_of_gpu(struct file *filep, · 3 unmodified lines
1399	goto err_unlock;	1115	goto err_unlock;
1400	}	1116	}
1401		1117
1402	ret = amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev,	1118	ret = amdgpu_amdkfd_gpuvm_free_memory_of_gpu(pdd->dev->adev,
1403	(struct kgd_mem *)mem, pdd->drm_priv, &size);	1119	(struct kgd_mem *)mem, pdd->drm_priv, &size);
1404		1120
1405	/* If freeing the buffer failed, leave the handle in place for	1121	/* If freeing the buffer failed, leave the handle in place for
@@ -1412,26 +1128,31 @@ static int kfd_ioctl_free_memory_of_gpu(struct file *filep, · 6 unmodified lines
1412	WRITE_ONCE(pdd->vram_usage, pdd->vram_usage - size);	1128	WRITE_ONCE(pdd->vram_usage, pdd->vram_usage - size);
1413		1129
1414	err_unlock:	1130	err_unlock:
		1131	err_pdd:
1415	mutex_unlock(&p->mutex);	1132	mutex_unlock(&p->mutex);
1416	return ret;	1133	return ret;
1417	}	1134	}
1418		1135
		1136	static bool kfd_flush_tlb_after_unmap(struct kfd_dev *dev)
		1137	{
		1138	return KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2) \|\|
		1139	(KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 1) &&
		1140	dev->adev->sdma.instance[0].fw_version >= 18) \|\|
		1141	KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 0);
		1142	}
		1143
1419	static int kfd_ioctl_map_memory_to_gpu(struct file *filep,	1144	static int kfd_ioctl_map_memory_to_gpu(struct file *filep,
1420	struct kfd_process p, void data)	1145	struct kfd_process p, void data)
1421	{	1146	{
1422	struct kfd_ioctl_map_memory_to_gpu_args *args = data;	1147	struct kfd_ioctl_map_memory_to_gpu_args *args = data;
1423	struct kfd_process_device pdd, peer_pdd;	1148	struct kfd_process_device pdd, peer_pdd;
1424	void *mem;	1149	void *mem;
1425	struct kfd_dev dev, peer;	1150	struct kfd_dev *dev;
1426	long err = 0;	1151	long err = 0;
1427	int i;	1152	int i;
1428	uint32_t *devices_arr = NULL;	1153	uint32_t *devices_arr = NULL;
1429	bool table_freed = false;	1154	bool table_freed = false;
1430		1155
1431	dev = kfd_device_by_id(GET_GPU_ID(args->handle));
1432	if (!dev)
1433	return -EINVAL;
1434
1435	if (!args->n_devices) {	1156	if (!args->n_devices) {
1436	pr_debug("Device IDs array empty\n");	1157	pr_debug("Device IDs array empty\n");
1437	return -EINVAL;	1158	return -EINVAL;
@@ -1455,6 +1176,12 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep, · 17 unmodified lines
1455	}	1176	}
1456		1177
1457	mutex_lock(&p->mutex);	1178	mutex_lock(&p->mutex);
		1179	pdd = kfd_process_device_data_by_id(p, GET_GPU_ID(args->handle));
		1180	if (!pdd) {
		1181	err = -EINVAL;
		1182	goto get_process_device_data_failed;
		1183	}
		1184	dev = pdd->dev;
1458		1185
1459	pdd = kfd_bind_process_to_device(dev, p);	1186	pdd = kfd_bind_process_to_device(dev, p);
1460	if (IS_ERR(pdd)) {	1187	if (IS_ERR(pdd)) {
@@ -1470,21 +1197,22 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep, · 9 unmodified lines
1470	}	1197	}
1471		1198
1472	for (i = args->n_success; i < args->n_devices; i++) {	1199	for (i = args->n_success; i < args->n_devices; i++) {
1473	peer = kfd_device_by_id(devices_arr[i]);	1200	peer_pdd = kfd_process_device_data_by_id(p, devices_arr[i]);
1474	if (!peer) {	1201	if (!peer_pdd) {
1475	pr_debug("Getting device by id failed for 0x%x\n",	1202	pr_debug("Getting device by id failed for 0x%x\n",
1476	devices_arr[i]);	1203	devices_arr[i]);
1477	err = -EINVAL;	1204	err = -EINVAL;
1478	goto get_mem_obj_from_handle_failed;	1205	goto get_mem_obj_from_handle_failed;
1479	}	1206	}
1480		1207
1481	peer_pdd = kfd_bind_process_to_device(peer, p);	1208	peer_pdd = kfd_bind_process_to_device(peer_pdd->dev, p);
1482	if (IS_ERR(peer_pdd)) {	1209	if (IS_ERR(peer_pdd)) {
1483	err = PTR_ERR(peer_pdd);	1210	err = PTR_ERR(peer_pdd);
1484	goto get_mem_obj_from_handle_failed;	1211	goto get_mem_obj_from_handle_failed;
1485	}	1212	}
		1213
1486	err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu(	1214	err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu(
1487	peer->adev, (struct kgd_mem *)mem,	1215	peer_pdd->dev->adev, (struct kgd_mem *)mem,
1488	peer_pdd->drm_priv, &table_freed);	1216	peer_pdd->drm_priv, &table_freed);
1489	if (err) {	1217	if (err) {
1490	pr_err("Failed to map to gpu %d/%d\n",	1218	pr_err("Failed to map to gpu %d/%d\n",
@@ -1503,12 +1231,9 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep, · 12 unmodified lines
1503	}	1231	}
1504		1232
1505	/* Flush TLBs after waiting for the page table updates to complete */	1233	/* Flush TLBs after waiting for the page table updates to complete */
1506	if (table_freed) {	1234	if (table_freed \|\| !kfd_flush_tlb_after_unmap(dev)) {
1507	for (i = 0; i < args->n_devices; i++) {	1235	for (i = 0; i < args->n_devices; i++) {
1508	peer = kfd_device_by_id(devices_arr[i]);	1236	peer_pdd = kfd_process_device_data_by_id(p, devices_arr[i]);
1509	if (WARN_ON_ONCE(!peer))
1510	continue;
1511	peer_pdd = kfd_get_process_device_data(peer, p);
1512	if (WARN_ON_ONCE(!peer_pdd))	1237	if (WARN_ON_ONCE(!peer_pdd))
1513	continue;	1238	continue;
1514	kfd_flush_tlb(peer_pdd, TLB_FLUSH_LEGACY);	1239	kfd_flush_tlb(peer_pdd, TLB_FLUSH_LEGACY);
@@ -1518,6 +1243,7 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep, · 3 unmodified lines
1518		1243
1519	return err;	1244	return err;
1520		1245
		1246	get_process_device_data_failed:
1521	bind_process_to_device_failed:	1247	bind_process_to_device_failed:
1522	get_mem_obj_from_handle_failed:	1248	get_mem_obj_from_handle_failed:
1523	map_memory_to_gpu_failed:	1249	map_memory_to_gpu_failed:
@@ -1535,14 +1261,9 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 11 unmodified lines
1535	struct kfd_ioctl_unmap_memory_from_gpu_args *args = data;	1261	struct kfd_ioctl_unmap_memory_from_gpu_args *args = data;
1536	struct kfd_process_device pdd, peer_pdd;	1262	struct kfd_process_device pdd, peer_pdd;
1537	void *mem;	1263	void *mem;
1538	struct kfd_dev dev, peer;
1539	long err = 0;	1264	long err = 0;
1540	uint32_t *devices_arr = NULL, i;	1265	uint32_t *devices_arr = NULL, i;
1541		1266
1542	dev = kfd_device_by_id(GET_GPU_ID(args->handle));
1543	if (!dev)
1544	return -EINVAL;
1545
1546	if (!args->n_devices) {	1267	if (!args->n_devices) {
1547	pr_debug("Device IDs array empty\n");	1268	pr_debug("Device IDs array empty\n");
1548	return -EINVAL;	1269	return -EINVAL;
@@ -1566,8 +1287,7 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 17 unmodified lines
1566	}	1287	}
1567		1288
1568	mutex_lock(&p->mutex);	1289	mutex_lock(&p->mutex);
1569		1290	pdd = kfd_process_device_data_by_id(p, GET_GPU_ID(args->handle));
1570	pdd = kfd_get_process_device_data(dev, p);
1571	if (!pdd) {	1291	if (!pdd) {
1572	err = -EINVAL;	1292	err = -EINVAL;
1573	goto bind_process_to_device_failed;	1293	goto bind_process_to_device_failed;
@@ -1581,19 +1301,13 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 7 unmodified lines
1581	}	1301	}
1582		1302
1583	for (i = args->n_success; i < args->n_devices; i++) {	1303	for (i = args->n_success; i < args->n_devices; i++) {
1584	peer = kfd_device_by_id(devices_arr[i]);	1304	peer_pdd = kfd_process_device_data_by_id(p, devices_arr[i]);
1585	if (!peer) {
1586	err = -EINVAL;
1587	goto get_mem_obj_from_handle_failed;
1588	}
1589
1590	peer_pdd = kfd_get_process_device_data(peer, p);
1591	if (!peer_pdd) {	1305	if (!peer_pdd) {
1592	err = -ENODEV;	1306	err = -EINVAL;
1593	goto get_mem_obj_from_handle_failed;	1307	goto get_mem_obj_from_handle_failed;
1594	}	1308	}
1595	err = amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(	1309	err = amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(
1596	peer->adev, (struct kgd_mem *)mem, peer_pdd->drm_priv);	1310	peer_pdd->dev->adev, (struct kgd_mem *)mem, peer_pdd->drm_priv);
1597	if (err) {	1311	if (err) {
1598	pr_err("Failed to unmap from gpu %d/%d\n",	1312	pr_err("Failed to unmap from gpu %d/%d\n",
1599	i, args->n_devices);	1313	i, args->n_devices);
@@ -1603,8 +1317,8 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 3 unmodified lines
1603	}	1317	}
1604	mutex_unlock(&p->mutex);	1318	mutex_unlock(&p->mutex);
1605		1319
1606	if (KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2)) {	1320	if (kfd_flush_tlb_after_unmap(pdd->dev)) {
1607	err = amdgpu_amdkfd_gpuvm_sync_memory(dev->adev,	1321	err = amdgpu_amdkfd_gpuvm_sync_memory(pdd->dev->adev,
1608	(struct kgd_mem *) mem, true);	1322	(struct kgd_mem *) mem, true);
1609	if (err) {	1323	if (err) {
1610	pr_debug("Sync memory failed, wait interrupted by user signal\n");	1324	pr_debug("Sync memory failed, wait interrupted by user signal\n");
@@ -1613,10 +1327,7 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 2 unmodified lines
1613		1327
1614	/* Flush TLBs after waiting for the page table updates to complete */	1328	/* Flush TLBs after waiting for the page table updates to complete */
1615	for (i = 0; i < args->n_devices; i++) {	1329	for (i = 0; i < args->n_devices; i++) {
1616	peer = kfd_device_by_id(devices_arr[i]);	1330	peer_pdd = kfd_process_device_data_by_id(p, devices_arr[i]);
1617	if (WARN_ON_ONCE(!peer))
1618	continue;
1619	peer_pdd = kfd_get_process_device_data(peer, p);
1620	if (WARN_ON_ONCE(!peer_pdd))	1331	if (WARN_ON_ONCE(!peer_pdd))
1621	continue;	1332	continue;
1622	kfd_flush_tlb(peer_pdd, TLB_FLUSH_HEAVYWEIGHT);	1333	kfd_flush_tlb(peer_pdd, TLB_FLUSH_HEAVYWEIGHT);
@@ -1736,29 +1447,29 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, · 113 unmodified lines
1736	struct kfd_ioctl_import_dmabuf_args *args = data;	1447	struct kfd_ioctl_import_dmabuf_args *args = data;
1737	struct kfd_process_device *pdd;	1448	struct kfd_process_device *pdd;
1738	struct dma_buf *dmabuf;	1449	struct dma_buf *dmabuf;
1739	struct kfd_dev *dev;
1740	int idr_handle;	1450	int idr_handle;
1741	uint64_t size;	1451	uint64_t size;
1742	void *mem;	1452	void *mem;
1743	int r;	1453	int r;
1744		1454
1745	dev = kfd_device_by_id(args->gpu_id);
1746	if (!dev)
1747	return -EINVAL;
1748
1749	dmabuf = dma_buf_get(args->dmabuf_fd);	1455	dmabuf = dma_buf_get(args->dmabuf_fd);
1750	if (IS_ERR(dmabuf))	1456	if (IS_ERR(dmabuf))
1751	return PTR_ERR(dmabuf);	1457	return PTR_ERR(dmabuf);
1752		1458
1753	mutex_lock(&p->mutex);	1459	mutex_lock(&p->mutex);
		1460	pdd = kfd_process_device_data_by_id(p, args->gpu_id);
		1461	if (!pdd) {
		1462	r = -EINVAL;
		1463	goto err_unlock;
		1464	}
1754		1465
1755	pdd = kfd_bind_process_to_device(dev, p);	1466	pdd = kfd_bind_process_to_device(pdd->dev, p);
1756	if (IS_ERR(pdd)) {	1467	if (IS_ERR(pdd)) {
1757	r = PTR_ERR(pdd);	1468	r = PTR_ERR(pdd);
1758	goto err_unlock;	1469	goto err_unlock;
1759	}	1470	}
1760		1471
1761	r = amdgpu_amdkfd_gpuvm_import_dmabuf(dev->adev, dmabuf,	1472	r = amdgpu_amdkfd_gpuvm_import_dmabuf(pdd->dev->adev, dmabuf,
1762	args->va_addr, pdd->drm_priv,	1473	args->va_addr, pdd->drm_priv,
1763	(struct kgd_mem **)&mem, &size,	1474	(struct kgd_mem **)&mem, &size,
1764	NULL);	1475	NULL);
@@ -1779,7 +1490,7 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, · 14 unmodified lines
1779	return 0;	1490	return 0;
1780		1491
1781	err_free:	1492	err_free:
1782	amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, (struct kgd_mem *)mem,	1493	amdgpu_amdkfd_gpuvm_free_memory_of_gpu(pdd->dev->adev, (struct kgd_mem *)mem,
1783	pdd->drm_priv, NULL);	1494	pdd->drm_priv, NULL);
1784	err_unlock:	1495	err_unlock:
1785	mutex_unlock(&p->mutex);	1496	mutex_unlock(&p->mutex);
@@ -1792,13 +1503,16 @@ static int kfd_ioctl_smi_events(struct file *filep, · 6 unmodified lines
1792	struct kfd_process p, void data)	1503	struct kfd_process p, void data)
1793	{	1504	{
1794	struct kfd_ioctl_smi_events_args *args = data;	1505	struct kfd_ioctl_smi_events_args *args = data;
1795	struct kfd_dev *dev;	1506	struct kfd_process_device *pdd;
1796		1507
1797	dev = kfd_device_by_id(args->gpuid);	1508	mutex_lock(&p->mutex);
1798	if (!dev)	1509
		1510	pdd = kfd_process_device_data_by_id(p, args->gpuid);
		1511	mutex_unlock(&p->mutex);
		1512	if (!pdd)
1799	return -EINVAL;	1513	return -EINVAL;
1800		1514
1801	return kfd_smi_event_open(dev, &args->anon_fd);	1515	return kfd_smi_event_open(pdd->dev, &args->anon_fd);
1802	}	1516	}
1803		1517
1804	static int kfd_ioctl_set_xnack_mode(struct file *filep,	1518	static int kfd_ioctl_set_xnack_mode(struct file *filep,
@@ -1840,13 +1554,9 @@ static int kfd_ioctl_svm(struct file filep, struct kfd_process p, void *data) · 35 unmodified lines
1840	if (!args->start_addr \|\| !args->size)	1554	if (!args->start_addr \|\| !args->size)
1841	return -EINVAL;	1555	return -EINVAL;
1842		1556
1843	mutex_lock(&p->mutex);
1844
1845	r = svm_ioctl(p, args->op, args->start_addr, args->size, args->nattr,	1557	r = svm_ioctl(p, args->op, args->start_addr, args->size, args->nattr,
1846	args->attrs);	1558	args->attrs);
1847		1559
1848	mutex_unlock(&p->mutex);
1849
1850	return r;	1560	return r;
1851	}	1561	}
1852	#else	1562	#else
@@ -1856,12 +1566,1030 @@ static int kfd_ioctl_svm(struct file filep, struct kfd_process p, void *data) · 3 unmodified lines
1856	}	1566	}
1857	#endif	1567	#endif
1858		1568
1859	#define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \	1569	static int criu_checkpoint_process(struct kfd_process *p,
1860	[_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \	1570	uint8_t __user *user_priv_data,
1861	.cmd_drv = 0, .name = #ioctl}	1571	uint64_t *priv_offset)
		1572	{
		1573	struct kfd_criu_process_priv_data process_priv;
		1574	int ret;
1862		1575
1863	/** Ioctl table */	1576	memset(&process_priv, 0, sizeof(process_priv));
1864	static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = {	1577
		1578	process_priv.version = KFD_CRIU_PRIV_VERSION;
		1579	/* For CR, we don't consider negative xnack mode which is used for
		1580	* querying without changing it, here 0 simply means disabled and 1
		1581	* means enabled so retry for finding a valid PTE.
		1582	*/
		1583	process_priv.xnack_mode = p->xnack_enabled ? 1 : 0;
		1584
		1585	ret = copy_to_user(user_priv_data + *priv_offset,
		1586	&process_priv, sizeof(process_priv));
		1587
		1588	if (ret) {
		1589	pr_err("Failed to copy process information to user\n");
		1590	ret = -EFAULT;
		1591	}
		1592
		1593	*priv_offset += sizeof(process_priv);
		1594	return ret;
		1595	}
		1596
		1597	static int criu_checkpoint_devices(struct kfd_process *p,
		1598	uint32_t num_devices,
		1599	uint8_t __user *user_addr,
		1600	uint8_t __user *user_priv_data,
		1601	uint64_t *priv_offset)
		1602	{
		1603	struct kfd_criu_device_priv_data *device_priv = NULL;
		1604	struct kfd_criu_device_bucket *device_buckets = NULL;
		1605	int ret = 0, i;
		1606
		1607	device_buckets = kvzalloc(num_devices * sizeof(*device_buckets), GFP_KERNEL);
		1608	if (!device_buckets) {
		1609	ret = -ENOMEM;
		1610	goto exit;
		1611	}
		1612
		1613	device_priv = kvzalloc(num_devices * sizeof(*device_priv), GFP_KERNEL);
		1614	if (!device_priv) {
		1615	ret = -ENOMEM;
		1616	goto exit;
		1617	}
		1618
		1619	for (i = 0; i < num_devices; i++) {
		1620	struct kfd_process_device *pdd = p->pdds[i];
		1621
		1622	device_buckets[i].user_gpu_id = pdd->user_gpu_id;
		1623	device_buckets[i].actual_gpu_id = pdd->dev->id;
		1624
		1625	/*
		1626	* priv_data does not contain useful information for now and is reserved for
		1627	* future use, so we do not set its contents.
		1628	*/
		1629	}
		1630
		1631	ret = copy_to_user(user_addr, device_buckets, num_devices * sizeof(*device_buckets));
		1632	if (ret) {
		1633	pr_err("Failed to copy device information to user\n");
		1634	ret = -EFAULT;
		1635	goto exit;
		1636	}
		1637
		1638	ret = copy_to_user(user_priv_data + *priv_offset,
		1639	device_priv,
		1640	num_devices * sizeof(*device_priv));
		1641	if (ret) {
		1642	pr_err("Failed to copy device information to user\n");
		1643	ret = -EFAULT;
		1644	}
		1645	priv_offset += num_devices sizeof(*device_priv);
		1646
		1647	exit:
		1648	kvfree(device_buckets);
		1649	kvfree(device_priv);
		1650	return ret;
		1651	}
		1652
		1653	static uint32_t get_process_num_bos(struct kfd_process *p)
		1654	{
		1655	uint32_t num_of_bos = 0;
		1656	int i;
		1657
		1658	/* Run over all PDDs of the process */
		1659	for (i = 0; i < p->n_pdds; i++) {
		1660	struct kfd_process_device *pdd = p->pdds[i];
		1661	void *mem;
		1662	int id;
		1663
		1664	idr_for_each_entry(&pdd->alloc_idr, mem, id) {
		1665	struct kgd_mem kgd_mem = (struct kgd_mem )mem;
		1666
		1667	if ((uint64_t)kgd_mem->va > pdd->gpuvm_base)
		1668	num_of_bos++;
		1669	}
		1670	}
		1671	return num_of_bos;
		1672	}
		1673
		1674	static int criu_get_prime_handle(struct drm_gem_object *gobj, int flags,
		1675	u32 *shared_fd)
		1676	{
		1677	struct dma_buf *dmabuf;
		1678	int ret;
		1679
		1680	dmabuf = amdgpu_gem_prime_export(gobj, flags);
		1681	if (IS_ERR(dmabuf)) {
		1682	ret = PTR_ERR(dmabuf);
		1683	pr_err("dmabuf export failed for the BO\n");
		1684	return ret;
		1685	}
		1686
		1687	ret = dma_buf_fd(dmabuf, flags);
		1688	if (ret < 0) {
		1689	pr_err("dmabuf create fd failed, ret:%d\n", ret);
		1690	goto out_free_dmabuf;
		1691	}
		1692
		1693	*shared_fd = ret;
		1694	return 0;
		1695
		1696	out_free_dmabuf:
		1697	dma_buf_put(dmabuf);
		1698	return ret;
		1699	}
		1700
		1701	static int criu_checkpoint_bos(struct kfd_process *p,
		1702	uint32_t num_bos,
		1703	uint8_t __user *user_bos,
		1704	uint8_t __user *user_priv_data,
		1705	uint64_t *priv_offset)
		1706	{
		1707	struct kfd_criu_bo_bucket *bo_buckets;
		1708	struct kfd_criu_bo_priv_data *bo_privs;
		1709	int ret = 0, pdd_index, bo_index = 0, id;
		1710	void *mem;
		1711
		1712	bo_buckets = kvzalloc(num_bos * sizeof(*bo_buckets), GFP_KERNEL);
		1713	if (!bo_buckets)
		1714	return -ENOMEM;
		1715
		1716	bo_privs = kvzalloc(num_bos * sizeof(*bo_privs), GFP_KERNEL);
		1717	if (!bo_privs) {
		1718	ret = -ENOMEM;
		1719	goto exit;
		1720	}
		1721
		1722	for (pdd_index = 0; pdd_index < p->n_pdds; pdd_index++) {
		1723	struct kfd_process_device *pdd = p->pdds[pdd_index];
		1724	struct amdgpu_bo *dumper_bo;
		1725	struct kgd_mem *kgd_mem;
		1726
		1727	idr_for_each_entry(&pdd->alloc_idr, mem, id) {
		1728	struct kfd_criu_bo_bucket *bo_bucket;
		1729	struct kfd_criu_bo_priv_data *bo_priv;
		1730	int i, dev_idx = 0;
		1731
		1732	if (!mem) {
		1733	ret = -ENOMEM;
		1734	goto exit;
		1735	}
		1736
		1737	kgd_mem = (struct kgd_mem *)mem;
		1738	dumper_bo = kgd_mem->bo;
		1739
		1740	if ((uint64_t)kgd_mem->va <= pdd->gpuvm_base)
		1741	continue;
		1742
		1743	bo_bucket = &bo_buckets[bo_index];
		1744	bo_priv = &bo_privs[bo_index];
		1745
		1746	bo_bucket->gpu_id = pdd->user_gpu_id;
		1747	bo_bucket->addr = (uint64_t)kgd_mem->va;
		1748	bo_bucket->size = amdgpu_bo_size(dumper_bo);
		1749	bo_bucket->alloc_flags = (uint32_t)kgd_mem->alloc_flags;
		1750	bo_priv->idr_handle = id;
		1751
		1752	if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR) {
		1753	ret = amdgpu_ttm_tt_get_userptr(&dumper_bo->tbo,
		1754	&bo_priv->user_addr);
		1755	if (ret) {
		1756	pr_err("Failed to obtain user address for user-pointer bo\n");
		1757	goto exit;
		1758	}
		1759	}
		1760	if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) {
		1761	ret = criu_get_prime_handle(&dumper_bo->tbo.base,
		1762	bo_bucket->alloc_flags &
		1763	KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? DRM_RDWR : 0,
		1764	&bo_bucket->dmabuf_fd);
		1765	if (ret)
		1766	goto exit;
		1767	}
		1768	if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL)
		1769	bo_bucket->offset = KFD_MMAP_TYPE_DOORBELL \|
		1770	KFD_MMAP_GPU_ID(pdd->dev->id);
		1771	else if (bo_bucket->alloc_flags &
		1772	KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)
		1773	bo_bucket->offset = KFD_MMAP_TYPE_MMIO \|
		1774	KFD_MMAP_GPU_ID(pdd->dev->id);
		1775	else
		1776	bo_bucket->offset = amdgpu_bo_mmap_offset(dumper_bo);
		1777
		1778	for (i = 0; i < p->n_pdds; i++) {
		1779	if (amdgpu_amdkfd_bo_mapped_to_dev(p->pdds[i]->dev->adev, kgd_mem))
		1780	bo_priv->mapped_gpuids[dev_idx++] = p->pdds[i]->user_gpu_id;
		1781	}
		1782
		1783	pr_debug("bo_size = 0x%llx, bo_addr = 0x%llx bo_offset = 0x%llx\n"
		1784	"gpu_id = 0x%x alloc_flags = 0x%x idr_handle = 0x%x",
		1785	bo_bucket->size,
		1786	bo_bucket->addr,
		1787	bo_bucket->offset,
		1788	bo_bucket->gpu_id,
		1789	bo_bucket->alloc_flags,
		1790	bo_priv->idr_handle);
		1791	bo_index++;
		1792	}
		1793	}
		1794
		1795	ret = copy_to_user(user_bos, bo_buckets, num_bos * sizeof(*bo_buckets));
		1796	if (ret) {
		1797	pr_err("Failed to copy BO information to user\n");
		1798	ret = -EFAULT;
		1799	goto exit;
		1800	}
		1801
		1802	ret = copy_to_user(user_priv_data + priv_offset, bo_privs, num_bos sizeof(*bo_privs));
		1803	if (ret) {
		1804	pr_err("Failed to copy BO priv information to user\n");
		1805	ret = -EFAULT;
		1806	goto exit;
		1807	}
		1808
		1809	priv_offset += num_bos sizeof(*bo_privs);
		1810
		1811	exit:
		1812	while (ret && bo_index--) {
		1813	if (bo_buckets[bo_index].alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM)
		1814	close_fd(bo_buckets[bo_index].dmabuf_fd);
		1815	}
		1816
		1817	kvfree(bo_buckets);
		1818	kvfree(bo_privs);
		1819	return ret;
		1820	}
		1821
		1822	static int criu_get_process_object_info(struct kfd_process *p,
		1823	uint32_t *num_devices,
		1824	uint32_t *num_bos,
		1825	uint32_t *num_objects,
		1826	uint64_t *objs_priv_size)
		1827	{
		1828	uint64_t queues_priv_data_size, svm_priv_data_size, priv_size;
		1829	uint32_t num_queues, num_events, num_svm_ranges;
		1830	int ret;
		1831
		1832	*num_devices = p->n_pdds;
		1833	*num_bos = get_process_num_bos(p);
		1834
		1835	ret = kfd_process_get_queue_info(p, &num_queues, &queues_priv_data_size);
		1836	if (ret)
		1837	return ret;
		1838
		1839	num_events = kfd_get_num_events(p);
		1840
		1841	ret = svm_range_get_info(p, &num_svm_ranges, &svm_priv_data_size);
		1842	if (ret)
		1843	return ret;
		1844
		1845	*num_objects = num_queues + num_events + num_svm_ranges;
		1846
		1847	if (objs_priv_size) {
		1848	priv_size = sizeof(struct kfd_criu_process_priv_data);
		1849	priv_size += num_devices sizeof(struct kfd_criu_device_priv_data);
		1850	priv_size += num_bos sizeof(struct kfd_criu_bo_priv_data);
		1851	priv_size += queues_priv_data_size;
		1852	priv_size += num_events * sizeof(struct kfd_criu_event_priv_data);
		1853	priv_size += svm_priv_data_size;
		1854	*objs_priv_size = priv_size;
		1855	}
		1856	return 0;
		1857	}
		1858
		1859	static int criu_checkpoint(struct file *filep,
		1860	struct kfd_process *p,
		1861	struct kfd_ioctl_criu_args *args)
		1862	{
		1863	int ret;
		1864	uint32_t num_devices, num_bos, num_objects;
		1865	uint64_t priv_size, priv_offset = 0;
		1866
		1867	if (!args->devices \|\| !args->bos \|\| !args->priv_data)
		1868	return -EINVAL;
		1869
		1870	mutex_lock(&p->mutex);
		1871
		1872	if (!p->n_pdds) {
		1873	pr_err("No pdd for given process\n");
		1874	ret = -ENODEV;
		1875	goto exit_unlock;
		1876	}
		1877
		1878	/* Confirm all process queues are evicted */
		1879	if (!p->queues_paused) {
		1880	pr_err("Cannot dump process when queues are not in evicted state\n");
		1881	/* CRIU plugin did not call op PROCESS_INFO before checkpointing */
		1882	ret = -EINVAL;
		1883	goto exit_unlock;
		1884	}
		1885
		1886	ret = criu_get_process_object_info(p, &num_devices, &num_bos, &num_objects, &priv_size);
		1887	if (ret)
		1888	goto exit_unlock;
		1889
		1890	if (num_devices != args->num_devices \|\|
		1891	num_bos != args->num_bos \|\|
		1892	num_objects != args->num_objects \|\|
		1893	priv_size != args->priv_data_size) {
		1894
		1895	ret = -EINVAL;
		1896	goto exit_unlock;
		1897	}
		1898
		1899	/* each function will store private data inside priv_data and adjust priv_offset */
		1900	ret = criu_checkpoint_process(p, (uint8_t __user *)args->priv_data, &priv_offset);
		1901	if (ret)
		1902	goto exit_unlock;
		1903
		1904	ret = criu_checkpoint_devices(p, num_devices, (uint8_t __user *)args->devices,
		1905	(uint8_t __user *)args->priv_data, &priv_offset);
		1906	if (ret)
		1907	goto exit_unlock;
		1908
		1909	ret = criu_checkpoint_bos(p, num_bos, (uint8_t __user *)args->bos,
		1910	(uint8_t __user *)args->priv_data, &priv_offset);
		1911	if (ret)
		1912	goto exit_unlock;
		1913
		1914	if (num_objects) {
		1915	ret = kfd_criu_checkpoint_queues(p, (uint8_t __user *)args->priv_data,
		1916	&priv_offset);
		1917	if (ret)
		1918	goto close_bo_fds;
		1919
		1920	ret = kfd_criu_checkpoint_events(p, (uint8_t __user *)args->priv_data,
		1921	&priv_offset);
		1922	if (ret)
		1923	goto close_bo_fds;
		1924
		1925	ret = kfd_criu_checkpoint_svm(p, (uint8_t __user *)args->priv_data, &priv_offset);
		1926	if (ret)
		1927	goto close_bo_fds;
		1928	}
		1929
		1930	close_bo_fds:
		1931	if (ret) {
		1932	/* If IOCTL returns err, user assumes all FDs opened in criu_dump_bos are closed */
		1933	uint32_t i;
		1934	struct kfd_criu_bo_bucket bo_buckets = (struct kfd_criu_bo_bucket ) args->bos;
		1935
		1936	for (i = 0; i < num_bos; i++) {
		1937	if (bo_buckets[i].alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM)
		1938	close_fd(bo_buckets[i].dmabuf_fd);
		1939	}
		1940	}
		1941
		1942	exit_unlock:
		1943	mutex_unlock(&p->mutex);
		1944	if (ret)
		1945	pr_err("Failed to dump CRIU ret:%d\n", ret);
		1946	else
		1947	pr_debug("CRIU dump ret:%d\n", ret);
		1948
		1949	return ret;
		1950	}
		1951
		1952	static int criu_restore_process(struct kfd_process *p,
		1953	struct kfd_ioctl_criu_args *args,
		1954	uint64_t *priv_offset,
		1955	uint64_t max_priv_data_size)
		1956	{
		1957	int ret = 0;
		1958	struct kfd_criu_process_priv_data process_priv;
		1959
		1960	if (*priv_offset + sizeof(process_priv) > max_priv_data_size)
		1961	return -EINVAL;
		1962
		1963	ret = copy_from_user(&process_priv,
		1964	(void __user )(args->priv_data + priv_offset),
		1965	sizeof(process_priv));
		1966	if (ret) {
		1967	pr_err("Failed to copy process private information from user\n");
		1968	ret = -EFAULT;
		1969	goto exit;
		1970	}
		1971	*priv_offset += sizeof(process_priv);
		1972
		1973	if (process_priv.version != KFD_CRIU_PRIV_VERSION) {
		1974	pr_err("Invalid CRIU API version (checkpointed:%d current:%d)\n",
		1975	process_priv.version, KFD_CRIU_PRIV_VERSION);
		1976	return -EINVAL;
		1977	}
		1978
		1979	pr_debug("Setting XNACK mode\n");
		1980	if (process_priv.xnack_mode && !kfd_process_xnack_mode(p, true)) {
		1981	pr_err("xnack mode cannot be set\n");
		1982	ret = -EPERM;
		1983	goto exit;
		1984	} else {
		1985	pr_debug("set xnack mode: %d\n", process_priv.xnack_mode);
		1986	p->xnack_enabled = process_priv.xnack_mode;
		1987	}
		1988
		1989	exit:
		1990	return ret;
		1991	}
		1992
		1993	static int criu_restore_devices(struct kfd_process *p,
		1994	struct kfd_ioctl_criu_args *args,
		1995	uint64_t *priv_offset,
		1996	uint64_t max_priv_data_size)
		1997	{
		1998	struct kfd_criu_device_bucket *device_buckets;
		1999	struct kfd_criu_device_priv_data *device_privs;
		2000	int ret = 0;
		2001	uint32_t i;
		2002
		2003	if (args->num_devices != p->n_pdds)
		2004	return -EINVAL;
		2005
		2006	if (priv_offset + (args->num_devices sizeof(*device_privs)) > max_priv_data_size)
		2007	return -EINVAL;
		2008
		2009	device_buckets = kmalloc_array(args->num_devices, sizeof(*device_buckets), GFP_KERNEL);
		2010	if (!device_buckets)
		2011	return -ENOMEM;
		2012
		2013	ret = copy_from_user(device_buckets, (void __user *)args->devices,
		2014	args->num_devices * sizeof(*device_buckets));
		2015	if (ret) {
		2016	pr_err("Failed to copy devices buckets from user\n");
		2017	ret = -EFAULT;
		2018	goto exit;
		2019	}
		2020
		2021	for (i = 0; i < args->num_devices; i++) {
		2022	struct kfd_dev *dev;
		2023	struct kfd_process_device *pdd;
		2024	struct file *drm_file;
		2025
		2026	/* device private data is not currently used */
		2027
		2028	if (!device_buckets[i].user_gpu_id) {
		2029	pr_err("Invalid user gpu_id\n");
		2030	ret = -EINVAL;
		2031	goto exit;
		2032	}
		2033
		2034	dev = kfd_device_by_id(device_buckets[i].actual_gpu_id);
		2035	if (!dev) {
		2036	pr_err("Failed to find device with gpu_id = %x\n",
		2037	device_buckets[i].actual_gpu_id);
		2038	ret = -EINVAL;
		2039	goto exit;
		2040	}
		2041
		2042	pdd = kfd_get_process_device_data(dev, p);
		2043	if (!pdd) {
		2044	pr_err("Failed to get pdd for gpu_id = %x\n",
		2045	device_buckets[i].actual_gpu_id);
		2046	ret = -EINVAL;
		2047	goto exit;
		2048	}
		2049	pdd->user_gpu_id = device_buckets[i].user_gpu_id;
		2050
		2051	drm_file = fget(device_buckets[i].drm_fd);
		2052	if (!drm_file) {
		2053	pr_err("Invalid render node file descriptor sent from plugin (%d)\n",
		2054	device_buckets[i].drm_fd);
		2055	ret = -EINVAL;
		2056	goto exit;
		2057	}
		2058
		2059	if (pdd->drm_file) {
		2060	ret = -EINVAL;
		2061	goto exit;
		2062	}
		2063
		2064	/* create the vm using render nodes for kfd pdd */
		2065	if (kfd_process_device_init_vm(pdd, drm_file)) {
		2066	pr_err("could not init vm for given pdd\n");
		2067	/* On success, the PDD keeps the drm_file reference */
		2068	fput(drm_file);
		2069	ret = -EINVAL;
		2070	goto exit;
		2071	}
		2072	/*
		2073	* pdd now already has the vm bound to render node so below api won't create a new
		2074	* exclusive kfd mapping but use existing one with renderDXXX but is still needed
		2075	* for iommu v2 binding and runtime pm.
		2076	*/
		2077	pdd = kfd_bind_process_to_device(dev, p);
		2078	if (IS_ERR(pdd)) {
		2079	ret = PTR_ERR(pdd);
		2080	goto exit;
		2081	}
		2082	}
		2083
		2084	/*
		2085	* We are not copying device private data from user as we are not using the data for now,
		2086	* but we still adjust for its private data.
		2087	*/
		2088	priv_offset += args->num_devices sizeof(*device_privs);
		2089
		2090	exit:
		2091	kfree(device_buckets);
		2092	return ret;
		2093	}
		2094
		2095	static int criu_restore_bos(struct kfd_process *p,
		2096	struct kfd_ioctl_criu_args *args,
		2097	uint64_t *priv_offset,
		2098	uint64_t max_priv_data_size)
		2099	{
		2100	struct kfd_criu_bo_bucket *bo_buckets;
		2101	struct kfd_criu_bo_priv_data *bo_privs;
		2102	const bool criu_resume = true;
		2103	bool flush_tlbs = false;
		2104	int ret = 0, j = 0;
		2105	uint32_t i = 0;
		2106
		2107	if (priv_offset + (args->num_bos sizeof(*bo_privs)) > max_priv_data_size)
		2108	return -EINVAL;
		2109
		2110	/* Prevent MMU notifications until stage-4 IOCTL (CRIU_RESUME) is received */
		2111	amdgpu_amdkfd_block_mmu_notifications(p->kgd_process_info);
		2112
		2113	bo_buckets = kvmalloc_array(args->num_bos, sizeof(*bo_buckets), GFP_KERNEL);
		2114	if (!bo_buckets)
		2115	return -ENOMEM;
		2116
		2117	ret = copy_from_user(bo_buckets, (void __user *)args->bos,
		2118	args->num_bos * sizeof(*bo_buckets));
		2119	if (ret) {
		2120	pr_err("Failed to copy BOs information from user\n");
		2121	ret = -EFAULT;
		2122	goto exit;
		2123	}
		2124
		2125	bo_privs = kvmalloc_array(args->num_bos, sizeof(*bo_privs), GFP_KERNEL);
		2126	if (!bo_privs) {
		2127	ret = -ENOMEM;
		2128	goto exit;
		2129	}
		2130
		2131	ret = copy_from_user(bo_privs, (void __user )args->priv_data + priv_offset,
		2132	args->num_bos * sizeof(*bo_privs));
		2133	if (ret) {
		2134	pr_err("Failed to copy BOs information from user\n");
		2135	ret = -EFAULT;
		2136	goto exit;
		2137	}
		2138	priv_offset += args->num_bos sizeof(*bo_privs);
		2139
		2140	/* Create and map new BOs */
		2141	for (; i < args->num_bos; i++) {
		2142	struct kfd_criu_bo_bucket *bo_bucket;
		2143	struct kfd_criu_bo_priv_data *bo_priv;
		2144	struct kfd_dev *dev;
		2145	struct kfd_process_device *pdd;
		2146	struct kgd_mem *kgd_mem;
		2147	void *mem;
		2148	u64 offset;
		2149	int idr_handle;
		2150
		2151	bo_bucket = &bo_buckets[i];
		2152	bo_priv = &bo_privs[i];
		2153
		2154	pr_debug("kfd restore ioctl - bo_bucket[%d]:\n", i);
		2155	pr_debug("size = 0x%llx, bo_addr = 0x%llx bo_offset = 0x%llx\n"
		2156	"gpu_id = 0x%x alloc_flags = 0x%x\n"
		2157	"idr_handle = 0x%x\n",
		2158	bo_bucket->size,
		2159	bo_bucket->addr,
		2160	bo_bucket->offset,
		2161	bo_bucket->gpu_id,
		2162	bo_bucket->alloc_flags,
		2163	bo_priv->idr_handle);
		2164
		2165	pdd = kfd_process_device_data_by_id(p, bo_bucket->gpu_id);
		2166	if (!pdd) {
		2167	pr_err("Failed to get pdd\n");
		2168	ret = -ENODEV;
		2169	goto exit;
		2170	}
		2171	dev = pdd->dev;
		2172
		2173	if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL) {
		2174	pr_debug("restore ioctl: KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL\n");
		2175	if (bo_bucket->size != kfd_doorbell_process_slice(dev)) {
		2176	ret = -EINVAL;
		2177	goto exit;
		2178	}
		2179	offset = kfd_get_process_doorbells(pdd);
		2180	} else if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP) {
		2181	/* MMIO BOs need remapped bus address */
		2182	pr_debug("restore ioctl :KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP\n");
		2183	if (bo_bucket->size != PAGE_SIZE) {
		2184	pr_err("Invalid page size\n");
		2185	ret = -EINVAL;
		2186	goto exit;
		2187	}
		2188	offset = dev->adev->rmmio_remap.bus_addr;
		2189	if (!offset) {
		2190	pr_err("amdgpu_amdkfd_get_mmio_remap_phys_addr failed\n");
		2191	ret = -ENOMEM;
		2192	goto exit;
		2193	}
		2194	} else if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR) {
		2195	offset = bo_priv->user_addr;
		2196	}
		2197	/* Create the BO */
		2198	ret = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(dev->adev,
		2199	bo_bucket->addr,
		2200	bo_bucket->size,
		2201	pdd->drm_priv,
		2202	(struct kgd_mem **) &mem,
		2203	&offset,
		2204	bo_bucket->alloc_flags,
		2205	criu_resume);
		2206	if (ret) {
		2207	pr_err("Could not create the BO\n");
		2208	ret = -ENOMEM;
		2209	goto exit;
		2210	}
		2211	pr_debug("New BO created: size = 0x%llx, bo_addr = 0x%llx bo_offset = 0x%llx\n",
		2212	bo_bucket->size, bo_bucket->addr, offset);
		2213
		2214	/* Restore previuos IDR handle */
		2215	pr_debug("Restoring old IDR handle for the BO");
		2216	idr_handle = idr_alloc(&pdd->alloc_idr, mem,
		2217	bo_priv->idr_handle,
		2218	bo_priv->idr_handle + 1, GFP_KERNEL);
		2219
		2220	if (idr_handle < 0) {
		2221	pr_err("Could not allocate idr\n");
		2222	amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev,
		2223	(struct kgd_mem *)mem,
		2224	pdd->drm_priv, NULL);
		2225	ret = -ENOMEM;
		2226	goto exit;
		2227	}
		2228
		2229	if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL)
		2230	bo_bucket->restored_offset = KFD_MMAP_TYPE_DOORBELL \|
		2231	KFD_MMAP_GPU_ID(pdd->dev->id);
		2232	if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP) {
		2233	bo_bucket->restored_offset = KFD_MMAP_TYPE_MMIO \|
		2234	KFD_MMAP_GPU_ID(pdd->dev->id);
		2235	} else if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_GTT) {
		2236	bo_bucket->restored_offset = offset;
		2237	pr_debug("updating offset for GTT\n");
		2238	} else if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) {
		2239	bo_bucket->restored_offset = offset;
		2240	/* Update the VRAM usage count */
		2241	WRITE_ONCE(pdd->vram_usage, pdd->vram_usage + bo_bucket->size);
		2242	pr_debug("updating offset for VRAM\n");
		2243	}
		2244
		2245	/* now map these BOs to GPU/s */
		2246	for (j = 0; j < p->n_pdds; j++) {
		2247	struct kfd_dev *peer;
		2248	struct kfd_process_device *peer_pdd;
		2249	bool table_freed = false;
		2250
		2251	if (!bo_priv->mapped_gpuids[j])
		2252	break;
		2253
		2254	peer_pdd = kfd_process_device_data_by_id(p, bo_priv->mapped_gpuids[j]);
		2255	if (!peer_pdd) {
		2256	ret = -EINVAL;
		2257	goto exit;
		2258	}
		2259	peer = peer_pdd->dev;
		2260
		2261	peer_pdd = kfd_bind_process_to_device(peer, p);
		2262	if (IS_ERR(peer_pdd)) {
		2263	ret = PTR_ERR(peer_pdd);
		2264	goto exit;
		2265	}
		2266	pr_debug("map mem in restore ioctl -> 0x%llx\n",
		2267	((struct kgd_mem *)mem)->va);
		2268	ret = amdgpu_amdkfd_gpuvm_map_memory_to_gpu(peer->adev,
		2269	(struct kgd_mem *)mem, peer_pdd->drm_priv, &table_freed);
		2270	if (ret) {
		2271	pr_err("Failed to map to gpu %d/%d\n", j, p->n_pdds);
		2272	goto exit;
		2273	}
		2274	if (table_freed)
		2275	flush_tlbs = true;
		2276	}
		2277
		2278	ret = amdgpu_amdkfd_gpuvm_sync_memory(dev->adev,
		2279	(struct kgd_mem *) mem, true);
		2280	if (ret) {
		2281	pr_debug("Sync memory failed, wait interrupted by user signal\n");
		2282	goto exit;
		2283	}
		2284
		2285	pr_debug("map memory was successful for the BO\n");
		2286	/* create the dmabuf object and export the bo */
		2287	kgd_mem = (struct kgd_mem *)mem;
		2288	if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) {
		2289	ret = criu_get_prime_handle(&kgd_mem->bo->tbo.base,
		2290	DRM_RDWR,
		2291	&bo_bucket->dmabuf_fd);
		2292	if (ret)
		2293	goto exit;
		2294	}
		2295	} /* done */
		2296
		2297	if (flush_tlbs) {
		2298	/* Flush TLBs after waiting for the page table updates to complete */
		2299	for (j = 0; j < p->n_pdds; j++) {
		2300	struct kfd_dev *peer;
		2301	struct kfd_process_device *pdd = p->pdds[j];
		2302	struct kfd_process_device *peer_pdd;
		2303
		2304	peer = kfd_device_by_id(pdd->dev->id);
		2305	if (WARN_ON_ONCE(!peer))
		2306	continue;
		2307	peer_pdd = kfd_get_process_device_data(peer, p);
		2308	if (WARN_ON_ONCE(!peer_pdd))
		2309	continue;
		2310	kfd_flush_tlb(peer_pdd, TLB_FLUSH_LEGACY);
		2311	}
		2312	}
		2313
		2314	/* Copy only the buckets back so user can read bo_buckets[N].restored_offset */
		2315	ret = copy_to_user((void __user *)args->bos,
		2316	bo_buckets,
		2317	(args->num_bos * sizeof(*bo_buckets)));
		2318	if (ret)
		2319	ret = -EFAULT;
		2320
		2321	exit:
		2322	while (ret && i--) {
		2323	if (bo_buckets[i].alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM)
		2324	close_fd(bo_buckets[i].dmabuf_fd);
		2325	}
		2326	kvfree(bo_buckets);
		2327	kvfree(bo_privs);
		2328	return ret;
		2329	}
		2330
		2331	static int criu_restore_objects(struct file *filep,
		2332	struct kfd_process *p,
		2333	struct kfd_ioctl_criu_args *args,
		2334	uint64_t *priv_offset,
		2335	uint64_t max_priv_data_size)
		2336	{
		2337	int ret = 0;
		2338	uint32_t i;
		2339
		2340	BUILD_BUG_ON(offsetof(struct kfd_criu_queue_priv_data, object_type));
		2341	BUILD_BUG_ON(offsetof(struct kfd_criu_event_priv_data, object_type));
		2342	BUILD_BUG_ON(offsetof(struct kfd_criu_svm_range_priv_data, object_type));
		2343
		2344	for (i = 0; i < args->num_objects; i++) {
		2345	uint32_t object_type;
		2346
		2347	if (*priv_offset + sizeof(object_type) > max_priv_data_size) {
		2348	pr_err("Invalid private data size\n");
		2349	return -EINVAL;
		2350	}
		2351
		2352	ret = get_user(object_type, (uint32_t __user )(args->priv_data + priv_offset));
		2353	if (ret) {
		2354	pr_err("Failed to copy private information from user\n");
		2355	goto exit;
		2356	}
		2357
		2358	switch (object_type) {
		2359	case KFD_CRIU_OBJECT_TYPE_QUEUE:
		2360	ret = kfd_criu_restore_queue(p, (uint8_t __user *)args->priv_data,
		2361	priv_offset, max_priv_data_size);
		2362	if (ret)
		2363	goto exit;
		2364	break;
		2365	case KFD_CRIU_OBJECT_TYPE_EVENT:
		2366	ret = kfd_criu_restore_event(filep, p, (uint8_t __user *)args->priv_data,
		2367	priv_offset, max_priv_data_size);
		2368	if (ret)
		2369	goto exit;
		2370	break;
		2371	case KFD_CRIU_OBJECT_TYPE_SVM_RANGE:
		2372	ret = kfd_criu_restore_svm(p, (uint8_t __user *)args->priv_data,
		2373	priv_offset, max_priv_data_size);
		2374	if (ret)
		2375	goto exit;
		2376	break;
		2377	default:
		2378	pr_err("Invalid object type:%u at index:%d\n", object_type, i);
		2379	ret = -EINVAL;
		2380	goto exit;
		2381	}
		2382	}
		2383	exit:
		2384	return ret;
		2385	}
		2386
		2387	static int criu_restore(struct file *filep,
		2388	struct kfd_process *p,
		2389	struct kfd_ioctl_criu_args *args)
		2390	{
		2391	uint64_t priv_offset = 0;
		2392	int ret = 0;
		2393
		2394	pr_debug("CRIU restore (num_devices:%u num_bos:%u num_objects:%u priv_data_size:%llu)\n",
		2395	args->num_devices, args->num_bos, args->num_objects, args->priv_data_size);
		2396
		2397	if (!args->bos \|\| !args->devices \|\| !args->priv_data \|\| !args->priv_data_size \|\|
		2398	!args->num_devices \|\| !args->num_bos)
		2399	return -EINVAL;
		2400
		2401	mutex_lock(&p->mutex);
		2402
		2403	/*
		2404	* Set the process to evicted state to avoid running any new queues before all the memory
		2405	* mappings are ready.
		2406	*/
		2407	ret = kfd_process_evict_queues(p);
		2408	if (ret)
		2409	goto exit_unlock;
		2410
		2411	/* Each function will adjust priv_offset based on how many bytes they consumed */
		2412	ret = criu_restore_process(p, args, &priv_offset, args->priv_data_size);
		2413	if (ret)
		2414	goto exit_unlock;
		2415
		2416	ret = criu_restore_devices(p, args, &priv_offset, args->priv_data_size);
		2417	if (ret)
		2418	goto exit_unlock;
		2419
		2420	ret = criu_restore_bos(p, args, &priv_offset, args->priv_data_size);
		2421	if (ret)
		2422	goto exit_unlock;
		2423
		2424	ret = criu_restore_objects(filep, p, args, &priv_offset, args->priv_data_size);
		2425	if (ret)
		2426	goto exit_unlock;
		2427
		2428	if (priv_offset != args->priv_data_size) {
		2429	pr_err("Invalid private data size\n");
		2430	ret = -EINVAL;
		2431	}
		2432
		2433	exit_unlock:
		2434	mutex_unlock(&p->mutex);
		2435	if (ret)
		2436	pr_err("Failed to restore CRIU ret:%d\n", ret);
		2437	else
		2438	pr_debug("CRIU restore successful\n");
		2439
		2440	return ret;
		2441	}
		2442
		2443	static int criu_unpause(struct file *filep,
		2444	struct kfd_process *p,
		2445	struct kfd_ioctl_criu_args *args)
		2446	{
		2447	int ret;
		2448
		2449	mutex_lock(&p->mutex);
		2450
		2451	if (!p->queues_paused) {
		2452	mutex_unlock(&p->mutex);
		2453	return -EINVAL;
		2454	}
		2455
		2456	ret = kfd_process_restore_queues(p);
		2457	if (ret)
		2458	pr_err("Failed to unpause queues ret:%d\n", ret);
		2459	else
		2460	p->queues_paused = false;
		2461
		2462	mutex_unlock(&p->mutex);
		2463
		2464	return ret;
		2465	}
		2466
		2467	static int criu_resume(struct file *filep,
		2468	struct kfd_process *p,
		2469	struct kfd_ioctl_criu_args *args)
		2470	{
		2471	struct kfd_process *target = NULL;
		2472	struct pid *pid = NULL;
		2473	int ret = 0;
		2474
		2475	pr_debug("Inside %s, target pid for criu restore: %d\n", __func__,
		2476	args->pid);
		2477
		2478	pid = find_get_pid(args->pid);
		2479	if (!pid) {
		2480	pr_err("Cannot find pid info for %i\n", args->pid);
		2481	return -ESRCH;
		2482	}
		2483
		2484	pr_debug("calling kfd_lookup_process_by_pid\n");
		2485	target = kfd_lookup_process_by_pid(pid);
		2486
		2487	put_pid(pid);
		2488
		2489	if (!target) {
		2490	pr_debug("Cannot find process info for %i\n", args->pid);
		2491	return -ESRCH;
		2492	}
		2493
		2494	mutex_lock(&target->mutex);
		2495	ret = kfd_criu_resume_svm(target);
		2496	if (ret) {
		2497	pr_err("kfd_criu_resume_svm failed for %i\n", args->pid);
		2498	goto exit;
		2499	}
		2500
		2501	ret = amdgpu_amdkfd_criu_resume(target->kgd_process_info);
		2502	if (ret)
		2503	pr_err("amdgpu_amdkfd_criu_resume failed for %i\n", args->pid);
		2504
		2505	exit:
		2506	mutex_unlock(&target->mutex);
		2507
		2508	kfd_unref_process(target);
		2509	return ret;
		2510	}
		2511
		2512	static int criu_process_info(struct file *filep,
		2513	struct kfd_process *p,
		2514	struct kfd_ioctl_criu_args *args)
		2515	{
		2516	int ret = 0;
		2517
		2518	mutex_lock(&p->mutex);
		2519
		2520	if (!p->n_pdds) {
		2521	pr_err("No pdd for given process\n");
		2522	ret = -ENODEV;
		2523	goto err_unlock;
		2524	}
		2525
		2526	ret = kfd_process_evict_queues(p);
		2527	if (ret)
		2528	goto err_unlock;
		2529
		2530	p->queues_paused = true;
		2531
		2532	args->pid = task_pid_nr_ns(p->lead_thread,
		2533	task_active_pid_ns(p->lead_thread));
		2534
		2535	ret = criu_get_process_object_info(p, &args->num_devices, &args->num_bos,
		2536	&args->num_objects, &args->priv_data_size);
		2537	if (ret)
		2538	goto err_unlock;
		2539
		2540	dev_dbg(kfd_device, "Num of devices:%u bos:%u objects:%u priv_data_size:%lld\n",
		2541	args->num_devices, args->num_bos, args->num_objects,
		2542	args->priv_data_size);
		2543
		2544	err_unlock:
		2545	if (ret) {
		2546	kfd_process_restore_queues(p);
		2547	p->queues_paused = false;
		2548	}
		2549	mutex_unlock(&p->mutex);
		2550	return ret;
		2551	}
		2552
		2553	static int kfd_ioctl_criu(struct file filep, struct kfd_process p, void *data)
		2554	{
		2555	struct kfd_ioctl_criu_args *args = data;
		2556	int ret;
		2557
		2558	dev_dbg(kfd_device, "CRIU operation: %d\n", args->op);
		2559	switch (args->op) {
		2560	case KFD_CRIU_OP_PROCESS_INFO:
		2561	ret = criu_process_info(filep, p, args);
		2562	break;
		2563	case KFD_CRIU_OP_CHECKPOINT:
		2564	ret = criu_checkpoint(filep, p, args);
		2565	break;
		2566	case KFD_CRIU_OP_UNPAUSE:
		2567	ret = criu_unpause(filep, p, args);
		2568	break;
		2569	case KFD_CRIU_OP_RESTORE:
		2570	ret = criu_restore(filep, p, args);
		2571	break;
		2572	case KFD_CRIU_OP_RESUME:
		2573	ret = criu_resume(filep, p, args);
		2574	break;
		2575	default:
		2576	dev_dbg(kfd_device, "Unsupported CRIU operation:%d\n", args->op);
		2577	ret = -EINVAL;
		2578	break;
		2579	}
		2580
		2581	if (ret)
		2582	dev_dbg(kfd_device, "CRIU operation:%d err:%d\n", args->op, ret);
		2583
		2584	return ret;
		2585	}
		2586
		2587	#define AMDKFD_IOCTL_DEF(ioctl, _func, _flags) \
		2588	[_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func, .flags = _flags, \
		2589	.cmd_drv = 0, .name = #ioctl}
		2590
		2591	/** Ioctl table */
		2592	static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = {
1865	AMDKFD_IOCTL_DEF(AMDKFD_IOC_GET_VERSION,	2593	AMDKFD_IOCTL_DEF(AMDKFD_IOC_GET_VERSION,
1866	kfd_ioctl_get_version, 0),	2594	kfd_ioctl_get_version, 0),
1867		2595
@@ -1898,16 +2626,16 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { · 30 unmodified lines
1898	AMDKFD_IOCTL_DEF(AMDKFD_IOC_WAIT_EVENTS,	2626	AMDKFD_IOCTL_DEF(AMDKFD_IOC_WAIT_EVENTS,
1899	kfd_ioctl_wait_events, 0),	2627	kfd_ioctl_wait_events, 0),
1900		2628
1901	AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_REGISTER,	2629	AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_REGISTER_DEPRECATED,
1902	kfd_ioctl_dbg_register, 0),	2630	kfd_ioctl_dbg_register, 0),
1903		2631
1904	AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_UNREGISTER,	2632	AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_UNREGISTER_DEPRECATED,
1905	kfd_ioctl_dbg_unregister, 0),	2633	kfd_ioctl_dbg_unregister, 0),
1906		2634
1907	AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_ADDRESS_WATCH,	2635	AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_ADDRESS_WATCH_DEPRECATED,
1908	kfd_ioctl_dbg_address_watch, 0),	2636	kfd_ioctl_dbg_address_watch, 0),
1909		2637
1910	AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_WAVE_CONTROL,	2638	AMDKFD_IOCTL_DEF(AMDKFD_IOC_DBG_WAVE_CONTROL_DEPRECATED,
1911	kfd_ioctl_dbg_wave_control, 0),	2639	kfd_ioctl_dbg_wave_control, 0),
1912		2640
1913	AMDKFD_IOCTL_DEF(AMDKFD_IOC_SET_SCRATCH_BACKING_VA,	2641	AMDKFD_IOCTL_DEF(AMDKFD_IOC_SET_SCRATCH_BACKING_VA,
@@ -1959,6 +2687,10 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { · 45 unmodified lines
1959		2687
1960	AMDKFD_IOCTL_DEF(AMDKFD_IOC_SET_XNACK_MODE,	2688	AMDKFD_IOCTL_DEF(AMDKFD_IOC_SET_XNACK_MODE,
1961	kfd_ioctl_set_xnack_mode, 0),	2689	kfd_ioctl_set_xnack_mode, 0),
		2690
		2691	AMDKFD_IOCTL_DEF(AMDKFD_IOC_CRIU_OP,
		2692	kfd_ioctl_criu, KFD_IOC_FLAG_CHECKPOINT_RESTORE),
		2693
1962	};	2694	};
1963		2695
1964	#define AMDKFD_CORE_IOCTL_COUNT ARRAY_SIZE(amdkfd_ioctls)	2696	#define AMDKFD_CORE_IOCTL_COUNT ARRAY_SIZE(amdkfd_ioctls)
@@ -1973,6 +2705,7 @@ static long kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) · 8 unmodified lines
1973	char *kdata = NULL;	2705	char *kdata = NULL;
1974	unsigned int usize, asize;	2706	unsigned int usize, asize;
1975	int retcode = -EINVAL;	2707	int retcode = -EINVAL;
		2708	bool ptrace_attached = false;
1976		2709
1977	if (nr >= AMDKFD_CORE_IOCTL_COUNT)	2710	if (nr >= AMDKFD_CORE_IOCTL_COUNT)
1978	goto err_i1;	2711	goto err_i1;
@@ -1998,7 +2731,15 @@ static long kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) · 19 unmodified lines
1998	* processes need to create their own KFD device context.	2731	* processes need to create their own KFD device context.
1999	*/	2732	*/
2000	process = filep->private_data;	2733	process = filep->private_data;
2001	if (process->lead_thread != current->group_leader) {	2734
		2735	rcu_read_lock();
		2736	if ((ioctl->flags & KFD_IOC_FLAG_CHECKPOINT_RESTORE) &&
		2737	ptrace_parent(process->lead_thread) == current)
		2738	ptrace_attached = true;
		2739	rcu_read_unlock();
		2740
		2741	if (process->lead_thread != current->group_leader
		2742	&& !ptrace_attached) {
2002	dev_dbg(kfd_device, "Using KFD FD in wrong process\n");	2743	dev_dbg(kfd_device, "Using KFD FD in wrong process\n");
2003	retcode = -EBADF;	2744	retcode = -EBADF;
2004	goto err_i1;	2745	goto err_i1;
@@ -2013,6 +2754,19 @@ static long kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) · 8 unmodified lines
2013	goto err_i1;	2754	goto err_i1;
2014	}	2755	}
2015		2756
		2757	/*
		2758	* Versions of docker shipped in Ubuntu 18.xx and 20.xx do not support
		2759	* CAP_CHECKPOINT_RESTORE, so we also allow access if CAP_SYS_ADMIN as CAP_SYS_ADMIN is a
		2760	* more priviledged access.
		2761	*/
		2762	if (unlikely(ioctl->flags & KFD_IOC_FLAG_CHECKPOINT_RESTORE)) {
		2763	if (!capable(CAP_CHECKPOINT_RESTORE) &&
		2764	!capable(CAP_SYS_ADMIN)) {
		2765	retcode = -EACCES;
		2766	goto err_i1;
		2767	}
		2768	}
		2769
2016	if (cmd & (IOC_IN \| IOC_OUT)) {	2770	if (cmd & (IOC_IN \| IOC_OUT)) {
2017	if (asize <= sizeof(stack_kdata)) {	2771	if (asize <= sizeof(stack_kdata)) {
2018	kdata = stack_kdata;	2772	kdata = stack_kdata;

drivers/gpu/drm/amd/amdkfd/kfd_crat.c+4−2

+// SPDX-License-Identifier: GPL-2.0 OR MIT
 /*
- * Copyright 2015-2017 Advanced Micro Devices, Inc.
+ * Copyright 2015-2022 Advanced Micro Devices, Inc.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
  * copy of this software and associated documentation files (the "Software"),
 		num_of_cache_types = ARRAY_SIZE(vegam_cache_info);
 		break;
 	default:
-		switch(KFD_GC_VERSION(kdev)) {
+		switch (KFD_GC_VERSION(kdev)) {
 		case IP_VERSION(9, 0, 1):
 			pcache_info = vega10_cache_info;
 			num_of_cache_types = ARRAY_SIZE(vega10_cache_info);
 		case IP_VERSION(10, 1, 10):
 		case IP_VERSION(10, 1, 2):
 		case IP_VERSION(10, 1, 3):
+		case IP_VERSION(10, 1, 4):
 			pcache_info = navi10_cache_info;
 			num_of_cache_types = ARRAY_SIZE(navi10_cache_info);
 			break;

drivers/gpu/drm/amd/amdkfd/kfd_crat.h+3−2

+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
 /*
- * Copyright 2014 Advanced Micro Devices, Inc.
+ * Copyright 2014-2022 Advanced Micro Devices, Inc.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
  * copy of this software and associated documentation files (the "Software"),
 #define CRAT_IOLINK_FLAGS_NO_ATOMICS_32_BIT	(1 << 2)
 #define CRAT_IOLINK_FLAGS_NO_ATOMICS_64_BIT	(1 << 3)
 #define CRAT_IOLINK_FLAGS_NO_PEER_TO_PEER_DMA	(1 << 4)
-#define CRAT_IOLINK_FLAGS_BI_DIRECTIONAL 	(1 << 31)
+#define CRAT_IOLINK_FLAGS_BI_DIRECTIONAL	(1 << 31)
 #define CRAT_IOLINK_FLAGS_RESERVED_MASK		0x7fffffe0
 /*

drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c+0−845

@@ -1,845 +0,0 @@
1	/*
2	* Copyright 2014 Advanced Micro Devices, Inc.
3	*
4	* Permission is hereby granted, free of charge, to any person obtaining a
5	* copy of this software and associated documentation files (the "Software"),
6	* to deal in the Software without restriction, including without limitation
7	* the rights to use, copy, modify, merge, publish, distribute, sublicense,
8	* and/or sell copies of the Software, and to permit persons to whom the
9	* Software is furnished to do so, subject to the following conditions:
10	*
11	* The above copyright notice and this permission notice shall be included in
12	* all copies or substantial portions of the Software.
13	*
14	* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15	* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16	* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
17	* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
18	* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
19	* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
20	* OTHER DEALINGS IN THE SOFTWARE.
21	*
22	*/
23
24	#include <linux/types.h>
25	#include <linux/kernel.h>
26	#include <linux/log2.h>
27	#include <linux/sched.h>
28	#include <linux/slab.h>
29	#include <linux/mutex.h>
30	#include <linux/device.h>
31
32	#include "kfd_pm4_headers.h"
33	#include "kfd_pm4_headers_diq.h"
34	#include "kfd_kernel_queue.h"
35	#include "kfd_priv.h"
36	#include "kfd_pm4_opcodes.h"
37	#include "cik_regs.h"
38	#include "kfd_dbgmgr.h"
39	#include "kfd_dbgdev.h"
40	#include "kfd_device_queue_manager.h"
41
42	static void dbgdev_address_watch_disable_nodiq(struct kfd_dev *dev)
43	{
44	dev->kfd2kgd->address_watch_disable(dev->adev);
45	}
46
47	static int dbgdev_diq_submit_ib(struct kfd_dbgdev *dbgdev,
48	u32 pasid, uint64_t vmid0_address,
49	uint32_t *packet_buff, size_t size_in_bytes)
50	{
51	struct pm4__release_mem *rm_packet;
52	struct pm4__indirect_buffer_pasid *ib_packet;
53	struct kfd_mem_obj *mem_obj;
54	size_t pq_packets_size_in_bytes;
55	union ULARGE_INTEGER *largep;
56	union ULARGE_INTEGER addr;
57	struct kernel_queue *kq;
58	uint64_t *rm_state;
59	unsigned int *ib_packet_buff;
60	int status;
61
62	if (WARN_ON(!size_in_bytes))
63	return -EINVAL;
64
65	kq = dbgdev->kq;
66
67	pq_packets_size_in_bytes = sizeof(struct pm4__release_mem) +
68	sizeof(struct pm4__indirect_buffer_pasid);
69
70	/*
71	* We acquire a buffer from DIQ
72	* The receive packet buff will be sitting on the Indirect Buffer
73	* and in the PQ we put the IB packet + sync packet(s).
74	*/
75	status = kq_acquire_packet_buffer(kq,
76	pq_packets_size_in_bytes / sizeof(uint32_t),
77	&ib_packet_buff);
78	if (status) {
79	pr_err("kq_acquire_packet_buffer failed\n");
80	return status;
81	}
82
83	memset(ib_packet_buff, 0, pq_packets_size_in_bytes);
84
85	ib_packet = (struct pm4__indirect_buffer_pasid *) (ib_packet_buff);
86
87	ib_packet->header.count = 3;
88	ib_packet->header.opcode = IT_INDIRECT_BUFFER_PASID;
89	ib_packet->header.type = PM4_TYPE_3;
90
91	largep = (union ULARGE_INTEGER *) &vmid0_address;
92
93	ib_packet->bitfields2.ib_base_lo = largep->u.low_part >> 2;
94	ib_packet->bitfields3.ib_base_hi = largep->u.high_part;
95
96	ib_packet->control = (1 << 23) \| (1 << 31) \|
97	((size_in_bytes / 4) & 0xfffff);
98
99	ib_packet->bitfields5.pasid = pasid;
100
101	/*
102	* for now we use release mem for GPU-CPU synchronization
103	* Consider WaitRegMem + WriteData as a better alternative
104	* we get a GART allocations ( gpu/cpu mapping),
105	* for the sync variable, and wait until:
106	* (a) Sync with HW
107	* (b) Sync var is written by CP to mem.
108	*/
109	rm_packet = (struct pm4__release_mem *) (ib_packet_buff +
110	(sizeof(struct pm4__indirect_buffer_pasid) /
111	sizeof(unsigned int)));
112
113	status = kfd_gtt_sa_allocate(dbgdev->dev, sizeof(uint64_t),
114	&mem_obj);
115
116	if (status) {
117	pr_err("Failed to allocate GART memory\n");
118	kq_rollback_packet(kq);
119	return status;
120	}
121
122	rm_state = (uint64_t *) mem_obj->cpu_ptr;
123
124	*rm_state = QUEUESTATE__ACTIVE_COMPLETION_PENDING;
125
126	rm_packet->header.opcode = IT_RELEASE_MEM;
127	rm_packet->header.type = PM4_TYPE_3;
128	rm_packet->header.count = sizeof(struct pm4__release_mem) / 4 - 2;
129
130	rm_packet->bitfields2.event_type = CACHE_FLUSH_AND_INV_TS_EVENT;
131	rm_packet->bitfields2.event_index =
132	event_index___release_mem__end_of_pipe;
133
134	rm_packet->bitfields2.cache_policy = cache_policy___release_mem__lru;
135	rm_packet->bitfields2.atc = 0;
136	rm_packet->bitfields2.tc_wb_action_ena = 1;
137
138	addr.quad_part = mem_obj->gpu_addr;
139
140	rm_packet->bitfields4.address_lo_32b = addr.u.low_part >> 2;
141	rm_packet->address_hi = addr.u.high_part;
142
143	rm_packet->bitfields3.data_sel =
144	data_sel___release_mem__send_64_bit_data;
145
146	rm_packet->bitfields3.int_sel =
147	int_sel___release_mem__send_data_after_write_confirm;
148
149	rm_packet->bitfields3.dst_sel =
150	dst_sel___release_mem__memory_controller;
151
152	rm_packet->data_lo = QUEUESTATE__ACTIVE;
153
154	kq_submit_packet(kq);
155
156	/* Wait till CP writes sync code: */
157	status = amdkfd_fence_wait_timeout(
158	rm_state,
159	QUEUESTATE__ACTIVE, 1500);
160
161	kfd_gtt_sa_free(dbgdev->dev, mem_obj);
162
163	return status;
164	}
165
166	static int dbgdev_register_nodiq(struct kfd_dbgdev *dbgdev)
167	{
168	/*
169	* no action is needed in this case,
170	* just make sure diq will not be used
171	*/
172
173	dbgdev->kq = NULL;
174
175	return 0;
176	}
177
178	static int dbgdev_register_diq(struct kfd_dbgdev *dbgdev)
179	{
180	struct queue_properties properties;
181	unsigned int qid;
182	struct kernel_queue *kq = NULL;
183	int status;
184
185	properties.type = KFD_QUEUE_TYPE_DIQ;
186
187	status = pqm_create_queue(dbgdev->pqm, dbgdev->dev, NULL,
188	&properties, &qid, NULL);
189
190	if (status) {
191	pr_err("Failed to create DIQ\n");
192	return status;
193	}
194
195	pr_debug("DIQ Created with queue id: %d\n", qid);
196
197	kq = pqm_get_kernel_queue(dbgdev->pqm, qid);
198
199	if (!kq) {
200	pr_err("Error getting DIQ\n");
201	pqm_destroy_queue(dbgdev->pqm, qid);
202	return -EFAULT;
203	}
204
205	dbgdev->kq = kq;
206
207	return status;
208	}
209
210	static int dbgdev_unregister_nodiq(struct kfd_dbgdev *dbgdev)
211	{
212	/* disable watch address */
213	dbgdev_address_watch_disable_nodiq(dbgdev->dev);
214	return 0;
215	}
216
217	static int dbgdev_unregister_diq(struct kfd_dbgdev *dbgdev)
218	{
219	/* todo - disable address watch */
220	int status;
221
222	status = pqm_destroy_queue(dbgdev->pqm,
223	dbgdev->kq->queue->properties.queue_id);
224	dbgdev->kq = NULL;
225
226	return status;
227	}
228
229	static void dbgdev_address_watch_set_registers(
230	const struct dbg_address_watch_info *adw_info,
231	union TCP_WATCH_ADDR_H_BITS *addrHi,
232	union TCP_WATCH_ADDR_L_BITS *addrLo,
233	union TCP_WATCH_CNTL_BITS *cntl,
234	unsigned int index, unsigned int vmid)
235	{
236	union ULARGE_INTEGER addr;
237
238	addr.quad_part = 0;
239	addrHi->u32All = 0;
240	addrLo->u32All = 0;
241	cntl->u32All = 0;
242
243	if (adw_info->watch_mask)
244	cntl->bitfields.mask =
245	(uint32_t) (adw_info->watch_mask[index] &
246	ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK);
247	else
248	cntl->bitfields.mask = ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK;
249
250	addr.quad_part = (unsigned long long) adw_info->watch_address[index];
251
252	addrHi->bitfields.addr = addr.u.high_part &
253	ADDRESS_WATCH_REG_ADDHIGH_MASK;
254	addrLo->bitfields.addr =
255	(addr.u.low_part >> ADDRESS_WATCH_REG_ADDLOW_SHIFT);
256
257	cntl->bitfields.mode = adw_info->watch_mode[index];
258	cntl->bitfields.vmid = (uint32_t) vmid;
259	/* for now assume it is an ATC address */
260	cntl->u32All \|= ADDRESS_WATCH_REG_CNTL_ATC_BIT;
261
262	pr_debug("\t\t%20s %08x\n", "set reg mask :", cntl->bitfields.mask);
263	pr_debug("\t\t%20s %08x\n", "set reg add high :",
264	addrHi->bitfields.addr);
265	pr_debug("\t\t%20s %08x\n", "set reg add low :",
266	addrLo->bitfields.addr);
267	}
268
269	static int dbgdev_address_watch_nodiq(struct kfd_dbgdev *dbgdev,
270	struct dbg_address_watch_info *adw_info)
271	{
272	union TCP_WATCH_ADDR_H_BITS addrHi;
273	union TCP_WATCH_ADDR_L_BITS addrLo;
274	union TCP_WATCH_CNTL_BITS cntl;
275	struct kfd_process_device *pdd;
276	unsigned int i;
277
278	/* taking the vmid for that process on the safe way using pdd */
279	pdd = kfd_get_process_device_data(dbgdev->dev,
280	adw_info->process);
281	if (!pdd) {
282	pr_err("Failed to get pdd for wave control no DIQ\n");
283	return -EFAULT;
284	}
285
286	addrHi.u32All = 0;
287	addrLo.u32All = 0;
288	cntl.u32All = 0;
289
290	if ((adw_info->num_watch_points > MAX_WATCH_ADDRESSES) \|\|
291	(adw_info->num_watch_points == 0)) {
292	pr_err("num_watch_points is invalid\n");
293	return -EINVAL;
294	}
295
296	if (!adw_info->watch_mode \|\| !adw_info->watch_address) {
297	pr_err("adw_info fields are not valid\n");
298	return -EINVAL;
299	}
300
301	for (i = 0; i < adw_info->num_watch_points; i++) {
302	dbgdev_address_watch_set_registers(adw_info, &addrHi, &addrLo,
303	&cntl, i, pdd->qpd.vmid);
304
305	pr_debug("\t\t%30s\n", "* * * * * * * * * * * * * * * * * *");
306	pr_debug("\t\t%20s %08x\n", "register index :", i);
307	pr_debug("\t\t%20s %08x\n", "vmid is :", pdd->qpd.vmid);
308	pr_debug("\t\t%20s %08x\n", "Address Low is :",
309	addrLo.bitfields.addr);
310	pr_debug("\t\t%20s %08x\n", "Address high is :",
311	addrHi.bitfields.addr);
312	pr_debug("\t\t%20s %08x\n", "Address high is :",
313	addrHi.bitfields.addr);
314	pr_debug("\t\t%20s %08x\n", "Control Mask is :",
315	cntl.bitfields.mask);
316	pr_debug("\t\t%20s %08x\n", "Control Mode is :",
317	cntl.bitfields.mode);
318	pr_debug("\t\t%20s %08x\n", "Control Vmid is :",
319	cntl.bitfields.vmid);
320	pr_debug("\t\t%20s %08x\n", "Control atc is :",
321	cntl.bitfields.atc);
322	pr_debug("\t\t%30s\n", "* * * * * * * * * * * * * * * * * *");
323
324	pdd->dev->kfd2kgd->address_watch_execute(
325	dbgdev->dev->adev,
326	i,
327	cntl.u32All,
328	addrHi.u32All,
329	addrLo.u32All);
330	}
331
332	return 0;
333	}
334
335	static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev,
336	struct dbg_address_watch_info *adw_info)
337	{
338	struct pm4__set_config_reg *packets_vec;
339	union TCP_WATCH_ADDR_H_BITS addrHi;
340	union TCP_WATCH_ADDR_L_BITS addrLo;
341	union TCP_WATCH_CNTL_BITS cntl;
342	struct kfd_mem_obj *mem_obj;
343	unsigned int aw_reg_add_dword;
344	uint32_t *packet_buff_uint;
345	unsigned int i;
346	int status;
347	size_t ib_size = sizeof(struct pm4__set_config_reg) * 4;
348	/* we do not control the vmid in DIQ mode, just a place holder */
349	unsigned int vmid = 0;
350
351	addrHi.u32All = 0;
352	addrLo.u32All = 0;
353	cntl.u32All = 0;
354
355	if ((adw_info->num_watch_points > MAX_WATCH_ADDRESSES) \|\|
356	(adw_info->num_watch_points == 0)) {
357	pr_err("num_watch_points is invalid\n");
358	return -EINVAL;
359	}
360
361	if (!adw_info->watch_mode \|\| !adw_info->watch_address) {
362	pr_err("adw_info fields are not valid\n");
363	return -EINVAL;
364	}
365
366	status = kfd_gtt_sa_allocate(dbgdev->dev, ib_size, &mem_obj);
367
368	if (status) {
369	pr_err("Failed to allocate GART memory\n");
370	return status;
371	}
372
373	packet_buff_uint = mem_obj->cpu_ptr;
374
375	memset(packet_buff_uint, 0, ib_size);
376
377	packets_vec = (struct pm4__set_config_reg *) (packet_buff_uint);
378
379	packets_vec[0].header.count = 1;
380	packets_vec[0].header.opcode = IT_SET_CONFIG_REG;
381	packets_vec[0].header.type = PM4_TYPE_3;
382	packets_vec[0].bitfields2.vmid_shift = ADDRESS_WATCH_CNTL_OFFSET;
383	packets_vec[0].bitfields2.insert_vmid = 1;
384	packets_vec[1].ordinal1 = packets_vec[0].ordinal1;
385	packets_vec[1].bitfields2.insert_vmid = 0;
386	packets_vec[2].ordinal1 = packets_vec[0].ordinal1;
387	packets_vec[2].bitfields2.insert_vmid = 0;
388	packets_vec[3].ordinal1 = packets_vec[0].ordinal1;
389	packets_vec[3].bitfields2.vmid_shift = ADDRESS_WATCH_CNTL_OFFSET;
390	packets_vec[3].bitfields2.insert_vmid = 1;
391
392	for (i = 0; i < adw_info->num_watch_points; i++) {
393	dbgdev_address_watch_set_registers(adw_info,
394	&addrHi,
395	&addrLo,
396	&cntl,
397	i,
398	vmid);
399
400	pr_debug("\t\t%30s\n", "* * * * * * * * * * * * * * * * * *");
401	pr_debug("\t\t%20s %08x\n", "register index :", i);
402	pr_debug("\t\t%20s %08x\n", "vmid is :", vmid);
403	pr_debug("\t\t%20s %p\n", "Add ptr is :",
404	adw_info->watch_address);
405	pr_debug("\t\t%20s %08llx\n", "Add is :",
406	adw_info->watch_address[i]);
407	pr_debug("\t\t%20s %08x\n", "Address Low is :",
408	addrLo.bitfields.addr);
409	pr_debug("\t\t%20s %08x\n", "Address high is :",
410	addrHi.bitfields.addr);
411	pr_debug("\t\t%20s %08x\n", "Control Mask is :",
412	cntl.bitfields.mask);
413	pr_debug("\t\t%20s %08x\n", "Control Mode is :",
414	cntl.bitfields.mode);
415	pr_debug("\t\t%20s %08x\n", "Control Vmid is :",
416	cntl.bitfields.vmid);
417	pr_debug("\t\t%20s %08x\n", "Control atc is :",
418	cntl.bitfields.atc);
419	pr_debug("\t\t%30s\n", "* * * * * * * * * * * * * * * * * *");
420
421	aw_reg_add_dword =
422	dbgdev->dev->kfd2kgd->address_watch_get_offset(
423	dbgdev->dev->adev,
424	i,
425	ADDRESS_WATCH_REG_CNTL);
426
427	packets_vec[0].bitfields2.reg_offset =
428	aw_reg_add_dword - AMD_CONFIG_REG_BASE;
429
430	packets_vec[0].reg_data[0] = cntl.u32All;
431
432	aw_reg_add_dword =
433	dbgdev->dev->kfd2kgd->address_watch_get_offset(
434	dbgdev->dev->adev,
435	i,
436	ADDRESS_WATCH_REG_ADDR_HI);
437
438	packets_vec[1].bitfields2.reg_offset =
439	aw_reg_add_dword - AMD_CONFIG_REG_BASE;
440	packets_vec[1].reg_data[0] = addrHi.u32All;
441
442	aw_reg_add_dword =
443	dbgdev->dev->kfd2kgd->address_watch_get_offset(
444	dbgdev->dev->adev,
445	i,
446	ADDRESS_WATCH_REG_ADDR_LO);
447
448	packets_vec[2].bitfields2.reg_offset =
449	aw_reg_add_dword - AMD_CONFIG_REG_BASE;
450	packets_vec[2].reg_data[0] = addrLo.u32All;
451
452	/* enable watch flag if address is not zero*/
453	if (adw_info->watch_address[i] > 0)
454	cntl.bitfields.valid = 1;
455	else
456	cntl.bitfields.valid = 0;
457
458	aw_reg_add_dword =
459	dbgdev->dev->kfd2kgd->address_watch_get_offset(
460	dbgdev->dev->adev,
461	i,
462	ADDRESS_WATCH_REG_CNTL);
463
464	packets_vec[3].bitfields2.reg_offset =
465	aw_reg_add_dword - AMD_CONFIG_REG_BASE;
466	packets_vec[3].reg_data[0] = cntl.u32All;
467
468	status = dbgdev_diq_submit_ib(
469	dbgdev,
470	adw_info->process->pasid,
471	mem_obj->gpu_addr,
472	packet_buff_uint,
473	ib_size);
474
475	if (status) {
476	pr_err("Failed to submit IB to DIQ\n");
477	break;
478	}
479	}
480
481	kfd_gtt_sa_free(dbgdev->dev, mem_obj);
482	return status;
483	}
484
485	static int dbgdev_wave_control_set_registers(
486	struct dbg_wave_control_info *wac_info,
487	union SQ_CMD_BITS *in_reg_sq_cmd,
488	union GRBM_GFX_INDEX_BITS *in_reg_gfx_index)
489	{
490	int status = 0;
491	union SQ_CMD_BITS reg_sq_cmd;
492	union GRBM_GFX_INDEX_BITS reg_gfx_index;
493	struct HsaDbgWaveMsgAMDGen2 *pMsg;
494
495	reg_sq_cmd.u32All = 0;
496	reg_gfx_index.u32All = 0;
497	pMsg = &wac_info->dbgWave_msg.DbgWaveMsg.WaveMsgInfoGen2;
498
499	switch (wac_info->mode) {
500	/* Send command to single wave */
501	case HSA_DBG_WAVEMODE_SINGLE:
502	/*
503	* Limit access to the process waves only,
504	* by setting vmid check
505	*/
506	reg_sq_cmd.bits.check_vmid = 1;
507	reg_sq_cmd.bits.simd_id = pMsg->ui32.SIMD;
508	reg_sq_cmd.bits.wave_id = pMsg->ui32.WaveId;
509	reg_sq_cmd.bits.mode = SQ_IND_CMD_MODE_SINGLE;
510
511	reg_gfx_index.bits.sh_index = pMsg->ui32.ShaderArray;
512	reg_gfx_index.bits.se_index = pMsg->ui32.ShaderEngine;
513	reg_gfx_index.bits.instance_index = pMsg->ui32.HSACU;
514
515	break;
516
517	/* Send command to all waves with matching VMID */
518	case HSA_DBG_WAVEMODE_BROADCAST_PROCESS:
519
520	reg_gfx_index.bits.sh_broadcast_writes = 1;
521	reg_gfx_index.bits.se_broadcast_writes = 1;
522	reg_gfx_index.bits.instance_broadcast_writes = 1;
523
524	reg_sq_cmd.bits.mode = SQ_IND_CMD_MODE_BROADCAST;
525
526	break;
527
528	/* Send command to all CU waves with matching VMID */
529	case HSA_DBG_WAVEMODE_BROADCAST_PROCESS_CU:
530
531	reg_sq_cmd.bits.check_vmid = 1;
532	reg_sq_cmd.bits.mode = SQ_IND_CMD_MODE_BROADCAST;
533
534	reg_gfx_index.bits.sh_index = pMsg->ui32.ShaderArray;
535	reg_gfx_index.bits.se_index = pMsg->ui32.ShaderEngine;
536	reg_gfx_index.bits.instance_index = pMsg->ui32.HSACU;
537
538	break;
539
540	default:
541	return -EINVAL;
542	}
543
544	switch (wac_info->operand) {
545	case HSA_DBG_WAVEOP_HALT:
546	reg_sq_cmd.bits.cmd = SQ_IND_CMD_CMD_HALT;
547	break;
548
549	case HSA_DBG_WAVEOP_RESUME:
550	reg_sq_cmd.bits.cmd = SQ_IND_CMD_CMD_RESUME;
551	break;
552
553	case HSA_DBG_WAVEOP_KILL:
554	reg_sq_cmd.bits.cmd = SQ_IND_CMD_CMD_KILL;
555	break;
556
557	case HSA_DBG_WAVEOP_DEBUG:
558	reg_sq_cmd.bits.cmd = SQ_IND_CMD_CMD_DEBUG;
559	break;
560
561	case HSA_DBG_WAVEOP_TRAP:
562	if (wac_info->trapId < MAX_TRAPID) {
563	reg_sq_cmd.bits.cmd = SQ_IND_CMD_CMD_TRAP;
564	reg_sq_cmd.bits.trap_id = wac_info->trapId;
565	} else {
566	status = -EINVAL;
567	}
568	break;
569
570	default:
571	status = -EINVAL;
572	break;
573	}
574
575	if (status == 0) {
576	*in_reg_sq_cmd = reg_sq_cmd;
577	*in_reg_gfx_index = reg_gfx_index;
578	}
579
580	return status;
581	}
582
583	static int dbgdev_wave_control_diq(struct kfd_dbgdev *dbgdev,
584	struct dbg_wave_control_info *wac_info)
585	{
586
587	int status;
588	union SQ_CMD_BITS reg_sq_cmd;
589	union GRBM_GFX_INDEX_BITS reg_gfx_index;
590	struct kfd_mem_obj *mem_obj;
591	uint32_t *packet_buff_uint;
592	struct pm4__set_config_reg *packets_vec;
593	size_t ib_size = sizeof(struct pm4__set_config_reg) * 3;
594
595	reg_sq_cmd.u32All = 0;
596
597	status = dbgdev_wave_control_set_registers(wac_info, &reg_sq_cmd,
598	&reg_gfx_index);
599	if (status) {
600	pr_err("Failed to set wave control registers\n");
601	return status;
602	}
603
604	/* we do not control the VMID in DIQ, so reset it to a known value */
605	reg_sq_cmd.bits.vm_id = 0;
606
607	pr_debug("\t\t %30s\n", "* * * * * * * * * * * * * * * * * *");
608
609	pr_debug("\t\t mode is: %u\n", wac_info->mode);
610	pr_debug("\t\t operand is: %u\n", wac_info->operand);
611	pr_debug("\t\t trap id is: %u\n", wac_info->trapId);
612	pr_debug("\t\t msg value is: %u\n",
613	wac_info->dbgWave_msg.DbgWaveMsg.WaveMsgInfoGen2.Value);
614	pr_debug("\t\t vmid is: N/A\n");
615
616	pr_debug("\t\t chk_vmid is : %u\n", reg_sq_cmd.bitfields.check_vmid);
617	pr_debug("\t\t command is : %u\n", reg_sq_cmd.bitfields.cmd);
618	pr_debug("\t\t queue id is : %u\n", reg_sq_cmd.bitfields.queue_id);
619	pr_debug("\t\t simd id is : %u\n", reg_sq_cmd.bitfields.simd_id);
620	pr_debug("\t\t mode is : %u\n", reg_sq_cmd.bitfields.mode);
621	pr_debug("\t\t vm_id is : %u\n", reg_sq_cmd.bitfields.vm_id);
622	pr_debug("\t\t wave_id is : %u\n", reg_sq_cmd.bitfields.wave_id);
623
624	pr_debug("\t\t ibw is : %u\n",
625	reg_gfx_index.bitfields.instance_broadcast_writes);
626	pr_debug("\t\t ii is : %u\n",
627	reg_gfx_index.bitfields.instance_index);
628	pr_debug("\t\t sebw is : %u\n",
629	reg_gfx_index.bitfields.se_broadcast_writes);
630	pr_debug("\t\t se_ind is : %u\n", reg_gfx_index.bitfields.se_index);
631	pr_debug("\t\t sh_ind is : %u\n", reg_gfx_index.bitfields.sh_index);
632	pr_debug("\t\t sbw is : %u\n",
633	reg_gfx_index.bitfields.sh_broadcast_writes);
634
635	pr_debug("\t\t %30s\n", "* * * * * * * * * * * * * * * * * *");
636
637	status = kfd_gtt_sa_allocate(dbgdev->dev, ib_size, &mem_obj);
638
639	if (status != 0) {
640	pr_err("Failed to allocate GART memory\n");
641	return status;
642	}
643
644	packet_buff_uint = mem_obj->cpu_ptr;
645
646	memset(packet_buff_uint, 0, ib_size);
647
648	packets_vec = (struct pm4__set_config_reg *) packet_buff_uint;
649	packets_vec[0].header.count = 1;
650	packets_vec[0].header.opcode = IT_SET_UCONFIG_REG;
651	packets_vec[0].header.type = PM4_TYPE_3;
652	packets_vec[0].bitfields2.reg_offset =
653	GRBM_GFX_INDEX / 4 - USERCONFIG_REG_BASE;
654
655	packets_vec[0].bitfields2.insert_vmid = 0;
656	packets_vec[0].reg_data[0] = reg_gfx_index.u32All;
657
658	packets_vec[1].header.count = 1;
659	packets_vec[1].header.opcode = IT_SET_CONFIG_REG;
660	packets_vec[1].header.type = PM4_TYPE_3;
661	packets_vec[1].bitfields2.reg_offset = SQ_CMD / 4 - AMD_CONFIG_REG_BASE;
662
663	packets_vec[1].bitfields2.vmid_shift = SQ_CMD_VMID_OFFSET;
664	packets_vec[1].bitfields2.insert_vmid = 1;
665	packets_vec[1].reg_data[0] = reg_sq_cmd.u32All;
666
667	/* Restore the GRBM_GFX_INDEX register */
668
669	reg_gfx_index.u32All = 0;
670	reg_gfx_index.bits.sh_broadcast_writes = 1;
671	reg_gfx_index.bits.instance_broadcast_writes = 1;
672	reg_gfx_index.bits.se_broadcast_writes = 1;
673
674
675	packets_vec[2].ordinal1 = packets_vec[0].ordinal1;
676	packets_vec[2].bitfields2.reg_offset =
677	GRBM_GFX_INDEX / 4 - USERCONFIG_REG_BASE;
678
679	packets_vec[2].bitfields2.insert_vmid = 0;
680	packets_vec[2].reg_data[0] = reg_gfx_index.u32All;
681
682	status = dbgdev_diq_submit_ib(
683	dbgdev,
684	wac_info->process->pasid,
685	mem_obj->gpu_addr,
686	packet_buff_uint,
687	ib_size);
688
689	if (status)
690	pr_err("Failed to submit IB to DIQ\n");
691
692	kfd_gtt_sa_free(dbgdev->dev, mem_obj);
693
694	return status;
695	}
696
697	static int dbgdev_wave_control_nodiq(struct kfd_dbgdev *dbgdev,
698	struct dbg_wave_control_info *wac_info)
699	{
700	int status;
701	union SQ_CMD_BITS reg_sq_cmd;
702	union GRBM_GFX_INDEX_BITS reg_gfx_index;
703	struct kfd_process_device *pdd;
704
705	reg_sq_cmd.u32All = 0;
706
707	/* taking the VMID for that process on the safe way using PDD */
708	pdd = kfd_get_process_device_data(dbgdev->dev, wac_info->process);
709
710	if (!pdd) {
711	pr_err("Failed to get pdd for wave control no DIQ\n");
712	return -EFAULT;
713	}
714	status = dbgdev_wave_control_set_registers(wac_info, &reg_sq_cmd,
715	&reg_gfx_index);
716	if (status) {
717	pr_err("Failed to set wave control registers\n");
718	return status;
719	}
720
721	/* for non DIQ we need to patch the VMID: */
722
723	reg_sq_cmd.bits.vm_id = pdd->qpd.vmid;
724
725	pr_debug("\t\t %30s\n", "* * * * * * * * * * * * * * * * * *");
726
727	pr_debug("\t\t mode is: %u\n", wac_info->mode);
728	pr_debug("\t\t operand is: %u\n", wac_info->operand);
729	pr_debug("\t\t trap id is: %u\n", wac_info->trapId);
730	pr_debug("\t\t msg value is: %u\n",
731	wac_info->dbgWave_msg.DbgWaveMsg.WaveMsgInfoGen2.Value);
732	pr_debug("\t\t vmid is: %u\n", pdd->qpd.vmid);
733
734	pr_debug("\t\t chk_vmid is : %u\n", reg_sq_cmd.bitfields.check_vmid);
735	pr_debug("\t\t command is : %u\n", reg_sq_cmd.bitfields.cmd);
736	pr_debug("\t\t queue id is : %u\n", reg_sq_cmd.bitfields.queue_id);
737	pr_debug("\t\t simd id is : %u\n", reg_sq_cmd.bitfields.simd_id);
738	pr_debug("\t\t mode is : %u\n", reg_sq_cmd.bitfields.mode);
739	pr_debug("\t\t vm_id is : %u\n", reg_sq_cmd.bitfields.vm_id);
740	pr_debug("\t\t wave_id is : %u\n", reg_sq_cmd.bitfields.wave_id);
741
742	pr_debug("\t\t ibw is : %u\n",
743	reg_gfx_index.bitfields.instance_broadcast_writes);
744	pr_debug("\t\t ii is : %u\n",
745	reg_gfx_index.bitfields.instance_index);
746	pr_debug("\t\t sebw is : %u\n",
747	reg_gfx_index.bitfields.se_broadcast_writes);
748	pr_debug("\t\t se_ind is : %u\n", reg_gfx_index.bitfields.se_index);
749	pr_debug("\t\t sh_ind is : %u\n", reg_gfx_index.bitfields.sh_index);
750	pr_debug("\t\t sbw is : %u\n",
751	reg_gfx_index.bitfields.sh_broadcast_writes);
752
753	pr_debug("\t\t %30s\n", "* * * * * * * * * * * * * * * * * *");
754
755	return dbgdev->dev->kfd2kgd->wave_control_execute(dbgdev->dev->adev,
756	reg_gfx_index.u32All,
757	reg_sq_cmd.u32All);
758	}
759
760	int dbgdev_wave_reset_wavefronts(struct kfd_dev dev, struct kfd_process p)
761	{
762	int status = 0;
763	unsigned int vmid;
764	uint16_t queried_pasid;
765	union SQ_CMD_BITS reg_sq_cmd;
766	union GRBM_GFX_INDEX_BITS reg_gfx_index;
767	struct kfd_process_device *pdd;
768	struct dbg_wave_control_info wac_info;
769	int first_vmid_to_scan = dev->vm_info.first_vmid_kfd;
770	int last_vmid_to_scan = dev->vm_info.last_vmid_kfd;
771
772	reg_sq_cmd.u32All = 0;
773	status = 0;
774
775	wac_info.mode = HSA_DBG_WAVEMODE_BROADCAST_PROCESS;
776	wac_info.operand = HSA_DBG_WAVEOP_KILL;
777
778	pr_debug("Killing all process wavefronts\n");
779
780	/* Scan all registers in the range ATC_VMID8_PASID_MAPPING ..
781	* ATC_VMID15_PASID_MAPPING
782	* to check which VMID the current process is mapped to.
783	*/
784
785	for (vmid = first_vmid_to_scan; vmid <= last_vmid_to_scan; vmid++) {
786	status = dev->kfd2kgd->get_atc_vmid_pasid_mapping_info
787	(dev->adev, vmid, &queried_pasid);
788
789	if (status && queried_pasid == p->pasid) {
790	pr_debug("Killing wave fronts of vmid %d and pasid 0x%x\n",
791	vmid, p->pasid);
792	break;
793	}
794	}
795
796	if (vmid > last_vmid_to_scan) {
797	pr_err("Didn't find vmid for pasid 0x%x\n", p->pasid);
798	return -EFAULT;
799	}
800
801	/* taking the VMID for that process on the safe way using PDD */
802	pdd = kfd_get_process_device_data(dev, p);
803	if (!pdd)
804	return -EFAULT;
805
806	status = dbgdev_wave_control_set_registers(&wac_info, &reg_sq_cmd,
807	&reg_gfx_index);
808	if (status != 0)
809	return -EINVAL;
810
811	/* for non DIQ we need to patch the VMID: */
812	reg_sq_cmd.bits.vm_id = vmid;
813
814	dev->kfd2kgd->wave_control_execute(dev->adev,
815	reg_gfx_index.u32All,
816	reg_sq_cmd.u32All);
817
818	return 0;
819	}
820
821	void kfd_dbgdev_init(struct kfd_dbgdev pdbgdev, struct kfd_dev pdev,
822	enum DBGDEV_TYPE type)
823	{
824	pdbgdev->dev = pdev;
825	pdbgdev->kq = NULL;
826	pdbgdev->type = type;
827	pdbgdev->pqm = NULL;
828
829	switch (type) {
830	case DBGDEV_TYPE_NODIQ:
831	pdbgdev->dbgdev_register = dbgdev_register_nodiq;
832	pdbgdev->dbgdev_unregister = dbgdev_unregister_nodiq;
833	pdbgdev->dbgdev_wave_control = dbgdev_wave_control_nodiq;
834	pdbgdev->dbgdev_address_watch = dbgdev_address_watch_nodiq;
835	break;
836	case DBGDEV_TYPE_DIQ:
837	default:
838	pdbgdev->dbgdev_register = dbgdev_register_diq;
839	pdbgdev->dbgdev_unregister = dbgdev_unregister_diq;
840	pdbgdev->dbgdev_wave_control = dbgdev_wave_control_diq;
841	pdbgdev->dbgdev_address_watch = dbgdev_address_watch_diq;
842	break;
843	}
844
845	}

drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.h+0−230

@@ -1,230 +0,0 @@
1	/*
2	* Copyright 2014 Advanced Micro Devices, Inc.
3	*
4	* Permission is hereby granted, free of charge, to any person obtaining a
5	* copy of this software and associated documentation files (the "Software"),
6	* to deal in the Software without restriction, including without limitation
7	* the rights to use, copy, modify, merge, publish, distribute, sublicense,
8	* and/or sell copies of the Software, and to permit persons to whom the
9	* Software is furnished to do so, subject to the following conditions:
10	*
11	* The above copyright notice and this permission notice shall be included in
12	* all copies or substantial portions of the Software.
13	*
14	* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15	* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16	* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
17	* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
18	* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
19	* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
20	* OTHER DEALINGS IN THE SOFTWARE.
21	*/
22
23	#ifndef KFD_DBGDEV_H_
24	#define KFD_DBGDEV_H_
25
26	enum {
27	SQ_CMD_VMID_OFFSET = 28,
28	ADDRESS_WATCH_CNTL_OFFSET = 24
29	};
30
31	enum {
32	PRIV_QUEUE_SYNC_TIME_MS = 200
33	};
34
35	/* CONTEXT reg space definition */
36	enum {
37	CONTEXT_REG_BASE = 0xA000,
38	CONTEXT_REG_END = 0xA400,
39	CONTEXT_REG_SIZE = CONTEXT_REG_END - CONTEXT_REG_BASE
40	};
41
42	/* USER CONFIG reg space definition */
43	enum {
44	USERCONFIG_REG_BASE = 0xC000,
45	USERCONFIG_REG_END = 0x10000,
46	USERCONFIG_REG_SIZE = USERCONFIG_REG_END - USERCONFIG_REG_BASE
47	};
48
49	/* CONFIG reg space definition */
50	enum {
51	AMD_CONFIG_REG_BASE = 0x2000, /* in dwords */
52	AMD_CONFIG_REG_END = 0x2B00,
53	AMD_CONFIG_REG_SIZE = AMD_CONFIG_REG_END - AMD_CONFIG_REG_BASE
54	};
55
56	/* SH reg space definition */
57	enum {
58	SH_REG_BASE = 0x2C00,
59	SH_REG_END = 0x3000,
60	SH_REG_SIZE = SH_REG_END - SH_REG_BASE
61	};
62
63	/* SQ_CMD definitions */
64	#define SQ_CMD 0x8DEC
65
66	enum SQ_IND_CMD_CMD {
67	SQ_IND_CMD_CMD_NULL = 0x00000000,
68	SQ_IND_CMD_CMD_HALT = 0x00000001,
69	SQ_IND_CMD_CMD_RESUME = 0x00000002,
70	SQ_IND_CMD_CMD_KILL = 0x00000003,
71	SQ_IND_CMD_CMD_DEBUG = 0x00000004,
72	SQ_IND_CMD_CMD_TRAP = 0x00000005,
73	};
74
75	enum SQ_IND_CMD_MODE {
76	SQ_IND_CMD_MODE_SINGLE = 0x00000000,
77	SQ_IND_CMD_MODE_BROADCAST = 0x00000001,
78	SQ_IND_CMD_MODE_BROADCAST_QUEUE = 0x00000002,
79	SQ_IND_CMD_MODE_BROADCAST_PIPE = 0x00000003,
80	SQ_IND_CMD_MODE_BROADCAST_ME = 0x00000004,
81	};
82
83	union SQ_IND_INDEX_BITS {
84	struct {
85	uint32_t wave_id:4;
86	uint32_t simd_id:2;
87	uint32_t thread_id:6;
88	uint32_t:1;
89	uint32_t force_read:1;
90	uint32_t read_timeout:1;
91	uint32_t unindexed:1;
92	uint32_t index:16;
93
94	} bitfields, bits;
95	uint32_t u32All;
96	signed int i32All;
97	float f32All;
98	};
99
100	union SQ_IND_CMD_BITS {
101	struct {
102	uint32_t data:32;
103	} bitfields, bits;
104	uint32_t u32All;
105	signed int i32All;
106	float f32All;
107	};
108
109	union SQ_CMD_BITS {
110	struct {
111	uint32_t cmd:3;
112	uint32_t:1;
113	uint32_t mode:3;
114	uint32_t check_vmid:1;
115	uint32_t trap_id:3;
116	uint32_t:5;
117	uint32_t wave_id:4;
118	uint32_t simd_id:2;
119	uint32_t:2;
120	uint32_t queue_id:3;
121	uint32_t:1;
122	uint32_t vm_id:4;
123	} bitfields, bits;
124	uint32_t u32All;
125	signed int i32All;
126	float f32All;
127	};
128
129	union SQ_IND_DATA_BITS {
130	struct {
131	uint32_t data:32;
132	} bitfields, bits;
133	uint32_t u32All;
134	signed int i32All;
135	float f32All;
136	};
137
138	union GRBM_GFX_INDEX_BITS {
139	struct {
140	uint32_t instance_index:8;
141	uint32_t sh_index:8;
142	uint32_t se_index:8;
143	uint32_t:5;
144	uint32_t sh_broadcast_writes:1;
145	uint32_t instance_broadcast_writes:1;
146	uint32_t se_broadcast_writes:1;
147	} bitfields, bits;
148	uint32_t u32All;
149	signed int i32All;
150	float f32All;
151	};
152
153	union TCP_WATCH_ADDR_H_BITS {
154	struct {
155	uint32_t addr:16;
156	uint32_t:16;
157
158	} bitfields, bits;
159	uint32_t u32All;
160	signed int i32All;
161	float f32All;
162	};
163
164	union TCP_WATCH_ADDR_L_BITS {
165	struct {
166	uint32_t:6;
167	uint32_t addr:26;
168	} bitfields, bits;
169	uint32_t u32All;
170	signed int i32All;
171	float f32All;
172	};
173
174	enum {
175	QUEUESTATE__INVALID = 0, /* so by default we'll get invalid state */
176	QUEUESTATE__ACTIVE_COMPLETION_PENDING,
177	QUEUESTATE__ACTIVE
178	};
179
180	union ULARGE_INTEGER {
181	struct {
182	uint32_t low_part;
183	uint32_t high_part;
184	} u;
185	unsigned long long quad_part;
186	};
187
188
189	#define KFD_CIK_VMID_START_OFFSET (8)
190	#define KFD_CIK_VMID_END_OFFSET (KFD_CIK_VMID_START_OFFSET + (8))
191
192
193	void kfd_dbgdev_init(struct kfd_dbgdev pdbgdev, struct kfd_dev pdev,
194	enum DBGDEV_TYPE type);
195
196	union TCP_WATCH_CNTL_BITS {
197	struct {
198	uint32_t mask:24;
199	uint32_t vmid:4;
200	uint32_t atc:1;
201	uint32_t mode:2;
202	uint32_t valid:1;
203	} bitfields, bits;
204	uint32_t u32All;
205	signed int i32All;
206	float f32All;
207	};
208
209	enum {
210	ADDRESS_WATCH_REG_CNTL_ATC_BIT = 0x10000000UL,
211	ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK = 0x00FFFFFF,
212	ADDRESS_WATCH_REG_ADDLOW_MASK_EXTENSION = 0x03000000,
213	/* extend the mask to 26 bits in order to match the low address field */
214	ADDRESS_WATCH_REG_ADDLOW_SHIFT = 6,
215	ADDRESS_WATCH_REG_ADDHIGH_MASK = 0xFFFF
216	};
217
218	enum {
219	MAX_TRAPID = 8, /* 3 bits in the bitfield. */
220	MAX_WATCH_ADDRESSES = 4
221	};
222
223	enum {
224	ADDRESS_WATCH_REG_ADDR_HI = 0,
225	ADDRESS_WATCH_REG_ADDR_LO,
226	ADDRESS_WATCH_REG_CNTL,
227	ADDRESS_WATCH_REG_MAX
228	};
229
230	#endif /* KFD_DBGDEV_H_ */

drivers/gpu/drm/amd/amdkfd/kfd_dbgmgr.c+0−158

@@ -1,158 +0,0 @@
1	/*
2	* Copyright 2014 Advanced Micro Devices, Inc.
3	*
4	* Permission is hereby granted, free of charge, to any person obtaining a
5	* copy of this software and associated documentation files (the "Software"),
6	* to deal in the Software without restriction, including without limitation
7	* the rights to use, copy, modify, merge, publish, distribute, sublicense,
8	* and/or sell copies of the Software, and to permit persons to whom the
9	* Software is furnished to do so, subject to the following conditions:
10	*
11	* The above copyright notice and this permission notice shall be included in
12	* all copies or substantial portions of the Software.
13	*
14	* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15	* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16	* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
17	* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
18	* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
19	* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
20	* OTHER DEALINGS IN THE SOFTWARE.
21	*
22	*/
23	#include <linux/types.h>
24	#include <linux/kernel.h>
25	#include <linux/log2.h>
26	#include <linux/sched.h>
27	#include <linux/slab.h>
28	#include <linux/device.h>
29
30	#include "kfd_priv.h"
31	#include "cik_regs.h"
32	#include "kfd_pm4_headers.h"
33	#include "kfd_pm4_headers_diq.h"
34	#include "kfd_dbgmgr.h"
35	#include "kfd_dbgdev.h"
36	#include "kfd_device_queue_manager.h"
37
38	static DEFINE_MUTEX(kfd_dbgmgr_mutex);
39
40	struct mutex *kfd_get_dbgmgr_mutex(void)
41	{
42	return &kfd_dbgmgr_mutex;
43	}
44
45
46	static void kfd_dbgmgr_uninitialize(struct kfd_dbgmgr *pmgr)
47	{
48	kfree(pmgr->dbgdev);
49
50	pmgr->dbgdev = NULL;
51	pmgr->pasid = 0;
52	pmgr->dev = NULL;
53	}
54
55	void kfd_dbgmgr_destroy(struct kfd_dbgmgr *pmgr)
56	{
57	if (pmgr) {
58	kfd_dbgmgr_uninitialize(pmgr);
59	kfree(pmgr);
60	}
61	}
62
63	bool kfd_dbgmgr_create(struct kfd_dbgmgr *ppmgr, struct kfd_dev pdev)
64	{
65	enum DBGDEV_TYPE type = DBGDEV_TYPE_DIQ;
66	struct kfd_dbgmgr *new_buff;
67
68	if (WARN_ON(!pdev->init_complete))
69	return false;
70
71	new_buff = kfd_alloc_struct(new_buff);
72	if (!new_buff) {
73	pr_err("Failed to allocate dbgmgr instance\n");
74	return false;
75	}
76
77	new_buff->pasid = 0;
78	new_buff->dev = pdev;
79	new_buff->dbgdev = kfd_alloc_struct(new_buff->dbgdev);
80	if (!new_buff->dbgdev) {
81	pr_err("Failed to allocate dbgdev instance\n");
82	kfree(new_buff);
83	return false;
84	}
85
86	/* get actual type of DBGDevice cpsch or not */
87	if (pdev->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS)
88	type = DBGDEV_TYPE_NODIQ;
89
90	kfd_dbgdev_init(new_buff->dbgdev, pdev, type);
91	*ppmgr = new_buff;
92
93	return true;
94	}
95
96	long kfd_dbgmgr_register(struct kfd_dbgmgr pmgr, struct kfd_process p)
97	{
98	if (pmgr->pasid != 0) {
99	pr_debug("H/W debugger is already active using pasid 0x%x\n",
100	pmgr->pasid);
101	return -EBUSY;
102	}
103
104	/* remember pasid */
105	pmgr->pasid = p->pasid;
106
107	/* provide the pqm for diq generation */
108	pmgr->dbgdev->pqm = &p->pqm;
109
110	/* activate the actual registering */
111	pmgr->dbgdev->dbgdev_register(pmgr->dbgdev);
112
113	return 0;
114	}
115
116	long kfd_dbgmgr_unregister(struct kfd_dbgmgr pmgr, struct kfd_process p)
117	{
118	/* Is the requests coming from the already registered process? */
119	if (pmgr->pasid != p->pasid) {
120	pr_debug("H/W debugger is not registered by calling pasid 0x%x\n",
121	p->pasid);
122	return -EINVAL;
123	}
124
125	pmgr->dbgdev->dbgdev_unregister(pmgr->dbgdev);
126
127	pmgr->pasid = 0;
128
129	return 0;
130	}
131
132	long kfd_dbgmgr_wave_control(struct kfd_dbgmgr *pmgr,
133	struct dbg_wave_control_info *wac_info)
134	{
135	/* Is the requests coming from the already registered process? */
136	if (pmgr->pasid != wac_info->process->pasid) {
137	pr_debug("H/W debugger support was not registered for requester pasid 0x%x\n",
138	wac_info->process->pasid);
139	return -EINVAL;
140	}
141
142	return (long) pmgr->dbgdev->dbgdev_wave_control(pmgr->dbgdev, wac_info);
143	}
144
145	long kfd_dbgmgr_address_watch(struct kfd_dbgmgr *pmgr,
146	struct dbg_address_watch_info *adw_info)
147	{
148	/* Is the requests coming from the already registered process? */
149	if (pmgr->pasid != adw_info->process->pasid) {
150	pr_debug("H/W debugger support was not registered for requester pasid 0x%x\n",
151	adw_info->process->pasid);
152	return -EINVAL;
153	}
154
155	return (long) pmgr->dbgdev->dbgdev_address_watch(pmgr->dbgdev,
156	adw_info);
157	}
158

drivers/gpu/drm/amd/amdkfd/kfd_dbgmgr.h+0−293

@@ -1,293 +0,0 @@
1	/*
2	* Copyright 2014 Advanced Micro Devices, Inc.
3	*
4	* Permission is hereby granted, free of charge, to any person obtaining a
5	* copy of this software and associated documentation files (the "Software"),
6	* to deal in the Software without restriction, including without limitation
7	* the rights to use, copy, modify, merge, publish, distribute, sublicense,
8	* and/or sell copies of the Software, and to permit persons to whom the
9	* Software is furnished to do so, subject to the following conditions:
10	*
11	* The above copyright notice and this permission notice shall be included in
12	* all copies or substantial portions of the Software.
13	*
14	* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15	* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16	* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
17	* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
18	* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
19	* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
20	* OTHER DEALINGS IN THE SOFTWARE.
21	*
22	*/
23
24	#ifndef KFD_DBGMGR_H_
25	#define KFD_DBGMGR_H_
26
27	#include "kfd_priv.h"
28
29	/* must align with hsakmttypes definition */
30	#pragma pack(push, 4)
31
32	enum HSA_DBG_WAVEOP {
33	HSA_DBG_WAVEOP_HALT = 1, /* Halts a wavefront */
34	HSA_DBG_WAVEOP_RESUME = 2, /* Resumes a wavefront */
35	HSA_DBG_WAVEOP_KILL = 3, /* Kills a wavefront */
36	HSA_DBG_WAVEOP_DEBUG = 4, /* Causes wavefront to enter dbg mode */
37	HSA_DBG_WAVEOP_TRAP = 5, /* Causes wavefront to take a trap */
38	HSA_DBG_NUM_WAVEOP = 5,
39	HSA_DBG_MAX_WAVEOP = 0xFFFFFFFF
40	};
41
42	enum HSA_DBG_WAVEMODE {
43	/* send command to a single wave */
44	HSA_DBG_WAVEMODE_SINGLE = 0,
45	/*
46	* Broadcast to all wavefronts of all processes is not
47	* supported for HSA user mode
48	*/
49
50	/* send to waves within current process */
51	HSA_DBG_WAVEMODE_BROADCAST_PROCESS = 2,
52	/* send to waves within current process on CU */
53	HSA_DBG_WAVEMODE_BROADCAST_PROCESS_CU = 3,
54	HSA_DBG_NUM_WAVEMODE = 3,
55	HSA_DBG_MAX_WAVEMODE = 0xFFFFFFFF
56	};
57
58	enum HSA_DBG_WAVEMSG_TYPE {
59	HSA_DBG_WAVEMSG_AUTO = 0,
60	HSA_DBG_WAVEMSG_USER = 1,
61	HSA_DBG_WAVEMSG_ERROR = 2,
62	HSA_DBG_NUM_WAVEMSG,
63	HSA_DBG_MAX_WAVEMSG = 0xFFFFFFFF
64	};
65
66	enum HSA_DBG_WATCH_MODE {
67	HSA_DBG_WATCH_READ = 0, /* Read operations only */
68	HSA_DBG_WATCH_NONREAD = 1, /* Write or Atomic operations only */
69	HSA_DBG_WATCH_ATOMIC = 2, /* Atomic Operations only */
70	HSA_DBG_WATCH_ALL = 3, /* Read, Write or Atomic operations */
71	HSA_DBG_WATCH_NUM,
72	HSA_DBG_WATCH_SIZE = 0xFFFFFFFF
73	};
74
75	/* This structure is hardware specific and may change in the future */
76	struct HsaDbgWaveMsgAMDGen2 {
77	union {
78	struct ui32 {
79	uint32_t UserData:8; /* user data */
80	uint32_t ShaderArray:1; /* Shader array */
81	uint32_t Priv:1; /* Privileged */
82	uint32_t Reserved0:4; /* Reserved, should be 0 */
83	uint32_t WaveId:4; /* wave id */
84	uint32_t SIMD:2; /* SIMD id */
85	uint32_t HSACU:4; /* Compute unit */
86	uint32_t ShaderEngine:2;/* Shader engine */
87	uint32_t MessageType:2; /* see HSA_DBG_WAVEMSG_TYPE */
88	uint32_t Reserved1:4; /* Reserved, should be 0 */
89	} ui32;
90	uint32_t Value;
91	};
92	uint32_t Reserved2;
93	};
94
95	union HsaDbgWaveMessageAMD {
96	struct HsaDbgWaveMsgAMDGen2 WaveMsgInfoGen2;
97	/* for future HsaDbgWaveMsgAMDGen3; */
98	};
99
100	struct HsaDbgWaveMessage {
101	void MemoryVA; / ptr to associated host-accessible data */
102	union HsaDbgWaveMessageAMD DbgWaveMsg;
103	};
104
105	/*
106	* TODO: This definitions to be MOVED to kfd_event, once it is implemented.
107	*
108	* HSA sync primitive, Event and HW Exception notification API definitions.
109	* The API functions allow the runtime to define a so-called sync-primitive,
110	* a SW object combining a user-mode provided "syncvar" and a scheduler event
111	* that can be signaled through a defined GPU interrupt. A syncvar is
112	* a process virtual memory location of a certain size that can be accessed
113	* by CPU and GPU shader code within the process to set and query the content
114	* within that memory. The definition of the content is determined by the HSA
115	* runtime and potentially GPU shader code interfacing with the HSA runtime.
116	* The syncvar values may be commonly written through an PM4 WRITE_DATA packet
117	* in the user mode instruction stream. The OS scheduler event is typically
118	* associated and signaled by an interrupt issued by the GPU, but other HSA
119	* system interrupt conditions from other HW (e.g. IOMMUv2) may be surfaced
120	* by the KFD by this mechanism, too.
121	*/
122
123	/* these are the new definitions for events */
124	enum HSA_EVENTTYPE {
125	HSA_EVENTTYPE_SIGNAL = 0, /* user-mode generated GPU signal */
126	HSA_EVENTTYPE_NODECHANGE = 1, /* HSA node change (attach/detach) */
127	HSA_EVENTTYPE_DEVICESTATECHANGE = 2, /* HSA device state change
128	* (start/stop)
129	*/
130	HSA_EVENTTYPE_HW_EXCEPTION = 3, /* GPU shader exception event */
131	HSA_EVENTTYPE_SYSTEM_EVENT = 4, /* GPU SYSCALL with parameter info */
132	HSA_EVENTTYPE_DEBUG_EVENT = 5, /* GPU signal for debugging */
133	HSA_EVENTTYPE_PROFILE_EVENT = 6,/* GPU signal for profiling */
134	HSA_EVENTTYPE_QUEUE_EVENT = 7, /* GPU signal queue idle state
135	* (EOP pm4)
136	*/
137	/* ... */
138	HSA_EVENTTYPE_MAXID,
139	HSA_EVENTTYPE_TYPE_SIZE = 0xFFFFFFFF
140	};
141
142	/* Sub-definitions for various event types: Syncvar */
143	struct HsaSyncVar {
144	union SyncVar {
145	void UserData; / pointer to user mode data */
146	uint64_t UserDataPtrValue; /* 64bit compatibility of value */
147	} SyncVar;
148	uint64_t SyncVarSize;
149	};
150
151	/* Sub-definitions for various event types: NodeChange */
152
153	enum HSA_EVENTTYPE_NODECHANGE_FLAGS {
154	HSA_EVENTTYPE_NODECHANGE_ADD = 0,
155	HSA_EVENTTYPE_NODECHANGE_REMOVE = 1,
156	HSA_EVENTTYPE_NODECHANGE_SIZE = 0xFFFFFFFF
157	};
158
159	struct HsaNodeChange {
160	/* HSA node added/removed on the platform */
161	enum HSA_EVENTTYPE_NODECHANGE_FLAGS Flags;
162	};
163
164	/* Sub-definitions for various event types: DeviceStateChange */
165	enum HSA_EVENTTYPE_DEVICESTATECHANGE_FLAGS {
166	/* device started (and available) */
167	HSA_EVENTTYPE_DEVICESTATUSCHANGE_START = 0,
168	/* device stopped (i.e. unavailable) */
169	HSA_EVENTTYPE_DEVICESTATUSCHANGE_STOP = 1,
170	HSA_EVENTTYPE_DEVICESTATUSCHANGE_SIZE = 0xFFFFFFFF
171	};
172
173	enum HSA_DEVICE {
174	HSA_DEVICE_CPU = 0,
175	HSA_DEVICE_GPU = 1,
176	MAX_HSA_DEVICE = 2
177	};
178
179	struct HsaDeviceStateChange {
180	uint32_t NodeId; /* F-NUMA node that contains the device */
181	enum HSA_DEVICE Device; /* device type: GPU or CPU */
182	enum HSA_EVENTTYPE_DEVICESTATECHANGE_FLAGS Flags; /* event flags */
183	};
184
185	struct HsaEventData {
186	enum HSA_EVENTTYPE EventType; /* event type */
187	union EventData {
188	/*
189	* return data associated with HSA_EVENTTYPE_SIGNAL
190	* and other events
191	*/
192	struct HsaSyncVar SyncVar;
193
194	/* data associated with HSA_EVENTTYPE_NODE_CHANGE */
195	struct HsaNodeChange NodeChangeState;
196
197	/* data associated with HSA_EVENTTYPE_DEVICE_STATE_CHANGE */
198	struct HsaDeviceStateChange DeviceState;
199	} EventData;
200
201	/* the following data entries are internal to the KFD & thunk itself */
202
203	/* internal thunk store for Event data (OsEventHandle) */
204	uint64_t HWData1;
205	/* internal thunk store for Event data (HWAddress) */
206	uint64_t HWData2;
207	/* internal thunk store for Event data (HWData) */
208	uint32_t HWData3;
209	};
210
211	struct HsaEventDescriptor {
212	/* event type to allocate */
213	enum HSA_EVENTTYPE EventType;
214	/* H-NUMA node containing GPU device that is event source */
215	uint32_t NodeId;
216	/* pointer to user mode syncvar data, syncvar->UserDataPtrValue
217	* may be NULL
218	*/
219	struct HsaSyncVar SyncVar;
220	};
221
222	struct HsaEvent {
223	uint32_t EventId;
224	struct HsaEventData EventData;
225	};
226
227	#pragma pack(pop)
228
229	enum DBGDEV_TYPE {
230	DBGDEV_TYPE_ILLEGAL = 0,
231	DBGDEV_TYPE_NODIQ = 1,
232	DBGDEV_TYPE_DIQ = 2,
233	DBGDEV_TYPE_TEST = 3
234	};
235
236	struct dbg_address_watch_info {
237	struct kfd_process *process;
238	enum HSA_DBG_WATCH_MODE *watch_mode;
239	uint64_t *watch_address;
240	uint64_t *watch_mask;
241	struct HsaEvent *watch_event;
242	uint32_t num_watch_points;
243	};
244
245	struct dbg_wave_control_info {
246	struct kfd_process *process;
247	uint32_t trapId;
248	enum HSA_DBG_WAVEOP operand;
249	enum HSA_DBG_WAVEMODE mode;
250	struct HsaDbgWaveMessage dbgWave_msg;
251	};
252
253	struct kfd_dbgdev {
254
255	/* The device that owns this data. */
256	struct kfd_dev *dev;
257
258	/* kernel queue for DIQ */
259	struct kernel_queue *kq;
260
261	/* a pointer to the pqm of the calling process */
262	struct process_queue_manager *pqm;
263
264	/* type of debug device ( DIQ, non DIQ, etc. ) */
265	enum DBGDEV_TYPE type;
266
267	/* virtualized function pointers to device dbg */
268	int (dbgdev_register)(struct kfd_dbgdev dbgdev);
269	int (dbgdev_unregister)(struct kfd_dbgdev dbgdev);
270	int (dbgdev_address_watch)(struct kfd_dbgdev dbgdev,
271	struct dbg_address_watch_info *adw_info);
272	int (dbgdev_wave_control)(struct kfd_dbgdev dbgdev,
273	struct dbg_wave_control_info *wac_info);
274
275	};
276
277	struct kfd_dbgmgr {
278	u32 pasid;
279	struct kfd_dev *dev;
280	struct kfd_dbgdev *dbgdev;
281	};
282
283	/* prototypes for debug manager functions */
284	struct mutex *kfd_get_dbgmgr_mutex(void);
285	void kfd_dbgmgr_destroy(struct kfd_dbgmgr *pmgr);
286	bool kfd_dbgmgr_create(struct kfd_dbgmgr *ppmgr, struct kfd_dev pdev);
287	long kfd_dbgmgr_register(struct kfd_dbgmgr pmgr, struct kfd_process p);
288	long kfd_dbgmgr_unregister(struct kfd_dbgmgr pmgr, struct kfd_process p);
289	long kfd_dbgmgr_wave_control(struct kfd_dbgmgr *pmgr,
290	struct dbg_wave_control_info *wac_info);
291	long kfd_dbgmgr_address_watch(struct kfd_dbgmgr *pmgr,
292	struct dbg_address_watch_info *adw_info);
293	#endif /* KFD_DBGMGR_H_ */

drivers/gpu/drm/amd/amdkfd/kfd_debugfs.c+3−2

+// SPDX-License-Identifier: GPL-2.0 OR MIT
 /*
- * Copyright 2016-2017 Advanced Micro Devices, Inc.
+ * Copyright 2016-2022 Advanced Micro Devices, Inc.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
  * copy of this software and associated documentation files (the "Software"),
 }
 static int kfd_debugfs_hang_hws_read(struct seq_file *m, void *data)
 {
-	seq_printf(m, "echo gpu_id > hang_hws\n");
+	seq_puts(m, "echo gpu_id > hang_hws\n");
 	return 0;
 }

drivers/gpu/drm/amd/amdkfd/kfd_device.c+34−33

@@ -1,5 +1,6 @@
		1	// SPDX-License-Identifier: GPL-2.0 OR MIT
1	/*	2	/*
2	* Copyright 2014 Advanced Micro Devices, Inc.	3	* Copyright 2014-2022 Advanced Micro Devices, Inc.
3	*	4	*
4	* Permission is hereby granted, free of charge, to any person obtaining a	5	* Permission is hereby granted, free of charge, to any person obtaining a
5	* copy of this software and associated documentation files (the "Software"),	6	* copy of this software and associated documentation files (the "Software"),
@@ -64,34 +65,33 @@ static void kfd_device_info_set_sdma_queue_num(struct kfd_dev *kfd) · 58 unmodified lines
64	uint32_t sdma_version = kfd->adev->ip_versions[SDMA0_HWIP][0];	65	uint32_t sdma_version = kfd->adev->ip_versions[SDMA0_HWIP][0];
65		66
66	switch (sdma_version) {	67	switch (sdma_version) {
67	case IP_VERSION(4, 0, 0):/* VEGA10 */	68	case IP_VERSION(4, 0, 0):/* VEGA10 */
68	case IP_VERSION(4, 0, 1):/* VEGA12 */	69	case IP_VERSION(4, 0, 1):/* VEGA12 */
69	case IP_VERSION(4, 1, 0):/* RAVEN */	70	case IP_VERSION(4, 1, 0):/* RAVEN */
70	case IP_VERSION(4, 1, 1):/* RAVEN */	71	case IP_VERSION(4, 1, 1):/* RAVEN */
71	case IP_VERSION(4, 1, 2):/* RENOIR */	72	case IP_VERSION(4, 1, 2):/* RENOIR */
72	case IP_VERSION(5, 2, 1):/* VANGOGH */	73	case IP_VERSION(5, 2, 1):/* VANGOGH */
73	case IP_VERSION(5, 2, 3):/* YELLOW_CARP */	74	case IP_VERSION(5, 2, 3):/* YELLOW_CARP */
74	kfd->device_info.num_sdma_queues_per_engine = 2;	75	kfd->device_info.num_sdma_queues_per_engine = 2;
75	break;	76	break;
76	case IP_VERSION(4, 2, 0):/* VEGA20 */	77	case IP_VERSION(4, 2, 0):/* VEGA20 */
77	case IP_VERSION(4, 2, 2):/* ARCTURUS */	78	case IP_VERSION(4, 2, 2):/* ARCTURUS */
78	case IP_VERSION(4, 4, 0):/* ALDEBARAN */	79	case IP_VERSION(4, 4, 0):/* ALDEBARAN */
79	case IP_VERSION(5, 0, 0):/* NAVI10 */	80	case IP_VERSION(5, 0, 0):/* NAVI10 */
80	case IP_VERSION(5, 0, 1):/* CYAN_SKILLFISH */	81	case IP_VERSION(5, 0, 1):/* CYAN_SKILLFISH */
81	case IP_VERSION(5, 0, 2):/* NAVI14 */	82	case IP_VERSION(5, 0, 2):/* NAVI14 */
82	case IP_VERSION(5, 0, 5):/* NAVI12 */	83	case IP_VERSION(5, 0, 5):/* NAVI12 */
83	case IP_VERSION(5, 2, 0):/* SIENNA_CICHLID */	84	case IP_VERSION(5, 2, 0):/* SIENNA_CICHLID */
84	case IP_VERSION(5, 2, 2):/* NAVY_FLOUNDER */	85	case IP_VERSION(5, 2, 2):/* NAVY_FLOUNDER */
85	case IP_VERSION(5, 2, 4):/* DIMGREY_CAVEFISH */	86	case IP_VERSION(5, 2, 4):/* DIMGREY_CAVEFISH */
86	case IP_VERSION(5, 2, 5):/* BEIGE_GOBY */	87	case IP_VERSION(5, 2, 5):/* BEIGE_GOBY */
87	kfd->device_info.num_sdma_queues_per_engine = 8;	88	kfd->device_info.num_sdma_queues_per_engine = 8;
88	break;	89	break;
89	default:	90	default:
90	dev_warn(kfd_device,	91	dev_warn(kfd_device,
91	"Default sdma queue per engine(8) is set due to "	92	"Default sdma queue per engine(8) is set due to mismatch of sdma ip block(SDMA_HWIP:0x%x).\n",
92	"mismatch of sdma ip block(SDMA_HWIP:0x%x).\n",	93	sdma_version);
93	sdma_version);	94	kfd->device_info.num_sdma_queues_per_engine = 8;
94	kfd->device_info.num_sdma_queues_per_engine = 8;
95	}	95	}
96	}	96	}
97		97
@@ -111,6 +111,7 @@ static void kfd_device_info_set_event_interrupt_class(struct kfd_dev *kfd) · 13 unmodified lines
111	case IP_VERSION(10, 3, 1): /* VANGOGH */	111	case IP_VERSION(10, 3, 1): /* VANGOGH */
112	case IP_VERSION(10, 3, 3): /* YELLOW_CARP */	112	case IP_VERSION(10, 3, 3): /* YELLOW_CARP */
113	case IP_VERSION(10, 1, 3): /* CYAN_SKILLFISH */	113	case IP_VERSION(10, 1, 3): /* CYAN_SKILLFISH */
		114	case IP_VERSION(10, 1, 4):
114	case IP_VERSION(10, 1, 10): /* NAVI10 */	115	case IP_VERSION(10, 1, 10): /* NAVI10 */
115	case IP_VERSION(10, 1, 2): /* NAVI12 */	116	case IP_VERSION(10, 1, 2): /* NAVI12 */
116	case IP_VERSION(10, 1, 1): /* NAVI14 */	117	case IP_VERSION(10, 1, 1): /* NAVI14 */
@@ -308,6 +309,7 @@ struct kfd_dev kgd2kfd_probe(struct amdgpu_device adev, bool vf) · 191 unmodified lines
308	break;	309	break;
309	/* Cyan Skillfish */	310	/* Cyan Skillfish */
310	case IP_VERSION(10, 1, 3):	311	case IP_VERSION(10, 1, 3):
		312	case IP_VERSION(10, 1, 4):
311	gfx_target_version = 100103;	313	gfx_target_version = 100103;
312	if (!vf)	314	if (!vf)
313	f2g = &gfx_v10_kfd2kgd;	315	f2g = &gfx_v10_kfd2kgd;
@@ -437,7 +439,8 @@ static int kfd_gws_init(struct kfd_dev *kfd) · 123 unmodified lines
437	return ret;	439	return ret;
438	}	440	}
439		441
440	static void kfd_smi_init(struct kfd_dev *dev) {	442	static void kfd_smi_init(struct kfd_dev *dev)
		443	{
441	INIT_LIST_HEAD(&dev->smi_clients);	444	INIT_LIST_HEAD(&dev->smi_clients);
442	spin_lock_init(&dev->smi_lock);	445	spin_lock_init(&dev->smi_lock);
443	}	446	}
@@ -570,14 +573,12 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, · 126 unmodified lines
570		573
571	svm_migrate_init(kfd->adev);	574	svm_migrate_init(kfd->adev);
572		575
573	if(kgd2kfd_resume_iommu(kfd))	576	if (kgd2kfd_resume_iommu(kfd))
574	goto device_iommu_error;	577	goto device_iommu_error;
575		578
576	if (kfd_resume(kfd))	579	if (kfd_resume(kfd))
577	goto kfd_resume_error;	580	goto kfd_resume_error;
578		581
579	kfd->dbgmgr = NULL;
580
581	if (kfd_topology_add_device(kfd)) {	582	if (kfd_topology_add_device(kfd)) {
582	dev_err(kfd_device, "Error adding device to topology\n");	583	dev_err(kfd_device, "Error adding device to topology\n");
583	goto kfd_topology_add_device_error;	584	goto kfd_topology_add_device_error;

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c+221−48

@@ -1,5 +1,6 @@
		1	// SPDX-License-Identifier: GPL-2.0 OR MIT
1	/*	2	/*
2	* Copyright 2014 Advanced Micro Devices, Inc.	3	* Copyright 2014-2022 Advanced Micro Devices, Inc.
3	*	4	*
4	* Permission is hereby granted, free of charge, to any person obtaining a	5	* Permission is hereby granted, free of charge, to any person obtaining a
5	* copy of this software and associated documentation files (the "Software"),	6	* copy of this software and associated documentation files (the "Software"),
@@ -58,7 +59,7 @@ static inline void deallocate_hqd(struct device_queue_manager *dqm, · 52 unmodified lines
58	struct queue *q);	59	struct queue *q);
59	static int allocate_hqd(struct device_queue_manager dqm, struct queue q);	60	static int allocate_hqd(struct device_queue_manager dqm, struct queue q);
60	static int allocate_sdma_queue(struct device_queue_manager *dqm,	61	static int allocate_sdma_queue(struct device_queue_manager *dqm,
61	struct queue *q);	62	struct queue q, const uint32_t restore_sdma_id);
62	static void kfd_process_hw_exception(struct work_struct *work);	63	static void kfd_process_hw_exception(struct work_struct *work);
63		64
64	static inline	65	static inline
@@ -144,7 +145,13 @@ static void decrement_queue_count(struct device_queue_manager *dqm, · 79 unmodified lines
144	dqm->active_cp_queue_count--;	145	dqm->active_cp_queue_count--;
145	}	146	}
146		147
147	static int allocate_doorbell(struct qcm_process_device qpd, struct queue q)	148	/*
		149	* Allocate a doorbell ID to this queue.
		150	* If doorbell_id is passed in, make sure requested ID is valid then allocate it.
		151	*/
		152	static int allocate_doorbell(struct qcm_process_device *qpd,
		153	struct queue *q,
		154	uint32_t const *restore_id)
148	{	155	{
149	struct kfd_dev *dev = qpd->dqm->dev;	156	struct kfd_dev *dev = qpd->dqm->dev;
150		157
@@ -152,6 +159,10 @@ static int allocate_doorbell(struct qcm_process_device qpd, struct queue q) · 1 unmodified line
152	/* On pre-SOC15 chips we need to use the queue ID to	159	/* On pre-SOC15 chips we need to use the queue ID to
153	* preserve the user mode ABI.	160	* preserve the user mode ABI.
154	*/	161	*/
		162
		163	if (restore_id && *restore_id != q->properties.queue_id)
		164	return -EINVAL;
		165
155	q->doorbell_id = q->properties.queue_id;	166	q->doorbell_id = q->properties.queue_id;
156	} else if (q->properties.type == KFD_QUEUE_TYPE_SDMA \|\|	167	} else if (q->properties.type == KFD_QUEUE_TYPE_SDMA \|\|
157	q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) {	168	q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) {
@@ -160,25 +171,37 @@ static int allocate_doorbell(struct qcm_process_device qpd, struct queue q) · 2 unmodified lines
160	* The doobell index distance between RLC (2i) and (2i+1)	171	* The doobell index distance between RLC (2i) and (2i+1)
161	* for a SDMA engine is 512.	172	* for a SDMA engine is 512.
162	*/	173	*/
163	uint32_t *idx_offset =
164	dev->shared_resources.sdma_doorbell_idx;
165		174
166	q->doorbell_id = idx_offset[q->properties.sdma_engine_id]	175	uint32_t *idx_offset = dev->shared_resources.sdma_doorbell_idx;
167	+ (q->properties.sdma_queue_id & 1)	176	uint32_t valid_id = idx_offset[q->properties.sdma_engine_id]
168	* KFD_QUEUE_DOORBELL_MIRROR_OFFSET	177	+ (q->properties.sdma_queue_id & 1)
169	+ (q->properties.sdma_queue_id >> 1);	178	* KFD_QUEUE_DOORBELL_MIRROR_OFFSET
		179	+ (q->properties.sdma_queue_id >> 1);
		180
		181	if (restore_id && *restore_id != valid_id)
		182	return -EINVAL;
		183	q->doorbell_id = valid_id;
170	} else {	184	} else {
171	/* For CP queues on SOC15 reserve a free doorbell ID */	185	/* For CP queues on SOC15 */
172	unsigned int found;	186	if (restore_id) {
173		187	/* make sure that ID is free */
174	found = find_first_zero_bit(qpd->doorbell_bitmap,	188	if (__test_and_set_bit(*restore_id, qpd->doorbell_bitmap))
175	KFD_MAX_NUM_OF_QUEUES_PER_PROCESS);	189	return -EINVAL;
176	if (found >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) {	190
177	pr_debug("No doorbells available");	191	q->doorbell_id = *restore_id;
178	return -EBUSY;	192	} else {
		193	/* or reserve a free doorbell ID */
		194	unsigned int found;
		195
		196	found = find_first_zero_bit(qpd->doorbell_bitmap,
		197	KFD_MAX_NUM_OF_QUEUES_PER_PROCESS);
		198	if (found >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) {
		199	pr_debug("No doorbells available");
		200	return -EBUSY;
		201	}
		202	set_bit(found, qpd->doorbell_bitmap);
		203	q->doorbell_id = found;
179	}	204	}
180	set_bit(found, qpd->doorbell_bitmap);
181	q->doorbell_id = found;
182	}	205	}
183		206
184	q->properties.doorbell_off =	207	q->properties.doorbell_off =
@@ -299,7 +322,9 @@ static void deallocate_vmid(struct device_queue_manager *dqm, · 114 unmodified lines
299		322
300	static int create_queue_nocpsch(struct device_queue_manager *dqm,	323	static int create_queue_nocpsch(struct device_queue_manager *dqm,
301	struct queue *q,	324	struct queue *q,
302	struct qcm_process_device *qpd)	325	struct qcm_process_device *qpd,
		326	const struct kfd_criu_queue_priv_data *qd,
		327	const void restore_mqd, const void restore_ctl_stack)
303	{	328	{
304	struct mqd_manager *mqd_mgr;	329	struct mqd_manager *mqd_mgr;
305	int retval;	330	int retval;
@@ -339,13 +364,13 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm, · 33 unmodified lines
339	q->pipe, q->queue);	364	q->pipe, q->queue);
340	} else if (q->properties.type == KFD_QUEUE_TYPE_SDMA \|\|	365	} else if (q->properties.type == KFD_QUEUE_TYPE_SDMA \|\|
341	q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) {	366	q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) {
342	retval = allocate_sdma_queue(dqm, q);	367	retval = allocate_sdma_queue(dqm, q, qd ? &qd->sdma_id : NULL);
343	if (retval)	368	if (retval)
344	goto deallocate_vmid;	369	goto deallocate_vmid;
345	dqm->asic_ops.init_sdma_vm(dqm, q, qpd);	370	dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
346	}	371	}
347		372
348	retval = allocate_doorbell(qpd, q);	373	retval = allocate_doorbell(qpd, q, qd ? &qd->doorbell_id : NULL);
349	if (retval)	374	if (retval)
350	goto out_deallocate_hqd;	375	goto out_deallocate_hqd;
351		376
@@ -358,8 +383,15 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm, · 6 unmodified lines
358	retval = -ENOMEM;	383	retval = -ENOMEM;
359	goto out_deallocate_doorbell;	384	goto out_deallocate_doorbell;
360	}	385	}
361	mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,	386
362	&q->gart_mqd_addr, &q->properties);	387	if (qd)
		388	mqd_mgr->restore_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj, &q->gart_mqd_addr,
		389	&q->properties, restore_mqd, restore_ctl_stack,
		390	qd->ctl_stack_size);
		391	else
		392	mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
		393	&q->gart_mqd_addr, &q->properties);
		394
363	if (q->properties.is_active) {	395	if (q->properties.is_active) {
364	if (!dqm->sched_running) {	396	if (!dqm->sched_running) {
365	WARN_ONCE(1, "Load non-HWS mqd while stopped\n");	397	WARN_ONCE(1, "Load non-HWS mqd while stopped\n");
@@ -449,6 +481,65 @@ static inline void deallocate_hqd(struct device_queue_manager *dqm, · 83 unmodified lines
449	dqm->allocated_queues[q->pipe] \|= (1 << q->queue);	481	dqm->allocated_queues[q->pipe] \|= (1 << q->queue);
450	}	482	}
451		483
		484	#define SQ_IND_CMD_CMD_KILL 0x00000003
		485	#define SQ_IND_CMD_MODE_BROADCAST 0x00000001
		486
		487	static int dbgdev_wave_reset_wavefronts(struct kfd_dev dev, struct kfd_process p)
		488	{
		489	int status = 0;
		490	unsigned int vmid;
		491	uint16_t queried_pasid;
		492	union SQ_CMD_BITS reg_sq_cmd;
		493	union GRBM_GFX_INDEX_BITS reg_gfx_index;
		494	struct kfd_process_device *pdd;
		495	int first_vmid_to_scan = dev->vm_info.first_vmid_kfd;
		496	int last_vmid_to_scan = dev->vm_info.last_vmid_kfd;
		497
		498	reg_sq_cmd.u32All = 0;
		499	reg_gfx_index.u32All = 0;
		500
		501	pr_debug("Killing all process wavefronts\n");
		502
		503	/* Scan all registers in the range ATC_VMID8_PASID_MAPPING ..
		504	* ATC_VMID15_PASID_MAPPING
		505	* to check which VMID the current process is mapped to.
		506	*/
		507
		508	for (vmid = first_vmid_to_scan; vmid <= last_vmid_to_scan; vmid++) {
		509	status = dev->kfd2kgd->get_atc_vmid_pasid_mapping_info
		510	(dev->adev, vmid, &queried_pasid);
		511
		512	if (status && queried_pasid == p->pasid) {
		513	pr_debug("Killing wave fronts of vmid %d and pasid 0x%x\n",
		514	vmid, p->pasid);
		515	break;
		516	}
		517	}
		518
		519	if (vmid > last_vmid_to_scan) {
		520	pr_err("Didn't find vmid for pasid 0x%x\n", p->pasid);
		521	return -EFAULT;

264 more lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h+55−7

This diff is large: its 112 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_cik.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v10.c+2−10

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_v9.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager_vi.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_events.c+281−35

This diff is large: its 474 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_events.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c+3−2

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c+6−5

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_iommu.c+2−13

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_iommu.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_migrate.c+11−13

This diff is large: its 80 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_module.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c+66−1

This diff is large: its 83 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h+43−1

This diff is large: its 61 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c+74−52

This diff is large: its 197 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c+86−69

This diff is large: its 242 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c+100−75

This diff is large: its 254 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c+92−59

This diff is large: its 231 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c+5−6

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c+16−36

This diff is large: its 93 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c+5−24

This diff is large: its 57 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_pasid.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_pm4_headers.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_pm4_headers_ai.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_pm4_headers_aldebaran.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_pm4_headers_diq.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_pm4_headers_vi.h+4−3

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_pm4_opcodes.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_priv.h+171−24

This diff is large: its 345 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_process.c+67−28

This diff is large: its 207 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c+371−4

This diff is large: its 422 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_queue.c+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c+3−2

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_smi_events.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_svm.c+380−57

This diff is large: its 601 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_svm.h+39−0

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_topology.c+5−4

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_topology.h+2−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c+61−47

This diff is large: its 352 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h+13−0

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c+92−30

This diff is large: its 335 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c+186−33

This diff is large: its 281 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c+280−9

This diff is large: its 369 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.h+14−0

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c+88−160

This diff is large: its 487 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_psr.c+2−4

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/Makefile+4−3

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c+0−2

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/bios/command_table2.c+0−4

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/bios/command_table_helper2.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/Makefile+10−0

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c+16−0

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c+5−2

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c+8−8

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_clk_mgr.c+13−13

This diff is large: its 78 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.c+20−7

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn31/dcn31_smu.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.c+706−0

This diff is large: its 707 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_clk_mgr.h+49−0

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_smu.c+328−0

This diff is large: its 329 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn316/dcn316_smu.h+128−0

This diff is large: its 129 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc.c+71−13

This diff is large: its 166 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_link.c+366−501

This diff is large: its 1545 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c+853−535

This diff is large: its 2393 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c+23−1

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c+0−917

This diff is large: its 918 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_resource.c+74−26

This diff is large: its 263 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_stream.c+16−0

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc.h+20−21

This diff is large: its 153 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_dp_types.h+17−42

This diff is large: its 240 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_helper.c+60−0

This diff is large: its 67 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_hw_types.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_link.h+14−7

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_stream.h+0−6

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_types.h+0−6

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_aux.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c+9−8

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.c+14−11

This diff is large: its 43 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.h+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c+4−0

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c+37−95

This diff is large: its 308 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c+2−7

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c+9−9

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c+0−15

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.h+0−3

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dccg.h+13−2

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c+9−32

This diff is large: its 63 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c+9−12

This diff is large: its 43 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn20/dcn20_stream_encoder.c+0−2

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c+0−2

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c+3−4

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn302/Makefile+0−12

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c+12−308

This diff is large: its 359 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.h+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn303/dcn303_resource.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dccg.c+81−25

This diff is large: its 256 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_dccg.h+6−0

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.c+11−0

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hpo_dp_stream_encoder.h+6−3

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hubbub.c+32−29

This diff is large: its 265 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_hwseq.c+21−7

This diff is large: its 83 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_optc.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn31/dcn31_resource.c+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn316/Makefilerenamed from drivers/gpu/drm/amd/display/dc/calcs/Makefile+12−24

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn316/dcn316_resource.c+2306−0

This diff is large: its 2307 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dcn316/dcn316_resource.h+42−0

This diff is large: its 43 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dm_helpers.h+3−3

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml/Makefile+11−1

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml/calcs/bw_fixed.crenamed from drivers/gpu/drm/amd/display/dc/calcs/bw_fixed.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml/calcs/calcs_logger.hrenamed from drivers/gpu/drm/amd/display/dc/calcs/calcs_logger.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml/calcs/custom_float.crenamed from drivers/gpu/drm/amd/display/dc/calcs/custom_float.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml/calcs/dce_calcs.crenamed from drivers/gpu/drm/amd/display/dc/calcs/dce_calcs.c+23−25

This diff is large: its 59 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml/calcs/dcn_calc_auto.crenamed from drivers/gpu/drm/amd/display/dc/calcs/dcn_calc_auto.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml/calcs/dcn_calc_auto.hrenamed from drivers/gpu/drm/amd/display/dc/calcs/dcn_calc_auto.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml/calcs/dcn_calc_math.crenamed from drivers/gpu/drm/amd/display/dc/calcs/dcn_calc_math.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml/calcs/dcn_calcs.crenamed from drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml/dcn302/dcn302_fpu.c+357−0

This diff is large: its 358 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml/dcn302/dcn302_fpu.h+32−0

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml/dcn31/display_mode_vba_31.c+26−3

This diff is large: its 88 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.c+24−0

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.h+3−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/gpio/dcn30/hw_factory_dcn30.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/gpio/dcn30/hw_translate_dcn30.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/gpio/hw_factory.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/gpio/hw_translate.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/core_types.h+1−12

This diff is large: its 62 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h+44−12

This diff is large: its 91 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/dccg.h+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h+0−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h+2−3

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/stream_encoder.h+4−6

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h+0−2

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw_sequencer_private.h+0−2

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/link_dpcd.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/link_hwss.h+44−42

This diff is large: its 95 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/reg_helper.h+34−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/resource.h+6−10

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/irq/dcn30/irq_service_dcn30.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/Makefile+30−0

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.c+137−0

This diff is large: its 138 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_hwss_dio.h+53−0

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.c+51−0

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_hwss_dpia.h+34−0

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.c+254−0

This diff is large: its 255 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_dp.h+35−0

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_frl.c+43−0

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_hwss_hpo_frl.h+34−0

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/virtual/Makefile+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/virtual/virtual_link_encoder.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/virtual/virtual_link_hwss.c+43−0

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/virtual/virtual_link_hwss.h+34−0

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/dmub_srv.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h+8−5

This diff is large: its 58 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/Makefile+2−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_dcn31.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_dcn316.c+62−0

This diff is large: its 63 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_dcn316.h+33−0

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c+4−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/bios_parser_types.h+0−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/dal_asic_id.h+6−1

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/dal_types.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/ddc_service_types.h+5−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/dpcd_defs.h+0−4

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/grph_object_defs.h+0−2

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/grph_object_id.h+0−2

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/link_service_types.h+0−12

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/inc/mod_info_packet.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c+13−12

This diff is large: its 58 lines aren’t shown here.

drivers/gpu/drm/amd/include/amd_shared.h+1−2

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_1_6_offset.h+15682−0

This diff is large: its 15683 lines aren’t shown here.

@@ -115,6 +115,7 @@ Code Seq# Include File Comments · 114 unmodified lines
115	'B' 00-1F linux/cciss_ioctl.h conflict!	115	'B' 00-1F linux/cciss_ioctl.h conflict!
116	'B' 00-0F include/linux/pmu.h conflict!	116	'B' 00-0F include/linux/pmu.h conflict!
117	'B' C0-FF advanced bbus <mailto:maassen@uni-freiburg.de>	117	'B' C0-FF advanced bbus <mailto:maassen@uni-freiburg.de>
		118	'B' 00-0F xen/xenbus_dev.h conflict!
118	'C' all linux/soundcard.h conflict!	119	'C' all linux/soundcard.h conflict!
119	'C' 01-2F linux/capi.h conflict!	120	'C' 01-2F linux/capi.h conflict!
120	'C' F0-FF drivers/net/wan/cosa.h conflict!	121	'C' F0-FF drivers/net/wan/cosa.h conflict!
@@ -134,6 +135,7 @@ Code Seq# Include File Comments · 13 unmodified lines
134	'F' 80-8F linux/arcfb.h conflict!	135	'F' 80-8F linux/arcfb.h conflict!
135	'F' DD video/sstfb.h conflict!	136	'F' DD video/sstfb.h conflict!
136	'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict!	137	'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict!
		138	'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict!
137	'H' 00-7F linux/hiddev.h conflict!	139	'H' 00-7F linux/hiddev.h conflict!
138	'H' 00-0F linux/hidraw.h conflict!	140	'H' 00-0F linux/hidraw.h conflict!
139	'H' 01 linux/mei.h conflict!	141	'H' 01 linux/mei.h conflict!
@@ -176,6 +178,7 @@ Code Seq# Include File Comments · 36 unmodified lines
176	'P' 60-6F sound/sscape_ioctl.h conflict!	178	'P' 60-6F sound/sscape_ioctl.h conflict!
177	'P' 00-0F drivers/usb/class/usblp.c conflict!	179	'P' 00-0F drivers/usb/class/usblp.c conflict!
178	'P' 01-09 drivers/misc/pci_endpoint_test.c conflict!	180	'P' 01-09 drivers/misc/pci_endpoint_test.c conflict!
		181	'P' 00-0F xen/privcmd.h conflict!
179	'Q' all linux/soundcard.h	182	'Q' all linux/soundcard.h
180	'R' 00-1F linux/random.h conflict!	183	'R' 00-1F linux/random.h conflict!
181	'R' 01 linux/rfkill.h conflict!	184	'R' 01 linux/rfkill.h conflict!