lael-labs/linux

Documentation/accel/amdxdna/amdnpu.rst+281−0

@@ -0,0 +1,281 @@
	1	.. SPDX-License-Identifier: GPL-2.0-only
	2
	3	.. include:: <isonum.txt>
	4
	5	=========
	6	AMD NPU
	7	=========
	8
	9	:Copyright: \|copy\| 2024 Advanced Micro Devices, Inc.
	10	:Author: Sonal Santan <sonal.santan@amd.com>
	11
	12	Overview
	13	========
	14
	15	AMD NPU (Neural Processing Unit) is a multi-user AI inference accelerator
	16	integrated into AMD client APU. NPU enables efficient execution of Machine
	17	Learning applications like CNN, LLM, etc. NPU is based on
	18	`AMD XDNA Architecture`_. NPU is managed by amdxdna driver.
	19
	20
	21	Hardware Description
	22	====================
	23
	24	AMD NPU consists of the following hardware components:
	25
	26	AMD XDNA Array
	27	--------------
	28
	29	AMD XDNA Array comprises of 2D array of compute and memory tiles built with
	30	`AMD AI Engine Technology`_. Each column has 4 rows of compute tiles and 1
	31	row of memory tile. Each compute tile contains a VLIW processor with its own
	32	dedicated program and data memory. The memory tile acts as L2 memory. The 2D
	33	array can be partitioned at a column boundary creating a spatially isolated
	34	partition which can be bound to a workload context.
	35
	36	Each column also has dedicated DMA engines to move data between host DDR and
	37	memory tile.
	38
	39	AMD Phoenix and AMD Hawk Point client NPU have a 4x5 topology, i.e., 4 rows of
	40	compute tiles arranged into 5 columns. AMD Strix Point client APU have 4x8
	41	topology, i.e., 4 rows of compute tiles arranged into 8 columns.
	42
	43	Shared L2 Memory
	44	----------------
	45
	46	The single row of memory tiles create a pool of software managed on chip L2
	47	memory. DMA engines are used to move data between host DDR and memory tiles.
	48	AMD Phoenix and AMD Hawk Point NPUs have a total of 2560 KB of L2 memory.
	49	AMD Strix Point NPU has a total of 4096 KB of L2 memory.
	50
	51	Microcontroller
	52	---------------
	53
	54	A microcontroller runs NPU Firmware which is responsible for command processing,
	55	XDNA Array partition setup, XDNA Array configuration, workload context
	56	management and workload orchestration.
	57
	58	NPU Firmware uses a dedicated instance of an isolated non-privileged context
	59	called ERT to service each workload context. ERT is also used to execute user
	60	provided ``ctrlcode`` associated with the workload context.
	61
	62	NPU Firmware uses a single isolated privileged context called MERT to service
	63	management commands from the amdxdna driver.
	64
	65	Mailboxes
	66	---------
	67
	68	The microcontroller and amdxdna driver use a privileged channel for management
	69	tasks like setting up of contexts, telemetry, query, error handling, setting up
	70	user channel, etc. As mentioned before, privileged channel requests are
	71	serviced by MERT. The privileged channel is bound to a single mailbox.
	72
	73	The microcontroller and amdxdna driver use a dedicated user channel per
	74	workload context. The user channel is primarily used for submitting work to
	75	the NPU. As mentioned before, a user channel requests are serviced by an
	76	instance of ERT. Each user channel is bound to its own dedicated mailbox.
	77
	78	PCIe EP
	79	-------
	80
	81	NPU is visible to the x86 host CPU as a PCIe device with multiple BARs and some
	82	MSI-X interrupt vectors. NPU uses a dedicated high bandwidth SoC level fabric
	83	for reading or writing into host memory. Each instance of ERT gets its own
	84	dedicated MSI-X interrupt. MERT gets a single instance of MSI-X interrupt.
	85
	86	The number of PCIe BARs varies depending on the specific device. Based on their
	87	functions, PCIe BARs can generally be categorized into the following types.
	88
	89	* PSP BAR: Expose the AMD PSP (Platform Security Processor) function
	90	* SMU BAR: Expose the AMD SMU (System Management Unit) function
	91	* SRAM BAR: Expose ring buffers for the mailbox
	92	* Mailbox BAR: Expose the mailbox control registers (head, tail and ISR
	93	registers etc.)
	94	* Public Register BAR: Expose public registers
	95
	96	On specific devices, the above-mentioned BAR type might be combined into a
	97	single physical PCIe BAR. Or a module might require two physical PCIe BARs to
	98	be fully functional. For example,
	99
	100	* On AMD Phoenix device, PSP, SMU, Public Register BARs are on PCIe BAR index 0.
	101	* On AMD Strix Point device, Mailbox and Public Register BARs are on PCIe BAR
	102	index 0. The PSP has some registers in PCIe BAR index 0 (Public Register BAR)
	103	and PCIe BAR index 4 (PSP BAR).
	104
	105	Process Isolation Hardware
	106	--------------------------
	107
	108	As explained before, XDNA Array can be dynamically divided into isolated
	109	spatial partitions, each of which may have one or more columns. The spatial
	110	partition is setup by programming the column isolation registers by the
	111	microcontroller. Each spatial partition is associated with a PASID which is
	112	also programmed by the microcontroller. Hence multiple spatial partitions in
	113	the NPU can make concurrent host access protected by PASID.
	114
	115	The NPU FW itself uses microcontroller MMU enforced isolated contexts for
	116	servicing user and privileged channel requests.
	117
	118
	119	Mixed Spatial and Temporal Scheduling
	120	=====================================
	121
	122	AMD XDNA architecture supports mixed spatial and temporal (time sharing)
	123	scheduling of 2D array. This means that spatial partitions may be setup and
	124	torn down dynamically to accommodate various workloads. A spatial partition
	125	may be exclusively bound to one workload context while another partition may
	126	be temporarily bound to more than one workload contexts. The microcontroller
	127	updates the PASID for a temporarily shared partition to match the context that
	128	has been bound to the partition at any moment.
	129
	130	Resource Solver
	131	---------------
	132
	133	The Resource Solver component of the amdxdna driver manages the allocation
	134	of 2D array among various workloads. Every workload describes the number
	135	of columns required to run the NPU binary in its metadata. The Resource Solver
	136	component uses hints passed by the workload and its own heuristics to
	137	decide 2D array (re)partition strategy and mapping of workloads for spatial and
	138	temporal sharing of columns. The FW enforces the context-to-column(s) resource
	139	binding decisions made by the Resource Solver.
	140
	141	AMD Phoenix and AMD Hawk Point client NPU can support 6 concurrent workload
	142	contexts. AMD Strix Point can support 16 concurrent workload contexts.
	143
	144
	145	Application Binaries
	146	====================
	147
	148	A NPU application workload is comprised of two separate binaries which are
	149	generated by the NPU compiler.
	150
	151	1. AMD XDNA Array overlay, which is used to configure a NPU spatial partition.
	152	The overlay contains instructions for setting up the stream switch
	153	configuration and ELF for the compute tiles. The overlay is loaded on the
	154	spatial partition bound to the workload by the associated ERT instance.
	155	Refer to the
	156	`Versal Adaptive SoC AIE-ML Architecture Manual (AM020)`_ for more details.
	157
	158	2. ``ctrlcode``, used for orchestrating the overlay loaded on the spatial
	159	partition. ``ctrlcode`` is executed by the ERT running in protected mode on
	160	the microcontroller in the context of the workload. ``ctrlcode`` is made up
	161	of a sequence of opcodes named ``XAie_TxnOpcode``. Refer to the
	162	`AI Engine Run Time`_ for more details.
	163
	164
	165	Special Host Buffers
	166	====================
	167
	168	Per-context Instruction Buffer
	169	------------------------------
	170
	171	Every workload context uses a host resident 64 MB buffer which is memory
	172	mapped into the ERT instance created to service the workload. The ``ctrlcode``
	173	used by the workload is copied into this special memory. This buffer is
	174	protected by PASID like all other input/output buffers used by that workload.
	175	Instruction buffer is also mapped into the user space of the workload.
	176
	177	Global Privileged Buffer
	178	------------------------
	179
	180	In addition, the driver also allocates a single buffer for maintenance tasks
	181	like recording errors from MERT. This global buffer uses the global IOMMU
	182	domain and is only accessible by MERT.
	183
	184
	185	High-level Use Flow
	186	===================
	187
	188	Here are the steps to run a workload on AMD NPU:
	189
	190	1. Compile the workload into an overlay and a ``ctrlcode`` binary.
	191	2. Userspace opens a context in the driver and provides the overlay.
	192	3. The driver checks with the Resource Solver for provisioning a set of columns
	193	for the workload.
	194	4. The driver then asks MERT to create a context on the device with the desired
	195	columns.
	196	5. MERT then creates an instance of ERT. MERT also maps the Instruction Buffer
	197	into ERT memory.
	198	6. The userspace then copies the ``ctrlcode`` to the Instruction Buffer.
	199	7. Userspace then creates a command buffer with pointers to input, output, and
	200	instruction buffer; it then submits command buffer with the driver and goes
	201	to sleep waiting for completion.
	202	8. The driver sends the command over the Mailbox to ERT.
	203	9. ERT executes the ``ctrlcode`` in the instruction buffer.
	204	10. Execution of the ``ctrlcode`` kicks off DMAs to and from the host DDR while
	205	AMD XDNA Array is running.
	206	11. When ERT reaches end of ``ctrlcode``, it raises an MSI-X to send completion
	207	signal to the driver which then wakes up the waiting workload.
	208
	209
	210	Boot Flow
	211	=========
	212
	213	amdxdna driver uses PSP to securely load signed NPU FW and kick off the boot
	214	of the NPU microcontroller. amdxdna driver then waits for the alive signal in
	215	a special location on BAR 0. The NPU is switched off during SoC suspend and
	216	turned on after resume where the NPU FW is reloaded, and the handshake is
	217	performed again.
	218
	219
	220	Userspace components
	221	====================
	222
	223	Compiler
	224	--------
	225
	226	Peano is an LLVM based open-source compiler for AMD XDNA Array compute tile
	227	available at:
	228	https://github.com/Xilinx/llvm-aie
	229
	230	The open-source IREE compiler supports graph compilation of ML models for AMD
	231	NPU and uses Peano underneath. It is available at:
	232	https://github.com/nod-ai/iree-amd-aie
	233
	234	Usermode Driver (UMD)
	235	---------------------
	236
	237	The open-source XRT runtime stack interfaces with amdxdna kernel driver. XRT
	238	can be found at:
	239	https://github.com/Xilinx/XRT
	240
	241	The open-source XRT shim for NPU is can be found at:
	242	https://github.com/amd/xdna-driver
	243
	244
	245	DMA Operation
	246	=============
	247
	248	DMA operation instructions are encoded in the ``ctrlcode`` as
	249	``XAIE_IO_BLOCKWRITE`` opcode. When ERT executes ``XAIE_IO_BLOCKWRITE``, DMA
	250	operations between host DDR and L2 memory are effected.
	251
	252
	253	Error Handling
	254	==============
	255
	256	When MERT detects an error in AMD XDNA Array, it pauses execution for that
	257	workload context and sends an asynchronous message to the driver over the
	258	privileged channel. The driver then sends a buffer pointer to MERT to capture
	259	the register states for the partition bound to faulting workload context. The
	260	driver then decodes the error by reading the contents of the buffer pointer.
	261
	262
	263	Telemetry
	264	=========
	265
	266	MERT can report various kinds of telemetry information like the following:
	267
	268	* L1 interrupt counter
	269	* DMA counter
	270	* Deep Sleep counter
	271	* etc.
	272
	273
	274	References
	275	==========
	276
	277	- `AMD XDNA Architecture <https://www.amd.com/en/technologies/xdna.html>`_
	278	- `AMD AI Engine Technology <https://www.xilinx.com/products/technology/ai-engine.html>`_
	279	- `Peano <https://github.com/Xilinx/llvm-aie>`_
	280	- `Versal Adaptive SoC AIE-ML Architecture Manual (AM020) <https://docs.amd.com/r/en-US/am020-versal-aie-ml>`_
	281	- `AI Engine Run Time <https://github.com/Xilinx/aie-rt/tree/release/main_aig>`_

Documentation/accel/amdxdna/index.rst+11−0

@@ -0,0 +1,11 @@
	1	.. SPDX-License-Identifier: GPL-2.0-only
	2
	3	=====================================
	4	accel/amdxdna NPU driver
	5	=====================================
	6
	7	The accel/amdxdna driver supports the AMD NPU (Neural Processing Unit).
	8
	9	.. toctree::
	10
	11	amdnpu

Documentation/accel/index.rst+1−0

@@ -8,6 +8,7 @@ Compute Accelerators · 7 unmodified lines
8	:maxdepth: 1	8	:maxdepth: 1
9		9
10	introduction	10	introduction
		11	amdxdna/index
11	qaic/index	12	qaic/index
12		13
13	.. only:: subproject and html	14	.. only:: subproject and html

Documentation/devicetree/bindings/display/brcm,bcm2711-hdmi.yaml+2−0

@@ -14,6 +14,8 @@ properties: · 13 unmodified lines
14	enum:	14	enum:
15	- brcm,bcm2711-hdmi0	15	- brcm,bcm2711-hdmi0
16	- brcm,bcm2711-hdmi1	16	- brcm,bcm2711-hdmi1
		17	- brcm,bcm2712-hdmi0
		18	- brcm,bcm2712-hdmi1
17		19
18	reg:	20	reg:
19	items:	21	items:

Documentation/devicetree/bindings/display/brcm,bcm2835-hvs.yaml+4−1

   compatible:
     enum:
       - brcm,bcm2711-hvs
+      - brcm,bcm2712-hvs
       - brcm,bcm2835-hvs
   reg:
   properties:
     compatible:
       contains:
-        const: brcm,bcm2711-hvs
+        enum:
+          - brcm,bcm2711-hvs
+          - brcm,bcm2712-hvs
 then:
   required:

Documentation/devicetree/bindings/display/brcm,bcm2835-pixelvalve0.yaml+3−0

       - brcm,bcm2711-pixelvalve2
       - brcm,bcm2711-pixelvalve3
       - brcm,bcm2711-pixelvalve4
+      - brcm,bcm2712-pixelvalve0
+      - brcm,bcm2712-pixelvalve1
+      - brcm,bcm2712-pixelvalve2
   reg:
     maxItems: 1

Documentation/devicetree/bindings/display/brcm,bcm2835-txp.yaml+4−1

 properties:
   compatible:
-    const: brcm,bcm2835-txp
+    enum:
+      - brcm,bcm2712-mop
+      - brcm,bcm2712-moplet
+      - brcm,bcm2835-txp
   reg:
     maxItems: 1

Documentation/devicetree/bindings/display/brcm,bcm2835-vc4.yaml+1−0

@@ -18,6 +18,7 @@ properties: · 17 unmodified lines
18	compatible:	18	compatible:
19	enum:	19	enum:
20	- brcm,bcm2711-vc5	20	- brcm,bcm2711-vc5
		21	- brcm,bcm2712-vc6
21	- brcm,bcm2835-vc4	22	- brcm,bcm2835-vc4
22	- brcm,cygnus-vc4	23	- brcm,cygnus-vc4
23		24

Documentation/devicetree/bindings/display/panel/samsung,atna33xc20.yaml+2−0

@@ -23,6 +23,8 @@ properties: · 22 unmodified lines
23	- samsung,atna45af01	23	- samsung,atna45af01
24	# Samsung 14.5" 3K (2944x1840 pixels) eDP AMOLED panel	24	# Samsung 14.5" 3K (2944x1840 pixels) eDP AMOLED panel
25	- samsung,atna45dc02	25	- samsung,atna45dc02
		26	# Samsung 15.6" 3K (2880x1620 pixels) eDP AMOLED panel
		27	- samsung,atna56ac03
26	- const: samsung,atna33xc20	28	- const: samsung,atna33xc20
27		29
28	enable-gpios: true	30	enable-gpios: true

Documentation/gpu/drm-kms-helpers.rst+3−0

@@ -221,6 +221,9 @@ Panel Helper Reference · 220 unmodified lines
221	.. kernel-doc:: drivers/gpu/drm/drm_panel_orientation_quirks.c	221	.. kernel-doc:: drivers/gpu/drm/drm_panel_orientation_quirks.c
222	:export:	222	:export:
223		223
		224	.. kernel-doc:: drivers/gpu/drm/drm_panel_backlight_quirks.c
		225	:export:
		226
224	Panel Self Refresh Helper Reference	227	Panel Self Refresh Helper Reference
225	===================================	228	===================================
226		229

MAINTAINERS+13−1

@@ -1194,6 +1194,17 @@ L: linux-spi@vger.kernel.org · 1193 unmodified lines
1194	S: Supported	1194	S: Supported
1195	F: drivers/spi/spi-amd.c	1195	F: drivers/spi/spi-amd.c
1196		1196
		1197	AMD XDNA DRIVER
		1198	M: Min Ma <min.ma@amd.com>
		1199	M: Lizhi Hou <lizhi.hou@amd.com>
		1200	L: dri-devel@lists.freedesktop.org
		1201	S: Supported
		1202	T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
		1203	F: Documentation/accel/amdxdna/
		1204	F: drivers/accel/amdxdna/
		1205	F: include/trace/events/amdxdna.h
		1206	F: include/uapi/drm/amdxdna_accel.h
		1207
1197	AMD XGBE DRIVER	1208	AMD XGBE DRIVER
1198	M: "Shyam Sundar S K" <Shyam-sundar.S-k@amd.com>	1209	M: "Shyam Sundar S K" <Shyam-sundar.S-k@amd.com>
1199	L: netdev@vger.kernel.org	1210	L: netdev@vger.kernel.org
@@ -7385,7 +7396,7 @@ L: virtualization@lists.linux.dev · 6185 unmodified lines
7385	S: Obsolete	7396	S: Obsolete
7386	W: https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/	7397	W: https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/
7387	T: git https://gitlab.freedesktop.org/drm/misc/kernel.git	7398	T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
7388	F: drivers/gpu/drm/tiny/cirrus.c	7399	F: drivers/gpu/drm/tiny/cirrus-qemu.c
7389		7400
7390	DRM DRIVER FOR QXL VIRTUAL GPU	7401	DRM DRIVER FOR QXL VIRTUAL GPU
7391	M: Dave Airlie <airlied@redhat.com>	7402	M: Dave Airlie <airlied@redhat.com>
@@ -7796,6 +7807,7 @@ F: drivers/gpu/drm/rockchip/ · 404 unmodified lines
7796		7807
7797	DRM DRIVERS FOR STI	7808	DRM DRIVERS FOR STI
7798	M: Alain Volmat <alain.volmat@foss.st.com>	7809	M: Alain Volmat <alain.volmat@foss.st.com>
		7810	M: Raphael Gallais-Pou <rgallaispou@gmail.com>
7799	L: dri-devel@lists.freedesktop.org	7811	L: dri-devel@lists.freedesktop.org
7800	S: Maintained	7812	S: Maintained
7801	T: git https://gitlab.freedesktop.org/drm/misc/kernel.git	7813	T: git https://gitlab.freedesktop.org/drm/misc/kernel.git

drivers/accel/Kconfig+1−0

@@ -24,6 +24,7 @@ menuconfig DRM_ACCEL · 23 unmodified lines
24	different device files, called accel/accel* (in /dev, sysfs	24	different device files, called accel/accel* (in /dev, sysfs
25	and debugfs).	25	and debugfs).
26		26
		27	source "drivers/accel/amdxdna/Kconfig"
27	source "drivers/accel/habanalabs/Kconfig"	28	source "drivers/accel/habanalabs/Kconfig"
28	source "drivers/accel/ivpu/Kconfig"	29	source "drivers/accel/ivpu/Kconfig"
29	source "drivers/accel/qaic/Kconfig"	30	source "drivers/accel/qaic/Kconfig"

drivers/accel/Makefile+1−0

@@ -1,5 +1,6 @@
1	# SPDX-License-Identifier: GPL-2.0-only	1	# SPDX-License-Identifier: GPL-2.0-only
2		2
		3	obj-$(CONFIG_DRM_ACCEL_AMDXDNA) += amdxdna/
3	obj-$(CONFIG_DRM_ACCEL_HABANALABS) += habanalabs/	4	obj-$(CONFIG_DRM_ACCEL_HABANALABS) += habanalabs/
4	obj-$(CONFIG_DRM_ACCEL_IVPU) += ivpu/	5	obj-$(CONFIG_DRM_ACCEL_IVPU) += ivpu/
5	obj-$(CONFIG_DRM_ACCEL_QAIC) += qaic/	6	obj-$(CONFIG_DRM_ACCEL_QAIC) += qaic/

drivers/accel/amdxdna/Kconfig+18−0

@@ -0,0 +1,18 @@
	1	# SPDX-License-Identifier: GPL-2.0-only
	2
	3	config DRM_ACCEL_AMDXDNA
	4	tristate "AMD AI Engine"
	5	depends on AMD_IOMMU
	6	depends on DRM_ACCEL
	7	depends on PCI && HAS_IOMEM
	8	depends on X86_64
	9	select DRM_SCHED
	10	select DRM_GEM_SHMEM_HELPER
	11	select FW_LOADER
	12	select HMM_MIRROR
	13	help
	14	Choose this option to enable support for NPU integrated into AMD
	15	client CPUs like AMD Ryzen AI 300 Series. AMD NPU can be used to
	16	accelerate machine learning applications.
	17
	18	If "M" is selected, the driver module will be amdxdna.

drivers/accel/amdxdna/Makefile+21−0

@@ -0,0 +1,21 @@
	1	# SPDX-License-Identifier: GPL-2.0-only
	2
	3	amdxdna-y := \
	4	aie2_ctx.o \
	5	aie2_error.o \
	6	aie2_message.o \
	7	aie2_pci.o \
	8	aie2_psp.o \
	9	aie2_smu.o \
	10	aie2_solver.o \
	11	amdxdna_ctx.o \
	12	amdxdna_gem.o \
	13	amdxdna_mailbox.o \
	14	amdxdna_mailbox_helper.o \
	15	amdxdna_pci_drv.o \
	16	amdxdna_sysfs.o \
	17	npu1_regs.o \
	18	npu2_regs.o \
	19	npu4_regs.o \
	20	npu5_regs.o
	21	obj-$(CONFIG_DRM_ACCEL_AMDXDNA) = amdxdna.o

drivers/accel/amdxdna/TODO+5−0

@@ -0,0 +1,5 @@
	1	- Replace idr with xa
	2	- Add import and export BO support
	3	- Add debugfs support
	4	- Add debug BO support
	5	- Improve power management

drivers/accel/amdxdna/aie2_ctx.c+900−0

@@ -0,0 +1,900 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/drm_gem.h>
	9	#include <drm/drm_gem_shmem_helper.h>
	10	#include <drm/drm_print.h>
	11	#include <drm/drm_syncobj.h>
	12	#include <linux/hmm.h>
	13	#include <linux/types.h>
	14	#include <trace/events/amdxdna.h>
	15
	16	#include "aie2_msg_priv.h"
	17	#include "aie2_pci.h"
	18	#include "aie2_solver.h"
	19	#include "amdxdna_ctx.h"
	20	#include "amdxdna_gem.h"
	21	#include "amdxdna_mailbox.h"
	22	#include "amdxdna_pci_drv.h"
	23
	24	bool force_cmdlist;
	25	module_param(force_cmdlist, bool, 0600);
	26	MODULE_PARM_DESC(force_cmdlist, "Force use command list (Default false)");
	27
	28	#define HWCTX_MAX_TIMEOUT 60000 /* milliseconds */
	29
	30	static void aie2_job_release(struct kref *ref)
	31	{
	32	struct amdxdna_sched_job *job;
	33
	34	job = container_of(ref, struct amdxdna_sched_job, refcnt);
	35	amdxdna_sched_job_cleanup(job);
	36	if (job->out_fence)
	37	dma_fence_put(job->out_fence);
	38	kfree(job);
	39	}
	40
	41	static void aie2_job_put(struct amdxdna_sched_job *job)
	42	{
	43	kref_put(&job->refcnt, aie2_job_release);
	44	}
	45
	46	/* The bad_job is used in aie2_sched_job_timedout, otherwise, set it to NULL */
	47	static void aie2_hwctx_stop(struct amdxdna_dev xdna, struct amdxdna_hwctx hwctx,
	48	struct drm_sched_job *bad_job)
	49	{
	50	drm_sched_stop(&hwctx->priv->sched, bad_job);
	51	aie2_destroy_context(xdna->dev_handle, hwctx);
	52	}
	53
	54	static int aie2_hwctx_restart(struct amdxdna_dev xdna, struct amdxdna_hwctx hwctx)
	55	{
	56	struct amdxdna_gem_obj *heap = hwctx->priv->heap;
	57	int ret;
	58
	59	ret = aie2_create_context(xdna->dev_handle, hwctx);
	60	if (ret) {
	61	XDNA_ERR(xdna, "Create hwctx failed, ret %d", ret);
	62	goto out;
	63	}
	64
	65	ret = aie2_map_host_buf(xdna->dev_handle, hwctx->fw_ctx_id,
	66	heap->mem.userptr, heap->mem.size);
	67	if (ret) {
	68	XDNA_ERR(xdna, "Map host buf failed, ret %d", ret);
	69	goto out;
	70	}
	71
	72	if (hwctx->status != HWCTX_STAT_READY) {
	73	XDNA_DBG(xdna, "hwctx is not ready, status %d", hwctx->status);
	74	goto out;
	75	}
	76
	77	ret = aie2_config_cu(hwctx);
	78	if (ret) {
	79	XDNA_ERR(xdna, "Config cu failed, ret %d", ret);
	80	goto out;
	81	}
	82
	83	out:
	84	drm_sched_start(&hwctx->priv->sched, 0);
	85	XDNA_DBG(xdna, "%s restarted, ret %d", hwctx->name, ret);
	86	return ret;
	87	}
	88
	89	void aie2_restart_ctx(struct amdxdna_client *client)
	90	{
	91	struct amdxdna_dev *xdna = client->xdna;
	92	struct amdxdna_hwctx *hwctx;
	93	int next = 0;
	94
	95	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	96	mutex_lock(&client->hwctx_lock);
	97	idr_for_each_entry_continue(&client->hwctx_idr, hwctx, next) {
	98	if (hwctx->status != HWCTX_STAT_STOP)
	99	continue;
	100
	101	hwctx->status = hwctx->old_status;
	102	XDNA_DBG(xdna, "Resetting %s", hwctx->name);
	103	aie2_hwctx_restart(xdna, hwctx);
	104	}
	105	mutex_unlock(&client->hwctx_lock);
	106	}
	107
	108	static struct dma_fence aie2_cmd_get_out_fence(struct amdxdna_hwctx hwctx, u64 seq)
	109	{
	110	struct dma_fence fence, out_fence = NULL;
	111	int ret;
	112
	113	fence = drm_syncobj_fence_get(hwctx->priv->syncobj);
	114	if (!fence)
	115	return NULL;
	116
	117	ret = dma_fence_chain_find_seqno(&fence, seq);
	118	if (ret)
	119	goto out;
	120
	121	out_fence = dma_fence_get(dma_fence_chain_contained(fence));
	122
	123	out:
	124	dma_fence_put(fence);
	125	return out_fence;
	126	}
	127
	128	static void aie2_hwctx_wait_for_idle(struct amdxdna_hwctx *hwctx)
	129	{
	130	struct dma_fence *fence;
	131
	132	fence = aie2_cmd_get_out_fence(hwctx, hwctx->priv->seq - 1);
	133	if (!fence)
	134	return;
	135
	136	dma_fence_wait(fence, false);
	137	dma_fence_put(fence);
	138	}
	139
	140	void aie2_hwctx_suspend(struct amdxdna_hwctx *hwctx)
	141	{
	142	struct amdxdna_dev *xdna = hwctx->client->xdna;
	143
	144	/*
	145	* Command timeout is unlikely. But if it happens, it doesn't
	146	* break the system. aie2_hwctx_stop() will destroy mailbox
	147	* and abort all commands.
	148	*/
	149	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	150	aie2_hwctx_wait_for_idle(hwctx);
	151	aie2_hwctx_stop(xdna, hwctx, NULL);
	152	hwctx->old_status = hwctx->status;
	153	hwctx->status = HWCTX_STAT_STOP;
	154	}
	155
	156	void aie2_hwctx_resume(struct amdxdna_hwctx *hwctx)
	157	{
	158	struct amdxdna_dev *xdna = hwctx->client->xdna;
	159
	160	/*
	161	* The resume path cannot guarantee that mailbox channel can be
	162	* regenerated. If this happen, when submit message to this
	163	* mailbox channel, error will return.
	164	*/
	165	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	166	hwctx->status = hwctx->old_status;
	167	aie2_hwctx_restart(xdna, hwctx);
	168	}
	169
	170	static void
	171	aie2_sched_notify(struct amdxdna_sched_job *job)
	172	{
	173	struct dma_fence *fence = job->fence;
	174
	175	trace_xdna_job(&job->base, job->hwctx->name, "signaled fence", job->seq);
	176	job->hwctx->priv->completed++;
	177	dma_fence_signal(fence);
	178
	179	up(&job->hwctx->priv->job_sem);
	180	job->job_done = true;
	181	dma_fence_put(fence);
	182	mmput(job->mm);
	183	aie2_job_put(job);
	184	}
	185
	186	static int
	187	aie2_sched_resp_handler(void handle, const u32 data, size_t size)
	188	{
	189	struct amdxdna_sched_job *job = handle;
	190	struct amdxdna_gem_obj *cmd_abo;
	191	u32 ret = 0;
	192	u32 status;
	193
	194	cmd_abo = job->cmd_bo;
	195
	196	if (unlikely(!data))
	197	goto out;
	198
	199	if (unlikely(size != sizeof(u32))) {
	200	amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT);
	201	ret = -EINVAL;
	202	goto out;
	203	}
	204
	205	status = *data;
	206	XDNA_DBG(job->hwctx->client->xdna, "Resp status 0x%x", status);
	207	if (status == AIE2_STATUS_SUCCESS)
	208	amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_COMPLETED);
	209	else
	210	amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ERROR);
	211
	212	out:
	213	aie2_sched_notify(job);
	214	return ret;
	215	}
	216
	217	static int
	218	aie2_sched_nocmd_resp_handler(void handle, const u32 data, size_t size)
	219	{
	220	struct amdxdna_sched_job *job = handle;
	221	u32 ret = 0;
	222	u32 status;
	223
	224	if (unlikely(!data))
	225	goto out;
	226
	227	if (unlikely(size != sizeof(u32))) {
	228	ret = -EINVAL;
	229	goto out;
	230	}
	231
	232	status = *data;
	233	XDNA_DBG(job->hwctx->client->xdna, "Resp status 0x%x", status);
	234
	235	out:
	236	aie2_sched_notify(job);
	237	return ret;
	238	}
	239
	240	static int
	241	aie2_sched_cmdlist_resp_handler(void handle, const u32 data, size_t size)
	242	{
	243	struct amdxdna_sched_job *job = handle;
	244	struct amdxdna_gem_obj *cmd_abo;
	245	struct cmd_chain_resp *resp;
	246	struct amdxdna_dev *xdna;
	247	u32 fail_cmd_status;
	248	u32 fail_cmd_idx;
	249	u32 ret = 0;
	250
	251	cmd_abo = job->cmd_bo;
	252	if (unlikely(!data) \|\| unlikely(size != sizeof(u32) * 3)) {
	253	amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT);
	254	ret = -EINVAL;
	255	goto out;
	256	}
	257
	258	resp = (struct cmd_chain_resp *)data;
	259	xdna = job->hwctx->client->xdna;
	260	XDNA_DBG(xdna, "Status 0x%x", resp->status);
	261	if (resp->status == AIE2_STATUS_SUCCESS) {
	262	amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_COMPLETED);
	263	goto out;
	264	}
	265
	266	/* Slow path to handle error, read from ringbuf on BAR */
	267	fail_cmd_idx = resp->fail_cmd_idx;
	268	fail_cmd_status = resp->fail_cmd_status;
	269	XDNA_DBG(xdna, "Failed cmd idx %d, status 0x%x",
	270	fail_cmd_idx, fail_cmd_status);
	271
	272	if (fail_cmd_status == AIE2_STATUS_SUCCESS) {
	273	amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT);
	274	ret = -EINVAL;
	275	goto out;
	276	}
	277	amdxdna_cmd_set_state(cmd_abo, fail_cmd_status);
	278
	279	if (amdxdna_cmd_get_op(cmd_abo) == ERT_CMD_CHAIN) {
	280	struct amdxdna_cmd_chain *cc = amdxdna_cmd_get_payload(cmd_abo, NULL);
	281
	282	cc->error_index = fail_cmd_idx;
	283	if (cc->error_index >= cc->command_count)
	284	cc->error_index = 0;
	285	}
	286	out:
	287	aie2_sched_notify(job);
	288	return ret;
	289	}
	290
	291	static struct dma_fence *
	292	aie2_sched_job_run(struct drm_sched_job *sched_job)
	293	{
	294	struct amdxdna_sched_job *job = drm_job_to_xdna_job(sched_job);
	295	struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;
	296	struct amdxdna_hwctx *hwctx = job->hwctx;
	297	struct dma_fence *fence;
	298	int ret;
	299
	300	if (!mmget_not_zero(job->mm))
	301	return ERR_PTR(-ESRCH);
	302
	303	kref_get(&job->refcnt);
	304	fence = dma_fence_get(job->fence);
	305
	306	if (unlikely(!cmd_abo)) {
	307	ret = aie2_sync_bo(hwctx, job, aie2_sched_nocmd_resp_handler);
	308	goto out;
	309	}
	310
	311	amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_NEW);
	312
	313	if (amdxdna_cmd_get_op(cmd_abo) == ERT_CMD_CHAIN)
	314	ret = aie2_cmdlist_multi_execbuf(hwctx, job, aie2_sched_cmdlist_resp_handler);
	315	else if (force_cmdlist)
	316	ret = aie2_cmdlist_single_execbuf(hwctx, job, aie2_sched_cmdlist_resp_handler);
	317	else
	318	ret = aie2_execbuf(hwctx, job, aie2_sched_resp_handler);
	319
	320	out:
	321	if (ret) {
	322	dma_fence_put(job->fence);
	323	aie2_job_put(job);
	324	mmput(job->mm);
	325	fence = ERR_PTR(ret);
	326	}
	327	trace_xdna_job(sched_job, hwctx->name, "sent to device", job->seq);
	328
	329	return fence;
	330	}
	331
	332	static void aie2_sched_job_free(struct drm_sched_job *sched_job)
	333	{
	334	struct amdxdna_sched_job *job = drm_job_to_xdna_job(sched_job);
	335	struct amdxdna_hwctx *hwctx = job->hwctx;
	336
	337	trace_xdna_job(sched_job, hwctx->name, "job free", job->seq);
	338	if (!job->job_done)
	339	up(&hwctx->priv->job_sem);
	340
	341	drm_sched_job_cleanup(sched_job);
	342	aie2_job_put(job);
	343	}
	344
	345	static enum drm_gpu_sched_stat
	346	aie2_sched_job_timedout(struct drm_sched_job *sched_job)
	347	{
	348	struct amdxdna_sched_job *job = drm_job_to_xdna_job(sched_job);
	349	struct amdxdna_hwctx *hwctx = job->hwctx;
	350	struct amdxdna_dev *xdna;
	351
	352	xdna = hwctx->client->xdna;
	353	trace_xdna_job(sched_job, hwctx->name, "job timedout", job->seq);
	354	mutex_lock(&xdna->dev_lock);
	355	aie2_hwctx_stop(xdna, hwctx, sched_job);
	356
	357	aie2_hwctx_restart(xdna, hwctx);
	358	mutex_unlock(&xdna->dev_lock);
	359
	360	return DRM_GPU_SCHED_STAT_NOMINAL;
	361	}
	362
	363	const struct drm_sched_backend_ops sched_ops = {
	364	.run_job = aie2_sched_job_run,
	365	.free_job = aie2_sched_job_free,
	366	.timedout_job = aie2_sched_job_timedout,
	367	};
	368
	369	static int aie2_hwctx_col_list(struct amdxdna_hwctx *hwctx)
	370	{
	371	struct amdxdna_dev *xdna = hwctx->client->xdna;
	372	struct amdxdna_dev_hdl *ndev;
	373	int start, end, first, last;
	374	u32 width = 1, entries = 0;
	375	int i;
	376
	377	if (!hwctx->num_tiles) {
	378	XDNA_ERR(xdna, "Number of tiles is zero");
	379	return -EINVAL;
	380	}
	381
	382	ndev = xdna->dev_handle;
	383	if (unlikely(!ndev->metadata.core.row_count)) {
	384	XDNA_WARN(xdna, "Core tile row count is zero");
	385	return -EINVAL;
	386	}
	387
	388	hwctx->num_col = hwctx->num_tiles / ndev->metadata.core.row_count;
	389	if (!hwctx->num_col \|\| hwctx->num_col > ndev->total_col) {
	390	XDNA_ERR(xdna, "Invalid num_col %d", hwctx->num_col);
	391	return -EINVAL;
	392	}
	393
	394	if (ndev->priv->col_align == COL_ALIGN_NATURE)
	395	width = hwctx->num_col;
	396
	397	/*
	398	* In range [start, end], find out columns that is multiple of width.
	399	* 'first' is the first column,
	400	* 'last' is the last column,
	401	* 'entries' is the total number of columns.
	402	*/
	403	start = xdna->dev_info->first_col;
	404	end = ndev->total_col - hwctx->num_col;
	405	if (start > 0 && end == 0) {
	406	XDNA_DBG(xdna, "Force start from col 0");
	407	start = 0;
	408	}
	409	first = start + (width - start % width) % width;
	410	last = end - end % width;
	411	if (last >= first)
	412	entries = (last - first) / width + 1;
	413	XDNA_DBG(xdna, "start %d end %d first %d last %d",
	414	start, end, first, last);
	415
	416	if (unlikely(!entries)) {
	417	XDNA_ERR(xdna, "Start %d end %d width %d",
	418	start, end, width);
	419	return -EINVAL;
	420	}
	421
	422	hwctx->col_list = kmalloc_array(entries, sizeof(*hwctx->col_list), GFP_KERNEL);
	423	if (!hwctx->col_list)
	424	return -ENOMEM;
	425
	426	hwctx->col_list_len = entries;
	427	hwctx->col_list[0] = first;
	428	for (i = 1; i < entries; i++)
	429	hwctx->col_list[i] = hwctx->col_list[i - 1] + width;
	430
	431	print_hex_dump_debug("col_list: ", DUMP_PREFIX_OFFSET, 16, 4, hwctx->col_list,
	432	entries * sizeof(*hwctx->col_list), false);
	433	return 0;
	434	}
	435
	436	static int aie2_alloc_resource(struct amdxdna_hwctx *hwctx)
	437	{
	438	struct amdxdna_dev *xdna = hwctx->client->xdna;
	439	struct alloc_requests *xrs_req;
	440	int ret;
	441
	442	xrs_req = kzalloc(sizeof(*xrs_req), GFP_KERNEL);
	443	if (!xrs_req)
	444	return -ENOMEM;
	445
	446	xrs_req->cdo.start_cols = hwctx->col_list;
	447	xrs_req->cdo.cols_len = hwctx->col_list_len;
	448	xrs_req->cdo.ncols = hwctx->num_col;
	449	xrs_req->cdo.qos_cap.opc = hwctx->max_opc;
	450
	451	xrs_req->rqos.gops = hwctx->qos.gops;
	452	xrs_req->rqos.fps = hwctx->qos.fps;
	453	xrs_req->rqos.dma_bw = hwctx->qos.dma_bandwidth;
	454	xrs_req->rqos.latency = hwctx->qos.latency;
	455	xrs_req->rqos.exec_time = hwctx->qos.frame_exec_time;
	456	xrs_req->rqos.priority = hwctx->qos.priority;
	457
	458	xrs_req->rid = (uintptr_t)hwctx;
	459
	460	ret = xrs_allocate_resource(xdna->xrs_hdl, xrs_req, hwctx);
	461	if (ret)
	462	XDNA_ERR(xdna, "Allocate AIE resource failed, ret %d", ret);
	463
	464	kfree(xrs_req);
	465	return ret;
	466	}
	467
	468	static void aie2_release_resource(struct amdxdna_hwctx *hwctx)
	469	{
	470	struct amdxdna_dev *xdna = hwctx->client->xdna;
	471	int ret;
	472
	473	ret = xrs_release_resource(xdna->xrs_hdl, (uintptr_t)hwctx);
	474	if (ret)
	475	XDNA_ERR(xdna, "Release AIE resource failed, ret %d", ret);
	476	}
	477
	478	static int aie2_ctx_syncobj_create(struct amdxdna_hwctx *hwctx)
	479	{
	480	struct amdxdna_dev *xdna = hwctx->client->xdna;
	481	struct drm_file *filp = hwctx->client->filp;
	482	struct drm_syncobj *syncobj;
	483	u32 hdl;
	484	int ret;
	485
	486	hwctx->syncobj_hdl = AMDXDNA_INVALID_FENCE_HANDLE;
	487
	488	ret = drm_syncobj_create(&syncobj, 0, NULL);
	489	if (ret) {
	490	XDNA_ERR(xdna, "Create ctx syncobj failed, ret %d", ret);
	491	return ret;
	492	}
	493	ret = drm_syncobj_get_handle(filp, syncobj, &hdl);
	494	if (ret) {
	495	drm_syncobj_put(syncobj);
	496	XDNA_ERR(xdna, "Create ctx syncobj handle failed, ret %d", ret);
	497	return ret;
	498	}
	499	hwctx->priv->syncobj = syncobj;
	500	hwctx->syncobj_hdl = hdl;
	501
	502	return 0;
	503	}
	504
	505	static void aie2_ctx_syncobj_destroy(struct amdxdna_hwctx *hwctx)
	506	{
	507	/*
	508	* The syncobj_hdl is owned by user space and will be cleaned up
	509	* separately.
	510	*/
	511	drm_syncobj_put(hwctx->priv->syncobj);
	512	}
	513
	514	int aie2_hwctx_init(struct amdxdna_hwctx *hwctx)
	515	{
	516	struct amdxdna_client *client = hwctx->client;
	517	struct amdxdna_dev *xdna = client->xdna;
	518	struct drm_gpu_scheduler *sched;
	519	struct amdxdna_hwctx_priv *priv;
	520	struct amdxdna_gem_obj *heap;
	521	int i, ret;
	522
	523	priv = kzalloc(sizeof(*hwctx->priv), GFP_KERNEL);
	524	if (!priv)
	525	return -ENOMEM;
	526	hwctx->priv = priv;
	527
	528	mutex_lock(&client->mm_lock);
	529	heap = client->dev_heap;
	530	if (!heap) {
	531	XDNA_ERR(xdna, "The client dev heap object not exist");
	532	mutex_unlock(&client->mm_lock);
	533	ret = -ENOENT;
	534	goto free_priv;
	535	}
	536	drm_gem_object_get(to_gobj(heap));
	537	mutex_unlock(&client->mm_lock);
	538	priv->heap = heap;
	539	sema_init(&priv->job_sem, HWCTX_MAX_CMDS);
	540
	541	ret = amdxdna_gem_pin(heap);
	542	if (ret) {
	543	XDNA_ERR(xdna, "Dev heap pin failed, ret %d", ret);
	544	goto put_heap;
	545	}
	546
	547	for (i = 0; i < ARRAY_SIZE(priv->cmd_buf); i++) {
	548	struct amdxdna_gem_obj *abo;
	549	struct amdxdna_drm_create_bo args = {
	550	.flags = 0,
	551	.type = AMDXDNA_BO_DEV,
	552	.vaddr = 0,
	553	.size = MAX_CHAIN_CMDBUF_SIZE,
	554	};
	555
	556	abo = amdxdna_drm_alloc_dev_bo(&xdna->ddev, &args, client->filp, true);
	557	if (IS_ERR(abo)) {
	558	ret = PTR_ERR(abo);
	559	goto free_cmd_bufs;
	560	}
	561
	562	XDNA_DBG(xdna, "Command buf %d addr 0x%llx size 0x%lx",
	563	i, abo->mem.dev_addr, abo->mem.size);
	564	priv->cmd_buf[i] = abo;
	565	}
	566
	567	sched = &priv->sched;
	568	mutex_init(&priv->io_lock);
	569
	570	fs_reclaim_acquire(GFP_KERNEL);
	571	might_lock(&priv->io_lock);
	572	fs_reclaim_release(GFP_KERNEL);
	573
	574	ret = drm_sched_init(sched, &sched_ops, NULL, DRM_SCHED_PRIORITY_COUNT,
	575	HWCTX_MAX_CMDS, 0, msecs_to_jiffies(HWCTX_MAX_TIMEOUT),
	576	NULL, NULL, hwctx->name, xdna->ddev.dev);
	577	if (ret) {
	578	XDNA_ERR(xdna, "Failed to init DRM scheduler. ret %d", ret);
	579	goto free_cmd_bufs;
	580	}
	581
	582	ret = drm_sched_entity_init(&priv->entity, DRM_SCHED_PRIORITY_NORMAL,
	583	&sched, 1, NULL);
	584	if (ret) {
	585	XDNA_ERR(xdna, "Failed to initial sched entiry. ret %d", ret);
	586	goto free_sched;
	587	}
	588
	589	ret = aie2_hwctx_col_list(hwctx);
	590	if (ret) {
	591	XDNA_ERR(xdna, "Create col list failed, ret %d", ret);
	592	goto free_entity;
	593	}
	594
	595	ret = aie2_alloc_resource(hwctx);
	596	if (ret) {
	597	XDNA_ERR(xdna, "Alloc hw resource failed, ret %d", ret);
	598	goto free_col_list;
	599	}
	600
	601	ret = aie2_map_host_buf(xdna->dev_handle, hwctx->fw_ctx_id,
	602	heap->mem.userptr, heap->mem.size);
	603	if (ret) {
	604	XDNA_ERR(xdna, "Map host buffer failed, ret %d", ret);
	605	goto release_resource;
	606	}
	607
	608	ret = aie2_ctx_syncobj_create(hwctx);
	609	if (ret) {
	610	XDNA_ERR(xdna, "Create syncobj failed, ret %d", ret);
	611	goto release_resource;
	612	}
	613
	614	hwctx->status = HWCTX_STAT_INIT;
	615
	616	XDNA_DBG(xdna, "hwctx %s init completed", hwctx->name);
	617
	618	return 0;
	619
	620	release_resource:
	621	aie2_release_resource(hwctx);
	622	free_col_list:
	623	kfree(hwctx->col_list);
	624	free_entity:
	625	drm_sched_entity_destroy(&priv->entity);
	626	free_sched:
	627	drm_sched_fini(&priv->sched);
	628	free_cmd_bufs:
	629	for (i = 0; i < ARRAY_SIZE(priv->cmd_buf); i++) {
	630	if (!priv->cmd_buf[i])
	631	continue;
	632	drm_gem_object_put(to_gobj(priv->cmd_buf[i]));
	633	}
	634	amdxdna_gem_unpin(heap);
	635	put_heap:
	636	drm_gem_object_put(to_gobj(heap));
	637	free_priv:
	638	kfree(priv);
	639	return ret;
	640	}
	641
	642	void aie2_hwctx_fini(struct amdxdna_hwctx *hwctx)
	643	{
	644	struct amdxdna_dev *xdna;
	645	int idx;
	646
	647	xdna = hwctx->client->xdna;
	648	drm_sched_wqueue_stop(&hwctx->priv->sched);
	649
	650	/* Now, scheduler will not send command to device. */
	651	aie2_release_resource(hwctx);
	652
	653	/*
	654	* All submitted commands are aborted.
	655	* Restart scheduler queues to cleanup jobs. The amdxdna_sched_job_run()
	656	* will return NODEV if it is called.
	657	*/
	658	drm_sched_wqueue_start(&hwctx->priv->sched);
	659
	660	aie2_hwctx_wait_for_idle(hwctx);
	661	drm_sched_entity_destroy(&hwctx->priv->entity);
	662	drm_sched_fini(&hwctx->priv->sched);
	663	aie2_ctx_syncobj_destroy(hwctx);
	664
	665	XDNA_DBG(xdna, "%s sequence number %lld", hwctx->name, hwctx->priv->seq);
	666
	667	for (idx = 0; idx < ARRAY_SIZE(hwctx->priv->cmd_buf); idx++)
	668	drm_gem_object_put(to_gobj(hwctx->priv->cmd_buf[idx]));
	669	amdxdna_gem_unpin(hwctx->priv->heap);
	670	drm_gem_object_put(to_gobj(hwctx->priv->heap));
	671
	672	mutex_destroy(&hwctx->priv->io_lock);
	673	kfree(hwctx->col_list);
	674	kfree(hwctx->priv);
	675	kfree(hwctx->cus);
	676	}
	677
	678	static int aie2_hwctx_cu_config(struct amdxdna_hwctx hwctx, void buf, u32 size)
	679	{
	680	struct amdxdna_hwctx_param_config_cu *config = buf;
	681	struct amdxdna_dev *xdna = hwctx->client->xdna;
	682	u32 total_size;
	683	int ret;
	684
	685	XDNA_DBG(xdna, "Config %d CU to %s", config->num_cus, hwctx->name);
	686	if (hwctx->status != HWCTX_STAT_INIT) {
	687	XDNA_ERR(xdna, "Not support re-config CU");
	688	return -EINVAL;
	689	}
	690
	691	if (!config->num_cus) {
	692	XDNA_ERR(xdna, "Number of CU is zero");
	693	return -EINVAL;
	694	}
	695
	696	total_size = struct_size(config, cu_configs, config->num_cus);
	697	if (total_size > size) {
	698	XDNA_ERR(xdna, "CU config larger than size");
	699	return -EINVAL;
	700	}
	701
	702	hwctx->cus = kmemdup(config, total_size, GFP_KERNEL);
	703	if (!hwctx->cus)
	704	return -ENOMEM;
	705
	706	ret = aie2_config_cu(hwctx);
	707	if (ret) {
	708	XDNA_ERR(xdna, "Config CU to firmware failed, ret %d", ret);
	709	goto free_cus;
	710	}
	711
	712	wmb(); /* To avoid locking in command submit when check status */
	713	hwctx->status = HWCTX_STAT_READY;
	714
	715	return 0;
	716
	717	free_cus:
	718	kfree(hwctx->cus);
	719	hwctx->cus = NULL;
	720	return ret;
	721	}
	722
	723	int aie2_hwctx_config(struct amdxdna_hwctx hwctx, u32 type, u64 value, void buf, u32 size)
	724	{
	725	struct amdxdna_dev *xdna = hwctx->client->xdna;
	726
	727	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	728	switch (type) {
	729	case DRM_AMDXDNA_HWCTX_CONFIG_CU:
	730	return aie2_hwctx_cu_config(hwctx, buf, size);
	731	case DRM_AMDXDNA_HWCTX_ASSIGN_DBG_BUF:
	732	case DRM_AMDXDNA_HWCTX_REMOVE_DBG_BUF:
	733	return -EOPNOTSUPP;
	734	default:
	735	XDNA_DBG(xdna, "Not supported type %d", type);
	736	return -EOPNOTSUPP;
	737	}
	738	}
	739
	740	static int aie2_populate_range(struct amdxdna_gem_obj *abo)
	741	{
	742	struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev);
	743	struct mm_struct *mm = abo->mem.notifier.mm;
	744	struct hmm_range range = { 0 };
	745	unsigned long timeout;
	746	int ret;
	747
	748	XDNA_INFO_ONCE(xdna, "populate memory range %llx size %lx",
	749	abo->mem.userptr, abo->mem.size);
	750	range.notifier = &abo->mem.notifier;
	751	range.start = abo->mem.userptr;
	752	range.end = abo->mem.userptr + abo->mem.size;
	753	range.hmm_pfns = abo->mem.pfns;
	754	range.default_flags = HMM_PFN_REQ_FAULT;
	755
	756	if (!mmget_not_zero(mm))
	757	return -EFAULT;
	758
	759	timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
	760	again:
	761	range.notifier_seq = mmu_interval_read_begin(&abo->mem.notifier);
	762	mmap_read_lock(mm);
	763	ret = hmm_range_fault(&range);
	764	mmap_read_unlock(mm);
	765	if (ret) {
	766	if (time_after(jiffies, timeout)) {
	767	ret = -ETIME;
	768	goto put_mm;
	769	}
	770
	771	if (ret == -EBUSY)
	772	goto again;
	773
	774	goto put_mm;
	775	}
	776
	777	down_read(&xdna->notifier_lock);
	778	if (mmu_interval_read_retry(&abo->mem.notifier, range.notifier_seq)) {
	779	up_read(&xdna->notifier_lock);
	780	goto again;
	781	}
	782	abo->mem.map_invalid = false;
	783	up_read(&xdna->notifier_lock);
	784
	785	put_mm:
	786	mmput(mm);
	787	return ret;
	788	}
	789
	790	int aie2_cmd_submit(struct amdxdna_hwctx hwctx, struct amdxdna_sched_job job, u64 *seq)
	791	{
	792	struct amdxdna_dev *xdna = hwctx->client->xdna;
	793	struct ww_acquire_ctx acquire_ctx;
	794	struct dma_fence_chain *chain;
	795	struct amdxdna_gem_obj *abo;
	796	unsigned long timeout = 0;
	797	int ret, i;
	798
	799	ret = down_interruptible(&hwctx->priv->job_sem);
	800	if (ret) {
	801	XDNA_ERR(xdna, "Grab job sem failed, ret %d", ret);
	802	return ret;
	803	}
	804
	805	chain = dma_fence_chain_alloc();
	806	if (!chain) {
	807	XDNA_ERR(xdna, "Alloc fence chain failed");
	808	ret = -ENOMEM;
	809	goto up_sem;
	810	}
	811
	812	ret = drm_sched_job_init(&job->base, &hwctx->priv->entity, 1, hwctx);
	813	if (ret) {
	814	XDNA_ERR(xdna, "DRM job init failed, ret %d", ret);
	815	goto free_chain;
	816	}
	817
	818	retry:
	819	ret = drm_gem_lock_reservations(job->bos, job->bo_cnt, &acquire_ctx);
	820	if (ret) {
	821	XDNA_WARN(xdna, "Failed to lock BOs, ret %d", ret);
	822	goto cleanup_job;
	823	}
	824
	825	for (i = 0; i < job->bo_cnt; i++) {
	826	ret = dma_resv_reserve_fences(job->bos[i]->resv, 1);
	827	if (ret) {
	828	XDNA_WARN(xdna, "Failed to reserve fences %d", ret);
	829	drm_gem_unlock_reservations(job->bos, job->bo_cnt, &acquire_ctx);
	830	goto cleanup_job;
	831	}
	832	}
	833
	834	down_read(&xdna->notifier_lock);
	835	for (i = 0; i < job->bo_cnt; i++) {
	836	abo = to_xdna_obj(job->bos[i]);
	837	if (abo->mem.map_invalid) {
	838	up_read(&xdna->notifier_lock);
	839	drm_gem_unlock_reservations(job->bos, job->bo_cnt, &acquire_ctx);
	840	if (!timeout) {
	841	timeout = jiffies +
	842	msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
	843	} else if (time_after(jiffies, timeout)) {
	844	ret = -ETIME;
	845	goto cleanup_job;
	846	}
	847
	848	ret = aie2_populate_range(abo);
	849	if (ret)
	850	goto cleanup_job;
	851	goto retry;
	852	}
	853	}
	854
	855	mutex_lock(&hwctx->priv->io_lock);
	856	drm_sched_job_arm(&job->base);
	857	job->out_fence = dma_fence_get(&job->base.s_fence->finished);
	858	for (i = 0; i < job->bo_cnt; i++)
	859	dma_resv_add_fence(job->bos[i]->resv, job->out_fence, DMA_RESV_USAGE_WRITE);
	860	job->seq = hwctx->priv->seq++;
	861	kref_get(&job->refcnt);
	862	drm_sched_entity_push_job(&job->base);
	863
	864	*seq = job->seq;
	865	drm_syncobj_add_point(hwctx->priv->syncobj, chain, job->out_fence, *seq);
	866	mutex_unlock(&hwctx->priv->io_lock);
	867
	868	up_read(&xdna->notifier_lock);
	869	drm_gem_unlock_reservations(job->bos, job->bo_cnt, &acquire_ctx);
	870
	871	aie2_job_put(job);
	872
	873	return 0;
	874
	875	cleanup_job:
	876	drm_sched_job_cleanup(&job->base);
	877	free_chain:
	878	dma_fence_chain_free(chain);
	879	up_sem:
	880	up(&hwctx->priv->job_sem);
	881	job->job_done = true;
	882	return ret;
	883	}
	884
	885	void aie2_hmm_invalidate(struct amdxdna_gem_obj *abo,
	886	unsigned long cur_seq)
	887	{
	888	struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev);
	889	struct drm_gem_object *gobj = to_gobj(abo);
	890	long ret;
	891
	892	down_write(&xdna->notifier_lock);
	893	abo->mem.map_invalid = true;
	894	mmu_interval_set_seq(&abo->mem.notifier, cur_seq);
	895	up_write(&xdna->notifier_lock);
	896	ret = dma_resv_wait_timeout(gobj->resv, DMA_RESV_USAGE_BOOKKEEP,
	897	true, MAX_SCHEDULE_TIMEOUT);
	898	if (!ret \|\| ret == -ERESTARTSYS)
	899	XDNA_ERR(xdna, "Failed to wait for bo, ret %ld", ret);
	900	}

drivers/accel/amdxdna/aie2_error.c+360−0

@@ -0,0 +1,360 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/drm_cache.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/drm_print.h>
	9	#include <drm/gpu_scheduler.h>
	10	#include <linux/dma-mapping.h>
	11	#include <linux/kthread.h>
	12	#include <linux/kernel.h>
	13
	14	#include "aie2_msg_priv.h"
	15	#include "aie2_pci.h"
	16	#include "amdxdna_mailbox.h"
	17	#include "amdxdna_pci_drv.h"
	18
	19	struct async_event {
	20	struct amdxdna_dev_hdl *ndev;
	21	struct async_event_msg_resp resp;
	22	struct workqueue_struct *wq;
	23	struct work_struct work;
	24	u8 *buf;
	25	dma_addr_t addr;
	26	u32 size;
	27	};
	28
	29	struct async_events {
	30	struct workqueue_struct *wq;
	31	u8 *buf;
	32	dma_addr_t addr;
	33	u32 size;
	34	u32 event_cnt;
	35	struct async_event event[] __counted_by(event_cnt);
	36	};
	37
	38	/*
	39	* Below enum, struct and lookup tables are porting from XAIE util header file.
	40	*
	41	* Below data is defined by AIE device and it is used for decode error message
	42	* from the device.
	43	*/
	44
	45	enum aie_module_type {
	46	AIE_MEM_MOD = 0,
	47	AIE_CORE_MOD,
	48	AIE_PL_MOD,
	49	};
	50
	51	enum aie_error_category {
	52	AIE_ERROR_SATURATION = 0,
	53	AIE_ERROR_FP,
	54	AIE_ERROR_STREAM,
	55	AIE_ERROR_ACCESS,
	56	AIE_ERROR_BUS,
	57	AIE_ERROR_INSTRUCTION,
	58	AIE_ERROR_ECC,
	59	AIE_ERROR_LOCK,
	60	AIE_ERROR_DMA,
	61	AIE_ERROR_MEM_PARITY,
	62	/* Unknown is not from XAIE, added for better category */
	63	AIE_ERROR_UNKNOWN,
	64	};
	65
	66	/* Don't pack, unless XAIE side changed */
	67	struct aie_error {
	68	__u8 row;
	69	__u8 col;
	70	__u32 mod_type;
	71	__u8 event_id;
	72	};
	73
	74	struct aie_err_info {
	75	u32 err_cnt;
	76	u32 ret_code;
	77	u32 rsvd;
	78	struct aie_error payload[] __counted_by(err_cnt);
	79	};
	80
	81	struct aie_event_category {
	82	u8 event_id;
	83	enum aie_error_category category;
	84	};
	85
	86	#define EVENT_CATEGORY(id, cat) { id, cat }
	87	static const struct aie_event_category aie_ml_mem_event_cat[] = {
	88	EVENT_CATEGORY(88U, AIE_ERROR_ECC),
	89	EVENT_CATEGORY(90U, AIE_ERROR_ECC),
	90	EVENT_CATEGORY(91U, AIE_ERROR_MEM_PARITY),
	91	EVENT_CATEGORY(92U, AIE_ERROR_MEM_PARITY),
	92	EVENT_CATEGORY(93U, AIE_ERROR_MEM_PARITY),
	93	EVENT_CATEGORY(94U, AIE_ERROR_MEM_PARITY),
	94	EVENT_CATEGORY(95U, AIE_ERROR_MEM_PARITY),
	95	EVENT_CATEGORY(96U, AIE_ERROR_MEM_PARITY),
	96	EVENT_CATEGORY(97U, AIE_ERROR_DMA),
	97	EVENT_CATEGORY(98U, AIE_ERROR_DMA),
	98	EVENT_CATEGORY(99U, AIE_ERROR_DMA),
	99	EVENT_CATEGORY(100U, AIE_ERROR_DMA),
	100	EVENT_CATEGORY(101U, AIE_ERROR_LOCK),
	101	};
	102
	103	static const struct aie_event_category aie_ml_core_event_cat[] = {
	104	EVENT_CATEGORY(55U, AIE_ERROR_ACCESS),
	105	EVENT_CATEGORY(56U, AIE_ERROR_STREAM),
	106	EVENT_CATEGORY(57U, AIE_ERROR_STREAM),
	107	EVENT_CATEGORY(58U, AIE_ERROR_BUS),
	108	EVENT_CATEGORY(59U, AIE_ERROR_INSTRUCTION),
	109	EVENT_CATEGORY(60U, AIE_ERROR_ACCESS),
	110	EVENT_CATEGORY(62U, AIE_ERROR_ECC),
	111	EVENT_CATEGORY(64U, AIE_ERROR_ECC),
	112	EVENT_CATEGORY(65U, AIE_ERROR_ACCESS),
	113	EVENT_CATEGORY(66U, AIE_ERROR_ACCESS),
	114	EVENT_CATEGORY(67U, AIE_ERROR_LOCK),
	115	EVENT_CATEGORY(70U, AIE_ERROR_INSTRUCTION),
	116	EVENT_CATEGORY(71U, AIE_ERROR_STREAM),
	117	EVENT_CATEGORY(72U, AIE_ERROR_BUS),
	118	};
	119
	120	static const struct aie_event_category aie_ml_mem_tile_event_cat[] = {
	121	EVENT_CATEGORY(130U, AIE_ERROR_ECC),
	122	EVENT_CATEGORY(132U, AIE_ERROR_ECC),
	123	EVENT_CATEGORY(133U, AIE_ERROR_DMA),
	124	EVENT_CATEGORY(134U, AIE_ERROR_DMA),
	125	EVENT_CATEGORY(135U, AIE_ERROR_STREAM),
	126	EVENT_CATEGORY(136U, AIE_ERROR_STREAM),
	127	EVENT_CATEGORY(137U, AIE_ERROR_STREAM),
	128	EVENT_CATEGORY(138U, AIE_ERROR_BUS),
	129	EVENT_CATEGORY(139U, AIE_ERROR_LOCK),
	130	};
	131
	132	static const struct aie_event_category aie_ml_shim_tile_event_cat[] = {
	133	EVENT_CATEGORY(64U, AIE_ERROR_BUS),
	134	EVENT_CATEGORY(65U, AIE_ERROR_STREAM),
	135	EVENT_CATEGORY(66U, AIE_ERROR_STREAM),
	136	EVENT_CATEGORY(67U, AIE_ERROR_BUS),
	137	EVENT_CATEGORY(68U, AIE_ERROR_BUS),
	138	EVENT_CATEGORY(69U, AIE_ERROR_BUS),
	139	EVENT_CATEGORY(70U, AIE_ERROR_BUS),
	140	EVENT_CATEGORY(71U, AIE_ERROR_BUS),
	141	EVENT_CATEGORY(72U, AIE_ERROR_DMA),
	142	EVENT_CATEGORY(73U, AIE_ERROR_DMA),
	143	EVENT_CATEGORY(74U, AIE_ERROR_LOCK),
	144	};
	145
	146	static enum aie_error_category
	147	aie_get_error_category(u8 row, u8 event_id, enum aie_module_type mod_type)
	148	{
	149	const struct aie_event_category *lut;
	150	int num_entry;
	151	int i;
	152
	153	switch (mod_type) {
	154	case AIE_PL_MOD:
	155	lut = aie_ml_shim_tile_event_cat;
	156	num_entry = ARRAY_SIZE(aie_ml_shim_tile_event_cat);
	157	break;
	158	case AIE_CORE_MOD:
	159	lut = aie_ml_core_event_cat;
	160	num_entry = ARRAY_SIZE(aie_ml_core_event_cat);
	161	break;
	162	case AIE_MEM_MOD:
	163	if (row == 1) {
	164	lut = aie_ml_mem_tile_event_cat;
	165	num_entry = ARRAY_SIZE(aie_ml_mem_tile_event_cat);
	166	} else {
	167	lut = aie_ml_mem_event_cat;
	168	num_entry = ARRAY_SIZE(aie_ml_mem_event_cat);
	169	}
	170	break;
	171	default:
	172	return AIE_ERROR_UNKNOWN;
	173	}
	174
	175	for (i = 0; i < num_entry; i++) {
	176	if (event_id != lut[i].event_id)
	177	continue;
	178
	179	return lut[i].category;
	180	}
	181
	182	return AIE_ERROR_UNKNOWN;
	183	}
	184
	185	static u32 aie2_error_backtrack(struct amdxdna_dev_hdl ndev, void err_info, u32 num_err)
	186	{
	187	struct aie_error *errs = err_info;
	188	u32 err_col = 0; /* assume that AIE has less than 32 columns */
	189	int i;
	190
	191	/* Get err column bitmap */
	192	for (i = 0; i < num_err; i++) {
	193	struct aie_error *err = &errs[i];
	194	enum aie_error_category cat;
	195
	196	cat = aie_get_error_category(err->row, err->event_id, err->mod_type);
	197	XDNA_ERR(ndev->xdna, "Row: %d, Col: %d, module %d, event ID %d, category %d",
	198	err->row, err->col, err->mod_type,
	199	err->event_id, cat);
	200
	201	if (err->col >= 32) {
	202	XDNA_WARN(ndev->xdna, "Invalid column number");
	203	break;
	204	}
	205
	206	err_col \|= (1 << err->col);
	207	}
	208
	209	return err_col;
	210	}
	211
	212	static int aie2_error_async_cb(void handle, const u32 data, size_t size)
	213	{
	214	struct async_event_msg_resp *resp;
	215	struct async_event *e = handle;
	216
	217	if (data) {
	218	resp = (struct async_event_msg_resp *)data;
	219	e->resp.type = resp->type;
	220	wmb(); /* Update status in the end, so that no lock for here */
	221	e->resp.status = resp->status;
	222	}
	223	queue_work(e->wq, &e->work);
	224	return 0;
	225	}
	226
	227	static int aie2_error_event_send(struct async_event *e)
	228	{
	229	drm_clflush_virt_range(e->buf, e->size); /* device can access */
	230	return aie2_register_asyn_event_msg(e->ndev, e->addr, e->size, e,
	231	aie2_error_async_cb);
	232	}
	233
	234	static void aie2_error_worker(struct work_struct *err_work)
	235	{
	236	struct aie_err_info *info;
	237	struct amdxdna_dev *xdna;
	238	struct async_event *e;
	239	u32 max_err;
	240	u32 err_col;
	241
	242	e = container_of(err_work, struct async_event, work);
	243
	244	xdna = e->ndev->xdna;
	245
	246	if (e->resp.status == MAX_AIE2_STATUS_CODE)
	247	return;
	248
	249	e->resp.status = MAX_AIE2_STATUS_CODE;
	250
	251	print_hex_dump_debug("AIE error: ", DUMP_PREFIX_OFFSET, 16, 4,
	252	e->buf, 0x100, false);
	253
	254	info = (struct aie_err_info *)e->buf;
	255	XDNA_DBG(xdna, "Error count %d return code %d", info->err_cnt, info->ret_code);
	256
	257	max_err = (e->size - sizeof(*info)) / sizeof(struct aie_error);
	258	if (unlikely(info->err_cnt > max_err)) {
	259	WARN_ONCE(1, "Error count too large %d\n", info->err_cnt);
	260	return;
	261	}
	262	err_col = aie2_error_backtrack(e->ndev, info->payload, info->err_cnt);
	263	if (!err_col) {
	264	XDNA_WARN(xdna, "Did not get error column");
	265	return;
	266	}
	267
	268	mutex_lock(&xdna->dev_lock);
	269	/* Re-sent this event to firmware */
	270	if (aie2_error_event_send(e))
	271	XDNA_WARN(xdna, "Unable to register async event");
	272	mutex_unlock(&xdna->dev_lock);
	273	}
	274
	275	int aie2_error_async_events_send(struct amdxdna_dev_hdl *ndev)
	276	{
	277	struct amdxdna_dev *xdna = ndev->xdna;
	278	struct async_event *e;
	279	int i, ret;
	280
	281	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	282	for (i = 0; i < ndev->async_events->event_cnt; i++) {
	283	e = &ndev->async_events->event[i];
	284	ret = aie2_error_event_send(e);
	285	if (ret)
	286	return ret;
	287	}
	288
	289	return 0;
	290	}
	291
	292	void aie2_error_async_events_free(struct amdxdna_dev_hdl *ndev)
	293	{
	294	struct amdxdna_dev *xdna = ndev->xdna;
	295	struct async_events *events;
	296
	297	events = ndev->async_events;
	298
	299	mutex_unlock(&xdna->dev_lock);
	300	destroy_workqueue(events->wq);
	301	mutex_lock(&xdna->dev_lock);
	302
	303	dma_free_noncoherent(xdna->ddev.dev, events->size, events->buf,
	304	events->addr, DMA_FROM_DEVICE);
	305	kfree(events);
	306	}
	307
	308	int aie2_error_async_events_alloc(struct amdxdna_dev_hdl *ndev)
	309	{
	310	struct amdxdna_dev *xdna = ndev->xdna;
	311	u32 total_col = ndev->total_col;
	312	u32 total_size = ASYNC_BUF_SIZE * total_col;
	313	struct async_events *events;
	314	int i, ret;
	315
	316	events = kzalloc(struct_size(events, event, total_col), GFP_KERNEL);
	317	if (!events)
	318	return -ENOMEM;
	319
	320	events->buf = dma_alloc_noncoherent(xdna->ddev.dev, total_size, &events->addr,
	321	DMA_FROM_DEVICE, GFP_KERNEL);
	322	if (!events->buf) {
	323	ret = -ENOMEM;
	324	goto free_events;
	325	}
	326	events->size = total_size;
	327	events->event_cnt = total_col;
	328
	329	events->wq = alloc_ordered_workqueue("async_wq", 0);
	330	if (!events->wq) {
	331	ret = -ENOMEM;
	332	goto free_buf;
	333	}
	334
	335	for (i = 0; i < events->event_cnt; i++) {
	336	struct async_event *e = &events->event[i];
	337	u32 offset = i * ASYNC_BUF_SIZE;
	338
	339	e->ndev = ndev;
	340	e->wq = events->wq;
	341	e->buf = &events->buf[offset];
	342	e->addr = events->addr + offset;
	343	e->size = ASYNC_BUF_SIZE;
	344	e->resp.status = MAX_AIE2_STATUS_CODE;
	345	INIT_WORK(&e->work, aie2_error_worker);
	346	}
	347
	348	ndev->async_events = events;
	349
	350	XDNA_DBG(xdna, "Async event count %d, buf total size 0x%x",
	351	events->event_cnt, events->size);
	352	return 0;
	353
	354	free_buf:
	355	dma_free_noncoherent(xdna->ddev.dev, events->size, events->buf,
	356	events->addr, DMA_FROM_DEVICE);
	357	free_events:
	358	kfree(events);
	359	return ret;
	360	}

drivers/accel/amdxdna/aie2_message.c+791−0

@@ -0,0 +1,791 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_cache.h>
	8	#include <drm/drm_device.h>
	9	#include <drm/drm_gem.h>
	10	#include <drm/drm_gem_shmem_helper.h>
	11	#include <drm/drm_print.h>
	12	#include <drm/gpu_scheduler.h>
	13	#include <linux/bitfield.h>
	14	#include <linux/errno.h>
	15	#include <linux/pci.h>
	16	#include <linux/types.h>
	17
	18	#include "aie2_msg_priv.h"
	19	#include "aie2_pci.h"
	20	#include "amdxdna_ctx.h"
	21	#include "amdxdna_gem.h"
	22	#include "amdxdna_mailbox.h"
	23	#include "amdxdna_mailbox_helper.h"
	24	#include "amdxdna_pci_drv.h"
	25
	26	#define DECLARE_AIE2_MSG(name, op) \
	27	DECLARE_XDNA_MSG_COMMON(name, op, MAX_AIE2_STATUS_CODE)
	28
	29	static int aie2_send_mgmt_msg_wait(struct amdxdna_dev_hdl *ndev,
	30	struct xdna_mailbox_msg *msg)
	31	{
	32	struct amdxdna_dev *xdna = ndev->xdna;
	33	struct xdna_notify *hdl = msg->handle;
	34	int ret;
	35
	36	if (!ndev->mgmt_chann)
	37	return -ENODEV;
	38
	39	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	40	ret = xdna_send_msg_wait(xdna, ndev->mgmt_chann, msg);
	41	if (ret == -ETIME) {
	42	xdna_mailbox_stop_channel(ndev->mgmt_chann);
	43	xdna_mailbox_destroy_channel(ndev->mgmt_chann);
	44	ndev->mgmt_chann = NULL;
	45	}
	46
	47	if (!ret && *hdl->data != AIE2_STATUS_SUCCESS) {
	48	XDNA_ERR(xdna, "command opcode 0x%x failed, status 0x%x",
	49	msg->opcode, *hdl->data);
	50	ret = -EINVAL;
	51	}
	52
	53	return ret;
	54	}
	55
	56	int aie2_suspend_fw(struct amdxdna_dev_hdl *ndev)
	57	{
	58	DECLARE_AIE2_MSG(suspend, MSG_OP_SUSPEND);
	59
	60	return aie2_send_mgmt_msg_wait(ndev, &msg);
	61	}
	62
	63	int aie2_resume_fw(struct amdxdna_dev_hdl *ndev)
	64	{
	65	DECLARE_AIE2_MSG(suspend, MSG_OP_RESUME);
	66
	67	return aie2_send_mgmt_msg_wait(ndev, &msg);
	68	}
	69
	70	int aie2_set_runtime_cfg(struct amdxdna_dev_hdl *ndev, u32 type, u64 value)
	71	{
	72	DECLARE_AIE2_MSG(set_runtime_cfg, MSG_OP_SET_RUNTIME_CONFIG);
	73
	74	req.type = type;
	75	req.value = value;
	76
	77	return aie2_send_mgmt_msg_wait(ndev, &msg);
	78	}
	79
	80	int aie2_get_runtime_cfg(struct amdxdna_dev_hdl ndev, u32 type, u64 value)
	81	{
	82	DECLARE_AIE2_MSG(get_runtime_cfg, MSG_OP_GET_RUNTIME_CONFIG);
	83	int ret;
	84
	85	req.type = type;
	86	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	87	if (ret) {
	88	XDNA_ERR(ndev->xdna, "Failed to get runtime config, ret %d", ret);
	89	return ret;
	90	}
	91
	92	*value = resp.value;
	93	return 0;
	94	}
	95
	96	int aie2_check_protocol_version(struct amdxdna_dev_hdl *ndev)
	97	{
	98	DECLARE_AIE2_MSG(protocol_version, MSG_OP_GET_PROTOCOL_VERSION);
	99	struct amdxdna_dev *xdna = ndev->xdna;
	100	int ret;
	101
	102	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	103	if (ret) {
	104	XDNA_ERR(xdna, "Failed to get protocol version, ret %d", ret);
	105	return ret;
	106	}
	107
	108	if (resp.major != ndev->priv->protocol_major) {
	109	XDNA_ERR(xdna, "Incompatible firmware protocol version major %d minor %d",
	110	resp.major, resp.minor);
	111	return -EINVAL;
	112	}
	113
	114	if (resp.minor < ndev->priv->protocol_minor) {
	115	XDNA_ERR(xdna, "Firmware minor version smaller than supported");
	116	return -EINVAL;
	117	}
	118
	119	return 0;
	120	}
	121
	122	int aie2_assign_mgmt_pasid(struct amdxdna_dev_hdl *ndev, u16 pasid)
	123	{
	124	DECLARE_AIE2_MSG(assign_mgmt_pasid, MSG_OP_ASSIGN_MGMT_PASID);
	125
	126	req.pasid = pasid;
	127
	128	return aie2_send_mgmt_msg_wait(ndev, &msg);
	129	}
	130
	131	int aie2_query_aie_version(struct amdxdna_dev_hdl ndev, struct aie_version version)
	132	{
	133	DECLARE_AIE2_MSG(aie_version_info, MSG_OP_QUERY_AIE_VERSION);
	134	struct amdxdna_dev *xdna = ndev->xdna;
	135	int ret;
	136
	137	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	138	if (ret)
	139	return ret;
	140
	141	XDNA_DBG(xdna, "Query AIE version - major: %u minor: %u completed",
	142	resp.major, resp.minor);
	143
	144	version->major = resp.major;
	145	version->minor = resp.minor;
	146
	147	return 0;
	148	}
	149
	150	int aie2_query_aie_metadata(struct amdxdna_dev_hdl ndev, struct aie_metadata metadata)
	151	{
	152	DECLARE_AIE2_MSG(aie_tile_info, MSG_OP_QUERY_AIE_TILE_INFO);
	153	int ret;
	154
	155	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	156	if (ret)
	157	return ret;
	158
	159	metadata->size = resp.info.size;
	160	metadata->cols = resp.info.cols;
	161	metadata->rows = resp.info.rows;
	162
	163	metadata->version.major = resp.info.major;
	164	metadata->version.minor = resp.info.minor;
	165
	166	metadata->core.row_count = resp.info.core_rows;
	167	metadata->core.row_start = resp.info.core_row_start;
	168	metadata->core.dma_channel_count = resp.info.core_dma_channels;
	169	metadata->core.lock_count = resp.info.core_locks;
	170	metadata->core.event_reg_count = resp.info.core_events;
	171
	172	metadata->mem.row_count = resp.info.mem_rows;
	173	metadata->mem.row_start = resp.info.mem_row_start;
	174	metadata->mem.dma_channel_count = resp.info.mem_dma_channels;
	175	metadata->mem.lock_count = resp.info.mem_locks;
	176	metadata->mem.event_reg_count = resp.info.mem_events;
	177
	178	metadata->shim.row_count = resp.info.shim_rows;
	179	metadata->shim.row_start = resp.info.shim_row_start;
	180	metadata->shim.dma_channel_count = resp.info.shim_dma_channels;
	181	metadata->shim.lock_count = resp.info.shim_locks;
	182	metadata->shim.event_reg_count = resp.info.shim_events;
	183
	184	return 0;
	185	}
	186
	187	int aie2_query_firmware_version(struct amdxdna_dev_hdl *ndev,
	188	struct amdxdna_fw_ver *fw_ver)
	189	{
	190	DECLARE_AIE2_MSG(firmware_version, MSG_OP_GET_FIRMWARE_VERSION);
	191	int ret;
	192
	193	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	194	if (ret)
	195	return ret;
	196
	197	fw_ver->major = resp.major;
	198	fw_ver->minor = resp.minor;
	199	fw_ver->sub = resp.sub;
	200	fw_ver->build = resp.build;
	201
	202	return 0;
	203	}
	204
	205	int aie2_create_context(struct amdxdna_dev_hdl ndev, struct amdxdna_hwctx hwctx)
	206	{
	207	DECLARE_AIE2_MSG(create_ctx, MSG_OP_CREATE_CONTEXT);
	208	struct amdxdna_dev *xdna = ndev->xdna;
	209	struct xdna_mailbox_chann_res x2i;
	210	struct xdna_mailbox_chann_res i2x;
	211	struct cq_pair *cq_pair;
	212	u32 intr_reg;
	213	int ret;
	214
	215	req.aie_type = 1;
	216	req.start_col = hwctx->start_col;
	217	req.num_col = hwctx->num_col;
	218	req.num_cq_pairs_requested = 1;
	219	req.pasid = hwctx->client->pasid;
	220	req.context_priority = 2;
	221
	222	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	223	if (ret)
	224	return ret;
	225
	226	hwctx->fw_ctx_id = resp.context_id;
	227	WARN_ONCE(hwctx->fw_ctx_id == -1, "Unexpected context id");
	228
	229	cq_pair = &resp.cq_pair[0];
	230	x2i.mb_head_ptr_reg = AIE2_MBOX_OFF(ndev, cq_pair->x2i_q.head_addr);
	231	x2i.mb_tail_ptr_reg = AIE2_MBOX_OFF(ndev, cq_pair->x2i_q.tail_addr);
	232	x2i.rb_start_addr = AIE2_SRAM_OFF(ndev, cq_pair->x2i_q.buf_addr);
	233	x2i.rb_size = cq_pair->x2i_q.buf_size;
	234
	235	i2x.mb_head_ptr_reg = AIE2_MBOX_OFF(ndev, cq_pair->i2x_q.head_addr);
	236	i2x.mb_tail_ptr_reg = AIE2_MBOX_OFF(ndev, cq_pair->i2x_q.tail_addr);
	237	i2x.rb_start_addr = AIE2_SRAM_OFF(ndev, cq_pair->i2x_q.buf_addr);
	238	i2x.rb_size = cq_pair->i2x_q.buf_size;
	239
	240	ret = pci_irq_vector(to_pci_dev(xdna->ddev.dev), resp.msix_id);
	241	if (ret == -EINVAL) {
	242	XDNA_ERR(xdna, "not able to create channel");
	243	goto out_destroy_context;
	244	}
	245
	246	intr_reg = i2x.mb_head_ptr_reg + 4;
	247	hwctx->priv->mbox_chann = xdna_mailbox_create_channel(ndev->mbox, &x2i, &i2x,
	248	intr_reg, ret);
	249	if (!hwctx->priv->mbox_chann) {
	250	XDNA_ERR(xdna, "not able to create channel");
	251	ret = -EINVAL;
	252	goto out_destroy_context;
	253	}
	254
	255	XDNA_DBG(xdna, "%s mailbox channel irq: %d, msix_id: %d",
	256	hwctx->name, ret, resp.msix_id);
	257	XDNA_DBG(xdna, "%s created fw ctx %d pasid %d", hwctx->name,
	258	hwctx->fw_ctx_id, hwctx->client->pasid);
	259
	260	return 0;
	261
	262	out_destroy_context:
	263	aie2_destroy_context(ndev, hwctx);
	264	return ret;
	265	}
	266
	267	int aie2_destroy_context(struct amdxdna_dev_hdl ndev, struct amdxdna_hwctx hwctx)
	268	{
	269	DECLARE_AIE2_MSG(destroy_ctx, MSG_OP_DESTROY_CONTEXT);
	270	struct amdxdna_dev *xdna = ndev->xdna;
	271	int ret;
	272
	273	if (hwctx->fw_ctx_id == -1)
	274	return 0;
	275
	276	xdna_mailbox_stop_channel(hwctx->priv->mbox_chann);
	277
	278	req.context_id = hwctx->fw_ctx_id;
	279	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	280	if (ret)
	281	XDNA_WARN(xdna, "%s destroy context failed, ret %d", hwctx->name, ret);
	282
	283	xdna_mailbox_destroy_channel(hwctx->priv->mbox_chann);
	284	XDNA_DBG(xdna, "%s destroyed fw ctx %d", hwctx->name,
	285	hwctx->fw_ctx_id);
	286	hwctx->priv->mbox_chann = NULL;
	287	hwctx->fw_ctx_id = -1;
	288
	289	return ret;
	290	}
	291
	292	int aie2_map_host_buf(struct amdxdna_dev_hdl *ndev, u32 context_id, u64 addr, u64 size)
	293	{
	294	DECLARE_AIE2_MSG(map_host_buffer, MSG_OP_MAP_HOST_BUFFER);
	295	struct amdxdna_dev *xdna = ndev->xdna;
	296	int ret;
	297
	298	req.context_id = context_id;
	299	req.buf_addr = addr;
	300	req.buf_size = size;
	301	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	302	if (ret)
	303	return ret;
	304
	305	XDNA_DBG(xdna, "fw ctx %d map host buf addr 0x%llx size 0x%llx",
	306	context_id, addr, size);
	307
	308	return 0;
	309	}
	310
	311	int aie2_query_status(struct amdxdna_dev_hdl ndev, char __user buf,
	312	u32 size, u32 *cols_filled)
	313	{
	314	DECLARE_AIE2_MSG(aie_column_info, MSG_OP_QUERY_COL_STATUS);
	315	struct amdxdna_dev *xdna = ndev->xdna;
	316	struct amdxdna_client *client;
	317	struct amdxdna_hwctx *hwctx;
	318	dma_addr_t dma_addr;
	319	u32 aie_bitmap = 0;
	320	u8 *buff_addr;
	321	int next = 0;
	322	int ret, idx;
	323
	324	buff_addr = dma_alloc_noncoherent(xdna->ddev.dev, size, &dma_addr,
	325	DMA_FROM_DEVICE, GFP_KERNEL);
	326	if (!buff_addr)
	327	return -ENOMEM;
	328
	329	/* Go through each hardware context and mark the AIE columns that are active */
	330	list_for_each_entry(client, &xdna->client_list, node) {
	331	idx = srcu_read_lock(&client->hwctx_srcu);
	332	idr_for_each_entry_continue(&client->hwctx_idr, hwctx, next)
	333	aie_bitmap \|= amdxdna_hwctx_col_map(hwctx);
	334	srcu_read_unlock(&client->hwctx_srcu, idx);
	335	}
	336
	337	*cols_filled = 0;
	338	req.dump_buff_addr = dma_addr;
	339	req.dump_buff_size = size;
	340	req.num_cols = hweight32(aie_bitmap);
	341	req.aie_bitmap = aie_bitmap;
	342
	343	drm_clflush_virt_range(buff_addr, size); /* device can access */
	344	ret = aie2_send_mgmt_msg_wait(ndev, &msg);
	345	if (ret) {
	346	XDNA_ERR(xdna, "Error during NPU query, status %d", ret);
	347	goto fail;
	348	}
	349
	350	if (resp.status != AIE2_STATUS_SUCCESS) {
	351	XDNA_ERR(xdna, "Query NPU status failed, status 0x%x", resp.status);
	352	ret = -EINVAL;
	353	goto fail;
	354	}
	355	XDNA_DBG(xdna, "Query NPU status completed");
	356
	357	if (size < resp.size) {
	358	ret = -EINVAL;
	359	XDNA_ERR(xdna, "Bad buffer size. Available: %u. Needs: %u", size, resp.size);
	360	goto fail;
	361	}
	362
	363	if (copy_to_user(buf, buff_addr, resp.size)) {
	364	ret = -EFAULT;
	365	XDNA_ERR(xdna, "Failed to copy NPU status to user space");
	366	goto fail;
	367	}
	368
	369	*cols_filled = aie_bitmap;
	370
	371	fail:
	372	dma_free_noncoherent(xdna->ddev.dev, size, buff_addr, dma_addr, DMA_FROM_DEVICE);
	373	return ret;
	374	}
	375
	376	int aie2_register_asyn_event_msg(struct amdxdna_dev_hdl *ndev, dma_addr_t addr, u32 size,
	377	void handle, int (cb)(void, const u32 , size_t))
	378	{
	379	struct async_event_msg_req req = { 0 };
	380	struct xdna_mailbox_msg msg = {
	381	.send_data = (u8 *)&req,
	382	.send_size = sizeof(req),
	383	.handle = handle,
	384	.opcode = MSG_OP_REGISTER_ASYNC_EVENT_MSG,
	385	.notify_cb = cb,
	386	};
	387
	388	req.buf_addr = addr;
	389	req.buf_size = size;
	390
	391	XDNA_DBG(ndev->xdna, "Register addr 0x%llx size 0x%x", addr, size);
	392	return xdna_mailbox_send_msg(ndev->mgmt_chann, &msg, TX_TIMEOUT);
	393	}
	394
	395	int aie2_config_cu(struct amdxdna_hwctx *hwctx)
	396	{
	397	struct mailbox_channel *chann = hwctx->priv->mbox_chann;
	398	struct amdxdna_dev *xdna = hwctx->client->xdna;
	399	u32 shift = xdna->dev_info->dev_mem_buf_shift;
	400	DECLARE_AIE2_MSG(config_cu, MSG_OP_CONFIG_CU);
	401	struct drm_gem_object *gobj;
	402	struct amdxdna_gem_obj *abo;
	403	int ret, i;
	404
	405	if (!chann)
	406	return -ENODEV;
	407
	408	if (hwctx->cus->num_cus > MAX_NUM_CUS) {
	409	XDNA_DBG(xdna, "Exceed maximum CU %d", MAX_NUM_CUS);
	410	return -EINVAL;
	411	}
	412
	413	for (i = 0; i < hwctx->cus->num_cus; i++) {
	414	struct amdxdna_cu_config *cu = &hwctx->cus->cu_configs[i];
	415
	416	gobj = drm_gem_object_lookup(hwctx->client->filp, cu->cu_bo);
	417	if (!gobj) {
	418	XDNA_ERR(xdna, "Lookup GEM object failed");
	419	return -EINVAL;
	420	}
	421	abo = to_xdna_obj(gobj);
	422
	423	if (abo->type != AMDXDNA_BO_DEV) {
	424	drm_gem_object_put(gobj);
	425	XDNA_ERR(xdna, "Invalid BO type");
	426	return -EINVAL;
	427	}
	428
	429	req.cfgs[i] = FIELD_PREP(AIE2_MSG_CFG_CU_PDI_ADDR,
	430	abo->mem.dev_addr >> shift);
	431	req.cfgs[i] \|= FIELD_PREP(AIE2_MSG_CFG_CU_FUNC, cu->cu_func);
	432	XDNA_DBG(xdna, "CU %d full addr 0x%llx, cfg 0x%x", i,
	433	abo->mem.dev_addr, req.cfgs[i]);
	434	drm_gem_object_put(gobj);
	435	}
	436	req.num_cus = hwctx->cus->num_cus;
	437
	438	ret = xdna_send_msg_wait(xdna, chann, &msg);
	439	if (ret == -ETIME)
	440	aie2_destroy_context(xdna->dev_handle, hwctx);
	441
	442	if (resp.status == AIE2_STATUS_SUCCESS) {
	443	XDNA_DBG(xdna, "Configure %d CUs, ret %d", req.num_cus, ret);
	444	return 0;
	445	}
	446
	447	XDNA_ERR(xdna, "Command opcode 0x%x failed, status 0x%x ret %d",
	448	msg.opcode, resp.status, ret);
	449	return ret;
	450	}
	451
	452	int aie2_execbuf(struct amdxdna_hwctx hwctx, struct amdxdna_sched_job job,
	453	int (notify_cb)(void , const u32 *, size_t))
	454	{
	455	struct mailbox_channel *chann = hwctx->priv->mbox_chann;
	456	struct amdxdna_dev *xdna = hwctx->client->xdna;
	457	struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;
	458	union {
	459	struct execute_buffer_req ebuf;
	460	struct exec_dpu_req dpu;
	461	} req;
	462	struct xdna_mailbox_msg msg;
	463	u32 payload_len;
	464	void *payload;
	465	int cu_idx;
	466	int ret;
	467	u32 op;
	468
	469	if (!chann)
	470	return -ENODEV;
	471
	472	payload = amdxdna_cmd_get_payload(cmd_abo, &payload_len);
	473	if (!payload) {
	474	XDNA_ERR(xdna, "Invalid command, cannot get payload");
	475	return -EINVAL;
	476	}
	477
	478	cu_idx = amdxdna_cmd_get_cu_idx(cmd_abo);
	479	if (cu_idx < 0) {
	480	XDNA_DBG(xdna, "Invalid cu idx");
	481	return -EINVAL;
	482	}
	483
	484	op = amdxdna_cmd_get_op(cmd_abo);
	485	switch (op) {
	486	case ERT_START_CU:
	487	if (unlikely(payload_len > sizeof(req.ebuf.payload)))
	488	XDNA_DBG(xdna, "Invalid ebuf payload len: %d", payload_len);
	489	req.ebuf.cu_idx = cu_idx;
	490	memcpy(req.ebuf.payload, payload, sizeof(req.ebuf.payload));
	491	msg.send_size = sizeof(req.ebuf);
	492	msg.opcode = MSG_OP_EXECUTE_BUFFER_CF;
	493	break;
	494	case ERT_START_NPU: {
	495	struct amdxdna_cmd_start_npu *sn = payload;
	496
	497	if (unlikely(payload_len - sizeof(*sn) > sizeof(req.dpu.payload)))
	498	XDNA_DBG(xdna, "Invalid dpu payload len: %d", payload_len);
	499	req.dpu.inst_buf_addr = sn->buffer;
	500	req.dpu.inst_size = sn->buffer_size;
	501	req.dpu.inst_prop_cnt = sn->prop_count;
	502	req.dpu.cu_idx = cu_idx;
	503	memcpy(req.dpu.payload, sn->prop_args, sizeof(req.dpu.payload));
	504	msg.send_size = sizeof(req.dpu);
	505	msg.opcode = MSG_OP_EXEC_DPU;
	506	break;
	507	}
	508	default:
	509	XDNA_DBG(xdna, "Invalid ERT cmd op code: %d", op);
	510	return -EINVAL;
	511	}
	512	msg.handle = job;
	513	msg.notify_cb = notify_cb;
	514	msg.send_data = (u8 *)&req;
	515	print_hex_dump_debug("cmd: ", DUMP_PREFIX_OFFSET, 16, 4, &req,
	516	0x40, false);
	517
	518	ret = xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);
	519	if (ret) {
	520	XDNA_ERR(xdna, "Send message failed");
	521	return ret;
	522	}
	523
	524	return 0;
	525	}
	526
	527	static int
	528	aie2_cmdlist_fill_one_slot_cf(void *cmd_buf, u32 offset,
	529	struct amdxdna_gem_obj abo, u32 size)
	530	{
	531	struct cmd_chain_slot_execbuf_cf *buf = cmd_buf + offset;
	532	int cu_idx = amdxdna_cmd_get_cu_idx(abo);
	533	u32 payload_len;
	534	void *payload;
	535
	536	if (cu_idx < 0)
	537	return -EINVAL;
	538
	539	payload = amdxdna_cmd_get_payload(abo, &payload_len);
	540	if (!payload)
	541	return -EINVAL;
	542
	543	if (!slot_cf_has_space(offset, payload_len))
	544	return -ENOSPC;
	545
	546	buf->cu_idx = cu_idx;
	547	buf->arg_cnt = payload_len / sizeof(u32);
	548	memcpy(buf->args, payload, payload_len);
	549	/* Accurate buf size to hint firmware to do necessary copy */
	550	size = sizeof(buf) + payload_len;
	551	return 0;
	552	}
	553
	554	static int
	555	aie2_cmdlist_fill_one_slot_dpu(void *cmd_buf, u32 offset,
	556	struct amdxdna_gem_obj abo, u32 size)
	557	{
	558	struct cmd_chain_slot_dpu *buf = cmd_buf + offset;
	559	int cu_idx = amdxdna_cmd_get_cu_idx(abo);
	560	struct amdxdna_cmd_start_npu *sn;
	561	u32 payload_len;
	562	void *payload;
	563	u32 arg_sz;
	564
	565	if (cu_idx < 0)
	566	return -EINVAL;
	567
	568	payload = amdxdna_cmd_get_payload(abo, &payload_len);
	569	if (!payload)
	570	return -EINVAL;
	571	sn = payload;
	572	arg_sz = payload_len - sizeof(*sn);
	573	if (payload_len < sizeof(*sn) \|\| arg_sz > MAX_DPU_ARGS_SIZE)
	574	return -EINVAL;
	575
	576	if (!slot_dpu_has_space(offset, arg_sz))
	577	return -ENOSPC;
	578
	579	buf->inst_buf_addr = sn->buffer;
	580	buf->inst_size = sn->buffer_size;
	581	buf->inst_prop_cnt = sn->prop_count;
	582	buf->cu_idx = cu_idx;
	583	buf->arg_cnt = arg_sz / sizeof(u32);
	584	memcpy(buf->args, sn->prop_args, arg_sz);
	585
	586	/* Accurate buf size to hint firmware to do necessary copy */
	587	size += sizeof(buf) + arg_sz;
	588	return 0;
	589	}
	590
	591	static int
	592	aie2_cmdlist_fill_one_slot(u32 op, struct amdxdna_gem_obj *cmdbuf_abo, u32 offset,
	593	struct amdxdna_gem_obj abo, u32 size)
	594	{
	595	u32 this_op = amdxdna_cmd_get_op(abo);
	596	void *cmd_buf = cmdbuf_abo->mem.kva;
	597	int ret;
	598
	599	if (this_op != op) {
	600	ret = -EINVAL;
	601	goto done;
	602	}
	603
	604	switch (op) {
	605	case ERT_START_CU:
	606	ret = aie2_cmdlist_fill_one_slot_cf(cmd_buf, offset, abo, size);
	607	break;
	608	case ERT_START_NPU:
	609	ret = aie2_cmdlist_fill_one_slot_dpu(cmd_buf, offset, abo, size);
	610	break;
	611	default:
	612	ret = -EOPNOTSUPP;
	613	}
	614
	615	done:
	616	if (ret) {
	617	XDNA_ERR(abo->client->xdna, "Can't fill slot for cmd op %d ret %d",
	618	op, ret);
	619	}
	620	return ret;
	621	}
	622
	623	static inline struct amdxdna_gem_obj *
	624	aie2_cmdlist_get_cmd_buf(struct amdxdna_sched_job *job)
	625	{
	626	int idx = get_job_idx(job->seq);
	627
	628	return job->hwctx->priv->cmd_buf[idx];
	629	}
	630
	631	static void
	632	aie2_cmdlist_prepare_request(struct cmd_chain_req *req,
	633	struct amdxdna_gem_obj *cmdbuf_abo, u32 size, u32 cnt)
	634	{
	635	req->buf_addr = cmdbuf_abo->mem.dev_addr;
	636	req->buf_size = size;
	637	req->count = cnt;
	638	drm_clflush_virt_range(cmdbuf_abo->mem.kva, size);
	639	XDNA_DBG(cmdbuf_abo->client->xdna, "Command buf addr 0x%llx size 0x%x count %d",
	640	req->buf_addr, size, cnt);
	641	}
	642
	643	static inline u32
	644	aie2_cmd_op_to_msg_op(u32 op)
	645	{
	646	switch (op) {
	647	case ERT_START_CU:
	648	return MSG_OP_CHAIN_EXEC_BUFFER_CF;
	649	case ERT_START_NPU:
	650	return MSG_OP_CHAIN_EXEC_DPU;
	651	default:
	652	return MSG_OP_MAX_OPCODE;
	653	}
	654	}
	655
	656	int aie2_cmdlist_multi_execbuf(struct amdxdna_hwctx *hwctx,
	657	struct amdxdna_sched_job *job,
	658	int (notify_cb)(void , const u32 *, size_t))
	659	{
	660	struct amdxdna_gem_obj *cmdbuf_abo = aie2_cmdlist_get_cmd_buf(job);
	661	struct mailbox_channel *chann = hwctx->priv->mbox_chann;
	662	struct amdxdna_client *client = hwctx->client;
	663	struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;
	664	struct amdxdna_cmd_chain *payload;
	665	struct xdna_mailbox_msg msg;
	666	struct cmd_chain_req req;
	667	u32 payload_len;
	668	u32 offset = 0;
	669	u32 size;
	670	int ret;
	671	u32 op;
	672	u32 i;
	673
	674	op = amdxdna_cmd_get_op(cmd_abo);
	675	payload = amdxdna_cmd_get_payload(cmd_abo, &payload_len);
	676	if (op != ERT_CMD_CHAIN \|\| !payload \|\|
	677	payload_len < struct_size(payload, data, payload->command_count))
	678	return -EINVAL;
	679
	680	for (i = 0; i < payload->command_count; i++) {
	681	u32 boh = (u32)(payload->data[i]);
	682	struct amdxdna_gem_obj *abo;
	683
	684	abo = amdxdna_gem_get_obj(client, boh, AMDXDNA_BO_CMD);
	685	if (!abo) {
	686	XDNA_ERR(client->xdna, "Failed to find cmd BO %d", boh);
	687	return -ENOENT;
	688	}
	689
	690	/* All sub-cmd should have same op, use the first one. */
	691	if (i == 0)
	692	op = amdxdna_cmd_get_op(abo);
	693
	694	ret = aie2_cmdlist_fill_one_slot(op, cmdbuf_abo, offset, abo, &size);
	695	amdxdna_gem_put_obj(abo);
	696	if (ret)
	697	return -EINVAL;
	698
	699	offset += size;
	700	}
	701
	702	/* The offset is the accumulated total size of the cmd buffer */
	703	aie2_cmdlist_prepare_request(&req, cmdbuf_abo, offset, payload->command_count);
	704
	705	msg.opcode = aie2_cmd_op_to_msg_op(op);
	706	if (msg.opcode == MSG_OP_MAX_OPCODE)
	707	return -EOPNOTSUPP;
	708	msg.handle = job;
	709	msg.notify_cb = notify_cb;
	710	msg.send_data = (u8 *)&req;
	711	msg.send_size = sizeof(req);
	712	ret = xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);
	713	if (ret) {
	714	XDNA_ERR(hwctx->client->xdna, "Send message failed");
	715	return ret;
	716	}
	717
	718	return 0;
	719	}
	720
	721	int aie2_cmdlist_single_execbuf(struct amdxdna_hwctx *hwctx,
	722	struct amdxdna_sched_job *job,
	723	int (notify_cb)(void , const u32 *, size_t))
	724	{
	725	struct amdxdna_gem_obj *cmdbuf_abo = aie2_cmdlist_get_cmd_buf(job);
	726	struct mailbox_channel *chann = hwctx->priv->mbox_chann;
	727	struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;
	728	struct xdna_mailbox_msg msg;
	729	struct cmd_chain_req req;
	730	u32 size;
	731	int ret;
	732	u32 op;
	733
	734	op = amdxdna_cmd_get_op(cmd_abo);
	735	ret = aie2_cmdlist_fill_one_slot(op, cmdbuf_abo, 0, cmd_abo, &size);
	736	if (ret)
	737	return ret;
	738
	739	aie2_cmdlist_prepare_request(&req, cmdbuf_abo, size, 1);
	740
	741	msg.opcode = aie2_cmd_op_to_msg_op(op);
	742	if (msg.opcode == MSG_OP_MAX_OPCODE)
	743	return -EOPNOTSUPP;
	744	msg.handle = job;
	745	msg.notify_cb = notify_cb;
	746	msg.send_data = (u8 *)&req;
	747	msg.send_size = sizeof(req);
	748	ret = xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);
	749	if (ret) {
	750	XDNA_ERR(hwctx->client->xdna, "Send message failed");
	751	return ret;
	752	}
	753
	754	return 0;
	755	}
	756
	757	int aie2_sync_bo(struct amdxdna_hwctx hwctx, struct amdxdna_sched_job job,
	758	int (notify_cb)(void , const u32 *, size_t))
	759	{
	760	struct mailbox_channel *chann = hwctx->priv->mbox_chann;
	761	struct amdxdna_gem_obj *abo = to_xdna_obj(job->bos[0]);
	762	struct amdxdna_dev *xdna = hwctx->client->xdna;
	763	struct xdna_mailbox_msg msg;
	764	struct sync_bo_req req;
	765	int ret = 0;
	766
	767	req.src_addr = 0;
	768	req.dst_addr = abo->mem.dev_addr - hwctx->client->dev_heap->mem.dev_addr;
	769	req.size = abo->mem.size;
	770
	771	/* Device to Host */
	772	req.type = FIELD_PREP(AIE2_MSG_SYNC_BO_SRC_TYPE, SYNC_BO_DEV_MEM) \|
	773	FIELD_PREP(AIE2_MSG_SYNC_BO_DST_TYPE, SYNC_BO_HOST_MEM);
	774
	775	XDNA_DBG(xdna, "sync %d bytes src(0x%llx) to dst(0x%llx) completed",
	776	req.size, req.src_addr, req.dst_addr);
	777
	778	msg.handle = job;
	779	msg.notify_cb = notify_cb;
	780	msg.send_data = (u8 *)&req;
	781	msg.send_size = sizeof(req);
	782	msg.opcode = MSG_OP_SYNC_BO;
	783
	784	ret = xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);
	785	if (ret) {
	786	XDNA_ERR(xdna, "Send message failed");
	787	return ret;
	788	}
	789
	790	return 0;
	791	}

drivers/accel/amdxdna/aie2_msg_priv.h+370−0

@@ -0,0 +1,370 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _AIE2_MSG_PRIV_H_
	7	#define _AIE2_MSG_PRIV_H_
	8
	9	enum aie2_msg_opcode {
	10	MSG_OP_CREATE_CONTEXT = 0x2,
	11	MSG_OP_DESTROY_CONTEXT = 0x3,
	12	MSG_OP_SYNC_BO = 0x7,
	13	MSG_OP_EXECUTE_BUFFER_CF = 0xC,
	14	MSG_OP_QUERY_COL_STATUS = 0xD,
	15	MSG_OP_QUERY_AIE_TILE_INFO = 0xE,
	16	MSG_OP_QUERY_AIE_VERSION = 0xF,
	17	MSG_OP_EXEC_DPU = 0x10,
	18	MSG_OP_CONFIG_CU = 0x11,
	19	MSG_OP_CHAIN_EXEC_BUFFER_CF = 0x12,
	20	MSG_OP_CHAIN_EXEC_DPU = 0x13,
	21	MSG_OP_MAX_XRT_OPCODE,
	22	MSG_OP_SUSPEND = 0x101,
	23	MSG_OP_RESUME = 0x102,
	24	MSG_OP_ASSIGN_MGMT_PASID = 0x103,
	25	MSG_OP_INVOKE_SELF_TEST = 0x104,
	26	MSG_OP_MAP_HOST_BUFFER = 0x106,
	27	MSG_OP_GET_FIRMWARE_VERSION = 0x108,
	28	MSG_OP_SET_RUNTIME_CONFIG = 0x10A,
	29	MSG_OP_GET_RUNTIME_CONFIG = 0x10B,
	30	MSG_OP_REGISTER_ASYNC_EVENT_MSG = 0x10C,
	31	MSG_OP_MAX_DRV_OPCODE,
	32	MSG_OP_GET_PROTOCOL_VERSION = 0x301,
	33	MSG_OP_MAX_OPCODE
	34	};
	35
	36	enum aie2_msg_status {
	37	AIE2_STATUS_SUCCESS = 0x0,
	38	/* AIE Error codes */
	39	AIE2_STATUS_AIE_SATURATION_ERROR = 0x1000001,
	40	AIE2_STATUS_AIE_FP_ERROR = 0x1000002,
	41	AIE2_STATUS_AIE_STREAM_ERROR = 0x1000003,
	42	AIE2_STATUS_AIE_ACCESS_ERROR = 0x1000004,
	43	AIE2_STATUS_AIE_BUS_ERROR = 0x1000005,
	44	AIE2_STATUS_AIE_INSTRUCTION_ERROR = 0x1000006,
	45	AIE2_STATUS_AIE_ECC_ERROR = 0x1000007,
	46	AIE2_STATUS_AIE_LOCK_ERROR = 0x1000008,
	47	AIE2_STATUS_AIE_DMA_ERROR = 0x1000009,
	48	AIE2_STATUS_AIE_MEM_PARITY_ERROR = 0x100000a,
	49	AIE2_STATUS_AIE_PWR_CFG_ERROR = 0x100000b,
	50	AIE2_STATUS_AIE_BACKTRACK_ERROR = 0x100000c,
	51	AIE2_STATUS_MAX_AIE_STATUS_CODE,
	52	/* MGMT ERT Error codes */
	53	AIE2_STATUS_MGMT_ERT_SELF_TEST_FAILURE = 0x2000001,
	54	AIE2_STATUS_MGMT_ERT_HASH_MISMATCH,
	55	AIE2_STATUS_MGMT_ERT_NOAVAIL,
	56	AIE2_STATUS_MGMT_ERT_INVALID_PARAM,
	57	AIE2_STATUS_MGMT_ERT_ENTER_SUSPEND_FAILURE,
	58	AIE2_STATUS_MGMT_ERT_BUSY,
	59	AIE2_STATUS_MGMT_ERT_APPLICATION_ACTIVE,
	60	MAX_MGMT_ERT_STATUS_CODE,
	61	/* APP ERT Error codes */
	62	AIE2_STATUS_APP_ERT_FIRST_ERROR = 0x3000001,
	63	AIE2_STATUS_APP_INVALID_INSTR,
	64	AIE2_STATUS_APP_LOAD_PDI_FAIL,
	65	MAX_APP_ERT_STATUS_CODE,
	66	/* NPU RTOS Error Codes */
	67	AIE2_STATUS_INVALID_INPUT_BUFFER = 0x4000001,
	68	AIE2_STATUS_INVALID_COMMAND,
	69	AIE2_STATUS_INVALID_PARAM,
	70	AIE2_STATUS_INVALID_OPERATION = 0x4000006,
	71	AIE2_STATUS_ASYNC_EVENT_MSGS_FULL,
	72	AIE2_STATUS_MAX_RTOS_STATUS_CODE,
	73	MAX_AIE2_STATUS_CODE
	74	};
	75
	76	struct assign_mgmt_pasid_req {
	77	__u16 pasid;
	78	__u16 reserved;
	79	} __packed;
	80
	81	struct assign_mgmt_pasid_resp {
	82	enum aie2_msg_status status;
	83	} __packed;
	84
	85	struct map_host_buffer_req {
	86	__u32 context_id;
	87	__u64 buf_addr;
	88	__u64 buf_size;
	89	} __packed;
	90
	91	struct map_host_buffer_resp {
	92	enum aie2_msg_status status;
	93	} __packed;
	94
	95	#define MAX_CQ_PAIRS 2
	96	struct cq_info {
	97	__u32 head_addr;
	98	__u32 tail_addr;
	99	__u32 buf_addr;
	100	__u32 buf_size;
	101	};
	102
	103	struct cq_pair {
	104	struct cq_info x2i_q;
	105	struct cq_info i2x_q;
	106	};
	107
	108	struct create_ctx_req {
	109	__u32 aie_type;
	110	__u8 start_col;
	111	__u8 num_col;
	112	__u16 reserved;
	113	__u8 num_cq_pairs_requested;
	114	__u8 reserved1;
	115	__u16 pasid;
	116	__u32 pad[2];
	117	__u32 sec_comm_target_type;
	118	__u32 context_priority;
	119	} __packed;
	120
	121	struct create_ctx_resp {
	122	enum aie2_msg_status status;
	123	__u32 context_id;
	124	__u16 msix_id;
	125	__u8 num_cq_pairs_allocated;
	126	__u8 reserved;
	127	struct cq_pair cq_pair[MAX_CQ_PAIRS];
	128	} __packed;
	129
	130	struct destroy_ctx_req {
	131	__u32 context_id;
	132	} __packed;
	133
	134	struct destroy_ctx_resp {
	135	enum aie2_msg_status status;
	136	} __packed;
	137
	138	struct execute_buffer_req {
	139	__u32 cu_idx;
	140	__u32 payload[19];
	141	} __packed;
	142
	143	struct exec_dpu_req {
	144	__u64 inst_buf_addr;
	145	__u32 inst_size;
	146	__u32 inst_prop_cnt;
	147	__u32 cu_idx;
	148	__u32 payload[35];
	149	} __packed;
	150
	151	struct execute_buffer_resp {
	152	enum aie2_msg_status status;
	153	} __packed;
	154
	155	struct aie_tile_info {
	156	__u32 size;
	157	__u16 major;
	158	__u16 minor;
	159	__u16 cols;
	160	__u16 rows;
	161	__u16 core_rows;
	162	__u16 mem_rows;
	163	__u16 shim_rows;
	164	__u16 core_row_start;
	165	__u16 mem_row_start;
	166	__u16 shim_row_start;
	167	__u16 core_dma_channels;
	168	__u16 mem_dma_channels;
	169	__u16 shim_dma_channels;
	170	__u16 core_locks;
	171	__u16 mem_locks;
	172	__u16 shim_locks;
	173	__u16 core_events;
	174	__u16 mem_events;
	175	__u16 shim_events;
	176	__u16 reserved;
	177	};
	178
	179	struct aie_tile_info_req {
	180	__u32 reserved;
	181	} __packed;
	182
	183	struct aie_tile_info_resp {
	184	enum aie2_msg_status status;
	185	struct aie_tile_info info;
	186	} __packed;
	187
	188	struct aie_version_info_req {
	189	__u32 reserved;
	190	} __packed;
	191
	192	struct aie_version_info_resp {
	193	enum aie2_msg_status status;
	194	__u16 major;
	195	__u16 minor;
	196	} __packed;
	197
	198	struct aie_column_info_req {
	199	__u64 dump_buff_addr;
	200	__u32 dump_buff_size;
	201	__u32 num_cols;
	202	__u32 aie_bitmap;
	203	} __packed;
	204
	205	struct aie_column_info_resp {
	206	enum aie2_msg_status status;
	207	__u32 size;
	208	} __packed;
	209
	210	struct suspend_req {
	211	__u32 place_holder;
	212	} __packed;
	213
	214	struct suspend_resp {
	215	enum aie2_msg_status status;
	216	} __packed;
	217
	218	struct resume_req {
	219	__u32 place_holder;
	220	} __packed;
	221
	222	struct resume_resp {
	223	enum aie2_msg_status status;
	224	} __packed;
	225
	226	struct check_header_hash_req {
	227	__u64 hash_high;
	228	__u64 hash_low;
	229	} __packed;
	230
	231	struct check_header_hash_resp {
	232	enum aie2_msg_status status;
	233	} __packed;
	234
	235	struct query_error_req {
	236	__u64 buf_addr;
	237	__u32 buf_size;
	238	__u32 next_row;
	239	__u32 next_column;
	240	__u32 next_module;
	241	} __packed;
	242
	243	struct query_error_resp {
	244	enum aie2_msg_status status;
	245	__u32 num_err;
	246	__u32 has_next_err;
	247	__u32 next_row;
	248	__u32 next_column;
	249	__u32 next_module;
	250	} __packed;
	251
	252	struct protocol_version_req {
	253	__u32 reserved;
	254	} __packed;
	255
	256	struct protocol_version_resp {
	257	enum aie2_msg_status status;
	258	__u32 major;
	259	__u32 minor;
	260	} __packed;
	261
	262	struct firmware_version_req {
	263	__u32 reserved;
	264	} __packed;
	265
	266	struct firmware_version_resp {
	267	enum aie2_msg_status status;
	268	__u32 major;
	269	__u32 minor;
	270	__u32 sub;
	271	__u32 build;
	272	} __packed;
	273
	274	#define MAX_NUM_CUS 32
	275	#define AIE2_MSG_CFG_CU_PDI_ADDR GENMASK(16, 0)
	276	#define AIE2_MSG_CFG_CU_FUNC GENMASK(24, 17)
	277	struct config_cu_req {
	278	__u32 num_cus;
	279	__u32 cfgs[MAX_NUM_CUS];
	280	} __packed;
	281
	282	struct config_cu_resp {
	283	enum aie2_msg_status status;
	284	} __packed;
	285
	286	struct set_runtime_cfg_req {
	287	__u32 type;
	288	__u64 value;
	289	} __packed;
	290
	291	struct set_runtime_cfg_resp {
	292	enum aie2_msg_status status;
	293	} __packed;
	294
	295	struct get_runtime_cfg_req {
	296	__u32 type;
	297	} __packed;
	298
	299	struct get_runtime_cfg_resp {
	300	enum aie2_msg_status status;
	301	__u64 value;
	302	} __packed;
	303
	304	enum async_event_type {
	305	ASYNC_EVENT_TYPE_AIE_ERROR,
	306	ASYNC_EVENT_TYPE_EXCEPTION,
	307	MAX_ASYNC_EVENT_TYPE
	308	};
	309
	310	#define ASYNC_BUF_SIZE SZ_8K
	311	struct async_event_msg_req {
	312	__u64 buf_addr;
	313	__u32 buf_size;
	314	} __packed;
	315
	316	struct async_event_msg_resp {
	317	enum aie2_msg_status status;
	318	enum async_event_type type;
	319	} __packed;
	320
	321	#define MAX_CHAIN_CMDBUF_SIZE SZ_4K
	322	#define slot_cf_has_space(offset, payload_size) \
	323	(MAX_CHAIN_CMDBUF_SIZE - ((offset) + (payload_size)) > \
	324	offsetof(struct cmd_chain_slot_execbuf_cf, args[0]))
	325	struct cmd_chain_slot_execbuf_cf {
	326	__u32 cu_idx;
	327	__u32 arg_cnt;
	328	__u32 args[] __counted_by(arg_cnt);
	329	};
	330
	331	#define slot_dpu_has_space(offset, payload_size) \
	332	(MAX_CHAIN_CMDBUF_SIZE - ((offset) + (payload_size)) > \
	333	offsetof(struct cmd_chain_slot_dpu, args[0]))
	334	struct cmd_chain_slot_dpu {
	335	__u64 inst_buf_addr;
	336	__u32 inst_size;
	337	__u32 inst_prop_cnt;
	338	__u32 cu_idx;
	339	__u32 arg_cnt;
	340	#define MAX_DPU_ARGS_SIZE (34 * sizeof(__u32))
	341	__u32 args[] __counted_by(arg_cnt);
	342	};
	343
	344	struct cmd_chain_req {
	345	__u64 buf_addr;
	346	__u32 buf_size;
	347	__u32 count;
	348	} __packed;
	349
	350	struct cmd_chain_resp {
	351	enum aie2_msg_status status;
	352	__u32 fail_cmd_idx;
	353	enum aie2_msg_status fail_cmd_status;
	354	} __packed;
	355
	356	#define AIE2_MSG_SYNC_BO_SRC_TYPE GENMASK(3, 0)
	357	#define AIE2_MSG_SYNC_BO_DST_TYPE GENMASK(7, 4)
	358	struct sync_bo_req {
	359	__u64 src_addr;
	360	__u64 dst_addr;
	361	__u32 size;
	362	#define SYNC_BO_DEV_MEM 0
	363	#define SYNC_BO_HOST_MEM 2
	364	__u32 type;
	365	} __packed;
	366
	367	struct sync_bo_resp {
	368	enum aie2_msg_status status;
	369	} __packed;
	370	#endif /* _AIE2_MSG_PRIV_H_ */

drivers/accel/amdxdna/aie2_pci.c+762−0

@@ -0,0 +1,762 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/drm_drv.h>
	9	#include <drm/drm_gem_shmem_helper.h>
	10	#include <drm/drm_managed.h>
	11	#include <drm/drm_print.h>
	12	#include <drm/gpu_scheduler.h>
	13	#include <linux/errno.h>
	14	#include <linux/firmware.h>
	15	#include <linux/iommu.h>
	16	#include <linux/iopoll.h>
	17	#include <linux/pci.h>
	18
	19	#include "aie2_msg_priv.h"
	20	#include "aie2_pci.h"
	21	#include "aie2_solver.h"
	22	#include "amdxdna_ctx.h"
	23	#include "amdxdna_gem.h"
	24	#include "amdxdna_mailbox.h"
	25	#include "amdxdna_pci_drv.h"
	26
	27	int aie2_max_col = XRS_MAX_COL;
	28	module_param(aie2_max_col, uint, 0600);
	29	MODULE_PARM_DESC(aie2_max_col, "Maximum column could be used");
	30
	31	/*
	32	* The management mailbox channel is allocated by firmware.
	33	* The related register and ring buffer information is on SRAM BAR.
	34	* This struct is the register layout.
	35	*/
	36	struct mgmt_mbox_chann_info {
	37	u32 x2i_tail;
	38	u32 x2i_head;
	39	u32 x2i_buf;
	40	u32 x2i_buf_sz;
	41	u32 i2x_tail;
	42	u32 i2x_head;
	43	u32 i2x_buf;
	44	u32 i2x_buf_sz;
	45	};
	46
	47	static void aie2_dump_chann_info_debug(struct amdxdna_dev_hdl *ndev)
	48	{
	49	struct amdxdna_dev *xdna = ndev->xdna;
	50
	51	XDNA_DBG(xdna, "i2x tail 0x%x", ndev->mgmt_i2x.mb_tail_ptr_reg);
	52	XDNA_DBG(xdna, "i2x head 0x%x", ndev->mgmt_i2x.mb_head_ptr_reg);
	53	XDNA_DBG(xdna, "i2x ringbuf 0x%x", ndev->mgmt_i2x.rb_start_addr);
	54	XDNA_DBG(xdna, "i2x rsize 0x%x", ndev->mgmt_i2x.rb_size);
	55	XDNA_DBG(xdna, "x2i tail 0x%x", ndev->mgmt_x2i.mb_tail_ptr_reg);
	56	XDNA_DBG(xdna, "x2i head 0x%x", ndev->mgmt_x2i.mb_head_ptr_reg);
	57	XDNA_DBG(xdna, "x2i ringbuf 0x%x", ndev->mgmt_x2i.rb_start_addr);
	58	XDNA_DBG(xdna, "x2i rsize 0x%x", ndev->mgmt_x2i.rb_size);
	59	XDNA_DBG(xdna, "x2i chann index 0x%x", ndev->mgmt_chan_idx);
	60	}
	61
	62	static int aie2_get_mgmt_chann_info(struct amdxdna_dev_hdl *ndev)
	63	{
	64	struct mgmt_mbox_chann_info info_regs;
	65	struct xdna_mailbox_chann_res *i2x;
	66	struct xdna_mailbox_chann_res *x2i;
	67	u32 addr, off;
	68	u32 *reg;
	69	int ret;
	70	int i;
	71
	72	/*
	73	* Once firmware is alive, it will write management channel
	74	* information in SRAM BAR and write the address of that information
	75	* at FW_ALIVE_OFF offset in SRMA BAR.
	76	*
	77	* Read a non-zero value from FW_ALIVE_OFF implies that firmware
	78	* is alive.
	79	*/
	80	ret = readx_poll_timeout(readl, SRAM_GET_ADDR(ndev, FW_ALIVE_OFF),
	81	addr, addr, AIE2_INTERVAL, AIE2_TIMEOUT);
	82	if (ret \|\| !addr)
	83	return -ETIME;
	84
	85	off = AIE2_SRAM_OFF(ndev, addr);
	86	reg = (u32 *)&info_regs;
	87	for (i = 0; i < sizeof(info_regs) / sizeof(u32); i++)
	88	reg[i] = readl(ndev->sram_base + off + i * sizeof(u32));
	89
	90	i2x = &ndev->mgmt_i2x;
	91	x2i = &ndev->mgmt_x2i;
	92
	93	i2x->mb_head_ptr_reg = AIE2_MBOX_OFF(ndev, info_regs.i2x_head);
	94	i2x->mb_tail_ptr_reg = AIE2_MBOX_OFF(ndev, info_regs.i2x_tail);
	95	i2x->rb_start_addr = AIE2_SRAM_OFF(ndev, info_regs.i2x_buf);
	96	i2x->rb_size = info_regs.i2x_buf_sz;
	97
	98	x2i->mb_head_ptr_reg = AIE2_MBOX_OFF(ndev, info_regs.x2i_head);
	99	x2i->mb_tail_ptr_reg = AIE2_MBOX_OFF(ndev, info_regs.x2i_tail);
	100	x2i->rb_start_addr = AIE2_SRAM_OFF(ndev, info_regs.x2i_buf);
	101	x2i->rb_size = info_regs.x2i_buf_sz;
	102	ndev->mgmt_chan_idx = CHANN_INDEX(ndev, x2i->rb_start_addr);
	103
	104	aie2_dump_chann_info_debug(ndev);
	105
	106	/* Must clear address at FW_ALIVE_OFF */
	107	writel(0, SRAM_GET_ADDR(ndev, FW_ALIVE_OFF));
	108
	109	return 0;
	110	}
	111
	112	static int aie2_runtime_cfg(struct amdxdna_dev_hdl *ndev)
	113	{
	114	const struct rt_config *cfg = &ndev->priv->rt_config;
	115	u64 value;
	116	int ret;
	117
	118	ret = aie2_set_runtime_cfg(ndev, cfg->type, cfg->value);
	119	if (ret) {
	120	XDNA_ERR(ndev->xdna, "Set runtime type %d value %d failed",
	121	cfg->type, cfg->value);
	122	return ret;
	123	}
	124
	125	ret = aie2_get_runtime_cfg(ndev, cfg->type, &value);
	126	if (ret) {
	127	XDNA_ERR(ndev->xdna, "Get runtime cfg failed");
	128	return ret;
	129	}
	130
	131	if (value != cfg->value)
	132	return -EINVAL;
	133
	134	return 0;
	135	}
	136
	137	static int aie2_xdna_reset(struct amdxdna_dev_hdl *ndev)
	138	{
	139	int ret;
	140
	141	ret = aie2_suspend_fw(ndev);
	142	if (ret) {
	143	XDNA_ERR(ndev->xdna, "Suspend firmware failed");
	144	return ret;
	145	}
	146
	147	ret = aie2_resume_fw(ndev);
	148	if (ret) {
	149	XDNA_ERR(ndev->xdna, "Resume firmware failed");
	150	return ret;
	151	}
	152
	153	return 0;
	154	}
	155
	156	static int aie2_mgmt_fw_init(struct amdxdna_dev_hdl *ndev)
	157	{
	158	int ret;
	159
	160	ret = aie2_check_protocol_version(ndev);
	161	if (ret) {
	162	XDNA_ERR(ndev->xdna, "Check header hash failed");
	163	return ret;
	164	}
	165
	166	ret = aie2_runtime_cfg(ndev);
	167	if (ret) {
	168	XDNA_ERR(ndev->xdna, "Runtime config failed");
	169	return ret;
	170	}
	171
	172	ret = aie2_assign_mgmt_pasid(ndev, 0);
	173	if (ret) {
	174	XDNA_ERR(ndev->xdna, "Can not assign PASID");
	175	return ret;
	176	}
	177
	178	ret = aie2_xdna_reset(ndev);
	179	if (ret) {
	180	XDNA_ERR(ndev->xdna, "Reset firmware failed");
	181	return ret;
	182	}
	183
	184	if (!ndev->async_events)
	185	return 0;
	186
	187	ret = aie2_error_async_events_send(ndev);
	188	if (ret) {
	189	XDNA_ERR(ndev->xdna, "Send async events failed");
	190	return ret;
	191	}
	192
	193	return 0;
	194	}
	195
	196	static int aie2_mgmt_fw_query(struct amdxdna_dev_hdl *ndev)
	197	{
	198	int ret;
	199
	200	ret = aie2_query_firmware_version(ndev, &ndev->xdna->fw_ver);
	201	if (ret) {
	202	XDNA_ERR(ndev->xdna, "query firmware version failed");
	203	return ret;
	204	}
	205
	206	ret = aie2_query_aie_version(ndev, &ndev->version);
	207	if (ret) {
	208	XDNA_ERR(ndev->xdna, "Query AIE version failed");
	209	return ret;
	210	}
	211
	212	ret = aie2_query_aie_metadata(ndev, &ndev->metadata);
	213	if (ret) {
	214	XDNA_ERR(ndev->xdna, "Query AIE metadata failed");
	215	return ret;
	216	}
	217
	218	return 0;
	219	}
	220
	221	static void aie2_mgmt_fw_fini(struct amdxdna_dev_hdl *ndev)
	222	{
	223	if (aie2_suspend_fw(ndev))
	224	XDNA_ERR(ndev->xdna, "Suspend_fw failed");
	225	XDNA_DBG(ndev->xdna, "Firmware suspended");
	226	}
	227
	228	static int aie2_xrs_load(void cb_arg, struct xrs_action_load action)
	229	{
	230	struct amdxdna_hwctx *hwctx = cb_arg;
	231	struct amdxdna_dev *xdna;
	232	int ret;
	233
	234	xdna = hwctx->client->xdna;
	235
	236	hwctx->start_col = action->part.start_col;
	237	hwctx->num_col = action->part.ncols;
	238	ret = aie2_create_context(xdna->dev_handle, hwctx);
	239	if (ret)
	240	XDNA_ERR(xdna, "create context failed, ret %d", ret);
	241
	242	return ret;
	243	}
	244
	245	static int aie2_xrs_unload(void *cb_arg)
	246	{
	247	struct amdxdna_hwctx *hwctx = cb_arg;
	248	struct amdxdna_dev *xdna;
	249	int ret;
	250
	251	xdna = hwctx->client->xdna;
	252
	253	ret = aie2_destroy_context(xdna->dev_handle, hwctx);
	254	if (ret)
	255	XDNA_ERR(xdna, "destroy context failed, ret %d", ret);
	256
	257	return ret;
	258	}
	259
	260	static struct xrs_action_ops aie2_xrs_actions = {
	261	.load = aie2_xrs_load,
	262	.unload = aie2_xrs_unload,
	263	};
	264
	265	static void aie2_hw_stop(struct amdxdna_dev *xdna)
	266	{
	267	struct pci_dev *pdev = to_pci_dev(xdna->ddev.dev);
	268	struct amdxdna_dev_hdl *ndev = xdna->dev_handle;
	269
	270	aie2_mgmt_fw_fini(ndev);
	271	xdna_mailbox_stop_channel(ndev->mgmt_chann);
	272	xdna_mailbox_destroy_channel(ndev->mgmt_chann);
	273	aie2_psp_stop(ndev->psp_hdl);
	274	aie2_smu_fini(ndev);
	275	pci_disable_device(pdev);
	276	}
	277
	278	static int aie2_hw_start(struct amdxdna_dev *xdna)
	279	{
	280	struct pci_dev *pdev = to_pci_dev(xdna->ddev.dev);
	281	struct amdxdna_dev_hdl *ndev = xdna->dev_handle;
	282	struct xdna_mailbox_res mbox_res;
	283	u32 xdna_mailbox_intr_reg;
	284	int mgmt_mb_irq, ret;
	285
	286	ret = pci_enable_device(pdev);
	287	if (ret) {
	288	XDNA_ERR(xdna, "failed to enable device, ret %d", ret);
	289	return ret;
	290	}
	291	pci_set_master(pdev);
	292
	293	ret = aie2_smu_init(ndev);
	294	if (ret) {
	295	XDNA_ERR(xdna, "failed to init smu, ret %d", ret);
	296	goto disable_dev;
	297	}
	298
	299	ret = aie2_psp_start(ndev->psp_hdl);
	300	if (ret) {
	301	XDNA_ERR(xdna, "failed to start psp, ret %d", ret);
	302	goto fini_smu;
	303	}
	304
	305	ret = aie2_get_mgmt_chann_info(ndev);
	306	if (ret) {
	307	XDNA_ERR(xdna, "firmware is not alive");
	308	goto stop_psp;
	309	}
	310
	311	mbox_res.ringbuf_base = (u64)ndev->sram_base;
	312	mbox_res.ringbuf_size = pci_resource_len(pdev, xdna->dev_info->sram_bar);
	313	mbox_res.mbox_base = (u64)ndev->mbox_base;
	314	mbox_res.mbox_size = MBOX_SIZE(ndev);
	315	mbox_res.name = "xdna_mailbox";
	316	ndev->mbox = xdnam_mailbox_create(&xdna->ddev, &mbox_res);
	317	if (!ndev->mbox) {
	318	XDNA_ERR(xdna, "failed to create mailbox device");
	319	ret = -ENODEV;
	320	goto stop_psp;
	321	}
	322
	323	mgmt_mb_irq = pci_irq_vector(pdev, ndev->mgmt_chan_idx);
	324	if (mgmt_mb_irq < 0) {
	325	ret = mgmt_mb_irq;
	326	XDNA_ERR(xdna, "failed to alloc irq vector, ret %d", ret);
	327	goto stop_psp;
	328	}
	329
	330	xdna_mailbox_intr_reg = ndev->mgmt_i2x.mb_head_ptr_reg + 4;
	331	ndev->mgmt_chann = xdna_mailbox_create_channel(ndev->mbox,
	332	&ndev->mgmt_x2i,
	333	&ndev->mgmt_i2x,
	334	xdna_mailbox_intr_reg,
	335	mgmt_mb_irq);
	336	if (!ndev->mgmt_chann) {
	337	XDNA_ERR(xdna, "failed to create management mailbox channel");
	338	ret = -EINVAL;
	339	goto stop_psp;
	340	}
	341
	342	ret = aie2_mgmt_fw_init(ndev);
	343	if (ret) {
	344	XDNA_ERR(xdna, "initial mgmt firmware failed, ret %d", ret);
	345	goto destroy_mgmt_chann;
	346	}
	347
	348	return 0;
	349
	350	destroy_mgmt_chann:
	351	xdna_mailbox_stop_channel(ndev->mgmt_chann);
	352	xdna_mailbox_destroy_channel(ndev->mgmt_chann);
	353	stop_psp:
	354	aie2_psp_stop(ndev->psp_hdl);
	355	fini_smu:
	356	aie2_smu_fini(ndev);
	357	disable_dev:
	358	pci_disable_device(pdev);
	359
	360	return ret;
	361	}
	362
	363	static int aie2_init(struct amdxdna_dev *xdna)
	364	{
	365	struct pci_dev *pdev = to_pci_dev(xdna->ddev.dev);
	366	void __iomem *tbl[PCI_NUM_RESOURCES] = {0};
	367	struct init_config xrs_cfg = { 0 };
	368	struct amdxdna_dev_hdl *ndev;
	369	struct psp_config psp_conf;
	370	const struct firmware *fw;
	371	unsigned long bars = 0;
	372	int i, nvec, ret;
	373
	374	ndev = drmm_kzalloc(&xdna->ddev, sizeof(*ndev), GFP_KERNEL);
	375	if (!ndev)
	376	return -ENOMEM;
	377
	378	ndev->priv = xdna->dev_info->dev_priv;
	379	ndev->xdna = xdna;
	380
	381	ret = request_firmware(&fw, ndev->priv->fw_path, &pdev->dev);
	382	if (ret) {
	383	XDNA_ERR(xdna, "failed to request_firmware %s, ret %d",
	384	ndev->priv->fw_path, ret);
	385	return ret;
	386	}
	387
	388	ret = pcim_enable_device(pdev);
	389	if (ret) {
	390	XDNA_ERR(xdna, "pcim enable device failed, ret %d", ret);
	391	goto release_fw;
	392	}
	393
	394	for (i = 0; i < PSP_MAX_REGS; i++)
	395	set_bit(PSP_REG_BAR(ndev, i), &bars);
	396
	397	set_bit(xdna->dev_info->sram_bar, &bars);
	398	set_bit(xdna->dev_info->smu_bar, &bars);
	399	set_bit(xdna->dev_info->mbox_bar, &bars);
	400
	401	for (i = 0; i < PCI_NUM_RESOURCES; i++) {
	402	if (!test_bit(i, &bars))
	403	continue;
	404	tbl[i] = pcim_iomap(pdev, i, 0);
	405	if (!tbl[i]) {
	406	XDNA_ERR(xdna, "map bar %d failed", i);
	407	ret = -ENOMEM;
	408	goto release_fw;
	409	}
	410	}
	411
	412	ndev->sram_base = tbl[xdna->dev_info->sram_bar];
	413	ndev->smu_base = tbl[xdna->dev_info->smu_bar];
	414	ndev->mbox_base = tbl[xdna->dev_info->mbox_bar];
	415
	416	ret = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
	417	if (ret) {
	418	XDNA_ERR(xdna, "Failed to set DMA mask: %d", ret);
	419	goto release_fw;
	420	}
	421
	422	nvec = pci_msix_vec_count(pdev);
	423	if (nvec <= 0) {
	424	XDNA_ERR(xdna, "does not get number of interrupt vector");
	425	ret = -EINVAL;
	426	goto release_fw;
	427	}
	428
	429	ret = pci_alloc_irq_vectors(pdev, nvec, nvec, PCI_IRQ_MSIX);
	430	if (ret < 0) {
	431	XDNA_ERR(xdna, "failed to alloc irq vectors, ret %d", ret);
	432	goto release_fw;
	433	}
	434
	435	ret = iommu_dev_enable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA);
	436	if (ret) {
	437	XDNA_ERR(xdna, "Enable PASID failed, ret %d", ret);
	438	goto free_irq;
	439	}
	440
	441	psp_conf.fw_size = fw->size;
	442	psp_conf.fw_buf = fw->data;
	443	for (i = 0; i < PSP_MAX_REGS; i++)
	444	psp_conf.psp_regs[i] = tbl[PSP_REG_BAR(ndev, i)] + PSP_REG_OFF(ndev, i);
	445	ndev->psp_hdl = aie2m_psp_create(&xdna->ddev, &psp_conf);
	446	if (!ndev->psp_hdl) {
	447	XDNA_ERR(xdna, "failed to create psp");
	448	ret = -ENOMEM;
	449	goto disable_sva;
	450	}
	451	xdna->dev_handle = ndev;
	452
	453	ret = aie2_hw_start(xdna);
	454	if (ret) {
	455	XDNA_ERR(xdna, "start npu failed, ret %d", ret);
	456	goto disable_sva;
	457	}
	458
	459	ret = aie2_mgmt_fw_query(ndev);
	460	if (ret) {
	461	XDNA_ERR(xdna, "Query firmware failed, ret %d", ret);
	462	goto stop_hw;
	463	}
	464	ndev->total_col = min(aie2_max_col, ndev->metadata.cols);
	465
	466	xrs_cfg.clk_list.num_levels = 3;
	467	xrs_cfg.clk_list.cu_clk_list[0] = 0;
	468	xrs_cfg.clk_list.cu_clk_list[1] = 800;
	469	xrs_cfg.clk_list.cu_clk_list[2] = 1000;
	470	xrs_cfg.sys_eff_factor = 1;
	471	xrs_cfg.ddev = &xdna->ddev;
	472	xrs_cfg.actions = &aie2_xrs_actions;
	473	xrs_cfg.total_col = ndev->total_col;
	474
	475	xdna->xrs_hdl = xrsm_init(&xrs_cfg);
	476	if (!xdna->xrs_hdl) {
	477	XDNA_ERR(xdna, "Initialize resolver failed");
	478	ret = -EINVAL;
	479	goto stop_hw;
	480	}
	481
	482	ret = aie2_error_async_events_alloc(ndev);
	483	if (ret) {
	484	XDNA_ERR(xdna, "Allocate async events failed, ret %d", ret);
	485	goto stop_hw;
	486	}
	487
	488	ret = aie2_error_async_events_send(ndev);
	489	if (ret) {
	490	XDNA_ERR(xdna, "Send async events failed, ret %d", ret);
	491	goto async_event_free;
	492	}
	493
	494	/* Issue a command to make sure firmware handled async events */
	495	ret = aie2_query_firmware_version(ndev, &ndev->xdna->fw_ver);
	496	if (ret) {
	497	XDNA_ERR(xdna, "Re-query firmware version failed");
	498	goto async_event_free;
	499	}
	500
	501	release_firmware(fw);
	502	return 0;
	503
	504	async_event_free:
	505	aie2_error_async_events_free(ndev);
	506	stop_hw:
	507	aie2_hw_stop(xdna);
	508	disable_sva:
	509	iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA);
	510	free_irq:
	511	pci_free_irq_vectors(pdev);
	512	release_fw:
	513	release_firmware(fw);
	514
	515	return ret;
	516	}
	517
	518	static void aie2_fini(struct amdxdna_dev *xdna)
	519	{
	520	struct pci_dev *pdev = to_pci_dev(xdna->ddev.dev);
	521	struct amdxdna_dev_hdl *ndev = xdna->dev_handle;
	522
	523	aie2_hw_stop(xdna);
	524	aie2_error_async_events_free(ndev);
	525	iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA);
	526	pci_free_irq_vectors(pdev);
	527	}
	528
	529	static int aie2_get_aie_status(struct amdxdna_client *client,
	530	struct amdxdna_drm_get_info *args)
	531	{
	532	struct amdxdna_drm_query_aie_status status;
	533	struct amdxdna_dev *xdna = client->xdna;
	534	struct amdxdna_dev_hdl *ndev;
	535	int ret;
	536
	537	ndev = xdna->dev_handle;
	538	if (copy_from_user(&status, u64_to_user_ptr(args->buffer), sizeof(status))) {
	539	XDNA_ERR(xdna, "Failed to copy AIE request into kernel");
	540	return -EFAULT;
	541	}
	542
	543	if (ndev->metadata.cols * ndev->metadata.size < status.buffer_size) {
	544	XDNA_ERR(xdna, "Invalid buffer size. Given Size: %u. Need Size: %u.",
	545	status.buffer_size, ndev->metadata.cols * ndev->metadata.size);
	546	return -EINVAL;
	547	}
	548
	549	ret = aie2_query_status(ndev, u64_to_user_ptr(status.buffer),
	550	status.buffer_size, &status.cols_filled);
	551	if (ret) {
	552	XDNA_ERR(xdna, "Failed to get AIE status info. Ret: %d", ret);
	553	return ret;
	554	}
	555
	556	if (copy_to_user(u64_to_user_ptr(args->buffer), &status, sizeof(status))) {
	557	XDNA_ERR(xdna, "Failed to copy AIE request info to user space");
	558	return -EFAULT;
	559	}
	560
	561	return 0;
	562	}
	563
	564	static int aie2_get_aie_metadata(struct amdxdna_client *client,
	565	struct amdxdna_drm_get_info *args)
	566	{
	567	struct amdxdna_drm_query_aie_metadata *meta;
	568	struct amdxdna_dev *xdna = client->xdna;
	569	struct amdxdna_dev_hdl *ndev;
	570	int ret = 0;
	571
	572	ndev = xdna->dev_handle;
	573	meta = kzalloc(sizeof(*meta), GFP_KERNEL);
	574	if (!meta)
	575	return -ENOMEM;
	576
	577	meta->col_size = ndev->metadata.size;
	578	meta->cols = ndev->metadata.cols;
	579	meta->rows = ndev->metadata.rows;
	580
	581	meta->version.major = ndev->metadata.version.major;
	582	meta->version.minor = ndev->metadata.version.minor;
	583
	584	meta->core.row_count = ndev->metadata.core.row_count;
	585	meta->core.row_start = ndev->metadata.core.row_start;
	586	meta->core.dma_channel_count = ndev->metadata.core.dma_channel_count;
	587	meta->core.lock_count = ndev->metadata.core.lock_count;
	588	meta->core.event_reg_count = ndev->metadata.core.event_reg_count;
	589
	590	meta->mem.row_count = ndev->metadata.mem.row_count;
	591	meta->mem.row_start = ndev->metadata.mem.row_start;
	592	meta->mem.dma_channel_count = ndev->metadata.mem.dma_channel_count;
	593	meta->mem.lock_count = ndev->metadata.mem.lock_count;
	594	meta->mem.event_reg_count = ndev->metadata.mem.event_reg_count;
	595
	596	meta->shim.row_count = ndev->metadata.shim.row_count;
	597	meta->shim.row_start = ndev->metadata.shim.row_start;
	598	meta->shim.dma_channel_count = ndev->metadata.shim.dma_channel_count;
	599	meta->shim.lock_count = ndev->metadata.shim.lock_count;
	600	meta->shim.event_reg_count = ndev->metadata.shim.event_reg_count;
	601
	602	if (copy_to_user(u64_to_user_ptr(args->buffer), meta, sizeof(*meta)))
	603	ret = -EFAULT;
	604
	605	kfree(meta);
	606	return ret;
	607	}
	608
	609	static int aie2_get_aie_version(struct amdxdna_client *client,
	610	struct amdxdna_drm_get_info *args)
	611	{
	612	struct amdxdna_drm_query_aie_version version;
	613	struct amdxdna_dev *xdna = client->xdna;
	614	struct amdxdna_dev_hdl *ndev;
	615
	616	ndev = xdna->dev_handle;
	617	version.major = ndev->version.major;
	618	version.minor = ndev->version.minor;
	619
	620	if (copy_to_user(u64_to_user_ptr(args->buffer), &version, sizeof(version)))
	621	return -EFAULT;
	622
	623	return 0;
	624	}
	625
	626	static int aie2_get_clock_metadata(struct amdxdna_client *client,
	627	struct amdxdna_drm_get_info *args)
	628	{
	629	struct amdxdna_drm_query_clock_metadata *clock;
	630	struct amdxdna_dev *xdna = client->xdna;
	631	struct amdxdna_dev_hdl *ndev;
	632	int ret = 0;
	633
	634	ndev = xdna->dev_handle;
	635	clock = kzalloc(sizeof(*clock), GFP_KERNEL);
	636	if (!clock)
	637	return -ENOMEM;
	638
	639	memcpy(clock->mp_npu_clock.name, ndev->mp_npu_clock.name,
	640	sizeof(clock->mp_npu_clock.name));
	641	clock->mp_npu_clock.freq_mhz = ndev->mp_npu_clock.freq_mhz;
	642	memcpy(clock->h_clock.name, ndev->h_clock.name, sizeof(clock->h_clock.name));
	643	clock->h_clock.freq_mhz = ndev->h_clock.freq_mhz;
	644
	645	if (copy_to_user(u64_to_user_ptr(args->buffer), clock, sizeof(*clock)))
	646	ret = -EFAULT;
	647
	648	kfree(clock);
	649	return ret;
	650	}
	651
	652	static int aie2_get_hwctx_status(struct amdxdna_client *client,
	653	struct amdxdna_drm_get_info *args)
	654	{
	655	struct amdxdna_drm_query_hwctx __user *buf;
	656	struct amdxdna_dev *xdna = client->xdna;
	657	struct amdxdna_drm_query_hwctx *tmp;
	658	struct amdxdna_client *tmp_client;
	659	struct amdxdna_hwctx *hwctx;
	660	bool overflow = false;
	661	u32 req_bytes = 0;
	662	u32 hw_i = 0;
	663	int ret = 0;
	664	int next;
	665	int idx;
	666
	667	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	668
	669	tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
	670	if (!tmp)
	671	return -ENOMEM;
	672
	673	buf = u64_to_user_ptr(args->buffer);
	674	list_for_each_entry(tmp_client, &xdna->client_list, node) {
	675	idx = srcu_read_lock(&tmp_client->hwctx_srcu);
	676	next = 0;
	677	idr_for_each_entry_continue(&tmp_client->hwctx_idr, hwctx, next) {
	678	req_bytes += sizeof(*tmp);
	679	if (args->buffer_size < req_bytes) {
	680	/* Continue iterating to get the required size */
	681	overflow = true;
	682	continue;
	683	}
	684
	685	memset(tmp, 0, sizeof(*tmp));
	686	tmp->pid = tmp_client->pid;
	687	tmp->context_id = hwctx->id;
	688	tmp->start_col = hwctx->start_col;
	689	tmp->num_col = hwctx->num_col;
	690	tmp->command_submissions = hwctx->priv->seq;
	691	tmp->command_completions = hwctx->priv->completed;
	692
	693	if (copy_to_user(&buf[hw_i], tmp, sizeof(*tmp))) {
	694	ret = -EFAULT;
	695	srcu_read_unlock(&tmp_client->hwctx_srcu, idx);
	696	goto out;
	697	}
	698	hw_i++;
	699	}
	700	srcu_read_unlock(&tmp_client->hwctx_srcu, idx);
	701	}
	702
	703	if (overflow) {
	704	XDNA_ERR(xdna, "Invalid buffer size. Given: %u Need: %u.",
	705	args->buffer_size, req_bytes);
	706	ret = -EINVAL;
	707	}
	708
	709	out:
	710	kfree(tmp);
	711	args->buffer_size = req_bytes;
	712	return ret;
	713	}
	714
	715	static int aie2_get_info(struct amdxdna_client client, struct amdxdna_drm_get_info args)
	716	{
	717	struct amdxdna_dev *xdna = client->xdna;
	718	int ret, idx;
	719
	720	if (!drm_dev_enter(&xdna->ddev, &idx))
	721	return -ENODEV;
	722
	723	switch (args->param) {
	724	case DRM_AMDXDNA_QUERY_AIE_STATUS:
	725	ret = aie2_get_aie_status(client, args);
	726	break;
	727	case DRM_AMDXDNA_QUERY_AIE_METADATA:
	728	ret = aie2_get_aie_metadata(client, args);
	729	break;
	730	case DRM_AMDXDNA_QUERY_AIE_VERSION:
	731	ret = aie2_get_aie_version(client, args);
	732	break;
	733	case DRM_AMDXDNA_QUERY_CLOCK_METADATA:
	734	ret = aie2_get_clock_metadata(client, args);
	735	break;
	736	case DRM_AMDXDNA_QUERY_HW_CONTEXTS:
	737	ret = aie2_get_hwctx_status(client, args);
	738	break;
	739	default:
	740	XDNA_ERR(xdna, "Not supported request parameter %u", args->param);
	741	ret = -EOPNOTSUPP;
	742	}
	743	XDNA_DBG(xdna, "Got param %d", args->param);
	744
	745	drm_dev_exit(idx);
	746	return ret;
	747	}
	748
	749	const struct amdxdna_dev_ops aie2_ops = {
	750	.init = aie2_init,
	751	.fini = aie2_fini,
	752	.resume = aie2_hw_start,
	753	.suspend = aie2_hw_stop,
	754	.get_aie_info = aie2_get_info,
	755	.hwctx_init = aie2_hwctx_init,
	756	.hwctx_fini = aie2_hwctx_fini,
	757	.hwctx_config = aie2_hwctx_config,
	758	.cmd_submit = aie2_cmd_submit,
	759	.hmm_invalidate = aie2_hmm_invalidate,
	760	.hwctx_suspend = aie2_hwctx_suspend,
	761	.hwctx_resume = aie2_hwctx_resume,
	762	};

drivers/accel/amdxdna/aie2_pci.h+259−0

@@ -0,0 +1,259 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _AIE2_PCI_H_
	7	#define _AIE2_PCI_H_
	8
	9	#include <linux/semaphore.h>
	10
	11	#include "amdxdna_mailbox.h"
	12
	13	#define AIE2_INTERVAL 20000 /* us */
	14	#define AIE2_TIMEOUT 1000000 /* us */
	15
	16	/* Firmware determines device memory base address and size */
	17	#define AIE2_DEVM_BASE 0x4000000
	18	#define AIE2_DEVM_SIZE SZ_64M
	19
	20	#define NDEV2PDEV(ndev) (to_pci_dev((ndev)->xdna->ddev.dev))
	21
	22	#define AIE2_SRAM_OFF(ndev, addr) ((addr) - (ndev)->priv->sram_dev_addr)
	23	#define AIE2_MBOX_OFF(ndev, addr) ((addr) - (ndev)->priv->mbox_dev_addr)
	24
	25	#define PSP_REG_BAR(ndev, idx) ((ndev)->priv->psp_regs_off[(idx)].bar_idx)
	26	#define PSP_REG_OFF(ndev, idx) ((ndev)->priv->psp_regs_off[(idx)].offset)
	27	#define SRAM_REG_OFF(ndev, idx) ((ndev)->priv->sram_offs[(idx)].offset)
	28
	29	#define SMU_REG(ndev, idx) \
	30	({ \
	31	typeof(ndev) _ndev = ndev; \
	32	((_ndev)->smu_base + (_ndev)->priv->smu_regs_off[(idx)].offset); \
	33	})
	34	#define SRAM_GET_ADDR(ndev, idx) \
	35	({ \
	36	typeof(ndev) _ndev = ndev; \
	37	((_ndev)->sram_base + SRAM_REG_OFF((_ndev), (idx))); \
	38	})
	39
	40	#define CHAN_SLOT_SZ SZ_8K
	41	#define CHANN_INDEX(ndev, rbuf_off) \
	42	(((rbuf_off) - SRAM_REG_OFF((ndev), MBOX_CHANN_OFF)) / CHAN_SLOT_SZ)
	43
	44	#define MBOX_SIZE(ndev) \
	45	({ \
	46	typeof(ndev) _ndev = (ndev); \
	47	((_ndev)->priv->mbox_size) ? (_ndev)->priv->mbox_size : \
	48	pci_resource_len(NDEV2PDEV(_ndev), (_ndev)->xdna->dev_info->mbox_bar); \
	49	})
	50
	51	#define SMU_MPNPUCLK_FREQ_MAX(ndev) ((ndev)->priv->smu_mpnpuclk_freq_max)
	52	#define SMU_HCLK_FREQ_MAX(ndev) ((ndev)->priv->smu_hclk_freq_max)
	53
	54	enum aie2_smu_reg_idx {
	55	SMU_CMD_REG = 0,
	56	SMU_ARG_REG,
	57	SMU_INTR_REG,
	58	SMU_RESP_REG,
	59	SMU_OUT_REG,
	60	SMU_MAX_REGS /* Keep this at the end */
	61	};
	62
	63	enum aie2_sram_reg_idx {
	64	MBOX_CHANN_OFF = 0,
	65	FW_ALIVE_OFF,
	66	SRAM_MAX_INDEX /* Keep this at the end */
	67	};
	68
	69	enum psp_reg_idx {
	70	PSP_CMD_REG = 0,
	71	PSP_ARG0_REG,
	72	PSP_ARG1_REG,
	73	PSP_ARG2_REG,
	74	PSP_NUM_IN_REGS, /* number of input registers */
	75	PSP_INTR_REG = PSP_NUM_IN_REGS,
	76	PSP_STATUS_REG,
	77	PSP_RESP_REG,
	78	PSP_MAX_REGS /* Keep this at the end */
	79	};
	80
	81	struct amdxdna_client;
	82	struct amdxdna_fw_ver;
	83	struct amdxdna_hwctx;
	84	struct amdxdna_sched_job;
	85
	86	struct psp_config {
	87	const void *fw_buf;
	88	u32 fw_size;
	89	void __iomem *psp_regs[PSP_MAX_REGS];
	90	};
	91
	92	struct aie_version {
	93	u16 major;
	94	u16 minor;
	95	};
	96
	97	struct aie_tile_metadata {
	98	u16 row_count;
	99	u16 row_start;
	100	u16 dma_channel_count;
	101	u16 lock_count;
	102	u16 event_reg_count;
	103	};
	104
	105	struct aie_metadata {
	106	u32 size;
	107	u16 cols;
	108	u16 rows;
	109	struct aie_version version;
	110	struct aie_tile_metadata core;
	111	struct aie_tile_metadata mem;
	112	struct aie_tile_metadata shim;
	113	};
	114
	115	struct clock_entry {
	116	char name[16];
	117	u32 freq_mhz;
	118	};
	119
	120	struct rt_config {
	121	u32 type;
	122	u32 value;
	123	};
	124
	125	/*
	126	* Define the maximum number of pending commands in a hardware context.
	127	* Must be power of 2!
	128	*/
	129	#define HWCTX_MAX_CMDS 4
	130	#define get_job_idx(seq) ((seq) & (HWCTX_MAX_CMDS - 1))
	131	struct amdxdna_hwctx_priv {
	132	struct amdxdna_gem_obj *heap;
	133	void *mbox_chann;
	134
	135	struct drm_gpu_scheduler sched;
	136	struct drm_sched_entity entity;
	137
	138	struct mutex io_lock; /* protect seq and cmd order */
	139	struct wait_queue_head job_free_wq;
	140	u32 num_pending;
	141	u64 seq;
	142	struct semaphore job_sem;
	143	bool job_done;
	144
	145	/* Completed job counter */
	146	u64 completed;
	147
	148	struct amdxdna_gem_obj *cmd_buf[HWCTX_MAX_CMDS];
	149	struct drm_syncobj *syncobj;
	150	};
	151
	152	struct amdxdna_dev_hdl {
	153	struct amdxdna_dev *xdna;
	154	const struct amdxdna_dev_priv *priv;
	155	void __iomem *sram_base;
	156	void __iomem *smu_base;
	157	void __iomem *mbox_base;
	158	struct psp_device *psp_hdl;
	159
	160	struct xdna_mailbox_chann_res mgmt_x2i;
	161	struct xdna_mailbox_chann_res mgmt_i2x;
	162	u32 mgmt_chan_idx;
	163
	164	u32 total_col;
	165	struct aie_version version;
	166	struct aie_metadata metadata;
	167	struct clock_entry mp_npu_clock;
	168	struct clock_entry h_clock;
	169
	170	/* Mailbox and the management channel */
	171	struct mailbox *mbox;
	172	struct mailbox_channel *mgmt_chann;
	173	struct async_events *async_events;
	174	};
	175
	176	#define DEFINE_BAR_OFFSET(reg_name, bar, reg_addr) \
	177	[reg_name] = {bar##_BAR_INDEX, (reg_addr) - bar##_BAR_BASE}
	178
	179	struct aie2_bar_off_pair {
	180	int bar_idx;
	181	u32 offset;
	182	};
	183
	184	struct amdxdna_dev_priv {
	185	const char *fw_path;
	186	u64 protocol_major;
	187	u64 protocol_minor;
	188	struct rt_config rt_config;
	189	#define COL_ALIGN_NONE 0
	190	#define COL_ALIGN_NATURE 1
	191	u32 col_align;
	192	u32 mbox_dev_addr;
	193	/* If mbox_size is 0, use BAR size. See MBOX_SIZE macro */
	194	u32 mbox_size;
	195	u32 sram_dev_addr;
	196	struct aie2_bar_off_pair sram_offs[SRAM_MAX_INDEX];
	197	struct aie2_bar_off_pair psp_regs_off[PSP_MAX_REGS];
	198	struct aie2_bar_off_pair smu_regs_off[SMU_MAX_REGS];
	199	u32 smu_mpnpuclk_freq_max;
	200	u32 smu_hclk_freq_max;
	201	};
	202
	203	extern const struct amdxdna_dev_ops aie2_ops;
	204
	205	/* aie2_smu.c */
	206	int aie2_smu_init(struct amdxdna_dev_hdl *ndev);
	207	void aie2_smu_fini(struct amdxdna_dev_hdl *ndev);
	208
	209	/* aie2_psp.c */
	210	struct psp_device aie2m_psp_create(struct drm_device ddev, struct psp_config *conf);
	211	int aie2_psp_start(struct psp_device *psp);
	212	void aie2_psp_stop(struct psp_device *psp);
	213
	214	/* aie2_error.c */
	215	int aie2_error_async_events_alloc(struct amdxdna_dev_hdl *ndev);
	216	void aie2_error_async_events_free(struct amdxdna_dev_hdl *ndev);
	217	int aie2_error_async_events_send(struct amdxdna_dev_hdl *ndev);
	218	int aie2_error_async_msg_thread(void *data);
	219
	220	/* aie2_message.c */
	221	int aie2_suspend_fw(struct amdxdna_dev_hdl *ndev);
	222	int aie2_resume_fw(struct amdxdna_dev_hdl *ndev);
	223	int aie2_set_runtime_cfg(struct amdxdna_dev_hdl *ndev, u32 type, u64 value);
	224	int aie2_get_runtime_cfg(struct amdxdna_dev_hdl ndev, u32 type, u64 value);
	225	int aie2_check_protocol_version(struct amdxdna_dev_hdl *ndev);
	226	int aie2_assign_mgmt_pasid(struct amdxdna_dev_hdl *ndev, u16 pasid);
	227	int aie2_query_aie_version(struct amdxdna_dev_hdl ndev, struct aie_version version);
	228	int aie2_query_aie_metadata(struct amdxdna_dev_hdl ndev, struct aie_metadata metadata);
	229	int aie2_query_firmware_version(struct amdxdna_dev_hdl *ndev,
	230	struct amdxdna_fw_ver *fw_ver);
	231	int aie2_create_context(struct amdxdna_dev_hdl ndev, struct amdxdna_hwctx hwctx);
	232	int aie2_destroy_context(struct amdxdna_dev_hdl ndev, struct amdxdna_hwctx hwctx);
	233	int aie2_map_host_buf(struct amdxdna_dev_hdl *ndev, u32 context_id, u64 addr, u64 size);
	234	int aie2_query_status(struct amdxdna_dev_hdl ndev, char buf, u32 size, u32 *cols_filled);
	235	int aie2_register_asyn_event_msg(struct amdxdna_dev_hdl *ndev, dma_addr_t addr, u32 size,
	236	void handle, int (cb)(void, const u32 , size_t));
	237	int aie2_config_cu(struct amdxdna_hwctx *hwctx);
	238	int aie2_execbuf(struct amdxdna_hwctx hwctx, struct amdxdna_sched_job job,
	239	int (notify_cb)(void , const u32 *, size_t));
	240	int aie2_cmdlist_single_execbuf(struct amdxdna_hwctx *hwctx,
	241	struct amdxdna_sched_job *job,
	242	int (notify_cb)(void , const u32 *, size_t));
	243	int aie2_cmdlist_multi_execbuf(struct amdxdna_hwctx *hwctx,
	244	struct amdxdna_sched_job *job,
	245	int (notify_cb)(void , const u32 *, size_t));
	246	int aie2_sync_bo(struct amdxdna_hwctx hwctx, struct amdxdna_sched_job job,
	247	int (notify_cb)(void , const u32 *, size_t));
	248
	249	/* aie2_hwctx.c */
	250	int aie2_hwctx_init(struct amdxdna_hwctx *hwctx);
	251	void aie2_hwctx_fini(struct amdxdna_hwctx *hwctx);
	252	int aie2_hwctx_config(struct amdxdna_hwctx hwctx, u32 type, u64 value, void buf, u32 size);
	253	void aie2_hwctx_suspend(struct amdxdna_hwctx *hwctx);
	254	void aie2_hwctx_resume(struct amdxdna_hwctx *hwctx);
	255	int aie2_cmd_submit(struct amdxdna_hwctx hwctx, struct amdxdna_sched_job job, u64 *seq);
	256	void aie2_hmm_invalidate(struct amdxdna_gem_obj *abo, unsigned long cur_seq);
	257	void aie2_restart_ctx(struct amdxdna_client *client);
	258
	259	#endif /* _AIE2_PCI_H_ */

drivers/accel/amdxdna/aie2_psp.c+146−0

@@ -0,0 +1,146 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/drm_device.h>
	7	#include <drm/drm_gem_shmem_helper.h>
	8	#include <drm/drm_managed.h>
	9	#include <drm/drm_print.h>
	10	#include <drm/gpu_scheduler.h>
	11	#include <linux/bitfield.h>
	12	#include <linux/iopoll.h>
	13
	14	#include "aie2_pci.h"
	15	#include "amdxdna_mailbox.h"
	16	#include "amdxdna_pci_drv.h"
	17
	18	#define PSP_STATUS_READY BIT(31)
	19
	20	/* PSP commands */
	21	#define PSP_VALIDATE 1
	22	#define PSP_START 2
	23	#define PSP_RELEASE_TMR 3
	24
	25	/* PSP special arguments */
	26	#define PSP_START_COPY_FW 1
	27
	28	/* PSP response error code */
	29	#define PSP_ERROR_CANCEL 0xFFFF0002
	30	#define PSP_ERROR_BAD_STATE 0xFFFF0007
	31
	32	#define PSP_FW_ALIGN 0x10000
	33	#define PSP_POLL_INTERVAL 20000 /* us */
	34	#define PSP_POLL_TIMEOUT 1000000 /* us */
	35
	36	#define PSP_REG(p, reg) ((p)->psp_regs[reg])
	37
	38	struct psp_device {
	39	struct drm_device *ddev;
	40	struct psp_config conf;
	41	u32 fw_buf_sz;
	42	u64 fw_paddr;
	43	void *fw_buffer;
	44	void __iomem *psp_regs[PSP_MAX_REGS];
	45	};
	46
	47	static int psp_exec(struct psp_device psp, u32 reg_vals)
	48	{
	49	u32 resp_code;
	50	int ret, i;
	51	u32 ready;
	52
	53	/* Write command and argument registers */
	54	for (i = 0; i < PSP_NUM_IN_REGS; i++)
	55	writel(reg_vals[i], PSP_REG(psp, i));
	56
	57	/* clear and set PSP INTR register to kick off */
	58	writel(0, PSP_REG(psp, PSP_INTR_REG));
	59	writel(1, PSP_REG(psp, PSP_INTR_REG));
	60
	61	/* PSP should be busy. Wait for ready, so we know task is done. */
	62	ret = readx_poll_timeout(readl, PSP_REG(psp, PSP_STATUS_REG), ready,
	63	FIELD_GET(PSP_STATUS_READY, ready),
	64	PSP_POLL_INTERVAL, PSP_POLL_TIMEOUT);
	65	if (ret) {
	66	drm_err(psp->ddev, "PSP is not ready, ret 0x%x", ret);
	67	return ret;
	68	}
	69
	70	resp_code = readl(PSP_REG(psp, PSP_RESP_REG));
	71	if (resp_code) {
	72	drm_err(psp->ddev, "fw return error 0x%x", resp_code);
	73	return -EIO;
	74	}
	75
	76	return 0;
	77	}
	78
	79	void aie2_psp_stop(struct psp_device *psp)
	80	{
	81	u32 reg_vals[PSP_NUM_IN_REGS] = { PSP_RELEASE_TMR, };
	82	int ret;
	83
	84	ret = psp_exec(psp, reg_vals);
	85	if (ret)
	86	drm_err(psp->ddev, "release tmr failed, ret %d", ret);
	87	}
	88
	89	int aie2_psp_start(struct psp_device *psp)
	90	{
	91	u32 reg_vals[PSP_NUM_IN_REGS];
	92	int ret;
	93
	94	reg_vals[0] = PSP_VALIDATE;
	95	reg_vals[1] = lower_32_bits(psp->fw_paddr);
	96	reg_vals[2] = upper_32_bits(psp->fw_paddr);
	97	reg_vals[3] = psp->fw_buf_sz;
	98
	99	ret = psp_exec(psp, reg_vals);
	100	if (ret) {
	101	drm_err(psp->ddev, "failed to validate fw, ret %d", ret);
	102	return ret;
	103	}
	104
	105	memset(reg_vals, 0, sizeof(reg_vals));
	106	reg_vals[0] = PSP_START;
	107	reg_vals[1] = PSP_START_COPY_FW;
	108	ret = psp_exec(psp, reg_vals);
	109	if (ret) {
	110	drm_err(psp->ddev, "failed to start fw, ret %d", ret);
	111	return ret;
	112	}
	113
	114	return 0;
	115	}
	116
	117	struct psp_device aie2m_psp_create(struct drm_device ddev, struct psp_config *conf)
	118	{
	119	struct psp_device *psp;
	120	u64 offset;
	121
	122	psp = drmm_kzalloc(ddev, sizeof(*psp), GFP_KERNEL);
	123	if (!psp)
	124	return NULL;
	125
	126	psp->ddev = ddev;
	127	memcpy(psp->psp_regs, conf->psp_regs, sizeof(psp->psp_regs));
	128
	129	psp->fw_buf_sz = ALIGN(conf->fw_size, PSP_FW_ALIGN) + PSP_FW_ALIGN;
	130	psp->fw_buffer = drmm_kmalloc(ddev, psp->fw_buf_sz, GFP_KERNEL);
	131	if (!psp->fw_buffer) {
	132	drm_err(ddev, "no memory for fw buffer");
	133	return NULL;
	134	}
	135
	136	/*
	137	* AMD Platform Security Processor(PSP) requires host physical
	138	* address to load NPU firmware.
	139	*/
	140	psp->fw_paddr = virt_to_phys(psp->fw_buffer);
	141	offset = ALIGN(psp->fw_paddr, PSP_FW_ALIGN) - psp->fw_paddr;
	142	psp->fw_paddr += offset;
	143	memcpy(psp->fw_buffer + offset, conf->fw_buf, conf->fw_size);
	144
	145	return psp;
	146	}

drivers/accel/amdxdna/aie2_smu.c+119−0

@@ -0,0 +1,119 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/drm_device.h>
	7	#include <drm/drm_gem_shmem_helper.h>
	8	#include <drm/drm_print.h>
	9	#include <drm/gpu_scheduler.h>
	10	#include <linux/iopoll.h>
	11
	12	#include "aie2_pci.h"
	13	#include "amdxdna_pci_drv.h"
	14
	15	#define SMU_RESULT_OK 1
	16
	17	/* SMU commands */
	18	#define AIE2_SMU_POWER_ON 0x3
	19	#define AIE2_SMU_POWER_OFF 0x4
	20	#define AIE2_SMU_SET_MPNPUCLK_FREQ 0x5
	21	#define AIE2_SMU_SET_HCLK_FREQ 0x6
	22
	23	static int aie2_smu_exec(struct amdxdna_dev_hdl *ndev, u32 reg_cmd, u32 reg_arg)
	24	{
	25	u32 resp;
	26	int ret;
	27
	28	writel(0, SMU_REG(ndev, SMU_RESP_REG));
	29	writel(reg_arg, SMU_REG(ndev, SMU_ARG_REG));
	30	writel(reg_cmd, SMU_REG(ndev, SMU_CMD_REG));
	31
	32	/* Clear and set SMU_INTR_REG to kick off */
	33	writel(0, SMU_REG(ndev, SMU_INTR_REG));
	34	writel(1, SMU_REG(ndev, SMU_INTR_REG));
	35
	36	ret = readx_poll_timeout(readl, SMU_REG(ndev, SMU_RESP_REG), resp,
	37	resp, AIE2_INTERVAL, AIE2_TIMEOUT);
	38	if (ret) {
	39	XDNA_ERR(ndev->xdna, "smu cmd %d timed out", reg_cmd);
	40	return ret;
	41	}
	42
	43	if (resp != SMU_RESULT_OK) {
	44	XDNA_ERR(ndev->xdna, "smu cmd %d failed, 0x%x", reg_cmd, resp);
	45	return -EINVAL;
	46	}
	47
	48	return 0;
	49	}
	50
	51	static int aie2_smu_set_mpnpu_clock_freq(struct amdxdna_dev_hdl *ndev, u32 freq_mhz)
	52	{
	53	int ret;
	54
	55	if (!freq_mhz \|\| freq_mhz > SMU_MPNPUCLK_FREQ_MAX(ndev)) {
	56	XDNA_ERR(ndev->xdna, "invalid mpnpu clock freq %d", freq_mhz);
	57	return -EINVAL;
	58	}
	59
	60	ndev->mp_npu_clock.freq_mhz = freq_mhz;
	61	ret = aie2_smu_exec(ndev, AIE2_SMU_SET_MPNPUCLK_FREQ, freq_mhz);
	62	if (!ret)
	63	XDNA_INFO_ONCE(ndev->xdna, "set mpnpu_clock = %d mhz", freq_mhz);
	64
	65	return ret;
	66	}
	67
	68	static int aie2_smu_set_hclock_freq(struct amdxdna_dev_hdl *ndev, u32 freq_mhz)
	69	{
	70	int ret;
	71
	72	if (!freq_mhz \|\| freq_mhz > SMU_HCLK_FREQ_MAX(ndev)) {
	73	XDNA_ERR(ndev->xdna, "invalid hclock freq %d", freq_mhz);
	74	return -EINVAL;
	75	}
	76
	77	ndev->h_clock.freq_mhz = freq_mhz;
	78	ret = aie2_smu_exec(ndev, AIE2_SMU_SET_HCLK_FREQ, freq_mhz);
	79	if (!ret)
	80	XDNA_INFO_ONCE(ndev->xdna, "set npu_hclock = %d mhz", freq_mhz);
	81
	82	return ret;
	83	}
	84
	85	int aie2_smu_init(struct amdxdna_dev_hdl *ndev)
	86	{
	87	int ret;
	88
	89	ret = aie2_smu_exec(ndev, AIE2_SMU_POWER_ON, 0);
	90	if (ret) {
	91	XDNA_ERR(ndev->xdna, "Power on failed, ret %d", ret);
	92	return ret;
	93	}
	94
	95	ret = aie2_smu_set_mpnpu_clock_freq(ndev, SMU_MPNPUCLK_FREQ_MAX(ndev));
	96	if (ret) {
	97	XDNA_ERR(ndev->xdna, "Set mpnpu clk freq failed, ret %d", ret);
	98	return ret;
	99	}
	100	snprintf(ndev->mp_npu_clock.name, sizeof(ndev->mp_npu_clock.name), "MP-NPU Clock");
	101
	102	ret = aie2_smu_set_hclock_freq(ndev, SMU_HCLK_FREQ_MAX(ndev));
	103	if (ret) {
	104	XDNA_ERR(ndev->xdna, "Set hclk freq failed, ret %d", ret);
	105	return ret;
	106	}
	107	snprintf(ndev->h_clock.name, sizeof(ndev->h_clock.name), "H Clock");
	108
	109	return 0;
	110	}
	111
	112	void aie2_smu_fini(struct amdxdna_dev_hdl *ndev)
	113	{
	114	int ret;
	115
	116	ret = aie2_smu_exec(ndev, AIE2_SMU_POWER_OFF, 0);
	117	if (ret)
	118	XDNA_ERR(ndev->xdna, "Power off failed, ret %d", ret);
	119	}

drivers/accel/amdxdna/aie2_solver.c+330−0

@@ -0,0 +1,330 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/drm_device.h>
	7	#include <drm/drm_managed.h>
	8	#include <drm/drm_print.h>
	9	#include <linux/bitops.h>
	10	#include <linux/bitmap.h>
	11
	12	#include "aie2_solver.h"
	13
	14	struct partition_node {
	15	struct list_head list;
	16	u32 nshared; /* # shared requests */
	17	u32 start_col; /* start column */
	18	u32 ncols; /* # columns */
	19	bool exclusive; /* can not be shared if set */
	20	};
	21
	22	struct solver_node {
	23	struct list_head list;
	24	u64 rid; /* Request ID from consumer */
	25
	26	struct partition_node *pt_node;
	27	void *cb_arg;
	28	u32 cols_len;
	29	u32 start_cols[] __counted_by(cols_len);
	30	};
	31
	32	struct solver_rgroup {
	33	u32 rgid;
	34	u32 nnode;
	35	u32 npartition_node;
	36
	37	DECLARE_BITMAP(resbit, XRS_MAX_COL);
	38	struct list_head node_list;
	39	struct list_head pt_node_list;
	40	};
	41
	42	struct solver_state {
	43	struct solver_rgroup rgp;
	44	struct init_config cfg;
	45	struct xrs_action_ops *actions;
	46	};
	47
	48	static u32 calculate_gops(struct aie_qos *rqos)
	49	{
	50	u32 service_rate = 0;
	51
	52	if (rqos->latency)
	53	service_rate = (1000 / rqos->latency);
	54
	55	if (rqos->fps > service_rate)
	56	return rqos->fps * rqos->gops;
	57
	58	return service_rate * rqos->gops;
	59	}
	60
	61	/*
	62	* qos_meet() - Check the QOS request can be met.
	63	*/
	64	static int qos_meet(struct solver_state xrs, struct aie_qos rqos, u32 cgops)
	65	{
	66	u32 request_gops = calculate_gops(rqos) * xrs->cfg.sys_eff_factor;
	67
	68	if (request_gops <= cgops)
	69	return 0;
	70
	71	return -EINVAL;
	72	}
	73
	74	/*
	75	* sanity_check() - Do a basic sanity check on allocation request.
	76	*/
	77	static int sanity_check(struct solver_state xrs, struct alloc_requests req)
	78	{
	79	struct cdo_parts *cdop = &req->cdo;
	80	struct aie_qos *rqos = &req->rqos;
	81	u32 cu_clk_freq;
	82
	83	if (cdop->ncols > xrs->cfg.total_col)
	84	return -EINVAL;
	85
	86	/*
	87	* We can find at least one CDOs groups that meet the
	88	* GOPs requirement.
	89	*/
	90	cu_clk_freq = xrs->cfg.clk_list.cu_clk_list[xrs->cfg.clk_list.num_levels - 1];
	91
	92	if (qos_meet(xrs, rqos, cdop->qos_cap.opc * cu_clk_freq / 1000))
	93	return -EINVAL;
	94
	95	return 0;
	96	}
	97
	98	static struct solver_node rg_search_node(struct solver_rgroup rgp, u64 rid)
	99	{
	100	struct solver_node *node;
	101
	102	list_for_each_entry(node, &rgp->node_list, list) {
	103	if (node->rid == rid)
	104	return node;
	105	}
	106
	107	return NULL;
	108	}
	109
	110	static void remove_partition_node(struct solver_rgroup *rgp,
	111	struct partition_node *pt_node)
	112	{
	113	pt_node->nshared--;
	114	if (pt_node->nshared > 0)
	115	return;
	116
	117	list_del(&pt_node->list);
	118	rgp->npartition_node--;
	119
	120	bitmap_clear(rgp->resbit, pt_node->start_col, pt_node->ncols);
	121	kfree(pt_node);
	122	}
	123
	124	static void remove_solver_node(struct solver_rgroup *rgp,
	125	struct solver_node *node)
	126	{
	127	list_del(&node->list);
	128	rgp->nnode--;
	129
	130	if (node->pt_node)
	131	remove_partition_node(rgp, node->pt_node);
	132
	133	kfree(node);
	134	}
	135
	136	static int get_free_partition(struct solver_state *xrs,
	137	struct solver_node *snode,
	138	struct alloc_requests *req)
	139	{
	140	struct partition_node *pt_node;
	141	u32 ncols = req->cdo.ncols;
	142	u32 col, i;
	143
	144	for (i = 0; i < snode->cols_len; i++) {
	145	col = snode->start_cols[i];
	146	if (find_next_bit(xrs->rgp.resbit, XRS_MAX_COL, col) >= col + ncols)
	147	break;
	148	}
	149
	150	if (i == snode->cols_len)
	151	return -ENODEV;
	152
	153	pt_node = kzalloc(sizeof(*pt_node), GFP_KERNEL);
	154	if (!pt_node)
	155	return -ENOMEM;
	156
	157	pt_node->nshared = 1;
	158	pt_node->start_col = col;
	159	pt_node->ncols = ncols;
	160
	161	/*
	162	* Before fully support latency in QoS, if a request
	163	* specifies a non-zero latency value, it will not share
	164	* the partition with other requests.
	165	*/
	166	if (req->rqos.latency)
	167	pt_node->exclusive = true;
	168
	169	list_add_tail(&pt_node->list, &xrs->rgp.pt_node_list);
	170	xrs->rgp.npartition_node++;
	171	bitmap_set(xrs->rgp.resbit, pt_node->start_col, pt_node->ncols);
	172
	173	snode->pt_node = pt_node;
	174
	175	return 0;
	176	}
	177
	178	static int allocate_partition(struct solver_state *xrs,
	179	struct solver_node *snode,
	180	struct alloc_requests *req)
	181	{
	182	struct partition_node pt_node, rpt_node = NULL;
	183	int idx, ret;
	184
	185	ret = get_free_partition(xrs, snode, req);
	186	if (!ret)
	187	return ret;
	188
	189	/* try to get a share-able partition */
	190	list_for_each_entry(pt_node, &xrs->rgp.pt_node_list, list) {
	191	if (pt_node->exclusive)
	192	continue;
	193
	194	if (rpt_node && pt_node->nshared >= rpt_node->nshared)
	195	continue;
	196
	197	for (idx = 0; idx < snode->cols_len; idx++) {
	198	if (snode->start_cols[idx] != pt_node->start_col)
	199	continue;
	200
	201	if (req->cdo.ncols != pt_node->ncols)
	202	continue;
	203
	204	rpt_node = pt_node;
	205	break;
	206	}
	207	}
	208
	209	if (!rpt_node)
	210	return -ENODEV;
	211
	212	rpt_node->nshared++;
	213	snode->pt_node = rpt_node;
	214
	215	return 0;
	216	}
	217
	218	static struct solver_node create_solver_node(struct solver_state xrs,
	219	struct alloc_requests *req)
	220	{
	221	struct cdo_parts *cdop = &req->cdo;
	222	struct solver_node *node;
	223	int ret;
	224
	225	node = kzalloc(struct_size(node, start_cols, cdop->cols_len), GFP_KERNEL);
	226	if (!node)
	227	return ERR_PTR(-ENOMEM);
	228
	229	node->rid = req->rid;
	230	node->cols_len = cdop->cols_len;
	231	memcpy(node->start_cols, cdop->start_cols, cdop->cols_len * sizeof(u32));
	232
	233	ret = allocate_partition(xrs, node, req);
	234	if (ret)
	235	goto free_node;
	236
	237	list_add_tail(&node->list, &xrs->rgp.node_list);
	238	xrs->rgp.nnode++;
	239	return node;
	240
	241	free_node:
	242	kfree(node);
	243	return ERR_PTR(ret);
	244	}
	245
	246	static void fill_load_action(struct solver_state *xrs,
	247	struct solver_node *snode,
	248	struct xrs_action_load *action)
	249	{
	250	action->rid = snode->rid;
	251	action->part.start_col = snode->pt_node->start_col;
	252	action->part.ncols = snode->pt_node->ncols;
	253	}
	254
	255	int xrs_allocate_resource(void hdl, struct alloc_requests req, void *cb_arg)
	256	{
	257	struct xrs_action_load load_act;
	258	struct solver_node *snode;
	259	struct solver_state *xrs;
	260	int ret;
	261
	262	xrs = (struct solver_state *)hdl;
	263
	264	ret = sanity_check(xrs, req);
	265	if (ret) {
	266	drm_err(xrs->cfg.ddev, "invalid request");
	267	return ret;
	268	}
	269
	270	if (rg_search_node(&xrs->rgp, req->rid)) {
	271	drm_err(xrs->cfg.ddev, "rid %lld is in-use", req->rid);
	272	return -EEXIST;
	273	}
	274
	275	snode = create_solver_node(xrs, req);
	276	if (IS_ERR(snode))
	277	return PTR_ERR(snode);
	278
	279	fill_load_action(xrs, snode, &load_act);
	280	ret = xrs->cfg.actions->load(cb_arg, &load_act);
	281	if (ret)
	282	goto free_node;
	283
	284	snode->cb_arg = cb_arg;
	285
	286	drm_dbg(xrs->cfg.ddev, "start col %d ncols %d\n",
	287	snode->pt_node->start_col, snode->pt_node->ncols);
	288
	289	return 0;
	290
	291	free_node:
	292	remove_solver_node(&xrs->rgp, snode);
	293
	294	return ret;
	295	}
	296
	297	int xrs_release_resource(void *hdl, u64 rid)
	298	{
	299	struct solver_state *xrs = hdl;
	300	struct solver_node *node;
	301
	302	node = rg_search_node(&xrs->rgp, rid);
	303	if (!node) {
	304	drm_err(xrs->cfg.ddev, "node not exist");
	305	return -ENODEV;
	306	}
	307
	308	xrs->cfg.actions->unload(node->cb_arg);
	309	remove_solver_node(&xrs->rgp, node);
	310
	311	return 0;
	312	}
	313
	314	void xrsm_init(struct init_config cfg)
	315	{
	316	struct solver_rgroup *rgp;
	317	struct solver_state *xrs;
	318
	319	xrs = drmm_kzalloc(cfg->ddev, sizeof(*xrs), GFP_KERNEL);
	320	if (!xrs)
	321	return NULL;
	322
	323	memcpy(&xrs->cfg, cfg, sizeof(*cfg));
	324
	325	rgp = &xrs->rgp;
	326	INIT_LIST_HEAD(&rgp->node_list);
	327	INIT_LIST_HEAD(&rgp->pt_node_list);
	328
	329	return xrs;
	330	}

drivers/accel/amdxdna/aie2_solver.h+154−0

@@ -0,0 +1,154 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _AIE2_SOLVER_H
	7	#define _AIE2_SOLVER_H
	8
	9	#define XRS_MAX_COL 128
	10
	11	/*
	12	* Structure used to describe a partition. A partition is column based
	13	* allocation unit described by its start column and number of columns.
	14	*/
	15	struct aie_part {
	16	u32 start_col;
	17	u32 ncols;
	18	};
	19
	20	/*
	21	* The QoS capabilities of a given AIE partition.
	22	*/
	23	struct aie_qos_cap {
	24	u32 opc; /* operations per cycle */
	25	u32 dma_bw; /* DMA bandwidth */
	26	};
	27
	28	/*
	29	* QoS requirement of a resource allocation.
	30	*/
	31	struct aie_qos {
	32	u32 gops; /* Giga operations */
	33	u32 fps; /* Frames per second */
	34	u32 dma_bw; /* DMA bandwidth */
	35	u32 latency; /* Frame response latency */
	36	u32 exec_time; /* Frame execution time */
	37	u32 priority; /* Request priority */
	38	};
	39
	40	/*
	41	* Structure used to describe a relocatable CDO (Configuration Data Object).
	42	*/
	43	struct cdo_parts {
	44	u32 start_cols; / Start column array */
	45	u32 cols_len; /* Length of start column array */
	46	u32 ncols; /* # of column */
	47	struct aie_qos_cap qos_cap; /* CDO QoS capabilities */
	48	};
	49
	50	/*
	51	* Structure used to describe a request to allocate.
	52	*/
	53	struct alloc_requests {
	54	u64 rid;
	55	struct cdo_parts cdo;
	56	struct aie_qos rqos; /* Requested QoS */
	57	};
	58
	59	/*
	60	* Load callback argument
	61	*/
	62	struct xrs_action_load {
	63	u32 rid;
	64	struct aie_part part;
	65	};
	66
	67	/*
	68	* Define the power level available
	69	*
	70	* POWER_LEVEL_MIN:
	71	* Lowest power level. Usually set when all actions are unloaded.
	72	*
	73	* POWER_LEVEL_n
	74	* Power levels 0 - n, is a step increase in system frequencies
	75	*/
	76	enum power_level {
	77	POWER_LEVEL_MIN = 0x0,
	78	POWER_LEVEL_0 = 0x1,
	79	POWER_LEVEL_1 = 0x2,
	80	POWER_LEVEL_2 = 0x3,
	81	POWER_LEVEL_3 = 0x4,
	82	POWER_LEVEL_4 = 0x5,
	83	POWER_LEVEL_5 = 0x6,
	84	POWER_LEVEL_6 = 0x7,
	85	POWER_LEVEL_7 = 0x8,
	86	POWER_LEVEL_NUM,
	87	};
	88
	89	/*
	90	* Structure used to describe the frequency table.
	91	* Resource solver chooses the frequency from the table
	92	* to meet the QOS requirements.
	93	*/
	94	struct clk_list_info {
	95	u32 num_levels; /* available power levels */
	96	u32 cu_clk_list[POWER_LEVEL_NUM]; /* available aie clock frequencies in Mhz*/
	97	};
	98
	99	struct xrs_action_ops {
	100	int (load)(void cb_arg, struct xrs_action_load *action);
	101	int (unload)(void cb_arg);
	102	};
	103
	104	/*
	105	* Structure used to describe information for solver during initialization.
	106	*/
	107	struct init_config {
	108	u32 total_col;
	109	u32 sys_eff_factor; /* system efficiency factor */
	110	u32 latency_adj; /* latency adjustment in ms */
	111	struct clk_list_info clk_list; /* List of frequencies available in system */
	112	struct drm_device *ddev;
	113	struct xrs_action_ops *actions;
	114	};
	115
	116	/*
	117	* xrsm_init() - Register resource solver. Resource solver client needs
	118	* to call this function to register itself.
	119	*
	120	* @cfg: The system metrics for resource solver to use
	121	*
	122	* Return: A resource solver handle
	123	*
	124	* Note: We should only create one handle per AIE array to be managed.
	125	*/
	126	void xrsm_init(struct init_config cfg);
	127
	128	/*
	129	* xrs_allocate_resource() - Request to allocate resources for a given context
	130	* and a partition metadata. (See struct part_meta)
	131	*
	132	* @hdl: Resource solver handle obtained from xrs_init()
	133	* @req: Input to the Resource solver including request id
	134	* and partition metadata.
	135	* @cb_arg: callback argument pointer
	136	*
	137	* Return: 0 when successful.
	138	* Or standard error number when failing
	139	*
	140	* Note:
	141	* There is no lock mechanism inside resource solver. So it is
	142	* the caller's responsibility to lock down XCLBINs and grab
	143	* necessary lock.
	144	*/
	145	int xrs_allocate_resource(void hdl, struct alloc_requests req, void *cb_arg);
	146
	147	/*
	148	* xrs_release_resource() - Request to free resources for a given context.
	149	*
	150	* @hdl: Resource solver handle obtained from xrs_init()
	151	* @rid: The Request ID to identify the requesting context
	152	*/
	153	int xrs_release_resource(void *hdl, u64 rid);
	154	#endif /* _AIE2_SOLVER_H */

drivers/accel/amdxdna/amdxdna_ctx.c+553−0

@@ -0,0 +1,553 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/drm_drv.h>
	9	#include <drm/drm_file.h>
	10	#include <drm/drm_gem.h>
	11	#include <drm/drm_gem_shmem_helper.h>
	12	#include <drm/drm_print.h>
	13	#include <drm/gpu_scheduler.h>
	14	#include <trace/events/amdxdna.h>
	15
	16	#include "amdxdna_ctx.h"
	17	#include "amdxdna_gem.h"
	18	#include "amdxdna_pci_drv.h"
	19
	20	#define MAX_HWCTX_ID 255
	21	#define MAX_ARG_COUNT 4095
	22
	23	struct amdxdna_fence {
	24	struct dma_fence base;
	25	spinlock_t lock; /* for base */
	26	struct amdxdna_hwctx *hwctx;
	27	};
	28
	29	static const char amdxdna_fence_get_driver_name(struct dma_fence fence)
	30	{
	31	return KBUILD_MODNAME;
	32	}
	33
	34	static const char amdxdna_fence_get_timeline_name(struct dma_fence fence)
	35	{
	36	struct amdxdna_fence *xdna_fence;
	37
	38	xdna_fence = container_of(fence, struct amdxdna_fence, base);
	39
	40	return xdna_fence->hwctx->name;
	41	}
	42
	43	static const struct dma_fence_ops fence_ops = {
	44	.get_driver_name = amdxdna_fence_get_driver_name,
	45	.get_timeline_name = amdxdna_fence_get_timeline_name,
	46	};
	47
	48	static struct dma_fence amdxdna_fence_create(struct amdxdna_hwctx hwctx)
	49	{
	50	struct amdxdna_fence *fence;
	51
	52	fence = kzalloc(sizeof(*fence), GFP_KERNEL);
	53	if (!fence)
	54	return NULL;
	55
	56	fence->hwctx = hwctx;
	57	spin_lock_init(&fence->lock);
	58	dma_fence_init(&fence->base, &fence_ops, &fence->lock, hwctx->id, 0);
	59	return &fence->base;
	60	}
	61
	62	void amdxdna_hwctx_suspend(struct amdxdna_client *client)
	63	{
	64	struct amdxdna_dev *xdna = client->xdna;
	65	struct amdxdna_hwctx *hwctx;
	66	int next = 0;
	67
	68	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	69	mutex_lock(&client->hwctx_lock);
	70	idr_for_each_entry_continue(&client->hwctx_idr, hwctx, next)
	71	xdna->dev_info->ops->hwctx_suspend(hwctx);
	72	mutex_unlock(&client->hwctx_lock);
	73	}
	74
	75	void amdxdna_hwctx_resume(struct amdxdna_client *client)
	76	{
	77	struct amdxdna_dev *xdna = client->xdna;
	78	struct amdxdna_hwctx *hwctx;
	79	int next = 0;
	80
	81	drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
	82	mutex_lock(&client->hwctx_lock);
	83	idr_for_each_entry_continue(&client->hwctx_idr, hwctx, next)
	84	xdna->dev_info->ops->hwctx_resume(hwctx);
	85	mutex_unlock(&client->hwctx_lock);
	86	}
	87
	88	static void amdxdna_hwctx_destroy_rcu(struct amdxdna_hwctx *hwctx,
	89	struct srcu_struct *ss)
	90	{
	91	struct amdxdna_dev *xdna = hwctx->client->xdna;
	92
	93	synchronize_srcu(ss);
	94
	95	/* At this point, user is not able to submit new commands */
	96	mutex_lock(&xdna->dev_lock);
	97	xdna->dev_info->ops->hwctx_fini(hwctx);
	98	mutex_unlock(&xdna->dev_lock);
	99
	100	kfree(hwctx->name);
	101	kfree(hwctx);
	102	}
	103
	104	void amdxdna_cmd_get_payload(struct amdxdna_gem_obj abo, u32 *size)
	105	{
	106	struct amdxdna_cmd *cmd = abo->mem.kva;
	107	u32 num_masks, count;
	108
	109	if (amdxdna_cmd_get_op(abo) == ERT_CMD_CHAIN)
	110	num_masks = 0;
	111	else
	112	num_masks = 1 + FIELD_GET(AMDXDNA_CMD_EXTRA_CU_MASK, cmd->header);
	113
	114	if (size) {
	115	count = FIELD_GET(AMDXDNA_CMD_COUNT, cmd->header);
	116	if (unlikely(count <= num_masks)) {
	117	*size = 0;
	118	return NULL;
	119	}
	120	size = (count - num_masks) sizeof(u32);
	121	}
	122	return &cmd->data[num_masks];
	123	}
	124
	125	int amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo)
	126	{
	127	struct amdxdna_cmd *cmd = abo->mem.kva;
	128	u32 num_masks, i;
	129	u32 *cu_mask;
	130
	131	if (amdxdna_cmd_get_op(abo) == ERT_CMD_CHAIN)
	132	return -1;
	133
	134	num_masks = 1 + FIELD_GET(AMDXDNA_CMD_EXTRA_CU_MASK, cmd->header);
	135	cu_mask = cmd->data;
	136	for (i = 0; i < num_masks; i++) {
	137	if (cu_mask[i])
	138	return ffs(cu_mask[i]) - 1;
	139	}
	140
	141	return -1;
	142	}
	143
	144	/*
	145	* This should be called in close() and remove(). DO NOT call in other syscalls.
	146	* This guarantee that when hwctx and resources will be released, if user
	147	* doesn't call amdxdna_drm_destroy_hwctx_ioctl.
	148	*/
	149	void amdxdna_hwctx_remove_all(struct amdxdna_client *client)
	150	{
	151	struct amdxdna_hwctx *hwctx;
	152	int next = 0;
	153
	154	mutex_lock(&client->hwctx_lock);
	155	idr_for_each_entry_continue(&client->hwctx_idr, hwctx, next) {
	156	XDNA_DBG(client->xdna, "PID %d close HW context %d",
	157	client->pid, hwctx->id);
	158	idr_remove(&client->hwctx_idr, hwctx->id);
	159	mutex_unlock(&client->hwctx_lock);
	160	amdxdna_hwctx_destroy_rcu(hwctx, &client->hwctx_srcu);
	161	mutex_lock(&client->hwctx_lock);
	162	}
	163	mutex_unlock(&client->hwctx_lock);
	164	}
	165
	166	int amdxdna_drm_create_hwctx_ioctl(struct drm_device dev, void data, struct drm_file *filp)
	167	{
	168	struct amdxdna_client *client = filp->driver_priv;
	169	struct amdxdna_drm_create_hwctx *args = data;
	170	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	171	struct amdxdna_hwctx *hwctx;
	172	int ret, idx;
	173
	174	if (args->ext \|\| args->ext_flags)
	175	return -EINVAL;
	176
	177	if (!drm_dev_enter(dev, &idx))
	178	return -ENODEV;
	179
	180	hwctx = kzalloc(sizeof(*hwctx), GFP_KERNEL);
	181	if (!hwctx) {
	182	ret = -ENOMEM;
	183	goto exit;
	184	}
	185
	186	if (copy_from_user(&hwctx->qos, u64_to_user_ptr(args->qos_p), sizeof(hwctx->qos))) {
	187	XDNA_ERR(xdna, "Access QoS info failed");
	188	ret = -EFAULT;
	189	goto free_hwctx;
	190	}
	191
	192	hwctx->client = client;
	193	hwctx->fw_ctx_id = -1;
	194	hwctx->num_tiles = args->num_tiles;
	195	hwctx->mem_size = args->mem_size;
	196	hwctx->max_opc = args->max_opc;
	197	mutex_lock(&client->hwctx_lock);
	198	ret = idr_alloc_cyclic(&client->hwctx_idr, hwctx, 0, MAX_HWCTX_ID, GFP_KERNEL);
	199	if (ret < 0) {
	200	mutex_unlock(&client->hwctx_lock);
	201	XDNA_ERR(xdna, "Allocate hwctx ID failed, ret %d", ret);
	202	goto free_hwctx;
	203	}
	204	hwctx->id = ret;
	205	mutex_unlock(&client->hwctx_lock);
	206
	207	hwctx->name = kasprintf(GFP_KERNEL, "hwctx.%d.%d", client->pid, hwctx->id);
	208	if (!hwctx->name) {
	209	ret = -ENOMEM;
	210	goto rm_id;
	211	}
	212
	213	mutex_lock(&xdna->dev_lock);
	214	ret = xdna->dev_info->ops->hwctx_init(hwctx);
	215	if (ret) {
	216	mutex_unlock(&xdna->dev_lock);
	217	XDNA_ERR(xdna, "Init hwctx failed, ret %d", ret);
	218	goto free_name;
	219	}
	220	args->handle = hwctx->id;
	221	args->syncobj_handle = hwctx->syncobj_hdl;
	222	mutex_unlock(&xdna->dev_lock);
	223
	224	XDNA_DBG(xdna, "PID %d create HW context %d, ret %d", client->pid, args->handle, ret);
	225	drm_dev_exit(idx);
	226	return 0;
	227
	228	free_name:
	229	kfree(hwctx->name);
	230	rm_id:
	231	mutex_lock(&client->hwctx_lock);
	232	idr_remove(&client->hwctx_idr, hwctx->id);
	233	mutex_unlock(&client->hwctx_lock);
	234	free_hwctx:
	235	kfree(hwctx);
	236	exit:
	237	drm_dev_exit(idx);
	238	return ret;
	239	}
	240
	241	int amdxdna_drm_destroy_hwctx_ioctl(struct drm_device dev, void data, struct drm_file *filp)
	242	{
	243	struct amdxdna_client *client = filp->driver_priv;
	244	struct amdxdna_drm_destroy_hwctx *args = data;
	245	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	246	struct amdxdna_hwctx *hwctx;
	247	int ret = 0, idx;
	248
	249	if (!drm_dev_enter(dev, &idx))
	250	return -ENODEV;
	251
	252	/*
	253	* Use hwctx_lock to achieve exclusion with other hwctx writers,
	254	* SRCU to synchronize with exec/wait command ioctls.
	255	*
	256	* The pushed jobs are handled by DRM scheduler during destroy.
	257	*/
	258	mutex_lock(&client->hwctx_lock);
	259	hwctx = idr_find(&client->hwctx_idr, args->handle);
	260	if (!hwctx) {
	261	mutex_unlock(&client->hwctx_lock);
	262	ret = -EINVAL;
	263	XDNA_DBG(xdna, "PID %d HW context %d not exist",
	264	client->pid, args->handle);
	265	goto out;
	266	}
	267	idr_remove(&client->hwctx_idr, hwctx->id);
	268	mutex_unlock(&client->hwctx_lock);
	269
	270	amdxdna_hwctx_destroy_rcu(hwctx, &client->hwctx_srcu);
	271
	272	XDNA_DBG(xdna, "PID %d destroyed HW context %d", client->pid, args->handle);
	273	out:
	274	drm_dev_exit(idx);
	275	return ret;
	276	}
	277
	278	int amdxdna_drm_config_hwctx_ioctl(struct drm_device dev, void data, struct drm_file *filp)
	279	{
	280	struct amdxdna_client *client = filp->driver_priv;
	281	struct amdxdna_drm_config_hwctx *args = data;
	282	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	283	struct amdxdna_hwctx *hwctx;
	284	int ret, idx;
	285	u32 buf_size;
	286	void *buf;
	287	u64 val;
	288
	289	if (!xdna->dev_info->ops->hwctx_config)
	290	return -EOPNOTSUPP;
	291
	292	val = args->param_val;
	293	buf_size = args->param_val_size;
	294
	295	switch (args->param_type) {
	296	case DRM_AMDXDNA_HWCTX_CONFIG_CU:
	297	/* For those types that param_val is pointer */
	298	if (buf_size > PAGE_SIZE) {
	299	XDNA_ERR(xdna, "Config CU param buffer too large");
	300	return -E2BIG;
	301	}
	302
	303	/* Hwctx needs to keep buf */
	304	buf = kzalloc(PAGE_SIZE, GFP_KERNEL);
	305	if (!buf)
	306	return -ENOMEM;
	307
	308	if (copy_from_user(buf, u64_to_user_ptr(val), buf_size)) {
	309	kfree(buf);
	310	return -EFAULT;
	311	}
	312
	313	break;
	314	case DRM_AMDXDNA_HWCTX_ASSIGN_DBG_BUF:
	315	case DRM_AMDXDNA_HWCTX_REMOVE_DBG_BUF:
	316	/* For those types that param_val is a value */
	317	buf = NULL;
	318	buf_size = 0;
	319	break;
	320	default:
	321	XDNA_DBG(xdna, "Unknown HW context config type %d", args->param_type);
	322	return -EINVAL;
	323	}
	324
	325	mutex_lock(&xdna->dev_lock);
	326	idx = srcu_read_lock(&client->hwctx_srcu);
	327	hwctx = idr_find(&client->hwctx_idr, args->handle);
	328	if (!hwctx) {
	329	XDNA_DBG(xdna, "PID %d failed to get hwctx %d", client->pid, args->handle);
	330	ret = -EINVAL;
	331	goto unlock_srcu;
	332	}
	333
	334	ret = xdna->dev_info->ops->hwctx_config(hwctx, args->param_type, val, buf, buf_size);
	335
	336	unlock_srcu:
	337	srcu_read_unlock(&client->hwctx_srcu, idx);
	338	mutex_unlock(&xdna->dev_lock);
	339	kfree(buf);
	340	return ret;
	341	}
	342
	343	static void
	344	amdxdna_arg_bos_put(struct amdxdna_sched_job *job)
	345	{
	346	int i;
	347
	348	for (i = 0; i < job->bo_cnt; i++) {
	349	if (!job->bos[i])
	350	break;
	351	drm_gem_object_put(job->bos[i]);
	352	}
	353	}
	354
	355	static int
	356	amdxdna_arg_bos_lookup(struct amdxdna_client *client,
	357	struct amdxdna_sched_job *job,
	358	u32 *bo_hdls, u32 bo_cnt)
	359	{
	360	struct drm_gem_object *gobj;
	361	int i, ret;
	362
	363	job->bo_cnt = bo_cnt;
	364	for (i = 0; i < job->bo_cnt; i++) {
	365	struct amdxdna_gem_obj *abo;
	366
	367	gobj = drm_gem_object_lookup(client->filp, bo_hdls[i]);
	368	if (!gobj) {
	369	ret = -ENOENT;
	370	goto put_shmem_bo;
	371	}
	372	abo = to_xdna_obj(gobj);
	373
	374	mutex_lock(&abo->lock);
	375	if (abo->pinned) {
	376	mutex_unlock(&abo->lock);
	377	job->bos[i] = gobj;
	378	continue;
	379	}
	380
	381	ret = amdxdna_gem_pin_nolock(abo);
	382	if (ret) {
	383	mutex_unlock(&abo->lock);
	384	drm_gem_object_put(gobj);
	385	goto put_shmem_bo;
	386	}
	387	abo->pinned = true;
	388	mutex_unlock(&abo->lock);
	389
	390	job->bos[i] = gobj;
	391	}
	392
	393	return 0;
	394
	395	put_shmem_bo:
	396	amdxdna_arg_bos_put(job);
	397	return ret;
	398	}
	399
	400	void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job)
	401	{
	402	trace_amdxdna_debug_point(job->hwctx->name, job->seq, "job release");
	403	amdxdna_arg_bos_put(job);
	404	amdxdna_gem_put_obj(job->cmd_bo);
	405	}
	406
	407	int amdxdna_cmd_submit(struct amdxdna_client *client,
	408	u32 cmd_bo_hdl, u32 *arg_bo_hdls, u32 arg_bo_cnt,
	409	u32 hwctx_hdl, u64 *seq)
	410	{
	411	struct amdxdna_dev *xdna = client->xdna;
	412	struct amdxdna_sched_job *job;
	413	struct amdxdna_hwctx *hwctx;
	414	int ret, idx;
	415
	416	XDNA_DBG(xdna, "Command BO hdl %d, Arg BO count %d", cmd_bo_hdl, arg_bo_cnt);
	417	job = kzalloc(struct_size(job, bos, arg_bo_cnt), GFP_KERNEL);
	418	if (!job)
	419	return -ENOMEM;
	420
	421	if (cmd_bo_hdl != AMDXDNA_INVALID_BO_HANDLE) {
	422	job->cmd_bo = amdxdna_gem_get_obj(client, cmd_bo_hdl, AMDXDNA_BO_CMD);
	423	if (!job->cmd_bo) {
	424	XDNA_ERR(xdna, "Failed to get cmd bo from %d", cmd_bo_hdl);
	425	ret = -EINVAL;
	426	goto free_job;
	427	}
	428	} else {
	429	job->cmd_bo = NULL;
	430	}
	431
	432	ret = amdxdna_arg_bos_lookup(client, job, arg_bo_hdls, arg_bo_cnt);
	433	if (ret) {
	434	XDNA_ERR(xdna, "Argument BOs lookup failed, ret %d", ret);
	435	goto cmd_put;
	436	}
	437
	438	idx = srcu_read_lock(&client->hwctx_srcu);
	439	hwctx = idr_find(&client->hwctx_idr, hwctx_hdl);
	440	if (!hwctx) {
	441	XDNA_DBG(xdna, "PID %d failed to get hwctx %d",
	442	client->pid, hwctx_hdl);
	443	ret = -EINVAL;
	444	goto unlock_srcu;
	445	}
	446
	447	if (hwctx->status != HWCTX_STAT_READY) {
	448	XDNA_ERR(xdna, "HW Context is not ready");
	449	ret = -EINVAL;
	450	goto unlock_srcu;
	451	}
	452
	453	job->hwctx = hwctx;
	454	job->mm = current->mm;
	455
	456	job->fence = amdxdna_fence_create(hwctx);
	457	if (!job->fence) {
	458	XDNA_ERR(xdna, "Failed to create fence");
	459	ret = -ENOMEM;
	460	goto unlock_srcu;
	461	}
	462	kref_init(&job->refcnt);
	463
	464	ret = xdna->dev_info->ops->cmd_submit(hwctx, job, seq);
	465	if (ret)
	466	goto put_fence;
	467
	468	/*
	469	* The amdxdna_hwctx_destroy_rcu() will release hwctx and associated
	470	* resource after synchronize_srcu(). The submitted jobs should be
	471	* handled by the queue, for example DRM scheduler, in device layer.
	472	* For here we can unlock SRCU.
	473	*/
	474	srcu_read_unlock(&client->hwctx_srcu, idx);
	475	trace_amdxdna_debug_point(hwctx->name, *seq, "job pushed");
	476
	477	return 0;
	478
	479	put_fence:
	480	dma_fence_put(job->fence);
	481	unlock_srcu:
	482	srcu_read_unlock(&client->hwctx_srcu, idx);
	483	amdxdna_arg_bos_put(job);
	484	cmd_put:
	485	amdxdna_gem_put_obj(job->cmd_bo);
	486	free_job:
	487	kfree(job);
	488	return ret;
	489	}
	490
	491	/*
	492	* The submit command ioctl submits a command to firmware. One firmware command
	493	* may contain multiple command BOs for processing as a whole.
	494	* The command sequence number is returned which can be used for wait command ioctl.
	495	*/
	496	static int amdxdna_drm_submit_execbuf(struct amdxdna_client *client,
	497	struct amdxdna_drm_exec_cmd *args)
	498	{
	499	struct amdxdna_dev *xdna = client->xdna;
	500	u32 *arg_bo_hdls;
	501	u32 cmd_bo_hdl;
	502	int ret;
	503
	504	if (!args->arg_count \|\| args->arg_count > MAX_ARG_COUNT) {
	505	XDNA_ERR(xdna, "Invalid arg bo count %d", args->arg_count);
	506	return -EINVAL;
	507	}
	508
	509	/* Only support single command for now. */
	510	if (args->cmd_count != 1) {
	511	XDNA_ERR(xdna, "Invalid cmd bo count %d", args->cmd_count);
	512	return -EINVAL;
	513	}
	514
	515	cmd_bo_hdl = (u32)args->cmd_handles;
	516	arg_bo_hdls = kcalloc(args->arg_count, sizeof(u32), GFP_KERNEL);
	517	if (!arg_bo_hdls)
	518	return -ENOMEM;
	519	ret = copy_from_user(arg_bo_hdls, u64_to_user_ptr(args->args),
	520	args->arg_count * sizeof(u32));
	521	if (ret) {
	522	ret = -EFAULT;
	523	goto free_cmd_bo_hdls;
	524	}
	525
	526	ret = amdxdna_cmd_submit(client, cmd_bo_hdl, arg_bo_hdls,
	527	args->arg_count, args->hwctx, &args->seq);
	528	if (ret)
	529	XDNA_DBG(xdna, "Submit cmds failed, ret %d", ret);
	530
	531	free_cmd_bo_hdls:
	532	kfree(arg_bo_hdls);
	533	if (!ret)
	534	XDNA_DBG(xdna, "Pushed cmd %lld to scheduler", args->seq);
	535	return ret;
	536	}
	537
	538	int amdxdna_drm_submit_cmd_ioctl(struct drm_device dev, void data, struct drm_file *filp)
	539	{
	540	struct amdxdna_client *client = filp->driver_priv;
	541	struct amdxdna_drm_exec_cmd *args = data;
	542
	543	if (args->ext \|\| args->ext_flags)
	544	return -EINVAL;
	545
	546	switch (args->type) {
	547	case AMDXDNA_CMD_SUBMIT_EXEC_BUF:
	548	return amdxdna_drm_submit_execbuf(client, args);
	549	}
	550
	551	XDNA_ERR(client->xdna, "Invalid command type %d", args->type);
	552	return -EINVAL;
	553	}

drivers/accel/amdxdna/amdxdna_ctx.h+162−0

@@ -0,0 +1,162 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _AMDXDNA_CTX_H_
	7	#define _AMDXDNA_CTX_H_
	8
	9	#include <linux/bitfield.h>
	10
	11	#include "amdxdna_gem.h"
	12
	13	struct amdxdna_hwctx_priv;
	14
	15	enum ert_cmd_opcode {
	16	ERT_START_CU = 0,
	17	ERT_CMD_CHAIN = 19,
	18	ERT_START_NPU = 20,
	19	};
	20
	21	enum ert_cmd_state {
	22	ERT_CMD_STATE_INVALID,
	23	ERT_CMD_STATE_NEW,
	24	ERT_CMD_STATE_QUEUED,
	25	ERT_CMD_STATE_RUNNING,
	26	ERT_CMD_STATE_COMPLETED,
	27	ERT_CMD_STATE_ERROR,
	28	ERT_CMD_STATE_ABORT,
	29	ERT_CMD_STATE_SUBMITTED,
	30	ERT_CMD_STATE_TIMEOUT,
	31	ERT_CMD_STATE_NORESPONSE,
	32	};
	33
	34	/*
	35	* Interpretation of the beginning of data payload for ERT_START_NPU in
	36	* amdxdna_cmd. The rest of the payload in amdxdna_cmd is regular kernel args.
	37	*/
	38	struct amdxdna_cmd_start_npu {
	39	u64 buffer; /* instruction buffer address */
	40	u32 buffer_size; /* size of buffer in bytes */
	41	u32 prop_count; /* properties count */
	42	u32 prop_args[]; /* properties and regular kernel arguments */
	43	};
	44
	45	/*
	46	* Interpretation of the beginning of data payload for ERT_CMD_CHAIN in
	47	* amdxdna_cmd. The rest of the payload in amdxdna_cmd is cmd BO handles.
	48	*/
	49	struct amdxdna_cmd_chain {
	50	u32 command_count;
	51	u32 submit_index;
	52	u32 error_index;
	53	u32 reserved[3];
	54	u64 data[] __counted_by(command_count);
	55	};
	56
	57	/* Exec buffer command header format */
	58	#define AMDXDNA_CMD_STATE GENMASK(3, 0)
	59	#define AMDXDNA_CMD_EXTRA_CU_MASK GENMASK(11, 10)
	60	#define AMDXDNA_CMD_COUNT GENMASK(22, 12)
	61	#define AMDXDNA_CMD_OPCODE GENMASK(27, 23)
	62	struct amdxdna_cmd {
	63	u32 header;
	64	u32 data[];
	65	};
	66
	67	struct amdxdna_hwctx {
	68	struct amdxdna_client *client;
	69	struct amdxdna_hwctx_priv *priv;
	70	char *name;
	71
	72	u32 id;
	73	u32 max_opc;
	74	u32 num_tiles;
	75	u32 mem_size;
	76	u32 fw_ctx_id;
	77	u32 col_list_len;
	78	u32 *col_list;
	79	u32 start_col;
	80	u32 num_col;
	81	#define HWCTX_STAT_INIT 0
	82	#define HWCTX_STAT_READY 1
	83	#define HWCTX_STAT_STOP 2
	84	u32 status;
	85	u32 old_status;
	86
	87	struct amdxdna_qos_info qos;
	88	struct amdxdna_hwctx_param_config_cu *cus;
	89	u32 syncobj_hdl;
	90	};
	91
	92	#define drm_job_to_xdna_job(j) \
	93	container_of(j, struct amdxdna_sched_job, base)
	94
	95	struct amdxdna_sched_job {
	96	struct drm_sched_job base;
	97	struct kref refcnt;
	98	struct amdxdna_hwctx *hwctx;
	99	struct mm_struct *mm;
	100	/* The fence to notice DRM scheduler that job is done by hardware */
	101	struct dma_fence *fence;
	102	/* user can wait on this fence */
	103	struct dma_fence *out_fence;
	104	bool job_done;
	105	u64 seq;
	106	struct amdxdna_gem_obj *cmd_bo;
	107	size_t bo_cnt;
	108	struct drm_gem_object *bos[] __counted_by(bo_cnt);
	109	};
	110
	111	static inline u32
	112	amdxdna_cmd_get_op(struct amdxdna_gem_obj *abo)
	113	{
	114	struct amdxdna_cmd *cmd = abo->mem.kva;
	115
	116	return FIELD_GET(AMDXDNA_CMD_OPCODE, cmd->header);
	117	}
	118
	119	static inline void
	120	amdxdna_cmd_set_state(struct amdxdna_gem_obj *abo, enum ert_cmd_state s)
	121	{
	122	struct amdxdna_cmd *cmd = abo->mem.kva;
	123
	124	cmd->header &= ~AMDXDNA_CMD_STATE;
	125	cmd->header \|= FIELD_PREP(AMDXDNA_CMD_STATE, s);
	126	}
	127
	128	static inline enum ert_cmd_state
	129	amdxdna_cmd_get_state(struct amdxdna_gem_obj *abo)
	130	{
	131	struct amdxdna_cmd *cmd = abo->mem.kva;
	132
	133	return FIELD_GET(AMDXDNA_CMD_STATE, cmd->header);
	134	}
	135
	136	void amdxdna_cmd_get_payload(struct amdxdna_gem_obj abo, u32 *size);
	137	int amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo);
	138
	139	static inline u32 amdxdna_hwctx_col_map(struct amdxdna_hwctx *hwctx)
	140	{
	141	return GENMASK(hwctx->start_col + hwctx->num_col - 1,
	142	hwctx->start_col);
	143	}
	144
	145	void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job);
	146	void amdxdna_hwctx_remove_all(struct amdxdna_client *client);
	147	void amdxdna_hwctx_suspend(struct amdxdna_client *client);
	148	void amdxdna_hwctx_resume(struct amdxdna_client *client);
	149
	150	int amdxdna_cmd_submit(struct amdxdna_client *client,
	151	u32 cmd_bo_hdls, u32 *arg_bo_hdls, u32 arg_bo_cnt,
	152	u32 hwctx_hdl, u64 *seq);
	153
	154	int amdxdna_cmd_wait(struct amdxdna_client *client, u32 hwctx_hdl,
	155	u64 seq, u32 timeout);
	156
	157	int amdxdna_drm_create_hwctx_ioctl(struct drm_device dev, void data, struct drm_file *filp);
	158	int amdxdna_drm_config_hwctx_ioctl(struct drm_device dev, void data, struct drm_file *filp);
	159	int amdxdna_drm_destroy_hwctx_ioctl(struct drm_device dev, void data, struct drm_file *filp);
	160	int amdxdna_drm_submit_cmd_ioctl(struct drm_device dev, void data, struct drm_file *filp);
	161
	162	#endif /* _AMDXDNA_CTX_H_ */

drivers/accel/amdxdna/amdxdna_gem.c+622−0

@@ -0,0 +1,622 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_cache.h>
	8	#include <drm/drm_device.h>
	9	#include <drm/drm_gem.h>
	10	#include <drm/drm_gem_shmem_helper.h>
	11	#include <drm/gpu_scheduler.h>
	12	#include <linux/iosys-map.h>
	13	#include <linux/vmalloc.h>
	14
	15	#include "amdxdna_ctx.h"
	16	#include "amdxdna_gem.h"
	17	#include "amdxdna_pci_drv.h"
	18
	19	#define XDNA_MAX_CMD_BO_SIZE SZ_32K
	20
	21	static int
	22	amdxdna_gem_insert_node_locked(struct amdxdna_gem_obj *abo, bool use_vmap)
	23	{
	24	struct amdxdna_client *client = abo->client;
	25	struct amdxdna_dev *xdna = client->xdna;
	26	struct amdxdna_mem *mem = &abo->mem;
	27	u64 offset;
	28	u32 align;
	29	int ret;
	30
	31	align = 1 << max(PAGE_SHIFT, xdna->dev_info->dev_mem_buf_shift);
	32	ret = drm_mm_insert_node_generic(&abo->dev_heap->mm, &abo->mm_node,
	33	mem->size, align,
	34	0, DRM_MM_INSERT_BEST);
	35	if (ret) {
	36	XDNA_ERR(xdna, "Failed to alloc dev bo memory, ret %d", ret);
	37	return ret;
	38	}
	39
	40	mem->dev_addr = abo->mm_node.start;
	41	offset = mem->dev_addr - abo->dev_heap->mem.dev_addr;
	42	mem->userptr = abo->dev_heap->mem.userptr + offset;
	43	mem->pages = &abo->dev_heap->base.pages[offset >> PAGE_SHIFT];
	44	mem->nr_pages = mem->size >> PAGE_SHIFT;
	45
	46	if (use_vmap) {
	47	mem->kva = vmap(mem->pages, mem->nr_pages, VM_MAP, PAGE_KERNEL);
	48	if (!mem->kva) {
	49	XDNA_ERR(xdna, "Failed to vmap");
	50	drm_mm_remove_node(&abo->mm_node);
	51	return -EFAULT;
	52	}
	53	}
	54
	55	return 0;
	56	}
	57
	58	static void amdxdna_gem_obj_free(struct drm_gem_object *gobj)
	59	{
	60	struct amdxdna_dev *xdna = to_xdna_dev(gobj->dev);
	61	struct amdxdna_gem_obj *abo = to_xdna_obj(gobj);
	62	struct iosys_map map = IOSYS_MAP_INIT_VADDR(abo->mem.kva);
	63
	64	XDNA_DBG(xdna, "BO type %d xdna_addr 0x%llx", abo->type, abo->mem.dev_addr);
	65	if (abo->pinned)
	66	amdxdna_gem_unpin(abo);
	67
	68	if (abo->type == AMDXDNA_BO_DEV) {
	69	mutex_lock(&abo->client->mm_lock);
	70	drm_mm_remove_node(&abo->mm_node);
	71	mutex_unlock(&abo->client->mm_lock);
	72
	73	vunmap(abo->mem.kva);
	74	drm_gem_object_put(to_gobj(abo->dev_heap));
	75	drm_gem_object_release(gobj);
	76	mutex_destroy(&abo->lock);
	77	kfree(abo);
	78	return;
	79	}
	80
	81	if (abo->type == AMDXDNA_BO_DEV_HEAP)
	82	drm_mm_takedown(&abo->mm);
	83
	84	drm_gem_vunmap_unlocked(gobj, &map);
	85	mutex_destroy(&abo->lock);
	86	drm_gem_shmem_free(&abo->base);
	87	}
	88
	89	static const struct drm_gem_object_funcs amdxdna_gem_dev_obj_funcs = {
	90	.free = amdxdna_gem_obj_free,
	91	};
	92
	93	static bool amdxdna_hmm_invalidate(struct mmu_interval_notifier *mni,
	94	const struct mmu_notifier_range *range,
	95	unsigned long cur_seq)
	96	{
	97	struct amdxdna_gem_obj *abo = container_of(mni, struct amdxdna_gem_obj,
	98	mem.notifier);
	99	struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev);
	100
	101	XDNA_DBG(xdna, "Invalid range 0x%llx, 0x%lx, type %d",
	102	abo->mem.userptr, abo->mem.size, abo->type);
	103
	104	if (!mmu_notifier_range_blockable(range))
	105	return false;
	106
	107	xdna->dev_info->ops->hmm_invalidate(abo, cur_seq);
	108
	109	return true;
	110	}
	111
	112	static const struct mmu_interval_notifier_ops amdxdna_hmm_ops = {
	113	.invalidate = amdxdna_hmm_invalidate,
	114	};
	115
	116	static void amdxdna_hmm_unregister(struct amdxdna_gem_obj *abo)
	117	{
	118	struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev);
	119
	120	if (!xdna->dev_info->ops->hmm_invalidate)
	121	return;
	122
	123	mmu_interval_notifier_remove(&abo->mem.notifier);
	124	kvfree(abo->mem.pfns);
	125	abo->mem.pfns = NULL;
	126	}
	127
	128	static int amdxdna_hmm_register(struct amdxdna_gem_obj *abo, unsigned long addr,
	129	size_t len)
	130	{
	131	struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev);
	132	u32 nr_pages;
	133	int ret;
	134
	135	if (!xdna->dev_info->ops->hmm_invalidate)
	136	return 0;
	137
	138	if (abo->mem.pfns)
	139	return -EEXIST;
	140
	141	nr_pages = (PAGE_ALIGN(addr + len) - (addr & PAGE_MASK)) >> PAGE_SHIFT;
	142	abo->mem.pfns = kvcalloc(nr_pages, sizeof(*abo->mem.pfns),
	143	GFP_KERNEL);
	144	if (!abo->mem.pfns)
	145	return -ENOMEM;
	146
	147	ret = mmu_interval_notifier_insert_locked(&abo->mem.notifier,
	148	current->mm,
	149	addr,
	150	len,
	151	&amdxdna_hmm_ops);
	152	if (ret) {
	153	XDNA_ERR(xdna, "Insert mmu notifier failed, ret %d", ret);
	154	kvfree(abo->mem.pfns);
	155	}
	156	abo->mem.userptr = addr;
	157
	158	return ret;
	159	}
	160
	161	static int amdxdna_gem_obj_mmap(struct drm_gem_object *gobj,
	162	struct vm_area_struct *vma)
	163	{
	164	struct amdxdna_gem_obj *abo = to_xdna_obj(gobj);
	165	unsigned long num_pages;
	166	int ret;
	167
	168	ret = amdxdna_hmm_register(abo, vma->vm_start, gobj->size);
	169	if (ret)
	170	return ret;
	171
	172	ret = drm_gem_shmem_mmap(&abo->base, vma);
	173	if (ret)
	174	goto hmm_unreg;
	175
	176	num_pages = gobj->size >> PAGE_SHIFT;
	177	/* Try to insert the pages */
	178	vm_flags_mod(vma, VM_MIXEDMAP, VM_PFNMAP);
	179	ret = vm_insert_pages(vma, vma->vm_start, abo->base.pages, &num_pages);
	180	if (ret)
	181	XDNA_ERR(abo->client->xdna, "Failed insert pages, ret %d", ret);
	182
	183	return 0;
	184
	185	hmm_unreg:
	186	amdxdna_hmm_unregister(abo);
	187	return ret;
	188	}
	189
	190	static vm_fault_t amdxdna_gem_vm_fault(struct vm_fault *vmf)
	191	{
	192	return drm_gem_shmem_vm_ops.fault(vmf);
	193	}
	194
	195	static void amdxdna_gem_vm_open(struct vm_area_struct *vma)
	196	{
	197	drm_gem_shmem_vm_ops.open(vma);
	198	}
	199
	200	static void amdxdna_gem_vm_close(struct vm_area_struct *vma)
	201	{
	202	struct drm_gem_object *gobj = vma->vm_private_data;
	203
	204	amdxdna_hmm_unregister(to_xdna_obj(gobj));
	205	drm_gem_shmem_vm_ops.close(vma);
	206	}
	207
	208	static const struct vm_operations_struct amdxdna_gem_vm_ops = {
	209	.fault = amdxdna_gem_vm_fault,
	210	.open = amdxdna_gem_vm_open,
	211	.close = amdxdna_gem_vm_close,
	212	};
	213
	214	static const struct drm_gem_object_funcs amdxdna_gem_shmem_funcs = {
	215	.free = amdxdna_gem_obj_free,
	216	.print_info = drm_gem_shmem_object_print_info,
	217	.pin = drm_gem_shmem_object_pin,
	218	.unpin = drm_gem_shmem_object_unpin,
	219	.get_sg_table = drm_gem_shmem_object_get_sg_table,
	220	.vmap = drm_gem_shmem_object_vmap,
	221	.vunmap = drm_gem_shmem_object_vunmap,
	222	.mmap = amdxdna_gem_obj_mmap,
	223	.vm_ops = &amdxdna_gem_vm_ops,
	224	};
	225
	226	static struct amdxdna_gem_obj *
	227	amdxdna_gem_create_obj(struct drm_device *dev, size_t size)
	228	{
	229	struct amdxdna_gem_obj *abo;
	230
	231	abo = kzalloc(sizeof(*abo), GFP_KERNEL);
	232	if (!abo)
	233	return ERR_PTR(-ENOMEM);
	234
	235	abo->pinned = false;
	236	abo->assigned_hwctx = AMDXDNA_INVALID_CTX_HANDLE;
	237	mutex_init(&abo->lock);
	238
	239	abo->mem.userptr = AMDXDNA_INVALID_ADDR;
	240	abo->mem.dev_addr = AMDXDNA_INVALID_ADDR;
	241	abo->mem.size = size;
	242
	243	return abo;
	244	}
	245
	246	/* For drm_driver->gem_create_object callback */
	247	struct drm_gem_object *
	248	amdxdna_gem_create_object_cb(struct drm_device *dev, size_t size)
	249	{
	250	struct amdxdna_gem_obj *abo;
	251
	252	abo = amdxdna_gem_create_obj(dev, size);
	253	if (IS_ERR(abo))
	254	return ERR_CAST(abo);
	255
	256	to_gobj(abo)->funcs = &amdxdna_gem_shmem_funcs;
	257
	258	return to_gobj(abo);
	259	}
	260
	261	static struct amdxdna_gem_obj *
	262	amdxdna_drm_alloc_shmem(struct drm_device *dev,
	263	struct amdxdna_drm_create_bo *args,
	264	struct drm_file *filp)
	265	{
	266	struct amdxdna_client *client = filp->driver_priv;
	267	struct drm_gem_shmem_object *shmem;
	268	struct amdxdna_gem_obj *abo;
	269
	270	shmem = drm_gem_shmem_create(dev, args->size);
	271	if (IS_ERR(shmem))
	272	return ERR_CAST(shmem);
	273
	274	shmem->map_wc = false;
	275
	276	abo = to_xdna_obj(&shmem->base);
	277	abo->client = client;
	278	abo->type = AMDXDNA_BO_SHMEM;
	279
	280	return abo;
	281	}
	282
	283	static struct amdxdna_gem_obj *
	284	amdxdna_drm_create_dev_heap(struct drm_device *dev,
	285	struct amdxdna_drm_create_bo *args,
	286	struct drm_file *filp)
	287	{
	288	struct amdxdna_client *client = filp->driver_priv;
	289	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	290	struct drm_gem_shmem_object *shmem;
	291	struct amdxdna_gem_obj *abo;
	292	int ret;
	293
	294	if (args->size > xdna->dev_info->dev_mem_size) {
	295	XDNA_DBG(xdna, "Invalid dev heap size 0x%llx, limit 0x%lx",
	296	args->size, xdna->dev_info->dev_mem_size);
	297	return ERR_PTR(-EINVAL);
	298	}
	299
	300	mutex_lock(&client->mm_lock);
	301	if (client->dev_heap) {
	302	XDNA_DBG(client->xdna, "dev heap is already created");
	303	ret = -EBUSY;
	304	goto mm_unlock;
	305	}
	306
	307	shmem = drm_gem_shmem_create(dev, args->size);
	308	if (IS_ERR(shmem)) {
	309	ret = PTR_ERR(shmem);
	310	goto mm_unlock;
	311	}
	312
	313	shmem->map_wc = false;
	314	abo = to_xdna_obj(&shmem->base);
	315
	316	abo->type = AMDXDNA_BO_DEV_HEAP;
	317	abo->client = client;
	318	abo->mem.dev_addr = client->xdna->dev_info->dev_mem_base;
	319	drm_mm_init(&abo->mm, abo->mem.dev_addr, abo->mem.size);
	320
	321	client->dev_heap = abo;
	322	drm_gem_object_get(to_gobj(abo));
	323	mutex_unlock(&client->mm_lock);
	324
	325	return abo;
	326
	327	mm_unlock:
	328	mutex_unlock(&client->mm_lock);
	329	return ERR_PTR(ret);
	330	}
	331
	332	struct amdxdna_gem_obj *
	333	amdxdna_drm_alloc_dev_bo(struct drm_device *dev,
	334	struct amdxdna_drm_create_bo *args,
	335	struct drm_file *filp, bool use_vmap)
	336	{
	337	struct amdxdna_client *client = filp->driver_priv;
	338	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	339	size_t aligned_sz = PAGE_ALIGN(args->size);
	340	struct amdxdna_gem_obj abo, heap;
	341	int ret;
	342
	343	mutex_lock(&client->mm_lock);
	344	heap = client->dev_heap;
	345	if (!heap) {
	346	ret = -EINVAL;
	347	goto mm_unlock;
	348	}
	349
	350	if (heap->mem.userptr == AMDXDNA_INVALID_ADDR) {
	351	XDNA_ERR(xdna, "Invalid dev heap userptr");
	352	ret = -EINVAL;
	353	goto mm_unlock;
	354	}
	355
	356	if (args->size > heap->mem.size) {
	357	XDNA_ERR(xdna, "Invalid dev bo size 0x%llx, limit 0x%lx",
	358	args->size, heap->mem.size);
	359	ret = -EINVAL;
	360	goto mm_unlock;
	361	}
	362
	363	abo = amdxdna_gem_create_obj(&xdna->ddev, aligned_sz);
	364	if (IS_ERR(abo)) {
	365	ret = PTR_ERR(abo);
	366	goto mm_unlock;
	367	}
	368	to_gobj(abo)->funcs = &amdxdna_gem_dev_obj_funcs;
	369	abo->type = AMDXDNA_BO_DEV;
	370	abo->client = client;
	371	abo->dev_heap = heap;
	372	ret = amdxdna_gem_insert_node_locked(abo, use_vmap);
	373	if (ret) {
	374	XDNA_ERR(xdna, "Failed to alloc dev bo memory, ret %d", ret);
	375	goto mm_unlock;
	376	}
	377
	378	drm_gem_object_get(to_gobj(heap));
	379	drm_gem_private_object_init(&xdna->ddev, to_gobj(abo), aligned_sz);
	380
	381	mutex_unlock(&client->mm_lock);
	382	return abo;
	383
	384	mm_unlock:
	385	mutex_unlock(&client->mm_lock);
	386	return ERR_PTR(ret);
	387	}
	388
	389	static struct amdxdna_gem_obj *
	390	amdxdna_drm_create_cmd_bo(struct drm_device *dev,
	391	struct amdxdna_drm_create_bo *args,
	392	struct drm_file *filp)
	393	{
	394	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	395	struct drm_gem_shmem_object *shmem;
	396	struct amdxdna_gem_obj *abo;
	397	struct iosys_map map;
	398	int ret;
	399
	400	if (args->size > XDNA_MAX_CMD_BO_SIZE) {
	401	XDNA_ERR(xdna, "Command bo size 0x%llx too large", args->size);
	402	return ERR_PTR(-EINVAL);
	403	}
	404
	405	if (args->size < sizeof(struct amdxdna_cmd)) {
	406	XDNA_DBG(xdna, "Command BO size 0x%llx too small", args->size);
	407	return ERR_PTR(-EINVAL);
	408	}
	409
	410	shmem = drm_gem_shmem_create(dev, args->size);
	411	if (IS_ERR(shmem))
	412	return ERR_CAST(shmem);
	413
	414	shmem->map_wc = false;
	415	abo = to_xdna_obj(&shmem->base);
	416
	417	abo->type = AMDXDNA_BO_CMD;
	418	abo->client = filp->driver_priv;
	419
	420	ret = drm_gem_vmap_unlocked(to_gobj(abo), &map);
	421	if (ret) {
	422	XDNA_ERR(xdna, "Vmap cmd bo failed, ret %d", ret);
	423	goto release_obj;
	424	}
	425	abo->mem.kva = map.vaddr;
	426
	427	return abo;
	428
	429	release_obj:
	430	drm_gem_shmem_free(shmem);
	431	return ERR_PTR(ret);
	432	}
	433
	434	int amdxdna_drm_create_bo_ioctl(struct drm_device dev, void data, struct drm_file *filp)
	435	{
	436	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	437	struct amdxdna_drm_create_bo *args = data;
	438	struct amdxdna_gem_obj *abo;
	439	int ret;
	440
	441	if (args->flags \|\| args->vaddr \|\| !args->size)
	442	return -EINVAL;
	443
	444	XDNA_DBG(xdna, "BO arg type %d vaddr 0x%llx size 0x%llx flags 0x%llx",
	445	args->type, args->vaddr, args->size, args->flags);
	446	switch (args->type) {
	447	case AMDXDNA_BO_SHMEM:
	448	abo = amdxdna_drm_alloc_shmem(dev, args, filp);
	449	break;
	450	case AMDXDNA_BO_DEV_HEAP:
	451	abo = amdxdna_drm_create_dev_heap(dev, args, filp);
	452	break;
	453	case AMDXDNA_BO_DEV:
	454	abo = amdxdna_drm_alloc_dev_bo(dev, args, filp, false);
	455	break;
	456	case AMDXDNA_BO_CMD:
	457	abo = amdxdna_drm_create_cmd_bo(dev, args, filp);
	458	break;
	459	default:
	460	return -EINVAL;
	461	}
	462	if (IS_ERR(abo))
	463	return PTR_ERR(abo);
	464
	465	/* ready to publish object to userspace */
	466	ret = drm_gem_handle_create(filp, to_gobj(abo), &args->handle);
	467	if (ret) {
	468	XDNA_ERR(xdna, "Create handle failed");
	469	goto put_obj;
	470	}
	471
	472	XDNA_DBG(xdna, "BO hdl %d type %d userptr 0x%llx xdna_addr 0x%llx size 0x%lx",
	473	args->handle, args->type, abo->mem.userptr,
	474	abo->mem.dev_addr, abo->mem.size);
	475	put_obj:
	476	/* Dereference object reference. Handle holds it now. */
	477	drm_gem_object_put(to_gobj(abo));
	478	return ret;
	479	}
	480
	481	int amdxdna_gem_pin_nolock(struct amdxdna_gem_obj *abo)
	482	{
	483	struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev);
	484	int ret;
	485
	486	switch (abo->type) {
	487	case AMDXDNA_BO_SHMEM:
	488	case AMDXDNA_BO_DEV_HEAP:
	489	ret = drm_gem_shmem_pin(&abo->base);
	490	break;
	491	case AMDXDNA_BO_DEV:
	492	ret = drm_gem_shmem_pin(&abo->dev_heap->base);
	493	break;
	494	default:
	495	ret = -EOPNOTSUPP;
	496	}
	497
	498	XDNA_DBG(xdna, "BO type %d ret %d", abo->type, ret);
	499	return ret;
	500	}
	501
	502	int amdxdna_gem_pin(struct amdxdna_gem_obj *abo)
	503	{
	504	int ret;
	505
	506	if (abo->type == AMDXDNA_BO_DEV)
	507	abo = abo->dev_heap;
	508
	509	mutex_lock(&abo->lock);
	510	ret = amdxdna_gem_pin_nolock(abo);
	511	mutex_unlock(&abo->lock);
	512
	513	return ret;
	514	}
	515
	516	void amdxdna_gem_unpin(struct amdxdna_gem_obj *abo)
	517	{
	518	if (abo->type == AMDXDNA_BO_DEV)
	519	abo = abo->dev_heap;
	520
	521	mutex_lock(&abo->lock);
	522	drm_gem_shmem_unpin(&abo->base);
	523	mutex_unlock(&abo->lock);
	524	}
	525
	526	struct amdxdna_gem_obj amdxdna_gem_get_obj(struct amdxdna_client client,
	527	u32 bo_hdl, u8 bo_type)
	528	{
	529	struct amdxdna_dev *xdna = client->xdna;
	530	struct amdxdna_gem_obj *abo;
	531	struct drm_gem_object *gobj;
	532
	533	gobj = drm_gem_object_lookup(client->filp, bo_hdl);
	534	if (!gobj) {
	535	XDNA_DBG(xdna, "Can not find bo %d", bo_hdl);
	536	return NULL;
	537	}
	538
	539	abo = to_xdna_obj(gobj);
	540	if (bo_type == AMDXDNA_BO_INVALID \|\| abo->type == bo_type)
	541	return abo;
	542
	543	drm_gem_object_put(gobj);
	544	return NULL;
	545	}
	546
	547	int amdxdna_drm_get_bo_info_ioctl(struct drm_device dev, void data, struct drm_file *filp)
	548	{
	549	struct amdxdna_drm_get_bo_info *args = data;
	550	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	551	struct amdxdna_gem_obj *abo;
	552	struct drm_gem_object *gobj;
	553	int ret = 0;
	554
	555	if (args->ext \|\| args->ext_flags)
	556	return -EINVAL;
	557
	558	gobj = drm_gem_object_lookup(filp, args->handle);
	559	if (!gobj) {
	560	XDNA_DBG(xdna, "Lookup GEM object %d failed", args->handle);
	561	return -ENOENT;
	562	}
	563
	564	abo = to_xdna_obj(gobj);
	565	args->vaddr = abo->mem.userptr;
	566	args->xdna_addr = abo->mem.dev_addr;
	567
	568	if (abo->type != AMDXDNA_BO_DEV)
	569	args->map_offset = drm_vma_node_offset_addr(&gobj->vma_node);
	570	else
	571	args->map_offset = AMDXDNA_INVALID_ADDR;
	572
	573	XDNA_DBG(xdna, "BO hdl %d map_offset 0x%llx vaddr 0x%llx xdna_addr 0x%llx",
	574	args->handle, args->map_offset, args->vaddr, args->xdna_addr);
	575
	576	drm_gem_object_put(gobj);
	577	return ret;
	578	}
	579
	580	/*
	581	* The sync bo ioctl is to make sure the CPU cache is in sync with memory.
	582	* This is required because NPU is not cache coherent device. CPU cache
	583	* flushing/invalidation is expensive so it is best to handle this outside
	584	* of the command submission path. This ioctl allows explicit cache
	585	* flushing/invalidation outside of the critical path.
	586	*/
	587	int amdxdna_drm_sync_bo_ioctl(struct drm_device *dev,
	588	void data, struct drm_file filp)
	589	{
	590	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	591	struct amdxdna_drm_sync_bo *args = data;
	592	struct amdxdna_gem_obj *abo;
	593	struct drm_gem_object *gobj;
	594	int ret;
	595
	596	gobj = drm_gem_object_lookup(filp, args->handle);
	597	if (!gobj) {
	598	XDNA_ERR(xdna, "Lookup GEM object failed");
	599	return -ENOENT;
	600	}
	601	abo = to_xdna_obj(gobj);
	602
	603	ret = amdxdna_gem_pin(abo);
	604	if (ret) {
	605	XDNA_ERR(xdna, "Pin BO %d failed, ret %d", args->handle, ret);
	606	goto put_obj;
	607	}
	608
	609	if (abo->type == AMDXDNA_BO_DEV)
	610	drm_clflush_pages(abo->mem.pages, abo->mem.nr_pages);
	611	else
	612	drm_clflush_pages(abo->base.pages, gobj->size >> PAGE_SHIFT);
	613
	614	amdxdna_gem_unpin(abo);
	615
	616	XDNA_DBG(xdna, "Sync bo %d offset 0x%llx, size 0x%llx\n",
	617	args->handle, args->offset, args->size);
	618
	619	put_obj:
	620	drm_gem_object_put(gobj);
	621	return ret;
	622	}

drivers/accel/amdxdna/amdxdna_gem.h+65−0

@@ -0,0 +1,65 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _AMDXDNA_GEM_H_
	7	#define _AMDXDNA_GEM_H_
	8
	9	struct amdxdna_mem {
	10	u64 userptr;
	11	void *kva;
	12	u64 dev_addr;
	13	size_t size;
	14	struct page **pages;
	15	u32 nr_pages;
	16	struct mmu_interval_notifier notifier;
	17	unsigned long *pfns;
	18	bool map_invalid;
	19	};
	20
	21	struct amdxdna_gem_obj {
	22	struct drm_gem_shmem_object base;
	23	struct amdxdna_client *client;
	24	u8 type;
	25	bool pinned;
	26	struct mutex lock; /* Protects: pinned */
	27	struct amdxdna_mem mem;
	28
	29	/* Below members is uninitialized when needed */
	30	struct drm_mm mm; /* For AMDXDNA_BO_DEV_HEAP */
	31	struct amdxdna_gem_obj dev_heap; / For AMDXDNA_BO_DEV */
	32	struct drm_mm_node mm_node; /* For AMDXDNA_BO_DEV */
	33	u32 assigned_hwctx;
	34	};
	35
	36	#define to_gobj(obj) (&(obj)->base.base)
	37
	38	static inline struct amdxdna_gem_obj to_xdna_obj(struct drm_gem_object gobj)
	39	{
	40	return container_of(gobj, struct amdxdna_gem_obj, base.base);
	41	}
	42
	43	struct amdxdna_gem_obj amdxdna_gem_get_obj(struct amdxdna_client client,
	44	u32 bo_hdl, u8 bo_type);
	45	static inline void amdxdna_gem_put_obj(struct amdxdna_gem_obj *abo)
	46	{
	47	drm_gem_object_put(to_gobj(abo));
	48	}
	49
	50	struct drm_gem_object *
	51	amdxdna_gem_create_object_cb(struct drm_device *dev, size_t size);
	52	struct amdxdna_gem_obj *
	53	amdxdna_drm_alloc_dev_bo(struct drm_device *dev,
	54	struct amdxdna_drm_create_bo *args,
	55	struct drm_file *filp, bool use_vmap);
	56
	57	int amdxdna_gem_pin_nolock(struct amdxdna_gem_obj *abo);
	58	int amdxdna_gem_pin(struct amdxdna_gem_obj *abo);
	59	void amdxdna_gem_unpin(struct amdxdna_gem_obj *abo);
	60
	61	int amdxdna_drm_create_bo_ioctl(struct drm_device dev, void data, struct drm_file *filp);
	62	int amdxdna_drm_get_bo_info_ioctl(struct drm_device dev, void data, struct drm_file *filp);
	63	int amdxdna_drm_sync_bo_ioctl(struct drm_device dev, void data, struct drm_file *filp);
	64
	65	#endif /* _AMDXDNA_GEM_H_ */

drivers/accel/amdxdna/amdxdna_mailbox.c+576−0

@@ -0,0 +1,576 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/drm_device.h>
	7	#include <drm/drm_managed.h>
	8	#include <linux/bitfield.h>
	9	#include <linux/iopoll.h>
	10
	11	#define CREATE_TRACE_POINTS
	12	#include <trace/events/amdxdna.h>
	13
	14	#include "amdxdna_mailbox.h"
	15
	16	#define MB_ERR(chann, fmt, args...) \
	17	({ \
	18	typeof(chann) _chann = chann; \
	19	dev_err((_chann)->mb->dev, "xdna_mailbox.%d: "fmt, \
	20	(_chann)->msix_irq, ##args); \
	21	})
	22	#define MB_DBG(chann, fmt, args...) \
	23	({ \
	24	typeof(chann) _chann = chann; \
	25	dev_dbg((_chann)->mb->dev, "xdna_mailbox.%d: "fmt, \
	26	(_chann)->msix_irq, ##args); \
	27	})
	28	#define MB_WARN_ONCE(chann, fmt, args...) \
	29	({ \
	30	typeof(chann) _chann = chann; \
	31	dev_warn_once((_chann)->mb->dev, "xdna_mailbox.%d: "fmt, \
	32	(_chann)->msix_irq, ##args); \
	33	})
	34
	35	#define MAGIC_VAL 0x1D000000U
	36	#define MAGIC_VAL_MASK 0xFF000000
	37	#define MAX_MSG_ID_ENTRIES 256
	38	#define MSG_RX_TIMER 200 /* milliseconds */
	39	#define MAILBOX_NAME "xdna_mailbox"
	40
	41	enum channel_res_type {
	42	CHAN_RES_X2I,
	43	CHAN_RES_I2X,
	44	CHAN_RES_NUM
	45	};
	46
	47	struct mailbox {
	48	struct device *dev;
	49	struct xdna_mailbox_res res;
	50	};
	51
	52	struct mailbox_channel {
	53	struct mailbox *mb;
	54	struct xdna_mailbox_chann_res res[CHAN_RES_NUM];
	55	int msix_irq;
	56	u32 iohub_int_addr;
	57	struct idr chan_idr;
	58	spinlock_t chan_idr_lock; /* protect chan_idr */
	59	u32 x2i_tail;
	60
	61	/* Received msg related fields */
	62	struct workqueue_struct *work_q;
	63	struct work_struct rx_work;
	64	u32 i2x_head;
	65	bool bad_state;
	66	};
	67
	68	#define MSG_BODY_SZ GENMASK(10, 0)
	69	#define MSG_PROTO_VER GENMASK(23, 16)
	70	struct xdna_msg_header {
	71	__u32 total_size;
	72	__u32 sz_ver;
	73	__u32 id;
	74	__u32 opcode;
	75	} __packed;
	76
	77	static_assert(sizeof(struct xdna_msg_header) == 16);
	78
	79	struct mailbox_pkg {
	80	struct xdna_msg_header header;
	81	__u32 payload[];
	82	};
	83
	84	/* The protocol version. */
	85	#define MSG_PROTOCOL_VERSION 0x1
	86	/* The tombstone value. */
	87	#define TOMBSTONE 0xDEADFACE
	88
	89	struct mailbox_msg {
	90	void *handle;
	91	int (notify_cb)(void handle, const u32 *data, size_t size);
	92	size_t pkg_size; /* package size in bytes */
	93	struct mailbox_pkg pkg;
	94	};
	95
	96	static void mailbox_reg_write(struct mailbox_channel *mb_chann, u32 mbox_reg, u32 data)
	97	{
	98	struct xdna_mailbox_res *mb_res = &mb_chann->mb->res;
	99	u64 ringbuf_addr = mb_res->mbox_base + mbox_reg;
	100
	101	writel(data, (void *)ringbuf_addr);
	102	}
	103
	104	static u32 mailbox_reg_read(struct mailbox_channel *mb_chann, u32 mbox_reg)
	105	{
	106	struct xdna_mailbox_res *mb_res = &mb_chann->mb->res;
	107	u64 ringbuf_addr = mb_res->mbox_base + mbox_reg;
	108
	109	return readl((void *)ringbuf_addr);
	110	}
	111
	112	static int mailbox_reg_read_non_zero(struct mailbox_channel mb_chann, u32 mbox_reg, u32 val)
	113	{
	114	struct xdna_mailbox_res *mb_res = &mb_chann->mb->res;
	115	u64 ringbuf_addr = mb_res->mbox_base + mbox_reg;
	116	int ret, value;
	117
	118	/* Poll till value is not zero */
	119	ret = readx_poll_timeout(readl, (void *)ringbuf_addr, value,
	120	value, 1 /* us */, 100);
	121	if (ret < 0)
	122	return ret;
	123
	124	*val = value;
	125	return 0;
	126	}
	127
	128	static inline void
	129	mailbox_set_headptr(struct mailbox_channel *mb_chann, u32 headptr_val)
	130	{
	131	mailbox_reg_write(mb_chann, mb_chann->res[CHAN_RES_I2X].mb_head_ptr_reg, headptr_val);
	132	mb_chann->i2x_head = headptr_val;
	133	}
	134
	135	static inline void
	136	mailbox_set_tailptr(struct mailbox_channel *mb_chann, u32 tailptr_val)
	137	{
	138	mailbox_reg_write(mb_chann, mb_chann->res[CHAN_RES_X2I].mb_tail_ptr_reg, tailptr_val);
	139	mb_chann->x2i_tail = tailptr_val;
	140	}
	141
	142	static inline u32
	143	mailbox_get_headptr(struct mailbox_channel *mb_chann, enum channel_res_type type)
	144	{
	145	return mailbox_reg_read(mb_chann, mb_chann->res[type].mb_head_ptr_reg);
	146	}
	147
	148	static inline u32
	149	mailbox_get_tailptr(struct mailbox_channel *mb_chann, enum channel_res_type type)
	150	{
	151	return mailbox_reg_read(mb_chann, mb_chann->res[type].mb_tail_ptr_reg);
	152	}
	153
	154	static inline u32
	155	mailbox_get_ringbuf_size(struct mailbox_channel *mb_chann, enum channel_res_type type)
	156	{
	157	return mb_chann->res[type].rb_size;
	158	}
	159
	160	static inline int mailbox_validate_msgid(int msg_id)
	161	{
	162	return (msg_id & MAGIC_VAL_MASK) == MAGIC_VAL;
	163	}
	164
	165	static int mailbox_acquire_msgid(struct mailbox_channel mb_chann, struct mailbox_msg mb_msg)
	166	{
	167	unsigned long flags;
	168	int msg_id;
	169
	170	spin_lock_irqsave(&mb_chann->chan_idr_lock, flags);
	171	msg_id = idr_alloc_cyclic(&mb_chann->chan_idr, mb_msg, 0,
	172	MAX_MSG_ID_ENTRIES, GFP_NOWAIT);
	173	spin_unlock_irqrestore(&mb_chann->chan_idr_lock, flags);
	174	if (msg_id < 0)
	175	return msg_id;
	176
	177	/*
	178	* The IDR becomes less efficient when dealing with larger IDs.
	179	* Thus, add MAGIC_VAL to the higher bits.
	180	*/
	181	msg_id \|= MAGIC_VAL;
	182	return msg_id;
	183	}
	184
	185	static void mailbox_release_msgid(struct mailbox_channel *mb_chann, int msg_id)
	186	{
	187	unsigned long flags;
	188
	189	msg_id &= ~MAGIC_VAL_MASK;
	190	spin_lock_irqsave(&mb_chann->chan_idr_lock, flags);
	191	idr_remove(&mb_chann->chan_idr, msg_id);
	192	spin_unlock_irqrestore(&mb_chann->chan_idr_lock, flags);
	193	}
	194
	195	static int mailbox_release_msg(int id, void p, void data)
	196	{
	197	struct mailbox_channel *mb_chann = data;
	198	struct mailbox_msg *mb_msg = p;
	199
	200	MB_DBG(mb_chann, "msg_id 0x%x msg opcode 0x%x",
	201	mb_msg->pkg.header.id, mb_msg->pkg.header.opcode);
	202	mb_msg->notify_cb(mb_msg->handle, NULL, 0);
	203	kfree(mb_msg);
	204
	205	return 0;
	206	}
	207
	208	static int
	209	mailbox_send_msg(struct mailbox_channel mb_chann, struct mailbox_msg mb_msg)
	210	{
	211	u32 ringbuf_size;
	212	u32 head, tail;
	213	u32 start_addr;
	214	u64 write_addr;
	215	u32 tmp_tail;
	216
	217	head = mailbox_get_headptr(mb_chann, CHAN_RES_X2I);
	218	tail = mb_chann->x2i_tail;
	219	ringbuf_size = mailbox_get_ringbuf_size(mb_chann, CHAN_RES_X2I);
	220	start_addr = mb_chann->res[CHAN_RES_X2I].rb_start_addr;
	221	tmp_tail = tail + mb_msg->pkg_size;
	222
	223	if (tail < head && tmp_tail >= head)
	224	goto no_space;
	225
	226	if (tail >= head && (tmp_tail > ringbuf_size - sizeof(u32) &&
	227	mb_msg->pkg_size >= head))
	228	goto no_space;
	229
	230	if (tail >= head && tmp_tail > ringbuf_size - sizeof(u32)) {
	231	write_addr = mb_chann->mb->res.ringbuf_base + start_addr + tail;
	232	writel(TOMBSTONE, (void *)write_addr);
	233
	234	/* tombstone is set. Write from the start of the ringbuf */
	235	tail = 0;
	236	}
	237
	238	write_addr = mb_chann->mb->res.ringbuf_base + start_addr + tail;
	239	memcpy_toio((void *)write_addr, &mb_msg->pkg, mb_msg->pkg_size);
	240	mailbox_set_tailptr(mb_chann, tail + mb_msg->pkg_size);
	241
	242	trace_mbox_set_tail(MAILBOX_NAME, mb_chann->msix_irq,
	243	mb_msg->pkg.header.opcode,
	244	mb_msg->pkg.header.id);
	245
	246	return 0;
	247
	248	no_space:
	249	return -ENOSPC;
	250	}
	251
	252	static int
	253	mailbox_get_resp(struct mailbox_channel mb_chann, struct xdna_msg_header header,
	254	void *data)
	255	{
	256	struct mailbox_msg *mb_msg;
	257	unsigned long flags;
	258	int msg_id;
	259	int ret;
	260
	261	msg_id = header->id;
	262	if (!mailbox_validate_msgid(msg_id)) {
	263	MB_ERR(mb_chann, "Bad message ID 0x%x", msg_id);
	264	return -EINVAL;
	265	}
	266
	267	msg_id &= ~MAGIC_VAL_MASK;
	268	spin_lock_irqsave(&mb_chann->chan_idr_lock, flags);
	269	mb_msg = idr_find(&mb_chann->chan_idr, msg_id);
	270	if (!mb_msg) {
	271	MB_ERR(mb_chann, "Cannot find msg 0x%x", msg_id);
	272	spin_unlock_irqrestore(&mb_chann->chan_idr_lock, flags);
	273	return -EINVAL;
	274	}
	275	idr_remove(&mb_chann->chan_idr, msg_id);
	276	spin_unlock_irqrestore(&mb_chann->chan_idr_lock, flags);
	277
	278	MB_DBG(mb_chann, "opcode 0x%x size %d id 0x%x",
	279	header->opcode, header->total_size, header->id);
	280	ret = mb_msg->notify_cb(mb_msg->handle, data, header->total_size);
	281	if (unlikely(ret))
	282	MB_ERR(mb_chann, "Message callback ret %d", ret);
	283
	284	kfree(mb_msg);
	285	return ret;
	286	}
	287
	288	static int mailbox_get_msg(struct mailbox_channel *mb_chann)
	289	{
	290	struct xdna_msg_header header;
	291	u32 msg_size, rest;
	292	u32 ringbuf_size;
	293	u32 head, tail;
	294	u32 start_addr;
	295	u64 read_addr;
	296	int ret;
	297
	298	if (mailbox_reg_read_non_zero(mb_chann, mb_chann->res[CHAN_RES_I2X].mb_tail_ptr_reg, &tail))
	299	return -EINVAL;
	300	head = mb_chann->i2x_head;
	301	ringbuf_size = mailbox_get_ringbuf_size(mb_chann, CHAN_RES_I2X);
	302	start_addr = mb_chann->res[CHAN_RES_I2X].rb_start_addr;
	303
	304	if (unlikely(tail > ringbuf_size \|\| !IS_ALIGNED(tail, 4))) {
	305	MB_WARN_ONCE(mb_chann, "Invalid tail 0x%x", tail);
	306	return -EINVAL;
	307	}
	308
	309	/* ringbuf empty */
	310	if (head == tail)
	311	return -ENOENT;
	312
	313	if (head == ringbuf_size)
	314	head = 0;
	315
	316	/* Peek size of the message or TOMBSTONE */
	317	read_addr = mb_chann->mb->res.ringbuf_base + start_addr + head;
	318	header.total_size = readl((void *)read_addr);
	319	/* size is TOMBSTONE, set next read from 0 */
	320	if (header.total_size == TOMBSTONE) {
	321	if (head < tail) {
	322	MB_WARN_ONCE(mb_chann, "Tombstone, head 0x%x tail 0x%x",
	323	head, tail);
	324	return -EINVAL;
	325	}
	326	mailbox_set_headptr(mb_chann, 0);
	327	return 0;
	328	}
	329
	330	if (unlikely(!header.total_size \|\| !IS_ALIGNED(header.total_size, 4))) {
	331	MB_WARN_ONCE(mb_chann, "Invalid total size 0x%x", header.total_size);
	332	return -EINVAL;
	333	}
	334	msg_size = sizeof(header) + header.total_size;
	335
	336	if (msg_size > ringbuf_size - head \|\| msg_size > tail - head) {
	337	MB_WARN_ONCE(mb_chann, "Invalid message size %d, tail %d, head %d",
	338	msg_size, tail, head);
	339	return -EINVAL;
	340	}
	341
	342	rest = sizeof(header) - sizeof(u32);
	343	read_addr += sizeof(u32);
	344	memcpy_fromio((u32 )&header + 1, (void )read_addr, rest);
	345	read_addr += rest;
	346
	347	ret = mailbox_get_resp(mb_chann, &header, (u32 *)read_addr);
	348
	349	mailbox_set_headptr(mb_chann, head + msg_size);
	350	/* After update head, it can equal to ringbuf_size. This is expected. */
	351	trace_mbox_set_head(MAILBOX_NAME, mb_chann->msix_irq,
	352	header.opcode, header.id);
	353
	354	return ret;
	355	}
	356
	357	static irqreturn_t mailbox_irq_handler(int irq, void *p)
	358	{
	359	struct mailbox_channel *mb_chann = p;
	360
	361	trace_mbox_irq_handle(MAILBOX_NAME, irq);
	362	/* Schedule a rx_work to call the callback functions */
	363	queue_work(mb_chann->work_q, &mb_chann->rx_work);
	364	/* Clear IOHUB register */
	365	mailbox_reg_write(mb_chann, mb_chann->iohub_int_addr, 0);
	366
	367	return IRQ_HANDLED;
	368	}
	369
	370	static void mailbox_rx_worker(struct work_struct *rx_work)
	371	{
	372	struct mailbox_channel *mb_chann;
	373	int ret;
	374
	375	mb_chann = container_of(rx_work, struct mailbox_channel, rx_work);
	376
	377	if (READ_ONCE(mb_chann->bad_state)) {
	378	MB_ERR(mb_chann, "Channel in bad state, work aborted");
	379	return;
	380	}
	381
	382	while (1) {
	383	/*
	384	* If return is 0, keep consuming next message, until there is
	385	* no messages or an error happened.
	386	*/
	387	ret = mailbox_get_msg(mb_chann);
	388	if (ret == -ENOENT)
	389	break;
	390
	391	/* Other error means device doesn't look good, disable irq. */
	392	if (unlikely(ret)) {
	393	MB_ERR(mb_chann, "Unexpected ret %d, disable irq", ret);
	394	WRITE_ONCE(mb_chann->bad_state, true);
	395	disable_irq(mb_chann->msix_irq);
	396	break;
	397	}
	398	}
	399	}
	400
	401	int xdna_mailbox_send_msg(struct mailbox_channel *mb_chann,
	402	const struct xdna_mailbox_msg *msg, u64 tx_timeout)
	403	{
	404	struct xdna_msg_header *header;
	405	struct mailbox_msg *mb_msg;
	406	size_t pkg_size;
	407	int ret;
	408
	409	pkg_size = sizeof(*header) + msg->send_size;
	410	if (pkg_size > mailbox_get_ringbuf_size(mb_chann, CHAN_RES_X2I)) {
	411	MB_ERR(mb_chann, "Message size larger than ringbuf size");
	412	return -EINVAL;
	413	}
	414
	415	if (unlikely(!IS_ALIGNED(msg->send_size, 4))) {
	416	MB_ERR(mb_chann, "Message must be 4 bytes align");
	417	return -EINVAL;
	418	}
	419
	420	/* The fist word in payload can NOT be TOMBSTONE */
	421	if (unlikely(((u32 *)msg->send_data)[0] == TOMBSTONE)) {
	422	MB_ERR(mb_chann, "Tomb stone in data");
	423	return -EINVAL;
	424	}
	425
	426	if (READ_ONCE(mb_chann->bad_state)) {
	427	MB_ERR(mb_chann, "Channel in bad state");
	428	return -EPIPE;
	429	}
	430
	431	mb_msg = kzalloc(sizeof(*mb_msg) + pkg_size, GFP_KERNEL);
	432	if (!mb_msg)
	433	return -ENOMEM;
	434
	435	mb_msg->handle = msg->handle;
	436	mb_msg->notify_cb = msg->notify_cb;
	437	mb_msg->pkg_size = pkg_size;
	438
	439	header = &mb_msg->pkg.header;
	440	/*
	441	* Hardware use total_size and size to split huge message.
	442	* We do not support it here. Thus the values are the same.
	443	*/
	444	header->total_size = msg->send_size;
	445	header->sz_ver = FIELD_PREP(MSG_BODY_SZ, msg->send_size) \|
	446	FIELD_PREP(MSG_PROTO_VER, MSG_PROTOCOL_VERSION);
	447	header->opcode = msg->opcode;
	448	memcpy(mb_msg->pkg.payload, msg->send_data, msg->send_size);
	449
	450	ret = mailbox_acquire_msgid(mb_chann, mb_msg);
	451	if (unlikely(ret < 0)) {
	452	MB_ERR(mb_chann, "mailbox_acquire_msgid failed");
	453	goto msg_id_failed;
	454	}
	455	header->id = ret;
	456
	457	MB_DBG(mb_chann, "opcode 0x%x size %d id 0x%x",
	458	header->opcode, header->total_size, header->id);
	459
	460	ret = mailbox_send_msg(mb_chann, mb_msg);
	461	if (ret) {
	462	MB_DBG(mb_chann, "Error in mailbox send msg, ret %d", ret);
	463	goto release_id;
	464	}
	465
	466	return 0;
	467
	468	release_id:
	469	mailbox_release_msgid(mb_chann, header->id);
	470	msg_id_failed:
	471	kfree(mb_msg);
	472	return ret;
	473	}
	474
	475	struct mailbox_channel *
	476	xdna_mailbox_create_channel(struct mailbox *mb,
	477	const struct xdna_mailbox_chann_res *x2i,
	478	const struct xdna_mailbox_chann_res *i2x,
	479	u32 iohub_int_addr,
	480	int mb_irq)
	481	{
	482	struct mailbox_channel *mb_chann;
	483	int ret;
	484
	485	if (!is_power_of_2(x2i->rb_size) \|\| !is_power_of_2(i2x->rb_size)) {
	486	pr_err("Ring buf size must be power of 2");
	487	return NULL;
	488	}
	489
	490	mb_chann = kzalloc(sizeof(*mb_chann), GFP_KERNEL);
	491	if (!mb_chann)
	492	return NULL;
	493
	494	mb_chann->mb = mb;
	495	mb_chann->msix_irq = mb_irq;
	496	mb_chann->iohub_int_addr = iohub_int_addr;
	497	memcpy(&mb_chann->res[CHAN_RES_X2I], x2i, sizeof(*x2i));
	498	memcpy(&mb_chann->res[CHAN_RES_I2X], i2x, sizeof(*i2x));
	499
	500	spin_lock_init(&mb_chann->chan_idr_lock);
	501	idr_init(&mb_chann->chan_idr);
	502	mb_chann->x2i_tail = mailbox_get_tailptr(mb_chann, CHAN_RES_X2I);
	503	mb_chann->i2x_head = mailbox_get_headptr(mb_chann, CHAN_RES_I2X);
	504
	505	INIT_WORK(&mb_chann->rx_work, mailbox_rx_worker);
	506	mb_chann->work_q = create_singlethread_workqueue(MAILBOX_NAME);
	507	if (!mb_chann->work_q) {
	508	MB_ERR(mb_chann, "Create workqueue failed");
	509	goto free_and_out;
	510	}
	511
	512	/* Everything look good. Time to enable irq handler */
	513	ret = request_irq(mb_irq, mailbox_irq_handler, 0, MAILBOX_NAME, mb_chann);
	514	if (ret) {
	515	MB_ERR(mb_chann, "Failed to request irq %d ret %d", mb_irq, ret);
	516	goto destroy_wq;
	517	}
	518
	519	mb_chann->bad_state = false;
	520
	521	MB_DBG(mb_chann, "Mailbox channel created (irq: %d)", mb_chann->msix_irq);
	522	return mb_chann;
	523
	524	destroy_wq:
	525	destroy_workqueue(mb_chann->work_q);
	526	free_and_out:
	527	kfree(mb_chann);
	528	return NULL;
	529	}
	530
	531	int xdna_mailbox_destroy_channel(struct mailbox_channel *mb_chann)
	532	{
	533	if (!mb_chann)
	534	return 0;
	535
	536	MB_DBG(mb_chann, "IRQ disabled and RX work cancelled");
	537	free_irq(mb_chann->msix_irq, mb_chann);
	538	destroy_workqueue(mb_chann->work_q);
	539	/* We can clean up and release resources */
	540
	541	idr_for_each(&mb_chann->chan_idr, mailbox_release_msg, mb_chann);
	542	idr_destroy(&mb_chann->chan_idr);
	543
	544	MB_DBG(mb_chann, "Mailbox channel destroyed, irq: %d", mb_chann->msix_irq);
	545	kfree(mb_chann);
	546	return 0;
	547	}
	548
	549	void xdna_mailbox_stop_channel(struct mailbox_channel *mb_chann)
	550	{
	551	if (!mb_chann)
	552	return;
	553
	554	/* Disable an irq and wait. This might sleep. */
	555	disable_irq(mb_chann->msix_irq);
	556
	557	/* Cancel RX work and wait for it to finish */
	558	cancel_work_sync(&mb_chann->rx_work);
	559	MB_DBG(mb_chann, "IRQ disabled and RX work cancelled");
	560	}
	561
	562	struct mailbox xdnam_mailbox_create(struct drm_device ddev,
	563	const struct xdna_mailbox_res *res)
	564	{
	565	struct mailbox *mb;
	566
	567	mb = drmm_kzalloc(ddev, sizeof(*mb), GFP_KERNEL);
	568	if (!mb)
	569	return NULL;
	570	mb->dev = ddev->dev;
	571
	572	/* mailbox and ring buf base and size information */
	573	memcpy(&mb->res, res, sizeof(*res));
	574
	575	return mb;
	576	}

drivers/accel/amdxdna/amdxdna_mailbox.h+124−0

@@ -0,0 +1,124 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _AIE2_MAILBOX_H_
	7	#define _AIE2_MAILBOX_H_
	8
	9	struct mailbox;
	10	struct mailbox_channel;
	11
	12	/*
	13	* xdna_mailbox_msg - message struct
	14	*
	15	* @opcode: opcode for firmware
	16	* @handle: handle used for the notify callback
	17	* @notify_cb: callback function to notify the sender when there is response
	18	* @send_data: pointing to sending data
	19	* @send_size: size of the sending data
	20	*
	21	* The mailbox will split the sending data in to multiple firmware message if
	22	* the size of the data is too big. This is transparent to the sender. The
	23	* sender will receive one notification.
	24	*/
	25	struct xdna_mailbox_msg {
	26	u32 opcode;
	27	void *handle;
	28	int (notify_cb)(void handle, const u32 *data, size_t size);
	29	u8 *send_data;
	30	size_t send_size;
	31	};
	32
	33	/*
	34	* xdna_mailbox_res - mailbox hardware resource
	35	*
	36	* @ringbuf_base: ring buffer base address
	37	* @ringbuf_size: ring buffer size
	38	* @mbox_base: mailbox base address
	39	* @mbox_size: mailbox size
	40	*/
	41	struct xdna_mailbox_res {
	42	u64 ringbuf_base;
	43	size_t ringbuf_size;
	44	u64 mbox_base;
	45	size_t mbox_size;
	46	const char *name;
	47	};
	48
	49	/*
	50	* xdna_mailbox_chann_res - resources
	51	*
	52	* @rb_start_addr: ring buffer start address
	53	* @rb_size: ring buffer size
	54	* @mb_head_ptr_reg: mailbox head pointer register
	55	* @mb_tail_ptr_reg: mailbox tail pointer register
	56	*/
	57	struct xdna_mailbox_chann_res {
	58	u32 rb_start_addr;
	59	u32 rb_size;
	60	u32 mb_head_ptr_reg;
	61	u32 mb_tail_ptr_reg;
	62	};
	63
	64	/*
	65	* xdna_mailbox_create() -- create mailbox subsystem and initialize
	66	*
	67	* @ddev: device pointer
	68	* @res: SRAM and mailbox resources
	69	*
	70	* Return: If success, return a handle of mailbox subsystem.
	71	* Otherwise, return NULL pointer.
	72	*/
	73	struct mailbox xdnam_mailbox_create(struct drm_device ddev,
	74	const struct xdna_mailbox_res *res);
	75
	76	/*
	77	* xdna_mailbox_create_channel() -- Create a mailbox channel instance
	78	*
	79	* @mailbox: the handle return from xdna_mailbox_create()
	80	* @x2i: host to firmware mailbox resources
	81	* @i2x: firmware to host mailbox resources
	82	* @xdna_mailbox_intr_reg: register addr of MSI-X interrupt
	83	* @mb_irq: Linux IRQ number associated with mailbox MSI-X interrupt vector index
	84	*
	85	* Return: If success, return a handle of mailbox channel. Otherwise, return NULL.
	86	*/
	87	struct mailbox_channel *
	88	xdna_mailbox_create_channel(struct mailbox *mailbox,
	89	const struct xdna_mailbox_chann_res *x2i,
	90	const struct xdna_mailbox_chann_res *i2x,
	91	u32 xdna_mailbox_intr_reg,
	92	int mb_irq);
	93
	94	/*
	95	* xdna_mailbox_destroy_channel() -- destroy mailbox channel
	96	*
	97	* @mailbox_chann: the handle return from xdna_mailbox_create_channel()
	98	*
	99	* Return: if success, return 0. otherwise return error code
	100	*/
	101	int xdna_mailbox_destroy_channel(struct mailbox_channel *mailbox_chann);
	102
	103	/*
	104	* xdna_mailbox_stop_channel() -- stop mailbox channel
	105	*
	106	* @mailbox_chann: the handle return from xdna_mailbox_create_channel()
	107	*
	108	* Return: if success, return 0. otherwise return error code
	109	*/
	110	void xdna_mailbox_stop_channel(struct mailbox_channel *mailbox_chann);
	111
	112	/*
	113	* xdna_mailbox_send_msg() -- Send a message
	114	*
	115	* @mailbox_chann: Mailbox channel handle
	116	* @msg: message struct for message information
	117	* @tx_timeout: the timeout value for sending the message in ms.
	118	*
	119	* Return: If success return 0, otherwise, return error code
	120	*/
	121	int xdna_mailbox_send_msg(struct mailbox_channel *mailbox_chann,
	122	const struct xdna_mailbox_msg *msg, u64 tx_timeout);
	123
	124	#endif /* _AIE2_MAILBOX_ */

drivers/accel/amdxdna/amdxdna_mailbox_helper.c+61−0

@@ -0,0 +1,61 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/drm_print.h>
	9	#include <drm/drm_gem.h>
	10	#include <drm/drm_gem_shmem_helper.h>
	11	#include <drm/gpu_scheduler.h>
	12	#include <linux/completion.h>
	13
	14	#include "amdxdna_gem.h"
	15	#include "amdxdna_mailbox.h"
	16	#include "amdxdna_mailbox_helper.h"
	17	#include "amdxdna_pci_drv.h"
	18
	19	int xdna_msg_cb(void handle, const u32 data, size_t size)
	20	{
	21	struct xdna_notify *cb_arg = handle;
	22	int ret;
	23
	24	if (unlikely(!data))
	25	goto out;
	26
	27	if (unlikely(cb_arg->size != size)) {
	28	cb_arg->error = -EINVAL;
	29	goto out;
	30	}
	31
	32	print_hex_dump_debug("resp data: ", DUMP_PREFIX_OFFSET,
	33	16, 4, data, cb_arg->size, true);
	34	memcpy(cb_arg->data, data, cb_arg->size);
	35	out:
	36	ret = cb_arg->error;
	37	complete(&cb_arg->comp);
	38	return ret;
	39	}
	40
	41	int xdna_send_msg_wait(struct amdxdna_dev xdna, struct mailbox_channel chann,
	42	struct xdna_mailbox_msg *msg)
	43	{
	44	struct xdna_notify *hdl = msg->handle;
	45	int ret;
	46
	47	ret = xdna_mailbox_send_msg(chann, msg, TX_TIMEOUT);
	48	if (ret) {
	49	XDNA_ERR(xdna, "Send message failed, ret %d", ret);
	50	return ret;
	51	}
	52
	53	ret = wait_for_completion_timeout(&hdl->comp,
	54	msecs_to_jiffies(RX_TIMEOUT));
	55	if (!ret) {
	56	XDNA_ERR(xdna, "Wait for completion timeout");
	57	return -ETIME;
	58	}
	59
	60	return hdl->error;
	61	}

drivers/accel/amdxdna/amdxdna_mailbox_helper.h+42−0

@@ -0,0 +1,42 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _AMDXDNA_MAILBOX_HELPER_H
	7	#define _AMDXDNA_MAILBOX_HELPER_H
	8
	9	#define TX_TIMEOUT 2000 /* milliseconds */
	10	#define RX_TIMEOUT 5000 /* milliseconds */
	11
	12	struct amdxdna_dev;
	13
	14	struct xdna_notify {
	15	struct completion comp;
	16	u32 *data;
	17	size_t size;
	18	int error;
	19	};
	20
	21	#define DECLARE_XDNA_MSG_COMMON(name, op, status) \
	22	struct name##_req req = { 0 }; \
	23	struct name##_resp resp = { status }; \
	24	struct xdna_notify hdl = { \
	25	.error = 0, \
	26	.data = (u32 *)&resp, \
	27	.size = sizeof(resp), \
	28	.comp = COMPLETION_INITIALIZER_ONSTACK(hdl.comp), \
	29	}; \
	30	struct xdna_mailbox_msg msg = { \
	31	.send_data = (u8 *)&req, \
	32	.send_size = sizeof(req), \
	33	.handle = &hdl, \
	34	.opcode = op, \
	35	.notify_cb = xdna_msg_cb, \
	36	}
	37
	38	int xdna_msg_cb(void handle, const u32 data, size_t size);
	39	int xdna_send_msg_wait(struct amdxdna_dev xdna, struct mailbox_channel chann,
	40	struct xdna_mailbox_msg *msg);
	41
	42	#endif /* _AMDXDNA_MAILBOX_HELPER_H */

drivers/accel/amdxdna/amdxdna_pci_drv.c+409−0

@@ -0,0 +1,409 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_accel.h>
	8	#include <drm/drm_drv.h>
	9	#include <drm/drm_gem.h>
	10	#include <drm/drm_gem_shmem_helper.h>
	11	#include <drm/drm_ioctl.h>
	12	#include <drm/drm_managed.h>
	13	#include <drm/gpu_scheduler.h>
	14	#include <linux/iommu.h>
	15	#include <linux/pci.h>
	16	#include <linux/pm_runtime.h>
	17
	18	#include "amdxdna_ctx.h"
	19	#include "amdxdna_gem.h"
	20	#include "amdxdna_pci_drv.h"
	21
	22	#define AMDXDNA_AUTOSUSPEND_DELAY 5000 /* milliseconds */
	23
	24	/*
	25	* Bind the driver base on (vendor_id, device_id) pair and later use the
	26	* (device_id, rev_id) pair as a key to select the devices. The devices with
	27	* same device_id have very similar interface to host driver.
	28	*/
	29	static const struct pci_device_id pci_ids[] = {
	30	{ PCI_DEVICE(PCI_VENDOR_ID_AMD, 0x1502) },
	31	{ PCI_DEVICE(PCI_VENDOR_ID_AMD, 0x17f0) },
	32	{0}
	33	};
	34
	35	MODULE_DEVICE_TABLE(pci, pci_ids);
	36
	37	static const struct amdxdna_device_id amdxdna_ids[] = {
	38	{ 0x1502, 0x0, &dev_npu1_info },
	39	{ 0x17f0, 0x0, &dev_npu2_info },
	40	{ 0x17f0, 0x10, &dev_npu4_info },
	41	{ 0x17f0, 0x11, &dev_npu5_info },
	42	{0}
	43	};
	44
	45	static int amdxdna_drm_open(struct drm_device ddev, struct drm_file filp)
	46	{
	47	struct amdxdna_dev *xdna = to_xdna_dev(ddev);
	48	struct amdxdna_client *client;
	49	int ret;
	50
	51	ret = pm_runtime_resume_and_get(ddev->dev);
	52	if (ret) {
	53	XDNA_ERR(xdna, "Failed to get rpm, ret %d", ret);
	54	return ret;
	55	}
	56
	57	client = kzalloc(sizeof(*client), GFP_KERNEL);
	58	if (!client) {
	59	ret = -ENOMEM;
	60	goto put_rpm;
	61	}
	62
	63	client->pid = pid_nr(filp->pid);
	64	client->xdna = xdna;
	65
	66	client->sva = iommu_sva_bind_device(xdna->ddev.dev, current->mm);
	67	if (IS_ERR(client->sva)) {
	68	ret = PTR_ERR(client->sva);
	69	XDNA_ERR(xdna, "SVA bind device failed, ret %d", ret);
	70	goto failed;
	71	}
	72	client->pasid = iommu_sva_get_pasid(client->sva);
	73	if (client->pasid == IOMMU_PASID_INVALID) {
	74	XDNA_ERR(xdna, "SVA get pasid failed");
	75	ret = -ENODEV;
	76	goto unbind_sva;
	77	}
	78	mutex_init(&client->hwctx_lock);
	79	init_srcu_struct(&client->hwctx_srcu);
	80	idr_init_base(&client->hwctx_idr, AMDXDNA_INVALID_CTX_HANDLE + 1);
	81	mutex_init(&client->mm_lock);
	82
	83	mutex_lock(&xdna->dev_lock);
	84	list_add_tail(&client->node, &xdna->client_list);
	85	mutex_unlock(&xdna->dev_lock);
	86
	87	filp->driver_priv = client;
	88	client->filp = filp;
	89
	90	XDNA_DBG(xdna, "pid %d opened", client->pid);
	91	return 0;
	92
	93	unbind_sva:
	94	iommu_sva_unbind_device(client->sva);
	95	failed:
	96	kfree(client);
	97	put_rpm:
	98	pm_runtime_mark_last_busy(ddev->dev);
	99	pm_runtime_put_autosuspend(ddev->dev);
	100
	101	return ret;
	102	}
	103
	104	static void amdxdna_drm_close(struct drm_device ddev, struct drm_file filp)
	105	{
	106	struct amdxdna_client *client = filp->driver_priv;
	107	struct amdxdna_dev *xdna = to_xdna_dev(ddev);
	108
	109	XDNA_DBG(xdna, "closing pid %d", client->pid);
	110
	111	idr_destroy(&client->hwctx_idr);
	112	cleanup_srcu_struct(&client->hwctx_srcu);
	113	mutex_destroy(&client->hwctx_lock);
	114	mutex_destroy(&client->mm_lock);
	115	if (client->dev_heap)
	116	drm_gem_object_put(to_gobj(client->dev_heap));
	117
	118	iommu_sva_unbind_device(client->sva);
	119
	120	XDNA_DBG(xdna, "pid %d closed", client->pid);
	121	kfree(client);
	122	pm_runtime_mark_last_busy(ddev->dev);
	123	pm_runtime_put_autosuspend(ddev->dev);
	124	}
	125
	126	static int amdxdna_flush(struct file *f, fl_owner_t id)
	127	{
	128	struct drm_file *filp = f->private_data;
	129	struct amdxdna_client *client = filp->driver_priv;
	130	struct amdxdna_dev *xdna = client->xdna;
	131	int idx;
	132
	133	XDNA_DBG(xdna, "PID %d flushing...", client->pid);
	134	if (!drm_dev_enter(&xdna->ddev, &idx))
	135	return 0;
	136
	137	mutex_lock(&xdna->dev_lock);
	138	list_del_init(&client->node);
	139	mutex_unlock(&xdna->dev_lock);
	140	amdxdna_hwctx_remove_all(client);
	141
	142	drm_dev_exit(idx);
	143	return 0;
	144	}
	145
	146	static int amdxdna_drm_get_info_ioctl(struct drm_device dev, void data, struct drm_file *filp)
	147	{
	148	struct amdxdna_client *client = filp->driver_priv;
	149	struct amdxdna_dev *xdna = to_xdna_dev(dev);
	150	struct amdxdna_drm_get_info *args = data;
	151	int ret;
	152
	153	if (!xdna->dev_info->ops->get_aie_info)
	154	return -EOPNOTSUPP;
	155
	156	XDNA_DBG(xdna, "Request parameter %u", args->param);
	157	mutex_lock(&xdna->dev_lock);
	158	ret = xdna->dev_info->ops->get_aie_info(client, args);
	159	mutex_unlock(&xdna->dev_lock);
	160	return ret;
	161	}
	162
	163	static const struct drm_ioctl_desc amdxdna_drm_ioctls[] = {
	164	/* Context */
	165	DRM_IOCTL_DEF_DRV(AMDXDNA_CREATE_HWCTX, amdxdna_drm_create_hwctx_ioctl, 0),
	166	DRM_IOCTL_DEF_DRV(AMDXDNA_DESTROY_HWCTX, amdxdna_drm_destroy_hwctx_ioctl, 0),
	167	DRM_IOCTL_DEF_DRV(AMDXDNA_CONFIG_HWCTX, amdxdna_drm_config_hwctx_ioctl, 0),
	168	/* BO */
	169	DRM_IOCTL_DEF_DRV(AMDXDNA_CREATE_BO, amdxdna_drm_create_bo_ioctl, 0),
	170	DRM_IOCTL_DEF_DRV(AMDXDNA_GET_BO_INFO, amdxdna_drm_get_bo_info_ioctl, 0),
	171	DRM_IOCTL_DEF_DRV(AMDXDNA_SYNC_BO, amdxdna_drm_sync_bo_ioctl, 0),
	172	/* Execution */
	173	DRM_IOCTL_DEF_DRV(AMDXDNA_EXEC_CMD, amdxdna_drm_submit_cmd_ioctl, 0),
	174	/* AIE hardware */
	175	DRM_IOCTL_DEF_DRV(AMDXDNA_GET_INFO, amdxdna_drm_get_info_ioctl, 0),
	176	};
	177
	178	static const struct file_operations amdxdna_fops = {
	179	.owner = THIS_MODULE,
	180	.open = accel_open,
	181	.release = drm_release,
	182	.flush = amdxdna_flush,
	183	.unlocked_ioctl = drm_ioctl,
	184	.compat_ioctl = drm_compat_ioctl,
	185	.poll = drm_poll,
	186	.read = drm_read,
	187	.llseek = noop_llseek,
	188	.mmap = drm_gem_mmap,
	189	.fop_flags = FOP_UNSIGNED_OFFSET,
	190	};
	191
	192	const struct drm_driver amdxdna_drm_drv = {
	193	.driver_features = DRIVER_GEM \| DRIVER_COMPUTE_ACCEL \|
	194	DRIVER_SYNCOBJ \| DRIVER_SYNCOBJ_TIMELINE,
	195	.fops = &amdxdna_fops,
	196	.name = "amdxdna_accel_driver",
	197	.desc = "AMD XDNA DRM implementation",
	198	.open = amdxdna_drm_open,
	199	.postclose = amdxdna_drm_close,
	200	.ioctls = amdxdna_drm_ioctls,
	201	.num_ioctls = ARRAY_SIZE(amdxdna_drm_ioctls),
	202
	203	.gem_create_object = amdxdna_gem_create_object_cb,
	204	};
	205
	206	static const struct amdxdna_dev_info *
	207	amdxdna_get_dev_info(struct pci_dev *pdev)
	208	{
	209	int i;
	210
	211	for (i = 0; i < ARRAY_SIZE(amdxdna_ids); i++) {
	212	if (pdev->device == amdxdna_ids[i].device &&
	213	pdev->revision == amdxdna_ids[i].revision)
	214	return amdxdna_ids[i].dev_info;
	215	}
	216	return NULL;
	217	}
	218
	219	static int amdxdna_probe(struct pci_dev pdev, const struct pci_device_id id)
	220	{
	221	struct device *dev = &pdev->dev;
	222	struct amdxdna_dev *xdna;
	223	int ret;
	224
	225	xdna = devm_drm_dev_alloc(dev, &amdxdna_drm_drv, typeof(*xdna), ddev);
	226	if (IS_ERR(xdna))
	227	return PTR_ERR(xdna);
	228
	229	xdna->dev_info = amdxdna_get_dev_info(pdev);
	230	if (!xdna->dev_info)
	231	return -ENODEV;
	232
	233	drmm_mutex_init(&xdna->ddev, &xdna->dev_lock);
	234	init_rwsem(&xdna->notifier_lock);
	235	INIT_LIST_HEAD(&xdna->client_list);
	236	pci_set_drvdata(pdev, xdna);
	237
	238	if (IS_ENABLED(CONFIG_LOCKDEP)) {
	239	fs_reclaim_acquire(GFP_KERNEL);
	240	might_lock(&xdna->notifier_lock);
	241	fs_reclaim_release(GFP_KERNEL);
	242	}
	243
	244	mutex_lock(&xdna->dev_lock);
	245	ret = xdna->dev_info->ops->init(xdna);
	246	mutex_unlock(&xdna->dev_lock);
	247	if (ret) {
	248	XDNA_ERR(xdna, "Hardware init failed, ret %d", ret);
	249	return ret;
	250	}
	251
	252	ret = amdxdna_sysfs_init(xdna);
	253	if (ret) {
	254	XDNA_ERR(xdna, "Create amdxdna attrs failed: %d", ret);
	255	goto failed_dev_fini;
	256	}
	257
	258	pm_runtime_set_autosuspend_delay(dev, AMDXDNA_AUTOSUSPEND_DELAY);
	259	pm_runtime_use_autosuspend(dev);
	260	pm_runtime_allow(dev);
	261
	262	ret = drm_dev_register(&xdna->ddev, 0);
	263	if (ret) {
	264	XDNA_ERR(xdna, "DRM register failed, ret %d", ret);
	265	pm_runtime_forbid(dev);
	266	goto failed_sysfs_fini;
	267	}
	268
	269	pm_runtime_mark_last_busy(dev);
	270	pm_runtime_put_autosuspend(dev);
	271	return 0;
	272
	273	failed_sysfs_fini:
	274	amdxdna_sysfs_fini(xdna);
	275	failed_dev_fini:
	276	mutex_lock(&xdna->dev_lock);
	277	xdna->dev_info->ops->fini(xdna);
	278	mutex_unlock(&xdna->dev_lock);
	279	return ret;
	280	}
	281
	282	static void amdxdna_remove(struct pci_dev *pdev)
	283	{
	284	struct amdxdna_dev *xdna = pci_get_drvdata(pdev);
	285	struct device *dev = &pdev->dev;
	286	struct amdxdna_client *client;
	287
	288	pm_runtime_get_noresume(dev);
	289	pm_runtime_forbid(dev);
	290
	291	drm_dev_unplug(&xdna->ddev);
	292	amdxdna_sysfs_fini(xdna);
	293
	294	mutex_lock(&xdna->dev_lock);
	295	client = list_first_entry_or_null(&xdna->client_list,
	296	struct amdxdna_client, node);
	297	while (client) {
	298	list_del_init(&client->node);
	299	mutex_unlock(&xdna->dev_lock);
	300
	301	amdxdna_hwctx_remove_all(client);
	302
	303	mutex_lock(&xdna->dev_lock);
	304	client = list_first_entry_or_null(&xdna->client_list,
	305	struct amdxdna_client, node);
	306	}
	307
	308	xdna->dev_info->ops->fini(xdna);
	309	mutex_unlock(&xdna->dev_lock);
	310	}
	311
	312	static int amdxdna_dev_suspend_nolock(struct amdxdna_dev *xdna)
	313	{
	314	if (xdna->dev_info->ops->suspend)
	315	xdna->dev_info->ops->suspend(xdna);
	316
	317	return 0;
	318	}
	319
	320	static int amdxdna_dev_resume_nolock(struct amdxdna_dev *xdna)
	321	{
	322	if (xdna->dev_info->ops->resume)
	323	return xdna->dev_info->ops->resume(xdna);
	324
	325	return 0;
	326	}
	327
	328	static int amdxdna_pmops_suspend(struct device *dev)
	329	{
	330	struct amdxdna_dev *xdna = pci_get_drvdata(to_pci_dev(dev));
	331	struct amdxdna_client *client;
	332
	333	mutex_lock(&xdna->dev_lock);
	334	list_for_each_entry(client, &xdna->client_list, node)
	335	amdxdna_hwctx_suspend(client);
	336
	337	amdxdna_dev_suspend_nolock(xdna);
	338	mutex_unlock(&xdna->dev_lock);
	339
	340	return 0;
	341	}
	342
	343	static int amdxdna_pmops_resume(struct device *dev)
	344	{
	345	struct amdxdna_dev *xdna = pci_get_drvdata(to_pci_dev(dev));
	346	struct amdxdna_client *client;
	347	int ret;
	348
	349	XDNA_INFO(xdna, "firmware resuming...");
	350	mutex_lock(&xdna->dev_lock);
	351	ret = amdxdna_dev_resume_nolock(xdna);
	352	if (ret) {
	353	XDNA_ERR(xdna, "resume NPU firmware failed");
	354	mutex_unlock(&xdna->dev_lock);
	355	return ret;
	356	}
	357
	358	XDNA_INFO(xdna, "hardware context resuming...");
	359	list_for_each_entry(client, &xdna->client_list, node)
	360	amdxdna_hwctx_resume(client);
	361	mutex_unlock(&xdna->dev_lock);
	362
	363	return 0;
	364	}
	365
	366	static int amdxdna_rpmops_suspend(struct device *dev)
	367	{
	368	struct amdxdna_dev *xdna = pci_get_drvdata(to_pci_dev(dev));
	369	int ret;
	370
	371	mutex_lock(&xdna->dev_lock);
	372	ret = amdxdna_dev_suspend_nolock(xdna);
	373	mutex_unlock(&xdna->dev_lock);
	374
	375	XDNA_DBG(xdna, "Runtime suspend done ret: %d", ret);
	376	return ret;
	377	}
	378
	379	static int amdxdna_rpmops_resume(struct device *dev)
	380	{
	381	struct amdxdna_dev *xdna = pci_get_drvdata(to_pci_dev(dev));
	382	int ret;
	383
	384	mutex_lock(&xdna->dev_lock);
	385	ret = amdxdna_dev_resume_nolock(xdna);
	386	mutex_unlock(&xdna->dev_lock);
	387
	388	XDNA_DBG(xdna, "Runtime resume done ret: %d", ret);
	389	return ret;
	390	}
	391
	392	static const struct dev_pm_ops amdxdna_pm_ops = {
	393	SET_SYSTEM_SLEEP_PM_OPS(amdxdna_pmops_suspend, amdxdna_pmops_resume)
	394	SET_RUNTIME_PM_OPS(amdxdna_rpmops_suspend, amdxdna_rpmops_resume, NULL)
	395	};
	396
	397	static struct pci_driver amdxdna_pci_driver = {
	398	.name = KBUILD_MODNAME,
	399	.id_table = pci_ids,
	400	.probe = amdxdna_probe,
	401	.remove = amdxdna_remove,
	402	.driver.pm = &amdxdna_pm_ops,
	403	};
	404
	405	module_pci_driver(amdxdna_pci_driver);
	406
	407	MODULE_LICENSE("GPL");
	408	MODULE_AUTHOR("XRT Team <runtimeca39d@amd.com>");
	409	MODULE_DESCRIPTION("amdxdna driver");

drivers/accel/amdxdna/amdxdna_pci_drv.h+123−0

@@ -0,0 +1,123 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _AMDXDNA_PCI_DRV_H_
	7	#define _AMDXDNA_PCI_DRV_H_
	8
	9	#define XDNA_INFO(xdna, fmt, args...) drm_info(&(xdna)->ddev, fmt, ##args)
	10	#define XDNA_WARN(xdna, fmt, args...) drm_warn(&(xdna)->ddev, "%s: "fmt, __func__, ##args)
	11	#define XDNA_ERR(xdna, fmt, args...) drm_err(&(xdna)->ddev, "%s: "fmt, __func__, ##args)
	12	#define XDNA_DBG(xdna, fmt, args...) drm_dbg(&(xdna)->ddev, fmt, ##args)
	13	#define XDNA_INFO_ONCE(xdna, fmt, args...) drm_info_once(&(xdna)->ddev, fmt, ##args)
	14
	15	#define to_xdna_dev(drm_dev) \
	16	((struct amdxdna_dev *)container_of(drm_dev, struct amdxdna_dev, ddev))
	17
	18	extern const struct drm_driver amdxdna_drm_drv;
	19
	20	struct amdxdna_client;
	21	struct amdxdna_dev;
	22	struct amdxdna_drm_get_info;
	23	struct amdxdna_gem_obj;
	24	struct amdxdna_hwctx;
	25	struct amdxdna_sched_job;
	26
	27	/*
	28	* struct amdxdna_dev_ops - Device hardware operation callbacks
	29	*/
	30	struct amdxdna_dev_ops {
	31	int (init)(struct amdxdna_dev xdna);
	32	void (fini)(struct amdxdna_dev xdna);
	33	int (resume)(struct amdxdna_dev xdna);
	34	void (suspend)(struct amdxdna_dev xdna);
	35	int (hwctx_init)(struct amdxdna_hwctx hwctx);
	36	void (hwctx_fini)(struct amdxdna_hwctx hwctx);
	37	int (hwctx_config)(struct amdxdna_hwctx hwctx, u32 type, u64 value, void *buf, u32 size);
	38	void (hmm_invalidate)(struct amdxdna_gem_obj abo, unsigned long cur_seq);
	39	void (hwctx_suspend)(struct amdxdna_hwctx hwctx);
	40	void (hwctx_resume)(struct amdxdna_hwctx hwctx);
	41	int (cmd_submit)(struct amdxdna_hwctx hwctx, struct amdxdna_sched_job job, u64 seq);
	42	int (get_aie_info)(struct amdxdna_client client, struct amdxdna_drm_get_info *args);
	43	};
	44
	45	/*
	46	* struct amdxdna_dev_info - Device hardware information
	47	* Record device static information, like reg, mbox, PSP, SMU bar index
	48	*/
	49	struct amdxdna_dev_info {
	50	int reg_bar;
	51	int mbox_bar;
	52	int sram_bar;
	53	int psp_bar;
	54	int smu_bar;
	55	int device_type;
	56	int first_col;
	57	u32 dev_mem_buf_shift;
	58	u64 dev_mem_base;
	59	size_t dev_mem_size;
	60	char *vbnv;
	61	const struct amdxdna_dev_priv *dev_priv;
	62	const struct amdxdna_dev_ops *ops;
	63	};
	64
	65	struct amdxdna_fw_ver {
	66	u32 major;
	67	u32 minor;
	68	u32 sub;
	69	u32 build;
	70	};
	71
	72	struct amdxdna_dev {
	73	struct drm_device ddev;
	74	struct amdxdna_dev_hdl *dev_handle;
	75	const struct amdxdna_dev_info *dev_info;
	76	void *xrs_hdl;
	77
	78	struct mutex dev_lock; /* per device lock */
	79	struct list_head client_list;
	80	struct amdxdna_fw_ver fw_ver;
	81	struct rw_semaphore notifier_lock; /* for mmu notifier*/
	82	};
	83
	84	/*
	85	* struct amdxdna_device_id - PCI device info
	86	*/
	87	struct amdxdna_device_id {
	88	unsigned short device;
	89	u8 revision;
	90	const struct amdxdna_dev_info *dev_info;
	91	};
	92
	93	/*
	94	* struct amdxdna_client - amdxdna client
	95	* A per fd data structure for managing context and other user process stuffs.
	96	*/
	97	struct amdxdna_client {
	98	struct list_head node;
	99	pid_t pid;
	100	struct mutex hwctx_lock; /* protect hwctx */
	101	/* do NOT wait this srcu when hwctx_lock is held */
	102	struct srcu_struct hwctx_srcu;
	103	struct idr hwctx_idr;
	104	struct amdxdna_dev *xdna;
	105	struct drm_file *filp;
	106
	107	struct mutex mm_lock; /* protect memory related */
	108	struct amdxdna_gem_obj *dev_heap;
	109
	110	struct iommu_sva *sva;
	111	int pasid;
	112	};
	113
	114	/* Add device info below */
	115	extern const struct amdxdna_dev_info dev_npu1_info;
	116	extern const struct amdxdna_dev_info dev_npu2_info;
	117	extern const struct amdxdna_dev_info dev_npu4_info;
	118	extern const struct amdxdna_dev_info dev_npu5_info;
	119
	120	int amdxdna_sysfs_init(struct amdxdna_dev *xdna);
	121	void amdxdna_sysfs_fini(struct amdxdna_dev *xdna);
	122
	123	#endif /* _AMDXDNA_PCI_DRV_H_ */

drivers/accel/amdxdna/amdxdna_sysfs.c+67−0

@@ -0,0 +1,67 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/drm_gem_shmem_helper.h>
	9	#include <drm/drm_print.h>
	10	#include <drm/gpu_scheduler.h>
	11	#include <linux/types.h>
	12
	13	#include "amdxdna_gem.h"
	14	#include "amdxdna_pci_drv.h"
	15
	16	static ssize_t vbnv_show(struct device dev, struct device_attribute attr, char *buf)
	17	{
	18	struct amdxdna_dev *xdna = dev_get_drvdata(dev);
	19
	20	return sprintf(buf, "%s\n", xdna->dev_info->vbnv);
	21	}
	22	static DEVICE_ATTR_RO(vbnv);
	23
	24	static ssize_t device_type_show(struct device dev, struct device_attribute attr, char *buf)
	25	{
	26	struct amdxdna_dev *xdna = dev_get_drvdata(dev);
	27
	28	return sprintf(buf, "%d\n", xdna->dev_info->device_type);
	29	}
	30	static DEVICE_ATTR_RO(device_type);
	31
	32	static ssize_t fw_version_show(struct device dev, struct device_attribute attr, char *buf)
	33	{
	34	struct amdxdna_dev *xdna = dev_get_drvdata(dev);
	35
	36	return sprintf(buf, "%d.%d.%d.%d\n", xdna->fw_ver.major,
	37	xdna->fw_ver.minor, xdna->fw_ver.sub,
	38	xdna->fw_ver.build);
	39	}
	40	static DEVICE_ATTR_RO(fw_version);
	41
	42	static struct attribute *amdxdna_attrs[] = {
	43	&dev_attr_device_type.attr,
	44	&dev_attr_vbnv.attr,
	45	&dev_attr_fw_version.attr,
	46	NULL,
	47	};
	48
	49	static struct attribute_group amdxdna_attr_group = {
	50	.attrs = amdxdna_attrs,
	51	};
	52
	53	int amdxdna_sysfs_init(struct amdxdna_dev *xdna)
	54	{
	55	int ret;
	56
	57	ret = sysfs_create_group(&xdna->ddev.dev->kobj, &amdxdna_attr_group);
	58	if (ret)
	59	XDNA_ERR(xdna, "Create attr group failed");
	60
	61	return ret;
	62	}
	63
	64	void amdxdna_sysfs_fini(struct amdxdna_dev *xdna)
	65	{
	66	sysfs_remove_group(&xdna->ddev.dev->kobj, &amdxdna_attr_group);
	67	}

drivers/accel/amdxdna/npu1_regs.c+101−0

@@ -0,0 +1,101 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/gpu_scheduler.h>
	9	#include <linux/sizes.h>
	10
	11	#include "aie2_pci.h"
	12	#include "amdxdna_mailbox.h"
	13	#include "amdxdna_pci_drv.h"
	14
	15	/* Address definition from NPU1 docs */
	16	#define MPNPU_PUB_SEC_INTR 0x3010090
	17	#define MPNPU_PUB_PWRMGMT_INTR 0x3010094
	18	#define MPNPU_PUB_SCRATCH2 0x30100A0
	19	#define MPNPU_PUB_SCRATCH3 0x30100A4
	20	#define MPNPU_PUB_SCRATCH4 0x30100A8
	21	#define MPNPU_PUB_SCRATCH5 0x30100AC
	22	#define MPNPU_PUB_SCRATCH6 0x30100B0
	23	#define MPNPU_PUB_SCRATCH7 0x30100B4
	24	#define MPNPU_PUB_SCRATCH9 0x30100BC
	25
	26	#define MPNPU_SRAM_X2I_MAILBOX_0 0x30A0000
	27	#define MPNPU_SRAM_X2I_MAILBOX_1 0x30A2000
	28	#define MPNPU_SRAM_I2X_MAILBOX_15 0x30BF000
	29
	30	#define MPNPU_APERTURE0_BASE 0x3000000
	31	#define MPNPU_APERTURE1_BASE 0x3080000
	32	#define MPNPU_APERTURE2_BASE 0x30C0000
	33
	34	/* PCIe BAR Index for NPU1 */
	35	#define NPU1_REG_BAR_INDEX 0
	36	#define NPU1_MBOX_BAR_INDEX 4
	37	#define NPU1_PSP_BAR_INDEX 0
	38	#define NPU1_SMU_BAR_INDEX 0
	39	#define NPU1_SRAM_BAR_INDEX 2
	40	/* Associated BARs and Apertures */
	41	#define NPU1_REG_BAR_BASE MPNPU_APERTURE0_BASE
	42	#define NPU1_MBOX_BAR_BASE MPNPU_APERTURE2_BASE
	43	#define NPU1_PSP_BAR_BASE MPNPU_APERTURE0_BASE
	44	#define NPU1_SMU_BAR_BASE MPNPU_APERTURE0_BASE
	45	#define NPU1_SRAM_BAR_BASE MPNPU_APERTURE1_BASE
	46
	47	#define NPU1_RT_CFG_TYPE_PDI_LOAD 2
	48	#define NPU1_RT_CFG_VAL_PDI_LOAD_MGMT 0
	49	#define NPU1_RT_CFG_VAL_PDI_LOAD_APP 1
	50
	51	#define NPU1_MPNPUCLK_FREQ_MAX 600
	52	#define NPU1_HCLK_FREQ_MAX 1024
	53
	54	const struct amdxdna_dev_priv npu1_dev_priv = {
	55	.fw_path = "amdnpu/1502_00/npu.sbin",
	56	.protocol_major = 0x5,
	57	.protocol_minor = 0x1,
	58	.rt_config = {NPU1_RT_CFG_TYPE_PDI_LOAD, NPU1_RT_CFG_VAL_PDI_LOAD_APP},
	59	.col_align = COL_ALIGN_NONE,
	60	.mbox_dev_addr = NPU1_MBOX_BAR_BASE,
	61	.mbox_size = 0, /* Use BAR size */
	62	.sram_dev_addr = NPU1_SRAM_BAR_BASE,
	63	.sram_offs = {
	64	DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU1_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
	65	DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU1_SRAM, MPNPU_SRAM_I2X_MAILBOX_15),
	66	},
	67	.psp_regs_off = {
	68	DEFINE_BAR_OFFSET(PSP_CMD_REG, NPU1_PSP, MPNPU_PUB_SCRATCH2),
	69	DEFINE_BAR_OFFSET(PSP_ARG0_REG, NPU1_PSP, MPNPU_PUB_SCRATCH3),
	70	DEFINE_BAR_OFFSET(PSP_ARG1_REG, NPU1_PSP, MPNPU_PUB_SCRATCH4),
	71	DEFINE_BAR_OFFSET(PSP_ARG2_REG, NPU1_PSP, MPNPU_PUB_SCRATCH9),
	72	DEFINE_BAR_OFFSET(PSP_INTR_REG, NPU1_PSP, MPNPU_PUB_SEC_INTR),
	73	DEFINE_BAR_OFFSET(PSP_STATUS_REG, NPU1_PSP, MPNPU_PUB_SCRATCH2),
	74	DEFINE_BAR_OFFSET(PSP_RESP_REG, NPU1_PSP, MPNPU_PUB_SCRATCH3),
	75	},
	76	.smu_regs_off = {
	77	DEFINE_BAR_OFFSET(SMU_CMD_REG, NPU1_SMU, MPNPU_PUB_SCRATCH5),
	78	DEFINE_BAR_OFFSET(SMU_ARG_REG, NPU1_SMU, MPNPU_PUB_SCRATCH7),
	79	DEFINE_BAR_OFFSET(SMU_INTR_REG, NPU1_SMU, MPNPU_PUB_PWRMGMT_INTR),
	80	DEFINE_BAR_OFFSET(SMU_RESP_REG, NPU1_SMU, MPNPU_PUB_SCRATCH6),
	81	DEFINE_BAR_OFFSET(SMU_OUT_REG, NPU1_SMU, MPNPU_PUB_SCRATCH7),
	82	},
	83	.smu_mpnpuclk_freq_max = NPU1_MPNPUCLK_FREQ_MAX,
	84	.smu_hclk_freq_max = NPU1_HCLK_FREQ_MAX,
	85	};
	86
	87	const struct amdxdna_dev_info dev_npu1_info = {
	88	.reg_bar = NPU1_REG_BAR_INDEX,
	89	.mbox_bar = NPU1_MBOX_BAR_INDEX,
	90	.sram_bar = NPU1_SRAM_BAR_INDEX,
	91	.psp_bar = NPU1_PSP_BAR_INDEX,
	92	.smu_bar = NPU1_SMU_BAR_INDEX,
	93	.first_col = 1,
	94	.dev_mem_buf_shift = 15, /* 32 KiB aligned */
	95	.dev_mem_base = AIE2_DEVM_BASE,
	96	.dev_mem_size = AIE2_DEVM_SIZE,
	97	.vbnv = "RyzenAI-npu1",
	98	.device_type = AMDXDNA_DEV_TYPE_KMQ,
	99	.dev_priv = &npu1_dev_priv,
	100	.ops = &aie2_ops,
	101	};

drivers/accel/amdxdna/npu2_regs.c+118−0

@@ -0,0 +1,118 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/gpu_scheduler.h>
	9	#include <linux/sizes.h>
	10
	11	#include "aie2_pci.h"
	12	#include "amdxdna_mailbox.h"
	13	#include "amdxdna_pci_drv.h"
	14
	15	/* NPU Public Registers on MpNPUAxiXbar (refer to Diag npu_registers.h) */
	16	#define MPNPU_PUB_SEC_INTR 0x3010060
	17	#define MPNPU_PUB_PWRMGMT_INTR 0x3010064
	18	#define MPNPU_PUB_SCRATCH0 0x301006C
	19	#define MPNPU_PUB_SCRATCH1 0x3010070
	20	#define MPNPU_PUB_SCRATCH2 0x3010074
	21	#define MPNPU_PUB_SCRATCH3 0x3010078
	22	#define MPNPU_PUB_SCRATCH4 0x301007C
	23	#define MPNPU_PUB_SCRATCH5 0x3010080
	24	#define MPNPU_PUB_SCRATCH6 0x3010084
	25	#define MPNPU_PUB_SCRATCH7 0x3010088
	26	#define MPNPU_PUB_SCRATCH8 0x301008C
	27	#define MPNPU_PUB_SCRATCH9 0x3010090
	28	#define MPNPU_PUB_SCRATCH10 0x3010094
	29	#define MPNPU_PUB_SCRATCH11 0x3010098
	30	#define MPNPU_PUB_SCRATCH12 0x301009C
	31	#define MPNPU_PUB_SCRATCH13 0x30100A0
	32	#define MPNPU_PUB_SCRATCH14 0x30100A4
	33	#define MPNPU_PUB_SCRATCH15 0x30100A8
	34	#define MP0_C2PMSG_73 0x3810A24
	35	#define MP0_C2PMSG_123 0x3810AEC
	36
	37	#define MP1_C2PMSG_0 0x3B10900
	38	#define MP1_C2PMSG_60 0x3B109F0
	39	#define MP1_C2PMSG_61 0x3B109F4
	40
	41	#define MPNPU_SRAM_X2I_MAILBOX_0 0x3600000
	42	#define MPNPU_SRAM_X2I_MAILBOX_15 0x361E000
	43	#define MPNPU_SRAM_X2I_MAILBOX_31 0x363E000
	44	#define MPNPU_SRAM_I2X_MAILBOX_31 0x363F000
	45
	46	#define MMNPU_APERTURE0_BASE 0x3000000
	47	#define MMNPU_APERTURE1_BASE 0x3600000
	48	#define MMNPU_APERTURE3_BASE 0x3810000
	49	#define MMNPU_APERTURE4_BASE 0x3B10000
	50
	51	/* PCIe BAR Index for NPU2 */
	52	#define NPU2_REG_BAR_INDEX 0
	53	#define NPU2_MBOX_BAR_INDEX 0
	54	#define NPU2_PSP_BAR_INDEX 4
	55	#define NPU2_SMU_BAR_INDEX 5
	56	#define NPU2_SRAM_BAR_INDEX 2
	57	/* Associated BARs and Apertures */
	58	#define NPU2_REG_BAR_BASE MMNPU_APERTURE0_BASE
	59	#define NPU2_MBOX_BAR_BASE MMNPU_APERTURE0_BASE
	60	#define NPU2_PSP_BAR_BASE MMNPU_APERTURE3_BASE
	61	#define NPU2_SMU_BAR_BASE MMNPU_APERTURE4_BASE
	62	#define NPU2_SRAM_BAR_BASE MMNPU_APERTURE1_BASE
	63
	64	#define NPU2_RT_CFG_TYPE_PDI_LOAD 5
	65	#define NPU2_RT_CFG_VAL_PDI_LOAD_MGMT 0
	66	#define NPU2_RT_CFG_VAL_PDI_LOAD_APP 1
	67
	68	#define NPU2_MPNPUCLK_FREQ_MAX 1267
	69	#define NPU2_HCLK_FREQ_MAX 1800
	70
	71	const struct amdxdna_dev_priv npu2_dev_priv = {
	72	.fw_path = "amdnpu/17f0_00/npu.sbin",
	73	.protocol_major = 0x6,
	74	.protocol_minor = 0x1,
	75	.rt_config = {NPU2_RT_CFG_TYPE_PDI_LOAD, NPU2_RT_CFG_VAL_PDI_LOAD_APP},
	76	.col_align = COL_ALIGN_NATURE,
	77	.mbox_dev_addr = NPU2_MBOX_BAR_BASE,
	78	.mbox_size = 0, /* Use BAR size */
	79	.sram_dev_addr = NPU2_SRAM_BAR_BASE,
	80	.sram_offs = {
	81	DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU2_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
	82	DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU2_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),
	83	},
	84	.psp_regs_off = {
	85	DEFINE_BAR_OFFSET(PSP_CMD_REG, NPU2_PSP, MP0_C2PMSG_123),
	86	DEFINE_BAR_OFFSET(PSP_ARG0_REG, NPU2_REG, MPNPU_PUB_SCRATCH3),
	87	DEFINE_BAR_OFFSET(PSP_ARG1_REG, NPU2_REG, MPNPU_PUB_SCRATCH4),
	88	DEFINE_BAR_OFFSET(PSP_ARG2_REG, NPU2_REG, MPNPU_PUB_SCRATCH9),
	89	DEFINE_BAR_OFFSET(PSP_INTR_REG, NPU2_PSP, MP0_C2PMSG_73),
	90	DEFINE_BAR_OFFSET(PSP_STATUS_REG, NPU2_PSP, MP0_C2PMSG_123),
	91	DEFINE_BAR_OFFSET(PSP_RESP_REG, NPU2_REG, MPNPU_PUB_SCRATCH3),
	92	},
	93	.smu_regs_off = {
	94	DEFINE_BAR_OFFSET(SMU_CMD_REG, NPU2_SMU, MP1_C2PMSG_0),
	95	DEFINE_BAR_OFFSET(SMU_ARG_REG, NPU2_SMU, MP1_C2PMSG_60),
	96	DEFINE_BAR_OFFSET(SMU_INTR_REG, NPU2_SMU, MMNPU_APERTURE4_BASE),
	97	DEFINE_BAR_OFFSET(SMU_RESP_REG, NPU2_SMU, MP1_C2PMSG_61),
	98	DEFINE_BAR_OFFSET(SMU_OUT_REG, NPU2_SMU, MP1_C2PMSG_60),
	99	},
	100	.smu_mpnpuclk_freq_max = NPU2_MPNPUCLK_FREQ_MAX,
	101	.smu_hclk_freq_max = NPU2_HCLK_FREQ_MAX,
	102	};
	103
	104	const struct amdxdna_dev_info dev_npu2_info = {
	105	.reg_bar = NPU2_REG_BAR_INDEX,
	106	.mbox_bar = NPU2_MBOX_BAR_INDEX,
	107	.sram_bar = NPU2_SRAM_BAR_INDEX,
	108	.psp_bar = NPU2_PSP_BAR_INDEX,
	109	.smu_bar = NPU2_SMU_BAR_INDEX,
	110	.first_col = 0,
	111	.dev_mem_buf_shift = 15, /* 32 KiB aligned */
	112	.dev_mem_base = AIE2_DEVM_BASE,
	113	.dev_mem_size = AIE2_DEVM_SIZE,
	114	.vbnv = "RyzenAI-npu2",
	115	.device_type = AMDXDNA_DEV_TYPE_KMQ,
	116	.dev_priv = &npu2_dev_priv,
	117	.ops = &aie2_ops, /* NPU2 can share NPU1's callback */
	118	};

drivers/accel/amdxdna/npu4_regs.c+118−0

@@ -0,0 +1,118 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/gpu_scheduler.h>
	9	#include <linux/sizes.h>
	10
	11	#include "aie2_pci.h"
	12	#include "amdxdna_mailbox.h"
	13	#include "amdxdna_pci_drv.h"
	14
	15	/* NPU Public Registers on MpNPUAxiXbar (refer to Diag npu_registers.h) */
	16	#define MPNPU_PUB_SEC_INTR 0x3010060
	17	#define MPNPU_PUB_PWRMGMT_INTR 0x3010064
	18	#define MPNPU_PUB_SCRATCH0 0x301006C
	19	#define MPNPU_PUB_SCRATCH1 0x3010070
	20	#define MPNPU_PUB_SCRATCH2 0x3010074
	21	#define MPNPU_PUB_SCRATCH3 0x3010078
	22	#define MPNPU_PUB_SCRATCH4 0x301007C
	23	#define MPNPU_PUB_SCRATCH5 0x3010080
	24	#define MPNPU_PUB_SCRATCH6 0x3010084
	25	#define MPNPU_PUB_SCRATCH7 0x3010088
	26	#define MPNPU_PUB_SCRATCH8 0x301008C
	27	#define MPNPU_PUB_SCRATCH9 0x3010090
	28	#define MPNPU_PUB_SCRATCH10 0x3010094
	29	#define MPNPU_PUB_SCRATCH11 0x3010098
	30	#define MPNPU_PUB_SCRATCH12 0x301009C
	31	#define MPNPU_PUB_SCRATCH13 0x30100A0
	32	#define MPNPU_PUB_SCRATCH14 0x30100A4
	33	#define MPNPU_PUB_SCRATCH15 0x30100A8
	34	#define MP0_C2PMSG_73 0x3810A24
	35	#define MP0_C2PMSG_123 0x3810AEC
	36
	37	#define MP1_C2PMSG_0 0x3B10900
	38	#define MP1_C2PMSG_60 0x3B109F0
	39	#define MP1_C2PMSG_61 0x3B109F4
	40
	41	#define MPNPU_SRAM_X2I_MAILBOX_0 0x3600000
	42	#define MPNPU_SRAM_X2I_MAILBOX_15 0x361E000
	43	#define MPNPU_SRAM_X2I_MAILBOX_31 0x363E000
	44	#define MPNPU_SRAM_I2X_MAILBOX_31 0x363F000
	45
	46	#define MMNPU_APERTURE0_BASE 0x3000000
	47	#define MMNPU_APERTURE1_BASE 0x3600000
	48	#define MMNPU_APERTURE3_BASE 0x3810000
	49	#define MMNPU_APERTURE4_BASE 0x3B10000
	50
	51	/* PCIe BAR Index for NPU4 */
	52	#define NPU4_REG_BAR_INDEX 0
	53	#define NPU4_MBOX_BAR_INDEX 0
	54	#define NPU4_PSP_BAR_INDEX 4
	55	#define NPU4_SMU_BAR_INDEX 5
	56	#define NPU4_SRAM_BAR_INDEX 2
	57	/* Associated BARs and Apertures */
	58	#define NPU4_REG_BAR_BASE MMNPU_APERTURE0_BASE
	59	#define NPU4_MBOX_BAR_BASE MMNPU_APERTURE0_BASE
	60	#define NPU4_PSP_BAR_BASE MMNPU_APERTURE3_BASE
	61	#define NPU4_SMU_BAR_BASE MMNPU_APERTURE4_BASE
	62	#define NPU4_SRAM_BAR_BASE MMNPU_APERTURE1_BASE
	63
	64	#define NPU4_RT_CFG_TYPE_PDI_LOAD 5
	65	#define NPU4_RT_CFG_VAL_PDI_LOAD_MGMT 0
	66	#define NPU4_RT_CFG_VAL_PDI_LOAD_APP 1
	67
	68	#define NPU4_MPNPUCLK_FREQ_MAX 1267
	69	#define NPU4_HCLK_FREQ_MAX 1800
	70
	71	const struct amdxdna_dev_priv npu4_dev_priv = {
	72	.fw_path = "amdnpu/17f0_10/npu.sbin",
	73	.protocol_major = 0x6,
	74	.protocol_minor = 0x1,
	75	.rt_config = {NPU4_RT_CFG_TYPE_PDI_LOAD, NPU4_RT_CFG_VAL_PDI_LOAD_APP},
	76	.col_align = COL_ALIGN_NATURE,
	77	.mbox_dev_addr = NPU4_MBOX_BAR_BASE,
	78	.mbox_size = 0, /* Use BAR size */
	79	.sram_dev_addr = NPU4_SRAM_BAR_BASE,
	80	.sram_offs = {
	81	DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU4_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
	82	DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU4_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),
	83	},
	84	.psp_regs_off = {
	85	DEFINE_BAR_OFFSET(PSP_CMD_REG, NPU4_PSP, MP0_C2PMSG_123),
	86	DEFINE_BAR_OFFSET(PSP_ARG0_REG, NPU4_REG, MPNPU_PUB_SCRATCH3),
	87	DEFINE_BAR_OFFSET(PSP_ARG1_REG, NPU4_REG, MPNPU_PUB_SCRATCH4),
	88	DEFINE_BAR_OFFSET(PSP_ARG2_REG, NPU4_REG, MPNPU_PUB_SCRATCH9),
	89	DEFINE_BAR_OFFSET(PSP_INTR_REG, NPU4_PSP, MP0_C2PMSG_73),
	90	DEFINE_BAR_OFFSET(PSP_STATUS_REG, NPU4_PSP, MP0_C2PMSG_123),
	91	DEFINE_BAR_OFFSET(PSP_RESP_REG, NPU4_REG, MPNPU_PUB_SCRATCH3),
	92	},
	93	.smu_regs_off = {
	94	DEFINE_BAR_OFFSET(SMU_CMD_REG, NPU4_SMU, MP1_C2PMSG_0),
	95	DEFINE_BAR_OFFSET(SMU_ARG_REG, NPU4_SMU, MP1_C2PMSG_60),
	96	DEFINE_BAR_OFFSET(SMU_INTR_REG, NPU4_SMU, MMNPU_APERTURE4_BASE),
	97	DEFINE_BAR_OFFSET(SMU_RESP_REG, NPU4_SMU, MP1_C2PMSG_61),
	98	DEFINE_BAR_OFFSET(SMU_OUT_REG, NPU4_SMU, MP1_C2PMSG_60),
	99	},
	100	.smu_mpnpuclk_freq_max = NPU4_MPNPUCLK_FREQ_MAX,
	101	.smu_hclk_freq_max = NPU4_HCLK_FREQ_MAX,
	102	};
	103
	104	const struct amdxdna_dev_info dev_npu4_info = {
	105	.reg_bar = NPU4_REG_BAR_INDEX,
	106	.mbox_bar = NPU4_MBOX_BAR_INDEX,
	107	.sram_bar = NPU4_SRAM_BAR_INDEX,
	108	.psp_bar = NPU4_PSP_BAR_INDEX,
	109	.smu_bar = NPU4_SMU_BAR_INDEX,
	110	.first_col = 0,
	111	.dev_mem_buf_shift = 15, /* 32 KiB aligned */
	112	.dev_mem_base = AIE2_DEVM_BASE,
	113	.dev_mem_size = AIE2_DEVM_SIZE,
	114	.vbnv = "RyzenAI-npu4",
	115	.device_type = AMDXDNA_DEV_TYPE_KMQ,
	116	.dev_priv = &npu4_dev_priv,
	117	.ops = &aie2_ops, /* NPU4 can share NPU1's callback */
	118	};

drivers/accel/amdxdna/npu5_regs.c+118−0

@@ -0,0 +1,118 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright (C) 2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#include <drm/amdxdna_accel.h>
	7	#include <drm/drm_device.h>
	8	#include <drm/gpu_scheduler.h>
	9	#include <linux/sizes.h>
	10
	11	#include "aie2_pci.h"
	12	#include "amdxdna_mailbox.h"
	13	#include "amdxdna_pci_drv.h"
	14
	15	/* NPU Public Registers on MpNPUAxiXbar (refer to Diag npu_registers.h) */
	16	#define MPNPU_PUB_SEC_INTR 0x3010060
	17	#define MPNPU_PUB_PWRMGMT_INTR 0x3010064
	18	#define MPNPU_PUB_SCRATCH0 0x301006C
	19	#define MPNPU_PUB_SCRATCH1 0x3010070
	20	#define MPNPU_PUB_SCRATCH2 0x3010074
	21	#define MPNPU_PUB_SCRATCH3 0x3010078
	22	#define MPNPU_PUB_SCRATCH4 0x301007C
	23	#define MPNPU_PUB_SCRATCH5 0x3010080
	24	#define MPNPU_PUB_SCRATCH6 0x3010084
	25	#define MPNPU_PUB_SCRATCH7 0x3010088
	26	#define MPNPU_PUB_SCRATCH8 0x301008C
	27	#define MPNPU_PUB_SCRATCH9 0x3010090
	28	#define MPNPU_PUB_SCRATCH10 0x3010094
	29	#define MPNPU_PUB_SCRATCH11 0x3010098
	30	#define MPNPU_PUB_SCRATCH12 0x301009C
	31	#define MPNPU_PUB_SCRATCH13 0x30100A0
	32	#define MPNPU_PUB_SCRATCH14 0x30100A4
	33	#define MPNPU_PUB_SCRATCH15 0x30100A8
	34	#define MP0_C2PMSG_73 0x3810A24
	35	#define MP0_C2PMSG_123 0x3810AEC
	36
	37	#define MP1_C2PMSG_0 0x3B10900
	38	#define MP1_C2PMSG_60 0x3B109F0
	39	#define MP1_C2PMSG_61 0x3B109F4
	40
	41	#define MPNPU_SRAM_X2I_MAILBOX_0 0x3600000
	42	#define MPNPU_SRAM_X2I_MAILBOX_15 0x361E000
	43	#define MPNPU_SRAM_X2I_MAILBOX_31 0x363E000
	44	#define MPNPU_SRAM_I2X_MAILBOX_31 0x363F000
	45
	46	#define MMNPU_APERTURE0_BASE 0x3000000
	47	#define MMNPU_APERTURE1_BASE 0x3600000
	48	#define MMNPU_APERTURE3_BASE 0x3810000
	49	#define MMNPU_APERTURE4_BASE 0x3B10000
	50
	51	/* PCIe BAR Index for NPU5 */
	52	#define NPU5_REG_BAR_INDEX 0
	53	#define NPU5_MBOX_BAR_INDEX 0
	54	#define NPU5_PSP_BAR_INDEX 4
	55	#define NPU5_SMU_BAR_INDEX 5
	56	#define NPU5_SRAM_BAR_INDEX 2
	57	/* Associated BARs and Apertures */
	58	#define NPU5_REG_BAR_BASE MMNPU_APERTURE0_BASE
	59	#define NPU5_MBOX_BAR_BASE MMNPU_APERTURE0_BASE
	60	#define NPU5_PSP_BAR_BASE MMNPU_APERTURE3_BASE
	61	#define NPU5_SMU_BAR_BASE MMNPU_APERTURE4_BASE
	62	#define NPU5_SRAM_BAR_BASE MMNPU_APERTURE1_BASE
	63
	64	#define NPU5_RT_CFG_TYPE_PDI_LOAD 5
	65	#define NPU5_RT_CFG_VAL_PDI_LOAD_MGMT 0
	66	#define NPU5_RT_CFG_VAL_PDI_LOAD_APP 1
	67
	68	#define NPU5_MPNPUCLK_FREQ_MAX 1267
	69	#define NPU5_HCLK_FREQ_MAX 1800
	70
	71	const struct amdxdna_dev_priv npu5_dev_priv = {
	72	.fw_path = "amdnpu/17f0_11/npu.sbin",
	73	.protocol_major = 0x6,
	74	.protocol_minor = 0x1,
	75	.rt_config = {NPU5_RT_CFG_TYPE_PDI_LOAD, NPU5_RT_CFG_VAL_PDI_LOAD_APP},
	76	.col_align = COL_ALIGN_NATURE,
	77	.mbox_dev_addr = NPU5_MBOX_BAR_BASE,
	78	.mbox_size = 0, /* Use BAR size */
	79	.sram_dev_addr = NPU5_SRAM_BAR_BASE,
	80	.sram_offs = {
	81	DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU5_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
	82	DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU5_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),
	83	},
	84	.psp_regs_off = {
	85	DEFINE_BAR_OFFSET(PSP_CMD_REG, NPU5_PSP, MP0_C2PMSG_123),
	86	DEFINE_BAR_OFFSET(PSP_ARG0_REG, NPU5_REG, MPNPU_PUB_SCRATCH3),
	87	DEFINE_BAR_OFFSET(PSP_ARG1_REG, NPU5_REG, MPNPU_PUB_SCRATCH4),
	88	DEFINE_BAR_OFFSET(PSP_ARG2_REG, NPU5_REG, MPNPU_PUB_SCRATCH9),
	89	DEFINE_BAR_OFFSET(PSP_INTR_REG, NPU5_PSP, MP0_C2PMSG_73),
	90	DEFINE_BAR_OFFSET(PSP_STATUS_REG, NPU5_PSP, MP0_C2PMSG_123),
	91	DEFINE_BAR_OFFSET(PSP_RESP_REG, NPU5_REG, MPNPU_PUB_SCRATCH3),
	92	},
	93	.smu_regs_off = {
	94	DEFINE_BAR_OFFSET(SMU_CMD_REG, NPU5_SMU, MP1_C2PMSG_0),
	95	DEFINE_BAR_OFFSET(SMU_ARG_REG, NPU5_SMU, MP1_C2PMSG_60),
	96	DEFINE_BAR_OFFSET(SMU_INTR_REG, NPU5_SMU, MMNPU_APERTURE4_BASE),
	97	DEFINE_BAR_OFFSET(SMU_RESP_REG, NPU5_SMU, MP1_C2PMSG_61),
	98	DEFINE_BAR_OFFSET(SMU_OUT_REG, NPU5_SMU, MP1_C2PMSG_60),
	99	},
	100	.smu_mpnpuclk_freq_max = NPU5_MPNPUCLK_FREQ_MAX,
	101	.smu_hclk_freq_max = NPU5_HCLK_FREQ_MAX,
	102	};
	103
	104	const struct amdxdna_dev_info dev_npu5_info = {
	105	.reg_bar = NPU5_REG_BAR_INDEX,
	106	.mbox_bar = NPU5_MBOX_BAR_INDEX,
	107	.sram_bar = NPU5_SRAM_BAR_INDEX,
	108	.psp_bar = NPU5_PSP_BAR_INDEX,
	109	.smu_bar = NPU5_SMU_BAR_INDEX,
	110	.first_col = 0,
	111	.dev_mem_buf_shift = 15, /* 32 KiB aligned */
	112	.dev_mem_base = AIE2_DEVM_BASE,
	113	.dev_mem_size = AIE2_DEVM_SIZE,
	114	.vbnv = "RyzenAI-npu5",
	115	.device_type = AMDXDNA_DEV_TYPE_KMQ,
	116	.dev_priv = &npu5_dev_priv,
	117	.ops = &aie2_ops,
	118	};

drivers/accel/habanalabs/common/habanalabs_drv.c+0−1

@@ -101,7 +101,6 @@ static const struct drm_driver hl_driver = { · 100 unmodified lines
101	.major = LINUX_VERSION_MAJOR,	101	.major = LINUX_VERSION_MAJOR,
102	.minor = LINUX_VERSION_PATCHLEVEL,	102	.minor = LINUX_VERSION_PATCHLEVEL,
103	.patchlevel = LINUX_VERSION_SUBLEVEL,	103	.patchlevel = LINUX_VERSION_SUBLEVEL,
104	.date = "20190505",
105		104
106	.fops = &hl_fops,	105	.fops = &hl_fops,
107	.open = hl_device_open,	106	.open = hl_device_open,

drivers/accel/ivpu/ivpu_drv.c+0−8

@@ -458,15 +458,7 @@ static const struct drm_driver driver = { · 457 unmodified lines
458	.name = DRIVER_NAME,	458	.name = DRIVER_NAME,
459	.desc = DRIVER_DESC,	459	.desc = DRIVER_DESC,
460		460
461	#ifdef DRIVER_DATE
462	.date = DRIVER_DATE,
463	.major = DRIVER_MAJOR,
464	.minor = DRIVER_MINOR,
465	.patchlevel = DRIVER_PATCHLEVEL,
466	#else
467	.date = UTS_RELEASE,
468	.major = 1,	461	.major = 1,
469	#endif
470	};	462	};
471		463
472	static void ivpu_context_abort_invalid(struct ivpu_device *vdev)	464	static void ivpu_context_abort_invalid(struct ivpu_device *vdev)

drivers/accel/ivpu/ivpu_pm.c+1−1

@@ -78,8 +78,8 @@ static int ivpu_resume(struct ivpu_device *vdev) · 77 unmodified lines
78	int ret;	78	int ret;
79		79
80	retry:	80	retry:
81	pci_restore_state(to_pci_dev(vdev->drm.dev));
82	pci_set_power_state(to_pci_dev(vdev->drm.dev), PCI_D0);	81	pci_set_power_state(to_pci_dev(vdev->drm.dev), PCI_D0);
		82	pci_restore_state(to_pci_dev(vdev->drm.dev));
83		83
84	ret = ivpu_hw_power_up(vdev);	84	ret = ivpu_hw_power_up(vdev);
85	if (ret) {	85	if (ret) {

drivers/accel/qaic/qaic_drv.c+0−1

@@ -208,7 +208,6 @@ static const struct drm_driver qaic_accel_driver = { · 207 unmodified lines
208		208
209	.name = QAIC_NAME,	209	.name = QAIC_NAME,
210	.desc = QAIC_DESC,	210	.desc = QAIC_DESC,
211	.date = "20190618",
212		211
213	.fops = &qaic_accel_fops,	212	.fops = &qaic_accel_fops,
214	.open = qaic_open,	213	.open = qaic_open,

drivers/accel/qaic/sahara.c+1−2

@@ -772,8 +772,7 @@ static void sahara_mhi_remove(struct mhi_device *mhi_dev) · 771 unmodified lines
772		772
773	cancel_work_sync(&context->fw_work);	773	cancel_work_sync(&context->fw_work);
774	cancel_work_sync(&context->dump_work);	774	cancel_work_sync(&context->dump_work);
775	if (context->mem_dump)	775	vfree(context->mem_dump);
776	vfree(context->mem_dump);
777	sahara_release_image(context);	776	sahara_release_image(context);
778	mhi_unprepare_from_transfer(mhi_dev);	777	mhi_unprepare_from_transfer(mhi_dev);
779	}	778	}

drivers/gpu/drm/Kconfig+5−71

@@ -217,77 +217,7 @@ config DRM_CLIENT · 216 unmodified lines
217	option. Drivers that support the default clients should	217	option. Drivers that support the default clients should
218	select DRM_CLIENT_SELECTION instead.	218	select DRM_CLIENT_SELECTION instead.
219		219
220	config DRM_CLIENT_LIB	220	source "drivers/gpu/drm/clients/Kconfig"
221	tristate
222	depends on DRM
223	select DRM_KMS_HELPER if DRM_FBDEV_EMULATION
224	select FB_CORE if DRM_FBDEV_EMULATION
225	help
226	This option enables the DRM client library and selects all
227	modules and components according to the enabled clients.
228
229	config DRM_CLIENT_SELECTION
230	tristate
231	depends on DRM
232	select DRM_CLIENT_LIB if DRM_FBDEV_EMULATION
233	help
234	Drivers that support in-kernel DRM clients have to select this
235	option.
236
237	config DRM_CLIENT_SETUP
238	bool
239	depends on DRM_CLIENT_SELECTION
240	help
241	Enables the DRM client selection. DRM drivers that support the
242	default clients should select DRM_CLIENT_SELECTION instead.
243
244	menu "Supported DRM clients"
245	depends on DRM_CLIENT_SELECTION
246
247	config DRM_FBDEV_EMULATION
248	bool "Enable legacy fbdev support for your modesetting driver"
249	depends on DRM_CLIENT_SELECTION
250	select DRM_CLIENT
251	select DRM_CLIENT_SETUP
252	select FRAMEBUFFER_CONSOLE_DETECT_PRIMARY if FRAMEBUFFER_CONSOLE
253	default FB
254	help
255	Choose this option if you have a need for the legacy fbdev
256	support. Note that this support also provides the linux console
257	support on top of your modesetting driver.
258
259	If in doubt, say "Y".
260
261	config DRM_FBDEV_OVERALLOC
262	int "Overallocation of the fbdev buffer"
263	depends on DRM_FBDEV_EMULATION
264	default 100
265	help
266	Defines the fbdev buffer overallocation in percent. Default
267	is 100. Typical values for double buffering will be 200,
268	triple buffering 300.
269
270	config DRM_FBDEV_LEAK_PHYS_SMEM
271	bool "Shamelessly allow leaking of fbdev physical address (DANGEROUS)"
272	depends on DRM_FBDEV_EMULATION && EXPERT
273	default n
274	help
275	In order to keep user-space compatibility, we want in certain
276	use-cases to keep leaking the fbdev physical address to the
277	user-space program handling the fbdev buffer.
278	This affects, not only, Amlogic, Allwinner or Rockchip devices
279	with ARM Mali GPUs using an userspace Blob.
280	This option is not supported by upstream developers and should be
281	removed as soon as possible and be considered as a broken and
282	legacy behaviour from a modern fbdev device driver.
283
284	Please send any bug reports when using this to your proprietary
285	software vendor that requires this.
286
287	If in doubt, say "N" or spread the word to your closed source
288	library vendor.
289
290	endmenu
291		221
292	config DRM_LOAD_EDID_FIRMWARE	222	config DRM_LOAD_EDID_FIRMWARE
293	bool "Allow to specify an EDID data set instead of probing for it"	223	bool "Allow to specify an EDID data set instead of probing for it"
@@ -526,6 +456,10 @@ config DRM_HYPERV · 232 unmodified lines
526	config DRM_EXPORT_FOR_TESTS	456	config DRM_EXPORT_FOR_TESTS
527	bool	457	bool
528		458
		459	# Separate option as not all DRM drivers use it
		460	config DRM_PANEL_BACKLIGHT_QUIRKS
		461	tristate
		462
529	config DRM_LIB_RANDOM	463	config DRM_LIB_RANDOM
530	bool	464	bool
531	default n	465	default n

drivers/gpu/drm/Makefile+2−8

 obj-$(CONFIG_DRM)	+= drm.o
 obj-$(CONFIG_DRM_PANEL_ORIENTATION_QUIRKS) += drm_panel_orientation_quirks.o
+obj-$(CONFIG_DRM_PANEL_BACKLIGHT_QUIRKS) += drm_panel_backlight_quirks.o
 #
 # Memory-management helpers
 drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fb_helper.o
 obj-$(CONFIG_DRM_KMS_HELPER) += drm_kms_helper.o
-#
-# DRM clients
-#
-drm_client_lib-y := drm_client_setup.o
-drm_client_lib-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fbdev_client.o
-obj-$(CONFIG_DRM_CLIENT_LIB) += drm_client_lib.o
 #
 # Drivers and the rest
 #
 obj-$(CONFIG_DRM_MIPI_DBI) += drm_mipi_dbi.o
 obj-$(CONFIG_DRM_MIPI_DSI) += drm_mipi_dsi.o
 obj-y			+= arm/
+obj-y			+= clients/
 obj-y			+= display/
 obj-$(CONFIG_DRM_TTM)	+= ttm/
 obj-$(CONFIG_DRM_SCHED)	+= scheduler/

drivers/gpu/drm/amd/amdgpu/Kconfig+1−0

@@ -26,6 +26,7 @@ config DRM_AMDGPU · 25 unmodified lines
26	select DRM_BUDDY	26	select DRM_BUDDY
27	select DRM_SUBALLOC_HELPER	27	select DRM_SUBALLOC_HELPER
28	select DRM_EXEC	28	select DRM_EXEC
		29	select DRM_PANEL_BACKLIGHT_QUIRKS
29	# amdgpu depends on ACPI_VIDEO when ACPI is enabled, for select to work	30	# amdgpu depends on ACPI_VIDEO when ACPI is enabled, for select to work
30	# ACPI_VIDEO's dependencies must also be selected.	31	# ACPI_VIDEO's dependencies must also be selected.
31	select INPUT if ACPI	32	select INPUT if ACPI

drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c+1−3

  */
 #include <drm/amdgpu_drm.h>
-#include <drm/drm_client_setup.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_ttm.h>
 #include <drm/drm_gem.h>
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = KMS_DRIVER_MAJOR,
 	.minor = KMS_DRIVER_MINOR,
 	.patchlevel = KMS_DRIVER_PATCHLEVEL,
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = KMS_DRIVER_MAJOR,
 	.minor = KMS_DRIVER_MINOR,
 	.patchlevel = KMS_DRIVER_PATCHLEVEL,

drivers/gpu/drm/amd/amdgpu/amdgpu_drv.h+0−1

@@ -40,7 +40,6 @@ · 39 unmodified lines
40		40
41	#define DRIVER_NAME "amdgpu"	41	#define DRIVER_NAME "amdgpu"
42	#define DRIVER_DESC "AMD GPU"	42	#define DRIVER_DESC "AMD GPU"
43	#define DRIVER_DATE "20150101"
44		43
45	extern const struct drm_driver amdgpu_partition_driver;	44	extern const struct drm_driver amdgpu_partition_driver;
46		45

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c+6−0

 #include <drm/drm_fourcc.h>
 #include <drm/drm_edid.h>
 #include <drm/drm_eld.h>
+#include <drm/drm_utils.h>
 #include <drm/drm_vblank.h>
 #include <drm/drm_audio_component.h>
 #include <drm/drm_gem_atomic_helper.h>
 	struct drm_connector *conn_base;
 	struct amdgpu_device *adev;
 	struct drm_luminance_range_info *luminance_range;
+	int min_input_signal_override;
 	if (aconnector->bl_idx == -1 ||
 	    aconnector->dc_link->connector_signal != SIGNAL_TYPE_EDP)
 		caps->aux_min_input_signal = 0;
 		caps->aux_max_input_signal = 512;
 	}
+	min_input_signal_override = drm_get_panel_min_brightness_quirk(aconnector->drm_edid);
+	if (min_input_signal_override >= 0)
+		caps->min_input_signal = min_input_signal_override;
 }
 void amdgpu_dm_update_connector_after_detect(

drivers/gpu/drm/arm/display/komeda/komeda_drv.c+1−1

@@ -9,7 +9,7 @@ · 8 unmodified lines
9	#include <linux/of.h>	9	#include <linux/of.h>
10	#include <linux/platform_device.h>	10	#include <linux/platform_device.h>
11	#include <linux/pm_runtime.h>	11	#include <linux/pm_runtime.h>
12	#include <drm/drm_client_setup.h>	12	#include <drm/clients/drm_client_setup.h>
13	#include <drm/drm_module.h>	13	#include <drm/drm_module.h>
14	#include <drm/drm_of.h>	14	#include <drm/drm_of.h>
15	#include "komeda_dev.h"	15	#include "komeda_dev.h"

drivers/gpu/drm/arm/display/komeda/komeda_kms.c+0−1

@@ -63,7 +63,6 @@ static const struct drm_driver komeda_kms_driver = { · 62 unmodified lines
63	.fops = &komeda_cma_fops,	63	.fops = &komeda_cma_fops,
64	.name = "komeda",	64	.name = "komeda",
65	.desc = "Arm Komeda Display Processor driver",	65	.desc = "Arm Komeda Display Processor driver",
66	.date = "20181101",
67	.major = 0,	66	.major = 0,
68	.minor = 1,	67	.minor = 1,
69	};	68	};

drivers/gpu/drm/arm/hdlcd_drv.c+1−2

 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_crtc.h>
 #include <drm/drm_debugfs.h>
 #include <drm/drm_drv.h>
 	.fops = &fops,
 	.name = "hdlcd",
 	.desc = "ARM HDLCD Controller DRM",
-	.date = "20151021",
 	.major = 1,
 	.minor = 0,
 };

drivers/gpu/drm/arm/malidp_drv.c+1−2

 #include <linux/pm_runtime.h>
 #include <linux/debugfs.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_crtc.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 	.fops = &fops,
 	.name = "mali-dp",
 	.desc = "ARM Mali Display Processor driver",
-	.date = "20160106",
 	.major = 1,
 	.minor = 0,
 };

drivers/gpu/drm/armada/armada_drv.c+1−2

 #include <linux/of_graph.h>
 #include <linux/platform_device.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_ioctl.h>
 #include <drm/drm_managed.h>
 	.minor			= 0,
 	.name			= "armada-drm",
 	.desc			= "Armada SoC DRM",
-	.date			= "20120730",
 	.driver_features	= DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC,
 	.ioctls			= armada_ioctls,
 	.num_ioctls = ARRAY_SIZE(armada_ioctls),

drivers/gpu/drm/aspeed/aspeed_gfx_drv.c+1−2

 #include <linux/regmap.h>
 #include <linux/reset.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_device.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	.fops = &fops,
 	.name = "aspeed-gfx-drm",
 	.desc = "ASPEED GFX DRM",
-	.date = "20180319",
 	.major = 1,
 	.minor = 0,
 };

drivers/gpu/drm/ast/ast_drv.c+1−2

 #include <linux/of.h>
 #include <linux/pci.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_shmem.h>
 #include <drm/drm_gem_shmem_helper.h>
 	.fops = &ast_fops,
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/ast/ast_drv.h+0−1

@@ -43,7 +43,6 @@ · 42 unmodified lines
43		43
44	#define DRIVER_NAME "ast"	44	#define DRIVER_NAME "ast"
45	#define DRIVER_DESC "AST"	45	#define DRIVER_DESC "AST"
46	#define DRIVER_DATE "20120228"
47		46
48	#define DRIVER_MAJOR 0	47	#define DRIVER_MAJOR 0
49	#define DRIVER_MINOR 1	48	#define DRIVER_MINOR 1

drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c+1−2

 #include <linux/pm_runtime.h>
 #include <linux/platform_device.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_fourcc.h>
 	.fops = &fops,
 	.name = "atmel-hlcdc",
 	.desc = "Atmel HLCD Controller DRM",
-	.date = "20141504",
 	.major = 1,
 	.minor = 0,
 };

drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-hdcp.c+0−28

@@ -500,34 +500,6 @@ static void cdns_mhdp_hdcp_prop_work(struct work_struct *work) · 499 unmodified lines
500	drm_modeset_unlock(&dev->mode_config.connection_mutex);	500	drm_modeset_unlock(&dev->mode_config.connection_mutex);
501	}	501	}
502		502
503	int cdns_mhdp_hdcp_set_lc(struct cdns_mhdp_device mhdp, u8 val)
504	{
505	int ret;
506
507	mutex_lock(&mhdp->mbox_mutex);
508	ret = cdns_mhdp_secure_mailbox_send(mhdp, MB_MODULE_ID_HDCP_GENERAL,
509	HDCP_GENERAL_SET_LC_128,
510	16, val);
511	mutex_unlock(&mhdp->mbox_mutex);
512
513	return ret;
514	}
515
516	int
517	cdns_mhdp_hdcp_set_public_key_param(struct cdns_mhdp_device *mhdp,
518	struct cdns_hdcp_tx_public_key_param *val)
519	{
520	int ret;
521
522	mutex_lock(&mhdp->mbox_mutex);
523	ret = cdns_mhdp_secure_mailbox_send(mhdp, MB_MODULE_ID_HDCP_TX,
524	HDCP2X_TX_SET_PUBLIC_KEY_PARAMS,
525	sizeof(val), (u8 )val);
526	mutex_unlock(&mhdp->mbox_mutex);
527
528	return ret;
529	}
530
531	int cdns_mhdp_hdcp_enable(struct cdns_mhdp_device *mhdp, u8 content_type)	503	int cdns_mhdp_hdcp_enable(struct cdns_mhdp_device *mhdp, u8 content_type)
532	{	504	{
533	int ret;	505	int ret;

drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-hdcp.h+0−3

 	u8 E[DLP_E];
 };
-int cdns_mhdp_hdcp_set_public_key_param(struct cdns_mhdp_device *mhdp,
-					struct cdns_hdcp_tx_public_key_param *val);
-int cdns_mhdp_hdcp_set_lc(struct cdns_mhdp_device *mhdp, u8 *val);
 int cdns_mhdp_hdcp_enable(struct cdns_mhdp_device *mhdp, u8 content_type);
 int cdns_mhdp_hdcp_disable(struct cdns_mhdp_device *mhdp);
 void cdns_mhdp_hdcp_init(struct cdns_mhdp_device *mhdp);

drivers/gpu/drm/bridge/chipone-icn6211.c+1−1

@@ -785,7 +785,7 @@ static struct mipi_dsi_driver chipone_dsi_driver = { · 784 unmodified lines
785	},	785	},
786	};	786	};
787		787
788	static struct i2c_device_id chipone_i2c_id[] = {	788	static const struct i2c_device_id chipone_i2c_id[] = {
789	{ "chipone,icn6211" },	789	{ "chipone,icn6211" },
790	{},	790	{},
791	};	791	};

drivers/gpu/drm/bridge/lontium-lt9211.c+1−1

@@ -773,7 +773,7 @@ static void lt9211_remove(struct i2c_client *client) · 772 unmodified lines
773	drm_bridge_remove(&ctx->bridge);	773	drm_bridge_remove(&ctx->bridge);
774	}	774	}
775		775
776	static struct i2c_device_id lt9211_id[] = {	776	static const struct i2c_device_id lt9211_id[] = {
777	{ "lontium,lt9211" },	777	{ "lontium,lt9211" },
778	{},	778	{},
779	};	779	};

drivers/gpu/drm/bridge/lontium-lt9611.c+1−1

@@ -1235,7 +1235,7 @@ static void lt9611_remove(struct i2c_client *client) · 1234 unmodified lines
1235	of_node_put(lt9611->dsi0_node);	1235	of_node_put(lt9611->dsi0_node);
1236	}	1236	}
1237		1237
1238	static struct i2c_device_id lt9611_id[] = {	1238	static const struct i2c_device_id lt9611_id[] = {
1239	{ "lontium,lt9611", 0 },	1239	{ "lontium,lt9611", 0 },
1240	{}	1240	{}
1241	};	1241	};

drivers/gpu/drm/bridge/lontium-lt9611uxc.c+1−1

@@ -913,7 +913,7 @@ static void lt9611uxc_remove(struct i2c_client *client) · 912 unmodified lines
913	of_node_put(lt9611uxc->dsi0_node);	913	of_node_put(lt9611uxc->dsi0_node);
914	}	914	}
915		915
916	static struct i2c_device_id lt9611uxc_id[] = {	916	static const struct i2c_device_id lt9611uxc_id[] = {
917	{ "lontium,lt9611uxc", 0 },	917	{ "lontium,lt9611uxc", 0 },
918	{ /* sentinel */ }	918	{ /* sentinel */ }
919	};	919	};

drivers/gpu/drm/bridge/ti-sn65dsi83.c+1−1

@@ -732,7 +732,7 @@ static void sn65dsi83_remove(struct i2c_client *client) · 731 unmodified lines
732	drm_bridge_remove(&ctx->bridge);	732	drm_bridge_remove(&ctx->bridge);
733	}	733	}
734		734
735	static struct i2c_device_id sn65dsi83_id[] = {	735	static const struct i2c_device_id sn65dsi83_id[] = {
736	{ "ti,sn65dsi83", MODEL_SN65DSI83 },	736	{ "ti,sn65dsi83", MODEL_SN65DSI83 },
737	{ "ti,sn65dsi84", MODEL_SN65DSI84 },	737	{ "ti,sn65dsi84", MODEL_SN65DSI84 },
738	{},	738	{},

drivers/gpu/drm/bridge/ti-sn65dsi86.c+1−1

@@ -1970,7 +1970,7 @@ static int ti_sn65dsi86_probe(struct i2c_client *client) · 1969 unmodified lines
1970	return ti_sn65dsi86_add_aux_device(pdata, &pdata->aux_aux, "aux");	1970	return ti_sn65dsi86_add_aux_device(pdata, &pdata->aux_aux, "aux");
1971	}	1971	}
1972		1972
1973	static struct i2c_device_id ti_sn65dsi86_id[] = {	1973	static const struct i2c_device_id ti_sn65dsi86_id[] = {
1974	{ "ti,sn65dsi86", 0},	1974	{ "ti,sn65dsi86", 0},
1975	{},	1975	{},
1976	};	1976	};

drivers/gpu/drm/clients/Kconfig+73−0

@@ -0,0 +1,73 @@
	1	# SPDX-License-Identifier: GPL-2.0-only
	2
	3	config DRM_CLIENT_LIB
	4	tristate
	5	depends on DRM
	6	select DRM_KMS_HELPER if DRM_FBDEV_EMULATION
	7	select FB_CORE if DRM_FBDEV_EMULATION
	8	help
	9	This option enables the DRM client library and selects all
	10	modules and components according to the enabled clients.
	11
	12	config DRM_CLIENT_SELECTION
	13	tristate
	14	depends on DRM
	15	select DRM_CLIENT_LIB if DRM_FBDEV_EMULATION
	16	help
	17	Drivers that support in-kernel DRM clients have to select this
	18	option.
	19
	20	config DRM_CLIENT_SETUP
	21	bool
	22	depends on DRM_CLIENT_SELECTION
	23	help
	24	Enables the DRM client selection. DRM drivers that support the
	25	default clients should select DRM_CLIENT_SELECTION instead.
	26
	27	menu "Supported DRM clients"
	28	depends on DRM_CLIENT_SELECTION
	29
	30	config DRM_FBDEV_EMULATION
	31	bool "Enable legacy fbdev support for your modesetting driver"
	32	depends on DRM_CLIENT_SELECTION
	33	select DRM_CLIENT
	34	select DRM_CLIENT_SETUP
	35	select FRAMEBUFFER_CONSOLE_DETECT_PRIMARY if FRAMEBUFFER_CONSOLE
	36	default FB
	37	help
	38	Choose this option if you have a need for the legacy fbdev
	39	support. Note that this support also provides the linux console
	40	support on top of your modesetting driver.
	41
	42	If in doubt, say "Y".
	43
	44	config DRM_FBDEV_OVERALLOC
	45	int "Overallocation of the fbdev buffer"
	46	depends on DRM_FBDEV_EMULATION
	47	default 100
	48	help
	49	Defines the fbdev buffer overallocation in percent. Default
	50	is 100. Typical values for double buffering will be 200,
	51	triple buffering 300.
	52
	53	config DRM_FBDEV_LEAK_PHYS_SMEM
	54	bool "Shamelessly allow leaking of fbdev physical address (DANGEROUS)"
	55	depends on DRM_FBDEV_EMULATION && EXPERT
	56	default n
	57	help
	58	In order to keep user-space compatibility, we want in certain
	59	use-cases to keep leaking the fbdev physical address to the
	60	user-space program handling the fbdev buffer.
	61	This affects, not only, Amlogic, Allwinner or Rockchip devices
	62	with ARM Mali GPUs using a userspace Blob.
	63	This option is not supported by upstream developers and should be
	64	removed as soon as possible and be considered as a broken and
	65	legacy behaviour from a modern fbdev device driver.
	66
	67	Please send any bug reports when using this to your proprietary
	68	software vendor that requires this.
	69
	70	If in doubt, say "N" or spread the word to your closed source
	71	library vendor.
	72
	73	endmenu

drivers/gpu/drm/clients/Makefile+5−0

@@ -0,0 +1,5 @@
	1	# SPDX-License-Identifier: GPL-2.0
	2
	3	drm_client_lib-y := drm_client_setup.o
	4	drm_client_lib-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fbdev_client.o
	5	obj-$(CONFIG_DRM_CLIENT_LIB) += drm_client_lib.o

drivers/gpu/drm/clients/drm_client_internal.hrenamed from include/drm/drm_fbdev_client.h+2−2

@@ -1,7 +1,7 @@
1	/* SPDX-License-Identifier: MIT */	1	/* SPDX-License-Identifier: MIT */
2		2
3	#ifndef DRM_FBDEV_CLIENT_H	3	#ifndef DRM_CLIENT_INTERNAL_H
4	#define DRM_FBDEV_CLIENT_H	4	#define DRM_CLIENT_INTERNAL_H
5		5
6	struct drm_device;	6	struct drm_device;
7	struct drm_format_info;	7	struct drm_format_info;

drivers/gpu/drm/clients/drm_client_setup.crenamed from drivers/gpu/drm/drm_client_setup.c+3−2

 // SPDX-License-Identifier: MIT
-#include <drm/drm_client_setup.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_device.h>
-#include <drm/drm_fbdev_client.h>
 #include <drm/drm_fourcc.h>
 #include <drm/drm_print.h>
+#include "drm_client_internal.h"
 /**
  * drm_client_setup() - Setup in-kernel DRM clients
  * @dev: DRM device

drivers/gpu/drm/clients/drm_fbdev_client.crenamed from drivers/gpu/drm/drm_fbdev_client.c+2−2

 #include <drm/drm_client.h>
 #include <drm/drm_crtc_helper.h>
 #include <drm/drm_drv.h>
-#include <drm/drm_fbdev_client.h>
 #include <drm/drm_fb_helper.h>
 #include <drm/drm_fourcc.h>
 #include <drm/drm_print.h>
+#include "drm_client_internal.h"
 /*
  * struct drm_client_funcs
  */
 	kfree(fb_helper);
 	return ret;
 }
-EXPORT_SYMBOL(drm_fbdev_client_setup);

drivers/gpu/drm/display/drm_dp_helper.c+124−1

@@ -22,15 +22,16 @@ · 21 unmodified lines
22		22
23	#include <linux/backlight.h>	23	#include <linux/backlight.h>
24	#include <linux/delay.h>	24	#include <linux/delay.h>
		25	#include <linux/dynamic_debug.h>
25	#include <linux/errno.h>	26	#include <linux/errno.h>
26	#include <linux/i2c.h>	27	#include <linux/i2c.h>
27	#include <linux/init.h>	28	#include <linux/init.h>
		29	#include <linux/iopoll.h>
28	#include <linux/kernel.h>	30	#include <linux/kernel.h>
29	#include <linux/module.h>	31	#include <linux/module.h>
30	#include <linux/sched.h>	32	#include <linux/sched.h>
31	#include <linux/seq_file.h>	33	#include <linux/seq_file.h>
32	#include <linux/string_helpers.h>	34	#include <linux/string_helpers.h>
33	#include <linux/dynamic_debug.h>
34		35
35	#include <drm/display/drm_dp_helper.h>	36	#include <drm/display/drm_dp_helper.h>
36	#include <drm/display/drm_dp_mst_helper.h>	37	#include <drm/display/drm_dp_mst_helper.h>
@@ -779,6 +780,128 @@ int drm_dp_dpcd_read_phy_link_status(struct drm_dp_aux *aux, · 742 unmodified lines
779	}	780	}
780	EXPORT_SYMBOL(drm_dp_dpcd_read_phy_link_status);	781	EXPORT_SYMBOL(drm_dp_dpcd_read_phy_link_status);
781		782
		783	static int read_payload_update_status(struct drm_dp_aux *aux)
		784	{
		785	int ret;
		786	u8 status;
		787
		788	ret = drm_dp_dpcd_readb(aux, DP_PAYLOAD_TABLE_UPDATE_STATUS, &status);
		789	if (ret < 0)
		790	return ret;
		791
		792	return status;
		793	}
		794
		795	/**
		796	* drm_dp_dpcd_write_payload() - Write Virtual Channel information to payload table
		797	* @aux: DisplayPort AUX channel
		798	* @vcpid: Virtual Channel Payload ID
		799	* @start_time_slot: Starting time slot
		800	* @time_slot_count: Time slot count
		801	*
		802	* Write the Virtual Channel payload allocation table, checking the payload
		803	* update status and retrying as necessary.
		804	*
		805	* Returns:
		806	* 0 on success, negative error otherwise
		807	*/
		808	int drm_dp_dpcd_write_payload(struct drm_dp_aux *aux,
		809	int vcpid, u8 start_time_slot, u8 time_slot_count)
		810	{
		811	u8 payload_alloc[3], status;
		812	int ret;
		813	int retries = 0;
		814
		815	drm_dp_dpcd_writeb(aux, DP_PAYLOAD_TABLE_UPDATE_STATUS,
		816	DP_PAYLOAD_TABLE_UPDATED);
		817
		818	payload_alloc[0] = vcpid;
		819	payload_alloc[1] = start_time_slot;
		820	payload_alloc[2] = time_slot_count;
		821
		822	ret = drm_dp_dpcd_write(aux, DP_PAYLOAD_ALLOCATE_SET, payload_alloc, 3);
		823	if (ret != 3) {
		824	drm_dbg_kms(aux->drm_dev, "failed to write payload allocation %d\n", ret);
		825	goto fail;
		826	}
		827
		828	retry:
		829	ret = drm_dp_dpcd_readb(aux, DP_PAYLOAD_TABLE_UPDATE_STATUS, &status);
		830	if (ret < 0) {
		831	drm_dbg_kms(aux->drm_dev, "failed to read payload table status %d\n", ret);
		832	goto fail;
		833	}
		834
		835	if (!(status & DP_PAYLOAD_TABLE_UPDATED)) {
		836	retries++;
		837	if (retries < 20) {
		838	usleep_range(10000, 20000);
		839	goto retry;
		840	}
		841	drm_dbg_kms(aux->drm_dev, "status not set after read payload table status %d\n",
		842	status);
		843	ret = -EINVAL;
		844	goto fail;
		845	}
		846	ret = 0;
		847	fail:
		848	return ret;
		849	}
		850	EXPORT_SYMBOL(drm_dp_dpcd_write_payload);
		851
		852	/**
		853	* drm_dp_dpcd_clear_payload() - Clear the entire VC Payload ID table
		854	* @aux: DisplayPort AUX channel
		855	*
		856	* Clear the entire VC Payload ID table.
		857	*
		858	* Returns: 0 on success, negative error code on errors.
		859	*/
		860	int drm_dp_dpcd_clear_payload(struct drm_dp_aux *aux)
		861	{
		862	return drm_dp_dpcd_write_payload(aux, 0, 0, 0x3f);
		863	}
		864	EXPORT_SYMBOL(drm_dp_dpcd_clear_payload);
		865
		866	/**
		867	* drm_dp_dpcd_poll_act_handled() - Poll for ACT handled status
		868	* @aux: DisplayPort AUX channel
		869	* @timeout_ms: Timeout in ms
		870	*
		871	* Try waiting for the sink to finish updating its payload table by polling for
		872	* the ACT handled bit of DP_PAYLOAD_TABLE_UPDATE_STATUS for up to @timeout_ms
		873	* milliseconds, defaulting to 3000 ms if 0.
		874	*
		875	* Returns:
		876	* 0 if the ACT was handled in time, negative error code on failure.
		877	*/
		878	int drm_dp_dpcd_poll_act_handled(struct drm_dp_aux *aux, int timeout_ms)
		879	{
		880	int ret, status;
		881
		882	/* default to 3 seconds, this is arbitrary */
		883	timeout_ms = timeout_ms ?: 3000;
		884
		885	ret = readx_poll_timeout(read_payload_update_status, aux, status,
		886	status & DP_PAYLOAD_ACT_HANDLED \|\| status < 0,
		887	200, timeout_ms * USEC_PER_MSEC);
		888	if (ret < 0 && status >= 0) {
		889	drm_err(aux->drm_dev, "Failed to get ACT after %d ms, last status: %02x\n",
		890	timeout_ms, status);
		891	return -EINVAL;
		892	} else if (status < 0) {
		893	/*
		894	* Failure here isn't unexpected - the hub may have
		895	* just been unplugged
		896	*/
		897	drm_dbg_kms(aux->drm_dev, "Failed to read payload table status: %d\n", status);
		898	return status;
		899	}
		900
		901	return 0;
		902	}
		903	EXPORT_SYMBOL(drm_dp_dpcd_poll_act_handled);
		904
782	static bool is_edid_digital_input_dp(const struct drm_edid *drm_edid)	905	static bool is_edid_digital_input_dp(const struct drm_edid *drm_edid)
783	{	906	{
784	/* FIXME: get rid of drm_edid_raw() */	907	/* FIXME: get rid of drm_edid_raw() */

drivers/gpu/drm/display/drm_dp_mst_topology.c+5−83

@@ -29,7 +29,6 @@ · 28 unmodified lines
29	#include <linux/random.h>	29	#include <linux/random.h>
30	#include <linux/sched.h>	30	#include <linux/sched.h>
31	#include <linux/seq_file.h>	31	#include <linux/seq_file.h>
32	#include <linux/iopoll.h>
33		32
34	#if IS_ENABLED(CONFIG_DRM_DEBUG_DP_MST_TOPOLOGY_REFS)	33	#if IS_ENABLED(CONFIG_DRM_DEBUG_DP_MST_TOPOLOGY_REFS)
35	#include <linux/stacktrace.h>	34	#include <linux/stacktrace.h>
@@ -68,9 +67,6 @@ static bool dump_dp_payload_table(struct drm_dp_mst_topology_mgr *mgr, · 32 unmodified lines
68		67
69	static void drm_dp_mst_topology_put_port(struct drm_dp_mst_port *port);	68	static void drm_dp_mst_topology_put_port(struct drm_dp_mst_port *port);
70		69
71	static int drm_dp_dpcd_write_payload(struct drm_dp_mst_topology_mgr *mgr,
72	int id, u8 start_slot, u8 num_slots);
73
74	static int drm_dp_send_dpcd_read(struct drm_dp_mst_topology_mgr *mgr,	70	static int drm_dp_send_dpcd_read(struct drm_dp_mst_topology_mgr *mgr,
75	struct drm_dp_mst_port *port,	71	struct drm_dp_mst_port *port,
76	int offset, int size, u8 *bytes);	72	int offset, int size, u8 *bytes);
@@ -3267,7 +3263,7 @@ EXPORT_SYMBOL(drm_dp_send_query_stream_enc_status); · 3190 unmodified lines
3267	static int drm_dp_create_payload_at_dfp(struct drm_dp_mst_topology_mgr *mgr,	3263	static int drm_dp_create_payload_at_dfp(struct drm_dp_mst_topology_mgr *mgr,
3268	struct drm_dp_mst_atomic_payload *payload)	3264	struct drm_dp_mst_atomic_payload *payload)
3269	{	3265	{
3270	return drm_dp_dpcd_write_payload(mgr, payload->vcpi, payload->vc_start_slot,	3266	return drm_dp_dpcd_write_payload(mgr->aux, payload->vcpi, payload->vc_start_slot,
3271	payload->time_slots);	3267	payload->time_slots);
3272	}	3268	}
3273		3269
@@ -3298,7 +3294,7 @@ static void drm_dp_destroy_payload_at_remote_and_dfp(struct drm_dp_mst_topology_ · 24 unmodified lines
3298	}	3294	}
3299		3295
3300	if (payload->payload_allocation_status == DRM_DP_MST_PAYLOAD_ALLOCATION_DFP)	3296	if (payload->payload_allocation_status == DRM_DP_MST_PAYLOAD_ALLOCATION_DFP)
3301	drm_dp_dpcd_write_payload(mgr, payload->vcpi, payload->vc_start_slot, 0);	3297	drm_dp_dpcd_write_payload(mgr->aux, payload->vcpi, payload->vc_start_slot, 0);
3302	}	3298	}
3303		3299
3304	/**	3300	/**
@@ -3686,7 +3682,7 @@ int drm_dp_mst_topology_mgr_set_mst(struct drm_dp_mst_topology_mgr *mgr, bool ms · 381 unmodified lines
3686	goto out_unlock;	3682	goto out_unlock;
3687		3683
3688	/* Write reset payload */	3684	/* Write reset payload */
3689	drm_dp_dpcd_write_payload(mgr, 0, 0, 0x3f);	3685	drm_dp_dpcd_clear_payload(mgr->aux);
3690		3686
3691	drm_dp_mst_queue_probe_work(mgr);	3687	drm_dp_mst_queue_probe_work(mgr);
3692		3688
@@ -4747,61 +4743,6 @@ void drm_dp_mst_update_slots(struct drm_dp_mst_topology_state *mst_state, uint8_ · 1054 unmodified lines
4747	}	4743	}
4748	EXPORT_SYMBOL(drm_dp_mst_update_slots);	4744	EXPORT_SYMBOL(drm_dp_mst_update_slots);
4749		4745
4750	static int drm_dp_dpcd_write_payload(struct drm_dp_mst_topology_mgr *mgr,
4751	int id, u8 start_slot, u8 num_slots)
4752	{
4753	u8 payload_alloc[3], status;
4754	int ret;
4755	int retries = 0;
4756
4757	drm_dp_dpcd_writeb(mgr->aux, DP_PAYLOAD_TABLE_UPDATE_STATUS,
4758	DP_PAYLOAD_TABLE_UPDATED);
4759
4760	payload_alloc[0] = id;
4761	payload_alloc[1] = start_slot;
4762	payload_alloc[2] = num_slots;
4763
4764	ret = drm_dp_dpcd_write(mgr->aux, DP_PAYLOAD_ALLOCATE_SET, payload_alloc, 3);
4765	if (ret != 3) {
4766	drm_dbg_kms(mgr->dev, "failed to write payload allocation %d\n", ret);
4767	goto fail;
4768	}
4769
4770	retry:
4771	ret = drm_dp_dpcd_readb(mgr->aux, DP_PAYLOAD_TABLE_UPDATE_STATUS, &status);
4772	if (ret < 0) {
4773	drm_dbg_kms(mgr->dev, "failed to read payload table status %d\n", ret);
4774	goto fail;
4775	}
4776
4777	if (!(status & DP_PAYLOAD_TABLE_UPDATED)) {
4778	retries++;
4779	if (retries < 20) {
4780	usleep_range(10000, 20000);
4781	goto retry;
4782	}
4783	drm_dbg_kms(mgr->dev, "status not set after read payload table status %d\n",
4784	status);
4785	ret = -EINVAL;
4786	goto fail;
4787	}
4788	ret = 0;
4789	fail:
4790	return ret;
4791	}
4792
4793	static int do_get_act_status(struct drm_dp_aux *aux)
4794	{
4795	int ret;
4796	u8 status;
4797
4798	ret = drm_dp_dpcd_readb(aux, DP_PAYLOAD_TABLE_UPDATE_STATUS, &status);
4799	if (ret < 0)
4800	return ret;
4801
4802	return status;
4803	}
4804
4805	/**	4746	/**
4806	* drm_dp_check_act_status() - Polls for ACT handled status.	4747	* drm_dp_check_act_status() - Polls for ACT handled status.
4807	* @mgr: manager to use	4748	* @mgr: manager to use
@@ -4819,28 +4760,9 @@ int drm_dp_check_act_status(struct drm_dp_mst_topology_mgr *mgr) · 11 unmodified lines
4819	* There doesn't seem to be any recommended retry count or timeout in	4760	* There doesn't seem to be any recommended retry count or timeout in
4820	* the MST specification. Since some hubs have been observed to take	4761	* the MST specification. Since some hubs have been observed to take
4821	* over 1 second to update their payload allocations under certain	4762	* over 1 second to update their payload allocations under certain
4822	* conditions, we use a rather large timeout value.	4763	* conditions, we use a rather large timeout value of 3 seconds.
4823	*/	4764	*/
4824	const int timeout_ms = 3000;	4765	return drm_dp_dpcd_poll_act_handled(mgr->aux, 3000);
4825	int ret, status;
4826
4827	ret = readx_poll_timeout(do_get_act_status, mgr->aux, status,
4828	status & DP_PAYLOAD_ACT_HANDLED \|\| status < 0,
4829	200, timeout_ms * USEC_PER_MSEC);
4830	if (ret < 0 && status >= 0) {
4831	drm_err(mgr->dev, "Failed to get ACT after %dms, last status: %02x\n",
4832	timeout_ms, status);
4833	return -EINVAL;
4834	} else if (status < 0) {
4835	/*
4836	* Failure here isn't unexpected - the hub may have
4837	* just been unplugged
4838	*/
4839	drm_dbg_kms(mgr->dev, "Failed to read payload table status: %d\n", status);
4840	return status;
4841	}
4842
4843	return 0;
4844	}	4766	}
4845	EXPORT_SYMBOL(drm_dp_check_act_status);	4767	EXPORT_SYMBOL(drm_dp_check_act_status);
4846		4768

drivers/gpu/drm/drm_of.c+3−1

  * Gets parent DSI bus for a DSI device controlled through a bus other
  * than MIPI-DCS (SPI, I2C, etc.) using the Device Tree.
  *
+ * This function assumes that the device's port@0 is the DSI input.
+ *
  * Returns pointer to mipi_dsi_host if successful, -EINVAL if the
  * request is unsupported, -EPROBE_DEFER if the DSI host is found but
  * not available, or -ENODEV otherwise.
 	/*
 	 * Get first endpoint child from device.
 	 */
-	endpoint = of_graph_get_next_endpoint(dev->of_node, NULL);
+	endpoint = of_graph_get_endpoint_by_regs(dev->of_node, 0, -1);
 	if (!endpoint)
 		return ERR_PTR(-ENODEV);

drivers/gpu/drm/drm_panel_backlight_quirks.c+94−0

@@ -0,0 +1,94 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2
	3	#include <linux/array_size.h>
	4	#include <linux/dmi.h>
	5	#include <linux/mod_devicetable.h>
	6	#include <linux/module.h>
	7	#include <drm/drm_edid.h>
	8	#include <drm/drm_utils.h>
	9
	10	struct drm_panel_min_backlight_quirk {
	11	struct {
	12	enum dmi_field field;
	13	const char * const value;
	14	} dmi_match;
	15	struct drm_edid_ident ident;
	16	u8 min_brightness;
	17	};
	18
	19	static const struct drm_panel_min_backlight_quirk drm_panel_min_backlight_quirks[] = {
	20	/* 13 inch matte panel */
	21	{
	22	.dmi_match.field = DMI_BOARD_VENDOR,
	23	.dmi_match.value = "Framework",
	24	.ident.panel_id = drm_edid_encode_panel_id('B', 'O', 'E', 0x0bca),
	25	.ident.name = "NE135FBM-N41",
	26	.min_brightness = 0,
	27	},
	28	/* 13 inch glossy panel */
	29	{
	30	.dmi_match.field = DMI_BOARD_VENDOR,
	31	.dmi_match.value = "Framework",
	32	.ident.panel_id = drm_edid_encode_panel_id('B', 'O', 'E', 0x095f),
	33	.ident.name = "NE135FBM-N41",
	34	.min_brightness = 0,
	35	},
	36	/* 13 inch 2.8k panel */
	37	{
	38	.dmi_match.field = DMI_BOARD_VENDOR,
	39	.dmi_match.value = "Framework",
	40	.ident.panel_id = drm_edid_encode_panel_id('B', 'O', 'E', 0x0cb4),
	41	.ident.name = "NE135A1M-NY1",
	42	.min_brightness = 0,
	43	},
	44	};
	45
	46	static bool drm_panel_min_backlight_quirk_matches(const struct drm_panel_min_backlight_quirk *quirk,
	47	const struct drm_edid *edid)
	48	{
	49	if (!dmi_match(quirk->dmi_match.field, quirk->dmi_match.value))
	50	return false;
	51
	52	if (!drm_edid_match(edid, &quirk->ident))
	53	return false;
	54
	55	return true;
	56	}
	57
	58	/**
	59	* drm_get_panel_min_brightness_quirk - Get minimum supported brightness level for a panel.
	60	* @edid: EDID of the panel to check
	61	*
	62	* This function checks for platform specific (e.g. DMI based) quirks
	63	* providing info on the minimum backlight brightness for systems where this
	64	* cannot be probed correctly from the hard-/firm-ware.
	65	*
	66	* Returns:
	67	* A negative error value or
	68	* an override value in the range [0, 255] representing 0-100% to be scaled to
	69	* the drivers target range.
	70	*/
	71	int drm_get_panel_min_brightness_quirk(const struct drm_edid *edid)
	72	{
	73	const struct drm_panel_min_backlight_quirk *quirk;
	74	size_t i;
	75
	76	if (!IS_ENABLED(CONFIG_DMI))
	77	return -ENODATA;
	78
	79	if (!edid)
	80	return -EINVAL;
	81
	82	for (i = 0; i < ARRAY_SIZE(drm_panel_min_backlight_quirks); i++) {
	83	quirk = &drm_panel_min_backlight_quirks[i];
	84
	85	if (drm_panel_min_backlight_quirk_matches(quirk, edid))
	86	return quirk->min_brightness;
	87	}
	88
	89	return -ENODATA;
	90	}
	91	EXPORT_SYMBOL(drm_get_panel_min_brightness_quirk);
	92
	93	MODULE_DESCRIPTION("Quirks for panel backlight overrides");
	94	MODULE_LICENSE("GPL");

drivers/gpu/drm/etnaviv/etnaviv_drv.c+0−1

@@ -503,7 +503,6 @@ static const struct drm_driver etnaviv_drm_driver = { · 502 unmodified lines
503	.fops = &fops,	503	.fops = &fops,
504	.name = "etnaviv",	504	.name = "etnaviv",
505	.desc = "etnaviv DRM",	505	.desc = "etnaviv DRM",
506	.date = "20151214",
507	.major = 1,	506	.major = 1,
508	.minor = 4,	507	.minor = 4,
509	};	508	};

drivers/gpu/drm/exynos/exynos_drm_drv.c+1−3

 #include <linux/pm_runtime.h>
 #include <linux/uaccess.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_file.h>
 #include <drm/drm_fourcc.h>
 #define DRIVER_NAME	"exynos"
 #define DRIVER_DESC	"Samsung SoC DRM"
-#define DRIVER_DATE	"20180330"
 /*
  * Interface history:
 	.fops			= &exynos_drm_driver_fops,
 	.name	= DRIVER_NAME,
 	.desc	= DRIVER_DESC,
-	.date	= DRIVER_DATE,
 	.major	= DRIVER_MAJOR,
 	.minor	= DRIVER_MINOR,
 };

drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c+1−2

 #include <linux/pm_runtime.h>
 #include <linux/regmap.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	.fops			= &fsl_dcu_drm_fops,
 	.name			= "fsl-dcu-drm",
 	.desc			= "Freescale DCU DRM",
-	.date			= "20160425",
 	.major			= 1,
 	.minor			= 1,
 };

drivers/gpu/drm/gma500/psb_drv.c+1−2

 #include <acpi/video.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_file.h>
 #include <drm/drm_ioctl.h>
 	.fops = &psb_gem_fops,
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL

drivers/gpu/drm/gma500/psb_drv.h+0−1

@@ -26,7 +26,6 @@ · 25 unmodified lines
26		26
27	#define DRIVER_NAME "gma500"	27	#define DRIVER_NAME "gma500"
28	#define DRIVER_DESC "DRM driver for the Intel GMA500, GMA600, GMA3600, GMA3650"	28	#define DRIVER_DESC "DRM driver for the Intel GMA500, GMA600, GMA3600, GMA3650"
29	#define DRIVER_DATE "20140314"
30		29
31	#define DRIVER_MAJOR 1	30	#define DRIVER_MAJOR 1
32	#define DRIVER_MINOR 0	31	#define DRIVER_MINOR 0

drivers/gpu/drm/gud/gud_drv.c+1−2

 #include <linux/vmalloc.h>
 #include <linux/workqueue.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_blend.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_debugfs.h>
 #include <drm/drm_drv.h>
 	.name			= "gud",
 	.desc			= "Generic USB Display",
-	.date			= "20200422",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/hisilicon/hibmc/Kconfig+1−1

@@ -1,7 +1,7 @@
1	# SPDX-License-Identifier: GPL-2.0-only	1	# SPDX-License-Identifier: GPL-2.0-only
2	config DRM_HISI_HIBMC	2	config DRM_HISI_HIBMC
3	tristate "DRM Support for Hisilicon Hibmc"	3	tristate "DRM Support for Hisilicon Hibmc"
4	depends on DRM && PCI && (ARM64 \|\| COMPILE_TEST)	4	depends on DRM && PCI
5	depends on MMU	5	depends on MMU
6	select DRM_CLIENT_SELECTION	6	select DRM_CLIENT_SELECTION
7	select DRM_KMS_HELPER	7	select DRM_KMS_HELPER

drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c+1−2

 #include <linux/module.h>
 #include <linux/pci.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_ttm.h>
 #include <drm/drm_gem_framebuffer_helper.h>
 	.driver_features	= DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC,
 	.fops			= &hibmc_fops,
 	.name			= "hibmc",
-	.date			= "20160828",
 	.desc			= "hibmc drm driver",
 	.major			= 1,
 	.minor			= 0,

drivers/gpu/drm/hisilicon/kirin/kirin_drm_ade.c+0−1

@@ -929,7 +929,6 @@ static const struct drm_driver ade_driver = { · 928 unmodified lines
929	DRM_FBDEV_DMA_DRIVER_OPS,	929	DRM_FBDEV_DMA_DRIVER_OPS,
930	.name = "kirin",	930	.name = "kirin",
931	.desc = "Hisilicon Kirin620 SoC DRM Driver",	931	.desc = "Hisilicon Kirin620 SoC DRM Driver",
932	.date = "20150718",
933	.major = 1,	932	.major = 1,
934	.minor = 0,	933	.minor = 0,
935	};	934	};

drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c+1−1

 #include <linux/of_graph.h>
 #include <linux/platform_device.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_gem_dma_helper.h>
 #include <drm/drm_gem_framebuffer_helper.h>

drivers/gpu/drm/hyperv/hyperv_drm_drv.c+1−3

 #include <linux/module.h>
 #include <linux/pci.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_shmem.h>
 #include <drm/drm_gem_shmem_helper.h>
 #define DRIVER_NAME "hyperv_drm"
 #define DRIVER_DESC "DRM driver for Hyper-V synthetic video device"
-#define DRIVER_DATE "2020"
 #define DRIVER_MAJOR 1
 #define DRIVER_MINOR 0
 	.name		 = DRIVER_NAME,
 	.desc		 = DRIVER_DESC,
-	.date		 = DRIVER_DATE,
 	.major		 = DRIVER_MAJOR,
 	.minor		 = DRIVER_MINOR,

drivers/gpu/drm/i915/i915_driver.c+0−1

@@ -1785,7 +1785,6 @@ static const struct drm_driver i915_drm_driver = { · 1784 unmodified lines
1785	.fops = &i915_driver_fops,	1785	.fops = &i915_driver_fops,
1786	.name = DRIVER_NAME,	1786	.name = DRIVER_NAME,
1787	.desc = DRIVER_DESC,	1787	.desc = DRIVER_DESC,
1788	.date = DRIVER_DATE,
1789	.major = DRIVER_MAJOR,	1788	.major = DRIVER_MAJOR,
1790	.minor = DRIVER_MINOR,	1789	.minor = DRIVER_MINOR,
1791	.patchlevel = DRIVER_PATCHLEVEL,	1790	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/i915/i915_driver.h+0−1

@@ -15,7 +15,6 @@ struct drm_printer; · 14 unmodified lines
15		15
16	#define DRIVER_NAME "i915"	16	#define DRIVER_NAME "i915"
17	#define DRIVER_DESC "Intel Graphics"	17	#define DRIVER_DESC "Intel Graphics"
18	#define DRIVER_DATE "20230929"
19	#define DRIVER_TIMESTAMP 1695980603	18	#define DRIVER_TIMESTAMP 1695980603
20		19
21	extern const struct dev_pm_ops i915_pm_ops;	20	extern const struct dev_pm_ops i915_pm_ops;

drivers/gpu/drm/i915/i915_gpu_error.c+0−1

@@ -841,7 +841,6 @@ static void __err_print_to_sgl(struct drm_i915_error_state_buf *m, · 840 unmodified lines
841	err_printf(m, "Kernel: %s %s\n",	841	err_printf(m, "Kernel: %s %s\n",
842	init_utsname()->release,	842	init_utsname()->release,
843	init_utsname()->machine);	843	init_utsname()->machine);
844	err_printf(m, "Driver: %s\n", DRIVER_DATE);
845	ts = ktime_to_timespec64(error->time);	844	ts = ktime_to_timespec64(error->time);
846	err_printf(m, "Time: %lld s %ld us\n",	845	err_printf(m, "Time: %lld s %ld us\n",
847	(s64)ts.tv_sec, ts.tv_nsec / NSEC_PER_USEC);	846	(s64)ts.tv_sec, ts.tv_nsec / NSEC_PER_USEC);

drivers/gpu/drm/imagination/pvr_drv.c+0−1

@@ -1387,7 +1387,6 @@ static struct drm_driver pvr_drm_driver = { · 1386 unmodified lines
1387		1387
1388	.name = PVR_DRIVER_NAME,	1388	.name = PVR_DRIVER_NAME,
1389	.desc = PVR_DRIVER_DESC,	1389	.desc = PVR_DRIVER_DESC,
1390	.date = PVR_DRIVER_DATE,
1391	.major = PVR_DRIVER_MAJOR,	1390	.major = PVR_DRIVER_MAJOR,
1392	.minor = PVR_DRIVER_MINOR,	1391	.minor = PVR_DRIVER_MINOR,
1393	.patchlevel = PVR_DRIVER_PATCHLEVEL,	1392	.patchlevel = PVR_DRIVER_PATCHLEVEL,

drivers/gpu/drm/imagination/pvr_drv.h+0−1

@@ -9,7 +9,6 @@ · 8 unmodified lines
9		9
10	#define PVR_DRIVER_NAME "powervr"	10	#define PVR_DRIVER_NAME "powervr"
11	#define PVR_DRIVER_DESC "Imagination PowerVR (Series 6 and later) & IMG Graphics"	11	#define PVR_DRIVER_DESC "Imagination PowerVR (Series 6 and later) & IMG Graphics"
12	#define PVR_DRIVER_DATE "20230904"
13		12
14	/*	13	/*
15	* Driver interface version:	14	* Driver interface version:

drivers/gpu/drm/imx/dcss/dcss-kms.c+1−2

  * Copyright 2019 NXP.
  */
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_bridge.h>
 #include <drm/drm_bridge_connector.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	.fops			= &dcss_cma_fops,
 	.name			= "imx-dcss",
 	.desc			= "i.MX8MQ Display Subsystem",
-	.date			= "20190917",
 	.major			= 1,
 	.minor			= 0,
 	.patchlevel		= 0,

drivers/gpu/drm/imx/ipuv3/imx-drm-core.c+1−2

 #include <video/imx-ipu-v3.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	.fops			= &imx_drm_driver_fops,
 	.name			= "imx-drm",
 	.desc			= "i.MX DRM graphics",
-	.date			= "20120507",
 	.major			= 1,
 	.minor			= 0,
 	.patchlevel		= 0,

drivers/gpu/drm/imx/lcdc/imx-lcdc.c+1−2

 // SPDX-License-Identifier: GPL-2.0-only
 // SPDX-FileCopyrightText: 2020 Marian Cichy <M.Cichy@pengutronix.de>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_bridge.h>
 #include <drm/drm_bridge_connector.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 	DRM_FBDEV_DMA_DRIVER_OPS,
 	.name = "imx-lcdc",
 	.desc = "i.MX LCDC driver",
-	.date = "20200716",
 };
 static const struct of_device_id imx_lcdc_of_dev_id[] = {

drivers/gpu/drm/ingenic/ingenic-drm-drv.c+1−2

 #include <linux/pm.h>
 #include <linux/regmap.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_bridge.h>
 #include <drm/drm_bridge_connector.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_color_mgmt.h>
 #include <drm/drm_crtc.h>
 #include <drm/drm_damage_helper.h>
 	.driver_features	= DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC,
 	.name			= "ingenic-drm",
 	.desc			= "DRM module for Ingenic SoCs",
-	.date			= "20200716",
 	.major			= 1,
 	.minor			= 1,
 	.patchlevel		= 0,

drivers/gpu/drm/kmb/kmb_drv.c+1−2

 #include <linux/pm_runtime.h>
 #include <linux/regmap.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	DRM_FBDEV_DMA_DRIVER_OPS,
 	.name = "kmb-drm",
 	.desc = "KEEMBAY DISPLAY DRIVER",
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 };

drivers/gpu/drm/kmb/kmb_drv.h+0−1

@@ -16,7 +16,6 @@ · 15 unmodified lines
16	#define KMB_MIN_WIDTH 1920 /Max width in pixels /	16	#define KMB_MIN_WIDTH 1920 /Max width in pixels /
17	#define KMB_MIN_HEIGHT 1080 /Max height in pixels /	17	#define KMB_MIN_HEIGHT 1080 /Max height in pixels /
18		18
19	#define DRIVER_DATE "20210223"
20	#define DRIVER_MAJOR 1	19	#define DRIVER_MAJOR 1
21	#define DRIVER_MINOR 1	20	#define DRIVER_MINOR 1
22		21

drivers/gpu/drm/lima/lima_drv.c+0−1

@@ -271,7 +271,6 @@ static const struct drm_driver lima_drm_driver = { · 270 unmodified lines
271	.fops = &lima_drm_driver_fops,	271	.fops = &lima_drm_driver_fops,
272	.name = "lima",	272	.name = "lima",
273	.desc = "lima DRM",	273	.desc = "lima DRM",
274	.date = "20191231",
275	.major = 1,	274	.major = 1,
276	.minor = 1,	275	.minor = 1,
277	.patchlevel = 0,	276	.patchlevel = 0,

drivers/gpu/drm/logicvc/logicvc_drm.c+1−2

 #include <linux/regmap.h>
 #include <linux/types.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_fourcc.h>
 	.fops				= &logicvc_drm_fops,
 	.name				= "logicvc-drm",
 	.desc				= "Xylon LogiCVC DRM driver",
-	.date				= "20200403",
 	.major				= 1,
 	.minor				= 0,

drivers/gpu/drm/loongson/lsdc_drv.c+4−6

 #include <linux/pci.h>
 #include <linux/vgaarb.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_ttm.h>
 #include <drm/drm_gem_framebuffer_helper.h>
 #define DRIVER_AUTHOR               "Sui Jingfeng <suijingfeng@loongson.cn>"
 #define DRIVER_NAME                 "loongson"
 #define DRIVER_DESC                 "drm driver for loongson graphics"
-#define DRIVER_DATE                 "20220701"
 #define DRIVER_MAJOR                1
 #define DRIVER_MINOR                0
 #define DRIVER_PATCHLEVEL           0
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,
 	lsdc_gem_init(ddev);
 	/* Bar 0 of the DC device contains the MMIO register's base address */
-	ldev->reg_base = pcim_iomap(pdev, 0, 0);
+	ldev->reg_base = pcim_iomap_region(pdev, 0, "lsdc");
-	if (!ldev->reg_base)
+	if (IS_ERR(ldev->reg_base))
-		return ERR_PTR(-ENODEV);
+		return ldev->reg_base;
 	spin_lock_init(&ldev->reglock);

drivers/gpu/drm/mcde/mcde_drv.c+1−2

 #include <linux/slab.h>
 #include <linux/delay.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_bridge.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fb_dma_helper.h>
 #include <drm/drm_fbdev_dma.h>
 	.fops = &drm_fops,
 	.name = "mcde",
 	.desc = DRIVER_DESC,
-	.date = "20180529",
 	.major = 1,
 	.minor = 0,
 	.patchlevel = 0,

drivers/gpu/drm/mediatek/mtk_drm_drv.c+1−3

 #include <linux/pm_runtime.h>
 #include <linux/dma-mapping.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_fourcc.h>
 #define DRIVER_NAME "mediatek"
 #define DRIVER_DESC "Mediatek SoC DRM"
-#define DRIVER_DATE "20150513"
 #define DRIVER_MAJOR 1
 #define DRIVER_MINOR 0
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 };

drivers/gpu/drm/meson/meson_drv.c+1−2

 #include <linux/platform_device.h>
 #include <linux/soc/amlogic/meson-canvas.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	.fops			= &fops,
 	.name			= DRIVER_NAME,
 	.desc			= DRIVER_DESC,
-	.date			= "20161109",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/mgag200/mgag200_drv.c+1−2

 #include <linux/module.h>
 #include <linux/pci.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_shmem.h>
 #include <drm/drm_file.h>
 	.fops = &mgag200_driver_fops,
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/mgag200/mgag200_drv.h+0−1

@@ -25,7 +25,6 @@ · 24 unmodified lines
25		25
26	#define DRIVER_NAME "mgag200"	26	#define DRIVER_NAME "mgag200"
27	#define DRIVER_DESC "MGA G200 SE"	27	#define DRIVER_DESC "MGA G200 SE"
28	#define DRIVER_DATE "20110418"
29		28
30	#define DRIVER_MAJOR 1	29	#define DRIVER_MAJOR 1
31	#define DRIVER_MINOR 0	30	#define DRIVER_MINOR 0

drivers/gpu/drm/msm/msm_drv.c+1−2

 #include <linux/of_address.h>
 #include <linux/uaccess.h>
-#include <drm/drm_client_setup.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_file.h>
 #include <drm/drm_ioctl.h>
 	.fops               = &fops,
 	.name               = "msm",
 	.desc               = "MSM Snapdragon DRM",
-	.date               = "20130625",
 	.major              = MSM_VERSION_MAJOR,
 	.minor              = MSM_VERSION_MINOR,
 	.patchlevel         = MSM_VERSION_PATCHLEVEL,

drivers/gpu/drm/mxsfb/lcdif_drv.c+1−2

 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_bridge.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_encoder.h>
 #include <drm/drm_fbdev_dma.h>
 	.fops	= &fops,
 	.name	= "imx-lcdif",
 	.desc	= "i.MX LCDIF Controller DRM",
-	.date	= "20220417",
 	.major	= 1,
 	.minor	= 0,
 };

drivers/gpu/drm/mxsfb/mxsfb_drv.c+1−2

 #include <linux/property.h>
 #include <linux/pm_runtime.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_bridge.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_connector.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 	.fops	= &fops,
 	.name	= "mxsfb-drm",
 	.desc	= "MXSFB Controller DRM",
-	.date	= "20160824",
 	.major	= 1,
 	.minor	= 0,
 };

drivers/gpu/drm/nouveau/include/nvif/log.h+51−0

@@ -0,0 +1,51 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/* SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. */
	3
	4	#ifndef __NVIF_LOG_H__
	5	#define __NVIF_LOG_H__
	6
	7	#ifdef CONFIG_DEBUG_FS
	8
	9	/**
	10	* nvif_log - structure for tracking logging buffers
	11	* @entry: an entry in a list of struct nvif_logs
	12	* @shutdown: pointer to function to call to clean up
	13	*
	14	* Structure used to track logging buffers so that they can be cleaned up
	15	* when the module exits.
	16	*
	17	* The @shutdown function is called when the module exits. It should free all
	18	* backing resources, such as logging buffers.
	19	*/
	20	struct nvif_log {
	21	struct list_head entry;
	22	void (shutdown)(struct nvif_log log);
	23	};
	24
	25	/**
	26	* nvif_logs - linked list of nvif_log objects
	27	*/
	28	struct nvif_logs {
	29	struct list_head head;
	30	};
	31
	32	#define NVIF_LOGS_DECLARE(logs) \
	33	struct nvif_logs logs = { LIST_HEAD_INIT(logs.head) }
	34
	35	static inline void nvif_log_shutdown(struct nvif_logs *logs)
	36	{
	37	if (!list_empty(&logs->head)) {
	38	struct nvif_log log, n;
	39
	40	list_for_each_entry_safe(log, n, &logs->head, entry) {
	41	/* shutdown() should also delete the log entry */
	42	log->shutdown(log);
	43	}
	44	}
	45	}
	46
	47	extern struct nvif_logs gsp_logs;
	48
	49	#endif
	50
	51	#endif

drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h+21−0

@@ -5,10 +5,13 @@ · 4 unmodified lines
5	#include <core/falcon.h>	5	#include <core/falcon.h>
6	#include <core/firmware.h>	6	#include <core/firmware.h>
7		7
		8	#include <linux/debugfs.h>
		9
8	#define GSP_PAGE_SHIFT 12	10	#define GSP_PAGE_SHIFT 12
9	#define GSP_PAGE_SIZE BIT(GSP_PAGE_SHIFT)	11	#define GSP_PAGE_SIZE BIT(GSP_PAGE_SHIFT)
10		12
11	struct nvkm_gsp_mem {	13	struct nvkm_gsp_mem {
		14	struct device *dev;
12	size_t size;	15	size_t size;
13	void *data;	16	void *data;
14	dma_addr_t addr;	17	dma_addr_t addr;
@@ -219,6 +222,24 @@ struct nvkm_gsp { · 204 unmodified lines
219		222
220	/* The size of the registry RPC */	223	/* The size of the registry RPC */
221	size_t registry_rpc_size;	224	size_t registry_rpc_size;
		225
		226	#ifdef CONFIG_DEBUG_FS
		227	/*
		228	* Logging buffers in debugfs. The wrapper objects need to remain
		229	* in memory until the dentry is deleted.
		230	*/
		231	struct {
		232	struct dentry *parent;
		233	struct dentry *init;
		234	struct dentry *rm;
		235	struct dentry *intr;
		236	struct dentry *pmu;
		237	} debugfs;
		238	struct debugfs_blob_wrapper blob_init;
		239	struct debugfs_blob_wrapper blob_intr;
		240	struct debugfs_blob_wrapper blob_rm;
		241	struct debugfs_blob_wrapper blob_pmu;
		242	#endif
222	};	243	};
223		244
224	static inline bool	245	static inline bool

drivers/gpu/drm/nouveau/nouveau_debugfs.c+16−0

@@ -313,3 +313,19 @@ nouveau_debugfs_fini(struct nouveau_drm *drm) · 312 unmodified lines
313	kfree(drm->debugfs);	313	kfree(drm->debugfs);
314	drm->debugfs = NULL;	314	drm->debugfs = NULL;
315	}	315	}
		316
		317	int
		318	nouveau_module_debugfs_init(void)
		319	{
		320	nouveau_debugfs_root = debugfs_create_dir("nouveau", NULL);
		321	if (IS_ERR(nouveau_debugfs_root))
		322	return PTR_ERR(nouveau_debugfs_root);
		323
		324	return 0;
		325	}
		326
		327	void
		328	nouveau_module_debugfs_fini(void)
		329	{
		330	debugfs_remove(nouveau_debugfs_root);
		331	}

drivers/gpu/drm/nouveau/nouveau_debugfs.h+16−0

@@ -21,6 +21,11 @@ nouveau_debugfs(struct drm_device *dev) · 20 unmodified lines
21	extern void nouveau_drm_debugfs_init(struct drm_minor *);	21	extern void nouveau_drm_debugfs_init(struct drm_minor *);
22	extern int nouveau_debugfs_init(struct nouveau_drm *);	22	extern int nouveau_debugfs_init(struct nouveau_drm *);
23	extern void nouveau_debugfs_fini(struct nouveau_drm *);	23	extern void nouveau_debugfs_fini(struct nouveau_drm *);
		24
		25	extern struct dentry *nouveau_debugfs_root;
		26
		27	int nouveau_module_debugfs_init(void);
		28	void nouveau_module_debugfs_fini(void);
24	#else	29	#else
25	static inline void	30	static inline void
26	nouveau_drm_debugfs_init(struct drm_minor *minor)	31	nouveau_drm_debugfs_init(struct drm_minor *minor)
@@ -37,6 +42,17 @@ nouveau_debugfs_fini(struct nouveau_drm *drm) · 10 unmodified lines
37	{	42	{
38	}	43	}
39		44
		45	static inline int
		46	nouveau_module_debugfs_init(void)
		47	{
		48	return 0;
		49	}
		50
		51	static inline void
		52	nouveau_module_debugfs_fini(void)
		53	{
		54	}
		55
40	#endif	56	#endif
41		57
42	#endif	58	#endif

drivers/gpu/drm/nouveau/nouveau_drm.c+36−9

 #include <linux/vga_switcheroo.h>
 #include <linux/mmu_notifier.h>
 #include <linux/dynamic_debug.h>
+#include <linux/debugfs.h>
-#include <drm/drm_client_setup.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_ttm.h>
 #include <drm/drm_gem_ttm_helper.h>
 #include <nvif/fifo.h>
 #include <nvif/push006c.h>
 #include <nvif/user.h>
+#include <nvif/log.h>
 #include <nvif/class.h>
 #include <nvif/cl0002.h>
 static struct drm_driver driver_pci;
 static struct drm_driver driver_platform;
+#ifdef CONFIG_DEBUG_FS
+struct dentry *nouveau_debugfs_root;
+/**
+ * gsp_logs - list of nvif_log GSP-RM logging buffers
+ *
+ * Head pointer to a a list of nvif_log buffers that is created for each GPU
+ * upon GSP shutdown if the "keep_gsp_logging" command-line parameter is
+ * specified. This is used to track the alternative debugfs entries for the
+ * GSP-RM logs.
+ */
+NVIF_LOGS_DECLARE(gsp_logs);
+#endif
 static u64
 nouveau_pci_name(struct pci_dev *pdev)
 {
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-#ifdef GIT_REVISION
-	.date = GIT_REVISION,
-#else
-	.date = DRIVER_DATE,
-#endif
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,
 static int __init
 nouveau_drm_init(void)
 {
+	int ret;
 	driver_pci = driver_stub;
 	driver_platform = driver_stub;
 	if (!nouveau_modeset)
 		return 0;
+	ret = nouveau_module_debugfs_init();
+	if (ret)
+		return ret;
 #ifdef CONFIG_NOUVEAU_PLATFORM_DRIVER
 	platform_driver_register(&nouveau_platform_driver);
 #endif
 	nouveau_backlight_ctor();
 #ifdef CONFIG_PCI
-	return pci_register_driver(&nouveau_drm_pci_driver);
+	ret = pci_register_driver(&nouveau_drm_pci_driver);
-#else
+	if (ret) {
-	return 0;
+		nouveau_module_debugfs_fini();
+		return ret;
+	}
 #endif
+	return 0;
 }
 static void __exit
 #endif
 	if (IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM))
 		mmu_notifier_synchronize();
+#ifdef CONFIG_DEBUG_FS
+	nvif_log_shutdown(&gsp_logs);
+#endif
+	nouveau_module_debugfs_fini();
 }
 module_init(nouveau_drm_init);

drivers/gpu/drm/nouveau/nouveau_drv.h+0−1

@@ -7,7 +7,6 @@ · 6 unmodified lines
7		7
8	#define DRIVER_NAME "nouveau"	8	#define DRIVER_NAME "nouveau"
9	#define DRIVER_DESC "nVidia Riva/TNT/GeForce/Quadro/Tesla/Tegra K1+"	9	#define DRIVER_DESC "nVidia Riva/TNT/GeForce/Quadro/Tesla/Tegra K1+"
10	#define DRIVER_DATE "20120801"
11		10
12	#define DRIVER_MAJOR 1	11	#define DRIVER_MAJOR 1
13	#define DRIVER_MINOR 4	12	#define DRIVER_MINOR 4

drivers/gpu/drm/nouveau/nvkm/subdev/gsp/r535.c+482−26

@@ -26,6 +26,7 @@ · 25 unmodified lines
26	#include <subdev/vfn.h>	26	#include <subdev/vfn.h>
27	#include <engine/fifo/chan.h>	27	#include <engine/fifo/chan.h>
28	#include <engine/sec2.h>	28	#include <engine/sec2.h>
		29	#include <nvif/log.h>
29		30
30	#include <nvfw/fw.h>	31	#include <nvfw/fw.h>
31		32
@@ -57,6 +58,8 @@ · 25 unmodified lines
57	#include <linux/ctype.h>	58	#include <linux/ctype.h>
58	#include <linux/parser.h>	59	#include <linux/parser.h>
59		60
		61	extern struct dentry *nouveau_debugfs_root;
		62
60	#define GSP_MSG_MIN_SIZE GSP_PAGE_SIZE	63	#define GSP_MSG_MIN_SIZE GSP_PAGE_SIZE
61	#define GSP_MSG_MAX_SIZE GSP_PAGE_MIN_SIZE * 16	64	#define GSP_MSG_MAX_SIZE GSP_PAGE_MIN_SIZE * 16
62		65
@@ -121,6 +124,8 @@ r535_gsp_msgq_wait(struct nvkm_gsp gsp, u32 repc, u32 prepc, int *ptime) · 58 unmodified lines
121	return mqe->data;	124	return mqe->data;
122	}	125	}
123		126
		127	size = ALIGN(repc + GSP_MSG_HDR_SIZE, GSP_PAGE_SIZE);
		128
124	msg = kvmalloc(repc, GFP_KERNEL);	129	msg = kvmalloc(repc, GFP_KERNEL);
125	if (!msg)	130	if (!msg)
126	return ERR_PTR(-ENOMEM);	131	return ERR_PTR(-ENOMEM);
@@ -129,19 +134,15 @@ r535_gsp_msgq_wait(struct nvkm_gsp gsp, u32 repc, u32 prepc, int *ptime) · 2 unmodified lines
129	len = min_t(u32, repc, len);	134	len = min_t(u32, repc, len);
130	memcpy(msg, mqe->data, len);	135	memcpy(msg, mqe->data, len);
131		136
132	rptr += DIV_ROUND_UP(len, GSP_PAGE_SIZE);
133	if (rptr == gsp->msgq.cnt)
134	rptr = 0;
135
136	repc -= len;	137	repc -= len;
137		138
138	if (repc) {	139	if (repc) {
139	mqe = (void )((u8 )gsp->shm.msgq.ptr + 0x1000 + 0 * 0x1000);	140	mqe = (void )((u8 )gsp->shm.msgq.ptr + 0x1000 + 0 * 0x1000);
140	memcpy(msg + len, mqe, repc);	141	memcpy(msg + len, mqe, repc);
141
142	rptr += DIV_ROUND_UP(repc, GSP_PAGE_SIZE);
143	}	142	}
144		143
		144	rptr = (rptr + DIV_ROUND_UP(size, GSP_PAGE_SIZE)) % gsp->msgq.cnt;
		145
145	mb();	146	mb();
146	(*gsp->msgq.rptr) = rptr;	147	(*gsp->msgq.rptr) = rptr;
147	return msg;	148	return msg;
@@ -163,7 +164,7 @@ r535_gsp_cmdq_push(struct nvkm_gsp gsp, void argv) · 15 unmodified lines
163	u64 *end;	164	u64 *end;
164	u64 csum = 0;	165	u64 csum = 0;
165	int free, time = 1000000;	166	int free, time = 1000000;
166	u32 wptr, size;	167	u32 wptr, size, step;
167	u32 off = 0;	168	u32 off = 0;
168		169
169	argc = ALIGN(GSP_MSG_HDR_SIZE + argc, GSP_PAGE_SIZE);	170	argc = ALIGN(GSP_MSG_HDR_SIZE + argc, GSP_PAGE_SIZE);
@@ -197,7 +198,9 @@ r535_gsp_cmdq_push(struct nvkm_gsp gsp, void argv) · 27 unmodified lines
197	}	198	}
198		199
199	cqe = (void )((u8 )gsp->shm.cmdq.ptr + 0x1000 + wptr * 0x1000);	200	cqe = (void )((u8 )gsp->shm.cmdq.ptr + 0x1000 + wptr * 0x1000);
200	size = min_t(u32, argc, (gsp->cmdq.cnt - wptr) * GSP_PAGE_SIZE);	201	step = min_t(u32, free, (gsp->cmdq.cnt - wptr));
		202	size = min_t(u32, argc, step * GSP_PAGE_SIZE);
		203
201	memcpy(cqe, (u8 *)cmd + off, size);	204	memcpy(cqe, (u8 *)cmd + off, size);
202		205
203	wptr += DIV_ROUND_UP(size, 0x1000);	206	wptr += DIV_ROUND_UP(size, 0x1000);
@@ -1000,7 +1003,7 @@ r535_gsp_rpc_get_gsp_static_info(struct nvkm_gsp *gsp) · 796 unmodified lines
1000	}	1003	}
1001		1004
1002	static void	1005	static void
1003	nvkm_gsp_mem_dtor(struct nvkm_gsp gsp, struct nvkm_gsp_mem mem)	1006	nvkm_gsp_mem_dtor(struct nvkm_gsp_mem *mem)
1004	{	1007	{
1005	if (mem->data) {	1008	if (mem->data) {
1006	/*	1009	/*
@@ -1009,19 +1012,35 @@ nvkm_gsp_mem_dtor(struct nvkm_gsp gsp, struct nvkm_gsp_mem mem) · 2 unmodified lines
1009	*/	1012	*/
1010	memset(mem->data, 0xFF, mem->size);	1013	memset(mem->data, 0xFF, mem->size);
1011		1014
1012	dma_free_coherent(gsp->subdev.device->dev, mem->size, mem->data, mem->addr);	1015	dma_free_coherent(mem->dev, mem->size, mem->data, mem->addr);
		1016	put_device(mem->dev);
		1017
1013	memset(mem, 0, sizeof(*mem));	1018	memset(mem, 0, sizeof(*mem));
1014	}	1019	}
1015	}	1020	}
1016		1021
		1022	/**
		1023	* nvkm_gsp_mem_ctor - constructor for nvkm_gsp_mem objects
		1024	* @gsp: gsp pointer
		1025	* @size: number of bytes to allocate
		1026	* @mem: nvkm_gsp_mem object to initialize
		1027	*
		1028	* Allocates a block of memory for use with GSP.
		1029	*
		1030	* This memory block can potentially out-live the driver's remove() callback,
		1031	* so we take a device reference to ensure its lifetime. The reference is
		1032	* dropped in the destructor.
		1033	*/
1017	static int	1034	static int
1018	nvkm_gsp_mem_ctor(struct nvkm_gsp gsp, size_t size, struct nvkm_gsp_mem mem)	1035	nvkm_gsp_mem_ctor(struct nvkm_gsp gsp, size_t size, struct nvkm_gsp_mem mem)
1019	{	1036	{
1020	mem->size = size;
1021	mem->data = dma_alloc_coherent(gsp->subdev.device->dev, size, &mem->addr, GFP_KERNEL);	1037	mem->data = dma_alloc_coherent(gsp->subdev.device->dev, size, &mem->addr, GFP_KERNEL);
1022	if (WARN_ON(!mem->data))	1038	if (WARN_ON(!mem->data))
1023	return -ENOMEM;	1039	return -ENOMEM;
1024		1040
		1041	mem->size = size;
		1042	mem->dev = get_device(gsp->subdev.device->dev);
		1043
1025	return 0;	1044	return 0;
1026	}	1045	}
1027		1046
@@ -1054,8 +1073,8 @@ r535_gsp_postinit(struct nvkm_gsp *gsp) · 26 unmodified lines
1054	nvkm_wr32(device, 0x110004, 0x00000040);	1073	nvkm_wr32(device, 0x110004, 0x00000040);
1055		1074
1056	/* Release the DMA buffers that were needed only for boot and init */	1075	/* Release the DMA buffers that were needed only for boot and init */
1057	nvkm_gsp_mem_dtor(gsp, &gsp->boot.fw);	1076	nvkm_gsp_mem_dtor(&gsp->boot.fw);
1058	nvkm_gsp_mem_dtor(gsp, &gsp->libos);	1077	nvkm_gsp_mem_dtor(&gsp->libos);
1059		1078
1060	return ret;	1079	return ret;
1061	}	1080	}
@@ -2060,6 +2079,215 @@ r535_gsp_rmargs_init(struct nvkm_gsp *gsp, bool resume) · 998 unmodified lines
2060	return 0;	2079	return 0;
2061	}	2080	}
2062		2081
		2082	#ifdef CONFIG_DEBUG_FS
		2083
		2084	/*
		2085	* If GSP-RM load fails, then the GSP nvkm object will be deleted, the logging
		2086	* debugfs entries will be deleted, and it will not be possible to debug the
		2087	* load failure. The keep_gsp_logging parameter tells Nouveau to copy the
		2088	* logging buffers to new debugfs entries, and these entries are retained
		2089	* until the driver unloads.
		2090	*/
		2091	static bool keep_gsp_logging;
		2092	module_param(keep_gsp_logging, bool, 0444);
		2093	MODULE_PARM_DESC(keep_gsp_logging,
		2094	"Migrate the GSP-RM logging debugfs entries upon exit");
		2095
		2096	/*
		2097	* GSP-RM uses a pseudo-class mechanism to define of a variety of per-"engine"
		2098	* data structures, and each engine has a "class ID" genererated by a
		2099	* pre-processor. This is the class ID for the PMU.
		2100	*/
		2101	#define NV_GSP_MSG_EVENT_UCODE_LIBOS_CLASS_PMU 0xf3d722
		2102
		2103	/**
		2104	* rpc_ucode_libos_print_v1E_08 - RPC payload for libos print buffers
		2105	* @ucode_eng_desc: the engine descriptor
		2106	* @libos_print_buf_size: the size of the libos_print_buf[]
		2107	* @libos_print_buf: the actual buffer
		2108	*
		2109	* The engine descriptor is divided into 31:8 "class ID" and 7:0 "instance
		2110	* ID". We only care about messages from PMU.
		2111	*/
		2112	struct rpc_ucode_libos_print_v1e_08 {
		2113	u32 ucode_eng_desc;
		2114	u32 libos_print_buf_size;
		2115	u8 libos_print_buf[];
		2116	};
		2117
		2118	/**
		2119	* r535_gsp_msg_libos_print - capture log message from the PMU
		2120	* @priv: gsp pointer
		2121	* @fn: function number (ignored)
		2122	* @repv: pointer to libos print RPC
		2123	* @repc: message size
		2124	*
		2125	* Called when we receive a UCODE_LIBOS_PRINT event RPC from GSP-RM. This RPC
		2126	* contains the contents of the libos print buffer from PMU. It is typically
		2127	* only written to when PMU encounters an error.
		2128	*
		2129	* Technically this RPC can be used to pass print buffers from any number of
		2130	* GSP-RM engines, but we only expect to receive them for the PMU.
		2131	*
		2132	* For the PMU, the buffer is 4K in size and the RPC always contains the full
		2133	* contents.
		2134	*/
		2135	static int
		2136	r535_gsp_msg_libos_print(void priv, u32 fn, void repv, u32 repc)
		2137	{
		2138	struct nvkm_gsp *gsp = priv;
		2139	struct nvkm_subdev *subdev = &gsp->subdev;
		2140	struct rpc_ucode_libos_print_v1e_08 *rpc = repv;
		2141	unsigned int class = rpc->ucode_eng_desc >> 8;
		2142
		2143	nvkm_debug(subdev, "received libos print from class 0x%x for %u bytes\n",
		2144	class, rpc->libos_print_buf_size);
		2145
		2146	if (class != NV_GSP_MSG_EVENT_UCODE_LIBOS_CLASS_PMU) {
		2147	nvkm_warn(subdev,
		2148	"received libos print from unknown class 0x%x\n",
		2149	class);
		2150	return -ENOMSG;
		2151	}
		2152
		2153	if (rpc->libos_print_buf_size > GSP_PAGE_SIZE) {
		2154	nvkm_error(subdev, "libos print is too large (%u bytes)\n",
		2155	rpc->libos_print_buf_size);
		2156	return -E2BIG;
		2157	}
		2158
		2159	memcpy(gsp->blob_pmu.data, rpc->libos_print_buf, rpc->libos_print_buf_size);
		2160
		2161	return 0;
		2162	}
		2163
		2164	/**
		2165	* create_debufgs - create a blob debugfs entry
		2166	* @gsp: gsp pointer
		2167	* @name: name of this dentry
		2168	* @blob: blob wrapper
		2169	*
		2170	* Creates a debugfs entry for a logging buffer with the name 'name'.
		2171	*/
		2172	static struct dentry create_debugfs(struct nvkm_gsp gsp, const char *name,
		2173	struct debugfs_blob_wrapper *blob)
		2174	{
		2175	struct dentry *dent;
		2176
		2177	dent = debugfs_create_blob(name, 0444, gsp->debugfs.parent, blob);
		2178	if (IS_ERR(dent)) {
		2179	nvkm_error(&gsp->subdev,
		2180	"failed to create %s debugfs entry\n", name);
		2181	return NULL;
		2182	}
		2183
		2184	/*
		2185	* For some reason, debugfs_create_blob doesn't set the size of the
		2186	* dentry, so do that here. See [1]
		2187	*
		2188	* [1] https://lore.kernel.org/r/linux-fsdevel/20240207200619.3354549-1-ttabi@nvidia.com/
		2189	*/
		2190	i_size_write(d_inode(dent), blob->size);
		2191
		2192	return dent;
		2193	}
		2194
		2195	/**
		2196	* r535_gsp_libos_debugfs_init - create logging debugfs entries
		2197	* @gsp: gsp pointer
		2198	*
		2199	* Create the debugfs entries. This exposes the log buffers to userspace so
		2200	* that an external tool can parse it.
		2201	*
		2202	* The 'logpmu' contains exception dumps from the PMU. It is written via an
		2203	* RPC sent from GSP-RM and must be only 4KB. We create it here because it's
		2204	* only useful if there is a debugfs entry to expose it. If we get the PMU
		2205	* logging RPC and there is no debugfs entry, the RPC is just ignored.
		2206	*
		2207	* The blob_init, blob_rm, and blob_pmu objects can't be transient
		2208	* because debugfs_create_blob doesn't copy them.
		2209	*
		2210	* NOTE: OpenRM loads the logging elf image and prints the log messages
		2211	* in real-time. We may add that capability in the future, but that
		2212	* requires loading ELF images that are not distributed with the driver and
		2213	* adding the parsing code to Nouveau.
		2214	*
		2215	* Ideally, this should be part of nouveau_debugfs_init(), but that function
		2216	* is called too late. We really want to create these debugfs entries before
		2217	* r535_gsp_booter_load() is called, so that if GSP-RM fails to initialize,
		2218	* there could still be a log to capture.
		2219	*/
		2220	static void
		2221	r535_gsp_libos_debugfs_init(struct nvkm_gsp *gsp)
		2222	{
		2223	struct device *dev = gsp->subdev.device->dev;
		2224
		2225	/* Create a new debugfs directory with a name unique to this GPU. */
		2226	gsp->debugfs.parent = debugfs_create_dir(dev_name(dev), nouveau_debugfs_root);
		2227	if (IS_ERR(gsp->debugfs.parent)) {
		2228	nvkm_error(&gsp->subdev,
		2229	"failed to create %s debugfs root\n", dev_name(dev));
		2230	return;
		2231	}
		2232
		2233	gsp->blob_init.data = gsp->loginit.data;
		2234	gsp->blob_init.size = gsp->loginit.size;
		2235	gsp->blob_intr.data = gsp->logintr.data;
		2236	gsp->blob_intr.size = gsp->logintr.size;
		2237	gsp->blob_rm.data = gsp->logrm.data;
		2238	gsp->blob_rm.size = gsp->logrm.size;
		2239
		2240	gsp->debugfs.init = create_debugfs(gsp, "loginit", &gsp->blob_init);
		2241	if (!gsp->debugfs.init)
		2242	goto error;
		2243
		2244	gsp->debugfs.intr = create_debugfs(gsp, "logintr", &gsp->blob_intr);
		2245	if (!gsp->debugfs.intr)
		2246	goto error;
		2247
		2248	gsp->debugfs.rm = create_debugfs(gsp, "logrm", &gsp->blob_rm);
		2249	if (!gsp->debugfs.rm)
		2250	goto error;
		2251
		2252	/*
		2253	* Since the PMU buffer is copied from an RPC, it doesn't need to be
		2254	* a DMA buffer.
		2255	*/
		2256	gsp->blob_pmu.size = GSP_PAGE_SIZE;
		2257	gsp->blob_pmu.data = kzalloc(gsp->blob_pmu.size, GFP_KERNEL);
		2258	if (!gsp->blob_pmu.data)
		2259	goto error;
		2260
		2261	gsp->debugfs.pmu = create_debugfs(gsp, "logpmu", &gsp->blob_pmu);
		2262	if (!gsp->debugfs.pmu) {
		2263	kfree(gsp->blob_pmu.data);
		2264	goto error;
		2265	}
		2266
		2267	i_size_write(d_inode(gsp->debugfs.init), gsp->blob_init.size);
		2268	i_size_write(d_inode(gsp->debugfs.intr), gsp->blob_intr.size);
		2269	i_size_write(d_inode(gsp->debugfs.rm), gsp->blob_rm.size);
		2270	i_size_write(d_inode(gsp->debugfs.pmu), gsp->blob_pmu.size);
		2271
		2272	r535_gsp_msg_ntfy_add(gsp, NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT,
		2273	r535_gsp_msg_libos_print, gsp);
		2274
		2275	nvkm_debug(&gsp->subdev, "created debugfs GSP-RM logging entries\n");
		2276
		2277	if (keep_gsp_logging) {
		2278	nvkm_info(&gsp->subdev,
		2279	"logging buffers will be retained on failure\n");
		2280	}
		2281
		2282	return;
		2283
		2284	error:
		2285	debugfs_remove(gsp->debugfs.parent);
		2286	gsp->debugfs.parent = NULL;
		2287	}
		2288
		2289	#endif
		2290
2063	static inline u64	2291	static inline u64
2064	r535_gsp_libos_id8(const char *name)	2292	r535_gsp_libos_id8(const char *name)
2065	{	2293	{
@@ -2110,7 +2338,11 @@ static void create_pte_array(u64 *ptes, dma_addr_t addr, size_t size) · 44 unmodified lines
2110	* written to directly by GSP-RM and can be any multiple of GSP_PAGE_SIZE.	2338	* written to directly by GSP-RM and can be any multiple of GSP_PAGE_SIZE.
2111	*	2339	*
2112	* The physical address map for the log buffer is stored in the buffer	2340	* The physical address map for the log buffer is stored in the buffer
2113	* itself, starting with offset 1. Offset 0 contains the "put" pointer.	2341	* itself, starting with offset 1. Offset 0 contains the "put" pointer (pp).
		2342	* Initially, pp is equal to 0. If the buffer has valid logging data in it,
		2343	* then pp points to index into the buffer where the next logging entry will
		2344	* be written. Therefore, the logging data is valid if:
		2345	* 1 <= pp < sizeof(buffer)/sizeof(u64)
2114	*	2346	*
2115	* The GSP only understands 4K pages (GSP_PAGE_SIZE), so even if the kernel is	2347	* The GSP only understands 4K pages (GSP_PAGE_SIZE), so even if the kernel is
2116	* configured for a larger page size (e.g. 64K pages), we need to give	2348	* configured for a larger page size (e.g. 64K pages), we need to give
@@ -2181,6 +2413,11 @@ r535_gsp_libos_init(struct nvkm_gsp *gsp) · 64 unmodified lines
2181	args[3].size = gsp->rmargs.size;	2413	args[3].size = gsp->rmargs.size;
2182	args[3].kind = LIBOS_MEMORY_REGION_CONTIGUOUS;	2414	args[3].kind = LIBOS_MEMORY_REGION_CONTIGUOUS;
2183	args[3].loc = LIBOS_MEMORY_REGION_LOC_SYSMEM;	2415	args[3].loc = LIBOS_MEMORY_REGION_LOC_SYSMEM;
		2416
		2417	#ifdef CONFIG_DEBUG_FS
		2418	r535_gsp_libos_debugfs_init(gsp);
		2419	#endif
		2420
2184	return 0;	2421	return 0;
2185	}	2422	}
2186		2423
@@ -2234,8 +2471,8 @@ static void · 47 unmodified lines
2234	nvkm_gsp_radix3_dtor(struct nvkm_gsp gsp, struct nvkm_gsp_radix3 rx3)	2471	nvkm_gsp_radix3_dtor(struct nvkm_gsp gsp, struct nvkm_gsp_radix3 rx3)
2235	{	2472	{
2236	nvkm_gsp_sg_free(gsp->subdev.device, &rx3->lvl2);	2473	nvkm_gsp_sg_free(gsp->subdev.device, &rx3->lvl2);
2237	nvkm_gsp_mem_dtor(gsp, &rx3->lvl1);	2474	nvkm_gsp_mem_dtor(&rx3->lvl1);
2238	nvkm_gsp_mem_dtor(gsp, &rx3->lvl0);	2475	nvkm_gsp_mem_dtor(&rx3->lvl0);
2239	}	2476	}
2240		2477
2241	/**	2478	/**
@@ -2323,9 +2560,9 @@ nvkm_gsp_radix3_sg(struct nvkm_gsp gsp, struct sg_table sgt, u64 size, · 81 unmodified lines
2323		2560
2324	if (ret) {	2561	if (ret) {
2325	lvl2_fail:	2562	lvl2_fail:
2326	nvkm_gsp_mem_dtor(gsp, &rx3->lvl1);	2563	nvkm_gsp_mem_dtor(&rx3->lvl1);
2327	lvl1_fail:	2564	lvl1_fail:
2328	nvkm_gsp_mem_dtor(gsp, &rx3->lvl0);	2565	nvkm_gsp_mem_dtor(&rx3->lvl0);
2329	}	2566	}
2330		2567
2331	return ret;	2568	return ret;
@@ -2417,7 +2654,7 @@ r535_gsp_init(struct nvkm_gsp *gsp) · 85 unmodified lines
2417		2654
2418	done:	2655	done:
2419	if (gsp->sr.meta.data) {	2656	if (gsp->sr.meta.data) {
2420	nvkm_gsp_mem_dtor(gsp, &gsp->sr.meta);	2657	nvkm_gsp_mem_dtor(&gsp->sr.meta);
2421	nvkm_gsp_radix3_dtor(gsp, &gsp->sr.radix3);	2658	nvkm_gsp_radix3_dtor(gsp, &gsp->sr.radix3);
2422	nvkm_gsp_sg_free(gsp->subdev.device, &gsp->sr.sgt);	2659	nvkm_gsp_sg_free(gsp->subdev.device, &gsp->sr.sgt);
2423	return ret;	2660	return ret;
@@ -2491,6 +2728,222 @@ r535_gsp_dtor_fws(struct nvkm_gsp *gsp) · 67 unmodified lines
2491	gsp->fws.rm = NULL;	2728	gsp->fws.rm = NULL;
2492	}	2729	}
2493		2730
		2731	#ifdef CONFIG_DEBUG_FS
		2732
		2733	struct r535_gsp_log {
		2734	struct nvif_log log;
		2735
		2736	/*
		2737	* Logging buffers in debugfs. The wrapper objects need to remain
		2738	* in memory until the dentry is deleted.
		2739	*/
		2740	struct dentry *debugfs_logging_dir;
		2741	struct debugfs_blob_wrapper blob_init;
		2742	struct debugfs_blob_wrapper blob_intr;
		2743	struct debugfs_blob_wrapper blob_rm;
		2744	struct debugfs_blob_wrapper blob_pmu;
		2745	};
		2746
		2747	/**
		2748	* r535_debugfs_shutdown - delete GSP-RM logging buffers for one GPU
		2749	* @_log: nvif_log struct for this GPU
		2750	*
		2751	* Called when the driver is shutting down, to clean up the retained GSP-RM
		2752	* logging buffers.
		2753	*/
		2754	static void r535_debugfs_shutdown(struct nvif_log *_log)
		2755	{
		2756	struct r535_gsp_log *log = container_of(_log, struct r535_gsp_log, log);
		2757
		2758	debugfs_remove(log->debugfs_logging_dir);
		2759
		2760	kfree(log->blob_init.data);
		2761	kfree(log->blob_intr.data);
		2762	kfree(log->blob_rm.data);
		2763	kfree(log->blob_pmu.data);
		2764
		2765	/* We also need to delete the list object */
		2766	kfree(log);
		2767	}
		2768
		2769	/**
		2770	* is_empty - return true if the logging buffer was never written to
		2771	* @b: blob wrapper with ->data field pointing to logging buffer
		2772	*
		2773	* The first 64-bit field of loginit, and logintr, and logrm is the 'put'
		2774	* pointer, and it is initialized to 0. It's a dword-based index into the
		2775	* circular buffer, indicating where the next printf write will be made.
		2776	*
		2777	* If the pointer is still 0 when GSP-RM is shut down, that means that the
		2778	* buffer was never written to, so it can be ignored.
		2779	*
		2780	* This test also works for logpmu, even though it doesn't have a put pointer.
		2781	*/
		2782	static bool is_empty(const struct debugfs_blob_wrapper *b)
		2783	{
		2784	u64 *put = b->data;
		2785
		2786	return put ? (*put == 0) : true;
		2787	}
		2788
		2789	/**
		2790	* r535_gsp_copy_log - preserve the logging buffers in a blob
		2791	*
		2792	* When GSP shuts down, the nvkm_gsp object and all its memory is deleted.
		2793	* To preserve the logging buffers, the buffers need to be copied, but only
		2794	* if they actually have data.
		2795	*/
		2796	static int r535_gsp_copy_log(struct dentry *parent,
		2797	const char *name,
		2798	const struct debugfs_blob_wrapper *s,
		2799	struct debugfs_blob_wrapper *t)
		2800	{
		2801	struct dentry *dent;
		2802	void *p;
		2803
		2804	if (is_empty(s))
		2805	return 0;
		2806
		2807	/* The original buffers will be deleted */
		2808	p = kmemdup(s->data, s->size, GFP_KERNEL);
		2809	if (!p)
		2810	return -ENOMEM;
		2811
		2812	t->data = p;
		2813	t->size = s->size;
		2814
		2815	dent = debugfs_create_blob(name, 0444, parent, t);
		2816	if (IS_ERR(dent)) {
		2817	kfree(p);
		2818	memset(t, 0, sizeof(*t));
		2819	return PTR_ERR(dent);
		2820	}
		2821
		2822	i_size_write(d_inode(dent), t->size);
		2823
		2824	return 0;
		2825	}
		2826
		2827	/**
		2828	* r535_gsp_retain_logging - copy logging buffers to new debugfs root
		2829	* @gsp: gsp pointer
		2830	*
		2831	* If keep_gsp_logging is enabled, then we want to preserve the GSP-RM logging
		2832	* buffers and their debugfs entries, but all those objects would normally
		2833	* deleted if GSP-RM fails to load.
		2834	*
		2835	* To preserve the logging buffers, we need to:
		2836	*
		2837	* 1) Allocate new buffers and copy the logs into them, so that the original
		2838	* DMA buffers can be released.
		2839	*
		2840	* 2) Preserve the directories. We don't need to save single dentries because
		2841	* we're going to delete the parent when the
		2842	*
		2843	* If anything fails in this process, then all the dentries need to be
		2844	* deleted. We don't need to deallocate the original logging buffers because
		2845	* the caller will do that regardless.
		2846	*/
		2847	static void r535_gsp_retain_logging(struct nvkm_gsp *gsp)
		2848	{
		2849	struct device *dev = gsp->subdev.device->dev;
		2850	struct r535_gsp_log *log = NULL;
		2851	int ret;
		2852
		2853	if (!keep_gsp_logging \|\| !gsp->debugfs.parent) {
		2854	/* Nothing to do */
		2855	goto exit;
		2856	}
		2857
		2858	/* Check to make sure at least one buffer has data. */
		2859	if (is_empty(&gsp->blob_init) && is_empty(&gsp->blob_intr) &&
		2860	is_empty(&gsp->blob_rm) && is_empty(&gsp->blob_rm)) {
		2861	nvkm_warn(&gsp->subdev, "all logging buffers are empty\n");
		2862	goto exit;
		2863	}
		2864
		2865	log = kzalloc(sizeof(*log), GFP_KERNEL);
		2866	if (!log)
		2867	goto error;
		2868
		2869	/*
		2870	* Since the nvkm_gsp object is going away, the debugfs_blob_wrapper
		2871	* objects are also being deleted, which means the dentries will no
		2872	* longer be valid. Delete the existing entries so that we can create
		2873	* new ones with the same name.
		2874	*/
		2875	debugfs_remove(gsp->debugfs.init);
		2876	debugfs_remove(gsp->debugfs.intr);
		2877	debugfs_remove(gsp->debugfs.rm);
		2878	debugfs_remove(gsp->debugfs.pmu);
		2879
		2880	ret = r535_gsp_copy_log(gsp->debugfs.parent, "loginit", &gsp->blob_init, &log->blob_init);
		2881	if (ret)
		2882	goto error;
		2883
		2884	ret = r535_gsp_copy_log(gsp->debugfs.parent, "logintr", &gsp->blob_intr, &log->blob_intr);
		2885	if (ret)
		2886	goto error;
		2887
		2888	ret = r535_gsp_copy_log(gsp->debugfs.parent, "logrm", &gsp->blob_rm, &log->blob_rm);
		2889	if (ret)
		2890	goto error;
		2891
		2892	ret = r535_gsp_copy_log(gsp->debugfs.parent, "logpmu", &gsp->blob_pmu, &log->blob_pmu);
		2893	if (ret)
		2894	goto error;
		2895
		2896	/* The nvkm_gsp object is going away, so save the dentry */
		2897	log->debugfs_logging_dir = gsp->debugfs.parent;
		2898
		2899	log->log.shutdown = r535_debugfs_shutdown;
		2900	list_add(&log->log.entry, &gsp_logs.head);
		2901
		2902	nvkm_warn(&gsp->subdev,
		2903	"logging buffers migrated to /sys/kernel/debug/nouveau/%s\n",
		2904	dev_name(dev));
		2905
		2906	return;
		2907
		2908	error:
		2909	nvkm_warn(&gsp->subdev, "failed to migrate logging buffers\n");
		2910
		2911	exit:
		2912	debugfs_remove(gsp->debugfs.parent);
		2913
		2914	if (log) {
		2915	kfree(log->blob_init.data);
		2916	kfree(log->blob_intr.data);
		2917	kfree(log->blob_rm.data);
		2918	kfree(log->blob_pmu.data);
		2919	kfree(log);
		2920	}
		2921	}
		2922
		2923	#endif
		2924
		2925	/**
		2926	* r535_gsp_libos_debugfs_fini - cleanup/retain log buffers on shutdown
		2927	* @gsp: gsp pointer
		2928	*
		2929	* If the log buffers are exposed via debugfs, the data for those entries
		2930	* needs to be cleaned up when the GSP device shuts down.
		2931	*/
		2932	static void
		2933	r535_gsp_libos_debugfs_fini(struct nvkm_gsp __maybe_unused *gsp)
		2934	{
		2935	#ifdef CONFIG_DEBUG_FS
		2936	r535_gsp_retain_logging(gsp);
		2937
		2938	/*
		2939	* Unlike the other buffers, the PMU blob is a kmalloc'd buffer that
		2940	* exists only if the debugfs entries were created.
		2941	*/
		2942	kfree(gsp->blob_pmu.data);
		2943	gsp->blob_pmu.data = NULL;
		2944	#endif
		2945	}
		2946
2494	void	2947	void
2495	r535_gsp_dtor(struct nvkm_gsp *gsp)	2948	r535_gsp_dtor(struct nvkm_gsp *gsp)
2496	{	2949	{
@@ -2498,7 +2951,7 @@ r535_gsp_dtor(struct nvkm_gsp *gsp) · 1 unmodified line
2498	mutex_destroy(&gsp->client_id.mutex);	2951	mutex_destroy(&gsp->client_id.mutex);
2499		2952
2500	nvkm_gsp_radix3_dtor(gsp, &gsp->radix3);	2953	nvkm_gsp_radix3_dtor(gsp, &gsp->radix3);
2501	nvkm_gsp_mem_dtor(gsp, &gsp->sig);	2954	nvkm_gsp_mem_dtor(&gsp->sig);
2502	nvkm_firmware_dtor(&gsp->fw);	2955	nvkm_firmware_dtor(&gsp->fw);
2503		2956
2504	nvkm_falcon_fw_dtor(&gsp->booter.unload);	2957	nvkm_falcon_fw_dtor(&gsp->booter.unload);
@@ -2509,12 +2962,15 @@ r535_gsp_dtor(struct nvkm_gsp *gsp) · 4 unmodified lines
2509		2962
2510	r535_gsp_dtor_fws(gsp);	2963	r535_gsp_dtor_fws(gsp);
2511		2964
2512	nvkm_gsp_mem_dtor(gsp, &gsp->rmargs);	2965	nvkm_gsp_mem_dtor(&gsp->rmargs);
2513	nvkm_gsp_mem_dtor(gsp, &gsp->wpr_meta);	2966	nvkm_gsp_mem_dtor(&gsp->wpr_meta);
2514	nvkm_gsp_mem_dtor(gsp, &gsp->shm.mem);	2967	nvkm_gsp_mem_dtor(&gsp->shm.mem);
2515	nvkm_gsp_mem_dtor(gsp, &gsp->loginit);	2968
2516	nvkm_gsp_mem_dtor(gsp, &gsp->logintr);	2969	r535_gsp_libos_debugfs_fini(gsp);
2517	nvkm_gsp_mem_dtor(gsp, &gsp->logrm);	2970
		2971	nvkm_gsp_mem_dtor(&gsp->loginit);
		2972	nvkm_gsp_mem_dtor(&gsp->logintr);
		2973	nvkm_gsp_mem_dtor(&gsp->logrm);
2518	}	2974	}
2519		2975
2520	int	2976	int

drivers/gpu/drm/omapdrm/omap_drv.c+0−2

 #define DRIVER_NAME		MODULE_NAME
 #define DRIVER_DESC		"OMAP DRM"
-#define DRIVER_DATE		"20110917"
 #define DRIVER_MAJOR		1
 #define DRIVER_MINOR		0
 #define DRIVER_PATCHLEVEL	0
 	.fops = &omapdriver_fops,
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/omapdrm/omap_fbdev.c+1−1

@@ -6,7 +6,7 @@ · 5 unmodified lines
6		6
7	#include <linux/fb.h>	7	#include <linux/fb.h>
8		8
9	#include <drm/drm_client_setup.h>	9	#include <drm/clients/drm_client_setup.h>
10	#include <drm/drm_drv.h>	10	#include <drm/drm_drv.h>
11	#include <drm/drm_crtc_helper.h>	11	#include <drm/drm_crtc_helper.h>
12	#include <drm/drm_fb_helper.h>	12	#include <drm/drm_fb_helper.h>

drivers/gpu/drm/panel/panel-edp.c+8−0

 	.powered_on_to_enable = 200,
 };
+static const struct panel_delay delay_200_150_e50 = {
+	.hpd_absent = 200,
+	.unprepare = 150,
+	.enable = 50,
+};
 #define EDP_PANEL_ENTRY(vend_chr_0, vend_chr_1, vend_chr_2, product_id, _delay, _name) \
 { \
 	.ident = { \
 	EDP_PANEL_ENTRY('B', 'O', 'E', 0x0b56, &delay_200_500_e80, "NT140FHM-N47"),
 	EDP_PANEL_ENTRY('B', 'O', 'E', 0x0b66, &delay_200_500_e80, "NE140WUM-N6G"),
 	EDP_PANEL_ENTRY('B', 'O', 'E', 0x0c20, &delay_200_500_e80, "NT140FHM-N47"),
+	EDP_PANEL_ENTRY('B', 'O', 'E', 0x0c93, &delay_200_500_e200, "Unknown"),
 	EDP_PANEL_ENTRY('B', 'O', 'E', 0x0cb6, &delay_200_500_e200, "NT116WHM-N44"),
 	EDP_PANEL_ENTRY('B', 'O', 'E', 0x0cfa, &delay_200_500_e50, "NV116WHM-A4D"),
 	EDP_PANEL_ENTRY('K', 'D', 'B', 0x1118, &delay_200_500_e50, "KD116N29-30NK-A005"),
 	EDP_PANEL_ENTRY('K', 'D', 'B', 0x1120, &delay_200_500_e80_d50, "116N29-30NK-C007"),
 	EDP_PANEL_ENTRY('K', 'D', 'B', 0x1212, &delay_200_500_e50, "KD116N0930A16"),
+	EDP_PANEL_ENTRY('K', 'D', 'B', 0x1707, &delay_200_150_e50, "KD116N2130B12"),
 	EDP_PANEL_ENTRY('K', 'D', 'C', 0x044f, &delay_200_500_e50, "KD116N9-30NH-F3"),
 	EDP_PANEL_ENTRY('K', 'D', 'C', 0x05f1, &delay_200_500_e80_d50, "KD116N5-30NV-G7"),

drivers/gpu/drm/panel/panel-raspberrypi-touchscreen.c+1−1

@@ -400,7 +400,7 @@ static int rpi_touchscreen_probe(struct i2c_client *i2c) · 399 unmodified lines
400	rpi_touchscreen_i2c_write(ts, REG_POWERON, 0);	400	rpi_touchscreen_i2c_write(ts, REG_POWERON, 0);
401		401
402	/* Look up the DSI host. It needs to probe before we do. */	402	/* Look up the DSI host. It needs to probe before we do. */
403	endpoint = of_graph_get_next_endpoint(dev->of_node, NULL);	403	endpoint = of_graph_get_endpoint_by_regs(dev->of_node, 0, -1);
404	if (!endpoint)	404	if (!endpoint)
405	return -ENODEV;	405	return -ENODEV;
406		406

drivers/gpu/drm/panfrost/panfrost_drv.c+0−1

@@ -636,7 +636,6 @@ static const struct drm_driver panfrost_drm_driver = { · 635 unmodified lines
636	.fops = &panfrost_drm_driver_fops,	636	.fops = &panfrost_drm_driver_fops,
637	.name = "panfrost",	637	.name = "panfrost",
638	.desc = "panfrost DRM",	638	.desc = "panfrost DRM",
639	.date = "20180908",
640	.major = 1,	639	.major = 1,
641	.minor = 3,	640	.minor = 3,
642		641

drivers/gpu/drm/panfrost/panfrost_gpu.c+4−0

 	 */
 	GPU_MODEL(g57, 0x9003,
 		GPU_REV(g57, 0, 0)),
+	/* MediaTek MT8188 Mali-G57 MC3 */
+	GPU_MODEL(g57, 0x9093,
+		GPU_REV(g57, 0, 0)),
 };
 static void panfrost_gpu_init_features(struct panfrost_device *pfdev)

drivers/gpu/drm/panthor/panthor_device.c+21−1

@@ -22,6 +22,24 @@ · 21 unmodified lines
22	#include "panthor_regs.h"	22	#include "panthor_regs.h"
23	#include "panthor_sched.h"	23	#include "panthor_sched.h"
24		24
		25	static int panthor_gpu_coherency_init(struct panthor_device *ptdev)
		26	{
		27	ptdev->coherent = device_get_dma_attr(ptdev->base.dev) == DEV_DMA_COHERENT;
		28
		29	if (!ptdev->coherent)
		30	return 0;
		31
		32	/* Check if the ACE-Lite coherency protocol is actually supported by the GPU.
		33	* ACE protocol has never been supported for command stream frontend GPUs.
		34	*/
		35	if ((gpu_read(ptdev, GPU_COHERENCY_FEATURES) &
		36	GPU_COHERENCY_PROT_BIT(ACE_LITE)))
		37	return 0;
		38
		39	drm_err(&ptdev->base, "Coherency not supported by the device");
		40	return -ENOTSUPP;
		41	}
		42
25	static int panthor_clk_init(struct panthor_device *ptdev)	43	static int panthor_clk_init(struct panthor_device *ptdev)
26	{	44	{
27	ptdev->clks.core = devm_clk_get(ptdev->base.dev, NULL);	45	ptdev->clks.core = devm_clk_get(ptdev->base.dev, NULL);
@@ -156,7 +174,9 @@ int panthor_device_init(struct panthor_device *ptdev) · 128 unmodified lines
156	struct page *p;	174	struct page *p;
157	int ret;	175	int ret;
158		176
159	ptdev->coherent = device_get_dma_attr(ptdev->base.dev) == DEV_DMA_COHERENT;	177	ret = panthor_gpu_coherency_init(ptdev);
		178	if (ret)
		179	return ret;
160		180
161	init_completion(&ptdev->unplug.done);	181	init_completion(&ptdev->unplug.done);
162	ret = drmm_mutex_init(&ptdev->base, &ptdev->unplug.lock);	182	ret = drmm_mutex_init(&ptdev->base, &ptdev->unplug.lock);

drivers/gpu/drm/panthor/panthor_drv.c+0−1

@@ -1505,7 +1505,6 @@ static const struct drm_driver panthor_drm_driver = { · 1504 unmodified lines
1505	.fops = &panthor_drm_driver_fops,	1505	.fops = &panthor_drm_driver_fops,
1506	.name = "panthor",	1506	.name = "panthor",
1507	.desc = "Panthor DRM driver",	1507	.desc = "Panthor DRM driver",
1508	.date = "20230801",
1509	.major = 1,	1508	.major = 1,
1510	.minor = 2,	1509	.minor = 2,
1511		1510

drivers/gpu/drm/panthor/panthor_fw.c+41−39

 #define CSF_FW_BINARY_ENTRY_UPDATE					BIT(30)
 #define CSF_FW_BINARY_ENTRY_OPTIONAL					BIT(31)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_RD					BIT(0)
+#define CSF_FW_BINARY_IFACE_ENTRY_RD					BIT(0)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_WR					BIT(1)
+#define CSF_FW_BINARY_IFACE_ENTRY_WR					BIT(1)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_EX					BIT(2)
+#define CSF_FW_BINARY_IFACE_ENTRY_EX					BIT(2)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_NONE			(0 << 3)
+#define CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_NONE			(0 << 3)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_CACHED			(1 << 3)
+#define CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_CACHED			(1 << 3)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_UNCACHED_COHERENT	(2 << 3)
+#define CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_UNCACHED_COHERENT		(2 << 3)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_CACHED_COHERENT		(3 << 3)
+#define CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_CACHED_COHERENT		(3 << 3)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_MASK			GENMASK(4, 3)
+#define CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_MASK			GENMASK(4, 3)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_PROT				BIT(5)
+#define CSF_FW_BINARY_IFACE_ENTRY_PROT					BIT(5)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_SHARED				BIT(30)
+#define CSF_FW_BINARY_IFACE_ENTRY_SHARED				BIT(30)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_ZERO				BIT(31)
+#define CSF_FW_BINARY_IFACE_ENTRY_ZERO					BIT(31)
-#define CSF_FW_BINARY_IFACE_ENTRY_RD_SUPPORTED_FLAGS			\
+#define CSF_FW_BINARY_IFACE_ENTRY_SUPPORTED_FLAGS			\
-	(CSF_FW_BINARY_IFACE_ENTRY_RD_RD |				\
+	(CSF_FW_BINARY_IFACE_ENTRY_RD |					\
-	 CSF_FW_BINARY_IFACE_ENTRY_RD_WR |				\
+	 CSF_FW_BINARY_IFACE_ENTRY_WR |					\
-	 CSF_FW_BINARY_IFACE_ENTRY_RD_EX |				\
+	 CSF_FW_BINARY_IFACE_ENTRY_EX |					\
-	 CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_MASK |			\
+	 CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_MASK |			\
-	 CSF_FW_BINARY_IFACE_ENTRY_RD_PROT |				\
+	 CSF_FW_BINARY_IFACE_ENTRY_PROT |				\
-	 CSF_FW_BINARY_IFACE_ENTRY_RD_SHARED  |				\
+	 CSF_FW_BINARY_IFACE_ENTRY_SHARED  |				\
-	 CSF_FW_BINARY_IFACE_ENTRY_RD_ZERO)
+	 CSF_FW_BINARY_IFACE_ENTRY_ZERO)
 /**
  * struct panthor_fw_binary_section_entry_hdr - Describes a section of FW binary
 	int ret;
 	if (!section->data.size &&
-	    !(section->flags & CSF_FW_BINARY_IFACE_ENTRY_RD_ZERO))
+	    !(section->flags & CSF_FW_BINARY_IFACE_ENTRY_ZERO))
 		return;
 	ret = panthor_kernel_bo_vmap(section->mem);
 		return;
 	memcpy(section->mem->kmap, section->data.buf, section->data.size);
-	if (section->flags & CSF_FW_BINARY_IFACE_ENTRY_RD_ZERO) {
+	if (section->flags & CSF_FW_BINARY_IFACE_ENTRY_ZERO) {
 		memset(section->mem->kmap + section->data.size, 0,
 		       panthor_kernel_bo_size(section->mem) - section->data.size);
 	}
 		return -EINVAL;
 	}
-	if (hdr.flags & ~CSF_FW_BINARY_IFACE_ENTRY_RD_SUPPORTED_FLAGS) {
+	if (hdr.flags & ~CSF_FW_BINARY_IFACE_ENTRY_SUPPORTED_FLAGS) {
 		drm_err(&ptdev->base, "Firmware contains interface with unsupported flags (0x%x)\n",
 			hdr.flags);
 		return -EINVAL;
 	}
-	if (hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_RD_PROT) {
+	if (hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_PROT) {
 		drm_warn(&ptdev->base,
 			 "Firmware protected mode entry not be supported, ignoring");
 		return 0;
 	}
 	if (hdr.va.start == CSF_MCU_SHARED_REGION_START &&
-	    !(hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_RD_SHARED)) {
+	    !(hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_SHARED)) {
 		drm_err(&ptdev->base,
 			"Interface at 0x%llx must be shared", CSF_MCU_SHARED_REGION_START);
 		return -EINVAL;
 	section_size = hdr.va.end - hdr.va.start;
 	if (section_size) {
-		u32 cache_mode = hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_MASK;
+		u32 cache_mode = hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_MASK;
 		struct panthor_gem_object *bo;
 		u32 vm_map_flags = 0;
 		struct sg_table *sgt;
 		u64 va = hdr.va.start;
-		if (!(hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_RD_WR))
+		if (!(hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_WR))
 			vm_map_flags |= DRM_PANTHOR_VM_BIND_OP_MAP_READONLY;
-		if (!(hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_RD_EX))
+		if (!(hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_EX))
 			vm_map_flags |= DRM_PANTHOR_VM_BIND_OP_MAP_NOEXEC;
-		/* TODO: CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_*_COHERENT are mapped to
+		/* TODO: CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_*_COHERENT are mapped to
 		 * non-cacheable for now. We might want to introduce a new
 		 * IOMMU_xxx flag (or abuse IOMMU_MMIO, which maps to device
 		 * memory and is currently not used by our driver) for
 		 * AS_MEMATTR_AARCH64_SHARED memory, so we can take benefit
 		 * of IO-coherent systems.
 		 */
-		if (cache_mode != CSF_FW_BINARY_IFACE_ENTRY_RD_CACHE_MODE_CACHED)
+		if (cache_mode != CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_CACHED)
 			vm_map_flags |= DRM_PANTHOR_VM_BIND_OP_MAP_UNCACHED;
 		section->mem = panthor_kernel_bo_create(ptdev, panthor_fw_vm(ptdev),
 		if (drm_WARN_ON(&ptdev->base, section->mem->va_node.start != hdr.va.start))
 			return -EINVAL;
-		if (section->flags & CSF_FW_BINARY_IFACE_ENTRY_RD_SHARED) {
+		if (section->flags & CSF_FW_BINARY_IFACE_ENTRY_SHARED) {
 			ret = panthor_kernel_bo_vmap(section->mem);
 			if (ret)
 				return ret;
 	list_for_each_entry(section, &ptdev->fw->sections, node) {
 		struct sg_table *sgt;
-		if (!full_reload && !(section->flags & CSF_FW_BINARY_IFACE_ENTRY_RD_WR))
+		if (!full_reload && !(section->flags & CSF_FW_BINARY_IFACE_ENTRY_WR))
 			continue;
 		panthor_fw_init_section_mem(ptdev, section);
 		panthor_fw_update_reqs(glb_iface, req, GLB_HALT, GLB_HALT);
 		gpu_write(ptdev, CSF_DOORBELL(CSF_GLB_DOORBELL_ID), 1);
 		if (!readl_poll_timeout(ptdev->iomem + MCU_STATUS, status,
-					status == MCU_STATUS_HALT, 10, 100000) &&
+					status == MCU_STATUS_HALT, 10, 100000)) {
-		    glb_iface->output->halt_status == PANTHOR_FW_HALT_OK) {
 			ptdev->fw->fast_reset = true;
 		} else {
 			drm_warn(&ptdev->base, "Failed to cleanly suspend MCU");
 		}
-		/* The FW detects 0 -> 1 transitions. Make sure we reset
-		 * the HALT bit before the FW is rebooted.
-		 */
-		panthor_fw_update_reqs(glb_iface, req, 0, GLB_HALT);
 	}
 	panthor_job_irq_suspend(&ptdev->fw->irq);
 	 * the FW sections. If it fails, go for a full reset.
 	 */
 	if (ptdev->fw->fast_reset) {
+		/* The FW detects 0 -> 1 transitions. Make sure we reset
+		 * the HALT bit before the FW is rebooted.
+		 * This is not needed on a slow reset because FW sections are
+		 * re-initialized.
+		 */
+		struct panthor_fw_global_iface *glb_iface = panthor_fw_get_glb_iface(ptdev);
+		panthor_fw_update_reqs(glb_iface, req, 0, GLB_HALT);
 		ret = panthor_fw_start(ptdev);
 		if (!ret)
 			goto out;

drivers/gpu/drm/panthor/panthor_gpu.c+9−0

 	 GPU_IRQ_RESET_COMPLETED | \
 	 GPU_IRQ_CLEAN_CACHES_COMPLETED)
+static void panthor_gpu_coherency_set(struct panthor_device *ptdev)
+{
+	gpu_write(ptdev, GPU_COHERENCY_PROTOCOL,
+		ptdev->coherent ? GPU_COHERENCY_PROT_BIT(ACE_LITE) : GPU_COHERENCY_NONE);
+}
 static void panthor_gpu_init_info(struct panthor_device *ptdev)
 {
 	const struct panthor_model *model;
 			      hweight64(ptdev->gpu_info.shader_present));
 	}
+	/* Set the desired coherency mode before the power up of L2 */
+	panthor_gpu_coherency_set(ptdev);
 	return panthor_gpu_power_on(ptdev, L2, 1, 20000);
 }

drivers/gpu/drm/panthor/panthor_mmu.c+15−8

 	return pool;
 }
-static u64 mair_to_memattr(u64 mair)
+static u64 mair_to_memattr(u64 mair, bool coherent)
 {
 	u64 memattr = 0;
 	u32 i;
 				   AS_MEMATTR_AARCH64_SH_MIDGARD_INNER |
 				   AS_MEMATTR_AARCH64_INNER_ALLOC_EXPL(false, false);
 		} else {
-			/* Use SH_CPU_INNER mode so SH_IS, which is used when
-			 * IOMMU_CACHE is set, actually maps to the standard
-			 * definition of inner-shareable and not Mali's
-			 * internal-shareable mode.
-			 */
 			out_attr = AS_MEMATTR_AARCH64_INNER_OUTER_WB |
-				   AS_MEMATTR_AARCH64_SH_CPU_INNER |
 				   AS_MEMATTR_AARCH64_INNER_ALLOC_EXPL(inner & 1, inner & 2);
+			/* Use SH_MIDGARD_INNER mode when device isn't coherent,
+			 * so SH_IS, which is used when IOMMU_CACHE is set, maps
+			 * to Mali's internal-shareable mode. As per the Mali
+			 * Spec, inner and outer-shareable modes aren't allowed
+			 * for WB memory when coherency is disabled.
+			 * Use SH_CPU_INNER mode when coherency is enabled, so
+			 * that SH_IS actually maps to the standard definition of
+			 * inner-shareable.
+			 */
+			if (!coherent)
+				out_attr |= AS_MEMATTR_AARCH64_SH_MIDGARD_INNER;
+			else
+				out_attr |= AS_MEMATTR_AARCH64_SH_CPU_INNER;
 		}
 		memattr |= (u64)out_attr << (8 * i);
 		goto err_sched_fini;
 	mair = io_pgtable_ops_to_pgtable(vm->pgtbl_ops)->cfg.arm_lpae_s1_cfg.mair;
-	vm->memattr = mair_to_memattr(mair);
+	vm->memattr = mair_to_memattr(mair, ptdev->coherent);
 	mutex_lock(&ptdev->mmu->vm.lock);
 	list_add_tail(&vm->node, &ptdev->mmu->vm.list);

drivers/gpu/drm/pl111/pl111_drv.c+1−2

 #include <linux/shmem_fs.h>
 #include <linux/slab.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_bridge.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_fourcc.h>
 	.fops = &drm_fops,
 	.name = "pl111",
 	.desc = DRIVER_DESC,
-	.date = "20170317",
 	.major = 1,
 	.minor = 0,
 	.patchlevel = 0,

drivers/gpu/drm/qxl/Kconfig+1−0

@@ -6,6 +6,7 @@ config DRM_QXL · 5 unmodified lines
6	select DRM_KMS_HELPER	6	select DRM_KMS_HELPER
7	select DRM_TTM	7	select DRM_TTM
8	select DRM_TTM_HELPER	8	select DRM_TTM_HELPER
		9	select DRM_EXEC
9	select CRC32	10	select CRC32
10	help	11	help
11	QXL virtual GPU for Spice virtualization desktop integration.	12	QXL virtual GPU for Spice virtualization desktop integration.

drivers/gpu/drm/qxl/qxl_drv.c+1−2

 #include <linux/pci.h>
 #include <linux/vgaarb.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_ttm.h>
 #include <drm/drm_file.h>
 	.num_ioctls = ARRAY_SIZE(qxl_ioctls),
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = 0,
 	.minor = 1,
 	.patchlevel = 0,

drivers/gpu/drm/qxl/qxl_drv.h+4−4

 #include <drm/drm_crtc.h>
 #include <drm/drm_encoder.h>
+#include <drm/drm_exec.h>
 #include <drm/drm_gem_ttm_helper.h>
 #include <drm/drm_ioctl.h>
 #include <drm/drm_gem.h>
 #include <drm/qxl_drm.h>
 #include <drm/ttm/ttm_bo.h>
-#include <drm/ttm/ttm_execbuf_util.h>
 #include <drm/ttm/ttm_placement.h>
 #include "qxl_dev.h"
 #define DRIVER_NAME		"qxl"
 #define DRIVER_DESC		"RH QXL"
-#define DRIVER_DATE		"20120117"
 #define DRIVER_MAJOR 0
 #define DRIVER_MINOR 1
 };
 struct qxl_bo_list {
-	struct ttm_validate_buffer tv;
+	struct qxl_bo		*bo;
+	struct list_head	list;
 };
 struct qxl_crtc {
 	struct qxl_bo *release_bo;
 	uint32_t release_offset;
 	uint32_t surface_release_id;
-	struct ww_acquire_ctx ticket;
+	struct drm_exec	exec;
 	struct list_head bos;
 };

drivers/gpu/drm/qxl/qxl_release.c+35−33

@@ -121,13 +121,11 @@ qxl_release_free_list(struct qxl_release *release) · 120 unmodified lines
121	{	121	{
122	while (!list_empty(&release->bos)) {	122	while (!list_empty(&release->bos)) {
123	struct qxl_bo_list *entry;	123	struct qxl_bo_list *entry;
124	struct qxl_bo *bo;
125		124
126	entry = container_of(release->bos.next,	125	entry = container_of(release->bos.next,
127	struct qxl_bo_list, tv.head);	126	struct qxl_bo_list, list);
128	bo = to_qxl_bo(entry->tv.bo);	127	qxl_bo_unref(&entry->bo);
129	qxl_bo_unref(&bo);	128	list_del(&entry->list);
130	list_del(&entry->tv.head);
131	kfree(entry);	129	kfree(entry);
132	}	130	}
133	release->release_bo = NULL;	131	release->release_bo = NULL;
@@ -172,8 +170,8 @@ int qxl_release_list_add(struct qxl_release release, struct qxl_bo bo) · 38 unmodified lines
172	{	170	{
173	struct qxl_bo_list *entry;	171	struct qxl_bo_list *entry;
174		172
175	list_for_each_entry(entry, &release->bos, tv.head) {	173	list_for_each_entry(entry, &release->bos, list) {
176	if (entry->tv.bo == &bo->tbo)	174	if (entry->bo == bo)
177	return 0;	175	return 0;
178	}	176	}
179		177
@@ -182,9 +180,8 @@ int qxl_release_list_add(struct qxl_release release, struct qxl_bo bo) · 2 unmodified lines
182	return -ENOMEM;	180	return -ENOMEM;
183		181
184	qxl_bo_ref(bo);	182	qxl_bo_ref(bo);
185	entry->tv.bo = &bo->tbo;	183	entry->bo = bo;
186	entry->tv.num_shared = 0;	184	list_add_tail(&entry->list, &release->bos);
187	list_add_tail(&entry->tv.head, &release->bos);
188	return 0;	185	return 0;
189	}	186	}
190		187
@@ -221,21 +218,28 @@ int qxl_release_reserve_list(struct qxl_release *release, bool no_intr) · 30 unmodified lines
221	if (list_is_singular(&release->bos))	218	if (list_is_singular(&release->bos))
222	return 0;	219	return 0;
223		220
224	ret = ttm_eu_reserve_buffers(&release->ticket, &release->bos,	221	drm_exec_init(&release->exec, no_intr ? 0 :
225	!no_intr, NULL);	222	DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
226	if (ret)	223	drm_exec_until_all_locked(&release->exec) {
227	return ret;	224	list_for_each_entry(entry, &release->bos, list) {
228		225	ret = drm_exec_prepare_obj(&release->exec,
229	list_for_each_entry(entry, &release->bos, tv.head) {	226	&entry->bo->tbo.base,
230	struct qxl_bo *bo = to_qxl_bo(entry->tv.bo);	227	1);
231		228	drm_exec_retry_on_contention(&release->exec);
232	ret = qxl_release_validate_bo(bo);	229	if (ret)
233	if (ret) {	230	goto error;
234	ttm_eu_backoff_reservation(&release->ticket, &release->bos);
235	return ret;
236	}	231	}
237	}	232	}
		233
		234	list_for_each_entry(entry, &release->bos, list) {
		235	ret = qxl_release_validate_bo(entry->bo);
		236	if (ret)
		237	goto error;
		238	}
238	return 0;	239	return 0;
		240	error:
		241	drm_exec_fini(&release->exec);
		242	return ret;
239	}	243	}
240		244
241	void qxl_release_backoff_reserve_list(struct qxl_release *release)	245	void qxl_release_backoff_reserve_list(struct qxl_release *release)
@@ -245,7 +249,7 @@ void qxl_release_backoff_reserve_list(struct qxl_release *release) · 3 unmodified lines
245	if (list_is_singular(&release->bos))	249	if (list_is_singular(&release->bos))
246	return;	250	return;
247		251
248	ttm_eu_backoff_reservation(&release->ticket, &release->bos);	252	drm_exec_fini(&release->exec);
249	}	253	}
250		254
251	int qxl_alloc_surface_release_reserved(struct qxl_device *qdev,	255	int qxl_alloc_surface_release_reserved(struct qxl_device *qdev,
@@ -404,18 +408,18 @@ void qxl_release_unmap(struct qxl_device *qdev, · 152 unmodified lines
404		408
405	void qxl_release_fence_buffer_objects(struct qxl_release *release)	409	void qxl_release_fence_buffer_objects(struct qxl_release *release)
406	{	410	{
407	struct ttm_buffer_object *bo;
408	struct ttm_device *bdev;	411	struct ttm_device *bdev;
409	struct ttm_validate_buffer *entry;	412	struct qxl_bo_list *entry;
410	struct qxl_device *qdev;	413	struct qxl_device *qdev;
		414	struct qxl_bo *bo;
411		415
412	/* if only one object on the release its the release itself	416	/* if only one object on the release its the release itself
413	since these objects are pinned no need to reserve */	417	since these objects are pinned no need to reserve */
414	if (list_is_singular(&release->bos) \|\| list_empty(&release->bos))	418	if (list_is_singular(&release->bos) \|\| list_empty(&release->bos))
415	return;	419	return;
416		420
417	bo = list_first_entry(&release->bos, struct ttm_validate_buffer, head)->bo;	421	bo = list_first_entry(&release->bos, struct qxl_bo_list, list)->bo;
418	bdev = bo->bdev;	422	bdev = bo->tbo.bdev;
419	qdev = container_of(bdev, struct qxl_device, mman.bdev);	423	qdev = container_of(bdev, struct qxl_device, mman.bdev);
420		424
421	/*	425	/*
@@ -426,14 +430,12 @@ void qxl_release_fence_buffer_objects(struct qxl_release *release) · 4 unmodified lines
426	release->id \| 0xf0000000, release->base.seqno);	430	release->id \| 0xf0000000, release->base.seqno);
427	trace_dma_fence_emit(&release->base);	431	trace_dma_fence_emit(&release->base);
428		432
429	list_for_each_entry(entry, &release->bos, head) {	433	list_for_each_entry(entry, &release->bos, list) {
430	bo = entry->bo;	434	bo = entry->bo;
431		435
432	dma_resv_add_fence(bo->base.resv, &release->base,	436	dma_resv_add_fence(bo->tbo.base.resv, &release->base,
433	DMA_RESV_USAGE_READ);	437	DMA_RESV_USAGE_READ);
434	ttm_bo_move_to_lru_tail_unlocked(bo);	438	ttm_bo_move_to_lru_tail_unlocked(&bo->tbo);
435	dma_resv_unlock(bo->base.resv);
436	}	439	}
437	ww_acquire_fini(&release->ticket);	440	drm_exec_fini(&release->exec);
438	}	441	}
439

drivers/gpu/drm/radeon/Kconfig+1−0

@@ -13,6 +13,7 @@ config DRM_RADEON · 12 unmodified lines
13	select DRM_TTM	13	select DRM_TTM
14	select DRM_TTM_HELPER	14	select DRM_TTM_HELPER
15	select FB_IOMEM_HELPERS if DRM_FBDEV_EMULATION	15	select FB_IOMEM_HELPERS if DRM_FBDEV_EMULATION
		16	select DRM_EXEC
16	select SND_HDA_COMPONENT if SND_HDA_CORE	17	select SND_HDA_COMPONENT if SND_HDA_CORE
17	select POWER_SUPPLY	18	select POWER_SUPPLY
18	select HWMON	19	select HWMON

drivers/gpu/drm/radeon/radeon.h+4−3

 #include <drm/ttm/ttm_bo.h>
 #include <drm/ttm/ttm_placement.h>
-#include <drm/ttm/ttm_execbuf_util.h>
+#include <drm/drm_exec.h>
 #include <drm/drm_gem.h>
 #include <drm/drm_audio_component.h>
 #include <drm/drm_suballoc.h>
 struct radeon_bo_list {
 	struct radeon_bo		*robj;
-	struct ttm_validate_buffer	tv;
+	struct list_head		list;
+	bool				shared;
 	uint64_t			gpu_offset;
 	unsigned			preferred_domains;
 	unsigned			allowed_domains;
 	struct radeon_bo_list	*vm_bos;
 	struct list_head	validated;
 	unsigned		dma_reloc_idx;
+	struct drm_exec		exec;
 	/* indices of various chunks */
 	struct radeon_cs_chunk  *chunk_ib;
 	struct radeon_cs_chunk  *chunk_relocs;
 	u32			cs_flags;
 	u32			ring;
 	s32			priority;
-	struct ww_acquire_ctx	ticket;
 };
 static inline u32 radeon_get_ib_value(struct radeon_cs_parser *p, int idx)

drivers/gpu/drm/radeon/radeon_cs.c+23−22

@@ -182,11 +182,8 @@ static int radeon_cs_parser_relocs(struct radeon_cs_parser *p) · 181 unmodified lines
182	}	182	}
183	}	183	}
184		184
185	p->relocs[i].tv.bo = &p->relocs[i].robj->tbo;	185	p->relocs[i].shared = !r->write_domain;
186	p->relocs[i].tv.num_shared = !r->write_domain;	186	radeon_cs_buckets_add(&buckets, &p->relocs[i].list, priority);
187
188	radeon_cs_buckets_add(&buckets, &p->relocs[i].tv.head,
189	priority);
190	}	187	}
191		188
192	radeon_cs_buckets_get_list(&buckets, &p->validated);	189	radeon_cs_buckets_get_list(&buckets, &p->validated);
@@ -197,7 +194,7 @@ static int radeon_cs_parser_relocs(struct radeon_cs_parser *p) · 4 unmodified lines
197	if (need_mmap_lock)	194	if (need_mmap_lock)
198	mmap_read_lock(current->mm);	195	mmap_read_lock(current->mm);
199		196
200	r = radeon_bo_list_validate(p->rdev, &p->ticket, &p->validated, p->ring);	197	r = radeon_bo_list_validate(p->rdev, &p->exec, &p->validated, p->ring);
201		198
202	if (need_mmap_lock)	199	if (need_mmap_lock)
203	mmap_read_unlock(current->mm);	200	mmap_read_unlock(current->mm);
@@ -253,12 +250,11 @@ static int radeon_cs_sync_rings(struct radeon_cs_parser *p) · 49 unmodified lines
253	struct radeon_bo_list *reloc;	250	struct radeon_bo_list *reloc;
254	int r;	251	int r;
255		252
256	list_for_each_entry(reloc, &p->validated, tv.head) {	253	list_for_each_entry(reloc, &p->validated, list) {
257	struct dma_resv *resv;	254	struct dma_resv *resv;
258		255
259	resv = reloc->robj->tbo.base.resv;	256	resv = reloc->robj->tbo.base.resv;
260	r = radeon_sync_resv(p->rdev, &p->ib.sync, resv,	257	r = radeon_sync_resv(p->rdev, &p->ib.sync, resv, reloc->shared);
261	reloc->tv.num_shared);
262	if (r)	258	if (r)
263	return r;	259	return r;
264	}	260	}
@@ -276,6 +272,7 @@ int radeon_cs_parser_init(struct radeon_cs_parser p, void data) · 11 unmodified lines
276	s32 priority = 0;	272	s32 priority = 0;
277		273
278	INIT_LIST_HEAD(&p->validated);	274	INIT_LIST_HEAD(&p->validated);
		275	drm_exec_init(&p->exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
279		276
280	if (!cs->num_chunks) {	277	if (!cs->num_chunks) {
281	return 0;	278	return 0;
@@ -397,8 +394,8 @@ int radeon_cs_parser_init(struct radeon_cs_parser p, void data) · 115 unmodified lines
397	static int cmp_size_smaller_first(void priv, const struct list_head a,	394	static int cmp_size_smaller_first(void priv, const struct list_head a,
398	const struct list_head *b)	395	const struct list_head *b)
399	{	396	{
400	struct radeon_bo_list *la = list_entry(a, struct radeon_bo_list, tv.head);	397	struct radeon_bo_list *la = list_entry(a, struct radeon_bo_list, list);
401	struct radeon_bo_list *lb = list_entry(b, struct radeon_bo_list, tv.head);	398	struct radeon_bo_list *lb = list_entry(b, struct radeon_bo_list, list);
402		399
403	/* Sort A before B if A is smaller. */	400	/* Sort A before B if A is smaller. */
404	if (la->robj->tbo.base.size > lb->robj->tbo.base.size)	401	if (la->robj->tbo.base.size > lb->robj->tbo.base.size)
@@ -417,11 +414,13 @@ static int cmp_size_smaller_first(void priv, const struct list_head a, · 12 unmodified lines
417	* If error is set than unvalidate buffer, otherwise just free memory	414	* If error is set than unvalidate buffer, otherwise just free memory
418	* used by parsing context.	415	* used by parsing context.
419	**/	416	**/
420	static void radeon_cs_parser_fini(struct radeon_cs_parser *parser, int error, bool backoff)	417	static void radeon_cs_parser_fini(struct radeon_cs_parser *parser, int error)
421	{	418	{
422	unsigned i;	419	unsigned i;
423		420
424	if (!error) {	421	if (!error) {
		422	struct radeon_bo_list *reloc;
		423
425	/* Sort the buffer list from the smallest to largest buffer,	424	/* Sort the buffer list from the smallest to largest buffer,
426	* which affects the order of buffers in the LRU list.	425	* which affects the order of buffers in the LRU list.
427	* This assures that the smallest buffers are added first	426	* This assures that the smallest buffers are added first
@@ -433,15 +432,17 @@ static void radeon_cs_parser_fini(struct radeon_cs_parser *parser, int error, bo · 5 unmodified lines
433	* per frame under memory pressure.	432	* per frame under memory pressure.
434	*/	433	*/
435	list_sort(NULL, &parser->validated, cmp_size_smaller_first);	434	list_sort(NULL, &parser->validated, cmp_size_smaller_first);
436		435	list_for_each_entry(reloc, &parser->validated, list) {
437	ttm_eu_fence_buffer_objects(&parser->ticket,	436	dma_resv_add_fence(reloc->robj->tbo.base.resv,
438	&parser->validated,	437	&parser->ib.fence->base,
439	&parser->ib.fence->base);	438	reloc->shared ?
440	} else if (backoff) {	439	DMA_RESV_USAGE_READ :
441	ttm_eu_backoff_reservation(&parser->ticket,	440	DMA_RESV_USAGE_WRITE);
442	&parser->validated);	441	}
443	}	442	}
444		443
		444	drm_exec_fini(&parser->exec);
		445
445	if (parser->relocs != NULL) {	446	if (parser->relocs != NULL) {
446	for (i = 0; i < parser->nrelocs; i++) {	447	for (i = 0; i < parser->nrelocs; i++) {
447	struct radeon_bo *bo = parser->relocs[i].robj;	448	struct radeon_bo *bo = parser->relocs[i].robj;
@@ -693,7 +694,7 @@ int radeon_cs_ioctl(struct drm_device dev, void data, struct drm_file *filp) · 245 unmodified lines
693	r = radeon_cs_parser_init(&parser, data);	694	r = radeon_cs_parser_init(&parser, data);
694	if (r) {	695	if (r) {
695	DRM_ERROR("Failed to initialize parser !\n");	696	DRM_ERROR("Failed to initialize parser !\n");
696	radeon_cs_parser_fini(&parser, r, false);	697	radeon_cs_parser_fini(&parser, r);
697	up_read(&rdev->exclusive_lock);	698	up_read(&rdev->exclusive_lock);
698	r = radeon_cs_handle_lockup(rdev, r);	699	r = radeon_cs_handle_lockup(rdev, r);
699	return r;	700	return r;
@@ -707,7 +708,7 @@ int radeon_cs_ioctl(struct drm_device dev, void data, struct drm_file *filp) · 7 unmodified lines
707	}	708	}
708		709
709	if (r) {	710	if (r) {
710	radeon_cs_parser_fini(&parser, r, false);	711	radeon_cs_parser_fini(&parser, r);
711	up_read(&rdev->exclusive_lock);	712	up_read(&rdev->exclusive_lock);
712	r = radeon_cs_handle_lockup(rdev, r);	713	r = radeon_cs_handle_lockup(rdev, r);
713	return r;	714	return r;
@@ -724,7 +725,7 @@ int radeon_cs_ioctl(struct drm_device dev, void data, struct drm_file *filp) · 10 unmodified lines
724	goto out;	725	goto out;
725	}	726	}
726	out:	727	out:
727	radeon_cs_parser_fini(&parser, r, true);	728	radeon_cs_parser_fini(&parser, r);
728	up_read(&rdev->exclusive_lock);	729	up_read(&rdev->exclusive_lock);
729	r = radeon_cs_handle_lockup(rdev, r);	730	r = radeon_cs_handle_lockup(rdev, r);
730	return r;	731	return r;

drivers/gpu/drm/radeon/radeon_drv.c+1−2

 #include <linux/mmu_notifier.h>
 #include <linux/pci.h>
-#include <drm/drm_client_setup.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_file.h>
 #include <drm/drm_fourcc.h>
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = KMS_DRIVER_MAJOR,
 	.minor = KMS_DRIVER_MINOR,
 	.patchlevel = KMS_DRIVER_PATCHLEVEL,

drivers/gpu/drm/radeon/radeon_drv.h+0−1

@@ -43,7 +43,6 @@ · 42 unmodified lines
43		43
44	#define DRIVER_NAME "radeon"	44	#define DRIVER_NAME "radeon"
45	#define DRIVER_DESC "ATI Radeon"	45	#define DRIVER_DESC "ATI Radeon"
46	#define DRIVER_DATE "20080528"
47		46
48	/* Interface history:	47	/* Interface history:
49	*	48	*

drivers/gpu/drm/radeon/radeon_gem.c+22−17

 static void radeon_gem_va_update_vm(struct radeon_device *rdev,
 				    struct radeon_bo_va *bo_va)
 {
-	struct ttm_validate_buffer tv, *entry;
+	struct radeon_bo_list *vm_bos, *entry;
-	struct radeon_bo_list *vm_bos;
-	struct ww_acquire_ctx ticket;
 	struct list_head list;
+	struct drm_exec exec;
 	unsigned domain;
 	int r;
 	INIT_LIST_HEAD(&list);
-	tv.bo = &bo_va->bo->tbo;
-	tv.num_shared = 1;
-	list_add(&tv.head, &list);
 	vm_bos = radeon_vm_get_bos(rdev, bo_va->vm, &list);
 	if (!vm_bos)
 		return;
-	r = ttm_eu_reserve_buffers(&ticket, &list, true, NULL);
+	drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
-	if (r)
+	drm_exec_until_all_locked(&exec) {
-		goto error_free;
+		list_for_each_entry(entry, &list, list) {
+			r = drm_exec_prepare_obj(&exec, &entry->robj->tbo.base,
+						 1);
+			drm_exec_retry_on_contention(&exec);
+			if (unlikely(r))
+				goto error_cleanup;
+		}
-	list_for_each_entry(entry, &list, head) {
+		r = drm_exec_prepare_obj(&exec, &bo_va->bo->tbo.base, 1);
-		domain = radeon_mem_type_to_domain(entry->bo->resource->mem_type);
+		drm_exec_retry_on_contention(&exec);
+		if (unlikely(r))
+			goto error_cleanup;
+	}
+	list_for_each_entry(entry, &list, list) {
+		domain = radeon_mem_type_to_domain(entry->robj->tbo.resource->mem_type);
 		/* if anything is swapped out don't swap it in here,
 		   just abort and wait for the next CS */
 		if (domain == RADEON_GEM_DOMAIN_CPU)
-			goto error_unreserve;
+			goto error_cleanup;
 	}
 	mutex_lock(&bo_va->vm->mutex);
 error_unlock:
 	mutex_unlock(&bo_va->vm->mutex);
-error_unreserve:
+error_cleanup:
-	ttm_eu_backoff_reservation(&ticket, &list);
+	drm_exec_fini(&exec);
-error_free:
 	kvfree(vm_bos);
 	if (r && r != -ERESTARTSYS)

drivers/gpu/drm/radeon/radeon_object.c+11−14

 }
 int radeon_bo_list_validate(struct radeon_device *rdev,
-			    struct ww_acquire_ctx *ticket,
+			    struct drm_exec *exec,
 			    struct list_head *head, int ring)
 {
 	struct ttm_operation_ctx ctx = { true, false };
 	struct radeon_bo_list *lobj;
-	struct list_head duplicates;
-	int r;
 	u64 bytes_moved = 0, initial_bytes_moved;
 	u64 bytes_moved_threshold = radeon_bo_get_threshold_for_moves(rdev);
+	int r;
-	INIT_LIST_HEAD(&duplicates);
+	drm_exec_until_all_locked(exec) {
-	r = ttm_eu_reserve_buffers(ticket, head, true, &duplicates);
+		list_for_each_entry(lobj, head, list) {
-	if (unlikely(r != 0)) {
+			r = drm_exec_prepare_obj(exec, &lobj->robj->tbo.base,
-		return r;
+						 1);
+			drm_exec_retry_on_contention(exec);
+			if (unlikely(r && r != -EALREADY))
+				return r;
+		}
 	}
-	list_for_each_entry(lobj, head, tv.head) {
+	list_for_each_entry(lobj, head, list) {
 		struct radeon_bo *bo = lobj->robj;
 		if (!bo->tbo.pin_count) {
 			u32 domain = lobj->preferred_domains;
 					domain = lobj->allowed_domains;
 					goto retry;
 				}
-				ttm_eu_backoff_reservation(ticket, head);
 				return r;
 			}
 		}
 		lobj->tiling_flags = bo->tiling_flags;
 	}
-	list_for_each_entry(lobj, &duplicates, tv.head) {
-		lobj->gpu_offset = radeon_bo_gpu_offset(lobj->robj);
-		lobj->tiling_flags = lobj->robj->tiling_flags;
-	}
 	return 0;
 }

drivers/gpu/drm/radeon/radeon_object.h+1−1

@@ -152,7 +152,7 @@ extern void radeon_bo_force_delete(struct radeon_device *rdev); · 151 unmodified lines
152	extern int radeon_bo_init(struct radeon_device *rdev);	152	extern int radeon_bo_init(struct radeon_device *rdev);
153	extern void radeon_bo_fini(struct radeon_device *rdev);	153	extern void radeon_bo_fini(struct radeon_device *rdev);
154	extern int radeon_bo_list_validate(struct radeon_device *rdev,	154	extern int radeon_bo_list_validate(struct radeon_device *rdev,
155	struct ww_acquire_ctx *ticket,	155	struct drm_exec *exec,
156	struct list_head *head, int ring);	156	struct list_head *head, int ring);
157	extern int radeon_bo_set_tiling_flags(struct radeon_bo *bo,	157	extern int radeon_bo_set_tiling_flags(struct radeon_bo *bo,
158	u32 tiling_flags, u32 pitch);	158	u32 tiling_flags, u32 pitch);

drivers/gpu/drm/radeon/radeon_vm.c+4−6

 	list[0].robj = vm->page_directory;
 	list[0].preferred_domains = RADEON_GEM_DOMAIN_VRAM;
 	list[0].allowed_domains = RADEON_GEM_DOMAIN_VRAM;
-	list[0].tv.bo = &vm->page_directory->tbo;
+	list[0].shared = true;
-	list[0].tv.num_shared = 1;
 	list[0].tiling_flags = 0;
-	list_add(&list[0].tv.head, head);
+	list_add(&list[0].list, head);
 	for (i = 0, idx = 1; i <= vm->max_pde_used; i++) {
 		if (!vm->page_tables[i].bo)
 		list[idx].robj = vm->page_tables[i].bo;
 		list[idx].preferred_domains = RADEON_GEM_DOMAIN_VRAM;
 		list[idx].allowed_domains = RADEON_GEM_DOMAIN_VRAM;
-		list[idx].tv.bo = &list[idx].robj->tbo;
+		list[idx].shared = true;
-		list[idx].tv.num_shared = 1;
 		list[idx].tiling_flags = 0;
-		list_add(&list[idx++].tv.head, head);
+		list_add(&list[idx++].list, head);
 	}
 	return list;

drivers/gpu/drm/renesas/rcar-du/rcar_du_drv.c+1−2

 #include <linux/slab.h>
 #include <linux/wait.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	.fops			= &rcar_du_fops,
 	.name			= "rcar-du",
 	.desc			= "Renesas R-Car Display Unit",
-	.date			= "20130110",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/renesas/rz-du/rzg2l_du_drv.c+1−2

 #include <linux/of.h>
 #include <linux/platform_device.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	.fops			= &rzg2l_du_fops,
 	.name			= "rzg2l-du",
 	.desc			= "Renesas RZ/G2L Display Unit",
-	.date			= "20230410",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/renesas/rz-du/rzg2l_du_kms.c+3−3

@@ -311,11 +311,11 @@ int rzg2l_du_modeset_init(struct rzg2l_du_device *rcdu) · 310 unmodified lines
311	dev->mode_config.helper_private = &rzg2l_du_mode_config_helper;	311	dev->mode_config.helper_private = &rzg2l_du_mode_config_helper;
312		312
313	/*	313	/*
314	* The RZ DU uses the VSP1 for memory access, and is limited	314	* The RZ DU was designed to support a frame size of 1920x1200 (landscape)
315	* to frame sizes of 1920x1080.	315	* or 1200x1920 (portrait).
316	*/	316	*/
317	dev->mode_config.max_width = 1920;	317	dev->mode_config.max_width = 1920;
318	dev->mode_config.max_height = 1080;	318	dev->mode_config.max_height = 1920;
319		319
320	rcdu->num_crtcs = hweight8(rcdu->info->channels_mask);	320	rcdu->num_crtcs = hweight8(rcdu->info->channels_mask);
321		321

drivers/gpu/drm/renesas/shmobile/shmob_drm_drv.c+1−2

 #include <linux/pm_runtime.h>
 #include <linux/slab.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_fourcc.h>
 	.fops			= &shmob_drm_fops,
 	.name			= "shmob-drm",
 	.desc			= "Renesas SH Mobile DRM",
-	.date			= "20120424",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/rockchip/analogix_dp-rockchip.c+1−1

@@ -386,7 +386,7 @@ static int rockchip_dp_probe(struct platform_device *pdev) · 385 unmodified lines
386	return -ENODEV;	386	return -ENODEV;
387		387
388	ret = drm_of_find_panel_or_bridge(dev->of_node, 1, 0, &panel, NULL);	388	ret = drm_of_find_panel_or_bridge(dev->of_node, 1, 0, &panel, NULL);
389	if (ret < 0)	389	if (ret < 0 && ret != -ENODEV)
390	return ret;	390	return ret;
391		391
392	dp = devm_kzalloc(dev, sizeof(*dp), GFP_KERNEL);	392	dp = devm_kzalloc(dev, sizeof(*dp), GFP_KERNEL);

drivers/gpu/drm/rockchip/cdn-dp-core.c+1−8

@@ -947,9 +947,6 @@ static void cdn_dp_pd_event_work(struct work_struct *work) · 946 unmodified lines
947	{	947	{
948	struct cdn_dp_device *dp = container_of(work, struct cdn_dp_device,	948	struct cdn_dp_device *dp = container_of(work, struct cdn_dp_device,
949	event_work);	949	event_work);
950	struct drm_connector *connector = &dp->connector;
951	enum drm_connector_status old_status;
952
953	int ret;	950	int ret;
954		951
955	mutex_lock(&dp->lock);	952	mutex_lock(&dp->lock);
@@ -1009,11 +1006,7 @@ static void cdn_dp_pd_event_work(struct work_struct *work) · 53 unmodified lines
1009		1006
1010	out:	1007	out:
1011	mutex_unlock(&dp->lock);	1008	mutex_unlock(&dp->lock);
1012		1009	drm_connector_helper_hpd_irq_event(&dp->connector);
1013	old_status = connector->status;
1014	connector->status = connector->funcs->detect(connector, false);
1015	if (old_status != connector->status)
1016	drm_kms_helper_hotplug_event(dp->drm_dev);
1017	}	1010	}
1018		1011
1019	static int cdn_dp_pd_event(struct notifier_block *nb,	1012	static int cdn_dp_pd_event(struct notifier_block *nb,

drivers/gpu/drm/rockchip/rockchip_drm_drv.c+1−3

 #include <linux/console.h>
 #include <linux/iommu.h>
-#include <drm/drm_client_setup.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 #define DRIVER_NAME	"rockchip"
 #define DRIVER_DESC	"RockChip Soc DRM"
-#define DRIVER_DATE	"20140818"
 #define DRIVER_MAJOR	1
 #define DRIVER_MINOR	0
 	.fops			= &rockchip_drm_driver_fops,
 	.name	= DRIVER_NAME,
 	.desc	= DRIVER_DESC,
-	.date	= DRIVER_DATE,
 	.major	= DRIVER_MAJOR,
 	.minor	= DRIVER_MINOR,
 };

drivers/gpu/drm/rockchip/rockchip_drm_vop2.c+202−14

@@ -278,6 +278,15 @@ static u32 vop2_readl(struct vop2 *vop2, u32 offset) · 277 unmodified lines
278	return val;	278	return val;
279	}	279	}
280		280
		281	static u32 vop2_vp_read(struct vop2_video_port *vp, u32 offset)
		282	{
		283	u32 val;
		284
		285	regmap_read(vp->vop2->map, vp->data->offset + offset, &val);
		286
		287	return val;
		288	}
		289
281	static void vop2_win_write(const struct vop2_win *win, unsigned int reg, u32 v)	290	static void vop2_win_write(const struct vop2_win *win, unsigned int reg, u32 v)
282	{	291	{
283	regmap_field_write(win->reg[reg], v);	292	regmap_field_write(win->reg[reg], v);
@@ -998,6 +1007,67 @@ static void vop2_disable(struct vop2 *vop2) · 714 unmodified lines
998	clk_disable_unprepare(vop2->hclk);	1007	clk_disable_unprepare(vop2->hclk);
999	}	1008	}
1000		1009
		1010	static bool vop2_vp_dsp_lut_is_enabled(struct vop2_video_port *vp)
		1011	{
		1012	u32 dsp_ctrl = vop2_vp_read(vp, RK3568_VP_DSP_CTRL);
		1013
		1014	return dsp_ctrl & RK3568_VP_DSP_CTRL__DSP_LUT_EN;
		1015	}
		1016
		1017	static void vop2_vp_dsp_lut_disable(struct vop2_video_port *vp)
		1018	{
		1019	u32 dsp_ctrl = vop2_vp_read(vp, RK3568_VP_DSP_CTRL);
		1020
		1021	dsp_ctrl &= ~RK3568_VP_DSP_CTRL__DSP_LUT_EN;
		1022	vop2_vp_write(vp, RK3568_VP_DSP_CTRL, dsp_ctrl);
		1023	}
		1024
		1025	static bool vop2_vp_dsp_lut_poll_disabled(struct vop2_video_port *vp)
		1026	{
		1027	u32 dsp_ctrl;
		1028	int ret = readx_poll_timeout(vop2_vp_dsp_lut_is_enabled, vp, dsp_ctrl,
		1029	!dsp_ctrl, 5, 30 * 1000);
		1030	if (ret) {
		1031	drm_err(vp->vop2->drm, "display LUT RAM enable timeout!\n");
		1032	return false;
		1033	}
		1034
		1035	return true;
		1036	}
		1037
		1038	static void vop2_vp_dsp_lut_enable(struct vop2_video_port *vp)
		1039	{
		1040	u32 dsp_ctrl = vop2_vp_read(vp, RK3568_VP_DSP_CTRL);
		1041
		1042	dsp_ctrl \|= RK3568_VP_DSP_CTRL__DSP_LUT_EN;
		1043	vop2_vp_write(vp, RK3568_VP_DSP_CTRL, dsp_ctrl);
		1044	}
		1045
		1046	static void vop2_vp_dsp_lut_update_enable(struct vop2_video_port *vp)
		1047	{
		1048	u32 dsp_ctrl = vop2_vp_read(vp, RK3568_VP_DSP_CTRL);
		1049
		1050	dsp_ctrl \|= RK3588_VP_DSP_CTRL__GAMMA_UPDATE_EN;
		1051	vop2_vp_write(vp, RK3568_VP_DSP_CTRL, dsp_ctrl);
		1052	}
		1053
		1054	static inline bool vop2_supports_seamless_gamma_lut_update(struct vop2 *vop2)
		1055	{
		1056	return (vop2->data->soc_id != 3566 && vop2->data->soc_id != 3568);
		1057	}
		1058
		1059	static bool vop2_gamma_lut_in_use(struct vop2 vop2, struct vop2_video_port vp)
		1060	{
		1061	const int nr_vps = vop2->data->nr_vps;
		1062	int gamma_en_vp_id;
		1063
		1064	for (gamma_en_vp_id = 0; gamma_en_vp_id < nr_vps; gamma_en_vp_id++)
		1065	if (vop2_vp_dsp_lut_is_enabled(&vop2->vps[gamma_en_vp_id]))
		1066	break;
		1067
		1068	return gamma_en_vp_id != nr_vps && gamma_en_vp_id != vp->id;
		1069	}
		1070
1001	static void vop2_crtc_atomic_disable(struct drm_crtc *crtc,	1071	static void vop2_crtc_atomic_disable(struct drm_crtc *crtc,
1002	struct drm_atomic_state *state)	1072	struct drm_atomic_state *state)
1003	{	1073	{
@@ -1271,8 +1341,9 @@ static void vop2_plane_atomic_update(struct drm_plane *plane, · 267 unmodified lines
1271	dsp_w = drm_rect_width(dest);	1341	dsp_w = drm_rect_width(dest);
1272		1342
1273	if (dest->x1 + dsp_w > adjusted_mode->hdisplay) {	1343	if (dest->x1 + dsp_w > adjusted_mode->hdisplay) {
1274	drm_err(vop2->drm, "vp%d %s dest->x1[%d] + dsp_w[%d] exceed mode hdisplay[%d]\n",	1344	drm_dbg_kms(vop2->drm,
1275	vp->id, win->data->name, dest->x1, dsp_w, adjusted_mode->hdisplay);	1345	"vp%d %s dest->x1[%d] + dsp_w[%d] exceed mode hdisplay[%d]\n",
		1346	vp->id, win->data->name, dest->x1, dsp_w, adjusted_mode->hdisplay);
1276	dsp_w = adjusted_mode->hdisplay - dest->x1;	1347	dsp_w = adjusted_mode->hdisplay - dest->x1;
1277	if (dsp_w < 4)	1348	if (dsp_w < 4)
1278	dsp_w = 4;	1349	dsp_w = 4;
@@ -1282,8 +1353,9 @@ static void vop2_plane_atomic_update(struct drm_plane *plane, · 3 unmodified lines
1282	dsp_h = drm_rect_height(dest);	1353	dsp_h = drm_rect_height(dest);
1283		1354
1284	if (dest->y1 + dsp_h > adjusted_mode->vdisplay) {	1355	if (dest->y1 + dsp_h > adjusted_mode->vdisplay) {
1285	drm_err(vop2->drm, "vp%d %s dest->y1[%d] + dsp_h[%d] exceed mode vdisplay[%d]\n",	1356	drm_dbg_kms(vop2->drm,
1286	vp->id, win->data->name, dest->y1, dsp_h, adjusted_mode->vdisplay);	1357	"vp%d %s dest->y1[%d] + dsp_h[%d] exceed mode vdisplay[%d]\n",
		1358	vp->id, win->data->name, dest->y1, dsp_h, adjusted_mode->vdisplay);
1287	dsp_h = adjusted_mode->vdisplay - dest->y1;	1359	dsp_h = adjusted_mode->vdisplay - dest->y1;
1288	if (dsp_h < 4)	1360	if (dsp_h < 4)
1289	dsp_h = 4;	1361	dsp_h = 4;
@@ -1296,15 +1368,15 @@ static void vop2_plane_atomic_update(struct drm_plane *plane, · 6 unmodified lines
1296	*/	1368	*/
1297	if (!(win->data->feature & WIN_FEATURE_AFBDC)) {	1369	if (!(win->data->feature & WIN_FEATURE_AFBDC)) {
1298	if (actual_w > dsp_w && (actual_w & 0xf) == 1) {	1370	if (actual_w > dsp_w && (actual_w & 0xf) == 1) {
1299	drm_err(vop2->drm, "vp%d %s act_w[%d] MODE 16 == 1\n",	1371	drm_dbg_kms(vop2->drm, "vp%d %s act_w[%d] MODE 16 == 1\n",
1300	vp->id, win->data->name, actual_w);	1372	vp->id, win->data->name, actual_w);
1301	actual_w -= 1;	1373	actual_w -= 1;
1302	}	1374	}
1303	}	1375	}
1304		1376
1305	if (afbc_en && actual_w % 4) {	1377	if (afbc_en && actual_w % 4) {
1306	drm_err(vop2->drm, "vp%d %s actual_w[%d] not 4 pixel aligned\n",	1378	drm_dbg_kms(vop2->drm, "vp%d %s actual_w[%d] not 4 pixel aligned\n",
1307	vp->id, win->data->name, actual_w);	1379	vp->id, win->data->name, actual_w);
1308	actual_w = ALIGN_DOWN(actual_w, 4);	1380	actual_w = ALIGN_DOWN(actual_w, 4);
1309	}	1381	}
1310		1382
@@ -1341,8 +1413,8 @@ static void vop2_plane_atomic_update(struct drm_plane *plane, · 30 unmodified lines
1341	*/	1413	*/
1342	stride = (fb->pitches[0] << 3) / bpp;	1414	stride = (fb->pitches[0] << 3) / bpp;
1343	if ((stride & 0x3f) && (xmirror \|\| rotate_90 \|\| rotate_270))	1415	if ((stride & 0x3f) && (xmirror \|\| rotate_90 \|\| rotate_270))
1344	drm_err(vop2->drm, "vp%d %s stride[%d] not 64 pixel aligned\n",	1416	drm_dbg_kms(vop2->drm, "vp%d %s stride[%d] not 64 pixel aligned\n",
1345	vp->id, win->data->name, stride);	1417	vp->id, win->data->name, stride);
1346		1418
1347	uv_swap = vop2_afbc_uv_swap(fb->format->format);	1419	uv_swap = vop2_afbc_uv_swap(fb->format->format);
1348	/*	1420	/*
@@ -1482,6 +1554,77 @@ static bool vop2_crtc_mode_fixup(struct drm_crtc *crtc, · 133 unmodified lines
1482	return true;	1554	return true;
1483	}	1555	}
1484		1556
		1557	static void vop2_crtc_write_gamma_lut(struct vop2 vop2, struct drm_crtc crtc)
		1558	{
		1559	const struct vop2_video_port *vp = to_vop2_video_port(crtc);
		1560	const struct vop2_video_port_data *vp_data = &vop2->data->vp[vp->id];
		1561	struct drm_color_lut *lut = crtc->state->gamma_lut->data;
		1562	unsigned int i, bpc = ilog2(vp_data->gamma_lut_len);
		1563	u32 word;
		1564
		1565	for (i = 0; i < crtc->gamma_size; i++) {
		1566	word = (drm_color_lut_extract(lut[i].blue, bpc) << (2 * bpc)) \|
		1567	(drm_color_lut_extract(lut[i].green, bpc) << bpc) \|
		1568	drm_color_lut_extract(lut[i].red, bpc);
		1569
		1570	writel(word, vop2->lut_regs + i * 4);
		1571	}
		1572	}
		1573
		1574	static void vop2_crtc_atomic_set_gamma_seamless(struct vop2 *vop2,
		1575	struct vop2_video_port *vp,
		1576	struct drm_crtc *crtc)
		1577	{
		1578	vop2_writel(vop2, RK3568_LUT_PORT_SEL,
		1579	FIELD_PREP(RK3588_LUT_PORT_SEL__GAMMA_AHB_WRITE_SEL, vp->id));
		1580	vop2_vp_dsp_lut_enable(vp);
		1581	vop2_crtc_write_gamma_lut(vop2, crtc);
		1582	vop2_vp_dsp_lut_update_enable(vp);
		1583	}
		1584
		1585	static void vop2_crtc_atomic_set_gamma_rk356x(struct vop2 *vop2,
		1586	struct vop2_video_port *vp,
		1587	struct drm_crtc *crtc)
		1588	{
		1589	vop2_vp_dsp_lut_disable(vp);
		1590	vop2_cfg_done(vp);
		1591	if (!vop2_vp_dsp_lut_poll_disabled(vp))
		1592	return;
		1593
		1594	vop2_writel(vop2, RK3568_LUT_PORT_SEL, vp->id);
		1595	vop2_crtc_write_gamma_lut(vop2, crtc);
		1596	vop2_vp_dsp_lut_enable(vp);
		1597	}
		1598
		1599	static void vop2_crtc_atomic_try_set_gamma(struct vop2 *vop2,
		1600	struct vop2_video_port *vp,
		1601	struct drm_crtc *crtc,
		1602	struct drm_crtc_state *crtc_state)
		1603	{
		1604	if (!vop2->lut_regs \|\| !crtc_state->color_mgmt_changed)
		1605	return;
		1606
		1607	if (!crtc_state->gamma_lut) {
		1608	vop2_vp_dsp_lut_disable(vp);
		1609	return;
		1610	}
		1611
		1612	if (vop2_supports_seamless_gamma_lut_update(vop2))
		1613	vop2_crtc_atomic_set_gamma_seamless(vop2, vp, crtc);
		1614	else
		1615	vop2_crtc_atomic_set_gamma_rk356x(vop2, vp, crtc);
		1616	}
		1617
		1618	static inline void vop2_crtc_atomic_try_set_gamma_locked(struct vop2 *vop2,
		1619	struct vop2_video_port *vp,
		1620	struct drm_crtc *crtc,
		1621	struct drm_crtc_state *crtc_state)
		1622	{
		1623	vop2_lock(vop2);
		1624	vop2_crtc_atomic_try_set_gamma(vop2, vp, crtc, crtc_state);
		1625	vop2_unlock(vop2);
		1626	}
		1627
1485	static void vop2_dither_setup(struct drm_crtc crtc, u32 dsp_ctrl)	1628	static void vop2_dither_setup(struct drm_crtc crtc, u32 dsp_ctrl)
1486	{	1629	{
1487	struct rockchip_crtc_state *vcstate = to_rockchip_crtc_state(crtc->state);	1630	struct rockchip_crtc_state *vcstate = to_rockchip_crtc_state(crtc->state);
@@ -1721,9 +1864,9 @@ static unsigned long rk3588_calc_cru_cfg(struct vop2_video_port *vp, int id, · 233 unmodified lines
1721	else	1864	else
1722	dclk_out_rate = v_pixclk >> 2;	1865	dclk_out_rate = v_pixclk >> 2;
1723		1866
1724	dclk_rate = rk3588_calc_dclk(dclk_out_rate, 600000);	1867	dclk_rate = rk3588_calc_dclk(dclk_out_rate, 600000000);
1725	if (!dclk_rate) {	1868	if (!dclk_rate) {
1726	drm_err(vop2->drm, "DP dclk_out_rate out of range, dclk_out_rate: %ld KHZ\n",	1869	drm_err(vop2->drm, "DP dclk_out_rate out of range, dclk_out_rate: %ld Hz\n",
1727	dclk_out_rate);	1870	dclk_out_rate);
1728	return 0;	1871	return 0;
1729	}	1872	}
@@ -1738,9 +1881,9 @@ static unsigned long rk3588_calc_cru_cfg(struct vop2_video_port *vp, int id, · 8 unmodified lines
1738	* dclk_rate = N * dclk_core_rate N = (1,2,4 ),	1881	* dclk_rate = N * dclk_core_rate N = (1,2,4 ),
1739	* we get a little factor here	1882	* we get a little factor here
1740	*/	1883	*/
1741	dclk_rate = rk3588_calc_dclk(dclk_out_rate, 600000);	1884	dclk_rate = rk3588_calc_dclk(dclk_out_rate, 600000000);
1742	if (!dclk_rate) {	1885	if (!dclk_rate) {
1743	drm_err(vop2->drm, "MIPI dclk out of range, dclk_out_rate: %ld KHZ\n",	1886	drm_err(vop2->drm, "MIPI dclk out of range, dclk_out_rate: %ld Hz\n",
1744	dclk_out_rate);	1887	dclk_out_rate);
1745	return 0;	1888	return 0;
1746	}	1889	}
@@ -2057,11 +2200,40 @@ static void vop2_crtc_atomic_enable(struct drm_crtc *crtc, · 310 unmodified lines
2057		2200
2058	vop2_vp_write(vp, RK3568_VP_DSP_CTRL, dsp_ctrl);	2201	vop2_vp_write(vp, RK3568_VP_DSP_CTRL, dsp_ctrl);
2059		2202
		2203	vop2_crtc_atomic_try_set_gamma(vop2, vp, crtc, crtc_state);
		2204
2060	drm_crtc_vblank_on(crtc);	2205	drm_crtc_vblank_on(crtc);
2061		2206
2062	vop2_unlock(vop2);	2207	vop2_unlock(vop2);
2063	}	2208	}
2064		2209
		2210	static int vop2_crtc_atomic_check_gamma(struct vop2_video_port *vp,
		2211	struct drm_crtc *crtc,
		2212	struct drm_atomic_state *state,
		2213	struct drm_crtc_state *crtc_state)
		2214	{
		2215	struct vop2 *vop2 = vp->vop2;
		2216	unsigned int len;
		2217
		2218	if (!vp->vop2->lut_regs \|\| !crtc_state->color_mgmt_changed \|\|
		2219	!crtc_state->gamma_lut)
		2220	return 0;
		2221
		2222	len = drm_color_lut_size(crtc_state->gamma_lut);
		2223	if (len != crtc->gamma_size) {
		2224	drm_dbg(vop2->drm, "Invalid LUT size; got %d, expected %d\n",
		2225	len, crtc->gamma_size);
		2226	return -EINVAL;
		2227	}
		2228
		2229	if (!vop2_supports_seamless_gamma_lut_update(vop2) && vop2_gamma_lut_in_use(vop2, vp)) {
		2230	drm_info(vop2->drm, "Gamma LUT can be enabled for only one CRTC at a time\n");
		2231	return -EINVAL;
		2232	}
		2233
		2234	return 0;
		2235	}
		2236
2065	static int vop2_crtc_atomic_check(struct drm_crtc *crtc,	2237	static int vop2_crtc_atomic_check(struct drm_crtc *crtc,
2066	struct drm_atomic_state *state)	2238	struct drm_atomic_state *state)
2067	{	2239	{
@@ -2069,6 +2241,11 @@ static int vop2_crtc_atomic_check(struct drm_crtc *crtc, · 1 unmodified line
2069	struct drm_plane *plane;	2241	struct drm_plane *plane;
2070	int nplanes = 0;	2242	int nplanes = 0;
2071	struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);	2243	struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
		2244	int ret;
		2245
		2246	ret = vop2_crtc_atomic_check_gamma(vp, crtc, state, crtc_state);
		2247	if (ret)
		2248	return ret;
2072		2249
2073	drm_atomic_crtc_state_for_each_plane(plane, crtc_state)	2250	drm_atomic_crtc_state_for_each_plane(plane, crtc_state)
2074	nplanes++;	2251	nplanes++;
@@ -2487,7 +2664,13 @@ static void vop2_crtc_atomic_begin(struct drm_crtc *crtc, · 412 unmodified lines
2487	static void vop2_crtc_atomic_flush(struct drm_crtc *crtc,	2664	static void vop2_crtc_atomic_flush(struct drm_crtc *crtc,
2488	struct drm_atomic_state *state)	2665	struct drm_atomic_state *state)
2489	{	2666	{
		2667	struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
2490	struct vop2_video_port *vp = to_vop2_video_port(crtc);	2668	struct vop2_video_port *vp = to_vop2_video_port(crtc);
		2669	struct vop2 *vop2 = vp->vop2;
		2670
		2671	/* In case of modeset, gamma lut update already happened in atomic enable */
		2672	if (!drm_atomic_crtc_needs_modeset(crtc_state))
		2673	vop2_crtc_atomic_try_set_gamma_locked(vop2, vp, crtc, crtc_state);
2491		2674
2492	vop2_post_config(crtc);	2675	vop2_post_config(crtc);
2493		2676
@@ -2790,7 +2973,12 @@ static int vop2_create_crtcs(struct vop2 *vop2) · 296 unmodified lines
2790	}	2973	}
2791		2974
2792	drm_crtc_helper_add(&vp->crtc, &vop2_crtc_helper_funcs);	2975	drm_crtc_helper_add(&vp->crtc, &vop2_crtc_helper_funcs);
		2976	if (vop2->lut_regs) {
		2977	const struct vop2_video_port_data *vp_data = &vop2_data->vp[vp->id];
2793		2978
		2979	drm_mode_crtc_set_gamma_size(&vp->crtc, vp_data->gamma_lut_len);
		2980	drm_crtc_enable_color_mgmt(&vp->crtc, 0, false, vp_data->gamma_lut_len);
		2981	}
2794	init_completion(&vp->dsp_hold_completion);	2982	init_completion(&vp->dsp_hold_completion);
2795	}	2983	}
2796		2984

drivers/gpu/drm/rockchip/rockchip_drm_vop2.h+5−0

 #define RK3568_REG_CFG_DONE__GLB_CFG_DONE_EN		BIT(15)
 #define RK3568_VP_DSP_CTRL__STANDBY			BIT(31)
+#define RK3568_VP_DSP_CTRL__DSP_LUT_EN			BIT(28)
 #define RK3568_VP_DSP_CTRL__DITHER_DOWN_MODE		BIT(20)
 #define RK3568_VP_DSP_CTRL__DITHER_DOWN_SEL		GENMASK(19, 18)
 #define RK3568_VP_DSP_CTRL__DITHER_DOWN_EN		BIT(17)
 #define RK3568_VP_DSP_CTRL__CORE_DCLK_DIV		BIT(4)
 #define RK3568_VP_DSP_CTRL__OUT_MODE			GENMASK(3, 0)
+#define RK3588_VP_DSP_CTRL__GAMMA_UPDATE_EN		BIT(22)
 #define RK3588_VP_CLK_CTRL__DCLK_OUT_DIV		GENMASK(3, 2)
 #define RK3588_VP_CLK_CTRL__DCLK_CORE_DIV		GENMASK(1, 0)
 #define RK3588_DSP_IF_POL__DP1_PIN_POL			GENMASK(14, 12)
 #define RK3588_DSP_IF_POL__DP0_PIN_POL			GENMASK(10, 8)
+#define RK3588_LUT_PORT_SEL__GAMMA_AHB_WRITE_SEL	GENMASK(13, 12)
 #define RK3568_VP0_MIPI_CTRL__DCLK_DIV2_PHASE_LOCK	BIT(5)
 #define RK3568_VP0_MIPI_CTRL__DCLK_DIV2			BIT(4)

drivers/gpu/drm/solomon/ssd130x.c+1−3

 #include <linux/pwm.h>
 #include <linux/regulator/consumer.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_crtc_helper.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_edid.h>
 #define DRIVER_NAME	"ssd130x"
 #define DRIVER_DESC	"DRM driver for Solomon SSD13xx OLED displays"
-#define DRIVER_DATE	"20220131"
 #define DRIVER_MAJOR	1
 #define DRIVER_MINOR	0
 	DRM_FBDEV_SHMEM_DRIVER_OPS,
 	.name			= DRIVER_NAME,
 	.desc			= DRIVER_DESC,
-	.date			= DRIVER_DATE,
 	.major			= DRIVER_MAJOR,
 	.minor			= DRIVER_MINOR,
 	.driver_features	= DRIVER_ATOMIC | DRIVER_GEM | DRIVER_MODESET,

drivers/gpu/drm/sprd/sprd_drm.c+0−2

 #define DRIVER_NAME	"sprd"
 #define DRIVER_DESC	"Spreadtrum SoCs' DRM Driver"
-#define DRIVER_DATE	"20200201"
 #define DRIVER_MAJOR	1
 #define DRIVER_MINOR	0
 	.name			= DRIVER_NAME,
 	.desc			= DRIVER_DESC,
-	.date			= DRIVER_DATE,
 	.major			= DRIVER_MAJOR,
 	.minor			= DRIVER_MINOR,
 };

drivers/gpu/drm/sti/sti_drv.c+1−3

 #include <linux/of_platform.h>
 #include <linux/platform_device.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_debugfs.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #define DRIVER_NAME	"sti"
 #define DRIVER_DESC	"STMicroelectronics SoC DRM"
-#define DRIVER_DATE	"20140601"
 #define DRIVER_MAJOR	1
 #define DRIVER_MINOR	0
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 };

drivers/gpu/drm/stm/drv.c+1−2

 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_fourcc.h>
 	.driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC,
 	.name = "stm",
 	.desc = "STMicroelectronics SoC DRM",
-	.date = "20170330",
 	.major = 1,
 	.minor = 0,
 	.patchlevel = 0,

drivers/gpu/drm/sun4i/sun4i_drv.c+1−2

 #include <linux/of_reserved_mem.h>
 #include <linux/platform_device.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_dma_helper.h>
 	.fops			= &sun4i_drv_fops,
 	.name			= "sun4i-drm",
 	.desc			= "Allwinner sun4i Display Engine",
-	.date			= "20150629",
 	.major			= 1,
 	.minor			= 0,

drivers/gpu/drm/tegra/drm.c+1−3

 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_debugfs.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fourcc.h>
 #define DRIVER_NAME "tegra"
 #define DRIVER_DESC "NVIDIA Tegra graphics"
-#define DRIVER_DATE "20120330"
 #define DRIVER_MAJOR 1
 #define DRIVER_MINOR 0
 #define DRIVER_PATCHLEVEL 0
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/tidss/tidss_dispc.c+16−12

 {
 	dispc_irq_t old_mask = dispc_k2g_read_irqenable(dispc);
-	/* clear the irqstatus for newly enabled irqs */
+	/* clear the irqstatus for irqs that will be enabled */
 	dispc_k2g_clear_irqstatus(dispc, (mask ^ old_mask) & mask);
 	dispc_k2g_vp_set_irqenable(dispc, 0, mask);
 	dispc_write(dispc, DISPC_IRQENABLE_SET, (1 << 0) | (1 << 7));
+	/* clear the irqstatus for irqs that were disabled */
+	dispc_k2g_clear_irqstatus(dispc, (mask ^ old_mask) & old_mask);
 	/* flush posted write */
 	dispc_k2g_read_irqenable(dispc);
 }
 void dispc_k3_clear_irqstatus(struct dispc_device *dispc, dispc_irq_t clearmask)
 {
 	unsigned int i;
-	u32 top_clear = 0;
 	for (i = 0; i < dispc->feat->num_vps; ++i) {
-		if (clearmask & DSS_IRQ_VP_MASK(i)) {
+		if (clearmask & DSS_IRQ_VP_MASK(i))
 			dispc_k3_vp_write_irqstatus(dispc, i, clearmask);
-			top_clear |= BIT(i);
-		}
 	}
 	for (i = 0; i < dispc->feat->num_planes; ++i) {
-		if (clearmask & DSS_IRQ_PLANE_MASK(i)) {
+		if (clearmask & DSS_IRQ_PLANE_MASK(i))
 			dispc_k3_vid_write_irqstatus(dispc, i, clearmask);
-			top_clear |= BIT(4 + i);
-		}
 	}
-	if (dispc->feat->subrev == DISPC_K2G)
-		return;
-	dispc_write(dispc, DISPC_IRQSTATUS, top_clear);
+	/* always clear the top level irqstatus */
+	dispc_write(dispc, DISPC_IRQSTATUS, dispc_read(dispc, DISPC_IRQSTATUS));
 	/* Flush posted writes */
 	dispc_read(dispc, DISPC_IRQSTATUS);
 	old_mask = dispc_k3_read_irqenable(dispc);
-	/* clear the irqstatus for newly enabled irqs */
+	/* clear the irqstatus for irqs that will be enabled */
 	dispc_k3_clear_irqstatus(dispc, (old_mask ^ mask) & mask);
 	for (i = 0; i < dispc->feat->num_vps; ++i) {
 	if (main_disable)
 		dispc_write(dispc, DISPC_IRQENABLE_CLR, main_disable);
+	/* clear the irqstatus for irqs that were disabled */
+	dispc_k3_clear_irqstatus(dispc, (old_mask ^ mask) & old_mask);
 	/* Flush posted writes */
 	dispc_read(dispc, DISPC_IRQENABLE_SET);
 }
  */
 static void dispc_softreset_k2g(struct dispc_device *dispc)
 {
+	unsigned long flags;
+	spin_lock_irqsave(&dispc->tidss->irq_lock, flags);
 	dispc_set_irqenable(dispc, 0);
 	dispc_read_and_clear_irqstatus(dispc);
+	spin_unlock_irqrestore(&dispc->tidss->irq_lock, flags);
 	for (unsigned int vp_idx = 0; vp_idx < dispc->feat->num_vps; ++vp_idx)
 		VP_REG_FLD_MOD(dispc, vp_idx, DISPC_VP_CONTROL, 0, 0, 0);

drivers/gpu/drm/tidss/tidss_drv.c+2−3

 #include <linux/module.h>
 #include <linux/pm_runtime.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_crtc.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 	DRM_FBDEV_DMA_DRIVER_OPS,
 	.name			= "tidss",
 	.desc			= "TI Keystone DSS",
-	.date			= "20180215",
 	.major			= 1,
 	.minor			= 0,
 };
 	platform_set_drvdata(pdev, tidss);
-	spin_lock_init(&tidss->wait_lock);
+	spin_lock_init(&tidss->irq_lock);
 	ret = dispc_init(tidss);
 	if (ret) {

drivers/gpu/drm/tidss/tidss_drv.h+3−2

@@ -29,8 +29,9 @@ struct tidss_device { · 28 unmodified lines
29		29
30	unsigned int irq;	30	unsigned int irq;
31		31
32	spinlock_t wait_lock; /* protects the irq masks */	32	/* protects the irq masks field and irqenable/irqstatus registers */
33	dispc_irq_t irq_mask; /* enabled irqs in addition to wait_list */	33	spinlock_t irq_lock;
		34	dispc_irq_t irq_mask; /* enabled irqs */
34	};	35	};
35		36
36	#define to_tidss(__dev) container_of(__dev, struct tidss_device, ddev)	37	#define to_tidss(__dev) container_of(__dev, struct tidss_device, ddev)

drivers/gpu/drm/tidss/tidss_irq.c+23−11

 #include "tidss_irq.h"
 #include "tidss_plane.h"
-/* call with wait_lock and dispc runtime held */
 static void tidss_irq_update(struct tidss_device *tidss)
 {
-	assert_spin_locked(&tidss->wait_lock);
+	assert_spin_locked(&tidss->irq_lock);
 	dispc_set_irqenable(tidss->dispc, tidss->irq_mask);
 }
 	u32 hw_videoport = tcrtc->hw_videoport;
 	unsigned long flags;
-	spin_lock_irqsave(&tidss->wait_lock, flags);
+	spin_lock_irqsave(&tidss->irq_lock, flags);
 	tidss->irq_mask |= DSS_IRQ_VP_VSYNC_EVEN(hw_videoport) |
 			   DSS_IRQ_VP_VSYNC_ODD(hw_videoport);
 	tidss_irq_update(tidss);
-	spin_unlock_irqrestore(&tidss->wait_lock, flags);
+	spin_unlock_irqrestore(&tidss->irq_lock, flags);
 }
 void tidss_irq_disable_vblank(struct drm_crtc *crtc)
 	u32 hw_videoport = tcrtc->hw_videoport;
 	unsigned long flags;
-	spin_lock_irqsave(&tidss->wait_lock, flags);
+	spin_lock_irqsave(&tidss->irq_lock, flags);
 	tidss->irq_mask &= ~(DSS_IRQ_VP_VSYNC_EVEN(hw_videoport) |
 			     DSS_IRQ_VP_VSYNC_ODD(hw_videoport));
 	tidss_irq_update(tidss);
-	spin_unlock_irqrestore(&tidss->wait_lock, flags);
+	spin_unlock_irqrestore(&tidss->irq_lock, flags);
 }
 static irqreturn_t tidss_irq_handler(int irq, void *arg)
 	unsigned int id;
 	dispc_irq_t irqstatus;
+	spin_lock(&tidss->irq_lock);
 	irqstatus = dispc_read_and_clear_irqstatus(tidss->dispc);
+	spin_unlock(&tidss->irq_lock);
 	for (id = 0; id < tidss->num_crtcs; id++) {
 		struct drm_crtc *crtc = tidss->crtcs[id];
 			tidss_crtc_error_irq(crtc, irqstatus);
 	}
-	if (irqstatus & DSS_IRQ_DEVICE_OCP_ERR)
+	for (unsigned int i = 0; i < tidss->num_planes; ++i) {
-		dev_err_ratelimited(tidss->dev, "OCP error\n");
+		struct drm_plane *plane = tidss->planes[i];
+		struct tidss_plane *tplane = to_tidss_plane(plane);
+		if (irqstatus & DSS_IRQ_PLANE_FIFO_UNDERFLOW(tplane->hw_plane_id))
+			tidss_plane_error_irq(plane, irqstatus);
+	}
 	return IRQ_HANDLED;
 }
 {
 	unsigned long flags;
-	spin_lock_irqsave(&tidss->wait_lock, flags);
+	spin_lock_irqsave(&tidss->irq_lock, flags);
 	tidss_irq_update(tidss);
-	spin_unlock_irqrestore(&tidss->wait_lock, flags);
+	spin_unlock_irqrestore(&tidss->irq_lock, flags);
 }
 int tidss_irq_install(struct drm_device *ddev, unsigned int irq)
 	if (ret)
 		return ret;
-	tidss->irq_mask = DSS_IRQ_DEVICE_OCP_ERR;
+	tidss->irq_mask = 0;
 	for (unsigned int i = 0; i < tidss->num_crtcs; ++i) {
 		struct tidss_crtc *tcrtc = to_tidss_crtc(tidss->crtcs[i]);
 		tidss->irq_mask |= DSS_IRQ_VP_FRAME_DONE(tcrtc->hw_videoport);
 	}
+	for (unsigned int i = 0; i < tidss->num_planes; ++i) {
+		struct tidss_plane *tplane = to_tidss_plane(tidss->planes[i]);
+		tidss->irq_mask |= DSS_IRQ_PLANE_FIFO_UNDERFLOW(tplane->hw_plane_id);
+	}
 	return 0;
 }

drivers/gpu/drm/tidss/tidss_irq.h+1−3

  * bit use   |D  |fou|FEOL|FEOL|FEOL|FEOL|  UUUU  |          |
  * bit number|0  |1-3|4-7 |8-11|  12-19  | 20-23  |  24-31   |
  *
- * device bits:	D = OCP error
+ * device bits:	D = Unused
  * WB bits:	f = frame done wb, o = wb buffer overflow,
  *		u = wb buffer uncomplete
  * vp bits:	F = frame done, E = vsync even, O = vsync odd, L = sync lost
  * plane bits:	U = fifo underflow
  */
-#define DSS_IRQ_DEVICE_OCP_ERR			BIT(0)
 #define DSS_IRQ_DEVICE_FRAMEDONEWB		BIT(1)
 #define DSS_IRQ_DEVICE_WBBUFFEROVERFLOW		BIT(2)
 #define DSS_IRQ_DEVICE_WBUNCOMPLETEERROR	BIT(3)

drivers/gpu/drm/tidss/tidss_plane.c+8−0

@@ -18,6 +18,14 @@ · 17 unmodified lines
18	#include "tidss_drv.h"	18	#include "tidss_drv.h"
19	#include "tidss_plane.h"	19	#include "tidss_plane.h"
20		20
		21	void tidss_plane_error_irq(struct drm_plane *plane, u64 irqstatus)
		22	{
		23	struct tidss_plane *tplane = to_tidss_plane(plane);
		24
		25	dev_err_ratelimited(plane->dev->dev, "Plane%u underflow (irq %llx)\n",
		26	tplane->hw_plane_id, irqstatus);
		27	}
		28
21	/* drm_plane_helper_funcs */	29	/* drm_plane_helper_funcs */
22		30
23	static int tidss_plane_atomic_check(struct drm_plane *plane,	31	static int tidss_plane_atomic_check(struct drm_plane *plane,

drivers/gpu/drm/tidss/tidss_plane.h+2−0

@@ -22,4 +22,6 @@ struct tidss_plane tidss_plane_create(struct tidss_device tidss, · 21 unmodified lines
22	u32 crtc_mask, const u32 *formats,	22	u32 crtc_mask, const u32 *formats,
23	u32 num_formats);	23	u32 num_formats);
24		24
		25	void tidss_plane_error_irq(struct drm_plane *plane, u64 irqstatus);
		26
25	#endif	27	#endif

drivers/gpu/drm/tilcdc/tilcdc_drv.c+1−2

 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_debugfs.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 	.fops               = &fops,
 	.name               = "tilcdc",
 	.desc               = "TI LCD Controller DRM",
-	.date               = "20121205",
 	.major              = 1,
 	.minor              = 0,
 };

drivers/gpu/drm/tiny/Makefile+1−1

@@ -2,7 +2,7 @@ · 1 unmodified line
2		2
3	obj-$(CONFIG_DRM_ARCPGU) += arcpgu.o	3	obj-$(CONFIG_DRM_ARCPGU) += arcpgu.o
4	obj-$(CONFIG_DRM_BOCHS) += bochs.o	4	obj-$(CONFIG_DRM_BOCHS) += bochs.o
5	obj-$(CONFIG_DRM_CIRRUS_QEMU) += cirrus.o	5	obj-$(CONFIG_DRM_CIRRUS_QEMU) += cirrus-qemu.o
6	obj-$(CONFIG_DRM_GM12U320) += gm12u320.o	6	obj-$(CONFIG_DRM_GM12U320) += gm12u320.o
7	obj-$(CONFIG_DRM_OFDRM) += ofdrm.o	7	obj-$(CONFIG_DRM_OFDRM) += ofdrm.o
8	obj-$(CONFIG_DRM_PANEL_MIPI_DBI) += panel-mipi-dbi.o	8	obj-$(CONFIG_DRM_PANEL_MIPI_DBI) += panel-mipi-dbi.o

drivers/gpu/drm/tiny/arcpgu.c+3−3

  */
 #include <linux/clk.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_debugfs.h>
 #include <drm/drm_device.h>
 #include <drm/drm_drv.h>
 	 * There is only one output port inside each device. It is linked with
 	 * encoder endpoint.
 	 */
-	endpoint_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
+	endpoint_node = of_graph_get_endpoint_by_regs(pdev->dev.of_node, 0, -1);
 	if (endpoint_node) {
 		encoder_node = of_graph_get_remote_port_parent(endpoint_node);
 		of_node_put(endpoint_node);
 	.driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_ATOMIC,
 	.name = "arcpgu",
 	.desc = "ARC PGU Controller",
-	.date = "20160219",
 	.major = 1,
 	.minor = 0,
 	.patchlevel = 0,

drivers/gpu/drm/tiny/bochs.c+1−2

 #include <linux/module.h>
 #include <linux/pci.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_edid.h>
 	.fops			= &bochs_fops,
 	.name			= "bochs-drm",
 	.desc			= "bochs dispi vga interface (qemu stdvga)",
-	.date			= "20130925",
 	.major			= 1,
 	.minor			= 0,
 	DRM_GEM_SHMEM_DRIVER_OPS,

drivers/gpu/drm/tiny/cirrus-qemu.crenamed from drivers/gpu/drm/tiny/cirrus.c+4−6

 #include <video/cirrus.h>
 #include <video/vga.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_atomic_state_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_connector.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_module.h>
 #include <drm/drm_probe_helper.h>
-#define DRIVER_NAME "cirrus"
+#define DRIVER_NAME "cirrus-qemu"
 #define DRIVER_DESC "qemu cirrus vga"
-#define DRIVER_DATE "2019"
 #define DRIVER_MAJOR 2
 #define DRIVER_MINOR 0
 	encoder = &cirrus->encoder;
 	ret = drm_encoder_init(dev, encoder, &cirrus_encoder_funcs,
-			       DRM_MODE_ENCODER_DAC, NULL);
+			       DRM_MODE_ENCODER_VIRTUAL, NULL);
 	if (ret)
 		return ret;
 	encoder->possible_crtcs = drm_crtc_mask(crtc);
 	connector = &cirrus->connector;
 	ret = drm_connector_init(dev, connector, &cirrus_connector_funcs,
-				 DRM_MODE_CONNECTOR_VGA);
+				 DRM_MODE_CONNECTOR_VIRTUAL);
 	if (ret)
 		return ret;
 	drm_connector_helper_add(connector, &cirrus_connector_helper_funcs);
 	.name		 = DRIVER_NAME,
 	.desc		 = DRIVER_DESC,
-	.date		 = DRIVER_DATE,
 	.major		 = DRIVER_MAJOR,
 	.minor		 = DRIVER_MINOR,

drivers/gpu/drm/tiny/gm12u320.c+1−3

 #include <linux/pm.h>
 #include <linux/usb.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_atomic_state_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_connector.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_drv.h>
 #define DRIVER_NAME		"gm12u320"
 #define DRIVER_DESC		"Grain Media GM12U320 USB projector display"
-#define DRIVER_DATE		"2019"
 #define DRIVER_MAJOR		1
 #define DRIVER_MINOR		0
 	.name		 = DRIVER_NAME,
 	.desc		 = DRIVER_DESC,
-	.date		 = DRIVER_DATE,
 	.major		 = DRIVER_MAJOR,
 	.minor		 = DRIVER_MINOR,

drivers/gpu/drm/tiny/hx8357d.c+1−2

 #include <linux/property.h>
 #include <linux/spi/spi.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_atomic_helper.h>
 	.debugfs_init		= mipi_dbi_debugfs_init,
 	.name			= "hx8357d",
 	.desc			= "HX8357D",
-	.date			= "20181023",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/ili9163.c+1−2

 #include <linux/property.h>
 #include <linux/spi/spi.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_atomic_helper.h>
 	.debugfs_init		= mipi_dbi_debugfs_init,
 	.name			= "ili9163",
 	.desc			= "Ilitek ILI9163",
-	.date			= "20210208",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/ili9225.c+1−2

 #include <linux/spi/spi.h>
 #include <video/mipi_display.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fb_dma_helper.h>
 	DRM_FBDEV_DMA_DRIVER_OPS,
 	.name			= "ili9225",
 	.desc			= "Ilitek ILI9225",
-	.date			= "20171106",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/ili9341.c+1−2

 #include <linux/property.h>
 #include <linux/spi/spi.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_atomic_helper.h>
 	.debugfs_init		= mipi_dbi_debugfs_init,
 	.name			= "ili9341",
 	.desc			= "Ilitek ILI9341",
-	.date			= "20180514",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/ili9486.c+1−2

 #include <video/mipi_display.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_atomic_helper.h>
 	.debugfs_init		= mipi_dbi_debugfs_init,
 	.name			= "ili9486",
 	.desc			= "Ilitek ILI9486",
-	.date			= "20200118",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/mi0283qt.c+1−2

 #include <linux/regulator/consumer.h>
 #include <linux/spi/spi.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_atomic_helper.h>
 	.debugfs_init		= mipi_dbi_debugfs_init,
 	.name			= "mi0283qt",
 	.desc			= "Multi-Inno MI0283QT",
-	.date			= "20160614",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/ofdrm.c+1−3

 #include <linux/pci.h>
 #include <linux/platform_device.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_state_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_connector.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_device.h>
 #define DRIVER_NAME	"ofdrm"
 #define DRIVER_DESC	"DRM driver for OF platform devices"
-#define DRIVER_DATE	"20220501"
 #define DRIVER_MAJOR	1
 #define DRIVER_MINOR	0
 	DRM_FBDEV_SHMEM_DRIVER_OPS,
 	.name			= DRIVER_NAME,
 	.desc			= DRIVER_DESC,
-	.date			= DRIVER_DATE,
 	.major			= DRIVER_MAJOR,
 	.minor			= DRIVER_MINOR,
 	.driver_features	= DRIVER_ATOMIC | DRIVER_GEM | DRIVER_MODESET,

drivers/gpu/drm/tiny/panel-mipi-dbi.c+1−2

 #include <linux/regulator/consumer.h>
 #include <linux/spi/spi.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_atomic_helper.h>
 	.debugfs_init		= mipi_dbi_debugfs_init,
 	.name			= "panel-mipi-dbi",
 	.desc			= "MIPI DBI compatible display panel",
-	.date			= "20220103",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/repaper.c+1−2

 #include <linux/spi/spi.h>
 #include <linux/thermal.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_connector.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_drv.h>
 	DRM_FBDEV_DMA_DRIVER_OPS,
 	.name			= "repaper",
 	.desc			= "Pervasive Displays RePaper e-ink panels",
-	.date			= "20170405",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/sharp-memory.c+1−2

 // SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_connector.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_drv.h>
 	DRM_FBDEV_DMA_DRIVER_OPS,
 	.name			= "sharp_memory_display",
 	.desc			= "Sharp Display Memory LCD",
-	.date			= "20231129",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/simpledrm.c+1−3

 #include <linux/pm_domain.h>
 #include <linux/regulator/consumer.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_state_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_connector.h>
 #include <drm/drm_crtc_helper.h>
 #include <drm/drm_damage_helper.h>
 #define DRIVER_NAME	"simpledrm"
 #define DRIVER_DESC	"DRM driver for simple-framebuffer platform devices"
-#define DRIVER_DATE	"20200625"
 #define DRIVER_MAJOR	1
 #define DRIVER_MINOR	0
 	DRM_FBDEV_SHMEM_DRIVER_OPS,
 	.name			= DRIVER_NAME,
 	.desc			= DRIVER_DESC,
-	.date			= DRIVER_DATE,
 	.major			= DRIVER_MAJOR,
 	.minor			= DRIVER_MINOR,
 	.driver_features	= DRIVER_ATOMIC | DRIVER_GEM | DRIVER_MODESET,

drivers/gpu/drm/tiny/st7586.c+1−2

 #include <linux/spi/spi.h>
 #include <video/mipi_display.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_damage_helper.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fb_dma_helper.h>
 	.debugfs_init		= mipi_dbi_debugfs_init,
 	.name			= "st7586",
 	.desc			= "Sitronix ST7586",
-	.date			= "20170801",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/tiny/st7735r.c+1−2

 #include <linux/spi/spi.h>
 #include <video/mipi_display.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_gem_atomic_helper.h>
 	.debugfs_init		= mipi_dbi_debugfs_init,
 	.name			= "st7735r",
 	.desc			= "Sitronix ST7735R",
-	.date			= "20171128",
 	.major			= 1,
 	.minor			= 0,
 };

drivers/gpu/drm/ttm/ttm_bo_vm.c+7−7

@@ -58,13 +58,13 @@ static vm_fault_t ttm_bo_vm_fault_idle(struct ttm_buffer_object *bo, · 57 unmodified lines
58	if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT)	58	if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT)
59	return VM_FAULT_RETRY;	59	return VM_FAULT_RETRY;
60		60
61	ttm_bo_get(bo);	61	drm_gem_object_get(&bo->base);
62	mmap_read_unlock(vmf->vma->vm_mm);	62	mmap_read_unlock(vmf->vma->vm_mm);
63	(void)dma_resv_wait_timeout(bo->base.resv,	63	(void)dma_resv_wait_timeout(bo->base.resv,
64	DMA_RESV_USAGE_KERNEL, true,	64	DMA_RESV_USAGE_KERNEL, true,
65	MAX_SCHEDULE_TIMEOUT);	65	MAX_SCHEDULE_TIMEOUT);
66	dma_resv_unlock(bo->base.resv);	66	dma_resv_unlock(bo->base.resv);
67	ttm_bo_put(bo);	67	drm_gem_object_put(&bo->base);
68	return VM_FAULT_RETRY;	68	return VM_FAULT_RETRY;
69	}	69	}
70		70
@@ -130,12 +130,12 @@ vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo, · 59 unmodified lines
130	*/	130	*/
131	if (fault_flag_allow_retry_first(vmf->flags)) {	131	if (fault_flag_allow_retry_first(vmf->flags)) {
132	if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) {	132	if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) {
133	ttm_bo_get(bo);	133	drm_gem_object_get(&bo->base);
134	mmap_read_unlock(vmf->vma->vm_mm);	134	mmap_read_unlock(vmf->vma->vm_mm);
135	if (!dma_resv_lock_interruptible(bo->base.resv,	135	if (!dma_resv_lock_interruptible(bo->base.resv,
136	NULL))	136	NULL))
137	dma_resv_unlock(bo->base.resv);	137	dma_resv_unlock(bo->base.resv);
138	ttm_bo_put(bo);	138	drm_gem_object_put(&bo->base);
139	}	139	}
140		140
141	return VM_FAULT_RETRY;	141	return VM_FAULT_RETRY;
@@ -353,7 +353,7 @@ void ttm_bo_vm_open(struct vm_area_struct *vma) · 211 unmodified lines
353		353
354	WARN_ON(bo->bdev->dev_mapping != vma->vm_file->f_mapping);	354	WARN_ON(bo->bdev->dev_mapping != vma->vm_file->f_mapping);
355		355
356	ttm_bo_get(bo);	356	drm_gem_object_get(&bo->base);
357	}	357	}
358	EXPORT_SYMBOL(ttm_bo_vm_open);	358	EXPORT_SYMBOL(ttm_bo_vm_open);
359		359
@@ -361,7 +361,7 @@ void ttm_bo_vm_close(struct vm_area_struct *vma) · 1 unmodified line
361	{	361	{
362	struct ttm_buffer_object *bo = vma->vm_private_data;	362	struct ttm_buffer_object *bo = vma->vm_private_data;
363		363
364	ttm_bo_put(bo);	364	drm_gem_object_put(&bo->base);
365	vma->vm_private_data = NULL;	365	vma->vm_private_data = NULL;
366	}	366	}
367	EXPORT_SYMBOL(ttm_bo_vm_close);	367	EXPORT_SYMBOL(ttm_bo_vm_close);
@@ -462,7 +462,7 @@ int ttm_bo_mmap_obj(struct vm_area_struct vma, struct ttm_buffer_object bo) · 94 unmodified lines
462	if (is_cow_mapping(vma->vm_flags))	462	if (is_cow_mapping(vma->vm_flags))
463	return -EINVAL;	463	return -EINVAL;
464		464
465	ttm_bo_get(bo);	465	drm_gem_object_get(&bo->base);
466		466
467	/*	467	/*
468	* Drivers may want to override the vm_ops field. Otherwise we	468	* Drivers may want to override the vm_ops field. Otherwise we

drivers/gpu/drm/tve200/tve200_drv.c+1−2

 #include <linux/shmem_fs.h>
 #include <linux/slab.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_bridge.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_fourcc.h>
 	.fops = &drm_fops,
 	.name = "tve200",
 	.desc = DRIVER_DESC,
-	.date = "20170703",
 	.major = 1,
 	.minor = 0,
 	.patchlevel = 0,

drivers/gpu/drm/udl/udl_drv.c+1−2

 #include <linux/module.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_drv.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_fbdev_shmem.h>
 #include <drm/drm_file.h>
 #include <drm/drm_gem_shmem_helper.h>
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/udl/udl_drv.h+0−1

@@ -26,7 +26,6 @@ struct drm_mode_create_dumb; · 25 unmodified lines
26		26
27	#define DRIVER_NAME "udl"	27	#define DRIVER_NAME "udl"
28	#define DRIVER_DESC "DisplayLink"	28	#define DRIVER_DESC "DisplayLink"
29	#define DRIVER_DATE "20120220"
30		29
31	#define DRIVER_MAJOR 0	30	#define DRIVER_MAJOR 0
32	#define DRIVER_MINOR 0	31	#define DRIVER_MINOR 0

drivers/gpu/drm/v3d/v3d_debugfs.c+2−2

@@ -237,8 +237,8 @@ static int v3d_measure_clock(struct seq_file m, void unused) · 236 unmodified lines
237	if (v3d->ver >= 40) {	237	if (v3d->ver >= 40) {
238	int cycle_count_reg = V3D_PCTR_CYCLE_COUNT(v3d->ver);	238	int cycle_count_reg = V3D_PCTR_CYCLE_COUNT(v3d->ver);
239	V3D_CORE_WRITE(core, V3D_V4_PCTR_0_SRC_0_3,	239	V3D_CORE_WRITE(core, V3D_V4_PCTR_0_SRC_0_3,
240	V3D_SET_FIELD(cycle_count_reg,	240	V3D_SET_FIELD_VER(cycle_count_reg,
241	V3D_PCTR_S0));	241	V3D_PCTR_S0, v3d->ver));
242	V3D_CORE_WRITE(core, V3D_V4_PCTR_0_CLR, 1);	242	V3D_CORE_WRITE(core, V3D_V4_PCTR_0_CLR, 1);
243	V3D_CORE_WRITE(core, V3D_V4_PCTR_0_EN, 1);	243	V3D_CORE_WRITE(core, V3D_V4_PCTR_0_EN, 1);
244	} else {	244	} else {

drivers/gpu/drm/v3d/v3d_drv.c+1−2

 #define DRIVER_NAME "v3d"
 #define DRIVER_DESC "Broadcom V3D graphics"
-#define DRIVER_DATE "20180419"
 #define DRIVER_MAJOR 1
 #define DRIVER_MINOR 0
 #define DRIVER_PATCHLEVEL 0
 	DRM_IOCTL_DEF_DRV(V3D_PERFMON_GET_VALUES, v3d_perfmon_get_values_ioctl, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(V3D_SUBMIT_CPU, v3d_submit_cpu_ioctl, DRM_RENDER_ALLOW | DRM_AUTH),
 	DRM_IOCTL_DEF_DRV(V3D_PERFMON_GET_COUNTER, v3d_perfmon_get_counter_ioctl, DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(V3D_PERFMON_SET_GLOBAL, v3d_perfmon_set_global_ioctl, DRM_RENDER_ALLOW),
 };
 static const struct drm_driver v3d_drm_driver = {
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/v3d/v3d_drv.h+8−0

 		u32 num_allocated;
 		u32 pages_allocated;
 	} bo_stats;
+	/* To support a performance analysis tool in user space, we require
+	 * a single, globally configured performance monitor (perfmon) for
+	 * all jobs.
+	 */
+	struct v3d_perfmon *global_perfmon;
 };
 static inline struct v3d_dev *
 				 struct drm_file *file_priv);
 int v3d_perfmon_get_counter_ioctl(struct drm_device *dev, void *data,
 				  struct drm_file *file_priv);
+int v3d_perfmon_set_global_ioctl(struct drm_device *dev, void *data,
+				 struct drm_file *file_priv);
 /* v3d_sysfs.c */
 int v3d_sysfs_init(struct device *dev);

drivers/gpu/drm/v3d/v3d_perfmon.c+50−7

@@ -240,17 +240,18 @@ void v3d_perfmon_start(struct v3d_dev v3d, struct v3d_perfmon perfmon) · 239 unmodified lines
240		240
241	for (i = 0; i < ncounters; i++) {	241	for (i = 0; i < ncounters; i++) {
242	u32 source = i / 4;	242	u32 source = i / 4;
243	u32 channel = V3D_SET_FIELD(perfmon->counters[i], V3D_PCTR_S0);	243	u32 channel = V3D_SET_FIELD_VER(perfmon->counters[i], V3D_PCTR_S0,
		244	v3d->ver);
244		245
245	i++;	246	i++;
246	channel \|= V3D_SET_FIELD(i < ncounters ? perfmon->counters[i] : 0,	247	channel \|= V3D_SET_FIELD_VER(i < ncounters ? perfmon->counters[i] : 0,
247	V3D_PCTR_S1);	248	V3D_PCTR_S1, v3d->ver);
248	i++;	249	i++;
249	channel \|= V3D_SET_FIELD(i < ncounters ? perfmon->counters[i] : 0,	250	channel \|= V3D_SET_FIELD_VER(i < ncounters ? perfmon->counters[i] : 0,
250	V3D_PCTR_S2);	251	V3D_PCTR_S2, v3d->ver);
251	i++;	252	i++;
252	channel \|= V3D_SET_FIELD(i < ncounters ? perfmon->counters[i] : 0,	253	channel \|= V3D_SET_FIELD_VER(i < ncounters ? perfmon->counters[i] : 0,
253	V3D_PCTR_S3);	254	V3D_PCTR_S3, v3d->ver);
254	V3D_CORE_WRITE(0, V3D_V4_PCTR_0_SRC_X(source), channel);	255	V3D_CORE_WRITE(0, V3D_V4_PCTR_0_SRC_X(source), channel);
255	}	256	}
256		257
@@ -312,6 +313,9 @@ static int v3d_perfmon_idr_del(int id, void elem, void data) · 55 unmodified lines
312	if (perfmon == v3d->active_perfmon)	313	if (perfmon == v3d->active_perfmon)
313	v3d_perfmon_stop(v3d, perfmon, false);	314	v3d_perfmon_stop(v3d, perfmon, false);
314		315
		316	/* If the global perfmon is being destroyed, set it to NULL */
		317	cmpxchg(&v3d->global_perfmon, perfmon, NULL);
		318
315	v3d_perfmon_put(perfmon);	319	v3d_perfmon_put(perfmon);
316		320
317	return 0;	321	return 0;
@@ -383,6 +387,7 @@ int v3d_perfmon_destroy_ioctl(struct drm_device dev, void data, · 65 unmodified lines
383	{	387	{
384	struct v3d_file_priv *v3d_priv = file_priv->driver_priv;	388	struct v3d_file_priv *v3d_priv = file_priv->driver_priv;
385	struct drm_v3d_perfmon_destroy *req = data;	389	struct drm_v3d_perfmon_destroy *req = data;
		390	struct v3d_dev *v3d = v3d_priv->v3d;
386	struct v3d_perfmon *perfmon;	391	struct v3d_perfmon *perfmon;
387		392
388	mutex_lock(&v3d_priv->perfmon.lock);	393	mutex_lock(&v3d_priv->perfmon.lock);
@@ -392,6 +397,13 @@ int v3d_perfmon_destroy_ioctl(struct drm_device dev, void data, · 3 unmodified lines
392	if (!perfmon)	397	if (!perfmon)
393	return -EINVAL;	398	return -EINVAL;
394		399
		400	/* If the active perfmon is being destroyed, stop it first */
		401	if (perfmon == v3d->active_perfmon)
		402	v3d_perfmon_stop(v3d, perfmon, false);
		403
		404	/* If the global perfmon is being destroyed, set it to NULL */
		405	cmpxchg(&v3d->global_perfmon, perfmon, NULL);
		406
395	v3d_perfmon_put(perfmon);	407	v3d_perfmon_put(perfmon);
396		408
397	return 0;	409	return 0;
@@ -451,3 +463,34 @@ int v3d_perfmon_get_counter_ioctl(struct drm_device dev, void data, · 53 unmodified lines
451		463
452	return 0;	464	return 0;
453	}	465	}
		466
		467	int v3d_perfmon_set_global_ioctl(struct drm_device dev, void data,
		468	struct drm_file *file_priv)
		469	{
		470	struct v3d_file_priv *v3d_priv = file_priv->driver_priv;
		471	struct drm_v3d_perfmon_set_global *req = data;
		472	struct v3d_dev *v3d = to_v3d_dev(dev);
		473	struct v3d_perfmon *perfmon;
		474
		475	if (req->flags & ~DRM_V3D_PERFMON_CLEAR_GLOBAL)
		476	return -EINVAL;
		477
		478	perfmon = v3d_perfmon_find(v3d_priv, req->id);
		479	if (!perfmon)
		480	return -EINVAL;
		481
		482	/* If the request is to clear the global performance monitor */
		483	if (req->flags & DRM_V3D_PERFMON_CLEAR_GLOBAL) {
		484	if (!v3d->global_perfmon)
		485	return -EINVAL;
		486
		487	xchg(&v3d->global_perfmon, NULL);
		488
		489	return 0;
		490	}
		491
		492	if (cmpxchg(&v3d->global_perfmon, NULL, perfmon))
		493	return -EBUSY;
		494
		495	return 0;
		496	}

drivers/gpu/drm/v3d/v3d_regs.h+17−12

@@ -15,6 +15,14 @@ · 14 unmodified lines
15	fieldval & field##_MASK; \	15	fieldval & field##_MASK; \
16	})	16	})
17		17
		18	#define V3D_SET_FIELD_VER(value, field, ver) \
		19	({ \
		20	typeof(ver) _ver = (ver); \
		21	u32 fieldval = (value) << field##_SHIFT(_ver); \
		22	WARN_ON((fieldval & ~field##_MASK(_ver)) != 0); \
		23	fieldval & field##_MASK(_ver); \
		24	})
		25
18	#define V3D_GET_FIELD(word, field) (((word) & field##_MASK) >> \	26	#define V3D_GET_FIELD(word, field) (((word) & field##_MASK) >> \
19	field##_SHIFT)	27	field##_SHIFT)
20		28
@@ -354,18 +362,15 @@ · 333 unmodified lines
354	#define V3D_V4_PCTR_0_SRC_28_31 0x0067c	362	#define V3D_V4_PCTR_0_SRC_28_31 0x0067c
355	#define V3D_V4_PCTR_0_SRC_X(x) (V3D_V4_PCTR_0_SRC_0_3 + \	363	#define V3D_V4_PCTR_0_SRC_X(x) (V3D_V4_PCTR_0_SRC_0_3 + \
356	4 * (x))	364	4 * (x))
357	# define V3D_PCTR_S0_MASK V3D_MASK(6, 0)	365	# define V3D_PCTR_S0_MASK(ver) (((ver) >= 71) ? V3D_MASK(7, 0) : V3D_MASK(6, 0))
358	# define V3D_V7_PCTR_S0_MASK V3D_MASK(7, 0)	366	# define V3D_PCTR_S0_SHIFT(ver) 0
359	# define V3D_PCTR_S0_SHIFT 0	367	# define V3D_PCTR_S1_MASK(ver) (((ver) >= 71) ? V3D_MASK(15, 8) : V3D_MASK(14, 8))
360	# define V3D_PCTR_S1_MASK V3D_MASK(14, 8)	368	# define V3D_PCTR_S1_SHIFT(ver) 8
361	# define V3D_V7_PCTR_S1_MASK V3D_MASK(15, 8)	369	# define V3D_PCTR_S2_MASK(ver) (((ver) >= 71) ? V3D_MASK(23, 16) : V3D_MASK(22, 16))
362	# define V3D_PCTR_S1_SHIFT 8	370	# define V3D_PCTR_S2_SHIFT(ver) 16
363	# define V3D_PCTR_S2_MASK V3D_MASK(22, 16)	371	# define V3D_PCTR_S3_MASK(ver) (((ver) >= 71) ? V3D_MASK(31, 24) : V3D_MASK(30, 24))
364	# define V3D_V7_PCTR_S2_MASK V3D_MASK(23, 16)	372	# define V3D_PCTR_S3_SHIFT(ver) 24
365	# define V3D_PCTR_S2_SHIFT 16	373
366	# define V3D_PCTR_S3_MASK V3D_MASK(30, 24)
367	# define V3D_V7_PCTR_S3_MASK V3D_MASK(31, 24)
368	# define V3D_PCTR_S3_SHIFT 24
369	#define V3D_PCTR_CYCLE_COUNT(ver) ((ver >= 71) ? 0 : 32)	374	#define V3D_PCTR_CYCLE_COUNT(ver) ((ver >= 71) ? 0 : 32)
370		375
371	/* Output values of the counters. */	376	/* Output values of the counters. */

drivers/gpu/drm/v3d/v3d_sched.c+11−3

 static void
 v3d_switch_perfmon(struct v3d_dev *v3d, struct v3d_job *job)
 {
-	if (job->perfmon != v3d->active_perfmon)
+	struct v3d_perfmon *perfmon = v3d->global_perfmon;
+	if (!perfmon)
+		perfmon = job->perfmon;
+	if (perfmon == v3d->active_perfmon)
+		return;
+	if (perfmon != v3d->active_perfmon)
 		v3d_perfmon_stop(v3d, v3d->active_perfmon, true);
-	if (job->perfmon && v3d->active_perfmon != job->perfmon)
+	if (perfmon && v3d->active_perfmon != perfmon)
-		v3d_perfmon_start(v3d, job->perfmon);
+		v3d_perfmon_start(v3d, perfmon);
 }
 static void

drivers/gpu/drm/v3d/v3d_submit.c+10−0

 		goto fail;
 	if (args->perfmon_id) {
+		if (v3d->global_perfmon) {
+			ret = -EAGAIN;
+			goto fail_perfmon;
+		}
 		render->base.perfmon = v3d_perfmon_find(v3d_priv,
 							args->perfmon_id);
 		goto fail;
 	if (args->perfmon_id) {
+		if (v3d->global_perfmon) {
+			ret = -EAGAIN;
+			goto fail_perfmon;
+		}
 		job->base.perfmon = v3d_perfmon_find(v3d_priv,
 						     args->perfmon_id);
 		if (!job->base.perfmon) {

drivers/gpu/drm/vboxvideo/vbox_drv.c+1−2

 #include <linux/pci.h>
 #include <linux/vt_kern.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_ttm.h>
 #include <drm/drm_file.h>
 	.fops = &vbox_fops,
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/vboxvideo/vbox_drv.h+0−1

@@ -25,7 +25,6 @@ · 24 unmodified lines
25		25
26	#define DRIVER_NAME "vboxvideo"	26	#define DRIVER_NAME "vboxvideo"
27	#define DRIVER_DESC "Oracle VM VirtualBox Graphics Card"	27	#define DRIVER_DESC "Oracle VM VirtualBox Graphics Card"
28	#define DRIVER_DATE "20130823"
29		28
30	#define DRIVER_MAJOR 1	29	#define DRIVER_MAJOR 1
31	#define DRIVER_MINOR 0	30	#define DRIVER_MINOR 0

drivers/gpu/drm/vc4/tests/vc4_mock.c+4−4

@@ -51,8 +51,8 @@ struct vc4_mock_desc { · 50 unmodified lines
51		51
52	static const struct vc4_mock_desc vc4_mock =	52	static const struct vc4_mock_desc vc4_mock =
53	VC4_MOCK_DESC(	53	VC4_MOCK_DESC(
54	VC4_MOCK_CRTC_DESC(&vc4_txp_crtc_data,	54	VC4_MOCK_CRTC_DESC(&bcm2835_txp_data.base,
55	VC4_MOCK_OUTPUT_DESC(VC4_ENCODER_TYPE_TXP,	55	VC4_MOCK_OUTPUT_DESC(VC4_ENCODER_TYPE_TXP0,
56	DRM_MODE_ENCODER_VIRTUAL,	56	DRM_MODE_ENCODER_VIRTUAL,
57	DRM_MODE_CONNECTOR_WRITEBACK)),	57	DRM_MODE_CONNECTOR_WRITEBACK)),
58	VC4_MOCK_PIXELVALVE_DESC(&bcm2835_pv0_data,	58	VC4_MOCK_PIXELVALVE_DESC(&bcm2835_pv0_data,
@@ -77,8 +77,8 @@ static const struct vc4_mock_desc vc4_mock = · 18 unmodified lines
77		77
78	static const struct vc4_mock_desc vc5_mock =	78	static const struct vc4_mock_desc vc5_mock =
79	VC4_MOCK_DESC(	79	VC4_MOCK_DESC(
80	VC4_MOCK_CRTC_DESC(&vc4_txp_crtc_data,	80	VC4_MOCK_CRTC_DESC(&bcm2835_txp_data.base,
81	VC4_MOCK_OUTPUT_DESC(VC4_ENCODER_TYPE_TXP,	81	VC4_MOCK_OUTPUT_DESC(VC4_ENCODER_TYPE_TXP0,
82	DRM_MODE_ENCODER_VIRTUAL,	82	DRM_MODE_ENCODER_VIRTUAL,
83	DRM_MODE_CONNECTOR_WRITEBACK)),	83	DRM_MODE_CONNECTOR_WRITEBACK)),
84	VC4_MOCK_PIXELVALVE_DESC(&bcm2711_pv0_data,	84	VC4_MOCK_PIXELVALVE_DESC(&bcm2711_pv0_data,

drivers/gpu/drm/vc4/tests/vc4_test_pv_muxing.c+53−53

@@ -90,7 +90,7 @@ static const struct encoder_constraint vc4_encoder_constraints[] = { · 89 unmodified lines
90	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DSI0, 0),	90	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DSI0, 0),
91	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_HDMI0, 1),	91	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_HDMI0, 1),
92	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_VEC, 1),	92	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_VEC, 1),
93	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_TXP, 2),	93	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_TXP0, 2),
94	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DSI1, 2),	94	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DSI1, 2),
95	};	95	};
96		96
@@ -98,7 +98,7 @@ static const struct encoder_constraint vc5_encoder_constraints[] = { · 1 unmodified line
98	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DPI, 0),	98	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DPI, 0),
99	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DSI0, 0),	99	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DSI0, 0),
100	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_VEC, 1),	100	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_VEC, 1),
101	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_TXP, 0, 2),	101	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_TXP0, 0, 2),
102	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DSI1, 0, 1, 2),	102	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_DSI1, 0, 1, 2),
103	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_HDMI0, 0, 1, 2),	103	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_HDMI0, 0, 1, 2),
104	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_HDMI1, 0, 1, 2),	104	ENCODER_CONSTRAINT(VC4_ENCODER_TYPE_HDMI1, 0, 1, 2),
@@ -207,7 +207,7 @@ static const struct pv_muxing_param vc4_test_pv_muxing_params[] = { · 102 unmodified lines
207	VC4_PV_MUXING_TEST("1 output: DSI1",	207	VC4_PV_MUXING_TEST("1 output: DSI1",
208	VC4_ENCODER_TYPE_DSI1),	208	VC4_ENCODER_TYPE_DSI1),
209	VC4_PV_MUXING_TEST("1 output: TXP",	209	VC4_PV_MUXING_TEST("1 output: TXP",
210	VC4_ENCODER_TYPE_TXP),	210	VC4_ENCODER_TYPE_TXP0),
211	VC4_PV_MUXING_TEST("2 outputs: DSI0, HDMI0",	211	VC4_PV_MUXING_TEST("2 outputs: DSI0, HDMI0",
212	VC4_ENCODER_TYPE_DSI0,	212	VC4_ENCODER_TYPE_DSI0,
213	VC4_ENCODER_TYPE_HDMI0),	213	VC4_ENCODER_TYPE_HDMI0),
@@ -219,7 +219,7 @@ static const struct pv_muxing_param vc4_test_pv_muxing_params[] = { · 5 unmodified lines
219	VC4_ENCODER_TYPE_DSI1),	219	VC4_ENCODER_TYPE_DSI1),
220	VC4_PV_MUXING_TEST("2 outputs: DSI0, TXP",	220	VC4_PV_MUXING_TEST("2 outputs: DSI0, TXP",
221	VC4_ENCODER_TYPE_DSI0,	221	VC4_ENCODER_TYPE_DSI0,
222	VC4_ENCODER_TYPE_TXP),	222	VC4_ENCODER_TYPE_TXP0),
223	VC4_PV_MUXING_TEST("2 outputs: DPI, HDMI0",	223	VC4_PV_MUXING_TEST("2 outputs: DPI, HDMI0",
224	VC4_ENCODER_TYPE_DPI,	224	VC4_ENCODER_TYPE_DPI,
225	VC4_ENCODER_TYPE_HDMI0),	225	VC4_ENCODER_TYPE_HDMI0),
@@ -231,19 +231,19 @@ static const struct pv_muxing_param vc4_test_pv_muxing_params[] = { · 5 unmodified lines
231	VC4_ENCODER_TYPE_DSI1),	231	VC4_ENCODER_TYPE_DSI1),
232	VC4_PV_MUXING_TEST("2 outputs: DPI, TXP",	232	VC4_PV_MUXING_TEST("2 outputs: DPI, TXP",
233	VC4_ENCODER_TYPE_DPI,	233	VC4_ENCODER_TYPE_DPI,
234	VC4_ENCODER_TYPE_TXP),	234	VC4_ENCODER_TYPE_TXP0),
235	VC4_PV_MUXING_TEST("2 outputs: HDMI0, DSI1",	235	VC4_PV_MUXING_TEST("2 outputs: HDMI0, DSI1",
236	VC4_ENCODER_TYPE_HDMI0,	236	VC4_ENCODER_TYPE_HDMI0,
237	VC4_ENCODER_TYPE_DSI1),	237	VC4_ENCODER_TYPE_DSI1),
238	VC4_PV_MUXING_TEST("2 outputs: HDMI0, TXP",	238	VC4_PV_MUXING_TEST("2 outputs: HDMI0, TXP",
239	VC4_ENCODER_TYPE_HDMI0,	239	VC4_ENCODER_TYPE_HDMI0,
240	VC4_ENCODER_TYPE_TXP),	240	VC4_ENCODER_TYPE_TXP0),
241	VC4_PV_MUXING_TEST("2 outputs: VEC, DSI1",	241	VC4_PV_MUXING_TEST("2 outputs: VEC, DSI1",
242	VC4_ENCODER_TYPE_VEC,	242	VC4_ENCODER_TYPE_VEC,
243	VC4_ENCODER_TYPE_DSI1),	243	VC4_ENCODER_TYPE_DSI1),
244	VC4_PV_MUXING_TEST("2 outputs: VEC, TXP",	244	VC4_PV_MUXING_TEST("2 outputs: VEC, TXP",
245	VC4_ENCODER_TYPE_VEC,	245	VC4_ENCODER_TYPE_VEC,
246	VC4_ENCODER_TYPE_TXP),	246	VC4_ENCODER_TYPE_TXP0),
247	VC4_PV_MUXING_TEST("3 outputs: DSI0, HDMI0, DSI1",	247	VC4_PV_MUXING_TEST("3 outputs: DSI0, HDMI0, DSI1",
248	VC4_ENCODER_TYPE_DSI0,	248	VC4_ENCODER_TYPE_DSI0,
249	VC4_ENCODER_TYPE_HDMI0,	249	VC4_ENCODER_TYPE_HDMI0,
@@ -251,7 +251,7 @@ static const struct pv_muxing_param vc4_test_pv_muxing_params[] = { · 1 unmodified line
251	VC4_PV_MUXING_TEST("3 outputs: DSI0, HDMI0, TXP",	251	VC4_PV_MUXING_TEST("3 outputs: DSI0, HDMI0, TXP",
252	VC4_ENCODER_TYPE_DSI0,	252	VC4_ENCODER_TYPE_DSI0,
253	VC4_ENCODER_TYPE_HDMI0,	253	VC4_ENCODER_TYPE_HDMI0,
254	VC4_ENCODER_TYPE_TXP),	254	VC4_ENCODER_TYPE_TXP0),
255	VC4_PV_MUXING_TEST("3 outputs: DSI0, VEC, DSI1",	255	VC4_PV_MUXING_TEST("3 outputs: DSI0, VEC, DSI1",
256	VC4_ENCODER_TYPE_DSI0,	256	VC4_ENCODER_TYPE_DSI0,
257	VC4_ENCODER_TYPE_VEC,	257	VC4_ENCODER_TYPE_VEC,
@@ -259,7 +259,7 @@ static const struct pv_muxing_param vc4_test_pv_muxing_params[] = { · 1 unmodified line
259	VC4_PV_MUXING_TEST("3 outputs: DSI0, VEC, TXP",	259	VC4_PV_MUXING_TEST("3 outputs: DSI0, VEC, TXP",
260	VC4_ENCODER_TYPE_DSI0,	260	VC4_ENCODER_TYPE_DSI0,
261	VC4_ENCODER_TYPE_VEC,	261	VC4_ENCODER_TYPE_VEC,
262	VC4_ENCODER_TYPE_TXP),	262	VC4_ENCODER_TYPE_TXP0),
263	VC4_PV_MUXING_TEST("3 outputs: DPI, HDMI0, DSI1",	263	VC4_PV_MUXING_TEST("3 outputs: DPI, HDMI0, DSI1",
264	VC4_ENCODER_TYPE_DPI,	264	VC4_ENCODER_TYPE_DPI,
265	VC4_ENCODER_TYPE_HDMI0,	265	VC4_ENCODER_TYPE_HDMI0,
@@ -267,7 +267,7 @@ static const struct pv_muxing_param vc4_test_pv_muxing_params[] = { · 1 unmodified line
267	VC4_PV_MUXING_TEST("3 outputs: DPI, HDMI0, TXP",	267	VC4_PV_MUXING_TEST("3 outputs: DPI, HDMI0, TXP",
268	VC4_ENCODER_TYPE_DPI,	268	VC4_ENCODER_TYPE_DPI,
269	VC4_ENCODER_TYPE_HDMI0,	269	VC4_ENCODER_TYPE_HDMI0,
270	VC4_ENCODER_TYPE_TXP),	270	VC4_ENCODER_TYPE_TXP0),
271	VC4_PV_MUXING_TEST("3 outputs: DPI, VEC, DSI1",	271	VC4_PV_MUXING_TEST("3 outputs: DPI, VEC, DSI1",
272	VC4_ENCODER_TYPE_DPI,	272	VC4_ENCODER_TYPE_DPI,
273	VC4_ENCODER_TYPE_VEC,	273	VC4_ENCODER_TYPE_VEC,
@@ -275,7 +275,7 @@ static const struct pv_muxing_param vc4_test_pv_muxing_params[] = { · 1 unmodified line
275	VC4_PV_MUXING_TEST("3 outputs: DPI, VEC, TXP",	275	VC4_PV_MUXING_TEST("3 outputs: DPI, VEC, TXP",
276	VC4_ENCODER_TYPE_DPI,	276	VC4_ENCODER_TYPE_DPI,
277	VC4_ENCODER_TYPE_VEC,	277	VC4_ENCODER_TYPE_VEC,
278	VC4_ENCODER_TYPE_TXP),	278	VC4_ENCODER_TYPE_TXP0),
279	};	279	};
280		280
281	KUNIT_ARRAY_PARAM(vc4_test_pv_muxing,	281	KUNIT_ARRAY_PARAM(vc4_test_pv_muxing,
@@ -287,7 +287,7 @@ static const struct pv_muxing_param vc4_test_pv_muxing_invalid_params[] = { · 5 unmodified lines
287	VC4_ENCODER_TYPE_DPI,	287	VC4_ENCODER_TYPE_DPI,
288	VC4_ENCODER_TYPE_DSI0),	288	VC4_ENCODER_TYPE_DSI0),
289	VC4_PV_MUXING_TEST("TXP/DSI1 Conflict",	289	VC4_PV_MUXING_TEST("TXP/DSI1 Conflict",
290	VC4_ENCODER_TYPE_TXP,	290	VC4_ENCODER_TYPE_TXP0,
291	VC4_ENCODER_TYPE_DSI1),	291	VC4_ENCODER_TYPE_DSI1),
292	VC4_PV_MUXING_TEST("HDMI0/VEC Conflict",	292	VC4_PV_MUXING_TEST("HDMI0/VEC Conflict",
293	VC4_ENCODER_TYPE_HDMI0,	293	VC4_ENCODER_TYPE_HDMI0,
@@ -296,22 +296,22 @@ static const struct pv_muxing_param vc4_test_pv_muxing_invalid_params[] = { · 2 unmodified lines
296	VC4_ENCODER_TYPE_DSI0,	296	VC4_ENCODER_TYPE_DSI0,
297	VC4_ENCODER_TYPE_HDMI0,	297	VC4_ENCODER_TYPE_HDMI0,
298	VC4_ENCODER_TYPE_DSI1,	298	VC4_ENCODER_TYPE_DSI1,
299	VC4_ENCODER_TYPE_TXP),	299	VC4_ENCODER_TYPE_TXP0),
300	VC4_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, DSI1, TXP",	300	VC4_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, DSI1, TXP",
301	VC4_ENCODER_TYPE_DSI0,	301	VC4_ENCODER_TYPE_DSI0,
302	VC4_ENCODER_TYPE_VEC,	302	VC4_ENCODER_TYPE_VEC,
303	VC4_ENCODER_TYPE_DSI1,	303	VC4_ENCODER_TYPE_DSI1,
304	VC4_ENCODER_TYPE_TXP),	304	VC4_ENCODER_TYPE_TXP0),
305	VC4_PV_MUXING_TEST("More than 3 outputs: DPI, HDMI0, DSI1, TXP",	305	VC4_PV_MUXING_TEST("More than 3 outputs: DPI, HDMI0, DSI1, TXP",
306	VC4_ENCODER_TYPE_DPI,	306	VC4_ENCODER_TYPE_DPI,
307	VC4_ENCODER_TYPE_HDMI0,	307	VC4_ENCODER_TYPE_HDMI0,
308	VC4_ENCODER_TYPE_DSI1,	308	VC4_ENCODER_TYPE_DSI1,
309	VC4_ENCODER_TYPE_TXP),	309	VC4_ENCODER_TYPE_TXP0),
310	VC4_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, DSI1, TXP",	310	VC4_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, DSI1, TXP",
311	VC4_ENCODER_TYPE_DPI,	311	VC4_ENCODER_TYPE_DPI,
312	VC4_ENCODER_TYPE_VEC,	312	VC4_ENCODER_TYPE_VEC,
313	VC4_ENCODER_TYPE_DSI1,	313	VC4_ENCODER_TYPE_DSI1,
314	VC4_ENCODER_TYPE_TXP),	314	VC4_ENCODER_TYPE_TXP0),
315	};	315	};
316		316
317	KUNIT_ARRAY_PARAM(vc4_test_pv_muxing_invalid,	317	KUNIT_ARRAY_PARAM(vc4_test_pv_muxing_invalid,
@@ -342,7 +342,7 @@ static const struct pv_muxing_param vc5_test_pv_muxing_params[] = { · 24 unmodified lines
342	VC4_ENCODER_TYPE_HDMI1),	342	VC4_ENCODER_TYPE_HDMI1),
343	VC5_PV_MUXING_TEST("2 outputs: DPI, TXP",	343	VC5_PV_MUXING_TEST("2 outputs: DPI, TXP",
344	VC4_ENCODER_TYPE_DPI,	344	VC4_ENCODER_TYPE_DPI,
345	VC4_ENCODER_TYPE_TXP),	345	VC4_ENCODER_TYPE_TXP0),
346	VC5_PV_MUXING_TEST("2 outputs: DPI, VEC",	346	VC5_PV_MUXING_TEST("2 outputs: DPI, VEC",
347	VC4_ENCODER_TYPE_DPI,	347	VC4_ENCODER_TYPE_DPI,
348	VC4_ENCODER_TYPE_VEC),	348	VC4_ENCODER_TYPE_VEC),
@@ -360,7 +360,7 @@ static const struct pv_muxing_param vc5_test_pv_muxing_params[] = { · 11 unmodified lines
360	VC4_ENCODER_TYPE_HDMI1),	360	VC4_ENCODER_TYPE_HDMI1),
361	VC5_PV_MUXING_TEST("2 outputs: DSI0, TXP",	361	VC5_PV_MUXING_TEST("2 outputs: DSI0, TXP",
362	VC4_ENCODER_TYPE_DSI0,	362	VC4_ENCODER_TYPE_DSI0,
363	VC4_ENCODER_TYPE_TXP),	363	VC4_ENCODER_TYPE_TXP0),
364	VC5_PV_MUXING_TEST("2 outputs: DSI0, VEC",	364	VC5_PV_MUXING_TEST("2 outputs: DSI0, VEC",
365	VC4_ENCODER_TYPE_DSI0,	365	VC4_ENCODER_TYPE_DSI0,
366	VC4_ENCODER_TYPE_VEC),	366	VC4_ENCODER_TYPE_VEC),
@@ -372,7 +372,7 @@ static const struct pv_muxing_param vc5_test_pv_muxing_params[] = { · 5 unmodified lines
372	VC4_ENCODER_TYPE_VEC),	372	VC4_ENCODER_TYPE_VEC),
373	VC5_PV_MUXING_TEST("2 outputs: DSI1, TXP",	373	VC5_PV_MUXING_TEST("2 outputs: DSI1, TXP",
374	VC4_ENCODER_TYPE_DSI1,	374	VC4_ENCODER_TYPE_DSI1,
375	VC4_ENCODER_TYPE_TXP),	375	VC4_ENCODER_TYPE_TXP0),
376	VC5_PV_MUXING_TEST("2 outputs: DSI1, HDMI0",	376	VC5_PV_MUXING_TEST("2 outputs: DSI1, HDMI0",
377	VC4_ENCODER_TYPE_DSI1,	377	VC4_ENCODER_TYPE_DSI1,
378	VC4_ENCODER_TYPE_HDMI0),	378	VC4_ENCODER_TYPE_HDMI0),
@@ -384,7 +384,7 @@ static const struct pv_muxing_param vc5_test_pv_muxing_params[] = { · 5 unmodified lines
384	VC4_ENCODER_TYPE_VEC),	384	VC4_ENCODER_TYPE_VEC),
385	VC5_PV_MUXING_TEST("2 outputs: HDMI0, TXP",	385	VC5_PV_MUXING_TEST("2 outputs: HDMI0, TXP",
386	VC4_ENCODER_TYPE_HDMI0,	386	VC4_ENCODER_TYPE_HDMI0,
387	VC4_ENCODER_TYPE_TXP),	387	VC4_ENCODER_TYPE_TXP0),
388	VC5_PV_MUXING_TEST("2 outputs: HDMI0, HDMI1",	388	VC5_PV_MUXING_TEST("2 outputs: HDMI0, HDMI1",
389	VC4_ENCODER_TYPE_HDMI0,	389	VC4_ENCODER_TYPE_HDMI0,
390	VC4_ENCODER_TYPE_HDMI1),	390	VC4_ENCODER_TYPE_HDMI1),
@@ -393,14 +393,14 @@ static const struct pv_muxing_param vc5_test_pv_muxing_params[] = { · 2 unmodified lines
393	VC4_ENCODER_TYPE_VEC),	393	VC4_ENCODER_TYPE_VEC),
394	VC5_PV_MUXING_TEST("2 outputs: HDMI1, TXP",	394	VC5_PV_MUXING_TEST("2 outputs: HDMI1, TXP",
395	VC4_ENCODER_TYPE_HDMI1,	395	VC4_ENCODER_TYPE_HDMI1,
396	VC4_ENCODER_TYPE_TXP),	396	VC4_ENCODER_TYPE_TXP0),
397	VC5_PV_MUXING_TEST("2 outputs: TXP, VEC",	397	VC5_PV_MUXING_TEST("2 outputs: TXP, VEC",
398	VC4_ENCODER_TYPE_TXP,	398	VC4_ENCODER_TYPE_TXP0,
399	VC4_ENCODER_TYPE_VEC),	399	VC4_ENCODER_TYPE_VEC),
400	VC5_PV_MUXING_TEST("3 outputs: DPI, VEC, TXP",	400	VC5_PV_MUXING_TEST("3 outputs: DPI, VEC, TXP",
401	VC4_ENCODER_TYPE_DPI,	401	VC4_ENCODER_TYPE_DPI,
402	VC4_ENCODER_TYPE_VEC,	402	VC4_ENCODER_TYPE_VEC,
403	VC4_ENCODER_TYPE_TXP),	403	VC4_ENCODER_TYPE_TXP0),
404	VC5_PV_MUXING_TEST("3 outputs: DPI, VEC, DSI1",	404	VC5_PV_MUXING_TEST("3 outputs: DPI, VEC, DSI1",
405	VC4_ENCODER_TYPE_DPI,	405	VC4_ENCODER_TYPE_DPI,
406	VC4_ENCODER_TYPE_VEC,	406	VC4_ENCODER_TYPE_VEC,
@@ -415,15 +415,15 @@ static const struct pv_muxing_param vc5_test_pv_muxing_params[] = { · 8 unmodified lines
415	VC4_ENCODER_TYPE_HDMI1),	415	VC4_ENCODER_TYPE_HDMI1),
416	VC5_PV_MUXING_TEST("3 outputs: DPI, TXP, DSI1",	416	VC5_PV_MUXING_TEST("3 outputs: DPI, TXP, DSI1",
417	VC4_ENCODER_TYPE_DPI,	417	VC4_ENCODER_TYPE_DPI,
418	VC4_ENCODER_TYPE_TXP,	418	VC4_ENCODER_TYPE_TXP0,
419	VC4_ENCODER_TYPE_DSI1),	419	VC4_ENCODER_TYPE_DSI1),
420	VC5_PV_MUXING_TEST("3 outputs: DPI, TXP, HDMI0",	420	VC5_PV_MUXING_TEST("3 outputs: DPI, TXP, HDMI0",
421	VC4_ENCODER_TYPE_DPI,	421	VC4_ENCODER_TYPE_DPI,
422	VC4_ENCODER_TYPE_TXP,	422	VC4_ENCODER_TYPE_TXP0,
423	VC4_ENCODER_TYPE_HDMI0),	423	VC4_ENCODER_TYPE_HDMI0),
424	VC5_PV_MUXING_TEST("3 outputs: DPI, TXP, HDMI1",	424	VC5_PV_MUXING_TEST("3 outputs: DPI, TXP, HDMI1",
425	VC4_ENCODER_TYPE_DPI,	425	VC4_ENCODER_TYPE_DPI,
426	VC4_ENCODER_TYPE_TXP,	426	VC4_ENCODER_TYPE_TXP0,
427	VC4_ENCODER_TYPE_HDMI1),	427	VC4_ENCODER_TYPE_HDMI1),
428	VC5_PV_MUXING_TEST("3 outputs: DPI, DSI1, HDMI0",	428	VC5_PV_MUXING_TEST("3 outputs: DPI, DSI1, HDMI0",
429	VC4_ENCODER_TYPE_DPI,	429	VC4_ENCODER_TYPE_DPI,
@@ -440,7 +440,7 @@ static const struct pv_muxing_param vc5_test_pv_muxing_params[] = { · 10 unmodified lines
440	VC5_PV_MUXING_TEST("3 outputs: DSI0, VEC, TXP",	440	VC5_PV_MUXING_TEST("3 outputs: DSI0, VEC, TXP",
441	VC4_ENCODER_TYPE_DSI0,	441	VC4_ENCODER_TYPE_DSI0,
442	VC4_ENCODER_TYPE_VEC,	442	VC4_ENCODER_TYPE_VEC,
443	VC4_ENCODER_TYPE_TXP),	443	VC4_ENCODER_TYPE_TXP0),
444	VC5_PV_MUXING_TEST("3 outputs: DSI0, VEC, DSI1",	444	VC5_PV_MUXING_TEST("3 outputs: DSI0, VEC, DSI1",
445	VC4_ENCODER_TYPE_DSI0,	445	VC4_ENCODER_TYPE_DSI0,
446	VC4_ENCODER_TYPE_VEC,	446	VC4_ENCODER_TYPE_VEC,
@@ -455,15 +455,15 @@ static const struct pv_muxing_param vc5_test_pv_muxing_params[] = { · 8 unmodified lines
455	VC4_ENCODER_TYPE_HDMI1),	455	VC4_ENCODER_TYPE_HDMI1),
456	VC5_PV_MUXING_TEST("3 outputs: DSI0, TXP, DSI1",	456	VC5_PV_MUXING_TEST("3 outputs: DSI0, TXP, DSI1",
457	VC4_ENCODER_TYPE_DSI0,	457	VC4_ENCODER_TYPE_DSI0,
458	VC4_ENCODER_TYPE_TXP,	458	VC4_ENCODER_TYPE_TXP0,
459	VC4_ENCODER_TYPE_DSI1),	459	VC4_ENCODER_TYPE_DSI1),
460	VC5_PV_MUXING_TEST("3 outputs: DSI0, TXP, HDMI0",	460	VC5_PV_MUXING_TEST("3 outputs: DSI0, TXP, HDMI0",
461	VC4_ENCODER_TYPE_DSI0,	461	VC4_ENCODER_TYPE_DSI0,
462	VC4_ENCODER_TYPE_TXP,	462	VC4_ENCODER_TYPE_TXP0,
463	VC4_ENCODER_TYPE_HDMI0),	463	VC4_ENCODER_TYPE_HDMI0),
464	VC5_PV_MUXING_TEST("3 outputs: DSI0, TXP, HDMI1",	464	VC5_PV_MUXING_TEST("3 outputs: DSI0, TXP, HDMI1",
465	VC4_ENCODER_TYPE_DSI0,	465	VC4_ENCODER_TYPE_DSI0,
466	VC4_ENCODER_TYPE_TXP,	466	VC4_ENCODER_TYPE_TXP0,
467	VC4_ENCODER_TYPE_HDMI1),	467	VC4_ENCODER_TYPE_HDMI1),
468	VC5_PV_MUXING_TEST("3 outputs: DSI0, DSI1, HDMI0",	468	VC5_PV_MUXING_TEST("3 outputs: DSI0, DSI1, HDMI0",
469	VC4_ENCODER_TYPE_DSI0,	469	VC4_ENCODER_TYPE_DSI0,
@@ -490,17 +490,17 @@ static const struct pv_muxing_param vc5_test_pv_muxing_invalid_params[] = { · 20 unmodified lines
490	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, DSI1",	490	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, DSI1",
491	VC4_ENCODER_TYPE_DPI,	491	VC4_ENCODER_TYPE_DPI,
492	VC4_ENCODER_TYPE_VEC,	492	VC4_ENCODER_TYPE_VEC,
493	VC4_ENCODER_TYPE_TXP,	493	VC4_ENCODER_TYPE_TXP0,
494	VC4_ENCODER_TYPE_DSI1),	494	VC4_ENCODER_TYPE_DSI1),
495	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, HDMI0",	495	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, HDMI0",
496	VC4_ENCODER_TYPE_DPI,	496	VC4_ENCODER_TYPE_DPI,
497	VC4_ENCODER_TYPE_VEC,	497	VC4_ENCODER_TYPE_VEC,
498	VC4_ENCODER_TYPE_TXP,	498	VC4_ENCODER_TYPE_TXP0,
499	VC4_ENCODER_TYPE_HDMI0),	499	VC4_ENCODER_TYPE_HDMI0),
500	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, HDMI1",	500	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, HDMI1",
501	VC4_ENCODER_TYPE_DPI,	501	VC4_ENCODER_TYPE_DPI,
502	VC4_ENCODER_TYPE_VEC,	502	VC4_ENCODER_TYPE_VEC,
503	VC4_ENCODER_TYPE_TXP,	503	VC4_ENCODER_TYPE_TXP0,
504	VC4_ENCODER_TYPE_HDMI1),	504	VC4_ENCODER_TYPE_HDMI1),
505	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, DSI1, HDMI0",	505	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, DSI1, HDMI0",
506	VC4_ENCODER_TYPE_DPI,	506	VC4_ENCODER_TYPE_DPI,
@@ -519,17 +519,17 @@ static const struct pv_muxing_param vc5_test_pv_muxing_invalid_params[] = { · 12 unmodified lines
519	VC4_ENCODER_TYPE_HDMI1),	519	VC4_ENCODER_TYPE_HDMI1),
520	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, TXP, DSI1, HDMI0",	520	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, TXP, DSI1, HDMI0",
521	VC4_ENCODER_TYPE_DPI,	521	VC4_ENCODER_TYPE_DPI,
522	VC4_ENCODER_TYPE_TXP,	522	VC4_ENCODER_TYPE_TXP0,
523	VC4_ENCODER_TYPE_DSI1,	523	VC4_ENCODER_TYPE_DSI1,
524	VC4_ENCODER_TYPE_HDMI0),	524	VC4_ENCODER_TYPE_HDMI0),
525	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, TXP, DSI1, HDMI1",	525	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, TXP, DSI1, HDMI1",
526	VC4_ENCODER_TYPE_DPI,	526	VC4_ENCODER_TYPE_DPI,
527	VC4_ENCODER_TYPE_TXP,	527	VC4_ENCODER_TYPE_TXP0,
528	VC4_ENCODER_TYPE_DSI1,	528	VC4_ENCODER_TYPE_DSI1,
529	VC4_ENCODER_TYPE_HDMI1),	529	VC4_ENCODER_TYPE_HDMI1),
530	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, TXP, HDMI0, HDMI1",	530	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, TXP, HDMI0, HDMI1",
531	VC4_ENCODER_TYPE_DPI,	531	VC4_ENCODER_TYPE_DPI,
532	VC4_ENCODER_TYPE_TXP,	532	VC4_ENCODER_TYPE_TXP0,
533	VC4_ENCODER_TYPE_HDMI0,	533	VC4_ENCODER_TYPE_HDMI0,
534	VC4_ENCODER_TYPE_HDMI1),	534	VC4_ENCODER_TYPE_HDMI1),
535	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, DSI1, HDMI0, HDMI1",	535	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, DSI1, HDMI0, HDMI1",
@@ -540,19 +540,19 @@ static const struct pv_muxing_param vc5_test_pv_muxing_invalid_params[] = { · 4 unmodified lines
540	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, DSI1, HDMI0",	540	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, DSI1, HDMI0",
541	VC4_ENCODER_TYPE_DPI,	541	VC4_ENCODER_TYPE_DPI,
542	VC4_ENCODER_TYPE_VEC,	542	VC4_ENCODER_TYPE_VEC,
543	VC4_ENCODER_TYPE_TXP,	543	VC4_ENCODER_TYPE_TXP0,
544	VC4_ENCODER_TYPE_DSI1,	544	VC4_ENCODER_TYPE_DSI1,
545	VC4_ENCODER_TYPE_HDMI0),	545	VC4_ENCODER_TYPE_HDMI0),
546	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, DSI1, HDMI1",	546	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, DSI1, HDMI1",
547	VC4_ENCODER_TYPE_DPI,	547	VC4_ENCODER_TYPE_DPI,
548	VC4_ENCODER_TYPE_VEC,	548	VC4_ENCODER_TYPE_VEC,
549	VC4_ENCODER_TYPE_TXP,	549	VC4_ENCODER_TYPE_TXP0,
550	VC4_ENCODER_TYPE_DSI1,	550	VC4_ENCODER_TYPE_DSI1,
551	VC4_ENCODER_TYPE_HDMI1),	551	VC4_ENCODER_TYPE_HDMI1),
552	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, HDMI0, HDMI1",	552	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, HDMI0, HDMI1",
553	VC4_ENCODER_TYPE_DPI,	553	VC4_ENCODER_TYPE_DPI,
554	VC4_ENCODER_TYPE_VEC,	554	VC4_ENCODER_TYPE_VEC,
555	VC4_ENCODER_TYPE_TXP,	555	VC4_ENCODER_TYPE_TXP0,
556	VC4_ENCODER_TYPE_HDMI0,	556	VC4_ENCODER_TYPE_HDMI0,
557	VC4_ENCODER_TYPE_HDMI1),	557	VC4_ENCODER_TYPE_HDMI1),
558	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, DSI1, HDMI0, HDMI1",	558	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, DSI1, HDMI0, HDMI1",
@@ -563,24 +563,24 @@ static const struct pv_muxing_param vc5_test_pv_muxing_invalid_params[] = { · 4 unmodified lines
563	VC4_ENCODER_TYPE_HDMI1),	563	VC4_ENCODER_TYPE_HDMI1),
564	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, TXP, DSI1, HDMI0, HDMI1",	564	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, TXP, DSI1, HDMI0, HDMI1",
565	VC4_ENCODER_TYPE_DPI,	565	VC4_ENCODER_TYPE_DPI,
566	VC4_ENCODER_TYPE_TXP,	566	VC4_ENCODER_TYPE_TXP0,
567	VC4_ENCODER_TYPE_DSI1,	567	VC4_ENCODER_TYPE_DSI1,
568	VC4_ENCODER_TYPE_HDMI0,	568	VC4_ENCODER_TYPE_HDMI0,
569	VC4_ENCODER_TYPE_HDMI1),	569	VC4_ENCODER_TYPE_HDMI1),
570	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, DSI1",	570	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, DSI1",
571	VC4_ENCODER_TYPE_DSI0,	571	VC4_ENCODER_TYPE_DSI0,
572	VC4_ENCODER_TYPE_VEC,	572	VC4_ENCODER_TYPE_VEC,
573	VC4_ENCODER_TYPE_TXP,	573	VC4_ENCODER_TYPE_TXP0,
574	VC4_ENCODER_TYPE_DSI1),	574	VC4_ENCODER_TYPE_DSI1),
575	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, HDMI0",	575	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, HDMI0",
576	VC4_ENCODER_TYPE_DSI0,	576	VC4_ENCODER_TYPE_DSI0,
577	VC4_ENCODER_TYPE_VEC,	577	VC4_ENCODER_TYPE_VEC,
578	VC4_ENCODER_TYPE_TXP,	578	VC4_ENCODER_TYPE_TXP0,
579	VC4_ENCODER_TYPE_HDMI0),	579	VC4_ENCODER_TYPE_HDMI0),
580	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, HDMI1",	580	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, HDMI1",
581	VC4_ENCODER_TYPE_DSI0,	581	VC4_ENCODER_TYPE_DSI0,
582	VC4_ENCODER_TYPE_VEC,	582	VC4_ENCODER_TYPE_VEC,
583	VC4_ENCODER_TYPE_TXP,	583	VC4_ENCODER_TYPE_TXP0,
584	VC4_ENCODER_TYPE_HDMI1),	584	VC4_ENCODER_TYPE_HDMI1),
585	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, DSI1, HDMI0",	585	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, DSI1, HDMI0",
586	VC4_ENCODER_TYPE_DSI0,	586	VC4_ENCODER_TYPE_DSI0,
@@ -599,17 +599,17 @@ static const struct pv_muxing_param vc5_test_pv_muxing_invalid_params[] = { · 12 unmodified lines
599	VC4_ENCODER_TYPE_HDMI1),	599	VC4_ENCODER_TYPE_HDMI1),
600	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, TXP, DSI1, HDMI0",	600	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, TXP, DSI1, HDMI0",
601	VC4_ENCODER_TYPE_DSI0,	601	VC4_ENCODER_TYPE_DSI0,
602	VC4_ENCODER_TYPE_TXP,	602	VC4_ENCODER_TYPE_TXP0,
603	VC4_ENCODER_TYPE_DSI1,	603	VC4_ENCODER_TYPE_DSI1,
604	VC4_ENCODER_TYPE_HDMI0),	604	VC4_ENCODER_TYPE_HDMI0),
605	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, TXP, DSI1, HDMI1",	605	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, TXP, DSI1, HDMI1",
606	VC4_ENCODER_TYPE_DSI0,	606	VC4_ENCODER_TYPE_DSI0,
607	VC4_ENCODER_TYPE_TXP,	607	VC4_ENCODER_TYPE_TXP0,
608	VC4_ENCODER_TYPE_DSI1,	608	VC4_ENCODER_TYPE_DSI1,
609	VC4_ENCODER_TYPE_HDMI1),	609	VC4_ENCODER_TYPE_HDMI1),
610	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, TXP, HDMI0, HDMI1",	610	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, TXP, HDMI0, HDMI1",
611	VC4_ENCODER_TYPE_DSI0,	611	VC4_ENCODER_TYPE_DSI0,
612	VC4_ENCODER_TYPE_TXP,	612	VC4_ENCODER_TYPE_TXP0,
613	VC4_ENCODER_TYPE_HDMI0,	613	VC4_ENCODER_TYPE_HDMI0,
614	VC4_ENCODER_TYPE_HDMI1),	614	VC4_ENCODER_TYPE_HDMI1),
615	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, DSI1, HDMI0, HDMI1",	615	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, DSI1, HDMI0, HDMI1",
@@ -620,19 +620,19 @@ static const struct pv_muxing_param vc5_test_pv_muxing_invalid_params[] = { · 4 unmodified lines
620	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, DSI1, HDMI0",	620	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, DSI1, HDMI0",
621	VC4_ENCODER_TYPE_DSI0,	621	VC4_ENCODER_TYPE_DSI0,
622	VC4_ENCODER_TYPE_VEC,	622	VC4_ENCODER_TYPE_VEC,
623	VC4_ENCODER_TYPE_TXP,	623	VC4_ENCODER_TYPE_TXP0,
624	VC4_ENCODER_TYPE_DSI1,	624	VC4_ENCODER_TYPE_DSI1,
625	VC4_ENCODER_TYPE_HDMI0),	625	VC4_ENCODER_TYPE_HDMI0),
626	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, DSI1, HDMI1",	626	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, DSI1, HDMI1",
627	VC4_ENCODER_TYPE_DSI0,	627	VC4_ENCODER_TYPE_DSI0,
628	VC4_ENCODER_TYPE_VEC,	628	VC4_ENCODER_TYPE_VEC,
629	VC4_ENCODER_TYPE_TXP,	629	VC4_ENCODER_TYPE_TXP0,
630	VC4_ENCODER_TYPE_DSI1,	630	VC4_ENCODER_TYPE_DSI1,
631	VC4_ENCODER_TYPE_HDMI1),	631	VC4_ENCODER_TYPE_HDMI1),
632	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, HDMI0, HDMI1",	632	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, HDMI0, HDMI1",
633	VC4_ENCODER_TYPE_DSI0,	633	VC4_ENCODER_TYPE_DSI0,
634	VC4_ENCODER_TYPE_VEC,	634	VC4_ENCODER_TYPE_VEC,
635	VC4_ENCODER_TYPE_TXP,	635	VC4_ENCODER_TYPE_TXP0,
636	VC4_ENCODER_TYPE_HDMI0,	636	VC4_ENCODER_TYPE_HDMI0,
637	VC4_ENCODER_TYPE_HDMI1),	637	VC4_ENCODER_TYPE_HDMI1),
638	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, DSI1, HDMI0, HDMI1",	638	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, DSI1, HDMI0, HDMI1",
@@ -643,27 +643,27 @@ static const struct pv_muxing_param vc5_test_pv_muxing_invalid_params[] = { · 4 unmodified lines
643	VC4_ENCODER_TYPE_HDMI1),	643	VC4_ENCODER_TYPE_HDMI1),
644	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, TXP, DSI1, HDMI0, HDMI1",	644	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, TXP, DSI1, HDMI0, HDMI1",
645	VC4_ENCODER_TYPE_DSI0,	645	VC4_ENCODER_TYPE_DSI0,
646	VC4_ENCODER_TYPE_TXP,	646	VC4_ENCODER_TYPE_TXP0,
647	VC4_ENCODER_TYPE_DSI1,	647	VC4_ENCODER_TYPE_DSI1,
648	VC4_ENCODER_TYPE_HDMI0,	648	VC4_ENCODER_TYPE_HDMI0,
649	VC4_ENCODER_TYPE_HDMI1),	649	VC4_ENCODER_TYPE_HDMI1),
650	VC5_PV_MUXING_TEST("More than 3 outputs: VEC, TXP, DSI1, HDMI0, HDMI1",	650	VC5_PV_MUXING_TEST("More than 3 outputs: VEC, TXP, DSI1, HDMI0, HDMI1",
651	VC4_ENCODER_TYPE_VEC,	651	VC4_ENCODER_TYPE_VEC,
652	VC4_ENCODER_TYPE_TXP,	652	VC4_ENCODER_TYPE_TXP0,
653	VC4_ENCODER_TYPE_DSI1,	653	VC4_ENCODER_TYPE_DSI1,
654	VC4_ENCODER_TYPE_HDMI0,	654	VC4_ENCODER_TYPE_HDMI0,
655	VC4_ENCODER_TYPE_HDMI1),	655	VC4_ENCODER_TYPE_HDMI1),
656	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, DSI1, HDMI0, HDMI1",	656	VC5_PV_MUXING_TEST("More than 3 outputs: DPI, VEC, TXP, DSI1, HDMI0, HDMI1",
657	VC4_ENCODER_TYPE_DPI,	657	VC4_ENCODER_TYPE_DPI,
658	VC4_ENCODER_TYPE_VEC,	658	VC4_ENCODER_TYPE_VEC,
659	VC4_ENCODER_TYPE_TXP,	659	VC4_ENCODER_TYPE_TXP0,
660	VC4_ENCODER_TYPE_DSI1,	660	VC4_ENCODER_TYPE_DSI1,
661	VC4_ENCODER_TYPE_HDMI0,	661	VC4_ENCODER_TYPE_HDMI0,
662	VC4_ENCODER_TYPE_HDMI1),	662	VC4_ENCODER_TYPE_HDMI1),
663	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, DSI1, HDMI0, HDMI1",	663	VC5_PV_MUXING_TEST("More than 3 outputs: DSI0, VEC, TXP, DSI1, HDMI0, HDMI1",
664	VC4_ENCODER_TYPE_DSI0,	664	VC4_ENCODER_TYPE_DSI0,
665	VC4_ENCODER_TYPE_VEC,	665	VC4_ENCODER_TYPE_VEC,
666	VC4_ENCODER_TYPE_TXP,	666	VC4_ENCODER_TYPE_TXP0,
667	VC4_ENCODER_TYPE_DSI1,	667	VC4_ENCODER_TYPE_DSI1,
668	VC4_ENCODER_TYPE_HDMI0,	668	VC4_ENCODER_TYPE_HDMI0,
669	VC4_ENCODER_TYPE_HDMI1),	669	VC4_ENCODER_TYPE_HDMI1),

drivers/gpu/drm/vc4/vc4_crtc.c+84−12

@@ -83,13 +83,22 @@ static unsigned int · 82 unmodified lines
83	vc4_crtc_get_cob_allocation(struct vc4_dev *vc4, unsigned int channel)	83	vc4_crtc_get_cob_allocation(struct vc4_dev *vc4, unsigned int channel)
84	{	84	{
85	struct vc4_hvs *hvs = vc4->hvs;	85	struct vc4_hvs *hvs = vc4->hvs;
86	u32 dispbase = HVS_READ(SCALER_DISPBASEX(channel));	86	u32 dispbase, top, base;
		87
87	/* Top/base are supposed to be 4-pixel aligned, but the	88	/* Top/base are supposed to be 4-pixel aligned, but the
88	* Raspberry Pi firmware fills the low bits (which are	89	* Raspberry Pi firmware fills the low bits (which are
89	* presumably ignored).	90	* presumably ignored).
90	*/	91	*/
91	u32 top = VC4_GET_FIELD(dispbase, SCALER_DISPBASEX_TOP) & ~3;	92
92	u32 base = VC4_GET_FIELD(dispbase, SCALER_DISPBASEX_BASE) & ~3;	93	if (vc4->gen >= VC4_GEN_6_C) {
		94	dispbase = HVS_READ(SCALER6_DISPX_COB(channel));
		95	top = VC4_GET_FIELD(dispbase, SCALER6_DISPX_COB_TOP) & ~3;
		96	base = VC4_GET_FIELD(dispbase, SCALER6_DISPX_COB_BASE) & ~3;
		97	} else {
		98	dispbase = HVS_READ(SCALER_DISPBASEX(channel));
		99	top = VC4_GET_FIELD(dispbase, SCALER_DISPBASEX_TOP) & ~3;
		100	base = VC4_GET_FIELD(dispbase, SCALER_DISPBASEX_BASE) & ~3;
		101	}
93		102
94	return top - base + 4;	103	return top - base + 4;
95	}	104	}
@@ -122,7 +131,10 @@ static bool vc4_crtc_get_scanout_position(struct drm_crtc *crtc, · 26 unmodified lines
122	* Read vertical scanline which is currently composed for our	131	* Read vertical scanline which is currently composed for our
123	* pixelvalve by the HVS, and also the scaler status.	132	* pixelvalve by the HVS, and also the scaler status.
124	*/	133	*/
125	val = HVS_READ(SCALER_DISPSTATX(channel));	134	if (vc4->gen >= VC4_GEN_6_C)
		135	val = HVS_READ(SCALER6_DISPX_STATUS(channel));
		136	else
		137	val = HVS_READ(SCALER_DISPSTATX(channel));
126		138
127	/* Get optional system timestamp after query. */	139	/* Get optional system timestamp after query. */
128	if (etime)	140	if (etime)
@@ -131,7 +143,12 @@ static bool vc4_crtc_get_scanout_position(struct drm_crtc *crtc, · 2 unmodified lines
131	/* preempt_enable_rt() should go right here in PREEMPT_RT patchset. */	143	/* preempt_enable_rt() should go right here in PREEMPT_RT patchset. */
132		144
133	/* Vertical position of hvs composed scanline. */	145	/* Vertical position of hvs composed scanline. */
134	*vpos = VC4_GET_FIELD(val, SCALER_DISPSTATX_LINE);	146
		147	if (vc4->gen >= VC4_GEN_6_C)
		148	*vpos = VC4_GET_FIELD(val, SCALER6_DISPX_STATUS_YLINE);
		149	else
		150	*vpos = VC4_GET_FIELD(val, SCALER_DISPSTATX_LINE);
		151
135	*hpos = 0;	152	*hpos = 0;
136		153
137	if (mode->flags & DRM_MODE_FLAG_INTERLACE) {	154	if (mode->flags & DRM_MODE_FLAG_INTERLACE) {
@@ -223,6 +240,11 @@ static u32 vc4_get_fifo_full_level(struct vc4_crtc *vc4_crtc, u32 format) · 85 unmodified lines
223	const struct vc4_crtc_data *crtc_data = vc4_crtc_to_vc4_crtc_data(vc4_crtc);	240	const struct vc4_crtc_data *crtc_data = vc4_crtc_to_vc4_crtc_data(vc4_crtc);
224	const struct vc4_pv_data *pv_data = vc4_crtc_to_vc4_pv_data(vc4_crtc);	241	const struct vc4_pv_data *pv_data = vc4_crtc_to_vc4_pv_data(vc4_crtc);
225	struct vc4_dev *vc4 = to_vc4_dev(vc4_crtc->base.dev);	242	struct vc4_dev *vc4 = to_vc4_dev(vc4_crtc->base.dev);
		243
		244	/*
		245	* NOTE: Could we use register 0x68 (PV_HW_CFG1) to get the FIFO
		246	* size?
		247	*/
226	u32 fifo_len_bytes = pv_data->fifo_depth;	248	u32 fifo_len_bytes = pv_data->fifo_depth;
227		249
228	/*	250	/*
@@ -404,6 +426,7 @@ static void vc4_crtc_config_pv(struct drm_crtc crtc, struct drm_encoder encode · 175 unmodified lines
404	*/	426	*/
405	CRTC_WRITE(PV_V_CONTROL,	427	CRTC_WRITE(PV_V_CONTROL,
406	PV_VCONTROL_CONTINUOUS \|	428	PV_VCONTROL_CONTINUOUS \|
		429	(vc4->gen >= VC4_GEN_6_C ? PV_VCONTROL_ODD_TIMING : 0) \|
407	(is_dsi ? PV_VCONTROL_DSI : 0) \|	430	(is_dsi ? PV_VCONTROL_DSI : 0) \|
408	PV_VCONTROL_INTERLACE \|	431	PV_VCONTROL_INTERLACE \|
409	(odd_field_first	432	(odd_field_first
@@ -415,6 +438,7 @@ static void vc4_crtc_config_pv(struct drm_crtc crtc, struct drm_encoder encode · 5 unmodified lines
415	} else {	438	} else {
416	CRTC_WRITE(PV_V_CONTROL,	439	CRTC_WRITE(PV_V_CONTROL,
417	PV_VCONTROL_CONTINUOUS \|	440	PV_VCONTROL_CONTINUOUS \|
		441	(vc4->gen >= VC4_GEN_6_C ? PV_VCONTROL_ODD_TIMING : 0) \|
418	(is_dsi ? PV_VCONTROL_DSI : 0));	442	(is_dsi ? PV_VCONTROL_DSI : 0));
419	CRTC_WRITE(PV_VSYNCD_EVEN, 0);	443	CRTC_WRITE(PV_VSYNCD_EVEN, 0);
420	}	444	}
@@ -429,11 +453,17 @@ static void vc4_crtc_config_pv(struct drm_crtc crtc, struct drm_encoder encode · 8 unmodified lines
429	if (is_dsi)	453	if (is_dsi)
430	CRTC_WRITE(PV_HACT_ACT, mode->hdisplay * pixel_rep);	454	CRTC_WRITE(PV_HACT_ACT, mode->hdisplay * pixel_rep);
431		455
432	if (vc4->gen == VC4_GEN_5)	456	if (vc4->gen >= VC4_GEN_5)
433	CRTC_WRITE(PV_MUX_CFG,	457	CRTC_WRITE(PV_MUX_CFG,
434	VC4_SET_FIELD(PV_MUX_CFG_RGB_PIXEL_MUX_MODE_NO_SWAP,	458	VC4_SET_FIELD(PV_MUX_CFG_RGB_PIXEL_MUX_MODE_NO_SWAP,
435	PV_MUX_CFG_RGB_PIXEL_MUX_MODE));	459	PV_MUX_CFG_RGB_PIXEL_MUX_MODE));
436		460
		461	if (vc4->gen >= VC4_GEN_6_C)
		462	CRTC_WRITE(PV_PIPE_INIT_CTRL,
		463	VC4_SET_FIELD(1, PV_PIPE_INIT_CTRL_PV_INIT_WIDTH) \|
		464	VC4_SET_FIELD(1, PV_PIPE_INIT_CTRL_PV_INIT_IDLE) \|
		465	PV_PIPE_INIT_CTRL_PV_INIT_EN);
		466
437	CRTC_WRITE(PV_CONTROL, PV_CONTROL_FIFO_CLR \|	467	CRTC_WRITE(PV_CONTROL, PV_CONTROL_FIFO_CLR \|
438	vc4_crtc_get_fifo_full_level_bits(vc4_crtc, format) \|	468	vc4_crtc_get_fifo_full_level_bits(vc4_crtc, format) \|
439	VC4_SET_FIELD(format, PV_CONTROL_FORMAT) \|	469	VC4_SET_FIELD(format, PV_CONTROL_FORMAT) \|
@@ -459,8 +489,10 @@ static void require_hvs_enabled(struct drm_device *dev) · 19 unmodified lines
459	struct vc4_dev *vc4 = to_vc4_dev(dev);	489	struct vc4_dev *vc4 = to_vc4_dev(dev);
460	struct vc4_hvs *hvs = vc4->hvs;	490	struct vc4_hvs *hvs = vc4->hvs;
461		491
462	WARN_ON_ONCE((HVS_READ(SCALER_DISPCTRL) & SCALER_DISPCTRL_ENABLE) !=	492	if (vc4->gen >= VC4_GEN_6_C)
463	SCALER_DISPCTRL_ENABLE);	493	WARN_ON_ONCE(!(HVS_READ(SCALER6_CONTROL) & SCALER6_CONTROL_HVS_EN));
		494	else
		495	WARN_ON_ONCE(!(HVS_READ(SCALER_DISPCTRL) & SCALER_DISPCTRL_ENABLE));
464	}	496	}
465		497
466	static int vc4_crtc_disable(struct drm_crtc *crtc,	498	static int vc4_crtc_disable(struct drm_crtc *crtc,
@@ -530,7 +562,11 @@ int vc4_crtc_disable_at_boot(struct drm_crtc *crtc) · 63 unmodified lines
530	if (!(of_device_is_compatible(vc4_crtc->pdev->dev.of_node,	562	if (!(of_device_is_compatible(vc4_crtc->pdev->dev.of_node,
531	"brcm,bcm2711-pixelvalve2") \|\|	563	"brcm,bcm2711-pixelvalve2") \|\|
532	of_device_is_compatible(vc4_crtc->pdev->dev.of_node,	564	of_device_is_compatible(vc4_crtc->pdev->dev.of_node,
533	"brcm,bcm2711-pixelvalve4")))	565	"brcm,bcm2711-pixelvalve4") \|\|
		566	of_device_is_compatible(vc4_crtc->pdev->dev.of_node,
		567	"brcm,bcm2712-pixelvalve0") \|\|
		568	of_device_is_compatible(vc4_crtc->pdev->dev.of_node,
		569	"brcm,bcm2712-pixelvalve1")))
534	return 0;	570	return 0;
535		571
536	if (!(CRTC_READ(PV_CONTROL) & PV_CONTROL_EN))	572	if (!(CRTC_READ(PV_CONTROL) & PV_CONTROL_EN))
@@ -789,14 +825,21 @@ static void vc4_crtc_handle_page_flip(struct vc4_crtc *vc4_crtc) · 252 unmodified lines
789	struct drm_device *dev = crtc->dev;	825	struct drm_device *dev = crtc->dev;
790	struct vc4_dev *vc4 = to_vc4_dev(dev);	826	struct vc4_dev *vc4 = to_vc4_dev(dev);
791	struct vc4_hvs *hvs = vc4->hvs;	827	struct vc4_hvs *hvs = vc4->hvs;
		828	unsigned int current_dlist;
792	u32 chan = vc4_crtc->current_hvs_channel;	829	u32 chan = vc4_crtc->current_hvs_channel;
793	unsigned long flags;	830	unsigned long flags;
794		831
795	spin_lock_irqsave(&dev->event_lock, flags);	832	spin_lock_irqsave(&dev->event_lock, flags);
796	spin_lock(&vc4_crtc->irq_lock);	833	spin_lock(&vc4_crtc->irq_lock);
		834
		835	if (vc4->gen >= VC4_GEN_6_C)
		836	current_dlist = VC4_GET_FIELD(HVS_READ(SCALER6_DISPX_DL(chan)),
		837	SCALER6_DISPX_DL_LACT);
		838	else
		839	current_dlist = HVS_READ(SCALER_DISPLACTX(chan));
		840
797	if (vc4_crtc->event &&	841	if (vc4_crtc->event &&
798	(vc4_crtc->current_dlist == HVS_READ(SCALER_DISPLACTX(chan)) \|\|	842	(vc4_crtc->current_dlist == current_dlist \|\| vc4_crtc->feeds_txp)) {
799	vc4_crtc->feeds_txp)) {
800	drm_crtc_send_vblank_event(crtc, vc4_crtc->event);	843	drm_crtc_send_vblank_event(crtc, vc4_crtc->event);
801	vc4_crtc->event = NULL;	844	vc4_crtc->event = NULL;
802	drm_crtc_vblank_put(crtc);	845	drm_crtc_vblank_put(crtc);
@@ -807,7 +850,8 @@ static void vc4_crtc_handle_page_flip(struct vc4_crtc *vc4_crtc) · 4 unmodified lines
807	* the CRTC and encoder already reconfigured, leading to	850	* the CRTC and encoder already reconfigured, leading to
808	* underruns. This can be seen when reconfiguring the CRTC.	851	* underruns. This can be seen when reconfiguring the CRTC.
809	*/	852	*/
810	vc4_hvs_unmask_underrun(hvs, chan);	853	if (vc4->gen < VC4_GEN_6_C)
		854	vc4_hvs_unmask_underrun(hvs, chan);
811	}	855	}
812	spin_unlock(&vc4_crtc->irq_lock);	856	spin_unlock(&vc4_crtc->irq_lock);
813	spin_unlock_irqrestore(&dev->event_lock, flags);	857	spin_unlock_irqrestore(&dev->event_lock, flags);
@@ -1265,6 +1309,32 @@ const struct vc4_pv_data bcm2711_pv4_data = { · 451 unmodified lines
1265	},	1309	},
1266	};	1310	};
1267		1311
		1312	const struct vc4_pv_data bcm2712_pv0_data = {
		1313	.base = {
		1314	.debugfs_name = "crtc0_regs",
		1315	.hvs_available_channels = BIT(0),
		1316	.hvs_output = 0,
		1317	},
		1318	.fifo_depth = 64,
		1319	.pixels_per_clock = 1,
		1320	.encoder_types = {
		1321	[0] = VC4_ENCODER_TYPE_HDMI0,
		1322	},
		1323	};
		1324
		1325	const struct vc4_pv_data bcm2712_pv1_data = {
		1326	.base = {
		1327	.debugfs_name = "crtc1_regs",
		1328	.hvs_available_channels = BIT(1),
		1329	.hvs_output = 1,
		1330	},
		1331	.fifo_depth = 64,
		1332	.pixels_per_clock = 1,
		1333	.encoder_types = {
		1334	[0] = VC4_ENCODER_TYPE_HDMI1,
		1335	},
		1336	};
		1337
1268	static const struct of_device_id vc4_crtc_dt_match[] = {	1338	static const struct of_device_id vc4_crtc_dt_match[] = {
1269	{ .compatible = "brcm,bcm2835-pixelvalve0", .data = &bcm2835_pv0_data },	1339	{ .compatible = "brcm,bcm2835-pixelvalve0", .data = &bcm2835_pv0_data },
1270	{ .compatible = "brcm,bcm2835-pixelvalve1", .data = &bcm2835_pv1_data },	1340	{ .compatible = "brcm,bcm2835-pixelvalve1", .data = &bcm2835_pv1_data },
@@ -1274,6 +1344,8 @@ static const struct of_device_id vc4_crtc_dt_match[] = { · 3 unmodified lines
1274	{ .compatible = "brcm,bcm2711-pixelvalve2", .data = &bcm2711_pv2_data },	1344	{ .compatible = "brcm,bcm2711-pixelvalve2", .data = &bcm2711_pv2_data },
1275	{ .compatible = "brcm,bcm2711-pixelvalve3", .data = &bcm2711_pv3_data },	1345	{ .compatible = "brcm,bcm2711-pixelvalve3", .data = &bcm2711_pv3_data },
1276	{ .compatible = "brcm,bcm2711-pixelvalve4", .data = &bcm2711_pv4_data },	1346	{ .compatible = "brcm,bcm2711-pixelvalve4", .data = &bcm2711_pv4_data },
		1347	{ .compatible = "brcm,bcm2712-pixelvalve0", .data = &bcm2712_pv0_data },
		1348	{ .compatible = "brcm,bcm2712-pixelvalve1", .data = &bcm2712_pv1_data },
1277	{}	1349	{}
1278	};	1350	};
1279		1351

drivers/gpu/drm/vc4/vc4_drv.c+13−11

 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_dma.h>
 #include <drm/drm_fourcc.h>
 #define DRIVER_NAME "vc4"
 #define DRIVER_DESC "Broadcom VC4 graphics"
-#define DRIVER_DATE "20140616"
 #define DRIVER_MAJOR 0
 #define DRIVER_MINOR 0
 #define DRIVER_PATCHLEVEL 0
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,
 static const struct of_device_id vc4_dma_range_matches[] = {
 	{ .compatible = "brcm,bcm2711-hvs" },
+	{ .compatible = "brcm,bcm2712-hvs" },
 	{ .compatible = "brcm,bcm2835-hvs" },
 	{ .compatible = "brcm,bcm2835-v3d" },
 	{ .compatible = "brcm,cygnus-v3d" },
 	dev->coherent_dma_mask = DMA_BIT_MASK(32);
-	if (of_device_is_compatible(dev->of_node, "brcm,bcm2711-vc5"))
+	gen = (enum vc4_gen)of_device_get_match_data(dev);
-		gen = VC4_GEN_5;
-	else
-		gen = VC4_GEN_4;
 	if (gen > VC4_GEN_4)
 		driver = &vc5_drm_driver;
 	else
 		driver = &vc4_drm_driver;
+	if (gen >= VC4_GEN_6_C)
+		dma_set_mask_and_coherent(dev, DMA_BIT_MASK(36));
+	else
+		dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
 	node = of_find_matching_node_and_match(NULL, vc4_dma_range_matches,
 					       NULL);
 	if (node) {
 }
 static const struct of_device_id vc4_of_match[] = {
-	{ .compatible = "brcm,bcm2711-vc5", },
+	{ .compatible = "brcm,bcm2711-vc5", .data = (void *)VC4_GEN_5 },
-	{ .compatible = "brcm,bcm2835-vc4", },
+	/* NB GEN_6_C will be corrected on D0 hw to GEN_6_D via vc4_hvs_bind */
-	{ .compatible = "brcm,cygnus-vc4", },
+	{ .compatible = "brcm,bcm2712-vc6", .data = (void *)VC4_GEN_6_C },
+	{ .compatible = "brcm,bcm2835-vc4", .data = (void *)VC4_GEN_4 },
+	{ .compatible = "brcm,cygnus-vc4", .data = (void *)VC4_GEN_4 },
 	{},
 };
 MODULE_DEVICE_TABLE(of, vc4_of_match);

drivers/gpu/drm/vc4/vc4_drv.h+52−2

 enum vc4_gen {
 	VC4_GEN_4,
 	VC4_GEN_5,
+	VC4_GEN_6_C,
+	VC4_GEN_6_D,
 };
 struct vc4_dev {
 	struct debugfs_regset32 regset;
 };
+#define VC4_NUM_UPM_HANDLES 32
+struct vc4_upm_refcounts {
+	refcount_t refcount;
+	/* Allocation size */
+	size_t size;
+	/* Our allocation in UPM for prefetching. */
+	struct drm_mm_node upm;
+	/* Pointer back to the HVS structure */
+	struct vc4_hvs *hvs;
+};
+#define HVS_NUM_CHANNELS 3
 struct vc4_hvs {
 	struct vc4_dev *vc4;
 	struct platform_device *pdev;
 	unsigned int dlist_mem_size;
 	struct clk *core_clk;
+	struct clk *disp_clk;
 	unsigned long max_core_rate;
 	 * list.  Units are dwords.
 	 */
 	struct drm_mm dlist_mm;
 	/* Memory manager for the LBM memory used by HVS scaling. */
 	struct drm_mm lbm_mm;
+	/* Memory manager for the UPM memory used for prefetching. */
+	struct drm_mm upm_mm;
+	struct ida upm_handles;
+	struct vc4_upm_refcounts upm_refcounts[VC4_NUM_UPM_HANDLES + 1];
 	spinlock_t mm_lock;
 	struct drm_mm_node mitchell_netravali_filter;
 };
 #define HVS_NUM_CHANNELS 3
+#define HVS_UBM_WORD_SIZE 256
 struct vc4_hvs_state {
 	struct drm_private_state base;
 	/* Our allocation in LBM for temporary storage during scaling. */
 	struct drm_mm_node lbm;
+	/* The Unified Pre-Fetcher Handle */
+	unsigned int upm_handle[DRM_FORMAT_MAX_PLANES];
+	/* Number of lines to pre-fetch */
+	unsigned int upm_buffer_lines;
 	/* Set when the plane has per-pixel alpha content or does not cover
 	 * the entire screen. This is a hint to the CRTC that it might need
 	 * to enable background color fill.
 	VC4_ENCODER_TYPE_DSI1,
 	VC4_ENCODER_TYPE_SMI,
 	VC4_ENCODER_TYPE_DPI,
-	VC4_ENCODER_TYPE_TXP,
+	VC4_ENCODER_TYPE_TXP0,
+	VC4_ENCODER_TYPE_TXP1,
 };
 struct vc4_encoder {
 	int hvs_output;
 };
-extern const struct vc4_crtc_data vc4_txp_crtc_data;
+struct vc4_txp_data {
+	struct vc4_crtc_data	base;
+	enum vc4_encoder_type encoder_type;
+	unsigned int high_addr_ptr_reg;
+	unsigned int has_byte_enable:1;
+	unsigned int size_minus_one:1;
+	unsigned int supports_40bit_addresses:1;
+};
+extern const struct vc4_txp_data bcm2835_txp_data;
 struct vc4_pv_data {
 	struct vc4_crtc_data	base;
 extern const struct vc4_pv_data bcm2711_pv2_data;
 extern const struct vc4_pv_data bcm2711_pv3_data;
 extern const struct vc4_pv_data bcm2711_pv4_data;
+extern const struct vc4_pv_data bcm2712_pv0_data;
+extern const struct vc4_pv_data bcm2712_pv1_data;
 struct vc4_crtc {
 	struct drm_crtc base;
 		writel(val, hvs->regs + (offset));					\
 	} while (0)
+#define HVS_READ6(offset) \
+	HVS_READ(hvs->vc4->gen == VC4_GEN_6_C ? SCALER6_ ## offset : SCALER6D_ ## offset)
+#define HVS_WRITE6(offset, val) \
+	HVS_WRITE(hvs->vc4->gen == VC4_GEN_6_C ? SCALER6_ ## offset : SCALER6D_ ## offset, val)
 #define VC4_REG32(reg) { .name = #reg, .offset = reg }
 struct vc4_exec_info {

drivers/gpu/drm/vc4/vc4_hdmi.c+99−8

@@ -845,6 +845,7 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder, · 844 unmodified lines
845	{	845	{
846	struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);	846	struct vc4_hdmi *vc4_hdmi = encoder_to_vc4_hdmi(encoder);
847	struct drm_device *drm = vc4_hdmi->connector.dev;	847	struct drm_device *drm = vc4_hdmi->connector.dev;
		848	struct vc4_dev *vc4 = to_vc4_dev(drm);
848	unsigned long flags;	849	unsigned long flags;
849	int idx;	850	int idx;
850		851
@@ -861,14 +862,25 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder, · 10 unmodified lines
861		862
862	HDMI_WRITE(HDMI_VID_CTL, HDMI_READ(HDMI_VID_CTL) \| VC4_HD_VID_CTL_CLRRGB);	863	HDMI_WRITE(HDMI_VID_CTL, HDMI_READ(HDMI_VID_CTL) \| VC4_HD_VID_CTL_CLRRGB);
863		864
		865	if (vc4->gen >= VC4_GEN_6_C)
		866	HDMI_WRITE(HDMI_VID_CTL, HDMI_READ(HDMI_VID_CTL) \|
		867	VC4_HD_VID_CTL_BLANKPIX);
		868
864	spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);	869	spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
865		870
866	mdelay(1);	871	mdelay(1);
867		872
868	spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);	873	/*
869	HDMI_WRITE(HDMI_VID_CTL,	874	* TODO: This should work on BCM2712, but doesn't for some
870	HDMI_READ(HDMI_VID_CTL) & ~VC4_HD_VID_CTL_ENABLE);	875	* reason and result in a system lockup.
871	spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);	876	*/
		877	if (vc4->gen < VC4_GEN_6_C) {
		878	spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
		879	HDMI_WRITE(HDMI_VID_CTL,
		880	HDMI_READ(HDMI_VID_CTL) &
		881	~VC4_HD_VID_CTL_ENABLE);
		882	spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
		883	}
872		884
873	vc4_hdmi_disable_scrambling(encoder);	885	vc4_hdmi_disable_scrambling(encoder);
874		886
@@ -1488,7 +1500,6 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder, · 613 unmodified lines
1488	goto err_put_runtime_pm;	1500	goto err_put_runtime_pm;
1489	}	1501	}
1490		1502
1491
1492	vc4_hdmi_cec_update_clk_div(vc4_hdmi);	1503	vc4_hdmi_cec_update_clk_div(vc4_hdmi);
1493		1504
1494	if (tmds_char_rate > 297000000)	1505	if (tmds_char_rate > 297000000)
@@ -1594,6 +1605,8 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder, · 99 unmodified lines
1594	spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);	1605	spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
1595		1606
1596	HDMI_WRITE(HDMI_VID_CTL,	1607	HDMI_WRITE(HDMI_VID_CTL,
		1608	(HDMI_READ(HDMI_VID_CTL) &
		1609	~(VC4_HD_VID_CTL_VSYNC_LOW \| VC4_HD_VID_CTL_HSYNC_LOW)) \|
1597	VC4_HD_VID_CTL_ENABLE \|	1610	VC4_HD_VID_CTL_ENABLE \|
1598	VC4_HD_VID_CTL_CLRRGB \|	1611	VC4_HD_VID_CTL_CLRRGB \|
1599	VC4_HD_VID_CTL_UNDERFLOW_ENABLE \|	1612	VC4_HD_VID_CTL_UNDERFLOW_ENABLE \|
@@ -2110,18 +2123,33 @@ static int vc4_hdmi_audio_prepare(struct device dev, void data, · 510 unmodified lines
2110	VC4_HDMI_AUDIO_PACKET_CEA_MASK);	2123	VC4_HDMI_AUDIO_PACKET_CEA_MASK);
2111		2124
2112	/* Set the MAI threshold */	2125	/* Set the MAI threshold */
2113	if (vc4->gen >= VC4_GEN_5)	2126	switch (vc4->gen) {
		2127	case VC4_GEN_6_D:
		2128	HDMI_WRITE(HDMI_MAI_THR,
		2129	VC4_SET_FIELD(0x10, VC6_D_HD_MAI_THR_PANICHIGH) \|
		2130	VC4_SET_FIELD(0x10, VC6_D_HD_MAI_THR_PANICLOW) \|
		2131	VC4_SET_FIELD(0x1c, VC6_D_HD_MAI_THR_DREQHIGH) \|
		2132	VC4_SET_FIELD(0x1c, VC6_D_HD_MAI_THR_DREQLOW));
		2133	break;
		2134	case VC4_GEN_6_C:
		2135	case VC4_GEN_5:
2114	HDMI_WRITE(HDMI_MAI_THR,	2136	HDMI_WRITE(HDMI_MAI_THR,
2115	VC4_SET_FIELD(0x10, VC4_HD_MAI_THR_PANICHIGH) \|	2137	VC4_SET_FIELD(0x10, VC4_HD_MAI_THR_PANICHIGH) \|
2116	VC4_SET_FIELD(0x10, VC4_HD_MAI_THR_PANICLOW) \|	2138	VC4_SET_FIELD(0x10, VC4_HD_MAI_THR_PANICLOW) \|
2117	VC4_SET_FIELD(0x1c, VC4_HD_MAI_THR_DREQHIGH) \|	2139	VC4_SET_FIELD(0x1c, VC4_HD_MAI_THR_DREQHIGH) \|
2118	VC4_SET_FIELD(0x1c, VC4_HD_MAI_THR_DREQLOW));	2140	VC4_SET_FIELD(0x1c, VC4_HD_MAI_THR_DREQLOW));
2119	else	2141	break;
		2142	case VC4_GEN_4:
2120	HDMI_WRITE(HDMI_MAI_THR,	2143	HDMI_WRITE(HDMI_MAI_THR,
2121	VC4_SET_FIELD(0x8, VC4_HD_MAI_THR_PANICHIGH) \|	2144	VC4_SET_FIELD(0x8, VC4_HD_MAI_THR_PANICHIGH) \|
2122	VC4_SET_FIELD(0x8, VC4_HD_MAI_THR_PANICLOW) \|	2145	VC4_SET_FIELD(0x8, VC4_HD_MAI_THR_PANICLOW) \|
2123	VC4_SET_FIELD(0x6, VC4_HD_MAI_THR_DREQHIGH) \|	2146	VC4_SET_FIELD(0x6, VC4_HD_MAI_THR_DREQHIGH) \|
2124	VC4_SET_FIELD(0x8, VC4_HD_MAI_THR_DREQLOW));	2147	VC4_SET_FIELD(0x8, VC4_HD_MAI_THR_DREQLOW));
		2148	break;
		2149	default:
		2150	drm_err(drm, "Unknown VC4 generation: %d", vc4->gen);
		2151	break;
		2152	}
2125		2153
2126	HDMI_WRITE(HDMI_MAI_CONFIG,	2154	HDMI_WRITE(HDMI_MAI_CONFIG,
2127	VC4_HDMI_MAI_CONFIG_BIT_REVERSE \|	2155	VC4_HDMI_MAI_CONFIG_BIT_REVERSE \|
@@ -3121,6 +3149,7 @@ static int vc4_hdmi_runtime_suspend(struct device *dev) · 993 unmodified lines
3121	{	3149	{
3122	struct vc4_hdmi *vc4_hdmi = dev_get_drvdata(dev);	3150	struct vc4_hdmi *vc4_hdmi = dev_get_drvdata(dev);
3123		3151
		3152	clk_disable_unprepare(vc4_hdmi->audio_clock);
3124	clk_disable_unprepare(vc4_hdmi->hsm_clock);	3153	clk_disable_unprepare(vc4_hdmi->hsm_clock);
3125		3154
3126	return 0;	3155	return 0;
@@ -3153,6 +3182,10 @@ static int vc4_hdmi_runtime_resume(struct device *dev) · 26 unmodified lines
3153	goto err_disable_clk;	3182	goto err_disable_clk;
3154	}	3183	}
3155		3184
		3185	ret = clk_prepare_enable(vc4_hdmi->audio_clock);
		3186	if (ret)
		3187	goto err_disable_clk;
		3188
3156	if (vc4_hdmi->variant->reset)	3189	if (vc4_hdmi->variant->reset)
3157	vc4_hdmi->variant->reset(vc4_hdmi);	3190	vc4_hdmi->variant->reset(vc4_hdmi);
3158		3191
@@ -3273,7 +3306,9 @@ static int vc4_hdmi_bind(struct device dev, struct device master, void *data) · 114 unmodified lines
3273	return ret;	3306	return ret;
3274		3307
3275	if ((of_device_is_compatible(dev->of_node, "brcm,bcm2711-hdmi0") \|\|	3308	if ((of_device_is_compatible(dev->of_node, "brcm,bcm2711-hdmi0") \|\|
3276	of_device_is_compatible(dev->of_node, "brcm,bcm2711-hdmi1")) &&	3309	of_device_is_compatible(dev->of_node, "brcm,bcm2711-hdmi1") \|\|
		3310	of_device_is_compatible(dev->of_node, "brcm,bcm2712-hdmi0") \|\|
		3311	of_device_is_compatible(dev->of_node, "brcm,bcm2712-hdmi1")) &&
3277	HDMI_READ(HDMI_VID_CTL) & VC4_HD_VID_CTL_ENABLE) {	3312	HDMI_READ(HDMI_VID_CTL) & VC4_HD_VID_CTL_ENABLE) {
3278	clk_prepare_enable(vc4_hdmi->pixel_clock);	3313	clk_prepare_enable(vc4_hdmi->pixel_clock);
3279	clk_prepare_enable(vc4_hdmi->hsm_clock);	3314	clk_prepare_enable(vc4_hdmi->hsm_clock);
@@ -3407,10 +3442,66 @@ static const struct vc4_hdmi_variant bcm2711_hdmi1_variant = { · 127 unmodified lines
3407	.hp_detect = vc5_hdmi_hp_detect,	3442	.hp_detect = vc5_hdmi_hp_detect,
3408	};	3443	};
3409		3444
		3445	static const struct vc4_hdmi_variant bcm2712_hdmi0_variant = {
		3446	.encoder_type = VC4_ENCODER_TYPE_HDMI0,
		3447	.debugfs_name = "hdmi0_regs",
		3448	.card_name = "vc4-hdmi-0",
		3449	.max_pixel_clock = 600000000,
		3450	.registers = vc6_hdmi_hdmi0_fields,
		3451	.num_registers = ARRAY_SIZE(vc6_hdmi_hdmi0_fields),
		3452	.phy_lane_mapping = {
		3453	PHY_LANE_0,
		3454	PHY_LANE_1,
		3455	PHY_LANE_2,
		3456	PHY_LANE_CK,
		3457	},
		3458	.unsupported_odd_h_timings = false,
		3459	.external_irq_controller = true,
		3460
		3461	.init_resources = vc5_hdmi_init_resources,
		3462	.csc_setup = vc5_hdmi_csc_setup,
		3463	.reset = vc5_hdmi_reset,
		3464	.set_timings = vc5_hdmi_set_timings,
		3465	.phy_init = vc6_hdmi_phy_init,
		3466	.phy_disable = vc6_hdmi_phy_disable,
		3467	.channel_map = vc5_hdmi_channel_map,
		3468	.supports_hdr = true,
		3469	.hp_detect = vc5_hdmi_hp_detect,
		3470	};
		3471
		3472	static const struct vc4_hdmi_variant bcm2712_hdmi1_variant = {
		3473	.encoder_type = VC4_ENCODER_TYPE_HDMI1,
		3474	.debugfs_name = "hdmi1_regs",
		3475	.card_name = "vc4-hdmi-1",
		3476	.max_pixel_clock = 600000000,
		3477	.registers = vc6_hdmi_hdmi1_fields,
		3478	.num_registers = ARRAY_SIZE(vc6_hdmi_hdmi1_fields),
		3479	.phy_lane_mapping = {
		3480	PHY_LANE_0,
		3481	PHY_LANE_1,
		3482	PHY_LANE_2,
		3483	PHY_LANE_CK,
		3484	},
		3485	.unsupported_odd_h_timings = false,
		3486	.external_irq_controller = true,
		3487
		3488	.init_resources = vc5_hdmi_init_resources,
		3489	.csc_setup = vc5_hdmi_csc_setup,
		3490	.reset = vc5_hdmi_reset,
		3491	.set_timings = vc5_hdmi_set_timings,
		3492	.phy_init = vc6_hdmi_phy_init,
		3493	.phy_disable = vc6_hdmi_phy_disable,
		3494	.channel_map = vc5_hdmi_channel_map,
		3495	.supports_hdr = true,
		3496	.hp_detect = vc5_hdmi_hp_detect,
		3497	};
		3498
3410	static const struct of_device_id vc4_hdmi_dt_match[] = {	3499	static const struct of_device_id vc4_hdmi_dt_match[] = {
3411	{ .compatible = "brcm,bcm2835-hdmi", .data = &bcm2835_variant },	3500	{ .compatible = "brcm,bcm2835-hdmi", .data = &bcm2835_variant },
3412	{ .compatible = "brcm,bcm2711-hdmi0", .data = &bcm2711_hdmi0_variant },	3501	{ .compatible = "brcm,bcm2711-hdmi0", .data = &bcm2711_hdmi0_variant },
3413	{ .compatible = "brcm,bcm2711-hdmi1", .data = &bcm2711_hdmi1_variant },	3502	{ .compatible = "brcm,bcm2711-hdmi1", .data = &bcm2711_hdmi1_variant },
		3503	{ .compatible = "brcm,bcm2712-hdmi0", .data = &bcm2712_hdmi0_variant },
		3504	{ .compatible = "brcm,bcm2712-hdmi1", .data = &bcm2712_hdmi1_variant },
3414	{}	3505	{}
3415	};	3506	};
3416		3507

drivers/gpu/drm/vc4/vc4_hdmi.h+4−0

@@ -237,4 +237,8 @@ void vc5_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi); · 236 unmodified lines
237	void vc5_hdmi_phy_rng_enable(struct vc4_hdmi *vc4_hdmi);	237	void vc5_hdmi_phy_rng_enable(struct vc4_hdmi *vc4_hdmi);
238	void vc5_hdmi_phy_rng_disable(struct vc4_hdmi *vc4_hdmi);	238	void vc5_hdmi_phy_rng_disable(struct vc4_hdmi *vc4_hdmi);
239		239
		240	void vc6_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
		241	struct drm_connector_state *conn_state);
		242	void vc6_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi);
		243
240	#endif /* _VC4_HDMI_H_ */	244	#endif /* _VC4_HDMI_H_ */

drivers/gpu/drm/vc4/vc4_hdmi_phy.c+640−0

@@ -125,6 +125,48 @@ · 124 unmodified lines
125	#define VC4_HDMI_RM_FORMAT_SHIFT_SHIFT 24	125	#define VC4_HDMI_RM_FORMAT_SHIFT_SHIFT 24
126	#define VC4_HDMI_RM_FORMAT_SHIFT_MASK VC4_MASK(25, 24)	126	#define VC4_HDMI_RM_FORMAT_SHIFT_MASK VC4_MASK(25, 24)
127		127
		128	#define VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_BG_PWRUP BIT(8)
		129	#define VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_LDO_PWRUP BIT(7)
		130	#define VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_BIAS_PWRUP BIT(6)
		131	#define VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_RNDGEN_PWRUP BIT(4)
		132	#define VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_TX_CK_PWRUP BIT(3)
		133	#define VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_TX_2_PWRUP BIT(2)
		134	#define VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_TX_1_PWRUP BIT(1)
		135	#define VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_TX_0_PWRUP BIT(0)
		136
		137	#define VC6_HDMI_TX_PHY_PLL_REFCLK_REFCLK_SEL_CMOS BIT(13)
		138	#define VC6_HDMI_TX_PHY_PLL_REFCLK_REFFRQ_MASK VC4_MASK(9, 0)
		139
		140	#define VC6_HDMI_TX_PHY_PLL_POST_KDIV_CLK0_SEL_MASK VC4_MASK(3, 2)
		141	#define VC6_HDMI_TX_PHY_PLL_POST_KDIV_KDIV_MASK VC4_MASK(1, 0)
		142
		143	#define VC6_HDMI_TX_PHY_PLL_VCOCLK_DIV_VCODIV_EN BIT(10)
		144	#define VC6_HDMI_TX_PHY_PLL_VCOCLK_DIV_VCODIV_MASK VC4_MASK(9, 0)
		145
		146	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_CTL_MASK VC4_MASK(31, 28)
		147	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_ENABLE_MASK VC4_MASK(27, 27)
		148	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_RATE_CTL_MASK VC4_MASK(26, 26)
		149	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_POST_TAP_EN_MASK VC4_MASK(25, 25)
		150	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_LDMOS_BIAS_CTL_MASK VC4_MASK(24, 23)
		151	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_COM_MODE_LDMOS_EN_MASK VC4_MASK(22, 22)
		152	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EDGE_SEL_MASK VC4_MASK(21, 21)
		153	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_HS_EN_MASK VC4_MASK(20, 20)
		154	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_TERM_CTL_MASK VC4_MASK(19, 18)
		155	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_EN_MASK VC4_MASK(17, 17)
		156	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_EN_MASK VC4_MASK(16, 16)
		157	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_CTL_MASK VC4_MASK(15, 12)
		158	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_HS_EN_MASK VC4_MASK(11, 11)
		159	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_MAIN_TAP_CURRENT_SELECT_MASK VC4_MASK(10, 8)
		160	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_POST_TAP_CURRENT_SELECT_MASK VC4_MASK(7, 5)
		161	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_LOADING_MASK VC4_MASK(4, 3)
		162	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_DRIVING_MASK VC4_MASK(2, 1)
		163	#define VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_PRE_TAP_EN_MASK VC4_MASK(0, 0)
		164
		165	#define VC6_HDMI_TX_PHY_PLL_RESET_CTL_PLL_PLLPOST_RESETB BIT(1)
		166	#define VC6_HDMI_TX_PHY_PLL_RESET_CTL_PLL_RESETB BIT(0)
		167
		168	#define VC6_HDMI_TX_PHY_PLL_POWERUP_CTL_PLL_PWRUP BIT(0)
		169
128	#define OSCILLATOR_FREQUENCY 54000000	170	#define OSCILLATOR_FREQUENCY 54000000
129		171
130	void vc4_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,	172	void vc4_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
@@ -558,3 +600,601 @@ void vc5_hdmi_phy_rng_disable(struct vc4_hdmi *vc4_hdmi) · 427 unmodified lines
558	VC4_HDMI_TX_PHY_POWERDOWN_CTL_RNDGEN_PWRDN);	600	VC4_HDMI_TX_PHY_POWERDOWN_CTL_RNDGEN_PWRDN);
559	spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);	601	spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
560	}	602	}
		603
		604	#define VC6_VCO_MIN_FREQ (8ULL * 1000 * 1000 * 1000)
		605	#define VC6_VCO_MAX_FREQ (12ULL * 1000 * 1000 * 1000)
		606
		607	static unsigned long long
		608	vc6_phy_get_vco_freq(unsigned long long tmds_rate, unsigned int *vco_div)
		609	{
		610	unsigned int min_div;
		611	unsigned int max_div;
		612	unsigned int div;
		613
		614	div = 0;
		615	while (tmds_rate * div * 10 < VC6_VCO_MIN_FREQ)
		616	div++;
		617	min_div = div;
		618
		619	while (tmds_rate * (div + 1) * 10 < VC6_VCO_MAX_FREQ)
		620	div++;
		621	max_div = div;
		622
		623	div = min_div + (max_div - min_div) / 2;
		624
		625	*vco_div = div;
		626	return tmds_rate * div * 10;
		627	}
		628
		629	struct vc6_phy_lane_settings {
		630	unsigned int ext_current_ctl:4;
		631	unsigned int ffe_enable:1;
		632	unsigned int slew_rate_ctl:1;
		633	unsigned int ffe_post_tap_en:1;
		634	unsigned int ldmos_bias_ctl:2;
		635	unsigned int com_mode_ldmos_en:1;
		636	unsigned int edge_sel:1;
		637	unsigned int ext_current_src_hs_en:1;
		638	unsigned int term_ctl:2;
		639	unsigned int ext_current_src_en:1;
		640	unsigned int int_current_src_en:1;
		641	unsigned int int_current_ctl:4;
		642	unsigned int int_current_src_hs_en:1;
		643	unsigned int main_tap_current_select:3;
		644	unsigned int post_tap_current_select:3;
		645	unsigned int slew_ctl_slow_loading:2;
		646	unsigned int slew_ctl_slow_driving:2;
		647	unsigned int ffe_pre_tap_en:1;
		648	};
		649
		650	struct vc6_phy_settings {
		651	unsigned long long min_rate;
		652	unsigned long long max_rate;
		653	struct vc6_phy_lane_settings channel[3];
		654	struct vc6_phy_lane_settings clock;
		655	};
		656
		657	static const struct vc6_phy_settings vc6_hdmi_phy_settings[] = {
		658	{
		659	0, 222000000,
		660	{
		661	{
		662	/* 200mA */
		663	.ext_current_ctl = 8,
		664
		665	/* 0.85V */
		666	.ldmos_bias_ctl = 1,
		667
		668	/* Enable External Current Source */
		669	.ext_current_src_en = 1,
		670
		671	/* 200mA */
		672	.int_current_ctl = 8,
		673
		674	/* 17.6 mA */
		675	.main_tap_current_select = 7,
		676	},
		677	{
		678	/* 200mA */
		679	.ext_current_ctl = 8,
		680
		681	/* 0.85V */
		682	.ldmos_bias_ctl = 1,
		683
		684	/* Enable External Current Source */
		685	.ext_current_src_en = 1,
		686
		687	/* 200mA */
		688	.int_current_ctl = 8,
		689
		690	/* 17.6 mA */
		691	.main_tap_current_select = 7,
		692	},
		693	{
		694	/* 200mA */
		695	.ext_current_ctl = 8,
		696
		697	/* 0.85V */
		698	.ldmos_bias_ctl = 1,
		699
		700	/* Enable External Current Source */
		701	.ext_current_src_en = 1,
		702
		703	/* 200mA */
		704	.int_current_ctl = 8,
		705
		706	/* 17.6 mA */
		707	.main_tap_current_select = 7,
		708	},
		709	},
		710	{
		711	/* 200mA */
		712	.ext_current_ctl = 8,
		713
		714	/* 0.85V */
		715	.ldmos_bias_ctl = 1,
		716
		717	/* Enable External Current Source */
		718	.ext_current_src_en = 1,
		719
		720	/* 200mA */
		721	.int_current_ctl = 8,
		722
		723	/* 17.6 mA */
		724	.main_tap_current_select = 7,
		725	},
		726	},
		727	{
		728	222000001, 297000000,
		729	{
		730	{
		731	/* 200mA and 180mA ?! */
		732	.ext_current_ctl = 12,
		733
		734	/* 0.85V */
		735	.ldmos_bias_ctl = 1,
		736
		737	/* 100 Ohm */
		738	.term_ctl = 1,
		739
		740	/* Enable External Current Source */
		741	.ext_current_src_en = 1,
		742
		743	/* Enable Internal Current Source */
		744	.int_current_src_en = 1,
		745	},
		746	{
		747	/* 200mA and 180mA ?! */
		748	.ext_current_ctl = 12,
		749
		750	/* 0.85V */
		751	.ldmos_bias_ctl = 1,
		752
		753	/* 100 Ohm */
		754	.term_ctl = 1,
		755
		756	/* Enable External Current Source */
		757	.ext_current_src_en = 1,
		758
		759	/* Enable Internal Current Source */
		760	.int_current_src_en = 1,
		761	},
		762	{
		763	/* 200mA and 180mA ?! */
		764	.ext_current_ctl = 12,
		765
		766	/* 0.85V */
		767	.ldmos_bias_ctl = 1,
		768
		769	/* 100 Ohm */
		770	.term_ctl = 1,
		771
		772	/* Enable External Current Source */
		773	.ext_current_src_en = 1,
		774
		775	/* Enable Internal Current Source */
		776	.int_current_src_en = 1,
		777	},
		778	},
		779	{
		780	/* 200mA and 180mA ?! */
		781	.ext_current_ctl = 12,
		782
		783	/* 0.85V */
		784	.ldmos_bias_ctl = 1,
		785
		786	/* 100 Ohm */
		787	.term_ctl = 1,
		788
		789	/* Enable External Current Source */
		790	.ext_current_src_en = 1,
		791
		792	/* Enable Internal Current Source */
		793	.int_current_src_en = 1,
		794
		795	/* Internal Current Source Half Swing Enable*/
		796	.int_current_src_hs_en = 1,
		797	},
		798	},
		799	{
		800	297000001, 597000044,
		801	{
		802	{
		803	/* 200mA */
		804	.ext_current_ctl = 8,
		805
		806	/* Normal Slew Rate Control */
		807	.slew_rate_ctl = 1,
		808
		809	/* 0.85V */
		810	.ldmos_bias_ctl = 1,
		811
		812	/* 50 Ohms */
		813	.term_ctl = 3,
		814
		815	/* Enable External Current Source */
		816	.ext_current_src_en = 1,
		817
		818	/* Enable Internal Current Source */
		819	.int_current_src_en = 1,
		820
		821	/* 200mA */
		822	.int_current_ctl = 8,
		823
		824	/* 17.6 mA */
		825	.main_tap_current_select = 7,
		826	},
		827	{
		828	/* 200mA */
		829	.ext_current_ctl = 8,
		830
		831	/* Normal Slew Rate Control */
		832	.slew_rate_ctl = 1,
		833
		834	/* 0.85V */
		835	.ldmos_bias_ctl = 1,
		836
		837	/* 50 Ohms */
		838	.term_ctl = 3,
		839
		840	/* Enable External Current Source */
		841	.ext_current_src_en = 1,
		842
		843	/* Enable Internal Current Source */
		844	.int_current_src_en = 1,
		845
		846	/* 200mA */
		847	.int_current_ctl = 8,
		848
		849	/* 17.6 mA */
		850	.main_tap_current_select = 7,
		851	},
		852	{
		853	/* 200mA */
		854	.ext_current_ctl = 8,
		855
		856	/* Normal Slew Rate Control */
		857	.slew_rate_ctl = 1,
		858
		859	/* 0.85V */
		860	.ldmos_bias_ctl = 1,
		861
		862	/* 50 Ohms */
		863	.term_ctl = 3,
		864
		865	/* Enable External Current Source */
		866	.ext_current_src_en = 1,
		867
		868	/* Enable Internal Current Source */
		869	.int_current_src_en = 1,
		870
		871	/* 200mA */
		872	.int_current_ctl = 8,
		873
		874	/* 17.6 mA */
		875	.main_tap_current_select = 7,
		876	},
		877	},
		878	{
		879	/* 200mA */
		880	.ext_current_ctl = 8,
		881
		882	/* Normal Slew Rate Control */
		883	.slew_rate_ctl = 1,
		884
		885	/* 0.85V */
		886	.ldmos_bias_ctl = 1,
		887
		888	/* External Current Source Half Swing Enable*/
		889	.ext_current_src_hs_en = 1,
		890
		891	/* 50 Ohms */
		892	.term_ctl = 3,
		893
		894	/* Enable External Current Source */
		895	.ext_current_src_en = 1,
		896
		897	/* Enable Internal Current Source */
		898	.int_current_src_en = 1,
		899
		900	/* 200mA */
		901	.int_current_ctl = 8,
		902
		903	/* Internal Current Source Half Swing Enable*/
		904	.int_current_src_hs_en = 1,
		905
		906	/* 17.6 mA */
		907	.main_tap_current_select = 7,
		908	},
		909	},
		910	};
		911
		912	static const struct vc6_phy_settings *
		913	vc6_phy_get_settings(unsigned long long tmds_rate)
		914	{
		915	unsigned int count = ARRAY_SIZE(vc6_hdmi_phy_settings);
		916	unsigned int i;
		917
		918	for (i = 0; i < count; i++) {
		919	const struct vc6_phy_settings *s = &vc6_hdmi_phy_settings[i];
		920
		921	if (tmds_rate >= s->min_rate && tmds_rate <= s->max_rate)
		922	return s;
		923	}
		924
		925	/*
		926	* If the pixel clock exceeds our max setting, try the max
		927	* setting anyway.
		928	*/
		929	return &vc6_hdmi_phy_settings[count - 1];
		930	}
		931
		932	static const struct vc6_phy_lane_settings *
		933	vc6_phy_get_channel_settings(enum vc4_hdmi_phy_channel chan,
		934	unsigned long long tmds_rate)
		935	{
		936	const struct vc6_phy_settings *settings = vc6_phy_get_settings(tmds_rate);
		937
		938	if (chan == PHY_LANE_CK)
		939	return &settings->clock;
		940
		941	return &settings->channel[chan];
		942	}
		943
		944	static void vc6_hdmi_reset_phy(struct vc4_hdmi *vc4_hdmi)
		945	{
		946	lockdep_assert_held(&vc4_hdmi->hw_lock);
		947
		948	HDMI_WRITE(HDMI_TX_PHY_RESET_CTL, 0);
		949	HDMI_WRITE(HDMI_TX_PHY_POWERUP_CTL, 0);
		950	}
		951
		952	void vc6_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
		953	struct drm_connector_state *conn_state)
		954	{
		955	const struct vc6_phy_lane_settings *chan0_settings;
		956	const struct vc6_phy_lane_settings *chan1_settings;
		957	const struct vc6_phy_lane_settings *chan2_settings;
		958	const struct vc6_phy_lane_settings *clock_settings;
		959	const struct vc4_hdmi_variant *variant = vc4_hdmi->variant;
		960	unsigned long long pixel_freq = conn_state->hdmi.tmds_char_rate;
		961	unsigned long long vco_freq;
		962	unsigned char word_sel;
		963	unsigned long flags;
		964	unsigned int vco_div;
		965
		966	vco_freq = vc6_phy_get_vco_freq(pixel_freq, &vco_div);
		967
		968	spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
		969
		970	vc6_hdmi_reset_phy(vc4_hdmi);
		971
		972	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_0, 0x810c6000);
		973	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_1, 0x00b8c451);
		974	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_2, 0x46402e31);
		975	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_3, 0x00b8c005);
		976	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_4, 0x42410261);
		977	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_5, 0xcc021001);
		978	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_6, 0xc8301c80);
		979	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_7, 0xb0804444);
		980	HDMI_WRITE(HDMI_TX_PHY_PLL_MISC_8, 0xf80f8000);
		981
		982	HDMI_WRITE(HDMI_TX_PHY_PLL_REFCLK,
		983	VC6_HDMI_TX_PHY_PLL_REFCLK_REFCLK_SEL_CMOS \|
		984	VC4_SET_FIELD(54, VC6_HDMI_TX_PHY_PLL_REFCLK_REFFRQ));
		985
		986	HDMI_WRITE(HDMI_TX_PHY_RESET_CTL, 0x7f);
		987
		988	HDMI_WRITE(HDMI_RM_OFFSET,
		989	VC4_HDMI_RM_OFFSET_ONLY \|
		990	VC4_SET_FIELD(phy_get_rm_offset(vco_freq),
		991	VC4_HDMI_RM_OFFSET_OFFSET));
		992
		993	HDMI_WRITE(HDMI_TX_PHY_PLL_VCOCLK_DIV,
		994	VC6_HDMI_TX_PHY_PLL_VCOCLK_DIV_VCODIV_EN \|
		995	VC4_SET_FIELD(vco_div,
		996	VC6_HDMI_TX_PHY_PLL_VCOCLK_DIV_VCODIV));
		997
		998	HDMI_WRITE(HDMI_TX_PHY_PLL_CFG,
		999	VC4_SET_FIELD(0, VC4_HDMI_TX_PHY_PLL_CFG_PDIV));
		1000
		1001	HDMI_WRITE(HDMI_TX_PHY_PLL_POST_KDIV,
		1002	VC4_SET_FIELD(2, VC6_HDMI_TX_PHY_PLL_POST_KDIV_CLK0_SEL) \|
		1003	VC4_SET_FIELD(1, VC6_HDMI_TX_PHY_PLL_POST_KDIV_KDIV));
		1004
		1005	chan0_settings =
		1006	vc6_phy_get_channel_settings(variant->phy_lane_mapping[PHY_LANE_0],
		1007	pixel_freq);
		1008	HDMI_WRITE(HDMI_TX_PHY_CTL_0,
		1009	VC4_SET_FIELD(chan0_settings->ext_current_ctl,
		1010	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_CTL) \|
		1011	VC4_SET_FIELD(chan0_settings->ffe_enable,
		1012	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_ENABLE) \|
		1013	VC4_SET_FIELD(chan0_settings->slew_rate_ctl,
		1014	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_RATE_CTL) \|
		1015	VC4_SET_FIELD(chan0_settings->ffe_post_tap_en,
		1016	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_POST_TAP_EN) \|
		1017	VC4_SET_FIELD(chan0_settings->ldmos_bias_ctl,
		1018	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_LDMOS_BIAS_CTL) \|
		1019	VC4_SET_FIELD(chan0_settings->com_mode_ldmos_en,
		1020	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_COM_MODE_LDMOS_EN) \|
		1021	VC4_SET_FIELD(chan0_settings->edge_sel,
		1022	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EDGE_SEL) \|
		1023	VC4_SET_FIELD(chan0_settings->ext_current_src_hs_en,
		1024	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_HS_EN) \|
		1025	VC4_SET_FIELD(chan0_settings->term_ctl,
		1026	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_TERM_CTL) \|
		1027	VC4_SET_FIELD(chan0_settings->ext_current_src_en,
		1028	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_EN) \|
		1029	VC4_SET_FIELD(chan0_settings->int_current_src_en,
		1030	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_EN) \|
		1031	VC4_SET_FIELD(chan0_settings->int_current_ctl,
		1032	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_CTL) \|
		1033	VC4_SET_FIELD(chan0_settings->int_current_src_hs_en,
		1034	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_HS_EN) \|
		1035	VC4_SET_FIELD(chan0_settings->main_tap_current_select,
		1036	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_MAIN_TAP_CURRENT_SELECT) \|
		1037	VC4_SET_FIELD(chan0_settings->post_tap_current_select,
		1038	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_POST_TAP_CURRENT_SELECT) \|
		1039	VC4_SET_FIELD(chan0_settings->slew_ctl_slow_loading,
		1040	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_LOADING) \|
		1041	VC4_SET_FIELD(chan0_settings->slew_ctl_slow_driving,
		1042	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_DRIVING) \|
		1043	VC4_SET_FIELD(chan0_settings->ffe_pre_tap_en,
		1044	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_PRE_TAP_EN));
		1045
		1046	chan1_settings =
		1047	vc6_phy_get_channel_settings(variant->phy_lane_mapping[PHY_LANE_1],
		1048	pixel_freq);
		1049	HDMI_WRITE(HDMI_TX_PHY_CTL_1,
		1050	VC4_SET_FIELD(chan1_settings->ext_current_ctl,
		1051	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_CTL) \|
		1052	VC4_SET_FIELD(chan1_settings->ffe_enable,
		1053	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_ENABLE) \|
		1054	VC4_SET_FIELD(chan1_settings->slew_rate_ctl,
		1055	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_RATE_CTL) \|
		1056	VC4_SET_FIELD(chan1_settings->ffe_post_tap_en,
		1057	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_POST_TAP_EN) \|
		1058	VC4_SET_FIELD(chan1_settings->ldmos_bias_ctl,
		1059	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_LDMOS_BIAS_CTL) \|
		1060	VC4_SET_FIELD(chan1_settings->com_mode_ldmos_en,
		1061	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_COM_MODE_LDMOS_EN) \|
		1062	VC4_SET_FIELD(chan1_settings->edge_sel,
		1063	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EDGE_SEL) \|
		1064	VC4_SET_FIELD(chan1_settings->ext_current_src_hs_en,
		1065	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_HS_EN) \|
		1066	VC4_SET_FIELD(chan1_settings->term_ctl,
		1067	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_TERM_CTL) \|
		1068	VC4_SET_FIELD(chan1_settings->ext_current_src_en,
		1069	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_EN) \|
		1070	VC4_SET_FIELD(chan1_settings->int_current_src_en,
		1071	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_EN) \|
		1072	VC4_SET_FIELD(chan1_settings->int_current_ctl,
		1073	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_CTL) \|
		1074	VC4_SET_FIELD(chan1_settings->int_current_src_hs_en,
		1075	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_HS_EN) \|
		1076	VC4_SET_FIELD(chan1_settings->main_tap_current_select,
		1077	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_MAIN_TAP_CURRENT_SELECT) \|
		1078	VC4_SET_FIELD(chan1_settings->post_tap_current_select,
		1079	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_POST_TAP_CURRENT_SELECT) \|
		1080	VC4_SET_FIELD(chan1_settings->slew_ctl_slow_loading,
		1081	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_LOADING) \|
		1082	VC4_SET_FIELD(chan1_settings->slew_ctl_slow_driving,
		1083	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_DRIVING) \|
		1084	VC4_SET_FIELD(chan1_settings->ffe_pre_tap_en,
		1085	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_PRE_TAP_EN));
		1086
		1087	chan2_settings =
		1088	vc6_phy_get_channel_settings(variant->phy_lane_mapping[PHY_LANE_2],
		1089	pixel_freq);
		1090	HDMI_WRITE(HDMI_TX_PHY_CTL_2,
		1091	VC4_SET_FIELD(chan2_settings->ext_current_ctl,
		1092	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_CTL) \|
		1093	VC4_SET_FIELD(chan2_settings->ffe_enable,
		1094	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_ENABLE) \|
		1095	VC4_SET_FIELD(chan2_settings->slew_rate_ctl,
		1096	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_RATE_CTL) \|
		1097	VC4_SET_FIELD(chan2_settings->ffe_post_tap_en,
		1098	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_POST_TAP_EN) \|
		1099	VC4_SET_FIELD(chan2_settings->ldmos_bias_ctl,
		1100	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_LDMOS_BIAS_CTL) \|
		1101	VC4_SET_FIELD(chan2_settings->com_mode_ldmos_en,
		1102	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_COM_MODE_LDMOS_EN) \|
		1103	VC4_SET_FIELD(chan2_settings->edge_sel,
		1104	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EDGE_SEL) \|
		1105	VC4_SET_FIELD(chan2_settings->ext_current_src_hs_en,
		1106	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_HS_EN) \|
		1107	VC4_SET_FIELD(chan2_settings->term_ctl,
		1108	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_TERM_CTL) \|
		1109	VC4_SET_FIELD(chan2_settings->ext_current_src_en,
		1110	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_EN) \|
		1111	VC4_SET_FIELD(chan2_settings->int_current_src_en,
		1112	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_EN) \|
		1113	VC4_SET_FIELD(chan2_settings->int_current_ctl,
		1114	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_CTL) \|
		1115	VC4_SET_FIELD(chan2_settings->int_current_src_hs_en,
		1116	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_HS_EN) \|
		1117	VC4_SET_FIELD(chan2_settings->main_tap_current_select,
		1118	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_MAIN_TAP_CURRENT_SELECT) \|
		1119	VC4_SET_FIELD(chan2_settings->post_tap_current_select,
		1120	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_POST_TAP_CURRENT_SELECT) \|
		1121	VC4_SET_FIELD(chan2_settings->slew_ctl_slow_loading,
		1122	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_LOADING) \|
		1123	VC4_SET_FIELD(chan2_settings->slew_ctl_slow_driving,
		1124	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_DRIVING) \|
		1125	VC4_SET_FIELD(chan2_settings->ffe_pre_tap_en,
		1126	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_PRE_TAP_EN));
		1127
		1128	clock_settings =
		1129	vc6_phy_get_channel_settings(variant->phy_lane_mapping[PHY_LANE_CK],
		1130	pixel_freq);
		1131	HDMI_WRITE(HDMI_TX_PHY_CTL_CK,
		1132	VC4_SET_FIELD(clock_settings->ext_current_ctl,
		1133	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_CTL) \|
		1134	VC4_SET_FIELD(clock_settings->ffe_enable,
		1135	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_ENABLE) \|
		1136	VC4_SET_FIELD(clock_settings->slew_rate_ctl,
		1137	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_RATE_CTL) \|
		1138	VC4_SET_FIELD(clock_settings->ffe_post_tap_en,
		1139	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_POST_TAP_EN) \|
		1140	VC4_SET_FIELD(clock_settings->ldmos_bias_ctl,
		1141	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_LDMOS_BIAS_CTL) \|
		1142	VC4_SET_FIELD(clock_settings->com_mode_ldmos_en,
		1143	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_COM_MODE_LDMOS_EN) \|
		1144	VC4_SET_FIELD(clock_settings->edge_sel,
		1145	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EDGE_SEL) \|
		1146	VC4_SET_FIELD(clock_settings->ext_current_src_hs_en,
		1147	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_HS_EN) \|
		1148	VC4_SET_FIELD(clock_settings->term_ctl,
		1149	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_TERM_CTL) \|
		1150	VC4_SET_FIELD(clock_settings->ext_current_src_en,
		1151	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_EXT_CURRENT_SRC_EN) \|
		1152	VC4_SET_FIELD(clock_settings->int_current_src_en,
		1153	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_EN) \|
		1154	VC4_SET_FIELD(clock_settings->int_current_ctl,
		1155	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_CTL) \|
		1156	VC4_SET_FIELD(clock_settings->int_current_src_hs_en,
		1157	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_INT_CURRENT_SRC_HS_EN) \|
		1158	VC4_SET_FIELD(clock_settings->main_tap_current_select,
		1159	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_MAIN_TAP_CURRENT_SELECT) \|
		1160	VC4_SET_FIELD(clock_settings->post_tap_current_select,
		1161	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_POST_TAP_CURRENT_SELECT) \|
		1162	VC4_SET_FIELD(clock_settings->slew_ctl_slow_loading,
		1163	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_LOADING) \|
		1164	VC4_SET_FIELD(clock_settings->slew_ctl_slow_driving,
		1165	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_SLEW_CTL_SLOW_DRIVING) \|
		1166	VC4_SET_FIELD(clock_settings->ffe_pre_tap_en,
		1167	VC6_HDMI_TX_PHY_HDMI_CTRL_CHX_FFE_PRE_TAP_EN));
		1168
		1169	if (pixel_freq >= 340000000)
		1170	word_sel = 3;
		1171	else
		1172	word_sel = 0;
		1173	HDMI_WRITE(HDMI_TX_PHY_TMDS_CLK_WORD_SEL, word_sel);
		1174
		1175	HDMI_WRITE(HDMI_TX_PHY_POWERUP_CTL,
		1176	VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_BG_PWRUP \|
		1177	VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_LDO_PWRUP \|
		1178	VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_BIAS_PWRUP \|
		1179	VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_TX_CK_PWRUP \|
		1180	VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_TX_2_PWRUP \|
		1181	VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_TX_1_PWRUP \|
		1182	VC6_HDMI_TX_PHY_HDMI_POWERUP_CTL_TX_0_PWRUP);
		1183
		1184	HDMI_WRITE(HDMI_TX_PHY_PLL_POWERUP_CTL,
		1185	VC6_HDMI_TX_PHY_PLL_POWERUP_CTL_PLL_PWRUP);
		1186
		1187	HDMI_WRITE(HDMI_TX_PHY_PLL_RESET_CTL,
		1188	HDMI_READ(HDMI_TX_PHY_PLL_RESET_CTL) &
		1189	~VC6_HDMI_TX_PHY_PLL_RESET_CTL_PLL_RESETB);
		1190
		1191	HDMI_WRITE(HDMI_TX_PHY_PLL_RESET_CTL,
		1192	HDMI_READ(HDMI_TX_PHY_PLL_RESET_CTL) \|
		1193	VC6_HDMI_TX_PHY_PLL_RESET_CTL_PLL_RESETB);
		1194
		1195	spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
		1196	}
		1197
		1198	void vc6_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi)
		1199	{
		1200	}

drivers/gpu/drm/vc4/vc4_hdmi_regs.h+217−0

@@ -111,13 +111,30 @@ enum vc4_hdmi_field { · 110 unmodified lines
111	HDMI_TX_PHY_CTL_1,	111	HDMI_TX_PHY_CTL_1,
112	HDMI_TX_PHY_CTL_2,	112	HDMI_TX_PHY_CTL_2,
113	HDMI_TX_PHY_CTL_3,	113	HDMI_TX_PHY_CTL_3,
		114	HDMI_TX_PHY_CTL_CK,
114	HDMI_TX_PHY_PLL_CALIBRATION_CONFIG_1,	115	HDMI_TX_PHY_PLL_CALIBRATION_CONFIG_1,
115	HDMI_TX_PHY_PLL_CALIBRATION_CONFIG_2,	116	HDMI_TX_PHY_PLL_CALIBRATION_CONFIG_2,
116	HDMI_TX_PHY_PLL_CALIBRATION_CONFIG_4,	117	HDMI_TX_PHY_PLL_CALIBRATION_CONFIG_4,
117	HDMI_TX_PHY_PLL_CFG,	118	HDMI_TX_PHY_PLL_CFG,
		119	HDMI_TX_PHY_PLL_CFG_PDIV,
118	HDMI_TX_PHY_PLL_CTL_0,	120	HDMI_TX_PHY_PLL_CTL_0,
119	HDMI_TX_PHY_PLL_CTL_1,	121	HDMI_TX_PHY_PLL_CTL_1,
		122	HDMI_TX_PHY_PLL_MISC_0,
		123	HDMI_TX_PHY_PLL_MISC_1,
		124	HDMI_TX_PHY_PLL_MISC_2,
		125	HDMI_TX_PHY_PLL_MISC_3,
		126	HDMI_TX_PHY_PLL_MISC_4,
		127	HDMI_TX_PHY_PLL_MISC_5,
		128	HDMI_TX_PHY_PLL_MISC_6,
		129	HDMI_TX_PHY_PLL_MISC_7,
		130	HDMI_TX_PHY_PLL_MISC_8,
		131	HDMI_TX_PHY_PLL_POST_KDIV,
		132	HDMI_TX_PHY_PLL_POWERUP_CTL,
		133	HDMI_TX_PHY_PLL_REFCLK,
		134	HDMI_TX_PHY_PLL_RESET_CTL,
		135	HDMI_TX_PHY_PLL_VCOCLK_DIV,
120	HDMI_TX_PHY_POWERDOWN_CTL,	136	HDMI_TX_PHY_POWERDOWN_CTL,
		137	HDMI_TX_PHY_POWERUP_CTL,
121	HDMI_TX_PHY_RESET_CTL,	138	HDMI_TX_PHY_RESET_CTL,
122	HDMI_TX_PHY_TMDS_CLK_WORD_SEL,	139	HDMI_TX_PHY_TMDS_CLK_WORD_SEL,
123	HDMI_VEC_INTERFACE_CFG,	140	HDMI_VEC_INTERFACE_CFG,
@@ -411,6 +428,206 @@ static const struct vc4_hdmi_register __maybe_unused vc5_hdmi_hdmi1_fields[] = { · 287 unmodified lines
411	VC5_CSC_REG(HDMI_CSC_CHANNEL_CTL, 0x02c),	428	VC5_CSC_REG(HDMI_CSC_CHANNEL_CTL, 0x02c),
412	};	429	};
413		430
		431	static const struct vc4_hdmi_register __maybe_unused vc6_hdmi_hdmi0_fields[] = {
		432	VC4_HD_REG(HDMI_DVP_CTL, 0x0000),
		433	VC4_HD_REG(HDMI_MAI_CTL, 0x0010),
		434	VC4_HD_REG(HDMI_MAI_THR, 0x0014),
		435	VC4_HD_REG(HDMI_MAI_FMT, 0x0018),
		436	VC4_HD_REG(HDMI_MAI_DATA, 0x001c),
		437	VC4_HD_REG(HDMI_MAI_SMP, 0x0020),
		438	VC4_HD_REG(HDMI_VID_CTL, 0x0044),
		439	VC4_HD_REG(HDMI_FRAME_COUNT, 0x0060),
		440
		441	VC4_HDMI_REG(HDMI_FIFO_CTL, 0x07c),
		442	VC4_HDMI_REG(HDMI_AUDIO_PACKET_CONFIG, 0x0c0),
		443	VC4_HDMI_REG(HDMI_RAM_PACKET_CONFIG, 0x0c4),
		444	VC4_HDMI_REG(HDMI_RAM_PACKET_STATUS, 0x0cc),
		445	VC4_HDMI_REG(HDMI_CRP_CFG, 0x0d0),
		446	VC4_HDMI_REG(HDMI_CTS_0, 0x0d4),
		447	VC4_HDMI_REG(HDMI_CTS_1, 0x0d8),
		448	VC4_HDMI_REG(HDMI_SCHEDULER_CONTROL, 0x0e8),
		449	VC4_HDMI_REG(HDMI_HORZA, 0x0ec),
		450	VC4_HDMI_REG(HDMI_HORZB, 0x0f0),
		451	VC4_HDMI_REG(HDMI_VERTA0, 0x0f4),
		452	VC4_HDMI_REG(HDMI_VERTB0, 0x0f8),
		453	VC4_HDMI_REG(HDMI_VERTA1, 0x100),
		454	VC4_HDMI_REG(HDMI_VERTB1, 0x104),
		455	VC4_HDMI_REG(HDMI_MISC_CONTROL, 0x114),
		456	VC4_HDMI_REG(HDMI_MAI_CHANNEL_MAP, 0x0a4),
		457	VC4_HDMI_REG(HDMI_MAI_CONFIG, 0x0a8),
		458	VC4_HDMI_REG(HDMI_FORMAT_DET_1, 0x148),
		459	VC4_HDMI_REG(HDMI_FORMAT_DET_2, 0x14c),
		460	VC4_HDMI_REG(HDMI_FORMAT_DET_3, 0x150),
		461	VC4_HDMI_REG(HDMI_FORMAT_DET_4, 0x158),
		462	VC4_HDMI_REG(HDMI_FORMAT_DET_5, 0x15c),
		463	VC4_HDMI_REG(HDMI_FORMAT_DET_6, 0x160),
		464	VC4_HDMI_REG(HDMI_FORMAT_DET_7, 0x164),
		465	VC4_HDMI_REG(HDMI_FORMAT_DET_8, 0x168),
		466	VC4_HDMI_REG(HDMI_FORMAT_DET_9, 0x16c),
		467	VC4_HDMI_REG(HDMI_FORMAT_DET_10, 0x170),
		468	VC4_HDMI_REG(HDMI_DEEP_COLOR_CONFIG_1, 0x18c),
		469	VC4_HDMI_REG(HDMI_GCP_CONFIG, 0x194),
		470	VC4_HDMI_REG(HDMI_GCP_WORD_1, 0x198),
		471	VC4_HDMI_REG(HDMI_HOTPLUG, 0x1c8),
		472	VC4_HDMI_REG(HDMI_SCRAMBLER_CTL, 0x1e4),
		473
		474	VC5_DVP_REG(HDMI_CLOCK_STOP, 0x0bc),
		475	VC5_DVP_REG(HDMI_VEC_INTERFACE_CFG, 0x0f0),
		476	VC5_DVP_REG(HDMI_VEC_INTERFACE_XBAR, 0x0f4),
		477
		478	VC5_PHY_REG(HDMI_TX_PHY_RESET_CTL, 0x000),
		479	VC5_PHY_REG(HDMI_TX_PHY_POWERUP_CTL, 0x004),
		480	VC5_PHY_REG(HDMI_TX_PHY_CTL_0, 0x008),
		481	VC5_PHY_REG(HDMI_TX_PHY_CTL_1, 0x00c),
		482	VC5_PHY_REG(HDMI_TX_PHY_CTL_2, 0x010),
		483	VC5_PHY_REG(HDMI_TX_PHY_CTL_CK, 0x014),
		484	VC5_PHY_REG(HDMI_TX_PHY_PLL_REFCLK, 0x01c),
		485	VC5_PHY_REG(HDMI_TX_PHY_PLL_POST_KDIV, 0x028),
		486	VC5_PHY_REG(HDMI_TX_PHY_PLL_VCOCLK_DIV, 0x02c),
		487	VC5_PHY_REG(HDMI_TX_PHY_PLL_CFG, 0x044),
		488	VC5_PHY_REG(HDMI_TX_PHY_TMDS_CLK_WORD_SEL, 0x054),
		489	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_0, 0x060),
		490	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_1, 0x064),
		491	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_2, 0x068),
		492	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_3, 0x06c),
		493	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_4, 0x070),
		494	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_5, 0x074),
		495	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_6, 0x078),
		496	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_7, 0x07c),
		497	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_8, 0x080),
		498	VC5_PHY_REG(HDMI_TX_PHY_PLL_RESET_CTL, 0x190),
		499	VC5_PHY_REG(HDMI_TX_PHY_PLL_POWERUP_CTL, 0x194),
		500
		501	VC5_RM_REG(HDMI_RM_CONTROL, 0x000),
		502	VC5_RM_REG(HDMI_RM_OFFSET, 0x018),
		503	VC5_RM_REG(HDMI_RM_FORMAT, 0x01c),
		504
		505	VC5_RAM_REG(HDMI_RAM_PACKET_START, 0x000),
		506
		507	VC5_CEC_REG(HDMI_CEC_CNTRL_1, 0x010),
		508	VC5_CEC_REG(HDMI_CEC_CNTRL_2, 0x014),
		509	VC5_CEC_REG(HDMI_CEC_CNTRL_3, 0x018),
		510	VC5_CEC_REG(HDMI_CEC_CNTRL_4, 0x01c),
		511	VC5_CEC_REG(HDMI_CEC_CNTRL_5, 0x020),
		512	VC5_CEC_REG(HDMI_CEC_TX_DATA_1, 0x028),
		513	VC5_CEC_REG(HDMI_CEC_TX_DATA_2, 0x02c),
		514	VC5_CEC_REG(HDMI_CEC_TX_DATA_3, 0x030),
		515	VC5_CEC_REG(HDMI_CEC_TX_DATA_4, 0x034),
		516	VC5_CEC_REG(HDMI_CEC_RX_DATA_1, 0x038),
		517	VC5_CEC_REG(HDMI_CEC_RX_DATA_2, 0x03c),
		518	VC5_CEC_REG(HDMI_CEC_RX_DATA_3, 0x040),
		519	VC5_CEC_REG(HDMI_CEC_RX_DATA_4, 0x044),
		520
		521	VC5_CSC_REG(HDMI_CSC_CTL, 0x000),
		522	VC5_CSC_REG(HDMI_CSC_12_11, 0x004),
		523	VC5_CSC_REG(HDMI_CSC_14_13, 0x008),
		524	VC5_CSC_REG(HDMI_CSC_22_21, 0x00c),
		525	VC5_CSC_REG(HDMI_CSC_24_23, 0x010),
		526	VC5_CSC_REG(HDMI_CSC_32_31, 0x014),
		527	VC5_CSC_REG(HDMI_CSC_34_33, 0x018),
		528	VC5_CSC_REG(HDMI_CSC_CHANNEL_CTL, 0x02c),
		529	};
		530
		531	static const struct vc4_hdmi_register __maybe_unused vc6_hdmi_hdmi1_fields[] = {
		532	VC4_HD_REG(HDMI_DVP_CTL, 0x0000),
		533	VC4_HD_REG(HDMI_MAI_CTL, 0x0030),
		534	VC4_HD_REG(HDMI_MAI_THR, 0x0034),
		535	VC4_HD_REG(HDMI_MAI_FMT, 0x0038),
		536	VC4_HD_REG(HDMI_MAI_DATA, 0x003c),
		537	VC4_HD_REG(HDMI_MAI_SMP, 0x0040),
		538	VC4_HD_REG(HDMI_VID_CTL, 0x0048),
		539	VC4_HD_REG(HDMI_FRAME_COUNT, 0x0064),
		540
		541	VC4_HDMI_REG(HDMI_FIFO_CTL, 0x07c),
		542	VC4_HDMI_REG(HDMI_AUDIO_PACKET_CONFIG, 0x0c0),
		543	VC4_HDMI_REG(HDMI_RAM_PACKET_CONFIG, 0x0c4),
		544	VC4_HDMI_REG(HDMI_RAM_PACKET_STATUS, 0x0cc),
		545	VC4_HDMI_REG(HDMI_CRP_CFG, 0x0d0),
		546	VC4_HDMI_REG(HDMI_CTS_0, 0x0d4),
		547	VC4_HDMI_REG(HDMI_CTS_1, 0x0d8),
		548	VC4_HDMI_REG(HDMI_SCHEDULER_CONTROL, 0x0e8),
		549	VC4_HDMI_REG(HDMI_HORZA, 0x0ec),
		550	VC4_HDMI_REG(HDMI_HORZB, 0x0f0),
		551	VC4_HDMI_REG(HDMI_VERTA0, 0x0f4),
		552	VC4_HDMI_REG(HDMI_VERTB0, 0x0f8),
		553	VC4_HDMI_REG(HDMI_VERTA1, 0x100),
		554	VC4_HDMI_REG(HDMI_VERTB1, 0x104),
		555	VC4_HDMI_REG(HDMI_MISC_CONTROL, 0x114),
		556	VC4_HDMI_REG(HDMI_MAI_CHANNEL_MAP, 0x0a4),
		557	VC4_HDMI_REG(HDMI_MAI_CONFIG, 0x0a8),
		558	VC4_HDMI_REG(HDMI_FORMAT_DET_1, 0x148),
		559	VC4_HDMI_REG(HDMI_FORMAT_DET_2, 0x14c),
		560	VC4_HDMI_REG(HDMI_FORMAT_DET_3, 0x150),
		561	VC4_HDMI_REG(HDMI_FORMAT_DET_4, 0x158),
		562	VC4_HDMI_REG(HDMI_FORMAT_DET_5, 0x15c),
		563	VC4_HDMI_REG(HDMI_FORMAT_DET_6, 0x160),
		564	VC4_HDMI_REG(HDMI_FORMAT_DET_7, 0x164),
		565	VC4_HDMI_REG(HDMI_FORMAT_DET_8, 0x168),
		566	VC4_HDMI_REG(HDMI_FORMAT_DET_9, 0x16c),
		567	VC4_HDMI_REG(HDMI_FORMAT_DET_10, 0x170),
		568	VC4_HDMI_REG(HDMI_DEEP_COLOR_CONFIG_1, 0x18c),
		569	VC4_HDMI_REG(HDMI_GCP_CONFIG, 0x194),
		570	VC4_HDMI_REG(HDMI_GCP_WORD_1, 0x198),
		571	VC4_HDMI_REG(HDMI_HOTPLUG, 0x1c8),
		572	VC4_HDMI_REG(HDMI_SCRAMBLER_CTL, 0x1e4),
		573
		574	VC5_DVP_REG(HDMI_CLOCK_STOP, 0x0bc),
		575	VC5_DVP_REG(HDMI_VEC_INTERFACE_CFG, 0x0f0),
		576	VC5_DVP_REG(HDMI_VEC_INTERFACE_XBAR, 0x0f4),
		577
		578	VC5_PHY_REG(HDMI_TX_PHY_RESET_CTL, 0x000),
		579	VC5_PHY_REG(HDMI_TX_PHY_POWERUP_CTL, 0x004),
		580	VC5_PHY_REG(HDMI_TX_PHY_CTL_0, 0x008),
		581	VC5_PHY_REG(HDMI_TX_PHY_CTL_1, 0x00c),
		582	VC5_PHY_REG(HDMI_TX_PHY_CTL_2, 0x010),
		583	VC5_PHY_REG(HDMI_TX_PHY_CTL_CK, 0x014),
		584	VC5_PHY_REG(HDMI_TX_PHY_PLL_REFCLK, 0x01c),
		585	VC5_PHY_REG(HDMI_TX_PHY_PLL_POST_KDIV, 0x028),
		586	VC5_PHY_REG(HDMI_TX_PHY_PLL_VCOCLK_DIV, 0x02c),
		587	VC5_PHY_REG(HDMI_TX_PHY_PLL_CFG, 0x044),
		588	VC5_PHY_REG(HDMI_TX_PHY_TMDS_CLK_WORD_SEL, 0x054),
		589	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_0, 0x060),
		590	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_1, 0x064),
		591	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_2, 0x068),
		592	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_3, 0x06c),
		593	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_4, 0x070),
		594	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_5, 0x074),
		595	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_6, 0x078),
		596	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_7, 0x07c),
		597	VC5_PHY_REG(HDMI_TX_PHY_PLL_MISC_8, 0x080),
		598	VC5_PHY_REG(HDMI_TX_PHY_PLL_RESET_CTL, 0x190),
		599	VC5_PHY_REG(HDMI_TX_PHY_PLL_POWERUP_CTL, 0x194),
		600
		601	VC5_RM_REG(HDMI_RM_CONTROL, 0x000),
		602	VC5_RM_REG(HDMI_RM_OFFSET, 0x018),
		603	VC5_RM_REG(HDMI_RM_FORMAT, 0x01c),
		604
		605	VC5_RAM_REG(HDMI_RAM_PACKET_START, 0x000),
		606
		607	VC5_CEC_REG(HDMI_CEC_CNTRL_1, 0x010),
		608	VC5_CEC_REG(HDMI_CEC_CNTRL_2, 0x014),
		609	VC5_CEC_REG(HDMI_CEC_CNTRL_3, 0x018),
		610	VC5_CEC_REG(HDMI_CEC_CNTRL_4, 0x01c),
		611	VC5_CEC_REG(HDMI_CEC_CNTRL_5, 0x020),
		612	VC5_CEC_REG(HDMI_CEC_TX_DATA_1, 0x028),
		613	VC5_CEC_REG(HDMI_CEC_TX_DATA_2, 0x02c),
		614	VC5_CEC_REG(HDMI_CEC_TX_DATA_3, 0x030),
		615	VC5_CEC_REG(HDMI_CEC_TX_DATA_4, 0x034),
		616	VC5_CEC_REG(HDMI_CEC_RX_DATA_1, 0x038),
		617	VC5_CEC_REG(HDMI_CEC_RX_DATA_2, 0x03c),
		618	VC5_CEC_REG(HDMI_CEC_RX_DATA_3, 0x040),
		619	VC5_CEC_REG(HDMI_CEC_RX_DATA_4, 0x044),
		620
		621	VC5_CSC_REG(HDMI_CSC_CTL, 0x000),
		622	VC5_CSC_REG(HDMI_CSC_12_11, 0x004),
		623	VC5_CSC_REG(HDMI_CSC_14_13, 0x008),
		624	VC5_CSC_REG(HDMI_CSC_22_21, 0x00c),
		625	VC5_CSC_REG(HDMI_CSC_24_23, 0x010),
		626	VC5_CSC_REG(HDMI_CSC_32_31, 0x014),
		627	VC5_CSC_REG(HDMI_CSC_34_33, 0x018),
		628	VC5_CSC_REG(HDMI_CSC_CHANNEL_CTL, 0x02c),
		629	};
		630
414	static inline	631	static inline
415	void __iomem __vc4_hdmi_get_field_base(struct vc4_hdmi hdmi,	632	void __iomem __vc4_hdmi_get_field_base(struct vc4_hdmi hdmi,
416	enum vc4_hdmi_regs reg)	633	enum vc4_hdmi_regs reg)

drivers/gpu/drm/vc4/vc4_hvs.c+691−46

@@ -67,6 +67,140 @@ static const struct debugfs_reg32 vc4_hvs_regs[] = { · 66 unmodified lines
67	VC4_REG32(SCALER_OLEDCOEF2),	67	VC4_REG32(SCALER_OLEDCOEF2),
68	};	68	};
69		69
		70	static const struct debugfs_reg32 vc6_hvs_regs[] = {
		71	VC4_REG32(SCALER6_VERSION),
		72	VC4_REG32(SCALER6_CXM_SIZE),
		73	VC4_REG32(SCALER6_LBM_SIZE),
		74	VC4_REG32(SCALER6_UBM_SIZE),
		75	VC4_REG32(SCALER6_COBA_SIZE),
		76	VC4_REG32(SCALER6_COB_SIZE),
		77	VC4_REG32(SCALER6_CONTROL),
		78	VC4_REG32(SCALER6_FETCHER_STATUS),
		79	VC4_REG32(SCALER6_FETCH_STATUS),
		80	VC4_REG32(SCALER6_HANDLE_ERROR),
		81	VC4_REG32(SCALER6_DISP0_CTRL0),
		82	VC4_REG32(SCALER6_DISP0_CTRL1),
		83	VC4_REG32(SCALER6_DISP0_BGND),
		84	VC4_REG32(SCALER6_DISP0_LPTRS),
		85	VC4_REG32(SCALER6_DISP0_COB),
		86	VC4_REG32(SCALER6_DISP0_STATUS),
		87	VC4_REG32(SCALER6_DISP0_DL),
		88	VC4_REG32(SCALER6_DISP0_RUN),
		89	VC4_REG32(SCALER6_DISP1_CTRL0),
		90	VC4_REG32(SCALER6_DISP1_CTRL1),
		91	VC4_REG32(SCALER6_DISP1_BGND),
		92	VC4_REG32(SCALER6_DISP1_LPTRS),
		93	VC4_REG32(SCALER6_DISP1_COB),
		94	VC4_REG32(SCALER6_DISP1_STATUS),
		95	VC4_REG32(SCALER6_DISP1_DL),
		96	VC4_REG32(SCALER6_DISP1_RUN),
		97	VC4_REG32(SCALER6_DISP2_CTRL0),
		98	VC4_REG32(SCALER6_DISP2_CTRL1),
		99	VC4_REG32(SCALER6_DISP2_BGND),
		100	VC4_REG32(SCALER6_DISP2_LPTRS),
		101	VC4_REG32(SCALER6_DISP2_COB),
		102	VC4_REG32(SCALER6_DISP2_STATUS),
		103	VC4_REG32(SCALER6_DISP2_DL),
		104	VC4_REG32(SCALER6_DISP2_RUN),
		105	VC4_REG32(SCALER6_EOLN),
		106	VC4_REG32(SCALER6_DL_STATUS),
		107	VC4_REG32(SCALER6_BFG_MISC),
		108	VC4_REG32(SCALER6_QOS0),
		109	VC4_REG32(SCALER6_PROF0),
		110	VC4_REG32(SCALER6_QOS1),
		111	VC4_REG32(SCALER6_PROF1),
		112	VC4_REG32(SCALER6_QOS2),
		113	VC4_REG32(SCALER6_PROF2),
		114	VC4_REG32(SCALER6_PRI_MAP0),
		115	VC4_REG32(SCALER6_PRI_MAP1),
		116	VC4_REG32(SCALER6_HISTCTRL),
		117	VC4_REG32(SCALER6_HISTBIN0),
		118	VC4_REG32(SCALER6_HISTBIN1),
		119	VC4_REG32(SCALER6_HISTBIN2),
		120	VC4_REG32(SCALER6_HISTBIN3),
		121	VC4_REG32(SCALER6_HISTBIN4),
		122	VC4_REG32(SCALER6_HISTBIN5),
		123	VC4_REG32(SCALER6_HISTBIN6),
		124	VC4_REG32(SCALER6_HISTBIN7),
		125	VC4_REG32(SCALER6_HDR_CFG_REMAP),
		126	VC4_REG32(SCALER6_COL_SPACE),
		127	VC4_REG32(SCALER6_HVS_ID),
		128	VC4_REG32(SCALER6_CFC1),
		129	VC4_REG32(SCALER6_DISP_UPM_ISO0),
		130	VC4_REG32(SCALER6_DISP_UPM_ISO1),
		131	VC4_REG32(SCALER6_DISP_UPM_ISO2),
		132	VC4_REG32(SCALER6_DISP_LBM_ISO0),
		133	VC4_REG32(SCALER6_DISP_LBM_ISO1),
		134	VC4_REG32(SCALER6_DISP_LBM_ISO2),
		135	VC4_REG32(SCALER6_DISP_COB_ISO0),
		136	VC4_REG32(SCALER6_DISP_COB_ISO1),
		137	VC4_REG32(SCALER6_DISP_COB_ISO2),
		138	VC4_REG32(SCALER6_BAD_COB),
		139	VC4_REG32(SCALER6_BAD_LBM),
		140	VC4_REG32(SCALER6_BAD_UPM),
		141	VC4_REG32(SCALER6_BAD_AXI),
		142	};
		143
		144	static const struct debugfs_reg32 vc6_d_hvs_regs[] = {
		145	VC4_REG32(SCALER6D_VERSION),
		146	VC4_REG32(SCALER6D_CXM_SIZE),
		147	VC4_REG32(SCALER6D_LBM_SIZE),
		148	VC4_REG32(SCALER6D_UBM_SIZE),
		149	VC4_REG32(SCALER6D_COBA_SIZE),
		150	VC4_REG32(SCALER6D_COB_SIZE),
		151	VC4_REG32(SCALER6D_CONTROL),
		152	VC4_REG32(SCALER6D_FETCHER_STATUS),
		153	VC4_REG32(SCALER6D_FETCH_STATUS),
		154	VC4_REG32(SCALER6D_HANDLE_ERROR),
		155	VC4_REG32(SCALER6D_DISP0_CTRL0),
		156	VC4_REG32(SCALER6D_DISP0_CTRL1),
		157	VC4_REG32(SCALER6D_DISP0_BGND0),
		158	VC4_REG32(SCALER6D_DISP0_BGND1),
		159	VC4_REG32(SCALER6D_DISP0_LPTRS),
		160	VC4_REG32(SCALER6D_DISP0_COB),
		161	VC4_REG32(SCALER6D_DISP0_STATUS),
		162	VC4_REG32(SCALER6D_DISP0_DL),
		163	VC4_REG32(SCALER6D_DISP0_RUN),
		164	VC4_REG32(SCALER6D_DISP1_CTRL0),
		165	VC4_REG32(SCALER6D_DISP1_CTRL1),
		166	VC4_REG32(SCALER6D_DISP1_BGND0),
		167	VC4_REG32(SCALER6D_DISP1_BGND1),
		168	VC4_REG32(SCALER6D_DISP1_LPTRS),
		169	VC4_REG32(SCALER6D_DISP1_COB),
		170	VC4_REG32(SCALER6D_DISP1_STATUS),
		171	VC4_REG32(SCALER6D_DISP1_DL),
		172	VC4_REG32(SCALER6D_DISP1_RUN),
		173	VC4_REG32(SCALER6D_DISP2_CTRL0),
		174	VC4_REG32(SCALER6D_DISP2_CTRL1),
		175	VC4_REG32(SCALER6D_DISP2_BGND0),
		176	VC4_REG32(SCALER6D_DISP2_BGND1),
		177	VC4_REG32(SCALER6D_DISP2_LPTRS),
		178	VC4_REG32(SCALER6D_DISP2_COB),
		179	VC4_REG32(SCALER6D_DISP2_STATUS),
		180	VC4_REG32(SCALER6D_DISP2_DL),
		181	VC4_REG32(SCALER6D_DISP2_RUN),
		182	VC4_REG32(SCALER6D_EOLN),
		183	VC4_REG32(SCALER6D_DL_STATUS),
		184	VC4_REG32(SCALER6D_QOS0),
		185	VC4_REG32(SCALER6D_PROF0),
		186	VC4_REG32(SCALER6D_QOS1),
		187	VC4_REG32(SCALER6D_PROF1),
		188	VC4_REG32(SCALER6D_QOS2),
		189	VC4_REG32(SCALER6D_PROF2),
		190	VC4_REG32(SCALER6D_PRI_MAP0),
		191	VC4_REG32(SCALER6D_PRI_MAP1),
		192	VC4_REG32(SCALER6D_HISTCTRL),
		193	VC4_REG32(SCALER6D_HISTBIN0),
		194	VC4_REG32(SCALER6D_HISTBIN1),
		195	VC4_REG32(SCALER6D_HISTBIN2),
		196	VC4_REG32(SCALER6D_HISTBIN3),
		197	VC4_REG32(SCALER6D_HISTBIN4),
		198	VC4_REG32(SCALER6D_HISTBIN5),
		199	VC4_REG32(SCALER6D_HISTBIN6),
		200	VC4_REG32(SCALER6D_HISTBIN7),
		201	VC4_REG32(SCALER6D_HVS_ID),
		202	};
		203
70	void vc4_hvs_dump_state(struct vc4_hvs *hvs)	204	void vc4_hvs_dump_state(struct vc4_hvs *hvs)
71	{	205	{
72	struct drm_device *drm = &hvs->vc4->base;	206	struct drm_device *drm = &hvs->vc4->base;
@@ -145,6 +279,76 @@ static int vc4_hvs_debugfs_dlist(struct seq_file m, void data) · 72 unmodified lines
145	return 0;	279	return 0;
146	}	280	}
147		281
		282	static int vc6_hvs_debugfs_dlist(struct seq_file m, void data)
		283	{
		284	struct drm_info_node *node = m->private;
		285	struct drm_device *dev = node->minor->dev;
		286	struct vc4_dev *vc4 = to_vc4_dev(dev);
		287	struct vc4_hvs *hvs = vc4->hvs;
		288	struct drm_printer p = drm_seq_file_printer(m);
		289	unsigned int dlist_mem_size = hvs->dlist_mem_size;
		290	unsigned int next_entry_start;
		291	unsigned int i;
		292
		293	for (i = 0; i < SCALER_CHANNELS_COUNT; i++) {
		294	unsigned int active_dlist, dispstat;
		295	unsigned int j;
		296
		297	dispstat = VC4_GET_FIELD(HVS_READ(SCALER6_DISPX_STATUS(i)),
		298	SCALER6_DISPX_STATUS_MODE);
		299	if (dispstat == SCALER6_DISPX_STATUS_MODE_DISABLED \|\|
		300	dispstat == SCALER6_DISPX_STATUS_MODE_EOF) {
		301	drm_printf(&p, "HVS chan %u disabled\n", i);
		302	continue;
		303	}
		304
		305	drm_printf(&p, "HVS chan %u:\n", i);
		306
		307	active_dlist = VC4_GET_FIELD(HVS_READ(SCALER6_DISPX_DL(i)),
		308	SCALER6_DISPX_DL_LACT);
		309	next_entry_start = 0;
		310
		311	for (j = active_dlist; j < dlist_mem_size; j++) {
		312	u32 dlist_word;
		313
		314	dlist_word = readl((u32 __iomem *)vc4->hvs->dlist + j);
		315	drm_printf(&p, "dlist: %02d: 0x%08x\n", j,
		316	dlist_word);
		317	if (!next_entry_start \|\|
		318	next_entry_start == j) {
		319	if (dlist_word & SCALER_CTL0_END)
		320	break;
		321	next_entry_start = j +
		322	VC4_GET_FIELD(dlist_word,
		323	SCALER_CTL0_SIZE);
		324	}
		325	}
		326	}
		327
		328	return 0;
		329	}
		330
		331	static int vc6_hvs_debugfs_upm_allocs(struct seq_file m, void data)
		332	{
		333	struct drm_debugfs_entry *entry = m->private;
		334	struct drm_device *dev = entry->dev;
		335	struct vc4_dev *vc4 = to_vc4_dev(dev);
		336	struct vc4_hvs *hvs = vc4->hvs;
		337	struct drm_printer p = drm_seq_file_printer(m);
		338	struct vc4_upm_refcounts *refcount;
		339	unsigned int i;
		340
		341	drm_printf(&p, "UPM Handles:\n");
		342	for (i = 1; i <= VC4_NUM_UPM_HANDLES; i++) {
		343	refcount = &hvs->upm_refcounts[i];
		344	drm_printf(&p, "handle %u: refcount %u, size %zu [%08llx + %08llx]\n",
		345	i, refcount_read(&refcount->refcount), refcount->size,
		346	refcount->upm.start, refcount->upm.size);
		347	}
		348
		349	return 0;
		350	}
		351
148	/* The filter kernel is composed of dwords each containing 3 9-bit	352	/* The filter kernel is composed of dwords each containing 3 9-bit
149	* signed integers packed next to each other.	353	* signed integers packed next to each other.
150	*/	354	*/
@@ -215,12 +419,15 @@ static int vc4_hvs_upload_linear_kernel(struct vc4_hvs *hvs, · 64 unmodified lines
215	static void vc4_hvs_lut_load(struct vc4_hvs *hvs,	419	static void vc4_hvs_lut_load(struct vc4_hvs *hvs,
216	struct vc4_crtc *vc4_crtc)	420	struct vc4_crtc *vc4_crtc)
217	{	421	{
218	struct drm_device *drm = &hvs->vc4->base;	422	struct vc4_dev *vc4 = hvs->vc4;
		423	struct drm_device *drm = &vc4->base;
219	struct drm_crtc *crtc = &vc4_crtc->base;	424	struct drm_crtc *crtc = &vc4_crtc->base;
220	struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc->state);	425	struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc->state);
221	int idx;	426	int idx;
222	u32 i;	427	u32 i;
223		428
		429	WARN_ON_ONCE(vc4->gen > VC4_GEN_5);
		430
224	if (!drm_dev_enter(drm, &idx))	431	if (!drm_dev_enter(drm, &idx))
225	return;	432	return;
226		433
@@ -265,26 +472,57 @@ static void vc4_hvs_update_gamma_lut(struct vc4_hvs *hvs, · 38 unmodified lines
265		472
266	u8 vc4_hvs_get_fifo_frame_count(struct vc4_hvs *hvs, unsigned int fifo)	473	u8 vc4_hvs_get_fifo_frame_count(struct vc4_hvs *hvs, unsigned int fifo)
267	{	474	{
268	struct drm_device *drm = &hvs->vc4->base;	475	struct vc4_dev *vc4 = hvs->vc4;
		476	struct drm_device *drm = &vc4->base;
269	u8 field = 0;	477	u8 field = 0;
270	int idx;	478	int idx;
271		479
		480	WARN_ON_ONCE(vc4->gen > VC4_GEN_6_D);
		481
272	if (!drm_dev_enter(drm, &idx))	482	if (!drm_dev_enter(drm, &idx))
273	return 0;	483	return 0;
274		484
275	switch (fifo) {	485	switch (vc4->gen) {
276	case 0:	486	case VC4_GEN_6_C:
277	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT1),	487	case VC4_GEN_6_D:
278	SCALER_DISPSTAT1_FRCNT0);	488	field = VC4_GET_FIELD(HVS_READ(SCALER6_DISPX_STATUS(fifo)),
		489	SCALER6_DISPX_STATUS_FRCNT);
279	break;	490	break;
280	case 1:	491	case VC4_GEN_5:
281	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT1),	492	switch (fifo) {
282	SCALER_DISPSTAT1_FRCNT1);	493	case 0:
		494	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT1),
		495	SCALER5_DISPSTAT1_FRCNT0);
		496	break;
		497	case 1:
		498	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT1),
		499	SCALER5_DISPSTAT1_FRCNT1);
		500	break;
		501	case 2:
		502	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT2),
		503	SCALER5_DISPSTAT2_FRCNT2);
		504	break;
		505	}
283	break;	506	break;
284	case 2:	507	case VC4_GEN_4:
285	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT2),	508	switch (fifo) {
286	SCALER_DISPSTAT2_FRCNT2);	509	case 0:
		510	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT1),
		511	SCALER_DISPSTAT1_FRCNT0);
		512	break;
		513	case 1:
		514	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT1),
		515	SCALER_DISPSTAT1_FRCNT1);
		516	break;
		517	case 2:
		518	field = VC4_GET_FIELD(HVS_READ(SCALER_DISPSTAT2),
		519	SCALER_DISPSTAT2_FRCNT2);
		520	break;
		521	}
287	break;	522	break;
		523	default:
		524	drm_err(drm, "Unknown VC4 generation: %d", vc4->gen);
		525	return 0;
288	}	526	}
289		527
290	drm_dev_exit(idx);	528	drm_dev_exit(idx);
@@ -297,6 +535,8 @@ int vc4_hvs_get_fifo_from_output(struct vc4_hvs *hvs, unsigned int output) · 6 unmodified lines
297	u32 reg;	535	u32 reg;
298	int ret;	536	int ret;
299		537
		538	WARN_ON_ONCE(vc4->gen > VC4_GEN_6_D);
		539
300	switch (vc4->gen) {	540	switch (vc4->gen) {
301	case VC4_GEN_4:	541	case VC4_GEN_4:
302	return output;	542	return output;
@@ -352,6 +592,24 @@ int vc4_hvs_get_fifo_from_output(struct vc4_hvs *hvs, unsigned int output) · 49 unmodified lines
352	return -EPIPE;	592	return -EPIPE;
353	}	593	}
354		594
		595	case VC4_GEN_6_C:
		596	case VC4_GEN_6_D:
		597	switch (output) {
		598	case 0:
		599	return 0;
		600
		601	case 2:
		602	return 2;
		603
		604	case 1:
		605	case 3:
		606	case 4:
		607	return 1;
		608
		609	default:
		610	return -EPIPE;
		611	}
		612
355	default:	613	default:
356	return -EPIPE;	614	return -EPIPE;
357	}	615	}
@@ -370,6 +628,8 @@ static int vc4_hvs_init_channel(struct vc4_hvs hvs, struct drm_crtc crtc, · 12 unmodified lines
370	u32 dispctrl;	628	u32 dispctrl;
371	int idx;	629	int idx;
372		630
		631	WARN_ON_ONCE(vc4->gen > VC4_GEN_5);
		632
373	if (!drm_dev_enter(drm, &idx))	633	if (!drm_dev_enter(drm, &idx))
374	return -ENODEV;	634	return -ENODEV;
375		635
@@ -420,11 +680,50 @@ static int vc4_hvs_init_channel(struct vc4_hvs hvs, struct drm_crtc crtc, · 44 unmodified lines
420	return 0;	680	return 0;
421	}	681	}
422		682
423	void vc4_hvs_stop_channel(struct vc4_hvs *hvs, unsigned int chan)	683	static int vc6_hvs_init_channel(struct vc4_hvs hvs, struct drm_crtc crtc,
		684	struct drm_display_mode *mode, bool oneshot)
424	{	685	{
425	struct drm_device *drm = &hvs->vc4->base;	686	struct vc4_dev *vc4 = hvs->vc4;
		687	struct drm_device *drm = &vc4->base;
		688	struct vc4_crtc_state *vc4_crtc_state = to_vc4_crtc_state(crtc->state);
		689	unsigned int chan = vc4_crtc_state->assigned_channel;
		690	bool interlace = mode->flags & DRM_MODE_FLAG_INTERLACE;
		691	u32 disp_ctrl1;
		692	int idx;
		693
		694	WARN_ON_ONCE(vc4->gen < VC4_GEN_6_C);
		695
		696	if (!drm_dev_enter(drm, &idx))
		697	return -ENODEV;
		698
		699	HVS_WRITE(SCALER6_DISPX_CTRL0(chan), SCALER6_DISPX_CTRL0_RESET);
		700
		701	disp_ctrl1 = HVS_READ(SCALER6_DISPX_CTRL1(chan));
		702	disp_ctrl1 &= ~SCALER6_DISPX_CTRL1_INTLACE;
		703	HVS_WRITE(SCALER6_DISPX_CTRL1(chan),
		704	disp_ctrl1 \| (interlace ? SCALER6_DISPX_CTRL1_INTLACE : 0));
		705
		706	HVS_WRITE(SCALER6_DISPX_CTRL0(chan),
		707	SCALER6_DISPX_CTRL0_ENB \|
		708	VC4_SET_FIELD(mode->hdisplay - 1,
		709	SCALER6_DISPX_CTRL0_FWIDTH) \|
		710	(oneshot ? SCALER6_DISPX_CTRL0_ONESHOT : 0) \|
		711	VC4_SET_FIELD(mode->vdisplay - 1,
		712	SCALER6_DISPX_CTRL0_LINES));
		713
		714	drm_dev_exit(idx);
		715
		716	return 0;
		717	}
		718
		719	static void __vc4_hvs_stop_channel(struct vc4_hvs *hvs, unsigned int chan)
		720	{
		721	struct vc4_dev *vc4 = hvs->vc4;
		722	struct drm_device *drm = &vc4->base;
426	int idx;	723	int idx;
427		724
		725	WARN_ON_ONCE(vc4->gen > VC4_GEN_5);
		726
428	if (!drm_dev_enter(drm, &idx))	727	if (!drm_dev_enter(drm, &idx))
429	return;	728	return;
430		729
@@ -449,6 +748,44 @@ out: · 18 unmodified lines
449	drm_dev_exit(idx);	748	drm_dev_exit(idx);
450	}	749	}
451		750
		751	static void __vc6_hvs_stop_channel(struct vc4_hvs *hvs, unsigned int chan)
		752	{
		753	struct vc4_dev *vc4 = hvs->vc4;
		754	struct drm_device *drm = &vc4->base;
		755	int idx;
		756
		757	WARN_ON_ONCE(vc4->gen < VC4_GEN_6_C);
		758
		759	if (!drm_dev_enter(drm, &idx))
		760	return;
		761
		762	if (!(HVS_READ(SCALER6_DISPX_CTRL0(chan)) & SCALER6_DISPX_CTRL0_ENB))
		763	goto out;
		764
		765	HVS_WRITE(SCALER6_DISPX_CTRL0(chan),
		766	HVS_READ(SCALER6_DISPX_CTRL0(chan)) \| SCALER6_DISPX_CTRL0_RESET);
		767
		768	HVS_WRITE(SCALER6_DISPX_CTRL0(chan),
		769	HVS_READ(SCALER6_DISPX_CTRL0(chan)) & ~SCALER6_DISPX_CTRL0_ENB);
		770
		771	WARN_ON_ONCE(VC4_GET_FIELD(HVS_READ(SCALER6_DISPX_STATUS(chan)),
		772	SCALER6_DISPX_STATUS_MODE) !=
		773	SCALER6_DISPX_STATUS_MODE_DISABLED);
		774
		775	out:
		776	drm_dev_exit(idx);
		777	}
		778
		779	void vc4_hvs_stop_channel(struct vc4_hvs *hvs, unsigned int chan)
		780	{
		781	struct vc4_dev *vc4 = hvs->vc4;
		782
		783	if (vc4->gen >= VC4_GEN_6_C)
		784	__vc6_hvs_stop_channel(hvs, chan);
		785	else
		786	__vc4_hvs_stop_channel(hvs, chan);
		787	}
		788
452	int vc4_hvs_atomic_check(struct drm_crtc crtc, struct drm_atomic_state state)	789	int vc4_hvs_atomic_check(struct drm_crtc crtc, struct drm_atomic_state state)
453	{	790	{
454	struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);	791	struct drm_crtc_state *crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
@@ -505,8 +842,13 @@ static void vc4_hvs_install_dlist(struct drm_crtc *crtc) · 50 unmodified lines
505	if (!drm_dev_enter(dev, &idx))	842	if (!drm_dev_enter(dev, &idx))
506	return;	843	return;
507		844
508	HVS_WRITE(SCALER_DISPLISTX(vc4_state->assigned_channel),	845	if (vc4->gen >= VC4_GEN_6_C)
509	vc4_state->mm.start);	846	HVS_WRITE(SCALER6_DISPX_LPTRS(vc4_state->assigned_channel),
		847	VC4_SET_FIELD(vc4_state->mm.start,
		848	SCALER6_DISPX_LPTRS_HEADE));
		849	else
		850	HVS_WRITE(SCALER_DISPLISTX(vc4_state->assigned_channel),
		851	vc4_state->mm.start);
510		852
511	drm_dev_exit(idx);	853	drm_dev_exit(idx);
512	}	854	}
@@ -561,7 +903,11 @@ void vc4_hvs_atomic_enable(struct drm_crtc *crtc, · 48 unmodified lines
561		903
562	vc4_hvs_install_dlist(crtc);	904	vc4_hvs_install_dlist(crtc);
563	vc4_hvs_update_dlist(crtc);	905	vc4_hvs_update_dlist(crtc);
564	vc4_hvs_init_channel(vc4->hvs, crtc, mode, oneshot);	906
		907	if (vc4->gen >= VC4_GEN_6_C)
		908	vc6_hvs_init_channel(vc4->hvs, crtc, mode, oneshot);
		909	else
		910	vc4_hvs_init_channel(vc4->hvs, crtc, mode, oneshot);
565	}	911	}
566		912
567	void vc4_hvs_atomic_disable(struct drm_crtc *crtc,	913	void vc4_hvs_atomic_disable(struct drm_crtc *crtc,
@@ -590,13 +936,15 @@ void vc4_hvs_atomic_flush(struct drm_crtc *crtc, · 22 unmodified lines
590	struct drm_plane *plane;	936	struct drm_plane *plane;
591	struct vc4_plane_state *vc4_plane_state;	937	struct vc4_plane_state *vc4_plane_state;
592	bool debug_dump_regs = false;	938	bool debug_dump_regs = false;
593	bool enable_bg_fill = false;	939	bool enable_bg_fill = true;
594	u32 __iomem *dlist_start = vc4->hvs->dlist + vc4_state->mm.start;	940	u32 __iomem *dlist_start = vc4->hvs->dlist + vc4_state->mm.start;
595	u32 __iomem *dlist_next = dlist_start;	941	u32 __iomem *dlist_next = dlist_start;
596	unsigned int zpos = 0;	942	unsigned int zpos = 0;
597	bool found = false;	943	bool found = false;
598	int idx;	944	int idx;
599		945
		946	WARN_ON_ONCE(vc4->gen > VC4_GEN_6_D);
		947
600	if (!drm_dev_enter(dev, &idx)) {	948	if (!drm_dev_enter(dev, &idx)) {
601	vc4_crtc_send_vblank(crtc);	949	vc4_crtc_send_vblank(crtc);
602	return;	950	return;
@@ -645,13 +993,26 @@ void vc4_hvs_atomic_flush(struct drm_crtc *crtc, · 42 unmodified lines
645		993
646	WARN_ON_ONCE(dlist_next - dlist_start != vc4_state->mm.size);	994	WARN_ON_ONCE(dlist_next - dlist_start != vc4_state->mm.size);
647		995
648	if (enable_bg_fill)	996	if (vc4->gen >= VC4_GEN_6_C) {
649	/* This sets a black background color fill, as is the case	997	/* This sets a black background color fill, as is the case
650	* with other DRM drivers.	998	* with other DRM drivers.
651	*/	999	*/
		1000	if (enable_bg_fill)
		1001	HVS_WRITE(SCALER6_DISPX_CTRL1(channel),
		1002	HVS_READ(SCALER6_DISPX_CTRL1(channel)) \|
		1003	SCALER6_DISPX_CTRL1_BGENB);
		1004	else
		1005	HVS_WRITE(SCALER6_DISPX_CTRL1(channel),
		1006	HVS_READ(SCALER6_DISPX_CTRL1(channel)) &
		1007	~SCALER6_DISPX_CTRL1_BGENB);
		1008	} else {
		1009	/* we can actually run with a lower core clock when background
		1010	* fill is enabled on VC4_GEN_5 so leave it enabled always.
		1011	*/
652	HVS_WRITE(SCALER_DISPBKGNDX(channel),	1012	HVS_WRITE(SCALER_DISPBKGNDX(channel),
653	HVS_READ(SCALER_DISPBKGNDX(channel)) \|	1013	HVS_READ(SCALER_DISPBKGNDX(channel)) \|
654	SCALER_DISPBKGND_FILL);	1014	SCALER_DISPBKGND_FILL);
		1015	}
655		1016
656	/* Only update DISPLIST if the CRTC was already running and is not	1017	/* Only update DISPLIST if the CRTC was already running and is not
657	* being disabled.	1018	* being disabled.
@@ -668,6 +1029,8 @@ void vc4_hvs_atomic_flush(struct drm_crtc *crtc, · 10 unmodified lines
668	if (crtc->state->color_mgmt_changed) {	1029	if (crtc->state->color_mgmt_changed) {
669	u32 dispbkgndx = HVS_READ(SCALER_DISPBKGNDX(channel));	1030	u32 dispbkgndx = HVS_READ(SCALER_DISPBKGNDX(channel));
670		1031
		1032	WARN_ON_ONCE(vc4->gen > VC4_GEN_5);
		1033
671	if (crtc->state->gamma_lut) {	1034	if (crtc->state->gamma_lut) {
672	vc4_hvs_update_gamma_lut(hvs, vc4_crtc);	1035	vc4_hvs_update_gamma_lut(hvs, vc4_crtc);
673	dispbkgndx \|= SCALER_DISPBKGND_GAMMA;	1036	dispbkgndx \|= SCALER_DISPBKGND_GAMMA;
@@ -697,6 +1060,8 @@ void vc4_hvs_mask_underrun(struct vc4_hvs *hvs, int channel) · 23 unmodified lines
697	u32 dispctrl;	1060	u32 dispctrl;
698	int idx;	1061	int idx;
699		1062
		1063	WARN_ON(vc4->gen > VC4_GEN_5);
		1064
700	if (!drm_dev_enter(drm, &idx))	1065	if (!drm_dev_enter(drm, &idx))
701	return;	1066	return;
702		1067
@@ -717,6 +1082,8 @@ void vc4_hvs_unmask_underrun(struct vc4_hvs *hvs, int channel) · 14 unmodified lines
717	u32 dispctrl;	1082	u32 dispctrl;
718	int idx;	1083	int idx;
719		1084
		1085	WARN_ON(vc4->gen > VC4_GEN_5);
		1086
720	if (!drm_dev_enter(drm, &idx))	1087	if (!drm_dev_enter(drm, &idx))
721	return;	1088	return;
722		1089
@@ -751,6 +1118,8 @@ static irqreturn_t vc4_hvs_irq_handler(int irq, void *data) · 28 unmodified lines
751	u32 status;	1118	u32 status;
752	u32 dspeislur;	1119	u32 dspeislur;
753		1120
		1121	WARN_ON(vc4->gen > VC4_GEN_5);
		1122
754	/*	1123	/*
755	* NOTE: We don't need to protect the register access using	1124	* NOTE: We don't need to protect the register access using
756	* drm_dev_enter() there because the interrupt handler lifetime	1125	* drm_dev_enter() there because the interrupt handler lifetime
@@ -802,7 +1171,12 @@ int vc4_hvs_debugfs_init(struct drm_minor *minor) · 45 unmodified lines
802	minor->debugfs_root,	1171	minor->debugfs_root,
803	&vc4->load_tracker_enabled);	1172	&vc4->load_tracker_enabled);
804		1173
805	drm_debugfs_add_file(drm, "hvs_dlists", vc4_hvs_debugfs_dlist, NULL);	1174	if (vc4->gen >= VC4_GEN_6_C) {
		1175	drm_debugfs_add_file(drm, "hvs_dlists", vc6_hvs_debugfs_dlist, NULL);
		1176	drm_debugfs_add_file(drm, "hvs_upm", vc6_hvs_debugfs_upm_allocs, NULL);
		1177	} else {
		1178	drm_debugfs_add_file(drm, "hvs_dlists", vc4_hvs_debugfs_dlist, NULL);
		1179	}
806		1180
807	drm_debugfs_add_file(drm, "hvs_underrun", vc4_hvs_debugfs_underrun, NULL);	1181	drm_debugfs_add_file(drm, "hvs_underrun", vc4_hvs_debugfs_underrun, NULL);
808		1182
@@ -817,6 +1191,10 @@ struct vc4_hvs __vc4_hvs_alloc(struct vc4_dev vc4, · 8 unmodified lines
817	{	1191	{
818	struct drm_device *drm = &vc4->base;	1192	struct drm_device *drm = &vc4->base;
819	struct vc4_hvs *hvs;	1193	struct vc4_hvs *hvs;
		1194	unsigned int dlist_start;
		1195	size_t dlist_size;
		1196	size_t lbm_size;
		1197	unsigned int i;
820		1198
821	hvs = drmm_kzalloc(drm, sizeof(*hvs), GFP_KERNEL);	1199	hvs = drmm_kzalloc(drm, sizeof(*hvs), GFP_KERNEL);
822	if (!hvs)	1200	if (!hvs)
@@ -828,27 +1206,94 @@ struct vc4_hvs __vc4_hvs_alloc(struct vc4_dev vc4, · 5 unmodified lines
828		1206
829	spin_lock_init(&hvs->mm_lock);	1207	spin_lock_init(&hvs->mm_lock);
830		1208
831	/* Set up the HVS display list memory manager. We never	1209	switch (vc4->gen) {
832	* overwrite the setup from the bootloader (just 128b out of	1210	case VC4_GEN_4:
833	* our 16K), since we don't want to scramble the screen when	1211	case VC4_GEN_5:
834	* transitioning from the firmware's boot setup to runtime.	1212	/* Set up the HVS display list memory manager. We never
835	*/	1213	* overwrite the setup from the bootloader (just 128b
836	hvs->dlist_mem_size = (SCALER_DLIST_SIZE >> 2) - HVS_BOOTLOADER_DLIST_END;	1214	* out of our 16K), since we don't want to scramble the
837	drm_mm_init(&hvs->dlist_mm,	1215	* screen when transitioning from the firmware's boot
838	HVS_BOOTLOADER_DLIST_END,	1216	* setup to runtime.
839	hvs->dlist_mem_size);	1217	*/
		1218	dlist_start = HVS_BOOTLOADER_DLIST_END;
		1219	dlist_size = (SCALER_DLIST_SIZE >> 2) - HVS_BOOTLOADER_DLIST_END;
		1220	break;
		1221
		1222	case VC4_GEN_6_C:
		1223	case VC4_GEN_6_D:
		1224	dlist_start = HVS_BOOTLOADER_DLIST_END;
		1225
		1226	/*
		1227	* If we are running a test, it means that we can't
		1228	* access a register. Use a plausible size then.
		1229	*/
		1230	if (!kunit_get_current_test())
		1231	dlist_size = HVS_READ(SCALER6_CXM_SIZE);
		1232	else
		1233	dlist_size = 4096;
		1234
		1235	for (i = 0; i < VC4_NUM_UPM_HANDLES; i++) {
		1236	refcount_set(&hvs->upm_refcounts[i].refcount, 0);
		1237	hvs->upm_refcounts[i].hvs = hvs;
		1238	}
		1239
		1240	break;
		1241
		1242	default:
		1243	drm_err(drm, "Unknown VC4 generation: %d", vc4->gen);
		1244	return ERR_PTR(-ENODEV);
		1245	}
		1246
		1247	drm_mm_init(&hvs->dlist_mm, dlist_start, dlist_size);
		1248
		1249	hvs->dlist_mem_size = dlist_size;
840		1250
841	/* Set up the HVS LBM memory manager. We could have some more	1251	/* Set up the HVS LBM memory manager. We could have some more
842	* complicated data structure that allowed reuse of LBM areas	1252	* complicated data structure that allowed reuse of LBM areas
843	* between planes when they don't overlap on the screen, but	1253	* between planes when they don't overlap on the screen, but
844	* for now we just allocate globally.	1254	* for now we just allocate globally.
845	*/	1255	*/
846	if (vc4->gen == VC4_GEN_4)	1256
		1257	switch (vc4->gen) {
		1258	case VC4_GEN_4:
847	/* 48k words of 2x12-bit pixels */	1259	/* 48k words of 2x12-bit pixels */
848	drm_mm_init(&hvs->lbm_mm, 0, 48 * 1024);	1260	lbm_size = 48 * SZ_1K;
849	else	1261	break;
		1262
		1263	case VC4_GEN_5:
850	/* 60k words of 4x12-bit pixels */	1264	/* 60k words of 4x12-bit pixels */
851	drm_mm_init(&hvs->lbm_mm, 0, 60 * 1024);	1265	lbm_size = 60 * SZ_1K;
		1266	break;
		1267
		1268	case VC4_GEN_6_C:
		1269	case VC4_GEN_6_D:
		1270	/*
		1271	* If we are running a test, it means that we can't
		1272	* access a register. Use a plausible size then.
		1273	*/
		1274	lbm_size = 1024;
		1275	break;
		1276
		1277	default:
		1278	drm_err(drm, "Unknown VC4 generation: %d", vc4->gen);
		1279	return ERR_PTR(-ENODEV);
		1280	}
		1281
		1282	drm_mm_init(&hvs->lbm_mm, 0, lbm_size);
		1283
		1284	if (vc4->gen >= VC4_GEN_6_C) {
		1285	ida_init(&hvs->upm_handles);
		1286
		1287	/*
		1288	* NOTE: On BCM2712, the size can also be read through
		1289	* the SCALER_UBM_SIZE register. We would need to do a
		1290	* register access though, which we can't do with kunit
		1291	* that also uses this function to create its mock
		1292	* device.
		1293	*/
		1294	drm_mm_init(&hvs->upm_mm, 0, 1024 * HVS_UBM_WORD_SIZE);
		1295	}
		1296
852		1297
853	vc4->hvs = hvs;	1298	vc4->hvs = hvs;
854		1299
@@ -945,10 +1390,150 @@ static int vc4_hvs_hw_init(struct vc4_hvs *hvs) · 90 unmodified lines
945	return 0;	1390	return 0;
946	}	1391	}
947		1392
		1393	#define CFC1_N_NL_CSC_CTRL(x) (0xa000 + ((x) * 0x3000))
		1394	#define CFC1_N_MA_CSC_COEFF_C00(x) (0xa008 + ((x) * 0x3000))
		1395	#define CFC1_N_MA_CSC_COEFF_C01(x) (0xa00c + ((x) * 0x3000))
		1396	#define CFC1_N_MA_CSC_COEFF_C02(x) (0xa010 + ((x) * 0x3000))
		1397	#define CFC1_N_MA_CSC_COEFF_C03(x) (0xa014 + ((x) * 0x3000))
		1398	#define CFC1_N_MA_CSC_COEFF_C04(x) (0xa018 + ((x) * 0x3000))
		1399	#define CFC1_N_MA_CSC_COEFF_C10(x) (0xa01c + ((x) * 0x3000))
		1400	#define CFC1_N_MA_CSC_COEFF_C11(x) (0xa020 + ((x) * 0x3000))
		1401	#define CFC1_N_MA_CSC_COEFF_C12(x) (0xa024 + ((x) * 0x3000))
		1402	#define CFC1_N_MA_CSC_COEFF_C13(x) (0xa028 + ((x) * 0x3000))
		1403	#define CFC1_N_MA_CSC_COEFF_C14(x) (0xa02c + ((x) * 0x3000))
		1404	#define CFC1_N_MA_CSC_COEFF_C20(x) (0xa030 + ((x) * 0x3000))
		1405	#define CFC1_N_MA_CSC_COEFF_C21(x) (0xa034 + ((x) * 0x3000))
		1406	#define CFC1_N_MA_CSC_COEFF_C22(x) (0xa038 + ((x) * 0x3000))
		1407	#define CFC1_N_MA_CSC_COEFF_C23(x) (0xa03c + ((x) * 0x3000))
		1408	#define CFC1_N_MA_CSC_COEFF_C24(x) (0xa040 + ((x) * 0x3000))
		1409
		1410	#define SCALER_PI_CMP_CSC_RED0(x) (0x200 + ((x) * 0x40))
		1411	#define SCALER_PI_CMP_CSC_RED1(x) (0x204 + ((x) * 0x40))
		1412	#define SCALER_PI_CMP_CSC_RED_CLAMP(x) (0x208 + ((x) * 0x40))
		1413	#define SCALER_PI_CMP_CSC_CFG(x) (0x20c + ((x) * 0x40))
		1414	#define SCALER_PI_CMP_CSC_GREEN0(x) (0x210 + ((x) * 0x40))
		1415	#define SCALER_PI_CMP_CSC_GREEN1(x) (0x214 + ((x) * 0x40))
		1416	#define SCALER_PI_CMP_CSC_GREEN_CLAMP(x) (0x218 + ((x) * 0x40))
		1417	#define SCALER_PI_CMP_CSC_BLUE0(x) (0x220 + ((x) * 0x40))
		1418	#define SCALER_PI_CMP_CSC_BLUE1(x) (0x224 + ((x) * 0x40))
		1419	#define SCALER_PI_CMP_CSC_BLUE_CLAMP(x) (0x228 + ((x) * 0x40))
		1420
		1421	/* 4 S2.22 multiplication factors, and 1 S9.15 addititive element for each of 3
		1422	* output components
		1423	*/
		1424	struct vc6_csc_coeff_entry {
		1425	u32 csc[3][5];
		1426	};
		1427
		1428	static const struct vc6_csc_coeff_entry csc_coeffs[2][3] = {
		1429	[DRM_COLOR_YCBCR_LIMITED_RANGE] = {
		1430	[DRM_COLOR_YCBCR_BT601] = {
		1431	.csc = {
		1432	{ 0x004A8542, 0x0, 0x0066254A, 0x0, 0xFF908A0D },
		1433	{ 0x004A8542, 0xFFE6ED5D, 0xFFCBF856, 0x0, 0x0043C9A3 },
		1434	{ 0x004A8542, 0x00811A54, 0x0, 0x0, 0xFF759502 }
		1435	}
		1436	},
		1437	[DRM_COLOR_YCBCR_BT709] = {
		1438	.csc = {
		1439	{ 0x004A8542, 0x0, 0x0072BC44, 0x0, 0xFF83F312 },
		1440	{ 0x004A8542, 0xFFF25A22, 0xFFDDE4D0, 0x0, 0x00267064 },
		1441	{ 0x004A8542, 0x00873197, 0x0, 0x0, 0xFF6F7DC0 }
		1442	}
		1443	},
		1444	[DRM_COLOR_YCBCR_BT2020] = {
		1445	.csc = {
		1446	{ 0x004A8542, 0x0, 0x006B4A17, 0x0, 0xFF8B653F },
		1447	{ 0x004A8542, 0xFFF402D9, 0xFFDDE4D0, 0x0, 0x0024C7AE },
		1448	{ 0x004A8542, 0x008912CC, 0x0, 0x0, 0xFF6D9C8B }
		1449	}
		1450	}
		1451	},
		1452	[DRM_COLOR_YCBCR_FULL_RANGE] = {
		1453	[DRM_COLOR_YCBCR_BT601] = {
		1454	.csc = {
		1455	{ 0x00400000, 0x0, 0x0059BA5E, 0x0, 0xFFA645A1 },
		1456	{ 0x00400000, 0xFFE9F9AC, 0xFFD24B97, 0x0, 0x0043BABB },
		1457	{ 0x00400000, 0x00716872, 0x0, 0x0, 0xFF8E978D }
		1458	}
		1459	},
		1460	[DRM_COLOR_YCBCR_BT709] = {
		1461	.csc = {
		1462	{ 0x00400000, 0x0, 0x0064C985, 0x0, 0xFF9B367A },
		1463	{ 0x00400000, 0xFFF402E1, 0xFFE20A40, 0x0, 0x0029F2DE },
		1464	{ 0x00400000, 0x0076C226, 0x0, 0x0, 0xFF893DD9 }
		1465	}
		1466	},
		1467	[DRM_COLOR_YCBCR_BT2020] = {
		1468	.csc = {
		1469	{ 0x00400000, 0x0, 0x005E3F14, 0x0, 0xFFA1C0EB },
		1470	{ 0x00400000, 0xFFF577F6, 0xFFDB580F, 0x0, 0x002F2FFA },
		1471	{ 0x00400000, 0x007868DB, 0x0, 0x0, 0xFF879724 }
		1472	}
		1473	}
		1474	}
		1475	};
		1476
		1477	static int vc6_hvs_hw_init(struct vc4_hvs *hvs)
		1478	{
		1479	const struct vc6_csc_coeff_entry *coeffs;
		1480	unsigned int i;
		1481
		1482	HVS_WRITE(SCALER6_CONTROL,
		1483	SCALER6_CONTROL_HVS_EN \|
		1484	VC4_SET_FIELD(8, SCALER6_CONTROL_PF_LINES) \|
		1485	VC4_SET_FIELD(15, SCALER6_CONTROL_MAX_REQS));
		1486
		1487	/* Set HVS arbiter priority to max */
		1488	HVS_WRITE(SCALER6(PRI_MAP0), 0xffffffff);
		1489	HVS_WRITE(SCALER6(PRI_MAP1), 0xffffffff);
		1490
		1491	if (hvs->vc4->gen == VC4_GEN_6_C) {
		1492	for (i = 0; i < 6; i++) {
		1493	coeffs = &csc_coeffs[i / 3][i % 3];
		1494
		1495	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C00(i), coeffs->csc[0][0]);
		1496	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C01(i), coeffs->csc[0][1]);
		1497	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C02(i), coeffs->csc[0][2]);
		1498	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C03(i), coeffs->csc[0][3]);
		1499	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C04(i), coeffs->csc[0][4]);
		1500
		1501	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C10(i), coeffs->csc[1][0]);
		1502	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C11(i), coeffs->csc[1][1]);
		1503	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C12(i), coeffs->csc[1][2]);
		1504	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C13(i), coeffs->csc[1][3]);
		1505	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C14(i), coeffs->csc[1][4]);
		1506
		1507	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C20(i), coeffs->csc[2][0]);
		1508	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C21(i), coeffs->csc[2][1]);
		1509	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C22(i), coeffs->csc[2][2]);
		1510	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C23(i), coeffs->csc[2][3]);
		1511	HVS_WRITE(CFC1_N_MA_CSC_COEFF_C24(i), coeffs->csc[2][4]);
		1512
		1513	HVS_WRITE(CFC1_N_NL_CSC_CTRL(i), BIT(15));
		1514	}
		1515	} else {
		1516	for (i = 0; i < 8; i++) {
		1517	HVS_WRITE(SCALER_PI_CMP_CSC_RED0(i), 0x1f002566);
		1518	HVS_WRITE(SCALER_PI_CMP_CSC_RED1(i), 0x3994);
		1519	HVS_WRITE(SCALER_PI_CMP_CSC_RED_CLAMP(i), 0xfff00000);
		1520	HVS_WRITE(SCALER_PI_CMP_CSC_CFG(i), 0x1);
		1521	HVS_WRITE(SCALER_PI_CMP_CSC_GREEN0(i), 0x18002566);
		1522	HVS_WRITE(SCALER_PI_CMP_CSC_GREEN1(i), 0xf927eee2);
		1523	HVS_WRITE(SCALER_PI_CMP_CSC_GREEN_CLAMP(i), 0xfff00000);
		1524	HVS_WRITE(SCALER_PI_CMP_CSC_BLUE0(i), 0x18002566);
		1525	HVS_WRITE(SCALER_PI_CMP_CSC_BLUE1(i), 0x43d80000);
		1526	HVS_WRITE(SCALER_PI_CMP_CSC_BLUE_CLAMP(i), 0xfff00000);
		1527	}
		1528	}
		1529
		1530	return 0;
		1531	}
		1532
948	static int vc4_hvs_cob_init(struct vc4_hvs *hvs)	1533	static int vc4_hvs_cob_init(struct vc4_hvs *hvs)
949	{	1534	{
950	struct vc4_dev *vc4 = hvs->vc4;	1535	struct vc4_dev *vc4 = hvs->vc4;
951	u32 reg, top;	1536	u32 reg, top, base;
952		1537
953	/*	1538	/*
954	* Recompute Composite Output Buffer (COB) allocations for the	1539	* Recompute Composite Output Buffer (COB) allocations for the
@@ -1009,6 +1594,32 @@ static int vc4_hvs_cob_init(struct vc4_hvs *hvs) · 54 unmodified lines
1009	HVS_WRITE(SCALER_DISPBASE0, reg);	1594	HVS_WRITE(SCALER_DISPBASE0, reg);
1010	break;	1595	break;
1011		1596
		1597	case VC4_GEN_6_C:
		1598	case VC4_GEN_6_D:
		1599	#define VC6_COB_LINE_WIDTH 3840
		1600	#define VC6_COB_NUM_LINES 4
		1601	base = 0;
		1602	top = 3840;
		1603
		1604	HVS_WRITE(SCALER6_DISPX_COB(2),
		1605	VC4_SET_FIELD(top, SCALER6_DISPX_COB_TOP) \|
		1606	VC4_SET_FIELD(base, SCALER6_DISPX_COB_BASE));
		1607
		1608	base = top + 16;
		1609	top += VC6_COB_LINE_WIDTH * VC6_COB_NUM_LINES;
		1610
		1611	HVS_WRITE(SCALER6_DISPX_COB(1),
		1612	VC4_SET_FIELD(top, SCALER6_DISPX_COB_TOP) \|
		1613	VC4_SET_FIELD(base, SCALER6_DISPX_COB_BASE));
		1614
		1615	base = top + 16;
		1616	top += VC6_COB_LINE_WIDTH * VC6_COB_NUM_LINES;
		1617
		1618	HVS_WRITE(SCALER6_DISPX_COB(0),
		1619	VC4_SET_FIELD(top, SCALER6_DISPX_COB_TOP) \|
		1620	VC4_SET_FIELD(base, SCALER6_DISPX_COB_BASE));
		1621	break;
		1622
1012	default:	1623	default:
1013	return -EINVAL;	1624	return -EINVAL;
1014	}	1625	}
@@ -1034,10 +1645,23 @@ static int vc4_hvs_bind(struct device dev, struct device master, void *data) · 19 unmodified lines
1034	return PTR_ERR(hvs);	1645	return PTR_ERR(hvs);
1035		1646
1036	hvs->regset.base = hvs->regs;	1647	hvs->regset.base = hvs->regs;
1037	hvs->regset.regs = vc4_hvs_regs;
1038	hvs->regset.nregs = ARRAY_SIZE(vc4_hvs_regs);
1039		1648
1040	if (vc4->gen == VC4_GEN_5) {	1649	if (vc4->gen == VC4_GEN_6_C) {
		1650	hvs->regset.regs = vc6_hvs_regs;
		1651	hvs->regset.nregs = ARRAY_SIZE(vc6_hvs_regs);
		1652
		1653	if (VC4_GET_FIELD(HVS_READ(SCALER6_VERSION), SCALER6_VERSION) ==
		1654	SCALER6_VERSION_D0) {
		1655	vc4->gen = VC4_GEN_6_D;
		1656	hvs->regset.regs = vc6_d_hvs_regs;
		1657	hvs->regset.nregs = ARRAY_SIZE(vc6_d_hvs_regs);
		1658	}
		1659	} else {
		1660	hvs->regset.regs = vc4_hvs_regs;
		1661	hvs->regset.nregs = ARRAY_SIZE(vc4_hvs_regs);
		1662	}
		1663
		1664	if (vc4->gen >= VC4_GEN_5) {
1041	struct rpi_firmware *firmware;	1665	struct rpi_firmware *firmware;
1042	struct device_node *node;	1666	struct device_node *node;
1043	unsigned int max_rate;	1667	unsigned int max_rate;
@@ -1051,12 +1675,20 @@ static int vc4_hvs_bind(struct device dev, struct device master, void *data) · 7 unmodified lines
1051	if (!firmware)	1675	if (!firmware)
1052	return -EPROBE_DEFER;	1676	return -EPROBE_DEFER;
1053		1677
1054	hvs->core_clk = devm_clk_get(&pdev->dev, NULL);	1678	hvs->core_clk = devm_clk_get(&pdev->dev,
		1679	(vc4->gen >= VC4_GEN_6_C) ? "core" : NULL);
1055	if (IS_ERR(hvs->core_clk)) {	1680	if (IS_ERR(hvs->core_clk)) {
1056	dev_err(&pdev->dev, "Couldn't get core clock\n");	1681	dev_err(&pdev->dev, "Couldn't get core clock\n");
1057	return PTR_ERR(hvs->core_clk);	1682	return PTR_ERR(hvs->core_clk);
1058	}	1683	}
1059		1684
		1685	hvs->disp_clk = devm_clk_get(&pdev->dev,
		1686	(vc4->gen >= VC4_GEN_6_C) ? "disp" : NULL);
		1687	if (IS_ERR(hvs->disp_clk)) {
		1688	dev_err(&pdev->dev, "Couldn't get disp clock\n");
		1689	return PTR_ERR(hvs->disp_clk);
		1690	}
		1691
1060	max_rate = rpi_firmware_clk_get_max_rate(firmware,	1692	max_rate = rpi_firmware_clk_get_max_rate(firmware,
1061	RPI_FIRMWARE_CORE_CLK_ID);	1693	RPI_FIRMWARE_CORE_CLK_ID);
1062	rpi_firmware_put(firmware);	1694	rpi_firmware_put(firmware);
@@ -1073,14 +1705,23 @@ static int vc4_hvs_bind(struct device dev, struct device master, void *data) · 10 unmodified lines
1073	dev_err(&pdev->dev, "Couldn't enable the core clock\n");	1705	dev_err(&pdev->dev, "Couldn't enable the core clock\n");
1074	return ret;	1706	return ret;
1075	}	1707	}
		1708
		1709	ret = clk_prepare_enable(hvs->disp_clk);
		1710	if (ret) {
		1711	dev_err(&pdev->dev, "Couldn't enable the disp clock\n");
		1712	return ret;
		1713	}
1076	}	1714	}
1077		1715
1078	if (vc4->gen == VC4_GEN_4)	1716	if (vc4->gen >= VC4_GEN_5)
1079	hvs->dlist = hvs->regs + SCALER_DLIST_START;
1080	else
1081	hvs->dlist = hvs->regs + SCALER5_DLIST_START;	1717	hvs->dlist = hvs->regs + SCALER5_DLIST_START;
		1718	else
		1719	hvs->dlist = hvs->regs + SCALER_DLIST_START;
1082		1720
1083	ret = vc4_hvs_hw_init(hvs);	1721	if (vc4->gen >= VC4_GEN_6_C)
		1722	ret = vc6_hvs_hw_init(hvs);
		1723	else
		1724	ret = vc4_hvs_hw_init(hvs);
1084	if (ret)	1725	if (ret)
1085	return ret;	1726	return ret;
1086		1727
@@ -1097,10 +1738,12 @@ static int vc4_hvs_bind(struct device dev, struct device master, void *data) · 10 unmodified lines
1097	if (ret)	1738	if (ret)
1098	return ret;	1739	return ret;
1099		1740
1100	ret = devm_request_irq(dev, platform_get_irq(pdev, 0),	1741	if (vc4->gen < VC4_GEN_6_C) {
1101	vc4_hvs_irq_handler, 0, "vc4 hvs", drm);	1742	ret = devm_request_irq(dev, platform_get_irq(pdev, 0),
1102	if (ret)	1743	vc4_hvs_irq_handler, 0, "vc4 hvs", drm);
1103	return ret;	1744	if (ret)
		1745	return ret;
		1746	}
1104		1747
1105	return 0;	1748	return 0;
1106	}	1749	}
@@ -1125,6 +1768,7 @@ static void vc4_hvs_unbind(struct device dev, struct device master, · 18 unmodified lines
1125	drm_mm_remove_node(node);	1768	drm_mm_remove_node(node);
1126	drm_mm_takedown(&vc4->hvs->lbm_mm);	1769	drm_mm_takedown(&vc4->hvs->lbm_mm);
1127		1770
		1771	clk_disable_unprepare(hvs->disp_clk);
1128	clk_disable_unprepare(hvs->core_clk);	1772	clk_disable_unprepare(hvs->core_clk);
1129		1773
1130	vc4->hvs = NULL;	1774	vc4->hvs = NULL;
@@ -1147,6 +1791,7 @@ static void vc4_hvs_dev_remove(struct platform_device *pdev) · 16 unmodified lines
1147		1791
1148	static const struct of_device_id vc4_hvs_dt_match[] = {	1792	static const struct of_device_id vc4_hvs_dt_match[] = {
1149	{ .compatible = "brcm,bcm2711-hvs" },	1793	{ .compatible = "brcm,bcm2711-hvs" },
		1794	{ .compatible = "brcm,bcm2712-hvs" },
1150	{ .compatible = "brcm,bcm2835-hvs" },	1795	{ .compatible = "brcm,bcm2835-hvs" },
1151	{}	1796	{}
1152	};	1797	};

drivers/gpu/drm/vc4/vc4_kms.c+91−14

@@ -138,6 +138,8 @@ vc4_ctm_commit(struct vc4_dev vc4, struct drm_atomic_state state) · 137 unmodified lines
138	struct vc4_ctm_state *ctm_state = to_vc4_ctm_state(vc4->ctm_manager.state);	138	struct vc4_ctm_state *ctm_state = to_vc4_ctm_state(vc4->ctm_manager.state);
139	struct drm_color_ctm *ctm = ctm_state->ctm;	139	struct drm_color_ctm *ctm = ctm_state->ctm;
140		140
		141	WARN_ON_ONCE(vc4->gen > VC4_GEN_5);
		142
141	if (ctm_state->fifo) {	143	if (ctm_state->fifo) {
142	HVS_WRITE(SCALER_OLEDCOEF2,	144	HVS_WRITE(SCALER_OLEDCOEF2,
143	VC4_SET_FIELD(vc4_ctm_s31_32_to_s0_9(ctm->matrix[0]),	145	VC4_SET_FIELD(vc4_ctm_s31_32_to_s0_9(ctm->matrix[0]),
@@ -213,6 +215,8 @@ static void vc4_hvs_pv_muxing_commit(struct vc4_dev *vc4, · 69 unmodified lines
213	struct drm_crtc *crtc;	215	struct drm_crtc *crtc;
214	unsigned int i;	216	unsigned int i;
215		217
		218	WARN_ON_ONCE(vc4->gen != VC4_GEN_4);
		219
216	for_each_new_crtc_in_state(state, crtc, crtc_state, i) {	220	for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
217	struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);	221	struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
218	struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);	222	struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);
@@ -256,6 +260,8 @@ static void vc5_hvs_pv_muxing_commit(struct vc4_dev *vc4, · 37 unmodified lines
256	unsigned int i;	260	unsigned int i;
257	u32 reg;	261	u32 reg;
258		262
		263	WARN_ON_ONCE(vc4->gen != VC4_GEN_5);
		264
259	for_each_new_crtc_in_state(state, crtc, crtc_state, i) {	265	for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
260	struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);	266	struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);
261	struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);	267	struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
@@ -320,17 +326,62 @@ static void vc5_hvs_pv_muxing_commit(struct vc4_dev *vc4, · 58 unmodified lines
320	}	326	}
321	}	327	}
322		328
		329	static void vc6_hvs_pv_muxing_commit(struct vc4_dev *vc4,
		330	struct drm_atomic_state *state)
		331	{
		332	struct vc4_hvs *hvs = vc4->hvs;
		333	struct drm_crtc_state *crtc_state;
		334	struct drm_crtc *crtc;
		335	unsigned int i;
		336
		337	WARN_ON_ONCE(vc4->gen != VC4_GEN_6_C && vc4->gen != VC4_GEN_6_D);
		338
		339	for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
		340	struct vc4_crtc_state *vc4_state = to_vc4_crtc_state(crtc_state);
		341	struct vc4_encoder *vc4_encoder;
		342	struct drm_encoder *encoder;
		343	unsigned char mux;
		344	u32 reg;
		345
		346	if (!vc4_state->update_muxing)
		347	continue;
		348
		349	if (vc4_state->assigned_channel != 1)
		350	continue;
		351
		352	encoder = vc4_get_crtc_encoder(crtc, crtc_state);
		353	vc4_encoder = to_vc4_encoder(encoder);
		354	switch (vc4_encoder->type) {
		355	case VC4_ENCODER_TYPE_HDMI1:
		356	mux = 0;
		357	break;
		358
		359	case VC4_ENCODER_TYPE_TXP1:
		360	mux = 2;
		361	break;
		362
		363	default:
		364	drm_err(&vc4->base, "Unhandled encoder type for PV muxing %d",
		365	vc4_encoder->type);
		366	mux = 0;
		367	break;
		368	}
		369
		370	reg = HVS_READ(SCALER6_CONTROL);
		371	HVS_WRITE(SCALER6_CONTROL,
		372	(reg & ~SCALER6_CONTROL_DSP1_TARGET_MASK) \|
		373	VC4_SET_FIELD(mux, SCALER6_CONTROL_DSP1_TARGET));
		374	}
		375	}
		376
323	static void vc4_atomic_commit_tail(struct drm_atomic_state *state)	377	static void vc4_atomic_commit_tail(struct drm_atomic_state *state)
324	{	378	{
325	struct drm_device *dev = state->dev;	379	struct drm_device *dev = state->dev;
326	struct vc4_dev *vc4 = to_vc4_dev(dev);	380	struct vc4_dev *vc4 = to_vc4_dev(dev);
327	struct vc4_hvs *hvs = vc4->hvs;	381	struct vc4_hvs *hvs = vc4->hvs;
328	struct drm_crtc_state *new_crtc_state;
329	struct vc4_hvs_state *new_hvs_state;	382	struct vc4_hvs_state *new_hvs_state;
330	struct drm_crtc *crtc;
331	struct vc4_hvs_state *old_hvs_state;	383	struct vc4_hvs_state *old_hvs_state;
332	unsigned int channel;	384	unsigned int channel;
333	int i;
334		385
335	old_hvs_state = vc4_hvs_get_old_global_state(state);	386	old_hvs_state = vc4_hvs_get_old_global_state(state);
336	if (WARN_ON(IS_ERR(old_hvs_state)))	387	if (WARN_ON(IS_ERR(old_hvs_state)))
@@ -340,14 +391,20 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state) · 3 unmodified lines
340	if (WARN_ON(IS_ERR(new_hvs_state)))	391	if (WARN_ON(IS_ERR(new_hvs_state)))
341	return;	392	return;
342		393
343	for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {	394	if (vc4->gen < VC4_GEN_6_C) {
344	struct vc4_crtc_state *vc4_crtc_state;	395	struct drm_crtc_state *new_crtc_state;
		396	struct drm_crtc *crtc;
		397	int i;
345		398
346	if (!new_crtc_state->commit)	399	for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
347	continue;	400	struct vc4_crtc_state *vc4_crtc_state;
		401
		402	if (!new_crtc_state->commit)
		403	continue;
348		404
349	vc4_crtc_state = to_vc4_crtc_state(new_crtc_state);	405	vc4_crtc_state = to_vc4_crtc_state(new_crtc_state);
350	vc4_hvs_mask_underrun(hvs, vc4_crtc_state->assigned_channel);	406	vc4_hvs_mask_underrun(hvs, vc4_crtc_state->assigned_channel);
		407	}
351	}	408	}
352		409
353	for (channel = 0; channel < HVS_NUM_CHANNELS; channel++) {	410	for (channel = 0; channel < HVS_NUM_CHANNELS; channel++) {
@@ -382,16 +439,32 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state) · 28 unmodified lines
382	* modeset.	439	* modeset.
383	*/	440	*/
384	WARN_ON(clk_set_min_rate(hvs->core_clk, core_rate));	441	WARN_ON(clk_set_min_rate(hvs->core_clk, core_rate));
		442	WARN_ON(clk_set_min_rate(hvs->disp_clk, core_rate));
385	}	443	}
386		444
387	drm_atomic_helper_commit_modeset_disables(dev, state);	445	drm_atomic_helper_commit_modeset_disables(dev, state);
388		446
389	vc4_ctm_commit(vc4, state);	447	if (vc4->gen <= VC4_GEN_5)
		448	vc4_ctm_commit(vc4, state);
390		449
391	if (vc4->gen == VC4_GEN_5)	450	switch (vc4->gen) {
392	vc5_hvs_pv_muxing_commit(vc4, state);	451	case VC4_GEN_4:
393	else
394	vc4_hvs_pv_muxing_commit(vc4, state);	452	vc4_hvs_pv_muxing_commit(vc4, state);
		453	break;
		454
		455	case VC4_GEN_5:
		456	vc5_hvs_pv_muxing_commit(vc4, state);
		457	break;
		458
		459	case VC4_GEN_6_C:
		460	case VC4_GEN_6_D:
		461	vc6_hvs_pv_muxing_commit(vc4, state);
		462	break;
		463
		464	default:
		465	drm_err(dev, "Unknown VC4 generation: %d", vc4->gen);
		466	break;
		467	}
395		468
396	drm_atomic_helper_commit_planes(dev, state,	469	drm_atomic_helper_commit_planes(dev, state,
397	DRM_PLANE_COMMIT_ACTIVE_ONLY);	470	DRM_PLANE_COMMIT_ACTIVE_ONLY);
@@ -418,6 +491,7 @@ static void vc4_atomic_commit_tail(struct drm_atomic_state *state) · 20 unmodified lines
418	* requirements.	491	* requirements.
419	*/	492	*/
420	WARN_ON(clk_set_min_rate(hvs->core_clk, core_rate));	493	WARN_ON(clk_set_min_rate(hvs->core_clk, core_rate));
		494	WARN_ON(clk_set_min_rate(hvs->disp_clk, core_rate));
421		495
422	drm_dbg(dev, "Core clock actual rate: %lu Hz\n",	496	drm_dbg(dev, "Core clock actual rate: %lu Hz\n",
423	clk_get_rate(hvs->core_clk));	497	clk_get_rate(hvs->core_clk));
@@ -1056,7 +1130,10 @@ int vc4_kms_load(struct drm_device *dev) · 632 unmodified lines
1056	return ret;	1130	return ret;
1057	}	1131	}
1058		1132
1059	if (vc4->gen == VC4_GEN_5) {	1133	if (vc4->gen >= VC4_GEN_6_C) {
		1134	dev->mode_config.max_width = 8192;
		1135	dev->mode_config.max_height = 8192;
		1136	} else if (vc4->gen >= VC4_GEN_5) {
1060	dev->mode_config.max_width = 7680;	1137	dev->mode_config.max_width = 7680;
1061	dev->mode_config.max_height = 7680;	1138	dev->mode_config.max_height = 7680;
1062	} else {	1139	} else {

drivers/gpu/drm/vc4/vc4_plane.c+832−34

@@ -278,7 +278,10 @@ static bool plane_enabled(struct drm_plane_state *state) · 277 unmodified lines
278		278
279	static struct drm_plane_state vc4_plane_duplicate_state(struct drm_plane plane)	279	static struct drm_plane_state vc4_plane_duplicate_state(struct drm_plane plane)
280	{	280	{
		281	struct vc4_dev *vc4 = to_vc4_dev(plane->dev);
		282	struct vc4_hvs *hvs = vc4->hvs;
281	struct vc4_plane_state *vc4_state;	283	struct vc4_plane_state *vc4_state;
		284	unsigned int i;
282		285
283	if (WARN_ON(!plane->state))	286	if (WARN_ON(!plane->state))
284	return NULL;	287	return NULL;
@@ -288,6 +291,12 @@ static struct drm_plane_state vc4_plane_duplicate_state(struct drm_plane plane · 3 unmodified lines
288	return NULL;	291	return NULL;
289		292
290	memset(&vc4_state->lbm, 0, sizeof(vc4_state->lbm));	293	memset(&vc4_state->lbm, 0, sizeof(vc4_state->lbm));
		294
		295	for (i = 0; i < DRM_FORMAT_MAX_PLANES; i++) {
		296	if (vc4_state->upm_handle[i])
		297	refcount_inc(&hvs->upm_refcounts[vc4_state->upm_handle[i]].refcount);
		298	}
		299
291	vc4_state->dlist_initialized = 0;	300	vc4_state->dlist_initialized = 0;
292		301
293	__drm_atomic_helper_plane_duplicate_state(plane, &vc4_state->base);	302	__drm_atomic_helper_plane_duplicate_state(plane, &vc4_state->base);
@@ -306,18 +315,47 @@ static struct drm_plane_state vc4_plane_duplicate_state(struct drm_plane plane · 12 unmodified lines
306	return &vc4_state->base;	315	return &vc4_state->base;
307	}	316	}
308		317
		318	static void vc4_plane_release_upm_ida(struct vc4_hvs *hvs, unsigned int upm_handle)
		319	{
		320	struct vc4_upm_refcounts *refcount = &hvs->upm_refcounts[upm_handle];
		321	unsigned long irqflags;
		322
		323	spin_lock_irqsave(&hvs->mm_lock, irqflags);
		324	drm_mm_remove_node(&refcount->upm);
		325	spin_unlock_irqrestore(&hvs->mm_lock, irqflags);
		326	refcount->upm.start = 0;
		327	refcount->upm.size = 0;
		328	refcount->size = 0;
		329
		330	ida_free(&hvs->upm_handles, upm_handle);
		331	}
		332
309	static void vc4_plane_destroy_state(struct drm_plane *plane,	333	static void vc4_plane_destroy_state(struct drm_plane *plane,
310	struct drm_plane_state *state)	334	struct drm_plane_state *state)
311	{	335	{
312	struct vc4_dev *vc4 = to_vc4_dev(plane->dev);	336	struct vc4_dev *vc4 = to_vc4_dev(plane->dev);
		337	struct vc4_hvs *hvs = vc4->hvs;
313	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);	338	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		339	unsigned int i;
314		340
315	if (drm_mm_node_allocated(&vc4_state->lbm)) {	341	if (drm_mm_node_allocated(&vc4_state->lbm)) {
316	unsigned long irqflags;	342	unsigned long irqflags;
317		343
318	spin_lock_irqsave(&vc4->hvs->mm_lock, irqflags);	344	spin_lock_irqsave(&hvs->mm_lock, irqflags);
319	drm_mm_remove_node(&vc4_state->lbm);	345	drm_mm_remove_node(&vc4_state->lbm);
320	spin_unlock_irqrestore(&vc4->hvs->mm_lock, irqflags);	346	spin_unlock_irqrestore(&hvs->mm_lock, irqflags);
		347	}
		348
		349	for (i = 0; i < DRM_FORMAT_MAX_PLANES; i++) {
		350	struct vc4_upm_refcounts *refcount;
		351
		352	if (!vc4_state->upm_handle[i])
		353	continue;
		354
		355	refcount = &hvs->upm_refcounts[vc4_state->upm_handle[i]];
		356
		357	if (refcount_dec_and_test(&refcount->refcount))
		358	vc4_plane_release_upm_ida(hvs, vc4_state->upm_handle[i]);
321	}	359	}
322		360
323	kfree(vc4_state->dlist);	361	kfree(vc4_state->dlist);
@@ -528,8 +566,11 @@ static int vc4_plane_setup_clipping_and_scaling(struct drm_plane_state *state) · 204 unmodified lines
528		566
529	static void vc4_write_tpz(struct vc4_plane_state *vc4_state, u32 src, u32 dst)	567	static void vc4_write_tpz(struct vc4_plane_state *vc4_state, u32 src, u32 dst)
530	{	568	{
		569	struct vc4_dev *vc4 = to_vc4_dev(vc4_state->base.plane->dev);
531	u32 scale, recip;	570	u32 scale, recip;
532		571
		572	WARN_ON_ONCE(vc4->gen > VC4_GEN_6_D);
		573
533	scale = src / dst;	574	scale = src / dst;
534		575
535	/* The specs note that while the reciprocal would be defined	576	/* The specs note that while the reciprocal would be defined
@@ -538,6 +579,11 @@ static void vc4_write_tpz(struct vc4_plane_state *vc4_state, u32 src, u32 dst) · 2 unmodified lines
538	recip = ~0 / scale;	579	recip = ~0 / scale;
539		580
540	vc4_dlist_write(vc4_state,	581	vc4_dlist_write(vc4_state,
		582	/*
		583	* The BCM2712 is lacking BIT(31) compared to
		584	* the previous generations, but we don't use
		585	* it.
		586	*/
541	VC4_SET_FIELD(scale, SCALER_TPZ0_SCALE) \|	587	VC4_SET_FIELD(scale, SCALER_TPZ0_SCALE) \|
542	VC4_SET_FIELD(0, SCALER_TPZ0_IPHASE));	588	VC4_SET_FIELD(0, SCALER_TPZ0_IPHASE));
543	vc4_dlist_write(vc4_state,	589	vc4_dlist_write(vc4_state,
@@ -550,10 +596,13 @@ static void vc4_write_tpz(struct vc4_plane_state *vc4_state, u32 src, u32 dst) · 6 unmodified lines
550	static void vc4_write_ppf(struct vc4_plane_state *vc4_state, u32 src, u32 dst,	596	static void vc4_write_ppf(struct vc4_plane_state *vc4_state, u32 src, u32 dst,
551	u32 xy, int channel)	597	u32 xy, int channel)
552	{	598	{
		599	struct vc4_dev *vc4 = to_vc4_dev(vc4_state->base.plane->dev);
553	u32 scale = src / dst;	600	u32 scale = src / dst;
554	s32 offset, offset2;	601	s32 offset, offset2;
555	s32 phase;	602	s32 phase;
556		603
		604	WARN_ON_ONCE(vc4->gen > VC4_GEN_6_D);
		605
557	/*	606	/*
558	* Start the phase at 1/2 pixel from the 1st pixel at src_x.	607	* Start the phase at 1/2 pixel from the 1st pixel at src_x.
559	* 1/4 pixel for YUV.	608	* 1/4 pixel for YUV.
@@ -598,10 +647,15 @@ static void vc4_write_ppf(struct vc4_plane_state *vc4_state, u32 src, u32 dst, · 38 unmodified lines
598	vc4_dlist_write(vc4_state,	647	vc4_dlist_write(vc4_state,
599	SCALER_PPF_AGC \|	648	SCALER_PPF_AGC \|
600	VC4_SET_FIELD(scale, SCALER_PPF_SCALE) \|	649	VC4_SET_FIELD(scale, SCALER_PPF_SCALE) \|
		650	/*
		651	* The register layout documentation is slightly
		652	* different to setup the phase in the BCM2712,
		653	* but they seem equivalent.
		654	*/
601	VC4_SET_FIELD(phase, SCALER_PPF_IPHASE));	655	VC4_SET_FIELD(phase, SCALER_PPF_IPHASE));
602	}	656	}
603		657
604	static u32 vc4_lbm_size(struct drm_plane_state *state)	658	static u32 __vc4_lbm_size(struct drm_plane_state *state)
605	{	659	{
606	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);	660	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
607	struct vc4_dev *vc4 = to_vc4_dev(state->plane->dev);	661	struct vc4_dev *vc4 = to_vc4_dev(state->plane->dev);
@@ -649,11 +703,139 @@ static u32 vc4_lbm_size(struct drm_plane_state *state) · 41 unmodified lines
649	return lbm;	703	return lbm;
650	}	704	}
651		705
		706	static unsigned int vc4_lbm_words_per_component(const struct drm_plane_state *state,
		707	unsigned int channel)
		708	{
		709	const struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		710
		711	switch (vc4_state->y_scaling[channel]) {
		712	case VC4_SCALING_PPF:
		713	return 4;
		714
		715	case VC4_SCALING_TPZ:
		716	return 2;
		717
		718	default:
		719	return 0;
		720	}
		721	}
		722
		723	static unsigned int vc4_lbm_components(const struct drm_plane_state *state,
		724	unsigned int channel)
		725	{
		726	const struct drm_format_info *info = state->fb->format;
		727	const struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		728
		729	if (vc4_state->y_scaling[channel] == VC4_SCALING_NONE)
		730	return 0;
		731
		732	if (info->is_yuv)
		733	return channel ? 2 : 1;
		734
		735	if (info->has_alpha)
		736	return 4;
		737
		738	return 3;
		739	}
		740
		741	static unsigned int vc4_lbm_channel_size(const struct drm_plane_state *state,
		742	unsigned int channel)
		743	{
		744	const struct drm_format_info *info = state->fb->format;
		745	const struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		746	unsigned int channels_scaled = 0;
		747	unsigned int components, words, wpc;
		748	unsigned int width, lines;
		749	unsigned int i;
		750
		751	/* LBM is meant to use the smaller of source or dest width, but there
		752	* is a issue with UV scaling that the size required for the second
		753	* channel is based on the source width only.
		754	*/
		755	if (info->hsub > 1 && channel == 1)
		756	width = state->src_w >> 16;
		757	else
		758	width = min(state->src_w >> 16, state->crtc_w);
		759	width = round_up(width / info->hsub, 4);
		760
		761	wpc = vc4_lbm_words_per_component(state, channel);
		762	if (!wpc)
		763	return 0;
		764
		765	components = vc4_lbm_components(state, channel);
		766	if (!components)
		767	return 0;
		768
		769	if (state->alpha != DRM_BLEND_ALPHA_OPAQUE && info->has_alpha)
		770	components -= 1;
		771
		772	words = width * wpc * components;
		773
		774	lines = DIV_ROUND_UP(words, 128 / info->hsub);
		775
		776	for (i = 0; i < 2; i++)
		777	if (vc4_state->y_scaling[channel] != VC4_SCALING_NONE)
		778	channels_scaled++;
		779
		780	if (channels_scaled == 1)
		781	lines = lines / 2;
		782
		783	return lines;
		784	}
		785
		786	static unsigned int __vc6_lbm_size(const struct drm_plane_state *state)
		787	{
		788	const struct drm_format_info *info = state->fb->format;
		789
		790	if (info->hsub > 1)
		791	return max(vc4_lbm_channel_size(state, 0),
		792	vc4_lbm_channel_size(state, 1));
		793	else
		794	return vc4_lbm_channel_size(state, 0);
		795	}
		796
		797	static u32 vc4_lbm_size(struct drm_plane_state *state)
		798	{
		799	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		800	struct vc4_dev *vc4 = to_vc4_dev(state->plane->dev);
		801
		802	/* LBM is not needed when there's no vertical scaling. */
		803	if (vc4_state->y_scaling[0] == VC4_SCALING_NONE &&
		804	vc4_state->y_scaling[1] == VC4_SCALING_NONE)
		805	return 0;
		806
		807	if (vc4->gen >= VC4_GEN_6_C)
		808	return __vc6_lbm_size(state);
		809	else
		810	return __vc4_lbm_size(state);
		811	}
		812
		813	static size_t vc6_upm_size(const struct drm_plane_state *state,
		814	unsigned int plane)
		815	{
		816	const struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		817	unsigned int stride = state->fb->pitches[plane];
		818
		819	/*
		820	* TODO: This only works for raster formats, and is sub-optimal
		821	* for buffers with a stride aligned on 32 bytes.
		822	*/
		823	unsigned int words_per_line = (stride + 62) / 32;
		824	unsigned int fetch_region_size = words_per_line * 32;
		825	unsigned int buffer_lines = 2 << vc4_state->upm_buffer_lines;
		826	unsigned int buffer_size = fetch_region_size * buffer_lines;
		827
		828	return ALIGN(buffer_size, HVS_UBM_WORD_SIZE);
		829	}
		830
652	static void vc4_write_scaling_parameters(struct drm_plane_state *state,	831	static void vc4_write_scaling_parameters(struct drm_plane_state *state,
653	int channel)	832	int channel)
654	{	833	{
		834	struct vc4_dev *vc4 = to_vc4_dev(state->plane->dev);
655	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);	835	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
656		836
		837	WARN_ON_ONCE(vc4->gen > VC4_GEN_6_D);
		838
657	/* Ch0 H-PPF Word 0: Scaling Parameters */	839	/* Ch0 H-PPF Word 0: Scaling Parameters */
658	if (vc4_state->x_scaling[channel] == VC4_SCALING_PPF) {	840	if (vc4_state->x_scaling[channel] == VC4_SCALING_PPF) {
659	vc4_write_ppf(vc4_state, vc4_state->src_w[channel],	841	vc4_write_ppf(vc4_state, vc4_state->src_w[channel],
@@ -750,6 +932,10 @@ static int vc4_plane_allocate_lbm(struct drm_plane_state *state) · 90 unmodified lines
750	if (!lbm_size)	932	if (!lbm_size)
751	return 0;	933	return 0;
752		934
		935	/*
		936	* NOTE: BCM2712 doesn't need to be aligned, since the size
		937	* returned by vc4_lbm_size() is in words already.
		938	*/
753	if (vc4->gen == VC4_GEN_5)	939	if (vc4->gen == VC4_GEN_5)
754	lbm_size = ALIGN(lbm_size, 64);	940	lbm_size = ALIGN(lbm_size, 64);
755	else if (vc4->gen == VC4_GEN_4)	941	else if (vc4->gen == VC4_GEN_4)
@@ -787,6 +973,108 @@ static int vc4_plane_allocate_lbm(struct drm_plane_state *state) · 31 unmodified lines
787	return 0;	973	return 0;
788	}	974	}
789		975
		976	static int vc6_plane_allocate_upm(struct drm_plane_state *state)
		977	{
		978	const struct drm_format_info *info = state->fb->format;
		979	struct drm_device *drm = state->plane->dev;
		980	struct vc4_dev *vc4 = to_vc4_dev(drm);
		981	struct vc4_hvs *hvs = vc4->hvs;
		982	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		983	unsigned int i;
		984	int ret;
		985
		986	WARN_ON_ONCE(vc4->gen < VC4_GEN_6_C);
		987
		988	vc4_state->upm_buffer_lines = SCALER6_PTR0_UPM_BUFF_SIZE_2_LINES;
		989
		990	for (i = 0; i < info->num_planes; i++) {
		991	struct vc4_upm_refcounts *refcount;
		992	int upm_handle;
		993	unsigned long irqflags;
		994	size_t upm_size;
		995
		996	upm_size = vc6_upm_size(state, i);
		997	if (!upm_size)
		998	return -EINVAL;
		999	upm_handle = vc4_state->upm_handle[i];
		1000
		1001	if (upm_handle &&
		1002	hvs->upm_refcounts[upm_handle].size == upm_size) {
		1003	/* Allocation is the same size as the previous user of
		1004	* the plane. Keep the allocation.
		1005	*/
		1006	vc4_state->upm_handle[i] = upm_handle;
		1007	} else {
		1008	if (upm_handle &&
		1009	refcount_dec_and_test(&hvs->upm_refcounts[upm_handle].refcount)) {
		1010	vc4_plane_release_upm_ida(hvs, upm_handle);
		1011	vc4_state->upm_handle[i] = 0;
		1012	}
		1013
		1014	upm_handle = ida_alloc_range(&hvs->upm_handles, 1,
		1015	VC4_NUM_UPM_HANDLES,
		1016	GFP_KERNEL);
		1017	if (upm_handle < 0) {
		1018	drm_dbg(drm, "Out of upm_handles\n");
		1019	return upm_handle;
		1020	}
		1021	vc4_state->upm_handle[i] = upm_handle;
		1022
		1023	refcount = &hvs->upm_refcounts[upm_handle];
		1024	refcount_set(&refcount->refcount, 1);
		1025	refcount->size = upm_size;
		1026
		1027	spin_lock_irqsave(&hvs->mm_lock, irqflags);
		1028	ret = drm_mm_insert_node_generic(&hvs->upm_mm,
		1029	&refcount->upm,
		1030	upm_size, HVS_UBM_WORD_SIZE,
		1031	0, 0);
		1032	spin_unlock_irqrestore(&hvs->mm_lock, irqflags);
		1033	if (ret) {
		1034	drm_err(drm, "Failed to allocate UPM entry: %d\n", ret);
		1035	refcount_set(&refcount->refcount, 0);
		1036	ida_free(&hvs->upm_handles, upm_handle);
		1037	vc4_state->upm_handle[i] = 0;
		1038	return ret;
		1039	}
		1040	}
		1041
		1042	refcount = &hvs->upm_refcounts[upm_handle];
		1043	vc4_state->dlist[vc4_state->ptr0_offset[i]] \|=
		1044	VC4_SET_FIELD(refcount->upm.start / HVS_UBM_WORD_SIZE,
		1045	SCALER6_PTR0_UPM_BASE) \|
		1046	VC4_SET_FIELD(vc4_state->upm_handle[i] - 1,
		1047	SCALER6_PTR0_UPM_HANDLE) \|
		1048	VC4_SET_FIELD(vc4_state->upm_buffer_lines,
		1049	SCALER6_PTR0_UPM_BUFF_SIZE);
		1050	}
		1051
		1052	return 0;
		1053	}
		1054
		1055	static void vc6_plane_free_upm(struct drm_plane_state *state)
		1056	{
		1057	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		1058	struct drm_device *drm = state->plane->dev;
		1059	struct vc4_dev *vc4 = to_vc4_dev(drm);
		1060	struct vc4_hvs *hvs = vc4->hvs;
		1061	unsigned int i;
		1062
		1063	WARN_ON_ONCE(vc4->gen < VC4_GEN_6_C);
		1064
		1065	for (i = 0; i < DRM_FORMAT_MAX_PLANES; i++) {
		1066	unsigned int upm_handle;
		1067
		1068	upm_handle = vc4_state->upm_handle[i];
		1069	if (!upm_handle)
		1070	continue;
		1071
		1072	if (refcount_dec_and_test(&hvs->upm_refcounts[upm_handle].refcount))
		1073	vc4_plane_release_upm_ida(hvs, upm_handle);
		1074	vc4_state->upm_handle[i] = 0;
		1075	}
		1076	}
		1077
790	/*	1078	/*
791	* The colorspace conversion matrices are held in 3 entries in the dlist.	1079	* The colorspace conversion matrices are held in 3 entries in the dlist.
792	* Create an array of them, with entries for each full and limited mode, and	1080	* Create an array of them, with entries for each full and limited mode, and
@@ -834,6 +1122,11 @@ static const u32 colorspace_coeffs[2][DRM_COLOR_ENCODING_MAX][3] = { · 41 unmodified lines
834		1122
835	static u32 vc4_hvs4_get_alpha_blend_mode(struct drm_plane_state *state)	1123	static u32 vc4_hvs4_get_alpha_blend_mode(struct drm_plane_state *state)
836	{	1124	{
		1125	struct drm_device *dev = state->state->dev;
		1126	struct vc4_dev *vc4 = to_vc4_dev(dev);
		1127
		1128	WARN_ON_ONCE(vc4->gen != VC4_GEN_4);
		1129
837	if (!state->fb->format->has_alpha)	1130	if (!state->fb->format->has_alpha)
838	return VC4_SET_FIELD(SCALER_POS2_ALPHA_MODE_FIXED,	1131	return VC4_SET_FIELD(SCALER_POS2_ALPHA_MODE_FIXED,
839	SCALER_POS2_ALPHA_MODE);	1132	SCALER_POS2_ALPHA_MODE);
@@ -855,25 +1148,56 @@ static u32 vc4_hvs4_get_alpha_blend_mode(struct drm_plane_state *state) · 15 unmodified lines
855		1148
856	static u32 vc4_hvs5_get_alpha_blend_mode(struct drm_plane_state *state)	1149	static u32 vc4_hvs5_get_alpha_blend_mode(struct drm_plane_state *state)
857	{	1150	{
858	if (!state->fb->format->has_alpha)	1151	struct drm_device *dev = state->state->dev;
859	return VC4_SET_FIELD(SCALER5_CTL2_ALPHA_MODE_FIXED,	1152	struct vc4_dev *vc4 = to_vc4_dev(dev);
860	SCALER5_CTL2_ALPHA_MODE);
861		1153
862	switch (state->pixel_blend_mode) {	1154	WARN_ON_ONCE(vc4->gen != VC4_GEN_5 && vc4->gen != VC4_GEN_6_C &&
863	case DRM_MODE_BLEND_PIXEL_NONE:	1155	vc4->gen != VC4_GEN_6_D);
864	return VC4_SET_FIELD(SCALER5_CTL2_ALPHA_MODE_FIXED,	1156
865	SCALER5_CTL2_ALPHA_MODE);	1157	switch (vc4->gen) {
866	default:	1158	default:
867	case DRM_MODE_BLEND_PREMULTI:	1159	case VC4_GEN_5:
868	return VC4_SET_FIELD(SCALER5_CTL2_ALPHA_MODE_PIPELINE,	1160	case VC4_GEN_6_C:
869	SCALER5_CTL2_ALPHA_MODE) \|	1161	if (!state->fb->format->has_alpha)
870	SCALER5_CTL2_ALPHA_PREMULT;	1162	return VC4_SET_FIELD(SCALER5_CTL2_ALPHA_MODE_FIXED,
871	case DRM_MODE_BLEND_COVERAGE:	1163	SCALER5_CTL2_ALPHA_MODE);
872	return VC4_SET_FIELD(SCALER5_CTL2_ALPHA_MODE_PIPELINE,	1164
873	SCALER5_CTL2_ALPHA_MODE);	1165	switch (state->pixel_blend_mode) {
		1166	case DRM_MODE_BLEND_PIXEL_NONE:
		1167	return VC4_SET_FIELD(SCALER5_CTL2_ALPHA_MODE_FIXED,
		1168	SCALER5_CTL2_ALPHA_MODE);
		1169	default:
		1170	case DRM_MODE_BLEND_PREMULTI:
		1171	return VC4_SET_FIELD(SCALER5_CTL2_ALPHA_MODE_PIPELINE,
		1172	SCALER5_CTL2_ALPHA_MODE) \|
		1173	SCALER5_CTL2_ALPHA_PREMULT;
		1174	case DRM_MODE_BLEND_COVERAGE:
		1175	return VC4_SET_FIELD(SCALER5_CTL2_ALPHA_MODE_PIPELINE,
		1176	SCALER5_CTL2_ALPHA_MODE);
		1177	}
		1178	case VC4_GEN_6_D:
		1179	/* 2712-D configures fixed alpha mode in CTL0 */
		1180	return state->pixel_blend_mode == DRM_MODE_BLEND_PREMULTI ?
		1181	SCALER5_CTL2_ALPHA_PREMULT : 0;
874	}	1182	}
875	}	1183	}
876		1184
		1185	static u32 vc4_hvs6_get_alpha_mask_mode(struct drm_plane_state *state)
		1186	{
		1187	struct drm_device *dev = state->state->dev;
		1188	struct vc4_dev *vc4 = to_vc4_dev(dev);
		1189
		1190	WARN_ON_ONCE(vc4->gen != VC4_GEN_6_C && vc4->gen != VC4_GEN_6_D);
		1191
		1192	if (vc4->gen == VC4_GEN_6_D &&
		1193	(!state->fb->format->has_alpha \|\|
		1194	state->pixel_blend_mode == DRM_MODE_BLEND_PIXEL_NONE))
		1195	return VC4_SET_FIELD(SCALER6D_CTL0_ALPHA_MASK_FIXED,
		1196	SCALER6_CTL0_ALPHA_MASK);
		1197
		1198	return VC4_SET_FIELD(SCALER6_CTL0_ALPHA_MASK_NONE, SCALER6_CTL0_ALPHA_MASK);
		1199	}
		1200
877	/* Writes out a full display list for an active plane to the plane's	1201	/* Writes out a full display list for an active plane to the plane's
878	* private dlist state.	1202	* private dlist state.
879	*/	1203	*/
@@ -906,6 +1230,13 @@ static int vc4_plane_mode_set(struct drm_plane *plane, · 26 unmodified lines
906	if (ret)	1230	if (ret)
907	return ret;	1231	return ret;
908		1232
		1233	if (!vc4_state->src_w[0] \|\| !vc4_state->src_h[0] \|\|
		1234	!vc4_state->crtc_w \|\| !vc4_state->crtc_h) {
		1235	/* 0 source size probably means the plane is offscreen */
		1236	vc4_state->dlist_initialized = 1;
		1237	return 0;
		1238	}
		1239
909	width = vc4_state->src_w[0] >> 16;	1240	width = vc4_state->src_w[0] >> 16;
910	height = vc4_state->src_h[0] >> 16;	1241	height = vc4_state->src_h[0] >> 16;
911		1242
@@ -1363,6 +1694,427 @@ static int vc4_plane_mode_set(struct drm_plane *plane, · 451 unmodified lines
1363	return 0;	1694	return 0;
1364	}	1695	}
1365		1696
		1697	static u32 vc6_plane_get_csc_mode(struct vc4_plane_state *vc4_state)
		1698	{
		1699	struct drm_plane_state *state = &vc4_state->base;
		1700	struct vc4_dev *vc4 = to_vc4_dev(state->plane->dev);
		1701	u32 ret = 0;
		1702
		1703	if (vc4_state->is_yuv) {
		1704	enum drm_color_encoding color_encoding = state->color_encoding;
		1705	enum drm_color_range color_range = state->color_range;
		1706
		1707	/* CSC pre-loaded with:
		1708	* 0 = BT601 limited range
		1709	* 1 = BT709 limited range
		1710	* 2 = BT2020 limited range
		1711	* 3 = BT601 full range
		1712	* 4 = BT709 full range
		1713	* 5 = BT2020 full range
		1714	*/
		1715	if (color_encoding > DRM_COLOR_YCBCR_BT2020)
		1716	color_encoding = DRM_COLOR_YCBCR_BT601;
		1717	if (color_range > DRM_COLOR_YCBCR_FULL_RANGE)
		1718	color_range = DRM_COLOR_YCBCR_LIMITED_RANGE;
		1719
		1720	if (vc4->gen == VC4_GEN_6_C) {
		1721	ret \|= SCALER6C_CTL2_CSC_ENABLE;
		1722	ret \|= VC4_SET_FIELD(color_encoding + (color_range * 3),
		1723	SCALER6C_CTL2_BRCM_CFC_CONTROL);
		1724	} else {
		1725	ret \|= SCALER6D_CTL2_CSC_ENABLE;
		1726	ret \|= VC4_SET_FIELD(color_encoding + (color_range * 3),
		1727	SCALER6D_CTL2_BRCM_CFC_CONTROL);
		1728	}
		1729	}
		1730
		1731	return ret;
		1732	}
		1733
		1734	static int vc6_plane_mode_set(struct drm_plane *plane,
		1735	struct drm_plane_state *state)
		1736	{
		1737	struct drm_device *drm = plane->dev;
		1738	struct vc4_dev *vc4 = to_vc4_dev(drm);
		1739	struct vc4_plane_state *vc4_state = to_vc4_plane_state(state);
		1740	struct drm_framebuffer *fb = state->fb;
		1741	const struct hvs_format *format = vc4_get_hvs_format(fb->format->format);
		1742	u64 base_format_mod = fourcc_mod_broadcom_mod(fb->modifier);
		1743	int num_planes = fb->format->num_planes;
		1744	u32 h_subsample = fb->format->hsub;
		1745	u32 v_subsample = fb->format->vsub;
		1746	bool mix_plane_alpha;
		1747	bool covers_screen;
		1748	u32 scl0, scl1, pitch0;
		1749	u32 tiling, src_x, src_y;
		1750	u32 width, height;
		1751	u32 hvs_format = format->hvs;
		1752	u32 offsets[3] = { 0 };
		1753	unsigned int rotation;
		1754	int ret, i;
		1755
		1756	if (vc4_state->dlist_initialized)
		1757	return 0;
		1758
		1759	ret = vc4_plane_setup_clipping_and_scaling(state);
		1760	if (ret)
		1761	return ret;
		1762
		1763	if (!vc4_state->src_w[0] \|\| !vc4_state->src_h[0] \|\|
		1764	!vc4_state->crtc_w \|\| !vc4_state->crtc_h) {
		1765	/* 0 source size probably means the plane is offscreen.
		1766	* 0 destination size is a redundant plane.
		1767	*/
		1768	vc4_state->dlist_initialized = 1;
		1769	return 0;
		1770	}
		1771
		1772	width = vc4_state->src_w[0] >> 16;
		1773	height = vc4_state->src_h[0] >> 16;
		1774
		1775	/* SCL1 is used for Cb/Cr scaling of planar formats. For RGB
		1776	* and 4:4:4, scl1 should be set to scl0 so both channels of
		1777	* the scaler do the same thing. For YUV, the Y plane needs
		1778	* to be put in channel 1 and Cb/Cr in channel 0, so we swap
		1779	* the scl fields here.
		1780	*/
		1781	if (num_planes == 1) {
		1782	scl0 = vc4_get_scl_field(state, 0);
		1783	scl1 = scl0;
		1784	} else {
		1785	scl0 = vc4_get_scl_field(state, 1);
		1786	scl1 = vc4_get_scl_field(state, 0);
		1787	}
		1788
		1789	rotation = drm_rotation_simplify(state->rotation,
		1790	DRM_MODE_ROTATE_0 \|
		1791	DRM_MODE_REFLECT_X \|
		1792	DRM_MODE_REFLECT_Y);
		1793
		1794	/* We must point to the last line when Y reflection is enabled. */
		1795	src_y = vc4_state->src_y >> 16;
		1796	if (rotation & DRM_MODE_REFLECT_Y)
		1797	src_y += height - 1;
		1798
		1799	src_x = vc4_state->src_x >> 16;
		1800
		1801	switch (base_format_mod) {
		1802	case DRM_FORMAT_MOD_LINEAR:
		1803	tiling = SCALER6_CTL0_ADDR_MODE_LINEAR;
		1804
		1805	/* Adjust the base pointer to the first pixel to be scanned
		1806	* out.
		1807	*/
		1808	for (i = 0; i < num_planes; i++) {
		1809	offsets[i] += src_y / (i ? v_subsample : 1) * fb->pitches[i];
		1810	offsets[i] += src_x / (i ? h_subsample : 1) * fb->format->cpp[i];
		1811	}
		1812
		1813	break;
		1814
		1815	case DRM_FORMAT_MOD_BROADCOM_SAND128:
		1816	case DRM_FORMAT_MOD_BROADCOM_SAND256: {
		1817	uint32_t param = fourcc_mod_broadcom_param(fb->modifier);
		1818	u32 components_per_word;
		1819	u32 starting_offset;
		1820	u32 fetch_count;
		1821
		1822	if (param > SCALER_TILE_HEIGHT_MASK) {
		1823	DRM_DEBUG_KMS("SAND height too large (%d)\n",
		1824	param);
		1825	return -EINVAL;
		1826	}
		1827
		1828	if (fb->format->format == DRM_FORMAT_P030) {
		1829	hvs_format = HVS_PIXEL_FORMAT_YCBCR_10BIT;
		1830	tiling = SCALER6_CTL0_ADDR_MODE_128B;
		1831	} else {
		1832	hvs_format = HVS_PIXEL_FORMAT_YCBCR_YUV420_2PLANE;
		1833
		1834	switch (base_format_mod) {
		1835	case DRM_FORMAT_MOD_BROADCOM_SAND128:
		1836	tiling = SCALER6_CTL0_ADDR_MODE_128B;
		1837	break;
		1838	case DRM_FORMAT_MOD_BROADCOM_SAND256:
		1839	tiling = SCALER6_CTL0_ADDR_MODE_256B;
		1840	break;
		1841	default:
		1842	return -EINVAL;
		1843	}
		1844	}
		1845
		1846	/* Adjust the base pointer to the first pixel to be scanned
		1847	* out.
		1848	*
		1849	* For P030, y_ptr [31:4] is the 128bit word for the start pixel
		1850	* y_ptr [3:0] is the pixel (0-11) contained within that 128bit
		1851	* word that should be taken as the first pixel.
		1852	* Ditto uv_ptr [31:4] vs [3:0], however [3:0] contains the
		1853	* element within the 128bit word, eg for pixel 3 the value
		1854	* should be 6.
		1855	*/
		1856	for (i = 0; i < num_planes; i++) {
		1857	u32 tile_w, tile, x_off, pix_per_tile;
		1858
		1859	if (fb->format->format == DRM_FORMAT_P030) {
		1860	/*
		1861	* Spec says: bits [31:4] of the given address
		1862	* should point to the 128-bit word containing
		1863	* the desired starting pixel, and bits[3:0]
		1864	* should be between 0 and 11, indicating which
		1865	* of the 12-pixels in that 128-bit word is the
		1866	* first pixel to be used
		1867	*/
		1868	u32 remaining_pixels = src_x % 96;
		1869	u32 aligned = remaining_pixels / 12;
		1870	u32 last_bits = remaining_pixels % 12;
		1871
		1872	x_off = aligned * 16 + last_bits;
		1873	tile_w = 128;
		1874	pix_per_tile = 96;
		1875	} else {
		1876	switch (base_format_mod) {
		1877	case DRM_FORMAT_MOD_BROADCOM_SAND128:
		1878	tile_w = 128;
		1879	break;
		1880	case DRM_FORMAT_MOD_BROADCOM_SAND256:
		1881	tile_w = 256;
		1882	break;
		1883	default:
		1884	return -EINVAL;
		1885	}
		1886	pix_per_tile = tile_w / fb->format->cpp[0];
		1887	x_off = (src_x % pix_per_tile) /
		1888	(i ? h_subsample : 1) *
		1889	fb->format->cpp[i];
		1890	}
		1891
		1892	tile = src_x / pix_per_tile;
		1893
		1894	offsets[i] += param * tile_w * tile;
		1895	offsets[i] += src_y / (i ? v_subsample : 1) * tile_w;
		1896	offsets[i] += x_off & ~(i ? 1 : 0);
		1897	}
		1898
		1899	components_per_word = fb->format->format == DRM_FORMAT_P030 ? 24 : 32;
		1900	starting_offset = src_x % components_per_word;
		1901	fetch_count = (width + starting_offset + components_per_word - 1) /
		1902	components_per_word;
		1903
		1904	pitch0 = VC4_SET_FIELD(param, SCALER6_PTR2_PITCH) \|
		1905	VC4_SET_FIELD(fetch_count - 1, SCALER6_PTR2_FETCH_COUNT);
		1906	break;
		1907	}
		1908
		1909	default:
		1910	DRM_DEBUG_KMS("Unsupported FB tiling flag 0x%16llx",
		1911	(long long)fb->modifier);
		1912	return -EINVAL;
		1913	}
		1914
		1915	/* fetch an extra pixel if we don't actually line up with the left edge. */
		1916	if ((vc4_state->src_x & 0xffff) && vc4_state->src_x < (state->fb->width << 16))
		1917	width++;
		1918
		1919	/* same for the right side */
		1920	if (((vc4_state->src_x + vc4_state->src_w[0]) & 0xffff) &&
		1921	vc4_state->src_x + vc4_state->src_w[0] < (state->fb->width << 16))
		1922	width++;
		1923
		1924	/* now for the top */
		1925	if ((vc4_state->src_y & 0xffff) && vc4_state->src_y < (state->fb->height << 16))
		1926	height++;
		1927
		1928	/* and the bottom */
		1929	if (((vc4_state->src_y + vc4_state->src_h[0]) & 0xffff) &&
		1930	vc4_state->src_y + vc4_state->src_h[0] < (state->fb->height << 16))
		1931	height++;
		1932
		1933	/* for YUV444 hardware wants double the width, otherwise it doesn't
		1934	* fetch full width of chroma
		1935	*/
		1936	if (format->drm == DRM_FORMAT_YUV444 \|\| format->drm == DRM_FORMAT_YVU444)
		1937	width <<= 1;
		1938
		1939	/* Don't waste cycles mixing with plane alpha if the set alpha
		1940	* is opaque or there is no per-pixel alpha information.
		1941	* In any case we use the alpha property value as the fixed alpha.
		1942	*/
		1943	mix_plane_alpha = state->alpha != DRM_BLEND_ALPHA_OPAQUE &&
		1944	fb->format->has_alpha;
		1945
		1946	/* Control Word 0: Scaling Configuration & Element Validity*/
		1947	vc4_dlist_write(vc4_state,
		1948	SCALER6_CTL0_VALID \|
		1949	VC4_SET_FIELD(tiling, SCALER6_CTL0_ADDR_MODE) \|
		1950	vc4_hvs6_get_alpha_mask_mode(state) \|
		1951	(vc4_state->is_unity ? SCALER6_CTL0_UNITY : 0) \|
		1952	VC4_SET_FIELD(format->pixel_order_hvs5, SCALER6_CTL0_ORDERRGBA) \|
		1953	VC4_SET_FIELD(scl1, SCALER6_CTL0_SCL1_MODE) \|
		1954	VC4_SET_FIELD(scl0, SCALER6_CTL0_SCL0_MODE) \|
		1955	VC4_SET_FIELD(hvs_format, SCALER6_CTL0_PIXEL_FORMAT));
		1956
		1957	/* Position Word 0: Image Position */
		1958	vc4_state->pos0_offset = vc4_state->dlist_count;
		1959	vc4_dlist_write(vc4_state,
		1960	VC4_SET_FIELD(vc4_state->crtc_y, SCALER6_POS0_START_Y) \|
		1961	(rotation & DRM_MODE_REFLECT_X ? SCALER6_POS0_HFLIP : 0) \|
		1962	VC4_SET_FIELD(vc4_state->crtc_x, SCALER6_POS0_START_X));
		1963
		1964	/* Control Word 2: Alpha Value & CSC */
		1965	vc4_dlist_write(vc4_state,
		1966	vc6_plane_get_csc_mode(vc4_state) \|
		1967	vc4_hvs5_get_alpha_blend_mode(state) \|
		1968	(mix_plane_alpha ? SCALER6_CTL2_ALPHA_MIX : 0) \|
		1969	VC4_SET_FIELD(state->alpha >> 4, SCALER5_CTL2_ALPHA));
		1970
		1971	/* Position Word 1: Scaled Image Dimensions */
		1972	if (!vc4_state->is_unity)
		1973	vc4_dlist_write(vc4_state,
		1974	VC4_SET_FIELD(vc4_state->crtc_h - 1,
		1975	SCALER6_POS1_SCL_LINES) \|
		1976	VC4_SET_FIELD(vc4_state->crtc_w - 1,
		1977	SCALER6_POS1_SCL_WIDTH));
		1978
		1979	/* Position Word 2: Source Image Size */
		1980	vc4_state->pos2_offset = vc4_state->dlist_count;
		1981	vc4_dlist_write(vc4_state,
		1982	VC4_SET_FIELD(height - 1,
		1983	SCALER6_POS2_SRC_LINES) \|
		1984	VC4_SET_FIELD(width - 1,
		1985	SCALER6_POS2_SRC_WIDTH));
		1986
		1987	/* Position Word 3: Context */
		1988	vc4_dlist_write(vc4_state, 0xc0c0c0c0);
		1989
		1990	/*
		1991	* TODO: This only covers Raster Scan Order planes
		1992	*/
		1993	for (i = 0; i < num_planes; i++) {
		1994	struct drm_gem_dma_object *bo = drm_fb_dma_get_gem_obj(fb, i);
		1995	dma_addr_t paddr = bo->dma_addr + fb->offsets[i] + offsets[i];
		1996
		1997	/* Pointer Word 0 */
		1998	vc4_state->ptr0_offset[i] = vc4_state->dlist_count;
		1999	vc4_dlist_write(vc4_state,
		2000	(rotation & DRM_MODE_REFLECT_Y ? SCALER6_PTR0_VFLIP : 0) \|
		2001	/*
		2002	* The UPM buffer will be allocated in
		2003	* vc6_plane_allocate_upm().
		2004	*/
		2005	VC4_SET_FIELD(upper_32_bits(paddr) & 0xff,
		2006	SCALER6_PTR0_UPPER_ADDR));
		2007
		2008	/* Pointer Word 1 */
		2009	vc4_dlist_write(vc4_state, lower_32_bits(paddr));
		2010
		2011	/* Pointer Word 2 */
		2012	if (base_format_mod != DRM_FORMAT_MOD_BROADCOM_SAND128 &&
		2013	base_format_mod != DRM_FORMAT_MOD_BROADCOM_SAND256) {
		2014	vc4_dlist_write(vc4_state,
		2015	VC4_SET_FIELD(fb->pitches[i],
		2016	SCALER6_PTR2_PITCH));
		2017	} else {
		2018	vc4_dlist_write(vc4_state, pitch0);
		2019	}
		2020	}
		2021
		2022	/*
		2023	* Palette Word 0
		2024	* TODO: We're not using the palette mode
		2025	*/
		2026
		2027	/*
		2028	* Trans Word 0
		2029	* TODO: It's only relevant if we set the trans_rgb bit in the
		2030	* control word 0, and we don't at the moment.
		2031	*/
		2032
		2033	vc4_state->lbm_offset = 0;
		2034
		2035	if (!vc4_state->is_unity \|\| fb->format->is_yuv) {
		2036	/*
		2037	* Reserve a slot for the LBM Base Address. The real value will
		2038	* be set when calling vc4_plane_allocate_lbm().
		2039	*/
		2040	if (vc4_state->y_scaling[0] != VC4_SCALING_NONE \|\|
		2041	vc4_state->y_scaling[1] != VC4_SCALING_NONE) {
		2042	vc4_state->lbm_offset = vc4_state->dlist_count;
		2043	vc4_dlist_counter_increment(vc4_state);
		2044	}
		2045
		2046	if (vc4_state->x_scaling[0] != VC4_SCALING_NONE \|\|
		2047	vc4_state->x_scaling[1] != VC4_SCALING_NONE \|\|
		2048	vc4_state->y_scaling[0] != VC4_SCALING_NONE \|\|
		2049	vc4_state->y_scaling[1] != VC4_SCALING_NONE) {
		2050	if (num_planes > 1)
		2051	/*
		2052	* Emit Cb/Cr as channel 0 and Y as channel
		2053	* 1. This matches how we set up scl0/scl1
		2054	* above.
		2055	*/
		2056	vc4_write_scaling_parameters(state, 1);
		2057
		2058	vc4_write_scaling_parameters(state, 0);
		2059	}
		2060
		2061	/*
		2062	* If any PPF setup was done, then all the kernel
		2063	* pointers get uploaded.
		2064	*/
		2065	if (vc4_state->x_scaling[0] == VC4_SCALING_PPF \|\|
		2066	vc4_state->y_scaling[0] == VC4_SCALING_PPF \|\|
		2067	vc4_state->x_scaling[1] == VC4_SCALING_PPF \|\|
		2068	vc4_state->y_scaling[1] == VC4_SCALING_PPF) {
		2069	u32 kernel =
		2070	VC4_SET_FIELD(vc4->hvs->mitchell_netravali_filter.start,
		2071	SCALER_PPF_KERNEL_OFFSET);
		2072
		2073	/* HPPF plane 0 */
		2074	vc4_dlist_write(vc4_state, kernel);
		2075	/* VPPF plane 0 */
		2076	vc4_dlist_write(vc4_state, kernel);
		2077	/* HPPF plane 1 */
		2078	vc4_dlist_write(vc4_state, kernel);
		2079	/* VPPF plane 1 */
		2080	vc4_dlist_write(vc4_state, kernel);
		2081	}
		2082	}
		2083
		2084	vc4_dlist_write(vc4_state, SCALER6_CTL0_END);
		2085
		2086	vc4_state->dlist[0] \|=
		2087	VC4_SET_FIELD(vc4_state->dlist_count, SCALER6_CTL0_NEXT);
		2088
		2089	/* crtc_* are already clipped coordinates. */
		2090	covers_screen = vc4_state->crtc_x == 0 && vc4_state->crtc_y == 0 &&
		2091	vc4_state->crtc_w == state->crtc->mode.hdisplay &&
		2092	vc4_state->crtc_h == state->crtc->mode.vdisplay;
		2093
		2094	/*
		2095	* Background fill might be necessary when the plane has per-pixel
		2096	* alpha content or a non-opaque plane alpha and could blend from the
		2097	* background or does not cover the entire screen.
		2098	*/
		2099	vc4_state->needs_bg_fill = fb->format->has_alpha \|\| !covers_screen \|\|
		2100	state->alpha != DRM_BLEND_ALPHA_OPAQUE;
		2101
		2102	/*
		2103	* Flag the dlist as initialized to avoid checking it twice in case
		2104	* the async update check already called vc4_plane_mode_set() and
		2105	* decided to fallback to sync update because async update was not
		2106	* possible.
		2107	*/
		2108	vc4_state->dlist_initialized = 1;
		2109
		2110	vc4_plane_calc_load(state);
		2111
		2112	drm_dbg_driver(drm, "[PLANE:%d:%s] Computed DLIST size: %u\n",
		2113	plane->base.id, plane->name, vc4_state->dlist_count);
		2114
		2115	return 0;
		2116	}
		2117
1366	/* If a modeset involves changing the setup of a plane, the atomic	2118	/* If a modeset involves changing the setup of a plane, the atomic
1367	* infrastructure will call this to validate a proposed plane setup.	2119	* infrastructure will call this to validate a proposed plane setup.
1368	* However, if a plane isn't getting updated, this (and the	2120	* However, if a plane isn't getting updated, this (and the
@@ -1373,6 +2125,7 @@ static int vc4_plane_mode_set(struct drm_plane *plane, · 4 unmodified lines
1373	static int vc4_plane_atomic_check(struct drm_plane *plane,	2125	static int vc4_plane_atomic_check(struct drm_plane *plane,
1374	struct drm_atomic_state *state)	2126	struct drm_atomic_state *state)
1375	{	2127	{
		2128	struct vc4_dev *vc4 = to_vc4_dev(plane->dev);
1376	struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,	2129	struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
1377	plane);	2130	plane);
1378	struct vc4_plane_state *vc4_state = to_vc4_plane_state(new_plane_state);	2131	struct vc4_plane_state *vc4_state = to_vc4_plane_state(new_plane_state);
@@ -1380,17 +2133,38 @@ static int vc4_plane_atomic_check(struct drm_plane *plane, · 1 unmodified line
1380		2133
1381	vc4_state->dlist_count = 0;	2134	vc4_state->dlist_count = 0;
1382		2135
1383	if (!plane_enabled(new_plane_state))	2136	if (!plane_enabled(new_plane_state)) {
		2137	struct drm_plane_state *old_plane_state =
		2138	drm_atomic_get_old_plane_state(state, plane);
		2139
		2140	if (vc4->gen >= VC4_GEN_6_C && old_plane_state &&
		2141	plane_enabled(old_plane_state)) {
		2142	vc6_plane_free_upm(new_plane_state);
		2143	}
1384	return 0;	2144	return 0;
		2145	}
1385		2146
1386	ret = vc4_plane_mode_set(plane, new_plane_state);	2147	if (vc4->gen >= VC4_GEN_6_C)
		2148	ret = vc6_plane_mode_set(plane, new_plane_state);
		2149	else
		2150	ret = vc4_plane_mode_set(plane, new_plane_state);
1387	if (ret)	2151	if (ret)
1388	return ret;	2152	return ret;
1389		2153
		2154	if (!vc4_state->src_w[0] \|\| !vc4_state->src_h[0] \|\|
		2155	!vc4_state->crtc_w \|\| !vc4_state->crtc_h)
		2156	return 0;
		2157
1390	ret = vc4_plane_allocate_lbm(new_plane_state);	2158	ret = vc4_plane_allocate_lbm(new_plane_state);
1391	if (ret)	2159	if (ret)
1392	return ret;	2160	return ret;
1393		2161
		2162	if (vc4->gen >= VC4_GEN_6_C) {
		2163	ret = vc6_plane_allocate_upm(new_plane_state);
		2164	if (ret)
		2165	return ret;
		2166	}
		2167
1394	return 0;	2168	return 0;
1395	}	2169	}
1396		2170
@@ -1439,7 +2213,8 @@ void vc4_plane_async_set_fb(struct drm_plane plane, struct drm_framebuffer fb) · 42 unmodified lines
1439	{	2213	{
1440	struct vc4_plane_state *vc4_state = to_vc4_plane_state(plane->state);	2214	struct vc4_plane_state *vc4_state = to_vc4_plane_state(plane->state);
1441	struct drm_gem_dma_object *bo = drm_fb_dma_get_gem_obj(fb, 0);	2215	struct drm_gem_dma_object *bo = drm_fb_dma_get_gem_obj(fb, 0);
1442	uint32_t addr;	2216	struct vc4_dev *vc4 = to_vc4_dev(plane->dev);
		2217	dma_addr_t dma_addr = bo->dma_addr + fb->offsets[0];
1443	int idx;	2218	int idx;
1444		2219
1445	if (!drm_dev_enter(plane->dev, &idx))	2220	if (!drm_dev_enter(plane->dev, &idx))
@@ -1449,19 +2224,38 @@ void vc4_plane_async_set_fb(struct drm_plane plane, struct drm_framebuffer fb) · 3 unmodified lines
1449	* because this is only called on the primary plane.	2224	* because this is only called on the primary plane.
1450	*/	2225	*/
1451	WARN_ON_ONCE(plane->state->crtc_x < 0 \|\| plane->state->crtc_y < 0);	2226	WARN_ON_ONCE(plane->state->crtc_x < 0 \|\| plane->state->crtc_y < 0);
1452	addr = bo->dma_addr + fb->offsets[0];
1453		2227
1454	/* Write the new address into the hardware immediately. The	2228	if (vc4->gen == VC4_GEN_6_C) {
1455	* scanout will start from this address as soon as the FIFO	2229	u32 value;
1456	* needs to refill with pixels.
1457	*/
1458	writel(addr, &vc4_state->hw_dlist[vc4_state->ptr0_offset[0]]);
1459		2230
1460	/* Also update the CPU-side dlist copy, so that any later	2231	value = vc4_state->dlist[vc4_state->ptr0_offset[0]] &
1461	* atomic updates that don't do a new modeset on our plane	2232	~SCALER6_PTR0_UPPER_ADDR_MASK;
1462	* also use our updated address.	2233	value \|= VC4_SET_FIELD(upper_32_bits(dma_addr) & 0xff,
1463	*/	2234	SCALER6_PTR0_UPPER_ADDR);
1464	vc4_state->dlist[vc4_state->ptr0_offset[0]] = addr;	2235
		2236	writel(value, &vc4_state->hw_dlist[vc4_state->ptr0_offset[0]]);
		2237	vc4_state->dlist[vc4_state->ptr0_offset[0]] = value;
		2238
		2239	value = lower_32_bits(dma_addr);
		2240	writel(value, &vc4_state->hw_dlist[vc4_state->ptr0_offset[0] + 1]);
		2241	vc4_state->dlist[vc4_state->ptr0_offset[0] + 1] = value;
		2242	} else {
		2243	u32 addr;
		2244
		2245	addr = (u32)dma_addr;
		2246
		2247	/* Write the new address into the hardware immediately. The
		2248	* scanout will start from this address as soon as the FIFO
		2249	* needs to refill with pixels.
		2250	*/
		2251	writel(addr, &vc4_state->hw_dlist[vc4_state->ptr0_offset[0]]);
		2252
		2253	/* Also update the CPU-side dlist copy, so that any later
		2254	* atomic updates that don't do a new modeset on our plane
		2255	* also use our updated address.
		2256	*/
		2257	vc4_state->dlist[vc4_state->ptr0_offset[0]] = addr;
		2258	}
1465		2259
1466	drm_dev_exit(idx);	2260	drm_dev_exit(idx);
1467	}	2261	}
@@ -1543,13 +2337,17 @@ static void vc4_plane_atomic_async_update(struct drm_plane *plane, · 75 unmodified lines
1543	static int vc4_plane_atomic_async_check(struct drm_plane *plane,	2337	static int vc4_plane_atomic_async_check(struct drm_plane *plane,
1544	struct drm_atomic_state *state)	2338	struct drm_atomic_state *state)
1545	{	2339	{
		2340	struct vc4_dev *vc4 = to_vc4_dev(plane->dev);
1546	struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,	2341	struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
1547	plane);	2342	plane);
1548	struct vc4_plane_state old_vc4_state, new_vc4_state;	2343	struct vc4_plane_state old_vc4_state, new_vc4_state;
1549	int ret;	2344	int ret;
1550	u32 i;	2345	u32 i;
1551		2346
1552	ret = vc4_plane_mode_set(plane, new_plane_state);	2347	if (vc4->gen <= VC4_GEN_5)
		2348	ret = vc4_plane_mode_set(plane, new_plane_state);
		2349	else
		2350	ret = vc6_plane_mode_set(plane, new_plane_state);
1553	if (ret)	2351	if (ret)
1554	return ret;	2352	return ret;
1555		2353
@@ -1723,7 +2521,7 @@ struct drm_plane vc4_plane_init(struct drm_device dev, · 167 unmodified lines
1723	};	2521	};
1724		2522
1725	for (i = 0; i < ARRAY_SIZE(hvs_formats); i++) {	2523	for (i = 0; i < ARRAY_SIZE(hvs_formats); i++) {
1726	if (!hvs_formats[i].hvs5_only \|\| vc4->gen == VC4_GEN_5) {	2524	if (!hvs_formats[i].hvs5_only \|\| vc4->gen >= VC4_GEN_5) {
1727	formats[num_formats] = hvs_formats[i].drm;	2525	formats[num_formats] = hvs_formats[i].drm;
1728	num_formats++;	2526	num_formats++;
1729	}	2527	}
@@ -1738,7 +2536,7 @@ struct drm_plane vc4_plane_init(struct drm_device dev, · 8 unmodified lines
1738	return ERR_CAST(vc4_plane);	2536	return ERR_CAST(vc4_plane);
1739	plane = &vc4_plane->base;	2537	plane = &vc4_plane->base;
1740		2538
1741	if (vc4->gen == VC4_GEN_5)	2539	if (vc4->gen >= VC4_GEN_5)
1742	drm_plane_helper_add(plane, &vc5_plane_helper_funcs);	2540	drm_plane_helper_add(plane, &vc5_plane_helper_funcs);
1743	else	2541	else
1744	drm_plane_helper_add(plane, &vc4_plane_helper_funcs);	2542	drm_plane_helper_add(plane, &vc4_plane_helper_funcs);

drivers/gpu/drm/vc4/vc4_regs.h+297−0

@@ -19,6 +19,20 @@ · 18 unmodified lines
19		19
20	#define VC4_GET_FIELD(word, field) FIELD_GET(field##_MASK, word)	20	#define VC4_GET_FIELD(word, field) FIELD_GET(field##_MASK, word)
21		21
		22	#define VC6_SET_FIELD(value, field) \
		23	({ \
		24	WARN_ON(!FIELD_FIT(hvs->vc4->gen == VC4_GEN_6_C ? \
		25	SCALER6_ ## field ## _MASK : \
		26	SCALER6D_ ## field ## _MASK, value));\
		27	FIELD_PREP(hvs->vc4->gen == VC4_GEN_6_C ? \
		28	SCALER6_ ## field ## _MASK : \
		29	SCALER6D_ ## field ## _MASK, value); \
		30	})
		31
		32	#define VC6_GET_FIELD(word, field) FIELD_GET(hvs->vc4->gen == VC4_GEN_6_C ? \
		33	SCALER6_ ## field ## _MASK : \
		34	SCALER6D_ ## field ## _MASK, word)
		35
22	#define V3D_IDENT0 0x00000	36	#define V3D_IDENT0 0x00000
23	# define V3D_EXPECTED_IDENT0 \	37	# define V3D_EXPECTED_IDENT0 \
24	((2 << 24) \| \	38	((2 << 24) \| \
@@ -155,6 +169,7 @@ · 130 unmodified lines
155	# define PV_CONTROL_EN BIT(0)	169	# define PV_CONTROL_EN BIT(0)
156		170
157	#define PV_V_CONTROL 0x04	171	#define PV_V_CONTROL 0x04
		172	# define PV_VCONTROL_ODD_TIMING BIT(29)
158	# define PV_VCONTROL_ODD_DELAY_MASK VC4_MASK(22, 6)	173	# define PV_VCONTROL_ODD_DELAY_MASK VC4_MASK(22, 6)
159	# define PV_VCONTROL_ODD_DELAY_SHIFT 6	174	# define PV_VCONTROL_ODD_DELAY_SHIFT 6
160	# define PV_VCONTROL_ODD_FIRST BIT(5)	175	# define PV_VCONTROL_ODD_FIRST BIT(5)
@@ -215,6 +230,11 @@ · 54 unmodified lines
215	# define PV_MUX_CFG_RGB_PIXEL_MUX_MODE_SHIFT 2	230	# define PV_MUX_CFG_RGB_PIXEL_MUX_MODE_SHIFT 2
216	# define PV_MUX_CFG_RGB_PIXEL_MUX_MODE_NO_SWAP 8	231	# define PV_MUX_CFG_RGB_PIXEL_MUX_MODE_NO_SWAP 8
217		232
		233	#define PV_PIPE_INIT_CTRL 0x94
		234	# define PV_PIPE_INIT_CTRL_PV_INIT_WIDTH_MASK VC4_MASK(11, 8)
		235	# define PV_PIPE_INIT_CTRL_PV_INIT_IDLE_MASK VC4_MASK(7, 4)
		236	# define PV_PIPE_INIT_CTRL_PV_INIT_EN BIT(0)
		237
218	#define SCALER_CHANNELS_COUNT 3	238	#define SCALER_CHANNELS_COUNT 3
219		239
220	#define SCALER_DISPCTRL 0x00000000	240	#define SCALER_DISPCTRL 0x00000000
@@ -418,6 +438,10 @@ · 197 unmodified lines
418	# define SCALER_DISPSTAT1_FRCNT0_SHIFT 18	438	# define SCALER_DISPSTAT1_FRCNT0_SHIFT 18
419	# define SCALER_DISPSTAT1_FRCNT1_MASK VC4_MASK(17, 12)	439	# define SCALER_DISPSTAT1_FRCNT1_MASK VC4_MASK(17, 12)
420	# define SCALER_DISPSTAT1_FRCNT1_SHIFT 12	440	# define SCALER_DISPSTAT1_FRCNT1_SHIFT 12
		441	# define SCALER5_DISPSTAT1_FRCNT0_MASK VC4_MASK(25, 20)
		442	# define SCALER5_DISPSTAT1_FRCNT0_SHIFT 20
		443	# define SCALER5_DISPSTAT1_FRCNT1_MASK VC4_MASK(19, 14)
		444	# define SCALER5_DISPSTAT1_FRCNT1_SHIFT 14
421		445
422	#define SCALER_DISPSTATX(x) (SCALER_DISPSTAT0 + \	446	#define SCALER_DISPSTATX(x) (SCALER_DISPSTAT0 + \
423	(x) * (SCALER_DISPSTAT1 - \	447	(x) * (SCALER_DISPSTAT1 - \
@@ -436,6 +460,8 @@ · 12 unmodified lines
436	#define SCALER_DISPSTAT2 0x00000068	460	#define SCALER_DISPSTAT2 0x00000068
437	# define SCALER_DISPSTAT2_FRCNT2_MASK VC4_MASK(17, 12)	461	# define SCALER_DISPSTAT2_FRCNT2_MASK VC4_MASK(17, 12)
438	# define SCALER_DISPSTAT2_FRCNT2_SHIFT 12	462	# define SCALER_DISPSTAT2_FRCNT2_SHIFT 12
		463	# define SCALER5_DISPSTAT2_FRCNT2_MASK VC4_MASK(19, 14)
		464	# define SCALER5_DISPSTAT2_FRCNT2_SHIFT 14
439		465
440	#define SCALER_DISPBASE2 0x0000006c	466	#define SCALER_DISPBASE2 0x0000006c
441	#define SCALER_DISPALPHA2 0x00000070	467	#define SCALER_DISPALPHA2 0x00000070
@@ -514,6 +540,206 @@ · 72 unmodified lines
514		540
515	#define SCALER5_DLIST_START 0x00004000	541	#define SCALER5_DLIST_START 0x00004000
516		542
		543	#define SCALER6_VERSION 0x00000000
		544	# define SCALER6_VERSION_MASK VC4_MASK(7, 0)
		545	# define SCALER6_VERSION_C0 0x00000053
		546	# define SCALER6_VERSION_D0 0x00000054
		547	#define SCALER6_CXM_SIZE 0x00000004
		548	#define SCALER6_LBM_SIZE 0x00000008
		549	#define SCALER6_UBM_SIZE 0x0000000c
		550	#define SCALER6_COBA_SIZE 0x00000010
		551	#define SCALER6_COB_SIZE 0x00000014
		552
		553	#define SCALER6_CONTROL 0x00000020
		554	# define SCALER6_CONTROL_HVS_EN BIT(31)
		555	# define SCALER6_CONTROL_PF_LINES_MASK VC4_MASK(22, 18)
		556	# define SCALER6_CONTROL_ABORT_ON_EMPTY BIT(16)
		557	# define SCALER6_CONTROL_DSP1_TARGET_MASK VC4_MASK(13, 12)
		558	# define SCALER6_CONTROL_MAX_REQS_MASK VC4_MASK(7, 4)
		559
		560	#define SCALER6_FETCHER_STATUS 0x00000024
		561	#define SCALER6_FETCH_STATUS 0x00000028
		562	#define SCALER6_HANDLE_ERROR 0x0000002c
		563
		564	#define SCALER6_DISP0_CTRL0 0x00000030
		565	#define SCALER6_DISPX_CTRL0(x) ((hvs->vc4->gen == VC4_GEN_6_C) ? \
		566	(SCALER6_DISP0_CTRL0 + ((x) * (SCALER6_DISP1_CTRL0 - SCALER6_DISP0_CTRL0))) : \
		567	(SCALER6D_DISP0_CTRL0 + ((x) * (SCALER6D_DISP1_CTRL0 - SCALER6D_DISP0_CTRL0))))
		568	# define SCALER6_DISPX_CTRL0_ENB BIT(31)
		569	# define SCALER6_DISPX_CTRL0_RESET BIT(30)
		570	# define SCALER6_DISPX_CTRL0_FWIDTH_MASK VC4_MASK(28, 16)
		571	# define SCALER6_DISPX_CTRL0_ONESHOT BIT(15)
		572	# define SCALER6_DISPX_CTRL0_ONECTX_MASK VC4_MASK(14, 13)
		573	# define SCALER6_DISPX_CTRL0_LINES_MASK VC4_MASK(12, 0)
		574
		575	#define SCALER6_DISP0_CTRL1 0x00000034
		576	#define SCALER6_DISPX_CTRL1(x) ((hvs->vc4->gen == VC4_GEN_6_C) ? \
		577	(SCALER6_DISP0_CTRL1 + ((x) * (SCALER6_DISP1_CTRL1 - SCALER6_DISP0_CTRL1))) : \
		578	(SCALER6D_DISP0_CTRL1 + ((x) * (SCALER6D_DISP1_CTRL1 - SCALER6D_DISP0_CTRL1))))
		579	# define SCALER6_DISPX_CTRL1_BGENB BIT(8)
		580	# define SCALER6_DISPX_CTRL1_INTLACE BIT(0)
		581
		582	#define SCALER6_DISP0_BGND 0x00000038
		583	#define SCALER6_DISPX_BGND(x) ((hvs->vc4->gen == VC4_GEN_6_C) ? \
		584	(SCALER6_DISP0_BGND + ((x) * (SCALER6_DISP1_BGND - SCALER6_DISP0_BGND))) : \
		585	(SCALER6D_DISP0_BGND + ((x) * (SCALER6D_DISP1_BGND - SCALER6D_DISP0_BGND))))
		586
		587	#define SCALER6_DISP0_LPTRS 0x0000003c
		588	#define SCALER6_DISPX_LPTRS(x) ((hvs->vc4->gen == VC4_GEN_6_C) ? \
		589	(SCALER6_DISP0_LPTRS + ((x) * (SCALER6_DISP1_LPTRS - SCALER6_DISP0_LPTRS))) : \
		590	(SCALER6D_DISP0_LPTRS + ((x) * (SCALER6D_DISP1_LPTRS - SCALER6D_DISP0_LPTRS))))
		591	# define SCALER6_DISPX_LPTRS_HEADE_MASK VC4_MASK(11, 0)
		592
		593	#define SCALER6_DISP0_COB 0x00000040
		594	#define SCALER6_DISPX_COB(x) ((hvs->vc4->gen == VC4_GEN_6_C) ? \
		595	(SCALER6_DISP0_COB + ((x) * (SCALER6_DISP1_COB - SCALER6_DISP0_COB))) : \
		596	(SCALER6D_DISP0_COB + ((x) * (SCALER6D_DISP1_COB - SCALER6D_DISP0_COB))))
		597	# define SCALER6_DISPX_COB_TOP_MASK VC4_MASK(31, 16)
		598	# define SCALER6_DISPX_COB_BASE_MASK VC4_MASK(15, 0)
		599
		600	#define SCALER6_DISP0_STATUS 0x00000044
		601	#define SCALER6_DISPX_STATUS(x) ((hvs->vc4->gen == VC4_GEN_6_C) ? \
		602	(SCALER6_DISP0_STATUS + ((x) * (SCALER6_DISP1_STATUS - SCALER6_DISP0_STATUS))) : \
		603	(SCALER6D_DISP0_STATUS + ((x) * (SCALER6D_DISP1_STATUS - SCALER6D_DISP0_STATUS))))
		604	# define SCALER6_DISPX_STATUS_EMPTY BIT(22)
		605	# define SCALER6_DISPX_STATUS_FRCNT_MASK VC4_MASK(21, 16)
		606	# define SCALER6_DISPX_STATUS_OFIELD BIT(15)
		607	# define SCALER6_DISPX_STATUS_MODE_MASK VC4_MASK(14, 13)
		608	# define SCALER6_DISPX_STATUS_MODE_DISABLED 0
		609	# define SCALER6_DISPX_STATUS_MODE_INIT 1
		610	# define SCALER6_DISPX_STATUS_MODE_RUN 2
		611	# define SCALER6_DISPX_STATUS_MODE_EOF 3
		612	# define SCALER6_DISPX_STATUS_YLINE_MASK VC4_MASK(12, 0)
		613
		614	#define SCALER6_DISP0_DL 0x00000048
		615
		616	#define SCALER6_DISPX_DL(x) ((hvs->vc4->gen == VC4_GEN_6_C) ? \
		617	(SCALER6_DISP0_DL + ((x) * (SCALER6_DISP1_DL - SCALER6_DISP0_DL))) : \
		618	(SCALER6D_DISP0_DL + ((x) * (SCALER6D_DISP1_DL - SCALER6D_DISP0_DL))))
		619	# define SCALER6_DISPX_DL_LACT_MASK VC4_MASK(11, 0)
		620
		621	#define SCALER6_DISP0_RUN 0x0000004c
		622	#define SCALER6_DISP1_CTRL0 0x00000050
		623	#define SCALER6_DISP1_CTRL1 0x00000054
		624	#define SCALER6_DISP1_BGND 0x00000058
		625	#define SCALER6_DISP1_LPTRS 0x0000005c
		626	#define SCALER6_DISP1_COB 0x00000060
		627	#define SCALER6_DISP1_STATUS 0x00000064
		628	#define SCALER6_DISP1_DL 0x00000068
		629	#define SCALER6_DISP1_RUN 0x0000006c
		630	#define SCALER6_DISP2_CTRL0 0x00000070
		631	#define SCALER6_DISP2_CTRL1 0x00000074
		632	#define SCALER6_DISP2_BGND 0x00000078
		633	#define SCALER6_DISP2_LPTRS 0x0000007c
		634	#define SCALER6_DISP2_COB 0x00000080
		635	#define SCALER6_DISP2_STATUS 0x00000084
		636	#define SCALER6_DISP2_DL 0x00000088
		637	#define SCALER6_DISP2_RUN 0x0000008c
		638	#define SCALER6_EOLN 0x00000090
		639	#define SCALER6_DL_STATUS 0x00000094
		640	#define SCALER6_BFG_MISC 0x0000009c
		641	#define SCALER6_QOS0 0x000000a0
		642	#define SCALER6_PROF0 0x000000a4
		643	#define SCALER6_QOS1 0x000000a8
		644	#define SCALER6_PROF1 0x000000ac
		645	#define SCALER6_QOS2 0x000000b0
		646	#define SCALER6_PROF2 0x000000b4
		647	#define SCALER6_PRI_MAP0 0x000000b8
		648	#define SCALER6_PRI_MAP1 0x000000bc
		649	#define SCALER6_HISTCTRL 0x000000c0
		650	#define SCALER6_HISTBIN0 0x000000c4
		651	#define SCALER6_HISTBIN1 0x000000c8
		652	#define SCALER6_HISTBIN2 0x000000cc
		653	#define SCALER6_HISTBIN3 0x000000d0
		654	#define SCALER6_HISTBIN4 0x000000d4
		655	#define SCALER6_HISTBIN5 0x000000d8
		656	#define SCALER6_HISTBIN6 0x000000dc
		657	#define SCALER6_HISTBIN7 0x000000e0
		658	#define SCALER6_HDR_CFG_REMAP 0x000000f4
		659	#define SCALER6_COL_SPACE 0x000000f8
		660	#define SCALER6_HVS_ID 0x000000fc
		661	#define SCALER6_CFC1 0x00000100
		662	#define SCALER6_DISP_UPM_ISO0 0x00000200
		663	#define SCALER6_DISP_UPM_ISO1 0x00000204
		664	#define SCALER6_DISP_UPM_ISO2 0x00000208
		665	#define SCALER6_DISP_LBM_ISO0 0x0000020c
		666	#define SCALER6_DISP_LBM_ISO1 0x00000210
		667	#define SCALER6_DISP_LBM_ISO2 0x00000214
		668	#define SCALER6_DISP_COB_ISO0 0x00000218
		669	#define SCALER6_DISP_COB_ISO1 0x0000021c
		670	#define SCALER6_DISP_COB_ISO2 0x00000220
		671	#define SCALER6_BAD_COB 0x00000224
		672	#define SCALER6_BAD_LBM 0x00000228
		673	#define SCALER6_BAD_UPM 0x0000022c
		674	#define SCALER6_BAD_AXI 0x00000230
		675
		676	#define SCALER6D_VERSION 0x00000000
		677	#define SCALER6D_CXM_SIZE 0x00000004
		678	#define SCALER6D_LBM_SIZE 0x00000008
		679	#define SCALER6D_UBM_SIZE 0x0000000c
		680	#define SCALER6D_COBA_SIZE 0x00000010
		681	#define SCALER6D_COB_SIZE 0x00000014
		682	#define SCALER6D_CONTROL 0x00000020
		683	#define SCALER6D_FETCHER_STATUS 0x00000024
		684	#define SCALER6D_FETCH_STATUS 0x00000028
		685	#define SCALER6D_HANDLE_ERROR 0x0000002c
		686	#define SCALER6D_EOLN 0x00000030
		687	#define SCALER6D_DL_STATUS 0x00000034
		688	#define SCALER6D_PRI_MAP0 0x00000038
		689	#define SCALER6D_PRI_MAP1 0x0000003c
		690	#define SCALER6D_HISTCTRL 0x000000d0
		691	#define SCALER6D_HISTBIN0 0x000000d4
		692	#define SCALER6D_HISTBIN1 0x000000d8
		693	#define SCALER6D_HISTBIN2 0x000000dc
		694	#define SCALER6D_HISTBIN3 0x000000e0
		695	#define SCALER6D_HISTBIN4 0x000000e4
		696	#define SCALER6D_HISTBIN5 0x000000e8
		697	#define SCALER6D_HISTBIN6 0x000000ec
		698	#define SCALER6D_HISTBIN7 0x000000f0
		699	#define SCALER6D_HVS_ID 0x000000fc
		700
		701	#define SCALER6D_DISP0_CTRL0 0x00000100
		702	#define SCALER6D_DISP0_CTRL1 0x00000104
		703	#define SCALER6D_DISP0_BGND 0x00000108
		704	#define SCALER6D_DISP0_LPTRS 0x00000110
		705	#define SCALER6D_DISP0_COB 0x00000114
		706	#define SCALER6D_DISP0_STATUS 0x00000118
		707	#define SCALER6D_DISP0_CTRL0 0x00000100
		708	#define SCALER6D_DISP0_CTRL1 0x00000104
		709	#define SCALER6D_DISP0_BGND0 0x00000108
		710	#define SCALER6D_DISP0_BGND1 0x0000010c
		711	#define SCALER6D_DISP0_LPTRS 0x00000110
		712	#define SCALER6D_DISP0_COB 0x00000114
		713	#define SCALER6D_DISP0_STATUS 0x00000118
		714	#define SCALER6D_DISP0_DL 0x0000011c
		715	#define SCALER6D_DISP0_RUN 0x00000120
		716	#define SCALER6D_QOS0 0x00000124
		717	#define SCALER6D_PROF0 0x00000128
		718	#define SCALER6D_DISP1_CTRL0 0x00000140
		719	#define SCALER6D_DISP1_CTRL1 0x00000144
		720	#define SCALER6D_DISP1_BGND0 0x00000148
		721	#define SCALER6D_DISP1_BGND1 0x0000014c
		722	#define SCALER6D_DISP1_LPTRS 0x00000150
		723	#define SCALER6D_DISP1_COB 0x00000154
		724	#define SCALER6D_DISP1_STATUS 0x00000158
		725	#define SCALER6D_DISP1_DL 0x0000015c
		726	#define SCALER6D_DISP1_RUN 0x00000160
		727	#define SCALER6D_QOS1 0x00000164
		728	#define SCALER6D_PROF1 0x00000168
		729	#define SCALER6D_DISP2_CTRL0 0x00000180
		730	#define SCALER6D_DISP2_CTRL1 0x00000184
		731	#define SCALER6D_DISP2_BGND0 0x00000188
		732	#define SCALER6D_DISP2_BGND1 0x0000018c
		733	#define SCALER6D_DISP2_LPTRS 0x00000190
		734	#define SCALER6D_DISP2_COB 0x00000194
		735	#define SCALER6D_DISP2_STATUS 0x00000198
		736	#define SCALER6D_DISP2_DL 0x0000019c
		737	#define SCALER6D_DISP2_RUN 0x000001a0
		738	#define SCALER6D_QOS2 0x000001a4
		739	#define SCALER6D_PROF2 0x000001a8
		740
		741	#define SCALER6(x) ((hvs->vc4->gen == VC4_GEN_6_C) ? SCALER6_ ## x : SCALER6D_ ## x)
		742
517	# define VC4_HDMI_SW_RESET_FORMAT_DETECT BIT(1)	743	# define VC4_HDMI_SW_RESET_FORMAT_DETECT BIT(1)
518	# define VC4_HDMI_SW_RESET_HDMI BIT(0)	744	# define VC4_HDMI_SW_RESET_HDMI BIT(0)
519		745
@@ -761,6 +987,15 @@ enum { · 241 unmodified lines
761	# define VC4_HD_MAI_THR_DREQLOW_MASK VC4_MASK(5, 0)	987	# define VC4_HD_MAI_THR_DREQLOW_MASK VC4_MASK(5, 0)
762	# define VC4_HD_MAI_THR_DREQLOW_SHIFT 0	988	# define VC4_HD_MAI_THR_DREQLOW_SHIFT 0
763		989
		990	# define VC6_D_HD_MAI_THR_PANICHIGH_MASK VC4_MASK(29, 23)
		991	# define VC6_D_HD_MAI_THR_PANICHIGH_SHIFT 23
		992	# define VC6_D_HD_MAI_THR_PANICLOW_MASK VC4_MASK(21, 15)
		993	# define VC6_D_HD_MAI_THR_PANICLOW_SHIFT 15
		994	# define VC6_D_HD_MAI_THR_DREQHIGH_MASK VC4_MASK(13, 7)
		995	# define VC6_D_HD_MAI_THR_DREQHIGH_SHIFT 7
		996	# define VC6_D_HD_MAI_THR_DREQLOW_MASK VC4_MASK(6, 0)
		997	# define VC6_D_HD_MAI_THR_DREQLOW_SHIFT 0
		998
764	/* Divider from HDMI HSM clock to MAI serial clock. Sampling period	999	/* Divider from HDMI HSM clock to MAI serial clock. Sampling period
765	* converges to N / (M + 1) cycles.	1000	* converges to N / (M + 1) cycles.
766	*/	1001	*/
@@ -968,6 +1203,9 @@ enum hvs_pixel_format { · 201 unmodified lines
968	#define SCALER5_CTL2_ALPHA_MASK VC4_MASK(15, 4)	1203	#define SCALER5_CTL2_ALPHA_MASK VC4_MASK(15, 4)
969	#define SCALER5_CTL2_ALPHA_SHIFT 4	1204	#define SCALER5_CTL2_ALPHA_SHIFT 4
970		1205
		1206	#define SCALER6D_CTL2_CSC_ENABLE BIT(19)
		1207	#define SCALER6D_CTL2_BRCM_CFC_CONTROL_MASK VC4_MASK(22, 20)
		1208
971	#define SCALER_POS1_SCL_HEIGHT_MASK VC4_MASK(27, 16)	1209	#define SCALER_POS1_SCL_HEIGHT_MASK VC4_MASK(27, 16)
972	#define SCALER_POS1_SCL_HEIGHT_SHIFT 16	1210	#define SCALER_POS1_SCL_HEIGHT_SHIFT 16
973		1211
@@ -1109,4 +1347,63 @@ enum hvs_pixel_format { · 135 unmodified lines
1109	#define SCALER_PITCH0_TILE_WIDTH_R_MASK VC4_MASK(6, 0)	1347	#define SCALER_PITCH0_TILE_WIDTH_R_MASK VC4_MASK(6, 0)
1110	#define SCALER_PITCH0_TILE_WIDTH_R_SHIFT 0	1348	#define SCALER_PITCH0_TILE_WIDTH_R_SHIFT 0
1111		1349
		1350	#define SCALER6_CTL0_END BIT(31)
		1351	#define SCALER6_CTL0_VALID BIT(30)
		1352	#define SCALER6_CTL0_NEXT_MASK VC4_MASK(29, 24)
		1353	#define SCALER6_CTL0_RGB_TRANS BIT(23)
		1354	#define SCALER6_CTL0_ADDR_MODE_MASK VC4_MASK(22, 20)
		1355	#define SCALER6_CTL0_ADDR_MODE_LINEAR 0
		1356	#define SCALER6_CTL0_ADDR_MODE_128B 1
		1357	#define SCALER6_CTL0_ADDR_MODE_256B 2
		1358	#define SCALER6_CTL0_ADDR_MODE_MAP8 3
		1359	#define SCALER6_CTL0_ADDR_MODE_UIF 4
		1360
		1361	#define SCALER6_CTL0_ALPHA_MASK_MASK VC4_MASK(19, 18)
		1362	#define SCALER6_CTL0_ALPHA_MASK_NONE 0
		1363	#define SCALER6D_CTL0_ALPHA_MASK_FIXED 3
		1364	#define SCALER6_CTL0_UNITY BIT(15)
		1365	#define SCALER6_CTL0_ORDERRGBA_MASK VC4_MASK(14, 13)
		1366	#define SCALER6_CTL0_SCL1_MODE_MASK VC4_MASK(10, 8)
		1367	#define SCALER6_CTL0_SCL0_MODE_MASK VC4_MASK(7, 5)
		1368	#define SCALER6_CTL0_PIXEL_FORMAT_MASK VC4_MASK(4, 0)
		1369
		1370	#define SCALER6_POS0_START_Y_MASK VC4_MASK(28, 16)
		1371	#define SCALER6_POS0_HFLIP BIT(15)
		1372	#define SCALER6_POS0_START_X_MASK VC4_MASK(12, 0)
		1373
		1374	#define SCALER6_CTL2_ALPHA_MODE_MASK VC4_MASK(31, 30)
		1375	#define SCALER6_CTL2_ALPHA_PREMULT BIT(29)
		1376	#define SCALER6_CTL2_ALPHA_MIX BIT(28)
		1377	#define SCALER6_CTL2_BFG BIT(26)
		1378	#define SCALER6C_CTL2_CSC_ENABLE BIT(25)
		1379	#define SCALER6C_CTL2_BRCM_CFC_CONTROL_MASK VC4_MASK(18, 16)
		1380	#define SCALER6_CTL2_ALPHA_MASK VC4_MASK(15, 4)
		1381
		1382	#define SCALER6_POS1_SCL_LINES_MASK VC4_MASK(28, 16)
		1383	#define SCALER6_POS1_SCL_WIDTH_MASK VC4_MASK(12, 0)
		1384
		1385	#define SCALER6_POS2_SRC_LINES_MASK VC4_MASK(28, 16)
		1386	#define SCALER6_POS2_SRC_WIDTH_MASK VC4_MASK(12, 0)
		1387
		1388	#define SCALER6_PTR0_VFLIP BIT(31)
		1389	#define SCALER6_PTR0_UPM_BASE_MASK VC4_MASK(28, 16)
		1390	#define SCALER6_PTR0_UPM_HANDLE_MASK VC4_MASK(14, 10)
		1391	#define SCALER6_PTR0_UPM_BUFF_SIZE_MASK VC4_MASK(9, 8)
		1392	#define SCALER6_PTR0_UPM_BUFF_SIZE_16_LINES 3
		1393	#define SCALER6_PTR0_UPM_BUFF_SIZE_8_LINES 2
		1394	#define SCALER6_PTR0_UPM_BUFF_SIZE_4_LINES 1
		1395	#define SCALER6_PTR0_UPM_BUFF_SIZE_2_LINES 0
		1396	#define SCALER6_PTR0_UPPER_ADDR_MASK VC4_MASK(7, 0)
		1397
		1398	#define SCALER6_PTR2_ALPHA_BPP_MASK VC4_MASK(31, 31)
		1399	#define SCALER6_PTR2_ALPHA_BPP_1BPP 1
		1400	#define SCALER6_PTR2_ALPHA_BPP_8BPP 0
		1401	#define SCALER6_PTR2_ALPHA_ORDER_MASK VC4_MASK(30, 30)
		1402	#define SCALER6_PTR2_ALPHA_ORDER_MSB_TO_LSB 1
		1403	#define SCALER6_PTR2_ALPHA_ORDER_LSB_TO_MSB 0
		1404	#define SCALER6_PTR2_ALPHA_OFFS_MASK VC4_MASK(29, 27)
		1405	#define SCALER6_PTR2_LSKIP_MASK VC4_MASK(26, 24)
		1406	#define SCALER6_PTR2_PITCH_MASK VC4_MASK(16, 0)
		1407	#define SCALER6_PTR2_FETCH_COUNT_MASK VC4_MASK(26, 16)
		1408
1112	#endif /* VC4_REGS_H */	1409	#endif /* VC4_REGS_H */

drivers/gpu/drm/vc4/vc4_txp.c+78−13

 /* Number of lines received and committed to memory. */
 #define TXP_PROGRESS		0x10
+#define TXP_DST_PTR_HIGH_MOPLET	0x1c
+#define TXP_DST_PTR_HIGH_MOP	0x24
 #define TXP_READ(offset)								\
 	({										\
 		kunit_fail_current_test("Accessing a register in a unit test!\n");	\
 struct vc4_txp {
 	struct vc4_crtc	base;
+	const struct vc4_txp_data *data;
 	struct platform_device *pdev;
 	struct drm_connector_state *conn_state = drm_atomic_get_new_connector_state(state,
 										    conn);
 	struct vc4_txp *txp = connector_to_vc4_txp(conn);
+	const struct vc4_txp_data *txp_data = txp->data;
 	struct drm_gem_dma_object *gem;
 	struct drm_display_mode *mode;
 	struct drm_framebuffer *fb;
+	unsigned int hdisplay;
+	unsigned int vdisplay;
+	dma_addr_t addr;
 	u32 ctrl;
 	int idx;
 	int i;
 		return;
 	ctrl = TXP_GO | TXP_EI |
-	       VC4_SET_FIELD(0xf, TXP_BYTE_ENABLE) |
 	       VC4_SET_FIELD(txp_fmts[i], TXP_FORMAT);
+	if (txp_data->has_byte_enable)
+		ctrl |= VC4_SET_FIELD(0xf, TXP_BYTE_ENABLE);
 	if (fb->format->has_alpha)
 		ctrl |= TXP_ALPHA_ENABLE;
 	else
 		return;
 	gem = drm_fb_dma_get_gem_obj(fb, 0);
-	TXP_WRITE(TXP_DST_PTR, gem->dma_addr + fb->offsets[0]);
+	addr = gem->dma_addr + fb->offsets[0];
+	TXP_WRITE(TXP_DST_PTR, lower_32_bits(addr));
+	if (txp_data->supports_40bit_addresses)
+		TXP_WRITE(txp_data->high_addr_ptr_reg, upper_32_bits(addr) & 0xff);
 	TXP_WRITE(TXP_DST_PITCH, fb->pitches[0]);
+	hdisplay = mode->hdisplay ?: 1;
+	vdisplay = mode->vdisplay ?: 1;
+	if (txp_data->size_minus_one) {
+		hdisplay -= 1;
+		vdisplay -= 1;
+	}
 	TXP_WRITE(TXP_DIM,
-		  VC4_SET_FIELD(mode->hdisplay, TXP_WIDTH) |
+		  VC4_SET_FIELD(hdisplay, TXP_WIDTH) |
-		  VC4_SET_FIELD(mode->vdisplay, TXP_HEIGHT));
+		  VC4_SET_FIELD(vdisplay, TXP_HEIGHT));
 	TXP_WRITE(TXP_DST_CTRL, ctrl);
 static void vc4_txp_encoder_disable(struct drm_encoder *encoder)
 {
 	struct drm_device *drm = encoder->dev;
+	struct vc4_dev *vc4 = to_vc4_dev(drm);
 	struct vc4_txp *txp = encoder_to_vc4_txp(encoder);
 	int idx;
 		WARN_ON(TXP_READ(TXP_DST_CTRL) & TXP_BUSY);
 	}
-	TXP_WRITE(TXP_DST_CTRL, TXP_POWERDOWN);
+	if (vc4->gen < VC4_GEN_6_C)
+		TXP_WRITE(TXP_DST_CTRL, TXP_POWERDOWN);
 	drm_dev_exit(idx);
 }
 	return IRQ_HANDLED;
 }
-const struct vc4_crtc_data vc4_txp_crtc_data = {
+static const struct vc4_txp_data bcm2712_mop_data = {
-	.name = "txp",
+	.base = {
-	.debugfs_name = "txp_regs",
+		.name = "mop",
-	.hvs_available_channels = BIT(2),
+		.debugfs_name = "mop_regs",
-	.hvs_output = 2,
+		.hvs_available_channels = BIT(2),
+		.hvs_output = 2,
+	},
+	.encoder_type = VC4_ENCODER_TYPE_TXP0,
+	.high_addr_ptr_reg = TXP_DST_PTR_HIGH_MOP,
+	.has_byte_enable = true,
+	.size_minus_one = true,
+	.supports_40bit_addresses = true,
+};
+static const struct vc4_txp_data bcm2712_moplet_data = {
+	.base = {
+		.name = "moplet",
+		.debugfs_name = "moplet_regs",
+		.hvs_available_channels = BIT(1),
+		.hvs_output = 4,
+	},
+	.encoder_type = VC4_ENCODER_TYPE_TXP1,
+	.high_addr_ptr_reg = TXP_DST_PTR_HIGH_MOPLET,
+	.size_minus_one = true,
+	.supports_40bit_addresses = true,
+};
+const struct vc4_txp_data bcm2835_txp_data = {
+	.base = {
+		.name = "txp",
+		.debugfs_name = "txp_regs",
+		.hvs_available_channels = BIT(2),
+		.hvs_output = 2,
+	},
+	.encoder_type = VC4_ENCODER_TYPE_TXP0,
+	.has_byte_enable = true,
 };
 static int vc4_txp_bind(struct device *dev, struct device *master, void *data)
 {
 	struct platform_device *pdev = to_platform_device(dev);
 	struct drm_device *drm = dev_get_drvdata(master);
+	const struct vc4_txp_data *txp_data;
 	struct vc4_encoder *vc4_encoder;
 	struct drm_encoder *encoder;
 	struct vc4_crtc *vc4_crtc;
 	if (!txp)
 		return -ENOMEM;
+	txp_data = of_device_get_match_data(dev);
+	if (!txp_data)
+		return -ENODEV;
+	txp->data = txp_data;
 	txp->pdev = pdev;
 	txp->regs = vc4_ioremap_regs(pdev, 0);
 	if (IS_ERR(txp->regs))
 	vc4_crtc->regset.regs = txp_regs;
 	vc4_crtc->regset.nregs = ARRAY_SIZE(txp_regs);
-	ret = vc4_crtc_init(drm, pdev, vc4_crtc, &vc4_txp_crtc_data,
+	ret = vc4_crtc_init(drm, pdev, vc4_crtc, &txp_data->base,
 			    &vc4_txp_crtc_funcs, &vc4_txp_crtc_helper_funcs, true);
 	if (ret)
 		return ret;
 	vc4_encoder = &txp->encoder;
-	txp->encoder.type = VC4_ENCODER_TYPE_TXP;
+	txp->encoder.type = txp_data->encoder_type;
 	encoder = &vc4_encoder->base;
 	encoder->possible_crtcs = drm_crtc_mask(&vc4_crtc->base);
 }
 static const struct of_device_id vc4_txp_dt_match[] = {
-	{ .compatible = "brcm,bcm2835-txp" },
+	{ .compatible = "brcm,bcm2712-mop", .data = &bcm2712_mop_data },
+	{ .compatible = "brcm,bcm2712-moplet", .data = &bcm2712_moplet_data },
+	{ .compatible = "brcm,bcm2835-txp", .data = &bcm2835_txp_data },
 	{ /* sentinel */ },
 };

drivers/gpu/drm/vgem/vgem_drv.c+0−2

 #define DRIVER_NAME	"vgem"
 #define DRIVER_DESC	"Virtual GEM provider"
-#define DRIVER_DATE	"20120112"
 #define DRIVER_MAJOR	1
 #define DRIVER_MINOR	0
 	.name	= DRIVER_NAME,
 	.desc	= DRIVER_DESC,
-	.date	= DRIVER_DATE,
 	.major	= DRIVER_MAJOR,
 	.minor	= DRIVER_MINOR,
 };

drivers/gpu/drm/virtio/virtgpu_drv.c+1−3

 #include <linux/poll.h>
 #include <linux/wait.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_shmem.h>
 #include <drm/drm_file.h>
 	.postclose = virtio_gpu_driver_postclose,
 	.dumb_create = virtio_gpu_mode_dumb_create,
-	.dumb_map_offset = virtio_gpu_mode_dumb_mmap,
 	DRM_FBDEV_SHMEM_DRIVER_OPS,
 	.name = DRIVER_NAME,
 	.desc = DRIVER_DESC,
-	.date = DRIVER_DATE,
 	.major = DRIVER_MAJOR,
 	.minor = DRIVER_MINOR,
 	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/virtio/virtgpu_drv.h+17−4

 #define DRIVER_NAME "virtio_gpu"
 #define DRIVER_DESC "virtio GPU"
-#define DRIVER_DATE "0"
 #define DRIVER_MAJOR 0
 #define DRIVER_MINOR 1
 struct virtio_gpu_object {
 	struct drm_gem_shmem_object base;
+	struct sg_table *sgt;
 	uint32_t hw_res_handle;
 	bool dumb;
 	bool created;
+	bool attached;
 	bool host3d_blob, guest_blob;
 	uint32_t blob_mem, blob_flags;
 #define to_virtio_gpu_framebuffer(x) \
 	container_of(x, struct virtio_gpu_framebuffer, base)
+struct virtio_gpu_plane_state {
+	struct drm_plane_state base;
+	struct virtio_gpu_fence *fence;
+};
+#define to_virtio_gpu_plane_state(x) \
+	container_of(x, struct virtio_gpu_plane_state, base)
 struct virtio_gpu_queue {
 	struct virtqueue *vq;
 	spinlock_t qlock;
 int virtio_gpu_mode_dumb_create(struct drm_file *file_priv,
 				struct drm_device *dev,
 				struct drm_mode_create_dumb *args);
-int virtio_gpu_mode_dumb_mmap(struct drm_file *file_priv,
-			      struct drm_device *dev,
-			      uint32_t handle, uint64_t *offset_p);
 struct virtio_gpu_object_array *virtio_gpu_array_alloc(u32 nents);
 struct virtio_gpu_object_array*
 			      struct virtio_gpu_object *obj,
 			      struct virtio_gpu_mem_entry *ents,
 			      unsigned int nents);
+void virtio_gpu_object_detach(struct virtio_gpu_device *vgdev,
+			      struct virtio_gpu_object *obj,
+			      struct virtio_gpu_fence *fence);
+int virtio_gpu_detach_object_fenced(struct virtio_gpu_object *bo);
 void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
 			    struct virtio_gpu_output *output);
 int virtio_gpu_cmd_get_display_info(struct virtio_gpu_device *vgdev);
 struct drm_gem_object *virtgpu_gem_prime_import_sg_table(
 	struct drm_device *dev, struct dma_buf_attachment *attach,
 	struct sg_table *sgt);
+int virtgpu_dma_buf_import_sgt(struct virtio_gpu_mem_entry **ents,
+			       unsigned int *nents,
+			       struct virtio_gpu_object *bo,
+			       struct dma_buf_attachment *attach);
 /* virtgpu_debugfs.c */
 void virtio_gpu_debugfs_init(struct drm_minor *minor);

drivers/gpu/drm/virtio/virtgpu_gem.c+5−18

 	return ret;
 }
-int virtio_gpu_mode_dumb_mmap(struct drm_file *file_priv,
-			      struct drm_device *dev,
-			      uint32_t handle, uint64_t *offset_p)
-{
-	struct drm_gem_object *gobj;
-	BUG_ON(!offset_p);
-	gobj = drm_gem_object_lookup(file_priv, handle);
-	if (gobj == NULL)
-		return -ENOENT;
-	*offset_p = drm_vma_node_offset_addr(&gobj->vma_node);
-	drm_gem_object_put(gobj);
-	return 0;
-}
 int virtio_gpu_gem_object_open(struct drm_gem_object *obj,
 			       struct drm_file *file)
 {
 	/* the context might still be missing when the first ioctl is
 	 * DRM_IOCTL_MODE_CREATE_DUMB or DRM_IOCTL_PRIME_FD_TO_HANDLE
 	 */
-	virtio_gpu_create_context(obj->dev, file);
+	if (!vgdev->has_context_init)
+		virtio_gpu_create_context(obj->dev, file);
 	objs = virtio_gpu_array_alloc(1);
 	if (!objs)
 		return -ENOMEM;
 	virtio_gpu_array_add_obj(objs, obj);
-	virtio_gpu_cmd_context_attach_resource(vgdev, vfpriv->ctx_id,
+	if (vfpriv->ctx_id)
-					       objs);
+		virtio_gpu_cmd_context_attach_resource(vgdev, vfpriv->ctx_id, objs);
 out_notify:
 	virtio_gpu_notify(vgdev);
 	return 0;

drivers/gpu/drm/virtio/virtgpu_ioctl.c+3−3

@@ -80,9 +80,9 @@ static int virtio_gpu_map_ioctl(struct drm_device dev, void data, · 79 unmodified lines
80	struct virtio_gpu_device *vgdev = dev->dev_private;	80	struct virtio_gpu_device *vgdev = dev->dev_private;
81	struct drm_virtgpu_map *virtio_gpu_map = data;	81	struct drm_virtgpu_map *virtio_gpu_map = data;
82		82
83	return virtio_gpu_mode_dumb_mmap(file, vgdev->ddev,	83	return drm_gem_dumb_map_offset(file, vgdev->ddev,
84	virtio_gpu_map->handle,	84	virtio_gpu_map->handle,
85	&virtio_gpu_map->offset);	85	&virtio_gpu_map->offset);
86	}	86	}
87		87
88	static int virtio_gpu_getparam_ioctl(struct drm_device dev, void data,	88	static int virtio_gpu_getparam_ioctl(struct drm_device dev, void data,

drivers/gpu/drm/virtio/virtgpu_object.c+24−0

@@ -80,6 +80,9 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo) · 79 unmodified lines
80	drm_gem_free_mmap_offset(&vram->base.base.base);	80	drm_gem_free_mmap_offset(&vram->base.base.base);
81	drm_gem_object_release(&vram->base.base.base);	81	drm_gem_object_release(&vram->base.base.base);
82	kfree(vram);	82	kfree(vram);
		83	} else {
		84	drm_gem_object_release(&bo->base.base);
		85	kfree(bo);
83	}	86	}
84	}	87	}
85		88
@@ -97,6 +100,27 @@ static void virtio_gpu_free_object(struct drm_gem_object *obj) · 11 unmodified lines
97	virtio_gpu_cleanup_object(bo);	100	virtio_gpu_cleanup_object(bo);
98	}	101	}
99		102
		103	int virtio_gpu_detach_object_fenced(struct virtio_gpu_object *bo)
		104	{
		105	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
		106	struct virtio_gpu_fence *fence;
		107
		108	if (!bo->attached)
		109	return 0;
		110
		111	fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
		112	if (!fence)
		113	return -ENOMEM;
		114
		115	virtio_gpu_object_detach(vgdev, bo, fence);
		116	virtio_gpu_notify(vgdev);
		117
		118	dma_fence_wait(&fence->f, false);
		119	dma_fence_put(&fence->f);
		120
		121	return 0;
		122	}
		123
100	static const struct drm_gem_object_funcs virtio_gpu_shmem_funcs = {	124	static const struct drm_gem_object_funcs virtio_gpu_shmem_funcs = {
101	.free = virtio_gpu_free_object,	125	.free = virtio_gpu_free_object,
102	.open = virtio_gpu_gem_object_open,	126	.open = virtio_gpu_gem_object_open,

drivers/gpu/drm/virtio/virtgpu_plane.c+106−19

@@ -26,6 +26,8 @@ · 25 unmodified lines
26	#include <drm/drm_atomic_helper.h>	26	#include <drm/drm_atomic_helper.h>
27	#include <drm/drm_damage_helper.h>	27	#include <drm/drm_damage_helper.h>
28	#include <drm/drm_fourcc.h>	28	#include <drm/drm_fourcc.h>
		29	#include <drm/drm_gem_atomic_helper.h>
		30	#include <linux/virtio_dma_buf.h>
29		31
30	#include "virtgpu_drv.h"	32	#include "virtgpu_drv.h"
31		33
@@ -66,11 +68,28 @@ uint32_t virtio_gpu_translate_format(uint32_t drm_fourcc) · 34 unmodified lines
66	return format;	68	return format;
67	}	69	}
68		70
		71	static struct
		72	drm_plane_state virtio_gpu_plane_duplicate_state(struct drm_plane plane)
		73	{
		74	struct virtio_gpu_plane_state *new;
		75
		76	if (WARN_ON(!plane->state))
		77	return NULL;
		78
		79	new = kzalloc(sizeof(*new), GFP_KERNEL);
		80	if (!new)
		81	return NULL;
		82
		83	__drm_atomic_helper_plane_duplicate_state(plane, &new->base);
		84
		85	return &new->base;
		86	}
		87
69	static const struct drm_plane_funcs virtio_gpu_plane_funcs = {	88	static const struct drm_plane_funcs virtio_gpu_plane_funcs = {
70	.update_plane = drm_atomic_helper_update_plane,	89	.update_plane = drm_atomic_helper_update_plane,
71	.disable_plane = drm_atomic_helper_disable_plane,	90	.disable_plane = drm_atomic_helper_disable_plane,
72	.reset = drm_atomic_helper_plane_reset,	91	.reset = drm_atomic_helper_plane_reset,
73	.atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,	92	.atomic_duplicate_state = virtio_gpu_plane_duplicate_state,
74	.atomic_destroy_state = drm_atomic_helper_plane_destroy_state,	93	.atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
75	};	94	};
76		95
@@ -138,11 +157,13 @@ static void virtio_gpu_resource_flush(struct drm_plane *plane, · 61 unmodified lines
138	struct drm_device *dev = plane->dev;	157	struct drm_device *dev = plane->dev;
139	struct virtio_gpu_device *vgdev = dev->dev_private;	158	struct virtio_gpu_device *vgdev = dev->dev_private;
140	struct virtio_gpu_framebuffer *vgfb;	159	struct virtio_gpu_framebuffer *vgfb;
		160	struct virtio_gpu_plane_state *vgplane_st;
141	struct virtio_gpu_object *bo;	161	struct virtio_gpu_object *bo;
142		162
143	vgfb = to_virtio_gpu_framebuffer(plane->state->fb);	163	vgfb = to_virtio_gpu_framebuffer(plane->state->fb);
		164	vgplane_st = to_virtio_gpu_plane_state(plane->state);
144	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);	165	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
145	if (vgfb->fence) {	166	if (vgplane_st->fence) {
146	struct virtio_gpu_object_array *objs;	167	struct virtio_gpu_object_array *objs;
147		168
148	objs = virtio_gpu_array_alloc(1);	169	objs = virtio_gpu_array_alloc(1);
@@ -151,13 +172,11 @@ static void virtio_gpu_resource_flush(struct drm_plane *plane, · 2 unmodified lines
151	virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]);	172	virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]);
152	virtio_gpu_array_lock_resv(objs);	173	virtio_gpu_array_lock_resv(objs);
153	virtio_gpu_cmd_resource_flush(vgdev, bo->hw_res_handle, x, y,	174	virtio_gpu_cmd_resource_flush(vgdev, bo->hw_res_handle, x, y,
154	width, height, objs, vgfb->fence);	175	width, height, objs,
		176	vgplane_st->fence);
155	virtio_gpu_notify(vgdev);	177	virtio_gpu_notify(vgdev);
156		178	dma_fence_wait_timeout(&vgplane_st->fence->f, true,
157	dma_fence_wait_timeout(&vgfb->fence->f, true,
158	msecs_to_jiffies(50));	179	msecs_to_jiffies(50));
159	dma_fence_put(&vgfb->fence->f);
160	vgfb->fence = NULL;
161	} else {	180	} else {
162	virtio_gpu_cmd_resource_flush(vgdev, bo->hw_res_handle, x, y,	181	virtio_gpu_cmd_resource_flush(vgdev, bo->hw_res_handle, x, y,
163	width, height, NULL, NULL);	182	width, height, NULL, NULL);
@@ -241,45 +260,113 @@ static void virtio_gpu_primary_plane_update(struct drm_plane *plane, · 77 unmodified lines
241	rect.y2 - rect.y1);	260	rect.y2 - rect.y1);
242	}	261	}
243		262
		263	static int virtio_gpu_prepare_imported_obj(struct drm_plane *plane,
		264	struct drm_plane_state *new_state,
		265	struct drm_gem_object *obj)
		266	{
		267	struct virtio_gpu_device *vgdev = plane->dev->dev_private;
		268	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(obj);
		269	struct dma_buf_attachment *attach = obj->import_attach;
		270	struct dma_resv *resv = attach->dmabuf->resv;
		271	struct virtio_gpu_mem_entry *ents = NULL;
		272	unsigned int nents;
		273	int ret;
		274
		275	dma_resv_lock(resv, NULL);
		276
		277	ret = dma_buf_pin(attach);
		278	if (ret) {
		279	dma_resv_unlock(resv);
		280	return ret;
		281	}
		282
		283	if (!bo->sgt) {
		284	ret = virtgpu_dma_buf_import_sgt(&ents, &nents,
		285	bo, attach);
		286	if (ret)
		287	goto err;
		288
		289	virtio_gpu_object_attach(vgdev, bo, ents, nents);
		290	}
		291
		292	dma_resv_unlock(resv);
		293	return 0;
		294
		295	err:
		296	dma_buf_unpin(attach);
		297	dma_resv_unlock(resv);
		298	return ret;
		299	}
		300
244	static int virtio_gpu_plane_prepare_fb(struct drm_plane *plane,	301	static int virtio_gpu_plane_prepare_fb(struct drm_plane *plane,
245	struct drm_plane_state *new_state)	302	struct drm_plane_state *new_state)
246	{	303	{
247	struct drm_device *dev = plane->dev;	304	struct drm_device *dev = plane->dev;
248	struct virtio_gpu_device *vgdev = dev->dev_private;	305	struct virtio_gpu_device *vgdev = dev->dev_private;
249	struct virtio_gpu_framebuffer *vgfb;	306	struct virtio_gpu_framebuffer *vgfb;
		307	struct virtio_gpu_plane_state *vgplane_st;
250	struct virtio_gpu_object *bo;	308	struct virtio_gpu_object *bo;
		309	struct drm_gem_object *obj;
		310	int ret;
251		311
252	if (!new_state->fb)	312	if (!new_state->fb)
253	return 0;	313	return 0;
254		314
255	vgfb = to_virtio_gpu_framebuffer(new_state->fb);	315	vgfb = to_virtio_gpu_framebuffer(new_state->fb);
		316	vgplane_st = to_virtio_gpu_plane_state(new_state);
256	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);	317	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
		318
		319	drm_gem_plane_helper_prepare_fb(plane, new_state);
		320
257	if (!bo \|\| (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob))	321	if (!bo \|\| (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob))
258	return 0;	322	return 0;
259		323
260	if (bo->dumb && (plane->state->fb != new_state->fb)) {	324	obj = new_state->fb->obj[0];
261	vgfb->fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context,	325	if (obj->import_attach) {
		326	ret = virtio_gpu_prepare_imported_obj(plane, new_state, obj);
		327	if (ret)
		328	return ret;
		329	}
		330
		331	if (bo->dumb \|\| obj->import_attach) {
		332	vgplane_st->fence = virtio_gpu_fence_alloc(vgdev,
		333	vgdev->fence_drv.context,
262	0);	334	0);
263	if (!vgfb->fence)	335	if (!vgplane_st->fence)
264	return -ENOMEM;	336	return -ENOMEM;
265	}	337	}
266		338
267	return 0;	339	return 0;
268	}	340	}
269		341
		342	static void virtio_gpu_cleanup_imported_obj(struct drm_gem_object *obj)
		343	{
		344	struct dma_buf_attachment *attach = obj->import_attach;
		345	struct dma_resv *resv = attach->dmabuf->resv;
		346
		347	dma_resv_lock(resv, NULL);
		348	dma_buf_unpin(attach);
		349	dma_resv_unlock(resv);
		350	}
		351
270	static void virtio_gpu_plane_cleanup_fb(struct drm_plane *plane,	352	static void virtio_gpu_plane_cleanup_fb(struct drm_plane *plane,
271	struct drm_plane_state *state)	353	struct drm_plane_state *state)
272	{	354	{
273	struct virtio_gpu_framebuffer *vgfb;	355	struct virtio_gpu_plane_state *vgplane_st;
		356	struct drm_gem_object *obj;
274		357
275	if (!state->fb)	358	if (!state->fb)
276	return;	359	return;
277		360
278	vgfb = to_virtio_gpu_framebuffer(state->fb);	361	vgplane_st = to_virtio_gpu_plane_state(state);
279	if (vgfb->fence) {	362	if (vgplane_st->fence) {
280	dma_fence_put(&vgfb->fence->f);	363	dma_fence_put(&vgplane_st->fence->f);
281	vgfb->fence = NULL;	364	vgplane_st->fence = NULL;
282	}	365	}
		366
		367	obj = state->fb->obj[0];
		368	if (obj->import_attach)
		369	virtio_gpu_cleanup_imported_obj(obj);
283	}	370	}
284		371
285	static void virtio_gpu_cursor_plane_update(struct drm_plane *plane,	372	static void virtio_gpu_cursor_plane_update(struct drm_plane *plane,
@@ -291,6 +378,7 @@ static void virtio_gpu_cursor_plane_update(struct drm_plane *plane, · 5 unmodified lines
291	struct virtio_gpu_device *vgdev = dev->dev_private;	378	struct virtio_gpu_device *vgdev = dev->dev_private;
292	struct virtio_gpu_output *output = NULL;	379	struct virtio_gpu_output *output = NULL;
293	struct virtio_gpu_framebuffer *vgfb;	380	struct virtio_gpu_framebuffer *vgfb;
		381	struct virtio_gpu_plane_state *vgplane_st;
294	struct virtio_gpu_object *bo = NULL;	382	struct virtio_gpu_object *bo = NULL;
295	uint32_t handle;	383	uint32_t handle;
296		384
@@ -303,6 +391,7 @@ static void virtio_gpu_cursor_plane_update(struct drm_plane *plane, · 6 unmodified lines
303		391
304	if (plane->state->fb) {	392	if (plane->state->fb) {
305	vgfb = to_virtio_gpu_framebuffer(plane->state->fb);	393	vgfb = to_virtio_gpu_framebuffer(plane->state->fb);
		394	vgplane_st = to_virtio_gpu_plane_state(plane->state);
306	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);	395	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
307	handle = bo->hw_res_handle;	396	handle = bo->hw_res_handle;
308	} else {	397	} else {
@@ -322,11 +411,9 @@ static void virtio_gpu_cursor_plane_update(struct drm_plane *plane, · 13 unmodified lines
322	(vgdev, 0,	411	(vgdev, 0,
323	plane->state->crtc_w,	412	plane->state->crtc_w,
324	plane->state->crtc_h,	413	plane->state->crtc_h,
325	0, 0, objs, vgfb->fence);	414	0, 0, objs, vgplane_st->fence);
326	virtio_gpu_notify(vgdev);	415	virtio_gpu_notify(vgdev);
327	dma_fence_wait(&vgfb->fence->f, true);	416	dma_fence_wait(&vgplane_st->fence->f, true);
328	dma_fence_put(&vgfb->fence->f);
329	vgfb->fence = NULL;
330	}	417	}
331		418
332	if (plane->state->fb != old_state->fb) {	419	if (plane->state->fb != old_state->fb) {

drivers/gpu/drm/virtio/virtgpu_prime.c+177−1

@@ -27,6 +27,8 @@ · 26 unmodified lines
27		27
28	#include "virtgpu_drv.h"	28	#include "virtgpu_drv.h"
29		29
		30	MODULE_IMPORT_NS("DMA_BUF");
		31
30	static int virtgpu_virtio_get_uuid(struct dma_buf *buf,	32	static int virtgpu_virtio_get_uuid(struct dma_buf *buf,
31	uuid_t *uuid)	33	uuid_t *uuid)
32	{	34	{
@@ -142,10 +144,159 @@ struct dma_buf virtgpu_gem_prime_export(struct drm_gem_object obj, · 109 unmodified lines
142	return buf;	144	return buf;
143	}	145	}
144		146
		147	int virtgpu_dma_buf_import_sgt(struct virtio_gpu_mem_entry **ents,
		148	unsigned int *nents,
		149	struct virtio_gpu_object *bo,
		150	struct dma_buf_attachment *attach)
		151	{
		152	struct scatterlist *sl;
		153	struct sg_table *sgt;
		154	long i, ret;
		155
		156	dma_resv_assert_held(attach->dmabuf->resv);
		157
		158	ret = dma_resv_wait_timeout(attach->dmabuf->resv,
		159	DMA_RESV_USAGE_KERNEL,
		160	false, MAX_SCHEDULE_TIMEOUT);
		161	if (ret <= 0)
		162	return ret < 0 ? ret : -ETIMEDOUT;
		163
		164	sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
		165	if (IS_ERR(sgt))
		166	return PTR_ERR(sgt);
		167
		168	*ents = kvmalloc_array(sgt->nents,
		169	sizeof(struct virtio_gpu_mem_entry),
		170	GFP_KERNEL);
		171	if (!(*ents)) {
		172	dma_buf_unmap_attachment(attach, sgt, DMA_BIDIRECTIONAL);
		173	return -ENOMEM;
		174	}
		175
		176	*nents = sgt->nents;
		177	for_each_sgtable_dma_sg(sgt, sl, i) {
		178	(*ents)[i].addr = cpu_to_le64(sg_dma_address(sl));
		179	(*ents)[i].length = cpu_to_le32(sg_dma_len(sl));
		180	(*ents)[i].padding = 0;
		181	}
		182
		183	bo->sgt = sgt;
		184	return 0;
		185	}
		186
		187	static void virtgpu_dma_buf_free_obj(struct drm_gem_object *obj)
		188	{
		189	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(obj);
		190	struct virtio_gpu_device *vgdev = obj->dev->dev_private;
		191	struct dma_buf_attachment *attach = obj->import_attach;
		192	struct dma_resv *resv = attach->dmabuf->resv;
		193
		194	if (attach) {
		195	dma_resv_lock(resv, NULL);
		196
		197	virtio_gpu_detach_object_fenced(bo);
		198
		199	if (bo->sgt)
		200	dma_buf_unmap_attachment(attach, bo->sgt,
		201	DMA_BIDIRECTIONAL);
		202
		203	dma_resv_unlock(resv);
		204
		205	dma_buf_detach(attach->dmabuf, attach);
		206	dma_buf_put(attach->dmabuf);
		207	}
		208
		209	if (bo->created) {
		210	virtio_gpu_cmd_unref_resource(vgdev, bo);
		211	virtio_gpu_notify(vgdev);
		212	return;
		213	}
		214	virtio_gpu_cleanup_object(bo);
		215	}
		216
		217	static int virtgpu_dma_buf_init_obj(struct drm_device *dev,
		218	struct virtio_gpu_object *bo,
		219	struct dma_buf_attachment *attach)
		220	{
		221	struct virtio_gpu_device *vgdev = dev->dev_private;
		222	struct virtio_gpu_object_params params = { 0 };
		223	struct dma_resv *resv = attach->dmabuf->resv;
		224	struct virtio_gpu_mem_entry *ents = NULL;
		225	unsigned int nents;
		226	int ret;
		227
		228	ret = virtio_gpu_resource_id_get(vgdev, &bo->hw_res_handle);
		229	if (ret) {
		230	virtgpu_dma_buf_free_obj(&bo->base.base);
		231	return ret;
		232	}
		233
		234	dma_resv_lock(resv, NULL);
		235
		236	ret = dma_buf_pin(attach);
		237	if (ret)
		238	goto err_pin;
		239
		240	ret = virtgpu_dma_buf_import_sgt(&ents, &nents, bo, attach);
		241	if (ret)
		242	goto err_import;
		243
		244	params.blob = true;
		245	params.blob_mem = VIRTGPU_BLOB_MEM_GUEST;
		246	params.blob_flags = VIRTGPU_BLOB_FLAG_USE_SHAREABLE;
		247	params.size = attach->dmabuf->size;
		248
		249	virtio_gpu_cmd_resource_create_blob(vgdev, bo, &params,
		250	ents, nents);
		251	bo->guest_blob = true;
		252	bo->attached = true;
		253
		254	dma_buf_unpin(attach);
		255	dma_resv_unlock(resv);
		256
		257	return 0;
		258
		259	err_import:
		260	dma_buf_unpin(attach);
		261	err_pin:
		262	dma_resv_unlock(resv);
		263	virtgpu_dma_buf_free_obj(&bo->base.base);
		264	return ret;
		265	}
		266
		267	static const struct drm_gem_object_funcs virtgpu_gem_dma_buf_funcs = {
		268	.free = virtgpu_dma_buf_free_obj,
		269	};
		270
		271	static void virtgpu_dma_buf_move_notify(struct dma_buf_attachment *attach)
		272	{
		273	struct drm_gem_object *obj = attach->importer_priv;
		274	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(obj);
		275
		276	if (bo->created && kref_read(&obj->refcount)) {
		277	virtio_gpu_detach_object_fenced(bo);
		278
		279	if (bo->sgt)
		280	dma_buf_unmap_attachment(attach, bo->sgt,
		281	DMA_BIDIRECTIONAL);
		282
		283	bo->sgt = NULL;
		284	}
		285	}
		286
		287	static const struct dma_buf_attach_ops virtgpu_dma_buf_attach_ops = {
		288	.allow_peer2peer = true,
		289	.move_notify = virtgpu_dma_buf_move_notify
		290	};
		291
145	struct drm_gem_object virtgpu_gem_prime_import(struct drm_device dev,	292	struct drm_gem_object virtgpu_gem_prime_import(struct drm_device dev,
146	struct dma_buf *buf)	293	struct dma_buf *buf)
147	{	294	{
		295	struct virtio_gpu_device *vgdev = dev->dev_private;
		296	struct dma_buf_attachment *attach;
		297	struct virtio_gpu_object *bo;
148	struct drm_gem_object *obj;	298	struct drm_gem_object *obj;
		299	int ret;
149		300
150	if (buf->ops == &virtgpu_dmabuf_ops.ops) {	301	if (buf->ops == &virtgpu_dmabuf_ops.ops) {
151	obj = buf->priv;	302	obj = buf->priv;
@@ -159,7 +310,32 @@ struct drm_gem_object virtgpu_gem_prime_import(struct drm_device dev, · 7 unmodified lines
159	}	310	}
160	}	311	}
161		312
162	return drm_gem_prime_import(dev, buf);	313	if (!vgdev->has_resource_blob \|\| vgdev->has_virgl_3d)
		314	return drm_gem_prime_import(dev, buf);
		315
		316	bo = kzalloc(sizeof(*bo), GFP_KERNEL);
		317	if (!bo)
		318	return ERR_PTR(-ENOMEM);
		319
		320	obj = &bo->base.base;
		321	obj->funcs = &virtgpu_gem_dma_buf_funcs;
		322	drm_gem_private_object_init(dev, obj, buf->size);
		323
		324	attach = dma_buf_dynamic_attach(buf, dev->dev,
		325	&virtgpu_dma_buf_attach_ops, obj);
		326	if (IS_ERR(attach)) {
		327	kfree(bo);
		328	return ERR_CAST(attach);
		329	}
		330
		331	obj->import_attach = attach;
		332	get_dma_buf(buf);
		333
		334	ret = virtgpu_dma_buf_init_obj(dev, bo, attach);
		335	if (ret < 0)
		336	return ERR_PTR(ret);
		337
		338	return obj;
163	}	339	}
164		340
165	struct drm_gem_object *virtgpu_gem_prime_import_sg_table(	341	struct drm_gem_object *virtgpu_gem_prime_import_sg_table(

drivers/gpu/drm/virtio/virtgpu_vq.c+35−0

@@ -645,6 +645,23 @@ virtio_gpu_cmd_resource_attach_backing(struct virtio_gpu_device *vgdev, · 644 unmodified lines
645	virtio_gpu_queue_fenced_ctrl_buffer(vgdev, vbuf, fence);	645	virtio_gpu_queue_fenced_ctrl_buffer(vgdev, vbuf, fence);
646	}	646	}
647		647
		648	static void
		649	virtio_gpu_cmd_resource_detach_backing(struct virtio_gpu_device *vgdev,
		650	uint32_t resource_id,
		651	struct virtio_gpu_fence *fence)
		652	{
		653	struct virtio_gpu_resource_detach_backing *cmd_p;
		654	struct virtio_gpu_vbuffer *vbuf;
		655
		656	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
		657	memset(cmd_p, 0, sizeof(*cmd_p));
		658
		659	cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_DETACH_BACKING);
		660	cmd_p->resource_id = cpu_to_le32(resource_id);
		661
		662	virtio_gpu_queue_fenced_ctrl_buffer(vgdev, vbuf, fence);
		663	}
		664
648	static void virtio_gpu_cmd_get_display_info_cb(struct virtio_gpu_device *vgdev,	665	static void virtio_gpu_cmd_get_display_info_cb(struct virtio_gpu_device *vgdev,
649	struct virtio_gpu_vbuffer *vbuf)	666	struct virtio_gpu_vbuffer *vbuf)
650	{	667	{
@@ -1103,8 +1120,26 @@ void virtio_gpu_object_attach(struct virtio_gpu_device *vgdev, · 452 unmodified lines
1103	struct virtio_gpu_mem_entry *ents,	1120	struct virtio_gpu_mem_entry *ents,
1104	unsigned int nents)	1121	unsigned int nents)
1105	{	1122	{
		1123	if (obj->attached)
		1124	return;
		1125
1106	virtio_gpu_cmd_resource_attach_backing(vgdev, obj->hw_res_handle,	1126	virtio_gpu_cmd_resource_attach_backing(vgdev, obj->hw_res_handle,
1107	ents, nents, NULL);	1127	ents, nents, NULL);
		1128
		1129	obj->attached = true;
		1130	}
		1131
		1132	void virtio_gpu_object_detach(struct virtio_gpu_device *vgdev,
		1133	struct virtio_gpu_object *obj,
		1134	struct virtio_gpu_fence *fence)
		1135	{
		1136	if (!obj->attached)
		1137	return;
		1138
		1139	virtio_gpu_cmd_resource_detach_backing(vgdev, obj->hw_res_handle,
		1140	fence);
		1141
		1142	obj->attached = false;
1108	}	1143	}
1109		1144
1110	void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,	1145	void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,

drivers/gpu/drm/vkms/vkms_composer.c+247−65

@@ -24,64 +24,33 @@ static u16 pre_mul_blend_channel(u16 src, u16 dst, u16 alpha) · 23 unmodified lines
24		24
25	/**	25	/**
26	* pre_mul_alpha_blend - alpha blending equation	26	* pre_mul_alpha_blend - alpha blending equation
27	* @frame_info: Source framebuffer's metadata
28	* @stage_buffer: The line with the pixels from src_plane	27	* @stage_buffer: The line with the pixels from src_plane
29	* @output_buffer: A line buffer that receives all the blends output	28	* @output_buffer: A line buffer that receives all the blends output
		29	* @x_start: The start offset
		30	* @pixel_count: The number of pixels to blend
30	*	31	*
31	* Using the information from the `frame_info`, this blends only the	32	* The pixels [@x_start;@x_start+@pixel_count) in stage_buffer are blended at
32	* necessary pixels from the `stage_buffer` to the `output_buffer`	33	* [@x_start;@x_start+@pixel_count) in output_buffer.
33	* using premultiplied blend formula.
34	*	34	*
35	* The current DRM assumption is that pixel color values have been already	35	* The current DRM assumption is that pixel color values have been already
36	* pre-multiplied with the alpha channel values. See more	36	* pre-multiplied with the alpha channel values. See more
37	* drm_plane_create_blend_mode_property(). Also, this formula assumes a	37	* drm_plane_create_blend_mode_property(). Also, this formula assumes a
38	* completely opaque background.	38	* completely opaque background.
39	*/	39	*/
40	static void pre_mul_alpha_blend(struct vkms_frame_info *frame_info,	40	static void pre_mul_alpha_blend(const struct line_buffer *stage_buffer,
41	struct line_buffer *stage_buffer,	41	struct line_buffer *output_buffer, int x_start, int pixel_count)
42	struct line_buffer *output_buffer)
43	{	42	{
44	int x_dst = frame_info->dst.x1;	43	struct pixel_argb_u16 *out = &output_buffer->pixels[x_start];
45	struct pixel_argb_u16 *out = output_buffer->pixels + x_dst;	44	const struct pixel_argb_u16 *in = &stage_buffer->pixels[x_start];
46	struct pixel_argb_u16 *in = stage_buffer->pixels;	45
47	int x_limit = min_t(size_t, drm_rect_width(&frame_info->dst),	46	for (int i = 0; i < pixel_count; i++) {
48	stage_buffer->n_pixels);	47	out[i].a = (u16)0xffff;
49		48	out[i].r = pre_mul_blend_channel(in[i].r, out[i].r, in[i].a);
50	for (int x = 0; x < x_limit; x++) {	49	out[i].g = pre_mul_blend_channel(in[i].g, out[i].g, in[i].a);
51	out[x].a = (u16)0xffff;	50	out[i].b = pre_mul_blend_channel(in[i].b, out[i].b, in[i].a);
52	out[x].r = pre_mul_blend_channel(in[x].r, out[x].r, in[x].a);
53	out[x].g = pre_mul_blend_channel(in[x].g, out[x].g, in[x].a);
54	out[x].b = pre_mul_blend_channel(in[x].b, out[x].b, in[x].a);
55	}	51	}
56	}	52	}
57		53
58	static int get_y_pos(struct vkms_frame_info *frame_info, int y)
59	{
60	if (frame_info->rotation & DRM_MODE_REFLECT_Y)
61	return drm_rect_height(&frame_info->rotated) - y - 1;
62
63	switch (frame_info->rotation & DRM_MODE_ROTATE_MASK) {
64	case DRM_MODE_ROTATE_90:
65	return frame_info->rotated.x2 - y - 1;
66	case DRM_MODE_ROTATE_270:
67	return y + frame_info->rotated.x1;
68	default:
69	return y;
70	}
71	}
72
73	static bool check_limit(struct vkms_frame_info *frame_info, int pos)
74	{
75	if (drm_rotation_90_or_270(frame_info->rotation)) {
76	if (pos >= 0 && pos < drm_rect_width(&frame_info->rotated))
77	return true;
78	} else {
79	if (pos >= frame_info->rotated.y1 && pos < frame_info->rotated.y2)
80	return true;
81	}
82
83	return false;
84	}
85		54
86	static void fill_background(const struct pixel_argb_u16 *background_color,	55	static void fill_background(const struct pixel_argb_u16 *background_color,
87	struct line_buffer *output_buffer)	56	struct line_buffer *output_buffer)
@@ -96,7 +65,7 @@ static u16 lerp_u16(u16 a, u16 b, s64 t) · 8 unmodified lines
96	s64 a_fp = drm_int2fixp(a);	65	s64 a_fp = drm_int2fixp(a);
97	s64 b_fp = drm_int2fixp(b);	66	s64 b_fp = drm_int2fixp(b);
98		67
99	s64 delta = drm_fixp_mul(b_fp - a_fp, t);	68	s64 delta = drm_fixp_mul(b_fp - a_fp, t);
100		69
101	return drm_fixp2int(a_fp + delta);	70	return drm_fixp2int(a_fp + delta);
102	}	71	}
@@ -163,6 +132,226 @@ static void apply_lut(const struct vkms_crtc_state *crtc_state, struct line_buff · 60 unmodified lines
163	}	132	}
164	}	133	}
165		134
		135	/**
		136	* direction_for_rotation() - Get the correct reading direction for a given rotation
		137	*
		138	* @rotation: Rotation to analyze. It correspond the field @frame_info.rotation.
		139	*
		140	* This function will use the @rotation setting of a source plane to compute the reading
		141	* direction in this plane which correspond to a "left to right writing" in the CRTC.
		142	* For example, if the buffer is reflected on X axis, the pixel must be read from right to left
		143	* to be written from left to right on the CRTC.
		144	*/
		145	static enum pixel_read_direction direction_for_rotation(unsigned int rotation)
		146	{
		147	struct drm_rect tmp_a, tmp_b;
		148	int x, y;
		149
		150	/*
		151	* Points A and B are depicted as zero-size rectangles on the CRTC.
		152	* The CRTC writing direction is from A to B. The plane reading direction
		153	* is discovered by inverse-transforming A and B.
		154	* The reading direction is computed by rotating the vector AB (top-left to top-right) in a
		155	* 1x1 square.
		156	*/
		157
		158	tmp_a = DRM_RECT_INIT(0, 0, 0, 0);
		159	tmp_b = DRM_RECT_INIT(1, 0, 0, 0);
		160	drm_rect_rotate_inv(&tmp_a, 1, 1, rotation);
		161	drm_rect_rotate_inv(&tmp_b, 1, 1, rotation);
		162
		163	x = tmp_b.x1 - tmp_a.x1;
		164	y = tmp_b.y1 - tmp_a.y1;
		165
		166	if (x == 1 && y == 0)
		167	return READ_LEFT_TO_RIGHT;
		168	else if (x == -1 && y == 0)
		169	return READ_RIGHT_TO_LEFT;
		170	else if (y == 1 && x == 0)
		171	return READ_TOP_TO_BOTTOM;
		172	else if (y == -1 && x == 0)
		173	return READ_BOTTOM_TO_TOP;
		174
		175	WARN_ONCE(true, "The inverse of the rotation gives an incorrect direction.");
		176	return READ_LEFT_TO_RIGHT;
		177	}
		178
		179	/**
		180	* clamp_line_coordinates() - Compute and clamp the coordinate to read and write during the blend
		181	* process.
		182	*
		183	* @direction: direction of the reading
		184	* @current_plane: current plane blended
		185	* @src_line: source line of the reading. Only the top-left coordinate is used. This rectangle
		186	* must be rotated and have a shape of 1*pixel_count if @direction is vertical and a shape of
		187	* pixel_count*1 if @direction is horizontal.
		188	* @src_x_start: x start coordinate for the line reading
		189	* @src_y_start: y start coordinate for the line reading
		190	* @dst_x_start: x coordinate to blend the read line
		191	* @pixel_count: number of pixels to blend
		192	*
		193	* This function is mainly a safety net to avoid reading outside the source buffer. As the
		194	* userspace should never ask to read outside the source plane, all the cases covered here should
		195	* be dead code.
		196	*/
		197	static void clamp_line_coordinates(enum pixel_read_direction direction,
		198	const struct vkms_plane_state *current_plane,
		199	const struct drm_rect src_line, int src_x_start,
		200	int src_y_start, int dst_x_start, int *pixel_count)
		201	{
		202	/* By default the start points are correct */
		203	*src_x_start = src_line->x1;
		204	*src_y_start = src_line->y1;
		205	*dst_x_start = current_plane->frame_info->dst.x1;
		206
		207	/* Get the correct number of pixel to blend, it depends of the direction */
		208	switch (direction) {
		209	case READ_LEFT_TO_RIGHT:
		210	case READ_RIGHT_TO_LEFT:
		211	*pixel_count = drm_rect_width(src_line);
		212	break;
		213	case READ_BOTTOM_TO_TOP:
		214	case READ_TOP_TO_BOTTOM:
		215	*pixel_count = drm_rect_height(src_line);
		216	break;
		217	}
		218
		219	/*
		220	* Clamp the coordinates to avoid reading outside the buffer
		221	*
		222	* This is mainly a security check to avoid reading outside the buffer, the userspace
		223	* should never request to read outside the source buffer.
		224	*/
		225	switch (direction) {
		226	case READ_LEFT_TO_RIGHT:
		227	case READ_RIGHT_TO_LEFT:
		228	if (*src_x_start < 0) {
		229	pixel_count += src_x_start;
		230	dst_x_start -= src_x_start;
		231	*src_x_start = 0;
		232	}
		233	if (src_x_start + pixel_count > current_plane->frame_info->fb->width)
		234	*pixel_count = max(0, (int)current_plane->frame_info->fb->width -
		235	*src_x_start);
		236	break;
		237	case READ_BOTTOM_TO_TOP:
		238	case READ_TOP_TO_BOTTOM:
		239	if (*src_y_start < 0) {
		240	pixel_count += src_y_start;
		241	dst_x_start -= src_y_start;
		242	*src_y_start = 0;
		243	}
		244	if (src_y_start + pixel_count > current_plane->frame_info->fb->height)
		245	*pixel_count = max(0, (int)current_plane->frame_info->fb->height -
		246	*src_y_start);
		247	break;
		248	}
		249	}
		250
		251	/**
		252	* blend_line() - Blend a line from a plane to the output buffer
		253	*
		254	* @current_plane: current plane to work on
		255	* @y: line to write in the output buffer
		256	* @crtc_x_limit: width of the output buffer
		257	* @stage_buffer: temporary buffer to convert the pixel line from the source buffer
		258	* @output_buffer: buffer to blend the read line into.
		259	*/
		260	static void blend_line(struct vkms_plane_state *current_plane, int y,
		261	int crtc_x_limit, struct line_buffer *stage_buffer,
		262	struct line_buffer *output_buffer)
		263	{
		264	int src_x_start, src_y_start, dst_x_start, pixel_count;
		265	struct drm_rect dst_line, tmp_src, src_line;
		266
		267	/* Avoid rendering useless lines */
		268	if (y < current_plane->frame_info->dst.y1 \|\|
		269	y >= current_plane->frame_info->dst.y2)
		270	return;
		271
		272	/*
		273	* dst_line is the line to copy. The initial coordinates are inside the
		274	* destination framebuffer, and then drm_rect_* helpers are used to
		275	* compute the correct position into the source framebuffer.
		276	*/
		277	dst_line = DRM_RECT_INIT(current_plane->frame_info->dst.x1, y,
		278	drm_rect_width(&current_plane->frame_info->dst),
		279	1);
		280
		281	drm_rect_fp_to_int(&tmp_src, &current_plane->frame_info->src);
		282
		283	/*
		284	* [1]: Clamping src_line to the crtc_x_limit to avoid writing outside of
		285	* the destination buffer
		286	*/
		287	dst_line.x1 = max_t(int, dst_line.x1, 0);
		288	dst_line.x2 = min_t(int, dst_line.x2, crtc_x_limit);
		289	/* The destination is completely outside of the crtc. */
		290	if (dst_line.x2 <= dst_line.x1)
		291	return;
		292
		293	src_line = dst_line;
		294
		295	/*
		296	* Transform the coordinate x/y from the crtc to coordinates into
		297	* coordinates for the src buffer.
		298	*
		299	* - Cancel the offset of the dst buffer.
		300	* - Invert the rotation. This assumes that
		301	* dst = drm_rect_rotate(src, rotation) (dst and src have the
		302	* same size, but can be rotated).
		303	* - Apply the offset of the source rectangle to the coordinate.
		304	*/
		305	drm_rect_translate(&src_line, -current_plane->frame_info->dst.x1,
		306	-current_plane->frame_info->dst.y1);
		307	drm_rect_rotate_inv(&src_line, drm_rect_width(&tmp_src),
		308	drm_rect_height(&tmp_src),
		309	current_plane->frame_info->rotation);
		310	drm_rect_translate(&src_line, tmp_src.x1, tmp_src.y1);
		311
		312	/* Get the correct reading direction in the source buffer. */
		313
		314	enum pixel_read_direction direction =
		315	direction_for_rotation(current_plane->frame_info->rotation);
		316
		317	/* [2]: Compute and clamp the number of pixel to read */
		318	clamp_line_coordinates(direction, current_plane, &src_line, &src_x_start, &src_y_start,
		319	&dst_x_start, &pixel_count);
		320
		321	if (pixel_count <= 0) {
		322	/* Nothing to read, so avoid multiple function calls */
		323	return;
		324	}
		325
		326	/*
		327	* Modify the starting point to take in account the rotation
		328	*
		329	* src_line is the top-left corner, so when reading READ_RIGHT_TO_LEFT or
		330	* READ_BOTTOM_TO_TOP, it must be changed to the top-right/bottom-left
		331	* corner.
		332	*/
		333	if (direction == READ_RIGHT_TO_LEFT) {
		334	// src_x_start is now the right point
		335	src_x_start += pixel_count - 1;
		336	} else if (direction == READ_BOTTOM_TO_TOP) {
		337	// src_y_start is now the bottom point
		338	src_y_start += pixel_count - 1;
		339	}
		340
		341	/*
		342	* Perform the conversion and the blending
		343	*
		344	* Here we know that the read line (x_start, y_start, pixel_count) is
		345	* inside the source buffer [2] and we don't write outside the stage
		346	* buffer [1].
		347	*/
		348	current_plane->pixel_read_line(current_plane, src_x_start, src_y_start, direction,
		349	pixel_count, &stage_buffer->pixels[dst_x_start]);
		350
		351	pre_mul_alpha_blend(stage_buffer, output_buffer,
		352	dst_x_start, pixel_count);
		353	}
		354
166	/**	355	/**
167	* blend - blend the pixels from all planes and compute crc	356	* blend - blend the pixels from all planes and compute crc
168	* @wb: The writeback frame buffer metadata	357	* @wb: The writeback frame buffer metadata
@@ -183,32 +372,25 @@ static void blend(struct vkms_writeback_job *wb, · 14 unmodified lines
183	{	372	{
184	struct vkms_plane_state **plane = crtc_state->active_planes;	373	struct vkms_plane_state **plane = crtc_state->active_planes;
185	u32 n_active_planes = crtc_state->num_active_planes;	374	u32 n_active_planes = crtc_state->num_active_planes;
186	int y_pos;
187		375
188	const struct pixel_argb_u16 background_color = { .a = 0xffff };	376	const struct pixel_argb_u16 background_color = { .a = 0xffff };
189		377
190	size_t crtc_y_limit = crtc_state->base.mode.vdisplay;	378	int crtc_y_limit = crtc_state->base.mode.vdisplay;
		379	int crtc_x_limit = crtc_state->base.mode.hdisplay;
191		380
192	/*	381	/*
193	* The planes are composed line-by-line to avoid heavy memory usage. It is a necessary	382	* The planes are composed line-by-line to avoid heavy memory usage. It is a necessary
194	* complexity to avoid poor blending performance.	383	* complexity to avoid poor blending performance.
195	*	384	*
196	* The function vkms_compose_row() is used to read a line, pixel-by-pixel, into the staging	385	* The function pixel_read_line callback is used to read a line, using an efficient
197	* buffer.	386	* algorithm for a specific format, into the staging buffer.
198	*/	387	*/
199	for (size_t y = 0; y < crtc_y_limit; y++) {	388	for (int y = 0; y < crtc_y_limit; y++) {
200	fill_background(&background_color, output_buffer);	389	fill_background(&background_color, output_buffer);
201		390
202	/* The active planes are composed associatively in z-order. */	391	/* The active planes are composed associatively in z-order. */
203	for (size_t i = 0; i < n_active_planes; i++) {	392	for (size_t i = 0; i < n_active_planes; i++) {
204	y_pos = get_y_pos(plane[i]->frame_info, y);	393	blend_line(plane[i], y, crtc_x_limit, stage_buffer, output_buffer);
205
206	if (!check_limit(plane[i]->frame_info, y_pos))
207	continue;
208
209	vkms_compose_row(stage_buffer, plane[i], y_pos);
210	pre_mul_alpha_blend(plane[i]->frame_info, stage_buffer,
211	output_buffer);
212	}	394	}
213		395
214	apply_lut(crtc_state, output_buffer);	396	apply_lut(crtc_state, output_buffer);
@@ -216,7 +398,7 @@ static void blend(struct vkms_writeback_job *wb, · 1 unmodified line
216	crc32 = crc32_le(crc32, (void *)output_buffer->pixels, row_size);	398	crc32 = crc32_le(crc32, (void *)output_buffer->pixels, row_size);
217		399
218	if (wb)	400	if (wb)
219	vkms_writeback_row(wb, output_buffer, y_pos);	401	vkms_writeback_row(wb, output_buffer, y);
220	}	402	}
221	}	403	}
222		404
@@ -227,7 +409,7 @@ static int check_format_funcs(struct vkms_crtc_state *crtc_state, · 4 unmodified lines
227	u32 n_active_planes = crtc_state->num_active_planes;	409	u32 n_active_planes = crtc_state->num_active_planes;
228		410
229	for (size_t i = 0; i < n_active_planes; i++)	411	for (size_t i = 0; i < n_active_planes; i++)
230	if (!planes[i]->pixel_read)	412	if (!planes[i]->pixel_read_line)
231	return -1;	413	return -1;
232		414
233	if (active_wb && !active_wb->pixel_write)	415	if (active_wb && !active_wb->pixel_write)
@@ -309,8 +491,8 @@ free_stage_buffer: · 75 unmodified lines
309	void vkms_composer_worker(struct work_struct *work)	491	void vkms_composer_worker(struct work_struct *work)
310	{	492	{
311	struct vkms_crtc_state *crtc_state = container_of(work,	493	struct vkms_crtc_state *crtc_state = container_of(work,
312	struct vkms_crtc_state,	494	struct vkms_crtc_state,
313	composer_work);	495	composer_work);
314	struct drm_crtc *crtc = crtc_state->base.crtc;	496	struct drm_crtc *crtc = crtc_state->base.crtc;
315	struct vkms_writeback_job *active_wb = crtc_state->active_writeback;	497	struct vkms_writeback_job *active_wb = crtc_state->active_writeback;
316	struct vkms_output *out = drm_crtc_to_vkms_output(crtc);	498	struct vkms_output *out = drm_crtc_to_vkms_output(crtc);
@@ -335,7 +517,7 @@ void vkms_composer_worker(struct work_struct *work) · 18 unmodified lines
335	crtc_state->gamma_lut.base = (struct drm_color_lut *)crtc->state->gamma_lut->data;	517	crtc_state->gamma_lut.base = (struct drm_color_lut *)crtc->state->gamma_lut->data;
336	crtc_state->gamma_lut.lut_length =	518	crtc_state->gamma_lut.lut_length =
337	crtc->state->gamma_lut->length / sizeof(struct drm_color_lut);	519	crtc->state->gamma_lut->length / sizeof(struct drm_color_lut);
338	max_lut_index_fp = drm_int2fixp(crtc_state->gamma_lut.lut_length - 1);	520	max_lut_index_fp = drm_int2fixp(crtc_state->gamma_lut.lut_length - 1);
339	crtc_state->gamma_lut.channel_value2index_ratio = drm_fixp_div(max_lut_index_fp,	521	crtc_state->gamma_lut.channel_value2index_ratio = drm_fixp_div(max_lut_index_fp,
340	u16_max_fp);	522	u16_max_fp);
341		523
@@ -374,7 +556,7 @@ void vkms_composer_worker(struct work_struct *work) · 32 unmodified lines
374	drm_crtc_add_crc_entry(crtc, true, frame_start++, &crc32);	556	drm_crtc_add_crc_entry(crtc, true, frame_start++, &crc32);
375	}	557	}
376		558
377	static const char * const pipe_crc_sources[] = {"auto"};	559	static const char *const pipe_crc_sources[] = { "auto" };
378		560
379	const char const vkms_get_crc_sources(struct drm_crtc *crtc,	561	const char const vkms_get_crc_sources(struct drm_crtc *crtc,
380	size_t *count)	562	size_t *count)

drivers/gpu/drm/vkms/vkms_crtc.c+2−4

 		return ret;
 	drm_for_each_plane_mask(plane, crtc->dev, crtc_state->plane_mask) {
-		plane_state = drm_atomic_get_existing_plane_state(crtc_state->state,
+		plane_state = drm_atomic_get_existing_plane_state(crtc_state->state, plane);
-								  plane);
 		WARN_ON(!plane_state);
 		if (!plane_state->visible)
 	i = 0;
 	drm_for_each_plane_mask(plane, crtc->dev, crtc_state->plane_mask) {
-		plane_state = drm_atomic_get_existing_plane_state(crtc_state->state,
+		plane_state = drm_atomic_get_existing_plane_state(crtc_state->state, plane);
-								  plane);
 		if (!plane_state->visible)
 			continue;

drivers/gpu/drm/vkms/vkms_drv.c+3−6

 #include <linux/platform_device.h>
 #include <linux/dma-mapping.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_gem.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_shmem.h>
 #include <drm/drm_file.h>
 #define DRIVER_NAME	"vkms"
 #define DRIVER_DESC	"Virtual Kernel Mode Setting"
-#define DRIVER_DATE	"20180514"
 #define DRIVER_MAJOR	1
 #define DRIVER_MINOR	0
 	drm_atomic_helper_wait_for_flip_done(dev, old_state);
 	for_each_old_crtc_in_state(old_state, crtc, old_crtc_state, i) {
-		struct vkms_crtc_state *vkms_state =
+		struct vkms_crtc_state *vkms_state = to_vkms_crtc_state(old_crtc_state);
-			to_vkms_crtc_state(old_crtc_state);
 		flush_work(&vkms_state->composer_work);
 	}
 	.name			= DRIVER_NAME,
 	.desc			= DRIVER_DESC,
-	.date			= DRIVER_DATE,
 	.major			= DRIVER_MAJOR,
 	.minor			= DRIVER_MINOR,
 };
 	dev->mode_config.preferred_depth = 0;
 	dev->mode_config.helper_private = &vkms_mode_config_helpers;
-	return vkms_output_init(vkmsdev, 0);
+	return vkms_output_init(vkmsdev);
 }
 static int vkms_create(struct vkms_config *config)

drivers/gpu/drm/vkms/vkms_drv.h+48−15

 struct vkms_frame_info {
 	struct drm_framebuffer *fb;
 	struct drm_rect src, dst;
-	struct drm_rect rotated;
 	struct iosys_map map[DRM_FORMAT_MAX_PLANES];
 	unsigned int rotation;
-	unsigned int offset;
-	unsigned int pitch;
-	unsigned int cpp;
 };
 struct pixel_argb_u16 {
 	struct pixel_argb_u16 *pixels;
 };
+/**
+ * typedef pixel_write_t - These functions are used to read a pixel from a
+ * &struct pixel_argb_u16, convert it in a specific format and write it in the @out_pixel
+ * buffer.
+ *
+ * @out_pixel: destination address to write the pixel
+ * @in_pixel: pixel to write
+ */
+typedef void (*pixel_write_t)(u8 *out_pixel, const struct pixel_argb_u16 *in_pixel);
 struct vkms_writeback_job {
 	struct iosys_map data[DRM_FORMAT_MAX_PLANES];
 	struct vkms_frame_info wb_frame_info;
-	void (*pixel_write)(u8 *dst_pixels, struct pixel_argb_u16 *in_pixel);
+	pixel_write_t pixel_write;
 };
+/**
+ * enum pixel_read_direction - Enum used internally by VKMS to represent a reading direction in a
+ * plane.
+ */
+enum pixel_read_direction {
+	READ_BOTTOM_TO_TOP,
+	READ_TOP_TO_BOTTOM,
+	READ_RIGHT_TO_LEFT,
+	READ_LEFT_TO_RIGHT
+};
+struct vkms_plane_state;
+/**
+ * typedef pixel_read_line_t - These functions are used to read a pixel line in the source frame,
+ * convert it to `struct pixel_argb_u16` and write it to @out_pixel.
+ *
+ * @plane: plane used as source for the pixel value
+ * @x_start: X (width) coordinate of the first pixel to copy. The caller must ensure that x_start
+ * is non-negative and smaller than @plane->frame_info->fb->width.
+ * @y_start: Y (height) coordinate of the first pixel to copy. The caller must ensure that y_start
+ * is non-negative and smaller than @plane->frame_info->fb->height.
+ * @direction: direction to use for the copy, starting at @x_start/@y_start
+ * @count: number of pixels to copy
+ * @out_pixel: pointer where to write the pixel values. They will be written from @out_pixel[0]
+ * (included) to @out_pixel[@count] (excluded). The caller must ensure that out_pixel have a
+ * length of at least @count.
+ */
+typedef void (*pixel_read_line_t)(const struct vkms_plane_state *plane, int x_start,
+				  int y_start, enum pixel_read_direction direction, int count,
+				  struct pixel_argb_u16 out_pixel[]);
 /**
  * struct vkms_plane_state - Driver specific plane state
  * @base: base plane state
  * @frame_info: data required for composing computation
- * @pixel_read: function to read a pixel in this plane. The creator of a struct vkms_plane_state
+ * @pixel_read_line: function to read a pixel line in this plane. The creator of a
- *	        must ensure that this pointer is valid
+ *		     struct vkms_plane_state must ensure that this pointer is valid
  */
 struct vkms_plane_state {
 	struct drm_shadow_plane_state base;
 	struct vkms_frame_info *frame_info;
-	void (*pixel_read)(u8 *src_buffer, struct pixel_argb_u16 *out_pixel);
+	pixel_read_line_t pixel_read_line;
 };
 struct vkms_plane {
  * vkms_output_init() - Initialize all sub-components needed for a VKMS device.
  *
  * @vkmsdev: VKMS device to initialize
- * @index: CRTC which can be attached to the planes. The caller must ensure that
- *	   @index is positive and less or equals to 31.
  */
-int vkms_output_init(struct vkms_device *vkmsdev, int index);
+int vkms_output_init(struct vkms_device *vkmsdev);
 /**
  * vkms_plane_init() - Initialize a plane
  *
  * @vkmsdev: VKMS device containing the plane
  * @type: type of plane to initialize
- * @index: CRTC which can be attached to the plane. The caller must ensure that
- *	   @index is positive and less or equals to 31.
  */
 struct vkms_plane *vkms_plane_init(struct vkms_device *vkmsdev,
-				   enum drm_plane_type type, int index);
+				   enum drm_plane_type type);
 /* CRC Support */
 const char *const *vkms_get_crc_sources(struct drm_crtc *crtc,
 /* Composer Support */
 void vkms_composer_worker(struct work_struct *work);
 void vkms_set_composer(struct vkms_output *out, bool enabled);
-void vkms_compose_row(struct line_buffer *stage_buffer, struct vkms_plane_state *plane, int y);
 void vkms_writeback_row(struct vkms_writeback_job *wb, const struct line_buffer *src_buffer, int y);
 /* Writeback */

drivers/gpu/drm/vkms/vkms_formats.c+287−122

@@ -10,21 +10,46 @@ · 9 unmodified lines
10	#include "vkms_formats.h"	10	#include "vkms_formats.h"
11		11
12	/**	12	/**
13	* pixel_offset() - Get the offset of the pixel at coordinates x/y in the first plane	13	* packed_pixels_offset() - Get the offset of the block containing the pixel at coordinates x/y
14	*	14	*
15	* @frame_info: Buffer metadata	15	* @frame_info: Buffer metadata
16	* @x: The x coordinate of the wanted pixel in the buffer	16	* @x: The x coordinate of the wanted pixel in the buffer
17	* @y: The y coordinate of the wanted pixel in the buffer	17	* @y: The y coordinate of the wanted pixel in the buffer
		18	* @plane_index: The index of the plane to use
		19	* @offset: The returned offset inside the buffer of the block
		20	* @rem_x: The returned X coordinate of the requested pixel in the block
		21	* @rem_y: The returned Y coordinate of the requested pixel in the block
18	*	22	*
19	* The caller must ensure that the framebuffer associated with this request uses a pixel format	23	* As some pixel formats store multiple pixels in a block (DRM_FORMAT_R* for example), some
20	* where block_h == block_w == 1.	24	* pixels are not individually addressable. This function return 3 values: the offset of the
21	* If this requirement is not fulfilled, the resulting offset can point to an other pixel or	25	* whole block, and the coordinate of the requested pixel inside this block.
22	* outside of the buffer.	26	* For example, if the format is DRM_FORMAT_R1 and the requested coordinate is 13,5, the offset
		27	* will point to the byte 5*pitches + 13/8 (second byte of the 5th line), and the rem_x/rem_y
		28	* coordinates will be (13 % 8, 5 % 1) = (5, 0)
		29	*
		30	* With this function, the caller just have to extract the correct pixel from the block.
23	*/	31	*/
24	static size_t pixel_offset(const struct vkms_frame_info *frame_info, int x, int y)	32	static void packed_pixels_offset(const struct vkms_frame_info *frame_info, int x, int y,
		33	int plane_index, int offset, int rem_x, int *rem_y)
25	{	34	{
26	return frame_info->offset + (y * frame_info->pitch)	35	struct drm_framebuffer *fb = frame_info->fb;
27	+ (x * frame_info->cpp);	36	const struct drm_format_info *format = frame_info->fb->format;
		37	/* Directly using x and y to multiply pitches and format->ccp is not sufficient because
		38	* in some formats a block can represent multiple pixels.
		39	*
		40	* Dividing x and y by the block size allows to extract the correct offset of the block
		41	* containing the pixel.
		42	*/
		43
		44	int block_x = x / drm_format_info_block_width(format, plane_index);
		45	int block_y = y / drm_format_info_block_height(format, plane_index);
		46	int block_pitch = fb->pitches[plane_index] * drm_format_info_block_height(format,
		47	plane_index);
		48	*rem_x = x % drm_format_info_block_width(format, plane_index);
		49	*rem_y = y % drm_format_info_block_height(format, plane_index);
		50	*offset = fb->offsets[plane_index] +
		51	block_y * block_pitch +
		52	block_x * format->char_per_block[plane_index];
28	}	53	}
29		54
30	/**	55	/**
@@ -34,145 +59,266 @@ static size_t pixel_offset(const struct vkms_frame_info *frame_info, int x, int · 3 unmodified lines
34	* @frame_info: Buffer metadata	59	* @frame_info: Buffer metadata
35	* @x: The x (width) coordinate inside the plane	60	* @x: The x (width) coordinate inside the plane
36	* @y: The y (height) coordinate inside the plane	61	* @y: The y (height) coordinate inside the plane
		62	* @plane_index: The index of the plane
		63	* @addr: The returned pointer
		64	* @rem_x: The returned X coordinate of the requested pixel in the block
		65	* @rem_y: The returned Y coordinate of the requested pixel in the block
37	*	66	*
38	* Takes the information stored in the frame_info, a pair of coordinates, and	67	* Takes the information stored in the frame_info, a pair of coordinates, and returns the address
39	* returns the address of the first color channel.	68	* of the block containing this pixel and the pixel position inside this block.
40	* This function assumes the channels are packed together, i.e. a color channel
41	* comes immediately after another in the memory. And therefore, this function
42	* doesn't work for YUV with chroma subsampling (e.g. YUV420 and NV21).
43	*	69	*
44	* The caller must ensure that the framebuffer associated with this request uses a pixel format	70	* See @packed_pixels_offset for details about rem_x/rem_y behavior.
45	* where block_h == block_w == 1, otherwise the returned pointer can be outside the buffer.
46	*/	71	*/
47	static void packed_pixels_addr(const struct vkms_frame_info frame_info,	72	static void packed_pixels_addr(const struct vkms_frame_info *frame_info,
48	int x, int y)	73	int x, int y, int plane_index, u8 *addr, int rem_x,
		74	int *rem_y)
49	{	75	{
50	size_t offset = pixel_offset(frame_info, x, y);	76	int offset;
51		77
52	return (u8 *)frame_info->map[0].vaddr + offset;	78	packed_pixels_offset(frame_info, x, y, plane_index, &offset, rem_x, rem_y);
		79	addr = (u8 )frame_info->map[0].vaddr + offset;
53	}	80	}
54		81
55	static void get_packed_src_addr(const struct vkms_frame_info frame_info, int y)	82	/**
		83	* get_block_step_bytes() - Common helper to compute the correct step value between each pixel block
		84	* to read in a certain direction.
		85	*
		86	* @fb: Framebuffer to iter on
		87	* @direction: Direction of the reading
		88	* @plane_index: Plane to get the step from
		89	*
		90	* As the returned count is the number of bytes between two consecutive blocks in a direction,
		91	* the caller may have to read multiple pixels before using the next one (for example, to read from
		92	* left to right in a DRM_FORMAT_R1 plane, each block contains 8 pixels, so the step must be used
		93	* only every 8 pixels).
		94	*/
		95	static int get_block_step_bytes(struct drm_framebuffer *fb, enum pixel_read_direction direction,
		96	int plane_index)
56	{	97	{
57	int x_src = frame_info->src.x1 >> 16;	98	switch (direction) {
58	int y_src = y - frame_info->rotated.y1 + (frame_info->src.y1 >> 16);	99	case READ_LEFT_TO_RIGHT:
		100	return fb->format->char_per_block[plane_index];
		101	case READ_RIGHT_TO_LEFT:
		102	return -fb->format->char_per_block[plane_index];
		103	case READ_TOP_TO_BOTTOM:
		104	return (int)fb->pitches[plane_index] * drm_format_info_block_width(fb->format,
		105	plane_index);
		106	case READ_BOTTOM_TO_TOP:
		107	return -(int)fb->pitches[plane_index] * drm_format_info_block_width(fb->format,
		108	plane_index);
		109	}
59		110
60	return packed_pixels_addr(frame_info, x_src, y_src);	111	return 0;
61	}	112	}
62		113
63	static int get_x_position(const struct vkms_frame_info *frame_info, int limit, int x)	114	/**
		115	* packed_pixels_addr_1x1() - Get the pointer to the block containing the pixel at the given
		116	* coordinates
		117	*
		118	* @frame_info: Buffer metadata
		119	* @x: The x (width) coordinate inside the plane
		120	* @y: The y (height) coordinate inside the plane
		121	* @plane_index: The index of the plane
		122	* @addr: The returned pointer
		123	*
		124	* This function can only be used with format where block_h == block_w == 1.
		125	*/
		126	static void packed_pixels_addr_1x1(const struct vkms_frame_info *frame_info,
		127	int x, int y, int plane_index, u8 **addr)
64	{	128	{
65	if (frame_info->rotation & (DRM_MODE_REFLECT_X \| DRM_MODE_ROTATE_270))	129	int offset, rem_x, rem_y;
66	return limit - x - 1;	130
67	return x;	131	WARN_ONCE(drm_format_info_block_width(frame_info->fb->format,
		132	plane_index) != 1,
		133	"%s() only support formats with block_w == 1", __func__);
		134	WARN_ONCE(drm_format_info_block_height(frame_info->fb->format,
		135	plane_index) != 1,
		136	"%s() only support formats with block_h == 1", __func__);
		137
		138	packed_pixels_offset(frame_info, x, y, plane_index, &offset, &rem_x,
		139	&rem_y);
		140	addr = (u8 )frame_info->map[0].vaddr + offset;
68	}	141	}
69		142
70	/*	143	/*
71	* The following functions take pixel data from the buffer and convert them to the format	144	* The following functions take pixel data (a, r, g, b, pixel, ...) and convert them to
72	* ARGB16161616 in @out_pixel.	145	* &struct pixel_argb_u16
73	*	146	*
74	* They are used in the vkms_compose_row() function to handle multiple formats.	147	* They are used in the `read_line`s functions to avoid duplicate work for some pixel formats.
75	*/	148	*/
76		149
77	static void ARGB8888_to_argb_u16(u8 src_pixels, struct pixel_argb_u16 out_pixel)	150	static struct pixel_argb_u16 argb_u16_from_u8888(u8 a, u8 r, u8 g, u8 b)
78	{	151	{
		152	struct pixel_argb_u16 out_pixel;
79	/*	153	/*
80	* The 257 is the "conversion ratio". This number is obtained by the	154	* The 257 is the "conversion ratio". This number is obtained by the
81	* (2^16 - 1) / (2^8 - 1) division. Which, in this case, tries to get	155	* (2^16 - 1) / (2^8 - 1) division. Which, in this case, tries to get
82	* the best color value in a pixel format with more possibilities.	156	* the best color value in a pixel format with more possibilities.
83	* A similar idea applies to others RGB color conversions.	157	* A similar idea applies to others RGB color conversions.
84	*/	158	*/
85	out_pixel->a = (u16)src_pixels[3] * 257;	159	out_pixel.a = (u16)a * 257;
86	out_pixel->r = (u16)src_pixels[2] * 257;	160	out_pixel.r = (u16)r * 257;
87	out_pixel->g = (u16)src_pixels[1] * 257;	161	out_pixel.g = (u16)g * 257;
88	out_pixel->b = (u16)src_pixels[0] * 257;	162	out_pixel.b = (u16)b * 257;
89	}
90		163
91	static void XRGB8888_to_argb_u16(u8 src_pixels, struct pixel_argb_u16 out_pixel)	164	return out_pixel;
92	{
93	out_pixel->a = (u16)0xffff;
94	out_pixel->r = (u16)src_pixels[2] * 257;
95	out_pixel->g = (u16)src_pixels[1] * 257;
96	out_pixel->b = (u16)src_pixels[0] * 257;
97	}	165	}
98		166
99	static void ARGB16161616_to_argb_u16(u8 src_pixels, struct pixel_argb_u16 out_pixel)	167	static struct pixel_argb_u16 argb_u16_from_u16161616(u16 a, u16 r, u16 g, u16 b)
100	{	168	{
101	__le16 pixels = (__force __le16 )src_pixels;	169	struct pixel_argb_u16 out_pixel;
102		170
103	out_pixel->a = le16_to_cpu(pixels[3]);	171	out_pixel.a = a;
104	out_pixel->r = le16_to_cpu(pixels[2]);	172	out_pixel.r = r;
105	out_pixel->g = le16_to_cpu(pixels[1]);	173	out_pixel.g = g;
106	out_pixel->b = le16_to_cpu(pixels[0]);	174	out_pixel.b = b;
		175
		176	return out_pixel;
107	}	177	}
108		178
109	static void XRGB16161616_to_argb_u16(u8 src_pixels, struct pixel_argb_u16 out_pixel)	179	static struct pixel_argb_u16 argb_u16_from_le16161616(__le16 a, __le16 r, __le16 g, __le16 b)
110	{	180	{
111	__le16 pixels = (__force __le16 )src_pixels;	181	return argb_u16_from_u16161616(le16_to_cpu(a), le16_to_cpu(r), le16_to_cpu(g),
112		182	le16_to_cpu(b));
113	out_pixel->a = (u16)0xffff;
114	out_pixel->r = le16_to_cpu(pixels[2]);
115	out_pixel->g = le16_to_cpu(pixels[1]);
116	out_pixel->b = le16_to_cpu(pixels[0]);
117	}	183	}
118		184
119	static void RGB565_to_argb_u16(u8 src_pixels, struct pixel_argb_u16 out_pixel)	185	static struct pixel_argb_u16 argb_u16_from_RGB565(const __le16 *pixel)
120	{	186	{
121	__le16 pixels = (__force __le16 )src_pixels;	187	struct pixel_argb_u16 out_pixel;
122		188
123	s64 fp_rb_ratio = drm_fixp_div(drm_int2fixp(65535), drm_int2fixp(31));	189	s64 fp_rb_ratio = drm_fixp_div(drm_int2fixp(65535), drm_int2fixp(31));
124	s64 fp_g_ratio = drm_fixp_div(drm_int2fixp(65535), drm_int2fixp(63));	190	s64 fp_g_ratio = drm_fixp_div(drm_int2fixp(65535), drm_int2fixp(63));
125		191
126	u16 rgb_565 = le16_to_cpu(*pixels);	192	u16 rgb_565 = le16_to_cpu(*pixel);
127	s64 fp_r = drm_int2fixp((rgb_565 >> 11) & 0x1f);	193	s64 fp_r = drm_int2fixp((rgb_565 >> 11) & 0x1f);
128	s64 fp_g = drm_int2fixp((rgb_565 >> 5) & 0x3f);	194	s64 fp_g = drm_int2fixp((rgb_565 >> 5) & 0x3f);
129	s64 fp_b = drm_int2fixp(rgb_565 & 0x1f);	195	s64 fp_b = drm_int2fixp(rgb_565 & 0x1f);
130		196
131	out_pixel->a = (u16)0xffff;	197	out_pixel.a = (u16)0xffff;
132	out_pixel->r = drm_fixp2int_round(drm_fixp_mul(fp_r, fp_rb_ratio));	198	out_pixel.r = drm_fixp2int_round(drm_fixp_mul(fp_r, fp_rb_ratio));
133	out_pixel->g = drm_fixp2int_round(drm_fixp_mul(fp_g, fp_g_ratio));	199	out_pixel.g = drm_fixp2int_round(drm_fixp_mul(fp_g, fp_g_ratio));
134	out_pixel->b = drm_fixp2int_round(drm_fixp_mul(fp_b, fp_rb_ratio));	200	out_pixel.b = drm_fixp2int_round(drm_fixp_mul(fp_b, fp_rb_ratio));
		201
		202	return out_pixel;
135	}	203	}
136		204
137	/**	205	/*
138	* vkms_compose_row - compose a single row of a plane	206	* The following functions are read_line function for each pixel format supported by VKMS.
139	* @stage_buffer: output line with the composed pixels	207	*
140	* @plane: state of the plane that is being composed	208	* They read a line starting at the point @x_start,@y_start following the @direction. The result
141	* @y: y coordinate of the row	209	* is stored in @out_pixel and in the format ARGB16161616.
		210	*
		211	* These functions are very repetitive, but the innermost pixel loops must be kept inside these
		212	* functions for performance reasons. Some benchmarking was done in [1] where having the innermost
		213	* loop factored out of these functions showed a slowdown by a factor of three.
142	*	214	*
143	* This function composes a single row of a plane. It gets the source pixels	215	* [1]: https://lore.kernel.org/dri-devel/d258c8dc-78e9-4509-9037-a98f7f33b3a3@riseup.net/
144	* through the y coordinate (see get_packed_src_addr()) and goes linearly
145	* through the source pixel, reading the pixels and converting it to
146	* ARGB16161616 (see the pixel_read() callback). For rotate-90 and rotate-270,
147	* the source pixels are not traversed linearly. The source pixels are queried
148	* on each iteration in order to traverse the pixels vertically.
149	*/	216	*/
150	void vkms_compose_row(struct line_buffer stage_buffer, struct vkms_plane_state plane, int y)	217
		218	static void ARGB8888_read_line(const struct vkms_plane_state *plane, int x_start, int y_start,
		219	enum pixel_read_direction direction, int count,
		220	struct pixel_argb_u16 out_pixel[])
151	{	221	{
152	struct pixel_argb_u16 *out_pixels = stage_buffer->pixels;	222	struct pixel_argb_u16 *end = out_pixel + count;
153	struct vkms_frame_info *frame_info = plane->frame_info;	223	u8 *src_pixels;
154	u8 *src_pixels = get_packed_src_addr(frame_info, y);
155	int limit = min_t(size_t, drm_rect_width(&frame_info->dst), stage_buffer->n_pixels);
156		224
157	for (size_t x = 0; x < limit; x++, src_pixels += frame_info->cpp) {	225	packed_pixels_addr_1x1(plane->frame_info, x_start, y_start, 0, &src_pixels);
158	int x_pos = get_x_position(frame_info, limit, x);
159		226
160	if (drm_rotation_90_or_270(frame_info->rotation))	227	int step = get_block_step_bytes(plane->frame_info->fb, direction, 0);
161	src_pixels = get_packed_src_addr(frame_info, x + frame_info->rotated.y1)
162	+ frame_info->cpp * y;
163		228
164	plane->pixel_read(src_pixels, &out_pixels[x_pos]);	229	while (out_pixel < end) {
		230	u8 px = (u8 )src_pixels;
		231	*out_pixel = argb_u16_from_u8888(px[3], px[2], px[1], px[0]);
		232	out_pixel += 1;
		233	src_pixels += step;
		234	}
		235	}
		236
		237	static void XRGB8888_read_line(const struct vkms_plane_state *plane, int x_start, int y_start,
		238	enum pixel_read_direction direction, int count,
		239	struct pixel_argb_u16 out_pixel[])
		240	{
		241	struct pixel_argb_u16 *end = out_pixel + count;
		242	u8 *src_pixels;
		243
		244	packed_pixels_addr_1x1(plane->frame_info, x_start, y_start, 0, &src_pixels);
		245
		246	int step = get_block_step_bytes(plane->frame_info->fb, direction, 0);
		247
		248	while (out_pixel < end) {
		249	u8 px = (u8 )src_pixels;
		250	*out_pixel = argb_u16_from_u8888(255, px[2], px[1], px[0]);
		251	out_pixel += 1;
		252	src_pixels += step;
		253	}
		254	}
		255
		256	static void ARGB16161616_read_line(const struct vkms_plane_state *plane, int x_start,
		257	int y_start, enum pixel_read_direction direction, int count,
		258	struct pixel_argb_u16 out_pixel[])
		259	{
		260	struct pixel_argb_u16 *end = out_pixel + count;
		261	u8 *src_pixels;
		262
		263	packed_pixels_addr_1x1(plane->frame_info, x_start, y_start, 0, &src_pixels);
		264
		265	int step = get_block_step_bytes(plane->frame_info->fb, direction, 0);
		266
		267	while (out_pixel < end) {
		268	u16 px = (u16 )src_pixels;
		269	*out_pixel = argb_u16_from_u16161616(px[3], px[2], px[1], px[0]);
		270	out_pixel += 1;
		271	src_pixels += step;
		272	}
		273	}
		274
		275	static void XRGB16161616_read_line(const struct vkms_plane_state *plane, int x_start,
		276	int y_start, enum pixel_read_direction direction, int count,
		277	struct pixel_argb_u16 out_pixel[])
		278	{
		279	struct pixel_argb_u16 *end = out_pixel + count;
		280	u8 *src_pixels;
		281
		282	packed_pixels_addr_1x1(plane->frame_info, x_start, y_start, 0, &src_pixels);
		283
		284	int step = get_block_step_bytes(plane->frame_info->fb, direction, 0);
		285
		286	while (out_pixel < end) {
		287	__le16 px = (__le16 )src_pixels;
		288	*out_pixel = argb_u16_from_le16161616(cpu_to_le16(0xFFFF), px[2], px[1], px[0]);
		289	out_pixel += 1;
		290	src_pixels += step;
		291	}
		292	}
		293
		294	static void RGB565_read_line(const struct vkms_plane_state *plane, int x_start,
		295	int y_start, enum pixel_read_direction direction, int count,
		296	struct pixel_argb_u16 out_pixel[])
		297	{
		298	struct pixel_argb_u16 *end = out_pixel + count;
		299	u8 *src_pixels;
		300
		301	packed_pixels_addr_1x1(plane->frame_info, x_start, y_start, 0, &src_pixels);
		302
		303	int step = get_block_step_bytes(plane->frame_info->fb, direction, 0);
		304
		305	while (out_pixel < end) {
		306	__le16 px = (__le16 )src_pixels;
		307
		308	*out_pixel = argb_u16_from_RGB565(px);
		309	out_pixel += 1;
		310	src_pixels += step;
165	}	311	}
166	}	312	}
167		313
168	/*	314	/*
169	* The following functions take one &struct pixel_argb_u16 and convert it to a specific format.	315	* The following functions take one &struct pixel_argb_u16 and convert it to a specific format.
170	* The result is stored in @dst_pixels.	316	* The result is stored in @out_pixel.
171	*	317	*
172	* They are used in vkms_writeback_row() to convert and store a pixel from the src_buffer to	318	* They are used in vkms_writeback_row() to convert and store a pixel from the src_buffer to
173	* the writeback buffer.	319	* the writeback buffer.
174	*/	320	*/
175	static void argb_u16_to_ARGB8888(u8 dst_pixels, struct pixel_argb_u16 in_pixel)	321	static void argb_u16_to_ARGB8888(u8 out_pixel, const struct pixel_argb_u16 in_pixel)
176	{	322	{
177	/*	323	/*
178	* This sequence below is important because the format's byte order is	324	* This sequence below is important because the format's byte order is
@@ -184,43 +330,43 @@ static void argb_u16_to_ARGB8888(u8 dst_pixels, struct pixel_argb_u16 in_pixel · 5 unmodified lines
184	* \| Addr + 2 \| = Red channel	330	* \| Addr + 2 \| = Red channel
185	* \| Addr + 3 \| = Alpha channel	331	* \| Addr + 3 \| = Alpha channel
186	*/	332	*/
187	dst_pixels[3] = DIV_ROUND_CLOSEST(in_pixel->a, 257);	333	out_pixel[3] = DIV_ROUND_CLOSEST(in_pixel->a, 257);
188	dst_pixels[2] = DIV_ROUND_CLOSEST(in_pixel->r, 257);	334	out_pixel[2] = DIV_ROUND_CLOSEST(in_pixel->r, 257);
189	dst_pixels[1] = DIV_ROUND_CLOSEST(in_pixel->g, 257);	335	out_pixel[1] = DIV_ROUND_CLOSEST(in_pixel->g, 257);
190	dst_pixels[0] = DIV_ROUND_CLOSEST(in_pixel->b, 257);	336	out_pixel[0] = DIV_ROUND_CLOSEST(in_pixel->b, 257);
191	}	337	}
192		338
193	static void argb_u16_to_XRGB8888(u8 dst_pixels, struct pixel_argb_u16 in_pixel)	339	static void argb_u16_to_XRGB8888(u8 out_pixel, const struct pixel_argb_u16 in_pixel)
194	{	340	{
195	dst_pixels[3] = 0xff;	341	out_pixel[3] = 0xff;
196	dst_pixels[2] = DIV_ROUND_CLOSEST(in_pixel->r, 257);	342	out_pixel[2] = DIV_ROUND_CLOSEST(in_pixel->r, 257);
197	dst_pixels[1] = DIV_ROUND_CLOSEST(in_pixel->g, 257);	343	out_pixel[1] = DIV_ROUND_CLOSEST(in_pixel->g, 257);
198	dst_pixels[0] = DIV_ROUND_CLOSEST(in_pixel->b, 257);	344	out_pixel[0] = DIV_ROUND_CLOSEST(in_pixel->b, 257);
199	}	345	}
200		346
201	static void argb_u16_to_ARGB16161616(u8 dst_pixels, struct pixel_argb_u16 in_pixel)	347	static void argb_u16_to_ARGB16161616(u8 out_pixel, const struct pixel_argb_u16 in_pixel)
202	{	348	{
203	__le16 pixels = (__force __le16 )dst_pixels;	349	__le16 pixel = (__le16 )out_pixel;
204		350
205	pixels[3] = cpu_to_le16(in_pixel->a);	351	pixel[3] = cpu_to_le16(in_pixel->a);
206	pixels[2] = cpu_to_le16(in_pixel->r);	352	pixel[2] = cpu_to_le16(in_pixel->r);
207	pixels[1] = cpu_to_le16(in_pixel->g);	353	pixel[1] = cpu_to_le16(in_pixel->g);
208	pixels[0] = cpu_to_le16(in_pixel->b);	354	pixel[0] = cpu_to_le16(in_pixel->b);
209	}	355	}
210		356
211	static void argb_u16_to_XRGB16161616(u8 dst_pixels, struct pixel_argb_u16 in_pixel)	357	static void argb_u16_to_XRGB16161616(u8 out_pixel, const struct pixel_argb_u16 in_pixel)
212	{	358	{
213	__le16 pixels = (__force __le16 )dst_pixels;	359	__le16 pixel = (__le16 )out_pixel;
214		360
215	pixels[3] = cpu_to_le16(0xffff);	361	pixel[3] = cpu_to_le16(0xffff);
216	pixels[2] = cpu_to_le16(in_pixel->r);	362	pixel[2] = cpu_to_le16(in_pixel->r);
217	pixels[1] = cpu_to_le16(in_pixel->g);	363	pixel[1] = cpu_to_le16(in_pixel->g);
218	pixels[0] = cpu_to_le16(in_pixel->b);	364	pixel[0] = cpu_to_le16(in_pixel->b);
219	}	365	}
220		366
221	static void argb_u16_to_RGB565(u8 dst_pixels, struct pixel_argb_u16 in_pixel)	367	static void argb_u16_to_RGB565(u8 out_pixel, const struct pixel_argb_u16 in_pixel)
222	{	368	{
223	__le16 pixels = (__force __le16 )dst_pixels;	369	__le16 pixel = (__le16 )out_pixel;
224		370
225	s64 fp_rb_ratio = drm_fixp_div(drm_int2fixp(65535), drm_int2fixp(31));	371	s64 fp_rb_ratio = drm_fixp_div(drm_int2fixp(65535), drm_int2fixp(31));
226	s64 fp_g_ratio = drm_fixp_div(drm_int2fixp(65535), drm_int2fixp(63));	372	s64 fp_g_ratio = drm_fixp_div(drm_int2fixp(65535), drm_int2fixp(63));
@@ -233,7 +379,7 @@ static void argb_u16_to_RGB565(u8 dst_pixels, struct pixel_argb_u16 in_pixel) · 6 unmodified lines
233	u16 g = drm_fixp2int(drm_fixp_div(fp_g, fp_g_ratio));	379	u16 g = drm_fixp2int(drm_fixp_div(fp_g, fp_g_ratio));
234	u16 b = drm_fixp2int(drm_fixp_div(fp_b, fp_rb_ratio));	380	u16 b = drm_fixp2int(drm_fixp_div(fp_b, fp_rb_ratio));
235		381
236	*pixels = cpu_to_le16(r << 11 \| g << 5 \| b);	382	*pixel = cpu_to_le16(r << 11 \| g << 5 \| b);
237	}	383	}
238		384
239	/**	385	/**
@@ -249,36 +395,47 @@ void vkms_writeback_row(struct vkms_writeback_job *wb, · 9 unmodified lines
249	{	395	{
250	struct vkms_frame_info *frame_info = &wb->wb_frame_info;	396	struct vkms_frame_info *frame_info = &wb->wb_frame_info;
251	int x_dst = frame_info->dst.x1;	397	int x_dst = frame_info->dst.x1;
252	u8 *dst_pixels = packed_pixels_addr(frame_info, x_dst, y);	398	u8 *dst_pixels;
		399	int rem_x, rem_y;
		400
		401	packed_pixels_addr(frame_info, x_dst, y, 0, &dst_pixels, &rem_x, &rem_y);
253	struct pixel_argb_u16 *in_pixels = src_buffer->pixels;	402	struct pixel_argb_u16 *in_pixels = src_buffer->pixels;
254	int x_limit = min_t(size_t, drm_rect_width(&frame_info->dst), src_buffer->n_pixels);	403	int x_limit = min_t(size_t, drm_rect_width(&frame_info->dst), src_buffer->n_pixels);
255		404
256	for (size_t x = 0; x < x_limit; x++, dst_pixels += frame_info->cpp)	405	for (size_t x = 0; x < x_limit; x++, dst_pixels += frame_info->fb->format->cpp[0])
257	wb->pixel_write(dst_pixels, &in_pixels[x]);	406	wb->pixel_write(dst_pixels, &in_pixels[x]);
258	}	407	}
259		408
260	/**	409	/**
261	* get_pixel_conversion_function() - Retrieve the correct read_pixel function for a specific	410	* get_pixel_read_line_function() - Retrieve the correct read_line function for a specific
262	* format. The returned pointer is NULL for unsupported pixel formats. The caller must ensure that	411	* format. The returned pointer is NULL for unsupported pixel formats. The caller must ensure that
263	* the pointer is valid before using it in a vkms_plane_state.	412	* the pointer is valid before using it in a vkms_plane_state.
264	*	413	*
265	* @format: DRM_FORMAT_* value for which to obtain a conversion function (see [drm_fourcc.h])	414	* @format: DRM_FORMAT_* value for which to obtain a conversion function (see [drm_fourcc.h])
266	*/	415	*/
267	void *get_pixel_conversion_function(u32 format)	416	pixel_read_line_t get_pixel_read_line_function(u32 format)
268	{	417	{
269	switch (format) {	418	switch (format) {
270	case DRM_FORMAT_ARGB8888:	419	case DRM_FORMAT_ARGB8888:
271	return &ARGB8888_to_argb_u16;	420	return &ARGB8888_read_line;
272	case DRM_FORMAT_XRGB8888:	421	case DRM_FORMAT_XRGB8888:
273	return &XRGB8888_to_argb_u16;	422	return &XRGB8888_read_line;
274	case DRM_FORMAT_ARGB16161616:	423	case DRM_FORMAT_ARGB16161616:
275	return &ARGB16161616_to_argb_u16;	424	return &ARGB16161616_read_line;
276	case DRM_FORMAT_XRGB16161616:	425	case DRM_FORMAT_XRGB16161616:
277	return &XRGB16161616_to_argb_u16;	426	return &XRGB16161616_read_line;
278	case DRM_FORMAT_RGB565:	427	case DRM_FORMAT_RGB565:
279	return &RGB565_to_argb_u16;	428	return &RGB565_read_line;
280	default:	429	default:
281	return NULL;	430	/*
		431	* This is a bug in vkms_plane_atomic_check(). All the supported
		432	* format must:
		433	* - Be listed in vkms_formats in vkms_plane.c
		434	* - Have a pixel_read callback defined here
		435	*/
		436	pr_err("Pixel format %p4cc is not supported by VKMS planes. This is a kernel bug, atomic check must forbid this configuration.\n",
		437	&format);
		438	BUG();
282	}	439	}
283	}	440	}
284		441
@@ -289,7 +446,7 @@ void *get_pixel_conversion_function(u32 format) · 4 unmodified lines
289	*	446	*
290	* @format: DRM_FORMAT_* value for which to obtain a conversion function (see [drm_fourcc.h])	447	* @format: DRM_FORMAT_* value for which to obtain a conversion function (see [drm_fourcc.h])
291	*/	448	*/
292	void *get_pixel_write_function(u32 format)	449	pixel_write_t get_pixel_write_function(u32 format)
293	{	450	{
294	switch (format) {	451	switch (format) {
295	case DRM_FORMAT_ARGB8888:	452	case DRM_FORMAT_ARGB8888:
@@ -303,6 +460,14 @@ void *get_pixel_write_function(u32 format) · 7 unmodified lines
303	case DRM_FORMAT_RGB565:	460	case DRM_FORMAT_RGB565:
304	return &argb_u16_to_RGB565;	461	return &argb_u16_to_RGB565;
305	default:	462	default:
306	return NULL;	463	/*
		464	* This is a bug in vkms_writeback_atomic_check. All the supported
		465	* format must:
		466	* - Be listed in vkms_wb_formats in vkms_writeback.c
		467	* - Have a pixel_write callback defined here
		468	*/
		469	pr_err("Pixel format %p4cc is not supported by VKMS writeback. This is a kernel bug, atomic check must forbid this configuration.\n",
		470	&format);
		471	BUG();
307	}	472	}
308	}	473	}

drivers/gpu/drm/vkms/vkms_formats.h+2−2

@@ -5,8 +5,8 @@ · 4 unmodified lines
5		5
6	#include "vkms_drv.h"	6	#include "vkms_drv.h"
7		7
8	void *get_pixel_conversion_function(u32 format);	8	pixel_read_line_t get_pixel_read_line_function(u32 format);
9		9
10	void *get_pixel_write_function(u32 format);	10	pixel_write_t get_pixel_write_function(u32 format);
11		11
12	#endif /* _VKMS_FORMATS_H_ */	12	#endif /* _VKMS_FORMATS_H_ */

drivers/gpu/drm/vkms/vkms_output.c+16−33

 	.get_modes    = vkms_conn_get_modes,
 };
-static int vkms_add_overlay_plane(struct vkms_device *vkmsdev, int index,
+int vkms_output_init(struct vkms_device *vkmsdev)
-				  struct drm_crtc *crtc)
-{
-	struct vkms_plane *overlay;
-	overlay = vkms_plane_init(vkmsdev, DRM_PLANE_TYPE_OVERLAY, index);
-	if (IS_ERR(overlay))
-		return PTR_ERR(overlay);
-	if (!overlay->base.possible_crtcs)
-		overlay->base.possible_crtcs = drm_crtc_mask(crtc);
-	return 0;
-}
-int vkms_output_init(struct vkms_device *vkmsdev, int index)
 {
 	struct vkms_output *output = &vkmsdev->output;
 	struct drm_device *dev = &vkmsdev->drm;
 	struct drm_connector *connector = &output->connector;
 	struct drm_encoder *encoder = &output->encoder;
 	struct drm_crtc *crtc = &output->crtc;
-	struct vkms_plane *primary, *cursor = NULL;
+	struct vkms_plane *primary, *overlay, *cursor = NULL;
 	int ret;
 	int writeback;
 	unsigned int n;
 	 * The overlay and cursor planes are not mandatory, but can be used to perform complex
 	 * composition.
 	 */
-	primary = vkms_plane_init(vkmsdev, DRM_PLANE_TYPE_PRIMARY, index);
+	primary = vkms_plane_init(vkmsdev, DRM_PLANE_TYPE_PRIMARY);
 	if (IS_ERR(primary))
 		return PTR_ERR(primary);
-	if (vkmsdev->config->overlay) {
-		for (n = 0; n < NUM_OVERLAY_PLANES; n++) {
-			ret = vkms_add_overlay_plane(vkmsdev, index, crtc);
-			if (ret)
-				return ret;
-		}
-	}
 	if (vkmsdev->config->cursor) {
-		cursor = vkms_plane_init(vkmsdev, DRM_PLANE_TYPE_CURSOR, index);
+		cursor = vkms_plane_init(vkmsdev, DRM_PLANE_TYPE_CURSOR);
 		if (IS_ERR(cursor))
 			return PTR_ERR(cursor);
 	}
-	/* [1]: Allocation of a CRTC, its index will be BIT(0) = 1 */
 	ret = vkms_crtc_init(dev, crtc, &primary->base, &cursor->base);
 	if (ret)
 		return ret;
+	if (vkmsdev->config->overlay) {
+		for (n = 0; n < NUM_OVERLAY_PLANES; n++) {
+			overlay = vkms_plane_init(vkmsdev, DRM_PLANE_TYPE_OVERLAY);
+			if (IS_ERR(overlay)) {
+				DRM_DEV_ERROR(dev->dev, "Failed to init vkms plane\n");
+				return PTR_ERR(overlay);
+			}
+			overlay->base.possible_crtcs = drm_crtc_mask(crtc);
+		}
+	}
 	ret = drm_connector_init(dev, connector, &vkms_connector_funcs,
 				 DRM_MODE_CONNECTOR_VIRTUAL);
 	if (ret) {
 		DRM_ERROR("Failed to init encoder\n");
 		goto err_encoder;
 	}
-	/*
+	encoder->possible_crtcs = drm_crtc_mask(crtc);
-	 * This is a hardcoded value to select crtc for the encoder.
-	 * BIT(0) here designate the first registered CRTC, the one allocated in [1]
-	 */
-	encoder->possible_crtcs = BIT(0);
 	ret = drm_connector_attach_encoder(connector, encoder);
 	if (ret) {

drivers/gpu/drm/vkms/vkms_plane.c+5−16

@@ -112,23 +112,12 @@ static void vkms_plane_atomic_update(struct drm_plane *plane, · 111 unmodified lines
112	frame_info = vkms_plane_state->frame_info;	112	frame_info = vkms_plane_state->frame_info;
113	memcpy(&frame_info->src, &new_state->src, sizeof(struct drm_rect));	113	memcpy(&frame_info->src, &new_state->src, sizeof(struct drm_rect));
114	memcpy(&frame_info->dst, &new_state->dst, sizeof(struct drm_rect));	114	memcpy(&frame_info->dst, &new_state->dst, sizeof(struct drm_rect));
115	memcpy(&frame_info->rotated, &new_state->dst, sizeof(struct drm_rect));
116	frame_info->fb = fb;	115	frame_info->fb = fb;
117	memcpy(&frame_info->map, &shadow_plane_state->data, sizeof(frame_info->map));	116	memcpy(&frame_info->map, &shadow_plane_state->data, sizeof(frame_info->map));
118	drm_framebuffer_get(frame_info->fb);	117	drm_framebuffer_get(frame_info->fb);
119	frame_info->rotation = drm_rotation_simplify(new_state->rotation, DRM_MODE_ROTATE_0 \|	118	frame_info->rotation = new_state->rotation;
120	DRM_MODE_ROTATE_90 \|	119
121	DRM_MODE_ROTATE_270 \|	120	vkms_plane_state->pixel_read_line = get_pixel_read_line_function(fmt);
122	DRM_MODE_REFLECT_X \|
123	DRM_MODE_REFLECT_Y);
124
125	drm_rect_rotate(&frame_info->rotated, drm_rect_width(&frame_info->rotated),
126	drm_rect_height(&frame_info->rotated), frame_info->rotation);
127
128	frame_info->offset = fb->offsets[0];
129	frame_info->pitch = fb->pitches[0];
130	frame_info->cpp = fb->format->cpp[0];
131	vkms_plane_state->pixel_read = get_pixel_conversion_function(fmt);
132	}	121	}
133		122
134	static int vkms_plane_atomic_check(struct drm_plane *plane,	123	static int vkms_plane_atomic_check(struct drm_plane *plane,
@@ -198,12 +187,12 @@ static const struct drm_plane_helper_funcs vkms_plane_helper_funcs = { · 63 unmodified lines
198	};	187	};
199		188
200	struct vkms_plane vkms_plane_init(struct vkms_device vkmsdev,	189	struct vkms_plane vkms_plane_init(struct vkms_device vkmsdev,
201	enum drm_plane_type type, int index)	190	enum drm_plane_type type)
202	{	191	{
203	struct drm_device *dev = &vkmsdev->drm;	192	struct drm_device *dev = &vkmsdev->drm;
204	struct vkms_plane *plane;	193	struct vkms_plane *plane;
205		194
206	plane = drmm_universal_plane_alloc(dev, struct vkms_plane, base, 1 << index,	195	plane = drmm_universal_plane_alloc(dev, struct vkms_plane, base, 0,
207	&vkms_plane_funcs,	196	&vkms_plane_funcs,
208	vkms_formats, ARRAY_SIZE(vkms_formats),	197	vkms_formats, ARRAY_SIZE(vkms_formats),
209	NULL, type, NULL);	198	NULL, type, NULL);

drivers/gpu/drm/vkms/vkms_writeback.c+0−5

 	crtc_state->active_writeback = active_wb;
 	crtc_state->wb_pending = true;
 	spin_unlock_irq(&output->composer_lock);
-	wb_frame_info->offset = fb->offsets[0];
-	wb_frame_info->pitch = fb->pitches[0];
-	wb_frame_info->cpp = fb->format->cpp[0];
 	drm_writeback_queue_job(wb_conn, connector_state);
 	active_wb->pixel_write = get_pixel_write_function(wb_format);
 	drm_rect_init(&wb_frame_info->src, 0, 0, crtc_width, crtc_height);

drivers/gpu/drm/vmwgfx/vmwgfx_drv.c+1−2

 #include "vmwgfx_vkms.h"
 #include "ttm_object.h"
-#include <drm/drm_client_setup.h>
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_fbdev_ttm.h>
 #include <drm/drm_gem_ttm_helper.h>
 	.fops = &vmwgfx_driver_fops,
 	.name = VMWGFX_DRIVER_NAME,
 	.desc = VMWGFX_DRIVER_DESC,
-	.date = VMWGFX_DRIVER_DATE,
 	.major = VMWGFX_DRIVER_MAJOR,
 	.minor = VMWGFX_DRIVER_MINOR,
 	.patchlevel = VMWGFX_DRIVER_PATCHLEVEL

drivers/gpu/drm/vmwgfx/vmwgfx_drv.h+0−1

@@ -57,7 +57,6 @@ · 56 unmodified lines
57		57
58		58
59	#define VMWGFX_DRIVER_NAME "vmwgfx"	59	#define VMWGFX_DRIVER_NAME "vmwgfx"
60	#define VMWGFX_DRIVER_DATE "20211206"
61	#define VMWGFX_DRIVER_MAJOR 2	60	#define VMWGFX_DRIVER_MAJOR 2
62	#define VMWGFX_DRIVER_MINOR 20	61	#define VMWGFX_DRIVER_MINOR 20
63	#define VMWGFX_DRIVER_PATCHLEVEL 0	62	#define VMWGFX_DRIVER_PATCHLEVEL 0

drivers/gpu/drm/vmwgfx/vmwgfx_ldu.c+0−1

@@ -479,7 +479,6 @@ static int vmw_ldu_init(struct vmw_private *dev_priv, unsigned unit) · 478 unmodified lines
479	}	479	}
480		480
481	drm_connector_helper_add(connector, &vmw_ldu_connector_helper_funcs);	481	drm_connector_helper_add(connector, &vmw_ldu_connector_helper_funcs);
482	connector->status = vmw_du_connector_detect(connector, true);
483		482
484	ret = drm_encoder_init(dev, encoder, &vmw_legacy_encoder_funcs,	483	ret = drm_encoder_init(dev, encoder, &vmw_legacy_encoder_funcs,
485	DRM_MODE_ENCODER_VIRTUAL, NULL);	484	DRM_MODE_ENCODER_VIRTUAL, NULL);

drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c+0−1

@@ -868,7 +868,6 @@ static int vmw_sou_init(struct vmw_private *dev_priv, unsigned unit) · 867 unmodified lines
868	}	868	}
869		869
870	drm_connector_helper_add(connector, &vmw_sou_connector_helper_funcs);	870	drm_connector_helper_add(connector, &vmw_sou_connector_helper_funcs);
871	connector->status = vmw_du_connector_detect(connector, true);
872		871
873	ret = drm_encoder_init(dev, encoder, &vmw_screen_object_encoder_funcs,	872	ret = drm_encoder_init(dev, encoder, &vmw_screen_object_encoder_funcs,
874	DRM_MODE_ENCODER_VIRTUAL, NULL);	873	DRM_MODE_ENCODER_VIRTUAL, NULL);

drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c+0−1

@@ -1593,7 +1593,6 @@ static int vmw_stdu_init(struct vmw_private *dev_priv, unsigned unit) · 1592 unmodified lines
1593	}	1593	}
1594		1594
1595	drm_connector_helper_add(connector, &vmw_stdu_connector_helper_funcs);	1595	drm_connector_helper_add(connector, &vmw_stdu_connector_helper_funcs);
1596	connector->status = vmw_du_connector_detect(connector, false);
1597		1596
1598	ret = drm_encoder_init(dev, encoder, &vmw_stdu_encoder_funcs,	1597	ret = drm_encoder_init(dev, encoder, &vmw_stdu_encoder_funcs,
1599	DRM_MODE_ENCODER_VIRTUAL, NULL);	1598	DRM_MODE_ENCODER_VIRTUAL, NULL);

drivers/gpu/drm/xe/xe_bo_types.h+0−1

@@ -10,7 +10,6 @@ · 9 unmodified lines
10		10
11	#include <drm/ttm/ttm_bo.h>	11	#include <drm/ttm/ttm_bo.h>
12	#include <drm/ttm/ttm_device.h>	12	#include <drm/ttm/ttm_device.h>
13	#include <drm/ttm/ttm_execbuf_util.h>
14	#include <drm/ttm/ttm_placement.h>	13	#include <drm/ttm/ttm_placement.h>
15		14
16	#include "xe_ggtt_types.h"	15	#include "xe_ggtt_types.h"

drivers/gpu/drm/xe/xe_device.c+0−1

@@ -269,7 +269,6 @@ static struct drm_driver driver = { · 268 unmodified lines
269	.fops = &xe_driver_fops,	269	.fops = &xe_driver_fops,
270	.name = DRIVER_NAME,	270	.name = DRIVER_NAME,
271	.desc = DRIVER_DESC,	271	.desc = DRIVER_DESC,
272	.date = DRIVER_DATE,
273	.major = DRIVER_MAJOR,	272	.major = DRIVER_MAJOR,
274	.minor = DRIVER_MINOR,	273	.minor = DRIVER_MINOR,
275	.patchlevel = DRIVER_PATCHLEVEL,	274	.patchlevel = DRIVER_PATCHLEVEL,

drivers/gpu/drm/xe/xe_drv.h+0−1

@@ -10,7 +10,6 @@ · 9 unmodified lines
10		10
11	#define DRIVER_NAME "xe"	11	#define DRIVER_NAME "xe"
12	#define DRIVER_DESC "Intel Xe Graphics"	12	#define DRIVER_DESC "Intel Xe Graphics"
13	#define DRIVER_DATE "20201103"
14		13
15	/* Interface history:	14	/* Interface history:
16	*	15	*

drivers/gpu/drm/xe/xe_gt_pagefault.c+0−1

@@ -10,7 +10,6 @@ · 9 unmodified lines
10		10
11	#include <drm/drm_exec.h>	11	#include <drm/drm_exec.h>
12	#include <drm/drm_managed.h>	12	#include <drm/drm_managed.h>
13	#include <drm/ttm/ttm_execbuf_util.h>
14		13
15	#include "abi/guc_actions_abi.h"	14	#include "abi/guc_actions_abi.h"
16	#include "xe_bo.h"	15	#include "xe_bo.h"

drivers/gpu/drm/xe/xe_vm.c+0−1

@@ -10,7 +10,6 @@ · 9 unmodified lines
10		10
11	#include <drm/drm_exec.h>	11	#include <drm/drm_exec.h>
12	#include <drm/drm_print.h>	12	#include <drm/drm_print.h>
13	#include <drm/ttm/ttm_execbuf_util.h>
14	#include <drm/ttm/ttm_tt.h>	13	#include <drm/ttm/ttm_tt.h>
15	#include <uapi/drm/xe_drm.h>	14	#include <uapi/drm/xe_drm.h>
16	#include <linux/ascii85.h>	15	#include <linux/ascii85.h>

drivers/gpu/drm/xe/xe_vm.h+0−1

@@ -17,7 +17,6 @@ struct drm_printer; · 16 unmodified lines
17	struct drm_file;	17	struct drm_file;
18		18
19	struct ttm_buffer_object;	19	struct ttm_buffer_object;
20	struct ttm_validate_buffer;
21		20
22	struct xe_exec_queue;	21	struct xe_exec_queue;
23	struct xe_file;	22	struct xe_file;

drivers/gpu/drm/xen/xen_drm_front.c+0−6

 	.fops                      = &xen_drm_dev_fops,
 	.name                      = "xendrm-du",
 	.desc                      = "Xen PV DRM Display Unit",
-	.date                      = "20180221",
 	.major                     = 1,
 	.minor                     = 0,
 	if (ret)
 		goto fail_register;
-	DRM_INFO("Initialized %s %d.%d.%d %s on minor %d\n",
-		 xen_drm_driver.name, xen_drm_driver.major,
-		 xen_drm_driver.minor, xen_drm_driver.patchlevel,
-		 xen_drm_driver.date, drm_dev->primary->index);
 	return 0;
 fail_register:

drivers/gpu/drm/xlnx/zynqmp_kms.c+1−2

  * - Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  */
+#include <drm/clients/drm_client_setup.h>
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_blend.h>
 #include <drm/drm_bridge.h>
 #include <drm/drm_bridge_connector.h>
-#include <drm/drm_client_setup.h>
 #include <drm/drm_connector.h>
 #include <drm/drm_crtc.h>
 #include <drm/drm_device.h>
 	.name				= "zynqmp-dpsub",
 	.desc				= "Xilinx DisplayPort Subsystem Driver",
-	.date				= "20130509",
 	.major				= 1,
 	.minor				= 0,
 };

include/drm/clients/drm_client_setup.hrenamed from include/drm/drm_client_setup.h+0−0

File renamed without changes.

include/drm/display/drm_dp_helper.h+5−0

 				     enum drm_dp_phy dp_phy,
 				     u8 link_status[DP_LINK_STATUS_SIZE]);
+int drm_dp_dpcd_write_payload(struct drm_dp_aux *aux,
+			      int vcpid, u8 start_time_slot, u8 time_slot_count);
+int drm_dp_dpcd_clear_payload(struct drm_dp_aux *aux);
+int drm_dp_dpcd_poll_act_handled(struct drm_dp_aux *aux, int timeout_ms);
 bool drm_dp_send_real_edid_checksum(struct drm_dp_aux *aux,
 				    u8 real_edid_checksum);

include/drm/drm_drv.h+0−2

@@ -401,8 +401,6 @@ struct drm_driver { · 400 unmodified lines
401	char *name;	401	char *name;
402	/** @desc: driver description */	402	/** @desc: driver description */
403	char *desc;	403	char *desc;
404	/** @date: driver date, unused, to be removed */
405	char *date;
406		404
407	/**	405	/**
408	* @driver_features:	406	* @driver_features:

include/drm/drm_utils.h+4−0

@@ -12,8 +12,12 @@ · 11 unmodified lines
12		12
13	#include <linux/types.h>	13	#include <linux/types.h>
14		14
		15	struct drm_edid;
		16
15	int drm_get_panel_orientation_quirk(int width, int height);	17	int drm_get_panel_orientation_quirk(int width, int height);
16		18
		19	int drm_get_panel_min_brightness_quirk(const struct drm_edid *edid);
		20
17	signed long drm_timeout_abs_to_jiffies(int64_t timeout_nsec);	21	signed long drm_timeout_abs_to_jiffies(int64_t timeout_nsec);
18		22
19	#endif	23	#endif

include/trace/events/amdxdna.h+101−0

@@ -0,0 +1,101 @@
	1	/* SPDX-License-Identifier: GPL-2.0 */
	2	/*
	3	* Copyright (C) 2023-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#undef TRACE_SYSTEM
	7	#define TRACE_SYSTEM amdxdna
	8
	9	#if !defined(_TRACE_AMDXDNA_H) \|\| defined(TRACE_HEADER_MULTI_READ)
	10	#define _TRACE_AMDXDNA_H
	11
	12	#include <drm/gpu_scheduler.h>
	13	#include <linux/tracepoint.h>
	14
	15	TRACE_EVENT(amdxdna_debug_point,
	16	TP_PROTO(const char name, u64 number, const char str),
	17
	18	TP_ARGS(name, number, str),
	19
	20	TP_STRUCT__entry(__string(name, name)
	21	__field(u64, number)
	22	__string(str, str)),
	23
	24	TP_fast_assign(__assign_str(name);
	25	__entry->number = number;
	26	__assign_str(str);),
	27
	28	TP_printk("%s:%llu %s", __get_str(name), __entry->number,
	29	__get_str(str))
	30	);
	31
	32	TRACE_EVENT(xdna_job,
	33	TP_PROTO(struct drm_sched_job sched_job, const char name, const char *str, u64 seq),
	34
	35	TP_ARGS(sched_job, name, str, seq),
	36
	37	TP_STRUCT__entry(__string(name, name)
	38	__string(str, str)
	39	__field(u64, fence_context)
	40	__field(u64, fence_seqno)
	41	__field(u64, seq)),
	42
	43	TP_fast_assign(__assign_str(name);
	44	__assign_str(str);
	45	__entry->fence_context = sched_job->s_fence->finished.context;
	46	__entry->fence_seqno = sched_job->s_fence->finished.seqno;
	47	__entry->seq = seq;),
	48
	49	TP_printk("fence=(context:%llu, seqno:%lld), %s seq#:%lld %s",
	50	__entry->fence_context, __entry->fence_seqno,
	51	__get_str(name), __entry->seq,
	52	__get_str(str))
	53	);
	54
	55	DECLARE_EVENT_CLASS(xdna_mbox_msg,
	56	TP_PROTO(char *name, u8 chann_id, u32 opcode, u32 msg_id),
	57
	58	TP_ARGS(name, chann_id, opcode, msg_id),
	59
	60	TP_STRUCT__entry(__string(name, name)
	61	__field(u32, chann_id)
	62	__field(u32, opcode)
	63	__field(u32, msg_id)),
	64
	65	TP_fast_assign(__assign_str(name);
	66	__entry->chann_id = chann_id;
	67	__entry->opcode = opcode;
	68	__entry->msg_id = msg_id;),
	69
	70	TP_printk("%s.%d id 0x%x opcode 0x%x", __get_str(name),
	71	__entry->chann_id, __entry->msg_id, __entry->opcode)
	72	);
	73
	74	DEFINE_EVENT(xdna_mbox_msg, mbox_set_tail,
	75	TP_PROTO(char *name, u8 chann_id, u32 opcode, u32 id),
	76	TP_ARGS(name, chann_id, opcode, id)
	77	);
	78
	79	DEFINE_EVENT(xdna_mbox_msg, mbox_set_head,
	80	TP_PROTO(char *name, u8 chann_id, u32 opcode, u32 id),
	81	TP_ARGS(name, chann_id, opcode, id)
	82	);
	83
	84	TRACE_EVENT(mbox_irq_handle,
	85	TP_PROTO(char *name, int irq),
	86
	87	TP_ARGS(name, irq),
	88
	89	TP_STRUCT__entry(__string(name, name)
	90	__field(int, irq)),
	91
	92	TP_fast_assign(__assign_str(name);
	93	__entry->irq = irq;),
	94
	95	TP_printk("%s.%d", __get_str(name), __entry->irq)
	96	);
	97
	98	#endif /* !defined(_TRACE_AMDXDNA_H) \|\| defined(TRACE_HEADER_MULTI_READ) */
	99
	100	/* This part must be outside protection */
	101	#include <trace/define_trace.h>

include/uapi/drm/amdxdna_accel.h+436−0

@@ -0,0 +1,436 @@
	1	/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
	2	/*
	3	* Copyright (C) 2022-2024, Advanced Micro Devices, Inc.
	4	*/
	5
	6	#ifndef _UAPI_AMDXDNA_ACCEL_H_
	7	#define _UAPI_AMDXDNA_ACCEL_H_
	8
	9	#include <linux/stddef.h>
	10	#include "drm.h"
	11
	12	#if defined(__cplusplus)
	13	extern "C" {
	14	#endif
	15
	16	#define AMDXDNA_INVALID_CMD_HANDLE (~0UL)
	17	#define AMDXDNA_INVALID_ADDR (~0UL)
	18	#define AMDXDNA_INVALID_CTX_HANDLE 0
	19	#define AMDXDNA_INVALID_BO_HANDLE 0
	20	#define AMDXDNA_INVALID_FENCE_HANDLE 0
	21
	22	enum amdxdna_device_type {
	23	AMDXDNA_DEV_TYPE_UNKNOWN = -1,
	24	AMDXDNA_DEV_TYPE_KMQ,
	25	};
	26
	27	enum amdxdna_drm_ioctl_id {
	28	DRM_AMDXDNA_CREATE_HWCTX,
	29	DRM_AMDXDNA_DESTROY_HWCTX,
	30	DRM_AMDXDNA_CONFIG_HWCTX,
	31	DRM_AMDXDNA_CREATE_BO,
	32	DRM_AMDXDNA_GET_BO_INFO,
	33	DRM_AMDXDNA_SYNC_BO,
	34	DRM_AMDXDNA_EXEC_CMD,
	35	DRM_AMDXDNA_GET_INFO,
	36	};
	37
	38	/**
	39	* struct qos_info - QoS information for driver.
	40	* @gops: Giga operations per second.
	41	* @fps: Frames per second.
	42	* @dma_bandwidth: DMA bandwidtha.
	43	* @latency: Frame response latency.
	44	* @frame_exec_time: Frame execution time.
	45	* @priority: Request priority.
	46	*
	47	* User program can provide QoS hints to driver.
	48	*/
	49	struct amdxdna_qos_info {
	50	__u32 gops;
	51	__u32 fps;
	52	__u32 dma_bandwidth;
	53	__u32 latency;
	54	__u32 frame_exec_time;
	55	__u32 priority;
	56	};
	57
	58	/**
	59	* struct amdxdna_drm_create_hwctx - Create hardware context.
	60	* @ext: MBZ.
	61	* @ext_flags: MBZ.
	62	* @qos_p: Address of QoS info.
	63	* @umq_bo: BO handle for user mode queue(UMQ).
	64	* @log_buf_bo: BO handle for log buffer.
	65	* @max_opc: Maximum operations per cycle.
	66	* @num_tiles: Number of AIE tiles.
	67	* @mem_size: Size of AIE tile memory.
	68	* @umq_doorbell: Returned offset of doorbell associated with UMQ.
	69	* @handle: Returned hardware context handle.
	70	* @syncobj_handle: Returned syncobj handle for command completion.
	71	*/
	72	struct amdxdna_drm_create_hwctx {
	73	__u64 ext;
	74	__u64 ext_flags;
	75	__u64 qos_p;
	76	__u32 umq_bo;
	77	__u32 log_buf_bo;
	78	__u32 max_opc;
	79	__u32 num_tiles;
	80	__u32 mem_size;
	81	__u32 umq_doorbell;
	82	__u32 handle;
	83	__u32 syncobj_handle;
	84	};
	85
	86	/**
	87	* struct amdxdna_drm_destroy_hwctx - Destroy hardware context.
	88	* @handle: Hardware context handle.
	89	* @pad: Structure padding.
	90	*/
	91	struct amdxdna_drm_destroy_hwctx {
	92	__u32 handle;
	93	__u32 pad;
	94	};
	95
	96	/**
	97	* struct amdxdna_cu_config - configuration for one CU
	98	* @cu_bo: CU configuration buffer bo handle.
	99	* @cu_func: Function of a CU.
	100	* @pad: Structure padding.
	101	*/
	102	struct amdxdna_cu_config {
	103	__u32 cu_bo;
	104	__u8 cu_func;
	105	__u8 pad[3];
	106	};
	107
	108	/**
	109	* struct amdxdna_hwctx_param_config_cu - configuration for CUs in hardware context
	110	* @num_cus: Number of CUs to configure.
	111	* @pad: Structure padding.
	112	* @cu_configs: Array of CU configurations of struct amdxdna_cu_config.
	113	*/
	114	struct amdxdna_hwctx_param_config_cu {
	115	__u16 num_cus;
	116	__u16 pad[3];

320 more lines aren’t shown here.

include/uapi/drm/v3d_drm.h+18−0

This diff is large: its 39 lines aren’t shown here.