lael-labs/linux

Documentation/gpu/drm-usage-stats.rst+19−2

@@ -112,6 +112,19 @@ larger value within a reasonable period. Upon observing a value lower than what · 111 unmodified lines
112	was previously read, userspace is expected to stay with that larger previous	112	was previously read, userspace is expected to stay with that larger previous
113	value until a monotonic update is seen.	113	value until a monotonic update is seen.
114		114
		115	- drm-total-cycles-<keystr>: <uint>
		116
		117	Engine identifier string must be the same as the one specified in the
		118	drm-cycles-<keystr> tag and shall contain the total number cycles for the given
		119	engine.
		120
		121	This is a timestamp in GPU unspecified unit that matches the update rate
		122	of drm-cycles-<keystr>. For drivers that implement this interface, the engine
		123	utilization can be calculated entirely on the GPU clock domain, without
		124	considering the CPU sleep time between 2 samples.
		125
		126	A driver may implement either this key or drm-maxfreq-<keystr>, but not both.
		127
115	- drm-maxfreq-<keystr>: <uint> [Hz\|MHz\|KHz]	128	- drm-maxfreq-<keystr>: <uint> [Hz\|MHz\|KHz]
116		129
117	Engine identifier string must be the same as the one specified in the	130	Engine identifier string must be the same as the one specified in the
@@ -121,6 +134,9 @@ percentage utilization of the engine, whereas drm-engine-<keystr> only reflects · 3 unmodified lines
121	time active without considering what frequency the engine is operating as a	134	time active without considering what frequency the engine is operating as a
122	percentage of its maximum frequency.	135	percentage of its maximum frequency.
123		136
		137	A driver may implement either this key or drm-total-cycles-<keystr>, but not
		138	both.
		139
124	Memory	140	Memory
125	^^^^^^	141	^^^^^^
126		142
@@ -168,5 +184,6 @@ be documented above and where possible, aligned with other drivers. · 41 unmodified lines
168	Driver specific implementations	184	Driver specific implementations
169	-------------------------------	185	-------------------------------
170		186
171	:ref:`i915-usage-stats`	187	* :ref:`i915-usage-stats`
172	:ref:`panfrost-usage-stats`	188	* :ref:`panfrost-usage-stats`
		189	* :ref:`xe-usage-stats`

Documentation/gpu/xe/index.rst+1−0

@@ -23,3 +23,4 @@ DG2, etc is provided to prototype the driver. · 22 unmodified lines
23	xe_firmware	23	xe_firmware
24	xe_tile	24	xe_tile
25	xe_debugging	25	xe_debugging
		26	xe-drm-usage-stats.rst

Documentation/gpu/xe/xe-drm-usage-stats.rst+10−0

@@ -0,0 +1,10 @@
	1	.. SPDX-License-Identifier: GPL-2.0+
	2
	3	.. _xe-usage-stats:
	4
	5	========================================
	6	Xe DRM client usage stats implementation
	7	========================================
	8
	9	.. kernel-doc:: drivers/gpu/drm/xe/xe_drm_client.c
	10	:doc: DRM Client usage stats

MAINTAINERS+0−1

@@ -11034,7 +11034,6 @@ F: include/uapi/drm/i915_drm.h · 11033 unmodified lines
11034		11034
11035	INTEL DRM XE DRIVER (Lunar Lake and newer)	11035	INTEL DRM XE DRIVER (Lunar Lake and newer)
11036	M: Lucas De Marchi <lucas.demarchi@intel.com>	11036	M: Lucas De Marchi <lucas.demarchi@intel.com>
11037	M: Oded Gabbay <ogabbay@kernel.org>
11038	M: Thomas Hellström <thomas.hellstrom@linux.intel.com>	11037	M: Thomas Hellström <thomas.hellstrom@linux.intel.com>
11039	L: intel-xe@lists.freedesktop.org	11038	L: intel-xe@lists.freedesktop.org
11040	S: Supported	11039	S: Supported

drivers/gpu/drm/i915/display/intel_vga.c+1−0

@@ -3,6 +3,7 @@ · 2 unmodified lines
3	* Copyright © 2019 Intel Corporation	3	* Copyright © 2019 Intel Corporation
4	*/	4	*/
5		5
		6	#include <linux/delay.h>
6	#include <linux/vgaarb.h>	7	#include <linux/vgaarb.h>
7		8
8	#include <video/vga.h>	9	#include <video/vga.h>

drivers/gpu/drm/xe/Kconfig.debug+0−10

@@ -61,16 +61,6 @@ config DRM_XE_DEBUG_MEM · 60 unmodified lines
61		61
62	If in doubt, say "N".	62	If in doubt, say "N".
63		63
64	config DRM_XE_SIMPLE_ERROR_CAPTURE
65	bool "Enable simple error capture to dmesg on job timeout"
66	default n
67	help
68	Choose this option when debugging an unexpected job timeout
69
70	Recommended for driver developers only.
71
72	If in doubt, say "N".
73
74	config DRM_XE_KUNIT_TEST	64	config DRM_XE_KUNIT_TEST
75	tristate "KUnit tests for the drm xe driver" if !KUNIT_ALL_TESTS	65	tristate "KUnit tests for the drm xe driver" if !KUNIT_ALL_TESTS
76	depends on DRM_XE && KUNIT && DEBUG_FS	66	depends on DRM_XE && KUNIT && DEBUG_FS

drivers/gpu/drm/xe/Makefile+9−25

 # Makefile for the drm device driver.  This driver provides support for the
 # Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher.
-# Unconditionally enable W=1 warnings locally
+# Enable W=1 warnings not enabled in drm subsystem Makefile
-# --- begin copy-paste W=1 warnings from scripts/Makefile.extrawarn
-subdir-ccflags-y += -Wextra -Wunused -Wno-unused-parameter
-subdir-ccflags-y += -Wmissing-declarations
-subdir-ccflags-y += $(call cc-option, -Wrestrict)
-subdir-ccflags-y += -Wmissing-format-attribute
-subdir-ccflags-y += -Wmissing-prototypes
-subdir-ccflags-y += -Wold-style-definition
-subdir-ccflags-y += -Wmissing-include-dirs
-subdir-ccflags-y += $(call cc-option, -Wunused-but-set-variable)
-subdir-ccflags-y += $(call cc-option, -Wunused-const-variable)
-subdir-ccflags-y += $(call cc-option, -Wpacked-not-aligned)
-subdir-ccflags-y += $(call cc-option, -Wformat-overflow)
 subdir-ccflags-y += $(call cc-option, -Wformat-truncation)
-subdir-ccflags-y += $(call cc-option, -Wstringop-truncation)
-# The following turn off the warnings enabled by -Wextra
-ifeq ($(findstring 2, $(KBUILD_EXTRA_WARN)),)
-subdir-ccflags-y += -Wno-missing-field-initializers
-subdir-ccflags-y += -Wno-type-limits
-subdir-ccflags-y += -Wno-shift-negative-value
-endif
-ifeq ($(findstring 3, $(KBUILD_EXTRA_WARN)),)
-subdir-ccflags-y += -Wno-sign-compare
-endif
-# --- end copy-paste
 # Enable -Werror in CI and development
 subdir-ccflags-$(CONFIG_DRM_XE_WERROR) += -Werror
 	xe_gt_mcr.o \
 	xe_gt_pagefault.o \
 	xe_gt_sysfs.o \
-	xe_gt_throttle_sysfs.o \
+	xe_gt_throttle.o \
 	xe_gt_tlb_invalidation.o \
 	xe_gt_topology.o \
 	xe_guc.o \
 	xe_uc_debugfs.o \
 	xe_uc_fw.o \
 	xe_vm.o \
+	xe_vram.o \
 	xe_vram_freq.o \
 	xe_wait_user_fence.o \
 	xe_wa.o \
 # graphics virtualization (SR-IOV) support
 xe-y += \
+	xe_gt_sriov_vf.o \
+	xe_gt_sriov_vf_debugfs.o \
 	xe_guc_relay.o \
 	xe_memirq.o \
 	xe_sriov.o
 	xe_gt_sriov_pf.o \
 	xe_gt_sriov_pf_config.o \
 	xe_gt_sriov_pf_control.o \
+	xe_gt_sriov_pf_debugfs.o \
+	xe_gt_sriov_pf_monitor.o \
 	xe_gt_sriov_pf_policy.o \
+	xe_gt_sriov_pf_service.o \
 	xe_lmtt.o \
 	xe_lmtt_2l.o \
 	xe_lmtt_ml.o \
+	xe_pci_sriov.o \
 	xe_sriov_pf.o
 # include helpers for tests even when XE is built-in

drivers/gpu/drm/xe/abi/guc_actions_sriov_abi.h+190−0

@@ -171,6 +171,36 @@ · 170 unmodified lines
171	#define VF2GUC_RELAY_TO_PF_REQUEST_MSG_n_RELAY_DATAx GUC_HXG_REQUEST_MSG_n_DATAn	171	#define VF2GUC_RELAY_TO_PF_REQUEST_MSG_n_RELAY_DATAx GUC_HXG_REQUEST_MSG_n_DATAn
172	#define VF2GUC_RELAY_TO_PF_REQUEST_MSG_NUM_RELAY_DATA GUC_RELAY_MSG_MAX_LEN	172	#define VF2GUC_RELAY_TO_PF_REQUEST_MSG_NUM_RELAY_DATA GUC_RELAY_MSG_MAX_LEN
173		173
		174	/**
		175	* DOC: GUC2PF_ADVERSE_EVENT
		176	*
		177	* This message is used by the GuC to notify PF about adverse events.
		178	*
		179	* This G2H message must be sent as `CTB HXG Message`_.
		180	*
		181	* +---+-------+--------------------------------------------------------------+
		182	* \| \| Bits \| Description \|
		183	* +===+=======+==============================================================+
		184	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_GUC_ \|
		185	* \| +-------+--------------------------------------------------------------+
		186	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_EVENT_ \|
		187	* \| +-------+--------------------------------------------------------------+
		188	* \| \| 27:16 \| DATA0 = MBZ \|
		189	* \| +-------+--------------------------------------------------------------+
		190	* \| \| 15:0 \| ACTION = _`GUC_ACTION_GUC2PF_ADVERSE_EVENT` = 0x5104 \|
		191	* +---+-------+--------------------------------------------------------------+
		192	* \| 1 \| 31:0 \| DATA1 = VFID - VF identifier \|
		193	* +---+-------+--------------------------------------------------------------+
		194	* \| 2 \| 31:0 \| DATA2 = THRESHOLD - key of the exceeded threshold \|
		195	* +---+-------+--------------------------------------------------------------+
		196	*/
		197	#define GUC_ACTION_GUC2PF_ADVERSE_EVENT 0x5104
		198
		199	#define GUC2PF_ADVERSE_EVENT_EVENT_MSG_LEN (GUC_HXG_EVENT_MSG_MIN_LEN + 2u)
		200	#define GUC2PF_ADVERSE_EVENT_EVENT_MSG_0_MBZ GUC_HXG_EVENT_MSG_0_DATA0
		201	#define GUC2PF_ADVERSE_EVENT_EVENT_MSG_1_VFID GUC_HXG_EVENT_MSG_n_DATAn
		202	#define GUC2PF_ADVERSE_EVENT_EVENT_MSG_2_THRESHOLD GUC_HXG_EVENT_MSG_n_DATAn
		203
174	/**	204	/**
175	* DOC: GUC2PF_VF_STATE_NOTIFY	205	* DOC: GUC2PF_VF_STATE_NOTIFY
176	*	206	*
@@ -213,6 +243,73 @@ · 36 unmodified lines
213	#define GUC_PF_NOTIFY_VF_PAUSE_DONE 3u	243	#define GUC_PF_NOTIFY_VF_PAUSE_DONE 3u
214	#define GUC_PF_NOTIFY_VF_FIXUP_DONE 4u	244	#define GUC_PF_NOTIFY_VF_FIXUP_DONE 4u
215		245
		246	/**
		247	* DOC: VF2GUC_MATCH_VERSION
		248	*
		249	* This action is used to match VF interface version used by VF and GuC.
		250	*
		251	* This message must be sent as `MMIO HXG Message`_.
		252	*
		253	* +---+-------+--------------------------------------------------------------+
		254	* \| \| Bits \| Description \|
		255	* +===+=======+==============================================================+
		256	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
		257	* \| +-------+--------------------------------------------------------------+
		258	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
		259	* \| +-------+--------------------------------------------------------------+
		260	* \| \| 27:16 \| DATA0 = MBZ \|
		261	* \| +-------+--------------------------------------------------------------+
		262	* \| \| 15:0 \| ACTION = _`GUC_ACTION_VF2GUC_MATCH_VERSION` = 0x5500 \|
		263	* +---+-------+--------------------------------------------------------------+
		264	* \| 1 \| 31:24 \| BRANCH - branch ID of the VF interface \|
		265	* \| \| \| (use BRANCH_ANY to request latest version supported by GuC) \|
		266	* \| +-------+--------------------------------------------------------------+
		267	* \| \| 23:16 \| MAJOR - major version of the VF interface \|
		268	* \| \| \| (use MAJOR_ANY to request latest version supported by GuC) \|
		269	* \| +-------+--------------------------------------------------------------+
		270	* \| \| 15:8 \| MINOR - minor version of the VF interface \|
		271	* \| \| \| (use MINOR_ANY to request latest version supported by GuC) \|
		272	* \| +-------+--------------------------------------------------------------+
		273	* \| \| 7:0 \| MBZ \|
		274	* +---+-------+--------------------------------------------------------------+
		275	*
		276	* +---+-------+--------------------------------------------------------------+
		277	* \| \| Bits \| Description \|
		278	* +===+=======+==============================================================+
		279	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_GUC_ \|
		280	* \| +-------+--------------------------------------------------------------+
		281	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
		282	* \| +-------+--------------------------------------------------------------+
		283	* \| \| 27:0 \| DATA0 = MBZ \|
		284	* +---+-------+--------------------------------------------------------------+
		285	* \| 1 \| 31:24 \| BRANCH - branch ID of the VF interface \|
		286	* \| +-------+--------------------------------------------------------------+
		287	* \| \| 23:16 \| MAJOR - major version of the VF interface \|
		288	* \| +-------+--------------------------------------------------------------+
		289	* \| \| 15:8 \| MINOR - minor version of the VF interface \|
		290	* \| +-------+--------------------------------------------------------------+
		291	* \| \| 7:0 \| PATCH - patch version of the VF interface \|
		292	* +---+-------+--------------------------------------------------------------+
		293	*/
		294	#define GUC_ACTION_VF2GUC_MATCH_VERSION 0x5500u
		295
		296	#define VF2GUC_MATCH_VERSION_REQUEST_MSG_LEN (GUC_HXG_REQUEST_MSG_MIN_LEN + 1u)
		297	#define VF2GUC_MATCH_VERSION_REQUEST_MSG_0_MBZ GUC_HXG_REQUEST_MSG_0_DATA0
		298	#define VF2GUC_MATCH_VERSION_REQUEST_MSG_1_BRANCH (0xffu << 24)
		299	#define GUC_VERSION_BRANCH_ANY 0
		300	#define VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MAJOR (0xffu << 16)
		301	#define GUC_VERSION_MAJOR_ANY 0
		302	#define VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MINOR (0xffu << 8)
		303	#define GUC_VERSION_MINOR_ANY 0
		304	#define VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MBZ (0xffu << 0)
		305
		306	#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_LEN (GUC_HXG_RESPONSE_MSG_MIN_LEN + 1u)
		307	#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_0_MBZ GUC_HXG_RESPONSE_MSG_0_DATA0
		308	#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_BRANCH (0xffu << 24)
		309	#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_MAJOR (0xffu << 16)
		310	#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_MINOR (0xffu << 8)
		311	#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_PATCH (0xffu << 0)
		312
216	/**	313	/**
217	* DOC: PF2GUC_UPDATE_VGT_POLICY	314	* DOC: PF2GUC_UPDATE_VGT_POLICY
218	*	315	*
@@ -367,4 +464,97 @@ · 148 unmodified lines
367	#define GUC_PF_TRIGGER_VF_FLR_START 4u	464	#define GUC_PF_TRIGGER_VF_FLR_START 4u
368	#define GUC_PF_TRIGGER_VF_FLR_FINISH 5u	465	#define GUC_PF_TRIGGER_VF_FLR_FINISH 5u
369		466
		467	/**
		468	* DOC: VF2GUC_VF_RESET
		469	*
		470	* This action is used by VF to reset GuC's VF state.
		471	*
		472	* This message must be sent as `MMIO HXG Message`_.
		473	*
		474	* +---+-------+--------------------------------------------------------------+
		475	* \| \| Bits \| Description \|
		476	* +===+=======+==============================================================+
		477	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
		478	* \| +-------+--------------------------------------------------------------+
		479	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
		480	* \| +-------+--------------------------------------------------------------+
		481	* \| \| 27:16 \| DATA0 = MBZ \|
		482	* \| +-------+--------------------------------------------------------------+
		483	* \| \| 15:0 \| ACTION = _`GUC_ACTION_VF2GUC_VF_RESET` = 0x5507 \|
		484	* +---+-------+--------------------------------------------------------------+
		485	*
		486	* +---+-------+--------------------------------------------------------------+
		487	* \| \| Bits \| Description \|
		488	* +===+=======+==============================================================+
		489	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_GUC_ \|
		490	* \| +-------+--------------------------------------------------------------+
		491	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
		492	* \| +-------+--------------------------------------------------------------+
		493	* \| \| 27:0 \| DATA0 = MBZ \|
		494	* +---+-------+--------------------------------------------------------------+
		495	*/
		496	#define GUC_ACTION_VF2GUC_VF_RESET 0x5507u
		497
		498	#define VF2GUC_VF_RESET_REQUEST_MSG_LEN GUC_HXG_REQUEST_MSG_MIN_LEN
		499	#define VF2GUC_VF_RESET_REQUEST_MSG_0_MBZ GUC_HXG_REQUEST_MSG_0_DATA0
		500
		501	#define VF2GUC_VF_RESET_RESPONSE_MSG_LEN GUC_HXG_RESPONSE_MSG_MIN_LEN
		502	#define VF2GUC_VF_RESET_RESPONSE_MSG_0_MBZ GUC_HXG_RESPONSE_MSG_0_DATA0
		503
		504	/**
		505	* DOC: VF2GUC_QUERY_SINGLE_KLV
		506	*
		507	* This action is used by VF to query value of the single KLV data.
		508	*
		509	* This message must be sent as `MMIO HXG Message`_.
		510	*
		511	* +---+-------+--------------------------------------------------------------+
		512	* \| \| Bits \| Description \|
		513	* +===+=======+==============================================================+
		514	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
		515	* \| +-------+--------------------------------------------------------------+
		516	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
		517	* \| +-------+--------------------------------------------------------------+
		518	* \| \| 27:16 \| MBZ \|
		519	* \| +-------+--------------------------------------------------------------+
		520	* \| \| 15:0 \| ACTION = _`GUC_ACTION_VF2GUC_QUERY_SINGLE_KLV` = 0x5509 \|
		521	* +---+-------+--------------------------------------------------------------+
		522	* \| 1 \| 31:16 \| MBZ \|
		523	* \| +-------+--------------------------------------------------------------+
		524	* \| \| 15:0 \| KEY - key for which value is requested \|
		525	* +---+-------+--------------------------------------------------------------+
		526	*
		527	* +---+-------+--------------------------------------------------------------+
		528	* \| \| Bits \| Description \|
		529	* +===+=======+==============================================================+
		530	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_GUC_ \|
		531	* \| +-------+--------------------------------------------------------------+
		532	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
		533	* \| +-------+--------------------------------------------------------------+
		534	* \| \| 27:16 \| MBZ \|
		535	* \| +-------+--------------------------------------------------------------+
		536	* \| \| 15:0 \| LENGTH - length of data in dwords \|
		537	* +---+-------+--------------------------------------------------------------+
		538	* \| 1 \| 31:0 \| VALUE32 - bits 31:0 of value if LENGTH >= 1 \|
		539	* +---+-------+--------------------------------------------------------------+
		540	* \| 2 \| 31:0 \| VALUE64 - bits 63:32 of value if LENGTH >= 2 \|
		541	* +---+-------+--------------------------------------------------------------+
		542	* \| 3 \| 31:0 \| VALUE96 - bits 95:64 of value if LENGTH >= 3 \|
		543	* +---+-------+--------------------------------------------------------------+
		544	*/
		545	#define GUC_ACTION_VF2GUC_QUERY_SINGLE_KLV 0x5509u
		546
		547	#define VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_LEN (GUC_HXG_REQUEST_MSG_MIN_LEN + 1u)
		548	#define VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_0_MBZ GUC_HXG_REQUEST_MSG_0_DATA0
		549	#define VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_1_MBZ (0xffffu << 16)
		550	#define VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_1_KEY (0xffffu << 0)
		551
		552	#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_MIN_LEN GUC_HXG_RESPONSE_MSG_MIN_LEN
		553	#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_MAX_LEN (GUC_HXG_RESPONSE_MSG_MIN_LEN + 3u)
		554	#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_0_MBZ (0xfffu << 16)
		555	#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_0_LENGTH (0xffffu << 0)
		556	#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_1_VALUE32 GUC_HXG_REQUEST_MSG_n_DATAn
		557	#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_2_VALUE64 GUC_HXG_REQUEST_MSG_n_DATAn
		558	#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_3_VALUE96 GUC_HXG_REQUEST_MSG_n_DATAn
		559
370	#endif	560	#endif

drivers/gpu/drm/xe/abi/guc_errors_abi.h+24−2

@@ -7,8 +7,12 @@ · 6 unmodified lines
7	#define _ABI_GUC_ERRORS_ABI_H	7	#define _ABI_GUC_ERRORS_ABI_H
8		8
9	enum xe_guc_response_status {	9	enum xe_guc_response_status {
10	XE_GUC_RESPONSE_STATUS_SUCCESS = 0x0,	10	XE_GUC_RESPONSE_STATUS_SUCCESS = 0x0,
11	XE_GUC_RESPONSE_STATUS_GENERIC_FAIL = 0xF000,	11	XE_GUC_RESPONSE_NOT_SUPPORTED = 0x20,
		12	XE_GUC_RESPONSE_NO_ATTRIBUTE_TABLE = 0x201,
		13	XE_GUC_RESPONSE_NO_DECRYPTION_KEY = 0x202,
		14	XE_GUC_RESPONSE_DECRYPTION_FAILED = 0x204,
		15	XE_GUC_RESPONSE_STATUS_GENERIC_FAIL = 0xF000,
12	};	16	};
13		17
14	enum xe_guc_load_status {	18	enum xe_guc_load_status {
@@ -17,6 +21,9 @@ enum xe_guc_load_status { · 2 unmodified lines
17	XE_GUC_LOAD_STATUS_ERROR_DEVID_BUILD_MISMATCH = 0x02,	21	XE_GUC_LOAD_STATUS_ERROR_DEVID_BUILD_MISMATCH = 0x02,
18	XE_GUC_LOAD_STATUS_GUC_PREPROD_BUILD_MISMATCH = 0x03,	22	XE_GUC_LOAD_STATUS_GUC_PREPROD_BUILD_MISMATCH = 0x03,
19	XE_GUC_LOAD_STATUS_ERROR_DEVID_INVALID_GUCTYPE = 0x04,	23	XE_GUC_LOAD_STATUS_ERROR_DEVID_INVALID_GUCTYPE = 0x04,
		24	XE_GUC_LOAD_STATUS_HWCONFIG_START = 0x05,
		25	XE_GUC_LOAD_STATUS_HWCONFIG_DONE = 0x06,
		26	XE_GUC_LOAD_STATUS_HWCONFIG_ERROR = 0x07,
20	XE_GUC_LOAD_STATUS_GDT_DONE = 0x10,	27	XE_GUC_LOAD_STATUS_GDT_DONE = 0x10,
21	XE_GUC_LOAD_STATUS_IDT_DONE = 0x20,	28	XE_GUC_LOAD_STATUS_IDT_DONE = 0x20,
22	XE_GUC_LOAD_STATUS_LAPIC_DONE = 0x30,	29	XE_GUC_LOAD_STATUS_LAPIC_DONE = 0x30,
@@ -34,4 +41,19 @@ enum xe_guc_load_status { · 11 unmodified lines
34	XE_GUC_LOAD_STATUS_READY = 0xF0,	41	XE_GUC_LOAD_STATUS_READY = 0xF0,
35	};	42	};
36		43
		44	enum xe_bootrom_load_status {
		45	XE_BOOTROM_STATUS_NO_KEY_FOUND = 0x13,
		46	XE_BOOTROM_STATUS_AES_PROD_KEY_FOUND = 0x1A,
		47	XE_BOOTROM_STATUS_PROD_KEY_CHECK_FAILURE = 0x2B,
		48	XE_BOOTROM_STATUS_RSA_FAILED = 0x50,
		49	XE_BOOTROM_STATUS_PAVPC_FAILED = 0x73,
		50	XE_BOOTROM_STATUS_WOPCM_FAILED = 0x74,
		51	XE_BOOTROM_STATUS_LOADLOC_FAILED = 0x75,
		52	XE_BOOTROM_STATUS_JUMP_PASSED = 0x76,
		53	XE_BOOTROM_STATUS_JUMP_FAILED = 0x77,
		54	XE_BOOTROM_STATUS_RC6CTXCONFIG_FAILED = 0x79,
		55	XE_BOOTROM_STATUS_MPUMAP_INCORRECT = 0x7A,
		56	XE_BOOTROM_STATUS_EXCEPTION = 0x7E,
		57	};
		58
37	#endif	59	#endif

drivers/gpu/drm/xe/abi/guc_klvs_abi.h+32−8

 #define GUC_KLV_0_LEN				(0xffffu << 0)
 #define GUC_KLV_n_VALUE				(0xffffffffu << 0)
+/**
+ * DOC: GuC Global Config KLVs
+ *
+ * `GuC KLV`_ keys available for use with HOST2GUC_SELF_CFG_.
+ *
+ * _`GUC_KLV_GLOBAL_CFG_GMD_ID` : 0x3000
+ *      Refers to 32 bit architecture version as reported by the HW IP.
+ *      This key is supported on MTL+ platforms only.
+ *      Requires GuC ABI 1.2+.
+ */
+#define GUC_KLV_GLOBAL_CFG_GMD_ID_KEY			0x3000u
+#define GUC_KLV_GLOBAL_CFG_GMD_ID_LEN			1u
 /**
  * DOC: GuC Self Config KLVs
  *
  *      granularity) since the GPUs clock time runs off a different crystal
  *      from the CPUs clock. Changing this KLV on a VF that is currently
  *      running a context wont take effect until a new context is scheduled in.
- *      That said, when the PF is changing this value from 0xFFFFFFFF to
+ *      That said, when the PF is changing this value from 0x0 to
- *      something else, it might never take effect if the VF is running an
+ *      a non-zero value, it might never take effect if the VF is running an
- *      inifinitely long compute or shader kernel. In such a scenario, the
+ *      infinitely long compute or shader kernel. In such a scenario, the
  *      PF would need to trigger a VM PAUSE and then change the KLV to force
  *      it to take effect. Such cases might typically happen on a 1PF+1VF
  *      Virtualization config enabled for heavier workloads like AI/ML.
  *
+ *      The max value for this KLV is 100 seconds, anything exceeding that
+ *      will be clamped to the max.
+ *
  *      :0: infinite exec quantum (default)
+ *      :100000: maximum exec quantum (100000ms == 100s)
  *
  * _`GUC_KLV_VF_CFG_PREEMPT_TIMEOUT` : 0x8A02
  *      This config sets the VF-preemption-timeout in microseconds.
  *      different crystal from the CPUs clock. Changing this KLV on a VF
  *      that is currently running a context wont take effect until a new
  *      context is scheduled in.
- *      That said, when the PF is changing this value from 0xFFFFFFFF to
+ *      That said, when the PF is changing this value from 0x0 to
- *      something else, it might never take effect if the VF is running an
+ *      a non-zero value, it might never take effect if the VF is running an
- *      inifinitely long compute or shader kernel.
+ *      infinitely long compute or shader kernel.
  *      In this case, the PF would need to trigger a VM PAUSE and then change
  *      the KLV to force it to take effect. Such cases might typically happen
  *      on a 1PF+1VF Virtualization config enabled for heavier workloads like
  *      AI/ML.
  *
+ *      The max value for this KLV is 100 seconds, anything exceeding that
+ *      will be clamped to the max.
+ *
  *      :0: no preemption timeout (default)
+ *      :100000000: maximum preemption timeout (100000000us == 100s)
  *
  * _`GUC_KLV_VF_CFG_THRESHOLD_CAT_ERR` : 0x8A03
  *      This config sets threshold for CAT errors caused by the VF.
 #define GUC_KLV_VF_CFG_EXEC_QUANTUM_KEY		0x8a01
 #define GUC_KLV_VF_CFG_EXEC_QUANTUM_LEN		1u
+#define GUC_KLV_VF_CFG_EXEC_QUANTUM_MAX_VALUE	100000u
-#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_KEY	0x8a02
+#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_KEY		0x8a02
-#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_LEN	1u
+#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_LEN		1u
+#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_MAX_VALUE	100000000u
 #define GUC_KLV_VF_CFG_THRESHOLD_CAT_ERR_KEY		0x8a03
 #define GUC_KLV_VF_CFG_THRESHOLD_CAT_ERR_LEN		1u

drivers/gpu/drm/xe/abi/guc_relay_actions_abi.h+169−1

@@ -1,11 +1,179 @@
1	/* SPDX-License-Identifier: MIT */	1	/* SPDX-License-Identifier: MIT */
2	/*	2	/*
3	* Copyright © 2023 Intel Corporation	3	* Copyright © 2023-2024 Intel Corporation
4	*/	4	*/
5		5
6	#ifndef _ABI_GUC_RELAY_ACTIONS_ABI_H_	6	#ifndef _ABI_GUC_RELAY_ACTIONS_ABI_H_
7	#define _ABI_GUC_RELAY_ACTIONS_ABI_H_	7	#define _ABI_GUC_RELAY_ACTIONS_ABI_H_
8		8
		9	#include "abi/guc_relay_communication_abi.h"
		10
		11	/**
		12	* DOC: GuC Relay VF/PF ABI Version
		13	*
		14	* The _`GUC_RELAY_VERSION_BASE` defines minimum VF/PF ABI version that
		15	* drivers must support. Currently this is version 1.0.
		16	*
		17	* The _`GUC_RELAY_VERSION_LATEST` defines latest VF/PF ABI version that
		18	* drivers may use. Currently this is version 1.0.
		19	*
		20	* Some platforms may require different base VF/PF ABI version.
		21	* No supported VF/PF ABI version can be 0.0.
		22	*/
		23
		24	#define GUC_RELAY_VERSION_BASE_MAJOR 1
		25	#define GUC_RELAY_VERSION_BASE_MINOR 0
		26
		27	#define GUC_RELAY_VERSION_LATEST_MAJOR 1
		28	#define GUC_RELAY_VERSION_LATEST_MINOR 0
		29
		30	/**
		31	* DOC: GuC Relay Actions
		32	*
		33	* The following actions are supported from VF/PF ABI version 1.0:
		34	*
		35	* * `VF2PF_HANDSHAKE`_
		36	* * `VF2PF_QUERY_RUNTIME`_
		37	*/
		38
		39	/**
		40	* DOC: VF2PF_HANDSHAKE
		41	*
		42	* This `Relay Message`_ is used by the VF to establish ABI version with the PF.
		43	*
		44	* Prior to exchanging any other messages, both VF driver and PF driver must
		45	* negotiate the VF/PF ABI version that will be used in their communication.
		46	*
		47	* The VF driver shall use @MAJOR and @MINOR fields to pass requested ABI version.
		48	* The VF driver may use special version 0.0 (both @MAJOR and @MINOR set to 0)
		49	* to request latest (or any) ABI version that is supported by the PF driver.
		50	*
		51	* This message definition shall be supported by all future ABI versions.
		52	* This message definition shall not be changed by future ABI versions.
		53	*
		54	* +---+-------+--------------------------------------------------------------+
		55	* \| \| Bits \| Description \|
		56	* +===+=======+==============================================================+
		57	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
		58	* \| +-------+--------------------------------------------------------------+
		59	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
		60	* \| +-------+--------------------------------------------------------------+
		61	* \| \| 27:16 \| DATA0 = MBZ \|
		62	* \| +-------+--------------------------------------------------------------+
		63	* \| \| 15:0 \| ACTION = _`GUC_RELAY_ACTION_VF2PF_HANDSHAKE` = 0x0001 \|
		64	* +---+-------+--------------------------------------------------------------+
		65	* \| 1 \| 31:16 \| MAJOR - requested major version of the VFPF interface \|
		66	* \| \| \| (use MAJOR_ANY to request latest version supported by PF) \|
		67	* \| +-------+--------------------------------------------------------------+
		68	* \| \| 15:0 \| MINOR - requested minor version of the VFPF interface \|
		69	* \| \| \| (use MINOR_ANY to request latest version supported by PF) \|
		70	* +---+-------+--------------------------------------------------------------+
		71	*
		72	* +---+-------+--------------------------------------------------------------+
		73	* \| \| Bits \| Description \|
		74	* +===+=======+==============================================================+
		75	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
		76	* \| +-------+--------------------------------------------------------------+
		77	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
		78	* \| +-------+--------------------------------------------------------------+
		79	* \| \| 27:0 \| DATA0 = MBZ \|
		80	* +---+-------+--------------------------------------------------------------+
		81	* \| 1 \| 31:16 \| MAJOR - agreed major version of the VFPF interface \|
		82	* \| +-------+--------------------------------------------------------------+
		83	* \| \| 15:0 \| MINOR - agreed minor version of the VFPF interface \|
		84	* +---+-------+--------------------------------------------------------------+
		85	*/
		86	#define GUC_RELAY_ACTION_VF2PF_HANDSHAKE 0x0001u
		87
		88	#define VF2PF_HANDSHAKE_REQUEST_MSG_LEN 2u
		89	#define VF2PF_HANDSHAKE_REQUEST_MSG_0_MBZ GUC_HXG_REQUEST_MSG_0_DATA0
		90	#define VF2PF_HANDSHAKE_REQUEST_MSG_1_MAJOR (0xffffu << 16)
		91	#define VF2PF_HANDSHAKE_MAJOR_ANY 0
		92	#define VF2PF_HANDSHAKE_REQUEST_MSG_1_MINOR (0xffffu << 0)
		93	#define VF2PF_HANDSHAKE_MINOR_ANY 0
		94
		95	#define VF2PF_HANDSHAKE_RESPONSE_MSG_LEN 2u
		96	#define VF2PF_HANDSHAKE_RESPONSE_MSG_0_MBZ GUC_HXG_RESPONSE_MSG_0_DATA0
		97	#define VF2PF_HANDSHAKE_RESPONSE_MSG_1_MAJOR (0xffffu << 16)
		98	#define VF2PF_HANDSHAKE_RESPONSE_MSG_1_MINOR (0xffffu << 0)
		99
		100	/**
		101	* DOC: VF2PF_QUERY_RUNTIME
		102	*
		103	* This `Relay Message`_ is used by the VF to query values of runtime registers.
		104	*
		105	* On some platforms, VF drivers may not have access to the some fuse registers
		106	* (referred here as 'runtime registers') and therefore VF drivers need to ask
		107	* the PF driver to obtain their values.
		108	*
		109	* However, the list of such registers, and their values, is fully owned and
		110	* maintained by the PF driver and the VF driver may only initiate the query
		111	* sequence and indicate in the @START field the starting index of the next
		112	* requested register from this predefined list.
		113	*
		114	* In the response, the PF driver will return tuple of 32-bit register offset and
		115	* the 32-bit value of that register (respectively @REG_OFFSET and @REG_VALUE).
		116	*
		117	* The VF driver can use @LIMIT field to limit number of returned register tuples.
		118	* If @LIMIT is unset then PF decides about number of returned register tuples.
		119	*
		120	* This message definition is supported from ABI version 1.0.
		121	*
		122	* +---+-------+--------------------------------------------------------------+
		123	* \| \| Bits \| Description \|
		124	* +===+=======+==============================================================+
		125	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
		126	* \| +-------+--------------------------------------------------------------+
		127	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
		128	* \| +-------+--------------------------------------------------------------+
		129	* \| \| 27:16 \| DATA0 = LIMIT - limit number of returned entries \|
		130	* \| \| \| (use zero to not enforce any limits on the response) \|
		131	* \| +-------+--------------------------------------------------------------+
		132	* \| \| 15:0 \| ACTION = _`GUC_RELAY_ACTION_VF2PF_QUERY_RUNTIME` = 0x0101 \|
		133	* +---+-------+--------------------------------------------------------------+
		134	* \| 1 \| 31:0 \| DATA1 = START - index of the first requested entry \|
		135	* +---+-------+--------------------------------------------------------------+
		136	*
		137	* +---+-------+--------------------------------------------------------------+
		138	* \| \| Bits \| Description \|
		139	* +===+=======+==============================================================+
		140	* \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
		141	* \| +-------+--------------------------------------------------------------+
		142	* \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
		143	* \| +-------+--------------------------------------------------------------+
		144	* \| \| 27:0 \| DATA0 = COUNT - number of entries included in response \|
		145	* +---+-------+--------------------------------------------------------------+
		146	* \| 1 \| 31:0 \| DATA1 = REMAINING - number of remaining entries \|
		147	* +---+-------+--------------------------------------------------------------+
		148	* \| 2 \| 31:0 \| DATA2 = REG_OFFSET - offset of register[START] \|
		149	* +---+-------+--------------------------------------------------------------+
		150	* \| 3 \| 31:0 \| DATA3 = REG_VALUE - value of register[START] \|
		151	* +---+-------+--------------------------------------------------------------+
		152	* \| \| \| \|
		153	* +---+-------+--------------------------------------------------------------+
		154	* \|n-1\| 31:0 \| REG_OFFSET - offset of register[START + x] \|
		155	* +---+-------+--------------------------------------------------------------+
		156	* \| n \| 31:0 \| REG_VALUE - value of register[START + x] \|
		157	* +---+-------+--------------------------------------------------------------+
		158	*/
		159	#define GUC_RELAY_ACTION_VF2PF_QUERY_RUNTIME 0x0101u
		160
		161	#define VF2PF_QUERY_RUNTIME_REQUEST_MSG_LEN 2u
		162	#define VF2PF_QUERY_RUNTIME_REQUEST_MSG_0_LIMIT GUC_HXG_REQUEST_MSG_0_DATA0
		163	#define VF2PF_QUERY_RUNTIME_NO_LIMIT 0u
		164	#define VF2PF_QUERY_RUNTIME_REQUEST_MSG_1_START GUC_HXG_REQUEST_MSG_n_DATAn
		165
		166	#define VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN (GUC_HXG_MSG_MIN_LEN + 1u)
		167	#define VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MAX_LEN \
		168	(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + VF2PF_QUERY_RUNTIME_MAX_COUNT * 2)
		169	#define VF2PF_QUERY_RUNTIME_RESPONSE_MSG_0_COUNT GUC_HXG_RESPONSE_MSG_0_DATA0
		170	#define VF2PF_QUERY_RUNTIME_MIN_COUNT 0
		171	#define VF2PF_QUERY_RUNTIME_MAX_COUNT \
		172	((GUC_RELAY_MSG_MAX_LEN - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) / 2)
		173	#define VF2PF_QUERY_RUNTIME_RESPONSE_MSG_1_REMAINING GUC_HXG_RESPONSE_MSG_n_DATAn
		174	#define VF2PF_QUERY_RUNTIME_RESPONSE_DATAn_REG_OFFSETx GUC_HXG_RESPONSE_MSG_n_DATAn
		175	#define VF2PF_QUERY_RUNTIME_RESPONSE_DATAn_REG_VALUEx GUC_HXG_RESPONSE_MSG_n_DATAn
		176
9	/**	177	/**
10	* DOC: GuC Relay Debug Actions	178	* DOC: GuC Relay Debug Actions
11	*	179	*

drivers/gpu/drm/xe/display/xe_display.c+28−15

 	return drmm_add_action_or_reset(&xe->drm, xe_display_fini_nommio, xe);
 }
-static void xe_display_fini_noirq(struct drm_device *dev, void *dummy)
+static void xe_display_fini_noirq(void *arg)
 {
-	struct xe_device *xe = to_xe_device(dev);
+	struct xe_device *xe = arg;
 	if (!xe->info.enable_display)
 		return;
 	intel_display_driver_remove_noirq(xe);
-	intel_power_domains_driver_remove(xe);
 }
 int xe_display_init_noirq(struct xe_device *xe)
 	if (err)
 		return err;
-	return drmm_add_action_or_reset(&xe->drm, xe_display_fini_noirq, NULL);
+	return devm_add_action_or_reset(xe->drm.dev, xe_display_fini_noirq, xe);
 }
-static void xe_display_fini_noaccel(struct drm_device *dev, void *dummy)
+static void xe_display_fini_noaccel(void *arg)
 {
-	struct xe_device *xe = to_xe_device(dev);
+	struct xe_device *xe = arg;
 	if (!xe->info.enable_display)
 		return;
 	if (err)
 		return err;
-	return drmm_add_action_or_reset(&xe->drm, xe_display_fini_noaccel, NULL);
+	return devm_add_action_or_reset(xe->drm.dev, xe_display_fini_noaccel, xe);
 }
 int xe_display_init(struct xe_device *xe)
 		return;
 	intel_display_driver_remove(xe);
-	intel_display_device_remove(xe);
 }
 /* IRQ-related functions */
 	return false;
 }
-void xe_display_pm_suspend(struct xe_device *xe)
+void xe_display_pm_suspend(struct xe_device *xe, bool runtime)
 {
 	bool s2idle = suspend_to_idle();
 	if (!xe->info.enable_display)
 	if (has_display(xe))
 		drm_kms_helper_poll_disable(&xe->drm);
-	intel_display_driver_suspend(xe);
+	if (!runtime)
+		intel_display_driver_suspend(xe);
 	intel_dp_mst_suspend(xe);
 	intel_power_domains_resume(xe);
 }
-void xe_display_pm_resume(struct xe_device *xe)
+void xe_display_pm_resume(struct xe_device *xe, bool runtime)
 {
 	if (!xe->info.enable_display)
 		return;
 	/* MST sideband requires HPD interrupts enabled */
 	intel_dp_mst_resume(xe);
-	intel_display_driver_resume(xe);
+	if (!runtime)
+		intel_display_driver_resume(xe);
 	intel_hpd_poll_disable(xe);
 	if (has_display(xe))
 	intel_power_domains_enable(xe);
 }
-void xe_display_probe(struct xe_device *xe)
+static void display_device_remove(struct drm_device *dev, void *arg)
+{
+	struct xe_device *xe = arg;
+	intel_display_device_remove(xe);
+}
+int xe_display_probe(struct xe_device *xe)
 {
+	int err;
 	if (!xe->info.enable_display)
 		goto no_display;
 	intel_display_device_probe(xe);
+	err = drmm_add_action_or_reset(&xe->drm, display_device_remove, xe);
+	if (err)
+		return err;
 	if (has_display(xe))
-		return;
+		return 0;
 no_display:
 	xe->info.enable_display = false;
 	unset_display_features(xe);
+	return 0;
 }

drivers/gpu/drm/xe/display/xe_display.h+6−6

drivers/gpu/drm/xe/display/xe_hdcp_gsc.c+1−0

@@ -13,6 +13,7 @@ · 12 unmodified lines
13	#include "xe_bo.h"	13	#include "xe_bo.h"
14	#include "xe_device.h"	14	#include "xe_device.h"
15	#include "xe_device_types.h"	15	#include "xe_device_types.h"
		16	#include "xe_force_wake.h"
16	#include "xe_gsc_proxy.h"	17	#include "xe_gsc_proxy.h"
17	#include "xe_gsc_submit.h"	18	#include "xe_gsc_submit.h"
18	#include "xe_gt.h"	19	#include "xe_gt.h"

drivers/gpu/drm/xe/instructions/xe_gpu_commands.hrenamed from drivers/gpu/drm/xe/regs/xe_gpu_commands.h+0−0

File renamed without changes.

drivers/gpu/drm/xe/regs/xe_bars.h+11−0

@@ -0,0 +1,11 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2024 Intel Corporation
	4	*/
	5	#ifndef _XE_BARS_H_
	6	#define _XE_BARS_H_
	7
	8	#define GTTMMADR_BAR 0 /* MMIO + GTT */
	9	#define LMEM_BAR 2 /* VRAM */
	10
	11	#endif

drivers/gpu/drm/xe/regs/xe_engine_regs.h+8−2

 #define GSCCS_RING_BASE				0x11a000
 #define RING_TAIL(base)				XE_REG((base) + 0x30)
+#define   TAIL_ADDR				REG_GENMASK(20, 3)
 #define RING_HEAD(base)				XE_REG((base) + 0x34)
-#define   HEAD_ADDR				0x001FFFFC
+#define   HEAD_ADDR				REG_GENMASK(20, 2)
 #define RING_START(base)			XE_REG((base) + 0x38)
 #define   RING_CTL_SIZE(size)			((size) - PAGE_SIZE) /* in bytes -> pages */
 #define   RING_CTL_SIZE(size)			((size) - PAGE_SIZE) /* in bytes -> pages */
+#define RING_START_UDW(base)			XE_REG((base) + 0x48)
 #define RING_PSMI_CTL(base)			XE_REG((base) + 0x50, XE_REG_OPTION_MASKED)
 #define   RC_SEMA_IDLE_MSG_DISABLE		REG_BIT(12)
 #define   WAIT_FOR_EVENT_POWER_DOWN_DISABLE	REG_BIT(7)
 #define RING_ACTHD_UDW(base)			XE_REG((base) + 0x5c)
 #define RING_DMA_FADD_UDW(base)			XE_REG((base) + 0x60)
 #define RING_IPEHR(base)			XE_REG((base) + 0x68)
+#define RING_INSTDONE(base)			XE_REG((base) + 0x6c)
 #define RING_ACTHD(base)			XE_REG((base) + 0x74)
 #define RING_DMA_FADD(base)			XE_REG((base) + 0x78)
 #define RING_HWS_PGA(base)			XE_REG((base) + 0x80)
 #define   FF_DOP_CLOCK_GATE_DISABLE		REG_BIT(1)
 #define   REPLAY_MODE_GRANULARITY		REG_BIT(0)
+#define INDIRECT_RING_STATE(base)		XE_REG((base) + 0x108)
 #define RING_BBADDR(base)			XE_REG((base) + 0x140)
 #define RING_BBADDR_UDW(base)			XE_REG((base) + 0x168)
 #define RING_EXECLIST_STATUS_HI(base)		XE_REG((base) + 0x234 + 4)
 #define RING_CONTEXT_CONTROL(base)		XE_REG((base) + 0x244, XE_REG_OPTION_MASKED)
+#define	  CTX_CTRL_INDIRECT_RING_STATE_ENABLE	REG_BIT(4)
 #define	  CTX_CTRL_INHIBIT_SYN_CTX_SWITCH	REG_BIT(3)
 #define	  CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT	REG_BIT(0)
 #define   RING_VALID_MASK			0x00000001
 #define   RING_VALID				0x00000001
 #define   STOP_RING				REG_BIT(8)
-#define   TAIL_ADDR				0x001FFFF8
 #define RING_CTX_TIMESTAMP(base)		XE_REG((base) + 0x3a8)
 #define CSBE_DEBUG_STATUS(base)			XE_REG((base) + 0x3fc)

drivers/gpu/drm/xe/regs/xe_gt_regs.h+61−6

 #define XELP_GLOBAL_MOCS(i)			XE_REG(0x4000 + (i) * 4)
 #define XEHP_GLOBAL_MOCS(i)			XE_REG_MCR(0x4000 + (i) * 4)
+#define   LE_SSE_MASK				REG_GENMASK(18, 17)
+#define   LE_SSE(value)				REG_FIELD_PREP(LE_SSE_MASK, value)
+#define   LE_COS_MASK				REG_GENMASK(16, 15)
+#define   LE_COS(value)				REG_FIELD_PREP(LE_COS_MASK)
+#define   LE_SCF_MASK				REG_BIT(14)
+#define   LE_SCF(value)				REG_FIELD_PREP(LE_SCF_MASK, value)
+#define   LE_PFM_MASK				REG_GENMASK(13, 11)
+#define   LE_PFM(value)				REG_FIELD_PREP(LE_PFM_MASK, value)
+#define   LE_SCC_MASK				REG_GENMASK(10, 8)
+#define   LE_SCC(value)				REG_FIELD_PREP(LE_SCC_MASK, value)
+#define   LE_RSC_MASK				REG_BIT(7)
+#define   LE_RSC(value)				REG_FIELD_PREP(LE_RSC_MASK, value)
+#define   LE_AOM_MASK				REG_BIT(6)
+#define   LE_AOM(value)				REG_FIELD_PREP(LE_AOM_MASK, value)
+#define   LE_LRUM_MASK				REG_GENMASK(5, 4)
+#define   LE_LRUM(value)			REG_FIELD_PREP(LE_LRUM_MASK, value)
+#define   LE_TGT_CACHE_MASK			REG_GENMASK(3, 2)
+#define   LE_TGT_CACHE(value)			REG_FIELD_PREP(LE_TGT_CACHE_MASK, value)
+#define   LE_CACHEABILITY_MASK			REG_GENMASK(1, 0)
+#define   LE_CACHEABILITY(value)		REG_FIELD_PREP(LE_CACHEABILITY_MASK, value)
 #define CCS_AUX_INV				XE_REG(0x4208)
 #define VD0_AUX_INV				XE_REG(0x4218)
 #define   FF_MODE2_TDS_TIMER_MASK		REG_GENMASK(23, 16)
 #define   FF_MODE2_TDS_TIMER_128		REG_FIELD_PREP(FF_MODE2_TDS_TIMER_MASK, 4)
+#define XEHPG_INSTDONE_GEOM_SVGUNIT		XE_REG_MCR(0x666c)
 #define CACHE_MODE_1				XE_REG(0x7004, XE_REG_OPTION_MASKED)
 #define   MSAA_OPTIMIZATION_REDUC_DISABLE	REG_BIT(11)
 #define   FLSH_IGNORES_PSD			REG_BIT(10)
 #define   FD_END_COLLECT			REG_BIT(5)
+#define SC_INSTDONE				XE_REG(0x7100)
+#define SC_INSTDONE_EXTRA			XE_REG(0x7104)
+#define SC_INSTDONE_EXTRA2			XE_REG(0x7108)
+#define XEHPG_SC_INSTDONE			XE_REG_MCR(0x7100)
+#define XEHPG_SC_INSTDONE_EXTRA			XE_REG_MCR(0x7104)
+#define XEHPG_SC_INSTDONE_EXTRA2		XE_REG_MCR(0x7108)
 #define COMMON_SLICE_CHICKEN4			XE_REG(0x7300, XE_REG_OPTION_MASKED)
 #define   DISABLE_TDC_LOAD_BALANCING_CALC	REG_BIT(6)
 #define   MAX_MSLICES				4
 #define   MEML3_EN_MASK				REG_GENMASK(3, 0)
+#define MIRROR_FUSE1				XE_REG(0x911c)
 #define XELP_EU_ENABLE				XE_REG(0x9134)	/* "_DISABLE" on Xe_LP */
 #define   XELP_EU_MASK				REG_GENMASK(7, 0)
+#define XELP_GT_SLICE_ENABLE			XE_REG(0x9138)
 #define XELP_GT_GEOMETRY_DSS_ENABLE		XE_REG(0x913c)
 #define GT_VEBOX_VDBOX_DISABLE			XE_REG(0x9140)
 #define   RC_CTL_RC6_ENABLE			REG_BIT(18)
 #define RC_STATE				XE_REG(0xa094)
 #define RC_IDLE_HYSTERSIS			XE_REG(0xa0ac)
+#define MEDIA_POWERGATE_IDLE_HYSTERESIS		XE_REG(0xa0c4)
+#define RENDER_POWERGATE_IDLE_HYSTERESIS	XE_REG(0xa0c8)
 #define PMINTRMSK				XE_REG(0xa168)
 #define   PMINTR_DISABLE_REDIRECT_TO_GUC	REG_BIT(31)
 #define FORCEWAKE_GT				XE_REG(0xa188)
-#define PG_ENABLE				XE_REG(0xa210)
+#define POWERGATE_ENABLE			XE_REG(0xa210)
-#define   VD2_MFXVDENC_POWERGATE_ENABLE		REG_BIT(8)
+#define   RENDER_POWERGATE_ENABLE		REG_BIT(0)
-#define   VD2_HCP_POWERGATE_ENABLE		REG_BIT(7)
+#define   MEDIA_POWERGATE_ENABLE		REG_BIT(1)
-#define   VD0_MFXVDENC_POWERGATE_ENABLE		REG_BIT(4)
+#define   VDN_HCP_POWERGATE_ENABLE(n)		REG_BIT(3 + 2 * (n))
-#define   VD0_HCP_POWERGATE_ENABLE		REG_BIT(3)
+#define   VDN_MFXVDENC_POWERGATE_ENABLE(n)	REG_BIT(4 + 2 * (n))
 #define CTC_MODE				XE_REG(0xa26c)
 #define   CTC_SHIFT_PARAMETER_MASK		REG_GENMASK(2, 1)
 #define   XEHPC_OVRLSCCC			REG_BIT(0)
 /* L3 Cache Control */
+#define LNCFCMOCS_REG_COUNT			32
 #define XELP_LNCFCMOCS(i)			XE_REG(0xb020 + (i) * 4)
 #define XEHP_LNCFCMOCS(i)			XE_REG_MCR(0xb020 + (i) * 4)
-#define LNCFCMOCS_REG_COUNT			32
+#define   L3_UPPER_LKUP_MASK			REG_BIT(23)
+#define   L3_UPPER_GLBGO_MASK			REG_BIT(22)
+#define   L3_UPPER_IDX_CACHEABILITY_MASK	REG_GENMASK(21, 20)
+#define   L3_UPPER_IDX_SCC_MASK			REG_GENMASK(19, 17)
+#define   L3_UPPER_IDX_ESC_MASK			REG_BIT(16)
+#define   L3_LKUP_MASK				REG_BIT(7)
+#define   L3_LKUP(value)			REG_FIELD_PREP(L3_LKUP_MASK, value)
+#define   L3_GLBGO_MASK				REG_BIT(6)
+#define   L3_GLBGO(value)			REG_FIELD_PREP(L3_GLBGO_MASK, value)
+#define   L3_CACHEABILITY_MASK			REG_GENMASK(5, 4)
+#define   L3_CACHEABILITY(value)		REG_FIELD_PREP(L3_CACHEABILITY_MASK, value)
+#define   L3_SCC_MASK				REG_GENMASK(3, 1)
+#define   L3_SCC(value)				REG_FIELD_PREP(L3_SCC_MASK, value)
+#define   L3_ESC_MASK				REG_BIT(0)
+#define   L3_ESC(value)				REG_FIELD_PREP(L3_ESC_MASK, value)
 #define XEHP_L3NODEARBCFG			XE_REG_MCR(0xb0b4)
 #define   XEHP_LNESPARE				REG_BIT(19)
 #define HALF_SLICE_CHICKEN5			XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
 #define   DISABLE_SAMPLE_G_PERFORMANCE		REG_BIT(0)
+#define SAMPLER_INSTDONE			XE_REG_MCR(0xe160)
+#define ROW_INSTDONE				XE_REG_MCR(0xe164)
 #define SAMPLER_MODE				XE_REG_MCR(0xe18c, XE_REG_OPTION_MASKED)
 #define   ENABLE_SMALLPL			REG_BIT(15)
 #define   SC_DISABLE_POWER_OPTIMIZATION_EBB	REG_BIT(9)
 #define HALF_SLICE_CHICKEN7				XE_REG_MCR(0xe194, XE_REG_OPTION_MASKED)
 #define   DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA	REG_BIT(15)
+#define   CLEAR_OPTIMIZATION_DISABLE			REG_BIT(6)
 #define CACHE_MODE_SS				XE_REG_MCR(0xe420, XE_REG_OPTION_MASKED)
 #define   DISABLE_ECC				REG_BIT(5)

drivers/gpu/drm/xe/regs/xe_guc_regs.h+2−0

@@ -40,6 +40,8 @@ · 39 unmodified lines
40	#define GS_BOOTROM_JUMP_PASSED REG_FIELD_PREP(GS_BOOTROM_MASK, 0x76)	40	#define GS_BOOTROM_JUMP_PASSED REG_FIELD_PREP(GS_BOOTROM_MASK, 0x76)
41	#define GS_MIA_IN_RESET REG_BIT(0)	41	#define GS_MIA_IN_RESET REG_BIT(0)
42		42
		43	#define GUC_HEADER_INFO XE_REG(0xc014)
		44
43	#define GUC_WOPCM_SIZE XE_REG(0xc050)	45	#define GUC_WOPCM_SIZE XE_REG(0xc050)
44	#define GUC_WOPCM_SIZE_MASK REG_GENMASK(31, 12)	46	#define GUC_WOPCM_SIZE_MASK REG_GENMASK(31, 12)
45	#define GUC_WOPCM_SIZE_LOCKED REG_BIT(0)	47	#define GUC_WOPCM_SIZE_LOCKED REG_BIT(0)

drivers/gpu/drm/xe/regs/xe_lrc_layout.h+8−0

 #define CTX_RING_TAIL			(0x06 + 1)
 #define CTX_RING_START			(0x08 + 1)
 #define CTX_RING_CTL			(0x0a + 1)
+#define CTX_TIMESTAMP			(0x22 + 1)
+#define CTX_INDIRECT_RING_STATE		(0x26 + 1)
 #define CTX_PDP0_UDW			(0x30 + 1)
 #define CTX_PDP0_LDW			(0x32 + 1)
 #define CTX_INT_SRC_REPORT_REG		(CTX_LRI_INT_REPORT_PTR + 3)
 #define CTX_INT_SRC_REPORT_PTR		(CTX_LRI_INT_REPORT_PTR + 4)
+#define INDIRECT_CTX_RING_HEAD		(0x02 + 1)
+#define INDIRECT_CTX_RING_TAIL		(0x04 + 1)
+#define INDIRECT_CTX_RING_START		(0x06 + 1)
+#define INDIRECT_CTX_RING_START_UDW	(0x08 + 1)
+#define INDIRECT_CTX_RING_CTL		(0x0a + 1)
 #endif

drivers/gpu/drm/xe/regs/xe_pcode_regs.h+7−0

@@ -18,4 +18,11 @@ · 17 unmodified lines
18	#define PVC_GT0_PLATFORM_ENERGY_STATUS XE_REG(0x28106c)	18	#define PVC_GT0_PLATFORM_ENERGY_STATUS XE_REG(0x28106c)
19	#define PVC_GT0_PACKAGE_POWER_SKU XE_REG(0x281080)	19	#define PVC_GT0_PACKAGE_POWER_SKU XE_REG(0x281080)
20		20
		21	#define BMG_PACKAGE_POWER_SKU XE_REG(0x138098)
		22	#define BMG_PACKAGE_POWER_SKU_UNIT XE_REG(0x1380dc)
		23	#define BMG_PACKAGE_ENERGY_STATUS XE_REG(0x138120)
		24	#define BMG_PACKAGE_RAPL_LIMIT XE_REG(0x138440)
		25	#define BMG_PLATFORM_ENERGY_STATUS XE_REG(0x138458)
		26	#define BMG_PLATFORM_POWER_LIMIT XE_REG(0x138460)
		27
21	#endif /* _XE_PCODE_REGS_H_ */	28	#endif /* _XE_PCODE_REGS_H_ */

drivers/gpu/drm/xe/regs/xe_regs.h+3−0

@@ -30,6 +30,9 @@ · 29 unmodified lines
30	#define XEHP_CLOCK_GATE_DIS XE_REG(0x101014)	30	#define XEHP_CLOCK_GATE_DIS XE_REG(0x101014)
31	#define SGSI_SIDECLK_DIS REG_BIT(17)	31	#define SGSI_SIDECLK_DIS REG_BIT(17)
32		32
		33	#define XEHP_MTCFG_ADDR XE_REG(0x101800)
		34	#define TILE_COUNT REG_GENMASK(15, 8)
		35
33	#define GGC XE_REG(0x108040)	36	#define GGC XE_REG(0x108040)
34	#define GMS_MASK REG_GENMASK(15, 8)	37	#define GMS_MASK REG_GENMASK(15, 8)
35	#define GGMS_MASK REG_GENMASK(7, 6)	38	#define GGMS_MASK REG_GENMASK(7, 6)

drivers/gpu/drm/xe/regs/xe_sriov_regs.h+3−0

@@ -14,6 +14,9 @@ · 13 unmodified lines
14	#define LMEM_EN REG_BIT(31)	14	#define LMEM_EN REG_BIT(31)
15	#define LMTT_DIR_PTR REG_GENMASK(30, 0) /* in multiples of 64KB */	15	#define LMTT_DIR_PTR REG_GENMASK(30, 0) /* in multiples of 64KB */
16		16
		17	#define VIRTUAL_CTRL_REG XE_REG(0x10108c)
		18	#define GUEST_GTT_UPDATE_EN REG_BIT(8)
		19
17	#define VF_CAP_REG XE_REG(0x1901f8, XE_REG_OPTION_VF)	20	#define VF_CAP_REG XE_REG(0x1901f8, XE_REG_OPTION_VF)
18	#define VF_CAP REG_BIT(0)	21	#define VF_CAP REG_BIT(0)
19		22

drivers/gpu/drm/xe/tests/Makefile+1−0

@@ -11,6 +11,7 @@ xe_live_test-y = xe_live_test_mod.o \ · 10 unmodified lines
11	# Normal kunit tests	11	# Normal kunit tests
12	obj-$(CONFIG_DRM_XE_KUNIT_TEST) += xe_test.o	12	obj-$(CONFIG_DRM_XE_KUNIT_TEST) += xe_test.o
13	xe_test-y = xe_test_mod.o \	13	xe_test-y = xe_test_mod.o \
		14	xe_args_test.o \
14	xe_pci_test.o \	15	xe_pci_test.o \
15	xe_rtp_test.o \	16	xe_rtp_test.o \
16	xe_wa_test.o	17	xe_wa_test.o

drivers/gpu/drm/xe/tests/xe_args_test.c+221−0

@@ -0,0 +1,221 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2	/*
	3	* Copyright © 2024 Intel Corporation
	4	*/
	5
	6	#include <kunit/test.h>
	7
	8	#include "xe_args.h"
	9
	10	static void call_args_example(struct kunit *test)
	11	{
	12	#define foo X, Y, Z, Q
	13	#define bar COUNT_ARGS(foo)
	14	#define buz CALL_ARGS(COUNT_ARGS, foo)
	15
	16	KUNIT_EXPECT_EQ(test, bar, 1);
	17	KUNIT_EXPECT_EQ(test, buz, 4);
	18
	19	#undef foo
	20	#undef bar
	21	#undef buz
	22	}
	23
	24	static void drop_first_arg_example(struct kunit *test)
	25	{
	26	#define foo X, Y, Z, Q
	27	#define bar CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(foo))
	28
	29	KUNIT_EXPECT_EQ(test, bar, 3);
	30
	31	#undef foo
	32	#undef bar
	33	}
	34
	35	static void first_arg_example(struct kunit *test)
	36	{
	37	int X = 1;
	38
	39	#define foo X, Y, Z, Q
	40	#define bar FIRST_ARG(foo)
	41
	42	KUNIT_EXPECT_EQ(test, bar, X);
	43	KUNIT_EXPECT_STREQ(test, __stringify(bar), "X");
	44
	45	#undef foo
	46	#undef bar
	47	}
	48
	49	static void last_arg_example(struct kunit *test)
	50	{
	51	int Q = 1;
	52
	53	#define foo X, Y, Z, Q
	54	#define bar LAST_ARG(foo)
	55
	56	KUNIT_EXPECT_EQ(test, bar, Q);
	57	KUNIT_EXPECT_STREQ(test, __stringify(bar), "Q");
	58
	59	#undef foo
	60	#undef bar
	61	}
	62
	63	static void pick_arg_example(struct kunit *test)
	64	{
	65	int Y = 1, Z = 2;
	66
	67	#define foo X, Y, Z, Q
	68	#define bar PICK_ARG(2, foo)
	69	#define buz PICK_ARG3(foo)
	70
	71	KUNIT_EXPECT_EQ(test, bar, Y);
	72	KUNIT_EXPECT_STREQ(test, __stringify(bar), "Y");
	73	KUNIT_EXPECT_EQ(test, buz, Z);
	74	KUNIT_EXPECT_STREQ(test, __stringify(buz), "Z");
	75
	76	#undef foo
	77	#undef bar
	78	#undef buz
	79	}
	80
	81	static void sep_comma_example(struct kunit *test)
	82	{
	83	#define foo(f) f(X) f(Y) f(Z) f(Q)
	84	#define bar DROP_FIRST_ARG(foo(ARGS_SEP_COMMA __stringify))
	85	#define buz CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(foo(ARGS_SEP_COMMA)))
	86
	87	static const char * const a[] = { bar };
	88
	89	KUNIT_EXPECT_STREQ(test, a[0], "X");
	90	KUNIT_EXPECT_STREQ(test, a[1], "Y");
	91	KUNIT_EXPECT_STREQ(test, a[2], "Z");
	92	KUNIT_EXPECT_STREQ(test, a[3], "Q");
	93
	94	KUNIT_EXPECT_EQ(test, buz, 4);
	95
	96	#undef foo
	97	#undef bar
	98	#undef buz
	99	}
	100
	101	#define NO_ARGS
	102	#define FOO_ARGS X, Y, Z, Q
	103	#define MAX_ARGS -1, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12
	104
	105	static void count_args_test(struct kunit *test)
	106	{
	107	int count;
	108
	109	/* COUNT_ARGS() counts to 12 */
	110
	111	count = COUNT_ARGS();
	112	KUNIT_EXPECT_EQ(test, count, 0);
	113
	114	count = COUNT_ARGS(1);
	115	KUNIT_EXPECT_EQ(test, count, 1);
	116
	117	count = COUNT_ARGS(a, b, c, d, e);
	118	KUNIT_EXPECT_EQ(test, count, 5);
	119
	120	count = COUNT_ARGS(a, b, c, d, e, f, g, h, i, j, k, l);
	121	KUNIT_EXPECT_EQ(test, count, 12);
	122
	123	/* COUNT_ARGS() does not expand params */
	124
	125	count = COUNT_ARGS(NO_ARGS);
	126	KUNIT_EXPECT_EQ(test, count, 1);
	127
	128	count = COUNT_ARGS(FOO_ARGS);
	129	KUNIT_EXPECT_EQ(test, count, 1);
	130	}
	131
	132	static void call_args_test(struct kunit *test)
	133	{
	134	int count;
	135
	136	count = CALL_ARGS(COUNT_ARGS, NO_ARGS);
	137	KUNIT_EXPECT_EQ(test, count, 0);
	138	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, NO_ARGS), 0);
	139	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, FOO_ARGS), 4);
	140	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, FOO_ARGS, FOO_ARGS), 8);
	141	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, MAX_ARGS), 12);
	142	}
	143
	144	static void drop_first_arg_test(struct kunit *test)
	145	{
	146	int Y = -2, Z = -3, Q = -4;
	147	int a[] = { DROP_FIRST_ARG(FOO_ARGS) };
	148
	149	KUNIT_EXPECT_EQ(test, DROP_FIRST_ARG(0, -1), -1);
	150	KUNIT_EXPECT_EQ(test, DROP_FIRST_ARG(DROP_FIRST_ARG(0, -1, -2)), -2);
	151
	152	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(FOO_ARGS)), 3);
	153	KUNIT_EXPECT_EQ(test, DROP_FIRST_ARG(DROP_FIRST_ARG(DROP_FIRST_ARG(FOO_ARGS))), -4);
	154	KUNIT_EXPECT_EQ(test, a[0], -2);
	155	KUNIT_EXPECT_EQ(test, a[1], -3);
	156	KUNIT_EXPECT_EQ(test, a[2], -4);
	157
	158	#define foo DROP_FIRST_ARG(FOO_ARGS)
	159	#define bar DROP_FIRST_ARG(DROP_FIRST_ARG(FOO_ARGS))
	160	#define buz DROP_FIRST_ARG(DROP_FIRST_ARG(DROP_FIRST_ARG(FOO_ARGS)))
	161
	162	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, foo), 3);
	163	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, bar), 2);
	164	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, buz), 1);
	165	KUNIT_EXPECT_STREQ(test, __stringify(buz), "Q");
	166
	167	#undef foo
	168	#undef bar
	169	#undef buz
	170	}
	171
	172	static void first_arg_test(struct kunit *test)
	173	{
	174	int X = -1;
	175	int a[] = { FIRST_ARG(FOO_ARGS) };
	176
	177	KUNIT_EXPECT_EQ(test, FIRST_ARG(-1, -2), -1);
	178
	179	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, FIRST_ARG(FOO_ARGS)), 1);
	180	KUNIT_EXPECT_EQ(test, FIRST_ARG(FOO_ARGS), -1);
	181	KUNIT_EXPECT_EQ(test, a[0], -1);
	182	KUNIT_EXPECT_STREQ(test, __stringify(FIRST_ARG(FOO_ARGS)), "X");
	183	}
	184
	185	static void last_arg_test(struct kunit *test)
	186	{
	187	int Q = -4;
	188	int a[] = { LAST_ARG(FOO_ARGS) };
	189
	190	KUNIT_EXPECT_EQ(test, LAST_ARG(-1, -2), -2);
	191
	192	KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, LAST_ARG(FOO_ARGS)), 1);
	193	KUNIT_EXPECT_EQ(test, LAST_ARG(FOO_ARGS), -4);
	194	KUNIT_EXPECT_EQ(test, a[0], -4);
	195	KUNIT_EXPECT_STREQ(test, __stringify(LAST_ARG(FOO_ARGS)), "Q");
	196
	197	KUNIT_EXPECT_EQ(test, LAST_ARG(MAX_ARGS), -12);
	198	KUNIT_EXPECT_STREQ(test, __stringify(LAST_ARG(MAX_ARGS)), "-12");
	199	}
	200
	201	static struct kunit_case args_tests[] = {
	202	KUNIT_CASE(count_args_test),
	203	KUNIT_CASE(call_args_example),
	204	KUNIT_CASE(call_args_test),
	205	KUNIT_CASE(drop_first_arg_example),
	206	KUNIT_CASE(drop_first_arg_test),
	207	KUNIT_CASE(first_arg_example),
	208	KUNIT_CASE(first_arg_test),
	209	KUNIT_CASE(last_arg_example),
	210	KUNIT_CASE(last_arg_test),
	211	KUNIT_CASE(pick_arg_example),
	212	KUNIT_CASE(sep_comma_example),
	213	{}
	214	};
	215
	216	static struct kunit_suite args_test_suite = {
	217	.name = "args",
	218	.test_cases = args_tests,
	219	};
	220
	221	kunit_test_suite(args_test_suite);

drivers/gpu/drm/xe/tests/xe_gt_sriov_pf_service_test.c+232−0

@@ -0,0 +1,232 @@
	1	// SPDX-License-Identifier: GPL-2.0 AND MIT
	2	/*
	3	* Copyright © 2024 Intel Corporation
	4	*/
	5
	6	#include <kunit/test.h>
	7
	8	#include "xe_device.h"
	9	#include "xe_kunit_helpers.h"
	10	#include "xe_pci_test.h"
	11
	12	static int pf_service_test_init(struct kunit *test)
	13	{
	14	struct xe_pci_fake_data fake = {
	15	.sriov_mode = XE_SRIOV_MODE_PF,
	16	.platform = XE_TIGERLAKE, /* some random platform */
	17	.subplatform = XE_SUBPLATFORM_NONE,
	18	};
	19	struct xe_device *xe;
	20	struct xe_gt *gt;
	21
	22	test->priv = &fake;
	23	xe_kunit_helper_xe_device_test_init(test);
	24
	25	xe = test->priv;
	26	KUNIT_ASSERT_EQ(test, xe_sriov_init(xe), 0);
	27
	28	gt = xe_device_get_gt(xe, 0);
	29	pf_init_versions(gt);
	30
	31	/*
	32	* sanity check:
	33	* - all supported platforms VF/PF ABI versions must be defined
	34	* - base version can't be newer than latest
	35	*/
	36	KUNIT_ASSERT_NE(test, 0, gt->sriov.pf.service.version.base.major);
	37	KUNIT_ASSERT_NE(test, 0, gt->sriov.pf.service.version.latest.major);
	38	KUNIT_ASSERT_LE(test, gt->sriov.pf.service.version.base.major,
	39	gt->sriov.pf.service.version.latest.major);
	40	if (gt->sriov.pf.service.version.base.major == gt->sriov.pf.service.version.latest.major)
	41	KUNIT_ASSERT_LE(test, gt->sriov.pf.service.version.base.minor,
	42	gt->sriov.pf.service.version.latest.minor);
	43
	44	test->priv = gt;
	45	return 0;
	46	}
	47
	48	static void pf_negotiate_any(struct kunit *test)
	49	{
	50	struct xe_gt *gt = test->priv;
	51	u32 major, minor;
	52
	53	KUNIT_ASSERT_EQ(test, 0,
	54	pf_negotiate_version(gt, VF2PF_HANDSHAKE_MAJOR_ANY,
	55	VF2PF_HANDSHAKE_MINOR_ANY,
	56	&major, &minor));
	57	KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	58	KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor);
	59	}
	60
	61	static void pf_negotiate_base_match(struct kunit *test)
	62	{
	63	struct xe_gt *gt = test->priv;
	64	u32 major, minor;
	65
	66	KUNIT_ASSERT_EQ(test, 0,
	67	pf_negotiate_version(gt,
	68	gt->sriov.pf.service.version.base.major,
	69	gt->sriov.pf.service.version.base.minor,
	70	&major, &minor));
	71	KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.base.major);
	72	KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.base.minor);
	73	}
	74
	75	static void pf_negotiate_base_newer(struct kunit *test)
	76	{
	77	struct xe_gt *gt = test->priv;
	78	u32 major, minor;
	79
	80	KUNIT_ASSERT_EQ(test, 0,
	81	pf_negotiate_version(gt,
	82	gt->sriov.pf.service.version.base.major,
	83	gt->sriov.pf.service.version.base.minor + 1,
	84	&major, &minor));
	85	KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.base.major);
	86	KUNIT_ASSERT_GE(test, minor, gt->sriov.pf.service.version.base.minor);
	87	if (gt->sriov.pf.service.version.base.major == gt->sriov.pf.service.version.latest.major)
	88	KUNIT_ASSERT_LE(test, minor, gt->sriov.pf.service.version.latest.minor);
	89	else
	90	KUNIT_FAIL(test, "FIXME: don't know how to test multi-version yet!\n");
	91	}
	92
	93	static void pf_negotiate_base_next(struct kunit *test)
	94	{
	95	struct xe_gt *gt = test->priv;
	96	u32 major, minor;
	97
	98	KUNIT_ASSERT_EQ(test, 0,
	99	pf_negotiate_version(gt,
	100	gt->sriov.pf.service.version.base.major + 1, 0,
	101	&major, &minor));
	102	KUNIT_ASSERT_GE(test, major, gt->sriov.pf.service.version.base.major);
	103	KUNIT_ASSERT_LE(test, major, gt->sriov.pf.service.version.latest.major);
	104	if (major == gt->sriov.pf.service.version.latest.major)
	105	KUNIT_ASSERT_LE(test, minor, gt->sriov.pf.service.version.latest.minor);
	106	else
	107	KUNIT_FAIL(test, "FIXME: don't know how to test multi-version yet!\n");
	108	}
	109
	110	static void pf_negotiate_base_older(struct kunit *test)
	111	{
	112	struct xe_gt *gt = test->priv;
	113	u32 major, minor;
	114
	115	if (!gt->sriov.pf.service.version.base.minor)
	116	kunit_skip(test, "no older minor\n");
	117
	118	KUNIT_ASSERT_NE(test, 0,
	119	pf_negotiate_version(gt,
	120	gt->sriov.pf.service.version.base.major,
	121	gt->sriov.pf.service.version.base.minor - 1,
	122	&major, &minor));
	123	}
	124
	125	static void pf_negotiate_base_prev(struct kunit *test)
	126	{
	127	struct xe_gt *gt = test->priv;
	128	u32 major, minor;
	129
	130	KUNIT_ASSERT_NE(test, 0,
	131	pf_negotiate_version(gt,
	132	gt->sriov.pf.service.version.base.major - 1, 1,
	133	&major, &minor));
	134	}
	135
	136	static void pf_negotiate_latest_match(struct kunit *test)
	137	{
	138	struct xe_gt *gt = test->priv;
	139	u32 major, minor;
	140
	141	KUNIT_ASSERT_EQ(test, 0,
	142	pf_negotiate_version(gt,
	143	gt->sriov.pf.service.version.latest.major,
	144	gt->sriov.pf.service.version.latest.minor,
	145	&major, &minor));
	146	KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	147	KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor);
	148	}
	149
	150	static void pf_negotiate_latest_newer(struct kunit *test)
	151	{
	152	struct xe_gt *gt = test->priv;
	153	u32 major, minor;
	154
	155	KUNIT_ASSERT_EQ(test, 0,
	156	pf_negotiate_version(gt,
	157	gt->sriov.pf.service.version.latest.major,
	158	gt->sriov.pf.service.version.latest.minor + 1,
	159	&major, &minor));
	160	KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	161	KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor);
	162	}
	163
	164	static void pf_negotiate_latest_next(struct kunit *test)
	165	{
	166	struct xe_gt *gt = test->priv;
	167	u32 major, minor;
	168
	169	KUNIT_ASSERT_EQ(test, 0,
	170	pf_negotiate_version(gt,
	171	gt->sriov.pf.service.version.latest.major + 1, 0,
	172	&major, &minor));
	173	KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	174	KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor);
	175	}
	176
	177	static void pf_negotiate_latest_older(struct kunit *test)
	178	{
	179	struct xe_gt *gt = test->priv;
	180	u32 major, minor;
	181
	182	if (!gt->sriov.pf.service.version.latest.minor)
	183	kunit_skip(test, "no older minor\n");
	184
	185	KUNIT_ASSERT_EQ(test, 0,
	186	pf_negotiate_version(gt,
	187	gt->sriov.pf.service.version.latest.major,
	188	gt->sriov.pf.service.version.latest.minor - 1,
	189	&major, &minor));
	190	KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	191	KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor - 1);
	192	}
	193
	194	static void pf_negotiate_latest_prev(struct kunit *test)
	195	{
	196	struct xe_gt *gt = test->priv;
	197	u32 major, minor;
	198
	199	if (gt->sriov.pf.service.version.base.major == gt->sriov.pf.service.version.latest.major)
	200	kunit_skip(test, "no prev major");
	201
	202	KUNIT_ASSERT_EQ(test, 0,
	203	pf_negotiate_version(gt,
	204	gt->sriov.pf.service.version.latest.major - 1,
	205	gt->sriov.pf.service.version.base.minor + 1,
	206	&major, &minor));
	207	KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major - 1);
	208	KUNIT_ASSERT_GE(test, major, gt->sriov.pf.service.version.base.major);
	209	}
	210
	211	static struct kunit_case pf_service_test_cases[] = {
	212	KUNIT_CASE(pf_negotiate_any),
	213	KUNIT_CASE(pf_negotiate_base_match),
	214	KUNIT_CASE(pf_negotiate_base_newer),
	215	KUNIT_CASE(pf_negotiate_base_next),
	216	KUNIT_CASE(pf_negotiate_base_older),
	217	KUNIT_CASE(pf_negotiate_base_prev),
	218	KUNIT_CASE(pf_negotiate_latest_match),
	219	KUNIT_CASE(pf_negotiate_latest_newer),
	220	KUNIT_CASE(pf_negotiate_latest_next),
	221	KUNIT_CASE(pf_negotiate_latest_older),
	222	KUNIT_CASE(pf_negotiate_latest_prev),
	223	{}
	224	};
	225
	226	static struct kunit_suite pf_service_suite = {
	227	.name = "pf_service",
	228	.test_cases = pf_service_test_cases,
	229	.init = pf_service_test_init,
	230	};
	231
	232	kunit_test_suite(pf_service_suite);

drivers/gpu/drm/xe/tests/xe_migrate.c+1−87

@@ -62,36 +62,6 @@ static int run_sanity_job(struct xe_migrate m, struct xe_device xe, · 61 unmodified lines
62	return 0;	62	return 0;
63	}	63	}
64		64
65	static void
66	sanity_populate_cb(struct xe_migrate_pt_update *pt_update,
67	struct xe_tile tile, struct iosys_map map, void *dst,
68	u32 qword_ofs, u32 num_qwords,
69	const struct xe_vm_pgtable_update *update)
70	{
71	struct migrate_test_params *p =
72	to_migrate_test_params(xe_cur_kunit_priv(XE_TEST_LIVE_MIGRATE));
73	int i;
74	u64 *ptr = dst;
75	u64 value;
76
77	for (i = 0; i < num_qwords; i++) {
78	value = (qword_ofs + i - update->ofs) * 0x1111111111111111ULL;
79	if (map)
80	xe_map_wr(tile_to_xe(tile), map, (qword_ofs + i) *
81	sizeof(u64), u64, value);
82	else
83	ptr[i] = value;
84	}
85
86	kunit_info(xe_cur_kunit(), "Used %s.\n", map ? "CPU" : "GPU");
87	if (p->force_gpu && map)
88	KUNIT_FAIL(xe_cur_kunit(), "GPU pagetable update used CPU.\n");
89	}
90
91	static const struct xe_migrate_pt_update_ops sanity_ops = {
92	.populate = sanity_populate_cb,
93	};
94
95	#define check(_retval, _expected, str, _test) \	65	#define check(_retval, _expected, str, _test) \
96	do { if ((_retval) != (_expected)) { \	66	do { if ((_retval) != (_expected)) { \
97	KUNIT_FAIL(_test, "Sanity check failed: " str \	67	KUNIT_FAIL(_test, "Sanity check failed: " str \
@@ -209,57 +179,6 @@ static void test_copy_vram(struct xe_migrate m, struct xe_bo bo, · 111 unmodified lines
209	test_copy(m, bo, test, region);	179	test_copy(m, bo, test, region);
210	}	180	}
211		181
212	static void test_pt_update(struct xe_migrate m, struct xe_bo pt,
213	struct kunit *test, bool force_gpu)
214	{
215	struct xe_device *xe = tile_to_xe(m->tile);
216	struct dma_fence *fence;
217	u64 retval, expected;
218	ktime_t then, now;
219	int i;
220
221	struct xe_vm_pgtable_update update = {
222	.ofs = 1,
223	.qwords = 0x10,
224	.pt_bo = pt,
225	};
226	struct xe_migrate_pt_update pt_update = {
227	.ops = &sanity_ops,
228	};
229	struct migrate_test_params p = {
230	.base.id = XE_TEST_LIVE_MIGRATE,
231	.force_gpu = force_gpu,
232	};
233
234	test->priv = &p;
235	/* Test xe_migrate_update_pgtables() updates the pagetable as expected */
236	expected = 0xf0f0f0f0f0f0f0f0ULL;
237	xe_map_memset(xe, &pt->vmap, 0, (u8)expected, pt->size);
238
239	then = ktime_get();
240	fence = xe_migrate_update_pgtables(m, m->q->vm, NULL, m->q, &update, 1,
241	NULL, 0, &pt_update);
242	now = ktime_get();
243	if (sanity_fence_failed(xe, fence, "Migration pagetable update", test))
244	return;
245
246	kunit_info(test, "Updating without syncing took %llu us,\n",
247	(unsigned long long)ktime_to_us(ktime_sub(now, then)));
248
249	dma_fence_put(fence);
250	retval = xe_map_rd(xe, &pt->vmap, 0, u64);
251	check(retval, expected, "PTE[0] must stay untouched", test);
252
253	for (i = 0; i < update.qwords; i++) {
254	retval = xe_map_rd(xe, &pt->vmap, (update.ofs + i) * 8, u64);
255	check(retval, i * 0x1111111111111111ULL, "PTE update", test);
256	}
257
258	retval = xe_map_rd(xe, &pt->vmap, 8 * (update.ofs + update.qwords),
259	u64);
260	check(retval, expected, "PTE[0x11] must stay untouched", test);
261	}
262
263	static void xe_migrate_sanity_test(struct xe_migrate m, struct kunit test)	182	static void xe_migrate_sanity_test(struct xe_migrate m, struct kunit test)
264	{	183	{
265	struct xe_tile *tile = m->tile;	184	struct xe_tile *tile = m->tile;
@@ -398,11 +317,6 @@ static void xe_migrate_sanity_test(struct xe_migrate m, struct kunit test) · 132 unmodified lines
398	test_copy_vram(m, big, test);	317	test_copy_vram(m, big, test);
399	}	318	}
400		319
401	kunit_info(test, "Testing page table update using CPU if GPU idle.\n");
402	test_pt_update(m, pt, test, false);
403	kunit_info(test, "Testing page table update using GPU\n");
404	test_pt_update(m, pt, test, true);
405
406	out:	320	out:
407	xe_bb_free(bb, NULL);	321	xe_bb_free(bb, NULL);
408	free_tiny:	322	free_tiny:
@@ -430,7 +344,7 @@ static int migrate_test_run_device(struct xe_device *xe) · 21 unmodified lines
430	struct xe_migrate *m = tile->migrate;	344	struct xe_migrate *m = tile->migrate;
431		345
432	kunit_info(test, "Testing tile id %d.\n", id);	346	kunit_info(test, "Testing tile id %d.\n", id);
433	xe_vm_lock(m->q->vm, true);	347	xe_vm_lock(m->q->vm, false);
434	xe_migrate_sanity_test(m, test);	348	xe_migrate_sanity_test(m, test);
435	xe_vm_unlock(m->q->vm);	349	xe_vm_unlock(m->q->vm);
436	}	350	}

drivers/gpu/drm/xe/xe_args.h+143−0

@@ -0,0 +1,143 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_ARGS_H_
	7	#define _XE_ARGS_H_
	8
	9	#include <linux/args.h>
	10
	11	/*
	12	* Why don't the following macros have the XE prefix?
	13	*
	14	* Once we find more potential users outside of the Xe driver, we plan to move
	15	* all of the following macros unchanged to linux/args.h.
	16	*/
	17
	18	/**
	19	* CALL_ARGS - Invoke a macro, but allow parameters to be expanded beforehand.
	20	* @f: name of the macro to invoke
	21	* @args: arguments for the macro
	22	*
	23	* This macro allows calling macros which names might generated or we want to
	24	* make sure it's arguments will be correctly expanded.
	25	*
	26	* Example:
	27	*
	28	* #define foo X,Y,Z,Q
	29	* #define bar COUNT_ARGS(foo)
	30	* #define buz CALL_ARGS(COUNT_ARGS, foo)
	31	*
	32	* With above definitions bar expands to 1 while buz expands to 4.
	33	*/
	34	#define CALL_ARGS(f, args...) __CALL_ARGS(f, args)
	35	#define __CALL_ARGS(f, args...) f(args)
	36
	37	/**
	38	* DROP_FIRST_ARG - Returns all arguments except the first one.
	39	* @args: arguments
	40	*
	41	* This helper macro allows manipulation the argument list before passing it
	42	* to the next level macro.
	43	*
	44	* Example:
	45	*
	46	* #define foo X,Y,Z,Q
	47	* #define bar CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(foo))
	48	*
	49	* With above definitions bar expands to 3.
	50	*/
	51	#define DROP_FIRST_ARG(args...) __DROP_FIRST_ARG(args)
	52	#define __DROP_FIRST_ARG(a, b...) b
	53
	54	/**
	55	* FIRST_ARG - Returns the first argument.
	56	* @args: arguments
	57	*
	58	* This helper macro allows manipulation the argument list before passing it
	59	* to the next level macro.
	60	*
	61	* Example:
	62	*
	63	* #define foo X,Y,Z,Q
	64	* #define bar FIRST_ARG(foo)
	65	*
	66	* With above definitions bar expands to X.
	67	*/
	68	#define FIRST_ARG(args...) __FIRST_ARG(args)
	69	#define __FIRST_ARG(a, b...) a
	70
	71	/**
	72	* LAST_ARG - Returns the last argument.
	73	* @args: arguments
	74	*
	75	* This helper macro allows manipulation the argument list before passing it
	76	* to the next level macro.
	77	*
	78	* Like COUNT_ARGS() this macro works up to 12 arguments.
	79	*
	80	* Example:
	81	*
	82	* #define foo X,Y,Z,Q
	83	* #define bar LAST_ARG(foo)
	84	*
	85	* With above definitions bar expands to Q.
	86	*/
	87	#define LAST_ARG(args...) __LAST_ARG(args)
	88	#define __LAST_ARG(args...) PICK_ARG(COUNT_ARGS(args), args)
	89
	90	/**
	91	* PICK_ARG - Returns the n-th argument.
	92	* @n: argument number to be returned
	93	* @args: arguments
	94	*
	95	* This helper macro allows manipulation the argument list before passing it
	96	* to the next level macro.
	97	*
	98	* Like COUNT_ARGS() this macro supports n up to 12.
	99	* Specialized macros PICK_ARG1() to PICK_ARG12() are also available.
	100	*
	101	* Example:
	102	*
	103	* #define foo X,Y,Z,Q
	104	* #define bar PICK_ARG(2, foo)
	105	* #define buz PICK_ARG3(foo)
	106	*
	107	* With above definitions bar expands to Y and buz expands to Z.
	108	*/
	109	#define PICK_ARG(n, args...) __PICK_ARG(n, args)
	110	#define __PICK_ARG(n, args...) CALL_ARGS(CONCATENATE(PICK_ARG, n), args)
	111	#define PICK_ARG1(args...) FIRST_ARG(args)
	112	#define PICK_ARG2(args...) PICK_ARG1(DROP_FIRST_ARG(args))
	113	#define PICK_ARG3(args...) PICK_ARG2(DROP_FIRST_ARG(args))
	114	#define PICK_ARG4(args...) PICK_ARG3(DROP_FIRST_ARG(args))
	115	#define PICK_ARG5(args...) PICK_ARG4(DROP_FIRST_ARG(args))
	116	#define PICK_ARG6(args...) PICK_ARG5(DROP_FIRST_ARG(args))
	117	#define PICK_ARG7(args...) PICK_ARG6(DROP_FIRST_ARG(args))
	118	#define PICK_ARG8(args...) PICK_ARG7(DROP_FIRST_ARG(args))
	119	#define PICK_ARG9(args...) PICK_ARG8(DROP_FIRST_ARG(args))
	120	#define PICK_ARG10(args...) PICK_ARG9(DROP_FIRST_ARG(args))
	121	#define PICK_ARG11(args...) PICK_ARG10(DROP_FIRST_ARG(args))
	122	#define PICK_ARG12(args...) PICK_ARG11(DROP_FIRST_ARG(args))
	123
	124	/**
	125	* ARGS_SEP_COMMA - Definition of a comma character.
	126	*
	127	* This definition can be used in cases where any intermediate macro expects
	128	* fixed number of arguments, but we want to pass more arguments which can
	129	* be properly evaluated only by the next level macro.
	130	*
	131	* Example:
	132	*
	133	* #define foo(f) f(X) f(Y) f(Z) f(Q)
	134	* #define bar DROP_FIRST_ARG(foo(ARGS_SEP_COMMA __stringify))
	135	* #define buz CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(foo(ARGS_SEP_COMMA)))
	136	*
	137	* With above definitions bar expands to
	138	* "X", "Y", "Z", "Q"
	139	* and buz expands to 4.
	140	*/
	141	#define ARGS_SEP_COMMA ,
	142
	143	#endif

drivers/gpu/drm/xe/xe_assert.h+2−2

@@ -109,11 +109,11 @@ · 108 unmodified lines
109	#define xe_assert_msg(xe, condition, msg, arg...) ({ \	109	#define xe_assert_msg(xe, condition, msg, arg...) ({ \
110	const struct xe_device *__xe = (xe); \	110	const struct xe_device *__xe = (xe); \
111	__xe_assert_msg(__xe, condition, \	111	__xe_assert_msg(__xe, condition, \
112	"platform: %d subplatform: %d\n" \	112	"platform: %s subplatform: %d\n" \
113	"graphics: %s %u.%02u step %s\n" \	113	"graphics: %s %u.%02u step %s\n" \
114	"media: %s %u.%02u step %s\n" \	114	"media: %s %u.%02u step %s\n" \
115	msg, \	115	msg, \
116	__xe->info.platform, __xe->info.subplatform, \	116	__xe->info.platform_name, __xe->info.subplatform, \
117	__xe->info.graphics_name, \	117	__xe->info.graphics_name, \
118	__xe->info.graphics_verx100 / 100, \	118	__xe->info.graphics_verx100 / 100, \
119	__xe->info.graphics_verx100 % 100, \	119	__xe->info.graphics_verx100 % 100, \

drivers/gpu/drm/xe/xe_bb.c+1−1

@@ -6,7 +6,7 @@ · 5 unmodified lines
6	#include "xe_bb.h"	6	#include "xe_bb.h"
7		7
8	#include "instructions/xe_mi_commands.h"	8	#include "instructions/xe_mi_commands.h"
9	#include "regs/xe_gpu_commands.h"	9	#include "xe_assert.h"
10	#include "xe_device.h"	10	#include "xe_device.h"
11	#include "xe_exec_queue_types.h"	11	#include "xe_exec_queue_types.h"
12	#include "xe_gt.h"	12	#include "xe_gt.h"

drivers/gpu/drm/xe/xe_bo.c+48−23

 	return bo->ttm.resource->mem_type == XE_PL_STOLEN;
 }
+/**
+ * xe_bo_has_single_placement - check if BO is placed only in one memory location
+ * @bo: The BO
+ *
+ * This function checks whether a given BO is placed in only one memory location.
+ *
+ * Returns: true if the BO is placed in a single memory location, false otherwise.
+ *
+ */
+bool xe_bo_has_single_placement(struct xe_bo *bo)
+{
+	return bo->placement.num_placement == 1;
+}
 /**
  * xe_bo_is_stolen_devmem - check if BO is of stolen type accessed via PCI BAR
  * @bo: The BO
 	return 0;
 }
+static void xe_tt_unmap_sg(struct ttm_tt *tt)
+{
+	struct xe_ttm_tt *xe_tt = container_of(tt, struct xe_ttm_tt, ttm);
+	if (xe_tt->sg) {
+		dma_unmap_sgtable(xe_tt->dev, xe_tt->sg,
+				  DMA_BIDIRECTIONAL, 0);
+		sg_free_table(xe_tt->sg);
+		xe_tt->sg = NULL;
+	}
+}
 struct sg_table *xe_bo_sg(struct xe_bo *bo)
 {
 	struct ttm_tt *tt = bo->ttm.ttm;
 	if (err)
 		return err;
-	/* A follow up may move this xe_bo_move when BO is moved to XE_PL_TT */
-	err = xe_tt_map_sg(tt);
-	if (err)
-		ttm_pool_free(&ttm_dev->pool, tt);
 	return err;
 }
 static void xe_ttm_tt_unpopulate(struct ttm_device *ttm_dev, struct ttm_tt *tt)
 {
-	struct xe_ttm_tt *xe_tt = container_of(tt, struct xe_ttm_tt, ttm);
 	if (tt->page_flags & TTM_TT_FLAG_EXTERNAL)
 		return;
-	if (xe_tt->sg) {
+	xe_tt_unmap_sg(tt);
-		dma_unmap_sgtable(xe_tt->dev, xe_tt->sg,
-				  DMA_BIDIRECTIONAL, 0);
-		sg_free_table(xe_tt->sg);
-		xe_tt->sg = NULL;
-	}
 	return ttm_pool_free(&ttm_dev->pool, tt);
 }
 	bool handle_system_ccs = (!IS_DGFX(xe) && xe_bo_needs_ccs_pages(bo) &&
 				  ttm && ttm_tt_is_populated(ttm)) ? true : false;
 	int ret = 0;
 	/* Bo creation path, moving to system or TT. */
 	if ((!old_mem && ttm) && !handle_system_ccs) {
-		ttm_bo_move_null(ttm_bo, new_mem);
+		if (new_mem->mem_type == XE_PL_TT)
-		return 0;
+			ret = xe_tt_map_sg(ttm);
+		if (!ret)
+			ttm_bo_move_null(ttm_bo, new_mem);
+		goto out;
 	}
 	if (ttm_bo->type == ttm_bo_type_sg) {
 		ret = xe_bo_move_notify(bo, ctx);
 		if (!ret)
 			ret = xe_bo_move_dmabuf(ttm_bo, new_mem);
-		goto out;
+		return ret;
 	}
 	tt_has_data = ttm && (ttm_tt_is_populated(ttm) ||
 	needs_clear = (ttm && ttm->page_flags & TTM_TT_FLAG_ZERO_ALLOC) ||
 		(!ttm && ttm_bo->type == ttm_bo_type_device);
+	if (new_mem->mem_type == XE_PL_TT) {
+		ret = xe_tt_map_sg(ttm);
+		if (ret)
+			goto out;
+	}
 	if ((move_lacks_source && !needs_clear)) {
 		ttm_bo_move_null(ttm_bo, new_mem);
 		goto out;
 	xe_pm_runtime_put(xe);
 out:
-	return ret;
+	if ((!ttm_bo->resource || ttm_bo->resource->mem_type == XE_PL_SYSTEM) &&
+	    ttm_bo->ttm)
+		xe_tt_unmap_sg(ttm_bo->ttm);
+	return ret;
 }
 /**
 	xe_assert(xe, xe_bo_is_pinned(bo));
 	xe_assert(xe, xe_bo_is_user(bo));
-	if (bo->ttm.pin_count == 1 && !list_empty(&bo->pinned_link)) {
+	spin_lock(&xe->pinned.lock);
-		spin_lock(&xe->pinned.lock);
+	if (bo->ttm.pin_count == 1 && !list_empty(&bo->pinned_link))
 		list_del_init(&bo->pinned_link);
-		spin_unlock(&xe->pinned.lock);
+	spin_unlock(&xe->pinned.lock);
-	}
 	ttm_bo_unpin(&bo->ttm);
 		struct ttm_place *place = &(bo->placements[0]);
 		if (mem_type_is_vram(place->mem_type)) {
-			xe_assert(xe, !list_empty(&bo->pinned_link));
 			spin_lock(&xe->pinned.lock);
+			xe_assert(xe, !list_empty(&bo->pinned_link));
 			list_del_init(&bo->pinned_link);
 			spin_unlock(&xe->pinned.lock);
 		}

drivers/gpu/drm/xe/xe_bo.h+1−0

@@ -206,6 +206,7 @@ bool mem_type_is_vram(u32 mem_type); · 205 unmodified lines
206	bool xe_bo_is_vram(struct xe_bo *bo);	206	bool xe_bo_is_vram(struct xe_bo *bo);
207	bool xe_bo_is_stolen(struct xe_bo *bo);	207	bool xe_bo_is_stolen(struct xe_bo *bo);
208	bool xe_bo_is_stolen_devmem(struct xe_bo *bo);	208	bool xe_bo_is_stolen_devmem(struct xe_bo *bo);
		209	bool xe_bo_has_single_placement(struct xe_bo *bo);
209	uint64_t vram_region_gpu_offset(struct ttm_resource *res);	210	uint64_t vram_region_gpu_offset(struct ttm_resource *res);
210		211
211	bool xe_bo_can_migrate(struct xe_bo *bo, u32 mem_type);	212	bool xe_bo_can_migrate(struct xe_bo *bo, u32 mem_type);

drivers/gpu/drm/xe/xe_debugfs.c+58−0

@@ -12,7 +12,10 @@ · 11 unmodified lines
12		12
13	#include "xe_bo.h"	13	#include "xe_bo.h"
14	#include "xe_device.h"	14	#include "xe_device.h"
		15	#include "xe_force_wake.h"
15	#include "xe_gt_debugfs.h"	16	#include "xe_gt_debugfs.h"
		17	#include "xe_gt_printk.h"
		18	#include "xe_guc_ads.h"
16	#include "xe_pm.h"	19	#include "xe_pm.h"
17	#include "xe_sriov.h"	20	#include "xe_sriov.h"
18	#include "xe_step.h"	21	#include "xe_step.h"
@@ -118,6 +121,58 @@ static const struct file_operations forcewake_all_fops = { · 99 unmodified lines
118	.release = forcewake_release,	121	.release = forcewake_release,
119	};	122	};
120		123
		124	static ssize_t wedged_mode_show(struct file f, char __user ubuf,
		125	size_t size, loff_t *pos)
		126	{
		127	struct xe_device *xe = file_inode(f)->i_private;
		128	char buf[32];
		129	int len = 0;
		130
		131	len = scnprintf(buf, sizeof(buf), "%d\n", xe->wedged.mode);
		132
		133	return simple_read_from_buffer(ubuf, size, pos, buf, len);
		134	}
		135
		136	static ssize_t wedged_mode_set(struct file f, const char __user ubuf,
		137	size_t size, loff_t *pos)
		138	{
		139	struct xe_device *xe = file_inode(f)->i_private;
		140	struct xe_gt *gt;
		141	u32 wedged_mode;
		142	ssize_t ret;
		143	u8 id;
		144
		145	ret = kstrtouint_from_user(ubuf, size, 0, &wedged_mode);
		146	if (ret)
		147	return ret;
		148
		149	if (wedged_mode > 2)
		150	return -EINVAL;
		151
		152	if (xe->wedged.mode == wedged_mode)
		153	return 0;
		154
		155	xe->wedged.mode = wedged_mode;
		156
		157	xe_pm_runtime_get(xe);
		158	for_each_gt(gt, xe, id) {
		159	ret = xe_guc_ads_scheduler_policy_toggle_reset(&gt->uc.guc.ads);
		160	if (ret) {
		161	xe_gt_err(gt, "Failed to update GuC ADS scheduler policy. GuC may still cause engine reset even with wedged_mode=2\n");
		162	return -EIO;
		163	}
		164	}
		165	xe_pm_runtime_put(xe);
		166
		167	return size;
		168	}
		169
		170	static const struct file_operations wedged_mode_fops = {
		171	.owner = THIS_MODULE,
		172	.read = wedged_mode_show,
		173	.write = wedged_mode_set,
		174	};
		175
121	void xe_debugfs_register(struct xe_device *xe)	176	void xe_debugfs_register(struct xe_device *xe)
122	{	177	{
123	struct ttm_device *bdev = &xe->ttm;	178	struct ttm_device *bdev = &xe->ttm;
@@ -135,6 +190,9 @@ void xe_debugfs_register(struct xe_device *xe) · 11 unmodified lines
135	debugfs_create_file("forcewake_all", 0400, root, xe,	190	debugfs_create_file("forcewake_all", 0400, root, xe,
136	&forcewake_all_fops);	191	&forcewake_all_fops);
137		192
		193	debugfs_create_file("wedged_mode", 0400, root, xe,
		194	&wedged_mode_fops);
		195
138	for (mem_type = XE_PL_VRAM0; mem_type <= XE_PL_VRAM1; ++mem_type) {	196	for (mem_type = XE_PL_VRAM0; mem_type <= XE_PL_VRAM1; ++mem_type) {
139	man = ttm_manager_type(bdev, mem_type);	197	man = ttm_manager_type(bdev, mem_type);
140		198

drivers/gpu/drm/xe/xe_devcoredump.c+17−2

 	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
 	ts = ktime_to_timespec64(ss->boot_time);
 	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
+	drm_printf(&p, "Process: %s\n", ss->process_name);
 	xe_device_snapshot_print(xe, &p);
 	drm_printf(&p, "\n**** GuC CT ****\n");
 	enum xe_hw_engine_id id;
 	u32 adj_logical_mask = q->logical_mask;
 	u32 width_mask = (0x1 << q->width) - 1;
+	const char *process_name = "no process";
+	struct task_struct *task = NULL;
 	int i;
 	bool cookie;
 	ss->snapshot_time = ktime_get_real();
 	ss->boot_time = ktime_get_boottime();
+	if (q->vm && q->vm->xef) {
+		task = get_pid_task(q->vm->xef->drm->pid, PIDTYPE_PID);
+		if (task)
+			process_name = task->comm;
+	}
+	strscpy(ss->process_name, process_name);
+	if (task)
+		put_task_struct(task);
 	ss->gt = q->gt;
 	INIT_WORK(&ss->work, xe_devcoredump_deferred_snap_work);
 		      xe_devcoredump_read, xe_devcoredump_free);
 }
-static void xe_driver_devcoredump_fini(struct drm_device *drm, void *arg)
+static void xe_driver_devcoredump_fini(void *arg)
 {
+	struct drm_device *drm = arg;
 	dev_coredump_put(drm->dev);
 }
 int xe_devcoredump_init(struct xe_device *xe)
 {
-	return drmm_add_action_or_reset(&xe->drm, xe_driver_devcoredump_fini, xe);
+	return devm_add_action_or_reset(xe->drm.dev, xe_driver_devcoredump_fini, &xe->drm);
 }
 #endif

drivers/gpu/drm/xe/xe_devcoredump_types.h+2−0

@@ -26,6 +26,8 @@ struct xe_devcoredump_snapshot { · 25 unmodified lines
26	ktime_t snapshot_time;	26	ktime_t snapshot_time;
27	/** @boot_time: Relative boot time so the uptime can be calculated. */	27	/** @boot_time: Relative boot time so the uptime can be calculated. */
28	ktime_t boot_time;	28	ktime_t boot_time;
		29	/** @process_name: Name of process that triggered this gpu hang */
		30	char process_name[TASK_COMM_LEN];
29		31
30	/** @gt: Affected GT, used by forcewake for delayed capture */	32	/** @gt: Affected GT, used by forcewake for delayed capture */
31	struct xe_gt *gt;	33	struct xe_gt *gt;

drivers/gpu/drm/xe/xe_device.c+78−29

 #include "xe_device.h"
+#include <linux/delay.h>
 #include <linux/units.h>
 #include <drm/drm_aperture.h>
 #include <drm/xe_drm.h>
 #include "display/xe_display.h"
+#include "instructions/xe_gpu_commands.h"
 #include "regs/xe_gt_regs.h"
 #include "regs/xe_regs.h"
 #include "xe_bo.h"
 #include "xe_drv.h"
 #include "xe_exec.h"
 #include "xe_exec_queue.h"
+#include "xe_force_wake.h"
 #include "xe_ggtt.h"
 #include "xe_gsc_proxy.h"
 #include "xe_gt.h"
 #include "xe_gt_mcr.h"
+#include "xe_gt_printk.h"
+#include "xe_gt_sriov_vf.h"
+#include "xe_guc.h"
 #include "xe_hwmon.h"
 #include "xe_irq.h"
 #include "xe_memirq.h"
 #include "xe_ttm_stolen_mgr.h"
 #include "xe_ttm_sys_mgr.h"
 #include "xe_vm.h"
+#include "xe_vram.h"
 #include "xe_wait_user_fence.h"
 static int xe_file_open(struct drm_device *dev, struct drm_file *file)
 	struct xe_exec_queue *q;
 	unsigned long idx;
-	mutex_lock(&xef->exec_queue.lock);
+	/*
+	 * No need for exec_queue.lock here as there is no contention for it
+	 * when FD is closing as IOCTLs presumably can't be modifying the
+	 * xarray. Taking exec_queue.lock here causes undue dependency on
+	 * vm->lock taken during xe_exec_queue_kill().
+	 */
 	xa_for_each(&xef->exec_queue.xa, idx, q) {
 		xe_exec_queue_kill(q);
 		xe_exec_queue_put(q);
 	}
-	mutex_unlock(&xef->exec_queue.lock);
 	xa_destroy(&xef->exec_queue.xa);
 	mutex_destroy(&xef->exec_queue.lock);
 	mutex_lock(&xef->vm.lock);
 	struct xe_device *xe = to_xe_device(file_priv->minor->dev);
 	long ret;
+	if (xe_device_wedged(xe))
+		return -ECANCELED;
 	ret = xe_pm_runtime_get_ioctl(xe);
 	if (ret >= 0)
 		ret = drm_ioctl(file, cmd, arg);
 	struct xe_device *xe = to_xe_device(file_priv->minor->dev);
 	long ret;
+	if (xe_device_wedged(xe))
+		return -ECANCELED;
 	ret = xe_pm_runtime_get_ioctl(xe);
 	if (ret >= 0)
 		ret = drm_compat_ioctl(file, cmd, arg);
 #endif
 };
-static void xe_driver_release(struct drm_device *dev)
-{
-	struct xe_device *xe = to_xe_device(dev);
-	pci_set_drvdata(to_pci_dev(xe->drm.dev), NULL);
-}
 static struct drm_driver driver = {
 	/* Don't use MTRRs here; the Xserver or userspace app should
 	 * deal with them for Intel hardware.
 #ifdef CONFIG_PROC_FS
 	.show_fdinfo = xe_drm_client_fdinfo,
 #endif
-	.release = &xe_driver_release,
 	.ioctls = xe_ioctls,
 	.num_ioctls = ARRAY_SIZE(xe_ioctls),
 	.fops = &xe_driver_fops,
 	init_waitqueue_head(&xe->ufence_wq);
-	drmm_mutex_init(&xe->drm, &xe->usm.lock);
+	err = drmm_mutex_init(&xe->drm, &xe->usm.lock);
+	if (err)
+		goto err;
 	xa_init_flags(&xe->usm.asid_to_vm, XA_FLAGS_ALLOC);
 	if (IS_ENABLED(CONFIG_DRM_XE_DEBUG)) {
 	xe_mmio_write32(gt, GU_DEBUG, DRIVERFLR_STATUS);
 }
-static void xe_driver_flr_fini(struct drm_device *drm, void *arg)
+static void xe_driver_flr_fini(void *arg)
 {
 	struct xe_device *xe = arg;
 		xe_driver_flr(xe);
 }
-static void xe_device_sanitize(struct drm_device *drm, void *arg)
+static void xe_device_sanitize(void *arg)
 {
 	struct xe_device *xe = arg;
 	struct xe_gt *gt;
 	if (err)
 		return err;
+	xe->wedged.mode = xe_modparam.wedged_mode;
 	return 0;
 }
 	if (err)
 		return err;
-	xe_mmio_probe_tiles(xe);
+	err = xe_mmio_probe_tiles(xe);
+	if (err)
+		return err;
 	xe_ttm_sys_mgr_init(xe);
-	for_each_gt(gt, xe, id)
+	for_each_gt(gt, xe, id) {
-		xe_force_wake_init_gt(gt, gt_to_fw(gt));
+		err = xe_gt_init_early(gt);
+		if (err)
+			return err;
+	}
 	for_each_tile(tile, xe, id) {
+		if (IS_SRIOV_VF(xe)) {
+			xe_guc_comm_init_early(&tile->primary_gt->uc.guc);
+			err = xe_gt_sriov_vf_bootstrap(tile->primary_gt);
+			if (err)
+				return err;
+			err = xe_gt_sriov_vf_query_config(tile->primary_gt);
+			if (err)
+				return err;
+		}
 		err = xe_ggtt_init_early(tile->mem.ggtt);
 		if (err)
 			return err;
 	err = xe_devcoredump_init(xe);
 	if (err)
 		return err;
-	err = drmm_add_action_or_reset(&xe->drm, xe_driver_flr_fini, xe);
+	err = devm_add_action_or_reset(xe->drm.dev, xe_driver_flr_fini, xe);
 	if (err)
 		return err;
-	for_each_gt(gt, xe, id)
-		xe_pcode_init(gt);
 	err = xe_display_init_noirq(xe);
 	if (err)
 		return err;
 	if (err)
 		goto err;
-	for_each_gt(gt, xe, id) {
-		err = xe_gt_init_early(gt);
-		if (err)
-			goto err_irq_shutdown;
-	}
 	err = xe_device_set_has_flat_ccs(xe);
 	if (err)
 		goto err_irq_shutdown;
-	err = xe_mmio_probe_vram(xe);
+	err = xe_vram_probe(xe);
 	if (err)
 		goto err_irq_shutdown;
 	xe_hwmon_register(xe);
-	return drmm_add_action_or_reset(&xe->drm, xe_device_sanitize, xe);
+	return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
 err_fini_display:
 	xe_display_driver_remove(xe);
 {
 	return address & GENMASK_ULL(xe->info.va_bits - 1, 0);
 }
+/**
+ * xe_device_declare_wedged - Declare device wedged
+ * @xe: xe device instance
+ *
+ * This is a final state that can only be cleared with a mudule
+ * re-probe (unbind + bind).
+ * In this state every IOCTL will be blocked so the GT cannot be used.
+ * In general it will be called upon any critical error such as gt reset
+ * failure or guc loading failure.
+ * If xe.wedged module parameter is set to 2, this function will be called
+ * on every single execution timeout (a.k.a. GPU hang) right after devcoredump
+ * snapshot capture. In this mode, GT reset won't be attempted so the state of
+ * the issue is preserved for further debugging.
+ */
+void xe_device_declare_wedged(struct xe_device *xe)
+{
+	if (xe->wedged.mode == 0) {
+		drm_dbg(&xe->drm, "Wedged mode is forcibly disabled\n");
+		return;
+	}
+	if (!atomic_xchg(&xe->wedged.flag, 1)) {
+		xe->needs_flr_on_fini = true;
+		drm_err(&xe->drm,
+			"CRITICAL: Xe has declared device %s as wedged.\n"
+			"IOCTLs and executions are blocked. Only a rebind may clear the failure\n"
+			"Please file a _new_ bug report at https://gitlab.freedesktop.org/drm/xe/kernel/issues/new\n",
+			dev_name(xe->drm.dev));
+	}
+}

drivers/gpu/drm/xe/xe_device.h+7−6

 #ifndef _XE_DEVICE_H_
 #define _XE_DEVICE_H_
-struct xe_exec_queue;
-struct xe_file;
 #include <drm/drm_util.h>
-#include "regs/xe_gpu_commands.h"
 #include "xe_device_types.h"
-#include "xe_force_wake.h"
-#include "xe_macros.h"
 static inline struct xe_device *to_xe_device(const struct drm_device *dev)
 {
 u64 xe_device_canonicalize_addr(struct xe_device *xe, u64 address);
 u64 xe_device_uncanonicalize_addr(struct xe_device *xe, u64 address);
+static inline bool xe_device_wedged(struct xe_device *xe)
+{
+	return atomic_read(&xe->wedged.flag);
+}
+void xe_device_declare_wedged(struct xe_device *xe);
 #endif

drivers/gpu/drm/xe/xe_device_sysfs.c+2−2

@@ -69,7 +69,7 @@ vram_d3cold_threshold_store(struct device dev, struct device_attribute attr, · 68 unmodified lines
69		69
70	static DEVICE_ATTR_RW(vram_d3cold_threshold);	70	static DEVICE_ATTR_RW(vram_d3cold_threshold);
71		71
72	static void xe_device_sysfs_fini(struct drm_device drm, void arg)	72	static void xe_device_sysfs_fini(void *arg)
73	{	73	{
74	struct xe_device *xe = arg;	74	struct xe_device *xe = arg;
75		75
@@ -85,5 +85,5 @@ int xe_device_sysfs_init(struct xe_device *xe) · 9 unmodified lines
85	if (ret)	85	if (ret)
86	return ret;	86	return ret;
87		87
88	return drmm_add_action_or_reset(&xe->drm, xe_device_sysfs_fini, xe);	88	return devm_add_action_or_reset(dev, xe_device_sysfs_fini, xe);
89	}	89	}

drivers/gpu/drm/xe/xe_device_types.h+20−3

 		struct {
 			/** @sriov.vf.memirq: Memory Based Interrupts. */
 			struct xe_memirq memirq;
+			/** @sriov.vf.ggtt_balloon: GGTT regions excluded from use. */
+			struct drm_mm_node ggtt_balloon[2];
 		} vf;
 	} sriov;
 	/** @info: device info */
 	struct intel_device_info {
+		/** @info.platform_name: platform name */
+		const char *platform_name;
 		/** @info.graphics_name: graphics IP name */
 		const char *graphics_name;
 		/** @info.media_name: media IP name */
 		u8 has_heci_gscfi:1;
 		/** @info.skip_guc_pc: Skip GuC based PM feature init */
 		u8 skip_guc_pc:1;
+		/** @info.has_atomic_enable_pte_bit: Device has atomic enable PTE bit */
+		u8 has_atomic_enable_pte_bit:1;
+		/** @info.has_device_atomics_on_smem: Supports device atomics on SMEM */
+		u8 has_device_atomics_on_smem:1;
 #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
 		struct {
 		/** @d3cold.allowed: Indicates if d3cold is a valid device state */
 		bool allowed;
-		/** @d3cold.power_lost: Indicates if card has really lost power. */
-		bool power_lost;
 		/**
 		 * @d3cold.vram_threshold:
 		 *
 	/** @needs_flr_on_fini: requests function-reset on fini */
 	bool needs_flr_on_fini;
+	/** @wedged: Struct to control Wedged States and mode */
+	struct {
+		/** @wedged.flag: Xe device faced a critical error and is now blocked. */
+		atomic_t flag;
+		/** @wedged.mode: Mode controlled by kernel parameter and debugfs */
+		int mode;
+	} wedged;
 	/* private: */
 #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
 		struct mutex lock;
 	} exec_queue;
+	/** @run_ticks: hw engine class run time in ticks for this drm client */
+	u64 run_ticks[XE_ENGINE_CLASS_MAX];
 	/** @client: drm client */
 	struct xe_drm_client *client;
 };

drivers/gpu/drm/xe/xe_drm_client.c+123−1

@@ -2,6 +2,7 @@ · 1 unmodified line
2	/*	2	/*
3	* Copyright © 2023 Intel Corporation	3	* Copyright © 2023 Intel Corporation
4	*/	4	*/
		5	#include "xe_drm_client.h"
5		6
6	#include <drm/drm_print.h>	7	#include <drm/drm_print.h>
7	#include <drm/xe_drm.h>	8	#include <drm/xe_drm.h>
@@ -12,9 +13,66 @@ · 4 unmodified lines
12	#include "xe_bo.h"	13	#include "xe_bo.h"
13	#include "xe_bo_types.h"	14	#include "xe_bo_types.h"
14	#include "xe_device_types.h"	15	#include "xe_device_types.h"
15	#include "xe_drm_client.h"	16	#include "xe_exec_queue.h"
		17	#include "xe_force_wake.h"
		18	#include "xe_gt.h"
		19	#include "xe_hw_engine.h"
		20	#include "xe_pm.h"
16	#include "xe_trace.h"	21	#include "xe_trace.h"
17		22
		23	/**
		24	* DOC: DRM Client usage stats
		25	*
		26	* The drm/xe driver implements the DRM client usage stats specification as
		27	* documented in :ref:`drm-client-usage-stats`.
		28	*
		29	* Example of the output showing the implemented key value pairs and entirety of
		30	* the currently possible format options:
		31	*
		32	* ::
		33	*
		34	* pos: 0
		35	* flags: 0100002
		36	* mnt_id: 26
		37	* ino: 685
		38	* drm-driver: xe
		39	* drm-client-id: 3
		40	* drm-pdev: 0000:03:00.0
		41	* drm-total-system: 0
		42	* drm-shared-system: 0
		43	* drm-active-system: 0
		44	* drm-resident-system: 0
		45	* drm-purgeable-system: 0
		46	* drm-total-gtt: 192 KiB
		47	* drm-shared-gtt: 0
		48	* drm-active-gtt: 0
		49	* drm-resident-gtt: 192 KiB
		50	* drm-total-vram0: 23992 KiB
		51	* drm-shared-vram0: 16 MiB
		52	* drm-active-vram0: 0
		53	* drm-resident-vram0: 23992 KiB
		54	* drm-total-stolen: 0
		55	* drm-shared-stolen: 0
		56	* drm-active-stolen: 0
		57	* drm-resident-stolen: 0
		58	* drm-cycles-rcs: 28257900
		59	* drm-total-cycles-rcs: 7655183225
		60	* drm-cycles-bcs: 0
		61	* drm-total-cycles-bcs: 7655183225
		62	* drm-cycles-vcs: 0
		63	* drm-total-cycles-vcs: 7655183225
		64	* drm-engine-capacity-vcs: 2
		65	* drm-cycles-vecs: 0
		66	* drm-total-cycles-vecs: 7655183225
		67	* drm-engine-capacity-vecs: 2
		68	* drm-cycles-ccs: 0
		69	* drm-total-cycles-ccs: 7655183225
		70	* drm-engine-capacity-ccs: 4
		71	*
		72	* Possible `drm-cycles-` key names are: `rcs`, `ccs`, `bcs`, `vcs`, `vecs` and
		73	* "other".
		74	*/
		75
18	/**	76	/**
19	* xe_drm_client_alloc() - Allocate drm client	77	* xe_drm_client_alloc() - Allocate drm client
20	* @void: No arg	78	* @void: No arg
@@ -179,6 +237,69 @@ static void show_meminfo(struct drm_printer p, struct drm_file file) · 158 unmodified lines
179	}	237	}
180	}	238	}
181		239
		240	static void show_run_ticks(struct drm_printer p, struct drm_file file)
		241	{
		242	unsigned long class, i, gt_id, capacity[XE_ENGINE_CLASS_MAX] = { };
		243	struct xe_file *xef = file->driver_priv;
		244	struct xe_device *xe = xef->xe;
		245	struct xe_gt *gt;
		246	struct xe_hw_engine *hwe;
		247	struct xe_exec_queue *q;
		248	u64 gpu_timestamp;
		249
		250	xe_pm_runtime_get(xe);
		251
		252	/* Accumulate all the exec queues from this client */
		253	mutex_lock(&xef->exec_queue.lock);
		254	xa_for_each(&xef->exec_queue.xa, i, q) {
		255	xe_exec_queue_update_run_ticks(q);
		256	xef->run_ticks[q->class] += q->run_ticks - q->old_run_ticks;
		257	q->old_run_ticks = q->run_ticks;
		258	}
		259	mutex_unlock(&xef->exec_queue.lock);
		260
		261	/* Get the total GPU cycles */
		262	for_each_gt(gt, xe, gt_id) {
		263	hwe = xe_gt_any_hw_engine(gt);
		264	if (!hwe)
		265	continue;
		266
		267	xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
		268	gpu_timestamp = xe_hw_engine_read_timestamp(hwe);
		269	xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
		270	break;
		271	}
		272
		273	xe_pm_runtime_put(xe);
		274
		275	if (unlikely(!hwe))
		276	return;
		277
		278	for (class = 0; class < XE_ENGINE_CLASS_MAX; class++) {
		279	const char *class_name;
		280
		281	for_each_gt(gt, xe, gt_id)
		282	capacity[class] += gt->user_engines.instances_per_class[class];
		283
		284	/*
		285	* Engines may be fused off or not exposed to userspace. Don't
		286	* return anything if this entire class is not available
		287	*/
		288	if (!capacity[class])
		289	continue;
		290
		291	class_name = xe_hw_engine_class_to_str(class);
		292	drm_printf(p, "drm-cycles-%s:\t%llu\n",
		293	class_name, xef->run_ticks[class]);
		294	drm_printf(p, "drm-total-cycles-%s:\t%llu\n",
		295	class_name, gpu_timestamp);
		296
		297	if (capacity[class] > 1)
		298	drm_printf(p, "drm-engine-capacity-%s:\t%lu\n",
		299	class_name, capacity[class]);
		300	}
		301	}
		302
182	/**	303	/**
183	* xe_drm_client_fdinfo() - Callback for fdinfo interface	304	* xe_drm_client_fdinfo() - Callback for fdinfo interface
184	* @p: The drm_printer ptr	305	* @p: The drm_printer ptr
@@ -192,5 +313,6 @@ static void show_meminfo(struct drm_printer p, struct drm_file file) · 7 unmodified lines
192	void xe_drm_client_fdinfo(struct drm_printer p, struct drm_file file)	313	void xe_drm_client_fdinfo(struct drm_printer p, struct drm_file file)
193	{	314	{
194	show_meminfo(p, file);	315	show_meminfo(p, file);
		316	show_run_ticks(p, file);
195	}	317	}
196	#endif	318	#endif

drivers/gpu/drm/xe/xe_exec_queue.c+47−30

 	if (extensions) {
 		/*
-		 * may set q->usm, must come before xe_lrc_init(),
+		 * may set q->usm, must come before xe_lrc_create(),
 		 * may overwrite q->sched_props, must come before q->ops->init()
 		 */
 		err = exec_queue_user_extensions(xe, q, extensions, 0);
 		}
 	}
-	if (xe_exec_queue_is_parallel(q)) {
-		q->parallel.composite_fence_ctx = dma_fence_context_alloc(1);
-		q->parallel.composite_fence_seqno = XE_FENCE_INITIAL_SEQNO;
-	}
 	return q;
 }
 static int __xe_exec_queue_init(struct xe_exec_queue *q)
 {
-	struct xe_device *xe = gt_to_xe(q->gt);
 	int i, err;
 	for (i = 0; i < q->width; ++i) {
-		err = xe_lrc_init(q->lrc + i, q->hwe, q, q->vm, SZ_16K);
+		q->lrc[i] = xe_lrc_create(q->hwe, q->vm, SZ_16K);
-		if (err)
+		if (IS_ERR(q->lrc[i])) {
+			err = PTR_ERR(q->lrc[i]);
 			goto err_lrc;
+		}
 	}
 	err = q->ops->init(q);
 	if (err)
 		goto err_lrc;
-	/*
-	 * Normally the user vm holds an rpm ref to keep the device
-	 * awake, and the context holds a ref for the vm, however for
-	 * some engines we use the kernels migrate vm underneath which offers no
-	 * such rpm ref, or we lack a vm. Make sure we keep a ref here, so we
-	 * can perform GuC CT actions when needed. Caller is expected to have
-	 * already grabbed the rpm ref outside any sensitive locks.
-	 */
-	if (!(q->flags & EXEC_QUEUE_FLAG_PERMANENT) && (q->flags & EXEC_QUEUE_FLAG_VM || !q->vm))
-		xe_pm_runtime_get_noresume(xe);
 	return 0;
 err_lrc:
 	for (i = i - 1; i >= 0; --i)
-		xe_lrc_finish(q->lrc + i);
+		xe_lrc_put(q->lrc[i]);
 	return err;
 }
 	int i;
 	for (i = 0; i < q->width; ++i)
-		xe_lrc_finish(q->lrc + i);
+		xe_lrc_put(q->lrc[i]);
-	if (!(q->flags & EXEC_QUEUE_FLAG_PERMANENT) && (q->flags & EXEC_QUEUE_FLAG_VM || !q->vm))
-		xe_pm_runtime_put(gt_to_xe(q->gt));
 	__xe_exec_queue_free(q);
 }
 static s32 xe_exec_queue_num_job_inflight(struct xe_exec_queue *q)
 {
-	return q->lrc->fence_ctx.next_seqno - xe_lrc_seqno(q->lrc) - 1;
+	return q->lrc[0]->fence_ctx.next_seqno - xe_lrc_seqno(q->lrc[0]) - 1;
 }
 /**
  */
 bool xe_exec_queue_ring_full(struct xe_exec_queue *q)
 {
-	struct xe_lrc *lrc = q->lrc;
+	struct xe_lrc *lrc = q->lrc[0];
 	s32 max_job = lrc->ring.size / MAX_JOB_SIZE_BYTES;
 	return xe_exec_queue_num_job_inflight(q) >= max_job;
 		int i;
 		for (i = 0; i < q->width; ++i) {
-			if (xe_lrc_seqno(&q->lrc[i]) !=
+			if (xe_lrc_seqno(q->lrc[i]) !=
-			    q->lrc[i].fence_ctx.next_seqno - 1)
+			    q->lrc[i]->fence_ctx.next_seqno - 1)
 				return false;
 		}
 		return true;
 	}
-	return xe_lrc_seqno(&q->lrc[0]) ==
+	return xe_lrc_seqno(q->lrc[0]) ==
-		q->lrc[0].fence_ctx.next_seqno - 1;
+		q->lrc[0]->fence_ctx.next_seqno - 1;
+}
+/**
+ * xe_exec_queue_update_run_ticks() - Update run time in ticks for this exec queue
+ * from hw
+ * @q: The exec queue
+ *
+ * Update the timestamp saved by HW for this exec queue and save run ticks
+ * calculated by using the delta from last update.
+ */
+void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q)
+{
+	struct xe_lrc *lrc;
+	u32 old_ts, new_ts;
+	/*
+	 * Jobs that are run during driver load may use an exec_queue, but are
+	 * not associated with a user xe file, so avoid accumulating busyness
+	 * for kernel specific work.
+	 */
+	if (!q->vm || !q->vm->xef)
+		return;
+	/*
+	 * Only sample the first LRC. For parallel submission, all of them are
+	 * scheduled together and we compensate that below by multiplying by
+	 * width - this may introduce errors if that premise is not true and
+	 * they don't exit 100% aligned. On the other hand, looping through
+	 * the LRCs and reading them in different time could also introduce
+	 * errors.
+	 */
+	lrc = q->lrc[0];
+	new_ts = xe_lrc_update_timestamp(lrc, &old_ts);
+	q->run_ticks += (new_ts - old_ts) * q->width;
 }
 void xe_exec_queue_kill(struct xe_exec_queue *q)

drivers/gpu/drm/xe/xe_exec_queue.h+10−0

 void xe_exec_queue_destroy(struct kref *ref);
 void xe_exec_queue_assign_name(struct xe_exec_queue *q, u32 instance);
+static inline struct xe_exec_queue *
+xe_exec_queue_get_unless_zero(struct xe_exec_queue *q)
+{
+	if (kref_get_unless_zero(&q->refcount))
+		return q;
+	return NULL;
+}
 struct xe_exec_queue *xe_exec_queue_lookup(struct xe_file *xef, u32 id);
 static inline struct xe_exec_queue *xe_exec_queue_get(struct xe_exec_queue *q)
 					       struct xe_vm *vm);
 void xe_exec_queue_last_fence_set(struct xe_exec_queue *e, struct xe_vm *vm,
 				  struct dma_fence *fence);
+void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q);
 #endif

drivers/gpu/drm/xe/xe_exec_queue_types.h+5−11

 		struct xe_guc_exec_queue *guc;
 	};
-	/**
-	 * @parallel: parallel submission state
-	 */
-	struct {
-		/** @parallel.composite_fence_ctx: context composite fence */
-		u64 composite_fence_ctx;
-		/** @parallel.composite_fence_seqno: seqno for composite fence */
-		u32 composite_fence_seqno;
-	} parallel;
 	/** @sched_props: scheduling properties */
 	struct {
 		/** @sched_props.timeslice_us: timeslice period in micro-seconds */
 	 * Protected by @vm's resv. Unused if @vm == NULL.
 	 */
 	u64 tlb_flush_seqno;
+	/** @old_run_ticks: prior hw engine class run time in ticks for this exec queue */
+	u64 old_run_ticks;
+	/** @run_ticks: hw engine class run time in ticks for this exec queue */
+	u64 run_ticks;
 	/** @lrc: logical ring context for this exec queue */
-	struct xe_lrc lrc[];
+	struct xe_lrc *lrc[];
 };
 /**

drivers/gpu/drm/xe/xe_execlist.c+6−6

 #include "instructions/xe_mi_commands.h"
 #include "regs/xe_engine_regs.h"
-#include "regs/xe_gpu_commands.h"
 #include "regs/xe_gt_regs.h"
 #include "regs/xe_lrc_layout.h"
 #include "xe_assert.h"
 			port->last_ctx_id = 1;
 	}
-	__start_lrc(port->hwe, exl->q->lrc, port->last_ctx_id);
+	__start_lrc(port->hwe, exl->q->lrc[0], port->last_ctx_id);
 	port->running_exl = exl;
 	exl->has_run = true;
 }
 	if (!port->running_exl)
 		return;
-	xe_lrc_write_ring(&port->hwe->kernel_lrc, noop, sizeof(noop));
+	xe_lrc_write_ring(port->hwe->kernel_lrc, noop, sizeof(noop));
-	__start_lrc(port->hwe, &port->hwe->kernel_lrc, 0);
+	__start_lrc(port->hwe, port->hwe->kernel_lrc, 0);
 	port->running_exl = NULL;
 }
 static bool xe_execlist_is_idle(struct xe_execlist_exec_queue *exl)
 {
-	struct xe_lrc *lrc = exl->q->lrc;
+	struct xe_lrc *lrc = exl->q->lrc[0];
 	return lrc->ring.tail == lrc->ring.old_tail;
 }
 {
 	struct xe_sched_job *job = to_xe_sched_job(drm_job);
+	xe_exec_queue_update_run_ticks(job->q);
 	xe_sched_job_put(job);
 }
 	exl->q = q;
 	err = drm_sched_init(&exl->sched, &drm_sched_ops, NULL, 1,
-			     q->lrc[0].ring.size / MAX_JOB_SIZE_BYTES,
+			     q->lrc[0]->ring.size / MAX_JOB_SIZE_BYTES,
 			     XE_SCHED_HANG_LIMIT, XE_SCHED_JOB_TIMEOUT,
 			     NULL, NULL, q->hwe->name,
 			     gt_to_xe(q->gt)->drm.dev);

drivers/gpu/drm/xe/xe_ggtt.c+28−4

 #include <linux/io-64-nonatomic-lo-hi.h>
 #include <linux/sizes.h>
+#include <drm/drm_drv.h>
 #include <drm/drm_managed.h>
 #include <drm/i915_drm.h>
 #include "xe_device.h"
 #include "xe_gt.h"
 #include "xe_gt_printk.h"
+#include "xe_gt_sriov_vf.h"
 #include "xe_gt_tlb_invalidation.h"
 #include "xe_map.h"
 #include "xe_pm.h"
 	struct xe_device *xe = tile_to_xe(ggtt->tile);
 	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
 	unsigned int gsm_size;
+	int err;
 	if (IS_SRIOV_VF(xe))
 		gsm_size = SZ_8M; /* GGTT is expected to be 4GiB */
 	mutex_init(&ggtt->lock);
 	primelockdep(ggtt);
-	return drmm_add_action_or_reset(&xe->drm, ggtt_fini_early, ggtt);
+	err = drmm_add_action_or_reset(&xe->drm, ggtt_fini_early, ggtt);
+	if (err)
+		return err;
+	if (IS_SRIOV_VF(xe)) {
+		err = xe_gt_sriov_vf_prepare_ggtt(xe_tile_get_gt(ggtt->tile, 0));
+		if (err)
+			return err;
+	}
+	return 0;
 }
 static void xe_ggtt_invalidate(struct xe_ggtt *ggtt);
 void xe_ggtt_remove_node(struct xe_ggtt *ggtt, struct drm_mm_node *node,
 			 bool invalidate)
 {
-	xe_pm_runtime_get_noresume(tile_to_xe(ggtt->tile));
+	struct xe_device *xe = tile_to_xe(ggtt->tile);
+	bool bound;
+	int idx;
+	bound = drm_dev_enter(&xe->drm, &idx);
+	if (bound)
+		xe_pm_runtime_get_noresume(xe);
 	mutex_lock(&ggtt->lock);
-	xe_ggtt_clear(ggtt, node->start, node->size);
+	if (bound)
+		xe_ggtt_clear(ggtt, node->start, node->size);
 	drm_mm_remove_node(node);
 	node->size = 0;
 	mutex_unlock(&ggtt->lock);
+	if (!bound)
+		return;
 	if (invalidate)
 		xe_ggtt_invalidate(ggtt);
-	xe_pm_runtime_put(tile_to_xe(ggtt->tile));
+	xe_pm_runtime_put(xe);
+	drm_dev_exit(idx);
 }
 void xe_ggtt_remove_bo(struct xe_ggtt *ggtt, struct xe_bo *bo)

drivers/gpu/drm/xe/xe_gsc.c+3−0

 #include "xe_gsc.h"
+#include <linux/delay.h>
 #include <drm/drm_managed.h>
 #include <generated/xe_wa_oob.h>
 #include "xe_bo.h"
 #include "xe_device.h"
 #include "xe_exec_queue.h"
+#include "xe_force_wake.h"
 #include "xe_gsc_proxy.h"
 #include "xe_gsc_submit.h"
 #include "xe_gt.h"

drivers/gpu/drm/xe/xe_gsc.h+2−1

@@ -6,8 +6,9 @@ · 5 unmodified lines
6	#ifndef _XE_GSC_H_	6	#ifndef _XE_GSC_H_
7	#define _XE_GSC_H_	7	#define _XE_GSC_H_
8		8
9	#include "xe_gsc_types.h"	9	#include <linux/types.h>
10		10
		11	struct xe_gsc;
11	struct xe_gt;	12	struct xe_gt;
12	struct xe_hw_engine;	13	struct xe_hw_engine;
13		14

drivers/gpu/drm/xe/xe_gsc_proxy.c+1−0

@@ -15,6 +15,7 @@ · 14 unmodified lines
15	#include "abi/gsc_proxy_commands_abi.h"	15	#include "abi/gsc_proxy_commands_abi.h"
16	#include "regs/xe_gsc_regs.h"	16	#include "regs/xe_gsc_regs.h"
17	#include "xe_bo.h"	17	#include "xe_bo.h"
		18	#include "xe_force_wake.h"
18	#include "xe_gsc.h"	19	#include "xe_gsc.h"
19	#include "xe_gsc_submit.h"	20	#include "xe_gsc_submit.h"
20	#include "xe_gt.h"	21	#include "xe_gt.h"

drivers/gpu/drm/xe/xe_gsc_submit.c+1−0

@@ -8,6 +8,7 @@ · 7 unmodified lines
8	#include <linux/poison.h>	8	#include <linux/poison.h>
9		9
10	#include "abi/gsc_command_header_abi.h"	10	#include "abi/gsc_command_header_abi.h"
		11	#include "xe_assert.h"
11	#include "xe_bb.h"	12	#include "xe_bb.h"
12	#include "xe_exec_queue.h"	13	#include "xe_exec_queue.h"
13	#include "xe_gt_printk.h"	14	#include "xe_gt_printk.h"

drivers/gpu/drm/xe/xe_gt.c+105−37

 #include "xe_migrate.h"
 #include "xe_mmio.h"
 #include "xe_pat.h"
+#include "xe_pcode.h"
 #include "xe_pm.h"
 #include "xe_mocs.h"
 #include "xe_reg_sr.h"
 #include "xe_wa.h"
 #include "xe_wopcm.h"
+static void gt_fini(struct drm_device *drm, void *arg)
+{
+	struct xe_gt *gt = arg;
+	destroy_workqueue(gt->ordered_wq);
+}
 struct xe_gt *xe_gt_alloc(struct xe_tile *tile)
 {
 	struct xe_gt *gt;
+	int err;
 	gt = drmm_kzalloc(&tile_to_xe(tile)->drm, sizeof(*gt), GFP_KERNEL);
 	if (!gt)
 	gt->tile = tile;
 	gt->ordered_wq = alloc_ordered_workqueue("gt-ordered-wq", 0);
+	err = drmm_add_action_or_reset(&gt_to_xe(gt)->drm, gt_fini, gt);
+	if (err)
+		return ERR_PTR(err);
 	return gt;
 }
  */
 void xe_gt_remove(struct xe_gt *gt)
 {
-	xe_uc_remove(&gt->uc);
-}
-static void gt_fini(struct drm_device *drm, void *arg)
-{
-	struct xe_gt *gt = arg;
 	int i;
-	destroy_workqueue(gt->ordered_wq);
+	xe_uc_remove(&gt->uc);
 	for (i = 0; i < XE_ENGINE_CLASS_MAX; ++i)
 		xe_hw_fence_irq_finish(&gt->fence_irq[i]);
 	if (q->hwe->class == XE_ENGINE_CLASS_RENDER)
 		/* Big enough to emit all of the context's 3DSTATE */
-		bb = xe_bb_new(gt, xe_lrc_size(gt_to_xe(gt), q->hwe->class), false);
+		bb = xe_bb_new(gt, xe_gt_lrc_size(gt, q->hwe->class), false);
 	else
 		/* Just pick a large BB size */
 		bb = xe_bb_new(gt, SZ_4K, false);
 		xe_tuning_process_lrc(hwe);
 		default_lrc = drmm_kzalloc(&xe->drm,
-					   xe_lrc_size(xe, hwe->class),
+					   xe_gt_lrc_size(gt, hwe->class),
 					   GFP_KERNEL);
 		if (!default_lrc)
 			return -ENOMEM;
 		}
 		xe_map_memcpy_from(xe, default_lrc,
-				   &q->lrc[0].bo->vmap,
+				   &q->lrc[0]->bo->vmap,
-				   xe_lrc_pphwsp_offset(&q->lrc[0]),
+				   xe_lrc_pphwsp_offset(q->lrc[0]),
-				   xe_lrc_size(xe, hwe->class));
+				   xe_gt_lrc_size(gt, hwe->class));
 		gt->default_lrc[hwe->class] = default_lrc;
 put_nop_q:
 			return err;
 	}
-	err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
-	if (err)
-		return err;
-	err = xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
-	if (err)
-		return err;
 	xe_reg_sr_init(&gt->reg_sr, "GT", gt_to_xe(gt));
 	err = xe_wa_init(gt);
 	xe_wa_process_oob(gt);
 	xe_tuning_process_gt(gt);
+	xe_force_wake_init_gt(gt, gt_to_fw(gt));
+	xe_pcode_init(gt);
 	return 0;
 }
 			xe_lmtt_init(&gt_to_tile(gt)->sriov.pf.lmtt);
 	}
-	err = xe_gt_idle_sysfs_init(&gt->gtidle);
-	if (err)
-		goto err_force_wake;
 	/* Enable per hw engine IRQs */
 	xe_irq_enable_hwe(gt);
 	if (err)
 		goto err_force_wake;
+	err = xe_uc_init_post_hwconfig(&gt->uc);
+	if (err)
+		goto err_force_wake;
 	if (!xe_gt_is_media_type(gt)) {
 		/*
 		 * USM has its only SA pool to non-block behind user operations
 		}
 	}
-	err = xe_uc_init_post_hwconfig(&gt->uc);
-	if (err)
-		goto err_force_wake;
 	err = xe_uc_init_hw(&gt->uc);
 	if (err)
 		goto err_force_wake;
 	if (IS_SRIOV_PF(gt_to_xe(gt)) && !xe_gt_is_media_type(gt))
 		xe_lmtt_init_hw(&gt_to_tile(gt)->sriov.pf.lmtt);
+	if (IS_SRIOV_PF(gt_to_xe(gt)))
+		xe_gt_sriov_pf_init_hw(gt);
 	err = xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL);
 	XE_WARN_ON(err);
 	if (err)
 		goto out;
-	xe_gt_topology_init(gt);
+	xe_gt_mcr_init_early(gt);
-	xe_gt_mcr_init(gt);
 	xe_pat_init(gt);
 	err = xe_uc_init(&gt->uc);
 	if (err)
 		goto out_fw;
-	/* XXX: Fake that we pull the engine mask from hwconfig blob */
+	xe_gt_topology_init(gt);
-	gt->info.engine_mask = gt->info.__engine_mask;
+	xe_gt_mcr_init(gt);
 out_fw:
 	xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
 	if (err)
 		return err;
+	err = xe_gt_idle_init(&gt->gtidle);
+	if (err)
+		return err;
 	err = xe_gt_freq_init(gt);
 	if (err)
 		return err;
 	if (err)
 		return err;
-	return drmm_add_action_or_reset(&gt_to_xe(gt)->drm, gt_fini, gt);
+	xe_gt_record_user_engines(gt);
+	return 0;
+}
+void xe_gt_record_user_engines(struct xe_gt *gt)
+{
+	struct xe_hw_engine *hwe;
+	enum xe_hw_engine_id id;
+	gt->user_engines.mask = 0;
+	memset(gt->user_engines.instances_per_class, 0,
+	       sizeof(gt->user_engines.instances_per_class));
+	for_each_hw_engine(hwe, gt, id) {
+		if (xe_hw_engine_is_reserved(hwe))
+			continue;
+		gt->user_engines.mask |= BIT_ULL(id);
+		gt->user_engines.instances_per_class[hwe->class]++;
+	}
+	xe_gt_assert(gt, (gt->user_engines.mask | gt->info.engine_mask)
+		     == gt->info.engine_mask);
 }
 static int do_gt_reset(struct xe_gt *gt)
 	return err;
 }
+static int vf_gt_restart(struct xe_gt *gt)
+{
+	int err;
+	err = xe_uc_sanitize_reset(&gt->uc);
+	if (err)
+		return err;
+	err = xe_uc_init_hw(&gt->uc);
+	if (err)
+		return err;
+	err = xe_uc_start(&gt->uc);
+	if (err)
+		return err;
+	return 0;
+}
 static int do_gt_restart(struct xe_gt *gt)
 {
 	struct xe_hw_engine *hwe;
 	enum xe_hw_engine_id id;
 	int err;
+	if (IS_SRIOV_VF(gt_to_xe(gt)))
+		return vf_gt_restart(gt);
 	xe_pat_init(gt);
 	xe_gt_mcr_set_implicit_defaults(gt);
 	if (IS_SRIOV_PF(gt_to_xe(gt)) && !xe_gt_is_media_type(gt))
 		xe_lmtt_init_hw(&gt_to_tile(gt)->sriov.pf.lmtt);
+	if (IS_SRIOV_PF(gt_to_xe(gt)))
+		xe_gt_sriov_pf_init_hw(gt);
 	xe_mocs_init(gt);
 	err = xe_uc_start(&gt->uc);
 	if (err)
 {
 	int err;
+	if (xe_device_wedged(gt_to_xe(gt)))
+		return -ECANCELED;
 	/* We only support GT resets with GuC submission */
 	if (!xe_device_uc_enabled(gt_to_xe(gt)))
 		return -ENODEV;
 	xe_uc_stop_prepare(&gt->uc);
 	xe_gt_pagefault_reset(gt);
-	err = xe_uc_stop(&gt->uc);
+	xe_uc_stop(&gt->uc);
-	if (err)
-		goto err_out;
 	xe_gt_tlb_invalidation_reset(gt);
 err_fail:
 	xe_gt_err(gt, "reset failed (%pe)\n", ERR_PTR(err));
-	gt_to_xe(gt)->needs_flr_on_fini = true;
+	xe_device_declare_wedged(gt_to_xe(gt));
 	return err;
 }
 	if (err)
 		goto err_force_wake;
+	xe_gt_idle_disable_pg(gt);
 	XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL));
 	xe_gt_dbg(gt, "suspended\n");
 	if (err)
 		goto err_force_wake;
+	xe_gt_idle_enable_pg(gt);
 	XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL));
 	xe_gt_dbg(gt, "resumed\n");
 	return NULL;
 }
+struct xe_hw_engine *xe_gt_any_hw_engine(struct xe_gt *gt)
+{
+	struct xe_hw_engine *hwe;
+	enum xe_hw_engine_id id;
+	for_each_hw_engine(hwe, gt, id)
+		return hwe;
+	return NULL;
+}

drivers/gpu/drm/xe/xe_gt.h+27−0

@@ -8,6 +8,7 @@ · 7 unmodified lines
8		8
9	#include <drm/drm_util.h>	9	#include <drm/drm_util.h>
10		10
		11	#include "xe_device.h"
11	#include "xe_device_types.h"	12	#include "xe_device_types.h"
12	#include "xe_hw_engine.h"	13	#include "xe_hw_engine.h"
13		14
@@ -37,6 +38,19 @@ int xe_gt_init_hwconfig(struct xe_gt *gt); · 23 unmodified lines
37	int xe_gt_init_early(struct xe_gt *gt);	38	int xe_gt_init_early(struct xe_gt *gt);
38	int xe_gt_init(struct xe_gt *gt);	39	int xe_gt_init(struct xe_gt *gt);
39	int xe_gt_record_default_lrcs(struct xe_gt *gt);	40	int xe_gt_record_default_lrcs(struct xe_gt *gt);
		41
		42	/**
		43	* xe_gt_record_user_engines - save data related to engines available to
		44	* usersapce
		45	* @gt: GT structure
		46	*
		47	* Walk the available HW engines from gt->info.engine_mask and calculate data
		48	* related to those engines that may be used by userspace. To be used whenever
		49	* available engines change in runtime (e.g. with ccs_mode) or during
		50	* initialization
		51	*/
		52	void xe_gt_record_user_engines(struct xe_gt *gt);
		53
40	void xe_gt_suspend_prepare(struct xe_gt *gt);	54	void xe_gt_suspend_prepare(struct xe_gt *gt);
41	int xe_gt_suspend(struct xe_gt *gt);	55	int xe_gt_suspend(struct xe_gt *gt);
42	int xe_gt_resume(struct xe_gt *gt);	56	int xe_gt_resume(struct xe_gt *gt);
@@ -53,11 +67,24 @@ void xe_gt_remove(struct xe_gt *gt); · 10 unmodified lines
53	struct xe_hw_engine *	67	struct xe_hw_engine *
54	xe_gt_any_hw_engine_by_reset_domain(struct xe_gt *gt, enum xe_engine_class class);	68	xe_gt_any_hw_engine_by_reset_domain(struct xe_gt *gt, enum xe_engine_class class);
55		69
		70	/**
		71	* xe_gt_any_hw_engine - scan the list of engines and return the
		72	* first available
		73	* @gt: GT structure
		74	*/
		75	struct xe_hw_engine xe_gt_any_hw_engine(struct xe_gt gt);
		76
56	struct xe_hw_engine xe_gt_hw_engine(struct xe_gt gt,	77	struct xe_hw_engine xe_gt_hw_engine(struct xe_gt gt,
57	enum xe_engine_class class,	78	enum xe_engine_class class,
58	u16 instance,	79	u16 instance,
59	bool logical);	80	bool logical);
60		81
		82	static inline bool xe_gt_has_indirect_ring_state(struct xe_gt *gt)
		83	{
		84	return gt->info.has_indirect_ring_state &&
		85	xe_device_uc_enabled(gt_to_xe(gt));
		86	}
		87
61	static inline bool xe_gt_is_media_type(struct xe_gt *gt)	88	static inline bool xe_gt_is_media_type(struct xe_gt *gt)
62	{	89	{
63	return gt->info.type == XE_GT_TYPE_MEDIA;	90	return gt->info.type == XE_GT_TYPE_MEDIA;

drivers/gpu/drm/xe/xe_gt_ccs_mode.c+6−4

 #include "xe_assert.h"
 #include "xe_gt.h"
 #include "xe_gt_ccs_mode.h"
+#include "xe_gt_printk.h"
 #include "xe_gt_sysfs.h"
 #include "xe_mmio.h"
 	xe_mmio_write32(gt, CCS_MODE, mode);
-	xe_gt_info(gt, "CCS_MODE=%x config:%08x, num_engines:%d, num_slices:%d\n",
+	xe_gt_dbg(gt, "CCS_MODE=%x config:%08x, num_engines:%d, num_slices:%d\n",
-		   mode, config, num_engines, num_slices);
+		  mode, config, num_engines, num_slices);
 }
 void xe_gt_apply_ccs_mode(struct xe_gt *gt)
 	if (gt->ccs_mode != num_engines) {
 		xe_gt_info(gt, "Setting compute mode to %d\n", num_engines);
 		gt->ccs_mode = num_engines;
+		xe_gt_record_user_engines(gt);
 		xe_gt_reset_async(gt);
 	}
 	NULL,
 };
-static void xe_gt_ccs_mode_sysfs_fini(struct drm_device *drm, void *arg)
+static void xe_gt_ccs_mode_sysfs_fini(void *arg)
 {
 	struct xe_gt *gt = arg;
 	if (err)
 		return err;
-	return drmm_add_action_or_reset(&xe->drm, xe_gt_ccs_mode_sysfs_fini, gt);
+	return devm_add_action_or_reset(xe->drm.dev, xe_gt_ccs_mode_sysfs_fini, gt);
 }

drivers/gpu/drm/xe/xe_gt_clock.c+1−0

@@ -7,6 +7,7 @@ · 6 unmodified lines
7		7
8	#include "regs/xe_gt_regs.h"	8	#include "regs/xe_gt_regs.h"
9	#include "regs/xe_regs.h"	9	#include "regs/xe_regs.h"
		10	#include "xe_assert.h"
10	#include "xe_device.h"	11	#include "xe_device.h"
11	#include "xe_gt.h"	12	#include "xe_gt.h"
12	#include "xe_macros.h"	13	#include "xe_macros.h"

drivers/gpu/drm/xe/xe_gt_debugfs.c+31−0

 #include "xe_ggtt.h"
 #include "xe_gt.h"
 #include "xe_gt_mcr.h"
+#include "xe_gt_sriov_pf_debugfs.h"
+#include "xe_gt_sriov_vf_debugfs.h"
 #include "xe_gt_topology.h"
 #include "xe_hw_engine.h"
 #include "xe_lrc.h"
 #include "xe_macros.h"
+#include "xe_mocs.h"
 #include "xe_pat.h"
 #include "xe_pm.h"
 #include "xe_reg_sr.h"
 #include "xe_reg_whitelist.h"
+#include "xe_sriov.h"
 #include "xe_uc_debugfs.h"
 #include "xe_wa.h"
 	return 0;
 }
+static int force_reset_sync(struct xe_gt *gt, struct drm_printer *p)
+{
+	xe_pm_runtime_get(gt_to_xe(gt));
+	xe_gt_reset_async(gt);
+	xe_pm_runtime_put(gt_to_xe(gt));
+	flush_work(&gt->reset.worker);
+	return 0;
+}
 static int sa_info(struct xe_gt *gt, struct drm_printer *p)
 {
 	struct xe_tile *tile = gt_to_tile(gt);
 	return 0;
 }
+static int mocs(struct xe_gt *gt, struct drm_printer *p)
+{
+	xe_pm_runtime_get(gt_to_xe(gt));
+	xe_mocs_dump(gt, p);
+	xe_pm_runtime_put(gt_to_xe(gt));
+	return 0;
+}
 static int rcs_default_lrc(struct xe_gt *gt, struct drm_printer *p)
 {
 	xe_pm_runtime_get(gt_to_xe(gt));
 static const struct drm_info_list debugfs_list[] = {
 	{"hw_engines", .show = xe_gt_debugfs_simple_show, .data = hw_engines},
 	{"force_reset", .show = xe_gt_debugfs_simple_show, .data = force_reset},
+	{"force_reset_sync", .show = xe_gt_debugfs_simple_show, .data = force_reset_sync},
 	{"sa_info", .show = xe_gt_debugfs_simple_show, .data = sa_info},
 	{"topology", .show = xe_gt_debugfs_simple_show, .data = topology},
 	{"steering", .show = xe_gt_debugfs_simple_show, .data = steering},
 	{"register-save-restore", .show = xe_gt_debugfs_simple_show, .data = register_save_restore},
 	{"workarounds", .show = xe_gt_debugfs_simple_show, .data = workarounds},
 	{"pat", .show = xe_gt_debugfs_simple_show, .data = pat},
+	{"mocs", .show = xe_gt_debugfs_simple_show, .data = mocs},
 	{"default_lrc_rcs", .show = xe_gt_debugfs_simple_show, .data = rcs_default_lrc},
 	{"default_lrc_ccs", .show = xe_gt_debugfs_simple_show, .data = ccs_default_lrc},
 	{"default_lrc_bcs", .show = xe_gt_debugfs_simple_show, .data = bcs_default_lrc},
 				 root, minor);
 	xe_uc_debugfs_register(&gt->uc, root);
+	if (IS_SRIOV_PF(xe))
+		xe_gt_sriov_pf_debugfs_register(gt, root);
+	else if (IS_SRIOV_VF(xe))
+		xe_gt_sriov_vf_debugfs_register(gt, root);
 }

drivers/gpu/drm/xe/xe_gt_freq.c+4−4

@@ -13,7 +13,7 @@ · 12 unmodified lines
13		13
14	#include "xe_device_types.h"	14	#include "xe_device_types.h"
15	#include "xe_gt_sysfs.h"	15	#include "xe_gt_sysfs.h"
16	#include "xe_gt_throttle_sysfs.h"	16	#include "xe_gt_throttle.h"
17	#include "xe_guc_pc.h"	17	#include "xe_guc_pc.h"
18	#include "xe_pm.h"	18	#include "xe_pm.h"
19		19
@@ -209,7 +209,7 @@ static const struct attribute *freq_attrs[] = { · 189 unmodified lines
209	NULL	209	NULL
210	};	210	};
211		211
212	static void freq_fini(struct drm_device drm, void arg)	212	static void freq_fini(void *arg)
213	{	213	{
214	struct kobject *kobj = arg;	214	struct kobject *kobj = arg;
215		215
@@ -237,7 +237,7 @@ int xe_gt_freq_init(struct xe_gt *gt) · 21 unmodified lines
237	if (!gt->freq)	237	if (!gt->freq)
238	return -ENOMEM;	238	return -ENOMEM;
239		239
240	err = drmm_add_action_or_reset(&xe->drm, freq_fini, gt->freq);	240	err = devm_add_action(xe->drm.dev, freq_fini, gt->freq);
241	if (err)	241	if (err)
242	return err;	242	return err;
243		243
@@ -245,5 +245,5 @@ int xe_gt_freq_init(struct xe_gt *gt) · 1 unmodified line
245	if (err)	245	if (err)
246	return err;	246	return err;
247		247
248	return xe_gt_throttle_sysfs_init(gt);	248	return xe_gt_throttle_init(gt);
249	}	249	}

drivers/gpu/drm/xe/xe_gt_idle.c+61−5

@@ -5,12 +5,14 @@ · 4 unmodified lines
5		5
6	#include <drm/drm_managed.h>	6	#include <drm/drm_managed.h>
7		7
		8	#include "xe_force_wake.h"
8	#include "xe_device.h"	9	#include "xe_device.h"
9	#include "xe_gt.h"	10	#include "xe_gt.h"
10	#include "xe_gt_idle.h"	11	#include "xe_gt_idle.h"
11	#include "xe_gt_sysfs.h"	12	#include "xe_gt_sysfs.h"
12	#include "xe_guc_pc.h"	13	#include "xe_guc_pc.h"
13	#include "regs/xe_gt_regs.h"	14	#include "regs/xe_gt_regs.h"
		15	#include "xe_macros.h"
14	#include "xe_mmio.h"	16	#include "xe_mmio.h"
15	#include "xe_pm.h"	17	#include "xe_pm.h"
16		18
@@ -92,6 +94,50 @@ static u64 get_residency_ms(struct xe_gt_idle *gtidle, u64 cur_residency) · 75 unmodified lines
92	return cur_residency;	94	return cur_residency;
93	}	95	}
94		96
		97	void xe_gt_idle_enable_pg(struct xe_gt *gt)
		98	{
		99	struct xe_device *xe = gt_to_xe(gt);
		100	u32 pg_enable;
		101	int i, j;
		102
		103	/* Disable CPG for PVC */
		104	if (xe->info.platform == XE_PVC)
		105	return;
		106
		107	xe_device_assert_mem_access(gt_to_xe(gt));
		108
		109	pg_enable = RENDER_POWERGATE_ENABLE \| MEDIA_POWERGATE_ENABLE;
		110
		111	for (i = XE_HW_ENGINE_VCS0, j = 0; i <= XE_HW_ENGINE_VCS7; ++i, ++j) {
		112	if ((gt->info.engine_mask & BIT(i)))
		113	pg_enable \|= (VDN_HCP_POWERGATE_ENABLE(j) \|
		114	VDN_MFXVDENC_POWERGATE_ENABLE(j));
		115	}
		116
		117	XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
		118	if (xe->info.skip_guc_pc) {
		119	/*
		120	* GuC sets the hysteresis value when GuC PC is enabled
		121	* else set it to 25 (25 * 1.28us)
		122	*/
		123	xe_mmio_write32(gt, MEDIA_POWERGATE_IDLE_HYSTERESIS, 25);
		124	xe_mmio_write32(gt, RENDER_POWERGATE_IDLE_HYSTERESIS, 25);
		125	}
		126
		127	xe_mmio_write32(gt, POWERGATE_ENABLE, pg_enable);
		128	XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
		129	}
		130
		131	void xe_gt_idle_disable_pg(struct xe_gt *gt)
		132	{
		133	xe_device_assert_mem_access(gt_to_xe(gt));
		134	XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
		135
		136	xe_mmio_write32(gt, POWERGATE_ENABLE, 0);
		137
		138	XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
		139	}
		140
95	static ssize_t name_show(struct device *dev,	141	static ssize_t name_show(struct device *dev,
96	struct device_attribute attr, char buff)	142	struct device_attribute attr, char buff)
97	{	143	{
@@ -144,15 +190,24 @@ static const struct attribute *gt_idle_attrs[] = { · 46 unmodified lines
144	NULL,	190	NULL,
145	};	191	};
146		192
147	static void gt_idle_sysfs_fini(struct drm_device drm, void arg)	193	static void gt_idle_fini(void *arg)
148	{	194	{
149	struct kobject *kobj = arg;	195	struct kobject *kobj = arg;
		196	struct xe_gt *gt = kobj_to_gt(kobj->parent);
		197
		198	xe_gt_idle_disable_pg(gt);
		199
		200	if (gt_to_xe(gt)->info.skip_guc_pc) {
		201	XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
		202	xe_gt_idle_disable_c6(gt);
		203	xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
		204	}
150		205
151	sysfs_remove_files(kobj, gt_idle_attrs);	206	sysfs_remove_files(kobj, gt_idle_attrs);
152	kobject_put(kobj);	207	kobject_put(kobj);
153	}	208	}
154		209
155	int xe_gt_idle_sysfs_init(struct xe_gt_idle *gtidle)	210	int xe_gt_idle_init(struct xe_gt_idle *gtidle)
156	{	211	{
157	struct xe_gt *gt = gtidle_to_gt(gtidle);	212	struct xe_gt *gt = gtidle_to_gt(gtidle);
158	struct xe_device *xe = gt_to_xe(gt);	213	struct xe_device *xe = gt_to_xe(gt);
@@ -181,7 +236,9 @@ int xe_gt_idle_sysfs_init(struct xe_gt_idle *gtidle) · 22 unmodified lines
181	return err;	236	return err;
182	}	237	}
183		238
184	return drmm_add_action_or_reset(&xe->drm, gt_idle_sysfs_fini, kobj);	239	xe_gt_idle_enable_pg(gt);
		240
		241	return devm_add_action_or_reset(xe->drm.dev, gt_idle_fini, kobj);
185	}	242	}
186		243
187	void xe_gt_idle_enable_c6(struct xe_gt *gt)	244	void xe_gt_idle_enable_c6(struct xe_gt *gt)
@@ -199,9 +256,8 @@ void xe_gt_idle_enable_c6(struct xe_gt *gt) · 11 unmodified lines
199	void xe_gt_idle_disable_c6(struct xe_gt *gt)	256	void xe_gt_idle_disable_c6(struct xe_gt *gt)
200	{	257	{
201	xe_device_assert_mem_access(gt_to_xe(gt));	258	xe_device_assert_mem_access(gt_to_xe(gt));
202	xe_force_wake_assert_held(gt_to_fw(gt), XE_FORCEWAKE_ALL);	259	xe_force_wake_assert_held(gt_to_fw(gt), XE_FW_GT);
203		260
204	xe_mmio_write32(gt, PG_ENABLE, 0);
205	xe_mmio_write32(gt, RC_CONTROL, 0);	261	xe_mmio_write32(gt, RC_CONTROL, 0);
206	xe_mmio_write32(gt, RC_STATE, 0);	262	xe_mmio_write32(gt, RC_STATE, 0);
207	}	263	}

drivers/gpu/drm/xe/xe_gt_idle.h+3−1

@@ -10,8 +10,10 @@ · 9 unmodified lines
10		10
11	struct xe_gt;	11	struct xe_gt;
12		12
13	int xe_gt_idle_sysfs_init(struct xe_gt_idle *gtidle);	13	int xe_gt_idle_init(struct xe_gt_idle *gtidle);
14	void xe_gt_idle_enable_c6(struct xe_gt *gt);	14	void xe_gt_idle_enable_c6(struct xe_gt *gt);
15	void xe_gt_idle_disable_c6(struct xe_gt *gt);	15	void xe_gt_idle_disable_c6(struct xe_gt *gt);
		16	void xe_gt_idle_enable_pg(struct xe_gt *gt);
		17	void xe_gt_idle_disable_pg(struct xe_gt *gt);
16		18
17	#endif /* _XE_GT_IDLE_H_ */	19	#endif /* _XE_GT_IDLE_H_ */

drivers/gpu/drm/xe/xe_gt_mcr.c+22−5

@@ -375,18 +375,35 @@ static const struct { · 374 unmodified lines
375	[IMPLICIT_STEERING] = { "IMPLICIT", NULL },	375	[IMPLICIT_STEERING] = { "IMPLICIT", NULL },
376	};	376	};
377		377
378	void xe_gt_mcr_init(struct xe_gt *gt)	378	/**
		379	* xe_gt_mcr_init_early - Early initialization of the MCR support
		380	* @gt: GT structure
		381	*
		382	* Perform early software only initialization of the MCR lock to allow
		383	* the synchronization on accessing the STEER_SEMAPHORE register and
		384	* use the xe_gt_mcr_multicast_write() function.
		385	*/
		386	void xe_gt_mcr_init_early(struct xe_gt *gt)
379	{	387	{
380	struct xe_device *xe = gt_to_xe(gt);
381
382	BUILD_BUG_ON(IMPLICIT_STEERING + 1 != NUM_STEERING_TYPES);	388	BUILD_BUG_ON(IMPLICIT_STEERING + 1 != NUM_STEERING_TYPES);
383	BUILD_BUG_ON(ARRAY_SIZE(xe_steering_types) != NUM_STEERING_TYPES);	389	BUILD_BUG_ON(ARRAY_SIZE(xe_steering_types) != NUM_STEERING_TYPES);
384		390
		391	spin_lock_init(&gt->mcr_lock);
		392	}
		393
		394	/**
		395	* xe_gt_mcr_init - Normal initialization of the MCR support
		396	* @gt: GT structure
		397	*
		398	* Perform normal initialization of the MCR for all usages.
		399	*/
		400	void xe_gt_mcr_init(struct xe_gt *gt)
		401	{
		402	struct xe_device *xe = gt_to_xe(gt);
		403
385	if (IS_SRIOV_VF(xe))	404	if (IS_SRIOV_VF(xe))
386	return;	405	return;
387		406
388	spin_lock_init(&gt->mcr_lock);
389
390	if (gt->info.type == XE_GT_TYPE_MEDIA) {	407	if (gt->info.type == XE_GT_TYPE_MEDIA) {
391	drm_WARN_ON(&xe->drm, MEDIA_VER(xe) < 13);	408	drm_WARN_ON(&xe->drm, MEDIA_VER(xe) < 13);
392		409

drivers/gpu/drm/xe/xe_gt_mcr.h+25−0

@@ -12,6 +12,7 @@ · 11 unmodified lines
12	struct drm_printer;	12	struct drm_printer;
13	struct xe_gt;	13	struct xe_gt;
14		14
		15	void xe_gt_mcr_init_early(struct xe_gt *gt);
15	void xe_gt_mcr_init(struct xe_gt *gt);	16	void xe_gt_mcr_init(struct xe_gt *gt);
16		17
17	void xe_gt_mcr_set_implicit_defaults(struct xe_gt *gt);	18	void xe_gt_mcr_set_implicit_defaults(struct xe_gt *gt);
@@ -40,4 +41,28 @@ void xe_gt_mcr_get_dss_steering(struct xe_gt gt, unsigned int dss, u16 group, · 22 unmodified lines
40	for_each_dss((dss), (gt)) \	41	for_each_dss((dss), (gt)) \
41	for_each_if((xe_gt_mcr_get_dss_steering((gt), (dss), &(group), &(instance)), true))	42	for_each_if((xe_gt_mcr_get_dss_steering((gt), (dss), &(group), &(instance)), true))
42		43
		44	/*
		45	* Loop over each DSS available for geometry and determine the group and
		46	* instance IDs that should be used to steer MCR accesses toward this DSS.
		47	* @dss: DSS ID to obtain steering for
		48	* @gt: GT structure
		49	* @group: steering group ID, data type: u16
		50	* @instance: steering instance ID, data type: u16
		51	*/
		52	#define for_each_geometry_dss(dss, gt, group, instance) \
		53	for_each_dss_steering(dss, gt, group, instance) \
		54	if (xe_gt_has_geometry_dss(gt, dss))
		55
		56	/*
		57	* Loop over each DSS available for compute and determine the group and
		58	* instance IDs that should be used to steer MCR accesses toward this DSS.
		59	* @dss: DSS ID to obtain steering for
		60	* @gt: GT structure
		61	* @group: steering group ID, data type: u16
		62	* @instance: steering instance ID, data type: u16
		63	*/
		64	#define for_each_compute_dss(dss, gt, group, instance) \
		65	for_each_dss_steering(dss, gt, group, instance) \
		66	if (xe_gt_has_compute_dss(gt, dss))
		67
43	#endif /* _XE_GT_MCR_H_ */	68	#endif /* _XE_GT_MCR_H_ */

drivers/gpu/drm/xe/xe_gt_pagefault.c+7−9

 #include "xe_guc.h"
 #include "xe_guc_ct.h"
 #include "xe_migrate.h"
-#include "xe_pt.h"
 #include "xe_trace.h"
 #include "xe_vm.h"
 		drm_exec_retry_on_contention(&exec);
 		if (ret)
 			goto unlock_dma_resv;
-	}
-	/* Bind VMA only to the GT that has faulted */
+		/* Bind VMA only to the GT that has faulted */
-	trace_xe_vma_pf_bind(vma);
+		trace_xe_vma_pf_bind(vma);
-	fence = __xe_pt_bind_vma(tile, vma, xe_tile_migrate_engine(tile), NULL, 0,
+		fence = xe_vma_rebind(vm, vma, BIT(tile->id));
-				 vma->tile_present & BIT(tile->id));
+		if (IS_ERR(fence)) {
-	if (IS_ERR(fence)) {
+			ret = PTR_ERR(fence);
-		ret = PTR_ERR(fence);
+			goto unlock_dma_resv;
-		goto unlock_dma_resv;
+		}
 	}
 	/*

drivers/gpu/drm/xe/xe_gt_sriov_pf.c+32−0

@@ -5,8 +5,12 @@ · 4 unmodified lines
5		5
6	#include <drm/drm_managed.h>	6	#include <drm/drm_managed.h>
7		7
		8	#include "regs/xe_sriov_regs.h"
		9
8	#include "xe_gt_sriov_pf.h"	10	#include "xe_gt_sriov_pf.h"
9	#include "xe_gt_sriov_pf_helpers.h"	11	#include "xe_gt_sriov_pf_helpers.h"
		12	#include "xe_gt_sriov_pf_service.h"
		13	#include "xe_mmio.h"
10		14
11	/*	15	/*
12	* VF's metadata is maintained in the flexible array where:	16	* VF's metadata is maintained in the flexible array where:
@@ -48,5 +52,33 @@ int xe_gt_sriov_pf_init_early(struct xe_gt *gt) · 35 unmodified lines
48	if (err)	52	if (err)
49	return err;	53	return err;
50		54
		55	err = xe_gt_sriov_pf_service_init(gt);
		56	if (err)
		57	return err;
		58
51	return 0;	59	return 0;
52	}	60	}
		61
		62	static bool pf_needs_enable_ggtt_guest_update(struct xe_device *xe)
		63	{
		64	return GRAPHICS_VERx100(xe) == 1200;
		65	}
		66
		67	static void pf_enable_ggtt_guest_update(struct xe_gt *gt)
		68	{
		69	xe_mmio_write32(gt, VIRTUAL_CTRL_REG, GUEST_GTT_UPDATE_EN);
		70	}
		71
		72	/**
		73	* xe_gt_sriov_pf_init_hw - Initialize SR-IOV hardware support.
		74	* @gt: the &xe_gt to initialize
		75	*
		76	* On some platforms the PF must explicitly enable VF's access to the GGTT.
		77	*/
		78	void xe_gt_sriov_pf_init_hw(struct xe_gt *gt)
		79	{
		80	if (pf_needs_enable_ggtt_guest_update(gt_to_xe(gt)))
		81	pf_enable_ggtt_guest_update(gt);
		82
		83	xe_gt_sriov_pf_service_update(gt);
		84	}

drivers/gpu/drm/xe/xe_gt_sriov_pf.h+5−0

 #ifdef CONFIG_PCI_IOV
 int xe_gt_sriov_pf_init_early(struct xe_gt *gt);
+void xe_gt_sriov_pf_init_hw(struct xe_gt *gt);
 #else
 static inline int xe_gt_sriov_pf_init_early(struct xe_gt *gt)
 {
 	return 0;
 }
+static inline void xe_gt_sriov_pf_init_hw(struct xe_gt *gt)
+{
+}
 #endif
 #endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c+99−6

@@ -25,6 +25,7 @@ · 24 unmodified lines
25	#include "xe_guc_fwif.h"	25	#include "xe_guc_fwif.h"
26	#include "xe_guc_id_mgr.h"	26	#include "xe_guc_id_mgr.h"
27	#include "xe_guc_klv_helpers.h"	27	#include "xe_guc_klv_helpers.h"
		28	#include "xe_guc_klv_thresholds_set.h"
28	#include "xe_guc_submit.h"	29	#include "xe_guc_submit.h"
29	#include "xe_lmtt.h"	30	#include "xe_lmtt.h"
30	#include "xe_map.h"	31	#include "xe_map.h"
@@ -187,14 +188,20 @@ static int pf_push_vf_cfg_dbs(struct xe_gt *gt, unsigned int vfid, u32 begin, u3 · 156 unmodified lines
187	return pf_push_vf_cfg_klvs(gt, vfid, 2, klvs, ARRAY_SIZE(klvs));	188	return pf_push_vf_cfg_klvs(gt, vfid, 2, klvs, ARRAY_SIZE(klvs));
188	}	189	}
189		190
190	static int pf_push_vf_cfg_exec_quantum(struct xe_gt *gt, unsigned int vfid, u32 exec_quantum)	191	static int pf_push_vf_cfg_exec_quantum(struct xe_gt gt, unsigned int vfid, u32 exec_quantum)
191	{	192	{
192	return pf_push_vf_cfg_u32(gt, vfid, GUC_KLV_VF_CFG_EXEC_QUANTUM_KEY, exec_quantum);	193	/* GuC will silently clamp values exceeding max */
		194	exec_quantum = min_t(u32, exec_quantum, GUC_KLV_VF_CFG_EXEC_QUANTUM_MAX_VALUE);
		195
		196	return pf_push_vf_cfg_u32(gt, vfid, GUC_KLV_VF_CFG_EXEC_QUANTUM_KEY, *exec_quantum);
193	}	197	}
194		198
195	static int pf_push_vf_cfg_preempt_timeout(struct xe_gt *gt, unsigned int vfid, u32 preempt_timeout)	199	static int pf_push_vf_cfg_preempt_timeout(struct xe_gt gt, unsigned int vfid, u32 preempt_timeout)
196	{	200	{
197	return pf_push_vf_cfg_u32(gt, vfid, GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_KEY, preempt_timeout);	201	/* GuC will silently clamp values exceeding max */
		202	preempt_timeout = min_t(u32, preempt_timeout, GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_MAX_VALUE);
		203
		204	return pf_push_vf_cfg_u32(gt, vfid, GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_KEY, *preempt_timeout);
198	}	205	}
199		206
200	static int pf_push_vf_cfg_lmem(struct xe_gt *gt, unsigned int vfid, u64 size)	207	static int pf_push_vf_cfg_lmem(struct xe_gt *gt, unsigned int vfid, u64 size)
@@ -202,6 +209,15 @@ static int pf_push_vf_cfg_lmem(struct xe_gt *gt, unsigned int vfid, u64 size) · 1 unmodified line
202	return pf_push_vf_cfg_u64(gt, vfid, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, size);	209	return pf_push_vf_cfg_u64(gt, vfid, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, size);
203	}	210	}
204		211
		212	static int pf_push_vf_cfg_threshold(struct xe_gt *gt, unsigned int vfid,
		213	enum xe_guc_klv_threshold_index index, u32 value)
		214	{
		215	u32 key = xe_guc_klv_threshold_index_to_key(index);
		216
		217	xe_gt_assert(gt, key);
		218	return pf_push_vf_cfg_u32(gt, vfid, key, value);
		219	}
		220
205	static struct xe_gt_sriov_config pf_pick_vf_config(struct xe_gt gt, unsigned int vfid)	221	static struct xe_gt_sriov_config pf_pick_vf_config(struct xe_gt gt, unsigned int vfid)
206	{	222	{
207	xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));	223	xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
@@ -1604,7 +1620,7 @@ static int pf_provision_exec_quantum(struct xe_gt *gt, unsigned int vfid, · 1396 unmodified lines
1604	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);	1620	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);
1605	int err;	1621	int err;
1606		1622
1607	err = pf_push_vf_cfg_exec_quantum(gt, vfid, exec_quantum);	1623	err = pf_push_vf_cfg_exec_quantum(gt, vfid, &exec_quantum);
1608	if (unlikely(err))	1624	if (unlikely(err))
1609	return err;	1625	return err;
1610		1626
@@ -1674,7 +1690,7 @@ static int pf_provision_preempt_timeout(struct xe_gt *gt, unsigned int vfid, · 63 unmodified lines
1674	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);	1690	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);
1675	int err;	1691	int err;
1676		1692
1677	err = pf_push_vf_cfg_preempt_timeout(gt, vfid, preempt_timeout);	1693	err = pf_push_vf_cfg_preempt_timeout(gt, vfid, &preempt_timeout);
1678	if (unlikely(err))	1694	if (unlikely(err))
1679	return err;	1695	return err;
1680		1696
@@ -1742,6 +1758,83 @@ static void pf_reset_config_sched(struct xe_gt gt, struct xe_gt_sriov_config c · 61 unmodified lines
1742	config->preempt_timeout = 0;	1758	config->preempt_timeout = 0;
1743	}	1759	}
1744		1760
		1761	static int pf_provision_threshold(struct xe_gt *gt, unsigned int vfid,
		1762	enum xe_guc_klv_threshold_index index, u32 value)
		1763	{
		1764	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);
		1765	int err;
		1766
		1767	err = pf_push_vf_cfg_threshold(gt, vfid, index, value);
		1768	if (unlikely(err))
		1769	return err;
		1770
		1771	config->thresholds[index] = value;
		1772
		1773	return 0;
		1774	}
		1775
		1776	static int pf_get_threshold(struct xe_gt *gt, unsigned int vfid,
		1777	enum xe_guc_klv_threshold_index index)
		1778	{
		1779	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);
		1780
		1781	return config->thresholds[index];
		1782	}
		1783
		1784	static const char *threshold_unit(u32 threshold)
		1785	{
		1786	return threshold ? "" : "(disabled)";
		1787	}
		1788
		1789	/**
		1790	* xe_gt_sriov_pf_config_set_threshold - Configure threshold for the VF.
		1791	* @gt: the &xe_gt
		1792	* @vfid: the VF identifier
		1793	* @index: the threshold index
		1794	* @value: requested value (0 means disabled)
		1795	*
		1796	* This function can only be called on PF.
		1797	*
		1798	* Return: 0 on success or a negative error code on failure.
		1799	*/
		1800	int xe_gt_sriov_pf_config_set_threshold(struct xe_gt *gt, unsigned int vfid,
		1801	enum xe_guc_klv_threshold_index index, u32 value)
		1802	{
		1803	u32 key = xe_guc_klv_threshold_index_to_key(index);
		1804	const char *name = xe_guc_klv_key_to_string(key);
		1805	int err;
		1806
		1807	mutex_lock(xe_gt_sriov_pf_master_mutex(gt));
		1808	err = pf_provision_threshold(gt, vfid, index, value);
		1809	mutex_unlock(xe_gt_sriov_pf_master_mutex(gt));
		1810
		1811	return pf_config_set_u32_done(gt, vfid, value,
		1812	xe_gt_sriov_pf_config_get_threshold(gt, vfid, index),
		1813	name, threshold_unit, err);
		1814	}
		1815
		1816	/**
		1817	* xe_gt_sriov_pf_config_get_threshold - Get VF's threshold.
		1818	* @gt: the &xe_gt
		1819	* @vfid: the VF identifier
		1820	* @index: the threshold index
		1821	*
		1822	* This function can only be called on PF.
		1823	*
		1824	* Return: value of VF's (or PF's) threshold.
		1825	*/
		1826	u32 xe_gt_sriov_pf_config_get_threshold(struct xe_gt *gt, unsigned int vfid,
		1827	enum xe_guc_klv_threshold_index index)
		1828	{
		1829	u32 value;
		1830
		1831	mutex_lock(xe_gt_sriov_pf_master_mutex(gt));
		1832	value = pf_get_threshold(gt, vfid, index);
		1833	mutex_unlock(xe_gt_sriov_pf_master_mutex(gt));
		1834
		1835	return value;
		1836	}
		1837
1745	static void pf_release_vf_config(struct xe_gt *gt, unsigned int vfid)	1838	static void pf_release_vf_config(struct xe_gt *gt, unsigned int vfid)
1746	{	1839	{
1747	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);	1840	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);

drivers/gpu/drm/xe/xe_gt_sriov_pf_config.h+6−0

 #include <linux/types.h>
+enum xe_guc_klv_threshold_index;
 struct drm_printer;
 struct xe_gt;
 int xe_gt_sriov_pf_config_set_preempt_timeout(struct xe_gt *gt, unsigned int vfid,
 					      u32 preempt_timeout);
+u32 xe_gt_sriov_pf_config_get_threshold(struct xe_gt *gt, unsigned int vfid,
+					enum xe_guc_klv_threshold_index index);
+int xe_gt_sriov_pf_config_set_threshold(struct xe_gt *gt, unsigned int vfid,
+					enum xe_guc_klv_threshold_index index, u32 value);
 int xe_gt_sriov_pf_config_set_fair(struct xe_gt *gt, unsigned int vfid, unsigned int num_vfs);
 int xe_gt_sriov_pf_config_release(struct xe_gt *gt, unsigned int vfid, bool force);
 int xe_gt_sriov_pf_config_push(struct xe_gt *gt, unsigned int vfid, bool refresh);

drivers/gpu/drm/xe/xe_gt_sriov_pf_config_types.h+4−0

 #include <drm/drm_mm.h>
+#include "xe_guc_klv_thresholds_set_types.h"
 struct xe_bo;
 /**
 	u32 exec_quantum;
 	/** @preempt_timeout: preemption timeout in microseconds. */
 	u32 preempt_timeout;
+	/** @thresholds: GuC thresholds for adverse events notifications. */
+	u32 thresholds[XE_GUC_KLV_NUM_THRESHOLDS];
 };
 /**

drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c+427−0

@@ -0,0 +1,427 @@
	1	// SPDX-License-Identifier: MIT
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#include <linux/debugfs.h>
	7
	8	#include <drm/drm_print.h>
	9	#include <drm/drm_debugfs.h>
	10
	11	#include "xe_bo.h"
	12	#include "xe_debugfs.h"
	13	#include "xe_device.h"
	14	#include "xe_gt.h"
	15	#include "xe_gt_debugfs.h"
	16	#include "xe_gt_sriov_pf_config.h"
	17	#include "xe_gt_sriov_pf_control.h"
	18	#include "xe_gt_sriov_pf_debugfs.h"
	19	#include "xe_gt_sriov_pf_helpers.h"
	20	#include "xe_gt_sriov_pf_monitor.h"
	21	#include "xe_gt_sriov_pf_policy.h"
	22	#include "xe_gt_sriov_pf_service.h"
	23	#include "xe_pm.h"
	24
	25	/*
	26	* /sys/kernel/debug/dri/0/
	27	* ├── gt0 # d_inode->i_private = gt
	28	* │ ├── pf # d_inode->i_private = gt
	29	* │ ├── vf1 # d_inode->i_private = VFID(1)
	30	* : :
	31	* │ ├── vfN # d_inode->i_private = VFID(N)
	32	*/
	33
	34	static void extract_priv(struct dentry d)
	35	{
	36	return d->d_inode->i_private;
	37	}
	38
	39	static struct xe_gt extract_gt(struct dentry d)
	40	{
	41	return extract_priv(d->d_parent);
	42	}
	43
	44	static unsigned int extract_vfid(struct dentry *d)
	45	{
	46	return extract_priv(d) == extract_gt(d) ? PFID : (uintptr_t)extract_priv(d);
	47	}
	48
	49	/*
	50	* /sys/kernel/debug/dri/0/
	51	* ├── gt0
	52	* │ ├── pf
	53	* │ │ ├── ggtt_available
	54	* │ │ ├── ggtt_provisioned
	55	* │ │ ├── contexts_provisioned
	56	* │ │ ├── doorbells_provisioned
	57	* │ │ ├── runtime_registers
	58	* │ │ ├── negotiated_versions
	59	* │ │ ├── adverse_events
	60	*/
	61
	62	static const struct drm_info_list pf_info[] = {
	63	{
	64	"ggtt_available",
	65	.show = xe_gt_debugfs_simple_show,
	66	.data = xe_gt_sriov_pf_config_print_available_ggtt,
	67	},
	68	{
	69	"ggtt_provisioned",
	70	.show = xe_gt_debugfs_simple_show,
	71	.data = xe_gt_sriov_pf_config_print_ggtt,
	72	},
	73	{
	74	"contexts_provisioned",
	75	.show = xe_gt_debugfs_simple_show,
	76	.data = xe_gt_sriov_pf_config_print_ctxs,
	77	},
	78	{
	79	"doorbells_provisioned",
	80	.show = xe_gt_debugfs_simple_show,
	81	.data = xe_gt_sriov_pf_config_print_dbs,
	82	},
	83	{
	84	"runtime_registers",
	85	.show = xe_gt_debugfs_simple_show,
	86	.data = xe_gt_sriov_pf_service_print_runtime,
	87	},
	88	{
	89	"negotiated_versions",
	90	.show = xe_gt_debugfs_simple_show,
	91	.data = xe_gt_sriov_pf_service_print_version,
	92	},
	93	{
	94	"adverse_events",
	95	.show = xe_gt_debugfs_simple_show,
	96	.data = xe_gt_sriov_pf_monitor_print_events,
	97	},
	98	};
	99
	100	/*
	101	* /sys/kernel/debug/dri/0/
	102	* ├── gt0
	103	* │ ├── pf
	104	* │ │ ├── reset_engine
	105	* │ │ ├── sample_period
	106	* │ │ ├── sched_if_idle
	107	*/
	108
	109	#define DEFINE_SRIOV_GT_POLICY_DEBUGFS_ATTRIBUTE(POLICY, TYPE, FORMAT) \
	110	\
	111	static int POLICY##_set(void *data, u64 val) \
	112	{ \
	113	struct xe_gt *gt = extract_gt(data); \
	114	struct xe_device *xe = gt_to_xe(gt); \
	115	int err; \
	116	\
	117	if (val > (TYPE)~0ull) \
	118	return -EOVERFLOW; \
	119	\
	120	xe_pm_runtime_get(xe); \
	121	err = xe_gt_sriov_pf_policy_set_##POLICY(gt, val); \
	122	xe_pm_runtime_put(xe); \
	123	\
	124	return err; \
	125	} \
	126	\
	127	static int POLICY##_get(void data, u64 val) \
	128	{ \
	129	struct xe_gt *gt = extract_gt(data); \
	130	\
	131	*val = xe_gt_sriov_pf_policy_get_##POLICY(gt); \
	132	return 0; \
	133	} \
	134	\
	135	DEFINE_DEBUGFS_ATTRIBUTE(POLICY##_fops, POLICY##_get, POLICY##_set, FORMAT)
	136
	137	DEFINE_SRIOV_GT_POLICY_DEBUGFS_ATTRIBUTE(reset_engine, bool, "%llu\n");
	138	DEFINE_SRIOV_GT_POLICY_DEBUGFS_ATTRIBUTE(sched_if_idle, bool, "%llu\n");
	139	DEFINE_SRIOV_GT_POLICY_DEBUGFS_ATTRIBUTE(sample_period, u32, "%llu\n");
	140
	141	static void pf_add_policy_attrs(struct xe_gt gt, struct dentry parent)
	142	{
	143	xe_gt_assert(gt, gt == extract_gt(parent));
	144	xe_gt_assert(gt, PFID == extract_vfid(parent));
	145
	146	debugfs_create_file_unsafe("reset_engine", 0644, parent, parent, &reset_engine_fops);
	147	debugfs_create_file_unsafe("sched_if_idle", 0644, parent, parent, &sched_if_idle_fops);
	148	debugfs_create_file_unsafe("sample_period_ms", 0644, parent, parent, &sample_period_fops);
	149	}
	150
	151	/*
	152	* /sys/kernel/debug/dri/0/
	153	* ├── gt0
	154	* │ ├── pf
	155	* │ │ ├── ggtt_spare
	156	* │ │ ├── lmem_spare
	157	* │ │ ├── doorbells_spare
	158	* │ │ ├── contexts_spare
	159	* │ │ ├── exec_quantum_ms
	160	* │ │ ├── preempt_timeout_us
	161	* │ ├── vf1
	162	* │ │ ├── ggtt_quota
	163	* │ │ ├── lmem_quota
	164	* │ │ ├── doorbells_quota
	165	* │ │ ├── contexts_quota
	166	* │ │ ├── exec_quantum_ms
	167	* │ │ ├── preempt_timeout_us
	168	*/
	169
	170	#define DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(CONFIG, TYPE, FORMAT) \
	171	\
	172	static int CONFIG##_set(void *data, u64 val) \
	173	{ \
	174	struct xe_gt *gt = extract_gt(data); \
	175	unsigned int vfid = extract_vfid(data); \
	176	struct xe_device *xe = gt_to_xe(gt); \
	177	int err; \
	178	\
	179	if (val > (TYPE)~0ull) \
	180	return -EOVERFLOW; \
	181	\
	182	xe_pm_runtime_get(xe); \
	183	err = xe_gt_sriov_pf_config_set_##CONFIG(gt, vfid, val); \
	184	xe_pm_runtime_put(xe); \
	185	\
	186	return err; \
	187	} \
	188	\
	189	static int CONFIG##_get(void data, u64 val) \
	190	{ \
	191	struct xe_gt *gt = extract_gt(data); \
	192	unsigned int vfid = extract_vfid(data); \
	193	\
	194	*val = xe_gt_sriov_pf_config_get_##CONFIG(gt, vfid); \
	195	return 0; \
	196	} \
	197	\
	198	DEFINE_DEBUGFS_ATTRIBUTE(CONFIG##_fops, CONFIG##_get, CONFIG##_set, FORMAT)
	199
	200	DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(ggtt, u64, "%llu\n");
	201	DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(lmem, u64, "%llu\n");
	202	DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(ctxs, u32, "%llu\n");
	203	DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(dbs, u32, "%llu\n");
	204	DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(exec_quantum, u32, "%llu\n");
	205	DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(preempt_timeout, u32, "%llu\n");
	206
	207	/*
	208	* /sys/kernel/debug/dri/0/
	209	* ├── gt0
	210	* │ ├── pf
	211	* │ │ ├── threshold_cat_error_count
	212	* │ │ ├── threshold_doorbell_time_us
	213	* │ │ ├── threshold_engine_reset_count
	214	* │ │ ├── threshold_guc_time_us
	215	* │ │ ├── threshold_irq_time_us
	216	* │ │ ├── threshold_page_fault_count
	217	* │ ├── vf1
	218	* │ │ ├── threshold_cat_error_count
	219	* │ │ ├── threshold_doorbell_time_us
	220	* │ │ ├── threshold_engine_reset_count
	221	* │ │ ├── threshold_guc_time_us
	222	* │ │ ├── threshold_irq_time_us
	223	* │ │ ├── threshold_page_fault_count
	224	*/
	225
	226	static int set_threshold(void *data, u64 val, enum xe_guc_klv_threshold_index index)
	227	{
	228	struct xe_gt *gt = extract_gt(data);
	229	unsigned int vfid = extract_vfid(data);
	230	struct xe_device *xe = gt_to_xe(gt);
	231	int err;
	232
	233	if (val > (u32)~0ull)
	234	return -EOVERFLOW;
	235
	236	xe_pm_runtime_get(xe);
	237	err = xe_gt_sriov_pf_config_set_threshold(gt, vfid, index, val);
	238	xe_pm_runtime_put(xe);
	239
	240	return err;
	241	}
	242
	243	static int get_threshold(void data, u64 val, enum xe_guc_klv_threshold_index index)
	244	{
	245	struct xe_gt *gt = extract_gt(data);
	246	unsigned int vfid = extract_vfid(data);
	247
	248	*val = xe_gt_sriov_pf_config_get_threshold(gt, vfid, index);
	249	return 0;
	250	}
	251
	252	#define DEFINE_SRIOV_GT_THRESHOLD_DEBUGFS_ATTRIBUTE(THRESHOLD, INDEX) \
	253	\
	254	static int THRESHOLD##_set(void *data, u64 val) \
	255	{ \
	256	return set_threshold(data, val, INDEX); \
	257	} \
	258	\
	259	static int THRESHOLD##_get(void data, u64 val) \
	260	{ \
	261	return get_threshold(data, val, INDEX); \
	262	} \
	263	\
	264	DEFINE_DEBUGFS_ATTRIBUTE(THRESHOLD##_fops, THRESHOLD##_get, THRESHOLD##_set, "%llu\n")
	265
	266	/* generate all threshold attributes */
	267	#define define_threshold_attribute(TAG, NAME, ...) \
	268	DEFINE_SRIOV_GT_THRESHOLD_DEBUGFS_ATTRIBUTE(NAME, MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG));
	269	MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_threshold_attribute)
	270	#undef define_threshold_attribute
	271
	272	static void pf_add_config_attrs(struct xe_gt gt, struct dentry parent, unsigned int vfid)
	273	{
	274	xe_gt_assert(gt, gt == extract_gt(parent));
	275	xe_gt_assert(gt, vfid == extract_vfid(parent));
	276
	277	if (!xe_gt_is_media_type(gt)) {
	278	debugfs_create_file_unsafe(vfid ? "ggtt_quota" : "ggtt_spare",
	279	0644, parent, parent, &ggtt_fops);
	280	if (IS_DGFX(gt_to_xe(gt)))
	281	debugfs_create_file_unsafe(vfid ? "lmem_quota" : "lmem_spare",
	282	0644, parent, parent, &lmem_fops);
	283	}
	284	debugfs_create_file_unsafe(vfid ? "doorbells_quota" : "doorbells_spare",
	285	0644, parent, parent, &dbs_fops);
	286	debugfs_create_file_unsafe(vfid ? "contexts_quota" : "contexts_spare",
	287	0644, parent, parent, &ctxs_fops);
	288	debugfs_create_file_unsafe("exec_quantum_ms", 0644, parent, parent,
	289	&exec_quantum_fops);
	290	debugfs_create_file_unsafe("preempt_timeout_us", 0644, parent, parent,
	291	&preempt_timeout_fops);
	292
	293	/* register all threshold attributes */
	294	#define register_threshold_attribute(TAG, NAME, ...) \
	295	debugfs_create_file_unsafe("threshold_" #NAME, 0644, parent, parent, \
	296	&NAME##_fops);
	297	MAKE_XE_GUC_KLV_THRESHOLDS_SET(register_threshold_attribute)
	298	#undef register_threshold_attribute
	299	}
	300
	301	/*
	302	* /sys/kernel/debug/dri/0/
	303	* ├── gt0
	304	* │ ├── vf1
	305	* │ │ ├── control { stop, pause, resume }
	306	*/
	307
	308	static const struct {
	309	const char *cmd;
	310	int (fn)(struct xe_gt gt, unsigned int vfid);
	311	} control_cmds[] = {
	312	{ "stop", xe_gt_sriov_pf_control_stop_vf },
	313	{ "pause", xe_gt_sriov_pf_control_pause_vf },
	314	{ "resume", xe_gt_sriov_pf_control_resume_vf },
	315	};
	316
	317	static ssize_t control_write(struct file file, const char __user buf, size_t count, loff_t *pos)
	318	{
	319	struct dentry *dent = file_dentry(file);
	320	struct dentry *parent = dent->d_parent;
	321	struct xe_gt *gt = extract_gt(parent);
	322	struct xe_device *xe = gt_to_xe(gt);
	323	unsigned int vfid = extract_vfid(parent);
	324	int ret = -EINVAL;
	325	char cmd[32];
	326	size_t n;
	327
	328	xe_gt_assert(gt, vfid);
	329	xe_gt_sriov_pf_assert_vfid(gt, vfid);
	330
	331	if (*pos)
	332	return -ESPIPE;
	333
	334	if (count > sizeof(cmd) - 1)
	335	return -EINVAL;
	336
	337	ret = simple_write_to_buffer(cmd, sizeof(cmd) - 1, pos, buf, count);
	338	if (ret < 0)
	339	return ret;
	340	cmd[ret] = '\0';
	341
	342	for (n = 0; n < ARRAY_SIZE(control_cmds); n++) {
	343	xe_gt_assert(gt, sizeof(cmd) > strlen(control_cmds[n].cmd));
	344
	345	if (sysfs_streq(cmd, control_cmds[n].cmd)) {
	346	xe_pm_runtime_get(xe);
	347	ret = control_cmds[n].fn ? (*control_cmds[n].fn)(gt, vfid) : 0;
	348	xe_pm_runtime_put(xe);
	349	break;
	350	}
	351	}
	352
	353	return (ret < 0) ? ret : count;
	354	}
	355
	356	static ssize_t control_read(struct file file, char __user buf, size_t count, loff_t *ppos)
	357	{
	358	char help[128];
	359	size_t n;
	360
	361	help[0] = '\0';
	362	for (n = 0; n < ARRAY_SIZE(control_cmds); n++) {
	363	strlcat(help, control_cmds[n].cmd, sizeof(help));
	364	strlcat(help, "\n", sizeof(help));
	365	}
	366
	367	return simple_read_from_buffer(buf, count, ppos, help, strlen(help));
	368	}
	369
	370	static const struct file_operations control_ops = {
	371	.owner = THIS_MODULE,
	372	.open = simple_open,
	373	.write = control_write,
	374	.read = control_read,
	375	.llseek = default_llseek,
	376	};
	377
	378	/**
	379	* xe_gt_sriov_pf_debugfs_register - Register SR-IOV PF specific entries in GT debugfs.
	380	* @gt: the &xe_gt to register
	381	* @root: the &dentry that represents the GT directory
	382	*
	383	* Register SR-IOV PF entries that are GT related and must be shown under GT debugfs.
	384	*/
	385	void xe_gt_sriov_pf_debugfs_register(struct xe_gt gt, struct dentry root)
	386	{
	387	struct xe_device *xe = gt_to_xe(gt);
	388	struct drm_minor *minor = xe->drm.primary;
	389	int n, totalvfs = xe_sriov_pf_get_totalvfs(xe);
	390	struct dentry *pfdentry;
	391	struct dentry *vfdentry;
	392	char buf[14]; /* should be enough up to "vf%u\0" for 2^32 - 1 */
	393
	394	xe_gt_assert(gt, IS_SRIOV_PF(xe));
	395	xe_gt_assert(gt, root->d_inode->i_private == gt);
	396
	397	/*
	398	* /sys/kernel/debug/dri/0/
	399	* ├── gt0
	400	* │ ├── pf
	401	*/
	402	pfdentry = debugfs_create_dir("pf", root);
	403	if (IS_ERR(pfdentry))
	404	return;
	405	pfdentry->d_inode->i_private = gt;
	406
	407	drm_debugfs_create_files(pf_info, ARRAY_SIZE(pf_info), pfdentry, minor);
	408	pf_add_policy_attrs(gt, pfdentry);
	409	pf_add_config_attrs(gt, pfdentry, PFID);
	410
	411	for (n = 1; n <= totalvfs; n++) {
	412	/*
	413	* /sys/kernel/debug/dri/0/
	414	* ├── gt0
	415	* │ ├── vf1
	416	* │ ├── vf2
	417	*/
	418	snprintf(buf, sizeof(buf), "vf%u", n);
	419	vfdentry = debugfs_create_dir(buf, root);
	420	if (IS_ERR(vfdentry))
	421	break;
	422	vfdentry->d_inode->i_private = (void *)(uintptr_t)n;
	423
	424	pf_add_config_attrs(gt, vfdentry, VFID(n));
	425	debugfs_create_file("control", 0600, vfdentry, NULL, &control_ops);
	426	}
	427	}

drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.h+18−0

@@ -0,0 +1,18 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_SRIOV_PF_DEBUGFS_H_
	7	#define _XE_GT_SRIOV_PF_DEBUGFS_H_
	8
	9	struct xe_gt;
	10	struct dentry;
	11
	12	#ifdef CONFIG_PCI_IOV
	13	void xe_gt_sriov_pf_debugfs_register(struct xe_gt gt, struct dentry root);
	14	#else
	15	static inline void xe_gt_sriov_pf_debugfs_register(struct xe_gt gt, struct dentry root) { }
	16	#endif
	17
	18	#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.c+147−0

@@ -0,0 +1,147 @@
	1	// SPDX-License-Identifier: MIT
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#include "abi/guc_actions_sriov_abi.h"
	7	#include "abi/guc_messages_abi.h"
	8
	9	#include "xe_gt_sriov_pf_config.h"
	10	#include "xe_gt_sriov_pf_helpers.h"
	11	#include "xe_gt_sriov_pf_monitor.h"
	12	#include "xe_gt_sriov_printk.h"
	13	#include "xe_guc_klv_helpers.h"
	14	#include "xe_guc_klv_thresholds_set.h"
	15
	16	/**
	17	* xe_gt_sriov_pf_monitor_flr - Cleanup VF data after VF FLR.
	18	* @gt: the &xe_gt
	19	* @vfid: the VF identifier
	20	*
	21	* On FLR this function will reset all event data related to the VF.
	22	* This function is for PF only.
	23	*/
	24	void xe_gt_sriov_pf_monitor_flr(struct xe_gt *gt, u32 vfid)
	25	{
	26	int e;
	27
	28	xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	29	xe_gt_sriov_pf_assert_vfid(gt, vfid);
	30
	31	for (e = 0; e < XE_GUC_KLV_NUM_THRESHOLDS; e++)
	32	gt->sriov.pf.vfs[vfid].monitor.guc.events[e] = 0;
	33	}
	34
	35	static void pf_update_event_counter(struct xe_gt *gt, u32 vfid,
	36	enum xe_guc_klv_threshold_index e)
	37	{
	38	xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	39	xe_gt_assert(gt, e < XE_GUC_KLV_NUM_THRESHOLDS);
	40
	41	gt->sriov.pf.vfs[vfid].monitor.guc.events[e]++;
	42	}
	43
	44	static int pf_handle_vf_threshold_event(struct xe_gt *gt, u32 vfid, u32 threshold)
	45	{
	46	char origin[8];
	47	int e;
	48
	49	e = xe_guc_klv_threshold_key_to_index(threshold);
	50	xe_sriov_function_name(vfid, origin, sizeof(origin));
	51
	52	/* was there a new KEY added that we missed? */
	53	if (unlikely(e < 0)) {
	54	xe_gt_sriov_notice(gt, "unknown threshold key %#x reported for %s\n",
	55	threshold, origin);
	56	return -ENOTCONN;
	57	}
	58
	59	xe_gt_sriov_dbg(gt, "%s exceeded threshold %u %s\n",
	60	origin, xe_gt_sriov_pf_config_get_threshold(gt, vfid, e),
	61	xe_guc_klv_key_to_string(threshold));
	62
	63	pf_update_event_counter(gt, vfid, e);
	64
	65	return 0;
	66	}
	67
	68	/**
	69	* xe_gt_sriov_pf_monitor_process_guc2pf - Handle adverse event notification from the GuC.
	70	* @gt: the &xe_gt
	71	* @msg: G2H event message
	72	* @len: length of the message
	73	*
	74	* This function is intended for PF only.
	75	*
	76	* Return: 0 on success or a negative error code on failure.
	77	*/
	78	int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt gt, const u32 msg, u32 len)
	79	{
	80	struct xe_device *xe = gt_to_xe(gt);
	81	u32 vfid;
	82	u32 threshold;
	83
	84	xe_gt_assert(gt, len >= GUC_HXG_MSG_MIN_LEN);
	85	xe_gt_assert(gt, FIELD_GET(GUC_HXG_MSG_0_ORIGIN, msg[0]) == GUC_HXG_ORIGIN_GUC);
	86	xe_gt_assert(gt, FIELD_GET(GUC_HXG_MSG_0_TYPE, msg[0]) == GUC_HXG_TYPE_EVENT);
	87	xe_gt_assert(gt, FIELD_GET(GUC_HXG_EVENT_MSG_0_ACTION, msg[0]) ==
	88	GUC_ACTION_GUC2PF_ADVERSE_EVENT);
	89
	90	if (unlikely(!IS_SRIOV_PF(xe)))
	91	return -EPROTO;
	92
	93	if (unlikely(FIELD_GET(GUC2PF_ADVERSE_EVENT_EVENT_MSG_0_MBZ, msg[0])))
	94	return -EPFNOSUPPORT;
	95
	96	if (unlikely(len < GUC2PF_ADVERSE_EVENT_EVENT_MSG_LEN))
	97	return -EPROTO;
	98
	99	vfid = FIELD_GET(GUC2PF_ADVERSE_EVENT_EVENT_MSG_1_VFID, msg[1]);
	100	threshold = FIELD_GET(GUC2PF_ADVERSE_EVENT_EVENT_MSG_2_THRESHOLD, msg[2]);
	101
	102	if (unlikely(vfid > xe_gt_sriov_pf_get_totalvfs(gt)))
	103	return -EINVAL;
	104
	105	return pf_handle_vf_threshold_event(gt, vfid, threshold);
	106	}
	107
	108	/**
	109	* xe_gt_sriov_pf_monitor_print_events - Print adverse events counters.
	110	* @gt: the &xe_gt to print events from
	111	* @p: the &drm_printer
	112	*
	113	* Print adverse events counters for all VFs.
	114	* VFs with no events are not printed.
	115	*
	116	* This function can only be called on PF.
	117	*/
	118	void xe_gt_sriov_pf_monitor_print_events(struct xe_gt gt, struct drm_printer p)
	119	{
	120	unsigned int n, total_vfs = xe_gt_sriov_pf_get_totalvfs(gt);
	121	const struct xe_gt_sriov_monitor *data;
	122	int e;
	123
	124	xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	125
	126	for (n = 1; n <= total_vfs; n++) {
	127	data = &gt->sriov.pf.vfs[n].monitor;
	128
	129	for (e = 0; e < XE_GUC_KLV_NUM_THRESHOLDS; e++)
	130	if (data->guc.events[e])
	131	break;
	132
	133	/* skip empty unless in debug mode */
	134	if (e >= XE_GUC_KLV_NUM_THRESHOLDS &&
	135	!IS_ENABLED(CONFIG_DRM_XE_DEBUG_SRIOV))
	136	continue;
	137
	138	#define __format(...) "%s:%u "
	139	#define __value(TAG, NAME, ...) , #NAME, data->guc.events[MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG)]
	140
	141	drm_printf(p, "VF%u:\t" MAKE_XE_GUC_KLV_THRESHOLDS_SET(__format) "\n",
	142	n MAKE_XE_GUC_KLV_THRESHOLDS_SET(__value));
	143
	144	#undef __format
	145	#undef __value
	146	}
	147	}

drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.h+27−0

@@ -0,0 +1,27 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_SRIOV_PF_MONITOR_H_
	7	#define _XE_GT_SRIOV_PF_MONITOR_H_
	8
	9	#include <linux/errno.h>
	10	#include <linux/types.h>
	11
	12	struct drm_printer;
	13	struct xe_gt;
	14
	15	void xe_gt_sriov_pf_monitor_flr(struct xe_gt *gt, u32 vfid);
	16	void xe_gt_sriov_pf_monitor_print_events(struct xe_gt gt, struct drm_printer p);
	17
	18	#ifdef CONFIG_PCI_IOV
	19	int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt gt, const u32 msg, u32 len);
	20	#else
	21	static inline int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt gt, const u32 msg, u32 len)
	22	{
	23	return -EPROTO;
	24	}
	25	#endif
	26
	27	#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor_types.h+22−0

@@ -0,0 +1,22 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_SRIOV_PF_MONITOR_TYPES_H_
	7	#define _XE_GT_SRIOV_PF_MONITOR_TYPES_H_
	8
	9	#include "xe_guc_klv_thresholds_set_types.h"
	10
	11	/**
	12	* struct xe_gt_sriov_monitor - GT level per-VF monitoring data.
	13	*/
	14	struct xe_gt_sriov_monitor {
	15	/** @guc: monitoring data related to the GuC. */
	16	struct {
	17	/** @guc.events: number of adverse events reported by the GuC. */
	18	unsigned int events[XE_GUC_KLV_NUM_THRESHOLDS];
	19	} guc;
	20	};
	21
	22	#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c+550−0

@@ -0,0 +1,550 @@
	1	// SPDX-License-Identifier: MIT
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#include <drm/drm_managed.h>
	7
	8	#include "abi/guc_actions_sriov_abi.h"
	9	#include "abi/guc_relay_actions_abi.h"
	10
	11	#include "regs/xe_gt_regs.h"
	12	#include "regs/xe_guc_regs.h"
	13	#include "regs/xe_regs.h"
	14
	15	#include "xe_mmio.h"
	16	#include "xe_gt_sriov_printk.h"
	17	#include "xe_gt_sriov_pf_helpers.h"
	18	#include "xe_gt_sriov_pf_service.h"
	19	#include "xe_gt_sriov_pf_service_types.h"
	20	#include "xe_guc_ct.h"
	21	#include "xe_guc_hxg_helpers.h"
	22
	23	static void pf_init_versions(struct xe_gt *gt)
	24	{
	25	BUILD_BUG_ON(!GUC_RELAY_VERSION_BASE_MAJOR && !GUC_RELAY_VERSION_BASE_MINOR);
	26	BUILD_BUG_ON(GUC_RELAY_VERSION_BASE_MAJOR > GUC_RELAY_VERSION_LATEST_MAJOR);
	27
	28	/* base versions may differ between platforms */
	29	gt->sriov.pf.service.version.base.major = GUC_RELAY_VERSION_BASE_MAJOR;
	30	gt->sriov.pf.service.version.base.minor = GUC_RELAY_VERSION_BASE_MINOR;
	31
	32	/* latest version is same for all platforms */
	33	gt->sriov.pf.service.version.latest.major = GUC_RELAY_VERSION_LATEST_MAJOR;
	34	gt->sriov.pf.service.version.latest.minor = GUC_RELAY_VERSION_LATEST_MINOR;
	35	}
	36
	37	/* Return: 0 on success or a negative error code on failure. */
	38	static int pf_negotiate_version(struct xe_gt *gt,
	39	u32 wanted_major, u32 wanted_minor,
	40	u32 major, u32 minor)
	41	{
	42	struct xe_gt_sriov_pf_service_version base = gt->sriov.pf.service.version.base;
	43	struct xe_gt_sriov_pf_service_version latest = gt->sriov.pf.service.version.latest;
	44
	45	xe_gt_assert(gt, base.major);
	46	xe_gt_assert(gt, base.major <= latest.major);
	47	xe_gt_assert(gt, (base.major < latest.major) \|\| (base.minor <= latest.minor));
	48
	49	/* VF doesn't care - return our latest */
	50	if (wanted_major == VF2PF_HANDSHAKE_MAJOR_ANY &&
	51	wanted_minor == VF2PF_HANDSHAKE_MINOR_ANY) {
	52	*major = latest.major;
	53	*minor = latest.minor;
	54	return 0;
	55	}
	56
	57	/* VF wants newer than our - return our latest */
	58	if (wanted_major > latest.major) {
	59	*major = latest.major;
	60	*minor = latest.minor;
	61	return 0;
	62	}
	63
	64	/* VF wants older than min required - reject */
	65	if (wanted_major < base.major \|\|
	66	(wanted_major == base.major && wanted_minor < base.minor)) {
	67	return -EPERM;
	68	}
	69
	70	/* previous major - return wanted, as we should still support it */
	71	if (wanted_major < latest.major) {
	72	/* XXX: we are not prepared for multi-versions yet */
	73	xe_gt_assert(gt, base.major == latest.major);
	74	return -ENOPKG;
	75	}
	76
	77	/* same major - return common minor */
	78	*major = wanted_major;
	79	*minor = min_t(u32, latest.minor, wanted_minor);
	80	return 0;
	81	}
	82
	83	static void pf_connect(struct xe_gt *gt, u32 vfid, u32 major, u32 minor)
	84	{
	85	xe_gt_sriov_pf_assert_vfid(gt, vfid);
	86	xe_gt_assert(gt, major \|\| minor);
	87
	88	gt->sriov.pf.vfs[vfid].version.major = major;
	89	gt->sriov.pf.vfs[vfid].version.minor = minor;
	90	}
	91
	92	static void pf_disconnect(struct xe_gt *gt, u32 vfid)
	93	{
	94	xe_gt_sriov_pf_assert_vfid(gt, vfid);
	95
	96	gt->sriov.pf.vfs[vfid].version.major = 0;
	97	gt->sriov.pf.vfs[vfid].version.minor = 0;
	98	}
	99
	100	static bool pf_is_negotiated(struct xe_gt *gt, u32 vfid, u32 major, u32 minor)
	101	{
	102	xe_gt_sriov_pf_assert_vfid(gt, vfid);
	103
	104	return major == gt->sriov.pf.vfs[vfid].version.major &&
	105	minor <= gt->sriov.pf.vfs[vfid].version.minor;
	106	}
	107
	108	static const struct xe_reg tgl_runtime_regs[] = {
	109	RPM_CONFIG0, /* _MMIO(0x0d00) */
	110	MIRROR_FUSE3, /* _MMIO(0x9118) */
	111	XELP_EU_ENABLE, /* _MMIO(0x9134) */
	112	XELP_GT_SLICE_ENABLE, /* _MMIO(0x9138) */
	113	XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	114	GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	115	CTC_MODE, /* _MMIO(0xa26c) */
	116	HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	117	TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	118	};
	119
	120	static const struct xe_reg ats_m_runtime_regs[] = {
	121	RPM_CONFIG0, /* _MMIO(0x0d00) */
	122	MIRROR_FUSE3, /* _MMIO(0x9118) */
	123	MIRROR_FUSE1, /* _MMIO(0x911c) */
	124	XELP_EU_ENABLE, /* _MMIO(0x9134) */
	125	XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	126	GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	127	XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
	128	CTC_MODE, /* _MMIO(0xa26c) */
	129	HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	130	TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	131	};
	132
	133	static const struct xe_reg pvc_runtime_regs[] = {
	134	RPM_CONFIG0, /* _MMIO(0x0d00) */
	135	MIRROR_FUSE3, /* _MMIO(0x9118) */
	136	XELP_EU_ENABLE, /* _MMIO(0x9134) */
	137	XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	138	GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	139	XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
	140	XEHPC_GT_COMPUTE_DSS_ENABLE_EXT,/* _MMIO(0x9148) */
	141	CTC_MODE, /* _MMIO(0xA26C) */
	142	HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	143	TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	144	};
	145
	146	static const struct xe_reg ver_1270_runtime_regs[] = {
	147	RPM_CONFIG0, /* _MMIO(0x0d00) */
	148	XEHP_FUSE4, /* _MMIO(0x9114) */
	149	MIRROR_FUSE3, /* _MMIO(0x9118) */
	150	MIRROR_FUSE1, /* _MMIO(0x911c) */
	151	XELP_EU_ENABLE, /* _MMIO(0x9134) */
	152	XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	153	GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	154	XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
	155	XEHPC_GT_COMPUTE_DSS_ENABLE_EXT,/* _MMIO(0x9148) */
	156	CTC_MODE, /* _MMIO(0xa26c) */
	157	HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	158	TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	159	};
	160
	161	static const struct xe_reg ver_2000_runtime_regs[] = {
	162	RPM_CONFIG0, /* _MMIO(0x0d00) */
	163	XEHP_FUSE4, /* _MMIO(0x9114) */
	164	MIRROR_FUSE3, /* _MMIO(0x9118) */
	165	MIRROR_FUSE1, /* _MMIO(0x911c) */
	166	XELP_EU_ENABLE, /* _MMIO(0x9134) */
	167	XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	168	GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	169	XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
	170	XEHPC_GT_COMPUTE_DSS_ENABLE_EXT,/* _MMIO(0x9148) */
	171	XE2_GT_COMPUTE_DSS_2, /* _MMIO(0x914c) */
	172	XE2_GT_GEOMETRY_DSS_1, /* _MMIO(0x9150) */
	173	XE2_GT_GEOMETRY_DSS_2, /* _MMIO(0x9154) */
	174	CTC_MODE, /* _MMIO(0xa26c) */
	175	HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	176	TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	177	};
	178
	179	static const struct xe_reg pick_runtime_regs(struct xe_device xe, unsigned int *count)
	180	{
	181	const struct xe_reg *regs;
	182
	183	if (GRAPHICS_VERx100(xe) >= 2000) {
	184	*count = ARRAY_SIZE(ver_2000_runtime_regs);
	185	regs = ver_2000_runtime_regs;
	186	} else if (GRAPHICS_VERx100(xe) >= 1270) {
	187	*count = ARRAY_SIZE(ver_1270_runtime_regs);
	188	regs = ver_1270_runtime_regs;
	189	} else if (GRAPHICS_VERx100(xe) == 1260) {
	190	*count = ARRAY_SIZE(pvc_runtime_regs);
	191	regs = pvc_runtime_regs;
	192	} else if (GRAPHICS_VERx100(xe) == 1255) {
	193	*count = ARRAY_SIZE(ats_m_runtime_regs);
	194	regs = ats_m_runtime_regs;
	195	} else if (GRAPHICS_VERx100(xe) == 1200) {
	196	*count = ARRAY_SIZE(tgl_runtime_regs);
	197	regs = tgl_runtime_regs;
	198	} else {
	199	regs = ERR_PTR(-ENOPKG);
	200	*count = 0;
	201	}
	202
	203	return regs;
	204	}
	205
	206	static int pf_alloc_runtime_info(struct xe_gt *gt)
	207	{
	208	struct xe_device *xe = gt_to_xe(gt);
	209	const struct xe_reg *regs;
	210	unsigned int size;
	211	u32 *values;
	212
	213	xe_gt_assert(gt, IS_SRIOV_PF(xe));
	214	xe_gt_assert(gt, !gt->sriov.pf.service.runtime.size);
	215	xe_gt_assert(gt, !gt->sriov.pf.service.runtime.regs);
	216	xe_gt_assert(gt, !gt->sriov.pf.service.runtime.values);
	217
	218	regs = pick_runtime_regs(xe, &size);
	219	if (IS_ERR(regs))
	220	return PTR_ERR(regs);
	221
	222	if (unlikely(!size))
	223	return 0;
	224
	225	values = drmm_kcalloc(&xe->drm, size, sizeof(u32), GFP_KERNEL);
	226	if (!values)
	227	return -ENOMEM;
	228
	229	gt->sriov.pf.service.runtime.size = size;
	230	gt->sriov.pf.service.runtime.regs = regs;
	231	gt->sriov.pf.service.runtime.values = values;
	232
	233	return 0;
	234	}
	235
	236	static void read_many(struct xe_gt *gt, unsigned int count,
	237	const struct xe_reg regs, u32 values)
	238	{
	239	while (count--)
	240	values++ = xe_mmio_read32(gt, regs++);
	241	}
	242
	243	static void pf_prepare_runtime_info(struct xe_gt *gt)
	244	{
	245	const struct xe_reg *regs;
	246	unsigned int size;
	247	u32 *values;
	248
	249	if (!gt->sriov.pf.service.runtime.size)
	250	return;
	251
	252	size = gt->sriov.pf.service.runtime.size;
	253	regs = gt->sriov.pf.service.runtime.regs;
	254	values = gt->sriov.pf.service.runtime.values;
	255
	256	read_many(gt, size, regs, values);
	257
	258	if (IS_ENABLED(CONFIG_DRM_XE_DEBUG_SRIOV)) {
	259	struct drm_printer p = xe_gt_info_printer(gt);
	260
	261	xe_gt_sriov_pf_service_print_runtime(gt, &p);
	262	}
	263	}
	264
	265	/**
	266	* xe_gt_sriov_pf_service_init - Early initialization of the GT SR-IOV PF services.
	267	* @gt: the &xe_gt to initialize
	268	*
	269	* Performs early initialization of the GT SR-IOV PF services, including preparation
	270	* of the runtime info that will be shared with VFs.
	271	*
	272	* This function can only be called on PF.
	273	*/
	274	int xe_gt_sriov_pf_service_init(struct xe_gt *gt)
	275	{
	276	int err;
	277
	278	pf_init_versions(gt);
	279
	280	err = pf_alloc_runtime_info(gt);
	281	if (unlikely(err))
	282	goto failed;
	283
	284	return 0;
	285	failed:
	286	xe_gt_sriov_err(gt, "Failed to initialize service (%pe)\n", ERR_PTR(err));
	287	return err;
	288	}
	289
	290	/**
	291	* xe_gt_sriov_pf_service_update - Update PF SR-IOV services.
	292	* @gt: the &xe_gt to update
	293	*
	294	* Updates runtime data shared with VFs.
	295	*
	296	* This function can be called more than once.
	297	* This function can only be called on PF.
	298	*/
	299	void xe_gt_sriov_pf_service_update(struct xe_gt *gt)
	300	{
	301	pf_prepare_runtime_info(gt);
	302	}
	303
	304	/**
	305	* xe_gt_sriov_pf_service_reset - Reset a connection with the VF.
	306	* @gt: the &xe_gt
	307	* @vfid: the VF identifier
	308	*
	309	* Reset a VF driver negotiated VF/PF ABI version.
	310	* After that point, the VF driver will have to perform new version handshake
	311	* to continue use of the PF services again.
	312	*
	313	* This function can only be called on PF.
	314	*/
	315	void xe_gt_sriov_pf_service_reset(struct xe_gt *gt, unsigned int vfid)
	316	{
	317	pf_disconnect(gt, vfid);
	318	}
	319
	320	/* Return: 0 on success or a negative error code on failure. */
	321	static int pf_process_handshake(struct xe_gt *gt, u32 vfid,
	322	u32 wanted_major, u32 wanted_minor,
	323	u32 major, u32 minor)
	324	{
	325	int err;
	326
	327	xe_gt_sriov_dbg_verbose(gt, "VF%u wants ABI version %u.%u\n",
	328	vfid, wanted_major, wanted_minor);
	329
	330	err = pf_negotiate_version(gt, wanted_major, wanted_minor, major, minor);
	331
	332	if (err < 0) {
	333	xe_gt_sriov_notice(gt, "VF%u failed to negotiate ABI %u.%u (%pe)\n",
	334	vfid, wanted_major, wanted_minor, ERR_PTR(err));
	335	pf_disconnect(gt, vfid);
	336	} else {
	337	xe_gt_sriov_dbg(gt, "VF%u negotiated ABI version %u.%u\n",
	338	vfid, major, minor);
	339	pf_connect(gt, vfid, major, minor);
	340	}
	341
	342	return 0;
	343	}
	344
	345	/* Return: length of the response message or a negative error code on failure. */
	346	static int pf_process_handshake_msg(struct xe_gt *gt, u32 origin,
	347	const u32 request, u32 len, u32 response, u32 size)
	348	{
	349	u32 wanted_major, wanted_minor;
	350	u32 major, minor;
	351	u32 mbz;
	352	int err;
	353
	354	if (unlikely(len != VF2PF_HANDSHAKE_REQUEST_MSG_LEN))
	355	return -EMSGSIZE;
	356
	357	mbz = FIELD_GET(VF2PF_HANDSHAKE_REQUEST_MSG_0_MBZ, request[0]);
	358	if (unlikely(mbz))
	359	return -EPFNOSUPPORT;
	360
	361	wanted_major = FIELD_GET(VF2PF_HANDSHAKE_REQUEST_MSG_1_MAJOR, request[1]);
	362	wanted_minor = FIELD_GET(VF2PF_HANDSHAKE_REQUEST_MSG_1_MINOR, request[1]);
	363
	364	err = pf_process_handshake(gt, origin, wanted_major, wanted_minor, &major, &minor);
	365	if (err < 0)
	366	return err;
	367
	368	xe_gt_assert(gt, major \|\| minor);
	369	xe_gt_assert(gt, size >= VF2PF_HANDSHAKE_RESPONSE_MSG_LEN);
	370
	371	response[0] = FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	372	FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_RESPONSE_SUCCESS) \|
	373	FIELD_PREP(GUC_HXG_RESPONSE_MSG_0_DATA0, 0);
	374	response[1] = FIELD_PREP(VF2PF_HANDSHAKE_RESPONSE_MSG_1_MAJOR, major) \|
	375	FIELD_PREP(VF2PF_HANDSHAKE_RESPONSE_MSG_1_MINOR, minor);
	376
	377	return VF2PF_HANDSHAKE_RESPONSE_MSG_LEN;
	378	}
	379
	380	struct reg_data {
	381	u32 offset;
	382	u32 value;
	383	} __packed;
	384	static_assert(hxg_sizeof(struct reg_data) == 2);
	385
	386	/* Return: number of entries copied or negative error code on failure. */
	387	static int pf_service_runtime_query(struct xe_gt *gt, u32 start, u32 limit,
	388	struct reg_data data, u32 remaining)
	389	{
	390	struct xe_gt_sriov_pf_service_runtime_regs *runtime;
	391	unsigned int count, i;
	392	u32 addr;
	393
	394	xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	395
	396	runtime = &gt->sriov.pf.service.runtime;
	397
	398	if (start > runtime->size)
	399	return -ERANGE;
	400
	401	count = min_t(u32, runtime->size - start, limit);
	402
	403	for (i = 0; i < count; ++i, ++data) {
	404	addr = runtime->regs[start + i].addr;
	405	data->offset = xe_mmio_adjusted_addr(gt, addr);
	406	data->value = runtime->values[start + i];
	407	}
	408
	409	*remaining = runtime->size - start - count;
	410	return count;
	411	}
	412
	413	/* Return: length of the response message or a negative error code on failure. */
	414	static int pf_process_runtime_query_msg(struct xe_gt *gt, u32 origin,
	415	const u32 msg, u32 msg_len, u32 response, u32 resp_size)
	416	{
	417	const u32 chunk_size = hxg_sizeof(struct reg_data);
	418	struct reg_data *reg_data_buf;
	419	u32 limit, start, max_chunks;
	420	u32 remaining = 0;
	421	int ret;
	422
	423	if (!pf_is_negotiated(gt, origin, 1, 0))
	424	return -EACCES;
	425	if (unlikely(msg_len > VF2PF_QUERY_RUNTIME_REQUEST_MSG_LEN))
	426	return -EMSGSIZE;
	427	if (unlikely(msg_len < VF2PF_QUERY_RUNTIME_REQUEST_MSG_LEN))
	428	return -EPROTO;
	429	if (unlikely(resp_size < VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN))
	430	return -EINVAL;
	431
	432	limit = FIELD_GET(VF2PF_QUERY_RUNTIME_REQUEST_MSG_0_LIMIT, msg[0]);
	433	start = FIELD_GET(VF2PF_QUERY_RUNTIME_REQUEST_MSG_1_START, msg[1]);
	434
	435	resp_size = min_t(u32, resp_size, VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MAX_LEN);
	436	max_chunks = (resp_size - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) / chunk_size;
	437	limit = limit == VF2PF_QUERY_RUNTIME_NO_LIMIT ? max_chunks : min_t(u32, max_chunks, limit);
	438	reg_data_buf = (void *)(response + VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN);
	439
	440	ret = pf_service_runtime_query(gt, start, limit, reg_data_buf, &remaining);
	441	if (ret < 0)
	442	return ret;
	443
	444	response[0] = FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	445	FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_RESPONSE_SUCCESS) \|
	446	FIELD_PREP(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_0_COUNT, ret);
	447	response[1] = FIELD_PREP(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_1_REMAINING, remaining);
	448
	449	return VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + ret * hxg_sizeof(struct reg_data);
	450	}
	451
	452	/**
	453	* xe_gt_sriov_pf_service_process_request - Service GT level SR-IOV request message from the VF.
	454	* @gt: the &xe_gt that provides the service
	455	* @origin: VF number that is requesting the service
	456	* @msg: request message
	457	* @msg_len: length of the request message (in dwords)
	458	* @response: placeholder for the response message
	459	* @resp_size: length of the response message buffer (in dwords)
	460	*
	461	* This function processes `Relay Message`_ request from the VF.
	462	*
	463	* Return: length of the response message or a negative error code on failure.
	464	*/
	465	int xe_gt_sriov_pf_service_process_request(struct xe_gt *gt, u32 origin,
	466	const u32 *msg, u32 msg_len,
	467	u32 *response, u32 resp_size)
	468	{
	469	u32 action, data __maybe_unused;
	470	int ret;
	471
	472	xe_gt_assert(gt, msg_len >= GUC_HXG_MSG_MIN_LEN);
	473	xe_gt_assert(gt, FIELD_GET(GUC_HXG_MSG_0_TYPE, msg[0]) == GUC_HXG_TYPE_REQUEST);
	474
	475	action = FIELD_GET(GUC_HXG_REQUEST_MSG_0_ACTION, msg[0]);
	476	data = FIELD_GET(GUC_HXG_REQUEST_MSG_0_DATA0, msg[0]);
	477	xe_gt_sriov_dbg_verbose(gt, "service action %#x:%u from VF%u\n",
	478	action, data, origin);
	479
	480	switch (action) {
	481	case GUC_RELAY_ACTION_VF2PF_HANDSHAKE:
	482	ret = pf_process_handshake_msg(gt, origin, msg, msg_len, response, resp_size);
	483	break;
	484	case GUC_RELAY_ACTION_VF2PF_QUERY_RUNTIME:
	485	ret = pf_process_runtime_query_msg(gt, origin, msg, msg_len, response, resp_size);
	486	break;
	487	default:
	488	ret = -EOPNOTSUPP;
	489	break;
	490	}
	491
	492	return ret;
	493	}
	494
	495	/**
	496	* xe_gt_sriov_pf_service_print_runtime - Print PF runtime data shared with VFs.
	497	* @gt: the &xe_gt
	498	* @p: the &drm_printer
	499	*
	500	* This function is for PF use only.
	501	*/
	502	int xe_gt_sriov_pf_service_print_runtime(struct xe_gt gt, struct drm_printer p)
	503	{
	504	const struct xe_reg *regs;
	505	unsigned int size;
	506	u32 *values;
	507
	508	xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	509
	510	size = gt->sriov.pf.service.runtime.size;
	511	regs = gt->sriov.pf.service.runtime.regs;
	512	values = gt->sriov.pf.service.runtime.values;
	513
	514	for (; size--; regs++, values++) {
	515	drm_printf(p, "reg[%#x] = %#x\n",
	516	xe_mmio_adjusted_addr(gt, regs->addr), *values);
	517	}
	518
	519	return 0;
	520	}
	521
	522	/**
	523	* xe_gt_sriov_pf_service_print_version - Print ABI versions negotiated with VFs.
	524	* @gt: the &xe_gt
	525	* @p: the &drm_printer
	526	*
	527	* This function is for PF use only.
	528	*/
	529	int xe_gt_sriov_pf_service_print_version(struct xe_gt gt, struct drm_printer p)
	530	{
	531	struct xe_device *xe = gt_to_xe(gt);
	532	unsigned int n, total_vfs = xe_sriov_pf_get_totalvfs(xe);
	533	struct xe_gt_sriov_pf_service_version *version;
	534
	535	xe_gt_assert(gt, IS_SRIOV_PF(xe));
	536
	537	for (n = 1; n <= total_vfs; n++) {
	538	version = &gt->sriov.pf.vfs[n].version;
	539	if (!version->major && !version->minor)
	540	continue;
	541
	542	drm_printf(p, "VF%u:\t%u.%u\n", n, version->major, version->minor);
	543	}
	544
	545	return 0;
	546	}
	547
	548	#if IS_BUILTIN(CONFIG_DRM_XE_KUNIT_TEST)
	549	#include "tests/xe_gt_sriov_pf_service_test.c"
	550	#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_service.h+36−0

@@ -0,0 +1,36 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_SRIOV_PF_SERVICE_H_
	7	#define _XE_GT_SRIOV_PF_SERVICE_H_
	8
	9	#include <linux/errno.h>
	10	#include <linux/types.h>
	11
	12	struct drm_printer;
	13	struct xe_gt;
	14
	15	int xe_gt_sriov_pf_service_init(struct xe_gt *gt);
	16	void xe_gt_sriov_pf_service_update(struct xe_gt *gt);
	17	void xe_gt_sriov_pf_service_reset(struct xe_gt *gt, unsigned int vfid);
	18
	19	int xe_gt_sriov_pf_service_print_version(struct xe_gt gt, struct drm_printer p);
	20	int xe_gt_sriov_pf_service_print_runtime(struct xe_gt gt, struct drm_printer p);
	21
	22	#ifdef CONFIG_PCI_IOV
	23	int xe_gt_sriov_pf_service_process_request(struct xe_gt *gt, u32 origin,
	24	const u32 *msg, u32 msg_len,
	25	u32 *response, u32 resp_size);
	26	#else
	27	static inline int
	28	xe_gt_sriov_pf_service_process_request(struct xe_gt *gt, u32 origin,
	29	const u32 *msg, u32 msg_len,
	30	u32 *response, u32 resp_size)
	31	{
	32	return -EPROTO;
	33	}
	34	#endif
	35
	36	#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_service_types.h+52−0

@@ -0,0 +1,52 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_SRIOV_PF_SERVICE_TYPES_H_
	7	#define _XE_GT_SRIOV_PF_SERVICE_TYPES_H_
	8
	9	#include <linux/types.h>
	10
	11	struct xe_reg;
	12
	13	/**
	14	* struct xe_gt_sriov_pf_service_version - VF/PF ABI Version.
	15	* @major: the major version of the VF/PF ABI
	16	* @minor: the minor version of the VF/PF ABI
	17	*
	18	* See `GuC Relay Communication`_.
	19	*/
	20	struct xe_gt_sriov_pf_service_version {
	21	u16 major;
	22	u16 minor;
	23	};
	24
	25	/**
	26	* struct xe_gt_sriov_pf_service_runtime_regs - Runtime data shared with VFs.
	27	* @regs: pointer to static array with register offsets.
	28	* @values: pointer to array with captured register values.
	29	* @size: size of the regs and value arrays.
	30	*/
	31	struct xe_gt_sriov_pf_service_runtime_regs {
	32	const struct xe_reg *regs;
	33	u32 *values;
	34	u32 size;
	35	};
	36
	37	/**
	38	* struct xe_gt_sriov_pf_service - Data used by the PF service.
	39	* @version: information about VF/PF ABI versions for current platform.
	40	* @version.base: lowest VF/PF ABI version that could be negotiated with VF.
	41	* @version.latest: latest VF/PF ABI version supported by the PF driver.
	42	* @runtime: runtime data shared with VFs.
	43	*/
	44	struct xe_gt_sriov_pf_service {
	45	struct {
	46	struct xe_gt_sriov_pf_service_version base;
	47	struct xe_gt_sriov_pf_service_version latest;
	48	} version;
	49	struct xe_gt_sriov_pf_service_runtime_regs runtime;
	50	};
	51
	52	#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_types.h+10−0

 #include <linux/types.h>
 #include "xe_gt_sriov_pf_config_types.h"
+#include "xe_gt_sriov_pf_monitor_types.h"
 #include "xe_gt_sriov_pf_policy_types.h"
+#include "xe_gt_sriov_pf_service_types.h"
 /**
  * struct xe_gt_sriov_metadata - GT level per-VF metadata.
 struct xe_gt_sriov_metadata {
 	/** @config: per-VF provisioning data. */
 	struct xe_gt_sriov_config config;
+	/** @monitor: per-VF monitoring data. */
+	struct xe_gt_sriov_monitor monitor;
+	/** @version: negotiated VF/PF ABI version */
+	struct xe_gt_sriov_pf_service_version version;
 };
 /**
  * struct xe_gt_sriov_pf - GT level PF virtualization data.
+ * @service: service data.
  * @policy: policy data.
  * @spare: PF-only provisioning configuration.
  * @vfs: metadata for all VFs.
  */
 struct xe_gt_sriov_pf {
+	struct xe_gt_sriov_pf_service service;
 	struct xe_gt_sriov_pf_policy policy;
 	struct xe_gt_sriov_spare_config spare;
 	struct xe_gt_sriov_metadata *vfs;

drivers/gpu/drm/xe/xe_gt_sriov_vf.c+979−0

@@ -0,0 +1,979 @@
	1	// SPDX-License-Identifier: MIT
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#include <linux/bitfield.h>
	7	#include <linux/bsearch.h>
	8
	9	#include <drm/drm_managed.h>
	10	#include <drm/drm_print.h>
	11
	12	#include "abi/guc_actions_sriov_abi.h"
	13	#include "abi/guc_communication_mmio_abi.h"
	14	#include "abi/guc_klvs_abi.h"
	15	#include "abi/guc_relay_actions_abi.h"
	16	#include "regs/xe_gt_regs.h"
	17	#include "regs/xe_gtt_defs.h"
	18
	19	#include "xe_assert.h"
	20	#include "xe_device.h"
	21	#include "xe_ggtt.h"
	22	#include "xe_gt_sriov_printk.h"
	23	#include "xe_gt_sriov_vf.h"
	24	#include "xe_gt_sriov_vf_types.h"
	25	#include "xe_guc.h"
	26	#include "xe_guc_hxg_helpers.h"
	27	#include "xe_guc_relay.h"
	28	#include "xe_mmio.h"
	29	#include "xe_sriov.h"
	30	#include "xe_uc_fw.h"
	31	#include "xe_wopcm.h"
	32
	33	#define make_u64_from_u32(hi, lo) ((u64)((u64)(u32)(hi) << 32 \| (u32)(lo)))
	34
	35	static int guc_action_vf_reset(struct xe_guc *guc)
	36	{
	37	u32 request[GUC_HXG_REQUEST_MSG_MIN_LEN] = {
	38	FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	39	FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	40	FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION, GUC_ACTION_VF2GUC_VF_RESET),
	41	};
	42	int ret;
	43
	44	ret = xe_guc_mmio_send(guc, request, ARRAY_SIZE(request));
	45
	46	return ret > 0 ? -EPROTO : ret;
	47	}
	48
	49	static int vf_reset_guc_state(struct xe_gt *gt)
	50	{
	51	struct xe_guc *guc = &gt->uc.guc;
	52	int err;
	53
	54	err = guc_action_vf_reset(guc);
	55	if (unlikely(err))
	56	xe_gt_sriov_err(gt, "Failed to reset GuC state (%pe)\n", ERR_PTR(err));
	57	return err;
	58	}
	59
	60	static int guc_action_match_version(struct xe_guc *guc,
	61	u32 wanted_branch, u32 wanted_major, u32 wanted_minor,
	62	u32 branch, u32 major, u32 minor, u32 patch)
	63	{
	64	u32 request[VF2GUC_MATCH_VERSION_REQUEST_MSG_LEN] = {
	65	FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	66	FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	67	FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION,
	68	GUC_ACTION_VF2GUC_MATCH_VERSION),
	69	FIELD_PREP(VF2GUC_MATCH_VERSION_REQUEST_MSG_1_BRANCH, wanted_branch) \|
	70	FIELD_PREP(VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MAJOR, wanted_major) \|
	71	FIELD_PREP(VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MINOR, wanted_minor),
	72	};
	73	u32 response[GUC_MAX_MMIO_MSG_LEN];
	74	int ret;
	75
	76	BUILD_BUG_ON(VF2GUC_MATCH_VERSION_RESPONSE_MSG_LEN > GUC_MAX_MMIO_MSG_LEN);
	77
	78	ret = xe_guc_mmio_send_recv(guc, request, ARRAY_SIZE(request), response);
	79	if (unlikely(ret < 0))
	80	return ret;
	81
	82	if (unlikely(FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_0_MBZ, response[0])))
	83	return -EPROTO;
	84
	85	*branch = FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_BRANCH, response[1]);
	86	*major = FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_MAJOR, response[1]);
	87	*minor = FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_MINOR, response[1]);
	88	*patch = FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_PATCH, response[1]);
	89
	90	return 0;
	91	}
	92
	93	static void vf_minimum_guc_version(struct xe_gt gt, u32 branch, u32 major, u32 minor)
	94	{
	95	struct xe_device *xe = gt_to_xe(gt);
	96
	97	switch (xe->info.platform) {
	98	case XE_TIGERLAKE ... XE_PVC:
	99	/* 1.1 this is current baseline for Xe driver */
	100	*branch = 0;
	101	*major = 1;
	102	*minor = 1;
	103	break;
	104	default:
	105	/* 1.2 has support for the GMD_ID KLV */
	106	*branch = 0;
	107	*major = 1;
	108	*minor = 2;
	109	break;
	110	}
	111	}
	112
	113	static void vf_wanted_guc_version(struct xe_gt gt, u32 branch, u32 major, u32 minor)
	114	{
	115	/* for now it's the same as minimum */
	116	return vf_minimum_guc_version(gt, branch, major, minor);
	117	}
	118
	119	static int vf_handshake_with_guc(struct xe_gt *gt)
	120	{
	121	struct xe_gt_sriov_vf_guc_version *guc_version = &gt->sriov.vf.guc_version;
	122	struct xe_guc *guc = &gt->uc.guc;
	123	u32 wanted_branch, wanted_major, wanted_minor;
	124	u32 branch, major, minor, patch;
	125	int err;
	126
	127	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	128
	129	/* select wanted version - prefer previous (if any) */
	130	if (guc_version->major \|\| guc_version->minor) {
	131	wanted_branch = guc_version->branch;
	132	wanted_major = guc_version->major;
	133	wanted_minor = guc_version->minor;
	134	} else {
	135	vf_wanted_guc_version(gt, &wanted_branch, &wanted_major, &wanted_minor);
	136	xe_gt_assert(gt, wanted_major != GUC_VERSION_MAJOR_ANY);
	137	}
	138
	139	err = guc_action_match_version(guc, wanted_branch, wanted_major, wanted_minor,
	140	&branch, &major, &minor, &patch);
	141	if (unlikely(err))
	142	goto fail;
	143
	144	/* we don't support interface version change */
	145	if ((guc_version->major \|\| guc_version->minor) &&
	146	(guc_version->branch != branch \|\| guc_version->major != major \|\|
	147	guc_version->minor != minor)) {
	148	xe_gt_sriov_err(gt, "New GuC interface version detected: %u.%u.%u.%u\n",
	149	branch, major, minor, patch);
	150	xe_gt_sriov_info(gt, "Previously used version was: %u.%u.%u.%u\n",
	151	guc_version->branch, guc_version->major,
	152	guc_version->minor, guc_version->patch);
	153	err = -EREMCHG;
	154	goto fail;
	155	}
	156
	157	/* illegal */
	158	if (major > wanted_major) {
	159	err = -EPROTO;
	160	goto unsupported;
	161	}
	162
	163	/* there's no fallback on major version. */
	164	if (major != wanted_major) {
	165	err = -ENOPKG;
	166	goto unsupported;
	167	}
	168
	169	/* check against minimum version supported by us */
	170	vf_minimum_guc_version(gt, &wanted_branch, &wanted_major, &wanted_minor);
	171	xe_gt_assert(gt, major != GUC_VERSION_MAJOR_ANY);
	172	if (major < wanted_major \|\| (major == wanted_major && minor < wanted_minor)) {
	173	err = -ENOKEY;
	174	goto unsupported;
	175	}
	176
	177	xe_gt_sriov_dbg(gt, "using GuC interface version %u.%u.%u.%u\n",
	178	branch, major, minor, patch);
	179
	180	guc_version->branch = branch;
	181	guc_version->major = major;
	182	guc_version->minor = minor;
	183	guc_version->patch = patch;
	184	return 0;
	185
	186	unsupported:
	187	xe_gt_sriov_err(gt, "Unsupported GuC version %u.%u.%u.%u (%pe)\n",
	188	branch, major, minor, patch, ERR_PTR(err));
	189	fail:
	190	xe_gt_sriov_err(gt, "Unable to confirm GuC version %u.%u (%pe)\n",
	191	wanted_major, wanted_minor, ERR_PTR(err));
	192
	193	/* try again with any just to query which version is supported */
	194	if (!guc_action_match_version(guc, GUC_VERSION_BRANCH_ANY,
	195	GUC_VERSION_MAJOR_ANY, GUC_VERSION_MINOR_ANY,
	196	&branch, &major, &minor, &patch))
	197	xe_gt_sriov_notice(gt, "GuC reports interface version %u.%u.%u.%u\n",
	198	branch, major, minor, patch);
	199	return err;
	200	}
	201
	202	/**
	203	* xe_gt_sriov_vf_bootstrap - Query and setup GuC ABI interface version.
	204	* @gt: the &xe_gt
	205	*
	206	* This function is for VF use only.
	207	* It requires functional `GuC MMIO based communication`_.
	208	*
	209	* Return: 0 on success or a negative error code on failure.
	210	*/
	211	int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt)
	212	{
	213	int err;
	214
	215	err = vf_reset_guc_state(gt);
	216	if (unlikely(err))
	217	return err;
	218
	219	err = vf_handshake_with_guc(gt);
	220	if (unlikely(err))
	221	return err;
	222
	223	return 0;
	224	}
	225
	226	static int guc_action_query_single_klv(struct xe_guc *guc, u32 key,
	227	u32 *value, u32 value_len)
	228	{
	229	u32 request[VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_LEN] = {
	230	FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	231	FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	232	FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION,
	233	GUC_ACTION_VF2GUC_QUERY_SINGLE_KLV),
	234	FIELD_PREP(VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_1_KEY, key),
	235	};
	236	u32 response[GUC_MAX_MMIO_MSG_LEN];
	237	u32 length;
	238	int ret;
	239
	240	BUILD_BUG_ON(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_MAX_LEN > GUC_MAX_MMIO_MSG_LEN);
	241	ret = xe_guc_mmio_send_recv(guc, request, ARRAY_SIZE(request), response);
	242	if (unlikely(ret < 0))
	243	return ret;
	244
	245	if (unlikely(FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_0_MBZ, response[0])))
	246	return -EPROTO;
	247
	248	length = FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_0_LENGTH, response[0]);
	249	if (unlikely(length > value_len))
	250	return -EOVERFLOW;
	251	if (unlikely(length < value_len))
	252	return -ENODATA;
	253
	254	switch (value_len) {
	255	default:
	256	xe_gt_WARN_ON(guc_to_gt(guc), value_len > 3);
	257	fallthrough;
	258	case 3:
	259	value[2] = FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_3_VALUE96, response[3]);
	260	fallthrough;
	261	case 2:
	262	value[1] = FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_2_VALUE64, response[2]);
	263	fallthrough;
	264	case 1:
	265	value[0] = FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_1_VALUE32, response[1]);
	266	fallthrough;
	267	case 0:
	268	break;
	269	}
	270
	271	return 0;
	272	}
	273
	274	static int guc_action_query_single_klv32(struct xe_guc guc, u32 key, u32 value32)
	275	{
	276	return guc_action_query_single_klv(guc, key, value32, hxg_sizeof(u32));
	277	}
	278
	279	static int guc_action_query_single_klv64(struct xe_guc guc, u32 key, u64 value64)
	280	{
	281	u32 value[2];
	282	int err;
	283
	284	err = guc_action_query_single_klv(guc, key, value, hxg_sizeof(value));
	285	if (unlikely(err))
	286	return err;
	287
	288	*value64 = make_u64_from_u32(value[1], value[0]);
	289	return 0;
	290	}
	291
	292	static bool has_gmdid(struct xe_device *xe)
	293	{
	294	return GRAPHICS_VERx100(xe) >= 1270;
	295	}
	296
	297	/**
	298	* xe_gt_sriov_vf_gmdid - Query GMDID over MMIO.
	299	* @gt: the &xe_gt
	300	*
	301	* This function is for VF use only.
	302	*
	303	* Return: value of GMDID KLV on success or 0 on failure.
	304	*/
	305	u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt)
	306	{
	307	const char *type = xe_gt_is_media_type(gt) ? "media" : "graphics";
	308	struct xe_guc *guc = &gt->uc.guc;
	309	u32 value;
	310	int err;
	311
	312	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	313	xe_gt_assert(gt, !GRAPHICS_VERx100(gt_to_xe(gt)) \|\| has_gmdid(gt_to_xe(gt)));
	314	xe_gt_assert(gt, gt->sriov.vf.guc_version.major > 1 \|\| gt->sriov.vf.guc_version.minor >= 2);
	315
	316	err = guc_action_query_single_klv32(guc, GUC_KLV_GLOBAL_CFG_GMD_ID_KEY, &value);
	317	if (unlikely(err)) {
	318	xe_gt_sriov_err(gt, "Failed to obtain %s GMDID (%pe)\n",
	319	type, ERR_PTR(err));
	320	return 0;
	321	}
	322
	323	xe_gt_sriov_dbg(gt, "%s GMDID = %#x\n", type, value);
	324	return value;
	325	}
	326
	327	static int vf_get_ggtt_info(struct xe_gt *gt)
	328	{
	329	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	330	struct xe_guc *guc = &gt->uc.guc;
	331	u64 start, size;
	332	int err;
	333
	334	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	335
	336	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
	337	if (unlikely(err))
	338	return err;
	339
	340	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_SIZE_KEY, &size);
	341	if (unlikely(err))
	342	return err;
	343
	344	if (config->ggtt_size && config->ggtt_size != size) {
	345	xe_gt_sriov_err(gt, "Unexpected GGTT reassignment: %lluK != %lluK\n",
	346	size / SZ_1K, config->ggtt_size / SZ_1K);
	347	return -EREMCHG;
	348	}
	349
	350	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
	351	start, start + size - 1, size / SZ_1K);
	352
	353	config->ggtt_base = start;
	354	config->ggtt_size = size;
	355
	356	return config->ggtt_size ? 0 : -ENODATA;
	357	}
	358
	359	static int vf_get_lmem_info(struct xe_gt *gt)
	360	{
	361	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	362	struct xe_guc *guc = &gt->uc.guc;
	363	char size_str[10];
	364	u64 size;
	365	int err;
	366
	367	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	368
	369	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
	370	if (unlikely(err))
	371	return err;
	372
	373	if (config->lmem_size && config->lmem_size != size) {
	374	xe_gt_sriov_err(gt, "Unexpected LMEM reassignment: %lluM != %lluM\n",
	375	size / SZ_1M, config->lmem_size / SZ_1M);
	376	return -EREMCHG;
	377	}
	378
	379	string_get_size(size, 1, STRING_UNITS_2, size_str, sizeof(size_str));
	380	xe_gt_sriov_dbg_verbose(gt, "LMEM %lluM %s\n", size / SZ_1M, size_str);
	381
	382	config->lmem_size = size;
	383
	384	return config->lmem_size ? 0 : -ENODATA;
	385	}
	386
	387	static int vf_get_submission_cfg(struct xe_gt *gt)
	388	{
	389	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	390	struct xe_guc *guc = &gt->uc.guc;
	391	u32 num_ctxs, num_dbs;
	392	int err;
	393
	394	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	395
	396	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
	397	if (unlikely(err))
	398	return err;
	399
	400	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_DOORBELLS_KEY, &num_dbs);
	401	if (unlikely(err))
	402	return err;
	403
	404	if (config->num_ctxs && config->num_ctxs != num_ctxs) {
	405	xe_gt_sriov_err(gt, "Unexpected CTXs reassignment: %u != %u\n",
	406	num_ctxs, config->num_ctxs);
	407	return -EREMCHG;
	408	}
	409	if (config->num_dbs && config->num_dbs != num_dbs) {
	410	xe_gt_sriov_err(gt, "Unexpected DBs reassignment: %u != %u\n",
	411	num_dbs, config->num_dbs);
	412	return -EREMCHG;
	413	}
	414
	415	xe_gt_sriov_dbg_verbose(gt, "CTXs %u DBs %u\n", num_ctxs, num_dbs);
	416
	417	config->num_ctxs = num_ctxs;
	418	config->num_dbs = num_dbs;
	419
	420	return config->num_ctxs ? 0 : -ENODATA;
	421	}
	422
	423	static void vf_cache_gmdid(struct xe_gt *gt)
	424	{
	425	xe_gt_assert(gt, has_gmdid(gt_to_xe(gt)));
	426	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	427
	428	gt->sriov.vf.runtime.gmdid = xe_gt_sriov_vf_gmdid(gt);
	429	}
	430
	431	/**
	432	* xe_gt_sriov_vf_query_config - Query SR-IOV config data over MMIO.
	433	* @gt: the &xe_gt
	434	*
	435	* This function is for VF use only.
	436	*
	437	* Return: 0 on success or a negative error code on failure.
	438	*/
	439	int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
	440	{
	441	struct xe_device *xe = gt_to_xe(gt);
	442	int err;
	443
	444	err = vf_get_ggtt_info(gt);
	445	if (unlikely(err))
	446	return err;
	447
	448	if (IS_DGFX(xe) && !xe_gt_is_media_type(gt)) {
	449	err = vf_get_lmem_info(gt);
	450	if (unlikely(err))
	451	return err;
	452	}
	453
	454	err = vf_get_submission_cfg(gt);
	455	if (unlikely(err))
	456	return err;
	457
	458	if (has_gmdid(xe))
	459	vf_cache_gmdid(gt);
	460
	461	return 0;
	462	}
	463
	464	/**
	465	* xe_gt_sriov_vf_guc_ids - VF GuC context IDs configuration.
	466	* @gt: the &xe_gt
	467	*
	468	* This function is for VF use only.
	469	*
	470	* Return: number of GuC context IDs assigned to VF.
	471	*/
	472	u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
	473	{
	474	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	475	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
	476	xe_gt_assert(gt, gt->sriov.vf.self_config.num_ctxs);
	477
	478	return gt->sriov.vf.self_config.num_ctxs;
	479	}
	480
	481	/**
	482	* xe_gt_sriov_vf_lmem - VF LMEM configuration.
	483	* @gt: the &xe_gt
	484	*
	485	* This function is for VF use only.
	486	*
	487	* Return: size of the LMEM assigned to VF.
	488	*/
	489	u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
	490	{
	491	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	492	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
	493	xe_gt_assert(gt, gt->sriov.vf.self_config.lmem_size);
	494
	495	return gt->sriov.vf.self_config.lmem_size;
	496	}
	497
	498	static int vf_balloon_ggtt(struct xe_gt *gt)
	499	{
	500	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	501	struct xe_tile *tile = gt_to_tile(gt);
	502	struct xe_ggtt *ggtt = tile->mem.ggtt;
	503	struct xe_device *xe = gt_to_xe(gt);
	504	u64 start, end;
	505	int err;
	506
	507	xe_gt_assert(gt, IS_SRIOV_VF(xe));
	508	xe_gt_assert(gt, !xe_gt_is_media_type(gt));
	509
	510	if (!config->ggtt_size)
	511	return -ENODATA;
	512
	513	/*
	514	* VF can only use part of the GGTT as allocated by the PF:
	515	*
	516	* WOPCM GUC_GGTT_TOP
	517	* \|<------------ Total GGTT size ------------------>\|
	518	*
	519	* VF GGTT base -->\|<- size ->\|
	520	*
	521	* +--------------------+----------+-----------------+
	522	* \|////////////////////\| block \|\\\\\\\\\\\\\\\\\\|
	523	* +--------------------+----------+-----------------+
	524	*
	525	* \|<--- balloon[0] --->\|<-- VF -->\|<-- balloon[1] ->\|
	526	*/
	527
	528	start = xe_wopcm_size(xe);
	529	end = config->ggtt_base;
	530	if (end != start) {
	531	err = xe_ggtt_balloon(ggtt, start, end, &tile->sriov.vf.ggtt_balloon[0]);
	532	if (err)
	533	goto failed;
	534	}
	535
	536	start = config->ggtt_base + config->ggtt_size;
	537	end = GUC_GGTT_TOP;
	538	if (end != start) {
	539	err = xe_ggtt_balloon(ggtt, start, end, &tile->sriov.vf.ggtt_balloon[1]);
	540	if (err)
	541	goto deballoon;
	542	}
	543
	544	return 0;
	545
	546	deballoon:
	547	xe_ggtt_deballoon(ggtt, &tile->sriov.vf.ggtt_balloon[0]);
	548	failed:
	549	return err;
	550	}
	551
	552	static void deballoon_ggtt(struct drm_device drm, void arg)
	553	{
	554	struct xe_tile *tile = arg;
	555	struct xe_ggtt *ggtt = tile->mem.ggtt;
	556
	557	xe_tile_assert(tile, IS_SRIOV_VF(tile_to_xe(tile)));
	558	xe_ggtt_deballoon(ggtt, &tile->sriov.vf.ggtt_balloon[1]);
	559	xe_ggtt_deballoon(ggtt, &tile->sriov.vf.ggtt_balloon[0]);
	560	}
	561
	562	/**
	563	* xe_gt_sriov_vf_prepare_ggtt - Prepare a VF's GGTT configuration.
	564	* @gt: the &xe_gt
	565	*
	566	* This function is for VF use only.
	567	*
	568	* Return: 0 on success or a negative error code on failure.
	569	*/
	570	int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt)
	571	{
	572	struct xe_tile *tile = gt_to_tile(gt);
	573	struct xe_device *xe = tile_to_xe(tile);
	574	int err;
	575
	576	if (xe_gt_is_media_type(gt))
	577	return 0;
	578
	579	err = vf_balloon_ggtt(gt);
	580	if (err)
	581	return err;
	582
	583	return drmm_add_action_or_reset(&xe->drm, deballoon_ggtt, tile);
	584	}
	585
	586	static int relay_action_handshake(struct xe_gt gt, u32 major, u32 *minor)
	587	{
	588	u32 request[VF2PF_HANDSHAKE_REQUEST_MSG_LEN] = {
	589	FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	590	FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	591	FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION, GUC_RELAY_ACTION_VF2PF_HANDSHAKE),
	592	FIELD_PREP(VF2PF_HANDSHAKE_REQUEST_MSG_1_MAJOR, *major) \|
	593	FIELD_PREP(VF2PF_HANDSHAKE_REQUEST_MSG_1_MINOR, *minor),
	594	};
	595	u32 response[VF2PF_HANDSHAKE_RESPONSE_MSG_LEN];
	596	int ret;
	597
	598	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	599
	600	ret = xe_guc_relay_send_to_pf(&gt->uc.guc.relay,
	601	request, ARRAY_SIZE(request),
	602	response, ARRAY_SIZE(response));
	603	if (unlikely(ret < 0))
	604	return ret;
	605
	606	if (unlikely(ret != VF2PF_HANDSHAKE_RESPONSE_MSG_LEN))
	607	return -EPROTO;
	608
	609	if (unlikely(FIELD_GET(VF2PF_HANDSHAKE_RESPONSE_MSG_0_MBZ, response[0])))
	610	return -EPROTO;
	611
	612	*major = FIELD_GET(VF2PF_HANDSHAKE_RESPONSE_MSG_1_MAJOR, response[1]);
	613	*minor = FIELD_GET(VF2PF_HANDSHAKE_RESPONSE_MSG_1_MINOR, response[1]);
	614
	615	return 0;
	616	}
	617
	618	static void vf_connect_pf(struct xe_gt *gt, u16 major, u16 minor)
	619	{
	620	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	621
	622	gt->sriov.vf.pf_version.major = major;
	623	gt->sriov.vf.pf_version.minor = minor;
	624	}
	625
	626	static void vf_disconnect_pf(struct xe_gt *gt)
	627	{
	628	vf_connect_pf(gt, 0, 0);
	629	}
	630
	631	static int vf_handshake_with_pf(struct xe_gt *gt)
	632	{
	633	u32 major_wanted = GUC_RELAY_VERSION_LATEST_MAJOR;
	634	u32 minor_wanted = GUC_RELAY_VERSION_LATEST_MINOR;
	635	u32 major = major_wanted, minor = minor_wanted;
	636	int err;
	637
	638	err = relay_action_handshake(gt, &major, &minor);
	639	if (unlikely(err))
	640	goto failed;
	641
	642	if (!major && !minor) {
	643	err = -ENODATA;
	644	goto failed;
	645	}
	646
	647	xe_gt_sriov_dbg(gt, "using VF/PF ABI %u.%u\n", major, minor);
	648	vf_connect_pf(gt, major, minor);
	649	return 0;
	650
	651	failed:
	652	xe_gt_sriov_err(gt, "Unable to confirm VF/PF ABI version %u.%u (%pe)\n",
	653	major, minor, ERR_PTR(err));
	654	vf_disconnect_pf(gt);
	655	return err;
	656	}
	657
	658	/**
	659	* xe_gt_sriov_vf_connect - Establish connection with the PF driver.
	660	* @gt: the &xe_gt
	661	*
	662	* This function is for VF use only.
	663	*
	664	* Return: 0 on success or a negative error code on failure.
	665	*/
	666	int xe_gt_sriov_vf_connect(struct xe_gt *gt)
	667	{
	668	int err;
	669
	670	err = vf_handshake_with_pf(gt);
	671	if (unlikely(err))
	672	goto failed;
	673
	674	return 0;
	675
	676	failed:
	677	xe_gt_sriov_err(gt, "Failed to get version info (%pe)\n", ERR_PTR(err));
	678	return err;
	679	}
	680
	681	static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
	682	{
	683	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	684
	685	return major == gt->sriov.vf.pf_version.major &&
	686	minor <= gt->sriov.vf.pf_version.minor;
	687	}
	688
	689	static int vf_prepare_runtime_info(struct xe_gt *gt, unsigned int num_regs)
	690	{
	691	struct vf_runtime_reg *regs = gt->sriov.vf.runtime.regs;
	692	unsigned int regs_size = round_up(num_regs, 4);
	693	struct xe_device *xe = gt_to_xe(gt);
	694
	695	xe_gt_assert(gt, IS_SRIOV_VF(xe));
	696
	697	if (regs) {
	698	if (num_regs <= gt->sriov.vf.runtime.regs_size) {
	699	memset(regs, 0, num_regs * sizeof(*regs));
	700	gt->sriov.vf.runtime.num_regs = num_regs;
	701	return 0;
	702	}
	703
	704	drmm_kfree(&xe->drm, regs);
	705	gt->sriov.vf.runtime.regs = NULL;
	706	gt->sriov.vf.runtime.num_regs = 0;
	707	gt->sriov.vf.runtime.regs_size = 0;
	708	}
	709
	710	regs = drmm_kcalloc(&xe->drm, regs_size, sizeof(*regs), GFP_KERNEL);
	711	if (unlikely(!regs))
	712	return -ENOMEM;
	713
	714	gt->sriov.vf.runtime.regs = regs;
	715	gt->sriov.vf.runtime.num_regs = num_regs;
	716	gt->sriov.vf.runtime.regs_size = regs_size;
	717	return 0;
	718	}
	719
	720	static int vf_query_runtime_info(struct xe_gt *gt)
	721	{
	722	u32 request[VF2PF_QUERY_RUNTIME_REQUEST_MSG_LEN];
	723	u32 response[VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + 32]; /* up to 16 regs */
	724	u32 limit = (ARRAY_SIZE(response) - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) / 2;
	725	u32 count, remaining, num, i;
	726	u32 start = 0;
	727	int ret;
	728
	729	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	730	xe_gt_assert(gt, limit);
	731
	732	/* this is part of the 1.0 PF/VF ABI */
	733	if (!vf_is_negotiated(gt, 1, 0))
	734	return -ENOPKG;
	735
	736	request[0] = FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	737	FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	738	FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION,
	739	GUC_RELAY_ACTION_VF2PF_QUERY_RUNTIME) \|
	740	FIELD_PREP(VF2PF_QUERY_RUNTIME_REQUEST_MSG_0_LIMIT, limit);
	741
	742	repeat:
	743	request[1] = FIELD_PREP(VF2PF_QUERY_RUNTIME_REQUEST_MSG_1_START, start);
	744	ret = xe_guc_relay_send_to_pf(&gt->uc.guc.relay,
	745	request, ARRAY_SIZE(request),
	746	response, ARRAY_SIZE(response));
	747	if (unlikely(ret < 0))
	748	goto failed;
	749
	750	if (unlikely(ret < VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN)) {
	751	ret = -EPROTO;
	752	goto failed;
	753	}
	754	if (unlikely((ret - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) % 2)) {
	755	ret = -EPROTO;
	756	goto failed;
	757	}
	758
	759	num = (ret - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) / 2;
	760	count = FIELD_GET(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_0_COUNT, response[0]);
	761	remaining = FIELD_GET(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_1_REMAINING, response[1]);
	762
	763	xe_gt_sriov_dbg_verbose(gt, "count=%u num=%u ret=%d start=%u remaining=%u\n",
	764	count, num, ret, start, remaining);
	765
	766	if (unlikely(count != num)) {
	767	ret = -EPROTO;
	768	goto failed;
	769	}
	770
	771	if (start == 0) {
	772	ret = vf_prepare_runtime_info(gt, num + remaining);
	773	if (unlikely(ret < 0))
	774	goto failed;
	775	} else if (unlikely(start + num > gt->sriov.vf.runtime.num_regs)) {
	776	ret = -EPROTO;
	777	goto failed;
	778	}
	779
	780	for (i = 0; i < num; ++i) {
	781	struct vf_runtime_reg *reg = &gt->sriov.vf.runtime.regs[start + i];
	782
	783	reg->offset = response[VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + 2 * i];
	784	reg->value = response[VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + 2 * i + 1];
	785	}
	786
	787	if (remaining) {
	788	start += num;
	789	goto repeat;
	790	}
	791
	792	return 0;
	793
	794	failed:
	795	vf_prepare_runtime_info(gt, 0);
	796	return ret;
	797	}
	798
	799	static void vf_show_runtime_info(struct xe_gt *gt)
	800	{
	801	struct vf_runtime_reg *vf_regs = gt->sriov.vf.runtime.regs;
	802	unsigned int size = gt->sriov.vf.runtime.num_regs;
	803
	804	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	805
	806	for (; size--; vf_regs++)
	807	xe_gt_sriov_dbg(gt, "runtime(%#x) = %#x\n",
	808	vf_regs->offset, vf_regs->value);
	809	}
	810
	811	/**
	812	* xe_gt_sriov_vf_query_runtime - Query SR-IOV runtime data.
	813	* @gt: the &xe_gt
	814	*
	815	* This function is for VF use only.
	816	*
	817	* Return: 0 on success or a negative error code on failure.
	818	*/
	819	int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt)
	820	{
	821	int err;
	822
	823	err = vf_query_runtime_info(gt);
	824	if (unlikely(err))
	825	goto failed;
	826
	827	if (IS_ENABLED(CONFIG_DRM_XE_DEBUG))
	828	vf_show_runtime_info(gt);
	829
	830	return 0;
	831
	832	failed:
	833	xe_gt_sriov_err(gt, "Failed to get runtime info (%pe)\n",
	834	ERR_PTR(err));
	835	return err;
	836	}
	837
	838	static int vf_runtime_reg_cmp(const void a, const void b)
	839	{
	840	const struct vf_runtime_reg *ra = a;
	841	const struct vf_runtime_reg *rb = b;
	842
	843	return (int)ra->offset - (int)rb->offset;
	844	}
	845
	846	static struct vf_runtime_reg vf_lookup_reg(struct xe_gt gt, u32 addr)
	847	{
	848	struct xe_gt_sriov_vf_runtime *runtime = &gt->sriov.vf.runtime;
	849	struct vf_runtime_reg key = { .offset = addr };
	850
	851	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	852
	853	return bsearch(&key, runtime->regs, runtime->regs_size, sizeof(key),
	854	vf_runtime_reg_cmp);
	855	}
	856
	857	/**
	858	* xe_gt_sriov_vf_read32 - Get a register value from the runtime data.
	859	* @gt: the &xe_gt
	860	* @reg: the register to read
	861	*
	862	* This function is for VF use only.
	863	* This function shall be called after VF has connected to PF.
	864	* This function is dedicated for registers that VFs can't read directly.
	865	*
	866	* Return: register value obtained from the PF or 0 if not found.
	867	*/
	868	u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg)
	869	{
	870	u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
	871	struct vf_runtime_reg *rr;
	872
	873	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	874	xe_gt_assert(gt, gt->sriov.vf.pf_version.major);
	875	xe_gt_assert(gt, !reg.vf);
	876
	877	if (reg.addr == GMD_ID.addr) {
	878	xe_gt_sriov_dbg_verbose(gt, "gmdid(%#x) = %#x\n",
	879	addr, gt->sriov.vf.runtime.gmdid);
	880	return gt->sriov.vf.runtime.gmdid;
	881	}
	882
	883	rr = vf_lookup_reg(gt, addr);
	884	if (!rr) {
	885	xe_gt_WARN(gt, IS_ENABLED(CONFIG_DRM_XE_DEBUG),
	886	"VF is trying to read an inaccessible register %#x+%#x\n",
	887	reg.addr, addr - reg.addr);
	888	return 0;
	889	}
	890
	891	xe_gt_sriov_dbg_verbose(gt, "runtime[%#x] = %#x\n", addr, rr->value);
	892	return rr->value;
	893	}
	894
	895	/**
	896	* xe_gt_sriov_vf_print_config - Print VF self config.
	897	* @gt: the &xe_gt
	898	* @p: the &drm_printer
	899	*
	900	* This function is for VF use only.
	901	*/
	902	void xe_gt_sriov_vf_print_config(struct xe_gt gt, struct drm_printer p)
	903	{
	904	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	905	struct xe_device *xe = gt_to_xe(gt);
	906	char buf[10];
	907
	908	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	909
	910	drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
	911	config->ggtt_base,
	912	config->ggtt_base + config->ggtt_size - 1);
	913
	914	string_get_size(config->ggtt_size, 1, STRING_UNITS_2, buf, sizeof(buf));
	915	drm_printf(p, "GGTT size:\t%llu (%s)\n", config->ggtt_size, buf);
	916
	917	if (IS_DGFX(xe) && !xe_gt_is_media_type(gt)) {
	918	string_get_size(config->lmem_size, 1, STRING_UNITS_2, buf, sizeof(buf));
	919	drm_printf(p, "LMEM size:\t%llu (%s)\n", config->lmem_size, buf);
	920	}
	921
	922	drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
	923	drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
	924	}
	925
	926	/**
	927	* xe_gt_sriov_vf_print_runtime - Print VF's runtime regs received from PF.
	928	* @gt: the &xe_gt
	929	* @p: the &drm_printer
	930	*
	931	* This function is for VF use only.
	932	*/
	933	void xe_gt_sriov_vf_print_runtime(struct xe_gt gt, struct drm_printer p)
	934	{
	935	struct vf_runtime_reg *vf_regs = gt->sriov.vf.runtime.regs;
	936	unsigned int size = gt->sriov.vf.runtime.num_regs;
	937
	938	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	939
	940	for (; size--; vf_regs++)
	941	drm_printf(p, "%#x = %#x\n", vf_regs->offset, vf_regs->value);
	942	}
	943
	944	/**
	945	* xe_gt_sriov_vf_print_version - Print VF ABI versions.
	946	* @gt: the &xe_gt
	947	* @p: the &drm_printer
	948	*
	949	* This function is for VF use only.
	950	*/
	951	void xe_gt_sriov_vf_print_version(struct xe_gt gt, struct drm_printer p)
	952	{
	953	struct xe_gt_sriov_vf_guc_version *guc_version = &gt->sriov.vf.guc_version;
	954	struct xe_gt_sriov_vf_relay_version *pf_version = &gt->sriov.vf.pf_version;
	955	u32 branch, major, minor;
	956
	957	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	958
	959	drm_printf(p, "GuC ABI:\n");
	960
	961	vf_minimum_guc_version(gt, &branch, &major, &minor);
	962	drm_printf(p, "\tbase:\t%u.%u.%u.*\n", branch, major, minor);
	963
	964	vf_wanted_guc_version(gt, &branch, &major, &minor);
	965	drm_printf(p, "\twanted:\t%u.%u.%u.*\n", branch, major, minor);
	966
	967	drm_printf(p, "\thandshake:\t%u.%u.%u.%u\n",
	968	guc_version->branch, guc_version->major,
	969	guc_version->minor, guc_version->patch);
	970
	971	drm_printf(p, "PF ABI:\n");
	972
	973	drm_printf(p, "\tbase:\t%u.%u\n",
	974	GUC_RELAY_VERSION_BASE_MAJOR, GUC_RELAY_VERSION_BASE_MINOR);
	975	drm_printf(p, "\twanted:\t%u.%u\n",
	976	GUC_RELAY_VERSION_LATEST_MAJOR, GUC_RELAY_VERSION_LATEST_MINOR);
	977	drm_printf(p, "\thandshake:\t%u.%u\n",
	978	pf_version->major, pf_version->minor);
	979	}

drivers/gpu/drm/xe/xe_gt_sriov_vf.h+30−0

@@ -0,0 +1,30 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_SRIOV_VF_H_
	7	#define _XE_GT_SRIOV_VF_H_
	8
	9	#include <linux/types.h>
	10
	11	struct drm_printer;
	12	struct xe_gt;
	13	struct xe_reg;
	14
	15	int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);
	16	int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
	17	int xe_gt_sriov_vf_connect(struct xe_gt *gt);
	18	int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
	19	int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt);
	20
	21	u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
	22	u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
	23	u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
	24	u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg);
	25
	26	void xe_gt_sriov_vf_print_config(struct xe_gt gt, struct drm_printer p);
	27	void xe_gt_sriov_vf_print_runtime(struct xe_gt gt, struct drm_printer p);
	28	void xe_gt_sriov_vf_print_version(struct xe_gt gt, struct drm_printer p);
	29
	30	#endif

drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.c+72−0

@@ -0,0 +1,72 @@
	1	// SPDX-License-Identifier: MIT
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#include <linux/debugfs.h>
	7
	8	#include <drm/drm_debugfs.h>
	9
	10	#include "xe_gt_debugfs.h"
	11	#include "xe_gt_sriov_vf.h"
	12	#include "xe_gt_sriov_vf_debugfs.h"
	13	#include "xe_gt_types.h"
	14	#include "xe_sriov.h"
	15
	16	/*
	17	* /sys/kernel/debug/dri/0/
	18	* ├── gt0
	19	* │ ├── vf
	20	* │ │ ├── self_config
	21	* │ │ ├── abi_versions
	22	* │ │ ├── runtime_regs
	23	*/
	24
	25	static const struct drm_info_list vf_info[] = {
	26	{
	27	"self_config",
	28	.show = xe_gt_debugfs_simple_show,
	29	.data = xe_gt_sriov_vf_print_config,
	30	},
	31	{
	32	"abi_versions",
	33	.show = xe_gt_debugfs_simple_show,
	34	.data = xe_gt_sriov_vf_print_version,
	35	},
	36	#if defined(CONFIG_DRM_XE_DEBUG) \|\| defined(CONFIG_DRM_XE_DEBUG_SRIOV)
	37	{
	38	"runtime_regs",
	39	.show = xe_gt_debugfs_simple_show,
	40	.data = xe_gt_sriov_vf_print_runtime,
	41	},
	42	#endif
	43	};
	44
	45	/**
	46	* xe_gt_sriov_vf_debugfs_register - Register SR-IOV VF specific entries in GT debugfs.
	47	* @gt: the &xe_gt to register
	48	* @root: the &dentry that represents the GT directory
	49	*
	50	* Register SR-IOV VF entries that are GT related and must be shown under GT debugfs.
	51	*/
	52	void xe_gt_sriov_vf_debugfs_register(struct xe_gt gt, struct dentry root)
	53	{
	54	struct xe_device *xe = gt_to_xe(gt);
	55	struct drm_minor *minor = xe->drm.primary;
	56	struct dentry *vfdentry;
	57
	58	xe_assert(xe, IS_SRIOV_VF(xe));
	59	xe_assert(xe, root->d_inode->i_private == gt);
	60
	61	/*
	62	* /sys/kernel/debug/dri/0/
	63	* ├── gt0
	64	* │ ├── vf
	65	*/
	66	vfdentry = debugfs_create_dir("vf", root);
	67	if (IS_ERR(vfdentry))
	68	return;
	69	vfdentry->d_inode->i_private = gt;
	70
	71	drm_debugfs_create_files(vf_info, ARRAY_SIZE(vf_info), vfdentry, minor);
	72	}

drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.h+14−0

@@ -0,0 +1,14 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_SRIOV_VF_DEBUGFS_H_
	7	#define _XE_GT_SRIOV_VF_DEBUGFS_H_
	8
	9	struct xe_gt;
	10	struct dentry;
	11
	12	void xe_gt_sriov_vf_debugfs_register(struct xe_gt gt, struct dentry root);
	13
	14	#endif

drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h+84−0

@@ -0,0 +1,84 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_SRIOV_VF_TYPES_H_
	7	#define _XE_GT_SRIOV_VF_TYPES_H_
	8
	9	#include <linux/types.h>
	10
	11	/**
	12	* struct xe_gt_sriov_vf_guc_version - GuC ABI version details.
	13	*/
	14	struct xe_gt_sriov_vf_guc_version {
	15	/** @branch: branch version. */
	16	u8 branch;
	17	/** @major: major version. */
	18	u8 major;
	19	/** @minor: minor version. */
	20	u8 minor;
	21	/** @patch: patch version. */
	22	u8 patch;
	23	};
	24
	25	/**
	26	* struct xe_gt_sriov_vf_relay_version - PF ABI version details.
	27	*/
	28	struct xe_gt_sriov_vf_relay_version {
	29	/** @major: major version. */
	30	u16 major;
	31	/** @minor: minor version. */
	32	u16 minor;
	33	};
	34
	35	/**
	36	* struct xe_gt_sriov_vf_selfconfig - VF configuration data.
	37	*/
	38	struct xe_gt_sriov_vf_selfconfig {
	39	/** @ggtt_base: assigned base offset of the GGTT region. */
	40	u64 ggtt_base;
	41	/** @ggtt_size: assigned size of the GGTT region. */
	42	u64 ggtt_size;
	43	/** @lmem_size: assigned size of the LMEM. */
	44	u64 lmem_size;
	45	/** @num_ctxs: assigned number of GuC submission context IDs. */
	46	u16 num_ctxs;
	47	/** @num_dbs: assigned number of GuC doorbells IDs. */
	48	u16 num_dbs;
	49	};
	50
	51	/**
	52	* struct xe_gt_sriov_vf_runtime - VF runtime data.
	53	*/
	54	struct xe_gt_sriov_vf_runtime {
	55	/** @gmdid: cached value of the GDMID register. */
	56	u32 gmdid;
	57	/** @regs_size: size of runtime register array. */
	58	u32 regs_size;
	59	/** @num_regs: number of runtime registers in the array. */
	60	u32 num_regs;
	61	/** @regs: pointer to array of register offset/value pairs. */
	62	struct vf_runtime_reg {
	63	/** @regs.offset: register offset. */
	64	u32 offset;
	65	/** @regs.value: register value. */
	66	u32 value;
	67	} *regs;
	68	};
	69
	70	/**
	71	* struct xe_gt_sriov_vf - GT level VF virtualization data.
	72	*/
	73	struct xe_gt_sriov_vf {
	74	/** @guc_version: negotiated GuC ABI version. */
	75	struct xe_gt_sriov_vf_guc_version guc_version;
	76	/** @self_config: resource configurations. */
	77	struct xe_gt_sriov_vf_selfconfig self_config;
	78	/** @pf_version: negotiated VF/PF ABI version. */
	79	struct xe_gt_sriov_vf_relay_version pf_version;
	80	/** @runtime: runtime data retrieved from the PF. */
	81	struct xe_gt_sriov_vf_runtime runtime;
	82	};
	83
	84	#endif

drivers/gpu/drm/xe/xe_gt_sysfs.c+2−2

@@ -22,7 +22,7 @@ static const struct kobj_type xe_gt_sysfs_kobj_type = { · 21 unmodified lines
22	.sysfs_ops = &kobj_sysfs_ops,	22	.sysfs_ops = &kobj_sysfs_ops,
23	};	23	};
24		24
25	static void gt_sysfs_fini(struct drm_device drm, void arg)	25	static void gt_sysfs_fini(void *arg)
26	{	26	{
27	struct xe_gt *gt = arg;	27	struct xe_gt *gt = arg;
28		28
@@ -51,5 +51,5 @@ int xe_gt_sysfs_init(struct xe_gt *gt) · 22 unmodified lines
51		51
52	gt->sysfs = &kg->base;	52	gt->sysfs = &kg->base;
53		53
54	return drmm_add_action_or_reset(&xe->drm, gt_sysfs_fini, gt);	54	return devm_add_action(xe->drm.dev, gt_sysfs_fini, gt);
55	}	55	}

drivers/gpu/drm/xe/xe_gt_throttle.crenamed from drivers/gpu/drm/xe/xe_gt_throttle_sysfs.c+15−15

@@ -9,14 +9,14 @@ · 8 unmodified lines
9	#include "xe_device.h"	9	#include "xe_device.h"
10	#include "xe_gt.h"	10	#include "xe_gt.h"
11	#include "xe_gt_sysfs.h"	11	#include "xe_gt_sysfs.h"
12	#include "xe_gt_throttle_sysfs.h"	12	#include "xe_gt_throttle.h"
13	#include "xe_mmio.h"	13	#include "xe_mmio.h"
14	#include "xe_pm.h"	14	#include "xe_pm.h"
15		15
16	/**	16	/**
17	* DOC: Xe GT Throttle	17	* DOC: Xe GT Throttle
18	*	18	*
19	* Provides sysfs entries for frequency throttle reasons in GT	19	* Provides sysfs entries and other helpers for frequency throttle reasons in GT
20	*	20	*
21	* device/gt#/freq0/throttle/status - Overall status	21	* device/gt#/freq0/throttle/status - Overall status
22	* device/gt#/freq0/throttle/reason_pl1 - Frequency throttle due to PL1	22	* device/gt#/freq0/throttle/reason_pl1 - Frequency throttle due to PL1
@@ -35,7 +35,7 @@ dev_to_gt(struct device *dev) · 12 unmodified lines
35	return kobj_to_gt(dev->kobj.parent);	35	return kobj_to_gt(dev->kobj.parent);
36	}	36	}
37		37
38	static u32 read_perf_limit_reasons(struct xe_gt *gt)	38	u32 xe_gt_throttle_get_limit_reasons(struct xe_gt *gt)
39	{	39	{
40	u32 reg;	40	u32 reg;
41		41
@@ -51,63 +51,63 @@ static u32 read_perf_limit_reasons(struct xe_gt *gt) · 9 unmodified lines
51		51
52	static u32 read_status(struct xe_gt *gt)	52	static u32 read_status(struct xe_gt *gt)
53	{	53	{
54	u32 status = read_perf_limit_reasons(gt) & GT0_PERF_LIMIT_REASONS_MASK;	54	u32 status = xe_gt_throttle_get_limit_reasons(gt) & GT0_PERF_LIMIT_REASONS_MASK;
55		55
56	return status;	56	return status;
57	}	57	}
58		58
59	static u32 read_reason_pl1(struct xe_gt *gt)	59	static u32 read_reason_pl1(struct xe_gt *gt)
60	{	60	{
61	u32 pl1 = read_perf_limit_reasons(gt) & POWER_LIMIT_1_MASK;	61	u32 pl1 = xe_gt_throttle_get_limit_reasons(gt) & POWER_LIMIT_1_MASK;
62		62
63	return pl1;	63	return pl1;
64	}	64	}
65		65
66	static u32 read_reason_pl2(struct xe_gt *gt)	66	static u32 read_reason_pl2(struct xe_gt *gt)
67	{	67	{
68	u32 pl2 = read_perf_limit_reasons(gt) & POWER_LIMIT_2_MASK;	68	u32 pl2 = xe_gt_throttle_get_limit_reasons(gt) & POWER_LIMIT_2_MASK;
69		69
70	return pl2;	70	return pl2;
71	}	71	}
72		72
73	static u32 read_reason_pl4(struct xe_gt *gt)	73	static u32 read_reason_pl4(struct xe_gt *gt)
74	{	74	{
75	u32 pl4 = read_perf_limit_reasons(gt) & POWER_LIMIT_4_MASK;	75	u32 pl4 = xe_gt_throttle_get_limit_reasons(gt) & POWER_LIMIT_4_MASK;
76		76
77	return pl4;	77	return pl4;
78	}	78	}
79		79
80	static u32 read_reason_thermal(struct xe_gt *gt)	80	static u32 read_reason_thermal(struct xe_gt *gt)
81	{	81	{
82	u32 thermal = read_perf_limit_reasons(gt) & THERMAL_LIMIT_MASK;	82	u32 thermal = xe_gt_throttle_get_limit_reasons(gt) & THERMAL_LIMIT_MASK;
83		83
84	return thermal;	84	return thermal;
85	}	85	}
86		86
87	static u32 read_reason_prochot(struct xe_gt *gt)	87	static u32 read_reason_prochot(struct xe_gt *gt)
88	{	88	{
89	u32 prochot = read_perf_limit_reasons(gt) & PROCHOT_MASK;	89	u32 prochot = xe_gt_throttle_get_limit_reasons(gt) & PROCHOT_MASK;
90		90
91	return prochot;	91	return prochot;
92	}	92	}
93		93
94	static u32 read_reason_ratl(struct xe_gt *gt)	94	static u32 read_reason_ratl(struct xe_gt *gt)
95	{	95	{
96	u32 ratl = read_perf_limit_reasons(gt) & RATL_MASK;	96	u32 ratl = xe_gt_throttle_get_limit_reasons(gt) & RATL_MASK;
97		97
98	return ratl;	98	return ratl;
99	}	99	}
100		100
101	static u32 read_reason_vr_thermalert(struct xe_gt *gt)	101	static u32 read_reason_vr_thermalert(struct xe_gt *gt)
102	{	102	{
103	u32 thermalert = read_perf_limit_reasons(gt) & VR_THERMALERT_MASK;	103	u32 thermalert = xe_gt_throttle_get_limit_reasons(gt) & VR_THERMALERT_MASK;
104		104
105	return thermalert;	105	return thermalert;
106	}	106	}
107		107
108	static u32 read_reason_vr_tdc(struct xe_gt *gt)	108	static u32 read_reason_vr_tdc(struct xe_gt *gt)
109	{	109	{
110	u32 tdc = read_perf_limit_reasons(gt) & VR_TDC_MASK;	110	u32 tdc = xe_gt_throttle_get_limit_reasons(gt) & VR_TDC_MASK;
111		111
112	return tdc;	112	return tdc;
113	}	113	}
@@ -229,14 +229,14 @@ static const struct attribute_group throttle_group_attrs = { · 115 unmodified lines
229	.attrs = throttle_attrs,	229	.attrs = throttle_attrs,
230	};	230	};
231		231
232	static void gt_throttle_sysfs_fini(struct drm_device drm, void arg)	232	static void gt_throttle_sysfs_fini(void *arg)
233	{	233	{
234	struct xe_gt *gt = arg;	234	struct xe_gt *gt = arg;
235		235
236	sysfs_remove_group(gt->freq, &throttle_group_attrs);	236	sysfs_remove_group(gt->freq, &throttle_group_attrs);
237	}	237	}
238		238
239	int xe_gt_throttle_sysfs_init(struct xe_gt *gt)	239	int xe_gt_throttle_init(struct xe_gt *gt)
240	{	240	{
241	struct xe_device *xe = gt_to_xe(gt);	241	struct xe_device *xe = gt_to_xe(gt);
242	int err;	242	int err;
@@ -245,5 +245,5 @@ int xe_gt_throttle_sysfs_init(struct xe_gt *gt) · 2 unmodified lines
245	if (err)	245	if (err)
246	return err;	246	return err;
247		247
248	return drmm_add_action_or_reset(&xe->drm, gt_throttle_sysfs_fini, gt);	248	return devm_add_action_or_reset(xe->drm.dev, gt_throttle_sysfs_fini, gt);
249	}	249	}

drivers/gpu/drm/xe/xe_gt_throttle.h+17−0

@@ -0,0 +1,17 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GT_THROTTLE_H_
	7	#define _XE_GT_THROTTLE_H_
	8
	9	#include <linux/types.h>
	10
	11	struct xe_gt;
	12
	13	int xe_gt_throttle_init(struct xe_gt *gt);
	14
	15	u32 xe_gt_throttle_get_limit_reasons(struct xe_gt *gt);
	16
	17	#endif /* _XE_GT_THROTTLE_H_ */

drivers/gpu/drm/xe/xe_gt_throttle_sysfs.h+0−16

@@ -1,16 +0,0 @@
1	/* SPDX-License-Identifier: MIT */
2	/*
3	* Copyright © 2023 Intel Corporation
4	*/
5
6	#ifndef _XE_GT_THROTTLE_SYSFS_H_
7	#define _XE_GT_THROTTLE_SYSFS_H_
8
9	#include <drm/drm_managed.h>
10
11	struct xe_gt;
12
13	int xe_gt_throttle_sysfs_init(struct xe_gt *gt);
14
15	#endif /* _XE_GT_THROTTLE_SYSFS_H_ */
16

drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c+47−17

 #include "abi/guc_actions_abi.h"
 #include "xe_device.h"
+#include "xe_force_wake.h"
 #include "xe_gt.h"
 #include "xe_gt_printk.h"
 #include "xe_guc.h"
 			return seqno;
 		xe_gt_tlb_invalidation_wait(gt, seqno);
-	} else if (xe_device_uc_enabled(xe)) {
+	} else if (xe_device_uc_enabled(xe) && !xe_device_wedged(xe)) {
 		xe_gt_WARN_ON(gt, xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
 		if (xe->info.platform == XE_PVC || GRAPHICS_VER(xe) >= 20) {
 			xe_mmio_write32(gt, PVC_GUC_TLB_INV_DESC1,
 }
 /**
- * xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA
+ * xe_gt_tlb_invalidation_range - Issue a TLB invalidation on this GT for an
+ * address range
+ *
  * @gt: graphics tile
  * @fence: invalidation fence which will be signal on TLB invalidation
  * completion, can be NULL
- * @vma: VMA to invalidate
+ * @start: start address
+ * @end: end address
+ * @asid: address space id
  *
  * Issue a range based TLB invalidation if supported, if not fallback to a full
  * TLB invalidation. Completion of TLB is asynchronous and caller can either use
  * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success,
  * negative error code on error.
  */
-int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
+int xe_gt_tlb_invalidation_range(struct xe_gt *gt,
-			       struct xe_gt_tlb_invalidation_fence *fence,
+				 struct xe_gt_tlb_invalidation_fence *fence,
-			       struct xe_vma *vma)
+				 u64 start, u64 end, u32 asid)
 {
 	struct xe_device *xe = gt_to_xe(gt);
 #define MAX_TLB_INVALIDATION_LEN	7
 	u32 action[MAX_TLB_INVALIDATION_LEN];
 	int len = 0;
-	xe_gt_assert(gt, vma);
 	/* Execlists not supported */
 	if (gt_to_xe(gt)->info.force_execlist) {
 		if (fence)
 	if (!xe->info.has_range_tlb_invalidation) {
 		action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_FULL);
 	} else {
-		u64 start = xe_vma_start(vma);
+		u64 orig_start = start;
-		u64 length = xe_vma_size(vma);
+		u64 length = end - start;
-		u64 align, end;
+		u64 align;
 		if (length < SZ_4K)
 			length = SZ_4K;
 		 * address mask covering the required range.
 		 */
 		align = roundup_pow_of_two(length);
-		start = ALIGN_DOWN(xe_vma_start(vma), align);
+		start = ALIGN_DOWN(start, align);
-		end = ALIGN(xe_vma_end(vma), align);
+		end = ALIGN(end, align);
 		length = align;
 		while (start + length < end) {
 			length <<= 1;
-			start = ALIGN_DOWN(xe_vma_start(vma), length);
+			start = ALIGN_DOWN(orig_start, length);
 		}
 		/*
 		 */
 		if (length >= SZ_2M) {
 			length = max_t(u64, SZ_16M, length);
-			start = ALIGN_DOWN(xe_vma_start(vma), length);
+			start = ALIGN_DOWN(orig_start, length);
 		}
 		xe_gt_assert(gt, length >= SZ_4K);
 		xe_gt_assert(gt, is_power_of_2(length));
-		xe_gt_assert(gt, !(length & GENMASK(ilog2(SZ_16M) - 1, ilog2(SZ_2M) + 1)));
+		xe_gt_assert(gt, !(length & GENMASK(ilog2(SZ_16M) - 1,
+						    ilog2(SZ_2M) + 1)));
 		xe_gt_assert(gt, IS_ALIGNED(start, length));
 		action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_PAGE_SELECTIVE);
-		action[len++] = xe_vma_vm(vma)->usm.asid;
+		action[len++] = asid;
 		action[len++] = lower_32_bits(start);
 		action[len++] = upper_32_bits(start);
 		action[len++] = ilog2(length) - ilog2(SZ_4K);
 	return send_tlb_invalidation(&gt->uc.guc, fence, action, len);
 }
+/**
+ * xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA
+ * @gt: graphics tile
+ * @fence: invalidation fence which will be signal on TLB invalidation
+ * completion, can be NULL
+ * @vma: VMA to invalidate
+ *
+ * Issue a range based TLB invalidation if supported, if not fallback to a full
+ * TLB invalidation. Completion of TLB is asynchronous and caller can either use
+ * the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for
+ * completion.
+ *
+ * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success,
+ * negative error code on error.
+ */
+int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
+			       struct xe_gt_tlb_invalidation_fence *fence,
+			       struct xe_vma *vma)
+{
+	xe_gt_assert(gt, vma);
+	return xe_gt_tlb_invalidation_range(gt, fence, xe_vma_start(vma),
+					    xe_vma_end(vma),
+					    xe_vma_vm(vma)->usm.asid);
+}
 /**
  * xe_gt_tlb_invalidation_wait - Wait for TLB to complete
  * @gt: graphics tile

drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h+3−0

 int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
 			       struct xe_gt_tlb_invalidation_fence *fence,
 			       struct xe_vma *vma);
+int xe_gt_tlb_invalidation_range(struct xe_gt *gt,
+				 struct xe_gt_tlb_invalidation_fence *fence,
+				 u64 start, u64 end, u32 asid);
 int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno);
 int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len);

drivers/gpu/drm/xe/xe_gt_topology.c+13−1

@@ -108,7 +108,9 @@ gen_l3_mask_from_pattern(struct xe_device *xe, xe_l3_bank_mask_t dst, · 107 unmodified lines
108	{	108	{
109	unsigned long bit;	109	unsigned long bit;
110		110
111	xe_assert(xe, fls(mask) <= patternbits);	111	xe_assert(xe, find_last_bit(pattern, XE_MAX_L3_BANK_MASK_BITS) < patternbits \|\|
		112	bitmap_empty(pattern, XE_MAX_L3_BANK_MASK_BITS));
		113	xe_assert(xe, !mask \|\| patternbits * (__fls(mask) + 1) <= XE_MAX_L3_BANK_MASK_BITS);
112	for_each_set_bit(bit, &mask, 32) {	114	for_each_set_bit(bit, &mask, 32) {
113	xe_l3_bank_mask_t shifted_pattern = {};	115	xe_l3_bank_mask_t shifted_pattern = {};
114		116
@@ -278,3 +280,13 @@ bool xe_gt_topology_has_dss_in_quadrant(struct xe_gt *gt, int quad) · 163 unmodified lines
278		280
279	return quad_first < (quad + 1) * dss_per_quad;	281	return quad_first < (quad + 1) * dss_per_quad;
280	}	282	}
		283
		284	bool xe_gt_has_geometry_dss(struct xe_gt *gt, unsigned int dss)
		285	{
		286	return test_bit(dss, gt->fuse_topo.g_dss_mask);
		287	}
		288
		289	bool xe_gt_has_compute_dss(struct xe_gt *gt, unsigned int dss)
		290	{
		291	return test_bit(dss, gt->fuse_topo.c_dss_mask);
		292	}

drivers/gpu/drm/xe/xe_gt_topology.h+3−0

@@ -33,4 +33,7 @@ bool xe_dss_mask_empty(const xe_dss_mask_t mask); · 32 unmodified lines
33	bool	33	bool
34	xe_gt_topology_has_dss_in_quadrant(struct xe_gt *gt, int quad);	34	xe_gt_topology_has_dss_in_quadrant(struct xe_gt *gt, int quad);
35		35
		36	bool xe_gt_has_geometry_dss(struct xe_gt *gt, unsigned int dss);
		37	bool xe_gt_has_compute_dss(struct xe_gt *gt, unsigned int dss);
		38
36	#endif /* _XE_GT_TOPOLOGY_H_ */	39	#endif /* _XE_GT_TOPOLOGY_H_ */

drivers/gpu/drm/xe/xe_gt_types.h+26−8

 #include "xe_force_wake_types.h"
 #include "xe_gt_idle_types.h"
 #include "xe_gt_sriov_pf_types.h"
+#include "xe_gt_sriov_vf_types.h"
 #include "xe_hw_engine_types.h"
 #include "xe_hw_fence_types.h"
 #include "xe_reg_sr_types.h"
 	struct {
 		/** @info.type: type of GT */
 		enum xe_gt_type type;
-		/** @info.id: Unique ID of this GT within the PCI Device */
-		u8 id;
 		/** @info.reference_clock: clock frequency */
 		u32 reference_clock;
-		/** @info.engine_mask: mask of engines present on GT */
-		u64 engine_mask;
 		/**
-		 * @info.__engine_mask: mask of engines present on GT read from
+		 * @info.engine_mask: mask of engines present on GT. Some of
-		 * xe_pci.c, used to fake reading the engine_mask from the
+		 * them may be reserved in runtime and not available for user.
-		 * hwconfig blob.
+		 * See @user_engines.mask
 		 */
-		u64 __engine_mask;
+		u64 engine_mask;
 		/** @info.gmdid: raw GMD_ID value from hardware */
 		u32 gmdid;
+		/** @info.id: Unique ID of this GT within the PCI Device */
+		u8 id;
+		/** @info.has_indirect_ring_state: GT has indirect ring state support */
+		u8 has_indirect_ring_state:1;
 	} info;
 	/**
 	union {
 		/** @sriov.pf: PF data. Valid only if driver is running as PF */
 		struct xe_gt_sriov_pf pf;
+		/** @sriov.vf: VF data. Valid only if driver is running as VF */
+		struct xe_gt_sriov_vf vf;
 	} sriov;
 	/**
 		/** @wa_active.oob: bitmap with active OOB workaroudns */
 		unsigned long *oob;
 	} wa_active;
+	/** @user_engines: engines present in GT and available to userspace */
+	struct {
+		/**
+		 * @user_engines.mask: like @info->engine_mask, but take in
+		 * consideration only engines available to userspace
+		 */
+		u64 mask;
+		/**
+		 * @user_engines.instances_per_class: aggregate per class the
+		 * number of engines available to userspace
+		 */
+		u8 instances_per_class[XE_ENGINE_CLASS_MAX];
+	} user_engines;
 };
 #endif

drivers/gpu/drm/xe/xe_guc.c+272−82

@@ -19,8 +19,11 @@ · 18 unmodified lines
19	#include "xe_force_wake.h"	19	#include "xe_force_wake.h"
20	#include "xe_gt.h"	20	#include "xe_gt.h"
21	#include "xe_gt_printk.h"	21	#include "xe_gt_printk.h"
		22	#include "xe_gt_sriov_vf.h"
		23	#include "xe_gt_throttle.h"
22	#include "xe_guc_ads.h"	24	#include "xe_guc_ads.h"
23	#include "xe_guc_ct.h"	25	#include "xe_guc_ct.h"
		26	#include "xe_guc_db_mgr.h"
24	#include "xe_guc_hwconfig.h"	27	#include "xe_guc_hwconfig.h"
25	#include "xe_guc_log.h"	28	#include "xe_guc_log.h"
26	#include "xe_guc_pc.h"	29	#include "xe_guc_pc.h"
@@ -239,7 +242,7 @@ static void guc_write_params(struct xe_guc *guc) · 212 unmodified lines
239	xe_mmio_write32(gt, SOFT_SCRATCH(1 + i), guc->params[i]);	242	xe_mmio_write32(gt, SOFT_SCRATCH(1 + i), guc->params[i]);
240	}	243	}
241		244
242	static void guc_fini(struct drm_device drm, void arg)	245	static void guc_fini_hw(void *arg)
243	{	246	{
244	struct xe_guc *guc = arg;	247	struct xe_guc *guc = arg;
245	struct xe_gt *gt = guc_to_gt(guc);	248	struct xe_gt *gt = guc_to_gt(guc);
@@ -293,6 +296,23 @@ static int xe_guc_realloc_post_hwconfig(struct xe_guc *guc) · 47 unmodified lines
293	return 0;	296	return 0;
294	}	297	}
295		298
		299	static int vf_guc_init(struct xe_guc *guc)
		300	{
		301	int err;
		302
		303	xe_guc_comm_init_early(guc);
		304
		305	err = xe_guc_ct_init(&guc->ct);
		306	if (err)
		307	return err;
		308
		309	err = xe_guc_relay_init(&guc->relay);
		310	if (err)
		311	return err;
		312
		313	return 0;
		314	}
		315
296	int xe_guc_init(struct xe_guc *guc)	316	int xe_guc_init(struct xe_guc *guc)
297	{	317	{
298	struct xe_device *xe = guc_to_xe(guc);	318	struct xe_device *xe = guc_to_xe(guc);
@@ -307,6 +327,13 @@ int xe_guc_init(struct xe_guc *guc) · 8 unmodified lines
307	if (!xe_uc_fw_is_enabled(&guc->fw))	327	if (!xe_uc_fw_is_enabled(&guc->fw))
308	return 0;	328	return 0;
309		329
		330	if (IS_SRIOV_VF(xe)) {
		331	ret = vf_guc_init(guc);
		332	if (ret)
		333	goto out;
		334	return 0;
		335	}
		336
310	ret = xe_guc_log_init(&guc->log);	337	ret = xe_guc_log_init(&guc->log);
311	if (ret)	338	if (ret)
312	goto out;	339	goto out;
@@ -323,7 +350,7 @@ int xe_guc_init(struct xe_guc *guc) · 10 unmodified lines
323	if (ret)	350	if (ret)
324	goto out;	351	goto out;
325		352
326	ret = drmm_add_action_or_reset(&xe->drm, guc_fini, guc);	353	ret = devm_add_action_or_reset(xe->drm.dev, guc_fini_hw, guc);
327	if (ret)	354	if (ret)
328	goto out;	355	goto out;
329		356
@@ -340,6 +367,19 @@ out: · 10 unmodified lines
340	return ret;	367	return ret;
341	}	368	}
342		369
		370	static int vf_guc_init_post_hwconfig(struct xe_guc *guc)
		371	{
		372	int err;
		373
		374	err = xe_guc_submit_init(guc, xe_gt_sriov_vf_guc_ids(guc_to_gt(guc)));
		375	if (err)
		376	return err;
		377
		378	/* XXX xe_guc_db_mgr_init not needed for now */
		379
		380	return 0;
		381	}
		382
343	/**	383	/**
344	* xe_guc_init_post_hwconfig - initialize GuC post hwconfig load	384	* xe_guc_init_post_hwconfig - initialize GuC post hwconfig load
345	* @guc: The GuC object	385	* @guc: The GuC object
@@ -350,12 +390,23 @@ int xe_guc_init_post_hwconfig(struct xe_guc *guc) · 4 unmodified lines
350	{	390	{
351	int ret;	391	int ret;
352		392
		393	if (IS_SRIOV_VF(guc_to_xe(guc)))
		394	return vf_guc_init_post_hwconfig(guc);
		395
353	ret = xe_guc_realloc_post_hwconfig(guc);	396	ret = xe_guc_realloc_post_hwconfig(guc);
354	if (ret)	397	if (ret)
355	return ret;	398	return ret;
356		399
357	guc_init_params_post_hwconfig(guc);	400	guc_init_params_post_hwconfig(guc);
358		401
		402	ret = xe_guc_submit_init(guc, ~0);
		403	if (ret)
		404	return ret;
		405
		406	ret = xe_guc_db_mgr_init(&guc->dbm, ~0);
		407	if (ret)
		408	return ret;
		409
359	ret = xe_guc_pc_init(&guc->pc);	410	ret = xe_guc_pc_init(&guc->pc);
360	if (ret)	411	if (ret)
361	return ret;	412	return ret;
@@ -379,6 +430,9 @@ int xe_guc_reset(struct xe_guc *guc) · 17 unmodified lines
379		430
380	xe_force_wake_assert_held(gt_to_fw(gt), XE_FW_GT);	431	xe_force_wake_assert_held(gt_to_fw(gt), XE_FW_GT);
381		432
		433	if (IS_SRIOV_VF(gt_to_xe(gt)))
		434	return xe_gt_sriov_vf_bootstrap(gt);
		435
382	xe_mmio_write32(gt, GDRST, GRDOM_GUC);	436	xe_mmio_write32(gt, GDRST, GRDOM_GUC);
383		437
384	ret = xe_mmio_wait32(gt, GDRST, GRDOM_GUC, 0, 5000, &gdrst, false);	438	ret = xe_mmio_wait32(gt, GDRST, GRDOM_GUC, 0, 5000, &gdrst, false);
@@ -451,58 +505,194 @@ static int guc_xfer_rsa(struct xe_guc *guc) · 66 unmodified lines
451	return 0;	505	return 0;
452	}	506	}
453		507
454	static int guc_wait_ucode(struct xe_guc *guc)	508	/*
		509	* Check a previously read GuC status register (GUC_STATUS) looking for
		510	* known terminal states (either completion or failure) of either the
		511	* microkernel status field or the boot ROM status field. Returns +1 for
		512	* successful completion, -1 for failure and 0 for any intermediate state.
		513	*/
		514	static int guc_load_done(u32 status)
455	{	515	{
456	struct xe_gt *gt = guc_to_gt(guc);	516	u32 uk_val = REG_FIELD_GET(GS_UKERNEL_MASK, status);
457	u32 status;	517	u32 br_val = REG_FIELD_GET(GS_BOOTROM_MASK, status);
458	int ret;	518
		519	switch (uk_val) {
		520	case XE_GUC_LOAD_STATUS_READY:
		521	return 1;
		522
		523	case XE_GUC_LOAD_STATUS_ERROR_DEVID_BUILD_MISMATCH:
		524	case XE_GUC_LOAD_STATUS_GUC_PREPROD_BUILD_MISMATCH:
		525	case XE_GUC_LOAD_STATUS_ERROR_DEVID_INVALID_GUCTYPE:
		526	case XE_GUC_LOAD_STATUS_HWCONFIG_ERROR:
		527	case XE_GUC_LOAD_STATUS_DPC_ERROR:
		528	case XE_GUC_LOAD_STATUS_EXCEPTION:
		529	case XE_GUC_LOAD_STATUS_INIT_DATA_INVALID:
		530	case XE_GUC_LOAD_STATUS_MPU_DATA_INVALID:
		531	case XE_GUC_LOAD_STATUS_INIT_MMIO_SAVE_RESTORE_INVALID:
		532	return -1;
		533	}
		534
		535	switch (br_val) {
		536	case XE_BOOTROM_STATUS_NO_KEY_FOUND:
		537	case XE_BOOTROM_STATUS_RSA_FAILED:
		538	case XE_BOOTROM_STATUS_PAVPC_FAILED:
		539	case XE_BOOTROM_STATUS_WOPCM_FAILED:
		540	case XE_BOOTROM_STATUS_LOADLOC_FAILED:
		541	case XE_BOOTROM_STATUS_JUMP_FAILED:
		542	case XE_BOOTROM_STATUS_RC6CTXCONFIG_FAILED:
		543	case XE_BOOTROM_STATUS_MPUMAP_INCORRECT:
		544	case XE_BOOTROM_STATUS_EXCEPTION:
		545	case XE_BOOTROM_STATUS_PROD_KEY_CHECK_FAILURE:
		546	return -1;
		547	}
		548
		549	return 0;
		550	}
		551
		552	static s32 guc_pc_get_cur_freq(struct xe_guc_pc *guc_pc)
		553	{
		554	u32 freq;
		555	int ret = xe_guc_pc_get_cur_freq(guc_pc, &freq);
		556
		557	return ret ? ret : freq;
		558	}
459		559
		560	/*
		561	* Wait for the GuC to start up.
		562	*
		563	* Measurements indicate this should take no more than 20ms (assuming the GT
		564	* clock is at maximum frequency). However, thermal throttling and other issues
		565	* can prevent the clock hitting max and thus making the load take significantly
		566	* longer. Allow up to 200ms as a safety margin for real world worst case situations.
		567	*
		568	* However, bugs anywhere from KMD to GuC to PCODE to fan failure in a CI farm can
		569	* lead to even longer times. E.g. if the GT is clamped to minimum frequency then
		570	* the load times can be in the seconds range. So the timeout is increased for debug
		571	* builds to ensure that problems can be correctly analysed. For release builds, the
		572	* timeout is kept short so that users don't wait forever to find out that there is a
		573	* problem. In either case, if the load took longer than is reasonable even with some
		574	* 'sensible' throttling, then flag a warning because something is not right.
		575	*
		576	* Note that there is a limit on how long an individual usleep_range() can wait for,
		577	* hence longer waits require wrapping a shorter wait in a loop.
		578	*
		579	* Note that the only reason an end user should hit the shorter timeout is in case of
		580	* extreme thermal throttling. And a system that is that hot during boot is probably
		581	* dead anyway!
		582	*/
		583	#if defined(CONFIG_DRM_XE_DEBUG)
		584	#define GUC_LOAD_RETRY_LIMIT 20
		585	#else
		586	#define GUC_LOAD_RETRY_LIMIT 3
		587	#endif
		588	#define GUC_LOAD_TIME_WARN_MS 200
		589
		590	static void guc_wait_ucode(struct xe_guc *guc)
		591	{
		592	struct xe_gt *gt = guc_to_gt(guc);
		593	struct xe_guc_pc *guc_pc = &gt->uc.guc.pc;
		594	ktime_t before, after, delta;
		595	int load_done;
		596	u32 status = 0;
		597	int count = 0;
		598	u64 delta_ms;
		599	u32 before_freq;
		600
		601	before_freq = xe_guc_pc_get_act_freq(guc_pc);
		602	before = ktime_get();
460	/*	603	/*
461	* Wait for the GuC to start up.	604	* Note, can't use any kind of timing information from the call to xe_mmio_wait.
462	* NB: Docs recommend not using the interrupt for completion.	605	* It could return a thousand intermediate stages at random times. Instead, must
463	* Measurements indicate this should take no more than 20ms	606	* manually track the total time taken and locally implement the timeout.
464	* (assuming the GT clock is at maximum frequency). So, a
465	* timeout here indicates that the GuC has failed and is unusable.
466	* (Higher levels of the driver may decide to reset the GuC and
467	* attempt the ucode load again if this happens.)
468	*
469	* FIXME: There is a known (but exceedingly unlikely) race condition
470	* where the asynchronous frequency management code could reduce
471	* the GT clock while a GuC reload is in progress (during a full
472	* GT reset). A fix is in progress but there are complex locking
473	* issues to be resolved. In the meantime bump the timeout to
474	* 200ms. Even at slowest clock, this should be sufficient. And
475	* in the working case, a larger timeout makes no difference.
476	*/	607	*/
477	ret = xe_mmio_wait32(gt, GUC_STATUS, GS_UKERNEL_MASK,	608	do {
478	FIELD_PREP(GS_UKERNEL_MASK, XE_GUC_LOAD_STATUS_READY),	609	u32 last_status = status & (GS_UKERNEL_MASK \| GS_BOOTROM_MASK);
479	200000, &status, false);	610	int ret;
480		611
481	if (ret) {	612	/*
482	xe_gt_info(gt, "GuC load failed: status = 0x%08X\n", status);	613	* Wait for any change (intermediate or terminal) in the status register.
483	xe_gt_info(gt, "GuC status: Reset = %u, BootROM = %#X, UKernel = %#X, MIA = %#X, Auth = %#X\n",	614	* Note, the return value is a don't care. The only failure code is timeout
484	REG_FIELD_GET(GS_MIA_IN_RESET, status),	615	* but the timeouts need to be accumulated over all the intermediate partial
485	REG_FIELD_GET(GS_BOOTROM_MASK, status),	616	* timeouts rather than allowing a huge timeout each time. So basically, need
486	REG_FIELD_GET(GS_UKERNEL_MASK, status),	617	* to treat a timeout no different to a value change.
487	REG_FIELD_GET(GS_MIA_MASK, status),	618	*/
488	REG_FIELD_GET(GS_AUTH_STATUS_MASK, status));	619	ret = xe_mmio_wait32_not(gt, GUC_STATUS, GS_UKERNEL_MASK \| GS_BOOTROM_MASK,
489		620	last_status, 1000 * 1000, &status, false);
490	if ((status & GS_BOOTROM_MASK) == GS_BOOTROM_RSA_FAILED) {	621	if (ret < 0)
491	xe_gt_info(gt, "GuC firmware signature verification failed\n");	622	count++;
492	ret = -ENOEXEC;	623	after = ktime_get();
		624	delta = ktime_sub(after, before);
		625	delta_ms = ktime_to_ms(delta);
		626
		627	load_done = guc_load_done(status);
		628	if (load_done != 0)
		629	break;
		630
		631	if (delta_ms >= (GUC_LOAD_RETRY_LIMIT * 1000))
		632	break;
		633
		634	xe_gt_dbg(gt, "load still in progress, timeouts = %d, freq = %dMHz (req %dMHz), status = 0x%08X [0x%02X/%02X]\n",
		635	count, xe_guc_pc_get_act_freq(guc_pc),
		636	guc_pc_get_cur_freq(guc_pc), status,
		637	REG_FIELD_GET(GS_BOOTROM_MASK, status),
		638	REG_FIELD_GET(GS_UKERNEL_MASK, status));
		639	} while (1);
		640
		641	if (load_done != 1) {
		642	u32 ukernel = REG_FIELD_GET(GS_UKERNEL_MASK, status);
		643	u32 bootrom = REG_FIELD_GET(GS_BOOTROM_MASK, status);
		644
		645	xe_gt_err(gt, "load failed: status = 0x%08X, time = %lldms, freq = %dMHz (req %dMHz), done = %d\n",
		646	status, delta_ms, xe_guc_pc_get_act_freq(guc_pc),
		647	guc_pc_get_cur_freq(guc_pc), load_done);
		648	xe_gt_err(gt, "load failed: status: Reset = %d, BootROM = 0x%02X, UKernel = 0x%02X, MIA = 0x%02X, Auth = 0x%02X\n",
		649	REG_FIELD_GET(GS_MIA_IN_RESET, status),
		650	bootrom, ukernel,
		651	REG_FIELD_GET(GS_MIA_MASK, status),
		652	REG_FIELD_GET(GS_AUTH_STATUS_MASK, status));
		653
		654	switch (bootrom) {
		655	case XE_BOOTROM_STATUS_NO_KEY_FOUND:
		656	xe_gt_err(gt, "invalid key requested, header = 0x%08X\n",
		657	xe_mmio_read32(gt, GUC_HEADER_INFO));
		658	break;
		659
		660	case XE_BOOTROM_STATUS_RSA_FAILED:
		661	xe_gt_err(gt, "firmware signature verification failed\n");
		662	break;
		663
		664	case XE_BOOTROM_STATUS_PROD_KEY_CHECK_FAILURE:
		665	xe_gt_err(gt, "firmware production part check failure\n");
		666	break;
493	}	667	}
494		668
495	if (REG_FIELD_GET(GS_UKERNEL_MASK, status) ==	669	switch (ukernel) {
496	XE_GUC_LOAD_STATUS_EXCEPTION) {	670	case XE_GUC_LOAD_STATUS_EXCEPTION:
497	xe_gt_info(gt, "GuC firmware exception. EIP: %#x\n",	671	xe_gt_err(gt, "firmware exception. EIP: %#x\n",
498	xe_mmio_read32(gt, SOFT_SCRATCH(13)));	672	xe_mmio_read32(gt, SOFT_SCRATCH(13)));
499	ret = -ENXIO;	673	break;
		674
		675	case XE_GUC_LOAD_STATUS_INIT_MMIO_SAVE_RESTORE_INVALID:
		676	xe_gt_err(gt, "illegal register in save/restore workaround list\n");
		677	break;
		678
		679	case XE_GUC_LOAD_STATUS_HWCONFIG_START:
		680	xe_gt_err(gt, "still extracting hwconfig table.\n");
		681	break;
500	}	682	}
		683
		684	xe_device_declare_wedged(gt_to_xe(gt));
		685	} else if (delta_ms > GUC_LOAD_TIME_WARN_MS) {
		686	xe_gt_warn(gt, "excessive init time: %lldms! [status = 0x%08X, timeouts = %d]\n",
		687	delta_ms, status, count);
		688	xe_gt_warn(gt, "excessive init time: [freq = %dMHz (req = %dMHz), before = %dMHz, perf_limit_reasons = 0x%08X]\n",
		689	xe_guc_pc_get_act_freq(guc_pc), guc_pc_get_cur_freq(guc_pc),
		690	before_freq, xe_gt_throttle_get_limit_reasons(gt));
501	} else {	691	} else {
502	xe_gt_dbg(gt, "GuC successfully loaded\n");	692	xe_gt_dbg(gt, "init took %lldms, freq = %dMHz (req = %dMHz), before = %dMHz, status = 0x%08X, timeouts = %d\n",
		693	delta_ms, xe_guc_pc_get_act_freq(guc_pc), guc_pc_get_cur_freq(guc_pc),
		694	before_freq, status, count);
503	}	695	}
504
505	return ret;
506	}	696	}
507		697
508	static int __xe_guc_upload(struct xe_guc *guc)	698	static int __xe_guc_upload(struct xe_guc *guc)
@@ -532,9 +722,7 @@ static int __xe_guc_upload(struct xe_guc *guc) · 23 unmodified lines
532	goto out;	722	goto out;
533		723
534	/* Wait for authentication */	724	/* Wait for authentication */
535	ret = guc_wait_ucode(guc);	725	guc_wait_ucode(guc);
536	if (ret)
537	goto out;
538		726
539	xe_uc_fw_change_status(&guc->fw, XE_UC_FIRMWARE_RUNNING);	727	xe_uc_fw_change_status(&guc->fw, XE_UC_FIRMWARE_RUNNING);
540	return 0;	728	return 0;
@@ -544,6 +732,38 @@ out: · 3 unmodified lines
544	return 0 /* FIXME: ret, don't want to stop load currently */;	732	return 0 /* FIXME: ret, don't want to stop load currently */;
545	}	733	}
546		734
		735	static int vf_guc_min_load_for_hwconfig(struct xe_guc *guc)
		736	{
		737	struct xe_gt *gt = guc_to_gt(guc);
		738	int ret;
		739
		740	ret = xe_gt_sriov_vf_bootstrap(gt);
		741	if (ret)
		742	return ret;
		743
		744	ret = xe_gt_sriov_vf_query_config(gt);
		745	if (ret)
		746	return ret;
		747
		748	ret = xe_guc_hwconfig_init(guc);
		749	if (ret)
		750	return ret;
		751
		752	ret = xe_guc_enable_communication(guc);
		753	if (ret)
		754	return ret;
		755
		756	ret = xe_gt_sriov_vf_connect(gt);
		757	if (ret)
		758	return ret;
		759
		760	ret = xe_gt_sriov_vf_query_runtime(gt);
		761	if (ret)
		762	return ret;
		763
		764	return 0;
		765	}
		766
547	/**	767	/**
548	* xe_guc_min_load_for_hwconfig - load minimal GuC and read hwconfig table	768	* xe_guc_min_load_for_hwconfig - load minimal GuC and read hwconfig table
549	* @guc: The GuC object	769	* @guc: The GuC object
@@ -559,6 +779,9 @@ int xe_guc_min_load_for_hwconfig(struct xe_guc *guc) · 9 unmodified lines
559	{	779	{
560	int ret;	780	int ret;
561		781
		782	if (IS_SRIOV_VF(guc_to_xe(guc)))
		783	return vf_guc_min_load_for_hwconfig(guc);
		784
562	xe_guc_ads_populate_minimal(&guc->ads);	785	xe_guc_ads_populate_minimal(&guc->ads);
563		786
564	/* Raise GT freq to speed up HuC/GuC load */	787	/* Raise GT freq to speed up HuC/GuC load */
@@ -891,17 +1114,11 @@ void xe_guc_stop_prepare(struct xe_guc *guc) · 326 unmodified lines
891	XE_WARN_ON(xe_guc_pc_stop(&guc->pc));	1114	XE_WARN_ON(xe_guc_pc_stop(&guc->pc));
892	}	1115	}
893		1116
894	int xe_guc_stop(struct xe_guc *guc)	1117	void xe_guc_stop(struct xe_guc *guc)
895	{	1118	{
896	int ret;
897
898	xe_guc_ct_stop(&guc->ct);	1119	xe_guc_ct_stop(&guc->ct);
899		1120
900	ret = xe_guc_submit_stop(guc);	1121	xe_guc_submit_stop(guc);
901	if (ret)
902	return ret;
903
904	return 0;
905	}	1122	}
906		1123
907	int xe_guc_start(struct xe_guc *guc)	1124	int xe_guc_start(struct xe_guc *guc)
@@ -950,30 +1167,3 @@ void xe_guc_print_info(struct xe_guc guc, struct drm_printer p) · 42 unmodified lines
950	xe_guc_ct_print(&guc->ct, p, false);	1167	xe_guc_ct_print(&guc->ct, p, false);
951	xe_guc_submit_print(guc, p);	1168	xe_guc_submit_print(guc, p);
952	}	1169	}
953
954	/**
955	* xe_guc_in_reset() - Detect if GuC MIA is in reset.
956	* @guc: The GuC object
957	*
958	* This function detects runtime resume from d3cold by leveraging
959	* GUC_STATUS, GUC doesn't get reset during d3hot,
960	* it strictly to be called from RPM resume handler.
961	*
962	* Return: true if failed to get forcewake or GuC MIA is in Reset,
963	* otherwise false.
964	*/
965	bool xe_guc_in_reset(struct xe_guc *guc)
966	{
967	struct xe_gt *gt = guc_to_gt(guc);
968	u32 status;
969	int err;
970
971	err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
972	if (err)
973	return true;
974
975	status = xe_mmio_read32(gt, GUC_STATUS);
976	xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
977
978	return status & GS_MIA_IN_RESET;
979	}

drivers/gpu/drm/xe/xe_guc.h+1−2

 int xe_guc_reset_prepare(struct xe_guc *guc);
 void xe_guc_reset_wait(struct xe_guc *guc);
 void xe_guc_stop_prepare(struct xe_guc *guc);
-int xe_guc_stop(struct xe_guc *guc);
+void xe_guc_stop(struct xe_guc *guc);
 int xe_guc_start(struct xe_guc *guc);
-bool xe_guc_in_reset(struct xe_guc *guc);
 static inline u16 xe_engine_class_to_guc_class(enum xe_engine_class class)
 {

drivers/gpu/drm/xe/xe_guc_ads.c+67−4

@@ -9,13 +9,16 @@ · 8 unmodified lines
9		9
10	#include <generated/xe_wa_oob.h>	10	#include <generated/xe_wa_oob.h>
11		11
		12	#include "abi/guc_actions_abi.h"
12	#include "regs/xe_engine_regs.h"	13	#include "regs/xe_engine_regs.h"
13	#include "regs/xe_gt_regs.h"	14	#include "regs/xe_gt_regs.h"
14	#include "regs/xe_guc_regs.h"	15	#include "regs/xe_guc_regs.h"
15	#include "xe_bo.h"	16	#include "xe_bo.h"
16	#include "xe_gt.h"	17	#include "xe_gt.h"
17	#include "xe_gt_ccs_mode.h"	18	#include "xe_gt_ccs_mode.h"
		19	#include "xe_gt_printk.h"
18	#include "xe_guc.h"	20	#include "xe_guc.h"
		21	#include "xe_guc_ct.h"
19	#include "xe_hw_engine.h"	22	#include "xe_hw_engine.h"
20	#include "xe_lrc.h"	23	#include "xe_lrc.h"
21	#include "xe_map.h"	24	#include "xe_map.h"
@@ -265,7 +268,6 @@ static u32 engine_enable_mask(struct xe_gt *gt, enum xe_engine_class class) · 243 unmodified lines
265		268
266	static size_t calculate_golden_lrc_size(struct xe_guc_ads *ads)	269	static size_t calculate_golden_lrc_size(struct xe_guc_ads *ads)
267	{	270	{
268	struct xe_device *xe = ads_to_xe(ads);
269	struct xe_gt *gt = ads_to_gt(ads);	271	struct xe_gt *gt = ads_to_gt(ads);
270	size_t total_size = 0, alloc_size, real_size;	272	size_t total_size = 0, alloc_size, real_size;
271	int class;	273	int class;
@@ -274,7 +276,7 @@ static size_t calculate_golden_lrc_size(struct xe_guc_ads *ads) · 2 unmodified lines
274	if (!engine_enable_mask(gt, class))	276	if (!engine_enable_mask(gt, class))
275	continue;	277	continue;
276		278
277	real_size = xe_lrc_size(xe, class);	279	real_size = xe_gt_lrc_size(gt, class);
278	alloc_size = PAGE_ALIGN(real_size);	280	alloc_size = PAGE_ALIGN(real_size);
279	total_size += alloc_size;	281	total_size += alloc_size;
280	}	282	}
@@ -440,11 +442,18 @@ int xe_guc_ads_init_post_hwconfig(struct xe_guc_ads *ads) · 159 unmodified lines
440		442
441	static void guc_policies_init(struct xe_guc_ads *ads)	443	static void guc_policies_init(struct xe_guc_ads *ads)
442	{	444	{
		445	struct xe_device *xe = ads_to_xe(ads);
		446	u32 global_flags = 0;
		447
443	ads_blob_write(ads, policies.dpc_promote_time,	448	ads_blob_write(ads, policies.dpc_promote_time,
444	GLOBAL_POLICY_DEFAULT_DPC_PROMOTE_TIME_US);	449	GLOBAL_POLICY_DEFAULT_DPC_PROMOTE_TIME_US);
445	ads_blob_write(ads, policies.max_num_work_items,	450	ads_blob_write(ads, policies.max_num_work_items,
446	GLOBAL_POLICY_MAX_NUM_WI);	451	GLOBAL_POLICY_MAX_NUM_WI);
447	ads_blob_write(ads, policies.global_flags, 0);	452
		453	if (xe->wedged.mode == 2)
		454	global_flags \|= GLOBAL_POLICY_DISABLE_ENGINE_RESET;
		455
		456	ads_blob_write(ads, policies.global_flags, global_flags);
448	ads_blob_write(ads, policies.is_valid, 1);	457	ads_blob_write(ads, policies.is_valid, 1);
449	}	458	}
450		459
@@ -765,7 +774,7 @@ static void guc_populate_golden_lrc(struct xe_guc_ads *ads) · 314 unmodified lines
765		774
766	xe_gt_assert(gt, gt->default_lrc[class]);	775	xe_gt_assert(gt, gt->default_lrc[class]);
767		776
768	real_size = xe_lrc_size(xe, class);	777	real_size = xe_gt_lrc_size(gt, class);
769	alloc_size = PAGE_ALIGN(real_size);	778	alloc_size = PAGE_ALIGN(real_size);
770	total_size += alloc_size;	779	total_size += alloc_size;
771		780
@@ -799,3 +808,57 @@ void xe_guc_ads_populate_post_load(struct xe_guc_ads *ads) · 27 unmodified lines
799	{	808	{
800	guc_populate_golden_lrc(ads);	809	guc_populate_golden_lrc(ads);
801	}	810	}
		811
		812	static int guc_ads_action_update_policies(struct xe_guc_ads *ads, u32 policy_offset)
		813	{
		814	struct xe_guc_ct *ct = &ads_to_guc(ads)->ct;
		815	u32 action[] = {
		816	XE_GUC_ACTION_GLOBAL_SCHED_POLICY_CHANGE,
		817	policy_offset
		818	};
		819
		820	return xe_guc_ct_send(ct, action, ARRAY_SIZE(action), 0, 0);
		821	}
		822
		823	/**
		824	* xe_guc_ads_scheduler_policy_toggle_reset - Toggle reset policy
		825	* @ads: Additional data structures object
		826	*
		827	* This function update the GuC's engine reset policy based on wedged.mode.
		828	*
		829	* Return: 0 on success, and negative error code otherwise.
		830	*/
		831	int xe_guc_ads_scheduler_policy_toggle_reset(struct xe_guc_ads *ads)
		832	{
		833	struct xe_device *xe = ads_to_xe(ads);
		834	struct xe_gt *gt = ads_to_gt(ads);
		835	struct xe_tile *tile = gt_to_tile(gt);
		836	struct guc_policies *policies;
		837	struct xe_bo *bo;
		838	int ret = 0;
		839
		840	policies = kmalloc(sizeof(*policies), GFP_KERNEL);
		841	if (!policies)
		842	return -ENOMEM;
		843
		844	policies->dpc_promote_time = ads_blob_read(ads, policies.dpc_promote_time);
		845	policies->max_num_work_items = ads_blob_read(ads, policies.max_num_work_items);
		846	policies->is_valid = 1;
		847	if (xe->wedged.mode == 2)
		848	policies->global_flags \|= GLOBAL_POLICY_DISABLE_ENGINE_RESET;
		849	else
		850	policies->global_flags &= ~GLOBAL_POLICY_DISABLE_ENGINE_RESET;
		851
		852	bo = xe_managed_bo_create_from_data(xe, tile, policies, sizeof(struct guc_policies),
		853	XE_BO_FLAG_VRAM_IF_DGFX(tile) \|
		854	XE_BO_FLAG_GGTT);
		855	if (IS_ERR(bo)) {
		856	ret = PTR_ERR(bo);
		857	goto out;
		858	}
		859
		860	ret = guc_ads_action_update_policies(ads, xe_bo_ggtt_addr(bo));
		861	out:
		862	kfree(policies);
		863	return ret;
		864	}

drivers/gpu/drm/xe/xe_guc_ads.h+2−1

 #ifndef _XE_GUC_ADS_H_
 #define _XE_GUC_ADS_H_
-#include "xe_guc_ads_types.h"
+struct xe_guc_ads;
 int xe_guc_ads_init(struct xe_guc_ads *ads);
 int xe_guc_ads_init_post_hwconfig(struct xe_guc_ads *ads);
 void xe_guc_ads_populate(struct xe_guc_ads *ads);
 void xe_guc_ads_populate_minimal(struct xe_guc_ads *ads);
 void xe_guc_ads_populate_post_load(struct xe_guc_ads *ads);
+int xe_guc_ads_scheduler_policy_toggle_reset(struct xe_guc_ads *ads);
 #endif

drivers/gpu/drm/xe/xe_guc_ct.c+4−0

 #include "xe_gt_pagefault.h"
 #include "xe_gt_printk.h"
 #include "xe_gt_sriov_pf_control.h"
+#include "xe_gt_sriov_pf_monitor.h"
 #include "xe_gt_tlb_invalidation.h"
 #include "xe_guc.h"
 #include "xe_guc_relay.h"
 	case GUC_ACTION_GUC2PF_VF_STATE_NOTIFY:
 		ret = xe_gt_sriov_pf_control_process_guc2pf(gt, hxg, hxg_len);
 		break;
+	case GUC_ACTION_GUC2PF_ADVERSE_EVENT:
+		ret = xe_gt_sriov_pf_monitor_process_guc2pf(gt, hxg, hxg_len);
+		break;
 	default:
 		xe_gt_err(gt, "unexpected G2H action 0x%04x\n", action);
 	}

drivers/gpu/drm/xe/xe_guc_db_mgr.c+2−1

@@ -106,7 +106,8 @@ int xe_guc_db_mgr_init(struct xe_guc_db_mgr *dbm, unsigned int count) · 105 unmodified lines
106	if (ret)	106	if (ret)
107	return ret;	107	return ret;
108	done:	108	done:
109	xe_gt_dbg(dbm_to_gt(dbm), "using %u doorbell(s)\n", dbm->count);	109	xe_gt_dbg(dbm_to_gt(dbm), "using %u doorbell%s\n",
		110	dbm->count, str_plural(dbm->count));
110	return 0;	111	return 0;
111	}	112	}
112		113

drivers/gpu/drm/xe/xe_guc_id_mgr.c+2−1

@@ -97,7 +97,8 @@ int xe_guc_id_mgr_init(struct xe_guc_id_mgr *idm, unsigned int limit) · 96 unmodified lines
97	if (ret)	97	if (ret)
98	return ret;	98	return ret;
99		99
100	xe_gt_info(idm_to_gt(idm), "using %u GUC ID(s)\n", idm->total);	100	xe_gt_info(idm_to_gt(idm), "using %u GUC ID%s\n",
		101	idm->total, str_plural(idm->total));
101	return 0;	102	return 0;
102	}	103	}
103		104

drivers/gpu/drm/xe/xe_guc_klv_helpers.c+12−0

 #include "abi/guc_klvs_abi.h"
 #include "xe_guc_klv_helpers.h"
+#include "xe_guc_klv_thresholds_set.h"
 #define make_u64(hi, lo) ((u64)((u64)(u32)(hi) << 32 | (u32)(lo)))
 		return "begin_db_id";
 	case GUC_KLV_VF_CFG_BEGIN_CONTEXT_ID_KEY:
 		return "begin_ctx_id";
+	/* VF CFG threshold keys */
+#define define_threshold_key_to_string_case(TAG, NAME, ...)	\
+								\
+	case MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY(TAG):		\
+		return #NAME;
+	/* private: auto-generated case statements */
+	MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_threshold_key_to_string_case)
+#undef define_threshold_key_to_string_case
 	default:
 		return "(unknown)";
 	}

drivers/gpu/drm/xe/xe_guc_klv_helpers.h+14−1

 #ifndef _XE_GUC_KLV_HELPERS_H_
 #define _XE_GUC_KLV_HELPERS_H_
+#include <linux/args.h>
 #include <linux/types.h>
 struct drm_printer;
 	(FIELD_PREP_CONST(GUC_KLV_0_KEY, (key)) | \
 	 FIELD_PREP_CONST(GUC_KLV_0_LEN, (len)))
+/**
+ * MAKE_GUC_KLV_KEY - Prepare KLV KEY name based on unique KLV definition tag.
+ * @TAG: unique tag of the KLV definition
+ */
+#define MAKE_GUC_KLV_KEY(TAG) CONCATENATE(CONCATENATE(GUC_KLV_, TAG), _KEY)
+/**
+ * MAKE_GUC_KLV_LEN - Prepare KLV LEN name based on unique KLV definition tag.
+ * @TAG: unique tag of the KLV definition
+ */
+#define MAKE_GUC_KLV_LEN(TAG) CONCATENATE(CONCATENATE(GUC_KLV_, TAG), _LEN)
 /**
  * PREP_GUC_KLV_TAG - Prepare KLV header value based on unique KLV definition tag.
  * @TAG: unique tag of the KLV definition
  * Return: value of the KLV header (u32).
  */
 #define PREP_GUC_KLV_TAG(TAG) \
-	PREP_GUC_KLV_CONST(GUC_KLV_##TAG##_KEY, GUC_KLV_##TAG##_LEN)
+	PREP_GUC_KLV_CONST(MAKE_GUC_KLV_KEY(TAG), MAKE_GUC_KLV_LEN(TAG))
 #endif

drivers/gpu/drm/xe/xe_guc_klv_thresholds_set.h+64−0

@@ -0,0 +1,64 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GUC_KLV_THRESHOLDS_SET_H_
	7	#define _XE_GUC_KLV_THRESHOLDS_SET_H_
	8
	9	#include "abi/guc_klvs_abi.h"
	10	#include "xe_guc_klv_helpers.h"
	11	#include "xe_guc_klv_thresholds_set_types.h"
	12
	13	/**
	14	* MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY - Prepare the name of the KLV key constant.
	15	* @TAG: unique tag of the GuC threshold KLV key.
	16	*/
	17	#define MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY(TAG) \
	18	MAKE_GUC_KLV_KEY(CONCATENATE(VF_CFG_THRESHOLD_, TAG))
	19
	20	/**
	21	* xe_guc_klv_threshold_key_to_index - Find index of the tracked GuC threshold.
	22	* @key: GuC threshold KLV key.
	23	*
	24	* This translation is automatically generated using &MAKE_XE_GUC_KLV_THRESHOLDS_SET.
	25	* Return: index of the GuC threshold KLV or -1 if not found.
	26	*/
	27	static inline int xe_guc_klv_threshold_key_to_index(u32 key)
	28	{
	29	switch (key) {
	30	#define define_xe_guc_klv_threshold_key_to_index_case(TAG, ...) \
	31	\
	32	case MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY(TAG): \
	33	return MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG);
	34
	35	/* private: auto-generated case statements */
	36	MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_xe_guc_klv_threshold_key_to_index_case)
	37	}
	38	return -1;
	39	#undef define_xe_guc_klv_threshold_key_to_index_case
	40	}
	41
	42	/**
	43	* xe_guc_klv_threshold_index_to_key - Get tracked GuC threshold KLV key.
	44	* @index: GuC threshold KLV index.
	45	*
	46	* This translation is automatically generated using &MAKE_XE_GUC_KLV_THRESHOLDS_SET.
	47	* Return: key of the GuC threshold KLV or 0 on malformed index.
	48	*/
	49	static inline u32 xe_guc_klv_threshold_index_to_key(enum xe_guc_klv_threshold_index index)
	50	{
	51	switch (index) {
	52	#define define_xe_guc_klv_threshold_index_to_key_case(TAG, ...) \
	53	\
	54	case MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG): \
	55	return MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY(TAG);
	56
	57	/* private: auto-generated case statements */
	58	MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_xe_guc_klv_threshold_index_to_key_case)
	59	}
	60	return 0; /* unreachable */
	61	#undef define_xe_guc_klv_threshold_index_to_key_case
	62	}
	63
	64	#endif

drivers/gpu/drm/xe/xe_guc_klv_thresholds_set_types.h+68−0

@@ -0,0 +1,68 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_GUC_KLV_THRESHOLDS_SET_TYPES_H_
	7	#define _XE_GUC_KLV_THRESHOLDS_SET_TYPES_H_
	8
	9	#include "xe_args.h"
	10
	11	/**
	12	* MAKE_XE_GUC_KLV_THRESHOLDS_SET - Generate various GuC thresholds definitions.
	13	* @define: name of the inner macro to expand.
	14	*
	15	* The GuC firmware is able to monitor VF's adverse activity and will notify the
	16	* PF driver once any threshold is exceeded.
	17	*
	18	* This super macro allows various conversions between the GuC adverse event
	19	* threshold KLV definitions and the driver code without repeating similar code
	20	* or risking missing some cases.
	21	*
	22	* For each GuC threshold definition, the inner macro &define will be provided
	23	* with the &TAG, that corresponds to the GuC threshold KLV key name defined by
	24	* ABI and the associated &NAME, that may be used in code or debugfs/sysfs::
	25	*
	26	* define(TAG, NAME)
	27	*/
	28	#define MAKE_XE_GUC_KLV_THRESHOLDS_SET(define) \
	29	define(CAT_ERR, cat_error_count) \
	30	define(ENGINE_RESET, engine_reset_count) \
	31	define(PAGE_FAULT, page_fault_count) \
	32	define(H2G_STORM, guc_time_us) \
	33	define(IRQ_STORM, irq_time_us) \
	34	define(DOORBELL_STORM, doorbell_time_us) \
	35	/* end */
	36
	37	/**
	38	* XE_GUC_KLV_NUM_THRESHOLDS - Number of GuC thresholds KLVs.
	39	*
	40	* Calculated automatically using &MAKE_XE_GUC_KLV_THRESHOLDS_SET.
	41	*/
	42	#define XE_GUC_KLV_NUM_THRESHOLDS \
	43	(CALL_ARGS(COUNT_ARGS, MAKE_XE_GUC_KLV_THRESHOLDS_SET(ARGS_SEP_COMMA)) - 1)
	44
	45	/**
	46	* MAKE_XE_GUC_KLV_THRESHOLD_INDEX - Create enumerator name.
	47	* @TAG: unique TAG of the enum xe_guc_klv_threshold_index.
	48	*/
	49	#define MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG) \
	50	CONCATENATE(XE_GUC_KLV_THRESHOLD_INDEX_, TAG)
	51
	52	/**
	53	* enum xe_guc_klv_threshold_index - Index of the tracked GuC threshold.
	54	*
	55	* This enum is automatically generated using &MAKE_XE_GUC_KLV_THRESHOLDS_SET.
	56	* All these generated enumerators will only be used by the also generated code.
	57	*/
	58	enum xe_guc_klv_threshold_index {
	59	#define define_xe_guc_klv_threshold_index_enum(TAG, ...) \
	60	\
	61	MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG),
	62
	63	/* private: auto-generated enum definitions */
	64	MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_xe_guc_klv_threshold_index_enum)
	65	#undef define_xe_guc_klv_threshold_index_enum
	66	};
	67
	68	#endif

drivers/gpu/drm/xe/xe_guc_pc.c+5−7

 #include "regs/xe_regs.h"
 #include "xe_bo.h"
 #include "xe_device.h"
+#include "xe_force_wake.h"
 #include "xe_gt.h"
 #include "xe_gt_idle.h"
 #include "xe_gt_sysfs.h"
 }
 /**
- * xe_guc_pc_fini - Finalize GuC's Power Conservation component
+ * xe_guc_pc_fini_hw - Finalize GuC's Power Conservation component
- * @drm: DRM device
  * @arg: opaque pointer that should point to Xe_GuC_PC instance
  */
-static void xe_guc_pc_fini(struct drm_device *drm, void *arg)
+static void xe_guc_pc_fini_hw(void *arg)
 {
 	struct xe_guc_pc *pc = arg;
 	struct xe_device *xe = pc_to_xe(pc);
-	if (xe->info.skip_guc_pc) {
+	if (xe_device_wedged(xe))
-		xe_gt_idle_disable_c6(pc_to_gt(pc));
 		return;
-	}
 	XE_WARN_ON(xe_force_wake_get(gt_to_fw(pc_to_gt(pc)), XE_FORCEWAKE_ALL));
 	XE_WARN_ON(xe_guc_pc_gucrc_disable(pc));
 	pc->bo = bo;
-	return drmm_add_action_or_reset(&xe->drm, xe_guc_pc_fini, pc);
+	return devm_add_action_or_reset(xe->drm.dev, xe_guc_pc_fini_hw, pc);
 }

drivers/gpu/drm/xe/xe_guc_pc.h+4−1

 #ifndef _XE_GUC_PC_H_
 #define _XE_GUC_PC_H_
-#include "xe_guc_pc_types.h"
+#include <linux/types.h>
+struct xe_guc_pc;
 int xe_guc_pc_init(struct xe_guc_pc *pc);
 int xe_guc_pc_start(struct xe_guc_pc *pc);
 u64 xe_guc_pc_rc6_residency(struct xe_guc_pc *pc);
 u64 xe_guc_pc_mc6_residency(struct xe_guc_pc *pc);
 void xe_guc_pc_init_early(struct xe_guc_pc *pc);
 #endif /* _XE_GUC_PC_H_ */

drivers/gpu/drm/xe/xe_guc_relay.c+14−3

 #include "xe_device.h"
 #include "xe_gt.h"
 #include "xe_gt_sriov_printk.h"
+#include "xe_gt_sriov_pf_service.h"
 #include "xe_guc.h"
 #include "xe_guc_ct.h"
 #include "xe_guc_hxg_helpers.h"
 static int relay_action_handler(struct xe_guc_relay *relay, u32 origin,
 				const u32 *msg, u32 len, u32 *response, u32 size)
 {
+	struct xe_gt *gt = relay_to_gt(relay);
 	u32 type;
 	int ret;
 	type = FIELD_GET(GUC_HXG_MSG_0_TYPE, msg[0]);
-	/* XXX: PF services will be added later */
+	if (IS_SRIOV_PF(relay_to_xe(relay)))
-	ret = -EOPNOTSUPP;
+		ret = xe_gt_sriov_pf_service_process_request(gt, origin, msg, len, response, size);
+	else
+		ret = -EOPNOTSUPP;
 	if (type == GUC_HXG_TYPE_EVENT)
 		relay_assert(relay, ret <= 0);
 static bool relay_needs_worker(struct xe_guc_relay *relay)
 {
-	return !list_empty(&relay->incoming_actions);
+	bool is_empty;
+	spin_lock(&relay->lock);
+	is_empty = list_empty(&relay->incoming_actions);
+	spin_unlock(&relay->lock);
+	return !is_empty;
 }
 static void relay_kick_worker(struct xe_guc_relay *relay)

drivers/gpu/drm/xe/xe_guc_submit.c+162−119

 #include "xe_macros.h"
 #include "xe_map.h"
 #include "xe_mocs.h"
+#include "xe_pm.h"
 #include "xe_ring_ops_types.h"
 #include "xe_sched_job.h"
 #include "xe_trace.h"
  * engine done being processed).
  */
 #define EXEC_QUEUE_STATE_REGISTERED		(1 << 0)
-#define ENGINE_STATE_ENABLED		(1 << 1)
+#define EXEC_QUEUE_STATE_ENABLED		(1 << 1)
-#define EXEC_QUEUE_STATE_PENDING_ENABLE	(1 << 2)
+#define EXEC_QUEUE_STATE_PENDING_ENABLE		(1 << 2)
 #define EXEC_QUEUE_STATE_PENDING_DISABLE	(1 << 3)
 #define EXEC_QUEUE_STATE_DESTROYED		(1 << 4)
-#define ENGINE_STATE_SUSPENDED		(1 << 5)
+#define EXEC_QUEUE_STATE_SUSPENDED		(1 << 5)
-#define EXEC_QUEUE_STATE_RESET		(1 << 6)
+#define EXEC_QUEUE_STATE_RESET			(1 << 6)
-#define ENGINE_STATE_KILLED		(1 << 7)
+#define EXEC_QUEUE_STATE_KILLED			(1 << 7)
+#define EXEC_QUEUE_STATE_WEDGED			(1 << 8)
 static bool exec_queue_registered(struct xe_exec_queue *q)
 {
 static bool exec_queue_enabled(struct xe_exec_queue *q)
 {
-	return atomic_read(&q->guc->state) & ENGINE_STATE_ENABLED;
+	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_ENABLED;
 }
 static void set_exec_queue_enabled(struct xe_exec_queue *q)
 {
-	atomic_or(ENGINE_STATE_ENABLED, &q->guc->state);
+	atomic_or(EXEC_QUEUE_STATE_ENABLED, &q->guc->state);
 }
 static void clear_exec_queue_enabled(struct xe_exec_queue *q)
 {
-	atomic_and(~ENGINE_STATE_ENABLED, &q->guc->state);
+	atomic_and(~EXEC_QUEUE_STATE_ENABLED, &q->guc->state);
 }
 static bool exec_queue_pending_enable(struct xe_exec_queue *q)
 static bool exec_queue_suspended(struct xe_exec_queue *q)
 {
-	return atomic_read(&q->guc->state) & ENGINE_STATE_SUSPENDED;
+	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_SUSPENDED;
 }
 static void set_exec_queue_suspended(struct xe_exec_queue *q)
 {
-	atomic_or(ENGINE_STATE_SUSPENDED, &q->guc->state);
+	atomic_or(EXEC_QUEUE_STATE_SUSPENDED, &q->guc->state);
 }
 static void clear_exec_queue_suspended(struct xe_exec_queue *q)
 {
-	atomic_and(~ENGINE_STATE_SUSPENDED, &q->guc->state);
+	atomic_and(~EXEC_QUEUE_STATE_SUSPENDED, &q->guc->state);
 }
 static bool exec_queue_reset(struct xe_exec_queue *q)
 static bool exec_queue_killed(struct xe_exec_queue *q)
 {
-	return atomic_read(&q->guc->state) & ENGINE_STATE_KILLED;
+	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_KILLED;
 }
 static void set_exec_queue_killed(struct xe_exec_queue *q)
 {
-	atomic_or(ENGINE_STATE_KILLED, &q->guc->state);
+	atomic_or(EXEC_QUEUE_STATE_KILLED, &q->guc->state);
 }
-static bool exec_queue_killed_or_banned(struct xe_exec_queue *q)
+static bool exec_queue_wedged(struct xe_exec_queue *q)
 {
-	return exec_queue_killed(q) || exec_queue_banned(q);
+	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_WEDGED;
+}
+static void set_exec_queue_wedged(struct xe_exec_queue *q)
+{
+	atomic_or(EXEC_QUEUE_STATE_WEDGED, &q->guc->state);
+}
+static bool exec_queue_killed_or_banned_or_wedged(struct xe_exec_queue *q)
+{
+	return exec_queue_banned(q) || (atomic_read(&q->guc->state) &
+		(EXEC_QUEUE_STATE_WEDGED | EXEC_QUEUE_STATE_KILLED));
 }
 #ifdef CONFIG_PROVE_LOCKING
 	free_submit_wq(guc);
 }
+static void guc_submit_wedged_fini(struct drm_device *drm, void *arg)
+{
+	struct xe_guc *guc = arg;
+	struct xe_exec_queue *q;
+	unsigned long index;
+	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
+		if (exec_queue_wedged(q))
+			xe_exec_queue_put(q);
+}
 static const struct xe_exec_queue_ops guc_exec_queue_ops;
 static void primelockdep(struct xe_guc *guc)
 	fs_reclaim_acquire(GFP_KERNEL);
 	mutex_lock(&guc->submission_state.lock);
-	might_lock(&guc->submission_state.suspend.lock);
 	mutex_unlock(&guc->submission_state.lock);
 	fs_reclaim_release(GFP_KERNEL);
 }
-int xe_guc_submit_init(struct xe_guc *guc)
+/**
+ * xe_guc_submit_init() - Initialize GuC submission.
+ * @guc: the &xe_guc to initialize
+ * @num_ids: number of GuC context IDs to use
+ *
+ * The bare-metal or PF driver can pass ~0 as &num_ids to indicate that all
+ * GuC context IDs supported by the GuC firmware should be used for submission.
+ *
+ * Only VF drivers will have to provide explicit number of GuC context IDs
+ * that they can use for submission.
+ *
+ * Return: 0 on success or a negative error code on failure.
+ */
+int xe_guc_submit_init(struct xe_guc *guc, unsigned int num_ids)
 {
 	struct xe_device *xe = guc_to_xe(guc);
 	struct xe_gt *gt = guc_to_gt(guc);
 	if (err)
 		return err;
-	err = xe_guc_id_mgr_init(&guc->submission_state.idm, ~0);
+	err = xe_guc_id_mgr_init(&guc->submission_state.idm, num_ids);
 	if (err)
 		return err;
 	xa_init(&guc->submission_state.exec_queue_lookup);
-	spin_lock_init(&guc->submission_state.suspend.lock);
-	guc->submission_state.suspend.context = dma_fence_context_alloc(1);
 	primelockdep(guc);
 	return drmm_add_action_or_reset(&xe->drm, guc_submit_fini, guc);
 	xe_map_wr_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
 			field_, val_)
-static void __register_mlrc_engine(struct xe_guc *guc,
+static void __register_mlrc_exec_queue(struct xe_guc *guc,
-				   struct xe_exec_queue *q,
+				       struct xe_exec_queue *q,
-				   struct guc_ctxt_registration_info *info)
+				       struct guc_ctxt_registration_info *info)
 {
 #define MAX_MLRC_REG_SIZE      (13 + XE_HW_ENGINE_MAX_INSTANCE * 2)
 	struct xe_device *xe = guc_to_xe(guc);
 	action[len++] = info->hwlrca_hi;
 	for (i = 1; i < q->width; ++i) {
-		struct xe_lrc *lrc = q->lrc + i;
+		struct xe_lrc *lrc = q->lrc[i];
 		action[len++] = lower_32_bits(xe_lrc_descriptor(lrc));
 		action[len++] = upper_32_bits(xe_lrc_descriptor(lrc));
 	xe_guc_ct_send(&guc->ct, action, len, 0, 0);
 }
-static void __register_engine(struct xe_guc *guc,
+static void __register_exec_queue(struct xe_guc *guc,
-			      struct guc_ctxt_registration_info *info)
+				  struct guc_ctxt_registration_info *info)
 {
 	u32 action[] = {
 		XE_GUC_ACTION_REGISTER_CONTEXT,
 	xe_guc_ct_send(&guc->ct, action, ARRAY_SIZE(action), 0, 0);
 }
-static void register_engine(struct xe_exec_queue *q)
+static void register_exec_queue(struct xe_exec_queue *q)
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
 	struct xe_device *xe = guc_to_xe(guc);
-	struct xe_lrc *lrc = q->lrc;
+	struct xe_lrc *lrc = q->lrc[0];
 	struct guc_ctxt_registration_info info;
 	xe_assert(xe, !exec_queue_registered(q));
 	set_exec_queue_registered(q);
 	trace_xe_exec_queue_register(q);
 	if (xe_exec_queue_is_parallel(q))
-		__register_mlrc_engine(guc, q, &info);
+		__register_mlrc_exec_queue(guc, q, &info);
 	else
-		__register_engine(guc, &info);
+		__register_exec_queue(guc, &info);
 	init_policies(guc, q);
 }
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
 	struct xe_device *xe = guc_to_xe(guc);
-	struct iosys_map map = xe_lrc_parallel_map(q->lrc);
+	struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
 	unsigned int sleep_period_ms = 1;
 #define AVAILABLE_SPACE \
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
 	struct xe_device *xe = guc_to_xe(guc);
-	struct iosys_map map = xe_lrc_parallel_map(q->lrc);
+	struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
 	u32 len_dw = wq_space_until_wrap(q) / sizeof(u32) - 1;
 	if (wq_wait_for_space(q, wq_space_until_wrap(q)))
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
 	struct xe_device *xe = guc_to_xe(guc);
-	struct iosys_map map = xe_lrc_parallel_map(q->lrc);
+	struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
 #define WQ_HEADER_SIZE	4	/* Includes 1 LRC address too */
 	u32 wqi[XE_HW_ENGINE_MAX_INSTANCE + (WQ_HEADER_SIZE - 1)];
 	u32 wqi_size = (q->width + (WQ_HEADER_SIZE - 1)) * sizeof(u32);
 	wqi[i++] = FIELD_PREP(WQ_TYPE_MASK, WQ_TYPE_MULTI_LRC) |
 		FIELD_PREP(WQ_LEN_MASK, len_dw);
-	wqi[i++] = xe_lrc_descriptor(q->lrc);
+	wqi[i++] = xe_lrc_descriptor(q->lrc[0]);
 	wqi[i++] = FIELD_PREP(WQ_GUC_ID_MASK, q->guc->id) |
-		FIELD_PREP(WQ_RING_TAIL_MASK, q->lrc->ring.tail / sizeof(u64));
+		FIELD_PREP(WQ_RING_TAIL_MASK, q->lrc[0]->ring.tail / sizeof(u64));
 	wqi[i++] = 0;
 	for (j = 1; j < q->width; ++j) {
-		struct xe_lrc *lrc = q->lrc + j;
+		struct xe_lrc *lrc = q->lrc[j];
 		wqi[i++] = lrc->ring.tail / sizeof(u64);
 	}
 	xe_device_wmb(xe);
-	map = xe_lrc_parallel_map(q->lrc);
+	map = xe_lrc_parallel_map(q->lrc[0]);
 	parallel_write(xe, map, wq_desc.tail, q->guc->wqi_tail);
 }
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
 	struct xe_device *xe = guc_to_xe(guc);
-	struct xe_lrc *lrc = q->lrc;
+	struct xe_lrc *lrc = q->lrc[0];
 	u32 action[3];
 	u32 g2h_len = 0;
 	u32 num_g2h = 0;
 	if (xe_exec_queue_is_parallel(q))
 		wq_item_append(q);
 	else
-		xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, lrc->ring.tail);
+		xe_lrc_set_ring_tail(lrc, lrc->ring.tail);
 	if (exec_queue_suspended(q) && !xe_exec_queue_is_parallel(q))
 		return;
 	trace_xe_sched_job_run(job);
-	if (!exec_queue_killed_or_banned(q) && !xe_sched_job_is_error(job)) {
+	if (!exec_queue_killed_or_banned_or_wedged(q) && !xe_sched_job_is_error(job)) {
 		if (!exec_queue_registered(q))
-			register_engine(q);
+			register_exec_queue(q);
 		if (!lr)	/* LR jobs are emitted in the exec IOCTL */
 			q->ring_ops->emit_job(job);
 		submit_exec_queue(q);
 {
 	struct xe_sched_job *job = to_xe_sched_job(drm_job);
+	xe_exec_queue_update_run_ticks(job->q);
 	trace_xe_sched_job_free(job);
 	xe_sched_job_put(job);
 }
 		       G2H_LEN_DW_DEREGISTER_CONTEXT, 2);
 }
-static void guc_exec_queue_print(struct xe_exec_queue *q, struct drm_printer *p);
-#if IS_ENABLED(CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE)
-static void simple_error_capture(struct xe_exec_queue *q)
-{
-	struct xe_guc *guc = exec_queue_to_guc(q);
-	struct xe_device *xe = guc_to_xe(guc);
-	struct drm_printer p = drm_err_printer(&xe->drm, NULL);
-	struct xe_hw_engine *hwe;
-	enum xe_hw_engine_id id;
-	u32 adj_logical_mask = q->logical_mask;
-	u32 width_mask = (0x1 << q->width) - 1;
-	int i;
-	bool cookie;
-	if (q->vm && !q->vm->error_capture.capture_once) {
-		q->vm->error_capture.capture_once = true;
-		cookie = dma_fence_begin_signalling();
-		for (i = 0; q->width > 1 && i < XE_HW_ENGINE_MAX_INSTANCE;) {
-			if (adj_logical_mask & BIT(i)) {
-				adj_logical_mask |= width_mask << i;
-				i += q->width;
-			} else {
-				++i;
-			}
-		}
-		if (xe_force_wake_get(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL))
-			xe_gt_info(guc_to_gt(guc),
-				   "failed to get forcewake for error capture");
-		xe_guc_ct_print(&guc->ct, &p, true);
-		guc_exec_queue_print(q, &p);
-		for_each_hw_engine(hwe, guc_to_gt(guc), id) {
-			if (hwe->class != q->hwe->class ||
-			    !(BIT(hwe->logical_instance) & adj_logical_mask))
-				continue;
-			xe_hw_engine_print(hwe, &p);
-		}
-		xe_analyze_vm(&p, q->vm, q->gt->info.id);
-		xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
-		dma_fence_end_signalling(cookie);
-	}
-}
-#else
-static void simple_error_capture(struct xe_exec_queue *q)
-{
-}
-#endif
 static void xe_guc_exec_queue_trigger_cleanup(struct xe_exec_queue *q)
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
 		xe_sched_tdr_queue_imm(&q->guc->sched);
 }
+static bool guc_submit_hint_wedged(struct xe_guc *guc)
+{
+	struct xe_device *xe = guc_to_xe(guc);
+	struct xe_exec_queue *q;
+	unsigned long index;
+	int err;
+	if (xe->wedged.mode != 2)
+		return false;
+	if (xe_device_wedged(xe))
+		return true;
+	xe_device_declare_wedged(xe);
+	xe_guc_submit_reset_prepare(guc);
+	xe_guc_ct_stop(&guc->ct);
+	err = drmm_add_action_or_reset(&guc_to_xe(guc)->drm,
+				       guc_submit_wedged_fini, guc);
+	if (err) {
+		drm_err(&xe->drm, "Failed to register xe_guc_submit clean-up on wedged.mode=2. Although device is wedged.\n");
+		return true; /* Device is wedged anyway */
+	}
+	mutex_lock(&guc->submission_state.lock);
+	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
+		if (xe_exec_queue_get_unless_zero(q))
+			set_exec_queue_wedged(q);
+	mutex_unlock(&guc->submission_state.lock);
+	return true;
+}
 static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)
 {
 	struct xe_guc_exec_queue *ge =
 	struct xe_guc *guc = exec_queue_to_guc(q);
 	struct xe_device *xe = guc_to_xe(guc);
 	struct xe_gpu_scheduler *sched = &ge->sched;
+	bool wedged;
 	xe_assert(xe, xe_exec_queue_is_lr(q));
 	trace_xe_exec_queue_lr_cleanup(q);
+	wedged = guc_submit_hint_wedged(exec_queue_to_guc(q));
 	/* Kill the run_job / process_msg entry points */
 	xe_sched_submission_stop(sched);
 	 * xe_guc_deregister_done_handler() which treats it as an unexpected
 	 * state.
 	 */
-	if (exec_queue_registered(q) && !exec_queue_destroyed(q)) {
+	if (!wedged && exec_queue_registered(q) && !exec_queue_destroyed(q)) {
 		struct xe_guc *guc = exec_queue_to_guc(q);
 		int ret;
 	struct xe_device *xe = guc_to_xe(exec_queue_to_guc(q));
 	int err = -ETIME;
 	int i = 0;
+	bool wedged;
 	/*
 	 * TDR has fired before free job worker. Common if exec queue
 		return DRM_GPU_SCHED_STAT_NOMINAL;
 	}
-	drm_notice(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
+	drm_notice(&xe->drm, "Timedout job: seqno=%u, lrc_seqno=%u, guc_id=%d, flags=0x%lx",
-		   xe_sched_job_seqno(job), q->guc->id, q->flags);
+		   xe_sched_job_seqno(job), xe_sched_job_lrc_seqno(job),
+		   q->guc->id, q->flags);
 	xe_gt_WARN(q->gt, q->flags & EXEC_QUEUE_FLAG_KERNEL,
 		   "Kernel-submitted job timed out\n");
 	xe_gt_WARN(q->gt, q->flags & EXEC_QUEUE_FLAG_VM && !exec_queue_killed(q),
 		   "VM job timed out on non-killed execqueue\n");
-	simple_error_capture(q);
+	if (!exec_queue_killed(q))
-	xe_devcoredump(job);
+		xe_devcoredump(job);
 	trace_xe_sched_job_timedout(job);
+	wedged = guc_submit_hint_wedged(exec_queue_to_guc(q));
 	/* Kill the run_job entry point */
 	xe_sched_submission_stop(sched);
 	 * Kernel jobs should never fail, nor should VM jobs if they do
 	 * somethings has gone wrong and the GT needs a reset
 	 */
-	if (q->flags & EXEC_QUEUE_FLAG_KERNEL ||
+	if (!wedged && (q->flags & EXEC_QUEUE_FLAG_KERNEL ||
-	    (q->flags & EXEC_QUEUE_FLAG_VM && !exec_queue_killed(q))) {
+			(q->flags & EXEC_QUEUE_FLAG_VM && !exec_queue_killed(q)))) {
 		if (!xe_sched_invalidate_job(job, 2)) {
 			xe_sched_add_pending_job(sched, job);
 			xe_sched_submission_start(sched);
 	}
 	/* Engine state now stable, disable scheduling if needed */
-	if (exec_queue_registered(q)) {
+	if (!wedged && exec_queue_registered(q)) {
 		struct xe_guc *guc = exec_queue_to_guc(q);
 		int ret;
 	 */
 	xe_sched_add_pending_job(sched, job);
 	xe_sched_submission_start(sched);
 	xe_guc_exec_queue_trigger_cleanup(q);
 	/* Mark all outstanding jobs as bad, thus completing them */
 	struct xe_exec_queue *q = ge->q;
 	struct xe_guc *guc = exec_queue_to_guc(q);
+	xe_pm_runtime_get(guc_to_xe(guc));
 	trace_xe_exec_queue_destroy(q);
 	if (xe_exec_queue_is_lr(q))
 	kfree(ge);
 	xe_exec_queue_fini(q);
+	xe_pm_runtime_put(guc_to_xe(guc));
 }
 static void guc_exec_queue_fini_async(struct xe_exec_queue *q)
 	INIT_WORK(&q->guc->fini_async, __guc_exec_queue_fini_async);
 	/* We must block on kernel engines so slabs are empty on driver unload */
-	if (q->flags & EXEC_QUEUE_FLAG_PERMANENT)
+	if (q->flags & EXEC_QUEUE_FLAG_PERMANENT || exec_queue_wedged(q))
 		__guc_exec_queue_fini_async(&q->guc->fini_async);
 	else
 		queue_work(system_wq, &q->guc->fini_async);
 static bool guc_exec_queue_allowed_to_change_state(struct xe_exec_queue *q)
 {
-	return !exec_queue_killed_or_banned(q) && exec_queue_registered(q);
+	return !exec_queue_killed_or_banned_or_wedged(q) && exec_queue_registered(q);
 }
 static void __guc_exec_queue_process_msg_set_sched_props(struct xe_sched_msg *msg)
 		  msecs_to_jiffies(q->sched_props.job_timeout_ms);
 	err = xe_sched_init(&ge->sched, &drm_sched_ops, &xe_sched_ops,
 			    get_submit_wq(guc),
-			    q->lrc[0].ring.size / MAX_JOB_SIZE_BYTES, 64,
+			    q->lrc[0]->ring.size / MAX_JOB_SIZE_BYTES, 64,
 			    timeout, guc_to_gt(guc)->ordered_wq, NULL,
 			    q->name, gt_to_xe(q->gt)->drm.dev);
 	if (err)
 {
 	struct xe_sched_msg *msg = q->guc->static_msgs + STATIC_MSG_CLEANUP;
-	if (!(q->flags & EXEC_QUEUE_FLAG_PERMANENT))
+	if (!(q->flags & EXEC_QUEUE_FLAG_PERMANENT) && !exec_queue_wedged(q))
 		guc_exec_queue_add_msg(q, msg, CLEANUP);
 	else
 		__guc_exec_queue_fini(exec_queue_to_guc(q), q);
 {
 	struct xe_sched_msg *msg;
-	if (q->sched_props.priority == priority || exec_queue_killed_or_banned(q))
+	if (q->sched_props.priority == priority ||
+	    exec_queue_killed_or_banned_or_wedged(q))
 		return 0;
 	msg = kmalloc(sizeof(*msg), GFP_KERNEL);
 	struct xe_sched_msg *msg;
 	if (q->sched_props.timeslice_us == timeslice_us ||
-	    exec_queue_killed_or_banned(q))
+	    exec_queue_killed_or_banned_or_wedged(q))
 		return 0;
 	msg = kmalloc(sizeof(*msg), GFP_KERNEL);
 	struct xe_sched_msg *msg;
 	if (q->sched_props.preempt_timeout_us == preempt_timeout_us ||
-	    exec_queue_killed_or_banned(q))
+	    exec_queue_killed_or_banned_or_wedged(q))
 		return 0;
 	msg = kmalloc(sizeof(*msg), GFP_KERNEL);
 {
 	struct xe_sched_msg *msg = q->guc->static_msgs + STATIC_MSG_SUSPEND;
-	if (exec_queue_killed_or_banned(q) || q->guc->suspend_pending)
+	if (exec_queue_killed_or_banned_or_wedged(q) || q->guc->suspend_pending)
 		return -EINVAL;
 	q->guc->suspend_pending = true;
 static bool guc_exec_queue_reset_status(struct xe_exec_queue *q)
 {
-	return exec_queue_reset(q);
+	return exec_queue_reset(q) || exec_queue_killed_or_banned_or_wedged(q);
 }
 /*
 		set_exec_queue_suspended(q);
 		suspend_fence_signal(q);
 	}
-	atomic_and(EXEC_QUEUE_STATE_DESTROYED | ENGINE_STATE_SUSPENDED,
+	atomic_and(EXEC_QUEUE_STATE_DESTROYED | EXEC_QUEUE_STATE_SUSPENDED,
 		   &q->guc->state);
 	q->guc->resume_time = 0;
 	trace_xe_exec_queue_stop(q);
 	 */
 	if (!(q->flags & (EXEC_QUEUE_FLAG_KERNEL | EXEC_QUEUE_FLAG_VM))) {
 		struct xe_sched_job *job = xe_sched_first_pending_job(sched);
+		bool ban = false;
 		if (job) {
 			if ((xe_sched_job_started(job) &&
 			    !xe_sched_job_completed(job)) ||
 			    xe_sched_invalidate_job(job, 2)) {
 				trace_xe_sched_job_ban(job);
-				xe_sched_tdr_queue_imm(&q->guc->sched);
+				ban = true;
-				set_exec_queue_banned(q);
 			}
+		} else if (xe_exec_queue_is_lr(q) &&
+			   (xe_lrc_ring_head(q->lrc[0]) != xe_lrc_ring_tail(q->lrc[0]))) {
+			ban = true;
+		}
+		if (ban) {
+			set_exec_queue_banned(q);
+			xe_guc_exec_queue_trigger_cleanup(q);
 		}
 	}
 }
 	wait_event(guc->ct.wq, !guc_read_stopped(guc));
 }
-int xe_guc_submit_stop(struct xe_guc *guc)
+void xe_guc_submit_stop(struct xe_guc *guc)
 {
 	struct xe_exec_queue *q;
 	unsigned long index;
 	 * creation which is protected by guc->submission_state.lock.
 	 */
-	return 0;
 }
 static void guc_exec_queue_start(struct xe_exec_queue *q)
 {
 	struct xe_gpu_scheduler *sched = &q->guc->sched;
-	if (!exec_queue_killed_or_banned(q)) {
+	if (!exec_queue_killed_or_banned_or_wedged(q)) {
 		int i;
 		trace_xe_exec_queue_resubmit(q);
 		for (i = 0; i < q->width; ++i)
-			xe_lrc_set_ring_head(q->lrc + i, q->lrc[i].ring.tail);
+			xe_lrc_set_ring_head(q->lrc[i], q->lrc[i]->ring.tail);
 		xe_sched_resubmit_jobs(sched);
 	}
 int xe_guc_exec_queue_reset_handler(struct xe_guc *guc, u32 *msg, u32 len)
 {
+	struct xe_gt *gt = guc_to_gt(guc);
 	struct xe_device *xe = guc_to_xe(guc);
 	struct xe_exec_queue *q;
 	u32 guc_id = msg[0];
 	if (unlikely(!q))
 		return -EPROTO;
-	drm_info(&xe->drm, "Engine reset: guc_id=%d", guc_id);
+	xe_gt_info(gt, "Engine reset: engine_class=%s, logical_mask: 0x%x, guc_id=%d",
+		   xe_hw_engine_class_to_str(q->class), q->logical_mask, guc_id);
 	/* FIXME: Do error capture, most likely async */
 int xe_guc_exec_queue_memory_cat_error_handler(struct xe_guc *guc, u32 *msg,
 					       u32 len)
 {
+	struct xe_gt *gt = guc_to_gt(guc);
 	struct xe_device *xe = guc_to_xe(guc);
 	struct xe_exec_queue *q;
 	u32 guc_id = msg[0];
 	if (unlikely(!q))
 		return -EPROTO;
-	drm_dbg(&xe->drm, "Engine memory cat error: guc_id=%d", guc_id);
+	xe_gt_dbg(gt, "Engine memory cat error: engine_class=%s, logical_mask: 0x%x, guc_id=%d",
+		  xe_hw_engine_class_to_str(q->class), q->logical_mask, guc_id);
 	trace_xe_exec_queue_memory_cat_error(q);
 	/* Treat the same as engine reset */
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
 	struct xe_device *xe = guc_to_xe(guc);
-	struct iosys_map map = xe_lrc_parallel_map(q->lrc);
+	struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
 	int i;
 	snapshot->guc.wqi_head = q->guc->wqi_head;
 	if (snapshot->lrc) {
 		for (i = 0; i < q->width; ++i) {
-			struct xe_lrc *lrc = q->lrc + i;
+			struct xe_lrc *lrc = q->lrc[i];
 			snapshot->lrc[i] = xe_lrc_snapshot_capture(lrc);
 		}

drivers/gpu/drm/xe/xe_guc_submit.h+2−2

@@ -12,11 +12,11 @@ struct drm_printer; · 11 unmodified lines
12	struct xe_exec_queue;	12	struct xe_exec_queue;
13	struct xe_guc;	13	struct xe_guc;
14		14
15	int xe_guc_submit_init(struct xe_guc *guc);	15	int xe_guc_submit_init(struct xe_guc *guc, unsigned int num_ids);
16		16
17	int xe_guc_submit_reset_prepare(struct xe_guc *guc);	17	int xe_guc_submit_reset_prepare(struct xe_guc *guc);
18	void xe_guc_submit_reset_wait(struct xe_guc *guc);	18	void xe_guc_submit_reset_wait(struct xe_guc *guc);
19	int xe_guc_submit_stop(struct xe_guc *guc);	19	void xe_guc_submit_stop(struct xe_guc *guc);
20	int xe_guc_submit_start(struct xe_guc *guc);	20	int xe_guc_submit_start(struct xe_guc *guc);
21		21
22	int xe_guc_sched_done_handler(struct xe_guc guc, u32 msg, u32 len);	22	int xe_guc_sched_done_handler(struct xe_guc guc, u32 msg, u32 len);

drivers/gpu/drm/xe/xe_guc_types.h+0−9

@@ -72,15 +72,6 @@ struct xe_guc { · 71 unmodified lines
72	atomic_t stopped;	72	atomic_t stopped;
73	/** @submission_state.lock: protects submission state */	73	/** @submission_state.lock: protects submission state */
74	struct mutex lock;	74	struct mutex lock;
75	/** @submission_state.suspend: suspend fence state */
76	struct {
77	/** @submission_state.suspend.lock: suspend fences lock */
78	spinlock_t lock;
79	/** @submission_state.suspend.context: suspend fences context */
80	u64 context;
81	/** @submission_state.suspend.seqno: suspend fences seqno */
82	u32 seqno;
83	} suspend;
84	#ifdef CONFIG_PROVE_LOCKING	75	#ifdef CONFIG_PROVE_LOCKING
85	#define NUM_SUBMIT_WQ 256	76	#define NUM_SUBMIT_WQ 256
86	/** @submission_state.submit_wq_pool: submission ordered workqueues pool */	77	/** @submission_state.submit_wq_pool: submission ordered workqueues pool */

drivers/gpu/drm/xe/xe_huc.c+2−0

@@ -5,6 +5,8 @@ · 4 unmodified lines
5		5
6	#include "xe_huc.h"	6	#include "xe_huc.h"
7		7
		8	#include <linux/delay.h>
		9
8	#include <drm/drm_managed.h>	10	#include <drm/drm_managed.h>
9		11
10	#include "abi/gsc_pxp_commands_abi.h"	12	#include "abi/gsc_pxp_commands_abi.h"

drivers/gpu/drm/xe/xe_huc.h+2−1

@@ -6,9 +6,10 @@ · 5 unmodified lines
6	#ifndef _XE_HUC_H_	6	#ifndef _XE_HUC_H_
7	#define _XE_HUC_H_	7	#define _XE_HUC_H_
8		8
9	#include "xe_huc_types.h"	9	#include <linux/types.h>
10		10
11	struct drm_printer;	11	struct drm_printer;
		12	struct xe_huc;
12		13
13	enum xe_huc_auth_types {	14	enum xe_huc_auth_types {
14	XE_HUC_AUTH_VIA_GUC = 0,	15	XE_HUC_AUTH_VIA_GUC = 0,

drivers/gpu/drm/xe/xe_hw_engine.c+183−10

@@ -18,6 +18,7 @@ · 17 unmodified lines
18	#include "xe_gt.h"	18	#include "xe_gt.h"
19	#include "xe_gt_ccs_mode.h"	19	#include "xe_gt_ccs_mode.h"
20	#include "xe_gt_printk.h"	20	#include "xe_gt_printk.h"
		21	#include "xe_gt_mcr.h"
21	#include "xe_gt_topology.h"	22	#include "xe_gt_topology.h"
22	#include "xe_hw_fence.h"	23	#include "xe_hw_fence.h"
23	#include "xe_irq.h"	24	#include "xe_irq.h"
@@ -267,7 +268,7 @@ static void hw_engine_fini(struct drm_device drm, void arg) · 243 unmodified lines
267		268
268	if (hwe->exl_port)	269	if (hwe->exl_port)
269	xe_execlist_port_destroy(hwe->exl_port);	270	xe_execlist_port_destroy(hwe->exl_port);
270	xe_lrc_finish(&hwe->kernel_lrc);	271	xe_lrc_put(hwe->kernel_lrc);
271		272
272	hwe->gt = NULL;	273	hwe->gt = NULL;
273	}	274	}
@@ -341,7 +342,7 @@ xe_hw_engine_setup_default_lrc_state(struct xe_hw_engine *hwe) · 67 unmodified lines
341	u32 blit_cctl_val = REG_FIELD_PREP(BLIT_CCTL_DST_MOCS_MASK, mocs_write_idx) \|	342	u32 blit_cctl_val = REG_FIELD_PREP(BLIT_CCTL_DST_MOCS_MASK, mocs_write_idx) \|
342	REG_FIELD_PREP(BLIT_CCTL_SRC_MOCS_MASK, mocs_read_idx);	343	REG_FIELD_PREP(BLIT_CCTL_SRC_MOCS_MASK, mocs_read_idx);
343	struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe);	344	struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe);
344	const struct xe_rtp_entry_sr lrc_was[] = {	345	const struct xe_rtp_entry_sr lrc_setup[] = {
345	/*	346	/*
346	* Some blitter commands do not have a field for MOCS, those	347	* Some blitter commands do not have a field for MOCS, those
347	* commands will use MOCS index pointed by BLIT_CCTL.	348	* commands will use MOCS index pointed by BLIT_CCTL.
@@ -373,7 +374,7 @@ xe_hw_engine_setup_default_lrc_state(struct xe_hw_engine *hwe) · 25 unmodified lines
373	{}	374	{}
374	};	375	};
375		376
376	xe_rtp_process_to_sr(&ctx, lrc_was, &hwe->reg_lrc);	377	xe_rtp_process_to_sr(&ctx, lrc_setup, &hwe->reg_lrc);
377	}	378	}
378		379
379	static void	380	static void
@@ -526,9 +527,11 @@ static int hw_engine_init(struct xe_gt gt, struct xe_hw_engine hwe, · 146 unmodified lines
526	goto err_name;	527	goto err_name;
527	}	528	}
528		529
529	err = xe_lrc_init(&hwe->kernel_lrc, hwe, NULL, NULL, SZ_16K);	530	hwe->kernel_lrc = xe_lrc_create(hwe, NULL, SZ_16K);
530	if (err)	531	if (IS_ERR(hwe->kernel_lrc)) {
		532	err = PTR_ERR(hwe->kernel_lrc);
531	goto err_hwsp;	533	goto err_hwsp;
		534	}
532		535
533	if (!xe_device_uc_enabled(xe)) {	536	if (!xe_device_uc_enabled(xe)) {
534	hwe->exl_port = xe_execlist_port_create(xe, hwe);	537	hwe->exl_port = xe_execlist_port_create(xe, hwe);
@@ -553,7 +556,7 @@ static int hw_engine_init(struct xe_gt gt, struct xe_hw_engine hwe, · 18 unmodified lines
553	return drmm_add_action_or_reset(&xe->drm, hw_engine_fini, hwe);	556	return drmm_add_action_or_reset(&xe->drm, hw_engine_fini, hwe);
554		557
555	err_kernel_lrc:	558	err_kernel_lrc:
556	xe_lrc_finish(&hwe->kernel_lrc);	559	xe_lrc_put(hwe->kernel_lrc);
557	err_hwsp:	560	err_hwsp:
558	xe_bo_unpin_map_no_vm(hwe->hwsp);	561	xe_bo_unpin_map_no_vm(hwe->hwsp);
559	err_name:	562	err_name:
@@ -716,6 +719,11 @@ static void check_gsc_availability(struct xe_gt *gt) · 156 unmodified lines
716	*/	719	*/
717	if (!xe_uc_fw_is_available(&gt->uc.gsc.fw)) {	720	if (!xe_uc_fw_is_available(&gt->uc.gsc.fw)) {
718	gt->info.engine_mask &= ~BIT(XE_HW_ENGINE_GSCCS0);	721	gt->info.engine_mask &= ~BIT(XE_HW_ENGINE_GSCCS0);
		722
		723	/* interrupts where previously enabled, so turn them off */
		724	xe_mmio_write32(gt, GUNIT_GSC_INTR_ENABLE, 0);
		725	xe_mmio_write32(gt, GUNIT_GSC_INTR_MASK, ~0);
		726
719	drm_info(&xe->drm, "gsccs disabled due to lack of FW\n");	727	drm_info(&xe->drm, "gsccs disabled due to lack of FW\n");
720	}	728	}
721	}	729	}
@@ -766,6 +774,57 @@ void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec) · 44 unmodified lines
766	xe_hw_fence_irq_run(hwe->fence_irq);	774	xe_hw_fence_irq_run(hwe->fence_irq);
767	}	775	}
768		776
		777	static bool
		778	is_slice_common_per_gslice(struct xe_device *xe)
		779	{
		780	return GRAPHICS_VERx100(xe) >= 1255;
		781	}
		782
		783	static void
		784	xe_hw_engine_snapshot_instdone_capture(struct xe_hw_engine *hwe,
		785	struct xe_hw_engine_snapshot *snapshot)
		786	{
		787	struct xe_gt *gt = hwe->gt;
		788	struct xe_device *xe = gt_to_xe(gt);
		789	unsigned int dss;
		790	u16 group, instance;
		791
		792	snapshot->reg.instdone.ring = hw_engine_mmio_read32(hwe, RING_INSTDONE(0));
		793
		794	if (snapshot->hwe->class != XE_ENGINE_CLASS_RENDER)
		795	return;
		796
		797	if (is_slice_common_per_gslice(xe) == false) {
		798	snapshot->reg.instdone.slice_common[0] =
		799	xe_mmio_read32(gt, SC_INSTDONE);
		800	snapshot->reg.instdone.slice_common_extra[0] =
		801	xe_mmio_read32(gt, SC_INSTDONE_EXTRA);
		802	snapshot->reg.instdone.slice_common_extra2[0] =
		803	xe_mmio_read32(gt, SC_INSTDONE_EXTRA2);
		804	} else {
		805	for_each_geometry_dss(dss, gt, group, instance) {
		806	snapshot->reg.instdone.slice_common[dss] =
		807	xe_gt_mcr_unicast_read(gt, XEHPG_SC_INSTDONE, group, instance);
		808	snapshot->reg.instdone.slice_common_extra[dss] =
		809	xe_gt_mcr_unicast_read(gt, XEHPG_SC_INSTDONE_EXTRA, group, instance);
		810	snapshot->reg.instdone.slice_common_extra2[dss] =
		811	xe_gt_mcr_unicast_read(gt, XEHPG_SC_INSTDONE_EXTRA2, group, instance);
		812	}
		813	}
		814
		815	for_each_geometry_dss(dss, gt, group, instance) {
		816	snapshot->reg.instdone.sampler[dss] =
		817	xe_gt_mcr_unicast_read(gt, SAMPLER_INSTDONE, group, instance);
		818	snapshot->reg.instdone.row[dss] =
		819	xe_gt_mcr_unicast_read(gt, ROW_INSTDONE, group, instance);
		820
		821	if (GRAPHICS_VERx100(xe) >= 1255)
		822	snapshot->reg.instdone.geom_svg[dss] =
		823	xe_gt_mcr_unicast_read(gt, XEHPG_INSTDONE_GEOM_SVGUNIT,
		824	group, instance);
		825	}
		826	}
		827
769	/**	828	/**
770	* xe_hw_engine_snapshot_capture - Take a quick snapshot of the HW Engine.	829	* xe_hw_engine_snapshot_capture - Take a quick snapshot of the HW Engine.
771	* @hwe: Xe HW Engine.	830	* @hwe: Xe HW Engine.
@@ -780,6 +839,7 @@ struct xe_hw_engine_snapshot * · 8 unmodified lines
780	xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe)	839	xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe)
781	{	840	{
782	struct xe_hw_engine_snapshot *snapshot;	841	struct xe_hw_engine_snapshot *snapshot;
		842	size_t len;
783	u64 val;	843	u64 val;
784		844
785	if (!xe_hw_engine_is_valid(hwe))	845	if (!xe_hw_engine_is_valid(hwe))
@@ -790,8 +850,30 @@ xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe) · 4 unmodified lines
790	if (!snapshot)	850	if (!snapshot)
791	return NULL;	851	return NULL;
792		852
		853	/* Because XE_MAX_DSS_FUSE_BITS is defined in xe_gt_types.h and it
		854	* includes xe_hw_engine_types.h the length of this 3 registers can't be
		855	* set in struct xe_hw_engine_snapshot, so here doing additional
		856	* allocations.
		857	*/
		858	len = (XE_MAX_DSS_FUSE_BITS * sizeof(u32));
		859	snapshot->reg.instdone.slice_common = kzalloc(len, GFP_ATOMIC);
		860	snapshot->reg.instdone.slice_common_extra = kzalloc(len, GFP_ATOMIC);
		861	snapshot->reg.instdone.slice_common_extra2 = kzalloc(len, GFP_ATOMIC);
		862	snapshot->reg.instdone.sampler = kzalloc(len, GFP_ATOMIC);
		863	snapshot->reg.instdone.row = kzalloc(len, GFP_ATOMIC);
		864	snapshot->reg.instdone.geom_svg = kzalloc(len, GFP_ATOMIC);
		865	if (!snapshot->reg.instdone.slice_common \|\|
		866	!snapshot->reg.instdone.slice_common_extra \|\|
		867	!snapshot->reg.instdone.slice_common_extra2 \|\|
		868	!snapshot->reg.instdone.sampler \|\|
		869	!snapshot->reg.instdone.row \|\|
		870	!snapshot->reg.instdone.geom_svg) {
		871	xe_hw_engine_snapshot_free(snapshot);
		872	return NULL;
		873	}
		874
793	snapshot->name = kstrdup(hwe->name, GFP_ATOMIC);	875	snapshot->name = kstrdup(hwe->name, GFP_ATOMIC);
794	snapshot->class = hwe->class;	876	snapshot->hwe = hwe;
795	snapshot->logical_instance = hwe->logical_instance;	877	snapshot->logical_instance = hwe->logical_instance;
796	snapshot->forcewake.domain = hwe->domain;	878	snapshot->forcewake.domain = hwe->domain;
797	snapshot->forcewake.ref = xe_force_wake_ref(gt_to_fw(hwe->gt),	879	snapshot->forcewake.ref = xe_force_wake_ref(gt_to_fw(hwe->gt),
@@ -828,6 +910,15 @@ xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe) · 30 unmodified lines
828	snapshot->reg.ring_hwstam = hw_engine_mmio_read32(hwe, RING_HWSTAM(0));	910	snapshot->reg.ring_hwstam = hw_engine_mmio_read32(hwe, RING_HWSTAM(0));
829	snapshot->reg.ring_hws_pga = hw_engine_mmio_read32(hwe, RING_HWS_PGA(0));	911	snapshot->reg.ring_hws_pga = hw_engine_mmio_read32(hwe, RING_HWS_PGA(0));
830	snapshot->reg.ring_start = hw_engine_mmio_read32(hwe, RING_START(0));	912	snapshot->reg.ring_start = hw_engine_mmio_read32(hwe, RING_START(0));
		913	if (GRAPHICS_VERx100(hwe->gt->tile->xe) >= 2000) {
		914	val = hw_engine_mmio_read32(hwe, RING_START_UDW(0));
		915	snapshot->reg.ring_start \|= val << 32;
		916	}
		917	if (xe_gt_has_indirect_ring_state(hwe->gt)) {
		918	snapshot->reg.indirect_ring_state =
		919	hw_engine_mmio_read32(hwe, INDIRECT_RING_STATE(0));
		920	}
		921
831	snapshot->reg.ring_head =	922	snapshot->reg.ring_head =
832	hw_engine_mmio_read32(hwe, RING_HEAD(0)) & HEAD_ADDR;	923	hw_engine_mmio_read32(hwe, RING_HEAD(0)) & HEAD_ADDR;
833	snapshot->reg.ring_tail =	924	snapshot->reg.ring_tail =
@@ -841,13 +932,57 @@ xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe) · 7 unmodified lines
841	snapshot->reg.ring_emr = hw_engine_mmio_read32(hwe, RING_EMR(0));	932	snapshot->reg.ring_emr = hw_engine_mmio_read32(hwe, RING_EMR(0));
842	snapshot->reg.ring_eir = hw_engine_mmio_read32(hwe, RING_EIR(0));	933	snapshot->reg.ring_eir = hw_engine_mmio_read32(hwe, RING_EIR(0));
843	snapshot->reg.ipehr = hw_engine_mmio_read32(hwe, RING_IPEHR(0));	934	snapshot->reg.ipehr = hw_engine_mmio_read32(hwe, RING_IPEHR(0));
		935	xe_hw_engine_snapshot_instdone_capture(hwe, snapshot);
844		936
845	if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)	937	if (snapshot->hwe->class == XE_ENGINE_CLASS_COMPUTE)
846	snapshot->reg.rcu_mode = xe_mmio_read32(hwe->gt, RCU_MODE);	938	snapshot->reg.rcu_mode = xe_mmio_read32(hwe->gt, RCU_MODE);
847		939
848	return snapshot;	940	return snapshot;
849	}	941	}
850		942
		943	static void
		944	xe_hw_engine_snapshot_instdone_print(struct xe_hw_engine_snapshot snapshot, struct drm_printer p)
		945	{
		946	struct xe_gt *gt = snapshot->hwe->gt;
		947	struct xe_device *xe = gt_to_xe(gt);
		948	u16 group, instance;
		949	unsigned int dss;
		950
		951	drm_printf(p, "\tRING_INSTDONE: 0x%08x\n", snapshot->reg.instdone.ring);
		952
		953	if (snapshot->hwe->class != XE_ENGINE_CLASS_RENDER)
		954	return;
		955
		956	if (is_slice_common_per_gslice(xe) == false) {
		957	drm_printf(p, "\tSC_INSTDONE[0]: 0x%08x\n",
		958	snapshot->reg.instdone.slice_common[0]);
		959	drm_printf(p, "\tSC_INSTDONE_EXTRA[0]: 0x%08x\n",
		960	snapshot->reg.instdone.slice_common_extra[0]);
		961	drm_printf(p, "\tSC_INSTDONE_EXTRA2[0]: 0x%08x\n",
		962	snapshot->reg.instdone.slice_common_extra2[0]);
		963	} else {
		964	for_each_geometry_dss(dss, gt, group, instance) {
		965	drm_printf(p, "\tSC_INSTDONE[%u]: 0x%08x\n", dss,
		966	snapshot->reg.instdone.slice_common[dss]);
		967	drm_printf(p, "\tSC_INSTDONE_EXTRA[%u]: 0x%08x\n", dss,
		968	snapshot->reg.instdone.slice_common_extra[dss]);
		969	drm_printf(p, "\tSC_INSTDONE_EXTRA2[%u]: 0x%08x\n", dss,
		970	snapshot->reg.instdone.slice_common_extra2[dss]);
		971	}
		972	}
		973
		974	for_each_geometry_dss(dss, gt, group, instance) {
		975	drm_printf(p, "\tSAMPLER_INSTDONE[%u]: 0x%08x\n", dss,
		976	snapshot->reg.instdone.sampler[dss]);
		977	drm_printf(p, "\tROW_INSTDONE[%u]: 0x%08x\n", dss,
		978	snapshot->reg.instdone.row[dss]);
		979
		980	if (GRAPHICS_VERx100(xe) >= 1255)
		981	drm_printf(p, "\tINSTDONE_GEOM_SVGUNIT[%u]: 0x%08x\n",
		982	dss, snapshot->reg.instdone.geom_svg[dss]);
		983	}
		984	}
		985
851	/**	986	/**
852	* xe_hw_engine_snapshot_print - Print out a given Xe HW Engine snapshot.	987	* xe_hw_engine_snapshot_print - Print out a given Xe HW Engine snapshot.
853	* @snapshot: Xe HW Engine snapshot object.	988	* @snapshot: Xe HW Engine snapshot object.
@@ -872,7 +1007,7 @@ void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot, · 18 unmodified lines
872	snapshot->reg.ring_execlist_status);	1007	snapshot->reg.ring_execlist_status);
873	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS: 0x%016llx\n",	1008	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS: 0x%016llx\n",
874	snapshot->reg.ring_execlist_sq_contents);	1009	snapshot->reg.ring_execlist_sq_contents);
875	drm_printf(p, "\tRING_START: 0x%08x\n", snapshot->reg.ring_start);	1010	drm_printf(p, "\tRING_START: 0x%016llx\n", snapshot->reg.ring_start);
876	drm_printf(p, "\tRING_HEAD: 0x%08x\n", snapshot->reg.ring_head);	1011	drm_printf(p, "\tRING_HEAD: 0x%08x\n", snapshot->reg.ring_head);
877	drm_printf(p, "\tRING_TAIL: 0x%08x\n", snapshot->reg.ring_tail);	1012	drm_printf(p, "\tRING_TAIL: 0x%08x\n", snapshot->reg.ring_tail);
878	drm_printf(p, "\tRING_CTL: 0x%08x\n", snapshot->reg.ring_ctl);	1013	drm_printf(p, "\tRING_CTL: 0x%08x\n", snapshot->reg.ring_ctl);
@@ -886,10 +1021,15 @@ void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot, · 7 unmodified lines
886	drm_printf(p, "\tACTHD: 0x%016llx\n", snapshot->reg.ring_acthd);	1021	drm_printf(p, "\tACTHD: 0x%016llx\n", snapshot->reg.ring_acthd);
887	drm_printf(p, "\tBBADDR: 0x%016llx\n", snapshot->reg.ring_bbaddr);	1022	drm_printf(p, "\tBBADDR: 0x%016llx\n", snapshot->reg.ring_bbaddr);
888	drm_printf(p, "\tDMA_FADDR: 0x%016llx\n", snapshot->reg.ring_dma_fadd);	1023	drm_printf(p, "\tDMA_FADDR: 0x%016llx\n", snapshot->reg.ring_dma_fadd);
		1024	drm_printf(p, "\tINDIRECT_RING_STATE: 0x%08x\n",
		1025	snapshot->reg.indirect_ring_state);
889	drm_printf(p, "\tIPEHR: 0x%08x\n", snapshot->reg.ipehr);	1026	drm_printf(p, "\tIPEHR: 0x%08x\n", snapshot->reg.ipehr);
890	if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)	1027	xe_hw_engine_snapshot_instdone_print(snapshot, p);
		1028
		1029	if (snapshot->hwe->class == XE_ENGINE_CLASS_COMPUTE)
891	drm_printf(p, "\tRCU_MODE: 0x%08x\n",	1030	drm_printf(p, "\tRCU_MODE: 0x%08x\n",
892	snapshot->reg.rcu_mode);	1031	snapshot->reg.rcu_mode);
		1032	drm_puts(p, "\n");
893	}	1033	}
894		1034
895	/**	1035	/**
@@ -904,6 +1044,12 @@ void xe_hw_engine_snapshot_free(struct xe_hw_engine_snapshot *snapshot) · 8 unmodified lines
904	if (!snapshot)	1044	if (!snapshot)
905	return;	1045	return;
906		1046
		1047	kfree(snapshot->reg.instdone.slice_common);
		1048	kfree(snapshot->reg.instdone.slice_common_extra);
		1049	kfree(snapshot->reg.instdone.slice_common_extra2);
		1050	kfree(snapshot->reg.instdone.sampler);
		1051	kfree(snapshot->reg.instdone.row);
		1052	kfree(snapshot->reg.instdone.geom_svg);
907	kfree(snapshot->name);	1053	kfree(snapshot->name);
908	kfree(snapshot);	1054	kfree(snapshot);
909	}	1055	}
@@ -955,3 +1101,30 @@ bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe) · 45 unmodified lines
955	return xe->info.has_usm && hwe->class == XE_ENGINE_CLASS_COPY &&	1101	return xe->info.has_usm && hwe->class == XE_ENGINE_CLASS_COPY &&
956	hwe->instance == gt->usm.reserved_bcs_instance;	1102	hwe->instance == gt->usm.reserved_bcs_instance;
957	}	1103	}
		1104
		1105	const char *xe_hw_engine_class_to_str(enum xe_engine_class class)
		1106	{
		1107	switch (class) {
		1108	case XE_ENGINE_CLASS_RENDER:
		1109	return "rcs";
		1110	case XE_ENGINE_CLASS_VIDEO_DECODE:
		1111	return "vcs";
		1112	case XE_ENGINE_CLASS_VIDEO_ENHANCE:
		1113	return "vecs";
		1114	case XE_ENGINE_CLASS_COPY:
		1115	return "bcs";
		1116	case XE_ENGINE_CLASS_OTHER:
		1117	return "other";
		1118	case XE_ENGINE_CLASS_COMPUTE:
		1119	return "ccs";
		1120	case XE_ENGINE_CLASS_MAX:
		1121	break;
		1122	}
		1123
		1124	return NULL;
		1125	}
		1126
		1127	u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe)
		1128	{
		1129	return xe_mmio_read64_2x32(hwe->gt, RING_TIMESTAMP(hwe->mmio_base));
		1130	}

drivers/gpu/drm/xe/xe_hw_engine.h+3−0

@@ -67,4 +67,7 @@ static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe) · 66 unmodified lines
67	return hwe->name;	67	return hwe->name;
68	}	68	}
69		69
		70	const char *xe_hw_engine_class_to_str(enum xe_engine_class class);
		71	u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe);
		72
70	#endif	73	#endif

drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c+6−24

 	NULL
 };
-static void kobj_xe_hw_engine_class_fini(struct drm_device *drm, void *arg)
+static void kobj_xe_hw_engine_class_fini(void *arg)
 {
 	struct kobject *kobj = arg;
 	}
 	keclass->xe = xe;
-	err = drmm_add_action_or_reset(&xe->drm, kobj_xe_hw_engine_class_fini,
+	err = devm_add_action_or_reset(xe->drm.dev, kobj_xe_hw_engine_class_fini,
 				       &keclass->base);
 	if (err)
 		return NULL;
 	return keclass;
 }
-static void hw_engine_class_defaults_fini(struct drm_device *drm, void *arg)
+static void hw_engine_class_defaults_fini(void *arg)
 {
 	struct kobject *kobj = arg;
 	if (err)
 		goto err_object;
-	return drmm_add_action_or_reset(&xe->drm, hw_engine_class_defaults_fini, kobj);
+	return devm_add_action_or_reset(xe->drm.dev, hw_engine_class_defaults_fini, kobj);
 err_object:
 	kobject_put(kobj);
 	.sysfs_ops = &xe_hw_engine_class_sysfs_ops,
 };
-static void hw_engine_class_sysfs_fini(struct drm_device *drm, void *arg)
+static void hw_engine_class_sysfs_fini(void *arg)
 {
 	struct kobject *kobj = arg;
 	kobject_put(kobj);
 }
-static const char *xe_hw_engine_class_to_str(enum xe_engine_class class)
-{
-	switch (class) {
-	case XE_ENGINE_CLASS_RENDER:
-		return "rcs";
-	case XE_ENGINE_CLASS_VIDEO_DECODE:
-		return "vcs";
-	case XE_ENGINE_CLASS_VIDEO_ENHANCE:
-		return "vecs";
-	case XE_ENGINE_CLASS_COPY:
-		return "bcs";
-	case XE_ENGINE_CLASS_COMPUTE:
-		return "ccs";
-	default:
-		return NULL;
-	}
-}
 /**
  * xe_hw_engine_class_sysfs_init - Init HW engine classes on GT.
  * @gt: Xe GT.
 			goto err_object;
 	}
-	return drmm_add_action_or_reset(&xe->drm, hw_engine_class_sysfs_fini, kobj);
+	return devm_add_action_or_reset(xe->drm.dev, hw_engine_class_sysfs_fini, kobj);
 err_object:
 	kobject_put(kobj);

drivers/gpu/drm/xe/xe_hw_engine_types.h+22−4

 	/** @hwsp: hardware status page buffer object */
 	struct xe_bo *hwsp;
 	/** @kernel_lrc: Kernel LRC (should be replaced /w an xe_engine) */
-	struct xe_lrc kernel_lrc;
+	struct xe_lrc *kernel_lrc;
 	/** @exl_port: execlists port */
 	struct xe_execlist_port *exl_port;
 	/** @fence_irq: fence IRQ to run when a hw engine IRQ is received */
 struct xe_hw_engine_snapshot {
 	/** @name: name of the hw engine */
 	char *name;
-	/** @class: class of this hw engine */
+	/** @hwe: hw engine */
-	enum xe_engine_class class;
+	struct xe_hw_engine *hwe;
 	/** @logical_instance: logical instance of this hw engine */
 	u16 logical_instance;
 	/** @forcewake: Force Wake information snapshot */
 		/** @reg.ring_hws_pga: RING_HWS_PGA */
 		u32 ring_hws_pga;
 		/** @reg.ring_start: RING_START */
-		u32 ring_start;
+		u64 ring_start;
 		/** @reg.ring_head: RING_HEAD */
 		u32 ring_head;
 		/** @reg.ring_tail: RING_TAIL */
 		u32 ring_emr;
 		/** @reg.ring_eir: RING_EIR */
 		u32 ring_eir;
+		/** @reg.indirect_ring_state: INDIRECT_RING_STATE */
+		u32 indirect_ring_state;
 		/** @reg.ipehr: IPEHR */
 		u32 ipehr;
 		/** @reg.rcu_mode: RCU_MODE */
 		u32 rcu_mode;
+		struct {
+			/** @reg.instdone.ring: RING_INSTDONE */
+			u32 ring;
+			/** @reg.instdone.slice_common: SC_INSTDONE */
+			u32 *slice_common;
+			/** @reg.instdone.slice_common_extra: SC_INSTDONE_EXTRA */
+			u32 *slice_common_extra;
+			/** @reg.instdone.slice_common_extra2: SC_INSTDONE_EXTRA2 */
+			u32 *slice_common_extra2;
+			/** @reg.instdone.sampler: SAMPLER_INSTDONE */
+			u32 *sampler;
+			/** @reg.instdone.row: ROW_INSTDONE */
+			u32 *row;
+			/** @reg.instdone.geom_svg: INSTDONE_GEOM_SVGUNIT */
+			u32 *geom_svg;
+		} instdone;
 	} reg;
 };

drivers/gpu/drm/xe/xe_hw_fence.c+47−12

@@ -208,23 +208,58 @@ static struct xe_hw_fence to_xe_hw_fence(struct dma_fence fence) · 207 unmodified lines
208	return container_of(fence, struct xe_hw_fence, dma);	208	return container_of(fence, struct xe_hw_fence, dma);
209	}	209	}
210		210
211	struct xe_hw_fence xe_hw_fence_create(struct xe_hw_fence_ctx ctx,	211	/**
212	struct iosys_map seqno_map)	212	* xe_hw_fence_alloc() - Allocate an hw fence.
		213	*
		214	* Allocate but don't initialize an hw fence.
		215	*
		216	* Return: Pointer to the allocated fence or
		217	* negative error pointer on error.
		218	*/
		219	struct dma_fence *xe_hw_fence_alloc(void)
213	{	220	{
214	struct xe_hw_fence *fence;	221	struct xe_hw_fence *hw_fence = fence_alloc();
215		222
216	fence = fence_alloc();	223	if (!hw_fence)
217	if (!fence)
218	return ERR_PTR(-ENOMEM);	224	return ERR_PTR(-ENOMEM);
219		225
220	fence->ctx = ctx;	226	return &hw_fence->dma;
221	fence->seqno_map = seqno_map;	227	}
222	INIT_LIST_HEAD(&fence->irq_link);
223		228
224	dma_fence_init(&fence->dma, &xe_hw_fence_ops, &ctx->irq->lock,	229	/**
225	ctx->dma_fence_ctx, ctx->next_seqno++);	230	* xe_hw_fence_free() - Free an hw fence.
		231	* @fence: Pointer to the fence to free.
		232	*
		233	* Frees an hw fence that hasn't yet been
		234	* initialized.
		235	*/
		236	void xe_hw_fence_free(struct dma_fence *fence)
		237	{
		238	fence_free(&fence->rcu);
		239	}
226		240
227	trace_xe_hw_fence_create(fence);	241	/**
		242	* xe_hw_fence_init() - Initialize an hw fence.
		243	* @fence: Pointer to the fence to initialize.
		244	* @ctx: Pointer to the struct xe_hw_fence_ctx fence context.
		245	* @seqno_map: Pointer to the map into where the seqno is blitted.
		246	*
		247	* Initializes a pre-allocated hw fence.
		248	* After initialization, the fence is subject to normal
		249	* dma-fence refcounting.
		250	*/
		251	void xe_hw_fence_init(struct dma_fence fence, struct xe_hw_fence_ctx ctx,
		252	struct iosys_map seqno_map)
		253	{
		254	struct xe_hw_fence *hw_fence =
		255	container_of(fence, typeof(*hw_fence), dma);
		256
		257	hw_fence->ctx = ctx;
		258	hw_fence->seqno_map = seqno_map;
		259	INIT_LIST_HEAD(&hw_fence->irq_link);
		260
		261	dma_fence_init(fence, &xe_hw_fence_ops, &ctx->irq->lock,
		262	ctx->dma_fence_ctx, ctx->next_seqno++);
228		263
229	return fence;	264	trace_xe_hw_fence_create(hw_fence);
230	}	265	}

drivers/gpu/drm/xe/xe_hw_fence.h+5−2

 			  struct xe_hw_fence_irq *irq, const char *name);
 void xe_hw_fence_ctx_finish(struct xe_hw_fence_ctx *ctx);
-struct xe_hw_fence *xe_hw_fence_create(struct xe_hw_fence_ctx *ctx,
+struct dma_fence *xe_hw_fence_alloc(void);
-				       struct iosys_map seqno_map);
+void xe_hw_fence_free(struct dma_fence *fence);
+void xe_hw_fence_init(struct dma_fence *fence, struct xe_hw_fence_ctx *ctx,
+		      struct iosys_map seqno_map);
 #endif

drivers/gpu/drm/xe/xe_hwmon.c+29−8

 	switch (hwmon_reg) {
 	case REG_PKG_RAPL_LIMIT:
-		if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG)
+		if (xe->info.platform == XE_BATTLEMAGE) {
+			if (channel == CHANNEL_PKG)
+				return BMG_PACKAGE_RAPL_LIMIT;
+			else
+				return BMG_PLATFORM_POWER_LIMIT;
+		} else if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG) {
 			return PVC_GT0_PACKAGE_RAPL_LIMIT;
-		else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG))
+		} else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG)) {
 			return PCU_CR_PACKAGE_RAPL_LIMIT;
+		}
 		break;
 	case REG_PKG_POWER_SKU:
-		if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG)
+		if (xe->info.platform == XE_BATTLEMAGE)
+			return BMG_PACKAGE_POWER_SKU;
+		else if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG)
 			return PVC_GT0_PACKAGE_POWER_SKU;
 		else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG))
 			return PCU_CR_PACKAGE_POWER_SKU;
 		break;
 	case REG_PKG_POWER_SKU_UNIT:
-		if (xe->info.platform == XE_PVC)
+		if (xe->info.platform == XE_BATTLEMAGE)
+			return BMG_PACKAGE_POWER_SKU_UNIT;
+		else if (xe->info.platform == XE_PVC)
 			return PVC_GT0_PACKAGE_POWER_SKU_UNIT;
 		else if (xe->info.platform == XE_DG2)
 			return PCU_CR_PACKAGE_POWER_SKU_UNIT;
 			return GT_PERF_STATUS;
 		break;
 	case REG_PKG_ENERGY_STATUS:
-		if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG)
+		if (xe->info.platform == XE_BATTLEMAGE) {
+			if (channel == CHANNEL_PKG)
+				return BMG_PACKAGE_ENERGY_STATUS;
+			else
+				return BMG_PLATFORM_ENERGY_STATUS;
+		} else if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG) {
 			return PVC_GT0_PLATFORM_ENERGY_STATUS;
-		else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG))
+		} else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG)) {
 			return PCU_CR_PACKAGE_ENERGY_STATUS;
+		}
 		break;
 	default:
 		drm_warn(&xe->drm, "Unknown xe hwmon reg id: %d\n", hwmon_reg);
 {
 	u32 uval;
+	/* hwmon sysfs attribute of current available only for package */
+	if (channel != CHANNEL_PKG)
+		return 0;
 	switch (attr) {
 	case hwmon_curr_crit:
-	case hwmon_curr_label:
-		if (channel == CHANNEL_PKG)
 			return (xe_hwmon_pcode_read_i1(hwmon->gt, &uval) ||
 				(uval & POWER_SETUP_I1_WATTS)) ? 0 : 0644;
+	case hwmon_curr_label:
+			return (xe_hwmon_pcode_read_i1(hwmon->gt, &uval) ||
+				(uval & POWER_SETUP_I1_WATTS)) ? 0 : 0444;
 		break;
 	default:
 		return 0;

drivers/gpu/drm/xe/xe_irq.c+3−3

@@ -663,7 +663,7 @@ static irq_handler_t xe_irq_handler(struct xe_device *xe) · 662 unmodified lines
663	return xelp_irq_handler;	663	return xelp_irq_handler;
664	}	664	}
665		665
666	static void irq_uninstall(struct drm_device drm, void arg)	666	static void irq_uninstall(void *arg)
667	{	667	{
668	struct xe_device *xe = arg;	668	struct xe_device *xe = arg;
669	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);	669	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
@@ -723,7 +723,7 @@ int xe_irq_install(struct xe_device *xe) · 53 unmodified lines
723		723
724	xe_irq_postinstall(xe);	724	xe_irq_postinstall(xe);
725		725
726	err = drmm_add_action_or_reset(&xe->drm, irq_uninstall, xe);	726	err = devm_add_action_or_reset(xe->drm.dev, irq_uninstall, xe);
727	if (err)	727	if (err)
728	goto free_irq_handler;	728	goto free_irq_handler;
729		729
@@ -737,7 +737,7 @@ free_irq_handler: · 7 unmodified lines
737		737
738	void xe_irq_shutdown(struct xe_device *xe)	738	void xe_irq_shutdown(struct xe_device *xe)
739	{	739	{
740	irq_uninstall(&xe->drm, xe);	740	irq_uninstall(xe);
741	}	741	}
742		742
743	void xe_irq_suspend(struct xe_device *xe)	743	void xe_irq_suspend(struct xe_device *xe)

drivers/gpu/drm/xe/xe_lrc.c+264−41

@@ -11,7 +11,6 @@ · 10 unmodified lines
11	#include "instructions/xe_gfxpipe_commands.h"	11	#include "instructions/xe_gfxpipe_commands.h"
12	#include "instructions/xe_gfx_state_commands.h"	12	#include "instructions/xe_gfx_state_commands.h"
13	#include "regs/xe_engine_regs.h"	13	#include "regs/xe_engine_regs.h"
14	#include "regs/xe_gpu_commands.h"
15	#include "regs/xe_lrc_layout.h"	14	#include "regs/xe_lrc_layout.h"
16	#include "xe_bb.h"	15	#include "xe_bb.h"
17	#include "xe_bo.h"	16	#include "xe_bo.h"
@@ -34,12 +33,15 @@ · 16 unmodified lines
34	#define LRC_ENGINE_CLASS GENMASK_ULL(63, 61)	33	#define LRC_ENGINE_CLASS GENMASK_ULL(63, 61)
35	#define LRC_ENGINE_INSTANCE GENMASK_ULL(53, 48)	34	#define LRC_ENGINE_INSTANCE GENMASK_ULL(53, 48)
36		35
		36	#define LRC_INDIRECT_RING_STATE_SIZE SZ_4K
		37
37	struct xe_lrc_snapshot {	38	struct xe_lrc_snapshot {
38	struct xe_bo *lrc_bo;	39	struct xe_bo *lrc_bo;
39	void *lrc_snapshot;	40	void *lrc_snapshot;
40	unsigned long lrc_size, lrc_offset;	41	unsigned long lrc_size, lrc_offset;
41		42
42	u32 context_desc;	43	u32 context_desc;
		44	u32 indirect_context_desc;
43	u32 head;	45	u32 head;
44	struct {	46	struct {
45	u32 internal;	47	u32 internal;
@@ -55,20 +57,25 @@ lrc_to_xe(struct xe_lrc *lrc) · 9 unmodified lines
55	return gt_to_xe(lrc->fence_ctx.gt);	57	return gt_to_xe(lrc->fence_ctx.gt);
56	}	58	}
57		59
58	size_t xe_lrc_size(struct xe_device *xe, enum xe_engine_class class)	60	size_t xe_gt_lrc_size(struct xe_gt *gt, enum xe_engine_class class)
59	{	61	{
		62	struct xe_device *xe = gt_to_xe(gt);
		63	size_t size;
		64
60	switch (class) {	65	switch (class) {
61	case XE_ENGINE_CLASS_RENDER:	66	case XE_ENGINE_CLASS_RENDER:
62	if (GRAPHICS_VER(xe) >= 20)	67	if (GRAPHICS_VER(xe) >= 20)
63	return 4 * SZ_4K;	68	size = 4 * SZ_4K;
64	else	69	else
65	return 14 * SZ_4K;	70	size = 14 * SZ_4K;
		71	break;
66	case XE_ENGINE_CLASS_COMPUTE:	72	case XE_ENGINE_CLASS_COMPUTE:
67	/* 14 pages since graphics_ver == 11 */	73	/* 14 pages since graphics_ver == 11 */
68	if (GRAPHICS_VER(xe) >= 20)	74	if (GRAPHICS_VER(xe) >= 20)
69	return 3 * SZ_4K;	75	size = 3 * SZ_4K;
70	else	76	else
71	return 14 * SZ_4K;	77	size = 14 * SZ_4K;
		78	break;
72	default:	79	default:
73	WARN(1, "Unknown engine class: %d", class);	80	WARN(1, "Unknown engine class: %d", class);
74	fallthrough;	81	fallthrough;
@@ -76,8 +83,14 @@ size_t xe_lrc_size(struct xe_device *xe, enum xe_engine_class class) · 1 unmodified line
76	case XE_ENGINE_CLASS_VIDEO_DECODE:	83	case XE_ENGINE_CLASS_VIDEO_DECODE:
77	case XE_ENGINE_CLASS_VIDEO_ENHANCE:	84	case XE_ENGINE_CLASS_VIDEO_ENHANCE:
78	case XE_ENGINE_CLASS_OTHER:	85	case XE_ENGINE_CLASS_OTHER:
79	return 2 * SZ_4K;	86	size = 2 * SZ_4K;
80	}	87	}
		88
		89	/* Add indirect ring state page */
		90	if (xe_gt_has_indirect_ring_state(gt))
		91	size += LRC_INDIRECT_RING_STATE_SIZE;
		92
		93	return size;
81	}	94	}
82		95
83	/*	96	/*
@@ -508,6 +521,32 @@ static const u8 xe2_xcs_offsets[] = { · 424 unmodified lines
508	0	521	0
509	};	522	};
510		523
		524	static const u8 xe2_indirect_ring_state_offsets[] = {
		525	NOP(1), /* [0x00] */
		526	LRI(5, POSTED), /* [0x01] */
		527	REG(0x034), /* [0x02] RING_BUFFER_HEAD */
		528	REG(0x030), /* [0x04] RING_BUFFER_TAIL */
		529	REG(0x038), /* [0x06] RING_BUFFER_START */
		530	REG(0x048), /* [0x08] RING_BUFFER_START_UDW */
		531	REG(0x03c), /* [0x0a] RING_BUFFER_CONTROL */
		532
		533	NOP(5), /* [0x0c] */
		534	LRI(9, POSTED), /* [0x11] */
		535	REG(0x168), /* [0x12] BB_ADDR_UDW */
		536	REG(0x140), /* [0x14] BB_ADDR */
		537	REG(0x110), /* [0x16] BB_STATE */
		538	REG16(0x588), /* [0x18] BB_STACK_WRITE_PORT */
		539	REG16(0x588), /* [0x20] BB_STACK_WRITE_PORT */
		540	REG16(0x588), /* [0x22] BB_STACK_WRITE_PORT */
		541	REG16(0x588), /* [0x24] BB_STACK_WRITE_PORT */
		542	REG16(0x588), /* [0x26] BB_STACK_WRITE_PORT */
		543	REG16(0x588), /* [0x28] BB_STACK_WRITE_PORT */
		544
		545	NOP(12), /* [0x00] */
		546
		547	0
		548	};
		549
511	#undef REG16	550	#undef REG16
512	#undef REG	551	#undef REG
513	#undef LRI	552	#undef LRI
@@ -546,6 +585,10 @@ static void set_context_control(u32 regs, struct xe_hw_engine hwe) · 32 unmodified lines
546	regs[CTX_CONTEXT_CONTROL] = _MASKED_BIT_ENABLE(CTX_CTRL_INHIBIT_SYN_CTX_SWITCH \|	585	regs[CTX_CONTEXT_CONTROL] = _MASKED_BIT_ENABLE(CTX_CTRL_INHIBIT_SYN_CTX_SWITCH \|
547	CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT);	586	CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT);
548		587
		588	if (xe_gt_has_indirect_ring_state(hwe->gt))
		589	regs[CTX_CONTEXT_CONTROL] \|=
		590	_MASKED_BIT_ENABLE(CTX_CTRL_INDIRECT_RING_STATE_ENABLE);
		591
549	/* TODO: Timestamp */	592	/* TODO: Timestamp */
550	}	593	}
551		594
@@ -589,6 +632,11 @@ static void reset_stop_ring(u32 regs, struct xe_hw_engine hwe) · 37 unmodified lines
589	regs[x + 1] \|= STOP_RING << 16;	632	regs[x + 1] \|= STOP_RING << 16;
590	}	633	}
591		634
		635	static inline bool xe_lrc_has_indirect_ring_state(struct xe_lrc *lrc)
		636	{
		637	return lrc->flags & XE_LRC_FLAG_INDIRECT_RING_STATE;
		638	}
		639
592	static inline u32 __xe_lrc_ring_offset(struct xe_lrc *lrc)	640	static inline u32 __xe_lrc_ring_offset(struct xe_lrc *lrc)
593	{	641	{
594	return 0;	642	return 0;
@@ -643,6 +691,12 @@ static inline u32 __xe_lrc_regs_offset(struct xe_lrc *lrc) · 48 unmodified lines
643	return xe_lrc_pphwsp_offset(lrc) + LRC_PPHWSP_SIZE;	691	return xe_lrc_pphwsp_offset(lrc) + LRC_PPHWSP_SIZE;
644	}	692	}
645		693
		694	static inline u32 __xe_lrc_indirect_ring_offset(struct xe_lrc *lrc)
		695	{
		696	/* Indirect ring state page is at the very end of LRC */
		697	return lrc->size - LRC_INDIRECT_RING_STATE_SIZE;
		698	}
		699
646	#define DECL_MAP_ADDR_HELPERS(elem) \	700	#define DECL_MAP_ADDR_HELPERS(elem) \
647	static inline struct iosys_map __xe_lrc_##elem##_map(struct xe_lrc *lrc) \	701	static inline struct iosys_map __xe_lrc_##elem##_map(struct xe_lrc *lrc) \
648	{ \	702	{ \
@@ -663,6 +717,7 @@ DECL_MAP_ADDR_HELPERS(seqno) · 14 unmodified lines
663	DECL_MAP_ADDR_HELPERS(regs)	717	DECL_MAP_ADDR_HELPERS(regs)
664	DECL_MAP_ADDR_HELPERS(start_seqno)	718	DECL_MAP_ADDR_HELPERS(start_seqno)
665	DECL_MAP_ADDR_HELPERS(parallel)	719	DECL_MAP_ADDR_HELPERS(parallel)
		720	DECL_MAP_ADDR_HELPERS(indirect_ring)
666		721
667	#undef DECL_MAP_ADDR_HELPERS	722	#undef DECL_MAP_ADDR_HELPERS
668		723
@@ -671,6 +726,35 @@ u32 xe_lrc_ggtt_addr(struct xe_lrc *lrc) · 2 unmodified lines
671	return __xe_lrc_pphwsp_ggtt_addr(lrc);	726	return __xe_lrc_pphwsp_ggtt_addr(lrc);
672	}	727	}
673		728
		729	u32 xe_lrc_indirect_ring_ggtt_addr(struct xe_lrc *lrc)
		730	{
		731	if (!xe_lrc_has_indirect_ring_state(lrc))
		732	return 0;
		733
		734	return __xe_lrc_indirect_ring_ggtt_addr(lrc);
		735	}
		736
		737	static u32 xe_lrc_read_indirect_ctx_reg(struct xe_lrc *lrc, int reg_nr)
		738	{
		739	struct xe_device *xe = lrc_to_xe(lrc);
		740	struct iosys_map map;
		741
		742	map = __xe_lrc_indirect_ring_map(lrc);
		743	iosys_map_incr(&map, reg_nr * sizeof(u32));
		744	return xe_map_read32(xe, &map);
		745	}
		746
		747	static void xe_lrc_write_indirect_ctx_reg(struct xe_lrc *lrc,
		748	int reg_nr, u32 val)
		749	{
		750	struct xe_device *xe = lrc_to_xe(lrc);
		751	struct iosys_map map;
		752
		753	map = __xe_lrc_indirect_ring_map(lrc);
		754	iosys_map_incr(&map, reg_nr * sizeof(u32));
		755	xe_map_write32(xe, &map, val);
		756	}
		757
674	u32 xe_lrc_read_ctx_reg(struct xe_lrc *lrc, int reg_nr)	758	u32 xe_lrc_read_ctx_reg(struct xe_lrc *lrc, int reg_nr)
675	{	759	{
676	struct xe_device *xe = lrc_to_xe(lrc);	760	struct xe_device *xe = lrc_to_xe(lrc);
@@ -693,20 +777,25 @@ void xe_lrc_write_ctx_reg(struct xe_lrc *lrc, int reg_nr, u32 val) · 16 unmodified lines
693		777
694	static void empty_lrc_data(struct xe_hw_engine hwe)	778	static void empty_lrc_data(struct xe_hw_engine hwe)
695	{	779	{
696	struct xe_device *xe = gt_to_xe(hwe->gt);	780	struct xe_gt *gt = hwe->gt;
697	void *data;	781	void *data;
698	u32 *regs;	782	u32 *regs;
699		783
700	data = kzalloc(xe_lrc_size(xe, hwe->class), GFP_KERNEL);	784	data = kzalloc(xe_gt_lrc_size(gt, hwe->class), GFP_KERNEL);
701	if (!data)	785	if (!data)
702	return NULL;	786	return NULL;
703		787
704	/* 1st page: Per-Process of HW status Page */	788	/* 1st page: Per-Process of HW status Page */
705	regs = data + LRC_PPHWSP_SIZE;	789	regs = data + LRC_PPHWSP_SIZE;
706	set_offsets(regs, reg_offsets(xe, hwe->class), hwe);	790	set_offsets(regs, reg_offsets(gt_to_xe(gt), hwe->class), hwe);
707	set_context_control(regs, hwe);	791	set_context_control(regs, hwe);
708	set_memory_based_intr(regs, hwe);	792	set_memory_based_intr(regs, hwe);
709	reset_stop_ring(regs, hwe);	793	reset_stop_ring(regs, hwe);
		794	if (xe_gt_has_indirect_ring_state(gt)) {
		795	regs = data + xe_gt_lrc_size(gt, hwe->class) -
		796	LRC_INDIRECT_RING_STATE_SIZE;
		797	set_offsets(regs, xe2_indirect_ring_state_offsets, hwe);
		798	}
710		799
711	return data;	800	return data;
712	}	801	}
@@ -719,11 +808,20 @@ static void xe_lrc_set_ppgtt(struct xe_lrc lrc, struct xe_vm vm) · 6 unmodified lines
719	xe_lrc_write_ctx_reg(lrc, CTX_PDP0_LDW, lower_32_bits(desc));	808	xe_lrc_write_ctx_reg(lrc, CTX_PDP0_LDW, lower_32_bits(desc));
720	}	809	}
721		810
		811	static void xe_lrc_finish(struct xe_lrc *lrc)
		812	{
		813	xe_hw_fence_ctx_finish(&lrc->fence_ctx);
		814	xe_bo_lock(lrc->bo, false);
		815	xe_bo_unpin(lrc->bo);
		816	xe_bo_unlock(lrc->bo);
		817	xe_bo_put(lrc->bo);
		818	}
		819
722	#define PVC_CTX_ASID (0x2e + 1)	820	#define PVC_CTX_ASID (0x2e + 1)
723	#define PVC_CTX_ACC_CTR_THOLD (0x2a + 1)	821	#define PVC_CTX_ACC_CTR_THOLD (0x2a + 1)
724		822
725	int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe,	823	static int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe,
726	struct xe_exec_queue q, struct xe_vm vm, u32 ring_size)	824	struct xe_vm *vm, u32 ring_size)
727	{	825	{
728	struct xe_gt *gt = hwe->gt;	826	struct xe_gt *gt = hwe->gt;
729	struct xe_tile *tile = gt_to_tile(gt);	827	struct xe_tile *tile = gt_to_tile(gt);
@@ -731,26 +829,32 @@ int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe, · 1 unmodified line
731	struct iosys_map map;	829	struct iosys_map map;
732	void *init_data = NULL;	830	void *init_data = NULL;
733	u32 arb_enable;	831	u32 arb_enable;
		832	u32 lrc_size;
734	int err;	833	int err;
735		834
		835	kref_init(&lrc->refcount);
736	lrc->flags = 0;	836	lrc->flags = 0;
		837	lrc_size = ring_size + xe_gt_lrc_size(gt, hwe->class);
		838	if (xe_gt_has_indirect_ring_state(gt))
		839	lrc->flags \|= XE_LRC_FLAG_INDIRECT_RING_STATE;
737		840
738	/*	841	/*
739	* FIXME: Perma-pinning LRC as we don't yet support moving GGTT address	842	* FIXME: Perma-pinning LRC as we don't yet support moving GGTT address
740	* via VM bind calls.	843	* via VM bind calls.
741	*/	844	*/
742	lrc->bo = xe_bo_create_pin_map(xe, tile, vm,	845	lrc->bo = xe_bo_create_pin_map(xe, tile, vm, lrc_size,
743	ring_size + xe_lrc_size(xe, hwe->class),	846	ttm_bo_type_kernel,
744	ttm_bo_type_kernel,	847	XE_BO_FLAG_VRAM_IF_DGFX(tile) \|
745	XE_BO_FLAG_VRAM_IF_DGFX(tile) \|	848	XE_BO_FLAG_GGTT \|
746	XE_BO_FLAG_GGTT \|	849	XE_BO_FLAG_GGTT_INVALIDATE);
747	XE_BO_FLAG_GGTT_INVALIDATE);
748	if (IS_ERR(lrc->bo))	850	if (IS_ERR(lrc->bo))
749	return PTR_ERR(lrc->bo);	851	return PTR_ERR(lrc->bo);
750		852
		853	lrc->size = lrc_size;
751	lrc->tile = gt_to_tile(hwe->gt);	854	lrc->tile = gt_to_tile(hwe->gt);
752	lrc->ring.size = ring_size;	855	lrc->ring.size = ring_size;
753	lrc->ring.tail = 0;	856	lrc->ring.tail = 0;
		857	lrc->ctx_timestamp = 0;
754		858
755	xe_hw_fence_ctx_init(&lrc->fence_ctx, hwe->gt,	859	xe_hw_fence_ctx_init(&lrc->fence_ctx, hwe->gt,
756	hwe->fence_irq, hwe->name);	860	hwe->fence_irq, hwe->name);
@@ -772,10 +876,10 @@ int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe, · 15 unmodified lines
772	xe_map_memset(xe, &map, 0, 0, LRC_PPHWSP_SIZE); /* PPHWSP */	876	xe_map_memset(xe, &map, 0, 0, LRC_PPHWSP_SIZE); /* PPHWSP */
773	xe_map_memcpy_to(xe, &map, LRC_PPHWSP_SIZE,	877	xe_map_memcpy_to(xe, &map, LRC_PPHWSP_SIZE,
774	gt->default_lrc[hwe->class] + LRC_PPHWSP_SIZE,	878	gt->default_lrc[hwe->class] + LRC_PPHWSP_SIZE,
775	xe_lrc_size(xe, hwe->class) - LRC_PPHWSP_SIZE);	879	xe_gt_lrc_size(gt, hwe->class) - LRC_PPHWSP_SIZE);
776	} else {	880	} else {
777	xe_map_memcpy_to(xe, &map, 0, init_data,	881	xe_map_memcpy_to(xe, &map, 0, init_data,
778	xe_lrc_size(xe, hwe->class));	882	xe_gt_lrc_size(gt, hwe->class));
779	kfree(init_data);	883	kfree(init_data);
780	}	884	}
781		885
@@ -786,11 +890,27 @@ int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe, · 4 unmodified lines
786	xe_drm_client_add_bo(vm->xef->client, lrc->bo);	890	xe_drm_client_add_bo(vm->xef->client, lrc->bo);
787	}	891	}
788		892
789	xe_lrc_write_ctx_reg(lrc, CTX_RING_START, __xe_lrc_ring_ggtt_addr(lrc));	893	if (xe_gt_has_indirect_ring_state(gt)) {
790	xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, 0);	894	xe_lrc_write_ctx_reg(lrc, CTX_INDIRECT_RING_STATE,
791	xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, lrc->ring.tail);	895	__xe_lrc_indirect_ring_ggtt_addr(lrc));
792	xe_lrc_write_ctx_reg(lrc, CTX_RING_CTL,	896
793	RING_CTL_SIZE(lrc->ring.size) \| RING_VALID);	897	xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_START,
		898	__xe_lrc_ring_ggtt_addr(lrc));
		899	xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_START_UDW, 0);
		900	xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_HEAD, 0);
		901	xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_TAIL, lrc->ring.tail);
		902	xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_CTL,
		903	RING_CTL_SIZE(lrc->ring.size) \| RING_VALID);
		904	} else {
		905	xe_lrc_write_ctx_reg(lrc, CTX_RING_START, __xe_lrc_ring_ggtt_addr(lrc));
		906	xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, 0);
		907	xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, lrc->ring.tail);
		908	xe_lrc_write_ctx_reg(lrc, CTX_RING_CTL,
		909	RING_CTL_SIZE(lrc->ring.size) \| RING_VALID);
		910	}
		911
		912	xe_lrc_write_ctx_reg(lrc, CTX_TIMESTAMP, 0);
		913
794	if (xe->info.has_asid && vm)	914	if (xe->info.has_asid && vm)
795	xe_lrc_write_ctx_reg(lrc, PVC_CTX_ASID, vm->usm.asid);	915	xe_lrc_write_ctx_reg(lrc, PVC_CTX_ASID, vm->usm.asid);
796		916
@@ -825,23 +945,81 @@ err_lrc_finish: · 28 unmodified lines
825	return err;	945	return err;
826	}	946	}
827		947
828	void xe_lrc_finish(struct xe_lrc *lrc)	948	/**
		949	* xe_lrc_create - Create a LRC
		950	* @hwe: Hardware Engine
		951	* @vm: The VM (address space)
		952	* @ring_size: LRC ring size
		953	*
		954	* Allocate and initialize the Logical Ring Context (LRC).
		955	*
		956	* Return pointer to created LRC upon success and an error pointer
		957	* upon failure.
		958	*/
		959	struct xe_lrc xe_lrc_create(struct xe_hw_engine hwe, struct xe_vm *vm,
		960	u32 ring_size)
829	{	961	{
830	xe_hw_fence_ctx_finish(&lrc->fence_ctx);	962	struct xe_lrc *lrc;
831	xe_bo_lock(lrc->bo, false);	963	int err;
832	xe_bo_unpin(lrc->bo);	964
833	xe_bo_unlock(lrc->bo);	965	lrc = kzalloc(sizeof(*lrc), GFP_KERNEL);
834	xe_bo_put(lrc->bo);	966	if (!lrc)
		967	return ERR_PTR(-ENOMEM);
		968
		969	err = xe_lrc_init(lrc, hwe, vm, ring_size);
		970	if (err) {
		971	kfree(lrc);
		972	return ERR_PTR(err);
		973	}
		974
		975	return lrc;
		976	}
		977
		978	/**
		979	* xe_lrc_destroy - Destroy the LRC
		980	* @ref: reference to LRC
		981	*
		982	* Called when ref == 0, release resources held by the Logical Ring Context
		983	* (LRC) and free the LRC memory.
		984	*/
		985	void xe_lrc_destroy(struct kref *ref)
		986	{
		987	struct xe_lrc *lrc = container_of(ref, struct xe_lrc, refcount);
		988
		989	xe_lrc_finish(lrc);
		990	kfree(lrc);
		991	}
		992
		993	void xe_lrc_set_ring_tail(struct xe_lrc *lrc, u32 tail)
		994	{
		995	if (xe_lrc_has_indirect_ring_state(lrc))
		996	xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_TAIL, tail);
		997	else
		998	xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, tail);
		999	}
		1000
		1001	u32 xe_lrc_ring_tail(struct xe_lrc *lrc)
		1002	{
		1003	if (xe_lrc_has_indirect_ring_state(lrc))
		1004	return xe_lrc_read_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_TAIL) & TAIL_ADDR;
		1005	else
		1006	return xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL) & TAIL_ADDR;
835	}	1007	}
836		1008
837	void xe_lrc_set_ring_head(struct xe_lrc *lrc, u32 head)	1009	void xe_lrc_set_ring_head(struct xe_lrc *lrc, u32 head)
838	{	1010	{
839	xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, head);	1011	if (xe_lrc_has_indirect_ring_state(lrc))
		1012	xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_HEAD, head);
		1013	else
		1014	xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, head);
840	}	1015	}
841		1016
842	u32 xe_lrc_ring_head(struct xe_lrc *lrc)	1017	u32 xe_lrc_ring_head(struct xe_lrc *lrc)
843	{	1018	{
844	return xe_lrc_read_ctx_reg(lrc, CTX_RING_HEAD) & HEAD_ADDR;	1019	if (xe_lrc_has_indirect_ring_state(lrc))
		1020	return xe_lrc_read_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_HEAD) & HEAD_ADDR;
		1021	else
		1022	return xe_lrc_read_ctx_reg(lrc, CTX_RING_HEAD) & HEAD_ADDR;
845	}	1023	}
846		1024
847	u32 xe_lrc_ring_space(struct xe_lrc *lrc)	1025	u32 xe_lrc_ring_space(struct xe_lrc *lrc)
@@ -901,10 +1079,43 @@ u32 xe_lrc_seqno_ggtt_addr(struct xe_lrc *lrc) · 53 unmodified lines
901	return __xe_lrc_seqno_ggtt_addr(lrc);	1079	return __xe_lrc_seqno_ggtt_addr(lrc);
902	}	1080	}
903		1081
904	struct dma_fence xe_lrc_create_seqno_fence(struct xe_lrc lrc)	1082	/**
		1083	* xe_lrc_alloc_seqno_fence() - Allocate an lrc seqno fence.
		1084	*
		1085	* Allocate but don't initialize an lrc seqno fence.
		1086	*
		1087	* Return: Pointer to the allocated fence or
		1088	* negative error pointer on error.
		1089	*/
		1090	struct dma_fence *xe_lrc_alloc_seqno_fence(void)
905	{	1091	{
906	return &xe_hw_fence_create(&lrc->fence_ctx,	1092	return xe_hw_fence_alloc();
907	__xe_lrc_seqno_map(lrc))->dma;	1093	}
		1094
		1095	/**
		1096	* xe_lrc_free_seqno_fence() - Free an lrc seqno fence.
		1097	* @fence: Pointer to the fence to free.
		1098	*
		1099	* Frees an lrc seqno fence that hasn't yet been
		1100	* initialized.
		1101	*/
		1102	void xe_lrc_free_seqno_fence(struct dma_fence *fence)
		1103	{
		1104	xe_hw_fence_free(fence);
		1105	}
		1106
		1107	/**
		1108	* xe_lrc_init_seqno_fence() - Initialize an lrc seqno fence.
		1109	* @lrc: Pointer to the lrc.
		1110	* @fence: Pointer to the fence to initialize.
		1111	*
		1112	* Initializes a pre-allocated lrc seqno fence.
		1113	* After initialization, the fence is subject to normal
		1114	* dma-fence refcounting.
		1115	*/
		1116	void xe_lrc_init_seqno_fence(struct xe_lrc lrc, struct dma_fence fence)
		1117	{
		1118	xe_hw_fence_init(fence, &lrc->fence_ctx, __xe_lrc_seqno_map(lrc));
908	}	1119	}
909		1120
910	s32 xe_lrc_seqno(struct xe_lrc *lrc)	1121	s32 xe_lrc_seqno(struct xe_lrc *lrc)
@@ -1214,7 +1425,7 @@ void xe_lrc_dump_default(struct drm_printer *p, · 303 unmodified lines
1214	* hardware status page.	1425	* hardware status page.
1215	*/	1426	*/
1216	dw = gt->default_lrc[hwe_class] + LRC_PPHWSP_SIZE;	1427	dw = gt->default_lrc[hwe_class] + LRC_PPHWSP_SIZE;
1217	remaining_dw = (xe_lrc_size(gt_to_xe(gt), hwe_class) - LRC_PPHWSP_SIZE) / 4;	1428	remaining_dw = (xe_gt_lrc_size(gt, hwe_class) - LRC_PPHWSP_SIZE) / 4;
1218		1429
1219	while (remaining_dw > 0) {	1430	while (remaining_dw > 0) {
1220	if ((*dw & XE_INSTR_CMD_TYPE) == XE_INSTR_MI) {	1431	if ((*dw & XE_INSTR_CMD_TYPE) == XE_INSTR_MI) {
@@ -1354,10 +1565,11 @@ struct xe_lrc_snapshot xe_lrc_snapshot_capture(struct xe_lrc lrc) · 133 unmodified lines
1354	if (!snapshot)	1565	if (!snapshot)
1355	return NULL;	1566	return NULL;
1356		1567
1357	snapshot->context_desc = lower_32_bits(xe_lrc_ggtt_addr(lrc));	1568	snapshot->context_desc = xe_lrc_ggtt_addr(lrc);
		1569	snapshot->indirect_context_desc = xe_lrc_indirect_ring_ggtt_addr(lrc);
1358	snapshot->head = xe_lrc_ring_head(lrc);	1570	snapshot->head = xe_lrc_ring_head(lrc);
1359	snapshot->tail.internal = lrc->ring.tail;	1571	snapshot->tail.internal = lrc->ring.tail;
1360	snapshot->tail.memory = xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL);	1572	snapshot->tail.memory = xe_lrc_ring_tail(lrc);
1361	snapshot->start_seqno = xe_lrc_start_seqno(lrc);	1573	snapshot->start_seqno = xe_lrc_start_seqno(lrc);
1362	snapshot->seqno = xe_lrc_seqno(lrc);	1574	snapshot->seqno = xe_lrc_seqno(lrc);
1363	snapshot->lrc_bo = xe_bo_get(lrc->bo);	1575	snapshot->lrc_bo = xe_bo_get(lrc->bo);
@@ -1382,7 +1594,7 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot) · 18 unmodified lines
1382	if (!snapshot->lrc_snapshot)	1594	if (!snapshot->lrc_snapshot)
1383	goto put_bo;	1595	goto put_bo;
1384		1596
1385	dma_resv_lock(bo->ttm.base.resv, NULL);	1597	xe_bo_lock(bo, false);
1386	if (!ttm_bo_vmap(&bo->ttm, &src)) {	1598	if (!ttm_bo_vmap(&bo->ttm, &src)) {
1387	xe_map_memcpy_from(xe_bo_device(bo),	1599	xe_map_memcpy_from(xe_bo_device(bo),
1388	snapshot->lrc_snapshot, &src, snapshot->lrc_offset,	1600	snapshot->lrc_snapshot, &src, snapshot->lrc_offset,
@@ -1392,7 +1604,7 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot) · 3 unmodified lines
1392	kvfree(snapshot->lrc_snapshot);	1604	kvfree(snapshot->lrc_snapshot);
1393	snapshot->lrc_snapshot = NULL;	1605	snapshot->lrc_snapshot = NULL;
1394	}	1606	}
1395	dma_resv_unlock(bo->ttm.base.resv);	1607	xe_bo_unlock(bo);
1396	put_bo:	1608	put_bo:
1397	xe_bo_put(bo);	1609	xe_bo_put(bo);
1398	}	1610	}
@@ -1405,6 +1617,8 @@ void xe_lrc_snapshot_print(struct xe_lrc_snapshot *snapshot, struct drm_printer · 6 unmodified lines
1405	return;	1617	return;
1406		1618
1407	drm_printf(p, "\tHW Context Desc: 0x%08x\n", snapshot->context_desc);	1619	drm_printf(p, "\tHW Context Desc: 0x%08x\n", snapshot->context_desc);
		1620	drm_printf(p, "\tHW Indirect Ring State: 0x%08x\n",
		1621	snapshot->indirect_context_desc);
1408	drm_printf(p, "\tLRC Head: (memory) %u\n", snapshot->head);	1622	drm_printf(p, "\tLRC Head: (memory) %u\n", snapshot->head);
1409	drm_printf(p, "\tLRC Tail: (internal) %u, (memory) %u\n",	1623	drm_printf(p, "\tLRC Tail: (internal) %u, (memory) %u\n",
1410	snapshot->tail.internal, snapshot->tail.memory);	1624	snapshot->tail.internal, snapshot->tail.memory);
@@ -1444,3 +1658,12 @@ void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot) · 33 unmodified lines
1444	xe_bo_put(snapshot->lrc_bo);	1658	xe_bo_put(snapshot->lrc_bo);
1445	kfree(snapshot);	1659	kfree(snapshot);
1446	}	1660	}
		1661
		1662	u32 xe_lrc_update_timestamp(struct xe_lrc lrc, u32 old_ts)
		1663	{
		1664	*old_ts = lrc->ctx_timestamp;
		1665
		1666	lrc->ctx_timestamp = xe_lrc_read_ctx_reg(lrc, CTX_TIMESTAMP);
		1667
		1668	return lrc->ctx_timestamp;
		1669	}

drivers/gpu/drm/xe/xe_lrc.h+53−5

@@ -5,6 +5,8 @@ · 4 unmodified lines
5	#ifndef _XE_LRC_H_	5	#ifndef _XE_LRC_H_
6	#define _XE_LRC_H_	6	#define _XE_LRC_H_
7		7
		8	#include <linux/types.h>
		9
8	#include "xe_lrc_types.h"	10	#include "xe_lrc_types.h"
9		11
10	struct drm_printer;	12	struct drm_printer;
@@ -12,23 +14,53 @@ struct xe_bb; · 1 unmodified line
12	struct xe_device;	14	struct xe_device;
13	struct xe_exec_queue;	15	struct xe_exec_queue;
14	enum xe_engine_class;	16	enum xe_engine_class;
		17	struct xe_gt;
15	struct xe_hw_engine;	18	struct xe_hw_engine;
		19	struct xe_lrc;
		20	struct xe_lrc_snapshot;
16	struct xe_vm;	21	struct xe_vm;
17		22
18	#define LRC_PPHWSP_SCRATCH_ADDR (0x34 * 4)	23	#define LRC_PPHWSP_SCRATCH_ADDR (0x34 * 4)
19		24
20	int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe,	25	struct xe_lrc xe_lrc_create(struct xe_hw_engine hwe, struct xe_vm *vm,
21	struct xe_exec_queue q, struct xe_vm vm, u32 ring_size);	26	u32 ring_size);
22	void xe_lrc_finish(struct xe_lrc *lrc);	27	void xe_lrc_destroy(struct kref *ref);
		28
		29	/**
		30	* xe_lrc_get - Get reference to the LRC
		31	* @lrc: Logical Ring Context
		32	*
		33	* Increment reference count of @lrc
		34	*/
		35	static inline struct xe_lrc xe_lrc_get(struct xe_lrc lrc)
		36	{
		37	kref_get(&lrc->refcount);
		38	return lrc;
		39	}
		40
		41	/**
		42	* xe_lrc_put - Put reference of the LRC
		43	* @lrc: Logical Ring Context
		44	*
		45	* Decrement reference count of @lrc, call xe_lrc_destroy when
		46	* reference count reaches 0.
		47	*/
		48	static inline void xe_lrc_put(struct xe_lrc *lrc)
		49	{
		50	kref_put(&lrc->refcount, xe_lrc_destroy);
		51	}
23		52
24	size_t xe_lrc_size(struct xe_device *xe, enum xe_engine_class class);	53	size_t xe_gt_lrc_size(struct xe_gt *gt, enum xe_engine_class class);
25	u32 xe_lrc_pphwsp_offset(struct xe_lrc *lrc);	54	u32 xe_lrc_pphwsp_offset(struct xe_lrc *lrc);
26		55
		56	void xe_lrc_set_ring_tail(struct xe_lrc *lrc, u32 tail);
		57	u32 xe_lrc_ring_tail(struct xe_lrc *lrc);
27	void xe_lrc_set_ring_head(struct xe_lrc *lrc, u32 head);	58	void xe_lrc_set_ring_head(struct xe_lrc *lrc, u32 head);
28	u32 xe_lrc_ring_head(struct xe_lrc *lrc);	59	u32 xe_lrc_ring_head(struct xe_lrc *lrc);
29	u32 xe_lrc_ring_space(struct xe_lrc *lrc);	60	u32 xe_lrc_ring_space(struct xe_lrc *lrc);
30	void xe_lrc_write_ring(struct xe_lrc lrc, const void data, size_t size);	61	void xe_lrc_write_ring(struct xe_lrc lrc, const void data, size_t size);
31		62
		63	u32 xe_lrc_indirect_ring_ggtt_addr(struct xe_lrc *lrc);
32	u32 xe_lrc_ggtt_addr(struct xe_lrc *lrc);	64	u32 xe_lrc_ggtt_addr(struct xe_lrc *lrc);
33	u32 xe_lrc_regs(struct xe_lrc lrc);	65	u32 xe_lrc_regs(struct xe_lrc lrc);
34		66
@@ -38,7 +70,9 @@ void xe_lrc_write_ctx_reg(struct xe_lrc *lrc, int reg_nr, u32 val); · 3 unmodified lines
38	u64 xe_lrc_descriptor(struct xe_lrc *lrc);	70	u64 xe_lrc_descriptor(struct xe_lrc *lrc);
39		71
40	u32 xe_lrc_seqno_ggtt_addr(struct xe_lrc *lrc);	72	u32 xe_lrc_seqno_ggtt_addr(struct xe_lrc *lrc);
41	struct dma_fence xe_lrc_create_seqno_fence(struct xe_lrc lrc);	73	struct dma_fence *xe_lrc_alloc_seqno_fence(void);
		74	void xe_lrc_free_seqno_fence(struct dma_fence *fence);
		75	void xe_lrc_init_seqno_fence(struct xe_lrc lrc, struct dma_fence fence);
42	s32 xe_lrc_seqno(struct xe_lrc *lrc);	76	s32 xe_lrc_seqno(struct xe_lrc *lrc);
43		77
44	u32 xe_lrc_start_seqno_ggtt_addr(struct xe_lrc *lrc);	78	u32 xe_lrc_start_seqno_ggtt_addr(struct xe_lrc *lrc);
@@ -60,4 +94,18 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot); · 15 unmodified lines
60	void xe_lrc_snapshot_print(struct xe_lrc_snapshot snapshot, struct drm_printer p);	94	void xe_lrc_snapshot_print(struct xe_lrc_snapshot snapshot, struct drm_printer p);
61	void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot);	95	void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot);
62		96
		97	/**
		98	* xe_lrc_update_timestamp - readout LRC timestamp and update cached value
		99	* @lrc: logical ring context for this exec queue
		100	* @old_ts: pointer where to save the previous timestamp
		101	*
		102	* Read the current timestamp for this LRC and update the cached value. The
		103	* previous cached value is also returned in @old_ts so the caller can calculate
		104	* the delta between 2 updates. Note that this is not intended to be called from
		105	* any place, but just by the paths updating the drm client utilization.
		106	*
		107	* Returns the current LRC timestamp
		108	*/
		109	u32 xe_lrc_update_timestamp(struct xe_lrc lrc, u32 old_ts);
		110
63	#endif	111	#endif

drivers/gpu/drm/xe/xe_lrc_types.h+12−0

 #ifndef _XE_LRC_TYPES_H_
 #define _XE_LRC_TYPES_H_
+#include <linux/kref.h>
 #include "xe_hw_fence_types.h"
 struct xe_bo;
 	 */
 	struct xe_bo *bo;
+	/** @size: size of lrc including any indirect ring state page */
+	u32 size;
 	/** @tile: tile which this LRC belongs to */
 	struct xe_tile *tile;
 	/** @flags: LRC flags */
+#define XE_LRC_FLAG_INDIRECT_RING_STATE		0x1
 	u32 flags;
+	/** @refcount: ref count of this lrc */
+	struct kref refcount;
 	/** @ring: submission ring state */
 	struct {
 		/** @ring.size: size of submission ring */
 	/** @fence_ctx: context for hw fence */
 	struct xe_hw_fence_ctx fence_ctx;
+	/** @ctx_timestamp: readout value of CTX_TIMESTAMP on last update */
+	u32 ctx_timestamp;
 };
 struct xe_lrc_snapshot;

drivers/gpu/drm/xe/xe_migrate.c+53−25

 #include <generated/xe_wa_oob.h>
+#include "instructions/xe_gpu_commands.h"
 #include "instructions/xe_mi_commands.h"
-#include "regs/xe_gpu_commands.h"
 #include "regs/xe_gtt_defs.h"
 #include "tests/xe_test.h"
 #include "xe_assert.h"
 #define MAX_PREEMPTDISABLE_TRANSFER SZ_8M /* Around 1ms. */
 #define MAX_CCS_LIMITED_TRANSFER SZ_4M /* XE_PAGE_SIZE * (FIELD_MAX(XE2_CCS_SIZE_MASK) + 1) */
-#define NUM_KERNEL_PDE 17
+#define NUM_KERNEL_PDE 15
 #define NUM_PT_SLOTS 32
 #define LEVEL0_PAGE_TABLE_ENCODE_SIZE SZ_2M
 #define MAX_NUM_PTE 512
 	struct xe_device *xe = tile_to_xe(tile);
 	u16 pat_index = xe->pat.idx[XE_CACHE_WB];
 	u8 id = tile->id;
-	u32 num_entries = NUM_PT_SLOTS, num_level = vm->pt_root[id]->level;
+	u32 num_entries = NUM_PT_SLOTS, num_level = vm->pt_root[id]->level,
+	    num_setup = num_level + 1;
 	u32 map_ofs, level, i;
 	struct xe_bo *bo, *batch = tile->mem.kernel_bb_pool->bo;
-	u64 entry;
+	u64 entry, pt30_ofs;
 	/* Can't bump NUM_PT_SLOTS too high */
 	BUILD_BUG_ON(NUM_PT_SLOTS > SZ_2M/XE_PAGE_SIZE);
 	if (IS_ERR(bo))
 		return PTR_ERR(bo);
-	entry = vm->pt_ops->pde_encode_bo(bo, bo->size - XE_PAGE_SIZE, pat_index);
+	/* PT31 reserved for 2M identity map */
+	pt30_ofs = bo->size - 2 * XE_PAGE_SIZE;
+	entry = vm->pt_ops->pde_encode_bo(bo, pt30_ofs, pat_index);
 	xe_pt_write(xe, &vm->pt_root[id]->bo->vmap, 0, entry);
-	map_ofs = (num_entries - num_level) * XE_PAGE_SIZE;
+	map_ofs = (num_entries - num_setup) * XE_PAGE_SIZE;
 	/* Map the entire BO in our level 0 pt */
 	for (i = 0, level = 0; i < num_entries; level++) {
 	}
 	/* Write PDE's that point to our BO. */
-	for (i = 0; i < num_entries - num_level; i++) {
+	for (i = 0; i < map_ofs / PAGE_SIZE; i++) {
 		entry = vm->pt_ops->pde_encode_bo(bo, (u64)i * XE_PAGE_SIZE,
 						  pat_index);
 	/* Identity map the entire vram at 256GiB offset */
 	if (IS_DGFX(xe)) {
 		u64 pos, ofs, flags;
+		/* XXX: Unclear if this should be usable_size? */
+		u64 vram_limit =  xe->mem.vram.actual_physical_size +
+			xe->mem.vram.dpa_base;
 		level = 2;
 		ofs = map_ofs + XE_PAGE_SIZE * level + 256 * 8;
 		flags = vm->pt_ops->pte_encode_addr(xe, 0, pat_index, level,
 						    true, 0);
+		xe_assert(xe, IS_ALIGNED(xe->mem.vram.usable_size, SZ_2M));
 		/*
-		 * Use 1GB pages, it shouldn't matter the physical amount of
+		 * Use 1GB pages when possible, last chunk always use 2M
-		 * vram is less, when we don't access it.
+		 * pages as mixing reserved memory (stolen, WOCPM) with a single
+		 * mapping is not allowed on certain platforms.
 		 */
-		for (pos = xe->mem.vram.dpa_base;
+		for (pos = xe->mem.vram.dpa_base; pos < vram_limit;
-		     pos < xe->mem.vram.actual_physical_size + xe->mem.vram.dpa_base;
+		     pos += SZ_1G, ofs += 8) {
-		     pos += SZ_1G, ofs += 8)
+			if (pos + SZ_1G >= vram_limit) {
+				u64 pt31_ofs = bo->size - XE_PAGE_SIZE;
+				entry = vm->pt_ops->pde_encode_bo(bo, pt31_ofs,
+								  pat_index);
+				xe_map_wr(xe, &bo->vmap, ofs, u64, entry);
+				flags = vm->pt_ops->pte_encode_addr(xe, 0,
+								    pat_index,
+								    level - 1,
+								    true, 0);
+				for (ofs = pt31_ofs; pos < vram_limit;
+				     pos += SZ_2M, ofs += 8)
+					xe_map_wr(xe, &bo->vmap, ofs, u64, pos | flags);
+				break;	/* Ensure pos == vram_limit assert correct */
+			}
 			xe_map_wr(xe, &bo->vmap, ofs, u64, pos | flags);
+		}
+		xe_assert(xe, pos == vram_limit);
 	}
 	/*
 	 * Example layout created above, with root level = 3:
 	 * [PT0...PT7]: kernel PT's for copy/clear; 64 or 4KiB PTE's
 	 * [PT8]: Kernel PT for VM_BIND, 4 KiB PTE's
-	 * [PT9...PT28]: Userspace PT's for VM_BIND, 4 KiB PTE's
+	 * [PT9...PT27]: Userspace PT's for VM_BIND, 4 KiB PTE's
-	 * [PT29 = PDE 0] [PT30 = PDE 1] [PT31 = PDE 2]
+	 * [PT28 = PDE 0] [PT29 = PDE 1] [PT30 = PDE 2] [PT31 = 2M vram identity map]
 	 *
 	 * This makes the lowest part of the VM point to the pagetables.
 	 * Hence the lowest 2M in the vm should point to itself, with a few writes
 	}
 	mutex_init(&m->job_mutex);
+	fs_reclaim_acquire(GFP_KERNEL);
+	might_lock(&m->job_mutex);
+	fs_reclaim_release(GFP_KERNEL);
 	err = drmm_add_action_or_reset(&xe->drm, xe_migrate_fini, m);
 	if (err)
 						  IS_DGFX(xe) ? dst_is_vram : dst_is_pltt,
 						  src_L0, ccs_ofs, copy_ccs);
-		mutex_lock(&m->job_mutex);
 		job = xe_bb_create_migration_job(m->q, bb,
 						 xe_migrate_batch_base(m, usm),
 						 update_idx);
 				goto err_job;
 		}
+		mutex_lock(&m->job_mutex);
 		xe_sched_job_arm(job);
 		dma_fence_put(fence);
 		fence = dma_fence_get(&job->drm.s_fence->finished);
 err_job:
 		xe_sched_job_put(job);
 err:
-		mutex_unlock(&m->job_mutex);
 		xe_bb_free(bb, NULL);
 err_sync:
 	 * all of the actual service copy engines (BCS1-BCS8) have been fused
 	 * off.
 	 */
-	return gt->info.__engine_mask & GENMASK(XE_HW_ENGINE_BCS8,
+	return gt->info.engine_mask & GENMASK(XE_HW_ENGINE_BCS8,
-						XE_HW_ENGINE_BCS1);
+					      XE_HW_ENGINE_BCS1);
 }
 static u32 emit_clear_cmd_len(struct xe_gt *gt)
 			flush_flags = MI_FLUSH_DW_CCS;
 		}
-		mutex_lock(&m->job_mutex);
 		job = xe_bb_create_migration_job(m->q, bb,
 						 xe_migrate_batch_base(m, usm),
 						 update_idx);
 				goto err_job;
 		}
+		mutex_lock(&m->job_mutex);
 		xe_sched_job_arm(job);
 		dma_fence_put(fence);
 		fence = dma_fence_get(&job->drm.s_fence->finished);
 err_job:
 		xe_sched_job_put(job);
 err:
-		mutex_unlock(&m->job_mutex);
 		xe_bb_free(bb, NULL);
 err_sync:
 		/* Sync partial copies if any. FIXME: job_mutex? */
 			write_pgtable(tile, bb, 0, &updates[i], pt_update);
 	}
-	if (!q)
-		mutex_lock(&m->job_mutex);
 	job = xe_bb_create_migration_job(q ?: m->q, bb,
 					 xe_migrate_batch_base(m, usm),
 					 update_idx);
 		if (err)
 			goto err_job;
 	}
+	if (!q)
+		mutex_lock(&m->job_mutex);
 	xe_sched_job_arm(job);
 	fence = dma_fence_get(&job->drm.s_fence->finished);
 	xe_sched_job_push(job);
 err_job:
 	xe_sched_job_put(job);
 err_bb:
-	if (!q)
-		mutex_unlock(&m->job_mutex);
 	xe_bb_free(bb, NULL);
 err:
 	drm_suballoc_free(sa_bo, NULL);

drivers/gpu/drm/xe/xe_mmio.c+99−343

@@ -3,337 +3,36 @@ · 2 unmodified lines
3	* Copyright © 2021-2023 Intel Corporation	3	* Copyright © 2021-2023 Intel Corporation
4	*/	4	*/
5		5
6	#include <linux/minmax.h>
7
8	#include "xe_mmio.h"	6	#include "xe_mmio.h"
9		7
		8	#include <linux/delay.h>
		9	#include <linux/io-64-nonatomic-lo-hi.h>
		10	#include <linux/minmax.h>
		11	#include <linux/pci.h>
		12
10	#include <drm/drm_managed.h>	13	#include <drm/drm_managed.h>
11	#include <drm/xe_drm.h>	14	#include <drm/drm_print.h>
12		15
13	#include "regs/xe_engine_regs.h"	16	#include "regs/xe_bars.h"
14	#include "regs/xe_gt_regs.h"
15	#include "regs/xe_regs.h"	17	#include "regs/xe_regs.h"
16	#include "xe_bo.h"
17	#include "xe_device.h"	18	#include "xe_device.h"
18	#include "xe_ggtt.h"
19	#include "xe_gt.h"	19	#include "xe_gt.h"
20	#include "xe_gt_mcr.h"	20	#include "xe_gt_printk.h"
		21	#include "xe_gt_sriov_vf.h"
21	#include "xe_macros.h"	22	#include "xe_macros.h"
22	#include "xe_module.h"
23	#include "xe_sriov.h"	23	#include "xe_sriov.h"
24	#include "xe_tile.h"
25
26	#define XEHP_MTCFG_ADDR XE_REG(0x101800)
27	#define TILE_COUNT REG_GENMASK(15, 8)
28
29	#define BAR_SIZE_SHIFT 20
30
31	static void
32	_resize_bar(struct xe_device *xe, int resno, resource_size_t size)
33	{
34	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
35	int bar_size = pci_rebar_bytes_to_size(size);
36	int ret;
37
38	if (pci_resource_len(pdev, resno))
39	pci_release_resource(pdev, resno);
40
41	ret = pci_resize_resource(pdev, resno, bar_size);
42	if (ret) {
43	drm_info(&xe->drm, "Failed to resize BAR%d to %dM (%pe). Consider enabling 'Resizable BAR' support in your BIOS\n",
44	resno, 1 << bar_size, ERR_PTR(ret));
45	return;
46	}
47
48	drm_info(&xe->drm, "BAR%d resized to %dM\n", resno, 1 << bar_size);
49	}
50
51	/*
52	* if force_vram_bar_size is set, attempt to set to the requested size
53	* else set to maximum possible size
54	*/
55	static void xe_resize_vram_bar(struct xe_device *xe)
56	{
57	u64 force_vram_bar_size = xe_modparam.force_vram_bar_size;
58	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
59	struct pci_bus *root = pdev->bus;
60	resource_size_t current_size;
61	resource_size_t rebar_size;
62	struct resource *root_res;
63	u32 bar_size_mask;
64	u32 pci_cmd;
65	int i;
66
67	/* gather some relevant info */
68	current_size = pci_resource_len(pdev, LMEM_BAR);
69	bar_size_mask = pci_rebar_get_possible_sizes(pdev, LMEM_BAR);
70
71	if (!bar_size_mask)
72	return;
73
74	/* set to a specific size? */
75	if (force_vram_bar_size) {
76	u32 bar_size_bit;
77
78	rebar_size = force_vram_bar_size * (resource_size_t)SZ_1M;
79
80	bar_size_bit = bar_size_mask & BIT(pci_rebar_bytes_to_size(rebar_size));
81
82	if (!bar_size_bit) {
83	drm_info(&xe->drm,
84	"Requested size: %lluMiB is not supported by rebar sizes: 0x%x. Leaving default: %lluMiB\n",
85	(u64)rebar_size >> 20, bar_size_mask, (u64)current_size >> 20);
86	return;
87	}
88
89	rebar_size = 1ULL << (__fls(bar_size_bit) + BAR_SIZE_SHIFT);
90
91	if (rebar_size == current_size)
92	return;
93	} else {
94	rebar_size = 1ULL << (__fls(bar_size_mask) + BAR_SIZE_SHIFT);
95
96	/* only resize if larger than current */
97	if (rebar_size <= current_size)
98	return;
99	}
100
101	drm_info(&xe->drm, "Attempting to resize bar from %lluMiB -> %lluMiB\n",
102	(u64)current_size >> 20, (u64)rebar_size >> 20);
103
104	while (root->parent)
105	root = root->parent;
106
107	pci_bus_for_each_resource(root, root_res, i) {
108	if (root_res && root_res->flags & (IORESOURCE_MEM \| IORESOURCE_MEM_64) &&
109	(u64)root_res->start > 0x100000000ul)
110	break;
111	}
112
113	if (!root_res) {
114	drm_info(&xe->drm, "Can't resize VRAM BAR - platform support is missing. Consider enabling 'Resizable BAR' support in your BIOS\n");
115	return;
116	}
117
118	pci_read_config_dword(pdev, PCI_COMMAND, &pci_cmd);
119	pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd & ~PCI_COMMAND_MEMORY);
120
121	_resize_bar(xe, LMEM_BAR, rebar_size);
122
123	pci_assign_unassigned_bus_resources(pdev->bus);
124	pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd);
125	}
126
127	static bool xe_pci_resource_valid(struct pci_dev *pdev, int bar)
128	{
129	if (!pci_resource_flags(pdev, bar))
130	return false;
131
132	if (pci_resource_flags(pdev, bar) & IORESOURCE_UNSET)
133	return false;
134
135	if (!pci_resource_len(pdev, bar))
136	return false;
137
138	return true;
139	}
140
141	static int xe_determine_lmem_bar_size(struct xe_device *xe)
142	{
143	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
144
145	if (!xe_pci_resource_valid(pdev, LMEM_BAR)) {
146	drm_err(&xe->drm, "pci resource is not valid\n");
147	return -ENXIO;
148	}
149
150	xe_resize_vram_bar(xe);
151
152	xe->mem.vram.io_start = pci_resource_start(pdev, LMEM_BAR);
153	xe->mem.vram.io_size = pci_resource_len(pdev, LMEM_BAR);
154	if (!xe->mem.vram.io_size)
155	return -EIO;
156
157	/* XXX: Need to change when xe link code is ready */
158	xe->mem.vram.dpa_base = 0;
159
160	/* set up a map to the total memory area. */
161	xe->mem.vram.mapping = ioremap_wc(xe->mem.vram.io_start, xe->mem.vram.io_size);
162
163	return 0;
164	}
165
166	static inline u64 get_flat_ccs_offset(struct xe_gt *gt, u64 tile_size)
167	{
168	struct xe_device *xe = gt_to_xe(gt);
169	u64 offset;
170	u32 reg;
171
172	if (GRAPHICS_VER(xe) >= 20) {
173	u64 ccs_size = tile_size / 512;
174	u64 offset_hi, offset_lo;
175	u32 nodes, num_enabled;
176
177	reg = xe_mmio_read32(gt, MIRROR_FUSE3);
178	nodes = REG_FIELD_GET(XE2_NODE_ENABLE_MASK, reg);
179	num_enabled = hweight32(nodes); /* Number of enabled l3 nodes */
180
181	reg = xe_gt_mcr_unicast_read_any(gt, XE2_FLAT_CCS_BASE_RANGE_LOWER);
182	offset_lo = REG_FIELD_GET(XE2_FLAT_CCS_BASE_LOWER_ADDR_MASK, reg);
183
184	reg = xe_gt_mcr_unicast_read_any(gt, XE2_FLAT_CCS_BASE_RANGE_UPPER);
185	offset_hi = REG_FIELD_GET(XE2_FLAT_CCS_BASE_UPPER_ADDR_MASK, reg);
186
187	offset = offset_hi << 32; /* HW view bits 39:32 */
188	offset \|= offset_lo << 6; /* HW view bits 31:6 */
189	offset = num_enabled; / convert to SW view */
190
191	/* We don't expect any holes */
192	xe_assert_msg(xe, offset == (xe_mmio_read64_2x32(gt, GSMBASE) - ccs_size),
193	"Hole between CCS and GSM.\n");
194	} else {
195	reg = xe_gt_mcr_unicast_read_any(gt, XEHP_FLAT_CCS_BASE_ADDR);
196	offset = (u64)REG_FIELD_GET(XEHP_FLAT_CCS_PTR, reg) * SZ_64K;
197	}
198
199	return offset;
200	}
201
202	/**
203	* xe_mmio_tile_vram_size() - Collect vram size and offset information
204	* @tile: tile to get info for
205	* @vram_size: available vram (size - device reserved portions)
206	* @tile_size: actual vram size
207	* @tile_offset: physical start point in the vram address space
208	*
209	* There are 4 places for size information:
210	* - io size (from pci_resource_len of LMEM bar) (only used for small bar and DG1)
211	* - TILEx size (actual vram size)
212	* - GSMBASE offset (TILEx - "stolen")
213	* - CSSBASE offset (TILEx - CSS space necessary)
214	*
215	* CSSBASE is always a lower/smaller offset then GSMBASE.
216	*
217	* The actual available size of memory is to the CCS or GSM base.
218	* NOTE: multi-tile bases will include the tile offset.
219	*
220	*/
221	static int xe_mmio_tile_vram_size(struct xe_tile tile, u64 vram_size,
222	u64 tile_size, u64 tile_offset)
223	{
224	struct xe_device *xe = tile_to_xe(tile);
225	struct xe_gt *gt = tile->primary_gt;
226	u64 offset;
227	int err;
228	u32 reg;
229
230	err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
231	if (err)
232	return err;
233
234	/* actual size */
235	if (unlikely(xe->info.platform == XE_DG1)) {
236	*tile_size = pci_resource_len(to_pci_dev(xe->drm.dev), LMEM_BAR);
237	*tile_offset = 0;
238	} else {
239	reg = xe_gt_mcr_unicast_read_any(gt, XEHP_TILE_ADDR_RANGE(gt->info.id));
240	tile_size = (u64)REG_FIELD_GET(GENMASK(14, 8), reg) SZ_1G;
241	tile_offset = (u64)REG_FIELD_GET(GENMASK(7, 1), reg) SZ_1G;
242	}
243
244	/* minus device usage */
245	if (xe->info.has_flat_ccs) {
246	offset = get_flat_ccs_offset(gt, *tile_size);
247	} else {
248	offset = xe_mmio_read64_2x32(gt, GSMBASE);
249	}
250
251	/* remove the tile offset so we have just the available size */
252	vram_size = offset - tile_offset;
253
254	return xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
255	}
256		24
257	int xe_mmio_probe_vram(struct xe_device *xe)	25	static void tiles_fini(void *arg)
258	{	26	{
		27	struct xe_device *xe = arg;
259	struct xe_tile *tile;	28	struct xe_tile *tile;
260	resource_size_t io_size;	29	int id;
261	u64 available_size = 0;
262	u64 total_size = 0;
263	u64 tile_offset;
264	u64 tile_size;
265	u64 vram_size;
266	int err;
267	u8 id;
268
269	if (!IS_DGFX(xe))
270	return 0;
271
272	/* Get the size of the root tile's vram for later accessibility comparison */
273	tile = xe_device_get_root_tile(xe);
274	err = xe_mmio_tile_vram_size(tile, &vram_size, &tile_size, &tile_offset);
275	if (err)
276	return err;
277
278	err = xe_determine_lmem_bar_size(xe);
279	if (err)
280	return err;
281
282	drm_info(&xe->drm, "VISIBLE VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
283	&xe->mem.vram.io_size);
284
285	io_size = xe->mem.vram.io_size;
286
287	/* tile specific ranges */
288	for_each_tile(tile, xe, id) {
289	err = xe_mmio_tile_vram_size(tile, &vram_size, &tile_size, &tile_offset);
290	if (err)
291	return err;
292
293	tile->mem.vram.actual_physical_size = tile_size;
294	tile->mem.vram.io_start = xe->mem.vram.io_start + tile_offset;
295	tile->mem.vram.io_size = min_t(u64, vram_size, io_size);
296
297	if (!tile->mem.vram.io_size) {
298	drm_err(&xe->drm, "Tile without any CPU visible VRAM. Aborting.\n");
299	return -ENODEV;
300	}
301
302	tile->mem.vram.dpa_base = xe->mem.vram.dpa_base + tile_offset;
303	tile->mem.vram.usable_size = vram_size;
304	tile->mem.vram.mapping = xe->mem.vram.mapping + tile_offset;
305
306	if (tile->mem.vram.io_size < tile->mem.vram.usable_size)
307	drm_info(&xe->drm, "Small BAR device\n");
308	drm_info(&xe->drm, "VRAM[%u, %u]: Actual physical size %pa, usable size exclude stolen %pa, CPU accessible size %pa\n", id,
309	tile->id, &tile->mem.vram.actual_physical_size, &tile->mem.vram.usable_size, &tile->mem.vram.io_size);
310	drm_info(&xe->drm, "VRAM[%u, %u]: DPA range: [%pa-%llx], io range: [%pa-%llx]\n", id, tile->id,
311	&tile->mem.vram.dpa_base, tile->mem.vram.dpa_base + (u64)tile->mem.vram.actual_physical_size,
312	&tile->mem.vram.io_start, tile->mem.vram.io_start + (u64)tile->mem.vram.io_size);
313
314	/* calculate total size using tile size to get the correct HW sizing */
315	total_size += tile_size;
316	available_size += vram_size;
317
318	if (total_size > xe->mem.vram.io_size) {
319	drm_info(&xe->drm, "VRAM: %pa is larger than resource %pa\n",
320	&total_size, &xe->mem.vram.io_size);
321	}
322
323	io_size -= min_t(u64, tile_size, io_size);
324	}
325
326	xe->mem.vram.actual_physical_size = total_size;
327
328	drm_info(&xe->drm, "Total VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
329	&xe->mem.vram.actual_physical_size);
330	drm_info(&xe->drm, "Available VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
331	&available_size);
332		30
333	return 0;	31	for_each_tile(tile, xe, id)
		32	tile->mmio.regs = NULL;
334	}	33	}
335		34
336	void xe_mmio_probe_tiles(struct xe_device *xe)	35	int xe_mmio_probe_tiles(struct xe_device *xe)
337	{	36	{
338	size_t tile_mmio_size = SZ_16M, tile_mmio_ext_size = xe->info.tile_mmio_ext_size;	37	size_t tile_mmio_size = SZ_16M, tile_mmio_ext_size = xe->info.tile_mmio_ext_size;
339	u8 id, tile_count = xe->info.tile_count;	38	u8 id, tile_count = xe->info.tile_count;
@@ -384,15 +83,16 @@ add_mmio_ext: · 44 unmodified lines
384	regs += tile_mmio_ext_size;	83	regs += tile_mmio_ext_size;
385	}	84	}
386	}	85	}
		86
		87	return devm_add_action_or_reset(xe->drm.dev, tiles_fini, xe);
387	}	88	}
388		89
389	static void mmio_fini(struct drm_device drm, void arg)	90	static void mmio_fini(void *arg)
390	{	91	{
391	struct xe_device *xe = arg;	92	struct xe_device *xe = arg;
392		93
393	pci_iounmap(to_pci_dev(xe->drm.dev), xe->mmio.regs);	94	pci_iounmap(to_pci_dev(xe->drm.dev), xe->mmio.regs);
394	if (xe->mem.vram.mapping)	95	xe->mmio.regs = NULL;
395	iounmap(xe->mem.vram.mapping);
396	}	96	}
397		97
398	int xe_mmio_init(struct xe_device *xe)	98	int xe_mmio_init(struct xe_device *xe)
@@ -407,7 +107,7 @@ int xe_mmio_init(struct xe_device *xe) · 8 unmodified lines
407	* registers (0-4MB), reserved space (4MB-8MB) and GGTT (8MB-16MB).	107	* registers (0-4MB), reserved space (4MB-8MB) and GGTT (8MB-16MB).
408	*/	108	*/
409	xe->mmio.size = pci_resource_len(pdev, mmio_bar);	109	xe->mmio.size = pci_resource_len(pdev, mmio_bar);
410	xe->mmio.regs = pci_iomap(pdev, mmio_bar, 0);	110	xe->mmio.regs = pci_iomap(pdev, mmio_bar, GTTMMADR_BAR);
411	if (xe->mmio.regs == NULL) {	111	if (xe->mmio.regs == NULL) {
412	drm_err(&xe->drm, "failed to map registers\n");	112	drm_err(&xe->drm, "failed to map registers\n");
413	return -EIO;	113	return -EIO;
@@ -417,47 +117,42 @@ int xe_mmio_init(struct xe_device *xe) · 3 unmodified lines
417	root_tile->mmio.size = SZ_16M;	117	root_tile->mmio.size = SZ_16M;
418	root_tile->mmio.regs = xe->mmio.regs;	118	root_tile->mmio.regs = xe->mmio.regs;
419		119
420	return drmm_add_action_or_reset(&xe->drm, mmio_fini, xe);	120	return devm_add_action_or_reset(xe->drm.dev, mmio_fini, xe);
421	}	121	}
422		122
423	u8 xe_mmio_read8(struct xe_gt *gt, struct xe_reg reg)	123	u8 xe_mmio_read8(struct xe_gt *gt, struct xe_reg reg)
424	{	124	{
425	struct xe_tile *tile = gt_to_tile(gt);	125	struct xe_tile *tile = gt_to_tile(gt);
		126	u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
426		127
427	if (reg.addr < gt->mmio.adj_limit)	128	return readb((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr);
428	reg.addr += gt->mmio.adj_offset;
429
430	return readb((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + reg.addr);
431	}	129	}
432		130
433	u16 xe_mmio_read16(struct xe_gt *gt, struct xe_reg reg)	131	u16 xe_mmio_read16(struct xe_gt *gt, struct xe_reg reg)
434	{	132	{
435	struct xe_tile *tile = gt_to_tile(gt);	133	struct xe_tile *tile = gt_to_tile(gt);
		134	u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
436		135
437	if (reg.addr < gt->mmio.adj_limit)	136	return readw((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr);
438	reg.addr += gt->mmio.adj_offset;
439
440	return readw((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + reg.addr);
441	}	137	}
442		138
443	void xe_mmio_write32(struct xe_gt *gt, struct xe_reg reg, u32 val)	139	void xe_mmio_write32(struct xe_gt *gt, struct xe_reg reg, u32 val)
444	{	140	{
445	struct xe_tile *tile = gt_to_tile(gt);	141	struct xe_tile *tile = gt_to_tile(gt);
		142	u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
446		143
447	if (reg.addr < gt->mmio.adj_limit)	144	writel(val, (reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr);
448	reg.addr += gt->mmio.adj_offset;
449
450	writel(val, (reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + reg.addr);
451	}	145	}
452		146
453	u32 xe_mmio_read32(struct xe_gt *gt, struct xe_reg reg)	147	u32 xe_mmio_read32(struct xe_gt *gt, struct xe_reg reg)
454	{	148	{
455	struct xe_tile *tile = gt_to_tile(gt);	149	struct xe_tile *tile = gt_to_tile(gt);
		150	u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
456		151
457	if (reg.addr < gt->mmio.adj_limit)	152	if (!reg.vf && IS_SRIOV_VF(gt_to_xe(gt)))
458	reg.addr += gt->mmio.adj_offset;	153	return xe_gt_sriov_vf_read32(gt, reg);
459		154
460	return readl((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + reg.addr);	155	return readl((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr);
461	}	156	}
462		157
463	u32 xe_mmio_rmw32(struct xe_gt *gt, struct xe_reg reg, u32 clr, u32 set)	158	u32 xe_mmio_rmw32(struct xe_gt *gt, struct xe_reg reg, u32 clr, u32 set)
@@ -486,10 +181,9 @@ bool xe_mmio_in_range(const struct xe_gt *gt, · 22 unmodified lines
486	const struct xe_mmio_range *range,	181	const struct xe_mmio_range *range,
487	struct xe_reg reg)	182	struct xe_reg reg)
488	{	183	{
489	if (reg.addr < gt->mmio.adj_limit)	184	u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
490	reg.addr += gt->mmio.adj_offset;
491		185
492	return range && reg.addr >= range->start && reg.addr <= range->end;	186	return range && addr >= range->start && addr <= range->end;
493	}	187	}
494		188
495	/**	189	/**
@@ -519,10 +213,11 @@ u64 xe_mmio_read64_2x32(struct xe_gt *gt, struct xe_reg reg) · 23 unmodified lines
519	struct xe_reg reg_udw = { .addr = reg.addr + 0x4 };	213	struct xe_reg reg_udw = { .addr = reg.addr + 0x4 };
520	u32 ldw, udw, oldudw, retries;	214	u32 ldw, udw, oldudw, retries;
521		215
522	if (reg.addr < gt->mmio.adj_limit) {	216	reg.addr = xe_mmio_adjusted_addr(gt, reg.addr);
523	reg.addr += gt->mmio.adj_offset;	217	reg_udw.addr = xe_mmio_adjusted_addr(gt, reg_udw.addr);
524	reg_udw.addr += gt->mmio.adj_offset;	218
525	}	219	/* we shouldn't adjust just one register address */
		220	xe_gt_assert(gt, reg_udw.addr == reg.addr + 0x4);
526		221
527	oldudw = xe_mmio_read32(gt, reg_udw);	222	oldudw = xe_mmio_read32(gt, reg_udw);
528	for (retries = 5; retries; --retries) {	223	for (retries = 5; retries; --retries) {
@@ -601,3 +296,64 @@ int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 t · 72 unmodified lines
601		296
602	return ret;	297	return ret;
603	}	298	}
		299
		300	/**
		301	* xe_mmio_wait32_not() - Wait for a register to return anything other than the given masked value
		302	* @gt: MMIO target GT
		303	* @reg: register to read value from
		304	* @mask: mask to be applied to the value read from the register
		305	* @val: value to match after applying the mask
		306	* @timeout_us: time out after this period of time. Wait logic tries to be
		307	* smart, applying an exponential backoff until @timeout_us is reached.
		308	* @out_val: if not NULL, points where to store the last unmasked value
		309	* @atomic: needs to be true if calling from an atomic context
		310	*
		311	* This function polls for a masked value to change from a given value and
		312	* returns zero on success or -ETIMEDOUT if timed out.
		313	*
		314	* Note that @timeout_us represents the minimum amount of time to wait before
		315	* giving up. The actual time taken by this function can be a little more than
		316	* @timeout_us for different reasons, specially in non-atomic contexts. Thus,
		317	* it is possible that this function succeeds even after @timeout_us has passed.
		318	*/
		319	int xe_mmio_wait32_not(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 timeout_us,
		320	u32 *out_val, bool atomic)
		321	{
		322	ktime_t cur = ktime_get_raw();
		323	const ktime_t end = ktime_add_us(cur, timeout_us);
		324	int ret = -ETIMEDOUT;
		325	s64 wait = 10;
		326	u32 read;
		327
		328	for (;;) {
		329	read = xe_mmio_read32(gt, reg);
		330	if ((read & mask) != val) {
		331	ret = 0;
		332	break;
		333	}
		334
		335	cur = ktime_get_raw();
		336	if (!ktime_before(cur, end))
		337	break;
		338
		339	if (ktime_after(ktime_add_us(cur, wait), end))
		340	wait = ktime_us_delta(end, cur);
		341
		342	if (atomic)
		343	udelay(wait);
		344	else
		345	usleep_range(wait, wait << 1);
		346	wait <<= 1;
		347	}
		348
		349	if (ret != 0) {
		350	read = xe_mmio_read32(gt, reg);
		351	if ((read & mask) != val)
		352	ret = 0;
		353	}
		354
		355	if (out_val)
		356	*out_val = read;
		357
		358	return ret;
		359	}

drivers/gpu/drm/xe/xe_mmio.h+11−11

@@ -6,22 +6,13 @@ · 5 unmodified lines
6	#ifndef _XE_MMIO_H_	6	#ifndef _XE_MMIO_H_
7	#define _XE_MMIO_H_	7	#define _XE_MMIO_H_
8		8
9	#include <linux/delay.h>
10	#include <linux/io-64-nonatomic-lo-hi.h>
11
12	#include "regs/xe_reg_defs.h"
13	#include "xe_device_types.h"
14	#include "xe_gt_printk.h"
15	#include "xe_gt_types.h"	9	#include "xe_gt_types.h"
16		10
17	struct drm_device;
18	struct drm_file;
19	struct xe_device;	11	struct xe_device;
20		12	struct xe_reg;
21	#define LMEM_BAR 2
22		13
23	int xe_mmio_init(struct xe_device *xe);	14	int xe_mmio_init(struct xe_device *xe);
24	void xe_mmio_probe_tiles(struct xe_device *xe);	15	int xe_mmio_probe_tiles(struct xe_device *xe);
25		16
26	u8 xe_mmio_read8(struct xe_gt *gt, struct xe_reg reg);	17	u8 xe_mmio_read8(struct xe_gt *gt, struct xe_reg reg);
27	u16 xe_mmio_read16(struct xe_gt *gt, struct xe_reg reg);	18	u16 xe_mmio_read16(struct xe_gt *gt, struct xe_reg reg);
@@ -35,5 +26,14 @@ int xe_mmio_probe_vram(struct xe_device *xe); · 7 unmodified lines
35	u64 xe_mmio_read64_2x32(struct xe_gt *gt, struct xe_reg reg);	26	u64 xe_mmio_read64_2x32(struct xe_gt *gt, struct xe_reg reg);
36	int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 timeout_us,	27	int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 timeout_us,
37	u32 *out_val, bool atomic);	28	u32 *out_val, bool atomic);
		29	int xe_mmio_wait32_not(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 timeout_us,
		30	u32 *out_val, bool atomic);
		31
		32	static inline u32 xe_mmio_adjusted_addr(const struct xe_gt *gt, u32 addr)
		33	{
		34	if (addr < gt->mmio.adj_limit)
		35	addr += gt->mmio.adj_offset;
		36	return addr;
		37	}
38		38
39	#endif	39	#endif

drivers/gpu/drm/xe/xe_mocs.c+260−41

@@ -9,10 +9,13 @@ · 8 unmodified lines
9	#include "xe_bo.h"	9	#include "xe_bo.h"
10	#include "xe_device.h"	10	#include "xe_device.h"
11	#include "xe_exec_queue.h"	11	#include "xe_exec_queue.h"
		12	#include "xe_force_wake.h"
12	#include "xe_gt.h"	13	#include "xe_gt.h"
13	#include "xe_gt_mcr.h"	14	#include "xe_gt_mcr.h"
		15	#include "xe_gt_printk.h"
14	#include "xe_mmio.h"	16	#include "xe_mmio.h"
15	#include "xe_platform_types.h"	17	#include "xe_platform_types.h"
		18	#include "xe_pm.h"
16	#include "xe_sriov.h"	19	#include "xe_sriov.h"
17	#include "xe_step_types.h"	20	#include "xe_step_types.h"
18		21
@@ -36,34 +39,23 @@ struct xe_mocs_entry { · 17 unmodified lines
36	u16 used;	39	u16 used;
37	};	40	};
38		41
		42	struct xe_mocs_info;
		43
		44	struct xe_mocs_ops {
		45	void (dump)(struct xe_mocs_info mocs, unsigned int flags,
		46	struct xe_gt gt, struct drm_printer p);
		47	};
		48
39	struct xe_mocs_info {	49	struct xe_mocs_info {
40	unsigned int size;	50	unsigned int size;
41	unsigned int n_entries;	51	unsigned int n_entries;
42	const struct xe_mocs_entry *table;	52	const struct xe_mocs_entry *table;
		53	const struct xe_mocs_ops *ops;
43	u8 uc_index;	54	u8 uc_index;
44	u8 wb_index;	55	u8 wb_index;
45	u8 unused_entries_index;	56	u8 unused_entries_index;
46	};	57	};
47		58
48	/* Defines for the tables (XXX_MOCS_0 - XXX_MOCS_63) */
49	#define _LE_CACHEABILITY(value) ((value) << 0)
50	#define _LE_TGT_CACHE(value) ((value) << 2)
51	#define LE_LRUM(value) ((value) << 4)
52	#define LE_AOM(value) ((value) << 6)
53	#define LE_RSC(value) ((value) << 7)
54	#define LE_SCC(value) ((value) << 8)
55	#define LE_PFM(value) ((value) << 11)
56	#define LE_SCF(value) ((value) << 14)
57	#define LE_COS(value) ((value) << 15)
58	#define LE_SSE(value) ((value) << 17)
59
60	/* Defines for the tables (LNCFMOCS0 - LNCFMOCS31) - two entries per word */
61	#define L3_ESC(value) ((value) << 0)
62	#define L3_SCC(value) ((value) << 1)
63	#define _L3_CACHEABILITY(value) ((value) << 4)
64	#define L3_GLBGO(value) ((value) << 6)
65	#define L3_LKUP(value) ((value) << 7)
66
67	/* Defines for the tables (GLOB_MOCS_0 - GLOB_MOCS_16) */	59	/* Defines for the tables (GLOB_MOCS_0 - GLOB_MOCS_16) */
68	#define IG_PAT REG_BIT(8)	60	#define IG_PAT REG_BIT(8)
69	#define L3_CACHE_POLICY_MASK REG_GENMASK(5, 4)	61	#define L3_CACHE_POLICY_MASK REG_GENMASK(5, 4)
@@ -80,22 +72,22 @@ struct xe_mocs_info { · 10 unmodified lines
80	* Note: LE_0_PAGETABLE works only up to Gen11; for newer gens it means	72	* Note: LE_0_PAGETABLE works only up to Gen11; for newer gens it means
81	* the same as LE_UC	73	* the same as LE_UC
82	*/	74	*/
83	#define LE_0_PAGETABLE _LE_CACHEABILITY(0)	75	#define LE_0_PAGETABLE LE_CACHEABILITY(0)
84	#define LE_1_UC _LE_CACHEABILITY(1)	76	#define LE_1_UC LE_CACHEABILITY(1)
85	#define LE_2_WT _LE_CACHEABILITY(2)	77	#define LE_2_WT LE_CACHEABILITY(2)
86	#define LE_3_WB _LE_CACHEABILITY(3)	78	#define LE_3_WB LE_CACHEABILITY(3)
87		79
88	/* Target cache */	80	/* Target cache */
89	#define LE_TC_0_PAGETABLE _LE_TGT_CACHE(0)	81	#define LE_TC_0_PAGETABLE LE_TGT_CACHE(0)
90	#define LE_TC_1_LLC _LE_TGT_CACHE(1)	82	#define LE_TC_1_LLC LE_TGT_CACHE(1)
91	#define LE_TC_2_LLC_ELLC _LE_TGT_CACHE(2)	83	#define LE_TC_2_LLC_ELLC LE_TGT_CACHE(2)
92	#define LE_TC_3_LLC_ELLC_ALT _LE_TGT_CACHE(3)	84	#define LE_TC_3_LLC_ELLC_ALT LE_TGT_CACHE(3)
93		85
94	/* L3 caching options */	86	/* L3 caching options */
95	#define L3_0_DIRECT _L3_CACHEABILITY(0)	87	#define L3_0_DIRECT L3_CACHEABILITY(0)
96	#define L3_1_UC _L3_CACHEABILITY(1)	88	#define L3_1_UC L3_CACHEABILITY(1)
97	#define L3_2_RESERVED _L3_CACHEABILITY(2)	89	#define L3_2_RESERVED L3_CACHEABILITY(2)
98	#define L3_3_WB _L3_CACHEABILITY(3)	90	#define L3_3_WB L3_CACHEABILITY(3)
99		91
100	/* L4 caching options */	92	/* L4 caching options */
101	#define L4_0_WB REG_FIELD_PREP(L4_CACHE_POLICY_MASK, 0)	93	#define L4_0_WB REG_FIELD_PREP(L4_CACHE_POLICY_MASK, 0)
@@ -107,6 +99,8 @@ struct xe_mocs_info { · 5 unmodified lines
107	#define XE2_L3_1_XD REG_FIELD_PREP(L3_CACHE_POLICY_MASK, 1)	99	#define XE2_L3_1_XD REG_FIELD_PREP(L3_CACHE_POLICY_MASK, 1)
108	#define XE2_L3_3_UC REG_FIELD_PREP(L3_CACHE_POLICY_MASK, 3)	100	#define XE2_L3_3_UC REG_FIELD_PREP(L3_CACHE_POLICY_MASK, 3)
109		101
		102	#define XE2_L3_CLOS_MASK REG_GENMASK(7, 6)
		103
110	#define MOCS_ENTRY(__idx, __control_value, __l3cc_value) \	104	#define MOCS_ENTRY(__idx, __control_value, __l3cc_value) \
111	[__idx] = { \	105	[__idx] = { \
112	.control_value = __control_value, \	106	.control_value = __control_value, \
@@ -255,6 +249,84 @@ static const struct xe_mocs_entry gen12_mocs_desc[] = { · 142 unmodified lines
255	L3_1_UC)	249	L3_1_UC)
256	};	250	};
257		251
		252	static bool regs_are_mcr(struct xe_gt *gt)
		253	{
		254	struct xe_device *xe = gt_to_xe(gt);
		255
		256	if (xe_gt_is_media_type(gt))
		257	return MEDIA_VER(xe) >= 20;
		258	else
		259	return GRAPHICS_VERx100(xe) >= 1250;
		260	}
		261
		262	static void xelp_lncf_dump(struct xe_mocs_info info, struct xe_gt gt, struct drm_printer *p)
		263	{
		264	unsigned int i, j;
		265	u32 reg_val;
		266
		267	drm_printf(p, "LNCFCMOCS[idx] = [ESC, SCC, L3CC] (value)\n\n");
		268
		269	for (i = 0, j = 0; i < (info->n_entries + 1) / 2; i++, j++) {
		270	if (regs_are_mcr(gt))
		271	reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_LNCFCMOCS(i));
		272	else
		273	reg_val = xe_mmio_read32(gt, XELP_LNCFCMOCS(i));
		274
		275	drm_printf(p, "LNCFCMOCS[%2d] = [%u, %u, %u] (%#8x)\n",
		276	j++,
		277	!!(reg_val & L3_ESC_MASK),
		278	REG_FIELD_GET(L3_SCC_MASK, reg_val),
		279	REG_FIELD_GET(L3_CACHEABILITY_MASK, reg_val),
		280	reg_val);
		281
		282	drm_printf(p, "LNCFCMOCS[%2d] = [%u, %u, %u] (%#8x)\n",
		283	j,
		284	!!(reg_val & L3_UPPER_IDX_ESC_MASK),
		285	REG_FIELD_GET(L3_UPPER_IDX_SCC_MASK, reg_val),
		286	REG_FIELD_GET(L3_UPPER_IDX_CACHEABILITY_MASK, reg_val),
		287	reg_val);
		288	}
		289	}
		290
		291	static void xelp_mocs_dump(struct xe_mocs_info *info, unsigned int flags,
		292	struct xe_gt gt, struct drm_printer p)
		293	{
		294	unsigned int i;
		295	u32 reg_val;
		296
		297	if (flags & HAS_GLOBAL_MOCS) {
		298	drm_printf(p, "Global mocs table configuration:\n");
		299	drm_printf(p, "GLOB_MOCS[idx] = [LeCC, TC, LRUM, AOM, RSC, SCC, PFM, SCF, CoS, SSE] (value)\n\n");
		300
		301	for (i = 0; i < info->n_entries; i++) {
		302	if (regs_are_mcr(gt))
		303	reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_GLOBAL_MOCS(i));
		304	else
		305	reg_val = xe_mmio_read32(gt, XELP_GLOBAL_MOCS(i));
		306
		307	drm_printf(p, "GLOB_MOCS[%2d] = [%u, %u, %u, %u, %u, %u, %u, %u, %u, %u ] (%#8x)\n",
		308	i,
		309	REG_FIELD_GET(LE_CACHEABILITY_MASK, reg_val),
		310	REG_FIELD_GET(LE_TGT_CACHE_MASK, reg_val),
		311	REG_FIELD_GET(LE_LRUM_MASK, reg_val),
		312	!!(reg_val & LE_AOM_MASK),
		313	!!(reg_val & LE_RSC_MASK),
		314	REG_FIELD_GET(LE_SCC_MASK, reg_val),
		315	REG_FIELD_GET(LE_PFM_MASK, reg_val),
		316	!!(reg_val & LE_SCF_MASK),
		317	REG_FIELD_GET(LE_COS_MASK, reg_val),
		318	REG_FIELD_GET(LE_SSE_MASK, reg_val),
		319	reg_val);
		320	}
		321	}
		322
		323	xelp_lncf_dump(info, gt, p);
		324	}
		325
		326	static const struct xe_mocs_ops xelp_mocs_ops = {
		327	.dump = xelp_mocs_dump,
		328	};
		329
258	static const struct xe_mocs_entry dg1_mocs_desc[] = {	330	static const struct xe_mocs_entry dg1_mocs_desc[] = {
259	/* UC */	331	/* UC */
260	MOCS_ENTRY(1, 0, L3_1_UC),	332	MOCS_ENTRY(1, 0, L3_1_UC),
@@ -291,6 +363,40 @@ static const struct xe_mocs_entry dg2_mocs_desc[] = { · 30 unmodified lines
291	MOCS_ENTRY(3, 0, L3_3_WB \| L3_LKUP(1)),	363	MOCS_ENTRY(3, 0, L3_3_WB \| L3_LKUP(1)),
292	};	364	};
293		365
		366	static void xehp_lncf_dump(struct xe_mocs_info *info, unsigned int flags,
		367	struct xe_gt gt, struct drm_printer p)
		368	{
		369	unsigned int i, j;
		370	u32 reg_val;
		371
		372	drm_printf(p, "LNCFCMOCS[idx] = [UCL3LOOKUP, GLBGO, L3CC] (value)\n\n");
		373
		374	for (i = 0, j = 0; i < (info->n_entries + 1) / 2; i++, j++) {
		375	if (regs_are_mcr(gt))
		376	reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_LNCFCMOCS(i));
		377	else
		378	reg_val = xe_mmio_read32(gt, XELP_LNCFCMOCS(i));
		379
		380	drm_printf(p, "LNCFCMOCS[%2d] = [%u, %u, %u] (%#8x)\n",
		381	j++,
		382	!!(reg_val & L3_LKUP_MASK),
		383	!!(reg_val & L3_GLBGO_MASK),
		384	REG_FIELD_GET(L3_CACHEABILITY_MASK, reg_val),
		385	reg_val);
		386
		387	drm_printf(p, "LNCFCMOCS[%2d] = [%u, %u, %u] (%#8x)\n",
		388	j,
		389	!!(reg_val & L3_UPPER_LKUP_MASK),
		390	!!(reg_val & L3_UPPER_GLBGO_MASK),
		391	REG_FIELD_GET(L3_UPPER_IDX_CACHEABILITY_MASK, reg_val),
		392	reg_val);
		393	}
		394	}
		395
		396	static const struct xe_mocs_ops xehp_mocs_ops = {
		397	.dump = xehp_lncf_dump,
		398	};
		399
294	static const struct xe_mocs_entry pvc_mocs_desc[] = {	400	static const struct xe_mocs_entry pvc_mocs_desc[] = {
295	/* Error */	401	/* Error */
296	MOCS_ENTRY(0, 0, L3_3_WB),	402	MOCS_ENTRY(0, 0, L3_3_WB),
@@ -302,6 +408,36 @@ static const struct xe_mocs_entry pvc_mocs_desc[] = { · 5 unmodified lines
302	MOCS_ENTRY(2, 0, L3_3_WB),	408	MOCS_ENTRY(2, 0, L3_3_WB),
303	};	409	};
304		410
		411	static void pvc_mocs_dump(struct xe_mocs_info info, unsigned int flags, struct xe_gt gt,
		412	struct drm_printer *p)
		413	{
		414	unsigned int i, j;
		415	u32 reg_val;
		416
		417	drm_printf(p, "LNCFCMOCS[idx] = [ L3CC ] (value)\n\n");
		418
		419	for (i = 0, j = 0; i < (info->n_entries + 1) / 2; i++, j++) {
		420	if (regs_are_mcr(gt))
		421	reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_LNCFCMOCS(i));
		422	else
		423	reg_val = xe_mmio_read32(gt, XELP_LNCFCMOCS(i));
		424
		425	drm_printf(p, "LNCFCMOCS[%2d] = [ %u ] (%#8x)\n",
		426	j++,
		427	REG_FIELD_GET(L3_CACHEABILITY_MASK, reg_val),
		428	reg_val);
		429
		430	drm_printf(p, "LNCFCMOCS[%2d] = [ %u ] (%#8x)\n",
		431	j,
		432	REG_FIELD_GET(L3_UPPER_IDX_CACHEABILITY_MASK, reg_val),
		433	reg_val);
		434	}
		435	}
		436
		437	static const struct xe_mocs_ops pvc_mocs_ops = {
		438	.dump = pvc_mocs_dump,
		439	};
		440
305	static const struct xe_mocs_entry mtl_mocs_desc[] = {	441	static const struct xe_mocs_entry mtl_mocs_desc[] = {
306	/* Error - Reserved for Non-Use */	442	/* Error - Reserved for Non-Use */
307	MOCS_ENTRY(0,	443	MOCS_ENTRY(0,
@@ -353,6 +489,36 @@ static const struct xe_mocs_entry mtl_mocs_desc[] = { · 45 unmodified lines
353	L3_GLBGO(1) \| L3_1_UC),	489	L3_GLBGO(1) \| L3_1_UC),
354	};	490	};
355		491
		492	static void mtl_mocs_dump(struct xe_mocs_info *info, unsigned int flags,
		493	struct xe_gt gt, struct drm_printer p)
		494	{
		495	unsigned int i;
		496	u32 reg_val;
		497
		498	drm_printf(p, "Global mocs table configuration:\n");
		499	drm_printf(p, "GLOB_MOCS[idx] = [IG_PAT, L4_CACHE_POLICY] (value)\n\n");
		500
		501	for (i = 0; i < info->n_entries; i++) {
		502	if (regs_are_mcr(gt))
		503	reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_GLOBAL_MOCS(i));
		504	else
		505	reg_val = xe_mmio_read32(gt, XELP_GLOBAL_MOCS(i));
		506
		507	drm_printf(p, "GLOB_MOCS[%2d] = [%u, %u] (%#8x)\n",
		508	i,
		509	!!(reg_val & IG_PAT),
		510	REG_FIELD_GET(L4_CACHE_POLICY_MASK, reg_val),
		511	reg_val);
		512	}
		513
		514	/* MTL lncf mocs table pattern is similar to that of xehp */
		515	xehp_lncf_dump(info, flags, gt, p);
		516	}
		517
		518	static const struct xe_mocs_ops mtl_mocs_ops = {
		519	.dump = mtl_mocs_dump,
		520	};
		521
356	static const struct xe_mocs_entry xe2_mocs_table[] = {	522	static const struct xe_mocs_entry xe2_mocs_table[] = {
357	/* Defer to PAT */	523	/* Defer to PAT */
358	MOCS_ENTRY(0, XE2_L3_0_WB \| L4_3_UC, 0),	524	MOCS_ENTRY(0, XE2_L3_0_WB \| L4_3_UC, 0),
@@ -366,6 +532,34 @@ static const struct xe_mocs_entry xe2_mocs_table[] = { · 7 unmodified lines
366	MOCS_ENTRY(4, IG_PAT \| XE2_L3_0_WB \| L4_0_WB, 0),	532	MOCS_ENTRY(4, IG_PAT \| XE2_L3_0_WB \| L4_0_WB, 0),
367	};	533	};
368		534
		535	static void xe2_mocs_dump(struct xe_mocs_info *info, unsigned int flags,
		536	struct xe_gt gt, struct drm_printer p)
		537	{
		538	unsigned int i;
		539	u32 reg_val;
		540
		541	drm_printf(p, "Global mocs table configuration:\n");
		542	drm_printf(p, "GLOB_MOCS[idx] = [IG_PAT, L3_CLOS, L3_CACHE_POLICY, L4_CACHE_POLICY] (value)\n\n");
		543
		544	for (i = 0; i < info->n_entries; i++) {
		545	if (regs_are_mcr(gt))
		546	reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_GLOBAL_MOCS(i));
		547	else
		548	reg_val = xe_mmio_read32(gt, XELP_GLOBAL_MOCS(i));
		549
		550	drm_printf(p, "GLOB_MOCS[%2d] = [%u, %u, %u] (%#8x)\n",
		551	i,
		552	!!(reg_val & IG_PAT),
		553	REG_FIELD_GET(XE2_L3_CLOS_MASK, reg_val),
		554	REG_FIELD_GET(L4_CACHE_POLICY_MASK, reg_val),
		555	reg_val);
		556	}
		557	}
		558
		559	static const struct xe_mocs_ops xe2_mocs_ops = {
		560	.dump = xe2_mocs_dump,
		561	};
		562
369	static unsigned int get_mocs_settings(struct xe_device *xe,	563	static unsigned int get_mocs_settings(struct xe_device *xe,
370	struct xe_mocs_info *info)	564	struct xe_mocs_info *info)
371	{	565	{
@@ -376,6 +570,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 4 unmodified lines
376	switch (xe->info.platform) {	570	switch (xe->info.platform) {
377	case XE_LUNARLAKE:	571	case XE_LUNARLAKE:
378	case XE_BATTLEMAGE:	572	case XE_BATTLEMAGE:
		573	info->ops = &xe2_mocs_ops;
379	info->size = ARRAY_SIZE(xe2_mocs_table);	574	info->size = ARRAY_SIZE(xe2_mocs_table);
380	info->table = xe2_mocs_table;	575	info->table = xe2_mocs_table;
381	info->n_entries = XE2_NUM_MOCS_ENTRIES;	576	info->n_entries = XE2_NUM_MOCS_ENTRIES;
@@ -384,6 +579,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 2 unmodified lines
384	info->unused_entries_index = 4;	579	info->unused_entries_index = 4;
385	break;	580	break;
386	case XE_PVC:	581	case XE_PVC:
		582	info->ops = &pvc_mocs_ops;
387	info->size = ARRAY_SIZE(pvc_mocs_desc);	583	info->size = ARRAY_SIZE(pvc_mocs_desc);
388	info->table = pvc_mocs_desc;	584	info->table = pvc_mocs_desc;
389	info->n_entries = PVC_NUM_MOCS_ENTRIES;	585	info->n_entries = PVC_NUM_MOCS_ENTRIES;
@@ -392,6 +588,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 2 unmodified lines
392	info->unused_entries_index = 2;	588	info->unused_entries_index = 2;
393	break;	589	break;
394	case XE_METEORLAKE:	590	case XE_METEORLAKE:
		591	info->ops = &mtl_mocs_ops;
395	info->size = ARRAY_SIZE(mtl_mocs_desc);	592	info->size = ARRAY_SIZE(mtl_mocs_desc);
396	info->table = mtl_mocs_desc;	593	info->table = mtl_mocs_desc;
397	info->n_entries = MTL_NUM_MOCS_ENTRIES;	594	info->n_entries = MTL_NUM_MOCS_ENTRIES;
@@ -399,6 +596,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 1 unmodified line
399	info->unused_entries_index = 1;	596	info->unused_entries_index = 1;
400	break;	597	break;
401	case XE_DG2:	598	case XE_DG2:
		599	info->ops = &xehp_mocs_ops;
402	info->size = ARRAY_SIZE(dg2_mocs_desc);	600	info->size = ARRAY_SIZE(dg2_mocs_desc);
403	info->table = dg2_mocs_desc;	601	info->table = dg2_mocs_desc;
404	info->uc_index = 1;	602	info->uc_index = 1;
@@ -410,6 +608,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 5 unmodified lines
410	info->unused_entries_index = 3;	608	info->unused_entries_index = 3;
411	break;	609	break;
412	case XE_DG1:	610	case XE_DG1:
		611	info->ops = &xelp_mocs_ops;
413	info->size = ARRAY_SIZE(dg1_mocs_desc);	612	info->size = ARRAY_SIZE(dg1_mocs_desc);
414	info->table = dg1_mocs_desc;	613	info->table = dg1_mocs_desc;
415	info->uc_index = 1;	614	info->uc_index = 1;
@@ -421,6 +620,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 5 unmodified lines
421	case XE_ALDERLAKE_S:	620	case XE_ALDERLAKE_S:
422	case XE_ALDERLAKE_P:	621	case XE_ALDERLAKE_P:
423	case XE_ALDERLAKE_N:	622	case XE_ALDERLAKE_N:
		623	info->ops = &xelp_mocs_ops;
424	info->size = ARRAY_SIZE(gen12_mocs_desc);	624	info->size = ARRAY_SIZE(gen12_mocs_desc);
425	info->table = gen12_mocs_desc;	625	info->table = gen12_mocs_desc;
426	info->n_entries = XELP_NUM_MOCS_ENTRIES;	626	info->n_entries = XELP_NUM_MOCS_ENTRIES;
@@ -442,6 +642,8 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 15 unmodified lines
442	*/	642	*/
443	xe_assert(xe, info->unused_entries_index != 0);	643	xe_assert(xe, info->unused_entries_index != 0);
444		644
		645	xe_assert(xe, !info->ops \|\| info->ops->dump);
		646
445	if (XE_WARN_ON(info->size > info->n_entries)) {	647	if (XE_WARN_ON(info->size > info->n_entries)) {
446	info->table = NULL;	648	info->table = NULL;
447	return 0;	649	return 0;
@@ -467,16 +669,6 @@ static u32 get_entry_control(const struct xe_mocs_info *info, · 19 unmodified lines
467	return info->table[info->unused_entries_index].control_value;	669	return info->table[info->unused_entries_index].control_value;
468	}	670	}
469		671
470	static bool regs_are_mcr(struct xe_gt *gt)
471	{
472	struct xe_device *xe = gt_to_xe(gt);
473
474	if (xe_gt_is_media_type(gt))
475	return MEDIA_VER(xe) >= 20;
476	else
477	return GRAPHICS_VERx100(xe) >= 1250;
478	}
479
480	static void __init_mocs_table(struct xe_gt *gt,	672	static void __init_mocs_table(struct xe_gt *gt,
481	const struct xe_mocs_info *info)	673	const struct xe_mocs_info *info)
482	{	674	{
@@ -578,6 +770,33 @@ void xe_mocs_init(struct xe_gt *gt) · 95 unmodified lines
578	init_l3cc_table(gt, &table);	770	init_l3cc_table(gt, &table);
579	}	771	}
580		772
		773	void xe_mocs_dump(struct xe_gt gt, struct drm_printer p)
		774	{
		775	struct xe_mocs_info table;
		776	unsigned int flags;
		777	u32 ret;
		778	struct xe_device *xe = gt_to_xe(gt);
		779
		780	flags = get_mocs_settings(xe, &table);
		781
		782	if (!table.ops->dump)
		783	return;
		784
		785	xe_pm_runtime_get_noresume(xe);
		786	ret = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
		787
		788	if (ret)
		789	goto err_fw;
		790
		791	table.ops->dump(&table, flags, gt, p);
		792
		793	xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
		794
		795	err_fw:
		796	xe_assert(xe, !ret);
		797	xe_pm_runtime_put(xe);
		798	}
		799
581	#if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)	800	#if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)
582	#include "tests/xe_mocs.c"	801	#include "tests/xe_mocs.c"
583	#endif	802	#endif

drivers/gpu/drm/xe/xe_mocs.h+8−3

 #ifndef _XE_MOCS_H_
 #define _XE_MOCS_H_
-#include <linux/types.h>
+struct drm_printer;
-struct xe_exec_queue;
 struct xe_gt;
 void xe_mocs_init_early(struct xe_gt *gt);
 void xe_mocs_init(struct xe_gt *gt);
+/**
+ * xe_mocs_dump - Dump mocs table
+ * @gt: GT structure
+ * @p: Printer to dump info to
+ */
+void xe_mocs_dump(struct xe_gt *gt, struct drm_printer *p);
 #endif

drivers/gpu/drm/xe/xe_module.c+5−0

 	.enable_display = true,
 	.guc_log_level = 5,
 	.force_probe = CONFIG_DRM_XE_FORCE_PROBE,
+	.wedged_mode = 1,
 	/* the rest are 0 by default */
 };
 		 "(0 = no VFs [default]; N = allow up to N VFs)");
 #endif
+module_param_named_unsafe(wedged_mode, xe_modparam.wedged_mode, int, 0600);
+MODULE_PARM_DESC(wedged_mode,
+		 "Module's default policy for the wedged mode - 0=never, 1=upon-critical-errors[default], 2=upon-any-hang");
 struct init_funcs {
 	int (*init)(void);
 	void (*exit)(void);

drivers/gpu/drm/xe/xe_module.h+1−0

@@ -21,6 +21,7 @@ struct xe_modparam { · 20 unmodified lines
21	#ifdef CONFIG_PCI_IOV	21	#ifdef CONFIG_PCI_IOV
22	unsigned int max_vfs;	22	unsigned int max_vfs;
23	#endif	23	#endif
		24	int wedged_mode;
24	};	25	};
25		26
26	extern struct xe_modparam xe_modparam;	27	extern struct xe_modparam xe_modparam;

drivers/gpu/drm/xe/xe_pat.c+1−0

@@ -10,6 +10,7 @@ · 9 unmodified lines
10	#include "regs/xe_reg_defs.h"	10	#include "regs/xe_reg_defs.h"
11	#include "xe_assert.h"	11	#include "xe_assert.h"
12	#include "xe_device.h"	12	#include "xe_device.h"
		13	#include "xe_force_wake.h"
13	#include "xe_gt.h"	14	#include "xe_gt.h"
14	#include "xe_gt_mcr.h"	15	#include "xe_gt_mcr.h"
15	#include "xe_mmio.h"	16	#include "xe_mmio.h"

drivers/gpu/drm/xe/xe_pci.c+80−26

 #include "xe_device.h"
 #include "xe_drv.h"
 #include "xe_gt.h"
+#include "xe_gt_sriov_vf.h"
+#include "xe_guc.h"
 #include "xe_macros.h"
 #include "xe_mmio.h"
 #include "xe_module.h"
+#include "xe_pci_sriov.h"
 #include "xe_pci_types.h"
 #include "xe_pm.h"
 #include "xe_sriov.h"
 	const u16 *pciidlist;
 };
-struct xe_gt_desc {
-	enum xe_gt_type type;
-	u32 mmio_adj_limit;
-	u32 mmio_adj_offset;
-};
 struct xe_device_desc {
 	/* Should only ever be set for platforms without GMD_ID */
 	const struct xe_graphics_desc *graphics;
 __diag_ignore_all("-Woverride-init", "Allow field overrides in table");
 #define PLATFORM(x)		\
-	.platform = (x),	\
+	.platform = XE_##x,	\
 	.platform_name = #x
 #define NOP(x)	x
 	.vram_flags = XE_VRAM_FLAGS_NEED64K,
 	.has_asid = 1,
+	.has_atomic_enable_pte_bit = 1,
 	.has_flat_ccs = 0,
 	.has_usm = 1,
 };
 #define XE2_GFX_FEATURES \
 	.dma_mask_size = 46, \
 	.has_asid = 1, \
+	.has_atomic_enable_pte_bit = 1, \
 	.has_flat_ccs = 1, \
+	.has_indirect_ring_state = 1, \
 	.has_range_tlb_invalidation = 1, \
 	.has_usm = 1, \
 	.va_bits = 48, \
 	.name = "Xe2_LPM / Xe2_HPM",
 	.hw_engine_mask =
 		GENMASK(XE_HW_ENGINE_VCS7, XE_HW_ENGINE_VCS0) |
-		GENMASK(XE_HW_ENGINE_VECS3, XE_HW_ENGINE_VECS0), /* TODO: GSC0 */
+		GENMASK(XE_HW_ENGINE_VECS3, XE_HW_ENGINE_VECS0) |
+		BIT(XE_HW_ENGINE_GSCCS0)
 };
 static const struct xe_device_desc tgl_desc = {
 	.graphics = &graphics_xelp,
 	.media = &media_xem,
-	PLATFORM(XE_TIGERLAKE),
+	PLATFORM(TIGERLAKE),
 	.has_display = true,
 	.has_llc = true,
 	.require_force_probe = true,
 static const struct xe_device_desc rkl_desc = {
 	.graphics = &graphics_xelp,
 	.media = &media_xem,
-	PLATFORM(XE_ROCKETLAKE),
+	PLATFORM(ROCKETLAKE),
 	.has_display = true,
 	.has_llc = true,
 	.require_force_probe = true,
 static const struct xe_device_desc adl_s_desc = {
 	.graphics = &graphics_xelp,
 	.media = &media_xem,
-	PLATFORM(XE_ALDERLAKE_S),
+	PLATFORM(ALDERLAKE_S),
 	.has_display = true,
 	.has_llc = true,
 	.require_force_probe = true,
 static const struct xe_device_desc adl_p_desc = {
 	.graphics = &graphics_xelp,
 	.media = &media_xem,
-	PLATFORM(XE_ALDERLAKE_P),
+	PLATFORM(ALDERLAKE_P),
 	.has_display = true,
 	.has_llc = true,
 	.require_force_probe = true,
 static const struct xe_device_desc adl_n_desc = {
 	.graphics = &graphics_xelp,
 	.media = &media_xem,
-	PLATFORM(XE_ALDERLAKE_N),
+	PLATFORM(ALDERLAKE_N),
 	.has_display = true,
 	.has_llc = true,
 	.require_force_probe = true,
 	.graphics = &graphics_xelpp,
 	.media = &media_xem,
 	DGFX_FEATURES,
-	PLATFORM(XE_DG1),
+	PLATFORM(DG1),
 	.has_display = true,
 	.has_heci_gscfi = 1,
 	.require_force_probe = true,
 #define DG2_FEATURES \
 	DGFX_FEATURES, \
-	PLATFORM(XE_DG2), \
+	PLATFORM(DG2), \
 	.has_heci_gscfi = 1, \
 	.subplatforms = (const struct xe_subplatform_desc[]) { \
 		{ XE_SUBPLATFORM_DG2_G10, "G10", dg2_g10_ids }, \
 static const __maybe_unused struct xe_device_desc pvc_desc = {
 	.graphics = &graphics_xehpc,
 	DGFX_FEATURES,
-	PLATFORM(XE_PVC),
+	PLATFORM(PVC),
 	.has_display = false,
 	.has_heci_gscfi = 1,
 	.require_force_probe = true,
 static const struct xe_device_desc mtl_desc = {
 	/* .graphics and .media determined via GMD_ID */
 	.require_force_probe = true,
-	PLATFORM(XE_METEORLAKE),
+	PLATFORM(METEORLAKE),
 	.has_display = true,
 };
 static const struct xe_device_desc lnl_desc = {
-	PLATFORM(XE_LUNARLAKE),
+	PLATFORM(LUNARLAKE),
 	.has_display = true,
 	.require_force_probe = true,
 };
 static const struct xe_device_desc bmg_desc __maybe_unused = {
 	DGFX_FEATURES,
-	PLATFORM(XE_BATTLEMAGE),
+	PLATFORM(BATTLEMAGE),
 	.require_force_probe = true,
 };
 	KUNIT_STATIC_STUB_REDIRECT(read_gmdid, xe, type, ver, revid);
-	if (type == GMDID_MEDIA)
+	if (IS_SRIOV_VF(xe)) {
-		gmdid_reg.addr += MEDIA_GT_GSI_OFFSET;
+		/*
+		 * To get the value of the GMDID register, VFs must obtain it
+		 * from the GuC using MMIO communication.
+		 *
+		 * Note that at this point the xe_gt is not fully uninitialized
+		 * and only basic access to MMIO registers is possible. To use
+		 * our existing GuC communication functions we must perform at
+		 * least basic xe_gt and xe_guc initialization.
+		 *
+		 * Since to obtain the value of GMDID_MEDIA we need to use the
+		 * media GuC, temporarly tweak the gt type.
+		 */
+		xe_gt_assert(gt, gt->info.type == XE_GT_TYPE_UNINITIALIZED);
+		if (type == GMDID_MEDIA) {
+			gt->info.id = 1;
+			gt->info.type = XE_GT_TYPE_MEDIA;
+		} else {
+			gt->info.id = 0;
+			gt->info.type = XE_GT_TYPE_MAIN;
+		}
+		xe_guc_comm_init_early(&gt->uc.guc);
+		/* Don't bother with GMDID if failed to negotiate the GuC ABI */
+		val = xe_gt_sriov_vf_bootstrap(gt) ? 0 : xe_gt_sriov_vf_gmdid(gt);
+		/*
+		 * Only undo xe_gt.info here, the remaining changes made above
+		 * will be overwritten as part of the regular initialization.
+		 */
+		gt->info.id = 0;
+		gt->info.type = XE_GT_TYPE_UNINITIALIZED;
+	} else {
+		/*
+		 * We need to apply the GSI offset explicitly here as at this
+		 * point the xe_gt is not fully uninitialized and only basic
+		 * access to MMIO registers is possible.
+		 */
+		if (type == GMDID_MEDIA)
+			gmdid_reg.addr += MEDIA_GT_GSI_OFFSET;
+		val = xe_mmio_read32(gt, gmdid_reg);
+	}
-	val = xe_mmio_read32(gt, gmdid_reg);
 	*ver = REG_FIELD_GET(GMD_ID_ARCH_MASK, val) * 100 + REG_FIELD_GET(GMD_ID_RELEASE_MASK, val);
 	*revid = REG_FIELD_GET(GMD_ID_REVID, val);
 }
 {
 	int err;
+	xe->info.platform_name = desc->platform_name;
 	xe->info.platform = desc->platform;
 	xe->info.subplatform = subplatform_desc ?
 		subplatform_desc->subplatform : XE_SUBPLATFORM_NONE;
 	xe->info.va_bits = graphics_desc->va_bits;
 	xe->info.vm_max_level = graphics_desc->vm_max_level;
 	xe->info.has_asid = graphics_desc->has_asid;
+	xe->info.has_atomic_enable_pte_bit = graphics_desc->has_atomic_enable_pte_bit;
+	if (xe->info.platform != XE_PVC)
+		xe->info.has_device_atomics_on_smem = 1;
 	xe->info.has_flat_ccs = graphics_desc->has_flat_ccs;
 	xe->info.has_range_tlb_invalidation = graphics_desc->has_range_tlb_invalidation;
 	xe->info.has_usm = graphics_desc->has_usm;
 		gt = tile->primary_gt;
 		gt->info.id = xe->info.gt_count++;
 		gt->info.type = XE_GT_TYPE_MAIN;
-		gt->info.__engine_mask = graphics_desc->hw_engine_mask;
+		gt->info.has_indirect_ring_state = graphics_desc->has_indirect_ring_state;
+		gt->info.engine_mask = graphics_desc->hw_engine_mask;
 		if (MEDIA_VER(xe) < 13 && media_desc)
-			gt->info.__engine_mask |= media_desc->hw_engine_mask;
+			gt->info.engine_mask |= media_desc->hw_engine_mask;
 		if (MEDIA_VER(xe) < 13 || !media_desc)
 			continue;
 		gt = tile->media_gt;
 		gt->info.type = XE_GT_TYPE_MEDIA;
-		gt->info.__engine_mask = media_desc->hw_engine_mask;
+		gt->info.has_indirect_ring_state = media_desc->has_indirect_ring_state;
+		gt->info.engine_mask = media_desc->hw_engine_mask;
 		gt->mmio.adj_offset = MEDIA_GT_GSI_OFFSET;
 		gt->mmio.adj_limit = MEDIA_GT_GSI_LENGTH;
 	if (err)
 		return err;
-	xe_display_probe(xe);
+	err = xe_display_probe(xe);
+	if (err)
+		return err;
 	drm_dbg(&xe->drm, "%s %s %04x:%04x dgfx:%d gfx:%s (%d.%02d) media:%s (%d.%02d) display:%s dma_m_s:%d tc:%d gscfi:%d",
 		desc->platform_name,
 	.probe = xe_pci_probe,
 	.remove = xe_pci_remove,
 	.shutdown = xe_pci_shutdown,
+#ifdef CONFIG_PCI_IOV
+	.sriov_configure = xe_pci_sriov_configure,
+#endif
 #ifdef CONFIG_PM_SLEEP
 	.driver.pm = &xe_pm_ops,
 #endif

drivers/gpu/drm/xe/xe_pci_sriov.c+143−0

@@ -0,0 +1,143 @@
	1	// SPDX-License-Identifier: MIT
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#include "xe_assert.h"
	7	#include "xe_device.h"
	8	#include "xe_gt_sriov_pf_config.h"
	9	#include "xe_pci_sriov.h"
	10	#include "xe_pm.h"
	11	#include "xe_sriov.h"
	12	#include "xe_sriov_pf_helpers.h"
	13	#include "xe_sriov_printk.h"
	14
	15	static int pf_provision_vfs(struct xe_device *xe, unsigned int num_vfs)
	16	{
	17	struct xe_gt *gt;
	18	unsigned int id;
	19	int result = 0, err;
	20
	21	for_each_gt(gt, xe, id) {
	22	err = xe_gt_sriov_pf_config_set_fair(gt, VFID(1), num_vfs);
	23	result = result ?: err;
	24	}
	25
	26	return result;
	27	}
	28
	29	static void pf_unprovision_vfs(struct xe_device *xe, unsigned int num_vfs)
	30	{
	31	struct xe_gt *gt;
	32	unsigned int id;
	33	unsigned int n;
	34
	35	for_each_gt(gt, xe, id)
	36	for (n = 1; n <= num_vfs; n++)
	37	xe_gt_sriov_pf_config_release(gt, n, true);
	38	}
	39
	40	static int pf_enable_vfs(struct xe_device *xe, int num_vfs)
	41	{
	42	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
	43	int total_vfs = xe_sriov_pf_get_totalvfs(xe);
	44	int err;
	45
	46	xe_assert(xe, IS_SRIOV_PF(xe));
	47	xe_assert(xe, num_vfs > 0);
	48	xe_assert(xe, num_vfs <= total_vfs);
	49	xe_sriov_dbg(xe, "enabling %u VF%s\n", num_vfs, str_plural(num_vfs));
	50
	51	/*
	52	* We must hold additional reference to the runtime PM to keep PF in D0
	53	* during VFs lifetime, as our VFs do not implement the PM capability.
	54	*
	55	* With PF being in D0 state, all VFs will also behave as in D0 state.
	56	* This will also keep GuC alive with all VFs' configurations.
	57	*
	58	* We will release this additional PM reference in pf_disable_vfs().
	59	*/
	60	xe_pm_runtime_get_noresume(xe);
	61
	62	err = pf_provision_vfs(xe, num_vfs);
	63	if (err < 0)
	64	goto failed;
	65
	66	err = pci_enable_sriov(pdev, num_vfs);
	67	if (err < 0)
	68	goto failed;
	69
	70	xe_sriov_info(xe, "Enabled %u of %u VF%s\n",
	71	num_vfs, total_vfs, str_plural(total_vfs));
	72	return num_vfs;
	73
	74	failed:
	75	pf_unprovision_vfs(xe, num_vfs);
	76	xe_pm_runtime_put(xe);
	77
	78	xe_sriov_notice(xe, "Failed to enable %u VF%s (%pe)\n",
	79	num_vfs, str_plural(num_vfs), ERR_PTR(err));
	80	return err;
	81	}
	82
	83	static int pf_disable_vfs(struct xe_device *xe)
	84	{
	85	struct device *dev = xe->drm.dev;
	86	struct pci_dev *pdev = to_pci_dev(dev);
	87	u16 num_vfs = pci_num_vf(pdev);
	88
	89	xe_assert(xe, IS_SRIOV_PF(xe));
	90	xe_sriov_dbg(xe, "disabling %u VF%s\n", num_vfs, str_plural(num_vfs));
	91
	92	if (!num_vfs)
	93	return 0;
	94
	95	pci_disable_sriov(pdev);
	96
	97	pf_unprovision_vfs(xe, num_vfs);
	98
	99	/* not needed anymore - see pf_enable_vfs() */
	100	xe_pm_runtime_put(xe);
	101
	102	xe_sriov_info(xe, "Disabled %u VF%s\n", num_vfs, str_plural(num_vfs));
	103	return 0;
	104	}
	105
	106	/**
	107	* xe_pci_sriov_configure - Configure SR-IOV (enable/disable VFs).
	108	* @pdev: the &pci_dev
	109	* @num_vfs: number of VFs to enable or zero to disable all VFs
	110	*
	111	* This is the Xe implementation of struct pci_driver.sriov_configure callback.
	112	*
	113	* This callback will be called by the PCI subsystem to enable or disable SR-IOV
	114	* Virtual Functions (VFs) as requested by the used via the PCI sysfs interface.
	115	*
	116	* Return: number of configured VFs or a negative error code on failure.
	117	*/
	118	int xe_pci_sriov_configure(struct pci_dev *pdev, int num_vfs)
	119	{
	120	struct xe_device *xe = pdev_to_xe_device(pdev);
	121	int ret;
	122
	123	if (!IS_SRIOV_PF(xe))
	124	return -ENODEV;
	125
	126	if (num_vfs < 0)
	127	return -EINVAL;
	128
	129	if (num_vfs > xe_sriov_pf_get_totalvfs(xe))
	130	return -ERANGE;
	131
	132	if (num_vfs && pci_num_vf(pdev))
	133	return -EBUSY;
	134
	135	xe_pm_runtime_get(xe);
	136	if (num_vfs > 0)
	137	ret = pf_enable_vfs(xe, num_vfs);
	138	else
	139	ret = pf_disable_vfs(xe);
	140	xe_pm_runtime_put(xe);
	141
	142	return ret;
	143	}

drivers/gpu/drm/xe/xe_pci_sriov.h+13−0

@@ -0,0 +1,13 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2023-2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_PCI_SRIOV_H_
	7	#define _XE_PCI_SRIOV_H_
	8
	9	struct pci_dev;
	10
	11	int xe_pci_sriov_configure(struct pci_dev *pdev, int num_vfs);
	12
	13	#endif

drivers/gpu/drm/xe/xe_pci_types.h+4−0

 	u8 max_remote_tiles:2;
 	u8 has_asid:1;
+	u8 has_atomic_enable_pte_bit:1;
 	u8 has_flat_ccs:1;
+	u8 has_indirect_ring_state:1;
 	u8 has_range_tlb_invalidation:1;
 	u8 has_usm:1;
 };
 	u8 rel;
 	u64 hw_engine_mask;	/* hardware engines provided by media IP */
+	u8 has_indirect_ring_state:1;
 };
 struct gmdid_map {

drivers/gpu/drm/xe/xe_pcode.c+5−1

 #include <drm/drm_managed.h>
+#include "xe_assert.h"
 #include "xe_device.h"
 #include "xe_gt.h"
 #include "xe_mmio.h"
 {
 	int slept, wait = 10;
+	xe_gt_assert(gt, timeout_us > 0);
 	for (slept = 0; slept < timeout_us; slept += wait) {
 		if (locked)
 			*status = pcode_mailbox_rw(gt, mbox, &request, NULL, 1, true,
 	u32 status;
 	int ret;
+	xe_gt_assert(gt, timeout_base_ms <= 3);
 	mutex_lock(&gt->pcode.lock);
 	ret = pcode_try_request(gt, mbox, request, reply_mask, reply, &status,
 	 */
 	drm_err(&gt_to_xe(gt)->drm,
 		"PCODE timeout, retrying with preemption disabled\n");
-	drm_WARN_ON_ONCE(&gt_to_xe(gt)->drm, timeout_base_ms > 1);
 	preempt_disable();
 	ret = pcode_try_request(gt, mbox, request, reply_mask, reply, &status,
 				true, 50 * 1000, true);

drivers/gpu/drm/xe/xe_pm.c+22−20

  */
 #ifdef CONFIG_LOCKDEP
-struct lockdep_map xe_pm_runtime_lockdep_map = {
+static struct lockdep_map xe_pm_runtime_lockdep_map = {
 	.name = "xe_pm_runtime_lockdep_map"
 };
 #endif
 	if (err)
 		goto err;
-	xe_display_pm_suspend(xe);
+	xe_display_pm_suspend(xe, false);
 	for_each_gt(gt, xe, id) {
 		err = xe_gt_suspend(gt);
 		if (err) {
-			xe_display_pm_resume(xe);
+			xe_display_pm_resume(xe, false);
 			goto err;
 		}
 	}
 	xe_irq_resume(xe);
-	xe_display_pm_resume(xe);
+	xe_display_pm_resume(xe, false);
 	for_each_gt(gt, xe, id)
 		xe_gt_resume(gt);
 		err = xe_bo_evict_all(xe);
 		if (err)
 			goto out;
+		xe_display_pm_suspend(xe, true);
 	}
 	for_each_gt(gt, xe, id) {
 	}
 	xe_irq_suspend(xe);
+	if (xe->d3cold.allowed)
+		xe_display_pm_suspend_late(xe);
 out:
+	if (err)
+		xe_display_pm_resume(xe, true);
 	lock_map_release(&xe_pm_runtime_lockdep_map);
 	xe_pm_write_callback_task(xe, NULL);
 	return err;
 	lock_map_acquire(&xe_pm_runtime_lockdep_map);
-	/*
+	if (xe->d3cold.allowed) {
-	 * It can be possible that xe has allowed d3cold but other pcie devices
-	 * in gfx card soc would have blocked d3cold, therefore card has not
-	 * really lost power. Detecting primary Gt power is sufficient.
-	 */
-	gt = xe_device_get_gt(xe, 0);
-	xe->d3cold.power_lost = xe_guc_in_reset(&gt->uc.guc);
-	if (xe->d3cold.allowed && xe->d3cold.power_lost) {
 		err = xe_pcode_ready(xe, true);
 		if (err)
 			goto out;
+		xe_display_pm_resume_early(xe);
 		/*
 		 * This only restores pinned memory which is the memory
 		 * required for the GT(s) to resume.
 	for_each_gt(gt, xe, id)
 		xe_gt_resume(gt);
-	if (xe->d3cold.allowed && xe->d3cold.power_lost) {
+	if (xe->d3cold.allowed) {
+		xe_display_pm_resume(xe, true);
 		err = xe_bo_restore_user(xe);
 		if (err)
 			goto out;
  * xe_pm_runtime_get_if_active - Get a runtime_pm reference if device active
  * @xe: xe device instance
  *
- * Returns: Any number greater than or equal to 0 for success, negative error
+ * Return: True if device is awake (regardless the previous number of references)
- * code otherwise.
+ * and a new reference was taken, false otherwise.
  */
-int xe_pm_runtime_get_if_active(struct xe_device *xe)
+bool xe_pm_runtime_get_if_active(struct xe_device *xe)
 {
-	return pm_runtime_get_if_active(xe->drm.dev);
+	return pm_runtime_get_if_active(xe->drm.dev) > 0;
 }
 /**
- * xe_pm_runtime_get_if_in_use - Get a runtime_pm reference and resume if needed
+ * xe_pm_runtime_get_if_in_use - Get a new reference if device is active with previous ref taken
  * @xe: xe device instance
  *
- * Returns: True if device is awake and the reference was taken, false otherwise.
+ * Return: True if device is awake, a previous reference had been already taken,
+ * and a new reference was now taken, false otherwise.
  */
 bool xe_pm_runtime_get_if_in_use(struct xe_device *xe)
 {

drivers/gpu/drm/xe/xe_pm.h+2−7

 #include <linux/pm_runtime.h>
-/*
+#define DEFAULT_VRAM_THRESHOLD 300 /* in MB */
- * TODO: Threshold = 0 will block D3Cold.
- *       Before we can move this to a higher value (like 300), we need to:
- *           1. rewrite the VRAM save / restore to avoid buffer object locks
- */
-#define DEFAULT_VRAM_THRESHOLD 0 /* in MB */
 struct xe_device;
 void xe_pm_runtime_get(struct xe_device *xe);
 int xe_pm_runtime_get_ioctl(struct xe_device *xe);
 void xe_pm_runtime_put(struct xe_device *xe);
-int xe_pm_runtime_get_if_active(struct xe_device *xe);
+bool xe_pm_runtime_get_if_active(struct xe_device *xe);
 bool xe_pm_runtime_get_if_in_use(struct xe_device *xe);
 void xe_pm_runtime_get_noresume(struct xe_device *xe);
 bool xe_pm_runtime_resume_and_get(struct xe_device *xe);

drivers/gpu/drm/xe/xe_preempt_fence.c+11−3

@@ -23,11 +23,19 @@ static void preempt_fence_work_func(struct work_struct *w) · 22 unmodified lines
23	q->ops->suspend_wait(q);	23	q->ops->suspend_wait(q);
24		24
25	dma_fence_signal(&pfence->base);	25	dma_fence_signal(&pfence->base);
26	dma_fence_end_signalling(cookie);	26	/*
27		27	* Opt for keep everything in the fence critical section. This looks really strange since we
		28	* have just signalled the fence, however the preempt fences are all signalled via single
		29	* global ordered-wq, therefore anything that happens in this callback can easily block
		30	* progress on the entire wq, which itself may prevent other published preempt fences from
		31	* ever signalling. Therefore try to keep everything here in the callback in the fence
		32	* critical section. For example if something below grabs a scary lock like vm->lock,
		33	* lockdep should complain since we also hold that lock whilst waiting on preempt fences to
		34	* complete.
		35	*/
28	xe_vm_queue_rebind_worker(q->vm);	36	xe_vm_queue_rebind_worker(q->vm);
29
30	xe_exec_queue_put(q);	37	xe_exec_queue_put(q);
		38	dma_fence_end_signalling(cookie);
31	}	39	}
32		40
33	static const char *	41	static const char *

drivers/gpu/drm/xe/xe_pt.c+56−13

 	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
 	int ret;
-	if ((vma->gpuva.flags & XE_VMA_ATOMIC_PTE_BIT) &&
+	/**
-	    (is_devmem || !IS_DGFX(xe)))
+	 * Default atomic expectations for different allocation scenarios are as follows:
-		xe_walk.default_pte |= XE_USM_PPGTT_PTE_AE;
+	 *
+	 * 1. Traditional API: When the VM is not in LR mode:
+	 *    - Device atomics are expected to function with all allocations.
+	 *
+	 * 2. Compute/SVM API: When the VM is in LR mode:
+	 *    - Device atomics are the default behavior when the bo is placed in a single region.
+	 *    - In all other cases device atomics will be disabled with AE=0 until an application
+	 *      request differently using a ioctl like madvise.
+	 */
+	if (vma->gpuva.flags & XE_VMA_ATOMIC_PTE_BIT) {
+		if (xe_vm_in_lr_mode(xe_vma_vm(vma))) {
+			if (bo && xe_bo_has_single_placement(bo))
+				xe_walk.default_pte |= XE_USM_PPGTT_PTE_AE;
+			/**
+			 * If a SMEM+LMEM allocation is backed by SMEM, a device
+			 * atomics will cause a gpu page fault and which then
+			 * gets migrated to LMEM, bind such allocations with
+			 * device atomics enabled.
+			 */
+			else if (is_devmem && !xe_bo_has_single_placement(bo))
+				xe_walk.default_pte |= XE_USM_PPGTT_PTE_AE;
+		} else {
+			xe_walk.default_pte |= XE_USM_PPGTT_PTE_AE;
+		}
+		/**
+		 * Unset AE if the platform(PVC) doesn't support it on an
+		 * allocation
+		 */
+		if (!xe->info.has_device_atomics_on_smem && !is_devmem)
+			xe_walk.default_pte &= ~XE_USM_PPGTT_PTE_AE;
+	}
 	if (is_devmem) {
 		xe_walk.default_pte |= XE_PPGTT_PTE_DM;
 	pgoff_t end_offset;
 	XE_WARN_ON(!*child);
-	XE_WARN_ON(!level && xe_child->is_compact);
+	XE_WARN_ON(!level);
 	/*
 	 * Note that we're called from an entry callback, and we're dealing
 		.tile = tile,
 	};
 	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
+	u8 pt_mask = (vma->tile_present & ~vma->tile_invalidated);
-	if (!(vma->tile_present & BIT(tile->id)))
+	if (!(pt_mask & BIT(tile->id)))
 		return false;
 	(void)xe_pt_walk_shared(&pt->base, pt->level, xe_vma_start(vma),
 struct invalidation_fence {
 	struct xe_gt_tlb_invalidation_fence base;
 	struct xe_gt *gt;
-	struct xe_vma *vma;
 	struct dma_fence *fence;
 	struct dma_fence_cb cb;
 	struct work_struct work;
+	u64 start;
+	u64 end;
+	u32 asid;
 };
 static const char *
 		container_of(w, struct invalidation_fence, work);
 	trace_xe_gt_tlb_invalidation_fence_work_func(&ifence->base);
-	xe_gt_tlb_invalidation_vma(ifence->gt, &ifence->base, ifence->vma);
+	xe_gt_tlb_invalidation_range(ifence->gt, &ifence->base, ifence->start,
+				     ifence->end, ifence->asid);
 }
 static int invalidation_fence_init(struct xe_gt *gt,
 				   struct invalidation_fence *ifence,
 				   struct dma_fence *fence,
-				   struct xe_vma *vma)
+				   u64 start, u64 end, u32 asid)
 {
 	int ret;
 	dma_fence_get(&ifence->base.base);	/* Ref for caller */
 	ifence->fence = fence;
 	ifence->gt = gt;
-	ifence->vma = vma;
+	ifence->start = start;
+	ifence->end = end;
+	ifence->asid = asid;
 	INIT_WORK(&ifence->work, invalidation_fence_work_func);
 	ret = dma_fence_add_callback(fence, &ifence->cb, invalidation_fence_cb);
 		/* TLB invalidation must be done before signaling rebind */
 		if (ifence) {
-			int err = invalidation_fence_init(tile->primary_gt, ifence, fence,
+			int err = invalidation_fence_init(tile->primary_gt,
-							  vma);
+							  ifence, fence,
+							  xe_vma_start(vma),
+							  xe_vma_end(vma),
+							  xe_vma_vm(vma)->usm.asid);
 			if (err) {
 				dma_fence_put(fence);
 				kfree(ifence);
 	struct xe_pt *xe_child = container_of(*child, typeof(*xe_child), base);
 	XE_WARN_ON(!*child);
-	XE_WARN_ON(!level && xe_child->is_compact);
+	XE_WARN_ON(!level);
 	xe_pt_check_kill(addr, next, level - 1, xe_child, action, walk);
 			dma_fence_wait(fence, false);
 		/* TLB invalidation must be done before signaling unbind */
-		err = invalidation_fence_init(tile->primary_gt, ifence, fence, vma);
+		err = invalidation_fence_init(tile->primary_gt, ifence, fence,
+					      xe_vma_start(vma),
+					      xe_vma_end(vma),
+					      xe_vma_vm(vma)->usm.asid);
 		if (err) {
 			dma_fence_put(fence);
 			kfree(ifence);

drivers/gpu/drm/xe/xe_query.c+9−1

 #include "xe_bo.h"
 #include "xe_device.h"
 #include "xe_exec_queue.h"
+#include "xe_force_wake.h"
 #include "xe_ggtt.h"
 #include "xe_gt.h"
 #include "xe_guc_hwconfig.h"
 static size_t calc_topo_query_size(struct xe_device *xe)
 {
 	return xe->info.gt_count *
-		(3 * sizeof(struct drm_xe_query_topology_mask) +
+		(4 * sizeof(struct drm_xe_query_topology_mask) +
 		 sizeof_field(struct xe_gt, fuse_topo.g_dss_mask) +
 		 sizeof_field(struct xe_gt, fuse_topo.c_dss_mask) +
+		 sizeof_field(struct xe_gt, fuse_topo.l3_bank_mask) +
 		 sizeof_field(struct xe_gt, fuse_topo.eu_mask_per_dss));
 }
 		if (err)
 			return err;
+		topo.type = DRM_XE_TOPO_L3_BANK;
+		err = copy_mask(&query_ptr, &topo, gt->fuse_topo.l3_bank_mask,
+				sizeof(gt->fuse_topo.l3_bank_mask));
+		if (err)
+			return err;
 		topo.type = DRM_XE_TOPO_EU_PER_DSS;
 		err = copy_mask(&query_ptr, &topo,
 				gt->fuse_topo.eu_mask_per_dss,

drivers/gpu/drm/xe/xe_reg_sr.h+2−2

 #ifndef _XE_REG_SR_
 #define _XE_REG_SR_
-#include "xe_reg_sr_types.h"
 /*
  * Reg save/restore bookkeeping
  */
 struct xe_device;
 struct xe_gt;
 struct xe_hw_engine;
+struct xe_reg_sr;
+struct xe_reg_sr_entry;
 struct drm_printer;
 int xe_reg_sr_init(struct xe_reg_sr *sr, const char *name, struct xe_device *xe);

drivers/gpu/drm/xe/xe_res_cursor.h+0−2

 				   struct xe_res_cursor *cur)
 {
 	XE_WARN_ON(!sg);
-	XE_WARN_ON(!IS_ALIGNED(start, PAGE_SIZE) ||
-		   !IS_ALIGNED(size, PAGE_SIZE));
 	cur->node = NULL;
 	cur->start = start;
 	cur->remaining = size;

drivers/gpu/drm/xe/xe_ring_ops.c+33−19

 #include <generated/xe_wa_oob.h>
+#include "instructions/xe_gpu_commands.h"
 #include "instructions/xe_mi_commands.h"
 #include "regs/xe_engine_regs.h"
-#include "regs/xe_gpu_commands.h"
 #include "regs/xe_gt_regs.h"
 #include "regs/xe_lrc_layout.h"
 #include "xe_exec_queue_types.h"
 	return i;
 }
+static int emit_flush_dw(u32 *dw, int i)
+{
+	dw[i++] = MI_FLUSH_DW | MI_FLUSH_IMM_DW;
+	dw[i++] = 0;
+	dw[i++] = 0;
+	dw[i++] = 0;
+	return i;
+}
 static int emit_flush_imm_ggtt(u32 addr, u32 value, bool invalidate_tlb,
 			       u32 *dw, int i)
 {
 	i = emit_bb_start(batch_addr, ppgtt_flag, dw, i);
-	if (job->user_fence.used)
+	if (job->user_fence.used) {
+		i = emit_flush_dw(dw, i);
 		i = emit_store_imm_ppgtt_posted(job->user_fence.addr,
 						job->user_fence.value,
 						dw, i);
+	}
 	i = emit_flush_imm_ggtt(xe_lrc_seqno_ggtt_addr(lrc), seqno, false, dw, i);
 	i = emit_bb_start(batch_addr, ppgtt_flag, dw, i);
-	if (job->user_fence.used)
+	if (job->user_fence.used) {
+		i = emit_flush_dw(dw, i);
 		i = emit_store_imm_ppgtt_posted(job->user_fence.addr,
 						job->user_fence.value,
 						dw, i);
+	}
 	i = emit_flush_imm_ggtt(xe_lrc_seqno_ggtt_addr(lrc), seqno, false, dw, i);
 	dw[i++] = MI_ARB_ON_OFF | MI_ARB_DISABLE; /* Enabled again below */
-	i = emit_bb_start(job->batch_addr[0], BIT(8), dw, i);
+	i = emit_bb_start(job->ptrs[0].batch_addr, BIT(8), dw, i);
 	if (!IS_SRIOV_VF(gt_to_xe(job->q->gt))) {
 		/* XXX: Do we need this? Leaving for now. */
 		dw[i++] = preparser_disable(false);
 	}
-	i = emit_bb_start(job->batch_addr[1], BIT(8), dw, i);
+	i = emit_bb_start(job->ptrs[1].batch_addr, BIT(8), dw, i);
 	dw[i++] = MI_FLUSH_DW | MI_INVALIDATE_TLB | job->migrate_flush_flags |
 		MI_FLUSH_DW_OP_STOREDW | MI_FLUSH_IMM_DW;
 	xe_gt_assert(gt, job->q->width <= 1); /* no parallel submission for GSCCS */
-	__emit_job_gen12_simple(job, job->q->lrc,
+	__emit_job_gen12_simple(job, job->q->lrc[0],
-				job->batch_addr[0],
+				job->ptrs[0].batch_addr,
-				xe_sched_job_seqno(job));
+				xe_sched_job_lrc_seqno(job));
 }
 static void emit_job_gen12_copy(struct xe_sched_job *job)
 	int i;
 	if (xe_sched_job_is_migration(job->q)) {
-		emit_migration_job_gen12(job, job->q->lrc,
+		emit_migration_job_gen12(job, job->q->lrc[0],
-					 xe_sched_job_seqno(job));
+					 xe_sched_job_lrc_seqno(job));
 		return;
 	}
 	for (i = 0; i < job->q->width; ++i)
-		__emit_job_gen12_simple(job, job->q->lrc + i,
+		__emit_job_gen12_simple(job, job->q->lrc[i],
-				        job->batch_addr[i],
+					job->ptrs[i].batch_addr,
-				        xe_sched_job_seqno(job));
+					xe_sched_job_lrc_seqno(job));
 }
 static void emit_job_gen12_video(struct xe_sched_job *job)
 	/* FIXME: Not doing parallel handshake for now */
 	for (i = 0; i < job->q->width; ++i)
-		__emit_job_gen12_video(job, job->q->lrc + i,
+		__emit_job_gen12_video(job, job->q->lrc[i],
-				       job->batch_addr[i],
+				       job->ptrs[i].batch_addr,
-				       xe_sched_job_seqno(job));
+				       xe_sched_job_lrc_seqno(job));
 }
 static void emit_job_gen12_render_compute(struct xe_sched_job *job)
 	int i;
 	for (i = 0; i < job->q->width; ++i)
-		__emit_job_gen12_render_compute(job, job->q->lrc + i,
+		__emit_job_gen12_render_compute(job, job->q->lrc[i],
-						job->batch_addr[i],
+						job->ptrs[i].batch_addr,
-						xe_sched_job_seqno(job));
+						xe_sched_job_lrc_seqno(job));
 }
 static const struct xe_ring_ops ring_ops_gen12_gsc = {

drivers/gpu/drm/xe/xe_rtp.c+6−0

@@ -323,3 +323,9 @@ bool xe_rtp_match_first_gslice_fused_off(const struct xe_gt *gt, · 322 unmodified lines
323		323
324	return dss >= dss_per_gslice;	324	return dss >= dss_per_gslice;
325	}	325	}
		326
		327	bool xe_rtp_match_when_media2000(const struct xe_gt *gt,
		328	const struct xe_hw_engine *hwe)
		329	{
		330	return (gt_to_xe(gt))->info.media_verx100 == 2000;
		331	}

drivers/gpu/drm/xe/xe_rtp.h+16−2

  *	};
  */
 #define XE_RTP_RULES(...)							\
-	.n_rules = _XE_COUNT_ARGS(__VA_ARGS__),					\
+	.n_rules = COUNT_ARGS(__VA_ARGS__),					\
 	.rules = (const struct xe_rtp_rule[]) {					\
 		XE_RTP_PASTE_FOREACH(RULE_, COMMA, (__VA_ARGS__))	\
 	}
  *	};
  */
 #define XE_RTP_ACTIONS(...)							\
-	.n_actions = _XE_COUNT_ARGS(__VA_ARGS__),				\
+	.n_actions = COUNT_ARGS(__VA_ARGS__),					\
 	.actions = (const struct xe_rtp_action[]) {				\
 		XE_RTP_PASTE_FOREACH(ACTION_, COMMA, (__VA_ARGS__))	\
 	}
 bool xe_rtp_match_first_gslice_fused_off(const struct xe_gt *gt,
 					 const struct xe_hw_engine *hwe);
+/*
+ * xe_rtp_match_when_media2000 - Match when media GT version 2000
+ *
+ * @gt: GT structure
+ * @hwe: Engine instance
+ *
+ * Its one of the case where we need to apply workaround on primary GT
+ * based on if media GT version 2000 is present. Thus this API will help
+ * us to match media version 2000.
+ *
+ * Returns: true if media GT version 2000, false otherwise.
+ */
+bool xe_rtp_match_when_media2000(const struct xe_gt *gt,
+				 const struct xe_hw_engine *hwe);
 #endif

drivers/gpu/drm/xe/xe_rtp_helpers.h+10−16

 #error "This header is supposed to be included by xe_rtp.h only"
 #endif
+#include "xe_args.h"
 /*
  * Helper macros - not to be used outside this header.
  */
 #define _XE_ESC(...) __VA_ARGS__
-#define _XE_COUNT_ARGS(...) _XE_ESC(__XE_COUNT_ARGS(__VA_ARGS__, 5, 4, 3, 2, 1,))
-#define __XE_COUNT_ARGS(_, _5, _4, _3, _2, X_, ...) X_
-#define _XE_FIRST(...) _XE_ESC(__XE_FIRST(__VA_ARGS__,))
-#define __XE_FIRST(x_, ...) x_
-#define _XE_TUPLE_TAIL(...) _XE_ESC(__XE_TUPLE_TAIL(__VA_ARGS__))
-#define __XE_TUPLE_TAIL(x_, ...) (__VA_ARGS__)
-#define _XE_DROP_FIRST(x_, ...) __VA_ARGS__
+#define _XE_TUPLE_TAIL(...) (DROP_FIRST_ARG(__VA_ARGS__))
-#define _XE_RTP_CONCAT(a, b) __XE_RTP_CONCAT(a, b)
+#define _XE_RTP_CONCAT(a, b) CONCATENATE(XE_RTP_, CONCATENATE(a, b))
-#define __XE_RTP_CONCAT(a, b) XE_RTP_ ## a ## b
 #define __XE_RTP_PASTE_SEP_COMMA		,
 #define __XE_RTP_PASTE_SEP_BITWISE_OR		|
  *
  *	XE_RTP_TEST_FOO BANANA XE_RTP_TEST_BAR
  */
-#define XE_RTP_PASTE_FOREACH(prefix_, sep_, args_) _XE_ESC(_XE_RTP_CONCAT(PASTE_, _XE_COUNT_ARGS args_)(prefix_, sep_, args_))
+#define XE_RTP_PASTE_FOREACH(prefix_, sep_, args_) _XE_RTP_CONCAT(PASTE_, COUNT_ARGS args_)(prefix_, sep_, args_)
-#define XE_RTP_PASTE_1(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, _XE_FIRST args_)
+#define XE_RTP_PASTE_1(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, FIRST_ARG args_)
-#define XE_RTP_PASTE_2(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, _XE_FIRST args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_1(prefix_, sep_, _XE_TUPLE_TAIL args_)
+#define XE_RTP_PASTE_2(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, FIRST_ARG args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_1(prefix_, sep_, _XE_TUPLE_TAIL args_)
-#define XE_RTP_PASTE_3(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, _XE_FIRST args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_2(prefix_, sep_, _XE_TUPLE_TAIL args_)
+#define XE_RTP_PASTE_3(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, FIRST_ARG args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_2(prefix_, sep_, _XE_TUPLE_TAIL args_)
-#define XE_RTP_PASTE_4(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, _XE_FIRST args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_3(prefix_, sep_, _XE_TUPLE_TAIL args_)
+#define XE_RTP_PASTE_4(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, FIRST_ARG args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_3(prefix_, sep_, _XE_TUPLE_TAIL args_)
 /*
  * XE_RTP_DROP_CAST - Drop cast to convert a compound statement to a initializer
  *
  *	{ .a = 10 }
  */
-#define XE_RTP_DROP_CAST(...) _XE_ESC(_XE_DROP_FIRST _XE_ESC __VA_ARGS__)
+#define XE_RTP_DROP_CAST(...) _XE_ESC(DROP_FIRST_ARG _XE_ESC __VA_ARGS__)
 #endif

drivers/gpu/drm/xe/xe_sched_job.c+100−81

@@ -6,7 +6,7 @@ · 5 unmodified lines
6	#include "xe_sched_job.h"	6	#include "xe_sched_job.h"
7		7
8	#include <drm/xe_drm.h>	8	#include <drm/xe_drm.h>
9	#include <linux/dma-fence-array.h>	9	#include <linux/dma-fence-chain.h>
10	#include <linux/slab.h>	10	#include <linux/slab.h>
11		11
12	#include "xe_device.h"	12	#include "xe_device.h"
@@ -29,7 +29,7 @@ int __init xe_sched_job_module_init(void) · 16 unmodified lines
29	xe_sched_job_slab =	29	xe_sched_job_slab =
30	kmem_cache_create("xe_sched_job",	30	kmem_cache_create("xe_sched_job",
31	sizeof(struct xe_sched_job) +	31	sizeof(struct xe_sched_job) +
32	sizeof(u64), 0,	32	sizeof(struct xe_job_ptrs), 0,
33	SLAB_HWCACHE_ALIGN, NULL);	33	SLAB_HWCACHE_ALIGN, NULL);
34	if (!xe_sched_job_slab)	34	if (!xe_sched_job_slab)
35	return -ENOMEM;	35	return -ENOMEM;
@@ -37,7 +37,7 @@ int __init xe_sched_job_module_init(void) · 1 unmodified line
37	xe_sched_job_parallel_slab =	37	xe_sched_job_parallel_slab =
38	kmem_cache_create("xe_sched_job_parallel",	38	kmem_cache_create("xe_sched_job_parallel",
39	sizeof(struct xe_sched_job) +	39	sizeof(struct xe_sched_job) +
40	sizeof(u64) *	40	sizeof(struct xe_job_ptrs) *
41	XE_HW_ENGINE_MAX_INSTANCE, 0,	41	XE_HW_ENGINE_MAX_INSTANCE, 0,
42	SLAB_HWCACHE_ALIGN, NULL);	42	SLAB_HWCACHE_ALIGN, NULL);
43	if (!xe_sched_job_parallel_slab) {	43	if (!xe_sched_job_parallel_slab) {
@@ -79,26 +79,33 @@ static struct xe_device job_to_xe(struct xe_sched_job job) · 35 unmodified lines
79	return gt_to_xe(job->q->gt);	79	return gt_to_xe(job->q->gt);
80	}	80	}
81		81
		82	/* Free unused pre-allocated fences */
		83	static void xe_sched_job_free_fences(struct xe_sched_job *job)
		84	{
		85	int i;
		86
		87	for (i = 0; i < job->q->width; ++i) {
		88	struct xe_job_ptrs *ptrs = &job->ptrs[i];
		89
		90	if (ptrs->lrc_fence)
		91	xe_lrc_free_seqno_fence(ptrs->lrc_fence);
		92	if (ptrs->chain_fence)
		93	dma_fence_chain_free(ptrs->chain_fence);
		94	}
		95	}
		96
82	struct xe_sched_job xe_sched_job_create(struct xe_exec_queue q,	97	struct xe_sched_job xe_sched_job_create(struct xe_exec_queue q,
83	u64 *batch_addr)	98	u64 *batch_addr)
84	{	99	{
85	struct xe_sched_job *job;
86	struct dma_fence **fences;
87	bool is_migration = xe_sched_job_is_migration(q);	100	bool is_migration = xe_sched_job_is_migration(q);
		101	struct xe_sched_job *job;
88	int err;	102	int err;
89	int i, j;	103	int i;
90	u32 width;	104	u32 width;
91		105
92	/* only a kernel context can submit a vm-less job */	106	/* only a kernel context can submit a vm-less job */
93	XE_WARN_ON(!q->vm && !(q->flags & EXEC_QUEUE_FLAG_KERNEL));	107	XE_WARN_ON(!q->vm && !(q->flags & EXEC_QUEUE_FLAG_KERNEL));
94		108
95	/* Migration and kernel engines have their own locking */
96	if (!(q->flags & (EXEC_QUEUE_FLAG_KERNEL \| EXEC_QUEUE_FLAG_VM))) {
97	lockdep_assert_held(&q->vm->lock);
98	if (!xe_vm_in_lr_mode(q->vm))
99	xe_vm_assert_held(q->vm);
100	}
101
102	job = job_alloc(xe_exec_queue_is_parallel(q) \|\| is_migration);	109	job = job_alloc(xe_exec_queue_is_parallel(q) \|\| is_migration);
103	if (!job)	110	if (!job)
104	return ERR_PTR(-ENOMEM);	111	return ERR_PTR(-ENOMEM);
@@ -111,44 +118,25 @@ struct xe_sched_job xe_sched_job_create(struct xe_exec_queue q, · 6 unmodified lines
111	if (err)	118	if (err)
112	goto err_free;	119	goto err_free;
113		120
114	if (!xe_exec_queue_is_parallel(q)) {	121	for (i = 0; i < q->width; ++i) {
115	job->fence = xe_lrc_create_seqno_fence(q->lrc);	122	struct dma_fence *fence = xe_lrc_alloc_seqno_fence();
116	if (IS_ERR(job->fence)) {	123	struct dma_fence_chain *chain;
117	err = PTR_ERR(job->fence);
118	goto err_sched_job;
119	}
120	} else {
121	struct dma_fence_array *cf;
122		124
123	fences = kmalloc_array(q->width, sizeof(*fences), GFP_KERNEL);	125	if (IS_ERR(fence)) {
124	if (!fences) {	126	err = PTR_ERR(fence);
125	err = -ENOMEM;
126	goto err_sched_job;	127	goto err_sched_job;
127	}	128	}
		129	job->ptrs[i].lrc_fence = fence;
128		130
129	for (j = 0; j < q->width; ++j) {	131	if (i + 1 == q->width)
130	fences[j] = xe_lrc_create_seqno_fence(q->lrc + j);	132	continue;
131	if (IS_ERR(fences[j])) {
132	err = PTR_ERR(fences[j]);
133	goto err_fences;
134	}
135	}
136		133
137	cf = dma_fence_array_create(q->width, fences,	134	chain = dma_fence_chain_alloc();
138	q->parallel.composite_fence_ctx,	135	if (!chain) {
139	q->parallel.composite_fence_seqno++,
140	false);
141	if (!cf) {
142	--q->parallel.composite_fence_seqno;
143	err = -ENOMEM;	136	err = -ENOMEM;
144	goto err_fences;	137	goto err_sched_job;
145	}	138	}
146		139	job->ptrs[i].chain_fence = chain;
147	/* Sanity check */
148	for (j = 0; j < q->width; ++j)
149	xe_assert(job_to_xe(job), cf->base.seqno == fences[j]->seqno);
150
151	job->fence = &cf->base;
152	}	140	}
153		141
154	width = q->width;	142	width = q->width;
@@ -156,23 +144,14 @@ struct xe_sched_job xe_sched_job_create(struct xe_exec_queue q, · 1 unmodified line
156	width = 2;	144	width = 2;
157		145
158	for (i = 0; i < width; ++i)	146	for (i = 0; i < width; ++i)
159	job->batch_addr[i] = batch_addr[i];	147	job->ptrs[i].batch_addr = batch_addr[i];
160
161	/* All other jobs require a VM to be open which has a ref */
162	if (unlikely(q->flags & EXEC_QUEUE_FLAG_KERNEL))
163	xe_pm_runtime_get_noresume(job_to_xe(job));
164	xe_device_assert_mem_access(job_to_xe(job));
165		148
		149	xe_pm_runtime_get_noresume(job_to_xe(job));
166	trace_xe_sched_job_create(job);	150	trace_xe_sched_job_create(job);
167	return job;	151	return job;
168		152
169	err_fences:
170	for (j = j - 1; j >= 0; --j) {
171	--q->lrc[j].fence_ctx.next_seqno;
172	dma_fence_put(fences[j]);
173	}
174	kfree(fences);
175	err_sched_job:	153	err_sched_job:
		154	xe_sched_job_free_fences(job);
176	drm_sched_job_cleanup(&job->drm);	155	drm_sched_job_cleanup(&job->drm);
177	err_free:	156	err_free:
178	xe_exec_queue_put(q);	157	xe_exec_queue_put(q);
@@ -191,36 +170,42 @@ void xe_sched_job_destroy(struct kref *ref) · 12 unmodified lines
191	{	170	{
192	struct xe_sched_job *job =	171	struct xe_sched_job *job =
193	container_of(ref, struct xe_sched_job, refcount);	172	container_of(ref, struct xe_sched_job, refcount);
		173	struct xe_device *xe = job_to_xe(job);
194		174
195	if (unlikely(job->q->flags & EXEC_QUEUE_FLAG_KERNEL))	175	xe_sched_job_free_fences(job);
196	xe_pm_runtime_put(job_to_xe(job));
197	xe_exec_queue_put(job->q);	176	xe_exec_queue_put(job->q);
198	dma_fence_put(job->fence);	177	dma_fence_put(job->fence);
199	drm_sched_job_cleanup(&job->drm);	178	drm_sched_job_cleanup(&job->drm);
200	job_free(job);	179	job_free(job);
		180	xe_pm_runtime_put(xe);
201	}	181	}
202		182
203	void xe_sched_job_set_error(struct xe_sched_job *job, int error)	183	/* Set the error status under the fence to avoid racing with signaling */
		184	static bool xe_fence_set_error(struct dma_fence *fence, int error)
204	{	185	{
205	if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &job->fence->flags))	186	unsigned long irq_flags;
206	return;	187	bool signaled;
207		188
208	dma_fence_set_error(job->fence, error);	189	spin_lock_irqsave(fence->lock, irq_flags);
		190	signaled = test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags);
		191	if (!signaled)
		192	dma_fence_set_error(fence, error);
		193	spin_unlock_irqrestore(fence->lock, irq_flags);
209		194
210	if (dma_fence_is_array(job->fence)) {	195	return signaled;
211	struct dma_fence_array *array =	196	}
212	to_dma_fence_array(job->fence);	197
213	struct dma_fence **child = array->fences;	198	void xe_sched_job_set_error(struct xe_sched_job *job, int error)
214	unsigned int nchild = array->num_fences;	199	{
		200	if (xe_fence_set_error(job->fence, error))
		201	return;
215		202
216	do {	203	if (dma_fence_is_chain(job->fence)) {
217	struct dma_fence current_fence = child++;	204	struct dma_fence *iter;
218		205
219	if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT,	206	dma_fence_chain_for_each(iter, job->fence)
220	&current_fence->flags))	207	xe_fence_set_error(dma_fence_chain_contained(iter),
221	continue;	208	error);
222	dma_fence_set_error(current_fence, error);
223	} while (--nchild);
224	}	209	}
225		210
226	trace_xe_sched_job_set_error(job);	211	trace_xe_sched_job_set_error(job);
@@ -231,30 +216,42 @@ void xe_sched_job_set_error(struct xe_sched_job *job, int error) · 4 unmodified lines
231		216
232	bool xe_sched_job_started(struct xe_sched_job *job)	217	bool xe_sched_job_started(struct xe_sched_job *job)
233	{	218	{
234	struct xe_lrc *lrc = job->q->lrc;	219	struct xe_lrc *lrc = job->q->lrc[0];
235		220
236	return !__dma_fence_is_later(xe_sched_job_seqno(job),	221	return !__dma_fence_is_later(xe_sched_job_lrc_seqno(job),
237	xe_lrc_start_seqno(lrc),	222	xe_lrc_start_seqno(lrc),
238	job->fence->ops);	223	dma_fence_chain_contained(job->fence)->ops);
239	}	224	}
240		225
241	bool xe_sched_job_completed(struct xe_sched_job *job)	226	bool xe_sched_job_completed(struct xe_sched_job *job)
242	{	227	{
243	struct xe_lrc *lrc = job->q->lrc;	228	struct xe_lrc *lrc = job->q->lrc[0];
244		229
245	/*	230	/*
246	* Can safely check just LRC[0] seqno as that is last seqno written when	231	* Can safely check just LRC[0] seqno as that is last seqno written when
247	* parallel handshake is done.	232	* parallel handshake is done.
248	*/	233	*/
249		234
250	return !__dma_fence_is_later(xe_sched_job_seqno(job), xe_lrc_seqno(lrc),	235	return !__dma_fence_is_later(xe_sched_job_lrc_seqno(job),
251	job->fence->ops);	236	xe_lrc_seqno(lrc),
		237	dma_fence_chain_contained(job->fence)->ops);
252	}	238	}
253		239
254	void xe_sched_job_arm(struct xe_sched_job *job)	240	void xe_sched_job_arm(struct xe_sched_job *job)
255	{	241	{
256	struct xe_exec_queue *q = job->q;	242	struct xe_exec_queue *q = job->q;
		243	struct dma_fence fence, prev;
257	struct xe_vm *vm = q->vm;	244	struct xe_vm *vm = q->vm;
		245	u64 seqno = 0;
		246	int i;
		247
		248	/* Migration and kernel engines have their own locking */
		249	if (IS_ENABLED(CONFIG_LOCKDEP) &&
		250	!(q->flags & (EXEC_QUEUE_FLAG_KERNEL \| EXEC_QUEUE_FLAG_VM))) {
		251	lockdep_assert_held(&q->vm->lock);
		252	if (!xe_vm_in_lr_mode(q->vm))
		253	xe_vm_assert_held(q->vm);
		254	}
258		255
259	if (vm && !xe_sched_job_is_migration(q) && !xe_vm_in_lr_mode(vm) &&	256	if (vm && !xe_sched_job_is_migration(q) && !xe_vm_in_lr_mode(vm) &&
260	(vm->batch_invalidate_tlb \|\| vm->tlb_flush_seqno != q->tlb_flush_seqno)) {	257	(vm->batch_invalidate_tlb \|\| vm->tlb_flush_seqno != q->tlb_flush_seqno)) {
@@ -263,6 +260,27 @@ void xe_sched_job_arm(struct xe_sched_job *job) · 2 unmodified lines
263	job->ring_ops_flush_tlb = true;	260	job->ring_ops_flush_tlb = true;
264	}	261	}
265		262
		263	/* Arm the pre-allocated fences */
		264	for (i = 0; i < q->width; prev = fence, ++i) {
		265	struct dma_fence_chain *chain;
		266
		267	fence = job->ptrs[i].lrc_fence;
		268	xe_lrc_init_seqno_fence(q->lrc[i], fence);
		269	job->ptrs[i].lrc_fence = NULL;
		270	if (!i) {
		271	job->lrc_seqno = fence->seqno;
		272	continue;
		273	} else {
		274	xe_assert(gt_to_xe(q->gt), job->lrc_seqno == fence->seqno);
		275	}
		276
		277	chain = job->ptrs[i - 1].chain_fence;
		278	dma_fence_chain_init(chain, prev, fence, seqno++);
		279	job->ptrs[i - 1].chain_fence = NULL;
		280	fence = &chain->base;
		281	}
		282
		283	job->fence = fence;
266	drm_sched_job_arm(&job->drm);	284	drm_sched_job_arm(&job->drm);
267	}	285	}
268		286
@@ -322,7 +340,8 @@ xe_sched_job_snapshot_capture(struct xe_sched_job *job) · 53 unmodified lines
322		340
323	snapshot->batch_addr_len = q->width;	341	snapshot->batch_addr_len = q->width;
324	for (i = 0; i < q->width; i++)	342	for (i = 0; i < q->width; i++)
325	snapshot->batch_addr[i] = xe_device_uncanonicalize_addr(xe, job->batch_addr[i]);	343	snapshot->batch_addr[i] =
		344	xe_device_uncanonicalize_addr(xe, job->ptrs[i].batch_addr);
326		345
327	return snapshot;	346	return snapshot;
328	}	347	}

drivers/gpu/drm/xe/xe_sched_job.h+6−1

 static inline u32 xe_sched_job_seqno(struct xe_sched_job *job)
 {
-	return job->fence->seqno;
+	return job->fence ? job->fence->seqno : 0;
+}
+static inline u32 xe_sched_job_lrc_seqno(struct xe_sched_job *job)
+{
+	return job->lrc_seqno;
 }
 static inline void

drivers/gpu/drm/xe/xe_sched_job_types.h+18−2

 #include <drm/gpu_scheduler.h>
 struct xe_exec_queue;
+struct dma_fence;
+struct dma_fence_chain;
+/**
+ * struct xe_job_ptrs - Per hw engine instance data
+ */
+struct xe_job_ptrs {
+	/** @lrc_fence: Pre-allocated uinitialized lrc fence.*/
+	struct dma_fence *lrc_fence;
+	/** @chain_fence: Pre-allocated ninitialized fence chain node. */
+	struct dma_fence_chain *chain_fence;
+	/** @batch_addr: Batch buffer address. */
+	u64 batch_addr;
+};
 /**
  * struct xe_sched_job - XE schedule job (batch buffer tracking)
 		/** @user_fence.value: write back value */
 		u64 value;
 	} user_fence;
+	/** @lrc_seqno: LRC seqno */
+	u32 lrc_seqno;
 	/** @migrate_flush_flags: Additional flush flags for migration jobs */
 	u32 migrate_flush_flags;
 	/** @ring_ops_flush_tlb: The ring ops need to flush TLB before payload. */
 	bool ring_ops_flush_tlb;
-	/** @batch_addr: batch buffer address of job */
+	/** @ptrs: per instance pointers. */
-	u64 batch_addr[];
+	struct xe_job_ptrs ptrs[];
 };
 struct xe_sched_job_snapshot {

drivers/gpu/drm/xe/xe_sriov.c+11−0

  */
 void xe_sriov_probe_early(struct xe_device *xe)
 {
+	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
 	enum xe_sriov_mode mode = XE_SRIOV_MODE_NONE;
 	bool has_sriov = xe->info.has_sriov;
 			mode = XE_SRIOV_MODE_VF;
 		else if (xe_sriov_pf_readiness(xe))
 			mode = XE_SRIOV_MODE_PF;
+	} else if (pci_sriov_get_totalvfs(pdev)) {
+		/*
+		 * Even if we have not enabled SR-IOV support using the
+		 * platform specific has_sriov flag, the hardware may still
+		 * report SR-IOV capability and the PCI layer may wrongly
+		 * advertise driver support to enable VFs. Explicitly reset
+		 * the number of supported VFs to zero to avoid confusion.
+		 */
+		drm_info(&xe->drm, "Support for SR-IOV is not available\n");
+		pci_sriov_set_totalvfs(pdev, 0);
 	}
 	xe_assert(xe, !xe->sriov.__mode);

drivers/gpu/drm/xe/xe_sync.c+15−0

@@ -338,6 +338,21 @@ err_out: · 337 unmodified lines
338	return ERR_PTR(-ENOMEM);	338	return ERR_PTR(-ENOMEM);
339	}	339	}
340		340
		341	/**
		342	* __xe_sync_ufence_get() - Get user fence from user fence
		343	* @ufence: input user fence
		344	*
		345	* Get a user fence reference from user fence
		346	*
		347	* Return: xe_user_fence pointer with reference
		348	*/
		349	struct xe_user_fence __xe_sync_ufence_get(struct xe_user_fence ufence)
		350	{
		351	user_fence_get(ufence);
		352
		353	return ufence;
		354	}
		355
341	/**	356	/**
342	* xe_sync_ufence_get() - Get user fence from sync	357	* xe_sync_ufence_get() - Get user fence from sync
343	* @sync: input sync	358	* @sync: input sync

drivers/gpu/drm/xe/xe_sync.h+1−0

@@ -37,6 +37,7 @@ static inline bool xe_sync_is_ufence(struct xe_sync_entry *sync) · 36 unmodified lines
37	return !!sync->ufence;	37	return !!sync->ufence;
38	}	38	}
39		39
		40	struct xe_user_fence __xe_sync_ufence_get(struct xe_user_fence ufence);
40	struct xe_user_fence xe_sync_ufence_get(struct xe_sync_entry sync);	41	struct xe_user_fence xe_sync_ufence_get(struct xe_sync_entry sync);
41	void xe_sync_ufence_put(struct xe_user_fence *ufence);	42	void xe_sync_ufence_put(struct xe_user_fence *ufence);
42	int xe_sync_ufence_get_status(struct xe_user_fence *ufence);	43	int xe_sync_ufence_get_status(struct xe_user_fence *ufence);

drivers/gpu/drm/xe/xe_tile_sysfs.c+2−2

@@ -22,7 +22,7 @@ static const struct kobj_type xe_tile_sysfs_kobj_type = { · 21 unmodified lines
22	.sysfs_ops = &kobj_sysfs_ops,	22	.sysfs_ops = &kobj_sysfs_ops,
23	};	23	};
24		24
25	static void tile_sysfs_fini(struct drm_device drm, void arg)	25	static void tile_sysfs_fini(void *arg)
26	{	26	{
27	struct xe_tile *tile = arg;	27	struct xe_tile *tile = arg;
28		28
@@ -55,5 +55,5 @@ int xe_tile_sysfs_init(struct xe_tile *tile) · 26 unmodified lines
55	if (err)	55	if (err)
56	return err;	56	return err;
57		57
58	return drmm_add_action_or_reset(&xe->drm, tile_sysfs_fini, tile);	58	return devm_add_action_or_reset(xe->drm.dev, tile_sysfs_fini, tile);
59	}	59	}

drivers/gpu/drm/xe/xe_trace.h+7−4

 		    TP_STRUCT__entry(
 			     __field(u32, seqno)
+			     __field(u32, lrc_seqno)
 			     __field(u16, guc_id)
 			     __field(u32, guc_state)
 			     __field(u32, flags)
 		    TP_fast_assign(
 			   __entry->seqno = xe_sched_job_seqno(job);
+			   __entry->lrc_seqno = xe_sched_job_lrc_seqno(job);
 			   __entry->guc_id = job->q->guc->id;
 			   __entry->guc_state =
 			   atomic_read(&job->q->guc->state);
 			   __entry->flags = job->q->flags;
-			   __entry->error = job->fence->error;
+			   __entry->error = job->fence ? job->fence->error : 0;
 			   __entry->fence = job->fence;
-			   __entry->batch_addr = (u64)job->batch_addr[0];
+			   __entry->batch_addr = (u64)job->ptrs[0].batch_addr;
 			   ),
-		    TP_printk("fence=%p, seqno=%u, guc_id=%d, batch_addr=0x%012llx, guc_state=0x%x, flags=0x%x, error=%d",
+		    TP_printk("fence=%p, seqno=%u, lrc_seqno=%u, guc_id=%d, batch_addr=0x%012llx, guc_state=0x%x, flags=0x%x, error=%d",
-			      __entry->fence, __entry->seqno, __entry->guc_id,
+			      __entry->fence, __entry->seqno,
+			      __entry->lrc_seqno, __entry->guc_id,
 			      __entry->batch_addr, __entry->guc_state,
 			      __entry->flags, __entry->error)
 );

drivers/gpu/drm/xe/xe_ttm_stolen_mgr.c+2−0

 #include <generated/xe_wa_oob.h>
+#include "regs/xe_bars.h"
 #include "regs/xe_gt_regs.h"
 #include "regs/xe_regs.h"
 #include "xe_bo.h"
 #include "xe_device.h"
 #include "xe_gt.h"
+#include "xe_gt_printk.h"
 #include "xe_mmio.h"
 #include "xe_res_cursor.h"
 #include "xe_sriov.h"

drivers/gpu/drm/xe/xe_uc.c+42−16

 #include "xe_uc.h"
+#include "xe_assert.h"
 #include "xe_device.h"
 #include "xe_gsc.h"
 #include "xe_gsc_proxy.h"
 #include "xe_gt.h"
+#include "xe_gt_printk.h"
+#include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
-#include "xe_guc_db_mgr.h"
 #include "xe_guc_pc.h"
-#include "xe_guc_submit.h"
 #include "xe_huc.h"
+#include "xe_sriov.h"
 #include "xe_uc_fw.h"
 #include "xe_wopcm.h"
 		goto err;
 	if (!xe_device_uc_enabled(uc_to_xe(uc)))
-		goto err;
+		return 0;
-	ret = xe_wopcm_init(&uc->wopcm);
+	if (IS_SRIOV_VF(uc_to_xe(uc)))
-	if (ret)
+		return 0;
-		goto err;
-	ret = xe_guc_submit_init(&uc->guc);
+	ret = xe_wopcm_init(&uc->wopcm);
 	if (ret)
 		goto err;
-	ret = xe_guc_db_mgr_init(&uc->guc.dbm, ~0);
+	return 0;
 err:
+	xe_gt_err(uc_to_gt(uc), "Failed to initialize uC (%pe)\n", ERR_PTR(ret));
 	return ret;
 }
 	return 0;
 }
+static int vf_uc_init_hw(struct xe_uc *uc)
+{
+	int err;
+	err = xe_uc_sanitize_reset(uc);
+	if (err)
+		return err;
+	err = xe_guc_enable_communication(&uc->guc);
+	if (err)
+		return err;
+	err = xe_gt_sriov_vf_connect(uc_to_gt(uc));
+	if (err)
+		return err;
+	uc->guc.submission_state.enabled = true;
+	err = xe_gt_record_default_lrcs(uc_to_gt(uc));
+	if (err)
+		return err;
+	return 0;
+}
 /*
  * Should be called during driver load, after every GT reset, and after every
  * suspend to reload / auth the firmwares.
 	if (!xe_device_uc_enabled(uc_to_xe(uc)))
 		return 0;
+	if (IS_SRIOV_VF(uc_to_xe(uc)))
+		return vf_uc_init_hw(uc);
 	ret = xe_huc_upload(&uc->huc);
 	if (ret)
 		return ret;
 	xe_guc_stop_prepare(&uc->guc);
 }
-int xe_uc_stop(struct xe_uc *uc)
+void xe_uc_stop(struct xe_uc *uc)
 {
 	/* GuC submission not enabled, nothing to do */
 	if (!xe_device_uc_enabled(uc_to_xe(uc)))
-		return 0;
+		return;
-	return xe_guc_stop(&uc->guc);
+	xe_guc_stop(&uc->guc);
 }
 int xe_uc_start(struct xe_uc *uc)
 int xe_uc_suspend(struct xe_uc *uc)
 {
-	int ret;
 	/* GuC submission not enabled, nothing to do */
 	if (!xe_device_uc_enabled(uc_to_xe(uc)))
 		return 0;
 	uc_reset_wait(uc);
-	ret = xe_uc_stop(uc);
+	xe_uc_stop(uc);
-	if (ret)
-		return ret;
 	return xe_guc_suspend(&uc->guc);
 }

drivers/gpu/drm/xe/xe_uc.h+2−2

@@ -6,7 +6,7 @@ · 5 unmodified lines
6	#ifndef _XE_UC_H_	6	#ifndef _XE_UC_H_
7	#define _XE_UC_H_	7	#define _XE_UC_H_
8		8
9	#include "xe_uc_types.h"	9	struct xe_uc;
10		10
11	int xe_uc_init(struct xe_uc *uc);	11	int xe_uc_init(struct xe_uc *uc);
12	int xe_uc_init_hwconfig(struct xe_uc *uc);	12	int xe_uc_init_hwconfig(struct xe_uc *uc);
@@ -16,7 +16,7 @@ int xe_uc_fini_hw(struct xe_uc *uc); · 3 unmodified lines
16	void xe_uc_gucrc_disable(struct xe_uc *uc);	16	void xe_uc_gucrc_disable(struct xe_uc *uc);
17	int xe_uc_reset_prepare(struct xe_uc *uc);	17	int xe_uc_reset_prepare(struct xe_uc *uc);
18	void xe_uc_stop_prepare(struct xe_uc *uc);	18	void xe_uc_stop_prepare(struct xe_uc *uc);
19	int xe_uc_stop(struct xe_uc *uc);	19	void xe_uc_stop(struct xe_uc *uc);
20	int xe_uc_start(struct xe_uc *uc);	20	int xe_uc_start(struct xe_uc *uc);
21	int xe_uc_suspend(struct xe_uc *uc);	21	int xe_uc_suspend(struct xe_uc *uc);
22	int xe_uc_sanitize_reset(struct xe_uc *uc);	22	int xe_uc_sanitize_reset(struct xe_uc *uc);

drivers/gpu/drm/xe/xe_uc_fw.c+5−0

 #include "xe_force_wake.h"
 #include "xe_gsc.h"
 #include "xe_gt.h"
+#include "xe_gt_printk.h"
 #include "xe_map.h"
 #include "xe_mmio.h"
 #include "xe_module.h"
 	uc_fw_auto_select(xe, uc_fw);
 	if (IS_SRIOV_VF(xe)) {
+		/* Only GuC/HuC are supported */
+		if (uc_fw->type != XE_UC_FW_TYPE_GUC &&
+		    uc_fw->type != XE_UC_FW_TYPE_HUC)
+			uc_fw->path = NULL;
 		/* VF will support only firmwares that driver can autoselect */
 		xe_uc_fw_change_status(uc_fw, uc_fw->path ?
 				       XE_UC_FIRMWARE_PRELOADED :

drivers/gpu/drm/xe/xe_vm.c+495−350

@@ -85,8 +85,8 @@ static bool preempt_fences_waiting(struct xe_vm *vm) · 84 unmodified lines
85		85
86	list_for_each_entry(q, &vm->preempt.exec_queues, compute.link) {	86	list_for_each_entry(q, &vm->preempt.exec_queues, compute.link) {
87	if (!q->compute.pfence \|\|	87	if (!q->compute.pfence \|\|
88	(q->compute.pfence && test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,	88	test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
89	&q->compute.pfence->flags))) {	89	&q->compute.pfence->flags)) {
90	return true;	90	return true;
91	}	91	}
92	}	92	}
@@ -315,19 +315,23 @@ int __xe_vm_userptr_needs_repin(struct xe_vm *vm) · 222 unmodified lines
315		315
316	#define XE_VM_REBIND_RETRY_TIMEOUT_MS 1000	316	#define XE_VM_REBIND_RETRY_TIMEOUT_MS 1000
317		317
318	static void xe_vm_kill(struct xe_vm *vm)	318	static void xe_vm_kill(struct xe_vm *vm, bool unlocked)
319	{	319	{
320	struct xe_exec_queue *q;	320	struct xe_exec_queue *q;
321		321
322	lockdep_assert_held(&vm->lock);	322	lockdep_assert_held(&vm->lock);
323		323
324	xe_vm_lock(vm, false);	324	if (unlocked)
		325	xe_vm_lock(vm, false);
		326
325	vm->flags \|= XE_VM_FLAG_BANNED;	327	vm->flags \|= XE_VM_FLAG_BANNED;
326	trace_xe_vm_kill(vm);	328	trace_xe_vm_kill(vm);
327		329
328	list_for_each_entry(q, &vm->preempt.exec_queues, compute.link)	330	list_for_each_entry(q, &vm->preempt.exec_queues, compute.link)
329	q->ops->kill(q);	331	q->ops->kill(q);
330	xe_vm_unlock(vm);	332
		333	if (unlocked)
		334	xe_vm_unlock(vm);
331		335
332	/* TODO: Inform user the VM is banned */	336	/* TODO: Inform user the VM is banned */
333	}	337	}
@@ -557,7 +561,7 @@ out_unlock_outer: · 223 unmodified lines
557		561
558	if (err) {	562	if (err) {
559	drm_warn(&vm->xe->drm, "VM worker error: %d\n", err);	563	drm_warn(&vm->xe->drm, "VM worker error: %d\n", err);
560	xe_vm_kill(vm);	564	xe_vm_kill(vm, true);
561	}	565	}
562	up_write(&vm->lock);	566	up_write(&vm->lock);
563		567
@@ -708,37 +712,116 @@ int xe_vm_userptr_check_repin(struct xe_vm *vm) · 144 unmodified lines
708	list_empty_careful(&vm->userptr.invalidated)) ? 0 : -EAGAIN;	712	list_empty_careful(&vm->userptr.invalidated)) ? 0 : -EAGAIN;
709	}	713	}
710		714
711	static struct dma_fence *	715	static void xe_vm_populate_rebind(struct xe_vma_op op, struct xe_vma vma,
712	xe_vm_bind_vma(struct xe_vma vma, struct xe_exec_queue q,	716	u8 tile_mask)
713	struct xe_sync_entry *syncs, u32 num_syncs,	717	{
714	bool first_op, bool last_op);	718	INIT_LIST_HEAD(&op->link);
		719	op->tile_mask = tile_mask;
		720	op->base.op = DRM_GPUVA_OP_MAP;
		721	op->base.map.va.addr = vma->gpuva.va.addr;
		722	op->base.map.va.range = vma->gpuva.va.range;
		723	op->base.map.gem.obj = vma->gpuva.gem.obj;
		724	op->base.map.gem.offset = vma->gpuva.gem.offset;
		725	op->map.vma = vma;
		726	op->map.immediate = true;
		727	op->map.dumpable = vma->gpuva.flags & XE_VMA_DUMPABLE;
		728	op->map.is_null = xe_vma_is_null(vma);
		729	}
		730
		731	static int xe_vm_ops_add_rebind(struct xe_vma_ops vops, struct xe_vma vma,
		732	u8 tile_mask)
		733	{
		734	struct xe_vma_op *op;
		735
		736	op = kzalloc(sizeof(*op), GFP_KERNEL);
		737	if (!op)
		738	return -ENOMEM;
		739
		740	xe_vm_populate_rebind(op, vma, tile_mask);
		741	list_add_tail(&op->link, &vops->list);
		742
		743	return 0;
		744	}
		745
		746	static struct dma_fence ops_execute(struct xe_vm vm,
		747	struct xe_vma_ops *vops);
		748	static void xe_vma_ops_init(struct xe_vma_ops vops, struct xe_vm vm,
		749	struct xe_exec_queue *q,
		750	struct xe_sync_entry *syncs, u32 num_syncs);
715		751
716	int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)	752	int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)
717	{	753	{
718	struct dma_fence *fence;	754	struct dma_fence *fence;
719	struct xe_vma vma, next;	755	struct xe_vma vma, next;
		756	struct xe_vma_ops vops;
		757	struct xe_vma_op op, next_op;
		758	int err;
720		759
721	lockdep_assert_held(&vm->lock);	760	lockdep_assert_held(&vm->lock);
722	if (xe_vm_in_lr_mode(vm) && !rebind_worker)	761	if ((xe_vm_in_lr_mode(vm) && !rebind_worker) \|\|
		762	list_empty(&vm->rebind_list))
723	return 0;	763	return 0;
724		764
		765	xe_vma_ops_init(&vops, vm, NULL, NULL, 0);
		766
725	xe_vm_assert_held(vm);	767	xe_vm_assert_held(vm);
726	list_for_each_entry_safe(vma, next, &vm->rebind_list,	768	list_for_each_entry(vma, &vm->rebind_list, combined_links.rebind) {
727	combined_links.rebind) {
728	xe_assert(vm->xe, vma->tile_present);	769	xe_assert(vm->xe, vma->tile_present);
729		770
730	list_del_init(&vma->combined_links.rebind);
731	if (rebind_worker)	771	if (rebind_worker)
732	trace_xe_vma_rebind_worker(vma);	772	trace_xe_vma_rebind_worker(vma);
733	else	773	else
734	trace_xe_vma_rebind_exec(vma);	774	trace_xe_vma_rebind_exec(vma);
735	fence = xe_vm_bind_vma(vma, NULL, NULL, 0, false, false);	775
736	if (IS_ERR(fence))	776	err = xe_vm_ops_add_rebind(&vops, vma,
737	return PTR_ERR(fence);	777	vma->tile_present);
		778	if (err)
		779	goto free_ops;
		780	}
		781
		782	fence = ops_execute(vm, &vops);
		783	if (IS_ERR(fence)) {
		784	err = PTR_ERR(fence);
		785	} else {
738	dma_fence_put(fence);	786	dma_fence_put(fence);
		787	list_for_each_entry_safe(vma, next, &vm->rebind_list,
		788	combined_links.rebind)
		789	list_del_init(&vma->combined_links.rebind);
		790	}
		791	free_ops:
		792	list_for_each_entry_safe(op, next_op, &vops.list, link) {
		793	list_del(&op->link);
		794	kfree(op);
739	}	795	}
740		796
741	return 0;	797	return err;
		798	}
		799
		800	struct dma_fence xe_vma_rebind(struct xe_vm vm, struct xe_vma *vma, u8 tile_mask)
		801	{
		802	struct dma_fence *fence = NULL;
		803	struct xe_vma_ops vops;
		804	struct xe_vma_op op, next_op;
		805	int err;
		806
		807	lockdep_assert_held(&vm->lock);
		808	xe_vm_assert_held(vm);
		809	xe_assert(vm->xe, xe_vm_in_fault_mode(vm));
		810
		811	xe_vma_ops_init(&vops, vm, NULL, NULL, 0);
		812
		813	err = xe_vm_ops_add_rebind(&vops, vma, tile_mask);
		814	if (err)
		815	return ERR_PTR(err);
		816
		817	fence = ops_execute(vm, &vops);
		818
		819	list_for_each_entry_safe(op, next_op, &vops.list, link) {
		820	list_del(&op->link);
		821	kfree(op);
		822	}
		823
		824	return fence;
742	}	825	}
743		826
744	static void xe_vma_free(struct xe_vma *vma)	827	static void xe_vma_free(struct xe_vma *vma)
@@ -805,7 +888,7 @@ static struct xe_vma xe_vma_create(struct xe_vm vm, · 60 unmodified lines
805	for_each_tile(tile, vm->xe, id)	888	for_each_tile(tile, vm->xe, id)
806	vma->tile_mask \|= 0x1 << id;	889	vma->tile_mask \|= 0x1 << id;
807		890
808	if (GRAPHICS_VER(vm->xe) >= 20 \|\| vm->xe->info.platform == XE_PVC)	891	if (vm->xe->info.has_atomic_enable_pte_bit)
809	vma->gpuva.flags \|= XE_VMA_ATOMIC_PTE_BIT;	892	vma->gpuva.flags \|= XE_VMA_ATOMIC_PTE_BIT;
810		893
811	vma->pat_index = pat_index;	894	vma->pat_index = pat_index;
@@ -1173,6 +1256,8 @@ static const struct xe_pt_ops xelp_pt_ops = { · 361 unmodified lines
1173	.pde_encode_bo = xelp_pde_encode_bo,	1256	.pde_encode_bo = xelp_pde_encode_bo,
1174	};	1257	};
1175		1258
		1259	static void vm_destroy_work_func(struct work_struct *w);
		1260
1176	/**	1261	/**
1177	* xe_vm_create_scratch() - Setup a scratch memory pagetable tree for the	1262	* xe_vm_create_scratch() - Setup a scratch memory pagetable tree for the
1178	* given tile and vm.	1263	* given tile and vm.
@@ -1252,6 +1337,8 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 73 unmodified lines
1252	init_rwsem(&vm->userptr.notifier_lock);	1337	init_rwsem(&vm->userptr.notifier_lock);
1253	spin_lock_init(&vm->userptr.invalidated_lock);	1338	spin_lock_init(&vm->userptr.invalidated_lock);
1254		1339
		1340	INIT_WORK(&vm->destroy_work, vm_destroy_work_func);
		1341
1255	INIT_LIST_HEAD(&vm->preempt.exec_queues);	1342	INIT_LIST_HEAD(&vm->preempt.exec_queues);
1256	vm->preempt.min_run_period_ms = 10; /* FIXME: Wire up to uAPI */	1343	vm->preempt.min_run_period_ms = 10; /* FIXME: Wire up to uAPI */
1257		1344
@@ -1260,7 +1347,13 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 2 unmodified lines
1260		1347
1261	vm->pt_ops = &xelp_pt_ops;	1348	vm->pt_ops = &xelp_pt_ops;
1262		1349
1263	if (!(flags & XE_VM_FLAG_MIGRATION))	1350	/*
		1351	* Long-running workloads are not protected by the scheduler references.
		1352	* By design, run_job for long-running workloads returns NULL and the
		1353	* scheduler drops all the references of it, hence protecting the VM
		1354	* for this case is necessary.
		1355	*/
		1356	if (flags & XE_VM_FLAG_LR_MODE)
1264	xe_pm_runtime_get_noresume(xe);	1357	xe_pm_runtime_get_noresume(xe);
1265		1358
1266	vm_resv_obj = drm_gpuvm_resv_object_alloc(&xe->drm);	1359	vm_resv_obj = drm_gpuvm_resv_object_alloc(&xe->drm);
@@ -1274,7 +1367,7 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 7 unmodified lines
1274		1367
1275	drm_gem_object_put(vm_resv_obj);	1368	drm_gem_object_put(vm_resv_obj);
1276		1369
1277	err = dma_resv_lock_interruptible(xe_vm_resv(vm), NULL);	1370	err = xe_vm_lock(vm, true);
1278	if (err)	1371	if (err)
1279	goto err_close;	1372	goto err_close;
1280		1373
@@ -1318,7 +1411,7 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 37 unmodified lines
1318		1411
1319	xe_pt_populate_empty(tile, vm, vm->pt_root[id]);	1412	xe_pt_populate_empty(tile, vm, vm->pt_root[id]);
1320	}	1413	}
1321	dma_resv_unlock(xe_vm_resv(vm));	1414	xe_vm_unlock(vm);
1322		1415
1323	/* Kernel migration VM shouldn't have a circular loop.. */	1416	/* Kernel migration VM shouldn't have a circular loop.. */
1324	if (!(flags & XE_VM_FLAG_MIGRATION)) {	1417	if (!(flags & XE_VM_FLAG_MIGRATION)) {
@@ -1360,7 +1453,7 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 35 unmodified lines
1360	return vm;	1453	return vm;
1361		1454
1362	err_unlock_close:	1455	err_unlock_close:
1363	dma_resv_unlock(xe_vm_resv(vm));	1456	xe_vm_unlock(vm);
1364	err_close:	1457	err_close:
1365	xe_vm_close_and_put(vm);	1458	xe_vm_close_and_put(vm);
1366	return ERR_PTR(err);	1459	return ERR_PTR(err);
@@ -1370,7 +1463,7 @@ err_no_resv: · 3 unmodified lines
1370	for_each_tile(tile, xe, id)	1463	for_each_tile(tile, xe, id)
1371	xe_range_fence_tree_fini(&vm->rftree[id]);	1464	xe_range_fence_tree_fini(&vm->rftree[id]);
1372	kfree(vm);	1465	kfree(vm);
1373	if (!(flags & XE_VM_FLAG_MIGRATION))	1466	if (flags & XE_VM_FLAG_LR_MODE)
1374	xe_pm_runtime_put(xe);	1467	xe_pm_runtime_put(xe);
1375	return ERR_PTR(err);	1468	return ERR_PTR(err);
1376	}	1469	}
@@ -1489,9 +1582,10 @@ void xe_vm_close_and_put(struct xe_vm *vm) · 112 unmodified lines
1489	xe_vm_put(vm);	1582	xe_vm_put(vm);
1490	}	1583	}
1491		1584
1492	static void xe_vm_free(struct drm_gpuvm *gpuvm)	1585	static void vm_destroy_work_func(struct work_struct *w)
1493	{	1586	{
1494	struct xe_vm *vm = container_of(gpuvm, struct xe_vm, gpuvm);	1587	struct xe_vm *vm =
		1588	container_of(w, struct xe_vm, destroy_work);
1495	struct xe_device *xe = vm->xe;	1589	struct xe_device *xe = vm->xe;
1496	struct xe_tile *tile;	1590	struct xe_tile *tile;
1497	u8 id;	1591	u8 id;
@@ -1504,7 +1598,7 @@ static void xe_vm_free(struct drm_gpuvm *gpuvm) · 6 unmodified lines
1504		1598
1505	mutex_destroy(&vm->snap_mutex);	1599	mutex_destroy(&vm->snap_mutex);
1506		1600
1507	if (!(vm->flags & XE_VM_FLAG_MIGRATION))	1601	if (vm->flags & XE_VM_FLAG_LR_MODE)
1508	xe_pm_runtime_put(xe);	1602	xe_pm_runtime_put(xe);
1509		1603
1510	for_each_tile(tile, xe, id)	1604	for_each_tile(tile, xe, id)
@@ -1514,6 +1608,14 @@ static void xe_vm_free(struct drm_gpuvm *gpuvm) · 3 unmodified lines
1514	kfree(vm);	1608	kfree(vm);
1515	}	1609	}
1516		1610
		1611	static void xe_vm_free(struct drm_gpuvm *gpuvm)
		1612	{
		1613	struct xe_vm *vm = container_of(gpuvm, struct xe_vm, gpuvm);
		1614
		1615	/* To destroy the VM we need to be able to sleep */
		1616	queue_work(system_unbound_wq, &vm->destroy_work);
		1617	}
		1618
1517	struct xe_vm xe_vm_lookup(struct xe_file xef, u32 id)	1619	struct xe_vm xe_vm_lookup(struct xe_file xef, u32 id)
1518	{	1620	{
1519	struct xe_vm *vm;	1621	struct xe_vm *vm;
@@ -1550,23 +1652,13 @@ xe_vm_unbind_vma(struct xe_vma vma, struct xe_exec_queue q, · 30 unmodified lines
1550	struct dma_fence *fence = NULL;	1652	struct dma_fence *fence = NULL;
1551	struct dma_fence **fences = NULL;	1653	struct dma_fence **fences = NULL;
1552	struct dma_fence_array *cf = NULL;	1654	struct dma_fence_array *cf = NULL;
1553	int cur_fence = 0, i;	1655	int cur_fence = 0;
1554	int number_tiles = hweight8(vma->tile_present);	1656	int number_tiles = hweight8(vma->tile_present);
1555	int err;	1657	int err;
1556	u8 id;	1658	u8 id;
1557		1659
1558	trace_xe_vma_unbind(vma);	1660	trace_xe_vma_unbind(vma);
1559		1661
1560	if (vma->ufence) {
1561	struct xe_user_fence * const f = vma->ufence;
1562
1563	if (!xe_sync_ufence_get_status(f))
1564	return ERR_PTR(-EBUSY);
1565
1566	vma->ufence = NULL;
1567	xe_sync_ufence_put(f);
1568	}
1569
1570	if (number_tiles > 1) {	1662	if (number_tiles > 1) {
1571	fences = kmalloc_array(number_tiles, sizeof(*fences),	1663	fences = kmalloc_array(number_tiles, sizeof(*fences),
1572	GFP_KERNEL);	1664	GFP_KERNEL);
@@ -1608,10 +1700,6 @@ next: · 35 unmodified lines
1608		1700
1609	fence = cf ? &cf->base : !fence ?	1701	fence = cf ? &cf->base : !fence ?
1610	xe_exec_queue_last_fence_get(wait_exec_queue, vm) : fence;	1702	xe_exec_queue_last_fence_get(wait_exec_queue, vm) : fence;
1611	if (last_op) {
1612	for (i = 0; i < num_syncs; i++)
1613	xe_sync_entry_signal(&syncs[i], fence);
1614	}
1615		1703
1616	return fence;	1704	return fence;
1617		1705
@@ -1628,15 +1716,15 @@ err_fences: · 10 unmodified lines
1628	static struct dma_fence *	1716	static struct dma_fence *
1629	xe_vm_bind_vma(struct xe_vma vma, struct xe_exec_queue q,	1717	xe_vm_bind_vma(struct xe_vma vma, struct xe_exec_queue q,
1630	struct xe_sync_entry *syncs, u32 num_syncs,	1718	struct xe_sync_entry *syncs, u32 num_syncs,
1631	bool first_op, bool last_op)	1719	u8 tile_mask, bool first_op, bool last_op)
1632	{	1720	{
1633	struct xe_tile *tile;	1721	struct xe_tile *tile;
1634	struct dma_fence *fence;	1722	struct dma_fence *fence;
1635	struct dma_fence **fences = NULL;	1723	struct dma_fence **fences = NULL;
1636	struct dma_fence_array *cf = NULL;	1724	struct dma_fence_array *cf = NULL;
1637	struct xe_vm *vm = xe_vma_vm(vma);	1725	struct xe_vm *vm = xe_vma_vm(vma);
1638	int cur_fence = 0, i;	1726	int cur_fence = 0;
1639	int number_tiles = hweight8(vma->tile_mask);	1727	int number_tiles = hweight8(tile_mask);
1640	int err;	1728	int err;
1641	u8 id;	1729	u8 id;
1642		1730
@@ -1650,7 +1738,7 @@ xe_vm_bind_vma(struct xe_vma vma, struct xe_exec_queue q, · 7 unmodified lines
1650	}	1738	}
1651		1739
1652	for_each_tile(tile, vm->xe, id) {	1740	for_each_tile(tile, vm->xe, id) {
1653	if (!(vma->tile_mask & BIT(id)))	1741	if (!(tile_mask & BIT(id)))
1654	goto next;	1742	goto next;
1655		1743
1656	fence = __xe_pt_bind_vma(tile, vma, q ? q : vm->q[id],	1744	fence = __xe_pt_bind_vma(tile, vma, q ? q : vm->q[id],
@@ -1682,12 +1770,6 @@ next: · 25 unmodified lines
1682	}	1770	}
1683	}	1771	}
1684		1772
1685	if (last_op) {
1686	for (i = 0; i < num_syncs; i++)
1687	xe_sync_entry_signal(&syncs[i],
1688	cf ? &cf->base : fence);
1689	}
1690
1691	return cf ? &cf->base : fence;	1773	return cf ? &cf->base : fence;
1692		1774
1693	err_fences:	1775	err_fences:
@@ -1715,87 +1797,46 @@ find_ufence_get(struct xe_sync_entry *syncs, u32 num_syncs) · 21 unmodified lines
1715	return NULL;	1797	return NULL;
1716	}	1798	}
1717		1799
1718	static int __xe_vm_bind(struct xe_vm vm, struct xe_vma vma,	1800	static struct dma_fence *
1719	struct xe_exec_queue q, struct xe_sync_entry syncs,	1801	xe_vm_bind(struct xe_vm vm, struct xe_vma vma, struct xe_exec_queue *q,
1720	u32 num_syncs, bool immediate, bool first_op,	1802	struct xe_bo bo, struct xe_sync_entry syncs, u32 num_syncs,
1721	bool last_op)	1803	u8 tile_mask, bool immediate, bool first_op, bool last_op)
1722	{	1804	{
1723	struct dma_fence *fence;	1805	struct dma_fence *fence;
1724	struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);	1806	struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
1725	struct xe_user_fence *ufence;
1726		1807
1727	xe_vm_assert_held(vm);	1808	xe_vm_assert_held(vm);
1728		1809	xe_bo_assert_held(bo);
1729	ufence = find_ufence_get(syncs, num_syncs);
1730	if (vma->ufence && ufence)
1731	xe_sync_ufence_put(vma->ufence);
1732
1733	vma->ufence = ufence ?: vma->ufence;
1734		1810
1735	if (immediate) {	1811	if (immediate) {
1736	fence = xe_vm_bind_vma(vma, q, syncs, num_syncs, first_op,	1812	fence = xe_vm_bind_vma(vma, q, syncs, num_syncs, tile_mask,
1737	last_op);	1813	first_op, last_op);
1738	if (IS_ERR(fence))	1814	if (IS_ERR(fence))
1739	return PTR_ERR(fence);	1815	return fence;
1740	} else {	1816	} else {
1741	int i;
1742
1743	xe_assert(vm->xe, xe_vm_in_fault_mode(vm));	1817	xe_assert(vm->xe, xe_vm_in_fault_mode(vm));
1744		1818
1745	fence = xe_exec_queue_last_fence_get(wait_exec_queue, vm);	1819	fence = xe_exec_queue_last_fence_get(wait_exec_queue, vm);
1746	if (last_op) {
1747	for (i = 0; i < num_syncs; i++)
1748	xe_sync_entry_signal(&syncs[i], fence);
1749	}
1750	}	1820	}
1751		1821
1752	if (last_op)	1822	return fence;
1753	xe_exec_queue_last_fence_set(wait_exec_queue, vm, fence);
1754	dma_fence_put(fence);
1755
1756	return 0;
1757	}
1758
1759	static int xe_vm_bind(struct xe_vm vm, struct xe_vma vma, struct xe_exec_queue *q,
1760	struct xe_bo bo, struct xe_sync_entry syncs,
1761	u32 num_syncs, bool immediate, bool first_op,
1762	bool last_op)
1763	{
1764	int err;
1765
1766	xe_vm_assert_held(vm);
1767	xe_bo_assert_held(bo);
1768
1769	if (bo && immediate) {
1770	err = xe_bo_validate(bo, vm, true);
1771	if (err)
1772	return err;
1773	}
1774
1775	return __xe_vm_bind(vm, vma, q, syncs, num_syncs, immediate, first_op,
1776	last_op);
1777	}	1823	}
1778		1824
1779	static int xe_vm_unbind(struct xe_vm vm, struct xe_vma vma,	1825	static struct dma_fence *
1780	struct xe_exec_queue q, struct xe_sync_entry syncs,	1826	xe_vm_unbind(struct xe_vm vm, struct xe_vma vma,
1781	u32 num_syncs, bool first_op, bool last_op)	1827	struct xe_exec_queue q, struct xe_sync_entry syncs,
		1828	u32 num_syncs, bool first_op, bool last_op)
1782	{	1829	{
1783	struct dma_fence *fence;	1830	struct dma_fence *fence;
1784	struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
1785		1831
1786	xe_vm_assert_held(vm);	1832	xe_vm_assert_held(vm);
1787	xe_bo_assert_held(xe_vma_bo(vma));	1833	xe_bo_assert_held(xe_vma_bo(vma));
1788		1834
1789	fence = xe_vm_unbind_vma(vma, q, syncs, num_syncs, first_op, last_op);	1835	fence = xe_vm_unbind_vma(vma, q, syncs, num_syncs, first_op, last_op);
1790	if (IS_ERR(fence))	1836	if (IS_ERR(fence))
1791	return PTR_ERR(fence);	1837	return fence;
1792		1838
1793	xe_vma_destroy(vma, fence);	1839	return fence;
1794	if (last_op)
1795	xe_exec_queue_last_fence_set(wait_exec_queue, vm, fence);
1796	dma_fence_put(fence);
1797
1798	return 0;
1799	}	1840	}
1800		1841
1801	#define ALL_DRM_XE_VM_CREATE_FLAGS (DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE \| \	1842	#define ALL_DRM_XE_VM_CREATE_FLAGS (DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE \| \
@@ -1938,40 +1979,18 @@ static const u32 region_to_mem_type[] = { · 136 unmodified lines
1938	XE_PL_VRAM1,	1979	XE_PL_VRAM1,
1939	};	1980	};
1940		1981
1941	static int xe_vm_prefetch(struct xe_vm vm, struct xe_vma vma,	1982	static struct dma_fence *
1942	struct xe_exec_queue *q, u32 region,	1983	xe_vm_prefetch(struct xe_vm vm, struct xe_vma vma,
1943	struct xe_sync_entry *syncs, u32 num_syncs,	1984	struct xe_exec_queue q, struct xe_sync_entry syncs,
1944	bool first_op, bool last_op)	1985	u32 num_syncs, bool first_op, bool last_op)
1945	{	1986	{
1946	struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);	1987	struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
1947	int err;
1948
1949	xe_assert(vm->xe, region < ARRAY_SIZE(region_to_mem_type));
1950
1951	if (!xe_vma_has_no_bo(vma)) {
1952	err = xe_bo_migrate(xe_vma_bo(vma), region_to_mem_type[region]);
1953	if (err)
1954	return err;
1955	}
1956		1988
1957	if (vma->tile_mask != (vma->tile_present & ~vma->tile_invalidated)) {	1989	if (vma->tile_mask != (vma->tile_present & ~vma->tile_invalidated)) {
1958	return xe_vm_bind(vm, vma, q, xe_vma_bo(vma), syncs, num_syncs,	1990	return xe_vm_bind(vm, vma, q, xe_vma_bo(vma), syncs, num_syncs,
1959	true, first_op, last_op);	1991	vma->tile_mask, true, first_op, last_op);
1960	} else {	1992	} else {
1961	int i;	1993	return xe_exec_queue_last_fence_get(wait_exec_queue, vm);
1962
1963	/* Nothing to do, signal fences now */
1964	if (last_op) {
1965	for (i = 0; i < num_syncs; i++) {
1966	struct dma_fence *fence =
1967	xe_exec_queue_last_fence_get(wait_exec_queue, vm);
1968
1969	xe_sync_entry_signal(&syncs[i], fence);
1970	dma_fence_put(fence);
1971	}
1972	}
1973
1974	return 0;
1975	}	1994	}
1976	}	1995	}
1977		1996
@@ -2267,23 +2286,28 @@ static int xe_vma_op_commit(struct xe_vm vm, struct xe_vma_op op) · 289 unmodified lines
2267	static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q,	2286	static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q,
2268	struct drm_gpuva_ops *ops,	2287	struct drm_gpuva_ops *ops,
2269	struct xe_sync_entry *syncs, u32 num_syncs,	2288	struct xe_sync_entry *syncs, u32 num_syncs,
2270	struct list_head *ops_list, bool last)	2289	struct xe_vma_ops *vops, bool last)
2271	{	2290	{
2272	struct xe_device *xe = vm->xe;	2291	struct xe_device *xe = vm->xe;
2273	struct xe_vma_op *last_op = NULL;	2292	struct xe_vma_op *last_op = NULL;
2274	struct drm_gpuva_op *__op;	2293	struct drm_gpuva_op *__op;
		2294	struct xe_tile *tile;
		2295	u8 id, tile_mask = 0;
2275	int err = 0;	2296	int err = 0;
2276		2297
2277	lockdep_assert_held_write(&vm->lock);	2298	lockdep_assert_held_write(&vm->lock);
2278		2299
		2300	for_each_tile(tile, vm->xe, id)
		2301	tile_mask \|= 0x1 << id;
		2302
2279	drm_gpuva_for_each_op(__op, ops) {	2303	drm_gpuva_for_each_op(__op, ops) {
2280	struct xe_vma_op *op = gpuva_op_to_vma_op(__op);	2304	struct xe_vma_op *op = gpuva_op_to_vma_op(__op);
2281	struct xe_vma *vma;	2305	struct xe_vma *vma;
2282	bool first = list_empty(ops_list);	2306	bool first = list_empty(&vops->list);
2283	unsigned int flags = 0;	2307	unsigned int flags = 0;
2284		2308
2285	INIT_LIST_HEAD(&op->link);	2309	INIT_LIST_HEAD(&op->link);
2286	list_add_tail(&op->link, ops_list);	2310	list_add_tail(&op->link, &vops->list);
2287		2311
2288	if (first) {	2312	if (first) {
2289	op->flags \|= XE_VMA_OP_FIRST;	2313	op->flags \|= XE_VMA_OP_FIRST;
@@ -2292,6 +2316,7 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q, · 2 unmodified lines
2292	}	2316	}
2293		2317
2294	op->q = q;	2318	op->q = q;
		2319	op->tile_mask = tile_mask;
2295		2320
2296	switch (op->base.op) {	2321	switch (op->base.op) {
2297	case DRM_GPUVA_OP_MAP:	2322	case DRM_GPUVA_OP_MAP:
@@ -2409,12 +2434,11 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q, · 111 unmodified lines
2409	}	2434	}
2410		2435
2411	/* FIXME: Unhandled corner case */	2436	/* FIXME: Unhandled corner case */
2412	XE_WARN_ON(!last_op && last && !list_empty(ops_list));	2437	XE_WARN_ON(!last_op && last && !list_empty(&vops->list));
2413		2438
2414	if (!last_op)	2439	if (!last_op)
2415	return 0;	2440	return 0;
2416		2441
2417	last_op->ops = ops;
2418	if (last) {	2442	if (last) {
2419	last_op->flags \|= XE_VMA_OP_LAST;	2443	last_op->flags \|= XE_VMA_OP_LAST;
2420	last_op->num_syncs = num_syncs;	2444	last_op->num_syncs = num_syncs;
@@ -2424,27 +2448,24 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q, · 3 unmodified lines
2424	return 0;	2448	return 0;
2425	}	2449	}
2426		2450
2427	static int op_execute(struct drm_exec exec, struct xe_vm vm,	2451	static struct dma_fence op_execute(struct xe_vm vm, struct xe_vma *vma,
2428	struct xe_vma vma, struct xe_vma_op op)	2452	struct xe_vma_op *op)
2429	{	2453	{
2430	int err;	2454	struct dma_fence *fence = NULL;
2431
2432	lockdep_assert_held_write(&vm->lock);
2433		2455
2434	err = xe_vm_lock_vma(exec, vma);	2456	lockdep_assert_held(&vm->lock);
2435	if (err)
2436	return err;
2437		2457
2438	xe_vm_assert_held(vm);	2458	xe_vm_assert_held(vm);
2439	xe_bo_assert_held(xe_vma_bo(vma));	2459	xe_bo_assert_held(xe_vma_bo(vma));
2440		2460
2441	switch (op->base.op) {	2461	switch (op->base.op) {
2442	case DRM_GPUVA_OP_MAP:	2462	case DRM_GPUVA_OP_MAP:
2443	err = xe_vm_bind(vm, vma, op->q, xe_vma_bo(vma),	2463	fence = xe_vm_bind(vm, vma, op->q, xe_vma_bo(vma),
2444	op->syncs, op->num_syncs,	2464	op->syncs, op->num_syncs,
2445	op->map.immediate \|\| !xe_vm_in_fault_mode(vm),	2465	op->tile_mask,
2446	op->flags & XE_VMA_OP_FIRST,	2466	op->map.immediate \|\| !xe_vm_in_fault_mode(vm),
2447	op->flags & XE_VMA_OP_LAST);	2467	op->flags & XE_VMA_OP_FIRST,
		2468	op->flags & XE_VMA_OP_LAST);
2448	break;	2469	break;
2449	case DRM_GPUVA_OP_REMAP:	2470	case DRM_GPUVA_OP_REMAP:
2450	{	2471	{
@@ -2454,37 +2475,41 @@ static int op_execute(struct drm_exec exec, struct xe_vm vm, · 3 unmodified lines
2454	if (!op->remap.unmap_done) {	2475	if (!op->remap.unmap_done) {
2455	if (prev \|\| next)	2476	if (prev \|\| next)
2456	vma->gpuva.flags \|= XE_VMA_FIRST_REBIND;	2477	vma->gpuva.flags \|= XE_VMA_FIRST_REBIND;
2457	err = xe_vm_unbind(vm, vma, op->q, op->syncs,	2478	fence = xe_vm_unbind(vm, vma, op->q, op->syncs,
2458	op->num_syncs,	2479	op->num_syncs,
2459	op->flags & XE_VMA_OP_FIRST,	2480	op->flags & XE_VMA_OP_FIRST,
2460	op->flags & XE_VMA_OP_LAST &&	2481	op->flags & XE_VMA_OP_LAST &&
2461	!prev && !next);	2482	!prev && !next);
2462	if (err)	2483	if (IS_ERR(fence))
2463	break;	2484	break;
2464	op->remap.unmap_done = true;	2485	op->remap.unmap_done = true;
2465	}	2486	}
2466		2487
2467	if (prev) {	2488	if (prev) {
2468	op->remap.prev->gpuva.flags \|= XE_VMA_LAST_REBIND;	2489	op->remap.prev->gpuva.flags \|= XE_VMA_LAST_REBIND;
2469	err = xe_vm_bind(vm, op->remap.prev, op->q,	2490	dma_fence_put(fence);
2470	xe_vma_bo(op->remap.prev), op->syncs,	2491	fence = xe_vm_bind(vm, op->remap.prev, op->q,
2471	op->num_syncs, true, false,	2492	xe_vma_bo(op->remap.prev), op->syncs,
2472	op->flags & XE_VMA_OP_LAST && !next);	2493	op->num_syncs,
		2494	op->remap.prev->tile_mask, true,
		2495	false,
		2496	op->flags & XE_VMA_OP_LAST && !next);
2473	op->remap.prev->gpuva.flags &= ~XE_VMA_LAST_REBIND;	2497	op->remap.prev->gpuva.flags &= ~XE_VMA_LAST_REBIND;
2474	if (err)	2498	if (IS_ERR(fence))
2475	break;	2499	break;
2476	op->remap.prev = NULL;	2500	op->remap.prev = NULL;
2477	}	2501	}
2478		2502
2479	if (next) {	2503	if (next) {
2480	op->remap.next->gpuva.flags \|= XE_VMA_LAST_REBIND;	2504	op->remap.next->gpuva.flags \|= XE_VMA_LAST_REBIND;
2481	err = xe_vm_bind(vm, op->remap.next, op->q,	2505	dma_fence_put(fence);
2482	xe_vma_bo(op->remap.next),	2506	fence = xe_vm_bind(vm, op->remap.next, op->q,
2483	op->syncs, op->num_syncs,	2507	xe_vma_bo(op->remap.next),
2484	true, false,	2508	op->syncs, op->num_syncs,
2485	op->flags & XE_VMA_OP_LAST);	2509	op->remap.next->tile_mask, true,
		2510	false, op->flags & XE_VMA_OP_LAST);
2486	op->remap.next->gpuva.flags &= ~XE_VMA_LAST_REBIND;	2511	op->remap.next->gpuva.flags &= ~XE_VMA_LAST_REBIND;
2487	if (err)	2512	if (IS_ERR(fence))
2488	break;	2513	break;
2489	op->remap.next = NULL;	2514	op->remap.next = NULL;
2490	}	2515	}
@@ -2492,43 +2517,35 @@ static int op_execute(struct drm_exec exec, struct xe_vm vm, · 1 unmodified line
2492	break;	2517	break;
2493	}	2518	}
2494	case DRM_GPUVA_OP_UNMAP:	2519	case DRM_GPUVA_OP_UNMAP:
2495	err = xe_vm_unbind(vm, vma, op->q, op->syncs,	2520	fence = xe_vm_unbind(vm, vma, op->q, op->syncs,
2496	op->num_syncs, op->flags & XE_VMA_OP_FIRST,	2521	op->num_syncs, op->flags & XE_VMA_OP_FIRST,
2497	op->flags & XE_VMA_OP_LAST);	2522	op->flags & XE_VMA_OP_LAST);
2498	break;	2523	break;
2499	case DRM_GPUVA_OP_PREFETCH:	2524	case DRM_GPUVA_OP_PREFETCH:
2500	err = xe_vm_prefetch(vm, vma, op->q, op->prefetch.region,	2525	fence = xe_vm_prefetch(vm, vma, op->q, op->syncs, op->num_syncs,
2501	op->syncs, op->num_syncs,	2526	op->flags & XE_VMA_OP_FIRST,
2502	op->flags & XE_VMA_OP_FIRST,	2527	op->flags & XE_VMA_OP_LAST);
2503	op->flags & XE_VMA_OP_LAST);
2504	break;	2528	break;
2505	default:	2529	default:
2506	drm_warn(&vm->xe->drm, "NOT POSSIBLE");	2530	drm_warn(&vm->xe->drm, "NOT POSSIBLE");
2507	}	2531	}
2508		2532
2509	if (err)	2533	if (IS_ERR(fence))
2510	trace_xe_vma_fail(vma);	2534	trace_xe_vma_fail(vma);
2511		2535
2512	return err;	2536	return fence;
2513	}	2537	}
2514		2538
2515	static int __xe_vma_op_execute(struct xe_vm vm, struct xe_vma vma,	2539	static struct dma_fence *
2516	struct xe_vma_op *op)	2540	__xe_vma_op_execute(struct xe_vm vm, struct xe_vma vma,
		2541	struct xe_vma_op *op)
2517	{	2542	{
2518	struct drm_exec exec;	2543	struct dma_fence *fence;
2519	int err;	2544	int err;
2520		2545
2521	retry_userptr:	2546	retry_userptr:
2522	drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);	2547	fence = op_execute(vm, vma, op);
2523	drm_exec_until_all_locked(&exec) {	2548	if (IS_ERR(fence) && PTR_ERR(fence) == -EAGAIN) {
2524	err = op_execute(&exec, vm, vma, op);
2525	drm_exec_retry_on_contention(&exec);
2526	if (err)
2527	break;
2528	}
2529	drm_exec_fini(&exec);
2530
2531	if (err == -EAGAIN) {
2532	lockdep_assert_held_write(&vm->lock);	2549	lockdep_assert_held_write(&vm->lock);
2533		2550
2534	if (op->base.op == DRM_GPUVA_OP_REMAP) {	2551	if (op->base.op == DRM_GPUVA_OP_REMAP) {
@@ -2545,22 +2562,24 @@ retry_userptr: · 10 unmodified lines
2545	if (!err)	2562	if (!err)
2546	goto retry_userptr;	2563	goto retry_userptr;
2547		2564
		2565	fence = ERR_PTR(err);
2548	trace_xe_vma_fail(vma);	2566	trace_xe_vma_fail(vma);
2549	}	2567	}
2550	}	2568	}
2551		2569
2552	return err;	2570	return fence;
2553	}	2571	}
2554		2572
2555	static int xe_vma_op_execute(struct xe_vm vm, struct xe_vma_op op)	2573	static struct dma_fence *
		2574	xe_vma_op_execute(struct xe_vm vm, struct xe_vma_op op)
2556	{	2575	{
2557	int ret = 0;	2576	struct dma_fence *fence = ERR_PTR(-ENOMEM);
2558		2577
2559	lockdep_assert_held_write(&vm->lock);	2578	lockdep_assert_held(&vm->lock);
2560		2579
2561	switch (op->base.op) {	2580	switch (op->base.op) {
2562	case DRM_GPUVA_OP_MAP:	2581	case DRM_GPUVA_OP_MAP:
2563	ret = __xe_vma_op_execute(vm, op->map.vma, op);	2582	fence = __xe_vma_op_execute(vm, op->map.vma, op);
2564	break;	2583	break;
2565	case DRM_GPUVA_OP_REMAP:	2584	case DRM_GPUVA_OP_REMAP:
2566	{	2585	{
@@ -2573,42 +2592,23 @@ static int xe_vma_op_execute(struct xe_vm vm, struct xe_vma_op op) · 6 unmodified lines
2573	else	2592	else
2574	vma = op->remap.next;	2593	vma = op->remap.next;
2575		2594
2576	ret = __xe_vma_op_execute(vm, vma, op);	2595	fence = __xe_vma_op_execute(vm, vma, op);
2577	break;	2596	break;
2578	}	2597	}
2579	case DRM_GPUVA_OP_UNMAP:	2598	case DRM_GPUVA_OP_UNMAP:
2580	ret = __xe_vma_op_execute(vm, gpuva_to_vma(op->base.unmap.va),	2599	fence = __xe_vma_op_execute(vm, gpuva_to_vma(op->base.unmap.va),
2581	op);	2600	op);
2582	break;	2601	break;
2583	case DRM_GPUVA_OP_PREFETCH:	2602	case DRM_GPUVA_OP_PREFETCH:
2584	ret = __xe_vma_op_execute(vm,	2603	fence = __xe_vma_op_execute(vm,
2585	gpuva_to_vma(op->base.prefetch.va),	2604	gpuva_to_vma(op->base.prefetch.va),
2586	op);	2605	op);
2587	break;	2606	break;
2588	default:	2607	default:
2589	drm_warn(&vm->xe->drm, "NOT POSSIBLE");	2608	drm_warn(&vm->xe->drm, "NOT POSSIBLE");
2590	}	2609	}
2591		2610
2592	return ret;	2611	return fence;
2593	}
2594
2595	static void xe_vma_op_cleanup(struct xe_vm vm, struct xe_vma_op op)
2596	{
2597	bool last = op->flags & XE_VMA_OP_LAST;
2598
2599	if (last) {
2600	while (op->num_syncs--)
2601	xe_sync_entry_cleanup(&op->syncs[op->num_syncs]);
2602	kfree(op->syncs);
2603	if (op->q)
2604	xe_exec_queue_put(op->q);
2605	}
2606	if (!list_empty(&op->link))
2607	list_del(&op->link);
2608	if (op->ops)
2609	drm_gpuva_ops_free(&vm->gpuvm, op->ops);
2610	if (last)
2611	xe_vm_put(vm);
2612	}	2612	}
2613		2613
2614	static void xe_vma_op_unwind(struct xe_vm vm, struct xe_vma_op op,	2614	static void xe_vma_op_unwind(struct xe_vm vm, struct xe_vma_op op,
@@ -2687,34 +2687,223 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, · 72 unmodified lines
2687	op->flags & XE_VMA_OP_PREV_COMMITTED,	2687	op->flags & XE_VMA_OP_PREV_COMMITTED,
2688	op->flags & XE_VMA_OP_NEXT_COMMITTED);	2688	op->flags & XE_VMA_OP_NEXT_COMMITTED);
2689	}	2689	}
		2690	}
		2691	}
2690		2692
2691	drm_gpuva_ops_free(&vm->gpuvm, __ops);	2693	static int vma_lock_and_validate(struct drm_exec exec, struct xe_vma vma,
		2694	bool validate)
		2695	{
		2696	struct xe_bo *bo = xe_vma_bo(vma);
		2697	int err = 0;
		2698
		2699	if (bo) {
		2700	if (!bo->vm)
		2701	err = drm_exec_lock_obj(exec, &bo->ttm.base);
		2702	if (!err && validate)
		2703	err = xe_bo_validate(bo, xe_vma_vm(vma), true);
2692	}	2704	}
		2705
		2706	return err;
2693	}	2707	}
2694		2708
2695	static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,	2709	static int check_ufence(struct xe_vma *vma)
2696	struct list_head *ops_list)	2710	{
		2711	if (vma->ufence) {
		2712	struct xe_user_fence * const f = vma->ufence;
		2713
		2714	if (!xe_sync_ufence_get_status(f))
		2715	return -EBUSY;
		2716
		2717	vma->ufence = NULL;
		2718	xe_sync_ufence_put(f);
		2719	}
		2720
		2721	return 0;
		2722	}
		2723
		2724	static int op_lock_and_prep(struct drm_exec exec, struct xe_vm vm,
		2725	struct xe_vma_op *op)
		2726	{
		2727	int err = 0;
		2728
		2729	switch (op->base.op) {
		2730	case DRM_GPUVA_OP_MAP:
		2731	err = vma_lock_and_validate(exec, op->map.vma,
		2732	!xe_vm_in_fault_mode(vm) \|\|
		2733	op->map.immediate);
		2734	break;
		2735	case DRM_GPUVA_OP_REMAP:
		2736	err = check_ufence(gpuva_to_vma(op->base.remap.unmap->va));
		2737	if (err)
		2738	break;
		2739
		2740	err = vma_lock_and_validate(exec,
		2741	gpuva_to_vma(op->base.remap.unmap->va),
		2742	false);
		2743	if (!err && op->remap.prev)
		2744	err = vma_lock_and_validate(exec, op->remap.prev, true);
		2745	if (!err && op->remap.next)
		2746	err = vma_lock_and_validate(exec, op->remap.next, true);
		2747	break;
		2748	case DRM_GPUVA_OP_UNMAP:
		2749	err = check_ufence(gpuva_to_vma(op->base.unmap.va));
		2750	if (err)
		2751	break;
		2752
		2753	err = vma_lock_and_validate(exec,
		2754	gpuva_to_vma(op->base.unmap.va),
		2755	false);
		2756	break;
		2757	case DRM_GPUVA_OP_PREFETCH:
		2758	{
		2759	struct xe_vma *vma = gpuva_to_vma(op->base.prefetch.va);
		2760	u32 region = op->prefetch.region;
		2761
		2762	xe_assert(vm->xe, region <= ARRAY_SIZE(region_to_mem_type));
		2763
		2764	err = vma_lock_and_validate(exec,
		2765	gpuva_to_vma(op->base.prefetch.va),
		2766	false);
		2767	if (!err && !xe_vma_has_no_bo(vma))
		2768	err = xe_bo_migrate(xe_vma_bo(vma),
		2769	region_to_mem_type[region]);
		2770	break;
		2771	}
		2772	default:
		2773	drm_warn(&vm->xe->drm, "NOT POSSIBLE");
		2774	}
		2775
		2776	return err;
		2777	}
		2778
		2779	static int vm_bind_ioctl_ops_lock_and_prep(struct drm_exec *exec,
		2780	struct xe_vm *vm,
		2781	struct xe_vma_ops *vops)
		2782	{
		2783	struct xe_vma_op *op;
		2784	int err;
		2785
		2786	err = drm_exec_lock_obj(exec, xe_vm_obj(vm));
		2787	if (err)
		2788	return err;
		2789
		2790	list_for_each_entry(op, &vops->list, link) {
		2791	err = op_lock_and_prep(exec, vm, op);
		2792	if (err)
		2793	return err;
		2794	}
		2795
		2796	return 0;
		2797	}
		2798
		2799	static struct dma_fence ops_execute(struct xe_vm vm,
		2800	struct xe_vma_ops *vops)
2697	{	2801	{
2698	struct xe_vma_op op, next;	2802	struct xe_vma_op op, next;
		2803	struct dma_fence *fence = NULL;
		2804
		2805	list_for_each_entry_safe(op, next, &vops->list, link) {
		2806	dma_fence_put(fence);
		2807	fence = xe_vma_op_execute(vm, op);
		2808	if (IS_ERR(fence)) {
		2809	drm_warn(&vm->xe->drm, "VM op(%d) failed with %ld",
		2810	op->base.op, PTR_ERR(fence));
		2811	fence = ERR_PTR(-ENOSPC);
		2812	break;
		2813	}
		2814	}
		2815
		2816	return fence;
		2817	}
		2818
		2819	static void vma_add_ufence(struct xe_vma vma, struct xe_user_fence ufence)
		2820	{
		2821	if (vma->ufence)
		2822	xe_sync_ufence_put(vma->ufence);
		2823	vma->ufence = __xe_sync_ufence_get(ufence);
		2824	}
		2825
		2826	static void op_add_ufence(struct xe_vm vm, struct xe_vma_op op,
		2827	struct xe_user_fence *ufence)
		2828	{
		2829	switch (op->base.op) {
		2830	case DRM_GPUVA_OP_MAP:
		2831	vma_add_ufence(op->map.vma, ufence);
		2832	break;
		2833	case DRM_GPUVA_OP_REMAP:
		2834	if (op->remap.prev)
		2835	vma_add_ufence(op->remap.prev, ufence);
		2836	if (op->remap.next)
		2837	vma_add_ufence(op->remap.next, ufence);
		2838	break;
		2839	case DRM_GPUVA_OP_UNMAP:
		2840	break;
		2841	case DRM_GPUVA_OP_PREFETCH:
		2842	vma_add_ufence(gpuva_to_vma(op->base.prefetch.va), ufence);
		2843	break;
		2844	default:
		2845	drm_warn(&vm->xe->drm, "NOT POSSIBLE");
		2846	}
		2847	}
		2848
		2849	static void vm_bind_ioctl_ops_fini(struct xe_vm vm, struct xe_vma_ops vops,
		2850	struct dma_fence *fence)
		2851	{
		2852	struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, vops->q);
		2853	struct xe_user_fence *ufence;
		2854	struct xe_vma_op *op;
		2855	int i;
		2856
		2857	ufence = find_ufence_get(vops->syncs, vops->num_syncs);
		2858	list_for_each_entry(op, &vops->list, link) {
		2859	if (ufence)
		2860	op_add_ufence(vm, op, ufence);
		2861
		2862	if (op->base.op == DRM_GPUVA_OP_UNMAP)
		2863	xe_vma_destroy(gpuva_to_vma(op->base.unmap.va), fence);
		2864	else if (op->base.op == DRM_GPUVA_OP_REMAP)
		2865	xe_vma_destroy(gpuva_to_vma(op->base.remap.unmap->va),
		2866	fence);
		2867	}
		2868	if (ufence)
		2869	xe_sync_ufence_put(ufence);
		2870	for (i = 0; i < vops->num_syncs; i++)
		2871	xe_sync_entry_signal(vops->syncs + i, fence);
		2872	xe_exec_queue_last_fence_set(wait_exec_queue, vm, fence);
		2873	dma_fence_put(fence);
		2874	}
		2875
		2876	static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,
		2877	struct xe_vma_ops *vops)
		2878	{
		2879	struct drm_exec exec;
		2880	struct dma_fence *fence;
2699	int err;	2881	int err;
2700		2882
2701	lockdep_assert_held_write(&vm->lock);	2883	lockdep_assert_held_write(&vm->lock);
2702		2884
2703	list_for_each_entry_safe(op, next, ops_list, link) {	2885	drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT \|
2704	err = xe_vma_op_execute(vm, op);	2886	DRM_EXEC_IGNORE_DUPLICATES, 0);
2705	if (err) {	2887	drm_exec_until_all_locked(&exec) {
2706	drm_warn(&vm->xe->drm, "VM op(%d) failed with %d",	2888	err = vm_bind_ioctl_ops_lock_and_prep(&exec, vm, vops);
2707	op->base.op, err);	2889	drm_exec_retry_on_contention(&exec);
2708	/*	2890	if (err)
2709	* FIXME: Killing VM rather than proper error handling	2891	goto unlock;
2710	*/	2892
2711	xe_vm_kill(vm);	2893	fence = ops_execute(vm, vops);
2712	return -ENOSPC;	2894	if (IS_ERR(fence)) {
		2895	err = PTR_ERR(fence);
		2896	/* FIXME: Killing VM rather than proper error handling */
		2897	xe_vm_kill(vm, false);
		2898	goto unlock;
		2899	} else {
		2900	vm_bind_ioctl_ops_fini(vm, vops, fence);
2713	}	2901	}
2714	xe_vma_op_cleanup(vm, op);
2715	}	2902	}
2716		2903
2717	return 0;	2904	unlock:
		2905	drm_exec_fini(&exec);
		2906	return err;
2718	}	2907	}
2719		2908
2720	#define SUPPORTED_FLAGS \	2909	#define SUPPORTED_FLAGS \
@@ -2862,6 +3051,58 @@ static int vm_bind_ioctl_signal_fences(struct xe_vm *vm, · 141 unmodified lines
2862	return err;	3051	return err;
2863	}	3052	}
2864		3053
		3054	static void xe_vma_ops_init(struct xe_vma_ops vops, struct xe_vm vm,
		3055	struct xe_exec_queue *q,
		3056	struct xe_sync_entry *syncs, u32 num_syncs)
		3057	{
		3058	memset(vops, 0, sizeof(*vops));
		3059	INIT_LIST_HEAD(&vops->list);
		3060	vops->vm = vm;
		3061	vops->q = q;
		3062	vops->syncs = syncs;
		3063	vops->num_syncs = num_syncs;
		3064	}
		3065
		3066	static int xe_vm_bind_ioctl_validate_bo(struct xe_device xe, struct xe_bo bo,
		3067	u64 addr, u64 range, u64 obj_offset,
		3068	u16 pat_index)
		3069	{
		3070	u16 coh_mode;
		3071
		3072	if (XE_IOCTL_DBG(xe, range > bo->size) \|\|
		3073	XE_IOCTL_DBG(xe, obj_offset >
		3074	bo->size - range)) {
		3075	return -EINVAL;
		3076	}
		3077
		3078	if (bo->flags & XE_BO_FLAG_INTERNAL_64K) {
		3079	if (XE_IOCTL_DBG(xe, obj_offset &
		3080	XE_64K_PAGE_MASK) \|\|
		3081	XE_IOCTL_DBG(xe, addr & XE_64K_PAGE_MASK) \|\|
		3082	XE_IOCTL_DBG(xe, range & XE_64K_PAGE_MASK)) {
		3083	return -EINVAL;
		3084	}
		3085	}
		3086
		3087	coh_mode = xe_pat_index_get_coh_mode(xe, pat_index);
		3088	if (bo->cpu_caching) {
		3089	if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE &&
		3090	bo->cpu_caching == DRM_XE_GEM_CPU_CACHING_WB)) {
		3091	return -EINVAL;
		3092	}
		3093	} else if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE)) {
		3094	/*
		3095	* Imported dma-buf from a different device should
		3096	* require 1way or 2way coherency since we don't know
		3097	* how it was mapped on the CPU. Just assume is it
		3098	* potentially cached on CPU side.
		3099	*/
		3100	return -EINVAL;
		3101	}
		3102
		3103	return 0;
		3104	}
		3105
2865	int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file)	3106	int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file)
2866	{	3107	{
2867	struct xe_device *xe = to_xe_device(dev);	3108	struct xe_device *xe = to_xe_device(dev);
@@ -2875,7 +3116,7 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 7 unmodified lines
2875	u32 num_syncs, num_ufence = 0;	3116	u32 num_syncs, num_ufence = 0;
2876	struct xe_sync_entry *syncs = NULL;	3117	struct xe_sync_entry *syncs = NULL;
2877	struct drm_xe_vm_bind_op *bind_ops;	3118	struct drm_xe_vm_bind_op *bind_ops;
2878	LIST_HEAD(ops_list);	3119	struct xe_vma_ops vops;
2879	int err;	3120	int err;
2880	int i;	3121	int i;
2881		3122
@@ -2945,7 +3186,6 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 63 unmodified lines
2945	u32 obj = bind_ops[i].obj;	3186	u32 obj = bind_ops[i].obj;
2946	u64 obj_offset = bind_ops[i].obj_offset;	3187	u64 obj_offset = bind_ops[i].obj_offset;
2947	u16 pat_index = bind_ops[i].pat_index;	3188	u16 pat_index = bind_ops[i].pat_index;
2948	u16 coh_mode;
2949		3189
2950	if (!obj)	3190	if (!obj)
2951	continue;	3191	continue;
@@ -2957,40 +3197,10 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 5 unmodified lines
2957	}	3197	}
2958	bos[i] = gem_to_xe_bo(gem_obj);	3198	bos[i] = gem_to_xe_bo(gem_obj);
2959		3199
2960	if (XE_IOCTL_DBG(xe, range > bos[i]->size) \|\|	3200	err = xe_vm_bind_ioctl_validate_bo(xe, bos[i], addr, range,
2961	XE_IOCTL_DBG(xe, obj_offset >	3201	obj_offset, pat_index);
2962	bos[i]->size - range)) {	3202	if (err)
2963	err = -EINVAL;
2964	goto put_obj;
2965	}
2966
2967	if (bos[i]->flags & XE_BO_FLAG_INTERNAL_64K) {
2968	if (XE_IOCTL_DBG(xe, obj_offset &
2969	XE_64K_PAGE_MASK) \|\|
2970	XE_IOCTL_DBG(xe, addr & XE_64K_PAGE_MASK) \|\|
2971	XE_IOCTL_DBG(xe, range & XE_64K_PAGE_MASK)) {
2972	err = -EINVAL;
2973	goto put_obj;
2974	}
2975	}
2976
2977	coh_mode = xe_pat_index_get_coh_mode(xe, pat_index);
2978	if (bos[i]->cpu_caching) {
2979	if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE &&
2980	bos[i]->cpu_caching == DRM_XE_GEM_CPU_CACHING_WB)) {
2981	err = -EINVAL;
2982	goto put_obj;
2983	}
2984	} else if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE)) {
2985	/*
2986	* Imported dma-buf from a different device should
2987	* require 1way or 2way coherency since we don't know
2988	* how it was mapped on the CPU. Just assume is it
2989	* potentially cached on CPU side.
2990	*/
2991	err = -EINVAL;
2992	goto put_obj;	3203	goto put_obj;
2993	}
2994	}	3204	}
2995		3205
2996	if (args->num_syncs) {	3206	if (args->num_syncs) {
@@ -3026,6 +3236,7 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 29 unmodified lines
3026	goto free_syncs;	3236	goto free_syncs;
3027	}	3237	}
3028		3238
		3239	xe_vma_ops_init(&vops, vm, q, syncs, num_syncs);
3029	for (i = 0; i < args->num_binds; ++i) {	3240	for (i = 0; i < args->num_binds; ++i) {
3030	u64 range = bind_ops[i].range;	3241	u64 range = bind_ops[i].range;
3031	u64 addr = bind_ops[i].addr;	3242	u64 addr = bind_ops[i].addr;
@@ -3045,42 +3256,25 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 13 unmodified lines
3045	}	3256	}
3046		3257
3047	err = vm_bind_ioctl_ops_parse(vm, q, ops[i], syncs, num_syncs,	3258	err = vm_bind_ioctl_ops_parse(vm, q, ops[i], syncs, num_syncs,
3048	&ops_list,	3259	&vops, i == args->num_binds - 1);
3049	i == args->num_binds - 1);
3050	if (err)	3260	if (err)
3051	goto unwind_ops;	3261	goto unwind_ops;
3052	}	3262	}
3053		3263
3054	/* Nothing to do */	3264	/* Nothing to do */
3055	if (list_empty(&ops_list)) {	3265	if (list_empty(&vops.list)) {
3056	err = -ENODATA;	3266	err = -ENODATA;
3057	goto unwind_ops;	3267	goto unwind_ops;
3058	}	3268	}
3059		3269
3060	xe_vm_get(vm);	3270	err = vm_bind_ioctl_ops_execute(vm, &vops);
3061	if (q)
3062	xe_exec_queue_get(q);
3063
3064	err = vm_bind_ioctl_ops_execute(vm, &ops_list);
3065
3066	up_write(&vm->lock);
3067
3068	if (q)
3069	xe_exec_queue_put(q);
3070	xe_vm_put(vm);
3071
3072	for (i = 0; bos && i < args->num_binds; ++i)
3073	xe_bo_put(bos[i]);
3074
3075	kvfree(bos);
3076	kvfree(ops);
3077	if (args->num_binds > 1)
3078	kvfree(bind_ops);
3079
3080	return err;
3081		3271
3082	unwind_ops:	3272	unwind_ops:
3083	vm_bind_ioctl_ops_unwind(vm, ops, args->num_binds);	3273	if (err && err != -ENODATA)
		3274	vm_bind_ioctl_ops_unwind(vm, ops, args->num_binds);
		3275	for (i = args->num_binds - 1; i >= 0; --i)
		3276	if (ops[i])
		3277	drm_gpuva_ops_free(&vm->gpuvm, ops[i]);
3084	free_syncs:	3278	free_syncs:
3085	if (err == -ENODATA)	3279	if (err == -ENODATA)
3086	err = vm_bind_ioctl_signal_fences(vm, q, syncs, num_syncs);	3280	err = vm_bind_ioctl_signal_fences(vm, q, syncs, num_syncs);
@@ -3201,55 +3395,6 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) · 114 unmodified lines
3201	return 0;	3395	return 0;
3202	}	3396	}
3203		3397
3204	int xe_analyze_vm(struct drm_printer p, struct xe_vm vm, int gt_id)
3205	{
3206	struct drm_gpuva *gpuva;
3207	bool is_vram;
3208	uint64_t addr;
3209
3210	if (!down_read_trylock(&vm->lock)) {
3211	drm_printf(p, " Failed to acquire VM lock to dump capture");
3212	return 0;
3213	}
3214	if (vm->pt_root[gt_id]) {
3215	addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0, XE_PAGE_SIZE);
3216	is_vram = xe_bo_is_vram(vm->pt_root[gt_id]->bo);
3217	drm_printf(p, " VM root: A:0x%llx %s\n", addr,
3218	is_vram ? "VRAM" : "SYS");
3219	}
3220
3221	drm_gpuvm_for_each_va(gpuva, &vm->gpuvm) {
3222	struct xe_vma *vma = gpuva_to_vma(gpuva);
3223	bool is_userptr = xe_vma_is_userptr(vma);
3224	bool is_null = xe_vma_is_null(vma);
3225
3226	if (is_null) {
3227	addr = 0;
3228	} else if (is_userptr) {
3229	struct sg_table *sg = to_userptr_vma(vma)->userptr.sg;
3230	struct xe_res_cursor cur;
3231
3232	if (sg) {
3233	xe_res_first_sg(sg, 0, XE_PAGE_SIZE, &cur);
3234	addr = xe_res_dma(&cur);
3235	} else {
3236	addr = 0;
3237	}
3238	} else {
3239	addr = __xe_bo_addr(xe_vma_bo(vma), 0, XE_PAGE_SIZE);
3240	is_vram = xe_bo_is_vram(xe_vma_bo(vma));
3241	}
3242	drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
3243	xe_vma_start(vma), xe_vma_end(vma) - 1,
3244	xe_vma_size(vma),
3245	addr, is_null ? "NULL" : is_userptr ? "USR" :
3246	is_vram ? "VRAM" : "SYS");
3247	}
3248	up_read(&vm->lock);
3249
3250	return 0;
3251	}
3252
3253	struct xe_vm_snapshot {	3398	struct xe_vm_snapshot {
3254	unsigned long num_snaps;	3399	unsigned long num_snaps;
3255	struct {	3400	struct {
@@ -3339,7 +3484,7 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) · 83 unmodified lines
3339	}	3484	}
3340		3485
3341	if (bo) {	3486	if (bo) {
3342	dma_resv_lock(bo->ttm.base.resv, NULL);	3487	xe_bo_lock(bo, false);
3343	err = ttm_bo_vmap(&bo->ttm, &src);	3488	err = ttm_bo_vmap(&bo->ttm, &src);
3344	if (!err) {	3489	if (!err) {
3345	xe_map_memcpy_from(xe_bo_device(bo),	3490	xe_map_memcpy_from(xe_bo_device(bo),
@@ -3348,7 +3493,7 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) · 2 unmodified lines
3348	snap->snap[i].len);	3493	snap->snap[i].len);
3349	ttm_bo_vunmap(&bo->ttm, &src);	3494	ttm_bo_vunmap(&bo->ttm, &src);
3350	}	3495	}
3351	dma_resv_unlock(bo->ttm.base.resv);	3496	xe_bo_unlock(bo);
3352	} else {	3497	} else {
3353	void __user userptr = (void __user )(size_t)snap->snap[i].bo_ofs;	3498	void __user userptr = (void __user )(size_t)snap->snap[i].bo_ofs;
3354		3499

drivers/gpu/drm/xe/xe_vm.h+3−2

 #ifndef _XE_VM_H_
 #define _XE_VM_H_
+#include "xe_assert.h"
 #include "xe_bo_types.h"
 #include "xe_macros.h"
 #include "xe_map.h"
 int xe_vm_userptr_check_repin(struct xe_vm *vm);
 int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker);
+struct dma_fence *xe_vma_rebind(struct xe_vm *vm, struct xe_vma *vma,
+				u8 tile_mask);
 int xe_vm_invalidate_vma(struct xe_vma *vma);
 bool xe_vm_validate_should_retry(struct drm_exec *exec, int err, ktime_t *end);
-int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);
 int xe_vm_lock_vma(struct drm_exec *exec, struct xe_vma *vma);
 int xe_vm_validate_rebind(struct xe_vm *vm, struct drm_exec *exec,

drivers/gpu/drm/xe/xe_vm_doc.h+12−12

drivers/gpu/drm/xe/xe_vm_types.h+24−5

 	 */
 	struct list_head rebind_list;
+	/**
+	 * @destroy_work: worker to destroy VM, needed as a dma_fence signaling
+	 * from an irq context can be last put and the destroy needs to be able
+	 * to sleep.
+	 */
+	struct work_struct destroy_work;
 	/**
 	 * @rftree: range fence tree to track updates to page table structure.
 	 * Used to implement conflict tracking between independent bind engines.
 struct xe_vma_op {
 	/** @base: GPUVA base operation */
 	struct drm_gpuva_op base;
-	/**
-	 * @ops: GPUVA ops, when set call drm_gpuva_ops_free after this
-	 * operations is processed
-	 */
-	struct drm_gpuva_ops *ops;
 	/** @q: exec queue for this operation */
 	struct xe_exec_queue *q;
 	/**
 	struct list_head link;
 	/** @flags: operation flags */
 	enum xe_vma_op_flags flags;
+	/** @tile_mask: Tile mask for operation */
+	u8 tile_mask;
 	union {
 		/** @map: VMA map operation specific data */
 		struct xe_vma_op_prefetch prefetch;
 	};
 };
+/** struct xe_vma_ops - VMA operations */
+struct xe_vma_ops {
+	/** @list: list of VMA operations */
+	struct list_head list;
+	/** @vm: VM */
+	struct xe_vm *vm;
+	/** @q: exec queue these operations */
+	struct xe_exec_queue *q;
+	/** @syncs: syncs these operation */
+	struct xe_sync_entry *syncs;
+	/** @num_syncs: number of syncs */
+	u32 num_syncs;
+};
 #endif

drivers/gpu/drm/xe/xe_vram.c+368−0

@@ -0,0 +1,368 @@
	1	// SPDX-License-Identifier: MIT
	2	/*
	3	* Copyright © 2021-2024 Intel Corporation
	4	*/
	5
	6	#include <linux/pci.h>
	7
	8	#include <drm/drm_managed.h>
	9	#include <drm/drm_print.h>
	10
	11	#include "regs/xe_bars.h"
	12	#include "regs/xe_gt_regs.h"
	13	#include "regs/xe_regs.h"
	14	#include "xe_assert.h"
	15	#include "xe_device.h"
	16	#include "xe_force_wake.h"
	17	#include "xe_gt_mcr.h"
	18	#include "xe_gt_sriov_vf.h"
	19	#include "xe_mmio.h"
	20	#include "xe_module.h"
	21	#include "xe_sriov.h"
	22	#include "xe_vram.h"
	23
	24	#define BAR_SIZE_SHIFT 20
	25
	26	static void
	27	_resize_bar(struct xe_device *xe, int resno, resource_size_t size)
	28	{
	29	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
	30	int bar_size = pci_rebar_bytes_to_size(size);
	31	int ret;
	32
	33	if (pci_resource_len(pdev, resno))
	34	pci_release_resource(pdev, resno);
	35
	36	ret = pci_resize_resource(pdev, resno, bar_size);
	37	if (ret) {
	38	drm_info(&xe->drm, "Failed to resize BAR%d to %dM (%pe). Consider enabling 'Resizable BAR' support in your BIOS\n",
	39	resno, 1 << bar_size, ERR_PTR(ret));
	40	return;
	41	}
	42
	43	drm_info(&xe->drm, "BAR%d resized to %dM\n", resno, 1 << bar_size);
	44	}
	45
	46	/*
	47	* if force_vram_bar_size is set, attempt to set to the requested size
	48	* else set to maximum possible size
	49	*/
	50	static void resize_vram_bar(struct xe_device *xe)
	51	{
	52	u64 force_vram_bar_size = xe_modparam.force_vram_bar_size;
	53	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
	54	struct pci_bus *root = pdev->bus;
	55	resource_size_t current_size;
	56	resource_size_t rebar_size;
	57	struct resource *root_res;
	58	u32 bar_size_mask;
	59	u32 pci_cmd;
	60	int i;
	61
	62	/* gather some relevant info */
	63	current_size = pci_resource_len(pdev, LMEM_BAR);
	64	bar_size_mask = pci_rebar_get_possible_sizes(pdev, LMEM_BAR);
	65
	66	if (!bar_size_mask)
	67	return;
	68
	69	/* set to a specific size? */
	70	if (force_vram_bar_size) {
	71	u32 bar_size_bit;
	72
	73	rebar_size = force_vram_bar_size * (resource_size_t)SZ_1M;
	74
	75	bar_size_bit = bar_size_mask & BIT(pci_rebar_bytes_to_size(rebar_size));
	76
	77	if (!bar_size_bit) {
	78	drm_info(&xe->drm,
	79	"Requested size: %lluMiB is not supported by rebar sizes: 0x%x. Leaving default: %lluMiB\n",
	80	(u64)rebar_size >> 20, bar_size_mask, (u64)current_size >> 20);
	81	return;
	82	}
	83
	84	rebar_size = 1ULL << (__fls(bar_size_bit) + BAR_SIZE_SHIFT);
	85
	86	if (rebar_size == current_size)
	87	return;
	88	} else {
	89	rebar_size = 1ULL << (__fls(bar_size_mask) + BAR_SIZE_SHIFT);
	90
	91	/* only resize if larger than current */
	92	if (rebar_size <= current_size)
	93	return;
	94	}
	95
	96	drm_info(&xe->drm, "Attempting to resize bar from %lluMiB -> %lluMiB\n",
	97	(u64)current_size >> 20, (u64)rebar_size >> 20);
	98
	99	while (root->parent)
	100	root = root->parent;
	101
	102	pci_bus_for_each_resource(root, root_res, i) {
	103	if (root_res && root_res->flags & (IORESOURCE_MEM \| IORESOURCE_MEM_64) &&
	104	(u64)root_res->start > 0x100000000ul)
	105	break;
	106	}
	107
	108	if (!root_res) {
	109	drm_info(&xe->drm, "Can't resize VRAM BAR - platform support is missing. Consider enabling 'Resizable BAR' support in your BIOS\n");
	110	return;
	111	}
	112
	113	pci_read_config_dword(pdev, PCI_COMMAND, &pci_cmd);
	114	pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd & ~PCI_COMMAND_MEMORY);
	115
	116	_resize_bar(xe, LMEM_BAR, rebar_size);
	117
	118	pci_assign_unassigned_bus_resources(pdev->bus);
	119	pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd);
	120	}
	121
	122	static bool resource_is_valid(struct pci_dev *pdev, int bar)
	123	{
	124	if (!pci_resource_flags(pdev, bar))
	125	return false;
	126
	127	if (pci_resource_flags(pdev, bar) & IORESOURCE_UNSET)
	128	return false;
	129
	130	if (!pci_resource_len(pdev, bar))
	131	return false;
	132
	133	return true;
	134	}
	135
	136	static int determine_lmem_bar_size(struct xe_device *xe)
	137	{
	138	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
	139
	140	if (!resource_is_valid(pdev, LMEM_BAR)) {
	141	drm_err(&xe->drm, "pci resource is not valid\n");
	142	return -ENXIO;
	143	}
	144
	145	resize_vram_bar(xe);
	146
	147	xe->mem.vram.io_start = pci_resource_start(pdev, LMEM_BAR);
	148	xe->mem.vram.io_size = pci_resource_len(pdev, LMEM_BAR);
	149	if (!xe->mem.vram.io_size)
	150	return -EIO;
	151
	152	/* XXX: Need to change when xe link code is ready */
	153	xe->mem.vram.dpa_base = 0;
	154
	155	/* set up a map to the total memory area. */
	156	xe->mem.vram.mapping = ioremap_wc(xe->mem.vram.io_start, xe->mem.vram.io_size);
	157
	158	return 0;
	159	}
	160
	161	static inline u64 get_flat_ccs_offset(struct xe_gt *gt, u64 tile_size)
	162	{
	163	struct xe_device *xe = gt_to_xe(gt);
	164	u64 offset;
	165	u32 reg;
	166
	167	if (GRAPHICS_VER(xe) >= 20) {
	168	u64 ccs_size = tile_size / 512;
	169	u64 offset_hi, offset_lo;
	170	u32 nodes, num_enabled;
	171
	172	reg = xe_mmio_read32(gt, MIRROR_FUSE3);
	173	nodes = REG_FIELD_GET(XE2_NODE_ENABLE_MASK, reg);
	174	num_enabled = hweight32(nodes); /* Number of enabled l3 nodes */
	175
	176	reg = xe_gt_mcr_unicast_read_any(gt, XE2_FLAT_CCS_BASE_RANGE_LOWER);
	177	offset_lo = REG_FIELD_GET(XE2_FLAT_CCS_BASE_LOWER_ADDR_MASK, reg);
	178
	179	reg = xe_gt_mcr_unicast_read_any(gt, XE2_FLAT_CCS_BASE_RANGE_UPPER);
	180	offset_hi = REG_FIELD_GET(XE2_FLAT_CCS_BASE_UPPER_ADDR_MASK, reg);
	181
	182	offset = offset_hi << 32; /* HW view bits 39:32 */
	183	offset \|= offset_lo << 6; /* HW view bits 31:6 */
	184	offset = num_enabled; / convert to SW view */
	185
	186	/* We don't expect any holes */
	187	xe_assert_msg(xe, offset == (xe_mmio_read64_2x32(gt, GSMBASE) - ccs_size),
	188	"Hole between CCS and GSM.\n");
	189	} else {
	190	reg = xe_gt_mcr_unicast_read_any(gt, XEHP_FLAT_CCS_BASE_ADDR);
	191	offset = (u64)REG_FIELD_GET(XEHP_FLAT_CCS_PTR, reg) * SZ_64K;
	192	}
	193
	194	return offset;
	195	}
	196
	197	/*
	198	* tile_vram_size() - Collect vram size and offset information
	199	* @tile: tile to get info for
	200	* @vram_size: available vram (size - device reserved portions)
	201	* @tile_size: actual vram size
	202	* @tile_offset: physical start point in the vram address space
	203	*
	204	* There are 4 places for size information:
	205	* - io size (from pci_resource_len of LMEM bar) (only used for small bar and DG1)
	206	* - TILEx size (actual vram size)
	207	* - GSMBASE offset (TILEx - "stolen")
	208	* - CSSBASE offset (TILEx - CSS space necessary)
	209	*
	210	* CSSBASE is always a lower/smaller offset then GSMBASE.
	211	*
	212	* The actual available size of memory is to the CCS or GSM base.
	213	* NOTE: multi-tile bases will include the tile offset.
	214	*
	215	*/
	216	static int tile_vram_size(struct xe_tile tile, u64 vram_size,
	217	u64 tile_size, u64 tile_offset)
	218	{
	219	struct xe_device *xe = tile_to_xe(tile);
	220	struct xe_gt *gt = tile->primary_gt;
	221	u64 offset;
	222	int err;
	223	u32 reg;
	224
	225	if (IS_SRIOV_VF(xe)) {
	226	struct xe_tile *t;
	227	int id;
	228
	229	offset = 0;
	230	for_each_tile(t, xe, id)
	231	for_each_if(t->id < tile->id)
	232	offset += xe_gt_sriov_vf_lmem(t->primary_gt);
	233
	234	*tile_size = xe_gt_sriov_vf_lmem(gt);
	235	vram_size = tile_size;
	236	*tile_offset = offset;
	237
	238	return 0;
	239	}
	240
	241	err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
	242	if (err)
	243	return err;
	244
	245	/* actual size */
	246	if (unlikely(xe->info.platform == XE_DG1)) {
	247	*tile_size = pci_resource_len(to_pci_dev(xe->drm.dev), LMEM_BAR);
	248	*tile_offset = 0;
	249	} else {
	250	reg = xe_gt_mcr_unicast_read_any(gt, XEHP_TILE_ADDR_RANGE(gt->info.id));
	251	tile_size = (u64)REG_FIELD_GET(GENMASK(14, 8), reg) SZ_1G;
	252	tile_offset = (u64)REG_FIELD_GET(GENMASK(7, 1), reg) SZ_1G;
	253	}
	254
	255	/* minus device usage */
	256	if (xe->info.has_flat_ccs) {
	257	offset = get_flat_ccs_offset(gt, *tile_size);
	258	} else {
	259	offset = xe_mmio_read64_2x32(gt, GSMBASE);
	260	}
	261
	262	/* remove the tile offset so we have just the available size */
	263	vram_size = offset - tile_offset;
	264
	265	return xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
	266	}
	267
	268	static void vram_fini(void *arg)
	269	{
	270	struct xe_device *xe = arg;
	271	struct xe_tile *tile;
	272	int id;
	273
	274	if (xe->mem.vram.mapping)
	275	iounmap(xe->mem.vram.mapping);
	276
	277	xe->mem.vram.mapping = NULL;
	278
	279	for_each_tile(tile, xe, id)
	280	tile->mem.vram.mapping = NULL;
	281	}
	282
	283	/**
	284	* xe_vram_probe() - Probe VRAM configuration
	285	* @xe: the &xe_device
	286	*
	287	* Collect VRAM size and offset information for all tiles.
	288	*
	289	* Return: 0 on success, error code on failure
	290	*/
	291	int xe_vram_probe(struct xe_device *xe)
	292	{
	293	struct xe_tile *tile;
	294	resource_size_t io_size;
	295	u64 available_size = 0;
	296	u64 total_size = 0;
	297	u64 tile_offset;
	298	u64 tile_size;
	299	u64 vram_size;
	300	int err;
	301	u8 id;
	302
	303	if (!IS_DGFX(xe))
	304	return 0;
	305
	306	/* Get the size of the root tile's vram for later accessibility comparison */
	307	tile = xe_device_get_root_tile(xe);
	308	err = tile_vram_size(tile, &vram_size, &tile_size, &tile_offset);
	309	if (err)
	310	return err;
	311
	312	err = determine_lmem_bar_size(xe);
	313	if (err)
	314	return err;
	315
	316	drm_info(&xe->drm, "VISIBLE VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
	317	&xe->mem.vram.io_size);
	318
	319	io_size = xe->mem.vram.io_size;
	320
	321	/* tile specific ranges */
	322	for_each_tile(tile, xe, id) {
	323	err = tile_vram_size(tile, &vram_size, &tile_size, &tile_offset);
	324	if (err)
	325	return err;
	326
	327	tile->mem.vram.actual_physical_size = tile_size;
	328	tile->mem.vram.io_start = xe->mem.vram.io_start + tile_offset;
	329	tile->mem.vram.io_size = min_t(u64, vram_size, io_size);
	330
	331	if (!tile->mem.vram.io_size) {
	332	drm_err(&xe->drm, "Tile without any CPU visible VRAM. Aborting.\n");
	333	return -ENODEV;
	334	}
	335
	336	tile->mem.vram.dpa_base = xe->mem.vram.dpa_base + tile_offset;
	337	tile->mem.vram.usable_size = vram_size;
	338	tile->mem.vram.mapping = xe->mem.vram.mapping + tile_offset;
	339
	340	if (tile->mem.vram.io_size < tile->mem.vram.usable_size)
	341	drm_info(&xe->drm, "Small BAR device\n");
	342	drm_info(&xe->drm, "VRAM[%u, %u]: Actual physical size %pa, usable size exclude stolen %pa, CPU accessible size %pa\n", id,
	343	tile->id, &tile->mem.vram.actual_physical_size, &tile->mem.vram.usable_size, &tile->mem.vram.io_size);
	344	drm_info(&xe->drm, "VRAM[%u, %u]: DPA range: [%pa-%llx], io range: [%pa-%llx]\n", id, tile->id,
	345	&tile->mem.vram.dpa_base, tile->mem.vram.dpa_base + (u64)tile->mem.vram.actual_physical_size,
	346	&tile->mem.vram.io_start, tile->mem.vram.io_start + (u64)tile->mem.vram.io_size);
	347
	348	/* calculate total size using tile size to get the correct HW sizing */
	349	total_size += tile_size;
	350	available_size += vram_size;
	351
	352	if (total_size > xe->mem.vram.io_size) {
	353	drm_info(&xe->drm, "VRAM: %pa is larger than resource %pa\n",
	354	&total_size, &xe->mem.vram.io_size);
	355	}
	356
	357	io_size -= min_t(u64, tile_size, io_size);
	358	}
	359
	360	xe->mem.vram.actual_physical_size = total_size;
	361
	362	drm_info(&xe->drm, "Total VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
	363	&xe->mem.vram.actual_physical_size);
	364	drm_info(&xe->drm, "Available VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
	365	&available_size);
	366
	367	return devm_add_action_or_reset(xe->drm.dev, vram_fini, xe);
	368	}

drivers/gpu/drm/xe/xe_vram.h+13−0

@@ -0,0 +1,13 @@
	1	/* SPDX-License-Identifier: MIT */
	2	/*
	3	* Copyright © 2024 Intel Corporation
	4	*/
	5
	6	#ifndef _XE_VRAM_H_
	7	#define _XE_VRAM_H_
	8
	9	struct xe_device;
	10
	11	int xe_vram_probe(struct xe_device *xe);
	12
	13	#endif

drivers/gpu/drm/xe/xe_vram_freq.c+2−2

@@ -87,7 +87,7 @@ static const struct attribute_group freq_group_attrs = { · 86 unmodified lines
87	.attrs = freq_attrs,	87	.attrs = freq_attrs,
88	};	88	};
89		89
90	static void vram_freq_sysfs_fini(struct drm_device drm, void arg)	90	static void vram_freq_sysfs_fini(void *arg)
91	{	91	{
92	struct kobject *kobj = arg;	92	struct kobject *kobj = arg;
93		93
@@ -122,5 +122,5 @@ int xe_vram_freq_sysfs_init(struct xe_tile *tile) · 28 unmodified lines
122	return err;	122	return err;
123	}	123	}
124		124
125	return drmm_add_action_or_reset(&xe->drm, vram_freq_sysfs_fini, kobj);	125	return devm_add_action_or_reset(xe->drm.dev, vram_freq_sysfs_fini, kobj);
126	}	126	}

drivers/gpu/drm/xe/xe_wa.c+34−5

 	},
 	{ XE_RTP_NAME("14020316580"),
 	  XE_RTP_RULES(MEDIA_VERSION(1301)),
-	  XE_RTP_ACTIONS(CLR(PG_ENABLE,
+	  XE_RTP_ACTIONS(CLR(POWERGATE_ENABLE,
-			     VD0_HCP_POWERGATE_ENABLE |
+			     VDN_HCP_POWERGATE_ENABLE(0) |
-			     VD0_MFXVDENC_POWERGATE_ENABLE |
+			     VDN_MFXVDENC_POWERGATE_ENABLE(0) |
-			     VD2_HCP_POWERGATE_ENABLE |
+			     VDN_HCP_POWERGATE_ENABLE(2) |
-			     VD2_MFXVDENC_POWERGATE_ENABLE)),
+			     VDN_MFXVDENC_POWERGATE_ENABLE(2))),
 	},
 	{ XE_RTP_NAME("14019449301"),
 	  XE_RTP_RULES(MEDIA_VERSION(1301), ENGINE_CLASS(VIDEO_DECODE)),
 		       FUNC(xe_rtp_match_first_render_or_compute)),
 	  XE_RTP_ACTIONS(SET(LSC_CHICKEN_BIT_0, WR_REQ_CHAINING_DIS))
 	},
+	{ XE_RTP_NAME("14021402888"),
+	  XE_RTP_RULES(GRAPHICS_VERSION(2001), ENGINE_CLASS(RENDER)),
+	  XE_RTP_ACTIONS(SET(HALF_SLICE_CHICKEN7, CLEAR_OPTIMIZATION_DISABLE))
+	},
 	/* Xe2_HPM */
 	  XE_RTP_RULES(GRAPHICS_VERSION_RANGE(2001, 2004), ENGINE_CLASS(RENDER)),
 	  XE_RTP_ACTIONS(SET(COMMON_SLICE_CHICKEN1, DISABLE_BOTTOM_CLIP_RECTANGLE_TEST))
 	},
+	{ XE_RTP_NAME("14021567978"),
+	  XE_RTP_RULES(GRAPHICS_VERSION_RANGE(2001, XE_RTP_END_VERSION_UNDEFINED),
+		       ENGINE_CLASS(RENDER)),
+	  XE_RTP_ACTIONS(SET(CHICKEN_RASTER_2, TBIMR_FAST_CLIP))
+	},
+	{ XE_RTP_NAME("14020756599"),
+	  XE_RTP_RULES(GRAPHICS_VERSION(2004), ENGINE_CLASS(RENDER)),
+	  XE_RTP_ACTIONS(SET(WM_CHICKEN3, HIZ_PLANE_COMPRESSION_DIS))
+	},
 	/* Xe2_HPG */
 	{ XE_RTP_NAME("15010599737"),
 	  XE_RTP_RULES(GRAPHICS_VERSION(2001), ENGINE_CLASS(RENDER)),
 	  XE_RTP_ACTIONS(SET(WM_CHICKEN3, HIZ_PLANE_COMPRESSION_DIS))
 	},
+	{ XE_RTP_NAME("14021490052"),
+	  XE_RTP_RULES(GRAPHICS_VERSION(2001), ENGINE_CLASS(RENDER)),
+	  XE_RTP_ACTIONS(SET(FF_MODE,
+			     DIS_MESH_PARTIAL_AUTOSTRIP |
+			     DIS_MESH_AUTOSTRIP),
+			 SET(VFLSKPD,
+			     DIS_PARTIAL_AUTOSTRIP |
+			     DIS_AUTOSTRIP))
+	},
+	/* Xe2_LPM */
+	{ XE_RTP_NAME("14020756599"),
+	  XE_RTP_RULES(ENGINE_CLASS(RENDER), FUNC(xe_rtp_match_when_media2000)),
+	  XE_RTP_ACTIONS(SET(WM_CHICKEN3, HIZ_PLANE_COMPRESSION_DIS))
+	},
 	{}
 };

include/uapi/drm/xe_drm.h+2−0

  *    containing the following in mask:
  *    ``DSS_COMPUTE    ff ff ff ff 00 00 00 00``
  *    means 32 DSS are available for compute.
+ *  - %DRM_XE_TOPO_L3_BANK - To query the mask of enabled L3 banks
  *  - %DRM_XE_TOPO_EU_PER_DSS - To query the mask of Execution Units (EU)
  *    available per Dual Sub Slices (DSS). For example a query response
  *    containing the following in mask:
 #define DRM_XE_TOPO_DSS_GEOMETRY	1
 #define DRM_XE_TOPO_DSS_COMPUTE		2
+#define DRM_XE_TOPO_L3_BANK		3
 #define DRM_XE_TOPO_EU_PER_DSS		4
 	/** @type: type of mask */
 	__u16 type;

@@ -18,7 +18,7 @@ void xe_display_driver_remove(struct xe_device *xe); · 17 unmodified lines
18		18
19	int xe_display_create(struct xe_device *xe);	19	int xe_display_create(struct xe_device *xe);
20		20
21	void xe_display_probe(struct xe_device *xe);	21	int xe_display_probe(struct xe_device *xe);
22		22
23	int xe_display_init_nommio(struct xe_device *xe);	23	int xe_display_init_nommio(struct xe_device *xe);
24	int xe_display_init_noirq(struct xe_device *xe);	24	int xe_display_init_noirq(struct xe_device *xe);
@@ -34,10 +34,10 @@ void xe_display_irq_enable(struct xe_device *xe, u32 gu_misc_iir); · 9 unmodified lines
34	void xe_display_irq_reset(struct xe_device *xe);	34	void xe_display_irq_reset(struct xe_device *xe);
35	void xe_display_irq_postinstall(struct xe_device xe, struct xe_gt gt);	35	void xe_display_irq_postinstall(struct xe_device xe, struct xe_gt gt);
36		36
37	void xe_display_pm_suspend(struct xe_device *xe);	37	void xe_display_pm_suspend(struct xe_device *xe, bool runtime);
38	void xe_display_pm_suspend_late(struct xe_device *xe);	38	void xe_display_pm_suspend_late(struct xe_device *xe);
39	void xe_display_pm_resume_early(struct xe_device *xe);	39	void xe_display_pm_resume_early(struct xe_device *xe);
40	void xe_display_pm_resume(struct xe_device *xe);	40	void xe_display_pm_resume(struct xe_device *xe, bool runtime);
41		41
42	#else	42	#else
43		43
@@ -47,7 +47,7 @@ static inline void xe_display_driver_remove(struct xe_device *xe) {} · 3 unmodified lines
47		47
48	static inline int xe_display_create(struct xe_device *xe) { return 0; }	48	static inline int xe_display_create(struct xe_device *xe) { return 0; }
49		49
50	static inline void xe_display_probe(struct xe_device *xe) { }	50	static inline int xe_display_probe(struct xe_device *xe) { return 0; }
51		51
52	static inline int xe_display_init_nommio(struct xe_device *xe) { return 0; }	52	static inline int xe_display_init_nommio(struct xe_device *xe) { return 0; }
53	static inline int xe_display_init_noirq(struct xe_device *xe) { return 0; }	53	static inline int xe_display_init_noirq(struct xe_device *xe) { return 0; }
@@ -63,10 +63,10 @@ static inline void xe_display_irq_enable(struct xe_device *xe, u32 gu_misc_iir) · 9 unmodified lines
63	static inline void xe_display_irq_reset(struct xe_device *xe) {}	63	static inline void xe_display_irq_reset(struct xe_device *xe) {}
64	static inline void xe_display_irq_postinstall(struct xe_device xe, struct xe_gt gt) {}	64	static inline void xe_display_irq_postinstall(struct xe_device xe, struct xe_gt gt) {}
65		65
66	static inline void xe_display_pm_suspend(struct xe_device *xe) {}	66	static inline void xe_display_pm_suspend(struct xe_device *xe, bool runtime) {}
67	static inline void xe_display_pm_suspend_late(struct xe_device *xe) {}	67	static inline void xe_display_pm_suspend_late(struct xe_device *xe) {}
68	static inline void xe_display_pm_resume_early(struct xe_device *xe) {}	68	static inline void xe_display_pm_resume_early(struct xe_device *xe) {}
69	static inline void xe_display_pm_resume(struct xe_device *xe) {}	69	static inline void xe_display_pm_resume(struct xe_device *xe, bool runtime) {}
70		70
71	#endif /* CONFIG_DRM_XE_DISPLAY */	71	#endif /* CONFIG_DRM_XE_DISPLAY */
72	#endif /* _XE_DISPLAY_H_ */	72	#endif /* _XE_DISPLAY_H_ */

@@ -25,7 +25,7 @@ · 24 unmodified lines
25	* VM bind (create GPU mapping for a BO or userptr)	25	* VM bind (create GPU mapping for a BO or userptr)
26	* ================================================	26	* ================================================
27	*	27	*
28	* Creates GPU mapings for a BO or userptr within a VM. VM binds uses the same	28	* Creates GPU mappings for a BO or userptr within a VM. VM binds uses the same
29	* in / out fence interface (struct drm_xe_sync) as execs which allows users to	29	* in / out fence interface (struct drm_xe_sync) as execs which allows users to
30	* think of binds and execs as more or less the same operation.	30	* think of binds and execs as more or less the same operation.
31	*	31	*
@@ -190,8 +190,8 @@ · 158 unmodified lines
190	* Deferred binds in fault mode	190	* Deferred binds in fault mode
191	* ----------------------------	191	* ----------------------------
192	*	192	*
193	* In a VM is in fault mode (TODO: link to fault mode), new bind operations that	193	* If a VM is in fault mode (TODO: link to fault mode), new bind operations that
194	* create mappings are by default are deferred to the page fault handler (first	194	* create mappings are by default deferred to the page fault handler (first
195	* use). This behavior can be overriden by setting the flag	195	* use). This behavior can be overriden by setting the flag
196	* DRM_XE_VM_BIND_FLAG_IMMEDIATE which indicates to creating the mapping	196	* DRM_XE_VM_BIND_FLAG_IMMEDIATE which indicates to creating the mapping
197	* immediately.	197	* immediately.
@@ -225,7 +225,7 @@ · 27 unmodified lines
225	*	225	*
226	* A VM in compute mode enables long running workloads and ultra low latency	226	* A VM in compute mode enables long running workloads and ultra low latency
227	* submission (ULLS). ULLS is implemented via a continuously running batch +	227	* submission (ULLS). ULLS is implemented via a continuously running batch +
228	* semaphores. This enables to the user to insert jump to new batch commands	228	* semaphores. This enables the user to insert jump to new batch commands
229	* into the continuously running batch. In both cases these batches exceed the	229	* into the continuously running batch. In both cases these batches exceed the
230	* time a dma fence is allowed to exist for before signaling, as such dma fences	230	* time a dma fence is allowed to exist for before signaling, as such dma fences
231	* are not used when a VM is in compute mode. User fences (TODO: link user fence	231	* are not used when a VM is in compute mode. User fences (TODO: link user fence
@@ -244,7 +244,7 @@ · 12 unmodified lines
244	* Once all preempt fences are signaled for a VM the kernel can safely move the	244	* Once all preempt fences are signaled for a VM the kernel can safely move the
245	* memory and kick the rebind worker which resumes all the engines execution.	245	* memory and kick the rebind worker which resumes all the engines execution.
246	*	246	*
247	* A preempt fence, for every engine using the VM, is installed the VM's	247	* A preempt fence, for every engine using the VM, is installed into the VM's
248	* dma-resv DMA_RESV_USAGE_PREEMPT_FENCE slot. The same preempt fence, for every	248	* dma-resv DMA_RESV_USAGE_PREEMPT_FENCE slot. The same preempt fence, for every
249	* engine using the VM, is also installed into the same dma-resv slot of every	249	* engine using the VM, is also installed into the same dma-resv slot of every
250	* external BO mapped in the VM.	250	* external BO mapped in the VM.
@@ -314,7 +314,7 @@ · 63 unmodified lines
314	* signaling, and memory allocation is usually required to resolve a page	314	* signaling, and memory allocation is usually required to resolve a page
315	* fault, but memory allocation is not allowed to gate dma fence signaling. As	315	* fault, but memory allocation is not allowed to gate dma fence signaling. As
316	* such, dma fences are not allowed when VM is in fault mode. Because dma-fences	316	* such, dma fences are not allowed when VM is in fault mode. Because dma-fences
317	* are not allowed, long running workloads and ULLS are enabled on a faulting	317	* are not allowed, only long running workloads and ULLS are enabled on a faulting
318	* VM.	318	* VM.
319	*	319	*
320	* Defered VM binds	320	* Defered VM binds
@@ -399,14 +399,14 @@ · 78 unmodified lines
399	* Notice no rebind is issued in the access counter handler as the rebind will	399	* Notice no rebind is issued in the access counter handler as the rebind will
400	* be issued on next page fault.	400	* be issued on next page fault.
401	*	401	*
402	* Cavets with eviction / user pointer invalidation	402	* Caveats with eviction / user pointer invalidation
403	* ------------------------------------------------	403	* -------------------------------------------------
404	*	404	*
405	* In the case of eviction and user pointer invalidation on a faulting VM, there	405	* In the case of eviction and user pointer invalidation on a faulting VM, there
406	* is no need to issue a rebind rather we just need to blow away the page tables	406	* is no need to issue a rebind rather we just need to blow away the page tables
407	* for the VMAs and the page fault handler will rebind the VMAs when they fault.	407	* for the VMAs and the page fault handler will rebind the VMAs when they fault.
408	* The cavet is to update / read the page table structure the VM global lock is	408	* The caveat is to update / read the page table structure the VM global lock is
409	* neeeed. In both the case of eviction and user pointer invalidation locks are	409	* needed. In both the case of eviction and user pointer invalidation locks are
410	* held which make acquiring the VM global lock impossible. To work around this	410	* held which make acquiring the VM global lock impossible. To work around this
411	* every VMA maintains a list of leaf page table entries which should be written	411	* every VMA maintains a list of leaf page table entries which should be written
412	* to zero to blow away the VMA's page tables. After writing zero to these	412	* to zero to blow away the VMA's page tables. After writing zero to these
@@ -427,9 +427,9 @@ · 14 unmodified lines
427	* VM global lock (vm->lock) - rw semaphore lock. Outer most lock which protects	427	* VM global lock (vm->lock) - rw semaphore lock. Outer most lock which protects
428	* the list of userptrs mapped in the VM, the list of engines using this VM, and	428	* the list of userptrs mapped in the VM, the list of engines using this VM, and
429	* the array of external BOs mapped in the VM. When adding or removing any of the	429	* the array of external BOs mapped in the VM. When adding or removing any of the
430	* aforemented state from the VM should acquire this lock in write mode. The VM	430	* aforementioned state from the VM should acquire this lock in write mode. The VM
431	* bind path also acquires this lock in write while the exec / compute mode	431	* bind path also acquires this lock in write while the exec / compute mode
432	* rebind worker acquire this lock in read mode.	432	* rebind worker acquires this lock in read mode.
433	*	433	*
434	* VM dma-resv lock (vm->ttm.base.resv->lock) - WW lock. Protects VM dma-resv	434	* VM dma-resv lock (vm->ttm.base.resv->lock) - WW lock. Protects VM dma-resv
435	* slots which is shared with any private BO in the VM. Expected to be acquired	435	* slots which is shared with any private BO in the VM. Expected to be acquired