Merge tag 'drm-xe-next-2024-06-06' of https://gitlab.freedesktop.org/drm/xe/kernel into drm-next

UAPI Changes: - Expose the L3 bank mask (Francois) Cross-subsystem Changes: - Update Xe driver maintainers (Oded) Display (i915): - Add missing include to intel_vga.c (Michal Wajdeczko) Driver Changes: - Fix Display (xe-only) detection for ADL-N (Lucas) - Runtime PM fixes that enabled PC-10 and D3Cold (Francois, Rodrigo) - Fix unexpected silent drm backmerge issues (Thomas) - More (a lot more) preparation for SR-IOV support (Michal Wajdeczko) - Devcoredump fixes and improvements (Jose, Tejas, Matt Brost) - Introduce device 'wedged' state (Rodrigo) - Improve debug and info messages (Michal Wajdeczko, Rodrigo, Nirmoy) - Adding or fixing workarounds (Tejas, Shekhar, Lucas, Bommu) - Check result of drmm_mutex_init (Michal Wajdeczko) - Enlarge the critical dma fence area for preempt fences (Matt Auld) - Prevent UAF in VM's rebind work (Matt Auld) - GuC submit related clean-ups and fixes (Matt Brost, Himal, Jonathan, Niranjana) - Prefer local helpers to perform dma reservation locking (Himal) - Spelling and typo fixes (Colin, Francois) - Prep patches for 1 job per VM bind IOCTL (no uapi change yet) (Matt Brost) - Remove uninitialized end var from xe_gt_tlb_invalidation_range (Nirmoy) - GSC related changes targeting LNL support (Daniele) - Fix assert in L3 bank mask generation (Francois) - Perform dma_map when moving system buffer objects to TT (Thomas) - Add helpers for manipulating macro arguments (Michal Wajdeczko) - Refactor default device atomic settings (Nirmoy) - Add debugfs node to dump mocs (Janga) - Use ordered WQ for G2H handler (Matt Brost) - Clean up and fixes in header includes (Michal Wajdeczko) - Prefer flexible-array over deprecated zero-lenght ones (Lucas) - Add Indirect Ring State support (Niranjana) - Fix UBSAN shift-out-of-bounds failure (Shuicheng) - HWMon fixes and additions (Karthik) - Clean-up refactor around probe init functions (Lucas, Michal Wajdeczko) - Fix PCODE init function (Himal) - Only use reserved BCS instances for usm migrate exec queue (Matt Brost) - Only zap PTEs as needed (Matt Brost) - Per client usage info (Lucas) - Core hotunplug improvements converting stuff towards devm (Matt Auld) - Don't emit false error if running in execlist mode (Michal Wajdeczko) - Remove unused struct (Dr. David) - Support/debug for slow GuC loads (John Harrison) - Decouple job seqno and lrc seqno (Matt Brost) - Allow migrate vm gpu submissions from reclaim context (Thomas) - Rename drm-client running time to run_ticks and fix a UAF (Umesh) - Check empty pinned BO list with lock held (Nirmoy) - Drop undesired prefix from the platform name (Michal Wajdeczko) - Remove unwanted mutex locking on xe file close (Niranjana) - Replace format-less snprintf() with strscpy() (Arnd) - Other general clean-ups on registers definitions and function names (Michal Wajdeczko) - Add kernel-doc to some xe_lrc interfaces (Niranajana) - Use missing lock in relay_needs_worker (Nirmoy) - Drop redundant W=1 warnings from Makefile (Jani) - Simplify if condition in preempt fences code (Thorsten) - Flush engine buffers before signalling user fence on all engines (Andrzej) - Don't overmap identity VRAM mapping (Matt Brost) - Do not dereference NULL job->fence in trace points (Matt Brost) - Add synchronous gt reset debugfs (Jonathan) - Xe gt_idle fixes (Riana) Signed-off-by: Dave Airlie <airlied@redhat.com>

Dave Airlie committed 2 years ago

Commit 79570662 parents 83a7eef 6800e63163 files · +8025 −1778Browse files

Unified · Split

163 files

Documentation/gpu/drm-usage-stats.rst +19 −2
Documentation/gpu/xe/index.rst +1 −0
Documentation/gpu/xe/xe-drm-usage-stats.rst +10 −0
MAINTAINERS +0 −1
drivers/gpu/drm/i915/display/intel_vga.c +1 −0
drivers/gpu/drm/xe/Kconfig.debug +0 −10
drivers/gpu/drm/xe/Makefile +9 −25
drivers/gpu/drm/xe/abi/guc_actions_sriov_abi.h +190 −0
drivers/gpu/drm/xe/abi/guc_errors_abi.h +24 −2
drivers/gpu/drm/xe/abi/guc_klvs_abi.h +32 −8
drivers/gpu/drm/xe/abi/guc_relay_actions_abi.h +169 −1
drivers/gpu/drm/xe/display/xe_display.c +28 −15
drivers/gpu/drm/xe/display/xe_display.h +6 −6
drivers/gpu/drm/xe/display/xe_hdcp_gsc.c +1 −0
drivers/gpu/drm/xe/instructions/xe_gpu_commands.h +0 −0
drivers/gpu/drm/xe/regs/xe_bars.h +11 −0
drivers/gpu/drm/xe/regs/xe_engine_regs.h +8 −2
drivers/gpu/drm/xe/regs/xe_gt_regs.h +61 −6
drivers/gpu/drm/xe/regs/xe_guc_regs.h +2 −0
drivers/gpu/drm/xe/regs/xe_lrc_layout.h +8 −0
drivers/gpu/drm/xe/regs/xe_pcode_regs.h +7 −0
drivers/gpu/drm/xe/regs/xe_regs.h +3 −0
drivers/gpu/drm/xe/regs/xe_sriov_regs.h +3 −0
drivers/gpu/drm/xe/tests/Makefile +1 −0
drivers/gpu/drm/xe/tests/xe_args_test.c +221 −0
drivers/gpu/drm/xe/tests/xe_gt_sriov_pf_service_test.c +232 −0
drivers/gpu/drm/xe/tests/xe_migrate.c +1 −87
drivers/gpu/drm/xe/xe_args.h +143 −0
drivers/gpu/drm/xe/xe_assert.h +2 −2
drivers/gpu/drm/xe/xe_bb.c +1 −1
drivers/gpu/drm/xe/xe_bo.c +48 −23
drivers/gpu/drm/xe/xe_bo.h +1 −0
drivers/gpu/drm/xe/xe_debugfs.c +58 −0
drivers/gpu/drm/xe/xe_devcoredump.c +17 −2
drivers/gpu/drm/xe/xe_devcoredump_types.h +2 −0
drivers/gpu/drm/xe/xe_device.c +78 −29
drivers/gpu/drm/xe/xe_device.h +7 −6
drivers/gpu/drm/xe/xe_device_sysfs.c +2 −2
drivers/gpu/drm/xe/xe_device_types.h +20 −3
drivers/gpu/drm/xe/xe_drm_client.c +123 −1
drivers/gpu/drm/xe/xe_exec_queue.c +47 −30
drivers/gpu/drm/xe/xe_exec_queue.h +10 −0
drivers/gpu/drm/xe/xe_exec_queue_types.h +5 −11
drivers/gpu/drm/xe/xe_execlist.c +6 −6
drivers/gpu/drm/xe/xe_ggtt.c +28 −4
drivers/gpu/drm/xe/xe_gsc.c +3 −0
drivers/gpu/drm/xe/xe_gsc.h +2 −1
drivers/gpu/drm/xe/xe_gsc_proxy.c +1 −0
drivers/gpu/drm/xe/xe_gsc_submit.c +1 −0
drivers/gpu/drm/xe/xe_gt.c +105 −37
drivers/gpu/drm/xe/xe_gt.h +27 −0
drivers/gpu/drm/xe/xe_gt_ccs_mode.c +6 −4
drivers/gpu/drm/xe/xe_gt_clock.c +1 −0
drivers/gpu/drm/xe/xe_gt_debugfs.c +31 −0
drivers/gpu/drm/xe/xe_gt_freq.c +4 −4
drivers/gpu/drm/xe/xe_gt_idle.c +61 −5
drivers/gpu/drm/xe/xe_gt_idle.h +3 −1
drivers/gpu/drm/xe/xe_gt_mcr.c +22 −5
drivers/gpu/drm/xe/xe_gt_mcr.h +25 −0
drivers/gpu/drm/xe/xe_gt_pagefault.c +7 −9
drivers/gpu/drm/xe/xe_gt_sriov_pf.c +32 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf.h +5 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c +99 −6
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.h +6 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_config_types.h +4 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c +427 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.h +18 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.c +147 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.h +27 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor_types.h +22 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c +550 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_service.h +36 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_service_types.h +52 −0
drivers/gpu/drm/xe/xe_gt_sriov_pf_types.h +10 −0
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +979 −0
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +30 −0
drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.c +72 −0
drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.h +14 −0
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +84 −0
drivers/gpu/drm/xe/xe_gt_sysfs.c +2 −2
drivers/gpu/drm/xe/xe_gt_throttle.c +15 −15
drivers/gpu/drm/xe/xe_gt_throttle.h +17 −0
drivers/gpu/drm/xe/xe_gt_throttle_sysfs.h +0 −16
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c +47 −17
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h +3 −0
drivers/gpu/drm/xe/xe_gt_topology.c +13 −1
drivers/gpu/drm/xe/xe_gt_topology.h +3 −0
drivers/gpu/drm/xe/xe_gt_types.h +26 −8
drivers/gpu/drm/xe/xe_guc.c +272 −82
drivers/gpu/drm/xe/xe_guc.h +1 −2
drivers/gpu/drm/xe/xe_guc_ads.c +67 −4
drivers/gpu/drm/xe/xe_guc_ads.h +2 −1
drivers/gpu/drm/xe/xe_guc_ct.c +4 −0
drivers/gpu/drm/xe/xe_guc_db_mgr.c +2 −1
drivers/gpu/drm/xe/xe_guc_id_mgr.c +2 −1
drivers/gpu/drm/xe/xe_guc_klv_helpers.c +12 −0
drivers/gpu/drm/xe/xe_guc_klv_helpers.h +14 −1
drivers/gpu/drm/xe/xe_guc_klv_thresholds_set.h +64 −0
drivers/gpu/drm/xe/xe_guc_klv_thresholds_set_types.h +68 −0
drivers/gpu/drm/xe/xe_guc_pc.c +5 −7
drivers/gpu/drm/xe/xe_guc_pc.h +4 −1
drivers/gpu/drm/xe/xe_guc_relay.c +14 −3
drivers/gpu/drm/xe/xe_guc_submit.c +162 −119
drivers/gpu/drm/xe/xe_guc_submit.h +2 −2
drivers/gpu/drm/xe/xe_guc_types.h +0 −9
drivers/gpu/drm/xe/xe_huc.c +2 −0
drivers/gpu/drm/xe/xe_huc.h +2 −1
drivers/gpu/drm/xe/xe_hw_engine.c +183 −10
drivers/gpu/drm/xe/xe_hw_engine.h +3 −0
drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c +6 −24
drivers/gpu/drm/xe/xe_hw_engine_types.h +22 −4
drivers/gpu/drm/xe/xe_hw_fence.c +47 −12
drivers/gpu/drm/xe/xe_hw_fence.h +5 −2
drivers/gpu/drm/xe/xe_hwmon.c +29 −8
drivers/gpu/drm/xe/xe_irq.c +3 −3
drivers/gpu/drm/xe/xe_lrc.c +264 −41
drivers/gpu/drm/xe/xe_lrc.h +53 −5
drivers/gpu/drm/xe/xe_lrc_types.h +12 −0
drivers/gpu/drm/xe/xe_migrate.c +53 −25
drivers/gpu/drm/xe/xe_mmio.c +99 −343
drivers/gpu/drm/xe/xe_mmio.h +11 −11
drivers/gpu/drm/xe/xe_mocs.c +260 −41
drivers/gpu/drm/xe/xe_mocs.h +8 −3
drivers/gpu/drm/xe/xe_module.c +5 −0
drivers/gpu/drm/xe/xe_module.h +1 −0
drivers/gpu/drm/xe/xe_pat.c +1 −0
drivers/gpu/drm/xe/xe_pci.c +80 −26
drivers/gpu/drm/xe/xe_pci_sriov.c +143 −0
drivers/gpu/drm/xe/xe_pci_sriov.h +13 −0
drivers/gpu/drm/xe/xe_pci_types.h +4 −0
drivers/gpu/drm/xe/xe_pcode.c +5 −1
drivers/gpu/drm/xe/xe_pm.c +22 −20
drivers/gpu/drm/xe/xe_pm.h +2 −7
drivers/gpu/drm/xe/xe_preempt_fence.c +11 −3
drivers/gpu/drm/xe/xe_pt.c +56 −13
drivers/gpu/drm/xe/xe_query.c +9 −1
drivers/gpu/drm/xe/xe_reg_sr.h +2 −2
drivers/gpu/drm/xe/xe_res_cursor.h +0 −2
drivers/gpu/drm/xe/xe_ring_ops.c +33 −19
drivers/gpu/drm/xe/xe_rtp.c +6 −0
drivers/gpu/drm/xe/xe_rtp.h +16 −2
drivers/gpu/drm/xe/xe_rtp_helpers.h +10 −16
drivers/gpu/drm/xe/xe_sched_job.c +100 −81
drivers/gpu/drm/xe/xe_sched_job.h +6 −1
drivers/gpu/drm/xe/xe_sched_job_types.h +18 −2
drivers/gpu/drm/xe/xe_sriov.c +11 −0
drivers/gpu/drm/xe/xe_sync.c +15 −0
drivers/gpu/drm/xe/xe_sync.h +1 −0
drivers/gpu/drm/xe/xe_tile_sysfs.c +2 −2
drivers/gpu/drm/xe/xe_trace.h +7 −4
drivers/gpu/drm/xe/xe_ttm_stolen_mgr.c +2 −0
drivers/gpu/drm/xe/xe_uc.c +42 −16
drivers/gpu/drm/xe/xe_uc.h +2 −2
drivers/gpu/drm/xe/xe_uc_fw.c +5 −0
drivers/gpu/drm/xe/xe_vm.c +495 −350
drivers/gpu/drm/xe/xe_vm.h +3 −2
drivers/gpu/drm/xe/xe_vm_doc.h +12 −12
drivers/gpu/drm/xe/xe_vm_types.h +24 −5
drivers/gpu/drm/xe/xe_vram.c +368 −0
drivers/gpu/drm/xe/xe_vram.h +13 −0
drivers/gpu/drm/xe/xe_vram_freq.c +2 −2
drivers/gpu/drm/xe/xe_wa.c +34 −5
include/uapi/drm/xe_drm.h +2 −0

Documentation/gpu/drm-usage-stats.rst+19−2

@@ -112,6 +112,19 @@ larger value within a reasonable period. Upon observing a value lower than what · 111 unmodified lines
112	112	was previously read, userspace is expected to stay with that larger previous
113	113	value until a monotonic update is seen.
114	114
	115	+- drm-total-cycles-<keystr>: <uint>
	116	+
	117	+Engine identifier string must be the same as the one specified in the
	118	+drm-cycles-<keystr> tag and shall contain the total number cycles for the given
	119	+engine.
	120	+
	121	+This is a timestamp in GPU unspecified unit that matches the update rate
	122	+of drm-cycles-<keystr>. For drivers that implement this interface, the engine
	123	+utilization can be calculated entirely on the GPU clock domain, without
	124	+considering the CPU sleep time between 2 samples.
	125	+
	126	+A driver may implement either this key or drm-maxfreq-<keystr>, but not both.
	127	+
115	128	- drm-maxfreq-<keystr>: <uint> [Hz\|MHz\|KHz]
116	129
117	130	Engine identifier string must be the same as the one specified in the
@@ -121,6 +134,9 @@ percentage utilization of the engine, whereas drm-engine-<keystr> only reflects · 3 unmodified lines
121	134	time active without considering what frequency the engine is operating as a
122	135	percentage of its maximum frequency.
123	136
	137	+A driver may implement either this key or drm-total-cycles-<keystr>, but not
	138	+both.
	139	+
124	140	Memory
125	141	^^^^^^
126	142
@@ -168,5 +184,6 @@ be documented above and where possible, aligned with other drivers. · 41 unmodified lines
168	184	Driver specific implementations
169	185	-------------------------------
170	186
171		−:ref:`i915-usage-stats`
172		−:ref:`panfrost-usage-stats`
	187	+* :ref:`i915-usage-stats`
	188	+* :ref:`panfrost-usage-stats`
	189	+* :ref:`xe-usage-stats`

Documentation/gpu/xe/index.rst+1−0

@@ -23,3 +23,4 @@ DG2, etc is provided to prototype the driver. · 22 unmodified lines
23	23	xe_firmware
24	24	xe_tile
25	25	xe_debugging
	26	+ xe-drm-usage-stats.rst

Documentation/gpu/xe/xe-drm-usage-stats.rst+10−0

@@ -0,0 +1,10 @@
	1	+.. SPDX-License-Identifier: GPL-2.0+
	2	+
	3	+.. _xe-usage-stats:
	4	+
	5	+========================================
	6	+Xe DRM client usage stats implementation
	7	+========================================
	8	+
	9	+.. kernel-doc:: drivers/gpu/drm/xe/xe_drm_client.c
	10	+ :doc: DRM Client usage stats

MAINTAINERS+0−1

@@ -11034,7 +11034,6 @@ F: include/uapi/drm/i915_drm.h · 11033 unmodified lines
11034	11034
11035	11035	INTEL DRM XE DRIVER (Lunar Lake and newer)
11036	11036	M: Lucas De Marchi <lucas.demarchi@intel.com>
11037		−M: Oded Gabbay <ogabbay@kernel.org>
11038	11037	M: Thomas Hellström <thomas.hellstrom@linux.intel.com>
11039	11038	L: intel-xe@lists.freedesktop.org
11040	11039	S: Supported

drivers/gpu/drm/i915/display/intel_vga.c+1−0

@@ -3,6 +3,7 @@ · 2 unmodified lines
3	3	* Copyright © 2019 Intel Corporation
4	4	*/
5	5
	6	+#include <linux/delay.h>
6	7	#include <linux/vgaarb.h>
7	8
8	9	#include <video/vga.h>

drivers/gpu/drm/xe/Kconfig.debug+0−10

@@ -61,16 +61,6 @@ config DRM_XE_DEBUG_MEM · 60 unmodified lines
61	61
62	62	If in doubt, say "N".
63	63
64		−config DRM_XE_SIMPLE_ERROR_CAPTURE
65		− bool "Enable simple error capture to dmesg on job timeout"
66		− default n
67		− help
68		− Choose this option when debugging an unexpected job timeout
69		−
70		− Recommended for driver developers only.
71		−
72		− If in doubt, say "N".
73		−
74	64	config DRM_XE_KUNIT_TEST
75	65	tristate "KUnit tests for the drm xe driver" if !KUNIT_ALL_TESTS
76	66	depends on DRM_XE && KUNIT && DEBUG_FS

drivers/gpu/drm/xe/Makefile+9−25

@@ -3,31 +3,8 @@ · 2 unmodified lines
3	3	# Makefile for the drm device driver. This driver provides support for the
4	4	# Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher.
5	5
6		−# Unconditionally enable W=1 warnings locally
7		−# --- begin copy-paste W=1 warnings from scripts/Makefile.extrawarn
8		−subdir-ccflags-y += -Wextra -Wunused -Wno-unused-parameter
9		−subdir-ccflags-y += -Wmissing-declarations
10		−subdir-ccflags-y += $(call cc-option, -Wrestrict)
11		−subdir-ccflags-y += -Wmissing-format-attribute
12		−subdir-ccflags-y += -Wmissing-prototypes
13		−subdir-ccflags-y += -Wold-style-definition
14		−subdir-ccflags-y += -Wmissing-include-dirs
15		−subdir-ccflags-y += $(call cc-option, -Wunused-but-set-variable)
16		−subdir-ccflags-y += $(call cc-option, -Wunused-const-variable)
17		−subdir-ccflags-y += $(call cc-option, -Wpacked-not-aligned)
18		−subdir-ccflags-y += $(call cc-option, -Wformat-overflow)
	6	+# Enable W=1 warnings not enabled in drm subsystem Makefile
19	7	subdir-ccflags-y += $(call cc-option, -Wformat-truncation)
20		−subdir-ccflags-y += $(call cc-option, -Wstringop-truncation)
21		−# The following turn off the warnings enabled by -Wextra
22		−ifeq ($(findstring 2, $(KBUILD_EXTRA_WARN)),)
23		−subdir-ccflags-y += -Wno-missing-field-initializers
24		−subdir-ccflags-y += -Wno-type-limits
25		−subdir-ccflags-y += -Wno-shift-negative-value
26		−endif
27		−ifeq ($(findstring 3, $(KBUILD_EXTRA_WARN)),)
28		−subdir-ccflags-y += -Wno-sign-compare
29		−endif
30		−# --- end copy-paste
31	8
32	9	# Enable -Werror in CI and development
33	10	subdir-ccflags-$(CONFIG_DRM_XE_WERROR) += -Werror
@@ -89,7 +66,7 @@ xe-y += xe_bb.o \ · 55 unmodified lines
89	66	xe_gt_mcr.o \
90	67	xe_gt_pagefault.o \
91	68	xe_gt_sysfs.o \
92		− xe_gt_throttle_sysfs.o \
	69	+ xe_gt_throttle.o \
93	70	xe_gt_tlb_invalidation.o \
94	71	xe_gt_topology.o \
95	72	xe_guc.o \
@@ -143,6 +120,7 @@ xe-y += xe_bb.o \ · 47 unmodified lines
143	120	xe_uc_debugfs.o \
144	121	xe_uc_fw.o \
145	122	xe_vm.o \
	123	+ xe_vram.o \
146	124	xe_vram_freq.o \
147	125	xe_wait_user_fence.o \
148	126	xe_wa.o \
@@ -155,6 +133,8 @@ xe-$(CONFIG_HWMON) += xe_hwmon.o · 6 unmodified lines
155	133
156	134	# graphics virtualization (SR-IOV) support
157	135	xe-y += \
	136	+ xe_gt_sriov_vf.o \
	137	+ xe_gt_sriov_vf_debugfs.o \
158	138	xe_guc_relay.o \
159	139	xe_memirq.o \
160	140	xe_sriov.o
@@ -163,10 +143,14 @@ xe-$(CONFIG_PCI_IOV) += \ · 2 unmodified lines
163	143	xe_gt_sriov_pf.o \
164	144	xe_gt_sriov_pf_config.o \
165	145	xe_gt_sriov_pf_control.o \
	146	+ xe_gt_sriov_pf_debugfs.o \
	147	+ xe_gt_sriov_pf_monitor.o \
166	148	xe_gt_sriov_pf_policy.o \
	149	+ xe_gt_sriov_pf_service.o \
167	150	xe_lmtt.o \
168	151	xe_lmtt_2l.o \
169	152	xe_lmtt_ml.o \
	153	+ xe_pci_sriov.o \
170	154	xe_sriov_pf.o
171	155
172	156	# include helpers for tests even when XE is built-in

drivers/gpu/drm/xe/abi/guc_actions_sriov_abi.h+190−0

@@ -171,6 +171,36 @@ · 170 unmodified lines
171	171	#define VF2GUC_RELAY_TO_PF_REQUEST_MSG_n_RELAY_DATAx GUC_HXG_REQUEST_MSG_n_DATAn
172	172	#define VF2GUC_RELAY_TO_PF_REQUEST_MSG_NUM_RELAY_DATA GUC_RELAY_MSG_MAX_LEN
173	173
	174	+/**
	175	+ * DOC: GUC2PF_ADVERSE_EVENT
	176	+ *
	177	+ * This message is used by the GuC to notify PF about adverse events.
	178	+ *
	179	+ * This G2H message must be sent as `CTB HXG Message`_.
	180	+ *
	181	+ * +---+-------+--------------------------------------------------------------+
	182	+ * \| \| Bits \| Description \|
	183	+ * +===+=======+==============================================================+
	184	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_GUC_ \|
	185	+ * \| +-------+--------------------------------------------------------------+
	186	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_EVENT_ \|
	187	+ * \| +-------+--------------------------------------------------------------+
	188	+ * \| \| 27:16 \| DATA0 = MBZ \|
	189	+ * \| +-------+--------------------------------------------------------------+
	190	+ * \| \| 15:0 \| ACTION = _`GUC_ACTION_GUC2PF_ADVERSE_EVENT` = 0x5104 \|
	191	+ * +---+-------+--------------------------------------------------------------+
	192	+ * \| 1 \| 31:0 \| DATA1 = VFID - VF identifier \|
	193	+ * +---+-------+--------------------------------------------------------------+
	194	+ * \| 2 \| 31:0 \| DATA2 = THRESHOLD - key of the exceeded threshold \|
	195	+ * +---+-------+--------------------------------------------------------------+
	196	+ */
	197	+#define GUC_ACTION_GUC2PF_ADVERSE_EVENT 0x5104
	198	+
	199	+#define GUC2PF_ADVERSE_EVENT_EVENT_MSG_LEN (GUC_HXG_EVENT_MSG_MIN_LEN + 2u)
	200	+#define GUC2PF_ADVERSE_EVENT_EVENT_MSG_0_MBZ GUC_HXG_EVENT_MSG_0_DATA0
	201	+#define GUC2PF_ADVERSE_EVENT_EVENT_MSG_1_VFID GUC_HXG_EVENT_MSG_n_DATAn
	202	+#define GUC2PF_ADVERSE_EVENT_EVENT_MSG_2_THRESHOLD GUC_HXG_EVENT_MSG_n_DATAn
	203	+
174	204	/**
175	205	* DOC: GUC2PF_VF_STATE_NOTIFY
176	206	*
@@ -213,6 +243,73 @@ · 36 unmodified lines
213	243	#define GUC_PF_NOTIFY_VF_PAUSE_DONE 3u
214	244	#define GUC_PF_NOTIFY_VF_FIXUP_DONE 4u
215	245
	246	+/**
	247	+ * DOC: VF2GUC_MATCH_VERSION
	248	+ *
	249	+ * This action is used to match VF interface version used by VF and GuC.
	250	+ *
	251	+ * This message must be sent as `MMIO HXG Message`_.
	252	+ *
	253	+ * +---+-------+--------------------------------------------------------------+
	254	+ * \| \| Bits \| Description \|
	255	+ * +===+=======+==============================================================+
	256	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
	257	+ * \| +-------+--------------------------------------------------------------+
	258	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
	259	+ * \| +-------+--------------------------------------------------------------+
	260	+ * \| \| 27:16 \| DATA0 = MBZ \|
	261	+ * \| +-------+--------------------------------------------------------------+
	262	+ * \| \| 15:0 \| ACTION = _`GUC_ACTION_VF2GUC_MATCH_VERSION` = 0x5500 \|
	263	+ * +---+-------+--------------------------------------------------------------+
	264	+ * \| 1 \| 31:24 \| BRANCH - branch ID of the VF interface \|
	265	+ * \| \| \| (use BRANCH_ANY to request latest version supported by GuC) \|
	266	+ * \| +-------+--------------------------------------------------------------+
	267	+ * \| \| 23:16 \| MAJOR - major version of the VF interface \|
	268	+ * \| \| \| (use MAJOR_ANY to request latest version supported by GuC) \|
	269	+ * \| +-------+--------------------------------------------------------------+
	270	+ * \| \| 15:8 \| MINOR - minor version of the VF interface \|
	271	+ * \| \| \| (use MINOR_ANY to request latest version supported by GuC) \|
	272	+ * \| +-------+--------------------------------------------------------------+
	273	+ * \| \| 7:0 \| MBZ \|
	274	+ * +---+-------+--------------------------------------------------------------+
	275	+ *
	276	+ * +---+-------+--------------------------------------------------------------+
	277	+ * \| \| Bits \| Description \|
	278	+ * +===+=======+==============================================================+
	279	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_GUC_ \|
	280	+ * \| +-------+--------------------------------------------------------------+
	281	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
	282	+ * \| +-------+--------------------------------------------------------------+
	283	+ * \| \| 27:0 \| DATA0 = MBZ \|
	284	+ * +---+-------+--------------------------------------------------------------+
	285	+ * \| 1 \| 31:24 \| BRANCH - branch ID of the VF interface \|
	286	+ * \| +-------+--------------------------------------------------------------+
	287	+ * \| \| 23:16 \| MAJOR - major version of the VF interface \|
	288	+ * \| +-------+--------------------------------------------------------------+
	289	+ * \| \| 15:8 \| MINOR - minor version of the VF interface \|
	290	+ * \| +-------+--------------------------------------------------------------+
	291	+ * \| \| 7:0 \| PATCH - patch version of the VF interface \|
	292	+ * +---+-------+--------------------------------------------------------------+
	293	+ */
	294	+#define GUC_ACTION_VF2GUC_MATCH_VERSION 0x5500u
	295	+
	296	+#define VF2GUC_MATCH_VERSION_REQUEST_MSG_LEN (GUC_HXG_REQUEST_MSG_MIN_LEN + 1u)
	297	+#define VF2GUC_MATCH_VERSION_REQUEST_MSG_0_MBZ GUC_HXG_REQUEST_MSG_0_DATA0
	298	+#define VF2GUC_MATCH_VERSION_REQUEST_MSG_1_BRANCH (0xffu << 24)
	299	+#define GUC_VERSION_BRANCH_ANY 0
	300	+#define VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MAJOR (0xffu << 16)
	301	+#define GUC_VERSION_MAJOR_ANY 0
	302	+#define VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MINOR (0xffu << 8)
	303	+#define GUC_VERSION_MINOR_ANY 0
	304	+#define VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MBZ (0xffu << 0)
	305	+
	306	+#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_LEN (GUC_HXG_RESPONSE_MSG_MIN_LEN + 1u)
	307	+#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_0_MBZ GUC_HXG_RESPONSE_MSG_0_DATA0
	308	+#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_BRANCH (0xffu << 24)
	309	+#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_MAJOR (0xffu << 16)
	310	+#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_MINOR (0xffu << 8)
	311	+#define VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_PATCH (0xffu << 0)
	312	+
216	313	/**
217	314	* DOC: PF2GUC_UPDATE_VGT_POLICY
218	315	*
@@ -367,4 +464,97 @@ · 148 unmodified lines
367	464	#define GUC_PF_TRIGGER_VF_FLR_START 4u
368	465	#define GUC_PF_TRIGGER_VF_FLR_FINISH 5u
369	466
	467	+/**
	468	+ * DOC: VF2GUC_VF_RESET
	469	+ *
	470	+ * This action is used by VF to reset GuC's VF state.
	471	+ *
	472	+ * This message must be sent as `MMIO HXG Message`_.
	473	+ *
	474	+ * +---+-------+--------------------------------------------------------------+
	475	+ * \| \| Bits \| Description \|
	476	+ * +===+=======+==============================================================+
	477	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
	478	+ * \| +-------+--------------------------------------------------------------+
	479	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
	480	+ * \| +-------+--------------------------------------------------------------+
	481	+ * \| \| 27:16 \| DATA0 = MBZ \|
	482	+ * \| +-------+--------------------------------------------------------------+
	483	+ * \| \| 15:0 \| ACTION = _`GUC_ACTION_VF2GUC_VF_RESET` = 0x5507 \|
	484	+ * +---+-------+--------------------------------------------------------------+
	485	+ *
	486	+ * +---+-------+--------------------------------------------------------------+
	487	+ * \| \| Bits \| Description \|
	488	+ * +===+=======+==============================================================+
	489	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_GUC_ \|
	490	+ * \| +-------+--------------------------------------------------------------+
	491	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
	492	+ * \| +-------+--------------------------------------------------------------+
	493	+ * \| \| 27:0 \| DATA0 = MBZ \|
	494	+ * +---+-------+--------------------------------------------------------------+
	495	+ */
	496	+#define GUC_ACTION_VF2GUC_VF_RESET 0x5507u
	497	+
	498	+#define VF2GUC_VF_RESET_REQUEST_MSG_LEN GUC_HXG_REQUEST_MSG_MIN_LEN
	499	+#define VF2GUC_VF_RESET_REQUEST_MSG_0_MBZ GUC_HXG_REQUEST_MSG_0_DATA0
	500	+
	501	+#define VF2GUC_VF_RESET_RESPONSE_MSG_LEN GUC_HXG_RESPONSE_MSG_MIN_LEN
	502	+#define VF2GUC_VF_RESET_RESPONSE_MSG_0_MBZ GUC_HXG_RESPONSE_MSG_0_DATA0
	503	+
	504	+/**
	505	+ * DOC: VF2GUC_QUERY_SINGLE_KLV
	506	+ *
	507	+ * This action is used by VF to query value of the single KLV data.
	508	+ *
	509	+ * This message must be sent as `MMIO HXG Message`_.
	510	+ *
	511	+ * +---+-------+--------------------------------------------------------------+
	512	+ * \| \| Bits \| Description \|
	513	+ * +===+=======+==============================================================+
	514	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
	515	+ * \| +-------+--------------------------------------------------------------+
	516	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
	517	+ * \| +-------+--------------------------------------------------------------+
	518	+ * \| \| 27:16 \| MBZ \|
	519	+ * \| +-------+--------------------------------------------------------------+
	520	+ * \| \| 15:0 \| ACTION = _`GUC_ACTION_VF2GUC_QUERY_SINGLE_KLV` = 0x5509 \|
	521	+ * +---+-------+--------------------------------------------------------------+
	522	+ * \| 1 \| 31:16 \| MBZ \|
	523	+ * \| +-------+--------------------------------------------------------------+
	524	+ * \| \| 15:0 \| KEY - key for which value is requested \|
	525	+ * +---+-------+--------------------------------------------------------------+
	526	+ *
	527	+ * +---+-------+--------------------------------------------------------------+
	528	+ * \| \| Bits \| Description \|
	529	+ * +===+=======+==============================================================+
	530	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_GUC_ \|
	531	+ * \| +-------+--------------------------------------------------------------+
	532	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
	533	+ * \| +-------+--------------------------------------------------------------+
	534	+ * \| \| 27:16 \| MBZ \|
	535	+ * \| +-------+--------------------------------------------------------------+
	536	+ * \| \| 15:0 \| LENGTH - length of data in dwords \|
	537	+ * +---+-------+--------------------------------------------------------------+
	538	+ * \| 1 \| 31:0 \| VALUE32 - bits 31:0 of value if LENGTH >= 1 \|
	539	+ * +---+-------+--------------------------------------------------------------+
	540	+ * \| 2 \| 31:0 \| VALUE64 - bits 63:32 of value if LENGTH >= 2 \|
	541	+ * +---+-------+--------------------------------------------------------------+
	542	+ * \| 3 \| 31:0 \| VALUE96 - bits 95:64 of value if LENGTH >= 3 \|
	543	+ * +---+-------+--------------------------------------------------------------+
	544	+ */
	545	+#define GUC_ACTION_VF2GUC_QUERY_SINGLE_KLV 0x5509u
	546	+
	547	+#define VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_LEN (GUC_HXG_REQUEST_MSG_MIN_LEN + 1u)
	548	+#define VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_0_MBZ GUC_HXG_REQUEST_MSG_0_DATA0
	549	+#define VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_1_MBZ (0xffffu << 16)
	550	+#define VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_1_KEY (0xffffu << 0)
	551	+
	552	+#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_MIN_LEN GUC_HXG_RESPONSE_MSG_MIN_LEN
	553	+#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_MAX_LEN (GUC_HXG_RESPONSE_MSG_MIN_LEN + 3u)
	554	+#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_0_MBZ (0xfffu << 16)
	555	+#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_0_LENGTH (0xffffu << 0)
	556	+#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_1_VALUE32 GUC_HXG_REQUEST_MSG_n_DATAn
	557	+#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_2_VALUE64 GUC_HXG_REQUEST_MSG_n_DATAn
	558	+#define VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_3_VALUE96 GUC_HXG_REQUEST_MSG_n_DATAn
	559	+
370	560	#endif

drivers/gpu/drm/xe/abi/guc_errors_abi.h+24−2

@@ -7,8 +7,12 @@ · 6 unmodified lines
7	7	#define _ABI_GUC_ERRORS_ABI_H
8	8
9	9	enum xe_guc_response_status {
10		− XE_GUC_RESPONSE_STATUS_SUCCESS = 0x0,
11		− XE_GUC_RESPONSE_STATUS_GENERIC_FAIL = 0xF000,
	10	+ XE_GUC_RESPONSE_STATUS_SUCCESS = 0x0,
	11	+ XE_GUC_RESPONSE_NOT_SUPPORTED = 0x20,
	12	+ XE_GUC_RESPONSE_NO_ATTRIBUTE_TABLE = 0x201,
	13	+ XE_GUC_RESPONSE_NO_DECRYPTION_KEY = 0x202,
	14	+ XE_GUC_RESPONSE_DECRYPTION_FAILED = 0x204,
	15	+ XE_GUC_RESPONSE_STATUS_GENERIC_FAIL = 0xF000,
12	16	};
13	17
14	18	enum xe_guc_load_status {
@@ -17,6 +21,9 @@ enum xe_guc_load_status { · 2 unmodified lines
17	21	XE_GUC_LOAD_STATUS_ERROR_DEVID_BUILD_MISMATCH = 0x02,
18	22	XE_GUC_LOAD_STATUS_GUC_PREPROD_BUILD_MISMATCH = 0x03,
19	23	XE_GUC_LOAD_STATUS_ERROR_DEVID_INVALID_GUCTYPE = 0x04,
	24	+ XE_GUC_LOAD_STATUS_HWCONFIG_START = 0x05,
	25	+ XE_GUC_LOAD_STATUS_HWCONFIG_DONE = 0x06,
	26	+ XE_GUC_LOAD_STATUS_HWCONFIG_ERROR = 0x07,
20	27	XE_GUC_LOAD_STATUS_GDT_DONE = 0x10,
21	28	XE_GUC_LOAD_STATUS_IDT_DONE = 0x20,
22	29	XE_GUC_LOAD_STATUS_LAPIC_DONE = 0x30,
@@ -34,4 +41,19 @@ enum xe_guc_load_status { · 11 unmodified lines
34	41	XE_GUC_LOAD_STATUS_READY = 0xF0,
35	42	};
36	43
	44	+enum xe_bootrom_load_status {
	45	+ XE_BOOTROM_STATUS_NO_KEY_FOUND = 0x13,
	46	+ XE_BOOTROM_STATUS_AES_PROD_KEY_FOUND = 0x1A,
	47	+ XE_BOOTROM_STATUS_PROD_KEY_CHECK_FAILURE = 0x2B,
	48	+ XE_BOOTROM_STATUS_RSA_FAILED = 0x50,
	49	+ XE_BOOTROM_STATUS_PAVPC_FAILED = 0x73,
	50	+ XE_BOOTROM_STATUS_WOPCM_FAILED = 0x74,
	51	+ XE_BOOTROM_STATUS_LOADLOC_FAILED = 0x75,
	52	+ XE_BOOTROM_STATUS_JUMP_PASSED = 0x76,
	53	+ XE_BOOTROM_STATUS_JUMP_FAILED = 0x77,
	54	+ XE_BOOTROM_STATUS_RC6CTXCONFIG_FAILED = 0x79,
	55	+ XE_BOOTROM_STATUS_MPUMAP_INCORRECT = 0x7A,
	56	+ XE_BOOTROM_STATUS_EXCEPTION = 0x7E,
	57	+};
	58	+
37	59	#endif

drivers/gpu/drm/xe/abi/guc_klvs_abi.h+32−8

@@ -35,6 +35,20 @@ · 34 unmodified lines
35	35	#define GUC_KLV_0_LEN (0xffffu << 0)
36	36	#define GUC_KLV_n_VALUE (0xffffffffu << 0)
37	37
	38	+/**
	39	+ * DOC: GuC Global Config KLVs
	40	+ *
	41	+ * `GuC KLV`_ keys available for use with HOST2GUC_SELF_CFG_.
	42	+ *
	43	+ * _`GUC_KLV_GLOBAL_CFG_GMD_ID` : 0x3000
	44	+ * Refers to 32 bit architecture version as reported by the HW IP.
	45	+ * This key is supported on MTL+ platforms only.
	46	+ * Requires GuC ABI 1.2+.
	47	+ */
	48	+
	49	+#define GUC_KLV_GLOBAL_CFG_GMD_ID_KEY 0x3000u
	50	+#define GUC_KLV_GLOBAL_CFG_GMD_ID_LEN 1u
	51	+
38	52	/**
39	53	* DOC: GuC Self Config KLVs
40	54	*
@@ -194,14 +208,18 @@ enum { · 153 unmodified lines
194	208	* granularity) since the GPUs clock time runs off a different crystal
195	209	* from the CPUs clock. Changing this KLV on a VF that is currently
196	210	* running a context wont take effect until a new context is scheduled in.
197		− * That said, when the PF is changing this value from 0xFFFFFFFF to
198		− * something else, it might never take effect if the VF is running an
199		− * inifinitely long compute or shader kernel. In such a scenario, the
	211	+ * That said, when the PF is changing this value from 0x0 to
	212	+ * a non-zero value, it might never take effect if the VF is running an
	213	+ * infinitely long compute or shader kernel. In such a scenario, the
200	214	* PF would need to trigger a VM PAUSE and then change the KLV to force
201	215	* it to take effect. Such cases might typically happen on a 1PF+1VF
202	216	* Virtualization config enabled for heavier workloads like AI/ML.
203	217	*
	218	+ * The max value for this KLV is 100 seconds, anything exceeding that
	219	+ * will be clamped to the max.
	220	+ *
204	221	* :0: infinite exec quantum (default)
	222	+ * :100000: maximum exec quantum (100000ms == 100s)
205	223	*
206	224	* _`GUC_KLV_VF_CFG_PREEMPT_TIMEOUT` : 0x8A02
207	225	* This config sets the VF-preemption-timeout in microseconds.
@@ -211,15 +229,19 @@ enum { · 3 unmodified lines
211	229	* different crystal from the CPUs clock. Changing this KLV on a VF
212	230	* that is currently running a context wont take effect until a new
213	231	* context is scheduled in.
214		− * That said, when the PF is changing this value from 0xFFFFFFFF to
215		− * something else, it might never take effect if the VF is running an
216		− * inifinitely long compute or shader kernel.
	232	+ * That said, when the PF is changing this value from 0x0 to
	233	+ * a non-zero value, it might never take effect if the VF is running an
	234	+ * infinitely long compute or shader kernel.
217	235	* In this case, the PF would need to trigger a VM PAUSE and then change
218	236	* the KLV to force it to take effect. Such cases might typically happen
219	237	* on a 1PF+1VF Virtualization config enabled for heavier workloads like
220	238	* AI/ML.
221	239	*
	240	+ * The max value for this KLV is 100 seconds, anything exceeding that
	241	+ * will be clamped to the max.
	242	+ *
222	243	* :0: no preemption timeout (default)
	244	+ * :100000000: maximum preemption timeout (100000000us == 100s)
223	245	*
224	246	* _`GUC_KLV_VF_CFG_THRESHOLD_CAT_ERR` : 0x8A03
225	247	* This config sets threshold for CAT errors caused by the VF.
@@ -291,9 +313,11 @@ enum { · 65 unmodified lines
291	313
292	314	#define GUC_KLV_VF_CFG_EXEC_QUANTUM_KEY 0x8a01
293	315	#define GUC_KLV_VF_CFG_EXEC_QUANTUM_LEN 1u
	316	+#define GUC_KLV_VF_CFG_EXEC_QUANTUM_MAX_VALUE 100000u
294	317
295		−#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_KEY 0x8a02
296		−#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_LEN 1u
	318	+#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_KEY 0x8a02
	319	+#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_LEN 1u
	320	+#define GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_MAX_VALUE 100000000u
297	321
298	322	#define GUC_KLV_VF_CFG_THRESHOLD_CAT_ERR_KEY 0x8a03
299	323	#define GUC_KLV_VF_CFG_THRESHOLD_CAT_ERR_LEN 1u

drivers/gpu/drm/xe/abi/guc_relay_actions_abi.h+169−1

@@ -1,11 +1,179 @@
1	1	/* SPDX-License-Identifier: MIT */
2	2	/*
3		− * Copyright © 2023 Intel Corporation
	3	+ * Copyright © 2023-2024 Intel Corporation
4	4	*/
5	5
6	6	#ifndef _ABI_GUC_RELAY_ACTIONS_ABI_H_
7	7	#define _ABI_GUC_RELAY_ACTIONS_ABI_H_
8	8
	9	+#include "abi/guc_relay_communication_abi.h"
	10	+
	11	+/**
	12	+ * DOC: GuC Relay VF/PF ABI Version
	13	+ *
	14	+ * The _`GUC_RELAY_VERSION_BASE` defines minimum VF/PF ABI version that
	15	+ * drivers must support. Currently this is version 1.0.
	16	+ *
	17	+ * The _`GUC_RELAY_VERSION_LATEST` defines latest VF/PF ABI version that
	18	+ * drivers may use. Currently this is version 1.0.
	19	+ *
	20	+ * Some platforms may require different base VF/PF ABI version.
	21	+ * No supported VF/PF ABI version can be 0.0.
	22	+ */
	23	+
	24	+#define GUC_RELAY_VERSION_BASE_MAJOR 1
	25	+#define GUC_RELAY_VERSION_BASE_MINOR 0
	26	+
	27	+#define GUC_RELAY_VERSION_LATEST_MAJOR 1
	28	+#define GUC_RELAY_VERSION_LATEST_MINOR 0
	29	+
	30	+/**
	31	+ * DOC: GuC Relay Actions
	32	+ *
	33	+ * The following actions are supported from VF/PF ABI version 1.0:
	34	+ *
	35	+ * * `VF2PF_HANDSHAKE`_
	36	+ * * `VF2PF_QUERY_RUNTIME`_
	37	+ */
	38	+
	39	+/**
	40	+ * DOC: VF2PF_HANDSHAKE
	41	+ *
	42	+ * This `Relay Message`_ is used by the VF to establish ABI version with the PF.
	43	+ *
	44	+ * Prior to exchanging any other messages, both VF driver and PF driver must
	45	+ * negotiate the VF/PF ABI version that will be used in their communication.
	46	+ *
	47	+ * The VF driver shall use @MAJOR and @MINOR fields to pass requested ABI version.
	48	+ * The VF driver may use special version 0.0 (both @MAJOR and @MINOR set to 0)
	49	+ * to request latest (or any) ABI version that is supported by the PF driver.
	50	+ *
	51	+ * This message definition shall be supported by all future ABI versions.
	52	+ * This message definition shall not be changed by future ABI versions.
	53	+ *
	54	+ * +---+-------+--------------------------------------------------------------+
	55	+ * \| \| Bits \| Description \|
	56	+ * +===+=======+==============================================================+
	57	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
	58	+ * \| +-------+--------------------------------------------------------------+
	59	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
	60	+ * \| +-------+--------------------------------------------------------------+
	61	+ * \| \| 27:16 \| DATA0 = MBZ \|
	62	+ * \| +-------+--------------------------------------------------------------+
	63	+ * \| \| 15:0 \| ACTION = _`GUC_RELAY_ACTION_VF2PF_HANDSHAKE` = 0x0001 \|
	64	+ * +---+-------+--------------------------------------------------------------+
	65	+ * \| 1 \| 31:16 \| MAJOR - requested major version of the VFPF interface \|
	66	+ * \| \| \| (use MAJOR_ANY to request latest version supported by PF) \|
	67	+ * \| +-------+--------------------------------------------------------------+
	68	+ * \| \| 15:0 \| MINOR - requested minor version of the VFPF interface \|
	69	+ * \| \| \| (use MINOR_ANY to request latest version supported by PF) \|
	70	+ * +---+-------+--------------------------------------------------------------+
	71	+ *
	72	+ * +---+-------+--------------------------------------------------------------+
	73	+ * \| \| Bits \| Description \|
	74	+ * +===+=======+==============================================================+
	75	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
	76	+ * \| +-------+--------------------------------------------------------------+
	77	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
	78	+ * \| +-------+--------------------------------------------------------------+
	79	+ * \| \| 27:0 \| DATA0 = MBZ \|
	80	+ * +---+-------+--------------------------------------------------------------+
	81	+ * \| 1 \| 31:16 \| MAJOR - agreed major version of the VFPF interface \|
	82	+ * \| +-------+--------------------------------------------------------------+
	83	+ * \| \| 15:0 \| MINOR - agreed minor version of the VFPF interface \|
	84	+ * +---+-------+--------------------------------------------------------------+
	85	+ */
	86	+#define GUC_RELAY_ACTION_VF2PF_HANDSHAKE 0x0001u
	87	+
	88	+#define VF2PF_HANDSHAKE_REQUEST_MSG_LEN 2u
	89	+#define VF2PF_HANDSHAKE_REQUEST_MSG_0_MBZ GUC_HXG_REQUEST_MSG_0_DATA0
	90	+#define VF2PF_HANDSHAKE_REQUEST_MSG_1_MAJOR (0xffffu << 16)
	91	+#define VF2PF_HANDSHAKE_MAJOR_ANY 0
	92	+#define VF2PF_HANDSHAKE_REQUEST_MSG_1_MINOR (0xffffu << 0)
	93	+#define VF2PF_HANDSHAKE_MINOR_ANY 0
	94	+
	95	+#define VF2PF_HANDSHAKE_RESPONSE_MSG_LEN 2u
	96	+#define VF2PF_HANDSHAKE_RESPONSE_MSG_0_MBZ GUC_HXG_RESPONSE_MSG_0_DATA0
	97	+#define VF2PF_HANDSHAKE_RESPONSE_MSG_1_MAJOR (0xffffu << 16)
	98	+#define VF2PF_HANDSHAKE_RESPONSE_MSG_1_MINOR (0xffffu << 0)
	99	+
	100	+/**
	101	+ * DOC: VF2PF_QUERY_RUNTIME
	102	+ *
	103	+ * This `Relay Message`_ is used by the VF to query values of runtime registers.
	104	+ *
	105	+ * On some platforms, VF drivers may not have access to the some fuse registers
	106	+ * (referred here as 'runtime registers') and therefore VF drivers need to ask
	107	+ * the PF driver to obtain their values.
	108	+ *
	109	+ * However, the list of such registers, and their values, is fully owned and
	110	+ * maintained by the PF driver and the VF driver may only initiate the query
	111	+ * sequence and indicate in the @START field the starting index of the next
	112	+ * requested register from this predefined list.
	113	+ *
	114	+ * In the response, the PF driver will return tuple of 32-bit register offset and
	115	+ * the 32-bit value of that register (respectively @REG_OFFSET and @REG_VALUE).
	116	+ *
	117	+ * The VF driver can use @LIMIT field to limit number of returned register tuples.
	118	+ * If @LIMIT is unset then PF decides about number of returned register tuples.
	119	+ *
	120	+ * This message definition is supported from ABI version 1.0.
	121	+ *
	122	+ * +---+-------+--------------------------------------------------------------+
	123	+ * \| \| Bits \| Description \|
	124	+ * +===+=======+==============================================================+
	125	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
	126	+ * \| +-------+--------------------------------------------------------------+
	127	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_REQUEST_ \|
	128	+ * \| +-------+--------------------------------------------------------------+
	129	+ * \| \| 27:16 \| DATA0 = LIMIT - limit number of returned entries \|
	130	+ * \| \| \| (use zero to not enforce any limits on the response) \|
	131	+ * \| +-------+--------------------------------------------------------------+
	132	+ * \| \| 15:0 \| ACTION = _`GUC_RELAY_ACTION_VF2PF_QUERY_RUNTIME` = 0x0101 \|
	133	+ * +---+-------+--------------------------------------------------------------+
	134	+ * \| 1 \| 31:0 \| DATA1 = START - index of the first requested entry \|
	135	+ * +---+-------+--------------------------------------------------------------+
	136	+ *
	137	+ * +---+-------+--------------------------------------------------------------+
	138	+ * \| \| Bits \| Description \|
	139	+ * +===+=======+==============================================================+
	140	+ * \| 0 \| 31 \| ORIGIN = GUC_HXG_ORIGIN_HOST_ \|
	141	+ * \| +-------+--------------------------------------------------------------+
	142	+ * \| \| 30:28 \| TYPE = GUC_HXG_TYPE_RESPONSE_SUCCESS_ \|
	143	+ * \| +-------+--------------------------------------------------------------+
	144	+ * \| \| 27:0 \| DATA0 = COUNT - number of entries included in response \|
	145	+ * +---+-------+--------------------------------------------------------------+
	146	+ * \| 1 \| 31:0 \| DATA1 = REMAINING - number of remaining entries \|
	147	+ * +---+-------+--------------------------------------------------------------+
	148	+ * \| 2 \| 31:0 \| DATA2 = REG_OFFSET - offset of register[START] \|
	149	+ * +---+-------+--------------------------------------------------------------+
	150	+ * \| 3 \| 31:0 \| DATA3 = REG_VALUE - value of register[START] \|
	151	+ * +---+-------+--------------------------------------------------------------+
	152	+ * \| \| \| \|
	153	+ * +---+-------+--------------------------------------------------------------+
	154	+ * \|n-1\| 31:0 \| REG_OFFSET - offset of register[START + x] \|
	155	+ * +---+-------+--------------------------------------------------------------+
	156	+ * \| n \| 31:0 \| REG_VALUE - value of register[START + x] \|
	157	+ * +---+-------+--------------------------------------------------------------+
	158	+ */
	159	+#define GUC_RELAY_ACTION_VF2PF_QUERY_RUNTIME 0x0101u
	160	+
	161	+#define VF2PF_QUERY_RUNTIME_REQUEST_MSG_LEN 2u
	162	+#define VF2PF_QUERY_RUNTIME_REQUEST_MSG_0_LIMIT GUC_HXG_REQUEST_MSG_0_DATA0
	163	+#define VF2PF_QUERY_RUNTIME_NO_LIMIT 0u
	164	+#define VF2PF_QUERY_RUNTIME_REQUEST_MSG_1_START GUC_HXG_REQUEST_MSG_n_DATAn
	165	+
	166	+#define VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN (GUC_HXG_MSG_MIN_LEN + 1u)
	167	+#define VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MAX_LEN \
	168	+ (VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + VF2PF_QUERY_RUNTIME_MAX_COUNT * 2)
	169	+#define VF2PF_QUERY_RUNTIME_RESPONSE_MSG_0_COUNT GUC_HXG_RESPONSE_MSG_0_DATA0
	170	+#define VF2PF_QUERY_RUNTIME_MIN_COUNT 0
	171	+#define VF2PF_QUERY_RUNTIME_MAX_COUNT \
	172	+ ((GUC_RELAY_MSG_MAX_LEN - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) / 2)
	173	+#define VF2PF_QUERY_RUNTIME_RESPONSE_MSG_1_REMAINING GUC_HXG_RESPONSE_MSG_n_DATAn
	174	+#define VF2PF_QUERY_RUNTIME_RESPONSE_DATAn_REG_OFFSETx GUC_HXG_RESPONSE_MSG_n_DATAn
	175	+#define VF2PF_QUERY_RUNTIME_RESPONSE_DATAn_REG_VALUEx GUC_HXG_RESPONSE_MSG_n_DATAn
	176	+
9	177	/**
10	178	* DOC: GuC Relay Debug Actions
11	179	*

drivers/gpu/drm/xe/display/xe_display.c+28−15

@@ -126,15 +126,14 @@ int xe_display_init_nommio(struct xe_device *xe) · 125 unmodified lines
126	126	return drmm_add_action_or_reset(&xe->drm, xe_display_fini_nommio, xe);
127	127	}
128	128
129		−static void xe_display_fini_noirq(struct drm_device dev, void dummy)
	129	+static void xe_display_fini_noirq(void *arg)
130	130	{
131		− struct xe_device *xe = to_xe_device(dev);
	131	+ struct xe_device *xe = arg;
132	132
133	133	if (!xe->info.enable_display)
134	134	return;
135	135
136	136	intel_display_driver_remove_noirq(xe);
137		− intel_power_domains_driver_remove(xe);
138	137	}
139	138
140	139	int xe_display_init_noirq(struct xe_device *xe)
@@ -163,12 +162,12 @@ int xe_display_init_noirq(struct xe_device *xe) · 22 unmodified lines
163	162	if (err)
164	163	return err;
165	164
166		− return drmm_add_action_or_reset(&xe->drm, xe_display_fini_noirq, NULL);
	165	+ return devm_add_action_or_reset(xe->drm.dev, xe_display_fini_noirq, xe);
167	166	}
168	167
169		−static void xe_display_fini_noaccel(struct drm_device dev, void dummy)
	168	+static void xe_display_fini_noaccel(void *arg)
170	169	{
171		− struct xe_device *xe = to_xe_device(dev);
	170	+ struct xe_device *xe = arg;
172	171
173	172	if (!xe->info.enable_display)
174	173	return;
@@ -187,7 +186,7 @@ int xe_display_init_noaccel(struct xe_device *xe) · 12 unmodified lines
187	186	if (err)
188	187	return err;
189	188
190		− return drmm_add_action_or_reset(&xe->drm, xe_display_fini_noaccel, NULL);
	189	+ return devm_add_action_or_reset(xe->drm.dev, xe_display_fini_noaccel, xe);
191	190	}
192	191
193	192	int xe_display_init(struct xe_device *xe)
@@ -235,8 +234,6 @@ void xe_display_driver_remove(struct xe_device *xe) · 41 unmodified lines
235	234	return;
236	235
237	236	intel_display_driver_remove(xe);
238		−
239		− intel_display_device_remove(xe);
240	237	}
241	238
242	239	/* IRQ-related functions */
@@ -300,7 +297,7 @@ static bool suspend_to_idle(void) · 57 unmodified lines
300	297	return false;
301	298	}
302	299
303		−void xe_display_pm_suspend(struct xe_device *xe)
	300	+void xe_display_pm_suspend(struct xe_device *xe, bool runtime)
304	301	{
305	302	bool s2idle = suspend_to_idle();
306	303	if (!xe->info.enable_display)
@@ -314,7 +311,8 @@ void xe_display_pm_suspend(struct xe_device *xe) · 7 unmodified lines
314	311	if (has_display(xe))
315	312	drm_kms_helper_poll_disable(&xe->drm);
316	313
317		− intel_display_driver_suspend(xe);
	314	+ if (!runtime)
	315	+ intel_display_driver_suspend(xe);
318	316
319	317	intel_dp_mst_suspend(xe);
320	318
@@ -350,7 +348,7 @@ void xe_display_pm_resume_early(struct xe_device *xe) · 29 unmodified lines
350	348	intel_power_domains_resume(xe);
351	349	}
352	350
353		−void xe_display_pm_resume(struct xe_device *xe)
	351	+void xe_display_pm_resume(struct xe_device *xe, bool runtime)
354	352	{
355	353	if (!xe->info.enable_display)
356	354	return;
@@ -365,7 +363,8 @@ void xe_display_pm_resume(struct xe_device *xe) · 8 unmodified lines
365	363
366	364	/* MST sideband requires HPD interrupts enabled */
367	365	intel_dp_mst_resume(xe);
368		− intel_display_driver_resume(xe);
	366	+ if (!runtime)
	367	+ intel_display_driver_resume(xe);
369	368
370	369	intel_hpd_poll_disable(xe);
371	370	if (has_display(xe))
@@ -378,17 +377,31 @@ void xe_display_pm_resume(struct xe_device *xe) · 6 unmodified lines
378	377	intel_power_domains_enable(xe);
379	378	}
380	379
381		−void xe_display_probe(struct xe_device *xe)
	380	+static void display_device_remove(struct drm_device dev, void arg)
	381	+{
	382	+ struct xe_device *xe = arg;
	383	+
	384	+ intel_display_device_remove(xe);
	385	+}
	386	+
	387	+int xe_display_probe(struct xe_device *xe)
382	388	{
	389	+ int err;
	390	+
383	391	if (!xe->info.enable_display)
384	392	goto no_display;
385	393
386	394	intel_display_device_probe(xe);
387	395
	396	+ err = drmm_add_action_or_reset(&xe->drm, display_device_remove, xe);
	397	+ if (err)
	398	+ return err;
	399	+
388	400	if (has_display(xe))
389		− return;
	401	+ return 0;
390	402
391	403	no_display:
392	404	xe->info.enable_display = false;
393	405	unset_display_features(xe);
	406	+ return 0;
394	407	}

drivers/gpu/drm/xe/display/xe_display.h+6−6

@@ -18,7 +18,7 @@ void xe_display_driver_remove(struct xe_device *xe); · 17 unmodified lines
18	18
19	19	int xe_display_create(struct xe_device *xe);
20	20
21		−void xe_display_probe(struct xe_device *xe);
	21	+int xe_display_probe(struct xe_device *xe);
22	22
23	23	int xe_display_init_nommio(struct xe_device *xe);
24	24	int xe_display_init_noirq(struct xe_device *xe);
@@ -34,10 +34,10 @@ void xe_display_irq_enable(struct xe_device *xe, u32 gu_misc_iir); · 9 unmodified lines
34	34	void xe_display_irq_reset(struct xe_device *xe);
35	35	void xe_display_irq_postinstall(struct xe_device xe, struct xe_gt gt);
36	36
37		−void xe_display_pm_suspend(struct xe_device *xe);
	37	+void xe_display_pm_suspend(struct xe_device *xe, bool runtime);
38	38	void xe_display_pm_suspend_late(struct xe_device *xe);
39	39	void xe_display_pm_resume_early(struct xe_device *xe);
40		−void xe_display_pm_resume(struct xe_device *xe);
	40	+void xe_display_pm_resume(struct xe_device *xe, bool runtime);
41	41
42	42	#else
43	43
@@ -47,7 +47,7 @@ static inline void xe_display_driver_remove(struct xe_device *xe) {} · 3 unmodified lines
47	47
48	48	static inline int xe_display_create(struct xe_device *xe) { return 0; }
49	49
50		−static inline void xe_display_probe(struct xe_device *xe) { }
	50	+static inline int xe_display_probe(struct xe_device *xe) { return 0; }
51	51
52	52	static inline int xe_display_init_nommio(struct xe_device *xe) { return 0; }
53	53	static inline int xe_display_init_noirq(struct xe_device *xe) { return 0; }
@@ -63,10 +63,10 @@ static inline void xe_display_irq_enable(struct xe_device *xe, u32 gu_misc_iir) · 9 unmodified lines
63	63	static inline void xe_display_irq_reset(struct xe_device *xe) {}
64	64	static inline void xe_display_irq_postinstall(struct xe_device xe, struct xe_gt gt) {}
65	65
66		−static inline void xe_display_pm_suspend(struct xe_device *xe) {}
	66	+static inline void xe_display_pm_suspend(struct xe_device *xe, bool runtime) {}
67	67	static inline void xe_display_pm_suspend_late(struct xe_device *xe) {}
68	68	static inline void xe_display_pm_resume_early(struct xe_device *xe) {}
69		−static inline void xe_display_pm_resume(struct xe_device *xe) {}
	69	+static inline void xe_display_pm_resume(struct xe_device *xe, bool runtime) {}
70	70
71	71	#endif /* CONFIG_DRM_XE_DISPLAY */
72	72	#endif /* _XE_DISPLAY_H_ */

drivers/gpu/drm/xe/display/xe_hdcp_gsc.c+1−0

@@ -13,6 +13,7 @@ · 12 unmodified lines
13	13	#include "xe_bo.h"
14	14	#include "xe_device.h"
15	15	#include "xe_device_types.h"
	16	+#include "xe_force_wake.h"
16	17	#include "xe_gsc_proxy.h"
17	18	#include "xe_gsc_submit.h"
18	19	#include "xe_gt.h"

drivers/gpu/drm/xe/instructions/xe_gpu_commands.hrenamed from drivers/gpu/drm/xe/regs/xe_gpu_commands.h+0−0

File renamed without changes.

drivers/gpu/drm/xe/regs/xe_bars.h+11−0

@@ -0,0 +1,11 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2024 Intel Corporation
	4	+ */
	5	+#ifndef _XE_BARS_H_
	6	+#define _XE_BARS_H_
	7	+
	8	+#define GTTMMADR_BAR 0 /* MMIO + GTT */
	9	+#define LMEM_BAR 2 /* VRAM */
	10	+
	11	+#endif

drivers/gpu/drm/xe/regs/xe_engine_regs.h+8−2

@@ -44,9 +44,10 @@ · 43 unmodified lines
44	44	#define GSCCS_RING_BASE 0x11a000
45	45
46	46	#define RING_TAIL(base) XE_REG((base) + 0x30)
	47	+#define TAIL_ADDR REG_GENMASK(20, 3)
47	48
48	49	#define RING_HEAD(base) XE_REG((base) + 0x34)
49		−#define HEAD_ADDR 0x001FFFFC
	50	+#define HEAD_ADDR REG_GENMASK(20, 2)
50	51
51	52	#define RING_START(base) XE_REG((base) + 0x38)
52	53
@@ -54,6 +55,8 @@ · 1 unmodified line
54	55	#define RING_CTL_SIZE(size) ((size) - PAGE_SIZE) /* in bytes -> pages */
55	56	#define RING_CTL_SIZE(size) ((size) - PAGE_SIZE) /* in bytes -> pages */
56	57
	58	+#define RING_START_UDW(base) XE_REG((base) + 0x48)
	59	+
57	60	#define RING_PSMI_CTL(base) XE_REG((base) + 0x50, XE_REG_OPTION_MASKED)
58	61	#define RC_SEMA_IDLE_MSG_DISABLE REG_BIT(12)
59	62	#define WAIT_FOR_EVENT_POWER_DOWN_DISABLE REG_BIT(7)
@@ -65,6 +68,7 @@ · 5 unmodified lines
65	68	#define RING_ACTHD_UDW(base) XE_REG((base) + 0x5c)
66	69	#define RING_DMA_FADD_UDW(base) XE_REG((base) + 0x60)
67	70	#define RING_IPEHR(base) XE_REG((base) + 0x68)
	71	+#define RING_INSTDONE(base) XE_REG((base) + 0x6c)
68	72	#define RING_ACTHD(base) XE_REG((base) + 0x74)
69	73	#define RING_DMA_FADD(base) XE_REG((base) + 0x78)
70	74	#define RING_HWS_PGA(base) XE_REG((base) + 0x80)
@@ -108,6 +112,8 @@ · 37 unmodified lines
108	112	#define FF_DOP_CLOCK_GATE_DISABLE REG_BIT(1)
109	113	#define REPLAY_MODE_GRANULARITY REG_BIT(0)
110	114
	115	+#define INDIRECT_RING_STATE(base) XE_REG((base) + 0x108)
	116	+
111	117	#define RING_BBADDR(base) XE_REG((base) + 0x140)
112	118	#define RING_BBADDR_UDW(base) XE_REG((base) + 0x168)
113	119
@@ -123,6 +129,7 @@ · 9 unmodified lines
123	129	#define RING_EXECLIST_STATUS_HI(base) XE_REG((base) + 0x234 + 4)
124	130
125	131	#define RING_CONTEXT_CONTROL(base) XE_REG((base) + 0x244, XE_REG_OPTION_MASKED)
	132	+#define CTX_CTRL_INDIRECT_RING_STATE_ENABLE REG_BIT(4)
126	133	#define CTX_CTRL_INHIBIT_SYN_CTX_SWITCH REG_BIT(3)
127	134	#define CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT REG_BIT(0)
128	135
@@ -135,7 +142,6 @@ · 6 unmodified lines
135	142	#define RING_VALID_MASK 0x00000001
136	143	#define RING_VALID 0x00000001
137	144	#define STOP_RING REG_BIT(8)
138		−#define TAIL_ADDR 0x001FFFF8
139	145
140	146	#define RING_CTX_TIMESTAMP(base) XE_REG((base) + 0x3a8)
141	147	#define CSBE_DEBUG_STATUS(base) XE_REG((base) + 0x3fc)

drivers/gpu/drm/xe/regs/xe_gt_regs.h+61−6

@@ -59,6 +59,27 @@ · 58 unmodified lines
59	59
60	60	#define XELP_GLOBAL_MOCS(i) XE_REG(0x4000 + (i) * 4)
61	61	#define XEHP_GLOBAL_MOCS(i) XE_REG_MCR(0x4000 + (i) * 4)
	62	+#define LE_SSE_MASK REG_GENMASK(18, 17)
	63	+#define LE_SSE(value) REG_FIELD_PREP(LE_SSE_MASK, value)
	64	+#define LE_COS_MASK REG_GENMASK(16, 15)
	65	+#define LE_COS(value) REG_FIELD_PREP(LE_COS_MASK)
	66	+#define LE_SCF_MASK REG_BIT(14)
	67	+#define LE_SCF(value) REG_FIELD_PREP(LE_SCF_MASK, value)
	68	+#define LE_PFM_MASK REG_GENMASK(13, 11)
	69	+#define LE_PFM(value) REG_FIELD_PREP(LE_PFM_MASK, value)
	70	+#define LE_SCC_MASK REG_GENMASK(10, 8)
	71	+#define LE_SCC(value) REG_FIELD_PREP(LE_SCC_MASK, value)
	72	+#define LE_RSC_MASK REG_BIT(7)
	73	+#define LE_RSC(value) REG_FIELD_PREP(LE_RSC_MASK, value)
	74	+#define LE_AOM_MASK REG_BIT(6)
	75	+#define LE_AOM(value) REG_FIELD_PREP(LE_AOM_MASK, value)
	76	+#define LE_LRUM_MASK REG_GENMASK(5, 4)
	77	+#define LE_LRUM(value) REG_FIELD_PREP(LE_LRUM_MASK, value)
	78	+#define LE_TGT_CACHE_MASK REG_GENMASK(3, 2)
	79	+#define LE_TGT_CACHE(value) REG_FIELD_PREP(LE_TGT_CACHE_MASK, value)
	80	+#define LE_CACHEABILITY_MASK REG_GENMASK(1, 0)
	81	+#define LE_CACHEABILITY(value) REG_FIELD_PREP(LE_CACHEABILITY_MASK, value)
	82	+
62	83	#define CCS_AUX_INV XE_REG(0x4208)
63	84
64	85	#define VD0_AUX_INV XE_REG(0x4218)
@@ -98,6 +119,8 @@ · 33 unmodified lines
98	119	#define FF_MODE2_TDS_TIMER_MASK REG_GENMASK(23, 16)
99	120	#define FF_MODE2_TDS_TIMER_128 REG_FIELD_PREP(FF_MODE2_TDS_TIMER_MASK, 4)
100	121
	122	+#define XEHPG_INSTDONE_GEOM_SVGUNIT XE_REG_MCR(0x666c)
	123	+
101	124	#define CACHE_MODE_1 XE_REG(0x7004, XE_REG_OPTION_MASKED)
102	125	#define MSAA_OPTIMIZATION_REDUC_DISABLE REG_BIT(11)
103	126
@@ -115,6 +138,14 @@ · 11 unmodified lines
115	138	#define FLSH_IGNORES_PSD REG_BIT(10)
116	139	#define FD_END_COLLECT REG_BIT(5)
117	140
	141	+#define SC_INSTDONE XE_REG(0x7100)
	142	+#define SC_INSTDONE_EXTRA XE_REG(0x7104)
	143	+#define SC_INSTDONE_EXTRA2 XE_REG(0x7108)
	144	+
	145	+#define XEHPG_SC_INSTDONE XE_REG_MCR(0x7100)
	146	+#define XEHPG_SC_INSTDONE_EXTRA XE_REG_MCR(0x7104)
	147	+#define XEHPG_SC_INSTDONE_EXTRA2 XE_REG_MCR(0x7108)
	148	+
118	149	#define COMMON_SLICE_CHICKEN4 XE_REG(0x7300, XE_REG_OPTION_MASKED)
119	150	#define DISABLE_TDC_LOAD_BALANCING_CALC REG_BIT(6)
120	151
@@ -173,8 +204,11 @@ · 52 unmodified lines
173	204	#define MAX_MSLICES 4
174	205	#define MEML3_EN_MASK REG_GENMASK(3, 0)
175	206
	207	+#define MIRROR_FUSE1 XE_REG(0x911c)
	208	+
176	209	#define XELP_EU_ENABLE XE_REG(0x9134) /* "_DISABLE" on Xe_LP */
177	210	#define XELP_EU_MASK REG_GENMASK(7, 0)
	211	+#define XELP_GT_SLICE_ENABLE XE_REG(0x9138)
178	212	#define XELP_GT_GEOMETRY_DSS_ENABLE XE_REG(0x913c)
179	213
180	214	#define GT_VEBOX_VDBOX_DISABLE XE_REG(0x9140)
@@ -275,6 +309,8 @@ · 94 unmodified lines
275	309	#define RC_CTL_RC6_ENABLE REG_BIT(18)
276	310	#define RC_STATE XE_REG(0xa094)
277	311	#define RC_IDLE_HYSTERSIS XE_REG(0xa0ac)
	312	+#define MEDIA_POWERGATE_IDLE_HYSTERESIS XE_REG(0xa0c4)
	313	+#define RENDER_POWERGATE_IDLE_HYSTERESIS XE_REG(0xa0c8)
278	314
279	315	#define PMINTRMSK XE_REG(0xa168)
280	316	#define PMINTR_DISABLE_REDIRECT_TO_GUC REG_BIT(31)
@@ -282,11 +318,11 @@ · 1 unmodified line
282	318
283	319	#define FORCEWAKE_GT XE_REG(0xa188)
284	320
285		−#define PG_ENABLE XE_REG(0xa210)
286		−#define VD2_MFXVDENC_POWERGATE_ENABLE REG_BIT(8)
287		−#define VD2_HCP_POWERGATE_ENABLE REG_BIT(7)
288		−#define VD0_MFXVDENC_POWERGATE_ENABLE REG_BIT(4)
289		−#define VD0_HCP_POWERGATE_ENABLE REG_BIT(3)
	321	+#define POWERGATE_ENABLE XE_REG(0xa210)
	322	+#define RENDER_POWERGATE_ENABLE REG_BIT(0)
	323	+#define MEDIA_POWERGATE_ENABLE REG_BIT(1)
	324	+#define VDN_HCP_POWERGATE_ENABLE(n) REG_BIT(3 + 2 * (n))
	325	+#define VDN_MFXVDENC_POWERGATE_ENABLE(n) REG_BIT(4 + 2 * (n))
290	326
291	327	#define CTC_MODE XE_REG(0xa26c)
292	328	#define CTC_SHIFT_PARAMETER_MASK REG_GENMASK(2, 1)
@@ -301,9 +337,24 @@ · 8 unmodified lines
301	337	#define XEHPC_OVRLSCCC REG_BIT(0)
302	338
303	339	/* L3 Cache Control */
	340	+#define LNCFCMOCS_REG_COUNT 32
304	341	#define XELP_LNCFCMOCS(i) XE_REG(0xb020 + (i) * 4)
305	342	#define XEHP_LNCFCMOCS(i) XE_REG_MCR(0xb020 + (i) * 4)
306		−#define LNCFCMOCS_REG_COUNT 32
	343	+#define L3_UPPER_LKUP_MASK REG_BIT(23)
	344	+#define L3_UPPER_GLBGO_MASK REG_BIT(22)
	345	+#define L3_UPPER_IDX_CACHEABILITY_MASK REG_GENMASK(21, 20)
	346	+#define L3_UPPER_IDX_SCC_MASK REG_GENMASK(19, 17)
	347	+#define L3_UPPER_IDX_ESC_MASK REG_BIT(16)
	348	+#define L3_LKUP_MASK REG_BIT(7)
	349	+#define L3_LKUP(value) REG_FIELD_PREP(L3_LKUP_MASK, value)
	350	+#define L3_GLBGO_MASK REG_BIT(6)
	351	+#define L3_GLBGO(value) REG_FIELD_PREP(L3_GLBGO_MASK, value)
	352	+#define L3_CACHEABILITY_MASK REG_GENMASK(5, 4)
	353	+#define L3_CACHEABILITY(value) REG_FIELD_PREP(L3_CACHEABILITY_MASK, value)
	354	+#define L3_SCC_MASK REG_GENMASK(3, 1)
	355	+#define L3_SCC(value) REG_FIELD_PREP(L3_SCC_MASK, value)
	356	+#define L3_ESC_MASK REG_BIT(0)
	357	+#define L3_ESC(value) REG_FIELD_PREP(L3_ESC_MASK, value)
307	358
308	359	#define XEHP_L3NODEARBCFG XE_REG_MCR(0xb0b4)
309	360	#define XEHP_LNESPARE REG_BIT(19)
@@ -342,6 +393,9 @@ · 32 unmodified lines
342	393	#define HALF_SLICE_CHICKEN5 XE_REG_MCR(0xe188, XE_REG_OPTION_MASKED)
343	394	#define DISABLE_SAMPLE_G_PERFORMANCE REG_BIT(0)
344	395
	396	+#define SAMPLER_INSTDONE XE_REG_MCR(0xe160)
	397	+#define ROW_INSTDONE XE_REG_MCR(0xe164)
	398	+
345	399	#define SAMPLER_MODE XE_REG_MCR(0xe18c, XE_REG_OPTION_MASKED)
346	400	#define ENABLE_SMALLPL REG_BIT(15)
347	401	#define SC_DISABLE_POWER_OPTIMIZATION_EBB REG_BIT(9)
@@ -350,6 +404,7 @@ · 2 unmodified lines
350	404
351	405	#define HALF_SLICE_CHICKEN7 XE_REG_MCR(0xe194, XE_REG_OPTION_MASKED)
352	406	#define DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA REG_BIT(15)
	407	+#define CLEAR_OPTIMIZATION_DISABLE REG_BIT(6)
353	408
354	409	#define CACHE_MODE_SS XE_REG_MCR(0xe420, XE_REG_OPTION_MASKED)
355	410	#define DISABLE_ECC REG_BIT(5)

drivers/gpu/drm/xe/regs/xe_guc_regs.h+2−0

@@ -40,6 +40,8 @@ · 39 unmodified lines
40	40	#define GS_BOOTROM_JUMP_PASSED REG_FIELD_PREP(GS_BOOTROM_MASK, 0x76)
41	41	#define GS_MIA_IN_RESET REG_BIT(0)
42	42
	43	+#define GUC_HEADER_INFO XE_REG(0xc014)
	44	+
43	45	#define GUC_WOPCM_SIZE XE_REG(0xc050)
44	46	#define GUC_WOPCM_SIZE_MASK REG_GENMASK(31, 12)
45	47	#define GUC_WOPCM_SIZE_LOCKED REG_BIT(0)

drivers/gpu/drm/xe/regs/xe_lrc_layout.h+8−0

@@ -11,6 +11,8 @@ · 10 unmodified lines
11	11	#define CTX_RING_TAIL (0x06 + 1)
12	12	#define CTX_RING_START (0x08 + 1)
13	13	#define CTX_RING_CTL (0x0a + 1)
	14	+#define CTX_TIMESTAMP (0x22 + 1)
	15	+#define CTX_INDIRECT_RING_STATE (0x26 + 1)
14	16	#define CTX_PDP0_UDW (0x30 + 1)
15	17	#define CTX_PDP0_LDW (0x32 + 1)
16	18
@@ -23,4 +25,10 @@ · 6 unmodified lines
23	25	#define CTX_INT_SRC_REPORT_REG (CTX_LRI_INT_REPORT_PTR + 3)
24	26	#define CTX_INT_SRC_REPORT_PTR (CTX_LRI_INT_REPORT_PTR + 4)
25	27
	28	+#define INDIRECT_CTX_RING_HEAD (0x02 + 1)
	29	+#define INDIRECT_CTX_RING_TAIL (0x04 + 1)
	30	+#define INDIRECT_CTX_RING_START (0x06 + 1)
	31	+#define INDIRECT_CTX_RING_START_UDW (0x08 + 1)
	32	+#define INDIRECT_CTX_RING_CTL (0x0a + 1)
	33	+
26	34	#endif

drivers/gpu/drm/xe/regs/xe_pcode_regs.h+7−0

@@ -18,4 +18,11 @@ · 17 unmodified lines
18	18	#define PVC_GT0_PLATFORM_ENERGY_STATUS XE_REG(0x28106c)
19	19	#define PVC_GT0_PACKAGE_POWER_SKU XE_REG(0x281080)
20	20
	21	+#define BMG_PACKAGE_POWER_SKU XE_REG(0x138098)
	22	+#define BMG_PACKAGE_POWER_SKU_UNIT XE_REG(0x1380dc)
	23	+#define BMG_PACKAGE_ENERGY_STATUS XE_REG(0x138120)
	24	+#define BMG_PACKAGE_RAPL_LIMIT XE_REG(0x138440)
	25	+#define BMG_PLATFORM_ENERGY_STATUS XE_REG(0x138458)
	26	+#define BMG_PLATFORM_POWER_LIMIT XE_REG(0x138460)
	27	+
21	28	#endif /* _XE_PCODE_REGS_H_ */

drivers/gpu/drm/xe/regs/xe_regs.h+3−0

@@ -30,6 +30,9 @@ · 29 unmodified lines
30	30	#define XEHP_CLOCK_GATE_DIS XE_REG(0x101014)
31	31	#define SGSI_SIDECLK_DIS REG_BIT(17)
32	32
	33	+#define XEHP_MTCFG_ADDR XE_REG(0x101800)
	34	+#define TILE_COUNT REG_GENMASK(15, 8)
	35	+
33	36	#define GGC XE_REG(0x108040)
34	37	#define GMS_MASK REG_GENMASK(15, 8)
35	38	#define GGMS_MASK REG_GENMASK(7, 6)

drivers/gpu/drm/xe/regs/xe_sriov_regs.h+3−0

@@ -14,6 +14,9 @@ · 13 unmodified lines
14	14	#define LMEM_EN REG_BIT(31)
15	15	#define LMTT_DIR_PTR REG_GENMASK(30, 0) /* in multiples of 64KB */
16	16
	17	+#define VIRTUAL_CTRL_REG XE_REG(0x10108c)
	18	+#define GUEST_GTT_UPDATE_EN REG_BIT(8)
	19	+
17	20	#define VF_CAP_REG XE_REG(0x1901f8, XE_REG_OPTION_VF)
18	21	#define VF_CAP REG_BIT(0)
19	22

drivers/gpu/drm/xe/tests/Makefile+1−0

@@ -11,6 +11,7 @@ xe_live_test-y = xe_live_test_mod.o \ · 10 unmodified lines
11	11	# Normal kunit tests
12	12	obj-$(CONFIG_DRM_XE_KUNIT_TEST) += xe_test.o
13	13	xe_test-y = xe_test_mod.o \
	14	+ xe_args_test.o \
14	15	xe_pci_test.o \
15	16	xe_rtp_test.o \
16	17	xe_wa_test.o

drivers/gpu/drm/xe/tests/xe_args_test.c+221−0

@@ -0,0 +1,221 @@
	1	+// SPDX-License-Identifier: GPL-2.0
	2	+/*
	3	+ * Copyright © 2024 Intel Corporation
	4	+ */
	5	+
	6	+#include <kunit/test.h>
	7	+
	8	+#include "xe_args.h"
	9	+
	10	+static void call_args_example(struct kunit *test)
	11	+{
	12	+#define foo X, Y, Z, Q
	13	+#define bar COUNT_ARGS(foo)
	14	+#define buz CALL_ARGS(COUNT_ARGS, foo)
	15	+
	16	+ KUNIT_EXPECT_EQ(test, bar, 1);
	17	+ KUNIT_EXPECT_EQ(test, buz, 4);
	18	+
	19	+#undef foo
	20	+#undef bar
	21	+#undef buz
	22	+}
	23	+
	24	+static void drop_first_arg_example(struct kunit *test)
	25	+{
	26	+#define foo X, Y, Z, Q
	27	+#define bar CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(foo))
	28	+
	29	+ KUNIT_EXPECT_EQ(test, bar, 3);
	30	+
	31	+#undef foo
	32	+#undef bar
	33	+}
	34	+
	35	+static void first_arg_example(struct kunit *test)
	36	+{
	37	+ int X = 1;
	38	+
	39	+#define foo X, Y, Z, Q
	40	+#define bar FIRST_ARG(foo)
	41	+
	42	+ KUNIT_EXPECT_EQ(test, bar, X);
	43	+ KUNIT_EXPECT_STREQ(test, __stringify(bar), "X");
	44	+
	45	+#undef foo
	46	+#undef bar
	47	+}
	48	+
	49	+static void last_arg_example(struct kunit *test)
	50	+{
	51	+ int Q = 1;
	52	+
	53	+#define foo X, Y, Z, Q
	54	+#define bar LAST_ARG(foo)
	55	+
	56	+ KUNIT_EXPECT_EQ(test, bar, Q);
	57	+ KUNIT_EXPECT_STREQ(test, __stringify(bar), "Q");
	58	+
	59	+#undef foo
	60	+#undef bar
	61	+}
	62	+
	63	+static void pick_arg_example(struct kunit *test)
	64	+{
	65	+ int Y = 1, Z = 2;
	66	+
	67	+#define foo X, Y, Z, Q
	68	+#define bar PICK_ARG(2, foo)
	69	+#define buz PICK_ARG3(foo)
	70	+
	71	+ KUNIT_EXPECT_EQ(test, bar, Y);
	72	+ KUNIT_EXPECT_STREQ(test, __stringify(bar), "Y");
	73	+ KUNIT_EXPECT_EQ(test, buz, Z);
	74	+ KUNIT_EXPECT_STREQ(test, __stringify(buz), "Z");
	75	+
	76	+#undef foo
	77	+#undef bar
	78	+#undef buz
	79	+}
	80	+
	81	+static void sep_comma_example(struct kunit *test)
	82	+{
	83	+#define foo(f) f(X) f(Y) f(Z) f(Q)
	84	+#define bar DROP_FIRST_ARG(foo(ARGS_SEP_COMMA __stringify))
	85	+#define buz CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(foo(ARGS_SEP_COMMA)))
	86	+
	87	+ static const char * const a[] = { bar };
	88	+
	89	+ KUNIT_EXPECT_STREQ(test, a[0], "X");
	90	+ KUNIT_EXPECT_STREQ(test, a[1], "Y");
	91	+ KUNIT_EXPECT_STREQ(test, a[2], "Z");
	92	+ KUNIT_EXPECT_STREQ(test, a[3], "Q");
	93	+
	94	+ KUNIT_EXPECT_EQ(test, buz, 4);
	95	+
	96	+#undef foo
	97	+#undef bar
	98	+#undef buz
	99	+}
	100	+
	101	+#define NO_ARGS
	102	+#define FOO_ARGS X, Y, Z, Q
	103	+#define MAX_ARGS -1, -2, -3, -4, -5, -6, -7, -8, -9, -10, -11, -12
	104	+
	105	+static void count_args_test(struct kunit *test)
	106	+{
	107	+ int count;
	108	+
	109	+ /* COUNT_ARGS() counts to 12 */
	110	+
	111	+ count = COUNT_ARGS();
	112	+ KUNIT_EXPECT_EQ(test, count, 0);
	113	+
	114	+ count = COUNT_ARGS(1);
	115	+ KUNIT_EXPECT_EQ(test, count, 1);
	116	+
	117	+ count = COUNT_ARGS(a, b, c, d, e);
	118	+ KUNIT_EXPECT_EQ(test, count, 5);
	119	+
	120	+ count = COUNT_ARGS(a, b, c, d, e, f, g, h, i, j, k, l);
	121	+ KUNIT_EXPECT_EQ(test, count, 12);
	122	+
	123	+ /* COUNT_ARGS() does not expand params */
	124	+
	125	+ count = COUNT_ARGS(NO_ARGS);
	126	+ KUNIT_EXPECT_EQ(test, count, 1);
	127	+
	128	+ count = COUNT_ARGS(FOO_ARGS);
	129	+ KUNIT_EXPECT_EQ(test, count, 1);
	130	+}
	131	+
	132	+static void call_args_test(struct kunit *test)
	133	+{
	134	+ int count;
	135	+
	136	+ count = CALL_ARGS(COUNT_ARGS, NO_ARGS);
	137	+ KUNIT_EXPECT_EQ(test, count, 0);
	138	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, NO_ARGS), 0);
	139	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, FOO_ARGS), 4);
	140	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, FOO_ARGS, FOO_ARGS), 8);
	141	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, MAX_ARGS), 12);
	142	+}
	143	+
	144	+static void drop_first_arg_test(struct kunit *test)
	145	+{
	146	+ int Y = -2, Z = -3, Q = -4;
	147	+ int a[] = { DROP_FIRST_ARG(FOO_ARGS) };
	148	+
	149	+ KUNIT_EXPECT_EQ(test, DROP_FIRST_ARG(0, -1), -1);
	150	+ KUNIT_EXPECT_EQ(test, DROP_FIRST_ARG(DROP_FIRST_ARG(0, -1, -2)), -2);
	151	+
	152	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(FOO_ARGS)), 3);
	153	+ KUNIT_EXPECT_EQ(test, DROP_FIRST_ARG(DROP_FIRST_ARG(DROP_FIRST_ARG(FOO_ARGS))), -4);
	154	+ KUNIT_EXPECT_EQ(test, a[0], -2);
	155	+ KUNIT_EXPECT_EQ(test, a[1], -3);
	156	+ KUNIT_EXPECT_EQ(test, a[2], -4);
	157	+
	158	+#define foo DROP_FIRST_ARG(FOO_ARGS)
	159	+#define bar DROP_FIRST_ARG(DROP_FIRST_ARG(FOO_ARGS))
	160	+#define buz DROP_FIRST_ARG(DROP_FIRST_ARG(DROP_FIRST_ARG(FOO_ARGS)))
	161	+
	162	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, foo), 3);
	163	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, bar), 2);
	164	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, buz), 1);
	165	+ KUNIT_EXPECT_STREQ(test, __stringify(buz), "Q");
	166	+
	167	+#undef foo
	168	+#undef bar
	169	+#undef buz
	170	+}
	171	+
	172	+static void first_arg_test(struct kunit *test)
	173	+{
	174	+ int X = -1;
	175	+ int a[] = { FIRST_ARG(FOO_ARGS) };
	176	+
	177	+ KUNIT_EXPECT_EQ(test, FIRST_ARG(-1, -2), -1);
	178	+
	179	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, FIRST_ARG(FOO_ARGS)), 1);
	180	+ KUNIT_EXPECT_EQ(test, FIRST_ARG(FOO_ARGS), -1);
	181	+ KUNIT_EXPECT_EQ(test, a[0], -1);
	182	+ KUNIT_EXPECT_STREQ(test, __stringify(FIRST_ARG(FOO_ARGS)), "X");
	183	+}
	184	+
	185	+static void last_arg_test(struct kunit *test)
	186	+{
	187	+ int Q = -4;
	188	+ int a[] = { LAST_ARG(FOO_ARGS) };
	189	+
	190	+ KUNIT_EXPECT_EQ(test, LAST_ARG(-1, -2), -2);
	191	+
	192	+ KUNIT_EXPECT_EQ(test, CALL_ARGS(COUNT_ARGS, LAST_ARG(FOO_ARGS)), 1);
	193	+ KUNIT_EXPECT_EQ(test, LAST_ARG(FOO_ARGS), -4);
	194	+ KUNIT_EXPECT_EQ(test, a[0], -4);
	195	+ KUNIT_EXPECT_STREQ(test, __stringify(LAST_ARG(FOO_ARGS)), "Q");
	196	+
	197	+ KUNIT_EXPECT_EQ(test, LAST_ARG(MAX_ARGS), -12);
	198	+ KUNIT_EXPECT_STREQ(test, __stringify(LAST_ARG(MAX_ARGS)), "-12");
	199	+}
	200	+
	201	+static struct kunit_case args_tests[] = {
	202	+ KUNIT_CASE(count_args_test),
	203	+ KUNIT_CASE(call_args_example),
	204	+ KUNIT_CASE(call_args_test),
	205	+ KUNIT_CASE(drop_first_arg_example),
	206	+ KUNIT_CASE(drop_first_arg_test),
	207	+ KUNIT_CASE(first_arg_example),
	208	+ KUNIT_CASE(first_arg_test),
	209	+ KUNIT_CASE(last_arg_example),
	210	+ KUNIT_CASE(last_arg_test),
	211	+ KUNIT_CASE(pick_arg_example),
	212	+ KUNIT_CASE(sep_comma_example),
	213	+ {}
	214	+};
	215	+
	216	+static struct kunit_suite args_test_suite = {
	217	+ .name = "args",
	218	+ .test_cases = args_tests,
	219	+};
	220	+
	221	+kunit_test_suite(args_test_suite);

drivers/gpu/drm/xe/tests/xe_gt_sriov_pf_service_test.c+232−0

@@ -0,0 +1,232 @@
	1	+// SPDX-License-Identifier: GPL-2.0 AND MIT
	2	+/*
	3	+ * Copyright © 2024 Intel Corporation
	4	+ */
	5	+
	6	+#include <kunit/test.h>
	7	+
	8	+#include "xe_device.h"
	9	+#include "xe_kunit_helpers.h"
	10	+#include "xe_pci_test.h"
	11	+
	12	+static int pf_service_test_init(struct kunit *test)
	13	+{
	14	+ struct xe_pci_fake_data fake = {
	15	+ .sriov_mode = XE_SRIOV_MODE_PF,
	16	+ .platform = XE_TIGERLAKE, /* some random platform */
	17	+ .subplatform = XE_SUBPLATFORM_NONE,
	18	+ };
	19	+ struct xe_device *xe;
	20	+ struct xe_gt *gt;
	21	+
	22	+ test->priv = &fake;
	23	+ xe_kunit_helper_xe_device_test_init(test);
	24	+
	25	+ xe = test->priv;
	26	+ KUNIT_ASSERT_EQ(test, xe_sriov_init(xe), 0);
	27	+
	28	+ gt = xe_device_get_gt(xe, 0);
	29	+ pf_init_versions(gt);
	30	+
	31	+ /*
	32	+ * sanity check:
	33	+ * - all supported platforms VF/PF ABI versions must be defined
	34	+ * - base version can't be newer than latest
	35	+ */
	36	+ KUNIT_ASSERT_NE(test, 0, gt->sriov.pf.service.version.base.major);
	37	+ KUNIT_ASSERT_NE(test, 0, gt->sriov.pf.service.version.latest.major);
	38	+ KUNIT_ASSERT_LE(test, gt->sriov.pf.service.version.base.major,
	39	+ gt->sriov.pf.service.version.latest.major);
	40	+ if (gt->sriov.pf.service.version.base.major == gt->sriov.pf.service.version.latest.major)
	41	+ KUNIT_ASSERT_LE(test, gt->sriov.pf.service.version.base.minor,
	42	+ gt->sriov.pf.service.version.latest.minor);
	43	+
	44	+ test->priv = gt;
	45	+ return 0;
	46	+}
	47	+
	48	+static void pf_negotiate_any(struct kunit *test)
	49	+{
	50	+ struct xe_gt *gt = test->priv;
	51	+ u32 major, minor;
	52	+
	53	+ KUNIT_ASSERT_EQ(test, 0,
	54	+ pf_negotiate_version(gt, VF2PF_HANDSHAKE_MAJOR_ANY,
	55	+ VF2PF_HANDSHAKE_MINOR_ANY,
	56	+ &major, &minor));
	57	+ KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	58	+ KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor);
	59	+}
	60	+
	61	+static void pf_negotiate_base_match(struct kunit *test)
	62	+{
	63	+ struct xe_gt *gt = test->priv;
	64	+ u32 major, minor;
	65	+
	66	+ KUNIT_ASSERT_EQ(test, 0,
	67	+ pf_negotiate_version(gt,
	68	+ gt->sriov.pf.service.version.base.major,
	69	+ gt->sriov.pf.service.version.base.minor,
	70	+ &major, &minor));
	71	+ KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.base.major);
	72	+ KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.base.minor);
	73	+}
	74	+
	75	+static void pf_negotiate_base_newer(struct kunit *test)
	76	+{
	77	+ struct xe_gt *gt = test->priv;
	78	+ u32 major, minor;
	79	+
	80	+ KUNIT_ASSERT_EQ(test, 0,
	81	+ pf_negotiate_version(gt,
	82	+ gt->sriov.pf.service.version.base.major,
	83	+ gt->sriov.pf.service.version.base.minor + 1,
	84	+ &major, &minor));
	85	+ KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.base.major);
	86	+ KUNIT_ASSERT_GE(test, minor, gt->sriov.pf.service.version.base.minor);
	87	+ if (gt->sriov.pf.service.version.base.major == gt->sriov.pf.service.version.latest.major)
	88	+ KUNIT_ASSERT_LE(test, minor, gt->sriov.pf.service.version.latest.minor);
	89	+ else
	90	+ KUNIT_FAIL(test, "FIXME: don't know how to test multi-version yet!\n");
	91	+}
	92	+
	93	+static void pf_negotiate_base_next(struct kunit *test)
	94	+{
	95	+ struct xe_gt *gt = test->priv;
	96	+ u32 major, minor;
	97	+
	98	+ KUNIT_ASSERT_EQ(test, 0,
	99	+ pf_negotiate_version(gt,
	100	+ gt->sriov.pf.service.version.base.major + 1, 0,
	101	+ &major, &minor));
	102	+ KUNIT_ASSERT_GE(test, major, gt->sriov.pf.service.version.base.major);
	103	+ KUNIT_ASSERT_LE(test, major, gt->sriov.pf.service.version.latest.major);
	104	+ if (major == gt->sriov.pf.service.version.latest.major)
	105	+ KUNIT_ASSERT_LE(test, minor, gt->sriov.pf.service.version.latest.minor);
	106	+ else
	107	+ KUNIT_FAIL(test, "FIXME: don't know how to test multi-version yet!\n");
	108	+}
	109	+
	110	+static void pf_negotiate_base_older(struct kunit *test)
	111	+{
	112	+ struct xe_gt *gt = test->priv;
	113	+ u32 major, minor;
	114	+
	115	+ if (!gt->sriov.pf.service.version.base.minor)
	116	+ kunit_skip(test, "no older minor\n");
	117	+
	118	+ KUNIT_ASSERT_NE(test, 0,
	119	+ pf_negotiate_version(gt,
	120	+ gt->sriov.pf.service.version.base.major,
	121	+ gt->sriov.pf.service.version.base.minor - 1,
	122	+ &major, &minor));
	123	+}
	124	+
	125	+static void pf_negotiate_base_prev(struct kunit *test)
	126	+{
	127	+ struct xe_gt *gt = test->priv;
	128	+ u32 major, minor;
	129	+
	130	+ KUNIT_ASSERT_NE(test, 0,
	131	+ pf_negotiate_version(gt,
	132	+ gt->sriov.pf.service.version.base.major - 1, 1,
	133	+ &major, &minor));
	134	+}
	135	+
	136	+static void pf_negotiate_latest_match(struct kunit *test)
	137	+{
	138	+ struct xe_gt *gt = test->priv;
	139	+ u32 major, minor;
	140	+
	141	+ KUNIT_ASSERT_EQ(test, 0,
	142	+ pf_negotiate_version(gt,
	143	+ gt->sriov.pf.service.version.latest.major,
	144	+ gt->sriov.pf.service.version.latest.minor,
	145	+ &major, &minor));
	146	+ KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	147	+ KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor);
	148	+}
	149	+
	150	+static void pf_negotiate_latest_newer(struct kunit *test)
	151	+{
	152	+ struct xe_gt *gt = test->priv;
	153	+ u32 major, minor;
	154	+
	155	+ KUNIT_ASSERT_EQ(test, 0,
	156	+ pf_negotiate_version(gt,
	157	+ gt->sriov.pf.service.version.latest.major,
	158	+ gt->sriov.pf.service.version.latest.minor + 1,
	159	+ &major, &minor));
	160	+ KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	161	+ KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor);
	162	+}
	163	+
	164	+static void pf_negotiate_latest_next(struct kunit *test)
	165	+{
	166	+ struct xe_gt *gt = test->priv;
	167	+ u32 major, minor;
	168	+
	169	+ KUNIT_ASSERT_EQ(test, 0,
	170	+ pf_negotiate_version(gt,
	171	+ gt->sriov.pf.service.version.latest.major + 1, 0,
	172	+ &major, &minor));
	173	+ KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	174	+ KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor);
	175	+}
	176	+
	177	+static void pf_negotiate_latest_older(struct kunit *test)
	178	+{
	179	+ struct xe_gt *gt = test->priv;
	180	+ u32 major, minor;
	181	+
	182	+ if (!gt->sriov.pf.service.version.latest.minor)
	183	+ kunit_skip(test, "no older minor\n");
	184	+
	185	+ KUNIT_ASSERT_EQ(test, 0,
	186	+ pf_negotiate_version(gt,
	187	+ gt->sriov.pf.service.version.latest.major,
	188	+ gt->sriov.pf.service.version.latest.minor - 1,
	189	+ &major, &minor));
	190	+ KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major);
	191	+ KUNIT_ASSERT_EQ(test, minor, gt->sriov.pf.service.version.latest.minor - 1);
	192	+}
	193	+
	194	+static void pf_negotiate_latest_prev(struct kunit *test)
	195	+{
	196	+ struct xe_gt *gt = test->priv;
	197	+ u32 major, minor;
	198	+
	199	+ if (gt->sriov.pf.service.version.base.major == gt->sriov.pf.service.version.latest.major)
	200	+ kunit_skip(test, "no prev major");
	201	+
	202	+ KUNIT_ASSERT_EQ(test, 0,
	203	+ pf_negotiate_version(gt,
	204	+ gt->sriov.pf.service.version.latest.major - 1,
	205	+ gt->sriov.pf.service.version.base.minor + 1,
	206	+ &major, &minor));
	207	+ KUNIT_ASSERT_EQ(test, major, gt->sriov.pf.service.version.latest.major - 1);
	208	+ KUNIT_ASSERT_GE(test, major, gt->sriov.pf.service.version.base.major);
	209	+}
	210	+
	211	+static struct kunit_case pf_service_test_cases[] = {
	212	+ KUNIT_CASE(pf_negotiate_any),
	213	+ KUNIT_CASE(pf_negotiate_base_match),
	214	+ KUNIT_CASE(pf_negotiate_base_newer),
	215	+ KUNIT_CASE(pf_negotiate_base_next),
	216	+ KUNIT_CASE(pf_negotiate_base_older),
	217	+ KUNIT_CASE(pf_negotiate_base_prev),
	218	+ KUNIT_CASE(pf_negotiate_latest_match),
	219	+ KUNIT_CASE(pf_negotiate_latest_newer),
	220	+ KUNIT_CASE(pf_negotiate_latest_next),
	221	+ KUNIT_CASE(pf_negotiate_latest_older),
	222	+ KUNIT_CASE(pf_negotiate_latest_prev),
	223	+ {}
	224	+};
	225	+
	226	+static struct kunit_suite pf_service_suite = {
	227	+ .name = "pf_service",
	228	+ .test_cases = pf_service_test_cases,
	229	+ .init = pf_service_test_init,
	230	+};
	231	+
	232	+kunit_test_suite(pf_service_suite);

drivers/gpu/drm/xe/tests/xe_migrate.c+1−87

@@ -62,36 +62,6 @@ static int run_sanity_job(struct xe_migrate m, struct xe_device xe, · 61 unmodified lines
62	62	return 0;
63	63	}
64	64
65		−static void
66		−sanity_populate_cb(struct xe_migrate_pt_update *pt_update,
67		− struct xe_tile tile, struct iosys_map map, void *dst,
68		− u32 qword_ofs, u32 num_qwords,
69		− const struct xe_vm_pgtable_update *update)
70		−{
71		− struct migrate_test_params *p =
72		− to_migrate_test_params(xe_cur_kunit_priv(XE_TEST_LIVE_MIGRATE));
73		− int i;
74		− u64 *ptr = dst;
75		− u64 value;
76		−
77		− for (i = 0; i < num_qwords; i++) {
78		− value = (qword_ofs + i - update->ofs) * 0x1111111111111111ULL;
79		− if (map)
80		− xe_map_wr(tile_to_xe(tile), map, (qword_ofs + i) *
81		− sizeof(u64), u64, value);
82		− else
83		− ptr[i] = value;
84		− }
85		−
86		− kunit_info(xe_cur_kunit(), "Used %s.\n", map ? "CPU" : "GPU");
87		− if (p->force_gpu && map)
88		− KUNIT_FAIL(xe_cur_kunit(), "GPU pagetable update used CPU.\n");
89		−}
90		−
91		−static const struct xe_migrate_pt_update_ops sanity_ops = {
92		− .populate = sanity_populate_cb,
93		−};
94		−
95	65	#define check(_retval, _expected, str, _test) \
96	66	do { if ((_retval) != (_expected)) { \
97	67	KUNIT_FAIL(_test, "Sanity check failed: " str \
@@ -209,57 +179,6 @@ static void test_copy_vram(struct xe_migrate m, struct xe_bo bo, · 111 unmodified lines
209	179	test_copy(m, bo, test, region);
210	180	}
211	181
212		−static void test_pt_update(struct xe_migrate m, struct xe_bo pt,
213		− struct kunit *test, bool force_gpu)
214		−{
215		− struct xe_device *xe = tile_to_xe(m->tile);
216		− struct dma_fence *fence;
217		− u64 retval, expected;
218		− ktime_t then, now;
219		− int i;
220		−
221		− struct xe_vm_pgtable_update update = {
222		− .ofs = 1,
223		− .qwords = 0x10,
224		− .pt_bo = pt,
225		− };
226		− struct xe_migrate_pt_update pt_update = {
227		− .ops = &sanity_ops,
228		− };
229		− struct migrate_test_params p = {
230		− .base.id = XE_TEST_LIVE_MIGRATE,
231		− .force_gpu = force_gpu,
232		− };
233		−
234		− test->priv = &p;
235		− /* Test xe_migrate_update_pgtables() updates the pagetable as expected */
236		− expected = 0xf0f0f0f0f0f0f0f0ULL;
237		− xe_map_memset(xe, &pt->vmap, 0, (u8)expected, pt->size);
238		−
239		− then = ktime_get();
240		− fence = xe_migrate_update_pgtables(m, m->q->vm, NULL, m->q, &update, 1,
241		− NULL, 0, &pt_update);
242		− now = ktime_get();
243		− if (sanity_fence_failed(xe, fence, "Migration pagetable update", test))
244		− return;
245		−
246		− kunit_info(test, "Updating without syncing took %llu us,\n",
247		− (unsigned long long)ktime_to_us(ktime_sub(now, then)));
248		−
249		− dma_fence_put(fence);
250		− retval = xe_map_rd(xe, &pt->vmap, 0, u64);
251		− check(retval, expected, "PTE[0] must stay untouched", test);
252		−
253		− for (i = 0; i < update.qwords; i++) {
254		− retval = xe_map_rd(xe, &pt->vmap, (update.ofs + i) * 8, u64);
255		− check(retval, i * 0x1111111111111111ULL, "PTE update", test);
256		− }
257		−
258		− retval = xe_map_rd(xe, &pt->vmap, 8 * (update.ofs + update.qwords),
259		− u64);
260		− check(retval, expected, "PTE[0x11] must stay untouched", test);
261		−}
262		−
263	182	static void xe_migrate_sanity_test(struct xe_migrate m, struct kunit test)
264	183	{
265	184	struct xe_tile *tile = m->tile;
@@ -398,11 +317,6 @@ static void xe_migrate_sanity_test(struct xe_migrate m, struct kunit test) · 132 unmodified lines
398	317	test_copy_vram(m, big, test);
399	318	}
400	319
401		− kunit_info(test, "Testing page table update using CPU if GPU idle.\n");
402		− test_pt_update(m, pt, test, false);
403		− kunit_info(test, "Testing page table update using GPU\n");
404		− test_pt_update(m, pt, test, true);
405		−
406	320	out:
407	321	xe_bb_free(bb, NULL);
408	322	free_tiny:
@@ -430,7 +344,7 @@ static int migrate_test_run_device(struct xe_device *xe) · 21 unmodified lines
430	344	struct xe_migrate *m = tile->migrate;
431	345
432	346	kunit_info(test, "Testing tile id %d.\n", id);
433		− xe_vm_lock(m->q->vm, true);
	347	+ xe_vm_lock(m->q->vm, false);
434	348	xe_migrate_sanity_test(m, test);
435	349	xe_vm_unlock(m->q->vm);
436	350	}

drivers/gpu/drm/xe/xe_args.h+143−0

@@ -0,0 +1,143 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_ARGS_H_
	7	+#define _XE_ARGS_H_
	8	+
	9	+#include <linux/args.h>
	10	+
	11	+/*
	12	+ * Why don't the following macros have the XE prefix?
	13	+ *
	14	+ * Once we find more potential users outside of the Xe driver, we plan to move
	15	+ * all of the following macros unchanged to linux/args.h.
	16	+ */
	17	+
	18	+/**
	19	+ * CALL_ARGS - Invoke a macro, but allow parameters to be expanded beforehand.
	20	+ * @f: name of the macro to invoke
	21	+ * @args: arguments for the macro
	22	+ *
	23	+ * This macro allows calling macros which names might generated or we want to
	24	+ * make sure it's arguments will be correctly expanded.
	25	+ *
	26	+ * Example:
	27	+ *
	28	+ * #define foo X,Y,Z,Q
	29	+ * #define bar COUNT_ARGS(foo)
	30	+ * #define buz CALL_ARGS(COUNT_ARGS, foo)
	31	+ *
	32	+ * With above definitions bar expands to 1 while buz expands to 4.
	33	+ */
	34	+#define CALL_ARGS(f, args...) __CALL_ARGS(f, args)
	35	+#define __CALL_ARGS(f, args...) f(args)
	36	+
	37	+/**
	38	+ * DROP_FIRST_ARG - Returns all arguments except the first one.
	39	+ * @args: arguments
	40	+ *
	41	+ * This helper macro allows manipulation the argument list before passing it
	42	+ * to the next level macro.
	43	+ *
	44	+ * Example:
	45	+ *
	46	+ * #define foo X,Y,Z,Q
	47	+ * #define bar CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(foo))
	48	+ *
	49	+ * With above definitions bar expands to 3.
	50	+ */
	51	+#define DROP_FIRST_ARG(args...) __DROP_FIRST_ARG(args)
	52	+#define __DROP_FIRST_ARG(a, b...) b
	53	+
	54	+/**
	55	+ * FIRST_ARG - Returns the first argument.
	56	+ * @args: arguments
	57	+ *
	58	+ * This helper macro allows manipulation the argument list before passing it
	59	+ * to the next level macro.
	60	+ *
	61	+ * Example:
	62	+ *
	63	+ * #define foo X,Y,Z,Q
	64	+ * #define bar FIRST_ARG(foo)
	65	+ *
	66	+ * With above definitions bar expands to X.
	67	+ */
	68	+#define FIRST_ARG(args...) __FIRST_ARG(args)
	69	+#define __FIRST_ARG(a, b...) a
	70	+
	71	+/**
	72	+ * LAST_ARG - Returns the last argument.
	73	+ * @args: arguments
	74	+ *
	75	+ * This helper macro allows manipulation the argument list before passing it
	76	+ * to the next level macro.
	77	+ *
	78	+ * Like COUNT_ARGS() this macro works up to 12 arguments.
	79	+ *
	80	+ * Example:
	81	+ *
	82	+ * #define foo X,Y,Z,Q
	83	+ * #define bar LAST_ARG(foo)
	84	+ *
	85	+ * With above definitions bar expands to Q.
	86	+ */
	87	+#define LAST_ARG(args...) __LAST_ARG(args)
	88	+#define __LAST_ARG(args...) PICK_ARG(COUNT_ARGS(args), args)
	89	+
	90	+/**
	91	+ * PICK_ARG - Returns the n-th argument.
	92	+ * @n: argument number to be returned
	93	+ * @args: arguments
	94	+ *
	95	+ * This helper macro allows manipulation the argument list before passing it
	96	+ * to the next level macro.
	97	+ *
	98	+ * Like COUNT_ARGS() this macro supports n up to 12.
	99	+ * Specialized macros PICK_ARG1() to PICK_ARG12() are also available.
	100	+ *
	101	+ * Example:
	102	+ *
	103	+ * #define foo X,Y,Z,Q
	104	+ * #define bar PICK_ARG(2, foo)
	105	+ * #define buz PICK_ARG3(foo)
	106	+ *
	107	+ * With above definitions bar expands to Y and buz expands to Z.
	108	+ */
	109	+#define PICK_ARG(n, args...) __PICK_ARG(n, args)
	110	+#define __PICK_ARG(n, args...) CALL_ARGS(CONCATENATE(PICK_ARG, n), args)
	111	+#define PICK_ARG1(args...) FIRST_ARG(args)
	112	+#define PICK_ARG2(args...) PICK_ARG1(DROP_FIRST_ARG(args))
	113	+#define PICK_ARG3(args...) PICK_ARG2(DROP_FIRST_ARG(args))
	114	+#define PICK_ARG4(args...) PICK_ARG3(DROP_FIRST_ARG(args))
	115	+#define PICK_ARG5(args...) PICK_ARG4(DROP_FIRST_ARG(args))
	116	+#define PICK_ARG6(args...) PICK_ARG5(DROP_FIRST_ARG(args))
	117	+#define PICK_ARG7(args...) PICK_ARG6(DROP_FIRST_ARG(args))
	118	+#define PICK_ARG8(args...) PICK_ARG7(DROP_FIRST_ARG(args))
	119	+#define PICK_ARG9(args...) PICK_ARG8(DROP_FIRST_ARG(args))
	120	+#define PICK_ARG10(args...) PICK_ARG9(DROP_FIRST_ARG(args))
	121	+#define PICK_ARG11(args...) PICK_ARG10(DROP_FIRST_ARG(args))
	122	+#define PICK_ARG12(args...) PICK_ARG11(DROP_FIRST_ARG(args))
	123	+
	124	+/**
	125	+ * ARGS_SEP_COMMA - Definition of a comma character.
	126	+ *
	127	+ * This definition can be used in cases where any intermediate macro expects
	128	+ * fixed number of arguments, but we want to pass more arguments which can
	129	+ * be properly evaluated only by the next level macro.
	130	+ *
	131	+ * Example:
	132	+ *
	133	+ * #define foo(f) f(X) f(Y) f(Z) f(Q)
	134	+ * #define bar DROP_FIRST_ARG(foo(ARGS_SEP_COMMA __stringify))
	135	+ * #define buz CALL_ARGS(COUNT_ARGS, DROP_FIRST_ARG(foo(ARGS_SEP_COMMA)))
	136	+ *
	137	+ * With above definitions bar expands to
	138	+ * "X", "Y", "Z", "Q"
	139	+ * and buz expands to 4.
	140	+ */
	141	+#define ARGS_SEP_COMMA ,
	142	+
	143	+#endif

drivers/gpu/drm/xe/xe_assert.h+2−2

@@ -109,11 +109,11 @@ · 108 unmodified lines
109	109	#define xe_assert_msg(xe, condition, msg, arg...) ({ \
110	110	const struct xe_device *__xe = (xe); \
111	111	__xe_assert_msg(__xe, condition, \
112		− "platform: %d subplatform: %d\n" \
	112	+ "platform: %s subplatform: %d\n" \
113	113	"graphics: %s %u.%02u step %s\n" \
114	114	"media: %s %u.%02u step %s\n" \
115	115	msg, \
116		− __xe->info.platform, __xe->info.subplatform, \
	116	+ __xe->info.platform_name, __xe->info.subplatform, \
117	117	__xe->info.graphics_name, \
118	118	__xe->info.graphics_verx100 / 100, \
119	119	__xe->info.graphics_verx100 % 100, \

drivers/gpu/drm/xe/xe_bb.c+1−1

@@ -6,7 +6,7 @@ · 5 unmodified lines
6	6	#include "xe_bb.h"
7	7
8	8	#include "instructions/xe_mi_commands.h"
9		−#include "regs/xe_gpu_commands.h"
	9	+#include "xe_assert.h"
10	10	#include "xe_device.h"
11	11	#include "xe_exec_queue_types.h"
12	12	#include "xe_gt.h"

drivers/gpu/drm/xe/xe_bo.c+48−23

@@ -95,6 +95,20 @@ bool xe_bo_is_stolen(struct xe_bo *bo) · 94 unmodified lines
95	95	return bo->ttm.resource->mem_type == XE_PL_STOLEN;
96	96	}
97	97
	98	+/**
	99	+ * xe_bo_has_single_placement - check if BO is placed only in one memory location
	100	+ * @bo: The BO
	101	+ *
	102	+ * This function checks whether a given BO is placed in only one memory location.
	103	+ *
	104	+ * Returns: true if the BO is placed in a single memory location, false otherwise.
	105	+ *
	106	+ */
	107	+bool xe_bo_has_single_placement(struct xe_bo *bo)
	108	+{
	109	+ return bo->placement.num_placement == 1;
	110	+}
	111	+
98	112	/**
99	113	* xe_bo_is_stolen_devmem - check if BO is of stolen type accessed via PCI BAR
100	114	* @bo: The BO
@@ -302,6 +316,18 @@ static int xe_tt_map_sg(struct ttm_tt *tt) · 201 unmodified lines
302	316	return 0;
303	317	}
304	318
	319	+static void xe_tt_unmap_sg(struct ttm_tt *tt)
	320	+{
	321	+ struct xe_ttm_tt *xe_tt = container_of(tt, struct xe_ttm_tt, ttm);
	322	+
	323	+ if (xe_tt->sg) {
	324	+ dma_unmap_sgtable(xe_tt->dev, xe_tt->sg,
	325	+ DMA_BIDIRECTIONAL, 0);
	326	+ sg_free_table(xe_tt->sg);
	327	+ xe_tt->sg = NULL;
	328	+ }
	329	+}
	330	+
305	331	struct sg_table xe_bo_sg(struct xe_bo bo)
306	332	{
307	333	struct ttm_tt *tt = bo->ttm.ttm;
@@ -377,27 +403,15 @@ static int xe_ttm_tt_populate(struct ttm_device ttm_dev, struct ttm_tt tt, · 69 unmodified lines
377	403	if (err)
378	404	return err;
379	405
380		− /* A follow up may move this xe_bo_move when BO is moved to XE_PL_TT */
381		− err = xe_tt_map_sg(tt);
382		− if (err)
383		− ttm_pool_free(&ttm_dev->pool, tt);
384		−
385	406	return err;
386	407	}
387	408
388	409	static void xe_ttm_tt_unpopulate(struct ttm_device ttm_dev, struct ttm_tt tt)
389	410	{
390		− struct xe_ttm_tt *xe_tt = container_of(tt, struct xe_ttm_tt, ttm);
391		−
392	411	if (tt->page_flags & TTM_TT_FLAG_EXTERNAL)
393	412	return;
394	413
395		− if (xe_tt->sg) {
396		− dma_unmap_sgtable(xe_tt->dev, xe_tt->sg,
397		− DMA_BIDIRECTIONAL, 0);
398		− sg_free_table(xe_tt->sg);
399		− xe_tt->sg = NULL;
400		− }
	414	+ xe_tt_unmap_sg(tt);
401	415
402	416	return ttm_pool_free(&ttm_dev->pool, tt);
403	417	}
@@ -628,17 +642,21 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · 224 unmodified lines
628	642	bool handle_system_ccs = (!IS_DGFX(xe) && xe_bo_needs_ccs_pages(bo) &&
629	643	ttm && ttm_tt_is_populated(ttm)) ? true : false;
630	644	int ret = 0;
	645	+
631	646	/* Bo creation path, moving to system or TT. */
632	647	if ((!old_mem && ttm) && !handle_system_ccs) {
633		− ttm_bo_move_null(ttm_bo, new_mem);
634		− return 0;
	648	+ if (new_mem->mem_type == XE_PL_TT)
	649	+ ret = xe_tt_map_sg(ttm);
	650	+ if (!ret)
	651	+ ttm_bo_move_null(ttm_bo, new_mem);
	652	+ goto out;
635	653	}
636	654
637	655	if (ttm_bo->type == ttm_bo_type_sg) {
638	656	ret = xe_bo_move_notify(bo, ctx);
639	657	if (!ret)
640	658	ret = xe_bo_move_dmabuf(ttm_bo, new_mem);
641		− goto out;
	659	+ return ret;
642	660	}
643	661
644	662	tt_has_data = ttm && (ttm_tt_is_populated(ttm) \|\|
@@ -650,6 +668,12 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · 5 unmodified lines
650	668	needs_clear = (ttm && ttm->page_flags & TTM_TT_FLAG_ZERO_ALLOC) \|\|
651	669	(!ttm && ttm_bo->type == ttm_bo_type_device);
652	670
	671	+ if (new_mem->mem_type == XE_PL_TT) {
	672	+ ret = xe_tt_map_sg(ttm);
	673	+ if (ret)
	674	+ goto out;
	675	+ }
	676	+
653	677	if ((move_lacks_source && !needs_clear)) {
654	678	ttm_bo_move_null(ttm_bo, new_mem);
655	679	goto out;
@@ -786,8 +810,11 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · 130 unmodified lines
786	810	xe_pm_runtime_put(xe);
787	811
788	812	out:
789		− return ret;
	813	+ if ((!ttm_bo->resource \|\| ttm_bo->resource->mem_type == XE_PL_SYSTEM) &&
	814	+ ttm_bo->ttm)
	815	+ xe_tt_unmap_sg(ttm_bo->ttm);
790	816
	817	+ return ret;
791	818	}
792	819
793	820	/**
@@ -1731,11 +1758,10 @@ void xe_bo_unpin_external(struct xe_bo *bo) · 937 unmodified lines
1731	1758	xe_assert(xe, xe_bo_is_pinned(bo));
1732	1759	xe_assert(xe, xe_bo_is_user(bo));
1733	1760
1734		− if (bo->ttm.pin_count == 1 && !list_empty(&bo->pinned_link)) {
1735		− spin_lock(&xe->pinned.lock);
	1761	+ spin_lock(&xe->pinned.lock);
	1762	+ if (bo->ttm.pin_count == 1 && !list_empty(&bo->pinned_link))
1736	1763	list_del_init(&bo->pinned_link);
1737		− spin_unlock(&xe->pinned.lock);
1738		− }
	1764	+ spin_unlock(&xe->pinned.lock);
1739	1765
1740	1766	ttm_bo_unpin(&bo->ttm);
1741	1767
@@ -1758,9 +1784,8 @@ void xe_bo_unpin(struct xe_bo *bo) · 16 unmodified lines
1758	1784	struct ttm_place *place = &(bo->placements[0]);
1759	1785
1760	1786	if (mem_type_is_vram(place->mem_type)) {
1761		− xe_assert(xe, !list_empty(&bo->pinned_link));
1762		−
1763	1787	spin_lock(&xe->pinned.lock);
	1788	+ xe_assert(xe, !list_empty(&bo->pinned_link));
1764	1789	list_del_init(&bo->pinned_link);
1765	1790	spin_unlock(&xe->pinned.lock);
1766	1791	}

drivers/gpu/drm/xe/xe_bo.h+1−0

@@ -206,6 +206,7 @@ bool mem_type_is_vram(u32 mem_type); · 205 unmodified lines
206	206	bool xe_bo_is_vram(struct xe_bo *bo);
207	207	bool xe_bo_is_stolen(struct xe_bo *bo);
208	208	bool xe_bo_is_stolen_devmem(struct xe_bo *bo);
	209	+bool xe_bo_has_single_placement(struct xe_bo *bo);
209	210	uint64_t vram_region_gpu_offset(struct ttm_resource *res);
210	211
211	212	bool xe_bo_can_migrate(struct xe_bo *bo, u32 mem_type);

drivers/gpu/drm/xe/xe_debugfs.c+58−0

@@ -12,7 +12,10 @@ · 11 unmodified lines
12	12
13	13	#include "xe_bo.h"
14	14	#include "xe_device.h"
	15	+#include "xe_force_wake.h"
15	16	#include "xe_gt_debugfs.h"
	17	+#include "xe_gt_printk.h"
	18	+#include "xe_guc_ads.h"
16	19	#include "xe_pm.h"
17	20	#include "xe_sriov.h"
18	21	#include "xe_step.h"
@@ -118,6 +121,58 @@ static const struct file_operations forcewake_all_fops = { · 99 unmodified lines
118	121	.release = forcewake_release,
119	122	};
120	123
	124	+static ssize_t wedged_mode_show(struct file f, char __user ubuf,
	125	+ size_t size, loff_t *pos)
	126	+{
	127	+ struct xe_device *xe = file_inode(f)->i_private;
	128	+ char buf[32];
	129	+ int len = 0;
	130	+
	131	+ len = scnprintf(buf, sizeof(buf), "%d\n", xe->wedged.mode);
	132	+
	133	+ return simple_read_from_buffer(ubuf, size, pos, buf, len);
	134	+}
	135	+
	136	+static ssize_t wedged_mode_set(struct file f, const char __user ubuf,
	137	+ size_t size, loff_t *pos)
	138	+{
	139	+ struct xe_device *xe = file_inode(f)->i_private;
	140	+ struct xe_gt *gt;
	141	+ u32 wedged_mode;
	142	+ ssize_t ret;
	143	+ u8 id;
	144	+
	145	+ ret = kstrtouint_from_user(ubuf, size, 0, &wedged_mode);
	146	+ if (ret)
	147	+ return ret;
	148	+
	149	+ if (wedged_mode > 2)
	150	+ return -EINVAL;
	151	+
	152	+ if (xe->wedged.mode == wedged_mode)
	153	+ return 0;
	154	+
	155	+ xe->wedged.mode = wedged_mode;
	156	+
	157	+ xe_pm_runtime_get(xe);
	158	+ for_each_gt(gt, xe, id) {
	159	+ ret = xe_guc_ads_scheduler_policy_toggle_reset(&gt->uc.guc.ads);
	160	+ if (ret) {
	161	+ xe_gt_err(gt, "Failed to update GuC ADS scheduler policy. GuC may still cause engine reset even with wedged_mode=2\n");
	162	+ return -EIO;
	163	+ }
	164	+ }
	165	+ xe_pm_runtime_put(xe);
	166	+
	167	+ return size;
	168	+}
	169	+
	170	+static const struct file_operations wedged_mode_fops = {
	171	+ .owner = THIS_MODULE,
	172	+ .read = wedged_mode_show,
	173	+ .write = wedged_mode_set,
	174	+};
	175	+
121	176	void xe_debugfs_register(struct xe_device *xe)
122	177	{
123	178	struct ttm_device *bdev = &xe->ttm;
@@ -135,6 +190,9 @@ void xe_debugfs_register(struct xe_device *xe) · 11 unmodified lines
135	190	debugfs_create_file("forcewake_all", 0400, root, xe,
136	191	&forcewake_all_fops);
137	192
	193	+ debugfs_create_file("wedged_mode", 0400, root, xe,
	194	+ &wedged_mode_fops);
	195	+
138	196	for (mem_type = XE_PL_VRAM0; mem_type <= XE_PL_VRAM1; ++mem_type) {
139	197	man = ttm_manager_type(bdev, mem_type);
140	198

drivers/gpu/drm/xe/xe_devcoredump.c+17−2

@@ -110,6 +110,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset, · 109 unmodified lines
110	110	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
111	111	ts = ktime_to_timespec64(ss->boot_time);
112	112	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
	113	+ drm_printf(&p, "Process: %s\n", ss->process_name);
113	114	xe_device_snapshot_print(xe, &p);
114	115
115	116	drm_printf(&p, "\n** GuC CT **\n");
@@ -166,12 +167,24 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump, · 50 unmodified lines
166	167	enum xe_hw_engine_id id;
167	168	u32 adj_logical_mask = q->logical_mask;
168	169	u32 width_mask = (0x1 << q->width) - 1;
	170	+ const char *process_name = "no process";
	171	+ struct task_struct *task = NULL;
	172	+
169	173	int i;
170	174	bool cookie;
171	175
172	176	ss->snapshot_time = ktime_get_real();
173	177	ss->boot_time = ktime_get_boottime();
174	178
	179	+ if (q->vm && q->vm->xef) {
	180	+ task = get_pid_task(q->vm->xef->drm->pid, PIDTYPE_PID);
	181	+ if (task)
	182	+ process_name = task->comm;
	183	+ }
	184	+ strscpy(ss->process_name, process_name);
	185	+ if (task)
	186	+ put_task_struct(task);
	187	+
175	188	ss->gt = q->gt;
176	189	INIT_WORK(&ss->work, xe_devcoredump_deferred_snap_work);
177	190
@@ -238,13 +251,15 @@ void xe_devcoredump(struct xe_sched_job *job) · 60 unmodified lines
238	251	xe_devcoredump_read, xe_devcoredump_free);
239	252	}
240	253
241		−static void xe_driver_devcoredump_fini(struct drm_device drm, void arg)
	254	+static void xe_driver_devcoredump_fini(void *arg)
242	255	{
	256	+ struct drm_device *drm = arg;
	257	+
243	258	dev_coredump_put(drm->dev);
244	259	}
245	260
246	261	int xe_devcoredump_init(struct xe_device *xe)
247	262	{
248		− return drmm_add_action_or_reset(&xe->drm, xe_driver_devcoredump_fini, xe);
	263	+ return devm_add_action_or_reset(xe->drm.dev, xe_driver_devcoredump_fini, &xe->drm);
249	264	}
250	265	#endif

drivers/gpu/drm/xe/xe_devcoredump_types.h+2−0

@@ -26,6 +26,8 @@ struct xe_devcoredump_snapshot { · 25 unmodified lines
26	26	ktime_t snapshot_time;
27	27	/** @boot_time: Relative boot time so the uptime can be calculated. */
28	28	ktime_t boot_time;
	29	+ /** @process_name: Name of process that triggered this gpu hang */
	30	+ char process_name[TASK_COMM_LEN];
29	31
30	32	/** @gt: Affected GT, used by forcewake for delayed capture */
31	33	struct xe_gt *gt;

drivers/gpu/drm/xe/xe_device.c+78−29

@@ -5,6 +5,7 @@ · 4 unmodified lines
5	5
6	6	#include "xe_device.h"
7	7
	8	+#include <linux/delay.h>
8	9	#include <linux/units.h>
9	10
10	11	#include <drm/drm_aperture.h>
@@ -17,6 +18,7 @@ · 6 unmodified lines
17	18	#include <drm/xe_drm.h>
18	19
19	20	#include "display/xe_display.h"
	21	+#include "instructions/xe_gpu_commands.h"
20	22	#include "regs/xe_gt_regs.h"
21	23	#include "regs/xe_regs.h"
22	24	#include "xe_bo.h"
@@ -27,10 +29,14 @@ · 4 unmodified lines
27	29	#include "xe_drv.h"
28	30	#include "xe_exec.h"
29	31	#include "xe_exec_queue.h"
	32	+#include "xe_force_wake.h"
30	33	#include "xe_ggtt.h"
31	34	#include "xe_gsc_proxy.h"
32	35	#include "xe_gt.h"
33	36	#include "xe_gt_mcr.h"
	37	+#include "xe_gt_printk.h"
	38	+#include "xe_gt_sriov_vf.h"
	39	+#include "xe_guc.h"
34	40	#include "xe_hwmon.h"
35	41	#include "xe_irq.h"
36	42	#include "xe_memirq.h"
@@ -45,6 +51,7 @@ · 8 unmodified lines
45	51	#include "xe_ttm_stolen_mgr.h"
46	52	#include "xe_ttm_sys_mgr.h"
47	53	#include "xe_vm.h"
	54	+#include "xe_vram.h"
48	55	#include "xe_wait_user_fence.h"
49	56
50	57	static int xe_file_open(struct drm_device dev, struct drm_file file)
@@ -90,12 +97,16 @@ static void xe_file_close(struct drm_device dev, struct drm_file file) · 39 unmodified lines
90	97	struct xe_exec_queue *q;
91	98	unsigned long idx;
92	99
93		− mutex_lock(&xef->exec_queue.lock);
	100	+ /*
	101	+ * No need for exec_queue.lock here as there is no contention for it
	102	+ * when FD is closing as IOCTLs presumably can't be modifying the
	103	+ * xarray. Taking exec_queue.lock here causes undue dependency on
	104	+ * vm->lock taken during xe_exec_queue_kill().
	105	+ */
94	106	xa_for_each(&xef->exec_queue.xa, idx, q) {
95	107	xe_exec_queue_kill(q);
96	108	xe_exec_queue_put(q);
97	109	}
98		− mutex_unlock(&xef->exec_queue.lock);
99	110	xa_destroy(&xef->exec_queue.xa);
100	111	mutex_destroy(&xef->exec_queue.lock);
101	112	mutex_lock(&xef->vm.lock);
@@ -138,6 +149,9 @@ static long xe_drm_ioctl(struct file *file, unsigned int cmd, unsigned long arg) · 36 unmodified lines
138	149	struct xe_device *xe = to_xe_device(file_priv->minor->dev);
139	150	long ret;
140	151
	152	+ if (xe_device_wedged(xe))
	153	+ return -ECANCELED;
	154	+
141	155	ret = xe_pm_runtime_get_ioctl(xe);
142	156	if (ret >= 0)
143	157	ret = drm_ioctl(file, cmd, arg);
@@ -153,6 +167,9 @@ static long xe_drm_compat_ioctl(struct file *file, unsigned int cmd, unsigned lo · 9 unmodified lines
153	167	struct xe_device *xe = to_xe_device(file_priv->minor->dev);
154	168	long ret;
155	169
	170	+ if (xe_device_wedged(xe))
	171	+ return -ECANCELED;
	172	+
156	173	ret = xe_pm_runtime_get_ioctl(xe);
157	174	if (ret >= 0)
158	175	ret = drm_compat_ioctl(file, cmd, arg);
@@ -180,13 +197,6 @@ static const struct file_operations xe_driver_fops = { · 21 unmodified lines
180	197	#endif
181	198	};
182	199
183		−static void xe_driver_release(struct drm_device *dev)
184		−{
185		− struct xe_device *xe = to_xe_device(dev);
186		−
187		− pci_set_drvdata(to_pci_dev(xe->drm.dev), NULL);
188		−}
189		−
190	200	static struct drm_driver driver = {
191	201	/* Don't use MTRRs here; the Xserver or userspace app should
192	202	* deal with them for Intel hardware.
@@ -205,8 +215,6 @@ static struct drm_driver driver = { · 12 unmodified lines
205	215	#ifdef CONFIG_PROC_FS
206	216	.show_fdinfo = xe_drm_client_fdinfo,
207	217	#endif
208		− .release = &xe_driver_release,
209		−
210	218	.ioctls = xe_ioctls,
211	219	.num_ioctls = ARRAY_SIZE(xe_ioctls),
212	220	.fops = &xe_driver_fops,
@@ -269,7 +277,10 @@ struct xe_device xe_device_create(struct pci_dev pdev, · 56 unmodified lines
269	277
270	278	init_waitqueue_head(&xe->ufence_wq);
271	279
272		− drmm_mutex_init(&xe->drm, &xe->usm.lock);
	280	+ err = drmm_mutex_init(&xe->drm, &xe->usm.lock);
	281	+ if (err)
	282	+ goto err;
	283	+
273	284	xa_init_flags(&xe->usm.asid_to_vm, XA_FLAGS_ALLOC);
274	285
275	286	if (IS_ENABLED(CONFIG_DRM_XE_DEBUG)) {
@@ -378,7 +389,7 @@ static void xe_driver_flr(struct xe_device *xe) · 102 unmodified lines
378	389	xe_mmio_write32(gt, GU_DEBUG, DRIVERFLR_STATUS);
379	390	}
380	391
381		−static void xe_driver_flr_fini(struct drm_device drm, void arg)
	392	+static void xe_driver_flr_fini(void *arg)
382	393	{
383	394	struct xe_device *xe = arg;
384	395
@@ -386,7 +397,7 @@ static void xe_driver_flr_fini(struct drm_device drm, void arg) · 1 unmodified line
386	397	xe_driver_flr(xe);
387	398	}
388	399
389		−static void xe_device_sanitize(struct drm_device drm, void arg)
	400	+static void xe_device_sanitize(void *arg)
390	401	{
391	402	struct xe_device *xe = arg;
392	403	struct xe_gt *gt;
@@ -501,6 +512,8 @@ int xe_device_probe_early(struct xe_device *xe) · 108 unmodified lines
501	512	if (err)
502	513	return err;
503	514
	515	+ xe->wedged.mode = xe_modparam.wedged_mode;
	516	+
504	517	return 0;
505	518	}
506	519
@@ -551,14 +564,28 @@ int xe_device_probe(struct xe_device *xe) · 44 unmodified lines
551	564	if (err)
552	565	return err;
553	566
554		− xe_mmio_probe_tiles(xe);
	567	+ err = xe_mmio_probe_tiles(xe);
	568	+ if (err)
	569	+ return err;
555	570
556	571	xe_ttm_sys_mgr_init(xe);
557	572
558		− for_each_gt(gt, xe, id)
559		− xe_force_wake_init_gt(gt, gt_to_fw(gt));
	573	+ for_each_gt(gt, xe, id) {
	574	+ err = xe_gt_init_early(gt);
	575	+ if (err)
	576	+ return err;
	577	+ }
560	578
561	579	for_each_tile(tile, xe, id) {
	580	+ if (IS_SRIOV_VF(xe)) {
	581	+ xe_guc_comm_init_early(&tile->primary_gt->uc.guc);
	582	+ err = xe_gt_sriov_vf_bootstrap(tile->primary_gt);
	583	+ if (err)
	584	+ return err;
	585	+ err = xe_gt_sriov_vf_query_config(tile->primary_gt);
	586	+ if (err)
	587	+ return err;
	588	+ }
562	589	err = xe_ggtt_init_early(tile->mem.ggtt);
563	590	if (err)
564	591	return err;
@@ -578,13 +605,10 @@ int xe_device_probe(struct xe_device *xe) · 13 unmodified lines
578	605	err = xe_devcoredump_init(xe);
579	606	if (err)
580	607	return err;
581		− err = drmm_add_action_or_reset(&xe->drm, xe_driver_flr_fini, xe);
	608	+ err = devm_add_action_or_reset(xe->drm.dev, xe_driver_flr_fini, xe);
582	609	if (err)
583	610	return err;
584	611
585		− for_each_gt(gt, xe, id)
586		− xe_pcode_init(gt);
587		−
588	612	err = xe_display_init_noirq(xe);
589	613	if (err)
590	614	return err;
@@ -593,17 +617,11 @@ int xe_device_probe(struct xe_device *xe) · 2 unmodified lines
593	617	if (err)
594	618	goto err;
595	619
596		− for_each_gt(gt, xe, id) {
597		− err = xe_gt_init_early(gt);
598		− if (err)
599		− goto err_irq_shutdown;
600		− }
601		−
602	620	err = xe_device_set_has_flat_ccs(xe);
603	621	if (err)
604	622	goto err_irq_shutdown;
605	623
606		− err = xe_mmio_probe_vram(xe);
	624	+ err = xe_vram_probe(xe);
607	625	if (err)
608	626	goto err_irq_shutdown;
609	627
@@ -650,7 +668,7 @@ int xe_device_probe(struct xe_device *xe) · 40 unmodified lines
650	668
651	669	xe_hwmon_register(xe);
652	670
653		− return drmm_add_action_or_reset(&xe->drm, xe_device_sanitize, xe);
	671	+ return devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
654	672
655	673	err_fini_display:
656	674	xe_display_driver_remove(xe);
@@ -759,3 +777,34 @@ u64 xe_device_uncanonicalize_addr(struct xe_device *xe, u64 address) · 102 unmodified lines
759	777	{
760	778	return address & GENMASK_ULL(xe->info.va_bits - 1, 0);
761	779	}
	780	+
	781	+/**
	782	+ * xe_device_declare_wedged - Declare device wedged
	783	+ * @xe: xe device instance
	784	+ *
	785	+ * This is a final state that can only be cleared with a mudule
	786	+ * re-probe (unbind + bind).
	787	+ * In this state every IOCTL will be blocked so the GT cannot be used.
	788	+ * In general it will be called upon any critical error such as gt reset
	789	+ * failure or guc loading failure.
	790	+ * If xe.wedged module parameter is set to 2, this function will be called
	791	+ * on every single execution timeout (a.k.a. GPU hang) right after devcoredump
	792	+ * snapshot capture. In this mode, GT reset won't be attempted so the state of
	793	+ * the issue is preserved for further debugging.
	794	+ */
	795	+void xe_device_declare_wedged(struct xe_device *xe)
	796	+{
	797	+ if (xe->wedged.mode == 0) {
	798	+ drm_dbg(&xe->drm, "Wedged mode is forcibly disabled\n");
	799	+ return;
	800	+ }
	801	+
	802	+ if (!atomic_xchg(&xe->wedged.flag, 1)) {
	803	+ xe->needs_flr_on_fini = true;
	804	+ drm_err(&xe->drm,
	805	+ "CRITICAL: Xe has declared device %s as wedged.\n"
	806	+ "IOCTLs and executions are blocked. Only a rebind may clear the failure\n"
	807	+ "Please file a _new_ bug report at https://gitlab.freedesktop.org/drm/xe/kernel/issues/new\n",
	808	+ dev_name(xe->drm.dev));
	809	+ }
	810	+}

drivers/gpu/drm/xe/xe_device.h+7−6

@@ -6,15 +6,9 @@ · 5 unmodified lines
6	6	#ifndef _XE_DEVICE_H_
7	7	#define _XE_DEVICE_H_
8	8
9		−struct xe_exec_queue;
10		−struct xe_file;
11		−
12	9	#include <drm/drm_util.h>
13	10
14		−#include "regs/xe_gpu_commands.h"
15	11	#include "xe_device_types.h"
16		−#include "xe_force_wake.h"
17		−#include "xe_macros.h"
18	12
19	13	static inline struct xe_device to_xe_device(const struct drm_device dev)
20	14	{
@@ -167,4 +161,11 @@ void xe_device_snapshot_print(struct xe_device xe, struct drm_printer p); · 146 unmodified lines
167	161	u64 xe_device_canonicalize_addr(struct xe_device *xe, u64 address);
168	162	u64 xe_device_uncanonicalize_addr(struct xe_device *xe, u64 address);
169	163
	164	+static inline bool xe_device_wedged(struct xe_device *xe)
	165	+{
	166	+ return atomic_read(&xe->wedged.flag);
	167	+}
	168	+
	169	+void xe_device_declare_wedged(struct xe_device *xe);
	170	+
170	171	#endif

drivers/gpu/drm/xe/xe_device_sysfs.c+2−2

@@ -69,7 +69,7 @@ vram_d3cold_threshold_store(struct device dev, struct device_attribute attr, · 68 unmodified lines
69	69
70	70	static DEVICE_ATTR_RW(vram_d3cold_threshold);
71	71
72		−static void xe_device_sysfs_fini(struct drm_device drm, void arg)
	72	+static void xe_device_sysfs_fini(void *arg)
73	73	{
74	74	struct xe_device *xe = arg;
75	75
@@ -85,5 +85,5 @@ int xe_device_sysfs_init(struct xe_device *xe) · 9 unmodified lines
85	85	if (ret)
86	86	return ret;
87	87
88		− return drmm_add_action_or_reset(&xe->drm, xe_device_sysfs_fini, xe);
	88	+ return devm_add_action_or_reset(dev, xe_device_sysfs_fini, xe);
89	89	}

drivers/gpu/drm/xe/xe_device_types.h+20−3

@@ -196,6 +196,9 @@ struct xe_tile { · 195 unmodified lines
196	196	struct {
197	197	/** @sriov.vf.memirq: Memory Based Interrupts. */
198	198	struct xe_memirq memirq;
	199	+
	200	+ /** @sriov.vf.ggtt_balloon: GGTT regions excluded from use. */
	201	+ struct drm_mm_node ggtt_balloon[2];
199	202	} vf;
200	203	} sriov;
201	204
@@ -218,6 +221,8 @@ struct xe_device { · 16 unmodified lines
218	221
219	222	/** @info: device info */
220	223	struct intel_device_info {
	224	+ /** @info.platform_name: platform name */
	225	+ const char *platform_name;
221	226	/** @info.graphics_name: graphics IP name */
222	227	const char *graphics_name;
223	228	/** @info.media_name: media IP name */
@@ -281,6 +286,10 @@ struct xe_device { · 57 unmodified lines
281	286	u8 has_heci_gscfi:1;
282	287	/** @info.skip_guc_pc: Skip GuC based PM feature init */
283	288	u8 skip_guc_pc:1;
	289	+ /** @info.has_atomic_enable_pte_bit: Device has atomic enable PTE bit */
	290	+ u8 has_atomic_enable_pte_bit:1;
	291	+ /** @info.has_device_atomics_on_smem: Supports device atomics on SMEM */
	292	+ u8 has_device_atomics_on_smem:1;
284	293
285	294	#if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
286	295	struct {
@@ -427,9 +436,6 @@ struct xe_device { · 140 unmodified lines
427	436	/** @d3cold.allowed: Indicates if d3cold is a valid device state */
428	437	bool allowed;
429	438
430		− /** @d3cold.power_lost: Indicates if card has really lost power. */
431		− bool power_lost;
432		−
433	439	/**
434	440	* @d3cold.vram_threshold:
435	441	*
@@ -459,6 +465,14 @@ struct xe_device { · 23 unmodified lines
459	465	/** @needs_flr_on_fini: requests function-reset on fini */
460	466	bool needs_flr_on_fini;
461	467
	468	+ /** @wedged: Struct to control Wedged States and mode */
	469	+ struct {
	470	+ /** @wedged.flag: Xe device faced a critical error and is now blocked. */
	471	+ atomic_t flag;
	472	+ /** @wedged.mode: Mode controlled by kernel parameter and debugfs */
	473	+ int mode;
	474	+ } wedged;
	475	+
462	476	/* private: */
463	477
464	478	#if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
@@ -547,6 +561,9 @@ struct xe_file { · 82 unmodified lines
547	561	struct mutex lock;
548	562	} exec_queue;
549	563
	564	+ /** @run_ticks: hw engine class run time in ticks for this drm client */
	565	+ u64 run_ticks[XE_ENGINE_CLASS_MAX];
	566	+
550	567	/** @client: drm client */
551	568	struct xe_drm_client *client;
552	569	};

drivers/gpu/drm/xe/xe_drm_client.c+123−1

@@ -2,6 +2,7 @@ · 1 unmodified line
2	2	/*
3	3	* Copyright © 2023 Intel Corporation
4	4	*/
	5	+#include "xe_drm_client.h"
5	6
6	7	#include <drm/drm_print.h>
7	8	#include <drm/xe_drm.h>
@@ -12,9 +13,66 @@ · 4 unmodified lines
12	13	#include "xe_bo.h"
13	14	#include "xe_bo_types.h"
14	15	#include "xe_device_types.h"
15		−#include "xe_drm_client.h"
	16	+#include "xe_exec_queue.h"
	17	+#include "xe_force_wake.h"
	18	+#include "xe_gt.h"
	19	+#include "xe_hw_engine.h"
	20	+#include "xe_pm.h"
16	21	#include "xe_trace.h"
17	22
	23	+/**
	24	+ * DOC: DRM Client usage stats
	25	+ *
	26	+ * The drm/xe driver implements the DRM client usage stats specification as
	27	+ * documented in :ref:`drm-client-usage-stats`.
	28	+ *
	29	+ * Example of the output showing the implemented key value pairs and entirety of
	30	+ * the currently possible format options:
	31	+ *
	32	+ * ::
	33	+ *
	34	+ * pos: 0
	35	+ * flags: 0100002
	36	+ * mnt_id: 26
	37	+ * ino: 685
	38	+ * drm-driver: xe
	39	+ * drm-client-id: 3
	40	+ * drm-pdev: 0000:03:00.0
	41	+ * drm-total-system: 0
	42	+ * drm-shared-system: 0
	43	+ * drm-active-system: 0
	44	+ * drm-resident-system: 0
	45	+ * drm-purgeable-system: 0
	46	+ * drm-total-gtt: 192 KiB
	47	+ * drm-shared-gtt: 0
	48	+ * drm-active-gtt: 0
	49	+ * drm-resident-gtt: 192 KiB
	50	+ * drm-total-vram0: 23992 KiB
	51	+ * drm-shared-vram0: 16 MiB
	52	+ * drm-active-vram0: 0
	53	+ * drm-resident-vram0: 23992 KiB
	54	+ * drm-total-stolen: 0
	55	+ * drm-shared-stolen: 0
	56	+ * drm-active-stolen: 0
	57	+ * drm-resident-stolen: 0
	58	+ * drm-cycles-rcs: 28257900
	59	+ * drm-total-cycles-rcs: 7655183225
	60	+ * drm-cycles-bcs: 0
	61	+ * drm-total-cycles-bcs: 7655183225
	62	+ * drm-cycles-vcs: 0
	63	+ * drm-total-cycles-vcs: 7655183225
	64	+ * drm-engine-capacity-vcs: 2
	65	+ * drm-cycles-vecs: 0
	66	+ * drm-total-cycles-vecs: 7655183225
	67	+ * drm-engine-capacity-vecs: 2
	68	+ * drm-cycles-ccs: 0
	69	+ * drm-total-cycles-ccs: 7655183225
	70	+ * drm-engine-capacity-ccs: 4
	71	+ *
	72	+ * Possible `drm-cycles-` key names are: `rcs`, `ccs`, `bcs`, `vcs`, `vecs` and
	73	+ * "other".
	74	+ */
	75	+
18	76	/**
19	77	* xe_drm_client_alloc() - Allocate drm client
20	78	* @void: No arg
@@ -179,6 +237,69 @@ static void show_meminfo(struct drm_printer p, struct drm_file file) · 158 unmodified lines
179	237	}
180	238	}
181	239
	240	+static void show_run_ticks(struct drm_printer p, struct drm_file file)
	241	+{
	242	+ unsigned long class, i, gt_id, capacity[XE_ENGINE_CLASS_MAX] = { };
	243	+ struct xe_file *xef = file->driver_priv;
	244	+ struct xe_device *xe = xef->xe;
	245	+ struct xe_gt *gt;
	246	+ struct xe_hw_engine *hwe;
	247	+ struct xe_exec_queue *q;
	248	+ u64 gpu_timestamp;
	249	+
	250	+ xe_pm_runtime_get(xe);
	251	+
	252	+ /* Accumulate all the exec queues from this client */
	253	+ mutex_lock(&xef->exec_queue.lock);
	254	+ xa_for_each(&xef->exec_queue.xa, i, q) {
	255	+ xe_exec_queue_update_run_ticks(q);
	256	+ xef->run_ticks[q->class] += q->run_ticks - q->old_run_ticks;
	257	+ q->old_run_ticks = q->run_ticks;
	258	+ }
	259	+ mutex_unlock(&xef->exec_queue.lock);
	260	+
	261	+ /* Get the total GPU cycles */
	262	+ for_each_gt(gt, xe, gt_id) {
	263	+ hwe = xe_gt_any_hw_engine(gt);
	264	+ if (!hwe)
	265	+ continue;
	266	+
	267	+ xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
	268	+ gpu_timestamp = xe_hw_engine_read_timestamp(hwe);
	269	+ xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
	270	+ break;
	271	+ }
	272	+
	273	+ xe_pm_runtime_put(xe);
	274	+
	275	+ if (unlikely(!hwe))
	276	+ return;
	277	+
	278	+ for (class = 0; class < XE_ENGINE_CLASS_MAX; class++) {
	279	+ const char *class_name;
	280	+
	281	+ for_each_gt(gt, xe, gt_id)
	282	+ capacity[class] += gt->user_engines.instances_per_class[class];
	283	+
	284	+ /*
	285	+ * Engines may be fused off or not exposed to userspace. Don't
	286	+ * return anything if this entire class is not available
	287	+ */
	288	+ if (!capacity[class])
	289	+ continue;
	290	+
	291	+ class_name = xe_hw_engine_class_to_str(class);
	292	+ drm_printf(p, "drm-cycles-%s:\t%llu\n",
	293	+ class_name, xef->run_ticks[class]);
	294	+ drm_printf(p, "drm-total-cycles-%s:\t%llu\n",
	295	+ class_name, gpu_timestamp);
	296	+
	297	+ if (capacity[class] > 1)
	298	+ drm_printf(p, "drm-engine-capacity-%s:\t%lu\n",
	299	+ class_name, capacity[class]);
	300	+ }
	301	+}
	302	+
182	303	/**
183	304	* xe_drm_client_fdinfo() - Callback for fdinfo interface
184	305	* @p: The drm_printer ptr
@@ -192,5 +313,6 @@ static void show_meminfo(struct drm_printer p, struct drm_file file) · 7 unmodified lines
192	313	void xe_drm_client_fdinfo(struct drm_printer p, struct drm_file file)
193	314	{
194	315	show_meminfo(p, file);
	316	+ show_run_ticks(p, file);
195	317	}
196	318	#endif

drivers/gpu/drm/xe/xe_exec_queue.c+47−30

@@ -86,7 +86,7 @@ static struct xe_exec_queue __xe_exec_queue_alloc(struct xe_device xe, · 85 unmodified lines
86	86
87	87	if (extensions) {
88	88	/*
89		− * may set q->usm, must come before xe_lrc_init(),
	89	+ * may set q->usm, must come before xe_lrc_create(),
90	90	* may overwrite q->sched_props, must come before q->ops->init()
91	91	*/
92	92	err = exec_queue_user_extensions(xe, q, extensions, 0);
@@ -96,45 +96,30 @@ static struct xe_exec_queue __xe_exec_queue_alloc(struct xe_device xe, · 3 unmodified lines
96	96	}
97	97	}
98	98
99		− if (xe_exec_queue_is_parallel(q)) {
100		− q->parallel.composite_fence_ctx = dma_fence_context_alloc(1);
101		− q->parallel.composite_fence_seqno = XE_FENCE_INITIAL_SEQNO;
102		− }
103		−
104	99	return q;
105	100	}
106	101
107	102	static int __xe_exec_queue_init(struct xe_exec_queue *q)
108	103	{
109		− struct xe_device *xe = gt_to_xe(q->gt);
110	104	int i, err;
111	105
112	106	for (i = 0; i < q->width; ++i) {
113		− err = xe_lrc_init(q->lrc + i, q->hwe, q, q->vm, SZ_16K);
114		− if (err)
	107	+ q->lrc[i] = xe_lrc_create(q->hwe, q->vm, SZ_16K);
	108	+ if (IS_ERR(q->lrc[i])) {
	109	+ err = PTR_ERR(q->lrc[i]);
115	110	goto err_lrc;
	111	+ }
116	112	}
117	113
118	114	err = q->ops->init(q);
119	115	if (err)
120	116	goto err_lrc;
121	117
122		− /*
123		− * Normally the user vm holds an rpm ref to keep the device
124		− * awake, and the context holds a ref for the vm, however for
125		− * some engines we use the kernels migrate vm underneath which offers no
126		− * such rpm ref, or we lack a vm. Make sure we keep a ref here, so we
127		− * can perform GuC CT actions when needed. Caller is expected to have
128		− * already grabbed the rpm ref outside any sensitive locks.
129		− */
130		− if (!(q->flags & EXEC_QUEUE_FLAG_PERMANENT) && (q->flags & EXEC_QUEUE_FLAG_VM \|\| !q->vm))
131		− xe_pm_runtime_get_noresume(xe);
132		−
133	118	return 0;
134	119
135	120	err_lrc:
136	121	for (i = i - 1; i >= 0; --i)
137		− xe_lrc_finish(q->lrc + i);
	122	+ xe_lrc_put(q->lrc[i]);
138	123	return err;
139	124	}
140	125
@@ -215,9 +200,7 @@ void xe_exec_queue_fini(struct xe_exec_queue *q) · 74 unmodified lines
215	200	int i;
216	201
217	202	for (i = 0; i < q->width; ++i)
218		− xe_lrc_finish(q->lrc + i);
219		− if (!(q->flags & EXEC_QUEUE_FLAG_PERMANENT) && (q->flags & EXEC_QUEUE_FLAG_VM \|\| !q->vm))
220		− xe_pm_runtime_put(gt_to_xe(q->gt));
	203	+ xe_lrc_put(q->lrc[i]);
221	204	__xe_exec_queue_free(q);
222	205	}
223	206
@@ -720,7 +703,7 @@ bool xe_exec_queue_is_lr(struct xe_exec_queue *q) · 496 unmodified lines
720	703
721	704	static s32 xe_exec_queue_num_job_inflight(struct xe_exec_queue *q)
722	705	{
723		− return q->lrc->fence_ctx.next_seqno - xe_lrc_seqno(q->lrc) - 1;
	706	+ return q->lrc[0]->fence_ctx.next_seqno - xe_lrc_seqno(q->lrc[0]) - 1;
724	707	}
725	708
726	709	/**
@@ -731,7 +714,7 @@ static s32 xe_exec_queue_num_job_inflight(struct xe_exec_queue *q) · 4 unmodified lines
731	714	*/
732	715	bool xe_exec_queue_ring_full(struct xe_exec_queue *q)
733	716	{
734		− struct xe_lrc *lrc = q->lrc;
	717	+ struct xe_lrc *lrc = q->lrc[0];
735	718	s32 max_job = lrc->ring.size / MAX_JOB_SIZE_BYTES;
736	719
737	720	return xe_exec_queue_num_job_inflight(q) >= max_job;
@@ -757,16 +740,50 @@ bool xe_exec_queue_is_idle(struct xe_exec_queue *q) · 19 unmodified lines
757	740	int i;
758	741
759	742	for (i = 0; i < q->width; ++i) {
760		− if (xe_lrc_seqno(&q->lrc[i]) !=
761		− q->lrc[i].fence_ctx.next_seqno - 1)
	743	+ if (xe_lrc_seqno(q->lrc[i]) !=
	744	+ q->lrc[i]->fence_ctx.next_seqno - 1)
762	745	return false;
763	746	}
764	747
765	748	return true;
766	749	}
767	750
768		− return xe_lrc_seqno(&q->lrc[0]) ==
769		− q->lrc[0].fence_ctx.next_seqno - 1;
	751	+ return xe_lrc_seqno(q->lrc[0]) ==
	752	+ q->lrc[0]->fence_ctx.next_seqno - 1;
	753	+}
	754	+
	755	+/**
	756	+ * xe_exec_queue_update_run_ticks() - Update run time in ticks for this exec queue
	757	+ * from hw
	758	+ * @q: The exec queue
	759	+ *
	760	+ * Update the timestamp saved by HW for this exec queue and save run ticks
	761	+ * calculated by using the delta from last update.
	762	+ */
	763	+void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q)
	764	+{
	765	+ struct xe_lrc *lrc;
	766	+ u32 old_ts, new_ts;
	767	+
	768	+ /*
	769	+ * Jobs that are run during driver load may use an exec_queue, but are
	770	+ * not associated with a user xe file, so avoid accumulating busyness
	771	+ * for kernel specific work.
	772	+ */
	773	+ if (!q->vm \|\| !q->vm->xef)
	774	+ return;
	775	+
	776	+ /*
	777	+ * Only sample the first LRC. For parallel submission, all of them are
	778	+ * scheduled together and we compensate that below by multiplying by
	779	+ * width - this may introduce errors if that premise is not true and
	780	+ * they don't exit 100% aligned. On the other hand, looping through
	781	+ * the LRCs and reading them in different time could also introduce
	782	+ * errors.
	783	+ */
	784	+ lrc = q->lrc[0];
	785	+ new_ts = xe_lrc_update_timestamp(lrc, &old_ts);
	786	+ q->run_ticks += (new_ts - old_ts) * q->width;
770	787	}
771	788
772	789	void xe_exec_queue_kill(struct xe_exec_queue *q)

drivers/gpu/drm/xe/xe_exec_queue.h+10−0

@@ -26,6 +26,15 @@ void xe_exec_queue_fini(struct xe_exec_queue *q); · 25 unmodified lines
26	26	void xe_exec_queue_destroy(struct kref *ref);
27	27	void xe_exec_queue_assign_name(struct xe_exec_queue *q, u32 instance);
28	28
	29	+static inline struct xe_exec_queue *
	30	+xe_exec_queue_get_unless_zero(struct xe_exec_queue *q)
	31	+{
	32	+ if (kref_get_unless_zero(&q->refcount))
	33	+ return q;
	34	+
	35	+ return NULL;
	36	+}
	37	+
29	38	struct xe_exec_queue xe_exec_queue_lookup(struct xe_file xef, u32 id);
30	39
31	40	static inline struct xe_exec_queue xe_exec_queue_get(struct xe_exec_queue q)
@@ -66,5 +75,6 @@ struct dma_fence xe_exec_queue_last_fence_get(struct xe_exec_queue e, · 34 unmodified lines
66	75	struct xe_vm *vm);
67	76	void xe_exec_queue_last_fence_set(struct xe_exec_queue e, struct xe_vm vm,
68	77	struct dma_fence *fence);
	78	+void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q);
69	79
70	80	#endif

drivers/gpu/drm/xe/xe_exec_queue_types.h+5−11

@@ -103,16 +103,6 @@ struct xe_exec_queue { · 102 unmodified lines
103	103	struct xe_guc_exec_queue *guc;
104	104	};
105	105
106		− /**
107		− * @parallel: parallel submission state
108		− */
109		− struct {
110		− /** @parallel.composite_fence_ctx: context composite fence */
111		− u64 composite_fence_ctx;
112		− /** @parallel.composite_fence_seqno: seqno for composite fence */
113		− u32 composite_fence_seqno;
114		− } parallel;
115		−
116	106	/** @sched_props: scheduling properties */
117	107	struct {
118	108	/** @sched_props.timeslice_us: timeslice period in micro-seconds */
@@ -151,8 +141,12 @@ struct xe_exec_queue { · 32 unmodified lines
151	141	* Protected by @vm's resv. Unused if @vm == NULL.
152	142	*/
153	143	u64 tlb_flush_seqno;
	144	+ /** @old_run_ticks: prior hw engine class run time in ticks for this exec queue */
	145	+ u64 old_run_ticks;
	146	+ /** @run_ticks: hw engine class run time in ticks for this exec queue */
	147	+ u64 run_ticks;
154	148	/** @lrc: logical ring context for this exec queue */
155		− struct xe_lrc lrc[];
	149	+ struct xe_lrc *lrc[];
156	150	};
157	151
158	152	/**

drivers/gpu/drm/xe/xe_execlist.c+6−6

@@ -9,7 +9,6 @@ · 8 unmodified lines
9	9
10	10	#include "instructions/xe_mi_commands.h"
11	11	#include "regs/xe_engine_regs.h"
12		−#include "regs/xe_gpu_commands.h"
13	12	#include "regs/xe_gt_regs.h"
14	13	#include "regs/xe_lrc_layout.h"
15	14	#include "xe_assert.h"
@@ -110,7 +109,7 @@ static void __xe_execlist_port_start(struct xe_execlist_port *port, · 94 unmodified lines
110	109	port->last_ctx_id = 1;
111	110	}
112	111
113		− __start_lrc(port->hwe, exl->q->lrc, port->last_ctx_id);
	112	+ __start_lrc(port->hwe, exl->q->lrc[0], port->last_ctx_id);
114	113	port->running_exl = exl;
115	114	exl->has_run = true;
116	115	}
@@ -124,14 +123,14 @@ static void __xe_execlist_port_idle(struct xe_execlist_port *port) · 7 unmodified lines
124	123	if (!port->running_exl)
125	124	return;
126	125
127		− xe_lrc_write_ring(&port->hwe->kernel_lrc, noop, sizeof(noop));
128		− __start_lrc(port->hwe, &port->hwe->kernel_lrc, 0);
	126	+ xe_lrc_write_ring(port->hwe->kernel_lrc, noop, sizeof(noop));
	127	+ __start_lrc(port->hwe, port->hwe->kernel_lrc, 0);
129	128	port->running_exl = NULL;
130	129	}
131	130
132	131	static bool xe_execlist_is_idle(struct xe_execlist_exec_queue *exl)
133	132	{
134		− struct xe_lrc *lrc = exl->q->lrc;
	133	+ struct xe_lrc *lrc = exl->q->lrc[0];
135	134
136	135	return lrc->ring.tail == lrc->ring.old_tail;
137	136	}
@@ -307,6 +306,7 @@ static void execlist_job_free(struct drm_sched_job *drm_job) · 169 unmodified lines
307	306	{
308	307	struct xe_sched_job *job = to_xe_sched_job(drm_job);
309	308
	309	+ xe_exec_queue_update_run_ticks(job->q);
310	310	xe_sched_job_put(job);
311	311	}
312	312
@@ -333,7 +333,7 @@ static int execlist_exec_queue_init(struct xe_exec_queue *q) · 20 unmodified lines
333	333	exl->q = q;
334	334
335	335	err = drm_sched_init(&exl->sched, &drm_sched_ops, NULL, 1,
336		− q->lrc[0].ring.size / MAX_JOB_SIZE_BYTES,
	336	+ q->lrc[0]->ring.size / MAX_JOB_SIZE_BYTES,
337	337	XE_SCHED_HANG_LIMIT, XE_SCHED_JOB_TIMEOUT,
338	338	NULL, NULL, q->hwe->name,
339	339	gt_to_xe(q->gt)->drm.dev);

drivers/gpu/drm/xe/xe_ggtt.c+28−4

@@ -8,6 +8,7 @@ · 7 unmodified lines
8	8	#include <linux/io-64-nonatomic-lo-hi.h>
9	9	#include <linux/sizes.h>
10	10
	11	+#include <drm/drm_drv.h>
11	12	#include <drm/drm_managed.h>
12	13	#include <drm/i915_drm.h>
13	14
@@ -19,6 +20,7 @@ · 5 unmodified lines
19	20	#include "xe_device.h"
20	21	#include "xe_gt.h"
21	22	#include "xe_gt_printk.h"
	23	+#include "xe_gt_sriov_vf.h"
22	24	#include "xe_gt_tlb_invalidation.h"
23	25	#include "xe_map.h"
24	26	#include "xe_pm.h"
@@ -140,6 +142,7 @@ int xe_ggtt_init_early(struct xe_ggtt *ggtt) · 115 unmodified lines
140	142	struct xe_device *xe = tile_to_xe(ggtt->tile);
141	143	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
142	144	unsigned int gsm_size;
	145	+ int err;
143	146
144	147	if (IS_SRIOV_VF(xe))
145	148	gsm_size = SZ_8M; /* GGTT is expected to be 4GiB */
@@ -193,7 +196,17 @@ int xe_ggtt_init_early(struct xe_ggtt *ggtt) · 47 unmodified lines
193	196	mutex_init(&ggtt->lock);
194	197	primelockdep(ggtt);
195	198
196		− return drmm_add_action_or_reset(&xe->drm, ggtt_fini_early, ggtt);
	199	+ err = drmm_add_action_or_reset(&xe->drm, ggtt_fini_early, ggtt);
	200	+ if (err)
	201	+ return err;
	202	+
	203	+ if (IS_SRIOV_VF(xe)) {
	204	+ err = xe_gt_sriov_vf_prepare_ggtt(xe_tile_get_gt(ggtt->tile, 0));
	205	+ if (err)
	206	+ return err;
	207	+ }
	208	+
	209	+ return 0;
197	210	}
198	211
199	212	static void xe_ggtt_invalidate(struct xe_ggtt *ggtt);
@@ -433,18 +446,29 @@ int xe_ggtt_insert_bo(struct xe_ggtt ggtt, struct xe_bo bo) · 233 unmodified lines
433	446	void xe_ggtt_remove_node(struct xe_ggtt ggtt, struct drm_mm_node node,
434	447	bool invalidate)
435	448	{
436		− xe_pm_runtime_get_noresume(tile_to_xe(ggtt->tile));
	449	+ struct xe_device *xe = tile_to_xe(ggtt->tile);
	450	+ bool bound;
	451	+ int idx;
	452	+
	453	+ bound = drm_dev_enter(&xe->drm, &idx);
	454	+ if (bound)
	455	+ xe_pm_runtime_get_noresume(xe);
437	456
438	457	mutex_lock(&ggtt->lock);
439		− xe_ggtt_clear(ggtt, node->start, node->size);
	458	+ if (bound)
	459	+ xe_ggtt_clear(ggtt, node->start, node->size);
440	460	drm_mm_remove_node(node);
441	461	node->size = 0;
442	462	mutex_unlock(&ggtt->lock);
443	463
	464	+ if (!bound)
	465	+ return;
	466	+
444	467	if (invalidate)
445	468	xe_ggtt_invalidate(ggtt);
446	469
447		− xe_pm_runtime_put(tile_to_xe(ggtt->tile));
	470	+ xe_pm_runtime_put(xe);
	471	+ drm_dev_exit(idx);
448	472	}
449	473
450	474	void xe_ggtt_remove_bo(struct xe_ggtt ggtt, struct xe_bo bo)

drivers/gpu/drm/xe/xe_gsc.c+3−0

@@ -5,6 +5,8 @@ · 4 unmodified lines
5	5
6	6	#include "xe_gsc.h"
7	7
	8	+#include <linux/delay.h>
	9	+
8	10	#include <drm/drm_managed.h>
9	11
10	12	#include <generated/xe_wa_oob.h>
@@ -14,6 +16,7 @@ · 3 unmodified lines
14	16	#include "xe_bo.h"
15	17	#include "xe_device.h"
16	18	#include "xe_exec_queue.h"
	19	+#include "xe_force_wake.h"
17	20	#include "xe_gsc_proxy.h"
18	21	#include "xe_gsc_submit.h"
19	22	#include "xe_gt.h"

drivers/gpu/drm/xe/xe_gsc.h+2−1

@@ -6,8 +6,9 @@ · 5 unmodified lines
6	6	#ifndef _XE_GSC_H_
7	7	#define _XE_GSC_H_
8	8
9		−#include "xe_gsc_types.h"
	9	+#include <linux/types.h>
10	10
	11	+struct xe_gsc;
11	12	struct xe_gt;
12	13	struct xe_hw_engine;
13	14

drivers/gpu/drm/xe/xe_gsc_proxy.c+1−0

@@ -15,6 +15,7 @@ · 14 unmodified lines
15	15	#include "abi/gsc_proxy_commands_abi.h"
16	16	#include "regs/xe_gsc_regs.h"
17	17	#include "xe_bo.h"
	18	+#include "xe_force_wake.h"
18	19	#include "xe_gsc.h"
19	20	#include "xe_gsc_submit.h"
20	21	#include "xe_gt.h"

drivers/gpu/drm/xe/xe_gsc_submit.c+1−0

@@ -8,6 +8,7 @@ · 7 unmodified lines
8	8	#include <linux/poison.h>
9	9
10	10	#include "abi/gsc_command_header_abi.h"
	11	+#include "xe_assert.h"
11	12	#include "xe_bb.h"
12	13	#include "xe_exec_queue.h"
13	14	#include "xe_gt_printk.h"

drivers/gpu/drm/xe/xe_gt.c+105−37

@@ -44,6 +44,7 @@ · 43 unmodified lines
44	44	#include "xe_migrate.h"
45	45	#include "xe_mmio.h"
46	46	#include "xe_pat.h"
	47	+#include "xe_pcode.h"
47	48	#include "xe_pm.h"
48	49	#include "xe_mocs.h"
49	50	#include "xe_reg_sr.h"
@@ -57,9 +58,17 @@ · 7 unmodified lines
57	58	#include "xe_wa.h"
58	59	#include "xe_wopcm.h"
59	60
	61	+static void gt_fini(struct drm_device drm, void arg)
	62	+{
	63	+ struct xe_gt *gt = arg;
	64	+
	65	+ destroy_workqueue(gt->ordered_wq);
	66	+}
	67	+
60	68	struct xe_gt xe_gt_alloc(struct xe_tile tile)
61	69	{
62	70	struct xe_gt *gt;
	71	+ int err;
63	72
64	73	gt = drmm_kzalloc(&tile_to_xe(tile)->drm, sizeof(*gt), GFP_KERNEL);
65	74	if (!gt)
@@ -68,6 +77,10 @@ struct xe_gt xe_gt_alloc(struct xe_tile tile) · 2 unmodified lines
68	77	gt->tile = tile;
69	78	gt->ordered_wq = alloc_ordered_workqueue("gt-ordered-wq", 0);
70	79
	80	+ err = drmm_add_action_or_reset(&gt_to_xe(gt)->drm, gt_fini, gt);
	81	+ if (err)
	82	+ return ERR_PTR(err);
	83	+
71	84	return gt;
72	85	}
73	86
@@ -90,15 +103,9 @@ void xe_gt_sanitize(struct xe_gt *gt) · 16 unmodified lines
90	103	*/
91	104	void xe_gt_remove(struct xe_gt *gt)
92	105	{
93		− xe_uc_remove(&gt->uc);
94		−}
95		−
96		−static void gt_fini(struct drm_device drm, void arg)
97		−{
98		− struct xe_gt *gt = arg;
99	106	int i;
100	107
101		− destroy_workqueue(gt->ordered_wq);
	108	+ xe_uc_remove(&gt->uc);
102	109
103	110	for (i = 0; i < XE_ENGINE_CLASS_MAX; ++i)
104	111	xe_hw_fence_irq_finish(&gt->fence_irq[i]);
@@ -160,7 +167,7 @@ static int emit_wa_job(struct xe_gt gt, struct xe_exec_queue q) · 55 unmodified lines
160	167
161	168	if (q->hwe->class == XE_ENGINE_CLASS_RENDER)
162	169	/* Big enough to emit all of the context's 3DSTATE */
163		− bb = xe_bb_new(gt, xe_lrc_size(gt_to_xe(gt), q->hwe->class), false);
	170	+ bb = xe_bb_new(gt, xe_gt_lrc_size(gt, q->hwe->class), false);
164	171	else
165	172	/* Just pick a large BB size */
166	173	bb = xe_bb_new(gt, SZ_4K, false);
@@ -244,7 +251,7 @@ int xe_gt_record_default_lrcs(struct xe_gt *gt) · 77 unmodified lines
244	251	xe_tuning_process_lrc(hwe);
245	252
246	253	default_lrc = drmm_kzalloc(&xe->drm,
247		− xe_lrc_size(xe, hwe->class),
	254	+ xe_gt_lrc_size(gt, hwe->class),
248	255	GFP_KERNEL);
249	256	if (!default_lrc)
250	257	return -ENOMEM;
@@ -292,9 +299,9 @@ int xe_gt_record_default_lrcs(struct xe_gt *gt) · 41 unmodified lines
292	299	}
293	300
294	301	xe_map_memcpy_from(xe, default_lrc,
295		− &q->lrc[0].bo->vmap,
296		− xe_lrc_pphwsp_offset(&q->lrc[0]),
297		− xe_lrc_size(xe, hwe->class));
	302	+ &q->lrc[0]->bo->vmap,
	303	+ xe_lrc_pphwsp_offset(q->lrc[0]),
	304	+ xe_gt_lrc_size(gt, hwe->class));
298	305
299	306	gt->default_lrc[hwe->class] = default_lrc;
300	307	put_nop_q:
@@ -318,14 +325,6 @@ int xe_gt_init_early(struct xe_gt *gt) · 17 unmodified lines
318	325	return err;
319	326	}
320	327
321		− err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
322		− if (err)
323		− return err;
324		−
325		− err = xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
326		− if (err)
327		− return err;
328		−
329	328	xe_reg_sr_init(&gt->reg_sr, "GT", gt_to_xe(gt));
330	329
331	330	err = xe_wa_init(gt);
@@ -336,6 +335,9 @@ int xe_gt_init_early(struct xe_gt *gt) · 4 unmodified lines
336	335	xe_wa_process_oob(gt);
337	336	xe_tuning_process_gt(gt);
338	337
	338	+ xe_force_wake_init_gt(gt, gt_to_fw(gt));
	339	+ xe_pcode_init(gt);
	340	+
339	341	return 0;
340	342	}
341	343
@@ -366,10 +368,6 @@ static int gt_fw_domain_init(struct xe_gt *gt) · 24 unmodified lines
366	368	xe_lmtt_init(&gt_to_tile(gt)->sriov.pf.lmtt);
367	369	}
368	370
369		− err = xe_gt_idle_sysfs_init(&gt->gtidle);
370		− if (err)
371		− goto err_force_wake;
372		−
373	371	/* Enable per hw engine IRQs */
374	372	xe_irq_enable_hwe(gt);
375	373
@@ -434,6 +432,10 @@ static int all_fw_domain_init(struct xe_gt *gt) · 58 unmodified lines
434	432	if (err)
435	433	goto err_force_wake;
436	434
	435	+ err = xe_uc_init_post_hwconfig(&gt->uc);
	436	+ if (err)
	437	+ goto err_force_wake;
	438	+
437	439	if (!xe_gt_is_media_type(gt)) {
438	440	/*
439	441	* USM has its only SA pool to non-block behind user operations
@@ -460,10 +462,6 @@ static int all_fw_domain_init(struct xe_gt *gt) · 20 unmodified lines
460	462	}
461	463	}
462	464
463		− err = xe_uc_init_post_hwconfig(&gt->uc);
464		− if (err)
465		− goto err_force_wake;
466		−
467	465	err = xe_uc_init_hw(&gt->uc);
468	466	if (err)
469	467	goto err_force_wake;
@@ -477,6 +475,9 @@ static int all_fw_domain_init(struct xe_gt *gt) · 7 unmodified lines
477	475	if (IS_SRIOV_PF(gt_to_xe(gt)) && !xe_gt_is_media_type(gt))
478	476	xe_lmtt_init_hw(&gt_to_tile(gt)->sriov.pf.lmtt);
479	477
	478	+ if (IS_SRIOV_PF(gt_to_xe(gt)))
	479	+ xe_gt_sriov_pf_init_hw(gt);
	480	+
480	481	err = xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL);
481	482	XE_WARN_ON(err);
482	483
@@ -503,8 +504,7 @@ int xe_gt_init_hwconfig(struct xe_gt *gt) · 20 unmodified lines
503	504	if (err)
504	505	goto out;
505	506
506		− xe_gt_topology_init(gt);
507		− xe_gt_mcr_init(gt);
	507	+ xe_gt_mcr_init_early(gt);
508	508	xe_pat_init(gt);
509	509
510	510	err = xe_uc_init(&gt->uc);
@@ -515,8 +515,8 @@ int xe_gt_init_hwconfig(struct xe_gt *gt) · 4 unmodified lines
515	515	if (err)
516	516	goto out_fw;
517	517
518		− /* XXX: Fake that we pull the engine mask from hwconfig blob */
519		− gt->info.engine_mask = gt->info.__engine_mask;
	518	+ xe_gt_topology_init(gt);
	519	+ xe_gt_mcr_init(gt);
520	520
521	521	out_fw:
522	522	xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
@@ -554,6 +554,10 @@ int xe_gt_init(struct xe_gt *gt) · 31 unmodified lines
554	554	if (err)
555	555	return err;
556	556
	557	+ err = xe_gt_idle_init(&gt->gtidle);
	558	+ if (err)
	559	+ return err;
	560	+
557	561	err = xe_gt_freq_init(gt);
558	562	if (err)
559	563	return err;
@@ -564,7 +568,30 @@ int xe_gt_init(struct xe_gt *gt) · 4 unmodified lines
564	568	if (err)
565	569	return err;
566	570
567		− return drmm_add_action_or_reset(&gt_to_xe(gt)->drm, gt_fini, gt);
	571	+ xe_gt_record_user_engines(gt);
	572	+
	573	+ return 0;
	574	+}
	575	+
	576	+void xe_gt_record_user_engines(struct xe_gt *gt)
	577	+{
	578	+ struct xe_hw_engine *hwe;
	579	+ enum xe_hw_engine_id id;
	580	+
	581	+ gt->user_engines.mask = 0;
	582	+ memset(gt->user_engines.instances_per_class, 0,
	583	+ sizeof(gt->user_engines.instances_per_class));
	584	+
	585	+ for_each_hw_engine(hwe, gt, id) {
	586	+ if (xe_hw_engine_is_reserved(hwe))
	587	+ continue;
	588	+
	589	+ gt->user_engines.mask \|= BIT_ULL(id);
	590	+ gt->user_engines.instances_per_class[hwe->class]++;
	591	+ }
	592	+
	593	+ xe_gt_assert(gt, (gt->user_engines.mask \| gt->info.engine_mask)
	594	+ == gt->info.engine_mask);
568	595	}
569	596
570	597	static int do_gt_reset(struct xe_gt *gt)
@@ -584,12 +611,34 @@ static int do_gt_reset(struct xe_gt *gt) · 13 unmodified lines
584	611	return err;
585	612	}
586	613
	614	+static int vf_gt_restart(struct xe_gt *gt)
	615	+{
	616	+ int err;
	617	+
	618	+ err = xe_uc_sanitize_reset(&gt->uc);
	619	+ if (err)
	620	+ return err;
	621	+
	622	+ err = xe_uc_init_hw(&gt->uc);
	623	+ if (err)
	624	+ return err;
	625	+
	626	+ err = xe_uc_start(&gt->uc);
	627	+ if (err)
	628	+ return err;
	629	+
	630	+ return 0;
	631	+}
	632	+
587	633	static int do_gt_restart(struct xe_gt *gt)
588	634	{
589	635	struct xe_hw_engine *hwe;
590	636	enum xe_hw_engine_id id;
591	637	int err;
592	638
	639	+ if (IS_SRIOV_VF(gt_to_xe(gt)))
	640	+ return vf_gt_restart(gt);
	641	+
593	642	xe_pat_init(gt);
594	643
595	644	xe_gt_mcr_set_implicit_defaults(gt);
@@ -613,6 +662,9 @@ static int do_gt_restart(struct xe_gt *gt) · 17 unmodified lines
613	662	if (IS_SRIOV_PF(gt_to_xe(gt)) && !xe_gt_is_media_type(gt))
614	663	xe_lmtt_init_hw(&gt_to_tile(gt)->sriov.pf.lmtt);
615	664
	665	+ if (IS_SRIOV_PF(gt_to_xe(gt)))
	666	+ xe_gt_sriov_pf_init_hw(gt);
	667	+
616	668	xe_mocs_init(gt);
617	669	err = xe_uc_start(&gt->uc);
618	670	if (err)
@@ -633,6 +685,9 @@ static int gt_reset(struct xe_gt *gt) · 14 unmodified lines
633	685	{
634	686	int err;
635	687
	688	+ if (xe_device_wedged(gt_to_xe(gt)))
	689	+ return -ECANCELED;
	690	+
636	691	/* We only support GT resets with GuC submission */
637	692	if (!xe_device_uc_enabled(gt_to_xe(gt)))
638	693	return -ENODEV;
@@ -655,9 +710,7 @@ static int gt_reset(struct xe_gt *gt) · 16 unmodified lines
655	710	xe_uc_stop_prepare(&gt->uc);
656	711	xe_gt_pagefault_reset(gt);
657	712
658		− err = xe_uc_stop(&gt->uc);
659		− if (err)
660		− goto err_out;
	713	+ xe_uc_stop(&gt->uc);
661	714
662	715	xe_gt_tlb_invalidation_reset(gt);
663	716
@@ -685,7 +738,7 @@ err_msg: · 21 unmodified lines
685	738	err_fail:
686	739	xe_gt_err(gt, "reset failed (%pe)\n", ERR_PTR(err));
687	740
688		− gt_to_xe(gt)->needs_flr_on_fini = true;
	741	+ xe_device_declare_wedged(gt_to_xe(gt));
689	742
690	743	return err;
691	744	}
@@ -733,6 +786,8 @@ int xe_gt_suspend(struct xe_gt *gt) · 41 unmodified lines
733	786	if (err)
734	787	goto err_force_wake;
735	788
	789	+ xe_gt_idle_disable_pg(gt);
	790	+
736	791	XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL));
737	792	xe_gt_dbg(gt, "suspended\n");
738	793
@@ -759,6 +814,8 @@ int xe_gt_resume(struct xe_gt *gt) · 20 unmodified lines
759	814	if (err)
760	815	goto err_force_wake;
761	816
	817	+ xe_gt_idle_enable_pg(gt);
	818	+
762	819	XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL));
763	820	xe_gt_dbg(gt, "resumed\n");
764	821
@@ -810,3 +867,14 @@ struct xe_hw_engine xe_gt_any_hw_engine_by_reset_domain(struct xe_gt gt, · 45 unmodified lines
810	867
811	868	return NULL;
812	869	}
	870	+
	871	+struct xe_hw_engine xe_gt_any_hw_engine(struct xe_gt gt)
	872	+{
	873	+ struct xe_hw_engine *hwe;
	874	+ enum xe_hw_engine_id id;
	875	+
	876	+ for_each_hw_engine(hwe, gt, id)
	877	+ return hwe;
	878	+
	879	+ return NULL;
	880	+}

drivers/gpu/drm/xe/xe_gt.h+27−0

@@ -8,6 +8,7 @@ · 7 unmodified lines
8	8
9	9	#include <drm/drm_util.h>
10	10
	11	+#include "xe_device.h"
11	12	#include "xe_device_types.h"
12	13	#include "xe_hw_engine.h"
13	14
@@ -37,6 +38,19 @@ int xe_gt_init_hwconfig(struct xe_gt *gt); · 23 unmodified lines
37	38	int xe_gt_init_early(struct xe_gt *gt);
38	39	int xe_gt_init(struct xe_gt *gt);
39	40	int xe_gt_record_default_lrcs(struct xe_gt *gt);
	41	+
	42	+/**
	43	+ * xe_gt_record_user_engines - save data related to engines available to
	44	+ * usersapce
	45	+ * @gt: GT structure
	46	+ *
	47	+ * Walk the available HW engines from gt->info.engine_mask and calculate data
	48	+ * related to those engines that may be used by userspace. To be used whenever
	49	+ * available engines change in runtime (e.g. with ccs_mode) or during
	50	+ * initialization
	51	+ */
	52	+void xe_gt_record_user_engines(struct xe_gt *gt);
	53	+
40	54	void xe_gt_suspend_prepare(struct xe_gt *gt);
41	55	int xe_gt_suspend(struct xe_gt *gt);
42	56	int xe_gt_resume(struct xe_gt *gt);
@@ -53,11 +67,24 @@ void xe_gt_remove(struct xe_gt *gt); · 10 unmodified lines
53	67	struct xe_hw_engine *
54	68	xe_gt_any_hw_engine_by_reset_domain(struct xe_gt *gt, enum xe_engine_class class);
55	69
	70	+/**
	71	+ * xe_gt_any_hw_engine - scan the list of engines and return the
	72	+ * first available
	73	+ * @gt: GT structure
	74	+ */
	75	+struct xe_hw_engine xe_gt_any_hw_engine(struct xe_gt gt);
	76	+
56	77	struct xe_hw_engine xe_gt_hw_engine(struct xe_gt gt,
57	78	enum xe_engine_class class,
58	79	u16 instance,
59	80	bool logical);
60	81
	82	+static inline bool xe_gt_has_indirect_ring_state(struct xe_gt *gt)
	83	+{
	84	+ return gt->info.has_indirect_ring_state &&
	85	+ xe_device_uc_enabled(gt_to_xe(gt));
	86	+}
	87	+
61	88	static inline bool xe_gt_is_media_type(struct xe_gt *gt)
62	89	{
63	90	return gt->info.type == XE_GT_TYPE_MEDIA;

drivers/gpu/drm/xe/xe_gt_ccs_mode.c+6−4

@@ -9,6 +9,7 @@ · 8 unmodified lines
9	9	#include "xe_assert.h"
10	10	#include "xe_gt.h"
11	11	#include "xe_gt_ccs_mode.h"
	12	+#include "xe_gt_printk.h"
12	13	#include "xe_gt_sysfs.h"
13	14	#include "xe_mmio.h"
14	15
@@ -68,8 +69,8 @@ static void __xe_gt_apply_ccs_mode(struct xe_gt *gt, u32 num_engines) · 53 unmodified lines
68	69
69	70	xe_mmio_write32(gt, CCS_MODE, mode);
70	71
71		− xe_gt_info(gt, "CCS_MODE=%x config:%08x, num_engines:%d, num_slices:%d\n",
72		− mode, config, num_engines, num_slices);
	72	+ xe_gt_dbg(gt, "CCS_MODE=%x config:%08x, num_engines:%d, num_slices:%d\n",
	73	+ mode, config, num_engines, num_slices);
73	74	}
74	75
75	76	void xe_gt_apply_ccs_mode(struct xe_gt *gt)
@@ -134,6 +135,7 @@ ccs_mode_store(struct device kdev, struct device_attribute attr, · 58 unmodified lines
134	135	if (gt->ccs_mode != num_engines) {
135	136	xe_gt_info(gt, "Setting compute mode to %d\n", num_engines);
136	137	gt->ccs_mode = num_engines;
	138	+ xe_gt_record_user_engines(gt);
137	139	xe_gt_reset_async(gt);
138	140	}
139	141
@@ -150,7 +152,7 @@ static const struct attribute *gt_ccs_mode_attrs[] = { · 10 unmodified lines
150	152	NULL,
151	153	};
152	154
153		−static void xe_gt_ccs_mode_sysfs_fini(struct drm_device drm, void arg)
	155	+static void xe_gt_ccs_mode_sysfs_fini(void *arg)
154	156	{
155	157	struct xe_gt *gt = arg;
156	158
@@ -182,5 +184,5 @@ int xe_gt_ccs_mode_sysfs_init(struct xe_gt *gt) · 25 unmodified lines
182	184	if (err)
183	185	return err;
184	186
185		− return drmm_add_action_or_reset(&xe->drm, xe_gt_ccs_mode_sysfs_fini, gt);
	187	+ return devm_add_action_or_reset(xe->drm.dev, xe_gt_ccs_mode_sysfs_fini, gt);
186	188	}

drivers/gpu/drm/xe/xe_gt_clock.c+1−0

@@ -7,6 +7,7 @@ · 6 unmodified lines
7	7
8	8	#include "regs/xe_gt_regs.h"
9	9	#include "regs/xe_regs.h"
	10	+#include "xe_assert.h"
10	11	#include "xe_device.h"
11	12	#include "xe_gt.h"
12	13	#include "xe_macros.h"

drivers/gpu/drm/xe/xe_gt_debugfs.c+31−0

@@ -15,14 +15,18 @@ · 14 unmodified lines
15	15	#include "xe_ggtt.h"
16	16	#include "xe_gt.h"
17	17	#include "xe_gt_mcr.h"
	18	+#include "xe_gt_sriov_pf_debugfs.h"
	19	+#include "xe_gt_sriov_vf_debugfs.h"
18	20	#include "xe_gt_topology.h"
19	21	#include "xe_hw_engine.h"
20	22	#include "xe_lrc.h"
21	23	#include "xe_macros.h"
	24	+#include "xe_mocs.h"
22	25	#include "xe_pat.h"
23	26	#include "xe_pm.h"
24	27	#include "xe_reg_sr.h"
25	28	#include "xe_reg_whitelist.h"
	29	+#include "xe_sriov.h"
26	30	#include "xe_uc_debugfs.h"
27	31	#include "xe_wa.h"
28	32
@@ -112,6 +116,17 @@ static int force_reset(struct xe_gt gt, struct drm_printer p) · 83 unmodified lines
112	116	return 0;
113	117	}
114	118
	119	+static int force_reset_sync(struct xe_gt gt, struct drm_printer p)
	120	+{
	121	+ xe_pm_runtime_get(gt_to_xe(gt));
	122	+ xe_gt_reset_async(gt);
	123	+ xe_pm_runtime_put(gt_to_xe(gt));
	124	+
	125	+ flush_work(&gt->reset.worker);
	126	+
	127	+ return 0;
	128	+}
	129	+
115	130	static int sa_info(struct xe_gt gt, struct drm_printer p)
116	131	{
117	132	struct xe_tile *tile = gt_to_tile(gt);
@@ -200,6 +215,15 @@ static int pat(struct xe_gt gt, struct drm_printer p) · 82 unmodified lines
200	215	return 0;
201	216	}
202	217
	218	+static int mocs(struct xe_gt gt, struct drm_printer p)
	219	+{
	220	+ xe_pm_runtime_get(gt_to_xe(gt));
	221	+ xe_mocs_dump(gt, p);
	222	+ xe_pm_runtime_put(gt_to_xe(gt));
	223	+
	224	+ return 0;
	225	+}
	226	+
203	227	static int rcs_default_lrc(struct xe_gt gt, struct drm_printer p)
204	228	{
205	229	xe_pm_runtime_get(gt_to_xe(gt));
@@ -248,6 +272,7 @@ static int vecs_default_lrc(struct xe_gt gt, struct drm_printer p) · 42 unmodified lines
248	272	static const struct drm_info_list debugfs_list[] = {
249	273	{"hw_engines", .show = xe_gt_debugfs_simple_show, .data = hw_engines},
250	274	{"force_reset", .show = xe_gt_debugfs_simple_show, .data = force_reset},
	275	+ {"force_reset_sync", .show = xe_gt_debugfs_simple_show, .data = force_reset_sync},
251	276	{"sa_info", .show = xe_gt_debugfs_simple_show, .data = sa_info},
252	277	{"topology", .show = xe_gt_debugfs_simple_show, .data = topology},
253	278	{"steering", .show = xe_gt_debugfs_simple_show, .data = steering},
@@ -255,6 +280,7 @@ static const struct drm_info_list debugfs_list[] = { · 1 unmodified line
255	280	{"register-save-restore", .show = xe_gt_debugfs_simple_show, .data = register_save_restore},
256	281	{"workarounds", .show = xe_gt_debugfs_simple_show, .data = workarounds},
257	282	{"pat", .show = xe_gt_debugfs_simple_show, .data = pat},
	283	+ {"mocs", .show = xe_gt_debugfs_simple_show, .data = mocs},
258	284	{"default_lrc_rcs", .show = xe_gt_debugfs_simple_show, .data = rcs_default_lrc},
259	285	{"default_lrc_ccs", .show = xe_gt_debugfs_simple_show, .data = ccs_default_lrc},
260	286	{"default_lrc_bcs", .show = xe_gt_debugfs_simple_show, .data = bcs_default_lrc},
@@ -290,4 +316,9 @@ void xe_gt_debugfs_register(struct xe_gt *gt) · 29 unmodified lines
290	316	root, minor);
291	317
292	318	xe_uc_debugfs_register(&gt->uc, root);
	319	+
	320	+ if (IS_SRIOV_PF(xe))
	321	+ xe_gt_sriov_pf_debugfs_register(gt, root);
	322	+ else if (IS_SRIOV_VF(xe))
	323	+ xe_gt_sriov_vf_debugfs_register(gt, root);
293	324	}

drivers/gpu/drm/xe/xe_gt_freq.c+4−4

@@ -13,7 +13,7 @@ · 12 unmodified lines
13	13
14	14	#include "xe_device_types.h"
15	15	#include "xe_gt_sysfs.h"
16		−#include "xe_gt_throttle_sysfs.h"
	16	+#include "xe_gt_throttle.h"
17	17	#include "xe_guc_pc.h"
18	18	#include "xe_pm.h"
19	19
@@ -209,7 +209,7 @@ static const struct attribute *freq_attrs[] = { · 189 unmodified lines
209	209	NULL
210	210	};
211	211
212		−static void freq_fini(struct drm_device drm, void arg)
	212	+static void freq_fini(void *arg)
213	213	{
214	214	struct kobject *kobj = arg;
215	215
@@ -237,7 +237,7 @@ int xe_gt_freq_init(struct xe_gt *gt) · 21 unmodified lines
237	237	if (!gt->freq)
238	238	return -ENOMEM;
239	239
240		− err = drmm_add_action_or_reset(&xe->drm, freq_fini, gt->freq);
	240	+ err = devm_add_action(xe->drm.dev, freq_fini, gt->freq);
241	241	if (err)
242	242	return err;
243	243
@@ -245,5 +245,5 @@ int xe_gt_freq_init(struct xe_gt *gt) · 1 unmodified line
245	245	if (err)
246	246	return err;
247	247
248		− return xe_gt_throttle_sysfs_init(gt);
	248	+ return xe_gt_throttle_init(gt);
249	249	}

drivers/gpu/drm/xe/xe_gt_idle.c+61−5

@@ -5,12 +5,14 @@ · 4 unmodified lines
5	5
6	6	#include <drm/drm_managed.h>
7	7
	8	+#include "xe_force_wake.h"
8	9	#include "xe_device.h"
9	10	#include "xe_gt.h"
10	11	#include "xe_gt_idle.h"
11	12	#include "xe_gt_sysfs.h"
12	13	#include "xe_guc_pc.h"
13	14	#include "regs/xe_gt_regs.h"
	15	+#include "xe_macros.h"
14	16	#include "xe_mmio.h"
15	17	#include "xe_pm.h"
16	18
@@ -92,6 +94,50 @@ static u64 get_residency_ms(struct xe_gt_idle *gtidle, u64 cur_residency) · 75 unmodified lines
92	94	return cur_residency;
93	95	}
94	96
	97	+void xe_gt_idle_enable_pg(struct xe_gt *gt)
	98	+{
	99	+ struct xe_device *xe = gt_to_xe(gt);
	100	+ u32 pg_enable;
	101	+ int i, j;
	102	+
	103	+ /* Disable CPG for PVC */
	104	+ if (xe->info.platform == XE_PVC)
	105	+ return;
	106	+
	107	+ xe_device_assert_mem_access(gt_to_xe(gt));
	108	+
	109	+ pg_enable = RENDER_POWERGATE_ENABLE \| MEDIA_POWERGATE_ENABLE;
	110	+
	111	+ for (i = XE_HW_ENGINE_VCS0, j = 0; i <= XE_HW_ENGINE_VCS7; ++i, ++j) {
	112	+ if ((gt->info.engine_mask & BIT(i)))
	113	+ pg_enable \|= (VDN_HCP_POWERGATE_ENABLE(j) \|
	114	+ VDN_MFXVDENC_POWERGATE_ENABLE(j));
	115	+ }
	116	+
	117	+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
	118	+ if (xe->info.skip_guc_pc) {
	119	+ /*
	120	+ * GuC sets the hysteresis value when GuC PC is enabled
	121	+ * else set it to 25 (25 * 1.28us)
	122	+ */
	123	+ xe_mmio_write32(gt, MEDIA_POWERGATE_IDLE_HYSTERESIS, 25);
	124	+ xe_mmio_write32(gt, RENDER_POWERGATE_IDLE_HYSTERESIS, 25);
	125	+ }
	126	+
	127	+ xe_mmio_write32(gt, POWERGATE_ENABLE, pg_enable);
	128	+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
	129	+}
	130	+
	131	+void xe_gt_idle_disable_pg(struct xe_gt *gt)
	132	+{
	133	+ xe_device_assert_mem_access(gt_to_xe(gt));
	134	+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
	135	+
	136	+ xe_mmio_write32(gt, POWERGATE_ENABLE, 0);
	137	+
	138	+ XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FW_GT));
	139	+}
	140	+
95	141	static ssize_t name_show(struct device *dev,
96	142	struct device_attribute attr, char buff)
97	143	{
@@ -144,15 +190,24 @@ static const struct attribute *gt_idle_attrs[] = { · 46 unmodified lines
144	190	NULL,
145	191	};
146	192
147		−static void gt_idle_sysfs_fini(struct drm_device drm, void arg)
	193	+static void gt_idle_fini(void *arg)
148	194	{
149	195	struct kobject *kobj = arg;
	196	+ struct xe_gt *gt = kobj_to_gt(kobj->parent);
	197	+
	198	+ xe_gt_idle_disable_pg(gt);
	199	+
	200	+ if (gt_to_xe(gt)->info.skip_guc_pc) {
	201	+ XE_WARN_ON(xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
	202	+ xe_gt_idle_disable_c6(gt);
	203	+ xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
	204	+ }
150	205
151	206	sysfs_remove_files(kobj, gt_idle_attrs);
152	207	kobject_put(kobj);
153	208	}
154	209
155		−int xe_gt_idle_sysfs_init(struct xe_gt_idle *gtidle)
	210	+int xe_gt_idle_init(struct xe_gt_idle *gtidle)
156	211	{
157	212	struct xe_gt *gt = gtidle_to_gt(gtidle);
158	213	struct xe_device *xe = gt_to_xe(gt);
@@ -181,7 +236,9 @@ int xe_gt_idle_sysfs_init(struct xe_gt_idle *gtidle) · 22 unmodified lines
181	236	return err;
182	237	}
183	238
184		− return drmm_add_action_or_reset(&xe->drm, gt_idle_sysfs_fini, kobj);
	239	+ xe_gt_idle_enable_pg(gt);
	240	+
	241	+ return devm_add_action_or_reset(xe->drm.dev, gt_idle_fini, kobj);
185	242	}
186	243
187	244	void xe_gt_idle_enable_c6(struct xe_gt *gt)
@@ -199,9 +256,8 @@ void xe_gt_idle_enable_c6(struct xe_gt *gt) · 11 unmodified lines
199	256	void xe_gt_idle_disable_c6(struct xe_gt *gt)
200	257	{
201	258	xe_device_assert_mem_access(gt_to_xe(gt));
202		− xe_force_wake_assert_held(gt_to_fw(gt), XE_FORCEWAKE_ALL);
	259	+ xe_force_wake_assert_held(gt_to_fw(gt), XE_FW_GT);
203	260
204		− xe_mmio_write32(gt, PG_ENABLE, 0);
205	261	xe_mmio_write32(gt, RC_CONTROL, 0);
206	262	xe_mmio_write32(gt, RC_STATE, 0);
207	263	}

drivers/gpu/drm/xe/xe_gt_idle.h+3−1

@@ -10,8 +10,10 @@ · 9 unmodified lines
10	10
11	11	struct xe_gt;
12	12
13		−int xe_gt_idle_sysfs_init(struct xe_gt_idle *gtidle);
	13	+int xe_gt_idle_init(struct xe_gt_idle *gtidle);
14	14	void xe_gt_idle_enable_c6(struct xe_gt *gt);
15	15	void xe_gt_idle_disable_c6(struct xe_gt *gt);
	16	+void xe_gt_idle_enable_pg(struct xe_gt *gt);
	17	+void xe_gt_idle_disable_pg(struct xe_gt *gt);
16	18
17	19	#endif /* _XE_GT_IDLE_H_ */

drivers/gpu/drm/xe/xe_gt_mcr.c+22−5

@@ -375,18 +375,35 @@ static const struct { · 374 unmodified lines
375	375	[IMPLICIT_STEERING] = { "IMPLICIT", NULL },
376	376	};
377	377
378		−void xe_gt_mcr_init(struct xe_gt *gt)
	378	+/**
	379	+ * xe_gt_mcr_init_early - Early initialization of the MCR support
	380	+ * @gt: GT structure
	381	+ *
	382	+ * Perform early software only initialization of the MCR lock to allow
	383	+ * the synchronization on accessing the STEER_SEMAPHORE register and
	384	+ * use the xe_gt_mcr_multicast_write() function.
	385	+ */
	386	+void xe_gt_mcr_init_early(struct xe_gt *gt)
379	387	{
380		− struct xe_device *xe = gt_to_xe(gt);
381		−
382	388	BUILD_BUG_ON(IMPLICIT_STEERING + 1 != NUM_STEERING_TYPES);
383	389	BUILD_BUG_ON(ARRAY_SIZE(xe_steering_types) != NUM_STEERING_TYPES);
384	390
	391	+ spin_lock_init(&gt->mcr_lock);
	392	+}
	393	+
	394	+/**
	395	+ * xe_gt_mcr_init - Normal initialization of the MCR support
	396	+ * @gt: GT structure
	397	+ *
	398	+ * Perform normal initialization of the MCR for all usages.
	399	+ */
	400	+void xe_gt_mcr_init(struct xe_gt *gt)
	401	+{
	402	+ struct xe_device *xe = gt_to_xe(gt);
	403	+
385	404	if (IS_SRIOV_VF(xe))
386	405	return;
387	406
388		− spin_lock_init(&gt->mcr_lock);
389		−
390	407	if (gt->info.type == XE_GT_TYPE_MEDIA) {
391	408	drm_WARN_ON(&xe->drm, MEDIA_VER(xe) < 13);
392	409

drivers/gpu/drm/xe/xe_gt_mcr.h+25−0

@@ -12,6 +12,7 @@ · 11 unmodified lines
12	12	struct drm_printer;
13	13	struct xe_gt;
14	14
	15	+void xe_gt_mcr_init_early(struct xe_gt *gt);
15	16	void xe_gt_mcr_init(struct xe_gt *gt);
16	17
17	18	void xe_gt_mcr_set_implicit_defaults(struct xe_gt *gt);
@@ -40,4 +41,28 @@ void xe_gt_mcr_get_dss_steering(struct xe_gt gt, unsigned int dss, u16 group, · 22 unmodified lines
40	41	for_each_dss((dss), (gt)) \
41	42	for_each_if((xe_gt_mcr_get_dss_steering((gt), (dss), &(group), &(instance)), true))
42	43
	44	+/*
	45	+ * Loop over each DSS available for geometry and determine the group and
	46	+ * instance IDs that should be used to steer MCR accesses toward this DSS.
	47	+ * @dss: DSS ID to obtain steering for
	48	+ * @gt: GT structure
	49	+ * @group: steering group ID, data type: u16
	50	+ * @instance: steering instance ID, data type: u16
	51	+ */
	52	+#define for_each_geometry_dss(dss, gt, group, instance) \
	53	+ for_each_dss_steering(dss, gt, group, instance) \
	54	+ if (xe_gt_has_geometry_dss(gt, dss))
	55	+
	56	+/*
	57	+ * Loop over each DSS available for compute and determine the group and
	58	+ * instance IDs that should be used to steer MCR accesses toward this DSS.
	59	+ * @dss: DSS ID to obtain steering for
	60	+ * @gt: GT structure
	61	+ * @group: steering group ID, data type: u16
	62	+ * @instance: steering instance ID, data type: u16
	63	+ */
	64	+#define for_each_compute_dss(dss, gt, group, instance) \
	65	+ for_each_dss_steering(dss, gt, group, instance) \
	66	+ if (xe_gt_has_compute_dss(gt, dss))
	67	+
43	68	#endif /* _XE_GT_MCR_H_ */

drivers/gpu/drm/xe/xe_gt_pagefault.c+7−9

@@ -19,7 +19,6 @@ · 18 unmodified lines
19	19	#include "xe_guc.h"
20	20	#include "xe_guc_ct.h"
21	21	#include "xe_migrate.h"
22		−#include "xe_pt.h"
23	22	#include "xe_trace.h"
24	23	#include "xe_vm.h"
25	24
@@ -204,15 +203,14 @@ retry_userptr: · 178 unmodified lines
204	203	drm_exec_retry_on_contention(&exec);
205	204	if (ret)
206	205	goto unlock_dma_resv;
207		− }
208	206
209		− /* Bind VMA only to the GT that has faulted */
210		− trace_xe_vma_pf_bind(vma);
211		− fence = __xe_pt_bind_vma(tile, vma, xe_tile_migrate_engine(tile), NULL, 0,
212		− vma->tile_present & BIT(tile->id));
213		− if (IS_ERR(fence)) {
214		− ret = PTR_ERR(fence);
215		− goto unlock_dma_resv;
	207	+ /* Bind VMA only to the GT that has faulted */
	208	+ trace_xe_vma_pf_bind(vma);
	209	+ fence = xe_vma_rebind(vm, vma, BIT(tile->id));
	210	+ if (IS_ERR(fence)) {
	211	+ ret = PTR_ERR(fence);
	212	+ goto unlock_dma_resv;
	213	+ }
216	214	}
217	215
218	216	/*

drivers/gpu/drm/xe/xe_gt_sriov_pf.c+32−0

@@ -5,8 +5,12 @@ · 4 unmodified lines
5	5
6	6	#include <drm/drm_managed.h>
7	7
	8	+#include "regs/xe_sriov_regs.h"
	9	+
8	10	#include "xe_gt_sriov_pf.h"
9	11	#include "xe_gt_sriov_pf_helpers.h"
	12	+#include "xe_gt_sriov_pf_service.h"
	13	+#include "xe_mmio.h"
10	14
11	15	/*
12	16	* VF's metadata is maintained in the flexible array where:
@@ -48,5 +52,33 @@ int xe_gt_sriov_pf_init_early(struct xe_gt *gt) · 35 unmodified lines
48	52	if (err)
49	53	return err;
50	54
	55	+ err = xe_gt_sriov_pf_service_init(gt);
	56	+ if (err)
	57	+ return err;
	58	+
51	59	return 0;
52	60	}
	61	+
	62	+static bool pf_needs_enable_ggtt_guest_update(struct xe_device *xe)
	63	+{
	64	+ return GRAPHICS_VERx100(xe) == 1200;
	65	+}
	66	+
	67	+static void pf_enable_ggtt_guest_update(struct xe_gt *gt)
	68	+{
	69	+ xe_mmio_write32(gt, VIRTUAL_CTRL_REG, GUEST_GTT_UPDATE_EN);
	70	+}
	71	+
	72	+/**
	73	+ * xe_gt_sriov_pf_init_hw - Initialize SR-IOV hardware support.
	74	+ * @gt: the &xe_gt to initialize
	75	+ *
	76	+ * On some platforms the PF must explicitly enable VF's access to the GGTT.
	77	+ */
	78	+void xe_gt_sriov_pf_init_hw(struct xe_gt *gt)
	79	+{
	80	+ if (pf_needs_enable_ggtt_guest_update(gt_to_xe(gt)))
	81	+ pf_enable_ggtt_guest_update(gt);
	82	+
	83	+ xe_gt_sriov_pf_service_update(gt);
	84	+}

drivers/gpu/drm/xe/xe_gt_sriov_pf.h+5−0

@@ -10,11 +10,16 @@ struct xe_gt; · 9 unmodified lines
10	10
11	11	#ifdef CONFIG_PCI_IOV
12	12	int xe_gt_sriov_pf_init_early(struct xe_gt *gt);
	13	+void xe_gt_sriov_pf_init_hw(struct xe_gt *gt);
13	14	#else
14	15	static inline int xe_gt_sriov_pf_init_early(struct xe_gt *gt)
15	16	{
16	17	return 0;
17	18	}
	19	+
	20	+static inline void xe_gt_sriov_pf_init_hw(struct xe_gt *gt)
	21	+{
	22	+}
18	23	#endif
19	24
20	25	#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c+99−6

@@ -25,6 +25,7 @@ · 24 unmodified lines
25	25	#include "xe_guc_fwif.h"
26	26	#include "xe_guc_id_mgr.h"
27	27	#include "xe_guc_klv_helpers.h"
	28	+#include "xe_guc_klv_thresholds_set.h"
28	29	#include "xe_guc_submit.h"
29	30	#include "xe_lmtt.h"
30	31	#include "xe_map.h"
@@ -187,14 +188,20 @@ static int pf_push_vf_cfg_dbs(struct xe_gt *gt, unsigned int vfid, u32 begin, u3 · 156 unmodified lines
187	188	return pf_push_vf_cfg_klvs(gt, vfid, 2, klvs, ARRAY_SIZE(klvs));
188	189	}
189	190
190		−static int pf_push_vf_cfg_exec_quantum(struct xe_gt *gt, unsigned int vfid, u32 exec_quantum)
	191	+static int pf_push_vf_cfg_exec_quantum(struct xe_gt gt, unsigned int vfid, u32 exec_quantum)
191	192	{
192		− return pf_push_vf_cfg_u32(gt, vfid, GUC_KLV_VF_CFG_EXEC_QUANTUM_KEY, exec_quantum);
	193	+ /* GuC will silently clamp values exceeding max */
	194	+ exec_quantum = min_t(u32, exec_quantum, GUC_KLV_VF_CFG_EXEC_QUANTUM_MAX_VALUE);
	195	+
	196	+ return pf_push_vf_cfg_u32(gt, vfid, GUC_KLV_VF_CFG_EXEC_QUANTUM_KEY, *exec_quantum);
193	197	}
194	198
195		−static int pf_push_vf_cfg_preempt_timeout(struct xe_gt *gt, unsigned int vfid, u32 preempt_timeout)
	199	+static int pf_push_vf_cfg_preempt_timeout(struct xe_gt gt, unsigned int vfid, u32 preempt_timeout)
196	200	{
197		− return pf_push_vf_cfg_u32(gt, vfid, GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_KEY, preempt_timeout);
	201	+ /* GuC will silently clamp values exceeding max */
	202	+ preempt_timeout = min_t(u32, preempt_timeout, GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_MAX_VALUE);
	203	+
	204	+ return pf_push_vf_cfg_u32(gt, vfid, GUC_KLV_VF_CFG_PREEMPT_TIMEOUT_KEY, *preempt_timeout);
198	205	}
199	206
200	207	static int pf_push_vf_cfg_lmem(struct xe_gt *gt, unsigned int vfid, u64 size)
@@ -202,6 +209,15 @@ static int pf_push_vf_cfg_lmem(struct xe_gt *gt, unsigned int vfid, u64 size) · 1 unmodified line
202	209	return pf_push_vf_cfg_u64(gt, vfid, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, size);
203	210	}
204	211
	212	+static int pf_push_vf_cfg_threshold(struct xe_gt *gt, unsigned int vfid,
	213	+ enum xe_guc_klv_threshold_index index, u32 value)
	214	+{
	215	+ u32 key = xe_guc_klv_threshold_index_to_key(index);
	216	+
	217	+ xe_gt_assert(gt, key);
	218	+ return pf_push_vf_cfg_u32(gt, vfid, key, value);
	219	+}
	220	+
205	221	static struct xe_gt_sriov_config pf_pick_vf_config(struct xe_gt gt, unsigned int vfid)
206	222	{
207	223	xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
@@ -1604,7 +1620,7 @@ static int pf_provision_exec_quantum(struct xe_gt *gt, unsigned int vfid, · 1396 unmodified lines
1604	1620	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);
1605	1621	int err;
1606	1622
1607		− err = pf_push_vf_cfg_exec_quantum(gt, vfid, exec_quantum);
	1623	+ err = pf_push_vf_cfg_exec_quantum(gt, vfid, &exec_quantum);
1608	1624	if (unlikely(err))
1609	1625	return err;
1610	1626
@@ -1674,7 +1690,7 @@ static int pf_provision_preempt_timeout(struct xe_gt *gt, unsigned int vfid, · 63 unmodified lines
1674	1690	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);
1675	1691	int err;
1676	1692
1677		− err = pf_push_vf_cfg_preempt_timeout(gt, vfid, preempt_timeout);
	1693	+ err = pf_push_vf_cfg_preempt_timeout(gt, vfid, &preempt_timeout);
1678	1694	if (unlikely(err))
1679	1695	return err;
1680	1696
@@ -1742,6 +1758,83 @@ static void pf_reset_config_sched(struct xe_gt gt, struct xe_gt_sriov_config c · 61 unmodified lines
1742	1758	config->preempt_timeout = 0;
1743	1759	}
1744	1760
	1761	+static int pf_provision_threshold(struct xe_gt *gt, unsigned int vfid,
	1762	+ enum xe_guc_klv_threshold_index index, u32 value)
	1763	+{
	1764	+ struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);
	1765	+ int err;
	1766	+
	1767	+ err = pf_push_vf_cfg_threshold(gt, vfid, index, value);
	1768	+ if (unlikely(err))
	1769	+ return err;
	1770	+
	1771	+ config->thresholds[index] = value;
	1772	+
	1773	+ return 0;
	1774	+}
	1775	+
	1776	+static int pf_get_threshold(struct xe_gt *gt, unsigned int vfid,
	1777	+ enum xe_guc_klv_threshold_index index)
	1778	+{
	1779	+ struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);
	1780	+
	1781	+ return config->thresholds[index];
	1782	+}
	1783	+
	1784	+static const char *threshold_unit(u32 threshold)
	1785	+{
	1786	+ return threshold ? "" : "(disabled)";
	1787	+}
	1788	+
	1789	+/**
	1790	+ * xe_gt_sriov_pf_config_set_threshold - Configure threshold for the VF.
	1791	+ * @gt: the &xe_gt
	1792	+ * @vfid: the VF identifier
	1793	+ * @index: the threshold index
	1794	+ * @value: requested value (0 means disabled)
	1795	+ *
	1796	+ * This function can only be called on PF.
	1797	+ *
	1798	+ * Return: 0 on success or a negative error code on failure.
	1799	+ */
	1800	+int xe_gt_sriov_pf_config_set_threshold(struct xe_gt *gt, unsigned int vfid,
	1801	+ enum xe_guc_klv_threshold_index index, u32 value)
	1802	+{
	1803	+ u32 key = xe_guc_klv_threshold_index_to_key(index);
	1804	+ const char *name = xe_guc_klv_key_to_string(key);
	1805	+ int err;
	1806	+
	1807	+ mutex_lock(xe_gt_sriov_pf_master_mutex(gt));
	1808	+ err = pf_provision_threshold(gt, vfid, index, value);
	1809	+ mutex_unlock(xe_gt_sriov_pf_master_mutex(gt));
	1810	+
	1811	+ return pf_config_set_u32_done(gt, vfid, value,
	1812	+ xe_gt_sriov_pf_config_get_threshold(gt, vfid, index),
	1813	+ name, threshold_unit, err);
	1814	+}
	1815	+
	1816	+/**
	1817	+ * xe_gt_sriov_pf_config_get_threshold - Get VF's threshold.
	1818	+ * @gt: the &xe_gt
	1819	+ * @vfid: the VF identifier
	1820	+ * @index: the threshold index
	1821	+ *
	1822	+ * This function can only be called on PF.
	1823	+ *
	1824	+ * Return: value of VF's (or PF's) threshold.
	1825	+ */
	1826	+u32 xe_gt_sriov_pf_config_get_threshold(struct xe_gt *gt, unsigned int vfid,
	1827	+ enum xe_guc_klv_threshold_index index)
	1828	+{
	1829	+ u32 value;
	1830	+
	1831	+ mutex_lock(xe_gt_sriov_pf_master_mutex(gt));
	1832	+ value = pf_get_threshold(gt, vfid, index);
	1833	+ mutex_unlock(xe_gt_sriov_pf_master_mutex(gt));
	1834	+
	1835	+ return value;
	1836	+}
	1837	+
1745	1838	static void pf_release_vf_config(struct xe_gt *gt, unsigned int vfid)
1746	1839	{
1747	1840	struct xe_gt_sriov_config *config = pf_pick_vf_config(gt, vfid);

drivers/gpu/drm/xe/xe_gt_sriov_pf_config.h+6−0

@@ -8,6 +8,7 @@ · 7 unmodified lines
8	8
9	9	#include <linux/types.h>
10	10
	11	+enum xe_guc_klv_threshold_index;
11	12	struct drm_printer;
12	13	struct xe_gt;
13	14
@@ -43,6 +44,11 @@ u32 xe_gt_sriov_pf_config_get_preempt_timeout(struct xe_gt *gt, unsigned int vfi · 29 unmodified lines
43	44	int xe_gt_sriov_pf_config_set_preempt_timeout(struct xe_gt *gt, unsigned int vfid,
44	45	u32 preempt_timeout);
45	46
	47	+u32 xe_gt_sriov_pf_config_get_threshold(struct xe_gt *gt, unsigned int vfid,
	48	+ enum xe_guc_klv_threshold_index index);
	49	+int xe_gt_sriov_pf_config_set_threshold(struct xe_gt *gt, unsigned int vfid,
	50	+ enum xe_guc_klv_threshold_index index, u32 value);
	51	+
46	52	int xe_gt_sriov_pf_config_set_fair(struct xe_gt *gt, unsigned int vfid, unsigned int num_vfs);
47	53	int xe_gt_sriov_pf_config_release(struct xe_gt *gt, unsigned int vfid, bool force);
48	54	int xe_gt_sriov_pf_config_push(struct xe_gt *gt, unsigned int vfid, bool refresh);

drivers/gpu/drm/xe/xe_gt_sriov_pf_config_types.h+4−0

@@ -8,6 +8,8 @@ · 7 unmodified lines
8	8
9	9	#include <drm/drm_mm.h>
10	10
	11	+#include "xe_guc_klv_thresholds_set_types.h"
	12	+
11	13	struct xe_bo;
12	14
13	15	/**
@@ -32,6 +34,8 @@ struct xe_gt_sriov_config { · 18 unmodified lines
32	34	u32 exec_quantum;
33	35	/** @preempt_timeout: preemption timeout in microseconds. */
34	36	u32 preempt_timeout;
	37	+ /** @thresholds: GuC thresholds for adverse events notifications. */
	38	+ u32 thresholds[XE_GUC_KLV_NUM_THRESHOLDS];
35	39	};
36	40
37	41	/**

drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c+427−0

@@ -0,0 +1,427 @@
	1	+// SPDX-License-Identifier: MIT
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#include <linux/debugfs.h>
	7	+
	8	+#include <drm/drm_print.h>
	9	+#include <drm/drm_debugfs.h>
	10	+
	11	+#include "xe_bo.h"
	12	+#include "xe_debugfs.h"
	13	+#include "xe_device.h"
	14	+#include "xe_gt.h"
	15	+#include "xe_gt_debugfs.h"
	16	+#include "xe_gt_sriov_pf_config.h"
	17	+#include "xe_gt_sriov_pf_control.h"
	18	+#include "xe_gt_sriov_pf_debugfs.h"
	19	+#include "xe_gt_sriov_pf_helpers.h"
	20	+#include "xe_gt_sriov_pf_monitor.h"
	21	+#include "xe_gt_sriov_pf_policy.h"
	22	+#include "xe_gt_sriov_pf_service.h"
	23	+#include "xe_pm.h"
	24	+
	25	+/*
	26	+ * /sys/kernel/debug/dri/0/
	27	+ * ├── gt0 # d_inode->i_private = gt
	28	+ * │ ├── pf # d_inode->i_private = gt
	29	+ * │ ├── vf1 # d_inode->i_private = VFID(1)
	30	+ * : :
	31	+ * │ ├── vfN # d_inode->i_private = VFID(N)
	32	+ */
	33	+
	34	+static void extract_priv(struct dentry d)
	35	+{
	36	+ return d->d_inode->i_private;
	37	+}
	38	+
	39	+static struct xe_gt extract_gt(struct dentry d)
	40	+{
	41	+ return extract_priv(d->d_parent);
	42	+}
	43	+
	44	+static unsigned int extract_vfid(struct dentry *d)
	45	+{
	46	+ return extract_priv(d) == extract_gt(d) ? PFID : (uintptr_t)extract_priv(d);
	47	+}
	48	+
	49	+/*
	50	+ * /sys/kernel/debug/dri/0/
	51	+ * ├── gt0
	52	+ * │ ├── pf
	53	+ * │ │ ├── ggtt_available
	54	+ * │ │ ├── ggtt_provisioned
	55	+ * │ │ ├── contexts_provisioned
	56	+ * │ │ ├── doorbells_provisioned
	57	+ * │ │ ├── runtime_registers
	58	+ * │ │ ├── negotiated_versions
	59	+ * │ │ ├── adverse_events
	60	+ */
	61	+
	62	+static const struct drm_info_list pf_info[] = {
	63	+ {
	64	+ "ggtt_available",
	65	+ .show = xe_gt_debugfs_simple_show,
	66	+ .data = xe_gt_sriov_pf_config_print_available_ggtt,
	67	+ },
	68	+ {
	69	+ "ggtt_provisioned",
	70	+ .show = xe_gt_debugfs_simple_show,
	71	+ .data = xe_gt_sriov_pf_config_print_ggtt,
	72	+ },
	73	+ {
	74	+ "contexts_provisioned",
	75	+ .show = xe_gt_debugfs_simple_show,
	76	+ .data = xe_gt_sriov_pf_config_print_ctxs,
	77	+ },
	78	+ {
	79	+ "doorbells_provisioned",
	80	+ .show = xe_gt_debugfs_simple_show,
	81	+ .data = xe_gt_sriov_pf_config_print_dbs,
	82	+ },
	83	+ {
	84	+ "runtime_registers",
	85	+ .show = xe_gt_debugfs_simple_show,
	86	+ .data = xe_gt_sriov_pf_service_print_runtime,
	87	+ },
	88	+ {
	89	+ "negotiated_versions",
	90	+ .show = xe_gt_debugfs_simple_show,
	91	+ .data = xe_gt_sriov_pf_service_print_version,
	92	+ },
	93	+ {
	94	+ "adverse_events",
	95	+ .show = xe_gt_debugfs_simple_show,
	96	+ .data = xe_gt_sriov_pf_monitor_print_events,
	97	+ },
	98	+};
	99	+
	100	+/*
	101	+ * /sys/kernel/debug/dri/0/
	102	+ * ├── gt0
	103	+ * │ ├── pf
	104	+ * │ │ ├── reset_engine
	105	+ * │ │ ├── sample_period
	106	+ * │ │ ├── sched_if_idle
	107	+ */
	108	+
	109	+#define DEFINE_SRIOV_GT_POLICY_DEBUGFS_ATTRIBUTE(POLICY, TYPE, FORMAT) \
	110	+ \
	111	+static int POLICY##_set(void *data, u64 val) \
	112	+{ \
	113	+ struct xe_gt *gt = extract_gt(data); \
	114	+ struct xe_device *xe = gt_to_xe(gt); \
	115	+ int err; \
	116	+ \
	117	+ if (val > (TYPE)~0ull) \
	118	+ return -EOVERFLOW; \
	119	+ \
	120	+ xe_pm_runtime_get(xe); \
	121	+ err = xe_gt_sriov_pf_policy_set_##POLICY(gt, val); \
	122	+ xe_pm_runtime_put(xe); \
	123	+ \
	124	+ return err; \
	125	+} \
	126	+ \
	127	+static int POLICY##_get(void data, u64 val) \
	128	+{ \
	129	+ struct xe_gt *gt = extract_gt(data); \
	130	+ \
	131	+ *val = xe_gt_sriov_pf_policy_get_##POLICY(gt); \
	132	+ return 0; \
	133	+} \
	134	+ \
	135	+DEFINE_DEBUGFS_ATTRIBUTE(POLICY##_fops, POLICY##_get, POLICY##_set, FORMAT)
	136	+
	137	+DEFINE_SRIOV_GT_POLICY_DEBUGFS_ATTRIBUTE(reset_engine, bool, "%llu\n");
	138	+DEFINE_SRIOV_GT_POLICY_DEBUGFS_ATTRIBUTE(sched_if_idle, bool, "%llu\n");
	139	+DEFINE_SRIOV_GT_POLICY_DEBUGFS_ATTRIBUTE(sample_period, u32, "%llu\n");
	140	+
	141	+static void pf_add_policy_attrs(struct xe_gt gt, struct dentry parent)
	142	+{
	143	+ xe_gt_assert(gt, gt == extract_gt(parent));
	144	+ xe_gt_assert(gt, PFID == extract_vfid(parent));
	145	+
	146	+ debugfs_create_file_unsafe("reset_engine", 0644, parent, parent, &reset_engine_fops);
	147	+ debugfs_create_file_unsafe("sched_if_idle", 0644, parent, parent, &sched_if_idle_fops);
	148	+ debugfs_create_file_unsafe("sample_period_ms", 0644, parent, parent, &sample_period_fops);
	149	+}
	150	+
	151	+/*
	152	+ * /sys/kernel/debug/dri/0/
	153	+ * ├── gt0
	154	+ * │ ├── pf
	155	+ * │ │ ├── ggtt_spare
	156	+ * │ │ ├── lmem_spare
	157	+ * │ │ ├── doorbells_spare
	158	+ * │ │ ├── contexts_spare
	159	+ * │ │ ├── exec_quantum_ms
	160	+ * │ │ ├── preempt_timeout_us
	161	+ * │ ├── vf1
	162	+ * │ │ ├── ggtt_quota
	163	+ * │ │ ├── lmem_quota
	164	+ * │ │ ├── doorbells_quota
	165	+ * │ │ ├── contexts_quota
	166	+ * │ │ ├── exec_quantum_ms
	167	+ * │ │ ├── preempt_timeout_us
	168	+ */
	169	+
	170	+#define DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(CONFIG, TYPE, FORMAT) \
	171	+ \
	172	+static int CONFIG##_set(void *data, u64 val) \
	173	+{ \
	174	+ struct xe_gt *gt = extract_gt(data); \
	175	+ unsigned int vfid = extract_vfid(data); \
	176	+ struct xe_device *xe = gt_to_xe(gt); \
	177	+ int err; \
	178	+ \
	179	+ if (val > (TYPE)~0ull) \
	180	+ return -EOVERFLOW; \
	181	+ \
	182	+ xe_pm_runtime_get(xe); \
	183	+ err = xe_gt_sriov_pf_config_set_##CONFIG(gt, vfid, val); \
	184	+ xe_pm_runtime_put(xe); \
	185	+ \
	186	+ return err; \
	187	+} \
	188	+ \
	189	+static int CONFIG##_get(void data, u64 val) \
	190	+{ \
	191	+ struct xe_gt *gt = extract_gt(data); \
	192	+ unsigned int vfid = extract_vfid(data); \
	193	+ \
	194	+ *val = xe_gt_sriov_pf_config_get_##CONFIG(gt, vfid); \
	195	+ return 0; \
	196	+} \
	197	+ \
	198	+DEFINE_DEBUGFS_ATTRIBUTE(CONFIG##_fops, CONFIG##_get, CONFIG##_set, FORMAT)
	199	+
	200	+DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(ggtt, u64, "%llu\n");
	201	+DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(lmem, u64, "%llu\n");
	202	+DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(ctxs, u32, "%llu\n");
	203	+DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(dbs, u32, "%llu\n");
	204	+DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(exec_quantum, u32, "%llu\n");
	205	+DEFINE_SRIOV_GT_CONFIG_DEBUGFS_ATTRIBUTE(preempt_timeout, u32, "%llu\n");
	206	+
	207	+/*
	208	+ * /sys/kernel/debug/dri/0/
	209	+ * ├── gt0
	210	+ * │ ├── pf
	211	+ * │ │ ├── threshold_cat_error_count
	212	+ * │ │ ├── threshold_doorbell_time_us
	213	+ * │ │ ├── threshold_engine_reset_count
	214	+ * │ │ ├── threshold_guc_time_us
	215	+ * │ │ ├── threshold_irq_time_us
	216	+ * │ │ ├── threshold_page_fault_count
	217	+ * │ ├── vf1
	218	+ * │ │ ├── threshold_cat_error_count
	219	+ * │ │ ├── threshold_doorbell_time_us
	220	+ * │ │ ├── threshold_engine_reset_count
	221	+ * │ │ ├── threshold_guc_time_us
	222	+ * │ │ ├── threshold_irq_time_us
	223	+ * │ │ ├── threshold_page_fault_count
	224	+ */
	225	+
	226	+static int set_threshold(void *data, u64 val, enum xe_guc_klv_threshold_index index)
	227	+{
	228	+ struct xe_gt *gt = extract_gt(data);
	229	+ unsigned int vfid = extract_vfid(data);
	230	+ struct xe_device *xe = gt_to_xe(gt);
	231	+ int err;
	232	+
	233	+ if (val > (u32)~0ull)
	234	+ return -EOVERFLOW;
	235	+
	236	+ xe_pm_runtime_get(xe);
	237	+ err = xe_gt_sriov_pf_config_set_threshold(gt, vfid, index, val);
	238	+ xe_pm_runtime_put(xe);
	239	+
	240	+ return err;
	241	+}
	242	+
	243	+static int get_threshold(void data, u64 val, enum xe_guc_klv_threshold_index index)
	244	+{
	245	+ struct xe_gt *gt = extract_gt(data);
	246	+ unsigned int vfid = extract_vfid(data);
	247	+
	248	+ *val = xe_gt_sriov_pf_config_get_threshold(gt, vfid, index);
	249	+ return 0;
	250	+}
	251	+
	252	+#define DEFINE_SRIOV_GT_THRESHOLD_DEBUGFS_ATTRIBUTE(THRESHOLD, INDEX) \
	253	+ \
	254	+static int THRESHOLD##_set(void *data, u64 val) \
	255	+{ \
	256	+ return set_threshold(data, val, INDEX); \
	257	+} \
	258	+ \
	259	+static int THRESHOLD##_get(void data, u64 val) \
	260	+{ \
	261	+ return get_threshold(data, val, INDEX); \
	262	+} \
	263	+ \
	264	+DEFINE_DEBUGFS_ATTRIBUTE(THRESHOLD##_fops, THRESHOLD##_get, THRESHOLD##_set, "%llu\n")
	265	+
	266	+/* generate all threshold attributes */
	267	+#define define_threshold_attribute(TAG, NAME, ...) \
	268	+ DEFINE_SRIOV_GT_THRESHOLD_DEBUGFS_ATTRIBUTE(NAME, MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG));
	269	+MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_threshold_attribute)
	270	+#undef define_threshold_attribute
	271	+
	272	+static void pf_add_config_attrs(struct xe_gt gt, struct dentry parent, unsigned int vfid)
	273	+{
	274	+ xe_gt_assert(gt, gt == extract_gt(parent));
	275	+ xe_gt_assert(gt, vfid == extract_vfid(parent));
	276	+
	277	+ if (!xe_gt_is_media_type(gt)) {
	278	+ debugfs_create_file_unsafe(vfid ? "ggtt_quota" : "ggtt_spare",
	279	+ 0644, parent, parent, &ggtt_fops);
	280	+ if (IS_DGFX(gt_to_xe(gt)))
	281	+ debugfs_create_file_unsafe(vfid ? "lmem_quota" : "lmem_spare",
	282	+ 0644, parent, parent, &lmem_fops);
	283	+ }
	284	+ debugfs_create_file_unsafe(vfid ? "doorbells_quota" : "doorbells_spare",
	285	+ 0644, parent, parent, &dbs_fops);
	286	+ debugfs_create_file_unsafe(vfid ? "contexts_quota" : "contexts_spare",
	287	+ 0644, parent, parent, &ctxs_fops);
	288	+ debugfs_create_file_unsafe("exec_quantum_ms", 0644, parent, parent,
	289	+ &exec_quantum_fops);
	290	+ debugfs_create_file_unsafe("preempt_timeout_us", 0644, parent, parent,
	291	+ &preempt_timeout_fops);
	292	+
	293	+ /* register all threshold attributes */
	294	+#define register_threshold_attribute(TAG, NAME, ...) \
	295	+ debugfs_create_file_unsafe("threshold_" #NAME, 0644, parent, parent, \
	296	+ &NAME##_fops);
	297	+ MAKE_XE_GUC_KLV_THRESHOLDS_SET(register_threshold_attribute)
	298	+#undef register_threshold_attribute
	299	+}
	300	+
	301	+/*
	302	+ * /sys/kernel/debug/dri/0/
	303	+ * ├── gt0
	304	+ * │ ├── vf1
	305	+ * │ │ ├── control { stop, pause, resume }
	306	+ */
	307	+
	308	+static const struct {
	309	+ const char *cmd;
	310	+ int (fn)(struct xe_gt gt, unsigned int vfid);
	311	+} control_cmds[] = {
	312	+ { "stop", xe_gt_sriov_pf_control_stop_vf },
	313	+ { "pause", xe_gt_sriov_pf_control_pause_vf },
	314	+ { "resume", xe_gt_sriov_pf_control_resume_vf },
	315	+};
	316	+
	317	+static ssize_t control_write(struct file file, const char __user buf, size_t count, loff_t *pos)
	318	+{
	319	+ struct dentry *dent = file_dentry(file);
	320	+ struct dentry *parent = dent->d_parent;
	321	+ struct xe_gt *gt = extract_gt(parent);
	322	+ struct xe_device *xe = gt_to_xe(gt);
	323	+ unsigned int vfid = extract_vfid(parent);
	324	+ int ret = -EINVAL;
	325	+ char cmd[32];
	326	+ size_t n;
	327	+
	328	+ xe_gt_assert(gt, vfid);
	329	+ xe_gt_sriov_pf_assert_vfid(gt, vfid);
	330	+
	331	+ if (*pos)
	332	+ return -ESPIPE;
	333	+
	334	+ if (count > sizeof(cmd) - 1)
	335	+ return -EINVAL;
	336	+
	337	+ ret = simple_write_to_buffer(cmd, sizeof(cmd) - 1, pos, buf, count);
	338	+ if (ret < 0)
	339	+ return ret;
	340	+ cmd[ret] = '\0';
	341	+
	342	+ for (n = 0; n < ARRAY_SIZE(control_cmds); n++) {
	343	+ xe_gt_assert(gt, sizeof(cmd) > strlen(control_cmds[n].cmd));
	344	+
	345	+ if (sysfs_streq(cmd, control_cmds[n].cmd)) {
	346	+ xe_pm_runtime_get(xe);
	347	+ ret = control_cmds[n].fn ? (*control_cmds[n].fn)(gt, vfid) : 0;
	348	+ xe_pm_runtime_put(xe);
	349	+ break;
	350	+ }
	351	+ }
	352	+
	353	+ return (ret < 0) ? ret : count;
	354	+}
	355	+
	356	+static ssize_t control_read(struct file file, char __user buf, size_t count, loff_t *ppos)
	357	+{
	358	+ char help[128];
	359	+ size_t n;
	360	+
	361	+ help[0] = '\0';
	362	+ for (n = 0; n < ARRAY_SIZE(control_cmds); n++) {
	363	+ strlcat(help, control_cmds[n].cmd, sizeof(help));
	364	+ strlcat(help, "\n", sizeof(help));
	365	+ }
	366	+
	367	+ return simple_read_from_buffer(buf, count, ppos, help, strlen(help));
	368	+}
	369	+
	370	+static const struct file_operations control_ops = {
	371	+ .owner = THIS_MODULE,
	372	+ .open = simple_open,
	373	+ .write = control_write,
	374	+ .read = control_read,
	375	+ .llseek = default_llseek,
	376	+};
	377	+
	378	+/**
	379	+ * xe_gt_sriov_pf_debugfs_register - Register SR-IOV PF specific entries in GT debugfs.
	380	+ * @gt: the &xe_gt to register
	381	+ * @root: the &dentry that represents the GT directory
	382	+ *
	383	+ * Register SR-IOV PF entries that are GT related and must be shown under GT debugfs.
	384	+ */
	385	+void xe_gt_sriov_pf_debugfs_register(struct xe_gt gt, struct dentry root)
	386	+{
	387	+ struct xe_device *xe = gt_to_xe(gt);
	388	+ struct drm_minor *minor = xe->drm.primary;
	389	+ int n, totalvfs = xe_sriov_pf_get_totalvfs(xe);
	390	+ struct dentry *pfdentry;
	391	+ struct dentry *vfdentry;
	392	+ char buf[14]; /* should be enough up to "vf%u\0" for 2^32 - 1 */
	393	+
	394	+ xe_gt_assert(gt, IS_SRIOV_PF(xe));
	395	+ xe_gt_assert(gt, root->d_inode->i_private == gt);
	396	+
	397	+ /*
	398	+ * /sys/kernel/debug/dri/0/
	399	+ * ├── gt0
	400	+ * │ ├── pf
	401	+ */
	402	+ pfdentry = debugfs_create_dir("pf", root);
	403	+ if (IS_ERR(pfdentry))
	404	+ return;
	405	+ pfdentry->d_inode->i_private = gt;
	406	+
	407	+ drm_debugfs_create_files(pf_info, ARRAY_SIZE(pf_info), pfdentry, minor);
	408	+ pf_add_policy_attrs(gt, pfdentry);
	409	+ pf_add_config_attrs(gt, pfdentry, PFID);
	410	+
	411	+ for (n = 1; n <= totalvfs; n++) {
	412	+ /*
	413	+ * /sys/kernel/debug/dri/0/
	414	+ * ├── gt0
	415	+ * │ ├── vf1
	416	+ * │ ├── vf2
	417	+ */
	418	+ snprintf(buf, sizeof(buf), "vf%u", n);
	419	+ vfdentry = debugfs_create_dir(buf, root);
	420	+ if (IS_ERR(vfdentry))
	421	+ break;
	422	+ vfdentry->d_inode->i_private = (void *)(uintptr_t)n;
	423	+
	424	+ pf_add_config_attrs(gt, vfdentry, VFID(n));
	425	+ debugfs_create_file("control", 0600, vfdentry, NULL, &control_ops);
	426	+ }
	427	+}

drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.h+18−0

@@ -0,0 +1,18 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_SRIOV_PF_DEBUGFS_H_
	7	+#define _XE_GT_SRIOV_PF_DEBUGFS_H_
	8	+
	9	+struct xe_gt;
	10	+struct dentry;
	11	+
	12	+#ifdef CONFIG_PCI_IOV
	13	+void xe_gt_sriov_pf_debugfs_register(struct xe_gt gt, struct dentry root);
	14	+#else
	15	+static inline void xe_gt_sriov_pf_debugfs_register(struct xe_gt gt, struct dentry root) { }
	16	+#endif
	17	+
	18	+#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.c+147−0

@@ -0,0 +1,147 @@
	1	+// SPDX-License-Identifier: MIT
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#include "abi/guc_actions_sriov_abi.h"
	7	+#include "abi/guc_messages_abi.h"
	8	+
	9	+#include "xe_gt_sriov_pf_config.h"
	10	+#include "xe_gt_sriov_pf_helpers.h"
	11	+#include "xe_gt_sriov_pf_monitor.h"
	12	+#include "xe_gt_sriov_printk.h"
	13	+#include "xe_guc_klv_helpers.h"
	14	+#include "xe_guc_klv_thresholds_set.h"
	15	+
	16	+/**
	17	+ * xe_gt_sriov_pf_monitor_flr - Cleanup VF data after VF FLR.
	18	+ * @gt: the &xe_gt
	19	+ * @vfid: the VF identifier
	20	+ *
	21	+ * On FLR this function will reset all event data related to the VF.
	22	+ * This function is for PF only.
	23	+ */
	24	+void xe_gt_sriov_pf_monitor_flr(struct xe_gt *gt, u32 vfid)
	25	+{
	26	+ int e;
	27	+
	28	+ xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	29	+ xe_gt_sriov_pf_assert_vfid(gt, vfid);
	30	+
	31	+ for (e = 0; e < XE_GUC_KLV_NUM_THRESHOLDS; e++)
	32	+ gt->sriov.pf.vfs[vfid].monitor.guc.events[e] = 0;
	33	+}
	34	+
	35	+static void pf_update_event_counter(struct xe_gt *gt, u32 vfid,
	36	+ enum xe_guc_klv_threshold_index e)
	37	+{
	38	+ xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	39	+ xe_gt_assert(gt, e < XE_GUC_KLV_NUM_THRESHOLDS);
	40	+
	41	+ gt->sriov.pf.vfs[vfid].monitor.guc.events[e]++;
	42	+}
	43	+
	44	+static int pf_handle_vf_threshold_event(struct xe_gt *gt, u32 vfid, u32 threshold)
	45	+{
	46	+ char origin[8];
	47	+ int e;
	48	+
	49	+ e = xe_guc_klv_threshold_key_to_index(threshold);
	50	+ xe_sriov_function_name(vfid, origin, sizeof(origin));
	51	+
	52	+ /* was there a new KEY added that we missed? */
	53	+ if (unlikely(e < 0)) {
	54	+ xe_gt_sriov_notice(gt, "unknown threshold key %#x reported for %s\n",
	55	+ threshold, origin);
	56	+ return -ENOTCONN;
	57	+ }
	58	+
	59	+ xe_gt_sriov_dbg(gt, "%s exceeded threshold %u %s\n",
	60	+ origin, xe_gt_sriov_pf_config_get_threshold(gt, vfid, e),
	61	+ xe_guc_klv_key_to_string(threshold));
	62	+
	63	+ pf_update_event_counter(gt, vfid, e);
	64	+
	65	+ return 0;
	66	+}
	67	+
	68	+/**
	69	+ * xe_gt_sriov_pf_monitor_process_guc2pf - Handle adverse event notification from the GuC.
	70	+ * @gt: the &xe_gt
	71	+ * @msg: G2H event message
	72	+ * @len: length of the message
	73	+ *
	74	+ * This function is intended for PF only.
	75	+ *
	76	+ * Return: 0 on success or a negative error code on failure.
	77	+ */
	78	+int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt gt, const u32 msg, u32 len)
	79	+{
	80	+ struct xe_device *xe = gt_to_xe(gt);
	81	+ u32 vfid;
	82	+ u32 threshold;
	83	+
	84	+ xe_gt_assert(gt, len >= GUC_HXG_MSG_MIN_LEN);
	85	+ xe_gt_assert(gt, FIELD_GET(GUC_HXG_MSG_0_ORIGIN, msg[0]) == GUC_HXG_ORIGIN_GUC);
	86	+ xe_gt_assert(gt, FIELD_GET(GUC_HXG_MSG_0_TYPE, msg[0]) == GUC_HXG_TYPE_EVENT);
	87	+ xe_gt_assert(gt, FIELD_GET(GUC_HXG_EVENT_MSG_0_ACTION, msg[0]) ==
	88	+ GUC_ACTION_GUC2PF_ADVERSE_EVENT);
	89	+
	90	+ if (unlikely(!IS_SRIOV_PF(xe)))
	91	+ return -EPROTO;
	92	+
	93	+ if (unlikely(FIELD_GET(GUC2PF_ADVERSE_EVENT_EVENT_MSG_0_MBZ, msg[0])))
	94	+ return -EPFNOSUPPORT;
	95	+
	96	+ if (unlikely(len < GUC2PF_ADVERSE_EVENT_EVENT_MSG_LEN))
	97	+ return -EPROTO;
	98	+
	99	+ vfid = FIELD_GET(GUC2PF_ADVERSE_EVENT_EVENT_MSG_1_VFID, msg[1]);
	100	+ threshold = FIELD_GET(GUC2PF_ADVERSE_EVENT_EVENT_MSG_2_THRESHOLD, msg[2]);
	101	+
	102	+ if (unlikely(vfid > xe_gt_sriov_pf_get_totalvfs(gt)))
	103	+ return -EINVAL;
	104	+
	105	+ return pf_handle_vf_threshold_event(gt, vfid, threshold);
	106	+}
	107	+
	108	+/**
	109	+ * xe_gt_sriov_pf_monitor_print_events - Print adverse events counters.
	110	+ * @gt: the &xe_gt to print events from
	111	+ * @p: the &drm_printer
	112	+ *
	113	+ * Print adverse events counters for all VFs.
	114	+ * VFs with no events are not printed.
	115	+ *
	116	+ * This function can only be called on PF.
	117	+ */
	118	+void xe_gt_sriov_pf_monitor_print_events(struct xe_gt gt, struct drm_printer p)
	119	+{
	120	+ unsigned int n, total_vfs = xe_gt_sriov_pf_get_totalvfs(gt);
	121	+ const struct xe_gt_sriov_monitor *data;
	122	+ int e;
	123	+
	124	+ xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	125	+
	126	+ for (n = 1; n <= total_vfs; n++) {
	127	+ data = &gt->sriov.pf.vfs[n].monitor;
	128	+
	129	+ for (e = 0; e < XE_GUC_KLV_NUM_THRESHOLDS; e++)
	130	+ if (data->guc.events[e])
	131	+ break;
	132	+
	133	+ /* skip empty unless in debug mode */
	134	+ if (e >= XE_GUC_KLV_NUM_THRESHOLDS &&
	135	+ !IS_ENABLED(CONFIG_DRM_XE_DEBUG_SRIOV))
	136	+ continue;
	137	+
	138	+#define __format(...) "%s:%u "
	139	+#define __value(TAG, NAME, ...) , #NAME, data->guc.events[MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG)]
	140	+
	141	+ drm_printf(p, "VF%u:\t" MAKE_XE_GUC_KLV_THRESHOLDS_SET(__format) "\n",
	142	+ n MAKE_XE_GUC_KLV_THRESHOLDS_SET(__value));
	143	+
	144	+#undef __format
	145	+#undef __value
	146	+ }
	147	+}

drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor.h+27−0

@@ -0,0 +1,27 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_SRIOV_PF_MONITOR_H_
	7	+#define _XE_GT_SRIOV_PF_MONITOR_H_
	8	+
	9	+#include <linux/errno.h>
	10	+#include <linux/types.h>
	11	+
	12	+struct drm_printer;
	13	+struct xe_gt;
	14	+
	15	+void xe_gt_sriov_pf_monitor_flr(struct xe_gt *gt, u32 vfid);
	16	+void xe_gt_sriov_pf_monitor_print_events(struct xe_gt gt, struct drm_printer p);
	17	+
	18	+#ifdef CONFIG_PCI_IOV
	19	+int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt gt, const u32 msg, u32 len);
	20	+#else
	21	+static inline int xe_gt_sriov_pf_monitor_process_guc2pf(struct xe_gt gt, const u32 msg, u32 len)
	22	+{
	23	+ return -EPROTO;
	24	+}
	25	+#endif
	26	+
	27	+#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_monitor_types.h+22−0

@@ -0,0 +1,22 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_SRIOV_PF_MONITOR_TYPES_H_
	7	+#define _XE_GT_SRIOV_PF_MONITOR_TYPES_H_
	8	+
	9	+#include "xe_guc_klv_thresholds_set_types.h"
	10	+
	11	+/**
	12	+ * struct xe_gt_sriov_monitor - GT level per-VF monitoring data.
	13	+ */
	14	+struct xe_gt_sriov_monitor {
	15	+ /** @guc: monitoring data related to the GuC. */
	16	+ struct {
	17	+ /** @guc.events: number of adverse events reported by the GuC. */
	18	+ unsigned int events[XE_GUC_KLV_NUM_THRESHOLDS];
	19	+ } guc;
	20	+};
	21	+
	22	+#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c+550−0

@@ -0,0 +1,550 @@
	1	+// SPDX-License-Identifier: MIT
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#include <drm/drm_managed.h>
	7	+
	8	+#include "abi/guc_actions_sriov_abi.h"
	9	+#include "abi/guc_relay_actions_abi.h"
	10	+
	11	+#include "regs/xe_gt_regs.h"
	12	+#include "regs/xe_guc_regs.h"
	13	+#include "regs/xe_regs.h"
	14	+
	15	+#include "xe_mmio.h"
	16	+#include "xe_gt_sriov_printk.h"
	17	+#include "xe_gt_sriov_pf_helpers.h"
	18	+#include "xe_gt_sriov_pf_service.h"
	19	+#include "xe_gt_sriov_pf_service_types.h"
	20	+#include "xe_guc_ct.h"
	21	+#include "xe_guc_hxg_helpers.h"
	22	+
	23	+static void pf_init_versions(struct xe_gt *gt)
	24	+{
	25	+ BUILD_BUG_ON(!GUC_RELAY_VERSION_BASE_MAJOR && !GUC_RELAY_VERSION_BASE_MINOR);
	26	+ BUILD_BUG_ON(GUC_RELAY_VERSION_BASE_MAJOR > GUC_RELAY_VERSION_LATEST_MAJOR);
	27	+
	28	+ /* base versions may differ between platforms */
	29	+ gt->sriov.pf.service.version.base.major = GUC_RELAY_VERSION_BASE_MAJOR;
	30	+ gt->sriov.pf.service.version.base.minor = GUC_RELAY_VERSION_BASE_MINOR;
	31	+
	32	+ /* latest version is same for all platforms */
	33	+ gt->sriov.pf.service.version.latest.major = GUC_RELAY_VERSION_LATEST_MAJOR;
	34	+ gt->sriov.pf.service.version.latest.minor = GUC_RELAY_VERSION_LATEST_MINOR;
	35	+}
	36	+
	37	+/* Return: 0 on success or a negative error code on failure. */
	38	+static int pf_negotiate_version(struct xe_gt *gt,
	39	+ u32 wanted_major, u32 wanted_minor,
	40	+ u32 major, u32 minor)
	41	+{
	42	+ struct xe_gt_sriov_pf_service_version base = gt->sriov.pf.service.version.base;
	43	+ struct xe_gt_sriov_pf_service_version latest = gt->sriov.pf.service.version.latest;
	44	+
	45	+ xe_gt_assert(gt, base.major);
	46	+ xe_gt_assert(gt, base.major <= latest.major);
	47	+ xe_gt_assert(gt, (base.major < latest.major) \|\| (base.minor <= latest.minor));
	48	+
	49	+ /* VF doesn't care - return our latest */
	50	+ if (wanted_major == VF2PF_HANDSHAKE_MAJOR_ANY &&
	51	+ wanted_minor == VF2PF_HANDSHAKE_MINOR_ANY) {
	52	+ *major = latest.major;
	53	+ *minor = latest.minor;
	54	+ return 0;
	55	+ }
	56	+
	57	+ /* VF wants newer than our - return our latest */
	58	+ if (wanted_major > latest.major) {
	59	+ *major = latest.major;
	60	+ *minor = latest.minor;
	61	+ return 0;
	62	+ }
	63	+
	64	+ /* VF wants older than min required - reject */
	65	+ if (wanted_major < base.major \|\|
	66	+ (wanted_major == base.major && wanted_minor < base.minor)) {
	67	+ return -EPERM;
	68	+ }
	69	+
	70	+ /* previous major - return wanted, as we should still support it */
	71	+ if (wanted_major < latest.major) {
	72	+ /* XXX: we are not prepared for multi-versions yet */
	73	+ xe_gt_assert(gt, base.major == latest.major);
	74	+ return -ENOPKG;
	75	+ }
	76	+
	77	+ /* same major - return common minor */
	78	+ *major = wanted_major;
	79	+ *minor = min_t(u32, latest.minor, wanted_minor);
	80	+ return 0;
	81	+}
	82	+
	83	+static void pf_connect(struct xe_gt *gt, u32 vfid, u32 major, u32 minor)
	84	+{
	85	+ xe_gt_sriov_pf_assert_vfid(gt, vfid);
	86	+ xe_gt_assert(gt, major \|\| minor);
	87	+
	88	+ gt->sriov.pf.vfs[vfid].version.major = major;
	89	+ gt->sriov.pf.vfs[vfid].version.minor = minor;
	90	+}
	91	+
	92	+static void pf_disconnect(struct xe_gt *gt, u32 vfid)
	93	+{
	94	+ xe_gt_sriov_pf_assert_vfid(gt, vfid);
	95	+
	96	+ gt->sriov.pf.vfs[vfid].version.major = 0;
	97	+ gt->sriov.pf.vfs[vfid].version.minor = 0;
	98	+}
	99	+
	100	+static bool pf_is_negotiated(struct xe_gt *gt, u32 vfid, u32 major, u32 minor)
	101	+{
	102	+ xe_gt_sriov_pf_assert_vfid(gt, vfid);
	103	+
	104	+ return major == gt->sriov.pf.vfs[vfid].version.major &&
	105	+ minor <= gt->sriov.pf.vfs[vfid].version.minor;
	106	+}
	107	+
	108	+static const struct xe_reg tgl_runtime_regs[] = {
	109	+ RPM_CONFIG0, /* _MMIO(0x0d00) */
	110	+ MIRROR_FUSE3, /* _MMIO(0x9118) */
	111	+ XELP_EU_ENABLE, /* _MMIO(0x9134) */
	112	+ XELP_GT_SLICE_ENABLE, /* _MMIO(0x9138) */
	113	+ XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	114	+ GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	115	+ CTC_MODE, /* _MMIO(0xa26c) */
	116	+ HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	117	+ TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	118	+};
	119	+
	120	+static const struct xe_reg ats_m_runtime_regs[] = {
	121	+ RPM_CONFIG0, /* _MMIO(0x0d00) */
	122	+ MIRROR_FUSE3, /* _MMIO(0x9118) */
	123	+ MIRROR_FUSE1, /* _MMIO(0x911c) */
	124	+ XELP_EU_ENABLE, /* _MMIO(0x9134) */
	125	+ XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	126	+ GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	127	+ XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
	128	+ CTC_MODE, /* _MMIO(0xa26c) */
	129	+ HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	130	+ TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	131	+};
	132	+
	133	+static const struct xe_reg pvc_runtime_regs[] = {
	134	+ RPM_CONFIG0, /* _MMIO(0x0d00) */
	135	+ MIRROR_FUSE3, /* _MMIO(0x9118) */
	136	+ XELP_EU_ENABLE, /* _MMIO(0x9134) */
	137	+ XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	138	+ GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	139	+ XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
	140	+ XEHPC_GT_COMPUTE_DSS_ENABLE_EXT,/* _MMIO(0x9148) */
	141	+ CTC_MODE, /* _MMIO(0xA26C) */
	142	+ HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	143	+ TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	144	+};
	145	+
	146	+static const struct xe_reg ver_1270_runtime_regs[] = {
	147	+ RPM_CONFIG0, /* _MMIO(0x0d00) */
	148	+ XEHP_FUSE4, /* _MMIO(0x9114) */
	149	+ MIRROR_FUSE3, /* _MMIO(0x9118) */
	150	+ MIRROR_FUSE1, /* _MMIO(0x911c) */
	151	+ XELP_EU_ENABLE, /* _MMIO(0x9134) */
	152	+ XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	153	+ GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	154	+ XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
	155	+ XEHPC_GT_COMPUTE_DSS_ENABLE_EXT,/* _MMIO(0x9148) */
	156	+ CTC_MODE, /* _MMIO(0xa26c) */
	157	+ HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	158	+ TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	159	+};
	160	+
	161	+static const struct xe_reg ver_2000_runtime_regs[] = {
	162	+ RPM_CONFIG0, /* _MMIO(0x0d00) */
	163	+ XEHP_FUSE4, /* _MMIO(0x9114) */
	164	+ MIRROR_FUSE3, /* _MMIO(0x9118) */
	165	+ MIRROR_FUSE1, /* _MMIO(0x911c) */
	166	+ XELP_EU_ENABLE, /* _MMIO(0x9134) */
	167	+ XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
	168	+ GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
	169	+ XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
	170	+ XEHPC_GT_COMPUTE_DSS_ENABLE_EXT,/* _MMIO(0x9148) */
	171	+ XE2_GT_COMPUTE_DSS_2, /* _MMIO(0x914c) */
	172	+ XE2_GT_GEOMETRY_DSS_1, /* _MMIO(0x9150) */
	173	+ XE2_GT_GEOMETRY_DSS_2, /* _MMIO(0x9154) */
	174	+ CTC_MODE, /* _MMIO(0xa26c) */
	175	+ HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
	176	+ TIMESTAMP_OVERRIDE, /* _MMIO(0x44074) */
	177	+};
	178	+
	179	+static const struct xe_reg pick_runtime_regs(struct xe_device xe, unsigned int *count)
	180	+{
	181	+ const struct xe_reg *regs;
	182	+
	183	+ if (GRAPHICS_VERx100(xe) >= 2000) {
	184	+ *count = ARRAY_SIZE(ver_2000_runtime_regs);
	185	+ regs = ver_2000_runtime_regs;
	186	+ } else if (GRAPHICS_VERx100(xe) >= 1270) {
	187	+ *count = ARRAY_SIZE(ver_1270_runtime_regs);
	188	+ regs = ver_1270_runtime_regs;
	189	+ } else if (GRAPHICS_VERx100(xe) == 1260) {
	190	+ *count = ARRAY_SIZE(pvc_runtime_regs);
	191	+ regs = pvc_runtime_regs;
	192	+ } else if (GRAPHICS_VERx100(xe) == 1255) {
	193	+ *count = ARRAY_SIZE(ats_m_runtime_regs);
	194	+ regs = ats_m_runtime_regs;
	195	+ } else if (GRAPHICS_VERx100(xe) == 1200) {
	196	+ *count = ARRAY_SIZE(tgl_runtime_regs);
	197	+ regs = tgl_runtime_regs;
	198	+ } else {
	199	+ regs = ERR_PTR(-ENOPKG);
	200	+ *count = 0;
	201	+ }
	202	+
	203	+ return regs;
	204	+}
	205	+
	206	+static int pf_alloc_runtime_info(struct xe_gt *gt)
	207	+{
	208	+ struct xe_device *xe = gt_to_xe(gt);
	209	+ const struct xe_reg *regs;
	210	+ unsigned int size;
	211	+ u32 *values;
	212	+
	213	+ xe_gt_assert(gt, IS_SRIOV_PF(xe));
	214	+ xe_gt_assert(gt, !gt->sriov.pf.service.runtime.size);
	215	+ xe_gt_assert(gt, !gt->sriov.pf.service.runtime.regs);
	216	+ xe_gt_assert(gt, !gt->sriov.pf.service.runtime.values);
	217	+
	218	+ regs = pick_runtime_regs(xe, &size);
	219	+ if (IS_ERR(regs))
	220	+ return PTR_ERR(regs);
	221	+
	222	+ if (unlikely(!size))
	223	+ return 0;
	224	+
	225	+ values = drmm_kcalloc(&xe->drm, size, sizeof(u32), GFP_KERNEL);
	226	+ if (!values)
	227	+ return -ENOMEM;
	228	+
	229	+ gt->sriov.pf.service.runtime.size = size;
	230	+ gt->sriov.pf.service.runtime.regs = regs;
	231	+ gt->sriov.pf.service.runtime.values = values;
	232	+
	233	+ return 0;
	234	+}
	235	+
	236	+static void read_many(struct xe_gt *gt, unsigned int count,
	237	+ const struct xe_reg regs, u32 values)
	238	+{
	239	+ while (count--)
	240	+ values++ = xe_mmio_read32(gt, regs++);
	241	+}
	242	+
	243	+static void pf_prepare_runtime_info(struct xe_gt *gt)
	244	+{
	245	+ const struct xe_reg *regs;
	246	+ unsigned int size;
	247	+ u32 *values;
	248	+
	249	+ if (!gt->sriov.pf.service.runtime.size)
	250	+ return;
	251	+
	252	+ size = gt->sriov.pf.service.runtime.size;
	253	+ regs = gt->sriov.pf.service.runtime.regs;
	254	+ values = gt->sriov.pf.service.runtime.values;
	255	+
	256	+ read_many(gt, size, regs, values);
	257	+
	258	+ if (IS_ENABLED(CONFIG_DRM_XE_DEBUG_SRIOV)) {
	259	+ struct drm_printer p = xe_gt_info_printer(gt);
	260	+
	261	+ xe_gt_sriov_pf_service_print_runtime(gt, &p);
	262	+ }
	263	+}
	264	+
	265	+/**
	266	+ * xe_gt_sriov_pf_service_init - Early initialization of the GT SR-IOV PF services.
	267	+ * @gt: the &xe_gt to initialize
	268	+ *
	269	+ * Performs early initialization of the GT SR-IOV PF services, including preparation
	270	+ * of the runtime info that will be shared with VFs.
	271	+ *
	272	+ * This function can only be called on PF.
	273	+ */
	274	+int xe_gt_sriov_pf_service_init(struct xe_gt *gt)
	275	+{
	276	+ int err;
	277	+
	278	+ pf_init_versions(gt);
	279	+
	280	+ err = pf_alloc_runtime_info(gt);
	281	+ if (unlikely(err))
	282	+ goto failed;
	283	+
	284	+ return 0;
	285	+failed:
	286	+ xe_gt_sriov_err(gt, "Failed to initialize service (%pe)\n", ERR_PTR(err));
	287	+ return err;
	288	+}
	289	+
	290	+/**
	291	+ * xe_gt_sriov_pf_service_update - Update PF SR-IOV services.
	292	+ * @gt: the &xe_gt to update
	293	+ *
	294	+ * Updates runtime data shared with VFs.
	295	+ *
	296	+ * This function can be called more than once.
	297	+ * This function can only be called on PF.
	298	+ */
	299	+void xe_gt_sriov_pf_service_update(struct xe_gt *gt)
	300	+{
	301	+ pf_prepare_runtime_info(gt);
	302	+}
	303	+
	304	+/**
	305	+ * xe_gt_sriov_pf_service_reset - Reset a connection with the VF.
	306	+ * @gt: the &xe_gt
	307	+ * @vfid: the VF identifier
	308	+ *
	309	+ * Reset a VF driver negotiated VF/PF ABI version.
	310	+ * After that point, the VF driver will have to perform new version handshake
	311	+ * to continue use of the PF services again.
	312	+ *
	313	+ * This function can only be called on PF.
	314	+ */
	315	+void xe_gt_sriov_pf_service_reset(struct xe_gt *gt, unsigned int vfid)
	316	+{
	317	+ pf_disconnect(gt, vfid);
	318	+}
	319	+
	320	+/* Return: 0 on success or a negative error code on failure. */
	321	+static int pf_process_handshake(struct xe_gt *gt, u32 vfid,
	322	+ u32 wanted_major, u32 wanted_minor,
	323	+ u32 major, u32 minor)
	324	+{
	325	+ int err;
	326	+
	327	+ xe_gt_sriov_dbg_verbose(gt, "VF%u wants ABI version %u.%u\n",
	328	+ vfid, wanted_major, wanted_minor);
	329	+
	330	+ err = pf_negotiate_version(gt, wanted_major, wanted_minor, major, minor);
	331	+
	332	+ if (err < 0) {
	333	+ xe_gt_sriov_notice(gt, "VF%u failed to negotiate ABI %u.%u (%pe)\n",
	334	+ vfid, wanted_major, wanted_minor, ERR_PTR(err));
	335	+ pf_disconnect(gt, vfid);
	336	+ } else {
	337	+ xe_gt_sriov_dbg(gt, "VF%u negotiated ABI version %u.%u\n",
	338	+ vfid, major, minor);
	339	+ pf_connect(gt, vfid, major, minor);
	340	+ }
	341	+
	342	+ return 0;
	343	+}
	344	+
	345	+/* Return: length of the response message or a negative error code on failure. */
	346	+static int pf_process_handshake_msg(struct xe_gt *gt, u32 origin,
	347	+ const u32 request, u32 len, u32 response, u32 size)
	348	+{
	349	+ u32 wanted_major, wanted_minor;
	350	+ u32 major, minor;
	351	+ u32 mbz;
	352	+ int err;
	353	+
	354	+ if (unlikely(len != VF2PF_HANDSHAKE_REQUEST_MSG_LEN))
	355	+ return -EMSGSIZE;
	356	+
	357	+ mbz = FIELD_GET(VF2PF_HANDSHAKE_REQUEST_MSG_0_MBZ, request[0]);
	358	+ if (unlikely(mbz))
	359	+ return -EPFNOSUPPORT;
	360	+
	361	+ wanted_major = FIELD_GET(VF2PF_HANDSHAKE_REQUEST_MSG_1_MAJOR, request[1]);
	362	+ wanted_minor = FIELD_GET(VF2PF_HANDSHAKE_REQUEST_MSG_1_MINOR, request[1]);
	363	+
	364	+ err = pf_process_handshake(gt, origin, wanted_major, wanted_minor, &major, &minor);
	365	+ if (err < 0)
	366	+ return err;
	367	+
	368	+ xe_gt_assert(gt, major \|\| minor);
	369	+ xe_gt_assert(gt, size >= VF2PF_HANDSHAKE_RESPONSE_MSG_LEN);
	370	+
	371	+ response[0] = FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	372	+ FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_RESPONSE_SUCCESS) \|
	373	+ FIELD_PREP(GUC_HXG_RESPONSE_MSG_0_DATA0, 0);
	374	+ response[1] = FIELD_PREP(VF2PF_HANDSHAKE_RESPONSE_MSG_1_MAJOR, major) \|
	375	+ FIELD_PREP(VF2PF_HANDSHAKE_RESPONSE_MSG_1_MINOR, minor);
	376	+
	377	+ return VF2PF_HANDSHAKE_RESPONSE_MSG_LEN;
	378	+}
	379	+
	380	+struct reg_data {
	381	+ u32 offset;
	382	+ u32 value;
	383	+} __packed;
	384	+static_assert(hxg_sizeof(struct reg_data) == 2);
	385	+
	386	+/* Return: number of entries copied or negative error code on failure. */
	387	+static int pf_service_runtime_query(struct xe_gt *gt, u32 start, u32 limit,
	388	+ struct reg_data data, u32 remaining)
	389	+{
	390	+ struct xe_gt_sriov_pf_service_runtime_regs *runtime;
	391	+ unsigned int count, i;
	392	+ u32 addr;
	393	+
	394	+ xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	395	+
	396	+ runtime = &gt->sriov.pf.service.runtime;
	397	+
	398	+ if (start > runtime->size)
	399	+ return -ERANGE;
	400	+
	401	+ count = min_t(u32, runtime->size - start, limit);
	402	+
	403	+ for (i = 0; i < count; ++i, ++data) {
	404	+ addr = runtime->regs[start + i].addr;
	405	+ data->offset = xe_mmio_adjusted_addr(gt, addr);
	406	+ data->value = runtime->values[start + i];
	407	+ }
	408	+
	409	+ *remaining = runtime->size - start - count;
	410	+ return count;
	411	+}
	412	+
	413	+/* Return: length of the response message or a negative error code on failure. */
	414	+static int pf_process_runtime_query_msg(struct xe_gt *gt, u32 origin,
	415	+ const u32 msg, u32 msg_len, u32 response, u32 resp_size)
	416	+{
	417	+ const u32 chunk_size = hxg_sizeof(struct reg_data);
	418	+ struct reg_data *reg_data_buf;
	419	+ u32 limit, start, max_chunks;
	420	+ u32 remaining = 0;
	421	+ int ret;
	422	+
	423	+ if (!pf_is_negotiated(gt, origin, 1, 0))
	424	+ return -EACCES;
	425	+ if (unlikely(msg_len > VF2PF_QUERY_RUNTIME_REQUEST_MSG_LEN))
	426	+ return -EMSGSIZE;
	427	+ if (unlikely(msg_len < VF2PF_QUERY_RUNTIME_REQUEST_MSG_LEN))
	428	+ return -EPROTO;
	429	+ if (unlikely(resp_size < VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN))
	430	+ return -EINVAL;
	431	+
	432	+ limit = FIELD_GET(VF2PF_QUERY_RUNTIME_REQUEST_MSG_0_LIMIT, msg[0]);
	433	+ start = FIELD_GET(VF2PF_QUERY_RUNTIME_REQUEST_MSG_1_START, msg[1]);
	434	+
	435	+ resp_size = min_t(u32, resp_size, VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MAX_LEN);
	436	+ max_chunks = (resp_size - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) / chunk_size;
	437	+ limit = limit == VF2PF_QUERY_RUNTIME_NO_LIMIT ? max_chunks : min_t(u32, max_chunks, limit);
	438	+ reg_data_buf = (void *)(response + VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN);
	439	+
	440	+ ret = pf_service_runtime_query(gt, start, limit, reg_data_buf, &remaining);
	441	+ if (ret < 0)
	442	+ return ret;
	443	+
	444	+ response[0] = FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	445	+ FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_RESPONSE_SUCCESS) \|
	446	+ FIELD_PREP(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_0_COUNT, ret);
	447	+ response[1] = FIELD_PREP(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_1_REMAINING, remaining);
	448	+
	449	+ return VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + ret * hxg_sizeof(struct reg_data);
	450	+}
	451	+
	452	+/**
	453	+ * xe_gt_sriov_pf_service_process_request - Service GT level SR-IOV request message from the VF.
	454	+ * @gt: the &xe_gt that provides the service
	455	+ * @origin: VF number that is requesting the service
	456	+ * @msg: request message
	457	+ * @msg_len: length of the request message (in dwords)
	458	+ * @response: placeholder for the response message
	459	+ * @resp_size: length of the response message buffer (in dwords)
	460	+ *
	461	+ * This function processes `Relay Message`_ request from the VF.
	462	+ *
	463	+ * Return: length of the response message or a negative error code on failure.
	464	+ */
	465	+int xe_gt_sriov_pf_service_process_request(struct xe_gt *gt, u32 origin,
	466	+ const u32 *msg, u32 msg_len,
	467	+ u32 *response, u32 resp_size)
	468	+{
	469	+ u32 action, data __maybe_unused;
	470	+ int ret;
	471	+
	472	+ xe_gt_assert(gt, msg_len >= GUC_HXG_MSG_MIN_LEN);
	473	+ xe_gt_assert(gt, FIELD_GET(GUC_HXG_MSG_0_TYPE, msg[0]) == GUC_HXG_TYPE_REQUEST);
	474	+
	475	+ action = FIELD_GET(GUC_HXG_REQUEST_MSG_0_ACTION, msg[0]);
	476	+ data = FIELD_GET(GUC_HXG_REQUEST_MSG_0_DATA0, msg[0]);
	477	+ xe_gt_sriov_dbg_verbose(gt, "service action %#x:%u from VF%u\n",
	478	+ action, data, origin);
	479	+
	480	+ switch (action) {
	481	+ case GUC_RELAY_ACTION_VF2PF_HANDSHAKE:
	482	+ ret = pf_process_handshake_msg(gt, origin, msg, msg_len, response, resp_size);
	483	+ break;
	484	+ case GUC_RELAY_ACTION_VF2PF_QUERY_RUNTIME:
	485	+ ret = pf_process_runtime_query_msg(gt, origin, msg, msg_len, response, resp_size);
	486	+ break;
	487	+ default:
	488	+ ret = -EOPNOTSUPP;
	489	+ break;
	490	+ }
	491	+
	492	+ return ret;
	493	+}
	494	+
	495	+/**
	496	+ * xe_gt_sriov_pf_service_print_runtime - Print PF runtime data shared with VFs.
	497	+ * @gt: the &xe_gt
	498	+ * @p: the &drm_printer
	499	+ *
	500	+ * This function is for PF use only.
	501	+ */
	502	+int xe_gt_sriov_pf_service_print_runtime(struct xe_gt gt, struct drm_printer p)
	503	+{
	504	+ const struct xe_reg *regs;
	505	+ unsigned int size;
	506	+ u32 *values;
	507	+
	508	+ xe_gt_assert(gt, IS_SRIOV_PF(gt_to_xe(gt)));
	509	+
	510	+ size = gt->sriov.pf.service.runtime.size;
	511	+ regs = gt->sriov.pf.service.runtime.regs;
	512	+ values = gt->sriov.pf.service.runtime.values;
	513	+
	514	+ for (; size--; regs++, values++) {
	515	+ drm_printf(p, "reg[%#x] = %#x\n",
	516	+ xe_mmio_adjusted_addr(gt, regs->addr), *values);
	517	+ }
	518	+
	519	+ return 0;
	520	+}
	521	+
	522	+/**
	523	+ * xe_gt_sriov_pf_service_print_version - Print ABI versions negotiated with VFs.
	524	+ * @gt: the &xe_gt
	525	+ * @p: the &drm_printer
	526	+ *
	527	+ * This function is for PF use only.
	528	+ */
	529	+int xe_gt_sriov_pf_service_print_version(struct xe_gt gt, struct drm_printer p)
	530	+{
	531	+ struct xe_device *xe = gt_to_xe(gt);
	532	+ unsigned int n, total_vfs = xe_sriov_pf_get_totalvfs(xe);
	533	+ struct xe_gt_sriov_pf_service_version *version;
	534	+
	535	+ xe_gt_assert(gt, IS_SRIOV_PF(xe));
	536	+
	537	+ for (n = 1; n <= total_vfs; n++) {
	538	+ version = &gt->sriov.pf.vfs[n].version;
	539	+ if (!version->major && !version->minor)
	540	+ continue;
	541	+
	542	+ drm_printf(p, "VF%u:\t%u.%u\n", n, version->major, version->minor);
	543	+ }
	544	+
	545	+ return 0;
	546	+}
	547	+
	548	+#if IS_BUILTIN(CONFIG_DRM_XE_KUNIT_TEST)
	549	+#include "tests/xe_gt_sriov_pf_service_test.c"
	550	+#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_service.h+36−0

@@ -0,0 +1,36 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_SRIOV_PF_SERVICE_H_
	7	+#define _XE_GT_SRIOV_PF_SERVICE_H_
	8	+
	9	+#include <linux/errno.h>
	10	+#include <linux/types.h>
	11	+
	12	+struct drm_printer;
	13	+struct xe_gt;
	14	+
	15	+int xe_gt_sriov_pf_service_init(struct xe_gt *gt);
	16	+void xe_gt_sriov_pf_service_update(struct xe_gt *gt);
	17	+void xe_gt_sriov_pf_service_reset(struct xe_gt *gt, unsigned int vfid);
	18	+
	19	+int xe_gt_sriov_pf_service_print_version(struct xe_gt gt, struct drm_printer p);
	20	+int xe_gt_sriov_pf_service_print_runtime(struct xe_gt gt, struct drm_printer p);
	21	+
	22	+#ifdef CONFIG_PCI_IOV
	23	+int xe_gt_sriov_pf_service_process_request(struct xe_gt *gt, u32 origin,
	24	+ const u32 *msg, u32 msg_len,
	25	+ u32 *response, u32 resp_size);
	26	+#else
	27	+static inline int
	28	+xe_gt_sriov_pf_service_process_request(struct xe_gt *gt, u32 origin,
	29	+ const u32 *msg, u32 msg_len,
	30	+ u32 *response, u32 resp_size)
	31	+{
	32	+ return -EPROTO;
	33	+}
	34	+#endif
	35	+
	36	+#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_service_types.h+52−0

@@ -0,0 +1,52 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_SRIOV_PF_SERVICE_TYPES_H_
	7	+#define _XE_GT_SRIOV_PF_SERVICE_TYPES_H_
	8	+
	9	+#include <linux/types.h>
	10	+
	11	+struct xe_reg;
	12	+
	13	+/**
	14	+ * struct xe_gt_sriov_pf_service_version - VF/PF ABI Version.
	15	+ * @major: the major version of the VF/PF ABI
	16	+ * @minor: the minor version of the VF/PF ABI
	17	+ *
	18	+ * See `GuC Relay Communication`_.
	19	+ */
	20	+struct xe_gt_sriov_pf_service_version {
	21	+ u16 major;
	22	+ u16 minor;
	23	+};
	24	+
	25	+/**
	26	+ * struct xe_gt_sriov_pf_service_runtime_regs - Runtime data shared with VFs.
	27	+ * @regs: pointer to static array with register offsets.
	28	+ * @values: pointer to array with captured register values.
	29	+ * @size: size of the regs and value arrays.
	30	+ */
	31	+struct xe_gt_sriov_pf_service_runtime_regs {
	32	+ const struct xe_reg *regs;
	33	+ u32 *values;
	34	+ u32 size;
	35	+};
	36	+
	37	+/**
	38	+ * struct xe_gt_sriov_pf_service - Data used by the PF service.
	39	+ * @version: information about VF/PF ABI versions for current platform.
	40	+ * @version.base: lowest VF/PF ABI version that could be negotiated with VF.
	41	+ * @version.latest: latest VF/PF ABI version supported by the PF driver.
	42	+ * @runtime: runtime data shared with VFs.
	43	+ */
	44	+struct xe_gt_sriov_pf_service {
	45	+ struct {
	46	+ struct xe_gt_sriov_pf_service_version base;
	47	+ struct xe_gt_sriov_pf_service_version latest;
	48	+ } version;
	49	+ struct xe_gt_sriov_pf_service_runtime_regs runtime;
	50	+};
	51	+
	52	+#endif

drivers/gpu/drm/xe/xe_gt_sriov_pf_types.h+10−0

@@ -9,7 +9,9 @@ · 8 unmodified lines
9	9	#include <linux/types.h>
10	10
11	11	#include "xe_gt_sriov_pf_config_types.h"
	12	+#include "xe_gt_sriov_pf_monitor_types.h"
12	13	#include "xe_gt_sriov_pf_policy_types.h"
	14	+#include "xe_gt_sriov_pf_service_types.h"
13	15
14	16	/**
15	17	* struct xe_gt_sriov_metadata - GT level per-VF metadata.
@@ -17,15 +19,23 @@ · 1 unmodified line
17	19	struct xe_gt_sriov_metadata {
18	20	/** @config: per-VF provisioning data. */
19	21	struct xe_gt_sriov_config config;
	22	+
	23	+ /** @monitor: per-VF monitoring data. */
	24	+ struct xe_gt_sriov_monitor monitor;
	25	+
	26	+ /** @version: negotiated VF/PF ABI version */
	27	+ struct xe_gt_sriov_pf_service_version version;
20	28	};
21	29
22	30	/**
23	31	* struct xe_gt_sriov_pf - GT level PF virtualization data.
	32	+ * @service: service data.
24	33	* @policy: policy data.
25	34	* @spare: PF-only provisioning configuration.
26	35	* @vfs: metadata for all VFs.
27	36	*/
28	37	struct xe_gt_sriov_pf {
	38	+ struct xe_gt_sriov_pf_service service;
29	39	struct xe_gt_sriov_pf_policy policy;
30	40	struct xe_gt_sriov_spare_config spare;
31	41	struct xe_gt_sriov_metadata *vfs;

drivers/gpu/drm/xe/xe_gt_sriov_vf.c+979−0

@@ -0,0 +1,979 @@
	1	+// SPDX-License-Identifier: MIT
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#include <linux/bitfield.h>
	7	+#include <linux/bsearch.h>
	8	+
	9	+#include <drm/drm_managed.h>
	10	+#include <drm/drm_print.h>
	11	+
	12	+#include "abi/guc_actions_sriov_abi.h"
	13	+#include "abi/guc_communication_mmio_abi.h"
	14	+#include "abi/guc_klvs_abi.h"
	15	+#include "abi/guc_relay_actions_abi.h"
	16	+#include "regs/xe_gt_regs.h"
	17	+#include "regs/xe_gtt_defs.h"
	18	+
	19	+#include "xe_assert.h"
	20	+#include "xe_device.h"
	21	+#include "xe_ggtt.h"
	22	+#include "xe_gt_sriov_printk.h"
	23	+#include "xe_gt_sriov_vf.h"
	24	+#include "xe_gt_sriov_vf_types.h"
	25	+#include "xe_guc.h"
	26	+#include "xe_guc_hxg_helpers.h"
	27	+#include "xe_guc_relay.h"
	28	+#include "xe_mmio.h"
	29	+#include "xe_sriov.h"
	30	+#include "xe_uc_fw.h"
	31	+#include "xe_wopcm.h"
	32	+
	33	+#define make_u64_from_u32(hi, lo) ((u64)((u64)(u32)(hi) << 32 \| (u32)(lo)))
	34	+
	35	+static int guc_action_vf_reset(struct xe_guc *guc)
	36	+{
	37	+ u32 request[GUC_HXG_REQUEST_MSG_MIN_LEN] = {
	38	+ FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	39	+ FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	40	+ FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION, GUC_ACTION_VF2GUC_VF_RESET),
	41	+ };
	42	+ int ret;
	43	+
	44	+ ret = xe_guc_mmio_send(guc, request, ARRAY_SIZE(request));
	45	+
	46	+ return ret > 0 ? -EPROTO : ret;
	47	+}
	48	+
	49	+static int vf_reset_guc_state(struct xe_gt *gt)
	50	+{
	51	+ struct xe_guc *guc = &gt->uc.guc;
	52	+ int err;
	53	+
	54	+ err = guc_action_vf_reset(guc);
	55	+ if (unlikely(err))
	56	+ xe_gt_sriov_err(gt, "Failed to reset GuC state (%pe)\n", ERR_PTR(err));
	57	+ return err;
	58	+}
	59	+
	60	+static int guc_action_match_version(struct xe_guc *guc,
	61	+ u32 wanted_branch, u32 wanted_major, u32 wanted_minor,
	62	+ u32 branch, u32 major, u32 minor, u32 patch)
	63	+{
	64	+ u32 request[VF2GUC_MATCH_VERSION_REQUEST_MSG_LEN] = {
	65	+ FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	66	+ FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	67	+ FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION,
	68	+ GUC_ACTION_VF2GUC_MATCH_VERSION),
	69	+ FIELD_PREP(VF2GUC_MATCH_VERSION_REQUEST_MSG_1_BRANCH, wanted_branch) \|
	70	+ FIELD_PREP(VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MAJOR, wanted_major) \|
	71	+ FIELD_PREP(VF2GUC_MATCH_VERSION_REQUEST_MSG_1_MINOR, wanted_minor),
	72	+ };
	73	+ u32 response[GUC_MAX_MMIO_MSG_LEN];
	74	+ int ret;
	75	+
	76	+ BUILD_BUG_ON(VF2GUC_MATCH_VERSION_RESPONSE_MSG_LEN > GUC_MAX_MMIO_MSG_LEN);
	77	+
	78	+ ret = xe_guc_mmio_send_recv(guc, request, ARRAY_SIZE(request), response);
	79	+ if (unlikely(ret < 0))
	80	+ return ret;
	81	+
	82	+ if (unlikely(FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_0_MBZ, response[0])))
	83	+ return -EPROTO;
	84	+
	85	+ *branch = FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_BRANCH, response[1]);
	86	+ *major = FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_MAJOR, response[1]);
	87	+ *minor = FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_MINOR, response[1]);
	88	+ *patch = FIELD_GET(VF2GUC_MATCH_VERSION_RESPONSE_MSG_1_PATCH, response[1]);
	89	+
	90	+ return 0;
	91	+}
	92	+
	93	+static void vf_minimum_guc_version(struct xe_gt gt, u32 branch, u32 major, u32 minor)
	94	+{
	95	+ struct xe_device *xe = gt_to_xe(gt);
	96	+
	97	+ switch (xe->info.platform) {
	98	+ case XE_TIGERLAKE ... XE_PVC:
	99	+ /* 1.1 this is current baseline for Xe driver */
	100	+ *branch = 0;
	101	+ *major = 1;
	102	+ *minor = 1;
	103	+ break;
	104	+ default:
	105	+ /* 1.2 has support for the GMD_ID KLV */
	106	+ *branch = 0;
	107	+ *major = 1;
	108	+ *minor = 2;
	109	+ break;
	110	+ }
	111	+}
	112	+
	113	+static void vf_wanted_guc_version(struct xe_gt gt, u32 branch, u32 major, u32 minor)
	114	+{
	115	+ /* for now it's the same as minimum */
	116	+ return vf_minimum_guc_version(gt, branch, major, minor);
	117	+}
	118	+
	119	+static int vf_handshake_with_guc(struct xe_gt *gt)
	120	+{
	121	+ struct xe_gt_sriov_vf_guc_version *guc_version = &gt->sriov.vf.guc_version;
	122	+ struct xe_guc *guc = &gt->uc.guc;
	123	+ u32 wanted_branch, wanted_major, wanted_minor;
	124	+ u32 branch, major, minor, patch;
	125	+ int err;
	126	+
	127	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	128	+
	129	+ /* select wanted version - prefer previous (if any) */
	130	+ if (guc_version->major \|\| guc_version->minor) {
	131	+ wanted_branch = guc_version->branch;
	132	+ wanted_major = guc_version->major;
	133	+ wanted_minor = guc_version->minor;
	134	+ } else {
	135	+ vf_wanted_guc_version(gt, &wanted_branch, &wanted_major, &wanted_minor);
	136	+ xe_gt_assert(gt, wanted_major != GUC_VERSION_MAJOR_ANY);
	137	+ }
	138	+
	139	+ err = guc_action_match_version(guc, wanted_branch, wanted_major, wanted_minor,
	140	+ &branch, &major, &minor, &patch);
	141	+ if (unlikely(err))
	142	+ goto fail;
	143	+
	144	+ /* we don't support interface version change */
	145	+ if ((guc_version->major \|\| guc_version->minor) &&
	146	+ (guc_version->branch != branch \|\| guc_version->major != major \|\|
	147	+ guc_version->minor != minor)) {
	148	+ xe_gt_sriov_err(gt, "New GuC interface version detected: %u.%u.%u.%u\n",
	149	+ branch, major, minor, patch);
	150	+ xe_gt_sriov_info(gt, "Previously used version was: %u.%u.%u.%u\n",
	151	+ guc_version->branch, guc_version->major,
	152	+ guc_version->minor, guc_version->patch);
	153	+ err = -EREMCHG;
	154	+ goto fail;
	155	+ }
	156	+
	157	+ /* illegal */
	158	+ if (major > wanted_major) {
	159	+ err = -EPROTO;
	160	+ goto unsupported;
	161	+ }
	162	+
	163	+ /* there's no fallback on major version. */
	164	+ if (major != wanted_major) {
	165	+ err = -ENOPKG;
	166	+ goto unsupported;
	167	+ }
	168	+
	169	+ /* check against minimum version supported by us */
	170	+ vf_minimum_guc_version(gt, &wanted_branch, &wanted_major, &wanted_minor);
	171	+ xe_gt_assert(gt, major != GUC_VERSION_MAJOR_ANY);
	172	+ if (major < wanted_major \|\| (major == wanted_major && minor < wanted_minor)) {
	173	+ err = -ENOKEY;
	174	+ goto unsupported;
	175	+ }
	176	+
	177	+ xe_gt_sriov_dbg(gt, "using GuC interface version %u.%u.%u.%u\n",
	178	+ branch, major, minor, patch);
	179	+
	180	+ guc_version->branch = branch;
	181	+ guc_version->major = major;
	182	+ guc_version->minor = minor;
	183	+ guc_version->patch = patch;
	184	+ return 0;
	185	+
	186	+unsupported:
	187	+ xe_gt_sriov_err(gt, "Unsupported GuC version %u.%u.%u.%u (%pe)\n",
	188	+ branch, major, minor, patch, ERR_PTR(err));
	189	+fail:
	190	+ xe_gt_sriov_err(gt, "Unable to confirm GuC version %u.%u (%pe)\n",
	191	+ wanted_major, wanted_minor, ERR_PTR(err));
	192	+
	193	+ /* try again with any just to query which version is supported */
	194	+ if (!guc_action_match_version(guc, GUC_VERSION_BRANCH_ANY,
	195	+ GUC_VERSION_MAJOR_ANY, GUC_VERSION_MINOR_ANY,
	196	+ &branch, &major, &minor, &patch))
	197	+ xe_gt_sriov_notice(gt, "GuC reports interface version %u.%u.%u.%u\n",
	198	+ branch, major, minor, patch);
	199	+ return err;
	200	+}
	201	+
	202	+/**
	203	+ * xe_gt_sriov_vf_bootstrap - Query and setup GuC ABI interface version.
	204	+ * @gt: the &xe_gt
	205	+ *
	206	+ * This function is for VF use only.
	207	+ * It requires functional `GuC MMIO based communication`_.
	208	+ *
	209	+ * Return: 0 on success or a negative error code on failure.
	210	+ */
	211	+int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt)
	212	+{
	213	+ int err;
	214	+
	215	+ err = vf_reset_guc_state(gt);
	216	+ if (unlikely(err))
	217	+ return err;
	218	+
	219	+ err = vf_handshake_with_guc(gt);
	220	+ if (unlikely(err))
	221	+ return err;
	222	+
	223	+ return 0;
	224	+}
	225	+
	226	+static int guc_action_query_single_klv(struct xe_guc *guc, u32 key,
	227	+ u32 *value, u32 value_len)
	228	+{
	229	+ u32 request[VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_LEN] = {
	230	+ FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	231	+ FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	232	+ FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION,
	233	+ GUC_ACTION_VF2GUC_QUERY_SINGLE_KLV),
	234	+ FIELD_PREP(VF2GUC_QUERY_SINGLE_KLV_REQUEST_MSG_1_KEY, key),
	235	+ };
	236	+ u32 response[GUC_MAX_MMIO_MSG_LEN];
	237	+ u32 length;
	238	+ int ret;
	239	+
	240	+ BUILD_BUG_ON(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_MAX_LEN > GUC_MAX_MMIO_MSG_LEN);
	241	+ ret = xe_guc_mmio_send_recv(guc, request, ARRAY_SIZE(request), response);
	242	+ if (unlikely(ret < 0))
	243	+ return ret;
	244	+
	245	+ if (unlikely(FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_0_MBZ, response[0])))
	246	+ return -EPROTO;
	247	+
	248	+ length = FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_0_LENGTH, response[0]);
	249	+ if (unlikely(length > value_len))
	250	+ return -EOVERFLOW;
	251	+ if (unlikely(length < value_len))
	252	+ return -ENODATA;
	253	+
	254	+ switch (value_len) {
	255	+ default:
	256	+ xe_gt_WARN_ON(guc_to_gt(guc), value_len > 3);
	257	+ fallthrough;
	258	+ case 3:
	259	+ value[2] = FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_3_VALUE96, response[3]);
	260	+ fallthrough;
	261	+ case 2:
	262	+ value[1] = FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_2_VALUE64, response[2]);
	263	+ fallthrough;
	264	+ case 1:
	265	+ value[0] = FIELD_GET(VF2GUC_QUERY_SINGLE_KLV_RESPONSE_MSG_1_VALUE32, response[1]);
	266	+ fallthrough;
	267	+ case 0:
	268	+ break;
	269	+ }
	270	+
	271	+ return 0;
	272	+}
	273	+
	274	+static int guc_action_query_single_klv32(struct xe_guc guc, u32 key, u32 value32)
	275	+{
	276	+ return guc_action_query_single_klv(guc, key, value32, hxg_sizeof(u32));
	277	+}
	278	+
	279	+static int guc_action_query_single_klv64(struct xe_guc guc, u32 key, u64 value64)
	280	+{
	281	+ u32 value[2];
	282	+ int err;
	283	+
	284	+ err = guc_action_query_single_klv(guc, key, value, hxg_sizeof(value));
	285	+ if (unlikely(err))
	286	+ return err;
	287	+
	288	+ *value64 = make_u64_from_u32(value[1], value[0]);
	289	+ return 0;
	290	+}
	291	+
	292	+static bool has_gmdid(struct xe_device *xe)
	293	+{
	294	+ return GRAPHICS_VERx100(xe) >= 1270;
	295	+}
	296	+
	297	+/**
	298	+ * xe_gt_sriov_vf_gmdid - Query GMDID over MMIO.
	299	+ * @gt: the &xe_gt
	300	+ *
	301	+ * This function is for VF use only.
	302	+ *
	303	+ * Return: value of GMDID KLV on success or 0 on failure.
	304	+ */
	305	+u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt)
	306	+{
	307	+ const char *type = xe_gt_is_media_type(gt) ? "media" : "graphics";
	308	+ struct xe_guc *guc = &gt->uc.guc;
	309	+ u32 value;
	310	+ int err;
	311	+
	312	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	313	+ xe_gt_assert(gt, !GRAPHICS_VERx100(gt_to_xe(gt)) \|\| has_gmdid(gt_to_xe(gt)));
	314	+ xe_gt_assert(gt, gt->sriov.vf.guc_version.major > 1 \|\| gt->sriov.vf.guc_version.minor >= 2);
	315	+
	316	+ err = guc_action_query_single_klv32(guc, GUC_KLV_GLOBAL_CFG_GMD_ID_KEY, &value);
	317	+ if (unlikely(err)) {
	318	+ xe_gt_sriov_err(gt, "Failed to obtain %s GMDID (%pe)\n",
	319	+ type, ERR_PTR(err));
	320	+ return 0;
	321	+ }
	322	+
	323	+ xe_gt_sriov_dbg(gt, "%s GMDID = %#x\n", type, value);
	324	+ return value;
	325	+}
	326	+
	327	+static int vf_get_ggtt_info(struct xe_gt *gt)
	328	+{
	329	+ struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	330	+ struct xe_guc *guc = &gt->uc.guc;
	331	+ u64 start, size;
	332	+ int err;
	333	+
	334	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	335	+
	336	+ err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
	337	+ if (unlikely(err))
	338	+ return err;
	339	+
	340	+ err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_SIZE_KEY, &size);
	341	+ if (unlikely(err))
	342	+ return err;
	343	+
	344	+ if (config->ggtt_size && config->ggtt_size != size) {
	345	+ xe_gt_sriov_err(gt, "Unexpected GGTT reassignment: %lluK != %lluK\n",
	346	+ size / SZ_1K, config->ggtt_size / SZ_1K);
	347	+ return -EREMCHG;
	348	+ }
	349	+
	350	+ xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
	351	+ start, start + size - 1, size / SZ_1K);
	352	+
	353	+ config->ggtt_base = start;
	354	+ config->ggtt_size = size;
	355	+
	356	+ return config->ggtt_size ? 0 : -ENODATA;
	357	+}
	358	+
	359	+static int vf_get_lmem_info(struct xe_gt *gt)
	360	+{
	361	+ struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	362	+ struct xe_guc *guc = &gt->uc.guc;
	363	+ char size_str[10];
	364	+ u64 size;
	365	+ int err;
	366	+
	367	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	368	+
	369	+ err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
	370	+ if (unlikely(err))
	371	+ return err;
	372	+
	373	+ if (config->lmem_size && config->lmem_size != size) {
	374	+ xe_gt_sriov_err(gt, "Unexpected LMEM reassignment: %lluM != %lluM\n",
	375	+ size / SZ_1M, config->lmem_size / SZ_1M);
	376	+ return -EREMCHG;
	377	+ }
	378	+
	379	+ string_get_size(size, 1, STRING_UNITS_2, size_str, sizeof(size_str));
	380	+ xe_gt_sriov_dbg_verbose(gt, "LMEM %lluM %s\n", size / SZ_1M, size_str);
	381	+
	382	+ config->lmem_size = size;
	383	+
	384	+ return config->lmem_size ? 0 : -ENODATA;
	385	+}
	386	+
	387	+static int vf_get_submission_cfg(struct xe_gt *gt)
	388	+{
	389	+ struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	390	+ struct xe_guc *guc = &gt->uc.guc;
	391	+ u32 num_ctxs, num_dbs;
	392	+ int err;
	393	+
	394	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	395	+
	396	+ err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
	397	+ if (unlikely(err))
	398	+ return err;
	399	+
	400	+ err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_DOORBELLS_KEY, &num_dbs);
	401	+ if (unlikely(err))
	402	+ return err;
	403	+
	404	+ if (config->num_ctxs && config->num_ctxs != num_ctxs) {
	405	+ xe_gt_sriov_err(gt, "Unexpected CTXs reassignment: %u != %u\n",
	406	+ num_ctxs, config->num_ctxs);
	407	+ return -EREMCHG;
	408	+ }
	409	+ if (config->num_dbs && config->num_dbs != num_dbs) {
	410	+ xe_gt_sriov_err(gt, "Unexpected DBs reassignment: %u != %u\n",
	411	+ num_dbs, config->num_dbs);
	412	+ return -EREMCHG;
	413	+ }
	414	+
	415	+ xe_gt_sriov_dbg_verbose(gt, "CTXs %u DBs %u\n", num_ctxs, num_dbs);
	416	+
	417	+ config->num_ctxs = num_ctxs;
	418	+ config->num_dbs = num_dbs;
	419	+
	420	+ return config->num_ctxs ? 0 : -ENODATA;
	421	+}
	422	+
	423	+static void vf_cache_gmdid(struct xe_gt *gt)
	424	+{
	425	+ xe_gt_assert(gt, has_gmdid(gt_to_xe(gt)));
	426	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	427	+
	428	+ gt->sriov.vf.runtime.gmdid = xe_gt_sriov_vf_gmdid(gt);
	429	+}
	430	+
	431	+/**
	432	+ * xe_gt_sriov_vf_query_config - Query SR-IOV config data over MMIO.
	433	+ * @gt: the &xe_gt
	434	+ *
	435	+ * This function is for VF use only.
	436	+ *
	437	+ * Return: 0 on success or a negative error code on failure.
	438	+ */
	439	+int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
	440	+{
	441	+ struct xe_device *xe = gt_to_xe(gt);
	442	+ int err;
	443	+
	444	+ err = vf_get_ggtt_info(gt);
	445	+ if (unlikely(err))
	446	+ return err;
	447	+
	448	+ if (IS_DGFX(xe) && !xe_gt_is_media_type(gt)) {
	449	+ err = vf_get_lmem_info(gt);
	450	+ if (unlikely(err))
	451	+ return err;
	452	+ }
	453	+
	454	+ err = vf_get_submission_cfg(gt);
	455	+ if (unlikely(err))
	456	+ return err;
	457	+
	458	+ if (has_gmdid(xe))
	459	+ vf_cache_gmdid(gt);
	460	+
	461	+ return 0;
	462	+}
	463	+
	464	+/**
	465	+ * xe_gt_sriov_vf_guc_ids - VF GuC context IDs configuration.
	466	+ * @gt: the &xe_gt
	467	+ *
	468	+ * This function is for VF use only.
	469	+ *
	470	+ * Return: number of GuC context IDs assigned to VF.
	471	+ */
	472	+u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
	473	+{
	474	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	475	+ xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
	476	+ xe_gt_assert(gt, gt->sriov.vf.self_config.num_ctxs);
	477	+
	478	+ return gt->sriov.vf.self_config.num_ctxs;
	479	+}
	480	+
	481	+/**
	482	+ * xe_gt_sriov_vf_lmem - VF LMEM configuration.
	483	+ * @gt: the &xe_gt
	484	+ *
	485	+ * This function is for VF use only.
	486	+ *
	487	+ * Return: size of the LMEM assigned to VF.
	488	+ */
	489	+u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
	490	+{
	491	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	492	+ xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
	493	+ xe_gt_assert(gt, gt->sriov.vf.self_config.lmem_size);
	494	+
	495	+ return gt->sriov.vf.self_config.lmem_size;
	496	+}
	497	+
	498	+static int vf_balloon_ggtt(struct xe_gt *gt)
	499	+{
	500	+ struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	501	+ struct xe_tile *tile = gt_to_tile(gt);
	502	+ struct xe_ggtt *ggtt = tile->mem.ggtt;
	503	+ struct xe_device *xe = gt_to_xe(gt);
	504	+ u64 start, end;
	505	+ int err;
	506	+
	507	+ xe_gt_assert(gt, IS_SRIOV_VF(xe));
	508	+ xe_gt_assert(gt, !xe_gt_is_media_type(gt));
	509	+
	510	+ if (!config->ggtt_size)
	511	+ return -ENODATA;
	512	+
	513	+ /*
	514	+ * VF can only use part of the GGTT as allocated by the PF:
	515	+ *
	516	+ * WOPCM GUC_GGTT_TOP
	517	+ * \|<------------ Total GGTT size ------------------>\|
	518	+ *
	519	+ * VF GGTT base -->\|<- size ->\|
	520	+ *
	521	+ * +--------------------+----------+-----------------+
	522	+ * \|////////////////////\| block \|\\\\\\\\\\\\\\\\\\|
	523	+ * +--------------------+----------+-----------------+
	524	+ *
	525	+ * \|<--- balloon[0] --->\|<-- VF -->\|<-- balloon[1] ->\|
	526	+ */
	527	+
	528	+ start = xe_wopcm_size(xe);
	529	+ end = config->ggtt_base;
	530	+ if (end != start) {
	531	+ err = xe_ggtt_balloon(ggtt, start, end, &tile->sriov.vf.ggtt_balloon[0]);
	532	+ if (err)
	533	+ goto failed;
	534	+ }
	535	+
	536	+ start = config->ggtt_base + config->ggtt_size;
	537	+ end = GUC_GGTT_TOP;
	538	+ if (end != start) {
	539	+ err = xe_ggtt_balloon(ggtt, start, end, &tile->sriov.vf.ggtt_balloon[1]);
	540	+ if (err)
	541	+ goto deballoon;
	542	+ }
	543	+
	544	+ return 0;
	545	+
	546	+deballoon:
	547	+ xe_ggtt_deballoon(ggtt, &tile->sriov.vf.ggtt_balloon[0]);
	548	+failed:
	549	+ return err;
	550	+}
	551	+
	552	+static void deballoon_ggtt(struct drm_device drm, void arg)
	553	+{
	554	+ struct xe_tile *tile = arg;
	555	+ struct xe_ggtt *ggtt = tile->mem.ggtt;
	556	+
	557	+ xe_tile_assert(tile, IS_SRIOV_VF(tile_to_xe(tile)));
	558	+ xe_ggtt_deballoon(ggtt, &tile->sriov.vf.ggtt_balloon[1]);
	559	+ xe_ggtt_deballoon(ggtt, &tile->sriov.vf.ggtt_balloon[0]);
	560	+}
	561	+
	562	+/**
	563	+ * xe_gt_sriov_vf_prepare_ggtt - Prepare a VF's GGTT configuration.
	564	+ * @gt: the &xe_gt
	565	+ *
	566	+ * This function is for VF use only.
	567	+ *
	568	+ * Return: 0 on success or a negative error code on failure.
	569	+ */
	570	+int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt)
	571	+{
	572	+ struct xe_tile *tile = gt_to_tile(gt);
	573	+ struct xe_device *xe = tile_to_xe(tile);
	574	+ int err;
	575	+
	576	+ if (xe_gt_is_media_type(gt))
	577	+ return 0;
	578	+
	579	+ err = vf_balloon_ggtt(gt);
	580	+ if (err)
	581	+ return err;
	582	+
	583	+ return drmm_add_action_or_reset(&xe->drm, deballoon_ggtt, tile);
	584	+}
	585	+
	586	+static int relay_action_handshake(struct xe_gt gt, u32 major, u32 *minor)
	587	+{
	588	+ u32 request[VF2PF_HANDSHAKE_REQUEST_MSG_LEN] = {
	589	+ FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	590	+ FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	591	+ FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION, GUC_RELAY_ACTION_VF2PF_HANDSHAKE),
	592	+ FIELD_PREP(VF2PF_HANDSHAKE_REQUEST_MSG_1_MAJOR, *major) \|
	593	+ FIELD_PREP(VF2PF_HANDSHAKE_REQUEST_MSG_1_MINOR, *minor),
	594	+ };
	595	+ u32 response[VF2PF_HANDSHAKE_RESPONSE_MSG_LEN];
	596	+ int ret;
	597	+
	598	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	599	+
	600	+ ret = xe_guc_relay_send_to_pf(&gt->uc.guc.relay,
	601	+ request, ARRAY_SIZE(request),
	602	+ response, ARRAY_SIZE(response));
	603	+ if (unlikely(ret < 0))
	604	+ return ret;
	605	+
	606	+ if (unlikely(ret != VF2PF_HANDSHAKE_RESPONSE_MSG_LEN))
	607	+ return -EPROTO;
	608	+
	609	+ if (unlikely(FIELD_GET(VF2PF_HANDSHAKE_RESPONSE_MSG_0_MBZ, response[0])))
	610	+ return -EPROTO;
	611	+
	612	+ *major = FIELD_GET(VF2PF_HANDSHAKE_RESPONSE_MSG_1_MAJOR, response[1]);
	613	+ *minor = FIELD_GET(VF2PF_HANDSHAKE_RESPONSE_MSG_1_MINOR, response[1]);
	614	+
	615	+ return 0;
	616	+}
	617	+
	618	+static void vf_connect_pf(struct xe_gt *gt, u16 major, u16 minor)
	619	+{
	620	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	621	+
	622	+ gt->sriov.vf.pf_version.major = major;
	623	+ gt->sriov.vf.pf_version.minor = minor;
	624	+}
	625	+
	626	+static void vf_disconnect_pf(struct xe_gt *gt)
	627	+{
	628	+ vf_connect_pf(gt, 0, 0);
	629	+}
	630	+
	631	+static int vf_handshake_with_pf(struct xe_gt *gt)
	632	+{
	633	+ u32 major_wanted = GUC_RELAY_VERSION_LATEST_MAJOR;
	634	+ u32 minor_wanted = GUC_RELAY_VERSION_LATEST_MINOR;
	635	+ u32 major = major_wanted, minor = minor_wanted;
	636	+ int err;
	637	+
	638	+ err = relay_action_handshake(gt, &major, &minor);
	639	+ if (unlikely(err))
	640	+ goto failed;
	641	+
	642	+ if (!major && !minor) {
	643	+ err = -ENODATA;
	644	+ goto failed;
	645	+ }
	646	+
	647	+ xe_gt_sriov_dbg(gt, "using VF/PF ABI %u.%u\n", major, minor);
	648	+ vf_connect_pf(gt, major, minor);
	649	+ return 0;
	650	+
	651	+failed:
	652	+ xe_gt_sriov_err(gt, "Unable to confirm VF/PF ABI version %u.%u (%pe)\n",
	653	+ major, minor, ERR_PTR(err));
	654	+ vf_disconnect_pf(gt);
	655	+ return err;
	656	+}
	657	+
	658	+/**
	659	+ * xe_gt_sriov_vf_connect - Establish connection with the PF driver.
	660	+ * @gt: the &xe_gt
	661	+ *
	662	+ * This function is for VF use only.
	663	+ *
	664	+ * Return: 0 on success or a negative error code on failure.
	665	+ */
	666	+int xe_gt_sriov_vf_connect(struct xe_gt *gt)
	667	+{
	668	+ int err;
	669	+
	670	+ err = vf_handshake_with_pf(gt);
	671	+ if (unlikely(err))
	672	+ goto failed;
	673	+
	674	+ return 0;
	675	+
	676	+failed:
	677	+ xe_gt_sriov_err(gt, "Failed to get version info (%pe)\n", ERR_PTR(err));
	678	+ return err;
	679	+}
	680	+
	681	+static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
	682	+{
	683	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	684	+
	685	+ return major == gt->sriov.vf.pf_version.major &&
	686	+ minor <= gt->sriov.vf.pf_version.minor;
	687	+}
	688	+
	689	+static int vf_prepare_runtime_info(struct xe_gt *gt, unsigned int num_regs)
	690	+{
	691	+ struct vf_runtime_reg *regs = gt->sriov.vf.runtime.regs;
	692	+ unsigned int regs_size = round_up(num_regs, 4);
	693	+ struct xe_device *xe = gt_to_xe(gt);
	694	+
	695	+ xe_gt_assert(gt, IS_SRIOV_VF(xe));
	696	+
	697	+ if (regs) {
	698	+ if (num_regs <= gt->sriov.vf.runtime.regs_size) {
	699	+ memset(regs, 0, num_regs * sizeof(*regs));
	700	+ gt->sriov.vf.runtime.num_regs = num_regs;
	701	+ return 0;
	702	+ }
	703	+
	704	+ drmm_kfree(&xe->drm, regs);
	705	+ gt->sriov.vf.runtime.regs = NULL;
	706	+ gt->sriov.vf.runtime.num_regs = 0;
	707	+ gt->sriov.vf.runtime.regs_size = 0;
	708	+ }
	709	+
	710	+ regs = drmm_kcalloc(&xe->drm, regs_size, sizeof(*regs), GFP_KERNEL);
	711	+ if (unlikely(!regs))
	712	+ return -ENOMEM;
	713	+
	714	+ gt->sriov.vf.runtime.regs = regs;
	715	+ gt->sriov.vf.runtime.num_regs = num_regs;
	716	+ gt->sriov.vf.runtime.regs_size = regs_size;
	717	+ return 0;
	718	+}
	719	+
	720	+static int vf_query_runtime_info(struct xe_gt *gt)
	721	+{
	722	+ u32 request[VF2PF_QUERY_RUNTIME_REQUEST_MSG_LEN];
	723	+ u32 response[VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + 32]; /* up to 16 regs */
	724	+ u32 limit = (ARRAY_SIZE(response) - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) / 2;
	725	+ u32 count, remaining, num, i;
	726	+ u32 start = 0;
	727	+ int ret;
	728	+
	729	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	730	+ xe_gt_assert(gt, limit);
	731	+
	732	+ /* this is part of the 1.0 PF/VF ABI */
	733	+ if (!vf_is_negotiated(gt, 1, 0))
	734	+ return -ENOPKG;
	735	+
	736	+ request[0] = FIELD_PREP(GUC_HXG_MSG_0_ORIGIN, GUC_HXG_ORIGIN_HOST) \|
	737	+ FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) \|
	738	+ FIELD_PREP(GUC_HXG_REQUEST_MSG_0_ACTION,
	739	+ GUC_RELAY_ACTION_VF2PF_QUERY_RUNTIME) \|
	740	+ FIELD_PREP(VF2PF_QUERY_RUNTIME_REQUEST_MSG_0_LIMIT, limit);
	741	+
	742	+repeat:
	743	+ request[1] = FIELD_PREP(VF2PF_QUERY_RUNTIME_REQUEST_MSG_1_START, start);
	744	+ ret = xe_guc_relay_send_to_pf(&gt->uc.guc.relay,
	745	+ request, ARRAY_SIZE(request),
	746	+ response, ARRAY_SIZE(response));
	747	+ if (unlikely(ret < 0))
	748	+ goto failed;
	749	+
	750	+ if (unlikely(ret < VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN)) {
	751	+ ret = -EPROTO;
	752	+ goto failed;
	753	+ }
	754	+ if (unlikely((ret - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) % 2)) {
	755	+ ret = -EPROTO;
	756	+ goto failed;
	757	+ }
	758	+
	759	+ num = (ret - VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN) / 2;
	760	+ count = FIELD_GET(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_0_COUNT, response[0]);
	761	+ remaining = FIELD_GET(VF2PF_QUERY_RUNTIME_RESPONSE_MSG_1_REMAINING, response[1]);
	762	+
	763	+ xe_gt_sriov_dbg_verbose(gt, "count=%u num=%u ret=%d start=%u remaining=%u\n",
	764	+ count, num, ret, start, remaining);
	765	+
	766	+ if (unlikely(count != num)) {
	767	+ ret = -EPROTO;
	768	+ goto failed;
	769	+ }
	770	+
	771	+ if (start == 0) {
	772	+ ret = vf_prepare_runtime_info(gt, num + remaining);
	773	+ if (unlikely(ret < 0))
	774	+ goto failed;
	775	+ } else if (unlikely(start + num > gt->sriov.vf.runtime.num_regs)) {
	776	+ ret = -EPROTO;
	777	+ goto failed;
	778	+ }
	779	+
	780	+ for (i = 0; i < num; ++i) {
	781	+ struct vf_runtime_reg *reg = &gt->sriov.vf.runtime.regs[start + i];
	782	+
	783	+ reg->offset = response[VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + 2 * i];
	784	+ reg->value = response[VF2PF_QUERY_RUNTIME_RESPONSE_MSG_MIN_LEN + 2 * i + 1];
	785	+ }
	786	+
	787	+ if (remaining) {
	788	+ start += num;
	789	+ goto repeat;
	790	+ }
	791	+
	792	+ return 0;
	793	+
	794	+failed:
	795	+ vf_prepare_runtime_info(gt, 0);
	796	+ return ret;
	797	+}
	798	+
	799	+static void vf_show_runtime_info(struct xe_gt *gt)
	800	+{
	801	+ struct vf_runtime_reg *vf_regs = gt->sriov.vf.runtime.regs;
	802	+ unsigned int size = gt->sriov.vf.runtime.num_regs;
	803	+
	804	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	805	+
	806	+ for (; size--; vf_regs++)
	807	+ xe_gt_sriov_dbg(gt, "runtime(%#x) = %#x\n",
	808	+ vf_regs->offset, vf_regs->value);
	809	+}
	810	+
	811	+/**
	812	+ * xe_gt_sriov_vf_query_runtime - Query SR-IOV runtime data.
	813	+ * @gt: the &xe_gt
	814	+ *
	815	+ * This function is for VF use only.
	816	+ *
	817	+ * Return: 0 on success or a negative error code on failure.
	818	+ */
	819	+int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt)
	820	+{
	821	+ int err;
	822	+
	823	+ err = vf_query_runtime_info(gt);
	824	+ if (unlikely(err))
	825	+ goto failed;
	826	+
	827	+ if (IS_ENABLED(CONFIG_DRM_XE_DEBUG))
	828	+ vf_show_runtime_info(gt);
	829	+
	830	+ return 0;
	831	+
	832	+failed:
	833	+ xe_gt_sriov_err(gt, "Failed to get runtime info (%pe)\n",
	834	+ ERR_PTR(err));
	835	+ return err;
	836	+}
	837	+
	838	+static int vf_runtime_reg_cmp(const void a, const void b)
	839	+{
	840	+ const struct vf_runtime_reg *ra = a;
	841	+ const struct vf_runtime_reg *rb = b;
	842	+
	843	+ return (int)ra->offset - (int)rb->offset;
	844	+}
	845	+
	846	+static struct vf_runtime_reg vf_lookup_reg(struct xe_gt gt, u32 addr)
	847	+{
	848	+ struct xe_gt_sriov_vf_runtime *runtime = &gt->sriov.vf.runtime;
	849	+ struct vf_runtime_reg key = { .offset = addr };
	850	+
	851	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	852	+
	853	+ return bsearch(&key, runtime->regs, runtime->regs_size, sizeof(key),
	854	+ vf_runtime_reg_cmp);
	855	+}
	856	+
	857	+/**
	858	+ * xe_gt_sriov_vf_read32 - Get a register value from the runtime data.
	859	+ * @gt: the &xe_gt
	860	+ * @reg: the register to read
	861	+ *
	862	+ * This function is for VF use only.
	863	+ * This function shall be called after VF has connected to PF.
	864	+ * This function is dedicated for registers that VFs can't read directly.
	865	+ *
	866	+ * Return: register value obtained from the PF or 0 if not found.
	867	+ */
	868	+u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg)
	869	+{
	870	+ u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
	871	+ struct vf_runtime_reg *rr;
	872	+
	873	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	874	+ xe_gt_assert(gt, gt->sriov.vf.pf_version.major);
	875	+ xe_gt_assert(gt, !reg.vf);
	876	+
	877	+ if (reg.addr == GMD_ID.addr) {
	878	+ xe_gt_sriov_dbg_verbose(gt, "gmdid(%#x) = %#x\n",
	879	+ addr, gt->sriov.vf.runtime.gmdid);
	880	+ return gt->sriov.vf.runtime.gmdid;
	881	+ }
	882	+
	883	+ rr = vf_lookup_reg(gt, addr);
	884	+ if (!rr) {
	885	+ xe_gt_WARN(gt, IS_ENABLED(CONFIG_DRM_XE_DEBUG),
	886	+ "VF is trying to read an inaccessible register %#x+%#x\n",
	887	+ reg.addr, addr - reg.addr);
	888	+ return 0;
	889	+ }
	890	+
	891	+ xe_gt_sriov_dbg_verbose(gt, "runtime[%#x] = %#x\n", addr, rr->value);
	892	+ return rr->value;
	893	+}
	894	+
	895	+/**
	896	+ * xe_gt_sriov_vf_print_config - Print VF self config.
	897	+ * @gt: the &xe_gt
	898	+ * @p: the &drm_printer
	899	+ *
	900	+ * This function is for VF use only.
	901	+ */
	902	+void xe_gt_sriov_vf_print_config(struct xe_gt gt, struct drm_printer p)
	903	+{
	904	+ struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
	905	+ struct xe_device *xe = gt_to_xe(gt);
	906	+ char buf[10];
	907	+
	908	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	909	+
	910	+ drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
	911	+ config->ggtt_base,
	912	+ config->ggtt_base + config->ggtt_size - 1);
	913	+
	914	+ string_get_size(config->ggtt_size, 1, STRING_UNITS_2, buf, sizeof(buf));
	915	+ drm_printf(p, "GGTT size:\t%llu (%s)\n", config->ggtt_size, buf);
	916	+
	917	+ if (IS_DGFX(xe) && !xe_gt_is_media_type(gt)) {
	918	+ string_get_size(config->lmem_size, 1, STRING_UNITS_2, buf, sizeof(buf));
	919	+ drm_printf(p, "LMEM size:\t%llu (%s)\n", config->lmem_size, buf);
	920	+ }
	921	+
	922	+ drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
	923	+ drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
	924	+}
	925	+
	926	+/**
	927	+ * xe_gt_sriov_vf_print_runtime - Print VF's runtime regs received from PF.
	928	+ * @gt: the &xe_gt
	929	+ * @p: the &drm_printer
	930	+ *
	931	+ * This function is for VF use only.
	932	+ */
	933	+void xe_gt_sriov_vf_print_runtime(struct xe_gt gt, struct drm_printer p)
	934	+{
	935	+ struct vf_runtime_reg *vf_regs = gt->sriov.vf.runtime.regs;
	936	+ unsigned int size = gt->sriov.vf.runtime.num_regs;
	937	+
	938	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	939	+
	940	+ for (; size--; vf_regs++)
	941	+ drm_printf(p, "%#x = %#x\n", vf_regs->offset, vf_regs->value);
	942	+}
	943	+
	944	+/**
	945	+ * xe_gt_sriov_vf_print_version - Print VF ABI versions.
	946	+ * @gt: the &xe_gt
	947	+ * @p: the &drm_printer
	948	+ *
	949	+ * This function is for VF use only.
	950	+ */
	951	+void xe_gt_sriov_vf_print_version(struct xe_gt gt, struct drm_printer p)
	952	+{
	953	+ struct xe_gt_sriov_vf_guc_version *guc_version = &gt->sriov.vf.guc_version;
	954	+ struct xe_gt_sriov_vf_relay_version *pf_version = &gt->sriov.vf.pf_version;
	955	+ u32 branch, major, minor;
	956	+
	957	+ xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
	958	+
	959	+ drm_printf(p, "GuC ABI:\n");
	960	+
	961	+ vf_minimum_guc_version(gt, &branch, &major, &minor);
	962	+ drm_printf(p, "\tbase:\t%u.%u.%u.*\n", branch, major, minor);
	963	+
	964	+ vf_wanted_guc_version(gt, &branch, &major, &minor);
	965	+ drm_printf(p, "\twanted:\t%u.%u.%u.*\n", branch, major, minor);
	966	+
	967	+ drm_printf(p, "\thandshake:\t%u.%u.%u.%u\n",
	968	+ guc_version->branch, guc_version->major,
	969	+ guc_version->minor, guc_version->patch);
	970	+
	971	+ drm_printf(p, "PF ABI:\n");
	972	+
	973	+ drm_printf(p, "\tbase:\t%u.%u\n",
	974	+ GUC_RELAY_VERSION_BASE_MAJOR, GUC_RELAY_VERSION_BASE_MINOR);
	975	+ drm_printf(p, "\twanted:\t%u.%u\n",
	976	+ GUC_RELAY_VERSION_LATEST_MAJOR, GUC_RELAY_VERSION_LATEST_MINOR);
	977	+ drm_printf(p, "\thandshake:\t%u.%u\n",
	978	+ pf_version->major, pf_version->minor);
	979	+}

drivers/gpu/drm/xe/xe_gt_sriov_vf.h+30−0

@@ -0,0 +1,30 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_SRIOV_VF_H_
	7	+#define _XE_GT_SRIOV_VF_H_
	8	+
	9	+#include <linux/types.h>
	10	+
	11	+struct drm_printer;
	12	+struct xe_gt;
	13	+struct xe_reg;
	14	+
	15	+int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);
	16	+int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
	17	+int xe_gt_sriov_vf_connect(struct xe_gt *gt);
	18	+int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
	19	+int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt);
	20	+
	21	+u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
	22	+u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
	23	+u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
	24	+u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg);
	25	+
	26	+void xe_gt_sriov_vf_print_config(struct xe_gt gt, struct drm_printer p);
	27	+void xe_gt_sriov_vf_print_runtime(struct xe_gt gt, struct drm_printer p);
	28	+void xe_gt_sriov_vf_print_version(struct xe_gt gt, struct drm_printer p);
	29	+
	30	+#endif

drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.c+72−0

@@ -0,0 +1,72 @@
	1	+// SPDX-License-Identifier: MIT
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#include <linux/debugfs.h>
	7	+
	8	+#include <drm/drm_debugfs.h>
	9	+
	10	+#include "xe_gt_debugfs.h"
	11	+#include "xe_gt_sriov_vf.h"
	12	+#include "xe_gt_sriov_vf_debugfs.h"
	13	+#include "xe_gt_types.h"
	14	+#include "xe_sriov.h"
	15	+
	16	+/*
	17	+ * /sys/kernel/debug/dri/0/
	18	+ * ├── gt0
	19	+ * │ ├── vf
	20	+ * │ │ ├── self_config
	21	+ * │ │ ├── abi_versions
	22	+ * │ │ ├── runtime_regs
	23	+ */
	24	+
	25	+static const struct drm_info_list vf_info[] = {
	26	+ {
	27	+ "self_config",
	28	+ .show = xe_gt_debugfs_simple_show,
	29	+ .data = xe_gt_sriov_vf_print_config,
	30	+ },
	31	+ {
	32	+ "abi_versions",
	33	+ .show = xe_gt_debugfs_simple_show,
	34	+ .data = xe_gt_sriov_vf_print_version,
	35	+ },
	36	+#if defined(CONFIG_DRM_XE_DEBUG) \|\| defined(CONFIG_DRM_XE_DEBUG_SRIOV)
	37	+ {
	38	+ "runtime_regs",
	39	+ .show = xe_gt_debugfs_simple_show,
	40	+ .data = xe_gt_sriov_vf_print_runtime,
	41	+ },
	42	+#endif
	43	+};
	44	+
	45	+/**
	46	+ * xe_gt_sriov_vf_debugfs_register - Register SR-IOV VF specific entries in GT debugfs.
	47	+ * @gt: the &xe_gt to register
	48	+ * @root: the &dentry that represents the GT directory
	49	+ *
	50	+ * Register SR-IOV VF entries that are GT related and must be shown under GT debugfs.
	51	+ */
	52	+void xe_gt_sriov_vf_debugfs_register(struct xe_gt gt, struct dentry root)
	53	+{
	54	+ struct xe_device *xe = gt_to_xe(gt);
	55	+ struct drm_minor *minor = xe->drm.primary;
	56	+ struct dentry *vfdentry;
	57	+
	58	+ xe_assert(xe, IS_SRIOV_VF(xe));
	59	+ xe_assert(xe, root->d_inode->i_private == gt);
	60	+
	61	+ /*
	62	+ * /sys/kernel/debug/dri/0/
	63	+ * ├── gt0
	64	+ * │ ├── vf
	65	+ */
	66	+ vfdentry = debugfs_create_dir("vf", root);
	67	+ if (IS_ERR(vfdentry))
	68	+ return;
	69	+ vfdentry->d_inode->i_private = gt;
	70	+
	71	+ drm_debugfs_create_files(vf_info, ARRAY_SIZE(vf_info), vfdentry, minor);
	72	+}

drivers/gpu/drm/xe/xe_gt_sriov_vf_debugfs.h+14−0

@@ -0,0 +1,14 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_SRIOV_VF_DEBUGFS_H_
	7	+#define _XE_GT_SRIOV_VF_DEBUGFS_H_
	8	+
	9	+struct xe_gt;
	10	+struct dentry;
	11	+
	12	+void xe_gt_sriov_vf_debugfs_register(struct xe_gt gt, struct dentry root);
	13	+
	14	+#endif

drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h+84−0

@@ -0,0 +1,84 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_SRIOV_VF_TYPES_H_
	7	+#define _XE_GT_SRIOV_VF_TYPES_H_
	8	+
	9	+#include <linux/types.h>
	10	+
	11	+/**
	12	+ * struct xe_gt_sriov_vf_guc_version - GuC ABI version details.
	13	+ */
	14	+struct xe_gt_sriov_vf_guc_version {
	15	+ /** @branch: branch version. */
	16	+ u8 branch;
	17	+ /** @major: major version. */
	18	+ u8 major;
	19	+ /** @minor: minor version. */
	20	+ u8 minor;
	21	+ /** @patch: patch version. */
	22	+ u8 patch;
	23	+};
	24	+
	25	+/**
	26	+ * struct xe_gt_sriov_vf_relay_version - PF ABI version details.
	27	+ */
	28	+struct xe_gt_sriov_vf_relay_version {
	29	+ /** @major: major version. */
	30	+ u16 major;
	31	+ /** @minor: minor version. */
	32	+ u16 minor;
	33	+};
	34	+
	35	+/**
	36	+ * struct xe_gt_sriov_vf_selfconfig - VF configuration data.
	37	+ */
	38	+struct xe_gt_sriov_vf_selfconfig {
	39	+ /** @ggtt_base: assigned base offset of the GGTT region. */
	40	+ u64 ggtt_base;
	41	+ /** @ggtt_size: assigned size of the GGTT region. */
	42	+ u64 ggtt_size;
	43	+ /** @lmem_size: assigned size of the LMEM. */
	44	+ u64 lmem_size;
	45	+ /** @num_ctxs: assigned number of GuC submission context IDs. */
	46	+ u16 num_ctxs;
	47	+ /** @num_dbs: assigned number of GuC doorbells IDs. */
	48	+ u16 num_dbs;
	49	+};
	50	+
	51	+/**
	52	+ * struct xe_gt_sriov_vf_runtime - VF runtime data.
	53	+ */
	54	+struct xe_gt_sriov_vf_runtime {
	55	+ /** @gmdid: cached value of the GDMID register. */
	56	+ u32 gmdid;
	57	+ /** @regs_size: size of runtime register array. */
	58	+ u32 regs_size;
	59	+ /** @num_regs: number of runtime registers in the array. */
	60	+ u32 num_regs;
	61	+ /** @regs: pointer to array of register offset/value pairs. */
	62	+ struct vf_runtime_reg {
	63	+ /** @regs.offset: register offset. */
	64	+ u32 offset;
	65	+ /** @regs.value: register value. */
	66	+ u32 value;
	67	+ } *regs;
	68	+};
	69	+
	70	+/**
	71	+ * struct xe_gt_sriov_vf - GT level VF virtualization data.
	72	+ */
	73	+struct xe_gt_sriov_vf {
	74	+ /** @guc_version: negotiated GuC ABI version. */
	75	+ struct xe_gt_sriov_vf_guc_version guc_version;
	76	+ /** @self_config: resource configurations. */
	77	+ struct xe_gt_sriov_vf_selfconfig self_config;
	78	+ /** @pf_version: negotiated VF/PF ABI version. */
	79	+ struct xe_gt_sriov_vf_relay_version pf_version;
	80	+ /** @runtime: runtime data retrieved from the PF. */
	81	+ struct xe_gt_sriov_vf_runtime runtime;
	82	+};
	83	+
	84	+#endif

drivers/gpu/drm/xe/xe_gt_sysfs.c+2−2

@@ -22,7 +22,7 @@ static const struct kobj_type xe_gt_sysfs_kobj_type = { · 21 unmodified lines
22	22	.sysfs_ops = &kobj_sysfs_ops,
23	23	};
24	24
25		−static void gt_sysfs_fini(struct drm_device drm, void arg)
	25	+static void gt_sysfs_fini(void *arg)
26	26	{
27	27	struct xe_gt *gt = arg;
28	28
@@ -51,5 +51,5 @@ int xe_gt_sysfs_init(struct xe_gt *gt) · 22 unmodified lines
51	51
52	52	gt->sysfs = &kg->base;
53	53
54		− return drmm_add_action_or_reset(&xe->drm, gt_sysfs_fini, gt);
	54	+ return devm_add_action(xe->drm.dev, gt_sysfs_fini, gt);
55	55	}

drivers/gpu/drm/xe/xe_gt_throttle.crenamed from drivers/gpu/drm/xe/xe_gt_throttle_sysfs.c+15−15

@@ -9,14 +9,14 @@ · 8 unmodified lines
9	9	#include "xe_device.h"
10	10	#include "xe_gt.h"
11	11	#include "xe_gt_sysfs.h"
12		−#include "xe_gt_throttle_sysfs.h"
	12	+#include "xe_gt_throttle.h"
13	13	#include "xe_mmio.h"
14	14	#include "xe_pm.h"
15	15
16	16	/**
17	17	* DOC: Xe GT Throttle
18	18	*
19		− * Provides sysfs entries for frequency throttle reasons in GT
	19	+ * Provides sysfs entries and other helpers for frequency throttle reasons in GT
20	20	*
21	21	* device/gt#/freq0/throttle/status - Overall status
22	22	* device/gt#/freq0/throttle/reason_pl1 - Frequency throttle due to PL1
@@ -35,7 +35,7 @@ dev_to_gt(struct device *dev) · 12 unmodified lines
35	35	return kobj_to_gt(dev->kobj.parent);
36	36	}
37	37
38		−static u32 read_perf_limit_reasons(struct xe_gt *gt)
	38	+u32 xe_gt_throttle_get_limit_reasons(struct xe_gt *gt)
39	39	{
40	40	u32 reg;
41	41
@@ -51,63 +51,63 @@ static u32 read_perf_limit_reasons(struct xe_gt *gt) · 9 unmodified lines
51	51
52	52	static u32 read_status(struct xe_gt *gt)
53	53	{
54		− u32 status = read_perf_limit_reasons(gt) & GT0_PERF_LIMIT_REASONS_MASK;
	54	+ u32 status = xe_gt_throttle_get_limit_reasons(gt) & GT0_PERF_LIMIT_REASONS_MASK;
55	55
56	56	return status;
57	57	}
58	58
59	59	static u32 read_reason_pl1(struct xe_gt *gt)
60	60	{
61		− u32 pl1 = read_perf_limit_reasons(gt) & POWER_LIMIT_1_MASK;
	61	+ u32 pl1 = xe_gt_throttle_get_limit_reasons(gt) & POWER_LIMIT_1_MASK;
62	62
63	63	return pl1;
64	64	}
65	65
66	66	static u32 read_reason_pl2(struct xe_gt *gt)
67	67	{
68		− u32 pl2 = read_perf_limit_reasons(gt) & POWER_LIMIT_2_MASK;
	68	+ u32 pl2 = xe_gt_throttle_get_limit_reasons(gt) & POWER_LIMIT_2_MASK;
69	69
70	70	return pl2;
71	71	}
72	72
73	73	static u32 read_reason_pl4(struct xe_gt *gt)
74	74	{
75		− u32 pl4 = read_perf_limit_reasons(gt) & POWER_LIMIT_4_MASK;
	75	+ u32 pl4 = xe_gt_throttle_get_limit_reasons(gt) & POWER_LIMIT_4_MASK;
76	76
77	77	return pl4;
78	78	}
79	79
80	80	static u32 read_reason_thermal(struct xe_gt *gt)
81	81	{
82		− u32 thermal = read_perf_limit_reasons(gt) & THERMAL_LIMIT_MASK;
	82	+ u32 thermal = xe_gt_throttle_get_limit_reasons(gt) & THERMAL_LIMIT_MASK;
83	83
84	84	return thermal;
85	85	}
86	86
87	87	static u32 read_reason_prochot(struct xe_gt *gt)
88	88	{
89		− u32 prochot = read_perf_limit_reasons(gt) & PROCHOT_MASK;
	89	+ u32 prochot = xe_gt_throttle_get_limit_reasons(gt) & PROCHOT_MASK;
90	90
91	91	return prochot;
92	92	}
93	93
94	94	static u32 read_reason_ratl(struct xe_gt *gt)
95	95	{
96		− u32 ratl = read_perf_limit_reasons(gt) & RATL_MASK;
	96	+ u32 ratl = xe_gt_throttle_get_limit_reasons(gt) & RATL_MASK;
97	97
98	98	return ratl;
99	99	}
100	100
101	101	static u32 read_reason_vr_thermalert(struct xe_gt *gt)
102	102	{
103		− u32 thermalert = read_perf_limit_reasons(gt) & VR_THERMALERT_MASK;
	103	+ u32 thermalert = xe_gt_throttle_get_limit_reasons(gt) & VR_THERMALERT_MASK;
104	104
105	105	return thermalert;
106	106	}
107	107
108	108	static u32 read_reason_vr_tdc(struct xe_gt *gt)
109	109	{
110		− u32 tdc = read_perf_limit_reasons(gt) & VR_TDC_MASK;
	110	+ u32 tdc = xe_gt_throttle_get_limit_reasons(gt) & VR_TDC_MASK;
111	111
112	112	return tdc;
113	113	}
@@ -229,14 +229,14 @@ static const struct attribute_group throttle_group_attrs = { · 115 unmodified lines
229	229	.attrs = throttle_attrs,
230	230	};
231	231
232		−static void gt_throttle_sysfs_fini(struct drm_device drm, void arg)
	232	+static void gt_throttle_sysfs_fini(void *arg)
233	233	{
234	234	struct xe_gt *gt = arg;
235	235
236	236	sysfs_remove_group(gt->freq, &throttle_group_attrs);
237	237	}
238	238
239		−int xe_gt_throttle_sysfs_init(struct xe_gt *gt)
	239	+int xe_gt_throttle_init(struct xe_gt *gt)
240	240	{
241	241	struct xe_device *xe = gt_to_xe(gt);
242	242	int err;
@@ -245,5 +245,5 @@ int xe_gt_throttle_sysfs_init(struct xe_gt *gt) · 2 unmodified lines
245	245	if (err)
246	246	return err;
247	247
248		− return drmm_add_action_or_reset(&xe->drm, gt_throttle_sysfs_fini, gt);
	248	+ return devm_add_action_or_reset(xe->drm.dev, gt_throttle_sysfs_fini, gt);
249	249	}

drivers/gpu/drm/xe/xe_gt_throttle.h+17−0

@@ -0,0 +1,17 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GT_THROTTLE_H_
	7	+#define _XE_GT_THROTTLE_H_
	8	+
	9	+#include <linux/types.h>
	10	+
	11	+struct xe_gt;
	12	+
	13	+int xe_gt_throttle_init(struct xe_gt *gt);
	14	+
	15	+u32 xe_gt_throttle_get_limit_reasons(struct xe_gt *gt);
	16	+
	17	+#endif /* _XE_GT_THROTTLE_H_ */

drivers/gpu/drm/xe/xe_gt_throttle_sysfs.h+0−16

@@ -1,16 +0,0 @@
1		−/* SPDX-License-Identifier: MIT */
2		−/*
3		− * Copyright © 2023 Intel Corporation
4		− */
5		−
6		−#ifndef _XE_GT_THROTTLE_SYSFS_H_
7		−#define _XE_GT_THROTTLE_SYSFS_H_
8		−
9		−#include <drm/drm_managed.h>
10		−
11		−struct xe_gt;
12		−
13		−int xe_gt_throttle_sysfs_init(struct xe_gt *gt);
14		−
15		−#endif /* _XE_GT_THROTTLE_SYSFS_H_ */
16		−

drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c+47−17

@@ -7,6 +7,7 @@ · 6 unmodified lines
7	7
8	8	#include "abi/guc_actions_abi.h"
9	9	#include "xe_device.h"
	10	+#include "xe_force_wake.h"
10	11	#include "xe_gt.h"
11	12	#include "xe_gt_printk.h"
12	13	#include "xe_guc.h"
@@ -245,7 +246,7 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) · 232 unmodified lines
245	246	return seqno;
246	247
247	248	xe_gt_tlb_invalidation_wait(gt, seqno);
248		− } else if (xe_device_uc_enabled(xe)) {
	249	+ } else if (xe_device_uc_enabled(xe) && !xe_device_wedged(xe)) {
249	250	xe_gt_WARN_ON(gt, xe_force_wake_get(gt_to_fw(gt), XE_FW_GT));
250	251	if (xe->info.platform == XE_PVC \|\| GRAPHICS_VER(xe) >= 20) {
251	252	xe_mmio_write32(gt, PVC_GUC_TLB_INV_DESC1,
@@ -263,11 +264,15 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) · 11 unmodified lines
263	264	}
264	265
265	266	/**
266		− * xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA
	267	+ * xe_gt_tlb_invalidation_range - Issue a TLB invalidation on this GT for an
	268	+ * address range
	269	+ *
267	270	* @gt: graphics tile
268	271	* @fence: invalidation fence which will be signal on TLB invalidation
269	272	* completion, can be NULL
270		− * @vma: VMA to invalidate
	273	+ * @start: start address
	274	+ * @end: end address
	275	+ * @asid: address space id
271	276	*
272	277	* Issue a range based TLB invalidation if supported, if not fallback to a full
273	278	* TLB invalidation. Completion of TLB is asynchronous and caller can either use
@@ -277,17 +282,15 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) · 3 unmodified lines
277	282	* Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success,
278	283	* negative error code on error.
279	284	*/
280		−int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
281		− struct xe_gt_tlb_invalidation_fence *fence,
282		− struct xe_vma *vma)
	285	+int xe_gt_tlb_invalidation_range(struct xe_gt *gt,
	286	+ struct xe_gt_tlb_invalidation_fence *fence,
	287	+ u64 start, u64 end, u32 asid)
283	288	{
284	289	struct xe_device *xe = gt_to_xe(gt);
285	290	#define MAX_TLB_INVALIDATION_LEN 7
286	291	u32 action[MAX_TLB_INVALIDATION_LEN];
287	292	int len = 0;
288	293
289		− xe_gt_assert(gt, vma);
290		−
291	294	/* Execlists not supported */
292	295	if (gt_to_xe(gt)->info.force_execlist) {
293	296	if (fence)
@@ -301,9 +304,9 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, · 7 unmodified lines
301	304	if (!xe->info.has_range_tlb_invalidation) {
302	305	action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_FULL);
303	306	} else {
304		− u64 start = xe_vma_start(vma);
305		− u64 length = xe_vma_size(vma);
306		− u64 align, end;
	307	+ u64 orig_start = start;
	308	+ u64 length = end - start;
	309	+ u64 align;
307	310
308	311	if (length < SZ_4K)
309	312	length = SZ_4K;
@@ -315,12 +318,12 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, · 5 unmodified lines
315	318	* address mask covering the required range.
316	319	*/
317	320	align = roundup_pow_of_two(length);
318		− start = ALIGN_DOWN(xe_vma_start(vma), align);
319		− end = ALIGN(xe_vma_end(vma), align);
	321	+ start = ALIGN_DOWN(start, align);
	322	+ end = ALIGN(end, align);
320	323	length = align;
321	324	while (start + length < end) {
322	325	length <<= 1;
323		− start = ALIGN_DOWN(xe_vma_start(vma), length);
	326	+ start = ALIGN_DOWN(orig_start, length);
324	327	}
325	328
326	329	/*
@@ -329,16 +332,17 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, · 2 unmodified lines
329	332	*/
330	333	if (length >= SZ_2M) {
331	334	length = max_t(u64, SZ_16M, length);
332		− start = ALIGN_DOWN(xe_vma_start(vma), length);
	335	+ start = ALIGN_DOWN(orig_start, length);
333	336	}
334	337
335	338	xe_gt_assert(gt, length >= SZ_4K);
336	339	xe_gt_assert(gt, is_power_of_2(length));
337		− xe_gt_assert(gt, !(length & GENMASK(ilog2(SZ_16M) - 1, ilog2(SZ_2M) + 1)));
	340	+ xe_gt_assert(gt, !(length & GENMASK(ilog2(SZ_16M) - 1,
	341	+ ilog2(SZ_2M) + 1)));
338	342	xe_gt_assert(gt, IS_ALIGNED(start, length));
339	343
340	344	action[len++] = MAKE_INVAL_OP(XE_GUC_TLB_INVAL_PAGE_SELECTIVE);
341		− action[len++] = xe_vma_vm(vma)->usm.asid;
	345	+ action[len++] = asid;
342	346	action[len++] = lower_32_bits(start);
343	347	action[len++] = upper_32_bits(start);
344	348	action[len++] = ilog2(length) - ilog2(SZ_4K);
@@ -349,6 +353,32 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, · 4 unmodified lines
349	353	return send_tlb_invalidation(&gt->uc.guc, fence, action, len);
350	354	}
351	355
	356	+/**
	357	+ * xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA
	358	+ * @gt: graphics tile
	359	+ * @fence: invalidation fence which will be signal on TLB invalidation
	360	+ * completion, can be NULL
	361	+ * @vma: VMA to invalidate
	362	+ *
	363	+ * Issue a range based TLB invalidation if supported, if not fallback to a full
	364	+ * TLB invalidation. Completion of TLB is asynchronous and caller can either use
	365	+ * the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for
	366	+ * completion.
	367	+ *
	368	+ * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success,
	369	+ * negative error code on error.
	370	+ */
	371	+int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
	372	+ struct xe_gt_tlb_invalidation_fence *fence,
	373	+ struct xe_vma *vma)
	374	+{
	375	+ xe_gt_assert(gt, vma);
	376	+
	377	+ return xe_gt_tlb_invalidation_range(gt, fence, xe_vma_start(vma),
	378	+ xe_vma_end(vma),
	379	+ xe_vma_vm(vma)->usm.asid);
	380	+}
	381	+
352	382	/**
353	383	* xe_gt_tlb_invalidation_wait - Wait for TLB to complete
354	384	* @gt: graphics tile

drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h+3−0

@@ -20,6 +20,9 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt); · 19 unmodified lines
20	20	int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
21	21	struct xe_gt_tlb_invalidation_fence *fence,
22	22	struct xe_vma *vma);
	23	+int xe_gt_tlb_invalidation_range(struct xe_gt *gt,
	24	+ struct xe_gt_tlb_invalidation_fence *fence,
	25	+ u64 start, u64 end, u32 asid);
23	26	int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno);
24	27	int xe_guc_tlb_invalidation_done_handler(struct xe_guc guc, u32 msg, u32 len);
25	28

drivers/gpu/drm/xe/xe_gt_topology.c+13−1

@@ -108,7 +108,9 @@ gen_l3_mask_from_pattern(struct xe_device *xe, xe_l3_bank_mask_t dst, · 107 unmodified lines
108	108	{
109	109	unsigned long bit;
110	110
111		− xe_assert(xe, fls(mask) <= patternbits);
	111	+ xe_assert(xe, find_last_bit(pattern, XE_MAX_L3_BANK_MASK_BITS) < patternbits \|\|
	112	+ bitmap_empty(pattern, XE_MAX_L3_BANK_MASK_BITS));
	113	+ xe_assert(xe, !mask \|\| patternbits * (__fls(mask) + 1) <= XE_MAX_L3_BANK_MASK_BITS);
112	114	for_each_set_bit(bit, &mask, 32) {
113	115	xe_l3_bank_mask_t shifted_pattern = {};
114	116
@@ -278,3 +280,13 @@ bool xe_gt_topology_has_dss_in_quadrant(struct xe_gt *gt, int quad) · 163 unmodified lines
278	280
279	281	return quad_first < (quad + 1) * dss_per_quad;
280	282	}
	283	+
	284	+bool xe_gt_has_geometry_dss(struct xe_gt *gt, unsigned int dss)
	285	+{
	286	+ return test_bit(dss, gt->fuse_topo.g_dss_mask);
	287	+}
	288	+
	289	+bool xe_gt_has_compute_dss(struct xe_gt *gt, unsigned int dss)
	290	+{
	291	+ return test_bit(dss, gt->fuse_topo.c_dss_mask);
	292	+}

drivers/gpu/drm/xe/xe_gt_topology.h+3−0

@@ -33,4 +33,7 @@ bool xe_dss_mask_empty(const xe_dss_mask_t mask); · 32 unmodified lines
33	33	bool
34	34	xe_gt_topology_has_dss_in_quadrant(struct xe_gt *gt, int quad);
35	35
	36	+bool xe_gt_has_geometry_dss(struct xe_gt *gt, unsigned int dss);
	37	+bool xe_gt_has_compute_dss(struct xe_gt *gt, unsigned int dss);
	38	+
36	39	#endif /* _XE_GT_TOPOLOGY_H_ */

drivers/gpu/drm/xe/xe_gt_types.h+26−8

@@ -9,6 +9,7 @@ · 8 unmodified lines
9	9	#include "xe_force_wake_types.h"
10	10	#include "xe_gt_idle_types.h"
11	11	#include "xe_gt_sriov_pf_types.h"
	12	+#include "xe_gt_sriov_vf_types.h"
12	13	#include "xe_hw_engine_types.h"
13	14	#include "xe_hw_fence_types.h"
14	15	#include "xe_reg_sr_types.h"
@@ -110,20 +111,20 @@ struct xe_gt { · 95 unmodified lines
110	111	struct {
111	112	/** @info.type: type of GT */
112	113	enum xe_gt_type type;
113		− /** @info.id: Unique ID of this GT within the PCI Device */
114		− u8 id;
115	114	/** @info.reference_clock: clock frequency */
116	115	u32 reference_clock;
117		− /** @info.engine_mask: mask of engines present on GT */
118		− u64 engine_mask;
119	116	/**
120		− * @info.__engine_mask: mask of engines present on GT read from
121		− * xe_pci.c, used to fake reading the engine_mask from the
122		− * hwconfig blob.
	117	+ * @info.engine_mask: mask of engines present on GT. Some of
	118	+ * them may be reserved in runtime and not available for user.
	119	+ * See @user_engines.mask
123	120	*/
124		− u64 __engine_mask;
	121	+ u64 engine_mask;
125	122	/** @info.gmdid: raw GMD_ID value from hardware */
126	123	u32 gmdid;
	124	+ /** @info.id: Unique ID of this GT within the PCI Device */
	125	+ u8 id;
	126	+ /** @info.has_indirect_ring_state: GT has indirect ring state support */
	127	+ u8 has_indirect_ring_state:1;
127	128	} info;
128	129
129	130	/**
@@ -147,6 +148,8 @@ struct xe_gt { · 17 unmodified lines
147	148	union {
148	149	/** @sriov.pf: PF data. Valid only if driver is running as PF */
149	150	struct xe_gt_sriov_pf pf;
	151	+ /** @sriov.vf: VF data. Valid only if driver is running as VF */
	152	+ struct xe_gt_sriov_vf vf;
150	153	} sriov;
151	154
152	155	/**
@@ -369,6 +372,21 @@ struct xe_gt { · 216 unmodified lines
369	372	/** @wa_active.oob: bitmap with active OOB workaroudns */
370	373	unsigned long *oob;
371	374	} wa_active;
	375	+
	376	+ /** @user_engines: engines present in GT and available to userspace */
	377	+ struct {
	378	+ /**
	379	+ * @user_engines.mask: like @info->engine_mask, but take in
	380	+ * consideration only engines available to userspace
	381	+ */
	382	+ u64 mask;
	383	+
	384	+ /**
	385	+ * @user_engines.instances_per_class: aggregate per class the
	386	+ * number of engines available to userspace
	387	+ */
	388	+ u8 instances_per_class[XE_ENGINE_CLASS_MAX];
	389	+ } user_engines;
372	390	};
373	391
374	392	#endif

drivers/gpu/drm/xe/xe_guc.c+272−82

@@ -19,8 +19,11 @@ · 18 unmodified lines
19	19	#include "xe_force_wake.h"
20	20	#include "xe_gt.h"
21	21	#include "xe_gt_printk.h"
	22	+#include "xe_gt_sriov_vf.h"
	23	+#include "xe_gt_throttle.h"
22	24	#include "xe_guc_ads.h"
23	25	#include "xe_guc_ct.h"
	26	+#include "xe_guc_db_mgr.h"
24	27	#include "xe_guc_hwconfig.h"
25	28	#include "xe_guc_log.h"
26	29	#include "xe_guc_pc.h"
@@ -239,7 +242,7 @@ static void guc_write_params(struct xe_guc *guc) · 212 unmodified lines
239	242	xe_mmio_write32(gt, SOFT_SCRATCH(1 + i), guc->params[i]);
240	243	}
241	244
242		−static void guc_fini(struct drm_device drm, void arg)
	245	+static void guc_fini_hw(void *arg)
243	246	{
244	247	struct xe_guc *guc = arg;
245	248	struct xe_gt *gt = guc_to_gt(guc);
@@ -293,6 +296,23 @@ static int xe_guc_realloc_post_hwconfig(struct xe_guc *guc) · 47 unmodified lines
293	296	return 0;
294	297	}
295	298
	299	+static int vf_guc_init(struct xe_guc *guc)
	300	+{
	301	+ int err;
	302	+
	303	+ xe_guc_comm_init_early(guc);
	304	+
	305	+ err = xe_guc_ct_init(&guc->ct);
	306	+ if (err)
	307	+ return err;
	308	+
	309	+ err = xe_guc_relay_init(&guc->relay);
	310	+ if (err)
	311	+ return err;
	312	+
	313	+ return 0;
	314	+}
	315	+
296	316	int xe_guc_init(struct xe_guc *guc)
297	317	{
298	318	struct xe_device *xe = guc_to_xe(guc);
@@ -307,6 +327,13 @@ int xe_guc_init(struct xe_guc *guc) · 8 unmodified lines
307	327	if (!xe_uc_fw_is_enabled(&guc->fw))
308	328	return 0;
309	329
	330	+ if (IS_SRIOV_VF(xe)) {
	331	+ ret = vf_guc_init(guc);
	332	+ if (ret)
	333	+ goto out;
	334	+ return 0;
	335	+ }
	336	+
310	337	ret = xe_guc_log_init(&guc->log);
311	338	if (ret)
312	339	goto out;
@@ -323,7 +350,7 @@ int xe_guc_init(struct xe_guc *guc) · 10 unmodified lines
323	350	if (ret)
324	351	goto out;
325	352
326		− ret = drmm_add_action_or_reset(&xe->drm, guc_fini, guc);
	353	+ ret = devm_add_action_or_reset(xe->drm.dev, guc_fini_hw, guc);
327	354	if (ret)
328	355	goto out;
329	356
@@ -340,6 +367,19 @@ out: · 10 unmodified lines
340	367	return ret;
341	368	}
342	369
	370	+static int vf_guc_init_post_hwconfig(struct xe_guc *guc)
	371	+{
	372	+ int err;
	373	+
	374	+ err = xe_guc_submit_init(guc, xe_gt_sriov_vf_guc_ids(guc_to_gt(guc)));
	375	+ if (err)
	376	+ return err;
	377	+
	378	+ /* XXX xe_guc_db_mgr_init not needed for now */
	379	+
	380	+ return 0;
	381	+}
	382	+
343	383	/**
344	384	* xe_guc_init_post_hwconfig - initialize GuC post hwconfig load
345	385	* @guc: The GuC object
@@ -350,12 +390,23 @@ int xe_guc_init_post_hwconfig(struct xe_guc *guc) · 4 unmodified lines
350	390	{
351	391	int ret;
352	392
	393	+ if (IS_SRIOV_VF(guc_to_xe(guc)))
	394	+ return vf_guc_init_post_hwconfig(guc);
	395	+
353	396	ret = xe_guc_realloc_post_hwconfig(guc);
354	397	if (ret)
355	398	return ret;
356	399
357	400	guc_init_params_post_hwconfig(guc);
358	401
	402	+ ret = xe_guc_submit_init(guc, ~0);
	403	+ if (ret)
	404	+ return ret;
	405	+
	406	+ ret = xe_guc_db_mgr_init(&guc->dbm, ~0);
	407	+ if (ret)
	408	+ return ret;
	409	+
359	410	ret = xe_guc_pc_init(&guc->pc);
360	411	if (ret)
361	412	return ret;
@@ -379,6 +430,9 @@ int xe_guc_reset(struct xe_guc *guc) · 17 unmodified lines
379	430
380	431	xe_force_wake_assert_held(gt_to_fw(gt), XE_FW_GT);
381	432
	433	+ if (IS_SRIOV_VF(gt_to_xe(gt)))
	434	+ return xe_gt_sriov_vf_bootstrap(gt);
	435	+
382	436	xe_mmio_write32(gt, GDRST, GRDOM_GUC);
383	437
384	438	ret = xe_mmio_wait32(gt, GDRST, GRDOM_GUC, 0, 5000, &gdrst, false);
@@ -451,58 +505,194 @@ static int guc_xfer_rsa(struct xe_guc *guc) · 66 unmodified lines
451	505	return 0;
452	506	}
453	507
454		−static int guc_wait_ucode(struct xe_guc *guc)
	508	+/*
	509	+ * Check a previously read GuC status register (GUC_STATUS) looking for
	510	+ * known terminal states (either completion or failure) of either the
	511	+ * microkernel status field or the boot ROM status field. Returns +1 for
	512	+ * successful completion, -1 for failure and 0 for any intermediate state.
	513	+ */
	514	+static int guc_load_done(u32 status)
455	515	{
456		− struct xe_gt *gt = guc_to_gt(guc);
457		− u32 status;
458		− int ret;
	516	+ u32 uk_val = REG_FIELD_GET(GS_UKERNEL_MASK, status);
	517	+ u32 br_val = REG_FIELD_GET(GS_BOOTROM_MASK, status);
	518	+
	519	+ switch (uk_val) {
	520	+ case XE_GUC_LOAD_STATUS_READY:
	521	+ return 1;
	522	+
	523	+ case XE_GUC_LOAD_STATUS_ERROR_DEVID_BUILD_MISMATCH:
	524	+ case XE_GUC_LOAD_STATUS_GUC_PREPROD_BUILD_MISMATCH:
	525	+ case XE_GUC_LOAD_STATUS_ERROR_DEVID_INVALID_GUCTYPE:
	526	+ case XE_GUC_LOAD_STATUS_HWCONFIG_ERROR:
	527	+ case XE_GUC_LOAD_STATUS_DPC_ERROR:
	528	+ case XE_GUC_LOAD_STATUS_EXCEPTION:
	529	+ case XE_GUC_LOAD_STATUS_INIT_DATA_INVALID:
	530	+ case XE_GUC_LOAD_STATUS_MPU_DATA_INVALID:
	531	+ case XE_GUC_LOAD_STATUS_INIT_MMIO_SAVE_RESTORE_INVALID:
	532	+ return -1;
	533	+ }
	534	+
	535	+ switch (br_val) {
	536	+ case XE_BOOTROM_STATUS_NO_KEY_FOUND:
	537	+ case XE_BOOTROM_STATUS_RSA_FAILED:
	538	+ case XE_BOOTROM_STATUS_PAVPC_FAILED:
	539	+ case XE_BOOTROM_STATUS_WOPCM_FAILED:
	540	+ case XE_BOOTROM_STATUS_LOADLOC_FAILED:
	541	+ case XE_BOOTROM_STATUS_JUMP_FAILED:
	542	+ case XE_BOOTROM_STATUS_RC6CTXCONFIG_FAILED:
	543	+ case XE_BOOTROM_STATUS_MPUMAP_INCORRECT:
	544	+ case XE_BOOTROM_STATUS_EXCEPTION:
	545	+ case XE_BOOTROM_STATUS_PROD_KEY_CHECK_FAILURE:
	546	+ return -1;
	547	+ }
	548	+
	549	+ return 0;
	550	+}
	551	+
	552	+static s32 guc_pc_get_cur_freq(struct xe_guc_pc *guc_pc)
	553	+{
	554	+ u32 freq;
	555	+ int ret = xe_guc_pc_get_cur_freq(guc_pc, &freq);
	556	+
	557	+ return ret ? ret : freq;
	558	+}
459	559
	560	+/*
	561	+ * Wait for the GuC to start up.
	562	+ *
	563	+ * Measurements indicate this should take no more than 20ms (assuming the GT
	564	+ * clock is at maximum frequency). However, thermal throttling and other issues
	565	+ * can prevent the clock hitting max and thus making the load take significantly
	566	+ * longer. Allow up to 200ms as a safety margin for real world worst case situations.
	567	+ *
	568	+ * However, bugs anywhere from KMD to GuC to PCODE to fan failure in a CI farm can
	569	+ * lead to even longer times. E.g. if the GT is clamped to minimum frequency then
	570	+ * the load times can be in the seconds range. So the timeout is increased for debug
	571	+ * builds to ensure that problems can be correctly analysed. For release builds, the
	572	+ * timeout is kept short so that users don't wait forever to find out that there is a
	573	+ * problem. In either case, if the load took longer than is reasonable even with some
	574	+ * 'sensible' throttling, then flag a warning because something is not right.
	575	+ *
	576	+ * Note that there is a limit on how long an individual usleep_range() can wait for,
	577	+ * hence longer waits require wrapping a shorter wait in a loop.
	578	+ *
	579	+ * Note that the only reason an end user should hit the shorter timeout is in case of
	580	+ * extreme thermal throttling. And a system that is that hot during boot is probably
	581	+ * dead anyway!
	582	+ */
	583	+#if defined(CONFIG_DRM_XE_DEBUG)
	584	+#define GUC_LOAD_RETRY_LIMIT 20
	585	+#else
	586	+#define GUC_LOAD_RETRY_LIMIT 3
	587	+#endif
	588	+#define GUC_LOAD_TIME_WARN_MS 200
	589	+
	590	+static void guc_wait_ucode(struct xe_guc *guc)
	591	+{
	592	+ struct xe_gt *gt = guc_to_gt(guc);
	593	+ struct xe_guc_pc *guc_pc = &gt->uc.guc.pc;
	594	+ ktime_t before, after, delta;
	595	+ int load_done;
	596	+ u32 status = 0;
	597	+ int count = 0;
	598	+ u64 delta_ms;
	599	+ u32 before_freq;
	600	+
	601	+ before_freq = xe_guc_pc_get_act_freq(guc_pc);
	602	+ before = ktime_get();
460	603	/*
461		− * Wait for the GuC to start up.
462		− * NB: Docs recommend not using the interrupt for completion.
463		− * Measurements indicate this should take no more than 20ms
464		− * (assuming the GT clock is at maximum frequency). So, a
465		− * timeout here indicates that the GuC has failed and is unusable.
466		− * (Higher levels of the driver may decide to reset the GuC and
467		− * attempt the ucode load again if this happens.)
468		− *
469		− * FIXME: There is a known (but exceedingly unlikely) race condition
470		− * where the asynchronous frequency management code could reduce
471		− * the GT clock while a GuC reload is in progress (during a full
472		− * GT reset). A fix is in progress but there are complex locking
473		− * issues to be resolved. In the meantime bump the timeout to
474		− * 200ms. Even at slowest clock, this should be sufficient. And
475		− * in the working case, a larger timeout makes no difference.
	604	+ * Note, can't use any kind of timing information from the call to xe_mmio_wait.
	605	+ * It could return a thousand intermediate stages at random times. Instead, must
	606	+ * manually track the total time taken and locally implement the timeout.
476	607	*/
477		− ret = xe_mmio_wait32(gt, GUC_STATUS, GS_UKERNEL_MASK,
478		− FIELD_PREP(GS_UKERNEL_MASK, XE_GUC_LOAD_STATUS_READY),
479		− 200000, &status, false);
	608	+ do {
	609	+ u32 last_status = status & (GS_UKERNEL_MASK \| GS_BOOTROM_MASK);
	610	+ int ret;
480	611
481		− if (ret) {
482		− xe_gt_info(gt, "GuC load failed: status = 0x%08X\n", status);
483		− xe_gt_info(gt, "GuC status: Reset = %u, BootROM = %#X, UKernel = %#X, MIA = %#X, Auth = %#X\n",
484		− REG_FIELD_GET(GS_MIA_IN_RESET, status),
485		− REG_FIELD_GET(GS_BOOTROM_MASK, status),
486		− REG_FIELD_GET(GS_UKERNEL_MASK, status),
487		− REG_FIELD_GET(GS_MIA_MASK, status),
488		− REG_FIELD_GET(GS_AUTH_STATUS_MASK, status));
489		−
490		− if ((status & GS_BOOTROM_MASK) == GS_BOOTROM_RSA_FAILED) {
491		− xe_gt_info(gt, "GuC firmware signature verification failed\n");
492		− ret = -ENOEXEC;
	612	+ /*
	613	+ * Wait for any change (intermediate or terminal) in the status register.
	614	+ * Note, the return value is a don't care. The only failure code is timeout
	615	+ * but the timeouts need to be accumulated over all the intermediate partial
	616	+ * timeouts rather than allowing a huge timeout each time. So basically, need
	617	+ * to treat a timeout no different to a value change.
	618	+ */
	619	+ ret = xe_mmio_wait32_not(gt, GUC_STATUS, GS_UKERNEL_MASK \| GS_BOOTROM_MASK,
	620	+ last_status, 1000 * 1000, &status, false);
	621	+ if (ret < 0)
	622	+ count++;
	623	+ after = ktime_get();
	624	+ delta = ktime_sub(after, before);
	625	+ delta_ms = ktime_to_ms(delta);
	626	+
	627	+ load_done = guc_load_done(status);
	628	+ if (load_done != 0)
	629	+ break;
	630	+
	631	+ if (delta_ms >= (GUC_LOAD_RETRY_LIMIT * 1000))
	632	+ break;
	633	+
	634	+ xe_gt_dbg(gt, "load still in progress, timeouts = %d, freq = %dMHz (req %dMHz), status = 0x%08X [0x%02X/%02X]\n",
	635	+ count, xe_guc_pc_get_act_freq(guc_pc),
	636	+ guc_pc_get_cur_freq(guc_pc), status,
	637	+ REG_FIELD_GET(GS_BOOTROM_MASK, status),
	638	+ REG_FIELD_GET(GS_UKERNEL_MASK, status));
	639	+ } while (1);
	640	+
	641	+ if (load_done != 1) {
	642	+ u32 ukernel = REG_FIELD_GET(GS_UKERNEL_MASK, status);
	643	+ u32 bootrom = REG_FIELD_GET(GS_BOOTROM_MASK, status);
	644	+
	645	+ xe_gt_err(gt, "load failed: status = 0x%08X, time = %lldms, freq = %dMHz (req %dMHz), done = %d\n",
	646	+ status, delta_ms, xe_guc_pc_get_act_freq(guc_pc),
	647	+ guc_pc_get_cur_freq(guc_pc), load_done);
	648	+ xe_gt_err(gt, "load failed: status: Reset = %d, BootROM = 0x%02X, UKernel = 0x%02X, MIA = 0x%02X, Auth = 0x%02X\n",
	649	+ REG_FIELD_GET(GS_MIA_IN_RESET, status),
	650	+ bootrom, ukernel,
	651	+ REG_FIELD_GET(GS_MIA_MASK, status),
	652	+ REG_FIELD_GET(GS_AUTH_STATUS_MASK, status));
	653	+
	654	+ switch (bootrom) {
	655	+ case XE_BOOTROM_STATUS_NO_KEY_FOUND:
	656	+ xe_gt_err(gt, "invalid key requested, header = 0x%08X\n",
	657	+ xe_mmio_read32(gt, GUC_HEADER_INFO));
	658	+ break;
	659	+
	660	+ case XE_BOOTROM_STATUS_RSA_FAILED:
	661	+ xe_gt_err(gt, "firmware signature verification failed\n");
	662	+ break;
	663	+
	664	+ case XE_BOOTROM_STATUS_PROD_KEY_CHECK_FAILURE:
	665	+ xe_gt_err(gt, "firmware production part check failure\n");
	666	+ break;
493	667	}
494	668
495		− if (REG_FIELD_GET(GS_UKERNEL_MASK, status) ==
496		− XE_GUC_LOAD_STATUS_EXCEPTION) {
497		− xe_gt_info(gt, "GuC firmware exception. EIP: %#x\n",
498		− xe_mmio_read32(gt, SOFT_SCRATCH(13)));
499		− ret = -ENXIO;
	669	+ switch (ukernel) {
	670	+ case XE_GUC_LOAD_STATUS_EXCEPTION:
	671	+ xe_gt_err(gt, "firmware exception. EIP: %#x\n",
	672	+ xe_mmio_read32(gt, SOFT_SCRATCH(13)));
	673	+ break;
	674	+
	675	+ case XE_GUC_LOAD_STATUS_INIT_MMIO_SAVE_RESTORE_INVALID:
	676	+ xe_gt_err(gt, "illegal register in save/restore workaround list\n");
	677	+ break;
	678	+
	679	+ case XE_GUC_LOAD_STATUS_HWCONFIG_START:
	680	+ xe_gt_err(gt, "still extracting hwconfig table.\n");
	681	+ break;
500	682	}
	683	+
	684	+ xe_device_declare_wedged(gt_to_xe(gt));
	685	+ } else if (delta_ms > GUC_LOAD_TIME_WARN_MS) {
	686	+ xe_gt_warn(gt, "excessive init time: %lldms! [status = 0x%08X, timeouts = %d]\n",
	687	+ delta_ms, status, count);
	688	+ xe_gt_warn(gt, "excessive init time: [freq = %dMHz (req = %dMHz), before = %dMHz, perf_limit_reasons = 0x%08X]\n",
	689	+ xe_guc_pc_get_act_freq(guc_pc), guc_pc_get_cur_freq(guc_pc),
	690	+ before_freq, xe_gt_throttle_get_limit_reasons(gt));
501	691	} else {
502		− xe_gt_dbg(gt, "GuC successfully loaded\n");
	692	+ xe_gt_dbg(gt, "init took %lldms, freq = %dMHz (req = %dMHz), before = %dMHz, status = 0x%08X, timeouts = %d\n",
	693	+ delta_ms, xe_guc_pc_get_act_freq(guc_pc), guc_pc_get_cur_freq(guc_pc),
	694	+ before_freq, status, count);
503	695	}
504		−
505		− return ret;
506	696	}
507	697
508	698	static int __xe_guc_upload(struct xe_guc *guc)
@@ -532,9 +722,7 @@ static int __xe_guc_upload(struct xe_guc *guc) · 23 unmodified lines
532	722	goto out;
533	723
534	724	/* Wait for authentication */
535		− ret = guc_wait_ucode(guc);
536		− if (ret)
537		− goto out;
	725	+ guc_wait_ucode(guc);
538	726
539	727	xe_uc_fw_change_status(&guc->fw, XE_UC_FIRMWARE_RUNNING);
540	728	return 0;
@@ -544,6 +732,38 @@ out: · 3 unmodified lines
544	732	return 0 /* FIXME: ret, don't want to stop load currently */;
545	733	}
546	734
	735	+static int vf_guc_min_load_for_hwconfig(struct xe_guc *guc)
	736	+{
	737	+ struct xe_gt *gt = guc_to_gt(guc);
	738	+ int ret;
	739	+
	740	+ ret = xe_gt_sriov_vf_bootstrap(gt);
	741	+ if (ret)
	742	+ return ret;
	743	+
	744	+ ret = xe_gt_sriov_vf_query_config(gt);
	745	+ if (ret)
	746	+ return ret;
	747	+
	748	+ ret = xe_guc_hwconfig_init(guc);
	749	+ if (ret)
	750	+ return ret;
	751	+
	752	+ ret = xe_guc_enable_communication(guc);
	753	+ if (ret)
	754	+ return ret;
	755	+
	756	+ ret = xe_gt_sriov_vf_connect(gt);
	757	+ if (ret)
	758	+ return ret;
	759	+
	760	+ ret = xe_gt_sriov_vf_query_runtime(gt);
	761	+ if (ret)
	762	+ return ret;
	763	+
	764	+ return 0;
	765	+}
	766	+
547	767	/**
548	768	* xe_guc_min_load_for_hwconfig - load minimal GuC and read hwconfig table
549	769	* @guc: The GuC object
@@ -559,6 +779,9 @@ int xe_guc_min_load_for_hwconfig(struct xe_guc *guc) · 9 unmodified lines
559	779	{
560	780	int ret;
561	781
	782	+ if (IS_SRIOV_VF(guc_to_xe(guc)))
	783	+ return vf_guc_min_load_for_hwconfig(guc);
	784	+
562	785	xe_guc_ads_populate_minimal(&guc->ads);
563	786
564	787	/* Raise GT freq to speed up HuC/GuC load */
@@ -891,17 +1114,11 @@ void xe_guc_stop_prepare(struct xe_guc *guc) · 326 unmodified lines
891	1114	XE_WARN_ON(xe_guc_pc_stop(&guc->pc));
892	1115	}
893	1116
894		−int xe_guc_stop(struct xe_guc *guc)
	1117	+void xe_guc_stop(struct xe_guc *guc)
895	1118	{
896		− int ret;
897		−
898	1119	xe_guc_ct_stop(&guc->ct);
899	1120
900		− ret = xe_guc_submit_stop(guc);
901		− if (ret)
902		− return ret;
903		−
904		− return 0;
	1121	+ xe_guc_submit_stop(guc);
905	1122	}
906	1123
907	1124	int xe_guc_start(struct xe_guc *guc)
@@ -950,30 +1167,3 @@ void xe_guc_print_info(struct xe_guc guc, struct drm_printer p) · 42 unmodified lines
950	1167	xe_guc_ct_print(&guc->ct, p, false);
951	1168	xe_guc_submit_print(guc, p);
952	1169	}
953		−
954		−/**
955		− * xe_guc_in_reset() - Detect if GuC MIA is in reset.
956		− * @guc: The GuC object
957		− *
958		− * This function detects runtime resume from d3cold by leveraging
959		− * GUC_STATUS, GUC doesn't get reset during d3hot,
960		− * it strictly to be called from RPM resume handler.
961		− *
962		− * Return: true if failed to get forcewake or GuC MIA is in Reset,
963		− * otherwise false.
964		− */
965		−bool xe_guc_in_reset(struct xe_guc *guc)
966		−{
967		− struct xe_gt *gt = guc_to_gt(guc);
968		− u32 status;
969		− int err;
970		−
971		− err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
972		− if (err)
973		− return true;
974		−
975		− status = xe_mmio_read32(gt, GUC_STATUS);
976		− xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
977		−
978		− return status & GS_MIA_IN_RESET;
979		−}

drivers/gpu/drm/xe/xe_guc.h+1−2

@@ -35,9 +35,8 @@ void xe_guc_print_info(struct xe_guc guc, struct drm_printer p); · 34 unmodified lines
35	35	int xe_guc_reset_prepare(struct xe_guc *guc);
36	36	void xe_guc_reset_wait(struct xe_guc *guc);
37	37	void xe_guc_stop_prepare(struct xe_guc *guc);
38		−int xe_guc_stop(struct xe_guc *guc);
	38	+void xe_guc_stop(struct xe_guc *guc);
39	39	int xe_guc_start(struct xe_guc *guc);
40		−bool xe_guc_in_reset(struct xe_guc *guc);
41	40
42	41	static inline u16 xe_engine_class_to_guc_class(enum xe_engine_class class)
43	42	{

drivers/gpu/drm/xe/xe_guc_ads.c+67−4

@@ -9,13 +9,16 @@ · 8 unmodified lines
9	9
10	10	#include <generated/xe_wa_oob.h>
11	11
	12	+#include "abi/guc_actions_abi.h"
12	13	#include "regs/xe_engine_regs.h"
13	14	#include "regs/xe_gt_regs.h"
14	15	#include "regs/xe_guc_regs.h"
15	16	#include "xe_bo.h"
16	17	#include "xe_gt.h"
17	18	#include "xe_gt_ccs_mode.h"
	19	+#include "xe_gt_printk.h"
18	20	#include "xe_guc.h"
	21	+#include "xe_guc_ct.h"
19	22	#include "xe_hw_engine.h"
20	23	#include "xe_lrc.h"
21	24	#include "xe_map.h"
@@ -265,7 +268,6 @@ static u32 engine_enable_mask(struct xe_gt *gt, enum xe_engine_class class) · 243 unmodified lines
265	268
266	269	static size_t calculate_golden_lrc_size(struct xe_guc_ads *ads)
267	270	{
268		− struct xe_device *xe = ads_to_xe(ads);
269	271	struct xe_gt *gt = ads_to_gt(ads);
270	272	size_t total_size = 0, alloc_size, real_size;
271	273	int class;
@@ -274,7 +276,7 @@ static size_t calculate_golden_lrc_size(struct xe_guc_ads *ads) · 2 unmodified lines
274	276	if (!engine_enable_mask(gt, class))
275	277	continue;
276	278
277		− real_size = xe_lrc_size(xe, class);
	279	+ real_size = xe_gt_lrc_size(gt, class);
278	280	alloc_size = PAGE_ALIGN(real_size);
279	281	total_size += alloc_size;
280	282	}
@@ -440,11 +442,18 @@ int xe_guc_ads_init_post_hwconfig(struct xe_guc_ads *ads) · 159 unmodified lines
440	442
441	443	static void guc_policies_init(struct xe_guc_ads *ads)
442	444	{
	445	+ struct xe_device *xe = ads_to_xe(ads);
	446	+ u32 global_flags = 0;
	447	+
443	448	ads_blob_write(ads, policies.dpc_promote_time,
444	449	GLOBAL_POLICY_DEFAULT_DPC_PROMOTE_TIME_US);
445	450	ads_blob_write(ads, policies.max_num_work_items,
446	451	GLOBAL_POLICY_MAX_NUM_WI);
447		− ads_blob_write(ads, policies.global_flags, 0);
	452	+
	453	+ if (xe->wedged.mode == 2)
	454	+ global_flags \|= GLOBAL_POLICY_DISABLE_ENGINE_RESET;
	455	+
	456	+ ads_blob_write(ads, policies.global_flags, global_flags);
448	457	ads_blob_write(ads, policies.is_valid, 1);
449	458	}
450	459
@@ -765,7 +774,7 @@ static void guc_populate_golden_lrc(struct xe_guc_ads *ads) · 314 unmodified lines
765	774
766	775	xe_gt_assert(gt, gt->default_lrc[class]);
767	776
768		− real_size = xe_lrc_size(xe, class);
	777	+ real_size = xe_gt_lrc_size(gt, class);
769	778	alloc_size = PAGE_ALIGN(real_size);
770	779	total_size += alloc_size;
771	780
@@ -799,3 +808,57 @@ void xe_guc_ads_populate_post_load(struct xe_guc_ads *ads) · 27 unmodified lines
799	808	{
800	809	guc_populate_golden_lrc(ads);
801	810	}
	811	+
	812	+static int guc_ads_action_update_policies(struct xe_guc_ads *ads, u32 policy_offset)
	813	+{
	814	+ struct xe_guc_ct *ct = &ads_to_guc(ads)->ct;
	815	+ u32 action[] = {
	816	+ XE_GUC_ACTION_GLOBAL_SCHED_POLICY_CHANGE,
	817	+ policy_offset
	818	+ };
	819	+
	820	+ return xe_guc_ct_send(ct, action, ARRAY_SIZE(action), 0, 0);
	821	+}
	822	+
	823	+/**
	824	+ * xe_guc_ads_scheduler_policy_toggle_reset - Toggle reset policy
	825	+ * @ads: Additional data structures object
	826	+ *
	827	+ * This function update the GuC's engine reset policy based on wedged.mode.
	828	+ *
	829	+ * Return: 0 on success, and negative error code otherwise.
	830	+ */
	831	+int xe_guc_ads_scheduler_policy_toggle_reset(struct xe_guc_ads *ads)
	832	+{
	833	+ struct xe_device *xe = ads_to_xe(ads);
	834	+ struct xe_gt *gt = ads_to_gt(ads);
	835	+ struct xe_tile *tile = gt_to_tile(gt);
	836	+ struct guc_policies *policies;
	837	+ struct xe_bo *bo;
	838	+ int ret = 0;
	839	+
	840	+ policies = kmalloc(sizeof(*policies), GFP_KERNEL);
	841	+ if (!policies)
	842	+ return -ENOMEM;
	843	+
	844	+ policies->dpc_promote_time = ads_blob_read(ads, policies.dpc_promote_time);
	845	+ policies->max_num_work_items = ads_blob_read(ads, policies.max_num_work_items);
	846	+ policies->is_valid = 1;
	847	+ if (xe->wedged.mode == 2)
	848	+ policies->global_flags \|= GLOBAL_POLICY_DISABLE_ENGINE_RESET;
	849	+ else
	850	+ policies->global_flags &= ~GLOBAL_POLICY_DISABLE_ENGINE_RESET;
	851	+
	852	+ bo = xe_managed_bo_create_from_data(xe, tile, policies, sizeof(struct guc_policies),
	853	+ XE_BO_FLAG_VRAM_IF_DGFX(tile) \|
	854	+ XE_BO_FLAG_GGTT);
	855	+ if (IS_ERR(bo)) {
	856	+ ret = PTR_ERR(bo);
	857	+ goto out;
	858	+ }
	859	+
	860	+ ret = guc_ads_action_update_policies(ads, xe_bo_ggtt_addr(bo));
	861	+out:
	862	+ kfree(policies);
	863	+ return ret;
	864	+}

drivers/gpu/drm/xe/xe_guc_ads.h+2−1

@@ -6,12 +6,13 @@ · 5 unmodified lines
6	6	#ifndef _XE_GUC_ADS_H_
7	7	#define _XE_GUC_ADS_H_
8	8
9		−#include "xe_guc_ads_types.h"
	9	+struct xe_guc_ads;
10	10
11	11	int xe_guc_ads_init(struct xe_guc_ads *ads);
12	12	int xe_guc_ads_init_post_hwconfig(struct xe_guc_ads *ads);
13	13	void xe_guc_ads_populate(struct xe_guc_ads *ads);
14	14	void xe_guc_ads_populate_minimal(struct xe_guc_ads *ads);
15	15	void xe_guc_ads_populate_post_load(struct xe_guc_ads *ads);
	16	+int xe_guc_ads_scheduler_policy_toggle_reset(struct xe_guc_ads *ads);
16	17
17	18	#endif

drivers/gpu/drm/xe/xe_guc_ct.c+4−0

@@ -22,6 +22,7 @@ · 21 unmodified lines
22	22	#include "xe_gt_pagefault.h"
23	23	#include "xe_gt_printk.h"
24	24	#include "xe_gt_sriov_pf_control.h"
	25	+#include "xe_gt_sriov_pf_monitor.h"
25	26	#include "xe_gt_tlb_invalidation.h"
26	27	#include "xe_guc.h"
27	28	#include "xe_guc_relay.h"
@@ -1071,6 +1072,9 @@ static int process_g2h_msg(struct xe_guc_ct ct, u32 msg, u32 len) · 1043 unmodified lines
1071	1072	case GUC_ACTION_GUC2PF_VF_STATE_NOTIFY:
1072	1073	ret = xe_gt_sriov_pf_control_process_guc2pf(gt, hxg, hxg_len);
1073	1074	break;
	1075	+ case GUC_ACTION_GUC2PF_ADVERSE_EVENT:
	1076	+ ret = xe_gt_sriov_pf_monitor_process_guc2pf(gt, hxg, hxg_len);
	1077	+ break;
1074	1078	default:
1075	1079	xe_gt_err(gt, "unexpected G2H action 0x%04x\n", action);
1076	1080	}

drivers/gpu/drm/xe/xe_guc_db_mgr.c+2−1

@@ -106,7 +106,8 @@ int xe_guc_db_mgr_init(struct xe_guc_db_mgr *dbm, unsigned int count) · 105 unmodified lines
106	106	if (ret)
107	107	return ret;
108	108	done:
109		− xe_gt_dbg(dbm_to_gt(dbm), "using %u doorbell(s)\n", dbm->count);
	109	+ xe_gt_dbg(dbm_to_gt(dbm), "using %u doorbell%s\n",
	110	+ dbm->count, str_plural(dbm->count));
110	111	return 0;
111	112	}
112	113

drivers/gpu/drm/xe/xe_guc_id_mgr.c+2−1

@@ -97,7 +97,8 @@ int xe_guc_id_mgr_init(struct xe_guc_id_mgr *idm, unsigned int limit) · 96 unmodified lines
97	97	if (ret)
98	98	return ret;
99	99
100		− xe_gt_info(idm_to_gt(idm), "using %u GUC ID(s)\n", idm->total);
	100	+ xe_gt_info(idm_to_gt(idm), "using %u GUC ID%s\n",
	101	+ idm->total, str_plural(idm->total));
101	102	return 0;
102	103	}
103	104

drivers/gpu/drm/xe/xe_guc_klv_helpers.c+12−0

@@ -8,6 +8,7 @@ · 7 unmodified lines
8	8
9	9	#include "abi/guc_klvs_abi.h"
10	10	#include "xe_guc_klv_helpers.h"
	11	+#include "xe_guc_klv_thresholds_set.h"
11	12
12	13	#define make_u64(hi, lo) ((u64)((u64)(u32)(hi) << 32 \| (u32)(lo)))
13	14
@@ -48,6 +49,17 @@ const char *xe_guc_klv_key_to_string(u16 key) · 34 unmodified lines
48	49	return "begin_db_id";
49	50	case GUC_KLV_VF_CFG_BEGIN_CONTEXT_ID_KEY:
50	51	return "begin_ctx_id";
	52	+
	53	+ /* VF CFG threshold keys */
	54	+#define define_threshold_key_to_string_case(TAG, NAME, ...) \
	55	+ \
	56	+ case MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY(TAG): \
	57	+ return #NAME;
	58	+
	59	+ /* private: auto-generated case statements */
	60	+ MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_threshold_key_to_string_case)
	61	+#undef define_threshold_key_to_string_case
	62	+
51	63	default:
52	64	return "(unknown)";
53	65	}

drivers/gpu/drm/xe/xe_guc_klv_helpers.h+14−1

@@ -6,6 +6,7 @@ · 5 unmodified lines
6	6	#ifndef _XE_GUC_KLV_HELPERS_H_
7	7	#define _XE_GUC_KLV_HELPERS_H_
8	8
	9	+#include <linux/args.h>
9	10	#include <linux/types.h>
10	11
11	12	struct drm_printer;
@@ -37,6 +38,18 @@ int xe_guc_klv_count(const u32 *klvs, u32 num_dwords); · 25 unmodified lines
37	38	(FIELD_PREP_CONST(GUC_KLV_0_KEY, (key)) \| \
38	39	FIELD_PREP_CONST(GUC_KLV_0_LEN, (len)))
39	40
	41	+/**
	42	+ * MAKE_GUC_KLV_KEY - Prepare KLV KEY name based on unique KLV definition tag.
	43	+ * @TAG: unique tag of the KLV definition
	44	+ */
	45	+#define MAKE_GUC_KLV_KEY(TAG) CONCATENATE(CONCATENATE(GUC_KLV_, TAG), _KEY)
	46	+
	47	+/**
	48	+ * MAKE_GUC_KLV_LEN - Prepare KLV LEN name based on unique KLV definition tag.
	49	+ * @TAG: unique tag of the KLV definition
	50	+ */
	51	+#define MAKE_GUC_KLV_LEN(TAG) CONCATENATE(CONCATENATE(GUC_KLV_, TAG), _LEN)
	52	+
40	53	/**
41	54	* PREP_GUC_KLV_TAG - Prepare KLV header value based on unique KLV definition tag.
42	55	* @TAG: unique tag of the KLV definition
@@ -46,6 +59,6 @@ int xe_guc_klv_count(const u32 *klvs, u32 num_dwords); · 3 unmodified lines
46	59	* Return: value of the KLV header (u32).
47	60	*/
48	61	#define PREP_GUC_KLV_TAG(TAG) \
49		− PREP_GUC_KLV_CONST(GUC_KLV_##TAG##_KEY, GUC_KLV_##TAG##_LEN)
	62	+ PREP_GUC_KLV_CONST(MAKE_GUC_KLV_KEY(TAG), MAKE_GUC_KLV_LEN(TAG))
50	63
51	64	#endif

drivers/gpu/drm/xe/xe_guc_klv_thresholds_set.h+64−0

@@ -0,0 +1,64 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GUC_KLV_THRESHOLDS_SET_H_
	7	+#define _XE_GUC_KLV_THRESHOLDS_SET_H_
	8	+
	9	+#include "abi/guc_klvs_abi.h"
	10	+#include "xe_guc_klv_helpers.h"
	11	+#include "xe_guc_klv_thresholds_set_types.h"
	12	+
	13	+/**
	14	+ * MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY - Prepare the name of the KLV key constant.
	15	+ * @TAG: unique tag of the GuC threshold KLV key.
	16	+ */
	17	+#define MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY(TAG) \
	18	+ MAKE_GUC_KLV_KEY(CONCATENATE(VF_CFG_THRESHOLD_, TAG))
	19	+
	20	+/**
	21	+ * xe_guc_klv_threshold_key_to_index - Find index of the tracked GuC threshold.
	22	+ * @key: GuC threshold KLV key.
	23	+ *
	24	+ * This translation is automatically generated using &MAKE_XE_GUC_KLV_THRESHOLDS_SET.
	25	+ * Return: index of the GuC threshold KLV or -1 if not found.
	26	+ */
	27	+static inline int xe_guc_klv_threshold_key_to_index(u32 key)
	28	+{
	29	+ switch (key) {
	30	+#define define_xe_guc_klv_threshold_key_to_index_case(TAG, ...) \
	31	+ \
	32	+ case MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY(TAG): \
	33	+ return MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG);
	34	+
	35	+ /* private: auto-generated case statements */
	36	+ MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_xe_guc_klv_threshold_key_to_index_case)
	37	+ }
	38	+ return -1;
	39	+#undef define_xe_guc_klv_threshold_key_to_index_case
	40	+}
	41	+
	42	+/**
	43	+ * xe_guc_klv_threshold_index_to_key - Get tracked GuC threshold KLV key.
	44	+ * @index: GuC threshold KLV index.
	45	+ *
	46	+ * This translation is automatically generated using &MAKE_XE_GUC_KLV_THRESHOLDS_SET.
	47	+ * Return: key of the GuC threshold KLV or 0 on malformed index.
	48	+ */
	49	+static inline u32 xe_guc_klv_threshold_index_to_key(enum xe_guc_klv_threshold_index index)
	50	+{
	51	+ switch (index) {
	52	+#define define_xe_guc_klv_threshold_index_to_key_case(TAG, ...) \
	53	+ \
	54	+ case MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG): \
	55	+ return MAKE_GUC_KLV_VF_CFG_THRESHOLD_KEY(TAG);
	56	+
	57	+ /* private: auto-generated case statements */
	58	+ MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_xe_guc_klv_threshold_index_to_key_case)
	59	+ }
	60	+ return 0; /* unreachable */
	61	+#undef define_xe_guc_klv_threshold_index_to_key_case
	62	+}
	63	+
	64	+#endif

drivers/gpu/drm/xe/xe_guc_klv_thresholds_set_types.h+68−0

@@ -0,0 +1,68 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_GUC_KLV_THRESHOLDS_SET_TYPES_H_
	7	+#define _XE_GUC_KLV_THRESHOLDS_SET_TYPES_H_
	8	+
	9	+#include "xe_args.h"
	10	+
	11	+/**
	12	+ * MAKE_XE_GUC_KLV_THRESHOLDS_SET - Generate various GuC thresholds definitions.
	13	+ * @define: name of the inner macro to expand.
	14	+ *
	15	+ * The GuC firmware is able to monitor VF's adverse activity and will notify the
	16	+ * PF driver once any threshold is exceeded.
	17	+ *
	18	+ * This super macro allows various conversions between the GuC adverse event
	19	+ * threshold KLV definitions and the driver code without repeating similar code
	20	+ * or risking missing some cases.
	21	+ *
	22	+ * For each GuC threshold definition, the inner macro &define will be provided
	23	+ * with the &TAG, that corresponds to the GuC threshold KLV key name defined by
	24	+ * ABI and the associated &NAME, that may be used in code or debugfs/sysfs::
	25	+ *
	26	+ * define(TAG, NAME)
	27	+ */
	28	+#define MAKE_XE_GUC_KLV_THRESHOLDS_SET(define) \
	29	+ define(CAT_ERR, cat_error_count) \
	30	+ define(ENGINE_RESET, engine_reset_count) \
	31	+ define(PAGE_FAULT, page_fault_count) \
	32	+ define(H2G_STORM, guc_time_us) \
	33	+ define(IRQ_STORM, irq_time_us) \
	34	+ define(DOORBELL_STORM, doorbell_time_us) \
	35	+ /* end */
	36	+
	37	+/**
	38	+ * XE_GUC_KLV_NUM_THRESHOLDS - Number of GuC thresholds KLVs.
	39	+ *
	40	+ * Calculated automatically using &MAKE_XE_GUC_KLV_THRESHOLDS_SET.
	41	+ */
	42	+#define XE_GUC_KLV_NUM_THRESHOLDS \
	43	+ (CALL_ARGS(COUNT_ARGS, MAKE_XE_GUC_KLV_THRESHOLDS_SET(ARGS_SEP_COMMA)) - 1)
	44	+
	45	+/**
	46	+ * MAKE_XE_GUC_KLV_THRESHOLD_INDEX - Create enumerator name.
	47	+ * @TAG: unique TAG of the enum xe_guc_klv_threshold_index.
	48	+ */
	49	+#define MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG) \
	50	+ CONCATENATE(XE_GUC_KLV_THRESHOLD_INDEX_, TAG)
	51	+
	52	+/**
	53	+ * enum xe_guc_klv_threshold_index - Index of the tracked GuC threshold.
	54	+ *
	55	+ * This enum is automatically generated using &MAKE_XE_GUC_KLV_THRESHOLDS_SET.
	56	+ * All these generated enumerators will only be used by the also generated code.
	57	+ */
	58	+enum xe_guc_klv_threshold_index {
	59	+#define define_xe_guc_klv_threshold_index_enum(TAG, ...) \
	60	+ \
	61	+ MAKE_XE_GUC_KLV_THRESHOLD_INDEX(TAG),
	62	+
	63	+ /* private: auto-generated enum definitions */
	64	+ MAKE_XE_GUC_KLV_THRESHOLDS_SET(define_xe_guc_klv_threshold_index_enum)
	65	+#undef define_xe_guc_klv_threshold_index_enum
	66	+};
	67	+
	68	+#endif

drivers/gpu/drm/xe/xe_guc_pc.c+5−7

@@ -15,6 +15,7 @@ · 14 unmodified lines
15	15	#include "regs/xe_regs.h"
16	16	#include "xe_bo.h"
17	17	#include "xe_device.h"
	18	+#include "xe_force_wake.h"
18	19	#include "xe_gt.h"
19	20	#include "xe_gt_idle.h"
20	21	#include "xe_gt_sysfs.h"
@@ -888,19 +889,16 @@ int xe_guc_pc_stop(struct xe_guc_pc *pc) · 867 unmodified lines
888	889	}
889	890
890	891	/**
891		− * xe_guc_pc_fini - Finalize GuC's Power Conservation component
892		− * @drm: DRM device
	892	+ * xe_guc_pc_fini_hw - Finalize GuC's Power Conservation component
893	893	* @arg: opaque pointer that should point to Xe_GuC_PC instance
894	894	*/
895		−static void xe_guc_pc_fini(struct drm_device drm, void arg)
	895	+static void xe_guc_pc_fini_hw(void *arg)
896	896	{
897	897	struct xe_guc_pc *pc = arg;
898	898	struct xe_device *xe = pc_to_xe(pc);
899	899
900		− if (xe->info.skip_guc_pc) {
901		− xe_gt_idle_disable_c6(pc_to_gt(pc));
	900	+ if (xe_device_wedged(xe))
902	901	return;
903		− }
904	902
905	903	XE_WARN_ON(xe_force_wake_get(gt_to_fw(pc_to_gt(pc)), XE_FORCEWAKE_ALL));
906	904	XE_WARN_ON(xe_guc_pc_gucrc_disable(pc));
@@ -937,5 +935,5 @@ int xe_guc_pc_init(struct xe_guc_pc *pc) · 30 unmodified lines
937	935
938	936	pc->bo = bo;
939	937
940		− return drmm_add_action_or_reset(&xe->drm, xe_guc_pc_fini, pc);
	938	+ return devm_add_action_or_reset(xe->drm.dev, xe_guc_pc_fini_hw, pc);
941	939	}

drivers/gpu/drm/xe/xe_guc_pc.h+4−1

@@ -6,7 +6,9 @@ · 5 unmodified lines
6	6	#ifndef _XE_GUC_PC_H_
7	7	#define _XE_GUC_PC_H_
8	8
9		−#include "xe_guc_pc_types.h"
	9	+#include <linux/types.h>
	10	+
	11	+struct xe_guc_pc;
10	12
11	13	int xe_guc_pc_init(struct xe_guc_pc *pc);
12	14	int xe_guc_pc_start(struct xe_guc_pc *pc);
@@ -27,4 +29,5 @@ enum xe_gt_idle_state xe_guc_pc_c_status(struct xe_guc_pc *pc); · 14 unmodified lines
27	29	u64 xe_guc_pc_rc6_residency(struct xe_guc_pc *pc);
28	30	u64 xe_guc_pc_mc6_residency(struct xe_guc_pc *pc);
29	31	void xe_guc_pc_init_early(struct xe_guc_pc *pc);
	32	+
30	33	#endif /* _XE_GUC_PC_H_ */

drivers/gpu/drm/xe/xe_guc_relay.c+14−3

@@ -19,6 +19,7 @@ · 18 unmodified lines
19	19	#include "xe_device.h"
20	20	#include "xe_gt.h"
21	21	#include "xe_gt_sriov_printk.h"
	22	+#include "xe_gt_sriov_pf_service.h"
22	23	#include "xe_guc.h"
23	24	#include "xe_guc_ct.h"
24	25	#include "xe_guc_hxg_helpers.h"
@@ -664,6 +665,7 @@ static int relay_testloop_action_handler(struct xe_guc_relay *relay, u32 origin, · 639 unmodified lines
664	665	static int relay_action_handler(struct xe_guc_relay *relay, u32 origin,
665	666	const u32 msg, u32 len, u32 response, u32 size)
666	667	{
	668	+ struct xe_gt *gt = relay_to_gt(relay);
667	669	u32 type;
668	670	int ret;
669	671
@@ -674,8 +676,10 @@ static int relay_action_handler(struct xe_guc_relay *relay, u32 origin, · 4 unmodified lines
674	676
675	677	type = FIELD_GET(GUC_HXG_MSG_0_TYPE, msg[0]);
676	678
677		− /* XXX: PF services will be added later */
678		− ret = -EOPNOTSUPP;
	679	+ if (IS_SRIOV_PF(relay_to_xe(relay)))
	680	+ ret = xe_gt_sriov_pf_service_process_request(gt, origin, msg, len, response, size);
	681	+ else
	682	+ ret = -EOPNOTSUPP;
679	683
680	684	if (type == GUC_HXG_TYPE_EVENT)
681	685	relay_assert(relay, ret <= 0);
@@ -757,7 +761,14 @@ static void relay_process_incoming_action(struct xe_guc_relay *relay) · 75 unmodified lines
757	761
758	762	static bool relay_needs_worker(struct xe_guc_relay *relay)
759	763	{
760		− return !list_empty(&relay->incoming_actions);
	764	+ bool is_empty;
	765	+
	766	+ spin_lock(&relay->lock);
	767	+ is_empty = list_empty(&relay->incoming_actions);
	768	+ spin_unlock(&relay->lock);
	769	+
	770	+ return !is_empty;
	771	+
761	772	}
762	773
763	774	static void relay_kick_worker(struct xe_guc_relay *relay)

drivers/gpu/drm/xe/xe_guc_submit.c+162−119

@@ -35,6 +35,7 @@ · 34 unmodified lines
35	35	#include "xe_macros.h"
36	36	#include "xe_map.h"
37	37	#include "xe_mocs.h"
	38	+#include "xe_pm.h"
38	39	#include "xe_ring_ops_types.h"
39	40	#include "xe_sched_job.h"
40	41	#include "xe_trace.h"
@@ -52,13 +53,14 @@ exec_queue_to_guc(struct xe_exec_queue *q) · 11 unmodified lines
52	53	* engine done being processed).
53	54	*/
54	55	#define EXEC_QUEUE_STATE_REGISTERED (1 << 0)
55		−#define ENGINE_STATE_ENABLED (1 << 1)
56		−#define EXEC_QUEUE_STATE_PENDING_ENABLE (1 << 2)
	56	+#define EXEC_QUEUE_STATE_ENABLED (1 << 1)
	57	+#define EXEC_QUEUE_STATE_PENDING_ENABLE (1 << 2)
57	58	#define EXEC_QUEUE_STATE_PENDING_DISABLE (1 << 3)
58	59	#define EXEC_QUEUE_STATE_DESTROYED (1 << 4)
59		−#define ENGINE_STATE_SUSPENDED (1 << 5)
60		−#define EXEC_QUEUE_STATE_RESET (1 << 6)
61		−#define ENGINE_STATE_KILLED (1 << 7)
	60	+#define EXEC_QUEUE_STATE_SUSPENDED (1 << 5)
	61	+#define EXEC_QUEUE_STATE_RESET (1 << 6)
	62	+#define EXEC_QUEUE_STATE_KILLED (1 << 7)
	63	+#define EXEC_QUEUE_STATE_WEDGED (1 << 8)
62	64
63	65	static bool exec_queue_registered(struct xe_exec_queue *q)
64	66	{
@@ -77,17 +79,17 @@ static void clear_exec_queue_registered(struct xe_exec_queue *q) · 12 unmodified lines
77	79
78	80	static bool exec_queue_enabled(struct xe_exec_queue *q)
79	81	{
80		− return atomic_read(&q->guc->state) & ENGINE_STATE_ENABLED;
	82	+ return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_ENABLED;
81	83	}
82	84
83	85	static void set_exec_queue_enabled(struct xe_exec_queue *q)
84	86	{
85		− atomic_or(ENGINE_STATE_ENABLED, &q->guc->state);
	87	+ atomic_or(EXEC_QUEUE_STATE_ENABLED, &q->guc->state);
86	88	}
87	89
88	90	static void clear_exec_queue_enabled(struct xe_exec_queue *q)
89	91	{
90		− atomic_and(~ENGINE_STATE_ENABLED, &q->guc->state);
	92	+ atomic_and(~EXEC_QUEUE_STATE_ENABLED, &q->guc->state);
91	93	}
92	94
93	95	static bool exec_queue_pending_enable(struct xe_exec_queue *q)
@@ -142,17 +144,17 @@ static void set_exec_queue_banned(struct xe_exec_queue *q) · 48 unmodified lines
142	144
143	145	static bool exec_queue_suspended(struct xe_exec_queue *q)
144	146	{
145		− return atomic_read(&q->guc->state) & ENGINE_STATE_SUSPENDED;
	147	+ return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_SUSPENDED;
146	148	}
147	149
148	150	static void set_exec_queue_suspended(struct xe_exec_queue *q)
149	151	{
150		− atomic_or(ENGINE_STATE_SUSPENDED, &q->guc->state);
	152	+ atomic_or(EXEC_QUEUE_STATE_SUSPENDED, &q->guc->state);
151	153	}
152	154
153	155	static void clear_exec_queue_suspended(struct xe_exec_queue *q)
154	156	{
155		− atomic_and(~ENGINE_STATE_SUSPENDED, &q->guc->state);
	157	+ atomic_and(~EXEC_QUEUE_STATE_SUSPENDED, &q->guc->state);
156	158	}
157	159
158	160	static bool exec_queue_reset(struct xe_exec_queue *q)
@@ -167,17 +169,28 @@ static void set_exec_queue_reset(struct xe_exec_queue *q) · 8 unmodified lines
167	169
168	170	static bool exec_queue_killed(struct xe_exec_queue *q)
169	171	{
170		− return atomic_read(&q->guc->state) & ENGINE_STATE_KILLED;
	172	+ return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_KILLED;
171	173	}
172	174
173	175	static void set_exec_queue_killed(struct xe_exec_queue *q)
174	176	{
175		− atomic_or(ENGINE_STATE_KILLED, &q->guc->state);
	177	+ atomic_or(EXEC_QUEUE_STATE_KILLED, &q->guc->state);
176	178	}
177	179
178		−static bool exec_queue_killed_or_banned(struct xe_exec_queue *q)
	180	+static bool exec_queue_wedged(struct xe_exec_queue *q)
179	181	{
180		− return exec_queue_killed(q) \|\| exec_queue_banned(q);
	182	+ return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_WEDGED;
	183	+}
	184	+
	185	+static void set_exec_queue_wedged(struct xe_exec_queue *q)
	186	+{
	187	+ atomic_or(EXEC_QUEUE_STATE_WEDGED, &q->guc->state);
	188	+}
	189	+
	190	+static bool exec_queue_killed_or_banned_or_wedged(struct xe_exec_queue *q)
	191	+{
	192	+ return exec_queue_banned(q) \|\| (atomic_read(&q->guc->state) &
	193	+ (EXEC_QUEUE_STATE_WEDGED \| EXEC_QUEUE_STATE_KILLED));
181	194	}
182	195
183	196	#ifdef CONFIG_PROVE_LOCKING
@@ -240,6 +253,17 @@ static void guc_submit_fini(struct drm_device drm, void arg) · 56 unmodified lines
240	253	free_submit_wq(guc);
241	254	}
242	255
	256	+static void guc_submit_wedged_fini(struct drm_device drm, void arg)
	257	+{
	258	+ struct xe_guc *guc = arg;
	259	+ struct xe_exec_queue *q;
	260	+ unsigned long index;
	261	+
	262	+ xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
	263	+ if (exec_queue_wedged(q))
	264	+ xe_exec_queue_put(q);
	265	+}
	266	+
243	267	static const struct xe_exec_queue_ops guc_exec_queue_ops;
244	268
245	269	static void primelockdep(struct xe_guc *guc)
@@ -250,13 +274,25 @@ static void primelockdep(struct xe_guc *guc) · 4 unmodified lines
250	274	fs_reclaim_acquire(GFP_KERNEL);
251	275
252	276	mutex_lock(&guc->submission_state.lock);
253		− might_lock(&guc->submission_state.suspend.lock);
254	277	mutex_unlock(&guc->submission_state.lock);
255	278
256	279	fs_reclaim_release(GFP_KERNEL);
257	280	}
258	281
259		−int xe_guc_submit_init(struct xe_guc *guc)
	282	+/**
	283	+ * xe_guc_submit_init() - Initialize GuC submission.
	284	+ * @guc: the &xe_guc to initialize
	285	+ * @num_ids: number of GuC context IDs to use
	286	+ *
	287	+ * The bare-metal or PF driver can pass ~0 as &num_ids to indicate that all
	288	+ * GuC context IDs supported by the GuC firmware should be used for submission.
	289	+ *
	290	+ * Only VF drivers will have to provide explicit number of GuC context IDs
	291	+ * that they can use for submission.
	292	+ *
	293	+ * Return: 0 on success or a negative error code on failure.
	294	+ */
	295	+int xe_guc_submit_init(struct xe_guc *guc, unsigned int num_ids)
260	296	{
261	297	struct xe_device *xe = guc_to_xe(guc);
262	298	struct xe_gt *gt = guc_to_gt(guc);
@@ -266,7 +302,7 @@ int xe_guc_submit_init(struct xe_guc *guc) · 3 unmodified lines
266	302	if (err)
267	303	return err;
268	304
269		− err = xe_guc_id_mgr_init(&guc->submission_state.idm, ~0);
	305	+ err = xe_guc_id_mgr_init(&guc->submission_state.idm, num_ids);
270	306	if (err)
271	307	return err;
272	308
@@ -278,9 +314,6 @@ int xe_guc_submit_init(struct xe_guc *guc) · 5 unmodified lines
278	314
279	315	xa_init(&guc->submission_state.exec_queue_lookup);
280	316
281		− spin_lock_init(&guc->submission_state.suspend.lock);
282		− guc->submission_state.suspend.context = dma_fence_context_alloc(1);
283		−
284	317	primelockdep(guc);
285	318
286	319	return drmm_add_action_or_reset(&xe->drm, guc_submit_fini, guc);
@@ -430,9 +463,9 @@ static void set_min_preemption_timeout(struct xe_guc *guc, struct xe_exec_queue · 143 unmodified lines
430	463	xe_map_wr_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
431	464	field_, val_)
432	465
433		−static void __register_mlrc_engine(struct xe_guc *guc,
434		− struct xe_exec_queue *q,
435		− struct guc_ctxt_registration_info *info)
	466	+static void __register_mlrc_exec_queue(struct xe_guc *guc,
	467	+ struct xe_exec_queue *q,
	468	+ struct guc_ctxt_registration_info *info)
436	469	{
437	470	#define MAX_MLRC_REG_SIZE (13 + XE_HW_ENGINE_MAX_INSTANCE * 2)
438	471	struct xe_device *xe = guc_to_xe(guc);
@@ -457,7 +490,7 @@ static void __register_mlrc_engine(struct xe_guc *guc, · 18 unmodified lines
457	490	action[len++] = info->hwlrca_hi;
458	491
459	492	for (i = 1; i < q->width; ++i) {
460		− struct xe_lrc *lrc = q->lrc + i;
	493	+ struct xe_lrc *lrc = q->lrc[i];
461	494
462	495	action[len++] = lower_32_bits(xe_lrc_descriptor(lrc));
463	496	action[len++] = upper_32_bits(xe_lrc_descriptor(lrc));
@@ -469,8 +502,8 @@ static void __register_mlrc_engine(struct xe_guc *guc, · 5 unmodified lines
469	502	xe_guc_ct_send(&guc->ct, action, len, 0, 0);
470	503	}
471	504
472		−static void __register_engine(struct xe_guc *guc,
473		− struct guc_ctxt_registration_info *info)
	505	+static void __register_exec_queue(struct xe_guc *guc,
	506	+ struct guc_ctxt_registration_info *info)
474	507	{
475	508	u32 action[] = {
476	509	XE_GUC_ACTION_REGISTER_CONTEXT,
@@ -490,11 +523,11 @@ static void __register_engine(struct xe_guc *guc, · 13 unmodified lines
490	523	xe_guc_ct_send(&guc->ct, action, ARRAY_SIZE(action), 0, 0);
491	524	}
492	525
493		−static void register_engine(struct xe_exec_queue *q)
	526	+static void register_exec_queue(struct xe_exec_queue *q)
494	527	{
495	528	struct xe_guc *guc = exec_queue_to_guc(q);
496	529	struct xe_device *xe = guc_to_xe(guc);
497		− struct xe_lrc *lrc = q->lrc;
	530	+ struct xe_lrc *lrc = q->lrc[0];
498	531	struct guc_ctxt_registration_info info;
499	532
500	533	xe_assert(xe, !exec_queue_registered(q));
@@ -538,9 +571,9 @@ static void register_engine(struct xe_exec_queue *q) · 37 unmodified lines
538	571	set_exec_queue_registered(q);
539	572	trace_xe_exec_queue_register(q);
540	573	if (xe_exec_queue_is_parallel(q))
541		− __register_mlrc_engine(guc, q, &info);
	574	+ __register_mlrc_exec_queue(guc, q, &info);
542	575	else
543		− __register_engine(guc, &info);
	576	+ __register_exec_queue(guc, &info);
544	577	init_policies(guc, q);
545	578	}
546	579
@@ -553,7 +586,7 @@ static int wq_wait_for_space(struct xe_exec_queue *q, u32 wqi_size) · 6 unmodified lines
553	586	{
554	587	struct xe_guc *guc = exec_queue_to_guc(q);
555	588	struct xe_device *xe = guc_to_xe(guc);
556		− struct iosys_map map = xe_lrc_parallel_map(q->lrc);
	589	+ struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
557	590	unsigned int sleep_period_ms = 1;
558	591
559	592	#define AVAILABLE_SPACE \
@@ -581,7 +614,7 @@ static int wq_noop_append(struct xe_exec_queue *q) · 21 unmodified lines
581	614	{
582	615	struct xe_guc *guc = exec_queue_to_guc(q);
583	616	struct xe_device *xe = guc_to_xe(guc);
584		− struct iosys_map map = xe_lrc_parallel_map(q->lrc);
	617	+ struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
585	618	u32 len_dw = wq_space_until_wrap(q) / sizeof(u32) - 1;
586	619
587	620	if (wq_wait_for_space(q, wq_space_until_wrap(q)))
@@ -601,7 +634,7 @@ static void wq_item_append(struct xe_exec_queue *q) · 13 unmodified lines
601	634	{
602	635	struct xe_guc *guc = exec_queue_to_guc(q);
603	636	struct xe_device *xe = guc_to_xe(guc);
604		− struct iosys_map map = xe_lrc_parallel_map(q->lrc);
	637	+ struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
605	638	#define WQ_HEADER_SIZE 4 /* Includes 1 LRC address too */
606	639	u32 wqi[XE_HW_ENGINE_MAX_INSTANCE + (WQ_HEADER_SIZE - 1)];
607	640	u32 wqi_size = (q->width + (WQ_HEADER_SIZE - 1)) * sizeof(u32);
@@ -617,12 +650,12 @@ static void wq_item_append(struct xe_exec_queue *q) · 9 unmodified lines
617	650
618	651	wqi[i++] = FIELD_PREP(WQ_TYPE_MASK, WQ_TYPE_MULTI_LRC) \|
619	652	FIELD_PREP(WQ_LEN_MASK, len_dw);
620		− wqi[i++] = xe_lrc_descriptor(q->lrc);
	653	+ wqi[i++] = xe_lrc_descriptor(q->lrc[0]);
621	654	wqi[i++] = FIELD_PREP(WQ_GUC_ID_MASK, q->guc->id) \|
622		− FIELD_PREP(WQ_RING_TAIL_MASK, q->lrc->ring.tail / sizeof(u64));
	655	+ FIELD_PREP(WQ_RING_TAIL_MASK, q->lrc[0]->ring.tail / sizeof(u64));
623	656	wqi[i++] = 0;
624	657	for (j = 1; j < q->width; ++j) {
625		− struct xe_lrc *lrc = q->lrc + j;
	658	+ struct xe_lrc *lrc = q->lrc[j];
626	659
627	660	wqi[i++] = lrc->ring.tail / sizeof(u64);
628	661	}
@@ -637,7 +670,7 @@ static void wq_item_append(struct xe_exec_queue *q) · 8 unmodified lines
637	670
638	671	xe_device_wmb(xe);
639	672
640		− map = xe_lrc_parallel_map(q->lrc);
	673	+ map = xe_lrc_parallel_map(q->lrc[0]);
641	674	parallel_write(xe, map, wq_desc.tail, q->guc->wqi_tail);
642	675	}
643	676
@@ -646,7 +679,7 @@ static void submit_exec_queue(struct xe_exec_queue *q) · 2 unmodified lines
646	679	{
647	680	struct xe_guc *guc = exec_queue_to_guc(q);
648	681	struct xe_device *xe = guc_to_xe(guc);
649		− struct xe_lrc *lrc = q->lrc;
	682	+ struct xe_lrc *lrc = q->lrc[0];
650	683	u32 action[3];
651	684	u32 g2h_len = 0;
652	685	u32 num_g2h = 0;
@@ -658,7 +691,7 @@ static void submit_exec_queue(struct xe_exec_queue *q) · 5 unmodified lines
658	691	if (xe_exec_queue_is_parallel(q))
659	692	wq_item_append(q);
660	693	else
661		− xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, lrc->ring.tail);
	694	+ xe_lrc_set_ring_tail(lrc, lrc->ring.tail);
662	695
663	696	if (exec_queue_suspended(q) && !xe_exec_queue_is_parallel(q))
664	697	return;
@@ -708,9 +741,9 @@ guc_exec_queue_run_job(struct drm_sched_job *drm_job) · 43 unmodified lines
708	741
709	742	trace_xe_sched_job_run(job);
710	743
711		− if (!exec_queue_killed_or_banned(q) && !xe_sched_job_is_error(job)) {
	744	+ if (!exec_queue_killed_or_banned_or_wedged(q) && !xe_sched_job_is_error(job)) {
712	745	if (!exec_queue_registered(q))
713		− register_engine(q);
	746	+ register_exec_queue(q);
714	747	if (!lr) /* LR jobs are emitted in the exec IOCTL */
715	748	q->ring_ops->emit_job(job);
716	749	submit_exec_queue(q);
@@ -730,6 +763,8 @@ static void guc_exec_queue_free_job(struct drm_sched_job *drm_job) · 13 unmodified lines
730	763	{
731	764	struct xe_sched_job *job = to_xe_sched_job(drm_job);
732	765
	766	+ xe_exec_queue_update_run_ticks(job->q);
	767	+
733	768	trace_xe_sched_job_free(job);
734	769	xe_sched_job_put(job);
735	770	}
@@ -781,55 +816,6 @@ static void disable_scheduling_deregister(struct xe_guc *guc, · 45 unmodified lines
781	816	G2H_LEN_DW_DEREGISTER_CONTEXT, 2);
782	817	}
783	818
784		−static void guc_exec_queue_print(struct xe_exec_queue q, struct drm_printer p);
785		−
786		−#if IS_ENABLED(CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE)
787		−static void simple_error_capture(struct xe_exec_queue *q)
788		−{
789		− struct xe_guc *guc = exec_queue_to_guc(q);
790		− struct xe_device *xe = guc_to_xe(guc);
791		− struct drm_printer p = drm_err_printer(&xe->drm, NULL);
792		− struct xe_hw_engine *hwe;
793		− enum xe_hw_engine_id id;
794		− u32 adj_logical_mask = q->logical_mask;
795		− u32 width_mask = (0x1 << q->width) - 1;
796		− int i;
797		− bool cookie;
798		−
799		− if (q->vm && !q->vm->error_capture.capture_once) {
800		− q->vm->error_capture.capture_once = true;
801		− cookie = dma_fence_begin_signalling();
802		− for (i = 0; q->width > 1 && i < XE_HW_ENGINE_MAX_INSTANCE;) {
803		− if (adj_logical_mask & BIT(i)) {
804		− adj_logical_mask \|= width_mask << i;
805		− i += q->width;
806		− } else {
807		− ++i;
808		− }
809		− }
810		−
811		− if (xe_force_wake_get(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL))
812		− xe_gt_info(guc_to_gt(guc),
813		− "failed to get forcewake for error capture");
814		− xe_guc_ct_print(&guc->ct, &p, true);
815		− guc_exec_queue_print(q, &p);
816		− for_each_hw_engine(hwe, guc_to_gt(guc), id) {
817		− if (hwe->class != q->hwe->class \|\|
818		− !(BIT(hwe->logical_instance) & adj_logical_mask))
819		− continue;
820		− xe_hw_engine_print(hwe, &p);
821		− }
822		− xe_analyze_vm(&p, q->vm, q->gt->info.id);
823		− xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
824		− dma_fence_end_signalling(cookie);
825		− }
826		−}
827		−#else
828		−static void simple_error_capture(struct xe_exec_queue *q)
829		−{
830		−}
831		−#endif
832		−
833	819	static void xe_guc_exec_queue_trigger_cleanup(struct xe_exec_queue *q)
834	820	{
835	821	struct xe_guc *guc = exec_queue_to_guc(q);
@@ -844,6 +830,40 @@ static void xe_guc_exec_queue_trigger_cleanup(struct xe_exec_queue *q) · 8 unmodified lines
844	830	xe_sched_tdr_queue_imm(&q->guc->sched);
845	831	}
846	832
	833	+static bool guc_submit_hint_wedged(struct xe_guc *guc)
	834	+{
	835	+ struct xe_device *xe = guc_to_xe(guc);
	836	+ struct xe_exec_queue *q;
	837	+ unsigned long index;
	838	+ int err;
	839	+
	840	+ if (xe->wedged.mode != 2)
	841	+ return false;
	842	+
	843	+ if (xe_device_wedged(xe))
	844	+ return true;
	845	+
	846	+ xe_device_declare_wedged(xe);
	847	+
	848	+ xe_guc_submit_reset_prepare(guc);
	849	+ xe_guc_ct_stop(&guc->ct);
	850	+
	851	+ err = drmm_add_action_or_reset(&guc_to_xe(guc)->drm,
	852	+ guc_submit_wedged_fini, guc);
	853	+ if (err) {
	854	+ drm_err(&xe->drm, "Failed to register xe_guc_submit clean-up on wedged.mode=2. Although device is wedged.\n");
	855	+ return true; /* Device is wedged anyway */
	856	+ }
	857	+
	858	+ mutex_lock(&guc->submission_state.lock);
	859	+ xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
	860	+ if (xe_exec_queue_get_unless_zero(q))
	861	+ set_exec_queue_wedged(q);
	862	+ mutex_unlock(&guc->submission_state.lock);
	863	+
	864	+ return true;
	865	+}
	866	+
847	867	static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)
848	868	{
849	869	struct xe_guc_exec_queue *ge =
@@ -852,10 +872,13 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w) · 2 unmodified lines
852	872	struct xe_guc *guc = exec_queue_to_guc(q);
853	873	struct xe_device *xe = guc_to_xe(guc);
854	874	struct xe_gpu_scheduler *sched = &ge->sched;
	875	+ bool wedged;
855	876
856	877	xe_assert(xe, xe_exec_queue_is_lr(q));
857	878	trace_xe_exec_queue_lr_cleanup(q);
858	879
	880	+ wedged = guc_submit_hint_wedged(exec_queue_to_guc(q));
	881	+
859	882	/* Kill the run_job / process_msg entry points */
860	883	xe_sched_submission_stop(sched);
861	884
@@ -870,7 +893,7 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w) · 8 unmodified lines
870	893	* xe_guc_deregister_done_handler() which treats it as an unexpected
871	894	* state.
872	895	*/
873		− if (exec_queue_registered(q) && !exec_queue_destroyed(q)) {
	896	+ if (!wedged && exec_queue_registered(q) && !exec_queue_destroyed(q)) {
874	897	struct xe_guc *guc = exec_queue_to_guc(q);
875	898	int ret;
876	899
@@ -905,6 +928,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 28 unmodified lines
905	928	struct xe_device *xe = guc_to_xe(exec_queue_to_guc(q));
906	929	int err = -ETIME;
907	930	int i = 0;
	931	+ bool wedged;
908	932
909	933	/*
910	934	* TDR has fired before free job worker. Common if exec queue
@@ -916,18 +940,21 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 5 unmodified lines
916	940	return DRM_GPU_SCHED_STAT_NOMINAL;
917	941	}
918	942
919		− drm_notice(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
920		− xe_sched_job_seqno(job), q->guc->id, q->flags);
	943	+ drm_notice(&xe->drm, "Timedout job: seqno=%u, lrc_seqno=%u, guc_id=%d, flags=0x%lx",
	944	+ xe_sched_job_seqno(job), xe_sched_job_lrc_seqno(job),
	945	+ q->guc->id, q->flags);
921	946	xe_gt_WARN(q->gt, q->flags & EXEC_QUEUE_FLAG_KERNEL,
922	947	"Kernel-submitted job timed out\n");
923	948	xe_gt_WARN(q->gt, q->flags & EXEC_QUEUE_FLAG_VM && !exec_queue_killed(q),
924	949	"VM job timed out on non-killed execqueue\n");
925	950
926		− simple_error_capture(q);
927		− xe_devcoredump(job);
	951	+ if (!exec_queue_killed(q))
	952	+ xe_devcoredump(job);
928	953
929	954	trace_xe_sched_job_timedout(job);
930	955
	956	+ wedged = guc_submit_hint_wedged(exec_queue_to_guc(q));
	957	+
931	958	/* Kill the run_job entry point */
932	959	xe_sched_submission_stop(sched);
933	960
@@ -935,8 +962,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 1 unmodified line
935	962	* Kernel jobs should never fail, nor should VM jobs if they do
936	963	* somethings has gone wrong and the GT needs a reset
937	964	*/
938		− if (q->flags & EXEC_QUEUE_FLAG_KERNEL \|\|
939		− (q->flags & EXEC_QUEUE_FLAG_VM && !exec_queue_killed(q))) {
	965	+ if (!wedged && (q->flags & EXEC_QUEUE_FLAG_KERNEL \|\|
	966	+ (q->flags & EXEC_QUEUE_FLAG_VM && !exec_queue_killed(q)))) {
940	967	if (!xe_sched_invalidate_job(job, 2)) {
941	968	xe_sched_add_pending_job(sched, job);
942	969	xe_sched_submission_start(sched);
@@ -946,7 +973,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 3 unmodified lines
946	973	}
947	974
948	975	/* Engine state now stable, disable scheduling if needed */
949		− if (exec_queue_registered(q)) {
	976	+ if (!wedged && exec_queue_registered(q)) {
950	977	struct xe_guc *guc = exec_queue_to_guc(q);
951	978	int ret;
952	979
@@ -989,6 +1016,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 36 unmodified lines
989	1016	*/
990	1017	xe_sched_add_pending_job(sched, job);
991	1018	xe_sched_submission_start(sched);
	1019	+
992	1020	xe_guc_exec_queue_trigger_cleanup(q);
993	1021
994	1022	/* Mark all outstanding jobs as bad, thus completing them */
@@ -1011,6 +1039,7 @@ static void __guc_exec_queue_fini_async(struct work_struct *w) · 16 unmodified lines
1011	1039	struct xe_exec_queue *q = ge->q;
1012	1040	struct xe_guc *guc = exec_queue_to_guc(q);
1013	1041
	1042	+ xe_pm_runtime_get(guc_to_xe(guc));
1014	1043	trace_xe_exec_queue_destroy(q);
1015	1044
1016	1045	if (xe_exec_queue_is_lr(q))
@@ -1021,6 +1050,7 @@ static void __guc_exec_queue_fini_async(struct work_struct *w) · 4 unmodified lines
1021	1050
1022	1051	kfree(ge);
1023	1052	xe_exec_queue_fini(q);
	1053	+ xe_pm_runtime_put(guc_to_xe(guc));
1024	1054	}
1025	1055
1026	1056	static void guc_exec_queue_fini_async(struct xe_exec_queue *q)
@@ -1028,7 +1058,7 @@ static void guc_exec_queue_fini_async(struct xe_exec_queue *q) · 1 unmodified line
1028	1058	INIT_WORK(&q->guc->fini_async, __guc_exec_queue_fini_async);
1029	1059
1030	1060	/* We must block on kernel engines so slabs are empty on driver unload */
1031		− if (q->flags & EXEC_QUEUE_FLAG_PERMANENT)
	1061	+ if (q->flags & EXEC_QUEUE_FLAG_PERMANENT \|\| exec_queue_wedged(q))
1032	1062	__guc_exec_queue_fini_async(&q->guc->fini_async);
1033	1063	else
1034	1064	queue_work(system_wq, &q->guc->fini_async);
@@ -1063,7 +1093,7 @@ static void __guc_exec_queue_process_msg_cleanup(struct xe_sched_msg *msg) · 28 unmodified lines
1063	1093
1064	1094	static bool guc_exec_queue_allowed_to_change_state(struct xe_exec_queue *q)
1065	1095	{
1066		− return !exec_queue_killed_or_banned(q) && exec_queue_registered(q);
	1096	+ return !exec_queue_killed_or_banned_or_wedged(q) && exec_queue_registered(q);
1067	1097	}
1068	1098
1069	1099	static void __guc_exec_queue_process_msg_set_sched_props(struct xe_sched_msg *msg)
@@ -1206,7 +1236,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) · 136 unmodified lines
1206	1236	msecs_to_jiffies(q->sched_props.job_timeout_ms);
1207	1237	err = xe_sched_init(&ge->sched, &drm_sched_ops, &xe_sched_ops,
1208	1238	get_submit_wq(guc),
1209		− q->lrc[0].ring.size / MAX_JOB_SIZE_BYTES, 64,
	1239	+ q->lrc[0]->ring.size / MAX_JOB_SIZE_BYTES, 64,
1210	1240	timeout, guc_to_gt(guc)->ordered_wq, NULL,
1211	1241	q->name, gt_to_xe(q->gt)->drm.dev);
1212	1242	if (err)
@@ -1275,7 +1305,7 @@ static void guc_exec_queue_fini(struct xe_exec_queue *q) · 62 unmodified lines
1275	1305	{
1276	1306	struct xe_sched_msg *msg = q->guc->static_msgs + STATIC_MSG_CLEANUP;
1277	1307
1278		− if (!(q->flags & EXEC_QUEUE_FLAG_PERMANENT))
	1308	+ if (!(q->flags & EXEC_QUEUE_FLAG_PERMANENT) && !exec_queue_wedged(q))
1279	1309	guc_exec_queue_add_msg(q, msg, CLEANUP);
1280	1310	else
1281	1311	__guc_exec_queue_fini(exec_queue_to_guc(q), q);
@@ -1286,7 +1316,8 @@ static int guc_exec_queue_set_priority(struct xe_exec_queue *q, · 4 unmodified lines
1286	1316	{
1287	1317	struct xe_sched_msg *msg;
1288	1318
1289		− if (q->sched_props.priority == priority \|\| exec_queue_killed_or_banned(q))
	1319	+ if (q->sched_props.priority == priority \|\|
	1320	+ exec_queue_killed_or_banned_or_wedged(q))
1290	1321	return 0;
1291	1322
1292	1323	msg = kmalloc(sizeof(*msg), GFP_KERNEL);
@@ -1304,7 +1335,7 @@ static int guc_exec_queue_set_timeslice(struct xe_exec_queue *q, u32 timeslice_u · 11 unmodified lines
1304	1335	struct xe_sched_msg *msg;
1305	1336
1306	1337	if (q->sched_props.timeslice_us == timeslice_us \|\|
1307		− exec_queue_killed_or_banned(q))
	1338	+ exec_queue_killed_or_banned_or_wedged(q))
1308	1339	return 0;
1309	1340
1310	1341	msg = kmalloc(sizeof(*msg), GFP_KERNEL);
@@ -1323,7 +1354,7 @@ static int guc_exec_queue_set_preempt_timeout(struct xe_exec_queue *q, · 12 unmodified lines
1323	1354	struct xe_sched_msg *msg;
1324	1355
1325	1356	if (q->sched_props.preempt_timeout_us == preempt_timeout_us \|\|
1326		− exec_queue_killed_or_banned(q))
	1357	+ exec_queue_killed_or_banned_or_wedged(q))
1327	1358	return 0;
1328	1359
1329	1360	msg = kmalloc(sizeof(*msg), GFP_KERNEL);
@@ -1340,7 +1371,7 @@ static int guc_exec_queue_suspend(struct xe_exec_queue *q) · 10 unmodified lines
1340	1371	{
1341	1372	struct xe_sched_msg *msg = q->guc->static_msgs + STATIC_MSG_SUSPEND;
1342	1373
1343		− if (exec_queue_killed_or_banned(q) \|\| q->guc->suspend_pending)
	1374	+ if (exec_queue_killed_or_banned_or_wedged(q) \|\| q->guc->suspend_pending)
1344	1375	return -EINVAL;
1345	1376
1346	1377	q->guc->suspend_pending = true;
@@ -1370,7 +1401,7 @@ static void guc_exec_queue_resume(struct xe_exec_queue *q) · 23 unmodified lines
1370	1401
1371	1402	static bool guc_exec_queue_reset_status(struct xe_exec_queue *q)
1372	1403	{
1373		− return exec_queue_reset(q);
	1404	+ return exec_queue_reset(q) \|\| exec_queue_killed_or_banned_or_wedged(q);
1374	1405	}
1375	1406
1376	1407	/*
@@ -1411,7 +1442,7 @@ static void guc_exec_queue_stop(struct xe_guc guc, struct xe_exec_queue q) · 34 unmodified lines
1411	1442	set_exec_queue_suspended(q);
1412	1443	suspend_fence_signal(q);
1413	1444	}
1414		− atomic_and(EXEC_QUEUE_STATE_DESTROYED \| ENGINE_STATE_SUSPENDED,
	1445	+ atomic_and(EXEC_QUEUE_STATE_DESTROYED \| EXEC_QUEUE_STATE_SUSPENDED,
1415	1446	&q->guc->state);
1416	1447	q->guc->resume_time = 0;
1417	1448	trace_xe_exec_queue_stop(q);
@@ -1423,15 +1454,23 @@ static void guc_exec_queue_stop(struct xe_guc guc, struct xe_exec_queue q) · 5 unmodified lines
1423	1454	*/
1424	1455	if (!(q->flags & (EXEC_QUEUE_FLAG_KERNEL \| EXEC_QUEUE_FLAG_VM))) {
1425	1456	struct xe_sched_job *job = xe_sched_first_pending_job(sched);
	1457	+ bool ban = false;
1426	1458
1427	1459	if (job) {
1428	1460	if ((xe_sched_job_started(job) &&
1429	1461	!xe_sched_job_completed(job)) \|\|
1430	1462	xe_sched_invalidate_job(job, 2)) {
1431	1463	trace_xe_sched_job_ban(job);
1432		− xe_sched_tdr_queue_imm(&q->guc->sched);
1433		− set_exec_queue_banned(q);
	1464	+ ban = true;
1434	1465	}
	1466	+ } else if (xe_exec_queue_is_lr(q) &&
	1467	+ (xe_lrc_ring_head(q->lrc[0]) != xe_lrc_ring_tail(q->lrc[0]))) {
	1468	+ ban = true;
	1469	+ }
	1470	+
	1471	+ if (ban) {
	1472	+ set_exec_queue_banned(q);
	1473	+ xe_guc_exec_queue_trigger_cleanup(q);
1435	1474	}
1436	1475	}
1437	1476	}
@@ -1459,7 +1498,7 @@ void xe_guc_submit_reset_wait(struct xe_guc *guc) · 21 unmodified lines
1459	1498	wait_event(guc->ct.wq, !guc_read_stopped(guc));
1460	1499	}
1461	1500
1462		−int xe_guc_submit_stop(struct xe_guc *guc)
	1501	+void xe_guc_submit_stop(struct xe_guc *guc)
1463	1502	{
1464	1503	struct xe_exec_queue *q;
1465	1504	unsigned long index;
@@ -1479,19 +1518,18 @@ int xe_guc_submit_stop(struct xe_guc *guc) · 13 unmodified lines
1479	1518	* creation which is protected by guc->submission_state.lock.
1480	1519	*/
1481	1520
1482		− return 0;
1483	1521	}
1484	1522
1485	1523	static void guc_exec_queue_start(struct xe_exec_queue *q)
1486	1524	{
1487	1525	struct xe_gpu_scheduler *sched = &q->guc->sched;
1488	1526
1489		− if (!exec_queue_killed_or_banned(q)) {
	1527	+ if (!exec_queue_killed_or_banned_or_wedged(q)) {
1490	1528	int i;
1491	1529
1492	1530	trace_xe_exec_queue_resubmit(q);
1493	1531	for (i = 0; i < q->width; ++i)
1494		− xe_lrc_set_ring_head(q->lrc + i, q->lrc[i].ring.tail);
	1532	+ xe_lrc_set_ring_head(q->lrc[i], q->lrc[i]->ring.tail);
1495	1533	xe_sched_resubmit_jobs(sched);
1496	1534	}
1497	1535
@@ -1643,6 +1681,7 @@ int xe_guc_deregister_done_handler(struct xe_guc guc, u32 msg, u32 len) · 145 unmodified lines
1643	1681
1644	1682	int xe_guc_exec_queue_reset_handler(struct xe_guc guc, u32 msg, u32 len)
1645	1683	{
	1684	+ struct xe_gt *gt = guc_to_gt(guc);
1646	1685	struct xe_device *xe = guc_to_xe(guc);
1647	1686	struct xe_exec_queue *q;
1648	1687	u32 guc_id = msg[0];
@@ -1656,7 +1695,8 @@ int xe_guc_exec_queue_reset_handler(struct xe_guc guc, u32 msg, u32 len) · 7 unmodified lines
1656	1695	if (unlikely(!q))
1657	1696	return -EPROTO;
1658	1697
1659		− drm_info(&xe->drm, "Engine reset: guc_id=%d", guc_id);
	1698	+ xe_gt_info(gt, "Engine reset: engine_class=%s, logical_mask: 0x%x, guc_id=%d",
	1699	+ xe_hw_engine_class_to_str(q->class), q->logical_mask, guc_id);
1660	1700
1661	1701	/* FIXME: Do error capture, most likely async */
1662	1702
@@ -1678,6 +1718,7 @@ int xe_guc_exec_queue_reset_handler(struct xe_guc guc, u32 msg, u32 len) · 15 unmodified lines
1678	1718	int xe_guc_exec_queue_memory_cat_error_handler(struct xe_guc guc, u32 msg,
1679	1719	u32 len)
1680	1720	{
	1721	+ struct xe_gt *gt = guc_to_gt(guc);
1681	1722	struct xe_device *xe = guc_to_xe(guc);
1682	1723	struct xe_exec_queue *q;
1683	1724	u32 guc_id = msg[0];
@@ -1691,7 +1732,9 @@ int xe_guc_exec_queue_memory_cat_error_handler(struct xe_guc guc, u32 msg, · 7 unmodified lines
1691	1732	if (unlikely(!q))
1692	1733	return -EPROTO;
1693	1734
1694		− drm_dbg(&xe->drm, "Engine memory cat error: guc_id=%d", guc_id);
	1735	+ xe_gt_dbg(gt, "Engine memory cat error: engine_class=%s, logical_mask: 0x%x, guc_id=%d",
	1736	+ xe_hw_engine_class_to_str(q->class), q->logical_mask, guc_id);
	1737	+
1695	1738	trace_xe_exec_queue_memory_cat_error(q);
1696	1739
1697	1740	/* Treat the same as engine reset */
@@ -1732,7 +1775,7 @@ guc_exec_queue_wq_snapshot_capture(struct xe_exec_queue *q, · 34 unmodified lines
1732	1775	{
1733	1776	struct xe_guc *guc = exec_queue_to_guc(q);
1734	1777	struct xe_device *xe = guc_to_xe(guc);
1735		− struct iosys_map map = xe_lrc_parallel_map(q->lrc);
	1778	+ struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
1736	1779	int i;
1737	1780
1738	1781	snapshot->guc.wqi_head = q->guc->wqi_head;
@@ -1812,7 +1855,7 @@ xe_guc_exec_queue_snapshot_capture(struct xe_exec_queue *q) · 73 unmodified lines
1812	1855
1813	1856	if (snapshot->lrc) {
1814	1857	for (i = 0; i < q->width; ++i) {
1815		− struct xe_lrc *lrc = q->lrc + i;
	1858	+ struct xe_lrc *lrc = q->lrc[i];
1816	1859
1817	1860	snapshot->lrc[i] = xe_lrc_snapshot_capture(lrc);
1818	1861	}

drivers/gpu/drm/xe/xe_guc_submit.h+2−2

@@ -12,11 +12,11 @@ struct drm_printer; · 11 unmodified lines
12	12	struct xe_exec_queue;
13	13	struct xe_guc;
14	14
15		−int xe_guc_submit_init(struct xe_guc *guc);
	15	+int xe_guc_submit_init(struct xe_guc *guc, unsigned int num_ids);
16	16
17	17	int xe_guc_submit_reset_prepare(struct xe_guc *guc);
18	18	void xe_guc_submit_reset_wait(struct xe_guc *guc);
19		−int xe_guc_submit_stop(struct xe_guc *guc);
	19	+void xe_guc_submit_stop(struct xe_guc *guc);
20	20	int xe_guc_submit_start(struct xe_guc *guc);
21	21
22	22	int xe_guc_sched_done_handler(struct xe_guc guc, u32 msg, u32 len);

drivers/gpu/drm/xe/xe_guc_types.h+0−9

@@ -72,15 +72,6 @@ struct xe_guc { · 71 unmodified lines
72	72	atomic_t stopped;
73	73	/** @submission_state.lock: protects submission state */
74	74	struct mutex lock;
75		− /** @submission_state.suspend: suspend fence state */
76		− struct {
77		− /** @submission_state.suspend.lock: suspend fences lock */
78		− spinlock_t lock;
79		− /** @submission_state.suspend.context: suspend fences context */
80		− u64 context;
81		− /** @submission_state.suspend.seqno: suspend fences seqno */
82		− u32 seqno;
83		− } suspend;
84	75	#ifdef CONFIG_PROVE_LOCKING
85	76	#define NUM_SUBMIT_WQ 256
86	77	/** @submission_state.submit_wq_pool: submission ordered workqueues pool */

drivers/gpu/drm/xe/xe_huc.c+2−0

@@ -5,6 +5,8 @@ · 4 unmodified lines
5	5
6	6	#include "xe_huc.h"
7	7
	8	+#include <linux/delay.h>
	9	+
8	10	#include <drm/drm_managed.h>
9	11
10	12	#include "abi/gsc_pxp_commands_abi.h"

drivers/gpu/drm/xe/xe_huc.h+2−1

@@ -6,9 +6,10 @@ · 5 unmodified lines
6	6	#ifndef _XE_HUC_H_
7	7	#define _XE_HUC_H_
8	8
9		−#include "xe_huc_types.h"
	9	+#include <linux/types.h>
10	10
11	11	struct drm_printer;
	12	+struct xe_huc;
12	13
13	14	enum xe_huc_auth_types {
14	15	XE_HUC_AUTH_VIA_GUC = 0,

drivers/gpu/drm/xe/xe_hw_engine.c+183−10

@@ -18,6 +18,7 @@ · 17 unmodified lines
18	18	#include "xe_gt.h"
19	19	#include "xe_gt_ccs_mode.h"
20	20	#include "xe_gt_printk.h"
	21	+#include "xe_gt_mcr.h"
21	22	#include "xe_gt_topology.h"
22	23	#include "xe_hw_fence.h"
23	24	#include "xe_irq.h"
@@ -267,7 +268,7 @@ static void hw_engine_fini(struct drm_device drm, void arg) · 243 unmodified lines
267	268
268	269	if (hwe->exl_port)
269	270	xe_execlist_port_destroy(hwe->exl_port);
270		− xe_lrc_finish(&hwe->kernel_lrc);
	271	+ xe_lrc_put(hwe->kernel_lrc);
271	272
272	273	hwe->gt = NULL;
273	274	}
@@ -341,7 +342,7 @@ xe_hw_engine_setup_default_lrc_state(struct xe_hw_engine *hwe) · 67 unmodified lines
341	342	u32 blit_cctl_val = REG_FIELD_PREP(BLIT_CCTL_DST_MOCS_MASK, mocs_write_idx) \|
342	343	REG_FIELD_PREP(BLIT_CCTL_SRC_MOCS_MASK, mocs_read_idx);
343	344	struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe);
344		− const struct xe_rtp_entry_sr lrc_was[] = {
	345	+ const struct xe_rtp_entry_sr lrc_setup[] = {
345	346	/*
346	347	* Some blitter commands do not have a field for MOCS, those
347	348	* commands will use MOCS index pointed by BLIT_CCTL.
@@ -373,7 +374,7 @@ xe_hw_engine_setup_default_lrc_state(struct xe_hw_engine *hwe) · 25 unmodified lines
373	374	{}
374	375	};
375	376
376		− xe_rtp_process_to_sr(&ctx, lrc_was, &hwe->reg_lrc);
	377	+ xe_rtp_process_to_sr(&ctx, lrc_setup, &hwe->reg_lrc);
377	378	}
378	379
379	380	static void
@@ -526,9 +527,11 @@ static int hw_engine_init(struct xe_gt gt, struct xe_hw_engine hwe, · 146 unmodified lines
526	527	goto err_name;
527	528	}
528	529
529		− err = xe_lrc_init(&hwe->kernel_lrc, hwe, NULL, NULL, SZ_16K);
530		− if (err)
	530	+ hwe->kernel_lrc = xe_lrc_create(hwe, NULL, SZ_16K);
	531	+ if (IS_ERR(hwe->kernel_lrc)) {
	532	+ err = PTR_ERR(hwe->kernel_lrc);
531	533	goto err_hwsp;
	534	+ }
532	535
533	536	if (!xe_device_uc_enabled(xe)) {
534	537	hwe->exl_port = xe_execlist_port_create(xe, hwe);
@@ -553,7 +556,7 @@ static int hw_engine_init(struct xe_gt gt, struct xe_hw_engine hwe, · 18 unmodified lines
553	556	return drmm_add_action_or_reset(&xe->drm, hw_engine_fini, hwe);
554	557
555	558	err_kernel_lrc:
556		− xe_lrc_finish(&hwe->kernel_lrc);
	559	+ xe_lrc_put(hwe->kernel_lrc);
557	560	err_hwsp:
558	561	xe_bo_unpin_map_no_vm(hwe->hwsp);
559	562	err_name:
@@ -716,6 +719,11 @@ static void check_gsc_availability(struct xe_gt *gt) · 156 unmodified lines
716	719	*/
717	720	if (!xe_uc_fw_is_available(&gt->uc.gsc.fw)) {
718	721	gt->info.engine_mask &= ~BIT(XE_HW_ENGINE_GSCCS0);
	722	+
	723	+ /* interrupts where previously enabled, so turn them off */
	724	+ xe_mmio_write32(gt, GUNIT_GSC_INTR_ENABLE, 0);
	725	+ xe_mmio_write32(gt, GUNIT_GSC_INTR_MASK, ~0);
	726	+
719	727	drm_info(&xe->drm, "gsccs disabled due to lack of FW\n");
720	728	}
721	729	}
@@ -766,6 +774,57 @@ void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec) · 44 unmodified lines
766	774	xe_hw_fence_irq_run(hwe->fence_irq);
767	775	}
768	776
	777	+static bool
	778	+is_slice_common_per_gslice(struct xe_device *xe)
	779	+{
	780	+ return GRAPHICS_VERx100(xe) >= 1255;
	781	+}
	782	+
	783	+static void
	784	+xe_hw_engine_snapshot_instdone_capture(struct xe_hw_engine *hwe,
	785	+ struct xe_hw_engine_snapshot *snapshot)
	786	+{
	787	+ struct xe_gt *gt = hwe->gt;
	788	+ struct xe_device *xe = gt_to_xe(gt);
	789	+ unsigned int dss;
	790	+ u16 group, instance;
	791	+
	792	+ snapshot->reg.instdone.ring = hw_engine_mmio_read32(hwe, RING_INSTDONE(0));
	793	+
	794	+ if (snapshot->hwe->class != XE_ENGINE_CLASS_RENDER)
	795	+ return;
	796	+
	797	+ if (is_slice_common_per_gslice(xe) == false) {
	798	+ snapshot->reg.instdone.slice_common[0] =
	799	+ xe_mmio_read32(gt, SC_INSTDONE);
	800	+ snapshot->reg.instdone.slice_common_extra[0] =
	801	+ xe_mmio_read32(gt, SC_INSTDONE_EXTRA);
	802	+ snapshot->reg.instdone.slice_common_extra2[0] =
	803	+ xe_mmio_read32(gt, SC_INSTDONE_EXTRA2);
	804	+ } else {
	805	+ for_each_geometry_dss(dss, gt, group, instance) {
	806	+ snapshot->reg.instdone.slice_common[dss] =
	807	+ xe_gt_mcr_unicast_read(gt, XEHPG_SC_INSTDONE, group, instance);
	808	+ snapshot->reg.instdone.slice_common_extra[dss] =
	809	+ xe_gt_mcr_unicast_read(gt, XEHPG_SC_INSTDONE_EXTRA, group, instance);
	810	+ snapshot->reg.instdone.slice_common_extra2[dss] =
	811	+ xe_gt_mcr_unicast_read(gt, XEHPG_SC_INSTDONE_EXTRA2, group, instance);
	812	+ }
	813	+ }
	814	+
	815	+ for_each_geometry_dss(dss, gt, group, instance) {
	816	+ snapshot->reg.instdone.sampler[dss] =
	817	+ xe_gt_mcr_unicast_read(gt, SAMPLER_INSTDONE, group, instance);
	818	+ snapshot->reg.instdone.row[dss] =
	819	+ xe_gt_mcr_unicast_read(gt, ROW_INSTDONE, group, instance);
	820	+
	821	+ if (GRAPHICS_VERx100(xe) >= 1255)
	822	+ snapshot->reg.instdone.geom_svg[dss] =
	823	+ xe_gt_mcr_unicast_read(gt, XEHPG_INSTDONE_GEOM_SVGUNIT,
	824	+ group, instance);
	825	+ }
	826	+}
	827	+
769	828	/**
770	829	* xe_hw_engine_snapshot_capture - Take a quick snapshot of the HW Engine.
771	830	* @hwe: Xe HW Engine.
@@ -780,6 +839,7 @@ struct xe_hw_engine_snapshot * · 8 unmodified lines
780	839	xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe)
781	840	{
782	841	struct xe_hw_engine_snapshot *snapshot;
	842	+ size_t len;
783	843	u64 val;
784	844
785	845	if (!xe_hw_engine_is_valid(hwe))
@@ -790,8 +850,30 @@ xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe) · 4 unmodified lines
790	850	if (!snapshot)
791	851	return NULL;
792	852
	853	+ /* Because XE_MAX_DSS_FUSE_BITS is defined in xe_gt_types.h and it
	854	+ * includes xe_hw_engine_types.h the length of this 3 registers can't be
	855	+ * set in struct xe_hw_engine_snapshot, so here doing additional
	856	+ * allocations.
	857	+ */
	858	+ len = (XE_MAX_DSS_FUSE_BITS * sizeof(u32));
	859	+ snapshot->reg.instdone.slice_common = kzalloc(len, GFP_ATOMIC);
	860	+ snapshot->reg.instdone.slice_common_extra = kzalloc(len, GFP_ATOMIC);
	861	+ snapshot->reg.instdone.slice_common_extra2 = kzalloc(len, GFP_ATOMIC);
	862	+ snapshot->reg.instdone.sampler = kzalloc(len, GFP_ATOMIC);
	863	+ snapshot->reg.instdone.row = kzalloc(len, GFP_ATOMIC);
	864	+ snapshot->reg.instdone.geom_svg = kzalloc(len, GFP_ATOMIC);
	865	+ if (!snapshot->reg.instdone.slice_common \|\|
	866	+ !snapshot->reg.instdone.slice_common_extra \|\|
	867	+ !snapshot->reg.instdone.slice_common_extra2 \|\|
	868	+ !snapshot->reg.instdone.sampler \|\|
	869	+ !snapshot->reg.instdone.row \|\|
	870	+ !snapshot->reg.instdone.geom_svg) {
	871	+ xe_hw_engine_snapshot_free(snapshot);
	872	+ return NULL;
	873	+ }
	874	+
793	875	snapshot->name = kstrdup(hwe->name, GFP_ATOMIC);
794		− snapshot->class = hwe->class;
	876	+ snapshot->hwe = hwe;
795	877	snapshot->logical_instance = hwe->logical_instance;
796	878	snapshot->forcewake.domain = hwe->domain;
797	879	snapshot->forcewake.ref = xe_force_wake_ref(gt_to_fw(hwe->gt),
@@ -828,6 +910,15 @@ xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe) · 30 unmodified lines
828	910	snapshot->reg.ring_hwstam = hw_engine_mmio_read32(hwe, RING_HWSTAM(0));
829	911	snapshot->reg.ring_hws_pga = hw_engine_mmio_read32(hwe, RING_HWS_PGA(0));
830	912	snapshot->reg.ring_start = hw_engine_mmio_read32(hwe, RING_START(0));
	913	+ if (GRAPHICS_VERx100(hwe->gt->tile->xe) >= 2000) {
	914	+ val = hw_engine_mmio_read32(hwe, RING_START_UDW(0));
	915	+ snapshot->reg.ring_start \|= val << 32;
	916	+ }
	917	+ if (xe_gt_has_indirect_ring_state(hwe->gt)) {
	918	+ snapshot->reg.indirect_ring_state =
	919	+ hw_engine_mmio_read32(hwe, INDIRECT_RING_STATE(0));
	920	+ }
	921	+
831	922	snapshot->reg.ring_head =
832	923	hw_engine_mmio_read32(hwe, RING_HEAD(0)) & HEAD_ADDR;
833	924	snapshot->reg.ring_tail =
@@ -841,13 +932,57 @@ xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe) · 7 unmodified lines
841	932	snapshot->reg.ring_emr = hw_engine_mmio_read32(hwe, RING_EMR(0));
842	933	snapshot->reg.ring_eir = hw_engine_mmio_read32(hwe, RING_EIR(0));
843	934	snapshot->reg.ipehr = hw_engine_mmio_read32(hwe, RING_IPEHR(0));
	935	+ xe_hw_engine_snapshot_instdone_capture(hwe, snapshot);
844	936
845		− if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)
	937	+ if (snapshot->hwe->class == XE_ENGINE_CLASS_COMPUTE)
846	938	snapshot->reg.rcu_mode = xe_mmio_read32(hwe->gt, RCU_MODE);
847	939
848	940	return snapshot;
849	941	}
850	942
	943	+static void
	944	+xe_hw_engine_snapshot_instdone_print(struct xe_hw_engine_snapshot snapshot, struct drm_printer p)
	945	+{
	946	+ struct xe_gt *gt = snapshot->hwe->gt;
	947	+ struct xe_device *xe = gt_to_xe(gt);
	948	+ u16 group, instance;
	949	+ unsigned int dss;
	950	+
	951	+ drm_printf(p, "\tRING_INSTDONE: 0x%08x\n", snapshot->reg.instdone.ring);
	952	+
	953	+ if (snapshot->hwe->class != XE_ENGINE_CLASS_RENDER)
	954	+ return;
	955	+
	956	+ if (is_slice_common_per_gslice(xe) == false) {
	957	+ drm_printf(p, "\tSC_INSTDONE[0]: 0x%08x\n",
	958	+ snapshot->reg.instdone.slice_common[0]);
	959	+ drm_printf(p, "\tSC_INSTDONE_EXTRA[0]: 0x%08x\n",
	960	+ snapshot->reg.instdone.slice_common_extra[0]);
	961	+ drm_printf(p, "\tSC_INSTDONE_EXTRA2[0]: 0x%08x\n",
	962	+ snapshot->reg.instdone.slice_common_extra2[0]);
	963	+ } else {
	964	+ for_each_geometry_dss(dss, gt, group, instance) {
	965	+ drm_printf(p, "\tSC_INSTDONE[%u]: 0x%08x\n", dss,
	966	+ snapshot->reg.instdone.slice_common[dss]);
	967	+ drm_printf(p, "\tSC_INSTDONE_EXTRA[%u]: 0x%08x\n", dss,
	968	+ snapshot->reg.instdone.slice_common_extra[dss]);
	969	+ drm_printf(p, "\tSC_INSTDONE_EXTRA2[%u]: 0x%08x\n", dss,
	970	+ snapshot->reg.instdone.slice_common_extra2[dss]);
	971	+ }
	972	+ }
	973	+
	974	+ for_each_geometry_dss(dss, gt, group, instance) {
	975	+ drm_printf(p, "\tSAMPLER_INSTDONE[%u]: 0x%08x\n", dss,
	976	+ snapshot->reg.instdone.sampler[dss]);
	977	+ drm_printf(p, "\tROW_INSTDONE[%u]: 0x%08x\n", dss,
	978	+ snapshot->reg.instdone.row[dss]);
	979	+
	980	+ if (GRAPHICS_VERx100(xe) >= 1255)
	981	+ drm_printf(p, "\tINSTDONE_GEOM_SVGUNIT[%u]: 0x%08x\n",
	982	+ dss, snapshot->reg.instdone.geom_svg[dss]);
	983	+ }
	984	+}
	985	+
851	986	/**
852	987	* xe_hw_engine_snapshot_print - Print out a given Xe HW Engine snapshot.
853	988	* @snapshot: Xe HW Engine snapshot object.
@@ -872,7 +1007,7 @@ void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot, · 18 unmodified lines
872	1007	snapshot->reg.ring_execlist_status);
873	1008	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS: 0x%016llx\n",
874	1009	snapshot->reg.ring_execlist_sq_contents);
875		− drm_printf(p, "\tRING_START: 0x%08x\n", snapshot->reg.ring_start);
	1010	+ drm_printf(p, "\tRING_START: 0x%016llx\n", snapshot->reg.ring_start);
876	1011	drm_printf(p, "\tRING_HEAD: 0x%08x\n", snapshot->reg.ring_head);
877	1012	drm_printf(p, "\tRING_TAIL: 0x%08x\n", snapshot->reg.ring_tail);
878	1013	drm_printf(p, "\tRING_CTL: 0x%08x\n", snapshot->reg.ring_ctl);
@@ -886,10 +1021,15 @@ void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot, · 7 unmodified lines
886	1021	drm_printf(p, "\tACTHD: 0x%016llx\n", snapshot->reg.ring_acthd);
887	1022	drm_printf(p, "\tBBADDR: 0x%016llx\n", snapshot->reg.ring_bbaddr);
888	1023	drm_printf(p, "\tDMA_FADDR: 0x%016llx\n", snapshot->reg.ring_dma_fadd);
	1024	+ drm_printf(p, "\tINDIRECT_RING_STATE: 0x%08x\n",
	1025	+ snapshot->reg.indirect_ring_state);
889	1026	drm_printf(p, "\tIPEHR: 0x%08x\n", snapshot->reg.ipehr);
890		− if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)
	1027	+ xe_hw_engine_snapshot_instdone_print(snapshot, p);
	1028	+
	1029	+ if (snapshot->hwe->class == XE_ENGINE_CLASS_COMPUTE)
891	1030	drm_printf(p, "\tRCU_MODE: 0x%08x\n",
892	1031	snapshot->reg.rcu_mode);
	1032	+ drm_puts(p, "\n");
893	1033	}
894	1034
895	1035	/**
@@ -904,6 +1044,12 @@ void xe_hw_engine_snapshot_free(struct xe_hw_engine_snapshot *snapshot) · 8 unmodified lines
904	1044	if (!snapshot)
905	1045	return;
906	1046
	1047	+ kfree(snapshot->reg.instdone.slice_common);
	1048	+ kfree(snapshot->reg.instdone.slice_common_extra);
	1049	+ kfree(snapshot->reg.instdone.slice_common_extra2);
	1050	+ kfree(snapshot->reg.instdone.sampler);
	1051	+ kfree(snapshot->reg.instdone.row);
	1052	+ kfree(snapshot->reg.instdone.geom_svg);
907	1053	kfree(snapshot->name);
908	1054	kfree(snapshot);
909	1055	}
@@ -955,3 +1101,30 @@ bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe) · 45 unmodified lines
955	1101	return xe->info.has_usm && hwe->class == XE_ENGINE_CLASS_COPY &&
956	1102	hwe->instance == gt->usm.reserved_bcs_instance;
957	1103	}
	1104	+
	1105	+const char *xe_hw_engine_class_to_str(enum xe_engine_class class)
	1106	+{
	1107	+ switch (class) {
	1108	+ case XE_ENGINE_CLASS_RENDER:
	1109	+ return "rcs";
	1110	+ case XE_ENGINE_CLASS_VIDEO_DECODE:
	1111	+ return "vcs";
	1112	+ case XE_ENGINE_CLASS_VIDEO_ENHANCE:
	1113	+ return "vecs";
	1114	+ case XE_ENGINE_CLASS_COPY:
	1115	+ return "bcs";
	1116	+ case XE_ENGINE_CLASS_OTHER:
	1117	+ return "other";
	1118	+ case XE_ENGINE_CLASS_COMPUTE:
	1119	+ return "ccs";
	1120	+ case XE_ENGINE_CLASS_MAX:
	1121	+ break;
	1122	+ }
	1123	+
	1124	+ return NULL;
	1125	+}
	1126	+
	1127	+u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe)
	1128	+{
	1129	+ return xe_mmio_read64_2x32(hwe->gt, RING_TIMESTAMP(hwe->mmio_base));
	1130	+}

drivers/gpu/drm/xe/xe_hw_engine.h+3−0

@@ -67,4 +67,7 @@ static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe) · 66 unmodified lines
67	67	return hwe->name;
68	68	}
69	69
	70	+const char *xe_hw_engine_class_to_str(enum xe_engine_class class);
	71	+u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe);
	72	+
70	73	#endif

drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c+6−24

@@ -492,7 +492,7 @@ static const struct attribute * const files[] = { · 491 unmodified lines
492	492	NULL
493	493	};
494	494
495		−static void kobj_xe_hw_engine_class_fini(struct drm_device drm, void arg)
	495	+static void kobj_xe_hw_engine_class_fini(void *arg)
496	496	{
497	497	struct kobject *kobj = arg;
498	498
@@ -517,7 +517,7 @@ kobj_xe_hw_engine_class(struct xe_device xe, struct kobject parent, const char · 18 unmodified lines
517	517	}
518	518	keclass->xe = xe;
519	519
520		− err = drmm_add_action_or_reset(&xe->drm, kobj_xe_hw_engine_class_fini,
	520	+ err = devm_add_action_or_reset(xe->drm.dev, kobj_xe_hw_engine_class_fini,
521	521	&keclass->base);
522	522	if (err)
523	523	return NULL;
@@ -525,7 +525,7 @@ kobj_xe_hw_engine_class(struct xe_device xe, struct kobject parent, const char · 1 unmodified line
525	525	return keclass;
526	526	}
527	527
528		−static void hw_engine_class_defaults_fini(struct drm_device drm, void arg)
	528	+static void hw_engine_class_defaults_fini(void *arg)
529	529	{
530	530	struct kobject *kobj = arg;
531	531
@@ -552,7 +552,7 @@ static int xe_add_hw_engine_class_defaults(struct xe_device *xe, · 20 unmodified lines
552	552	if (err)
553	553	goto err_object;
554	554
555		− return drmm_add_action_or_reset(&xe->drm, hw_engine_class_defaults_fini, kobj);
	555	+ return devm_add_action_or_reset(xe->drm.dev, hw_engine_class_defaults_fini, kobj);
556	556
557	557	err_object:
558	558	kobject_put(kobj);
@@ -611,31 +611,13 @@ static const struct kobj_type xe_hw_engine_sysfs_kobj_type = { · 52 unmodified lines
611	611	.sysfs_ops = &xe_hw_engine_class_sysfs_ops,
612	612	};
613	613
614		−static void hw_engine_class_sysfs_fini(struct drm_device drm, void arg)
	614	+static void hw_engine_class_sysfs_fini(void *arg)
615	615	{
616	616	struct kobject *kobj = arg;
617	617
618	618	kobject_put(kobj);
619	619	}
620	620
621		−static const char *xe_hw_engine_class_to_str(enum xe_engine_class class)
622		−{
623		− switch (class) {
624		− case XE_ENGINE_CLASS_RENDER:
625		− return "rcs";
626		− case XE_ENGINE_CLASS_VIDEO_DECODE:
627		− return "vcs";
628		− case XE_ENGINE_CLASS_VIDEO_ENHANCE:
629		− return "vecs";
630		− case XE_ENGINE_CLASS_COPY:
631		− return "bcs";
632		− case XE_ENGINE_CLASS_COMPUTE:
633		− return "ccs";
634		− default:
635		− return NULL;
636		− }
637		−}
638		−
639	621	/**
640	622	* xe_hw_engine_class_sysfs_init - Init HW engine classes on GT.
641	623	* @gt: Xe GT.
@@ -698,7 +680,7 @@ int xe_hw_engine_class_sysfs_init(struct xe_gt *gt) · 56 unmodified lines
698	680	goto err_object;
699	681	}
700	682
701		− return drmm_add_action_or_reset(&xe->drm, hw_engine_class_sysfs_fini, kobj);
	683	+ return devm_add_action_or_reset(xe->drm.dev, hw_engine_class_sysfs_fini, kobj);
702	684
703	685	err_object:
704	686	kobject_put(kobj);

drivers/gpu/drm/xe/xe_hw_engine_types.h+22−4

@@ -137,7 +137,7 @@ struct xe_hw_engine { · 136 unmodified lines
137	137	/** @hwsp: hardware status page buffer object */
138	138	struct xe_bo *hwsp;
139	139	/** @kernel_lrc: Kernel LRC (should be replaced /w an xe_engine) */
140		− struct xe_lrc kernel_lrc;
	140	+ struct xe_lrc *kernel_lrc;
141	141	/** @exl_port: execlists port */
142	142	struct xe_execlist_port *exl_port;
143	143	/** @fence_irq: fence IRQ to run when a hw engine IRQ is received */
@@ -158,8 +158,8 @@ struct xe_hw_engine { · 14 unmodified lines
158	158	struct xe_hw_engine_snapshot {
159	159	/** @name: name of the hw engine */
160	160	char *name;
161		− /** @class: class of this hw engine */
162		− enum xe_engine_class class;
	161	+ /** @hwe: hw engine */
	162	+ struct xe_hw_engine *hwe;
163	163	/** @logical_instance: logical instance of this hw engine */
164	164	u16 logical_instance;
165	165	/** @forcewake: Force Wake information snapshot */
@@ -188,7 +188,7 @@ struct xe_hw_engine_snapshot { · 22 unmodified lines
188	188	/** @reg.ring_hws_pga: RING_HWS_PGA */
189	189	u32 ring_hws_pga;
190	190	/** @reg.ring_start: RING_START */
191		− u32 ring_start;
	191	+ u64 ring_start;
192	192	/** @reg.ring_head: RING_HEAD */
193	193	u32 ring_head;
194	194	/** @reg.ring_tail: RING_TAIL */
@@ -207,10 +207,28 @@ struct xe_hw_engine_snapshot { · 12 unmodified lines
207	207	u32 ring_emr;
208	208	/** @reg.ring_eir: RING_EIR */
209	209	u32 ring_eir;
	210	+ /** @reg.indirect_ring_state: INDIRECT_RING_STATE */
	211	+ u32 indirect_ring_state;
210	212	/** @reg.ipehr: IPEHR */
211	213	u32 ipehr;
212	214	/** @reg.rcu_mode: RCU_MODE */
213	215	u32 rcu_mode;
	216	+ struct {
	217	+ /** @reg.instdone.ring: RING_INSTDONE */
	218	+ u32 ring;
	219	+ /** @reg.instdone.slice_common: SC_INSTDONE */
	220	+ u32 *slice_common;
	221	+ /** @reg.instdone.slice_common_extra: SC_INSTDONE_EXTRA */
	222	+ u32 *slice_common_extra;
	223	+ /** @reg.instdone.slice_common_extra2: SC_INSTDONE_EXTRA2 */
	224	+ u32 *slice_common_extra2;
	225	+ /** @reg.instdone.sampler: SAMPLER_INSTDONE */
	226	+ u32 *sampler;
	227	+ /** @reg.instdone.row: ROW_INSTDONE */
	228	+ u32 *row;
	229	+ /** @reg.instdone.geom_svg: INSTDONE_GEOM_SVGUNIT */
	230	+ u32 *geom_svg;
	231	+ } instdone;
214	232	} reg;
215	233	};
216	234

drivers/gpu/drm/xe/xe_hw_fence.c+47−12

@@ -208,23 +208,58 @@ static struct xe_hw_fence to_xe_hw_fence(struct dma_fence fence) · 207 unmodified lines
208	208	return container_of(fence, struct xe_hw_fence, dma);
209	209	}
210	210
211		−struct xe_hw_fence xe_hw_fence_create(struct xe_hw_fence_ctx ctx,
212		− struct iosys_map seqno_map)
	211	+/**
	212	+ * xe_hw_fence_alloc() - Allocate an hw fence.
	213	+ *
	214	+ * Allocate but don't initialize an hw fence.
	215	+ *
	216	+ * Return: Pointer to the allocated fence or
	217	+ * negative error pointer on error.
	218	+ */
	219	+struct dma_fence *xe_hw_fence_alloc(void)
213	220	{
214		− struct xe_hw_fence *fence;
	221	+ struct xe_hw_fence *hw_fence = fence_alloc();
215	222
216		− fence = fence_alloc();
217		− if (!fence)
	223	+ if (!hw_fence)
218	224	return ERR_PTR(-ENOMEM);
219	225
220		− fence->ctx = ctx;
221		− fence->seqno_map = seqno_map;
222		− INIT_LIST_HEAD(&fence->irq_link);
	226	+ return &hw_fence->dma;
	227	+}
223	228
224		− dma_fence_init(&fence->dma, &xe_hw_fence_ops, &ctx->irq->lock,
225		− ctx->dma_fence_ctx, ctx->next_seqno++);
	229	+/**
	230	+ * xe_hw_fence_free() - Free an hw fence.
	231	+ * @fence: Pointer to the fence to free.
	232	+ *
	233	+ * Frees an hw fence that hasn't yet been
	234	+ * initialized.
	235	+ */
	236	+void xe_hw_fence_free(struct dma_fence *fence)
	237	+{
	238	+ fence_free(&fence->rcu);
	239	+}
226	240
227		− trace_xe_hw_fence_create(fence);
	241	+/**
	242	+ * xe_hw_fence_init() - Initialize an hw fence.
	243	+ * @fence: Pointer to the fence to initialize.
	244	+ * @ctx: Pointer to the struct xe_hw_fence_ctx fence context.
	245	+ * @seqno_map: Pointer to the map into where the seqno is blitted.
	246	+ *
	247	+ * Initializes a pre-allocated hw fence.
	248	+ * After initialization, the fence is subject to normal
	249	+ * dma-fence refcounting.
	250	+ */
	251	+void xe_hw_fence_init(struct dma_fence fence, struct xe_hw_fence_ctx ctx,
	252	+ struct iosys_map seqno_map)
	253	+{
	254	+ struct xe_hw_fence *hw_fence =
	255	+ container_of(fence, typeof(*hw_fence), dma);
	256	+
	257	+ hw_fence->ctx = ctx;
	258	+ hw_fence->seqno_map = seqno_map;
	259	+ INIT_LIST_HEAD(&hw_fence->irq_link);
	260	+
	261	+ dma_fence_init(fence, &xe_hw_fence_ops, &ctx->irq->lock,
	262	+ ctx->dma_fence_ctx, ctx->next_seqno++);
228	263
229		− return fence;
	264	+ trace_xe_hw_fence_create(hw_fence);
230	265	}

drivers/gpu/drm/xe/xe_hw_fence.h+5−2

@@ -24,7 +24,10 @@ void xe_hw_fence_ctx_init(struct xe_hw_fence_ctx ctx, struct xe_gt gt, · 23 unmodified lines
24	24	struct xe_hw_fence_irq irq, const char name);
25	25	void xe_hw_fence_ctx_finish(struct xe_hw_fence_ctx *ctx);
26	26
27		−struct xe_hw_fence xe_hw_fence_create(struct xe_hw_fence_ctx ctx,
28		− struct iosys_map seqno_map);
	27	+struct dma_fence *xe_hw_fence_alloc(void);
29	28
	29	+void xe_hw_fence_free(struct dma_fence *fence);
	30	+
	31	+void xe_hw_fence_init(struct dma_fence fence, struct xe_hw_fence_ctx ctx,
	32	+ struct iosys_map seqno_map);
30	33	#endif

drivers/gpu/drm/xe/xe_hwmon.c+29−8

@@ -86,19 +86,29 @@ static struct xe_reg xe_hwmon_get_reg(struct xe_hwmon *hwmon, enum xe_hwmon_reg · 85 unmodified lines
86	86
87	87	switch (hwmon_reg) {
88	88	case REG_PKG_RAPL_LIMIT:
89		− if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG)
	89	+ if (xe->info.platform == XE_BATTLEMAGE) {
	90	+ if (channel == CHANNEL_PKG)
	91	+ return BMG_PACKAGE_RAPL_LIMIT;
	92	+ else
	93	+ return BMG_PLATFORM_POWER_LIMIT;
	94	+ } else if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG) {
90	95	return PVC_GT0_PACKAGE_RAPL_LIMIT;
91		− else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG))
	96	+ } else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG)) {
92	97	return PCU_CR_PACKAGE_RAPL_LIMIT;
	98	+ }
93	99	break;
94	100	case REG_PKG_POWER_SKU:
95		− if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG)
	101	+ if (xe->info.platform == XE_BATTLEMAGE)
	102	+ return BMG_PACKAGE_POWER_SKU;
	103	+ else if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG)
96	104	return PVC_GT0_PACKAGE_POWER_SKU;
97	105	else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG))
98	106	return PCU_CR_PACKAGE_POWER_SKU;
99	107	break;
100	108	case REG_PKG_POWER_SKU_UNIT:
101		− if (xe->info.platform == XE_PVC)
	109	+ if (xe->info.platform == XE_BATTLEMAGE)
	110	+ return BMG_PACKAGE_POWER_SKU_UNIT;
	111	+ else if (xe->info.platform == XE_PVC)
102	112	return PVC_GT0_PACKAGE_POWER_SKU_UNIT;
103	113	else if (xe->info.platform == XE_DG2)
104	114	return PCU_CR_PACKAGE_POWER_SKU_UNIT;
@@ -108,10 +118,16 @@ static struct xe_reg xe_hwmon_get_reg(struct xe_hwmon *hwmon, enum xe_hwmon_reg · 3 unmodified lines
108	118	return GT_PERF_STATUS;
109	119	break;
110	120	case REG_PKG_ENERGY_STATUS:
111		− if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG)
	121	+ if (xe->info.platform == XE_BATTLEMAGE) {
	122	+ if (channel == CHANNEL_PKG)
	123	+ return BMG_PACKAGE_ENERGY_STATUS;
	124	+ else
	125	+ return BMG_PLATFORM_ENERGY_STATUS;
	126	+ } else if (xe->info.platform == XE_PVC && channel == CHANNEL_PKG) {
112	127	return PVC_GT0_PLATFORM_ENERGY_STATUS;
113		− else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG))
	128	+ } else if ((xe->info.platform == XE_DG2) && (channel == CHANNEL_PKG)) {
114	129	return PCU_CR_PACKAGE_ENERGY_STATUS;
	130	+ }
115	131	break;
116	132	default:
117	133	drm_warn(&xe->drm, "Unknown xe hwmon reg id: %d\n", hwmon_reg);
@@ -550,12 +566,17 @@ xe_hwmon_curr_is_visible(const struct xe_hwmon *hwmon, u32 attr, int channel) · 432 unmodified lines
550	566	{
551	567	u32 uval;
552	568
	569	+ /* hwmon sysfs attribute of current available only for package */
	570	+ if (channel != CHANNEL_PKG)
	571	+ return 0;
	572	+
553	573	switch (attr) {
554	574	case hwmon_curr_crit:
555		− case hwmon_curr_label:
556		− if (channel == CHANNEL_PKG)
557	575	return (xe_hwmon_pcode_read_i1(hwmon->gt, &uval) \|\|
558	576	(uval & POWER_SETUP_I1_WATTS)) ? 0 : 0644;
	577	+ case hwmon_curr_label:
	578	+ return (xe_hwmon_pcode_read_i1(hwmon->gt, &uval) \|\|
	579	+ (uval & POWER_SETUP_I1_WATTS)) ? 0 : 0444;
559	580	break;
560	581	default:
561	582	return 0;

drivers/gpu/drm/xe/xe_irq.c+3−3

@@ -663,7 +663,7 @@ static irq_handler_t xe_irq_handler(struct xe_device *xe) · 662 unmodified lines
663	663	return xelp_irq_handler;
664	664	}
665	665
666		−static void irq_uninstall(struct drm_device drm, void arg)
	666	+static void irq_uninstall(void *arg)
667	667	{
668	668	struct xe_device *xe = arg;
669	669	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
@@ -723,7 +723,7 @@ int xe_irq_install(struct xe_device *xe) · 53 unmodified lines
723	723
724	724	xe_irq_postinstall(xe);
725	725
726		− err = drmm_add_action_or_reset(&xe->drm, irq_uninstall, xe);
	726	+ err = devm_add_action_or_reset(xe->drm.dev, irq_uninstall, xe);
727	727	if (err)
728	728	goto free_irq_handler;
729	729
@@ -737,7 +737,7 @@ free_irq_handler: · 7 unmodified lines
737	737
738	738	void xe_irq_shutdown(struct xe_device *xe)
739	739	{
740		− irq_uninstall(&xe->drm, xe);
	740	+ irq_uninstall(xe);
741	741	}
742	742
743	743	void xe_irq_suspend(struct xe_device *xe)

drivers/gpu/drm/xe/xe_lrc.c+264−41

@@ -11,7 +11,6 @@ · 10 unmodified lines
11	11	#include "instructions/xe_gfxpipe_commands.h"
12	12	#include "instructions/xe_gfx_state_commands.h"
13	13	#include "regs/xe_engine_regs.h"
14		−#include "regs/xe_gpu_commands.h"
15	14	#include "regs/xe_lrc_layout.h"
16	15	#include "xe_bb.h"
17	16	#include "xe_bo.h"
@@ -34,12 +33,15 @@ · 16 unmodified lines
34	33	#define LRC_ENGINE_CLASS GENMASK_ULL(63, 61)
35	34	#define LRC_ENGINE_INSTANCE GENMASK_ULL(53, 48)
36	35
	36	+#define LRC_INDIRECT_RING_STATE_SIZE SZ_4K
	37	+
37	38	struct xe_lrc_snapshot {
38	39	struct xe_bo *lrc_bo;
39	40	void *lrc_snapshot;
40	41	unsigned long lrc_size, lrc_offset;
41	42
42	43	u32 context_desc;
	44	+ u32 indirect_context_desc;
43	45	u32 head;
44	46	struct {
45	47	u32 internal;
@@ -55,20 +57,25 @@ lrc_to_xe(struct xe_lrc *lrc) · 9 unmodified lines
55	57	return gt_to_xe(lrc->fence_ctx.gt);
56	58	}
57	59
58		−size_t xe_lrc_size(struct xe_device *xe, enum xe_engine_class class)
	60	+size_t xe_gt_lrc_size(struct xe_gt *gt, enum xe_engine_class class)
59	61	{
	62	+ struct xe_device *xe = gt_to_xe(gt);
	63	+ size_t size;
	64	+
60	65	switch (class) {
61	66	case XE_ENGINE_CLASS_RENDER:
62	67	if (GRAPHICS_VER(xe) >= 20)
63		− return 4 * SZ_4K;
	68	+ size = 4 * SZ_4K;
64	69	else
65		− return 14 * SZ_4K;
	70	+ size = 14 * SZ_4K;
	71	+ break;
66	72	case XE_ENGINE_CLASS_COMPUTE:
67	73	/* 14 pages since graphics_ver == 11 */
68	74	if (GRAPHICS_VER(xe) >= 20)
69		− return 3 * SZ_4K;
	75	+ size = 3 * SZ_4K;
70	76	else
71		− return 14 * SZ_4K;
	77	+ size = 14 * SZ_4K;
	78	+ break;
72	79	default:
73	80	WARN(1, "Unknown engine class: %d", class);
74	81	fallthrough;
@@ -76,8 +83,14 @@ size_t xe_lrc_size(struct xe_device *xe, enum xe_engine_class class) · 1 unmodified line
76	83	case XE_ENGINE_CLASS_VIDEO_DECODE:
77	84	case XE_ENGINE_CLASS_VIDEO_ENHANCE:
78	85	case XE_ENGINE_CLASS_OTHER:
79		− return 2 * SZ_4K;
	86	+ size = 2 * SZ_4K;
80	87	}
	88	+
	89	+ /* Add indirect ring state page */
	90	+ if (xe_gt_has_indirect_ring_state(gt))
	91	+ size += LRC_INDIRECT_RING_STATE_SIZE;
	92	+
	93	+ return size;
81	94	}
82	95
83	96	/*
@@ -508,6 +521,32 @@ static const u8 xe2_xcs_offsets[] = { · 424 unmodified lines
508	521	0
509	522	};
510	523
	524	+static const u8 xe2_indirect_ring_state_offsets[] = {
	525	+ NOP(1), /* [0x00] */
	526	+ LRI(5, POSTED), /* [0x01] */
	527	+ REG(0x034), /* [0x02] RING_BUFFER_HEAD */
	528	+ REG(0x030), /* [0x04] RING_BUFFER_TAIL */
	529	+ REG(0x038), /* [0x06] RING_BUFFER_START */
	530	+ REG(0x048), /* [0x08] RING_BUFFER_START_UDW */
	531	+ REG(0x03c), /* [0x0a] RING_BUFFER_CONTROL */
	532	+
	533	+ NOP(5), /* [0x0c] */
	534	+ LRI(9, POSTED), /* [0x11] */
	535	+ REG(0x168), /* [0x12] BB_ADDR_UDW */
	536	+ REG(0x140), /* [0x14] BB_ADDR */
	537	+ REG(0x110), /* [0x16] BB_STATE */
	538	+ REG16(0x588), /* [0x18] BB_STACK_WRITE_PORT */
	539	+ REG16(0x588), /* [0x20] BB_STACK_WRITE_PORT */
	540	+ REG16(0x588), /* [0x22] BB_STACK_WRITE_PORT */
	541	+ REG16(0x588), /* [0x24] BB_STACK_WRITE_PORT */
	542	+ REG16(0x588), /* [0x26] BB_STACK_WRITE_PORT */
	543	+ REG16(0x588), /* [0x28] BB_STACK_WRITE_PORT */
	544	+
	545	+ NOP(12), /* [0x00] */
	546	+
	547	+ 0
	548	+};
	549	+
511	550	#undef REG16
512	551	#undef REG
513	552	#undef LRI
@@ -546,6 +585,10 @@ static void set_context_control(u32 regs, struct xe_hw_engine hwe) · 32 unmodified lines
546	585	regs[CTX_CONTEXT_CONTROL] = _MASKED_BIT_ENABLE(CTX_CTRL_INHIBIT_SYN_CTX_SWITCH \|
547	586	CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT);
548	587
	588	+ if (xe_gt_has_indirect_ring_state(hwe->gt))
	589	+ regs[CTX_CONTEXT_CONTROL] \|=
	590	+ _MASKED_BIT_ENABLE(CTX_CTRL_INDIRECT_RING_STATE_ENABLE);
	591	+
549	592	/* TODO: Timestamp */
550	593	}
551	594
@@ -589,6 +632,11 @@ static void reset_stop_ring(u32 regs, struct xe_hw_engine hwe) · 37 unmodified lines
589	632	regs[x + 1] \|= STOP_RING << 16;
590	633	}
591	634
	635	+static inline bool xe_lrc_has_indirect_ring_state(struct xe_lrc *lrc)
	636	+{
	637	+ return lrc->flags & XE_LRC_FLAG_INDIRECT_RING_STATE;
	638	+}
	639	+
592	640	static inline u32 __xe_lrc_ring_offset(struct xe_lrc *lrc)
593	641	{
594	642	return 0;
@@ -643,6 +691,12 @@ static inline u32 __xe_lrc_regs_offset(struct xe_lrc *lrc) · 48 unmodified lines
643	691	return xe_lrc_pphwsp_offset(lrc) + LRC_PPHWSP_SIZE;
644	692	}
645	693
	694	+static inline u32 __xe_lrc_indirect_ring_offset(struct xe_lrc *lrc)
	695	+{
	696	+ /* Indirect ring state page is at the very end of LRC */
	697	+ return lrc->size - LRC_INDIRECT_RING_STATE_SIZE;
	698	+}
	699	+
646	700	#define DECL_MAP_ADDR_HELPERS(elem) \
647	701	static inline struct iosys_map __xe_lrc_##elem##_map(struct xe_lrc *lrc) \
648	702	{ \
@@ -663,6 +717,7 @@ DECL_MAP_ADDR_HELPERS(seqno) · 14 unmodified lines
663	717	DECL_MAP_ADDR_HELPERS(regs)
664	718	DECL_MAP_ADDR_HELPERS(start_seqno)
665	719	DECL_MAP_ADDR_HELPERS(parallel)
	720	+DECL_MAP_ADDR_HELPERS(indirect_ring)
666	721
667	722	#undef DECL_MAP_ADDR_HELPERS
668	723
@@ -671,6 +726,35 @@ u32 xe_lrc_ggtt_addr(struct xe_lrc *lrc) · 2 unmodified lines
671	726	return __xe_lrc_pphwsp_ggtt_addr(lrc);
672	727	}
673	728
	729	+u32 xe_lrc_indirect_ring_ggtt_addr(struct xe_lrc *lrc)
	730	+{
	731	+ if (!xe_lrc_has_indirect_ring_state(lrc))
	732	+ return 0;
	733	+
	734	+ return __xe_lrc_indirect_ring_ggtt_addr(lrc);
	735	+}
	736	+
	737	+static u32 xe_lrc_read_indirect_ctx_reg(struct xe_lrc *lrc, int reg_nr)
	738	+{
	739	+ struct xe_device *xe = lrc_to_xe(lrc);
	740	+ struct iosys_map map;
	741	+
	742	+ map = __xe_lrc_indirect_ring_map(lrc);
	743	+ iosys_map_incr(&map, reg_nr * sizeof(u32));
	744	+ return xe_map_read32(xe, &map);
	745	+}
	746	+
	747	+static void xe_lrc_write_indirect_ctx_reg(struct xe_lrc *lrc,
	748	+ int reg_nr, u32 val)
	749	+{
	750	+ struct xe_device *xe = lrc_to_xe(lrc);
	751	+ struct iosys_map map;
	752	+
	753	+ map = __xe_lrc_indirect_ring_map(lrc);
	754	+ iosys_map_incr(&map, reg_nr * sizeof(u32));
	755	+ xe_map_write32(xe, &map, val);
	756	+}
	757	+
674	758	u32 xe_lrc_read_ctx_reg(struct xe_lrc *lrc, int reg_nr)
675	759	{
676	760	struct xe_device *xe = lrc_to_xe(lrc);
@@ -693,20 +777,25 @@ void xe_lrc_write_ctx_reg(struct xe_lrc *lrc, int reg_nr, u32 val) · 16 unmodified lines
693	777
694	778	static void empty_lrc_data(struct xe_hw_engine hwe)
695	779	{
696		− struct xe_device *xe = gt_to_xe(hwe->gt);
	780	+ struct xe_gt *gt = hwe->gt;
697	781	void *data;
698	782	u32 *regs;
699	783
700		− data = kzalloc(xe_lrc_size(xe, hwe->class), GFP_KERNEL);
	784	+ data = kzalloc(xe_gt_lrc_size(gt, hwe->class), GFP_KERNEL);
701	785	if (!data)
702	786	return NULL;
703	787
704	788	/* 1st page: Per-Process of HW status Page */
705	789	regs = data + LRC_PPHWSP_SIZE;
706		− set_offsets(regs, reg_offsets(xe, hwe->class), hwe);
	790	+ set_offsets(regs, reg_offsets(gt_to_xe(gt), hwe->class), hwe);
707	791	set_context_control(regs, hwe);
708	792	set_memory_based_intr(regs, hwe);
709	793	reset_stop_ring(regs, hwe);
	794	+ if (xe_gt_has_indirect_ring_state(gt)) {
	795	+ regs = data + xe_gt_lrc_size(gt, hwe->class) -
	796	+ LRC_INDIRECT_RING_STATE_SIZE;
	797	+ set_offsets(regs, xe2_indirect_ring_state_offsets, hwe);
	798	+ }
710	799
711	800	return data;
712	801	}
@@ -719,11 +808,20 @@ static void xe_lrc_set_ppgtt(struct xe_lrc lrc, struct xe_vm vm) · 6 unmodified lines
719	808	xe_lrc_write_ctx_reg(lrc, CTX_PDP0_LDW, lower_32_bits(desc));
720	809	}
721	810
	811	+static void xe_lrc_finish(struct xe_lrc *lrc)
	812	+{
	813	+ xe_hw_fence_ctx_finish(&lrc->fence_ctx);
	814	+ xe_bo_lock(lrc->bo, false);
	815	+ xe_bo_unpin(lrc->bo);
	816	+ xe_bo_unlock(lrc->bo);
	817	+ xe_bo_put(lrc->bo);
	818	+}
	819	+
722	820	#define PVC_CTX_ASID (0x2e + 1)
723	821	#define PVC_CTX_ACC_CTR_THOLD (0x2a + 1)
724	822
725		−int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe,
726		− struct xe_exec_queue q, struct xe_vm vm, u32 ring_size)
	823	+static int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe,
	824	+ struct xe_vm *vm, u32 ring_size)
727	825	{
728	826	struct xe_gt *gt = hwe->gt;
729	827	struct xe_tile *tile = gt_to_tile(gt);
@@ -731,26 +829,32 @@ int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe, · 1 unmodified line
731	829	struct iosys_map map;
732	830	void *init_data = NULL;
733	831	u32 arb_enable;
	832	+ u32 lrc_size;
734	833	int err;
735	834
	835	+ kref_init(&lrc->refcount);
736	836	lrc->flags = 0;
	837	+ lrc_size = ring_size + xe_gt_lrc_size(gt, hwe->class);
	838	+ if (xe_gt_has_indirect_ring_state(gt))
	839	+ lrc->flags \|= XE_LRC_FLAG_INDIRECT_RING_STATE;
737	840
738	841	/*
739	842	* FIXME: Perma-pinning LRC as we don't yet support moving GGTT address
740	843	* via VM bind calls.
741	844	*/
742		− lrc->bo = xe_bo_create_pin_map(xe, tile, vm,
743		− ring_size + xe_lrc_size(xe, hwe->class),
744		− ttm_bo_type_kernel,
745		− XE_BO_FLAG_VRAM_IF_DGFX(tile) \|
746		− XE_BO_FLAG_GGTT \|
747		− XE_BO_FLAG_GGTT_INVALIDATE);
	845	+ lrc->bo = xe_bo_create_pin_map(xe, tile, vm, lrc_size,
	846	+ ttm_bo_type_kernel,
	847	+ XE_BO_FLAG_VRAM_IF_DGFX(tile) \|
	848	+ XE_BO_FLAG_GGTT \|
	849	+ XE_BO_FLAG_GGTT_INVALIDATE);
748	850	if (IS_ERR(lrc->bo))
749	851	return PTR_ERR(lrc->bo);
750	852
	853	+ lrc->size = lrc_size;
751	854	lrc->tile = gt_to_tile(hwe->gt);
752	855	lrc->ring.size = ring_size;
753	856	lrc->ring.tail = 0;
	857	+ lrc->ctx_timestamp = 0;
754	858
755	859	xe_hw_fence_ctx_init(&lrc->fence_ctx, hwe->gt,
756	860	hwe->fence_irq, hwe->name);
@@ -772,10 +876,10 @@ int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe, · 15 unmodified lines
772	876	xe_map_memset(xe, &map, 0, 0, LRC_PPHWSP_SIZE); /* PPHWSP */
773	877	xe_map_memcpy_to(xe, &map, LRC_PPHWSP_SIZE,
774	878	gt->default_lrc[hwe->class] + LRC_PPHWSP_SIZE,
775		− xe_lrc_size(xe, hwe->class) - LRC_PPHWSP_SIZE);
	879	+ xe_gt_lrc_size(gt, hwe->class) - LRC_PPHWSP_SIZE);
776	880	} else {
777	881	xe_map_memcpy_to(xe, &map, 0, init_data,
778		− xe_lrc_size(xe, hwe->class));
	882	+ xe_gt_lrc_size(gt, hwe->class));
779	883	kfree(init_data);
780	884	}
781	885
@@ -786,11 +890,27 @@ int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe, · 4 unmodified lines
786	890	xe_drm_client_add_bo(vm->xef->client, lrc->bo);
787	891	}
788	892
789		− xe_lrc_write_ctx_reg(lrc, CTX_RING_START, __xe_lrc_ring_ggtt_addr(lrc));
790		− xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, 0);
791		− xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, lrc->ring.tail);
792		− xe_lrc_write_ctx_reg(lrc, CTX_RING_CTL,
793		− RING_CTL_SIZE(lrc->ring.size) \| RING_VALID);
	893	+ if (xe_gt_has_indirect_ring_state(gt)) {
	894	+ xe_lrc_write_ctx_reg(lrc, CTX_INDIRECT_RING_STATE,
	895	+ __xe_lrc_indirect_ring_ggtt_addr(lrc));
	896	+
	897	+ xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_START,
	898	+ __xe_lrc_ring_ggtt_addr(lrc));
	899	+ xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_START_UDW, 0);
	900	+ xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_HEAD, 0);
	901	+ xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_TAIL, lrc->ring.tail);
	902	+ xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_CTL,
	903	+ RING_CTL_SIZE(lrc->ring.size) \| RING_VALID);
	904	+ } else {
	905	+ xe_lrc_write_ctx_reg(lrc, CTX_RING_START, __xe_lrc_ring_ggtt_addr(lrc));
	906	+ xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, 0);
	907	+ xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, lrc->ring.tail);
	908	+ xe_lrc_write_ctx_reg(lrc, CTX_RING_CTL,
	909	+ RING_CTL_SIZE(lrc->ring.size) \| RING_VALID);
	910	+ }
	911	+
	912	+ xe_lrc_write_ctx_reg(lrc, CTX_TIMESTAMP, 0);
	913	+
794	914	if (xe->info.has_asid && vm)
795	915	xe_lrc_write_ctx_reg(lrc, PVC_CTX_ASID, vm->usm.asid);
796	916
@@ -825,23 +945,81 @@ err_lrc_finish: · 28 unmodified lines
825	945	return err;
826	946	}
827	947
828		−void xe_lrc_finish(struct xe_lrc *lrc)
	948	+/**
	949	+ * xe_lrc_create - Create a LRC
	950	+ * @hwe: Hardware Engine
	951	+ * @vm: The VM (address space)
	952	+ * @ring_size: LRC ring size
	953	+ *
	954	+ * Allocate and initialize the Logical Ring Context (LRC).
	955	+ *
	956	+ * Return pointer to created LRC upon success and an error pointer
	957	+ * upon failure.
	958	+ */
	959	+struct xe_lrc xe_lrc_create(struct xe_hw_engine hwe, struct xe_vm *vm,
	960	+ u32 ring_size)
829	961	{
830		− xe_hw_fence_ctx_finish(&lrc->fence_ctx);
831		− xe_bo_lock(lrc->bo, false);
832		− xe_bo_unpin(lrc->bo);
833		− xe_bo_unlock(lrc->bo);
834		− xe_bo_put(lrc->bo);
	962	+ struct xe_lrc *lrc;
	963	+ int err;
	964	+
	965	+ lrc = kzalloc(sizeof(*lrc), GFP_KERNEL);
	966	+ if (!lrc)
	967	+ return ERR_PTR(-ENOMEM);
	968	+
	969	+ err = xe_lrc_init(lrc, hwe, vm, ring_size);
	970	+ if (err) {
	971	+ kfree(lrc);
	972	+ return ERR_PTR(err);
	973	+ }
	974	+
	975	+ return lrc;
	976	+}
	977	+
	978	+/**
	979	+ * xe_lrc_destroy - Destroy the LRC
	980	+ * @ref: reference to LRC
	981	+ *
	982	+ * Called when ref == 0, release resources held by the Logical Ring Context
	983	+ * (LRC) and free the LRC memory.
	984	+ */
	985	+void xe_lrc_destroy(struct kref *ref)
	986	+{
	987	+ struct xe_lrc *lrc = container_of(ref, struct xe_lrc, refcount);
	988	+
	989	+ xe_lrc_finish(lrc);
	990	+ kfree(lrc);
	991	+}
	992	+
	993	+void xe_lrc_set_ring_tail(struct xe_lrc *lrc, u32 tail)
	994	+{
	995	+ if (xe_lrc_has_indirect_ring_state(lrc))
	996	+ xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_TAIL, tail);
	997	+ else
	998	+ xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, tail);
	999	+}
	1000	+
	1001	+u32 xe_lrc_ring_tail(struct xe_lrc *lrc)
	1002	+{
	1003	+ if (xe_lrc_has_indirect_ring_state(lrc))
	1004	+ return xe_lrc_read_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_TAIL) & TAIL_ADDR;
	1005	+ else
	1006	+ return xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL) & TAIL_ADDR;
835	1007	}
836	1008
837	1009	void xe_lrc_set_ring_head(struct xe_lrc *lrc, u32 head)
838	1010	{
839		− xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, head);
	1011	+ if (xe_lrc_has_indirect_ring_state(lrc))
	1012	+ xe_lrc_write_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_HEAD, head);
	1013	+ else
	1014	+ xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, head);
840	1015	}
841	1016
842	1017	u32 xe_lrc_ring_head(struct xe_lrc *lrc)
843	1018	{
844		− return xe_lrc_read_ctx_reg(lrc, CTX_RING_HEAD) & HEAD_ADDR;
	1019	+ if (xe_lrc_has_indirect_ring_state(lrc))
	1020	+ return xe_lrc_read_indirect_ctx_reg(lrc, INDIRECT_CTX_RING_HEAD) & HEAD_ADDR;
	1021	+ else
	1022	+ return xe_lrc_read_ctx_reg(lrc, CTX_RING_HEAD) & HEAD_ADDR;
845	1023	}
846	1024
847	1025	u32 xe_lrc_ring_space(struct xe_lrc *lrc)
@@ -901,10 +1079,43 @@ u32 xe_lrc_seqno_ggtt_addr(struct xe_lrc *lrc) · 53 unmodified lines
901	1079	return __xe_lrc_seqno_ggtt_addr(lrc);
902	1080	}
903	1081
904		−struct dma_fence xe_lrc_create_seqno_fence(struct xe_lrc lrc)
	1082	+/**
	1083	+ * xe_lrc_alloc_seqno_fence() - Allocate an lrc seqno fence.
	1084	+ *
	1085	+ * Allocate but don't initialize an lrc seqno fence.
	1086	+ *
	1087	+ * Return: Pointer to the allocated fence or
	1088	+ * negative error pointer on error.
	1089	+ */
	1090	+struct dma_fence *xe_lrc_alloc_seqno_fence(void)
905	1091	{
906		− return &xe_hw_fence_create(&lrc->fence_ctx,
907		− __xe_lrc_seqno_map(lrc))->dma;
	1092	+ return xe_hw_fence_alloc();
	1093	+}
	1094	+
	1095	+/**
	1096	+ * xe_lrc_free_seqno_fence() - Free an lrc seqno fence.
	1097	+ * @fence: Pointer to the fence to free.
	1098	+ *
	1099	+ * Frees an lrc seqno fence that hasn't yet been
	1100	+ * initialized.
	1101	+ */
	1102	+void xe_lrc_free_seqno_fence(struct dma_fence *fence)
	1103	+{
	1104	+ xe_hw_fence_free(fence);
	1105	+}
	1106	+
	1107	+/**
	1108	+ * xe_lrc_init_seqno_fence() - Initialize an lrc seqno fence.
	1109	+ * @lrc: Pointer to the lrc.
	1110	+ * @fence: Pointer to the fence to initialize.
	1111	+ *
	1112	+ * Initializes a pre-allocated lrc seqno fence.
	1113	+ * After initialization, the fence is subject to normal
	1114	+ * dma-fence refcounting.
	1115	+ */
	1116	+void xe_lrc_init_seqno_fence(struct xe_lrc lrc, struct dma_fence fence)
	1117	+{
	1118	+ xe_hw_fence_init(fence, &lrc->fence_ctx, __xe_lrc_seqno_map(lrc));
908	1119	}
909	1120
910	1121	s32 xe_lrc_seqno(struct xe_lrc *lrc)
@@ -1214,7 +1425,7 @@ void xe_lrc_dump_default(struct drm_printer *p, · 303 unmodified lines
1214	1425	* hardware status page.
1215	1426	*/
1216	1427	dw = gt->default_lrc[hwe_class] + LRC_PPHWSP_SIZE;
1217		− remaining_dw = (xe_lrc_size(gt_to_xe(gt), hwe_class) - LRC_PPHWSP_SIZE) / 4;
	1428	+ remaining_dw = (xe_gt_lrc_size(gt, hwe_class) - LRC_PPHWSP_SIZE) / 4;
1218	1429
1219	1430	while (remaining_dw > 0) {
1220	1431	if ((*dw & XE_INSTR_CMD_TYPE) == XE_INSTR_MI) {
@@ -1354,10 +1565,11 @@ struct xe_lrc_snapshot xe_lrc_snapshot_capture(struct xe_lrc lrc) · 133 unmodified lines
1354	1565	if (!snapshot)
1355	1566	return NULL;
1356	1567
1357		− snapshot->context_desc = lower_32_bits(xe_lrc_ggtt_addr(lrc));
	1568	+ snapshot->context_desc = xe_lrc_ggtt_addr(lrc);
	1569	+ snapshot->indirect_context_desc = xe_lrc_indirect_ring_ggtt_addr(lrc);
1358	1570	snapshot->head = xe_lrc_ring_head(lrc);
1359	1571	snapshot->tail.internal = lrc->ring.tail;
1360		− snapshot->tail.memory = xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL);
	1572	+ snapshot->tail.memory = xe_lrc_ring_tail(lrc);
1361	1573	snapshot->start_seqno = xe_lrc_start_seqno(lrc);
1362	1574	snapshot->seqno = xe_lrc_seqno(lrc);
1363	1575	snapshot->lrc_bo = xe_bo_get(lrc->bo);
@@ -1382,7 +1594,7 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot) · 18 unmodified lines
1382	1594	if (!snapshot->lrc_snapshot)
1383	1595	goto put_bo;
1384	1596
1385		− dma_resv_lock(bo->ttm.base.resv, NULL);
	1597	+ xe_bo_lock(bo, false);
1386	1598	if (!ttm_bo_vmap(&bo->ttm, &src)) {
1387	1599	xe_map_memcpy_from(xe_bo_device(bo),
1388	1600	snapshot->lrc_snapshot, &src, snapshot->lrc_offset,
@@ -1392,7 +1604,7 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot) · 3 unmodified lines
1392	1604	kvfree(snapshot->lrc_snapshot);
1393	1605	snapshot->lrc_snapshot = NULL;
1394	1606	}
1395		− dma_resv_unlock(bo->ttm.base.resv);
	1607	+ xe_bo_unlock(bo);
1396	1608	put_bo:
1397	1609	xe_bo_put(bo);
1398	1610	}
@@ -1405,6 +1617,8 @@ void xe_lrc_snapshot_print(struct xe_lrc_snapshot *snapshot, struct drm_printer · 6 unmodified lines
1405	1617	return;
1406	1618
1407	1619	drm_printf(p, "\tHW Context Desc: 0x%08x\n", snapshot->context_desc);
	1620	+ drm_printf(p, "\tHW Indirect Ring State: 0x%08x\n",
	1621	+ snapshot->indirect_context_desc);
1408	1622	drm_printf(p, "\tLRC Head: (memory) %u\n", snapshot->head);
1409	1623	drm_printf(p, "\tLRC Tail: (internal) %u, (memory) %u\n",
1410	1624	snapshot->tail.internal, snapshot->tail.memory);
@@ -1444,3 +1658,12 @@ void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot) · 33 unmodified lines
1444	1658	xe_bo_put(snapshot->lrc_bo);
1445	1659	kfree(snapshot);
1446	1660	}
	1661	+
	1662	+u32 xe_lrc_update_timestamp(struct xe_lrc lrc, u32 old_ts)
	1663	+{
	1664	+ *old_ts = lrc->ctx_timestamp;
	1665	+
	1666	+ lrc->ctx_timestamp = xe_lrc_read_ctx_reg(lrc, CTX_TIMESTAMP);
	1667	+
	1668	+ return lrc->ctx_timestamp;
	1669	+}

drivers/gpu/drm/xe/xe_lrc.h+53−5

@@ -5,6 +5,8 @@ · 4 unmodified lines
5	5	#ifndef _XE_LRC_H_
6	6	#define _XE_LRC_H_
7	7
	8	+#include <linux/types.h>
	9	+
8	10	#include "xe_lrc_types.h"
9	11
10	12	struct drm_printer;
@@ -12,23 +14,53 @@ struct xe_bb; · 1 unmodified line
12	14	struct xe_device;
13	15	struct xe_exec_queue;
14	16	enum xe_engine_class;
	17	+struct xe_gt;
15	18	struct xe_hw_engine;
	19	+struct xe_lrc;
	20	+struct xe_lrc_snapshot;
16	21	struct xe_vm;
17	22
18	23	#define LRC_PPHWSP_SCRATCH_ADDR (0x34 * 4)
19	24
20		−int xe_lrc_init(struct xe_lrc lrc, struct xe_hw_engine hwe,
21		− struct xe_exec_queue q, struct xe_vm vm, u32 ring_size);
22		−void xe_lrc_finish(struct xe_lrc *lrc);
	25	+struct xe_lrc xe_lrc_create(struct xe_hw_engine hwe, struct xe_vm *vm,
	26	+ u32 ring_size);
	27	+void xe_lrc_destroy(struct kref *ref);
	28	+
	29	+/**
	30	+ * xe_lrc_get - Get reference to the LRC
	31	+ * @lrc: Logical Ring Context
	32	+ *
	33	+ * Increment reference count of @lrc
	34	+ */
	35	+static inline struct xe_lrc xe_lrc_get(struct xe_lrc lrc)
	36	+{
	37	+ kref_get(&lrc->refcount);
	38	+ return lrc;
	39	+}
	40	+
	41	+/**
	42	+ * xe_lrc_put - Put reference of the LRC
	43	+ * @lrc: Logical Ring Context
	44	+ *
	45	+ * Decrement reference count of @lrc, call xe_lrc_destroy when
	46	+ * reference count reaches 0.
	47	+ */
	48	+static inline void xe_lrc_put(struct xe_lrc *lrc)
	49	+{
	50	+ kref_put(&lrc->refcount, xe_lrc_destroy);
	51	+}
23	52
24		−size_t xe_lrc_size(struct xe_device *xe, enum xe_engine_class class);
	53	+size_t xe_gt_lrc_size(struct xe_gt *gt, enum xe_engine_class class);
25	54	u32 xe_lrc_pphwsp_offset(struct xe_lrc *lrc);
26	55
	56	+void xe_lrc_set_ring_tail(struct xe_lrc *lrc, u32 tail);
	57	+u32 xe_lrc_ring_tail(struct xe_lrc *lrc);
27	58	void xe_lrc_set_ring_head(struct xe_lrc *lrc, u32 head);
28	59	u32 xe_lrc_ring_head(struct xe_lrc *lrc);
29	60	u32 xe_lrc_ring_space(struct xe_lrc *lrc);
30	61	void xe_lrc_write_ring(struct xe_lrc lrc, const void data, size_t size);
31	62
	63	+u32 xe_lrc_indirect_ring_ggtt_addr(struct xe_lrc *lrc);
32	64	u32 xe_lrc_ggtt_addr(struct xe_lrc *lrc);
33	65	u32 xe_lrc_regs(struct xe_lrc lrc);
34	66
@@ -38,7 +70,9 @@ void xe_lrc_write_ctx_reg(struct xe_lrc *lrc, int reg_nr, u32 val); · 3 unmodified lines
38	70	u64 xe_lrc_descriptor(struct xe_lrc *lrc);
39	71
40	72	u32 xe_lrc_seqno_ggtt_addr(struct xe_lrc *lrc);
41		−struct dma_fence xe_lrc_create_seqno_fence(struct xe_lrc lrc);
	73	+struct dma_fence *xe_lrc_alloc_seqno_fence(void);
	74	+void xe_lrc_free_seqno_fence(struct dma_fence *fence);
	75	+void xe_lrc_init_seqno_fence(struct xe_lrc lrc, struct dma_fence fence);
42	76	s32 xe_lrc_seqno(struct xe_lrc *lrc);
43	77
44	78	u32 xe_lrc_start_seqno_ggtt_addr(struct xe_lrc *lrc);
@@ -60,4 +94,18 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot); · 15 unmodified lines
60	94	void xe_lrc_snapshot_print(struct xe_lrc_snapshot snapshot, struct drm_printer p);
61	95	void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot);
62	96
	97	+/**
	98	+ * xe_lrc_update_timestamp - readout LRC timestamp and update cached value
	99	+ * @lrc: logical ring context for this exec queue
	100	+ * @old_ts: pointer where to save the previous timestamp
	101	+ *
	102	+ * Read the current timestamp for this LRC and update the cached value. The
	103	+ * previous cached value is also returned in @old_ts so the caller can calculate
	104	+ * the delta between 2 updates. Note that this is not intended to be called from
	105	+ * any place, but just by the paths updating the drm client utilization.
	106	+ *
	107	+ * Returns the current LRC timestamp
	108	+ */
	109	+u32 xe_lrc_update_timestamp(struct xe_lrc lrc, u32 old_ts);
	110	+
63	111	#endif

drivers/gpu/drm/xe/xe_lrc_types.h+12−0

@@ -6,6 +6,8 @@ · 5 unmodified lines
6	6	#ifndef _XE_LRC_TYPES_H_
7	7	#define _XE_LRC_TYPES_H_
8	8
	9	+#include <linux/kref.h>
	10	+
9	11	#include "xe_hw_fence_types.h"
10	12
11	13	struct xe_bo;
@@ -20,12 +22,19 @@ struct xe_lrc { · 8 unmodified lines
20	22	*/
21	23	struct xe_bo *bo;
22	24
	25	+ /** @size: size of lrc including any indirect ring state page */
	26	+ u32 size;
	27	+
23	28	/** @tile: tile which this LRC belongs to */
24	29	struct xe_tile *tile;
25	30
26	31	/** @flags: LRC flags */
	32	+#define XE_LRC_FLAG_INDIRECT_RING_STATE 0x1
27	33	u32 flags;
28	34
	35	+ /** @refcount: ref count of this lrc */
	36	+ struct kref refcount;
	37	+
29	38	/** @ring: submission ring state */
30	39	struct {
31	40	/** @ring.size: size of submission ring */
@@ -41,6 +50,9 @@ struct xe_lrc { · 9 unmodified lines
41	50
42	51	/** @fence_ctx: context for hw fence */
43	52	struct xe_hw_fence_ctx fence_ctx;
	53	+
	54	+ /** @ctx_timestamp: readout value of CTX_TIMESTAMP on last update */
	55	+ u32 ctx_timestamp;
44	56	};
45	57
46	58	struct xe_lrc_snapshot;

drivers/gpu/drm/xe/xe_migrate.c+53−25

@@ -14,8 +14,8 @@ · 13 unmodified lines
14	14
15	15	#include <generated/xe_wa_oob.h>
16	16
	17	+#include "instructions/xe_gpu_commands.h"
17	18	#include "instructions/xe_mi_commands.h"
18		−#include "regs/xe_gpu_commands.h"
19	19	#include "regs/xe_gtt_defs.h"
20	20	#include "tests/xe_test.h"
21	21	#include "xe_assert.h"
@@ -69,7 +69,7 @@ struct xe_migrate { · 47 unmodified lines
69	69
70	70	#define MAX_PREEMPTDISABLE_TRANSFER SZ_8M /* Around 1ms. */
71	71	#define MAX_CCS_LIMITED_TRANSFER SZ_4M /* XE_PAGE_SIZE * (FIELD_MAX(XE2_CCS_SIZE_MASK) + 1) */
72		−#define NUM_KERNEL_PDE 17
	72	+#define NUM_KERNEL_PDE 15
73	73	#define NUM_PT_SLOTS 32
74	74	#define LEVEL0_PAGE_TABLE_ENCODE_SIZE SZ_2M
75	75	#define MAX_NUM_PTE 512
@@ -137,10 +137,11 @@ static int xe_migrate_prepare_vm(struct xe_tile tile, struct xe_migrate m, · 61 unmodified lines
137	137	struct xe_device *xe = tile_to_xe(tile);
138	138	u16 pat_index = xe->pat.idx[XE_CACHE_WB];
139	139	u8 id = tile->id;
140		− u32 num_entries = NUM_PT_SLOTS, num_level = vm->pt_root[id]->level;
	140	+ u32 num_entries = NUM_PT_SLOTS, num_level = vm->pt_root[id]->level,
	141	+ num_setup = num_level + 1;
141	142	u32 map_ofs, level, i;
142	143	struct xe_bo bo, batch = tile->mem.kernel_bb_pool->bo;
143		− u64 entry;
	144	+ u64 entry, pt30_ofs;
144	145
145	146	/* Can't bump NUM_PT_SLOTS too high */
146	147	BUILD_BUG_ON(NUM_PT_SLOTS > SZ_2M/XE_PAGE_SIZE);
@@ -160,10 +161,12 @@ static int xe_migrate_prepare_vm(struct xe_tile tile, struct xe_migrate m, · 13 unmodified lines
160	161	if (IS_ERR(bo))
161	162	return PTR_ERR(bo);
162	163
163		− entry = vm->pt_ops->pde_encode_bo(bo, bo->size - XE_PAGE_SIZE, pat_index);
	164	+ /* PT31 reserved for 2M identity map */
	165	+ pt30_ofs = bo->size - 2 * XE_PAGE_SIZE;
	166	+ entry = vm->pt_ops->pde_encode_bo(bo, pt30_ofs, pat_index);
164	167	xe_pt_write(xe, &vm->pt_root[id]->bo->vmap, 0, entry);
165	168
166		− map_ofs = (num_entries - num_level) * XE_PAGE_SIZE;
	169	+ map_ofs = (num_entries - num_setup) * XE_PAGE_SIZE;
167	170
168	171	/* Map the entire BO in our level 0 pt */
169	172	for (i = 0, level = 0; i < num_entries; level++) {
@@ -234,7 +237,7 @@ static int xe_migrate_prepare_vm(struct xe_tile tile, struct xe_migrate m, · 64 unmodified lines
234	237	}
235	238
236	239	/* Write PDE's that point to our BO. */
237		− for (i = 0; i < num_entries - num_level; i++) {
	240	+ for (i = 0; i < map_ofs / PAGE_SIZE; i++) {
238	241	entry = vm->pt_ops->pde_encode_bo(bo, (u64)i * XE_PAGE_SIZE,
239	242	pat_index);
240	243
@@ -252,28 +255,54 @@ static int xe_migrate_prepare_vm(struct xe_tile tile, struct xe_migrate m, · 11 unmodified lines
252	255	/* Identity map the entire vram at 256GiB offset */
253	256	if (IS_DGFX(xe)) {
254	257	u64 pos, ofs, flags;
	258	+ /* XXX: Unclear if this should be usable_size? */
	259	+ u64 vram_limit = xe->mem.vram.actual_physical_size +
	260	+ xe->mem.vram.dpa_base;
255	261
256	262	level = 2;
257	263	ofs = map_ofs + XE_PAGE_SIZE * level + 256 * 8;
258	264	flags = vm->pt_ops->pte_encode_addr(xe, 0, pat_index, level,
259	265	true, 0);
260	266
	267	+ xe_assert(xe, IS_ALIGNED(xe->mem.vram.usable_size, SZ_2M));
	268	+
261	269	/*
262		− * Use 1GB pages, it shouldn't matter the physical amount of
263		− * vram is less, when we don't access it.
	270	+ * Use 1GB pages when possible, last chunk always use 2M
	271	+ * pages as mixing reserved memory (stolen, WOCPM) with a single
	272	+ * mapping is not allowed on certain platforms.
264	273	*/
265		− for (pos = xe->mem.vram.dpa_base;
266		− pos < xe->mem.vram.actual_physical_size + xe->mem.vram.dpa_base;
267		− pos += SZ_1G, ofs += 8)
	274	+ for (pos = xe->mem.vram.dpa_base; pos < vram_limit;
	275	+ pos += SZ_1G, ofs += 8) {
	276	+ if (pos + SZ_1G >= vram_limit) {
	277	+ u64 pt31_ofs = bo->size - XE_PAGE_SIZE;
	278	+
	279	+ entry = vm->pt_ops->pde_encode_bo(bo, pt31_ofs,
	280	+ pat_index);
	281	+ xe_map_wr(xe, &bo->vmap, ofs, u64, entry);
	282	+
	283	+ flags = vm->pt_ops->pte_encode_addr(xe, 0,
	284	+ pat_index,
	285	+ level - 1,
	286	+ true, 0);
	287	+
	288	+ for (ofs = pt31_ofs; pos < vram_limit;
	289	+ pos += SZ_2M, ofs += 8)
	290	+ xe_map_wr(xe, &bo->vmap, ofs, u64, pos \| flags);
	291	+ break; /* Ensure pos == vram_limit assert correct */
	292	+ }
	293	+
268	294	xe_map_wr(xe, &bo->vmap, ofs, u64, pos \| flags);
	295	+ }
	296	+
	297	+ xe_assert(xe, pos == vram_limit);
269	298	}
270	299
271	300	/*
272	301	* Example layout created above, with root level = 3:
273	302	* [PT0...PT7]: kernel PT's for copy/clear; 64 or 4KiB PTE's
274	303	* [PT8]: Kernel PT for VM_BIND, 4 KiB PTE's
275		− * [PT9...PT28]: Userspace PT's for VM_BIND, 4 KiB PTE's
276		− * [PT29 = PDE 0] [PT30 = PDE 1] [PT31 = PDE 2]
	304	+ * [PT9...PT27]: Userspace PT's for VM_BIND, 4 KiB PTE's
	305	+ * [PT28 = PDE 0] [PT29 = PDE 1] [PT30 = PDE 2] [PT31 = 2M vram identity map]
277	306	*
278	307	* This makes the lowest part of the VM point to the pagetables.
279	308	* Hence the lowest 2M in the vm should point to itself, with a few writes
@@ -383,6 +412,9 @@ struct xe_migrate xe_migrate_init(struct xe_tile tile) · 103 unmodified lines
383	412	}
384	413
385	414	mutex_init(&m->job_mutex);
	415	+ fs_reclaim_acquire(GFP_KERNEL);
	416	+ might_lock(&m->job_mutex);
	417	+ fs_reclaim_release(GFP_KERNEL);
386	418
387	419	err = drmm_add_action_or_reset(&xe->drm, xe_migrate_fini, m);
388	420	if (err)
@@ -807,7 +839,6 @@ struct dma_fence xe_migrate_copy(struct xe_migrate m, · 418 unmodified lines
807	839	IS_DGFX(xe) ? dst_is_vram : dst_is_pltt,
808	840	src_L0, ccs_ofs, copy_ccs);
809	841
810		− mutex_lock(&m->job_mutex);
811	842	job = xe_bb_create_migration_job(m->q, bb,
812	843	xe_migrate_batch_base(m, usm),
813	844	update_idx);
@@ -827,6 +858,7 @@ struct dma_fence xe_migrate_copy(struct xe_migrate m, · 13 unmodified lines
827	858	goto err_job;
828	859	}
829	860
	861	+ mutex_lock(&m->job_mutex);
830	862	xe_sched_job_arm(job);
831	863	dma_fence_put(fence);
832	864	fence = dma_fence_get(&job->drm.s_fence->finished);
@@ -844,7 +876,6 @@ struct dma_fence xe_migrate_copy(struct xe_migrate m, · 11 unmodified lines
844	876	err_job:
845	877	xe_sched_job_put(job);
846	878	err:
847		− mutex_unlock(&m->job_mutex);
848	879	xe_bb_free(bb, NULL);
849	880
850	881	err_sync:
@@ -934,8 +965,8 @@ static bool has_service_copy_support(struct xe_gt *gt) · 83 unmodified lines
934	965	* all of the actual service copy engines (BCS1-BCS8) have been fused
935	966	* off.
936	967	*/
937		− return gt->info.__engine_mask & GENMASK(XE_HW_ENGINE_BCS8,
938		− XE_HW_ENGINE_BCS1);
	968	+ return gt->info.engine_mask & GENMASK(XE_HW_ENGINE_BCS8,
	969	+ XE_HW_ENGINE_BCS1);
939	970	}
940	971
941	972	static u32 emit_clear_cmd_len(struct xe_gt *gt)
@@ -1044,7 +1075,6 @@ struct dma_fence xe_migrate_clear(struct xe_migrate m, · 102 unmodified lines
1044	1075	flush_flags = MI_FLUSH_DW_CCS;
1045	1076	}
1046	1077
1047		− mutex_lock(&m->job_mutex);
1048	1078	job = xe_bb_create_migration_job(m->q, bb,
1049	1079	xe_migrate_batch_base(m, usm),
1050	1080	update_idx);
@@ -1067,6 +1097,7 @@ struct dma_fence xe_migrate_clear(struct xe_migrate m, · 16 unmodified lines
1067	1097	goto err_job;
1068	1098	}
1069	1099
	1100	+ mutex_lock(&m->job_mutex);
1070	1101	xe_sched_job_arm(job);
1071	1102	dma_fence_put(fence);
1072	1103	fence = dma_fence_get(&job->drm.s_fence->finished);
@@ -1083,7 +1114,6 @@ struct dma_fence xe_migrate_clear(struct xe_migrate m, · 10 unmodified lines
1083	1114	err_job:
1084	1115	xe_sched_job_put(job);
1085	1116	err:
1086		− mutex_unlock(&m->job_mutex);
1087	1117	xe_bb_free(bb, NULL);
1088	1118	err_sync:
1089	1119	/* Sync partial copies if any. FIXME: job_mutex? */
@@ -1377,9 +1407,6 @@ xe_migrate_update_pgtables(struct xe_migrate *m, · 287 unmodified lines
1377	1407	write_pgtable(tile, bb, 0, &updates[i], pt_update);
1378	1408	}
1379	1409
1380		− if (!q)
1381		− mutex_lock(&m->job_mutex);
1382		−
1383	1410	job = xe_bb_create_migration_job(q ?: m->q, bb,
1384	1411	xe_migrate_batch_base(m, usm),
1385	1412	update_idx);
@@ -1420,6 +1447,9 @@ xe_migrate_update_pgtables(struct xe_migrate *m, · 34 unmodified lines
1420	1447	if (err)
1421	1448	goto err_job;
1422	1449	}
	1450	+ if (!q)
	1451	+ mutex_lock(&m->job_mutex);
	1452	+
1423	1453	xe_sched_job_arm(job);
1424	1454	fence = dma_fence_get(&job->drm.s_fence->finished);
1425	1455	xe_sched_job_push(job);
@@ -1435,8 +1465,6 @@ xe_migrate_update_pgtables(struct xe_migrate *m, · 9 unmodified lines
1435	1465	err_job:
1436	1466	xe_sched_job_put(job);
1437	1467	err_bb:
1438		− if (!q)
1439		− mutex_unlock(&m->job_mutex);
1440	1468	xe_bb_free(bb, NULL);
1441	1469	err:
1442	1470	drm_suballoc_free(sa_bo, NULL);

drivers/gpu/drm/xe/xe_mmio.c+99−343

@@ -3,337 +3,36 @@ · 2 unmodified lines
3	3	* Copyright © 2021-2023 Intel Corporation
4	4	*/
5	5
6		−#include <linux/minmax.h>
7		−
8	6	#include "xe_mmio.h"
9	7
	8	+#include <linux/delay.h>
	9	+#include <linux/io-64-nonatomic-lo-hi.h>
	10	+#include <linux/minmax.h>
	11	+#include <linux/pci.h>
	12	+
10	13	#include <drm/drm_managed.h>
11		−#include <drm/xe_drm.h>
	14	+#include <drm/drm_print.h>
12	15
13		−#include "regs/xe_engine_regs.h"
14		−#include "regs/xe_gt_regs.h"
	16	+#include "regs/xe_bars.h"
15	17	#include "regs/xe_regs.h"
16		−#include "xe_bo.h"
17	18	#include "xe_device.h"
18		−#include "xe_ggtt.h"
19	19	#include "xe_gt.h"
20		−#include "xe_gt_mcr.h"
	20	+#include "xe_gt_printk.h"
	21	+#include "xe_gt_sriov_vf.h"
21	22	#include "xe_macros.h"
22		−#include "xe_module.h"
23	23	#include "xe_sriov.h"
24		−#include "xe_tile.h"
25		−
26		−#define XEHP_MTCFG_ADDR XE_REG(0x101800)
27		−#define TILE_COUNT REG_GENMASK(15, 8)
28		−
29		−#define BAR_SIZE_SHIFT 20
30		−
31		−static void
32		−_resize_bar(struct xe_device *xe, int resno, resource_size_t size)
33		−{
34		− struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
35		− int bar_size = pci_rebar_bytes_to_size(size);
36		− int ret;
37		−
38		− if (pci_resource_len(pdev, resno))
39		− pci_release_resource(pdev, resno);
40		−
41		− ret = pci_resize_resource(pdev, resno, bar_size);
42		− if (ret) {
43		− drm_info(&xe->drm, "Failed to resize BAR%d to %dM (%pe). Consider enabling 'Resizable BAR' support in your BIOS\n",
44		− resno, 1 << bar_size, ERR_PTR(ret));
45		− return;
46		− }
47		−
48		− drm_info(&xe->drm, "BAR%d resized to %dM\n", resno, 1 << bar_size);
49		−}
50		−
51		−/*
52		− * if force_vram_bar_size is set, attempt to set to the requested size
53		− * else set to maximum possible size
54		− */
55		−static void xe_resize_vram_bar(struct xe_device *xe)
56		−{
57		− u64 force_vram_bar_size = xe_modparam.force_vram_bar_size;
58		− struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
59		− struct pci_bus *root = pdev->bus;
60		− resource_size_t current_size;
61		− resource_size_t rebar_size;
62		− struct resource *root_res;
63		− u32 bar_size_mask;
64		− u32 pci_cmd;
65		− int i;
66		−
67		− /* gather some relevant info */
68		− current_size = pci_resource_len(pdev, LMEM_BAR);
69		− bar_size_mask = pci_rebar_get_possible_sizes(pdev, LMEM_BAR);
70		−
71		− if (!bar_size_mask)
72		− return;
73		−
74		− /* set to a specific size? */
75		− if (force_vram_bar_size) {
76		− u32 bar_size_bit;
77		−
78		− rebar_size = force_vram_bar_size * (resource_size_t)SZ_1M;
79		−
80		− bar_size_bit = bar_size_mask & BIT(pci_rebar_bytes_to_size(rebar_size));
81		−
82		− if (!bar_size_bit) {
83		− drm_info(&xe->drm,
84		− "Requested size: %lluMiB is not supported by rebar sizes: 0x%x. Leaving default: %lluMiB\n",
85		− (u64)rebar_size >> 20, bar_size_mask, (u64)current_size >> 20);
86		− return;
87		− }
88		−
89		− rebar_size = 1ULL << (__fls(bar_size_bit) + BAR_SIZE_SHIFT);
90		−
91		− if (rebar_size == current_size)
92		− return;
93		− } else {
94		− rebar_size = 1ULL << (__fls(bar_size_mask) + BAR_SIZE_SHIFT);
95		−
96		− /* only resize if larger than current */
97		− if (rebar_size <= current_size)
98		− return;
99		− }
100		−
101		− drm_info(&xe->drm, "Attempting to resize bar from %lluMiB -> %lluMiB\n",
102		− (u64)current_size >> 20, (u64)rebar_size >> 20);
103		−
104		− while (root->parent)
105		− root = root->parent;
106		−
107		− pci_bus_for_each_resource(root, root_res, i) {
108		− if (root_res && root_res->flags & (IORESOURCE_MEM \| IORESOURCE_MEM_64) &&
109		− (u64)root_res->start > 0x100000000ul)
110		− break;
111		− }
112		−
113		− if (!root_res) {
114		− drm_info(&xe->drm, "Can't resize VRAM BAR - platform support is missing. Consider enabling 'Resizable BAR' support in your BIOS\n");
115		− return;
116		− }
117		−
118		− pci_read_config_dword(pdev, PCI_COMMAND, &pci_cmd);
119		− pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd & ~PCI_COMMAND_MEMORY);
120		−
121		− _resize_bar(xe, LMEM_BAR, rebar_size);
122		−
123		− pci_assign_unassigned_bus_resources(pdev->bus);
124		− pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd);
125		−}
126		−
127		−static bool xe_pci_resource_valid(struct pci_dev *pdev, int bar)
128		−{
129		− if (!pci_resource_flags(pdev, bar))
130		− return false;
131		−
132		− if (pci_resource_flags(pdev, bar) & IORESOURCE_UNSET)
133		− return false;
134		−
135		− if (!pci_resource_len(pdev, bar))
136		− return false;
137		−
138		− return true;
139		−}
140		−
141		−static int xe_determine_lmem_bar_size(struct xe_device *xe)
142		−{
143		− struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
144		−
145		− if (!xe_pci_resource_valid(pdev, LMEM_BAR)) {
146		− drm_err(&xe->drm, "pci resource is not valid\n");
147		− return -ENXIO;
148		− }
149		−
150		− xe_resize_vram_bar(xe);
151		−
152		− xe->mem.vram.io_start = pci_resource_start(pdev, LMEM_BAR);
153		− xe->mem.vram.io_size = pci_resource_len(pdev, LMEM_BAR);
154		− if (!xe->mem.vram.io_size)
155		− return -EIO;
156		−
157		− /* XXX: Need to change when xe link code is ready */
158		− xe->mem.vram.dpa_base = 0;
159		−
160		− /* set up a map to the total memory area. */
161		− xe->mem.vram.mapping = ioremap_wc(xe->mem.vram.io_start, xe->mem.vram.io_size);
162		−
163		− return 0;
164		−}
165		−
166		−static inline u64 get_flat_ccs_offset(struct xe_gt *gt, u64 tile_size)
167		−{
168		− struct xe_device *xe = gt_to_xe(gt);
169		− u64 offset;
170		− u32 reg;
171		−
172		− if (GRAPHICS_VER(xe) >= 20) {
173		− u64 ccs_size = tile_size / 512;
174		− u64 offset_hi, offset_lo;
175		− u32 nodes, num_enabled;
176		−
177		− reg = xe_mmio_read32(gt, MIRROR_FUSE3);
178		− nodes = REG_FIELD_GET(XE2_NODE_ENABLE_MASK, reg);
179		− num_enabled = hweight32(nodes); /* Number of enabled l3 nodes */
180		−
181		− reg = xe_gt_mcr_unicast_read_any(gt, XE2_FLAT_CCS_BASE_RANGE_LOWER);
182		− offset_lo = REG_FIELD_GET(XE2_FLAT_CCS_BASE_LOWER_ADDR_MASK, reg);
183		−
184		− reg = xe_gt_mcr_unicast_read_any(gt, XE2_FLAT_CCS_BASE_RANGE_UPPER);
185		− offset_hi = REG_FIELD_GET(XE2_FLAT_CCS_BASE_UPPER_ADDR_MASK, reg);
186		−
187		− offset = offset_hi << 32; /* HW view bits 39:32 */
188		− offset \|= offset_lo << 6; /* HW view bits 31:6 */
189		− offset = num_enabled; / convert to SW view */
190		−
191		− /* We don't expect any holes */
192		− xe_assert_msg(xe, offset == (xe_mmio_read64_2x32(gt, GSMBASE) - ccs_size),
193		− "Hole between CCS and GSM.\n");
194		− } else {
195		− reg = xe_gt_mcr_unicast_read_any(gt, XEHP_FLAT_CCS_BASE_ADDR);
196		− offset = (u64)REG_FIELD_GET(XEHP_FLAT_CCS_PTR, reg) * SZ_64K;
197		− }
198		−
199		− return offset;
200		−}
201		−
202		−/**
203		− * xe_mmio_tile_vram_size() - Collect vram size and offset information
204		− * @tile: tile to get info for
205		− * @vram_size: available vram (size - device reserved portions)
206		− * @tile_size: actual vram size
207		− * @tile_offset: physical start point in the vram address space
208		− *
209		− * There are 4 places for size information:
210		− * - io size (from pci_resource_len of LMEM bar) (only used for small bar and DG1)
211		− * - TILEx size (actual vram size)
212		− * - GSMBASE offset (TILEx - "stolen")
213		− * - CSSBASE offset (TILEx - CSS space necessary)
214		− *
215		− * CSSBASE is always a lower/smaller offset then GSMBASE.
216		− *
217		− * The actual available size of memory is to the CCS or GSM base.
218		− * NOTE: multi-tile bases will include the tile offset.
219		− *
220		− */
221		−static int xe_mmio_tile_vram_size(struct xe_tile tile, u64 vram_size,
222		− u64 tile_size, u64 tile_offset)
223		−{
224		− struct xe_device *xe = tile_to_xe(tile);
225		− struct xe_gt *gt = tile->primary_gt;
226		− u64 offset;
227		− int err;
228		− u32 reg;
229		−
230		− err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
231		− if (err)
232		− return err;
233		−
234		− /* actual size */
235		− if (unlikely(xe->info.platform == XE_DG1)) {
236		− *tile_size = pci_resource_len(to_pci_dev(xe->drm.dev), LMEM_BAR);
237		− *tile_offset = 0;
238		− } else {
239		− reg = xe_gt_mcr_unicast_read_any(gt, XEHP_TILE_ADDR_RANGE(gt->info.id));
240		− tile_size = (u64)REG_FIELD_GET(GENMASK(14, 8), reg) SZ_1G;
241		− tile_offset = (u64)REG_FIELD_GET(GENMASK(7, 1), reg) SZ_1G;
242		− }
243		−
244		− /* minus device usage */
245		− if (xe->info.has_flat_ccs) {
246		− offset = get_flat_ccs_offset(gt, *tile_size);
247		− } else {
248		− offset = xe_mmio_read64_2x32(gt, GSMBASE);
249		− }
250		−
251		− /* remove the tile offset so we have just the available size */
252		− vram_size = offset - tile_offset;
253		−
254		− return xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
255		−}
256	24
257		−int xe_mmio_probe_vram(struct xe_device *xe)
	25	+static void tiles_fini(void *arg)
258	26	{
	27	+ struct xe_device *xe = arg;
259	28	struct xe_tile *tile;
260		− resource_size_t io_size;
261		− u64 available_size = 0;
262		− u64 total_size = 0;
263		− u64 tile_offset;
264		− u64 tile_size;
265		− u64 vram_size;
266		− int err;
267		− u8 id;
268		−
269		− if (!IS_DGFX(xe))
270		− return 0;
271		−
272		− /* Get the size of the root tile's vram for later accessibility comparison */
273		− tile = xe_device_get_root_tile(xe);
274		− err = xe_mmio_tile_vram_size(tile, &vram_size, &tile_size, &tile_offset);
275		− if (err)
276		− return err;
277		−
278		− err = xe_determine_lmem_bar_size(xe);
279		− if (err)
280		− return err;
281		−
282		− drm_info(&xe->drm, "VISIBLE VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
283		− &xe->mem.vram.io_size);
284		−
285		− io_size = xe->mem.vram.io_size;
286		−
287		− /* tile specific ranges */
288		− for_each_tile(tile, xe, id) {
289		− err = xe_mmio_tile_vram_size(tile, &vram_size, &tile_size, &tile_offset);
290		− if (err)
291		− return err;
292		−
293		− tile->mem.vram.actual_physical_size = tile_size;
294		− tile->mem.vram.io_start = xe->mem.vram.io_start + tile_offset;
295		− tile->mem.vram.io_size = min_t(u64, vram_size, io_size);
296		−
297		− if (!tile->mem.vram.io_size) {
298		− drm_err(&xe->drm, "Tile without any CPU visible VRAM. Aborting.\n");
299		− return -ENODEV;
300		− }
301		−
302		− tile->mem.vram.dpa_base = xe->mem.vram.dpa_base + tile_offset;
303		− tile->mem.vram.usable_size = vram_size;
304		− tile->mem.vram.mapping = xe->mem.vram.mapping + tile_offset;
305		−
306		− if (tile->mem.vram.io_size < tile->mem.vram.usable_size)
307		− drm_info(&xe->drm, "Small BAR device\n");
308		− drm_info(&xe->drm, "VRAM[%u, %u]: Actual physical size %pa, usable size exclude stolen %pa, CPU accessible size %pa\n", id,
309		− tile->id, &tile->mem.vram.actual_physical_size, &tile->mem.vram.usable_size, &tile->mem.vram.io_size);
310		− drm_info(&xe->drm, "VRAM[%u, %u]: DPA range: [%pa-%llx], io range: [%pa-%llx]\n", id, tile->id,
311		− &tile->mem.vram.dpa_base, tile->mem.vram.dpa_base + (u64)tile->mem.vram.actual_physical_size,
312		− &tile->mem.vram.io_start, tile->mem.vram.io_start + (u64)tile->mem.vram.io_size);
313		−
314		− /* calculate total size using tile size to get the correct HW sizing */
315		− total_size += tile_size;
316		− available_size += vram_size;
317		−
318		− if (total_size > xe->mem.vram.io_size) {
319		− drm_info(&xe->drm, "VRAM: %pa is larger than resource %pa\n",
320		− &total_size, &xe->mem.vram.io_size);
321		− }
322		−
323		− io_size -= min_t(u64, tile_size, io_size);
324		− }
325		−
326		− xe->mem.vram.actual_physical_size = total_size;
327		−
328		− drm_info(&xe->drm, "Total VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
329		− &xe->mem.vram.actual_physical_size);
330		− drm_info(&xe->drm, "Available VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
331		− &available_size);
	29	+ int id;
332	30
333		− return 0;
	31	+ for_each_tile(tile, xe, id)
	32	+ tile->mmio.regs = NULL;
334	33	}
335	34
336		−void xe_mmio_probe_tiles(struct xe_device *xe)
	35	+int xe_mmio_probe_tiles(struct xe_device *xe)
337	36	{
338	37	size_t tile_mmio_size = SZ_16M, tile_mmio_ext_size = xe->info.tile_mmio_ext_size;
339	38	u8 id, tile_count = xe->info.tile_count;
@@ -384,15 +83,16 @@ add_mmio_ext: · 44 unmodified lines
384	83	regs += tile_mmio_ext_size;
385	84	}
386	85	}
	86	+
	87	+ return devm_add_action_or_reset(xe->drm.dev, tiles_fini, xe);
387	88	}
388	89
389		−static void mmio_fini(struct drm_device drm, void arg)
	90	+static void mmio_fini(void *arg)
390	91	{
391	92	struct xe_device *xe = arg;
392	93
393	94	pci_iounmap(to_pci_dev(xe->drm.dev), xe->mmio.regs);
394		− if (xe->mem.vram.mapping)
395		− iounmap(xe->mem.vram.mapping);
	95	+ xe->mmio.regs = NULL;
396	96	}
397	97
398	98	int xe_mmio_init(struct xe_device *xe)
@@ -407,7 +107,7 @@ int xe_mmio_init(struct xe_device *xe) · 8 unmodified lines
407	107	* registers (0-4MB), reserved space (4MB-8MB) and GGTT (8MB-16MB).
408	108	*/
409	109	xe->mmio.size = pci_resource_len(pdev, mmio_bar);
410		− xe->mmio.regs = pci_iomap(pdev, mmio_bar, 0);
	110	+ xe->mmio.regs = pci_iomap(pdev, mmio_bar, GTTMMADR_BAR);
411	111	if (xe->mmio.regs == NULL) {
412	112	drm_err(&xe->drm, "failed to map registers\n");
413	113	return -EIO;
@@ -417,47 +117,42 @@ int xe_mmio_init(struct xe_device *xe) · 3 unmodified lines
417	117	root_tile->mmio.size = SZ_16M;
418	118	root_tile->mmio.regs = xe->mmio.regs;
419	119
420		− return drmm_add_action_or_reset(&xe->drm, mmio_fini, xe);
	120	+ return devm_add_action_or_reset(xe->drm.dev, mmio_fini, xe);
421	121	}
422	122
423	123	u8 xe_mmio_read8(struct xe_gt *gt, struct xe_reg reg)
424	124	{
425	125	struct xe_tile *tile = gt_to_tile(gt);
	126	+ u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
426	127
427		− if (reg.addr < gt->mmio.adj_limit)
428		− reg.addr += gt->mmio.adj_offset;
429		−
430		− return readb((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + reg.addr);
	128	+ return readb((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr);
431	129	}
432	130
433	131	u16 xe_mmio_read16(struct xe_gt *gt, struct xe_reg reg)
434	132	{
435	133	struct xe_tile *tile = gt_to_tile(gt);
	134	+ u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
436	135
437		− if (reg.addr < gt->mmio.adj_limit)
438		− reg.addr += gt->mmio.adj_offset;
439		−
440		− return readw((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + reg.addr);
	136	+ return readw((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr);
441	137	}
442	138
443	139	void xe_mmio_write32(struct xe_gt *gt, struct xe_reg reg, u32 val)
444	140	{
445	141	struct xe_tile *tile = gt_to_tile(gt);
	142	+ u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
446	143
447		− if (reg.addr < gt->mmio.adj_limit)
448		− reg.addr += gt->mmio.adj_offset;
449		−
450		− writel(val, (reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + reg.addr);
	144	+ writel(val, (reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr);
451	145	}
452	146
453	147	u32 xe_mmio_read32(struct xe_gt *gt, struct xe_reg reg)
454	148	{
455	149	struct xe_tile *tile = gt_to_tile(gt);
	150	+ u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
456	151
457		− if (reg.addr < gt->mmio.adj_limit)
458		− reg.addr += gt->mmio.adj_offset;
	152	+ if (!reg.vf && IS_SRIOV_VF(gt_to_xe(gt)))
	153	+ return xe_gt_sriov_vf_read32(gt, reg);
459	154
460		− return readl((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + reg.addr);
	155	+ return readl((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr);
461	156	}
462	157
463	158	u32 xe_mmio_rmw32(struct xe_gt *gt, struct xe_reg reg, u32 clr, u32 set)
@@ -486,10 +181,9 @@ bool xe_mmio_in_range(const struct xe_gt *gt, · 22 unmodified lines
486	181	const struct xe_mmio_range *range,
487	182	struct xe_reg reg)
488	183	{
489		− if (reg.addr < gt->mmio.adj_limit)
490		− reg.addr += gt->mmio.adj_offset;
	184	+ u32 addr = xe_mmio_adjusted_addr(gt, reg.addr);
491	185
492		− return range && reg.addr >= range->start && reg.addr <= range->end;
	186	+ return range && addr >= range->start && addr <= range->end;
493	187	}
494	188
495	189	/**
@@ -519,10 +213,11 @@ u64 xe_mmio_read64_2x32(struct xe_gt *gt, struct xe_reg reg) · 23 unmodified lines
519	213	struct xe_reg reg_udw = { .addr = reg.addr + 0x4 };
520	214	u32 ldw, udw, oldudw, retries;
521	215
522		− if (reg.addr < gt->mmio.adj_limit) {
523		− reg.addr += gt->mmio.adj_offset;
524		− reg_udw.addr += gt->mmio.adj_offset;
525		− }
	216	+ reg.addr = xe_mmio_adjusted_addr(gt, reg.addr);
	217	+ reg_udw.addr = xe_mmio_adjusted_addr(gt, reg_udw.addr);
	218	+
	219	+ /* we shouldn't adjust just one register address */
	220	+ xe_gt_assert(gt, reg_udw.addr == reg.addr + 0x4);
526	221
527	222	oldudw = xe_mmio_read32(gt, reg_udw);
528	223	for (retries = 5; retries; --retries) {
@@ -601,3 +296,64 @@ int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 t · 72 unmodified lines
601	296
602	297	return ret;
603	298	}
	299	+
	300	+/**
	301	+ * xe_mmio_wait32_not() - Wait for a register to return anything other than the given masked value
	302	+ * @gt: MMIO target GT
	303	+ * @reg: register to read value from
	304	+ * @mask: mask to be applied to the value read from the register
	305	+ * @val: value to match after applying the mask
	306	+ * @timeout_us: time out after this period of time. Wait logic tries to be
	307	+ * smart, applying an exponential backoff until @timeout_us is reached.
	308	+ * @out_val: if not NULL, points where to store the last unmasked value
	309	+ * @atomic: needs to be true if calling from an atomic context
	310	+ *
	311	+ * This function polls for a masked value to change from a given value and
	312	+ * returns zero on success or -ETIMEDOUT if timed out.
	313	+ *
	314	+ * Note that @timeout_us represents the minimum amount of time to wait before
	315	+ * giving up. The actual time taken by this function can be a little more than
	316	+ * @timeout_us for different reasons, specially in non-atomic contexts. Thus,
	317	+ * it is possible that this function succeeds even after @timeout_us has passed.
	318	+ */
	319	+int xe_mmio_wait32_not(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 timeout_us,
	320	+ u32 *out_val, bool atomic)
	321	+{
	322	+ ktime_t cur = ktime_get_raw();
	323	+ const ktime_t end = ktime_add_us(cur, timeout_us);
	324	+ int ret = -ETIMEDOUT;
	325	+ s64 wait = 10;
	326	+ u32 read;
	327	+
	328	+ for (;;) {
	329	+ read = xe_mmio_read32(gt, reg);
	330	+ if ((read & mask) != val) {
	331	+ ret = 0;
	332	+ break;
	333	+ }
	334	+
	335	+ cur = ktime_get_raw();
	336	+ if (!ktime_before(cur, end))
	337	+ break;
	338	+
	339	+ if (ktime_after(ktime_add_us(cur, wait), end))
	340	+ wait = ktime_us_delta(end, cur);
	341	+
	342	+ if (atomic)
	343	+ udelay(wait);
	344	+ else
	345	+ usleep_range(wait, wait << 1);
	346	+ wait <<= 1;
	347	+ }
	348	+
	349	+ if (ret != 0) {
	350	+ read = xe_mmio_read32(gt, reg);
	351	+ if ((read & mask) != val)
	352	+ ret = 0;
	353	+ }
	354	+
	355	+ if (out_val)
	356	+ *out_val = read;
	357	+
	358	+ return ret;
	359	+}

drivers/gpu/drm/xe/xe_mmio.h+11−11

@@ -6,22 +6,13 @@ · 5 unmodified lines
6	6	#ifndef _XE_MMIO_H_
7	7	#define _XE_MMIO_H_
8	8
9		−#include <linux/delay.h>
10		−#include <linux/io-64-nonatomic-lo-hi.h>
11		−
12		−#include "regs/xe_reg_defs.h"
13		−#include "xe_device_types.h"
14		−#include "xe_gt_printk.h"
15	9	#include "xe_gt_types.h"
16	10
17		−struct drm_device;
18		−struct drm_file;
19	11	struct xe_device;
20		−
21		−#define LMEM_BAR 2
	12	+struct xe_reg;
22	13
23	14	int xe_mmio_init(struct xe_device *xe);
24		−void xe_mmio_probe_tiles(struct xe_device *xe);
	15	+int xe_mmio_probe_tiles(struct xe_device *xe);
25	16
26	17	u8 xe_mmio_read8(struct xe_gt *gt, struct xe_reg reg);
27	18	u16 xe_mmio_read16(struct xe_gt *gt, struct xe_reg reg);
@@ -35,5 +26,14 @@ int xe_mmio_probe_vram(struct xe_device *xe); · 7 unmodified lines
35	26	u64 xe_mmio_read64_2x32(struct xe_gt *gt, struct xe_reg reg);
36	27	int xe_mmio_wait32(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 timeout_us,
37	28	u32 *out_val, bool atomic);
	29	+int xe_mmio_wait32_not(struct xe_gt *gt, struct xe_reg reg, u32 mask, u32 val, u32 timeout_us,
	30	+ u32 *out_val, bool atomic);
	31	+
	32	+static inline u32 xe_mmio_adjusted_addr(const struct xe_gt *gt, u32 addr)
	33	+{
	34	+ if (addr < gt->mmio.adj_limit)
	35	+ addr += gt->mmio.adj_offset;
	36	+ return addr;
	37	+}
38	38
39	39	#endif

drivers/gpu/drm/xe/xe_mocs.c+260−41

@@ -9,10 +9,13 @@ · 8 unmodified lines
9	9	#include "xe_bo.h"
10	10	#include "xe_device.h"
11	11	#include "xe_exec_queue.h"
	12	+#include "xe_force_wake.h"
12	13	#include "xe_gt.h"
13	14	#include "xe_gt_mcr.h"
	15	+#include "xe_gt_printk.h"
14	16	#include "xe_mmio.h"
15	17	#include "xe_platform_types.h"
	18	+#include "xe_pm.h"
16	19	#include "xe_sriov.h"
17	20	#include "xe_step_types.h"
18	21
@@ -36,34 +39,23 @@ struct xe_mocs_entry { · 17 unmodified lines
36	39	u16 used;
37	40	};
38	41
	42	+struct xe_mocs_info;
	43	+
	44	+struct xe_mocs_ops {
	45	+ void (dump)(struct xe_mocs_info mocs, unsigned int flags,
	46	+ struct xe_gt gt, struct drm_printer p);
	47	+};
	48	+
39	49	struct xe_mocs_info {
40	50	unsigned int size;
41	51	unsigned int n_entries;
42	52	const struct xe_mocs_entry *table;
	53	+ const struct xe_mocs_ops *ops;
43	54	u8 uc_index;
44	55	u8 wb_index;
45	56	u8 unused_entries_index;
46	57	};
47	58
48		−/* Defines for the tables (XXX_MOCS_0 - XXX_MOCS_63) */
49		−#define _LE_CACHEABILITY(value) ((value) << 0)
50		−#define _LE_TGT_CACHE(value) ((value) << 2)
51		−#define LE_LRUM(value) ((value) << 4)
52		−#define LE_AOM(value) ((value) << 6)
53		−#define LE_RSC(value) ((value) << 7)
54		−#define LE_SCC(value) ((value) << 8)
55		−#define LE_PFM(value) ((value) << 11)
56		−#define LE_SCF(value) ((value) << 14)
57		−#define LE_COS(value) ((value) << 15)
58		−#define LE_SSE(value) ((value) << 17)
59		−
60		−/* Defines for the tables (LNCFMOCS0 - LNCFMOCS31) - two entries per word */
61		−#define L3_ESC(value) ((value) << 0)
62		−#define L3_SCC(value) ((value) << 1)
63		−#define _L3_CACHEABILITY(value) ((value) << 4)
64		−#define L3_GLBGO(value) ((value) << 6)
65		−#define L3_LKUP(value) ((value) << 7)
66		−
67	59	/* Defines for the tables (GLOB_MOCS_0 - GLOB_MOCS_16) */
68	60	#define IG_PAT REG_BIT(8)
69	61	#define L3_CACHE_POLICY_MASK REG_GENMASK(5, 4)
@@ -80,22 +72,22 @@ struct xe_mocs_info { · 10 unmodified lines
80	72	* Note: LE_0_PAGETABLE works only up to Gen11; for newer gens it means
81	73	* the same as LE_UC
82	74	*/
83		−#define LE_0_PAGETABLE _LE_CACHEABILITY(0)
84		−#define LE_1_UC _LE_CACHEABILITY(1)
85		−#define LE_2_WT _LE_CACHEABILITY(2)
86		−#define LE_3_WB _LE_CACHEABILITY(3)
	75	+#define LE_0_PAGETABLE LE_CACHEABILITY(0)
	76	+#define LE_1_UC LE_CACHEABILITY(1)
	77	+#define LE_2_WT LE_CACHEABILITY(2)
	78	+#define LE_3_WB LE_CACHEABILITY(3)
87	79
88	80	/* Target cache */
89		−#define LE_TC_0_PAGETABLE _LE_TGT_CACHE(0)
90		−#define LE_TC_1_LLC _LE_TGT_CACHE(1)
91		−#define LE_TC_2_LLC_ELLC _LE_TGT_CACHE(2)
92		−#define LE_TC_3_LLC_ELLC_ALT _LE_TGT_CACHE(3)
	81	+#define LE_TC_0_PAGETABLE LE_TGT_CACHE(0)
	82	+#define LE_TC_1_LLC LE_TGT_CACHE(1)
	83	+#define LE_TC_2_LLC_ELLC LE_TGT_CACHE(2)
	84	+#define LE_TC_3_LLC_ELLC_ALT LE_TGT_CACHE(3)
93	85
94	86	/* L3 caching options */
95		−#define L3_0_DIRECT _L3_CACHEABILITY(0)
96		−#define L3_1_UC _L3_CACHEABILITY(1)
97		−#define L3_2_RESERVED _L3_CACHEABILITY(2)
98		−#define L3_3_WB _L3_CACHEABILITY(3)
	87	+#define L3_0_DIRECT L3_CACHEABILITY(0)
	88	+#define L3_1_UC L3_CACHEABILITY(1)
	89	+#define L3_2_RESERVED L3_CACHEABILITY(2)
	90	+#define L3_3_WB L3_CACHEABILITY(3)
99	91
100	92	/* L4 caching options */
101	93	#define L4_0_WB REG_FIELD_PREP(L4_CACHE_POLICY_MASK, 0)
@@ -107,6 +99,8 @@ struct xe_mocs_info { · 5 unmodified lines
107	99	#define XE2_L3_1_XD REG_FIELD_PREP(L3_CACHE_POLICY_MASK, 1)
108	100	#define XE2_L3_3_UC REG_FIELD_PREP(L3_CACHE_POLICY_MASK, 3)
109	101
	102	+#define XE2_L3_CLOS_MASK REG_GENMASK(7, 6)
	103	+
110	104	#define MOCS_ENTRY(__idx, __control_value, __l3cc_value) \
111	105	[__idx] = { \
112	106	.control_value = __control_value, \
@@ -255,6 +249,84 @@ static const struct xe_mocs_entry gen12_mocs_desc[] = { · 142 unmodified lines
255	249	L3_1_UC)
256	250	};
257	251
	252	+static bool regs_are_mcr(struct xe_gt *gt)
	253	+{
	254	+ struct xe_device *xe = gt_to_xe(gt);
	255	+
	256	+ if (xe_gt_is_media_type(gt))
	257	+ return MEDIA_VER(xe) >= 20;
	258	+ else
	259	+ return GRAPHICS_VERx100(xe) >= 1250;
	260	+}
	261	+
	262	+static void xelp_lncf_dump(struct xe_mocs_info info, struct xe_gt gt, struct drm_printer *p)
	263	+{
	264	+ unsigned int i, j;
	265	+ u32 reg_val;
	266	+
	267	+ drm_printf(p, "LNCFCMOCS[idx] = [ESC, SCC, L3CC] (value)\n\n");
	268	+
	269	+ for (i = 0, j = 0; i < (info->n_entries + 1) / 2; i++, j++) {
	270	+ if (regs_are_mcr(gt))
	271	+ reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_LNCFCMOCS(i));
	272	+ else
	273	+ reg_val = xe_mmio_read32(gt, XELP_LNCFCMOCS(i));
	274	+
	275	+ drm_printf(p, "LNCFCMOCS[%2d] = [%u, %u, %u] (%#8x)\n",
	276	+ j++,
	277	+ !!(reg_val & L3_ESC_MASK),
	278	+ REG_FIELD_GET(L3_SCC_MASK, reg_val),
	279	+ REG_FIELD_GET(L3_CACHEABILITY_MASK, reg_val),
	280	+ reg_val);
	281	+
	282	+ drm_printf(p, "LNCFCMOCS[%2d] = [%u, %u, %u] (%#8x)\n",
	283	+ j,
	284	+ !!(reg_val & L3_UPPER_IDX_ESC_MASK),
	285	+ REG_FIELD_GET(L3_UPPER_IDX_SCC_MASK, reg_val),
	286	+ REG_FIELD_GET(L3_UPPER_IDX_CACHEABILITY_MASK, reg_val),
	287	+ reg_val);
	288	+ }
	289	+}
	290	+
	291	+static void xelp_mocs_dump(struct xe_mocs_info *info, unsigned int flags,
	292	+ struct xe_gt gt, struct drm_printer p)
	293	+{
	294	+ unsigned int i;
	295	+ u32 reg_val;
	296	+
	297	+ if (flags & HAS_GLOBAL_MOCS) {
	298	+ drm_printf(p, "Global mocs table configuration:\n");
	299	+ drm_printf(p, "GLOB_MOCS[idx] = [LeCC, TC, LRUM, AOM, RSC, SCC, PFM, SCF, CoS, SSE] (value)\n\n");
	300	+
	301	+ for (i = 0; i < info->n_entries; i++) {
	302	+ if (regs_are_mcr(gt))
	303	+ reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_GLOBAL_MOCS(i));
	304	+ else
	305	+ reg_val = xe_mmio_read32(gt, XELP_GLOBAL_MOCS(i));
	306	+
	307	+ drm_printf(p, "GLOB_MOCS[%2d] = [%u, %u, %u, %u, %u, %u, %u, %u, %u, %u ] (%#8x)\n",
	308	+ i,
	309	+ REG_FIELD_GET(LE_CACHEABILITY_MASK, reg_val),
	310	+ REG_FIELD_GET(LE_TGT_CACHE_MASK, reg_val),
	311	+ REG_FIELD_GET(LE_LRUM_MASK, reg_val),
	312	+ !!(reg_val & LE_AOM_MASK),
	313	+ !!(reg_val & LE_RSC_MASK),
	314	+ REG_FIELD_GET(LE_SCC_MASK, reg_val),
	315	+ REG_FIELD_GET(LE_PFM_MASK, reg_val),
	316	+ !!(reg_val & LE_SCF_MASK),
	317	+ REG_FIELD_GET(LE_COS_MASK, reg_val),
	318	+ REG_FIELD_GET(LE_SSE_MASK, reg_val),
	319	+ reg_val);
	320	+ }
	321	+ }
	322	+
	323	+ xelp_lncf_dump(info, gt, p);
	324	+}
	325	+
	326	+static const struct xe_mocs_ops xelp_mocs_ops = {
	327	+ .dump = xelp_mocs_dump,
	328	+};
	329	+
258	330	static const struct xe_mocs_entry dg1_mocs_desc[] = {
259	331	/* UC */
260	332	MOCS_ENTRY(1, 0, L3_1_UC),
@@ -291,6 +363,40 @@ static const struct xe_mocs_entry dg2_mocs_desc[] = { · 30 unmodified lines
291	363	MOCS_ENTRY(3, 0, L3_3_WB \| L3_LKUP(1)),
292	364	};
293	365
	366	+static void xehp_lncf_dump(struct xe_mocs_info *info, unsigned int flags,
	367	+ struct xe_gt gt, struct drm_printer p)
	368	+{
	369	+ unsigned int i, j;
	370	+ u32 reg_val;
	371	+
	372	+ drm_printf(p, "LNCFCMOCS[idx] = [UCL3LOOKUP, GLBGO, L3CC] (value)\n\n");
	373	+
	374	+ for (i = 0, j = 0; i < (info->n_entries + 1) / 2; i++, j++) {
	375	+ if (regs_are_mcr(gt))
	376	+ reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_LNCFCMOCS(i));
	377	+ else
	378	+ reg_val = xe_mmio_read32(gt, XELP_LNCFCMOCS(i));
	379	+
	380	+ drm_printf(p, "LNCFCMOCS[%2d] = [%u, %u, %u] (%#8x)\n",
	381	+ j++,
	382	+ !!(reg_val & L3_LKUP_MASK),
	383	+ !!(reg_val & L3_GLBGO_MASK),
	384	+ REG_FIELD_GET(L3_CACHEABILITY_MASK, reg_val),
	385	+ reg_val);
	386	+
	387	+ drm_printf(p, "LNCFCMOCS[%2d] = [%u, %u, %u] (%#8x)\n",
	388	+ j,
	389	+ !!(reg_val & L3_UPPER_LKUP_MASK),
	390	+ !!(reg_val & L3_UPPER_GLBGO_MASK),
	391	+ REG_FIELD_GET(L3_UPPER_IDX_CACHEABILITY_MASK, reg_val),
	392	+ reg_val);
	393	+ }
	394	+}
	395	+
	396	+static const struct xe_mocs_ops xehp_mocs_ops = {
	397	+ .dump = xehp_lncf_dump,
	398	+};
	399	+
294	400	static const struct xe_mocs_entry pvc_mocs_desc[] = {
295	401	/* Error */
296	402	MOCS_ENTRY(0, 0, L3_3_WB),
@@ -302,6 +408,36 @@ static const struct xe_mocs_entry pvc_mocs_desc[] = { · 5 unmodified lines
302	408	MOCS_ENTRY(2, 0, L3_3_WB),
303	409	};
304	410
	411	+static void pvc_mocs_dump(struct xe_mocs_info info, unsigned int flags, struct xe_gt gt,
	412	+ struct drm_printer *p)
	413	+{
	414	+ unsigned int i, j;
	415	+ u32 reg_val;
	416	+
	417	+ drm_printf(p, "LNCFCMOCS[idx] = [ L3CC ] (value)\n\n");
	418	+
	419	+ for (i = 0, j = 0; i < (info->n_entries + 1) / 2; i++, j++) {
	420	+ if (regs_are_mcr(gt))
	421	+ reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_LNCFCMOCS(i));
	422	+ else
	423	+ reg_val = xe_mmio_read32(gt, XELP_LNCFCMOCS(i));
	424	+
	425	+ drm_printf(p, "LNCFCMOCS[%2d] = [ %u ] (%#8x)\n",
	426	+ j++,
	427	+ REG_FIELD_GET(L3_CACHEABILITY_MASK, reg_val),
	428	+ reg_val);
	429	+
	430	+ drm_printf(p, "LNCFCMOCS[%2d] = [ %u ] (%#8x)\n",
	431	+ j,
	432	+ REG_FIELD_GET(L3_UPPER_IDX_CACHEABILITY_MASK, reg_val),
	433	+ reg_val);
	434	+ }
	435	+}
	436	+
	437	+static const struct xe_mocs_ops pvc_mocs_ops = {
	438	+ .dump = pvc_mocs_dump,
	439	+};
	440	+
305	441	static const struct xe_mocs_entry mtl_mocs_desc[] = {
306	442	/* Error - Reserved for Non-Use */
307	443	MOCS_ENTRY(0,
@@ -353,6 +489,36 @@ static const struct xe_mocs_entry mtl_mocs_desc[] = { · 45 unmodified lines
353	489	L3_GLBGO(1) \| L3_1_UC),
354	490	};
355	491
	492	+static void mtl_mocs_dump(struct xe_mocs_info *info, unsigned int flags,
	493	+ struct xe_gt gt, struct drm_printer p)
	494	+{
	495	+ unsigned int i;
	496	+ u32 reg_val;
	497	+
	498	+ drm_printf(p, "Global mocs table configuration:\n");
	499	+ drm_printf(p, "GLOB_MOCS[idx] = [IG_PAT, L4_CACHE_POLICY] (value)\n\n");
	500	+
	501	+ for (i = 0; i < info->n_entries; i++) {
	502	+ if (regs_are_mcr(gt))
	503	+ reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_GLOBAL_MOCS(i));
	504	+ else
	505	+ reg_val = xe_mmio_read32(gt, XELP_GLOBAL_MOCS(i));
	506	+
	507	+ drm_printf(p, "GLOB_MOCS[%2d] = [%u, %u] (%#8x)\n",
	508	+ i,
	509	+ !!(reg_val & IG_PAT),
	510	+ REG_FIELD_GET(L4_CACHE_POLICY_MASK, reg_val),
	511	+ reg_val);
	512	+ }
	513	+
	514	+ /* MTL lncf mocs table pattern is similar to that of xehp */
	515	+ xehp_lncf_dump(info, flags, gt, p);
	516	+}
	517	+
	518	+static const struct xe_mocs_ops mtl_mocs_ops = {
	519	+ .dump = mtl_mocs_dump,
	520	+};
	521	+
356	522	static const struct xe_mocs_entry xe2_mocs_table[] = {
357	523	/* Defer to PAT */
358	524	MOCS_ENTRY(0, XE2_L3_0_WB \| L4_3_UC, 0),
@@ -366,6 +532,34 @@ static const struct xe_mocs_entry xe2_mocs_table[] = { · 7 unmodified lines
366	532	MOCS_ENTRY(4, IG_PAT \| XE2_L3_0_WB \| L4_0_WB, 0),
367	533	};
368	534
	535	+static void xe2_mocs_dump(struct xe_mocs_info *info, unsigned int flags,
	536	+ struct xe_gt gt, struct drm_printer p)
	537	+{
	538	+ unsigned int i;
	539	+ u32 reg_val;
	540	+
	541	+ drm_printf(p, "Global mocs table configuration:\n");
	542	+ drm_printf(p, "GLOB_MOCS[idx] = [IG_PAT, L3_CLOS, L3_CACHE_POLICY, L4_CACHE_POLICY] (value)\n\n");
	543	+
	544	+ for (i = 0; i < info->n_entries; i++) {
	545	+ if (regs_are_mcr(gt))
	546	+ reg_val = xe_gt_mcr_unicast_read_any(gt, XEHP_GLOBAL_MOCS(i));
	547	+ else
	548	+ reg_val = xe_mmio_read32(gt, XELP_GLOBAL_MOCS(i));
	549	+
	550	+ drm_printf(p, "GLOB_MOCS[%2d] = [%u, %u, %u] (%#8x)\n",
	551	+ i,
	552	+ !!(reg_val & IG_PAT),
	553	+ REG_FIELD_GET(XE2_L3_CLOS_MASK, reg_val),
	554	+ REG_FIELD_GET(L4_CACHE_POLICY_MASK, reg_val),
	555	+ reg_val);
	556	+ }
	557	+}
	558	+
	559	+static const struct xe_mocs_ops xe2_mocs_ops = {
	560	+ .dump = xe2_mocs_dump,
	561	+};
	562	+
369	563	static unsigned int get_mocs_settings(struct xe_device *xe,
370	564	struct xe_mocs_info *info)
371	565	{
@@ -376,6 +570,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 4 unmodified lines
376	570	switch (xe->info.platform) {
377	571	case XE_LUNARLAKE:
378	572	case XE_BATTLEMAGE:
	573	+ info->ops = &xe2_mocs_ops;
379	574	info->size = ARRAY_SIZE(xe2_mocs_table);
380	575	info->table = xe2_mocs_table;
381	576	info->n_entries = XE2_NUM_MOCS_ENTRIES;
@@ -384,6 +579,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 2 unmodified lines
384	579	info->unused_entries_index = 4;
385	580	break;
386	581	case XE_PVC:
	582	+ info->ops = &pvc_mocs_ops;
387	583	info->size = ARRAY_SIZE(pvc_mocs_desc);
388	584	info->table = pvc_mocs_desc;
389	585	info->n_entries = PVC_NUM_MOCS_ENTRIES;
@@ -392,6 +588,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 2 unmodified lines
392	588	info->unused_entries_index = 2;
393	589	break;
394	590	case XE_METEORLAKE:
	591	+ info->ops = &mtl_mocs_ops;
395	592	info->size = ARRAY_SIZE(mtl_mocs_desc);
396	593	info->table = mtl_mocs_desc;
397	594	info->n_entries = MTL_NUM_MOCS_ENTRIES;
@@ -399,6 +596,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 1 unmodified line
399	596	info->unused_entries_index = 1;
400	597	break;
401	598	case XE_DG2:
	599	+ info->ops = &xehp_mocs_ops;
402	600	info->size = ARRAY_SIZE(dg2_mocs_desc);
403	601	info->table = dg2_mocs_desc;
404	602	info->uc_index = 1;
@@ -410,6 +608,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 5 unmodified lines
410	608	info->unused_entries_index = 3;
411	609	break;
412	610	case XE_DG1:
	611	+ info->ops = &xelp_mocs_ops;
413	612	info->size = ARRAY_SIZE(dg1_mocs_desc);
414	613	info->table = dg1_mocs_desc;
415	614	info->uc_index = 1;
@@ -421,6 +620,7 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 5 unmodified lines
421	620	case XE_ALDERLAKE_S:
422	621	case XE_ALDERLAKE_P:
423	622	case XE_ALDERLAKE_N:
	623	+ info->ops = &xelp_mocs_ops;
424	624	info->size = ARRAY_SIZE(gen12_mocs_desc);
425	625	info->table = gen12_mocs_desc;
426	626	info->n_entries = XELP_NUM_MOCS_ENTRIES;
@@ -442,6 +642,8 @@ static unsigned int get_mocs_settings(struct xe_device *xe, · 15 unmodified lines
442	642	*/
443	643	xe_assert(xe, info->unused_entries_index != 0);
444	644
	645	+ xe_assert(xe, !info->ops \|\| info->ops->dump);
	646	+
445	647	if (XE_WARN_ON(info->size > info->n_entries)) {
446	648	info->table = NULL;
447	649	return 0;
@@ -467,16 +669,6 @@ static u32 get_entry_control(const struct xe_mocs_info *info, · 19 unmodified lines
467	669	return info->table[info->unused_entries_index].control_value;
468	670	}
469	671
470		−static bool regs_are_mcr(struct xe_gt *gt)
471		−{
472		− struct xe_device *xe = gt_to_xe(gt);
473		−
474		− if (xe_gt_is_media_type(gt))
475		− return MEDIA_VER(xe) >= 20;
476		− else
477		− return GRAPHICS_VERx100(xe) >= 1250;
478		−}
479		−
480	672	static void __init_mocs_table(struct xe_gt *gt,
481	673	const struct xe_mocs_info *info)
482	674	{
@@ -578,6 +770,33 @@ void xe_mocs_init(struct xe_gt *gt) · 95 unmodified lines
578	770	init_l3cc_table(gt, &table);
579	771	}
580	772
	773	+void xe_mocs_dump(struct xe_gt gt, struct drm_printer p)
	774	+{
	775	+ struct xe_mocs_info table;
	776	+ unsigned int flags;
	777	+ u32 ret;
	778	+ struct xe_device *xe = gt_to_xe(gt);
	779	+
	780	+ flags = get_mocs_settings(xe, &table);
	781	+
	782	+ if (!table.ops->dump)
	783	+ return;
	784	+
	785	+ xe_pm_runtime_get_noresume(xe);
	786	+ ret = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
	787	+
	788	+ if (ret)
	789	+ goto err_fw;
	790	+
	791	+ table.ops->dump(&table, flags, gt, p);
	792	+
	793	+ xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
	794	+
	795	+err_fw:
	796	+ xe_assert(xe, !ret);
	797	+ xe_pm_runtime_put(xe);
	798	+}
	799	+
581	800	#if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)
582	801	#include "tests/xe_mocs.c"
583	802	#endif

drivers/gpu/drm/xe/xe_mocs.h+8−3

@@ -6,12 +6,17 @@ · 5 unmodified lines
6	6	#ifndef _XE_MOCS_H_
7	7	#define _XE_MOCS_H_
8	8
9		−#include <linux/types.h>
10		−
11		−struct xe_exec_queue;
	9	+struct drm_printer;
12	10	struct xe_gt;
13	11
14	12	void xe_mocs_init_early(struct xe_gt *gt);
15	13	void xe_mocs_init(struct xe_gt *gt);
16	14
	15	+/**
	16	+ * xe_mocs_dump - Dump mocs table
	17	+ * @gt: GT structure
	18	+ * @p: Printer to dump info to
	19	+ */
	20	+void xe_mocs_dump(struct xe_gt gt, struct drm_printer p);
	21	+
17	22	#endif

drivers/gpu/drm/xe/xe_module.c+5−0

@@ -17,6 +17,7 @@ struct xe_modparam xe_modparam = { · 16 unmodified lines
17	17	.enable_display = true,
18	18	.guc_log_level = 5,
19	19	.force_probe = CONFIG_DRM_XE_FORCE_PROBE,
	20	+ .wedged_mode = 1,
20	21	/* the rest are 0 by default */
21	22	};
22	23
@@ -55,6 +56,10 @@ MODULE_PARM_DESC(max_vfs, · 32 unmodified lines
55	56	"(0 = no VFs [default]; N = allow up to N VFs)");
56	57	#endif
57	58
	59	+module_param_named_unsafe(wedged_mode, xe_modparam.wedged_mode, int, 0600);
	60	+MODULE_PARM_DESC(wedged_mode,
	61	+ "Module's default policy for the wedged mode - 0=never, 1=upon-critical-errors[default], 2=upon-any-hang");
	62	+
58	63	struct init_funcs {
59	64	int (*init)(void);
60	65	void (*exit)(void);

drivers/gpu/drm/xe/xe_module.h+1−0

@@ -21,6 +21,7 @@ struct xe_modparam { · 20 unmodified lines
21	21	#ifdef CONFIG_PCI_IOV
22	22	unsigned int max_vfs;
23	23	#endif
	24	+ int wedged_mode;
24	25	};
25	26
26	27	extern struct xe_modparam xe_modparam;

drivers/gpu/drm/xe/xe_pat.c+1−0

@@ -10,6 +10,7 @@ · 9 unmodified lines
10	10	#include "regs/xe_reg_defs.h"
11	11	#include "xe_assert.h"
12	12	#include "xe_device.h"
	13	+#include "xe_force_wake.h"
13	14	#include "xe_gt.h"
14	15	#include "xe_gt_mcr.h"
15	16	#include "xe_mmio.h"

drivers/gpu/drm/xe/xe_pci.c+80−26

@@ -20,9 +20,12 @@ · 19 unmodified lines
20	20	#include "xe_device.h"
21	21	#include "xe_drv.h"
22	22	#include "xe_gt.h"
	23	+#include "xe_gt_sriov_vf.h"
	24	+#include "xe_guc.h"
23	25	#include "xe_macros.h"
24	26	#include "xe_mmio.h"
25	27	#include "xe_module.h"
	28	+#include "xe_pci_sriov.h"
26	29	#include "xe_pci_types.h"
27	30	#include "xe_pm.h"
28	31	#include "xe_sriov.h"
@@ -40,12 +43,6 @@ struct xe_subplatform_desc { · 11 unmodified lines
40	43	const u16 *pciidlist;
41	44	};
42	45
43		−struct xe_gt_desc {
44		− enum xe_gt_type type;
45		− u32 mmio_adj_limit;
46		− u32 mmio_adj_offset;
47		−};
48		−
49	46	struct xe_device_desc {
50	47	/* Should only ever be set for platforms without GMD_ID */
51	48	const struct xe_graphics_desc *graphics;
@@ -74,7 +71,7 @@ __diag_push(); · 22 unmodified lines
74	71	__diag_ignore_all("-Woverride-init", "Allow field overrides in table");
75	72
76	73	#define PLATFORM(x) \
77		− .platform = (x), \
	74	+ .platform = XE_##x, \
78	75	.platform_name = #x
79	76
80	77	#define NOP(x) x
@@ -146,6 +143,7 @@ static const struct xe_graphics_desc graphics_xehpc = { · 65 unmodified lines
146	143	.vram_flags = XE_VRAM_FLAGS_NEED64K,
147	144
148	145	.has_asid = 1,
	146	+ .has_atomic_enable_pte_bit = 1,
149	147	.has_flat_ccs = 0,
150	148	.has_usm = 1,
151	149	};
@@ -163,7 +161,9 @@ static const struct xe_graphics_desc graphics_xelpg = { · 11 unmodified lines
163	161	#define XE2_GFX_FEATURES \
164	162	.dma_mask_size = 46, \
165	163	.has_asid = 1, \
	164	+ .has_atomic_enable_pte_bit = 1, \
166	165	.has_flat_ccs = 1, \
	166	+ .has_indirect_ring_state = 1, \
167	167	.has_range_tlb_invalidation = 1, \
168	168	.has_usm = 1, \
169	169	.va_bits = 48, \
@@ -211,13 +211,14 @@ static const struct xe_media_desc media_xe2 = { · 41 unmodified lines
211	211	.name = "Xe2_LPM / Xe2_HPM",
212	212	.hw_engine_mask =
213	213	GENMASK(XE_HW_ENGINE_VCS7, XE_HW_ENGINE_VCS0) \|
214		− GENMASK(XE_HW_ENGINE_VECS3, XE_HW_ENGINE_VECS0), /* TODO: GSC0 */
	214	+ GENMASK(XE_HW_ENGINE_VECS3, XE_HW_ENGINE_VECS0) \|
	215	+ BIT(XE_HW_ENGINE_GSCCS0)
215	216	};
216	217
217	218	static const struct xe_device_desc tgl_desc = {
218	219	.graphics = &graphics_xelp,
219	220	.media = &media_xem,
220		− PLATFORM(XE_TIGERLAKE),
	221	+ PLATFORM(TIGERLAKE),
221	222	.has_display = true,
222	223	.has_llc = true,
223	224	.require_force_probe = true,
@@ -226,7 +227,7 @@ static const struct xe_device_desc tgl_desc = { · 2 unmodified lines
226	227	static const struct xe_device_desc rkl_desc = {
227	228	.graphics = &graphics_xelp,
228	229	.media = &media_xem,
229		− PLATFORM(XE_ROCKETLAKE),
	230	+ PLATFORM(ROCKETLAKE),
230	231	.has_display = true,
231	232	.has_llc = true,
232	233	.require_force_probe = true,
@@ -237,7 +238,7 @@ static const u16 adls_rpls_ids[] = { XE_RPLS_IDS(NOP), 0 }; · 4 unmodified lines
237	238	static const struct xe_device_desc adl_s_desc = {
238	239	.graphics = &graphics_xelp,
239	240	.media = &media_xem,
240		− PLATFORM(XE_ALDERLAKE_S),
	241	+ PLATFORM(ALDERLAKE_S),
241	242	.has_display = true,
242	243	.has_llc = true,
243	244	.require_force_probe = true,
@@ -252,7 +253,7 @@ static const u16 adlp_rplu_ids[] = { XE_RPLU_IDS(NOP), 0 }; · 8 unmodified lines
252	253	static const struct xe_device_desc adl_p_desc = {
253	254	.graphics = &graphics_xelp,
254	255	.media = &media_xem,
255		− PLATFORM(XE_ALDERLAKE_P),
	256	+ PLATFORM(ALDERLAKE_P),
256	257	.has_display = true,
257	258	.has_llc = true,
258	259	.require_force_probe = true,
@@ -265,7 +266,7 @@ static const struct xe_device_desc adl_p_desc = { · 6 unmodified lines
265	266	static const struct xe_device_desc adl_n_desc = {
266	267	.graphics = &graphics_xelp,
267	268	.media = &media_xem,
268		− PLATFORM(XE_ALDERLAKE_N),
	269	+ PLATFORM(ALDERLAKE_N),
269	270	.has_display = true,
270	271	.has_llc = true,
271	272	.require_force_probe = true,
@@ -278,7 +279,7 @@ static const struct xe_device_desc dg1_desc = { · 6 unmodified lines
278	279	.graphics = &graphics_xelpp,
279	280	.media = &media_xem,
280	281	DGFX_FEATURES,
281		− PLATFORM(XE_DG1),
	282	+ PLATFORM(DG1),
282	283	.has_display = true,
283	284	.has_heci_gscfi = 1,
284	285	.require_force_probe = true,
@@ -290,7 +291,7 @@ static const u16 dg2_g12_ids[] = { XE_DG2_G12_IDS(NOP), 0 }; · 5 unmodified lines
290	291
291	292	#define DG2_FEATURES \
292	293	DGFX_FEATURES, \
293		− PLATFORM(XE_DG2), \
	294	+ PLATFORM(DG2), \
294	295	.has_heci_gscfi = 1, \
295	296	.subplatforms = (const struct xe_subplatform_desc[]) { \
296	297	{ XE_SUBPLATFORM_DG2_G10, "G10", dg2_g10_ids }, \
@@ -320,7 +321,7 @@ static const struct xe_device_desc dg2_desc = { · 23 unmodified lines
320	321	static const __maybe_unused struct xe_device_desc pvc_desc = {
321	322	.graphics = &graphics_xehpc,
322	323	DGFX_FEATURES,
323		− PLATFORM(XE_PVC),
	324	+ PLATFORM(PVC),
324	325	.has_display = false,
325	326	.has_heci_gscfi = 1,
326	327	.require_force_probe = true,
@@ -329,19 +330,19 @@ static const __maybe_unused struct xe_device_desc pvc_desc = { · 2 unmodified lines
329	330	static const struct xe_device_desc mtl_desc = {
330	331	/* .graphics and .media determined via GMD_ID */
331	332	.require_force_probe = true,
332		− PLATFORM(XE_METEORLAKE),
	333	+ PLATFORM(METEORLAKE),
333	334	.has_display = true,
334	335	};
335	336
336	337	static const struct xe_device_desc lnl_desc = {
337		− PLATFORM(XE_LUNARLAKE),
	338	+ PLATFORM(LUNARLAKE),
338	339	.has_display = true,
339	340	.require_force_probe = true,
340	341	};
341	342
342	343	static const struct xe_device_desc bmg_desc __maybe_unused = {
343	344	DGFX_FEATURES,
344		− PLATFORM(XE_BATTLEMAGE),
	345	+ PLATFORM(BATTLEMAGE),
345	346	.require_force_probe = true,
346	347	};
347	348
@@ -470,10 +471,52 @@ static void read_gmdid(struct xe_device xe, enum xe_gmdid_type type, u32 ver, · 122 unmodified lines
470	471
471	472	KUNIT_STATIC_STUB_REDIRECT(read_gmdid, xe, type, ver, revid);
472	473
473		− if (type == GMDID_MEDIA)
474		− gmdid_reg.addr += MEDIA_GT_GSI_OFFSET;
	474	+ if (IS_SRIOV_VF(xe)) {
	475	+ /*
	476	+ * To get the value of the GMDID register, VFs must obtain it
	477	+ * from the GuC using MMIO communication.
	478	+ *
	479	+ * Note that at this point the xe_gt is not fully uninitialized
	480	+ * and only basic access to MMIO registers is possible. To use
	481	+ * our existing GuC communication functions we must perform at
	482	+ * least basic xe_gt and xe_guc initialization.
	483	+ *
	484	+ * Since to obtain the value of GMDID_MEDIA we need to use the
	485	+ * media GuC, temporarly tweak the gt type.
	486	+ */
	487	+ xe_gt_assert(gt, gt->info.type == XE_GT_TYPE_UNINITIALIZED);
	488	+
	489	+ if (type == GMDID_MEDIA) {
	490	+ gt->info.id = 1;
	491	+ gt->info.type = XE_GT_TYPE_MEDIA;
	492	+ } else {
	493	+ gt->info.id = 0;
	494	+ gt->info.type = XE_GT_TYPE_MAIN;
	495	+ }
	496	+
	497	+ xe_guc_comm_init_early(&gt->uc.guc);
	498	+
	499	+ /* Don't bother with GMDID if failed to negotiate the GuC ABI */
	500	+ val = xe_gt_sriov_vf_bootstrap(gt) ? 0 : xe_gt_sriov_vf_gmdid(gt);
	501	+
	502	+ /*
	503	+ * Only undo xe_gt.info here, the remaining changes made above
	504	+ * will be overwritten as part of the regular initialization.
	505	+ */
	506	+ gt->info.id = 0;
	507	+ gt->info.type = XE_GT_TYPE_UNINITIALIZED;
	508	+ } else {
	509	+ /*
	510	+ * We need to apply the GSI offset explicitly here as at this
	511	+ * point the xe_gt is not fully uninitialized and only basic
	512	+ * access to MMIO registers is possible.
	513	+ */
	514	+ if (type == GMDID_MEDIA)
	515	+ gmdid_reg.addr += MEDIA_GT_GSI_OFFSET;
	516	+
	517	+ val = xe_mmio_read32(gt, gmdid_reg);
	518	+ }
475	519
476		− val = xe_mmio_read32(gt, gmdid_reg);
477	520	ver = REG_FIELD_GET(GMD_ID_ARCH_MASK, val) 100 + REG_FIELD_GET(GMD_ID_RELEASE_MASK, val);
478	521	*revid = REG_FIELD_GET(GMD_ID_REVID, val);
479	522	}
@@ -554,6 +597,7 @@ static int xe_info_init_early(struct xe_device *xe, · 74 unmodified lines
554	597	{
555	598	int err;
556	599
	600	+ xe->info.platform_name = desc->platform_name;
557	601	xe->info.platform = desc->platform;
558	602	xe->info.subplatform = subplatform_desc ?
559	603	subplatform_desc->subplatform : XE_SUBPLATFORM_NONE;
@@ -628,6 +672,9 @@ static int xe_info_init(struct xe_device *xe, · 68 unmodified lines
628	672	xe->info.va_bits = graphics_desc->va_bits;
629	673	xe->info.vm_max_level = graphics_desc->vm_max_level;
630	674	xe->info.has_asid = graphics_desc->has_asid;
	675	+ xe->info.has_atomic_enable_pte_bit = graphics_desc->has_atomic_enable_pte_bit;
	676	+ if (xe->info.platform != XE_PVC)
	677	+ xe->info.has_device_atomics_on_smem = 1;
631	678	xe->info.has_flat_ccs = graphics_desc->has_flat_ccs;
632	679	xe->info.has_range_tlb_invalidation = graphics_desc->has_range_tlb_invalidation;
633	680	xe->info.has_usm = graphics_desc->has_usm;
@@ -655,9 +702,10 @@ static int xe_info_init(struct xe_device *xe, · 21 unmodified lines
655	702	gt = tile->primary_gt;
656	703	gt->info.id = xe->info.gt_count++;
657	704	gt->info.type = XE_GT_TYPE_MAIN;
658		− gt->info.__engine_mask = graphics_desc->hw_engine_mask;
	705	+ gt->info.has_indirect_ring_state = graphics_desc->has_indirect_ring_state;
	706	+ gt->info.engine_mask = graphics_desc->hw_engine_mask;
659	707	if (MEDIA_VER(xe) < 13 && media_desc)
660		− gt->info.__engine_mask \|= media_desc->hw_engine_mask;
	708	+ gt->info.engine_mask \|= media_desc->hw_engine_mask;
661	709
662	710	if (MEDIA_VER(xe) < 13 \|\| !media_desc)
663	711	continue;
@@ -672,7 +720,8 @@ static int xe_info_init(struct xe_device *xe, · 8 unmodified lines
672	720
673	721	gt = tile->media_gt;
674	722	gt->info.type = XE_GT_TYPE_MEDIA;
675		− gt->info.__engine_mask = media_desc->hw_engine_mask;
	723	+ gt->info.has_indirect_ring_state = media_desc->has_indirect_ring_state;
	724	+ gt->info.engine_mask = media_desc->hw_engine_mask;
676	725	gt->mmio.adj_offset = MEDIA_GT_GSI_OFFSET;
677	726	gt->mmio.adj_limit = MEDIA_GT_GSI_LENGTH;
678	727
@@ -757,7 +806,9 @@ static int xe_pci_probe(struct pci_dev pdev, const struct pci_device_id ent) · 78 unmodified lines
757	806	if (err)
758	807	return err;
759	808
760		− xe_display_probe(xe);
	809	+ err = xe_display_probe(xe);
	810	+ if (err)
	811	+ return err;
761	812
762	813	drm_dbg(&xe->drm, "%s %s %04x:%04x dgfx:%d gfx:%s (%d.%02d) media:%s (%d.%02d) display:%s dma_m_s:%d tc:%d gscfi:%d",
763	814	desc->platform_name,
@@ -951,6 +1002,9 @@ static struct pci_driver xe_pci_driver = { · 187 unmodified lines
951	1002	.probe = xe_pci_probe,
952	1003	.remove = xe_pci_remove,
953	1004	.shutdown = xe_pci_shutdown,
	1005	+#ifdef CONFIG_PCI_IOV
	1006	+ .sriov_configure = xe_pci_sriov_configure,
	1007	+#endif
954	1008	#ifdef CONFIG_PM_SLEEP
955	1009	.driver.pm = &xe_pm_ops,
956	1010	#endif

drivers/gpu/drm/xe/xe_pci_sriov.c+143−0

@@ -0,0 +1,143 @@
	1	+// SPDX-License-Identifier: MIT
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#include "xe_assert.h"
	7	+#include "xe_device.h"
	8	+#include "xe_gt_sriov_pf_config.h"
	9	+#include "xe_pci_sriov.h"
	10	+#include "xe_pm.h"
	11	+#include "xe_sriov.h"
	12	+#include "xe_sriov_pf_helpers.h"
	13	+#include "xe_sriov_printk.h"
	14	+
	15	+static int pf_provision_vfs(struct xe_device *xe, unsigned int num_vfs)
	16	+{
	17	+ struct xe_gt *gt;
	18	+ unsigned int id;
	19	+ int result = 0, err;
	20	+
	21	+ for_each_gt(gt, xe, id) {
	22	+ err = xe_gt_sriov_pf_config_set_fair(gt, VFID(1), num_vfs);
	23	+ result = result ?: err;
	24	+ }
	25	+
	26	+ return result;
	27	+}
	28	+
	29	+static void pf_unprovision_vfs(struct xe_device *xe, unsigned int num_vfs)
	30	+{
	31	+ struct xe_gt *gt;
	32	+ unsigned int id;
	33	+ unsigned int n;
	34	+
	35	+ for_each_gt(gt, xe, id)
	36	+ for (n = 1; n <= num_vfs; n++)
	37	+ xe_gt_sriov_pf_config_release(gt, n, true);
	38	+}
	39	+
	40	+static int pf_enable_vfs(struct xe_device *xe, int num_vfs)
	41	+{
	42	+ struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
	43	+ int total_vfs = xe_sriov_pf_get_totalvfs(xe);
	44	+ int err;
	45	+
	46	+ xe_assert(xe, IS_SRIOV_PF(xe));
	47	+ xe_assert(xe, num_vfs > 0);
	48	+ xe_assert(xe, num_vfs <= total_vfs);
	49	+ xe_sriov_dbg(xe, "enabling %u VF%s\n", num_vfs, str_plural(num_vfs));
	50	+
	51	+ /*
	52	+ * We must hold additional reference to the runtime PM to keep PF in D0
	53	+ * during VFs lifetime, as our VFs do not implement the PM capability.
	54	+ *
	55	+ * With PF being in D0 state, all VFs will also behave as in D0 state.
	56	+ * This will also keep GuC alive with all VFs' configurations.
	57	+ *
	58	+ * We will release this additional PM reference in pf_disable_vfs().
	59	+ */
	60	+ xe_pm_runtime_get_noresume(xe);
	61	+
	62	+ err = pf_provision_vfs(xe, num_vfs);
	63	+ if (err < 0)
	64	+ goto failed;
	65	+
	66	+ err = pci_enable_sriov(pdev, num_vfs);
	67	+ if (err < 0)
	68	+ goto failed;
	69	+
	70	+ xe_sriov_info(xe, "Enabled %u of %u VF%s\n",
	71	+ num_vfs, total_vfs, str_plural(total_vfs));
	72	+ return num_vfs;
	73	+
	74	+failed:
	75	+ pf_unprovision_vfs(xe, num_vfs);
	76	+ xe_pm_runtime_put(xe);
	77	+
	78	+ xe_sriov_notice(xe, "Failed to enable %u VF%s (%pe)\n",
	79	+ num_vfs, str_plural(num_vfs), ERR_PTR(err));
	80	+ return err;
	81	+}
	82	+
	83	+static int pf_disable_vfs(struct xe_device *xe)
	84	+{
	85	+ struct device *dev = xe->drm.dev;
	86	+ struct pci_dev *pdev = to_pci_dev(dev);
	87	+ u16 num_vfs = pci_num_vf(pdev);
	88	+
	89	+ xe_assert(xe, IS_SRIOV_PF(xe));
	90	+ xe_sriov_dbg(xe, "disabling %u VF%s\n", num_vfs, str_plural(num_vfs));
	91	+
	92	+ if (!num_vfs)
	93	+ return 0;
	94	+
	95	+ pci_disable_sriov(pdev);
	96	+
	97	+ pf_unprovision_vfs(xe, num_vfs);
	98	+
	99	+ /* not needed anymore - see pf_enable_vfs() */
	100	+ xe_pm_runtime_put(xe);
	101	+
	102	+ xe_sriov_info(xe, "Disabled %u VF%s\n", num_vfs, str_plural(num_vfs));
	103	+ return 0;
	104	+}
	105	+
	106	+/**
	107	+ * xe_pci_sriov_configure - Configure SR-IOV (enable/disable VFs).
	108	+ * @pdev: the &pci_dev
	109	+ * @num_vfs: number of VFs to enable or zero to disable all VFs
	110	+ *
	111	+ * This is the Xe implementation of struct pci_driver.sriov_configure callback.
	112	+ *
	113	+ * This callback will be called by the PCI subsystem to enable or disable SR-IOV
	114	+ * Virtual Functions (VFs) as requested by the used via the PCI sysfs interface.
	115	+ *
	116	+ * Return: number of configured VFs or a negative error code on failure.
	117	+ */
	118	+int xe_pci_sriov_configure(struct pci_dev *pdev, int num_vfs)
	119	+{
	120	+ struct xe_device *xe = pdev_to_xe_device(pdev);
	121	+ int ret;
	122	+
	123	+ if (!IS_SRIOV_PF(xe))
	124	+ return -ENODEV;
	125	+
	126	+ if (num_vfs < 0)
	127	+ return -EINVAL;
	128	+
	129	+ if (num_vfs > xe_sriov_pf_get_totalvfs(xe))
	130	+ return -ERANGE;
	131	+
	132	+ if (num_vfs && pci_num_vf(pdev))
	133	+ return -EBUSY;
	134	+
	135	+ xe_pm_runtime_get(xe);
	136	+ if (num_vfs > 0)
	137	+ ret = pf_enable_vfs(xe, num_vfs);
	138	+ else
	139	+ ret = pf_disable_vfs(xe);
	140	+ xe_pm_runtime_put(xe);
	141	+
	142	+ return ret;
	143	+}

drivers/gpu/drm/xe/xe_pci_sriov.h+13−0

@@ -0,0 +1,13 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2023-2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_PCI_SRIOV_H_
	7	+#define _XE_PCI_SRIOV_H_
	8	+
	9	+struct pci_dev;
	10	+
	11	+int xe_pci_sriov_configure(struct pci_dev *pdev, int num_vfs);
	12	+
	13	+#endif

drivers/gpu/drm/xe/xe_pci_types.h+4−0

@@ -25,7 +25,9 @@ struct xe_graphics_desc { · 24 unmodified lines
25	25	u8 max_remote_tiles:2;
26	26
27	27	u8 has_asid:1;
	28	+ u8 has_atomic_enable_pte_bit:1;
28	29	u8 has_flat_ccs:1;
	30	+ u8 has_indirect_ring_state:1;
29	31	u8 has_range_tlb_invalidation:1;
30	32	u8 has_usm:1;
31	33	};
@@ -36,6 +38,8 @@ struct xe_media_desc { · 4 unmodified lines
36	38	u8 rel;
37	39
38	40	u64 hw_engine_mask; /* hardware engines provided by media IP */
	41	+
	42	+ u8 has_indirect_ring_state:1;
39	43	};
40	44
41	45	struct gmdid_map {

drivers/gpu/drm/xe/xe_pcode.c+5−1

@@ -10,6 +10,7 @@ · 9 unmodified lines
10	10
11	11	#include <drm/drm_managed.h>
12	12
	13	+#include "xe_assert.h"
13	14	#include "xe_device.h"
14	15	#include "xe_gt.h"
15	16	#include "xe_mmio.h"
@@ -124,6 +125,8 @@ static int pcode_try_request(struct xe_gt *gt, u32 mbox, · 108 unmodified lines
124	125	{
125	126	int slept, wait = 10;
126	127
	128	+ xe_gt_assert(gt, timeout_us > 0);
	129	+
127	130	for (slept = 0; slept < timeout_us; slept += wait) {
128	131	if (locked)
129	132	*status = pcode_mailbox_rw(gt, mbox, &request, NULL, 1, true,
@@ -169,6 +172,8 @@ int xe_pcode_request(struct xe_gt *gt, u32 mbox, u32 request, · 39 unmodified lines
169	172	u32 status;
170	173	int ret;
171	174
	175	+ xe_gt_assert(gt, timeout_base_ms <= 3);
	176	+
172	177	mutex_lock(&gt->pcode.lock);
173	178
174	179	ret = pcode_try_request(gt, mbox, request, reply_mask, reply, &status,
@@ -188,7 +193,6 @@ int xe_pcode_request(struct xe_gt *gt, u32 mbox, u32 request, · 13 unmodified lines
188	193	*/
189	194	drm_err(&gt_to_xe(gt)->drm,
190	195	"PCODE timeout, retrying with preemption disabled\n");
191		− drm_WARN_ON_ONCE(&gt_to_xe(gt)->drm, timeout_base_ms > 1);
192	196	preempt_disable();
193	197	ret = pcode_try_request(gt, mbox, request, reply_mask, reply, &status,
194	198	true, 50 * 1000, true);

drivers/gpu/drm/xe/xe_pm.c+22−20

@@ -69,7 +69,7 @@ · 68 unmodified lines
69	69	*/
70	70
71	71	#ifdef CONFIG_LOCKDEP
72		−struct lockdep_map xe_pm_runtime_lockdep_map = {
	72	+static struct lockdep_map xe_pm_runtime_lockdep_map = {
73	73	.name = "xe_pm_runtime_lockdep_map"
74	74	};
75	75	#endif
@@ -96,12 +96,12 @@ int xe_pm_suspend(struct xe_device *xe) · 20 unmodified lines
96	96	if (err)
97	97	goto err;
98	98
99		− xe_display_pm_suspend(xe);
	99	+ xe_display_pm_suspend(xe, false);
100	100
101	101	for_each_gt(gt, xe, id) {
102	102	err = xe_gt_suspend(gt);
103	103	if (err) {
104		− xe_display_pm_resume(xe);
	104	+ xe_display_pm_resume(xe, false);
105	105	goto err;
106	106	}
107	107	}
@@ -151,7 +151,7 @@ int xe_pm_resume(struct xe_device *xe) · 43 unmodified lines
151	151
152	152	xe_irq_resume(xe);
153	153
154		− xe_display_pm_resume(xe);
	154	+ xe_display_pm_resume(xe, false);
155	155
156	156	for_each_gt(gt, xe, id)
157	157	xe_gt_resume(gt);
@@ -366,6 +366,7 @@ int xe_pm_runtime_suspend(struct xe_device *xe) · 208 unmodified lines
366	366	err = xe_bo_evict_all(xe);
367	367	if (err)
368	368	goto out;
	369	+ xe_display_pm_suspend(xe, true);
369	370	}
370	371
371	372	for_each_gt(gt, xe, id) {
@@ -375,7 +376,12 @@ int xe_pm_runtime_suspend(struct xe_device *xe) · 3 unmodified lines
375	376	}
376	377
377	378	xe_irq_suspend(xe);
	379	+
	380	+ if (xe->d3cold.allowed)
	381	+ xe_display_pm_suspend_late(xe);
378	382	out:
	383	+ if (err)
	384	+ xe_display_pm_resume(xe, true);
379	385	lock_map_release(&xe_pm_runtime_lockdep_map);
380	386	xe_pm_write_callback_task(xe, NULL);
381	387	return err;
@@ -398,19 +404,13 @@ int xe_pm_runtime_resume(struct xe_device *xe) · 16 unmodified lines
398	404
399	405	lock_map_acquire(&xe_pm_runtime_lockdep_map);
400	406
401		− /*
402		− * It can be possible that xe has allowed d3cold but other pcie devices
403		− * in gfx card soc would have blocked d3cold, therefore card has not
404		− * really lost power. Detecting primary Gt power is sufficient.
405		− */
406		− gt = xe_device_get_gt(xe, 0);
407		− xe->d3cold.power_lost = xe_guc_in_reset(&gt->uc.guc);
408		−
409		− if (xe->d3cold.allowed && xe->d3cold.power_lost) {
	407	+ if (xe->d3cold.allowed) {
410	408	err = xe_pcode_ready(xe, true);
411	409	if (err)
412	410	goto out;
413	411
	412	+ xe_display_pm_resume_early(xe);
	413	+
414	414	/*
415	415	* This only restores pinned memory which is the memory
416	416	* required for the GT(s) to resume.
@@ -425,7 +425,8 @@ int xe_pm_runtime_resume(struct xe_device *xe) · 8 unmodified lines
425	425	for_each_gt(gt, xe, id)
426	426	xe_gt_resume(gt);
427	427
428		− if (xe->d3cold.allowed && xe->d3cold.power_lost) {
	428	+ if (xe->d3cold.allowed) {
	429	+ xe_display_pm_resume(xe, true);
429	430	err = xe_bo_restore_user(xe);
430	431	if (err)
431	432	goto out;
@@ -505,19 +506,20 @@ int xe_pm_runtime_get_ioctl(struct xe_device *xe) · 73 unmodified lines
505	506	* xe_pm_runtime_get_if_active - Get a runtime_pm reference if device active
506	507	* @xe: xe device instance
507	508	*
508		− * Returns: Any number greater than or equal to 0 for success, negative error
509		− * code otherwise.
	509	+ * Return: True if device is awake (regardless the previous number of references)
	510	+ * and a new reference was taken, false otherwise.
510	511	*/
511		−int xe_pm_runtime_get_if_active(struct xe_device *xe)
	512	+bool xe_pm_runtime_get_if_active(struct xe_device *xe)
512	513	{
513		− return pm_runtime_get_if_active(xe->drm.dev);
	514	+ return pm_runtime_get_if_active(xe->drm.dev) > 0;
514	515	}
515	516
516	517	/**
517		− * xe_pm_runtime_get_if_in_use - Get a runtime_pm reference and resume if needed
	518	+ * xe_pm_runtime_get_if_in_use - Get a new reference if device is active with previous ref taken
518	519	* @xe: xe device instance
519	520	*
520		− * Returns: True if device is awake and the reference was taken, false otherwise.
	521	+ * Return: True if device is awake, a previous reference had been already taken,
	522	+ * and a new reference was now taken, false otherwise.
521	523	*/
522	524	bool xe_pm_runtime_get_if_in_use(struct xe_device *xe)
523	525	{

drivers/gpu/drm/xe/xe_pm.h+2−7

@@ -8,12 +8,7 @@ · 7 unmodified lines
8	8
9	9	#include <linux/pm_runtime.h>
10	10
11		−/*
12		− * TODO: Threshold = 0 will block D3Cold.
13		− * Before we can move this to a higher value (like 300), we need to:
14		− * 1. rewrite the VRAM save / restore to avoid buffer object locks
15		− */
16		−#define DEFAULT_VRAM_THRESHOLD 0 /* in MB */
	11	+#define DEFAULT_VRAM_THRESHOLD 300 /* in MB */
17	12
18	13	struct xe_device;
19	14
@@ -29,7 +24,7 @@ int xe_pm_runtime_resume(struct xe_device *xe); · 9 unmodified lines
29	24	void xe_pm_runtime_get(struct xe_device *xe);
30	25	int xe_pm_runtime_get_ioctl(struct xe_device *xe);
31	26	void xe_pm_runtime_put(struct xe_device *xe);
32		−int xe_pm_runtime_get_if_active(struct xe_device *xe);
	27	+bool xe_pm_runtime_get_if_active(struct xe_device *xe);
33	28	bool xe_pm_runtime_get_if_in_use(struct xe_device *xe);
34	29	void xe_pm_runtime_get_noresume(struct xe_device *xe);
35	30	bool xe_pm_runtime_resume_and_get(struct xe_device *xe);

drivers/gpu/drm/xe/xe_preempt_fence.c+11−3

@@ -23,11 +23,19 @@ static void preempt_fence_work_func(struct work_struct *w) · 22 unmodified lines
23	23	q->ops->suspend_wait(q);
24	24
25	25	dma_fence_signal(&pfence->base);
26		− dma_fence_end_signalling(cookie);
27		−
	26	+ /*
	27	+ * Opt for keep everything in the fence critical section. This looks really strange since we
	28	+ * have just signalled the fence, however the preempt fences are all signalled via single
	29	+ * global ordered-wq, therefore anything that happens in this callback can easily block
	30	+ * progress on the entire wq, which itself may prevent other published preempt fences from
	31	+ * ever signalling. Therefore try to keep everything here in the callback in the fence
	32	+ * critical section. For example if something below grabs a scary lock like vm->lock,
	33	+ * lockdep should complain since we also hold that lock whilst waiting on preempt fences to
	34	+ * complete.
	35	+ */
28	36	xe_vm_queue_rebind_worker(q->vm);
29		−
30	37	xe_exec_queue_put(q);
	38	+ dma_fence_end_signalling(cookie);
31	39	}
32	40
33	41	static const char *

drivers/gpu/drm/xe/xe_pt.c+56−13

@@ -619,9 +619,40 @@ xe_pt_stage_bind(struct xe_tile tile, struct xe_vma vma, · 618 unmodified lines
619	619	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
620	620	int ret;
621	621
622		− if ((vma->gpuva.flags & XE_VMA_ATOMIC_PTE_BIT) &&
623		− (is_devmem \|\| !IS_DGFX(xe)))
624		− xe_walk.default_pte \|= XE_USM_PPGTT_PTE_AE;
	622	+ /**
	623	+ * Default atomic expectations for different allocation scenarios are as follows:
	624	+ *
	625	+ * 1. Traditional API: When the VM is not in LR mode:
	626	+ * - Device atomics are expected to function with all allocations.
	627	+ *
	628	+ * 2. Compute/SVM API: When the VM is in LR mode:
	629	+ * - Device atomics are the default behavior when the bo is placed in a single region.
	630	+ * - In all other cases device atomics will be disabled with AE=0 until an application
	631	+ * request differently using a ioctl like madvise.
	632	+ */
	633	+ if (vma->gpuva.flags & XE_VMA_ATOMIC_PTE_BIT) {
	634	+ if (xe_vm_in_lr_mode(xe_vma_vm(vma))) {
	635	+ if (bo && xe_bo_has_single_placement(bo))
	636	+ xe_walk.default_pte \|= XE_USM_PPGTT_PTE_AE;
	637	+ /**
	638	+ * If a SMEM+LMEM allocation is backed by SMEM, a device
	639	+ * atomics will cause a gpu page fault and which then
	640	+ * gets migrated to LMEM, bind such allocations with
	641	+ * device atomics enabled.
	642	+ */
	643	+ else if (is_devmem && !xe_bo_has_single_placement(bo))
	644	+ xe_walk.default_pte \|= XE_USM_PPGTT_PTE_AE;
	645	+ } else {
	646	+ xe_walk.default_pte \|= XE_USM_PPGTT_PTE_AE;
	647	+ }
	648	+
	649	+ /**
	650	+ * Unset AE if the platform(PVC) doesn't support it on an
	651	+ * allocation
	652	+ */
	653	+ if (!xe->info.has_device_atomics_on_smem && !is_devmem)
	654	+ xe_walk.default_pte &= ~XE_USM_PPGTT_PTE_AE;
	655	+ }
625	656
626	657	if (is_devmem) {
627	658	xe_walk.default_pte \|= XE_PPGTT_PTE_DM;
@@ -732,7 +763,7 @@ static int xe_pt_zap_ptes_entry(struct xe_ptw *parent, pgoff_t offset, · 104 unmodified lines
732	763	pgoff_t end_offset;
733	764
734	765	XE_WARN_ON(!*child);
735		− XE_WARN_ON(!level && xe_child->is_compact);
	766	+ XE_WARN_ON(!level);
736	767
737	768	/*
738	769	* Note that we're called from an entry callback, and we're dealing
@@ -781,8 +812,9 @@ bool xe_pt_zap_ptes(struct xe_tile tile, struct xe_vma vma) · 42 unmodified lines
781	812	.tile = tile,
782	813	};
783	814	struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
	815	+ u8 pt_mask = (vma->tile_present & ~vma->tile_invalidated);
784	816
785		− if (!(vma->tile_present & BIT(tile->id)))
	817	+ if (!(pt_mask & BIT(tile->id)))
786	818	return false;
787	819
788	820	(void)xe_pt_walk_shared(&pt->base, pt->level, xe_vma_start(vma),
@@ -1075,10 +1107,12 @@ static const struct xe_migrate_pt_update_ops userptr_bind_ops = { · 286 unmodified lines
1075	1107	struct invalidation_fence {
1076	1108	struct xe_gt_tlb_invalidation_fence base;
1077	1109	struct xe_gt *gt;
1078		− struct xe_vma *vma;
1079	1110	struct dma_fence *fence;
1080	1111	struct dma_fence_cb cb;
1081	1112	struct work_struct work;
	1113	+ u64 start;
	1114	+ u64 end;
	1115	+ u32 asid;
1082	1116	};
1083	1117
1084	1118	static const char *
@@ -1121,13 +1155,14 @@ static void invalidation_fence_work_func(struct work_struct *w) · 36 unmodified lines
1121	1155	container_of(w, struct invalidation_fence, work);
1122	1156
1123	1157	trace_xe_gt_tlb_invalidation_fence_work_func(&ifence->base);
1124		− xe_gt_tlb_invalidation_vma(ifence->gt, &ifence->base, ifence->vma);
	1158	+ xe_gt_tlb_invalidation_range(ifence->gt, &ifence->base, ifence->start,
	1159	+ ifence->end, ifence->asid);
1125	1160	}
1126	1161
1127	1162	static int invalidation_fence_init(struct xe_gt *gt,
1128	1163	struct invalidation_fence *ifence,
1129	1164	struct dma_fence *fence,
1130		− struct xe_vma *vma)
	1165	+ u64 start, u64 end, u32 asid)
1131	1166	{
1132	1167	int ret;
1133	1168
@@ -1144,7 +1179,9 @@ static int invalidation_fence_init(struct xe_gt *gt, · 10 unmodified lines
1144	1179	dma_fence_get(&ifence->base.base); /* Ref for caller */
1145	1180	ifence->fence = fence;
1146	1181	ifence->gt = gt;
1147		− ifence->vma = vma;
	1182	+ ifence->start = start;
	1183	+ ifence->end = end;
	1184	+ ifence->asid = asid;
1148	1185
1149	1186	INIT_WORK(&ifence->work, invalidation_fence_work_func);
1150	1187	ret = dma_fence_add_callback(fence, &ifence->cb, invalidation_fence_cb);
@@ -1295,8 +1332,11 @@ __xe_pt_bind_vma(struct xe_tile tile, struct xe_vma vma, struct xe_exec_queue · 144 unmodified lines
1295	1332
1296	1333	/* TLB invalidation must be done before signaling rebind */
1297	1334	if (ifence) {
1298		− int err = invalidation_fence_init(tile->primary_gt, ifence, fence,
1299		− vma);
	1335	+ int err = invalidation_fence_init(tile->primary_gt,
	1336	+ ifence, fence,
	1337	+ xe_vma_start(vma),
	1338	+ xe_vma_end(vma),
	1339	+ xe_vma_vm(vma)->usm.asid);
1300	1340	if (err) {
1301	1341	dma_fence_put(fence);
1302	1342	kfree(ifence);
@@ -1405,7 +1445,7 @@ static int xe_pt_stage_unbind_entry(struct xe_ptw *parent, pgoff_t offset, · 102 unmodified lines
1405	1445	struct xe_pt xe_child = container_of(child, typeof(*xe_child), base);
1406	1446
1407	1447	XE_WARN_ON(!*child);
1408		− XE_WARN_ON(!level && xe_child->is_compact);
	1448	+ XE_WARN_ON(!level);
1409	1449
1410	1450	xe_pt_check_kill(addr, next, level - 1, xe_child, action, walk);
1411	1451
@@ -1641,7 +1681,10 @@ __xe_pt_unbind_vma(struct xe_tile tile, struct xe_vma vma, struct xe_exec_queu · 229 unmodified lines
1641	1681	dma_fence_wait(fence, false);
1642	1682
1643	1683	/* TLB invalidation must be done before signaling unbind */
1644		− err = invalidation_fence_init(tile->primary_gt, ifence, fence, vma);
	1684	+ err = invalidation_fence_init(tile->primary_gt, ifence, fence,
	1685	+ xe_vma_start(vma),
	1686	+ xe_vma_end(vma),
	1687	+ xe_vma_vm(vma)->usm.asid);
1645	1688	if (err) {
1646	1689	dma_fence_put(fence);
1647	1690	kfree(ifence);

drivers/gpu/drm/xe/xe_query.c+9−1

@@ -16,6 +16,7 @@ · 15 unmodified lines
16	16	#include "xe_bo.h"
17	17	#include "xe_device.h"
18	18	#include "xe_exec_queue.h"
	19	+#include "xe_force_wake.h"
19	20	#include "xe_ggtt.h"
20	21	#include "xe_gt.h"
21	22	#include "xe_guc_hwconfig.h"
@@ -454,9 +455,10 @@ static int query_hwconfig(struct xe_device *xe, · 432 unmodified lines
454	455	static size_t calc_topo_query_size(struct xe_device *xe)
455	456	{
456	457	return xe->info.gt_count *
457		− (3 * sizeof(struct drm_xe_query_topology_mask) +
	458	+ (4 * sizeof(struct drm_xe_query_topology_mask) +
458	459	sizeof_field(struct xe_gt, fuse_topo.g_dss_mask) +
459	460	sizeof_field(struct xe_gt, fuse_topo.c_dss_mask) +
	461	+ sizeof_field(struct xe_gt, fuse_topo.l3_bank_mask) +
460	462	sizeof_field(struct xe_gt, fuse_topo.eu_mask_per_dss));
461	463	}
462	464
@@ -510,6 +512,12 @@ static int query_gt_topology(struct xe_device *xe, · 47 unmodified lines
510	512	if (err)
511	513	return err;
512	514
	515	+ topo.type = DRM_XE_TOPO_L3_BANK;
	516	+ err = copy_mask(&query_ptr, &topo, gt->fuse_topo.l3_bank_mask,
	517	+ sizeof(gt->fuse_topo.l3_bank_mask));
	518	+ if (err)
	519	+ return err;
	520	+
513	521	topo.type = DRM_XE_TOPO_EU_PER_DSS;
514	522	err = copy_mask(&query_ptr, &topo,
515	523	gt->fuse_topo.eu_mask_per_dss,

drivers/gpu/drm/xe/xe_reg_sr.h+2−2

@@ -6,8 +6,6 @@ · 5 unmodified lines
6	6	#ifndef _XE_REG_SR_
7	7	#define _XE_REG_SR_
8	8
9		−#include "xe_reg_sr_types.h"
10		−
11	9	/*
12	10	* Reg save/restore bookkeeping
13	11	*/
@@ -15,6 +13,8 @@ · 1 unmodified line
15	13	struct xe_device;
16	14	struct xe_gt;
17	15	struct xe_hw_engine;
	16	+struct xe_reg_sr;
	17	+struct xe_reg_sr_entry;
18	18	struct drm_printer;
19	19
20	20	int xe_reg_sr_init(struct xe_reg_sr sr, const char name, struct xe_device *xe);

drivers/gpu/drm/xe/xe_res_cursor.h+0−2

@@ -157,8 +157,6 @@ static inline void xe_res_first_sg(const struct sg_table *sg, · 156 unmodified lines
157	157	struct xe_res_cursor *cur)
158	158	{
159	159	XE_WARN_ON(!sg);
160		− XE_WARN_ON(!IS_ALIGNED(start, PAGE_SIZE) \|\|
161		− !IS_ALIGNED(size, PAGE_SIZE));
162	160	cur->node = NULL;
163	161	cur->start = start;
164	162	cur->remaining = size;

drivers/gpu/drm/xe/xe_ring_ops.c+33−19

@@ -7,9 +7,9 @@ · 6 unmodified lines
7	7
8	8	#include <generated/xe_wa_oob.h>
9	9
	10	+#include "instructions/xe_gpu_commands.h"
10	11	#include "instructions/xe_mi_commands.h"
11	12	#include "regs/xe_engine_regs.h"
12		−#include "regs/xe_gpu_commands.h"
13	13	#include "regs/xe_gt_regs.h"
14	14	#include "regs/xe_lrc_layout.h"
15	15	#include "xe_exec_queue_types.h"
@@ -80,6 +80,16 @@ static int emit_store_imm_ggtt(u32 addr, u32 value, u32 *dw, int i) · 64 unmodified lines
80	80	return i;
81	81	}
82	82
	83	+static int emit_flush_dw(u32 *dw, int i)
	84	+{
	85	+ dw[i++] = MI_FLUSH_DW \| MI_FLUSH_IMM_DW;
	86	+ dw[i++] = 0;
	87	+ dw[i++] = 0;
	88	+ dw[i++] = 0;
	89	+
	90	+ return i;
	91	+}
	92	+
83	93	static int emit_flush_imm_ggtt(u32 addr, u32 value, bool invalidate_tlb,
84	94	u32 *dw, int i)
85	95	{
@@ -234,10 +244,12 @@ static void __emit_job_gen12_simple(struct xe_sched_job job, struct xe_lrc lrc · 148 unmodified lines
234	244
235	245	i = emit_bb_start(batch_addr, ppgtt_flag, dw, i);
236	246
237		− if (job->user_fence.used)
	247	+ if (job->user_fence.used) {
	248	+ i = emit_flush_dw(dw, i);
238	249	i = emit_store_imm_ppgtt_posted(job->user_fence.addr,
239	250	job->user_fence.value,
240	251	dw, i);
	252	+ }
241	253
242	254	i = emit_flush_imm_ggtt(xe_lrc_seqno_ggtt_addr(lrc), seqno, false, dw, i);
243	255
@@ -293,10 +305,12 @@ static void __emit_job_gen12_video(struct xe_sched_job job, struct xe_lrc lrc, · 49 unmodified lines
293	305
294	306	i = emit_bb_start(batch_addr, ppgtt_flag, dw, i);
295	307
296		− if (job->user_fence.used)
	308	+ if (job->user_fence.used) {
	309	+ i = emit_flush_dw(dw, i);
297	310	i = emit_store_imm_ppgtt_posted(job->user_fence.addr,
298	311	job->user_fence.value,
299	312	dw, i);
	313	+ }
300	314
301	315	i = emit_flush_imm_ggtt(xe_lrc_seqno_ggtt_addr(lrc), seqno, false, dw, i);
302	316
@@ -366,7 +380,7 @@ static void emit_migration_job_gen12(struct xe_sched_job *job, · 63 unmodified lines
366	380
367	381	dw[i++] = MI_ARB_ON_OFF \| MI_ARB_DISABLE; /* Enabled again below */
368	382
369		− i = emit_bb_start(job->batch_addr[0], BIT(8), dw, i);
	383	+ i = emit_bb_start(job->ptrs[0].batch_addr, BIT(8), dw, i);
370	384
371	385	if (!IS_SRIOV_VF(gt_to_xe(job->q->gt))) {
372	386	/* XXX: Do we need this? Leaving for now. */
@@ -375,7 +389,7 @@ static void emit_migration_job_gen12(struct xe_sched_job *job, · 2 unmodified lines
375	389	dw[i++] = preparser_disable(false);
376	390	}
377	391
378		− i = emit_bb_start(job->batch_addr[1], BIT(8), dw, i);
	392	+ i = emit_bb_start(job->ptrs[1].batch_addr, BIT(8), dw, i);
379	393
380	394	dw[i++] = MI_FLUSH_DW \| MI_INVALIDATE_TLB \| job->migrate_flush_flags \|
381	395	MI_FLUSH_DW_OP_STOREDW \| MI_FLUSH_IMM_DW;
@@ -396,9 +410,9 @@ static void emit_job_gen12_gsc(struct xe_sched_job *job) · 14 unmodified lines
396	410
397	411	xe_gt_assert(gt, job->q->width <= 1); /* no parallel submission for GSCCS */
398	412
399		− __emit_job_gen12_simple(job, job->q->lrc,
400		− job->batch_addr[0],
401		− xe_sched_job_seqno(job));
	413	+ __emit_job_gen12_simple(job, job->q->lrc[0],
	414	+ job->ptrs[0].batch_addr,
	415	+ xe_sched_job_lrc_seqno(job));
402	416	}
403	417
404	418	static void emit_job_gen12_copy(struct xe_sched_job *job)
@@ -406,15 +420,15 @@ static void emit_job_gen12_copy(struct xe_sched_job *job) · 1 unmodified line
406	420	int i;
407	421
408	422	if (xe_sched_job_is_migration(job->q)) {
409		− emit_migration_job_gen12(job, job->q->lrc,
410		− xe_sched_job_seqno(job));
	423	+ emit_migration_job_gen12(job, job->q->lrc[0],
	424	+ xe_sched_job_lrc_seqno(job));
411	425	return;
412	426	}
413	427
414	428	for (i = 0; i < job->q->width; ++i)
415		− __emit_job_gen12_simple(job, job->q->lrc + i,
416		− job->batch_addr[i],
417		− xe_sched_job_seqno(job));
	429	+ __emit_job_gen12_simple(job, job->q->lrc[i],
	430	+ job->ptrs[i].batch_addr,
	431	+ xe_sched_job_lrc_seqno(job));
418	432	}
419	433
420	434	static void emit_job_gen12_video(struct xe_sched_job *job)
@@ -423,9 +437,9 @@ static void emit_job_gen12_video(struct xe_sched_job *job) · 2 unmodified lines
423	437
424	438	/* FIXME: Not doing parallel handshake for now */
425	439	for (i = 0; i < job->q->width; ++i)
426		− __emit_job_gen12_video(job, job->q->lrc + i,
427		− job->batch_addr[i],
428		− xe_sched_job_seqno(job));
	440	+ __emit_job_gen12_video(job, job->q->lrc[i],
	441	+ job->ptrs[i].batch_addr,
	442	+ xe_sched_job_lrc_seqno(job));
429	443	}
430	444
431	445	static void emit_job_gen12_render_compute(struct xe_sched_job *job)
@@ -433,9 +447,9 @@ static void emit_job_gen12_render_compute(struct xe_sched_job *job) · 1 unmodified line
433	447	int i;
434	448
435	449	for (i = 0; i < job->q->width; ++i)
436		− __emit_job_gen12_render_compute(job, job->q->lrc + i,
437		− job->batch_addr[i],
438		− xe_sched_job_seqno(job));
	450	+ __emit_job_gen12_render_compute(job, job->q->lrc[i],
	451	+ job->ptrs[i].batch_addr,
	452	+ xe_sched_job_lrc_seqno(job));
439	453	}
440	454
441	455	static const struct xe_ring_ops ring_ops_gen12_gsc = {

drivers/gpu/drm/xe/xe_rtp.c+6−0

@@ -323,3 +323,9 @@ bool xe_rtp_match_first_gslice_fused_off(const struct xe_gt *gt, · 322 unmodified lines
323	323
324	324	return dss >= dss_per_gslice;
325	325	}
	326	+
	327	+bool xe_rtp_match_when_media2000(const struct xe_gt *gt,
	328	+ const struct xe_hw_engine *hwe)
	329	+{
	330	+ return (gt_to_xe(gt))->info.media_verx100 == 2000;
	331	+}

drivers/gpu/drm/xe/xe_rtp.h+16−2

@@ -341,7 +341,7 @@ struct xe_reg_sr; · 340 unmodified lines
341	341	* };
342	342	*/
343	343	#define XE_RTP_RULES(...) \
344		− .n_rules = _XE_COUNT_ARGS(__VA_ARGS__), \
	344	+ .n_rules = COUNT_ARGS(__VA_ARGS__), \
345	345	.rules = (const struct xe_rtp_rule[]) { \
346	346	XE_RTP_PASTE_FOREACH(RULE_, COMMA, (__VA_ARGS__)) \
347	347	}
@@ -366,7 +366,7 @@ struct xe_reg_sr; · 18 unmodified lines
366	366	* };
367	367	*/
368	368	#define XE_RTP_ACTIONS(...) \
369		− .n_actions = _XE_COUNT_ARGS(__VA_ARGS__), \
	369	+ .n_actions = COUNT_ARGS(__VA_ARGS__), \
370	370	.actions = (const struct xe_rtp_action[]) { \
371	371	XE_RTP_PASTE_FOREACH(ACTION_, COMMA, (__VA_ARGS__)) \
372	372	}
@@ -427,4 +427,18 @@ bool xe_rtp_match_first_render_or_compute(const struct xe_gt *gt, · 54 unmodified lines
427	427	bool xe_rtp_match_first_gslice_fused_off(const struct xe_gt *gt,
428	428	const struct xe_hw_engine *hwe);
429	429
	430	+/*
	431	+ * xe_rtp_match_when_media2000 - Match when media GT version 2000
	432	+ *
	433	+ * @gt: GT structure
	434	+ * @hwe: Engine instance
	435	+ *
	436	+ * Its one of the case where we need to apply workaround on primary GT
	437	+ * based on if media GT version 2000 is present. Thus this API will help
	438	+ * us to match media version 2000.
	439	+ *
	440	+ * Returns: true if media GT version 2000, false otherwise.
	441	+ */
	442	+bool xe_rtp_match_when_media2000(const struct xe_gt *gt,
	443	+ const struct xe_hw_engine *hwe);
430	444	#endif

drivers/gpu/drm/xe/xe_rtp_helpers.h+10−16

@@ -10,22 +10,16 @@ · 9 unmodified lines
10	10	#error "This header is supposed to be included by xe_rtp.h only"
11	11	#endif
12	12
	13	+#include "xe_args.h"
	14	+
13	15	/*
14	16	* Helper macros - not to be used outside this header.
15	17	*/
16	18	#define _XE_ESC(...) __VA_ARGS__
17		−#define _XE_COUNT_ARGS(...) _XE_ESC(__XE_COUNT_ARGS(__VA_ARGS__, 5, 4, 3, 2, 1,))
18		−#define __XE_COUNT_ARGS(_, _5, _4, _3, _2, X_, ...) X_
19		−
20		−#define _XE_FIRST(...) _XE_ESC(__XE_FIRST(__VA_ARGS__,))
21		−#define __XE_FIRST(x_, ...) x_
22		−#define _XE_TUPLE_TAIL(...) _XE_ESC(__XE_TUPLE_TAIL(__VA_ARGS__))
23		−#define __XE_TUPLE_TAIL(x_, ...) (__VA_ARGS__)
24	19
25		−#define _XE_DROP_FIRST(x_, ...) __VA_ARGS__
	20	+#define _XE_TUPLE_TAIL(...) (DROP_FIRST_ARG(__VA_ARGS__))
26	21
27		−#define _XE_RTP_CONCAT(a, b) __XE_RTP_CONCAT(a, b)
28		−#define __XE_RTP_CONCAT(a, b) XE_RTP_ ## a ## b
	22	+#define _XE_RTP_CONCAT(a, b) CONCATENATE(XE_RTP_, CONCATENATE(a, b))
29	23
30	24	#define __XE_RTP_PASTE_SEP_COMMA ,
31	25	#define __XE_RTP_PASTE_SEP_BITWISE_OR \|
@@ -59,11 +53,11 @@ · 27 unmodified lines
59	53	*
60	54	* XE_RTP_TEST_FOO BANANA XE_RTP_TEST_BAR
61	55	*/
62		−#define XE_RTP_PASTE_FOREACH(prefix_, sep_, args_) _XE_ESC(_XE_RTP_CONCAT(PASTE_, _XE_COUNT_ARGS args_)(prefix_, sep_, args_))
63		−#define XE_RTP_PASTE_1(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, _XE_FIRST args_)
64		−#define XE_RTP_PASTE_2(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, _XE_FIRST args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_1(prefix_, sep_, _XE_TUPLE_TAIL args_)
65		−#define XE_RTP_PASTE_3(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, _XE_FIRST args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_2(prefix_, sep_, _XE_TUPLE_TAIL args_)
66		−#define XE_RTP_PASTE_4(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, _XE_FIRST args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_3(prefix_, sep_, _XE_TUPLE_TAIL args_)
	56	+#define XE_RTP_PASTE_FOREACH(prefix_, sep_, args_) _XE_RTP_CONCAT(PASTE_, COUNT_ARGS args_)(prefix_, sep_, args_)
	57	+#define XE_RTP_PASTE_1(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, FIRST_ARG args_)
	58	+#define XE_RTP_PASTE_2(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, FIRST_ARG args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_1(prefix_, sep_, _XE_TUPLE_TAIL args_)
	59	+#define XE_RTP_PASTE_3(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, FIRST_ARG args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_2(prefix_, sep_, _XE_TUPLE_TAIL args_)
	60	+#define XE_RTP_PASTE_4(prefix_, sep_, args_) _XE_RTP_CONCAT(prefix_, FIRST_ARG args_) __XE_RTP_PASTE_SEP_ ## sep_ XE_RTP_PASTE_3(prefix_, sep_, _XE_TUPLE_TAIL args_)
67	61
68	62	/*
69	63	* XE_RTP_DROP_CAST - Drop cast to convert a compound statement to a initializer
@@ -76,6 +70,6 @@ · 6 unmodified lines
76	70	*
77	71	* { .a = 10 }
78	72	*/
79		−#define XE_RTP_DROP_CAST(...) _XE_ESC(_XE_DROP_FIRST _XE_ESC __VA_ARGS__)
	73	+#define XE_RTP_DROP_CAST(...) _XE_ESC(DROP_FIRST_ARG _XE_ESC __VA_ARGS__)
80	74
81	75	#endif

drivers/gpu/drm/xe/xe_sched_job.c+100−81

@@ -6,7 +6,7 @@ · 5 unmodified lines
6	6	#include "xe_sched_job.h"
7	7
8	8	#include <drm/xe_drm.h>
9		−#include <linux/dma-fence-array.h>
	9	+#include <linux/dma-fence-chain.h>
10	10	#include <linux/slab.h>
11	11
12	12	#include "xe_device.h"
@@ -29,7 +29,7 @@ int __init xe_sched_job_module_init(void) · 16 unmodified lines
29	29	xe_sched_job_slab =
30	30	kmem_cache_create("xe_sched_job",
31	31	sizeof(struct xe_sched_job) +
32		− sizeof(u64), 0,
	32	+ sizeof(struct xe_job_ptrs), 0,
33	33	SLAB_HWCACHE_ALIGN, NULL);
34	34	if (!xe_sched_job_slab)
35	35	return -ENOMEM;
@@ -37,7 +37,7 @@ int __init xe_sched_job_module_init(void) · 1 unmodified line
37	37	xe_sched_job_parallel_slab =
38	38	kmem_cache_create("xe_sched_job_parallel",
39	39	sizeof(struct xe_sched_job) +
40		− sizeof(u64) *
	40	+ sizeof(struct xe_job_ptrs) *
41	41	XE_HW_ENGINE_MAX_INSTANCE, 0,
42	42	SLAB_HWCACHE_ALIGN, NULL);
43	43	if (!xe_sched_job_parallel_slab) {
@@ -79,26 +79,33 @@ static struct xe_device job_to_xe(struct xe_sched_job job) · 35 unmodified lines
79	79	return gt_to_xe(job->q->gt);
80	80	}
81	81
	82	+/* Free unused pre-allocated fences */
	83	+static void xe_sched_job_free_fences(struct xe_sched_job *job)
	84	+{
	85	+ int i;
	86	+
	87	+ for (i = 0; i < job->q->width; ++i) {
	88	+ struct xe_job_ptrs *ptrs = &job->ptrs[i];
	89	+
	90	+ if (ptrs->lrc_fence)
	91	+ xe_lrc_free_seqno_fence(ptrs->lrc_fence);
	92	+ if (ptrs->chain_fence)
	93	+ dma_fence_chain_free(ptrs->chain_fence);
	94	+ }
	95	+}
	96	+
82	97	struct xe_sched_job xe_sched_job_create(struct xe_exec_queue q,
83	98	u64 *batch_addr)
84	99	{
85		− struct xe_sched_job *job;
86		− struct dma_fence **fences;
87	100	bool is_migration = xe_sched_job_is_migration(q);
	101	+ struct xe_sched_job *job;
88	102	int err;
89		− int i, j;
	103	+ int i;
90	104	u32 width;
91	105
92	106	/* only a kernel context can submit a vm-less job */
93	107	XE_WARN_ON(!q->vm && !(q->flags & EXEC_QUEUE_FLAG_KERNEL));
94	108
95		− /* Migration and kernel engines have their own locking */
96		− if (!(q->flags & (EXEC_QUEUE_FLAG_KERNEL \| EXEC_QUEUE_FLAG_VM))) {
97		− lockdep_assert_held(&q->vm->lock);
98		− if (!xe_vm_in_lr_mode(q->vm))
99		− xe_vm_assert_held(q->vm);
100		− }
101		−
102	109	job = job_alloc(xe_exec_queue_is_parallel(q) \|\| is_migration);
103	110	if (!job)
104	111	return ERR_PTR(-ENOMEM);
@@ -111,44 +118,25 @@ struct xe_sched_job xe_sched_job_create(struct xe_exec_queue q, · 6 unmodified lines
111	118	if (err)
112	119	goto err_free;
113	120
114		− if (!xe_exec_queue_is_parallel(q)) {
115		− job->fence = xe_lrc_create_seqno_fence(q->lrc);
116		− if (IS_ERR(job->fence)) {
117		− err = PTR_ERR(job->fence);
118		− goto err_sched_job;
119		− }
120		− } else {
121		− struct dma_fence_array *cf;
	121	+ for (i = 0; i < q->width; ++i) {
	122	+ struct dma_fence *fence = xe_lrc_alloc_seqno_fence();
	123	+ struct dma_fence_chain *chain;
122	124
123		− fences = kmalloc_array(q->width, sizeof(*fences), GFP_KERNEL);
124		− if (!fences) {
125		− err = -ENOMEM;
	125	+ if (IS_ERR(fence)) {
	126	+ err = PTR_ERR(fence);
126	127	goto err_sched_job;
127	128	}
	129	+ job->ptrs[i].lrc_fence = fence;
128	130
129		− for (j = 0; j < q->width; ++j) {
130		− fences[j] = xe_lrc_create_seqno_fence(q->lrc + j);
131		− if (IS_ERR(fences[j])) {
132		− err = PTR_ERR(fences[j]);
133		− goto err_fences;
134		− }
135		− }
	131	+ if (i + 1 == q->width)
	132	+ continue;
136	133
137		− cf = dma_fence_array_create(q->width, fences,
138		− q->parallel.composite_fence_ctx,
139		− q->parallel.composite_fence_seqno++,
140		− false);
141		− if (!cf) {
142		− --q->parallel.composite_fence_seqno;
	134	+ chain = dma_fence_chain_alloc();
	135	+ if (!chain) {
143	136	err = -ENOMEM;
144		− goto err_fences;
	137	+ goto err_sched_job;
145	138	}
146		−
147		− /* Sanity check */
148		− for (j = 0; j < q->width; ++j)
149		− xe_assert(job_to_xe(job), cf->base.seqno == fences[j]->seqno);
150		−
151		− job->fence = &cf->base;
	139	+ job->ptrs[i].chain_fence = chain;
152	140	}
153	141
154	142	width = q->width;
@@ -156,23 +144,14 @@ struct xe_sched_job xe_sched_job_create(struct xe_exec_queue q, · 1 unmodified line
156	144	width = 2;
157	145
158	146	for (i = 0; i < width; ++i)
159		− job->batch_addr[i] = batch_addr[i];
160		−
161		− /* All other jobs require a VM to be open which has a ref */
162		− if (unlikely(q->flags & EXEC_QUEUE_FLAG_KERNEL))
163		− xe_pm_runtime_get_noresume(job_to_xe(job));
164		− xe_device_assert_mem_access(job_to_xe(job));
	147	+ job->ptrs[i].batch_addr = batch_addr[i];
165	148
	149	+ xe_pm_runtime_get_noresume(job_to_xe(job));
166	150	trace_xe_sched_job_create(job);
167	151	return job;
168	152
169		−err_fences:
170		− for (j = j - 1; j >= 0; --j) {
171		− --q->lrc[j].fence_ctx.next_seqno;
172		− dma_fence_put(fences[j]);
173		− }
174		− kfree(fences);
175	153	err_sched_job:
	154	+ xe_sched_job_free_fences(job);
176	155	drm_sched_job_cleanup(&job->drm);
177	156	err_free:
178	157	xe_exec_queue_put(q);
@@ -191,36 +170,42 @@ void xe_sched_job_destroy(struct kref *ref) · 12 unmodified lines
191	170	{
192	171	struct xe_sched_job *job =
193	172	container_of(ref, struct xe_sched_job, refcount);
	173	+ struct xe_device *xe = job_to_xe(job);
194	174
195		− if (unlikely(job->q->flags & EXEC_QUEUE_FLAG_KERNEL))
196		− xe_pm_runtime_put(job_to_xe(job));
	175	+ xe_sched_job_free_fences(job);
197	176	xe_exec_queue_put(job->q);
198	177	dma_fence_put(job->fence);
199	178	drm_sched_job_cleanup(&job->drm);
200	179	job_free(job);
	180	+ xe_pm_runtime_put(xe);
201	181	}
202	182
203		−void xe_sched_job_set_error(struct xe_sched_job *job, int error)
	183	+/* Set the error status under the fence to avoid racing with signaling */
	184	+static bool xe_fence_set_error(struct dma_fence *fence, int error)
204	185	{
205		− if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &job->fence->flags))
206		− return;
	186	+ unsigned long irq_flags;
	187	+ bool signaled;
207	188
208		− dma_fence_set_error(job->fence, error);
	189	+ spin_lock_irqsave(fence->lock, irq_flags);
	190	+ signaled = test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags);
	191	+ if (!signaled)
	192	+ dma_fence_set_error(fence, error);
	193	+ spin_unlock_irqrestore(fence->lock, irq_flags);
209	194
210		− if (dma_fence_is_array(job->fence)) {
211		− struct dma_fence_array *array =
212		− to_dma_fence_array(job->fence);
213		− struct dma_fence **child = array->fences;
214		− unsigned int nchild = array->num_fences;
	195	+ return signaled;
	196	+}
	197	+
	198	+void xe_sched_job_set_error(struct xe_sched_job *job, int error)
	199	+{
	200	+ if (xe_fence_set_error(job->fence, error))
	201	+ return;
215	202
216		− do {
217		− struct dma_fence current_fence = child++;
	203	+ if (dma_fence_is_chain(job->fence)) {
	204	+ struct dma_fence *iter;
218	205
219		− if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT,
220		− &current_fence->flags))
221		− continue;
222		− dma_fence_set_error(current_fence, error);
223		− } while (--nchild);
	206	+ dma_fence_chain_for_each(iter, job->fence)
	207	+ xe_fence_set_error(dma_fence_chain_contained(iter),
	208	+ error);
224	209	}
225	210
226	211	trace_xe_sched_job_set_error(job);
@@ -231,30 +216,42 @@ void xe_sched_job_set_error(struct xe_sched_job *job, int error) · 4 unmodified lines
231	216
232	217	bool xe_sched_job_started(struct xe_sched_job *job)
233	218	{
234		− struct xe_lrc *lrc = job->q->lrc;
	219	+ struct xe_lrc *lrc = job->q->lrc[0];
235	220
236		− return !__dma_fence_is_later(xe_sched_job_seqno(job),
	221	+ return !__dma_fence_is_later(xe_sched_job_lrc_seqno(job),
237	222	xe_lrc_start_seqno(lrc),
238		− job->fence->ops);
	223	+ dma_fence_chain_contained(job->fence)->ops);
239	224	}
240	225
241	226	bool xe_sched_job_completed(struct xe_sched_job *job)
242	227	{
243		− struct xe_lrc *lrc = job->q->lrc;
	228	+ struct xe_lrc *lrc = job->q->lrc[0];
244	229
245	230	/*
246	231	* Can safely check just LRC[0] seqno as that is last seqno written when
247	232	* parallel handshake is done.
248	233	*/
249	234
250		− return !__dma_fence_is_later(xe_sched_job_seqno(job), xe_lrc_seqno(lrc),
251		− job->fence->ops);
	235	+ return !__dma_fence_is_later(xe_sched_job_lrc_seqno(job),
	236	+ xe_lrc_seqno(lrc),
	237	+ dma_fence_chain_contained(job->fence)->ops);
252	238	}
253	239
254	240	void xe_sched_job_arm(struct xe_sched_job *job)
255	241	{
256	242	struct xe_exec_queue *q = job->q;
	243	+ struct dma_fence fence, prev;
257	244	struct xe_vm *vm = q->vm;
	245	+ u64 seqno = 0;
	246	+ int i;
	247	+
	248	+ /* Migration and kernel engines have their own locking */
	249	+ if (IS_ENABLED(CONFIG_LOCKDEP) &&
	250	+ !(q->flags & (EXEC_QUEUE_FLAG_KERNEL \| EXEC_QUEUE_FLAG_VM))) {
	251	+ lockdep_assert_held(&q->vm->lock);
	252	+ if (!xe_vm_in_lr_mode(q->vm))
	253	+ xe_vm_assert_held(q->vm);
	254	+ }
258	255
259	256	if (vm && !xe_sched_job_is_migration(q) && !xe_vm_in_lr_mode(vm) &&
260	257	(vm->batch_invalidate_tlb \|\| vm->tlb_flush_seqno != q->tlb_flush_seqno)) {
@@ -263,6 +260,27 @@ void xe_sched_job_arm(struct xe_sched_job *job) · 2 unmodified lines
263	260	job->ring_ops_flush_tlb = true;
264	261	}
265	262
	263	+ /* Arm the pre-allocated fences */
	264	+ for (i = 0; i < q->width; prev = fence, ++i) {
	265	+ struct dma_fence_chain *chain;
	266	+
	267	+ fence = job->ptrs[i].lrc_fence;
	268	+ xe_lrc_init_seqno_fence(q->lrc[i], fence);
	269	+ job->ptrs[i].lrc_fence = NULL;
	270	+ if (!i) {
	271	+ job->lrc_seqno = fence->seqno;
	272	+ continue;
	273	+ } else {
	274	+ xe_assert(gt_to_xe(q->gt), job->lrc_seqno == fence->seqno);
	275	+ }
	276	+
	277	+ chain = job->ptrs[i - 1].chain_fence;
	278	+ dma_fence_chain_init(chain, prev, fence, seqno++);
	279	+ job->ptrs[i - 1].chain_fence = NULL;
	280	+ fence = &chain->base;
	281	+ }
	282	+
	283	+ job->fence = fence;
266	284	drm_sched_job_arm(&job->drm);
267	285	}
268	286
@@ -322,7 +340,8 @@ xe_sched_job_snapshot_capture(struct xe_sched_job *job) · 53 unmodified lines
322	340
323	341	snapshot->batch_addr_len = q->width;
324	342	for (i = 0; i < q->width; i++)
325		− snapshot->batch_addr[i] = xe_device_uncanonicalize_addr(xe, job->batch_addr[i]);
	343	+ snapshot->batch_addr[i] =
	344	+ xe_device_uncanonicalize_addr(xe, job->ptrs[i].batch_addr);
326	345
327	346	return snapshot;
328	347	}

drivers/gpu/drm/xe/xe_sched_job.h+6−1

@@ -70,7 +70,12 @@ to_xe_sched_job(struct drm_sched_job *drm) · 69 unmodified lines
70	70
71	71	static inline u32 xe_sched_job_seqno(struct xe_sched_job *job)
72	72	{
73		− return job->fence->seqno;
	73	+ return job->fence ? job->fence->seqno : 0;
	74	+}
	75	+
	76	+static inline u32 xe_sched_job_lrc_seqno(struct xe_sched_job *job)
	77	+{
	78	+ return job->lrc_seqno;
74	79	}
75	80
76	81	static inline void

drivers/gpu/drm/xe/xe_sched_job_types.h+18−2

@@ -11,6 +11,20 @@ · 10 unmodified lines
11	11	#include <drm/gpu_scheduler.h>
12	12
13	13	struct xe_exec_queue;
	14	+struct dma_fence;
	15	+struct dma_fence_chain;
	16	+
	17	+/**
	18	+ * struct xe_job_ptrs - Per hw engine instance data
	19	+ */
	20	+struct xe_job_ptrs {
	21	+ /** @lrc_fence: Pre-allocated uinitialized lrc fence.*/
	22	+ struct dma_fence *lrc_fence;
	23	+ /** @chain_fence: Pre-allocated ninitialized fence chain node. */
	24	+ struct dma_fence_chain *chain_fence;
	25	+ /** @batch_addr: Batch buffer address. */
	26	+ u64 batch_addr;
	27	+};
14	28
15	29	/**
16	30	* struct xe_sched_job - XE schedule job (batch buffer tracking)
@@ -37,12 +51,14 @@ struct xe_sched_job { · 20 unmodified lines
37	51	/** @user_fence.value: write back value */
38	52	u64 value;
39	53	} user_fence;
	54	+ /** @lrc_seqno: LRC seqno */
	55	+ u32 lrc_seqno;
40	56	/** @migrate_flush_flags: Additional flush flags for migration jobs */
41	57	u32 migrate_flush_flags;
42	58	/** @ring_ops_flush_tlb: The ring ops need to flush TLB before payload. */
43	59	bool ring_ops_flush_tlb;
44		− /** @batch_addr: batch buffer address of job */
45		− u64 batch_addr[];
	60	+ /** @ptrs: per instance pointers. */
	61	+ struct xe_job_ptrs ptrs[];
46	62	};
47	63
48	64	struct xe_sched_job_snapshot {

drivers/gpu/drm/xe/xe_sriov.c+11−0

@@ -53,6 +53,7 @@ static bool test_is_vf(struct xe_device *xe) · 52 unmodified lines
53	53	*/
54	54	void xe_sriov_probe_early(struct xe_device *xe)
55	55	{
	56	+ struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
56	57	enum xe_sriov_mode mode = XE_SRIOV_MODE_NONE;
57	58	bool has_sriov = xe->info.has_sriov;
58	59
@@ -61,6 +62,16 @@ void xe_sriov_probe_early(struct xe_device *xe) · 2 unmodified lines
61	62	mode = XE_SRIOV_MODE_VF;
62	63	else if (xe_sriov_pf_readiness(xe))
63	64	mode = XE_SRIOV_MODE_PF;
	65	+ } else if (pci_sriov_get_totalvfs(pdev)) {
	66	+ /*
	67	+ * Even if we have not enabled SR-IOV support using the
	68	+ * platform specific has_sriov flag, the hardware may still
	69	+ * report SR-IOV capability and the PCI layer may wrongly
	70	+ * advertise driver support to enable VFs. Explicitly reset
	71	+ * the number of supported VFs to zero to avoid confusion.
	72	+ */
	73	+ drm_info(&xe->drm, "Support for SR-IOV is not available\n");
	74	+ pci_sriov_set_totalvfs(pdev, 0);
64	75	}
65	76
66	77	xe_assert(xe, !xe->sriov.__mode);

drivers/gpu/drm/xe/xe_sync.c+15−0

@@ -338,6 +338,21 @@ err_out: · 337 unmodified lines
338	338	return ERR_PTR(-ENOMEM);
339	339	}
340	340
	341	+/**
	342	+ * __xe_sync_ufence_get() - Get user fence from user fence
	343	+ * @ufence: input user fence
	344	+ *
	345	+ * Get a user fence reference from user fence
	346	+ *
	347	+ * Return: xe_user_fence pointer with reference
	348	+ */
	349	+struct xe_user_fence __xe_sync_ufence_get(struct xe_user_fence ufence)
	350	+{
	351	+ user_fence_get(ufence);
	352	+
	353	+ return ufence;
	354	+}
	355	+
341	356	/**
342	357	* xe_sync_ufence_get() - Get user fence from sync
343	358	* @sync: input sync

drivers/gpu/drm/xe/xe_sync.h+1−0

@@ -37,6 +37,7 @@ static inline bool xe_sync_is_ufence(struct xe_sync_entry *sync) · 36 unmodified lines
37	37	return !!sync->ufence;
38	38	}
39	39
	40	+struct xe_user_fence __xe_sync_ufence_get(struct xe_user_fence ufence);
40	41	struct xe_user_fence xe_sync_ufence_get(struct xe_sync_entry sync);
41	42	void xe_sync_ufence_put(struct xe_user_fence *ufence);
42	43	int xe_sync_ufence_get_status(struct xe_user_fence *ufence);

drivers/gpu/drm/xe/xe_tile_sysfs.c+2−2

@@ -22,7 +22,7 @@ static const struct kobj_type xe_tile_sysfs_kobj_type = { · 21 unmodified lines
22	22	.sysfs_ops = &kobj_sysfs_ops,
23	23	};
24	24
25		−static void tile_sysfs_fini(struct drm_device drm, void arg)
	25	+static void tile_sysfs_fini(void *arg)
26	26	{
27	27	struct xe_tile *tile = arg;
28	28
@@ -55,5 +55,5 @@ int xe_tile_sysfs_init(struct xe_tile *tile) · 26 unmodified lines
55	55	if (err)
56	56	return err;
57	57
58		− return drmm_add_action_or_reset(&xe->drm, tile_sysfs_fini, tile);
	58	+ return devm_add_action_or_reset(xe->drm.dev, tile_sysfs_fini, tile);
59	59	}

drivers/gpu/drm/xe/xe_trace.h+7−4

@@ -254,6 +254,7 @@ DECLARE_EVENT_CLASS(xe_sched_job, · 253 unmodified lines
254	254
255	255	TP_STRUCT__entry(
256	256	__field(u32, seqno)
	257	+ __field(u32, lrc_seqno)
257	258	__field(u16, guc_id)
258	259	__field(u32, guc_state)
259	260	__field(u32, flags)
@@ -264,17 +265,19 @@ DECLARE_EVENT_CLASS(xe_sched_job, · 4 unmodified lines
264	265
265	266	TP_fast_assign(
266	267	__entry->seqno = xe_sched_job_seqno(job);
	268	+ __entry->lrc_seqno = xe_sched_job_lrc_seqno(job);
267	269	__entry->guc_id = job->q->guc->id;
268	270	__entry->guc_state =
269	271	atomic_read(&job->q->guc->state);
270	272	__entry->flags = job->q->flags;
271		− __entry->error = job->fence->error;
	273	+ __entry->error = job->fence ? job->fence->error : 0;
272	274	__entry->fence = job->fence;
273		− __entry->batch_addr = (u64)job->batch_addr[0];
	275	+ __entry->batch_addr = (u64)job->ptrs[0].batch_addr;
274	276	),
275	277
276		− TP_printk("fence=%p, seqno=%u, guc_id=%d, batch_addr=0x%012llx, guc_state=0x%x, flags=0x%x, error=%d",
277		− __entry->fence, __entry->seqno, __entry->guc_id,
	278	+ TP_printk("fence=%p, seqno=%u, lrc_seqno=%u, guc_id=%d, batch_addr=0x%012llx, guc_state=0x%x, flags=0x%x, error=%d",
	279	+ __entry->fence, __entry->seqno,
	280	+ __entry->lrc_seqno, __entry->guc_id,
278	281	__entry->batch_addr, __entry->guc_state,
279	282	__entry->flags, __entry->error)
280	283	);

drivers/gpu/drm/xe/xe_ttm_stolen_mgr.c+2−0

@@ -13,11 +13,13 @@ · 12 unmodified lines
13	13
14	14	#include <generated/xe_wa_oob.h>
15	15
	16	+#include "regs/xe_bars.h"
16	17	#include "regs/xe_gt_regs.h"
17	18	#include "regs/xe_regs.h"
18	19	#include "xe_bo.h"
19	20	#include "xe_device.h"
20	21	#include "xe_gt.h"
	22	+#include "xe_gt_printk.h"
21	23	#include "xe_mmio.h"
22	24	#include "xe_res_cursor.h"
23	25	#include "xe_sriov.h"

drivers/gpu/drm/xe/xe_uc.c+42−16

@@ -5,15 +5,17 @@ · 4 unmodified lines
5	5
6	6	#include "xe_uc.h"
7	7
	8	+#include "xe_assert.h"
8	9	#include "xe_device.h"
9	10	#include "xe_gsc.h"
10	11	#include "xe_gsc_proxy.h"
11	12	#include "xe_gt.h"
	13	+#include "xe_gt_printk.h"
	14	+#include "xe_gt_sriov_vf.h"
12	15	#include "xe_guc.h"
13		−#include "xe_guc_db_mgr.h"
14	16	#include "xe_guc_pc.h"
15		−#include "xe_guc_submit.h"
16	17	#include "xe_huc.h"
	18	+#include "xe_sriov.h"
17	19	#include "xe_uc_fw.h"
18	20	#include "xe_wopcm.h"
19	21
@@ -51,19 +53,19 @@ int xe_uc_init(struct xe_uc *uc) · 31 unmodified lines
51	53	goto err;
52	54
53	55	if (!xe_device_uc_enabled(uc_to_xe(uc)))
54		− goto err;
	56	+ return 0;
55	57
56		− ret = xe_wopcm_init(&uc->wopcm);
57		− if (ret)
58		− goto err;
	58	+ if (IS_SRIOV_VF(uc_to_xe(uc)))
	59	+ return 0;
59	60
60		− ret = xe_guc_submit_init(&uc->guc);
	61	+ ret = xe_wopcm_init(&uc->wopcm);
61	62	if (ret)
62	63	goto err;
63	64
64		− ret = xe_guc_db_mgr_init(&uc->guc.dbm, ~0);
	65	+ return 0;
65	66
66	67	err:
	68	+ xe_gt_err(uc_to_gt(uc), "Failed to initialize uC (%pe)\n", ERR_PTR(ret));
67	69	return ret;
68	70	}
69	71
@@ -144,6 +146,31 @@ int xe_uc_init_hwconfig(struct xe_uc *uc) · 74 unmodified lines
144	146	return 0;
145	147	}
146	148
	149	+static int vf_uc_init_hw(struct xe_uc *uc)
	150	+{
	151	+ int err;
	152	+
	153	+ err = xe_uc_sanitize_reset(uc);
	154	+ if (err)
	155	+ return err;
	156	+
	157	+ err = xe_guc_enable_communication(&uc->guc);
	158	+ if (err)
	159	+ return err;
	160	+
	161	+ err = xe_gt_sriov_vf_connect(uc_to_gt(uc));
	162	+ if (err)
	163	+ return err;
	164	+
	165	+ uc->guc.submission_state.enabled = true;
	166	+
	167	+ err = xe_gt_record_default_lrcs(uc_to_gt(uc));
	168	+ if (err)
	169	+ return err;
	170	+
	171	+ return 0;
	172	+}
	173	+
147	174	/*
148	175	* Should be called during driver load, after every GT reset, and after every
149	176	* suspend to reload / auth the firmwares.
@@ -156,6 +183,9 @@ int xe_uc_init_hw(struct xe_uc *uc) · 6 unmodified lines
156	183	if (!xe_device_uc_enabled(uc_to_xe(uc)))
157	184	return 0;
158	185
	186	+ if (IS_SRIOV_VF(uc_to_xe(uc)))
	187	+ return vf_uc_init_hw(uc);
	188	+
159	189	ret = xe_huc_upload(&uc->huc);
160	190	if (ret)
161	191	return ret;
@@ -215,13 +245,13 @@ void xe_uc_stop_prepare(struct xe_uc *uc) · 53 unmodified lines
215	245	xe_guc_stop_prepare(&uc->guc);
216	246	}
217	247
218		−int xe_uc_stop(struct xe_uc *uc)
	248	+void xe_uc_stop(struct xe_uc *uc)
219	249	{
220	250	/* GuC submission not enabled, nothing to do */
221	251	if (!xe_device_uc_enabled(uc_to_xe(uc)))
222		− return 0;
	252	+ return;
223	253
224		− return xe_guc_stop(&uc->guc);
	254	+ xe_guc_stop(&uc->guc);
225	255	}
226	256
227	257	int xe_uc_start(struct xe_uc *uc)
@@ -247,17 +277,13 @@ again: · 19 unmodified lines
247	277
248	278	int xe_uc_suspend(struct xe_uc *uc)
249	279	{
250		− int ret;
251		−
252	280	/* GuC submission not enabled, nothing to do */
253	281	if (!xe_device_uc_enabled(uc_to_xe(uc)))
254	282	return 0;
255	283
256	284	uc_reset_wait(uc);
257	285
258		− ret = xe_uc_stop(uc);
259		− if (ret)
260		− return ret;
	286	+ xe_uc_stop(uc);
261	287
262	288	return xe_guc_suspend(&uc->guc);
263	289	}

drivers/gpu/drm/xe/xe_uc.h+2−2

@@ -6,7 +6,7 @@ · 5 unmodified lines
6	6	#ifndef _XE_UC_H_
7	7	#define _XE_UC_H_
8	8
9		−#include "xe_uc_types.h"
	9	+struct xe_uc;
10	10
11	11	int xe_uc_init(struct xe_uc *uc);
12	12	int xe_uc_init_hwconfig(struct xe_uc *uc);
@@ -16,7 +16,7 @@ int xe_uc_fini_hw(struct xe_uc *uc); · 3 unmodified lines
16	16	void xe_uc_gucrc_disable(struct xe_uc *uc);
17	17	int xe_uc_reset_prepare(struct xe_uc *uc);
18	18	void xe_uc_stop_prepare(struct xe_uc *uc);
19		−int xe_uc_stop(struct xe_uc *uc);
	19	+void xe_uc_stop(struct xe_uc *uc);
20	20	int xe_uc_start(struct xe_uc *uc);
21	21	int xe_uc_suspend(struct xe_uc *uc);
22	22	int xe_uc_sanitize_reset(struct xe_uc *uc);

drivers/gpu/drm/xe/xe_uc_fw.c+5−0

@@ -14,6 +14,7 @@ · 13 unmodified lines
14	14	#include "xe_force_wake.h"
15	15	#include "xe_gsc.h"
16	16	#include "xe_gt.h"
	17	+#include "xe_gt_printk.h"
17	18	#include "xe_map.h"
18	19	#include "xe_mmio.h"
19	20	#include "xe_module.h"
@@ -653,6 +654,10 @@ static int uc_fw_request(struct xe_uc_fw uc_fw, const struct firmware *firmwar · 633 unmodified lines
653	654	uc_fw_auto_select(xe, uc_fw);
654	655
655	656	if (IS_SRIOV_VF(xe)) {
	657	+ /* Only GuC/HuC are supported */
	658	+ if (uc_fw->type != XE_UC_FW_TYPE_GUC &&
	659	+ uc_fw->type != XE_UC_FW_TYPE_HUC)
	660	+ uc_fw->path = NULL;
656	661	/* VF will support only firmwares that driver can autoselect */
657	662	xe_uc_fw_change_status(uc_fw, uc_fw->path ?
658	663	XE_UC_FIRMWARE_PRELOADED :

drivers/gpu/drm/xe/xe_vm.c+495−350

@@ -85,8 +85,8 @@ static bool preempt_fences_waiting(struct xe_vm *vm) · 84 unmodified lines
85	85
86	86	list_for_each_entry(q, &vm->preempt.exec_queues, compute.link) {
87	87	if (!q->compute.pfence \|\|
88		− (q->compute.pfence && test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
89		− &q->compute.pfence->flags))) {
	88	+ test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
	89	+ &q->compute.pfence->flags)) {
90	90	return true;
91	91	}
92	92	}
@@ -315,19 +315,23 @@ int __xe_vm_userptr_needs_repin(struct xe_vm *vm) · 222 unmodified lines
315	315
316	316	#define XE_VM_REBIND_RETRY_TIMEOUT_MS 1000
317	317
318		−static void xe_vm_kill(struct xe_vm *vm)
	318	+static void xe_vm_kill(struct xe_vm *vm, bool unlocked)
319	319	{
320	320	struct xe_exec_queue *q;
321	321
322	322	lockdep_assert_held(&vm->lock);
323	323
324		− xe_vm_lock(vm, false);
	324	+ if (unlocked)
	325	+ xe_vm_lock(vm, false);
	326	+
325	327	vm->flags \|= XE_VM_FLAG_BANNED;
326	328	trace_xe_vm_kill(vm);
327	329
328	330	list_for_each_entry(q, &vm->preempt.exec_queues, compute.link)
329	331	q->ops->kill(q);
330		− xe_vm_unlock(vm);
	332	+
	333	+ if (unlocked)
	334	+ xe_vm_unlock(vm);
331	335
332	336	/* TODO: Inform user the VM is banned */
333	337	}
@@ -557,7 +561,7 @@ out_unlock_outer: · 223 unmodified lines
557	561
558	562	if (err) {
559	563	drm_warn(&vm->xe->drm, "VM worker error: %d\n", err);
560		− xe_vm_kill(vm);
	564	+ xe_vm_kill(vm, true);
561	565	}
562	566	up_write(&vm->lock);
563	567
@@ -708,37 +712,116 @@ int xe_vm_userptr_check_repin(struct xe_vm *vm) · 144 unmodified lines
708	712	list_empty_careful(&vm->userptr.invalidated)) ? 0 : -EAGAIN;
709	713	}
710	714
711		−static struct dma_fence *
712		−xe_vm_bind_vma(struct xe_vma vma, struct xe_exec_queue q,
713		− struct xe_sync_entry *syncs, u32 num_syncs,
714		− bool first_op, bool last_op);
	715	+static void xe_vm_populate_rebind(struct xe_vma_op op, struct xe_vma vma,
	716	+ u8 tile_mask)
	717	+{
	718	+ INIT_LIST_HEAD(&op->link);
	719	+ op->tile_mask = tile_mask;
	720	+ op->base.op = DRM_GPUVA_OP_MAP;
	721	+ op->base.map.va.addr = vma->gpuva.va.addr;
	722	+ op->base.map.va.range = vma->gpuva.va.range;
	723	+ op->base.map.gem.obj = vma->gpuva.gem.obj;
	724	+ op->base.map.gem.offset = vma->gpuva.gem.offset;
	725	+ op->map.vma = vma;
	726	+ op->map.immediate = true;
	727	+ op->map.dumpable = vma->gpuva.flags & XE_VMA_DUMPABLE;
	728	+ op->map.is_null = xe_vma_is_null(vma);
	729	+}
	730	+
	731	+static int xe_vm_ops_add_rebind(struct xe_vma_ops vops, struct xe_vma vma,
	732	+ u8 tile_mask)
	733	+{
	734	+ struct xe_vma_op *op;
	735	+
	736	+ op = kzalloc(sizeof(*op), GFP_KERNEL);
	737	+ if (!op)
	738	+ return -ENOMEM;
	739	+
	740	+ xe_vm_populate_rebind(op, vma, tile_mask);
	741	+ list_add_tail(&op->link, &vops->list);
	742	+
	743	+ return 0;
	744	+}
	745	+
	746	+static struct dma_fence ops_execute(struct xe_vm vm,
	747	+ struct xe_vma_ops *vops);
	748	+static void xe_vma_ops_init(struct xe_vma_ops vops, struct xe_vm vm,
	749	+ struct xe_exec_queue *q,
	750	+ struct xe_sync_entry *syncs, u32 num_syncs);
715	751
716	752	int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)
717	753	{
718	754	struct dma_fence *fence;
719	755	struct xe_vma vma, next;
	756	+ struct xe_vma_ops vops;
	757	+ struct xe_vma_op op, next_op;
	758	+ int err;
720	759
721	760	lockdep_assert_held(&vm->lock);
722		− if (xe_vm_in_lr_mode(vm) && !rebind_worker)
	761	+ if ((xe_vm_in_lr_mode(vm) && !rebind_worker) \|\|
	762	+ list_empty(&vm->rebind_list))
723	763	return 0;
724	764
	765	+ xe_vma_ops_init(&vops, vm, NULL, NULL, 0);
	766	+
725	767	xe_vm_assert_held(vm);
726		− list_for_each_entry_safe(vma, next, &vm->rebind_list,
727		− combined_links.rebind) {
	768	+ list_for_each_entry(vma, &vm->rebind_list, combined_links.rebind) {
728	769	xe_assert(vm->xe, vma->tile_present);
729	770
730		− list_del_init(&vma->combined_links.rebind);
731	771	if (rebind_worker)
732	772	trace_xe_vma_rebind_worker(vma);
733	773	else
734	774	trace_xe_vma_rebind_exec(vma);
735		− fence = xe_vm_bind_vma(vma, NULL, NULL, 0, false, false);
736		− if (IS_ERR(fence))
737		− return PTR_ERR(fence);
	775	+
	776	+ err = xe_vm_ops_add_rebind(&vops, vma,
	777	+ vma->tile_present);
	778	+ if (err)
	779	+ goto free_ops;
	780	+ }
	781	+
	782	+ fence = ops_execute(vm, &vops);
	783	+ if (IS_ERR(fence)) {
	784	+ err = PTR_ERR(fence);
	785	+ } else {
738	786	dma_fence_put(fence);
	787	+ list_for_each_entry_safe(vma, next, &vm->rebind_list,
	788	+ combined_links.rebind)
	789	+ list_del_init(&vma->combined_links.rebind);
	790	+ }
	791	+free_ops:
	792	+ list_for_each_entry_safe(op, next_op, &vops.list, link) {
	793	+ list_del(&op->link);
	794	+ kfree(op);
739	795	}
740	796
741		− return 0;
	797	+ return err;
	798	+}
	799	+
	800	+struct dma_fence xe_vma_rebind(struct xe_vm vm, struct xe_vma *vma, u8 tile_mask)
	801	+{
	802	+ struct dma_fence *fence = NULL;
	803	+ struct xe_vma_ops vops;
	804	+ struct xe_vma_op op, next_op;
	805	+ int err;
	806	+
	807	+ lockdep_assert_held(&vm->lock);
	808	+ xe_vm_assert_held(vm);
	809	+ xe_assert(vm->xe, xe_vm_in_fault_mode(vm));
	810	+
	811	+ xe_vma_ops_init(&vops, vm, NULL, NULL, 0);
	812	+
	813	+ err = xe_vm_ops_add_rebind(&vops, vma, tile_mask);
	814	+ if (err)
	815	+ return ERR_PTR(err);
	816	+
	817	+ fence = ops_execute(vm, &vops);
	818	+
	819	+ list_for_each_entry_safe(op, next_op, &vops.list, link) {
	820	+ list_del(&op->link);
	821	+ kfree(op);
	822	+ }
	823	+
	824	+ return fence;
742	825	}
743	826
744	827	static void xe_vma_free(struct xe_vma *vma)
@@ -805,7 +888,7 @@ static struct xe_vma xe_vma_create(struct xe_vm vm, · 60 unmodified lines
805	888	for_each_tile(tile, vm->xe, id)
806	889	vma->tile_mask \|= 0x1 << id;
807	890
808		− if (GRAPHICS_VER(vm->xe) >= 20 \|\| vm->xe->info.platform == XE_PVC)
	891	+ if (vm->xe->info.has_atomic_enable_pte_bit)
809	892	vma->gpuva.flags \|= XE_VMA_ATOMIC_PTE_BIT;
810	893
811	894	vma->pat_index = pat_index;
@@ -1173,6 +1256,8 @@ static const struct xe_pt_ops xelp_pt_ops = { · 361 unmodified lines
1173	1256	.pde_encode_bo = xelp_pde_encode_bo,
1174	1257	};
1175	1258
	1259	+static void vm_destroy_work_func(struct work_struct *w);
	1260	+
1176	1261	/**
1177	1262	* xe_vm_create_scratch() - Setup a scratch memory pagetable tree for the
1178	1263	* given tile and vm.
@@ -1252,6 +1337,8 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 73 unmodified lines
1252	1337	init_rwsem(&vm->userptr.notifier_lock);
1253	1338	spin_lock_init(&vm->userptr.invalidated_lock);
1254	1339
	1340	+ INIT_WORK(&vm->destroy_work, vm_destroy_work_func);
	1341	+
1255	1342	INIT_LIST_HEAD(&vm->preempt.exec_queues);
1256	1343	vm->preempt.min_run_period_ms = 10; /* FIXME: Wire up to uAPI */
1257	1344
@@ -1260,7 +1347,13 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 2 unmodified lines
1260	1347
1261	1348	vm->pt_ops = &xelp_pt_ops;
1262	1349
1263		− if (!(flags & XE_VM_FLAG_MIGRATION))
	1350	+ /*
	1351	+ * Long-running workloads are not protected by the scheduler references.
	1352	+ * By design, run_job for long-running workloads returns NULL and the
	1353	+ * scheduler drops all the references of it, hence protecting the VM
	1354	+ * for this case is necessary.
	1355	+ */
	1356	+ if (flags & XE_VM_FLAG_LR_MODE)
1264	1357	xe_pm_runtime_get_noresume(xe);
1265	1358
1266	1359	vm_resv_obj = drm_gpuvm_resv_object_alloc(&xe->drm);
@@ -1274,7 +1367,7 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 7 unmodified lines
1274	1367
1275	1368	drm_gem_object_put(vm_resv_obj);
1276	1369
1277		− err = dma_resv_lock_interruptible(xe_vm_resv(vm), NULL);
	1370	+ err = xe_vm_lock(vm, true);
1278	1371	if (err)
1279	1372	goto err_close;
1280	1373
@@ -1318,7 +1411,7 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 37 unmodified lines
1318	1411
1319	1412	xe_pt_populate_empty(tile, vm, vm->pt_root[id]);
1320	1413	}
1321		− dma_resv_unlock(xe_vm_resv(vm));
	1414	+ xe_vm_unlock(vm);
1322	1415
1323	1416	/* Kernel migration VM shouldn't have a circular loop.. */
1324	1417	if (!(flags & XE_VM_FLAG_MIGRATION)) {
@@ -1360,7 +1453,7 @@ struct xe_vm xe_vm_create(struct xe_device xe, u32 flags) · 35 unmodified lines
1360	1453	return vm;
1361	1454
1362	1455	err_unlock_close:
1363		− dma_resv_unlock(xe_vm_resv(vm));
	1456	+ xe_vm_unlock(vm);
1364	1457	err_close:
1365	1458	xe_vm_close_and_put(vm);
1366	1459	return ERR_PTR(err);
@@ -1370,7 +1463,7 @@ err_no_resv: · 3 unmodified lines
1370	1463	for_each_tile(tile, xe, id)
1371	1464	xe_range_fence_tree_fini(&vm->rftree[id]);
1372	1465	kfree(vm);
1373		− if (!(flags & XE_VM_FLAG_MIGRATION))
	1466	+ if (flags & XE_VM_FLAG_LR_MODE)
1374	1467	xe_pm_runtime_put(xe);
1375	1468	return ERR_PTR(err);
1376	1469	}
@@ -1489,9 +1582,10 @@ void xe_vm_close_and_put(struct xe_vm *vm) · 112 unmodified lines
1489	1582	xe_vm_put(vm);
1490	1583	}
1491	1584
1492		−static void xe_vm_free(struct drm_gpuvm *gpuvm)
	1585	+static void vm_destroy_work_func(struct work_struct *w)
1493	1586	{
1494		− struct xe_vm *vm = container_of(gpuvm, struct xe_vm, gpuvm);
	1587	+ struct xe_vm *vm =
	1588	+ container_of(w, struct xe_vm, destroy_work);
1495	1589	struct xe_device *xe = vm->xe;
1496	1590	struct xe_tile *tile;
1497	1591	u8 id;
@@ -1504,7 +1598,7 @@ static void xe_vm_free(struct drm_gpuvm *gpuvm) · 6 unmodified lines
1504	1598
1505	1599	mutex_destroy(&vm->snap_mutex);
1506	1600
1507		− if (!(vm->flags & XE_VM_FLAG_MIGRATION))
	1601	+ if (vm->flags & XE_VM_FLAG_LR_MODE)
1508	1602	xe_pm_runtime_put(xe);
1509	1603
1510	1604	for_each_tile(tile, xe, id)
@@ -1514,6 +1608,14 @@ static void xe_vm_free(struct drm_gpuvm *gpuvm) · 3 unmodified lines
1514	1608	kfree(vm);
1515	1609	}
1516	1610
	1611	+static void xe_vm_free(struct drm_gpuvm *gpuvm)
	1612	+{
	1613	+ struct xe_vm *vm = container_of(gpuvm, struct xe_vm, gpuvm);
	1614	+
	1615	+ /* To destroy the VM we need to be able to sleep */
	1616	+ queue_work(system_unbound_wq, &vm->destroy_work);
	1617	+}
	1618	+
1517	1619	struct xe_vm xe_vm_lookup(struct xe_file xef, u32 id)
1518	1620	{
1519	1621	struct xe_vm *vm;
@@ -1550,23 +1652,13 @@ xe_vm_unbind_vma(struct xe_vma vma, struct xe_exec_queue q, · 30 unmodified lines
1550	1652	struct dma_fence *fence = NULL;
1551	1653	struct dma_fence **fences = NULL;
1552	1654	struct dma_fence_array *cf = NULL;
1553		− int cur_fence = 0, i;
	1655	+ int cur_fence = 0;
1554	1656	int number_tiles = hweight8(vma->tile_present);
1555	1657	int err;
1556	1658	u8 id;
1557	1659
1558	1660	trace_xe_vma_unbind(vma);
1559	1661
1560		− if (vma->ufence) {
1561		− struct xe_user_fence * const f = vma->ufence;
1562		−
1563		− if (!xe_sync_ufence_get_status(f))
1564		− return ERR_PTR(-EBUSY);
1565		−
1566		− vma->ufence = NULL;
1567		− xe_sync_ufence_put(f);
1568		− }
1569		−
1570	1662	if (number_tiles > 1) {
1571	1663	fences = kmalloc_array(number_tiles, sizeof(*fences),
1572	1664	GFP_KERNEL);
@@ -1608,10 +1700,6 @@ next: · 35 unmodified lines
1608	1700
1609	1701	fence = cf ? &cf->base : !fence ?
1610	1702	xe_exec_queue_last_fence_get(wait_exec_queue, vm) : fence;
1611		− if (last_op) {
1612		− for (i = 0; i < num_syncs; i++)
1613		− xe_sync_entry_signal(&syncs[i], fence);
1614		− }
1615	1703
1616	1704	return fence;
1617	1705
@@ -1628,15 +1716,15 @@ err_fences: · 10 unmodified lines
1628	1716	static struct dma_fence *
1629	1717	xe_vm_bind_vma(struct xe_vma vma, struct xe_exec_queue q,
1630	1718	struct xe_sync_entry *syncs, u32 num_syncs,
1631		− bool first_op, bool last_op)
	1719	+ u8 tile_mask, bool first_op, bool last_op)
1632	1720	{
1633	1721	struct xe_tile *tile;
1634	1722	struct dma_fence *fence;
1635	1723	struct dma_fence **fences = NULL;
1636	1724	struct dma_fence_array *cf = NULL;
1637	1725	struct xe_vm *vm = xe_vma_vm(vma);
1638		− int cur_fence = 0, i;
1639		− int number_tiles = hweight8(vma->tile_mask);
	1726	+ int cur_fence = 0;
	1727	+ int number_tiles = hweight8(tile_mask);
1640	1728	int err;
1641	1729	u8 id;
1642	1730
@@ -1650,7 +1738,7 @@ xe_vm_bind_vma(struct xe_vma vma, struct xe_exec_queue q, · 7 unmodified lines
1650	1738	}
1651	1739
1652	1740	for_each_tile(tile, vm->xe, id) {
1653		− if (!(vma->tile_mask & BIT(id)))
	1741	+ if (!(tile_mask & BIT(id)))
1654	1742	goto next;
1655	1743
1656	1744	fence = __xe_pt_bind_vma(tile, vma, q ? q : vm->q[id],
@@ -1682,12 +1770,6 @@ next: · 25 unmodified lines
1682	1770	}
1683	1771	}
1684	1772
1685		− if (last_op) {
1686		− for (i = 0; i < num_syncs; i++)
1687		− xe_sync_entry_signal(&syncs[i],
1688		− cf ? &cf->base : fence);
1689		− }
1690		−
1691	1773	return cf ? &cf->base : fence;
1692	1774
1693	1775	err_fences:
@@ -1715,87 +1797,46 @@ find_ufence_get(struct xe_sync_entry *syncs, u32 num_syncs) · 21 unmodified lines
1715	1797	return NULL;
1716	1798	}
1717	1799
1718		−static int __xe_vm_bind(struct xe_vm vm, struct xe_vma vma,
1719		− struct xe_exec_queue q, struct xe_sync_entry syncs,
1720		− u32 num_syncs, bool immediate, bool first_op,
1721		− bool last_op)
	1800	+static struct dma_fence *
	1801	+xe_vm_bind(struct xe_vm vm, struct xe_vma vma, struct xe_exec_queue *q,
	1802	+ struct xe_bo bo, struct xe_sync_entry syncs, u32 num_syncs,
	1803	+ u8 tile_mask, bool immediate, bool first_op, bool last_op)
1722	1804	{
1723	1805	struct dma_fence *fence;
1724	1806	struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
1725		− struct xe_user_fence *ufence;
1726	1807
1727	1808	xe_vm_assert_held(vm);
1728		−
1729		− ufence = find_ufence_get(syncs, num_syncs);
1730		− if (vma->ufence && ufence)
1731		− xe_sync_ufence_put(vma->ufence);
1732		−
1733		− vma->ufence = ufence ?: vma->ufence;
	1809	+ xe_bo_assert_held(bo);
1734	1810
1735	1811	if (immediate) {
1736		− fence = xe_vm_bind_vma(vma, q, syncs, num_syncs, first_op,
1737		− last_op);
	1812	+ fence = xe_vm_bind_vma(vma, q, syncs, num_syncs, tile_mask,
	1813	+ first_op, last_op);
1738	1814	if (IS_ERR(fence))
1739		− return PTR_ERR(fence);
	1815	+ return fence;
1740	1816	} else {
1741		− int i;
1742		−
1743	1817	xe_assert(vm->xe, xe_vm_in_fault_mode(vm));
1744	1818
1745	1819	fence = xe_exec_queue_last_fence_get(wait_exec_queue, vm);
1746		− if (last_op) {
1747		− for (i = 0; i < num_syncs; i++)
1748		− xe_sync_entry_signal(&syncs[i], fence);
1749		− }
1750	1820	}
1751	1821
1752		− if (last_op)
1753		− xe_exec_queue_last_fence_set(wait_exec_queue, vm, fence);
1754		− dma_fence_put(fence);
1755		−
1756		− return 0;
1757		−}
1758		−
1759		−static int xe_vm_bind(struct xe_vm vm, struct xe_vma vma, struct xe_exec_queue *q,
1760		− struct xe_bo bo, struct xe_sync_entry syncs,
1761		− u32 num_syncs, bool immediate, bool first_op,
1762		− bool last_op)
1763		−{
1764		− int err;
1765		−
1766		− xe_vm_assert_held(vm);
1767		− xe_bo_assert_held(bo);
1768		−
1769		− if (bo && immediate) {
1770		− err = xe_bo_validate(bo, vm, true);
1771		− if (err)
1772		− return err;
1773		− }
1774		−
1775		− return __xe_vm_bind(vm, vma, q, syncs, num_syncs, immediate, first_op,
1776		− last_op);
	1822	+ return fence;
1777	1823	}
1778	1824
1779		−static int xe_vm_unbind(struct xe_vm vm, struct xe_vma vma,
1780		− struct xe_exec_queue q, struct xe_sync_entry syncs,
1781		− u32 num_syncs, bool first_op, bool last_op)
	1825	+static struct dma_fence *
	1826	+xe_vm_unbind(struct xe_vm vm, struct xe_vma vma,
	1827	+ struct xe_exec_queue q, struct xe_sync_entry syncs,
	1828	+ u32 num_syncs, bool first_op, bool last_op)
1782	1829	{
1783	1830	struct dma_fence *fence;
1784		− struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
1785	1831
1786	1832	xe_vm_assert_held(vm);
1787	1833	xe_bo_assert_held(xe_vma_bo(vma));
1788	1834
1789	1835	fence = xe_vm_unbind_vma(vma, q, syncs, num_syncs, first_op, last_op);
1790	1836	if (IS_ERR(fence))
1791		− return PTR_ERR(fence);
	1837	+ return fence;
1792	1838
1793		− xe_vma_destroy(vma, fence);
1794		− if (last_op)
1795		− xe_exec_queue_last_fence_set(wait_exec_queue, vm, fence);
1796		− dma_fence_put(fence);
1797		−
1798		− return 0;
	1839	+ return fence;
1799	1840	}
1800	1841
1801	1842	#define ALL_DRM_XE_VM_CREATE_FLAGS (DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE \| \
@@ -1938,40 +1979,18 @@ static const u32 region_to_mem_type[] = { · 136 unmodified lines
1938	1979	XE_PL_VRAM1,
1939	1980	};
1940	1981
1941		−static int xe_vm_prefetch(struct xe_vm vm, struct xe_vma vma,
1942		− struct xe_exec_queue *q, u32 region,
1943		− struct xe_sync_entry *syncs, u32 num_syncs,
1944		− bool first_op, bool last_op)
	1982	+static struct dma_fence *
	1983	+xe_vm_prefetch(struct xe_vm vm, struct xe_vma vma,
	1984	+ struct xe_exec_queue q, struct xe_sync_entry syncs,
	1985	+ u32 num_syncs, bool first_op, bool last_op)
1945	1986	{
1946	1987	struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
1947		− int err;
1948		−
1949		− xe_assert(vm->xe, region < ARRAY_SIZE(region_to_mem_type));
1950		−
1951		− if (!xe_vma_has_no_bo(vma)) {
1952		− err = xe_bo_migrate(xe_vma_bo(vma), region_to_mem_type[region]);
1953		− if (err)
1954		− return err;
1955		− }
1956	1988
1957	1989	if (vma->tile_mask != (vma->tile_present & ~vma->tile_invalidated)) {
1958	1990	return xe_vm_bind(vm, vma, q, xe_vma_bo(vma), syncs, num_syncs,
1959		− true, first_op, last_op);
	1991	+ vma->tile_mask, true, first_op, last_op);
1960	1992	} else {
1961		− int i;
1962		−
1963		− /* Nothing to do, signal fences now */
1964		− if (last_op) {
1965		− for (i = 0; i < num_syncs; i++) {
1966		− struct dma_fence *fence =
1967		− xe_exec_queue_last_fence_get(wait_exec_queue, vm);
1968		−
1969		− xe_sync_entry_signal(&syncs[i], fence);
1970		− dma_fence_put(fence);
1971		− }
1972		− }
1973		−
1974		− return 0;
	1993	+ return xe_exec_queue_last_fence_get(wait_exec_queue, vm);
1975	1994	}
1976	1995	}
1977	1996
@@ -2267,23 +2286,28 @@ static int xe_vma_op_commit(struct xe_vm vm, struct xe_vma_op op) · 289 unmodified lines
2267	2286	static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q,
2268	2287	struct drm_gpuva_ops *ops,
2269	2288	struct xe_sync_entry *syncs, u32 num_syncs,
2270		− struct list_head *ops_list, bool last)
	2289	+ struct xe_vma_ops *vops, bool last)
2271	2290	{
2272	2291	struct xe_device *xe = vm->xe;
2273	2292	struct xe_vma_op *last_op = NULL;
2274	2293	struct drm_gpuva_op *__op;
	2294	+ struct xe_tile *tile;
	2295	+ u8 id, tile_mask = 0;
2275	2296	int err = 0;
2276	2297
2277	2298	lockdep_assert_held_write(&vm->lock);
2278	2299
	2300	+ for_each_tile(tile, vm->xe, id)
	2301	+ tile_mask \|= 0x1 << id;
	2302	+
2279	2303	drm_gpuva_for_each_op(__op, ops) {
2280	2304	struct xe_vma_op *op = gpuva_op_to_vma_op(__op);
2281	2305	struct xe_vma *vma;
2282		− bool first = list_empty(ops_list);
	2306	+ bool first = list_empty(&vops->list);
2283	2307	unsigned int flags = 0;
2284	2308
2285	2309	INIT_LIST_HEAD(&op->link);
2286		− list_add_tail(&op->link, ops_list);
	2310	+ list_add_tail(&op->link, &vops->list);
2287	2311
2288	2312	if (first) {
2289	2313	op->flags \|= XE_VMA_OP_FIRST;
@@ -2292,6 +2316,7 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q, · 2 unmodified lines
2292	2316	}
2293	2317
2294	2318	op->q = q;
	2319	+ op->tile_mask = tile_mask;
2295	2320
2296	2321	switch (op->base.op) {
2297	2322	case DRM_GPUVA_OP_MAP:
@@ -2409,12 +2434,11 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q, · 111 unmodified lines
2409	2434	}
2410	2435
2411	2436	/* FIXME: Unhandled corner case */
2412		− XE_WARN_ON(!last_op && last && !list_empty(ops_list));
	2437	+ XE_WARN_ON(!last_op && last && !list_empty(&vops->list));
2413	2438
2414	2439	if (!last_op)
2415	2440	return 0;
2416	2441
2417		− last_op->ops = ops;
2418	2442	if (last) {
2419	2443	last_op->flags \|= XE_VMA_OP_LAST;
2420	2444	last_op->num_syncs = num_syncs;
@@ -2424,27 +2448,24 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm vm, struct xe_exec_queue q, · 3 unmodified lines
2424	2448	return 0;
2425	2449	}
2426	2450
2427		−static int op_execute(struct drm_exec exec, struct xe_vm vm,
2428		− struct xe_vma vma, struct xe_vma_op op)
	2451	+static struct dma_fence op_execute(struct xe_vm vm, struct xe_vma *vma,
	2452	+ struct xe_vma_op *op)
2429	2453	{
2430		− int err;
2431		−
2432		− lockdep_assert_held_write(&vm->lock);
	2454	+ struct dma_fence *fence = NULL;
2433	2455
2434		− err = xe_vm_lock_vma(exec, vma);
2435		− if (err)
2436		− return err;
	2456	+ lockdep_assert_held(&vm->lock);
2437	2457
2438	2458	xe_vm_assert_held(vm);
2439	2459	xe_bo_assert_held(xe_vma_bo(vma));
2440	2460
2441	2461	switch (op->base.op) {
2442	2462	case DRM_GPUVA_OP_MAP:
2443		− err = xe_vm_bind(vm, vma, op->q, xe_vma_bo(vma),
2444		− op->syncs, op->num_syncs,
2445		− op->map.immediate \|\| !xe_vm_in_fault_mode(vm),
2446		− op->flags & XE_VMA_OP_FIRST,
2447		− op->flags & XE_VMA_OP_LAST);
	2463	+ fence = xe_vm_bind(vm, vma, op->q, xe_vma_bo(vma),
	2464	+ op->syncs, op->num_syncs,
	2465	+ op->tile_mask,
	2466	+ op->map.immediate \|\| !xe_vm_in_fault_mode(vm),
	2467	+ op->flags & XE_VMA_OP_FIRST,
	2468	+ op->flags & XE_VMA_OP_LAST);
2448	2469	break;
2449	2470	case DRM_GPUVA_OP_REMAP:
2450	2471	{
@@ -2454,37 +2475,41 @@ static int op_execute(struct drm_exec exec, struct xe_vm vm, · 3 unmodified lines
2454	2475	if (!op->remap.unmap_done) {
2455	2476	if (prev \|\| next)
2456	2477	vma->gpuva.flags \|= XE_VMA_FIRST_REBIND;
2457		− err = xe_vm_unbind(vm, vma, op->q, op->syncs,
2458		− op->num_syncs,
2459		− op->flags & XE_VMA_OP_FIRST,
2460		− op->flags & XE_VMA_OP_LAST &&
2461		− !prev && !next);
2462		− if (err)
	2478	+ fence = xe_vm_unbind(vm, vma, op->q, op->syncs,
	2479	+ op->num_syncs,
	2480	+ op->flags & XE_VMA_OP_FIRST,
	2481	+ op->flags & XE_VMA_OP_LAST &&
	2482	+ !prev && !next);
	2483	+ if (IS_ERR(fence))
2463	2484	break;
2464	2485	op->remap.unmap_done = true;
2465	2486	}
2466	2487
2467	2488	if (prev) {
2468	2489	op->remap.prev->gpuva.flags \|= XE_VMA_LAST_REBIND;
2469		− err = xe_vm_bind(vm, op->remap.prev, op->q,
2470		− xe_vma_bo(op->remap.prev), op->syncs,
2471		− op->num_syncs, true, false,
2472		− op->flags & XE_VMA_OP_LAST && !next);
	2490	+ dma_fence_put(fence);
	2491	+ fence = xe_vm_bind(vm, op->remap.prev, op->q,
	2492	+ xe_vma_bo(op->remap.prev), op->syncs,
	2493	+ op->num_syncs,
	2494	+ op->remap.prev->tile_mask, true,
	2495	+ false,
	2496	+ op->flags & XE_VMA_OP_LAST && !next);
2473	2497	op->remap.prev->gpuva.flags &= ~XE_VMA_LAST_REBIND;
2474		− if (err)
	2498	+ if (IS_ERR(fence))
2475	2499	break;
2476	2500	op->remap.prev = NULL;
2477	2501	}
2478	2502
2479	2503	if (next) {
2480	2504	op->remap.next->gpuva.flags \|= XE_VMA_LAST_REBIND;
2481		− err = xe_vm_bind(vm, op->remap.next, op->q,
2482		− xe_vma_bo(op->remap.next),
2483		− op->syncs, op->num_syncs,
2484		− true, false,
2485		− op->flags & XE_VMA_OP_LAST);
	2505	+ dma_fence_put(fence);
	2506	+ fence = xe_vm_bind(vm, op->remap.next, op->q,
	2507	+ xe_vma_bo(op->remap.next),
	2508	+ op->syncs, op->num_syncs,
	2509	+ op->remap.next->tile_mask, true,
	2510	+ false, op->flags & XE_VMA_OP_LAST);
2486	2511	op->remap.next->gpuva.flags &= ~XE_VMA_LAST_REBIND;
2487		− if (err)
	2512	+ if (IS_ERR(fence))
2488	2513	break;
2489	2514	op->remap.next = NULL;
2490	2515	}
@@ -2492,43 +2517,35 @@ static int op_execute(struct drm_exec exec, struct xe_vm vm, · 1 unmodified line
2492	2517	break;
2493	2518	}
2494	2519	case DRM_GPUVA_OP_UNMAP:
2495		− err = xe_vm_unbind(vm, vma, op->q, op->syncs,
2496		− op->num_syncs, op->flags & XE_VMA_OP_FIRST,
2497		− op->flags & XE_VMA_OP_LAST);
	2520	+ fence = xe_vm_unbind(vm, vma, op->q, op->syncs,
	2521	+ op->num_syncs, op->flags & XE_VMA_OP_FIRST,
	2522	+ op->flags & XE_VMA_OP_LAST);
2498	2523	break;
2499	2524	case DRM_GPUVA_OP_PREFETCH:
2500		− err = xe_vm_prefetch(vm, vma, op->q, op->prefetch.region,
2501		− op->syncs, op->num_syncs,
2502		− op->flags & XE_VMA_OP_FIRST,
2503		− op->flags & XE_VMA_OP_LAST);
	2525	+ fence = xe_vm_prefetch(vm, vma, op->q, op->syncs, op->num_syncs,
	2526	+ op->flags & XE_VMA_OP_FIRST,
	2527	+ op->flags & XE_VMA_OP_LAST);
2504	2528	break;
2505	2529	default:
2506	2530	drm_warn(&vm->xe->drm, "NOT POSSIBLE");
2507	2531	}
2508	2532
2509		− if (err)
	2533	+ if (IS_ERR(fence))
2510	2534	trace_xe_vma_fail(vma);
2511	2535
2512		− return err;
	2536	+ return fence;
2513	2537	}
2514	2538
2515		−static int __xe_vma_op_execute(struct xe_vm vm, struct xe_vma vma,
2516		− struct xe_vma_op *op)
	2539	+static struct dma_fence *
	2540	+__xe_vma_op_execute(struct xe_vm vm, struct xe_vma vma,
	2541	+ struct xe_vma_op *op)
2517	2542	{
2518		− struct drm_exec exec;
	2543	+ struct dma_fence *fence;
2519	2544	int err;
2520	2545
2521	2546	retry_userptr:
2522		− drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
2523		− drm_exec_until_all_locked(&exec) {
2524		− err = op_execute(&exec, vm, vma, op);
2525		− drm_exec_retry_on_contention(&exec);
2526		− if (err)
2527		− break;
2528		− }
2529		− drm_exec_fini(&exec);
2530		−
2531		− if (err == -EAGAIN) {
	2547	+ fence = op_execute(vm, vma, op);
	2548	+ if (IS_ERR(fence) && PTR_ERR(fence) == -EAGAIN) {
2532	2549	lockdep_assert_held_write(&vm->lock);
2533	2550
2534	2551	if (op->base.op == DRM_GPUVA_OP_REMAP) {
@@ -2545,22 +2562,24 @@ retry_userptr: · 10 unmodified lines
2545	2562	if (!err)
2546	2563	goto retry_userptr;
2547	2564
	2565	+ fence = ERR_PTR(err);
2548	2566	trace_xe_vma_fail(vma);
2549	2567	}
2550	2568	}
2551	2569
2552		− return err;
	2570	+ return fence;
2553	2571	}
2554	2572
2555		−static int xe_vma_op_execute(struct xe_vm vm, struct xe_vma_op op)
	2573	+static struct dma_fence *
	2574	+xe_vma_op_execute(struct xe_vm vm, struct xe_vma_op op)
2556	2575	{
2557		− int ret = 0;
	2576	+ struct dma_fence *fence = ERR_PTR(-ENOMEM);
2558	2577
2559		− lockdep_assert_held_write(&vm->lock);
	2578	+ lockdep_assert_held(&vm->lock);
2560	2579
2561	2580	switch (op->base.op) {
2562	2581	case DRM_GPUVA_OP_MAP:
2563		− ret = __xe_vma_op_execute(vm, op->map.vma, op);
	2582	+ fence = __xe_vma_op_execute(vm, op->map.vma, op);
2564	2583	break;
2565	2584	case DRM_GPUVA_OP_REMAP:
2566	2585	{
@@ -2573,42 +2592,23 @@ static int xe_vma_op_execute(struct xe_vm vm, struct xe_vma_op op) · 6 unmodified lines
2573	2592	else
2574	2593	vma = op->remap.next;
2575	2594
2576		− ret = __xe_vma_op_execute(vm, vma, op);
	2595	+ fence = __xe_vma_op_execute(vm, vma, op);
2577	2596	break;
2578	2597	}
2579	2598	case DRM_GPUVA_OP_UNMAP:
2580		− ret = __xe_vma_op_execute(vm, gpuva_to_vma(op->base.unmap.va),
2581		− op);
	2599	+ fence = __xe_vma_op_execute(vm, gpuva_to_vma(op->base.unmap.va),
	2600	+ op);
2582	2601	break;
2583	2602	case DRM_GPUVA_OP_PREFETCH:
2584		− ret = __xe_vma_op_execute(vm,
2585		− gpuva_to_vma(op->base.prefetch.va),
2586		− op);
	2603	+ fence = __xe_vma_op_execute(vm,
	2604	+ gpuva_to_vma(op->base.prefetch.va),
	2605	+ op);
2587	2606	break;
2588	2607	default:
2589	2608	drm_warn(&vm->xe->drm, "NOT POSSIBLE");
2590	2609	}
2591	2610
2592		− return ret;
2593		−}
2594		−
2595		−static void xe_vma_op_cleanup(struct xe_vm vm, struct xe_vma_op op)
2596		−{
2597		− bool last = op->flags & XE_VMA_OP_LAST;
2598		−
2599		− if (last) {
2600		− while (op->num_syncs--)
2601		− xe_sync_entry_cleanup(&op->syncs[op->num_syncs]);
2602		− kfree(op->syncs);
2603		− if (op->q)
2604		− xe_exec_queue_put(op->q);
2605		− }
2606		− if (!list_empty(&op->link))
2607		− list_del(&op->link);
2608		− if (op->ops)
2609		− drm_gpuva_ops_free(&vm->gpuvm, op->ops);
2610		− if (last)
2611		− xe_vm_put(vm);
	2611	+ return fence;
2612	2612	}
2613	2613
2614	2614	static void xe_vma_op_unwind(struct xe_vm vm, struct xe_vma_op op,
@@ -2687,34 +2687,223 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, · 72 unmodified lines
2687	2687	op->flags & XE_VMA_OP_PREV_COMMITTED,
2688	2688	op->flags & XE_VMA_OP_NEXT_COMMITTED);
2689	2689	}
	2690	+ }
	2691	+}
2690	2692
2691		− drm_gpuva_ops_free(&vm->gpuvm, __ops);
	2693	+static int vma_lock_and_validate(struct drm_exec exec, struct xe_vma vma,
	2694	+ bool validate)
	2695	+{
	2696	+ struct xe_bo *bo = xe_vma_bo(vma);
	2697	+ int err = 0;
	2698	+
	2699	+ if (bo) {
	2700	+ if (!bo->vm)
	2701	+ err = drm_exec_lock_obj(exec, &bo->ttm.base);
	2702	+ if (!err && validate)
	2703	+ err = xe_bo_validate(bo, xe_vma_vm(vma), true);
2692	2704	}
	2705	+
	2706	+ return err;
2693	2707	}
2694	2708
2695		−static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,
2696		− struct list_head *ops_list)
	2709	+static int check_ufence(struct xe_vma *vma)
	2710	+{
	2711	+ if (vma->ufence) {
	2712	+ struct xe_user_fence * const f = vma->ufence;
	2713	+
	2714	+ if (!xe_sync_ufence_get_status(f))
	2715	+ return -EBUSY;
	2716	+
	2717	+ vma->ufence = NULL;
	2718	+ xe_sync_ufence_put(f);
	2719	+ }
	2720	+
	2721	+ return 0;
	2722	+}
	2723	+
	2724	+static int op_lock_and_prep(struct drm_exec exec, struct xe_vm vm,
	2725	+ struct xe_vma_op *op)
	2726	+{
	2727	+ int err = 0;
	2728	+
	2729	+ switch (op->base.op) {
	2730	+ case DRM_GPUVA_OP_MAP:
	2731	+ err = vma_lock_and_validate(exec, op->map.vma,
	2732	+ !xe_vm_in_fault_mode(vm) \|\|
	2733	+ op->map.immediate);
	2734	+ break;
	2735	+ case DRM_GPUVA_OP_REMAP:
	2736	+ err = check_ufence(gpuva_to_vma(op->base.remap.unmap->va));
	2737	+ if (err)
	2738	+ break;
	2739	+
	2740	+ err = vma_lock_and_validate(exec,
	2741	+ gpuva_to_vma(op->base.remap.unmap->va),
	2742	+ false);
	2743	+ if (!err && op->remap.prev)
	2744	+ err = vma_lock_and_validate(exec, op->remap.prev, true);
	2745	+ if (!err && op->remap.next)
	2746	+ err = vma_lock_and_validate(exec, op->remap.next, true);
	2747	+ break;
	2748	+ case DRM_GPUVA_OP_UNMAP:
	2749	+ err = check_ufence(gpuva_to_vma(op->base.unmap.va));
	2750	+ if (err)
	2751	+ break;
	2752	+
	2753	+ err = vma_lock_and_validate(exec,
	2754	+ gpuva_to_vma(op->base.unmap.va),
	2755	+ false);
	2756	+ break;
	2757	+ case DRM_GPUVA_OP_PREFETCH:
	2758	+ {
	2759	+ struct xe_vma *vma = gpuva_to_vma(op->base.prefetch.va);
	2760	+ u32 region = op->prefetch.region;
	2761	+
	2762	+ xe_assert(vm->xe, region <= ARRAY_SIZE(region_to_mem_type));
	2763	+
	2764	+ err = vma_lock_and_validate(exec,
	2765	+ gpuva_to_vma(op->base.prefetch.va),
	2766	+ false);
	2767	+ if (!err && !xe_vma_has_no_bo(vma))
	2768	+ err = xe_bo_migrate(xe_vma_bo(vma),
	2769	+ region_to_mem_type[region]);
	2770	+ break;
	2771	+ }
	2772	+ default:
	2773	+ drm_warn(&vm->xe->drm, "NOT POSSIBLE");
	2774	+ }
	2775	+
	2776	+ return err;
	2777	+}
	2778	+
	2779	+static int vm_bind_ioctl_ops_lock_and_prep(struct drm_exec *exec,
	2780	+ struct xe_vm *vm,
	2781	+ struct xe_vma_ops *vops)
	2782	+{
	2783	+ struct xe_vma_op *op;
	2784	+ int err;
	2785	+
	2786	+ err = drm_exec_lock_obj(exec, xe_vm_obj(vm));
	2787	+ if (err)
	2788	+ return err;
	2789	+
	2790	+ list_for_each_entry(op, &vops->list, link) {
	2791	+ err = op_lock_and_prep(exec, vm, op);
	2792	+ if (err)
	2793	+ return err;
	2794	+ }
	2795	+
	2796	+ return 0;
	2797	+}
	2798	+
	2799	+static struct dma_fence ops_execute(struct xe_vm vm,
	2800	+ struct xe_vma_ops *vops)
2697	2801	{
2698	2802	struct xe_vma_op op, next;
	2803	+ struct dma_fence *fence = NULL;
	2804	+
	2805	+ list_for_each_entry_safe(op, next, &vops->list, link) {
	2806	+ dma_fence_put(fence);
	2807	+ fence = xe_vma_op_execute(vm, op);
	2808	+ if (IS_ERR(fence)) {
	2809	+ drm_warn(&vm->xe->drm, "VM op(%d) failed with %ld",
	2810	+ op->base.op, PTR_ERR(fence));
	2811	+ fence = ERR_PTR(-ENOSPC);
	2812	+ break;
	2813	+ }
	2814	+ }
	2815	+
	2816	+ return fence;
	2817	+}
	2818	+
	2819	+static void vma_add_ufence(struct xe_vma vma, struct xe_user_fence ufence)
	2820	+{
	2821	+ if (vma->ufence)
	2822	+ xe_sync_ufence_put(vma->ufence);
	2823	+ vma->ufence = __xe_sync_ufence_get(ufence);
	2824	+}
	2825	+
	2826	+static void op_add_ufence(struct xe_vm vm, struct xe_vma_op op,
	2827	+ struct xe_user_fence *ufence)
	2828	+{
	2829	+ switch (op->base.op) {
	2830	+ case DRM_GPUVA_OP_MAP:
	2831	+ vma_add_ufence(op->map.vma, ufence);
	2832	+ break;
	2833	+ case DRM_GPUVA_OP_REMAP:
	2834	+ if (op->remap.prev)
	2835	+ vma_add_ufence(op->remap.prev, ufence);
	2836	+ if (op->remap.next)
	2837	+ vma_add_ufence(op->remap.next, ufence);
	2838	+ break;
	2839	+ case DRM_GPUVA_OP_UNMAP:
	2840	+ break;
	2841	+ case DRM_GPUVA_OP_PREFETCH:
	2842	+ vma_add_ufence(gpuva_to_vma(op->base.prefetch.va), ufence);
	2843	+ break;
	2844	+ default:
	2845	+ drm_warn(&vm->xe->drm, "NOT POSSIBLE");
	2846	+ }
	2847	+}
	2848	+
	2849	+static void vm_bind_ioctl_ops_fini(struct xe_vm vm, struct xe_vma_ops vops,
	2850	+ struct dma_fence *fence)
	2851	+{
	2852	+ struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, vops->q);
	2853	+ struct xe_user_fence *ufence;
	2854	+ struct xe_vma_op *op;
	2855	+ int i;
	2856	+
	2857	+ ufence = find_ufence_get(vops->syncs, vops->num_syncs);
	2858	+ list_for_each_entry(op, &vops->list, link) {
	2859	+ if (ufence)
	2860	+ op_add_ufence(vm, op, ufence);
	2861	+
	2862	+ if (op->base.op == DRM_GPUVA_OP_UNMAP)
	2863	+ xe_vma_destroy(gpuva_to_vma(op->base.unmap.va), fence);
	2864	+ else if (op->base.op == DRM_GPUVA_OP_REMAP)
	2865	+ xe_vma_destroy(gpuva_to_vma(op->base.remap.unmap->va),
	2866	+ fence);
	2867	+ }
	2868	+ if (ufence)
	2869	+ xe_sync_ufence_put(ufence);
	2870	+ for (i = 0; i < vops->num_syncs; i++)
	2871	+ xe_sync_entry_signal(vops->syncs + i, fence);
	2872	+ xe_exec_queue_last_fence_set(wait_exec_queue, vm, fence);
	2873	+ dma_fence_put(fence);
	2874	+}
	2875	+
	2876	+static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,
	2877	+ struct xe_vma_ops *vops)
	2878	+{
	2879	+ struct drm_exec exec;
	2880	+ struct dma_fence *fence;
2699	2881	int err;
2700	2882
2701	2883	lockdep_assert_held_write(&vm->lock);
2702	2884
2703		− list_for_each_entry_safe(op, next, ops_list, link) {
2704		− err = xe_vma_op_execute(vm, op);
2705		− if (err) {
2706		− drm_warn(&vm->xe->drm, "VM op(%d) failed with %d",
2707		− op->base.op, err);
2708		− /*
2709		− * FIXME: Killing VM rather than proper error handling
2710		− */
2711		− xe_vm_kill(vm);
2712		− return -ENOSPC;
	2885	+ drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT \|
	2886	+ DRM_EXEC_IGNORE_DUPLICATES, 0);
	2887	+ drm_exec_until_all_locked(&exec) {
	2888	+ err = vm_bind_ioctl_ops_lock_and_prep(&exec, vm, vops);
	2889	+ drm_exec_retry_on_contention(&exec);
	2890	+ if (err)
	2891	+ goto unlock;
	2892	+
	2893	+ fence = ops_execute(vm, vops);
	2894	+ if (IS_ERR(fence)) {
	2895	+ err = PTR_ERR(fence);
	2896	+ /* FIXME: Killing VM rather than proper error handling */
	2897	+ xe_vm_kill(vm, false);
	2898	+ goto unlock;
	2899	+ } else {
	2900	+ vm_bind_ioctl_ops_fini(vm, vops, fence);
2713	2901	}
2714		− xe_vma_op_cleanup(vm, op);
2715	2902	}
2716	2903
2717		− return 0;
	2904	+unlock:
	2905	+ drm_exec_fini(&exec);
	2906	+ return err;
2718	2907	}
2719	2908
2720	2909	#define SUPPORTED_FLAGS \
@@ -2862,6 +3051,58 @@ static int vm_bind_ioctl_signal_fences(struct xe_vm *vm, · 141 unmodified lines
2862	3051	return err;
2863	3052	}
2864	3053
	3054	+static void xe_vma_ops_init(struct xe_vma_ops vops, struct xe_vm vm,
	3055	+ struct xe_exec_queue *q,
	3056	+ struct xe_sync_entry *syncs, u32 num_syncs)
	3057	+{
	3058	+ memset(vops, 0, sizeof(*vops));
	3059	+ INIT_LIST_HEAD(&vops->list);
	3060	+ vops->vm = vm;
	3061	+ vops->q = q;
	3062	+ vops->syncs = syncs;
	3063	+ vops->num_syncs = num_syncs;
	3064	+}
	3065	+
	3066	+static int xe_vm_bind_ioctl_validate_bo(struct xe_device xe, struct xe_bo bo,
	3067	+ u64 addr, u64 range, u64 obj_offset,
	3068	+ u16 pat_index)
	3069	+{
	3070	+ u16 coh_mode;
	3071	+
	3072	+ if (XE_IOCTL_DBG(xe, range > bo->size) \|\|
	3073	+ XE_IOCTL_DBG(xe, obj_offset >
	3074	+ bo->size - range)) {
	3075	+ return -EINVAL;
	3076	+ }
	3077	+
	3078	+ if (bo->flags & XE_BO_FLAG_INTERNAL_64K) {
	3079	+ if (XE_IOCTL_DBG(xe, obj_offset &
	3080	+ XE_64K_PAGE_MASK) \|\|
	3081	+ XE_IOCTL_DBG(xe, addr & XE_64K_PAGE_MASK) \|\|
	3082	+ XE_IOCTL_DBG(xe, range & XE_64K_PAGE_MASK)) {
	3083	+ return -EINVAL;
	3084	+ }
	3085	+ }
	3086	+
	3087	+ coh_mode = xe_pat_index_get_coh_mode(xe, pat_index);
	3088	+ if (bo->cpu_caching) {
	3089	+ if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE &&
	3090	+ bo->cpu_caching == DRM_XE_GEM_CPU_CACHING_WB)) {
	3091	+ return -EINVAL;
	3092	+ }
	3093	+ } else if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE)) {
	3094	+ /*
	3095	+ * Imported dma-buf from a different device should
	3096	+ * require 1way or 2way coherency since we don't know
	3097	+ * how it was mapped on the CPU. Just assume is it
	3098	+ * potentially cached on CPU side.
	3099	+ */
	3100	+ return -EINVAL;
	3101	+ }
	3102	+
	3103	+ return 0;
	3104	+}
	3105	+
2865	3106	int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file)
2866	3107	{
2867	3108	struct xe_device *xe = to_xe_device(dev);
@@ -2875,7 +3116,7 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 7 unmodified lines
2875	3116	u32 num_syncs, num_ufence = 0;
2876	3117	struct xe_sync_entry *syncs = NULL;
2877	3118	struct drm_xe_vm_bind_op *bind_ops;
2878		− LIST_HEAD(ops_list);
	3119	+ struct xe_vma_ops vops;
2879	3120	int err;
2880	3121	int i;
2881	3122
@@ -2945,7 +3186,6 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 63 unmodified lines
2945	3186	u32 obj = bind_ops[i].obj;
2946	3187	u64 obj_offset = bind_ops[i].obj_offset;
2947	3188	u16 pat_index = bind_ops[i].pat_index;
2948		− u16 coh_mode;
2949	3189
2950	3190	if (!obj)
2951	3191	continue;
@@ -2957,40 +3197,10 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 5 unmodified lines
2957	3197	}
2958	3198	bos[i] = gem_to_xe_bo(gem_obj);
2959	3199
2960		− if (XE_IOCTL_DBG(xe, range > bos[i]->size) \|\|
2961		− XE_IOCTL_DBG(xe, obj_offset >
2962		− bos[i]->size - range)) {
2963		− err = -EINVAL;
2964		− goto put_obj;
2965		− }
2966		−
2967		− if (bos[i]->flags & XE_BO_FLAG_INTERNAL_64K) {
2968		− if (XE_IOCTL_DBG(xe, obj_offset &
2969		− XE_64K_PAGE_MASK) \|\|
2970		− XE_IOCTL_DBG(xe, addr & XE_64K_PAGE_MASK) \|\|
2971		− XE_IOCTL_DBG(xe, range & XE_64K_PAGE_MASK)) {
2972		− err = -EINVAL;
2973		− goto put_obj;
2974		− }
2975		− }
2976		−
2977		− coh_mode = xe_pat_index_get_coh_mode(xe, pat_index);
2978		− if (bos[i]->cpu_caching) {
2979		− if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE &&
2980		− bos[i]->cpu_caching == DRM_XE_GEM_CPU_CACHING_WB)) {
2981		− err = -EINVAL;
2982		− goto put_obj;
2983		− }
2984		− } else if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE)) {
2985		− /*
2986		− * Imported dma-buf from a different device should
2987		− * require 1way or 2way coherency since we don't know
2988		− * how it was mapped on the CPU. Just assume is it
2989		− * potentially cached on CPU side.
2990		− */
2991		− err = -EINVAL;
	3200	+ err = xe_vm_bind_ioctl_validate_bo(xe, bos[i], addr, range,
	3201	+ obj_offset, pat_index);
	3202	+ if (err)
2992	3203	goto put_obj;
2993		− }
2994	3204	}
2995	3205
2996	3206	if (args->num_syncs) {
@@ -3026,6 +3236,7 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 29 unmodified lines
3026	3236	goto free_syncs;
3027	3237	}
3028	3238
	3239	+ xe_vma_ops_init(&vops, vm, q, syncs, num_syncs);
3029	3240	for (i = 0; i < args->num_binds; ++i) {
3030	3241	u64 range = bind_ops[i].range;
3031	3242	u64 addr = bind_ops[i].addr;
@@ -3045,42 +3256,25 @@ int xe_vm_bind_ioctl(struct drm_device dev, void data, struct drm_file *file) · 13 unmodified lines
3045	3256	}
3046	3257
3047	3258	err = vm_bind_ioctl_ops_parse(vm, q, ops[i], syncs, num_syncs,
3048		− &ops_list,
3049		− i == args->num_binds - 1);
	3259	+ &vops, i == args->num_binds - 1);
3050	3260	if (err)
3051	3261	goto unwind_ops;
3052	3262	}
3053	3263
3054	3264	/* Nothing to do */
3055		− if (list_empty(&ops_list)) {
	3265	+ if (list_empty(&vops.list)) {
3056	3266	err = -ENODATA;
3057	3267	goto unwind_ops;
3058	3268	}
3059	3269
3060		− xe_vm_get(vm);
3061		− if (q)
3062		− xe_exec_queue_get(q);
3063		−
3064		− err = vm_bind_ioctl_ops_execute(vm, &ops_list);
3065		−
3066		− up_write(&vm->lock);
3067		−
3068		− if (q)
3069		− xe_exec_queue_put(q);
3070		− xe_vm_put(vm);
3071		−
3072		− for (i = 0; bos && i < args->num_binds; ++i)
3073		− xe_bo_put(bos[i]);
3074		−
3075		− kvfree(bos);
3076		− kvfree(ops);
3077		− if (args->num_binds > 1)
3078		− kvfree(bind_ops);
3079		−
3080		− return err;
	3270	+ err = vm_bind_ioctl_ops_execute(vm, &vops);
3081	3271
3082	3272	unwind_ops:
3083		− vm_bind_ioctl_ops_unwind(vm, ops, args->num_binds);
	3273	+ if (err && err != -ENODATA)
	3274	+ vm_bind_ioctl_ops_unwind(vm, ops, args->num_binds);
	3275	+ for (i = args->num_binds - 1; i >= 0; --i)
	3276	+ if (ops[i])
	3277	+ drm_gpuva_ops_free(&vm->gpuvm, ops[i]);
3084	3278	free_syncs:
3085	3279	if (err == -ENODATA)
3086	3280	err = vm_bind_ioctl_signal_fences(vm, q, syncs, num_syncs);
@@ -3201,55 +3395,6 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) · 114 unmodified lines
3201	3395	return 0;
3202	3396	}
3203	3397
3204		−int xe_analyze_vm(struct drm_printer p, struct xe_vm vm, int gt_id)
3205		−{
3206		− struct drm_gpuva *gpuva;
3207		− bool is_vram;
3208		− uint64_t addr;
3209		−
3210		− if (!down_read_trylock(&vm->lock)) {
3211		− drm_printf(p, " Failed to acquire VM lock to dump capture");
3212		− return 0;
3213		− }
3214		− if (vm->pt_root[gt_id]) {
3215		− addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0, XE_PAGE_SIZE);
3216		− is_vram = xe_bo_is_vram(vm->pt_root[gt_id]->bo);
3217		− drm_printf(p, " VM root: A:0x%llx %s\n", addr,
3218		− is_vram ? "VRAM" : "SYS");
3219		− }
3220		−
3221		− drm_gpuvm_for_each_va(gpuva, &vm->gpuvm) {
3222		− struct xe_vma *vma = gpuva_to_vma(gpuva);
3223		− bool is_userptr = xe_vma_is_userptr(vma);
3224		− bool is_null = xe_vma_is_null(vma);
3225		−
3226		− if (is_null) {
3227		− addr = 0;
3228		− } else if (is_userptr) {
3229		− struct sg_table *sg = to_userptr_vma(vma)->userptr.sg;
3230		− struct xe_res_cursor cur;
3231		−
3232		− if (sg) {
3233		− xe_res_first_sg(sg, 0, XE_PAGE_SIZE, &cur);
3234		− addr = xe_res_dma(&cur);
3235		− } else {
3236		− addr = 0;
3237		− }
3238		− } else {
3239		− addr = __xe_bo_addr(xe_vma_bo(vma), 0, XE_PAGE_SIZE);
3240		− is_vram = xe_bo_is_vram(xe_vma_bo(vma));
3241		− }
3242		− drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
3243		− xe_vma_start(vma), xe_vma_end(vma) - 1,
3244		− xe_vma_size(vma),
3245		− addr, is_null ? "NULL" : is_userptr ? "USR" :
3246		− is_vram ? "VRAM" : "SYS");
3247		− }
3248		− up_read(&vm->lock);
3249		−
3250		− return 0;
3251		−}
3252		−
3253	3398	struct xe_vm_snapshot {
3254	3399	unsigned long num_snaps;
3255	3400	struct {
@@ -3339,7 +3484,7 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) · 83 unmodified lines
3339	3484	}
3340	3485
3341	3486	if (bo) {
3342		− dma_resv_lock(bo->ttm.base.resv, NULL);
	3487	+ xe_bo_lock(bo, false);
3343	3488	err = ttm_bo_vmap(&bo->ttm, &src);
3344	3489	if (!err) {
3345	3490	xe_map_memcpy_from(xe_bo_device(bo),
@@ -3348,7 +3493,7 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) · 2 unmodified lines
3348	3493	snap->snap[i].len);
3349	3494	ttm_bo_vunmap(&bo->ttm, &src);
3350	3495	}
3351		− dma_resv_unlock(bo->ttm.base.resv);
	3496	+ xe_bo_unlock(bo);
3352	3497	} else {
3353	3498	void __user userptr = (void __user )(size_t)snap->snap[i].bo_ofs;
3354	3499

drivers/gpu/drm/xe/xe_vm.h+3−2

@@ -6,6 +6,7 @@ · 5 unmodified lines
6	6	#ifndef _XE_VM_H_
7	7	#define _XE_VM_H_
8	8
	9	+#include "xe_assert.h"
9	10	#include "xe_bo_types.h"
10	11	#include "xe_macros.h"
11	12	#include "xe_map.h"
@@ -208,6 +209,8 @@ int __xe_vm_userptr_needs_repin(struct xe_vm *vm); · 196 unmodified lines
208	209	int xe_vm_userptr_check_repin(struct xe_vm *vm);
209	210
210	211	int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker);
	212	+struct dma_fence xe_vma_rebind(struct xe_vm vm, struct xe_vma *vma,
	213	+ u8 tile_mask);
211	214
212	215	int xe_vm_invalidate_vma(struct xe_vma *vma);
213	216
@@ -240,8 +243,6 @@ int xe_vma_userptr_check_repin(struct xe_userptr_vma *uvma); · 26 unmodified lines
240	243
241	244	bool xe_vm_validate_should_retry(struct drm_exec exec, int err, ktime_t end);
242	245
243		−int xe_analyze_vm(struct drm_printer p, struct xe_vm vm, int gt_id);
244		−
245	246	int xe_vm_lock_vma(struct drm_exec exec, struct xe_vma vma);
246	247
247	248	int xe_vm_validate_rebind(struct xe_vm vm, struct drm_exec exec,

drivers/gpu/drm/xe/xe_vm_doc.h+12−12

@@ -25,7 +25,7 @@ · 24 unmodified lines
25	25	* VM bind (create GPU mapping for a BO or userptr)
26	26	* ================================================
27	27	*
28		− * Creates GPU mapings for a BO or userptr within a VM. VM binds uses the same
	28	+ * Creates GPU mappings for a BO or userptr within a VM. VM binds uses the same
29	29	* in / out fence interface (struct drm_xe_sync) as execs which allows users to
30	30	* think of binds and execs as more or less the same operation.
31	31	*
@@ -190,8 +190,8 @@ · 158 unmodified lines
190	190	* Deferred binds in fault mode
191	191	* ----------------------------
192	192	*
193		− * In a VM is in fault mode (TODO: link to fault mode), new bind operations that
194		− * create mappings are by default are deferred to the page fault handler (first
	193	+ * If a VM is in fault mode (TODO: link to fault mode), new bind operations that
	194	+ * create mappings are by default deferred to the page fault handler (first
195	195	* use). This behavior can be overriden by setting the flag
196	196	* DRM_XE_VM_BIND_FLAG_IMMEDIATE which indicates to creating the mapping
197	197	* immediately.
@@ -225,7 +225,7 @@ · 27 unmodified lines
225	225	*
226	226	* A VM in compute mode enables long running workloads and ultra low latency
227	227	* submission (ULLS). ULLS is implemented via a continuously running batch +
228		− * semaphores. This enables to the user to insert jump to new batch commands
	228	+ * semaphores. This enables the user to insert jump to new batch commands
229	229	* into the continuously running batch. In both cases these batches exceed the
230	230	* time a dma fence is allowed to exist for before signaling, as such dma fences
231	231	* are not used when a VM is in compute mode. User fences (TODO: link user fence
@@ -244,7 +244,7 @@ · 12 unmodified lines
244	244	* Once all preempt fences are signaled for a VM the kernel can safely move the
245	245	* memory and kick the rebind worker which resumes all the engines execution.
246	246	*
247		− * A preempt fence, for every engine using the VM, is installed the VM's
	247	+ * A preempt fence, for every engine using the VM, is installed into the VM's
248	248	* dma-resv DMA_RESV_USAGE_PREEMPT_FENCE slot. The same preempt fence, for every
249	249	* engine using the VM, is also installed into the same dma-resv slot of every
250	250	* external BO mapped in the VM.
@@ -314,7 +314,7 @@ · 63 unmodified lines
314	314	* signaling, and memory allocation is usually required to resolve a page
315	315	* fault, but memory allocation is not allowed to gate dma fence signaling. As
316	316	* such, dma fences are not allowed when VM is in fault mode. Because dma-fences
317		− * are not allowed, long running workloads and ULLS are enabled on a faulting
	317	+ * are not allowed, only long running workloads and ULLS are enabled on a faulting
318	318	* VM.
319	319	*
320	320	* Defered VM binds
@@ -399,14 +399,14 @@ · 78 unmodified lines
399	399	* Notice no rebind is issued in the access counter handler as the rebind will
400	400	* be issued on next page fault.
401	401	*
402		− * Cavets with eviction / user pointer invalidation
403		− * ------------------------------------------------
	402	+ * Caveats with eviction / user pointer invalidation
	403	+ * -------------------------------------------------
404	404	*
405	405	* In the case of eviction and user pointer invalidation on a faulting VM, there
406	406	* is no need to issue a rebind rather we just need to blow away the page tables
407	407	* for the VMAs and the page fault handler will rebind the VMAs when they fault.
408		− * The cavet is to update / read the page table structure the VM global lock is
409		− * neeeed. In both the case of eviction and user pointer invalidation locks are
	408	+ * The caveat is to update / read the page table structure the VM global lock is
	409	+ * needed. In both the case of eviction and user pointer invalidation locks are
410	410	* held which make acquiring the VM global lock impossible. To work around this
411	411	* every VMA maintains a list of leaf page table entries which should be written
412	412	* to zero to blow away the VMA's page tables. After writing zero to these
@@ -427,9 +427,9 @@ · 14 unmodified lines
427	427	* VM global lock (vm->lock) - rw semaphore lock. Outer most lock which protects
428	428	* the list of userptrs mapped in the VM, the list of engines using this VM, and
429	429	* the array of external BOs mapped in the VM. When adding or removing any of the
430		− * aforemented state from the VM should acquire this lock in write mode. The VM
	430	+ * aforementioned state from the VM should acquire this lock in write mode. The VM
431	431	* bind path also acquires this lock in write while the exec / compute mode
432		− * rebind worker acquire this lock in read mode.
	432	+ * rebind worker acquires this lock in read mode.
433	433	*
434	434	* VM dma-resv lock (vm->ttm.base.resv->lock) - WW lock. Protects VM dma-resv
435	435	* slots which is shared with any private BO in the VM. Expected to be acquired

drivers/gpu/drm/xe/xe_vm_types.h+24−5

@@ -177,6 +177,13 @@ struct xe_vm { · 176 unmodified lines
177	177	*/
178	178	struct list_head rebind_list;
179	179
	180	+ /**
	181	+ * @destroy_work: worker to destroy VM, needed as a dma_fence signaling
	182	+ * from an irq context can be last put and the destroy needs to be able
	183	+ * to sleep.
	184	+ */
	185	+ struct work_struct destroy_work;
	186	+
180	187	/**
181	188	* @rftree: range fence tree to track updates to page table structure.
182	189	* Used to implement conflict tracking between independent bind engines.
@@ -323,11 +330,6 @@ enum xe_vma_op_flags { · 140 unmodified lines
323	330	struct xe_vma_op {
324	331	/** @base: GPUVA base operation */
325	332	struct drm_gpuva_op base;
326		− /**
327		− * @ops: GPUVA ops, when set call drm_gpuva_ops_free after this
328		− * operations is processed
329		− */
330		− struct drm_gpuva_ops *ops;
331	333	/** @q: exec queue for this operation */
332	334	struct xe_exec_queue *q;
333	335	/**
@@ -341,6 +343,8 @@ struct xe_vma_op { · 7 unmodified lines
341	343	struct list_head link;
342	344	/** @flags: operation flags */
343	345	enum xe_vma_op_flags flags;
	346	+ /** @tile_mask: Tile mask for operation */
	347	+ u8 tile_mask;
344	348
345	349	union {
346	350	/** @map: VMA map operation specific data */
@@ -351,4 +355,19 @@ struct xe_vma_op { · 4 unmodified lines
351	355	struct xe_vma_op_prefetch prefetch;
352	356	};
353	357	};
	358	+
	359	+/** struct xe_vma_ops - VMA operations */
	360	+struct xe_vma_ops {
	361	+ /** @list: list of VMA operations */
	362	+ struct list_head list;
	363	+ /** @vm: VM */
	364	+ struct xe_vm *vm;
	365	+ /** @q: exec queue these operations */
	366	+ struct xe_exec_queue *q;
	367	+ /** @syncs: syncs these operation */
	368	+ struct xe_sync_entry *syncs;
	369	+ /** @num_syncs: number of syncs */
	370	+ u32 num_syncs;
	371	+};
	372	+
354	373	#endif

drivers/gpu/drm/xe/xe_vram.c+368−0

@@ -0,0 +1,368 @@
	1	+// SPDX-License-Identifier: MIT
	2	+/*
	3	+ * Copyright © 2021-2024 Intel Corporation
	4	+ */
	5	+
	6	+#include <linux/pci.h>
	7	+
	8	+#include <drm/drm_managed.h>
	9	+#include <drm/drm_print.h>
	10	+
	11	+#include "regs/xe_bars.h"
	12	+#include "regs/xe_gt_regs.h"
	13	+#include "regs/xe_regs.h"
	14	+#include "xe_assert.h"
	15	+#include "xe_device.h"
	16	+#include "xe_force_wake.h"
	17	+#include "xe_gt_mcr.h"
	18	+#include "xe_gt_sriov_vf.h"
	19	+#include "xe_mmio.h"
	20	+#include "xe_module.h"
	21	+#include "xe_sriov.h"
	22	+#include "xe_vram.h"
	23	+
	24	+#define BAR_SIZE_SHIFT 20
	25	+
	26	+static void
	27	+_resize_bar(struct xe_device *xe, int resno, resource_size_t size)
	28	+{
	29	+ struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
	30	+ int bar_size = pci_rebar_bytes_to_size(size);
	31	+ int ret;
	32	+
	33	+ if (pci_resource_len(pdev, resno))
	34	+ pci_release_resource(pdev, resno);
	35	+
	36	+ ret = pci_resize_resource(pdev, resno, bar_size);
	37	+ if (ret) {
	38	+ drm_info(&xe->drm, "Failed to resize BAR%d to %dM (%pe). Consider enabling 'Resizable BAR' support in your BIOS\n",
	39	+ resno, 1 << bar_size, ERR_PTR(ret));
	40	+ return;
	41	+ }
	42	+
	43	+ drm_info(&xe->drm, "BAR%d resized to %dM\n", resno, 1 << bar_size);
	44	+}
	45	+
	46	+/*
	47	+ * if force_vram_bar_size is set, attempt to set to the requested size
	48	+ * else set to maximum possible size
	49	+ */
	50	+static void resize_vram_bar(struct xe_device *xe)
	51	+{
	52	+ u64 force_vram_bar_size = xe_modparam.force_vram_bar_size;
	53	+ struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
	54	+ struct pci_bus *root = pdev->bus;
	55	+ resource_size_t current_size;
	56	+ resource_size_t rebar_size;
	57	+ struct resource *root_res;
	58	+ u32 bar_size_mask;
	59	+ u32 pci_cmd;
	60	+ int i;
	61	+
	62	+ /* gather some relevant info */
	63	+ current_size = pci_resource_len(pdev, LMEM_BAR);
	64	+ bar_size_mask = pci_rebar_get_possible_sizes(pdev, LMEM_BAR);
	65	+
	66	+ if (!bar_size_mask)
	67	+ return;
	68	+
	69	+ /* set to a specific size? */
	70	+ if (force_vram_bar_size) {
	71	+ u32 bar_size_bit;
	72	+
	73	+ rebar_size = force_vram_bar_size * (resource_size_t)SZ_1M;
	74	+
	75	+ bar_size_bit = bar_size_mask & BIT(pci_rebar_bytes_to_size(rebar_size));
	76	+
	77	+ if (!bar_size_bit) {
	78	+ drm_info(&xe->drm,
	79	+ "Requested size: %lluMiB is not supported by rebar sizes: 0x%x. Leaving default: %lluMiB\n",
	80	+ (u64)rebar_size >> 20, bar_size_mask, (u64)current_size >> 20);
	81	+ return;
	82	+ }
	83	+
	84	+ rebar_size = 1ULL << (__fls(bar_size_bit) + BAR_SIZE_SHIFT);
	85	+
	86	+ if (rebar_size == current_size)
	87	+ return;
	88	+ } else {
	89	+ rebar_size = 1ULL << (__fls(bar_size_mask) + BAR_SIZE_SHIFT);
	90	+
	91	+ /* only resize if larger than current */
	92	+ if (rebar_size <= current_size)
	93	+ return;
	94	+ }
	95	+
	96	+ drm_info(&xe->drm, "Attempting to resize bar from %lluMiB -> %lluMiB\n",
	97	+ (u64)current_size >> 20, (u64)rebar_size >> 20);
	98	+
	99	+ while (root->parent)
	100	+ root = root->parent;
	101	+
	102	+ pci_bus_for_each_resource(root, root_res, i) {
	103	+ if (root_res && root_res->flags & (IORESOURCE_MEM \| IORESOURCE_MEM_64) &&
	104	+ (u64)root_res->start > 0x100000000ul)
	105	+ break;
	106	+ }
	107	+
	108	+ if (!root_res) {
	109	+ drm_info(&xe->drm, "Can't resize VRAM BAR - platform support is missing. Consider enabling 'Resizable BAR' support in your BIOS\n");
	110	+ return;
	111	+ }
	112	+
	113	+ pci_read_config_dword(pdev, PCI_COMMAND, &pci_cmd);
	114	+ pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd & ~PCI_COMMAND_MEMORY);
	115	+
	116	+ _resize_bar(xe, LMEM_BAR, rebar_size);
	117	+
	118	+ pci_assign_unassigned_bus_resources(pdev->bus);
	119	+ pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd);
	120	+}
	121	+
	122	+static bool resource_is_valid(struct pci_dev *pdev, int bar)
	123	+{
	124	+ if (!pci_resource_flags(pdev, bar))
	125	+ return false;
	126	+
	127	+ if (pci_resource_flags(pdev, bar) & IORESOURCE_UNSET)
	128	+ return false;
	129	+
	130	+ if (!pci_resource_len(pdev, bar))
	131	+ return false;
	132	+
	133	+ return true;
	134	+}
	135	+
	136	+static int determine_lmem_bar_size(struct xe_device *xe)
	137	+{
	138	+ struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
	139	+
	140	+ if (!resource_is_valid(pdev, LMEM_BAR)) {
	141	+ drm_err(&xe->drm, "pci resource is not valid\n");
	142	+ return -ENXIO;
	143	+ }
	144	+
	145	+ resize_vram_bar(xe);
	146	+
	147	+ xe->mem.vram.io_start = pci_resource_start(pdev, LMEM_BAR);
	148	+ xe->mem.vram.io_size = pci_resource_len(pdev, LMEM_BAR);
	149	+ if (!xe->mem.vram.io_size)
	150	+ return -EIO;
	151	+
	152	+ /* XXX: Need to change when xe link code is ready */
	153	+ xe->mem.vram.dpa_base = 0;
	154	+
	155	+ /* set up a map to the total memory area. */
	156	+ xe->mem.vram.mapping = ioremap_wc(xe->mem.vram.io_start, xe->mem.vram.io_size);
	157	+
	158	+ return 0;
	159	+}
	160	+
	161	+static inline u64 get_flat_ccs_offset(struct xe_gt *gt, u64 tile_size)
	162	+{
	163	+ struct xe_device *xe = gt_to_xe(gt);
	164	+ u64 offset;
	165	+ u32 reg;
	166	+
	167	+ if (GRAPHICS_VER(xe) >= 20) {
	168	+ u64 ccs_size = tile_size / 512;
	169	+ u64 offset_hi, offset_lo;
	170	+ u32 nodes, num_enabled;
	171	+
	172	+ reg = xe_mmio_read32(gt, MIRROR_FUSE3);
	173	+ nodes = REG_FIELD_GET(XE2_NODE_ENABLE_MASK, reg);
	174	+ num_enabled = hweight32(nodes); /* Number of enabled l3 nodes */
	175	+
	176	+ reg = xe_gt_mcr_unicast_read_any(gt, XE2_FLAT_CCS_BASE_RANGE_LOWER);
	177	+ offset_lo = REG_FIELD_GET(XE2_FLAT_CCS_BASE_LOWER_ADDR_MASK, reg);
	178	+
	179	+ reg = xe_gt_mcr_unicast_read_any(gt, XE2_FLAT_CCS_BASE_RANGE_UPPER);
	180	+ offset_hi = REG_FIELD_GET(XE2_FLAT_CCS_BASE_UPPER_ADDR_MASK, reg);
	181	+
	182	+ offset = offset_hi << 32; /* HW view bits 39:32 */
	183	+ offset \|= offset_lo << 6; /* HW view bits 31:6 */
	184	+ offset = num_enabled; / convert to SW view */
	185	+
	186	+ /* We don't expect any holes */
	187	+ xe_assert_msg(xe, offset == (xe_mmio_read64_2x32(gt, GSMBASE) - ccs_size),
	188	+ "Hole between CCS and GSM.\n");
	189	+ } else {
	190	+ reg = xe_gt_mcr_unicast_read_any(gt, XEHP_FLAT_CCS_BASE_ADDR);
	191	+ offset = (u64)REG_FIELD_GET(XEHP_FLAT_CCS_PTR, reg) * SZ_64K;
	192	+ }
	193	+
	194	+ return offset;
	195	+}
	196	+
	197	+/*
	198	+ * tile_vram_size() - Collect vram size and offset information
	199	+ * @tile: tile to get info for
	200	+ * @vram_size: available vram (size - device reserved portions)
	201	+ * @tile_size: actual vram size
	202	+ * @tile_offset: physical start point in the vram address space
	203	+ *
	204	+ * There are 4 places for size information:
	205	+ * - io size (from pci_resource_len of LMEM bar) (only used for small bar and DG1)
	206	+ * - TILEx size (actual vram size)
	207	+ * - GSMBASE offset (TILEx - "stolen")
	208	+ * - CSSBASE offset (TILEx - CSS space necessary)
	209	+ *
	210	+ * CSSBASE is always a lower/smaller offset then GSMBASE.
	211	+ *
	212	+ * The actual available size of memory is to the CCS or GSM base.
	213	+ * NOTE: multi-tile bases will include the tile offset.
	214	+ *
	215	+ */
	216	+static int tile_vram_size(struct xe_tile tile, u64 vram_size,
	217	+ u64 tile_size, u64 tile_offset)
	218	+{
	219	+ struct xe_device *xe = tile_to_xe(tile);
	220	+ struct xe_gt *gt = tile->primary_gt;
	221	+ u64 offset;
	222	+ int err;
	223	+ u32 reg;
	224	+
	225	+ if (IS_SRIOV_VF(xe)) {
	226	+ struct xe_tile *t;
	227	+ int id;
	228	+
	229	+ offset = 0;
	230	+ for_each_tile(t, xe, id)
	231	+ for_each_if(t->id < tile->id)
	232	+ offset += xe_gt_sriov_vf_lmem(t->primary_gt);
	233	+
	234	+ *tile_size = xe_gt_sriov_vf_lmem(gt);
	235	+ vram_size = tile_size;
	236	+ *tile_offset = offset;
	237	+
	238	+ return 0;
	239	+ }
	240	+
	241	+ err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
	242	+ if (err)
	243	+ return err;
	244	+
	245	+ /* actual size */
	246	+ if (unlikely(xe->info.platform == XE_DG1)) {
	247	+ *tile_size = pci_resource_len(to_pci_dev(xe->drm.dev), LMEM_BAR);
	248	+ *tile_offset = 0;
	249	+ } else {
	250	+ reg = xe_gt_mcr_unicast_read_any(gt, XEHP_TILE_ADDR_RANGE(gt->info.id));
	251	+ tile_size = (u64)REG_FIELD_GET(GENMASK(14, 8), reg) SZ_1G;
	252	+ tile_offset = (u64)REG_FIELD_GET(GENMASK(7, 1), reg) SZ_1G;
	253	+ }
	254	+
	255	+ /* minus device usage */
	256	+ if (xe->info.has_flat_ccs) {
	257	+ offset = get_flat_ccs_offset(gt, *tile_size);
	258	+ } else {
	259	+ offset = xe_mmio_read64_2x32(gt, GSMBASE);
	260	+ }
	261	+
	262	+ /* remove the tile offset so we have just the available size */
	263	+ vram_size = offset - tile_offset;
	264	+
	265	+ return xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
	266	+}
	267	+
	268	+static void vram_fini(void *arg)
	269	+{
	270	+ struct xe_device *xe = arg;
	271	+ struct xe_tile *tile;
	272	+ int id;
	273	+
	274	+ if (xe->mem.vram.mapping)
	275	+ iounmap(xe->mem.vram.mapping);
	276	+
	277	+ xe->mem.vram.mapping = NULL;
	278	+
	279	+ for_each_tile(tile, xe, id)
	280	+ tile->mem.vram.mapping = NULL;
	281	+}
	282	+
	283	+/**
	284	+ * xe_vram_probe() - Probe VRAM configuration
	285	+ * @xe: the &xe_device
	286	+ *
	287	+ * Collect VRAM size and offset information for all tiles.
	288	+ *
	289	+ * Return: 0 on success, error code on failure
	290	+ */
	291	+int xe_vram_probe(struct xe_device *xe)
	292	+{
	293	+ struct xe_tile *tile;
	294	+ resource_size_t io_size;
	295	+ u64 available_size = 0;
	296	+ u64 total_size = 0;
	297	+ u64 tile_offset;
	298	+ u64 tile_size;
	299	+ u64 vram_size;
	300	+ int err;
	301	+ u8 id;
	302	+
	303	+ if (!IS_DGFX(xe))
	304	+ return 0;
	305	+
	306	+ /* Get the size of the root tile's vram for later accessibility comparison */
	307	+ tile = xe_device_get_root_tile(xe);
	308	+ err = tile_vram_size(tile, &vram_size, &tile_size, &tile_offset);
	309	+ if (err)
	310	+ return err;
	311	+
	312	+ err = determine_lmem_bar_size(xe);
	313	+ if (err)
	314	+ return err;
	315	+
	316	+ drm_info(&xe->drm, "VISIBLE VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
	317	+ &xe->mem.vram.io_size);
	318	+
	319	+ io_size = xe->mem.vram.io_size;
	320	+
	321	+ /* tile specific ranges */
	322	+ for_each_tile(tile, xe, id) {
	323	+ err = tile_vram_size(tile, &vram_size, &tile_size, &tile_offset);
	324	+ if (err)
	325	+ return err;
	326	+
	327	+ tile->mem.vram.actual_physical_size = tile_size;
	328	+ tile->mem.vram.io_start = xe->mem.vram.io_start + tile_offset;
	329	+ tile->mem.vram.io_size = min_t(u64, vram_size, io_size);
	330	+
	331	+ if (!tile->mem.vram.io_size) {
	332	+ drm_err(&xe->drm, "Tile without any CPU visible VRAM. Aborting.\n");
	333	+ return -ENODEV;
	334	+ }
	335	+
	336	+ tile->mem.vram.dpa_base = xe->mem.vram.dpa_base + tile_offset;
	337	+ tile->mem.vram.usable_size = vram_size;
	338	+ tile->mem.vram.mapping = xe->mem.vram.mapping + tile_offset;
	339	+
	340	+ if (tile->mem.vram.io_size < tile->mem.vram.usable_size)
	341	+ drm_info(&xe->drm, "Small BAR device\n");
	342	+ drm_info(&xe->drm, "VRAM[%u, %u]: Actual physical size %pa, usable size exclude stolen %pa, CPU accessible size %pa\n", id,
	343	+ tile->id, &tile->mem.vram.actual_physical_size, &tile->mem.vram.usable_size, &tile->mem.vram.io_size);
	344	+ drm_info(&xe->drm, "VRAM[%u, %u]: DPA range: [%pa-%llx], io range: [%pa-%llx]\n", id, tile->id,
	345	+ &tile->mem.vram.dpa_base, tile->mem.vram.dpa_base + (u64)tile->mem.vram.actual_physical_size,
	346	+ &tile->mem.vram.io_start, tile->mem.vram.io_start + (u64)tile->mem.vram.io_size);
	347	+
	348	+ /* calculate total size using tile size to get the correct HW sizing */
	349	+ total_size += tile_size;
	350	+ available_size += vram_size;
	351	+
	352	+ if (total_size > xe->mem.vram.io_size) {
	353	+ drm_info(&xe->drm, "VRAM: %pa is larger than resource %pa\n",
	354	+ &total_size, &xe->mem.vram.io_size);
	355	+ }
	356	+
	357	+ io_size -= min_t(u64, tile_size, io_size);
	358	+ }
	359	+
	360	+ xe->mem.vram.actual_physical_size = total_size;
	361	+
	362	+ drm_info(&xe->drm, "Total VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
	363	+ &xe->mem.vram.actual_physical_size);
	364	+ drm_info(&xe->drm, "Available VRAM: %pa, %pa\n", &xe->mem.vram.io_start,
	365	+ &available_size);
	366	+
	367	+ return devm_add_action_or_reset(xe->drm.dev, vram_fini, xe);
	368	+}

drivers/gpu/drm/xe/xe_vram.h+13−0

@@ -0,0 +1,13 @@
	1	+/* SPDX-License-Identifier: MIT */
	2	+/*
	3	+ * Copyright © 2024 Intel Corporation
	4	+ */
	5	+
	6	+#ifndef _XE_VRAM_H_
	7	+#define _XE_VRAM_H_
	8	+
	9	+struct xe_device;
	10	+
	11	+int xe_vram_probe(struct xe_device *xe);
	12	+
	13	+#endif

drivers/gpu/drm/xe/xe_vram_freq.c+2−2

@@ -87,7 +87,7 @@ static const struct attribute_group freq_group_attrs = { · 86 unmodified lines
87	87	.attrs = freq_attrs,
88	88	};
89	89
90		−static void vram_freq_sysfs_fini(struct drm_device drm, void arg)
	90	+static void vram_freq_sysfs_fini(void *arg)
91	91	{
92	92	struct kobject *kobj = arg;
93	93
@@ -122,5 +122,5 @@ int xe_vram_freq_sysfs_init(struct xe_tile *tile) · 28 unmodified lines
122	122	return err;
123	123	}
124	124
125		− return drmm_add_action_or_reset(&xe->drm, vram_freq_sysfs_fini, kobj);
	125	+ return devm_add_action_or_reset(xe->drm.dev, vram_freq_sysfs_fini, kobj);
126	126	}

drivers/gpu/drm/xe/xe_wa.c+34−5

@@ -238,11 +238,11 @@ static const struct xe_rtp_entry_sr gt_was[] = { · 237 unmodified lines
238	238	},
239	239	{ XE_RTP_NAME("14020316580"),
240	240	XE_RTP_RULES(MEDIA_VERSION(1301)),
241		− XE_RTP_ACTIONS(CLR(PG_ENABLE,
242		− VD0_HCP_POWERGATE_ENABLE \|
243		− VD0_MFXVDENC_POWERGATE_ENABLE \|
244		− VD2_HCP_POWERGATE_ENABLE \|
245		− VD2_MFXVDENC_POWERGATE_ENABLE)),
	241	+ XE_RTP_ACTIONS(CLR(POWERGATE_ENABLE,
	242	+ VDN_HCP_POWERGATE_ENABLE(0) \|
	243	+ VDN_MFXVDENC_POWERGATE_ENABLE(0) \|
	244	+ VDN_HCP_POWERGATE_ENABLE(2) \|
	245	+ VDN_MFXVDENC_POWERGATE_ENABLE(2))),
246	246	},
247	247	{ XE_RTP_NAME("14019449301"),
248	248	XE_RTP_RULES(MEDIA_VERSION(1301), ENGINE_CLASS(VIDEO_DECODE)),
@@ -533,6 +533,10 @@ static const struct xe_rtp_entry_sr engine_was[] = { · 284 unmodified lines
533	533	FUNC(xe_rtp_match_first_render_or_compute)),
534	534	XE_RTP_ACTIONS(SET(LSC_CHICKEN_BIT_0, WR_REQ_CHAINING_DIS))
535	535	},
	536	+ { XE_RTP_NAME("14021402888"),
	537	+ XE_RTP_RULES(GRAPHICS_VERSION(2001), ENGINE_CLASS(RENDER)),
	538	+ XE_RTP_ACTIONS(SET(HALF_SLICE_CHICKEN7, CLEAR_OPTIMIZATION_DISABLE))
	539	+ },
536	540
537	541	/* Xe2_HPM */
538	542
@@ -668,6 +672,15 @@ static const struct xe_rtp_entry_sr lrc_was[] = { · 129 unmodified lines
668	672	XE_RTP_RULES(GRAPHICS_VERSION_RANGE(2001, 2004), ENGINE_CLASS(RENDER)),
669	673	XE_RTP_ACTIONS(SET(COMMON_SLICE_CHICKEN1, DISABLE_BOTTOM_CLIP_RECTANGLE_TEST))
670	674	},
	675	+ { XE_RTP_NAME("14021567978"),
	676	+ XE_RTP_RULES(GRAPHICS_VERSION_RANGE(2001, XE_RTP_END_VERSION_UNDEFINED),
	677	+ ENGINE_CLASS(RENDER)),
	678	+ XE_RTP_ACTIONS(SET(CHICKEN_RASTER_2, TBIMR_FAST_CLIP))
	679	+ },
	680	+ { XE_RTP_NAME("14020756599"),
	681	+ XE_RTP_RULES(GRAPHICS_VERSION(2004), ENGINE_CLASS(RENDER)),
	682	+ XE_RTP_ACTIONS(SET(WM_CHICKEN3, HIZ_PLANE_COMPRESSION_DIS))
	683	+ },
671	684
672	685	/* Xe2_HPG */
673	686	{ XE_RTP_NAME("15010599737"),
@@ -682,6 +695,22 @@ static const struct xe_rtp_entry_sr lrc_was[] = { · 8 unmodified lines
682	695	XE_RTP_RULES(GRAPHICS_VERSION(2001), ENGINE_CLASS(RENDER)),
683	696	XE_RTP_ACTIONS(SET(WM_CHICKEN3, HIZ_PLANE_COMPRESSION_DIS))
684	697	},
	698	+ { XE_RTP_NAME("14021490052"),
	699	+ XE_RTP_RULES(GRAPHICS_VERSION(2001), ENGINE_CLASS(RENDER)),
	700	+ XE_RTP_ACTIONS(SET(FF_MODE,
	701	+ DIS_MESH_PARTIAL_AUTOSTRIP \|
	702	+ DIS_MESH_AUTOSTRIP),
	703	+ SET(VFLSKPD,
	704	+ DIS_PARTIAL_AUTOSTRIP \|
	705	+ DIS_AUTOSTRIP))
	706	+ },
	707	+
	708	+ /* Xe2_LPM */
	709	+
	710	+ { XE_RTP_NAME("14020756599"),
	711	+ XE_RTP_RULES(ENGINE_CLASS(RENDER), FUNC(xe_rtp_match_when_media2000)),
	712	+ XE_RTP_ACTIONS(SET(WM_CHICKEN3, HIZ_PLANE_COMPRESSION_DIS))
	713	+ },
685	714
686	715	{}
687	716	};

include/uapi/drm/xe_drm.h+2−0

@@ -508,6 +508,7 @@ struct drm_xe_query_gt_list { · 507 unmodified lines
508	508	* containing the following in mask:
509	509	* ``DSS_COMPUTE ff ff ff ff 00 00 00 00``
510	510	* means 32 DSS are available for compute.
	511	+ * - %DRM_XE_TOPO_L3_BANK - To query the mask of enabled L3 banks
511	512	* - %DRM_XE_TOPO_EU_PER_DSS - To query the mask of Execution Units (EU)
512	513	* available per Dual Sub Slices (DSS). For example a query response
513	514	* containing the following in mask:
@@ -520,6 +521,7 @@ struct drm_xe_query_topology_mask { · 6 unmodified lines
520	521
521	522	#define DRM_XE_TOPO_DSS_GEOMETRY 1
522	523	#define DRM_XE_TOPO_DSS_COMPUTE 2
	524	+#define DRM_XE_TOPO_L3_BANK 3
523	525	#define DRM_XE_TOPO_EU_PER_DSS 4
524	526	/** @type: type of mask */
525	527	__u16 type;