History

Merge tag 'drm-xe-next-2025-04-17' of https://gitlab.freedesktop.org/drm/xe/kernel into drm-next

Core Changes: Fix drm_gpusvm kernel-doc (Lucas) Driver Changes: - Release guc ids before cancelling work (Tejas) - Remove a duplicated pc_start_call (Rodrigo) - Fix an incorrect assert in previous userptr fixes (Thomas) - Remove gen11 assertions and prefixes (Lucas) - Drop sentinels from arg to xe_rtp_process_to_src (Lucas) - Temporarily disable D3Cold on BMG (Rodrigo) - Fix MOCS debugfs LNCF readout (Tvrtko) - Some ring flush cleanups (Tvrtko) - Use unsigned int for alignment in fb pinning code (Tvrtko) - Retry and wait longer for GuC PC start (Rodrigo) - Recognize 3DSTATE_COARSE_PIXEL in LRC dumps (Matt Roper) - Remove reduntant check in xe_vm_create_ioctl() (Xin) - A bunch of SRIOV updates (Michal) - Add stats for SVM page-faults (Francois) - Fix an UAF (Harish) - Expose fan speed (Raag) - Fix exporting xe buffer objects multiple times (Tomasz) - Apply a workaround (Vinay) - Simplify pinned bo iteration (Thomas) - Remove an incorrect "static" keywork (Lucas) - Add support for separate firmware files on each GT (Lucas) - Survivability handling fixes (Lucas) - Allow to inject error in early probe (Lucas) - Fix unmet direct dependencies warning (Yue Haibing) - More error injection during probe (Francois) - Coding style fix (Maarten) - Additional stats support (Riana) - Add fault injection for xe_oa_alloc_regs (Nakshrtra) - Add a BMG PCI ID (Matt Roper) - Some SVM fixes and preliminary SVM multi-device work (Thomas) - Switch the migrate code from drm managed to dev managed (Aradhya) - Fix an out-of-bounds shift when invalidating TLB (Thomas) - Ensure fixed_slice_mode gets set after ccs_mode change (Niranjana) - Use local fence in error path of xe_migrate_clear (Matthew Brost) - More Workarounds (Julia) - Define sysfs_ops on all directories (Tejas) - Set power state to D3Cold during s2idle/s3 (Badal) - Devcoredump output fix (John) - Avoid plain 64-bit division (Arnd Bergmann) - Reword a debug message (John) - Don't print a hwconfig error message when forcing execlists (Stuart) - Restore an error code to avoid a smatch warning (Rodrigo) - Invalidate L3 read-only cachelines for geometry streams too (Kenneth) - Make PPHWSP size explicit in xe_gt_lrc_size() (Gustavo) - Add GT frequency events (Vinay) - Fix xe_pt_stage_bind_walk kerneldoc (Thomas) - Add a workaround (Aradhya) - Rework pinned save/restore (Matthew Auld, Matthew Brost) - Allow non-contig VRAM kernel BO (Matthew Auld) - Support non-contig VRAM provisioning for SRIOV (Matthew Auld) - Allow scratch-pages for unmapped parts of page-faulting VMs. (Oak) - Ensure XE_BO_FLAG_CPU_ADDR_MIRROR had a unique value (Matt Roper) - Fix taking an invalid lock on wedge (Lucas) - Configs and documentation for survivability mode (Riana) - Remove an unused macro (Shuicheng) - Work around a page-fault full error (Matt Brost) - Enable a SRIOV workaround (John) - Bump the recommended GuC version (John) - Allow to drop VRAM resizing (Lucas) - Don't expose privileged debugfs files if VF (Michal) - Don't show GGTT/LMEM debugfs files under media GT (Michal) - Adjust ring-buffer emission for maximum possible size (Tvrtko) - Fix notifier vs folio lock deadlock (Matthew Auld) - Stop relying on placement for dma-buf unmap Matthew Auld) Signed-off-by: Dave Airlie <airlied@redhat.com>

Dave Airlie committed 1 year ago

Commit d2b9e2f2 parents 55df7c0 d75588784 files · +2131 −820Browse files

Unified · Split

84 files
  1. Documentation/ABI/testing/sysfs-driver-intel-xe-hwmon +24 −0
  2. Documentation/gpu/xe/index.rst +1 −0
  3. Documentation/gpu/xe/xe_configfs.rst +10 −0
  4. Documentation/gpu/xe/xe_pcode.rst +7 −0
  5. drivers/gpu/drm/xe/Kconfig +14 −2
  6. drivers/gpu/drm/xe/Makefile +2 −1
  7. drivers/gpu/drm/xe/abi/guc_actions_abi.h +1 −0
  8. drivers/gpu/drm/xe/abi/guc_klvs_abi.h +1 −0
  9. drivers/gpu/drm/xe/display/intel_fbdev_fb.c +2 −2
  10. drivers/gpu/drm/xe/display/xe_plane_initial.c +1 −1
  11. drivers/gpu/drm/xe/instructions/xe_alu_commands.h +79 −0
  12. drivers/gpu/drm/xe/instructions/xe_gfxpipe_commands.h +1 −0
  13. drivers/gpu/drm/xe/instructions/xe_mi_commands.h +5 −0
  14. drivers/gpu/drm/xe/regs/xe_engine_regs.h +4 −0
  15. drivers/gpu/drm/xe/regs/xe_gt_regs.h +12 −1
  16. drivers/gpu/drm/xe/regs/xe_pcode_regs.h +3 −0
  17. drivers/gpu/drm/xe/tests/xe_bo.c +2 −2
  18. drivers/gpu/drm/xe/tests/xe_migrate.c +3 −6
  19. drivers/gpu/drm/xe/xe_bo.c +239 −164
  20. drivers/gpu/drm/xe/xe_bo.h +14 −9
  21. drivers/gpu/drm/xe/xe_bo_evict.c +179 −130
  22. drivers/gpu/drm/xe/xe_bo_evict.h +5 −2
  23. drivers/gpu/drm/xe/xe_bo_types.h +2 −0
  24. drivers/gpu/drm/xe/xe_configfs.c +250 −0
  25. drivers/gpu/drm/xe/xe_configfs.h +24 −0
  26. drivers/gpu/drm/xe/xe_device.c +19 −10
  27. drivers/gpu/drm/xe/xe_device_types.h +24 −8
  28. drivers/gpu/drm/xe/xe_dma_buf.c +1 −4
  29. drivers/gpu/drm/xe/xe_force_wake.c +0 −6
  30. drivers/gpu/drm/xe/xe_ggtt.c +1 −1
  31. drivers/gpu/drm/xe/xe_gt.c +63 −21
  32. drivers/gpu/drm/xe/xe_gt_debugfs.c +22 −8
  33. drivers/gpu/drm/xe/xe_gt_mcr.c +2 −1
  34. drivers/gpu/drm/xe/xe_gt_pagefault.c +10 −3
  35. drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c +14 −6
  36. drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c +49 −17
  37. drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c +0 −6
  38. drivers/gpu/drm/xe/xe_gt_stats.c +1 −0
  39. drivers/gpu/drm/xe/xe_gt_stats_types.h +1 −0
  40. drivers/gpu/drm/xe/xe_guc.c +2 −1
  41. drivers/gpu/drm/xe/xe_guc_ads.c +54 −33
  42. drivers/gpu/drm/xe/xe_guc_ct.c +3 −2
  43. drivers/gpu/drm/xe/xe_guc_debugfs.c +83 −65
  44. drivers/gpu/drm/xe/xe_guc_engine_activity.c +172 −28
  45. drivers/gpu/drm/xe/xe_guc_engine_activity.h +5 −2
  46. drivers/gpu/drm/xe/xe_guc_engine_activity_types.h +11 −1
  47. drivers/gpu/drm/xe/xe_guc_log.c +2 −1
  48. drivers/gpu/drm/xe/xe_guc_pc.c +17 −1
  49. drivers/gpu/drm/xe/xe_guc_pc.h +1 −0
  50. drivers/gpu/drm/xe/xe_guc_submit.c +9 −0
  51. drivers/gpu/drm/xe/xe_guc_types.h +5 −0
  52. drivers/gpu/drm/xe/xe_hmm.c +0 −24
  53. drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c +1 −0
  54. drivers/gpu/drm/xe/xe_hwmon.c +124 −1
  55. drivers/gpu/drm/xe/xe_lmtt.c +1 −1
  56. drivers/gpu/drm/xe/xe_lrc.c +18 −10
  57. drivers/gpu/drm/xe/xe_memirq.c +1 −1
  58. drivers/gpu/drm/xe/xe_migrate.c +15 −9
  59. drivers/gpu/drm/xe/xe_mmio.c +7 −4
  60. drivers/gpu/drm/xe/xe_module.c +7 −2
  61. drivers/gpu/drm/xe/xe_oa.c +1 −0
  62. drivers/gpu/drm/xe/xe_pci.c +19 −10
  63. drivers/gpu/drm/xe/xe_pci_sriov.c +20 −0
  64. drivers/gpu/drm/xe/xe_pcode.c +2 −0
  65. drivers/gpu/drm/xe/xe_pcode_api.h +3 −0
  66. drivers/gpu/drm/xe/xe_pm.c +4 −4
  67. drivers/gpu/drm/xe/xe_pmu.c +67 −10
  68. drivers/gpu/drm/xe/xe_pt.c +140 −98
  69. drivers/gpu/drm/xe/xe_query.c +1 −1
  70. drivers/gpu/drm/xe/xe_reg_sr.c +3 −0
  71. drivers/gpu/drm/xe/xe_ring_ops_types.h +1 −1
  72. drivers/gpu/drm/xe/xe_rtp.c +0 −3
  73. drivers/gpu/drm/xe/xe_sa.c +2 −1
  74. drivers/gpu/drm/xe/xe_survivability_mode.c +50 −19
  75. drivers/gpu/drm/xe/xe_survivability_mode.h +1 −0
  76. drivers/gpu/drm/xe/xe_svm.c +21 −2
  77. drivers/gpu/drm/xe/xe_svm.h +57 −23
  78. drivers/gpu/drm/xe/xe_uc_fw.c +55 −39
  79. drivers/gpu/drm/xe/xe_vm.c +29 −10
  80. drivers/gpu/drm/xe/xe_vm_types.h +2 −0
  81. drivers/gpu/drm/xe/xe_vram.c +4 −1
  82. drivers/gpu/drm/xe/xe_wa.c +12 −0
  83. drivers/gpu/drm/xe/xe_wa_oob.rules +2 −0
  84. include/uapi/drm/xe_drm.h +5 −1
Documentation/ABI/testing/sysfs-driver-intel-xe-hwmon+24−0
@@ -124,3 +124,27 @@ Contact: intel-xe@lists.freedesktop.org · 123 unmodified lines
124124 Description: RO. VRAM temperature in millidegree Celsius.
125125
126126 Only supported for particular Intel Xe graphics platforms.
127+
128+What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/fan1_input
129+Date: March 2025
130+KernelVersion: 6.14
131+Contact: intel-xe@lists.freedesktop.org
132+Description: RO. Fan 1 speed in RPM.
133+
134+ Only supported for particular Intel Xe graphics platforms.
135+
136+What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/fan2_input
137+Date: March 2025
138+KernelVersion: 6.14
139+Contact: intel-xe@lists.freedesktop.org
140+Description: RO. Fan 2 speed in RPM.
141+
142+ Only supported for particular Intel Xe graphics platforms.
143+
144+What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/fan3_input
145+Date: March 2025
146+KernelVersion: 6.14
147+Contact: intel-xe@lists.freedesktop.org
148+Description: RO. Fan 3 speed in RPM.
149+
150+ Only supported for particular Intel Xe graphics platforms.
Documentation/gpu/xe/index.rst+1−0
@@ -25,3 +25,4 @@ DG2, etc is provided to prototype the driver. · 24 unmodified lines
2525 xe_debugging
2626 xe_devcoredump
2727 xe-drm-usage-stats.rst
28+ xe_configfs
Documentation/gpu/xe/xe_configfs.rst+10−0
@@ -0,0 +1,10 @@
1+.. SPDX-License-Identifier: GPL-2.0+
2+
3+.. _xe_configfs:
4+
5+============
6+Xe Configfs
7+============
8+
9+.. kernel-doc:: drivers/gpu/drm/xe/xe_configfs.c
10+ :doc: Xe Configfs
Documentation/gpu/xe/xe_pcode.rst+7−0
@@ -12,3 +12,10 @@ Internal API · 11 unmodified lines
1212
1313 .. kernel-doc:: drivers/gpu/drm/xe/xe_pcode.c
1414 :internal:
15+
16+==================
17+Boot Survivability
18+==================
19+
20+.. kernel-doc:: drivers/gpu/drm/xe/xe_survivability_mode.c
21+ :doc: Xe Boot Survivability
drivers/gpu/drm/xe/Kconfig+14−2
@@ -39,7 +39,6 @@ config DRM_XE · 38 unmodified lines
3939 select DRM_TTM_HELPER
4040 select DRM_EXEC
4141 select DRM_GPUVM
42− select DRM_GPUSVM if !UML && DEVICE_PRIVATE
4342 select DRM_SCHED
4443 select MMU_NOTIFIER
4544 select WANT_DEV_COREDUMP
@@ -74,9 +73,22 @@ config DRM_XE_DP_TUNNEL · 28 unmodified lines
7473
7574 If in doubt say "Y".
7675
76+config DRM_XE_GPUSVM
77+ bool "Enable CPU to GPU address mirroring"
78+ depends on DRM_XE
79+ depends on !UML
80+ depends on DEVICE_PRIVATE
81+ default y
82+ select DRM_GPUSVM
83+ help
84+ Enable this option if you want support for CPU to GPU address
85+ mirroring.
86+
87+ If in doubut say "Y".
88+
7789 config DRM_XE_DEVMEM_MIRROR
7890 bool "Enable device memory mirror"
79− depends on DRM_XE
91+ depends on DRM_XE_GPUSVM
8092 select GET_FREE_REGION
8193 default y
8294 help
drivers/gpu/drm/xe/Makefile+2−1
@@ -125,12 +125,13 @@ xe-y += xe_bb.o \ · 124 unmodified lines
125125 xe_wopcm.o
126126
127127 xe-$(CONFIG_HMM_MIRROR) += xe_hmm.o
128−xe-$(CONFIG_DRM_GPUSVM) += xe_svm.o
128+xe-$(CONFIG_DRM_XE_GPUSVM) += xe_svm.o
129129
130130 # graphics hardware monitoring (HWMON) support
131131 xe-$(CONFIG_HWMON) += xe_hwmon.o
132132
133133 xe-$(CONFIG_PERF_EVENTS) += xe_pmu.o
134+xe-$(CONFIG_CONFIGFS_FS) += xe_configfs.o
134135
135136 # graphics virtualization (SR-IOV) support
136137 xe-y += \
drivers/gpu/drm/xe/abi/guc_actions_abi.h+1−0
@@ -141,6 +141,7 @@ enum xe_guc_action { · 140 unmodified lines
141141 XE_GUC_ACTION_CLIENT_SOFT_RESET = 0x5507,
142142 XE_GUC_ACTION_SET_ENG_UTIL_BUFF = 0x550A,
143143 XE_GUC_ACTION_SET_DEVICE_ENGINE_ACTIVITY_BUFFER = 0x550C,
144+ XE_GUC_ACTION_SET_FUNCTION_ENGINE_ACTIVITY_BUFFER = 0x550D,
144145 XE_GUC_ACTION_NOTIFY_MEMORY_CAT_ERROR = 0x6000,
145146 XE_GUC_ACTION_REPORT_PAGE_FAULT_REQ_DESC = 0x6002,
146147 XE_GUC_ACTION_PAGE_FAULT_RES_DESC = 0x6003,
drivers/gpu/drm/xe/abi/guc_klvs_abi.h+1−0
@@ -367,6 +367,7 @@ enum xe_guc_klv_ids { · 366 unmodified lines
367367 GUC_WA_KLV_NP_RD_WRITE_TO_CLEAR_RCSM_AT_CGP_LATE_RESTORE = 0x9008,
368368 GUC_WORKAROUND_KLV_ID_BACK_TO_BACK_RCS_ENGINE_RESET = 0x9009,
369369 GUC_WA_KLV_WAKE_POWER_DOMAINS_FOR_OUTBOUND_MMIO = 0x900a,
370+ GUC_WA_KLV_RESET_BB_STACK_PTR_ON_VF_SWITCH = 0x900b,
370371 };
371372
372373 #endif
drivers/gpu/drm/xe/display/intel_fbdev_fb.c+2−2
@@ -45,7 +45,7 @@ struct intel_framebuffer *intel_fbdev_fb_alloc(struct drm_fb_helper *helper, · 44 unmodified lines
4545 NULL, size,
4646 ttm_bo_type_kernel, XE_BO_FLAG_SCANOUT |
4747 XE_BO_FLAG_STOLEN |
48− XE_BO_FLAG_GGTT | XE_BO_FLAG_PINNED);
48+ XE_BO_FLAG_GGTT);
4949 if (!IS_ERR(obj))
5050 drm_info(&xe->drm, "Allocated fbdev into stolen\n");
5151 else
@@ -56,7 +56,7 @@ struct intel_framebuffer *intel_fbdev_fb_alloc(struct drm_fb_helper *helper, · 4 unmodified lines
5656 obj = xe_bo_create_pin_map(xe, xe_device_get_root_tile(xe), NULL, size,
5757 ttm_bo_type_kernel, XE_BO_FLAG_SCANOUT |
5858 XE_BO_FLAG_VRAM_IF_DGFX(xe_device_get_root_tile(xe)) |
59− XE_BO_FLAG_GGTT | XE_BO_FLAG_PINNED);
59+ XE_BO_FLAG_GGTT);
6060 }
6161
6262 if (IS_ERR(obj)) {
drivers/gpu/drm/xe/display/xe_plane_initial.c+1−1
@@ -83,7 +83,7 @@ initial_plane_bo(struct xe_device *xe, · 82 unmodified lines
8383 if (plane_config->size == 0)
8484 return NULL;
8585
86− flags = XE_BO_FLAG_PINNED | XE_BO_FLAG_SCANOUT | XE_BO_FLAG_GGTT;
86+ flags = XE_BO_FLAG_SCANOUT | XE_BO_FLAG_GGTT;
8787
8888 base = round_down(plane_config->base, page_size);
8989 if (IS_DGFX(xe)) {
drivers/gpu/drm/xe/instructions/xe_alu_commands.h+79−0
@@ -0,0 +1,79 @@
1+/* SPDX-License-Identifier: MIT */
2+/*
3+ * Copyright © 2025 Intel Corporation
4+ */
5+
6+#ifndef _XE_ALU_COMMANDS_H_
7+#define _XE_ALU_COMMANDS_H_
8+
9+#include "instructions/xe_instr_defs.h"
10+
11+/* Instruction Opcodes */
12+#define CS_ALU_OPCODE_NOOP 0x000
13+#define CS_ALU_OPCODE_FENCE_RD 0x001
14+#define CS_ALU_OPCODE_FENCE_WR 0x002
15+#define CS_ALU_OPCODE_LOAD 0x080
16+#define CS_ALU_OPCODE_LOADINV 0x480
17+#define CS_ALU_OPCODE_LOAD0 0x081
18+#define CS_ALU_OPCODE_LOAD1 0x481
19+#define CS_ALU_OPCODE_LOADIND 0x082
20+#define CS_ALU_OPCODE_ADD 0x100
21+#define CS_ALU_OPCODE_SUB 0x101
22+#define CS_ALU_OPCODE_AND 0x102
23+#define CS_ALU_OPCODE_OR 0x103
24+#define CS_ALU_OPCODE_XOR 0x104
25+#define CS_ALU_OPCODE_SHL 0x105
26+#define CS_ALU_OPCODE_SHR 0x106
27+#define CS_ALU_OPCODE_SAR 0x107
28+#define CS_ALU_OPCODE_STORE 0x180
29+#define CS_ALU_OPCODE_STOREINV 0x580
30+#define CS_ALU_OPCODE_STOREIND 0x181
31+
32+/* Instruction Operands */
33+#define CS_ALU_OPERAND_REG(n) REG_FIELD_PREP(GENMASK(3, 0), (n))
34+#define CS_ALU_OPERAND_REG0 0x0
35+#define CS_ALU_OPERAND_REG1 0x1
36+#define CS_ALU_OPERAND_REG2 0x2
37+#define CS_ALU_OPERAND_REG3 0x3
38+#define CS_ALU_OPERAND_REG4 0x4
39+#define CS_ALU_OPERAND_REG5 0x5
40+#define CS_ALU_OPERAND_REG6 0x6
41+#define CS_ALU_OPERAND_REG7 0x7
42+#define CS_ALU_OPERAND_REG8 0x8
43+#define CS_ALU_OPERAND_REG9 0x9
44+#define CS_ALU_OPERAND_REG10 0xa
45+#define CS_ALU_OPERAND_REG11 0xb
46+#define CS_ALU_OPERAND_REG12 0xc
47+#define CS_ALU_OPERAND_REG13 0xd
48+#define CS_ALU_OPERAND_REG14 0xe
49+#define CS_ALU_OPERAND_REG15 0xf
50+#define CS_ALU_OPERAND_SRCA 0x20
51+#define CS_ALU_OPERAND_SRCB 0x21
52+#define CS_ALU_OPERAND_ACCU 0x31
53+#define CS_ALU_OPERAND_ZF 0x32
54+#define CS_ALU_OPERAND_CF 0x33
55+#define CS_ALU_OPERAND_NA 0 /* N/A operand */
56+
57+/* Command Streamer ALU Instructions */
58+#define CS_ALU_INSTR(opcode, op1, op2) (REG_FIELD_PREP(GENMASK(31, 20), (opcode)) | \
59+ REG_FIELD_PREP(GENMASK(19, 10), (op1)) | \
60+ REG_FIELD_PREP(GENMASK(9, 0), (op2)))
61+
62+#define __CS_ALU_INSTR(opcode, op1, op2) CS_ALU_INSTR(CS_ALU_OPCODE_##opcode, \
63+ CS_ALU_OPERAND_##op1, \
64+ CS_ALU_OPERAND_##op2)
65+
66+#define CS_ALU_INSTR_NOOP __CS_ALU_INSTR(NOOP, NA, NA)
67+#define CS_ALU_INSTR_LOAD(op1, op2) __CS_ALU_INSTR(LOAD, op1, op2)
68+#define CS_ALU_INSTR_LOADINV(op1, op2) __CS_ALU_INSTR(LOADINV, op1, op2)
69+#define CS_ALU_INSTR_LOAD0(op1) __CS_ALU_INSTR(LOAD0, op1, NA)
70+#define CS_ALU_INSTR_LOAD1(op1) __CS_ALU_INSTR(LOAD1, op1, NA)
71+#define CS_ALU_INSTR_ADD __CS_ALU_INSTR(ADD, NA, NA)
72+#define CS_ALU_INSTR_SUB __CS_ALU_INSTR(SUB, NA, NA)
73+#define CS_ALU_INSTR_AND __CS_ALU_INSTR(AND, NA, NA)
74+#define CS_ALU_INSTR_OR __CS_ALU_INSTR(OR, NA, NA)
75+#define CS_ALU_INSTR_XOR __CS_ALU_INSTR(XOR, NA, NA)
76+#define CS_ALU_INSTR_STORE(op1, op2) __CS_ALU_INSTR(STORE, op1, op2)
77+#define CS_ALU_INSTR_STOREINV(op1, op2) __CS_ALU_INSTR(STOREINV, op1, op2)
78+
79+#endif
drivers/gpu/drm/xe/instructions/xe_gfxpipe_commands.h+1−0
@@ -137,6 +137,7 @@ · 136 unmodified lines
137137 #define CMD_3DSTATE_CLIP_MESH GFXPIPE_3D_CMD(0x0, 0x81)
138138 #define CMD_3DSTATE_SBE_MESH GFXPIPE_3D_CMD(0x0, 0x82)
139139 #define CMD_3DSTATE_CPSIZE_CONTROL_BUFFER GFXPIPE_3D_CMD(0x0, 0x83)
140+#define CMD_3DSTATE_COARSE_PIXEL GFXPIPE_3D_CMD(0x0, 0x89)
140141
141142 #define CMD_3DSTATE_DRAWING_RECTANGLE GFXPIPE_3D_CMD(0x1, 0x0)
142143 #define CMD_3DSTATE_CHROMA_KEY GFXPIPE_3D_CMD(0x1, 0x4)
drivers/gpu/drm/xe/instructions/xe_mi_commands.h+5−0
@@ -32,6 +32,7 @@ · 31 unmodified lines
3232 #define MI_BATCH_BUFFER_END __MI_INSTR(0xA)
3333 #define MI_TOPOLOGY_FILTER __MI_INSTR(0xD)
3434 #define MI_FORCE_WAKEUP __MI_INSTR(0x1D)
35+#define MI_MATH(n) (__MI_INSTR(0x1A) | XE_INSTR_NUM_DW((n) + 1))
3536
3637 #define MI_STORE_DATA_IMM __MI_INSTR(0x20)
3738 #define MI_SDI_GGTT REG_BIT(22)
@@ -61,6 +62,10 @@ · 23 unmodified lines
6162 #define MI_LOAD_REGISTER_MEM (__MI_INSTR(0x29) | XE_INSTR_NUM_DW(4))
6263 #define MI_LRM_USE_GGTT REG_BIT(22)
6364
65+#define MI_LOAD_REGISTER_REG (__MI_INSTR(0x2a) | XE_INSTR_NUM_DW(3))
66+#define MI_LRR_DST_CS_MMIO REG_BIT(19)
67+#define MI_LRR_SRC_CS_MMIO REG_BIT(18)
68+
6469 #define MI_COPY_MEM_MEM (__MI_INSTR(0x2e) | XE_INSTR_NUM_DW(5))
6570 #define MI_COPY_MEM_MEM_SRC_GGTT REG_BIT(22)
6671 #define MI_COPY_MEM_MEM_DST_GGTT REG_BIT(21)
drivers/gpu/drm/xe/regs/xe_engine_regs.h+4−0
@@ -188,6 +188,10 @@ · 187 unmodified lines
188188 #define PREEMPT_GPGPU_LEVEL_MASK PREEMPT_GPGPU_LEVEL(1, 1)
189189 #define PREEMPT_3D_OBJECT_LEVEL REG_BIT(0)
190190
191+#define CS_GPR_DATA(base, n) XE_REG((base) + 0x600 + (n) * 4)
192+#define CS_GPR_REG(base, n) CS_GPR_DATA((base), (n) * 2)
193+#define CS_GPR_REG_UDW(base, n) CS_GPR_DATA((base), (n) * 2 + 1)
194+
191195 #define VDBOX_CGCTL3F08(base) XE_REG((base) + 0x3f08)
192196 #define CG3DDISHRS_CLKGATE_DIS REG_BIT(5)
193197
drivers/gpu/drm/xe/regs/xe_gt_regs.h+12−1
@@ -62,7 +62,6 @@ · 61 unmodified lines
6262 #define LE_SSE_MASK REG_GENMASK(18, 17)
6363 #define LE_SSE(value) REG_FIELD_PREP(LE_SSE_MASK, value)
6464 #define LE_COS_MASK REG_GENMASK(16, 15)
65−#define LE_COS(value) REG_FIELD_PREP(LE_COS_MASK)
6665 #define LE_SCF_MASK REG_BIT(14)
6766 #define LE_SCF(value) REG_FIELD_PREP(LE_SCF_MASK, value)
6867 #define LE_PFM_MASK REG_GENMASK(13, 11)
@@ -392,6 +391,18 @@ · 323 unmodified lines
392391 #define XEHP_L3NODEARBCFG XE_REG_MCR(0xb0b4)
393392 #define XEHP_LNESPARE REG_BIT(19)
394393
394+#define LSN_VC_REG2 XE_REG_MCR(0xb0c8)
395+#define LSN_LNI_WGT_MASK REG_GENMASK(31, 28)
396+#define LSN_LNI_WGT(value) REG_FIELD_PREP(LSN_LNI_WGT_MASK, value)
397+#define LSN_LNE_WGT_MASK REG_GENMASK(27, 24)
398+#define LSN_LNE_WGT(value) REG_FIELD_PREP(LSN_LNE_WGT_MASK, value)
399+#define LSN_DIM_X_WGT_MASK REG_GENMASK(23, 20)
400+#define LSN_DIM_X_WGT(value) REG_FIELD_PREP(LSN_DIM_X_WGT_MASK, value)
401+#define LSN_DIM_Y_WGT_MASK REG_GENMASK(19, 16)
402+#define LSN_DIM_Y_WGT(value) REG_FIELD_PREP(LSN_DIM_Y_WGT_MASK, value)
403+#define LSN_DIM_Z_WGT_MASK REG_GENMASK(15, 12)
404+#define LSN_DIM_Z_WGT(value) REG_FIELD_PREP(LSN_DIM_Z_WGT_MASK, value)
405+
395406 #define L3SQCREG2 XE_REG_MCR(0xb104)
396407 #define COMPMEMRD256BOVRFETCHEN REG_BIT(20)
397408
drivers/gpu/drm/xe/regs/xe_pcode_regs.h+3−0
@@ -21,6 +21,9 @@ · 20 unmodified lines
2121 #define BMG_PACKAGE_POWER_SKU XE_REG(0x138098)
2222 #define BMG_PACKAGE_POWER_SKU_UNIT XE_REG(0x1380dc)
2323 #define BMG_PACKAGE_ENERGY_STATUS XE_REG(0x138120)
24+#define BMG_FAN_1_SPEED XE_REG(0x138140)
25+#define BMG_FAN_2_SPEED XE_REG(0x138170)
26+#define BMG_FAN_3_SPEED XE_REG(0x1381a0)
2427 #define BMG_VRAM_TEMPERATURE XE_REG(0x1382c0)
2528 #define BMG_PACKAGE_TEMPERATURE XE_REG(0x138434)
2629 #define BMG_PACKAGE_RAPL_LIMIT XE_REG(0x138440)
drivers/gpu/drm/xe/tests/xe_bo.c+2−2
@@ -252,7 +252,7 @@ static int evict_test_run_tile(struct xe_device *xe, struct xe_tile *tile, struc · 251 unmodified lines
252252
253253 for_each_gt(__gt, xe, id)
254254 xe_gt_sanitize(__gt);
255− err = xe_bo_restore_kernel(xe);
255+ err = xe_bo_restore_early(xe);
256256 /*
257257 * Snapshotting the CTB and copying back a potentially old
258258 * version seems risky, depending on what might have been
@@ -273,7 +273,7 @@ static int evict_test_run_tile(struct xe_device *xe, struct xe_tile *tile, struc · 14 unmodified lines
273273 goto cleanup_all;
274274 }
275275
276− err = xe_bo_restore_user(xe);
276+ err = xe_bo_restore_late(xe);
277277 if (err) {
278278 KUNIT_FAIL(test, "restore user err=%pe\n", ERR_PTR(err));
279279 goto cleanup_all;
drivers/gpu/drm/xe/tests/xe_migrate.c+3−6
@@ -202,8 +202,7 @@ static void xe_migrate_sanity_test(struct xe_migrate *m, struct kunit *test) · 201 unmodified lines
202202
203203 big = xe_bo_create_pin_map(xe, tile, m->q->vm, SZ_4M,
204204 ttm_bo_type_kernel,
205− XE_BO_FLAG_VRAM_IF_DGFX(tile) |
206− XE_BO_FLAG_PINNED);
205+ XE_BO_FLAG_VRAM_IF_DGFX(tile));
207206 if (IS_ERR(big)) {
208207 KUNIT_FAIL(test, "Failed to allocate bo: %li\n", PTR_ERR(big));
209208 goto vunmap;
@@ -211,8 +210,7 @@ static void xe_migrate_sanity_test(struct xe_migrate *m, struct kunit *test) · 1 unmodified line
211210
212211 pt = xe_bo_create_pin_map(xe, tile, m->q->vm, XE_PAGE_SIZE,
213212 ttm_bo_type_kernel,
214− XE_BO_FLAG_VRAM_IF_DGFX(tile) |
215− XE_BO_FLAG_PINNED);
213+ XE_BO_FLAG_VRAM_IF_DGFX(tile));
216214 if (IS_ERR(pt)) {
217215 KUNIT_FAIL(test, "Failed to allocate fake pt: %li\n",
218216 PTR_ERR(pt));
@@ -222,8 +220,7 @@ static void xe_migrate_sanity_test(struct xe_migrate *m, struct kunit *test) · 3 unmodified lines
222220 tiny = xe_bo_create_pin_map(xe, tile, m->q->vm,
223221 2 * SZ_4K,
224222 ttm_bo_type_kernel,
225− XE_BO_FLAG_VRAM_IF_DGFX(tile) |
226− XE_BO_FLAG_PINNED);
223+ XE_BO_FLAG_VRAM_IF_DGFX(tile));
227224 if (IS_ERR(tiny)) {
228225 KUNIT_FAIL(test, "Failed to allocate tiny fake pt: %li\n",
229226 PTR_ERR(tiny));
drivers/gpu/drm/xe/xe_bo.c+239−164
@@ -55,6 +55,8 @@ static struct ttm_placement sys_placement = { · 54 unmodified lines
5555 .placement = &sys_placement_flags,
5656 };
5757
58+static struct ttm_placement purge_placement;
59+
5860 static const struct ttm_place tt_placement_flags[] = {
5961 {
6062 .fpfn = 0,
@@ -189,11 +191,18 @@ static void try_add_system(struct xe_device *xe, struct xe_bo *bo, · 128 unmodified lines
189191
190192 static bool force_contiguous(u32 bo_flags)
191193 {
194+ if (bo_flags & XE_BO_FLAG_STOLEN)
195+ return true; /* users expect this */
196+ else if (bo_flags & XE_BO_FLAG_PINNED &&
197+ !(bo_flags & XE_BO_FLAG_PINNED_LATE_RESTORE))
198+ return true; /* needs vmap */
199+
192200 /*
193201 * For eviction / restore on suspend / resume objects pinned in VRAM
194202 * must be contiguous, also only contiguous BOs support xe_bo_vmap.
195203 */
196− return bo_flags & (XE_BO_FLAG_PINNED | XE_BO_FLAG_GGTT);
204+ return bo_flags & XE_BO_FLAG_NEEDS_CPU_ACCESS &&
205+ bo_flags & XE_BO_FLAG_PINNED;
197206 }
198207
199208 static void add_vram(struct xe_device *xe, struct xe_bo *bo,
@@ -281,6 +290,8 @@ int xe_bo_placement_for_flags(struct xe_device *xe, struct xe_bo *bo, · 81 unmodified lines
281290 static void xe_evict_flags(struct ttm_buffer_object *tbo,
282291 struct ttm_placement *placement)
283292 {
293+ struct xe_device *xe = container_of(tbo->bdev, typeof(*xe), ttm);
294+ bool device_unplugged = drm_dev_is_unplugged(&xe->drm);
284295 struct xe_bo *bo;
285296
286297 if (!xe_bo_is_xe_bo(tbo)) {
@@ -290,7 +301,7 @@ static void xe_evict_flags(struct ttm_buffer_object *tbo, · 3 unmodified lines
290301 return;
291302 }
292303
293− *placement = sys_placement;
304+ *placement = device_unplugged ? purge_placement : sys_placement;
294305 return;
295306 }
296307
@@ -300,6 +311,11 @@ static void xe_evict_flags(struct ttm_buffer_object *tbo, · 3 unmodified lines
300311 return;
301312 }
302313
314+ if (device_unplugged && !tbo->base.dma_buf) {
315+ *placement = purge_placement;
316+ return;
317+ }
318+
303319 /*
304320 * For xe, sg bos that are evicted to system just triggers a
305321 * rebind of the sg list upon subsequent validation to XE_PL_TT.
@@ -657,11 +673,20 @@ static int xe_bo_move_dmabuf(struct ttm_buffer_object *ttm_bo, · 351 unmodified lines
657673 struct xe_ttm_tt *xe_tt = container_of(ttm_bo->ttm, struct xe_ttm_tt,
658674 ttm);
659675 struct xe_device *xe = ttm_to_xe_device(ttm_bo->bdev);
676+ bool device_unplugged = drm_dev_is_unplugged(&xe->drm);
660677 struct sg_table *sg;
661678
662679 xe_assert(xe, attach);
663680 xe_assert(xe, ttm_bo->ttm);
664681
682+ if (device_unplugged && new_res->mem_type == XE_PL_SYSTEM &&
683+ ttm_bo->sg) {
684+ dma_resv_wait_timeout(ttm_bo->base.resv, DMA_RESV_USAGE_BOOKKEEP,
685+ false, MAX_SCHEDULE_TIMEOUT);
686+ dma_buf_unmap_attachment(attach, ttm_bo->sg, DMA_BIDIRECTIONAL);
687+ ttm_bo->sg = NULL;
688+ }
689+
665690 if (new_res->mem_type == XE_PL_SYSTEM)
666691 goto out;
667692
@@ -898,79 +923,44 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · 230 unmodified lines
898923 xe_pm_runtime_get_noresume(xe);
899924 }
900925
901− if (xe_bo_is_pinned(bo) && !xe_bo_is_user(bo)) {
902− /*
903− * Kernel memory that is pinned should only be moved on suspend
904− * / resume, some of the pinned memory is required for the
905− * device to resume / use the GPU to move other evicted memory
906− * (user memory) around. This likely could be optimized a bit
907− * further where we find the minimum set of pinned memory
908− * required for resume but for simplity doing a memcpy for all
909− * pinned memory.
910− */
911− ret = xe_bo_vmap(bo);
912− if (!ret) {
913− ret = ttm_bo_move_memcpy(ttm_bo, ctx, new_mem);
914−
915− /* Create a new VMAP once kernel BO back in VRAM */
916− if (!ret && resource_is_vram(new_mem)) {
917− struct xe_vram_region *vram = res_to_mem_region(new_mem);
918− void __iomem *new_addr = vram->mapping +
919− (new_mem->start << PAGE_SHIFT);
920−
921− if (XE_WARN_ON(new_mem->start == XE_BO_INVALID_OFFSET)) {
922− ret = -EINVAL;
923− xe_pm_runtime_put(xe);
924− goto out;
925− }
926+ if (move_lacks_source) {
927+ u32 flags = 0;
926928
927− xe_assert(xe, new_mem->start ==
928− bo->placements->fpfn);
929+ if (mem_type_is_vram(new_mem->mem_type))
930+ flags |= XE_MIGRATE_CLEAR_FLAG_FULL;
931+ else if (handle_system_ccs)
932+ flags |= XE_MIGRATE_CLEAR_FLAG_CCS_DATA;
929933
930− iosys_map_set_vaddr_iomem(&bo->vmap, new_addr);
931− }
932− }
934+ fence = xe_migrate_clear(migrate, bo, new_mem, flags);
933935 } else {
934− if (move_lacks_source) {
935− u32 flags = 0;
936−
937− if (mem_type_is_vram(new_mem->mem_type))
938− flags |= XE_MIGRATE_CLEAR_FLAG_FULL;
939− else if (handle_system_ccs)
940− flags |= XE_MIGRATE_CLEAR_FLAG_CCS_DATA;
941−
942− fence = xe_migrate_clear(migrate, bo, new_mem, flags);
943− }
944− else
945− fence = xe_migrate_copy(migrate, bo, bo, old_mem,
946− new_mem, handle_system_ccs);
947− if (IS_ERR(fence)) {
948− ret = PTR_ERR(fence);
949− xe_pm_runtime_put(xe);
950− goto out;
951− }
952− if (!move_lacks_source) {
953− ret = ttm_bo_move_accel_cleanup(ttm_bo, fence, evict,
954− true, new_mem);
955− if (ret) {
956− dma_fence_wait(fence, false);
957− ttm_bo_move_null(ttm_bo, new_mem);
958− ret = 0;
959− }
960− } else {
961− /*
962− * ttm_bo_move_accel_cleanup() may blow up if
963− * bo->resource == NULL, so just attach the
964− * fence and set the new resource.
965− */
966− dma_resv_add_fence(ttm_bo->base.resv, fence,
967− DMA_RESV_USAGE_KERNEL);
936+ fence = xe_migrate_copy(migrate, bo, bo, old_mem, new_mem,
937+ handle_system_ccs);
938+ }
939+ if (IS_ERR(fence)) {
940+ ret = PTR_ERR(fence);
941+ xe_pm_runtime_put(xe);
942+ goto out;
943+ }
944+ if (!move_lacks_source) {
945+ ret = ttm_bo_move_accel_cleanup(ttm_bo, fence, evict, true,
946+ new_mem);
947+ if (ret) {
948+ dma_fence_wait(fence, false);
968949 ttm_bo_move_null(ttm_bo, new_mem);
950+ ret = 0;
969951 }
970−
971− dma_fence_put(fence);
952+ } else {
953+ /*
954+ * ttm_bo_move_accel_cleanup() may blow up if
955+ * bo->resource == NULL, so just attach the
956+ * fence and set the new resource.
957+ */
958+ dma_resv_add_fence(ttm_bo->base.resv, fence,
959+ DMA_RESV_USAGE_KERNEL);
960+ ttm_bo_move_null(ttm_bo, new_mem);
972961 }
973962
963+ dma_fence_put(fence);
974964 xe_pm_runtime_put(xe);
975965
976966 out:
@@ -1107,59 +1097,93 @@ out_unref: · 130 unmodified lines
11071097 */
11081098 int xe_bo_evict_pinned(struct xe_bo *bo)
11091099 {
1110− struct ttm_place place = {
1111− .mem_type = XE_PL_TT,
1112− };
1113− struct ttm_placement placement = {
1114− .placement = &place,
1115− .num_placement = 1,
1116− };
1117− struct ttm_operation_ctx ctx = {
1118− .interruptible = false,
1119− .gfp_retry_mayfail = true,
1120− };
1121− struct ttm_resource *new_mem;
1122− int ret;
1100+ struct xe_device *xe = ttm_to_xe_device(bo->ttm.bdev);
1101+ struct xe_bo *backup;
1102+ bool unmap = false;
1103+ int ret = 0;
11231104
1124− xe_bo_assert_held(bo);
1105+ xe_bo_lock(bo, false);
11251106
1126− if (WARN_ON(!bo->ttm.resource))
1127− return -EINVAL;
1107+ if (WARN_ON(!bo->ttm.resource)) {
1108+ ret = -EINVAL;
1109+ goto out_unlock_bo;
1110+ }
11281111
1129− if (WARN_ON(!xe_bo_is_pinned(bo)))
1130− return -EINVAL;
1112+ if (WARN_ON(!xe_bo_is_pinned(bo))) {
1113+ ret = -EINVAL;
1114+ goto out_unlock_bo;
1115+ }
11311116
11321117 if (!xe_bo_is_vram(bo))
1133− return 0;
1118+ goto out_unlock_bo;
11341119
1135− ret = ttm_bo_mem_space(&bo->ttm, &placement, &new_mem, &ctx);
1136− if (ret)
1137− return ret;
1120+ if (bo->flags & XE_BO_FLAG_PINNED_NORESTORE)
1121+ goto out_unlock_bo;
11381122
1139− if (!bo->ttm.ttm) {
1140− bo->ttm.ttm = xe_ttm_tt_create(&bo->ttm, 0);
1141− if (!bo->ttm.ttm) {
1142− ret = -ENOMEM;
1143− goto err_res_free;
1144− }
1123+ backup = xe_bo_create_locked(xe, NULL, NULL, bo->size, ttm_bo_type_kernel,
1124+ XE_BO_FLAG_SYSTEM | XE_BO_FLAG_NEEDS_CPU_ACCESS |
1125+ XE_BO_FLAG_PINNED);
1126+ if (IS_ERR(backup)) {
1127+ ret = PTR_ERR(backup);
1128+ goto out_unlock_bo;
11451129 }
11461130
1147− ret = ttm_bo_populate(&bo->ttm, &ctx);
1148− if (ret)
1149− goto err_res_free;
1131+ if (xe_bo_is_user(bo) || (bo->flags & XE_BO_FLAG_PINNED_LATE_RESTORE)) {
1132+ struct xe_migrate *migrate;
1133+ struct dma_fence *fence;
11501134
1151− ret = dma_resv_reserve_fences(bo->ttm.base.resv, 1);
1152− if (ret)
1153− goto err_res_free;
1135+ if (bo->tile)
1136+ migrate = bo->tile->migrate;
1137+ else
1138+ migrate = mem_type_to_migrate(xe, bo->ttm.resource->mem_type);
11541139
1155− ret = xe_bo_move(&bo->ttm, false, &ctx, new_mem, NULL);
1156− if (ret)
1157− goto err_res_free;
1140+ ret = dma_resv_reserve_fences(bo->ttm.base.resv, 1);
1141+ if (ret)
1142+ goto out_backup;
11581143
1159− return 0;
1144+ ret = dma_resv_reserve_fences(backup->ttm.base.resv, 1);
1145+ if (ret)
1146+ goto out_backup;
11601147
1161−err_res_free:
1162− ttm_resource_free(&bo->ttm, &new_mem);
1148+ fence = xe_migrate_copy(migrate, bo, backup, bo->ttm.resource,
1149+ backup->ttm.resource, false);
1150+ if (IS_ERR(fence)) {
1151+ ret = PTR_ERR(fence);
1152+ goto out_backup;
1153+ }
1154+
1155+ dma_resv_add_fence(bo->ttm.base.resv, fence,
1156+ DMA_RESV_USAGE_KERNEL);
1157+ dma_resv_add_fence(backup->ttm.base.resv, fence,
1158+ DMA_RESV_USAGE_KERNEL);
1159+ dma_fence_put(fence);
1160+ } else {
1161+ ret = xe_bo_vmap(backup);
1162+ if (ret)
1163+ goto out_backup;
1164+
1165+ if (iosys_map_is_null(&bo->vmap)) {
1166+ ret = xe_bo_vmap(bo);
1167+ if (ret)
1168+ goto out_backup;
1169+ unmap = true;
1170+ }
1171+
1172+ xe_map_memcpy_from(xe, backup->vmap.vaddr, &bo->vmap, 0,
1173+ bo->size);
1174+ }
1175+
1176+ bo->backup_obj = backup;
1177+
1178+out_backup:
1179+ xe_bo_vunmap(backup);
1180+ xe_bo_unlock(backup);
1181+ if (ret)
1182+ xe_bo_put(backup);
1183+out_unlock_bo:
1184+ if (unmap)
1185+ xe_bo_vunmap(bo);
1186+ xe_bo_unlock(bo);
11631187 return ret;
11641188 }
11651189
@@ -1180,50 +1204,110 @@ int xe_bo_restore_pinned(struct xe_bo *bo) · 14 unmodified lines
11801204 .interruptible = false,
11811205 .gfp_retry_mayfail = false,
11821206 };
1183− struct ttm_resource *new_mem;
1184− struct ttm_place *place = &bo->placements[0];
1207+ struct xe_device *xe = ttm_to_xe_device(bo->ttm.bdev);
1208+ struct xe_bo *backup = bo->backup_obj;
1209+ bool unmap = false;
11851210 int ret;
11861211
1187− xe_bo_assert_held(bo);
1212+ if (!backup)
1213+ return 0;
11881214
1189− if (WARN_ON(!bo->ttm.resource))
1190− return -EINVAL;
1215+ xe_bo_lock(backup, false);
11911216
1192− if (WARN_ON(!xe_bo_is_pinned(bo)))
1193− return -EINVAL;
1217+ ret = ttm_bo_validate(&backup->ttm, &backup->placement, &ctx);
1218+ if (ret)
1219+ goto out_backup;
11941220
1195− if (WARN_ON(xe_bo_is_vram(bo)))
1196− return -EINVAL;
1221+ if (WARN_ON(!dma_resv_trylock(bo->ttm.base.resv))) {
1222+ ret = -EBUSY;
1223+ goto out_backup;
1224+ }
11971225
1198− if (WARN_ON(!bo->ttm.ttm && !xe_bo_is_stolen(bo)))
1199− return -EINVAL;
1226+ if (xe_bo_is_user(bo) || (bo->flags & XE_BO_FLAG_PINNED_LATE_RESTORE)) {
1227+ struct xe_migrate *migrate;
1228+ struct dma_fence *fence;
12001229
1201− if (!mem_type_is_vram(place->mem_type))
1202− return 0;
1230+ if (bo->tile)
1231+ migrate = bo->tile->migrate;
1232+ else
1233+ migrate = mem_type_to_migrate(xe, bo->ttm.resource->mem_type);
12031234
1204− ret = ttm_bo_mem_space(&bo->ttm, &bo->placement, &new_mem, &ctx);
1205− if (ret)
1206− return ret;
1235+ ret = dma_resv_reserve_fences(bo->ttm.base.resv, 1);
1236+ if (ret)
1237+ goto out_unlock_bo;
12071238
1208− ret = ttm_bo_populate(&bo->ttm, &ctx);
1209− if (ret)
1210− goto err_res_free;
1239+ ret = dma_resv_reserve_fences(backup->ttm.base.resv, 1);
1240+ if (ret)
1241+ goto out_unlock_bo;
12111242
1212− ret = dma_resv_reserve_fences(bo->ttm.base.resv, 1);
1213− if (ret)
1214− goto err_res_free;
1243+ fence = xe_migrate_copy(migrate, backup, bo,
1244+ backup->ttm.resource, bo->ttm.resource,
1245+ false);
1246+ if (IS_ERR(fence)) {
1247+ ret = PTR_ERR(fence);
1248+ goto out_unlock_bo;
1249+ }
12151250
1216− ret = xe_bo_move(&bo->ttm, false, &ctx, new_mem, NULL);
1217− if (ret)
1218− goto err_res_free;
1251+ dma_resv_add_fence(bo->ttm.base.resv, fence,
1252+ DMA_RESV_USAGE_KERNEL);
1253+ dma_resv_add_fence(backup->ttm.base.resv, fence,
1254+ DMA_RESV_USAGE_KERNEL);
1255+ dma_fence_put(fence);
1256+ } else {
1257+ ret = xe_bo_vmap(backup);
1258+ if (ret)
1259+ goto out_unlock_bo;
12191260
1220− return 0;
1261+ if (iosys_map_is_null(&bo->vmap)) {
1262+ ret = xe_bo_vmap(bo);
1263+ if (ret)
1264+ goto out_unlock_bo;
1265+ unmap = true;
1266+ }
12211267
1222−err_res_free:
1223− ttm_resource_free(&bo->ttm, &new_mem);
1268+ xe_map_memcpy_to(xe, &bo->vmap, 0, backup->vmap.vaddr,
1269+ bo->size);
1270+ }
1271+
1272+ bo->backup_obj = NULL;
1273+
1274+out_unlock_bo:
1275+ if (unmap)
1276+ xe_bo_vunmap(bo);
1277+ xe_bo_unlock(bo);
1278+out_backup:
1279+ xe_bo_vunmap(backup);
1280+ xe_bo_unlock(backup);
1281+ if (!bo->backup_obj)
1282+ xe_bo_put(backup);
12241283 return ret;
12251284 }
12261285
1286+int xe_bo_dma_unmap_pinned(struct xe_bo *bo)
1287+{
1288+ struct ttm_buffer_object *ttm_bo = &bo->ttm;
1289+ struct ttm_tt *tt = ttm_bo->ttm;
1290+
1291+ if (tt) {
1292+ struct xe_ttm_tt *xe_tt = container_of(tt, typeof(*xe_tt), ttm);
1293+
1294+ if (ttm_bo->type == ttm_bo_type_sg && ttm_bo->sg) {
1295+ dma_buf_unmap_attachment(ttm_bo->base.import_attach,
1296+ ttm_bo->sg,
1297+ DMA_BIDIRECTIONAL);
1298+ ttm_bo->sg = NULL;
1299+ xe_tt->sg = NULL;
1300+ } else if (xe_tt->sg) {
1301+ dma_unmap_sgtable(xe_tt->xe->drm.dev, xe_tt->sg,
1302+ DMA_BIDIRECTIONAL, 0);
1303+ sg_free_table(xe_tt->sg);
1304+ xe_tt->sg = NULL;
1305+ }
1306+ }
1307+
1308+ return 0;
1309+}
1310+
12271311 static unsigned long xe_ttm_io_mem_pfn(struct ttm_buffer_object *ttm_bo,
12281312 unsigned long page_offset)
12291313 {
@@ -1947,7 +2031,7 @@ struct xe_bo *xe_bo_create_pin_map_at_aligned(struct xe_device *xe, · 717 unmodified lines
19472031 flags |= XE_BO_FLAG_GGTT;
19482032
19492033 bo = xe_bo_create_locked_range(xe, tile, vm, size, start, end, type,
1950− flags | XE_BO_FLAG_NEEDS_CPU_ACCESS,
2034+ flags | XE_BO_FLAG_NEEDS_CPU_ACCESS | XE_BO_FLAG_PINNED,
19512035 alignment);
19522036 if (IS_ERR(bo))
19532037 return bo;
@@ -2049,7 +2133,8 @@ int xe_managed_bo_reinit_in_vram(struct xe_device *xe, struct xe_tile *tile, str · 95 unmodified lines
20492133 struct xe_bo *bo;
20502134 u32 dst_flags = XE_BO_FLAG_VRAM_IF_DGFX(tile) | XE_BO_FLAG_GGTT;
20512135
2052− dst_flags |= (*src)->flags & XE_BO_FLAG_GGTT_INVALIDATE;
2136+ dst_flags |= (*src)->flags & (XE_BO_FLAG_GGTT_INVALIDATE |
2137+ XE_BO_FLAG_PINNED_NORESTORE);
20532138
20542139 xe_assert(xe, IS_DGFX(xe));
20552140 xe_assert(xe, !(*src)->vmap.is_iomem);
@@ -2073,10 +2158,16 @@ uint64_t vram_region_gpu_offset(struct ttm_resource *res) · 17 unmodified lines
20732158 {
20742159 struct xe_device *xe = ttm_to_xe_device(res->bo->bdev);
20752160
2076− if (res->mem_type == XE_PL_STOLEN)
2161+ switch (res->mem_type) {
2162+ case XE_PL_STOLEN:
20772163 return xe_ttm_stolen_gpu_offset(xe);
2078−
2079− return res_to_mem_region(res)->dpa_base;
2164+ case XE_PL_TT:
2165+ case XE_PL_SYSTEM:
2166+ return 0;
2167+ default:
2168+ return res_to_mem_region(res)->dpa_base;
2169+ }
2170+ return 0;
20802171 }
20812172
20822173 /**
@@ -2102,12 +2193,9 @@ int xe_bo_pin_external(struct xe_bo *bo) · 19 unmodified lines
21022193 if (err)
21032194 return err;
21042195
2105− if (xe_bo_is_vram(bo)) {
2106− spin_lock(&xe->pinned.lock);
2107− list_add_tail(&bo->pinned_link,
2108− &xe->pinned.external_vram);
2109− spin_unlock(&xe->pinned.lock);
2110− }
2196+ spin_lock(&xe->pinned.lock);
2197+ list_add_tail(&bo->pinned_link, &xe->pinned.late.external);
2198+ spin_unlock(&xe->pinned.lock);
21112199 }
21122200
21132201 ttm_bo_pin(&bo->ttm);
@@ -2149,25 +2237,12 @@ int xe_bo_pin(struct xe_bo *bo) · 35 unmodified lines
21492237 if (err)
21502238 return err;
21512239
2152− /*
2153− * For pinned objects in on DGFX, which are also in vram, we expect
2154− * these to be in contiguous VRAM memory. Required eviction / restore
2155− * during suspend / resume (force restore to same physical address).
2156− */
2157− if (IS_DGFX(xe) && !(IS_ENABLED(CONFIG_DRM_XE_DEBUG) &&
2158− bo->flags & XE_BO_FLAG_INTERNAL_TEST)) {
2159− if (mem_type_is_vram(place->mem_type)) {
2160− xe_assert(xe, place->flags & TTM_PL_FLAG_CONTIGUOUS);
2161−
2162− place->fpfn = (xe_bo_addr(bo, 0, PAGE_SIZE) -
2163− vram_region_gpu_offset(bo->ttm.resource)) >> PAGE_SHIFT;
2164− place->lpfn = place->fpfn + (bo->size >> PAGE_SHIFT);
2165− }
2166− }
2167−
21682240 if (mem_type_is_vram(place->mem_type) || bo->flags & XE_BO_FLAG_GGTT) {
21692241 spin_lock(&xe->pinned.lock);
2170− list_add_tail(&bo->pinned_link, &xe->pinned.kernel_bo_present);
2242+ if (bo->flags & XE_BO_FLAG_PINNED_LATE_RESTORE)
2243+ list_add_tail(&bo->pinned_link, &xe->pinned.late.kernel_bo_present);
2244+ else
2245+ list_add_tail(&bo->pinned_link, &xe->pinned.early.kernel_bo_present);
21712246 spin_unlock(&xe->pinned.lock);
21722247 }
21732248
drivers/gpu/drm/xe/xe_bo.h+14−9
@@ -39,20 +39,23 @@ · 38 unmodified lines
3939 #define XE_BO_FLAG_NEEDS_64K BIT(15)
4040 #define XE_BO_FLAG_NEEDS_2M BIT(16)
4141 #define XE_BO_FLAG_GGTT_INVALIDATE BIT(17)
42−#define XE_BO_FLAG_GGTT0 BIT(18)
43−#define XE_BO_FLAG_GGTT1 BIT(19)
44−#define XE_BO_FLAG_GGTT2 BIT(20)
45−#define XE_BO_FLAG_GGTT3 BIT(21)
46−#define XE_BO_FLAG_GGTT_ALL (XE_BO_FLAG_GGTT0 | \
47− XE_BO_FLAG_GGTT1 | \
48− XE_BO_FLAG_GGTT2 | \
49− XE_BO_FLAG_GGTT3)
50−#define XE_BO_FLAG_CPU_ADDR_MIRROR BIT(22)
42+#define XE_BO_FLAG_PINNED_NORESTORE BIT(18)
43+#define XE_BO_FLAG_PINNED_LATE_RESTORE BIT(19)
44+#define XE_BO_FLAG_GGTT0 BIT(20)
45+#define XE_BO_FLAG_GGTT1 BIT(21)
46+#define XE_BO_FLAG_GGTT2 BIT(22)
47+#define XE_BO_FLAG_GGTT3 BIT(23)
48+#define XE_BO_FLAG_CPU_ADDR_MIRROR BIT(24)
5149
5250 /* this one is trigger internally only */
5351 #define XE_BO_FLAG_INTERNAL_TEST BIT(30)
5452 #define XE_BO_FLAG_INTERNAL_64K BIT(31)
5553
54+#define XE_BO_FLAG_GGTT_ALL (XE_BO_FLAG_GGTT0 | \
55+ XE_BO_FLAG_GGTT1 | \
56+ XE_BO_FLAG_GGTT2 | \
57+ XE_BO_FLAG_GGTT3)
58+
5659 #define XE_BO_FLAG_GGTTx(tile) \
5760 (XE_BO_FLAG_GGTT0 << (tile)->id)
5861
@@ -276,6 +279,8 @@ int xe_bo_evict(struct xe_bo *bo); · 217 unmodified lines
276279 int xe_bo_evict_pinned(struct xe_bo *bo);
277280 int xe_bo_restore_pinned(struct xe_bo *bo);
278281
282+int xe_bo_dma_unmap_pinned(struct xe_bo *bo);
283+
279284 extern const struct ttm_device_funcs xe_ttm_funcs;
280285 extern const char *const xe_mem_type_to_name[];
281286
drivers/gpu/drm/xe/xe_bo_evict.c+179−130
@@ -10,6 +10,42 @@ · 9 unmodified lines
1010 #include "xe_ggtt.h"
1111 #include "xe_tile.h"
1212
13+typedef int (*xe_pinned_fn)(struct xe_bo *bo);
14+
15+static int xe_bo_apply_to_pinned(struct xe_device *xe,
16+ struct list_head *pinned_list,
17+ struct list_head *new_list,
18+ const xe_pinned_fn pinned_fn)
19+{
20+ LIST_HEAD(still_in_list);
21+ struct xe_bo *bo;
22+ int ret = 0;
23+
24+ spin_lock(&xe->pinned.lock);
25+ while (!ret) {
26+ bo = list_first_entry_or_null(pinned_list, typeof(*bo),
27+ pinned_link);
28+ if (!bo)
29+ break;
30+ xe_bo_get(bo);
31+ list_move_tail(&bo->pinned_link, &still_in_list);
32+ spin_unlock(&xe->pinned.lock);
33+
34+ ret = pinned_fn(bo);
35+ if (ret && pinned_list != new_list) {
36+ spin_lock(&xe->pinned.lock);
37+ list_move(&bo->pinned_link, pinned_list);
38+ spin_unlock(&xe->pinned.lock);
39+ }
40+ xe_bo_put(bo);
41+ spin_lock(&xe->pinned.lock);
42+ }
43+ list_splice_tail(&still_in_list, new_list);
44+ spin_unlock(&xe->pinned.lock);
45+
46+ return ret;
47+}
48+
1349 /**
1450 * xe_bo_evict_all - evict all BOs from VRAM
1551 *
@@ -27,9 +63,7 @@ · 11 unmodified lines
2763 int xe_bo_evict_all(struct xe_device *xe)
2864 {
2965 struct ttm_device *bdev = &xe->ttm;
30− struct xe_bo *bo;
3166 struct xe_tile *tile;
32− struct list_head still_in_list;
3367 u32 mem_type;
3468 u8 id;
3569 int ret;
@@ -57,34 +91,13 @@ int xe_bo_evict_all(struct xe_device *xe) · 21 unmodified lines
5791 }
5892 }
5993
60− /* Pinned user memory in VRAM */
61− INIT_LIST_HEAD(&still_in_list);
62− spin_lock(&xe->pinned.lock);
63− for (;;) {
64− bo = list_first_entry_or_null(&xe->pinned.external_vram,
65− typeof(*bo), pinned_link);
66− if (!bo)
67− break;
68− xe_bo_get(bo);
69− list_move_tail(&bo->pinned_link, &still_in_list);
70− spin_unlock(&xe->pinned.lock);
71−
72− xe_bo_lock(bo, false);
73− ret = xe_bo_evict_pinned(bo);
74− xe_bo_unlock(bo);
75− xe_bo_put(bo);
76− if (ret) {
77− spin_lock(&xe->pinned.lock);
78− list_splice_tail(&still_in_list,
79− &xe->pinned.external_vram);
80− spin_unlock(&xe->pinned.lock);
81− return ret;
82− }
94+ ret = xe_bo_apply_to_pinned(xe, &xe->pinned.late.external,
95+ &xe->pinned.late.external,
96+ xe_bo_evict_pinned);
8397
84− spin_lock(&xe->pinned.lock);
85− }
86− list_splice_tail(&still_in_list, &xe->pinned.external_vram);
87− spin_unlock(&xe->pinned.lock);
98+ if (!ret)
99+ ret = xe_bo_apply_to_pinned(xe, &xe->pinned.late.kernel_bo_present,
100+ &xe->pinned.late.evicted, xe_bo_evict_pinned);
88101
89102 /*
90103 * Wait for all user BO to be evicted as those evictions depend on the
@@ -93,32 +106,49 @@ int xe_bo_evict_all(struct xe_device *xe) · 2 unmodified lines
93106 for_each_tile(tile, xe, id)
94107 xe_tile_migrate_wait(tile);
95108
96− spin_lock(&xe->pinned.lock);
97− for (;;) {
98− bo = list_first_entry_or_null(&xe->pinned.kernel_bo_present,
99− typeof(*bo), pinned_link);
100− if (!bo)
101− break;
102− xe_bo_get(bo);
103− list_move_tail(&bo->pinned_link, &xe->pinned.evicted);
104− spin_unlock(&xe->pinned.lock);
109+ if (ret)
110+ return ret;
105111
106− xe_bo_lock(bo, false);
107− ret = xe_bo_evict_pinned(bo);
108− xe_bo_unlock(bo);
109− xe_bo_put(bo);
110− if (ret)
111− return ret;
112+ return xe_bo_apply_to_pinned(xe, &xe->pinned.early.kernel_bo_present,
113+ &xe->pinned.early.evicted,
114+ xe_bo_evict_pinned);
115+}
112116
113− spin_lock(&xe->pinned.lock);
117+static int xe_bo_restore_and_map_ggtt(struct xe_bo *bo)
118+{
119+ struct xe_device *xe = xe_bo_device(bo);
120+ int ret;
121+
122+ ret = xe_bo_restore_pinned(bo);
123+ if (ret)
124+ return ret;
125+
126+ if (bo->flags & XE_BO_FLAG_GGTT) {
127+ struct xe_tile *tile;
128+ u8 id;
129+
130+ for_each_tile(tile, xe_bo_device(bo), id) {
131+ if (tile != bo->tile && !(bo->flags & XE_BO_FLAG_GGTTx(tile)))
132+ continue;
133+
134+ mutex_lock(&tile->mem.ggtt->lock);
135+ xe_ggtt_map_bo(tile->mem.ggtt, bo);
136+ mutex_unlock(&tile->mem.ggtt->lock);
137+ }
114138 }
115− spin_unlock(&xe->pinned.lock);
139+
140+ /*
141+ * We expect validate to trigger a move VRAM and our move code
142+ * should setup the iosys map.
143+ */
144+ xe_assert(xe, !(bo->flags & XE_BO_FLAG_PINNED_LATE_RESTORE) ||
145+ !iosys_map_is_null(&bo->vmap));
116146
117147 return 0;
118148 }
119149
120150 /**
121− * xe_bo_restore_kernel - restore kernel BOs to VRAM
151+ * xe_bo_restore_early - restore early phase kernel BOs to VRAM
122152 *
123153 * @xe: xe device
124154 *
@@ -128,111 +158,130 @@ int xe_bo_evict_all(struct xe_device *xe) · 3 unmodified lines
128158 * This function should be called early, before trying to init the GT, on device
129159 * resume.
130160 */
131−int xe_bo_restore_kernel(struct xe_device *xe)
161+int xe_bo_restore_early(struct xe_device *xe)
132162 {
133− struct xe_bo *bo;
134− int ret;
163+ return xe_bo_apply_to_pinned(xe, &xe->pinned.early.evicted,
164+ &xe->pinned.early.kernel_bo_present,
165+ xe_bo_restore_and_map_ggtt);
166+}
135167
136− spin_lock(&xe->pinned.lock);
137− for (;;) {
138− bo = list_first_entry_or_null(&xe->pinned.evicted,
139− typeof(*bo), pinned_link);
140− if (!bo)
141− break;
142− xe_bo_get(bo);
143− list_move_tail(&bo->pinned_link, &xe->pinned.kernel_bo_present);
144− spin_unlock(&xe->pinned.lock);
168+/**
169+ * xe_bo_restore_late - restore pinned late phase BOs
170+ *
171+ * @xe: xe device
172+ *
173+ * Move pinned user and kernel BOs which can use blitter from temporary
174+ * (typically system) memory to VRAM. All moves done via TTM calls.
175+ *
176+ * This function should be called late, after GT init, on device resume.
177+ */
178+int xe_bo_restore_late(struct xe_device *xe)
179+{
180+ struct xe_tile *tile;
181+ int ret, id;
145182
146− xe_bo_lock(bo, false);
147− ret = xe_bo_restore_pinned(bo);
148− xe_bo_unlock(bo);
149− if (ret) {
150− xe_bo_put(bo);
151− return ret;
152− }
183+ ret = xe_bo_apply_to_pinned(xe, &xe->pinned.late.evicted,
184+ &xe->pinned.late.kernel_bo_present,
185+ xe_bo_restore_and_map_ggtt);
153186
154− if (bo->flags & XE_BO_FLAG_GGTT) {
155− struct xe_tile *tile;
156− u8 id;
187+ for_each_tile(tile, xe, id)
188+ xe_tile_migrate_wait(tile);
157189
158− for_each_tile(tile, xe, id) {
159− if (tile != bo->tile && !(bo->flags & XE_BO_FLAG_GGTTx(tile)))
160− continue;
190+ if (ret)
191+ return ret;
161192
162− mutex_lock(&tile->mem.ggtt->lock);
163− xe_ggtt_map_bo(tile->mem.ggtt, bo);
164− mutex_unlock(&tile->mem.ggtt->lock);
165− }
166− }
193+ if (!IS_DGFX(xe))
194+ return 0;
167195
168− /*
169− * We expect validate to trigger a move VRAM and our move code
170− * should setup the iosys map.
171− */
172− xe_assert(xe, !iosys_map_is_null(&bo->vmap));
196+ /* Pinned user memory in VRAM should be validated on resume */
197+ ret = xe_bo_apply_to_pinned(xe, &xe->pinned.late.external,
198+ &xe->pinned.late.external,
199+ xe_bo_restore_pinned);
173200
174− xe_bo_put(bo);
201+ /* Wait for restore to complete */
202+ for_each_tile(tile, xe, id)
203+ xe_tile_migrate_wait(tile);
175204
176− spin_lock(&xe->pinned.lock);
177− }
178− spin_unlock(&xe->pinned.lock);
205+ return ret;
206+}
179207
180− return 0;
208+static void xe_bo_pci_dev_remove_pinned(struct xe_device *xe)
209+{
210+ struct xe_tile *tile;
211+ unsigned int id;
212+
213+ (void)xe_bo_apply_to_pinned(xe, &xe->pinned.late.external,
214+ &xe->pinned.late.external,
215+ xe_bo_dma_unmap_pinned);
216+ for_each_tile(tile, xe, id)
217+ xe_tile_migrate_wait(tile);
181218 }
182219
183220 /**
184− * xe_bo_restore_user - restore pinned user BOs to VRAM
185− *
186− * @xe: xe device
221+ * xe_bo_pci_dev_remove_all() - Handle bos when the pci_device is about to be removed
222+ * @xe: The xe device.
187223 *
188− * Move pinned user BOs from temporary (typically system) memory to VRAM via
189− * CPU. All moves done via TTM calls.
224+ * On pci_device removal we need to drop all dma mappings and move
225+ * the data of exported bos out to system. This includes SVM bos and
226+ * exported dma-buf bos. This is done by evicting all bos, but
227+ * the evict placement in xe_evict_flags() is chosen such that all
228+ * bos except those mentioned are purged, and thus their memory
229+ * is released.
190230 *
191− * This function should be called late, after GT init, on device resume.
231+ * For pinned bos, we're unmapping dma.
192232 */
193−int xe_bo_restore_user(struct xe_device *xe)
233+void xe_bo_pci_dev_remove_all(struct xe_device *xe)
194234 {
195− struct xe_bo *bo;
196− struct xe_tile *tile;
197− struct list_head still_in_list;
198− u8 id;
199− int ret;
235+ unsigned int mem_type;
200236
201− if (!IS_DGFX(xe))
202− return 0;
237+ /*
238+ * Move pagemap bos and exported dma-buf to system, and
239+ * purge everything else.
240+ */
241+ for (mem_type = XE_PL_VRAM1; mem_type >= XE_PL_TT; --mem_type) {
242+ struct ttm_resource_manager *man =
243+ ttm_manager_type(&xe->ttm, mem_type);
203244
204− /* Pinned user memory in VRAM should be validated on resume */
205− INIT_LIST_HEAD(&still_in_list);
206− spin_lock(&xe->pinned.lock);
207− for (;;) {
208− bo = list_first_entry_or_null(&xe->pinned.external_vram,
209− typeof(*bo), pinned_link);
210− if (!bo)
211− break;
212− list_move_tail(&bo->pinned_link, &still_in_list);
213− xe_bo_get(bo);
214− spin_unlock(&xe->pinned.lock);
245+ if (man) {
246+ int ret = ttm_resource_manager_evict_all(&xe->ttm, man);
215247
216− xe_bo_lock(bo, false);
217− ret = xe_bo_restore_pinned(bo);
218− xe_bo_unlock(bo);
219− xe_bo_put(bo);
220− if (ret) {
221− spin_lock(&xe->pinned.lock);
222− list_splice_tail(&still_in_list,
223− &xe->pinned.external_vram);
224− spin_unlock(&xe->pinned.lock);
225− return ret;
248+ drm_WARN_ON(&xe->drm, ret);
226249 }
227−
228− spin_lock(&xe->pinned.lock);
229250 }
230− list_splice_tail(&still_in_list, &xe->pinned.external_vram);
231− spin_unlock(&xe->pinned.lock);
232251
233− /* Wait for restore to complete */
234− for_each_tile(tile, xe, id)
235− xe_tile_migrate_wait(tile);
252+ xe_bo_pci_dev_remove_pinned(xe);
253+}
236254
237− return 0;
255+static void xe_bo_pinned_fini(void *arg)
256+{
257+ struct xe_device *xe = arg;
258+
259+ (void)xe_bo_apply_to_pinned(xe, &xe->pinned.late.kernel_bo_present,
260+ &xe->pinned.late.kernel_bo_present,
261+ xe_bo_dma_unmap_pinned);
262+ (void)xe_bo_apply_to_pinned(xe, &xe->pinned.early.kernel_bo_present,
263+ &xe->pinned.early.kernel_bo_present,
264+ xe_bo_dma_unmap_pinned);
265+}
266+
267+/**
268+ * xe_bo_pinned_init() - Initialize pinned bo tracking
269+ * @xe: The xe device.
270+ *
271+ * Initializes the lists and locks required for pinned bo
272+ * tracking and registers a callback to dma-unmap
273+ * any remaining pinned bos on pci device removal.
274+ *
275+ * Return: %0 on success, negative error code on error.
276+ */
277+int xe_bo_pinned_init(struct xe_device *xe)
278+{
279+ spin_lock_init(&xe->pinned.lock);
280+ INIT_LIST_HEAD(&xe->pinned.early.kernel_bo_present);
281+ INIT_LIST_HEAD(&xe->pinned.early.evicted);
282+ INIT_LIST_HEAD(&xe->pinned.late.kernel_bo_present);
283+ INIT_LIST_HEAD(&xe->pinned.late.evicted);
284+ INIT_LIST_HEAD(&xe->pinned.late.external);
285+
286+ return devm_add_action_or_reset(xe->drm.dev, xe_bo_pinned_fini, xe);
238287 }
drivers/gpu/drm/xe/xe_bo_evict.h+5−2
@@ -9,7 +9,10 @@ · 8 unmodified lines
99 struct xe_device;
1010
1111 int xe_bo_evict_all(struct xe_device *xe);
12−int xe_bo_restore_kernel(struct xe_device *xe);
13−int xe_bo_restore_user(struct xe_device *xe);
12+int xe_bo_restore_early(struct xe_device *xe);
13+int xe_bo_restore_late(struct xe_device *xe);
1414
15+void xe_bo_pci_dev_remove_all(struct xe_device *xe);
16+
17+int xe_bo_pinned_init(struct xe_device *xe);
1518 #endif
drivers/gpu/drm/xe/xe_bo_types.h+2−0
@@ -28,6 +28,8 @@ struct xe_vm; · 27 unmodified lines
2828 struct xe_bo {
2929 /** @ttm: TTM base buffer object */
3030 struct ttm_buffer_object ttm;
31+ /** @backup_obj: The backup object when pinned and suspended (vram only) */
32+ struct xe_bo *backup_obj;
3133 /** @size: Size of this buffer object */
3234 size_t size;
3335 /** @flags: flags for this buffer object */
drivers/gpu/drm/xe/xe_configfs.c+250−0
@@ -0,0 +1,250 @@
1+// SPDX-License-Identifier: MIT
2+/*
3+ * Copyright © 2025 Intel Corporation
4+ */
5+
6+#include <linux/configfs.h>
7+#include <linux/init.h>
8+#include <linux/module.h>
9+#include <linux/pci.h>
10+
11+#include "xe_configfs.h"
12+#include "xe_module.h"
13+
14+/**
15+ * DOC: Xe Configfs
16+ *
17+ * Overview
18+ * =========
19+ *
20+ * Configfs is a filesystem-based manager of kernel objects. XE KMD registers a
21+ * configfs subsystem called ``'xe'`` that creates a directory in the mounted configfs directory
22+ * The user can create devices under this directory and configure them as necessary
23+ * See Documentation/filesystems/configfs.rst for more information about how configfs works.
24+ *
25+ * Create devices
26+ * ===============
27+ *
28+ * In order to create a device, the user has to create a directory inside ``'xe'``::
29+ *
30+ * mkdir /sys/kernel/config/xe/0000:03:00.0/
31+ *
32+ * Every device created is populated by the driver with entries that can be
33+ * used to configure it::
34+ *
35+ * /sys/kernel/config/xe/
36+ * .. 0000:03:00.0/
37+ * ... survivability_mode
38+ *
39+ * Configure Attributes
40+ * ====================
41+ *
42+ * Survivability mode:
43+ * -------------------
44+ *
45+ * Enable survivability mode on supported cards. This setting only takes
46+ * effect when probing the device. Example to enable it::
47+ *
48+ * # echo 1 > /sys/kernel/config/xe/0000:03:00.0/survivability_mode
49+ * # echo 0000:03:00.0 > /sys/bus/pci/drivers/xe/bind (Enters survivability mode if supported)
50+ *
51+ * Remove devices
52+ * ==============
53+ *
54+ * The created device directories can be removed using ``rmdir``::
55+ *
56+ * rmdir /sys/kernel/config/xe/0000:03:00.0/
57+ */
58+
59+struct xe_config_device {
60+ struct config_group group;
61+
62+ bool survivability_mode;
63+
64+ /* protects attributes */
65+ struct mutex lock;
66+};
67+
68+static struct xe_config_device *to_xe_config_device(struct config_item *item)
69+{
70+ return container_of(to_config_group(item), struct xe_config_device, group);
71+}
72+
73+static ssize_t survivability_mode_show(struct config_item *item, char *page)
74+{
75+ struct xe_config_device *dev = to_xe_config_device(item);
76+
77+ return sprintf(page, "%d\n", dev->survivability_mode);
78+}
79+
80+static ssize_t survivability_mode_store(struct config_item *item, const char *page, size_t len)
81+{
82+ struct xe_config_device *dev = to_xe_config_device(item);
83+ bool survivability_mode;
84+ int ret;
85+
86+ ret = kstrtobool(page, &survivability_mode);
87+ if (ret)
88+ return ret;
89+
90+ mutex_lock(&dev->lock);
91+ dev->survivability_mode = survivability_mode;
92+ mutex_unlock(&dev->lock);
93+
94+ return len;
95+}
96+
97+CONFIGFS_ATTR(, survivability_mode);
98+
99+static struct configfs_attribute *xe_config_device_attrs[] = {
100+ &attr_survivability_mode,
101+ NULL,
102+};
103+
104+static void xe_config_device_release(struct config_item *item)
105+{
106+ struct xe_config_device *dev = to_xe_config_device(item);
107+
108+ mutex_destroy(&dev->lock);
109+ kfree(dev);
110+}
111+
112+static struct configfs_item_operations xe_config_device_ops = {
113+ .release = xe_config_device_release,
114+};
115+
116+static const struct config_item_type xe_config_device_type = {
117+ .ct_item_ops = &xe_config_device_ops,
118+ .ct_attrs = xe_config_device_attrs,
119+ .ct_owner = THIS_MODULE,
120+};
121+
122+static struct config_group *xe_config_make_device_group(struct config_group *group,
123+ const char *name)
124+{
125+ unsigned int domain, bus, slot, function;
126+ struct xe_config_device *dev;
127+ struct pci_dev *pdev;
128+ int ret;
129+
130+ ret = sscanf(name, "%04x:%02x:%02x.%x", &domain, &bus, &slot, &function);
131+ if (ret != 4)
132+ return ERR_PTR(-EINVAL);
133+
134+ pdev = pci_get_domain_bus_and_slot(domain, bus, PCI_DEVFN(slot, function));
135+ if (!pdev)
136+ return ERR_PTR(-EINVAL);
137+
138+ dev = kzalloc(sizeof(*dev), GFP_KERNEL);
139+ if (!dev)
140+ return ERR_PTR(-ENOMEM);
141+
142+ config_group_init_type_name(&dev->group, name, &xe_config_device_type);
143+
144+ mutex_init(&dev->lock);
145+
146+ return &dev->group;
147+}
148+
149+static struct configfs_group_operations xe_config_device_group_ops = {
150+ .make_group = xe_config_make_device_group,
151+};
152+
153+static const struct config_item_type xe_configfs_type = {
154+ .ct_group_ops = &xe_config_device_group_ops,
155+ .ct_owner = THIS_MODULE,
156+};
157+
158+static struct configfs_subsystem xe_configfs = {
159+ .su_group = {
160+ .cg_item = {
161+ .ci_namebuf = "xe",
162+ .ci_type = &xe_configfs_type,
163+ },
164+ },
165+};
166+
167+static struct xe_config_device *configfs_find_group(struct pci_dev *pdev)
168+{
169+ struct config_item *item;
170+ char name[64];
171+
172+ snprintf(name, sizeof(name), "%04x:%02x:%02x.%x", pci_domain_nr(pdev->bus),
173+ pdev->bus->number, PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn));
174+
175+ mutex_lock(&xe_configfs.su_mutex);
176+ item = config_group_find_item(&xe_configfs.su_group, name);
177+ mutex_unlock(&xe_configfs.su_mutex);
178+
179+ if (!item)
180+ return NULL;
181+
182+ return to_xe_config_device(item);
183+}
184+
185+/**
186+ * xe_configfs_get_survivability_mode - get configfs survivability mode attribute
187+ * @pdev: pci device
188+ *
189+ * find the configfs group that belongs to the pci device and return
190+ * the survivability mode attribute
191+ *
192+ * Return: survivability mode if config group is found, false otherwise
193+ */
194+bool xe_configfs_get_survivability_mode(struct pci_dev *pdev)
195+{
196+ struct xe_config_device *dev = configfs_find_group(pdev);
197+ bool mode;
198+
199+ if (!dev)
200+ return false;
201+
202+ mode = dev->survivability_mode;
203+ config_item_put(&dev->group.cg_item);
204+
205+ return mode;
206+}
207+
208+/**
209+ * xe_configfs_clear_survivability_mode - clear configfs survivability mode attribute
210+ * @pdev: pci device
211+ *
212+ * find the configfs group that belongs to the pci device and clear survivability
213+ * mode attribute
214+ */
215+void xe_configfs_clear_survivability_mode(struct pci_dev *pdev)
216+{
217+ struct xe_config_device *dev = configfs_find_group(pdev);
218+
219+ if (!dev)
220+ return;
221+
222+ mutex_lock(&dev->lock);
223+ dev->survivability_mode = 0;
224+ mutex_unlock(&dev->lock);
225+
226+ config_item_put(&dev->group.cg_item);
227+}
228+
229+int __init xe_configfs_init(void)
230+{
231+ struct config_group *root = &xe_configfs.su_group;
232+ int ret;
233+
234+ config_group_init(root);
235+ mutex_init(&xe_configfs.su_mutex);
236+ ret = configfs_register_subsystem(&xe_configfs);
237+ if (ret) {
238+ pr_err("Error %d while registering %s subsystem\n",
239+ ret, root->cg_item.ci_namebuf);
240+ return ret;
241+ }
242+
243+ return 0;
244+}
245+
246+void __exit xe_configfs_exit(void)
247+{
248+ configfs_unregister_subsystem(&xe_configfs);
249+}
250+
drivers/gpu/drm/xe/xe_configfs.h+24−0
@@ -0,0 +1,24 @@
1+/* SPDX-License-Identifier: MIT */
2+/*
3+ * Copyright © 2025 Intel Corporation
4+ */
5+#ifndef _XE_CONFIGFS_H_
6+#define _XE_CONFIGFS_H_
7+
8+#include <linux/types.h>
9+
10+struct pci_dev;
11+
12+#if IS_ENABLED(CONFIG_CONFIGFS_FS)
13+int xe_configfs_init(void);
14+void xe_configfs_exit(void);
15+bool xe_configfs_get_survivability_mode(struct pci_dev *pdev);
16+void xe_configfs_clear_survivability_mode(struct pci_dev *pdev);
17+#else
18+static inline int xe_configfs_init(void) { return 0; };
19+static inline void xe_configfs_exit(void) {};
20+static inline bool xe_configfs_get_survivability_mode(struct pci_dev *pdev) { return false; };
21+static inline void xe_configfs_clear_survivability_mode(struct pci_dev *pdev) {};
22+#endif
23+
24+#endif
drivers/gpu/drm/xe/xe_device.c+19−10
@@ -23,6 +23,7 @@ · 22 unmodified lines
2323 #include "regs/xe_gt_regs.h"
2424 #include "regs/xe_regs.h"
2525 #include "xe_bo.h"
26+#include "xe_bo_evict.h"
2627 #include "xe_debugfs.h"
2728 #include "xe_devcoredump.h"
2829 #include "xe_dma_buf.h"
@@ -467,10 +468,9 @@ struct xe_device *xe_device_create(struct pci_dev *pdev, · 438 unmodified lines
467468 xa_erase(&xe->usm.asid_to_vm, asid);
468469 }
469470
470− spin_lock_init(&xe->pinned.lock);
471− INIT_LIST_HEAD(&xe->pinned.kernel_bo_present);
472− INIT_LIST_HEAD(&xe->pinned.external_vram);
473− INIT_LIST_HEAD(&xe->pinned.evicted);
471+ err = xe_bo_pinned_init(xe);
472+ if (err)
473+ goto err;
474474
475475 xe->preempt_fence_wq = alloc_ordered_workqueue("xe-preempt-fence-wq",
476476 WQ_MEM_RECLAIM);
@@ -505,7 +505,15 @@ ALLOW_ERROR_INJECTION(xe_device_create, ERRNO); /* See xe_pci_probe() */ · 28 unmodified lines
505505
506506 static bool xe_driver_flr_disabled(struct xe_device *xe)
507507 {
508− return xe_mmio_read32(xe_root_tile_mmio(xe), GU_CNTL_PROTECTED) & DRIVERINT_FLR_DIS;
508+ if (IS_SRIOV_VF(xe))
509+ return true;
510+
511+ if (xe_mmio_read32(xe_root_tile_mmio(xe), GU_CNTL_PROTECTED) & DRIVERINT_FLR_DIS) {
512+ drm_info(&xe->drm, "Driver-FLR disabled by BIOS\n");
513+ return true;
514+ }
515+
516+ return false;
509517 }
510518
511519 /*
@@ -523,7 +531,7 @@ static bool xe_driver_flr_disabled(struct xe_device *xe) · 11 unmodified lines
523531 */
524532 static void __xe_driver_flr(struct xe_device *xe)
525533 {
526− const unsigned int flr_timeout = 3 * MICRO; /* specs recommend a 3s wait */
534+ const unsigned int flr_timeout = 3 * USEC_PER_SEC; /* specs recommend a 3s wait */
527535 struct xe_mmio *mmio = xe_root_tile_mmio(xe);
528536 int ret;
529537
@@ -569,10 +577,8 @@ static void __xe_driver_flr(struct xe_device *xe) · 39 unmodified lines
569577
570578 static void xe_driver_flr(struct xe_device *xe)
571579 {
572− if (xe_driver_flr_disabled(xe)) {
573− drm_info_once(&xe->drm, "BIOS Disabled Driver-FLR\n");
580+ if (xe_driver_flr_disabled(xe))
574581 return;
575− }
576582
577583 __xe_driver_flr(xe);
578584 }
@@ -706,7 +712,7 @@ int xe_device_probe_early(struct xe_device *xe) · 127 unmodified lines
706712 sriov_update_device_info(xe);
707713
708714 err = xe_pcode_probe_early(xe);
709− if (err) {
715+ if (err || xe_survivability_mode_is_requested(xe)) {
710716 int save_err = err;
711717
712718 /*
@@ -729,6 +735,7 @@ int xe_device_probe_early(struct xe_device *xe) · 16 unmodified lines
729735
730736 return 0;
731737 }
738+ALLOW_ERROR_INJECTION(xe_device_probe_early, ERRNO); /* See xe_pci_probe() */
732739
733740 static int probe_has_flat_ccs(struct xe_device *xe)
734741 {
@@ -932,6 +939,8 @@ void xe_device_remove(struct xe_device *xe) · 197 unmodified lines
932939 xe_display_unregister(xe);
933940
934941 drm_dev_unplug(&xe->drm);
942+
943+ xe_bo_pci_dev_remove_all(xe);
935944 }
936945
937946 void xe_device_shutdown(struct xe_device *xe)
drivers/gpu/drm/xe/xe_device_types.h+24−8
@@ -107,6 +107,9 @@ struct xe_vram_region { · 106 unmodified lines
107107 resource_size_t actual_physical_size;
108108 /** @mapping: pointer to VRAM mappable space */
109109 void __iomem *mapping;
110+ /** @ttm: VRAM TTM manager */
111+ struct xe_ttm_vram_mgr ttm;
112+#if IS_ENABLED(CONFIG_DRM_XE_DEVMEM_MIRROR)
110113 /** @pagemap: Used to remap device memory as ZONE_DEVICE */
111114 struct dev_pagemap pagemap;
112115 /**
@@ -120,8 +123,7 @@ struct xe_vram_region { · 7 unmodified lines
120123 * This is generated when remap device memory as ZONE_DEVICE
121124 */
122125 resource_size_t hpa_base;
123− /** @ttm: VRAM TTM manager */
124− struct xe_ttm_vram_mgr ttm;
126+#endif
125127 };
126128
127129 /**
@@ -314,6 +316,8 @@ struct xe_device { · 186 unmodified lines
314316 u8 has_atomic_enable_pte_bit:1;
315317 /** @info.has_device_atomics_on_smem: Supports device atomics on SMEM */
316318 u8 has_device_atomics_on_smem:1;
319+ /** @info.has_fan_control: Device supports fan control */
320+ u8 has_fan_control:1;
317321 /** @info.has_flat_ccs: Whether flat CCS metadata is used */
318322 u8 has_flat_ccs:1;
319323 /** @info.has_heci_cscfi: device has heci cscfi */
@@ -332,6 +336,8 @@ struct xe_device { · 12 unmodified lines
332336 u8 has_usm:1;
333337 /** @info.is_dgfx: is discrete device */
334338 u8 is_dgfx:1;
339+ /** @info.needs_scratch: needs scratch page for oob prefetch to work */
340+ u8 needs_scratch:1;
335341 /**
336342 * @info.probe_display: Probe display hardware. If set to
337343 * false, the driver will behave as if there is no display
@@ -418,12 +424,22 @@ struct xe_device { · 80 unmodified lines
418424 struct {
419425 /** @pinned.lock: protected pinned BO list state */
420426 spinlock_t lock;
421− /** @pinned.kernel_bo_present: pinned kernel BO that are present */
422− struct list_head kernel_bo_present;
423− /** @pinned.evicted: pinned BO that have been evicted */
424− struct list_head evicted;
425− /** @pinned.external_vram: pinned external BO in vram*/
426− struct list_head external_vram;
427+ /** @pinned.early: early pinned lists */
428+ struct {
429+ /** @pinned.early.kernel_bo_present: pinned kernel BO that are present */
430+ struct list_head kernel_bo_present;
431+ /** @pinned.early.evicted: pinned BO that have been evicted */
432+ struct list_head evicted;
433+ } early;
434+ /** @pinned.late: late pinned lists */
435+ struct {
436+ /** @pinned.late.kernel_bo_present: pinned kernel BO that are present */
437+ struct list_head kernel_bo_present;
438+ /** @pinned.late.evicted: pinned BO that have been evicted */
439+ struct list_head evicted;
440+ /** @pinned.external: pinned external and dma-buf. */
441+ struct list_head external;
442+ } late;
427443 } pinned;
428444
429445 /** @ufence_wq: user fence wait queue */
drivers/gpu/drm/xe/xe_dma_buf.c+1−4
@@ -145,10 +145,7 @@ static void xe_dma_buf_unmap(struct dma_buf_attachment *attach, · 144 unmodified lines
145145 struct sg_table *sgt,
146146 enum dma_data_direction dir)
147147 {
148− struct dma_buf *dma_buf = attach->dmabuf;
149− struct xe_bo *bo = gem_to_xe_bo(dma_buf->priv);
150−
151− if (!xe_bo_is_vram(bo)) {
148+ if (sg_page(sgt->sgl)) {
152149 dma_unmap_sgtable(attach->dev, sgt, dir, 0);
153150 sg_free_table(sgt);
154151 kfree(sgt);
drivers/gpu/drm/xe/xe_force_wake.c+0−6
@@ -49,9 +49,6 @@ void xe_force_wake_init_gt(struct xe_gt *gt, struct xe_force_wake *fw) · 48 unmodified lines
4949 fw->gt = gt;
5050 spin_lock_init(&fw->lock);
5151
52− /* Assuming gen11+ so assert this assumption is correct */
53− xe_gt_assert(gt, GRAPHICS_VER(gt_to_xe(gt)) >= 11);
54−
5552 if (xe->info.graphics_verx100 >= 1270) {
5653 init_domain(fw, XE_FW_DOMAIN_ID_GT,
5754 FORCEWAKE_GT,
@@ -67,9 +64,6 @@ void xe_force_wake_init_engines(struct xe_gt *gt, struct xe_force_wake *fw) · 9 unmodified lines
6764 {
6865 int i, j;
6966
70− /* Assuming gen11+ so assert this assumption is correct */
71− xe_gt_assert(gt, GRAPHICS_VER(gt_to_xe(gt)) >= 11);
72−
7367 if (!xe_gt_is_media_type(gt))
7468 init_domain(fw, XE_FW_DOMAIN_ID_RENDER,
7569 FORCEWAKE_RENDER,
drivers/gpu/drm/xe/xe_ggtt.c+1−1
@@ -365,7 +365,7 @@ int xe_ggtt_init(struct xe_ggtt *ggtt) · 364 unmodified lines
365365 * scratch entries, rather keep the scratch page in system memory on
366366 * platforms where 64K pages are needed for VRAM.
367367 */
368− flags = XE_BO_FLAG_PINNED;
368+ flags = 0;
369369 if (ggtt->flags & XE_GGTT_FLAGS_64K)
370370 flags |= XE_BO_FLAG_SYSTEM;
371371 else
drivers/gpu/drm/xe/xe_gt.c+63−21
@@ -12,8 +12,10 @@ · 11 unmodified lines
1212
1313 #include <generated/xe_wa_oob.h>
1414
15+#include "instructions/xe_alu_commands.h"
1516 #include "instructions/xe_gfxpipe_commands.h"
1617 #include "instructions/xe_mi_commands.h"
18+#include "regs/xe_engine_regs.h"
1719 #include "regs/xe_gt_regs.h"
1820 #include "xe_assert.h"
1921 #include "xe_bb.h"
@@ -176,15 +178,6 @@ static int emit_nop_job(struct xe_gt *gt, struct xe_exec_queue *q) · 156 unmodified lines
176178 return 0;
177179 }
178180
179−/*
180− * Convert back from encoded value to type-safe, only to be used when reg.mcr
181− * is true
182− */
183−static struct xe_reg_mcr to_xe_reg_mcr(const struct xe_reg reg)
184−{
185− return (const struct xe_reg_mcr){.__reg.raw = reg.raw };
186−}
187−
188181 static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q)
189182 {
190183 struct xe_reg_sr *sr = &q->hwe->reg_lrc;
@@ -194,6 +187,7 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 3 unmodified lines
194187 struct xe_bb *bb;
195188 struct dma_fence *fence;
196189 long timeout;
190+ int count_rmw = 0;
197191 int count = 0;
198192
199193 if (q->hwe->class == XE_ENGINE_CLASS_RENDER)
@@ -206,30 +200,32 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 6 unmodified lines
206200 if (IS_ERR(bb))
207201 return PTR_ERR(bb);
208202
209− xa_for_each(&sr->xa, idx, entry)
210− ++count;
203+ /* count RMW registers as those will be handled separately */
204+ xa_for_each(&sr->xa, idx, entry) {
205+ if (entry->reg.masked || entry->clr_bits == ~0)
206+ ++count;
207+ else
208+ ++count_rmw;
209+ }
211210
212− if (count) {
211+ if (count || count_rmw)
213212 xe_gt_dbg(gt, "LRC WA %s save-restore batch\n", sr->name);
214213
214+ if (count) {
215+ /* emit single LRI with all non RMW regs */
216+
215217 bb->cs[bb->len++] = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(count);
216218
217219 xa_for_each(&sr->xa, idx, entry) {
218220 struct xe_reg reg = entry->reg;
219− struct xe_reg_mcr reg_mcr = to_xe_reg_mcr(reg);
220221 u32 val;
221222
222− /*
223− * Skip reading the register if it's not really needed
224− */
225223 if (reg.masked)
226224 val = entry->clr_bits << 16;
227− else if (entry->clr_bits + 1)
228− val = (reg.mcr ?
229− xe_gt_mcr_unicast_read_any(gt, reg_mcr) :
230− xe_mmio_read32(&gt->mmio, reg)) & (~entry->clr_bits);
231− else
225+ else if (entry->clr_bits == ~0)
232226 val = 0;
227+ else
228+ continue;
233229
234230 val |= entry->set_bits;
235231
@@ -239,6 +235,52 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 3 unmodified lines
239235 }
240236 }
241237
238+ if (count_rmw) {
239+ /* emit MI_MATH for each RMW reg */
240+
241+ xa_for_each(&sr->xa, idx, entry) {
242+ if (entry->reg.masked || entry->clr_bits == ~0)
243+ continue;
244+
245+ bb->cs[bb->len++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO;
246+ bb->cs[bb->len++] = entry->reg.addr;
247+ bb->cs[bb->len++] = CS_GPR_REG(0, 0).addr;
248+
249+ bb->cs[bb->len++] = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(2) |
250+ MI_LRI_LRM_CS_MMIO;
251+ bb->cs[bb->len++] = CS_GPR_REG(0, 1).addr;
252+ bb->cs[bb->len++] = entry->clr_bits;
253+ bb->cs[bb->len++] = CS_GPR_REG(0, 2).addr;
254+ bb->cs[bb->len++] = entry->set_bits;
255+
256+ bb->cs[bb->len++] = MI_MATH(8);
257+ bb->cs[bb->len++] = CS_ALU_INSTR_LOAD(SRCA, REG0);
258+ bb->cs[bb->len++] = CS_ALU_INSTR_LOADINV(SRCB, REG1);
259+ bb->cs[bb->len++] = CS_ALU_INSTR_AND;
260+ bb->cs[bb->len++] = CS_ALU_INSTR_STORE(REG0, ACCU);
261+ bb->cs[bb->len++] = CS_ALU_INSTR_LOAD(SRCA, REG0);
262+ bb->cs[bb->len++] = CS_ALU_INSTR_LOAD(SRCB, REG2);
263+ bb->cs[bb->len++] = CS_ALU_INSTR_OR;
264+ bb->cs[bb->len++] = CS_ALU_INSTR_STORE(REG0, ACCU);
265+
266+ bb->cs[bb->len++] = MI_LOAD_REGISTER_REG | MI_LRR_SRC_CS_MMIO;
267+ bb->cs[bb->len++] = CS_GPR_REG(0, 0).addr;
268+ bb->cs[bb->len++] = entry->reg.addr;
269+
270+ xe_gt_dbg(gt, "REG[%#x] = ~%#x|%#x\n",
271+ entry->reg.addr, entry->clr_bits, entry->set_bits);
272+ }
273+
274+ /* reset used GPR */
275+ bb->cs[bb->len++] = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(3) | MI_LRI_LRM_CS_MMIO;
276+ bb->cs[bb->len++] = CS_GPR_REG(0, 0).addr;
277+ bb->cs[bb->len++] = 0;
278+ bb->cs[bb->len++] = CS_GPR_REG(0, 1).addr;
279+ bb->cs[bb->len++] = 0;
280+ bb->cs[bb->len++] = CS_GPR_REG(0, 2).addr;
281+ bb->cs[bb->len++] = 0;
282+ }
283+
242284 xe_lrc_emit_hwe_state_instructions(q, bb);
243285
244286 job = xe_bb_create_job(q, bb);
drivers/gpu/drm/xe/xe_gt_debugfs.c+22−8
@@ -299,20 +299,20 @@ static int hwconfig(struct xe_gt *gt, struct drm_printer *p) · 298 unmodified lines
299299 return 0;
300300 }
301301
302−static const struct drm_info_list debugfs_list[] = {
303− {"hw_engines", .show = xe_gt_debugfs_simple_show, .data = hw_engines},
302+/*
303+ * only for GT debugfs files which can be safely used on the VF as well:
304+ * - without access to the GT privileged registers
305+ * - without access to the PF specific data
306+ */
307+static const struct drm_info_list vf_safe_debugfs_list[] = {
304308 {"force_reset", .show = xe_gt_debugfs_simple_show, .data = force_reset},
305309 {"force_reset_sync", .show = xe_gt_debugfs_simple_show, .data = force_reset_sync},
306310 {"sa_info", .show = xe_gt_debugfs_simple_show, .data = sa_info},
307311 {"topology", .show = xe_gt_debugfs_simple_show, .data = topology},
308− {"steering", .show = xe_gt_debugfs_simple_show, .data = steering},
309312 {"ggtt", .show = xe_gt_debugfs_simple_show, .data = ggtt},
310− {"powergate_info", .show = xe_gt_debugfs_simple_show, .data = powergate_info},
311313 {"register-save-restore", .show = xe_gt_debugfs_simple_show, .data = register_save_restore},
312314 {"workarounds", .show = xe_gt_debugfs_simple_show, .data = workarounds},
313315 {"tunings", .show = xe_gt_debugfs_simple_show, .data = tunings},
314− {"pat", .show = xe_gt_debugfs_simple_show, .data = pat},
315− {"mocs", .show = xe_gt_debugfs_simple_show, .data = mocs},
316316 {"default_lrc_rcs", .show = xe_gt_debugfs_simple_show, .data = rcs_default_lrc},
317317 {"default_lrc_ccs", .show = xe_gt_debugfs_simple_show, .data = ccs_default_lrc},
318318 {"default_lrc_bcs", .show = xe_gt_debugfs_simple_show, .data = bcs_default_lrc},
@@ -322,6 +322,15 @@ static const struct drm_info_list debugfs_list[] = { · 3 unmodified lines
322322 {"hwconfig", .show = xe_gt_debugfs_simple_show, .data = hwconfig},
323323 };
324324
325+/* everything else should be added here */
326+static const struct drm_info_list pf_only_debugfs_list[] = {
327+ {"hw_engines", .show = xe_gt_debugfs_simple_show, .data = hw_engines},
328+ {"mocs", .show = xe_gt_debugfs_simple_show, .data = mocs},
329+ {"pat", .show = xe_gt_debugfs_simple_show, .data = pat},
330+ {"powergate_info", .show = xe_gt_debugfs_simple_show, .data = powergate_info},
331+ {"steering", .show = xe_gt_debugfs_simple_show, .data = steering},
332+};
333+
325334 void xe_gt_debugfs_register(struct xe_gt *gt)
326335 {
327336 struct xe_device *xe = gt_to_xe(gt);
@@ -345,10 +354,15 @@ void xe_gt_debugfs_register(struct xe_gt *gt) · 17 unmodified lines
345354 */
346355 root->d_inode->i_private = gt;
347356
348− drm_debugfs_create_files(debugfs_list,
349− ARRAY_SIZE(debugfs_list),
357+ drm_debugfs_create_files(vf_safe_debugfs_list,
358+ ARRAY_SIZE(vf_safe_debugfs_list),
350359 root, minor);
351360
361+ if (!IS_SRIOV_VF(xe))
362+ drm_debugfs_create_files(pf_only_debugfs_list,
363+ ARRAY_SIZE(pf_only_debugfs_list),
364+ root, minor);
365+
352366 xe_uc_debugfs_register(&gt->uc, root);
353367
354368 if (IS_SRIOV_PF(xe))
drivers/gpu/drm/xe/xe_gt_mcr.c+2−1
@@ -345,7 +345,8 @@ fallback: · 344 unmodified lines
345345 * Some older platforms don't have tables or don't have complete tables.
346346 * Newer platforms should always have the required info.
347347 */
348− if (GRAPHICS_VERx100(gt_to_xe(gt)) >= 2000)
348+ if (GRAPHICS_VERx100(gt_to_xe(gt)) >= 2000 &&
349+ !gt_to_xe(gt)->info.force_execlist)
349350 xe_gt_err(gt, "Slice/Subslice counts missing from hwconfig table; using typical fallback values\n");
350351
351352 if (gt_to_xe(gt)->info.platform == XE_PVC)
drivers/gpu/drm/xe/xe_gt_pagefault.c+10−3
@@ -240,7 +240,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) · 239 unmodified lines
240240 atomic = access_is_atomic(pf->access_type);
241241
242242 if (xe_vma_is_cpu_addr_mirror(vma))
243− err = xe_svm_handle_pagefault(vm, vma, gt_to_tile(gt),
243+ err = xe_svm_handle_pagefault(vm, vma, gt,
244244 pf->page_addr, atomic);
245245 else
246246 err = handle_vma_pagefault(gt, vma, atomic);
@@ -435,9 +435,16 @@ static int xe_alloc_pf_queue(struct xe_gt *gt, struct pf_queue *pf_queue) · 188 unmodified lines
435435 num_eus = bitmap_weight(gt->fuse_topo.eu_mask_per_dss,
436436 XE_MAX_EU_FUSE_BITS) * num_dss;
437437
438− /* user can issue separate page faults per EU and per CS */
438+ /*
439+ * user can issue separate page faults per EU and per CS
440+ *
441+ * XXX: Multiplier required as compute UMD are getting PF queue errors
442+ * without it. Follow on why this multiplier is required.
443+ */
444+#define PF_MULTIPLIER 8
439445 pf_queue->num_dw =
440− (num_eus + XE_NUM_HW_ENGINES) * PF_MSG_LEN_DW;
446+ (num_eus + XE_NUM_HW_ENGINES) * PF_MSG_LEN_DW * PF_MULTIPLIER;
447+#undef PF_MULTIPLIER
441448
442449 pf_queue->gt = gt;
443450 pf_queue->data = devm_kcalloc(xe->drm.dev, pf_queue->num_dw,
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c+14−6
@@ -1444,15 +1444,23 @@ static int pf_provision_vf_lmem(struct xe_gt *gt, unsigned int vfid, u64 size) · 1443 unmodified lines
14441444 return 0;
14451445
14461446 xe_gt_assert(gt, pf_get_lmem_alignment(gt) == SZ_2M);
1447− bo = xe_bo_create_pin_map(xe, tile, NULL,
1448− ALIGN(size, PAGE_SIZE),
1449− ttm_bo_type_kernel,
1450− XE_BO_FLAG_VRAM_IF_DGFX(tile) |
1451− XE_BO_FLAG_NEEDS_2M |
1452− XE_BO_FLAG_PINNED);
1447+ bo = xe_bo_create_locked(xe, tile, NULL,
1448+ ALIGN(size, PAGE_SIZE),
1449+ ttm_bo_type_kernel,
1450+ XE_BO_FLAG_VRAM_IF_DGFX(tile) |
1451+ XE_BO_FLAG_NEEDS_2M |
1452+ XE_BO_FLAG_PINNED |
1453+ XE_BO_FLAG_PINNED_LATE_RESTORE);
14531454 if (IS_ERR(bo))
14541455 return PTR_ERR(bo);
14551456
1457+ err = xe_bo_pin(bo);
1458+ xe_bo_unlock(bo);
1459+ if (unlikely(err)) {
1460+ xe_bo_put(bo);
1461+ return err;
1462+ }
1463+
14561464 config->lmem_obj = bo;
14571465
14581466 if (xe_device_has_lmtt(xe)) {
drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c+49−17
@@ -51,26 +51,17 @@ static unsigned int extract_vfid(struct dentry *d) · 50 unmodified lines
5151 * /sys/kernel/debug/dri/0/
5252 * ├── gt0
5353 * │   ├── pf
54− * │   │   ├── ggtt_available
55− * │   │   ├── ggtt_provisioned
5654 * │   │   ├── contexts_provisioned
5755 * │   │   ├── doorbells_provisioned
5856 * │   │   ├── runtime_registers
5957 * │   │   ├── negotiated_versions
6058 * │   │   ├── adverse_events
59+ * ├── gt1
60+ * │   ├── pf
61+ * │   │   ├── ...
6162 */
6263
6364 static const struct drm_info_list pf_info[] = {
64− {
65− "ggtt_available",
66− .show = xe_gt_debugfs_simple_show,
67− .data = xe_gt_sriov_pf_config_print_available_ggtt,
68− },
69− {
70− "ggtt_provisioned",
71− .show = xe_gt_debugfs_simple_show,
72− .data = xe_gt_sriov_pf_config_print_ggtt,
73− },
7465 {
7566 "contexts_provisioned",
7667 .show = xe_gt_debugfs_simple_show,
@@ -81,11 +72,6 @@ static const struct drm_info_list pf_info[] = { · 4 unmodified lines
8172 .show = xe_gt_debugfs_simple_show,
8273 .data = xe_gt_sriov_pf_config_print_dbs,
8374 },
84− {
85− "lmem_provisioned",
86− .show = xe_gt_debugfs_simple_show,
87− .data = xe_gt_sriov_pf_config_print_lmem,
88− },
8975 {
9076 "runtime_registers",
9177 .show = xe_gt_debugfs_simple_show,
@@ -103,6 +89,42 @@ static const struct drm_info_list pf_info[] = { · 11 unmodified lines
10389 },
10490 };
10591
92+/*
93+ * /sys/kernel/debug/dri/0/
94+ * ├── gt0
95+ * │   ├── pf
96+ * │   │   ├── ggtt_available
97+ * │   │   ├── ggtt_provisioned
98+ */
99+
100+static const struct drm_info_list pf_ggtt_info[] = {
101+ {
102+ "ggtt_available",
103+ .show = xe_gt_debugfs_simple_show,
104+ .data = xe_gt_sriov_pf_config_print_available_ggtt,
105+ },
106+ {
107+ "ggtt_provisioned",
108+ .show = xe_gt_debugfs_simple_show,
109+ .data = xe_gt_sriov_pf_config_print_ggtt,
110+ },
111+};
112+
113+/*
114+ * /sys/kernel/debug/dri/0/
115+ * ├── gt0
116+ * │   ├── pf
117+ * │   │   ├── lmem_provisioned
118+ */
119+
120+static const struct drm_info_list pf_lmem_info[] = {
121+ {
122+ "lmem_provisioned",
123+ .show = xe_gt_debugfs_simple_show,
124+ .data = xe_gt_sriov_pf_config_print_lmem,
125+ },
126+};
127+
106128 /*
107129 * /sys/kernel/debug/dri/0/
108130 * ├── gt0
@@ -532,6 +554,16 @@ void xe_gt_sriov_pf_debugfs_register(struct xe_gt *gt, struct dentry *root) · 423 unmodified lines
532554 pfdentry->d_inode->i_private = gt;
533555
534556 drm_debugfs_create_files(pf_info, ARRAY_SIZE(pf_info), pfdentry, minor);
557+ if (!xe_gt_is_media_type(gt)) {
558+ drm_debugfs_create_files(pf_ggtt_info,
559+ ARRAY_SIZE(pf_ggtt_info),
560+ pfdentry, minor);
561+ if (IS_DGFX(gt_to_xe(gt)))
562+ drm_debugfs_create_files(pf_lmem_info,
563+ ARRAY_SIZE(pf_lmem_info),
564+ pfdentry, minor);
565+ }
566+
535567 pf_add_policy_attrs(gt, pfdentry);
536568 pf_add_config_attrs(gt, pfdentry, PFID);
537569
drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c+0−6
@@ -112,7 +112,6 @@ static const struct xe_reg tgl_runtime_regs[] = { · 111 unmodified lines
112112 XELP_GT_SLICE_ENABLE, /* _MMIO(0x9138) */
113113 XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
114114 GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
115− CTC_MODE, /* _MMIO(0xa26c) */
116115 HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
117116 };
118117
@@ -124,7 +123,6 @@ static const struct xe_reg ats_m_runtime_regs[] = { · 5 unmodified lines
124123 XELP_GT_GEOMETRY_DSS_ENABLE, /* _MMIO(0x913c) */
125124 GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
126125 XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
127− CTC_MODE, /* _MMIO(0xa26c) */
128126 HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
129127 };
130128
@@ -136,7 +134,6 @@ static const struct xe_reg pvc_runtime_regs[] = { · 5 unmodified lines
136134 GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
137135 XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
138136 XEHPC_GT_COMPUTE_DSS_ENABLE_EXT,/* _MMIO(0x9148) */
139− CTC_MODE, /* _MMIO(0xA26C) */
140137 HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
141138 };
142139
@@ -150,7 +147,6 @@ static const struct xe_reg ver_1270_runtime_regs[] = { · 7 unmodified lines
150147 GT_VEBOX_VDBOX_DISABLE, /* _MMIO(0x9140) */
151148 XEHP_GT_COMPUTE_DSS_ENABLE, /* _MMIO(0x9144) */
152149 XEHPC_GT_COMPUTE_DSS_ENABLE_EXT,/* _MMIO(0x9148) */
153− CTC_MODE, /* _MMIO(0xa26c) */
154150 HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
155151 };
156152
@@ -167,7 +163,6 @@ static const struct xe_reg ver_2000_runtime_regs[] = { · 10 unmodified lines
167163 XE2_GT_COMPUTE_DSS_2, /* _MMIO(0x914c) */
168164 XE2_GT_GEOMETRY_DSS_1, /* _MMIO(0x9150) */
169165 XE2_GT_GEOMETRY_DSS_2, /* _MMIO(0x9154) */
170− CTC_MODE, /* _MMIO(0xa26c) */
171166 HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
172167 };
173168
@@ -185,7 +180,6 @@ static const struct xe_reg ver_3000_runtime_regs[] = { · 11 unmodified lines
185180 XE2_GT_COMPUTE_DSS_2, /* _MMIO(0x914c) */
186181 XE2_GT_GEOMETRY_DSS_1, /* _MMIO(0x9150) */
187182 XE2_GT_GEOMETRY_DSS_2, /* _MMIO(0x9154) */
188− CTC_MODE, /* _MMIO(0xa26c) */
189183 HUC_KERNEL_LOAD_INFO, /* _MMIO(0xc1dc) */
190184 };
191185
drivers/gpu/drm/xe/xe_gt_stats.c+1−0
@@ -27,6 +27,7 @@ void xe_gt_stats_incr(struct xe_gt *gt, const enum xe_gt_stats_id id, int incr) · 26 unmodified lines
2727 }
2828
2929 static const char *const stat_description[__XE_GT_STATS_NUM_IDS] = {
30+ "svm_pagefault_count",
3031 "tlb_inval_count",
3132 "vma_pagefault_count",
3233 "vma_pagefault_kb",
drivers/gpu/drm/xe/xe_gt_stats_types.h+1−0
@@ -7,6 +7,7 @@ · 6 unmodified lines
77 #define _XE_GT_STATS_TYPES_H_
88
99 enum xe_gt_stats_id {
10+ XE_GT_STATS_ID_SVM_PAGEFAULT_COUNT,
1011 XE_GT_STATS_ID_TLB_INVAL,
1112 XE_GT_STATS_ID_VMA_PAGEFAULT_COUNT,
1213 XE_GT_STATS_ID_VMA_PAGEFAULT_KB,
drivers/gpu/drm/xe/xe_guc.c+2−1
@@ -483,7 +483,8 @@ static int guc_g2g_alloc(struct xe_guc *guc) · 482 unmodified lines
483483 XE_BO_FLAG_VRAM_IF_DGFX(tile) |
484484 XE_BO_FLAG_GGTT |
485485 XE_BO_FLAG_GGTT_ALL |
486− XE_BO_FLAG_GGTT_INVALIDATE);
486+ XE_BO_FLAG_GGTT_INVALIDATE |
487+ XE_BO_FLAG_PINNED_NORESTORE);
487488 if (IS_ERR(bo))
488489 return PTR_ERR(bo);
489490
drivers/gpu/drm/xe/xe_guc_ads.c+54−33
@@ -376,6 +376,11 @@ static void guc_waklv_init(struct xe_guc_ads *ads) · 375 unmodified lines
376376 GUC_WORKAROUND_KLV_ID_BACK_TO_BACK_RCS_ENGINE_RESET,
377377 &offset, &remain);
378378
379+ if (GUC_FIRMWARE_VER(&gt->uc.guc) >= MAKE_GUC_VER(70, 44, 0) && XE_WA(gt, 16026508708))
380+ guc_waklv_enable_simple(ads,
381+ GUC_WA_KLV_RESET_BB_STACK_PTR_ON_VF_SWITCH,
382+ &offset, &remain);
383+
379384 size = guc_ads_waklv_size(ads) - remain;
380385 if (!size)
381386 return;
@@ -414,7 +419,8 @@ int xe_guc_ads_init(struct xe_guc_ads *ads) · 32 unmodified lines
414419 bo = xe_managed_bo_create_pin_map(xe, tile, guc_ads_size(ads) + MAX_GOLDEN_LRC_SIZE,
415420 XE_BO_FLAG_SYSTEM |
416421 XE_BO_FLAG_GGTT |
417− XE_BO_FLAG_GGTT_INVALIDATE);
422+ XE_BO_FLAG_GGTT_INVALIDATE |
423+ XE_BO_FLAG_PINNED_NORESTORE);
418424 if (IS_ERR(bo))
419425 return PTR_ERR(bo);
420426
@@ -490,24 +496,52 @@ static void fill_engine_enable_masks(struct xe_gt *gt, · 69 unmodified lines
490496 engine_enable_mask(gt, XE_ENGINE_CLASS_OTHER));
491497 }
492498
493−static void guc_prep_golden_lrc_null(struct xe_guc_ads *ads)
499+/*
500+ * Write the offsets corresponding to the golden LRCs. The actual data is
501+ * populated later by guc_golden_lrc_populate()
502+ */
503+static void guc_golden_lrc_init(struct xe_guc_ads *ads)
494504 {
495505 struct xe_device *xe = ads_to_xe(ads);
506+ struct xe_gt *gt = ads_to_gt(ads);
496507 struct iosys_map info_map = IOSYS_MAP_INIT_OFFSET(ads_to_map(ads),
497508 offsetof(struct __guc_ads_blob, system_info));
498− u8 guc_class;
509+ size_t alloc_size, real_size;
510+ u32 addr_ggtt, offset;
511+ int class;
512+
513+ offset = guc_ads_golden_lrc_offset(ads);
514+ addr_ggtt = xe_bo_ggtt_addr(ads->bo) + offset;
515+
516+ for (class = 0; class < XE_ENGINE_CLASS_MAX; ++class) {
517+ u8 guc_class;
518+
519+ guc_class = xe_engine_class_to_guc_class(class);
499520
500− for (guc_class = 0; guc_class <= GUC_MAX_ENGINE_CLASSES; ++guc_class) {
501521 if (!info_map_read(xe, &info_map,
502522 engine_enabled_masks[guc_class]))
503523 continue;
504524
525+ real_size = xe_gt_lrc_size(gt, class);
526+ alloc_size = PAGE_ALIGN(real_size);
527+
528+ /*
529+ * This interface is slightly confusing. We need to pass the
530+ * base address of the full golden context and the size of just
531+ * the engine state, which is the section of the context image
532+ * that starts after the execlists LRC registers. This is
533+ * required to allow the GuC to restore just the engine state
534+ * when a watchdog reset occurs.
535+ * We calculate the engine state size by removing the size of
536+ * what comes before it in the context image (which is identical
537+ * on all engines).
538+ */
505539 ads_blob_write(ads, ads.eng_state_size[guc_class],
506− guc_ads_golden_lrc_size(ads) -
507− xe_lrc_skip_size(xe));
540+ real_size - xe_lrc_skip_size(xe));
508541 ads_blob_write(ads, ads.golden_context_lrca[guc_class],
509− xe_bo_ggtt_addr(ads->bo) +
510− guc_ads_golden_lrc_offset(ads));
542+ addr_ggtt);
543+
544+ addr_ggtt += alloc_size;
511545 }
512546 }
513547
@@ -682,8 +716,8 @@ static int guc_capture_prep_lists(struct xe_guc_ads *ads) · 168 unmodified lines
682716 }
683717
684718 if (ads->capture_size != PAGE_ALIGN(total_size))
685− xe_gt_dbg(gt, "ADS capture alloc size changed from %d to %d\n",
686− ads->capture_size, PAGE_ALIGN(total_size));
719+ xe_gt_dbg(gt, "Updated ADS capture size %d (was %d)\n",
720+ PAGE_ALIGN(total_size), ads->capture_size);
687721 return PAGE_ALIGN(total_size);
688722 }
689723
@@ -857,7 +891,7 @@ void xe_guc_ads_populate_minimal(struct xe_guc_ads *ads) · 167 unmodified lines
857891
858892 xe_map_memset(ads_to_xe(ads), ads_to_map(ads), 0, 0, ads->bo->size);
859893 guc_policies_init(ads);
860− guc_prep_golden_lrc_null(ads);
894+ guc_golden_lrc_init(ads);
861895 guc_mapping_table_init_invalid(gt, &info_map);
862896 guc_doorbell_init(ads);
863897
@@ -883,7 +917,7 @@ void xe_guc_ads_populate(struct xe_guc_ads *ads) · 19 unmodified lines
883917 guc_policies_init(ads);
884918 fill_engine_enable_masks(gt, &info_map);
885919 guc_mmio_reg_state_init(ads);
886− guc_prep_golden_lrc_null(ads);
920+ guc_golden_lrc_init(ads);
887921 guc_mapping_table_init(gt, &info_map);
888922 guc_capture_prep_lists(ads);
889923 guc_doorbell_init(ads);
@@ -903,18 +937,22 @@ void xe_guc_ads_populate(struct xe_guc_ads *ads) · 13 unmodified lines
903937 guc_ads_private_data_offset(ads));
904938 }
905939
906−static void guc_populate_golden_lrc(struct xe_guc_ads *ads)
940+/*
941+ * After the golden LRC's are recorded for each engine class by the first
942+ * submission, copy them to the ADS, as initialized earlier by
943+ * guc_golden_lrc_init().
944+ */
945+static void guc_golden_lrc_populate(struct xe_guc_ads *ads)
907946 {
908947 struct xe_device *xe = ads_to_xe(ads);
909948 struct xe_gt *gt = ads_to_gt(ads);
910949 struct iosys_map info_map = IOSYS_MAP_INIT_OFFSET(ads_to_map(ads),
911950 offsetof(struct __guc_ads_blob, system_info));
912951 size_t total_size = 0, alloc_size, real_size;
913− u32 addr_ggtt, offset;
952+ u32 offset;
914953 int class;
915954
916955 offset = guc_ads_golden_lrc_offset(ads);
917− addr_ggtt = xe_bo_ggtt_addr(ads->bo) + offset;
918956
919957 for (class = 0; class < XE_ENGINE_CLASS_MAX; ++class) {
920958 u8 guc_class;
@@ -931,26 +969,9 @@ static void guc_populate_golden_lrc(struct xe_guc_ads *ads) · 10 unmodified lines
931969 alloc_size = PAGE_ALIGN(real_size);
932970 total_size += alloc_size;
933971
934− /*
935− * This interface is slightly confusing. We need to pass the
936− * base address of the full golden context and the size of just
937− * the engine state, which is the section of the context image
938− * that starts after the execlists LRC registers. This is
939− * required to allow the GuC to restore just the engine state
940− * when a watchdog reset occurs.
941− * We calculate the engine state size by removing the size of
942− * what comes before it in the context image (which is identical
943− * on all engines).
944− */
945− ads_blob_write(ads, ads.eng_state_size[guc_class],
946− real_size - xe_lrc_skip_size(xe));
947− ads_blob_write(ads, ads.golden_context_lrca[guc_class],
948− addr_ggtt);
949−
950972 xe_map_memcpy_to(xe, ads_to_map(ads), offset,
951973 gt->default_lrc[class], real_size);
952974
953− addr_ggtt += alloc_size;
954975 offset += alloc_size;
955976 }
956977
@@ -959,7 +980,7 @@ static void guc_populate_golden_lrc(struct xe_guc_ads *ads) · 2 unmodified lines
959980
960981 void xe_guc_ads_populate_post_load(struct xe_guc_ads *ads)
961982 {
962− guc_populate_golden_lrc(ads);
983+ guc_golden_lrc_populate(ads);
963984 }
964985
965986 static int guc_ads_action_update_policies(struct xe_guc_ads *ads, u32 policy_offset)
drivers/gpu/drm/xe/xe_guc_ct.c+3−2
@@ -238,7 +238,8 @@ int xe_guc_ct_init(struct xe_guc_ct *ct) · 237 unmodified lines
238238 bo = xe_managed_bo_create_pin_map(xe, tile, guc_ct_size(),
239239 XE_BO_FLAG_SYSTEM |
240240 XE_BO_FLAG_GGTT |
241− XE_BO_FLAG_GGTT_INVALIDATE);
241+ XE_BO_FLAG_GGTT_INVALIDATE |
242+ XE_BO_FLAG_PINNED_NORESTORE);
242243 if (IS_ERR(bo))
243244 return PTR_ERR(bo);
244245
@@ -1828,10 +1829,10 @@ static void ct_dead_print(struct xe_dead_ct *dead) · 1583 unmodified lines
18281829 return;
18291830 }
18301831
1831− drm_printf(&lp, "CTB is dead - reason=0x%X\n", dead->reason);
18321832
18331833 /* Can't generate a genuine core dump at this point, so just do the good bits */
18341834 drm_puts(&lp, "**** Xe Device Coredump ****\n");
1835+ drm_printf(&lp, "Reason: CTB is dead - 0x%X\n", dead->reason);
18351836 xe_device_snapshot_print(xe, &lp);
18361837
18371838 drm_printf(&lp, "**** GT #%d ****\n", gt->info.id);
drivers/gpu/drm/xe/xe_guc_debugfs.c+83−65
@@ -17,101 +17,119 @@ · 16 unmodified lines
1717 #include "xe_macros.h"
1818 #include "xe_pm.h"
1919
20−static struct xe_guc *node_to_guc(struct drm_info_node *node)
21−{
22− return node->info_ent->data;
23−}
24−
25−static int guc_info(struct seq_file *m, void *data)
20+/*
21+ * guc_debugfs_show - A show callback for struct drm_info_list
22+ * @m: the &seq_file
23+ * @data: data used by the drm debugfs helpers
24+ *
25+ * This callback can be used in struct drm_info_list to describe debugfs
26+ * files that are &xe_guc specific in similar way how we handle &xe_gt
27+ * specific files using &xe_gt_debugfs_simple_show.
28+ *
29+ * It is assumed that those debugfs files will be created on directory entry
30+ * which grandparent struct dentry d_inode->i_private points to &xe_gt.
31+ *
32+ * /sys/kernel/debug/dri/0/
33+ * ├── gt0 # dent->d_parent->d_parent (d_inode->i_private == gt)
34+ * │   ├── uc # dent->d_parent
35+ * │   │   ├── guc_info # dent
36+ * │   │   ├── guc_...
37+ *
38+ * This function assumes that &m->private will be set to the &struct
39+ * drm_info_node corresponding to the instance of the info on a given &struct
40+ * drm_minor (see struct drm_info_list.show for details).
41+ *
42+ * This function also assumes that struct drm_info_list.data will point to the
43+ * function code that will actually print a file content::
44+ *
45+ * int (*print)(struct xe_guc *, struct drm_printer *)
46+ *
47+ * Example::
48+ *
49+ * int foo(struct xe_guc *guc, struct drm_printer *p)
50+ * {
51+ * drm_printf(p, "enabled %d\n", guc->submission_state.enabled);
52+ * return 0;
53+ * }
54+ *
55+ * static const struct drm_info_list bar[] = {
56+ * { name = "foo", .show = guc_debugfs_show, .data = foo },
57+ * };
58+ *
59+ * parent = debugfs_create_dir("uc", gtdir);
60+ * drm_debugfs_create_files(bar, ARRAY_SIZE(bar), parent, minor);
61+ *
62+ * Return: 0 on success or a negative error code on failure.
63+ */
64+static int guc_debugfs_show(struct seq_file *m, void *data)
2665 {
27− struct xe_guc *guc = node_to_guc(m->private);
28− struct xe_device *xe = guc_to_xe(guc);
2966 struct drm_printer p = drm_seq_file_printer(m);
67+ struct drm_info_node *node = m->private;
68+ struct dentry *parent = node->dent->d_parent;
69+ struct dentry *grandparent = parent->d_parent;
70+ struct xe_gt *gt = grandparent->d_inode->i_private;
71+ struct xe_device *xe = gt_to_xe(gt);
72+ int (*print)(struct xe_guc *, struct drm_printer *) = node->info_ent->data;
73+ int ret;
3074
3175 xe_pm_runtime_get(xe);
32− xe_guc_print_info(guc, &p);
76+ ret = print(&gt->uc.guc, &p);
3377 xe_pm_runtime_put(xe);
3478
35− return 0;
79+ return ret;
3680 }
3781
38−static int guc_log(struct seq_file *m, void *data)
82+static int guc_log(struct xe_guc *guc, struct drm_printer *p)
3983 {
40− struct xe_guc *guc = node_to_guc(m->private);
41− struct xe_device *xe = guc_to_xe(guc);
42− struct drm_printer p = drm_seq_file_printer(m);
43−
44− xe_pm_runtime_get(xe);
45− xe_guc_log_print(&guc->log, &p);
46− xe_pm_runtime_put(xe);
47−
84+ xe_guc_log_print(&guc->log, p);
4885 return 0;
4986 }
5087
51−static int guc_log_dmesg(struct seq_file *m, void *data)
88+static int guc_log_dmesg(struct xe_guc *guc, struct drm_printer *p)
5289 {
53− struct xe_guc *guc = node_to_guc(m->private);
54− struct xe_device *xe = guc_to_xe(guc);
55−
56− xe_pm_runtime_get(xe);
5790 xe_guc_log_print_dmesg(&guc->log);
58− xe_pm_runtime_put(xe);
59−
6091 return 0;
6192 }
6293
63−static int guc_ctb(struct seq_file *m, void *data)
94+static int guc_ctb(struct xe_guc *guc, struct drm_printer *p)
6495 {
65− struct xe_guc *guc = node_to_guc(m->private);
66− struct xe_device *xe = guc_to_xe(guc);
67− struct drm_printer p = drm_seq_file_printer(m);
68−
69− xe_pm_runtime_get(xe);
70− xe_guc_ct_print(&guc->ct, &p, true);
71− xe_pm_runtime_put(xe);
72−
96+ xe_guc_ct_print(&guc->ct, p, true);
7397 return 0;
7498 }
7599
76−static int guc_pc(struct seq_file *m, void *data)
100+static int guc_pc(struct xe_guc *guc, struct drm_printer *p)
77101 {
78− struct xe_guc *guc = node_to_guc(m->private);
79− struct xe_device *xe = guc_to_xe(guc);
80− struct drm_printer p = drm_seq_file_printer(m);
81−
82− xe_pm_runtime_get(xe);
83− xe_guc_pc_print(&guc->pc, &p);
84− xe_pm_runtime_put(xe);
85−
102+ xe_guc_pc_print(&guc->pc, p);
86103 return 0;
87104 }
88105
89−static const struct drm_info_list debugfs_list[] = {
90− {"guc_info", guc_info, 0},
91− {"guc_log", guc_log, 0},
92− {"guc_log_dmesg", guc_log_dmesg, 0},
93− {"guc_ctb", guc_ctb, 0},
94− {"guc_pc", guc_pc, 0},
106+/*
107+ * only for GuC debugfs files which can be safely used on the VF as well:
108+ * - without access to the GuC privileged registers
109+ * - without access to the PF specific GuC objects
110+ */
111+static const struct drm_info_list vf_safe_debugfs_list[] = {
112+ { "guc_info", .show = guc_debugfs_show, .data = xe_guc_print_info },
113+ { "guc_ctb", .show = guc_debugfs_show, .data = guc_ctb },
114+};
115+
116+/* everything else should be added here */
117+static const struct drm_info_list pf_only_debugfs_list[] = {
118+ { "guc_log", .show = guc_debugfs_show, .data = guc_log },
119+ { "guc_log_dmesg", .show = guc_debugfs_show, .data = guc_log_dmesg },
120+ { "guc_pc", .show = guc_debugfs_show, .data = guc_pc },
95121 };
96122
97123 void xe_guc_debugfs_register(struct xe_guc *guc, struct dentry *parent)
98124 {
99125 struct drm_minor *minor = guc_to_xe(guc)->drm.primary;
100− struct drm_info_list *local;
101− int i;
102126
103−#define DEBUGFS_SIZE (ARRAY_SIZE(debugfs_list) * sizeof(struct drm_info_list))
104− local = drmm_kmalloc(&guc_to_xe(guc)->drm, DEBUGFS_SIZE, GFP_KERNEL);
105− if (!local)
106− return;
107−
108− memcpy(local, debugfs_list, DEBUGFS_SIZE);
109−#undef DEBUGFS_SIZE
110−
111− for (i = 0; i < ARRAY_SIZE(debugfs_list); ++i)
112− local[i].data = guc;
113−
114− drm_debugfs_create_files(local,
115− ARRAY_SIZE(debugfs_list),
127+ drm_debugfs_create_files(vf_safe_debugfs_list,
128+ ARRAY_SIZE(vf_safe_debugfs_list),
116129 parent, minor);
130+
131+ if (!IS_SRIOV_VF(guc_to_xe(guc)))
132+ drm_debugfs_create_files(pf_only_debugfs_list,
133+ ARRAY_SIZE(pf_only_debugfs_list),
134+ parent, minor);
117135 }
drivers/gpu/drm/xe/xe_guc_engine_activity.c+172−28
@@ -17,36 +17,61 @@ · 16 unmodified lines
1717 #include "xe_hw_engine.h"
1818 #include "xe_map.h"
1919 #include "xe_mmio.h"
20+#include "xe_sriov_pf_helpers.h"
2021 #include "xe_trace_guc.h"
2122
2223 #define TOTAL_QUANTA 0x8000
2324
24−static struct iosys_map engine_activity_map(struct xe_guc *guc, struct xe_hw_engine *hwe)
25+static struct iosys_map engine_activity_map(struct xe_guc *guc, struct xe_hw_engine *hwe,
26+ unsigned int index)
2527 {
2628 struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
27− struct engine_activity_buffer *buffer = &engine_activity->device_buffer;
29+ struct engine_activity_buffer *buffer;
2830 u16 guc_class = xe_engine_class_to_guc_class(hwe->class);
2931 size_t offset;
3032
31− offset = offsetof(struct guc_engine_activity_data,
33+ if (engine_activity->num_functions) {
34+ buffer = &engine_activity->function_buffer;
35+ offset = sizeof(struct guc_engine_activity_data) * index;
36+ } else {
37+ buffer = &engine_activity->device_buffer;
38+ offset = 0;
39+ }
40+
41+ offset += offsetof(struct guc_engine_activity_data,
3242 engine_activity[guc_class][hwe->logical_instance]);
3343
3444 return IOSYS_MAP_INIT_OFFSET(&buffer->activity_bo->vmap, offset);
3545 }
3646
37−static struct iosys_map engine_metadata_map(struct xe_guc *guc)
47+static struct iosys_map engine_metadata_map(struct xe_guc *guc,
48+ unsigned int index)
3849 {
3950 struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
40− struct engine_activity_buffer *buffer = &engine_activity->device_buffer;
51+ struct engine_activity_buffer *buffer;
52+ size_t offset;
4153
42− return buffer->metadata_bo->vmap;
54+ if (engine_activity->num_functions) {
55+ buffer = &engine_activity->function_buffer;
56+ offset = sizeof(struct guc_engine_activity_metadata) * index;
57+ } else {
58+ buffer = &engine_activity->device_buffer;
59+ offset = 0;
60+ }
61+
62+ return IOSYS_MAP_INIT_OFFSET(&buffer->metadata_bo->vmap, offset);
4363 }
4464
4565 static int allocate_engine_activity_group(struct xe_guc *guc)
4666 {
4767 struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
4868 struct xe_device *xe = guc_to_xe(guc);
49− u32 num_activity_group = 1; /* Will be modified for VF */
69+ u32 num_activity_group;
70+
71+ /*
72+ * An additional activity group is allocated for PF
73+ */
74+ num_activity_group = IS_SRIOV_PF(xe) ? xe_sriov_pf_get_totalvfs(xe) + 1 : 1;
5075
5176 engine_activity->eag = drmm_kcalloc(&xe->drm, num_activity_group,
5277 sizeof(struct engine_activity_group), GFP_KERNEL);
@@ -60,10 +85,11 @@ static int allocate_engine_activity_group(struct xe_guc *guc) · 7 unmodified lines
6085 }
6186
6287 static int allocate_engine_activity_buffers(struct xe_guc *guc,
63− struct engine_activity_buffer *buffer)
88+ struct engine_activity_buffer *buffer,
89+ int count)
6490 {
65− u32 metadata_size = sizeof(struct guc_engine_activity_metadata);
66− u32 size = sizeof(struct guc_engine_activity_data);
91+ u32 metadata_size = sizeof(struct guc_engine_activity_metadata) * count;
92+ u32 size = sizeof(struct guc_engine_activity_data) * count;
6793 struct xe_gt *gt = guc_to_gt(guc);
6894 struct xe_tile *tile = gt_to_tile(gt);
6995 struct xe_bo *bo, *metadata_bo;
@@ -118,10 +144,11 @@ static bool is_engine_activity_supported(struct xe_guc *guc) · 48 unmodified lines
118144 return true;
119145 }
120146
121−static struct engine_activity *hw_engine_to_engine_activity(struct xe_hw_engine *hwe)
147+static struct engine_activity *hw_engine_to_engine_activity(struct xe_hw_engine *hwe,
148+ unsigned int index)
122149 {
123150 struct xe_guc *guc = &hwe->gt->uc.guc;
124− struct engine_activity_group *eag = &guc->engine_activity.eag[0];
151+ struct engine_activity_group *eag = &guc->engine_activity.eag[index];
125152 u16 guc_class = xe_engine_class_to_guc_class(hwe->class);
126153
127154 return &eag->engine[guc_class][hwe->logical_instance];
@@ -138,9 +165,10 @@ static u64 cpu_ns_to_guc_tsc_tick(ktime_t ns, u32 freq) · 10 unmodified lines
138165 #define read_metadata_record(xe_, map_, field_) \
139166 xe_map_rd_field(xe_, map_, 0, struct guc_engine_activity_metadata, field_)
140167
141−static u64 get_engine_active_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe)
168+static u64 get_engine_active_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe,
169+ unsigned int index)
142170 {
143− struct engine_activity *ea = hw_engine_to_engine_activity(hwe);
171+ struct engine_activity *ea = hw_engine_to_engine_activity(hwe, index);
144172 struct guc_engine_activity *cached_activity = &ea->activity;
145173 struct guc_engine_activity_metadata *cached_metadata = &ea->metadata;
146174 struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
@@ -151,8 +179,8 @@ static u64 get_engine_active_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe) · 4 unmodified lines
151179 u64 active_ticks, gpm_ts;
152180 u16 change_num;
153181
154− activity_map = engine_activity_map(guc, hwe);
155− metadata_map = engine_metadata_map(guc);
182+ activity_map = engine_activity_map(guc, hwe, index);
183+ metadata_map = engine_metadata_map(guc, index);
156184 global_change_num = read_metadata_record(xe, &metadata_map, global_change_num);
157185
158186 /* GuC has not initialized activity data yet, return 0 */
@@ -194,9 +222,9 @@ update: · 35 unmodified lines
194222 return ea->total + ea->active;
195223 }
196224
197−static u64 get_engine_total_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe)
225+static u64 get_engine_total_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe, unsigned int index)
198226 {
199− struct engine_activity *ea = hw_engine_to_engine_activity(hwe);
227+ struct engine_activity *ea = hw_engine_to_engine_activity(hwe, index);
200228 struct guc_engine_activity_metadata *cached_metadata = &ea->metadata;
201229 struct guc_engine_activity *cached_activity = &ea->activity;
202230 struct iosys_map activity_map, metadata_map;
@@ -205,8 +233,8 @@ static u64 get_engine_total_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe) · 2 unmodified lines
205233 u64 numerator;
206234 u16 quanta_ratio;
207235
208− activity_map = engine_activity_map(guc, hwe);
209− metadata_map = engine_metadata_map(guc);
236+ activity_map = engine_activity_map(guc, hwe, index);
237+ metadata_map = engine_metadata_map(guc, index);
210238
211239 if (!cached_metadata->guc_tsc_frequency_hz)
212240 cached_metadata->guc_tsc_frequency_hz = read_metadata_record(xe, &metadata_map,
@@ -245,10 +273,35 @@ static int enable_engine_activity_stats(struct xe_guc *guc) · 32 unmodified lines
245273 return xe_guc_ct_send_block(&guc->ct, action, ARRAY_SIZE(action));
246274 }
247275
248−static void engine_activity_set_cpu_ts(struct xe_guc *guc)
276+static int enable_function_engine_activity_stats(struct xe_guc *guc, bool enable)
249277 {
250278 struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
251− struct engine_activity_group *eag = &engine_activity->eag[0];
279+ u32 metadata_ggtt_addr = 0, ggtt_addr = 0, num_functions = 0;
280+ struct engine_activity_buffer *buffer = &engine_activity->function_buffer;
281+ u32 action[6];
282+ int len = 0;
283+
284+ if (enable) {
285+ metadata_ggtt_addr = xe_bo_ggtt_addr(buffer->metadata_bo);
286+ ggtt_addr = xe_bo_ggtt_addr(buffer->activity_bo);
287+ num_functions = engine_activity->num_functions;
288+ }
289+
290+ action[len++] = XE_GUC_ACTION_SET_FUNCTION_ENGINE_ACTIVITY_BUFFER;
291+ action[len++] = num_functions;
292+ action[len++] = metadata_ggtt_addr;
293+ action[len++] = 0;
294+ action[len++] = ggtt_addr;
295+ action[len++] = 0;
296+
297+ /* Blocking here to ensure the buffers are ready before reading them */
298+ return xe_guc_ct_send_block(&guc->ct, action, ARRAY_SIZE(action));
299+}
300+
301+static void engine_activity_set_cpu_ts(struct xe_guc *guc, unsigned int index)
302+{
303+ struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
304+ struct engine_activity_group *eag = &engine_activity->eag[index];
252305 int i, j;
253306
254307 for (i = 0; i < GUC_MAX_ENGINE_CLASSES; i++)
@@ -265,34 +318,106 @@ static u32 gpm_timestamp_shift(struct xe_gt *gt) · 10 unmodified lines
265318 return 3 - REG_FIELD_GET(RPM_CONFIG0_CTC_SHIFT_PARAMETER_MASK, reg);
266319 }
267320
321+static bool is_function_valid(struct xe_guc *guc, unsigned int fn_id)
322+{
323+ struct xe_device *xe = guc_to_xe(guc);
324+ struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
325+
326+ if (!IS_SRIOV_PF(xe) && fn_id)
327+ return false;
328+
329+ if (engine_activity->num_functions && fn_id >= engine_activity->num_functions)
330+ return false;
331+
332+ return true;
333+}
334+
335+static int engine_activity_disable_function_stats(struct xe_guc *guc)
336+{
337+ struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
338+ struct engine_activity_buffer *buffer = &engine_activity->function_buffer;
339+ int ret;
340+
341+ if (!engine_activity->num_functions)
342+ return 0;
343+
344+ ret = enable_function_engine_activity_stats(guc, false);
345+ if (ret)
346+ return ret;
347+
348+ free_engine_activity_buffers(buffer);
349+ engine_activity->num_functions = 0;
350+
351+ return 0;
352+}
353+
354+static int engine_activity_enable_function_stats(struct xe_guc *guc, int num_vfs)
355+{
356+ struct xe_guc_engine_activity *engine_activity = &guc->engine_activity;
357+ struct engine_activity_buffer *buffer = &engine_activity->function_buffer;
358+ int ret, i;
359+
360+ if (!num_vfs)
361+ return 0;
362+
363+ /* This includes 1 PF and num_vfs */
364+ engine_activity->num_functions = num_vfs + 1;
365+
366+ ret = allocate_engine_activity_buffers(guc, buffer, engine_activity->num_functions);
367+ if (ret)
368+ return ret;
369+
370+ ret = enable_function_engine_activity_stats(guc, true);
371+ if (ret) {
372+ free_engine_activity_buffers(buffer);
373+ engine_activity->num_functions = 0;
374+ return ret;
375+ }
376+
377+ for (i = 0; i < engine_activity->num_functions; i++)
378+ engine_activity_set_cpu_ts(guc, i + 1);
379+
380+ return 0;
381+}
382+
268383 /**
269384 * xe_guc_engine_activity_active_ticks - Get engine active ticks
270385 * @guc: The GuC object
271386 * @hwe: The hw_engine object
387+ * @fn_id: function id to report on
272388 *
273389 * Return: accumulated ticks @hwe was active since engine activity stats were enabled.
274390 */
275−u64 xe_guc_engine_activity_active_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe)
391+u64 xe_guc_engine_activity_active_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe,
392+ unsigned int fn_id)
276393 {
277394 if (!xe_guc_engine_activity_supported(guc))
278395 return 0;
279396
280− return get_engine_active_ticks(guc, hwe);
397+ if (!is_function_valid(guc, fn_id))
398+ return 0;
399+
400+ return get_engine_active_ticks(guc, hwe, fn_id);
281401 }
282402
283403 /**
284404 * xe_guc_engine_activity_total_ticks - Get engine total ticks
285405 * @guc: The GuC object
286406 * @hwe: The hw_engine object
407+ * @fn_id: function id to report on
287408 *
288409 * Return: accumulated quanta of ticks allocated for the engine
289410 */
290−u64 xe_guc_engine_activity_total_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe)
411+u64 xe_guc_engine_activity_total_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe,
412+ unsigned int fn_id)
291413 {
292414 if (!xe_guc_engine_activity_supported(guc))
293415 return 0;
294416
295− return get_engine_total_ticks(guc, hwe);
417+ if (!is_function_valid(guc, fn_id))
418+ return 0;
419+
420+ return get_engine_total_ticks(guc, hwe, fn_id);
296421 }
297422
298423 /**
@@ -310,6 +435,25 @@ bool xe_guc_engine_activity_supported(struct xe_guc *guc) · 11 unmodified lines
310435 return engine_activity->supported;
311436 }
312437
438+/**
439+ * xe_guc_engine_activity_function_stats - Enable/Disable per-function engine activity stats
440+ * @guc: The GuC object
441+ * @num_vfs: number of vfs
442+ * @enable: true to enable, false otherwise
443+ *
444+ * Return: 0 on success, negative error code otherwise
445+ */
446+int xe_guc_engine_activity_function_stats(struct xe_guc *guc, int num_vfs, bool enable)
447+{
448+ if (!xe_guc_engine_activity_supported(guc))
449+ return 0;
450+
451+ if (enable)
452+ return engine_activity_enable_function_stats(guc, num_vfs);
453+
454+ return engine_activity_disable_function_stats(guc);
455+}
456+
313457 /**
314458 * xe_guc_engine_activity_enable_stats - Enable engine activity stats
315459 * @guc: The GuC object
@@ -327,7 +471,7 @@ void xe_guc_engine_activity_enable_stats(struct xe_guc *guc) · 11 unmodified lines
327471 if (ret)
328472 xe_gt_err(guc_to_gt(guc), "failed to enable activity stats%d\n", ret);
329473 else
330− engine_activity_set_cpu_ts(guc);
474+ engine_activity_set_cpu_ts(guc, 0);
331475 }
332476
333477 static void engine_activity_fini(void *arg)
@@ -360,7 +504,7 @@ int xe_guc_engine_activity_init(struct xe_guc *guc) · 26 unmodified lines
360504 return ret;
361505 }
362506
363− ret = allocate_engine_activity_buffers(guc, &engine_activity->device_buffer);
507+ ret = allocate_engine_activity_buffers(guc, &engine_activity->device_buffer, 1);
364508 if (ret) {
365509 xe_gt_err(gt, "failed to allocate engine activity buffers (%pe)\n", ERR_PTR(ret));
366510 return ret;
drivers/gpu/drm/xe/xe_guc_engine_activity.h+5−2
@@ -14,6 +14,9 @@ struct xe_guc; · 13 unmodified lines
1414 int xe_guc_engine_activity_init(struct xe_guc *guc);
1515 bool xe_guc_engine_activity_supported(struct xe_guc *guc);
1616 void xe_guc_engine_activity_enable_stats(struct xe_guc *guc);
17−u64 xe_guc_engine_activity_active_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe);
18−u64 xe_guc_engine_activity_total_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe);
17+int xe_guc_engine_activity_function_stats(struct xe_guc *guc, int num_vfs, bool enable);
18+u64 xe_guc_engine_activity_active_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe,
19+ unsigned int fn_id);
20+u64 xe_guc_engine_activity_total_ticks(struct xe_guc *guc, struct xe_hw_engine *hwe,
21+ unsigned int fn_id);
1922 #endif
drivers/gpu/drm/xe/xe_guc_engine_activity_types.h+11−1
@@ -79,14 +79,24 @@ struct xe_guc_engine_activity { · 78 unmodified lines
7979 /** @num_activity_group: number of activity groups */
8080 u32 num_activity_group;
8181
82+ /** @num_functions: number of functions */
83+ u32 num_functions;
84+
8285 /** @supported: indicates support for engine activity stats */
8386 bool supported;
8487
85− /** @eag: holds the device level engine activity data */
88+ /**
89+ * @eag: holds the device level engine activity data in native mode.
90+ * In SRIOV mode, points to an array with entries which holds the engine
91+ * activity data for PF and VF's
92+ */
8693 struct engine_activity_group *eag;
8794
8895 /** @device_buffer: buffer object for global engine activity */
8996 struct engine_activity_buffer device_buffer;
97+
98+ /** @function_buffer: buffer object for per-function engine activity */
99+ struct engine_activity_buffer function_buffer;
90100 };
91101 #endif
92102
drivers/gpu/drm/xe/xe_guc_log.c+2−1
@@ -260,7 +260,8 @@ int xe_guc_log_init(struct xe_guc_log *log) · 259 unmodified lines
260260 bo = xe_managed_bo_create_pin_map(xe, tile, guc_log_size(),
261261 XE_BO_FLAG_SYSTEM |
262262 XE_BO_FLAG_GGTT |
263− XE_BO_FLAG_GGTT_INVALIDATE);
263+ XE_BO_FLAG_GGTT_INVALIDATE |
264+ XE_BO_FLAG_PINNED_NORESTORE);
264265 if (IS_ERR(bo))
265266 return PTR_ERR(bo);
266267
drivers/gpu/drm/xe/xe_guc_pc.c+17−1
@@ -461,6 +461,21 @@ static u32 get_cur_freq(struct xe_gt *gt) · 460 unmodified lines
461461 return decode_freq(freq);
462462 }
463463
464+/**
465+ * xe_guc_pc_get_cur_freq_fw - With fw held, get requested frequency
466+ * @pc: The GuC PC
467+ *
468+ * Returns: the requested frequency for that GT instance
469+ */
470+u32 xe_guc_pc_get_cur_freq_fw(struct xe_guc_pc *pc)
471+{
472+ struct xe_gt *gt = pc_to_gt(pc);
473+
474+ xe_force_wake_assert_held(gt_to_fw(gt), XE_FW_GT);
475+
476+ return get_cur_freq(gt);
477+}
478+
464479 /**
465480 * xe_guc_pc_get_cur_freq - Get Current requested frequency
466481 * @pc: The GuC PC
@@ -1170,7 +1185,8 @@ int xe_guc_pc_init(struct xe_guc_pc *pc) · 703 unmodified lines
11701185 bo = xe_managed_bo_create_pin_map(xe, tile, size,
11711186 XE_BO_FLAG_VRAM_IF_DGFX(tile) |
11721187 XE_BO_FLAG_GGTT |
1173− XE_BO_FLAG_GGTT_INVALIDATE);
1188+ XE_BO_FLAG_GGTT_INVALIDATE |
1189+ XE_BO_FLAG_PINNED_NORESTORE);
11741190 if (IS_ERR(bo))
11751191 return PTR_ERR(bo);
11761192
drivers/gpu/drm/xe/xe_guc_pc.h+1−0
@@ -22,6 +22,7 @@ void xe_guc_pc_print(struct xe_guc_pc *pc, struct drm_printer *p); · 21 unmodified lines
2222
2323 u32 xe_guc_pc_get_act_freq(struct xe_guc_pc *pc);
2424 int xe_guc_pc_get_cur_freq(struct xe_guc_pc *pc, u32 *freq);
25+u32 xe_guc_pc_get_cur_freq_fw(struct xe_guc_pc *pc);
2526 u32 xe_guc_pc_get_rp0_freq(struct xe_guc_pc *pc);
2627 u32 xe_guc_pc_get_rpa_freq(struct xe_guc_pc *pc);
2728 u32 xe_guc_pc_get_rpe_freq(struct xe_guc_pc *pc);
drivers/gpu/drm/xe/xe_guc_submit.c+9−0
@@ -300,6 +300,8 @@ int xe_guc_submit_init(struct xe_guc *guc, unsigned int num_ids) · 299 unmodified lines
300300
301301 primelockdep(guc);
302302
303+ guc->submission_state.initialized = true;
304+
303305 return drmm_add_action_or_reset(&xe->drm, guc_submit_fini, guc);
304306 }
305307
@@ -834,6 +836,13 @@ void xe_guc_submit_wedge(struct xe_guc *guc) · 528 unmodified lines
834836
835837 xe_gt_assert(guc_to_gt(guc), guc_to_xe(guc)->wedged.mode);
836838
839+ /*
840+ * If device is being wedged even before submission_state is
841+ * initialized, there's nothing to do here.
842+ */
843+ if (!guc->submission_state.initialized)
844+ return;
845+
837846 err = devm_add_action_or_reset(guc_to_xe(guc)->drm.dev,
838847 guc_submit_wedged_fini, guc);
839848 if (err) {
drivers/gpu/drm/xe/xe_guc_types.h+5−0
@@ -89,6 +89,11 @@ struct xe_guc { · 88 unmodified lines
8989 struct mutex lock;
9090 /** @submission_state.enabled: submission is enabled */
9191 bool enabled;
92+ /**
93+ * @submission_state.initialized: mark when submission state is
94+ * even initialized - before that not even the lock is valid
95+ */
96+ bool initialized;
9297 /** @submission_state.fini_wq: submit fini wait queue */
9398 wait_queue_head_t fini_wq;
9499 } submission_state;
drivers/gpu/drm/xe/xe_hmm.c+0−24
@@ -19,29 +19,6 @@ static u64 xe_npages_in_range(unsigned long start, unsigned long end) · 18 unmodified lines
1919 return (end - start) >> PAGE_SHIFT;
2020 }
2121
22−/**
23− * xe_mark_range_accessed() - mark a range is accessed, so core mm
24− * have such information for memory eviction or write back to
25− * hard disk
26− * @range: the range to mark
27− * @write: if write to this range, we mark pages in this range
28− * as dirty
29− */
30−static void xe_mark_range_accessed(struct hmm_range *range, bool write)
31−{
32− struct page *page;
33− u64 i, npages;
34−
35− npages = xe_npages_in_range(range->start, range->end);
36− for (i = 0; i < npages; i++) {
37− page = hmm_pfn_to_page(range->hmm_pfns[i]);
38− if (write)
39− set_page_dirty_lock(page);
40−
41− mark_page_accessed(page);
42− }
43−}
44−
4522 static int xe_alloc_sg(struct xe_device *xe, struct sg_table *st,
4623 struct hmm_range *range, struct rw_semaphore *notifier_sem)
4724 {
@@ -331,7 +308,6 @@ int xe_hmm_userptr_populate_range(struct xe_userptr_vma *uvma, · 283 unmodified lines
331308 if (ret)
332309 goto out_unlock;
333310
334− xe_mark_range_accessed(&hmm_range, write);
335311 userptr->sg = &userptr->sgt;
336312 xe_hmm_userptr_set_mapped(uvma);
337313 userptr->notifier_seq = hmm_range.notifier_seq;
drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c+1−0
@@ -605,6 +605,7 @@ err_object: · 604 unmodified lines
605605 kobject_put(kobj);
606606 return err;
607607 }
608+ALLOW_ERROR_INJECTION(xe_add_hw_engine_class_defaults, ERRNO); /* See xe_pci_probe() */
608609
609610
610611 static void hw_engine_class_sysfs_fini(void *arg)
drivers/gpu/drm/xe/xe_hwmon.c+124−1
@@ -5,6 +5,7 @@ · 4 unmodified lines
55
66 #include <linux/hwmon-sysfs.h>
77 #include <linux/hwmon.h>
8+#include <linux/jiffies.h>
89 #include <linux/types.h>
910 #include <linux/units.h>
1011
@@ -27,6 +28,7 @@ enum xe_hwmon_reg { · 16 unmodified lines
2728 REG_PKG_POWER_SKU_UNIT,
2829 REG_GT_PERF_STATUS,
2930 REG_PKG_ENERGY_STATUS,
31+ REG_FAN_SPEED,
3032 };
3133
3234 enum xe_hwmon_reg_operation {
@@ -42,6 +44,13 @@ enum xe_hwmon_channel { · 9 unmodified lines
4244 CHANNEL_MAX,
4345 };
4446
47+enum xe_fan_channel {
48+ FAN_1,
49+ FAN_2,
50+ FAN_3,
51+ FAN_MAX,
52+};
53+
4554 /*
4655 * SF_* - scale factors for particular quantities according to hwmon spec.
4756 */
@@ -61,6 +70,16 @@ struct xe_hwmon_energy_info { · 13 unmodified lines
6170 long accum_energy;
6271 };
6372
73+/**
74+ * struct xe_hwmon_fan_info - to cache previous fan reading
75+ */
76+struct xe_hwmon_fan_info {
77+ /** @reg_val_prev: previous fan reg val */
78+ u32 reg_val_prev;
79+ /** @time_prev: previous timestamp */
80+ u64 time_prev;
81+};
82+
6483 /**
6584 * struct xe_hwmon - xe hwmon data structure
6685 */
@@ -79,6 +98,8 @@ struct xe_hwmon { · 12 unmodified lines
7998 int scl_shift_time;
8099 /** @ei: Energy info for energyN_input */
81100 struct xe_hwmon_energy_info ei[CHANNEL_MAX];
101+ /** @fi: Fan info for fanN_input */
102+ struct xe_hwmon_fan_info fi[FAN_MAX];
82103 };
83104
84105 static struct xe_reg xe_hwmon_get_reg(struct xe_hwmon *hwmon, enum xe_hwmon_reg hwmon_reg,
@@ -144,6 +165,14 @@ static struct xe_reg xe_hwmon_get_reg(struct xe_hwmon *hwmon, enum xe_hwmon_reg · 59 unmodified lines
144165 return PCU_CR_PACKAGE_ENERGY_STATUS;
145166 }
146167 break;
168+ case REG_FAN_SPEED:
169+ if (channel == FAN_1)
170+ return BMG_FAN_1_SPEED;
171+ else if (channel == FAN_2)
172+ return BMG_FAN_2_SPEED;
173+ else if (channel == FAN_3)
174+ return BMG_FAN_3_SPEED;
175+ break;
147176 default:
148177 drm_warn(&xe->drm, "Unknown xe hwmon reg id: %d\n", hwmon_reg);
149178 break;
@@ -454,6 +483,7 @@ static const struct hwmon_channel_info * const hwmon_info[] = { · 304 unmodified lines
454483 HWMON_CHANNEL_INFO(curr, HWMON_C_LABEL, HWMON_C_CRIT | HWMON_C_LABEL),
455484 HWMON_CHANNEL_INFO(in, HWMON_I_INPUT | HWMON_I_LABEL, HWMON_I_INPUT | HWMON_I_LABEL),
456485 HWMON_CHANNEL_INFO(energy, HWMON_E_INPUT | HWMON_E_LABEL, HWMON_E_INPUT | HWMON_E_LABEL),
486+ HWMON_CHANNEL_INFO(fan, HWMON_F_INPUT, HWMON_F_INPUT, HWMON_F_INPUT),
457487 NULL
458488 };
459489
@@ -480,6 +510,19 @@ static int xe_hwmon_pcode_write_i1(const struct xe_hwmon *hwmon, u32 uval) · 20 unmodified lines
480510 (uval & POWER_SETUP_I1_DATA_MASK));
481511 }
482512
513+static int xe_hwmon_pcode_read_fan_control(const struct xe_hwmon *hwmon, u32 subcmd, u32 *uval)
514+{
515+ struct xe_tile *root_tile = xe_device_get_root_tile(hwmon->xe);
516+
517+ /* Platforms that don't return correct value */
518+ if (hwmon->xe->info.platform == XE_DG2 && subcmd == FSC_READ_NUM_FANS) {
519+ *uval = 2;
520+ return 0;
521+ }
522+
523+ return xe_pcode_read(root_tile, PCODE_MBOX(FAN_SPEED_CONTROL, subcmd, 0), uval, NULL);
524+}
525+
483526 static int xe_hwmon_power_curr_crit_read(struct xe_hwmon *hwmon, int channel,
484527 long *value, u32 scale_factor)
485528 {
@@ -705,6 +748,75 @@ xe_hwmon_energy_read(struct xe_hwmon *hwmon, u32 attr, int channel, long *val) · 219 unmodified lines
705748 }
706749 }
707750
751+static umode_t
752+xe_hwmon_fan_is_visible(struct xe_hwmon *hwmon, u32 attr, int channel)
753+{
754+ u32 uval;
755+
756+ if (!hwmon->xe->info.has_fan_control)
757+ return 0;
758+
759+ switch (attr) {
760+ case hwmon_fan_input:
761+ if (xe_hwmon_pcode_read_fan_control(hwmon, FSC_READ_NUM_FANS, &uval))
762+ return 0;
763+
764+ return channel < uval ? 0444 : 0;
765+ default:
766+ return 0;
767+ }
768+}
769+
770+static int
771+xe_hwmon_fan_input_read(struct xe_hwmon *hwmon, int channel, long *val)
772+{
773+ struct xe_mmio *mmio = xe_root_tile_mmio(hwmon->xe);
774+ struct xe_hwmon_fan_info *fi = &hwmon->fi[channel];
775+ u64 rotations, time_now, time;
776+ u32 reg_val;
777+ int ret = 0;
778+
779+ mutex_lock(&hwmon->hwmon_lock);
780+
781+ reg_val = xe_mmio_read32(mmio, xe_hwmon_get_reg(hwmon, REG_FAN_SPEED, channel));
782+ time_now = get_jiffies_64();
783+
784+ /*
785+ * HW register value is accumulated count of pulses from PWM fan with the scale
786+ * of 2 pulses per rotation.
787+ */
788+ rotations = (reg_val - fi->reg_val_prev) / 2;
789+
790+ time = jiffies_delta_to_msecs(time_now - fi->time_prev);
791+ if (unlikely(!time)) {
792+ ret = -EAGAIN;
793+ goto unlock;
794+ }
795+
796+ /*
797+ * Calculate fan speed in RPM by time averaging two subsequent readings in minutes.
798+ * RPM = number of rotations * msecs per minute / time in msecs
799+ */
800+ *val = DIV_ROUND_UP_ULL(rotations * (MSEC_PER_SEC * 60), time);
801+
802+ fi->reg_val_prev = reg_val;
803+ fi->time_prev = time_now;
804+unlock:
805+ mutex_unlock(&hwmon->hwmon_lock);
806+ return ret;
807+}
808+
809+static int
810+xe_hwmon_fan_read(struct xe_hwmon *hwmon, u32 attr, int channel, long *val)
811+{
812+ switch (attr) {
813+ case hwmon_fan_input:
814+ return xe_hwmon_fan_input_read(hwmon, channel, val);
815+ default:
816+ return -EOPNOTSUPP;
817+ }
818+}
819+
708820 static umode_t
709821 xe_hwmon_is_visible(const void *drvdata, enum hwmon_sensor_types type,
710822 u32 attr, int channel)
@@ -730,6 +842,9 @@ xe_hwmon_is_visible(const void *drvdata, enum hwmon_sensor_types type, · 19 unmodified lines
730842 case hwmon_energy:
731843 ret = xe_hwmon_energy_is_visible(hwmon, attr, channel);
732844 break;
845+ case hwmon_fan:
846+ ret = xe_hwmon_fan_is_visible(hwmon, attr, channel);
847+ break;
733848 default:
734849 ret = 0;
735850 break;
@@ -765,6 +880,9 @@ xe_hwmon_read(struct device *dev, enum hwmon_sensor_types type, u32 attr, · 29 unmodified lines
765880 case hwmon_energy:
766881 ret = xe_hwmon_energy_read(hwmon, attr, channel, val);
767882 break;
883+ case hwmon_fan:
884+ ret = xe_hwmon_fan_read(hwmon, attr, channel, val);
885+ break;
768886 default:
769887 ret = -EOPNOTSUPP;
770888 break;
@@ -842,7 +960,7 @@ static void · 71 unmodified lines
842960 xe_hwmon_get_preregistration_info(struct xe_hwmon *hwmon)
843961 {
844962 struct xe_mmio *mmio = xe_root_tile_mmio(hwmon->xe);
845− long energy;
963+ long energy, fan_speed;
846964 u64 val_sku_unit = 0;
847965 int channel;
848966 struct xe_reg pkg_power_sku_unit;
@@ -866,6 +984,11 @@ xe_hwmon_get_preregistration_info(struct xe_hwmon *hwmon) · 17 unmodified lines
866984 for (channel = 0; channel < CHANNEL_MAX; channel++)
867985 if (xe_hwmon_is_visible(hwmon, hwmon_energy, hwmon_energy_input, channel))
868986 xe_hwmon_energy_get(hwmon, channel, &energy);
987+
988+ /* Initialize 'struct xe_hwmon_fan_info' with initial fan register reading. */
989+ for (channel = 0; channel < FAN_MAX; channel++)
990+ if (xe_hwmon_is_visible(hwmon, hwmon_fan, hwmon_fan_input, channel))
991+ xe_hwmon_fan_input_read(hwmon, channel, &fan_speed);
869992 }
870993
871994 static void xe_hwmon_mutex_destroy(void *arg)
drivers/gpu/drm/xe/xe_lmtt.c+1−1
@@ -71,7 +71,7 @@ static struct xe_lmtt_pt *lmtt_pt_alloc(struct xe_lmtt *lmtt, unsigned int level · 70 unmodified lines
7171 lmtt->ops->lmtt_pte_num(level)),
7272 ttm_bo_type_kernel,
7373 XE_BO_FLAG_VRAM_IF_DGFX(lmtt_to_tile(lmtt)) |
74− XE_BO_FLAG_NEEDS_64K | XE_BO_FLAG_PINNED);
74+ XE_BO_FLAG_NEEDS_64K);
7575 if (IS_ERR(bo)) {
7676 err = PTR_ERR(bo);
7777 goto out_free_pt;
drivers/gpu/drm/xe/xe_lrc.c+18−10
@@ -37,6 +37,7 @@ · 36 unmodified lines
3737 #define LRC_ENGINE_CLASS GENMASK_ULL(63, 61)
3838 #define LRC_ENGINE_INSTANCE GENMASK_ULL(53, 48)
3939
40+#define LRC_PPHWSP_SIZE SZ_4K
4041 #define LRC_INDIRECT_RING_STATE_SIZE SZ_4K
4142
4243 static struct xe_device *
@@ -50,19 +51,22 @@ size_t xe_gt_lrc_size(struct xe_gt *gt, enum xe_engine_class class) · 7 unmodified lines
5051 struct xe_device *xe = gt_to_xe(gt);
5152 size_t size;
5253
54+ /* Per-process HW status page (PPHWSP) */
55+ size = LRC_PPHWSP_SIZE;
56+
57+ /* Engine context image */
5358 switch (class) {
5459 case XE_ENGINE_CLASS_RENDER:
5560 if (GRAPHICS_VER(xe) >= 20)
56− size = 4 * SZ_4K;
61+ size += 3 * SZ_4K;
5762 else
58− size = 14 * SZ_4K;
63+ size += 13 * SZ_4K;
5964 break;
6065 case XE_ENGINE_CLASS_COMPUTE:
61− /* 14 pages since graphics_ver == 11 */
6266 if (GRAPHICS_VER(xe) >= 20)
63− size = 3 * SZ_4K;
67+ size += 2 * SZ_4K;
6468 else
65− size = 14 * SZ_4K;
69+ size += 13 * SZ_4K;
6670 break;
6771 default:
6872 WARN(1, "Unknown engine class: %d", class);
@@ -71,7 +75,7 @@ size_t xe_gt_lrc_size(struct xe_gt *gt, enum xe_engine_class class) · 2 unmodified lines
7175 case XE_ENGINE_CLASS_VIDEO_DECODE:
7276 case XE_ENGINE_CLASS_VIDEO_ENHANCE:
7377 case XE_ENGINE_CLASS_OTHER:
74− size = 2 * SZ_4K;
78+ size += 1 * SZ_4K;
7579 }
7680
7781 /* Add indirect ring state page */
@@ -650,7 +654,6 @@ u32 xe_lrc_pphwsp_offset(struct xe_lrc *lrc) · 572 unmodified lines
650654 #define LRC_START_SEQNO_PPHWSP_OFFSET (LRC_SEQNO_PPHWSP_OFFSET + 8)
651655 #define LRC_CTX_JOB_TIMESTAMP_OFFSET (LRC_START_SEQNO_PPHWSP_OFFSET + 8)
652656 #define LRC_PARALLEL_PPHWSP_OFFSET 2048
653−#define LRC_PPHWSP_SIZE SZ_4K
654657
655658 u32 xe_lrc_regs_offset(struct xe_lrc *lrc)
656659 {
@@ -893,6 +896,7 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, · 236 unmodified lines
893896 void *init_data = NULL;
894897 u32 arb_enable;
895898 u32 lrc_size;
899+ u32 bo_flags;
896900 int err;
897901
898902 kref_init(&lrc->refcount);
@@ -901,15 +905,18 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, · 2 unmodified lines
901905 if (xe_gt_has_indirect_ring_state(gt))
902906 lrc->flags |= XE_LRC_FLAG_INDIRECT_RING_STATE;
903907
908+ bo_flags = XE_BO_FLAG_VRAM_IF_DGFX(tile) | XE_BO_FLAG_GGTT |
909+ XE_BO_FLAG_GGTT_INVALIDATE;
910+ if (vm && vm->xef) /* userspace */
911+ bo_flags |= XE_BO_FLAG_PINNED_LATE_RESTORE;
912+
904913 /*
905914 * FIXME: Perma-pinning LRC as we don't yet support moving GGTT address
906915 * via VM bind calls.
907916 */
908917 lrc->bo = xe_bo_create_pin_map(xe, tile, vm, lrc_size,
909918 ttm_bo_type_kernel,
910− XE_BO_FLAG_VRAM_IF_DGFX(tile) |
911− XE_BO_FLAG_GGTT |
912− XE_BO_FLAG_GGTT_INVALIDATE);
919+ bo_flags);
913920 if (IS_ERR(lrc->bo))
914921 return PTR_ERR(lrc->bo);
915922
@@ -1445,6 +1452,7 @@ static int dump_gfxpipe_command(struct drm_printer *p, · 529 unmodified lines
14451452 MATCH3D(3DSTATE_CLIP_MESH);
14461453 MATCH3D(3DSTATE_SBE_MESH);
14471454 MATCH3D(3DSTATE_CPSIZE_CONTROL_BUFFER);
1455+ MATCH3D(3DSTATE_COARSE_PIXEL);
14481456
14491457 MATCH3D(3DSTATE_DRAWING_RECTANGLE);
14501458 MATCH3D(3DSTATE_CHROMA_KEY);
drivers/gpu/drm/xe/xe_memirq.c+1−1
@@ -86,7 +86,7 @@ static const char *guc_name(struct xe_guc *guc) · 85 unmodified lines
8686 * This object needs to be 4KiB aligned.
8787 *
8888 * - _`Interrupt Source Report Page`: this is the equivalent of the
89− * GEN11_GT_INTR_DWx registers, with each bit in those registers being
89+ * GT_INTR_DWx registers, with each bit in those registers being
9090 * mapped to a byte here. The offsets are the same, just bytes instead
9191 * of bits. This object needs to be cacheline aligned.
9292 *
drivers/gpu/drm/xe/xe_migrate.c+15−9
@@ -97,7 +97,7 @@ struct xe_exec_queue *xe_tile_migrate_exec_queue(struct xe_tile *tile) · 96 unmodified lines
9797 return tile->migrate->q;
9898 }
9999
100−static void xe_migrate_fini(struct drm_device *dev, void *arg)
100+static void xe_migrate_fini(void *arg)
101101 {
102102 struct xe_migrate *m = arg;
103103
@@ -209,7 +209,6 @@ static int xe_migrate_prepare_vm(struct xe_tile *tile, struct xe_migrate *m, · 105 unmodified lines
209209 num_entries * XE_PAGE_SIZE,
210210 ttm_bo_type_kernel,
211211 XE_BO_FLAG_VRAM_IF_DGFX(tile) |
212− XE_BO_FLAG_PINNED |
213212 XE_BO_FLAG_PAGETABLE);
214213 if (IS_ERR(bo))
215214 return PTR_ERR(bo);
@@ -401,7 +400,7 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile) · 185 unmodified lines
401400 struct xe_vm *vm;
402401 int err;
403402
404− m = drmm_kzalloc(&xe->drm, sizeof(*m), GFP_KERNEL);
403+ m = devm_kzalloc(xe->drm.dev, sizeof(*m), GFP_KERNEL);
405404 if (!m)
406405 return ERR_PTR(-ENOMEM);
407406
@@ -455,7 +454,7 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile) · 47 unmodified lines
455454 might_lock(&m->job_mutex);
456455 fs_reclaim_release(GFP_KERNEL);
457456
458− err = drmm_add_action_or_reset(&xe->drm, xe_migrate_fini, m);
457+ err = devm_add_action_or_reset(xe->drm.dev, xe_migrate_fini, m);
459458 if (err)
460459 return ERR_PTR(err);
461460
@@ -779,10 +778,12 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, · 317 unmodified lines
779778 bool dst_is_pltt = dst->mem_type == XE_PL_TT;
780779 bool src_is_vram = mem_type_is_vram(src->mem_type);
781780 bool dst_is_vram = mem_type_is_vram(dst->mem_type);
781+ bool type_device = src_bo->ttm.type == ttm_bo_type_device;
782+ bool needs_ccs_emit = type_device && xe_migrate_needs_ccs_emit(xe);
782783 bool copy_ccs = xe_device_has_flat_ccs(xe) &&
783784 xe_bo_needs_ccs_pages(src_bo) && xe_bo_needs_ccs_pages(dst_bo);
784785 bool copy_system_ccs = copy_ccs && (!src_is_vram || !dst_is_vram);
785− bool use_comp_pat = xe_device_has_flat_ccs(xe) &&
786+ bool use_comp_pat = type_device && xe_device_has_flat_ccs(xe) &&
786787 GRAPHICS_VER(xe) >= 20 && src_is_vram && !dst_is_vram;
787788
788789 /* Copying CCS between two different BOs is not supported yet. */
@@ -839,6 +840,7 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, · 50 unmodified lines
839840 avail_pts, avail_pts);
840841
841842 if (copy_system_ccs) {
843+ xe_assert(xe, type_device);
842844 ccs_size = xe_device_ccs_bytes(xe, src_L0);
843845 batch_size += pte_update_size(m, 0, NULL, &ccs_it, &ccs_size,
844846 &ccs_ofs, &ccs_pt, 0,
@@ -849,7 +851,7 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, · 4 unmodified lines
849851
850852 /* Add copy commands size here */
851853 batch_size += ((copy_only_ccs) ? 0 : EMIT_COPY_DW) +
852− ((xe_migrate_needs_ccs_emit(xe) ? EMIT_COPY_CCS_DW : 0));
854+ ((needs_ccs_emit ? EMIT_COPY_CCS_DW : 0));
853855
854856 bb = xe_bb_new(gt, batch_size, usm);
855857 if (IS_ERR(bb)) {
@@ -878,7 +880,7 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, · 22 unmodified lines
878880 if (!copy_only_ccs)
879881 emit_copy(gt, bb, src_L0_ofs, dst_L0_ofs, src_L0, XE_PAGE_SIZE);
880882
881− if (xe_migrate_needs_ccs_emit(xe))
883+ if (needs_ccs_emit)
882884 flush_flags = xe_migrate_ccs_copy(m, bb, src_L0_ofs,
883885 IS_DGFX(xe) ? src_is_vram : src_is_pltt,
884886 dst_L0_ofs,
@@ -1544,6 +1546,7 @@ void xe_migrate_wait(struct xe_migrate *m) · 659 unmodified lines
15441546 dma_fence_wait(m->fence, false);
15451547 }
15461548
1549+#if IS_ENABLED(CONFIG_DRM_XE_DEVMEM_MIRROR)
15471550 static u32 pte_update_cmd_size(u64 size)
15481551 {
15491552 u32 num_dword;
@@ -1608,6 +1611,7 @@ static struct dma_fence *xe_migrate_vram(struct xe_migrate *m, · 58 unmodified lines
16081611 {
16091612 struct xe_gt *gt = m->tile->primary_gt;
16101613 struct xe_device *xe = gt_to_xe(gt);
1614+ bool use_usm_batch = xe->info.has_usm;
16111615 struct dma_fence *fence = NULL;
16121616 u32 batch_size = 2;
16131617 u64 src_L0_ofs, dst_L0_ofs;
@@ -1624,7 +1628,7 @@ static struct dma_fence *xe_migrate_vram(struct xe_migrate *m, · 10 unmodified lines
16241628 batch_size += pte_update_cmd_size(round_update_size);
16251629 batch_size += EMIT_COPY_DW;
16261630
1627− bb = xe_bb_new(gt, batch_size, true);
1631+ bb = xe_bb_new(gt, batch_size, use_usm_batch);
16281632 if (IS_ERR(bb)) {
16291633 err = PTR_ERR(bb);
16301634 return ERR_PTR(err);
@@ -1649,7 +1653,7 @@ static struct dma_fence *xe_migrate_vram(struct xe_migrate *m, · 18 unmodified lines
16491653 XE_PAGE_SIZE);
16501654
16511655 job = xe_bb_create_migration_job(m->q, bb,
1652− xe_migrate_batch_base(m, true),
1656+ xe_migrate_batch_base(m, use_usm_batch),
16531657 update_idx);
16541658 if (IS_ERR(job)) {
16551659 err = PTR_ERR(job);
@@ -1719,6 +1723,8 @@ struct dma_fence *xe_migrate_from_vram(struct xe_migrate *m, · 63 unmodified lines
17191723 XE_MIGRATE_COPY_TO_SRAM);
17201724 }
17211725
1726+#endif
1727+
17221728 #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)
17231729 #include "tests/xe_migrate.c"
17241730 #endif
drivers/gpu/drm/xe/xe_mmio.c+7−4
@@ -138,6 +138,7 @@ int xe_mmio_probe_early(struct xe_device *xe) · 137 unmodified lines
138138
139139 return devm_add_action_or_reset(xe->drm.dev, mmio_fini, xe);
140140 }
141+ALLOW_ERROR_INJECTION(xe_mmio_probe_early, ERRNO); /* See xe_pci_probe() */
141142
142143 /**
143144 * xe_mmio_init() - Initialize an MMIO instance
@@ -204,8 +205,9 @@ void xe_mmio_write32(struct xe_mmio *mmio, struct xe_reg reg, u32 val) · 60 unmodified lines
204205
205206 trace_xe_reg_rw(mmio, true, addr, val, sizeof(val));
206207
207− if (!reg.vf && mmio->sriov_vf_gt)
208− xe_gt_sriov_vf_write32(mmio->sriov_vf_gt, reg, val);
208+ if (!reg.vf && IS_SRIOV_VF(mmio->tile->xe))
209+ xe_gt_sriov_vf_write32(mmio->sriov_vf_gt ?:
210+ mmio->tile->primary_gt, reg, val);
209211 else
210212 writel(val, mmio->regs + addr);
211213 }
@@ -218,8 +220,9 @@ u32 xe_mmio_read32(struct xe_mmio *mmio, struct xe_reg reg) · 6 unmodified lines
218220 /* Wa_15015404425 */
219221 mmio_flush_pending_writes(mmio);
220222
221− if (!reg.vf && mmio->sriov_vf_gt)
222− val = xe_gt_sriov_vf_read32(mmio->sriov_vf_gt, reg);
223+ if (!reg.vf && IS_SRIOV_VF(mmio->tile->xe))
224+ val = xe_gt_sriov_vf_read32(mmio->sriov_vf_gt ?:
225+ mmio->tile->primary_gt, reg);
223226 else
224227 val = readl(mmio->regs + addr);
225228
drivers/gpu/drm/xe/xe_module.c+7−2
@@ -11,6 +11,7 @@ · 10 unmodified lines
1111 #include <drm/drm_module.h>
1212
1313 #include "xe_drv.h"
14+#include "xe_configfs.h"
1415 #include "xe_hw_fence.h"
1516 #include "xe_pci.h"
1617 #include "xe_pm.h"
@@ -38,8 +39,8 @@ MODULE_PARM_DESC(force_execlist, "Force Execlist submission"); · 21 unmodified lines
3839 module_param_named(probe_display, xe_modparam.probe_display, bool, 0444);
3940 MODULE_PARM_DESC(probe_display, "Probe display HW, otherwise it's left untouched (default: true)");
4041
41−module_param_named(vram_bar_size, xe_modparam.force_vram_bar_size, uint, 0600);
42−MODULE_PARM_DESC(vram_bar_size, "Set the vram bar size(in MiB)");
42+module_param_named(vram_bar_size, xe_modparam.force_vram_bar_size, int, 0600);
43+MODULE_PARM_DESC(vram_bar_size, "Set the vram bar size (in MiB) - <0=disable-resize, 0=max-needed-size[default], >0=force-size");
4344
4445 module_param_named(guc_log_level, xe_modparam.guc_log_level, int, 0600);
4546 MODULE_PARM_DESC(guc_log_level, "GuC firmware logging level (0=disable, 1..5=enable with verbosity min..max)");
@@ -88,6 +89,10 @@ static const struct init_funcs init_funcs[] = { · 42 unmodified lines
8889 {
8990 .init = xe_check_nomodeset,
9091 },
92+ {
93+ .init = xe_configfs_init,
94+ .exit = xe_configfs_exit,
95+ },
9196 {
9297 .init = xe_hw_fence_module_init,
9398 .exit = xe_hw_fence_module_exit,
drivers/gpu/drm/xe/xe_oa.c+1−0
@@ -2221,6 +2221,7 @@ addr_err: · 2220 unmodified lines
22212221 kfree(oa_regs);
22222222 return ERR_PTR(err);
22232223 }
2224+ALLOW_ERROR_INJECTION(xe_oa_alloc_regs, ERRNO);
22242225
22252226 static ssize_t show_dynamic_id(struct kobject *kobj,
22262227 struct kobj_attribute *attr,
drivers/gpu/drm/xe/xe_pci.c+19−10
@@ -62,11 +62,13 @@ struct xe_device_desc { · 61 unmodified lines
6262 u8 is_dgfx:1;
6363
6464 u8 has_display:1;
65+ u8 has_fan_control:1;
6566 u8 has_heci_gscfi:1;
6667 u8 has_heci_cscfi:1;
6768 u8 has_llc:1;
6869 u8 has_pxp:1;
6970 u8 has_sriov:1;
71+ u8 needs_scratch:1;
7072 u8 skip_guc_pc:1;
7173 u8 skip_mtcfg:1;
7274 u8 skip_pcode:1;
@@ -302,6 +304,7 @@ static const struct xe_device_desc dg2_desc = { · 229 unmodified lines
302304
303305 DG2_FEATURES,
304306 .has_display = true,
307+ .has_fan_control = true,
305308 };
306309
307310 static const __maybe_unused struct xe_device_desc pvc_desc = {
@@ -329,6 +332,7 @@ static const struct xe_device_desc lnl_desc = { · 21 unmodified lines
329332 .dma_mask_size = 46,
330333 .has_display = true,
331334 .has_pxp = true,
335+ .needs_scratch = true,
332336 };
333337
334338 static const struct xe_device_desc bmg_desc = {
@@ -336,7 +340,9 @@ static const struct xe_device_desc bmg_desc = { · 1 unmodified line
336340 PLATFORM(BATTLEMAGE),
337341 .dma_mask_size = 46,
338342 .has_display = true,
343+ .has_fan_control = true,
339344 .has_heci_cscfi = 1,
345+ .needs_scratch = true,
340346 };
341347
342348 static const struct xe_device_desc ptl_desc = {
@@ -345,6 +351,7 @@ static const struct xe_device_desc ptl_desc = { · 2 unmodified lines
345351 .has_display = true,
346352 .has_sriov = true,
347353 .require_force_probe = true,
354+ .needs_scratch = true,
348355 };
349356
350357 #undef PLATFORM
@@ -575,6 +582,7 @@ static int xe_info_init_early(struct xe_device *xe, · 224 unmodified lines
575582
576583 xe->info.dma_mask_size = desc->dma_mask_size;
577584 xe->info.is_dgfx = desc->is_dgfx;
585+ xe->info.has_fan_control = desc->has_fan_control;
578586 xe->info.has_heci_gscfi = desc->has_heci_gscfi;
579587 xe->info.has_heci_cscfi = desc->has_heci_cscfi;
580588 xe->info.has_llc = desc->has_llc;
@@ -583,6 +591,7 @@ static int xe_info_init_early(struct xe_device *xe, · 2 unmodified lines
583591 xe->info.skip_guc_pc = desc->skip_guc_pc;
584592 xe->info.skip_mtcfg = desc->skip_mtcfg;
585593 xe->info.skip_pcode = desc->skip_pcode;
594+ xe->info.needs_scratch = desc->needs_scratch;
586595
587596 xe->info.probe_display = IS_ENABLED(CONFIG_DRM_XE_DISPLAY) &&
588597 xe_modparam.probe_display &&
@@ -803,18 +812,17 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 214 unmodified lines
803812 return err;
804813
805814 err = xe_device_probe_early(xe);
806− if (err) {
807− /*
808− * In Boot Survivability mode, no drm card is exposed and driver
809− * is loaded with bare minimum to allow for firmware to be
810− * flashed through mei. If early probe failed, but it managed to
811− * enable survivability mode, return success.
812− */
813− if (xe_survivability_mode_is_enabled(xe))
814− return 0;
815+ /*
816+ * In Boot Survivability mode, no drm card is exposed and driver
817+ * is loaded with bare minimum to allow for firmware to be
818+ * flashed through mei. Return success, if survivability mode
819+ * is enabled due to pcode failure or configfs being set
820+ */
821+ if (xe_survivability_mode_is_enabled(xe))
822+ return 0;
815823
824+ if (err)
816825 return err;
817− }
818826
819827 err = xe_info_init(xe, desc);
820828 if (err)
@@ -920,6 +928,7 @@ static int xe_pci_suspend(struct device *dev) · 99 unmodified lines
920928
921929 pci_save_state(pdev);
922930 pci_disable_device(pdev);
931+ pci_set_power_state(pdev, PCI_D3cold);
923932
924933 return 0;
925934 }
drivers/gpu/drm/xe/xe_pci_sriov.c+20−0
@@ -7,6 +7,7 @@ · 6 unmodified lines
77 #include "xe_device.h"
88 #include "xe_gt_sriov_pf_config.h"
99 #include "xe_gt_sriov_pf_control.h"
10+#include "xe_guc_engine_activity.h"
1011 #include "xe_pci_sriov.h"
1112 #include "xe_pm.h"
1213 #include "xe_sriov.h"
@@ -111,6 +112,20 @@ static void pf_link_vfs(struct xe_device *xe, int num_vfs) · 98 unmodified lines
111112 }
112113 }
113114
115+static void pf_engine_activity_stats(struct xe_device *xe, unsigned int num_vfs, bool enable)
116+{
117+ struct xe_gt *gt;
118+ unsigned int id;
119+ int ret = 0;
120+
121+ for_each_gt(gt, xe, id) {
122+ ret = xe_guc_engine_activity_function_stats(&gt->uc.guc, num_vfs, enable);
123+ if (ret)
124+ xe_sriov_info(xe, "Failed to %s engine activity function stats (%pe)\n",
125+ str_enable_disable(enable), ERR_PTR(ret));
126+ }
127+}
128+
114129 static int pf_enable_vfs(struct xe_device *xe, int num_vfs)
115130 {
116131 struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
@@ -145,6 +160,9 @@ static int pf_enable_vfs(struct xe_device *xe, int num_vfs) · 28 unmodified lines
145160
146161 xe_sriov_info(xe, "Enabled %u of %u VF%s\n",
147162 num_vfs, total_vfs, str_plural(total_vfs));
163+
164+ pf_engine_activity_stats(xe, num_vfs, true);
165+
148166 return num_vfs;
149167
150168 failed:
@@ -168,6 +186,8 @@ static int pf_disable_vfs(struct xe_device *xe) · 17 unmodified lines
168186 if (!num_vfs)
169187 return 0;
170188
189+ pf_engine_activity_stats(xe, num_vfs, false);
190+
171191 pci_disable_sriov(pdev);
172192
173193 pf_reset_vfs(xe, num_vfs);
drivers/gpu/drm/xe/xe_pcode.c+2−0
@@ -7,6 +7,7 @@ · 6 unmodified lines
77
88 #include <linux/delay.h>
99 #include <linux/errno.h>
10+#include <linux/error-injection.h>
1011
1112 #include <drm/drm_managed.h>
1213
@@ -323,3 +324,4 @@ int xe_pcode_probe_early(struct xe_device *xe) · 310 unmodified lines
323324 {
324325 return xe_pcode_ready(xe, false);
325326 }
327+ALLOW_ERROR_INJECTION(xe_pcode_probe_early, ERRNO); /* See xe_pci_probe */
drivers/gpu/drm/xe/xe_pcode_api.h+3−0
@@ -49,6 +49,9 @@ · 48 unmodified lines
4949 /* Domain IDs (param2) */
5050 #define PCODE_MBOX_DOMAIN_HBM 0x2
5151
52+#define FAN_SPEED_CONTROL 0x7D
53+#define FSC_READ_NUM_FANS 0x4
54+
5255 #define PCODE_SCRATCH(x) XE_REG(0x138320 + ((x) * 4))
5356 /* PCODE_SCRATCH0 */
5457 #define AUXINFO_REG_OFFSET REG_GENMASK(17, 15)
drivers/gpu/drm/xe/xe_pm.c+4−4
@@ -188,7 +188,7 @@ int xe_pm_resume(struct xe_device *xe) · 187 unmodified lines
188188 * This only restores pinned memory which is the memory required for the
189189 * GT(s) to resume.
190190 */
191− err = xe_bo_restore_kernel(xe);
191+ err = xe_bo_restore_early(xe);
192192 if (err)
193193 goto err;
194194
@@ -199,7 +199,7 @@ int xe_pm_resume(struct xe_device *xe) · 4 unmodified lines
199199
200200 xe_display_pm_resume(xe);
201201
202− err = xe_bo_restore_user(xe);
202+ err = xe_bo_restore_late(xe);
203203 if (err)
204204 goto err;
205205
@@ -484,7 +484,7 @@ int xe_pm_runtime_resume(struct xe_device *xe) · 278 unmodified lines
484484 * This only restores pinned memory which is the memory
485485 * required for the GT(s) to resume.
486486 */
487− err = xe_bo_restore_kernel(xe);
487+ err = xe_bo_restore_early(xe);
488488 if (err)
489489 goto out;
490490 }
@@ -497,7 +497,7 @@ int xe_pm_runtime_resume(struct xe_device *xe) · 6 unmodified lines
497497 xe_display_pm_runtime_resume(xe);
498498
499499 if (xe->d3cold.allowed) {
500− err = xe_bo_restore_user(xe);
500+ err = xe_bo_restore_late(xe);
501501 if (err)
502502 goto out;
503503 }
drivers/gpu/drm/xe/xe_pmu.c+67−10
@@ -10,9 +10,11 @@ · 9 unmodified lines
1010 #include "xe_force_wake.h"
1111 #include "xe_gt_idle.h"
1212 #include "xe_guc_engine_activity.h"
13+#include "xe_guc_pc.h"
1314 #include "xe_hw_engine.h"
1415 #include "xe_pm.h"
1516 #include "xe_pmu.h"
17+#include "xe_sriov_pf_helpers.h"
1618
1719 /**
1820 * DOC: Xe PMU (Performance Monitoring Unit)
@@ -32,9 +34,10 @@ · 13 unmodified lines
3234 * gt[60:63] Selects gt for the event
3335 * engine_class[20:27] Selects engine-class for event
3436 * engine_instance[12:19] Selects the engine-instance for the event
37+ * function[44:59] Selects the function of the event (SRIOV enabled)
3538 *
3639 * For engine specific events (engine-*), gt, engine_class and engine_instance parameters must be
37− * set as populated by DRM_XE_DEVICE_QUERY_ENGINES.
40+ * set as populated by DRM_XE_DEVICE_QUERY_ENGINES and function if SRIOV is enabled.
3841 *
3942 * For gt specific events (gt-*) gt parameter must be passed. All other parameters will be 0.
4043 *
@@ -49,6 +52,7 @@ · 8 unmodified lines
4952 */
5053
5154 #define XE_PMU_EVENT_GT_MASK GENMASK_ULL(63, 60)
55+#define XE_PMU_EVENT_FUNCTION_MASK GENMASK_ULL(59, 44)
5256 #define XE_PMU_EVENT_ENGINE_CLASS_MASK GENMASK_ULL(27, 20)
5357 #define XE_PMU_EVENT_ENGINE_INSTANCE_MASK GENMASK_ULL(19, 12)
5458 #define XE_PMU_EVENT_ID_MASK GENMASK_ULL(11, 0)
@@ -58,6 +62,11 @@ static unsigned int config_to_event_id(u64 config) · 3 unmodified lines
5862 return FIELD_GET(XE_PMU_EVENT_ID_MASK, config);
5963 }
6064
65+static unsigned int config_to_function_id(u64 config)
66+{
67+ return FIELD_GET(XE_PMU_EVENT_FUNCTION_MASK, config);
68+}
69+
6170 static unsigned int config_to_engine_class(u64 config)
6271 {
6372 return FIELD_GET(XE_PMU_EVENT_ENGINE_CLASS_MASK, config);
@@ -76,6 +85,8 @@ static unsigned int config_to_gt_id(u64 config) · 12 unmodified lines
7685 #define XE_PMU_EVENT_GT_C6_RESIDENCY 0x01
7786 #define XE_PMU_EVENT_ENGINE_ACTIVE_TICKS 0x02
7887 #define XE_PMU_EVENT_ENGINE_TOTAL_TICKS 0x03
88+#define XE_PMU_EVENT_GT_ACTUAL_FREQUENCY 0x04
89+#define XE_PMU_EVENT_GT_REQUESTED_FREQUENCY 0x05
7990
8091 static struct xe_gt *event_to_gt(struct perf_event *event)
8192 {
@@ -111,6 +122,14 @@ static bool is_engine_event(u64 config) · 29 unmodified lines
111122 event_id == XE_PMU_EVENT_ENGINE_ACTIVE_TICKS);
112123 }
113124
125+static bool is_gt_frequency_event(struct perf_event *event)
126+{
127+ u32 id = config_to_event_id(event->attr.config);
128+
129+ return id == XE_PMU_EVENT_GT_ACTUAL_FREQUENCY ||
130+ id == XE_PMU_EVENT_GT_REQUESTED_FREQUENCY;
131+}
132+
114133 static bool event_gt_forcewake(struct perf_event *event)
115134 {
116135 struct xe_device *xe = container_of(event->pmu, typeof(*xe), pmu.base);
@@ -118,7 +137,7 @@ static bool event_gt_forcewake(struct perf_event *event) · 1 unmodified line
118137 struct xe_gt *gt;
119138 unsigned int *fw_ref;
120139
121− if (!is_engine_event(config))
140+ if (!is_engine_event(config) && !is_gt_frequency_event(event))
122141 return true;
123142
124143 gt = xe_device_get_gt(xe, config_to_gt_id(config));
@@ -151,7 +170,7 @@ static bool event_supported(struct xe_pmu *pmu, unsigned int gt, · 26 unmodified lines
151170 static bool event_param_valid(struct perf_event *event)
152171 {
153172 struct xe_device *xe = container_of(event->pmu, typeof(*xe), pmu.base);
154− unsigned int engine_class, engine_instance;
173+ unsigned int engine_class, engine_instance, function_id;
155174 u64 config = event->attr.config;
156175 struct xe_gt *gt;
157176
@@ -161,16 +180,28 @@ static bool event_param_valid(struct perf_event *event) · 3 unmodified lines
161180
162181 engine_class = config_to_engine_class(config);
163182 engine_instance = config_to_engine_instance(config);
183+ function_id = config_to_function_id(config);
164184
165185 switch (config_to_event_id(config)) {
166186 case XE_PMU_EVENT_GT_C6_RESIDENCY:
167− if (engine_class || engine_instance)
187+ case XE_PMU_EVENT_GT_ACTUAL_FREQUENCY:
188+ case XE_PMU_EVENT_GT_REQUESTED_FREQUENCY:
189+ if (engine_class || engine_instance || function_id)
168190 return false;
169191 break;
170192 case XE_PMU_EVENT_ENGINE_ACTIVE_TICKS:
171193 case XE_PMU_EVENT_ENGINE_TOTAL_TICKS:
172194 if (!event_to_hwe(event))
173195 return false;
196+
197+ /* PF(0) and total vfs when SRIOV is enabled */
198+ if (IS_SRIOV_PF(xe)) {
199+ if (function_id > xe_sriov_pf_get_totalvfs(xe))
200+ return false;
201+ } else if (function_id) {
202+ return false;
203+ }
204+
174205 break;
175206 }
176207
@@ -242,13 +273,17 @@ static int xe_pmu_event_init(struct perf_event *event) · 65 unmodified lines
242273 static u64 read_engine_events(struct xe_gt *gt, struct perf_event *event)
243274 {
244275 struct xe_hw_engine *hwe;
245− u64 val = 0;
276+ unsigned int function_id;
277+ u64 config, val = 0;
278+
279+ config = event->attr.config;
280+ function_id = config_to_function_id(config);
246281
247282 hwe = event_to_hwe(event);
248− if (config_to_event_id(event->attr.config) == XE_PMU_EVENT_ENGINE_ACTIVE_TICKS)
249− val = xe_guc_engine_activity_active_ticks(&gt->uc.guc, hwe);
283+ if (config_to_event_id(config) == XE_PMU_EVENT_ENGINE_ACTIVE_TICKS)
284+ val = xe_guc_engine_activity_active_ticks(&gt->uc.guc, hwe, function_id);
250285 else
251− val = xe_guc_engine_activity_total_ticks(&gt->uc.guc, hwe);
286+ val = xe_guc_engine_activity_total_ticks(&gt->uc.guc, hwe, function_id);
252287
253288 return val;
254289 }
@@ -266,6 +301,10 @@ static u64 __xe_pmu_event_read(struct perf_event *event) · 11 unmodified lines
266301 case XE_PMU_EVENT_ENGINE_ACTIVE_TICKS:
267302 case XE_PMU_EVENT_ENGINE_TOTAL_TICKS:
268303 return read_engine_events(gt, event);
304+ case XE_PMU_EVENT_GT_ACTUAL_FREQUENCY:
305+ return xe_guc_pc_get_act_freq(&gt->uc.guc.pc);
306+ case XE_PMU_EVENT_GT_REQUESTED_FREQUENCY:
307+ return xe_guc_pc_get_cur_freq_fw(&gt->uc.guc.pc);
269308 }
270309
271310 return 0;
@@ -281,7 +320,14 @@ static void xe_pmu_event_update(struct perf_event *event) · 9 unmodified lines
281320 new = __xe_pmu_event_read(event);
282321 } while (!local64_try_cmpxchg(&hwc->prev_count, &prev, new));
283322
284− local64_add(new - prev, &event->count);
323+ /*
324+ * GT frequency is not a monotonically increasing counter, so add the
325+ * instantaneous value instead.
326+ */
327+ if (is_gt_frequency_event(event))
328+ local64_add(new, &event->count);
329+ else
330+ local64_add(new - prev, &event->count);
285331 }
286332
287333 static void xe_pmu_event_read(struct perf_event *event)
@@ -351,6 +397,7 @@ static void xe_pmu_event_del(struct perf_event *event, int flags) · 63 unmodified lines
351397 }
352398
353399 PMU_FORMAT_ATTR(gt, "config:60-63");
400+PMU_FORMAT_ATTR(function, "config:44-59");
354401 PMU_FORMAT_ATTR(engine_class, "config:20-27");
355402 PMU_FORMAT_ATTR(engine_instance, "config:12-19");
356403 PMU_FORMAT_ATTR(event, "config:0-11");
@@ -359,6 +406,7 @@ static struct attribute *pmu_format_attrs[] = { · 2 unmodified lines
359406 &format_attr_event.attr,
360407 &format_attr_engine_class.attr,
361408 &format_attr_engine_instance.attr,
409+ &format_attr_function.attr,
362410 &format_attr_gt.attr,
363411 NULL,
364412 };
@@ -419,6 +467,10 @@ static ssize_t event_attr_show(struct device *dev, · 54 unmodified lines
419467 XE_EVENT_ATTR_SIMPLE(gt-c6-residency, gt_c6_residency, XE_PMU_EVENT_GT_C6_RESIDENCY, "ms");
420468 XE_EVENT_ATTR_NOUNIT(engine-active-ticks, engine_active_ticks, XE_PMU_EVENT_ENGINE_ACTIVE_TICKS);
421469 XE_EVENT_ATTR_NOUNIT(engine-total-ticks, engine_total_ticks, XE_PMU_EVENT_ENGINE_TOTAL_TICKS);
470+XE_EVENT_ATTR_SIMPLE(gt-actual-frequency, gt_actual_frequency,
471+ XE_PMU_EVENT_GT_ACTUAL_FREQUENCY, "MHz");
472+XE_EVENT_ATTR_SIMPLE(gt-requested-frequency, gt_requested_frequency,
473+ XE_PMU_EVENT_GT_REQUESTED_FREQUENCY, "MHz");
422474
423475 static struct attribute *pmu_empty_event_attrs[] = {
424476 /* Empty - all events are added as groups with .attr_update() */
@@ -434,6 +486,8 @@ static const struct attribute_group *pmu_events_attr_update[] = { · 9 unmodified lines
434486 &pmu_group_gt_c6_residency,
435487 &pmu_group_engine_active_ticks,
436488 &pmu_group_engine_total_ticks,
489+ &pmu_group_gt_actual_frequency,
490+ &pmu_group_gt_requested_frequency,
437491 NULL,
438492 };
439493
@@ -442,8 +496,11 @@ static void set_supported_events(struct xe_pmu *pmu) · 2 unmodified lines
442496 struct xe_device *xe = container_of(pmu, typeof(*xe), pmu);
443497 struct xe_gt *gt = xe_device_get_gt(xe, 0);
444498
445− if (!xe->info.skip_guc_pc)
499+ if (!xe->info.skip_guc_pc) {
446500 pmu->supported_events |= BIT_ULL(XE_PMU_EVENT_GT_C6_RESIDENCY);
501+ pmu->supported_events |= BIT_ULL(XE_PMU_EVENT_GT_ACTUAL_FREQUENCY);
502+ pmu->supported_events |= BIT_ULL(XE_PMU_EVENT_GT_REQUESTED_FREQUENCY);
503+ }
447504
448505 if (xe_guc_engine_activity_supported(&gt->uc.guc)) {
449506 pmu->supported_events |= BIT_ULL(XE_PMU_EVENT_ENGINE_ACTIVE_TICKS);
drivers/gpu/drm/xe/xe_pt.c+140−98
@@ -103,6 +103,7 @@ struct xe_pt *xe_pt_create(struct xe_vm *vm, struct xe_tile *tile, · 102 unmodified lines
103103 {
104104 struct xe_pt *pt;
105105 struct xe_bo *bo;
106+ u32 bo_flags;
106107 int err;
107108
108109 if (level) {
@@ -115,14 +116,16 @@ struct xe_pt *xe_pt_create(struct xe_vm *vm, struct xe_tile *tile, · 6 unmodified lines
115116 if (!pt)
116117 return ERR_PTR(-ENOMEM);
117118
119+ bo_flags = XE_BO_FLAG_VRAM_IF_DGFX(tile) |
120+ XE_BO_FLAG_IGNORE_MIN_PAGE_SIZE |
121+ XE_BO_FLAG_NO_RESV_EVICT | XE_BO_FLAG_PAGETABLE;
122+ if (vm->xef) /* userspace */
123+ bo_flags |= XE_BO_FLAG_PINNED_LATE_RESTORE;
124+
118125 pt->level = level;
119126 bo = xe_bo_create_pin_map(vm->xe, tile, vm, SZ_4K,
120127 ttm_bo_type_kernel,
121− XE_BO_FLAG_VRAM_IF_DGFX(tile) |
122− XE_BO_FLAG_IGNORE_MIN_PAGE_SIZE |
123− XE_BO_FLAG_PINNED |
124− XE_BO_FLAG_NO_RESV_EVICT |
125− XE_BO_FLAG_PAGETABLE);
128+ bo_flags);
126129 if (IS_ERR(bo)) {
127130 err = PTR_ERR(bo);
128131 goto err_kfree;
@@ -269,8 +272,11 @@ struct xe_pt_update { · 140 unmodified lines
269272 bool preexisting;
270273 };
271274
275+/**
276+ * struct xe_pt_stage_bind_walk - Walk state for the stage_bind walk.
277+ */
272278 struct xe_pt_stage_bind_walk {
273− /** base: The base class. */
279+ /** @base: The base class. */
274280 struct xe_pt_walk base;
275281
276282 /* Input parameters for the walk */
@@ -278,15 +284,19 @@ struct xe_pt_stage_bind_walk { · 1 unmodified line
278284 struct xe_vm *vm;
279285 /** @tile: The tile we're building for. */
280286 struct xe_tile *tile;
281− /** @default_pte: PTE flag only template. No address is associated */
282− u64 default_pte;
287+ /** @default_vram_pte: PTE flag only template for VRAM. No address is associated */
288+ u64 default_vram_pte;
289+ /** @default_system_pte: PTE flag only template for System. No address is associated */
290+ u64 default_system_pte;
283291 /** @dma_offset: DMA offset to add to the PTE. */
284292 u64 dma_offset;
285293 /**
286− * @needs_64k: This address range enforces 64K alignment and
287− * granularity.
294+ * @needs_64K: This address range enforces 64K alignment and
295+ * granularity on VRAM.
288296 */
289297 bool needs_64K;
298+ /** @clear_pt: clear page table entries during the bind walk */
299+ bool clear_pt;
290300 /**
291301 * @vma: VMA being mapped
292302 */
@@ -299,6 +309,7 @@ struct xe_pt_stage_bind_walk { · 6 unmodified lines
299309 u64 va_curs_start;
300310
301311 /* Output */
312+ /** @wupd: Walk output data for page-table updates. */
302313 struct xe_walk_update {
303314 /** @wupd.entries: Caller provided storage. */
304315 struct xe_vm_pgtable_update *entries;
@@ -316,7 +327,7 @@ struct xe_pt_stage_bind_walk { · 11 unmodified lines
316327 u64 l0_end_addr;
317328 /** @addr_64K: The start address of the current 64K chunk. */
318329 u64 addr_64K;
319− /** @found_64: Whether @add_64K actually points to a 64K chunk. */
330+ /** @found_64K: Whether @add_64K actually points to a 64K chunk. */
320331 bool found_64K;
321332 };
322333
@@ -436,6 +447,10 @@ static bool xe_pt_hugepte_possible(u64 addr, u64 next, unsigned int level, · 113 unmodified lines
436447 if (xe_vma_is_null(xe_walk->vma))
437448 return true;
438449
450+ /* if we are clearing page table, no dma addresses*/
451+ if (xe_walk->clear_pt)
452+ return true;
453+
439454 /* Is the DMA address huge PTE size aligned? */
440455 size = next - addr;
441456 dma = addr - xe_walk->va_curs_start + xe_res_dma(xe_walk->curs);
@@ -515,24 +530,35 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset, · 73 unmodified lines
515530 if (level == 0 || xe_pt_hugepte_possible(addr, next, level, xe_walk)) {
516531 struct xe_res_cursor *curs = xe_walk->curs;
517532 bool is_null = xe_vma_is_null(xe_walk->vma);
533+ bool is_vram = is_null ? false : xe_res_is_vram(curs);
518534
519535 XE_WARN_ON(xe_walk->va_curs_start != addr);
520536
521− pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
522− xe_res_dma(curs) + xe_walk->dma_offset,
523− xe_walk->vma, pat_index, level);
524− pte |= xe_walk->default_pte;
537+ if (xe_walk->clear_pt) {
538+ pte = 0;
539+ } else {
540+ pte = vm->pt_ops->pte_encode_vma(is_null ? 0 :
541+ xe_res_dma(curs) +
542+ xe_walk->dma_offset,
543+ xe_walk->vma,
544+ pat_index, level);
545+ if (!is_null)
546+ pte |= is_vram ? xe_walk->default_vram_pte :
547+ xe_walk->default_system_pte;
525548
526− /*
527− * Set the XE_PTE_PS64 hint if possible, otherwise if
528− * this device *requires* 64K PTE size for VRAM, fail.
529− */
530− if (level == 0 && !xe_parent->is_compact) {
531− if (xe_pt_is_pte_ps64K(addr, next, xe_walk)) {
532− xe_walk->vma->gpuva.flags |= XE_VMA_PTE_64K;
533− pte |= XE_PTE_PS64;
534− } else if (XE_WARN_ON(xe_walk->needs_64K)) {
535− return -EINVAL;
549+ /*
550+ * Set the XE_PTE_PS64 hint if possible, otherwise if
551+ * this device *requires* 64K PTE size for VRAM, fail.
552+ */
553+ if (level == 0 && !xe_parent->is_compact) {
554+ if (xe_pt_is_pte_ps64K(addr, next, xe_walk)) {
555+ xe_walk->vma->gpuva.flags |=
556+ XE_VMA_PTE_64K;
557+ pte |= XE_PTE_PS64;
558+ } else if (XE_WARN_ON(xe_walk->needs_64K &&
559+ is_vram)) {
560+ return -EINVAL;
561+ }
536562 }
537563 }
538564
@@ -540,7 +566,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset, · 1 unmodified line
540566 if (unlikely(ret))
541567 return ret;
542568
543− if (!is_null)
569+ if (!is_null && !xe_walk->clear_pt)
544570 xe_res_next(curs, next - addr);
545571 xe_walk->va_curs_start = next;
546572 xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);
@@ -603,6 +629,44 @@ static const struct xe_pt_walk_ops xe_pt_stage_bind_ops = { · 56 unmodified lines
603629 .pt_entry = xe_pt_stage_bind_entry,
604630 };
605631
632+/*
633+ * Default atomic expectations for different allocation scenarios are as follows:
634+ *
635+ * 1. Traditional API: When the VM is not in LR mode:
636+ * - Device atomics are expected to function with all allocations.
637+ *
638+ * 2. Compute/SVM API: When the VM is in LR mode:
639+ * - Device atomics are the default behavior when the bo is placed in a single region.
640+ * - In all other cases device atomics will be disabled with AE=0 until an application
641+ * request differently using a ioctl like madvise.
642+ */
643+static bool xe_atomic_for_vram(struct xe_vm *vm)
644+{
645+ return true;
646+}
647+
648+static bool xe_atomic_for_system(struct xe_vm *vm, struct xe_bo *bo)
649+{
650+ struct xe_device *xe = vm->xe;
651+
652+ if (!xe->info.has_device_atomics_on_smem)
653+ return false;
654+
655+ /*
656+ * If a SMEM+LMEM allocation is backed by SMEM, a device
657+ * atomics will cause a gpu page fault and which then
658+ * gets migrated to LMEM, bind such allocations with
659+ * device atomics enabled.
660+ *
661+ * TODO: Revisit this. Perhaps add something like a
662+ * fault_on_atomics_in_system UAPI flag.
663+ * Note that this also prohibits GPU atomics in LR mode for
664+ * userptr and system memory on DGFX.
665+ */
666+ return (!IS_DGFX(xe) || (!xe_vm_in_lr_mode(vm) ||
667+ (bo && xe_bo_has_single_placement(bo))));
668+}
669+
606670 /**
607671 * xe_pt_stage_bind() - Build a disconnected page-table tree for a given address
608672 * range.
@@ -612,6 +676,7 @@ static const struct xe_pt_walk_ops xe_pt_stage_bind_ops = { · 3 unmodified lines
612676 * @entries: Storage for the update entries used for connecting the tree to
613677 * the main tree at commit time.
614678 * @num_entries: On output contains the number of @entries used.
679+ * @clear_pt: Clear the page table entries.
615680 *
616681 * This function builds a disconnected page-table tree for a given address
617682 * range. The tree is connected to the main vm tree for the gpu using
@@ -625,13 +690,13 @@ static const struct xe_pt_walk_ops xe_pt_stage_bind_ops = { · 7 unmodified lines
625690 static int
626691 xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,
627692 struct xe_svm_range *range,
628− struct xe_vm_pgtable_update *entries, u32 *num_entries)
693+ struct xe_vm_pgtable_update *entries,
694+ u32 *num_entries, bool clear_pt)
629695 {
630696 struct xe_device *xe = tile_to_xe(tile);
631697 struct xe_bo *bo = xe_vma_bo(vma);
632− bool is_devmem = !xe_vma_is_userptr(vma) && bo &&
633− (xe_bo_is_vram(bo) || xe_bo_is_stolen_devmem(bo));
634698 struct xe_res_cursor curs;
699+ struct xe_vm *vm = xe_vma_vm(vma);
635700 struct xe_pt_stage_bind_walk xe_walk = {
636701 .base = {
637702 .ops = &xe_pt_stage_bind_ops,
@@ -639,34 +704,31 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, · 1 unmodified line
639704 .max_level = XE_PT_HIGHEST_LEVEL,
640705 .staging = true,
641706 },
642− .vm = xe_vma_vm(vma),
707+ .vm = vm,
643708 .tile = tile,
644709 .curs = &curs,
645710 .va_curs_start = range ? range->base.itree.start :
646711 xe_vma_start(vma),
647712 .vma = vma,
648713 .wupd.entries = entries,
714+ .clear_pt = clear_pt,
649715 };
650− struct xe_pt *pt = xe_vma_vm(vma)->pt_root[tile->id];
716+ struct xe_pt *pt = vm->pt_root[tile->id];
651717 int ret;
652718
653719 if (range) {
654720 /* Move this entire thing to xe_svm.c? */
655− xe_svm_notifier_lock(xe_vma_vm(vma));
721+ xe_svm_notifier_lock(vm);
656722 if (!xe_svm_range_pages_valid(range)) {
657723 xe_svm_range_debug(range, "BIND PREPARE - RETRY");
658− xe_svm_notifier_unlock(xe_vma_vm(vma));
724+ xe_svm_notifier_unlock(vm);
659725 return -EAGAIN;
660726 }
661727 if (xe_svm_range_has_dma_mapping(range)) {
662728 xe_res_first_dma(range->base.dma_addr, 0,
663729 range->base.itree.last + 1 - range->base.itree.start,
664730 &curs);
665− is_devmem = xe_res_is_vram(&curs);
666− if (is_devmem)
667− xe_svm_range_debug(range, "BIND PREPARE - DMA VRAM");
668− else
669− xe_svm_range_debug(range, "BIND PREPARE - DMA");
731+ xe_svm_range_debug(range, "BIND PREPARE - MIXED");
670732 } else {
671733 xe_assert(xe, false);
672734 }
@@ -674,54 +736,21 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, · 1 unmodified line
674736 * Note, when unlocking the resource cursor dma addresses may become
675737 * stale, but the bind will be aborted anyway at commit time.
676738 */
677− xe_svm_notifier_unlock(xe_vma_vm(vma));
739+ xe_svm_notifier_unlock(vm);
678740 }
679741
680− xe_walk.needs_64K = (xe_vma_vm(vma)->flags & XE_VM_FLAG_64K) && is_devmem;
742+ xe_walk.needs_64K = (vm->flags & XE_VM_FLAG_64K);
743+ if (clear_pt)
744+ goto walk_pt;
681745
682− /**
683− * Default atomic expectations for different allocation scenarios are as follows:
684− *
685− * 1. Traditional API: When the VM is not in LR mode:
686− * - Device atomics are expected to function with all allocations.
687− *
688− * 2. Compute/SVM API: When the VM is in LR mode:
689− * - Device atomics are the default behavior when the bo is placed in a single region.
690− * - In all other cases device atomics will be disabled with AE=0 until an application
691− * request differently using a ioctl like madvise.
692− */
693746 if (vma->gpuva.flags & XE_VMA_ATOMIC_PTE_BIT) {
694− if (xe_vm_in_lr_mode(xe_vma_vm(vma))) {
695− if (bo && xe_bo_has_single_placement(bo))
696− xe_walk.default_pte |= XE_USM_PPGTT_PTE_AE;
697− /**
698− * If a SMEM+LMEM allocation is backed by SMEM, a device
699− * atomics will cause a gpu page fault and which then
700− * gets migrated to LMEM, bind such allocations with
701− * device atomics enabled.
702− */
703− else if (is_devmem)
704− xe_walk.default_pte |= XE_USM_PPGTT_PTE_AE;
705− } else {
706− xe_walk.default_pte |= XE_USM_PPGTT_PTE_AE;
707− }
708−
709− /**
710− * Unset AE if the platform(PVC) doesn't support it on an
711− * allocation
712− */
713− if (!xe->info.has_device_atomics_on_smem && !is_devmem)
714− xe_walk.default_pte &= ~XE_USM_PPGTT_PTE_AE;
715− }
716−
717− if (is_devmem) {
718− xe_walk.default_pte |= XE_PPGTT_PTE_DM;
719− xe_walk.dma_offset = bo ? vram_region_gpu_offset(bo->ttm.resource) : 0;
747+ xe_walk.default_vram_pte = xe_atomic_for_vram(vm) ? XE_USM_PPGTT_PTE_AE : 0;
748+ xe_walk.default_system_pte = xe_atomic_for_system(vm, bo) ?
749+ XE_USM_PPGTT_PTE_AE : 0;
720750 }
721751
722− if (!xe_vma_has_no_bo(vma) && xe_bo_is_stolen(bo))
723− xe_walk.dma_offset = xe_ttm_stolen_gpu_offset(xe_bo_device(bo));
724−
752+ xe_walk.default_vram_pte |= XE_PPGTT_PTE_DM;
753+ xe_walk.dma_offset = bo ? vram_region_gpu_offset(bo->ttm.resource) : 0;
725754 if (!range)
726755 xe_bo_assert_held(bo);
727756
@@ -739,6 +768,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, · 11 unmodified lines
739768 curs.size = xe_vma_size(vma);
740769 }
741770
771+walk_pt:
742772 ret = xe_pt_walk_range(&pt->base, pt->level,
743773 range ? range->base.itree.start : xe_vma_start(vma),
744774 range ? range->base.itree.last + 1 : xe_vma_end(vma),
@@ -1103,12 +1133,14 @@ static void xe_pt_free_bind(struct xe_vm_pgtable_update *entries, · 358 unmodified lines
11031133 static int
11041134 xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
11051135 struct xe_svm_range *range,
1106− struct xe_vm_pgtable_update *entries, u32 *num_entries)
1136+ struct xe_vm_pgtable_update *entries,
1137+ u32 *num_entries, bool invalidate_on_bind)
11071138 {
11081139 int err;
11091140
11101141 *num_entries = 0;
1111− err = xe_pt_stage_bind(tile, vma, range, entries, num_entries);
1142+ err = xe_pt_stage_bind(tile, vma, range, entries, num_entries,
1143+ invalidate_on_bind);
11121144 if (!err)
11131145 xe_tile_assert(tile, *num_entries);
11141146
@@ -1420,6 +1452,7 @@ static int xe_pt_userptr_pre_commit(struct xe_migrate_pt_update *pt_update) · 305 unmodified lines
14201452 return err;
14211453 }
14221454
1455+#if IS_ENABLED(CONFIG_DRM_XE_GPUSVM)
14231456 static int xe_pt_svm_pre_commit(struct xe_migrate_pt_update *pt_update)
14241457 {
14251458 struct xe_vm *vm = pt_update->vops->vm;
@@ -1453,6 +1486,7 @@ static int xe_pt_svm_pre_commit(struct xe_migrate_pt_update *pt_update) · 27 unmodified lines
14531486
14541487 return 0;
14551488 }
1489+#endif
14561490
14571491 struct invalidation_fence {
14581492 struct xe_gt_tlb_invalidation_fence base;
@@ -1791,7 +1825,7 @@ static int vma_reserve_fences(struct xe_device *xe, struct xe_vma *vma) · 332 unmodified lines
17911825
17921826 static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile,
17931827 struct xe_vm_pgtable_update_ops *pt_update_ops,
1794− struct xe_vma *vma)
1828+ struct xe_vma *vma, bool invalidate_on_bind)
17951829 {
17961830 u32 current_op = pt_update_ops->current_op;
17971831 struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[current_op];
@@ -1813,7 +1847,7 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile, · 15 unmodified lines
18131847 return err;
18141848
18151849 err = xe_pt_prepare_bind(tile, vma, NULL, pt_op->entries,
1816− &pt_op->num_entries);
1850+ &pt_op->num_entries, invalidate_on_bind);
18171851 if (!err) {
18181852 xe_tile_assert(tile, pt_op->num_entries <=
18191853 ARRAY_SIZE(pt_op->entries));
@@ -1835,11 +1869,11 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile, · 15 unmodified lines
18351869 * If !rebind, and scratch enabled VMs, there is a chance the scratch
18361870 * PTE is already cached in the TLB so it needs to be invalidated.
18371871 * On !LR VMs this is done in the ring ops preceding a batch, but on
1838− * non-faulting LR, in particular on user-space batch buffer chaining,
1839− * it needs to be done here.
1872+ * LR, in particular on user-space batch buffer chaining, it needs to
1873+ * be done here.
18401874 */
18411875 if ((!pt_op->rebind && xe_vm_has_scratch(vm) &&
1842− xe_vm_in_preempt_fence_mode(vm)))
1876+ xe_vm_in_lr_mode(vm)))
18431877 pt_update_ops->needs_invalidation = true;
18441878 else if (pt_op->rebind && !xe_vm_in_lr_mode(vm))
18451879 /* We bump also if batch_invalidate_tlb is true */
@@ -1875,7 +1909,7 @@ static int bind_range_prepare(struct xe_vm *vm, struct xe_tile *tile, · 29 unmodified lines
18751909 pt_op->rebind = BIT(tile->id) & range->tile_present;
18761910
18771911 err = xe_pt_prepare_bind(tile, vma, range, pt_op->entries,
1878− &pt_op->num_entries);
1912+ &pt_op->num_entries, false);
18791913 if (!err) {
18801914 xe_tile_assert(tile, pt_op->num_entries <=
18811915 ARRAY_SIZE(pt_op->entries));
@@ -1987,11 +2021,13 @@ static int op_prepare(struct xe_vm *vm, · 105 unmodified lines
19872021
19882022 switch (op->base.op) {
19892023 case DRM_GPUVA_OP_MAP:
1990− if ((!op->map.immediate && xe_vm_in_fault_mode(vm)) ||
2024+ if ((!op->map.immediate && xe_vm_in_fault_mode(vm) &&
2025+ !op->map.invalidate_on_bind) ||
19912026 op->map.is_cpu_addr_mirror)
19922027 break;
19932028
1994− err = bind_op_prepare(vm, tile, pt_update_ops, op->map.vma);
2029+ err = bind_op_prepare(vm, tile, pt_update_ops, op->map.vma,
2030+ op->map.invalidate_on_bind);
19952031 pt_update_ops->wait_vm_kernel = true;
19962032 break;
19972033 case DRM_GPUVA_OP_REMAP:
@@ -2005,12 +2041,12 @@ static int op_prepare(struct xe_vm *vm, · 7 unmodified lines
20052041
20062042 if (!err && op->remap.prev) {
20072043 err = bind_op_prepare(vm, tile, pt_update_ops,
2008− op->remap.prev);
2044+ op->remap.prev, false);
20092045 pt_update_ops->wait_vm_bookkeep = true;
20102046 }
20112047 if (!err && op->remap.next) {
20122048 err = bind_op_prepare(vm, tile, pt_update_ops,
2013− op->remap.next);
2049+ op->remap.next, false);
20142050 pt_update_ops->wait_vm_bookkeep = true;
20152051 }
20162052 break;
@@ -2032,7 +2068,7 @@ static int op_prepare(struct xe_vm *vm, · 15 unmodified lines
20322068 if (xe_vma_is_cpu_addr_mirror(vma))
20332069 break;
20342070
2035− err = bind_op_prepare(vm, tile, pt_update_ops, vma);
2071+ err = bind_op_prepare(vm, tile, pt_update_ops, vma, false);
20362072 pt_update_ops->wait_vm_kernel = true;
20372073 break;
20382074 }
@@ -2115,7 +2151,7 @@ ALLOW_ERROR_INJECTION(xe_pt_update_ops_prepare, ERRNO); · 76 unmodified lines
21152151 static void bind_op_commit(struct xe_vm *vm, struct xe_tile *tile,
21162152 struct xe_vm_pgtable_update_ops *pt_update_ops,
21172153 struct xe_vma *vma, struct dma_fence *fence,
2118− struct dma_fence *fence2)
2154+ struct dma_fence *fence2, bool invalidate_on_bind)
21192155 {
21202156 xe_tile_assert(tile, !xe_vma_is_cpu_addr_mirror(vma));
21212157
@@ -2132,6 +2168,8 @@ static void bind_op_commit(struct xe_vm *vm, struct xe_tile *tile, · 10 unmodified lines
21322168 }
21332169 vma->tile_present |= BIT(tile->id);
21342170 vma->tile_staged &= ~BIT(tile->id);
2171+ if (invalidate_on_bind)
2172+ vma->tile_invalidated |= BIT(tile->id);
21352173 if (xe_vma_is_userptr(vma)) {
21362174 lockdep_assert_held_read(&vm->userptr.notifier_lock);
21372175 to_userptr_vma(vma)->userptr.initial_bind = true;
@@ -2193,7 +2231,7 @@ static void op_commit(struct xe_vm *vm, · 55 unmodified lines
21932231 break;
21942232
21952233 bind_op_commit(vm, tile, pt_update_ops, op->map.vma, fence,
2196− fence2);
2234+ fence2, op->map.invalidate_on_bind);
21972235 break;
21982236 case DRM_GPUVA_OP_REMAP:
21992237 {
@@ -2206,10 +2244,10 @@ static void op_commit(struct xe_vm *vm, · 6 unmodified lines
22062244
22072245 if (op->remap.prev)
22082246 bind_op_commit(vm, tile, pt_update_ops, op->remap.prev,
2209− fence, fence2);
2247+ fence, fence2, false);
22102248 if (op->remap.next)
22112249 bind_op_commit(vm, tile, pt_update_ops, op->remap.next,
2212− fence, fence2);
2250+ fence, fence2, false);
22132251 break;
22142252 }
22152253 case DRM_GPUVA_OP_UNMAP:
@@ -2227,7 +2265,7 @@ static void op_commit(struct xe_vm *vm, · 11 unmodified lines
22272265
22282266 if (!xe_vma_is_cpu_addr_mirror(vma))
22292267 bind_op_commit(vm, tile, pt_update_ops, vma, fence,
2230− fence2);
2268+ fence2, false);
22312269 break;
22322270 }
22332271 case DRM_GPUVA_OP_DRIVER:
@@ -2257,11 +2295,15 @@ static const struct xe_migrate_pt_update_ops userptr_migrate_ops = { · 23 unmodified lines
22572295 .pre_commit = xe_pt_userptr_pre_commit,
22582296 };
22592297
2298+#if IS_ENABLED(CONFIG_DRM_XE_GPUSVM)
22602299 static const struct xe_migrate_pt_update_ops svm_migrate_ops = {
22612300 .populate = xe_vm_populate_pgtable,
22622301 .clear = xe_migrate_clear_pgtable_callback,
22632302 .pre_commit = xe_pt_svm_pre_commit,
22642303 };
2304+#else
2305+static const struct xe_migrate_pt_update_ops svm_migrate_ops;
2306+#endif
22652307
22662308 /**
22672309 * xe_pt_update_ops_run() - Run PT update operations
drivers/gpu/drm/xe/xe_query.c+1−1
@@ -340,7 +340,7 @@ static int query_config(struct xe_device *xe, struct drm_xe_device_query *query) · 339 unmodified lines
340340 if (xe_device_get_root_tile(xe)->mem.vram.usable_size)
341341 config->info[DRM_XE_QUERY_CONFIG_FLAGS] |=
342342 DRM_XE_QUERY_CONFIG_FLAG_HAS_VRAM;
343− if (xe->info.has_usm && IS_ENABLED(CONFIG_DRM_GPUSVM))
343+ if (xe->info.has_usm && IS_ENABLED(CONFIG_DRM_XE_GPUSVM))
344344 config->info[DRM_XE_QUERY_CONFIG_FLAGS] |=
345345 DRM_XE_QUERY_CONFIG_FLAG_HAS_CPU_ADDR_MIRROR;
346346 config->info[DRM_XE_QUERY_CONFIG_FLAGS] |=
drivers/gpu/drm/xe/xe_reg_sr.c+3−0
@@ -173,6 +173,9 @@ void xe_reg_sr_apply_mmio(struct xe_reg_sr *sr, struct xe_gt *gt) · 172 unmodified lines
173173 if (xa_empty(&sr->xa))
174174 return;
175175
176+ if (IS_SRIOV_VF(gt_to_xe(gt)))
177+ return;
178+
176179 xe_gt_dbg(gt, "Applying %s save-restore MMIOs\n", sr->name);
177180
178181 fw_ref = xe_force_wake_get(gt_to_fw(gt), XE_FORCEWAKE_ALL);
drivers/gpu/drm/xe/xe_ring_ops_types.h+1−1
@@ -8,7 +8,7 @@ · 7 unmodified lines
88
99 struct xe_sched_job;
1010
11−#define MAX_JOB_SIZE_DW 48
11+#define MAX_JOB_SIZE_DW 58
1212 #define MAX_JOB_SIZE_BYTES (MAX_JOB_SIZE_DW * 4)
1313
1414 /**
drivers/gpu/drm/xe/xe_rtp.c+0−3
@@ -258,9 +258,6 @@ void xe_rtp_process_to_sr(struct xe_rtp_process_ctx *ctx, · 257 unmodified lines
258258
259259 rtp_get_context(ctx, &hwe, &gt, &xe);
260260
261− if (IS_SRIOV_VF(xe))
262− return;
263−
264261 xe_assert(xe, entries);
265262
266263 for (entry = entries; entry - entries < n_entries; entry++) {
drivers/gpu/drm/xe/xe_sa.c+2−1
@@ -60,7 +60,8 @@ struct xe_sa_manager *__xe_sa_bo_manager_init(struct xe_tile *tile, u32 size, u3 · 59 unmodified lines
6060 bo = xe_managed_bo_create_pin_map(xe, tile, size,
6161 XE_BO_FLAG_VRAM_IF_DGFX(tile) |
6262 XE_BO_FLAG_GGTT |
63− XE_BO_FLAG_GGTT_INVALIDATE);
63+ XE_BO_FLAG_GGTT_INVALIDATE |
64+ XE_BO_FLAG_PINNED_NORESTORE);
6465 if (IS_ERR(bo)) {
6566 drm_err(&xe->drm, "Failed to prepare %uKiB BO for SA manager (%pe)\n",
6667 size / SZ_1K, bo);
drivers/gpu/drm/xe/xe_survivability_mode.c+50−19
@@ -10,6 +10,7 @@ · 9 unmodified lines
1010 #include <linux/pci.h>
1111 #include <linux/sysfs.h>
1212
13+#include "xe_configfs.h"
1314 #include "xe_device.h"
1415 #include "xe_gt.h"
1516 #include "xe_heci_gsc.h"
@@ -28,20 +29,32 @@ · 12 unmodified lines
2829 * This is implemented by loading the driver with bare minimum (no drm card) to allow the firmware
2930 * to be flashed through mei and collect telemetry. The driver's probe flow is modified
3031 * such that it enters survivability mode when pcode initialization is incomplete and boot status
31− * denotes a failure. The driver then populates the survivability_mode PCI sysfs indicating
32− * survivability mode and provides additional information required for debug
32+ * denotes a failure.
3333 *
34− * KMD exposes below admin-only readable sysfs in survivability mode
34+ * Survivability mode can also be entered manually using the survivability mode attribute available
35+ * through configfs which is beneficial in several usecases. It can be used to address scenarios
36+ * where pcode does not detect failure or for validation purposes. It can also be used in
37+ * In-Field-Repair (IFR) to repair a single card without impacting the other cards in a node.
3538 *
36− * device/survivability_mode: The presence of this file indicates that the card is in survivability
37− * mode. Also, provides additional information on why the driver entered
38− * survivability mode.
39+ * Use below command enable survivability mode manually::
3940 *
40− * Capability Information - Provides boot status
41− * Postcode Information - Provides information about the failure
42− * Overflow Information - Provides history of previous failures
43− * Auxiliary Information - Certain failures may have information in
44− * addition to postcode information
41+ * # echo 1 > /sys/kernel/config/xe/0000:03:00.0/survivability_mode
42+ *
43+ * Refer :ref:`xe_configfs` for more details on how to use configfs
44+ *
45+ * Survivability mode is indicated by the below admin-only readable sysfs which provides additional
46+ * debug information::
47+ *
48+ * /sys/bus/pci/devices/<device>/surivability_mode
49+ *
50+ * Capability Information:
51+ * Provides boot status
52+ * Postcode Information:
53+ * Provides information about the failure
54+ * Overflow Information
55+ * Provides history of previous failures
56+ * Auxiliary Information
57+ * Certain failures may have information in addition to postcode information
4558 */
4659
4760 static u32 aux_history_offset(u32 reg_value)
@@ -133,6 +146,7 @@ static void xe_survivability_mode_fini(void *arg) · 85 unmodified lines
133146 struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
134147 struct device *dev = &pdev->dev;
135148
149+ xe_configfs_clear_survivability_mode(pdev);
136150 sysfs_remove_file(&dev->kobj, &dev_attr_survivability_mode.attr);
137151 }
138152
@@ -186,24 +200,41 @@ bool xe_survivability_mode_is_enabled(struct xe_device *xe) · 47 unmodified lines
186200 return xe->survivability.mode;
187201 }
188202
189−/*
190− * survivability_mode_requested - check if it's possible to enable
191− * survivability mode and that was requested by firmware
203+/**
204+ * xe_survivability_mode_is_requested - check if it's possible to enable survivability
205+ * mode that was requested by firmware or userspace
206+ * @xe: xe device instance
192207 *
193− * This function reads the boot status from Pcode.
208+ * This function reads configfs and boot status from Pcode.
194209 *
195210 * Return: true if platform support is available and boot status indicates
196− * failure, false otherwise.
211+ * failure or if survivability mode is requested, false otherwise.
197212 */
198−static bool survivability_mode_requested(struct xe_device *xe)
213+bool xe_survivability_mode_is_requested(struct xe_device *xe)
199214 {
200215 struct xe_survivability *survivability = &xe->survivability;
201216 struct xe_mmio *mmio = xe_root_tile_mmio(xe);
217+ struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
202218 u32 data;
219+ bool survivability_mode;
203220
204− if (!IS_DGFX(xe) || xe->info.platform < XE_BATTLEMAGE || IS_SRIOV_VF(xe))
221+ if (!IS_DGFX(xe) || IS_SRIOV_VF(xe))
205222 return false;
206223
224+ survivability_mode = xe_configfs_get_survivability_mode(pdev);
225+
226+ if (xe->info.platform < XE_BATTLEMAGE) {
227+ if (survivability_mode) {
228+ dev_err(&pdev->dev, "Survivability Mode is not supported on this card\n");
229+ xe_configfs_clear_survivability_mode(pdev);
230+ }
231+ return false;
232+ }
233+
234+ /* Enable survivability mode if set via configfs */
235+ if (survivability_mode)
236+ return true;
237+
207238 data = xe_mmio_read32(mmio, PCODE_SCRATCH(0));
208239 survivability->boot_status = REG_FIELD_GET(BOOT_STATUS, data);
209240
@@ -226,7 +257,7 @@ int xe_survivability_mode_enable(struct xe_device *xe) · 16 unmodified lines
226257 struct xe_survivability_info *info;
227258 struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
228259
229− if (!survivability_mode_requested(xe))
260+ if (!xe_survivability_mode_is_requested(xe))
230261 return 0;
231262
232263 survivability->size = MAX_SCRATCH_MMIO;
drivers/gpu/drm/xe/xe_survivability_mode.h+1−0
@@ -12,5 +12,6 @@ struct xe_device; · 11 unmodified lines
1212
1313 int xe_survivability_mode_enable(struct xe_device *xe);
1414 bool xe_survivability_mode_is_enabled(struct xe_device *xe);
15+bool xe_survivability_mode_is_requested(struct xe_device *xe);
1516
1617 #endif /* _XE_SURVIVABILITY_MODE_H_ */
drivers/gpu/drm/xe/xe_svm.c+21−2
@@ -4,6 +4,7 @@ · 3 unmodified lines
44 */
55
66 #include "xe_bo.h"
7+#include "xe_gt_stats.h"
78 #include "xe_gt_tlb_invalidation.h"
89 #include "xe_migrate.h"
910 #include "xe_module.h"
@@ -339,6 +340,8 @@ static void xe_svm_garbage_collector_work_func(struct work_struct *w) · 329 unmodified lines
339340 up_write(&vm->lock);
340341 }
341342
343+#if IS_ENABLED(CONFIG_DRM_XE_DEVMEM_MIRROR)
344+
342345 static struct xe_vram_region *page_to_vr(struct page *page)
343346 {
344347 return container_of(page_pgmap(page), struct xe_vram_region, pagemap);
@@ -577,6 +580,8 @@ static const struct drm_gpusvm_devmem_ops gpusvm_devmem_ops = { · 232 unmodified lines
577580 .copy_to_ram = xe_svm_copy_to_ram,
578581 };
579582
583+#endif
584+
580585 static const struct drm_gpusvm_ops gpusvm_ops = {
581586 .range_alloc = xe_svm_range_alloc,
582587 .range_free = xe_svm_range_free,
@@ -650,6 +655,7 @@ static bool xe_svm_range_is_valid(struct xe_svm_range *range, · 67 unmodified lines
650655 return (range->tile_present & ~range->tile_invalidated) & BIT(tile->id);
651656 }
652657
658+#if IS_ENABLED(CONFIG_DRM_XE_DEVMEM_MIRROR)
653659 static struct xe_vram_region *tile_to_vr(struct xe_tile *tile)
654660 {
655661 return &tile->mem.vram;
@@ -711,12 +717,21 @@ unlock: · 55 unmodified lines
711717
712718 return err;
713719 }
720+#else
721+static int xe_svm_alloc_vram(struct xe_vm *vm, struct xe_tile *tile,
722+ struct xe_svm_range *range,
723+ const struct drm_gpusvm_ctx *ctx)
724+{
725+ return -EOPNOTSUPP;
726+}
727+#endif
728+
714729
715730 /**
716731 * xe_svm_handle_pagefault() - SVM handle page fault
717732 * @vm: The VM.
718733 * @vma: The CPU address mirror VMA.
719− * @tile: The tile upon the fault occurred.
734+ * @gt: The gt upon the fault occurred.
720735 * @fault_addr: The GPU fault address.
721736 * @atomic: The fault atomic access bit.
722737 *
@@ -726,7 +741,7 @@ unlock: · 3 unmodified lines
726741 * Return: 0 on success, negative error code on error.
727742 */
728743 int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma,
729− struct xe_tile *tile, u64 fault_addr,
744+ struct xe_gt *gt, u64 fault_addr,
730745 bool atomic)
731746 {
732747 struct drm_gpusvm_ctx ctx = {
@@ -740,12 +755,15 @@ int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma, · 7 unmodified lines
740755 struct drm_gpusvm_range *r;
741756 struct drm_exec exec;
742757 struct dma_fence *fence;
758+ struct xe_tile *tile = gt_to_tile(gt);
743759 ktime_t end = 0;
744760 int err;
745761
746762 lockdep_assert_held_write(&vm->lock);
747763 xe_assert(vm->xe, xe_vma_is_cpu_addr_mirror(vma));
748764
765+ xe_gt_stats_incr(gt, XE_GT_STATS_ID_SVM_PAGEFAULT_COUNT, 1);
766+
749767 retry:
750768 /* Always process UNMAPs first so view SVM ranges is current */
751769 err = xe_svm_garbage_collector(vm);
@@ -866,6 +884,7 @@ int xe_svm_bo_evict(struct xe_bo *bo) · 114 unmodified lines
866884 }
867885
868886 #if IS_ENABLED(CONFIG_DRM_XE_DEVMEM_MIRROR)
887+
869888 static struct drm_pagemap_device_addr
870889 xe_drm_pagemap_device_map(struct drm_pagemap *dpagemap,
871890 struct device *dev,
drivers/gpu/drm/xe/xe_svm.h+57−23
@@ -6,16 +6,19 @@ · 5 unmodified lines
66 #ifndef _XE_SVM_H_
77 #define _XE_SVM_H_
88
9+#if IS_ENABLED(CONFIG_DRM_XE_GPUSVM)
10+
911 #include <drm/drm_pagemap.h>
1012 #include <drm/drm_gpusvm.h>
1113
1214 #define XE_INTERCONNECT_VRAM DRM_INTERCONNECT_DRIVER
1315
1416 struct xe_bo;
15−struct xe_vram_region;
17+struct xe_gt;
1618 struct xe_tile;
1719 struct xe_vm;
1820 struct xe_vma;
21+struct xe_vram_region;
1922
2023 /** struct xe_svm_range - SVM range */
2124 struct xe_svm_range {
@@ -43,7 +46,6 @@ struct xe_svm_range { · 21 unmodified lines
4346 u8 skip_migrate :1;
4447 };
4548
46−#if IS_ENABLED(CONFIG_DRM_GPUSVM)
4749 /**
4850 * xe_svm_range_pages_valid() - SVM range pages valid
4951 * @range: SVM range
@@ -64,7 +66,7 @@ void xe_svm_fini(struct xe_vm *vm); · 14 unmodified lines
6466 void xe_svm_close(struct xe_vm *vm);
6567
6668 int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma,
67− struct xe_tile *tile, u64 fault_addr,
69+ struct xe_gt *gt, u64 fault_addr,
6870 bool atomic);
6971
7072 bool xe_svm_has_mapping(struct xe_vm *vm, u64 start, u64 end);
@@ -72,7 +74,50 @@ bool xe_svm_has_mapping(struct xe_vm *vm, u64 start, u64 end); · 1 unmodified line
7274 int xe_svm_bo_evict(struct xe_bo *bo);
7375
7476 void xe_svm_range_debug(struct xe_svm_range *range, const char *operation);
77+
78+/**
79+ * xe_svm_range_has_dma_mapping() - SVM range has DMA mapping
80+ * @range: SVM range
81+ *
82+ * Return: True if SVM range has a DMA mapping, False otherwise
83+ */
84+static inline bool xe_svm_range_has_dma_mapping(struct xe_svm_range *range)
85+{
86+ lockdep_assert_held(&range->base.gpusvm->notifier_lock);
87+ return range->base.flags.has_dma_mapping;
88+}
89+
90+#define xe_svm_assert_in_notifier(vm__) \
91+ lockdep_assert_held_write(&(vm__)->svm.gpusvm.notifier_lock)
92+
93+#define xe_svm_notifier_lock(vm__) \
94+ drm_gpusvm_notifier_lock(&(vm__)->svm.gpusvm)
95+
96+#define xe_svm_notifier_unlock(vm__) \
97+ drm_gpusvm_notifier_unlock(&(vm__)->svm.gpusvm)
98+
7599 #else
100+#include <linux/interval_tree.h>
101+
102+struct drm_pagemap_device_addr;
103+struct xe_bo;
104+struct xe_gt;
105+struct xe_vm;
106+struct xe_vma;
107+struct xe_tile;
108+struct xe_vram_region;
109+
110+#define XE_INTERCONNECT_VRAM 1
111+
112+struct xe_svm_range {
113+ struct {
114+ struct interval_tree_node itree;
115+ const struct drm_pagemap_device_addr *dma_addr;
116+ } base;
117+ u32 tile_present;
118+ u32 tile_invalidated;
119+};
120+
76121 static inline bool xe_svm_range_pages_valid(struct xe_svm_range *range)
77122 {
78123 return false;
@@ -102,7 +147,7 @@ void xe_svm_close(struct xe_vm *vm) · 23 unmodified lines
102147
103148 static inline
104149 int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma,
105− struct xe_tile *tile, u64 fault_addr,
150+ struct xe_gt *gt, u64 fault_addr,
106151 bool atomic)
107152 {
108153 return 0;
@@ -124,27 +169,16 @@ static inline · 15 unmodified lines
124169 void xe_svm_range_debug(struct xe_svm_range *range, const char *operation)
125170 {
126171 }
127−#endif
128172
129−/**
130− * xe_svm_range_has_dma_mapping() - SVM range has DMA mapping
131− * @range: SVM range
132− *
133− * Return: True if SVM range has a DMA mapping, False otherwise
134− */
135−static inline bool xe_svm_range_has_dma_mapping(struct xe_svm_range *range)
173+#define xe_svm_assert_in_notifier(...) do {} while (0)
174+#define xe_svm_range_has_dma_mapping(...) false
175+
176+static inline void xe_svm_notifier_lock(struct xe_vm *vm)
136177 {
137− lockdep_assert_held(&range->base.gpusvm->notifier_lock);
138− return range->base.flags.has_dma_mapping;
139178 }
140179
141−#define xe_svm_assert_in_notifier(vm__) \
142− lockdep_assert_held_write(&(vm__)->svm.gpusvm.notifier_lock)
143−
144−#define xe_svm_notifier_lock(vm__) \
145− drm_gpusvm_notifier_lock(&(vm__)->svm.gpusvm)
146−
147−#define xe_svm_notifier_unlock(vm__) \
148− drm_gpusvm_notifier_unlock(&(vm__)->svm.gpusvm)
149−
180+static inline void xe_svm_notifier_unlock(struct xe_vm *vm)
181+{
182+}
183+#endif
150184 #endif
drivers/gpu/drm/xe/xe_uc_fw.c+55−39
@@ -92,6 +92,8 @@ · 91 unmodified lines
9292
9393 struct uc_fw_entry {
9494 enum xe_platform platform;
95+ enum xe_gt_type gt_type;
96+
9597 struct {
9698 const char *path;
9799 u16 major;
@@ -106,32 +108,37 @@ struct fw_blobs_by_type { · 8 unmodified lines
106108 u32 count;
107109 };
108110
109−#define XE_GUC_FIRMWARE_DEFS(fw_def, mmp_ver, major_ver) \
110− fw_def(BATTLEMAGE, major_ver(xe, guc, bmg, 70, 29, 2)) \
111− fw_def(LUNARLAKE, major_ver(xe, guc, lnl, 70, 29, 2)) \
112− fw_def(METEORLAKE, major_ver(i915, guc, mtl, 70, 29, 2)) \
113− fw_def(DG2, major_ver(i915, guc, dg2, 70, 29, 2)) \
114− fw_def(DG1, major_ver(i915, guc, dg1, 70, 29, 2)) \
115− fw_def(ALDERLAKE_N, major_ver(i915, guc, tgl, 70, 29, 2)) \
116− fw_def(ALDERLAKE_P, major_ver(i915, guc, adlp, 70, 29, 2)) \
117− fw_def(ALDERLAKE_S, major_ver(i915, guc, tgl, 70, 29, 2)) \
118− fw_def(ROCKETLAKE, major_ver(i915, guc, tgl, 70, 29, 2)) \
119− fw_def(TIGERLAKE, major_ver(i915, guc, tgl, 70, 29, 2))
111+/*
112+ * Add an "ANY" define just to convey the meaning it's given here.
113+ */
114+#define XE_GT_TYPE_ANY XE_GT_TYPE_UNINITIALIZED
115+
116+#define XE_GUC_FIRMWARE_DEFS(fw_def, mmp_ver, major_ver) \
117+ fw_def(BATTLEMAGE, GT_TYPE_ANY, major_ver(xe, guc, bmg, 70, 44, 1)) \
118+ fw_def(LUNARLAKE, GT_TYPE_ANY, major_ver(xe, guc, lnl, 70, 44, 1)) \
119+ fw_def(METEORLAKE, GT_TYPE_ANY, major_ver(i915, guc, mtl, 70, 44, 1)) \
120+ fw_def(DG2, GT_TYPE_ANY, major_ver(i915, guc, dg2, 70, 44, 1)) \
121+ fw_def(DG1, GT_TYPE_ANY, major_ver(i915, guc, dg1, 70, 44, 1)) \
122+ fw_def(ALDERLAKE_N, GT_TYPE_ANY, major_ver(i915, guc, tgl, 70, 44, 1)) \
123+ fw_def(ALDERLAKE_P, GT_TYPE_ANY, major_ver(i915, guc, adlp, 70, 44, 1)) \
124+ fw_def(ALDERLAKE_S, GT_TYPE_ANY, major_ver(i915, guc, tgl, 70, 44, 1)) \
125+ fw_def(ROCKETLAKE, GT_TYPE_ANY, major_ver(i915, guc, tgl, 70, 44, 1)) \
126+ fw_def(TIGERLAKE, GT_TYPE_ANY, major_ver(i915, guc, tgl, 70, 44, 1))
120127
121128 #define XE_HUC_FIRMWARE_DEFS(fw_def, mmp_ver, no_ver) \
122− fw_def(BATTLEMAGE, no_ver(xe, huc, bmg)) \
123− fw_def(LUNARLAKE, no_ver(xe, huc, lnl)) \
124− fw_def(METEORLAKE, no_ver(i915, huc_gsc, mtl)) \
125− fw_def(DG1, no_ver(i915, huc, dg1)) \
126− fw_def(ALDERLAKE_P, no_ver(i915, huc, tgl)) \
127− fw_def(ALDERLAKE_S, no_ver(i915, huc, tgl)) \
128− fw_def(ROCKETLAKE, no_ver(i915, huc, tgl)) \
129− fw_def(TIGERLAKE, no_ver(i915, huc, tgl))
129+ fw_def(BATTLEMAGE, GT_TYPE_ANY, no_ver(xe, huc, bmg)) \
130+ fw_def(LUNARLAKE, GT_TYPE_ANY, no_ver(xe, huc, lnl)) \
131+ fw_def(METEORLAKE, GT_TYPE_ANY, no_ver(i915, huc_gsc, mtl)) \
132+ fw_def(DG1, GT_TYPE_ANY, no_ver(i915, huc, dg1)) \
133+ fw_def(ALDERLAKE_P, GT_TYPE_ANY, no_ver(i915, huc, tgl)) \
134+ fw_def(ALDERLAKE_S, GT_TYPE_ANY, no_ver(i915, huc, tgl)) \
135+ fw_def(ROCKETLAKE, GT_TYPE_ANY, no_ver(i915, huc, tgl)) \
136+ fw_def(TIGERLAKE, GT_TYPE_ANY, no_ver(i915, huc, tgl))
130137
131138 /* for the GSC FW we match the compatibility version and not the release one */
132139 #define XE_GSC_FIRMWARE_DEFS(fw_def, major_ver) \
133− fw_def(LUNARLAKE, major_ver(xe, gsc, lnl, 104, 1, 0)) \
134− fw_def(METEORLAKE, major_ver(i915, gsc, mtl, 102, 1, 0))
140+ fw_def(LUNARLAKE, GT_TYPE_ANY, major_ver(xe, gsc, lnl, 104, 1, 0)) \
141+ fw_def(METEORLAKE, GT_TYPE_ANY, major_ver(i915, gsc, mtl, 102, 1, 0))
135142
136143 #define MAKE_FW_PATH(dir__, uc__, shortname__, version__) \
137144 __stringify(dir__) "/" __stringify(shortname__) "_" __stringify(uc__) version__ ".bin"
@@ -159,12 +166,13 @@ struct fw_blobs_by_type { · 21 unmodified lines
159166 a, b, c }
160167
161168 /* All blobs need to be declared via MODULE_FIRMWARE() */
162−#define XE_UC_MODULE_FIRMWARE(platform__, fw_filename) \
169+#define XE_UC_MODULE_FIRMWARE(platform__, gt_type__, fw_filename) \
163170 MODULE_FIRMWARE(fw_filename);
164171
165−#define XE_UC_FW_ENTRY(platform__, entry__) \
172+#define XE_UC_FW_ENTRY(platform__, gt_type__, entry__) \
166173 { \
167174 .platform = XE_ ## platform__, \
175+ .gt_type = XE_ ## gt_type__, \
168176 entry__, \
169177 },
170178
@@ -222,30 +230,38 @@ uc_fw_auto_select(struct xe_device *xe, struct xe_uc_fw *uc_fw) · 51 unmodified lines
222230 [XE_UC_FW_TYPE_HUC] = { entries_huc, ARRAY_SIZE(entries_huc) },
223231 [XE_UC_FW_TYPE_GSC] = { entries_gsc, ARRAY_SIZE(entries_gsc) },
224232 };
225− static const struct uc_fw_entry *entries;
233+ struct xe_gt *gt = uc_fw_to_gt(uc_fw);
226234 enum xe_platform p = xe->info.platform;
235+ const struct uc_fw_entry *entries;
227236 u32 count;
228237 int i;
229238
230− xe_assert(xe, uc_fw->type < ARRAY_SIZE(blobs_all));
239+ xe_gt_assert(gt, uc_fw->type < ARRAY_SIZE(blobs_all));
240+ xe_gt_assert(gt, gt->info.type != XE_GT_TYPE_UNINITIALIZED);
241+
231242 entries = blobs_all[uc_fw->type].entries;
232243 count = blobs_all[uc_fw->type].count;
233244
234245 for (i = 0; i < count && p <= entries[i].platform; i++) {
235− if (p == entries[i].platform) {
236− uc_fw->path = entries[i].path;
237− uc_fw->versions.wanted.major = entries[i].major;
238− uc_fw->versions.wanted.minor = entries[i].minor;
239− uc_fw->versions.wanted.patch = entries[i].patch;
240− uc_fw->full_ver_required = entries[i].full_ver_required;
241−
242− if (uc_fw->type == XE_UC_FW_TYPE_GSC)
243− uc_fw->versions.wanted_type = XE_UC_FW_VER_COMPATIBILITY;
244− else
245− uc_fw->versions.wanted_type = XE_UC_FW_VER_RELEASE;
246−
247− break;
248− }
246+ if (p != entries[i].platform)
247+ continue;
248+
249+ if (entries[i].gt_type != XE_GT_TYPE_ANY &&
250+ entries[i].gt_type != gt->info.type)
251+ continue;
252+
253+ uc_fw->path = entries[i].path;
254+ uc_fw->versions.wanted.major = entries[i].major;
255+ uc_fw->versions.wanted.minor = entries[i].minor;
256+ uc_fw->versions.wanted.patch = entries[i].patch;
257+ uc_fw->full_ver_required = entries[i].full_ver_required;
258+
259+ if (uc_fw->type == XE_UC_FW_TYPE_GSC)
260+ uc_fw->versions.wanted_type = XE_UC_FW_VER_COMPATIBILITY;
261+ else
262+ uc_fw->versions.wanted_type = XE_UC_FW_VER_RELEASE;
263+
264+ break;
249265 }
250266 }
251267
drivers/gpu/drm/xe/xe_vm.c+29−10
@@ -2049,7 +2049,8 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data, · 2048 unmodified lines
20492049 return -EINVAL;
20502050
20512051 if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE &&
2052− args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE))
2052+ args->flags & DRM_XE_VM_CREATE_FLAG_FAULT_MODE &&
2053+ !xe->info.needs_scratch))
20532054 return -EINVAL;
20542055
20552056 if (XE_IOCTL_DBG(xe, !(args->flags & DRM_XE_VM_CREATE_FLAG_LR_MODE) &&
@@ -2201,6 +2202,20 @@ static void print_op(struct xe_device *xe, struct drm_gpuva_op *op) · 145 unmodified lines
22012202 }
22022203 #endif
22032204
2205+static bool __xe_vm_needs_clear_scratch_pages(struct xe_vm *vm, u32 bind_flags)
2206+{
2207+ if (!xe_vm_in_fault_mode(vm))
2208+ return false;
2209+
2210+ if (!xe_vm_has_scratch(vm))
2211+ return false;
2212+
2213+ if (bind_flags & DRM_XE_VM_BIND_FLAG_IMMEDIATE)
2214+ return false;
2215+
2216+ return true;
2217+}
2218+
22042219 /*
22052220 * Create operations list from IOCTL arguments, setup operations fields so parse
22062221 * and commit steps are decoupled from IOCTL arguments. This step can fail.
@@ -2273,6 +2288,8 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo, · 66 unmodified lines
22732288 DRM_XE_VM_BIND_FLAG_CPU_ADDR_MIRROR;
22742289 op->map.dumpable = flags & DRM_XE_VM_BIND_FLAG_DUMPABLE;
22752290 op->map.pat_index = pat_index;
2291+ op->map.invalidate_on_bind =
2292+ __xe_vm_needs_clear_scratch_pages(vm, flags);
22762293 } else if (__op->op == DRM_GPUVA_OP_PREFETCH) {
22772294 op->prefetch.region = prefetch_region;
22782295 }
@@ -2472,8 +2489,9 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct drm_gpuva_ops *ops, · 193 unmodified lines
24722489 return PTR_ERR(vma);
24732490
24742491 op->map.vma = vma;
2475− if ((op->map.immediate || !xe_vm_in_fault_mode(vm)) &&
2476− !op->map.is_cpu_addr_mirror)
2492+ if (((op->map.immediate || !xe_vm_in_fault_mode(vm)) &&
2493+ !op->map.is_cpu_addr_mirror) ||
2494+ op->map.invalidate_on_bind)
24772495 xe_vma_ops_incr_pt_update_ops(vops,
24782496 op->tile_mask);
24792497 break;
@@ -2726,9 +2744,10 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, · 246 unmodified lines
27262744
27272745 switch (op->base.op) {
27282746 case DRM_GPUVA_OP_MAP:
2729− err = vma_lock_and_validate(exec, op->map.vma,
2730− !xe_vm_in_fault_mode(vm) ||
2731− op->map.immediate);
2747+ if (!op->map.invalidate_on_bind)
2748+ err = vma_lock_and_validate(exec, op->map.vma,
2749+ !xe_vm_in_fault_mode(vm) ||
2750+ op->map.immediate);
27322751 break;
27332752 case DRM_GPUVA_OP_REMAP:
27342753 err = check_ufence(gpuva_to_vma(op->base.remap.unmap->va));
@@ -3109,7 +3128,7 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe, struct xe_vm *vm, · 374 unmodified lines
31093128
31103129 if (XE_IOCTL_DBG(xe, is_cpu_addr_mirror &&
31113130 (!xe_vm_in_fault_mode(vm) ||
3112− !IS_ENABLED(CONFIG_DRM_GPUSVM)))) {
3131+ !IS_ENABLED(CONFIG_DRM_XE_GPUSVM)))) {
31133132 err = -EINVAL;
31143133 goto free_bind_ops;
31153134 }
@@ -3243,7 +3262,7 @@ static int xe_vm_bind_ioctl_validate_bo(struct xe_device *xe, struct xe_bo *bo, · 127 unmodified lines
32433262 XE_64K_PAGE_MASK) ||
32443263 XE_IOCTL_DBG(xe, addr & XE_64K_PAGE_MASK) ||
32453264 XE_IOCTL_DBG(xe, range & XE_64K_PAGE_MASK)) {
3246− return -EINVAL;
3265+ return -EINVAL;
32473266 }
32483267 }
32493268
@@ -3251,7 +3270,7 @@ static int xe_vm_bind_ioctl_validate_bo(struct xe_device *xe, struct xe_bo *bo, · 1 unmodified line
32513270 if (bo->cpu_caching) {
32523271 if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE &&
32533272 bo->cpu_caching == DRM_XE_GEM_CPU_CACHING_WB)) {
3254− return -EINVAL;
3273+ return -EINVAL;
32553274 }
32563275 } else if (XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE)) {
32573276 /*
@@ -3260,7 +3279,7 @@ static int xe_vm_bind_ioctl_validate_bo(struct xe_device *xe, struct xe_bo *bo, · 2 unmodified lines
32603279 * how it was mapped on the CPU. Just assume is it
32613280 * potentially cached on CPU side.
32623281 */
3263− return -EINVAL;
3282+ return -EINVAL;
32643283 }
32653284
32663285 /* If a BO is protected it can only be mapped if the key is still valid */
drivers/gpu/drm/xe/xe_vm_types.h+2−0
@@ -330,6 +330,8 @@ struct xe_vma_op_map { · 329 unmodified lines
330330 bool is_cpu_addr_mirror;
331331 /** @dumpable: whether BO is dumped on GPU hang */
332332 bool dumpable;
333+ /** @invalidate: invalidate the VMA before bind */
334+ bool invalidate_on_bind;
333335 /** @pat_index: The pat index to use for this operation. */
334336 u16 pat_index;
335337 };
drivers/gpu/drm/xe/xe_vram.c+4−1
@@ -49,7 +49,7 @@ _resize_bar(struct xe_device *xe, int resno, resource_size_t size) · 48 unmodified lines
4949 */
5050 static void resize_vram_bar(struct xe_device *xe)
5151 {
52− u64 force_vram_bar_size = xe_modparam.force_vram_bar_size;
52+ int force_vram_bar_size = xe_modparam.force_vram_bar_size;
5353 struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
5454 struct pci_bus *root = pdev->bus;
5555 resource_size_t current_size;
@@ -66,6 +66,9 @@ static void resize_vram_bar(struct xe_device *xe) · 10 unmodified lines
6666 if (!bar_size_mask)
6767 return;
6868
69+ if (force_vram_bar_size < 0)
70+ return;
71+
6972 /* set to a specific size? */
7073 if (force_vram_bar_size) {
7174 u32 bar_size_bit;
drivers/gpu/drm/xe/xe_wa.c+12−0
@@ -230,6 +230,18 @@ static const struct xe_rtp_entry_sr gt_was[] = { · 229 unmodified lines
230230 XE_RTP_ENTRY_FLAG(FOREACH_ENGINE),
231231 },
232232
233+ /* Xe2_HPG */
234+
235+ { XE_RTP_NAME("16025250150"),
236+ XE_RTP_RULES(GRAPHICS_VERSION(2001)),
237+ XE_RTP_ACTIONS(SET(LSN_VC_REG2,
238+ LSN_LNI_WGT(1) |
239+ LSN_LNE_WGT(1) |
240+ LSN_DIM_X_WGT(1) |
241+ LSN_DIM_Y_WGT(1) |
242+ LSN_DIM_Z_WGT(1)))
243+ },
244+
233245 /* Xe2_HPM */
234246
235247 { XE_RTP_NAME("16021867713"),
drivers/gpu/drm/xe/xe_wa_oob.rules+2−0
@@ -57,3 +57,5 @@ no_media_l3 MEDIA_VERSION(3000) · 56 unmodified lines
5757 GRAPHICS_VERSION(1260), GRAPHICS_STEP(A0, B0)
5858 16023105232 GRAPHICS_VERSION_RANGE(2001, 3001)
5959 MEDIA_VERSION_RANGE(1301, 3000)
60+16026508708 GRAPHICS_VERSION_RANGE(1200, 3001)
61+ MEDIA_VERSION_RANGE(1300, 3000)
include/uapi/drm/xe_drm.h+5−1
@@ -917,7 +917,11 @@ struct drm_xe_gem_mmap_offset { · 916 unmodified lines
917917 * struct drm_xe_vm_create - Input of &DRM_IOCTL_XE_VM_CREATE
918918 *
919919 * The @flags can be:
920− * - %DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE
920+ * - %DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE - Map the whole virtual address
921+ * space of the VM to scratch page. A vm_bind would overwrite the scratch
922+ * page mapping. This flag is mutually exclusive with the
923+ * %DRM_XE_VM_CREATE_FLAG_FAULT_MODE flag, with an exception of on x2 and
924+ * xe3 platform.
921925 * - %DRM_XE_VM_CREATE_FLAG_LR_MODE - An LR, or Long Running VM accepts
922926 * exec submissions to its exec_queues that don't have an upper time
923927 * limit on the job execution time. But exec submissions to these

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t