History

Merge tag 'drm-next-2025-12-03' of https://gitlab.freedesktop.org/drm/kernel

Pull drm updates from Dave Airlie: "There was a rather late merge of a new color pipeline feature, that some userspace projects are blocked on, and has seen a lot of work in amdgpu. This should have seen some time in -next. There is additional support for this for Intel, that if it arrives in the next day or two I'll pass it on in another pull request and you can decide if you want to take it. Highlights: - Arm Ethos NPU accelerator driver - new DRM color pipeline support - amdgpu will now run discrete SI/CIK cards instead of radeon, which enables vulkan support in userspace - msm gets gen8 gpu support - initial Xe3P support in xe Full detail summary: New driver: - Arm Ethos-U65/U85 accel driver Core: - support the drm color pipeline in vkms/amdgfx - add support for drm colorop pipeline - add COLOR PIPELINE plane property - add DRM_CLIENT_CAP_PLANE_COLOR_PIPELINE - throttle dirty worker with vblank - use drm_for_each_bridge_in_chain_scoped in drm's bridge code - Ensure drm_client_modeset tests are enabled in UML - add simulated vblank interrupt - use in drivers - dumb buffer sizing helper - move freeing of drm client memory to driver - crtc sharpness strength property - stop using system_wq in scheduler/drivers - support emergency restore in drm-client Rust: - make slice::as_flattened usable on all supported rustc - add FromBytes::from_bytes_prefix() method - remove redundant device ptr from Rust GEM object - Change how AlwaysRefCounted is implemented for GEM objects gpuvm: - Add deferred vm_bo cleanup to GPUVM (for rust) atomic: - cleanup and improve state handling interfaces buddy: - optimize block management dma-buf: - heaps: Create heap per CMA reserved location - improve userspace documentation dp: - add POST_LT_ADJ_REQ training sequence - DPCD dSC quirk for synaptics panamera devices - helpers to query branch DSC max throughput ttm: - Rename ttm_bo_put to ttm_bo_fini - allow page protection flags on risc-v - rework pipelined eviction fence handling amdgpu: - enable amdgpu by default for SI/CI dGPUs - enable DC by default on SI - refactor CIK/SI enablement - add ABM KMS property - Re-enable DM idle optimizations - DC Analog encoders support - Powerplay fixes for fiji/iceland - Enable DC on bonaire by default - HMM cleanup - Add new RAS framework - DML2.1 updates - YCbCr420 fixes - DC FP fixes - DMUB fixes - LTTPR fixes - DTBCLK fixes - DMU cursor offload handling - Userq validation improvements - Unify shutdown callback handling - Suspend improvements - Power limit code cleanup - SR-IOV fixes - AUX backlight fixes - DCN 3.5 fixes - HDMI compliance fixes - DCN 4.0.1 cursor updates - DCN interrupt fix - DC KMS full update improvements - Add additional HDCP traces - DCN 3.2 fixes - DP MST fixes - Add support for new SR-IOV mailbox interface - UQ reset support - HDP flush rework - VCE1 support amdkfd: - HMM cleanups - Relax checks on save area overallocations - Fix GPU mappings after prefetch radeon: - refactor CIK/SI enablement xe: - Initial Xe3P support - panic support on VRAM for display - fix stolen size check - Loosen used tracking restriction - New SR-IOV debugfs structure and debugfs updates - Hide the GPU madvise flag behind a VM_BIND flag - Always expose VRAM provisioning data on discrete GPUs - Allow VRAM mappings for userptr when used with SVM - Allow pinning of p2p dma-buf - Use per-tile debugfs where appropriate - Add documentation for Execution Queues - PF improvements - VF migration recovery redesign work - User / Kernel VRAM partitioning - Update Tile-based messages - Allow configfs to disable specific GT types - VF provisioning and migration improvements - use SVM range helpers in PT layer - Initial CRI support - access VF registers using dedicated MMIO view - limit number of jobs per exec queue - add sriov_admin sysfs tree - more crescent island specific support - debugfs residency counter - SRIOV migration work - runtime registers for GFX 35 i915: - add initial Xe3p_LPD display version 35 support - Enable LNL+ content adaptive sharpness filter - Use optimized VRR guardband - Enable Xe3p LT PHY - enable FBC support for Xe3p_LPD display - add display 30.02 firmware support - refactor SKL+ watermark latency setup - refactor fbdev handling - call i915/xe runtime PM via function pointers - refactor i915/xe stolen memory/display interfaces - use display version instead of gfx version in display code - extend i915_display_info with Type-C port details - lots of display cleanups/refactorings - set O_LARGEFILE in __create_shmem - skuip guc communication warning on reset - fix time conversions - defeature DRRS on LNL+ - refactor intel_frontbuffer split between i915/xe/display - convert inteL_rom interfaces to struct drm_device - unify display register polling interfaces - aovid lock inversion when pinning to GGTT on CHV/BXT+VTD panel: - Add KD116N3730A08/A12, chromebook mt8189 - JT101TM023, LQ079L1SX01, - GLD070WX3-SL01 MIPI DSI - Samsung LTL106AL0, Samsung LTL106AL01 - Raystar RFF500F-AWH-DNN - Winstar WF70A8SYJHLNGA - Wanchanglong w552946aaa - Samsung SOFEF00 - Lenovo X13s panel - ilitek-ili9881c - add rpi 5" support - visionx-rm69299 - add backlight support - edp - support AUI B116XAN02.0 bridge: - improve ref counting - ti-sn65dsi86 - add support for DP mode with HPD - synopsis: support CEC, init timer with correct freq - ASL CS5263 DP-to-HDMI bridge support nova-core: - introduce bitfield! macro - introduce safe integer converters - GSP inits to fully booted state on Ampere - Use more future-proof register for GPU identification nova-drm: - select NOVA_CORE - 64-bit only nouveau: - improve reclocking on tegra 186+ - add large page and compression support msm: - GPU: - Gen8 support: A840 (Kaanapali) and X2-85 (Glymur) - A612 support - MDSS: - Added support for Glymur and QCS8300 platforms - DPU: - Enabled Quad-Pipe support, unlocking higher resolutions support - Added support for Glymur platform - Documented DPU on QCS8300 platform as supported - DisplayPort: - Added support for Glymur platform - Added support lame remapping inside DP block - Documented DisplayPort controller on QCS8300 and SM6150/QCS615 as supported tegra: - NVJPG driver panfrost: - display JM contexts over debugfs - export JM contexts to userspace - improve error and job handling panthor: - support custom ASN_HASH for mt8196 - support mali-G1 GPU - flush shmem write before mapping buffers uncached - make timeout per-queue instead of per-job mediatek: - MT8195/88 HDMIv2/DDCv2 support rockchip: - dsi: add support for RK3368 amdxdna: - enhance runtime PM - last hardware error reading uapi - support firmware debug output - add resource and telemetry data uapi - preemption support imx: - add driver for HDMI TX Parallel audio interface ivpu: - add support for user-managed preemption buffer - add userptr support - update JSM firware API to 3.33.0 - add better alloc/free warnings - fix page fault in unbind all bos - rework bind/unbind of imported buffers - enable MCA ECC signalling - split fw runtime and global memory buffers - add fdinfo memory statistics tidss: - convert to drm logging - logging cleanup ast: - refactor generation init paths - add per chip generation detect_tx_chip - set quirks for each chip model atmel-hlcdc: - set LCDC_ATTRE register in plane disable - set correct values for plane scaler solomon: - use drm helper for get_modes and move_valid sitronix: - fix output position when clearing screens qaic: - support dma-buf exports - support new firmware's READ_DATA implementation - sahara AIC200 image table update - add sysfs support - add coredump support - add uevents support - PM support sun4i: - layer refactors to decouple plane from output - improve DE33 support vc4: - switch to generic CEC helpers komeda: - use drm_ logging functions vkms: - configfs support for display configuration vgem: - fix fence timer deadlock etnaviv: - add HWDB entry for GC8000 Nano Ultra VIP r6205" * tag 'drm-next-2025-12-03' of https://gitlab.freedesktop.org/drm/kernel: (1869 commits) Revert "drm/amd: Skip power ungate during suspend for VPE" drm/amdgpu: use common defines for HUB faults drm/amdgpu/gmc12: add amdgpu_vm_handle_fault() handling drm/amdgpu/gmc11: add amdgpu_vm_handle_fault() handling drm/amdgpu: use static ids for ACP platform devs drm/amdgpu/sdma6: Update SDMA 6.0.3 FW version to include UMQ protected-fence fix drm/amdgpu: Forward VMID reservation errors drm/amdgpu/gmc8: Delegate VM faults to soft IRQ handler ring drm/amdgpu/gmc7: Delegate VM faults to soft IRQ handler ring drm/amdgpu/gmc6: Delegate VM faults to soft IRQ handler ring drm/amdgpu/gmc6: Cache VM fault info drm/amdgpu/gmc6: Don't print MC client as it's unknown drm/amdgpu/cz_ih: Enable soft IRQ handler ring drm/amdgpu/tonga_ih: Enable soft IRQ handler ring drm/amdgpu/iceland_ih: Enable soft IRQ handler ring drm/amdgpu/cik_ih: Enable soft IRQ handler ring drm/amdgpu/si_ih: Enable soft IRQ handler ring drm/amd/display: fix typo in display_mode_core_structs.h drm/amd/display: fix Smart Power OLED not working after S4 drm/amd/display: Move RGB-type check for audio sync to DCE HW sequence ...

Linus Torvalds committed 11 months ago

Commit 6dfafbd2 parents 94e244d 06926021799 files · +94407 −21674Browse files

Unified · Split

1799 files
  1. .clang-format +1 −1
  2. .mailmap +1 −0
  3. Documentation/ABI/stable/sysfs-driver-qaic +19 −0
  4. Documentation/ABI/testing/sysfs-driver-intel-xe-sriov +159 −0
  5. Documentation/accel/qaic/aic100.rst +23 −2
  6. Documentation/accel/qaic/qaic.rst +4 −4
  7. Documentation/devicetree/bindings/display/bridge/fsl,imx8mp-hdmi-tx.yaml +12 −0
  8. Documentation/devicetree/bindings/display/bridge/ite,it66121.yaml +1 −0
  9. Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml +52 −2
  10. Documentation/devicetree/bindings/display/bridge/simple-bridge.yaml +2 −0
  11. Documentation/devicetree/bindings/display/imx/fsl,imx8mp-hdmi-pai.yaml +69 −0
  12. Documentation/devicetree/bindings/display/msm/dp-controller.yaml +16 −1
  13. Documentation/devicetree/bindings/display/msm/gmu.yaml +59 −1
  14. Documentation/devicetree/bindings/display/msm/qcom,glymur-mdss.yaml +264 −0
  15. Documentation/devicetree/bindings/display/msm/qcom,qcs8300-mdss.yaml +286 −0
  16. Documentation/devicetree/bindings/display/msm/qcom,sm6150-mdss.yaml +25 −15
  17. Documentation/devicetree/bindings/display/msm/qcom,sm8650-dpu.yaml +11 −5
  18. Documentation/devicetree/bindings/display/panel/ilitek,il79900a.yaml +68 −0
  19. Documentation/devicetree/bindings/display/panel/ilitek,ili9881c.yaml +3 −0
  20. Documentation/devicetree/bindings/display/panel/lg,ld070wx3-sl01.yaml +60 −0
  21. Documentation/devicetree/bindings/display/panel/panel-lvds.yaml +2 −0
  22. Documentation/devicetree/bindings/display/panel/panel-simple-dsi.yaml +4 −26
  23. Documentation/devicetree/bindings/display/panel/panel-simple.yaml +6 −0
  24. Documentation/devicetree/bindings/display/panel/ronbo,rb070d30.yaml +5 −9
  25. Documentation/devicetree/bindings/display/panel/samsung,atna33xc20.yaml +2 −0
  26. Documentation/devicetree/bindings/display/panel/samsung,s6e3fc2x01.yaml +81 −0
  27. Documentation/devicetree/bindings/display/panel/samsung,sofef00.yaml +79 −0
  28. Documentation/devicetree/bindings/display/panel/sharp,lq079l1sx01.yaml +99 −0
  29. Documentation/devicetree/bindings/display/panel/synaptics,td4300-panel.yaml +89 −0
  30. Documentation/devicetree/bindings/display/renesas,rzg2l-du.yaml +3 −0
  31. Documentation/devicetree/bindings/display/rockchip/rockchip,dw-mipi-dsi.yaml +2 −0
  32. Documentation/devicetree/bindings/display/rockchip/rockchip,rk3588-dw-hdmi-qp.yaml +11 −0
  33. Documentation/devicetree/bindings/gpu/arm,mali-valhall-csf.yaml +40 −2
  34. Documentation/devicetree/bindings/gpu/img,powervr-rogue.yaml +43 −22
  35. Documentation/devicetree/bindings/iommu/arm,smmu.yaml +2 −0
  36. Documentation/devicetree/bindings/npu/arm,ethos.yaml +79 −0
  37. Documentation/devicetree/bindings/vendor-prefixes.yaml +6 −0
  38. Documentation/gpu/drm-kms-helpers.rst +12 −0
  39. Documentation/gpu/drm-kms.rst +15 −0
  40. Documentation/gpu/nova/core/todo.rst +0 −30
  41. Documentation/gpu/rfc/color_pipeline.rst +378 −0
  42. Documentation/gpu/rfc/index.rst +3 −0
  43. Documentation/gpu/todo.rst +37 −0
  44. Documentation/gpu/vkms.rst +100 −19
  45. Documentation/gpu/xe/index.rst +1 −0
  46. Documentation/gpu/xe/xe_exec_queue.rst +20 −0
  47. Documentation/gpu/xe/xe_gt_freq.rst +3 −0
  48. Documentation/userspace-api/dma-buf-heaps.rst +49 −10
  49. MAINTAINERS +37 −11
  50. arch/arm/boot/dts/st/stih410.dtsi +192 −134
  51. arch/arm64/boot/dts/freescale/imx95.dtsi +37 −0
  52. drivers/accel/Kconfig +1 −0
  53. drivers/accel/Makefile +1 −0
  54. drivers/accel/amdxdna/Makefile +1 −0
  55. drivers/accel/amdxdna/TODO +0 −1
  56. drivers/accel/amdxdna/aie2_ctx.c +165 −30
  57. drivers/accel/amdxdna/aie2_error.c +78 −17
  58. drivers/accel/amdxdna/aie2_message.c +471 −176
  59. drivers/accel/amdxdna/aie2_msg_priv.h +84 −4
  60. drivers/accel/amdxdna/aie2_pci.c +226 −43
  61. drivers/accel/amdxdna/aie2_pci.h +52 −2
  62. drivers/accel/amdxdna/aie2_smu.c +45 −4
  63. drivers/accel/amdxdna/amdxdna_ctx.c +67 −37
  64. drivers/accel/amdxdna/amdxdna_ctx.h +40 −5
  65. drivers/accel/amdxdna/amdxdna_error.h +59 −0
  66. drivers/accel/amdxdna/amdxdna_gem.c +24 −27
  67. drivers/accel/amdxdna/amdxdna_gem.h +6 −0
  68. drivers/accel/amdxdna/amdxdna_mailbox.c +9 −5
  69. drivers/accel/amdxdna/amdxdna_mailbox_helper.h +4 −2
  70. drivers/accel/amdxdna/amdxdna_pci_drv.c +11 −52
  71. drivers/accel/amdxdna/amdxdna_pci_drv.h +3 −0
  72. drivers/accel/amdxdna/amdxdna_pm.c +94 −0
  73. drivers/accel/amdxdna/amdxdna_pm.h +18 −0
  74. drivers/accel/amdxdna/npu1_regs.c +8 −0
  75. drivers/accel/amdxdna/npu2_regs.c +2 −0
  76. drivers/accel/amdxdna/npu4_regs.c +12 −0
  77. drivers/accel/amdxdna/npu5_regs.c +2 −0
  78. drivers/accel/amdxdna/npu6_regs.c +2 −0
  79. drivers/accel/ethosu/Kconfig +11 −0
  80. drivers/accel/ethosu/Makefile +4 −0
  81. drivers/accel/ethosu/ethosu_device.h +197 −0
  82. drivers/accel/ethosu/ethosu_drv.c +403 −0
  83. drivers/accel/ethosu/ethosu_drv.h +15 −0
  84. drivers/accel/ethosu/ethosu_gem.c +704 −0
  85. drivers/accel/ethosu/ethosu_gem.h +46 −0
  86. drivers/accel/ethosu/ethosu_job.c +497 −0
  87. drivers/accel/ethosu/ethosu_job.h +40 −0
  88. drivers/accel/ivpu/Makefile +1 −0
  89. drivers/accel/ivpu/ivpu_debugfs.c +14 −24
  90. drivers/accel/ivpu/ivpu_drv.c +15 −3
  91. drivers/accel/ivpu/ivpu_drv.h +5 −0
  92. drivers/accel/ivpu/ivpu_fw.c +151 −78
  93. drivers/accel/ivpu/ivpu_fw.h +13 −1
  94. drivers/accel/ivpu/ivpu_gem.c +117 −44
  95. drivers/accel/ivpu/ivpu_gem.h +20 −2
  96. drivers/accel/ivpu/ivpu_gem_userptr.c +213 −0
  97. drivers/accel/ivpu/ivpu_hw.c +52 −7
  98. drivers/accel/ivpu/ivpu_hw.h +4 −6
  99. drivers/accel/ivpu/ivpu_hw_btrs.c +18 −2
  100. drivers/accel/ivpu/ivpu_hw_btrs.h +1 −1
  101. drivers/accel/ivpu/ivpu_hw_btrs_lnl_reg.h +3 −0
  102. drivers/accel/ivpu/ivpu_hw_ip.c +10 −0
  103. drivers/accel/ivpu/ivpu_ipc.c +1 −1
  104. drivers/accel/ivpu/ivpu_job.c +178 −79
  105. drivers/accel/ivpu/ivpu_job.h +32 −17
  106. drivers/accel/ivpu/ivpu_mmu.c +1 −1
  107. drivers/accel/ivpu/ivpu_mmu_context.c +6 −3
  108. drivers/accel/ivpu/ivpu_mmu_context.h +1 −1
  109. drivers/accel/ivpu/ivpu_ms.c +17 −8
  110. drivers/accel/ivpu/ivpu_pm.c +11 −7
  111. drivers/accel/ivpu/ivpu_sysfs.c +2 −1
  112. drivers/accel/ivpu/vpu_jsm_api.h +417 −236
  113. drivers/accel/qaic/Kconfig +1 −0
  114. drivers/accel/qaic/Makefile +2 −0
  115. drivers/accel/qaic/qaic.h +39 −1
  116. drivers/accel/qaic/qaic_control.c +14 −11
  117. drivers/accel/qaic/qaic_data.c +116 −48
  118. drivers/accel/qaic/qaic_drv.c +115 −1
  119. drivers/accel/qaic/qaic_ras.c +3 −3
  120. drivers/accel/qaic/qaic_ssr.c +815 −0
  121. drivers/accel/qaic/qaic_ssr.h +17 −0
  122. drivers/accel/qaic/qaic_sysfs.c +109 −0
  123. drivers/accel/qaic/qaic_timesync.c +9 −0
  124. drivers/accel/qaic/qaic_timesync.h +3 −0
  125. drivers/accel/qaic/sahara.c +121 −43
  126. drivers/accel/rocket/rocket_gem.c +1 −0
  127. drivers/dma-buf/dma-fence.c +26 −26
  128. drivers/dma-buf/heaps/Kconfig +0 −10
  129. drivers/dma-buf/heaps/cma_heap.c +30 −17
  130. drivers/dma-buf/heaps/system_heap.c +24 −9
  131. drivers/dma-buf/sw_sync.c +4 −0
  132. drivers/gpu/drm/Makefile +5 −2
  133. drivers/gpu/drm/adp/adp_drv.c +1 −0
  134. drivers/gpu/drm/amd/amdgpu/Kconfig +16 −8
  135. drivers/gpu/drm/amd/amdgpu/Makefile +8 −2
  136. drivers/gpu/drm/amd/amdgpu/aldebaran.c +4 −0
  137. drivers/gpu/drm/amd/amdgpu/amdgpu.h +14 −15
  138. drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c +8 −2
  139. drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c +0 −1
  140. drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +1 −1
  141. drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +21 −11
  142. drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c +16 −13
  143. drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c +26 −10
  144. drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h +1 −1
  145. drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c +4 −12
  146. drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +11 −8
  147. drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +4 −30
  148. drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +284 −134
  149. drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c +156 −88
  150. drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h +12 −0
  151. drivers/gpu/drm/amd/amdgpu/amdgpu_display.c +59 −3
  152. drivers/gpu/drm/amd/amdgpu/amdgpu_display.h +7 −0
  153. drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +19 −0
  154. drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +104 −101
  155. drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +13 −132
  156. drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +37 −3
  157. drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +3 −0
  158. drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +10 −7
  159. drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c +71 −1
  160. drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h +1 −0
  161. drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +5 −0
  162. drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +1 −0
  163. drivers/gpu/drm/amd/amdgpu/amdgpu_hdp.c +16 −0
  164. drivers/gpu/drm/amd/amdgpu/amdgpu_hdp.h +4 −0
  165. drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c +62 −17
  166. drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.h +22 −2
  167. drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +26 −10
  168. drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c +17 −41
  169. drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +53 −16
  170. drivers/gpu/drm/amd/amdgpu/amdgpu_job.h +2 −1
  171. drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +0 −2
  172. drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +14 −2
  173. drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +2 −0
  174. drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h +2 −0
  175. drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +1 −0
  176. drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +16 −7
  177. drivers/gpu/drm/amd/amdgpu/amdgpu_rap.c +0 −1
  178. drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +322 −66
  179. drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h +37 −1
  180. drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c +394 −28
  181. drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.h +30 −0
  182. drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +74 −5
  183. drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +4 −5
  184. drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c +0 −1
  185. drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +41 −81
  186. drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +4 −25
  187. drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c +93 −54
  188. drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h +2 −0
  189. drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c +516 −124
  190. drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h +24 −5
  191. drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c +3 −2
  192. drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +108 −66
  193. drivers/gpu/drm/amd/amdgpu/amdgpu_vce.h +3 −0
  194. drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h +4 −2
  195. drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c +316 −27
  196. drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +27 −0
  197. drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +23 −16
  198. drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 −4
  199. drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c +10 −27
  200. drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h +76 −16
  201. drivers/gpu/drm/amd/amdgpu/cik_ih.c +12 −0
  202. drivers/gpu/drm/amd/amdgpu/cz_ih.c +10 −0
  203. drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c +3 −1
  204. drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c +7 −4
  205. drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c +3 −1
  206. drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c +1 −0
  207. drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +2 −1
  208. drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c +11 −14
  209. drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c +4 −2
  210. drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c +29 −0
  211. drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +29 −2
  212. drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +15 −7
  213. drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +6 −0
  214. drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +6 −0
  215. drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +8 −2
  216. drivers/gpu/drm/amd/amdgpu/iceland_ih.c +10 −0
  217. drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +38 −22
  218. drivers/gpu/drm/amd/amdgpu/mes_v11_0.c +4 −0
  219. drivers/gpu/drm/amd/amdgpu/mes_v12_0.c +4 −0
  220. drivers/gpu/drm/amd/amdgpu/mxgpu_nv.c +28 −12
  221. drivers/gpu/drm/amd/amdgpu/nbio_v7_9.c +13 −11
  222. drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c +6 −2
  223. drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c +2 −1
  224. drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c +5 −3
  225. drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c +3 −2
  226. drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c +2 −1
  227. drivers/gpu/drm/amd/amdgpu/si.c +11 −11
  228. drivers/gpu/drm/amd/amdgpu/si_ih.c +12 −0
  229. drivers/gpu/drm/amd/amdgpu/sid.h +0 −40
  230. drivers/gpu/drm/amd/amdgpu/soc15.c +0 −4
  231. drivers/gpu/drm/amd/amdgpu/tonga_ih.c +10 −0
  232. drivers/gpu/drm/amd/amdgpu/umc_v12_0.c +14 −0
  233. drivers/gpu/drm/amd/amdgpu/vce_v1_0.c +839 −0
  234. drivers/gpu/drm/amd/amdgpu/vce_v1_0.h +32 −0
  235. drivers/gpu/drm/amd/amdgpu/vce_v2_0.c +5 −0
  236. drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +5 −0
  237. drivers/gpu/drm/amd/amdgpu/vce_v4_0.c +5 −0
  238. drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +8 −4
  239. drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +8 −4
  240. drivers/gpu/drm/amd/amdkfd/kfd_events.c +0 −11
  241. drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c +6 −1
  242. drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +0 −1
  243. drivers/gpu/drm/amd/amdkfd/kfd_migrate.h +0 −1
  244. drivers/gpu/drm/amd/amdkfd/kfd_process.c +8 −4
  245. drivers/gpu/drm/amd/amdkfd/kfd_svm.c +13 −7
  246. drivers/gpu/drm/amd/amdkfd/kfd_svm.h +0 −1
  247. drivers/gpu/drm/amd/amdkfd/kfd_topology.c +3 −1
  248. drivers/gpu/drm/amd/display/amdgpu_dm/Makefile +2 −1
  249. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +281 −44
  250. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h +1 −0
  251. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c +748 −20
  252. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_colorop.c +209 −0
  253. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_colorop.h +36 −0
  254. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c +12 −14
  255. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c +3 −2
  256. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c +9 −9
  257. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c +1 −0
  258. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c +17 −0
  259. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c +39 −0
  260. drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_replay.c +1 −1
  261. drivers/gpu/drm/amd/display/dc/Makefile +1 −1
  262. drivers/gpu/drm/amd/display/dc/bios/bios_parser.c +89 −6
  263. drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c +4 −4
  264. drivers/gpu/drm/amd/display/dc/bios/command_table.c +286 −0
  265. drivers/gpu/drm/amd/display/dc/bios/command_table.h +6 −0
  266. drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_clk_mgr.c +4 −4
  267. drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c +85 −2
  268. drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.h +1 −0
  269. drivers/gpu/drm/amd/display/dc/clk_mgr/dcn35/dcn35_clk_mgr.c +34 −4
  270. drivers/gpu/drm/amd/display/dc/core/dc.c +918 −205
  271. drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c +3010 −205
  272. drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c +2 −2
  273. drivers/gpu/drm/amd/display/dc/core/dc_resource.c +112 −17
  274. drivers/gpu/drm/amd/display/dc/core/dc_state.c +2 −2
  275. drivers/gpu/drm/amd/display/dc/core/dc_stream.c +45 −11
  276. drivers/gpu/drm/amd/display/dc/dc.h +555 −26
  277. drivers/gpu/drm/amd/display/dc/dc_bios_types.h +9 −0
  278. drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c +126 −6
  279. drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h +43 −0
  280. drivers/gpu/drm/amd/display/dc/dc_dp_types.h +36 −0
  281. drivers/gpu/drm/amd/display/dc/dc_spl_translate.c +2 −0
  282. drivers/gpu/drm/amd/display/dc/dc_stream.h +5 −6
  283. drivers/gpu/drm/amd/display/dc/dc_types.h +23 −1
  284. drivers/gpu/drm/amd/display/dc/dccg/dcn20/dcn20_dccg.c +1 −1
  285. drivers/gpu/drm/amd/display/dc/dccg/dcn20/dcn20_dccg.h +63 −1
  286. drivers/gpu/drm/amd/display/dc/dccg/dcn31/dcn31_dccg.c +123 −0
  287. drivers/gpu/drm/amd/display/dc/dccg/dcn31/dcn31_dccg.h +2 −0
  288. drivers/gpu/drm/amd/display/dc/dccg/dcn314/dcn314_dccg.c +2 −1
  289. drivers/gpu/drm/amd/display/dc/dccg/dcn314/dcn314_dccg.h +1 −2
  290. drivers/gpu/drm/amd/display/dc/dccg/dcn35/dcn35_dccg.c +20 −4
  291. drivers/gpu/drm/amd/display/dc/dccg/dcn35/dcn35_dccg.h +11 −2
  292. drivers/gpu/drm/amd/display/dc/dccg/dcn401/dcn401_dccg.c +1 −0
  293. drivers/gpu/drm/amd/display/dc/dce/dce_abm.c +1 −1
  294. drivers/gpu/drm/amd/display/dc/dce/dce_audio.c +2 −1
  295. drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c +3 −3
  296. drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c +85 −0
  297. drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.h +9 −7
  298. drivers/gpu/drm/amd/display/dc/dce/dce_stream_encoder.c +14 −0
  299. drivers/gpu/drm/amd/display/dc/dce/dce_stream_encoder.h +5 −0
  300. drivers/gpu/drm/amd/display/dc/dce/dmub_hw_lock_mgr.c +27 −6
  301. drivers/gpu/drm/amd/display/dc/dce/dmub_hw_lock_mgr.h +11 −1
  302. drivers/gpu/drm/amd/display/dc/dce/dmub_replay.c +18 −2
  303. drivers/gpu/drm/amd/display/dc/dce/dmub_replay.h +3 −2
  304. drivers/gpu/drm/amd/display/dc/dml2/Makefile +0 −141
  305. drivers/gpu/drm/amd/display/dc/dml2_0/Makefile +140 −0
  306. drivers/gpu/drm/amd/display/dc/dml2_0/cmntypes.h +9 −9
  307. drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_core.c +2 −0
  308. drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_core.h +0 −0
  309. drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_core_structs.h +1 −2
  310. drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_lib_defines.h +1 −1
  311. drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_util.c +0 −0
  312. drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_util.h +0 −2
  313. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_translation_helper.c +0 −0
  314. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_translation_helper.h +0 −0
  315. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_utils.c +0 −0
  316. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_utils.h +0 −0
  317. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_wrapper.c +0 −4
  318. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_wrapper.h +0 −0
  319. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/bounding_boxes/dcn4_soc_bb.h +0 −1
  320. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml2_external_lib_deps.h +0 −0
  321. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top.h +0 −0
  322. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_dchub_registers.h +2 −1
  323. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_display_cfg_types.h +20 −3
  324. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_policy_types.h +0 −0
  325. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_soc_parameter_types.h +8 −3
  326. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_types.h +7 −0
  327. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4.c +1 −0
  328. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4.h +0 −0
  329. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4_calcs.c +75 −48
  330. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4_calcs.h +0 −0
  331. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_factory.c +2 −0
  332. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_factory.h +0 −0
  333. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_shared_types.h +30 −15
  334. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_utils.c +2 −0
  335. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_utils.h +0 −0
  336. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_dpmm/dml2_dpmm_dcn4.c +0 −0
  337. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_dpmm/dml2_dpmm_dcn4.h +0 −0
  338. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_dpmm/dml2_dpmm_factory.c +0 −0
  339. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_dpmm/dml2_dpmm_factory.h +0 −0
  340. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_mcg/dml2_mcg_dcn4.c +0 −0
  341. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_mcg/dml2_mcg_dcn4.h +1 −1
  342. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_mcg/dml2_mcg_factory.c +0 −0
  343. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_mcg/dml2_mcg_factory.h +0 −0
  344. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_dcn3.c +0 −0
  345. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_dcn3.h +0 −0
  346. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_dcn4_fams2.c +248 −229
  347. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_dcn4_fams2.h +0 −0
  348. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_factory.c +0 −0
  349. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_factory.h +1 −1
  350. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_standalone_libraries/lib_float_math.c +0 −0
  351. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_standalone_libraries/lib_float_math.h +0 −0
  352. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_interfaces.c +0 −0
  353. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_legacy.c +0 −0
  354. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_legacy.h +0 −0
  355. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_soc15.c +0 −0
  356. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_soc15.h +0 −0
  357. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/inc/dml2_debug.h +0 −0
  358. drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/inc/dml2_internal_shared_types.h +49 −27
  359. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_dc_resource_mgmt.c +0 −0
  360. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_dc_resource_mgmt.h +0 −0
  361. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_dc_types.h +0 −0
  362. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_internal_types.h +1 −1
  363. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_mall_phantom.c +1 −0
  364. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_mall_phantom.h +0 −0
  365. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_policy.c +0 −0
  366. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_policy.h +0 −0
  367. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_translation_helper.c +3 −0
  368. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_translation_helper.h +0 −0
  369. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_utils.c +0 −0
  370. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_utils.h +0 −0
  371. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_wrapper.c +0 −0
  372. drivers/gpu/drm/amd/display/dc/dml2_0/dml2_wrapper.h +0 −0
  373. drivers/gpu/drm/amd/display/dc/dml2_0/dml_assert.h +0 −0
  374. drivers/gpu/drm/amd/display/dc/dml2_0/dml_depedencies.h +1 −0
  375. drivers/gpu/drm/amd/display/dc/dml2_0/dml_display_rq_dlg_calc.c +0 −0
  376. drivers/gpu/drm/amd/display/dc/dml2_0/dml_display_rq_dlg_calc.h +0 −0
  377. drivers/gpu/drm/amd/display/dc/dml2_0/dml_logging.h +1 −0
  378. drivers/gpu/drm/amd/display/dc/dpp/dcn10/dcn10_dpp.c +14 −5
  379. drivers/gpu/drm/amd/display/dc/dpp/dcn10/dcn10_dpp.h +2 −2
  380. drivers/gpu/drm/amd/display/dc/dpp/dcn30/dcn30_dpp.c +28 −8
  381. drivers/gpu/drm/amd/display/dc/dpp/dcn30/dcn30_dpp.h +2 −0
  382. drivers/gpu/drm/amd/display/dc/dpp/dcn32/dcn32_dpp.c +1 −0
  383. drivers/gpu/drm/amd/display/dc/dpp/dcn35/dcn35_dpp.c +1 −0
  384. drivers/gpu/drm/amd/display/dc/dpp/dcn401/dcn401_dpp.c +1 −0
  385. drivers/gpu/drm/amd/display/dc/dpp/dcn401/dcn401_dpp_cm.c +28 −15
  386. drivers/gpu/drm/amd/display/dc/dsc/dcn20/dcn20_dsc.c +9 −1
  387. drivers/gpu/drm/amd/display/dc/dsc/dcn20/dcn20_dsc.h +1 −0
  388. drivers/gpu/drm/amd/display/dc/dsc/dcn35/dcn35_dsc.c +31 −1
  389. drivers/gpu/drm/amd/display/dc/dsc/dcn401/dcn401_dsc.c +1 −0
  390. drivers/gpu/drm/amd/display/dc/dsc/dsc.h +6 −0
  391. drivers/gpu/drm/amd/display/dc/hubbub/dcn30/dcn30_hubbub.c +7 −26
  392. drivers/gpu/drm/amd/display/dc/hubbub/dcn30/dcn30_hubbub.h +1 −5
  393. drivers/gpu/drm/amd/display/dc/hubbub/dcn31/dcn31_hubbub.c +4 −4
  394. drivers/gpu/drm/amd/display/dc/hubbub/dcn32/dcn32_hubbub.c +1 −2
  395. drivers/gpu/drm/amd/display/dc/hubbub/dcn35/dcn35_hubbub.c +51 −2
  396. drivers/gpu/drm/amd/display/dc/hubbub/dcn35/dcn35_hubbub.h +1 −0
  397. drivers/gpu/drm/amd/display/dc/hubbub/dcn401/dcn401_hubbub.c +1 −2
  398. drivers/gpu/drm/amd/display/dc/hubp/dcn10/dcn10_hubp.c +1 −0
  399. drivers/gpu/drm/amd/display/dc/hubp/dcn10/dcn10_hubp.h +133 −3
  400. drivers/gpu/drm/amd/display/dc/hubp/dcn20/dcn20_hubp.c +38 −31
  401. drivers/gpu/drm/amd/display/dc/hubp/dcn20/dcn20_hubp.h +6 −2
  402. drivers/gpu/drm/amd/display/dc/hubp/dcn21/dcn21_hubp.c +0 −2
  403. drivers/gpu/drm/amd/display/dc/hubp/dcn30/dcn30_hubp.c +121 −26
  404. drivers/gpu/drm/amd/display/dc/hubp/dcn30/dcn30_hubp.h +2 −0
  405. drivers/gpu/drm/amd/display/dc/hubp/dcn31/dcn31_hubp.c +1 −3
  406. drivers/gpu/drm/amd/display/dc/hubp/dcn32/dcn32_hubp.c +44 −30
  407. drivers/gpu/drm/amd/display/dc/hubp/dcn35/dcn35_hubp.c +1 −3
  408. drivers/gpu/drm/amd/display/dc/hubp/dcn401/dcn401_hubp.c +14 −14
  409. drivers/gpu/drm/amd/display/dc/hubp/dcn401/dcn401_hubp.h +1 −1
  410. drivers/gpu/drm/amd/display/dc/hwss/dce110/dce110_hwseq.c +86 −3
  411. drivers/gpu/drm/amd/display/dc/hwss/dcn10/dcn10_hwseq.c +4 −1
  412. drivers/gpu/drm/amd/display/dc/hwss/dcn20/dcn20_hwseq.c +4 −1
  413. drivers/gpu/drm/amd/display/dc/hwss/dcn30/dcn30_hwseq.c +40 −36
  414. drivers/gpu/drm/amd/display/dc/hwss/dcn31/dcn31_hwseq.c +2 −1
  415. drivers/gpu/drm/amd/display/dc/hwss/dcn314/dcn314_hwseq.c +1 −0
  416. drivers/gpu/drm/amd/display/dc/hwss/dcn32/dcn32_hwseq.c +1 −0
  417. drivers/gpu/drm/amd/display/dc/hwss/dcn35/dcn35_hwseq.c +139 −5
  418. drivers/gpu/drm/amd/display/dc/hwss/dcn35/dcn35_hwseq.h +8 −0
  419. drivers/gpu/drm/amd/display/dc/hwss/dcn35/dcn35_init.c +6 −0
  420. drivers/gpu/drm/amd/display/dc/hwss/dcn401/dcn401_hwseq.c +1390 −17
  421. drivers/gpu/drm/amd/display/dc/hwss/dcn401/dcn401_hwseq.h +102 −2
  422. drivers/gpu/drm/amd/display/dc/hwss/dcn401/dcn401_init.c +26 −2
  423. drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h +1440 −6
  424. drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer_private.h +36 −0
  425. drivers/gpu/drm/amd/display/dc/inc/core_types.h +6 −6
  426. drivers/gpu/drm/amd/display/dc/inc/hw/cursor_reg_cache.h +25 −3
  427. drivers/gpu/drm/amd/display/dc/inc/hw/dccg.h +120 −1
  428. drivers/gpu/drm/amd/display/dc/inc/hw/dchubbub.h +41 −9
  429. drivers/gpu/drm/amd/display/dc/inc/hw/dpp.h +16 −1
  430. drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h +5 −2
  431. drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h +49 −0
  432. drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h +2 −0
  433. drivers/gpu/drm/amd/display/dc/inc/hw/mem_input.h +1 −1
  434. drivers/gpu/drm/amd/display/dc/inc/hw/mpc.h +27 −0
  435. drivers/gpu/drm/amd/display/dc/inc/hw/opp.h +13 −0
  436. drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h +130 −0
  437. drivers/gpu/drm/amd/display/dc/inc/link_service.h +2 −2
  438. drivers/gpu/drm/amd/display/dc/inc/resource.h +1 −0
  439. drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_cts.c +2 −2
  440. drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dio.c +12 −7
  441. drivers/gpu/drm/amd/display/dc/link/link_detection.c +171 −1
  442. drivers/gpu/drm/amd/display/dc/link/link_dpms.c +10 −1
  443. drivers/gpu/drm/amd/display/dc/link/link_factory.c +59 −7
  444. drivers/gpu/drm/amd/display/dc/link/protocols/link_ddc.c +2 −1
  445. drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c +50 −3
  446. drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.c +15 −9
  447. drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_irq_handler.c +3 −1
  448. drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.c +135 −8
  449. drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.h +2 −2
  450. drivers/gpu/drm/amd/display/dc/mpc/dcn30/dcn30_mpc.c +16 −0
  451. drivers/gpu/drm/amd/display/dc/mpc/dcn30/dcn30_mpc.h +5 −0
  452. drivers/gpu/drm/amd/display/dc/mpc/dcn32/dcn32_mpc.c +1 −0
  453. drivers/gpu/drm/amd/display/dc/mpc/dcn401/dcn401_mpc.c +1 −0
  454. drivers/gpu/drm/amd/display/dc/opp/dcn10/dcn10_opp.c +13 −1
  455. drivers/gpu/drm/amd/display/dc/opp/dcn10/dcn10_opp.h +4 −2
  456. drivers/gpu/drm/amd/display/dc/opp/dcn20/dcn20_opp.c +13 −0
  457. drivers/gpu/drm/amd/display/dc/opp/dcn20/dcn20_opp.h +5 −1
  458. drivers/gpu/drm/amd/display/dc/opp/dcn35/dcn35_opp.c +13 −0
  459. drivers/gpu/drm/amd/display/dc/opp/dcn35/dcn35_opp.h +3 −1
  460. drivers/gpu/drm/amd/display/dc/optc/dcn10/dcn10_optc.h +37 −1
  461. drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.c +131 −0
  462. drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h +2 −0
  463. drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.c +1 −0
  464. drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.c +1 −0
  465. drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.c +1 −0
  466. drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.c +1 −0
  467. drivers/gpu/drm/amd/display/dc/resource/dce100/dce100_resource.c +33 −5
  468. drivers/gpu/drm/amd/display/dc/resource/dce110/dce110_resource.c +6 −1
  469. drivers/gpu/drm/amd/display/dc/resource/dce112/dce112_resource.c +7 −2
  470. drivers/gpu/drm/amd/display/dc/resource/dce120/dce120_resource.c +7 −2
  471. drivers/gpu/drm/amd/display/dc/resource/dce60/dce60_resource.c +27 −2
  472. drivers/gpu/drm/amd/display/dc/resource/dce80/dce80_resource.c +31 −3
  473. drivers/gpu/drm/amd/display/dc/resource/dcn10/dcn10_resource.c +6 −1
  474. drivers/gpu/drm/amd/display/dc/resource/dcn20/dcn20_resource.c +19 −16
  475. drivers/gpu/drm/amd/display/dc/resource/dcn201/dcn201_resource.c +19 −17
  476. drivers/gpu/drm/amd/display/dc/resource/dcn21/dcn21_resource.c +5 −1
  477. drivers/gpu/drm/amd/display/dc/resource/dcn30/dcn30_resource.c +5 −1
  478. drivers/gpu/drm/amd/display/dc/resource/dcn301/dcn301_resource.c +5 −1
  479. drivers/gpu/drm/amd/display/dc/resource/dcn302/dcn302_resource.c +5 −1
  480. drivers/gpu/drm/amd/display/dc/resource/dcn303/dcn303_resource.c +5 −1
  481. drivers/gpu/drm/amd/display/dc/resource/dcn31/dcn31_resource.c +5 −1
  482. drivers/gpu/drm/amd/display/dc/resource/dcn314/dcn314_resource.c +5 −1
  483. drivers/gpu/drm/amd/display/dc/resource/dcn315/dcn315_resource.c +6 −1
  484. drivers/gpu/drm/amd/display/dc/resource/dcn316/dcn316_resource.c +5 −1
  485. drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c +9 −4
  486. drivers/gpu/drm/amd/display/dc/resource/dcn321/dcn321_resource.c +5 −1
  487. drivers/gpu/drm/amd/display/dc/resource/dcn35/dcn35_resource.c +6 −2
  488. drivers/gpu/drm/amd/display/dc/resource/dcn351/dcn351_resource.c +6 −2
  489. drivers/gpu/drm/amd/display/dc/resource/dcn36/dcn36_resource.c +6 −2
  490. drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.c +7 −3
  491. drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h +2 −1
  492. drivers/gpu/drm/amd/display/dc/soc_and_ip_translator/dcn401/dcn401_soc_and_ip_translator.h +1 −1
  493. drivers/gpu/drm/amd/display/dc/sspl/dc_spl.c +15 −0
  494. drivers/gpu/drm/amd/display/dc/sspl/dc_spl_types.h +1 −0
  495. drivers/gpu/drm/amd/display/dmub/dmub_srv.h +31 −1
  496. drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h +708 −11
  497. drivers/gpu/drm/amd/display/dmub/src/dmub_dcn31.c +1 −0
  498. drivers/gpu/drm/amd/display/dmub/src/dmub_dcn32.c +31 −19
  499. drivers/gpu/drm/amd/display/dmub/src/dmub_dcn35.c +40 −0
  500. drivers/gpu/drm/amd/display/dmub/src/dmub_dcn35.h +2 −0
  501. drivers/gpu/drm/amd/display/dmub/src/dmub_dcn401.c +14 −3
  502. drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c +23 −10
  503. drivers/gpu/drm/amd/display/include/bios_parser_types.h +6 −5
  504. drivers/gpu/drm/amd/display/include/dpcd_defs.h +16 −0
  505. drivers/gpu/drm/amd/display/include/grph_object_ctrl_defs.h +1 −0
  506. drivers/gpu/drm/amd/display/include/grph_object_id.h +7 −0
  507. drivers/gpu/drm/amd/display/include/signal_types.h +12 −0
  508. drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c +3 −3
  509. drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h +2 −1
  510. drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c +11 −2
  511. drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_execution.c +38 −49
  512. drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_transition.c +32 −29
  513. drivers/gpu/drm/amd/display/modules/hdcp/hdcp_ddc.c +1 −1
  514. drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.c +2 −120
  515. drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.h +6 −0
  516. drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h +82 −66
  517. drivers/gpu/drm/amd/display/modules/power/power_helpers.c +33 −0
  518. drivers/gpu/drm/amd/display/modules/power/power_helpers.h +5 −0
  519. drivers/gpu/drm/amd/include/amd_shared.h +1 −0
  520. drivers/gpu/drm/amd/include/asic_reg/vce/vce_1_0_d.h +5 −0
  521. drivers/gpu/drm/amd/include/asic_reg/vce/vce_1_0_sh_mask.h +10 −0
  522. drivers/gpu/drm/amd/include/kgd_pp_interface.h +124 −1
  523. drivers/gpu/drm/amd/include/mes_v11_api_def.h +2 −1
  524. drivers/gpu/drm/amd/include/mes_v12_api_def.h +2 −1
  525. drivers/gpu/drm/amd/pm/amdgpu_dpm.c +18 −4
  526. drivers/gpu/drm/amd/pm/amdgpu_pm.c +21 −12
  527. drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h +2 −1
  528. drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.c +14 −14
  529. drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.h +0 −557
  530. drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c +5 −5
  531. drivers/gpu/drm/amd/pm/powerplay/smumgr/smu10_smumgr.c +2 −2
  532. drivers/gpu/drm/amd/pm/powerplay/smumgr/vega10_smumgr.c +2 −2
  533. drivers/gpu/drm/amd/pm/powerplay/smumgr/vega12_smumgr.c +2 −2
  534. drivers/gpu/drm/amd/pm/powerplay/smumgr/vega20_smumgr.c +4 −4
  535. drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c +56 −39
  536. drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h +20 −1
  537. drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v13_0_12_pmfw.h +7 −2
  538. drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v13_0_12_ppsmc.h +9 −11
  539. drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h +12 −2
  540. drivers/gpu/drm/amd/pm/swsmu/smu11/cyan_skillfish_ppt.c +3 −2
  541. drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c +9 −8
  542. drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c +3 −2
  543. drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c +6 −1
  544. drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c +11 −12
  545. drivers/gpu/drm/amd/pm/swsmu/smu12/renoir_ppt.c +4 −3
  546. drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c +4 −3
  547. drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_12_ppt.c +193 −56
  548. drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_4_ppt.c +3 −2
  549. drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_5_ppt.c +3 −2
  550. drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.c +235 −91
  551. drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.h +164 −1
  552. drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c +4 −3
  553. drivers/gpu/drm/amd/pm/swsmu/smu13/yellow_carp_ppt.c +3 −2
  554. drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_0_ppt.c +3 −2
  555. drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c +4 −3
  556. drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c +9 −5
  557. drivers/gpu/drm/amd/pm/swsmu/smu_cmn.h +67 −0
  558. drivers/gpu/drm/amd/pm/swsmu/smu_internal.h +1 −0
  559. drivers/gpu/drm/amd/ras/Makefile +34 −0
  560. drivers/gpu/drm/amd/ras/ras_mgr/Makefile +33 −0
  561. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_cmd.c +285 −0
  562. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_cmd.h +54 −0
  563. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_eeprom_i2c.c +182 −0
  564. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_eeprom_i2c.h +27 −0
  565. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_mgr.c +648 −0
  566. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_mgr.h +83 −0
  567. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_mp1_v13_0.c +94 −0
  568. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_mp1_v13_0.h +30 −0
  569. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_nbio_v7_9.c +125 −0
  570. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_nbio_v7_9.h +30 −0
  571. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_process.c +190 −0
  572. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_process.h +41 −0
  573. drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_sys.c +279 −0
  574. drivers/gpu/drm/amd/ras/ras_mgr/ras_sys.h +110 −0
  575. drivers/gpu/drm/amd/ras/rascore/Makefile +44 −0
  576. drivers/gpu/drm/amd/ras/rascore/ras.h +370 −0
  577. drivers/gpu/drm/amd/ras/rascore/ras_aca.c +672 −0
  578. drivers/gpu/drm/amd/ras/rascore/ras_aca.h +164 −0
  579. drivers/gpu/drm/amd/ras/rascore/ras_aca_v1_0.c +379 −0
  580. drivers/gpu/drm/amd/ras/rascore/ras_aca_v1_0.h +71 −0
  581. drivers/gpu/drm/amd/ras/rascore/ras_cmd.c +522 −0
  582. drivers/gpu/drm/amd/ras/rascore/ras_cmd.h +426 −0
  583. drivers/gpu/drm/amd/ras/rascore/ras_core.c +603 −0
  584. drivers/gpu/drm/amd/ras/rascore/ras_cper.c +315 −0
  585. drivers/gpu/drm/amd/ras/rascore/ras_cper.h +304 −0
  586. drivers/gpu/drm/amd/ras/rascore/ras_eeprom.c +1339 −0
  587. drivers/gpu/drm/amd/ras/rascore/ras_eeprom.h +197 −0
  588. drivers/gpu/drm/amd/ras/rascore/ras_gfx.c +70 −0
  589. drivers/gpu/drm/amd/ras/rascore/ras_gfx.h +43 −0
  590. drivers/gpu/drm/amd/ras/rascore/ras_gfx_v9_0.c +426 −0
  591. drivers/gpu/drm/amd/ras/rascore/ras_gfx_v9_0.h +259 −0
  592. drivers/gpu/drm/amd/ras/rascore/ras_log_ring.c +317 −0
  593. drivers/gpu/drm/amd/ras/rascore/ras_log_ring.h +93 −0
  594. drivers/gpu/drm/amd/ras/rascore/ras_mp1.c +81 −0
  595. drivers/gpu/drm/amd/ras/rascore/ras_mp1.h +50 −0
  596. drivers/gpu/drm/amd/ras/rascore/ras_mp1_v13_0.c +105 −0
  597. drivers/gpu/drm/amd/ras/rascore/ras_mp1_v13_0.h +30 −0
  598. drivers/gpu/drm/amd/ras/rascore/ras_nbio.c +96 −0
  599. drivers/gpu/drm/amd/ras/rascore/ras_nbio.h +46 −0
  600. drivers/gpu/drm/amd/ras/rascore/ras_nbio_v7_9.c +123 −0
  601. drivers/gpu/drm/amd/ras/rascore/ras_nbio_v7_9.h +31 −0
  602. drivers/gpu/drm/amd/ras/rascore/ras_process.c +322 −0
  603. drivers/gpu/drm/amd/ras/rascore/ras_process.h +53 −0
  604. drivers/gpu/drm/amd/ras/rascore/ras_psp.c +750 −0
  605. drivers/gpu/drm/amd/ras/rascore/ras_psp.h +145 −0
  606. drivers/gpu/drm/amd/ras/rascore/ras_psp_v13_0.c +46 −0
  607. drivers/gpu/drm/amd/ras/rascore/ras_psp_v13_0.h +31 −0
  608. drivers/gpu/drm/amd/ras/rascore/ras_ta_if.h +231 −0
  609. drivers/gpu/drm/amd/ras/rascore/ras_umc.c +707 −0
  610. drivers/gpu/drm/amd/ras/rascore/ras_umc.h +166 −0
  611. drivers/gpu/drm/amd/ras/rascore/ras_umc_v12_0.c +511 −0
  612. drivers/gpu/drm/amd/ras/rascore/ras_umc_v12_0.h +314 −0
  613. drivers/gpu/drm/arm/display/komeda/komeda_crtc.c +18 −13
  614. drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c +1 −0
  615. drivers/gpu/drm/arm/hdlcd_crtc.c +1 −0
  616. drivers/gpu/drm/arm/hdlcd_drv.c +1 −0
  617. drivers/gpu/drm/arm/malidp_drv.c +1 −0
  618. drivers/gpu/drm/arm/malidp_mw.c +1 −0
  619. drivers/gpu/drm/arm/malidp_planes.c +1 −1
  620. drivers/gpu/drm/armada/armada_crtc.c +1 −0
  621. drivers/gpu/drm/armada/armada_debugfs.c +1 −0
  622. drivers/gpu/drm/armada/armada_fb.c +1 −0
  623. drivers/gpu/drm/armada/armada_fbdev.c +2 −13
  624. drivers/gpu/drm/armada/armada_gem.c +1 −0
  625. drivers/gpu/drm/armada/armada_overlay.c +1 −0
  626. drivers/gpu/drm/armada/armada_plane.c +2 −6
  627. drivers/gpu/drm/ast/Makefile +2 −1
  628. drivers/gpu/drm/ast/ast_2000.c +108 −0
  629. drivers/gpu/drm/ast/ast_2100.c +92 −0
  630. drivers/gpu/drm/ast/ast_2200.c +92 −0
  631. drivers/gpu/drm/ast/ast_2300.c +135 −0
  632. drivers/gpu/drm/ast/ast_2400.c +100 −0
  633. drivers/gpu/drm/ast/ast_2500.c +106 −0
  634. drivers/gpu/drm/ast/ast_2600.c +72 −0
  635. drivers/gpu/drm/ast/ast_drv.c +66 −3
  636. drivers/gpu/drm/ast/ast_drv.h +92 −8
  637. drivers/gpu/drm/ast/ast_main.c +0 −268
  638. drivers/gpu/drm/ast/ast_mode.c +24 −34
  639. drivers/gpu/drm/ast/ast_tables.h +0 −60
  640. drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_crtc.c +11 −10
  641. drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c +8 −7
  642. drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.h +2 −1
  643. drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_output.c +2 −1
  644. drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_plane.c +43 −10
  645. drivers/gpu/drm/bridge/imx/Kconfig +11 −0
  646. drivers/gpu/drm/bridge/imx/Makefile +1 −0
  647. drivers/gpu/drm/bridge/imx/imx8mp-hdmi-pai.c +158 −0
  648. drivers/gpu/drm/bridge/imx/imx8mp-hdmi-tx.c +60 −5
  649. drivers/gpu/drm/bridge/imx/imx8qxp-ldb.c +1 −6
  650. drivers/gpu/drm/bridge/ite-it66121.c +33 −35
  651. drivers/gpu/drm/bridge/simple-bridge.c +10 −0
  652. drivers/gpu/drm/bridge/synopsys/Kconfig +8 −0
  653. drivers/gpu/drm/bridge/synopsys/dw-dp.c +2 −0
  654. drivers/gpu/drm/bridge/synopsys/dw-hdmi-gp-audio.c +5 −0
  655. drivers/gpu/drm/bridge/synopsys/dw-hdmi-qp.c +230 −5
  656. drivers/gpu/drm/bridge/synopsys/dw-hdmi-qp.h +14 −0
  657. drivers/gpu/drm/bridge/synopsys/dw-hdmi.c +17 −1
  658. drivers/gpu/drm/bridge/ti-sn65dsi86.c +112 −0
  659. drivers/gpu/drm/clients/drm_fbdev_client.c +23 −14
  660. drivers/gpu/drm/clients/drm_log.c +32 −11
  661. drivers/gpu/drm/display/drm_bridge_connector.c +48 −21
  662. drivers/gpu/drm/display/drm_dp_helper.c +213 −19
  663. drivers/gpu/drm/drm_atomic.c +200 −25
  664. drivers/gpu/drm/drm_atomic_helper.c +19 −5
  665. drivers/gpu/drm/drm_atomic_state_helper.c +5 −0
  666. drivers/gpu/drm/drm_atomic_uapi.c +160 −0
  667. drivers/gpu/drm/drm_bridge.c +52 −15
  668. drivers/gpu/drm/drm_buddy.c +239 −156
  669. drivers/gpu/drm/drm_client.c +96 −102
  670. drivers/gpu/drm/drm_client_event.c +15 −14
  671. drivers/gpu/drm/drm_client_modeset.c +44 −0
  672. drivers/gpu/drm/drm_client_sysrq.c +65 −0
  673. drivers/gpu/drm/drm_color_mgmt.c +43 −0
  674. drivers/gpu/drm/drm_colorop.c +599 −0
  675. drivers/gpu/drm/drm_connector.c +1 −0
  676. drivers/gpu/drm/drm_crtc.c +35 −0
  677. drivers/gpu/drm/drm_crtc_internal.h +1 −0
  678. drivers/gpu/drm/drm_displayid.c +43 −15
  679. drivers/gpu/drm/drm_displayid_internal.h +2 −0
  680. drivers/gpu/drm/drm_drv.c +3 −0
  681. drivers/gpu/drm/drm_dumb_buffers.c +160 −11
  682. drivers/gpu/drm/drm_edid.c +3 −0
  683. drivers/gpu/drm/drm_fb_helper.c +25 −113
  684. drivers/gpu/drm/drm_fbdev_dma.c +7 −18
  685. drivers/gpu/drm/drm_fbdev_shmem.c +6 −15
  686. drivers/gpu/drm/drm_fbdev_ttm.c +7 −17
  687. drivers/gpu/drm/drm_file.c +1 −1
  688. drivers/gpu/drm/drm_format_helper.c +0 −91
  689. drivers/gpu/drm/drm_framebuffer.c +1 −1
  690. drivers/gpu/drm/drm_gem.c +9 −19
  691. drivers/gpu/drm/drm_gem_atomic_helper.c +0 −2
  692. drivers/gpu/drm/drm_gem_dma_helper.c +7 −3
  693. drivers/gpu/drm/drm_gem_framebuffer_helper.c +1 −0
  694. drivers/gpu/drm/drm_gem_shmem_helper.c +73 −41
  695. drivers/gpu/drm/drm_gem_ttm_helper.c +1 −0
  696. drivers/gpu/drm/drm_gem_vram_helper.c +6 −5
  697. drivers/gpu/drm/drm_gpusvm.c +4 −2
  698. drivers/gpu/drm/drm_gpuvm.c +191 −0
  699. drivers/gpu/drm/drm_internal.h +11 −0
  700. drivers/gpu/drm/drm_ioctl.c +7 −0
  701. drivers/gpu/drm/drm_mipi_dbi.c +2 −1
  702. drivers/gpu/drm/drm_mm.c +1 −0
  703. drivers/gpu/drm/drm_mode_config.c +7 −0
  704. drivers/gpu/drm/drm_mode_object.c +18 −0
  705. drivers/gpu/drm/drm_modeset_helper.c +3 −3
  706. drivers/gpu/drm/drm_plane.c +59 −0
  707. drivers/gpu/drm/drm_prime.c +1 −0
  708. drivers/gpu/drm/drm_vblank.c +171 −3
  709. drivers/gpu/drm/drm_vblank_helper.c +176 −0
  710. drivers/gpu/drm/drm_vblank_work.c +1 −1
  711. drivers/gpu/drm/etnaviv/etnaviv_buffer.c +1 −0
  712. drivers/gpu/drm/etnaviv/etnaviv_drv.c +1 −0
  713. drivers/gpu/drm/etnaviv/etnaviv_gem.c +1 −0
  714. drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c +1 −0
  715. drivers/gpu/drm/etnaviv/etnaviv_gpu.c +2 −0
  716. drivers/gpu/drm/etnaviv/etnaviv_hwdb.c +32 −0
  717. drivers/gpu/drm/etnaviv/etnaviv_mmu.c +2 −0
  718. drivers/gpu/drm/exynos/exynos5433_drm_decon.c +1 −0
  719. drivers/gpu/drm/exynos/exynos7_drm_decon.c +1 −0
  720. drivers/gpu/drm/exynos/exynos_drm_fb.c +1 −0
  721. drivers/gpu/drm/exynos/exynos_drm_fbdev.c +2 −10
  722. drivers/gpu/drm/exynos/exynos_drm_fimd.c +1 −0
  723. drivers/gpu/drm/exynos/exynos_drm_g2d.c +1 −0
  724. drivers/gpu/drm/exynos/exynos_drm_gem.c +6 −3
  725. drivers/gpu/drm/exynos/exynos_drm_ipp.c +1 −0
  726. drivers/gpu/drm/exynos/exynos_drm_plane.c +2 −1
  727. drivers/gpu/drm/exynos/exynos_drm_vidi.c +1 −0
  728. drivers/gpu/drm/exynos/exynos_mixer.c +1 −0
  729. drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_plane.c +1 −0
  730. drivers/gpu/drm/gma500/backlight.c +2 −0
  731. drivers/gpu/drm/gma500/cdv_device.c +1 −0
  732. drivers/gpu/drm/gma500/cdv_intel_display.c +1 −0
  733. drivers/gpu/drm/gma500/cdv_intel_dp.c +1 −0
  734. drivers/gpu/drm/gma500/cdv_intel_hdmi.c +1 −0
  735. drivers/gpu/drm/gma500/cdv_intel_lvds.c +1 −0
  736. drivers/gpu/drm/gma500/fbdev.c +1 −57
  737. drivers/gpu/drm/gma500/gem.c +1 −0
  738. drivers/gpu/drm/gma500/intel_bios.c +1 −0
  739. drivers/gpu/drm/gma500/intel_gmbus.c +2 −0
  740. drivers/gpu/drm/gma500/mid_bios.c +1 −0
  741. drivers/gpu/drm/gma500/oaktrail_crtc.c +1 −0
  742. drivers/gpu/drm/gma500/oaktrail_hdmi.c +1 −0
  743. drivers/gpu/drm/gma500/oaktrail_hdmi_i2c.c +3 −0
  744. drivers/gpu/drm/gma500/oaktrail_lvds.c +1 −0
  745. drivers/gpu/drm/gma500/opregion.c +3 −0
  746. drivers/gpu/drm/gma500/psb_drv.c +1 −0
  747. drivers/gpu/drm/gma500/psb_intel_display.c +1 −0
  748. drivers/gpu/drm/gma500/psb_intel_lvds.c +1 −0
  749. drivers/gpu/drm/gma500/psb_intel_sdvo.c +1 −0
  750. drivers/gpu/drm/gma500/psb_irq.c +24 −13
  751. drivers/gpu/drm/gud/gud_connector.c +4 −4
  752. drivers/gpu/drm/gud/gud_drv.c +24 −21
  753. drivers/gpu/drm/gud/gud_pipe.c +6 −6
  754. drivers/gpu/drm/hisilicon/kirin/kirin_drm_ade.c +1 −0
  755. drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c +1 −0
  756. drivers/gpu/drm/hyperv/hyperv_drm_drv.c +1 −0
  757. drivers/gpu/drm/hyperv/hyperv_drm_modeset.c +12 −0
  758. drivers/gpu/drm/i915/Makefile +13 −0
  759. drivers/gpu/drm/i915/display/g4x_dp.c +1 −1
  760. drivers/gpu/drm/i915/display/hsw_ips.c +29 −34
  761. drivers/gpu/drm/i915/display/i9xx_plane.c +15 −19
  762. drivers/gpu/drm/i915/display/i9xx_plane.h +3 −2
  763. drivers/gpu/drm/i915/display/i9xx_wm.c +5 −4
  764. drivers/gpu/drm/i915/display/icl_dsi.c +18 −21
  765. drivers/gpu/drm/i915/display/intel_acpi.c +1 −1
  766. drivers/gpu/drm/i915/display/intel_alpm.c +39 −52
  767. drivers/gpu/drm/i915/display/intel_alpm.h +1 −1
  768. drivers/gpu/drm/i915/display/intel_backlight.c +1 −1
  769. drivers/gpu/drm/i915/display/intel_bios.c +4 −5
  770. drivers/gpu/drm/i915/display/intel_bo.c +30 −10
  771. drivers/gpu/drm/i915/display/intel_bo.h +4 −4
  772. drivers/gpu/drm/i915/display/intel_bw.c +45 −310
  773. drivers/gpu/drm/i915/display/intel_bw.h +1 −5
  774. drivers/gpu/drm/i915/display/intel_casf.c +290 −0
  775. drivers/gpu/drm/i915/display/intel_casf.h +21 −0
  776. drivers/gpu/drm/i915/display/intel_casf_regs.h +33 −0
  777. drivers/gpu/drm/i915/display/intel_cdclk.c +373 −139
  778. drivers/gpu/drm/i915/display/intel_cdclk.h +12 −4
  779. drivers/gpu/drm/i915/display/intel_color.c +9 −8
  780. drivers/gpu/drm/i915/display/intel_combo_phy.c +1 −1
  781. drivers/gpu/drm/i915/display/intel_connector.c +2 −1
  782. drivers/gpu/drm/i915/display/intel_crt.c +8 −8
  783. drivers/gpu/drm/i915/display/intel_crtc.c +97 −2
  784. drivers/gpu/drm/i915/display/intel_crtc.h +11 −0
  785. drivers/gpu/drm/i915/display/intel_crtc_state_dump.c +10 −6
  786. drivers/gpu/drm/i915/display/intel_cursor.c +26 −6
  787. drivers/gpu/drm/i915/display/intel_cursor.h +2 −0
  788. drivers/gpu/drm/i915/display/intel_cx0_phy.c +117 −132
  789. drivers/gpu/drm/i915/display/intel_cx0_phy.h +21 −0
  790. drivers/gpu/drm/i915/display/intel_cx0_phy_regs.h +20 −12
  791. drivers/gpu/drm/i915/display/intel_dbuf_bw.c +295 −0
  792. drivers/gpu/drm/i915/display/intel_dbuf_bw.h +37 −0
  793. drivers/gpu/drm/i915/display/intel_ddi.c +53 −24
  794. drivers/gpu/drm/i915/display/intel_ddi_buf_trans.c +81 −2
  795. drivers/gpu/drm/i915/display/intel_ddi_buf_trans.h +9 −0
  796. drivers/gpu/drm/i915/display/intel_de.h +54 −53
  797. drivers/gpu/drm/i915/display/intel_display.c +216 −146
  798. drivers/gpu/drm/i915/display/intel_display.h +8 −8
  799. drivers/gpu/drm/i915/display/intel_display_conversion.c +13 −7
  800. drivers/gpu/drm/i915/display/intel_display_core.h +30 −4
  801. drivers/gpu/drm/i915/display/intel_display_debugfs.c +9 −3
  802. drivers/gpu/drm/i915/display/intel_display_device.c +5 −1
  803. drivers/gpu/drm/i915/display/intel_display_device.h +7 −2
  804. drivers/gpu/drm/i915/display/intel_display_driver.c +10 −13
  805. drivers/gpu/drm/i915/display/intel_display_irq.c +103 −26
  806. drivers/gpu/drm/i915/display/intel_display_irq.h +5 −3
  807. drivers/gpu/drm/i915/display/intel_display_jiffies.h +43 −0
  808. drivers/gpu/drm/i915/display/intel_display_power.c +12 −9
  809. drivers/gpu/drm/i915/display/intel_display_power_map.c +14 −12
  810. drivers/gpu/drm/i915/display/intel_display_power_well.c +39 −19
  811. drivers/gpu/drm/i915/display/intel_display_reset.c +1 −0
  812. drivers/gpu/drm/i915/display/intel_display_rpm.c +13 −20
  813. drivers/gpu/drm/i915/display/intel_display_types.h +61 −12
  814. drivers/gpu/drm/i915/display/intel_display_utils.c +32 −0
  815. drivers/gpu/drm/i915/display/intel_display_utils.h +31 −0
  816. drivers/gpu/drm/i915/display/intel_display_wa.c +4 −1
  817. drivers/gpu/drm/i915/display/intel_display_wa.h +1 −0
  818. drivers/gpu/drm/i915/display/intel_dmc.c +42 −20
  819. drivers/gpu/drm/i915/display/intel_dmc_wl.c +12 −13
  820. drivers/gpu/drm/i915/display/intel_dp.c +237 −26
  821. drivers/gpu/drm/i915/display/intel_dp.h +11 −1
  822. drivers/gpu/drm/i915/display/intel_dp_aux.c +4 −4
  823. drivers/gpu/drm/i915/display/intel_dp_hdcp.c +7 −7
  824. drivers/gpu/drm/i915/display/intel_dp_link_training.c +2 −1
  825. drivers/gpu/drm/i915/display/intel_dp_mst.c +33 −16
  826. drivers/gpu/drm/i915/display/intel_dpio_phy.c +6 −6
  827. drivers/gpu/drm/i915/display/intel_dpll.c +32 −3
  828. drivers/gpu/drm/i915/display/intel_dpll_mgr.c +14 −15
  829. drivers/gpu/drm/i915/display/intel_dpll_mgr.h +11 −0
  830. drivers/gpu/drm/i915/display/intel_dpt.c +2 −0
  831. drivers/gpu/drm/i915/display/intel_dsb.c +33 −24
  832. drivers/gpu/drm/i915/display/intel_dsb.h +2 −2
  833. drivers/gpu/drm/i915/display/intel_dsi_vbt.c +24 −13
  834. drivers/gpu/drm/i915/display/intel_dvo.c +1 −1
  835. drivers/gpu/drm/i915/display/intel_fb.c +21 −20
  836. drivers/gpu/drm/i915/display/intel_fb_bo.c +2 −2
  837. drivers/gpu/drm/i915/display/intel_fb_bo.h +1 −2
  838. drivers/gpu/drm/i915/display/intel_fb_pin.c +2 −0
  839. drivers/gpu/drm/i915/display/intel_fbc.c +145 −58
  840. drivers/gpu/drm/i915/display/intel_fbc.h +3 −0
  841. drivers/gpu/drm/i915/display/intel_fbdev.c +60 −12
  842. drivers/gpu/drm/i915/display/intel_fbdev_fb.c +20 −38
  843. drivers/gpu/drm/i915/display/intel_fbdev_fb.h +8 −6
  844. drivers/gpu/drm/i915/display/intel_fdi.c +1 −1
  845. drivers/gpu/drm/i915/display/intel_flipq.c +7 −7
  846. drivers/gpu/drm/i915/display/intel_frontbuffer.c +23 −129
  847. drivers/gpu/drm/i915/display/intel_frontbuffer.h +6 −12
  848. drivers/gpu/drm/i915/display/intel_gmbus.c +2 −1
  849. drivers/gpu/drm/i915/display/intel_hdcp.c +24 −27
  850. drivers/gpu/drm/i915/display/intel_hdcp_gsc.c +1 −1
  851. drivers/gpu/drm/i915/display/intel_hdmi.c +17 −3
  852. drivers/gpu/drm/i915/display/intel_hdmi.h +1 −0
  853. drivers/gpu/drm/i915/display/intel_hotplug.c +3 −2
  854. drivers/gpu/drm/i915/display/intel_hotplug_irq.c +4 −1
  855. drivers/gpu/drm/i915/display/intel_link_bw.c +9 −8
  856. drivers/gpu/drm/i915/display/intel_link_bw.h +1 −1
  857. drivers/gpu/drm/i915/display/intel_lspcon.c +1 −1
  858. drivers/gpu/drm/i915/display/intel_lt_phy.c +2327 −0
  859. drivers/gpu/drm/i915/display/intel_lt_phy.h +47 −0
  860. drivers/gpu/drm/i915/display/intel_lt_phy_regs.h +90 −0
  861. drivers/gpu/drm/i915/display/intel_lvds.c +3 −3
  862. drivers/gpu/drm/i915/display/intel_modeset_setup.c +11 −3
  863. drivers/gpu/drm/i915/display/intel_modeset_verify.c +2 −0
  864. drivers/gpu/drm/i915/display/intel_overlay.c +2 −5
  865. drivers/gpu/drm/i915/display/intel_pch.c +2 −2
  866. drivers/gpu/drm/i915/display/intel_pch_display.c +6 −6
  867. drivers/gpu/drm/i915/display/intel_pch_refclk.c +5 −7
  868. drivers/gpu/drm/i915/display/intel_pfit.c +1 −1
  869. drivers/gpu/drm/i915/display/intel_pipe_crc.c +2 −0
  870. drivers/gpu/drm/i915/display/intel_plane.c +13 −52
  871. drivers/gpu/drm/i915/display/intel_plane.h +0 −3
  872. drivers/gpu/drm/i915/display/intel_plane_initial.c +9 −4
  873. drivers/gpu/drm/i915/display/intel_pmdemand.c +11 −10
  874. drivers/gpu/drm/i915/display/intel_pps.c +2 −1
  875. drivers/gpu/drm/i915/display/intel_psr.c +329 −110
  876. drivers/gpu/drm/i915/display/intel_psr.h +4 −1
  877. drivers/gpu/drm/i915/display/intel_qp_tables.c +1 −1
  878. drivers/gpu/drm/i915/display/intel_sbi.c +4 −2
  879. drivers/gpu/drm/i915/display/intel_snps_hdmi_pll.c +2 −0
  880. drivers/gpu/drm/i915/display/intel_snps_phy.c +5 −5
  881. drivers/gpu/drm/i915/display/intel_sprite.c +5 −7
  882. drivers/gpu/drm/i915/display/intel_tc.c +18 −5
  883. drivers/gpu/drm/i915/display/intel_tc.h +3 −0
  884. drivers/gpu/drm/i915/display/intel_vblank.c +14 −2
  885. drivers/gpu/drm/i915/display/intel_vblank.h +2 −0
  886. drivers/gpu/drm/i915/display/intel_vdsc.c +25 −1
  887. drivers/gpu/drm/i915/display/intel_vdsc.h +3 −0
  888. drivers/gpu/drm/i915/display/intel_vrr.c +312 −186
  889. drivers/gpu/drm/i915/display/intel_vrr.h +3 −2
  890. drivers/gpu/drm/i915/display/skl_prefill.c +157 −0
  891. drivers/gpu/drm/i915/display/skl_prefill.h +46 −0
  892. drivers/gpu/drm/i915/display/skl_scaler.c +221 −13
  893. drivers/gpu/drm/i915/display/skl_scaler.h +17 −0
  894. drivers/gpu/drm/i915/display/skl_universal_plane.c +109 −84
  895. drivers/gpu/drm/i915/display/skl_universal_plane_regs.h +18 −6
  896. drivers/gpu/drm/i915/display/skl_watermark.c +213 −168
  897. drivers/gpu/drm/i915/display/skl_watermark.h +3 −0
  898. drivers/gpu/drm/i915/display/skl_watermark_regs.h +28 −24
  899. drivers/gpu/drm/i915/display/vlv_clock.c +88 −0
  900. drivers/gpu/drm/i915/display/vlv_clock.h +38 −0
  901. drivers/gpu/drm/i915/display/vlv_dsi.c +28 −28
  902. drivers/gpu/drm/i915/display/vlv_dsi_pll.c +4 −4
  903. drivers/gpu/drm/i915/gem/i915_gem_context.c +1 −0
  904. drivers/gpu/drm/i915/gem/i915_gem_create.c +3 −2
  905. drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +2 −1
  906. drivers/gpu/drm/i915/gem/i915_gem_mman.c +3 −2
  907. drivers/gpu/drm/i915/gem/i915_gem_object.c +9 −8
  908. drivers/gpu/drm/i915/gem/i915_gem_object.h +1 −0
  909. drivers/gpu/drm/i915/gem/i915_gem_object_frontbuffer.c +103 −0
  910. drivers/gpu/drm/i915/gem/i915_gem_object_frontbuffer.h +18 −36
  911. drivers/gpu/drm/i915/gem/i915_gem_object_types.h +1 −1
  912. drivers/gpu/drm/i915/gem/i915_gem_pages.c +3 −1
  913. drivers/gpu/drm/i915/gem/i915_gem_phys.c +1 −0
  914. drivers/gpu/drm/i915/gem/i915_gem_shmem.c +12 −3
  915. drivers/gpu/drm/i915/gem/i915_gem_shrinker.c +2 −0
  916. drivers/gpu/drm/i915/gem/i915_gem_stolen.c +78 −27
  917. drivers/gpu/drm/i915/gem/i915_gem_stolen.h +16 −18
  918. drivers/gpu/drm/i915/gem/i915_gem_tiling.c +3 −2
  919. drivers/gpu/drm/i915/gem/i915_gem_ttm.c +5 −3
  920. drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.c +1 −0
  921. drivers/gpu/drm/i915/gem/i915_gem_userptr.c +2 −0
  922. drivers/gpu/drm/i915/gem/i915_gem_wait.c +7 −0
  923. drivers/gpu/drm/i915/gem/i915_gemfs.c +2 −0
  924. drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c +2 −0
  925. drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +2 −1
  926. drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +2 −0
  927. drivers/gpu/drm/i915/gt/gen2_engine_cs.c +4 −4
  928. drivers/gpu/drm/i915/gt/gen8_engine_cs.c +2 −0
  929. drivers/gpu/drm/i915/gt/intel_breadcrumbs.c +2 −0
  930. drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c +3 −0
  931. drivers/gpu/drm/i915/gt/intel_engine_user.c +2 −0
  932. drivers/gpu/drm/i915/gt/intel_execlists_submission.c +2 −0
  933. drivers/gpu/drm/i915/gt/intel_ggtt.c +1 −0
  934. drivers/gpu/drm/i915/gt/intel_ggtt_fencing.c +2 −0
  935. drivers/gpu/drm/i915/gt/intel_ggtt_gmch.c +1 −0
  936. drivers/gpu/drm/i915/gt/intel_gt_debugfs.c +2 −0
  937. drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.c +2 −0
  938. drivers/gpu/drm/i915/gt/intel_lrc.c +2 −0
  939. drivers/gpu/drm/i915/gt/intel_mocs.c +2 −0
  940. drivers/gpu/drm/i915/gt/intel_rc6.c +4 −1
  941. drivers/gpu/drm/i915/gt/intel_region_lmem.c +2 −0
  942. drivers/gpu/drm/i915/gt/intel_renderstate.c +2 −0
  943. drivers/gpu/drm/i915/gt/intel_rps.c +8 −10
  944. drivers/gpu/drm/i915/gt/intel_sa_media.c +1 −0
  945. drivers/gpu/drm/i915/gt/intel_sseu.c +2 −0
  946. drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c +2 −0
  947. drivers/gpu/drm/i915/gt/intel_timeline.c +1 −0
  948. drivers/gpu/drm/i915/gt/intel_tlb.h +1 −1
  949. drivers/gpu/drm/i915/gt/intel_wopcm.c +2 −0
  950. drivers/gpu/drm/i915/gt/intel_workarounds.c +5 −4
  951. drivers/gpu/drm/i915/gt/selftest_context.c +2 −0
  952. drivers/gpu/drm/i915/gt/selftest_execlists.c +3 −0
  953. drivers/gpu/drm/i915/gt/selftest_migrate.c +8 −1
  954. drivers/gpu/drm/i915/gt/uc/intel_gsc_uc_heci_cmd_submit.c +2 −0
  955. drivers/gpu/drm/i915/gvt/aperture_gm.c +2 −0
  956. drivers/gpu/drm/i915/gvt/cfg_space.c +2 −0
  957. drivers/gpu/drm/i915/gvt/cmd_parser.c +2 −0
  958. drivers/gpu/drm/i915/gvt/display.c +1 −0
  959. drivers/gpu/drm/i915/gvt/dmabuf.c +1 −0
  960. drivers/gpu/drm/i915/gvt/edid.c +1 −0
  961. drivers/gpu/drm/i915/gvt/gtt.c +2 −0
  962. drivers/gpu/drm/i915/gvt/handlers.c +1 −0
  963. drivers/gpu/drm/i915/gvt/interrupt.c +2 −0
  964. drivers/gpu/drm/i915/gvt/kvmgt.c +19 −17
  965. drivers/gpu/drm/i915/gvt/mmio.c +5 −2
  966. drivers/gpu/drm/i915/gvt/mmio_context.c +2 −0
  967. drivers/gpu/drm/i915/gvt/scheduler.c +2 −0
  968. drivers/gpu/drm/i915/gvt/vgpu.c +2 −0
  969. drivers/gpu/drm/i915/i915_cmd_parser.c +1 −0
  970. drivers/gpu/drm/i915/i915_config.c +1 −1
  971. drivers/gpu/drm/i915/i915_debugfs.c +1 −0
  972. drivers/gpu/drm/i915/i915_driver.c +87 −68
  973. drivers/gpu/drm/i915/i915_driver.h +2 −0
  974. drivers/gpu/drm/i915/i915_drv.h +6 −17
  975. drivers/gpu/drm/i915/i915_gem.c +3 −0
  976. drivers/gpu/drm/i915/i915_getparam.c +2 −0
  977. drivers/gpu/drm/i915/i915_irq.c +27 −81
  978. drivers/gpu/drm/i915/i915_jiffies.h +16 −0
  979. drivers/gpu/drm/i915/i915_mmio_range.c +18 −0
  980. drivers/gpu/drm/i915/i915_mmio_range.h +19 −0
  981. drivers/gpu/drm/i915/i915_module.c +1 −0
  982. drivers/gpu/drm/i915/i915_perf.c +28 −39
  983. drivers/gpu/drm/i915/i915_pmu.c +4 −2
  984. drivers/gpu/drm/i915/i915_query.c +2 −0
  985. drivers/gpu/drm/i915/i915_reg.h +1 −0
  986. drivers/gpu/drm/i915/i915_reg_defs.h +10 −0
  987. drivers/gpu/drm/i915/i915_request.c +2 −0
  988. drivers/gpu/drm/i915/i915_switcheroo.c +2 −0
  989. drivers/gpu/drm/i915/i915_sysfs.c +2 −0
  990. drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +2 −2
  991. drivers/gpu/drm/i915/i915_utils.c +1 −0
  992. drivers/gpu/drm/i915/i915_utils.h +4 −37
  993. drivers/gpu/drm/i915/i915_vgpu.c +2 −0
  994. drivers/gpu/drm/i915/i915_vma.c +6 −4
  995. drivers/gpu/drm/i915/intel_clock_gating.c +2 −0
  996. drivers/gpu/drm/i915/intel_gvt.c +2 −0
  997. drivers/gpu/drm/i915/intel_memory_region.c +1 −0
  998. drivers/gpu/drm/i915/intel_pcode.c +2 −0
  999. drivers/gpu/drm/i915/intel_region_ttm.c +1 −1
  1000. drivers/gpu/drm/i915/intel_runtime_pm.c +77 −0
  1001. drivers/gpu/drm/i915/intel_runtime_pm.h +3 −0
  1002. drivers/gpu/drm/i915/intel_step.c +2 −0
  1003. drivers/gpu/drm/i915/intel_uncore.c +9 −7
  1004. drivers/gpu/drm/i915/intel_uncore.h +1 −7
  1005. drivers/gpu/drm/i915/intel_wakeref.c +2 −0
  1006. drivers/gpu/drm/i915/pxp/intel_pxp.c +2 −0
  1007. drivers/gpu/drm/i915/pxp/intel_pxp_gsccs.c +2 −0
  1008. drivers/gpu/drm/i915/pxp/intel_pxp_huc.c +2 −0
  1009. drivers/gpu/drm/i915/pxp/intel_pxp_session.c +2 −0
  1010. drivers/gpu/drm/i915/selftests/i915_active.c +2 −0
  1011. drivers/gpu/drm/i915/selftests/i915_gem_gtt.c +4 −0
  1012. drivers/gpu/drm/i915/selftests/i915_request.c +2 −0
  1013. drivers/gpu/drm/i915/selftests/i915_selftest.c +1 −0
  1014. drivers/gpu/drm/i915/selftests/intel_uncore.c +2 −2
  1015. drivers/gpu/drm/i915/selftests/mock_gem_device.c +3 −1
  1016. drivers/gpu/drm/i915/soc/intel_dram.c +10 −5
  1017. drivers/gpu/drm/i915/soc/intel_dram.h +1 −0
  1018. drivers/gpu/drm/i915/soc/intel_gmch.c +1 −0
  1019. drivers/gpu/drm/i915/soc/intel_rom.c +4 −3
  1020. drivers/gpu/drm/i915/soc/intel_rom.h +3 −3
  1021. drivers/gpu/drm/i915/vlv_iosf_sb.c +2 −0
  1022. drivers/gpu/drm/imagination/pvr_ccb.c +1 −0
  1023. drivers/gpu/drm/imagination/pvr_device.c +1 −1
  1024. drivers/gpu/drm/imagination/pvr_fw.c +1 −0
  1025. drivers/gpu/drm/imagination/pvr_fw_meta.c +2 −0
  1026. drivers/gpu/drm/imagination/pvr_fw_trace.c +1 −0
  1027. drivers/gpu/drm/imagination/pvr_power.c +1 −0
  1028. drivers/gpu/drm/imagination/pvr_vm.c +1 −0
  1029. drivers/gpu/drm/imx/dc/dc-ed.c +4 −4
  1030. drivers/gpu/drm/imx/dc/dc-fg.c +2 −2
  1031. drivers/gpu/drm/imx/dc/dc-fu.c +5 −5
  1032. drivers/gpu/drm/imx/dc/dc-fu.h +2 −2
  1033. drivers/gpu/drm/imx/dc/dc-lb.c +14 −14
  1034. drivers/gpu/drm/imx/dc/dc-plane.c +1 −1
  1035. drivers/gpu/drm/imx/dcss/dcss-plane.c +3 −2
  1036. drivers/gpu/drm/imx/ipuv3/dw_hdmi-imx.c +0 −1
  1037. drivers/gpu/drm/imx/ipuv3/imx-drm-core.c +25 −6
  1038. drivers/gpu/drm/imx/ipuv3/imx-ldb.c +0 −1
  1039. drivers/gpu/drm/imx/ipuv3/imx-tve.c +10 −8
  1040. drivers/gpu/drm/imx/ipuv3/ipuv3-plane.c +2 −2
  1041. drivers/gpu/drm/imx/ipuv3/parallel-display.c +2 −3
  1042. drivers/gpu/drm/imx/lcdc/imx-lcdc.c +1 −0
  1043. drivers/gpu/drm/ingenic/ingenic-drm-drv.c +9 −4
  1044. drivers/gpu/drm/ingenic/ingenic-ipu.c +2 −2
  1045. drivers/gpu/drm/kmb/kmb_drv.c +1 −0
  1046. drivers/gpu/drm/kmb/kmb_plane.c +2 −2
  1047. drivers/gpu/drm/lima/lima_sched.c +2 −0
  1048. drivers/gpu/drm/logicvc/logicvc_layer.c +2 −2
  1049. drivers/gpu/drm/loongson/lsdc_benchmark.c +1 −0
  1050. drivers/gpu/drm/loongson/lsdc_crtc.c +1 −0
  1051. drivers/gpu/drm/loongson/lsdc_debugfs.c +1 −0
  1052. drivers/gpu/drm/loongson/lsdc_drv.c +1 −0
  1053. drivers/gpu/drm/loongson/lsdc_gem.c +10 −22
  1054. drivers/gpu/drm/loongson/lsdc_i2c.c +1 −0
  1055. drivers/gpu/drm/loongson/lsdc_irq.c +1 −0
  1056. drivers/gpu/drm/loongson/lsdc_output_7a1000.c +1 −0
  1057. drivers/gpu/drm/loongson/lsdc_output_7a2000.c +1 −0
  1058. drivers/gpu/drm/loongson/lsdc_pixpll.c +1 −0
  1059. drivers/gpu/drm/loongson/lsdc_plane.c +2 −1
  1060. drivers/gpu/drm/loongson/lsdc_ttm.c +3 −1
  1061. drivers/gpu/drm/mcde/mcde_clk_div.c +8 −5
  1062. drivers/gpu/drm/mcde/mcde_display.c +1 −0
  1063. drivers/gpu/drm/mediatek/Kconfig +22 −1
  1064. drivers/gpu/drm/mediatek/Makefile +3 −0
  1065. drivers/gpu/drm/mediatek/mtk_crtc.c +1 −0
  1066. drivers/gpu/drm/mediatek/mtk_ddp_comp.c +28 −5
  1067. drivers/gpu/drm/mediatek/mtk_ddp_comp.h +1 −1
  1068. drivers/gpu/drm/mediatek/mtk_disp_ccorr.c +1 −22
  1069. drivers/gpu/drm/mediatek/mtk_disp_ovl_adaptor.c +12 −0
  1070. drivers/gpu/drm/mediatek/mtk_dp.c +1 −0
  1071. drivers/gpu/drm/mediatek/mtk_drm_drv.c +2 −2
  1072. drivers/gpu/drm/mediatek/mtk_gem.c +1 −0
  1073. drivers/gpu/drm/mediatek/mtk_hdmi.c +21 −518
  1074. drivers/gpu/drm/mediatek/mtk_hdmi_common.c +456 −0
  1075. drivers/gpu/drm/mediatek/mtk_hdmi_common.h +198 −0
  1076. drivers/gpu/drm/mediatek/mtk_hdmi_ddc_v2.c +396 −0
  1077. drivers/gpu/drm/mediatek/mtk_hdmi_regs_v2.h +263 −0
  1078. drivers/gpu/drm/mediatek/mtk_hdmi_v2.c +1521 −0
  1079. drivers/gpu/drm/mediatek/mtk_plane.c +3 −1
  1080. drivers/gpu/drm/meson/meson_overlay.c +1 −0
  1081. drivers/gpu/drm/meson/meson_plane.c +1 −0
  1082. drivers/gpu/drm/mgag200/mgag200_drv.c +1 −0
  1083. drivers/gpu/drm/mgag200/mgag200_g200.c +1 −0
  1084. drivers/gpu/drm/mgag200/mgag200_g200eh.c +1 −0
  1085. drivers/gpu/drm/mgag200/mgag200_g200eh3.c +1 −0
  1086. drivers/gpu/drm/mgag200/mgag200_g200eh5.c +1 −0
  1087. drivers/gpu/drm/mgag200/mgag200_g200er.c +1 −0
  1088. drivers/gpu/drm/mgag200/mgag200_g200ev.c +1 −0
  1089. drivers/gpu/drm/mgag200/mgag200_g200ew3.c +1 −0
  1090. drivers/gpu/drm/mgag200/mgag200_g200se.c +1 −0
  1091. drivers/gpu/drm/mgag200/mgag200_g200wb.c +1 −0
  1092. drivers/gpu/drm/mgag200/mgag200_vga.c +1 −0
  1093. drivers/gpu/drm/mgag200/mgag200_vga_bmc.c +1 −0
  1094. drivers/gpu/drm/msm/Makefile +2 −0
  1095. drivers/gpu/drm/msm/adreno/a2xx_catalog.c +4 −3
  1096. drivers/gpu/drm/msm/adreno/a2xx_gpu.c +27 −25
  1097. drivers/gpu/drm/msm/adreno/a2xx_gpu.h +2 −0
  1098. drivers/gpu/drm/msm/adreno/a3xx_catalog.c +7 −6
  1099. drivers/gpu/drm/msm/adreno/a3xx_gpu.c +27 −25
  1100. drivers/gpu/drm/msm/adreno/a3xx_gpu.h +2 −0
  1101. drivers/gpu/drm/msm/adreno/a4xx_catalog.c +4 −3
  1102. drivers/gpu/drm/msm/adreno/a4xx_gpu.c +28 −26
  1103. drivers/gpu/drm/msm/adreno/a4xx_gpu.h +2 −0
  1104. drivers/gpu/drm/msm/adreno/a5xx_catalog.c +9 −8
  1105. drivers/gpu/drm/msm/adreno/a5xx_gpu.c +31 −30
  1106. drivers/gpu/drm/msm/adreno/a5xx_gpu.h +1 −0
  1107. drivers/gpu/drm/msm/adreno/a6xx_catalog.c +362 −23
  1108. drivers/gpu/drm/msm/adreno/a6xx_gmu.c +248 −82
  1109. drivers/gpu/drm/msm/adreno/a6xx_gmu.h +15 −10
  1110. drivers/gpu/drm/msm/adreno/a6xx_gpu.c +273 −165
  1111. drivers/gpu/drm/msm/adreno/a6xx_gpu.h +30 −1
  1112. drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c +3 −2
  1113. drivers/gpu/drm/msm/adreno/a6xx_gpu_state.h +37 −37
  1114. drivers/gpu/drm/msm/adreno/a6xx_hfi.c +71 −3
  1115. drivers/gpu/drm/msm/adreno/a6xx_hfi.h +17 −0
  1116. drivers/gpu/drm/msm/adreno/a8xx_gpu.c +1201 −0
  1117. drivers/gpu/drm/msm/adreno/adreno_device.c +3 −1
  1118. drivers/gpu/drm/msm/adreno/adreno_gen7_0_0_snapshot.h +210 −210
  1119. drivers/gpu/drm/msm/adreno/adreno_gen7_2_0_snapshot.h +166 −166
  1120. drivers/gpu/drm/msm/adreno/adreno_gen7_9_0_snapshot.h +235 −235
  1121. drivers/gpu/drm/msm/adreno/adreno_gpu.c +12 −0
  1122. drivers/gpu/drm/msm/adreno/adreno_gpu.h +46 −10
  1123. drivers/gpu/drm/msm/disp/dpu1/catalog/dpu_12_2_glymur.h +541 −0
  1124. drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c +73 −42
  1125. drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h +5 −3
  1126. drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c +20 −27
  1127. drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys.h +1 −1
  1128. drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c +1 −0
  1129. drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h +2 −1
  1130. drivers/gpu/drm/msm/disp/dpu1/dpu_hw_dsc.h +0 −6
  1131. drivers/gpu/drm/msm/disp/dpu1/dpu_hw_mdss.h +8 −0
  1132. drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c +1 −0
  1133. drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c +264 −164
  1134. drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h +4 −8
  1135. drivers/gpu/drm/msm/disp/dpu1/dpu_rm.c +5 −1
  1136. drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h +5 −5
  1137. drivers/gpu/drm/msm/disp/mdp4/mdp4_crtc.c +2 −1
  1138. drivers/gpu/drm/msm/disp/mdp5/mdp5_crtc.c +2 −1
  1139. drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c +3 −4
  1140. drivers/gpu/drm/msm/disp/msm_disp_snapshot.h +7 −6
  1141. drivers/gpu/drm/msm/dp/dp_ctrl.c +5 −5
  1142. drivers/gpu/drm/msm/dp/dp_display.c +9 −0
  1143. drivers/gpu/drm/msm/dp/dp_link.c +117 −0
  1144. drivers/gpu/drm/msm/dp/dp_link.h +5 −0
  1145. drivers/gpu/drm/msm/dp/dp_panel.c +9 −69
  1146. drivers/gpu/drm/msm/dp/dp_panel.h +0 −3
  1147. drivers/gpu/drm/msm/msm_fbdev.c +1 −10
  1148. drivers/gpu/drm/msm/msm_gem.c +28 −2
  1149. drivers/gpu/drm/msm/msm_gem_vma.c +22 −6
  1150. drivers/gpu/drm/msm/msm_gpu.c +13 −8
  1151. drivers/gpu/drm/msm/msm_mdss.c +2 −0
  1152. drivers/gpu/drm/msm/registers/adreno/a6xx.xml +1646 −553
  1153. drivers/gpu/drm/msm/registers/adreno/a6xx_enums.xml +1 −1
  1154. drivers/gpu/drm/msm/registers/adreno/a6xx_gmu.xml +153 −130
  1155. drivers/gpu/drm/msm/registers/adreno/a7xx_enums.xml +0 −7
  1156. drivers/gpu/drm/msm/registers/adreno/a8xx_descriptors.xml +121 −0
  1157. drivers/gpu/drm/msm/registers/adreno/a8xx_enums.xml +299 −0
  1158. drivers/gpu/drm/msm/registers/adreno/adreno_common.xml +12 −0
  1159. drivers/gpu/drm/msm/registers/adreno/adreno_pm4.xml +247 −114
  1160. drivers/gpu/drm/msm/registers/gen_header.py +12 −7
  1161. drivers/gpu/drm/mxsfb/lcdif_kms.c +1 −0
  1162. drivers/gpu/drm/mxsfb/mxsfb_kms.c +1 −0
  1163. drivers/gpu/drm/nouveau/Kconfig +1 −0
  1164. drivers/gpu/drm/nouveau/include/nvfw/hs.h +3 −1
  1165. drivers/gpu/drm/nouveau/include/nvkm/core/tegra.h +2 −0
  1166. drivers/gpu/drm/nouveau/include/nvkm/subdev/clk.h +1 −0
  1167. drivers/gpu/drm/nouveau/nouveau_bo.h +1 −1
  1168. drivers/gpu/drm/nouveau/nouveau_display.c +6 −5
  1169. drivers/gpu/drm/nouveau/nouveau_drv.h +4 −1
  1170. drivers/gpu/drm/nouveau/nouveau_gem.c +1 −1
  1171. drivers/gpu/drm/nouveau/nouveau_platform.c +20 −0
  1172. drivers/gpu/drm/nouveau/nouveau_ttm.c +4 −2
  1173. drivers/gpu/drm/nouveau/nouveau_uvmm.c +85 −17
  1174. drivers/gpu/drm/nouveau/nouveau_uvmm.h +1 −0
  1175. drivers/gpu/drm/nouveau/nvkm/engine/device/base.c +1 −0
  1176. drivers/gpu/drm/nouveau/nvkm/engine/device/tegra.c +4 −0
  1177. drivers/gpu/drm/nouveau/nvkm/subdev/clk/Kbuild +2 −0
  1178. drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a.c +5 −0
  1179. drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a.h +1 −0
  1180. drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a_devfreq.c +320 −0
  1181. drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a_devfreq.h +24 −0
  1182. drivers/gpu/drm/nouveau/nvkm/subdev/clk/gm20b.c +5 −0
  1183. drivers/gpu/drm/nouveau/nvkm/subdev/clk/gp10b.c +185 −0
  1184. drivers/gpu/drm/nouveau/nvkm/subdev/clk/gp10b.h +18 −0
  1185. drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c +1 −1
  1186. drivers/gpu/drm/nouveau/nvkm/subdev/fb/gb100.c +3 −0
  1187. drivers/gpu/drm/nouveau/nvkm/subdev/fb/gb202.c +3 −0
  1188. drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c +3 −0
  1189. drivers/gpu/drm/nouveau/nvkm/subdev/fb/gh100.c +3 −0
  1190. drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.c +3 −0
  1191. drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c +40 −29
  1192. drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c +2 −2
  1193. drivers/gpu/drm/nova/Kconfig +2 −0
  1194. drivers/gpu/drm/omapdrm/omap_crtc.c +1 −0
  1195. drivers/gpu/drm/omapdrm/omap_debugfs.c +1 −0
  1196. drivers/gpu/drm/omapdrm/omap_dmm_tiler.c +2 −0
  1197. drivers/gpu/drm/omapdrm/omap_drv.c +1 −0
  1198. drivers/gpu/drm/omapdrm/omap_encoder.c +1 −3
  1199. drivers/gpu/drm/omapdrm/omap_fb.c +1 −0
  1200. drivers/gpu/drm/omapdrm/omap_fbdev.c +2 −10
  1201. drivers/gpu/drm/omapdrm/omap_gem.c +8 −8
  1202. drivers/gpu/drm/omapdrm/omap_irq.c +1 −0
  1203. drivers/gpu/drm/omapdrm/omap_overlay.c +1 −0
  1204. drivers/gpu/drm/omapdrm/omap_plane.c +2 −1
  1205. drivers/gpu/drm/panel/Kconfig +56 −3
  1206. drivers/gpu/drm/panel/Makefile +4 −0
  1207. drivers/gpu/drm/panel/panel-edp.c +10 −0
  1208. drivers/gpu/drm/panel/panel-ilitek-ili9881c.c +883 −444
  1209. drivers/gpu/drm/panel/panel-ilitek-ili9882t.c +69 −0
  1210. drivers/gpu/drm/panel/panel-jadard-jd9365da-h3.c +9 −12
  1211. drivers/gpu/drm/panel/panel-lg-ld070wx3.c +184 −0
  1212. drivers/gpu/drm/panel/panel-newvision-nv3052c.c +74 −334
  1213. drivers/gpu/drm/panel/panel-ronbo-rb070d30.c +4 −4
  1214. drivers/gpu/drm/panel/panel-samsung-s6e3fc2x01.c +385 −0
  1215. drivers/gpu/drm/panel/panel-samsung-sofef00.c +67 −38
  1216. drivers/gpu/drm/panel/panel-sharp-lq079l1sx01.c +225 −0
  1217. drivers/gpu/drm/panel/panel-simple.c +96 −31
  1218. drivers/gpu/drm/panel/panel-synaptics-tddi.c +277 −0
  1219. drivers/gpu/drm/panel/panel-visionox-rm69299.c +69 −2
  1220. drivers/gpu/drm/panfrost/panfrost_devfreq.c +4 −2
  1221. drivers/gpu/drm/panfrost/panfrost_device.c +36 −32
  1222. drivers/gpu/drm/panfrost/panfrost_device.h +16 −8
  1223. drivers/gpu/drm/panfrost/panfrost_drv.c +183 −60
  1224. drivers/gpu/drm/panfrost/panfrost_dump.c +4 −4
  1225. drivers/gpu/drm/panfrost/panfrost_gem.c +5 −4
  1226. drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c +2 −2
  1227. drivers/gpu/drm/panfrost/panfrost_gpu.c +40 −26
  1228. drivers/gpu/drm/panfrost/panfrost_job.c +239 −97
  1229. drivers/gpu/drm/panfrost/panfrost_job.h +31 −7
  1230. drivers/gpu/drm/panfrost/panfrost_mmu.c +81 −34
  1231. drivers/gpu/drm/panfrost/panfrost_mmu.h +2 −1
  1232. drivers/gpu/drm/panfrost/panfrost_perfcnt.c +15 −11
  1233. drivers/gpu/drm/panthor/Makefile +1 −0
  1234. drivers/gpu/drm/panthor/panthor_devfreq.c +50 −14
  1235. drivers/gpu/drm/panthor/panthor_devfreq.h +2 −0
  1236. drivers/gpu/drm/panthor/panthor_device.c +37 −6
  1237. drivers/gpu/drm/panthor/panthor_device.h +22 −3
  1238. drivers/gpu/drm/panthor/panthor_drv.c +11 −2
  1239. drivers/gpu/drm/panthor/panthor_fw.c +115 −19
  1240. drivers/gpu/drm/panthor/panthor_fw.h +30 −2
  1241. drivers/gpu/drm/panthor/panthor_gem.c +7 −14
  1242. drivers/gpu/drm/panthor/panthor_gpu.c +34 −4
  1243. drivers/gpu/drm/panthor/panthor_gpu.h +1 −0
  1244. drivers/gpu/drm/panthor/panthor_heap.c +1 −0
  1245. drivers/gpu/drm/panthor/panthor_hw.c +104 −5
  1246. drivers/gpu/drm/panthor/panthor_hw.h +46 −1
  1247. drivers/gpu/drm/panthor/panthor_mmu.c +50 −96
  1248. drivers/gpu/drm/panthor/panthor_pwr.c +549 −0
  1249. drivers/gpu/drm/panthor/panthor_pwr.h +23 −0
  1250. drivers/gpu/drm/panthor/panthor_regs.h +83 −0
  1251. drivers/gpu/drm/panthor/panthor_sched.c +261 −103
  1252. drivers/gpu/drm/panthor/panthor_sched.h +2 −1
  1253. drivers/gpu/drm/pl111/pl111_display.c +9 −5
  1254. drivers/gpu/drm/qxl/qxl_cmd.c +1 −0
  1255. drivers/gpu/drm/qxl/qxl_debugfs.c +1 −0
  1256. drivers/gpu/drm/qxl/qxl_display.c +30 −0
  1257. drivers/gpu/drm/qxl/qxl_drv.c +1 −0
  1258. drivers/gpu/drm/qxl/qxl_gem.c +2 −1
  1259. drivers/gpu/drm/qxl/qxl_image.c +2 −0
  1260. drivers/gpu/drm/qxl/qxl_ioctl.c +2 −0
  1261. drivers/gpu/drm/qxl/qxl_irq.c +1 −0
  1262. drivers/gpu/drm/qxl/qxl_kms.c +1 −0
  1263. drivers/gpu/drm/qxl/qxl_release.c +2 −0
  1264. drivers/gpu/drm/qxl/qxl_ttm.c +2 −1
  1265. drivers/gpu/drm/radeon/radeon.h +1 −0
  1266. drivers/gpu/drm/radeon/radeon_acpi.c +0 −1
  1267. drivers/gpu/drm/radeon/radeon_connectors.c +5 −15
  1268. drivers/gpu/drm/radeon/radeon_device.c +2 −2
  1269. drivers/gpu/drm/radeon/radeon_display.c +0 −2
  1270. drivers/gpu/drm/radeon/radeon_drv.c +56 −32
  1271. drivers/gpu/drm/radeon/radeon_fbdev.c +1 −16
  1272. drivers/gpu/drm/radeon/radeon_gem.c +1 −1
  1273. drivers/gpu/drm/radeon/radeon_kms.c +0 −4
  1274. drivers/gpu/drm/radeon/radeon_ttm.c +4 −2
  1275. drivers/gpu/drm/renesas/rcar-du/rcar_du_crtc.c +2 −1
  1276. drivers/gpu/drm/renesas/rcar-du/rcar_du_drv.c +1 −0
  1277. drivers/gpu/drm/renesas/rcar-du/rcar_du_kms.c +5 −2
  1278. drivers/gpu/drm/renesas/rcar-du/rcar_mipi_dsi.c +34 −16
  1279. drivers/gpu/drm/renesas/rcar-du/rcar_mipi_dsi_regs.h +203 −184
  1280. drivers/gpu/drm/renesas/rz-du/Kconfig +1 −1
  1281. drivers/gpu/drm/renesas/rz-du/rzg2l_du_drv.c +1 −0
  1282. drivers/gpu/drm/rockchip/analogix_dp-rockchip.c +17 −25
  1283. drivers/gpu/drm/rockchip/cdn-dp-core.c +1 −0
  1284. drivers/gpu/drm/rockchip/cdn-dp-reg.c +2 −0
  1285. drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c +21 −0
  1286. drivers/gpu/drm/rockchip/dw_hdmi_qp-rockchip.c +119 −76
  1287. drivers/gpu/drm/rockchip/inno_hdmi.c +1 −0
  1288. drivers/gpu/drm/rockchip/rk3066_hdmi.c +1 −0
  1289. drivers/gpu/drm/rockchip/rockchip_drm_drv.c +4 −0
  1290. drivers/gpu/drm/rockchip/rockchip_drm_gem.c +7 −6
  1291. drivers/gpu/drm/rockchip/rockchip_drm_vop.c +4 −3
  1292. drivers/gpu/drm/rockchip/rockchip_drm_vop2.c +80 −62
  1293. drivers/gpu/drm/rockchip/rockchip_lvds.c +1 −0
  1294. drivers/gpu/drm/rockchip/rockchip_rgb.c +1 −0
  1295. drivers/gpu/drm/rockchip/rockchip_vop2_reg.c +42 −7
  1296. drivers/gpu/drm/rockchip/rockchip_vop_reg.c +1 −0
  1297. drivers/gpu/drm/scheduler/sched_main.c +16 −4
  1298. drivers/gpu/drm/scheduler/tests/sched_tests.h +1 −2
  1299. drivers/gpu/drm/sitronix/st7571-i2c.c +2 −1
  1300. drivers/gpu/drm/sitronix/st7586.c +1 −0
  1301. drivers/gpu/drm/sitronix/st7735r.c +1 −0
  1302. drivers/gpu/drm/solomon/ssd130x.c +36 −51
  1303. drivers/gpu/drm/sti/sti_cursor.c +1 −0
  1304. drivers/gpu/drm/sti/sti_drv.c +6 −13
  1305. drivers/gpu/drm/sti/sti_gdp.c +1 −0
  1306. drivers/gpu/drm/sti/sti_hda.c +5 −0
  1307. drivers/gpu/drm/sti/sti_hdmi.c +2 −0
  1308. drivers/gpu/drm/sti/sti_hqvdp.c +1 −0
  1309. drivers/gpu/drm/sti/sti_plane.c +1 −0
  1310. drivers/gpu/drm/stm/drv.c +1 −0
  1311. drivers/gpu/drm/stm/dw_mipi_dsi-stm.c +8 −6
  1312. drivers/gpu/drm/stm/ltdc.c +1 −0
  1313. drivers/gpu/drm/stm/lvds.c +7 −5
  1314. drivers/gpu/drm/sun4i/sun4i_backend.c +1 −0
  1315. drivers/gpu/drm/sun4i/sun4i_drv.c +1 −0
  1316. drivers/gpu/drm/sun4i/sun4i_frontend.c +1 −0
  1317. drivers/gpu/drm/sun4i/sun4i_hdmi_ddc_clk.c +7 −5
  1318. drivers/gpu/drm/sun4i/sun4i_tcon_dclk.c +10 −8
  1319. drivers/gpu/drm/sun4i/sun8i_csc.c +61 −52
  1320. drivers/gpu/drm/sun4i/sun8i_csc.h +4 −12
  1321. drivers/gpu/drm/sun4i/sun8i_mixer.c +142 −76
  1322. drivers/gpu/drm/sun4i/sun8i_mixer.h +43 −22
  1323. drivers/gpu/drm/sun4i/sun8i_ui_layer.c +76 −111
  1324. drivers/gpu/drm/sun4i/sun8i_ui_layer.h +5 −2
  1325. drivers/gpu/drm/sun4i/sun8i_ui_scaler.c +19 −25
  1326. drivers/gpu/drm/sun4i/sun8i_ui_scaler.h +2 −2
  1327. drivers/gpu/drm/sun4i/sun8i_vi_layer.c +93 −155
  1328. drivers/gpu/drm/sun4i/sun8i_vi_layer.h +5 −2
  1329. drivers/gpu/drm/sun4i/sun8i_vi_scaler.c +26 −25
  1330. drivers/gpu/drm/sun4i/sun8i_vi_scaler.h +4 −2
  1331. drivers/gpu/drm/sysfb/drm_sysfb_helper.h +32 −2
  1332. drivers/gpu/drm/sysfb/drm_sysfb_modeset.c +149 −4
  1333. drivers/gpu/drm/sysfb/efidrm.c +1 −0
  1334. drivers/gpu/drm/sysfb/ofdrm.c +1 −0
  1335. drivers/gpu/drm/sysfb/simpledrm.c +3 −1
  1336. drivers/gpu/drm/sysfb/vesadrm.c +3 −1
  1337. drivers/gpu/drm/tegra/Makefile +1 −0
  1338. drivers/gpu/drm/tegra/dc.c +2 −1
  1339. drivers/gpu/drm/tegra/drm.c +3 −0
  1340. drivers/gpu/drm/tegra/drm.h +1 −0
  1341. drivers/gpu/drm/tegra/dsi.c +29 −27
  1342. drivers/gpu/drm/tegra/fb.c +1 −0
  1343. drivers/gpu/drm/tegra/fbdev.c +1 −10
  1344. drivers/gpu/drm/tegra/gem.c +5 −3
  1345. drivers/gpu/drm/tegra/hdmi.c +3 −2
  1346. drivers/gpu/drm/tegra/hub.c +1 −0
  1347. drivers/gpu/drm/tegra/nvjpg.c +330 −0
  1348. drivers/gpu/drm/tegra/sor.c +3 −2
  1349. drivers/gpu/drm/tests/.kunitconfig +2 −0
  1350. drivers/gpu/drm/tests/Makefile +2 −1
  1351. drivers/gpu/drm/tests/drm_buddy_test.c +105 −0
  1352. drivers/gpu/drm/tests/drm_fixp_test.c +71 −0
  1353. drivers/gpu/drm/tests/drm_mm_test.c +1 −0
  1354. drivers/gpu/drm/tidss/tidss_crtc.c +24 −18
  1355. drivers/gpu/drm/tidss/tidss_dispc.c +49 −77
  1356. drivers/gpu/drm/tidss/tidss_dispc.h +1 −5
  1357. drivers/gpu/drm/tidss/tidss_drv.c +0 −16
  1358. drivers/gpu/drm/tidss/tidss_drv.h +2 −0
  1359. drivers/gpu/drm/tidss/tidss_kms.c +0 −4
  1360. drivers/gpu/drm/tidss/tidss_oldi.c +22 −0
  1361. drivers/gpu/drm/tidss/tidss_plane.c +0 −8
  1362. drivers/gpu/drm/tilcdc/tilcdc_crtc.c +1 −8
  1363. drivers/gpu/drm/tilcdc/tilcdc_plane.c +1 −2
  1364. drivers/gpu/drm/tiny/bochs.c +11 −0
  1365. drivers/gpu/drm/tiny/cirrus-qemu.c +12 −0
  1366. drivers/gpu/drm/tiny/gm12u320.c +1 −0
  1367. drivers/gpu/drm/tiny/hx8357d.c +1 −0
  1368. drivers/gpu/drm/tiny/ili9163.c +1 −0
  1369. drivers/gpu/drm/tiny/ili9225.c +1 −0
  1370. drivers/gpu/drm/tiny/ili9341.c +1 −0
  1371. drivers/gpu/drm/tiny/ili9486.c +1 −0
  1372. drivers/gpu/drm/tiny/mi0283qt.c +1 −0
  1373. drivers/gpu/drm/tiny/panel-mipi-dbi.c +1 −0
  1374. drivers/gpu/drm/tiny/pixpaper.c +1 −0
  1375. drivers/gpu/drm/tiny/repaper.c +1 −0
  1376. drivers/gpu/drm/ttm/tests/ttm_bo_test.c +14 −14
  1377. drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c +37 −36
  1378. drivers/gpu/drm/ttm/tests/ttm_device_test.c +13 −20
  1379. drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.c +9 −13
  1380. drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.h +2 −5
  1381. drivers/gpu/drm/ttm/tests/ttm_mock_manager.c +1 −0
  1382. drivers/gpu/drm/ttm/tests/ttm_pool_test.c +12 −12
  1383. drivers/gpu/drm/ttm/tests/ttm_resource_test.c +3 −2
  1384. drivers/gpu/drm/ttm/ttm_bo.c +35 −32
  1385. drivers/gpu/drm/ttm/ttm_bo_internal.h +2 −0
  1386. drivers/gpu/drm/ttm/ttm_bo_util.c +31 −7
  1387. drivers/gpu/drm/ttm/ttm_device.c +5 −4
  1388. drivers/gpu/drm/ttm/ttm_module.c +2 −1
  1389. drivers/gpu/drm/ttm/ttm_pool.c +26 −19
  1390. drivers/gpu/drm/ttm/ttm_pool_internal.h +25 −0
  1391. drivers/gpu/drm/ttm/ttm_resource.c +24 −13
  1392. drivers/gpu/drm/ttm/ttm_tt.c +7 −4
  1393. drivers/gpu/drm/tve200/tve200_display.c +1 −0
  1394. drivers/gpu/drm/udl/udl_edid.c +1 −0
  1395. drivers/gpu/drm/v3d/v3d_bo.c +2 −0
  1396. drivers/gpu/drm/v3d/v3d_debugfs.c +1 −0
  1397. drivers/gpu/drm/v3d/v3d_drv.c +1 −0
  1398. drivers/gpu/drm/v3d/v3d_gem.c +1 −0
  1399. drivers/gpu/drm/v3d/v3d_gemfs.c +2 −0
  1400. drivers/gpu/drm/v3d/v3d_irq.c +2 −0
  1401. drivers/gpu/drm/v3d/v3d_sched.c +1 −0
  1402. drivers/gpu/drm/v3d/v3d_submit.c +1 −0
  1403. drivers/gpu/drm/vboxvideo/vbox_irq.c +1 −0
  1404. drivers/gpu/drm/vboxvideo/vbox_main.c +1 −0
  1405. drivers/gpu/drm/vboxvideo/vbox_mode.c +5 −4
  1406. drivers/gpu/drm/vboxvideo/vbox_ttm.c +1 −0
  1407. drivers/gpu/drm/vc4/Kconfig +1 −0
  1408. drivers/gpu/drm/vc4/vc4_bo.c +1 −0
  1409. drivers/gpu/drm/vc4/vc4_debugfs.c +1 −0
  1410. drivers/gpu/drm/vc4/vc4_dpi.c +1 −0
  1411. drivers/gpu/drm/vc4/vc4_drv.c +1 −0
  1412. drivers/gpu/drm/vc4/vc4_dsi.c +1 −0
  1413. drivers/gpu/drm/vc4/vc4_gem.c +1 −0
  1414. drivers/gpu/drm/vc4/vc4_hdmi.c +55 −83
  1415. drivers/gpu/drm/vc4/vc4_hdmi.h +0 −1
  1416. drivers/gpu/drm/vc4/vc4_hvs.c +1 −0
  1417. drivers/gpu/drm/vc4/vc4_irq.c +1 −0
  1418. drivers/gpu/drm/vc4/vc4_kms.c +1 −0
  1419. drivers/gpu/drm/vc4/vc4_perfmon.c +2 −0
  1420. drivers/gpu/drm/vc4/vc4_plane.c +3 −4
  1421. drivers/gpu/drm/vc4/vc4_render_cl.c +2 −0
  1422. drivers/gpu/drm/vc4/vc4_txp.c +1 −0
  1423. drivers/gpu/drm/vc4/vc4_v3d.c +2 −0
  1424. drivers/gpu/drm/vc4/vc4_validate.c +2 −0
  1425. drivers/gpu/drm/vc4/vc4_validate_shaders.c +2 −0
  1426. drivers/gpu/drm/vc4/vc4_vec.c +1 −0
  1427. drivers/gpu/drm/vgem/vgem_fence.c +1 −1
  1428. drivers/gpu/drm/virtio/virtgpu_debugfs.c +1 −0
  1429. drivers/gpu/drm/virtio/virtgpu_display.c +33 −4
  1430. drivers/gpu/drm/virtio/virtgpu_drv.c +1 −0
  1431. drivers/gpu/drm/virtio/virtgpu_kms.c +1 −0
  1432. drivers/gpu/drm/virtio/virtgpu_object.c +2 −0
  1433. drivers/gpu/drm/virtio/virtgpu_plane.c +1 −0
  1434. drivers/gpu/drm/virtio/virtgpu_vq.c +1 −0
  1435. drivers/gpu/drm/vkms/Kconfig +1 −0
  1436. drivers/gpu/drm/vkms/Makefile +4 −1
  1437. drivers/gpu/drm/vkms/tests/Makefile +2 −1
  1438. drivers/gpu/drm/vkms/tests/vkms_color_test.c +414 −0
  1439. drivers/gpu/drm/vkms/tests/vkms_config_test.c +54 −17
  1440. drivers/gpu/drm/vkms/vkms_colorop.c +120 −0
  1441. drivers/gpu/drm/vkms/vkms_composer.c +119 −17
  1442. drivers/gpu/drm/vkms/vkms_composer.h +28 −0
  1443. drivers/gpu/drm/vkms/vkms_config.c +12 −3
  1444. drivers/gpu/drm/vkms/vkms_config.h +53 −1
  1445. drivers/gpu/drm/vkms/vkms_configfs.c +843 −0
  1446. drivers/gpu/drm/vkms/vkms_configfs.h +8 −0
  1447. drivers/gpu/drm/vkms/vkms_connector.c +35 −0
  1448. drivers/gpu/drm/vkms/vkms_connector.h +9 −0
  1449. drivers/gpu/drm/vkms/vkms_crtc.c +10 −78
  1450. drivers/gpu/drm/vkms/vkms_drv.c +24 −3
  1451. drivers/gpu/drm/vkms/vkms_drv.h +30 −4
  1452. drivers/gpu/drm/vkms/vkms_luts.c +811 −0
  1453. drivers/gpu/drm/vkms/vkms_luts.h +12 −0
  1454. drivers/gpu/drm/vkms/vkms_output.c +2 −5
  1455. drivers/gpu/drm/vkms/vkms_plane.c +8 −2
  1456. drivers/gpu/drm/vkms/vkms_writeback.c +1 −0
  1457. drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +2 −2
  1458. drivers/gpu/drm/vmwgfx/vmwgfx_drv.h +1 −0
  1459. drivers/gpu/drm/vmwgfx/vmwgfx_gem.c +1 −1
  1460. drivers/gpu/drm/vmwgfx/vmwgfx_kms.c +3 −0
  1461. drivers/gpu/drm/vmwgfx/vmwgfx_surface.c +4 −17
  1462. drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c +2 −4
  1463. drivers/gpu/drm/xe/Kconfig.debug +8 −8
  1464. drivers/gpu/drm/xe/Makefile +17 −3
  1465. drivers/gpu/drm/xe/abi/guc_actions_abi.h +0 −8
  1466. drivers/gpu/drm/xe/compat-i915-headers/gem/i915_gem_object.h +1 −3
  1467. drivers/gpu/drm/xe/compat-i915-headers/gem/i915_gem_stolen.h +30 −75
  1468. drivers/gpu/drm/xe/compat-i915-headers/i915_drv.h +0 −4
  1469. drivers/gpu/drm/xe/compat-i915-headers/i915_scheduler_types.h +0 −13
  1470. drivers/gpu/drm/xe/compat-i915-headers/i915_utils.h +8 −1
  1471. drivers/gpu/drm/xe/compat-i915-headers/i915_vma.h +0 −2
  1472. drivers/gpu/drm/xe/compat-i915-headers/intel_uncore.h +10 −21
  1473. drivers/gpu/drm/xe/display/ext/i915_utils.c +0 −27
  1474. drivers/gpu/drm/xe/display/intel_bo.c +51 −9
  1475. drivers/gpu/drm/xe/display/intel_fb_bo.c +1 −2
  1476. drivers/gpu/drm/xe/display/intel_fbdev_fb.c +24 −46
  1477. drivers/gpu/drm/xe/display/xe_display.c +16 −7
  1478. drivers/gpu/drm/xe/display/xe_display.h +2 −2
  1479. drivers/gpu/drm/xe/display/xe_display_rpm.c +31 −30
  1480. drivers/gpu/drm/xe/display/xe_display_rpm.h +11 −0
  1481. drivers/gpu/drm/xe/display/xe_display_wa.c +2 −1
  1482. drivers/gpu/drm/xe/display/xe_panic.c +36 −14
  1483. drivers/gpu/drm/xe/display/xe_plane_initial.c +2 −2
  1484. drivers/gpu/drm/xe/display/xe_stolen.c +123 −0
  1485. drivers/gpu/drm/xe/instructions/xe_gpu_commands.h +6 −0
  1486. drivers/gpu/drm/xe/regs/xe_engine_regs.h +4 −0
  1487. drivers/gpu/drm/xe/regs/xe_gt_regs.h +25 −5
  1488. drivers/gpu/drm/xe/regs/xe_i2c_regs.h +3 −0
  1489. drivers/gpu/drm/xe/regs/xe_irq_regs.h +6 −2
  1490. drivers/gpu/drm/xe/regs/xe_pmt.h +1 −0
  1491. drivers/gpu/drm/xe/regs/xe_regs.h +2 −0
  1492. drivers/gpu/drm/xe/tests/xe_dma_buf.c +15 −2
  1493. drivers/gpu/drm/xe/tests/xe_gt_sriov_pf_config_kunit.c +208 −0
  1494. drivers/gpu/drm/xe/tests/xe_pci.c +4 −2
  1495. drivers/gpu/drm/xe/tests/xe_pci_test.c +11 −5
  1496. drivers/gpu/drm/xe/tests/xe_rtp_test.c +4 −2
  1497. drivers/gpu/drm/xe/xe_bo.c +83 −34
  1498. drivers/gpu/drm/xe/xe_bo.h +3 −1
  1499. drivers/gpu/drm/xe/xe_bo_doc.h +4 −4
  1500. drivers/gpu/drm/xe/xe_bo_evict.c +9 −0
  1501. drivers/gpu/drm/xe/xe_configfs.c +282 −1
  1502. drivers/gpu/drm/xe/xe_configfs.h +8 −0
  1503. drivers/gpu/drm/xe/xe_debugfs.c +7 −9
  1504. drivers/gpu/drm/xe/xe_device.c +48 −33
  1505. drivers/gpu/drm/xe/xe_device_sysfs.c +1 −9
  1506. drivers/gpu/drm/xe/xe_device_types.h +36 −19
  1507. drivers/gpu/drm/xe/xe_device_wa_oob.rules +3 −0
  1508. drivers/gpu/drm/xe/xe_dma_buf.c +26 −15
  1509. drivers/gpu/drm/xe/xe_eu_stall.c +30 −6
  1510. drivers/gpu/drm/xe/xe_exec.c +12 −16
  1511. drivers/gpu/drm/xe/xe_exec_queue.c +160 −69
  1512. drivers/gpu/drm/xe/xe_exec_queue.h +22 −6
  1513. drivers/gpu/drm/xe/xe_exec_queue_types.h +13 −0
  1514. drivers/gpu/drm/xe/xe_execlist.c +1 −1
  1515. drivers/gpu/drm/xe/xe_force_wake_types.h +24 −2
  1516. drivers/gpu/drm/xe/xe_ggtt.c +149 −25
  1517. drivers/gpu/drm/xe/xe_ggtt.h +3 −0
  1518. drivers/gpu/drm/xe/xe_ggtt_types.h +2 −0
  1519. drivers/gpu/drm/xe/xe_gpu_scheduler.c +14 −13
  1520. drivers/gpu/drm/xe/xe_gpu_scheduler.h +22 −7
  1521. drivers/gpu/drm/xe/xe_gsc.c +2 −2
  1522. drivers/gpu/drm/xe/xe_gt.c +46 −50
  1523. drivers/gpu/drm/xe/xe_gt.h +19 −0
  1524. drivers/gpu/drm/xe/xe_gt_clock.c +0 −19
  1525. drivers/gpu/drm/xe/xe_gt_debugfs.c +45 −114
  1526. drivers/gpu/drm/xe/xe_gt_debugfs.h +1 −0
  1527. drivers/gpu/drm/xe/xe_gt_freq.c +17 −20
  1528. drivers/gpu/drm/xe/xe_gt_mcr.c +74 −6
  1529. drivers/gpu/drm/xe/xe_gt_pagefault.c +0 −679
  1530. drivers/gpu/drm/xe/xe_gt_pagefault.h +0 −19
  1531. drivers/gpu/drm/xe/xe_gt_sriov_pf.c +8 −28
  1532. drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c +308 −41
  1533. drivers/gpu/drm/xe/xe_gt_sriov_pf_config.h +16 −0
  1534. drivers/gpu/drm/xe/xe_gt_sriov_pf_control.c +705 −45
  1535. drivers/gpu/drm/xe/xe_gt_sriov_pf_control.h +12 −0
  1536. drivers/gpu/drm/xe/xe_gt_sriov_pf_control_types.h +32 −4
  1537. drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c +245 −216
  1538. drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.h +1 −0
  1539. drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.c +829 −188
  1540. drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.h +39 −9
  1541. drivers/gpu/drm/xe/xe_gt_sriov_pf_migration_types.h +11 −23
  1542. drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c +20 −1
  1543. drivers/gpu/drm/xe/xe_gt_sriov_pf_types.h +2 −3
  1544. drivers/gpu/drm/xe/xe_gt_sriov_printk.h +5 −2
  1545. drivers/gpu/drm/xe/xe_gt_sriov_vf.c +361 −104
  1546. drivers/gpu/drm/xe/xe_gt_sriov_vf.h +6 −5
  1547. drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +26 −8
  1548. drivers/gpu/drm/xe/xe_gt_throttle.c +187 −168
  1549. drivers/gpu/drm/xe/xe_gt_topology.c +21 −3
  1550. drivers/gpu/drm/xe/xe_gt_topology.h +3 −1
  1551. drivers/gpu/drm/xe/xe_gt_types.h +11 −68
  1552. drivers/gpu/drm/xe/xe_guard.h +119 −0
  1553. drivers/gpu/drm/xe/xe_guc.c +176 −152
  1554. drivers/gpu/drm/xe/xe_guc.h +1 −0
  1555. drivers/gpu/drm/xe/xe_guc_ads.c +6 −2
  1556. drivers/gpu/drm/xe/xe_guc_ads_types.h +1 −1
  1557. drivers/gpu/drm/xe/xe_guc_buf.c +46 −11
  1558. drivers/gpu/drm/xe/xe_guc_buf.h +2 −0
  1559. drivers/gpu/drm/xe/xe_guc_capture.c +28 −1
  1560. drivers/gpu/drm/xe/xe_guc_ct.c +136 −241
  1561. drivers/gpu/drm/xe/xe_guc_ct.h +11 −2
  1562. drivers/gpu/drm/xe/xe_guc_ct_types.h +1 −1
  1563. drivers/gpu/drm/xe/xe_guc_exec_queue_types.h +15 −0
  1564. drivers/gpu/drm/xe/xe_guc_fwif.h +1 −0
  1565. drivers/gpu/drm/xe/xe_guc_log_types.h +1 −1
  1566. drivers/gpu/drm/xe/xe_guc_pagefault.c +95 −0
  1567. drivers/gpu/drm/xe/xe_guc_pagefault.h +15 −0
  1568. drivers/gpu/drm/xe/xe_guc_pc.c +65 −82
  1569. drivers/gpu/drm/xe/xe_guc_pc_types.h +0 −4
  1570. drivers/gpu/drm/xe/xe_guc_relay.c +15 −2
  1571. drivers/gpu/drm/xe/xe_guc_relay_types.h +4 −0
  1572. drivers/gpu/drm/xe/xe_guc_submit.c +444 −162
  1573. drivers/gpu/drm/xe/xe_guc_submit.h +2 −5
  1574. drivers/gpu/drm/xe/xe_guc_tlb_inval.c +1 −1
  1575. drivers/gpu/drm/xe/xe_heci_gsc.c +2 −0
  1576. drivers/gpu/drm/xe/xe_huc.c +7 −3
  1577. drivers/gpu/drm/xe/xe_hw_engine.c +48 −14
  1578. drivers/gpu/drm/xe/xe_hwmon.c +0 −8
  1579. drivers/gpu/drm/xe/xe_i2c.c +27 −1
  1580. drivers/gpu/drm/xe/xe_i2c.h +4 −0
  1581. drivers/gpu/drm/xe/xe_irq.c +94 −44
  1582. drivers/gpu/drm/xe/xe_lmtt.c +5 −6
  1583. drivers/gpu/drm/xe/xe_lrc.c +13 −7
  1584. drivers/gpu/drm/xe/xe_lrc.h +14 −2
  1585. drivers/gpu/drm/xe/xe_map.h +2 −20
  1586. drivers/gpu/drm/xe/xe_memirq.c +48 −9
  1587. drivers/gpu/drm/xe/xe_memirq.h +2 −0
  1588. drivers/gpu/drm/xe/xe_migrate.c +323 −46
  1589. drivers/gpu/drm/xe/xe_migrate.h +16 −0
  1590. drivers/gpu/drm/xe/xe_migrate_doc.h +1 −1
  1591. drivers/gpu/drm/xe/xe_mmio.c +29 −0
  1592. drivers/gpu/drm/xe/xe_mmio.h +4 −0
  1593. drivers/gpu/drm/xe/xe_mocs.c +40 −2
  1594. drivers/gpu/drm/xe/xe_mocs.h +1 −7
  1595. drivers/gpu/drm/xe/xe_oa.c +11 −9
  1596. drivers/gpu/drm/xe/xe_oa_types.h +3 −0
  1597. drivers/gpu/drm/xe/xe_pagefault.c +445 −0
  1598. drivers/gpu/drm/xe/xe_pagefault.h +19 −0
  1599. drivers/gpu/drm/xe/xe_pagefault_types.h +136 −0
  1600. drivers/gpu/drm/xe/xe_pat.c +126 −19
  1601. drivers/gpu/drm/xe/xe_pat.h +6 −6
  1602. drivers/gpu/drm/xe/xe_pci.c +206 −78
  1603. drivers/gpu/drm/xe/xe_pci_sriov.c +56 −59
  1604. drivers/gpu/drm/xe/xe_pci_sriov.h +1 −0
  1605. drivers/gpu/drm/xe/xe_pci_types.h +6 −5
  1606. drivers/gpu/drm/xe/xe_pcode.c +26 −14
  1607. drivers/gpu/drm/xe/xe_pcode_api.h +0 −6
  1608. drivers/gpu/drm/xe/xe_platform_types.h +3 −0
  1609. drivers/gpu/drm/xe/xe_pm.c +66 −0
  1610. drivers/gpu/drm/xe/xe_pm.h +2 −0
  1611. drivers/gpu/drm/xe/xe_pmu.c +10 −1
  1612. drivers/gpu/drm/xe/xe_preempt_fence.c +11 −0
  1613. drivers/gpu/drm/xe/xe_preempt_fence_types.h +1 −1
  1614. drivers/gpu/drm/xe/xe_psmi.c +2 −2
  1615. drivers/gpu/drm/xe/xe_pt.c +42 −68
  1616. drivers/gpu/drm/xe/xe_query.c +1 −1
  1617. drivers/gpu/drm/xe/xe_range_fence.h +2 −2
  1618. drivers/gpu/drm/xe/xe_reg_whitelist.c +9 −1
  1619. drivers/gpu/drm/xe/xe_ring_ops.c +19 −4
  1620. drivers/gpu/drm/xe/xe_rtp.c +20 −11
  1621. drivers/gpu/drm/xe/xe_rtp.h +25 −5
  1622. drivers/gpu/drm/xe/xe_rtp_types.h +3 −1
  1623. drivers/gpu/drm/xe/xe_sa.c +21 −0
  1624. drivers/gpu/drm/xe/xe_sa.h +1 −0
  1625. drivers/gpu/drm/xe/xe_sched_job.c +5 −20
  1626. drivers/gpu/drm/xe/xe_sched_job.h +6 −7
  1627. drivers/gpu/drm/xe/xe_sched_job_types.h +10 −1
  1628. drivers/gpu/drm/xe/xe_sriov.c +2 −0
  1629. drivers/gpu/drm/xe/xe_sriov_packet.c +520 −0
  1630. drivers/gpu/drm/xe/xe_sriov_packet.h +30 −0
  1631. drivers/gpu/drm/xe/xe_sriov_packet_types.h +75 −0
  1632. drivers/gpu/drm/xe/xe_sriov_pf.c +134 −41
  1633. drivers/gpu/drm/xe/xe_sriov_pf.h +8 −14
  1634. drivers/gpu/drm/xe/xe_sriov_pf_control.c +279 −0
  1635. drivers/gpu/drm/xe/xe_sriov_pf_control.h +22 −0
  1636. drivers/gpu/drm/xe/xe_sriov_pf_debugfs.c +395 −0
  1637. drivers/gpu/drm/xe/xe_sriov_pf_debugfs.h +18 −0
  1638. drivers/gpu/drm/xe/xe_sriov_pf_helpers.h +27 −0
  1639. drivers/gpu/drm/xe/xe_sriov_pf_migration.c +340 −0
  1640. drivers/gpu/drm/xe/xe_sriov_pf_migration.h +29 −0
  1641. drivers/gpu/drm/xe/xe_sriov_pf_migration_types.h +37 −0
  1642. drivers/gpu/drm/xe/xe_sriov_pf_provision.c +438 −0
  1643. drivers/gpu/drm/xe/xe_sriov_pf_provision.h +45 −0
  1644. drivers/gpu/drm/xe/xe_sriov_pf_provision_types.h +36 −0
  1645. drivers/gpu/drm/xe/xe_sriov_pf_sysfs.c +647 −0
  1646. drivers/gpu/drm/xe/xe_sriov_pf_sysfs.h +16 −0
  1647. drivers/gpu/drm/xe/xe_sriov_pf_types.h +25 −0
  1648. drivers/gpu/drm/xe/xe_sriov_printk.h +6 −6
  1649. drivers/gpu/drm/xe/xe_sriov_vf.c +11 −275
  1650. drivers/gpu/drm/xe/xe_sriov_vf.h +1 −1
  1651. drivers/gpu/drm/xe/xe_sriov_vf_ccs.c +72 −2
  1652. drivers/gpu/drm/xe/xe_sriov_vf_ccs.h +1 −0
  1653. drivers/gpu/drm/xe/xe_sriov_vf_types.h +3 −7
  1654. drivers/gpu/drm/xe/xe_svm.c +2 −3
  1655. drivers/gpu/drm/xe/xe_sync.c +41 −33
  1656. drivers/gpu/drm/xe/xe_tile.c +9 −5
  1657. drivers/gpu/drm/xe/xe_tile_debugfs.c +13 −6
  1658. drivers/gpu/drm/xe/xe_tile_debugfs.h +3 −0
  1659. drivers/gpu/drm/xe/xe_tile_sriov_pf_debugfs.c +253 −0
  1660. drivers/gpu/drm/xe/xe_tile_sriov_pf_debugfs.h +15 −0
  1661. drivers/gpu/drm/xe/xe_tile_sriov_printk.h +33 −0
  1662. drivers/gpu/drm/xe/xe_tile_sriov_vf.c +104 −8
  1663. drivers/gpu/drm/xe/xe_tile_sriov_vf.h +7 −2
  1664. drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h +23 −0
  1665. drivers/gpu/drm/xe/xe_tlb_inval.h +1 −1
  1666. drivers/gpu/drm/xe/xe_tlb_inval_job.c +24 −7
  1667. drivers/gpu/drm/xe/xe_tlb_inval_job.h +3 −2
  1668. drivers/gpu/drm/xe/xe_trace.h +23 −0
  1669. drivers/gpu/drm/xe/xe_ttm_stolen_mgr.c +5 −3
  1670. drivers/gpu/drm/xe/xe_ttm_sys_mgr.c +3 −3
  1671. drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +3 −3
  1672. drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h +2 −2
  1673. drivers/gpu/drm/xe/xe_tuning.c +21 −6
  1674. drivers/gpu/drm/xe/xe_tuning.h +1 −1
  1675. drivers/gpu/drm/xe/xe_uc_fw_types.h +3 −3
  1676. drivers/gpu/drm/xe/xe_uc_types.h +1 −1
  1677. drivers/gpu/drm/xe/xe_userptr.c +3 −1
  1678. drivers/gpu/drm/xe/xe_validation.h +3 −3
  1679. drivers/gpu/drm/xe/xe_vm.c +90 −48
  1680. drivers/gpu/drm/xe/xe_vm_doc.h +4 −4
  1681. drivers/gpu/drm/xe/xe_vm_types.h +3 −7
  1682. drivers/gpu/drm/xe/xe_vram.c +23 −15
  1683. drivers/gpu/drm/xe/xe_wa.c +19 −8
  1684. drivers/gpu/drm/xe/xe_wa.h +1 −1
  1685. drivers/gpu/drm/xe/xe_wa_oob.rules +6 −11
  1686. drivers/gpu/drm/xen/xen_drm_front.c +1 −0
  1687. drivers/gpu/drm/xen/xen_drm_front_gem.c +1 −0
  1688. drivers/gpu/drm/xen/xen_drm_front_kms.c +1 −0
  1689. drivers/gpu/drm/xlnx/zynqmp_kms.c +5 −2
  1690. drivers/gpu/host1x/bus.c +12 −0
  1691. drivers/gpu/host1x/dev.c +12 −8
  1692. drivers/gpu/host1x/dev.h +3 −0
  1693. drivers/gpu/host1x/hw/channel_hw.c +66 −40
  1694. drivers/gpu/host1x/hw/intr_hw.c +47 −9
  1695. drivers/gpu/host1x/syncpt.c +1 −3
  1696. drivers/gpu/nova-core/bitfield.rs +330 −0
  1697. drivers/gpu/nova-core/dma.rs +16 −20
  1698. drivers/gpu/nova-core/driver.rs +24 −3
  1699. drivers/gpu/nova-core/falcon.rs +166 −115
  1700. drivers/gpu/nova-core/falcon/gsp.rs +27 −2
  1701. drivers/gpu/nova-core/falcon/hal.rs +10 −4
  1702. drivers/gpu/nova-core/falcon/hal/ga102.rs +24 −20
  1703. drivers/gpu/nova-core/falcon/sec2.rs +8 −2
  1704. drivers/gpu/nova-core/fb.rs +86 −16
  1705. drivers/gpu/nova-core/fb/hal.rs +4 −2
  1706. drivers/gpu/nova-core/fb/hal/ga100.rs +11 −5
  1707. drivers/gpu/nova-core/fb/hal/ga102.rs +5 −3
  1708. drivers/gpu/nova-core/fb/hal/tu102.rs +13 −12
  1709. drivers/gpu/nova-core/firmware.rs +20 −13
  1710. drivers/gpu/nova-core/firmware/booter.rs +54 −28
  1711. drivers/gpu/nova-core/firmware/fwsec.rs +105 −83
  1712. drivers/gpu/nova-core/firmware/gsp.rs +34 −19
  1713. drivers/gpu/nova-core/firmware/riscv.rs +19 −15
  1714. drivers/gpu/nova-core/gfw.rs +24 −24
  1715. drivers/gpu/nova-core/gpu.rs +88 −27
  1716. drivers/gpu/nova-core/gsp.rs +145 −6
  1717. drivers/gpu/nova-core/gsp/boot.rs +134 −19
  1718. drivers/gpu/nova-core/gsp/cmdq.rs +679 −0
  1719. drivers/gpu/nova-core/gsp/commands.rs +227 −0
  1720. drivers/gpu/nova-core/gsp/fw.rs +922 −1
  1721. drivers/gpu/nova-core/gsp/fw/commands.rs +128 −0
  1722. drivers/gpu/nova-core/gsp/fw/r570_144.rs +4 −2
  1723. drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +950 −0
  1724. drivers/gpu/nova-core/gsp/sequencer.rs +407 −0
  1725. drivers/gpu/nova-core/nova_core.rs +5 −0
  1726. drivers/gpu/nova-core/num.rs +217 −0
  1727. drivers/gpu/nova-core/regs.rs +78 −13
  1728. drivers/gpu/nova-core/regs/macros.rs +24 −263
  1729. drivers/gpu/nova-core/sbuffer.rs +227 −0
  1730. drivers/gpu/nova-core/util.rs +11 −22
  1731. drivers/gpu/nova-core/vbios.rs +181 −242
  1732. drivers/video/fbdev/Kconfig +4 −4
  1733. drivers/video/fbdev/core/Kconfig +1 −1
  1734. drivers/video/fbdev/core/bitblit.c +61 −61
  1735. drivers/video/fbdev/core/fbcon.c +236 −223
  1736. drivers/video/fbdev/core/fbcon.h +8 −9
  1737. drivers/video/fbdev/core/fbcon_ccw.c +78 −73
  1738. drivers/video/fbdev/core/fbcon_cw.c +78 −73
  1739. drivers/video/fbdev/core/fbcon_rotate.c +15 −32
  1740. drivers/video/fbdev/core/fbcon_rotate.h +15 −3
  1741. drivers/video/fbdev/core/fbcon_ud.c +86 −81
  1742. drivers/video/fbdev/core/softcursor.c +9 −9
  1743. drivers/video/fbdev/core/tileblit.c +18 −14
  1744. drivers/video/fbdev/simplefb.c +3 −3
  1745. include/drm/bridge/dw_hdmi.h +10 −1
  1746. include/drm/bridge/dw_hdmi_qp.h +6 −0
  1747. include/drm/display/drm_dp.h +6 −0
  1748. include/drm/display/drm_dp_helper.h +22 −0
  1749. include/drm/drm_atomic.h +193 −62
  1750. include/drm/drm_atomic_uapi.h +3 −0
  1751. include/drm/drm_bridge.h +57 −4
  1752. include/drm/drm_buddy.h +9 −4
  1753. include/drm/drm_client.h +24 −29
  1754. include/drm/drm_client_event.h +6 −6
  1755. include/drm/drm_color_mgmt.h +29 −0
  1756. include/drm/drm_colorop.h +464 −0
  1757. include/drm/drm_crtc.h +19 −1
  1758. include/drm/drm_device.h +8 −0
  1759. include/drm/drm_dumb_buffers.h +14 −0
  1760. include/drm/drm_edid.h +6 −0
  1761. include/drm/drm_fb_helper.h +2 −18
  1762. include/drm/drm_file.h +7 −0
  1763. include/drm/drm_fixed.h +17 −0
  1764. include/drm/drm_format_helper.h +0 −4
  1765. include/drm/drm_gem_shmem_helper.h +2 −0
  1766. include/drm/drm_gpusvm.h +4 −0
  1767. include/drm/drm_gpuvm.h +16 −0
  1768. include/drm/drm_mm.h +1 −1
  1769. include/drm/drm_mode_config.h +18 −0
  1770. include/drm/drm_modeset_helper_vtables.h +12 −0
  1771. include/drm/drm_plane.h +19 −0
  1772. include/drm/drm_vblank.h +32 −0
  1773. include/drm/drm_vblank_helper.h +56 −0
  1774. include/drm/gpu_scheduler.h +1 −1
  1775. include/drm/intel/display_member.h +42 −0
  1776. include/drm/intel/display_parent_interface.h +45 −0
  1777. include/drm/intel/pciids.h +18 −2
  1778. include/drm/ttm/ttm_allocation.h +12 −0
  1779. include/drm/ttm/ttm_bo.h +1 −1
  1780. include/drm/ttm/ttm_device.h +7 −1
  1781. include/drm/ttm/ttm_pool.h +3 −5
  1782. include/drm/ttm/ttm_resource.h +23 −9
  1783. include/linux/dma-buf/heaps/cma.h +16 −0
  1784. include/sound/asoundef.h +9 −0
  1785. include/uapi/drm/amdgpu_drm.h +0 −9
  1786. include/uapi/drm/amdxdna_accel.h +61 −0
  1787. include/uapi/drm/drm.h +15 −0
  1788. include/uapi/drm/drm_mode.h +183 −1
  1789. include/uapi/drm/ethosu_accel.h +261 −0
  1790. include/uapi/drm/ivpu_accel.h +63 −0
  1791. include/uapi/drm/panfrost_drm.h +123 −27
  1792. include/uapi/drm/xe_drm.h +5 −1
  1793. init/Kconfig +3 −0
  1794. kernel/dma/contiguous.c +11 −0
  1795. rust/kernel/drm/gem/mod.rs +24 −29
  1796. rust/kernel/lib.rs +4 −0
  1797. rust/kernel/prelude.rs +3 −0
  1798. rust/kernel/slice.rs +49 −0
  1799. rust/kernel/transmute.rs +63 −0
.clang-format+1−1
@@ -167,7 +167,7 @@ ForEachMacros: · 166 unmodified lines
167 - 'drm_connector_for_each_possible_encoder'167 - 'drm_connector_for_each_possible_encoder'
168 - 'drm_exec_for_each_locked_object'168 - 'drm_exec_for_each_locked_object'
169 - 'drm_exec_for_each_locked_object_reverse'169 - 'drm_exec_for_each_locked_object_reverse'
170 - 'drm_for_each_bridge_in_chain'170 - 'drm_for_each_bridge_in_chain_scoped'
171 - 'drm_for_each_connector_iter'171 - 'drm_for_each_connector_iter'
172 - 'drm_for_each_crtc'172 - 'drm_for_each_crtc'
173 - 'drm_for_each_crtc_reverse'173 - 'drm_for_each_crtc_reverse'
.mailmap+1−0
@@ -174,6 +174,7 @@ Carlos Bilbao <carlos.bilbao@kernel.org> <bilbao@vt.edu> · 173 unmodified lines
174Changbin Du <changbin.du@intel.com> <changbin.du@gmail.com>174Changbin Du <changbin.du@intel.com> <changbin.du@gmail.com>
175Chao Yu <chao@kernel.org> <chao2.yu@samsung.com>175Chao Yu <chao@kernel.org> <chao2.yu@samsung.com>
176Chao Yu <chao@kernel.org> <yuchao0@huawei.com>176Chao Yu <chao@kernel.org> <yuchao0@huawei.com>
177Chen-Yu Tsai <wens@kernel.org> <wens@csie.org>
177Chester Lin <chester62515@gmail.com> <clin@suse.com>178Chester Lin <chester62515@gmail.com> <clin@suse.com>
178Chris Chiu <chris.chiu@canonical.com> <chiu@endlessm.com>179Chris Chiu <chris.chiu@canonical.com> <chiu@endlessm.com>
179Chris Chiu <chris.chiu@canonical.com> <chiu@endlessos.org>180Chris Chiu <chris.chiu@canonical.com> <chiu@endlessos.org>
Documentation/ABI/stable/sysfs-driver-qaic+19−0
@@ -0,0 +1,19 @@
1What: /sys/bus/pci/drivers/qaic/XXXX:XX:XX.X/accel/accel<minor_nr>/dbc<N>_state
2Date: October 2025
3KernelVersion: 6.19
4Contact: Jeff Hugo <jeff.hugo@oss.qualcomm.com>
5Description: Represents the current state of DMA Bridge channel (DBC). Below are the possible
6 states:
7
8 =================== ==========================================================
9 IDLE (0) DBC is free and can be activated
10 ASSIGNED (1) DBC is activated and a workload is running on device
11 BEFORE_SHUTDOWN (2) Sub-system associated with this workload has crashed and
12 it will shutdown soon
13 AFTER_SHUTDOWN (3) Sub-system associated with this workload has crashed and
14 it has shutdown
15 BEFORE_POWER_UP (4) Sub-system associated with this workload is shutdown and
16 it will be powered up soon
17 AFTER_POWER_UP (5) Sub-system associated with this workload is now powered up
18 =================== ==========================================================
19Users: Any userspace application or clients interested in DBC state.
Documentation/ABI/testing/sysfs-driver-intel-xe-sriov+159−0
@@ -0,0 +1,159 @@
1What: /sys/bus/pci/drivers/xe/.../sriov_admin/
2Date: October 2025
3KernelVersion: 6.19
4Contact: intel-xe@lists.freedesktop.org
5Description:
6 This directory appears for the particular Intel Xe device when:
7
8 - device supports SR-IOV, and
9 - device is a Physical Function (PF), and
10 - driver support for the SR-IOV PF is enabled on given device.
11
12 This directory is used as a root for all attributes required to
13 manage both Physical Function (PF) and Virtual Functions (VFs).
14
15
16What: /sys/bus/pci/drivers/xe/.../sriov_admin/pf/
17Date: October 2025
18KernelVersion: 6.19
19Contact: intel-xe@lists.freedesktop.org
20Description:
21 This directory holds attributes related to the SR-IOV Physical
22 Function (PF).
23
24
25What: /sys/bus/pci/drivers/xe/.../sriov_admin/vf1/
26What: /sys/bus/pci/drivers/xe/.../sriov_admin/vf2/
27What: /sys/bus/pci/drivers/xe/.../sriov_admin/vf<N>/
28Date: October 2025
29KernelVersion: 6.19
30Contact: intel-xe@lists.freedesktop.org
31Description:
32 These directories hold attributes related to the SR-IOV Virtual
33 Functions (VFs).
34
35 Note that the VF number <N> is 1-based as described in PCI SR-IOV
36 specification as the Xe driver follows that naming schema.
37
38 There could be "vf1", "vf2" and so on, up to "vf<N>", where <N>
39 matches the value of the "sriov_totalvfs" attribute.
40
41
42What: /sys/bus/pci/drivers/xe/.../sriov_admin/pf/profile/exec_quantum_ms
43What: /sys/bus/pci/drivers/xe/.../sriov_admin/pf/profile/preempt_timeout_us
44What: /sys/bus/pci/drivers/xe/.../sriov_admin/pf/profile/sched_priority
45What: /sys/bus/pci/drivers/xe/.../sriov_admin/vf<n>/profile/exec_quantum_ms
46What: /sys/bus/pci/drivers/xe/.../sriov_admin/vf<n>/profile/preempt_timeout_us
47What: /sys/bus/pci/drivers/xe/.../sriov_admin/vf<n>/profile/sched_priority
48Date: October 2025
49KernelVersion: 6.19
50Contact: intel-xe@lists.freedesktop.org
51Description:
52 These files expose scheduling parameters for the PF and its VFs, and
53 are visible only on Intel Xe platforms that use time-sliced GPU sharing.
54 They can be changed even if VFs are enabled and running and reflect the
55 settings of all tiles/GTs assigned to the given function.
56
57 exec_quantum_ms: (RW) unsigned integer
58 The GT execution quantum (EQ) in [ms] for the given function.
59 Actual quantum value might be aligned per HW/FW requirements.
60
61 Default is 0 (unlimited).
62
63 preempt_timeout_us: (RW) unsigned integer
64 The GT preemption timeout in [us] of the given function.
65 Actual timeout value might be aligned per HW/FW requirements.
66
67 Default is 0 (unlimited).
68
69 sched_priority: (RW/RO) string
70 The GT scheduling priority of the given function.
71
72 "low" - function will be scheduled on the GPU for its EQ/PT
73 only if function has any work already submitted.
74
75 "normal" - functions will be scheduled on the GPU for its EQ/PT
76 irrespective of whether it has submitted a work or not.
77
78 "high" - function will be scheduled on the GPU for its EQ/PT
79 in the next time-slice after the current one completes
80 and function has a work submitted.
81
82 Default is "low".
83
84 When read, this file will display the current and available
85 scheduling priorities. The currently active priority level will
86 be enclosed in square brackets, like:
87
88 [low] normal high
89
90 This file can be read-only if changing the priority is not
91 supported.
92
93 Writes to these attributes may fail with errors like:
94 -EINVAL if provided input is malformed or not recognized,
95 -EPERM if change is not applicable on given HW/FW,
96 -EIO if FW refuses to change the provisioning.
97
98 Reads from these attributes may fail with:
99 -EUCLEAN if value is not consistent across all tiles/GTs.
100
101
102What: /sys/bus/pci/drivers/xe/.../sriov_admin/.bulk_profile/exec_quantum_ms
103What: /sys/bus/pci/drivers/xe/.../sriov_admin/.bulk_profile/preempt_timeout_us
104What: /sys/bus/pci/drivers/xe/.../sriov_admin/.bulk_profile/sched_priority
105Date: October 2025
106KernelVersion: 6.19
107Contact: intel-xe@lists.freedesktop.org
108Description:
109 These files allows bulk reconfiguration of the scheduling parameters
110 of the PF or VFs and are available only for Intel Xe platforms with
111 GPU sharing based on the time-slice basis. These scheduling parameters
112 can be changed even if VFs are enabled and running.
113
114 exec_quantum_ms: (WO) unsigned integer
115 The GT execution quantum (EQ) in [ms] to be applied to all functions.
116 See sriov_admin/{pf,vf<N>}/profile/exec_quantum_ms for more details.
117
118 preempt_timeout_us: (WO) unsigned integer
119 The GT preemption timeout (PT) in [us] to be applied to all functions.
120 See sriov_admin/{pf,vf<N>}/profile/preempt_timeout_us for more details.
121
122 sched_priority: (RW/RO) string
123 The GT scheduling priority to be applied for all functions.
124 See sriov_admin/{pf,vf<N>}/profile/sched_priority for more details.
125
126 Writes to these attributes may fail with errors like:
127 -EINVAL if provided input is malformed or not recognized,
128 -EPERM if change is not applicable on given HW/FW,
129 -EIO if FW refuses to change the provisioning.
130
131
132What: /sys/bus/pci/drivers/xe/.../sriov_admin/vf<n>/stop
133Date: October 2025
134KernelVersion: 6.19
135Contact: intel-xe@lists.freedesktop.org
136Description:
137 This file allows to control scheduling of the VF on the Intel Xe GPU
138 platforms. It allows to implement custom policy mechanism in case VFs
139 are misbehaving or triggering adverse events above defined thresholds.
140
141 stop: (WO) bool
142 All GT executions of given function shall be immediately stopped.
143 To allow scheduling this VF again, the VF FLR must be triggered.
144
145 Writes to this attribute may fail with errors like:
146 -EINVAL if provided input is malformed or not recognized,
147 -EPERM if change is not applicable on given HW/FW,
148 -EIO if FW refuses to change the scheduling.
149
150
151What: /sys/bus/pci/drivers/xe/.../sriov_admin/pf/device
152What: /sys/bus/pci/drivers/xe/.../sriov_admin/vf<n>/device
153Date: October 2025
154KernelVersion: 6.19
155Contact: intel-xe@lists.freedesktop.org
156Description:
157 These are symlinks to the underlying PCI device entry representing
158 given Xe SR-IOV function. For the PF, this link is always present.
159 For VFs, this link is present only for currently enabled VFs.
Documentation/accel/qaic/aic100.rst+23−2
@@ -487,8 +487,8 @@ one user crashes, the fallout of that should be limited to that workload and not · 486 unmodified lines
487impact other workloads. SSR accomplishes this.487impact other workloads. SSR accomplishes this.
488 488
489If a particular workload crashes, QSM notifies the host via the QAIC_SSR MHI489If a particular workload crashes, QSM notifies the host via the QAIC_SSR MHI
490channel. This notification identifies the workload by it's assigned DBC. A490channel. This notification identifies the workload by its assigned DBC. A
491multi-stage recovery process is then used to cleanup both sides, and get the491multi-stage recovery process is then used to cleanup both sides, and gets the
492DBC/NSPs into a working state.492DBC/NSPs into a working state.
493 493
494When SSR occurs, any state in the workload is lost. Any inputs that were in494When SSR occurs, any state in the workload is lost. Any inputs that were in
@@ -496,6 +496,27 @@ process, or queued by not yet serviced, are lost. The loaded artifacts will · 1 unmodified line
496remain in on-card DDR, but the host will need to re-activate the workload if496remain in on-card DDR, but the host will need to re-activate the workload if
497it desires to recover the workload.497it desires to recover the workload.
498 498
499When SSR occurs for a specific NSP, the assigned DBC goes through the
500following state transactions in order:
501
502DBC_STATE_BEFORE_SHUTDOWN
503 Indicates that the affected NSP was found in an unrecoverable error
504 condition.
505DBC_STATE_AFTER_SHUTDOWN
506 Indicates that the NSP is under reset.
507DBC_STATE_BEFORE_POWER_UP
508 Indicates that the NSP's debug information has been collected, and is
509 ready to be collected by the host (if desired). At that stage the NSP
510 is restarted by QSM.
511DBC_STATE_AFTER_POWER_UP
512 Indicates that the NSP has been restarted, fully operational and is
513 in idle state.
514
515SSR also has an optional crashdump collection feature. If enabled, the host can
516collect the memory dump for the crashed NSP and dump it to the user space via
517the dev_coredump subsystem. The host can also decline the crashdump collection
518request from the device.
519
499Reliability, Accessibility, Serviceability (RAS)520Reliability, Accessibility, Serviceability (RAS)
500================================================521================================================
501 522
Documentation/accel/qaic/qaic.rst+4−4
@@ -36,7 +36,7 @@ polling mode and reenables the IRQ line. · 35 unmodified lines
36This mitigation in QAIC is very effective. The same lprnet usecase that36This mitigation in QAIC is very effective. The same lprnet usecase that
37generates 100k IRQs per second (per /proc/interrupts) is reduced to roughly 6437generates 100k IRQs per second (per /proc/interrupts) is reduced to roughly 64
38IRQs over 5 minutes while keeping the host system stable, and having the same38IRQs over 5 minutes while keeping the host system stable, and having the same
39workload throughput performance (within run to run noise variation).39workload throughput performance (within run-to-run noise variation).
40 40
41Single MSI Mode41Single MSI Mode
42---------------42---------------
@@ -49,7 +49,7 @@ useful to be able to fall back to a single MSI when needed. · 6 unmodified lines
49To support this fallback, we allow the case where only one MSI is able to be49To support this fallback, we allow the case where only one MSI is able to be
50allocated, and share that one MSI between MHI and the DBCs. The device detects50allocated, and share that one MSI between MHI and the DBCs. The device detects
51when only one MSI has been configured and directs the interrupts for the DBCs51when only one MSI has been configured and directs the interrupts for the DBCs
52to the interrupt normally used for MHI. Unfortunately this means that the52to the interrupt normally used for MHI. Unfortunately, this means that the
53interrupt handlers for every DBC and MHI wake up for every interrupt that53interrupt handlers for every DBC and MHI wake up for every interrupt that
54arrives; however, the DBC threaded irq handlers only are started when work to be54arrives; however, the DBC threaded irq handlers only are started when work to be
55done is detected (MHI will always start its threaded handler).55done is detected (MHI will always start its threaded handler).
@@ -62,9 +62,9 @@ never disabled, allowing each new entry to the FIFO to trigger a new interrupt. · 6 unmodified lines
62Neural Network Control (NNC) Protocol62Neural Network Control (NNC) Protocol
63=====================================63=====================================
64 64
65The implementation of NNC is split between the KMD (QAIC) and UMD. In general65The implementation of NNC is split between the KMD (QAIC) and UMD. In general,
66QAIC understands how to encode/decode NNC wire protocol, and elements of the66QAIC understands how to encode/decode NNC wire protocol, and elements of the
67protocol which require kernel space knowledge to process (for example, mapping67protocol which requires kernel space knowledge to process (for example, mapping
68host memory to device IOVAs). QAIC understands the structure of a message, and68host memory to device IOVAs). QAIC understands the structure of a message, and
69all of the transactions. QAIC does not understand commands (the payload of a69all of the transactions. QAIC does not understand commands (the payload of a
70passthrough transaction).70passthrough transaction).
Documentation/devicetree/bindings/display/bridge/fsl,imx8mp-hdmi-tx.yaml+12−0
@@ -49,6 +49,10 @@ properties: · 48 unmodified lines
49 $ref: /schemas/graph.yaml#/properties/port49 $ref: /schemas/graph.yaml#/properties/port
50 description: HDMI output port50 description: HDMI output port
51 51
52 port@2:
53 $ref: /schemas/graph.yaml#/properties/port
54 description: Parallel audio input port
55
52 required:56 required:
53 - port@057 - port@0
54 - port@158 - port@1
@@ -98,5 +102,13 @@ examples: · 43 unmodified lines
98 remote-endpoint = <&hdmi0_con>;102 remote-endpoint = <&hdmi0_con>;
99 };103 };
100 };104 };
105
106 port@2 {
107 reg = <2>;
108
109 endpoint {
110 remote-endpoint = <&pai_to_hdmi_tx>;
111 };
112 };
101 };113 };
102 };114 };
Documentation/devicetree/bindings/display/bridge/ite,it66121.yaml+1−0
@@ -19,6 +19,7 @@ properties: · 18 unmodified lines
19 compatible:19 compatible:
20 enum:20 enum:
21 - ite,it6612121 - ite,it66121
22 - ite,it66122
22 - ite,it661023 - ite,it6610
23 24
24 reg:25 reg:
Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml+52−2
@@ -14,6 +14,9 @@ description: | · 13 unmodified lines
14 R-Car Gen4 SoCs. The encoder can operate in either DSI or CSI-2 mode, with up14 R-Car Gen4 SoCs. The encoder can operate in either DSI or CSI-2 mode, with up
15 to four data lanes.15 to four data lanes.
16 16
17allOf:
18 - $ref: /schemas/display/dsi-controller.yaml#
19
17properties:20properties:
18 compatible:21 compatible:
19 enum:22 enum:
@@ -80,14 +83,14 @@ required: · 60 unmodified lines
80 - resets83 - resets
81 - ports84 - ports
82 85
83additionalProperties: false86unevaluatedProperties: false
84 87
85examples:88examples:
86 - |89 - |
87 #include <dt-bindings/clock/r8a779a0-cpg-mssr.h>90 #include <dt-bindings/clock/r8a779a0-cpg-mssr.h>
88 #include <dt-bindings/power/r8a779a0-sysc.h>91 #include <dt-bindings/power/r8a779a0-sysc.h>
89 92
90 dsi0: dsi-encoder@fed80000 {93 dsi@fed80000 {
91 compatible = "renesas,r8a779a0-dsi-csi2-tx";94 compatible = "renesas,r8a779a0-dsi-csi2-tx";
92 reg = <0xfed80000 0x10000>;95 reg = <0xfed80000 0x10000>;
93 power-domains = <&sysc R8A779A0_PD_ALWAYS_ON>;96 power-domains = <&sysc R8A779A0_PD_ALWAYS_ON>;
@@ -117,4 +120,51 @@ examples: · 23 unmodified lines
117 };120 };
118 };121 };
119 };122 };
123
124 - |
125 #include <dt-bindings/clock/r8a779g0-cpg-mssr.h>
126 #include <dt-bindings/power/r8a779g0-sysc.h>
127
128 dsi@fed80000 {
129 #address-cells = <1>;
130 #size-cells = <0>;
131 compatible = "renesas,r8a779g0-dsi-csi2-tx";
132 reg = <0xfed80000 0x10000>;
133 clocks = <&cpg CPG_MOD 415>,
134 <&cpg CPG_CORE R8A779G0_CLK_DSIEXT>,
135 <&cpg CPG_CORE R8A779G0_CLK_DSIREF>;
136 clock-names = "fck", "dsi", "pll";
137 power-domains = <&sysc R8A779G0_PD_ALWAYS_ON>;
138 resets = <&cpg 415>;
139
140 ports {
141 #address-cells = <1>;
142 #size-cells = <0>;
143
144 port@0 {
145 reg = <0>;
146 };
147
148 port@1 {
149 reg = <1>;
150
151 dsi0port1_out: endpoint {
152 remote-endpoint = <&panel_in>;
153 data-lanes = <1 2>;
154 };
155 };
156 };
157
158 panel@0 {
159 reg = <0>;
160 compatible = "raspberrypi,dsi-7inch", "ilitek,ili9881c";
161 power-supply = <&vcc_lcd_reg>;
162
163 port {
164 panel_in: endpoint {
165 remote-endpoint = <&dsi0port1_out>;
166 };
167 };
168 };
169 };
120...170...
Documentation/devicetree/bindings/display/bridge/simple-bridge.yaml+2−0
@@ -27,7 +27,9 @@ properties: · 26 unmodified lines
27 - const: adi,adv712327 - const: adi,adv7123
28 - enum:28 - enum:
29 - adi,adv712329 - adi,adv7123
30 - asl-tek,cs5263
30 - dumb-vga-dac31 - dumb-vga-dac
32 - parade,ps185hdm
31 - radxa,ra62033 - radxa,ra620
32 - realtek,rtd217134 - realtek,rtd2171
33 - ti,opa36235 - ti,opa362
Documentation/devicetree/bindings/display/imx/fsl,imx8mp-hdmi-pai.yaml+69−0
@@ -0,0 +1,69 @@
1# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/imx/fsl,imx8mp-hdmi-pai.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Freescale i.MX8MP HDMI Parallel Audio Interface
8
9maintainers:
10 - Shengjiu Wang <shengjiu.wang@nxp.com>
11
12description:
13 The HDMI TX Parallel Audio Interface (HTX_PAI) is a bridge between the
14 Audio Subsystem to the HDMI TX Controller.
15
16properties:
17 compatible:
18 const: fsl,imx8mp-hdmi-pai
19
20 reg:
21 maxItems: 1
22
23 interrupts:
24 maxItems: 1
25
26 clocks:
27 maxItems: 1
28
29 clock-names:
30 const: apb
31
32 power-domains:
33 maxItems: 1
34
35 port:
36 $ref: /schemas/graph.yaml#/properties/port
37 description: Output to the HDMI TX controller.
38
39required:
40 - compatible
41 - reg
42 - interrupts
43 - clocks
44 - clock-names
45 - power-domains
46 - port
47
48additionalProperties: false
49
50examples:
51 - |
52 #include <dt-bindings/clock/imx8mp-clock.h>
53 #include <dt-bindings/power/imx8mp-power.h>
54
55 audio-bridge@32fc4800 {
56 compatible = "fsl,imx8mp-hdmi-pai";
57 reg = <0x32fc4800 0x800>;
58 interrupt-parent = <&irqsteer_hdmi>;
59 interrupts = <14>;
60 clocks = <&clk IMX8MP_CLK_HDMI_APB>;
61 clock-names = "apb";
62 power-domains = <&hdmi_blk_ctrl IMX8MP_HDMIBLK_PD_PAI>;
63
64 port {
65 pai_to_hdmi_tx: endpoint {
66 remote-endpoint = <&hdmi_tx_from_pai>;
67 };
68 };
69 };
Documentation/devicetree/bindings/display/msm/dp-controller.yaml+16−1
@@ -18,6 +18,7 @@ properties: · 17 unmodified lines
18 compatible:18 compatible:
19 oneOf:19 oneOf:
20 - enum:20 - enum:
21 - qcom,glymur-dp
21 - qcom,sa8775p-dp22 - qcom,sa8775p-dp
22 - qcom,sc7180-dp23 - qcom,sc7180-dp
23 - qcom,sc7280-dp24 - qcom,sc7280-dp
@@ -31,6 +32,11 @@ properties: · 7 unmodified lines
31 - qcom,sm8650-dp32 - qcom,sm8650-dp
32 - qcom,x1e80100-dp33 - qcom,x1e80100-dp
33 34
35 - items:
36 - enum:
37 - qcom,qcs8300-dp
38 - const: qcom,sa8775p-dp
39
34 - items:40 - items:
35 - enum:41 - enum:
36 - qcom,sm6350-dp42 - qcom,sm6350-dp
@@ -53,6 +59,12 @@ properties: · 16 unmodified lines
53 - qcom,sm8550-dp59 - qcom,sm8550-dp
54 - const: qcom,sm8350-dp60 - const: qcom,sm8350-dp
55 61
62 - items:
63 - enum:
64 - qcom,sm6150-dp
65 - const: qcom,sm8150-dp
66 - const: qcom,sm8350-dp
67
56 - items:68 - items:
57 - enum:69 - enum:
58 - qcom,sm8750-dp70 - qcom,sm8750-dp
@@ -195,9 +207,11 @@ allOf: · 136 unmodified lines
195 compatible:207 compatible:
196 contains:208 contains:
197 enum:209 enum:
210 - qcom,glymur-dp
198 - qcom,sa8775p-dp211 - qcom,sa8775p-dp
199 - qcom,x1e80100-dp212 - qcom,x1e80100-dp
200 then:213 then:
214 $ref: /schemas/sound/dai-common.yaml#
201 oneOf:215 oneOf:
202 - required:216 - required:
203 - aux-bus217 - aux-bus
@@ -239,6 +253,7 @@ allOf: · 35 unmodified lines
239 enum:253 enum:
240 # these platforms support 2 streams MST on some interfaces,254 # these platforms support 2 streams MST on some interfaces,
241 # others are SST only255 # others are SST only
256 - qcom,glymur-dp
242 - qcom,sc8280xp-dp257 - qcom,sc8280xp-dp
243 - qcom,x1e80100-dp258 - qcom,x1e80100-dp
244 then:259 then:
@@ -295,7 +310,7 @@ allOf: · 50 unmodified lines
295 minItems: 6310 minItems: 6
296 maxItems: 8311 maxItems: 8
297 312
298additionalProperties: false313unevaluatedProperties: false
299 314
300examples:315examples:
301 - |316 - |
Documentation/devicetree/bindings/display/msm/gmu.yaml+59−1
@@ -21,7 +21,7 @@ properties: · 20 unmodified lines
21 compatible:21 compatible:
22 oneOf:22 oneOf:
23 - items:23 - items:
24 - pattern: '^qcom,adreno-gmu-[67][0-9][0-9]\.[0-9]$'24 - pattern: '^qcom,adreno-gmu-[6-8][0-9][0-9]\.[0-9]$'
25 - const: qcom,adreno-gmu25 - const: qcom,adreno-gmu
26 - items:26 - items:
27 - pattern: '^qcom,adreno-gmu-x[1-9][0-9][0-9]\.[0-9]$'27 - pattern: '^qcom,adreno-gmu-x[1-9][0-9][0-9]\.[0-9]$'
@@ -299,6 +299,64 @@ allOf: · 271 unmodified lines
299 required:299 required:
300 - qcom,qmp300 - qcom,qmp
301 301
302 - if:
303 properties:
304 compatible:
305 contains:
306 const: qcom,adreno-gmu-840.1
307 then:
308 properties:
309 reg:
310 items:
311 - description: Core GMU registers
312 reg-names:
313 items:
314 - const: gmu
315 clocks:
316 items:
317 - description: GPU AHB clock
318 - description: GMU clock
319 - description: GPU CX clock
320 - description: GPU MEMNOC clock
321 - description: GMU HUB clock
322 clock-names:
323 items:
324 - const: ahb
325 - const: gmu
326 - const: cxo
327 - const: memnoc
328 - const: hub
329
330 - if:
331 properties:
332 compatible:
333 contains:
334 const: qcom,adreno-gmu-x285.1
335 then:
336 properties:
337 reg:
338 items:
339 - description: Core GMU registers
340 reg-names:
341 items:
342 - const: gmu
343 clocks:
344 items:
345 - description: GPU AHB clock
346 - description: GMU clock
347 - description: GPU CX clock
348 - description: GPU MEMNOC clock
349 - description: GMU HUB clock
350 - description: GMU RSCC HUB clock
351 clock-names:
352 items:
353 - const: ahb
354 - const: gmu
355 - const: cxo
356 - const: memnoc
357 - const: hub
358 - const: rscc
359
302 - if:360 - if:
303 properties:361 properties:
304 compatible:362 compatible:
Documentation/devicetree/bindings/display/msm/qcom,glymur-mdss.yaml+264−0
@@ -0,0 +1,264 @@
1# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/msm/qcom,glymur-mdss.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Qualcomm Glymur Display MDSS
8
9maintainers:
10 - Abel Vesa <abel.vesa@linaro.org>
11
12description:
13 Glymur MSM Mobile Display Subsystem(MDSS), which encapsulates sub-blocks like
14 DPU display controller, DP interfaces, etc.
15
16$ref: /schemas/display/msm/mdss-common.yaml#
17
18properties:
19 compatible:
20 const: qcom,glymur-mdss
21
22 clocks:
23 items:
24 - description: Display AHB
25 - description: Display hf AXI
26 - description: Display core
27
28 iommus:
29 maxItems: 1
30
31 interconnects:
32 items:
33 - description: Interconnect path from mdp0 port to the data bus
34 - description: Interconnect path from CPU to the reg bus
35
36 interconnect-names:
37 items:
38 - const: mdp0-mem
39 - const: cpu-cfg
40
41patternProperties:
42 "^display-controller@[0-9a-f]+$":
43 type: object
44 additionalProperties: true
45 properties:
46 compatible:
47 const: qcom,glymur-dpu
48
49 "^displayport-controller@[0-9a-f]+$":
50 type: object
51 additionalProperties: true
52 properties:
53 compatible:
54 const: qcom,glymur-dp
55
56 "^phy@[0-9a-f]+$":
57 type: object
58 additionalProperties: true
59 properties:
60 compatible:
61 const: qcom,glymur-dp-phy
62
63required:
64 - compatible
65
66unevaluatedProperties: false
67
68examples:
69 - |
70 #include <dt-bindings/clock/qcom,rpmh.h>
71 #include <dt-bindings/interconnect/qcom,icc.h>
72 #include <dt-bindings/interconnect/qcom,glymur-rpmh.h>
73 #include <dt-bindings/interrupt-controller/arm-gic.h>
74 #include <dt-bindings/phy/phy-qcom-qmp.h>
75 #include <dt-bindings/power/qcom,rpmhpd.h>
76
77 display-subsystem@ae00000 {
78 compatible = "qcom,glymur-mdss";
79 reg = <0x0ae00000 0x1000>;
80 reg-names = "mdss";
81
82 interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>;
83
84 clocks = <&dispcc_ahb_clk>,
85 <&gcc_disp_hf_axi_clk>,
86 <&dispcc_mdp_clk>;
87 clock-names = "bus", "nrt_bus", "core";
88
89 interconnects = <&mmss_noc MASTER_MDP QCOM_ICC_TAG_ALWAYS
90 &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
91 <&hsc_noc MASTER_APPSS_PROC QCOM_ICC_TAG_ACTIVE_ONLY
92 &config_noc SLAVE_DISPLAY_CFG QCOM_ICC_TAG_ACTIVE_ONLY>;
93 interconnect-names = "mdp0-mem",
94 "cpu-cfg";
95
96 resets = <&disp_cc_mdss_core_bcr>;
97
98 power-domains = <&mdss_gdsc>;
99
100 iommus = <&apps_smmu 0x1c00 0x2>;
101
102 interrupt-controller;
103 #interrupt-cells = <1>;
104
105 #address-cells = <1>;
106 #size-cells = <1>;
107 ranges;
108
109 display-controller@ae01000 {
110 compatible = "qcom,glymur-dpu";
111 reg = <0x0ae01000 0x8f000>,
112 <0x0aeb0000 0x2008>;
113 reg-names = "mdp", "vbif";
114
115 clocks = <&gcc_axi_clk>,
116 <&dispcc_ahb_clk>,
117 <&dispcc_mdp_lut_clk>,
118 <&dispcc_mdp_clk>,
119 <&dispcc_mdp_vsync_clk>;
120 clock-names = "nrt_bus",
121 "iface",
122 "lut",
123 "core",
124 "vsync";
125
126 assigned-clocks = <&dispcc_mdp_vsync_clk>;
127 assigned-clock-rates = <19200000>;
128
129 operating-points-v2 = <&mdp_opp_table>;
130 power-domains = <&rpmhpd RPMHPD_MMCX>;
131
132 interrupt-parent = <&mdss>;
133 interrupts = <0>;
134
135 ports {
136 #address-cells = <1>;
137 #size-cells = <0>;
138
139 port@0 {
140 reg = <0>;
141 dpu_intf1_out: endpoint {
142 remote-endpoint = <&dsi0_in>;
143 };
144 };
145
146 port@1 {
147 reg = <1>;
148 dpu_intf2_out: endpoint {
149 remote-endpoint = <&dsi1_in>;
150 };
151 };
152 };
153
154 mdp_opp_table: opp-table {
155 compatible = "operating-points-v2";
156
157 opp-200000000 {
158 opp-hz = /bits/ 64 <200000000>;
159 required-opps = <&rpmhpd_opp_low_svs>;
160 };
161
162 opp-325000000 {
163 opp-hz = /bits/ 64 <325000000>;
164 required-opps = <&rpmhpd_opp_svs>;
165 };
166
167 opp-375000000 {
168 opp-hz = /bits/ 64 <375000000>;
169 required-opps = <&rpmhpd_opp_svs_l1>;
170 };
171
172 opp-514000000 {
173 opp-hz = /bits/ 64 <514000000>;
174 required-opps = <&rpmhpd_opp_nom>;
175 };
176 };
177 };
178
179 displayport-controller@ae90000 {
180 compatible = "qcom,glymur-dp";
181 reg = <0xae90000 0x200>,
182 <0xae90200 0x200>,
183 <0xae90400 0x600>,
184 <0xae91000 0x400>,
185 <0xae91400 0x400>;
186
187 interrupt-parent = <&mdss>;
188 interrupts = <12>;
189
190 clocks = <&dispcc_mdss_ahb_clk>,
191 <&dispcc_dptx0_aux_clk>,
192 <&dispcc_dptx0_link_clk>,
193 <&dispcc_dptx0_link_intf_clk>,
194 <&dispcc_dptx0_pixel0_clk>,
195 <&dispcc_dptx0_pixel1_clk>;
196 clock-names = "core_iface",
197 "core_aux",
198 "ctrl_link",
199 "ctrl_link_iface",
200 "stream_pixel",
201 "stream_1_pixel";
202
203 assigned-clocks = <&dispcc_mdss_dptx0_link_clk_src>,
204 <&dispcc_mdss_dptx0_pixel0_clk_src>,
205 <&dispcc_mdss_dptx0_pixel1_clk_src>;
206 assigned-clock-parents = <&usb_1_ss0_qmpphy QMP_USB43DP_DP_LINK_CLK>,
207 <&usb_1_ss0_qmpphy QMP_USB43DP_DP_VCO_DIV_CLK>,
208 <&usb_1_ss0_qmpphy QMP_USB43DP_DP_VCO_DIV_CLK>;
209
210 operating-points-v2 = <&mdss_dp0_opp_table>;
211
212 power-domains = <&rpmhpd RPMHPD_MMCX>;
213
214 phys = <&usb_1_ss0_qmpphy QMP_USB43DP_DP_PHY>;
215 phy-names = "dp";
216
217 #sound-dai-cells = <0>;
218
219 ports {
220 #address-cells = <1>;
221 #size-cells = <0>;
222
223 port@0 {
224 reg = <0>;
225
226 mdss_dp0_in: endpoint {
227 remote-endpoint = <&mdss_intf0_out>;
228 };
229 };
230
231 port@1 {
232 reg = <1>;
233
234 mdss_dp0_out: endpoint {
235 };
236 };
237 };
238
239 mdss_dp0_opp_table: opp-table {
240 compatible = "operating-points-v2";
241
242 opp-160000000 {
243 opp-hz = /bits/ 64 <160000000>;
244 required-opps = <&rpmhpd_opp_low_svs>;
245 };
246
247 opp-270000000 {
248 opp-hz = /bits/ 64 <270000000>;
249 required-opps = <&rpmhpd_opp_svs>;
250 };
251
252 opp-540000000 {
253 opp-hz = /bits/ 64 <540000000>;
254 required-opps = <&rpmhpd_opp_svs_l1>;
255 };
256
257 opp-810000000 {
258 opp-hz = /bits/ 64 <810000000>;
259 required-opps = <&rpmhpd_opp_nom>;
260 };
261 };
262 };
263 };
264...
Documentation/devicetree/bindings/display/msm/qcom,qcs8300-mdss.yaml+286−0
@@ -0,0 +1,286 @@
1# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/msm/qcom,qcs8300-mdss.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Qualcomm Technologies, Inc. QCS8300 Display MDSS
8
9maintainers:
10 - Yongxing Mou <yongxing.mou@oss.qualcomm.com>
11
12description:
13 QCS8300 MSM Mobile Display Subsystem(MDSS), which encapsulates sub-blocks like
14 DPU display controller, DP interfaces and EDP etc.
15
16$ref: /schemas/display/msm/mdss-common.yaml#
17
18properties:
19 compatible:
20 const: qcom,qcs8300-mdss
21
22 clocks:
23 items:
24 - description: Display AHB
25 - description: Display hf AXI
26 - description: Display core
27
28 iommus:
29 maxItems: 1
30
31 interconnects:
32 maxItems: 3
33
34 interconnect-names:
35 maxItems: 3
36
37patternProperties:
38 "^display-controller@[0-9a-f]+$":
39 type: object
40 additionalProperties: true
41
42 properties:
43 compatible:
44 contains:
45 const: qcom,qcs8300-dpu
46
47 "^displayport-controller@[0-9a-f]+$":
48 type: object
49 additionalProperties: true
50
51 properties:
52 compatible:
53 contains:
54 const: qcom,qcs8300-dp
55
56 "^phy@[0-9a-f]+$":
57 type: object
58 additionalProperties: true
59 properties:
60 compatible:
61 contains:
62 const: qcom,qcs8300-edp-phy
63
64required:
65 - compatible
66
67unevaluatedProperties: false
68
69examples:
70 - |
71 #include <dt-bindings/interconnect/qcom,icc.h>
72 #include <dt-bindings/interrupt-controller/arm-gic.h>
73 #include <dt-bindings/clock/qcom,qcs8300-gcc.h>
74 #include <dt-bindings/clock/qcom,sa8775p-dispcc.h>
75 #include <dt-bindings/interconnect/qcom,qcs8300-rpmh.h>
76 #include <dt-bindings/power/qcom,rpmhpd.h>
77 #include <dt-bindings/power/qcom-rpmpd.h>
78
79 mdss: display-subsystem@ae00000 {
80 compatible = "qcom,qcs8300-mdss";
81 reg = <0x0ae00000 0x1000>;
82 reg-names = "mdss";
83
84 interconnects = <&mmss_noc MASTER_MDP0 QCOM_ICC_TAG_ACTIVE_ONLY
85 &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ACTIVE_ONLY>,
86 <&mmss_noc MASTER_MDP1 QCOM_ICC_TAG_ACTIVE_ONLY
87 &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ACTIVE_ONLY>,
88 <&gem_noc MASTER_APPSS_PROC QCOM_ICC_TAG_ACTIVE_ONLY
89 &config_noc SLAVE_DISPLAY_CFG QCOM_ICC_TAG_ACTIVE_ONLY>;
90 interconnect-names = "mdp0-mem",
91 "mdp1-mem",
92 "cpu-cfg";
93
94 resets = <&dispcc_core_bcr>;
95 power-domains = <&dispcc_gdsc>;
96
97 clocks = <&dispcc_ahb_clk>,
98 <&gcc GCC_DISP_HF_AXI_CLK>,
99 <&dispcc_mdp_clk>;
100
101 interrupts = <GIC_SPI 92 IRQ_TYPE_LEVEL_HIGH>;
102 interrupt-controller;
103 #interrupt-cells = <1>;
104
105 iommus = <&apps_smmu 0x1000 0x402>;
106
107 #address-cells = <1>;
108 #size-cells = <1>;
109 ranges;
110
111 display-controller@ae01000 {
112 compatible = "qcom,qcs8300-dpu", "qcom,sa8775p-dpu";
113 reg = <0x0ae01000 0x8f000>,
114 <0x0aeb0000 0x2008>;
115 reg-names = "mdp", "vbif";
116
117 clocks = <&gcc GCC_DISP_HF_AXI_CLK>,
118 <&dispcc0 MDSS_DISP_CC_MDSS_AHB_CLK>,
119 <&dispcc0 MDSS_DISP_CC_MDSS_MDP_LUT_CLK>,
120 <&dispcc0 MDSS_DISP_CC_MDSS_MDP_CLK>,
121 <&dispcc0 MDSS_DISP_CC_MDSS_VSYNC_CLK>;
122 clock-names = "nrt_bus",
123 "iface",
124 "lut",
125 "core",
126 "vsync";
127
128 assigned-clocks = <&dispcc0 MDSS_DISP_CC_MDSS_VSYNC_CLK>;
129 assigned-clock-rates = <19200000>;
130 operating-points-v2 = <&mdp_opp_table>;
131 power-domains = <&rpmhpd RPMHPD_MMCX>;
132
133 interrupt-parent = <&mdss>;
134 interrupts = <0>;
135 ports {
136 #address-cells = <1>;
137 #size-cells = <0>;
138 port@0 {
139 reg = <0>;
140
141 dpu_intf0_out: endpoint {
142 remote-endpoint = <&mdss_dp0_in>;
143 };
144 };
145 };
146
147 mdp_opp_table: opp-table {
148 compatible = "operating-points-v2";
149
150 opp-375000000 {
151 opp-hz = /bits/ 64 <375000000>;
152 required-opps = <&rpmhpd_opp_svs_l1>;
153 };
154
155 opp-500000000 {
156 opp-hz = /bits/ 64 <500000000>;
157 required-opps = <&rpmhpd_opp_nom>;
158 };
159
160 opp-575000000 {
161 opp-hz = /bits/ 64 <575000000>;
162 required-opps = <&rpmhpd_opp_turbo>;
163 };
164
165 opp-650000000 {
166 opp-hz = /bits/ 64 <650000000>;
167 required-opps = <&rpmhpd_opp_turbo_l1>;
168 };
169 };
170 };
171
172 mdss_dp0_phy: phy@aec2a00 {
173 compatible = "qcom,qcs8300-edp-phy", "qcom,sa8775p-edp-phy";
174
175 reg = <0x0aec2a00 0x200>,
176 <0x0aec2200 0xd0>,
177 <0x0aec2600 0xd0>,
178 <0x0aec2000 0x1c8>;
179
180 clocks = <&dispcc MDSS_DISP_CC_MDSS_DPTX0_AUX_CLK>,
181 <&dispcc MDSS_DISP_CC_MDSS_AHB_CLK>;
182 clock-names = "aux",
183 "cfg_ahb";
184
185 #clock-cells = <1>;
186 #phy-cells = <0>;
187
188 vdda-phy-supply = <&vreg_l1c>;
189 vdda-pll-supply = <&vreg_l4a>;
190 };
191
192 displayport-controller@af54000 {
193 compatible = "qcom,qcs8300-dp", "qcom,sa8775p-dp";
194
195 pinctrl-0 = <&dp_hot_plug_det>;
196 pinctrl-names = "default";
197
198 reg = <0xaf54000 0x104>,
199 <0xaf54200 0x0c0>,
200 <0xaf55000 0x770>,
201 <0xaf56000 0x09c>,
202 <0xaf57000 0x09c>,
203 <0xaf58000 0x09c>,
204 <0xaf59000 0x09c>,
205 <0xaf5a000 0x23c>,
206 <0xaf5b000 0x23c>;
207
208 interrupt-parent = <&mdss>;
209 interrupts = <12>;
210 clocks = <&dispcc0 MDSS_DISP_CC_MDSS_AHB_CLK>,
211 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_AUX_CLK>,
212 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_LINK_CLK>,
213 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_LINK_INTF_CLK>,
214 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_PIXEL0_CLK>,
215 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_PIXEL1_CLK>,
216 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_PIXEL2_CLK>,
217 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_PIXEL3_CLK>;
218 clock-names = "core_iface",
219 "core_aux",
220 "ctrl_link",
221 "ctrl_link_iface",
222 "stream_pixel",
223 "stream_1_pixel",
224 "stream_2_pixel",
225 "stream_3_pixel";
226 assigned-clocks = <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_LINK_CLK_SRC>,
227 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_PIXEL0_CLK_SRC>,
228 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_PIXEL1_CLK_SRC>,
229 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_PIXEL2_CLK_SRC>,
230 <&dispcc0 MDSS_DISP_CC_MDSS_DPTX0_PIXEL3_CLK_SRC>;
231 assigned-clock-parents = <&mdss_dp0_phy 0>,
232 <&mdss_dp0_phy 1>,
233 <&mdss_dp0_phy 1>,
234 <&mdss_dp0_phy 1>;
235 phys = <&mdss_dp0_phy>;
236 phy-names = "dp";
237 operating-points-v2 = <&dp_opp_table>;
238 power-domains = <&rpmhpd RPMHPD_MMCX>;
239
240 #sound-dai-cells = <0>;
241
242 ports {
243 #address-cells = <1>;
244 #size-cells = <0>;
245
246 port@0 {
247 reg = <0>;
248
249 mdss_dp0_in: endpoint {
250 remote-endpoint = <&dpu_intf0_out>;
251 };
252 };
253
254 port@1 {
255 reg = <1>;
256
257 mdss_dp_out: endpoint { };
258 };
259 };
260
261 dp_opp_table: opp-table {
262 compatible = "operating-points-v2";
263
264 opp-160000000 {
265 opp-hz = /bits/ 64 <160000000>;
266 required-opps = <&rpmhpd_opp_low_svs>;
267 };
268
269 opp-270000000 {
270 opp-hz = /bits/ 64 <270000000>;
271 required-opps = <&rpmhpd_opp_svs>;
272 };
273
274 opp-540000000 {
275 opp-hz = /bits/ 64 <540000000>;
276 required-opps = <&rpmhpd_opp_svs_l1>;
277 };
278
279 opp-810000000 {
280 opp-hz = /bits/ 64 <810000000>;
281 required-opps = <&rpmhpd_opp_nom>;
282 };
283 };
284 };
285 };
286...
Documentation/devicetree/bindings/display/msm/qcom,sm6150-mdss.yaml+25−15
@@ -51,6 +51,14 @@ patternProperties: · 50 unmodified lines
51 compatible:51 compatible:
52 const: qcom,sm6150-dpu52 const: qcom,sm6150-dpu
53 53
54 "^displayport-controller@[0-9a-f]+$":
55 type: object
56 additionalProperties: true
57 properties:
58 compatible:
59 contains:
60 const: qcom,sm6150-dp
61
54 "^dsi@[0-9a-f]+$":62 "^dsi@[0-9a-f]+$":
55 type: object63 type: object
56 additionalProperties: true64 additionalProperties: true
@@ -130,35 +138,37 @@ examples: · 73 unmodified lines
130 #size-cells = <0>;138 #size-cells = <0>;
131 139
132 port@0 {140 port@0 {
133 reg = <0>;141 reg = <0>;
134 dpu_intf0_out: endpoint {142
135 };143 dpu_intf0_out: endpoint {
144 };
136 };145 };
137 146
138 port@1 {147 port@1 {
139 reg = <1>;148 reg = <1>;
140 dpu_intf1_out: endpoint {149
141 remote-endpoint = <&mdss_dsi0_in>;150 dpu_intf1_out: endpoint {
142 };151 remote-endpoint = <&mdss_dsi0_in>;
152 };
143 };153 };
144 };154 };
145 155
146 mdp_opp_table: opp-table {156 mdp_opp_table: opp-table {
147 compatible = "operating-points-v2";157 compatible = "operating-points-v2";
148 158
149 opp-19200000 {159 opp-192000000 {
150 opp-hz = /bits/ 64 <19200000>;160 opp-hz = /bits/ 64 <192000000>;
151 required-opps = <&rpmhpd_opp_low_svs>;161 required-opps = <&rpmhpd_opp_low_svs>;
152 };162 };
153 163
154 opp-25600000 {164 opp-256000000 {
155 opp-hz = /bits/ 64 <25600000>;165 opp-hz = /bits/ 64 <256000000>;
156 required-opps = <&rpmhpd_opp_svs>;166 required-opps = <&rpmhpd_opp_svs>;
157 };167 };
158 168
159 opp-307200000 {169 opp-307200000 {
160 opp-hz = /bits/ 64 <307200000>;170 opp-hz = /bits/ 64 <307200000>;
161 required-opps = <&rpmhpd_opp_nom>;171 required-opps = <&rpmhpd_opp_nom>;
162 };172 };
163 };173 };
164 };174 };
Documentation/devicetree/bindings/display/msm/qcom,sm8650-dpu.yaml+11−5
@@ -13,11 +13,17 @@ $ref: /schemas/display/msm/dpu-common.yaml# · 12 unmodified lines
13 13
14properties:14properties:
15 compatible:15 compatible:
16 enum:16 oneOf:
17 - qcom,sa8775p-dpu17 - enum:
18 - qcom,sm8650-dpu18 - qcom,glymur-dpu
19 - qcom,sm8750-dpu19 - qcom,sa8775p-dpu
20 - qcom,x1e80100-dpu20 - qcom,sm8650-dpu
21 - qcom,sm8750-dpu
22 - qcom,x1e80100-dpu
23 - items:
24 - enum:
25 - qcom,qcs8300-dpu
26 - const: qcom,sa8775p-dpu
21 27
22 reg:28 reg:
23 items:29 items:
Documentation/devicetree/bindings/display/panel/ilitek,il79900a.yaml+68−0
@@ -0,0 +1,68 @@
1# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/panel/ilitek,il79900a.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Ilitek IL79900a based MIPI-DSI panels
8
9maintainers:
10 - Langyan Ye <yelangyan@huaqin.corp-partner.google.com>
11
12allOf:
13 - $ref: panel-common.yaml#
14
15properties:
16 compatible:
17 items:
18 - enum:
19 - tianma,tl121bvms07-00
20 - const: ilitek,il79900a
21
22 reg:
23 maxItems: 1
24 description: DSI virtual channel used by the panel
25
26 enable-gpios:
27 maxItems: 1
28 description: GPIO specifier for the enable pin
29
30 avdd-supply:
31 description: Positive analog voltage supply (AVDD)
32
33 avee-supply:
34 description: Negative analog voltage supply (AVEE)
35
36 pp1800-supply:
37 description: 1.8V logic voltage supply
38
39 backlight: true
40
41required:
42 - compatible
43 - reg
44 - enable-gpios
45 - avdd-supply
46 - avee-supply
47 - pp1800-supply
48
49additionalProperties: false
50
51examples:
52 - |
53 dsi {
54 #address-cells = <1>;
55 #size-cells = <0>;
56
57 panel@0 {
58 compatible = "tianma,tl121bvms07-00", "ilitek,il79900a";
59 reg = <0>;
60 enable-gpios = <&pio 25 0>;
61 avdd-supply = <&reg_avdd>;
62 avee-supply = <&reg_avee>;
63 pp1800-supply = <&reg_pp1800>;
64 backlight = <&backlight>;
65 };
66 };
67
68...
Documentation/devicetree/bindings/display/panel/ilitek,ili9881c.yaml+3−0
@@ -20,9 +20,11 @@ properties: · 19 unmodified lines
20 - bananapi,lhr050h4120 - bananapi,lhr050h41
21 - bestar,bsd1218-a101kl6821 - bestar,bsd1218-a101kl68
22 - feixin,k101-im2byl0222 - feixin,k101-im2byl02
23 - raspberrypi,dsi-5inch
23 - raspberrypi,dsi-7inch24 - raspberrypi,dsi-7inch
24 - startek,kd050hdfia02025 - startek,kd050hdfia020
25 - tdo,tl050hdv3526 - tdo,tl050hdv35
27 - wanchanglong,w552946aaa
26 - wanchanglong,w552946aba28 - wanchanglong,w552946aba
27 - const: ilitek,ili9881c29 - const: ilitek,ili9881c
28 30
@@ -30,6 +32,7 @@ properties: · 1 unmodified line
30 maxItems: 132 maxItems: 1
31 33
32 backlight: true34 backlight: true
35 port: true
33 power-supply: true36 power-supply: true
34 reset-gpios: true37 reset-gpios: true
35 rotation: true38 rotation: true
Documentation/devicetree/bindings/display/panel/lg,ld070wx3-sl01.yaml+60−0
@@ -0,0 +1,60 @@
1# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/panel/lg,ld070wx3-sl01.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: LG Corporation 7" WXGA TFT LCD panel
8
9maintainers:
10 - Svyatoslav Ryhel <clamor95@gmail.com>
11
12allOf:
13 - $ref: panel-common.yaml#
14
15properties:
16 compatible:
17 items:
18 - const: lg,ld070wx3-sl01
19
20 reg:
21 maxItems: 1
22
23 vdd-supply: true
24 vcc-supply: true
25
26 backlight: true
27 port: true
28
29required:
30 - compatible
31 - vdd-supply
32 - vcc-supply
33
34additionalProperties: false
35
36examples:
37 - |
38 #include <dt-bindings/gpio/gpio.h>
39
40 dsi {
41 #address-cells = <1>;
42 #size-cells = <0>;
43
44 panel@0 {
45 compatible = "lg,ld070wx3-sl01";
46 reg = <0>;
47
48 vdd-supply = <&vdd_3v3_lcd>;
49 vcc-supply = <&vcc_1v8_lcd>;
50
51 backlight = <&backlight>;
52
53 port {
54 endpoint {
55 remote-endpoint = <&dsi0_out>;
56 };
57 };
58 };
59 };
60...
Documentation/devicetree/bindings/display/panel/panel-lvds.yaml+2−0
@@ -59,6 +59,8 @@ properties: · 58 unmodified lines
59 # Jenson Display BL-JT60050-01A 7" WSVGA (1024x600) color TFT LCD LVDS panel59 # Jenson Display BL-JT60050-01A 7" WSVGA (1024x600) color TFT LCD LVDS panel
60 - jenson,bl-jt60050-01a60 - jenson,bl-jt60050-01a
61 - tbs,a711-panel61 - tbs,a711-panel
62 # Winstar WF70A8SYJHLNGA 7" WSVGA (1024x600) color TFT LCD LVDS panel
63 - winstar,wf70a8syjhlnga
62 64
63 - const: panel-lvds65 - const: panel-lvds
64 66
Documentation/devicetree/bindings/display/panel/panel-simple-dsi.yaml+4−26
@@ -19,6 +19,9 @@ description: | · 18 unmodified lines
19 19
20 If the panel is more advanced a dedicated binding file is required.20 If the panel is more advanced a dedicated binding file is required.
21 21
22allOf:
23 - $ref: panel-common.yaml#
24
22properties:25properties:
23 26
24 compatible:27 compatible:
@@ -42,8 +45,6 @@ properties: · 17 unmodified lines
42 - kingdisplay,kd097d0445 - kingdisplay,kd097d04
43 # LG ACX467AKM-7 4.95" 1080×1920 LCD Panel46 # LG ACX467AKM-7 4.95" 1080×1920 LCD Panel
44 - lg,acx467akm-747 - lg,acx467akm-7
45 # LG Corporation 7" WXGA TFT LCD panel
46 - lg,ld070wx3-sl01
47 # LG Corporation 5" HD TFT LCD panel48 # LG Corporation 5" HD TFT LCD panel
48 - lg,lh500wx1-sd0349 - lg,lh500wx1-sd03
49 # Lincoln LCD197 5" 1080x1920 LCD panel50 # Lincoln LCD197 5" 1080x1920 LCD panel
@@ -56,10 +57,6 @@ properties: · 6 unmodified lines
56 - panasonic,vvx10f034n0057 - panasonic,vvx10f034n00
57 # Samsung s6e3fa7 1080x2220 based AMS559NK06 AMOLED panel58 # Samsung s6e3fa7 1080x2220 based AMS559NK06 AMOLED panel
58 - samsung,s6e3fa7-ams559nk0659 - samsung,s6e3fa7-ams559nk06
59 # Samsung s6e3fc2x01 1080x2340 AMOLED panel
60 - samsung,s6e3fc2x01
61 # Samsung sofef00 1080x2280 AMOLED panel
62 - samsung,sofef00
63 # Shangai Top Display Optoelectronics 7" TL070WSH30 1024x600 TFT LCD panel60 # Shangai Top Display Optoelectronics 7" TL070WSH30 1024x600 TFT LCD panel
64 - tdo,tl070wsh3061 - tdo,tl070wsh30
65 62
@@ -72,31 +69,12 @@ properties: · 6 unmodified lines
72 reset-gpios: true69 reset-gpios: true
73 port: true70 port: true
74 power-supply: true71 power-supply: true
75 vddio-supply: true
76
77allOf:
78 - $ref: panel-common.yaml#
79 - if:
80 properties:
81 compatible:
82 enum:
83 - samsung,s6e3fc2x01
84 - samsung,sofef00
85 then:
86 properties:
87 power-supply: false
88 required:
89 - vddio-supply
90 else:
91 properties:
92 vddio-supply: false
93 required:
94 - power-supply
95 72
96additionalProperties: false73additionalProperties: false
97 74
98required:75required:
99 - compatible76 - compatible
77 - power-supply
100 - reg78 - reg
101 79
102examples:80examples:
Documentation/devicetree/bindings/display/panel/panel-simple.yaml+6−0
@@ -184,6 +184,8 @@ properties: · 183 unmodified lines
184 - innolux,n156bge-l21184 - innolux,n156bge-l21
185 # Innolux Corporation 7.0" WSVGA (1024x600) TFT LCD panel185 # Innolux Corporation 7.0" WSVGA (1024x600) TFT LCD panel
186 - innolux,zj070na-01p186 - innolux,zj070na-01p
187 # JuTouch Technology Co.. 10" JT101TM023 WXGA (1280 x 800) LVDS panel
188 - jutouch,jt101tm023
187 # Kaohsiung Opto-Electronics Inc. 5.7" QVGA (320 x 240) TFT LCD panel189 # Kaohsiung Opto-Electronics Inc. 5.7" QVGA (320 x 240) TFT LCD panel
188 - koe,tx14d24vm1bpa190 - koe,tx14d24vm1bpa
189 # Kaohsiung Opto-Electronics. TX31D200VM0BAA 12.3" HSXGA LVDS panel191 # Kaohsiung Opto-Electronics. TX31D200VM0BAA 12.3" HSXGA LVDS panel
@@ -268,6 +270,8 @@ properties: · 78 unmodified lines
268 - qiaodian,qd43003c0-40270 - qiaodian,qd43003c0-40
269 # Shenzhen QiShenglong Industrialist Co., Ltd. Gopher 2b 4.3" 480(RGB)x272 TFT LCD panel271 # Shenzhen QiShenglong Industrialist Co., Ltd. Gopher 2b 4.3" 480(RGB)x272 TFT LCD panel
270 - qishenglong,gopher2b-lcd272 - qishenglong,gopher2b-lcd
273 # Raystar Optronics, Inc. RFF500F-AWH-DNN 5.0" TFT 840x480
274 - raystar,rff500f-awh-dnn
271 # Rocktech Displays Ltd. RK101II01D-CT 10.1" TFT 1280x800275 # Rocktech Displays Ltd. RK101II01D-CT 10.1" TFT 1280x800
272 - rocktech,rk101ii01d-ct276 - rocktech,rk101ii01d-ct
273 # Rocktech Display Ltd. RK070ER9427 800(RGB)x480 TFT LCD panel277 # Rocktech Display Ltd. RK070ER9427 800(RGB)x480 TFT LCD panel
@@ -276,6 +280,8 @@ properties: · 2 unmodified lines
276 - rocktech,rk043fn48h280 - rocktech,rk043fn48h
277 # Samsung Electronics 10.1" WXGA (1280x800) TFT LCD panel281 # Samsung Electronics 10.1" WXGA (1280x800) TFT LCD panel
278 - samsung,ltl101al01282 - samsung,ltl101al01
283 # Samsung Electronics 10.6" FWXGA (1366x768) TFT LCD panel
284 - samsung,ltl106al01
279 # Samsung Electronics 10.1" WSVGA TFT LCD panel285 # Samsung Electronics 10.1" WSVGA TFT LCD panel
280 - samsung,ltn101nt05286 - samsung,ltn101nt05
281 # Satoz SAT050AT40H12R2 5.0" WVGA TFT LCD panel287 # Satoz SAT050AT40H12R2 5.0" WVGA TFT LCD panel
Documentation/devicetree/bindings/display/panel/ronbo,rb070d30.yaml+5−9
@@ -9,6 +9,9 @@ title: Ronbo RB070D30 DSI Display Panel · 8 unmodified lines
9maintainers:9maintainers:
10 - Maxime Ripard <mripard@kernel.org>10 - Maxime Ripard <mripard@kernel.org>
11 11
12allOf:
13 - $ref: panel-common.yaml#
14
12properties:15properties:
13 compatible:16 compatible:
14 const: ronbo,rb070d3017 const: ronbo,rb070d30
@@ -20,10 +23,6 @@ properties: · 5 unmodified lines
20 description: GPIO used for the power pin23 description: GPIO used for the power pin
21 maxItems: 124 maxItems: 1
22 25
23 reset-gpios:
24 description: GPIO used for the reset pin
25 maxItems: 1
26
27 shlr-gpios:26 shlr-gpios:
28 description: GPIO used for the shlr pin (horizontal flip)27 description: GPIO used for the shlr pin (horizontal flip)
29 maxItems: 128 maxItems: 1
@@ -35,10 +34,6 @@ properties: · 5 unmodified lines
35 vcc-lcd-supply:34 vcc-lcd-supply:
36 description: Power regulator35 description: Power regulator
37 36
38 backlight:
39 description: Backlight used by the panel
40 $ref: /schemas/types.yaml#/definitions/phandle
41
42required:37required:
43 - compatible38 - compatible
44 - power-gpios39 - power-gpios
@@ -47,5 +42,6 @@ required: · 2 unmodified lines
47 - shlr-gpios42 - shlr-gpios
48 - updn-gpios43 - updn-gpios
49 - vcc-lcd-supply44 - vcc-lcd-supply
45 - port
50 46
51additionalProperties: false47unevaluatedProperties: false
Documentation/devicetree/bindings/display/panel/samsung,atna33xc20.yaml+2−0
@@ -33,6 +33,8 @@ properties: · 32 unmodified lines
33 - samsung,atna45dc0233 - samsung,atna45dc02
34 # Samsung 15.6" 3K (2880x1620 pixels) eDP AMOLED panel34 # Samsung 15.6" 3K (2880x1620 pixels) eDP AMOLED panel
35 - samsung,atna56ac0335 - samsung,atna56ac03
36 # Samsung 16.0" 3K (2880x1800 pixels) eDP AMOLED panel
37 - samsung,atna60cl08
36 - const: samsung,atna33xc2038 - const: samsung,atna33xc20
37 39
38 enable-gpios: true40 enable-gpios: true
Documentation/devicetree/bindings/display/panel/samsung,s6e3fc2x01.yaml+81−0
@@ -0,0 +1,81 @@
1# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/panel/samsung,s6e3fc2x01.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Samsung S6E3FC2X01 AMOLED DDIC
8
9description: The S6E3FC2X01 is display driver IC with connected panel.
10
11maintainers:
12 - David Heidelberg <david@ixit.cz>
13
14allOf:
15 - $ref: panel-common.yaml#
16
17properties:
18 compatible:
19 items:
20 - enum:
21 # Samsung 6.41 inch, 1080x2340 pixels, 19.5:9 ratio
22 - samsung,s6e3fc2x01-ams641rw
23 - const: samsung,s6e3fc2x01
24
25 reg:
26 maxItems: 1
27
28 reset-gpios: true
29
30 port: true
31
32 vddio-supply:
33 description: VDD regulator
34
35 vci-supply:
36 description: VCI regulator
37
38 poc-supply:
39 description: POC regulator
40
41required:
42 - compatible
43 - reset-gpios
44 - vddio-supply
45 - vci-supply
46 - poc-supply
47
48unevaluatedProperties: false
49
50examples:
51 - |
52 #include <dt-bindings/gpio/gpio.h>
53
54 dsi {
55 #address-cells = <1>;
56 #size-cells = <0>;
57
58 panel@0 {
59 compatible = "samsung,s6e3fc2x01-ams641rw", "samsung,s6e3fc2x01";
60 reg = <0>;
61
62 vddio-supply = <&vreg_l14a_1p88>;
63 vci-supply = <&s2dos05_buck1>;
64 poc-supply = <&s2dos05_ldo1>;
65
66 te-gpios = <&tlmm 10 GPIO_ACTIVE_HIGH>;
67 reset-gpios = <&tlmm 6 GPIO_ACTIVE_HIGH>;
68
69 pinctrl-0 = <&sde_dsi_active &sde_te_active_sleep>;
70 pinctrl-1 = <&sde_dsi_suspend &sde_te_active_sleep>;
71 pinctrl-names = "default", "sleep";
72
73 port {
74 panel_in: endpoint {
75 remote-endpoint = <&mdss_dsi0_out>;
76 };
77 };
78 };
79 };
80
81...
Documentation/devicetree/bindings/display/panel/samsung,sofef00.yaml+79−0
@@ -0,0 +1,79 @@
1# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/panel/samsung,sofef00.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Samsung SOFEF00 AMOLED DDIC
8
9description: The SOFEF00 is display driver IC with connected panel.
10
11maintainers:
12 - David Heidelberg <david@ixit.cz>
13
14allOf:
15 - $ref: panel-common.yaml#
16
17properties:
18 compatible:
19 items:
20 - enum:
21 # Samsung 6.01 inch, 1080x2160 pixels, 18:9 ratio
22 - samsung,sofef00-ams601nt22
23 # Samsung 6.28 inch, 1080x2280 pixels, 19:9 ratio
24 - samsung,sofef00-ams628nw01
25 - const: samsung,sofef00
26
27 reg:
28 maxItems: 1
29
30 poc-supply:
31 description: POC regulator
32
33 vci-supply:
34 description: VCI regulator
35
36 vddio-supply:
37 description: VDD regulator
38
39required:
40 - compatible
41 - reset-gpios
42 - poc-supply
43 - vci-supply
44 - vddio-supply
45
46unevaluatedProperties: false
47
48examples:
49 - |
50 #include <dt-bindings/gpio/gpio.h>
51
52 dsi {
53 #address-cells = <1>;
54 #size-cells = <0>;
55
56 panel@0 {
57 compatible = "samsung,sofef00-ams628nw01", "samsung,sofef00";
58 reg = <0>;
59
60 vddio-supply = <&vreg_l14a_1p88>;
61 vci-supply = <&s2dos05_buck1>;
62 poc-supply = <&s2dos05_ldo1>;
63
64 te-gpios = <&tlmm 10 GPIO_ACTIVE_HIGH>;
65 reset-gpios = <&tlmm 6 GPIO_ACTIVE_HIGH>;
66
67 pinctrl-0 = <&panel_active>;
68 pinctrl-1 = <&panel_suspend>;
69 pinctrl-names = "default", "sleep";
70
71 port {
72 panel_in: endpoint {
73 remote-endpoint = <&mdss_dsi0_out>;
74 };
75 };
76 };
77 };
78
79...
Documentation/devicetree/bindings/display/panel/sharp,lq079l1sx01.yaml+99−0
@@ -0,0 +1,99 @@
1# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/panel/sharp,lq079l1sx01.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Sharp Microelectronics 7.9" WQXGA TFT LCD panel
8
9maintainers:
10 - Svyatoslav Ryhel <clamor95@gmail.com>
11
12description: >
13 This panel requires a dual-channel DSI host to operate and it supports
14 only left-right split mode, where each channel drives the left or right
15 half of the screen and only video mode.
16
17 Each of the DSI channels controls a separate DSI peripheral.
18 The peripheral driven by the first link (DSI-LINK1), left one, is
19 considered the primary peripheral and controls the device.
20
21allOf:
22 - $ref: panel-common-dual.yaml#
23
24properties:
25 compatible:
26 const: sharp,lq079l1sx01
27
28 reg:
29 maxItems: 1
30
31 avdd-supply:
32 description: regulator that supplies the analog voltage
33
34 vddio-supply:
35 description: regulator that supplies the I/O voltage
36
37 vsp-supply:
38 description: positive boost supply regulator
39
40 vsn-supply:
41 description: negative boost supply regulator
42
43 reset-gpios:
44 maxItems: 1
45
46 backlight: true
47 ports: true
48
49required:
50 - compatible
51 - reg
52 - avdd-supply
53 - vddio-supply
54 - ports
55
56additionalProperties: false
57
58examples:
59 - |
60 #include <dt-bindings/gpio/gpio.h>
61
62 dsi {
63 #address-cells = <1>;
64 #size-cells = <0>;
65
66 panel@0 {
67 compatible = "sharp,lq079l1sx01";
68 reg = <0>;
69
70 reset-gpios = <&gpio 59 GPIO_ACTIVE_LOW>;
71
72 avdd-supply = <&avdd_lcd>;
73 vddio-supply = <&vdd_lcd_io>;
74 vsp-supply = <&vsp_5v5_lcd>;
75 vsn-supply = <&vsn_5v5_lcd>;
76
77 backlight = <&backlight>;
78
79 ports {
80 #address-cells = <1>;
81 #size-cells = <0>;
82
83 port@0 {
84 reg = <0>;
85 panel_in0: endpoint {
86 remote-endpoint = <&dsi0_out>;
87 };
88 };
89
90 port@1 {
91 reg = <1>;
92 panel_in1: endpoint {
93 remote-endpoint = <&dsi1_out>;
94 };
95 };
96 };
97 };
98 };
99...
Documentation/devicetree/bindings/display/panel/synaptics,td4300-panel.yaml+89−0
@@ -0,0 +1,89 @@
1# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/display/panel/synaptics,td4300-panel.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Synaptics TDDI Display Panel Controller
8
9maintainers:
10 - Kaustabh Chakraborty <kauschluss@disroot.org>
11
12allOf:
13 - $ref: panel-common.yaml#
14
15properties:
16 compatible:
17 enum:
18 - syna,td4101-panel
19 - syna,td4300-panel
20
21 reg:
22 maxItems: 1
23
24 vio-supply:
25 description: core I/O voltage supply
26
27 vsn-supply:
28 description: negative voltage supply for analog circuits
29
30 vsp-supply:
31 description: positive voltage supply for analog circuits
32
33 backlight-gpios:
34 maxItems: 1
35 description: backlight enable GPIO
36
37 reset-gpios: true
38 width-mm: true
39 height-mm: true
40 panel-timing: true
41
42required:
43 - compatible
44 - reg
45 - width-mm
46 - height-mm
47 - panel-timing
48
49additionalProperties: false
50
51examples:
52 - |
53 #include <dt-bindings/gpio/gpio.h>
54
55 dsi {
56 #address-cells = <1>;
57 #size-cells = <0>;
58
59 panel@0 {
60 compatible = "syna,td4300-panel";
61 reg = <0>;
62
63 vio-supply = <&panel_vio_reg>;
64 vsn-supply = <&panel_vsn_reg>;
65 vsp-supply = <&panel_vsp_reg>;
66
67 backlight-gpios = <&gpd3 5 GPIO_ACTIVE_LOW>;
68 reset-gpios = <&gpd3 4 GPIO_ACTIVE_LOW>;
69
70 width-mm = <68>;
71 height-mm = <121>;
72
73 panel-timing {
74 clock-frequency = <144389520>;
75
76 hactive = <1080>;
77 hsync-len = <4>;
78 hfront-porch = <120>;
79 hback-porch = <32>;
80
81 vactive = <1920>;
82 vsync-len = <2>;
83 vfront-porch = <21>;
84 vback-porch = <4>;
85 };
86 };
87 };
88
89...
Documentation/devicetree/bindings/display/renesas,rzg2l-du.yaml+3−0
@@ -25,6 +25,9 @@ properties: · 24 unmodified lines
25 - enum:25 - enum:
26 - renesas,r9a07g054-du # RZ/V2L26 - renesas,r9a07g054-du # RZ/V2L
27 - const: renesas,r9a07g044-du # RZ/G2L fallback27 - const: renesas,r9a07g044-du # RZ/G2L fallback
28 - items:
29 - const: renesas,r9a09g056-du # RZ/V2N
30 - const: renesas,r9a09g057-du # RZ/V2H(P) fallback
28 31
29 reg:32 reg:
30 maxItems: 133 maxItems: 1
Documentation/devicetree/bindings/display/rockchip/rockchip,dw-mipi-dsi.yaml+2−0
@@ -17,6 +17,7 @@ properties: · 16 unmodified lines
17 - rockchip,px30-mipi-dsi17 - rockchip,px30-mipi-dsi
18 - rockchip,rk3128-mipi-dsi18 - rockchip,rk3128-mipi-dsi
19 - rockchip,rk3288-mipi-dsi19 - rockchip,rk3288-mipi-dsi
20 - rockchip,rk3368-mipi-dsi
20 - rockchip,rk3399-mipi-dsi21 - rockchip,rk3399-mipi-dsi
21 - rockchip,rk3568-mipi-dsi22 - rockchip,rk3568-mipi-dsi
22 - rockchip,rv1126-mipi-dsi23 - rockchip,rv1126-mipi-dsi
@@ -73,6 +74,7 @@ allOf: · 50 unmodified lines
73 enum:74 enum:
74 - rockchip,px30-mipi-dsi75 - rockchip,px30-mipi-dsi
75 - rockchip,rk3128-mipi-dsi76 - rockchip,rk3128-mipi-dsi
77 - rockchip,rk3368-mipi-dsi
76 - rockchip,rk3568-mipi-dsi78 - rockchip,rk3568-mipi-dsi
77 - rockchip,rv1126-mipi-dsi79 - rockchip,rv1126-mipi-dsi
78 80
Documentation/devicetree/bindings/display/rockchip/rockchip,rk3588-dw-hdmi-qp.yaml+11−0
@@ -113,6 +113,14 @@ properties: · 112 unmodified lines
113 description:113 description:
114 Additional HDMI QP related data is accessed through VO GRF regs.114 Additional HDMI QP related data is accessed through VO GRF regs.
115 115
116 frl-enable-gpios:
117 description:
118 Optional GPIO line to be asserted when operating in HDMI 2.1 FRL mode and
119 deasserted for HDMI 1.4/2.0 TMDS. It can be used to control external
120 voltage bias for HDMI data lines. When not present the HDMI encoder will
121 operate in TMDS mode only.
122 maxItems: 1
123
116required:124required:
117 - compatible125 - compatible
118 - reg126 - reg
@@ -132,8 +140,10 @@ unevaluatedProperties: false · 13 unmodified lines
132examples:140examples:
133 - |141 - |
134 #include <dt-bindings/clock/rockchip,rk3588-cru.h>142 #include <dt-bindings/clock/rockchip,rk3588-cru.h>
143 #include <dt-bindings/gpio/gpio.h>
135 #include <dt-bindings/interrupt-controller/arm-gic.h>144 #include <dt-bindings/interrupt-controller/arm-gic.h>
136 #include <dt-bindings/interrupt-controller/irq.h>145 #include <dt-bindings/interrupt-controller/irq.h>
146 #include <dt-bindings/pinctrl/rockchip.h>
137 #include <dt-bindings/power/rk3588-power.h>147 #include <dt-bindings/power/rk3588-power.h>
138 #include <dt-bindings/reset/rockchip,rk3588-cru.h>148 #include <dt-bindings/reset/rockchip,rk3588-cru.h>
139 149
@@ -164,6 +174,7 @@ examples: · 24 unmodified lines
164 rockchip,grf = <&sys_grf>;174 rockchip,grf = <&sys_grf>;
165 rockchip,vo-grf = <&vo1_grf>;175 rockchip,vo-grf = <&vo1_grf>;
166 #sound-dai-cells = <0>;176 #sound-dai-cells = <0>;
177 frl-enable-gpios = <&gpio4 RK_PB1 GPIO_ACTIVE_LOW>;
167 178
168 ports {179 ports {
169 #address-cells = <1>;180 #address-cells = <1>;
Documentation/devicetree/bindings/gpu/arm,mali-valhall-csf.yaml+40−2
@@ -18,6 +18,8 @@ properties: · 17 unmodified lines
18 oneOf:18 oneOf:
19 - items:19 - items:
20 - enum:20 - enum:
21 - mediatek,mt8196-mali
22 - nxp,imx95-mali # G310
21 - rockchip,rk3588-mali23 - rockchip,rk3588-mali
22 - const: arm,mali-valhall-csf # Mali Valhall GPU model/revision is fully discoverable24 - const: arm,mali-valhall-csf # Mali Valhall GPU model/revision is fully discoverable
23 25
@@ -44,7 +46,9 @@ properties: · 20 unmodified lines
44 minItems: 146 minItems: 1
45 items:47 items:
46 - const: core48 - const: core
47 - const: coregroup49 - enum:
50 - coregroup
51 - stacks
48 - const: stacks52 - const: stacks
49 53
50 mali-supply: true54 mali-supply: true
@@ -91,7 +95,6 @@ required: · 40 unmodified lines
91 - interrupts95 - interrupts
92 - interrupt-names96 - interrupt-names
93 - clocks97 - clocks
94 - mali-supply
95 98
96additionalProperties: false99additionalProperties: false
97 100
@@ -108,6 +111,29 @@ allOf: · 10 unmodified lines
108 power-domains:111 power-domains:
109 maxItems: 1112 maxItems: 1
110 power-domain-names: false113 power-domain-names: false
114 required:
115 - mali-supply
116 - if:
117 properties:
118 compatible:
119 contains:
120 const: mediatek,mt8196-mali
121 then:
122 properties:
123 mali-supply: false
124 sram-supply: false
125 operating-points-v2: false
126 power-domains:
127 maxItems: 1
128 power-domain-names: false
129 clocks:
130 maxItems: 2
131 clock-names:
132 items:
133 - const: core
134 - const: stacks
135 required:
136 - power-domains
111 137
112examples:138examples:
113 - |139 - |
@@ -143,5 +169,17 @@ examples: · 29 unmodified lines
143 };169 };
144 };170 };
145 };171 };
172 - |
173 gpu@48000000 {
174 compatible = "mediatek,mt8196-mali", "arm,mali-valhall-csf";
175 reg = <0x48000000 0x480000>;
176 clocks = <&gpufreq 0>, <&gpufreq 1>;
177 clock-names = "core", "stacks";
178 interrupts = <GIC_SPI 606 IRQ_TYPE_LEVEL_HIGH 0>,
179 <GIC_SPI 605 IRQ_TYPE_LEVEL_HIGH 0>,
180 <GIC_SPI 604 IRQ_TYPE_LEVEL_HIGH 0>;
181 interrupt-names = "job", "mmu", "gpu";
182 power-domains = <&gpufreq>;
183 };
146 184
147...185...
Documentation/devicetree/bindings/gpu/img,powervr-rogue.yaml+43−22
@@ -13,6 +13,16 @@ maintainers: · 12 unmodified lines
13properties:13properties:
14 compatible:14 compatible:
15 oneOf:15 oneOf:
16 - items:
17 - enum:
18 - renesas,r8a7796-gpu
19 - renesas,r8a77961-gpu
20 - const: img,img-gx6250
21 - const: img,img-rogue
22 - items:
23 - const: renesas,r8a77965-gpu
24 - const: img,img-ge7800
25 - const: img,img-rogue
16 - items:26 - items:
17 - enum:27 - enum:
18 - ti,am62-gpu28 - ti,am62-gpu
@@ -86,48 +96,56 @@ allOf: · 67 unmodified lines
86 properties:96 properties:
87 compatible:97 compatible:
88 contains:98 contains:
89 const: img,img-axe-1-16m99 enum:
100 - ti,am62-gpu
101 - ti,j721s2-gpu
90 then:102 then:
91 properties:103 properties:
92 power-domains:104 clocks:
93 items:
94 - description: Power domain A
95 power-domain-names:
96 maxItems: 1105 maxItems: 1
97 required:
98 - power-domains
99 - power-domain-names
100 106
101 - if:107 - if:
102 properties:108 properties:
103 compatible:109 compatible:
104 contains:110 contains:
105 const: thead,th1520-gpu111 enum:
112 - img,img-ge7800
113 - img,img-gx6250
114 - thead,th1520-gpu
106 then:115 then:
107 properties:116 properties:
108 clocks:117 clocks:
109 minItems: 3118 minItems: 3
110 clock-names:119 clock-names:
111 minItems: 3120 minItems: 3
121
122 - if:
123 properties:
124 compatible:
125 contains:
126 const: img,img-axe-1-16m
127 then:
128 properties:
112 power-domains:129 power-domains:
113 items:130 maxItems: 1
114 - description: The single, unified power domain for the GPU on the131 power-domain-names:
115 TH1520 SoC, integrating all internal IP power domains.132 maxItems: 1
116 power-domain-names: false
117 required:133 required:
118 - power-domains134 - power-domains
135 - power-domain-names
119 136
120 - if:137 - if:
121 properties:138 properties:
122 compatible:139 compatible:
123 contains:140 contains:
124 const: img,img-bxs-4-64141 enum:
142 - img,img-bxs-4-64
143 - img,img-ge7800
144 - img,img-gx6250
125 then:145 then:
126 properties:146 properties:
127 power-domains:147 power-domains:
128 items:148 minItems: 2
129 - description: Power domain A
130 - description: Power domain B
131 power-domain-names:149 power-domain-names:
132 minItems: 2150 minItems: 2
133 required:151 required:
@@ -138,13 +156,16 @@ allOf: · 4 unmodified lines
138 properties:156 properties:
139 compatible:157 compatible:
140 contains:158 contains:
141 enum:159 const: thead,th1520-gpu
142 - ti,am62-gpu
143 - ti,j721s2-gpu
144 then:160 then:
145 properties:161 properties:
146 clocks:162 power-domains:
147 maxItems: 1163 items:
164 - description: The single, unified power domain for the GPU on the
165 TH1520 SoC, integrating all internal IP power domains.
166 power-domain-names: false
167 required:
168 - power-domains
148 169
149examples:170examples:
150 - |171 - |
Documentation/devicetree/bindings/iommu/arm,smmu.yaml+2−0
@@ -89,6 +89,8 @@ properties: · 88 unmodified lines
89 - description: Qcom Adreno GPUs implementing "qcom,smmu-500" and "arm,mmu-500"89 - description: Qcom Adreno GPUs implementing "qcom,smmu-500" and "arm,mmu-500"
90 items:90 items:
91 - enum:91 - enum:
92 - qcom,glymur-smmu-500
93 - qcom,kaanapali-smmu-500
92 - qcom,milos-smmu-50094 - qcom,milos-smmu-500
93 - qcom,qcm2290-smmu-50095 - qcom,qcm2290-smmu-500
94 - qcom,qcs615-smmu-50096 - qcom,qcs615-smmu-500
Documentation/devicetree/bindings/npu/arm,ethos.yaml+79−0
@@ -0,0 +1,79 @@
1# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
2%YAML 1.2
3---
4$id: http://devicetree.org/schemas/npu/arm,ethos.yaml#
5$schema: http://devicetree.org/meta-schemas/core.yaml#
6
7title: Arm Ethos U65/U85
8
9maintainers:
10 - Rob Herring <robh@kernel.org>
11
12description: >
13 The Arm Ethos-U NPUs are designed for IoT inference applications. The NPUs
14 can accelerate 8-bit and 16-bit integer quantized networks:
15
16 Transformer networks (U85 only)
17 Convolutional Neural Networks (CNN)
18 Recurrent Neural Networks (RNN)
19
20 Further documentation is available here:
21
22 U65 TRM: https://developer.arm.com/documentation/102023/
23 U85 TRM: https://developer.arm.com/documentation/102685/
24
25properties:
26 compatible:
27 oneOf:
28 - items:
29 - enum:
30 - fsl,imx93-npu
31 - const: arm,ethos-u65
32 - items:
33 - {}
34 - const: arm,ethos-u85
35
36 reg:
37 maxItems: 1
38
39 interrupts:
40 maxItems: 1
41
42 clocks:
43 maxItems: 2
44
45 clock-names:
46 items:
47 - const: core
48 - const: apb
49
50 power-domains:
51 maxItems: 1
52
53 sram:
54 maxItems: 1
55
56required:
57 - compatible
58 - reg
59 - interrupts
60 - clocks
61
62additionalProperties: false
63
64examples:
65 - |
66 #include <dt-bindings/interrupt-controller/irq.h>
67 #include <dt-bindings/interrupt-controller/arm-gic.h>
68 #include <dt-bindings/clock/imx93-clock.h>
69
70 npu@4a900000 {
71 compatible = "fsl,imx93-npu", "arm,ethos-u65";
72 reg = <0x4a900000 0x1000>;
73 interrupts = <GIC_SPI 178 IRQ_TYPE_LEVEL_HIGH>;
74 power-domains = <&mlmix>;
75 clocks = <&clk IMX93_CLK_ML>, <&clk IMX93_CLK_ML_APB>;
76 clock-names = "core", "apb";
77 sram = <&sram>;
78 };
79...
Documentation/devicetree/bindings/vendor-prefixes.yaml+6−0
@@ -176,6 +176,8 @@ patternProperties: · 175 unmodified lines
176 description: All Sensors Corporation176 description: All Sensors Corporation
177 "^asix,.*":177 "^asix,.*":
178 description: ASIX Electronics Corporation178 description: ASIX Electronics Corporation
179 "^asl-tek,.*":
180 description: ASL Xiamen Technology Co., Ltd.
179 "^aspeed,.*":181 "^aspeed,.*":
180 description: ASPEED Technology Inc.182 description: ASPEED Technology Inc.
181 "^asrock,.*":183 "^asrock,.*":
@@ -835,6 +837,8 @@ patternProperties: · 653 unmodified lines
835 description: JOZ BV837 description: JOZ BV
836 "^jty,.*":838 "^jty,.*":
837 description: JTY839 description: JTY
840 "^jutouch,.*":
841 description: JuTouch Technology Co., Ltd.
838 "^kam,.*":842 "^kam,.*":
839 description: Kamstrup A/S843 description: Kamstrup A/S
840 "^karo,.*":844 "^karo,.*":
@@ -1323,6 +1327,8 @@ patternProperties: · 482 unmodified lines
1323 description: Raumfeld GmbH1327 description: Raumfeld GmbH
1324 "^raydium,.*":1328 "^raydium,.*":
1325 description: Raydium Semiconductor Corp.1329 description: Raydium Semiconductor Corp.
1330 "^raystar,.*":
1331 description: Raystar Optronics, Inc.
1326 "^rda,.*":1332 "^rda,.*":
1327 description: Unisoc Communications, Inc.1333 description: Unisoc Communications, Inc.
1328 "^realtek,.*":1334 "^realtek,.*":
Documentation/gpu/drm-kms-helpers.rst+12−0
@@ -92,6 +92,18 @@ GEM Atomic Helper Reference · 91 unmodified lines
92.. kernel-doc:: drivers/gpu/drm/drm_gem_atomic_helper.c92.. kernel-doc:: drivers/gpu/drm/drm_gem_atomic_helper.c
93 :export:93 :export:
94 94
95VBLANK Helper Reference
96-----------------------
97
98.. kernel-doc:: drivers/gpu/drm/drm_vblank_helper.c
99 :doc: overview
100
101.. kernel-doc:: include/drm/drm_vblank_helper.h
102 :internal:
103
104.. kernel-doc:: drivers/gpu/drm/drm_vblank_helper.c
105 :export:
106
95Simple KMS Helper Reference107Simple KMS Helper Reference
96===========================108===========================
97 109
Documentation/gpu/drm-kms.rst+15−0
@@ -413,6 +413,21 @@ Plane Panic Functions Reference · 412 unmodified lines
413.. kernel-doc:: drivers/gpu/drm/drm_panic.c413.. kernel-doc:: drivers/gpu/drm/drm_panic.c
414 :export:414 :export:
415 415
416Colorop Abstraction
417===================
418
419.. kernel-doc:: drivers/gpu/drm/drm_colorop.c
420 :doc: overview
421
422Colorop Functions Reference
423---------------------------
424
425.. kernel-doc:: include/drm/drm_colorop.h
426 :internal:
427
428.. kernel-doc:: drivers/gpu/drm/drm_colorop.c
429 :export:
430
416Display Modes Function Reference431Display Modes Function Reference
417================================432================================
418 433
Documentation/gpu/nova/core/todo.rst+0−30
@@ -44,25 +44,6 @@ automatically generates the corresponding mappings between a value and a number. · 43 unmodified lines
44| Complexity: Beginner44| Complexity: Beginner
45| Link: https://docs.rs/num/latest/num/trait.FromPrimitive.html45| Link: https://docs.rs/num/latest/num/trait.FromPrimitive.html
46 46
47Conversion from byte slices for types implementing FromBytes [TRSM]
48-------------------------------------------------------------------
49
50We retrieve several structures from byte streams coming from the BIOS or loaded
51firmware. At the moment converting the bytes slice into the proper type require
52an inelegant `unsafe` operation; this will go away once `FromBytes` implements
53a proper `from_bytes` method.
54
55| Complexity: Beginner
56
57CoherentAllocation improvements [COHA]
58--------------------------------------
59
60`CoherentAllocation` needs a safe way to write into the allocation, and to
61obtain slices within the allocation.
62
63| Complexity: Beginner
64| Contact: Abdiel Janulgue
65
66Generic register abstraction [REGA]47Generic register abstraction [REGA]
67-----------------------------------48-----------------------------------
68 49
@@ -153,17 +134,6 @@ A `num` core kernel module is being designed to provide these operations. · 84 unmodified lines
153| Complexity: Intermediate134| Complexity: Intermediate
154| Contact: Alexandre Courbot135| Contact: Alexandre Courbot
155 136
156Delay / Sleep abstractions [DLAY]
157---------------------------------
158
159Rust abstractions for the kernel's delay() and sleep() functions.
160
161FUJITA Tomonori plans to work on abstractions for read_poll_timeout_atomic()
162(and friends) [1].
163
164| Complexity: Beginner
165| Link: https://lore.kernel.org/netdev/20250228.080550.354359820929821928.fujita.tomonori@gmail.com/ [1]
166
167IRQ abstractions137IRQ abstractions
168----------------138----------------
169 139
Documentation/gpu/rfc/color_pipeline.rst+378−0
@@ -0,0 +1,378 @@
1.. SPDX-License-Identifier: GPL-2.0
2
3========================
4Linux Color Pipeline API
5========================
6
7What problem are we solving?
8============================
9
10We would like to support pre-, and post-blending complex color
11transformations in display controller hardware in order to allow for
12HW-supported HDR use-cases, as well as to provide support to
13color-managed applications, such as video or image editors.
14
15It is possible to support an HDR output on HW supporting the Colorspace
16and HDR Metadata drm_connector properties, but that requires the
17compositor or application to render and compose the content into one
18final buffer intended for display. Doing so is costly.
19
20Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
21operations to support color transformations. These operations are often
22implemented in fixed-function HW and therefore much more power efficient than
23performing similar operations via shaders or CPU.
24
25We would like to make use of this HW functionality to support complex color
26transformations with no, or minimal CPU or shader load. The switch between HW
27fixed-function blocks and shaders/CPU must be seamless with no visible
28difference when fallback to shaders/CPU is neceesary at any time.
29
30
31How are other OSes solving this problem?
32========================================
33
34The most widely supported use-cases regard HDR content, whether video or
35gaming.
36
37Most OSes will specify the source content format (color gamut, encoding transfer
38function, and other metadata, such as max and average light levels) to a driver.
39Drivers will then program their fixed-function HW accordingly to map from a
40source content buffer's space to a display's space.
41
42When fixed-function HW is not available the compositor will assemble a shader to
43ask the GPU to perform the transformation from the source content format to the
44display's format.
45
46A compositor's mapping function and a driver's mapping function are usually
47entirely separate concepts. On OSes where a HW vendor has no insight into
48closed-source compositor code such a vendor will tune their color management
49code to visually match the compositor's. On other OSes, where both mapping
50functions are open to an implementer they will ensure both mappings match.
51
52This results in mapping algorithm lock-in, meaning that no-one alone can
53experiment with or introduce new mapping algorithms and achieve
54consistent results regardless of which implementation path is taken.
55
56Why is Linux different?
57=======================
58
59Unlike other OSes, where there is one compositor for one or more drivers, on
60Linux we have a many-to-many relationship. Many compositors; many drivers.
61In addition each compositor vendor or community has their own view of how
62color management should be done. This is what makes Linux so beautiful.
63
64This means that a HW vendor can now no longer tune their driver to one
65compositor, as tuning it to one could make it look fairly different from
66another compositor's color mapping.
67
68We need a better solution.
69
70
71Descriptive API
72===============
73
74An API that describes the source and destination colorspaces is a descriptive
75API. It describes the input and output color spaces but does not describe
76how precisely they should be mapped. Such a mapping includes many minute
77design decision that can greatly affect the look of the final result.
78
79It is not feasible to describe such mapping with enough detail to ensure the
80same result from each implementation. In fact, these mappings are a very active
81research area.
82
83
84Prescriptive API
85================
86
87A prescriptive API describes not the source and destination colorspaces. It
88instead prescribes a recipe for how to manipulate pixel values to arrive at the
89desired outcome.
90
91This recipe is generally an ordered list of straight-forward operations,
92with clear mathematical definitions, such as 1D LUTs, 3D LUTs, matrices,
93or other operations that can be described in a precise manner.
94
95
96The Color Pipeline API
97======================
98
99HW color management pipelines can significantly differ between HW
100vendors in terms of availability, ordering, and capabilities of HW
101blocks. This makes a common definition of color management blocks and
102their ordering nigh impossible. Instead we are defining an API that
103allows user space to discover the HW capabilities in a generic manner,
104agnostic of specific drivers and hardware.
105
106
107drm_colorop Object
108==================
109
110To support the definition of color pipelines we define the DRM core
111object type drm_colorop. Individual drm_colorop objects will be chained
112via the NEXT property of a drm_colorop to constitute a color pipeline.
113Each drm_colorop object is unique, i.e., even if multiple color
114pipelines have the same operation they won't share the same drm_colorop
115object to describe that operation.
116
117Note that drivers are not expected to map drm_colorop objects statically
118to specific HW blocks. The mapping of drm_colorop objects is entirely a
119driver-internal detail and can be as dynamic or static as a driver needs
120it to be. See more in the Driver Implementation Guide section below.
121
122Each drm_colorop has three core properties:
123
124TYPE: An enumeration property, defining the type of transformation, such as
125* enumerated curve
126* custom (uniform) 1D LUT
127* 3x3 matrix
128* 3x4 matrix
129* 3D LUT
130* etc.
131
132Depending on the type of transformation other properties will describe
133more details.
134
135BYPASS: A boolean property that can be used to easily put a block into
136bypass mode. The BYPASS property is not mandatory for a colorop, as long
137as the entire pipeline can get bypassed by setting the COLOR_PIPELINE on
138a plane to '0'.
139
140NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
141drm_colorop is the last in the chain.
142
143An example of a drm_colorop object might look like one of these::
144
145 /* 1D enumerated curve */
146 Color operation 42
147 ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3 matrix, 3x4 matrix, 3D LUT, etc.} = 1D enumerated curve
148 ├─ "BYPASS": bool {true, false}
149 ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, sRGB inverse EOTF, PQ EOTF, PQ inverse EOTF, …}
150 └─ "NEXT": immutable color operation ID = 43
151
152 /* custom 4k entry 1D LUT */
153 Color operation 52
154 ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3 matrix, 3x4 matrix, 3D LUT, etc.} = 1D LUT
155 ├─ "BYPASS": bool {true, false}
156 ├─ "SIZE": immutable range = 4096
157 ├─ "DATA": blob
158 └─ "NEXT": immutable color operation ID = 0
159
160 /* 17^3 3D LUT */
161 Color operation 72
162 ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3 matrix, 3x4 matrix, 3D LUT, etc.} = 3D LUT
163 ├─ "BYPASS": bool {true, false}
164 ├─ "SIZE": immutable range = 17
165 ├─ "DATA": blob
166 └─ "NEXT": immutable color operation ID = 73
167
168drm_colorop extensibility
169-------------------------
170
171Unlike existing DRM core objects, like &drm_plane, drm_colorop is not
172extensible. This simplifies implementations and keeps all functionality
173for managing &drm_colorop objects in the DRM core.
174
175If there is a need one may introduce a simple &drm_colorop_funcs
176function table in the future, for example to support an IN_FORMATS
177property on a &drm_colorop.
178
179If a driver requires the ability to create a driver-specific colorop
180object they will need to add &drm_colorop func table support with
181support for the usual functions, like destroy, atomic_duplicate_state,
182and atomic_destroy_state.
183
184
185COLOR_PIPELINE Plane Property
186=============================
187
188Color Pipelines are created by a driver and advertised via a new
189COLOR_PIPELINE enum property on each plane. Values of the property
190always include object id 0, which is the default and means all color
191processing is disabled. Additional values will be the object IDs of the
192first drm_colorop in a pipeline. A driver can create and advertise none,
193one, or more possible color pipelines. A DRM client will select a color
194pipeline by setting the COLOR PIPELINE to the respective value.
195
196NOTE: Many DRM clients will set enumeration properties via the string
197value, often hard-coding it. Since this enumeration is generated based
198on the colorop object IDs it is important to perform the Color Pipeline
199Discovery, described below, instead of hard-coding color pipeline
200assignment. Drivers might generate the enum strings dynamically.
201Hard-coded strings might only work for specific drivers on a specific
202pieces of HW. Color Pipeline Discovery can work universally, as long as
203drivers implement the required color operations.
204
205The COLOR_PIPELINE property is only exposed when the
206DRM_CLIENT_CAP_PLANE_COLOR_PIPELINE is set. Drivers shall ignore any
207existing pre-blend color operations when this cap is set, such as
208COLOR_RANGE and COLOR_ENCODING. If drivers want to support COLOR_RANGE
209or COLOR_ENCODING functionality when the color pipeline client cap is
210set, they are expected to expose colorops in the pipeline to allow for
211the appropriate color transformation.
212
213Setting of the COLOR_PIPELINE plane property or drm_colorop properties
214is only allowed for userspace that sets this client cap.
215
216An example of a COLOR_PIPELINE property on a plane might look like this::
217
218 Plane 10
219 ├─ "TYPE": immutable enum {Overlay, Primary, Cursor} = Primary
220 ├─ …
221 └─ "COLOR_PIPELINE": enum {0, 42, 52} = 0
222
223
224Color Pipeline Discovery
225========================
226
227A DRM client wanting color management on a drm_plane will:
228
2291. Get the COLOR_PIPELINE property of the plane
2302. iterate all COLOR_PIPELINE enum values
2313. for each enum value walk the color pipeline (via the NEXT pointers)
232 and see if the available color operations are suitable for the
233 desired color management operations
234
235If userspace encounters an unknown or unsuitable color operation during
236discovery it does not need to reject the entire color pipeline outright,
237as long as the unknown or unsuitable colorop has a "BYPASS" property.
238Drivers will ensure that a bypassed block does not have any effect.
239
240An example of chained properties to define an AMD pre-blending color
241pipeline might look like this::
242
243 Plane 10
244 ├─ "TYPE" (immutable) = Primary
245 └─ "COLOR_PIPELINE": enum {0, 44} = 0
246
247 Color operation 44
248 ├─ "TYPE" (immutable) = 1D enumerated curve
249 ├─ "BYPASS": bool
250 ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, PQ EOTF} = sRGB EOTF
251 └─ "NEXT" (immutable) = 45
252
253 Color operation 45
254 ├─ "TYPE" (immutable) = 3x4 Matrix
255 ├─ "BYPASS": bool
256 ├─ "DATA": blob
257 └─ "NEXT" (immutable) = 46
258
259 Color operation 46
260 ├─ "TYPE" (immutable) = 1D enumerated curve
261 ├─ "BYPASS": bool
262 ├─ "CURVE_1D_TYPE": enum {sRGB Inverse EOTF, PQ Inverse EOTF} = sRGB EOTF
263 └─ "NEXT" (immutable) = 47
264
265 Color operation 47
266 ├─ "TYPE" (immutable) = 1D LUT
267 ├─ "SIZE": immutable range = 4096
268 ├─ "DATA": blob
269 └─ "NEXT" (immutable) = 48
270
271 Color operation 48
272 ├─ "TYPE" (immutable) = 3D LUT
273 ├─ "DATA": blob
274 └─ "NEXT" (immutable) = 49
275
276 Color operation 49
277 ├─ "TYPE" (immutable) = 1D enumerated curve
278 ├─ "BYPASS": bool
279 ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, PQ EOTF} = sRGB EOTF
280 └─ "NEXT" (immutable) = 0
281
282
283Color Pipeline Programming
284==========================
285
286Once a DRM client has found a suitable pipeline it will:
287
2881. Set the COLOR_PIPELINE enum value to the one pointing at the first
289 drm_colorop object of the desired pipeline
2902. Set the properties for all drm_colorop objects in the pipeline to the
291 desired values, setting BYPASS to true for unused drm_colorop blocks,
292 and false for enabled drm_colorop blocks
2933. Perform (TEST_ONLY or not) atomic commit with all the other KMS
294 states it wishes to change
295
296To configure the pipeline for an HDR10 PQ plane and blending in linear
297space, a compositor might perform an atomic commit with the following
298property values::
299
300 Plane 10
301 └─ "COLOR_PIPELINE" = 42
302
303 Color operation 42
304 └─ "BYPASS" = true
305
306 Color operation 44
307 └─ "BYPASS" = true
308
309 Color operation 45
310 └─ "BYPASS" = true
311
312 Color operation 46
313 └─ "BYPASS" = true
314
315 Color operation 47
316 ├─ "DATA" = Gamut mapping + tone mapping + night mode
317 └─ "BYPASS" = false
318
319 Color operation 48
320 ├─ "CURVE_1D_TYPE" = PQ EOTF
321 └─ "BYPASS" = false
322
323
324Driver Implementer's Guide
325==========================
326
327What does this all mean for driver implementations? As noted above the
328colorops can map to HW directly but don't need to do so. Here are some
329suggestions on how to think about creating your color pipelines:
330
331- Try to expose pipelines that use already defined colorops, even if
332 your hardware pipeline is split differently. This allows existing
333 userspace to immediately take advantage of the hardware.
334
335- Additionally, try to expose your actual hardware blocks as colorops.
336 Define new colorop types where you believe it can offer significant
337 benefits if userspace learns to program them.
338
339- Avoid defining new colorops for compound operations with very narrow
340 scope. If you have a hardware block for a special operation that
341 cannot be split further, you can expose that as a new colorop type.
342 However, try to not define colorops for "use cases", especially if
343 they require you to combine multiple hardware blocks.
344
345- Design new colorops as prescriptive, not descriptive; by the
346 mathematical formula, not by the assumed input and output.
347
348A defined colorop type must be deterministic. The exact behavior of the
349colorop must be documented entirely, whether via a mathematical formula
350or some other description. Its operation can depend only on its
351properties and input and nothing else, allowed error tolerance
352notwithstanding.
353
354
355Driver Forward/Backward Compatibility
356=====================================
357
358As this is uAPI drivers can't regress color pipelines that have been
359introduced for a given HW generation. New HW generations are free to
360abandon color pipelines advertised for previous generations.
361Nevertheless, it can be beneficial to carry support for existing color
362pipelines forward as those will likely already have support in DRM
363clients.
364
365Introducing new colorops to a pipeline is fine, as long as they can be
366bypassed or are purely informational. DRM clients implementing support
367for the pipeline can always skip unknown properties as long as they can
368be confident that doing so will not cause unexpected results.
369
370If a new colorop doesn't fall into one of the above categories
371(bypassable or informational) the modified pipeline would be unusable
372for user space. In this case a new pipeline should be defined.
373
374
375References
376==========
377
3781. https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
Documentation/gpu/rfc/index.rst+3−0
@@ -35,3 +35,6 @@ host such documentation: · 34 unmodified lines
35.. toctree::35.. toctree::
36 36
37 i915_vm_bind.rst37 i915_vm_bind.rst
38
39.. toctree::
40 color_pipeline.rst
Documentation/gpu/todo.rst+37−0
@@ -623,6 +623,43 @@ Contact: Thomas Zimmermann <tzimmermann@suse.de>, Simona Vetter · 622 unmodified lines
623 623
624Level: Advanced624Level: Advanced
625 625
626Implement a new DUMB_CREATE2 ioctl
627----------------------------------
628
629The current DUMB_CREATE ioctl is not well defined. Instead of a pixel and
630framebuffer format, it only accepts a color mode of vague semantics. Assuming
631a linear framebuffer, the color mode gives an idea of the supported pixel
632format. But userspace effectively has to guess the correct values. It really
633only works reliably with framebuffers in XRGB8888. Userspace has begun to
634workaround these limitations by computing arbitrary format's buffer sizes and
635calculating their sizes in terms of XRGB8888 pixels.
636
637One possible solution is a new ioctl DUMB_CREATE2. It should accept a DRM
638format and a format modifier to resolve the color mode's ambiguity. As
639framebuffers can be multi-planar, the new ioctl has to return the buffer size,
640pitch and GEM handle for each individual color plane.
641
642In the first step, the new ioctl can be limited to the current features of
643the existing DUMB_CREATE. Individual drivers can then be extended to support
644multi-planar formats. Rockchip might require this and would be a good candidate.
645
646It might also be helpful to userspace to query information about the size of
647a potential buffer, if allocated. Userspace would supply geometry and format;
648the kernel would return minimal allocation sizes and scanline pitch. There is
649interest to allocate that memory from another device and provide it to the
650DRM driver (say via dma-buf).
651
652Another requested feature is the ability to allocate a buffer by size, without
653format. Accelators use this for their buffer allocation and it could likely be
654generalized.
655
656In addition to the kernel implementation, there must be user-space support
657for the new ioctl. There's code in Mesa that might be able to use the new
658call.
659
660Contact: Thomas Zimmermann <tzimmermann@suse.de>
661
662Level: Advanced
626 663
627Better Testing664Better Testing
628==============665==============
Documentation/gpu/vkms.rst+100−19
@@ -51,6 +51,97 @@ To disable the driver, use :: · 50 unmodified lines
51 51
52 sudo modprobe -r vkms52 sudo modprobe -r vkms
53 53
54Configuring With Configfs
55=========================
56
57It is possible to create and configure multiple VKMS instances via configfs.
58
59Start by mounting configfs and loading VKMS::
60
61 sudo mount -t configfs none /config
62 sudo modprobe vkms
63
64Once VKMS is loaded, ``/config/vkms`` is created automatically. Each directory
65under ``/config/vkms`` represents a VKMS instance, create a new one::
66
67 sudo mkdir /config/vkms/my-vkms
68
69By default, the instance is disabled::
70
71 cat /config/vkms/my-vkms/enabled
72 0
73
74And directories are created for each configurable item of the display pipeline::
75
76 tree /config/vkms/my-vkms
77 ├── connectors
78 ├── crtcs
79 ├── enabled
80 ├── encoders
81 └── planes
82
83To add items to the display pipeline, create one or more directories under the
84available paths.
85
86Start by creating one or more planes::
87
88 sudo mkdir /config/vkms/my-vkms/planes/plane0
89
90Planes have 1 configurable attribute:
91
92- type: Plane type: 0 overlay, 1 primary, 2 cursor (same values as those
93 exposed by the "type" property of a plane)
94
95Continue by creating one or more CRTCs::
96
97 sudo mkdir /config/vkms/my-vkms/crtcs/crtc0
98
99CRTCs have 1 configurable attribute:
100
101- writeback: Enable or disable writeback connector support by writing 1 or 0
102
103Next, create one or more encoders::
104
105 sudo mkdir /config/vkms/my-vkms/encoders/encoder0
106
107Last but not least, create one or more connectors::
108
109 sudo mkdir /config/vkms/my-vkms/connectors/connector0
110
111Connectors have 1 configurable attribute:
112
113- status: Connection status: 1 connected, 2 disconnected, 3 unknown (same values
114 as those exposed by the "status" property of a connector)
115
116To finish the configuration, link the different pipeline items::
117
118 sudo ln -s /config/vkms/my-vkms/crtcs/crtc0 /config/vkms/my-vkms/planes/plane0/possible_crtcs
119 sudo ln -s /config/vkms/my-vkms/crtcs/crtc0 /config/vkms/my-vkms/encoders/encoder0/possible_crtcs
120 sudo ln -s /config/vkms/my-vkms/encoders/encoder0 /config/vkms/my-vkms/connectors/connector0/possible_encoders
121
122Since at least one primary plane is required, make sure to set the right type::
123
124 echo "1" | sudo tee /config/vkms/my-vkms/planes/plane0/type
125
126Once you are done configuring the VKMS instance, enable it::
127
128 echo "1" | sudo tee /config/vkms/my-vkms/enabled
129
130Finally, you can remove the VKMS instance disabling it::
131
132 echo "0" | sudo tee /config/vkms/my-vkms/enabled
133
134And removing the top level directory and its subdirectories::
135
136 sudo rm /config/vkms/my-vkms/planes/*/possible_crtcs/*
137 sudo rm /config/vkms/my-vkms/encoders/*/possible_crtcs/*
138 sudo rm /config/vkms/my-vkms/connectors/*/possible_encoders/*
139 sudo rmdir /config/vkms/my-vkms/planes/*
140 sudo rmdir /config/vkms/my-vkms/crtcs/*
141 sudo rmdir /config/vkms/my-vkms/encoders/*
142 sudo rmdir /config/vkms/my-vkms/connectors/*
143 sudo rmdir /config/vkms/my-vkms
144
54Testing With IGT145Testing With IGT
55================146================
56 147
@@ -68,26 +159,23 @@ To return to graphical mode, do:: · 11 unmodified lines
68 159
69 sudo systemctl isolate graphical.target160 sudo systemctl isolate graphical.target
70 161
71Once you are in text only mode, you can run tests using the --device switch162Once you are in text only mode, you can run tests using the IGT_FORCE_DRIVER
72or IGT_DEVICE variable to specify the device filter for the driver we want163variable to specify the device filter for the driver we want to test.
73to test. IGT_DEVICE can also be used with the run-test.sh script to run the164IGT_FORCE_DRIVER can also be used with the run-tests.sh script to run the
74tests for a specific driver::165tests for a specific driver::
75 166
76 sudo ./build/tests/<name of test> --device "sys:/sys/devices/platform/vkms"167 sudo IGT_FORCE_DRIVER="vkms" ./build/tests/<name of test>
77 sudo IGT_DEVICE="sys:/sys/devices/platform/vkms" ./build/tests/<name of test>168 sudo IGT_FORCE_DRIVER="vkms" ./scripts/run-tests.sh -t <name of test>
78 sudo IGT_DEVICE="sys:/sys/devices/platform/vkms" ./scripts/run-tests.sh -t <name of test>
79 169
80For example, to test the functionality of the writeback library,170For example, to test the functionality of the writeback library,
81we can run the kms_writeback test::171we can run the kms_writeback test::
82 172
83 sudo ./build/tests/kms_writeback --device "sys:/sys/devices/platform/vkms"173 sudo IGT_FORCE_DRIVER="vkms" ./build/tests/kms_writeback
84 sudo IGT_DEVICE="sys:/sys/devices/platform/vkms" ./build/tests/kms_writeback174 sudo IGT_FORCE_DRIVER="vkms" ./scripts/run-tests.sh -t kms_writeback
85 sudo IGT_DEVICE="sys:/sys/devices/platform/vkms" ./scripts/run-tests.sh -t kms_writeback
86 175
87You can also run subtests if you do not want to run the entire test::176You can also run subtests if you do not want to run the entire test::
88 177
89 sudo ./build/tests/kms_flip --run-subtest basic-plain-flip --device "sys:/sys/devices/platform/vkms"178 sudo IGT_FORCE_DRIVER="vkms" ./build/tests/kms_flip --run-subtest basic-plain-flip
90 sudo IGT_DEVICE="sys:/sys/devices/platform/vkms" ./build/tests/kms_flip --run-subtest basic-plain-flip
91 179
92Testing With KUnit180Testing With KUnit
93==================181==================
@@ -147,21 +235,14 @@ Runtime Configuration · 53 unmodified lines
147---------------------235---------------------
148 236
149We want to be able to reconfigure vkms instance without having to reload the237We want to be able to reconfigure vkms instance without having to reload the
150module. Use/Test-cases:238module through configfs. Use/Test-cases:
151 239
152- Hotplug/hotremove connectors on the fly (to be able to test DP MST handling240- Hotplug/hotremove connectors on the fly (to be able to test DP MST handling
153 of compositors).241 of compositors).
154 242
155- Configure planes/crtcs/connectors (we'd need some code to have more than 1 of
156 them first).
157
158- Change output configuration: Plug/unplug screens, change EDID, allow changing243- Change output configuration: Plug/unplug screens, change EDID, allow changing
159 the refresh rate.244 the refresh rate.
160 245
161The currently proposed solution is to expose vkms configuration through
162configfs. All existing module options should be supported through configfs
163too.
164
165Writeback support246Writeback support
166-----------------247-----------------
167 248
Documentation/gpu/xe/index.rst+1−0
@@ -14,6 +14,7 @@ DG2, etc is provided to prototype the driver. · 13 unmodified lines
14 xe_mm14 xe_mm
15 xe_map15 xe_map
16 xe_migrate16 xe_migrate
17 xe_exec_queue
17 xe_cs18 xe_cs
18 xe_pm19 xe_pm
19 xe_gt_freq20 xe_gt_freq
Documentation/gpu/xe/xe_exec_queue.rst+20−0
@@ -0,0 +1,20 @@
1.. SPDX-License-Identifier: (GPL-2.0+ OR MIT)
2
3===============
4Execution Queue
5===============
6
7.. kernel-doc:: drivers/gpu/drm/xe/xe_exec_queue.c
8 :doc: Execution Queue
9
10Internal API
11============
12
13.. kernel-doc:: drivers/gpu/drm/xe/xe_exec_queue_types.h
14 :internal:
15
16.. kernel-doc:: drivers/gpu/drm/xe/xe_exec_queue.h
17 :internal:
18
19.. kernel-doc:: drivers/gpu/drm/xe/xe_exec_queue.c
20 :internal:
Documentation/gpu/xe/xe_gt_freq.rst+3−0
@@ -7,6 +7,9 @@ Xe GT Frequency Management · 6 unmodified lines
7.. kernel-doc:: drivers/gpu/drm/xe/xe_gt_freq.c7.. kernel-doc:: drivers/gpu/drm/xe/xe_gt_freq.c
8 :doc: Xe GT Frequency Management8 :doc: Xe GT Frequency Management
9 9
10.. kernel-doc:: drivers/gpu/drm/xe/xe_gt_throttle.c
11 :doc: Xe GT Throttle
12
10Internal API13Internal API
11============14============
12 15
Documentation/userspace-api/dma-buf-heaps.rst+49−10
@@ -16,13 +16,52 @@ following heaps: · 15 unmodified lines
16 16
17 - The ``system`` heap allocates virtually contiguous, cacheable, buffers.17 - The ``system`` heap allocates virtually contiguous, cacheable, buffers.
18 18
19 - The ``cma`` heap allocates physically contiguous, cacheable,19 - The ``default_cma_region`` heap allocates physically contiguous,
20 buffers. Only present if a CMA region is present. Such a region is20 cacheable, buffers. Only present if a CMA region is present. Such a
21 usually created either through the kernel commandline through the21 region is usually created either through the kernel commandline
22 ``cma`` parameter, a memory region Device-Tree node with the22 through the ``cma`` parameter, a memory region Device-Tree node with
23 ``linux,cma-default`` property set, or through the ``CMA_SIZE_MBYTES`` or23 the ``linux,cma-default`` property set, or through the
24 ``CMA_SIZE_PERCENTAGE`` Kconfig options. The heap's name in devtmpfs is24 ``CMA_SIZE_MBYTES`` or ``CMA_SIZE_PERCENTAGE`` Kconfig options. Prior
25 ``default_cma_region``. For backwards compatibility, when the25 to Linux 6.17, its name wasn't stable and could be called
26 ``DMABUF_HEAPS_CMA_LEGACY`` Kconfig option is set, a duplicate node is26 ``reserved``, ``linux,cma``, or ``default-pool``, depending on the
27 created following legacy naming conventions; the legacy name might be27 platform.
28 ``reserved``, ``linux,cma``, or ``default-pool``.28
29 - A heap will be created for each reusable region in the device tree
30 with the ``shared-dma-pool`` compatible, using the full device tree
31 node name as its name. The buffer semantics are identical to
32 ``default-cma-region``.
33
34Naming Convention
35=================
36
37``dma-buf`` heaps name should meet a number of constraints:
38
39- The name must be stable, and must not change from one version to the other.
40 Userspace identifies heaps by their name, so if the names ever change, we
41 would be likely to introduce regressions.
42
43- The name must describe the memory region the heap will allocate from, and
44 must uniquely identify it in a given platform. Since userspace applications
45 use the heap name as the discriminant, it must be able to tell which heap it
46 wants to use reliably if there's multiple heaps.
47
48- The name must not mention implementation details, such as the allocator. The
49 heap driver will change over time, and implementation details when it was
50 introduced might not be relevant in the future.
51
52- The name should describe properties of the buffers that would be allocated.
53 Doing so will make heap identification easier for userspace. Such properties
54 are:
55
56 - ``contiguous`` for physically contiguous buffers;
57
58 - ``protected`` for encrypted buffers not accessible the OS;
59
60- The name may describe intended usage. Doing so will make heap identification
61 easier for userspace applications and users.
62
63For example, assuming a platform with a reserved memory region located
64at the RAM address 0x42000000, intended to allocate video framebuffers,
65physically contiguous, and backed by the CMA kernel allocator, good
66names would be ``memory@42000000-contiguous`` or ``video@42000000``, but
67``cma-video`` wouldn't.
MAINTAINERS+37−11
@@ -1081,7 +1081,7 @@ M: Austin Zheng <austin.zheng@amd.com> · 1080 unmodified lines
1081M: Jun Lei <jun.lei@amd.com>1081M: Jun Lei <jun.lei@amd.com>
1082S: Supported1082S: Supported
1083F: drivers/gpu/drm/amd/display/dc/dml/1083F: drivers/gpu/drm/amd/display/dc/dml/
1084F: drivers/gpu/drm/amd/display/dc/dml2/1084F: drivers/gpu/drm/amd/display/dc/dml2_0/
1085 1085
1086AMD FAM15H PROCESSOR POWER MONITORING DRIVER1086AMD FAM15H PROCESSOR POWER MONITORING DRIVER
1087M: Huang Rui <ray.huang@amd.com>1087M: Huang Rui <ray.huang@amd.com>
@@ -2022,6 +2022,15 @@ F: arch/arm64/include/asm/arch_timer.h · 934 unmodified lines
2022F: drivers/clocksource/arm_arch_timer.c2022F: drivers/clocksource/arm_arch_timer.c
2023F: drivers/clocksource/arm_arch_timer_mmio.c2023F: drivers/clocksource/arm_arch_timer_mmio.c
2024 2024
2025ARM ETHOS-U NPU DRIVER
2026M: Rob Herring (Arm) <robh@kernel.org>
2027M: Tomeu Vizoso <tomeu@tomeuvizoso.net>
2028L: dri-devel@lists.freedesktop.org
2029S: Supported
2030T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
2031F: drivers/accel/ethosu/
2032F: include/uapi/drm/ethosu_accel.h
2033
2025ARM GENERIC INTERRUPT CONTROLLER DRIVERS2034ARM GENERIC INTERRUPT CONTROLLER DRIVERS
2026M: Marc Zyngier <maz@kernel.org>2035M: Marc Zyngier <maz@kernel.org>
2027L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)2036L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
@@ -2097,7 +2106,8 @@ F: drivers/gpu/drm/arm/display/komeda/ · 69 unmodified lines
2097ARM MALI PANFROST DRM DRIVER2106ARM MALI PANFROST DRM DRIVER
2098M: Boris Brezillon <boris.brezillon@collabora.com>2107M: Boris Brezillon <boris.brezillon@collabora.com>
2099M: Rob Herring <robh@kernel.org>2108M: Rob Herring <robh@kernel.org>
2100R: Steven Price <steven.price@arm.com>2109M: Steven Price <steven.price@arm.com>
2110M: Adrián Larumbe <adrian.larumbe@collabora.com>
2101L: dri-devel@lists.freedesktop.org2111L: dri-devel@lists.freedesktop.org
2102S: Supported2112S: Supported
2103T: git https://gitlab.freedesktop.org/drm/misc/kernel.git2113T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
@@ -2317,7 +2327,7 @@ S: Maintained · 213 unmodified lines
2317F: drivers/clk/sunxi/2327F: drivers/clk/sunxi/
2318 2328
2319ARM/Allwinner sunXi SoC support2329ARM/Allwinner sunXi SoC support
2320M: Chen-Yu Tsai <wens@csie.org>2330M: Chen-Yu Tsai <wens@kernel.org>
2321M: Jernej Skrabec <jernej.skrabec@gmail.com>2331M: Jernej Skrabec <jernej.skrabec@gmail.com>
2322M: Samuel Holland <samuel@sholland.org>2332M: Samuel Holland <samuel@sholland.org>
2323L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)2333L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
@@ -7331,6 +7341,7 @@ F: Documentation/userspace-api/dma-buf-alloc-exchange.rst · 5007 unmodified lines
7331F: drivers/dma-buf/7341F: drivers/dma-buf/
7332F: include/linux/*fence.h7342F: include/linux/*fence.h
7333F: include/linux/dma-buf.h7343F: include/linux/dma-buf.h
7344F: include/linux/dma-buf/
7334F: include/linux/dma-resv.h7345F: include/linux/dma-resv.h
7335K: \bdma_(?:buf|fence|resv)\b7346K: \bdma_(?:buf|fence|resv)\b
7336 7347
@@ -7654,8 +7665,7 @@ F: drivers/accel/ · 317 unmodified lines
7654F: include/drm/drm_accel.h7665F: include/drm/drm_accel.h
7655 7666
7656DRM DRIVER FOR ALLWINNER DE2 AND DE3 ENGINE7667DRM DRIVER FOR ALLWINNER DE2 AND DE3 ENGINE
7657M: Maxime Ripard <mripard@kernel.org>7668M: Chen-Yu Tsai <wens@kernel.org>
7658M: Chen-Yu Tsai <wens@csie.org>
7659R: Jernej Skrabec <jernej.skrabec@gmail.com>7669R: Jernej Skrabec <jernej.skrabec@gmail.com>
7660L: dri-devel@lists.freedesktop.org7670L: dri-devel@lists.freedesktop.org
7661S: Supported7671S: Supported
@@ -7764,7 +7774,8 @@ F: Documentation/devicetree/bindings/display/panel/panel-edp.yaml · 102 unmodified lines
7764F: drivers/gpu/drm/panel/panel-edp.c7774F: drivers/gpu/drm/panel/panel-edp.c
7765 7775
7766DRM DRIVER FOR GENERIC USB DISPLAY7776DRM DRIVER FOR GENERIC USB DISPLAY
7767S: Orphan7777M: Ruben Wauters <rubenru09@aol.com>
7778S: Maintained
7768W: https://github.com/notro/gud/wiki7779W: https://github.com/notro/gud/wiki
7769T: git https://gitlab.freedesktop.org/drm/misc/kernel.git7780T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
7770F: drivers/gpu/drm/gud/7781F: drivers/gpu/drm/gud/
@@ -7886,6 +7897,7 @@ DRM DRIVER for Qualcomm Adreno GPUs · 115 unmodified lines
7886M: Rob Clark <robin.clark@oss.qualcomm.com>7897M: Rob Clark <robin.clark@oss.qualcomm.com>
7887R: Sean Paul <sean@poorly.run>7898R: Sean Paul <sean@poorly.run>
7888R: Konrad Dybcio <konradybcio@kernel.org>7899R: Konrad Dybcio <konradybcio@kernel.org>
7900R: Akhil P Oommen <akhilpo@oss.qualcomm.com>
7889L: linux-arm-msm@vger.kernel.org7901L: linux-arm-msm@vger.kernel.org
7890L: dri-devel@lists.freedesktop.org7902L: dri-devel@lists.freedesktop.org
7891L: freedreno@lists.freedesktop.org7903L: freedreno@lists.freedesktop.org
@@ -7905,7 +7917,7 @@ DRM DRIVER for Qualcomm display hardware · 13 unmodified lines
7905M: Rob Clark <robin.clark@oss.qualcomm.com>7917M: Rob Clark <robin.clark@oss.qualcomm.com>
7906M: Dmitry Baryshkov <lumag@kernel.org>7918M: Dmitry Baryshkov <lumag@kernel.org>
7907R: Abhinav Kumar <abhinav.kumar@linux.dev>7919R: Abhinav Kumar <abhinav.kumar@linux.dev>
7908R: Jessica Zhang <jessica.zhang@oss.qualcomm.com>7920R: Jessica Zhang <jesszhan0024@gmail.com>
7909R: Sean Paul <sean@poorly.run>7921R: Sean Paul <sean@poorly.run>
7910R: Marijn Suijten <marijn.suijten@somainline.org>7922R: Marijn Suijten <marijn.suijten@somainline.org>
7911L: linux-arm-msm@vger.kernel.org7923L: linux-arm-msm@vger.kernel.org
@@ -8074,12 +8086,25 @@ S: Maintained · 162 unmodified lines
8074F: Documentation/devicetree/bindings/display/panel/samsung,s6d7aa0.yaml8086F: Documentation/devicetree/bindings/display/panel/samsung,s6d7aa0.yaml
8075F: drivers/gpu/drm/panel/panel-samsung-s6d7aa0.c8087F: drivers/gpu/drm/panel/panel-samsung-s6d7aa0.c
8076 8088
8089DRM DRIVER FOR SAMSUNG S6E3FC2X01 DDIC
8090M: David Heidelberg <david@ixit.cz>
8091S: Maintained
8092F: Documentation/devicetree/bindings/display/panel/samsung,s6e3fc2x01.yaml
8093F: drivers/gpu/drm/panel/panel-samsung-s6e3fc2x01.c
8094
8077DRM DRIVER FOR SAMSUNG S6E3HA8 PANELS8095DRM DRIVER FOR SAMSUNG S6E3HA8 PANELS
8078M: Dzmitry Sankouski <dsankouski@gmail.com>8096M: Dzmitry Sankouski <dsankouski@gmail.com>
8079S: Maintained8097S: Maintained
8080F: Documentation/devicetree/bindings/display/panel/samsung,s6e3ha8.yaml8098F: Documentation/devicetree/bindings/display/panel/samsung,s6e3ha8.yaml
8081F: drivers/gpu/drm/panel/panel-samsung-s6e3ha8.c8099F: drivers/gpu/drm/panel/panel-samsung-s6e3ha8.c
8082 8100
8101DRM DRIVER FOR SAMSUNG SOFEF00 DDIC
8102M: David Heidelberg <david@ixit.cz>
8103M: Casey Connolly <casey.connolly@linaro.org>
8104S: Maintained
8105F: Documentation/devicetree/bindings/display/panel/samsung,sofef00.yaml
8106F: drivers/gpu/drm/panel/panel-samsung-sofef00.c
8107
8083DRM DRIVER FOR SHARP MEMORY LCD8108DRM DRIVER FOR SHARP MEMORY LCD
8084M: Alex Lanzano <lanzano.alex@gmail.com>8109M: Alex Lanzano <lanzano.alex@gmail.com>
8085S: Maintained8110S: Maintained
@@ -8264,12 +8289,12 @@ S: Supported · 178 unmodified lines
8264W: https://drm.pages.freedesktop.org/maintainer-tools/drm-rust.html8289W: https://drm.pages.freedesktop.org/maintainer-tools/drm-rust.html
8265T: git https://gitlab.freedesktop.org/drm/rust/kernel.git8290T: git https://gitlab.freedesktop.org/drm/rust/kernel.git
8266F: drivers/gpu/drm/nova/8291F: drivers/gpu/drm/nova/
8292F: drivers/gpu/drm/tyr/
8267F: drivers/gpu/nova-core/8293F: drivers/gpu/nova-core/
8268F: rust/kernel/drm/8294F: rust/kernel/drm/
8269 8295
8270DRM DRIVERS FOR ALLWINNER A108296DRM DRIVERS FOR ALLWINNER A10
8271M: Maxime Ripard <mripard@kernel.org>8297M: Chen-Yu Tsai <wens@kernel.org>
8272M: Chen-Yu Tsai <wens@csie.org>
8273L: dri-devel@lists.freedesktop.org8298L: dri-devel@lists.freedesktop.org
8274S: Supported8299S: Supported
8275T: git https://gitlab.freedesktop.org/drm/misc/kernel.git8300T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
@@ -8596,6 +8621,7 @@ S: Supported · 320 unmodified lines
8596T: git https://gitlab.freedesktop.org/drm/misc/kernel.git8621T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
8597F: drivers/gpu/drm/scheduler/8622F: drivers/gpu/drm/scheduler/
8598F: include/drm/gpu_scheduler.h8623F: include/drm/gpu_scheduler.h
8624F: include/drm/spsc_queue.h
8599 8625
8600DRM GPUVM8626DRM GPUVM
8601M: Danilo Krummrich <dakr@kernel.org>8627M: Danilo Krummrich <dakr@kernel.org>
@@ -8618,7 +8644,7 @@ F: drivers/gpu/drm/clients/drm_log.c · 16 unmodified lines
8618 8644
8619DRM PANEL DRIVERS8645DRM PANEL DRIVERS
8620M: Neil Armstrong <neil.armstrong@linaro.org>8646M: Neil Armstrong <neil.armstrong@linaro.org>
8621R: Jessica Zhang <jessica.zhang@oss.qualcomm.com>8647R: Jessica Zhang <jesszhan0024@gmail.com>
8622L: dri-devel@lists.freedesktop.org8648L: dri-devel@lists.freedesktop.org
8623S: Maintained8649S: Maintained
8624T: git https://gitlab.freedesktop.org/drm/misc/kernel.git8650T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
@@ -27832,7 +27858,7 @@ F: drivers/acpi/pmic/intel_pmic_xpower.c · 19207 unmodified lines
27832N: axp28827858N: axp288
27833 27859
27834X-POWERS MULTIFUNCTION PMIC DEVICE DRIVERS27860X-POWERS MULTIFUNCTION PMIC DEVICE DRIVERS
27835M: Chen-Yu Tsai <wens@csie.org>27861M: Chen-Yu Tsai <wens@kernel.org>
27836L: linux-kernel@vger.kernel.org27862L: linux-kernel@vger.kernel.org
27837S: Maintained27863S: Maintained
27838N: axp[128]27864N: axp[128]
arch/arm/boot/dts/st/stih410.dtsi+192−134
@@ -34,6 +34,41 @@ · 33 unmodified lines
34 status = "disabled";34 status = "disabled";
35 };35 };
36 36
37 display-subsystem {
38 compatible = "st,sti-display-subsystem";
39 ports = <&compositor>, <&hqvdp>, <&tvout>, <&sti_hdmi>;
40
41 assigned-clocks = <&clk_s_d2_quadfs 0>,
42 <&clk_s_d2_quadfs 1>,
43 <&clk_s_c0_pll1 0>,
44 <&clk_s_c0_flexgen CLK_COMPO_DVP>,
45 <&clk_s_c0_flexgen CLK_MAIN_DISP>,
46 <&clk_s_d2_flexgen CLK_PIX_MAIN_DISP>,
47 <&clk_s_d2_flexgen CLK_PIX_AUX_DISP>,
48 <&clk_s_d2_flexgen CLK_PIX_GDP1>,
49 <&clk_s_d2_flexgen CLK_PIX_GDP2>,
50 <&clk_s_d2_flexgen CLK_PIX_GDP3>,
51 <&clk_s_d2_flexgen CLK_PIX_GDP4>;
52
53 assigned-clock-parents = <0>,
54 <0>,
55 <0>,
56 <&clk_s_c0_pll1 0>,
57 <&clk_s_c0_pll1 0>,
58 <&clk_s_d2_quadfs 0>,
59 <&clk_s_d2_quadfs 1>,
60 <&clk_s_d2_quadfs 0>,
61 <&clk_s_d2_quadfs 0>,
62 <&clk_s_d2_quadfs 0>,
63 <&clk_s_d2_quadfs 0>;
64
65 assigned-clock-rates = <297000000>,
66 <297000000>,
67 <0>,
68 <400000000>,
69 <400000000>;
70 };
71
37 soc {72 soc {
38 ohci0: usb@9a03c00 {73 ohci0: usb@9a03c00 {
39 compatible = "st,st-ohci-300x";74 compatible = "st,st-ohci-300x";
@@ -99,151 +134,174 @@ · 59 unmodified lines
99 status = "disabled";134 status = "disabled";
100 };135 };
101 136
102 sti-display-subsystem@0 {137 compositor: display-controller@9d11000 {
103 compatible = "st,sti-display-subsystem";138 compatible = "st,stih407-compositor";
104 #address-cells = <1>;139 reg = <0x9d11000 0x1000>;
105 #size-cells = <1>;140
106 141 clock-names = "compo_main",
107 reg = <0 0>;142 "compo_aux",
108 assigned-clocks = <&clk_s_d2_quadfs 0>,143 "pix_main",
109 <&clk_s_d2_quadfs 1>,144 "pix_aux",
110 <&clk_s_c0_pll1 0>,145 "pix_gdp1",
111 <&clk_s_c0_flexgen CLK_COMPO_DVP>,146 "pix_gdp2",
112 <&clk_s_c0_flexgen CLK_MAIN_DISP>,147 "pix_gdp3",
113 <&clk_s_d2_flexgen CLK_PIX_MAIN_DISP>,148 "pix_gdp4",
114 <&clk_s_d2_flexgen CLK_PIX_AUX_DISP>,149 "main_parent",
115 <&clk_s_d2_flexgen CLK_PIX_GDP1>,150 "aux_parent";
116 <&clk_s_d2_flexgen CLK_PIX_GDP2>,151
117 <&clk_s_d2_flexgen CLK_PIX_GDP3>,152 clocks = <&clk_s_c0_flexgen CLK_COMPO_DVP>,
118 <&clk_s_d2_flexgen CLK_PIX_GDP4>;153 <&clk_s_c0_flexgen CLK_COMPO_DVP>,
119 154 <&clk_s_d2_flexgen CLK_PIX_MAIN_DISP>,
120 assigned-clock-parents = <0>,155 <&clk_s_d2_flexgen CLK_PIX_AUX_DISP>,
121 <0>,156 <&clk_s_d2_flexgen CLK_PIX_GDP1>,
122 <0>,157 <&clk_s_d2_flexgen CLK_PIX_GDP2>,
123 <&clk_s_c0_pll1 0>,158 <&clk_s_d2_flexgen CLK_PIX_GDP3>,
124 <&clk_s_c0_pll1 0>,159 <&clk_s_d2_flexgen CLK_PIX_GDP4>,
125 <&clk_s_d2_quadfs 0>,160 <&clk_s_d2_quadfs 0>,
126 <&clk_s_d2_quadfs 1>,161 <&clk_s_d2_quadfs 1>;
127 <&clk_s_d2_quadfs 0>,162
163 reset-names = "compo-main", "compo-aux";
164 resets = <&softreset STIH407_COMPO_SOFTRESET>,
165 <&softreset STIH407_COMPO_SOFTRESET>;
166 st,vtg = <&vtg_main>, <&vtg_aux>;
167
168 ports {
169 #address-cells = <1>;
170 #size-cells = <0>;
171
172 port@0 {
173 reg = <0>;
174 compo_main_out: endpoint {
175 remote-endpoint = <&tvout_in0>;
176 };
177 };
178
179 port@1 {
180 reg = <1>;
181 compo_aux_out: endpoint {
182 remote-endpoint = <&tvout_in1>;
183 };
184 };
185 };
186 };
187
188 tvout: encoder@8d08000 {
189 compatible = "st,stih407-tvout";
190 reg = <0x8d08000 0x1000>;
191 reg-names = "tvout-reg";
192 reset-names = "tvout";
193 resets = <&softreset STIH407_HDTVOUT_SOFTRESET>;
194 assigned-clocks = <&clk_s_d2_flexgen CLK_PIX_HDMI>,
195 <&clk_s_d2_flexgen CLK_TMDS_HDMI>,
196 <&clk_s_d2_flexgen CLK_REF_HDMIPHY>,
197 <&clk_s_d0_flexgen CLK_PCM_0>,
198 <&clk_s_d2_flexgen CLK_PIX_HDDAC>,
199 <&clk_s_d2_flexgen CLK_HDDAC>;
200
201 assigned-clock-parents = <&clk_s_d2_quadfs 0>,
202 <&clk_tmdsout_hdmi>,
128 <&clk_s_d2_quadfs 0>,203 <&clk_s_d2_quadfs 0>,
204 <&clk_s_d0_quadfs 0>,
129 <&clk_s_d2_quadfs 0>,205 <&clk_s_d2_quadfs 0>,
130 <&clk_s_d2_quadfs 0>;206 <&clk_s_d2_quadfs 0>;
131 207
132 assigned-clock-rates = <297000000>,208 ports {
133 <297000000>,
134 <0>,
135 <400000000>,
136 <400000000>;
137
138 ranges;
139
140 sti-compositor@9d11000 {
141 compatible = "st,stih407-compositor";
142 reg = <0x9d11000 0x1000>;
143
144 clock-names = "compo_main",
145 "compo_aux",
146 "pix_main",
147 "pix_aux",
148 "pix_gdp1",
149 "pix_gdp2",
150 "pix_gdp3",
151 "pix_gdp4",
152 "main_parent",
153 "aux_parent";
154
155 clocks = <&clk_s_c0_flexgen CLK_COMPO_DVP>,
156 <&clk_s_c0_flexgen CLK_COMPO_DVP>,
157 <&clk_s_d2_flexgen CLK_PIX_MAIN_DISP>,
158 <&clk_s_d2_flexgen CLK_PIX_AUX_DISP>,
159 <&clk_s_d2_flexgen CLK_PIX_GDP1>,
160 <&clk_s_d2_flexgen CLK_PIX_GDP2>,
161 <&clk_s_d2_flexgen CLK_PIX_GDP3>,
162 <&clk_s_d2_flexgen CLK_PIX_GDP4>,
163 <&clk_s_d2_quadfs 0>,
164 <&clk_s_d2_quadfs 1>;
165
166 reset-names = "compo-main", "compo-aux";
167 resets = <&softreset STIH407_COMPO_SOFTRESET>,
168 <&softreset STIH407_COMPO_SOFTRESET>;
169 st,vtg = <&vtg_main>, <&vtg_aux>;
170 };
171
172 sti-tvout@8d08000 {
173 compatible = "st,stih407-tvout";
174 reg = <0x8d08000 0x1000>;
175 reg-names = "tvout-reg";
176 reset-names = "tvout";
177 resets = <&softreset STIH407_HDTVOUT_SOFTRESET>;
178 #address-cells = <1>;209 #address-cells = <1>;
179 #size-cells = <1>;210 #size-cells = <0>;
180 assigned-clocks = <&clk_s_d2_flexgen CLK_PIX_HDMI>,211
181 <&clk_s_d2_flexgen CLK_TMDS_HDMI>,212 port@0 {
182 <&clk_s_d2_flexgen CLK_REF_HDMIPHY>,213 reg = <0>;
183 <&clk_s_d0_flexgen CLK_PCM_0>,214 tvout_in0: endpoint {
184 <&clk_s_d2_flexgen CLK_PIX_HDDAC>,215 remote-endpoint = <&compo_main_out>;
185 <&clk_s_d2_flexgen CLK_HDDAC>;216 };
186 217 };
187 assigned-clock-parents = <&clk_s_d2_quadfs 0>,218
188 <&clk_tmdsout_hdmi>,219 port@1 {
189 <&clk_s_d2_quadfs 0>,220 reg = <1>;
190 <&clk_s_d0_quadfs 0>,221 tvout_in1: endpoint {
191 <&clk_s_d2_quadfs 0>,222 remote-endpoint = <&compo_aux_out>;
192 <&clk_s_d2_quadfs 0>;223 };
224 };
225
226 port@2 {
227 reg = <2>;
228 tvout_out0: endpoint {
229 remote-endpoint = <&hdmi_in>;
230 };
231 };
232
233 port@3 {
234 reg = <3>;
235 tvout_out1: endpoint {
236 remote-endpoint = <&hda_in>;
237 };
238 };
193 };239 };
240 };
194 241
195 sti_hdmi: sti-hdmi@8d04000 {242 sti_hdmi: hdmi@8d04000 {
196 compatible = "st,stih407-hdmi";243 compatible = "st,stih407-hdmi";
197 reg = <0x8d04000 0x1000>;244 reg = <0x8d04000 0x1000>;
198 reg-names = "hdmi-reg";245 reg-names = "hdmi-reg";
199 #sound-dai-cells = <0>;246 #sound-dai-cells = <0>;
200 interrupts = <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>;247 interrupts = <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>;
201 interrupt-names = "irq";248 interrupt-names = "irq";
202 clock-names = "pix",249 clock-names = "pix",
203 "tmds",250 "tmds",
204 "phy",251 "phy",
205 "audio",252 "audio",
206 "main_parent",253 "main_parent",
207 "aux_parent";254 "aux_parent";
208 255
209 clocks = <&clk_s_d2_flexgen CLK_PIX_HDMI>,256 clocks = <&clk_s_d2_flexgen CLK_PIX_HDMI>,
210 <&clk_s_d2_flexgen CLK_TMDS_HDMI>,257 <&clk_s_d2_flexgen CLK_TMDS_HDMI>,
211 <&clk_s_d2_flexgen CLK_REF_HDMIPHY>,258 <&clk_s_d2_flexgen CLK_REF_HDMIPHY>,
212 <&clk_s_d0_flexgen CLK_PCM_0>,259 <&clk_s_d0_flexgen CLK_PCM_0>,
213 <&clk_s_d2_quadfs 0>,260 <&clk_s_d2_quadfs 0>,
214 <&clk_s_d2_quadfs 1>;261 <&clk_s_d2_quadfs 1>;
215 262
216 hdmi,hpd-gpio = <&pio5 3 GPIO_ACTIVE_LOW>;263 hdmi,hpd-gpio = <&pio5 3 GPIO_ACTIVE_LOW>;
217 reset-names = "hdmi";264 reset-names = "hdmi";
218 resets = <&softreset STIH407_HDMI_TX_PHY_SOFTRESET>;265 resets = <&softreset STIH407_HDMI_TX_PHY_SOFTRESET>;
219 ddc = <&hdmiddc>;266 ddc = <&hdmiddc>;
267
268 port {
269 hdmi_in: endpoint {
270 remote-endpoint = <&tvout_out0>;
271 };
220 };272 };
273 };
221 274
222 sti-hda@8d02000 {275 analog@8d02000 {
223 compatible = "st,stih407-hda";276 compatible = "st,stih407-hda";
224 status = "disabled";277 status = "disabled";
225 reg = <0x8d02000 0x400>, <0x92b0120 0x4>;278 reg = <0x8d02000 0x400>, <0x92b0120 0x4>;
226 reg-names = "hda-reg", "video-dacs-ctrl";279 reg-names = "hda-reg", "video-dacs-ctrl";
227 clock-names = "pix",280 clock-names = "pix",
228 "hddac",281 "hddac",
229 "main_parent",282 "main_parent",
230 "aux_parent";283 "aux_parent";
231 clocks = <&clk_s_d2_flexgen CLK_PIX_HDDAC>,284 clocks = <&clk_s_d2_flexgen CLK_PIX_HDDAC>,
232 <&clk_s_d2_flexgen CLK_HDDAC>,285 <&clk_s_d2_flexgen CLK_HDDAC>,
233 <&clk_s_d2_quadfs 0>,286 <&clk_s_d2_quadfs 0>,
234 <&clk_s_d2_quadfs 1>;287 <&clk_s_d2_quadfs 1>;
288
289 port {
290 hda_in: endpoint {
291 remote-endpoint = <&tvout_out1>;
292 };
235 };293 };
294 };
236 295
237 sti-hqvdp@9c00000 {296 hqvdp: plane@9c00000 {
238 compatible = "st,stih407-hqvdp";297 compatible = "st,stih407-hqvdp";
239 reg = <0x9C00000 0x100000>;298 reg = <0x9C00000 0x100000>;
240 clock-names = "hqvdp", "pix_main";299 clock-names = "hqvdp", "pix_main";
241 clocks = <&clk_s_c0_flexgen CLK_MAIN_DISP>,300 clocks = <&clk_s_c0_flexgen CLK_MAIN_DISP>,
242 <&clk_s_d2_flexgen CLK_PIX_MAIN_DISP>;301 <&clk_s_d2_flexgen CLK_PIX_MAIN_DISP>;
243 reset-names = "hqvdp";302 reset-names = "hqvdp";
244 resets = <&softreset STIH407_HDQVDP_SOFTRESET>;303 resets = <&softreset STIH407_HDQVDP_SOFTRESET>;
245 st,vtg = <&vtg_main>;304 st,vtg = <&vtg_main>;
246 };
247 };305 };
248 306
249 bdisp0:bdisp@9f10000 {307 bdisp0:bdisp@9f10000 {
arch/arm64/boot/dts/freescale/imx95.dtsi+37−0
@@ -250,6 +250,28 @@ · 249 unmodified lines
250 clock-output-names = "dummy";250 clock-output-names = "dummy";
251 };251 };
252 252
253 gpu_opp_table: opp-table {
254 compatible = "operating-points-v2";
255
256 opp-500000000 {
257 opp-hz = /bits/ 64 <500000000>;
258 opp-hz-real = /bits/ 64 <500000000>;
259 opp-microvolt = <920000>;
260 };
261
262 opp-800000000 {
263 opp-hz = /bits/ 64 <800000000>;
264 opp-hz-real = /bits/ 64 <800000000>;
265 opp-microvolt = <920000>;
266 };
267
268 opp-1000000000 {
269 opp-hz = /bits/ 64 <1000000000>;
270 opp-hz-real = /bits/ 64 <1000000000>;
271 opp-microvolt = <920000>;
272 };
273 };
274
253 clk_ext1: clock-ext1 {275 clk_ext1: clock-ext1 {
254 compatible = "fixed-clock";276 compatible = "fixed-clock";
255 #clock-cells = <0>;277 #clock-cells = <0>;
@@ -2139,6 +2161,21 @@ · 1883 unmodified lines
2139 };2161 };
2140 };2162 };
2141 2163
2164 gpu: gpu@4d900000 {
2165 compatible = "nxp,imx95-mali", "arm,mali-valhall-csf";
2166 reg = <0 0x4d900000 0 0x480000>;
2167 clocks = <&scmi_clk IMX95_CLK_GPU>, <&scmi_clk IMX95_CLK_GPUAPB>;
2168 clock-names = "core", "coregroup";
2169 interrupts = <GIC_SPI 289 IRQ_TYPE_LEVEL_HIGH>,
2170 <GIC_SPI 290 IRQ_TYPE_LEVEL_HIGH>,
2171 <GIC_SPI 288 IRQ_TYPE_LEVEL_HIGH>;
2172 interrupt-names = "job", "mmu", "gpu";
2173 operating-points-v2 = <&gpu_opp_table>;
2174 power-domains = <&scmi_devpd IMX95_PD_GPU>;
2175 #cooling-cells = <2>;
2176 dynamic-power-coefficient = <1013>;
2177 };
2178
2142 ddr-pmu@4e090dc0 {2179 ddr-pmu@4e090dc0 {
2143 compatible = "fsl,imx95-ddr-pmu", "fsl,imx93-ddr-pmu";2180 compatible = "fsl,imx95-ddr-pmu", "fsl,imx93-ddr-pmu";
2144 reg = <0x0 0x4e090dc0 0x0 0x200>;2181 reg = <0x0 0x4e090dc0 0x0 0x200>;
drivers/accel/Kconfig+1−0
@@ -25,6 +25,7 @@ menuconfig DRM_ACCEL · 24 unmodified lines
25 and debugfs).25 and debugfs).
26 26
27source "drivers/accel/amdxdna/Kconfig"27source "drivers/accel/amdxdna/Kconfig"
28source "drivers/accel/ethosu/Kconfig"
28source "drivers/accel/habanalabs/Kconfig"29source "drivers/accel/habanalabs/Kconfig"
29source "drivers/accel/ivpu/Kconfig"30source "drivers/accel/ivpu/Kconfig"
30source "drivers/accel/qaic/Kconfig"31source "drivers/accel/qaic/Kconfig"
drivers/accel/Makefile+1−0
@@ -1,6 +1,7 @@
1# SPDX-License-Identifier: GPL-2.0-only1# SPDX-License-Identifier: GPL-2.0-only
2 2
3obj-$(CONFIG_DRM_ACCEL_AMDXDNA) += amdxdna/3obj-$(CONFIG_DRM_ACCEL_AMDXDNA) += amdxdna/
4obj-$(CONFIG_DRM_ACCEL_ARM_ETHOSU) += ethosu/
4obj-$(CONFIG_DRM_ACCEL_HABANALABS) += habanalabs/5obj-$(CONFIG_DRM_ACCEL_HABANALABS) += habanalabs/
5obj-$(CONFIG_DRM_ACCEL_IVPU) += ivpu/6obj-$(CONFIG_DRM_ACCEL_IVPU) += ivpu/
6obj-$(CONFIG_DRM_ACCEL_QAIC) += qaic/7obj-$(CONFIG_DRM_ACCEL_QAIC) += qaic/
drivers/accel/amdxdna/Makefile+1−0
@@ -14,6 +14,7 @@ amdxdna-y := \ · 13 unmodified lines
14 amdxdna_mailbox.o \14 amdxdna_mailbox.o \
15 amdxdna_mailbox_helper.o \15 amdxdna_mailbox_helper.o \
16 amdxdna_pci_drv.o \16 amdxdna_pci_drv.o \
17 amdxdna_pm.o \
17 amdxdna_sysfs.o \18 amdxdna_sysfs.o \
18 amdxdna_ubuf.o \19 amdxdna_ubuf.o \
19 npu1_regs.o \20 npu1_regs.o \
drivers/accel/amdxdna/TODO+0−1
@@ -1,2 +1 @@
1- Add debugfs support1- Add debugfs support
2- Add debug BO support
drivers/accel/amdxdna/aie2_ctx.c+165−30
@@ -21,6 +21,7 @@ · 20 unmodified lines
21#include "amdxdna_gem.h"21#include "amdxdna_gem.h"
22#include "amdxdna_mailbox.h"22#include "amdxdna_mailbox.h"
23#include "amdxdna_pci_drv.h"23#include "amdxdna_pci_drv.h"
24#include "amdxdna_pm.h"
24 25
25static bool force_cmdlist;26static bool force_cmdlist;
26module_param(force_cmdlist, bool, 0600);27module_param(force_cmdlist, bool, 0600);
@@ -88,7 +89,7 @@ static int aie2_hwctx_restart(struct amdxdna_dev *xdna, struct amdxdna_hwctx *hw · 61 unmodified lines
88 goto out;89 goto out;
89 }90 }
90 91
91 ret = aie2_config_cu(hwctx);92 ret = aie2_config_cu(hwctx, NULL);
92 if (ret) {93 if (ret) {
93 XDNA_ERR(xdna, "Config cu failed, ret %d", ret);94 XDNA_ERR(xdna, "Config cu failed, ret %d", ret);
94 goto out;95 goto out;
@@ -167,14 +168,11 @@ static int aie2_hwctx_resume_cb(struct amdxdna_hwctx *hwctx, void *arg) · 72 unmodified lines
167 168
168int aie2_hwctx_resume(struct amdxdna_client *client)169int aie2_hwctx_resume(struct amdxdna_client *client)
169{170{
170 struct amdxdna_dev *xdna = client->xdna;
171
172 /*171 /*
173 * The resume path cannot guarantee that mailbox channel can be172 * The resume path cannot guarantee that mailbox channel can be
174 * regenerated. If this happen, when submit message to this173 * regenerated. If this happen, when submit message to this
175 * mailbox channel, error will return.174 * mailbox channel, error will return.
176 */175 */
177 drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
178 return amdxdna_hwctx_walk(client, NULL, aie2_hwctx_resume_cb);176 return amdxdna_hwctx_walk(client, NULL, aie2_hwctx_resume_cb);
179}177}
180 178
@@ -184,12 +182,13 @@ aie2_sched_notify(struct amdxdna_sched_job *job) · 3 unmodified lines
184 struct dma_fence *fence = job->fence;182 struct dma_fence *fence = job->fence;
185 183
186 trace_xdna_job(&job->base, job->hwctx->name, "signaled fence", job->seq);184 trace_xdna_job(&job->base, job->hwctx->name, "signaled fence", job->seq);
185
186 amdxdna_pm_suspend_put(job->hwctx->client->xdna);
187 job->hwctx->priv->completed++;187 job->hwctx->priv->completed++;
188 dma_fence_signal(fence);188 dma_fence_signal(fence);
189 189
190 up(&job->hwctx->priv->job_sem);190 up(&job->hwctx->priv->job_sem);
191 job->job_done = true;191 job->job_done = true;
192 dma_fence_put(fence);
193 mmput_async(job->mm);192 mmput_async(job->mm);
194 aie2_job_put(job);193 aie2_job_put(job);
195}194}
@@ -204,10 +203,13 @@ aie2_sched_resp_handler(void *handle, void __iomem *data, size_t size) · 8 unmodified lines
204 203
205 cmd_abo = job->cmd_bo;204 cmd_abo = job->cmd_bo;
206 205
207 if (unlikely(!data))206 if (unlikely(job->job_timeout)) {
207 amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_TIMEOUT);
208 ret = -EINVAL;
208 goto out;209 goto out;
210 }
209 211
210 if (unlikely(size != sizeof(u32))) {212 if (unlikely(!data) || unlikely(size != sizeof(u32))) {
211 amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT);213 amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT);
212 ret = -EINVAL;214 ret = -EINVAL;
213 goto out;215 goto out;
@@ -226,11 +228,10 @@ out: · 12 unmodified lines
226}228}
227 229
228static int230static int
229aie2_sched_nocmd_resp_handler(void *handle, void __iomem *data, size_t size)231aie2_sched_drvcmd_resp_handler(void *handle, void __iomem *data, size_t size)
230{232{
231 struct amdxdna_sched_job *job = handle;233 struct amdxdna_sched_job *job = handle;
232 int ret = 0;234 int ret = 0;
233 u32 status;
234 235
235 if (unlikely(!data))236 if (unlikely(!data))
236 goto out;237 goto out;
@@ -240,8 +241,7 @@ aie2_sched_nocmd_resp_handler(void *handle, void __iomem *data, size_t size) · 3 unmodified lines
240 goto out;241 goto out;
241 }242 }
242 243
243 status = readl(data);244 job->drv_cmd->result = readl(data);
244 XDNA_DBG(job->hwctx->client->xdna, "Resp status 0x%x", status);
245 245
246out:246out:
247 aie2_sched_notify(job);247 aie2_sched_notify(job);
@@ -260,6 +260,13 @@ aie2_sched_cmdlist_resp_handler(void *handle, void __iomem *data, size_t size) · 12 unmodified lines
260 int ret = 0;260 int ret = 0;
261 261
262 cmd_abo = job->cmd_bo;262 cmd_abo = job->cmd_bo;
263
264 if (unlikely(job->job_timeout)) {
265 amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_TIMEOUT);
266 ret = -EINVAL;
267 goto out;
268 }
269
263 if (unlikely(!data) || unlikely(size != sizeof(u32) * 3)) {270 if (unlikely(!data) || unlikely(size != sizeof(u32) * 3)) {
264 amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT);271 amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT);
265 ret = -EINVAL;272 ret = -EINVAL;
@@ -314,8 +321,18 @@ aie2_sched_job_run(struct drm_sched_job *sched_job) · 48 unmodified lines
314 kref_get(&job->refcnt);321 kref_get(&job->refcnt);
315 fence = dma_fence_get(job->fence);322 fence = dma_fence_get(job->fence);
316 323
317 if (unlikely(!cmd_abo)) {324 if (job->drv_cmd) {
318 ret = aie2_sync_bo(hwctx, job, aie2_sched_nocmd_resp_handler);325 switch (job->drv_cmd->opcode) {
326 case SYNC_DEBUG_BO:
327 ret = aie2_sync_bo(hwctx, job, aie2_sched_drvcmd_resp_handler);
328 break;
329 case ATTACH_DEBUG_BO:
330 ret = aie2_config_debug_bo(hwctx, job, aie2_sched_drvcmd_resp_handler);
331 break;
332 default:
333 ret = -EINVAL;
334 break;
335 }
319 goto out;336 goto out;
320 }337 }
321 338
@@ -362,6 +379,7 @@ aie2_sched_job_timedout(struct drm_sched_job *sched_job) · 40 unmodified lines
362 379
363 xdna = hwctx->client->xdna;380 xdna = hwctx->client->xdna;
364 trace_xdna_job(sched_job, hwctx->name, "job timedout", job->seq);381 trace_xdna_job(sched_job, hwctx->name, "job timedout", job->seq);
382 job->job_timeout = true;
365 mutex_lock(&xdna->dev_lock);383 mutex_lock(&xdna->dev_lock);
366 aie2_hwctx_stop(xdna, hwctx, sched_job);384 aie2_hwctx_stop(xdna, hwctx, sched_job);
367 385
@@ -531,13 +549,12 @@ int aie2_hwctx_init(struct amdxdna_hwctx *hwctx) · 163 unmodified lines
531 .num_rqs = DRM_SCHED_PRIORITY_COUNT,549 .num_rqs = DRM_SCHED_PRIORITY_COUNT,
532 .credit_limit = HWCTX_MAX_CMDS,550 .credit_limit = HWCTX_MAX_CMDS,
533 .timeout = msecs_to_jiffies(HWCTX_MAX_TIMEOUT),551 .timeout = msecs_to_jiffies(HWCTX_MAX_TIMEOUT),
534 .name = hwctx->name,552 .name = "amdxdna_js",
535 .dev = xdna->ddev.dev,553 .dev = xdna->ddev.dev,
536 };554 };
537 struct drm_gpu_scheduler *sched;555 struct drm_gpu_scheduler *sched;
538 struct amdxdna_hwctx_priv *priv;556 struct amdxdna_hwctx_priv *priv;
539 struct amdxdna_gem_obj *heap;557 struct amdxdna_gem_obj *heap;
540 struct amdxdna_dev_hdl *ndev;
541 int i, ret;558 int i, ret;
542 559
543 priv = kzalloc(sizeof(*hwctx->priv), GFP_KERNEL);560 priv = kzalloc(sizeof(*hwctx->priv), GFP_KERNEL);
@@ -610,10 +627,14 @@ int aie2_hwctx_init(struct amdxdna_hwctx *hwctx) · 66 unmodified lines
610 goto free_entity;627 goto free_entity;
611 }628 }
612 629
630 ret = amdxdna_pm_resume_get(xdna);
631 if (ret)
632 goto free_col_list;
633
613 ret = aie2_alloc_resource(hwctx);634 ret = aie2_alloc_resource(hwctx);
614 if (ret) {635 if (ret) {
615 XDNA_ERR(xdna, "Alloc hw resource failed, ret %d", ret);636 XDNA_ERR(xdna, "Alloc hw resource failed, ret %d", ret);
616 goto free_col_list;637 goto suspend_put;
617 }638 }
618 639
619 ret = aie2_map_host_buf(xdna->dev_handle, hwctx->fw_ctx_id,640 ret = aie2_map_host_buf(xdna->dev_handle, hwctx->fw_ctx_id,
@@ -628,10 +649,9 @@ int aie2_hwctx_init(struct amdxdna_hwctx *hwctx) · 8 unmodified lines
628 XDNA_ERR(xdna, "Create syncobj failed, ret %d", ret);649 XDNA_ERR(xdna, "Create syncobj failed, ret %d", ret);
629 goto release_resource;650 goto release_resource;
630 }651 }
652 amdxdna_pm_suspend_put(xdna);
631 653
632 hwctx->status = HWCTX_STAT_INIT;654 hwctx->status = HWCTX_STAT_INIT;
633 ndev = xdna->dev_handle;
634 ndev->hwctx_num++;
635 init_waitqueue_head(&priv->job_free_wq);655 init_waitqueue_head(&priv->job_free_wq);
636 656
637 XDNA_DBG(xdna, "hwctx %s init completed", hwctx->name);657 XDNA_DBG(xdna, "hwctx %s init completed", hwctx->name);
@@ -640,6 +660,8 @@ int aie2_hwctx_init(struct amdxdna_hwctx *hwctx) · 2 unmodified lines
640 660
641release_resource:661release_resource:
642 aie2_release_resource(hwctx);662 aie2_release_resource(hwctx);
663suspend_put:
664 amdxdna_pm_suspend_put(xdna);
643free_col_list:665free_col_list:
644 kfree(hwctx->col_list);666 kfree(hwctx->col_list);
645free_entity:667free_entity:
@@ -662,26 +684,25 @@ free_priv: · 16 unmodified lines
662 684
663void aie2_hwctx_fini(struct amdxdna_hwctx *hwctx)685void aie2_hwctx_fini(struct amdxdna_hwctx *hwctx)
664{686{
665 struct amdxdna_dev_hdl *ndev;
666 struct amdxdna_dev *xdna;687 struct amdxdna_dev *xdna;
667 int idx;688 int idx;
668 689
669 xdna = hwctx->client->xdna;690 xdna = hwctx->client->xdna;
670 ndev = xdna->dev_handle;
671 ndev->hwctx_num--;
672 691
673 XDNA_DBG(xdna, "%s sequence number %lld", hwctx->name, hwctx->priv->seq);692 XDNA_DBG(xdna, "%s sequence number %lld", hwctx->name, hwctx->priv->seq);
674 drm_sched_entity_destroy(&hwctx->priv->entity);
675
676 aie2_hwctx_wait_for_idle(hwctx);693 aie2_hwctx_wait_for_idle(hwctx);
677 694
678 /* Request fw to destroy hwctx and cancel the rest pending requests */695 /* Request fw to destroy hwctx and cancel the rest pending requests */
679 aie2_release_resource(hwctx);696 aie2_release_resource(hwctx);
680 697
698 mutex_unlock(&xdna->dev_lock);
699 drm_sched_entity_destroy(&hwctx->priv->entity);
700
681 /* Wait for all submitted jobs to be completed or canceled */701 /* Wait for all submitted jobs to be completed or canceled */
682 wait_event(hwctx->priv->job_free_wq,702 wait_event(hwctx->priv->job_free_wq,
683 atomic64_read(&hwctx->job_submit_cnt) ==703 atomic64_read(&hwctx->job_submit_cnt) ==
684 atomic64_read(&hwctx->job_free_cnt));704 atomic64_read(&hwctx->job_free_cnt));
705 mutex_lock(&xdna->dev_lock);
685 706
686 drm_sched_fini(&hwctx->priv->sched);707 drm_sched_fini(&hwctx->priv->sched);
687 aie2_ctx_syncobj_destroy(hwctx);708 aie2_ctx_syncobj_destroy(hwctx);
@@ -697,6 +718,14 @@ void aie2_hwctx_fini(struct amdxdna_hwctx *hwctx) · 9 unmodified lines
697 kfree(hwctx->cus);718 kfree(hwctx->cus);
698}719}
699 720
721static int aie2_config_cu_resp_handler(void *handle, void __iomem *data, size_t size)
722{
723 struct amdxdna_hwctx *hwctx = handle;
724
725 amdxdna_pm_suspend_put(hwctx->client->xdna);
726 return 0;
727}
728
700static int aie2_hwctx_cu_config(struct amdxdna_hwctx *hwctx, void *buf, u32 size)729static int aie2_hwctx_cu_config(struct amdxdna_hwctx *hwctx, void *buf, u32 size)
701{730{
702 struct amdxdna_hwctx_param_config_cu *config = buf;731 struct amdxdna_hwctx_param_config_cu *config = buf;
@@ -728,10 +757,14 @@ static int aie2_hwctx_cu_config(struct amdxdna_hwctx *hwctx, void *buf, u32 size · 25 unmodified lines
728 if (!hwctx->cus)757 if (!hwctx->cus)
729 return -ENOMEM;758 return -ENOMEM;
730 759
731 ret = aie2_config_cu(hwctx);760 ret = amdxdna_pm_resume_get(xdna);
761 if (ret)
762 goto free_cus;
763
764 ret = aie2_config_cu(hwctx, aie2_config_cu_resp_handler);
732 if (ret) {765 if (ret) {
733 XDNA_ERR(xdna, "Config CU to firmware failed, ret %d", ret);766 XDNA_ERR(xdna, "Config CU to firmware failed, ret %d", ret);
734 goto free_cus;767 goto pm_suspend_put;
735 }768 }
736 769
737 wmb(); /* To avoid locking in command submit when check status */770 wmb(); /* To avoid locking in command submit when check status */
@@ -739,12 +772,82 @@ static int aie2_hwctx_cu_config(struct amdxdna_hwctx *hwctx, void *buf, u32 size · 1 unmodified line
739 772
740 return 0;773 return 0;
741 774
775pm_suspend_put:
776 amdxdna_pm_suspend_put(xdna);
742free_cus:777free_cus:
743 kfree(hwctx->cus);778 kfree(hwctx->cus);
744 hwctx->cus = NULL;779 hwctx->cus = NULL;
745 return ret;780 return ret;
746}781}
747 782
783static void aie2_cmd_wait(struct amdxdna_hwctx *hwctx, u64 seq)
784{
785 struct dma_fence *out_fence = aie2_cmd_get_out_fence(hwctx, seq);
786
787 if (!out_fence) {
788 XDNA_ERR(hwctx->client->xdna, "Failed to get fence");
789 return;
790 }
791
792 dma_fence_wait_timeout(out_fence, false, MAX_SCHEDULE_TIMEOUT);
793 dma_fence_put(out_fence);
794}
795
796static int aie2_hwctx_cfg_debug_bo(struct amdxdna_hwctx *hwctx, u32 bo_hdl,
797 bool attach)
798{
799 struct amdxdna_client *client = hwctx->client;
800 struct amdxdna_dev *xdna = client->xdna;
801 struct amdxdna_drv_cmd cmd = { 0 };
802 struct amdxdna_gem_obj *abo;
803 u64 seq;
804 int ret;
805
806 abo = amdxdna_gem_get_obj(client, bo_hdl, AMDXDNA_BO_DEV);
807 if (!abo) {
808 XDNA_ERR(xdna, "Get bo %d failed", bo_hdl);
809 return -EINVAL;
810 }
811
812 if (attach) {
813 if (abo->assigned_hwctx != AMDXDNA_INVALID_CTX_HANDLE) {
814 ret = -EBUSY;
815 goto put_obj;
816 }
817 cmd.opcode = ATTACH_DEBUG_BO;
818 } else {
819 if (abo->assigned_hwctx != hwctx->id) {
820 ret = -EINVAL;
821 goto put_obj;
822 }
823 cmd.opcode = DETACH_DEBUG_BO;
824 }
825
826 ret = amdxdna_cmd_submit(client, &cmd, AMDXDNA_INVALID_BO_HANDLE,
827 &bo_hdl, 1, hwctx->id, &seq);
828 if (ret) {
829 XDNA_ERR(xdna, "Submit command failed");
830 goto put_obj;
831 }
832
833 aie2_cmd_wait(hwctx, seq);
834 if (cmd.result) {
835 XDNA_ERR(xdna, "Response failure 0x%x", cmd.result);
836 goto put_obj;
837 }
838
839 if (attach)
840 abo->assigned_hwctx = hwctx->id;
841 else
842 abo->assigned_hwctx = AMDXDNA_INVALID_CTX_HANDLE;
843
844 XDNA_DBG(xdna, "Config debug BO %d to %s", bo_hdl, hwctx->name);
845
846put_obj:
847 amdxdna_gem_put_obj(abo);
848 return ret;
849}
850
748int aie2_hwctx_config(struct amdxdna_hwctx *hwctx, u32 type, u64 value, void *buf, u32 size)851int aie2_hwctx_config(struct amdxdna_hwctx *hwctx, u32 type, u64 value, void *buf, u32 size)
749{852{
750 struct amdxdna_dev *xdna = hwctx->client->xdna;853 struct amdxdna_dev *xdna = hwctx->client->xdna;
@@ -754,14 +857,40 @@ int aie2_hwctx_config(struct amdxdna_hwctx *hwctx, u32 type, u64 value, void *bu · 3 unmodified lines
754 case DRM_AMDXDNA_HWCTX_CONFIG_CU:857 case DRM_AMDXDNA_HWCTX_CONFIG_CU:
755 return aie2_hwctx_cu_config(hwctx, buf, size);858 return aie2_hwctx_cu_config(hwctx, buf, size);
756 case DRM_AMDXDNA_HWCTX_ASSIGN_DBG_BUF:859 case DRM_AMDXDNA_HWCTX_ASSIGN_DBG_BUF:
860 return aie2_hwctx_cfg_debug_bo(hwctx, (u32)value, true);
757 case DRM_AMDXDNA_HWCTX_REMOVE_DBG_BUF:861 case DRM_AMDXDNA_HWCTX_REMOVE_DBG_BUF:
758 return -EOPNOTSUPP;862 return aie2_hwctx_cfg_debug_bo(hwctx, (u32)value, false);
759 default:863 default:
760 XDNA_DBG(xdna, "Not supported type %d", type);864 XDNA_DBG(xdna, "Not supported type %d", type);
761 return -EOPNOTSUPP;865 return -EOPNOTSUPP;
762 }866 }
763}867}
764 868
869int aie2_hwctx_sync_debug_bo(struct amdxdna_hwctx *hwctx, u32 debug_bo_hdl)
870{
871 struct amdxdna_client *client = hwctx->client;
872 struct amdxdna_dev *xdna = client->xdna;
873 struct amdxdna_drv_cmd cmd = { 0 };
874 u64 seq;
875 int ret;
876
877 cmd.opcode = SYNC_DEBUG_BO;
878 ret = amdxdna_cmd_submit(client, &cmd, AMDXDNA_INVALID_BO_HANDLE,
879 &debug_bo_hdl, 1, hwctx->id, &seq);
880 if (ret) {
881 XDNA_ERR(xdna, "Submit command failed");
882 return ret;
883 }
884
885 aie2_cmd_wait(hwctx, seq);
886 if (cmd.result) {
887 XDNA_ERR(xdna, "Response failure 0x%x", cmd.result);
888 return -EINVAL;
889 }
890
891 return 0;
892}
893
765static int aie2_populate_range(struct amdxdna_gem_obj *abo)894static int aie2_populate_range(struct amdxdna_gem_obj *abo)
766{895{
767 struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev);896 struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev);
@@ -862,11 +991,15 @@ int aie2_cmd_submit(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job, · 94 unmodified lines
862 goto free_chain;991 goto free_chain;
863 }992 }
864 993
994 ret = amdxdna_pm_resume_get(xdna);
995 if (ret)
996 goto cleanup_job;
997
865retry:998retry:
866 ret = drm_gem_lock_reservations(job->bos, job->bo_cnt, &acquire_ctx);999 ret = drm_gem_lock_reservations(job->bos, job->bo_cnt, &acquire_ctx);
867 if (ret) {1000 if (ret) {
868 XDNA_WARN(xdna, "Failed to lock BOs, ret %d", ret);1001 XDNA_WARN(xdna, "Failed to lock BOs, ret %d", ret);
869 goto cleanup_job;1002 goto suspend_put;
870 }1003 }
871 1004
872 for (i = 0; i < job->bo_cnt; i++) {1005 for (i = 0; i < job->bo_cnt; i++) {
@@ -874,7 +1007,7 @@ retry: · 1 unmodified line
874 if (ret) {1007 if (ret) {
875 XDNA_WARN(xdna, "Failed to reserve fences %d", ret);1008 XDNA_WARN(xdna, "Failed to reserve fences %d", ret);
876 drm_gem_unlock_reservations(job->bos, job->bo_cnt, &acquire_ctx);1009 drm_gem_unlock_reservations(job->bos, job->bo_cnt, &acquire_ctx);
877 goto cleanup_job;1010 goto suspend_put;
878 }1011 }
879 }1012 }
880 1013
@@ -889,12 +1022,12 @@ retry: · 8 unmodified lines
889 msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);1022 msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
890 } else if (time_after(jiffies, timeout)) {1023 } else if (time_after(jiffies, timeout)) {
891 ret = -ETIME;1024 ret = -ETIME;
892 goto cleanup_job;1025 goto suspend_put;
893 }1026 }
894 1027
895 ret = aie2_populate_range(abo);1028 ret = aie2_populate_range(abo);
896 if (ret)1029 if (ret)
897 goto cleanup_job;1030 goto suspend_put;
898 goto retry;1031 goto retry;
899 }1032 }
900 }1033 }
@@ -920,6 +1053,8 @@ retry: · 19 unmodified lines
920 1053
921 return 0;1054 return 0;
922 1055
1056suspend_put:
1057 amdxdna_pm_suspend_put(xdna);
923cleanup_job:1058cleanup_job:
924 drm_sched_job_cleanup(&job->base);1059 drm_sched_job_cleanup(&job->base);
925free_chain:1060free_chain:
drivers/accel/amdxdna/aie2_error.c+78−17
@@ -13,6 +13,7 @@ · 12 unmodified lines
13 13
14#include "aie2_msg_priv.h"14#include "aie2_msg_priv.h"
15#include "aie2_pci.h"15#include "aie2_pci.h"
16#include "amdxdna_error.h"
16#include "amdxdna_mailbox.h"17#include "amdxdna_mailbox.h"
17#include "amdxdna_pci_drv.h"18#include "amdxdna_pci_drv.h"
18 19
@@ -46,6 +47,7 @@ enum aie_module_type { · 27 unmodified lines
46 AIE_MEM_MOD = 0,47 AIE_MEM_MOD = 0,
47 AIE_CORE_MOD,48 AIE_CORE_MOD,
48 AIE_PL_MOD,49 AIE_PL_MOD,
50 AIE_UNKNOWN_MOD,
49};51};
50 52
51enum aie_error_category {53enum aie_error_category {
@@ -143,6 +145,31 @@ static const struct aie_event_category aie_ml_shim_tile_event_cat[] = { · 91 unmodified lines
143 EVENT_CATEGORY(74U, AIE_ERROR_LOCK),145 EVENT_CATEGORY(74U, AIE_ERROR_LOCK),
144};146};
145 147
148static const enum amdxdna_error_num aie_cat_err_num_map[] = {
149 [AIE_ERROR_SATURATION] = AMDXDNA_ERROR_NUM_AIE_SATURATION,
150 [AIE_ERROR_FP] = AMDXDNA_ERROR_NUM_AIE_FP,
151 [AIE_ERROR_STREAM] = AMDXDNA_ERROR_NUM_AIE_STREAM,
152 [AIE_ERROR_ACCESS] = AMDXDNA_ERROR_NUM_AIE_ACCESS,
153 [AIE_ERROR_BUS] = AMDXDNA_ERROR_NUM_AIE_BUS,
154 [AIE_ERROR_INSTRUCTION] = AMDXDNA_ERROR_NUM_AIE_INSTRUCTION,
155 [AIE_ERROR_ECC] = AMDXDNA_ERROR_NUM_AIE_ECC,
156 [AIE_ERROR_LOCK] = AMDXDNA_ERROR_NUM_AIE_LOCK,
157 [AIE_ERROR_DMA] = AMDXDNA_ERROR_NUM_AIE_DMA,
158 [AIE_ERROR_MEM_PARITY] = AMDXDNA_ERROR_NUM_AIE_MEM_PARITY,
159 [AIE_ERROR_UNKNOWN] = AMDXDNA_ERROR_NUM_UNKNOWN,
160};
161
162static_assert(ARRAY_SIZE(aie_cat_err_num_map) == AIE_ERROR_UNKNOWN + 1);
163
164static const enum amdxdna_error_module aie_err_mod_map[] = {
165 [AIE_MEM_MOD] = AMDXDNA_ERROR_MODULE_AIE_MEMORY,
166 [AIE_CORE_MOD] = AMDXDNA_ERROR_MODULE_AIE_CORE,
167 [AIE_PL_MOD] = AMDXDNA_ERROR_MODULE_AIE_PL,
168 [AIE_UNKNOWN_MOD] = AMDXDNA_ERROR_MODULE_UNKNOWN,
169};
170
171static_assert(ARRAY_SIZE(aie_err_mod_map) == AIE_UNKNOWN_MOD + 1);
172
146static enum aie_error_category173static enum aie_error_category
147aie_get_error_category(u8 row, u8 event_id, enum aie_module_type mod_type)174aie_get_error_category(u8 row, u8 event_id, enum aie_module_type mod_type)
148{175{
@@ -176,12 +203,40 @@ aie_get_error_category(u8 row, u8 event_id, enum aie_module_type mod_type) · 27 unmodified lines
176 if (event_id != lut[i].event_id)203 if (event_id != lut[i].event_id)
177 continue;204 continue;
178 205
206 if (lut[i].category > AIE_ERROR_UNKNOWN)
207 return AIE_ERROR_UNKNOWN;
208
179 return lut[i].category;209 return lut[i].category;
180 }210 }
181 211
182 return AIE_ERROR_UNKNOWN;212 return AIE_ERROR_UNKNOWN;
183}213}
184 214
215static void aie2_update_last_async_error(struct amdxdna_dev_hdl *ndev, void *err_info, u32 num_err)
216{
217 struct aie_error *errs = err_info;
218 enum amdxdna_error_module err_mod;
219 enum aie_error_category aie_err;
220 enum amdxdna_error_num err_num;
221 struct aie_error *last_err;
222
223 last_err = &errs[num_err - 1];
224 if (last_err->mod_type >= AIE_UNKNOWN_MOD) {
225 err_num = aie_cat_err_num_map[AIE_ERROR_UNKNOWN];
226 err_mod = aie_err_mod_map[AIE_UNKNOWN_MOD];
227 } else {
228 aie_err = aie_get_error_category(last_err->row,
229 last_err->event_id,
230 last_err->mod_type);
231 err_num = aie_cat_err_num_map[aie_err];
232 err_mod = aie_err_mod_map[last_err->mod_type];
233 }
234
235 ndev->last_async_err.err_code = AMDXDNA_ERROR_ENCODE(err_num, err_mod);
236 ndev->last_async_err.ts_us = ktime_to_us(ktime_get_real());
237 ndev->last_async_err.ex_err_code = AMDXDNA_EXTRA_ERR_ENCODE(last_err->row, last_err->col);
238}
239
185static u32 aie2_error_backtrack(struct amdxdna_dev_hdl *ndev, void *err_info, u32 num_err)240static u32 aie2_error_backtrack(struct amdxdna_dev_hdl *ndev, void *err_info, u32 num_err)
186{241{
187 struct aie_error *errs = err_info;242 struct aie_error *errs = err_info;
@@ -264,29 +319,14 @@ static void aie2_error_worker(struct work_struct *err_work) · 76 unmodified lines
264 }319 }
265 320
266 mutex_lock(&xdna->dev_lock);321 mutex_lock(&xdna->dev_lock);
322 aie2_update_last_async_error(e->ndev, info->payload, info->err_cnt);
323
267 /* Re-sent this event to firmware */324 /* Re-sent this event to firmware */
268 if (aie2_error_event_send(e))325 if (aie2_error_event_send(e))
269 XDNA_WARN(xdna, "Unable to register async event");326 XDNA_WARN(xdna, "Unable to register async event");
270 mutex_unlock(&xdna->dev_lock);327 mutex_unlock(&xdna->dev_lock);
271}328}
272 329
273int aie2_error_async_events_send(struct amdxdna_dev_hdl *ndev)
274{
275 struct amdxdna_dev *xdna = ndev->xdna;
276 struct async_event *e;
277 int i, ret;
278
279 drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
280 for (i = 0; i < ndev->async_events->event_cnt; i++) {
281 e = &ndev->async_events->event[i];
282 ret = aie2_error_event_send(e);
283 if (ret)
284 return ret;
285 }
286
287 return 0;
288}
289
290void aie2_error_async_events_free(struct amdxdna_dev_hdl *ndev)330void aie2_error_async_events_free(struct amdxdna_dev_hdl *ndev)
291{331{
292 struct amdxdna_dev *xdna = ndev->xdna;332 struct amdxdna_dev *xdna = ndev->xdna;
@@ -341,6 +381,10 @@ int aie2_error_async_events_alloc(struct amdxdna_dev_hdl *ndev) · 48 unmodified lines
341 e->size = ASYNC_BUF_SIZE;381 e->size = ASYNC_BUF_SIZE;
342 e->resp.status = MAX_AIE2_STATUS_CODE;382 e->resp.status = MAX_AIE2_STATUS_CODE;
343 INIT_WORK(&e->work, aie2_error_worker);383 INIT_WORK(&e->work, aie2_error_worker);
384
385 ret = aie2_error_event_send(e);
386 if (ret)
387 goto free_wq;
344 }388 }
345 389
346 ndev->async_events = events;390 ndev->async_events = events;
@@ -349,6 +393,8 @@ int aie2_error_async_events_alloc(struct amdxdna_dev_hdl *ndev) · 2 unmodified lines
349 events->event_cnt, events->size);393 events->event_cnt, events->size);
350 return 0;394 return 0;
351 395
396free_wq:
397 destroy_workqueue(events->wq);
352free_buf:398free_buf:
353 dma_free_noncoherent(xdna->ddev.dev, events->size, events->buf,399 dma_free_noncoherent(xdna->ddev.dev, events->size, events->buf,
354 events->addr, DMA_FROM_DEVICE);400 events->addr, DMA_FROM_DEVICE);
@@ -356,3 +402,18 @@ free_events: · 1 unmodified line
356 kfree(events);402 kfree(events);
357 return ret;403 return ret;
358}404}
405
406int aie2_get_array_async_error(struct amdxdna_dev_hdl *ndev, struct amdxdna_drm_get_array *args)
407{
408 struct amdxdna_dev *xdna = ndev->xdna;
409
410 drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
411
412 args->num_element = 1;
413 args->element_size = sizeof(ndev->last_async_err);
414 if (copy_to_user(u64_to_user_ptr(args->buffer),
415 &ndev->last_async_err, args->element_size))
416 return -EFAULT;
417
418 return 0;
419}
drivers/accel/amdxdna/aie2_message.c+471−176
@@ -27,6 +27,8 @@ · 26 unmodified lines
27#define DECLARE_AIE2_MSG(name, op) \27#define DECLARE_AIE2_MSG(name, op) \
28 DECLARE_XDNA_MSG_COMMON(name, op, MAX_AIE2_STATUS_CODE)28 DECLARE_XDNA_MSG_COMMON(name, op, MAX_AIE2_STATUS_CODE)
29 29
30#define EXEC_MSG_OPS(xdna) ((xdna)->dev_handle->exec_msg_ops)
31
30static int aie2_send_mgmt_msg_wait(struct amdxdna_dev_hdl *ndev,32static int aie2_send_mgmt_msg_wait(struct amdxdna_dev_hdl *ndev,
31 struct xdna_mailbox_msg *msg)33 struct xdna_mailbox_msg *msg)
32{34{
@@ -37,7 +39,7 @@ static int aie2_send_mgmt_msg_wait(struct amdxdna_dev_hdl *ndev, · 4 unmodified lines
37 if (!ndev->mgmt_chann)39 if (!ndev->mgmt_chann)
38 return -ENODEV;40 return -ENODEV;
39 41
40 drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));42 drm_WARN_ON(&xdna->ddev, xdna->rpm_on && !mutex_is_locked(&xdna->dev_lock));
41 ret = xdna_send_msg_wait(xdna, ndev->mgmt_chann, msg);43 ret = xdna_send_msg_wait(xdna, ndev->mgmt_chann, msg);
42 if (ret == -ETIME) {44 if (ret == -ETIME) {
43 xdna_mailbox_stop_channel(ndev->mgmt_chann);45 xdna_mailbox_stop_channel(ndev->mgmt_chann);
@@ -45,7 +47,7 @@ static int aie2_send_mgmt_msg_wait(struct amdxdna_dev_hdl *ndev, · 1 unmodified line
45 ndev->mgmt_chann = NULL;47 ndev->mgmt_chann = NULL;
46 }48 }
47 49
48 if (!ret && *hdl->data != AIE2_STATUS_SUCCESS) {50 if (!ret && *hdl->status != AIE2_STATUS_SUCCESS) {
49 XDNA_ERR(xdna, "command opcode 0x%x failed, status 0x%x",51 XDNA_ERR(xdna, "command opcode 0x%x failed, status 0x%x",
50 msg->opcode, *hdl->data);52 msg->opcode, *hdl->data);
51 ret = -EINVAL;53 ret = -EINVAL;
@@ -208,6 +210,14 @@ int aie2_create_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwct · 156 unmodified lines
208 hwctx->fw_ctx_id = resp.context_id;210 hwctx->fw_ctx_id = resp.context_id;
209 WARN_ONCE(hwctx->fw_ctx_id == -1, "Unexpected context id");211 WARN_ONCE(hwctx->fw_ctx_id == -1, "Unexpected context id");
210 212
213 if (ndev->force_preempt_enabled) {
214 ret = aie2_runtime_cfg(ndev, AIE2_RT_CFG_FORCE_PREEMPT, &hwctx->fw_ctx_id);
215 if (ret) {
216 XDNA_ERR(xdna, "failed to enable force preempt %d", ret);
217 return ret;
218 }
219 }
220
211 cq_pair = &resp.cq_pair[0];221 cq_pair = &resp.cq_pair[0];
212 x2i.mb_head_ptr_reg = AIE2_MBOX_OFF(ndev, cq_pair->x2i_q.head_addr);222 x2i.mb_head_ptr_reg = AIE2_MBOX_OFF(ndev, cq_pair->x2i_q.head_addr);
213 x2i.mb_tail_ptr_reg = AIE2_MBOX_OFF(ndev, cq_pair->x2i_q.tail_addr);223 x2i.mb_tail_ptr_reg = AIE2_MBOX_OFF(ndev, cq_pair->x2i_q.tail_addr);
@@ -233,6 +243,7 @@ int aie2_create_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwct · 19 unmodified lines
233 ret = -EINVAL;243 ret = -EINVAL;
234 goto out_destroy_context;244 goto out_destroy_context;
235 }245 }
246 ndev->hwctx_num++;
236 247
237 XDNA_DBG(xdna, "%s mailbox channel irq: %d, msix_id: %d",248 XDNA_DBG(xdna, "%s mailbox channel irq: %d, msix_id: %d",
238 hwctx->name, ret, resp.msix_id);249 hwctx->name, ret, resp.msix_id);
@@ -267,6 +278,7 @@ int aie2_destroy_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwc · 28 unmodified lines
267 hwctx->fw_ctx_id);278 hwctx->fw_ctx_id);
268 hwctx->priv->mbox_chann = NULL;279 hwctx->priv->mbox_chann = NULL;
269 hwctx->fw_ctx_id = -1;280 hwctx->fw_ctx_id = -1;
281 ndev->hwctx_num--;
270 282
271 return ret;283 return ret;
272}284}
@@ -332,11 +344,6 @@ int aie2_query_status(struct amdxdna_dev_hdl *ndev, char __user *buf, · 59 unmodified lines
332 goto fail;344 goto fail;
333 }345 }
334 346
335 if (resp.status != AIE2_STATUS_SUCCESS) {
336 XDNA_ERR(xdna, "Query NPU status failed, status 0x%x", resp.status);
337 ret = -EINVAL;
338 goto fail;
339 }
340 XDNA_DBG(xdna, "Query NPU status completed");347 XDNA_DBG(xdna, "Query NPU status completed");
341 348
342 if (size < resp.size) {349 if (size < resp.size) {
@@ -358,6 +365,55 @@ fail: · 15 unmodified lines
358 return ret;365 return ret;
359}366}
360 367
368int aie2_query_telemetry(struct amdxdna_dev_hdl *ndev,
369 char __user *buf, u32 size,
370 struct amdxdna_drm_query_telemetry_header *header)
371{
372 DECLARE_AIE2_MSG(get_telemetry, MSG_OP_GET_TELEMETRY);
373 struct amdxdna_dev *xdna = ndev->xdna;
374 dma_addr_t dma_addr;
375 u8 *addr;
376 int ret;
377
378 if (header->type >= MAX_TELEMETRY_TYPE)
379 return -EINVAL;
380
381 addr = dma_alloc_noncoherent(xdna->ddev.dev, size, &dma_addr,
382 DMA_FROM_DEVICE, GFP_KERNEL);
383 if (!addr)
384 return -ENOMEM;
385
386 req.buf_addr = dma_addr;
387 req.buf_size = size;
388 req.type = header->type;
389
390 drm_clflush_virt_range(addr, size); /* device can access */
391 ret = aie2_send_mgmt_msg_wait(ndev, &msg);
392 if (ret) {
393 XDNA_ERR(xdna, "Query telemetry failed, status %d", ret);
394 goto free_buf;
395 }
396
397 if (size < resp.size) {
398 ret = -EINVAL;
399 XDNA_ERR(xdna, "Bad buffer size. Available: %u. Needs: %u", size, resp.size);
400 goto free_buf;
401 }
402
403 if (copy_to_user(buf, addr, resp.size)) {
404 ret = -EFAULT;
405 XDNA_ERR(xdna, "Failed to copy telemetry to user space");
406 goto free_buf;
407 }
408
409 header->major = resp.major;
410 header->minor = resp.minor;
411
412free_buf:
413 dma_free_noncoherent(xdna->ddev.dev, size, addr, dma_addr, DMA_FROM_DEVICE);
414 return ret;
415}
416
361int aie2_register_asyn_event_msg(struct amdxdna_dev_hdl *ndev, dma_addr_t addr, u32 size,417int aie2_register_asyn_event_msg(struct amdxdna_dev_hdl *ndev, dma_addr_t addr, u32 size,
362 void *handle, int (*cb)(void*, void __iomem *, size_t))418 void *handle, int (*cb)(void*, void __iomem *, size_t))
363{419{
@@ -377,15 +433,17 @@ int aie2_register_asyn_event_msg(struct amdxdna_dev_hdl *ndev, dma_addr_t addr, · 13 unmodified lines
377 return xdna_mailbox_send_msg(ndev->mgmt_chann, &msg, TX_TIMEOUT);433 return xdna_mailbox_send_msg(ndev->mgmt_chann, &msg, TX_TIMEOUT);
378}434}
379 435
380int aie2_config_cu(struct amdxdna_hwctx *hwctx)436int aie2_config_cu(struct amdxdna_hwctx *hwctx,
437 int (*notify_cb)(void *, void __iomem *, size_t))
381{438{
382 struct mailbox_channel *chann = hwctx->priv->mbox_chann;439 struct mailbox_channel *chann = hwctx->priv->mbox_chann;
383 struct amdxdna_dev *xdna = hwctx->client->xdna;440 struct amdxdna_dev *xdna = hwctx->client->xdna;
384 u32 shift = xdna->dev_info->dev_mem_buf_shift;441 u32 shift = xdna->dev_info->dev_mem_buf_shift;
385 DECLARE_AIE2_MSG(config_cu, MSG_OP_CONFIG_CU);442 struct config_cu_req req = { 0 };
443 struct xdna_mailbox_msg msg;
386 struct drm_gem_object *gobj;444 struct drm_gem_object *gobj;
387 struct amdxdna_gem_obj *abo;445 struct amdxdna_gem_obj *abo;
388 int ret, i;446 int i;
389 447
390 if (!chann)448 if (!chann)
391 return -ENODEV;449 return -ENODEV;
@@ -423,191 +481,386 @@ int aie2_config_cu(struct amdxdna_hwctx *hwctx) · 31 unmodified lines
423 }481 }
424 req.num_cus = hwctx->cus->num_cus;482 req.num_cus = hwctx->cus->num_cus;
425 483
426 ret = xdna_send_msg_wait(xdna, chann, &msg);484 msg.send_data = (u8 *)&req;
427 if (ret == -ETIME)485 msg.send_size = sizeof(req);
428 aie2_destroy_context(xdna->dev_handle, hwctx);486 msg.handle = hwctx;
487 msg.opcode = MSG_OP_CONFIG_CU;
488 msg.notify_cb = notify_cb;
489 return xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);
490}
429 491
430 if (resp.status == AIE2_STATUS_SUCCESS) {492static int aie2_init_exec_cu_req(struct amdxdna_gem_obj *cmd_bo, void *req,
431 XDNA_DBG(xdna, "Configure %d CUs, ret %d", req.num_cus, ret);493 size_t *size, u32 *msg_op)
432 return 0;494{
433 }495 struct execute_buffer_req *cu_req = req;
496 u32 cmd_len;
497 void *cmd;
434 498
435 XDNA_ERR(xdna, "Command opcode 0x%x failed, status 0x%x ret %d",499 cmd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len);
436 msg.opcode, resp.status, ret);500 if (cmd_len > sizeof(cu_req->payload))
437 return ret;501 return -EINVAL;
502
503 cu_req->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo);
504 if (cu_req->cu_idx == INVALID_CU_IDX)
505 return -EINVAL;
506
507 memcpy(cu_req->payload, cmd, cmd_len);
508
509 *size = sizeof(*cu_req);
510 *msg_op = MSG_OP_EXECUTE_BUFFER_CF;
511 return 0;
438}512}
439 513
440int aie2_execbuf(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job,514static int aie2_init_exec_dpu_req(struct amdxdna_gem_obj *cmd_bo, void *req,
441 int (*notify_cb)(void *, void __iomem *, size_t))515 size_t *size, u32 *msg_op)
442{516{
443 struct mailbox_channel *chann = hwctx->priv->mbox_chann;517 struct exec_dpu_req *dpu_req = req;
444 struct amdxdna_dev *xdna = hwctx->client->xdna;518 struct amdxdna_cmd_start_npu *sn;
445 struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;519 u32 cmd_len;
446 union {
447 struct execute_buffer_req ebuf;
448 struct exec_dpu_req dpu;
449 } req;
450 struct xdna_mailbox_msg msg;
451 u32 payload_len;
452 void *payload;
453 int cu_idx;
454 int ret;
455 u32 op;
456 520
457 if (!chann)521 sn = amdxdna_cmd_get_payload(cmd_bo, &cmd_len);
458 return -ENODEV;522 if (cmd_len - sizeof(*sn) > sizeof(dpu_req->payload))
523 return -EINVAL;
459 524
460 payload = amdxdna_cmd_get_payload(cmd_abo, &payload_len);525 dpu_req->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo);
461 if (!payload) {526 if (dpu_req->cu_idx == INVALID_CU_IDX)
462 XDNA_ERR(xdna, "Invalid command, cannot get payload");
463 return -EINVAL;527 return -EINVAL;
464 }
465 528
466 cu_idx = amdxdna_cmd_get_cu_idx(cmd_abo);529 dpu_req->inst_buf_addr = sn->buffer;
467 if (cu_idx < 0) {530 dpu_req->inst_size = sn->buffer_size;
468 XDNA_DBG(xdna, "Invalid cu idx");531 dpu_req->inst_prop_cnt = sn->prop_count;
532 memcpy(dpu_req->payload, sn->prop_args, cmd_len - sizeof(*sn));
533
534 *size = sizeof(*dpu_req);
535 *msg_op = MSG_OP_EXEC_DPU;
536 return 0;
537}
538
539static void aie2_init_exec_chain_req(void *req, u64 slot_addr, size_t size, u32 cmd_cnt)
540{
541 struct cmd_chain_req *chain_req = req;
542
543 chain_req->buf_addr = slot_addr;
544 chain_req->buf_size = size;
545 chain_req->count = cmd_cnt;
546}
547
548static void aie2_init_npu_chain_req(void *req, u64 slot_addr, size_t size, u32 cmd_cnt)
549{
550 struct cmd_chain_npu_req *npu_chain_req = req;
551
552 npu_chain_req->flags = 0;
553 npu_chain_req->reserved = 0;
554 npu_chain_req->buf_addr = slot_addr;
555 npu_chain_req->buf_size = size;
556 npu_chain_req->count = cmd_cnt;
557}
558
559static int
560aie2_cmdlist_fill_cf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size)
561{
562 struct cmd_chain_slot_execbuf_cf *cf_slot = slot;
563 u32 cmd_len;
564 void *cmd;
565
566 cmd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len);
567 if (*size < sizeof(*cf_slot) + cmd_len)
469 return -EINVAL;568 return -EINVAL;
470 }
471 569
472 op = amdxdna_cmd_get_op(cmd_abo);570 cf_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo);
473 switch (op) {571 if (cf_slot->cu_idx == INVALID_CU_IDX)
572 return -EINVAL;
573
574 cf_slot->arg_cnt = cmd_len / sizeof(u32);
575 memcpy(cf_slot->args, cmd, cmd_len);
576 /* Accurate slot size to hint firmware to do necessary copy */
577 *size = sizeof(*cf_slot) + cmd_len;
578 return 0;
579}
580
581static int
582aie2_cmdlist_fill_dpu(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size)
583{
584 struct cmd_chain_slot_dpu *dpu_slot = slot;
585 struct amdxdna_cmd_start_npu *sn;
586 u32 cmd_len;
587 u32 arg_sz;
588
589 sn = amdxdna_cmd_get_payload(cmd_bo, &cmd_len);
590 arg_sz = cmd_len - sizeof(*sn);
591 if (cmd_len < sizeof(*sn) || arg_sz > MAX_DPU_ARGS_SIZE)
592 return -EINVAL;
593
594 if (*size < sizeof(*dpu_slot) + arg_sz)
595 return -EINVAL;
596
597 dpu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo);
598 if (dpu_slot->cu_idx == INVALID_CU_IDX)
599 return -EINVAL;
600
601 dpu_slot->inst_buf_addr = sn->buffer;
602 dpu_slot->inst_size = sn->buffer_size;
603 dpu_slot->inst_prop_cnt = sn->prop_count;
604 dpu_slot->arg_cnt = arg_sz / sizeof(u32);
605 memcpy(dpu_slot->args, sn->prop_args, arg_sz);
606
607 /* Accurate slot size to hint firmware to do necessary copy */
608 *size = sizeof(*dpu_slot) + arg_sz;
609 return 0;
610}
611
612static int aie2_cmdlist_unsupp(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size)
613{
614 return -EOPNOTSUPP;
615}
616
617static u32 aie2_get_chain_msg_op(u32 cmd_op)
618{
619 switch (cmd_op) {
474 case ERT_START_CU:620 case ERT_START_CU:
475 if (unlikely(payload_len > sizeof(req.ebuf.payload)))621 return MSG_OP_CHAIN_EXEC_BUFFER_CF;
476 XDNA_DBG(xdna, "Invalid ebuf payload len: %d", payload_len);622 case ERT_START_NPU:
477 req.ebuf.cu_idx = cu_idx;623 return MSG_OP_CHAIN_EXEC_DPU;
478 memcpy(req.ebuf.payload, payload, sizeof(req.ebuf.payload));624 default:
479 msg.send_size = sizeof(req.ebuf);
480 msg.opcode = MSG_OP_EXECUTE_BUFFER_CF;
481 break;
482 case ERT_START_NPU: {
483 struct amdxdna_cmd_start_npu *sn = payload;
484
485 if (unlikely(payload_len - sizeof(*sn) > sizeof(req.dpu.payload)))
486 XDNA_DBG(xdna, "Invalid dpu payload len: %d", payload_len);
487 req.dpu.inst_buf_addr = sn->buffer;
488 req.dpu.inst_size = sn->buffer_size;
489 req.dpu.inst_prop_cnt = sn->prop_count;
490 req.dpu.cu_idx = cu_idx;
491 memcpy(req.dpu.payload, sn->prop_args, sizeof(req.dpu.payload));
492 msg.send_size = sizeof(req.dpu);
493 msg.opcode = MSG_OP_EXEC_DPU;
494 break;625 break;
495 }626 }
496 default:627
497 XDNA_DBG(xdna, "Invalid ERT cmd op code: %d", op);628 return MSG_OP_MAX_OPCODE;
629}
630
631static struct aie2_exec_msg_ops legacy_exec_message_ops = {
632 .init_cu_req = aie2_init_exec_cu_req,
633 .init_dpu_req = aie2_init_exec_dpu_req,
634 .init_chain_req = aie2_init_exec_chain_req,
635 .fill_cf_slot = aie2_cmdlist_fill_cf,
636 .fill_dpu_slot = aie2_cmdlist_fill_dpu,
637 .fill_preempt_slot = aie2_cmdlist_unsupp,
638 .fill_elf_slot = aie2_cmdlist_unsupp,
639 .get_chain_msg_op = aie2_get_chain_msg_op,
640};
641
642static int
643aie2_cmdlist_fill_npu_cf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size)
644{
645 struct cmd_chain_slot_npu *npu_slot = slot;
646 u32 cmd_len;
647 void *cmd;
648
649 cmd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len);
650 if (*size < sizeof(*npu_slot) + cmd_len)
498 return -EINVAL;651 return -EINVAL;
499 }
500 msg.handle = job;
501 msg.notify_cb = notify_cb;
502 msg.send_data = (u8 *)&req;
503 print_hex_dump_debug("cmd: ", DUMP_PREFIX_OFFSET, 16, 4, &req,
504 0x40, false);
505 652
506 ret = xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);653 npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo);
507 if (ret) {654 if (npu_slot->cu_idx == INVALID_CU_IDX)
508 XDNA_ERR(xdna, "Send message failed");655 return -EINVAL;
509 return ret;
510 }
511 656
657 memset(npu_slot, 0, sizeof(*npu_slot));
658 npu_slot->type = EXEC_NPU_TYPE_NON_ELF;
659 npu_slot->arg_cnt = cmd_len / sizeof(u32);
660 memcpy(npu_slot->args, cmd, cmd_len);
661
662 *size = sizeof(*npu_slot) + cmd_len;
512 return 0;663 return 0;
513}664}
514 665
515static int666static int
516aie2_cmdlist_fill_one_slot_cf(void *cmd_buf, u32 offset,667aie2_cmdlist_fill_npu_dpu(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size)
517 struct amdxdna_gem_obj *abo, u32 *size)
518{668{
519 struct cmd_chain_slot_execbuf_cf *buf = cmd_buf + offset;669 struct cmd_chain_slot_npu *npu_slot = slot;
520 int cu_idx = amdxdna_cmd_get_cu_idx(abo);670 struct amdxdna_cmd_start_npu *sn;
521 u32 payload_len;671 u32 cmd_len;
522 void *payload;672 u32 arg_sz;
523 673
524 if (cu_idx < 0)674 sn = amdxdna_cmd_get_payload(cmd_bo, &cmd_len);
675 arg_sz = cmd_len - sizeof(*sn);
676 if (cmd_len < sizeof(*sn) || arg_sz > MAX_NPU_ARGS_SIZE)
525 return -EINVAL;677 return -EINVAL;
526 678
527 payload = amdxdna_cmd_get_payload(abo, &payload_len);679 if (*size < sizeof(*npu_slot) + arg_sz)
528 if (!payload)
529 return -EINVAL;680 return -EINVAL;
530 681
531 if (!slot_has_space(*buf, offset, payload_len))682 npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo);
532 return -ENOSPC;683 if (npu_slot->cu_idx == INVALID_CU_IDX)
684 return -EINVAL;
685
686 memset(npu_slot, 0, sizeof(*npu_slot));
687 npu_slot->type = EXEC_NPU_TYPE_PARTIAL_ELF;
688 npu_slot->inst_buf_addr = sn->buffer;
689 npu_slot->inst_size = sn->buffer_size;
690 npu_slot->inst_prop_cnt = sn->prop_count;
691 npu_slot->arg_cnt = arg_sz / sizeof(u32);
692 memcpy(npu_slot->args, sn->prop_args, arg_sz);
533 693
534 buf->cu_idx = cu_idx;694 *size = sizeof(*npu_slot) + arg_sz;
535 buf->arg_cnt = payload_len / sizeof(u32);
536 memcpy(buf->args, payload, payload_len);
537 /* Accurate buf size to hint firmware to do necessary copy */
538 *size = sizeof(*buf) + payload_len;
539 return 0;695 return 0;
540}696}
541 697
542static int698static int
543aie2_cmdlist_fill_one_slot_dpu(void *cmd_buf, u32 offset,699aie2_cmdlist_fill_npu_preempt(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size)
544 struct amdxdna_gem_obj *abo, u32 *size)
545{700{
546 struct cmd_chain_slot_dpu *buf = cmd_buf + offset;701 struct cmd_chain_slot_npu *npu_slot = slot;
547 int cu_idx = amdxdna_cmd_get_cu_idx(abo);702 struct amdxdna_cmd_preempt_data *pd;
548 struct amdxdna_cmd_start_npu *sn;703 u32 cmd_len;
549 u32 payload_len;
550 void *payload;
551 u32 arg_sz;704 u32 arg_sz;
552 705
553 if (cu_idx < 0)706 pd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len);
707 arg_sz = cmd_len - sizeof(*pd);
708 if (cmd_len < sizeof(*pd) || arg_sz > MAX_NPU_ARGS_SIZE)
554 return -EINVAL;709 return -EINVAL;
555 710
556 payload = amdxdna_cmd_get_payload(abo, &payload_len);711 if (*size < sizeof(*npu_slot) + arg_sz)
557 if (!payload)
558 return -EINVAL;712 return -EINVAL;
559 sn = payload;713
560 arg_sz = payload_len - sizeof(*sn);714 npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo);
561 if (payload_len < sizeof(*sn) || arg_sz > MAX_DPU_ARGS_SIZE)715 if (npu_slot->cu_idx == INVALID_CU_IDX)
562 return -EINVAL;716 return -EINVAL;
563 717
564 if (!slot_has_space(*buf, offset, arg_sz))718 memset(npu_slot, 0, sizeof(*npu_slot));
565 return -ENOSPC;719 npu_slot->type = EXEC_NPU_TYPE_PREEMPT;
720 npu_slot->inst_buf_addr = pd->inst_buf;
721 npu_slot->save_buf_addr = pd->save_buf;
722 npu_slot->restore_buf_addr = pd->restore_buf;
723 npu_slot->inst_size = pd->inst_size;
724 npu_slot->save_size = pd->save_size;
725 npu_slot->restore_size = pd->restore_size;
726 npu_slot->inst_prop_cnt = pd->inst_prop_cnt;
727 npu_slot->arg_cnt = arg_sz / sizeof(u32);
728 memcpy(npu_slot->args, pd->prop_args, arg_sz);
729
730 *size = sizeof(*npu_slot) + arg_sz;
731 return 0;
732}
733
734static int
735aie2_cmdlist_fill_npu_elf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size)
736{
737 struct cmd_chain_slot_npu *npu_slot = slot;
738 struct amdxdna_cmd_preempt_data *pd;
739 u32 cmd_len;
740 u32 arg_sz;
741
742 pd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len);
743 arg_sz = cmd_len - sizeof(*pd);
744 if (cmd_len < sizeof(*pd) || arg_sz > MAX_NPU_ARGS_SIZE)
745 return -EINVAL;
566 746
567 buf->inst_buf_addr = sn->buffer;747 if (*size < sizeof(*npu_slot) + arg_sz)
568 buf->inst_size = sn->buffer_size;748 return -EINVAL;
569 buf->inst_prop_cnt = sn->prop_count;
570 buf->cu_idx = cu_idx;
571 buf->arg_cnt = arg_sz / sizeof(u32);
572 memcpy(buf->args, sn->prop_args, arg_sz);
573 749
574 /* Accurate buf size to hint firmware to do necessary copy */750 memset(npu_slot, 0, sizeof(*npu_slot));
575 *size = sizeof(*buf) + arg_sz;751 npu_slot->type = EXEC_NPU_TYPE_ELF;
752 npu_slot->inst_buf_addr = pd->inst_buf;
753 npu_slot->save_buf_addr = pd->save_buf;
754 npu_slot->restore_buf_addr = pd->restore_buf;
755 npu_slot->inst_size = pd->inst_size;
756 npu_slot->save_size = pd->save_size;
757 npu_slot->restore_size = pd->restore_size;
758 npu_slot->inst_prop_cnt = pd->inst_prop_cnt;
759 npu_slot->arg_cnt = 1;
760 npu_slot->args[0] = AIE2_EXEC_BUFFER_KERNEL_OP_TXN;
761
762 *size = struct_size(npu_slot, args, npu_slot->arg_cnt);
576 return 0;763 return 0;
577}764}
578 765
579static int766static u32 aie2_get_npu_chain_msg_op(u32 cmd_op)
580aie2_cmdlist_fill_one_slot(u32 op, struct amdxdna_gem_obj *cmdbuf_abo, u32 offset,767{
581 struct amdxdna_gem_obj *abo, u32 *size)768 return MSG_OP_CHAIN_EXEC_NPU;
769}
770
771static struct aie2_exec_msg_ops npu_exec_message_ops = {
772 .init_cu_req = aie2_init_exec_cu_req,
773 .init_dpu_req = aie2_init_exec_dpu_req,
774 .init_chain_req = aie2_init_npu_chain_req,
775 .fill_cf_slot = aie2_cmdlist_fill_npu_cf,
776 .fill_dpu_slot = aie2_cmdlist_fill_npu_dpu,
777 .fill_preempt_slot = aie2_cmdlist_fill_npu_preempt,
778 .fill_elf_slot = aie2_cmdlist_fill_npu_elf,
779 .get_chain_msg_op = aie2_get_npu_chain_msg_op,
780};
781
782static int aie2_init_exec_req(void *req, struct amdxdna_gem_obj *cmd_abo,
783 size_t *size, u32 *msg_op)
582{784{
583 u32 this_op = amdxdna_cmd_get_op(abo);785 struct amdxdna_dev *xdna = cmd_abo->client->xdna;
584 void *cmd_buf = cmdbuf_abo->mem.kva;
585 int ret;786 int ret;
787 u32 op;
586 788
587 if (this_op != op) {
588 ret = -EINVAL;
589 goto done;
590 }
591 789
790 op = amdxdna_cmd_get_op(cmd_abo);
592 switch (op) {791 switch (op) {
593 case ERT_START_CU:792 case ERT_START_CU:
594 ret = aie2_cmdlist_fill_one_slot_cf(cmd_buf, offset, abo, size);793 ret = EXEC_MSG_OPS(xdna)->init_cu_req(cmd_abo, req, size, msg_op);
794 if (ret) {
795 XDNA_DBG(xdna, "Init CU req failed ret %d", ret);
796 return ret;
797 }
595 break;798 break;
596 case ERT_START_NPU:799 case ERT_START_NPU:
597 ret = aie2_cmdlist_fill_one_slot_dpu(cmd_buf, offset, abo, size);800 ret = EXEC_MSG_OPS(xdna)->init_dpu_req(cmd_abo, req, size, msg_op);
801 if (ret) {
802 XDNA_DBG(xdna, "Init DPU req failed ret %d", ret);
803 return ret;
804 }
805
598 break;806 break;
599 default:807 default:
808 XDNA_ERR(xdna, "Unsupported op %d", op);
600 ret = -EOPNOTSUPP;809 ret = -EOPNOTSUPP;
810 break;
601 }811 }
602 812
603done:813 return ret;
604 if (ret) {814}
605 XDNA_ERR(abo->client->xdna, "Can't fill slot for cmd op %d ret %d",815
606 op, ret);816static int
817aie2_cmdlist_fill_slot(void *slot, struct amdxdna_gem_obj *cmd_abo,
818 size_t *size, u32 *cmd_op)
819{
820 struct amdxdna_dev *xdna = cmd_abo->client->xdna;
821 int ret;
822 u32 op;
823
824 op = amdxdna_cmd_get_op(cmd_abo);
825 if (*cmd_op == ERT_INVALID_CMD)
826 *cmd_op = op;
827 else if (op != *cmd_op)
828 return -EINVAL;
829
830 switch (op) {
831 case ERT_START_CU:
832 ret = EXEC_MSG_OPS(xdna)->fill_cf_slot(cmd_abo, slot, size);
833 break;
834 case ERT_START_NPU:
835 ret = EXEC_MSG_OPS(xdna)->fill_dpu_slot(cmd_abo, slot, size);
836 break;
837 case ERT_START_NPU_PREEMPT:
838 if (!AIE2_FEATURE_ON(xdna->dev_handle, AIE2_PREEMPT))
839 return -EOPNOTSUPP;
840 ret = EXEC_MSG_OPS(xdna)->fill_preempt_slot(cmd_abo, slot, size);
841 break;
842 case ERT_START_NPU_PREEMPT_ELF:
843 if (!AIE2_FEATURE_ON(xdna->dev_handle, AIE2_PREEMPT))
844 return -EOPNOTSUPP;
845 ret = EXEC_MSG_OPS(xdna)->fill_elf_slot(cmd_abo, slot, size);
846 break;
847 default:
848 XDNA_INFO(xdna, "Unsupported op %d", op);
849 ret = -EOPNOTSUPP;
850 break;
607 }851 }
852
608 return ret;853 return ret;
609}854}
610 855
856void aie2_msg_init(struct amdxdna_dev_hdl *ndev)
857{
858 if (AIE2_FEATURE_ON(ndev, AIE2_NPU_COMMAND))
859 ndev->exec_msg_ops = &npu_exec_message_ops;
860 else
861 ndev->exec_msg_ops = &legacy_exec_message_ops;
862}
863
611static inline struct amdxdna_gem_obj *864static inline struct amdxdna_gem_obj *
612aie2_cmdlist_get_cmd_buf(struct amdxdna_sched_job *job)865aie2_cmdlist_get_cmd_buf(struct amdxdna_sched_job *job)
613{866{
@@ -616,29 +869,36 @@ aie2_cmdlist_get_cmd_buf(struct amdxdna_sched_job *job) · 2 unmodified lines
616 return job->hwctx->priv->cmd_buf[idx];869 return job->hwctx->priv->cmd_buf[idx];
617}870}
618 871
619static void872int aie2_execbuf(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job,
620aie2_cmdlist_prepare_request(struct cmd_chain_req *req,873 int (*notify_cb)(void *, void __iomem *, size_t))
621 struct amdxdna_gem_obj *cmdbuf_abo, u32 size, u32 cnt)
622{874{
623 req->buf_addr = cmdbuf_abo->mem.dev_addr;875 struct mailbox_channel *chann = hwctx->priv->mbox_chann;
624 req->buf_size = size;876 struct amdxdna_dev *xdna = hwctx->client->xdna;
625 req->count = cnt;877 struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;
626 drm_clflush_virt_range(cmdbuf_abo->mem.kva, size);878 struct xdna_mailbox_msg msg;
627 XDNA_DBG(cmdbuf_abo->client->xdna, "Command buf addr 0x%llx size 0x%x count %d",879 union exec_req req;
628 req->buf_addr, size, cnt);880 int ret;
629}
630 881
631static inline u32882 if (!chann)
632aie2_cmd_op_to_msg_op(u32 op)883 return -ENODEV;
633{884
634 switch (op) {885 ret = aie2_init_exec_req(&req, cmd_abo, &msg.send_size, &msg.opcode);
635 case ERT_START_CU:886 if (ret)
636 return MSG_OP_CHAIN_EXEC_BUFFER_CF;887 return ret;
637 case ERT_START_NPU:888
638 return MSG_OP_CHAIN_EXEC_DPU;889 msg.handle = job;
639 default:890 msg.notify_cb = notify_cb;
640 return MSG_OP_MAX_OPCODE;891 msg.send_data = (u8 *)&req;
892 print_hex_dump_debug("cmd: ", DUMP_PREFIX_OFFSET, 16, 4, &req,
893 0x40, false);
894
895 ret = xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);
896 if (ret) {
897 XDNA_ERR(xdna, "Send message failed");
898 return ret;
641 }899 }
900
901 return 0;
642}902}
643 903
644int aie2_cmdlist_multi_execbuf(struct amdxdna_hwctx *hwctx,904int aie2_cmdlist_multi_execbuf(struct amdxdna_hwctx *hwctx,
@@ -649,12 +909,13 @@ int aie2_cmdlist_multi_execbuf(struct amdxdna_hwctx *hwctx, · 4 unmodified lines
649 struct mailbox_channel *chann = hwctx->priv->mbox_chann;909 struct mailbox_channel *chann = hwctx->priv->mbox_chann;
650 struct amdxdna_client *client = hwctx->client;910 struct amdxdna_client *client = hwctx->client;
651 struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;911 struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;
912 struct amdxdna_dev *xdna = client->xdna;
652 struct amdxdna_cmd_chain *payload;913 struct amdxdna_cmd_chain *payload;
653 struct xdna_mailbox_msg msg;914 struct xdna_mailbox_msg msg;
654 struct cmd_chain_req req;915 union exec_chain_req req;
655 u32 payload_len;916 u32 payload_len;
656 u32 offset = 0;917 u32 offset = 0;
657 u32 size;918 size_t size;
658 int ret;919 int ret;
659 u32 op;920 u32 op;
660 u32 i;921 u32 i;
@@ -665,41 +926,42 @@ int aie2_cmdlist_multi_execbuf(struct amdxdna_hwctx *hwctx, · 4 unmodified lines
665 payload_len < struct_size(payload, data, payload->command_count))926 payload_len < struct_size(payload, data, payload->command_count))
666 return -EINVAL;927 return -EINVAL;
667 928
929 op = ERT_INVALID_CMD;
668 for (i = 0; i < payload->command_count; i++) {930 for (i = 0; i < payload->command_count; i++) {
669 u32 boh = (u32)(payload->data[i]);931 u32 boh = (u32)(payload->data[i]);
670 struct amdxdna_gem_obj *abo;932 struct amdxdna_gem_obj *abo;
671 933
672 abo = amdxdna_gem_get_obj(client, boh, AMDXDNA_BO_CMD);934 abo = amdxdna_gem_get_obj(client, boh, AMDXDNA_BO_CMD);
673 if (!abo) {935 if (!abo) {
674 XDNA_ERR(client->xdna, "Failed to find cmd BO %d", boh);936 XDNA_ERR(xdna, "Failed to find cmd BO %d", boh);
675 return -ENOENT;937 return -ENOENT;
676 }938 }
677 939
678 /* All sub-cmd should have same op, use the first one. */940 size = cmdbuf_abo->mem.size - offset;
679 if (i == 0)941 ret = aie2_cmdlist_fill_slot(cmdbuf_abo->mem.kva + offset,
680 op = amdxdna_cmd_get_op(abo);942 abo, &size, &op);
681
682 ret = aie2_cmdlist_fill_one_slot(op, cmdbuf_abo, offset, abo, &size);
683 amdxdna_gem_put_obj(abo);943 amdxdna_gem_put_obj(abo);
684 if (ret)944 if (ret)
685 return -EINVAL;945 return ret;
686 946
687 offset += size;947 offset += size;
688 }948 }
949 msg.opcode = EXEC_MSG_OPS(xdna)->get_chain_msg_op(op);
950 if (msg.opcode == MSG_OP_MAX_OPCODE)
951 return -EOPNOTSUPP;
689 952
690 /* The offset is the accumulated total size of the cmd buffer */953 /* The offset is the accumulated total size of the cmd buffer */
691 aie2_cmdlist_prepare_request(&req, cmdbuf_abo, offset, payload->command_count);954 EXEC_MSG_OPS(xdna)->init_chain_req(&req, cmdbuf_abo->mem.dev_addr,
955 offset, payload->command_count);
956 drm_clflush_virt_range(cmdbuf_abo->mem.kva, offset);
692 957
693 msg.opcode = aie2_cmd_op_to_msg_op(op);
694 if (msg.opcode == MSG_OP_MAX_OPCODE)
695 return -EOPNOTSUPP;
696 msg.handle = job;958 msg.handle = job;
697 msg.notify_cb = notify_cb;959 msg.notify_cb = notify_cb;
698 msg.send_data = (u8 *)&req;960 msg.send_data = (u8 *)&req;
699 msg.send_size = sizeof(req);961 msg.send_size = sizeof(req);
700 ret = xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);962 ret = xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);
701 if (ret) {963 if (ret) {
702 XDNA_ERR(hwctx->client->xdna, "Send message failed");964 XDNA_ERR(xdna, "Send message failed");
703 return ret;965 return ret;
704 }966 }
705 967
@@ -712,23 +974,27 @@ int aie2_cmdlist_single_execbuf(struct amdxdna_hwctx *hwctx, · 6 unmodified lines
712{974{
713 struct amdxdna_gem_obj *cmdbuf_abo = aie2_cmdlist_get_cmd_buf(job);975 struct amdxdna_gem_obj *cmdbuf_abo = aie2_cmdlist_get_cmd_buf(job);
714 struct mailbox_channel *chann = hwctx->priv->mbox_chann;976 struct mailbox_channel *chann = hwctx->priv->mbox_chann;
977 struct amdxdna_dev *xdna = hwctx->client->xdna;
715 struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;978 struct amdxdna_gem_obj *cmd_abo = job->cmd_bo;
716 struct xdna_mailbox_msg msg;979 struct xdna_mailbox_msg msg;
717 struct cmd_chain_req req;980 union exec_chain_req req;
718 u32 size;981 u32 op = ERT_INVALID_CMD;
982 size_t size;
719 int ret;983 int ret;
720 u32 op;
721 984
722 op = amdxdna_cmd_get_op(cmd_abo);985 size = cmdbuf_abo->mem.size;
723 ret = aie2_cmdlist_fill_one_slot(op, cmdbuf_abo, 0, cmd_abo, &size);986 ret = aie2_cmdlist_fill_slot(cmdbuf_abo->mem.kva, cmd_abo, &size, &op);
724 if (ret)987 if (ret)
725 return ret;988 return ret;
726 989
727 aie2_cmdlist_prepare_request(&req, cmdbuf_abo, size, 1);990 msg.opcode = EXEC_MSG_OPS(xdna)->get_chain_msg_op(op);
728
729 msg.opcode = aie2_cmd_op_to_msg_op(op);
730 if (msg.opcode == MSG_OP_MAX_OPCODE)991 if (msg.opcode == MSG_OP_MAX_OPCODE)
731 return -EOPNOTSUPP;992 return -EOPNOTSUPP;
993
994 EXEC_MSG_OPS(xdna)->init_chain_req(&req, cmdbuf_abo->mem.dev_addr,
995 size, 1);
996 drm_clflush_virt_range(cmdbuf_abo->mem.kva, size);
997
732 msg.handle = job;998 msg.handle = job;
733 msg.notify_cb = notify_cb;999 msg.notify_cb = notify_cb;
734 msg.send_data = (u8 *)&req;1000 msg.send_data = (u8 *)&req;
@@ -753,7 +1019,7 @@ int aie2_sync_bo(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job, · 18 unmodified lines
753 int ret = 0;1019 int ret = 0;
754 1020
755 req.src_addr = 0;1021 req.src_addr = 0;
756 req.dst_addr = abo->mem.dev_addr - hwctx->client->dev_heap->mem.dev_addr;1022 req.dst_addr = amdxdna_dev_bo_offset(abo);
757 req.size = abo->mem.size;1023 req.size = abo->mem.size;
758 1024
759 /* Device to Host */1025 /* Device to Host */
@@ -777,3 +1043,32 @@ int aie2_sync_bo(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job, · 17 unmodified lines
777 1043
778 return 0;1044 return 0;
779}1045}
1046
1047int aie2_config_debug_bo(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job,
1048 int (*notify_cb)(void *, void __iomem *, size_t))
1049{
1050 struct mailbox_channel *chann = hwctx->priv->mbox_chann;
1051 struct amdxdna_gem_obj *abo = to_xdna_obj(job->bos[0]);
1052 struct amdxdna_dev *xdna = hwctx->client->xdna;
1053 struct config_debug_bo_req req;
1054 struct xdna_mailbox_msg msg;
1055
1056 if (job->drv_cmd->opcode == ATTACH_DEBUG_BO)
1057 req.config = DEBUG_BO_REGISTER;
1058 else
1059 req.config = DEBUG_BO_UNREGISTER;
1060
1061 req.offset = amdxdna_dev_bo_offset(abo);
1062 req.size = abo->mem.size;
1063
1064 XDNA_DBG(xdna, "offset 0x%llx size 0x%llx config %d",
1065 req.offset, req.size, req.config);
1066
1067 msg.handle = job;
1068 msg.notify_cb = notify_cb;
1069 msg.send_data = (u8 *)&req;
1070 msg.send_size = sizeof(req);
1071 msg.opcode = MSG_OP_CONFIG_DEBUG_BO;
1072
1073 return xdna_mailbox_send_msg(chann, &msg, TX_TIMEOUT);
1074}
drivers/accel/amdxdna/aie2_msg_priv.h+84−4
@@ -9,7 +9,8 @@ · 8 unmodified lines
9enum aie2_msg_opcode {9enum aie2_msg_opcode {
10 MSG_OP_CREATE_CONTEXT = 0x2,10 MSG_OP_CREATE_CONTEXT = 0x2,
11 MSG_OP_DESTROY_CONTEXT = 0x3,11 MSG_OP_DESTROY_CONTEXT = 0x3,
12 MSG_OP_SYNC_BO = 0x7,12 MSG_OP_GET_TELEMETRY = 0x4,
13 MSG_OP_SYNC_BO = 0x7,
13 MSG_OP_EXECUTE_BUFFER_CF = 0xC,14 MSG_OP_EXECUTE_BUFFER_CF = 0xC,
14 MSG_OP_QUERY_COL_STATUS = 0xD,15 MSG_OP_QUERY_COL_STATUS = 0xD,
15 MSG_OP_QUERY_AIE_TILE_INFO = 0xE,16 MSG_OP_QUERY_AIE_TILE_INFO = 0xE,
@@ -18,6 +19,8 @@ enum aie2_msg_opcode { · 2 unmodified lines
18 MSG_OP_CONFIG_CU = 0x11,19 MSG_OP_CONFIG_CU = 0x11,
19 MSG_OP_CHAIN_EXEC_BUFFER_CF = 0x12,20 MSG_OP_CHAIN_EXEC_BUFFER_CF = 0x12,
20 MSG_OP_CHAIN_EXEC_DPU = 0x13,21 MSG_OP_CHAIN_EXEC_DPU = 0x13,
22 MSG_OP_CONFIG_DEBUG_BO = 0x14,
23 MSG_OP_CHAIN_EXEC_NPU = 0x18,
21 MSG_OP_MAX_XRT_OPCODE,24 MSG_OP_MAX_XRT_OPCODE,
22 MSG_OP_SUSPEND = 0x101,25 MSG_OP_SUSPEND = 0x101,
23 MSG_OP_RESUME = 0x102,26 MSG_OP_RESUME = 0x102,
@@ -135,6 +138,28 @@ struct destroy_ctx_resp { · 111 unmodified lines
135 enum aie2_msg_status status;138 enum aie2_msg_status status;
136} __packed;139} __packed;
137 140
141enum telemetry_type {
142 TELEMETRY_TYPE_DISABLED,
143 TELEMETRY_TYPE_HEALTH,
144 TELEMETRY_TYPE_ERROR_INFO,
145 TELEMETRY_TYPE_PROFILING,
146 TELEMETRY_TYPE_DEBUG,
147 MAX_TELEMETRY_TYPE
148};
149
150struct get_telemetry_req {
151 enum telemetry_type type;
152 __u64 buf_addr;
153 __u32 buf_size;
154} __packed;
155
156struct get_telemetry_resp {
157 __u32 major;
158 __u32 minor;
159 __u32 size;
160 enum aie2_msg_status status;
161} __packed;
162
138struct execute_buffer_req {163struct execute_buffer_req {
139 __u32 cu_idx;164 __u32 cu_idx;
140 __u32 payload[19];165 __u32 payload[19];
@@ -148,6 +173,18 @@ struct exec_dpu_req { · 7 unmodified lines
148 __u32 payload[35];173 __u32 payload[35];
149} __packed;174} __packed;
150 175
176enum exec_npu_type {
177 EXEC_NPU_TYPE_NON_ELF = 0x1,
178 EXEC_NPU_TYPE_PARTIAL_ELF = 0x2,
179 EXEC_NPU_TYPE_PREEMPT = 0x3,
180 EXEC_NPU_TYPE_ELF = 0x4,
181};
182
183union exec_req {
184 struct execute_buffer_req ebuf;
185 struct exec_dpu_req dpu_req;
186};
187
151struct execute_buffer_resp {188struct execute_buffer_resp {
152 enum aie2_msg_status status;189 enum aie2_msg_status status;
153} __packed;190} __packed;
@@ -319,9 +356,6 @@ struct async_event_msg_resp { · 165 unmodified lines
319} __packed;356} __packed;
320 357
321#define MAX_CHAIN_CMDBUF_SIZE SZ_4K358#define MAX_CHAIN_CMDBUF_SIZE SZ_4K
322#define slot_has_space(slot, offset, payload_size) \
323 (MAX_CHAIN_CMDBUF_SIZE >= (offset) + (payload_size) + \
324 sizeof(typeof(slot)))
325 359
326struct cmd_chain_slot_execbuf_cf {360struct cmd_chain_slot_execbuf_cf {
327 __u32 cu_idx;361 __u32 cu_idx;
@@ -339,12 +373,41 @@ struct cmd_chain_slot_dpu { · 11 unmodified lines
339 __u32 args[] __counted_by(arg_cnt);373 __u32 args[] __counted_by(arg_cnt);
340};374};
341 375
376#define MAX_NPU_ARGS_SIZE (26 * sizeof(__u32))
377#define AIE2_EXEC_BUFFER_KERNEL_OP_TXN 3
378struct cmd_chain_slot_npu {
379 enum exec_npu_type type;
380 u64 inst_buf_addr;
381 u64 save_buf_addr;
382 u64 restore_buf_addr;
383 u32 inst_size;
384 u32 save_size;
385 u32 restore_size;
386 u32 inst_prop_cnt;
387 u32 cu_idx;
388 u32 arg_cnt;
389 u32 args[] __counted_by(arg_cnt);
390} __packed;
391
342struct cmd_chain_req {392struct cmd_chain_req {
343 __u64 buf_addr;393 __u64 buf_addr;
344 __u32 buf_size;394 __u32 buf_size;
345 __u32 count;395 __u32 count;
346} __packed;396} __packed;
347 397
398struct cmd_chain_npu_req {
399 u32 flags;
400 u32 reserved;
401 u64 buf_addr;
402 u32 buf_size;
403 u32 count;
404} __packed;
405
406union exec_chain_req {
407 struct cmd_chain_npu_req npu_req;
408 struct cmd_chain_req req;
409};
410
348struct cmd_chain_resp {411struct cmd_chain_resp {
349 enum aie2_msg_status status;412 enum aie2_msg_status status;
350 __u32 fail_cmd_idx;413 __u32 fail_cmd_idx;
@@ -365,4 +428,21 @@ struct sync_bo_req { · 14 unmodified lines
365struct sync_bo_resp {428struct sync_bo_resp {
366 enum aie2_msg_status status;429 enum aie2_msg_status status;
367} __packed;430} __packed;
431
432#define DEBUG_BO_UNREGISTER 0
433#define DEBUG_BO_REGISTER 1
434struct config_debug_bo_req {
435 __u64 offset;
436 __u64 size;
437 /*
438 * config operations.
439 * DEBUG_BO_REGISTER: Register debug buffer
440 * DEBUG_BO_UNREGISTER: Unregister debug buffer
441 */
442 __u32 config;
443} __packed;
444
445struct config_debug_bo_resp {
446 enum aie2_msg_status status;
447} __packed;
368#endif /* _AIE2_MSG_PRIV_H_ */448#endif /* _AIE2_MSG_PRIV_H_ */
drivers/accel/amdxdna/aie2_pci.c+226−43
@@ -25,6 +25,7 @@ · 24 unmodified lines
25#include "amdxdna_gem.h"25#include "amdxdna_gem.h"
26#include "amdxdna_mailbox.h"26#include "amdxdna_mailbox.h"
27#include "amdxdna_pci_drv.h"27#include "amdxdna_pci_drv.h"
28#include "amdxdna_pm.h"
28 29
29static int aie2_max_col = XRS_MAX_COL;30static int aie2_max_col = XRS_MAX_COL;
30module_param(aie2_max_col, uint, 0600);31module_param(aie2_max_col, uint, 0600);
@@ -54,6 +55,7 @@ struct mgmt_mbox_chann_info { · 23 unmodified lines
54 55
55static int aie2_check_protocol(struct amdxdna_dev_hdl *ndev, u32 fw_major, u32 fw_minor)56static int aie2_check_protocol(struct amdxdna_dev_hdl *ndev, u32 fw_major, u32 fw_minor)
56{57{
58 const struct aie2_fw_feature_tbl *feature;
57 struct amdxdna_dev *xdna = ndev->xdna;59 struct amdxdna_dev *xdna = ndev->xdna;
58 60
59 /*61 /*
@@ -77,6 +79,17 @@ static int aie2_check_protocol(struct amdxdna_dev_hdl *ndev, u32 fw_major, u32 f · 17 unmodified lines
77 XDNA_ERR(xdna, "Firmware minor version smaller than supported");79 XDNA_ERR(xdna, "Firmware minor version smaller than supported");
78 return -EINVAL;80 return -EINVAL;
79 }81 }
82
83 for (feature = ndev->priv->fw_feature_tbl; feature && feature->min_minor;
84 feature++) {
85 if (fw_minor < feature->min_minor)
86 continue;
87 if (feature->max_minor > 0 && fw_minor > feature->max_minor)
88 continue;
89
90 set_bit(feature->feature, &ndev->feature_mask);
91 }
92
80 return 0;93 return 0;
81}94}
82 95
@@ -170,6 +183,10 @@ int aie2_runtime_cfg(struct amdxdna_dev_hdl *ndev, · 87 unmodified lines
170 if (cfg->category != category)183 if (cfg->category != category)
171 continue;184 continue;
172 185
186 if (cfg->feature_mask &&
187 bitmap_subset(&cfg->feature_mask, &ndev->feature_mask, AIE2_FEATURE_MAX))
188 continue;
189
173 value = val ? *val : cfg->value;190 value = val ? *val : cfg->value;
174 ret = aie2_set_runtime_cfg(ndev, cfg->type, value);191 ret = aie2_set_runtime_cfg(ndev, cfg->type, value);
175 if (ret) {192 if (ret) {
@@ -223,15 +240,6 @@ static int aie2_mgmt_fw_init(struct amdxdna_dev_hdl *ndev) · 47 unmodified lines
223 return ret;240 return ret;
224 }241 }
225 242
226 if (!ndev->async_events)
227 return 0;
228
229 ret = aie2_error_async_events_send(ndev);
230 if (ret) {
231 XDNA_ERR(ndev->xdna, "Send async events failed");
232 return ret;
233 }
234
235 return 0;243 return 0;
236}244}
237 245
@@ -257,6 +265,8 @@ static int aie2_mgmt_fw_query(struct amdxdna_dev_hdl *ndev) · 19 unmodified lines
257 return ret;265 return ret;
258 }266 }
259 267
268 ndev->total_col = min(aie2_max_col, ndev->metadata.cols);
269
260 return 0;270 return 0;
261}271}
262 272
@@ -338,6 +348,7 @@ static void aie2_hw_stop(struct amdxdna_dev *xdna) · 75 unmodified lines
338 ndev->mbox = NULL;348 ndev->mbox = NULL;
339 aie2_psp_stop(ndev->psp_hdl);349 aie2_psp_stop(ndev->psp_hdl);
340 aie2_smu_fini(ndev);350 aie2_smu_fini(ndev);
351 aie2_error_async_events_free(ndev);
341 pci_disable_device(pdev);352 pci_disable_device(pdev);
342 353
343 ndev->dev_status = AIE2_DEV_INIT;354 ndev->dev_status = AIE2_DEV_INIT;
@@ -424,6 +435,18 @@ static int aie2_hw_start(struct amdxdna_dev *xdna) · 80 unmodified lines
424 goto destroy_mgmt_chann;435 goto destroy_mgmt_chann;
425 }436 }
426 437
438 ret = aie2_mgmt_fw_query(ndev);
439 if (ret) {
440 XDNA_ERR(xdna, "failed to query fw, ret %d", ret);
441 goto destroy_mgmt_chann;
442 }
443
444 ret = aie2_error_async_events_alloc(ndev);
445 if (ret) {
446 XDNA_ERR(xdna, "Allocate async events failed, ret %d", ret);
447 goto destroy_mgmt_chann;
448 }
449
427 ndev->dev_status = AIE2_DEV_START;450 ndev->dev_status = AIE2_DEV_START;
428 451
429 return 0;452 return 0;
@@ -459,7 +482,6 @@ static int aie2_hw_resume(struct amdxdna_dev *xdna) · 29 unmodified lines
459 struct amdxdna_client *client;482 struct amdxdna_client *client;
460 int ret;483 int ret;
461 484
462 guard(mutex)(&xdna->dev_lock);
463 ret = aie2_hw_start(xdna);485 ret = aie2_hw_start(xdna);
464 if (ret) {486 if (ret) {
465 XDNA_ERR(xdna, "Start hardware failed, %d", ret);487 XDNA_ERR(xdna, "Start hardware failed, %d", ret);
@@ -565,13 +587,6 @@ static int aie2_init(struct amdxdna_dev *xdna) · 99 unmodified lines
565 goto release_fw;587 goto release_fw;
566 }588 }
567 589
568 ret = aie2_mgmt_fw_query(ndev);
569 if (ret) {
570 XDNA_ERR(xdna, "Query firmware failed, ret %d", ret);
571 goto stop_hw;
572 }
573 ndev->total_col = min(aie2_max_col, ndev->metadata.cols);
574
575 xrs_cfg.clk_list.num_levels = ndev->max_dpm_level + 1;590 xrs_cfg.clk_list.num_levels = ndev->max_dpm_level + 1;
576 for (i = 0; i < xrs_cfg.clk_list.num_levels; i++)591 for (i = 0; i < xrs_cfg.clk_list.num_levels; i++)
577 xrs_cfg.clk_list.cu_clk_list[i] = ndev->priv->dpm_clk_tbl[i].hclk;592 xrs_cfg.clk_list.cu_clk_list[i] = ndev->priv->dpm_clk_tbl[i].hclk;
@@ -587,30 +602,11 @@ static int aie2_init(struct amdxdna_dev *xdna) · 9 unmodified lines
587 goto stop_hw;602 goto stop_hw;
588 }603 }
589 604
590 ret = aie2_error_async_events_alloc(ndev);
591 if (ret) {
592 XDNA_ERR(xdna, "Allocate async events failed, ret %d", ret);
593 goto stop_hw;
594 }
595
596 ret = aie2_error_async_events_send(ndev);
597 if (ret) {
598 XDNA_ERR(xdna, "Send async events failed, ret %d", ret);
599 goto async_event_free;
600 }
601
602 /* Issue a command to make sure firmware handled async events */
603 ret = aie2_query_firmware_version(ndev, &ndev->xdna->fw_ver);
604 if (ret) {
605 XDNA_ERR(xdna, "Re-query firmware version failed");
606 goto async_event_free;
607 }
608
609 release_firmware(fw);605 release_firmware(fw);
606 aie2_msg_init(ndev);
607 amdxdna_pm_init(xdna);
610 return 0;608 return 0;
611 609
612async_event_free:
613 aie2_error_async_events_free(ndev);
614stop_hw:610stop_hw:
615 aie2_hw_stop(xdna);611 aie2_hw_stop(xdna);
616release_fw:612release_fw:
@@ -621,10 +617,8 @@ release_fw: · 4 unmodified lines
621 617
622static void aie2_fini(struct amdxdna_dev *xdna)618static void aie2_fini(struct amdxdna_dev *xdna)
623{619{
624 struct amdxdna_dev_hdl *ndev = xdna->dev_handle;620 amdxdna_pm_fini(xdna);
625
626 aie2_hw_stop(xdna);621 aie2_hw_stop(xdna);
627 aie2_error_async_events_free(ndev);
628}622}
629 623
630static int aie2_get_aie_status(struct amdxdna_client *client,624static int aie2_get_aie_status(struct amdxdna_client *client,
@@ -845,7 +839,120 @@ static int aie2_get_hwctx_status(struct amdxdna_client *client, · 214 unmodified lines
845 }839 }
846 840
847 args->buffer_size -= (u32)(array_args.buffer - args->buffer);841 args->buffer_size -= (u32)(array_args.buffer - args->buffer);
848 return ret;842 return 0;
843}
844
845static int aie2_query_resource_info(struct amdxdna_client *client,
846 struct amdxdna_drm_get_info *args)
847{
848 struct amdxdna_drm_get_resource_info res_info;
849 const struct amdxdna_dev_priv *priv;
850 struct amdxdna_dev_hdl *ndev;
851 struct amdxdna_dev *xdna;
852
853 xdna = client->xdna;
854 ndev = xdna->dev_handle;
855 priv = ndev->priv;
856
857 res_info.npu_clk_max = priv->dpm_clk_tbl[ndev->max_dpm_level].hclk;
858 res_info.npu_tops_max = ndev->max_tops;
859 res_info.npu_task_max = priv->hwctx_limit;
860 res_info.npu_tops_curr = ndev->curr_tops;
861 res_info.npu_task_curr = ndev->hwctx_num;
862
863 if (copy_to_user(u64_to_user_ptr(args->buffer), &res_info, sizeof(res_info)))
864 return -EFAULT;
865
866 return 0;
867}
868
869static int aie2_fill_hwctx_map(struct amdxdna_hwctx *hwctx, void *arg)
870{
871 struct amdxdna_dev *xdna = hwctx->client->xdna;
872 u32 *map = arg;
873
874 if (hwctx->fw_ctx_id >= xdna->dev_handle->priv->hwctx_limit) {
875 XDNA_ERR(xdna, "Invalid fw ctx id %d/%d ", hwctx->fw_ctx_id,
876 xdna->dev_handle->priv->hwctx_limit);
877 return -EINVAL;
878 }
879
880 map[hwctx->fw_ctx_id] = hwctx->id;
881 return 0;
882}
883
884static int aie2_get_telemetry(struct amdxdna_client *client,
885 struct amdxdna_drm_get_info *args)
886{
887 struct amdxdna_drm_query_telemetry_header *header __free(kfree) = NULL;
888 u32 telemetry_data_sz, header_sz, elem_num;
889 struct amdxdna_dev *xdna = client->xdna;
890 struct amdxdna_client *tmp_client;
891 int ret;
892
893 elem_num = xdna->dev_handle->priv->hwctx_limit;
894 header_sz = struct_size(header, map, elem_num);
895 if (args->buffer_size <= header_sz) {
896 XDNA_ERR(xdna, "Invalid buffer size");
897 return -EINVAL;
898 }
899
900 telemetry_data_sz = args->buffer_size - header_sz;
901 if (telemetry_data_sz > SZ_4M) {
902 XDNA_ERR(xdna, "Buffer size is too big, %d", telemetry_data_sz);
903 return -EINVAL;
904 }
905
906 header = kzalloc(header_sz, GFP_KERNEL);
907 if (!header)
908 return -ENOMEM;
909
910 if (copy_from_user(header, u64_to_user_ptr(args->buffer), sizeof(*header))) {
911 XDNA_ERR(xdna, "Failed to copy telemetry header from user");
912 return -EFAULT;
913 }
914
915 header->map_num_elements = elem_num;
916 list_for_each_entry(tmp_client, &xdna->client_list, node) {
917 ret = amdxdna_hwctx_walk(tmp_client, &header->map,
918 aie2_fill_hwctx_map);
919 if (ret)
920 return ret;
921 }
922
923 ret = aie2_query_telemetry(xdna->dev_handle,
924 u64_to_user_ptr(args->buffer + header_sz),
925 telemetry_data_sz, header);
926 if (ret) {
927 XDNA_ERR(xdna, "Query telemetry failed ret %d", ret);
928 return ret;
929 }
930
931 if (copy_to_user(u64_to_user_ptr(args->buffer), header, header_sz)) {
932 XDNA_ERR(xdna, "Copy header failed");
933 return -EFAULT;
934 }
935
936 return 0;
937}
938
939static int aie2_get_preempt_state(struct amdxdna_client *client,
940 struct amdxdna_drm_get_info *args)
941{
942 struct amdxdna_drm_attribute_state state = {};
943 struct amdxdna_dev *xdna = client->xdna;
944 struct amdxdna_dev_hdl *ndev;
945
946 ndev = xdna->dev_handle;
947 if (args->param == DRM_AMDXDNA_GET_FORCE_PREEMPT_STATE)
948 state.state = ndev->force_preempt_enabled;
949 else if (args->param == DRM_AMDXDNA_GET_FRAME_BOUNDARY_PREEMPT_STATE)
950 state.state = ndev->frame_boundary_preempt;
951
952 if (copy_to_user(u64_to_user_ptr(args->buffer), &state, sizeof(state)))
953 return -EFAULT;
954
955 return 0;
849}956}
850 957
851static int aie2_get_info(struct amdxdna_client *client, struct amdxdna_drm_get_info *args)958static int aie2_get_info(struct amdxdna_client *client, struct amdxdna_drm_get_info *args)
@@ -856,6 +963,10 @@ static int aie2_get_info(struct amdxdna_client *client, struct amdxdna_drm_get_i · 4 unmodified lines
856 if (!drm_dev_enter(&xdna->ddev, &idx))963 if (!drm_dev_enter(&xdna->ddev, &idx))
857 return -ENODEV;964 return -ENODEV;
858 965
966 ret = amdxdna_pm_resume_get(xdna);
967 if (ret)
968 goto dev_exit;
969
859 switch (args->param) {970 switch (args->param) {
860 case DRM_AMDXDNA_QUERY_AIE_STATUS:971 case DRM_AMDXDNA_QUERY_AIE_STATUS:
861 ret = aie2_get_aie_status(client, args);972 ret = aie2_get_aie_status(client, args);
@@ -878,12 +989,25 @@ static int aie2_get_info(struct amdxdna_client *client, struct amdxdna_drm_get_i · 16 unmodified lines
878 case DRM_AMDXDNA_GET_POWER_MODE:989 case DRM_AMDXDNA_GET_POWER_MODE:
879 ret = aie2_get_power_mode(client, args);990 ret = aie2_get_power_mode(client, args);
880 break;991 break;
992 case DRM_AMDXDNA_QUERY_TELEMETRY:
993 ret = aie2_get_telemetry(client, args);
994 break;
995 case DRM_AMDXDNA_QUERY_RESOURCE_INFO:
996 ret = aie2_query_resource_info(client, args);
997 break;
998 case DRM_AMDXDNA_GET_FORCE_PREEMPT_STATE:
999 case DRM_AMDXDNA_GET_FRAME_BOUNDARY_PREEMPT_STATE:
1000 ret = aie2_get_preempt_state(client, args);
1001 break;
881 default:1002 default:
882 XDNA_ERR(xdna, "Not supported request parameter %u", args->param);1003 XDNA_ERR(xdna, "Not supported request parameter %u", args->param);
883 ret = -EOPNOTSUPP;1004 ret = -EOPNOTSUPP;
884 }1005 }
1006
1007 amdxdna_pm_suspend_put(xdna);
885 XDNA_DBG(xdna, "Got param %d", args->param);1008 XDNA_DBG(xdna, "Got param %d", args->param);
886 1009
1010dev_exit:
887 drm_dev_exit(idx);1011 drm_dev_exit(idx);
888 return ret;1012 return ret;
889}1013}
@@ -898,6 +1022,12 @@ static int aie2_query_ctx_status_array(struct amdxdna_client *client, · 8 unmodified lines
898 1022
899 drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));1023 drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock));
900 1024
1025 if (args->element_size > SZ_4K || args->num_element > SZ_1K) {
1026 XDNA_DBG(xdna, "Invalid element size %d or number of element %d",
1027 args->element_size, args->num_element);
1028 return -EINVAL;
1029 }
1030
901 array_args.element_size = min(args->element_size,1031 array_args.element_size = min(args->element_size,
902 sizeof(struct amdxdna_drm_hwctx_entry));1032 sizeof(struct amdxdna_drm_hwctx_entry));
903 array_args.buffer = args->buffer;1033 array_args.buffer = args->buffer;
@@ -914,7 +1044,7 @@ static int aie2_query_ctx_status_array(struct amdxdna_client *client, · 10 unmodified lines
914 args->num_element = (u32)((array_args.buffer - args->buffer) /1044 args->num_element = (u32)((array_args.buffer - args->buffer) /
915 args->element_size);1045 args->element_size);
916 1046
917 return ret;1047 return 0;
918}1048}
919 1049
920static int aie2_get_array(struct amdxdna_client *client,1050static int aie2_get_array(struct amdxdna_client *client,
@@ -926,16 +1056,26 @@ static int aie2_get_array(struct amdxdna_client *client, · 5 unmodified lines
926 if (!drm_dev_enter(&xdna->ddev, &idx))1056 if (!drm_dev_enter(&xdna->ddev, &idx))
927 return -ENODEV;1057 return -ENODEV;
928 1058
1059 ret = amdxdna_pm_resume_get(xdna);
1060 if (ret)
1061 goto dev_exit;
1062
929 switch (args->param) {1063 switch (args->param) {
930 case DRM_AMDXDNA_HW_CONTEXT_ALL:1064 case DRM_AMDXDNA_HW_CONTEXT_ALL:
931 ret = aie2_query_ctx_status_array(client, args);1065 ret = aie2_query_ctx_status_array(client, args);
932 break;1066 break;
1067 case DRM_AMDXDNA_HW_LAST_ASYNC_ERR:
1068 ret = aie2_get_array_async_error(xdna->dev_handle, args);
1069 break;
933 default:1070 default:
934 XDNA_ERR(xdna, "Not supported request parameter %u", args->param);1071 XDNA_ERR(xdna, "Not supported request parameter %u", args->param);
935 ret = -EOPNOTSUPP;1072 ret = -EOPNOTSUPP;
936 }1073 }
1074
1075 amdxdna_pm_suspend_put(xdna);
937 XDNA_DBG(xdna, "Got param %d", args->param);1076 XDNA_DBG(xdna, "Got param %d", args->param);
938 1077
1078dev_exit:
939 drm_dev_exit(idx);1079 drm_dev_exit(idx);
940 return ret;1080 return ret;
941}1081}
@@ -965,6 +1105,38 @@ static int aie2_set_power_mode(struct amdxdna_client *client, · 23 unmodified lines
965 return aie2_pm_set_mode(xdna->dev_handle, power_mode);1105 return aie2_pm_set_mode(xdna->dev_handle, power_mode);
966}1106}
967 1107
1108static int aie2_set_preempt_state(struct amdxdna_client *client,
1109 struct amdxdna_drm_set_state *args)
1110{
1111 struct amdxdna_dev_hdl *ndev = client->xdna->dev_handle;
1112 struct amdxdna_drm_attribute_state state;
1113 u32 val;
1114 int ret;
1115
1116 if (copy_from_user(&state, u64_to_user_ptr(args->buffer), sizeof(state)))
1117 return -EFAULT;
1118
1119 if (state.state > 1)
1120 return -EINVAL;
1121
1122 if (XDNA_MBZ_DBG(client->xdna, state.pad, sizeof(state.pad)))
1123 return -EINVAL;
1124
1125 if (args->param == DRM_AMDXDNA_SET_FORCE_PREEMPT) {
1126 ndev->force_preempt_enabled = state.state;
1127 } else if (args->param == DRM_AMDXDNA_SET_FRAME_BOUNDARY_PREEMPT) {
1128 val = state.state;
1129 ret = aie2_runtime_cfg(ndev, AIE2_RT_CFG_FRAME_BOUNDARY_PREEMPT,
1130 &val);
1131 if (ret)
1132 return ret;
1133
1134 ndev->frame_boundary_preempt = state.state;
1135 }
1136
1137 return 0;
1138}
1139
968static int aie2_set_state(struct amdxdna_client *client,1140static int aie2_set_state(struct amdxdna_client *client,
969 struct amdxdna_drm_set_state *args)1141 struct amdxdna_drm_set_state *args)
970{1142{
@@ -974,16 +1146,26 @@ static int aie2_set_state(struct amdxdna_client *client, · 3 unmodified lines
974 if (!drm_dev_enter(&xdna->ddev, &idx))1146 if (!drm_dev_enter(&xdna->ddev, &idx))
975 return -ENODEV;1147 return -ENODEV;
976 1148
1149 ret = amdxdna_pm_resume_get(xdna);
1150 if (ret)
1151 goto dev_exit;
1152
977 switch (args->param) {1153 switch (args->param) {
978 case DRM_AMDXDNA_SET_POWER_MODE:1154 case DRM_AMDXDNA_SET_POWER_MODE:
979 ret = aie2_set_power_mode(client, args);1155 ret = aie2_set_power_mode(client, args);
980 break;1156 break;
1157 case DRM_AMDXDNA_SET_FORCE_PREEMPT:
1158 case DRM_AMDXDNA_SET_FRAME_BOUNDARY_PREEMPT:
1159 ret = aie2_set_preempt_state(client, args);
1160 break;
981 default:1161 default:
982 XDNA_ERR(xdna, "Not supported request parameter %u", args->param);1162 XDNA_ERR(xdna, "Not supported request parameter %u", args->param);
983 ret = -EOPNOTSUPP;1163 ret = -EOPNOTSUPP;
984 break;1164 break;
985 }1165 }
986 1166
1167 amdxdna_pm_suspend_put(xdna);
1168dev_exit:
987 drm_dev_exit(idx);1169 drm_dev_exit(idx);
988 return ret;1170 return ret;
989}1171}
@@ -998,6 +1180,7 @@ const struct amdxdna_dev_ops aie2_ops = { · 8 unmodified lines
998 .hwctx_init = aie2_hwctx_init,1180 .hwctx_init = aie2_hwctx_init,
999 .hwctx_fini = aie2_hwctx_fini,1181 .hwctx_fini = aie2_hwctx_fini,
1000 .hwctx_config = aie2_hwctx_config,1182 .hwctx_config = aie2_hwctx_config,
1183 .hwctx_sync_debug_bo = aie2_hwctx_sync_debug_bo,
1001 .cmd_submit = aie2_cmd_submit,1184 .cmd_submit = aie2_cmd_submit,
1002 .hmm_invalidate = aie2_hmm_invalidate,1185 .hmm_invalidate = aie2_hmm_invalidate,
1003 .get_array = aie2_get_array,1186 .get_array = aie2_get_array,
drivers/accel/amdxdna/aie2_pci.h+52−2
@@ -110,12 +110,15 @@ struct aie_metadata { · 109 unmodified lines
110enum rt_config_category {110enum rt_config_category {
111 AIE2_RT_CFG_INIT,111 AIE2_RT_CFG_INIT,
112 AIE2_RT_CFG_CLK_GATING,112 AIE2_RT_CFG_CLK_GATING,
113 AIE2_RT_CFG_FORCE_PREEMPT,
114 AIE2_RT_CFG_FRAME_BOUNDARY_PREEMPT,
113};115};
114 116
115struct rt_config {117struct rt_config {
116 u32 type;118 u32 type;
117 u32 value;119 u32 value;
118 u32 category;120 u32 category;
121 unsigned long feature_mask;
119};122};
120 123
121struct dpm_clk_freq {124struct dpm_clk_freq {
@@ -156,6 +159,19 @@ enum aie2_dev_status { · 34 unmodified lines
156 AIE2_DEV_START,159 AIE2_DEV_START,
157};160};
158 161
162struct aie2_exec_msg_ops {
163 int (*init_cu_req)(struct amdxdna_gem_obj *cmd_bo, void *req,
164 size_t *size, u32 *msg_op);
165 int (*init_dpu_req)(struct amdxdna_gem_obj *cmd_bo, void *req,
166 size_t *size, u32 *msg_op);
167 void (*init_chain_req)(void *req, u64 slot_addr, size_t size, u32 cmd_cnt);
168 int (*fill_cf_slot)(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size);
169 int (*fill_dpu_slot)(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size);
170 int (*fill_preempt_slot)(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size);
171 int (*fill_elf_slot)(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *size);
172 u32 (*get_chain_msg_op)(u32 cmd_op);
173};
174
159struct amdxdna_dev_hdl {175struct amdxdna_dev_hdl {
160 struct amdxdna_dev *xdna;176 struct amdxdna_dev *xdna;
161 const struct amdxdna_dev_priv *priv;177 const struct amdxdna_dev_priv *priv;
@@ -173,6 +189,8 @@ struct amdxdna_dev_hdl { · 11 unmodified lines
173 u32 total_col;189 u32 total_col;
174 struct aie_version version;190 struct aie_version version;
175 struct aie_metadata metadata;191 struct aie_metadata metadata;
192 unsigned long feature_mask;
193 struct aie2_exec_msg_ops *exec_msg_ops;
176 194
177 /* power management and clock*/195 /* power management and clock*/
178 enum amdxdna_power_mode_type pw_mode;196 enum amdxdna_power_mode_type pw_mode;
@@ -182,6 +200,10 @@ struct amdxdna_dev_hdl { · 3 unmodified lines
182 u32 clk_gating;200 u32 clk_gating;
183 u32 npuclk_freq;201 u32 npuclk_freq;
184 u32 hclk_freq;202 u32 hclk_freq;
203 u32 max_tops;
204 u32 curr_tops;
205 u32 force_preempt_enabled;
206 u32 frame_boundary_preempt;
185 207
186 /* Mailbox and the management channel */208 /* Mailbox and the management channel */
187 struct mailbox *mbox;209 struct mailbox *mbox;
@@ -190,6 +212,8 @@ struct amdxdna_dev_hdl { · 2 unmodified lines
190 212
191 enum aie2_dev_status dev_status;213 enum aie2_dev_status dev_status;
192 u32 hwctx_num;214 u32 hwctx_num;
215
216 struct amdxdna_async_error last_async_err;
193};217};
194 218
195#define DEFINE_BAR_OFFSET(reg_name, bar, reg_addr) \219#define DEFINE_BAR_OFFSET(reg_name, bar, reg_addr) \
@@ -204,12 +228,27 @@ struct aie2_hw_ops { · 8 unmodified lines
204 int (*set_dpm)(struct amdxdna_dev_hdl *ndev, u32 dpm_level);228 int (*set_dpm)(struct amdxdna_dev_hdl *ndev, u32 dpm_level);
205};229};
206 230
231enum aie2_fw_feature {
232 AIE2_NPU_COMMAND,
233 AIE2_PREEMPT,
234 AIE2_FEATURE_MAX
235};
236
237struct aie2_fw_feature_tbl {
238 enum aie2_fw_feature feature;
239 u32 max_minor;
240 u32 min_minor;
241};
242
243#define AIE2_FEATURE_ON(ndev, feature) test_bit(feature, &(ndev)->feature_mask)
244
207struct amdxdna_dev_priv {245struct amdxdna_dev_priv {
208 const char *fw_path;246 const char *fw_path;
209 u64 protocol_major;247 u64 protocol_major;
210 u64 protocol_minor;248 u64 protocol_minor;
211 const struct rt_config *rt_config;249 const struct rt_config *rt_config;
212 const struct dpm_clk_freq *dpm_clk_tbl;250 const struct dpm_clk_freq *dpm_clk_tbl;
251 const struct aie2_fw_feature_tbl *fw_feature_tbl;
213 252
214#define COL_ALIGN_NONE 0253#define COL_ALIGN_NONE 0
215#define COL_ALIGN_NATURE 1254#define COL_ALIGN_NATURE 1
@@ -217,6 +256,7 @@ struct amdxdna_dev_priv { · 1 unmodified line
217 u32 mbox_dev_addr;256 u32 mbox_dev_addr;
218 /* If mbox_size is 0, use BAR size. See MBOX_SIZE macro */257 /* If mbox_size is 0, use BAR size. See MBOX_SIZE macro */
219 u32 mbox_size;258 u32 mbox_size;
259 u32 hwctx_limit;
220 u32 sram_dev_addr;260 u32 sram_dev_addr;
221 struct aie2_bar_off_pair sram_offs[SRAM_MAX_INDEX];261 struct aie2_bar_off_pair sram_offs[SRAM_MAX_INDEX];
222 struct aie2_bar_off_pair psp_regs_off[PSP_MAX_REGS];262 struct aie2_bar_off_pair psp_regs_off[PSP_MAX_REGS];
@@ -234,6 +274,7 @@ extern const struct dpm_clk_freq npu1_dpm_clk_table[]; · 11 unmodified lines
234extern const struct dpm_clk_freq npu4_dpm_clk_table[];274extern const struct dpm_clk_freq npu4_dpm_clk_table[];
235extern const struct rt_config npu1_default_rt_cfg[];275extern const struct rt_config npu1_default_rt_cfg[];
236extern const struct rt_config npu4_default_rt_cfg[];276extern const struct rt_config npu4_default_rt_cfg[];
277extern const struct aie2_fw_feature_tbl npu4_fw_feature_table[];
237 278
238/* aie2_smu.c */279/* aie2_smu.c */
239int aie2_smu_init(struct amdxdna_dev_hdl *ndev);280int aie2_smu_init(struct amdxdna_dev_hdl *ndev);
@@ -253,10 +294,12 @@ void aie2_psp_stop(struct psp_device *psp); · 13 unmodified lines
253/* aie2_error.c */294/* aie2_error.c */
254int aie2_error_async_events_alloc(struct amdxdna_dev_hdl *ndev);295int aie2_error_async_events_alloc(struct amdxdna_dev_hdl *ndev);
255void aie2_error_async_events_free(struct amdxdna_dev_hdl *ndev);296void aie2_error_async_events_free(struct amdxdna_dev_hdl *ndev);
256int aie2_error_async_events_send(struct amdxdna_dev_hdl *ndev);
257int aie2_error_async_msg_thread(void *data);297int aie2_error_async_msg_thread(void *data);
298int aie2_get_array_async_error(struct amdxdna_dev_hdl *ndev,
299 struct amdxdna_drm_get_array *args);
258 300
259/* aie2_message.c */301/* aie2_message.c */
302void aie2_msg_init(struct amdxdna_dev_hdl *ndev);
260int aie2_suspend_fw(struct amdxdna_dev_hdl *ndev);303int aie2_suspend_fw(struct amdxdna_dev_hdl *ndev);
261int aie2_resume_fw(struct amdxdna_dev_hdl *ndev);304int aie2_resume_fw(struct amdxdna_dev_hdl *ndev);
262int aie2_set_runtime_cfg(struct amdxdna_dev_hdl *ndev, u32 type, u64 value);305int aie2_set_runtime_cfg(struct amdxdna_dev_hdl *ndev, u32 type, u64 value);
@@ -270,9 +313,13 @@ int aie2_create_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwct · 7 unmodified lines
270int aie2_destroy_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwctx);313int aie2_destroy_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwctx);
271int aie2_map_host_buf(struct amdxdna_dev_hdl *ndev, u32 context_id, u64 addr, u64 size);314int aie2_map_host_buf(struct amdxdna_dev_hdl *ndev, u32 context_id, u64 addr, u64 size);
272int aie2_query_status(struct amdxdna_dev_hdl *ndev, char __user *buf, u32 size, u32 *cols_filled);315int aie2_query_status(struct amdxdna_dev_hdl *ndev, char __user *buf, u32 size, u32 *cols_filled);
316int aie2_query_telemetry(struct amdxdna_dev_hdl *ndev,
317 char __user *buf, u32 size,
318 struct amdxdna_drm_query_telemetry_header *header);
273int aie2_register_asyn_event_msg(struct amdxdna_dev_hdl *ndev, dma_addr_t addr, u32 size,319int aie2_register_asyn_event_msg(struct amdxdna_dev_hdl *ndev, dma_addr_t addr, u32 size,
274 void *handle, int (*cb)(void*, void __iomem *, size_t));320 void *handle, int (*cb)(void*, void __iomem *, size_t));
275int aie2_config_cu(struct amdxdna_hwctx *hwctx);321int aie2_config_cu(struct amdxdna_hwctx *hwctx,
322 int (*notify_cb)(void *, void __iomem *, size_t));
276int aie2_execbuf(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job,323int aie2_execbuf(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job,
277 int (*notify_cb)(void *, void __iomem *, size_t));324 int (*notify_cb)(void *, void __iomem *, size_t));
278int aie2_cmdlist_single_execbuf(struct amdxdna_hwctx *hwctx,325int aie2_cmdlist_single_execbuf(struct amdxdna_hwctx *hwctx,
@@ -283,11 +330,14 @@ int aie2_cmdlist_multi_execbuf(struct amdxdna_hwctx *hwctx, · 4 unmodified lines
283 int (*notify_cb)(void *, void __iomem *, size_t));330 int (*notify_cb)(void *, void __iomem *, size_t));
284int aie2_sync_bo(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job,331int aie2_sync_bo(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job,
285 int (*notify_cb)(void *, void __iomem *, size_t));332 int (*notify_cb)(void *, void __iomem *, size_t));
333int aie2_config_debug_bo(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job,
334 int (*notify_cb)(void *, void __iomem *, size_t));
286 335
287/* aie2_hwctx.c */336/* aie2_hwctx.c */
288int aie2_hwctx_init(struct amdxdna_hwctx *hwctx);337int aie2_hwctx_init(struct amdxdna_hwctx *hwctx);
289void aie2_hwctx_fini(struct amdxdna_hwctx *hwctx);338void aie2_hwctx_fini(struct amdxdna_hwctx *hwctx);
290int aie2_hwctx_config(struct amdxdna_hwctx *hwctx, u32 type, u64 value, void *buf, u32 size);339int aie2_hwctx_config(struct amdxdna_hwctx *hwctx, u32 type, u64 value, void *buf, u32 size);
340int aie2_hwctx_sync_debug_bo(struct amdxdna_hwctx *hwctx, u32 debug_bo_hdl);
291void aie2_hwctx_suspend(struct amdxdna_client *client);341void aie2_hwctx_suspend(struct amdxdna_client *client);
292int aie2_hwctx_resume(struct amdxdna_client *client);342int aie2_hwctx_resume(struct amdxdna_client *client);
293int aie2_cmd_submit(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job, u64 *seq);343int aie2_cmd_submit(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job, u64 *seq);
drivers/accel/amdxdna/aie2_smu.c+45−4
@@ -11,6 +11,7 @@ · 10 unmodified lines
11 11
12#include "aie2_pci.h"12#include "aie2_pci.h"
13#include "amdxdna_pci_drv.h"13#include "amdxdna_pci_drv.h"
14#include "amdxdna_pm.h"
14 15
15#define SMU_RESULT_OK 116#define SMU_RESULT_OK 1
16 17
@@ -22,6 +23,13 @@ · 5 unmodified lines
22#define AIE2_SMU_SET_SOFT_DPMLEVEL 0x723#define AIE2_SMU_SET_SOFT_DPMLEVEL 0x7
23#define AIE2_SMU_SET_HARD_DPMLEVEL 0x824#define AIE2_SMU_SET_HARD_DPMLEVEL 0x8
24 25
26#define NPU4_DPM_TOPS(ndev, dpm_level) \
27({ \
28 typeof(ndev) _ndev = ndev; \
29 (4096 * (_ndev)->total_col * \
30 (_ndev)->priv->dpm_clk_tbl[dpm_level].hclk / 1000000); \
31})
32
25static int aie2_smu_exec(struct amdxdna_dev_hdl *ndev, u32 reg_cmd,33static int aie2_smu_exec(struct amdxdna_dev_hdl *ndev, u32 reg_cmd,
26 u32 reg_arg, u32 *out)34 u32 reg_arg, u32 *out)
27{35{
@@ -59,12 +67,16 @@ int npu1_set_dpm(struct amdxdna_dev_hdl *ndev, u32 dpm_level) · 31 unmodified lines
59 u32 freq;67 u32 freq;
60 int ret;68 int ret;
61 69
70 ret = amdxdna_pm_resume_get(ndev->xdna);
71 if (ret)
72 return ret;
73
62 ret = aie2_smu_exec(ndev, AIE2_SMU_SET_MPNPUCLK_FREQ,74 ret = aie2_smu_exec(ndev, AIE2_SMU_SET_MPNPUCLK_FREQ,
63 ndev->priv->dpm_clk_tbl[dpm_level].npuclk, &freq);75 ndev->priv->dpm_clk_tbl[dpm_level].npuclk, &freq);
64 if (ret) {76 if (ret) {
65 XDNA_ERR(ndev->xdna, "Set npu clock to %d failed, ret %d\n",77 XDNA_ERR(ndev->xdna, "Set npu clock to %d failed, ret %d\n",
66 ndev->priv->dpm_clk_tbl[dpm_level].npuclk, ret);78 ndev->priv->dpm_clk_tbl[dpm_level].npuclk, ret);
67 return ret;79 goto suspend_put;
68 }80 }
69 ndev->npuclk_freq = freq;81 ndev->npuclk_freq = freq;
70 82
@@ -73,49 +85,78 @@ int npu1_set_dpm(struct amdxdna_dev_hdl *ndev, u32 dpm_level) · 2 unmodified lines
73 if (ret) {85 if (ret) {
74 XDNA_ERR(ndev->xdna, "Set h clock to %d failed, ret %d\n",86 XDNA_ERR(ndev->xdna, "Set h clock to %d failed, ret %d\n",
75 ndev->priv->dpm_clk_tbl[dpm_level].hclk, ret);87 ndev->priv->dpm_clk_tbl[dpm_level].hclk, ret);
76 return ret;88 goto suspend_put;
77 }89 }
90
91 amdxdna_pm_suspend_put(ndev->xdna);
78 ndev->hclk_freq = freq;92 ndev->hclk_freq = freq;
79 ndev->dpm_level = dpm_level;93 ndev->dpm_level = dpm_level;
94 ndev->max_tops = 2 * ndev->total_col;
95 ndev->curr_tops = ndev->max_tops * freq / 1028;
80 96
81 XDNA_DBG(ndev->xdna, "MP-NPU clock %d, H clock %d\n",97 XDNA_DBG(ndev->xdna, "MP-NPU clock %d, H clock %d\n",
82 ndev->npuclk_freq, ndev->hclk_freq);98 ndev->npuclk_freq, ndev->hclk_freq);
83 99
84 return 0;100 return 0;
101
102suspend_put:
103 amdxdna_pm_suspend_put(ndev->xdna);
104 return ret;
85}105}
86 106
87int npu4_set_dpm(struct amdxdna_dev_hdl *ndev, u32 dpm_level)107int npu4_set_dpm(struct amdxdna_dev_hdl *ndev, u32 dpm_level)
88{108{
89 int ret;109 int ret;
90 110
111 ret = amdxdna_pm_resume_get(ndev->xdna);
112 if (ret)
113 return ret;
114
91 ret = aie2_smu_exec(ndev, AIE2_SMU_SET_HARD_DPMLEVEL, dpm_level, NULL);115 ret = aie2_smu_exec(ndev, AIE2_SMU_SET_HARD_DPMLEVEL, dpm_level, NULL);
92 if (ret) {116 if (ret) {
93 XDNA_ERR(ndev->xdna, "Set hard dpm level %d failed, ret %d ",117 XDNA_ERR(ndev->xdna, "Set hard dpm level %d failed, ret %d ",
94 dpm_level, ret);118 dpm_level, ret);
95 return ret;119 goto suspend_put;
96 }120 }
97 121
98 ret = aie2_smu_exec(ndev, AIE2_SMU_SET_SOFT_DPMLEVEL, dpm_level, NULL);122 ret = aie2_smu_exec(ndev, AIE2_SMU_SET_SOFT_DPMLEVEL, dpm_level, NULL);
99 if (ret) {123 if (ret) {
100 XDNA_ERR(ndev->xdna, "Set soft dpm level %d failed, ret %d",124 XDNA_ERR(ndev->xdna, "Set soft dpm level %d failed, ret %d",
101 dpm_level, ret);125 dpm_level, ret);
102 return ret;126 goto suspend_put;
103 }127 }
104 128
129 amdxdna_pm_suspend_put(ndev->xdna);
105 ndev->npuclk_freq = ndev->priv->dpm_clk_tbl[dpm_level].npuclk;130 ndev->npuclk_freq = ndev->priv->dpm_clk_tbl[dpm_level].npuclk;
106 ndev->hclk_freq = ndev->priv->dpm_clk_tbl[dpm_level].hclk;131 ndev->hclk_freq = ndev->priv->dpm_clk_tbl[dpm_level].hclk;
107 ndev->dpm_level = dpm_level;132 ndev->dpm_level = dpm_level;
133 ndev->max_tops = NPU4_DPM_TOPS(ndev, ndev->max_dpm_level);
134 ndev->curr_tops = NPU4_DPM_TOPS(ndev, dpm_level);
108 135
109 XDNA_DBG(ndev->xdna, "MP-NPU clock %d, H clock %d\n",136 XDNA_DBG(ndev->xdna, "MP-NPU clock %d, H clock %d\n",
110 ndev->npuclk_freq, ndev->hclk_freq);137 ndev->npuclk_freq, ndev->hclk_freq);
111 138
112 return 0;139 return 0;
140
141suspend_put:
142 amdxdna_pm_suspend_put(ndev->xdna);
143 return ret;
113}144}
114 145
115int aie2_smu_init(struct amdxdna_dev_hdl *ndev)146int aie2_smu_init(struct amdxdna_dev_hdl *ndev)
116{147{
117 int ret;148 int ret;
118 149
150 /*
151 * Failing to set power off indicates an unrecoverable hardware or
152 * firmware error.
153 */
154 ret = aie2_smu_exec(ndev, AIE2_SMU_POWER_OFF, 0, NULL);
155 if (ret) {
156 XDNA_ERR(ndev->xdna, "Access power failed, ret %d", ret);
157 return ret;
158 }
159
119 ret = aie2_smu_exec(ndev, AIE2_SMU_POWER_ON, 0, NULL);160 ret = aie2_smu_exec(ndev, AIE2_SMU_POWER_ON, 0, NULL);
120 if (ret) {161 if (ret) {
121 XDNA_ERR(ndev->xdna, "Power on failed, ret %d", ret);162 XDNA_ERR(ndev->xdna, "Power on failed, ret %d", ret);
drivers/accel/amdxdna/amdxdna_ctx.c+67−37
@@ -113,14 +113,14 @@ void *amdxdna_cmd_get_payload(struct amdxdna_gem_obj *abo, u32 *size) · 112 unmodified lines
113 return &cmd->data[num_masks];113 return &cmd->data[num_masks];
114}114}
115 115
116int amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo)116u32 amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo)
117{117{
118 struct amdxdna_cmd *cmd = abo->mem.kva;118 struct amdxdna_cmd *cmd = abo->mem.kva;
119 u32 num_masks, i;119 u32 num_masks, i;
120 u32 *cu_mask;120 u32 *cu_mask;
121 121
122 if (amdxdna_cmd_get_op(abo) == ERT_CMD_CHAIN)122 if (amdxdna_cmd_get_op(abo) == ERT_CMD_CHAIN)
123 return -1;123 return INVALID_CU_IDX;
124 124
125 num_masks = 1 + FIELD_GET(AMDXDNA_CMD_EXTRA_CU_MASK, cmd->header);125 num_masks = 1 + FIELD_GET(AMDXDNA_CMD_EXTRA_CU_MASK, cmd->header);
126 cu_mask = cmd->data;126 cu_mask = cmd->data;
@@ -129,7 +129,7 @@ int amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo) · 2 unmodified lines
129 return ffs(cu_mask[i]) - 1;129 return ffs(cu_mask[i]) - 1;
130 }130 }
131 131
132 return -1;132 return INVALID_CU_IDX;
133}133}
134 134
135/*135/*
@@ -161,19 +161,14 @@ int amdxdna_drm_create_hwctx_ioctl(struct drm_device *dev, void *data, struct dr · 25 unmodified lines
161 if (args->ext || args->ext_flags)161 if (args->ext || args->ext_flags)
162 return -EINVAL;162 return -EINVAL;
163 163
164 if (!drm_dev_enter(dev, &idx))
165 return -ENODEV;
166
167 hwctx = kzalloc(sizeof(*hwctx), GFP_KERNEL);164 hwctx = kzalloc(sizeof(*hwctx), GFP_KERNEL);
168 if (!hwctx) {165 if (!hwctx)
169 ret = -ENOMEM;166 return -ENOMEM;
170 goto exit;
171 }
172 167
173 if (copy_from_user(&hwctx->qos, u64_to_user_ptr(args->qos_p), sizeof(hwctx->qos))) {168 if (copy_from_user(&hwctx->qos, u64_to_user_ptr(args->qos_p), sizeof(hwctx->qos))) {
174 XDNA_ERR(xdna, "Access QoS info failed");169 XDNA_ERR(xdna, "Access QoS info failed");
175 ret = -EFAULT;170 kfree(hwctx);
176 goto free_hwctx;171 return -EFAULT;
177 }172 }
178 173
179 hwctx->client = client;174 hwctx->client = client;
@@ -181,30 +176,36 @@ int amdxdna_drm_create_hwctx_ioctl(struct drm_device *dev, void *data, struct dr · 1 unmodified line
181 hwctx->num_tiles = args->num_tiles;176 hwctx->num_tiles = args->num_tiles;
182 hwctx->mem_size = args->mem_size;177 hwctx->mem_size = args->mem_size;
183 hwctx->max_opc = args->max_opc;178 hwctx->max_opc = args->max_opc;
184 ret = xa_alloc_cyclic(&client->hwctx_xa, &hwctx->id, hwctx,179
185 XA_LIMIT(AMDXDNA_INVALID_CTX_HANDLE + 1, MAX_HWCTX_ID),180 guard(mutex)(&xdna->dev_lock);
186 &client->next_hwctxid, GFP_KERNEL);181
187 if (ret < 0) {182 if (!drm_dev_enter(dev, &idx)) {
188 XDNA_ERR(xdna, "Allocate hwctx ID failed, ret %d", ret);183 ret = -ENODEV;
189 goto free_hwctx;184 goto free_hwctx;
190 }185 }
191 186
192 hwctx->name = kasprintf(GFP_KERNEL, "hwctx.%d.%d", client->pid, hwctx->id);187 ret = xdna->dev_info->ops->hwctx_init(hwctx);
188 if (ret) {
189 XDNA_ERR(xdna, "Init hwctx failed, ret %d", ret);
190 goto dev_exit;
191 }
192
193 hwctx->name = kasprintf(GFP_KERNEL, "hwctx.%d.%d", client->pid, hwctx->fw_ctx_id);
193 if (!hwctx->name) {194 if (!hwctx->name) {
194 ret = -ENOMEM;195 ret = -ENOMEM;
195 goto rm_id;196 goto fini_hwctx;
196 }197 }
197 198
198 mutex_lock(&xdna->dev_lock);199 ret = xa_alloc_cyclic(&client->hwctx_xa, &hwctx->id, hwctx,
199 ret = xdna->dev_info->ops->hwctx_init(hwctx);200 XA_LIMIT(AMDXDNA_INVALID_CTX_HANDLE + 1, MAX_HWCTX_ID),
200 if (ret) {201 &client->next_hwctxid, GFP_KERNEL);
201 mutex_unlock(&xdna->dev_lock);202 if (ret < 0) {
202 XDNA_ERR(xdna, "Init hwctx failed, ret %d", ret);203 XDNA_ERR(xdna, "Allocate hwctx ID failed, ret %d", ret);
203 goto free_name;204 goto free_name;
204 }205 }
206
205 args->handle = hwctx->id;207 args->handle = hwctx->id;
206 args->syncobj_handle = hwctx->syncobj_hdl;208 args->syncobj_handle = hwctx->syncobj_hdl;
207 mutex_unlock(&xdna->dev_lock);
208 209
209 atomic64_set(&hwctx->job_submit_cnt, 0);210 atomic64_set(&hwctx->job_submit_cnt, 0);
210 atomic64_set(&hwctx->job_free_cnt, 0);211 atomic64_set(&hwctx->job_free_cnt, 0);
@@ -214,12 +215,12 @@ int amdxdna_drm_create_hwctx_ioctl(struct drm_device *dev, void *data, struct dr · 3 unmodified lines
214 215
215free_name:216free_name:
216 kfree(hwctx->name);217 kfree(hwctx->name);
217rm_id:218fini_hwctx:
218 xa_erase(&client->hwctx_xa, hwctx->id);219 xdna->dev_info->ops->hwctx_fini(hwctx);
220dev_exit:
221 drm_dev_exit(idx);
219free_hwctx:222free_hwctx:
220 kfree(hwctx);223 kfree(hwctx);
221exit:
222 drm_dev_exit(idx);
223 return ret;224 return ret;
224}225}
225 226
@@ -327,6 +328,38 @@ unlock_srcu: · 101 unmodified lines
327 return ret;328 return ret;
328}329}
329 330
331int amdxdna_hwctx_sync_debug_bo(struct amdxdna_client *client, u32 debug_bo_hdl)
332{
333 struct amdxdna_dev *xdna = client->xdna;
334 struct amdxdna_hwctx *hwctx;
335 struct amdxdna_gem_obj *abo;
336 struct drm_gem_object *gobj;
337 int ret, idx;
338
339 if (!xdna->dev_info->ops->hwctx_sync_debug_bo)
340 return -EOPNOTSUPP;
341
342 gobj = drm_gem_object_lookup(client->filp, debug_bo_hdl);
343 if (!gobj)
344 return -EINVAL;
345
346 abo = to_xdna_obj(gobj);
347 guard(mutex)(&xdna->dev_lock);
348 idx = srcu_read_lock(&client->hwctx_srcu);
349 hwctx = xa_load(&client->hwctx_xa, abo->assigned_hwctx);
350 if (!hwctx) {
351 ret = -EINVAL;
352 goto unlock_srcu;
353 }
354
355 ret = xdna->dev_info->ops->hwctx_sync_debug_bo(hwctx, debug_bo_hdl);
356
357unlock_srcu:
358 srcu_read_unlock(&client->hwctx_srcu, idx);
359 drm_gem_object_put(gobj);
360 return ret;
361}
362
330static void363static void
331amdxdna_arg_bos_put(struct amdxdna_sched_job *job)364amdxdna_arg_bos_put(struct amdxdna_sched_job *job)
332{365{
@@ -389,9 +422,11 @@ void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job) · 56 unmodified lines
389 trace_amdxdna_debug_point(job->hwctx->name, job->seq, "job release");422 trace_amdxdna_debug_point(job->hwctx->name, job->seq, "job release");
390 amdxdna_arg_bos_put(job);423 amdxdna_arg_bos_put(job);
391 amdxdna_gem_put_obj(job->cmd_bo);424 amdxdna_gem_put_obj(job->cmd_bo);
425 dma_fence_put(job->fence);
392}426}
393 427
394int amdxdna_cmd_submit(struct amdxdna_client *client,428int amdxdna_cmd_submit(struct amdxdna_client *client,
429 struct amdxdna_drv_cmd *drv_cmd,
395 u32 cmd_bo_hdl, u32 *arg_bo_hdls, u32 arg_bo_cnt,430 u32 cmd_bo_hdl, u32 *arg_bo_hdls, u32 arg_bo_cnt,
396 u32 hwctx_hdl, u64 *seq)431 u32 hwctx_hdl, u64 *seq)
397{432{
@@ -405,6 +440,8 @@ int amdxdna_cmd_submit(struct amdxdna_client *client, · 7 unmodified lines
405 if (!job)440 if (!job)
406 return -ENOMEM;441 return -ENOMEM;
407 442
443 job->drv_cmd = drv_cmd;
444
408 if (cmd_bo_hdl != AMDXDNA_INVALID_BO_HANDLE) {445 if (cmd_bo_hdl != AMDXDNA_INVALID_BO_HANDLE) {
409 job->cmd_bo = amdxdna_gem_get_obj(client, cmd_bo_hdl, AMDXDNA_BO_CMD);446 job->cmd_bo = amdxdna_gem_get_obj(client, cmd_bo_hdl, AMDXDNA_BO_CMD);
410 if (!job->cmd_bo) {447 if (!job->cmd_bo) {
@@ -412,8 +449,6 @@ int amdxdna_cmd_submit(struct amdxdna_client *client, · 1 unmodified line
412 ret = -EINVAL;449 ret = -EINVAL;
413 goto free_job;450 goto free_job;
414 }451 }
415 } else {
416 job->cmd_bo = NULL;
417 }452 }
418 453
419 ret = amdxdna_arg_bos_lookup(client, job, arg_bo_hdls, arg_bo_cnt);454 ret = amdxdna_arg_bos_lookup(client, job, arg_bo_hdls, arg_bo_cnt);
@@ -431,11 +466,6 @@ int amdxdna_cmd_submit(struct amdxdna_client *client, · 11 unmodified lines
431 goto unlock_srcu;466 goto unlock_srcu;
432 }467 }
433 468
434 if (hwctx->status != HWCTX_STAT_READY) {
435 XDNA_ERR(xdna, "HW Context is not ready");
436 ret = -EINVAL;
437 goto unlock_srcu;
438 }
439 469
440 job->hwctx = hwctx;470 job->hwctx = hwctx;
441 job->mm = current->mm;471 job->mm = current->mm;
@@ -512,7 +542,7 @@ static int amdxdna_drm_submit_execbuf(struct amdxdna_client *client, · 70 unmodified lines
512 }542 }
513 }543 }
514 544
515 ret = amdxdna_cmd_submit(client, cmd_bo_hdl, arg_bo_hdls,545 ret = amdxdna_cmd_submit(client, NULL, cmd_bo_hdl, arg_bo_hdls,
516 args->arg_count, args->hwctx, &args->seq);546 args->arg_count, args->hwctx, &args->seq);
517 if (ret)547 if (ret)
518 XDNA_DBG(xdna, "Submit cmds failed, ret %d", ret);548 XDNA_DBG(xdna, "Submit cmds failed, ret %d", ret);
drivers/accel/amdxdna/amdxdna_ctx.h+40−5
@@ -13,9 +13,12 @@ · 12 unmodified lines
13struct amdxdna_hwctx_priv;13struct amdxdna_hwctx_priv;
14 14
15enum ert_cmd_opcode {15enum ert_cmd_opcode {
16 ERT_START_CU = 0,16 ERT_START_CU = 0,
17 ERT_CMD_CHAIN = 19,17 ERT_CMD_CHAIN = 19,
18 ERT_START_NPU = 20,18 ERT_START_NPU = 20,
19 ERT_START_NPU_PREEMPT = 21,
20 ERT_START_NPU_PREEMPT_ELF = 22,
21 ERT_INVALID_CMD = ~0U,
19};22};
20 23
21enum ert_cmd_state {24enum ert_cmd_state {
@@ -54,6 +57,21 @@ struct amdxdna_cmd_chain { · 32 unmodified lines
54 u64 data[] __counted_by(command_count);57 u64 data[] __counted_by(command_count);
55};58};
56 59
60/*
61 * Interpretation of the beginning of data payload for ERT_START_NPU_PREEMPT in
62 * amdxdna_cmd. The rest of the payload in amdxdna_cmd is regular kernel args.
63 */
64struct amdxdna_cmd_preempt_data {
65 u64 inst_buf; /* instruction buffer address */
66 u64 save_buf; /* save buffer address */
67 u64 restore_buf; /* restore buffer address */
68 u32 inst_size; /* size of instruction buffer in bytes */
69 u32 save_size; /* size of save buffer in bytes */
70 u32 restore_size; /* size of restore buffer in bytes */
71 u32 inst_prop_cnt; /* properties count */
72 u32 prop_args[]; /* properties and regular kernel arguments */
73};
74
57/* Exec buffer command header format */75/* Exec buffer command header format */
58#define AMDXDNA_CMD_STATE GENMASK(3, 0)76#define AMDXDNA_CMD_STATE GENMASK(3, 0)
59#define AMDXDNA_CMD_EXTRA_CU_MASK GENMASK(11, 10)77#define AMDXDNA_CMD_EXTRA_CU_MASK GENMASK(11, 10)
@@ -64,6 +82,8 @@ struct amdxdna_cmd { · 4 unmodified lines
64 u32 data[];82 u32 data[];
65};83};
66 84
85#define INVALID_CU_IDX (~0U)
86
67struct amdxdna_hwctx {87struct amdxdna_hwctx {
68 struct amdxdna_client *client;88 struct amdxdna_client *client;
69 struct amdxdna_hwctx_priv *priv;89 struct amdxdna_hwctx_priv *priv;
@@ -95,6 +115,17 @@ struct amdxdna_hwctx { · 25 unmodified lines
95#define drm_job_to_xdna_job(j) \115#define drm_job_to_xdna_job(j) \
96 container_of(j, struct amdxdna_sched_job, base)116 container_of(j, struct amdxdna_sched_job, base)
97 117
118enum amdxdna_job_opcode {
119 SYNC_DEBUG_BO,
120 ATTACH_DEBUG_BO,
121 DETACH_DEBUG_BO,
122};
123
124struct amdxdna_drv_cmd {
125 enum amdxdna_job_opcode opcode;
126 u32 result;
127};
128
98struct amdxdna_sched_job {129struct amdxdna_sched_job {
99 struct drm_sched_job base;130 struct drm_sched_job base;
100 struct kref refcnt;131 struct kref refcnt;
@@ -105,7 +136,9 @@ struct amdxdna_sched_job { · 4 unmodified lines
105 /* user can wait on this fence */136 /* user can wait on this fence */
106 struct dma_fence *out_fence;137 struct dma_fence *out_fence;
107 bool job_done;138 bool job_done;
139 bool job_timeout;
108 u64 seq;140 u64 seq;
141 struct amdxdna_drv_cmd *drv_cmd;
109 struct amdxdna_gem_obj *cmd_bo;142 struct amdxdna_gem_obj *cmd_bo;
110 size_t bo_cnt;143 size_t bo_cnt;
111 struct drm_gem_object *bos[] __counted_by(bo_cnt);144 struct drm_gem_object *bos[] __counted_by(bo_cnt);
@@ -137,15 +170,17 @@ amdxdna_cmd_get_state(struct amdxdna_gem_obj *abo) · 25 unmodified lines
137}170}
138 171
139void *amdxdna_cmd_get_payload(struct amdxdna_gem_obj *abo, u32 *size);172void *amdxdna_cmd_get_payload(struct amdxdna_gem_obj *abo, u32 *size);
140int amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo);173u32 amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo);
141 174
142void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job);175void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job);
143void amdxdna_hwctx_remove_all(struct amdxdna_client *client);176void amdxdna_hwctx_remove_all(struct amdxdna_client *client);
144int amdxdna_hwctx_walk(struct amdxdna_client *client, void *arg,177int amdxdna_hwctx_walk(struct amdxdna_client *client, void *arg,
145 int (*walk)(struct amdxdna_hwctx *hwctx, void *arg));178 int (*walk)(struct amdxdna_hwctx *hwctx, void *arg));
179int amdxdna_hwctx_sync_debug_bo(struct amdxdna_client *client, u32 debug_bo_hdl);
146 180
147int amdxdna_cmd_submit(struct amdxdna_client *client,181int amdxdna_cmd_submit(struct amdxdna_client *client,
148 u32 cmd_bo_hdls, u32 *arg_bo_hdls, u32 arg_bo_cnt,182 struct amdxdna_drv_cmd *drv_cmd, u32 cmd_bo_hdls,
183 u32 *arg_bo_hdls, u32 arg_bo_cnt,
149 u32 hwctx_hdl, u64 *seq);184 u32 hwctx_hdl, u64 *seq);
150 185
151int amdxdna_cmd_wait(struct amdxdna_client *client, u32 hwctx_hdl,186int amdxdna_cmd_wait(struct amdxdna_client *client, u32 hwctx_hdl,
drivers/accel/amdxdna/amdxdna_error.h+59−0
@@ -0,0 +1,59 @@
1/* SPDX-License-Identifier: GPL-2.0 */
2/*
3 * Copyright (C) 2025, Advanced Micro Devices, Inc.
4 */
5
6#ifndef _AMDXDNA_ERROR_H_
7#define _AMDXDNA_ERROR_H_
8
9#include <linux/bitfield.h>
10#include <linux/bits.h>
11
12#define AMDXDNA_ERR_DRV_AIE 4
13#define AMDXDNA_ERR_SEV_CRITICAL 3
14#define AMDXDNA_ERR_CLASS_AIE 2
15
16#define AMDXDNA_ERR_NUM_MASK GENMASK_U64(15, 0)
17#define AMDXDNA_ERR_DRV_MASK GENMASK_U64(23, 16)
18#define AMDXDNA_ERR_SEV_MASK GENMASK_U64(31, 24)
19#define AMDXDNA_ERR_MOD_MASK GENMASK_U64(39, 32)
20#define AMDXDNA_ERR_CLASS_MASK GENMASK_U64(47, 40)
21
22enum amdxdna_error_num {
23 AMDXDNA_ERROR_NUM_AIE_SATURATION = 3,
24 AMDXDNA_ERROR_NUM_AIE_FP,
25 AMDXDNA_ERROR_NUM_AIE_STREAM,
26 AMDXDNA_ERROR_NUM_AIE_ACCESS,
27 AMDXDNA_ERROR_NUM_AIE_BUS,
28 AMDXDNA_ERROR_NUM_AIE_INSTRUCTION,
29 AMDXDNA_ERROR_NUM_AIE_ECC,
30 AMDXDNA_ERROR_NUM_AIE_LOCK,
31 AMDXDNA_ERROR_NUM_AIE_DMA,
32 AMDXDNA_ERROR_NUM_AIE_MEM_PARITY,
33 AMDXDNA_ERROR_NUM_UNKNOWN = 15,
34};
35
36enum amdxdna_error_module {
37 AMDXDNA_ERROR_MODULE_AIE_CORE = 3,
38 AMDXDNA_ERROR_MODULE_AIE_MEMORY,
39 AMDXDNA_ERROR_MODULE_AIE_SHIM,
40 AMDXDNA_ERROR_MODULE_AIE_NOC,
41 AMDXDNA_ERROR_MODULE_AIE_PL,
42 AMDXDNA_ERROR_MODULE_UNKNOWN = 8,
43};
44
45#define AMDXDNA_ERROR_ENCODE(err_num, err_mod) \
46 (FIELD_PREP(AMDXDNA_ERR_NUM_MASK, err_num) | \
47 FIELD_PREP_CONST(AMDXDNA_ERR_DRV_MASK, AMDXDNA_ERR_DRV_AIE) | \
48 FIELD_PREP_CONST(AMDXDNA_ERR_SEV_MASK, AMDXDNA_ERR_SEV_CRITICAL) | \
49 FIELD_PREP(AMDXDNA_ERR_MOD_MASK, err_mod) | \
50 FIELD_PREP_CONST(AMDXDNA_ERR_CLASS_MASK, AMDXDNA_ERR_CLASS_AIE))
51
52#define AMDXDNA_EXTRA_ERR_COL_MASK GENMASK_U64(7, 0)
53#define AMDXDNA_EXTRA_ERR_ROW_MASK GENMASK_U64(15, 8)
54
55#define AMDXDNA_EXTRA_ERR_ENCODE(row, col) \
56 (FIELD_PREP(AMDXDNA_EXTRA_ERR_COL_MASK, col) | \
57 FIELD_PREP(AMDXDNA_EXTRA_ERR_ROW_MASK, row))
58
59#endif /* _AMDXDNA_ERROR_H_ */
drivers/accel/amdxdna/amdxdna_gem.c+24−27
@@ -8,6 +8,7 @@ · 7 unmodified lines
8#include <drm/drm_device.h>8#include <drm/drm_device.h>
9#include <drm/drm_gem.h>9#include <drm/drm_gem.h>
10#include <drm/drm_gem_shmem_helper.h>10#include <drm/drm_gem_shmem_helper.h>
11#include <drm/drm_print.h>
11#include <drm/gpu_scheduler.h>12#include <drm/gpu_scheduler.h>
12#include <linux/dma-buf.h>13#include <linux/dma-buf.h>
13#include <linux/dma-direct.h>14#include <linux/dma-direct.h>
@@ -392,35 +393,33 @@ static const struct dma_buf_ops amdxdna_dmabuf_ops = { · 378 unmodified lines
392 .vunmap = drm_gem_dmabuf_vunmap,393 .vunmap = drm_gem_dmabuf_vunmap,
393};394};
394 395
395static int amdxdna_gem_obj_vmap(struct drm_gem_object *obj, struct iosys_map *map)396static int amdxdna_gem_obj_vmap(struct amdxdna_gem_obj *abo, void **vaddr)
396{397{
397 struct amdxdna_gem_obj *abo = to_xdna_obj(obj);398 struct iosys_map map = IOSYS_MAP_INIT_VADDR(NULL);
398 399 int ret;
399 iosys_map_clear(map);
400
401 dma_resv_assert_held(obj->resv);
402 400
403 if (is_import_bo(abo))401 if (is_import_bo(abo))
404 dma_buf_vmap(abo->dma_buf, map);402 ret = dma_buf_vmap_unlocked(abo->dma_buf, &map);
405 else403 else
406 drm_gem_shmem_object_vmap(obj, map);404 ret = drm_gem_vmap(to_gobj(abo), &map);
407 405
408 if (!map->vaddr)406 *vaddr = map.vaddr;
409 return -ENOMEM;407 return ret;
410
411 return 0;
412}408}
413 409
414static void amdxdna_gem_obj_vunmap(struct drm_gem_object *obj, struct iosys_map *map)410static void amdxdna_gem_obj_vunmap(struct amdxdna_gem_obj *abo)
415{411{
416 struct amdxdna_gem_obj *abo = to_xdna_obj(obj);412 struct iosys_map map;
413
414 if (!abo->mem.kva)
415 return;
417 416
418 dma_resv_assert_held(obj->resv);417 iosys_map_set_vaddr(&map, abo->mem.kva);
419 418
420 if (is_import_bo(abo))419 if (is_import_bo(abo))
421 dma_buf_vunmap(abo->dma_buf, map);420 dma_buf_vunmap_unlocked(abo->dma_buf, &map);
422 else421 else
423 drm_gem_shmem_object_vunmap(obj, map);422 drm_gem_vunmap(to_gobj(abo), &map);
424}423}
425 424
426static struct dma_buf *amdxdna_gem_prime_export(struct drm_gem_object *gobj, int flags)425static struct dma_buf *amdxdna_gem_prime_export(struct drm_gem_object *gobj, int flags)
@@ -455,7 +454,6 @@ static void amdxdna_gem_obj_free(struct drm_gem_object *gobj) · 28 unmodified lines
455{454{
456 struct amdxdna_dev *xdna = to_xdna_dev(gobj->dev);455 struct amdxdna_dev *xdna = to_xdna_dev(gobj->dev);
457 struct amdxdna_gem_obj *abo = to_xdna_obj(gobj);456 struct amdxdna_gem_obj *abo = to_xdna_obj(gobj);
458 struct iosys_map map = IOSYS_MAP_INIT_VADDR(abo->mem.kva);
459 457
460 XDNA_DBG(xdna, "BO type %d xdna_addr 0x%llx", abo->type, abo->mem.dev_addr);458 XDNA_DBG(xdna, "BO type %d xdna_addr 0x%llx", abo->type, abo->mem.dev_addr);
461 459
@@ -468,7 +466,7 @@ static void amdxdna_gem_obj_free(struct drm_gem_object *gobj) · 6 unmodified lines
468 if (abo->type == AMDXDNA_BO_DEV_HEAP)466 if (abo->type == AMDXDNA_BO_DEV_HEAP)
469 drm_mm_takedown(&abo->mm);467 drm_mm_takedown(&abo->mm);
470 468
471 drm_gem_vunmap(gobj, &map);469 amdxdna_gem_obj_vunmap(abo);
472 mutex_destroy(&abo->lock);470 mutex_destroy(&abo->lock);
473 471
474 if (is_import_bo(abo)) {472 if (is_import_bo(abo)) {
@@ -489,8 +487,8 @@ static const struct drm_gem_object_funcs amdxdna_gem_shmem_funcs = { · 14 unmodified lines
489 .pin = drm_gem_shmem_object_pin,487 .pin = drm_gem_shmem_object_pin,
490 .unpin = drm_gem_shmem_object_unpin,488 .unpin = drm_gem_shmem_object_unpin,
491 .get_sg_table = drm_gem_shmem_object_get_sg_table,489 .get_sg_table = drm_gem_shmem_object_get_sg_table,
492 .vmap = amdxdna_gem_obj_vmap,490 .vmap = drm_gem_shmem_object_vmap,
493 .vunmap = amdxdna_gem_obj_vunmap,491 .vunmap = drm_gem_shmem_object_vunmap,
494 .mmap = amdxdna_gem_obj_mmap,492 .mmap = amdxdna_gem_obj_mmap,
495 .vm_ops = &drm_gem_shmem_vm_ops,493 .vm_ops = &drm_gem_shmem_vm_ops,
496 .export = amdxdna_gem_prime_export,494 .export = amdxdna_gem_prime_export,
@@ -663,7 +661,6 @@ amdxdna_drm_create_dev_heap(struct drm_device *dev, · 166 unmodified lines
663 struct drm_file *filp)661 struct drm_file *filp)
664{662{
665 struct amdxdna_client *client = filp->driver_priv;663 struct amdxdna_client *client = filp->driver_priv;
666 struct iosys_map map = IOSYS_MAP_INIT_VADDR(NULL);
667 struct amdxdna_dev *xdna = to_xdna_dev(dev);664 struct amdxdna_dev *xdna = to_xdna_dev(dev);
668 struct amdxdna_gem_obj *abo;665 struct amdxdna_gem_obj *abo;
669 int ret;666 int ret;
@@ -692,12 +689,11 @@ amdxdna_drm_create_dev_heap(struct drm_device *dev, · 22 unmodified lines
692 abo->mem.dev_addr = client->xdna->dev_info->dev_mem_base;689 abo->mem.dev_addr = client->xdna->dev_info->dev_mem_base;
693 drm_mm_init(&abo->mm, abo->mem.dev_addr, abo->mem.size);690 drm_mm_init(&abo->mm, abo->mem.dev_addr, abo->mem.size);
694 691
695 ret = drm_gem_vmap(to_gobj(abo), &map);692 ret = amdxdna_gem_obj_vmap(abo, &abo->mem.kva);
696 if (ret) {693 if (ret) {
697 XDNA_ERR(xdna, "Vmap heap bo failed, ret %d", ret);694 XDNA_ERR(xdna, "Vmap heap bo failed, ret %d", ret);
698 goto release_obj;695 goto release_obj;
699 }696 }
700 abo->mem.kva = map.vaddr;
701 697
702 client->dev_heap = abo;698 client->dev_heap = abo;
703 drm_gem_object_get(to_gobj(abo));699 drm_gem_object_get(to_gobj(abo));
@@ -748,7 +744,6 @@ amdxdna_drm_create_cmd_bo(struct drm_device *dev, · 44 unmodified lines
748 struct amdxdna_drm_create_bo *args,744 struct amdxdna_drm_create_bo *args,
749 struct drm_file *filp)745 struct drm_file *filp)
750{746{
751 struct iosys_map map = IOSYS_MAP_INIT_VADDR(NULL);
752 struct amdxdna_dev *xdna = to_xdna_dev(dev);747 struct amdxdna_dev *xdna = to_xdna_dev(dev);
753 struct amdxdna_gem_obj *abo;748 struct amdxdna_gem_obj *abo;
754 int ret;749 int ret;
@@ -770,12 +765,11 @@ amdxdna_drm_create_cmd_bo(struct drm_device *dev, · 15 unmodified lines
770 abo->type = AMDXDNA_BO_CMD;765 abo->type = AMDXDNA_BO_CMD;
771 abo->client = filp->driver_priv;766 abo->client = filp->driver_priv;
772 767
773 ret = drm_gem_vmap(to_gobj(abo), &map);768 ret = amdxdna_gem_obj_vmap(abo, &abo->mem.kva);
774 if (ret) {769 if (ret) {
775 XDNA_ERR(xdna, "Vmap cmd bo failed, ret %d", ret);770 XDNA_ERR(xdna, "Vmap cmd bo failed, ret %d", ret);
776 goto release_obj;771 goto release_obj;
777 }772 }
778 abo->mem.kva = map.vaddr;
779 773
780 return abo;774 return abo;
781 775
@@ -969,6 +963,9 @@ int amdxdna_drm_sync_bo_ioctl(struct drm_device *dev, · 187 unmodified lines
969 XDNA_DBG(xdna, "Sync bo %d offset 0x%llx, size 0x%llx\n",963 XDNA_DBG(xdna, "Sync bo %d offset 0x%llx, size 0x%llx\n",
970 args->handle, args->offset, args->size);964 args->handle, args->offset, args->size);
971 965
966 if (args->direction == SYNC_DIRECT_FROM_DEVICE)
967 ret = amdxdna_hwctx_sync_debug_bo(abo->client, args->handle);
968
972put_obj:969put_obj:
973 drm_gem_object_put(gobj);970 drm_gem_object_put(gobj);
974 return ret;971 return ret;
drivers/accel/amdxdna/amdxdna_gem.h+6−0
@@ -7,6 +7,7 @@ · 6 unmodified lines
7#define _AMDXDNA_GEM_H_7#define _AMDXDNA_GEM_H_
8 8
9#include <linux/hmm.h>9#include <linux/hmm.h>
10#include "amdxdna_pci_drv.h"
10 11
11struct amdxdna_umap {12struct amdxdna_umap {
12 struct vm_area_struct *vma;13 struct vm_area_struct *vma;
@@ -62,6 +63,11 @@ static inline void amdxdna_gem_put_obj(struct amdxdna_gem_obj *abo) · 49 unmodified lines
62 drm_gem_object_put(to_gobj(abo));63 drm_gem_object_put(to_gobj(abo));
63}64}
64 65
66static inline u64 amdxdna_dev_bo_offset(struct amdxdna_gem_obj *abo)
67{
68 return abo->mem.dev_addr - abo->client->dev_heap->mem.dev_addr;
69}
70
65void amdxdna_umap_put(struct amdxdna_umap *mapp);71void amdxdna_umap_put(struct amdxdna_umap *mapp);
66 72
67struct drm_gem_object *73struct drm_gem_object *
drivers/accel/amdxdna/amdxdna_mailbox.c+9−5
@@ -194,7 +194,8 @@ static void mailbox_release_msg(struct mailbox_channel *mb_chann, · 193 unmodified lines
194{194{
195 MB_DBG(mb_chann, "msg_id 0x%x msg opcode 0x%x",195 MB_DBG(mb_chann, "msg_id 0x%x msg opcode 0x%x",
196 mb_msg->pkg.header.id, mb_msg->pkg.header.opcode);196 mb_msg->pkg.header.id, mb_msg->pkg.header.opcode);
197 mb_msg->notify_cb(mb_msg->handle, NULL, 0);197 if (mb_msg->notify_cb)
198 mb_msg->notify_cb(mb_msg->handle, NULL, 0);
198 kfree(mb_msg);199 kfree(mb_msg);
199}200}
200 201
@@ -248,7 +249,7 @@ mailbox_get_resp(struct mailbox_channel *mb_chann, struct xdna_msg_header *heade · 47 unmodified lines
248{249{
249 struct mailbox_msg *mb_msg;250 struct mailbox_msg *mb_msg;
250 int msg_id;251 int msg_id;
251 int ret;252 int ret = 0;
252 253
253 msg_id = header->id;254 msg_id = header->id;
254 if (!mailbox_validate_msgid(msg_id)) {255 if (!mailbox_validate_msgid(msg_id)) {
@@ -265,9 +266,11 @@ mailbox_get_resp(struct mailbox_channel *mb_chann, struct xdna_msg_header *heade · 10 unmodified lines
265 266
266 MB_DBG(mb_chann, "opcode 0x%x size %d id 0x%x",267 MB_DBG(mb_chann, "opcode 0x%x size %d id 0x%x",
267 header->opcode, header->total_size, header->id);268 header->opcode, header->total_size, header->id);
268 ret = mb_msg->notify_cb(mb_msg->handle, data, header->total_size);269 if (mb_msg->notify_cb) {
269 if (unlikely(ret))270 ret = mb_msg->notify_cb(mb_msg->handle, data, header->total_size);
270 MB_ERR(mb_chann, "Message callback ret %d", ret);271 if (unlikely(ret))
272 MB_ERR(mb_chann, "Message callback ret %d", ret);
273 }
271 274
272 kfree(mb_msg);275 kfree(mb_msg);
273 return ret;276 return ret;
@@ -513,6 +516,7 @@ xdna_mailbox_create_channel(struct mailbox *mb, · 239 unmodified lines
513 }516 }
514 517
515 mb_chann->bad_state = false;518 mb_chann->bad_state = false;
519 mailbox_reg_write(mb_chann, mb_chann->iohub_int_addr, 0);
516 520
517 MB_DBG(mb_chann, "Mailbox channel created (irq: %d)", mb_chann->msix_irq);521 MB_DBG(mb_chann, "Mailbox channel created (irq: %d)", mb_chann->msix_irq);
518 return mb_chann;522 return mb_chann;
drivers/accel/amdxdna/amdxdna_mailbox_helper.h+4−2
@@ -16,16 +16,18 @@ struct xdna_notify { · 15 unmodified lines
16 u32 *data;16 u32 *data;
17 size_t size;17 size_t size;
18 int error;18 int error;
19 u32 *status;
19};20};
20 21
21#define DECLARE_XDNA_MSG_COMMON(name, op, status) \22#define DECLARE_XDNA_MSG_COMMON(name, op, s) \
22 struct name##_req req = { 0 }; \23 struct name##_req req = { 0 }; \
23 struct name##_resp resp = { status }; \24 struct name##_resp resp = { .status = s }; \
24 struct xdna_notify hdl = { \25 struct xdna_notify hdl = { \
25 .error = 0, \26 .error = 0, \
26 .data = (u32 *)&resp, \27 .data = (u32 *)&resp, \
27 .size = sizeof(resp), \28 .size = sizeof(resp), \
28 .comp = COMPLETION_INITIALIZER_ONSTACK(hdl.comp), \29 .comp = COMPLETION_INITIALIZER_ONSTACK(hdl.comp), \
30 .status = (u32 *)&resp.status, \
29 }; \31 }; \
30 struct xdna_mailbox_msg msg = { \32 struct xdna_mailbox_msg msg = { \
31 .send_data = (u8 *)&req, \33 .send_data = (u8 *)&req, \
drivers/accel/amdxdna/amdxdna_pci_drv.c+11−52
@@ -13,13 +13,11 @@ · 12 unmodified lines
13#include <drm/gpu_scheduler.h>13#include <drm/gpu_scheduler.h>
14#include <linux/iommu.h>14#include <linux/iommu.h>
15#include <linux/pci.h>15#include <linux/pci.h>
16#include <linux/pm_runtime.h>
17 16
18#include "amdxdna_ctx.h"17#include "amdxdna_ctx.h"
19#include "amdxdna_gem.h"18#include "amdxdna_gem.h"
20#include "amdxdna_pci_drv.h"19#include "amdxdna_pci_drv.h"
21 20#include "amdxdna_pm.h"
22#define AMDXDNA_AUTOSUSPEND_DELAY 5000 /* milliseconds */
23 21
24MODULE_FIRMWARE("amdnpu/1502_00/npu.sbin");22MODULE_FIRMWARE("amdnpu/1502_00/npu.sbin");
25MODULE_FIRMWARE("amdnpu/17f0_10/npu.sbin");23MODULE_FIRMWARE("amdnpu/17f0_10/npu.sbin");
@@ -29,9 +27,14 @@ MODULE_FIRMWARE("amdnpu/17f0_20/npu.sbin"); · 3 unmodified lines
29/*27/*
30 * 0.0: Initial version28 * 0.0: Initial version
31 * 0.1: Support getting all hardware contexts by DRM_IOCTL_AMDXDNA_GET_ARRAY29 * 0.1: Support getting all hardware contexts by DRM_IOCTL_AMDXDNA_GET_ARRAY
30 * 0.2: Support getting last error hardware error
31 * 0.3: Support firmware debug buffer
32 * 0.4: Support getting resource information
33 * 0.5: Support getting telemetry data
34 * 0.6: Support preemption
32 */35 */
33#define AMDXDNA_DRIVER_MAJOR 036#define AMDXDNA_DRIVER_MAJOR 0
34#define AMDXDNA_DRIVER_MINOR 137#define AMDXDNA_DRIVER_MINOR 6
35 38
36/*39/*
37 * Bind the driver base on (vendor_id, device_id) pair and later use the40 * Bind the driver base on (vendor_id, device_id) pair and later use the
@@ -61,17 +64,9 @@ static int amdxdna_drm_open(struct drm_device *ddev, struct drm_file *filp) · 23 unmodified lines
61 struct amdxdna_client *client;64 struct amdxdna_client *client;
62 int ret;65 int ret;
63 66
64 ret = pm_runtime_resume_and_get(ddev->dev);
65 if (ret) {
66 XDNA_ERR(xdna, "Failed to get rpm, ret %d", ret);
67 return ret;
68 }
69
70 client = kzalloc(sizeof(*client), GFP_KERNEL);67 client = kzalloc(sizeof(*client), GFP_KERNEL);
71 if (!client) {68 if (!client)
72 ret = -ENOMEM;69 return -ENOMEM;
73 goto put_rpm;
74 }
75 70
76 client->pid = pid_nr(rcu_access_pointer(filp->pid));71 client->pid = pid_nr(rcu_access_pointer(filp->pid));
77 client->xdna = xdna;72 client->xdna = xdna;
@@ -106,9 +101,6 @@ unbind_sva: · 28 unmodified lines
106 iommu_sva_unbind_device(client->sva);101 iommu_sva_unbind_device(client->sva);
107failed:102failed:
108 kfree(client);103 kfree(client);
109put_rpm:
110 pm_runtime_mark_last_busy(ddev->dev);
111 pm_runtime_put_autosuspend(ddev->dev);
112 104
113 return ret;105 return ret;
114}106}
@@ -130,8 +122,6 @@ static void amdxdna_drm_close(struct drm_device *ddev, struct drm_file *filp) · 15 unmodified lines
130 122
131 XDNA_DBG(xdna, "pid %d closed", client->pid);123 XDNA_DBG(xdna, "pid %d closed", client->pid);
132 kfree(client);124 kfree(client);
133 pm_runtime_mark_last_busy(ddev->dev);
134 pm_runtime_put_autosuspend(ddev->dev);
135}125}
136 126
137static int amdxdna_flush(struct file *f, fl_owner_t id)127static int amdxdna_flush(struct file *f, fl_owner_t id)
@@ -310,19 +300,12 @@ static int amdxdna_probe(struct pci_dev *pdev, const struct pci_device_id *id) · 172 unmodified lines
310 goto failed_dev_fini;300 goto failed_dev_fini;
311 }301 }
312 302
313 pm_runtime_set_autosuspend_delay(dev, AMDXDNA_AUTOSUSPEND_DELAY);
314 pm_runtime_use_autosuspend(dev);
315 pm_runtime_allow(dev);
316
317 ret = drm_dev_register(&xdna->ddev, 0);303 ret = drm_dev_register(&xdna->ddev, 0);
318 if (ret) {304 if (ret) {
319 XDNA_ERR(xdna, "DRM register failed, ret %d", ret);305 XDNA_ERR(xdna, "DRM register failed, ret %d", ret);
320 pm_runtime_forbid(dev);
321 goto failed_sysfs_fini;306 goto failed_sysfs_fini;
322 }307 }
323 308
324 pm_runtime_mark_last_busy(dev);
325 pm_runtime_put_autosuspend(dev);
326 return 0;309 return 0;
327 310
328failed_sysfs_fini:311failed_sysfs_fini:
@@ -339,14 +322,10 @@ destroy_notifier_wq: · 10 unmodified lines
339static void amdxdna_remove(struct pci_dev *pdev)322static void amdxdna_remove(struct pci_dev *pdev)
340{323{
341 struct amdxdna_dev *xdna = pci_get_drvdata(pdev);324 struct amdxdna_dev *xdna = pci_get_drvdata(pdev);
342 struct device *dev = &pdev->dev;
343 struct amdxdna_client *client;325 struct amdxdna_client *client;
344 326
345 destroy_workqueue(xdna->notifier_wq);327 destroy_workqueue(xdna->notifier_wq);
346 328
347 pm_runtime_get_noresume(dev);
348 pm_runtime_forbid(dev);
349
350 drm_dev_unplug(&xdna->ddev);329 drm_dev_unplug(&xdna->ddev);
351 amdxdna_sysfs_fini(xdna);330 amdxdna_sysfs_fini(xdna);
352 331
@@ -365,29 +344,9 @@ static void amdxdna_remove(struct pci_dev *pdev) · 12 unmodified lines
365 mutex_unlock(&xdna->dev_lock);344 mutex_unlock(&xdna->dev_lock);
366}345}
367 346
368static int amdxdna_pmops_suspend(struct device *dev)
369{
370 struct amdxdna_dev *xdna = pci_get_drvdata(to_pci_dev(dev));
371
372 if (!xdna->dev_info->ops->suspend)
373 return -EOPNOTSUPP;
374
375 return xdna->dev_info->ops->suspend(xdna);
376}
377
378static int amdxdna_pmops_resume(struct device *dev)
379{
380 struct amdxdna_dev *xdna = pci_get_drvdata(to_pci_dev(dev));
381
382 if (!xdna->dev_info->ops->resume)
383 return -EOPNOTSUPP;
384
385 return xdna->dev_info->ops->resume(xdna);
386}
387
388static const struct dev_pm_ops amdxdna_pm_ops = {347static const struct dev_pm_ops amdxdna_pm_ops = {
389 SYSTEM_SLEEP_PM_OPS(amdxdna_pmops_suspend, amdxdna_pmops_resume)348 SYSTEM_SLEEP_PM_OPS(amdxdna_pm_suspend, amdxdna_pm_resume)
390 RUNTIME_PM_OPS(amdxdna_pmops_suspend, amdxdna_pmops_resume, NULL)349 RUNTIME_PM_OPS(amdxdna_pm_suspend, amdxdna_pm_resume, NULL)
391};350};
392 351
393static struct pci_driver amdxdna_pci_driver = {352static struct pci_driver amdxdna_pci_driver = {
drivers/accel/amdxdna/amdxdna_pci_drv.h+3−0
@@ -6,6 +6,7 @@ · 5 unmodified lines
6#ifndef _AMDXDNA_PCI_DRV_H_6#ifndef _AMDXDNA_PCI_DRV_H_
7#define _AMDXDNA_PCI_DRV_H_7#define _AMDXDNA_PCI_DRV_H_
8 8
9#include <drm/drm_print.h>
9#include <linux/workqueue.h>10#include <linux/workqueue.h>
10#include <linux/xarray.h>11#include <linux/xarray.h>
11 12
@@ -54,6 +55,7 @@ struct amdxdna_dev_ops { · 42 unmodified lines
54 int (*hwctx_init)(struct amdxdna_hwctx *hwctx);55 int (*hwctx_init)(struct amdxdna_hwctx *hwctx);
55 void (*hwctx_fini)(struct amdxdna_hwctx *hwctx);56 void (*hwctx_fini)(struct amdxdna_hwctx *hwctx);
56 int (*hwctx_config)(struct amdxdna_hwctx *hwctx, u32 type, u64 value, void *buf, u32 size);57 int (*hwctx_config)(struct amdxdna_hwctx *hwctx, u32 type, u64 value, void *buf, u32 size);
58 int (*hwctx_sync_debug_bo)(struct amdxdna_hwctx *hwctx, u32 debug_bo_hdl);
57 void (*hmm_invalidate)(struct amdxdna_gem_obj *abo, unsigned long cur_seq);59 void (*hmm_invalidate)(struct amdxdna_gem_obj *abo, unsigned long cur_seq);
58 int (*cmd_submit)(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job, u64 *seq);60 int (*cmd_submit)(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job, u64 *seq);
59 int (*get_aie_info)(struct amdxdna_client *client, struct amdxdna_drm_get_info *args);61 int (*get_aie_info)(struct amdxdna_client *client, struct amdxdna_drm_get_info *args);
@@ -99,6 +101,7 @@ struct amdxdna_dev { · 39 unmodified lines
99 struct amdxdna_fw_ver fw_ver;101 struct amdxdna_fw_ver fw_ver;
100 struct rw_semaphore notifier_lock; /* for mmu notifier*/102 struct rw_semaphore notifier_lock; /* for mmu notifier*/
101 struct workqueue_struct *notifier_wq;103 struct workqueue_struct *notifier_wq;
104 bool rpm_on;
102};105};
103 106
104/*107/*
drivers/accel/amdxdna/amdxdna_pm.c+94−0
@@ -0,0 +1,94 @@
1// SPDX-License-Identifier: GPL-2.0
2/*
3 * Copyright (C) 2025, Advanced Micro Devices, Inc.
4 */
5
6#include <drm/amdxdna_accel.h>
7#include <drm/drm_drv.h>
8#include <linux/pm_runtime.h>
9
10#include "amdxdna_pm.h"
11
12#define AMDXDNA_AUTOSUSPEND_DELAY 5000 /* milliseconds */
13
14int amdxdna_pm_suspend(struct device *dev)
15{
16 struct amdxdna_dev *xdna = to_xdna_dev(dev_get_drvdata(dev));
17 int ret = -EOPNOTSUPP;
18 bool rpm;
19
20 if (xdna->dev_info->ops->suspend) {
21 rpm = xdna->rpm_on;
22 xdna->rpm_on = false;
23 ret = xdna->dev_info->ops->suspend(xdna);
24 xdna->rpm_on = rpm;
25 }
26
27 XDNA_DBG(xdna, "Suspend done ret %d", ret);
28 return ret;
29}
30
31int amdxdna_pm_resume(struct device *dev)
32{
33 struct amdxdna_dev *xdna = to_xdna_dev(dev_get_drvdata(dev));
34 int ret = -EOPNOTSUPP;
35 bool rpm;
36
37 if (xdna->dev_info->ops->resume) {
38 rpm = xdna->rpm_on;
39 xdna->rpm_on = false;
40 ret = xdna->dev_info->ops->resume(xdna);
41 xdna->rpm_on = rpm;
42 }
43
44 XDNA_DBG(xdna, "Resume done ret %d", ret);
45 return ret;
46}
47
48int amdxdna_pm_resume_get(struct amdxdna_dev *xdna)
49{
50 struct device *dev = xdna->ddev.dev;
51 int ret;
52
53 if (!xdna->rpm_on)
54 return 0;
55
56 ret = pm_runtime_resume_and_get(dev);
57 if (ret) {
58 XDNA_ERR(xdna, "Resume failed: %d", ret);
59 pm_runtime_set_suspended(dev);
60 }
61
62 return ret;
63}
64
65void amdxdna_pm_suspend_put(struct amdxdna_dev *xdna)
66{
67 struct device *dev = xdna->ddev.dev;
68
69 if (!xdna->rpm_on)
70 return;
71
72 pm_runtime_put_autosuspend(dev);
73}
74
75void amdxdna_pm_init(struct amdxdna_dev *xdna)
76{
77 struct device *dev = xdna->ddev.dev;
78
79 pm_runtime_set_active(dev);
80 pm_runtime_set_autosuspend_delay(dev, AMDXDNA_AUTOSUSPEND_DELAY);
81 pm_runtime_use_autosuspend(dev);
82 pm_runtime_allow(dev);
83 pm_runtime_put_autosuspend(dev);
84 xdna->rpm_on = true;
85}
86
87void amdxdna_pm_fini(struct amdxdna_dev *xdna)
88{
89 struct device *dev = xdna->ddev.dev;
90
91 xdna->rpm_on = false;
92 pm_runtime_get_noresume(dev);
93 pm_runtime_forbid(dev);
94}
drivers/accel/amdxdna/amdxdna_pm.h+18−0
@@ -0,0 +1,18 @@
1/* SPDX-License-Identifier: GPL-2.0 */
2/*
3 * Copyright (C) 2025, Advanced Micro Devices, Inc.
4 */
5
6#ifndef _AMDXDNA_PM_H_
7#define _AMDXDNA_PM_H_
8
9#include "amdxdna_pci_drv.h"
10
11int amdxdna_pm_suspend(struct device *dev);
12int amdxdna_pm_resume(struct device *dev);
13int amdxdna_pm_resume_get(struct amdxdna_dev *xdna);
14void amdxdna_pm_suspend_put(struct amdxdna_dev *xdna);
15void amdxdna_pm_init(struct amdxdna_dev *xdna);
16void amdxdna_pm_fini(struct amdxdna_dev *xdna);
17
18#endif /* _AMDXDNA_PM_H_ */
drivers/accel/amdxdna/npu1_regs.c+8−0
@@ -46,6 +46,7 @@ · 45 unmodified lines
46 46
47const struct rt_config npu1_default_rt_cfg[] = {47const struct rt_config npu1_default_rt_cfg[] = {
48 { 2, 1, AIE2_RT_CFG_INIT }, /* PDI APP LOAD MODE */48 { 2, 1, AIE2_RT_CFG_INIT }, /* PDI APP LOAD MODE */
49 { 4, 1, AIE2_RT_CFG_INIT }, /* Debug BO */
49 { 1, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */50 { 1, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */
50 { 0 },51 { 0 },
51};52};
@@ -62,16 +63,23 @@ const struct dpm_clk_freq npu1_dpm_clk_table[] = { · 10 unmodified lines
62 { 0 }63 { 0 }
63};64};
64 65
66static const struct aie2_fw_feature_tbl npu1_fw_feature_table[] = {
67 { .feature = AIE2_NPU_COMMAND, .min_minor = 8 },
68 { 0 }
69};
70
65static const struct amdxdna_dev_priv npu1_dev_priv = {71static const struct amdxdna_dev_priv npu1_dev_priv = {
66 .fw_path = "amdnpu/1502_00/npu.sbin",72 .fw_path = "amdnpu/1502_00/npu.sbin",
67 .protocol_major = 0x5,73 .protocol_major = 0x5,
68 .protocol_minor = 0x7,74 .protocol_minor = 0x7,
69 .rt_config = npu1_default_rt_cfg,75 .rt_config = npu1_default_rt_cfg,
70 .dpm_clk_tbl = npu1_dpm_clk_table,76 .dpm_clk_tbl = npu1_dpm_clk_table,
77 .fw_feature_tbl = npu1_fw_feature_table,
71 .col_align = COL_ALIGN_NONE,78 .col_align = COL_ALIGN_NONE,
72 .mbox_dev_addr = NPU1_MBOX_BAR_BASE,79 .mbox_dev_addr = NPU1_MBOX_BAR_BASE,
73 .mbox_size = 0, /* Use BAR size */80 .mbox_size = 0, /* Use BAR size */
74 .sram_dev_addr = NPU1_SRAM_BAR_BASE,81 .sram_dev_addr = NPU1_SRAM_BAR_BASE,
82 .hwctx_limit = 6,
75 .sram_offs = {83 .sram_offs = {
76 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU1_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),84 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU1_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
77 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU1_SRAM, MPNPU_SRAM_I2X_MAILBOX_15),85 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU1_SRAM, MPNPU_SRAM_I2X_MAILBOX_15),
drivers/accel/amdxdna/npu2_regs.c+2−0
@@ -67,10 +67,12 @@ static const struct amdxdna_dev_priv npu2_dev_priv = { · 66 unmodified lines
67 .protocol_minor = 0x6,67 .protocol_minor = 0x6,
68 .rt_config = npu4_default_rt_cfg,68 .rt_config = npu4_default_rt_cfg,
69 .dpm_clk_tbl = npu4_dpm_clk_table,69 .dpm_clk_tbl = npu4_dpm_clk_table,
70 .fw_feature_tbl = npu4_fw_feature_table,
70 .col_align = COL_ALIGN_NATURE,71 .col_align = COL_ALIGN_NATURE,
71 .mbox_dev_addr = NPU2_MBOX_BAR_BASE,72 .mbox_dev_addr = NPU2_MBOX_BAR_BASE,
72 .mbox_size = 0, /* Use BAR size */73 .mbox_size = 0, /* Use BAR size */
73 .sram_dev_addr = NPU2_SRAM_BAR_BASE,74 .sram_dev_addr = NPU2_SRAM_BAR_BASE,
75 .hwctx_limit = 16,
74 .sram_offs = {76 .sram_offs = {
75 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU2_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),77 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU2_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
76 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU2_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),78 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU2_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),
drivers/accel/amdxdna/npu4_regs.c+12−0
@@ -63,10 +63,14 @@ · 62 unmodified lines
63 63
64const struct rt_config npu4_default_rt_cfg[] = {64const struct rt_config npu4_default_rt_cfg[] = {
65 { 5, 1, AIE2_RT_CFG_INIT }, /* PDI APP LOAD MODE */65 { 5, 1, AIE2_RT_CFG_INIT }, /* PDI APP LOAD MODE */
66 { 10, 1, AIE2_RT_CFG_INIT }, /* DEBUG BUF */
67 { 14, 0, AIE2_RT_CFG_INIT, BIT_U64(AIE2_PREEMPT) }, /* Frame boundary preemption */
66 { 1, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */68 { 1, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */
67 { 2, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */69 { 2, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */
68 { 3, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */70 { 3, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */
69 { 4, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */71 { 4, 1, AIE2_RT_CFG_CLK_GATING }, /* Clock gating on */
72 { 13, 0, AIE2_RT_CFG_FORCE_PREEMPT },
73 { 14, 0, AIE2_RT_CFG_FRAME_BOUNDARY_PREEMPT },
70 { 0 },74 { 0 },
71};75};
72 76
@@ -82,16 +86,24 @@ const struct dpm_clk_freq npu4_dpm_clk_table[] = { · 9 unmodified lines
82 { 0 }86 { 0 }
83};87};
84 88
89const struct aie2_fw_feature_tbl npu4_fw_feature_table[] = {
90 { .feature = AIE2_NPU_COMMAND, .min_minor = 15 },
91 { .feature = AIE2_PREEMPT, .min_minor = 12 },
92 { 0 }
93};
94
85static const struct amdxdna_dev_priv npu4_dev_priv = {95static const struct amdxdna_dev_priv npu4_dev_priv = {
86 .fw_path = "amdnpu/17f0_10/npu.sbin",96 .fw_path = "amdnpu/17f0_10/npu.sbin",
87 .protocol_major = 0x6,97 .protocol_major = 0x6,
88 .protocol_minor = 12,98 .protocol_minor = 12,
89 .rt_config = npu4_default_rt_cfg,99 .rt_config = npu4_default_rt_cfg,
90 .dpm_clk_tbl = npu4_dpm_clk_table,100 .dpm_clk_tbl = npu4_dpm_clk_table,
101 .fw_feature_tbl = npu4_fw_feature_table,
91 .col_align = COL_ALIGN_NATURE,102 .col_align = COL_ALIGN_NATURE,
92 .mbox_dev_addr = NPU4_MBOX_BAR_BASE,103 .mbox_dev_addr = NPU4_MBOX_BAR_BASE,
93 .mbox_size = 0, /* Use BAR size */104 .mbox_size = 0, /* Use BAR size */
94 .sram_dev_addr = NPU4_SRAM_BAR_BASE,105 .sram_dev_addr = NPU4_SRAM_BAR_BASE,
106 .hwctx_limit = 16,
95 .sram_offs = {107 .sram_offs = {
96 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU4_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),108 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU4_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
97 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU4_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),109 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU4_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),
drivers/accel/amdxdna/npu5_regs.c+2−0
@@ -67,10 +67,12 @@ static const struct amdxdna_dev_priv npu5_dev_priv = { · 66 unmodified lines
67 .protocol_minor = 12,67 .protocol_minor = 12,
68 .rt_config = npu4_default_rt_cfg,68 .rt_config = npu4_default_rt_cfg,
69 .dpm_clk_tbl = npu4_dpm_clk_table,69 .dpm_clk_tbl = npu4_dpm_clk_table,
70 .fw_feature_tbl = npu4_fw_feature_table,
70 .col_align = COL_ALIGN_NATURE,71 .col_align = COL_ALIGN_NATURE,
71 .mbox_dev_addr = NPU5_MBOX_BAR_BASE,72 .mbox_dev_addr = NPU5_MBOX_BAR_BASE,
72 .mbox_size = 0, /* Use BAR size */73 .mbox_size = 0, /* Use BAR size */
73 .sram_dev_addr = NPU5_SRAM_BAR_BASE,74 .sram_dev_addr = NPU5_SRAM_BAR_BASE,
75 .hwctx_limit = 16,
74 .sram_offs = {76 .sram_offs = {
75 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU5_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),77 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU5_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
76 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU5_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),78 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU5_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),
drivers/accel/amdxdna/npu6_regs.c+2−0
@@ -67,10 +67,12 @@ static const struct amdxdna_dev_priv npu6_dev_priv = { · 66 unmodified lines
67 .protocol_minor = 12,67 .protocol_minor = 12,
68 .rt_config = npu4_default_rt_cfg,68 .rt_config = npu4_default_rt_cfg,
69 .dpm_clk_tbl = npu4_dpm_clk_table,69 .dpm_clk_tbl = npu4_dpm_clk_table,
70 .fw_feature_tbl = npu4_fw_feature_table,
70 .col_align = COL_ALIGN_NATURE,71 .col_align = COL_ALIGN_NATURE,
71 .mbox_dev_addr = NPU6_MBOX_BAR_BASE,72 .mbox_dev_addr = NPU6_MBOX_BAR_BASE,
72 .mbox_size = 0, /* Use BAR size */73 .mbox_size = 0, /* Use BAR size */
73 .sram_dev_addr = NPU6_SRAM_BAR_BASE,74 .sram_dev_addr = NPU6_SRAM_BAR_BASE,
75 .hwctx_limit = 16,
74 .sram_offs = {76 .sram_offs = {
75 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU6_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),77 DEFINE_BAR_OFFSET(MBOX_CHANN_OFF, NPU6_SRAM, MPNPU_SRAM_X2I_MAILBOX_0),
76 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU6_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),78 DEFINE_BAR_OFFSET(FW_ALIVE_OFF, NPU6_SRAM, MPNPU_SRAM_X2I_MAILBOX_15),
drivers/accel/ethosu/Kconfig+11−0
@@ -0,0 +1,11 @@
1# SPDX-License-Identifier: GPL-2.0-only
2
3config DRM_ACCEL_ARM_ETHOSU
4 tristate "Arm Ethos-U65/U85 NPU"
5 depends on HAS_IOMEM
6 depends on DRM_ACCEL
7 select DRM_GEM_DMA_HELPER
8 select DRM_SCHED
9 select GENERIC_ALLOCATOR
10 help
11 Enables driver for Arm Ethos-U65/U85 NPUs
drivers/accel/ethosu/Makefile+4−0
@@ -0,0 +1,4 @@
1# SPDX-License-Identifier: GPL-2.0-only
2
3obj-$(CONFIG_DRM_ACCEL_ARM_ETHOSU) := ethosu.o
4ethosu-y += ethosu_drv.o ethosu_gem.o ethosu_job.o
drivers/accel/ethosu/ethosu_device.h+197−0
@@ -0,0 +1,197 @@
1/* SPDX-License-Identifier: GPL-2.0-only or MIT */
2/* Copyright 2025 Arm, Ltd. */
3
4#ifndef __ETHOSU_DEVICE_H__
5#define __ETHOSU_DEVICE_H__
6
7#include <linux/bitfield.h>
8#include <linux/bits.h>
9#include <linux/types.h>
10
11#include <drm/drm_device.h>
12#include <drm/gpu_scheduler.h>
13
14#include <drm/ethosu_accel.h>
15
16struct clk;
17struct gen_pool;
18
19#define NPU_REG_ID 0x0000
20#define NPU_REG_STATUS 0x0004
21#define NPU_REG_CMD 0x0008
22#define NPU_REG_RESET 0x000c
23#define NPU_REG_QBASE 0x0010
24#define NPU_REG_QBASE_HI 0x0014
25#define NPU_REG_QREAD 0x0018
26#define NPU_REG_QCONFIG 0x001c
27#define NPU_REG_QSIZE 0x0020
28#define NPU_REG_PROT 0x0024
29#define NPU_REG_CONFIG 0x0028
30#define NPU_REG_REGIONCFG 0x003c
31#define NPU_REG_AXILIMIT0 0x0040 // U65
32#define NPU_REG_AXILIMIT1 0x0044 // U65
33#define NPU_REG_AXILIMIT2 0x0048 // U65
34#define NPU_REG_AXILIMIT3 0x004c // U65
35#define NPU_REG_MEM_ATTR0 0x0040 // U85
36#define NPU_REG_MEM_ATTR1 0x0044 // U85
37#define NPU_REG_MEM_ATTR2 0x0048 // U85
38#define NPU_REG_MEM_ATTR3 0x004c // U85
39#define NPU_REG_AXI_SRAM 0x0050 // U85
40#define NPU_REG_AXI_EXT 0x0054 // U85
41
42#define NPU_REG_BASEP(x) (0x0080 + (x) * 8)
43#define NPU_REG_BASEP_HI(x) (0x0084 + (x) * 8)
44#define NPU_BASEP_REGION_MAX 8
45
46#define ID_ARCH_MAJOR_MASK GENMASK(31, 28)
47#define ID_ARCH_MINOR_MASK GENMASK(27, 20)
48#define ID_ARCH_PATCH_MASK GENMASK(19, 16)
49#define ID_VER_MAJOR_MASK GENMASK(11, 8)
50#define ID_VER_MINOR_MASK GENMASK(7, 4)
51
52#define CONFIG_MACS_PER_CC_MASK GENMASK(3, 0)
53#define CONFIG_CMD_STREAM_VER_MASK GENMASK(7, 4)
54
55#define STATUS_STATE_RUNNING BIT(0)
56#define STATUS_IRQ_RAISED BIT(1)
57#define STATUS_BUS_STATUS BIT(2)
58#define STATUS_RESET_STATUS BIT(3)
59#define STATUS_CMD_PARSE_ERR BIT(4)
60#define STATUS_CMD_END_REACHED BIT(5)
61
62#define CMD_CLEAR_IRQ BIT(1)
63#define CMD_TRANSITION_TO_RUN BIT(0)
64
65#define RESET_PENDING_CSL BIT(1)
66#define RESET_PENDING_CPL BIT(0)
67
68#define PROT_ACTIVE_CSL BIT(1)
69
70enum ethosu_cmds {
71 NPU_OP_CONV = 0x2,
72 NPU_OP_DEPTHWISE = 0x3,
73 NPU_OP_POOL = 0x5,
74 NPU_OP_ELEMENTWISE = 0x6,
75 NPU_OP_RESIZE = 0x7, // U85 only
76 NPU_OP_DMA_START = 0x10,
77 NPU_SET_IFM_PAD_TOP = 0x100,
78 NPU_SET_IFM_PAD_LEFT = 0x101,
79 NPU_SET_IFM_PAD_RIGHT = 0x102,
80 NPU_SET_IFM_PAD_BOTTOM = 0x103,
81 NPU_SET_IFM_DEPTH_M1 = 0x104,
82 NPU_SET_IFM_PRECISION = 0x105,
83 NPU_SET_IFM_BROADCAST = 0x108,
84 NPU_SET_IFM_WIDTH0_M1 = 0x10a,
85 NPU_SET_IFM_HEIGHT0_M1 = 0x10b,
86 NPU_SET_IFM_HEIGHT1_M1 = 0x10c,
87 NPU_SET_IFM_REGION = 0x10f,
88 NPU_SET_OFM_WIDTH_M1 = 0x111,
89 NPU_SET_OFM_HEIGHT_M1 = 0x112,
90 NPU_SET_OFM_DEPTH_M1 = 0x113,
91 NPU_SET_OFM_PRECISION = 0x114,
92 NPU_SET_OFM_WIDTH0_M1 = 0x11a,
93 NPU_SET_OFM_HEIGHT0_M1 = 0x11b,
94 NPU_SET_OFM_HEIGHT1_M1 = 0x11c,
95 NPU_SET_OFM_REGION = 0x11f,
96 NPU_SET_KERNEL_WIDTH_M1 = 0x120,
97 NPU_SET_KERNEL_HEIGHT_M1 = 0x121,
98 NPU_SET_KERNEL_STRIDE = 0x122,
99 NPU_SET_WEIGHT_REGION = 0x128,
100 NPU_SET_SCALE_REGION = 0x129,
101 NPU_SET_DMA0_SRC_REGION = 0x130,
102 NPU_SET_DMA0_DST_REGION = 0x131,
103 NPU_SET_DMA0_SIZE0 = 0x132,
104 NPU_SET_DMA0_SIZE1 = 0x133,
105 NPU_SET_IFM2_BROADCAST = 0x180,
106 NPU_SET_IFM2_PRECISION = 0x185,
107 NPU_SET_IFM2_WIDTH0_M1 = 0x18a,
108 NPU_SET_IFM2_HEIGHT0_M1 = 0x18b,
109 NPU_SET_IFM2_HEIGHT1_M1 = 0x18c,
110 NPU_SET_IFM2_REGION = 0x18f,
111 NPU_SET_IFM_BASE0 = 0x4000,
112 NPU_SET_IFM_BASE1 = 0x4001,
113 NPU_SET_IFM_BASE2 = 0x4002,
114 NPU_SET_IFM_BASE3 = 0x4003,
115 NPU_SET_IFM_STRIDE_X = 0x4004,
116 NPU_SET_IFM_STRIDE_Y = 0x4005,
117 NPU_SET_IFM_STRIDE_C = 0x4006,
118 NPU_SET_OFM_BASE0 = 0x4010,
119 NPU_SET_OFM_BASE1 = 0x4011,
120 NPU_SET_OFM_BASE2 = 0x4012,
121 NPU_SET_OFM_BASE3 = 0x4013,
122 NPU_SET_OFM_STRIDE_X = 0x4014,
123 NPU_SET_OFM_STRIDE_Y = 0x4015,
124 NPU_SET_OFM_STRIDE_C = 0x4016,
125 NPU_SET_WEIGHT_BASE = 0x4020,
126 NPU_SET_WEIGHT_LENGTH = 0x4021,
127 NPU_SET_SCALE_BASE = 0x4022,
128 NPU_SET_SCALE_LENGTH = 0x4023,
129 NPU_SET_DMA0_SRC = 0x4030,
130 NPU_SET_DMA0_DST = 0x4031,
131 NPU_SET_DMA0_LEN = 0x4032,
132 NPU_SET_DMA0_SRC_STRIDE0 = 0x4033,
133 NPU_SET_DMA0_SRC_STRIDE1 = 0x4034,
134 NPU_SET_DMA0_DST_STRIDE0 = 0x4035,
135 NPU_SET_DMA0_DST_STRIDE1 = 0x4036,
136 NPU_SET_IFM2_BASE0 = 0x4080,
137 NPU_SET_IFM2_BASE1 = 0x4081,
138 NPU_SET_IFM2_BASE2 = 0x4082,
139 NPU_SET_IFM2_BASE3 = 0x4083,
140 NPU_SET_IFM2_STRIDE_X = 0x4084,
141 NPU_SET_IFM2_STRIDE_Y = 0x4085,
142 NPU_SET_IFM2_STRIDE_C = 0x4086,
143 NPU_SET_WEIGHT1_BASE = 0x4090,
144 NPU_SET_WEIGHT1_LENGTH = 0x4091,
145 NPU_SET_SCALE1_BASE = 0x4092,
146 NPU_SET_WEIGHT2_BASE = 0x4092,
147 NPU_SET_SCALE1_LENGTH = 0x4093,
148 NPU_SET_WEIGHT2_LENGTH = 0x4093,
149 NPU_SET_WEIGHT3_BASE = 0x4094,
150 NPU_SET_WEIGHT3_LENGTH = 0x4095,
151};
152
153#define ETHOSU_SRAM_REGION 2 /* Matching Vela compiler */
154
155/**
156 * struct ethosu_device - Ethosu device
157 */
158struct ethosu_device {
159 /** @base: Base drm_device. */
160 struct drm_device base;
161
162 /** @iomem: CPU mapping of the registers. */
163 void __iomem *regs;
164
165 void __iomem *sram;
166 struct gen_pool *srampool;
167 dma_addr_t sramphys;
168
169 struct clk_bulk_data *clks;
170 int num_clks;
171 int irq;
172
173 struct drm_ethosu_npu_info npu_info;
174
175 struct ethosu_job *in_flight_job;
176 /* For in_flight_job and ethosu_job_hw_submit() */
177 struct mutex job_lock;
178
179 /* For dma_fence */
180 spinlock_t fence_lock;
181
182 struct drm_gpu_scheduler sched;
183 /* For ethosu_job_do_push() */
184 struct mutex sched_lock;
185 u64 fence_context;
186 u64 emit_seqno;
187};
188
189#define to_ethosu_device(drm_dev) \
190 ((struct ethosu_device *)container_of(drm_dev, struct ethosu_device, base))
191
192static inline bool ethosu_is_u65(const struct ethosu_device *ethosudev)
193{
194 return FIELD_GET(ID_ARCH_MAJOR_MASK, ethosudev->npu_info.id) == 1;
195}
196
197#endif
drivers/accel/ethosu/ethosu_drv.c+403−0
@@ -0,0 +1,403 @@
1// SPDX-License-Identifier: GPL-2.0-only or MIT
2// Copyright (C) 2025 Arm, Ltd.
3
4#include <linux/bitfield.h>
5#include <linux/clk.h>
6#include <linux/genalloc.h>
7#include <linux/io.h>
8#include <linux/iopoll.h>
9#include <linux/module.h>
10#include <linux/mod_devicetable.h>
11#include <linux/platform_device.h>
12#include <linux/pm_runtime.h>
13
14#include <drm/drm_drv.h>
15#include <drm/drm_ioctl.h>
16#include <drm/drm_utils.h>
17#include <drm/drm_gem.h>
18#include <drm/drm_accel.h>
19#include <drm/ethosu_accel.h>
20
21#include "ethosu_drv.h"
22#include "ethosu_device.h"
23#include "ethosu_gem.h"
24#include "ethosu_job.h"
25
26static int ethosu_ioctl_dev_query(struct drm_device *ddev, void *data,
27 struct drm_file *file)
28{
29 struct ethosu_device *ethosudev = to_ethosu_device(ddev);
30 struct drm_ethosu_dev_query *args = data;
31
32 if (!args->pointer) {
33 switch (args->type) {
34 case DRM_ETHOSU_DEV_QUERY_NPU_INFO:
35 args->size = sizeof(ethosudev->npu_info);
36 return 0;
37 default:
38 return -EINVAL;
39 }
40 }
41
42 switch (args->type) {
43 case DRM_ETHOSU_DEV_QUERY_NPU_INFO:
44 if (args->size < offsetofend(struct drm_ethosu_npu_info, sram_size))
45 return -EINVAL;
46 return copy_struct_to_user(u64_to_user_ptr(args->pointer),
47 args->size,
48 &ethosudev->npu_info,
49 sizeof(ethosudev->npu_info), NULL);
50 default:
51 return -EINVAL;
52 }
53}
54
55#define ETHOSU_BO_FLAGS DRM_ETHOSU_BO_NO_MMAP
56
57static int ethosu_ioctl_bo_create(struct drm_device *ddev, void *data,
58 struct drm_file *file)
59{
60 struct drm_ethosu_bo_create *args = data;
61 int cookie, ret;
62
63 if (!drm_dev_enter(ddev, &cookie))
64 return -ENODEV;
65
66 if (!args->size || (args->flags & ~ETHOSU_BO_FLAGS)) {
67 ret = -EINVAL;
68 goto out_dev_exit;
69 }
70
71 ret = ethosu_gem_create_with_handle(file, ddev, &args->size,
72 args->flags, &args->handle);
73
74out_dev_exit:
75 drm_dev_exit(cookie);
76 return ret;
77}
78
79static int ethosu_ioctl_bo_wait(struct drm_device *ddev, void *data,
80 struct drm_file *file)
81{
82 struct drm_ethosu_bo_wait *args = data;
83 int cookie, ret;
84 unsigned long timeout = drm_timeout_abs_to_jiffies(args->timeout_ns);
85
86 if (args->pad)
87 return -EINVAL;
88
89 if (!drm_dev_enter(ddev, &cookie))
90 return -ENODEV;
91
92 ret = drm_gem_dma_resv_wait(file, args->handle, true, timeout);
93
94 drm_dev_exit(cookie);
95 return ret;
96}
97
98static int ethosu_ioctl_bo_mmap_offset(struct drm_device *ddev, void *data,
99 struct drm_file *file)
100{
101 struct drm_ethosu_bo_mmap_offset *args = data;
102 struct drm_gem_object *obj;
103
104 if (args->pad)
105 return -EINVAL;
106
107 obj = drm_gem_object_lookup(file, args->handle);
108 if (!obj)
109 return -ENOENT;
110
111 args->offset = drm_vma_node_offset_addr(&obj->vma_node);
112 drm_gem_object_put(obj);
113 return 0;
114}
115
116static int ethosu_ioctl_cmdstream_bo_create(struct drm_device *ddev, void *data,
117 struct drm_file *file)
118{
119 struct drm_ethosu_cmdstream_bo_create *args = data;
120 int cookie, ret;
121
122 if (!drm_dev_enter(ddev, &cookie))
123 return -ENODEV;
124
125 if (!args->size || !args->data || args->pad || args->flags) {
126 ret = -EINVAL;
127 goto out_dev_exit;
128 }
129
130 args->flags |= DRM_ETHOSU_BO_NO_MMAP;
131
132 ret = ethosu_gem_cmdstream_create(file, ddev, args->size, args->data,
133 args->flags, &args->handle);
134
135out_dev_exit:
136 drm_dev_exit(cookie);
137 return ret;
138}
139
140static int ethosu_open(struct drm_device *ddev, struct drm_file *file)
141{
142 int ret = 0;
143
144 if (!try_module_get(THIS_MODULE))
145 return -EINVAL;
146
147 struct ethosu_file_priv __free(kfree) *priv = kzalloc(sizeof(*priv), GFP_KERNEL);
148 if (!priv) {
149 ret = -ENOMEM;
150 goto err_put_mod;
151 }
152 priv->edev = to_ethosu_device(ddev);
153
154 ret = ethosu_job_open(priv);
155 if (ret)
156 goto err_put_mod;
157
158 file->driver_priv = no_free_ptr(priv);
159 return 0;
160
161err_put_mod:
162 module_put(THIS_MODULE);
163 return ret;
164}
165
166static void ethosu_postclose(struct drm_device *ddev, struct drm_file *file)
167{
168 ethosu_job_close(file->driver_priv);
169 kfree(file->driver_priv);
170 module_put(THIS_MODULE);
171}
172
173static const struct drm_ioctl_desc ethosu_drm_driver_ioctls[] = {
174#define ETHOSU_IOCTL(n, func, flags) \
175 DRM_IOCTL_DEF_DRV(ETHOSU_##n, ethosu_ioctl_##func, flags)
176
177 ETHOSU_IOCTL(DEV_QUERY, dev_query, 0),
178 ETHOSU_IOCTL(BO_CREATE, bo_create, 0),
179 ETHOSU_IOCTL(BO_WAIT, bo_wait, 0),
180 ETHOSU_IOCTL(BO_MMAP_OFFSET, bo_mmap_offset, 0),
181 ETHOSU_IOCTL(CMDSTREAM_BO_CREATE, cmdstream_bo_create, 0),
182 ETHOSU_IOCTL(SUBMIT, submit, 0),
183};
184
185DEFINE_DRM_ACCEL_FOPS(ethosu_drm_driver_fops);
186
187/*
188 * Ethosu driver version:
189 * - 1.0 - initial interface
190 */
191static const struct drm_driver ethosu_drm_driver = {
192 .driver_features = DRIVER_COMPUTE_ACCEL | DRIVER_GEM,
193 .open = ethosu_open,
194 .postclose = ethosu_postclose,
195 .ioctls = ethosu_drm_driver_ioctls,
196 .num_ioctls = ARRAY_SIZE(ethosu_drm_driver_ioctls),
197 .fops = &ethosu_drm_driver_fops,
198 .name = "ethosu",
199 .desc = "Arm Ethos-U Accel driver",
200 .major = 1,
201 .minor = 0,
202
203 .gem_create_object = ethosu_gem_create_object,
204};
205
206#define U65_DRAM_AXI_LIMIT_CFG 0x1f3f0002
207#define U65_SRAM_AXI_LIMIT_CFG 0x1f3f00b0
208#define U85_AXI_EXT_CFG 0x00021f3f
209#define U85_AXI_SRAM_CFG 0x00021f3f
210#define U85_MEM_ATTR0_CFG 0x00000000
211#define U85_MEM_ATTR2_CFG 0x000000b7
212
213static int ethosu_reset(struct ethosu_device *ethosudev)
214{
215 int ret;
216 u32 reg;
217
218 writel_relaxed(RESET_PENDING_CSL, ethosudev->regs + NPU_REG_RESET);
219 ret = readl_poll_timeout(ethosudev->regs + NPU_REG_STATUS, reg,
220 !FIELD_GET(STATUS_RESET_STATUS, reg),
221 USEC_PER_MSEC, USEC_PER_SEC);
222 if (ret)
223 return ret;
224
225 if (!FIELD_GET(PROT_ACTIVE_CSL, readl_relaxed(ethosudev->regs + NPU_REG_PROT))) {
226 dev_warn(ethosudev->base.dev, "Could not reset to non-secure mode (PROT = %x)\n",
227 readl_relaxed(ethosudev->regs + NPU_REG_PROT));
228 }
229
230 /*
231 * Assign region 2 (SRAM) to AXI M0 (AXILIMIT0),
232 * everything else to AXI M1 (AXILIMIT2)
233 */
234 writel_relaxed(0x0000aa8a, ethosudev->regs + NPU_REG_REGIONCFG);
235 if (ethosu_is_u65(ethosudev)) {
236 writel_relaxed(U65_SRAM_AXI_LIMIT_CFG, ethosudev->regs + NPU_REG_AXILIMIT0);
237 writel_relaxed(U65_DRAM_AXI_LIMIT_CFG, ethosudev->regs + NPU_REG_AXILIMIT2);
238 } else {
239 writel_relaxed(U85_AXI_SRAM_CFG, ethosudev->regs + NPU_REG_AXI_SRAM);
240 writel_relaxed(U85_AXI_EXT_CFG, ethosudev->regs + NPU_REG_AXI_EXT);
241 writel_relaxed(U85_MEM_ATTR0_CFG, ethosudev->regs + NPU_REG_MEM_ATTR0); // SRAM
242 writel_relaxed(U85_MEM_ATTR2_CFG, ethosudev->regs + NPU_REG_MEM_ATTR2); // DRAM
243 }
244
245 if (ethosudev->sram)
246 memset_io(ethosudev->sram, 0, ethosudev->npu_info.sram_size);
247
248 return 0;
249}
250
251static int ethosu_device_resume(struct device *dev)
252{
253 struct ethosu_device *ethosudev = dev_get_drvdata(dev);
254 int ret;
255
256 ret = clk_bulk_prepare_enable(ethosudev->num_clks, ethosudev->clks);
257 if (ret)
258 return ret;
259
260 ret = ethosu_reset(ethosudev);
261 if (!ret)
262 return 0;
263
264 clk_bulk_disable_unprepare(ethosudev->num_clks, ethosudev->clks);
265 return ret;
266}
267
268static int ethosu_device_suspend(struct device *dev)
269{
270 struct ethosu_device *ethosudev = dev_get_drvdata(dev);
271
272 clk_bulk_disable_unprepare(ethosudev->num_clks, ethosudev->clks);
273 return 0;
274}
275
276static int ethosu_sram_init(struct ethosu_device *ethosudev)
277{
278 ethosudev->npu_info.sram_size = 0;
279
280 ethosudev->srampool = of_gen_pool_get(ethosudev->base.dev->of_node, "sram", 0);
281 if (!ethosudev->srampool)
282 return 0;
283
284 ethosudev->npu_info.sram_size = gen_pool_size(ethosudev->srampool);
285
286 ethosudev->sram = (void __iomem *)gen_pool_dma_alloc(ethosudev->srampool,
287 ethosudev->npu_info.sram_size,
288 &ethosudev->sramphys);
289 if (!ethosudev->sram) {
290 dev_err(ethosudev->base.dev, "failed to allocate from SRAM pool\n");
291 return -ENOMEM;
292 }
293
294 return 0;
295}
296
297static int ethosu_init(struct ethosu_device *ethosudev)
298{
299 int ret;
300 u32 id, config;
301
302 ret = ethosu_device_resume(ethosudev->base.dev);
303 if (ret)
304 return ret;
305
306 pm_runtime_set_autosuspend_delay(ethosudev->base.dev, 50);
307 pm_runtime_use_autosuspend(ethosudev->base.dev);
308 ret = devm_pm_runtime_set_active_enabled(ethosudev->base.dev);
309 if (ret)
310 return ret;
311 pm_runtime_get_noresume(ethosudev->base.dev);
312
313 ethosudev->npu_info.id = id = readl_relaxed(ethosudev->regs + NPU_REG_ID);
314 ethosudev->npu_info.config = config = readl_relaxed(ethosudev->regs + NPU_REG_CONFIG);
315
316 ethosu_sram_init(ethosudev);
317
318 dev_info(ethosudev->base.dev,
319 "Ethos-U NPU, arch v%ld.%ld.%ld, rev r%ldp%ld, cmd stream ver%ld, %d MACs, %dKB SRAM\n",
320 FIELD_GET(ID_ARCH_MAJOR_MASK, id),
321 FIELD_GET(ID_ARCH_MINOR_MASK, id),
322 FIELD_GET(ID_ARCH_PATCH_MASK, id),
323 FIELD_GET(ID_VER_MAJOR_MASK, id),
324 FIELD_GET(ID_VER_MINOR_MASK, id),
325 FIELD_GET(CONFIG_CMD_STREAM_VER_MASK, config),
326 1 << FIELD_GET(CONFIG_MACS_PER_CC_MASK, config),
327 ethosudev->npu_info.sram_size / 1024);
328
329 return 0;
330}
331
332static int ethosu_probe(struct platform_device *pdev)
333{
334 int ret;
335 struct ethosu_device *ethosudev;
336
337 ethosudev = devm_drm_dev_alloc(&pdev->dev, &ethosu_drm_driver,
338 struct ethosu_device, base);
339 if (IS_ERR(ethosudev))
340 return -ENOMEM;
341 platform_set_drvdata(pdev, ethosudev);
342
343 dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(40));
344
345 ethosudev->regs = devm_platform_ioremap_resource(pdev, 0);
346
347 ethosudev->num_clks = devm_clk_bulk_get_all(&pdev->dev, &ethosudev->clks);
348 if (ethosudev->num_clks < 0)
349 return ethosudev->num_clks;
350
351 ret = ethosu_job_init(ethosudev);
352 if (ret)
353 return ret;
354
355 ret = ethosu_init(ethosudev);
356 if (ret)
357 return ret;
358
359 ret = drm_dev_register(&ethosudev->base, 0);
360 if (ret)
361 pm_runtime_dont_use_autosuspend(ethosudev->base.dev);
362
363 pm_runtime_put_autosuspend(ethosudev->base.dev);
364 return ret;
365}
366
367static void ethosu_remove(struct platform_device *pdev)
368{
369 struct ethosu_device *ethosudev = dev_get_drvdata(&pdev->dev);
370
371 drm_dev_unregister(&ethosudev->base);
372 ethosu_job_fini(ethosudev);
373 if (ethosudev->sram)
374 gen_pool_free(ethosudev->srampool, (unsigned long)ethosudev->sram,
375 ethosudev->npu_info.sram_size);
376}
377
378static const struct of_device_id dt_match[] = {
379 { .compatible = "arm,ethos-u65" },
380 { .compatible = "arm,ethos-u85" },
381 {}
382};
383MODULE_DEVICE_TABLE(of, dt_match);
384
385static DEFINE_RUNTIME_DEV_PM_OPS(ethosu_pm_ops,
386 ethosu_device_suspend,
387 ethosu_device_resume,
388 NULL);
389
390static struct platform_driver ethosu_driver = {
391 .probe = ethosu_probe,
392 .remove = ethosu_remove,
393 .driver = {
394 .name = "ethosu",
395 .pm = pm_ptr(&ethosu_pm_ops),
396 .of_match_table = dt_match,
397 },
398};
399module_platform_driver(ethosu_driver);
400
401MODULE_AUTHOR("Rob Herring <robh@kernel.org>");
402MODULE_DESCRIPTION("Arm Ethos-U Accel Driver");
403MODULE_LICENSE("Dual MIT/GPL");
drivers/accel/ethosu/ethosu_drv.h+15−0
@@ -0,0 +1,15 @@
1/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
2/* Copyright 2025 Arm, Ltd. */
3#ifndef __ETHOSU_DRV_H__
4#define __ETHOSU_DRV_H__
5
6#include <drm/gpu_scheduler.h>
7
8struct ethosu_device;
9
10struct ethosu_file_priv {
11 struct ethosu_device *edev;
12 struct drm_sched_entity sched_entity;
13};
14
15#endif
drivers/accel/ethosu/ethosu_gem.c+704−0
@@ -0,0 +1,704 @@
1// SPDX-License-Identifier: GPL-2.0-only or MIT
2/* Copyright 2025 Arm, Ltd. */
3
4#include <linux/err.h>
5#include <linux/slab.h>
6
7#include <drm/ethosu_accel.h>
8
9#include "ethosu_device.h"
10#include "ethosu_gem.h"
11
12static void ethosu_gem_free_object(struct drm_gem_object *obj)
13{
14 struct ethosu_gem_object *bo = to_ethosu_bo(obj);
15
16 kfree(bo->info);
17 drm_gem_free_mmap_offset(&bo->base.base);
18 drm_gem_dma_free(&bo->base);
19}
20
21static int ethosu_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
22{
23 struct ethosu_gem_object *bo = to_ethosu_bo(obj);
24
25 /* Don't allow mmap on objects that have the NO_MMAP flag set. */
26 if (bo->flags & DRM_ETHOSU_BO_NO_MMAP)
27 return -EINVAL;
28
29 return drm_gem_dma_object_mmap(obj, vma);
30}
31
32static const struct drm_gem_object_funcs ethosu_gem_funcs = {
33 .free = ethosu_gem_free_object,
34 .print_info = drm_gem_dma_object_print_info,
35 .get_sg_table = drm_gem_dma_object_get_sg_table,
36 .vmap = drm_gem_dma_object_vmap,
37 .mmap = ethosu_gem_mmap,
38 .vm_ops = &drm_gem_dma_vm_ops,
39};
40
41/**
42 * ethosu_gem_create_object - Implementation of driver->gem_create_object.
43 * @ddev: DRM device
44 * @size: Size in bytes of the memory the object will reference
45 *
46 * This lets the GEM helpers allocate object structs for us, and keep
47 * our BO stats correct.
48 */
49struct drm_gem_object *ethosu_gem_create_object(struct drm_device *ddev, size_t size)
50{
51 struct ethosu_gem_object *obj;
52
53 obj = kzalloc(sizeof(*obj), GFP_KERNEL);
54 if (!obj)
55 return ERR_PTR(-ENOMEM);
56
57 obj->base.base.funcs = &ethosu_gem_funcs;
58 return &obj->base.base;
59}
60
61/**
62 * ethosu_gem_create_with_handle() - Create a GEM object and attach it to a handle.
63 * @file: DRM file.
64 * @ddev: DRM device.
65 * @size: Size of the GEM object to allocate.
66 * @flags: Combination of drm_ethosu_bo_flags flags.
67 * @handle: Pointer holding the handle pointing to the new GEM object.
68 *
69 * Return: Zero on success
70 */
71int ethosu_gem_create_with_handle(struct drm_file *file,
72 struct drm_device *ddev,
73 u64 *size, u32 flags, u32 *handle)
74{
75 struct drm_gem_dma_object *mem;
76 struct ethosu_gem_object *bo;
77 int ret;
78
79 mem = drm_gem_dma_create(ddev, *size);
80 if (IS_ERR(mem))
81 return PTR_ERR(mem);
82
83 bo = to_ethosu_bo(&mem->base);
84 bo->flags = flags;
85
86 /*
87 * Allocate an id of idr table where the obj is registered
88 * and handle has the id what user can see.
89 */
90 ret = drm_gem_handle_create(file, &mem->base, handle);
91 if (!ret)
92 *size = bo->base.base.size;
93
94 /* drop reference from allocate - handle holds it now. */
95 drm_gem_object_put(&mem->base);
96
97 return ret;
98}
99
100struct dma {
101 s8 region;
102 u64 len;
103 u64 offset;
104 s64 stride[2];
105};
106
107struct dma_state {
108 u16 size0;
109 u16 size1;
110 s8 mode;
111 struct dma src;
112 struct dma dst;
113};
114
115struct buffer {
116 u64 base;
117 u32 length;
118 s8 region;
119};
120
121struct feat_matrix {
122 u64 base[4];
123 s64 stride_x;
124 s64 stride_y;
125 s64 stride_c;
126 s8 region;
127 u8 broadcast;
128 u16 stride_kernel;
129 u16 precision;
130 u16 depth;
131 u16 width;
132 u16 width0;
133 u16 height[3];
134 u8 pad_top;
135 u8 pad_left;
136 u8 pad_bottom;
137 u8 pad_right;
138};
139
140struct cmd_state {
141 struct dma_state dma;
142 struct buffer scale[2];
143 struct buffer weight[4];
144 struct feat_matrix ofm;
145 struct feat_matrix ifm;
146 struct feat_matrix ifm2;
147};
148
149static void cmd_state_init(struct cmd_state *st)
150{
151 /* Initialize to all 1s to detect missing setup */
152 memset(st, 0xff, sizeof(*st));
153}
154
155static u64 cmd_to_addr(u32 *cmd)
156{
157 return ((u64)((cmd[0] & 0xff0000) << 16)) | cmd[1];
158}
159
160static u64 dma_length(struct ethosu_validated_cmdstream_info *info,
161 struct dma_state *dma_st, struct dma *dma)
162{
163 s8 mode = dma_st->mode;
164 u64 len = dma->len;
165
166 if (mode >= 1) {
167 len += dma->stride[0];
168 len *= dma_st->size0;
169 }
170 if (mode == 2) {
171 len += dma->stride[1];
172 len *= dma_st->size1;
173 }
174 if (dma->region >= 0)
175 info->region_size[dma->region] = max(info->region_size[dma->region],
176 len + dma->offset);
177
178 return len;
179}
180
181static u64 feat_matrix_length(struct ethosu_validated_cmdstream_info *info,
182 struct feat_matrix *fm,
183 u32 x, u32 y, u32 c)
184{
185 u32 element_size, storage = fm->precision >> 14;
186 int tile = 0;
187 u64 addr;
188
189 if (fm->region < 0)
190 return U64_MAX;
191
192 switch (storage) {
193 case 0:
194 if (x >= fm->width0 + 1) {
195 x -= fm->width0 + 1;
196 tile += 1;
197 }
198 if (y >= fm->height[tile] + 1) {
199 y -= fm->height[tile] + 1;
200 tile += 2;
201 }
202 break;
203 case 1:
204 if (y >= fm->height[1] + 1) {
205 y -= fm->height[1] + 1;
206 tile = 2;
207 } else if (y >= fm->height[0] + 1) {
208 y -= fm->height[0] + 1;
209 tile = 1;
210 }
211 break;
212 }
213 if (fm->base[tile] == U64_MAX)
214 return U64_MAX;
215
216 addr = fm->base[tile] + y * fm->stride_y;
217
218 switch ((fm->precision >> 6) & 0x3) { // format
219 case 0: //nhwc:
220 addr += x * fm->stride_x + c;
221 break;
222 case 1: //nhcwb16:
223 element_size = BIT((fm->precision >> 1) & 0x3);
224
225 addr += (c / 16) * fm->stride_c + (16 * x + (c & 0xf)) * element_size;
226 break;
227 }
228
229 info->region_size[fm->region] = max(info->region_size[fm->region], addr + 1);
230
231 return addr;
232}
233
234static int calc_sizes(struct drm_device *ddev,
235 struct ethosu_validated_cmdstream_info *info,
236 u16 op, struct cmd_state *st,
237 bool ifm, bool ifm2, bool weight, bool scale)
238{
239 u64 len;
240
241 if (ifm) {
242 if (st->ifm.stride_kernel == U16_MAX)
243 return -EINVAL;
244 u32 stride_y = ((st->ifm.stride_kernel >> 8) & 0x2) +
245 ((st->ifm.stride_kernel >> 1) & 0x1) + 1;
246 u32 stride_x = ((st->ifm.stride_kernel >> 5) & 0x2) +
247 (st->ifm.stride_kernel & 0x1) + 1;
248 u32 ifm_height = st->ofm.height[2] * stride_y +
249 st->ifm.height[2] - (st->ifm.pad_top + st->ifm.pad_bottom);
250 u32 ifm_width = st->ofm.width * stride_x +
251 st->ifm.width - (st->ifm.pad_left + st->ifm.pad_right);
252
253 len = feat_matrix_length(info, &st->ifm, ifm_width,
254 ifm_height, st->ifm.depth);
255 dev_dbg(ddev->dev, "op %d: IFM:%d:0x%llx-0x%llx\n",
256 op, st->ifm.region, st->ifm.base[0], len);
257 if (len == U64_MAX)
258 return -EINVAL;
259 }
260
261 if (ifm2) {
262 len = feat_matrix_length(info, &st->ifm2, st->ifm.depth,
263 0, st->ofm.depth);
264 dev_dbg(ddev->dev, "op %d: IFM2:%d:0x%llx-0x%llx\n",
265 op, st->ifm2.region, st->ifm2.base[0], len);
266 if (len == U64_MAX)
267 return -EINVAL;
268 }
269
270 if (weight) {
271 dev_dbg(ddev->dev, "op %d: W:%d:0x%llx-0x%llx\n",
272 op, st->weight[0].region, st->weight[0].base,
273 st->weight[0].base + st->weight[0].length - 1);
274 if (st->weight[0].region < 0 || st->weight[0].base == U64_MAX ||
275 st->weight[0].length == U32_MAX)
276 return -EINVAL;
277 info->region_size[st->weight[0].region] =
278 max(info->region_size[st->weight[0].region],
279 st->weight[0].base + st->weight[0].length);
280 }
281
282 if (scale) {
283 dev_dbg(ddev->dev, "op %d: S:%d:0x%llx-0x%llx\n",
284 op, st->scale[0].region, st->scale[0].base,
285 st->scale[0].base + st->scale[0].length - 1);
286 if (st->scale[0].region < 0 || st->scale[0].base == U64_MAX ||
287 st->scale[0].length == U32_MAX)
288 return -EINVAL;
289 info->region_size[st->scale[0].region] =
290 max(info->region_size[st->scale[0].region],
291 st->scale[0].base + st->scale[0].length);
292 }
293
294 len = feat_matrix_length(info, &st->ofm, st->ofm.width,
295 st->ofm.height[2], st->ofm.depth);
296 dev_dbg(ddev->dev, "op %d: OFM:%d:0x%llx-0x%llx\n",
297 op, st->ofm.region, st->ofm.base[0], len);
298 if (len == U64_MAX)
299 return -EINVAL;
300 info->output_region[st->ofm.region] = true;
301
302 return 0;
303}
304
305static int calc_sizes_elemwise(struct drm_device *ddev,
306 struct ethosu_validated_cmdstream_info *info,
307 u16 op, struct cmd_state *st,
308 bool ifm, bool ifm2)
309{
310 u32 height, width, depth;
311 u64 len;
312
313 if (ifm) {
314 height = st->ifm.broadcast & 0x1 ? 0 : st->ofm.height[2];
315 width = st->ifm.broadcast & 0x2 ? 0 : st->ofm.width;
316 depth = st->ifm.broadcast & 0x4 ? 0 : st->ofm.depth;
317
318 len = feat_matrix_length(info, &st->ifm, width,
319 height, depth);
320 dev_dbg(ddev->dev, "op %d: IFM:%d:0x%llx-0x%llx\n",
321 op, st->ifm.region, st->ifm.base[0], len);
322 if (len == U64_MAX)
323 return -EINVAL;
324 }
325
326 if (ifm2) {
327 height = st->ifm2.broadcast & 0x1 ? 0 : st->ofm.height[2];
328 width = st->ifm2.broadcast & 0x2 ? 0 : st->ofm.width;
329 depth = st->ifm2.broadcast & 0x4 ? 0 : st->ofm.depth;
330
331 len = feat_matrix_length(info, &st->ifm2, width,
332 height, depth);
333 dev_dbg(ddev->dev, "op %d: IFM2:%d:0x%llx-0x%llx\n",
334 op, st->ifm2.region, st->ifm2.base[0], len);
335 if (len == U64_MAX)
336 return -EINVAL;
337 }
338
339 len = feat_matrix_length(info, &st->ofm, st->ofm.width,
340 st->ofm.height[2], st->ofm.depth);
341 dev_dbg(ddev->dev, "op %d: OFM:%d:0x%llx-0x%llx\n",
342 op, st->ofm.region, st->ofm.base[0], len);
343 if (len == U64_MAX)
344 return -EINVAL;
345 info->output_region[st->ofm.region] = true;
346
347 return 0;
348}
349
350static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,
351 u32 __user *ucmds,
352 struct ethosu_gem_object *bo,
353 u32 size)
354{
355 struct ethosu_validated_cmdstream_info __free(kfree) *info = kzalloc(sizeof(*info), GFP_KERNEL);
356 struct ethosu_device *edev = to_ethosu_device(ddev);
357 u32 *bocmds = bo->base.vaddr;
358 struct cmd_state st;
359 int i, ret;
360
361 if (!info)
362 return -ENOMEM;
363 info->cmd_size = size;
364
365 cmd_state_init(&st);
366
367 for (i = 0; i < size / 4; i++) {
368 bool use_ifm, use_ifm2, use_scale;
369 u64 dstlen, srclen;
370 u16 cmd, param;
371 u32 cmds[2];
372 u64 addr;
373
374 if (get_user(cmds[0], ucmds++))
375 return -EFAULT;
376
377 bocmds[i] = cmds[0];
378
379 cmd = cmds[0];
380 param = cmds[0] >> 16;
381
382 if (cmd & 0x4000) {
383 if (get_user(cmds[1], ucmds++))
384 return -EFAULT;
385
386 i++;
387 bocmds[i] = cmds[1];
388 addr = cmd_to_addr(cmds);
389 }
390
391 switch (cmd) {
392 case NPU_OP_DMA_START:
393 srclen = dma_length(info, &st.dma, &st.dma.src);
394 dstlen = dma_length(info, &st.dma, &st.dma.dst);
395
396 if (st.dma.dst.region >= 0)
397 info->output_region[st.dma.dst.region] = true;
398 dev_dbg(ddev->dev, "cmd: DMA SRC:%d:0x%llx+0x%llx DST:%d:0x%llx+0x%llx\n",
399 st.dma.src.region, st.dma.src.offset, srclen,
400 st.dma.dst.region, st.dma.dst.offset, dstlen);
401 break;
402 case NPU_OP_CONV:
403 case NPU_OP_DEPTHWISE:
404 use_ifm2 = param & 0x1; // weights_ifm2
405 use_scale = !(st.ofm.precision & 0x100);
406 ret = calc_sizes(ddev, info, cmd, &st, true, use_ifm2,
407 !use_ifm2, use_scale);
408 if (ret)
409 return ret;
410 break;
411 case NPU_OP_POOL:
412 use_ifm = param != 0x4; // pooling mode
413 use_scale = !(st.ofm.precision & 0x100);
414 ret = calc_sizes(ddev, info, cmd, &st, use_ifm, false,
415 false, use_scale);
416 if (ret)
417 return ret;
418 break;
419 case NPU_OP_ELEMENTWISE:
420 use_ifm2 = !((st.ifm2.broadcast == 8) || (param == 5) ||
421 (param == 6) || (param == 7) || (param == 0x24));
422 use_ifm = st.ifm.broadcast != 8;
423 ret = calc_sizes_elemwise(ddev, info, cmd, &st, use_ifm, use_ifm2);
424 if (ret)
425 return ret;
426 break;
427 case NPU_OP_RESIZE: // U85 only
428 WARN_ON(1); // TODO
429 break;
430 case NPU_SET_KERNEL_WIDTH_M1:
431 st.ifm.width = param;
432 break;
433 case NPU_SET_KERNEL_HEIGHT_M1:
434 st.ifm.height[2] = param;
435 break;
436 case NPU_SET_KERNEL_STRIDE:
437 st.ifm.stride_kernel = param;
438 break;
439 case NPU_SET_IFM_PAD_TOP:
440 st.ifm.pad_top = param & 0x7f;
441 break;
442 case NPU_SET_IFM_PAD_LEFT:
443 st.ifm.pad_left = param & 0x7f;
444 break;
445 case NPU_SET_IFM_PAD_RIGHT:
446 st.ifm.pad_right = param & 0xff;
447 break;
448 case NPU_SET_IFM_PAD_BOTTOM:
449 st.ifm.pad_bottom = param & 0xff;
450 break;
451 case NPU_SET_IFM_DEPTH_M1:
452 st.ifm.depth = param;
453 break;
454 case NPU_SET_IFM_PRECISION:
455 st.ifm.precision = param;
456 break;
457 case NPU_SET_IFM_BROADCAST:
458 st.ifm.broadcast = param;
459 break;
460 case NPU_SET_IFM_REGION:
461 st.ifm.region = param & 0x7f;
462 break;
463 case NPU_SET_IFM_WIDTH0_M1:
464 st.ifm.width0 = param;
465 break;
466 case NPU_SET_IFM_HEIGHT0_M1:
467 st.ifm.height[0] = param;
468 break;
469 case NPU_SET_IFM_HEIGHT1_M1:
470 st.ifm.height[1] = param;
471 break;
472 case NPU_SET_IFM_BASE0:
473 case NPU_SET_IFM_BASE1:
474 case NPU_SET_IFM_BASE2:
475 case NPU_SET_IFM_BASE3:
476 st.ifm.base[cmd & 0x3] = addr;
477 break;
478 case NPU_SET_IFM_STRIDE_X:
479 st.ifm.stride_x = addr;
480 break;
481 case NPU_SET_IFM_STRIDE_Y:
482 st.ifm.stride_y = addr;
483 break;
484 case NPU_SET_IFM_STRIDE_C:
485 st.ifm.stride_c = addr;
486 break;
487
488 case NPU_SET_OFM_WIDTH_M1:
489 st.ofm.width = param;
490 break;
491 case NPU_SET_OFM_HEIGHT_M1:
492 st.ofm.height[2] = param;
493 break;
494 case NPU_SET_OFM_DEPTH_M1:
495 st.ofm.depth = param;
496 break;
497 case NPU_SET_OFM_PRECISION:
498 st.ofm.precision = param;
499 break;
500 case NPU_SET_OFM_REGION:
501 st.ofm.region = param & 0x7;
502 break;
503 case NPU_SET_OFM_WIDTH0_M1:
504 st.ofm.width0 = param;
505 break;
506 case NPU_SET_OFM_HEIGHT0_M1:
507 st.ofm.height[0] = param;
508 break;
509 case NPU_SET_OFM_HEIGHT1_M1:
510 st.ofm.height[1] = param;
511 break;
512 case NPU_SET_OFM_BASE0:
513 case NPU_SET_OFM_BASE1:
514 case NPU_SET_OFM_BASE2:
515 case NPU_SET_OFM_BASE3:
516 st.ofm.base[cmd & 0x3] = addr;
517 break;
518 case NPU_SET_OFM_STRIDE_X:
519 st.ofm.stride_x = addr;
520 break;
521 case NPU_SET_OFM_STRIDE_Y:
522 st.ofm.stride_y = addr;
523 break;
524 case NPU_SET_OFM_STRIDE_C:
525 st.ofm.stride_c = addr;
526 break;
527
528 case NPU_SET_IFM2_BROADCAST:
529 st.ifm2.broadcast = param;
530 break;
531 case NPU_SET_IFM2_PRECISION:
532 st.ifm2.precision = param;
533 break;
534 case NPU_SET_IFM2_REGION:
535 st.ifm2.region = param & 0x7;
536 break;
537 case NPU_SET_IFM2_WIDTH0_M1:
538 st.ifm2.width0 = param;
539 break;
540 case NPU_SET_IFM2_HEIGHT0_M1:
541 st.ifm2.height[0] = param;
542 break;
543 case NPU_SET_IFM2_HEIGHT1_M1:
544 st.ifm2.height[1] = param;
545 break;
546 case NPU_SET_IFM2_BASE0:
547 case NPU_SET_IFM2_BASE1:
548 case NPU_SET_IFM2_BASE2:
549 case NPU_SET_IFM2_BASE3:
550 st.ifm2.base[cmd & 0x3] = addr;
551 break;
552 case NPU_SET_IFM2_STRIDE_X:
553 st.ifm2.stride_x = addr;
554 break;
555 case NPU_SET_IFM2_STRIDE_Y:
556 st.ifm2.stride_y = addr;
557 break;
558 case NPU_SET_IFM2_STRIDE_C:
559 st.ifm2.stride_c = addr;
560 break;
561
562 case NPU_SET_WEIGHT_REGION:
563 st.weight[0].region = param & 0x7;
564 break;
565 case NPU_SET_SCALE_REGION:
566 st.scale[0].region = param & 0x7;
567 break;
568 case NPU_SET_WEIGHT_BASE:
569 st.weight[0].base = addr;
570 break;
571 case NPU_SET_WEIGHT_LENGTH:
572 st.weight[0].length = cmds[1];
573 break;
574 case NPU_SET_SCALE_BASE:
575 st.scale[0].base = addr;
576 break;
577 case NPU_SET_SCALE_LENGTH:
578 st.scale[0].length = cmds[1];
579 break;
580 case NPU_SET_WEIGHT1_BASE:
581 st.weight[1].base = addr;
582 break;
583 case NPU_SET_WEIGHT1_LENGTH:
584 st.weight[1].length = cmds[1];
585 break;
586 case NPU_SET_SCALE1_BASE: // NPU_SET_WEIGHT2_BASE (U85)
587 if (ethosu_is_u65(edev))
588 st.scale[1].base = addr;
589 else
590 st.weight[2].base = addr;
591 break;
592 case NPU_SET_SCALE1_LENGTH: // NPU_SET_WEIGHT2_LENGTH (U85)
593 if (ethosu_is_u65(edev))
594 st.scale[1].length = cmds[1];
595 else
596 st.weight[1].length = cmds[1];
597 break;
598 case NPU_SET_WEIGHT3_BASE:
599 st.weight[3].base = addr;
600 break;
601 case NPU_SET_WEIGHT3_LENGTH:
602 st.weight[3].length = cmds[1];
603 break;
604
605 case NPU_SET_DMA0_SRC_REGION:
606 if (param & 0x100)
607 st.dma.src.region = -1;
608 else
609 st.dma.src.region = param & 0x7;
610 st.dma.mode = (param >> 9) & 0x3;
611 break;
612 case NPU_SET_DMA0_DST_REGION:
613 if (param & 0x100)
614 st.dma.dst.region = -1;
615 else
616 st.dma.dst.region = param & 0x7;
617 break;
618 case NPU_SET_DMA0_SIZE0:
619 st.dma.size0 = param;
620 break;
621 case NPU_SET_DMA0_SIZE1:
622 st.dma.size1 = param;
623 break;
624 case NPU_SET_DMA0_SRC_STRIDE0:
625 st.dma.src.stride[0] = ((s64)addr << 24) >> 24;
626 break;
627 case NPU_SET_DMA0_SRC_STRIDE1:
628 st.dma.src.stride[1] = ((s64)addr << 24) >> 24;
629 break;
630 case NPU_SET_DMA0_DST_STRIDE0:
631 st.dma.dst.stride[0] = ((s64)addr << 24) >> 24;
632 break;
633 case NPU_SET_DMA0_DST_STRIDE1:
634 st.dma.dst.stride[1] = ((s64)addr << 24) >> 24;
635 break;
636 case NPU_SET_DMA0_SRC:
637 st.dma.src.offset = addr;
638 break;
639 case NPU_SET_DMA0_DST:
640 st.dma.dst.offset = addr;
641 break;
642 case NPU_SET_DMA0_LEN:
643 st.dma.src.len = st.dma.dst.len = addr;
644 break;
645 default:
646 break;
647 }
648 }
649
650 for (i = 0; i < NPU_BASEP_REGION_MAX; i++) {
651 if (!info->region_size[i])
652 continue;
653 dev_dbg(ddev->dev, "region %d max size: 0x%llx\n",
654 i, info->region_size[i]);
655 }
656
657 bo->info = no_free_ptr(info);
658 return 0;
659}
660
661/**
662 * ethosu_gem_cmdstream_create() - Create a GEM object and attach it to a handle.
663 * @file: DRM file.
664 * @ddev: DRM device.
665 * @exclusive_vm: Exclusive VM. Not NULL if the GEM object can't be shared.
666 * @size: Size of the GEM object to allocate.
667 * @flags: Combination of drm_ethosu_bo_flags flags.
668 * @handle: Pointer holding the handle pointing to the new GEM object.
669 *
670 * Return: Zero on success
671 */
672int ethosu_gem_cmdstream_create(struct drm_file *file,
673 struct drm_device *ddev,
674 u32 size, u64 data, u32 flags, u32 *handle)
675{
676 int ret;
677 struct drm_gem_dma_object *mem;
678 struct ethosu_gem_object *bo;
679
680 mem = drm_gem_dma_create(ddev, size);
681 if (IS_ERR(mem))
682 return PTR_ERR(mem);
683
684 bo = to_ethosu_bo(&mem->base);
685 bo->flags = flags;
686
687 ret = ethosu_gem_cmdstream_copy_and_validate(ddev,
688 (void __user *)(uintptr_t)data,
689 bo, size);
690 if (ret)
691 goto fail;
692
693 /*
694 * Allocate an id of idr table where the obj is registered
695 * and handle has the id what user can see.
696 */
697 ret = drm_gem_handle_create(file, &mem->base, handle);
698
699fail:
700 /* drop reference from allocate - handle holds it now. */
701 drm_gem_object_put(&mem->base);
702
703 return ret;
704}
drivers/accel/ethosu/ethosu_gem.h+46−0
@@ -0,0 +1,46 @@
1/* SPDX-License-Identifier: GPL-2.0 or MIT */
2/* Copyright 2025 Arm, Ltd. */
3
4#ifndef __ETHOSU_GEM_H__
5#define __ETHOSU_GEM_H__
6
7#include "ethosu_device.h"
8#include <drm/drm_gem_dma_helper.h>
9
10struct ethosu_validated_cmdstream_info {
11 u32 cmd_size;
12 u64 region_size[NPU_BASEP_REGION_MAX];
13 bool output_region[NPU_BASEP_REGION_MAX];
14};
15
16/**
17 * struct ethosu_gem_object - Driver specific GEM object.
18 */
19struct ethosu_gem_object {
20 /** @base: Inherit from drm_gem_shmem_object. */
21 struct drm_gem_dma_object base;
22
23 struct ethosu_validated_cmdstream_info *info;
24
25 /** @flags: Combination of drm_ethosu_bo_flags flags. */
26 u32 flags;
27};
28
29static inline
30struct ethosu_gem_object *to_ethosu_bo(struct drm_gem_object *obj)
31{
32 return container_of(to_drm_gem_dma_obj(obj), struct ethosu_gem_object, base);
33}
34
35struct drm_gem_object *ethosu_gem_create_object(struct drm_device *ddev,
36 size_t size);
37
38int ethosu_gem_create_with_handle(struct drm_file *file,
39 struct drm_device *ddev,
40 u64 *size, u32 flags, uint32_t *handle);
41
42int ethosu_gem_cmdstream_create(struct drm_file *file,
43 struct drm_device *ddev,
44 u32 size, u64 data, u32 flags, u32 *handle);
45
46#endif /* __ETHOSU_GEM_H__ */
drivers/accel/ethosu/ethosu_job.c+497−0
@@ -0,0 +1,497 @@
1// SPDX-License-Identifier: GPL-2.0-only OR MIT
2/* Copyright 2024-2025 Tomeu Vizoso <tomeu@tomeuvizoso.net> */
3/* Copyright 2025 Arm, Ltd. */
4
5#include <linux/bitfield.h>
6#include <linux/genalloc.h>
7#include <linux/interrupt.h>
8#include <linux/iopoll.h>
9#include <linux/platform_device.h>
10#include <linux/pm_runtime.h>
11
12#include <drm/drm_file.h>
13#include <drm/drm_gem.h>
14#include <drm/drm_gem_dma_helper.h>
15#include <drm/drm_print.h>
16#include <drm/ethosu_accel.h>
17
18#include "ethosu_device.h"
19#include "ethosu_drv.h"
20#include "ethosu_gem.h"
21#include "ethosu_job.h"
22
23#define JOB_TIMEOUT_MS 500
24
25static struct ethosu_job *to_ethosu_job(struct drm_sched_job *sched_job)
26{
27 return container_of(sched_job, struct ethosu_job, base);
28}
29
30static const char *ethosu_fence_get_driver_name(struct dma_fence *fence)
31{
32 return "ethosu";
33}
34
35static const char *ethosu_fence_get_timeline_name(struct dma_fence *fence)
36{
37 return "ethosu-npu";
38}
39
40static const struct dma_fence_ops ethosu_fence_ops = {
41 .get_driver_name = ethosu_fence_get_driver_name,
42 .get_timeline_name = ethosu_fence_get_timeline_name,
43};
44
45static void ethosu_job_hw_submit(struct ethosu_device *dev, struct ethosu_job *job)
46{
47 struct drm_gem_dma_object *cmd_bo = to_drm_gem_dma_obj(job->cmd_bo);
48 struct ethosu_validated_cmdstream_info *cmd_info = to_ethosu_bo(job->cmd_bo)->info;
49
50 for (int i = 0; i < job->region_cnt; i++) {
51 struct drm_gem_dma_object *bo;
52 int region = job->region_bo_num[i];
53
54 bo = to_drm_gem_dma_obj(job->region_bo[i]);
55 writel_relaxed(lower_32_bits(bo->dma_addr), dev->regs + NPU_REG_BASEP(region));
56 writel_relaxed(upper_32_bits(bo->dma_addr), dev->regs + NPU_REG_BASEP_HI(region));
57 dev_dbg(dev->base.dev, "Region %d base addr = %pad\n", region, &bo->dma_addr);
58 }
59
60 if (job->sram_size) {
61 writel_relaxed(lower_32_bits(dev->sramphys),
62 dev->regs + NPU_REG_BASEP(ETHOSU_SRAM_REGION));
63 writel_relaxed(upper_32_bits(dev->sramphys),
64 dev->regs + NPU_REG_BASEP_HI(ETHOSU_SRAM_REGION));
65 dev_dbg(dev->base.dev, "Region %d base addr = %pad (SRAM)\n",
66 ETHOSU_SRAM_REGION, &dev->sramphys);
67 }
68
69 writel_relaxed(lower_32_bits(cmd_bo->dma_addr), dev->regs + NPU_REG_QBASE);
70 writel_relaxed(upper_32_bits(cmd_bo->dma_addr), dev->regs + NPU_REG_QBASE_HI);
71 writel_relaxed(cmd_info->cmd_size, dev->regs + NPU_REG_QSIZE);
72
73 writel(CMD_TRANSITION_TO_RUN, dev->regs + NPU_REG_CMD);
74
75 dev_dbg(dev->base.dev,
76 "Submitted cmd at %pad to core\n", &cmd_bo->dma_addr);
77}
78
79static int ethosu_acquire_object_fences(struct ethosu_job *job)
80{
81 int i, ret;
82 struct drm_gem_object **bos = job->region_bo;
83 struct ethosu_validated_cmdstream_info *info = to_ethosu_bo(job->cmd_bo)->info;
84
85 for (i = 0; i < job->region_cnt; i++) {
86 bool is_write;
87
88 if (!bos[i])
89 break;
90
91 ret = dma_resv_reserve_fences(bos[i]->resv, 1);
92 if (ret)
93 return ret;
94
95 is_write = info->output_region[job->region_bo_num[i]];
96 ret = drm_sched_job_add_implicit_dependencies(&job->base, bos[i],
97 is_write);
98 if (ret)
99 return ret;
100 }
101
102 return 0;
103}
104
105static void ethosu_attach_object_fences(struct ethosu_job *job)
106{
107 int i;
108 struct dma_fence *fence = job->inference_done_fence;
109 struct drm_gem_object **bos = job->region_bo;
110 struct ethosu_validated_cmdstream_info *info = to_ethosu_bo(job->cmd_bo)->info;
111
112 for (i = 0; i < job->region_cnt; i++)
113 if (info->output_region[job->region_bo_num[i]])
114 dma_resv_add_fence(bos[i]->resv, fence, DMA_RESV_USAGE_WRITE);
115}
116
117static int ethosu_job_push(struct ethosu_job *job)
118{
119 struct ww_acquire_ctx acquire_ctx;
120 int ret;
121
122 ret = drm_gem_lock_reservations(job->region_bo, job->region_cnt, &acquire_ctx);
123 if (ret)
124 return ret;
125
126 ret = ethosu_acquire_object_fences(job);
127 if (ret)
128 goto out;
129
130 ret = pm_runtime_resume_and_get(job->dev->base.dev);
131 if (!ret) {
132 guard(mutex)(&job->dev->sched_lock);
133
134 drm_sched_job_arm(&job->base);
135 job->inference_done_fence = dma_fence_get(&job->base.s_fence->finished);
136 kref_get(&job->refcount); /* put by scheduler job completion */
137 drm_sched_entity_push_job(&job->base);
138 ethosu_attach_object_fences(job);
139 }
140
141out:
142 drm_gem_unlock_reservations(job->region_bo, job->region_cnt, &acquire_ctx);
143 return ret;
144}
145
146static void ethosu_job_cleanup(struct kref *ref)
147{
148 struct ethosu_job *job = container_of(ref, struct ethosu_job,
149 refcount);
150 unsigned int i;
151
152 pm_runtime_put_autosuspend(job->dev->base.dev);
153
154 dma_fence_put(job->done_fence);
155 dma_fence_put(job->inference_done_fence);
156
157 for (i = 0; i < job->region_cnt; i++)
158 drm_gem_object_put(job->region_bo[i]);
159
160 drm_gem_object_put(job->cmd_bo);
161
162 kfree(job);
163}
164
165static void ethosu_job_put(struct ethosu_job *job)
166{
167 kref_put(&job->refcount, ethosu_job_cleanup);
168}
169
170static void ethosu_job_free(struct drm_sched_job *sched_job)
171{
172 struct ethosu_job *job = to_ethosu_job(sched_job);
173
174 drm_sched_job_cleanup(sched_job);
175 ethosu_job_put(job);
176}
177
178static struct dma_fence *ethosu_job_run(struct drm_sched_job *sched_job)
179{
180 struct ethosu_job *job = to_ethosu_job(sched_job);
181 struct ethosu_device *dev = job->dev;
182 struct dma_fence *fence = job->done_fence;
183
184 if (unlikely(job->base.s_fence->finished.error))
185 return NULL;
186
187 dma_fence_init(fence, &ethosu_fence_ops, &dev->fence_lock,
188 dev->fence_context, ++dev->emit_seqno);
189 dma_fence_get(fence);
190
191 scoped_guard(mutex, &dev->job_lock) {
192 dev->in_flight_job = job;
193 ethosu_job_hw_submit(dev, job);
194 }
195
196 return fence;
197}
198
199static void ethosu_job_handle_irq(struct ethosu_device *dev)
200{
201 u32 status = readl_relaxed(dev->regs + NPU_REG_STATUS);
202
203 if (status & (STATUS_BUS_STATUS | STATUS_CMD_PARSE_ERR)) {
204 dev_err(dev->base.dev, "Error IRQ - %x\n", status);
205 drm_sched_fault(&dev->sched);
206 return;
207 }
208
209 scoped_guard(mutex, &dev->job_lock) {
210 if (dev->in_flight_job) {
211 dma_fence_signal(dev->in_flight_job->done_fence);
212 dev->in_flight_job = NULL;
213 }
214 }
215}
216
217static irqreturn_t ethosu_job_irq_handler_thread(int irq, void *data)
218{
219 struct ethosu_device *dev = data;
220
221 ethosu_job_handle_irq(dev);
222
223 return IRQ_HANDLED;
224}
225
226static irqreturn_t ethosu_job_irq_handler(int irq, void *data)
227{
228 struct ethosu_device *dev = data;
229 u32 status = readl_relaxed(dev->regs + NPU_REG_STATUS);
230
231 if (!(status & STATUS_IRQ_RAISED))
232 return IRQ_NONE;
233
234 writel_relaxed(CMD_CLEAR_IRQ, dev->regs + NPU_REG_CMD);
235 return IRQ_WAKE_THREAD;
236}
237
238static enum drm_gpu_sched_stat ethosu_job_timedout(struct drm_sched_job *bad)
239{
240 struct ethosu_job *job = to_ethosu_job(bad);
241 struct ethosu_device *dev = job->dev;
242 bool running;
243 u32 *bocmds = to_drm_gem_dma_obj(job->cmd_bo)->vaddr;
244 u32 cmdaddr;
245
246 cmdaddr = readl_relaxed(dev->regs + NPU_REG_QREAD);
247 running = FIELD_GET(STATUS_STATE_RUNNING, readl_relaxed(dev->regs + NPU_REG_STATUS));
248
249 if (running) {
250 int ret;
251 u32 reg;
252
253 ret = readl_relaxed_poll_timeout(dev->regs + NPU_REG_QREAD,
254 reg,
255 reg != cmdaddr,
256 USEC_PER_MSEC, 100 * USEC_PER_MSEC);
257
258 /* If still running and progress is being made, just return */
259 if (!ret)
260 return DRM_GPU_SCHED_STAT_NO_HANG;
261 }
262
263 dev_err(dev->base.dev, "NPU sched timed out: NPU %s, cmdstream offset 0x%x: 0x%x\n",
264 running ? "running" : "stopped",
265 cmdaddr, bocmds[cmdaddr / 4]);
266
267 drm_sched_stop(&dev->sched, bad);
268
269 scoped_guard(mutex, &dev->job_lock)
270 dev->in_flight_job = NULL;
271
272 /* Proceed with reset now. */
273 pm_runtime_force_suspend(dev->base.dev);
274 pm_runtime_force_resume(dev->base.dev);
275
276 /* Restart the scheduler */
277 drm_sched_start(&dev->sched, 0);
278
279 return DRM_GPU_SCHED_STAT_RESET;
280}
281
282static const struct drm_sched_backend_ops ethosu_sched_ops = {
283 .run_job = ethosu_job_run,
284 .timedout_job = ethosu_job_timedout,
285 .free_job = ethosu_job_free
286};
287
288int ethosu_job_init(struct ethosu_device *edev)
289{
290 struct device *dev = edev->base.dev;
291 struct drm_sched_init_args args = {
292 .ops = &ethosu_sched_ops,
293 .num_rqs = DRM_SCHED_PRIORITY_COUNT,
294 .credit_limit = 1,
295 .timeout = msecs_to_jiffies(JOB_TIMEOUT_MS),
296 .name = dev_name(dev),
297 .dev = dev,
298 };
299 int ret;
300
301 spin_lock_init(&edev->fence_lock);
302 ret = devm_mutex_init(dev, &edev->job_lock);
303 if (ret)
304 return ret;
305 ret = devm_mutex_init(dev, &edev->sched_lock);
306 if (ret)
307 return ret;
308
309 edev->irq = platform_get_irq(to_platform_device(dev), 0);
310 if (edev->irq < 0)
311 return edev->irq;
312
313 ret = devm_request_threaded_irq(dev, edev->irq,
314 ethosu_job_irq_handler,
315 ethosu_job_irq_handler_thread,
316 IRQF_SHARED, KBUILD_MODNAME,
317 edev);
318 if (ret) {
319 dev_err(dev, "failed to request irq\n");
320 return ret;
321 }
322
323 edev->fence_context = dma_fence_context_alloc(1);
324
325 ret = drm_sched_init(&edev->sched, &args);
326 if (ret) {
327 dev_err(dev, "Failed to create scheduler: %d\n", ret);
328 goto err_sched;
329 }
330
331 return 0;
332
333err_sched:
334 drm_sched_fini(&edev->sched);
335 return ret;
336}
337
338void ethosu_job_fini(struct ethosu_device *dev)
339{
340 drm_sched_fini(&dev->sched);
341}
342
343int ethosu_job_open(struct ethosu_file_priv *ethosu_priv)
344{
345 struct ethosu_device *dev = ethosu_priv->edev;
346 struct drm_gpu_scheduler *sched = &dev->sched;
347 int ret;
348
349 ret = drm_sched_entity_init(&ethosu_priv->sched_entity,
350 DRM_SCHED_PRIORITY_NORMAL,
351 &sched, 1, NULL);
352 return WARN_ON(ret);
353}
354
355void ethosu_job_close(struct ethosu_file_priv *ethosu_priv)
356{
357 struct drm_sched_entity *entity = &ethosu_priv->sched_entity;
358
359 drm_sched_entity_destroy(entity);
360}
361
362static int ethosu_ioctl_submit_job(struct drm_device *dev, struct drm_file *file,
363 struct drm_ethosu_job *job)
364{
365 struct ethosu_device *edev = to_ethosu_device(dev);
366 struct ethosu_file_priv *file_priv = file->driver_priv;
367 struct ethosu_job *ejob = NULL;
368 struct ethosu_validated_cmdstream_info *cmd_info;
369 int ret = 0;
370
371 /* BO region 2 is reserved if SRAM is used */
372 if (job->region_bo_handles[ETHOSU_SRAM_REGION] && job->sram_size)
373 return -EINVAL;
374
375 if (edev->npu_info.sram_size < job->sram_size)
376 return -EINVAL;
377
378 ejob = kzalloc(sizeof(*ejob), GFP_KERNEL);
379 if (!ejob)
380 return -ENOMEM;
381
382 kref_init(&ejob->refcount);
383
384 ejob->dev = edev;
385 ejob->sram_size = job->sram_size;
386
387 ejob->done_fence = kzalloc(sizeof(*ejob->done_fence), GFP_KERNEL);
388 if (!ejob->done_fence) {
389 ret = -ENOMEM;
390 goto out_cleanup_job;
391 }
392
393 ret = drm_sched_job_init(&ejob->base,
394 &file_priv->sched_entity,
395 1, NULL, file->client_id);
396 if (ret)
397 goto out_put_job;
398
399 ejob->cmd_bo = drm_gem_object_lookup(file, job->cmd_bo);
400 if (!ejob->cmd_bo) {
401 ret = -ENOENT;
402 goto out_cleanup_job;
403 }
404 cmd_info = to_ethosu_bo(ejob->cmd_bo)->info;
405 if (!cmd_info) {
406 ret = -EINVAL;
407 goto out_cleanup_job;
408 }
409
410 for (int i = 0; i < NPU_BASEP_REGION_MAX; i++) {
411 struct drm_gem_object *gem;
412
413 /* Can only omit a BO handle if the region is not used or used for SRAM */
414 if (!job->region_bo_handles[i] &&
415 (!cmd_info->region_size[i] || (i == ETHOSU_SRAM_REGION && job->sram_size)))
416 continue;
417
418 if (job->region_bo_handles[i] && !cmd_info->region_size[i]) {
419 dev_err(dev->dev,
420 "Cmdstream BO handle %d set for unused region %d\n",
421 job->region_bo_handles[i], i);
422 ret = -EINVAL;
423 goto out_cleanup_job;
424 }
425
426 gem = drm_gem_object_lookup(file, job->region_bo_handles[i]);
427 if (!gem) {
428 dev_err(dev->dev,
429 "Invalid BO handle %d for region %d\n",
430 job->region_bo_handles[i], i);
431 ret = -ENOENT;
432 goto out_cleanup_job;
433 }
434
435 ejob->region_bo[ejob->region_cnt] = gem;
436 ejob->region_bo_num[ejob->region_cnt] = i;
437 ejob->region_cnt++;
438
439 if (to_ethosu_bo(gem)->info) {
440 dev_err(dev->dev,
441 "Cmdstream BO handle %d used for region %d\n",
442 job->region_bo_handles[i], i);
443 ret = -EINVAL;
444 goto out_cleanup_job;
445 }
446
447 /* Verify the command stream doesn't have accesses outside the BO */
448 if (cmd_info->region_size[i] > gem->size) {
449 dev_err(dev->dev,
450 "cmd stream region %d size greater than BO size (%llu > %zu)\n",
451 i, cmd_info->region_size[i], gem->size);
452 ret = -EOVERFLOW;
453 goto out_cleanup_job;
454 }
455 }
456 ret = ethosu_job_push(ejob);
457
458out_cleanup_job:
459 if (ret)
460 drm_sched_job_cleanup(&ejob->base);
461out_put_job:
462 ethosu_job_put(ejob);
463
464 return ret;
465}
466
467int ethosu_ioctl_submit(struct drm_device *dev, void *data, struct drm_file *file)
468{
469 struct drm_ethosu_submit *args = data;
470 int ret = 0;
471 unsigned int i = 0;
472
473 if (args->pad) {
474 drm_dbg(dev, "Reserved field in drm_ethosu_submit struct should be 0.\n");
475 return -EINVAL;
476 }
477
478 struct drm_ethosu_job __free(kvfree) *jobs =
479 kvmalloc_array(args->job_count, sizeof(*jobs), GFP_KERNEL);
480 if (!jobs)
481 return -ENOMEM;
482
483 if (copy_from_user(jobs,
484 (void __user *)(uintptr_t)args->jobs,
485 args->job_count * sizeof(*jobs))) {
486 drm_dbg(dev, "Failed to copy incoming job array\n");
487 return -EFAULT;
488 }
489
490 for (i = 0; i < args->job_count; i++) {
491 ret = ethosu_ioctl_submit_job(dev, file, &jobs[i]);
492 if (ret)
493 return ret;
494 }
495
496 return 0;
497}
drivers/accel/ethosu/ethosu_job.h+40−0
@@ -0,0 +1,40 @@
1/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
2/* Copyright 2024-2025 Tomeu Vizoso <tomeu@tomeuvizoso.net> */
3/* Copyright 2025 Arm, Ltd. */
4
5#ifndef __ETHOSU_JOB_H__
6#define __ETHOSU_JOB_H__
7
8#include <linux/kref.h>
9#include <drm/gpu_scheduler.h>
10
11struct ethosu_device;
12struct ethosu_file_priv;
13
14struct ethosu_job {
15 struct drm_sched_job base;
16 struct ethosu_device *dev;
17
18 struct drm_gem_object *cmd_bo;
19 struct drm_gem_object *region_bo[NPU_BASEP_REGION_MAX];
20 u8 region_bo_num[NPU_BASEP_REGION_MAX];
21 u8 region_cnt;
22 u32 sram_size;
23
24 /* Fence to be signaled by drm-sched once its done with the job */
25 struct dma_fence *inference_done_fence;
26
27 /* Fence to be signaled by IRQ handler when the job is complete. */
28 struct dma_fence *done_fence;
29
30 struct kref refcount;
31};
32
33int ethosu_ioctl_submit(struct drm_device *dev, void *data, struct drm_file *file);
34
35int ethosu_job_init(struct ethosu_device *dev);
36void ethosu_job_fini(struct ethosu_device *dev);
37int ethosu_job_open(struct ethosu_file_priv *ethosu_priv);
38void ethosu_job_close(struct ethosu_file_priv *ethosu_priv);
39
40#endif
drivers/accel/ivpu/Makefile+1−0
@@ -6,6 +6,7 @@ intel_vpu-y := \ · 5 unmodified lines
6 ivpu_fw.o \6 ivpu_fw.o \
7 ivpu_fw_log.o \7 ivpu_fw_log.o \
8 ivpu_gem.o \8 ivpu_gem.o \
9 ivpu_gem_userptr.o \
9 ivpu_hw.o \10 ivpu_hw.o \
10 ivpu_hw_btrs.o \11 ivpu_hw_btrs.o \
11 ivpu_hw_ip.o \12 ivpu_hw_ip.o \
drivers/accel/ivpu/ivpu_debugfs.c+14−24
@@ -398,35 +398,25 @@ static int dct_active_set(void *data, u64 active_percent) · 397 unmodified lines
398 398
399DEFINE_DEBUGFS_ATTRIBUTE(ivpu_dct_fops, dct_active_get, dct_active_set, "%llu\n");399DEFINE_DEBUGFS_ATTRIBUTE(ivpu_dct_fops, dct_active_get, dct_active_set, "%llu\n");
400 400
401static void print_priority_band(struct seq_file *s, struct ivpu_hw_info *hw,
402 int band, const char *name)
403{
404 seq_printf(s, "%-9s: grace_period %9u process_grace_period %9u process_quantum %9u\n",
405 name,
406 hw->hws.grace_period[band],
407 hw->hws.process_grace_period[band],
408 hw->hws.process_quantum[band]);
409}
410
401static int priority_bands_show(struct seq_file *s, void *v)411static int priority_bands_show(struct seq_file *s, void *v)
402{412{
403 struct ivpu_device *vdev = s->private;413 struct ivpu_device *vdev = s->private;
404 struct ivpu_hw_info *hw = vdev->hw;414 struct ivpu_hw_info *hw = vdev->hw;
405 415
406 for (int band = VPU_JOB_SCHEDULING_PRIORITY_BAND_IDLE;416 print_priority_band(s, hw, VPU_JOB_SCHEDULING_PRIORITY_BAND_IDLE, "Idle");
407 band < VPU_JOB_SCHEDULING_PRIORITY_BAND_COUNT; band++) {417 print_priority_band(s, hw, VPU_JOB_SCHEDULING_PRIORITY_BAND_NORMAL, "Normal");
408 switch (band) {418 print_priority_band(s, hw, VPU_JOB_SCHEDULING_PRIORITY_BAND_FOCUS, "Focus");
409 case VPU_JOB_SCHEDULING_PRIORITY_BAND_IDLE:419 print_priority_band(s, hw, VPU_JOB_SCHEDULING_PRIORITY_BAND_REALTIME, "Realtime");
410 seq_puts(s, "Idle: ");
411 break;
412
413 case VPU_JOB_SCHEDULING_PRIORITY_BAND_NORMAL:
414 seq_puts(s, "Normal: ");
415 break;
416
417 case VPU_JOB_SCHEDULING_PRIORITY_BAND_FOCUS:
418 seq_puts(s, "Focus: ");
419 break;
420
421 case VPU_JOB_SCHEDULING_PRIORITY_BAND_REALTIME:
422 seq_puts(s, "Realtime: ");
423 break;
424 }
425
426 seq_printf(s, "grace_period %9u process_grace_period %9u process_quantum %9u\n",
427 hw->hws.grace_period[band], hw->hws.process_grace_period[band],
428 hw->hws.process_quantum[band]);
429 }
430 420
431 return 0;421 return 0;
432}422}
drivers/accel/ivpu/ivpu_drv.c+15−3
@@ -57,7 +57,7 @@ MODULE_PARM_DESC(pll_max_ratio, "Maximum PLL ratio used to set NPU frequency"); · 56 unmodified lines
57 57
58int ivpu_sched_mode = IVPU_SCHED_MODE_AUTO;58int ivpu_sched_mode = IVPU_SCHED_MODE_AUTO;
59module_param_named(sched_mode, ivpu_sched_mode, int, 0444);59module_param_named(sched_mode, ivpu_sched_mode, int, 0444);
60MODULE_PARM_DESC(sched_mode, "Scheduler mode: -1 - Use default scheduler, 0 - Use OS scheduler, 1 - Use HW scheduler");60MODULE_PARM_DESC(sched_mode, "Scheduler mode: -1 - Use default scheduler, 0 - Use OS scheduler (supported on 27XX - 50XX), 1 - Use HW scheduler");
61 61
62bool ivpu_disable_mmu_cont_pages;62bool ivpu_disable_mmu_cont_pages;
63module_param_named(disable_mmu_cont_pages, ivpu_disable_mmu_cont_pages, bool, 0444);63module_param_named(disable_mmu_cont_pages, ivpu_disable_mmu_cont_pages, bool, 0444);
@@ -134,6 +134,8 @@ bool ivpu_is_capable(struct ivpu_device *vdev, u32 capability) · 70 unmodified lines
134 return true;134 return true;
135 case DRM_IVPU_CAP_DMA_MEMORY_RANGE:135 case DRM_IVPU_CAP_DMA_MEMORY_RANGE:
136 return true;136 return true;
137 case DRM_IVPU_CAP_BO_CREATE_FROM_USERPTR:
138 return true;
137 case DRM_IVPU_CAP_MANAGE_CMDQ:139 case DRM_IVPU_CAP_MANAGE_CMDQ:
138 return vdev->fw->sched_mode == VPU_SCHEDULING_MODE_HW;140 return vdev->fw->sched_mode == VPU_SCHEDULING_MODE_HW;
139 default:141 default:
@@ -200,6 +202,9 @@ static int ivpu_get_param_ioctl(struct drm_device *dev, void *data, struct drm_f · 60 unmodified lines
200 case DRM_IVPU_PARAM_CAPABILITIES:202 case DRM_IVPU_PARAM_CAPABILITIES:
201 args->value = ivpu_is_capable(vdev, args->index);203 args->value = ivpu_is_capable(vdev, args->index);
202 break;204 break;
205 case DRM_IVPU_PARAM_PREEMPT_BUFFER_SIZE:
206 args->value = ivpu_fw_preempt_buf_size(vdev);
207 break;
203 default:208 default:
204 ret = -EINVAL;209 ret = -EINVAL;
205 break;210 break;
@@ -310,6 +315,7 @@ static const struct drm_ioctl_desc ivpu_drm_ioctls[] = { · 104 unmodified lines
310 DRM_IOCTL_DEF_DRV(IVPU_CMDQ_CREATE, ivpu_cmdq_create_ioctl, 0),315 DRM_IOCTL_DEF_DRV(IVPU_CMDQ_CREATE, ivpu_cmdq_create_ioctl, 0),
311 DRM_IOCTL_DEF_DRV(IVPU_CMDQ_DESTROY, ivpu_cmdq_destroy_ioctl, 0),316 DRM_IOCTL_DEF_DRV(IVPU_CMDQ_DESTROY, ivpu_cmdq_destroy_ioctl, 0),
312 DRM_IOCTL_DEF_DRV(IVPU_CMDQ_SUBMIT, ivpu_cmdq_submit_ioctl, 0),317 DRM_IOCTL_DEF_DRV(IVPU_CMDQ_SUBMIT, ivpu_cmdq_submit_ioctl, 0),
318 DRM_IOCTL_DEF_DRV(IVPU_BO_CREATE_FROM_USERPTR, ivpu_bo_create_from_userptr_ioctl, 0),
313};319};
314 320
315static int ivpu_wait_for_ready(struct ivpu_device *vdev)321static int ivpu_wait_for_ready(struct ivpu_device *vdev)
@@ -377,8 +383,7 @@ int ivpu_boot(struct ivpu_device *vdev) · 61 unmodified lines
377 drm_WARN_ON(&vdev->drm, atomic_read(&vdev->job_timeout_counter));383 drm_WARN_ON(&vdev->drm, atomic_read(&vdev->job_timeout_counter));
378 drm_WARN_ON(&vdev->drm, !xa_empty(&vdev->submitted_jobs_xa));384 drm_WARN_ON(&vdev->drm, !xa_empty(&vdev->submitted_jobs_xa));
379 385
380 /* Update boot params located at first 4KB of FW memory */386 ivpu_fw_boot_params_setup(vdev, ivpu_bo_vaddr(vdev->fw->mem_bp));
381 ivpu_fw_boot_params_setup(vdev, ivpu_bo_vaddr(vdev->fw->mem));
382 387
383 ret = ivpu_hw_boot_fw(vdev);388 ret = ivpu_hw_boot_fw(vdev);
384 if (ret) {389 if (ret) {
@@ -450,6 +455,9 @@ int ivpu_shutdown(struct ivpu_device *vdev) · 65 unmodified lines
450static const struct file_operations ivpu_fops = {455static const struct file_operations ivpu_fops = {
451 .owner = THIS_MODULE,456 .owner = THIS_MODULE,
452 DRM_ACCEL_FOPS,457 DRM_ACCEL_FOPS,
458#ifdef CONFIG_PROC_FS
459 .show_fdinfo = drm_show_fdinfo,
460#endif
453};461};
454 462
455static const struct drm_driver driver = {463static const struct drm_driver driver = {
@@ -464,6 +472,9 @@ static const struct drm_driver driver = { · 8 unmodified lines
464 .ioctls = ivpu_drm_ioctls,472 .ioctls = ivpu_drm_ioctls,
465 .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls),473 .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls),
466 .fops = &ivpu_fops,474 .fops = &ivpu_fops,
475#ifdef CONFIG_PROC_FS
476 .show_fdinfo = drm_show_memory_stats,
477#endif
467 478
468 .name = DRIVER_NAME,479 .name = DRIVER_NAME,
469 .desc = DRIVER_DESC,480 .desc = DRIVER_DESC,
@@ -705,6 +716,7 @@ static struct pci_device_id ivpu_pci_ids[] = { · 235 unmodified lines
705 { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_LNL) },716 { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_LNL) },
706 { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PTL_P) },717 { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PTL_P) },
707 { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_WCL) },718 { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_WCL) },
719 { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_NVL) },
708 { }720 { }
709};721};
710MODULE_DEVICE_TABLE(pci, ivpu_pci_ids);722MODULE_DEVICE_TABLE(pci, ivpu_pci_ids);
drivers/accel/ivpu/ivpu_drv.h+5−0
@@ -27,6 +27,7 @@ · 26 unmodified lines
27#define PCI_DEVICE_ID_LNL 0x643e27#define PCI_DEVICE_ID_LNL 0x643e
28#define PCI_DEVICE_ID_PTL_P 0xb03e28#define PCI_DEVICE_ID_PTL_P 0xb03e
29#define PCI_DEVICE_ID_WCL 0xfd3e29#define PCI_DEVICE_ID_WCL 0xfd3e
30#define PCI_DEVICE_ID_NVL 0xd71d
30 31
31#define IVPU_HW_IP_37XX 3732#define IVPU_HW_IP_37XX 37
32#define IVPU_HW_IP_40XX 4033#define IVPU_HW_IP_40XX 40
@@ -78,6 +79,7 @@ · 45 unmodified lines
78#define IVPU_DBG_KREF BIT(11)79#define IVPU_DBG_KREF BIT(11)
79#define IVPU_DBG_RPM BIT(12)80#define IVPU_DBG_RPM BIT(12)
80#define IVPU_DBG_MMU_MAP BIT(13)81#define IVPU_DBG_MMU_MAP BIT(13)
82#define IVPU_DBG_IOCTL BIT(14)
81 83
82#define ivpu_err(vdev, fmt, ...) \84#define ivpu_err(vdev, fmt, ...) \
83 drm_err(&(vdev)->drm, "%s(): " fmt, __func__, ##__VA_ARGS__)85 drm_err(&(vdev)->drm, "%s(): " fmt, __func__, ##__VA_ARGS__)
@@ -245,6 +247,8 @@ static inline int ivpu_hw_ip_gen(struct ivpu_device *vdev) · 161 unmodified lines
245 case PCI_DEVICE_ID_PTL_P:247 case PCI_DEVICE_ID_PTL_P:
246 case PCI_DEVICE_ID_WCL:248 case PCI_DEVICE_ID_WCL:
247 return IVPU_HW_IP_50XX;249 return IVPU_HW_IP_50XX;
250 case PCI_DEVICE_ID_NVL:
251 return IVPU_HW_IP_60XX;
248 default:252 default:
249 dump_stack();253 dump_stack();
250 ivpu_err(vdev, "Unknown NPU IP generation\n");254 ivpu_err(vdev, "Unknown NPU IP generation\n");
@@ -261,6 +265,7 @@ static inline int ivpu_hw_btrs_gen(struct ivpu_device *vdev) · 10 unmodified lines
261 case PCI_DEVICE_ID_LNL:265 case PCI_DEVICE_ID_LNL:
262 case PCI_DEVICE_ID_PTL_P:266 case PCI_DEVICE_ID_PTL_P:
263 case PCI_DEVICE_ID_WCL:267 case PCI_DEVICE_ID_WCL:
268 case PCI_DEVICE_ID_NVL:
264 return IVPU_HW_BTRS_LNL;269 return IVPU_HW_BTRS_LNL;
265 default:270 default:
266 dump_stack();271 dump_stack();
drivers/accel/ivpu/ivpu_fw.c+151−78
@@ -17,15 +17,10 @@ · 16 unmodified lines
17#include "ivpu_ipc.h"17#include "ivpu_ipc.h"
18#include "ivpu_pm.h"18#include "ivpu_pm.h"
19 19
20#define FW_GLOBAL_MEM_START (2ull * SZ_1G)
21#define FW_GLOBAL_MEM_END (3ull * SZ_1G)
22#define FW_SHARED_MEM_SIZE SZ_256M /* Must be aligned to FW_SHARED_MEM_ALIGNMENT */
23#define FW_SHARED_MEM_ALIGNMENT SZ_128K /* VPU MTRR limitation */
24#define FW_RUNTIME_MAX_SIZE SZ_512M
25#define FW_SHAVE_NN_MAX_SIZE SZ_2M20#define FW_SHAVE_NN_MAX_SIZE SZ_2M
26#define FW_RUNTIME_MIN_ADDR (FW_GLOBAL_MEM_START)
27#define FW_RUNTIME_MAX_ADDR (FW_GLOBAL_MEM_END - FW_SHARED_MEM_SIZE)
28#define FW_FILE_IMAGE_OFFSET (VPU_FW_HEADER_SIZE + FW_VERSION_HEADER_SIZE)21#define FW_FILE_IMAGE_OFFSET (VPU_FW_HEADER_SIZE + FW_VERSION_HEADER_SIZE)
22#define FW_PREEMPT_BUF_MIN_SIZE SZ_4K
23#define FW_PREEMPT_BUF_MAX_SIZE SZ_32M
29 24
30#define WATCHDOG_MSS_REDIRECT 3225#define WATCHDOG_MSS_REDIRECT 32
31#define WATCHDOG_NCE_REDIRECT 3326#define WATCHDOG_NCE_REDIRECT 33
@@ -61,12 +56,14 @@ static struct { · 29 unmodified lines
61 { IVPU_HW_IP_40XX, "intel/vpu/vpu_40xx_v0.0.bin" },56 { IVPU_HW_IP_40XX, "intel/vpu/vpu_40xx_v0.0.bin" },
62 { IVPU_HW_IP_50XX, "intel/vpu/vpu_50xx_v1.bin" },57 { IVPU_HW_IP_50XX, "intel/vpu/vpu_50xx_v1.bin" },
63 { IVPU_HW_IP_50XX, "intel/vpu/vpu_50xx_v0.0.bin" },58 { IVPU_HW_IP_50XX, "intel/vpu/vpu_50xx_v0.0.bin" },
59 { IVPU_HW_IP_60XX, "intel/vpu/vpu_60xx_v1.bin" },
64};60};
65 61
66/* Production fw_names from the table above */62/* Production fw_names from the table above */
67MODULE_FIRMWARE("intel/vpu/vpu_37xx_v1.bin");63MODULE_FIRMWARE("intel/vpu/vpu_37xx_v1.bin");
68MODULE_FIRMWARE("intel/vpu/vpu_40xx_v1.bin");64MODULE_FIRMWARE("intel/vpu/vpu_40xx_v1.bin");
69MODULE_FIRMWARE("intel/vpu/vpu_50xx_v1.bin");65MODULE_FIRMWARE("intel/vpu/vpu_50xx_v1.bin");
66MODULE_FIRMWARE("intel/vpu/vpu_60xx_v1.bin");
70 67
71static int ivpu_fw_request(struct ivpu_device *vdev)68static int ivpu_fw_request(struct ivpu_device *vdev)
72{69{
@@ -131,9 +128,14 @@ ivpu_fw_check_api_ver_lt(struct ivpu_device *vdev, const struct vpu_firmware_hea · 58 unmodified lines
131 return false;128 return false;
132}129}
133 130
134static bool is_within_range(u64 addr, size_t size, u64 range_start, size_t range_size)131bool ivpu_is_within_range(u64 addr, size_t size, struct ivpu_addr_range *range)
135{132{
136 if (addr < range_start || addr + size > range_start + range_size)133 u64 addr_end;
134
135 if (!range || check_add_overflow(addr, size, &addr_end))
136 return false;
137
138 if (addr < range->start || addr_end > range->end)
137 return false;139 return false;
138 140
139 return true;141 return true;
@@ -142,6 +144,12 @@ static bool is_within_range(u64 addr, size_t size, u64 range_start, size_t range · 2 unmodified lines
142static u32144static u32
143ivpu_fw_sched_mode_select(struct ivpu_device *vdev, const struct vpu_firmware_header *fw_hdr)145ivpu_fw_sched_mode_select(struct ivpu_device *vdev, const struct vpu_firmware_header *fw_hdr)
144{146{
147 if (ivpu_hw_ip_gen(vdev) >= IVPU_HW_IP_60XX &&
148 ivpu_sched_mode == VPU_SCHEDULING_MODE_OS) {
149 ivpu_warn(vdev, "OS sched mode is not supported, using HW mode\n");
150 return VPU_SCHEDULING_MODE_HW;
151 }
152
145 if (ivpu_sched_mode != IVPU_SCHED_MODE_AUTO)153 if (ivpu_sched_mode != IVPU_SCHED_MODE_AUTO)
146 return ivpu_sched_mode;154 return ivpu_sched_mode;
147 155
@@ -151,11 +159,56 @@ ivpu_fw_sched_mode_select(struct ivpu_device *vdev, const struct vpu_firmware_he · 3 unmodified lines
151 return VPU_SCHEDULING_MODE_HW;159 return VPU_SCHEDULING_MODE_HW;
152}160}
153 161
162static void
163ivpu_preemption_config_parse(struct ivpu_device *vdev, const struct vpu_firmware_header *fw_hdr)
164{
165 struct ivpu_fw_info *fw = vdev->fw;
166 u32 primary_preempt_buf_size, secondary_preempt_buf_size;
167
168 if (fw_hdr->preemption_buffer_1_max_size)
169 primary_preempt_buf_size = fw_hdr->preemption_buffer_1_max_size;
170 else
171 primary_preempt_buf_size = fw_hdr->preemption_buffer_1_size;
172
173 if (fw_hdr->preemption_buffer_2_max_size)
174 secondary_preempt_buf_size = fw_hdr->preemption_buffer_2_max_size;
175 else
176 secondary_preempt_buf_size = fw_hdr->preemption_buffer_2_size;
177
178 ivpu_dbg(vdev, FW_BOOT, "Preemption buffer size, primary: %u, secondary: %u\n",
179 primary_preempt_buf_size, secondary_preempt_buf_size);
180
181 if (primary_preempt_buf_size < FW_PREEMPT_BUF_MIN_SIZE ||
182 secondary_preempt_buf_size < FW_PREEMPT_BUF_MIN_SIZE) {
183 ivpu_warn(vdev, "Preemption buffers size too small\n");
184 return;
185 }
186
187 if (primary_preempt_buf_size > FW_PREEMPT_BUF_MAX_SIZE ||
188 secondary_preempt_buf_size > FW_PREEMPT_BUF_MAX_SIZE) {
189 ivpu_warn(vdev, "Preemption buffers size too big\n");
190 return;
191 }
192
193 if (fw->sched_mode != VPU_SCHEDULING_MODE_HW)
194 return;
195
196 if (ivpu_test_mode & IVPU_TEST_MODE_MIP_DISABLE)
197 return;
198
199 vdev->fw->primary_preempt_buf_size = ALIGN(primary_preempt_buf_size, PAGE_SIZE);
200 vdev->fw->secondary_preempt_buf_size = ALIGN(secondary_preempt_buf_size, PAGE_SIZE);
201}
202
154static int ivpu_fw_parse(struct ivpu_device *vdev)203static int ivpu_fw_parse(struct ivpu_device *vdev)
155{204{
156 struct ivpu_fw_info *fw = vdev->fw;205 struct ivpu_fw_info *fw = vdev->fw;
157 const struct vpu_firmware_header *fw_hdr = (const void *)fw->file->data;206 const struct vpu_firmware_header *fw_hdr = (const void *)fw->file->data;
158 u64 runtime_addr, image_load_addr, runtime_size, image_size;207 struct ivpu_addr_range fw_image_range;
208 u64 boot_params_addr, boot_params_size;
209 u64 fw_version_addr, fw_version_size;
210 u64 runtime_addr, runtime_size;
211 u64 image_load_addr, image_size;
159 212
160 if (fw->file->size <= FW_FILE_IMAGE_OFFSET) {213 if (fw->file->size <= FW_FILE_IMAGE_OFFSET) {
161 ivpu_err(vdev, "Firmware file is too small: %zu\n", fw->file->size);214 ivpu_err(vdev, "Firmware file is too small: %zu\n", fw->file->size);
@@ -167,18 +220,37 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 5 unmodified lines
167 return -EINVAL;220 return -EINVAL;
168 }221 }
169 222
170 runtime_addr = fw_hdr->boot_params_load_address;223 boot_params_addr = fw_hdr->boot_params_load_address;
171 runtime_size = fw_hdr->runtime_size;224 boot_params_size = SZ_4K;
172 image_load_addr = fw_hdr->image_load_address;
173 image_size = fw_hdr->image_size;
174 225
175 if (runtime_addr < FW_RUNTIME_MIN_ADDR || runtime_addr > FW_RUNTIME_MAX_ADDR) {226 if (!ivpu_is_within_range(boot_params_addr, boot_params_size, &vdev->hw->ranges.runtime)) {
176 ivpu_err(vdev, "Invalid firmware runtime address: 0x%llx\n", runtime_addr);227 ivpu_err(vdev, "Invalid boot params address: 0x%llx\n", boot_params_addr);
177 return -EINVAL;228 return -EINVAL;
178 }229 }
179 230
180 if (runtime_size < fw->file->size || runtime_size > FW_RUNTIME_MAX_SIZE) {231 fw_version_addr = fw_hdr->firmware_version_load_address;
181 ivpu_err(vdev, "Invalid firmware runtime size: %llu\n", runtime_size);232 fw_version_size = ALIGN(fw_hdr->firmware_version_size, SZ_4K);
233
234 if (fw_version_size != SZ_4K) {
235 ivpu_err(vdev, "Invalid firmware version size: %u\n",
236 fw_hdr->firmware_version_size);
237 return -EINVAL;
238 }
239
240 if (!ivpu_is_within_range(fw_version_addr, fw_version_size, &vdev->hw->ranges.runtime)) {
241 ivpu_err(vdev, "Invalid firmware version address: 0x%llx\n", fw_version_addr);
242 return -EINVAL;
243 }
244
245 runtime_addr = fw_hdr->image_load_address;
246 runtime_size = fw_hdr->runtime_size - boot_params_size - fw_version_size;
247
248 image_load_addr = fw_hdr->image_load_address;
249 image_size = fw_hdr->image_size;
250
251 if (!ivpu_is_within_range(runtime_addr, runtime_size, &vdev->hw->ranges.runtime)) {
252 ivpu_err(vdev, "Invalid firmware runtime address: 0x%llx and size %llu\n",
253 runtime_addr, runtime_size);
182 return -EINVAL;254 return -EINVAL;
183 }255 }
184 256
@@ -187,23 +259,25 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 2 unmodified lines
187 return -EINVAL;259 return -EINVAL;
188 }260 }
189 261
190 if (image_load_addr < runtime_addr ||262 if (!ivpu_is_within_range(image_load_addr, image_size, &vdev->hw->ranges.runtime)) {
191 image_load_addr + image_size > runtime_addr + runtime_size) {263 ivpu_err(vdev, "Invalid firmware load address: 0x%llx and size %llu\n",
192 ivpu_err(vdev, "Invalid firmware load address size: 0x%llx and size %llu\n",
193 image_load_addr, image_size);264 image_load_addr, image_size);
194 return -EINVAL;265 return -EINVAL;
195 }266 }
196 267
197 if (fw_hdr->shave_nn_fw_size > FW_SHAVE_NN_MAX_SIZE) {268 if (ivpu_hw_range_init(vdev, &fw_image_range, image_load_addr, image_size))
198 ivpu_err(vdev, "SHAVE NN firmware is too big: %u\n", fw_hdr->shave_nn_fw_size);
199 return -EINVAL;269 return -EINVAL;
200 }
201 270
202 if (fw_hdr->entry_point < image_load_addr ||271 if (!ivpu_is_within_range(fw_hdr->entry_point, SZ_4K, &fw_image_range)) {
203 fw_hdr->entry_point >= image_load_addr + image_size) {
204 ivpu_err(vdev, "Invalid entry point: 0x%llx\n", fw_hdr->entry_point);272 ivpu_err(vdev, "Invalid entry point: 0x%llx\n", fw_hdr->entry_point);
205 return -EINVAL;273 return -EINVAL;
206 }274 }
275
276 if (fw_hdr->shave_nn_fw_size > FW_SHAVE_NN_MAX_SIZE) {
277 ivpu_err(vdev, "SHAVE NN firmware is too big: %u\n", fw_hdr->shave_nn_fw_size);
278 return -EINVAL;
279 }
280
207 ivpu_dbg(vdev, FW_BOOT, "Header version: 0x%x, format 0x%x\n",281 ivpu_dbg(vdev, FW_BOOT, "Header version: 0x%x, format 0x%x\n",
208 fw_hdr->header_version, fw_hdr->image_format);282 fw_hdr->header_version, fw_hdr->image_format);
209 283
@@ -217,6 +291,10 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 7 unmodified lines
217 if (IVPU_FW_CHECK_API_COMPAT(vdev, fw_hdr, JSM, 3))291 if (IVPU_FW_CHECK_API_COMPAT(vdev, fw_hdr, JSM, 3))
218 return -EINVAL;292 return -EINVAL;
219 293
294 fw->boot_params_addr = boot_params_addr;
295 fw->boot_params_size = boot_params_size;
296 fw->fw_version_addr = fw_version_addr;
297 fw->fw_version_size = fw_version_size;
220 fw->runtime_addr = runtime_addr;298 fw->runtime_addr = runtime_addr;
221 fw->runtime_size = runtime_size;299 fw->runtime_size = runtime_size;
222 fw->image_load_offset = image_load_addr - runtime_addr;300 fw->image_load_offset = image_load_addr - runtime_addr;
@@ -235,22 +313,13 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 12 unmodified lines
235 fw->sched_mode = ivpu_fw_sched_mode_select(vdev, fw_hdr);313 fw->sched_mode = ivpu_fw_sched_mode_select(vdev, fw_hdr);
236 ivpu_info(vdev, "Scheduler mode: %s\n", fw->sched_mode ? "HW" : "OS");314 ivpu_info(vdev, "Scheduler mode: %s\n", fw->sched_mode ? "HW" : "OS");
237 315
238 if (fw_hdr->preemption_buffer_1_max_size)316 ivpu_preemption_config_parse(vdev, fw_hdr);
239 fw->primary_preempt_buf_size = fw_hdr->preemption_buffer_1_max_size;317 ivpu_dbg(vdev, FW_BOOT, "Mid-inference preemption %s supported\n",
240 else318 ivpu_fw_preempt_buf_size(vdev) ? "is" : "is not");
241 fw->primary_preempt_buf_size = fw_hdr->preemption_buffer_1_size;
242 319
243 if (fw_hdr->preemption_buffer_2_max_size)320 if (fw_hdr->ro_section_start_address &&
244 fw->secondary_preempt_buf_size = fw_hdr->preemption_buffer_2_max_size;321 !ivpu_is_within_range(fw_hdr->ro_section_start_address, fw_hdr->ro_section_size,
245 else322 &fw_image_range)) {
246 fw->secondary_preempt_buf_size = fw_hdr->preemption_buffer_2_size;
247 ivpu_dbg(vdev, FW_BOOT, "Preemption buffer sizes: primary %u, secondary %u\n",
248 fw->primary_preempt_buf_size, fw->secondary_preempt_buf_size);
249
250 if (fw_hdr->ro_section_start_address && !is_within_range(fw_hdr->ro_section_start_address,
251 fw_hdr->ro_section_size,
252 fw_hdr->image_load_address,
253 fw_hdr->image_size)) {
254 ivpu_err(vdev, "Invalid read-only section: start address 0x%llx, size %u\n",323 ivpu_err(vdev, "Invalid read-only section: start address 0x%llx, size %u\n",
255 fw_hdr->ro_section_start_address, fw_hdr->ro_section_size);324 fw_hdr->ro_section_start_address, fw_hdr->ro_section_size);
256 return -EINVAL;325 return -EINVAL;
@@ -259,12 +328,18 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 2 unmodified lines
259 fw->read_only_addr = fw_hdr->ro_section_start_address;328 fw->read_only_addr = fw_hdr->ro_section_start_address;
260 fw->read_only_size = fw_hdr->ro_section_size;329 fw->read_only_size = fw_hdr->ro_section_size;
261 330
262 ivpu_dbg(vdev, FW_BOOT, "Size: file %lu image %u runtime %u shavenn %u\n",331 ivpu_dbg(vdev, FW_BOOT, "Boot params: address 0x%llx, size %llu\n",
263 fw->file->size, fw->image_size, fw->runtime_size, fw->shave_nn_size);332 fw->boot_params_addr, fw->boot_params_size);
264 ivpu_dbg(vdev, FW_BOOT, "Address: runtime 0x%llx, load 0x%llx, entry point 0x%llx\n",333 ivpu_dbg(vdev, FW_BOOT, "FW version: address 0x%llx, size %llu\n",
265 fw->runtime_addr, image_load_addr, fw->entry_point);334 fw->fw_version_addr, fw->fw_version_size);
335 ivpu_dbg(vdev, FW_BOOT, "Runtime: address 0x%llx, size %u\n",
336 fw->runtime_addr, fw->runtime_size);
337 ivpu_dbg(vdev, FW_BOOT, "Image load offset: 0x%llx, size %u\n",
338 fw->image_load_offset, fw->image_size);
266 ivpu_dbg(vdev, FW_BOOT, "Read-only section: address 0x%llx, size %u\n",339 ivpu_dbg(vdev, FW_BOOT, "Read-only section: address 0x%llx, size %u\n",
267 fw->read_only_addr, fw->read_only_size);340 fw->read_only_addr, fw->read_only_size);
341 ivpu_dbg(vdev, FW_BOOT, "FW entry point: 0x%llx\n", fw->entry_point);
342 ivpu_dbg(vdev, FW_BOOT, "SHAVE NN size: %u\n", fw->shave_nn_size);
268 343
269 return 0;344 return 0;
270}345}
@@ -291,39 +366,33 @@ ivpu_fw_init_wa(struct ivpu_device *vdev) · 20 unmodified lines
291 IVPU_PRINT_WA(disable_d0i3_msg);366 IVPU_PRINT_WA(disable_d0i3_msg);
292}367}
293 368
294static int ivpu_fw_update_global_range(struct ivpu_device *vdev)
295{
296 struct ivpu_fw_info *fw = vdev->fw;
297 u64 start = ALIGN(fw->runtime_addr + fw->runtime_size, FW_SHARED_MEM_ALIGNMENT);
298 u64 size = FW_SHARED_MEM_SIZE;
299
300 if (start + size > FW_GLOBAL_MEM_END) {
301 ivpu_err(vdev, "No space for shared region, start %lld, size %lld\n", start, size);
302 return -EINVAL;
303 }
304
305 ivpu_hw_range_init(&vdev->hw->ranges.global, start, size);
306 return 0;
307}
308
309static int ivpu_fw_mem_init(struct ivpu_device *vdev)369static int ivpu_fw_mem_init(struct ivpu_device *vdev)
310{370{
311 struct ivpu_fw_info *fw = vdev->fw;371 struct ivpu_fw_info *fw = vdev->fw;
312 struct ivpu_addr_range fw_range;
313 int log_verb_size;372 int log_verb_size;
314 int ret;373 int ret;
315 374
316 ret = ivpu_fw_update_global_range(vdev);375 fw->mem_bp = ivpu_bo_create_runtime(vdev, fw->boot_params_addr, fw->boot_params_size,
317 if (ret)376 DRM_IVPU_BO_WC | DRM_IVPU_BO_MAPPABLE);
318 return ret;377 if (!fw->mem_bp) {
378 ivpu_err(vdev, "Failed to create firmware boot params memory buffer\n");
379 return -ENOMEM;
380 }
319 381
320 fw_range.start = fw->runtime_addr;382 fw->mem_fw_ver = ivpu_bo_create_runtime(vdev, fw->fw_version_addr, fw->fw_version_size,
321 fw_range.end = fw->runtime_addr + fw->runtime_size;383 DRM_IVPU_BO_WC | DRM_IVPU_BO_MAPPABLE);
322 fw->mem = ivpu_bo_create(vdev, &vdev->gctx, &fw_range, fw->runtime_size,384 if (!fw->mem_fw_ver) {
323 DRM_IVPU_BO_WC | DRM_IVPU_BO_MAPPABLE);385 ivpu_err(vdev, "Failed to create firmware version memory buffer\n");
386 ret = -ENOMEM;
387 goto err_free_bp;
388 }
389
390 fw->mem = ivpu_bo_create_runtime(vdev, fw->runtime_addr, fw->runtime_size,
391 DRM_IVPU_BO_WC | DRM_IVPU_BO_MAPPABLE);
324 if (!fw->mem) {392 if (!fw->mem) {
325 ivpu_err(vdev, "Failed to create firmware runtime memory buffer\n");393 ivpu_err(vdev, "Failed to create firmware runtime memory buffer\n");
326 return -ENOMEM;394 ret = -ENOMEM;
395 goto err_free_fw_ver;
327 }396 }
328 397
329 ret = ivpu_mmu_context_set_pages_ro(vdev, &vdev->gctx, fw->read_only_addr,398 ret = ivpu_mmu_context_set_pages_ro(vdev, &vdev->gctx, fw->read_only_addr,
@@ -372,6 +441,10 @@ err_free_log_crit: · 42 unmodified lines
372 ivpu_bo_free(fw->mem_log_crit);441 ivpu_bo_free(fw->mem_log_crit);
373err_free_fw_mem:442err_free_fw_mem:
374 ivpu_bo_free(fw->mem);443 ivpu_bo_free(fw->mem);
444err_free_fw_ver:
445 ivpu_bo_free(fw->mem_fw_ver);
446err_free_bp:
447 ivpu_bo_free(fw->mem_bp);
375 return ret;448 return ret;
376}449}
377 450
@@ -387,10 +460,14 @@ static void ivpu_fw_mem_fini(struct ivpu_device *vdev) · 9 unmodified lines
387 ivpu_bo_free(fw->mem_log_verb);460 ivpu_bo_free(fw->mem_log_verb);
388 ivpu_bo_free(fw->mem_log_crit);461 ivpu_bo_free(fw->mem_log_crit);
389 ivpu_bo_free(fw->mem);462 ivpu_bo_free(fw->mem);
463 ivpu_bo_free(fw->mem_fw_ver);
464 ivpu_bo_free(fw->mem_bp);
390 465
391 fw->mem_log_verb = NULL;466 fw->mem_log_verb = NULL;
392 fw->mem_log_crit = NULL;467 fw->mem_log_crit = NULL;
393 fw->mem = NULL;468 fw->mem = NULL;
469 fw->mem_fw_ver = NULL;
470 fw->mem_bp = NULL;
394}471}
395 472
396int ivpu_fw_init(struct ivpu_device *vdev)473int ivpu_fw_init(struct ivpu_device *vdev)
@@ -483,11 +560,6 @@ static void ivpu_fw_boot_params_print(struct ivpu_device *vdev, struct vpu_boot_ · 86 unmodified lines
483 ivpu_dbg(vdev, FW_BOOT, "boot_params.cache_defaults[VPU_BOOT_L2_CACHE_CFG_NN].cfg = 0x%x\n",560 ivpu_dbg(vdev, FW_BOOT, "boot_params.cache_defaults[VPU_BOOT_L2_CACHE_CFG_NN].cfg = 0x%x\n",
484 boot_params->cache_defaults[VPU_BOOT_L2_CACHE_CFG_NN].cfg);561 boot_params->cache_defaults[VPU_BOOT_L2_CACHE_CFG_NN].cfg);
485 562
486 ivpu_dbg(vdev, FW_BOOT, "boot_params.global_memory_allocator_base = 0x%llx\n",
487 boot_params->global_memory_allocator_base);
488 ivpu_dbg(vdev, FW_BOOT, "boot_params.global_memory_allocator_size = 0x%x\n",
489 boot_params->global_memory_allocator_size);
490
491 ivpu_dbg(vdev, FW_BOOT, "boot_params.shave_nn_fw_base = 0x%llx\n",563 ivpu_dbg(vdev, FW_BOOT, "boot_params.shave_nn_fw_base = 0x%llx\n",
492 boot_params->shave_nn_fw_base);564 boot_params->shave_nn_fw_base);
493 565
@@ -495,10 +567,6 @@ static void ivpu_fw_boot_params_print(struct ivpu_device *vdev, struct vpu_boot_ · 1 unmodified line
495 boot_params->watchdog_irq_mss);567 boot_params->watchdog_irq_mss);
496 ivpu_dbg(vdev, FW_BOOT, "boot_params.watchdog_irq_nce = 0x%x\n",568 ivpu_dbg(vdev, FW_BOOT, "boot_params.watchdog_irq_nce = 0x%x\n",
497 boot_params->watchdog_irq_nce);569 boot_params->watchdog_irq_nce);
498 ivpu_dbg(vdev, FW_BOOT, "boot_params.host_to_vpu_irq = 0x%x\n",
499 boot_params->host_to_vpu_irq);
500 ivpu_dbg(vdev, FW_BOOT, "boot_params.job_done_irq = 0x%x\n",
501 boot_params->job_done_irq);
502 570
503 ivpu_dbg(vdev, FW_BOOT, "boot_params.host_version_id = 0x%x\n",571 ivpu_dbg(vdev, FW_BOOT, "boot_params.host_version_id = 0x%x\n",
504 boot_params->host_version_id);572 boot_params->host_version_id);
@@ -546,6 +614,8 @@ static void ivpu_fw_boot_params_print(struct ivpu_device *vdev, struct vpu_boot_ · 41 unmodified lines
546 boot_params->system_time_us);614 boot_params->system_time_us);
547 ivpu_dbg(vdev, FW_BOOT, "boot_params.power_profile = 0x%x\n",615 ivpu_dbg(vdev, FW_BOOT, "boot_params.power_profile = 0x%x\n",
548 boot_params->power_profile);616 boot_params->power_profile);
617 ivpu_dbg(vdev, FW_BOOT, "boot_params.vpu_uses_ecc_mca_signal = 0x%x\n",
618 boot_params->vpu_uses_ecc_mca_signal);
549}619}
550 620
551void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params *boot_params)621void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params *boot_params)
@@ -572,6 +642,7 @@ void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params · 20 unmodified lines
572 return;642 return;
573 }643 }
574 644
645 memset(boot_params, 0, sizeof(*boot_params));
575 vdev->pm->is_warmboot = false;646 vdev->pm->is_warmboot = false;
576 647
577 boot_params->magic = VPU_BOOT_PARAMS_MAGIC;648 boot_params->magic = VPU_BOOT_PARAMS_MAGIC;
@@ -647,6 +718,8 @@ void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params · 69 unmodified lines
647 boot_params->d0i3_entry_vpu_ts = 0;718 boot_params->d0i3_entry_vpu_ts = 0;
648 if (IVPU_WA(disable_d0i2))719 if (IVPU_WA(disable_d0i2))
649 boot_params->power_profile |= BIT(1);720 boot_params->power_profile |= BIT(1);
721 boot_params->vpu_uses_ecc_mca_signal =
722 ivpu_hw_uses_ecc_mca_signal(vdev) ? VPU_BOOT_MCA_ECC_BOTH : 0;
650 723
651 boot_params->system_time_us = ktime_to_us(ktime_get_real());724 boot_params->system_time_us = ktime_to_us(ktime_get_real());
652 wmb(); /* Flush WC buffers after writing bootparams */725 wmb(); /* Flush WC buffers after writing bootparams */
drivers/accel/ivpu/ivpu_fw.h+13−1
@@ -1,6 +1,6 @@
1/* SPDX-License-Identifier: GPL-2.0-only */1/* SPDX-License-Identifier: GPL-2.0-only */
2/*2/*
3 * Copyright (C) 2020-2024 Intel Corporation3 * Copyright (C) 2020-2025 Intel Corporation
4 */4 */
5 5
6#ifndef __IVPU_FW_H__6#ifndef __IVPU_FW_H__
@@ -19,10 +19,16 @@ struct ivpu_fw_info { · 12 unmodified lines
19 const struct firmware *file;19 const struct firmware *file;
20 const char *name;20 const char *name;
21 char version[FW_VERSION_STR_SIZE];21 char version[FW_VERSION_STR_SIZE];
22 struct ivpu_bo *mem_bp;
23 struct ivpu_bo *mem_fw_ver;
22 struct ivpu_bo *mem;24 struct ivpu_bo *mem;
23 struct ivpu_bo *mem_shave_nn;25 struct ivpu_bo *mem_shave_nn;
24 struct ivpu_bo *mem_log_crit;26 struct ivpu_bo *mem_log_crit;
25 struct ivpu_bo *mem_log_verb;27 struct ivpu_bo *mem_log_verb;
28 u64 boot_params_addr;
29 u64 boot_params_size;
30 u64 fw_version_addr;
31 u64 fw_version_size;
26 u64 runtime_addr;32 u64 runtime_addr;
27 u32 runtime_size;33 u32 runtime_size;
28 u64 image_load_offset;34 u64 image_load_offset;
@@ -42,6 +48,7 @@ struct ivpu_fw_info { · 13 unmodified lines
42 u64 last_heartbeat;48 u64 last_heartbeat;
43};49};
44 50
51bool ivpu_is_within_range(u64 addr, size_t size, struct ivpu_addr_range *range);
45int ivpu_fw_init(struct ivpu_device *vdev);52int ivpu_fw_init(struct ivpu_device *vdev);
46void ivpu_fw_fini(struct ivpu_device *vdev);53void ivpu_fw_fini(struct ivpu_device *vdev);
47void ivpu_fw_load(struct ivpu_device *vdev);54void ivpu_fw_load(struct ivpu_device *vdev);
@@ -52,4 +59,9 @@ static inline bool ivpu_fw_is_cold_boot(struct ivpu_device *vdev) · 4 unmodified lines
52 return vdev->fw->entry_point == vdev->fw->cold_boot_entry_point;59 return vdev->fw->entry_point == vdev->fw->cold_boot_entry_point;
53}60}
54 61
62static inline u32 ivpu_fw_preempt_buf_size(struct ivpu_device *vdev)
63{
64 return vdev->fw->primary_preempt_buf_size + vdev->fw->secondary_preempt_buf_size;
65}
66
55#endif /* __IVPU_FW_H__ */67#endif /* __IVPU_FW_H__ */
drivers/accel/ivpu/ivpu_gem.c+117−44
@@ -15,6 +15,7 @@ · 14 unmodified lines
15#include <drm/drm_utils.h>15#include <drm/drm_utils.h>
16 16
17#include "ivpu_drv.h"17#include "ivpu_drv.h"
18#include "ivpu_fw.h"
18#include "ivpu_gem.h"19#include "ivpu_gem.h"
19#include "ivpu_hw.h"20#include "ivpu_hw.h"
20#include "ivpu_mmu.h"21#include "ivpu_mmu.h"
@@ -27,8 +28,8 @@ static const struct drm_gem_object_funcs ivpu_gem_funcs; · 6 unmodified lines
27static inline void ivpu_dbg_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, const char *action)28static inline void ivpu_dbg_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, const char *action)
28{29{
29 ivpu_dbg(vdev, BO,30 ivpu_dbg(vdev, BO,
30 "%6s: bo %8p vpu_addr %9llx size %8zu ctx %d has_pages %d dma_mapped %d mmu_mapped %d wc %d imported %d\n",31 "%6s: bo %8p size %9zu ctx %d vpu_addr %9llx pages %d sgt %d mmu_mapped %d wc %d imported %d\n",
31 action, bo, bo->vpu_addr, ivpu_bo_size(bo), bo->ctx_id,32 action, bo, ivpu_bo_size(bo), bo->ctx_id, bo->vpu_addr,
32 (bool)bo->base.pages, (bool)bo->base.sgt, bo->mmu_mapped, bo->base.map_wc,33 (bool)bo->base.pages, (bool)bo->base.sgt, bo->mmu_mapped, bo->base.map_wc,
33 (bool)drm_gem_is_imported(&bo->base.base));34 (bool)drm_gem_is_imported(&bo->base.base));
34}35}
@@ -43,22 +44,47 @@ static inline void ivpu_bo_unlock(struct ivpu_bo *bo) · 8 unmodified lines
43 dma_resv_unlock(bo->base.base.resv);44 dma_resv_unlock(bo->base.base.resv);
44}45}
45 46
47static struct sg_table *ivpu_bo_map_attachment(struct ivpu_device *vdev, struct ivpu_bo *bo)
48{
49 struct sg_table *sgt;
50
51 drm_WARN_ON(&vdev->drm, !bo->base.base.import_attach);
52
53 ivpu_bo_lock(bo);
54
55 sgt = bo->base.sgt;
56 if (!sgt) {
57 sgt = dma_buf_map_attachment(bo->base.base.import_attach, DMA_BIDIRECTIONAL);
58 if (IS_ERR(sgt))
59 ivpu_err(vdev, "Failed to map BO in IOMMU: %ld\n", PTR_ERR(sgt));
60 else
61 bo->base.sgt = sgt;
62 }
63
64 ivpu_bo_unlock(bo);
65
66 return sgt;
67}
68
46/*69/*
47 * ivpu_bo_pin() - pin the backing physical pages and map them to VPU.70 * ivpu_bo_bind() - pin the backing physical pages and map them to VPU.
48 *71 *
49 * This function pins physical memory pages, then maps the physical pages72 * This function pins physical memory pages, then maps the physical pages
50 * to IOMMU address space and finally updates the VPU MMU page tables73 * to IOMMU address space and finally updates the VPU MMU page tables
51 * to allow the VPU to translate VPU address to IOMMU address.74 * to allow the VPU to translate VPU address to IOMMU address.
52 */75 */
53int __must_check ivpu_bo_pin(struct ivpu_bo *bo)76int __must_check ivpu_bo_bind(struct ivpu_bo *bo)
54{77{
55 struct ivpu_device *vdev = ivpu_bo_to_vdev(bo);78 struct ivpu_device *vdev = ivpu_bo_to_vdev(bo);
56 struct sg_table *sgt;79 struct sg_table *sgt;
57 int ret = 0;80 int ret = 0;
58 81
59 ivpu_dbg_bo(vdev, bo, "pin");82 ivpu_dbg_bo(vdev, bo, "bind");
60 83
61 sgt = drm_gem_shmem_get_pages_sgt(&bo->base);84 if (bo->base.base.import_attach)
85 sgt = ivpu_bo_map_attachment(vdev, bo);
86 else
87 sgt = drm_gem_shmem_get_pages_sgt(&bo->base);
62 if (IS_ERR(sgt)) {88 if (IS_ERR(sgt)) {
63 ret = PTR_ERR(sgt);89 ret = PTR_ERR(sgt);
64 ivpu_err(vdev, "Failed to map BO in IOMMU: %d\n", ret);90 ivpu_err(vdev, "Failed to map BO in IOMMU: %d\n", ret);
@@ -70,7 +96,7 @@ int __must_check ivpu_bo_pin(struct ivpu_bo *bo) · 5 unmodified lines
70 if (!bo->mmu_mapped) {96 if (!bo->mmu_mapped) {
71 drm_WARN_ON(&vdev->drm, !bo->ctx);97 drm_WARN_ON(&vdev->drm, !bo->ctx);
72 ret = ivpu_mmu_context_map_sgt(vdev, bo->ctx, bo->vpu_addr, sgt,98 ret = ivpu_mmu_context_map_sgt(vdev, bo->ctx, bo->vpu_addr, sgt,
73 ivpu_bo_is_snooped(bo));99 ivpu_bo_is_snooped(bo), ivpu_bo_is_read_only(bo));
74 if (ret) {100 if (ret) {
75 ivpu_err(vdev, "Failed to map BO in MMU: %d\n", ret);101 ivpu_err(vdev, "Failed to map BO in MMU: %d\n", ret);
76 goto unlock;102 goto unlock;
@@ -99,9 +125,9 @@ ivpu_bo_alloc_vpu_addr(struct ivpu_bo *bo, struct ivpu_mmu_context *ctx, · 22 unmodified lines
99 ret = ivpu_mmu_context_insert_node(ctx, range, ivpu_bo_size(bo), &bo->mm_node);125 ret = ivpu_mmu_context_insert_node(ctx, range, ivpu_bo_size(bo), &bo->mm_node);
100 if (!ret) {126 if (!ret) {
101 bo->ctx = ctx;127 bo->ctx = ctx;
128 bo->ctx_id = ctx->id;
102 bo->vpu_addr = bo->mm_node.start;129 bo->vpu_addr = bo->mm_node.start;
103 } else {130 ivpu_dbg_bo(vdev, bo, "vaddr");
104 ivpu_err(vdev, "Failed to add BO to context %u: %d\n", ctx->id, ret);
105 }131 }
106 132
107 ivpu_bo_unlock(bo);133 ivpu_bo_unlock(bo);
@@ -115,7 +141,7 @@ static void ivpu_bo_unbind_locked(struct ivpu_bo *bo) · 7 unmodified lines
115{141{
116 struct ivpu_device *vdev = ivpu_bo_to_vdev(bo);142 struct ivpu_device *vdev = ivpu_bo_to_vdev(bo);
117 143
118 lockdep_assert(dma_resv_held(bo->base.base.resv) || !kref_read(&bo->base.base.refcount));144 dma_resv_assert_held(bo->base.base.resv);
119 145
120 if (bo->mmu_mapped) {146 if (bo->mmu_mapped) {
121 drm_WARN_ON(&vdev->drm, !bo->ctx);147 drm_WARN_ON(&vdev->drm, !bo->ctx);
@@ -130,13 +156,15 @@ static void ivpu_bo_unbind_locked(struct ivpu_bo *bo) · 8 unmodified lines
130 bo->ctx = NULL;156 bo->ctx = NULL;
131 }157 }
132 158
133 if (drm_gem_is_imported(&bo->base.base))
134 return;
135
136 if (bo->base.sgt) {159 if (bo->base.sgt) {
137 dma_unmap_sgtable(vdev->drm.dev, bo->base.sgt, DMA_BIDIRECTIONAL, 0);160 if (bo->base.base.import_attach) {
138 sg_free_table(bo->base.sgt);161 dma_buf_unmap_attachment(bo->base.base.import_attach,
139 kfree(bo->base.sgt);162 bo->base.sgt, DMA_BIDIRECTIONAL);
163 } else {
164 dma_unmap_sgtable(vdev->drm.dev, bo->base.sgt, DMA_BIDIRECTIONAL, 0);
165 sg_free_table(bo->base.sgt);
166 kfree(bo->base.sgt);
167 }
140 bo->base.sgt = NULL;168 bo->base.sgt = NULL;
141 }169 }
142}170}
@@ -182,10 +210,11 @@ struct drm_gem_object *ivpu_gem_create_object(struct drm_device *dev, size_t siz · 39 unmodified lines
182struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev,210struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev,
183 struct dma_buf *dma_buf)211 struct dma_buf *dma_buf)
184{212{
213 struct ivpu_device *vdev = to_ivpu_device(dev);
185 struct device *attach_dev = dev->dev;214 struct device *attach_dev = dev->dev;
186 struct dma_buf_attachment *attach;215 struct dma_buf_attachment *attach;
187 struct sg_table *sgt;
188 struct drm_gem_object *obj;216 struct drm_gem_object *obj;
217 struct ivpu_bo *bo;
189 int ret;218 int ret;
190 219
191 attach = dma_buf_attach(dma_buf, attach_dev);220 attach = dma_buf_attach(dma_buf, attach_dev);
@@ -194,25 +223,25 @@ struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, · 2 unmodified lines
194 223
195 get_dma_buf(dma_buf);224 get_dma_buf(dma_buf);
196 225
197 sgt = dma_buf_map_attachment_unlocked(attach, DMA_BIDIRECTIONAL);226 obj = drm_gem_shmem_prime_import_sg_table(dev, attach, NULL);
198 if (IS_ERR(sgt)) {
199 ret = PTR_ERR(sgt);
200 goto fail_detach;
201 }
202
203 obj = drm_gem_shmem_prime_import_sg_table(dev, attach, sgt);
204 if (IS_ERR(obj)) {227 if (IS_ERR(obj)) {
205 ret = PTR_ERR(obj);228 ret = PTR_ERR(obj);
206 goto fail_unmap;229 goto fail_detach;
207 }230 }
208 231
209 obj->import_attach = attach;232 obj->import_attach = attach;
210 obj->resv = dma_buf->resv;233 obj->resv = dma_buf->resv;
211 234
235 bo = to_ivpu_bo(obj);
236
237 mutex_lock(&vdev->bo_list_lock);
238 list_add_tail(&bo->bo_list_node, &vdev->bo_list);
239 mutex_unlock(&vdev->bo_list_lock);
240
241 ivpu_dbg(vdev, BO, "import: bo %8p size %9zu\n", bo, ivpu_bo_size(bo));
242
212 return obj;243 return obj;
213 244
214fail_unmap:
215 dma_buf_unmap_attachment_unlocked(attach, sgt, DMA_BIDIRECTIONAL);
216fail_detach:245fail_detach:
217 dma_buf_detach(dma_buf, attach);246 dma_buf_detach(dma_buf, attach);
218 dma_buf_put(dma_buf);247 dma_buf_put(dma_buf);
@@ -220,7 +249,7 @@ fail_detach: · 1 unmodified line
220 return ERR_PTR(ret);249 return ERR_PTR(ret);
221}250}
222 251
223static struct ivpu_bo *ivpu_bo_alloc(struct ivpu_device *vdev, u64 size, u32 flags, u32 ctx_id)252static struct ivpu_bo *ivpu_bo_alloc(struct ivpu_device *vdev, u64 size, u32 flags)
224{253{
225 struct drm_gem_shmem_object *shmem;254 struct drm_gem_shmem_object *shmem;
226 struct ivpu_bo *bo;255 struct ivpu_bo *bo;
@@ -238,7 +267,6 @@ static struct ivpu_bo *ivpu_bo_alloc(struct ivpu_device *vdev, u64 size, u32 fla · 11 unmodified lines
238 return ERR_CAST(shmem);267 return ERR_CAST(shmem);
239 268
240 bo = to_ivpu_bo(&shmem->base);269 bo = to_ivpu_bo(&shmem->base);
241 bo->ctx_id = ctx_id;
242 bo->base.map_wc = flags & DRM_IVPU_BO_WC;270 bo->base.map_wc = flags & DRM_IVPU_BO_WC;
243 bo->flags = flags;271 bo->flags = flags;
244 272
@@ -246,7 +274,7 @@ static struct ivpu_bo *ivpu_bo_alloc(struct ivpu_device *vdev, u64 size, u32 fla · 1 unmodified line
246 list_add_tail(&bo->bo_list_node, &vdev->bo_list);274 list_add_tail(&bo->bo_list_node, &vdev->bo_list);
247 mutex_unlock(&vdev->bo_list_lock);275 mutex_unlock(&vdev->bo_list_lock);
248 276
249 ivpu_dbg_bo(vdev, bo, "alloc");277 ivpu_dbg(vdev, BO, " alloc: bo %8p size %9llu\n", bo, size);
250 278
251 return bo;279 return bo;
252}280}
@@ -259,8 +287,8 @@ static int ivpu_gem_bo_open(struct drm_gem_object *obj, struct drm_file *file) · 6 unmodified lines
259 struct ivpu_addr_range *range;287 struct ivpu_addr_range *range;
260 288
261 if (bo->ctx) {289 if (bo->ctx) {
262 ivpu_warn(vdev, "Can't add BO to ctx %u: already in ctx %u\n",290 ivpu_dbg(vdev, IOCTL, "Can't add BO %pe to ctx %u: already in ctx %u\n",
263 file_priv->ctx.id, bo->ctx->id);291 bo, file_priv->ctx.id, bo->ctx->id);
264 return -EALREADY;292 return -EALREADY;
265 }293 }
266 294
@@ -281,23 +309,41 @@ static void ivpu_gem_bo_free(struct drm_gem_object *obj) · 14 unmodified lines
281 309
282 ivpu_dbg_bo(vdev, bo, "free");310 ivpu_dbg_bo(vdev, bo, "free");
283 311
312 drm_WARN_ON(&vdev->drm, list_empty(&bo->bo_list_node));
313
284 mutex_lock(&vdev->bo_list_lock);314 mutex_lock(&vdev->bo_list_lock);
285 list_del(&bo->bo_list_node);315 list_del(&bo->bo_list_node);
286 mutex_unlock(&vdev->bo_list_lock);
287 316
288 drm_WARN_ON(&vdev->drm, !drm_gem_is_imported(&bo->base.base) &&317 drm_WARN_ON(&vdev->drm, !drm_gem_is_imported(&bo->base.base) &&
289 !dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_READ));318 !dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_READ));
290 drm_WARN_ON(&vdev->drm, ivpu_bo_size(bo) == 0);319 drm_WARN_ON(&vdev->drm, ivpu_bo_size(bo) == 0);
291 drm_WARN_ON(&vdev->drm, bo->base.vaddr);320 drm_WARN_ON(&vdev->drm, bo->base.vaddr);
292 321
322 ivpu_bo_lock(bo);
293 ivpu_bo_unbind_locked(bo);323 ivpu_bo_unbind_locked(bo);
324 ivpu_bo_unlock(bo);
325
326 mutex_unlock(&vdev->bo_list_lock);
327
294 drm_WARN_ON(&vdev->drm, bo->mmu_mapped);328 drm_WARN_ON(&vdev->drm, bo->mmu_mapped);
295 drm_WARN_ON(&vdev->drm, bo->ctx);329 drm_WARN_ON(&vdev->drm, bo->ctx);
296 330
297 drm_WARN_ON(obj->dev, refcount_read(&bo->base.pages_use_count) > 1);331 drm_WARN_ON(obj->dev, refcount_read(&bo->base.pages_use_count) > 1);
332 drm_WARN_ON(obj->dev, bo->base.base.vma_node.vm_files.rb_node);
298 drm_gem_shmem_free(&bo->base);333 drm_gem_shmem_free(&bo->base);
299}334}
300 335
336static enum drm_gem_object_status ivpu_gem_status(struct drm_gem_object *obj)
337{
338 struct ivpu_bo *bo = to_ivpu_bo(obj);
339 enum drm_gem_object_status status = 0;
340
341 if (ivpu_bo_is_resident(bo))
342 status |= DRM_GEM_OBJECT_RESIDENT;
343
344 return status;
345}
346
301static const struct drm_gem_object_funcs ivpu_gem_funcs = {347static const struct drm_gem_object_funcs ivpu_gem_funcs = {
302 .free = ivpu_gem_bo_free,348 .free = ivpu_gem_bo_free,
303 .open = ivpu_gem_bo_open,349 .open = ivpu_gem_bo_open,
@@ -308,6 +354,7 @@ static const struct drm_gem_object_funcs ivpu_gem_funcs = { · 4 unmodified lines
308 .vmap = drm_gem_shmem_object_vmap,354 .vmap = drm_gem_shmem_object_vmap,
309 .vunmap = drm_gem_shmem_object_vunmap,355 .vunmap = drm_gem_shmem_object_vunmap,
310 .mmap = drm_gem_shmem_object_mmap,356 .mmap = drm_gem_shmem_object_mmap,
357 .status = ivpu_gem_status,
311 .vm_ops = &drm_gem_shmem_vm_ops,358 .vm_ops = &drm_gem_shmem_vm_ops,
312};359};
313 360
@@ -320,25 +367,33 @@ int ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *fi · 6 unmodified lines
320 struct ivpu_bo *bo;367 struct ivpu_bo *bo;
321 int ret;368 int ret;
322 369
323 if (args->flags & ~DRM_IVPU_BO_FLAGS)370 if (args->flags & ~DRM_IVPU_BO_FLAGS) {
371 ivpu_dbg(vdev, IOCTL, "Invalid BO flags 0x%x\n", args->flags);
324 return -EINVAL;372 return -EINVAL;
373 }
325 374
326 if (size == 0)375 if (size == 0) {
376 ivpu_dbg(vdev, IOCTL, "Invalid BO size %llu\n", args->size);
327 return -EINVAL;377 return -EINVAL;
378 }
328 379
329 bo = ivpu_bo_alloc(vdev, size, args->flags, file_priv->ctx.id);380 bo = ivpu_bo_alloc(vdev, size, args->flags);
330 if (IS_ERR(bo)) {381 if (IS_ERR(bo)) {
331 ivpu_err(vdev, "Failed to allocate BO: %pe (ctx %u size %llu flags 0x%x)",382 ivpu_dbg(vdev, IOCTL, "Failed to allocate BO: %pe ctx %u size %llu flags 0x%x\n",
332 bo, file_priv->ctx.id, args->size, args->flags);383 bo, file_priv->ctx.id, args->size, args->flags);
333 return PTR_ERR(bo);384 return PTR_ERR(bo);
334 }385 }
335 386
387 drm_WARN_ON(&vdev->drm, bo->base.base.handle_count != 0);
388
336 ret = drm_gem_handle_create(file, &bo->base.base, &args->handle);389 ret = drm_gem_handle_create(file, &bo->base.base, &args->handle);
337 if (ret)390 if (ret) {
338 ivpu_err(vdev, "Failed to create handle for BO: %pe (ctx %u size %llu flags 0x%x)",391 ivpu_dbg(vdev, IOCTL, "Failed to create handle for BO: %pe ctx %u size %llu flags 0x%x\n",
339 bo, file_priv->ctx.id, args->size, args->flags);392 bo, file_priv->ctx.id, args->size, args->flags);
340 else393 } else {
341 args->vpu_addr = bo->vpu_addr;394 args->vpu_addr = bo->vpu_addr;
395 drm_WARN_ON(&vdev->drm, bo->base.base.handle_count != 1);
396 }
342 397
343 drm_gem_object_put(&bo->base.base);398 drm_gem_object_put(&bo->base.base);
344 399
@@ -360,18 +415,21 @@ ivpu_bo_create(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, · 15 unmodified lines
360 drm_WARN_ON(&vdev->drm, !PAGE_ALIGNED(range->end));415 drm_WARN_ON(&vdev->drm, !PAGE_ALIGNED(range->end));
361 drm_WARN_ON(&vdev->drm, !PAGE_ALIGNED(size));416 drm_WARN_ON(&vdev->drm, !PAGE_ALIGNED(size));
362 417
363 bo = ivpu_bo_alloc(vdev, size, flags, IVPU_GLOBAL_CONTEXT_MMU_SSID);418 bo = ivpu_bo_alloc(vdev, size, flags);
364 if (IS_ERR(bo)) {419 if (IS_ERR(bo)) {
365 ivpu_err(vdev, "Failed to allocate BO: %pe (vpu_addr 0x%llx size %llu flags 0x%x)",420 ivpu_err(vdev, "Failed to allocate BO: %pe vpu_addr 0x%llx size %llu flags 0x%x\n",
366 bo, range->start, size, flags);421 bo, range->start, size, flags);
367 return NULL;422 return NULL;
368 }423 }
369 424
370 ret = ivpu_bo_alloc_vpu_addr(bo, ctx, range);425 ret = ivpu_bo_alloc_vpu_addr(bo, ctx, range);
371 if (ret)426 if (ret) {
427 ivpu_err(vdev, "Failed to allocate NPU address for BO: %pe ctx %u size %llu: %d\n",
428 bo, ctx->id, size, ret);
372 goto err_put;429 goto err_put;
430 }
373 431
374 ret = ivpu_bo_pin(bo);432 ret = ivpu_bo_bind(bo);
375 if (ret)433 if (ret)
376 goto err_put;434 goto err_put;
377 435
@@ -391,6 +449,21 @@ err_put: · 13 unmodified lines
391 return NULL;449 return NULL;
392}450}
393 451
452struct ivpu_bo *ivpu_bo_create_runtime(struct ivpu_device *vdev, u64 addr, u64 size, u32 flags)
453{
454 struct ivpu_addr_range range;
455
456 if (!ivpu_is_within_range(addr, size, &vdev->hw->ranges.runtime)) {
457 ivpu_err(vdev, "Invalid runtime BO address 0x%llx size %llu\n", addr, size);
458 return NULL;
459 }
460
461 if (ivpu_hw_range_init(vdev, &range, addr, size))
462 return NULL;
463
464 return ivpu_bo_create(vdev, &vdev->gctx, &range, size, flags);
465}
466
394struct ivpu_bo *ivpu_bo_create_global(struct ivpu_device *vdev, u64 size, u32 flags)467struct ivpu_bo *ivpu_bo_create_global(struct ivpu_device *vdev, u64 size, u32 flags)
395{468{
396 return ivpu_bo_create(vdev, &vdev->gctx, &vdev->hw->ranges.global, size, flags);469 return ivpu_bo_create(vdev, &vdev->gctx, &vdev->hw->ranges.global, size, flags);
drivers/accel/ivpu/ivpu_gem.h+20−2
@@ -1,6 +1,6 @@
1/* SPDX-License-Identifier: GPL-2.0-only */1/* SPDX-License-Identifier: GPL-2.0-only */
2/*2/*
3 * Copyright (C) 2020-2023 Intel Corporation3 * Copyright (C) 2020-2025 Intel Corporation
4 */4 */
5#ifndef __IVPU_GEM_H__5#ifndef __IVPU_GEM_H__
6#define __IVPU_GEM_H__6#define __IVPU_GEM_H__
@@ -24,19 +24,22 @@ struct ivpu_bo { · 17 unmodified lines
24 bool mmu_mapped;24 bool mmu_mapped;
25};25};
26 26
27int ivpu_bo_pin(struct ivpu_bo *bo);27int ivpu_bo_bind(struct ivpu_bo *bo);
28void ivpu_bo_unbind_all_bos_from_context(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx);28void ivpu_bo_unbind_all_bos_from_context(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx);
29 29
30struct drm_gem_object *ivpu_gem_create_object(struct drm_device *dev, size_t size);30struct drm_gem_object *ivpu_gem_create_object(struct drm_device *dev, size_t size);
31struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, struct dma_buf *dma_buf);31struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, struct dma_buf *dma_buf);
32struct ivpu_bo *ivpu_bo_create(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,32struct ivpu_bo *ivpu_bo_create(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,
33 struct ivpu_addr_range *range, u64 size, u32 flags);33 struct ivpu_addr_range *range, u64 size, u32 flags);
34struct ivpu_bo *ivpu_bo_create_runtime(struct ivpu_device *vdev, u64 addr, u64 size, u32 flags);
34struct ivpu_bo *ivpu_bo_create_global(struct ivpu_device *vdev, u64 size, u32 flags);35struct ivpu_bo *ivpu_bo_create_global(struct ivpu_device *vdev, u64 size, u32 flags);
35void ivpu_bo_free(struct ivpu_bo *bo);36void ivpu_bo_free(struct ivpu_bo *bo);
36 37
37int ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file);38int ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file);
38int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file);39int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file);
39int ivpu_bo_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file);40int ivpu_bo_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file);
41int ivpu_bo_create_from_userptr_ioctl(struct drm_device *dev, void *data,
42 struct drm_file *file);
40 43
41void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p);44void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p);
42void ivpu_bo_list_print(struct drm_device *dev);45void ivpu_bo_list_print(struct drm_device *dev);
@@ -74,6 +77,16 @@ static inline bool ivpu_bo_is_snooped(struct ivpu_bo *bo) · 31 unmodified lines
74 return ivpu_bo_cache_mode(bo) == DRM_IVPU_BO_CACHED;77 return ivpu_bo_cache_mode(bo) == DRM_IVPU_BO_CACHED;
75}78}
76 79
80static inline bool ivpu_bo_is_read_only(struct ivpu_bo *bo)
81{
82 return bo->flags & DRM_IVPU_BO_READ_ONLY;
83}
84
85static inline bool ivpu_bo_is_resident(struct ivpu_bo *bo)
86{
87 return !!bo->base.pages;
88}
89
77static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u32 vpu_addr)90static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u32 vpu_addr)
78{91{
79 if (vpu_addr < bo->vpu_addr)92 if (vpu_addr < bo->vpu_addr)
@@ -96,4 +109,9 @@ static inline u32 cpu_to_vpu_addr(struct ivpu_bo *bo, void *cpu_addr) · 16 unmodified lines
96 return bo->vpu_addr + (cpu_addr - ivpu_bo_vaddr(bo));109 return bo->vpu_addr + (cpu_addr - ivpu_bo_vaddr(bo));
97}110}
98 111
112static inline bool ivpu_bo_is_mappable(struct ivpu_bo *bo)
113{
114 return bo->flags & DRM_IVPU_BO_MAPPABLE;
115}
116
99#endif /* __IVPU_GEM_H__ */117#endif /* __IVPU_GEM_H__ */
drivers/accel/ivpu/ivpu_gem_userptr.c+213−0
@@ -0,0 +1,213 @@
1// SPDX-License-Identifier: GPL-2.0-only
2/*
3 * Copyright (C) 2020-2025 Intel Corporation
4 */
5
6#include <linux/dma-buf.h>
7#include <linux/err.h>
8#include <linux/highmem.h>
9#include <linux/mm.h>
10#include <linux/mman.h>
11#include <linux/scatterlist.h>
12#include <linux/slab.h>
13#include <linux/capability.h>
14
15#include <drm/drm_device.h>
16#include <drm/drm_file.h>
17#include <drm/drm_gem.h>
18
19#include "ivpu_drv.h"
20#include "ivpu_gem.h"
21
22static struct sg_table *
23ivpu_gem_userptr_dmabuf_map(struct dma_buf_attachment *attachment,
24 enum dma_data_direction direction)
25{
26 struct sg_table *sgt = attachment->dmabuf->priv;
27 int ret;
28
29 ret = dma_map_sgtable(attachment->dev, sgt, direction, DMA_ATTR_SKIP_CPU_SYNC);
30 if (ret)
31 return ERR_PTR(ret);
32
33 return sgt;
34}
35
36static void ivpu_gem_userptr_dmabuf_unmap(struct dma_buf_attachment *attachment,
37 struct sg_table *sgt,
38 enum dma_data_direction direction)
39{
40 dma_unmap_sgtable(attachment->dev, sgt, direction, DMA_ATTR_SKIP_CPU_SYNC);
41}
42
43static void ivpu_gem_userptr_dmabuf_release(struct dma_buf *dma_buf)
44{
45 struct sg_table *sgt = dma_buf->priv;
46 struct sg_page_iter page_iter;
47 struct page *page;
48
49 for_each_sgtable_page(sgt, &page_iter, 0) {
50 page = sg_page_iter_page(&page_iter);
51 unpin_user_page(page);
52 }
53
54 sg_free_table(sgt);
55 kfree(sgt);
56}
57
58static const struct dma_buf_ops ivpu_gem_userptr_dmabuf_ops = {
59 .map_dma_buf = ivpu_gem_userptr_dmabuf_map,
60 .unmap_dma_buf = ivpu_gem_userptr_dmabuf_unmap,
61 .release = ivpu_gem_userptr_dmabuf_release,
62};
63
64static struct dma_buf *
65ivpu_create_userptr_dmabuf(struct ivpu_device *vdev, void __user *user_ptr,
66 size_t size, uint32_t flags)
67{
68 struct dma_buf_export_info exp_info = {};
69 struct dma_buf *dma_buf;
70 struct sg_table *sgt;
71 struct page **pages;
72 unsigned long nr_pages = size >> PAGE_SHIFT;
73 unsigned int gup_flags = FOLL_LONGTERM;
74 int ret, i, pinned;
75
76 /* Add FOLL_WRITE only if the BO is not read-only */
77 if (!(flags & DRM_IVPU_BO_READ_ONLY))
78 gup_flags |= FOLL_WRITE;
79
80 pages = kvmalloc_array(nr_pages, sizeof(*pages), GFP_KERNEL);
81 if (!pages)
82 return ERR_PTR(-ENOMEM);
83
84 pinned = pin_user_pages_fast((unsigned long)user_ptr, nr_pages, gup_flags, pages);
85 if (pinned < 0) {
86 ret = pinned;
87 ivpu_dbg(vdev, IOCTL, "Failed to pin user pages: %d\n", ret);
88 goto free_pages_array;
89 }
90
91 if (pinned != nr_pages) {
92 ivpu_dbg(vdev, IOCTL, "Pinned %d pages, expected %lu\n", pinned, nr_pages);
93 ret = -EFAULT;
94 goto unpin_pages;
95 }
96
97 sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
98 if (!sgt) {
99 ret = -ENOMEM;
100 goto unpin_pages;
101 }
102
103 ret = sg_alloc_table_from_pages(sgt, pages, nr_pages, 0, size, GFP_KERNEL);
104 if (ret) {
105 ivpu_dbg(vdev, IOCTL, "Failed to create sg table: %d\n", ret);
106 goto free_sgt;
107 }
108
109 exp_info.exp_name = "ivpu_userptr_dmabuf";
110 exp_info.owner = THIS_MODULE;
111 exp_info.ops = &ivpu_gem_userptr_dmabuf_ops;
112 exp_info.size = size;
113 exp_info.flags = O_RDWR | O_CLOEXEC;
114 exp_info.priv = sgt;
115
116 dma_buf = dma_buf_export(&exp_info);
117 if (IS_ERR(dma_buf)) {
118 ret = PTR_ERR(dma_buf);
119 ivpu_dbg(vdev, IOCTL, "Failed to export userptr dma-buf: %d\n", ret);
120 goto free_sg_table;
121 }
122
123 kvfree(pages);
124 return dma_buf;
125
126free_sg_table:
127 sg_free_table(sgt);
128free_sgt:
129 kfree(sgt);
130unpin_pages:
131 for (i = 0; i < pinned; i++)
132 unpin_user_page(pages[i]);
133free_pages_array:
134 kvfree(pages);
135 return ERR_PTR(ret);
136}
137
138static struct ivpu_bo *
139ivpu_bo_create_from_userptr(struct ivpu_device *vdev, void __user *user_ptr,
140 size_t size, uint32_t flags)
141{
142 struct dma_buf *dma_buf;
143 struct drm_gem_object *obj;
144 struct ivpu_bo *bo;
145
146 dma_buf = ivpu_create_userptr_dmabuf(vdev, user_ptr, size, flags);
147 if (IS_ERR(dma_buf))
148 return ERR_CAST(dma_buf);
149
150 obj = ivpu_gem_prime_import(&vdev->drm, dma_buf);
151 if (IS_ERR(obj)) {
152 dma_buf_put(dma_buf);
153 return ERR_CAST(obj);
154 }
155
156 dma_buf_put(dma_buf);
157
158 bo = to_ivpu_bo(obj);
159 bo->flags = flags;
160
161 return bo;
162}
163
164int ivpu_bo_create_from_userptr_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
165{
166 struct drm_ivpu_bo_create_from_userptr *args = data;
167 struct ivpu_file_priv *file_priv = file->driver_priv;
168 struct ivpu_device *vdev = to_ivpu_device(dev);
169 void __user *user_ptr = u64_to_user_ptr(args->user_ptr);
170 struct ivpu_bo *bo;
171 int ret;
172
173 if (args->flags & ~(DRM_IVPU_BO_HIGH_MEM | DRM_IVPU_BO_DMA_MEM | DRM_IVPU_BO_READ_ONLY)) {
174 ivpu_dbg(vdev, IOCTL, "Invalid BO flags: 0x%x\n", args->flags);
175 return -EINVAL;
176 }
177
178 if (!args->user_ptr || !args->size) {
179 ivpu_dbg(vdev, IOCTL, "Userptr or size are zero: ptr %llx size %llu\n",
180 args->user_ptr, args->size);
181 return -EINVAL;
182 }
183
184 if (!PAGE_ALIGNED(args->user_ptr) || !PAGE_ALIGNED(args->size)) {
185 ivpu_dbg(vdev, IOCTL, "Userptr or size not page aligned: ptr %llx size %llu\n",
186 args->user_ptr, args->size);
187 return -EINVAL;
188 }
189
190 if (!access_ok(user_ptr, args->size)) {
191 ivpu_dbg(vdev, IOCTL, "Userptr is not accessible: ptr %llx size %llu\n",
192 args->user_ptr, args->size);
193 return -EFAULT;
194 }
195
196 bo = ivpu_bo_create_from_userptr(vdev, user_ptr, args->size, args->flags);
197 if (IS_ERR(bo))
198 return PTR_ERR(bo);
199
200 ret = drm_gem_handle_create(file, &bo->base.base, &args->handle);
201 if (ret) {
202 ivpu_dbg(vdev, IOCTL, "Failed to create handle for BO: %pe ctx %u size %llu flags 0x%x\n",
203 bo, file_priv->ctx.id, args->size, args->flags);
204 } else {
205 ivpu_dbg(vdev, BO, "Created userptr BO: handle=%u vpu_addr=0x%llx size=%llu flags=0x%x\n",
206 args->handle, bo->vpu_addr, args->size, bo->flags);
207 args->vpu_addr = bo->vpu_addr;
208 }
209
210 drm_gem_object_put(&bo->base.base);
211
212 return ret;
213}
drivers/accel/ivpu/ivpu_hw.c+52−7
@@ -8,6 +8,8 @@ · 7 unmodified lines
8#include "ivpu_hw_btrs.h"8#include "ivpu_hw_btrs.h"
9#include "ivpu_hw_ip.h"9#include "ivpu_hw_ip.h"
10 10
11#include <asm/msr-index.h>
12#include <asm/msr.h>
11#include <linux/dmi.h>13#include <linux/dmi.h>
12#include <linux/fault-inject.h>14#include <linux/fault-inject.h>
13#include <linux/pm_runtime.h>15#include <linux/pm_runtime.h>
@@ -20,6 +22,10 @@ module_param_named_unsafe(fail_hw, ivpu_fail_hw, charp, 0444); · 6 unmodified lines
20MODULE_PARM_DESC(fail_hw, "<interval>,<probability>,<space>,<times>");22MODULE_PARM_DESC(fail_hw, "<interval>,<probability>,<space>,<times>");
21#endif23#endif
22 24
25#define FW_SHARED_MEM_ALIGNMENT SZ_512K /* VPU MTRR limitation */
26
27#define ECC_MCA_SIGNAL_ENABLE_MASK 0xff
28
23static char *platform_to_str(u32 platform)29static char *platform_to_str(u32 platform)
24{30{
25 switch (platform) {31 switch (platform) {
@@ -147,19 +153,39 @@ static void priority_bands_init(struct ivpu_device *vdev) · 121 unmodified lines
147 vdev->hw->hws.process_quantum[VPU_JOB_SCHEDULING_PRIORITY_BAND_REALTIME] = 200000;153 vdev->hw->hws.process_quantum[VPU_JOB_SCHEDULING_PRIORITY_BAND_REALTIME] = 200000;
148}154}
149 155
156int ivpu_hw_range_init(struct ivpu_device *vdev, struct ivpu_addr_range *range, u64 start, u64 size)
157{
158 u64 end;
159
160 if (!range || check_add_overflow(start, size, &end)) {
161 ivpu_err(vdev, "Invalid range: start 0x%llx size %llu\n", start, size);
162 return -EINVAL;
163 }
164
165 range->start = start;
166 range->end = end;
167
168 return 0;
169}
170
150static void memory_ranges_init(struct ivpu_device *vdev)171static void memory_ranges_init(struct ivpu_device *vdev)
151{172{
152 if (ivpu_hw_ip_gen(vdev) == IVPU_HW_IP_37XX) {173 if (ivpu_hw_ip_gen(vdev) == IVPU_HW_IP_37XX) {
153 ivpu_hw_range_init(&vdev->hw->ranges.global, 0x80000000, SZ_512M);174 ivpu_hw_range_init(vdev, &vdev->hw->ranges.runtime, 0x84800000, SZ_64M);
154 ivpu_hw_range_init(&vdev->hw->ranges.user, 0x88000000, 511 * SZ_1M);175 ivpu_hw_range_init(vdev, &vdev->hw->ranges.global, 0x90000000, SZ_256M);
155 ivpu_hw_range_init(&vdev->hw->ranges.shave, 0x180000000, SZ_2G);176 ivpu_hw_range_init(vdev, &vdev->hw->ranges.user, 0xa0000000, 511 * SZ_1M);
156 ivpu_hw_range_init(&vdev->hw->ranges.dma, 0x200000000, SZ_128G);177 ivpu_hw_range_init(vdev, &vdev->hw->ranges.shave, 0x180000000, SZ_2G);
178 ivpu_hw_range_init(vdev, &vdev->hw->ranges.dma, 0x200000000, SZ_128G);
157 } else {179 } else {
158 ivpu_hw_range_init(&vdev->hw->ranges.global, 0x80000000, SZ_512M);180 ivpu_hw_range_init(vdev, &vdev->hw->ranges.runtime, 0x80000000, SZ_64M);
159 ivpu_hw_range_init(&vdev->hw->ranges.shave, 0x80000000, SZ_2G);181 ivpu_hw_range_init(vdev, &vdev->hw->ranges.global, 0x90000000, SZ_256M);
160 ivpu_hw_range_init(&vdev->hw->ranges.user, 0x100000000, SZ_256G);182 ivpu_hw_range_init(vdev, &vdev->hw->ranges.shave, 0x80000000, SZ_2G);
183 ivpu_hw_range_init(vdev, &vdev->hw->ranges.user, 0x100000000, SZ_256G);
161 vdev->hw->ranges.dma = vdev->hw->ranges.user;184 vdev->hw->ranges.dma = vdev->hw->ranges.user;
162 }185 }
186
187 drm_WARN_ON(&vdev->drm, !IS_ALIGNED(vdev->hw->ranges.global.start,
188 FW_SHARED_MEM_ALIGNMENT));
163}189}
164 190
165static int wp_enable(struct ivpu_device *vdev)191static int wp_enable(struct ivpu_device *vdev)
@@ -373,3 +399,22 @@ irqreturn_t ivpu_hw_irq_handler(int irq, void *ptr) · 207 unmodified lines
373 pm_runtime_mark_last_busy(vdev->drm.dev);399 pm_runtime_mark_last_busy(vdev->drm.dev);
374 return IRQ_HANDLED;400 return IRQ_HANDLED;
375}401}
402
403bool ivpu_hw_uses_ecc_mca_signal(struct ivpu_device *vdev)
404{
405 unsigned long long msr_integrity_caps;
406 int ret;
407
408 if (ivpu_hw_ip_gen(vdev) < IVPU_HW_IP_50XX)
409 return false;
410
411 ret = rdmsrq_safe(MSR_INTEGRITY_CAPS, &msr_integrity_caps);
412 if (ret) {
413 ivpu_warn(vdev, "Error reading MSR_INTEGRITY_CAPS: %d", ret);
414 return false;
415 }
416
417 ivpu_dbg(vdev, MISC, "MSR_INTEGRITY_CAPS: 0x%llx\n", msr_integrity_caps);
418
419 return msr_integrity_caps & ECC_MCA_SIGNAL_ENABLE_MASK;
420}
drivers/accel/ivpu/ivpu_hw.h+4−6
@@ -21,6 +21,7 @@ struct ivpu_hw_info { · 20 unmodified lines
21 bool (*ip_irq_handler)(struct ivpu_device *vdev, int irq);21 bool (*ip_irq_handler)(struct ivpu_device *vdev, int irq);
22 } irq;22 } irq;
23 struct {23 struct {
24 struct ivpu_addr_range runtime;
24 struct ivpu_addr_range global;25 struct ivpu_addr_range global;
25 struct ivpu_addr_range user;26 struct ivpu_addr_range user;
26 struct ivpu_addr_range shave;27 struct ivpu_addr_range shave;
@@ -51,6 +52,8 @@ struct ivpu_hw_info { · 24 unmodified lines
51};52};
52 53
53int ivpu_hw_init(struct ivpu_device *vdev);54int ivpu_hw_init(struct ivpu_device *vdev);
55int ivpu_hw_range_init(struct ivpu_device *vdev, struct ivpu_addr_range *range, u64 start,
56 u64 size);
54int ivpu_hw_power_up(struct ivpu_device *vdev);57int ivpu_hw_power_up(struct ivpu_device *vdev);
55int ivpu_hw_power_down(struct ivpu_device *vdev);58int ivpu_hw_power_down(struct ivpu_device *vdev);
56int ivpu_hw_reset(struct ivpu_device *vdev);59int ivpu_hw_reset(struct ivpu_device *vdev);
@@ -60,6 +63,7 @@ void ivpu_irq_handlers_init(struct ivpu_device *vdev); · 3 unmodified lines
60void ivpu_hw_irq_enable(struct ivpu_device *vdev);63void ivpu_hw_irq_enable(struct ivpu_device *vdev);
61void ivpu_hw_irq_disable(struct ivpu_device *vdev);64void ivpu_hw_irq_disable(struct ivpu_device *vdev);
62irqreturn_t ivpu_hw_irq_handler(int irq, void *ptr);65irqreturn_t ivpu_hw_irq_handler(int irq, void *ptr);
66bool ivpu_hw_uses_ecc_mca_signal(struct ivpu_device *vdev);
63 67
64static inline u32 ivpu_hw_btrs_irq_handler(struct ivpu_device *vdev, int irq)68static inline u32 ivpu_hw_btrs_irq_handler(struct ivpu_device *vdev, int irq)
65{69{
@@ -71,12 +75,6 @@ static inline u32 ivpu_hw_ip_irq_handler(struct ivpu_device *vdev, int irq) · 5 unmodified lines
71 return vdev->hw->irq.ip_irq_handler(vdev, irq);75 return vdev->hw->irq.ip_irq_handler(vdev, irq);
72}76}
73 77
74static inline void ivpu_hw_range_init(struct ivpu_addr_range *range, u64 start, u64 size)
75{
76 range->start = start;
77 range->end = start + size;
78}
79
80static inline u64 ivpu_hw_range_size(const struct ivpu_addr_range *range)78static inline u64 ivpu_hw_range_size(const struct ivpu_addr_range *range)
81{79{
82 return range->end - range->start;80 return range->end - range->start;
drivers/accel/ivpu/ivpu_hw_btrs.c+18−2
@@ -321,6 +321,14 @@ static int wait_for_pll_lock(struct ivpu_device *vdev, bool enable) · 320 unmodified lines
321 return REGB_POLL_FLD(VPU_HW_BTRS_MTL_PLL_STATUS, LOCK, exp_val, PLL_TIMEOUT_US);321 return REGB_POLL_FLD(VPU_HW_BTRS_MTL_PLL_STATUS, LOCK, exp_val, PLL_TIMEOUT_US);
322}322}
323 323
324static int wait_for_cdyn_deassert(struct ivpu_device *vdev)
325{
326 if (ivpu_hw_btrs_gen(vdev) == IVPU_HW_BTRS_MTL)
327 return 0;
328
329 return REGB_POLL_FLD(VPU_HW_BTRS_LNL_CDYN, CDYN, 0, PLL_TIMEOUT_US);
330}
331
324int ivpu_hw_btrs_wp_drive(struct ivpu_device *vdev, bool enable)332int ivpu_hw_btrs_wp_drive(struct ivpu_device *vdev, bool enable)
325{333{
326 struct wp_request wp;334 struct wp_request wp;
@@ -354,6 +362,14 @@ int ivpu_hw_btrs_wp_drive(struct ivpu_device *vdev, bool enable) · 27 unmodified lines
354 return ret;362 return ret;
355 }363 }
356 364
365 if (!enable) {
366 ret = wait_for_cdyn_deassert(vdev);
367 if (ret) {
368 ivpu_err(vdev, "Timed out waiting for CDYN deassert\n");
369 return ret;
370 }
371 }
372
357 return 0;373 return 0;
358}374}
359 375
@@ -673,7 +689,7 @@ bool ivpu_hw_btrs_irq_handler_lnl(struct ivpu_device *vdev, int irq) · 313 unmodified lines
673 689
674 if (REG_TEST_FLD(VPU_HW_BTRS_LNL_INTERRUPT_STAT, SURV_ERR, status)) {690 if (REG_TEST_FLD(VPU_HW_BTRS_LNL_INTERRUPT_STAT, SURV_ERR, status)) {
675 ivpu_dbg(vdev, IRQ, "Survivability IRQ\n");691 ivpu_dbg(vdev, IRQ, "Survivability IRQ\n");
676 queue_work(system_wq, &vdev->irq_dct_work);692 queue_work(system_percpu_wq, &vdev->irq_dct_work);
677 }693 }
678 694
679 if (REG_TEST_FLD(VPU_HW_BTRS_LNL_INTERRUPT_STAT, FREQ_CHANGE, status)) {695 if (REG_TEST_FLD(VPU_HW_BTRS_LNL_INTERRUPT_STAT, FREQ_CHANGE, status)) {
@@ -752,7 +768,7 @@ int ivpu_hw_btrs_dct_get_request(struct ivpu_device *vdev, bool *enable) · 72 unmodified lines
752 }768 }
753}769}
754 770
755void ivpu_hw_btrs_dct_set_status(struct ivpu_device *vdev, bool enable, u32 active_percent)771void ivpu_hw_btrs_dct_set_status(struct ivpu_device *vdev, bool enable, u8 active_percent)
756{772{
757 u32 val = 0;773 u32 val = 0;
758 u32 cmd = enable ? DCT_ENABLE : DCT_DISABLE;774 u32 cmd = enable ? DCT_ENABLE : DCT_DISABLE;
drivers/accel/ivpu/ivpu_hw_btrs.h+1−1
@@ -36,7 +36,7 @@ u32 ivpu_hw_btrs_dpu_freq_get(struct ivpu_device *vdev); · 35 unmodified lines
36bool ivpu_hw_btrs_irq_handler_mtl(struct ivpu_device *vdev, int irq);36bool ivpu_hw_btrs_irq_handler_mtl(struct ivpu_device *vdev, int irq);
37bool ivpu_hw_btrs_irq_handler_lnl(struct ivpu_device *vdev, int irq);37bool ivpu_hw_btrs_irq_handler_lnl(struct ivpu_device *vdev, int irq);
38int ivpu_hw_btrs_dct_get_request(struct ivpu_device *vdev, bool *enable);38int ivpu_hw_btrs_dct_get_request(struct ivpu_device *vdev, bool *enable);
39void ivpu_hw_btrs_dct_set_status(struct ivpu_device *vdev, bool enable, u32 active_percent);39void ivpu_hw_btrs_dct_set_status(struct ivpu_device *vdev, bool enable, u8 active_percent);
40u32 ivpu_hw_btrs_telemetry_offset_get(struct ivpu_device *vdev);40u32 ivpu_hw_btrs_telemetry_offset_get(struct ivpu_device *vdev);
41u32 ivpu_hw_btrs_telemetry_size_get(struct ivpu_device *vdev);41u32 ivpu_hw_btrs_telemetry_size_get(struct ivpu_device *vdev);
42u32 ivpu_hw_btrs_telemetry_enable_get(struct ivpu_device *vdev);42u32 ivpu_hw_btrs_telemetry_enable_get(struct ivpu_device *vdev);
drivers/accel/ivpu/ivpu_hw_btrs_lnl_reg.h+3−0
@@ -74,6 +74,9 @@ · 73 unmodified lines
74#define VPU_HW_BTRS_LNL_PLL_FREQ 0x00000148u74#define VPU_HW_BTRS_LNL_PLL_FREQ 0x00000148u
75#define VPU_HW_BTRS_LNL_PLL_FREQ_RATIO_MASK GENMASK(15, 0)75#define VPU_HW_BTRS_LNL_PLL_FREQ_RATIO_MASK GENMASK(15, 0)
76 76
77#define VPU_HW_BTRS_LNL_CDYN 0x0000014cu
78#define VPU_HW_BTRS_LNL_CDYN_CDYN_MASK GENMASK(15, 0)
79
77#define VPU_HW_BTRS_LNL_TILE_FUSE 0x00000150u80#define VPU_HW_BTRS_LNL_TILE_FUSE 0x00000150u
78#define VPU_HW_BTRS_LNL_TILE_FUSE_VALID_MASK BIT_MASK(0)81#define VPU_HW_BTRS_LNL_TILE_FUSE_VALID_MASK BIT_MASK(0)
79#define VPU_HW_BTRS_LNL_TILE_FUSE_CONFIG_MASK GENMASK(6, 1)82#define VPU_HW_BTRS_LNL_TILE_FUSE_CONFIG_MASK GENMASK(6, 1)
drivers/accel/ivpu/ivpu_hw_ip.c+10−0
@@ -691,6 +691,13 @@ static void pwr_island_delay_set(struct ivpu_device *vdev) · 690 unmodified lines
691 status = high ? 46 : 3;691 status = high ? 46 : 3;
692 break;692 break;
693 693
694 case PCI_DEVICE_ID_NVL:
695 post = high ? 198 : 17;
696 post1 = 0;
697 post2 = high ? 198 : 17;
698 status = 0;
699 break;
700
694 default:701 default:
695 dump_stack();702 dump_stack();
696 ivpu_err(vdev, "Unknown device ID\n");703 ivpu_err(vdev, "Unknown device ID\n");
@@ -889,6 +896,9 @@ static int soc_cpu_drive_40xx(struct ivpu_device *vdev, bool enable) · 192 unmodified lines
889 896
890static int soc_cpu_enable(struct ivpu_device *vdev)897static int soc_cpu_enable(struct ivpu_device *vdev)
891{898{
899 if (ivpu_hw_ip_gen(vdev) >= IVPU_HW_IP_60XX)
900 return 0;
901
892 return soc_cpu_drive_40xx(vdev, true);902 return soc_cpu_drive_40xx(vdev, true);
893}903}
894 904
drivers/accel/ivpu/ivpu_ipc.c+1−1
@@ -459,7 +459,7 @@ void ivpu_ipc_irq_handler(struct ivpu_device *vdev) · 458 unmodified lines
459 }459 }
460 }460 }
461 461
462 queue_work(system_wq, &vdev->irq_ipc_work);462 queue_work(system_percpu_wq, &vdev->irq_ipc_work);
463}463}
464 464
465void ivpu_ipc_irq_work_fn(struct work_struct *work)465void ivpu_ipc_irq_work_fn(struct work_struct *work)
drivers/accel/ivpu/ivpu_job.c+178−79
@@ -34,22 +34,20 @@ static void ivpu_cmdq_ring_db(struct ivpu_device *vdev, struct ivpu_cmdq *cmdq) · 33 unmodified lines
34static int ivpu_preemption_buffers_create(struct ivpu_device *vdev,34static int ivpu_preemption_buffers_create(struct ivpu_device *vdev,
35 struct ivpu_file_priv *file_priv, struct ivpu_cmdq *cmdq)35 struct ivpu_file_priv *file_priv, struct ivpu_cmdq *cmdq)
36{36{
37 u64 primary_size = ALIGN(vdev->fw->primary_preempt_buf_size, PAGE_SIZE);37 if (ivpu_fw_preempt_buf_size(vdev) == 0)
38 u64 secondary_size = ALIGN(vdev->fw->secondary_preempt_buf_size, PAGE_SIZE);
39
40 if (vdev->fw->sched_mode != VPU_SCHEDULING_MODE_HW ||
41 ivpu_test_mode & IVPU_TEST_MODE_MIP_DISABLE)
42 return 0;38 return 0;
43 39
44 cmdq->primary_preempt_buf = ivpu_bo_create(vdev, &file_priv->ctx, &vdev->hw->ranges.user,40 cmdq->primary_preempt_buf = ivpu_bo_create(vdev, &file_priv->ctx, &vdev->hw->ranges.user,
45 primary_size, DRM_IVPU_BO_WC);41 vdev->fw->primary_preempt_buf_size,
42 DRM_IVPU_BO_WC);
46 if (!cmdq->primary_preempt_buf) {43 if (!cmdq->primary_preempt_buf) {
47 ivpu_err(vdev, "Failed to create primary preemption buffer\n");44 ivpu_err(vdev, "Failed to create primary preemption buffer\n");
48 return -ENOMEM;45 return -ENOMEM;
49 }46 }
50 47
51 cmdq->secondary_preempt_buf = ivpu_bo_create(vdev, &file_priv->ctx, &vdev->hw->ranges.dma,48 cmdq->secondary_preempt_buf = ivpu_bo_create(vdev, &file_priv->ctx, &vdev->hw->ranges.dma,
52 secondary_size, DRM_IVPU_BO_WC);49 vdev->fw->secondary_preempt_buf_size,
50 DRM_IVPU_BO_WC);
53 if (!cmdq->secondary_preempt_buf) {51 if (!cmdq->secondary_preempt_buf) {
54 ivpu_err(vdev, "Failed to create secondary preemption buffer\n");52 ivpu_err(vdev, "Failed to create secondary preemption buffer\n");
55 goto err_free_primary;53 goto err_free_primary;
@@ -66,20 +64,39 @@ err_free_primary: · 10 unmodified lines
66static void ivpu_preemption_buffers_free(struct ivpu_device *vdev,64static void ivpu_preemption_buffers_free(struct ivpu_device *vdev,
67 struct ivpu_file_priv *file_priv, struct ivpu_cmdq *cmdq)65 struct ivpu_file_priv *file_priv, struct ivpu_cmdq *cmdq)
68{66{
69 if (vdev->fw->sched_mode != VPU_SCHEDULING_MODE_HW)
70 return;
71
72 if (cmdq->primary_preempt_buf)67 if (cmdq->primary_preempt_buf)
73 ivpu_bo_free(cmdq->primary_preempt_buf);68 ivpu_bo_free(cmdq->primary_preempt_buf);
74 if (cmdq->secondary_preempt_buf)69 if (cmdq->secondary_preempt_buf)
75 ivpu_bo_free(cmdq->secondary_preempt_buf);70 ivpu_bo_free(cmdq->secondary_preempt_buf);
76}71}
77 72
73static int ivpu_preemption_job_init(struct ivpu_device *vdev, struct ivpu_file_priv *file_priv,
74 struct ivpu_cmdq *cmdq, struct ivpu_job *job)
75{
76 int ret;
77
78 /* Use preemption buffer provided by the user space */
79 if (job->primary_preempt_buf)
80 return 0;
81
82 if (!cmdq->primary_preempt_buf) {
83 /* Allocate per command queue preemption buffers */
84 ret = ivpu_preemption_buffers_create(vdev, file_priv, cmdq);
85 if (ret)
86 return ret;
87 }
88
89 /* Use preemption buffers allocated by the kernel */
90 job->primary_preempt_buf = cmdq->primary_preempt_buf;
91 job->secondary_preempt_buf = cmdq->secondary_preempt_buf;
92
93 return 0;
94}
95
78static struct ivpu_cmdq *ivpu_cmdq_alloc(struct ivpu_file_priv *file_priv)96static struct ivpu_cmdq *ivpu_cmdq_alloc(struct ivpu_file_priv *file_priv)
79{97{
80 struct ivpu_device *vdev = file_priv->vdev;98 struct ivpu_device *vdev = file_priv->vdev;
81 struct ivpu_cmdq *cmdq;99 struct ivpu_cmdq *cmdq;
82 int ret;
83 100
84 cmdq = kzalloc(sizeof(*cmdq), GFP_KERNEL);101 cmdq = kzalloc(sizeof(*cmdq), GFP_KERNEL);
85 if (!cmdq)102 if (!cmdq)
@@ -89,10 +106,6 @@ static struct ivpu_cmdq *ivpu_cmdq_alloc(struct ivpu_file_priv *file_priv) · 3 unmodified lines
89 if (!cmdq->mem)106 if (!cmdq->mem)
90 goto err_free_cmdq;107 goto err_free_cmdq;
91 108
92 ret = ivpu_preemption_buffers_create(vdev, file_priv, cmdq);
93 if (ret)
94 ivpu_warn(vdev, "Failed to allocate preemption buffers, preemption limited\n");
95
96 return cmdq;109 return cmdq;
97 110
98err_free_cmdq:111err_free_cmdq:
@@ -219,11 +232,13 @@ static int ivpu_register_db(struct ivpu_file_priv *file_priv, struct ivpu_cmdq * · 120 unmodified lines
219 ret = ivpu_jsm_register_db(vdev, file_priv->ctx.id, cmdq->db_id,232 ret = ivpu_jsm_register_db(vdev, file_priv->ctx.id, cmdq->db_id,
220 cmdq->mem->vpu_addr, ivpu_bo_size(cmdq->mem));233 cmdq->mem->vpu_addr, ivpu_bo_size(cmdq->mem));
221 234
222 if (!ret)235 if (!ret) {
223 ivpu_dbg(vdev, JOB, "DB %d registered to cmdq %d ctx %d priority %d\n",236 ivpu_dbg(vdev, JOB, "DB %d registered to cmdq %d ctx %d priority %d\n",
224 cmdq->db_id, cmdq->id, file_priv->ctx.id, cmdq->priority);237 cmdq->db_id, cmdq->id, file_priv->ctx.id, cmdq->priority);
225 else238 } else {
226 xa_erase(&vdev->db_xa, cmdq->db_id);239 xa_erase(&vdev->db_xa, cmdq->db_id);
240 cmdq->db_id = 0;
241 }
227 242
228 return ret;243 return ret;
229}244}
@@ -333,7 +348,7 @@ static struct ivpu_cmdq *ivpu_cmdq_acquire(struct ivpu_file_priv *file_priv, u32 · 103 unmodified lines
333 348
334 cmdq = xa_load(&file_priv->cmdq_xa, cmdq_id);349 cmdq = xa_load(&file_priv->cmdq_xa, cmdq_id);
335 if (!cmdq) {350 if (!cmdq) {
336 ivpu_warn_ratelimited(vdev, "Failed to find command queue with ID: %u\n", cmdq_id);351 ivpu_dbg(vdev, IOCTL, "Failed to find command queue with ID: %u\n", cmdq_id);
337 return NULL;352 return NULL;
338 }353 }
339 354
@@ -427,17 +442,14 @@ static int ivpu_cmdq_push_job(struct ivpu_cmdq *cmdq, struct ivpu_job *job) · 87 unmodified lines
427 if (unlikely(ivpu_test_mode & IVPU_TEST_MODE_NULL_SUBMISSION))442 if (unlikely(ivpu_test_mode & IVPU_TEST_MODE_NULL_SUBMISSION))
428 entry->flags = VPU_JOB_FLAGS_NULL_SUBMISSION_MASK;443 entry->flags = VPU_JOB_FLAGS_NULL_SUBMISSION_MASK;
429 444
430 if (vdev->fw->sched_mode == VPU_SCHEDULING_MODE_HW) {445 if (job->primary_preempt_buf) {
431 if (cmdq->primary_preempt_buf) {446 entry->primary_preempt_buf_addr = job->primary_preempt_buf->vpu_addr;
432 entry->primary_preempt_buf_addr = cmdq->primary_preempt_buf->vpu_addr;447 entry->primary_preempt_buf_size = ivpu_bo_size(job->primary_preempt_buf);
433 entry->primary_preempt_buf_size = ivpu_bo_size(cmdq->primary_preempt_buf);448 }
434 }
435 449
436 if (cmdq->secondary_preempt_buf) {450 if (job->secondary_preempt_buf) {
437 entry->secondary_preempt_buf_addr = cmdq->secondary_preempt_buf->vpu_addr;451 entry->secondary_preempt_buf_addr = job->secondary_preempt_buf->vpu_addr;
438 entry->secondary_preempt_buf_size =452 entry->secondary_preempt_buf_size = ivpu_bo_size(job->secondary_preempt_buf);
439 ivpu_bo_size(cmdq->secondary_preempt_buf);
440 }
441 }453 }
442 454
443 wmb(); /* Ensure that tail is updated after filling entry */455 wmb(); /* Ensure that tail is updated after filling entry */
@@ -522,7 +534,7 @@ ivpu_job_create(struct ivpu_file_priv *file_priv, u32 engine_idx, u32 bo_count) · 78 unmodified lines
522 job->bo_count = bo_count;534 job->bo_count = bo_count;
523 job->done_fence = ivpu_fence_create(vdev);535 job->done_fence = ivpu_fence_create(vdev);
524 if (!job->done_fence) {536 if (!job->done_fence) {
525 ivpu_warn_ratelimited(vdev, "Failed to create a fence\n");537 ivpu_err(vdev, "Failed to create a fence\n");
526 goto err_free_job;538 goto err_free_job;
527 }539 }
528 540
@@ -552,21 +564,26 @@ static struct ivpu_job *ivpu_job_remove_from_submitted_jobs(struct ivpu_device * · 23 unmodified lines
552 return job;564 return job;
553}565}
554 566
555static int ivpu_job_signal_and_destroy(struct ivpu_device *vdev, u32 job_id, u32 job_status)567bool ivpu_job_handle_engine_error(struct ivpu_device *vdev, u32 job_id, u32 job_status)
556{568{
557 struct ivpu_job *job;
558
559 lockdep_assert_held(&vdev->submitted_jobs_lock);569 lockdep_assert_held(&vdev->submitted_jobs_lock);
560 570
561 job = xa_load(&vdev->submitted_jobs_xa, job_id);571 switch (job_status) {
562 if (!job)572 case VPU_JSM_STATUS_PROCESSING_ERR:
563 return -ENOENT;573 case VPU_JSM_STATUS_ENGINE_RESET_REQUIRED_MIN ... VPU_JSM_STATUS_ENGINE_RESET_REQUIRED_MAX:
574 {
575 struct ivpu_job *job = xa_load(&vdev->submitted_jobs_xa, job_id);
564 576
565 if (job_status == VPU_JSM_STATUS_MVNCI_CONTEXT_VIOLATION_HW) {577 if (!job)
578 return false;
579
580 /* Trigger an engine reset */
566 guard(mutex)(&job->file_priv->lock);581 guard(mutex)(&job->file_priv->lock);
567 582
583 job->job_status = job_status;
584
568 if (job->file_priv->has_mmu_faults)585 if (job->file_priv->has_mmu_faults)
569 return 0;586 return false;
570 587
571 /*588 /*
572 * Mark context as faulty and defer destruction of the job to jobs abort thread589 * Mark context as faulty and defer destruction of the job to jobs abort thread
@@ -574,23 +591,43 @@ static int ivpu_job_signal_and_destroy(struct ivpu_device *vdev, u32 job_id, u32 · 1 unmodified line
574 * status and ensure both are handled in the same way591 * status and ensure both are handled in the same way
575 */592 */
576 job->file_priv->has_mmu_faults = true;593 job->file_priv->has_mmu_faults = true;
577 queue_work(system_wq, &vdev->context_abort_work);594 queue_work(system_percpu_wq, &vdev->context_abort_work);
578 return 0;595 return true;
579 }596 }
597 default:
598 /* Complete job with error status, engine reset not required */
599 break;
600 }
601
602 return false;
603}
580 604
581 job = ivpu_job_remove_from_submitted_jobs(vdev, job_id);605static int ivpu_job_signal_and_destroy(struct ivpu_device *vdev, u32 job_id, u32 job_status)
606{
607 struct ivpu_job *job;
608
609 lockdep_assert_held(&vdev->submitted_jobs_lock);
610
611 job = xa_load(&vdev->submitted_jobs_xa, job_id);
582 if (!job)612 if (!job)
583 return -ENOENT;613 return -ENOENT;
584 614
585 if (job->file_priv->has_mmu_faults)615 ivpu_job_remove_from_submitted_jobs(vdev, job_id);
586 job_status = DRM_IVPU_JOB_STATUS_ABORTED;
587 616
588 job->bos[CMD_BUF_IDX]->job_status = job_status;617 if (job->job_status == VPU_JSM_STATUS_SUCCESS) {
618 if (job->file_priv->has_mmu_faults)
619 job->job_status = DRM_IVPU_JOB_STATUS_ABORTED;
620 else
621 job->job_status = job_status;
622 }
623
624 job->bos[CMD_BUF_IDX]->job_status = job->job_status;
589 dma_fence_signal(job->done_fence);625 dma_fence_signal(job->done_fence);
590 626
591 trace_job("done", job);627 trace_job("done", job);
592 ivpu_dbg(vdev, JOB, "Job complete: id %3u ctx %2d cmdq_id %u engine %d status 0x%x\n",628 ivpu_dbg(vdev, JOB, "Job complete: id %3u ctx %2d cmdq_id %u engine %d status 0x%x\n",
593 job->job_id, job->file_priv->ctx.id, job->cmdq_id, job->engine_idx, job_status);629 job->job_id, job->file_priv->ctx.id, job->cmdq_id, job->engine_idx,
630 job->job_status);
594 631
595 ivpu_job_destroy(job);632 ivpu_job_destroy(job);
596 ivpu_stop_job_timeout_detection(vdev);633 ivpu_stop_job_timeout_detection(vdev);
@@ -650,7 +687,6 @@ static int ivpu_job_submit(struct ivpu_job *job, u8 priority, u32 cmdq_id) · 53 unmodified lines
650 else687 else
651 cmdq = ivpu_cmdq_acquire(file_priv, cmdq_id);688 cmdq = ivpu_cmdq_acquire(file_priv, cmdq_id);
652 if (!cmdq) {689 if (!cmdq) {
653 ivpu_warn_ratelimited(vdev, "Failed to get job queue, ctx %d\n", file_priv->ctx.id);
654 ret = -EINVAL;690 ret = -EINVAL;
655 goto err_unlock;691 goto err_unlock;
656 }692 }
@@ -661,6 +697,13 @@ static int ivpu_job_submit(struct ivpu_job *job, u8 priority, u32 cmdq_id) · 4 unmodified lines
661 goto err_unlock;697 goto err_unlock;
662 }698 }
663 699
700 ret = ivpu_preemption_job_init(vdev, file_priv, cmdq, job);
701 if (ret) {
702 ivpu_err(vdev, "Failed to initialize preemption buffers for job %d: %d\n",
703 job->job_id, ret);
704 goto err_unlock;
705 }
706
664 job->cmdq_id = cmdq->id;707 job->cmdq_id = cmdq->id;
665 708
666 is_first_job = xa_empty(&vdev->submitted_jobs_xa);709 is_first_job = xa_empty(&vdev->submitted_jobs_xa);
@@ -714,7 +757,7 @@ err_unlock: · 47 unmodified lines
714 757
715static int758static int
716ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 *buf_handles,759ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 *buf_handles,
717 u32 buf_count, u32 commands_offset)760 u32 buf_count, u32 commands_offset, u32 preempt_buffer_index)
718{761{
719 struct ivpu_file_priv *file_priv = job->file_priv;762 struct ivpu_file_priv *file_priv = job->file_priv;
720 struct ivpu_device *vdev = file_priv->vdev;763 struct ivpu_device *vdev = file_priv->vdev;
@@ -727,40 +770,58 @@ ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 · 6 unmodified lines
727 for (i = 0; i < buf_count; i++) {770 for (i = 0; i < buf_count; i++) {
728 struct drm_gem_object *obj = drm_gem_object_lookup(file, buf_handles[i]);771 struct drm_gem_object *obj = drm_gem_object_lookup(file, buf_handles[i]);
729 772
730 if (!obj)773 if (!obj) {
774 ivpu_dbg(vdev, IOCTL, "Failed to lookup GEM object with handle %u\n",
775 buf_handles[i]);
731 return -ENOENT;776 return -ENOENT;
777 }
732 778
733 job->bos[i] = to_ivpu_bo(obj);779 job->bos[i] = to_ivpu_bo(obj);
734 780
735 ret = ivpu_bo_pin(job->bos[i]);781 ret = ivpu_bo_bind(job->bos[i]);
736 if (ret)782 if (ret)
737 return ret;783 return ret;
738 }784 }
739 785
740 bo = job->bos[CMD_BUF_IDX];786 bo = job->bos[CMD_BUF_IDX];
741 if (!dma_resv_test_signaled(bo->base.base.resv, DMA_RESV_USAGE_READ)) {787 if (!dma_resv_test_signaled(bo->base.base.resv, DMA_RESV_USAGE_READ)) {
742 ivpu_warn(vdev, "Buffer is already in use\n");788 ivpu_dbg(vdev, IOCTL, "Buffer is already in use by another job\n");
743 return -EBUSY;789 return -EBUSY;
744 }790 }
745 791
746 if (commands_offset >= ivpu_bo_size(bo)) {792 if (commands_offset >= ivpu_bo_size(bo)) {
747 ivpu_warn(vdev, "Invalid command buffer offset %u\n", commands_offset);793 ivpu_dbg(vdev, IOCTL, "Invalid commands offset %u for buffer size %zu\n",
794 commands_offset, ivpu_bo_size(bo));
748 return -EINVAL;795 return -EINVAL;
749 }796 }
750 797
751 job->cmd_buf_vpu_addr = bo->vpu_addr + commands_offset;798 job->cmd_buf_vpu_addr = bo->vpu_addr + commands_offset;
752 799
800 if (preempt_buffer_index) {
801 struct ivpu_bo *preempt_bo = job->bos[preempt_buffer_index];
802
803 if (ivpu_bo_size(preempt_bo) < ivpu_fw_preempt_buf_size(vdev)) {
804 ivpu_dbg(vdev, IOCTL, "Preemption buffer is too small\n");
805 return -EINVAL;
806 }
807 if (ivpu_bo_is_mappable(preempt_bo)) {
808 ivpu_dbg(vdev, IOCTL, "Preemption buffer cannot be mappable\n");
809 return -EINVAL;
810 }
811 job->primary_preempt_buf = preempt_bo;
812 }
813
753 ret = drm_gem_lock_reservations((struct drm_gem_object **)job->bos, buf_count,814 ret = drm_gem_lock_reservations((struct drm_gem_object **)job->bos, buf_count,
754 &acquire_ctx);815 &acquire_ctx);
755 if (ret) {816 if (ret) {
756 ivpu_warn(vdev, "Failed to lock reservations: %d\n", ret);817 ivpu_warn_ratelimited(vdev, "Failed to lock reservations: %d\n", ret);
757 return ret;818 return ret;
758 }819 }
759 820
760 for (i = 0; i < buf_count; i++) {821 for (i = 0; i < buf_count; i++) {
761 ret = dma_resv_reserve_fences(job->bos[i]->base.base.resv, 1);822 ret = dma_resv_reserve_fences(job->bos[i]->base.base.resv, 1);
762 if (ret) {823 if (ret) {
763 ivpu_warn(vdev, "Failed to reserve fences: %d\n", ret);824 ivpu_warn_ratelimited(vdev, "Failed to reserve fences: %d\n", ret);
764 goto unlock_reservations;825 goto unlock_reservations;
765 }826 }
766 }827 }
@@ -780,7 +841,7 @@ unlock_reservations: · 13 unmodified lines
780 841
781static int ivpu_submit(struct drm_file *file, struct ivpu_file_priv *file_priv, u32 cmdq_id,842static int ivpu_submit(struct drm_file *file, struct ivpu_file_priv *file_priv, u32 cmdq_id,
782 u32 buffer_count, u32 engine, void __user *buffers_ptr, u32 cmds_offset,843 u32 buffer_count, u32 engine, void __user *buffers_ptr, u32 cmds_offset,
783 u8 priority)844 u32 preempt_buffer_index, u8 priority)
784{845{
785 struct ivpu_device *vdev = file_priv->vdev;846 struct ivpu_device *vdev = file_priv->vdev;
786 struct ivpu_job *job;847 struct ivpu_job *job;
@@ -807,16 +868,14 @@ static int ivpu_submit(struct drm_file *file, struct ivpu_file_priv *file_priv, · 20 unmodified lines
807 868
808 job = ivpu_job_create(file_priv, engine, buffer_count);869 job = ivpu_job_create(file_priv, engine, buffer_count);
809 if (!job) {870 if (!job) {
810 ivpu_err(vdev, "Failed to create job\n");
811 ret = -ENOMEM;871 ret = -ENOMEM;
812 goto err_exit_dev;872 goto err_exit_dev;
813 }873 }
814 874
815 ret = ivpu_job_prepare_bos_for_submit(file, job, buf_handles, buffer_count, cmds_offset);875 ret = ivpu_job_prepare_bos_for_submit(file, job, buf_handles, buffer_count, cmds_offset,
816 if (ret) {876 preempt_buffer_index);
817 ivpu_err(vdev, "Failed to prepare job: %d\n", ret);877 if (ret)
818 goto err_destroy_job;878 goto err_destroy_job;
819 }
820 879
821 down_read(&vdev->pm->reset_lock);880 down_read(&vdev->pm->reset_lock);
822 ret = ivpu_job_submit(job, priority, cmdq_id);881 ret = ivpu_job_submit(job, priority, cmdq_id);
@@ -842,58 +901,91 @@ err_free_handles: · 19 unmodified lines
842int ivpu_submit_ioctl(struct drm_device *dev, void *data, struct drm_file *file)901int ivpu_submit_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
843{902{
844 struct ivpu_file_priv *file_priv = file->driver_priv;903 struct ivpu_file_priv *file_priv = file->driver_priv;
904 struct ivpu_device *vdev = file_priv->vdev;
845 struct drm_ivpu_submit *args = data;905 struct drm_ivpu_submit *args = data;
846 u8 priority;906 u8 priority;
847 907
848 if (args->engine != DRM_IVPU_ENGINE_COMPUTE)908 if (args->engine != DRM_IVPU_ENGINE_COMPUTE) {
909 ivpu_dbg(vdev, IOCTL, "Invalid engine %d\n", args->engine);
849 return -EINVAL;910 return -EINVAL;
911 }
850 912
851 if (args->priority > DRM_IVPU_JOB_PRIORITY_REALTIME)913 if (args->priority > DRM_IVPU_JOB_PRIORITY_REALTIME) {
914 ivpu_dbg(vdev, IOCTL, "Invalid priority %d\n", args->priority);
852 return -EINVAL;915 return -EINVAL;
916 }
853 917
854 if (args->buffer_count == 0 || args->buffer_count > JOB_MAX_BUFFER_COUNT)918 if (args->buffer_count == 0 || args->buffer_count > JOB_MAX_BUFFER_COUNT) {
919 ivpu_dbg(vdev, IOCTL, "Invalid buffer count %u\n", args->buffer_count);
855 return -EINVAL;920 return -EINVAL;
921 }
856 922
857 if (!IS_ALIGNED(args->commands_offset, 8))923 if (!IS_ALIGNED(args->commands_offset, 8)) {
924 ivpu_dbg(vdev, IOCTL, "Invalid commands offset %u\n", args->commands_offset);
858 return -EINVAL;925 return -EINVAL;
926 }
859 927
860 if (!file_priv->ctx.id)928 if (!file_priv->ctx.id) {
929 ivpu_dbg(vdev, IOCTL, "Context not initialized\n");
861 return -EINVAL;930 return -EINVAL;
931 }
862 932
863 if (file_priv->has_mmu_faults)933 if (file_priv->has_mmu_faults) {
934 ivpu_dbg(vdev, IOCTL, "Context %u has MMU faults\n", file_priv->ctx.id);
864 return -EBADFD;935 return -EBADFD;
936 }
865 937
866 priority = ivpu_job_to_jsm_priority(args->priority);938 priority = ivpu_job_to_jsm_priority(args->priority);
867 939
868 return ivpu_submit(file, file_priv, 0, args->buffer_count, args->engine,940 return ivpu_submit(file, file_priv, 0, args->buffer_count, args->engine,
869 (void __user *)args->buffers_ptr, args->commands_offset, priority);941 (void __user *)args->buffers_ptr, args->commands_offset, 0, priority);
870}942}
871 943
872int ivpu_cmdq_submit_ioctl(struct drm_device *dev, void *data, struct drm_file *file)944int ivpu_cmdq_submit_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
873{945{
874 struct ivpu_file_priv *file_priv = file->driver_priv;946 struct ivpu_file_priv *file_priv = file->driver_priv;
947 struct ivpu_device *vdev = file_priv->vdev;
875 struct drm_ivpu_cmdq_submit *args = data;948 struct drm_ivpu_cmdq_submit *args = data;
876 949
877 if (!ivpu_is_capable(file_priv->vdev, DRM_IVPU_CAP_MANAGE_CMDQ))950 if (!ivpu_is_capable(file_priv->vdev, DRM_IVPU_CAP_MANAGE_CMDQ)) {
951 ivpu_dbg(vdev, IOCTL, "Command queue management not supported\n");
878 return -ENODEV;952 return -ENODEV;
953 }
879 954
880 if (args->cmdq_id < IVPU_CMDQ_MIN_ID || args->cmdq_id > IVPU_CMDQ_MAX_ID)955 if (args->cmdq_id < IVPU_CMDQ_MIN_ID || args->cmdq_id > IVPU_CMDQ_MAX_ID) {
956 ivpu_dbg(vdev, IOCTL, "Invalid command queue ID %u\n", args->cmdq_id);
881 return -EINVAL;957 return -EINVAL;
958 }
882 959
883 if (args->buffer_count == 0 || args->buffer_count > JOB_MAX_BUFFER_COUNT)960 if (args->buffer_count == 0 || args->buffer_count > JOB_MAX_BUFFER_COUNT) {
961 ivpu_dbg(vdev, IOCTL, "Invalid buffer count %u\n", args->buffer_count);
884 return -EINVAL;962 return -EINVAL;
963 }
885 964
886 if (!IS_ALIGNED(args->commands_offset, 8))965 if (args->preempt_buffer_index >= args->buffer_count) {
966 ivpu_dbg(vdev, IOCTL, "Invalid preemption buffer index %u\n",
967 args->preempt_buffer_index);
887 return -EINVAL;968 return -EINVAL;
969 }
888 970
889 if (!file_priv->ctx.id)971 if (!IS_ALIGNED(args->commands_offset, 8)) {
972 ivpu_dbg(vdev, IOCTL, "Invalid commands offset %u\n", args->commands_offset);
890 return -EINVAL;973 return -EINVAL;
974 }
891 975
892 if (file_priv->has_mmu_faults)976 if (!file_priv->ctx.id) {
977 ivpu_dbg(vdev, IOCTL, "Context not initialized\n");
978 return -EINVAL;
979 }
980
981 if (file_priv->has_mmu_faults) {
982 ivpu_dbg(vdev, IOCTL, "Context %u has MMU faults\n", file_priv->ctx.id);
893 return -EBADFD;983 return -EBADFD;
984 }
894 985
895 return ivpu_submit(file, file_priv, args->cmdq_id, args->buffer_count, VPU_ENGINE_COMPUTE,986 return ivpu_submit(file, file_priv, args->cmdq_id, args->buffer_count, VPU_ENGINE_COMPUTE,
896 (void __user *)args->buffers_ptr, args->commands_offset, 0);987 (void __user *)args->buffers_ptr, args->commands_offset,
988 args->preempt_buffer_index, 0);
897}989}
898 990
899int ivpu_cmdq_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file)991int ivpu_cmdq_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
@@ -904,11 +996,15 @@ int ivpu_cmdq_create_ioctl(struct drm_device *dev, void *data, struct drm_file * · 4 unmodified lines
904 struct ivpu_cmdq *cmdq;996 struct ivpu_cmdq *cmdq;
905 int ret;997 int ret;
906 998
907 if (!ivpu_is_capable(vdev, DRM_IVPU_CAP_MANAGE_CMDQ))999 if (!ivpu_is_capable(vdev, DRM_IVPU_CAP_MANAGE_CMDQ)) {
1000 ivpu_dbg(vdev, IOCTL, "Command queue management not supported\n");
908 return -ENODEV;1001 return -ENODEV;
1002 }
909 1003
910 if (args->priority > DRM_IVPU_JOB_PRIORITY_REALTIME)1004 if (args->priority > DRM_IVPU_JOB_PRIORITY_REALTIME) {
1005 ivpu_dbg(vdev, IOCTL, "Invalid priority %d\n", args->priority);
911 return -EINVAL;1006 return -EINVAL;
1007 }
912 1008
913 ret = ivpu_rpm_get(vdev);1009 ret = ivpu_rpm_get(vdev);
914 if (ret < 0)1010 if (ret < 0)
@@ -936,8 +1032,10 @@ int ivpu_cmdq_destroy_ioctl(struct drm_device *dev, void *data, struct drm_file · 21 unmodified lines
936 u32 cmdq_id = 0;1032 u32 cmdq_id = 0;
937 int ret;1033 int ret;
938 1034
939 if (!ivpu_is_capable(vdev, DRM_IVPU_CAP_MANAGE_CMDQ))1035 if (!ivpu_is_capable(vdev, DRM_IVPU_CAP_MANAGE_CMDQ)) {
1036 ivpu_dbg(vdev, IOCTL, "Command queue management not supported\n");
940 return -ENODEV;1037 return -ENODEV;
1038 }
941 1039
942 ret = ivpu_rpm_get(vdev);1040 ret = ivpu_rpm_get(vdev);
943 if (ret < 0)1041 if (ret < 0)
@@ -984,7 +1082,9 @@ ivpu_job_done_callback(struct ivpu_device *vdev, struct ivpu_ipc_hdr *ipc_hdr, · 40 unmodified lines
984 payload = (struct vpu_ipc_msg_payload_job_done *)&jsm_msg->payload;1082 payload = (struct vpu_ipc_msg_payload_job_done *)&jsm_msg->payload;
985 1083
986 mutex_lock(&vdev->submitted_jobs_lock);1084 mutex_lock(&vdev->submitted_jobs_lock);
987 ivpu_job_signal_and_destroy(vdev, payload->job_id, payload->job_status);1085 if (!ivpu_job_handle_engine_error(vdev, payload->job_id, payload->job_status))
1086 /* No engine error, complete the job normally */
1087 ivpu_job_signal_and_destroy(vdev, payload->job_id, payload->job_status);
988 mutex_unlock(&vdev->submitted_jobs_lock);1088 mutex_unlock(&vdev->submitted_jobs_lock);
989}1089}
990 1090
@@ -1012,7 +1112,7 @@ void ivpu_context_abort_work_fn(struct work_struct *work) · 21 unmodified lines
1012 1112
1013 if (vdev->fw->sched_mode == VPU_SCHEDULING_MODE_HW)1113 if (vdev->fw->sched_mode == VPU_SCHEDULING_MODE_HW)
1014 if (ivpu_jsm_reset_engine(vdev, 0))1114 if (ivpu_jsm_reset_engine(vdev, 0))
1015 return;1115 goto runtime_put;
1016 1116
1017 mutex_lock(&vdev->context_list_lock);1117 mutex_lock(&vdev->context_list_lock);
1018 xa_for_each(&vdev->context_xa, ctx_id, file_priv) {1118 xa_for_each(&vdev->context_xa, ctx_id, file_priv) {
@@ -1036,7 +1136,7 @@ void ivpu_context_abort_work_fn(struct work_struct *work) · 17 unmodified lines
1036 goto runtime_put;1136 goto runtime_put;
1037 1137
1038 if (ivpu_jsm_hws_resume_engine(vdev, 0))1138 if (ivpu_jsm_hws_resume_engine(vdev, 0))
1039 return;1139 goto runtime_put;
1040 /*1140 /*
1041 * In hardware scheduling mode NPU already has stopped processing jobs1141 * In hardware scheduling mode NPU already has stopped processing jobs
1042 * and won't send us any further notifications, thus we have to free job related resources1142 * and won't send us any further notifications, thus we have to free job related resources
@@ -1049,6 +1149,5 @@ void ivpu_context_abort_work_fn(struct work_struct *work) · 6 unmodified lines
1049 mutex_unlock(&vdev->submitted_jobs_lock);1149 mutex_unlock(&vdev->submitted_jobs_lock);
1050 1150
1051runtime_put:1151runtime_put:
1052 pm_runtime_mark_last_busy(vdev->drm.dev);
1053 pm_runtime_put_autosuspend(vdev->drm.dev);1152 pm_runtime_put_autosuspend(vdev->drm.dev);
1054}1153}
drivers/accel/ivpu/ivpu_job.h+32−17
@@ -1,6 +1,6 @@
1/* SPDX-License-Identifier: GPL-2.0-only */1/* SPDX-License-Identifier: GPL-2.0-only */
2/*2/*
3 * Copyright (C) 2020-2024 Intel Corporation3 * Copyright (C) 2020-2025 Intel Corporation
4 */4 */
5 5
6#ifndef __IVPU_JOB_H__6#ifndef __IVPU_JOB_H__
@@ -15,12 +15,17 @@ struct ivpu_device; · 8 unmodified lines
15struct ivpu_file_priv;15struct ivpu_file_priv;
16 16
17/**17/**
18 * struct ivpu_cmdq - Object representing device queue used to send jobs.18 * struct ivpu_cmdq - Represents a command queue for submitting jobs to the VPU.
19 * @jobq: Pointer to job queue memory shared with the device19 * Tracks queue memory, preemption buffers, and metadata for job management.
20 * @mem: Memory allocated for the job queue, shared with device20 * @jobq: Pointer to job queue memory shared with the device
21 * @entry_count Number of job entries in the queue21 * @primary_preempt_buf: Primary preemption buffer for this queue (optional)
22 * @db_id: Doorbell assigned to this job queue22 * @secondary_preempt_buf: Secondary preemption buffer for this queue (optional)
23 * @db_registered: True if doorbell is registered in device23 * @mem: Memory allocated for the job queue, shared with device
24 * @entry_count: Number of job entries in the queue
25 * @id: Unique command queue ID
26 * @db_id: Doorbell ID assigned to this job queue
27 * @priority: Priority level of the command queue
28 * @is_legacy: True if this is a legacy command queue
24 */29 */
25struct ivpu_cmdq {30struct ivpu_cmdq {
26 struct vpu_job_queue *jobq;31 struct vpu_job_queue *jobq;
@@ -35,16 +40,22 @@ struct ivpu_cmdq { · 8 unmodified lines
35};40};
36 41
37/**42/**
38 * struct ivpu_job - KMD object that represents batchbuffer / DMA buffer.43 * struct ivpu_job - Representing a batch or DMA buffer submitted to the VPU.
39 * Each batch / DMA buffer is a job to be submitted and executed by the VPU FW.44 * Each job is a unit of execution, tracked by job_id for status reporting from VPU FW.
40 * This is a unit of execution, and be tracked by the job_id for45 * The structure holds all resources and metadata needed for job submission, execution,
41 * any status reporting from VPU FW through IPC JOB RET/DONE message.46 * and completion handling.
42 * @file_priv: The client that submitted this job47 * @vdev: Pointer to the VPU device
43 * @job_id: Job ID for KMD tracking and job status reporting from VPU FW48 * @file_priv: The client context that submitted this job
44 * @status: Status of the Job from IPC JOB RET/DONE message49 * @done_fence: Fence signaled when job completes
45 * @batch_buffer: CPU vaddr points to the batch buffer memory allocated for the job50 * @cmd_buf_vpu_addr: VPU address of the command buffer for this job
46 * @submit_status_offset: Offset within batch buffer where job completion handler51 * @cmdq_id: Command queue ID used for submission
47 will update the job status52 * @job_id: Unique job ID for tracking and status reporting
53 * @engine_idx: Engine index for job execution
54 * @job_status: Status reported by firmware for this job
55 * @primary_preempt_buf: Primary preemption buffer for job
56 * @secondary_preempt_buf: Secondary preemption buffer for job (optional)
57 * @bo_count: Number of buffer objects associated with this job
58 * @bos: Array of buffer objects used by the job (batch buffer is at index 0)
48 */59 */
49struct ivpu_job {60struct ivpu_job {
50 struct ivpu_device *vdev;61 struct ivpu_device *vdev;
@@ -54,6 +65,9 @@ struct ivpu_job { · 3 unmodified lines
54 u32 cmdq_id;65 u32 cmdq_id;
55 u32 job_id;66 u32 job_id;
56 u32 engine_idx;67 u32 engine_idx;
68 u32 job_status;
69 struct ivpu_bo *primary_preempt_buf;
70 struct ivpu_bo *secondary_preempt_buf;
57 size_t bo_count;71 size_t bo_count;
58 struct ivpu_bo *bos[] __counted_by(bo_count);72 struct ivpu_bo *bos[] __counted_by(bo_count);
59};73};
@@ -71,6 +85,7 @@ void ivpu_cmdq_abort_all_jobs(struct ivpu_device *vdev, u32 ctx_id, u32 cmdq_id) · 11 unmodified lines
71 85
72void ivpu_job_done_consumer_init(struct ivpu_device *vdev);86void ivpu_job_done_consumer_init(struct ivpu_device *vdev);
73void ivpu_job_done_consumer_fini(struct ivpu_device *vdev);87void ivpu_job_done_consumer_fini(struct ivpu_device *vdev);
88bool ivpu_job_handle_engine_error(struct ivpu_device *vdev, u32 job_id, u32 job_status);
74void ivpu_context_abort_work_fn(struct work_struct *work);89void ivpu_context_abort_work_fn(struct work_struct *work);
75 90
76void ivpu_jobs_abort_all(struct ivpu_device *vdev);91void ivpu_jobs_abort_all(struct ivpu_device *vdev);
drivers/accel/ivpu/ivpu_mmu.c+1−1
@@ -970,7 +970,7 @@ void ivpu_mmu_irq_evtq_handler(struct ivpu_device *vdev) · 969 unmodified lines
970 }970 }
971 }971 }
972 972
973 queue_work(system_wq, &vdev->context_abort_work);973 queue_work(system_percpu_wq, &vdev->context_abort_work);
974}974}
975 975
976void ivpu_mmu_evtq_dump(struct ivpu_device *vdev)976void ivpu_mmu_evtq_dump(struct ivpu_device *vdev)
drivers/accel/ivpu/ivpu_mmu_context.c+6−3
@@ -430,7 +430,7 @@ static void ivpu_mmu_context_unmap_pages(struct ivpu_mmu_context *ctx, u64 vpu_a · 429 unmodified lines
430 430
431int431int
432ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,432ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,
433 u64 vpu_addr, struct sg_table *sgt, bool llc_coherent)433 u64 vpu_addr, struct sg_table *sgt, bool llc_coherent, bool read_only)
434{434{
435 size_t start_vpu_addr = vpu_addr;435 size_t start_vpu_addr = vpu_addr;
436 struct scatterlist *sg;436 struct scatterlist *sg;
@@ -450,6 +450,8 @@ ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, · 13 unmodified lines
450 prot = IVPU_MMU_ENTRY_MAPPED;450 prot = IVPU_MMU_ENTRY_MAPPED;
451 if (llc_coherent)451 if (llc_coherent)
452 prot |= IVPU_MMU_ENTRY_FLAG_LLC_COHERENT;452 prot |= IVPU_MMU_ENTRY_FLAG_LLC_COHERENT;
453 if (read_only)
454 prot |= IVPU_MMU_ENTRY_FLAG_RO;
453 455
454 mutex_lock(&ctx->lock);456 mutex_lock(&ctx->lock);
455 457
@@ -527,7 +529,8 @@ ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ct · 71 unmodified lines
527 529
528 ret = ivpu_mmu_invalidate_tlb(vdev, ctx->id);530 ret = ivpu_mmu_invalidate_tlb(vdev, ctx->id);
529 if (ret)531 if (ret)
530 ivpu_warn(vdev, "Failed to invalidate TLB for ctx %u: %d\n", ctx->id, ret);532 ivpu_warn_ratelimited(vdev, "Failed to invalidate TLB for ctx %u: %d\n",
533 ctx->id, ret);
531}534}
532 535
533int536int
@@ -568,7 +571,7 @@ void ivpu_mmu_context_init(struct ivpu_device *vdev, struct ivpu_mmu_context *ct · 34 unmodified lines
568 mutex_init(&ctx->lock);571 mutex_init(&ctx->lock);
569 572
570 if (!context_id) {573 if (!context_id) {
571 start = vdev->hw->ranges.global.start;574 start = vdev->hw->ranges.runtime.start;
572 end = vdev->hw->ranges.shave.end;575 end = vdev->hw->ranges.shave.end;
573 } else {576 } else {
574 start = min_t(u64, vdev->hw->ranges.user.start, vdev->hw->ranges.shave.start);577 start = min_t(u64, vdev->hw->ranges.user.start, vdev->hw->ranges.shave.start);
drivers/accel/ivpu/ivpu_mmu_context.h+1−1
@@ -42,7 +42,7 @@ int ivpu_mmu_context_insert_node(struct ivpu_mmu_context *ctx, const struct ivpu · 41 unmodified lines
42void ivpu_mmu_context_remove_node(struct ivpu_mmu_context *ctx, struct drm_mm_node *node);42void ivpu_mmu_context_remove_node(struct ivpu_mmu_context *ctx, struct drm_mm_node *node);
43 43
44int ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,44int ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,
45 u64 vpu_addr, struct sg_table *sgt, bool llc_coherent);45 u64 vpu_addr, struct sg_table *sgt, bool llc_coherent, bool read_only);
46void ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,46void ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,
47 u64 vpu_addr, struct sg_table *sgt);47 u64 vpu_addr, struct sg_table *sgt);
48int ivpu_mmu_context_set_pages_ro(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,48int ivpu_mmu_context_set_pages_ro(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx,
drivers/accel/ivpu/ivpu_ms.c+17−8
@@ -8,6 +8,7 @@ · 7 unmodified lines
8 8
9#include "ivpu_drv.h"9#include "ivpu_drv.h"
10#include "ivpu_gem.h"10#include "ivpu_gem.h"
11#include "ivpu_hw.h"
11#include "ivpu_jsm_msg.h"12#include "ivpu_jsm_msg.h"
12#include "ivpu_ms.h"13#include "ivpu_ms.h"
13#include "ivpu_pm.h"14#include "ivpu_pm.h"
@@ -37,8 +38,8 @@ int ivpu_ms_start_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 23 unmodified lines
37 struct drm_ivpu_metric_streamer_start *args = data;38 struct drm_ivpu_metric_streamer_start *args = data;
38 struct ivpu_device *vdev = file_priv->vdev;39 struct ivpu_device *vdev = file_priv->vdev;
39 struct ivpu_ms_instance *ms;40 struct ivpu_ms_instance *ms;
40 u64 single_buff_size;
41 u32 sample_size;41 u32 sample_size;
42 u64 buf_size;
42 int ret;43 int ret;
43 44
44 if (!args->metric_group_mask || !args->read_period_samples ||45 if (!args->metric_group_mask || !args->read_period_samples ||
@@ -52,7 +53,8 @@ int ivpu_ms_start_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 7 unmodified lines
52 mutex_lock(&file_priv->ms_lock);53 mutex_lock(&file_priv->ms_lock);
53 54
54 if (get_instance_by_mask(file_priv, args->metric_group_mask)) {55 if (get_instance_by_mask(file_priv, args->metric_group_mask)) {
55 ivpu_err(vdev, "Instance already exists (mask %#llx)\n", args->metric_group_mask);56 ivpu_dbg(vdev, IOCTL, "Instance already exists (mask %#llx)\n",
57 args->metric_group_mask);
56 ret = -EALREADY;58 ret = -EALREADY;
57 goto unlock;59 goto unlock;
58 }60 }
@@ -69,12 +71,18 @@ int ivpu_ms_start_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 10 unmodified lines
69 if (ret)71 if (ret)
70 goto err_free_ms;72 goto err_free_ms;
71 73
72 single_buff_size = sample_size *74 buf_size = PAGE_ALIGN((u64)args->read_period_samples * sample_size *
73 ((u64)args->read_period_samples * MS_READ_PERIOD_MULTIPLIER);75 MS_READ_PERIOD_MULTIPLIER * MS_NUM_BUFFERS);
74 ms->bo = ivpu_bo_create_global(vdev, PAGE_ALIGN(single_buff_size * MS_NUM_BUFFERS),76 if (buf_size > ivpu_hw_range_size(&vdev->hw->ranges.global)) {
75 DRM_IVPU_BO_CACHED | DRM_IVPU_BO_MAPPABLE);77 ivpu_dbg(vdev, IOCTL, "Requested MS buffer size %llu exceeds range size %llu\n",
78 buf_size, ivpu_hw_range_size(&vdev->hw->ranges.global));
79 ret = -EINVAL;
80 goto err_free_ms;
81 }
82
83 ms->bo = ivpu_bo_create_global(vdev, buf_size, DRM_IVPU_BO_CACHED | DRM_IVPU_BO_MAPPABLE);
76 if (!ms->bo) {84 if (!ms->bo) {
77 ivpu_err(vdev, "Failed to allocate MS buffer (size %llu)\n", single_buff_size);85 ivpu_dbg(vdev, IOCTL, "Failed to allocate MS buffer (size %llu)\n", buf_size);
78 ret = -ENOMEM;86 ret = -ENOMEM;
79 goto err_free_ms;87 goto err_free_ms;
80 }88 }
@@ -175,7 +183,8 @@ int ivpu_ms_get_data_ioctl(struct drm_device *dev, void *data, struct drm_file * · 94 unmodified lines
175 183
176 ms = get_instance_by_mask(file_priv, args->metric_group_mask);184 ms = get_instance_by_mask(file_priv, args->metric_group_mask);
177 if (!ms) {185 if (!ms) {
178 ivpu_err(vdev, "Instance doesn't exist for mask: %#llx\n", args->metric_group_mask);186 ivpu_dbg(vdev, IOCTL, "Instance doesn't exist for mask: %#llx\n",
187 args->metric_group_mask);
179 ret = -EINVAL;188 ret = -EINVAL;
180 goto unlock;189 goto unlock;
181 }190 }
drivers/accel/ivpu/ivpu_pm.c+11−7
@@ -54,7 +54,7 @@ static void ivpu_pm_prepare_cold_boot(struct ivpu_device *vdev) · 53 unmodified lines
54static void ivpu_pm_prepare_warm_boot(struct ivpu_device *vdev)54static void ivpu_pm_prepare_warm_boot(struct ivpu_device *vdev)
55{55{
56 struct ivpu_fw_info *fw = vdev->fw;56 struct ivpu_fw_info *fw = vdev->fw;
57 struct vpu_boot_params *bp = ivpu_bo_vaddr(fw->mem);57 struct vpu_boot_params *bp = ivpu_bo_vaddr(fw->mem_bp);
58 58
59 if (!bp->save_restore_ret_address) {59 if (!bp->save_restore_ret_address) {
60 ivpu_pm_prepare_cold_boot(vdev);60 ivpu_pm_prepare_cold_boot(vdev);
@@ -186,7 +186,7 @@ void ivpu_pm_trigger_recovery(struct ivpu_device *vdev, const char *reason) · 125 unmodified lines
186 if (atomic_cmpxchg(&vdev->pm->reset_pending, 0, 1) == 0) {186 if (atomic_cmpxchg(&vdev->pm->reset_pending, 0, 1) == 0) {
187 ivpu_hw_diagnose_failure(vdev);187 ivpu_hw_diagnose_failure(vdev);
188 ivpu_hw_irq_disable(vdev); /* Disable IRQ early to protect from IRQ storm */188 ivpu_hw_irq_disable(vdev); /* Disable IRQ early to protect from IRQ storm */
189 queue_work(system_unbound_wq, &vdev->pm->recovery_work);189 queue_work(system_dfl_wq, &vdev->pm->recovery_work);
190 }190 }
191}191}
192 192
@@ -226,7 +226,8 @@ void ivpu_start_job_timeout_detection(struct ivpu_device *vdev) · 33 unmodified lines
226 unsigned long timeout_ms = ivpu_tdr_timeout_ms ? ivpu_tdr_timeout_ms : vdev->timeout.tdr;226 unsigned long timeout_ms = ivpu_tdr_timeout_ms ? ivpu_tdr_timeout_ms : vdev->timeout.tdr;
227 227
228 /* No-op if already queued */228 /* No-op if already queued */
229 queue_delayed_work(system_wq, &vdev->pm->job_timeout_work, msecs_to_jiffies(timeout_ms));229 queue_delayed_work(system_percpu_wq, &vdev->pm->job_timeout_work,
230 msecs_to_jiffies(timeout_ms));
230}231}
231 232
232void ivpu_stop_job_timeout_detection(struct ivpu_device *vdev)233void ivpu_stop_job_timeout_detection(struct ivpu_device *vdev)
@@ -359,7 +360,6 @@ int ivpu_rpm_get(struct ivpu_device *vdev) · 126 unmodified lines
359 360
360void ivpu_rpm_put(struct ivpu_device *vdev)361void ivpu_rpm_put(struct ivpu_device *vdev)
361{362{
362 pm_runtime_mark_last_busy(vdev->drm.dev);
363 pm_runtime_put_autosuspend(vdev->drm.dev);363 pm_runtime_put_autosuspend(vdev->drm.dev);
364}364}
365 365
@@ -428,7 +428,6 @@ void ivpu_pm_enable(struct ivpu_device *vdev) · 62 unmodified lines
428 struct device *dev = vdev->drm.dev;428 struct device *dev = vdev->drm.dev;
429 429
430 pm_runtime_allow(dev);430 pm_runtime_allow(dev);
431 pm_runtime_mark_last_busy(dev);
432 pm_runtime_put_autosuspend(dev);431 pm_runtime_put_autosuspend(dev);
433}432}
434 433
@@ -502,6 +501,11 @@ void ivpu_pm_irq_dct_work_fn(struct work_struct *work) · 67 unmodified lines
502 else501 else
503 ret = ivpu_pm_dct_disable(vdev);502 ret = ivpu_pm_dct_disable(vdev);
504 503
505 if (!ret)504 if (!ret) {
506 ivpu_hw_btrs_dct_set_status(vdev, enable, vdev->pm->dct_active_percent);505 /* Convert percent to U1.7 format */
506 u8 val = DIV_ROUND_CLOSEST(vdev->pm->dct_active_percent * 128, 100);
507
508 ivpu_hw_btrs_dct_set_status(vdev, enable, val);
509 }
510
507}511}
drivers/accel/ivpu/ivpu_sysfs.c+2−1
@@ -63,7 +63,8 @@ npu_memory_utilization_show(struct device *dev, struct device_attribute *attr, c · 62 unmodified lines
63 63
64 mutex_lock(&vdev->bo_list_lock);64 mutex_lock(&vdev->bo_list_lock);
65 list_for_each_entry(bo, &vdev->bo_list, bo_list_node)65 list_for_each_entry(bo, &vdev->bo_list, bo_list_node)
66 total_npu_memory += bo->base.base.size;66 if (ivpu_bo_is_resident(bo))
67 total_npu_memory += ivpu_bo_size(bo);
67 mutex_unlock(&vdev->bo_list_lock);68 mutex_unlock(&vdev->bo_list_lock);
68 69
69 return sysfs_emit(buf, "%lld\n", total_npu_memory);70 return sysfs_emit(buf, "%lld\n", total_npu_memory);
drivers/accel/ivpu/vpu_jsm_api.h+417−236
@@ -1,15 +1,16 @@
1/* SPDX-License-Identifier: MIT */1/* SPDX-License-Identifier: MIT */
2/*2/*
3 * Copyright (c) 2020-2024, Intel Corporation.3 * Copyright (c) 2020-2025, Intel Corporation.
4 */
5
6/**
7 * @addtogroup Jsm
8 * @{
4 */9 */
5 10
6/**11/**
7 * @file12 * @file
8 * @brief JSM shared definitions13 * @brief JSM shared definitions
9 *
10 * @ingroup Jsm
11 * @brief JSM shared definitions
12 * @{
13 */14 */
14#ifndef VPU_JSM_API_H15#ifndef VPU_JSM_API_H
15#define VPU_JSM_API_H16#define VPU_JSM_API_H
@@ -22,7 +23,7 @@ · 6 unmodified lines
22/*23/*
23 * Minor version changes when API backward compatibility is preserved.24 * Minor version changes when API backward compatibility is preserved.
24 */25 */
25#define VPU_JSM_API_VER_MINOR 2926#define VPU_JSM_API_VER_MINOR 33
26 27
27/*28/*
28 * API header changed (field names, documentation, formatting) but API itself has not been changed29 * API header changed (field names, documentation, formatting) but API itself has not been changed
@@ -71,9 +72,15 @@ · 42 unmodified lines
71#define VPU_JSM_STATUS_MVNCI_OUT_OF_RESOURCES 0xAU72#define VPU_JSM_STATUS_MVNCI_OUT_OF_RESOURCES 0xAU
72#define VPU_JSM_STATUS_MVNCI_NOT_IMPLEMENTED 0xBU73#define VPU_JSM_STATUS_MVNCI_NOT_IMPLEMENTED 0xBU
73#define VPU_JSM_STATUS_MVNCI_INTERNAL_ERROR 0xCU74#define VPU_JSM_STATUS_MVNCI_INTERNAL_ERROR 0xCU
74/* Job status returned when the job was preempted mid-inference */75/* @deprecated (use VPU_JSM_STATUS_PREEMPTED_MID_COMMAND instead) */
75#define VPU_JSM_STATUS_PREEMPTED_MID_INFERENCE 0xDU76#define VPU_JSM_STATUS_PREEMPTED_MID_INFERENCE 0xDU
77/* Job status returned when the job was preempted mid-command */
78#define VPU_JSM_STATUS_PREEMPTED_MID_COMMAND 0xDU
79/* Range of status codes that require engine reset */
80#define VPU_JSM_STATUS_ENGINE_RESET_REQUIRED_MIN 0xEU
76#define VPU_JSM_STATUS_MVNCI_CONTEXT_VIOLATION_HW 0xEU81#define VPU_JSM_STATUS_MVNCI_CONTEXT_VIOLATION_HW 0xEU
82#define VPU_JSM_STATUS_MVNCI_PREEMPTION_TIMED_OUT 0xFU
83#define VPU_JSM_STATUS_ENGINE_RESET_REQUIRED_MAX 0x1FU
77 84
78/*85/*
79 * Host <-> VPU IPC channels.86 * Host <-> VPU IPC channels.
@@ -134,11 +141,21 @@ enum { · 54 unmodified lines
134 * 2. Native fence queues are only supported on VPU 40xx onwards.141 * 2. Native fence queues are only supported on VPU 40xx onwards.
135 */142 */
136 VPU_JOB_QUEUE_FLAGS_USE_NATIVE_FENCE_MASK = (1 << 1U),143 VPU_JOB_QUEUE_FLAGS_USE_NATIVE_FENCE_MASK = (1 << 1U),
137
138 /*144 /*
139 * Enable turbo mode for testing NPU performance; not recommended for regular usage.145 * Enable turbo mode for testing NPU performance; not recommended for regular usage.
140 */146 */
141 VPU_JOB_QUEUE_FLAGS_TURBO_MODE = (1 << 2U)147 VPU_JOB_QUEUE_FLAGS_TURBO_MODE = (1 << 2U),
148 /*
149 * Queue error detection mode flag
150 * For 'interactive' queues (this bit not set), the FW will identify queues that have not
151 * completed a job inside the TDR timeout as in error as part of engine reset sequence.
152 * For 'non-interactive' queues (this bit set), the FW will identify queues that have not
153 * progressed the heartbeat inside the non-interactive no-progress timeout as in error as
154 * part of engine reset sequence. Additionally, there is an upper limit applied to these
155 * queues: even if they progress the heartbeat, if they run longer than non-interactive
156 * timeout, then the FW will also identify them as in error.
157 */
158 VPU_JOB_QUEUE_FLAGS_NON_INTERACTIVE = (1 << 3U)
142};159};
143 160
144/*161/*
@@ -209,7 +226,7 @@ enum { · 64 unmodified lines
209 */226 */
210#define VPU_INLINE_CMD_TYPE_FENCE_SIGNAL 0x2227#define VPU_INLINE_CMD_TYPE_FENCE_SIGNAL 0x2
211 228
212/*229/**
213 * Job scheduling priority bands for both hardware scheduling and OS scheduling.230 * Job scheduling priority bands for both hardware scheduling and OS scheduling.
214 */231 */
215enum vpu_job_scheduling_priority_band {232enum vpu_job_scheduling_priority_band {
@@ -220,16 +237,16 @@ enum vpu_job_scheduling_priority_band { · 4 unmodified lines
220 VPU_JOB_SCHEDULING_PRIORITY_BAND_COUNT = 4,237 VPU_JOB_SCHEDULING_PRIORITY_BAND_COUNT = 4,
221};238};
222 239
223/*240/**
224 * Job format.241 * Job format.
225 * Jobs defines the actual workloads to be executed by a given engine.242 * Jobs defines the actual workloads to be executed by a given engine.
226 */243 */
227struct vpu_job_queue_entry {244struct vpu_job_queue_entry {
228 /**< Address of VPU commands batch buffer */245 /** Address of VPU commands batch buffer */
229 u64 batch_buf_addr;246 u64 batch_buf_addr;
230 /**< Job ID */247 /** Job ID */
231 u32 job_id;248 u32 job_id;
232 /**< Flags bit field, see VPU_JOB_FLAGS_* above */249 /** Flags bit field, see VPU_JOB_FLAGS_* above */
233 u32 flags;250 u32 flags;
234 /**251 /**
235 * Doorbell ring timestamp taken by KMD from SoC's global system clock, in252 * Doorbell ring timestamp taken by KMD from SoC's global system clock, in
@@ -237,20 +254,20 @@ struct vpu_job_queue_entry { · 1 unmodified line
237 * to match other profiling timestamps.254 * to match other profiling timestamps.
238 */255 */
239 u64 doorbell_timestamp;256 u64 doorbell_timestamp;
240 /**< Extra id for job tracking, used only in the firmware perf traces */257 /** Extra id for job tracking, used only in the firmware perf traces */
241 u64 host_tracking_id;258 u64 host_tracking_id;
242 /**< Address of the primary preemption buffer to use for this job */259 /** Address of the primary preemption buffer to use for this job */
243 u64 primary_preempt_buf_addr;260 u64 primary_preempt_buf_addr;
244 /**< Size of the primary preemption buffer to use for this job */261 /** Size of the primary preemption buffer to use for this job */
245 u32 primary_preempt_buf_size;262 u32 primary_preempt_buf_size;
246 /**< Size of secondary preemption buffer to use for this job */263 /** Size of secondary preemption buffer to use for this job */
247 u32 secondary_preempt_buf_size;264 u32 secondary_preempt_buf_size;
248 /**< Address of secondary preemption buffer to use for this job */265 /** Address of secondary preemption buffer to use for this job */
249 u64 secondary_preempt_buf_addr;266 u64 secondary_preempt_buf_addr;
250 u64 reserved_0;267 u64 reserved_0;
251};268};
252 269
253/*270/**
254 * Inline command format.271 * Inline command format.
255 * Inline commands are the commands executed at scheduler level (typically,272 * Inline commands are the commands executed at scheduler level (typically,
256 * synchronization directives). Inline command and job objects must be of273 * synchronization directives). Inline command and job objects must be of
@@ -258,34 +275,36 @@ struct vpu_job_queue_entry { · 1 unmodified line
258 */275 */
259struct vpu_inline_cmd {276struct vpu_inline_cmd {
260 u64 reserved_0;277 u64 reserved_0;
261 /* Inline command type, see VPU_INLINE_CMD_TYPE_* defines. */278 /** Inline command type, see VPU_INLINE_CMD_TYPE_* defines. */
262 u32 type;279 u32 type;
263 /* Flags bit field, see VPU_JOB_FLAGS_* above. */280 /** Flags bit field, see VPU_JOB_FLAGS_* above. */
264 u32 flags;281 u32 flags;
265 /* Inline command payload. Depends on inline command type. */282 /** Inline command payload. Depends on inline command type. */
266 union {283 union payload {
267 /* Fence (wait and signal) commands' payload. */284 /** Fence (wait and signal) commands' payload. */
268 struct {285 struct fence {
269 /* Fence object handle. */286 /** Fence object handle. */
270 u64 fence_handle;287 u64 fence_handle;
271 /* User VA of the current fence value. */288 /** User VA of the current fence value. */
272 u64 current_value_va;289 u64 current_value_va;
273 /* User VA of the monitored fence value (read-only). */290 /** User VA of the monitored fence value (read-only). */
274 u64 monitored_value_va;291 u64 monitored_value_va;
275 /* Value to wait for or write in fence location. */292 /** Value to wait for or write in fence location. */
276 u64 value;293 u64 value;
277 /* User VA of the log buffer in which to add log entry on completion. */294 /** User VA of the log buffer in which to add log entry on completion. */
278 u64 log_buffer_va;295 u64 log_buffer_va;
279 /* NPU private data. */296 /** NPU private data. */
280 u64 npu_private_data;297 u64 npu_private_data;
281 } fence;298 } fence;
282 /* Other commands do not have a payload. */299 /**
283 /* Payload definition for future inline commands can be inserted here. */300 * Other commands do not have a payload:
301 * Payload definition for future inline commands can be inserted here.
302 */
284 u64 reserved_1[6];303 u64 reserved_1[6];
285 } payload;304 } payload;
286};305};
287 306
288/*307/**
289 * Job queue slots can be populated either with job objects or inline command objects.308 * Job queue slots can be populated either with job objects or inline command objects.
290 */309 */
291union vpu_jobq_slot {310union vpu_jobq_slot {
@@ -293,7 +312,7 @@ union vpu_jobq_slot { · 1 unmodified line
293 struct vpu_inline_cmd inline_cmd;312 struct vpu_inline_cmd inline_cmd;
294};313};
295 314
296/*315/**
297 * Job queue control registers.316 * Job queue control registers.
298 */317 */
299struct vpu_job_queue_header {318struct vpu_job_queue_header {
@@ -301,18 +320,18 @@ struct vpu_job_queue_header { · 1 unmodified line
301 u32 head;320 u32 head;
302 u32 tail;321 u32 tail;
303 u32 flags;322 u32 flags;
304 /* Set to 1 to indicate priority_band field is valid */323 /** Set to 1 to indicate priority_band field is valid */
305 u32 priority_band_valid;324 u32 priority_band_valid;
306 /*325 /**
307 * Priority for the work of this job queue, valid only if the HWS is NOT used326 * Priority for the work of this job queue, valid only if the HWS is NOT used
308 * and the `priority_band_valid` is set to 1. It is applied only during327 * and the @ref priority_band_valid is set to 1. It is applied only during
309 * the VPU_JSM_MSG_REGISTER_DB message processing.328 * the @ref VPU_JSM_MSG_REGISTER_DB message processing.
310 * The device firmware might use the `priority_band` to optimize the power329 * The device firmware might use the priority_band to optimize the power
311 * management logic, but it will not affect the order of jobs.330 * management logic, but it will not affect the order of jobs.
312 * Available priority bands: @see enum vpu_job_scheduling_priority_band331 * Available priority bands: @see enum vpu_job_scheduling_priority_band
313 */332 */
314 u32 priority_band;333 u32 priority_band;
315 /* Inside realtime band assigns a further priority, limited to 0..31 range */334 /** Inside realtime band assigns a further priority, limited to 0..31 range */
316 u32 realtime_priority_level;335 u32 realtime_priority_level;
317 u32 reserved_0[9];336 u32 reserved_0[9];
318};337};
@@ -337,16 +356,16 @@ enum vpu_trace_entity_type { · 18 unmodified lines
337 VPU_TRACE_ENTITY_TYPE_HW_COMPONENT = 2,356 VPU_TRACE_ENTITY_TYPE_HW_COMPONENT = 2,
338};357};
339 358
340/*359/**
341 * HWS specific log buffer header details.360 * HWS specific log buffer header details.
342 * Total size is 32 bytes.361 * Total size is 32 bytes.
343 */362 */
344struct vpu_hws_log_buffer_header {363struct vpu_hws_log_buffer_header {
345 /* Written by VPU after adding a log entry. Initialised by host to 0. */364 /** Written by VPU after adding a log entry. Initialised by host to 0. */
346 u32 first_free_entry_index;365 u32 first_free_entry_index;
347 /* Incremented by VPU every time the VPU writes the 0th entry; initialised by host to 0. */366 /** Incremented by VPU every time the VPU writes the 0th entry; initialised by host to 0. */
348 u32 wraparound_count;367 u32 wraparound_count;
349 /*368 /**
350 * This is the number of buffers that can be stored in the log buffer provided by the host.369 * This is the number of buffers that can be stored in the log buffer provided by the host.
351 * It is written by host before passing buffer to VPU. VPU should consider it read-only.370 * It is written by host before passing buffer to VPU. VPU should consider it read-only.
352 */371 */
@@ -354,14 +373,14 @@ struct vpu_hws_log_buffer_header { · 1 unmodified line
354 u64 reserved[2];373 u64 reserved[2];
355};374};
356 375
357/*376/**
358 * HWS specific log buffer entry details.377 * HWS specific log buffer entry details.
359 * Total size is 32 bytes.378 * Total size is 32 bytes.
360 */379 */
361struct vpu_hws_log_buffer_entry {380struct vpu_hws_log_buffer_entry {
362 /* VPU timestamp must be an invariant timer tick (not impacted by DVFS) */381 /** VPU timestamp must be an invariant timer tick (not impacted by DVFS) */
363 u64 vpu_timestamp;382 u64 vpu_timestamp;
364 /*383 /**
365 * Operation type:384 * Operation type:
366 * 0 - context state change385 * 0 - context state change
367 * 1 - queue new work386 * 1 - queue new work
@@ -371,7 +390,7 @@ struct vpu_hws_log_buffer_entry { · 3 unmodified lines
371 */390 */
372 u32 operation_type;391 u32 operation_type;
373 u32 reserved;392 u32 reserved;
374 /* Operation data depends on operation type */393 /** Operation data depends on operation type */
375 u64 operation_data[2];394 u64 operation_data[2];
376};395};
377 396
@@ -381,51 +400,54 @@ enum vpu_hws_native_fence_log_type { · 3 unmodified lines
381 VPU_HWS_NATIVE_FENCE_LOG_TYPE_SIGNALS = 2400 VPU_HWS_NATIVE_FENCE_LOG_TYPE_SIGNALS = 2
382};401};
383 402
384/* HWS native fence log buffer header. */403/** HWS native fence log buffer header. */
385struct vpu_hws_native_fence_log_header {404struct vpu_hws_native_fence_log_header {
386 union {405 union {
387 struct {406 struct {
388 /* Index of the first free entry in buffer. */407 /** Index of the first free entry in buffer. */
389 u32 first_free_entry_idx;408 u32 first_free_entry_idx;
390 /* Incremented each time NPU wraps around the buffer to write next entry. */409 /**
410 * Incremented whenever the NPU wraps around the buffer and writes
411 * to the first entry again.
412 */
391 u32 wraparound_count;413 u32 wraparound_count;
392 };414 };
393 /* Field allowing atomic update of both fields above. */415 /** Field allowing atomic update of both fields above. */
394 u64 atomic_wraparound_and_entry_idx;416 u64 atomic_wraparound_and_entry_idx;
395 };417 };
396 /* Log buffer type, see enum vpu_hws_native_fence_log_type. */418 /** Log buffer type, see enum vpu_hws_native_fence_log_type. */
397 u64 type;419 u64 type;
398 /* Allocated number of entries in the log buffer. */420 /** Allocated number of entries in the log buffer. */
399 u64 entry_nb;421 u64 entry_nb;
400 u64 reserved[2];422 u64 reserved[2];
401};423};
402 424
403/* Native fence log operation types. */425/** Native fence log operation types. */
404enum vpu_hws_native_fence_log_op {426enum vpu_hws_native_fence_log_op {
405 VPU_HWS_NATIVE_FENCE_LOG_OP_SIGNAL_EXECUTED = 0,427 VPU_HWS_NATIVE_FENCE_LOG_OP_SIGNAL_EXECUTED = 0,
406 VPU_HWS_NATIVE_FENCE_LOG_OP_WAIT_UNBLOCKED = 1428 VPU_HWS_NATIVE_FENCE_LOG_OP_WAIT_UNBLOCKED = 1
407};429};
408 430
409/* HWS native fence log entry. */431/** HWS native fence log entry. */
410struct vpu_hws_native_fence_log_entry {432struct vpu_hws_native_fence_log_entry {
411 /* Newly signaled/unblocked fence value. */433 /** Newly signaled/unblocked fence value. */
412 u64 fence_value;434 u64 fence_value;
413 /* Native fence object handle to which this operation belongs. */435 /** Native fence object handle to which this operation belongs. */
414 u64 fence_handle;436 u64 fence_handle;
415 /* Operation type, see enum vpu_hws_native_fence_log_op. */437 /** Operation type, see enum vpu_hws_native_fence_log_op. */
416 u64 op_type;438 u64 op_type;
417 u64 reserved_0;439 u64 reserved_0;
418 /*440 /**
419 * VPU_HWS_NATIVE_FENCE_LOG_OP_WAIT_UNBLOCKED only: Timestamp at which fence441 * VPU_HWS_NATIVE_FENCE_LOG_OP_WAIT_UNBLOCKED only: Timestamp at which fence
420 * wait was started (in NPU SysTime).442 * wait was started (in NPU SysTime).
421 */443 */
422 u64 fence_wait_start_ts;444 u64 fence_wait_start_ts;
423 u64 reserved_1;445 u64 reserved_1;
424 /* Timestamp at which fence operation was completed (in NPU SysTime). */446 /** Timestamp at which fence operation was completed (in NPU SysTime). */
425 u64 fence_end_ts;447 u64 fence_end_ts;
426};448};
427 449
428/* Native fence log buffer. */450/** Native fence log buffer. */
429struct vpu_hws_native_fence_log_buffer {451struct vpu_hws_native_fence_log_buffer {
430 struct vpu_hws_native_fence_log_header header;452 struct vpu_hws_native_fence_log_header header;
431 struct vpu_hws_native_fence_log_entry entry[];453 struct vpu_hws_native_fence_log_entry entry[];
@@ -435,10 +457,17 @@ struct vpu_hws_native_fence_log_buffer { · 3 unmodified lines
435 * Host <-> VPU IPC messages types.457 * Host <-> VPU IPC messages types.
436 */458 */
437enum vpu_ipc_msg_type {459enum vpu_ipc_msg_type {
460 /** Unsupported command */
438 VPU_JSM_MSG_UNKNOWN = 0xFFFFFFFF,461 VPU_JSM_MSG_UNKNOWN = 0xFFFFFFFF,
439 462
440 /* IPC Host -> Device, Async commands */463 /** IPC Host -> Device, base id for async commands */
441 VPU_JSM_MSG_ASYNC_CMD = 0x1100,464 VPU_JSM_MSG_ASYNC_CMD = 0x1100,
465 /**
466 * Reset engine. The NPU cancels all the jobs currently executing on the target
467 * engine making the engine become idle and then does a HW reset, before returning
468 * to the host.
469 * @see struct vpu_ipc_msg_payload_engine_reset
470 */
442 VPU_JSM_MSG_ENGINE_RESET = VPU_JSM_MSG_ASYNC_CMD,471 VPU_JSM_MSG_ENGINE_RESET = VPU_JSM_MSG_ASYNC_CMD,
443 /**472 /**
444 * Preempt engine. The NPU stops (preempts) all the jobs currently473 * Preempt engine. The NPU stops (preempts) all the jobs currently
@@ -448,10 +477,24 @@ enum vpu_ipc_msg_type { · 3 unmodified lines
448 * the target engine, but it stops processing them (until the queue doorbell477 * the target engine, but it stops processing them (until the queue doorbell
449 * is rung again); the host is responsible to reset the job queue, either478 * is rung again); the host is responsible to reset the job queue, either
450 * after preemption or when resubmitting jobs to the queue.479 * after preemption or when resubmitting jobs to the queue.
480 * @see vpu_ipc_msg_payload_engine_preempt
451 */481 */
452 VPU_JSM_MSG_ENGINE_PREEMPT = 0x1101,482 VPU_JSM_MSG_ENGINE_PREEMPT = 0x1101,
483 /**
484 * OS scheduling doorbell register command
485 * @see vpu_ipc_msg_payload_register_db
486 */
453 VPU_JSM_MSG_REGISTER_DB = 0x1102,487 VPU_JSM_MSG_REGISTER_DB = 0x1102,
488 /**
489 * OS scheduling doorbell unregister command
490 * @see vpu_ipc_msg_payload_unregister_db
491 */
454 VPU_JSM_MSG_UNREGISTER_DB = 0x1103,492 VPU_JSM_MSG_UNREGISTER_DB = 0x1103,
493 /**
494 * Query engine heartbeat. Heartbeat is expected to increase monotonically
495 * and increase while work is being progressed by NPU.
496 * @see vpu_ipc_msg_payload_query_engine_hb
497 */
455 VPU_JSM_MSG_QUERY_ENGINE_HB = 0x1104,498 VPU_JSM_MSG_QUERY_ENGINE_HB = 0x1104,
456 VPU_JSM_MSG_GET_POWER_LEVEL_COUNT = 0x1105,499 VPU_JSM_MSG_GET_POWER_LEVEL_COUNT = 0x1105,
457 VPU_JSM_MSG_GET_POWER_LEVEL = 0x1106,500 VPU_JSM_MSG_GET_POWER_LEVEL = 0x1106,
@@ -477,6 +520,7 @@ enum vpu_ipc_msg_type { · 19 unmodified lines
477 * aborted and removed from internal scheduling queues. All doorbells assigned520 * aborted and removed from internal scheduling queues. All doorbells assigned
478 * to the host_ssid are unregistered and any internal FW resources belonging to521 * to the host_ssid are unregistered and any internal FW resources belonging to
479 * the host_ssid are released.522 * the host_ssid are released.
523 * @see vpu_ipc_msg_payload_ssid_release
480 */524 */
481 VPU_JSM_MSG_SSID_RELEASE = 0x110e,525 VPU_JSM_MSG_SSID_RELEASE = 0x110e,
482 /**526 /**
@@ -504,43 +548,78 @@ enum vpu_ipc_msg_type { · 21 unmodified lines
504 * @see vpu_jsm_metric_streamer_start548 * @see vpu_jsm_metric_streamer_start
505 */549 */
506 VPU_JSM_MSG_METRIC_STREAMER_INFO = 0x1112,550 VPU_JSM_MSG_METRIC_STREAMER_INFO = 0x1112,
507 /** Control command: Priority band setup */551 /**
552 * Control command: Priority band setup
553 * @see vpu_ipc_msg_payload_hws_priority_band_setup
554 */
508 VPU_JSM_MSG_SET_PRIORITY_BAND_SETUP = 0x1113,555 VPU_JSM_MSG_SET_PRIORITY_BAND_SETUP = 0x1113,
509 /** Control command: Create command queue */556 /**
557 * Control command: Create command queue
558 * @see vpu_ipc_msg_payload_hws_create_cmdq
559 */
510 VPU_JSM_MSG_CREATE_CMD_QUEUE = 0x1114,560 VPU_JSM_MSG_CREATE_CMD_QUEUE = 0x1114,
511 /** Control command: Destroy command queue */561 /**
562 * Control command: Destroy command queue
563 * @see vpu_ipc_msg_payload_hws_destroy_cmdq
564 */
512 VPU_JSM_MSG_DESTROY_CMD_QUEUE = 0x1115,565 VPU_JSM_MSG_DESTROY_CMD_QUEUE = 0x1115,
513 /** Control command: Set context scheduling properties */566 /**
567 * Control command: Set context scheduling properties
568 * @see vpu_ipc_msg_payload_hws_set_context_sched_properties
569 */
514 VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES = 0x1116,570 VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES = 0x1116,
515 /*571 /**
516 * Register a doorbell to notify VPU of new work. The doorbell may later be572 * Register a doorbell to notify VPU of new work. The doorbell may later be
517 * deallocated or reassigned to another context.573 * deallocated or reassigned to another context.
574 * @see vpu_jsm_hws_register_db
518 */575 */
519 VPU_JSM_MSG_HWS_REGISTER_DB = 0x1117,576 VPU_JSM_MSG_HWS_REGISTER_DB = 0x1117,
520 /** Control command: Log buffer setting */577 /**
578 * Control command: Log buffer setting
579 * @see vpu_ipc_msg_payload_hws_set_scheduling_log
580 */
521 VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG = 0x1118,581 VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG = 0x1118,
522 /* Control command: Suspend command queue. */582 /**
583 * Control command: Suspend command queue.
584 * @see vpu_ipc_msg_payload_hws_suspend_cmdq
585 */
523 VPU_JSM_MSG_HWS_SUSPEND_CMDQ = 0x1119,586 VPU_JSM_MSG_HWS_SUSPEND_CMDQ = 0x1119,
524 /* Control command: Resume command queue */587 /**
588 * Control command: Resume command queue
589 * @see vpu_ipc_msg_payload_hws_resume_cmdq
590 */
525 VPU_JSM_MSG_HWS_RESUME_CMDQ = 0x111a,591 VPU_JSM_MSG_HWS_RESUME_CMDQ = 0x111a,
526 /* Control command: Resume engine after reset */592 /**
593 * Control command: Resume engine after reset
594 * @see vpu_ipc_msg_payload_hws_resume_engine
595 */
527 VPU_JSM_MSG_HWS_ENGINE_RESUME = 0x111b,596 VPU_JSM_MSG_HWS_ENGINE_RESUME = 0x111b,
528 /* Control command: Enable survivability/DCT mode */597 /**
598 * Control command: Enable survivability/DCT mode
599 * @see vpu_ipc_msg_payload_pwr_dct_control
600 */
529 VPU_JSM_MSG_DCT_ENABLE = 0x111c,601 VPU_JSM_MSG_DCT_ENABLE = 0x111c,
530 /* Control command: Disable survivability/DCT mode */602 /**
603 * Control command: Disable survivability/DCT mode
604 * This command has no payload
605 */
531 VPU_JSM_MSG_DCT_DISABLE = 0x111d,606 VPU_JSM_MSG_DCT_DISABLE = 0x111d,
532 /**607 /**
533 * Dump VPU state. To be used for debug purposes only.608 * Dump VPU state. To be used for debug purposes only.
534 * NOTE: Please introduce new ASYNC commands before this one. *609 * This command has no payload.
610 * NOTE: Please introduce new ASYNC commands before this one.
535 */611 */
536 VPU_JSM_MSG_STATE_DUMP = 0x11FF,612 VPU_JSM_MSG_STATE_DUMP = 0x11FF,
537 613
538 /* IPC Host -> Device, General commands */614 /** IPC Host -> Device, base id for general commands */
539 VPU_JSM_MSG_GENERAL_CMD = 0x1200,615 VPU_JSM_MSG_GENERAL_CMD = 0x1200,
616 /** Unsupported command */
540 VPU_JSM_MSG_BLOB_DEINIT_DEPRECATED = VPU_JSM_MSG_GENERAL_CMD,617 VPU_JSM_MSG_BLOB_DEINIT_DEPRECATED = VPU_JSM_MSG_GENERAL_CMD,
541 /**618 /**
542 * Control dyndbg behavior by executing a dyndbg command; equivalent to619 * Control dyndbg behavior by executing a dyndbg command; equivalent to
543 * Linux command: `echo '<dyndbg_cmd>' > <debugfs>/dynamic_debug/control`.620 * Linux command:
621 * @verbatim echo '<dyndbg_cmd>' > <debugfs>/dynamic_debug/control @endverbatim
622 * @see vpu_ipc_msg_payload_dyndbg_control
544 */623 */
545 VPU_JSM_MSG_DYNDBG_CONTROL = 0x1201,624 VPU_JSM_MSG_DYNDBG_CONTROL = 0x1201,
546 /**625 /**
@@ -548,17 +627,35 @@ enum vpu_ipc_msg_type { · 1 unmodified line
548 */627 */
549 VPU_JSM_MSG_PWR_D0I3_ENTER = 0x1202,628 VPU_JSM_MSG_PWR_D0I3_ENTER = 0x1202,
550 629
551 /* IPC Device -> Host, Job completion */630 /**
631 * IPC Device -> Host, Job completion
632 * @see struct vpu_ipc_msg_payload_job_done
633 */
552 VPU_JSM_MSG_JOB_DONE = 0x2100,634 VPU_JSM_MSG_JOB_DONE = 0x2100,
553 /* IPC Device -> Host, Fence signalled */635 /**
636 * IPC Device -> Host, Fence signalled
637 * @see vpu_ipc_msg_payload_native_fence_signalled
638 */
554 VPU_JSM_MSG_NATIVE_FENCE_SIGNALLED = 0x2101,639 VPU_JSM_MSG_NATIVE_FENCE_SIGNALLED = 0x2101,
555 640
556 /* IPC Device -> Host, Async command completion */641 /* IPC Device -> Host, Async command completion */
557 VPU_JSM_MSG_ASYNC_CMD_DONE = 0x2200,642 VPU_JSM_MSG_ASYNC_CMD_DONE = 0x2200,
643 /**
644 * IPC Device -> Host, engine reset complete
645 * @see vpu_ipc_msg_payload_engine_reset_done
646 */
558 VPU_JSM_MSG_ENGINE_RESET_DONE = VPU_JSM_MSG_ASYNC_CMD_DONE,647 VPU_JSM_MSG_ENGINE_RESET_DONE = VPU_JSM_MSG_ASYNC_CMD_DONE,
648 /**
649 * Preempt complete message
650 * @see vpu_ipc_msg_payload_engine_preempt_done
651 */
559 VPU_JSM_MSG_ENGINE_PREEMPT_DONE = 0x2201,652 VPU_JSM_MSG_ENGINE_PREEMPT_DONE = 0x2201,
560 VPU_JSM_MSG_REGISTER_DB_DONE = 0x2202,653 VPU_JSM_MSG_REGISTER_DB_DONE = 0x2202,
561 VPU_JSM_MSG_UNREGISTER_DB_DONE = 0x2203,654 VPU_JSM_MSG_UNREGISTER_DB_DONE = 0x2203,
655 /**
656 * Response to query engine heartbeat.
657 * @see vpu_ipc_msg_payload_query_engine_hb_done
658 */
562 VPU_JSM_MSG_QUERY_ENGINE_HB_DONE = 0x2204,659 VPU_JSM_MSG_QUERY_ENGINE_HB_DONE = 0x2204,
563 VPU_JSM_MSG_GET_POWER_LEVEL_COUNT_DONE = 0x2205,660 VPU_JSM_MSG_GET_POWER_LEVEL_COUNT_DONE = 0x2205,
564 VPU_JSM_MSG_GET_POWER_LEVEL_DONE = 0x2206,661 VPU_JSM_MSG_GET_POWER_LEVEL_DONE = 0x2206,
@@ -575,7 +672,10 @@ enum vpu_ipc_msg_type { · 10 unmodified lines
575 VPU_JSM_MSG_TRACE_GET_CAPABILITY_RSP = 0x220c,672 VPU_JSM_MSG_TRACE_GET_CAPABILITY_RSP = 0x220c,
576 /** Response to VPU_JSM_MSG_TRACE_GET_NAME. */673 /** Response to VPU_JSM_MSG_TRACE_GET_NAME. */
577 VPU_JSM_MSG_TRACE_GET_NAME_RSP = 0x220d,674 VPU_JSM_MSG_TRACE_GET_NAME_RSP = 0x220d,
578 /** Response to VPU_JSM_MSG_SSID_RELEASE. */675 /**
676 * Response to VPU_JSM_MSG_SSID_RELEASE.
677 * @see vpu_ipc_msg_payload_ssid_release
678 */
579 VPU_JSM_MSG_SSID_RELEASE_DONE = 0x220e,679 VPU_JSM_MSG_SSID_RELEASE_DONE = 0x220e,
580 /**680 /**
581 * Response to VPU_JSM_MSG_METRIC_STREAMER_START.681 * Response to VPU_JSM_MSG_METRIC_STREAMER_START.
@@ -605,37 +705,71 @@ enum vpu_ipc_msg_type { · 23 unmodified lines
605 /**705 /**
606 * Asynchronous event sent from the VPU to the host either when the current706 * Asynchronous event sent from the VPU to the host either when the current
607 * metric buffer is full or when the VPU has collected a multiple of707 * metric buffer is full or when the VPU has collected a multiple of
608 * @notify_sample_count samples as indicated through the start command708 * @ref vpu_jsm_metric_streamer_start::notify_sample_count samples as indicated
609 * (VPU_JSM_MSG_METRIC_STREAMER_START). Returns information about collected709 * through the start command (VPU_JSM_MSG_METRIC_STREAMER_START). Returns
610 * metric data.710 * information about collected metric data.
611 * @see vpu_jsm_metric_streamer_done711 * @see vpu_jsm_metric_streamer_done
612 */712 */
613 VPU_JSM_MSG_METRIC_STREAMER_NOTIFICATION = 0x2213,713 VPU_JSM_MSG_METRIC_STREAMER_NOTIFICATION = 0x2213,
614 /** Response to control command: Priority band setup */714 /**
715 * Response to control command: Priority band setup
716 * @see vpu_ipc_msg_payload_hws_priority_band_setup
717 */
615 VPU_JSM_MSG_SET_PRIORITY_BAND_SETUP_RSP = 0x2214,718 VPU_JSM_MSG_SET_PRIORITY_BAND_SETUP_RSP = 0x2214,
616 /** Response to control command: Create command queue */719 /**
720 * Response to control command: Create command queue
721 * @see vpu_ipc_msg_payload_hws_create_cmdq_rsp
722 */
617 VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP = 0x2215,723 VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP = 0x2215,
618 /** Response to control command: Destroy command queue */724 /**
725 * Response to control command: Destroy command queue
726 * @see vpu_ipc_msg_payload_hws_destroy_cmdq
727 */
619 VPU_JSM_MSG_DESTROY_CMD_QUEUE_RSP = 0x2216,728 VPU_JSM_MSG_DESTROY_CMD_QUEUE_RSP = 0x2216,
620 /** Response to control command: Set context scheduling properties */729 /**
730 * Response to control command: Set context scheduling properties
731 * @see vpu_ipc_msg_payload_hws_set_context_sched_properties
732 */
621 VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP = 0x2217,733 VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP = 0x2217,
622 /** Response to control command: Log buffer setting */734 /**
735 * Response to control command: Log buffer setting
736 * @see vpu_ipc_msg_payload_hws_set_scheduling_log
737 */
623 VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG_RSP = 0x2218,738 VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG_RSP = 0x2218,
624 /* IPC Device -> Host, HWS notify index entry of log buffer written */739 /**
740 * IPC Device -> Host, HWS notify index entry of log buffer written
741 * @see vpu_ipc_msg_payload_hws_scheduling_log_notification
742 */
625 VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION = 0x2219,743 VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION = 0x2219,
626 /* IPC Device -> Host, HWS completion of a context suspend request */744 /**
745 * IPC Device -> Host, HWS completion of a context suspend request
746 * @see vpu_ipc_msg_payload_hws_suspend_cmdq
747 */
627 VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE = 0x221a,748 VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE = 0x221a,
628 /* Response to control command: Resume command queue */749 /**
750 * Response to control command: Resume command queue
751 * @see vpu_ipc_msg_payload_hws_resume_cmdq
752 */
629 VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP = 0x221b,753 VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP = 0x221b,
630 /* Response to control command: Resume engine command response */754 /**
755 * Response to control command: Resume engine command response
756 * @see vpu_ipc_msg_payload_hws_resume_engine
757 */
631 VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE = 0x221c,758 VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE = 0x221c,
632 /* Response to control command: Enable survivability/DCT mode */759 /**
760 * Response to control command: Enable survivability/DCT mode
761 * This command has no payload
762 */
633 VPU_JSM_MSG_DCT_ENABLE_DONE = 0x221d,763 VPU_JSM_MSG_DCT_ENABLE_DONE = 0x221d,
634 /* Response to control command: Disable survivability/DCT mode */764 /**
765 * Response to control command: Disable survivability/DCT mode
766 * This command has no payload
767 */
635 VPU_JSM_MSG_DCT_DISABLE_DONE = 0x221e,768 VPU_JSM_MSG_DCT_DISABLE_DONE = 0x221e,
636 /**769 /**
637 * Response to state dump control command.770 * Response to state dump control command.
638 * NOTE: Please introduce new ASYNC responses before this one. *771 * This command has no payload.
772 * NOTE: Please introduce new ASYNC responses before this one.
639 */773 */
640 VPU_JSM_MSG_STATE_DUMP_RSP = 0x22FF,774 VPU_JSM_MSG_STATE_DUMP_RSP = 0x22FF,
641 775
@@ -653,57 +787,66 @@ enum vpu_ipc_msg_type { · 11 unmodified lines
653 787
654enum vpu_ipc_msg_status { VPU_JSM_MSG_FREE, VPU_JSM_MSG_ALLOCATED };788enum vpu_ipc_msg_status { VPU_JSM_MSG_FREE, VPU_JSM_MSG_ALLOCATED };
655 789
656/*790/**
657 * Host <-> LRT IPC message payload definitions791 * Engine reset request payload
792 * @see VPU_JSM_MSG_ENGINE_RESET
658 */793 */
659struct vpu_ipc_msg_payload_engine_reset {794struct vpu_ipc_msg_payload_engine_reset {
660 /* Engine to be reset. */795 /** Engine to be reset. */
661 u32 engine_idx;796 u32 engine_idx;
662 /* Reserved */797 /** Reserved */
663 u32 reserved_0;798 u32 reserved_0;
664};799};
665 800
801/**
802 * Engine preemption request struct
803 * @see VPU_JSM_MSG_ENGINE_PREEMPT
804 */
666struct vpu_ipc_msg_payload_engine_preempt {805struct vpu_ipc_msg_payload_engine_preempt {
667 /* Engine to be preempted. */806 /** Engine to be preempted. */
668 u32 engine_idx;807 u32 engine_idx;
669 /* ID of the preemption request. */808 /** ID of the preemption request. */
670 u32 preempt_id;809 u32 preempt_id;
671};810};
672 811
673/*812/**
674 * @brief Register doorbell command structure.813 * Register doorbell command structure.
675 * This structure supports doorbell registration for only OS scheduling.814 * This structure supports doorbell registration for only OS scheduling.
676 * @see VPU_JSM_MSG_REGISTER_DB815 * @see VPU_JSM_MSG_REGISTER_DB
677 */816 */
678struct vpu_ipc_msg_payload_register_db {817struct vpu_ipc_msg_payload_register_db {
679 /* Index of the doorbell to register. */818 /** Index of the doorbell to register. */
680 u32 db_idx;819 u32 db_idx;
681 /* Reserved */820 /** Reserved */
682 u32 reserved_0;821 u32 reserved_0;
683 /* Virtual address in Global GTT pointing to the start of job queue. */822 /** Virtual address in Global GTT pointing to the start of job queue. */
684 u64 jobq_base;823 u64 jobq_base;
685 /* Size of the job queue in bytes. */824 /** Size of the job queue in bytes. */
686 u32 jobq_size;825 u32 jobq_size;
687 /* Host sub-stream ID for the context assigned to the doorbell. */826 /** Host sub-stream ID for the context assigned to the doorbell. */
688 u32 host_ssid;827 u32 host_ssid;
689};828};
690 829
691/**830/**
692 * @brief Unregister doorbell command structure.831 * Unregister doorbell command structure.
693 * Request structure to unregister a doorbell for both HW and OS scheduling.832 * Request structure to unregister a doorbell for both HW and OS scheduling.
694 * @see VPU_JSM_MSG_UNREGISTER_DB833 * @see VPU_JSM_MSG_UNREGISTER_DB
695 */834 */
696struct vpu_ipc_msg_payload_unregister_db {835struct vpu_ipc_msg_payload_unregister_db {
697 /* Index of the doorbell to unregister. */836 /** Index of the doorbell to unregister. */
698 u32 db_idx;837 u32 db_idx;
699 /* Reserved */838 /** Reserved */
700 u32 reserved_0;839 u32 reserved_0;
701};840};
702 841
842/**
843 * Heartbeat request structure
844 * @see VPU_JSM_MSG_QUERY_ENGINE_HB
845 */
703struct vpu_ipc_msg_payload_query_engine_hb {846struct vpu_ipc_msg_payload_query_engine_hb {
704 /* Engine to return heartbeat value. */847 /** Engine to return heartbeat value. */
705 u32 engine_idx;848 u32 engine_idx;
706 /* Reserved */849 /** Reserved */
707 u32 reserved_0;850 u32 reserved_0;
708};851};
709 852
@@ -723,10 +866,14 @@ struct vpu_ipc_msg_payload_power_level { · 13 unmodified lines
723 u32 reserved_0;866 u32 reserved_0;
724};867};
725 868
869/**
870 * Structure for requesting ssid release
871 * @see VPU_JSM_MSG_SSID_RELEASE
872 */
726struct vpu_ipc_msg_payload_ssid_release {873struct vpu_ipc_msg_payload_ssid_release {
727 /* Host sub-stream ID for the context to be released. */874 /** Host sub-stream ID for the context to be released. */
728 u32 host_ssid;875 u32 host_ssid;
729 /* Reserved */876 /** Reserved */
730 u32 reserved_0;877 u32 reserved_0;
731};878};
732 879
@@ -752,7 +899,7 @@ struct vpu_jsm_metric_streamer_start { · 19 unmodified lines
752 u64 sampling_rate;899 u64 sampling_rate;
753 /**900 /**
754 * If > 0 the VPU will send a VPU_JSM_MSG_METRIC_STREAMER_NOTIFICATION message901 * If > 0 the VPU will send a VPU_JSM_MSG_METRIC_STREAMER_NOTIFICATION message
755 * after every @notify_sample_count samples is collected or dropped by the VPU.902 * after every @ref notify_sample_count samples is collected or dropped by the VPU.
756 * If set to UINT_MAX the VPU will only generate a notification when the metric903 * If set to UINT_MAX the VPU will only generate a notification when the metric
757 * buffer is full. If set to 0 the VPU will never generate a notification.904 * buffer is full. If set to 0 the VPU will never generate a notification.
758 */905 */
@@ -762,9 +909,9 @@ struct vpu_jsm_metric_streamer_start { · 3 unmodified lines
762 * Address and size of the buffer where the VPU will write metric data. The909 * Address and size of the buffer where the VPU will write metric data. The
763 * VPU writes all counters from enabled metric groups one after another. If910 * VPU writes all counters from enabled metric groups one after another. If
764 * there is no space left to write data at the next sample period the VPU911 * there is no space left to write data at the next sample period the VPU
765 * will switch to the next buffer (@see next_buffer_addr) and will optionally912 * will switch to the next buffer (@ref next_buffer_addr) and will optionally
766 * send a notification to the host driver if @notify_sample_count is non-zero.913 * send a notification to the host driver if @ref notify_sample_count is non-zero.
767 * If @next_buffer_addr is NULL the VPU will stop collecting metric data.914 * If @ref next_buffer_addr is NULL the VPU will stop collecting metric data.
768 */915 */
769 u64 buffer_addr;916 u64 buffer_addr;
770 u64 buffer_size;917 u64 buffer_size;
@@ -827,63 +974,80 @@ struct vpu_jsm_metric_streamer_update { · 56 unmodified lines
827 u64 next_buffer_size;974 u64 next_buffer_size;
828};975};
829 976
977/**
978 * Device -> host job completion message.
979 * @see VPU_JSM_MSG_JOB_DONE
980 */
830struct vpu_ipc_msg_payload_job_done {981struct vpu_ipc_msg_payload_job_done {
831 /* Engine to which the job was submitted. */982 /** Engine to which the job was submitted. */
832 u32 engine_idx;983 u32 engine_idx;
833 /* Index of the doorbell to which the job was submitted */984 /** Index of the doorbell to which the job was submitted */
834 u32 db_idx;985 u32 db_idx;
835 /* ID of the completed job */986 /** ID of the completed job */
836 u32 job_id;987 u32 job_id;
837 /* Status of the completed job */988 /** Status of the completed job */
838 u32 job_status;989 u32 job_status;
839 /* Host SSID */990 /** Host SSID */
840 u32 host_ssid;991 u32 host_ssid;
841 /* Zero Padding */992 /** Zero Padding */
842 u32 reserved_0;993 u32 reserved_0;
843 /* Command queue id */994 /** Command queue id */
844 u64 cmdq_id;995 u64 cmdq_id;
845};996};
846 997
847/*998/**
848 * Notification message upon native fence signalling.999 * Notification message upon native fence signalling.
849 * @see VPU_JSM_MSG_NATIVE_FENCE_SIGNALLED1000 * @see VPU_JSM_MSG_NATIVE_FENCE_SIGNALLED
850 */1001 */
851struct vpu_ipc_msg_payload_native_fence_signalled {1002struct vpu_ipc_msg_payload_native_fence_signalled {
852 /* Engine ID. */1003 /** Engine ID. */
853 u32 engine_idx;1004 u32 engine_idx;
854 /* Host SSID. */1005 /** Host SSID. */
855 u32 host_ssid;1006 u32 host_ssid;
856 /* CMDQ ID */1007 /** CMDQ ID */
857 u64 cmdq_id;1008 u64 cmdq_id;
858 /* Fence object handle. */1009 /** Fence object handle. */
859 u64 fence_handle;1010 u64 fence_handle;
860};1011};
861 1012
1013/**
1014 * vpu_ipc_msg_payload_engine_reset_done will contain an array of this structure
1015 * which contains which queues caused reset if FW was able to detect any error.
1016 * @see vpu_ipc_msg_payload_engine_reset_done
1017 */
862struct vpu_jsm_engine_reset_context {1018struct vpu_jsm_engine_reset_context {
863 /* Host SSID */1019 /** Host SSID */
864 u32 host_ssid;1020 u32 host_ssid;
865 /* Zero Padding */1021 /** Zero Padding */
866 u32 reserved_0;1022 u32 reserved_0;
867 /* Command queue id */1023 /** Command queue id */
868 u64 cmdq_id;1024 u64 cmdq_id;
869 /* See VPU_ENGINE_RESET_CONTEXT_* defines */1025 /** See VPU_ENGINE_RESET_CONTEXT_* defines */
870 u64 flags;1026 u64 flags;
871};1027};
872 1028
1029/**
1030 * Engine reset response.
1031 * @see VPU_JSM_MSG_ENGINE_RESET_DONE
1032 */
873struct vpu_ipc_msg_payload_engine_reset_done {1033struct vpu_ipc_msg_payload_engine_reset_done {
874 /* Engine ordinal */1034 /** Engine ordinal */
875 u32 engine_idx;1035 u32 engine_idx;
876 /* Number of impacted contexts */1036 /** Number of impacted contexts */
877 u32 num_impacted_contexts;1037 u32 num_impacted_contexts;
878 /* Array of impacted command queue ids and their flags */1038 /** Array of impacted command queue ids and their flags */
879 struct vpu_jsm_engine_reset_context1039 struct vpu_jsm_engine_reset_context
880 impacted_contexts[VPU_MAX_ENGINE_RESET_IMPACTED_CONTEXTS];1040 impacted_contexts[VPU_MAX_ENGINE_RESET_IMPACTED_CONTEXTS];
881};1041};
882 1042
1043/**
1044 * Preemption response struct
1045 * @see VPU_JSM_MSG_ENGINE_PREEMPT_DONE
1046 */
883struct vpu_ipc_msg_payload_engine_preempt_done {1047struct vpu_ipc_msg_payload_engine_preempt_done {
884 /* Engine preempted. */1048 /** Engine preempted. */
885 u32 engine_idx;1049 u32 engine_idx;
886 /* ID of the preemption request. */1050 /** ID of the preemption request. */
887 u32 preempt_id;1051 u32 preempt_id;
888};1052};
889 1053
@@ -912,12 +1076,16 @@ struct vpu_ipc_msg_payload_unregister_db_done { · 22 unmodified lines
912 u32 reserved_0;1076 u32 reserved_0;
913};1077};
914 1078
1079/**
1080 * Structure for heartbeat response
1081 * @see VPU_JSM_MSG_QUERY_ENGINE_HB_DONE
1082 */
915struct vpu_ipc_msg_payload_query_engine_hb_done {1083struct vpu_ipc_msg_payload_query_engine_hb_done {
916 /* Engine returning heartbeat value. */1084 /** Engine returning heartbeat value. */
917 u32 engine_idx;1085 u32 engine_idx;
918 /* Reserved */1086 /** Reserved */
919 u32 reserved_0;1087 u32 reserved_0;
920 /* Heartbeat value. */1088 /** Heartbeat value. */
921 u64 heartbeat;1089 u64 heartbeat;
922};1090};
923 1091
@@ -937,7 +1105,10 @@ struct vpu_ipc_msg_payload_get_power_level_count_done { · 13 unmodified lines
937 u8 power_limit[16];1105 u8 power_limit[16];
938};1106};
939 1107
940/* HWS priority band setup request / response */1108/**
1109 * HWS priority band setup request / response
1110 * @see VPU_JSM_MSG_SET_PRIORITY_BAND_SETUP
1111 */
941struct vpu_ipc_msg_payload_hws_priority_band_setup {1112struct vpu_ipc_msg_payload_hws_priority_band_setup {
942 /*1113 /*
943 * Grace period in 100ns units when preempting another priority band for1114 * Grace period in 100ns units when preempting another priority band for
@@ -964,15 +1135,23 @@ struct vpu_ipc_msg_payload_hws_priority_band_setup { · 20 unmodified lines
964 * TDR timeout value in milliseconds. Default value of 0 meaning no timeout.1135 * TDR timeout value in milliseconds. Default value of 0 meaning no timeout.
965 */1136 */
966 u32 tdr_timeout;1137 u32 tdr_timeout;
1138 /* Non-interactive queue timeout for no progress of heartbeat in milliseconds.
1139 * Default value of 0 meaning no timeout.
1140 */
1141 u32 non_interactive_no_progress_timeout;
1142 /*
1143 * Non-interactive queue upper limit timeout value in milliseconds. Default
1144 * value of 0 meaning no timeout.
1145 */
1146 u32 non_interactive_timeout;
967};1147};
968 1148
969/*1149/**
970 * @brief HWS create command queue request.1150 * @brief HWS create command queue request.
971 * Host will create a command queue via this command.1151 * Host will create a command queue via this command.
972 * Note: Cmdq group is a handle of an object which1152 * Note: Cmdq group is a handle of an object which
973 * may contain one or more command queues.1153 * may contain one or more command queues.
974 * @see VPU_JSM_MSG_CREATE_CMD_QUEUE1154 * @see VPU_JSM_MSG_CREATE_CMD_QUEUE
975 * @see VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP
976 */1155 */
977struct vpu_ipc_msg_payload_hws_create_cmdq {1156struct vpu_ipc_msg_payload_hws_create_cmdq {
978 /* Process id */1157 /* Process id */
@@ -993,66 +1172,73 @@ struct vpu_ipc_msg_payload_hws_create_cmdq { · 14 unmodified lines
993 u32 reserved_0;1172 u32 reserved_0;
994};1173};
995 1174
996/*1175/**
997 * @brief HWS create command queue response.1176 * HWS create command queue response.
998 * @see VPU_JSM_MSG_CREATE_CMD_QUEUE
999 * @see VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP1177 * @see VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP
1000 */1178 */
1001struct vpu_ipc_msg_payload_hws_create_cmdq_rsp {1179struct vpu_ipc_msg_payload_hws_create_cmdq_rsp {
1002 /* Process id */1180 /** Process id */
1003 u64 process_id;1181 u64 process_id;
1004 /* Host SSID */1182 /** Host SSID */
1005 u32 host_ssid;1183 u32 host_ssid;
1006 /* Engine for which queue is being created */1184 /** Engine for which queue is being created */
1007 u32 engine_idx;1185 u32 engine_idx;
1008 /* Command queue group */1186 /** Command queue group */
1009 u64 cmdq_group;1187 u64 cmdq_group;
1010 /* Command queue id */1188 /** Command queue id */
1011 u64 cmdq_id;1189 u64 cmdq_id;
1012};1190};
1013 1191
1014/* HWS destroy command queue request / response */1192/**
1193 * HWS destroy command queue request / response
1194 * @see VPU_JSM_MSG_DESTROY_CMD_QUEUE
1195 * @see VPU_JSM_MSG_DESTROY_CMD_QUEUE_RSP
1196 */
1015struct vpu_ipc_msg_payload_hws_destroy_cmdq {1197struct vpu_ipc_msg_payload_hws_destroy_cmdq {
1016 /* Host SSID */1198 /** Host SSID */
1017 u32 host_ssid;1199 u32 host_ssid;
1018 /* Zero Padding */1200 /** Zero Padding */
1019 u32 reserved;1201 u32 reserved;
1020 /* Command queue id */1202 /** Command queue id */
1021 u64 cmdq_id;1203 u64 cmdq_id;
1022};1204};
1023 1205
1024/* HWS set context scheduling properties request / response */1206/**
1207 * HWS set context scheduling properties request / response
1208 * @see VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES
1209 * @see VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP
1210 */
1025struct vpu_ipc_msg_payload_hws_set_context_sched_properties {1211struct vpu_ipc_msg_payload_hws_set_context_sched_properties {
1026 /* Host SSID */1212 /** Host SSID */
1027 u32 host_ssid;1213 u32 host_ssid;
1028 /* Zero Padding */1214 /** Zero Padding */
1029 u32 reserved_0;1215 u32 reserved_0;
1030 /* Command queue id */1216 /** Command queue id */
1031 u64 cmdq_id;1217 u64 cmdq_id;
1032 /*1218 /**
1033 * Priority band to assign to work of this context.1219 * Priority band to assign to work of this context.
1034 * Available priority bands: @see enum vpu_job_scheduling_priority_band1220 * Available priority bands: @see enum vpu_job_scheduling_priority_band
1035 */1221 */
1036 u32 priority_band;1222 u32 priority_band;
1037 /* Inside realtime band assigns a further priority */1223 /** Inside realtime band assigns a further priority */
1038 u32 realtime_priority_level;1224 u32 realtime_priority_level;
1039 /* Priority relative to other contexts in the same process */1225 /** Priority relative to other contexts in the same process */
1040 s32 in_process_priority;1226 s32 in_process_priority;
1041 /* Zero padding / Reserved */1227 /** Zero padding / Reserved */
1042 u32 reserved_1;1228 u32 reserved_1;
1043 /*1229 /**
1044 * Context quantum relative to other contexts of same priority in the same process1230 * Context quantum relative to other contexts of same priority in the same process
1045 * Minimum value supported by NPU is 1ms (10000 in 100ns units).1231 * Minimum value supported by NPU is 1ms (10000 in 100ns units).
1046 */1232 */
1047 u64 context_quantum;1233 u64 context_quantum;
1048 /* Grace period when preempting context of the same priority within the same process */1234 /** Grace period when preempting context of the same priority within the same process */
1049 u64 grace_period_same_priority;1235 u64 grace_period_same_priority;
1050 /* Grace period when preempting context of a lower priority within the same process */1236 /** Grace period when preempting context of a lower priority within the same process */
1051 u64 grace_period_lower_priority;1237 u64 grace_period_lower_priority;
1052};1238};
1053 1239
1054/*1240/**
1055 * @brief Register doorbell command structure.1241 * Register doorbell command structure.
1056 * This structure supports doorbell registration for both HW and OS scheduling.1242 * This structure supports doorbell registration for both HW and OS scheduling.
1057 * Note: Queue base and size are added here so that the same structure can be used for1243 * Note: Queue base and size are added here so that the same structure can be used for
1058 * OS scheduling and HW scheduling. For OS scheduling, cmdq_id will be ignored1244 * OS scheduling and HW scheduling. For OS scheduling, cmdq_id will be ignored
@@ -1061,27 +1247,27 @@ struct vpu_ipc_msg_payload_hws_set_context_sched_properties { · 2 unmodified lines
1061 * @see VPU_JSM_MSG_HWS_REGISTER_DB1247 * @see VPU_JSM_MSG_HWS_REGISTER_DB
1062 */1248 */
1063struct vpu_jsm_hws_register_db {1249struct vpu_jsm_hws_register_db {
1064 /* Index of the doorbell to register. */1250 /** Index of the doorbell to register. */
1065 u32 db_id;1251 u32 db_id;
1066 /* Host sub-stream ID for the context assigned to the doorbell. */1252 /** Host sub-stream ID for the context assigned to the doorbell. */
1067 u32 host_ssid;1253 u32 host_ssid;
1068 /* ID of the command queue associated with the doorbell. */1254 /** ID of the command queue associated with the doorbell. */
1069 u64 cmdq_id;1255 u64 cmdq_id;
1070 /* Virtual address pointing to the start of command queue. */1256 /** Virtual address pointing to the start of command queue. */
1071 u64 cmdq_base;1257 u64 cmdq_base;
1072 /* Size of the command queue in bytes. */1258 /** Size of the command queue in bytes. */
1073 u64 cmdq_size;1259 u64 cmdq_size;
1074};1260};
1075 1261
1076/*1262/**
1077 * @brief Structure to set another buffer to be used for scheduling-related logging.1263 * Structure to set another buffer to be used for scheduling-related logging.
1078 * The size of the logging buffer and the number of entries is defined as part of the1264 * The size of the logging buffer and the number of entries is defined as part of the
1079 * buffer itself as described next.1265 * buffer itself as described next.
1080 * The log buffer received from the host is made up of;1266 * The log buffer received from the host is made up of;
1081 * - header: 32 bytes in size, as shown in 'struct vpu_hws_log_buffer_header'.1267 * - header: 32 bytes in size, as shown in @ref vpu_hws_log_buffer_header.
1082 * The header contains the number of log entries in the buffer.1268 * The header contains the number of log entries in the buffer.
1083 * - log entry: 0 to n-1, each log entry is 32 bytes in size, as shown in1269 * - log entry: 0 to n-1, each log entry is 32 bytes in size, as shown in
1084 * 'struct vpu_hws_log_buffer_entry'.1270 * @ref vpu_hws_log_buffer_entry.
1085 * The entry contains the VPU timestamp, operation type and data.1271 * The entry contains the VPU timestamp, operation type and data.
1086 * The host should provide the notify index value of log buffer to VPU. This is a1272 * The host should provide the notify index value of log buffer to VPU. This is a
1087 * value defined within the log buffer and when written to will generate the1273 * value defined within the log buffer and when written to will generate the
@@ -1095,30 +1281,30 @@ struct vpu_jsm_hws_register_db { · 7 unmodified lines
1095 * @see VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION1281 * @see VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION
1096 */1282 */
1097struct vpu_ipc_msg_payload_hws_set_scheduling_log {1283struct vpu_ipc_msg_payload_hws_set_scheduling_log {
1098 /* Engine ordinal */1284 /** Engine ordinal */
1099 u32 engine_idx;1285 u32 engine_idx;
1100 /* Host SSID */1286 /** Host SSID */
1101 u32 host_ssid;1287 u32 host_ssid;
1102 /*1288 /**
1103 * VPU log buffer virtual address.1289 * VPU log buffer virtual address.
1104 * Set to 0 to disable logging for this engine.1290 * Set to 0 to disable logging for this engine.
1105 */1291 */
1106 u64 vpu_log_buffer_va;1292 u64 vpu_log_buffer_va;
1107 /*1293 /**
1108 * Notify index of log buffer. VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION1294 * Notify index of log buffer. VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION
1109 * is generated when an event log is written to this index.1295 * is generated when an event log is written to this index.
1110 */1296 */
1111 u64 notify_index;1297 u64 notify_index;
1112 /*1298 /**
1113 * Field is now deprecated, will be removed when KMD is updated to support removal1299 * Field is now deprecated, will be removed when KMD is updated to support removal
1114 */1300 */
1115 u32 enable_extra_events;1301 u32 enable_extra_events;
1116 /* Zero Padding */1302 /** Zero Padding */
1117 u32 reserved_0;1303 u32 reserved_0;
1118};1304};
1119 1305
1120/*1306/**
1121 * @brief The scheduling log notification is generated by VPU when it writes1307 * The scheduling log notification is generated by VPU when it writes
1122 * an event into the log buffer at the notify_index. VPU notifies host with1308 * an event into the log buffer at the notify_index. VPU notifies host with
1123 * VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION. This is an asynchronous1309 * VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION. This is an asynchronous
1124 * message from VPU to host.1310 * message from VPU to host.
@@ -1126,14 +1312,14 @@ struct vpu_ipc_msg_payload_hws_set_scheduling_log { · 1 unmodified line
1126 * @see VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG1312 * @see VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG
1127 */1313 */
1128struct vpu_ipc_msg_payload_hws_scheduling_log_notification {1314struct vpu_ipc_msg_payload_hws_scheduling_log_notification {
1129 /* Engine ordinal */1315 /** Engine ordinal */
1130 u32 engine_idx;1316 u32 engine_idx;
1131 /* Zero Padding */1317 /** Zero Padding */
1132 u32 reserved_0;1318 u32 reserved_0;
1133};1319};
1134 1320
1135/*1321/**
1136 * @brief HWS suspend command queue request and done structure.1322 * HWS suspend command queue request and done structure.
1137 * Host will request the suspend of contexts and VPU will;1323 * Host will request the suspend of contexts and VPU will;
1138 * - Suspend all work on this context1324 * - Suspend all work on this context
1139 * - Preempt any running work1325 * - Preempt any running work
@@ -1152,21 +1338,21 @@ struct vpu_ipc_msg_payload_hws_scheduling_log_notification { · 12 unmodified lines
1152 * @see VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE1338 * @see VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE
1153 */1339 */
1154struct vpu_ipc_msg_payload_hws_suspend_cmdq {1340struct vpu_ipc_msg_payload_hws_suspend_cmdq {
1155 /* Host SSID */1341 /** Host SSID */
1156 u32 host_ssid;1342 u32 host_ssid;
1157 /* Zero Padding */1343 /** Zero Padding */
1158 u32 reserved_0;1344 u32 reserved_0;
1159 /* Command queue id */1345 /** Command queue id */
1160 u64 cmdq_id;1346 u64 cmdq_id;
1161 /*1347 /**
1162 * Suspend fence value - reported by the VPU suspend context1348 * Suspend fence value - reported by the VPU suspend context
1163 * completed once suspend is complete.1349 * completed once suspend is complete.
1164 */1350 */
1165 u64 suspend_fence_value;1351 u64 suspend_fence_value;
1166};1352};
1167 1353
1168/*1354/**
1169 * @brief HWS Resume command queue request / response structure.1355 * HWS Resume command queue request / response structure.
1170 * Host will request the resume of a context;1356 * Host will request the resume of a context;
1171 * - VPU will resume all work on this context1357 * - VPU will resume all work on this context
1172 * - Scheduler will allow this context to be scheduled1358 * - Scheduler will allow this context to be scheduled
@@ -1174,25 +1360,25 @@ struct vpu_ipc_msg_payload_hws_suspend_cmdq { · 1 unmodified line
1174 * @see VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP1360 * @see VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP
1175 */1361 */
1176struct vpu_ipc_msg_payload_hws_resume_cmdq {1362struct vpu_ipc_msg_payload_hws_resume_cmdq {
1177 /* Host SSID */1363 /** Host SSID */
1178 u32 host_ssid;1364 u32 host_ssid;
1179 /* Zero Padding */1365 /** Zero Padding */
1180 u32 reserved_0;1366 u32 reserved_0;
1181 /* Command queue id */1367 /** Command queue id */
1182 u64 cmdq_id;1368 u64 cmdq_id;
1183};1369};
1184 1370
1185/*1371/**
1186 * @brief HWS Resume engine request / response structure.1372 * HWS Resume engine request / response structure.
1187 * After a HWS engine reset, all scheduling is stopped on VPU until a engine resume.1373 * After a HWS engine reset, all scheduling is stopped on VPU until an engine resume.
1188 * Host shall send this command to resume scheduling of any valid queue.1374 * Host shall send this command to resume scheduling of any valid queue.
1189 * @see VPU_JSM_MSG_HWS_RESUME_ENGINE1375 * @see VPU_JSM_MSG_HWS_ENGINE_RESUME
1190 * @see VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE1376 * @see VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE
1191 */1377 */
1192struct vpu_ipc_msg_payload_hws_resume_engine {1378struct vpu_ipc_msg_payload_hws_resume_engine {
1193 /* Engine to be resumed */1379 /** Engine to be resumed */
1194 u32 engine_idx;1380 u32 engine_idx;
1195 /* Reserved */1381 /** Reserved */
1196 u32 reserved_0;1382 u32 reserved_0;
1197};1383};
1198 1384
@@ -1326,7 +1512,7 @@ struct vpu_jsm_metric_streamer_done { · 127 unmodified lines
1326/**1512/**
1327 * Metric group description placed in the metric buffer after successful completion1513 * Metric group description placed in the metric buffer after successful completion
1328 * of the VPU_JSM_MSG_METRIC_STREAMER_INFO command. This is followed by one or more1514 * of the VPU_JSM_MSG_METRIC_STREAMER_INFO command. This is followed by one or more
1329 * @vpu_jsm_metric_counter_descriptor records.1515 * @ref vpu_jsm_metric_counter_descriptor records.
1330 * @see VPU_JSM_MSG_METRIC_STREAMER_INFO1516 * @see VPU_JSM_MSG_METRIC_STREAMER_INFO
1331 */1517 */
1332struct vpu_jsm_metric_group_descriptor {1518struct vpu_jsm_metric_group_descriptor {
@@ -1413,29 +1599,24 @@ struct vpu_jsm_metric_counter_descriptor { · 80 unmodified lines
1413};1599};
1414 1600
1415/**1601/**
1416 * Payload for VPU_JSM_MSG_DYNDBG_CONTROL requests.1602 * Payload for @ref VPU_JSM_MSG_DYNDBG_CONTROL requests.
1417 *1603 *
1418 * VPU_JSM_MSG_DYNDBG_CONTROL are used to control the VPU FW Dynamic Debug1604 * VPU_JSM_MSG_DYNDBG_CONTROL requests are used to control the VPU FW dynamic debug
1419 * feature, which allows developers to selectively enable / disable MVLOG_DEBUG1605 * feature, which allows developers to selectively enable/disable code to obtain
1420 * messages. This is equivalent to the Dynamic Debug functionality provided by1606 * additional FW information. This is equivalent to the dynamic debug functionality
1421 * Linux1607 * provided by Linux. The host can control dynamic debug behavior by sending dyndbg
1422 * (https://www.kernel.org/doc/html/latest/admin-guide/dynamic-debug-howto.html)1608 * commands, using the same syntax as for Linux dynamic debug commands.
1423 * The host can control Dynamic Debug behavior by sending dyndbg commands, which
1424 * have the same syntax as Linux
1425 * dyndbg commands.
1426 *1609 *
1427 * NOTE: in order for MVLOG_DEBUG messages to be actually printed, the host1610 * @see https://www.kernel.org/doc/html/latest/admin-guide/dynamic-debug-howto.html.
1428 * still has to set the logging level to MVLOG_DEBUG, using the
1429 * VPU_JSM_MSG_TRACE_SET_CONFIG command.
1430 *1611 *
1431 * The host can see the current dynamic debug configuration by executing a1612 * NOTE:
1432 * special 'show' command. The dyndbg configuration will be printed to the1613 * As the dynamic debug feature uses MVLOG messages to provide information, the host
1433 * configured logging destination using MVLOG_INFO logging level.1614 * must first set the logging level to MVLOG_DEBUG, using the @ref VPU_JSM_MSG_TRACE_SET_CONFIG
1615 * command.
1434 */1616 */
1435struct vpu_ipc_msg_payload_dyndbg_control {1617struct vpu_ipc_msg_payload_dyndbg_control {
1436 /**1618 /**
1437 * Dyndbg command (same format as Linux dyndbg); must be a NULL-terminated1619 * Dyndbg command to be executed.
1438 * string.
1439 */1620 */
1440 char dyndbg_cmd[VPU_DYNDBG_CMD_MAX_LEN];1621 char dyndbg_cmd[VPU_DYNDBG_CMD_MAX_LEN];
1441};1622};
@@ -1456,7 +1637,7 @@ struct vpu_ipc_msg_payload_pwr_d0i3_enter { · 14 unmodified lines
1456};1637};
1457 1638
1458/**1639/**
1459 * Payload for VPU_JSM_MSG_DCT_ENABLE message.1640 * Payload for @ref VPU_JSM_MSG_DCT_ENABLE message.
1460 *1641 *
1461 * Default values for DCT active/inactive times are 5.3ms and 30ms respectively,1642 * Default values for DCT active/inactive times are 5.3ms and 30ms respectively,
1462 * corresponding to a 85% duty cycle. This payload allows the host to tune these1643 * corresponding to a 85% duty cycle. This payload allows the host to tune these
@@ -1513,28 +1694,28 @@ union vpu_ipc_msg_payload { · 50 unmodified lines
1513 struct vpu_ipc_msg_payload_pwr_dct_control pwr_dct_control;1694 struct vpu_ipc_msg_payload_pwr_dct_control pwr_dct_control;
1514};1695};
1515 1696
1516/*1697/**
1517 * Host <-> LRT IPC message base structure.1698 * Host <-> NPU IPC message base structure.
1518 *1699 *
1519 * NOTE: All instances of this object must be aligned on a 64B boundary1700 * NOTE: All instances of this object must be aligned on a 64B boundary
1520 * to allow proper handling of VPU cache operations.1701 * to allow proper handling of VPU cache operations.
1521 */1702 */
1522struct vpu_jsm_msg {1703struct vpu_jsm_msg {
1523 /* Reserved */1704 /** Reserved */
1524 u64 reserved_0;1705 u64 reserved_0;
1525 /* Message type, see vpu_ipc_msg_type enum. */1706 /** Message type, see @ref vpu_ipc_msg_type. */
1526 u32 type;1707 u32 type;
1527 /* Buffer status, see vpu_ipc_msg_status enum. */1708 /** Buffer status, see @ref vpu_ipc_msg_status. */
1528 u32 status;1709 u32 status;
1529 /*1710 /**
1530 * Request ID, provided by the host in a request message and passed1711 * Request ID, provided by the host in a request message and passed
1531 * back by VPU in the response message.1712 * back by VPU in the response message.
1532 */1713 */
1533 u32 request_id;1714 u32 request_id;
1534 /* Request return code set by the VPU, see VPU_JSM_STATUS_* defines. */1715 /** Request return code set by the VPU, see VPU_JSM_STATUS_* defines. */
1535 u32 result;1716 u32 result;
1536 u64 reserved_1;1717 u64 reserved_1;
1537 /* Message payload depending on message type, see vpu_ipc_msg_payload union. */1718 /** Message payload depending on message type, see vpu_ipc_msg_payload union. */
1538 union vpu_ipc_msg_payload payload;1719 union vpu_ipc_msg_payload payload;
1539};1720};
1540 1721
drivers/accel/qaic/Kconfig+1−0
@@ -9,6 +9,7 @@ config DRM_ACCEL_QAIC · 8 unmodified lines
9 depends on PCI && HAS_IOMEM9 depends on PCI && HAS_IOMEM
10 depends on MHI_BUS10 depends on MHI_BUS
11 select CRC3211 select CRC32
12 select WANT_DEV_COREDUMP
12 help13 help
13 Enables driver for Qualcomm's Cloud AI accelerator PCIe cards that are14 Enables driver for Qualcomm's Cloud AI accelerator PCIe cards that are
14 designed to accelerate Deep Learning inference workloads.15 designed to accelerate Deep Learning inference workloads.
drivers/accel/qaic/Makefile+2−0
@@ -11,6 +11,8 @@ qaic-y := \ · 10 unmodified lines
11 qaic_data.o \11 qaic_data.o \
12 qaic_drv.o \12 qaic_drv.o \
13 qaic_ras.o \13 qaic_ras.o \
14 qaic_ssr.o \
15 qaic_sysfs.o \
14 qaic_timesync.o \16 qaic_timesync.o \
15 sahara.o17 sahara.o
16 18
drivers/accel/qaic/qaic.h+39−1
@@ -21,6 +21,7 @@ · 20 unmodified lines
21 21
22#define QAIC_DBC_BASE SZ_128K22#define QAIC_DBC_BASE SZ_128K
23#define QAIC_DBC_SIZE SZ_4K23#define QAIC_DBC_SIZE SZ_4K
24#define QAIC_SSR_DBC_SENTINEL U32_MAX /* No ongoing SSR sentinel */
24 25
25#define QAIC_NO_PARTITION -126#define QAIC_NO_PARTITION -1
26 27
@@ -47,6 +48,22 @@ enum __packed dev_states { · 20 unmodified lines
47 QAIC_ONLINE,48 QAIC_ONLINE,
48};49};
49 50
51enum dbc_states {
52 /* DBC is free and can be activated */
53 DBC_STATE_IDLE,
54 /* DBC is activated and a workload is running on device */
55 DBC_STATE_ASSIGNED,
56 /* Sub-system associated with this workload has crashed and it will shutdown soon */
57 DBC_STATE_BEFORE_SHUTDOWN,
58 /* Sub-system associated with this workload has crashed and it has shutdown */
59 DBC_STATE_AFTER_SHUTDOWN,
60 /* Sub-system associated with this workload is shutdown and it will be powered up soon */
61 DBC_STATE_BEFORE_POWER_UP,
62 /* Sub-system associated with this workload is now powered up */
63 DBC_STATE_AFTER_POWER_UP,
64 DBC_STATE_MAX,
65};
66
50extern bool datapath_polling;67extern bool datapath_polling;
51 68
52struct qaic_user {69struct qaic_user {
@@ -114,6 +131,8 @@ struct dma_bridge_chan { · 61 unmodified lines
114 unsigned int irq;131 unsigned int irq;
115 /* Polling work item to simulate interrupts */132 /* Polling work item to simulate interrupts */
116 struct work_struct poll_work;133 struct work_struct poll_work;
134 /* Represents various states of this DBC from enum dbc_states */
135 unsigned int state;
117};136};
118 137
119struct qaic_device {138struct qaic_device {
@@ -161,6 +180,8 @@ struct qaic_device { · 41 unmodified lines
161 struct mhi_device *qts_ch;180 struct mhi_device *qts_ch;
162 /* Work queue for tasks related to MHI "QAIC_TIMESYNC" channel */181 /* Work queue for tasks related to MHI "QAIC_TIMESYNC" channel */
163 struct workqueue_struct *qts_wq;182 struct workqueue_struct *qts_wq;
183 /* MHI "QAIC_TIMESYNC_PERIODIC" channel device */
184 struct mhi_device *mqts_ch;
164 /* Head of list of page allocated by MHI bootlog device */185 /* Head of list of page allocated by MHI bootlog device */
165 struct list_head bootlog;186 struct list_head bootlog;
166 /* MHI bootlog channel device */187 /* MHI bootlog channel device */
@@ -177,6 +198,14 @@ struct qaic_device { · 10 unmodified lines
177 unsigned int ue_count;198 unsigned int ue_count;
178 /* Un-correctable non-fatal error count */199 /* Un-correctable non-fatal error count */
179 unsigned int ue_nf_count;200 unsigned int ue_nf_count;
201 /* MHI SSR channel device */
202 struct mhi_device *ssr_ch;
203 /* Work queue for tasks related to MHI SSR device */
204 struct workqueue_struct *ssr_wq;
205 /* Buffer to collect SSR crashdump via SSR MHI channel */
206 void *ssr_mhi_buf;
207 /* DBC which is under SSR. Sentinel U32_MAX would mean that no SSR in progress */
208 u32 ssr_dbc;
180};209};
181 210
182struct qaic_drm_device {211struct qaic_drm_device {
@@ -195,6 +224,8 @@ struct qaic_drm_device { · 12 unmodified lines
195 struct list_head users;224 struct list_head users;
196 /* Synchronizes access to users list */225 /* Synchronizes access to users list */
197 struct mutex users_mutex;226 struct mutex users_mutex;
227 /* Pointer to array of DBC sysfs attributes */
228 void *sysfs_attrs;
198};229};
199 230
200struct qaic_bo {231struct qaic_bo {
@@ -317,6 +348,13 @@ int qaic_partial_execute_bo_ioctl(struct drm_device *dev, void *data, struct drm · 116 unmodified lines
317int qaic_wait_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv);348int qaic_wait_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv);
318int qaic_perf_stats_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv);349int qaic_perf_stats_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv);
319int qaic_detach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv);350int qaic_detach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv);
320void irq_polling_work(struct work_struct *work);351void qaic_irq_polling_work(struct work_struct *work);
352void qaic_dbc_enter_ssr(struct qaic_device *qdev, u32 dbc_id);
353void qaic_dbc_exit_ssr(struct qaic_device *qdev);
354
355/* qaic_sysfs.c */
356int qaic_sysfs_init(struct qaic_drm_device *qddev);
357void qaic_sysfs_remove(struct qaic_drm_device *qddev);
358void set_dbc_state(struct qaic_device *qdev, u32 dbc_id, unsigned int state);
321 359
322#endif /* _QAIC_H_ */360#endif /* _QAIC_H_ */
drivers/accel/qaic/qaic_control.c+14−11
@@ -17,6 +17,7 @@ · 16 unmodified lines
17#include <linux/overflow.h>17#include <linux/overflow.h>
18#include <linux/pci.h>18#include <linux/pci.h>
19#include <linux/scatterlist.h>19#include <linux/scatterlist.h>
20#include <linux/sched/signal.h>
20#include <linux/types.h>21#include <linux/types.h>
21#include <linux/uaccess.h>22#include <linux/uaccess.h>
22#include <linux/workqueue.h>23#include <linux/workqueue.h>
@@ -30,7 +31,7 @@ · 7 unmodified lines
30#define MANAGE_MAGIC_NUMBER ((__force __le32)0x43494151) /* "QAIC" in little endian */31#define MANAGE_MAGIC_NUMBER ((__force __le32)0x43494151) /* "QAIC" in little endian */
31#define QAIC_DBC_Q_GAP SZ_25632#define QAIC_DBC_Q_GAP SZ_256
32#define QAIC_DBC_Q_BUF_ALIGN SZ_4K33#define QAIC_DBC_Q_BUF_ALIGN SZ_4K
33#define QAIC_MANAGE_EXT_MSG_LENGTH SZ_64K /* Max DMA message length */34#define QAIC_MANAGE_WIRE_MSG_LENGTH SZ_64K /* Max DMA message length */
34#define QAIC_WRAPPER_MAX_SIZE SZ_4K35#define QAIC_WRAPPER_MAX_SIZE SZ_4K
35#define QAIC_MHI_RETRY_WAIT_MS 10036#define QAIC_MHI_RETRY_WAIT_MS 100
36#define QAIC_MHI_RETRY_MAX 2037#define QAIC_MHI_RETRY_MAX 20
@@ -309,6 +310,7 @@ static void save_dbc_buf(struct qaic_device *qdev, struct ioctl_resources *resou · 272 unmodified lines
309 enable_dbc(qdev, dbc_id, usr);310 enable_dbc(qdev, dbc_id, usr);
310 qdev->dbc[dbc_id].in_use = true;311 qdev->dbc[dbc_id].in_use = true;
311 resources->buf = NULL;312 resources->buf = NULL;
313 set_dbc_state(qdev, dbc_id, DBC_STATE_ASSIGNED);
312 }314 }
313}315}
314 316
@@ -367,7 +369,7 @@ static int encode_passthrough(struct qaic_device *qdev, void *trans, struct wrap · 52 unmodified lines
367 if (in_trans->hdr.len % 8 != 0)369 if (in_trans->hdr.len % 8 != 0)
368 return -EINVAL;370 return -EINVAL;
369 371
370 if (size_add(msg_hdr_len, in_trans->hdr.len) > QAIC_MANAGE_EXT_MSG_LENGTH)372 if (size_add(msg_hdr_len, in_trans->hdr.len) > QAIC_MANAGE_WIRE_MSG_LENGTH)
371 return -ENOSPC;373 return -ENOSPC;
372 374
373 trans_wrapper = add_wrapper(wrappers,375 trans_wrapper = add_wrapper(wrappers,
@@ -495,7 +497,7 @@ static int encode_addr_size_pairs(struct dma_xfer *xfer, struct wrapper_list *wr · 121 unmodified lines
495 497
496 nents = sgt->nents;498 nents = sgt->nents;
497 nents_dma = nents;499 nents_dma = nents;
498 *size = QAIC_MANAGE_EXT_MSG_LENGTH - msg_hdr_len - sizeof(**out_trans);500 *size = QAIC_MANAGE_WIRE_MSG_LENGTH - msg_hdr_len - sizeof(**out_trans);
499 for_each_sgtable_dma_sg(sgt, sg, i) {501 for_each_sgtable_dma_sg(sgt, sg, i) {
500 *size -= sizeof(*asp);502 *size -= sizeof(*asp);
501 /* Save 1K for possible follow-up transactions. */503 /* Save 1K for possible follow-up transactions. */
@@ -576,7 +578,7 @@ static int encode_dma(struct qaic_device *qdev, void *trans, struct wrapper_list · 74 unmodified lines
576 578
577 /* There should be enough space to hold at least one ASP entry. */579 /* There should be enough space to hold at least one ASP entry. */
578 if (size_add(msg_hdr_len, sizeof(*out_trans) + sizeof(struct wire_addr_size_pair)) >580 if (size_add(msg_hdr_len, sizeof(*out_trans) + sizeof(struct wire_addr_size_pair)) >
579 QAIC_MANAGE_EXT_MSG_LENGTH)581 QAIC_MANAGE_WIRE_MSG_LENGTH)
580 return -ENOMEM;582 return -ENOMEM;
581 583
582 xfer = kmalloc(sizeof(*xfer), GFP_KERNEL);584 xfer = kmalloc(sizeof(*xfer), GFP_KERNEL);
@@ -645,7 +647,7 @@ static int encode_activate(struct qaic_device *qdev, void *trans, struct wrapper · 62 unmodified lines
645 msg = &wrapper->msg;647 msg = &wrapper->msg;
646 msg_hdr_len = le32_to_cpu(msg->hdr.len);648 msg_hdr_len = le32_to_cpu(msg->hdr.len);
647 649
648 if (size_add(msg_hdr_len, sizeof(*out_trans)) > QAIC_MANAGE_MAX_MSG_LENGTH)650 if (size_add(msg_hdr_len, sizeof(*out_trans)) > QAIC_MANAGE_WIRE_MSG_LENGTH)
649 return -ENOSPC;651 return -ENOSPC;
650 652
651 if (!in_trans->queue_size)653 if (!in_trans->queue_size)
@@ -655,8 +657,9 @@ static int encode_activate(struct qaic_device *qdev, void *trans, struct wrapper · 3 unmodified lines
655 return -EINVAL;657 return -EINVAL;
656 658
657 nelem = in_trans->queue_size;659 nelem = in_trans->queue_size;
658 size = (get_dbc_req_elem_size() + get_dbc_rsp_elem_size()) * nelem;660 if (check_mul_overflow((u32)(get_dbc_req_elem_size() + get_dbc_rsp_elem_size()),
659 if (size / nelem != get_dbc_req_elem_size() + get_dbc_rsp_elem_size())661 nelem,
662 &size))
660 return -EINVAL;663 return -EINVAL;
661 664
662 if (size + QAIC_DBC_Q_GAP + QAIC_DBC_Q_BUF_ALIGN < size)665 if (size + QAIC_DBC_Q_GAP + QAIC_DBC_Q_BUF_ALIGN < size)
@@ -729,7 +732,7 @@ static int encode_status(struct qaic_device *qdev, void *trans, struct wrapper_l · 66 unmodified lines
729 msg = &wrapper->msg;732 msg = &wrapper->msg;
730 msg_hdr_len = le32_to_cpu(msg->hdr.len);733 msg_hdr_len = le32_to_cpu(msg->hdr.len);
731 734
732 if (size_add(msg_hdr_len, in_trans->hdr.len) > QAIC_MANAGE_MAX_MSG_LENGTH)735 if (size_add(msg_hdr_len, in_trans->hdr.len) > QAIC_MANAGE_WIRE_MSG_LENGTH)
733 return -ENOSPC;736 return -ENOSPC;
734 737
735 trans_wrapper = add_wrapper(wrappers, sizeof(*trans_wrapper));738 trans_wrapper = add_wrapper(wrappers, sizeof(*trans_wrapper));
@@ -810,7 +813,7 @@ static int encode_message(struct qaic_device *qdev, struct manage_msg *user_msg, · 74 unmodified lines
810 }813 }
811 814
812 if (ret)815 if (ret)
813 break;816 goto out;
814 }817 }
815 818
816 if (user_len != user_msg->len)819 if (user_len != user_msg->len)
@@ -921,6 +924,7 @@ static int decode_deactivate(struct qaic_device *qdev, void *trans, u32 *msg_len · 104 unmodified lines
921 }924 }
922 925
923 release_dbc(qdev, dbc_id);926 release_dbc(qdev, dbc_id);
927 set_dbc_state(qdev, dbc_id, DBC_STATE_IDLE);
924 *msg_len += sizeof(*in_trans);928 *msg_len += sizeof(*in_trans);
925 929
926 return 0;930 return 0;
@@ -1052,7 +1056,7 @@ static void *msg_xfer(struct qaic_device *qdev, struct wrapper_list *wrappers, u · 125 unmodified lines
1052 init_completion(&elem.xfer_done);1056 init_completion(&elem.xfer_done);
1053 if (likely(!qdev->cntl_lost_buf)) {1057 if (likely(!qdev->cntl_lost_buf)) {
1054 /*1058 /*
1055 * The max size of request to device is QAIC_MANAGE_EXT_MSG_LENGTH.1059 * The max size of request to device is QAIC_MANAGE_WIRE_MSG_LENGTH.
1056 * The max size of response from device is QAIC_MANAGE_MAX_MSG_LENGTH.1060 * The max size of response from device is QAIC_MANAGE_MAX_MSG_LENGTH.
1057 */1061 */
1058 out_buf = kmalloc(QAIC_MANAGE_MAX_MSG_LENGTH, GFP_KERNEL);1062 out_buf = kmalloc(QAIC_MANAGE_MAX_MSG_LENGTH, GFP_KERNEL);
@@ -1079,7 +1083,6 @@ static void *msg_xfer(struct qaic_device *qdev, struct wrapper_list *wrappers, u · 20 unmodified lines
1079 1083
1080 list_for_each_entry(w, &wrappers->list, list) {1084 list_for_each_entry(w, &wrappers->list, list) {
1081 kref_get(&w->ref_count);1085 kref_get(&w->ref_count);
1082 retry_count = 0;
1083 ret = mhi_queue_buf(qdev->cntl_ch, DMA_TO_DEVICE, &w->msg, w->len,1086 ret = mhi_queue_buf(qdev->cntl_ch, DMA_TO_DEVICE, &w->msg, w->len,
1084 list_is_last(&w->list, &wrappers->list) ? MHI_EOT : MHI_CHAIN);1087 list_is_last(&w->list, &wrappers->list) ? MHI_EOT : MHI_CHAIN);
1085 if (ret) {1088 if (ret) {
drivers/accel/qaic/qaic_data.c+116−48
@@ -18,6 +18,7 @@ · 17 unmodified lines
18#include <linux/scatterlist.h>18#include <linux/scatterlist.h>
19#include <linux/spinlock.h>19#include <linux/spinlock.h>
20#include <linux/srcu.h>20#include <linux/srcu.h>
21#include <linux/string.h>
21#include <linux/types.h>22#include <linux/types.h>
22#include <linux/uaccess.h>23#include <linux/uaccess.h>
23#include <linux/wait.h>24#include <linux/wait.h>
@@ -165,7 +166,7 @@ static void free_slice(struct kref *kref) · 141 unmodified lines
165 drm_gem_object_put(&slice->bo->base);166 drm_gem_object_put(&slice->bo->base);
166 sg_free_table(slice->sgt);167 sg_free_table(slice->sgt);
167 kfree(slice->sgt);168 kfree(slice->sgt);
168 kfree(slice->reqs);169 kvfree(slice->reqs);
169 kfree(slice);170 kfree(slice);
170}171}
171 172
@@ -404,7 +405,7 @@ static int qaic_map_one_slice(struct qaic_device *qdev, struct qaic_bo *bo, · 232 unmodified lines
404 goto free_sgt;405 goto free_sgt;
405 }406 }
406 407
407 slice->reqs = kcalloc(sgt->nents, sizeof(*slice->reqs), GFP_KERNEL);408 slice->reqs = kvcalloc(sgt->nents, sizeof(*slice->reqs), GFP_KERNEL);
408 if (!slice->reqs) {409 if (!slice->reqs) {
409 ret = -ENOMEM;410 ret = -ENOMEM;
410 goto free_slice;411 goto free_slice;
@@ -430,7 +431,7 @@ static int qaic_map_one_slice(struct qaic_device *qdev, struct qaic_bo *bo, · 19 unmodified lines
430 return 0;431 return 0;
431 432
432free_req:433free_req:
433 kfree(slice->reqs);434 kvfree(slice->reqs);
434free_slice:435free_slice:
435 kfree(slice);436 kfree(slice);
436free_sgt:437free_sgt:
@@ -643,8 +644,36 @@ static void qaic_free_object(struct drm_gem_object *obj) · 206 unmodified lines
643 kfree(bo);644 kfree(bo);
644}645}
645 646
647static struct sg_table *qaic_get_sg_table(struct drm_gem_object *obj)
648{
649 struct qaic_bo *bo = to_qaic_bo(obj);
650 struct scatterlist *sg, *sg_in;
651 struct sg_table *sgt, *sgt_in;
652 int i;
653
654 sgt_in = bo->sgt;
655
656 sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
657 if (!sgt)
658 return ERR_PTR(-ENOMEM);
659
660 if (sg_alloc_table(sgt, sgt_in->orig_nents, GFP_KERNEL)) {
661 kfree(sgt);
662 return ERR_PTR(-ENOMEM);
663 }
664
665 sg = sgt->sgl;
666 for_each_sgtable_sg(sgt_in, sg_in, i) {
667 memcpy(sg, sg_in, sizeof(*sg));
668 sg = sg_next(sg);
669 }
670
671 return sgt;
672}
673
646static const struct drm_gem_object_funcs qaic_gem_funcs = {674static const struct drm_gem_object_funcs qaic_gem_funcs = {
647 .free = qaic_free_object,675 .free = qaic_free_object,
676 .get_sg_table = qaic_get_sg_table,
648 .print_info = qaic_gem_print_info,677 .print_info = qaic_gem_print_info,
649 .mmap = qaic_gem_object_mmap,678 .mmap = qaic_gem_object_mmap,
650 .vm_ops = &drm_vm_ops,679 .vm_ops = &drm_vm_ops,
@@ -953,8 +982,9 @@ int qaic_attach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_fi · 302 unmodified lines
953 if (args->hdr.count == 0)982 if (args->hdr.count == 0)
954 return -EINVAL;983 return -EINVAL;
955 984
956 arg_size = args->hdr.count * sizeof(*slice_ent);985 if (check_mul_overflow((unsigned long)args->hdr.count,
957 if (arg_size / args->hdr.count != sizeof(*slice_ent))986 (unsigned long)sizeof(*slice_ent),
987 &arg_size))
958 return -EINVAL;988 return -EINVAL;
959 989
960 if (!(args->hdr.dir == DMA_TO_DEVICE || args->hdr.dir == DMA_FROM_DEVICE))990 if (!(args->hdr.dir == DMA_TO_DEVICE || args->hdr.dir == DMA_FROM_DEVICE))
@@ -984,18 +1014,12 @@ int qaic_attach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_fi · 23 unmodified lines
984 1014
985 user_data = u64_to_user_ptr(args->data);1015 user_data = u64_to_user_ptr(args->data);
986 1016
987 slice_ent = kzalloc(arg_size, GFP_KERNEL);1017 slice_ent = memdup_user(user_data, arg_size);
988 if (!slice_ent) {1018 if (IS_ERR(slice_ent)) {
989 ret = -EINVAL;1019 ret = PTR_ERR(slice_ent);
990 goto unlock_dev_srcu;1020 goto unlock_dev_srcu;
991 }1021 }
992 1022
993 ret = copy_from_user(slice_ent, user_data, arg_size);
994 if (ret) {
995 ret = -EFAULT;
996 goto free_slice_ent;
997 }
998
999 obj = drm_gem_object_lookup(file_priv, args->hdr.handle);1023 obj = drm_gem_object_lookup(file_priv, args->hdr.handle);
1000 if (!obj) {1024 if (!obj) {
1001 ret = -ENOENT;1025 ret = -ENOENT;
@@ -1023,6 +1047,11 @@ int qaic_attach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_fi · 21 unmodified lines
1023 goto unlock_ch_srcu;1047 goto unlock_ch_srcu;
1024 }1048 }
1025 1049
1050 if (dbc->id == qdev->ssr_dbc) {
1051 ret = -EPIPE;
1052 goto unlock_ch_srcu;
1053 }
1054
1026 ret = qaic_prepare_bo(qdev, bo, &args->hdr);1055 ret = qaic_prepare_bo(qdev, bo, &args->hdr);
1027 if (ret)1056 if (ret)
1028 goto unlock_ch_srcu;1057 goto unlock_ch_srcu;
@@ -1300,8 +1329,6 @@ static int __qaic_execute_bo_ioctl(struct drm_device *dev, void *data, struct dr · 271 unmodified lines
1300 int usr_rcu_id, qdev_rcu_id;1329 int usr_rcu_id, qdev_rcu_id;
1301 struct qaic_device *qdev;1330 struct qaic_device *qdev;
1302 struct qaic_user *usr;1331 struct qaic_user *usr;
1303 u8 __user *user_data;
1304 unsigned long n;
1305 u64 received_ts;1332 u64 received_ts;
1306 u32 queue_level;1333 u32 queue_level;
1307 u64 submit_ts;1334 u64 submit_ts;
@@ -1314,20 +1341,12 @@ static int __qaic_execute_bo_ioctl(struct drm_device *dev, void *data, struct dr · 6 unmodified lines
1314 received_ts = ktime_get_ns();1341 received_ts = ktime_get_ns();
1315 1342
1316 size = is_partial ? sizeof(struct qaic_partial_execute_entry) : sizeof(*exec);1343 size = is_partial ? sizeof(struct qaic_partial_execute_entry) : sizeof(*exec);
1317 n = (unsigned long)size * args->hdr.count;1344 if (args->hdr.count == 0)
1318 if (args->hdr.count == 0 || n / args->hdr.count != size)
1319 return -EINVAL;1345 return -EINVAL;
1320 1346
1321 user_data = u64_to_user_ptr(args->data);1347 exec = memdup_array_user(u64_to_user_ptr(args->data), args->hdr.count, size);
1322 1348 if (IS_ERR(exec))
1323 exec = kcalloc(args->hdr.count, size, GFP_KERNEL);1349 return PTR_ERR(exec);
1324 if (!exec)
1325 return -ENOMEM;
1326
1327 if (copy_from_user(exec, user_data, n)) {
1328 ret = -EFAULT;
1329 goto free_exec;
1330 }
1331 1350
1332 usr = file_priv->driver_priv;1351 usr = file_priv->driver_priv;
1333 usr_rcu_id = srcu_read_lock(&usr->qddev_lock);1352 usr_rcu_id = srcu_read_lock(&usr->qddev_lock);
@@ -1356,6 +1375,11 @@ static int __qaic_execute_bo_ioctl(struct drm_device *dev, void *data, struct dr · 22 unmodified lines
1356 goto release_ch_rcu;1375 goto release_ch_rcu;
1357 }1376 }
1358 1377
1378 if (dbc->id == qdev->ssr_dbc) {
1379 ret = -EPIPE;
1380 goto release_ch_rcu;
1381 }
1382
1359 ret = mutex_lock_interruptible(&dbc->req_lock);1383 ret = mutex_lock_interruptible(&dbc->req_lock);
1360 if (ret)1384 if (ret)
1361 goto release_ch_rcu;1385 goto release_ch_rcu;
@@ -1396,7 +1420,6 @@ unlock_dev_srcu: · 34 unmodified lines
1396 srcu_read_unlock(&qdev->dev_lock, qdev_rcu_id);1420 srcu_read_unlock(&qdev->dev_lock, qdev_rcu_id);
1397unlock_usr_srcu:1421unlock_usr_srcu:
1398 srcu_read_unlock(&usr->qddev_lock, usr_rcu_id);1422 srcu_read_unlock(&usr->qddev_lock, usr_rcu_id);
1399free_exec:
1400 kfree(exec);1423 kfree(exec);
1401 return ret;1424 return ret;
1402}1425}
@@ -1491,7 +1514,7 @@ irqreturn_t dbc_irq_handler(int irq, void *data) · 88 unmodified lines
1491 return IRQ_WAKE_THREAD;1514 return IRQ_WAKE_THREAD;
1492}1515}
1493 1516
1494void irq_polling_work(struct work_struct *work)1517void qaic_irq_polling_work(struct work_struct *work)
1495{1518{
1496 struct dma_bridge_chan *dbc = container_of(work, struct dma_bridge_chan, poll_work);1519 struct dma_bridge_chan *dbc = container_of(work, struct dma_bridge_chan, poll_work);
1497 unsigned long flags;1520 unsigned long flags;
@@ -1709,6 +1732,11 @@ int qaic_wait_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file · 211 unmodified lines
1709 goto unlock_ch_srcu;1732 goto unlock_ch_srcu;
1710 }1733 }
1711 1734
1735 if (dbc->id == qdev->ssr_dbc) {
1736 ret = -EPIPE;
1737 goto unlock_ch_srcu;
1738 }
1739
1712 obj = drm_gem_object_lookup(file_priv, args->handle);1740 obj = drm_gem_object_lookup(file_priv, args->handle);
1713 if (!obj) {1741 if (!obj) {
1714 ret = -ENOENT;1742 ret = -ENOENT;
@@ -1729,6 +1757,9 @@ int qaic_wait_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file · 14 unmodified lines
1729 if (!dbc->usr)1757 if (!dbc->usr)
1730 ret = -EPERM;1758 ret = -EPERM;
1731 1759
1760 if (dbc->id == qdev->ssr_dbc)
1761 ret = -EPIPE;
1762
1732put_obj:1763put_obj:
1733 drm_gem_object_put(obj);1764 drm_gem_object_put(obj);
1734unlock_ch_srcu:1765unlock_ch_srcu:
@@ -1749,7 +1780,8 @@ int qaic_perf_stats_bo_ioctl(struct drm_device *dev, void *data, struct drm_file · 14 unmodified lines
1749 struct qaic_device *qdev;1780 struct qaic_device *qdev;
1750 struct qaic_user *usr;1781 struct qaic_user *usr;
1751 struct qaic_bo *bo;1782 struct qaic_bo *bo;
1752 int ret, i;1783 int ret = 0;
1784 int i;
1753 1785
1754 usr = file_priv->driver_priv;1786 usr = file_priv->driver_priv;
1755 usr_rcu_id = srcu_read_lock(&usr->qddev_lock);1787 usr_rcu_id = srcu_read_lock(&usr->qddev_lock);
@@ -1770,18 +1802,12 @@ int qaic_perf_stats_bo_ioctl(struct drm_device *dev, void *data, struct drm_file · 14 unmodified lines
1770 goto unlock_dev_srcu;1802 goto unlock_dev_srcu;
1771 }1803 }
1772 1804
1773 ent = kcalloc(args->hdr.count, sizeof(*ent), GFP_KERNEL);1805 ent = memdup_array_user(u64_to_user_ptr(args->data), args->hdr.count, sizeof(*ent));
1774 if (!ent) {1806 if (IS_ERR(ent)) {
1775 ret = -EINVAL;1807 ret = PTR_ERR(ent);
1776 goto unlock_dev_srcu;1808 goto unlock_dev_srcu;
1777 }1809 }
1778 1810
1779 ret = copy_from_user(ent, u64_to_user_ptr(args->data), args->hdr.count * sizeof(*ent));
1780 if (ret) {
1781 ret = -EFAULT;
1782 goto free_ent;
1783 }
1784
1785 for (i = 0; i < args->hdr.count; i++) {1811 for (i = 0; i < args->hdr.count; i++) {
1786 obj = drm_gem_object_lookup(file_priv, ent[i].handle);1812 obj = drm_gem_object_lookup(file_priv, ent[i].handle);
1787 if (!obj) {1813 if (!obj) {
@@ -1789,6 +1815,16 @@ int qaic_perf_stats_bo_ioctl(struct drm_device *dev, void *data, struct drm_file · 1 unmodified line
1789 goto free_ent;1815 goto free_ent;
1790 }1816 }
1791 bo = to_qaic_bo(obj);1817 bo = to_qaic_bo(obj);
1818 if (!bo->sliced) {
1819 drm_gem_object_put(obj);
1820 ret = -EINVAL;
1821 goto free_ent;
1822 }
1823 if (bo->dbc->id != args->hdr.dbc_id) {
1824 drm_gem_object_put(obj);
1825 ret = -EINVAL;
1826 goto free_ent;
1827 }
1792 /*1828 /*
1793 * perf stats ioctl is called before wait ioctl is complete then1829 * perf stats ioctl is called before wait ioctl is complete then
1794 * the latency information is invalid.1830 * the latency information is invalid.
@@ -1927,6 +1963,17 @@ static void empty_xfer_list(struct qaic_device *qdev, struct dma_bridge_chan *db · 132 unmodified lines
1927 spin_unlock_irqrestore(&dbc->xfer_lock, flags);1963 spin_unlock_irqrestore(&dbc->xfer_lock, flags);
1928}1964}
1929 1965
1966static void sync_empty_xfer_list(struct qaic_device *qdev, struct dma_bridge_chan *dbc)
1967{
1968 empty_xfer_list(qdev, dbc);
1969 synchronize_srcu(&dbc->ch_lock);
1970 /*
1971 * Threads holding channel lock, may add more elements in the xfer_list.
1972 * Flush out these elements from xfer_list.
1973 */
1974 empty_xfer_list(qdev, dbc);
1975}
1976
1930int disable_dbc(struct qaic_device *qdev, u32 dbc_id, struct qaic_user *usr)1977int disable_dbc(struct qaic_device *qdev, u32 dbc_id, struct qaic_user *usr)
1931{1978{
1932 if (!qdev->dbc[dbc_id].usr || qdev->dbc[dbc_id].usr->handle != usr->handle)1979 if (!qdev->dbc[dbc_id].usr || qdev->dbc[dbc_id].usr->handle != usr->handle)
@@ -1941,7 +1988,7 @@ int disable_dbc(struct qaic_device *qdev, u32 dbc_id, struct qaic_user *usr) · 8 unmodified lines
1941 * enable_dbc - Enable the DBC. DBCs are disabled by removing the context of1988 * enable_dbc - Enable the DBC. DBCs are disabled by removing the context of
1942 * user. Add user context back to DBC to enable it. This function trusts the1989 * user. Add user context back to DBC to enable it. This function trusts the
1943 * DBC ID passed and expects the DBC to be disabled.1990 * DBC ID passed and expects the DBC to be disabled.
1944 * @qdev: Qranium device handle1991 * @qdev: qaic device handle
1945 * @dbc_id: ID of the DBC1992 * @dbc_id: ID of the DBC
1946 * @usr: User context1993 * @usr: User context
1947 */1994 */
@@ -1955,13 +2002,7 @@ void wakeup_dbc(struct qaic_device *qdev, u32 dbc_id) · 7 unmodified lines
1955 struct dma_bridge_chan *dbc = &qdev->dbc[dbc_id];2002 struct dma_bridge_chan *dbc = &qdev->dbc[dbc_id];
1956 2003
1957 dbc->usr = NULL;2004 dbc->usr = NULL;
1958 empty_xfer_list(qdev, dbc);2005 sync_empty_xfer_list(qdev, dbc);
1959 synchronize_srcu(&dbc->ch_lock);
1960 /*
1961 * Threads holding channel lock, may add more elements in the xfer_list.
1962 * Flush out these elements from xfer_list.
1963 */
1964 empty_xfer_list(qdev, dbc);
1965}2006}
1966 2007
1967void release_dbc(struct qaic_device *qdev, u32 dbc_id)2008void release_dbc(struct qaic_device *qdev, u32 dbc_id)
@@ -2002,3 +2043,30 @@ void qaic_data_get_fifo_info(struct dma_bridge_chan *dbc, u32 *head, u32 *tail) · 34 unmodified lines
2002 *head = readl(dbc->dbc_base + REQHP_OFF);2043 *head = readl(dbc->dbc_base + REQHP_OFF);
2003 *tail = readl(dbc->dbc_base + REQTP_OFF);2044 *tail = readl(dbc->dbc_base + REQTP_OFF);
2004}2045}
2046
2047/*
2048 * qaic_dbc_enter_ssr - Prepare to enter in sub system reset(SSR) for given DBC ID.
2049 * @qdev: qaic device handle
2050 * @dbc_id: ID of the DBC which will enter SSR
2051 *
2052 * The device will automatically deactivate the workload as not
2053 * all errors can be silently recovered. The user will be
2054 * notified and will need to decide the required recovery
2055 * action to take.
2056 */
2057void qaic_dbc_enter_ssr(struct qaic_device *qdev, u32 dbc_id)
2058{
2059 qdev->ssr_dbc = dbc_id;
2060 release_dbc(qdev, dbc_id);
2061}
2062
2063/*
2064 * qaic_dbc_exit_ssr - Prepare to exit from sub system reset(SSR) for given DBC ID.
2065 * @qdev: qaic device handle
2066 *
2067 * The DBC returns to an operational state and begins accepting work after exiting SSR.
2068 */
2069void qaic_dbc_exit_ssr(struct qaic_device *qdev)
2070{
2071 qdev->ssr_dbc = QAIC_SSR_DBC_SENTINEL;
2072}
drivers/accel/qaic/qaic_drv.c+115−1
@@ -30,6 +30,7 @@ · 29 unmodified lines
30#include "qaic.h"30#include "qaic.h"
31#include "qaic_debugfs.h"31#include "qaic_debugfs.h"
32#include "qaic_ras.h"32#include "qaic_ras.h"
33#include "qaic_ssr.h"
33#include "qaic_timesync.h"34#include "qaic_timesync.h"
34#include "sahara.h"35#include "sahara.h"
35 36
@@ -270,6 +271,13 @@ static int qaic_create_drm_device(struct qaic_device *qdev, s32 partition_id) · 234 unmodified lines
270 return ret;271 return ret;
271 }272 }
272 273
274 ret = qaic_sysfs_init(qddev);
275 if (ret) {
276 drm_dev_unregister(drm);
277 pci_dbg(qdev->pdev, "qaic_sysfs_init failed %d\n", ret);
278 return ret;
279 }
280
273 qaic_debugfs_init(qddev);281 qaic_debugfs_init(qddev);
274 282
275 return ret;283 return ret;
@@ -281,6 +289,7 @@ static void qaic_destroy_drm_device(struct qaic_device *qdev, s32 partition_id) · 5 unmodified lines
281 struct drm_device *drm = to_drm(qddev);289 struct drm_device *drm = to_drm(qddev);
282 struct qaic_user *usr;290 struct qaic_user *usr;
283 291
292 qaic_sysfs_remove(qddev);
284 drm_dev_unregister(drm);293 drm_dev_unregister(drm);
285 qddev->partition_id = 0;294 qddev->partition_id = 0;
286 /*295 /*
@@ -382,6 +391,7 @@ void qaic_dev_reset_clean_local_state(struct qaic_device *qdev) · 95 unmodified lines
382 qaic_notify_reset(qdev);391 qaic_notify_reset(qdev);
383 392
384 /* start tearing things down */393 /* start tearing things down */
394 qaic_clean_up_ssr(qdev);
385 for (i = 0; i < qdev->num_dbc; ++i)395 for (i = 0; i < qdev->num_dbc; ++i)
386 release_dbc(qdev, i);396 release_dbc(qdev, i);
387}397}
@@ -431,11 +441,18 @@ static struct qaic_device *create_qdev(struct pci_dev *pdev, · 43 unmodified lines
431 qdev->qts_wq = qaicm_wq_init(drm, "qaic_ts");441 qdev->qts_wq = qaicm_wq_init(drm, "qaic_ts");
432 if (IS_ERR(qdev->qts_wq))442 if (IS_ERR(qdev->qts_wq))
433 return NULL;443 return NULL;
444 qdev->ssr_wq = qaicm_wq_init(drm, "qaic_ssr");
445 if (IS_ERR(qdev->ssr_wq))
446 return NULL;
434 447
435 ret = qaicm_srcu_init(drm, &qdev->dev_lock);448 ret = qaicm_srcu_init(drm, &qdev->dev_lock);
436 if (ret)449 if (ret)
437 return NULL;450 return NULL;
438 451
452 ret = qaic_ssr_init(qdev, drm);
453 if (ret)
454 pci_info(pdev, "QAIC SSR crashdump collection not supported.\n");
455
439 qdev->qddev = qddev;456 qdev->qddev = qddev;
440 qdev->pdev = pdev;457 qdev->pdev = pdev;
441 qddev->qdev = qdev;458 qddev->qdev = qdev;
@@ -545,7 +562,7 @@ static int init_msi(struct qaic_device *qdev, struct pci_dev *pdev) · 103 unmodified lines
545 qdev->dbc[i].irq = pci_irq_vector(pdev, qdev->single_msi ? 0 : i + 1);562 qdev->dbc[i].irq = pci_irq_vector(pdev, qdev->single_msi ? 0 : i + 1);
546 if (!qdev->single_msi)563 if (!qdev->single_msi)
547 disable_irq_nosync(qdev->dbc[i].irq);564 disable_irq_nosync(qdev->dbc[i].irq);
548 INIT_WORK(&qdev->dbc[i].poll_work, irq_polling_work);565 INIT_WORK(&qdev->dbc[i].poll_work, qaic_irq_polling_work);
549 }566 }
550 }567 }
551 568
@@ -660,6 +677,92 @@ static const struct pci_error_handlers qaic_pci_err_handler = { · 108 unmodified lines
660 .reset_done = qaic_pci_reset_done,677 .reset_done = qaic_pci_reset_done,
661};678};
662 679
680static bool qaic_is_under_reset(struct qaic_device *qdev)
681{
682 int rcu_id;
683 bool ret;
684
685 rcu_id = srcu_read_lock(&qdev->dev_lock);
686 ret = qdev->dev_state != QAIC_ONLINE;
687 srcu_read_unlock(&qdev->dev_lock, rcu_id);
688 return ret;
689}
690
691static bool qaic_data_path_busy(struct qaic_device *qdev)
692{
693 bool ret = false;
694 int dev_rcu_id;
695 int i;
696
697 dev_rcu_id = srcu_read_lock(&qdev->dev_lock);
698 if (qdev->dev_state != QAIC_ONLINE) {
699 srcu_read_unlock(&qdev->dev_lock, dev_rcu_id);
700 return false;
701 }
702 for (i = 0; i < qdev->num_dbc; i++) {
703 struct dma_bridge_chan *dbc = &qdev->dbc[i];
704 unsigned long flags;
705 int ch_rcu_id;
706
707 ch_rcu_id = srcu_read_lock(&dbc->ch_lock);
708 if (!dbc->usr || !dbc->in_use) {
709 srcu_read_unlock(&dbc->ch_lock, ch_rcu_id);
710 continue;
711 }
712 spin_lock_irqsave(&dbc->xfer_lock, flags);
713 ret = !list_empty(&dbc->xfer_list);
714 spin_unlock_irqrestore(&dbc->xfer_lock, flags);
715 srcu_read_unlock(&dbc->ch_lock, ch_rcu_id);
716 if (ret)
717 break;
718 }
719 srcu_read_unlock(&qdev->dev_lock, dev_rcu_id);
720 return ret;
721}
722
723static int qaic_pm_suspend(struct device *dev)
724{
725 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(dev));
726
727 dev_dbg(dev, "Suspending..\n");
728 if (qaic_data_path_busy(qdev)) {
729 dev_dbg(dev, "Device's datapath is busy. Aborting suspend..\n");
730 return -EBUSY;
731 }
732 if (qaic_is_under_reset(qdev)) {
733 dev_dbg(dev, "Device is under reset. Aborting suspend..\n");
734 return -EBUSY;
735 }
736 qaic_mqts_ch_stop_timer(qdev->mqts_ch);
737 qaic_pci_reset_prepare(qdev->pdev);
738 pci_save_state(qdev->pdev);
739 pci_disable_device(qdev->pdev);
740 pci_set_power_state(qdev->pdev, PCI_D3hot);
741 return 0;
742}
743
744static int qaic_pm_resume(struct device *dev)
745{
746 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(dev));
747 int ret;
748
749 dev_dbg(dev, "Resuming..\n");
750 pci_set_power_state(qdev->pdev, PCI_D0);
751 pci_restore_state(qdev->pdev);
752 ret = pci_enable_device(qdev->pdev);
753 if (ret) {
754 dev_err(dev, "pci_enable_device failed on resume %d\n", ret);
755 return ret;
756 }
757 pci_set_master(qdev->pdev);
758 qaic_pci_reset_done(qdev->pdev);
759 return 0;
760}
761
762static const struct dev_pm_ops qaic_pm_ops = {
763 SYSTEM_SLEEP_PM_OPS(qaic_pm_suspend, qaic_pm_resume)
764};
765
663static struct pci_driver qaic_pci_driver = {766static struct pci_driver qaic_pci_driver = {
664 .name = QAIC_NAME,767 .name = QAIC_NAME,
665 .id_table = qaic_ids,768 .id_table = qaic_ids,
@@ -667,6 +770,9 @@ static struct pci_driver qaic_pci_driver = { · 1 unmodified line
667 .remove = qaic_pci_remove,770 .remove = qaic_pci_remove,
668 .shutdown = qaic_pci_shutdown,771 .shutdown = qaic_pci_shutdown,
669 .err_handler = &qaic_pci_err_handler,772 .err_handler = &qaic_pci_err_handler,
773 .driver = {
774 .pm = pm_sleep_ptr(&qaic_pm_ops),
775 },
670};776};
671 777
672static int __init qaic_init(void)778static int __init qaic_init(void)
@@ -702,9 +808,16 @@ static int __init qaic_init(void) · 29 unmodified lines
702 ret = qaic_ras_register();808 ret = qaic_ras_register();
703 if (ret)809 if (ret)
704 pr_debug("qaic: qaic_ras_register failed %d\n", ret);810 pr_debug("qaic: qaic_ras_register failed %d\n", ret);
811 ret = qaic_ssr_register();
812 if (ret) {
813 pr_debug("qaic: qaic_ssr_register failed %d\n", ret);
814 goto free_bootlog;
815 }
705 816
706 return 0;817 return 0;
707 818
819free_bootlog:
820 qaic_bootlog_unregister();
708free_mhi:821free_mhi:
709 mhi_driver_unregister(&qaic_mhi_driver);822 mhi_driver_unregister(&qaic_mhi_driver);
710free_pci:823free_pci:
@@ -730,6 +843,7 @@ static void __exit qaic_exit(void) · 19 unmodified lines
730 * reinitializing the link_up state after the cleanup is done.843 * reinitializing the link_up state after the cleanup is done.
731 */844 */
732 link_up = true;845 link_up = true;
846 qaic_ssr_unregister();
733 qaic_ras_unregister();847 qaic_ras_unregister();
734 qaic_bootlog_unregister();848 qaic_bootlog_unregister();
735 qaic_timesync_deinit();849 qaic_timesync_deinit();
drivers/accel/qaic/qaic_ras.c+3−3
@@ -514,21 +514,21 @@ static ssize_t ce_count_show(struct device *dev, struct device_attribute *attr, · 513 unmodified lines
514{514{
515 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(dev));515 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(dev));
516 516
517 return snprintf(buf, PAGE_SIZE, "%d\n", qdev->ce_count);517 return sysfs_emit(buf, "%d\n", qdev->ce_count);
518}518}
519 519
520static ssize_t ue_count_show(struct device *dev, struct device_attribute *attr, char *buf)520static ssize_t ue_count_show(struct device *dev, struct device_attribute *attr, char *buf)
521{521{
522 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(dev));522 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(dev));
523 523
524 return snprintf(buf, PAGE_SIZE, "%d\n", qdev->ue_count);524 return sysfs_emit(buf, "%d\n", qdev->ue_count);
525}525}
526 526
527static ssize_t ue_nonfatal_count_show(struct device *dev, struct device_attribute *attr, char *buf)527static ssize_t ue_nonfatal_count_show(struct device *dev, struct device_attribute *attr, char *buf)
528{528{
529 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(dev));529 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(dev));
530 530
531 return snprintf(buf, PAGE_SIZE, "%d\n", qdev->ue_nf_count);531 return sysfs_emit(buf, "%d\n", qdev->ue_nf_count);
532}532}
533 533
534static DEVICE_ATTR_RO(ce_count);534static DEVICE_ATTR_RO(ce_count);
drivers/accel/qaic/qaic_ssr.c+815−0
@@ -0,0 +1,815 @@
1// SPDX-License-Identifier: GPL-2.0-only
2
3/* Copyright (c) 2020-2021, The Linux Foundation. All rights reserved. */
4/* Copyright (c) 2021-2024 Qualcomm Innovation Center, Inc. All rights reserved. */
5
6#include <asm/byteorder.h>
7#include <drm/drm_file.h>
8#include <drm/drm_managed.h>
9#include <linux/devcoredump.h>
10#include <linux/device.h>
11#include <linux/kernel.h>
12#include <linux/mhi.h>
13#include <linux/workqueue.h>
14
15#include "qaic.h"
16#include "qaic_ssr.h"
17
18#define SSR_RESP_MSG_SZ 32
19#define SSR_MHI_BUF_SIZE SZ_64K
20#define SSR_MEM_READ_DATA_SIZE ((u64)SSR_MHI_BUF_SIZE - sizeof(struct ssr_crashdump))
21#define SSR_MEM_READ_CHUNK_SIZE ((u64)SSR_MEM_READ_DATA_SIZE - sizeof(struct ssr_memory_read_rsp))
22
23#define DEBUG_TRANSFER_INFO BIT(0)
24#define DEBUG_TRANSFER_INFO_RSP BIT(1)
25#define MEMORY_READ BIT(2)
26#define MEMORY_READ_RSP BIT(3)
27#define DEBUG_TRANSFER_DONE BIT(4)
28#define DEBUG_TRANSFER_DONE_RSP BIT(5)
29#define SSR_EVENT BIT(8)
30#define SSR_EVENT_RSP BIT(9)
31
32#define SSR_EVENT_NACK BIT(0)
33#define BEFORE_SHUTDOWN BIT(1)
34#define AFTER_SHUTDOWN BIT(2)
35#define BEFORE_POWER_UP BIT(3)
36#define AFTER_POWER_UP BIT(4)
37
38struct debug_info_table {
39 /* Save preferences. Default is mandatory */
40 u64 save_perf;
41 /* Base address of the debug region */
42 u64 mem_base;
43 /* Size of debug region in bytes */
44 u64 len;
45 /* Description */
46 char desc[20];
47 /* Filename of debug region */
48 char filename[20];
49};
50
51struct _ssr_hdr {
52 __le32 cmd;
53 __le32 len;
54 __le32 dbc_id;
55};
56
57struct ssr_hdr {
58 u32 cmd;
59 u32 len;
60 u32 dbc_id;
61};
62
63struct ssr_debug_transfer_info {
64 struct ssr_hdr hdr;
65 u32 resv;
66 u64 tbl_addr;
67 u64 tbl_len;
68} __packed;
69
70struct ssr_debug_transfer_info_rsp {
71 struct _ssr_hdr hdr;
72 __le32 ret;
73} __packed;
74
75struct ssr_memory_read {
76 struct _ssr_hdr hdr;
77 __le32 resv;
78 __le64 addr;
79 __le64 len;
80} __packed;
81
82struct ssr_memory_read_rsp {
83 struct _ssr_hdr hdr;
84 __le32 resv;
85 u8 data[];
86} __packed;
87
88struct ssr_debug_transfer_done {
89 struct _ssr_hdr hdr;
90 __le32 resv;
91} __packed;
92
93struct ssr_debug_transfer_done_rsp {
94 struct _ssr_hdr hdr;
95 __le32 ret;
96} __packed;
97
98struct ssr_event {
99 struct ssr_hdr hdr;
100 u32 event;
101} __packed;
102
103struct ssr_event_rsp {
104 struct _ssr_hdr hdr;
105 __le32 event;
106} __packed;
107
108struct ssr_resp {
109 /* Work struct to schedule work coming on QAIC_SSR channel */
110 struct work_struct work;
111 /* Root struct of device, used to access device resources */
112 struct qaic_device *qdev;
113 /* Buffer used by MHI for transfer requests */
114 u8 data[] __aligned(8);
115};
116
117/* SSR crashdump book keeping structure */
118struct ssr_dump_info {
119 /* DBC associated with this SSR crashdump */
120 struct dma_bridge_chan *dbc;
121 /*
122 * It will be used when we complete the crashdump download and switch
123 * to waiting on SSR events
124 */
125 struct ssr_resp *resp;
126 /* MEMORY READ request MHI buffer.*/
127 struct ssr_memory_read *read_buf_req;
128 /* TRUE: ->read_buf_req is queued for MHI transaction. FALSE: Otherwise */
129 bool read_buf_req_queued;
130 /* Address of table in host */
131 void *tbl_addr;
132 /* Total size of table */
133 u64 tbl_len;
134 /* Offset of table(->tbl_addr) where the new chunk will be dumped */
135 u64 tbl_off;
136 /* Address of table in device/target */
137 u64 tbl_addr_dev;
138 /* Ptr to the entire dump */
139 void *dump_addr;
140 /* Entire crashdump size */
141 u64 dump_sz;
142 /* Offset of crashdump(->dump_addr) where the new chunk will be dumped */
143 u64 dump_off;
144 /* Points to the table entry we are currently downloading */
145 struct debug_info_table *tbl_ent;
146 /* Offset in the current table entry(->tbl_ent) for next chuck */
147 u64 tbl_ent_off;
148};
149
150struct ssr_crashdump {
151 /*
152 * Points to a book keeping struct maintained by MHI SSR device while
153 * downloading a SSR crashdump. It is NULL when crashdump downloading
154 * not in progress.
155 */
156 struct ssr_dump_info *dump_info;
157 /* Work struct to schedule work coming on QAIC_SSR channel */
158 struct work_struct work;
159 /* Root struct of device, used to access device resources */
160 struct qaic_device *qdev;
161 /* Buffer used by MHI for transfer requests */
162 u8 data[];
163};
164
165#define QAIC_SSR_DUMP_V1_MAGIC 0x1234567890abcdef
166#define QAIC_SSR_DUMP_V1_VER 1
167struct dump_file_meta {
168 u64 magic;
169 u64 version;
170 u64 size; /* Total size of the entire dump */
171 u64 tbl_len; /* Length of the table in byte */
172};
173
174/*
175 * Layout of crashdump
176 * +------------------------------------------+
177 * | Crashdump Meta structure |
178 * | type: struct dump_file_meta |
179 * +------------------------------------------+
180 * | Crashdump Table |
181 * | type: array of struct debug_info_table |
182 * | |
183 * | |
184 * | |
185 * +------------------------------------------+
186 * | Crashdump |
187 * | |
188 * | |
189 * | |
190 * | |
191 * | |
192 * +------------------------------------------+
193 */
194
195static void free_ssr_dump_info(struct ssr_crashdump *ssr_crash)
196{
197 struct ssr_dump_info *dump_info = ssr_crash->dump_info;
198
199 ssr_crash->dump_info = NULL;
200 if (!dump_info)
201 return;
202 if (!dump_info->read_buf_req_queued)
203 kfree(dump_info->read_buf_req);
204 vfree(dump_info->tbl_addr);
205 vfree(dump_info->dump_addr);
206 kfree(dump_info);
207}
208
209void qaic_clean_up_ssr(struct qaic_device *qdev)
210{
211 struct ssr_crashdump *ssr_crash = qdev->ssr_mhi_buf;
212
213 if (!ssr_crash)
214 return;
215
216 qaic_dbc_exit_ssr(qdev);
217 free_ssr_dump_info(ssr_crash);
218}
219
220static int alloc_dump(struct ssr_dump_info *dump_info)
221{
222 struct debug_info_table *tbl_ent = dump_info->tbl_addr;
223 struct dump_file_meta *dump_meta;
224 u64 tbl_sz_lp = 0;
225 u64 dump_size = 0;
226
227 while (tbl_sz_lp < dump_info->tbl_len) {
228 le64_to_cpus(&tbl_ent->save_perf);
229 le64_to_cpus(&tbl_ent->mem_base);
230 le64_to_cpus(&tbl_ent->len);
231
232 if (tbl_ent->len == 0)
233 return -EINVAL;
234
235 dump_size += tbl_ent->len;
236 tbl_ent++;
237 tbl_sz_lp += sizeof(*tbl_ent);
238 }
239
240 dump_info->dump_sz = dump_size + dump_info->tbl_len + sizeof(*dump_meta);
241 dump_info->dump_addr = vzalloc(dump_info->dump_sz);
242 if (!dump_info->dump_addr)
243 return -ENOMEM;
244
245 /* Copy crashdump meta and table */
246 dump_meta = dump_info->dump_addr;
247 dump_meta->magic = QAIC_SSR_DUMP_V1_MAGIC;
248 dump_meta->version = QAIC_SSR_DUMP_V1_VER;
249 dump_meta->size = dump_info->dump_sz;
250 dump_meta->tbl_len = dump_info->tbl_len;
251 memcpy(dump_info->dump_addr + sizeof(*dump_meta), dump_info->tbl_addr, dump_info->tbl_len);
252 /* Offset by crashdump meta and table (copied above) */
253 dump_info->dump_off = dump_info->tbl_len + sizeof(*dump_meta);
254
255 return 0;
256}
257
258static int send_xfer_done(struct qaic_device *qdev, void *resp, u32 dbc_id)
259{
260 struct ssr_debug_transfer_done *xfer_done;
261 int ret;
262
263 xfer_done = kmalloc(sizeof(*xfer_done), GFP_KERNEL);
264 if (!xfer_done) {
265 ret = -ENOMEM;
266 goto out;
267 }
268
269 ret = mhi_queue_buf(qdev->ssr_ch, DMA_FROM_DEVICE, resp, SSR_RESP_MSG_SZ, MHI_EOT);
270 if (ret)
271 goto free_xfer_done;
272
273 xfer_done->hdr.cmd = cpu_to_le32(DEBUG_TRANSFER_DONE);
274 xfer_done->hdr.len = cpu_to_le32(sizeof(*xfer_done));
275 xfer_done->hdr.dbc_id = cpu_to_le32(dbc_id);
276
277 ret = mhi_queue_buf(qdev->ssr_ch, DMA_TO_DEVICE, xfer_done, sizeof(*xfer_done), MHI_EOT);
278 if (ret)
279 goto free_xfer_done;
280
281 return 0;
282
283free_xfer_done:
284 kfree(xfer_done);
285out:
286 return ret;
287}
288
289static int mem_read_req(struct qaic_device *qdev, u64 dest_addr, u64 dest_len)
290{
291 struct ssr_crashdump *ssr_crash = qdev->ssr_mhi_buf;
292 struct ssr_memory_read *read_buf_req;
293 struct ssr_dump_info *dump_info;
294 int ret;
295
296 dump_info = ssr_crash->dump_info;
297 ret = mhi_queue_buf(qdev->ssr_ch, DMA_FROM_DEVICE, ssr_crash->data, SSR_MEM_READ_DATA_SIZE,
298 MHI_EOT);
299 if (ret)
300 goto out;
301
302 read_buf_req = dump_info->read_buf_req;
303 read_buf_req->hdr.cmd = cpu_to_le32(MEMORY_READ);
304 read_buf_req->hdr.len = cpu_to_le32(sizeof(*read_buf_req));
305 read_buf_req->hdr.dbc_id = cpu_to_le32(qdev->ssr_dbc);
306 read_buf_req->addr = cpu_to_le64(dest_addr);
307 read_buf_req->len = cpu_to_le64(dest_len);
308
309 ret = mhi_queue_buf(qdev->ssr_ch, DMA_TO_DEVICE, read_buf_req, sizeof(*read_buf_req),
310 MHI_EOT);
311 if (!ret)
312 dump_info->read_buf_req_queued = true;
313
314out:
315 return ret;
316}
317
318static int ssr_copy_table(struct ssr_dump_info *dump_info, void *data, u64 len)
319{
320 if (len > dump_info->tbl_len - dump_info->tbl_off)
321 return -EINVAL;
322
323 memcpy(dump_info->tbl_addr + dump_info->tbl_off, data, len);
324 dump_info->tbl_off += len;
325
326 /* Entire table has been downloaded, alloc dump memory */
327 if (dump_info->tbl_off == dump_info->tbl_len) {
328 dump_info->tbl_ent = dump_info->tbl_addr;
329 return alloc_dump(dump_info);
330 }
331
332 return 0;
333}
334
335static int ssr_copy_dump(struct ssr_dump_info *dump_info, void *data, u64 len)
336{
337 struct debug_info_table *tbl_ent;
338
339 tbl_ent = dump_info->tbl_ent;
340
341 if (len > tbl_ent->len - dump_info->tbl_ent_off)
342 return -EINVAL;
343
344 memcpy(dump_info->dump_addr + dump_info->dump_off, data, len);
345 dump_info->dump_off += len;
346 dump_info->tbl_ent_off += len;
347
348 /*
349 * Current segment (a entry in table) of the crashdump is complete,
350 * move to next one
351 */
352 if (tbl_ent->len == dump_info->tbl_ent_off) {
353 dump_info->tbl_ent++;
354 dump_info->tbl_ent_off = 0;
355 }
356
357 return 0;
358}
359
360static void ssr_dump_worker(struct work_struct *work)
361{
362 struct ssr_crashdump *ssr_crash = container_of(work, struct ssr_crashdump, work);
363 struct qaic_device *qdev = ssr_crash->qdev;
364 struct ssr_memory_read_rsp *mem_rd_resp;
365 struct debug_info_table *tbl_ent;
366 struct ssr_dump_info *dump_info;
367 u64 dest_addr, dest_len;
368 struct _ssr_hdr *_hdr;
369 struct ssr_hdr hdr;
370 u64 data_len;
371 int ret;
372
373 mem_rd_resp = (struct ssr_memory_read_rsp *)ssr_crash->data;
374 _hdr = &mem_rd_resp->hdr;
375 hdr.cmd = le32_to_cpu(_hdr->cmd);
376 hdr.len = le32_to_cpu(_hdr->len);
377 hdr.dbc_id = le32_to_cpu(_hdr->dbc_id);
378
379 if (hdr.dbc_id != qdev->ssr_dbc)
380 goto reset_device;
381
382 dump_info = ssr_crash->dump_info;
383 if (!dump_info)
384 goto reset_device;
385
386 if (hdr.cmd != MEMORY_READ_RSP)
387 goto free_dump_info;
388
389 if (hdr.len > SSR_MEM_READ_DATA_SIZE)
390 goto free_dump_info;
391
392 data_len = hdr.len - sizeof(*mem_rd_resp);
393
394 if (dump_info->tbl_off < dump_info->tbl_len) /* Chunk belongs to table */
395 ret = ssr_copy_table(dump_info, mem_rd_resp->data, data_len);
396 else /* Chunk belongs to crashdump */
397 ret = ssr_copy_dump(dump_info, mem_rd_resp->data, data_len);
398
399 if (ret)
400 goto free_dump_info;
401
402 if (dump_info->tbl_off < dump_info->tbl_len) {
403 /* Continue downloading table */
404 dest_addr = dump_info->tbl_addr_dev + dump_info->tbl_off;
405 dest_len = min(SSR_MEM_READ_CHUNK_SIZE, dump_info->tbl_len - dump_info->tbl_off);
406 ret = mem_read_req(qdev, dest_addr, dest_len);
407 } else if (dump_info->dump_off < dump_info->dump_sz) {
408 /* Continue downloading crashdump */
409 tbl_ent = dump_info->tbl_ent;
410 dest_addr = tbl_ent->mem_base + dump_info->tbl_ent_off;
411 dest_len = min(SSR_MEM_READ_CHUNK_SIZE, tbl_ent->len - dump_info->tbl_ent_off);
412 ret = mem_read_req(qdev, dest_addr, dest_len);
413 } else {
414 /* Crashdump download complete */
415 ret = send_xfer_done(qdev, dump_info->resp->data, hdr.dbc_id);
416 }
417
418 /* Most likely a MHI xfer has failed */
419 if (ret)
420 goto free_dump_info;
421
422 return;
423
424free_dump_info:
425 /* Free the allocated memory */
426 free_ssr_dump_info(ssr_crash);
427reset_device:
428 /*
429 * After subsystem crashes in device crashdump collection begins but
430 * something went wrong while collecting crashdump, now instead of
431 * handling this error we just reset the device as the best effort has
432 * been made
433 */
434 mhi_soc_reset(qdev->mhi_cntrl);
435}
436
437static struct ssr_dump_info *alloc_dump_info(struct qaic_device *qdev,
438 struct ssr_debug_transfer_info *debug_info)
439{
440 struct ssr_dump_info *dump_info;
441 int ret;
442
443 le64_to_cpus(&debug_info->tbl_len);
444 le64_to_cpus(&debug_info->tbl_addr);
445
446 if (debug_info->tbl_len == 0 ||
447 debug_info->tbl_len % sizeof(struct debug_info_table) != 0) {
448 ret = -EINVAL;
449 goto out;
450 }
451
452 /* Allocate SSR crashdump book keeping structure */
453 dump_info = kzalloc(sizeof(*dump_info), GFP_KERNEL);
454 if (!dump_info) {
455 ret = -ENOMEM;
456 goto out;
457 }
458
459 /* Buffer used to send MEMORY READ request to device via MHI */
460 dump_info->read_buf_req = kzalloc(sizeof(*dump_info->read_buf_req), GFP_KERNEL);
461 if (!dump_info->read_buf_req) {
462 ret = -ENOMEM;
463 goto free_dump_info;
464 }
465
466 /* Crashdump meta table buffer */
467 dump_info->tbl_addr = vzalloc(debug_info->tbl_len);
468 if (!dump_info->tbl_addr) {
469 ret = -ENOMEM;
470 goto free_read_buf_req;
471 }
472
473 dump_info->tbl_addr_dev = debug_info->tbl_addr;
474 dump_info->tbl_len = debug_info->tbl_len;
475
476 return dump_info;
477
478free_read_buf_req:
479 kfree(dump_info->read_buf_req);
480free_dump_info:
481 kfree(dump_info);
482out:
483 return ERR_PTR(ret);
484}
485
486static int dbg_xfer_info_rsp(struct qaic_device *qdev, struct dma_bridge_chan *dbc,
487 struct ssr_debug_transfer_info *debug_info)
488{
489 struct ssr_debug_transfer_info_rsp *debug_rsp;
490 struct ssr_crashdump *ssr_crash = NULL;
491 int ret = 0, ret2;
492
493 debug_rsp = kmalloc(sizeof(*debug_rsp), GFP_KERNEL);
494 if (!debug_rsp)
495 return -ENOMEM;
496
497 if (!qdev->ssr_mhi_buf) {
498 ret = -ENOMEM;
499 goto send_rsp;
500 }
501
502 if (dbc->state != DBC_STATE_BEFORE_POWER_UP) {
503 ret = -EINVAL;
504 goto send_rsp;
505 }
506
507 ssr_crash = qdev->ssr_mhi_buf;
508 ssr_crash->dump_info = alloc_dump_info(qdev, debug_info);
509 if (IS_ERR(ssr_crash->dump_info)) {
510 ret = PTR_ERR(ssr_crash->dump_info);
511 ssr_crash->dump_info = NULL;
512 }
513
514send_rsp:
515 debug_rsp->hdr.cmd = cpu_to_le32(DEBUG_TRANSFER_INFO_RSP);
516 debug_rsp->hdr.len = cpu_to_le32(sizeof(*debug_rsp));
517 debug_rsp->hdr.dbc_id = cpu_to_le32(dbc->id);
518 /*
519 * 0 = Return an ACK confirming the host is ready to download crashdump
520 * 1 = Return an NACK confirming the host is not ready to download crashdump
521 */
522 debug_rsp->ret = cpu_to_le32(ret ? 1 : 0);
523
524 ret2 = mhi_queue_buf(qdev->ssr_ch, DMA_TO_DEVICE, debug_rsp, sizeof(*debug_rsp), MHI_EOT);
525 if (ret2) {
526 free_ssr_dump_info(ssr_crash);
527 kfree(debug_rsp);
528 return ret2;
529 }
530
531 return ret;
532}
533
534static void dbg_xfer_done_rsp(struct qaic_device *qdev, struct dma_bridge_chan *dbc,
535 struct ssr_debug_transfer_done_rsp *xfer_rsp)
536{
537 struct ssr_crashdump *ssr_crash = qdev->ssr_mhi_buf;
538 u32 status = le32_to_cpu(xfer_rsp->ret);
539 struct device *dev = &qdev->pdev->dev;
540 struct ssr_dump_info *dump_info;
541
542 dump_info = ssr_crash->dump_info;
543 if (!dump_info)
544 return;
545
546 if (status) {
547 free_ssr_dump_info(ssr_crash);
548 return;
549 }
550
551 dev_coredumpv(dev, dump_info->dump_addr, dump_info->dump_sz, GFP_KERNEL);
552 /* dev_coredumpv will free dump_info->dump_addr */
553 dump_info->dump_addr = NULL;
554 free_ssr_dump_info(ssr_crash);
555}
556
557static void ssr_worker(struct work_struct *work)
558{
559 struct ssr_resp *resp = container_of(work, struct ssr_resp, work);
560 struct ssr_hdr *hdr = (struct ssr_hdr *)resp->data;
561 struct ssr_dump_info *dump_info = NULL;
562 struct qaic_device *qdev = resp->qdev;
563 struct ssr_crashdump *ssr_crash;
564 struct ssr_event_rsp *event_rsp;
565 struct dma_bridge_chan *dbc;
566 struct ssr_event *event;
567 u32 ssr_event_ack;
568 int ret;
569
570 le32_to_cpus(&hdr->cmd);
571 le32_to_cpus(&hdr->len);
572 le32_to_cpus(&hdr->dbc_id);
573
574 if (hdr->len > SSR_RESP_MSG_SZ)
575 goto out;
576
577 if (hdr->dbc_id >= qdev->num_dbc)
578 goto out;
579
580 dbc = &qdev->dbc[hdr->dbc_id];
581
582 switch (hdr->cmd) {
583 case DEBUG_TRANSFER_INFO:
584 ret = dbg_xfer_info_rsp(qdev, dbc, (struct ssr_debug_transfer_info *)resp->data);
585 if (ret)
586 break;
587
588 ssr_crash = qdev->ssr_mhi_buf;
589 dump_info = ssr_crash->dump_info;
590 dump_info->dbc = dbc;
591 dump_info->resp = resp;
592
593 /* Start by downloading debug table */
594 ret = mem_read_req(qdev, dump_info->tbl_addr_dev,
595 min(dump_info->tbl_len, SSR_MEM_READ_CHUNK_SIZE));
596 if (ret) {
597 free_ssr_dump_info(ssr_crash);
598 break;
599 }
600
601 /*
602 * Till now everything went fine, which means that we will be
603 * collecting crashdump chunk by chunk. Do not queue a response
604 * buffer for SSR cmds till the crashdump is complete.
605 */
606 return;
607 case SSR_EVENT:
608 event = (struct ssr_event *)hdr;
609 le32_to_cpus(&event->event);
610 ssr_event_ack = event->event;
611 ssr_crash = qdev->ssr_mhi_buf;
612
613 switch (event->event) {
614 case BEFORE_SHUTDOWN:
615 set_dbc_state(qdev, hdr->dbc_id, DBC_STATE_BEFORE_SHUTDOWN);
616 qaic_dbc_enter_ssr(qdev, hdr->dbc_id);
617 break;
618 case AFTER_SHUTDOWN:
619 set_dbc_state(qdev, hdr->dbc_id, DBC_STATE_AFTER_SHUTDOWN);
620 break;
621 case BEFORE_POWER_UP:
622 set_dbc_state(qdev, hdr->dbc_id, DBC_STATE_BEFORE_POWER_UP);
623 break;
624 case AFTER_POWER_UP:
625 /*
626 * If dump info is a non NULL value it means that we
627 * have received this SSR event while downloading a
628 * crashdump for this DBC is still in progress. NACK
629 * the SSR event
630 */
631 if (ssr_crash && ssr_crash->dump_info) {
632 free_ssr_dump_info(ssr_crash);
633 ssr_event_ack = SSR_EVENT_NACK;
634 break;
635 }
636
637 set_dbc_state(qdev, hdr->dbc_id, DBC_STATE_AFTER_POWER_UP);
638 break;
639 default:
640 break;
641 }
642
643 event_rsp = kmalloc(sizeof(*event_rsp), GFP_KERNEL);
644 if (!event_rsp)
645 break;
646
647 event_rsp->hdr.cmd = cpu_to_le32(SSR_EVENT_RSP);
648 event_rsp->hdr.len = cpu_to_le32(sizeof(*event_rsp));
649 event_rsp->hdr.dbc_id = cpu_to_le32(hdr->dbc_id);
650 event_rsp->event = cpu_to_le32(ssr_event_ack);
651
652 ret = mhi_queue_buf(qdev->ssr_ch, DMA_TO_DEVICE, event_rsp, sizeof(*event_rsp),
653 MHI_EOT);
654 if (ret)
655 kfree(event_rsp);
656
657 if (event->event == AFTER_POWER_UP && ssr_event_ack != SSR_EVENT_NACK) {
658 qaic_dbc_exit_ssr(qdev);
659 set_dbc_state(qdev, hdr->dbc_id, DBC_STATE_IDLE);
660 }
661
662 break;
663 case DEBUG_TRANSFER_DONE_RSP:
664 dbg_xfer_done_rsp(qdev, dbc, (struct ssr_debug_transfer_done_rsp *)hdr);
665 break;
666 default:
667 break;
668 }
669
670out:
671 ret = mhi_queue_buf(qdev->ssr_ch, DMA_FROM_DEVICE, resp->data, SSR_RESP_MSG_SZ, MHI_EOT);
672 if (ret)
673 kfree(resp);
674}
675
676static int qaic_ssr_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id)
677{
678 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev));
679 struct ssr_resp *resp;
680 int ret;
681
682 ret = mhi_prepare_for_transfer(mhi_dev);
683 if (ret)
684 return ret;
685
686 resp = kzalloc(sizeof(*resp) + SSR_RESP_MSG_SZ, GFP_KERNEL);
687 if (!resp) {
688 mhi_unprepare_from_transfer(mhi_dev);
689 return -ENOMEM;
690 }
691
692 resp->qdev = qdev;
693 INIT_WORK(&resp->work, ssr_worker);
694
695 ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, resp->data, SSR_RESP_MSG_SZ, MHI_EOT);
696 if (ret) {
697 kfree(resp);
698 mhi_unprepare_from_transfer(mhi_dev);
699 return ret;
700 }
701
702 dev_set_drvdata(&mhi_dev->dev, qdev);
703 qdev->ssr_ch = mhi_dev;
704
705 return 0;
706}
707
708static void qaic_ssr_mhi_remove(struct mhi_device *mhi_dev)
709{
710 struct qaic_device *qdev;
711
712 qdev = dev_get_drvdata(&mhi_dev->dev);
713 mhi_unprepare_from_transfer(qdev->ssr_ch);
714 qdev->ssr_ch = NULL;
715}
716
717static void qaic_ssr_mhi_ul_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result)
718{
719 struct qaic_device *qdev = dev_get_drvdata(&mhi_dev->dev);
720 struct ssr_crashdump *ssr_crash = qdev->ssr_mhi_buf;
721 struct _ssr_hdr *hdr = mhi_result->buf_addr;
722 struct ssr_dump_info *dump_info;
723
724 if (mhi_result->transaction_status) {
725 kfree(mhi_result->buf_addr);
726 return;
727 }
728
729 /*
730 * MEMORY READ is used to download crashdump. And crashdump is
731 * downloaded chunk by chunk in a series of MEMORY READ SSR commands.
732 * Hence to avoid too many kmalloc() and kfree() of the same MEMORY READ
733 * request buffer, we allocate only one such buffer and free it only
734 * once.
735 */
736 if (le32_to_cpu(hdr->cmd) == MEMORY_READ) {
737 dump_info = ssr_crash->dump_info;
738 if (dump_info) {
739 dump_info->read_buf_req_queued = false;
740 return;
741 }
742 }
743
744 kfree(mhi_result->buf_addr);
745}
746
747static void qaic_ssr_mhi_dl_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result)
748{
749 struct ssr_resp *resp = container_of(mhi_result->buf_addr, struct ssr_resp, data);
750 struct qaic_device *qdev = dev_get_drvdata(&mhi_dev->dev);
751 struct ssr_crashdump *ssr_crash = qdev->ssr_mhi_buf;
752 bool memory_read_rsp = false;
753
754 if (ssr_crash && ssr_crash->data == mhi_result->buf_addr)
755 memory_read_rsp = true;
756
757 if (mhi_result->transaction_status) {
758 /* Do not free SSR crashdump buffer as it allocated via managed APIs */
759 if (!memory_read_rsp)
760 kfree(resp);
761 return;
762 }
763
764 if (memory_read_rsp)
765 queue_work(qdev->ssr_wq, &ssr_crash->work);
766 else
767 queue_work(qdev->ssr_wq, &resp->work);
768}
769
770static const struct mhi_device_id qaic_ssr_mhi_match_table[] = {
771 { .chan = "QAIC_SSR", },
772 {},
773};
774
775static struct mhi_driver qaic_ssr_mhi_driver = {
776 .id_table = qaic_ssr_mhi_match_table,
777 .remove = qaic_ssr_mhi_remove,
778 .probe = qaic_ssr_mhi_probe,
779 .ul_xfer_cb = qaic_ssr_mhi_ul_xfer_cb,
780 .dl_xfer_cb = qaic_ssr_mhi_dl_xfer_cb,
781 .driver = {
782 .name = "qaic_ssr",
783 },
784};
785
786int qaic_ssr_init(struct qaic_device *qdev, struct drm_device *drm)
787{
788 struct ssr_crashdump *ssr_crash;
789
790 qdev->ssr_dbc = QAIC_SSR_DBC_SENTINEL;
791
792 /*
793 * Device requests only one SSR at a time. So allocating only one
794 * buffer to download crashdump is good enough.
795 */
796 ssr_crash = drmm_kzalloc(drm, SSR_MHI_BUF_SIZE, GFP_KERNEL);
797 if (!ssr_crash)
798 return -ENOMEM;
799
800 ssr_crash->qdev = qdev;
801 INIT_WORK(&ssr_crash->work, ssr_dump_worker);
802 qdev->ssr_mhi_buf = ssr_crash;
803
804 return 0;
805}
806
807int qaic_ssr_register(void)
808{
809 return mhi_driver_register(&qaic_ssr_mhi_driver);
810}
811
812void qaic_ssr_unregister(void)
813{
814 mhi_driver_unregister(&qaic_ssr_mhi_driver);
815}
drivers/accel/qaic/qaic_ssr.h+17−0
@@ -0,0 +1,17 @@
1/* SPDX-License-Identifier: GPL-2.0-only
2 *
3 * Copyright (c) 2020, The Linux Foundation. All rights reserved.
4 * Copyright (c) 2021, 2024 Qualcomm Innovation Center, Inc. All rights reserved.
5 */
6
7#ifndef __QAIC_SSR_H__
8#define __QAIC_SSR_H__
9
10struct drm_device;
11struct qaic_device;
12
13int qaic_ssr_register(void);
14void qaic_ssr_unregister(void);
15void qaic_clean_up_ssr(struct qaic_device *qdev);
16int qaic_ssr_init(struct qaic_device *qdev, struct drm_device *drm);
17#endif /* __QAIC_SSR_H__ */
drivers/accel/qaic/qaic_sysfs.c+109−0
@@ -0,0 +1,109 @@
1// SPDX-License-Identifier: GPL-2.0-only
2
3/* Copyright (c) 2020-2025, The Linux Foundation. All rights reserved. */
4
5#include <drm/drm_file.h>
6#include <drm/drm_managed.h>
7#include <linux/device.h>
8#include <linux/kernel.h>
9#include <linux/kobject.h>
10#include <linux/mutex.h>
11#include <linux/sysfs.h>
12
13#include "qaic.h"
14
15#define NAME_LEN 14
16
17struct dbc_attribute {
18 struct device_attribute dev_attr;
19 u32 dbc_id;
20 char name[NAME_LEN];
21};
22
23static ssize_t dbc_state_show(struct device *dev, struct device_attribute *a, char *buf)
24{
25 struct dbc_attribute *dbc_attr = container_of(a, struct dbc_attribute, dev_attr);
26 struct drm_minor *minor = dev_get_drvdata(dev);
27 struct qaic_device *qdev;
28
29 qdev = to_qaic_device(minor->dev);
30 return sysfs_emit(buf, "%d\n", qdev->dbc[dbc_attr->dbc_id].state);
31}
32
33void set_dbc_state(struct qaic_device *qdev, u32 dbc_id, unsigned int state)
34{
35 struct device *kdev = to_accel_kdev(qdev->qddev);
36 char *envp[3] = {};
37 char state_str[16];
38 char id_str[12];
39
40 envp[0] = id_str;
41 envp[1] = state_str;
42
43 if (state >= DBC_STATE_MAX)
44 return;
45 if (dbc_id >= qdev->num_dbc)
46 return;
47 if (state == qdev->dbc[dbc_id].state)
48 return;
49
50 scnprintf(id_str, ARRAY_SIZE(id_str), "DBC_ID=%d", dbc_id);
51 scnprintf(state_str, ARRAY_SIZE(state_str), "DBC_STATE=%d", state);
52
53 qdev->dbc[dbc_id].state = state;
54 kobject_uevent_env(&kdev->kobj, KOBJ_CHANGE, envp);
55}
56
57int qaic_sysfs_init(struct qaic_drm_device *qddev)
58{
59 struct device *kdev = to_accel_kdev(qddev);
60 struct drm_device *drm = to_drm(qddev);
61 u32 num_dbc = qddev->qdev->num_dbc;
62 struct dbc_attribute *dbc_attrs;
63 int i, ret;
64
65 dbc_attrs = drmm_kcalloc(drm, num_dbc, sizeof(*dbc_attrs), GFP_KERNEL);
66 if (!dbc_attrs)
67 return -ENOMEM;
68
69 for (i = 0; i < num_dbc; ++i) {
70 struct dbc_attribute *dbc_attr = &dbc_attrs[i];
71
72 sysfs_attr_init(&dbc_attr->dev_attr.attr);
73 dbc_attr->dbc_id = i;
74 scnprintf(dbc_attr->name, NAME_LEN, "dbc%d_state", i);
75 dbc_attr->dev_attr.attr.name = dbc_attr->name;
76 dbc_attr->dev_attr.attr.mode = 0444;
77 dbc_attr->dev_attr.show = dbc_state_show;
78 ret = sysfs_create_file(&kdev->kobj, &dbc_attr->dev_attr.attr);
79 if (ret) {
80 int j;
81
82 for (j = 0; j < i; ++j) {
83 dbc_attr = &dbc_attrs[j];
84 sysfs_remove_file(&kdev->kobj, &dbc_attr->dev_attr.attr);
85 }
86 drmm_kfree(drm, dbc_attrs);
87 return ret;
88 }
89 }
90
91 qddev->sysfs_attrs = dbc_attrs;
92 return 0;
93}
94
95void qaic_sysfs_remove(struct qaic_drm_device *qddev)
96{
97 struct dbc_attribute *dbc_attrs = qddev->sysfs_attrs;
98 struct device *kdev = to_accel_kdev(qddev);
99 u32 num_dbc = qddev->qdev->num_dbc;
100 int i;
101
102 if (!dbc_attrs)
103 return;
104
105 qddev->sysfs_attrs = NULL;
106 for (i = 0; i < num_dbc; ++i)
107 sysfs_remove_file(&kdev->kobj, &dbc_attrs[i].dev_attr.attr);
108 drmm_kfree(to_drm(qddev), dbc_attrs);
109}
drivers/accel/qaic/qaic_timesync.c+9−0
@@ -171,6 +171,13 @@ mod_timer: · 170 unmodified lines
171 dev_err(mqtsdev->dev, "%s mod_timer error:%d\n", __func__, ret);171 dev_err(mqtsdev->dev, "%s mod_timer error:%d\n", __func__, ret);
172}172}
173 173
174void qaic_mqts_ch_stop_timer(struct mhi_device *mhi_dev)
175{
176 struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev);
177
178 timer_delete_sync(&mqtsdev->timer);
179}
180
174static int qaic_timesync_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id)181static int qaic_timesync_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id)
175{182{
176 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev));183 struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev));
@@ -206,6 +213,7 @@ static int qaic_timesync_probe(struct mhi_device *mhi_dev, const struct mhi_devi · 29 unmodified lines
206 timer->expires = jiffies + msecs_to_jiffies(timesync_delay_ms);213 timer->expires = jiffies + msecs_to_jiffies(timesync_delay_ms);
207 add_timer(timer);214 add_timer(timer);
208 dev_set_drvdata(&mhi_dev->dev, mqtsdev);215 dev_set_drvdata(&mhi_dev->dev, mqtsdev);
216 qdev->mqts_ch = mhi_dev;
209 217
210 return 0;218 return 0;
211 219
@@ -221,6 +229,7 @@ static void qaic_timesync_remove(struct mhi_device *mhi_dev) · 9 unmodified lines
221{229{
222 struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev);230 struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev);
223 231
232 mqtsdev->qdev->mqts_ch = NULL;
224 timer_delete_sync(&mqtsdev->timer);233 timer_delete_sync(&mqtsdev->timer);
225 mhi_unprepare_from_transfer(mqtsdev->mhi_dev);234 mhi_unprepare_from_transfer(mqtsdev->mhi_dev);
226 kfree(mqtsdev->sync_msg);235 kfree(mqtsdev->sync_msg);
drivers/accel/qaic/qaic_timesync.h+3−0
@@ -6,6 +6,9 @@ · 5 unmodified lines
6#ifndef __QAIC_TIMESYNC_H__6#ifndef __QAIC_TIMESYNC_H__
7#define __QAIC_TIMESYNC_H__7#define __QAIC_TIMESYNC_H__
8 8
9#include <linux/mhi.h>
10
9int qaic_timesync_init(void);11int qaic_timesync_init(void);
10void qaic_timesync_deinit(void);12void qaic_timesync_deinit(void);
13void qaic_mqts_ch_stop_timer(struct mhi_device *mhi_dev);
11#endif /* __QAIC_TIMESYNC_H__ */14#endif /* __QAIC_TIMESYNC_H__ */
drivers/accel/qaic/sahara.c+121−43
@@ -159,6 +159,7 @@ struct sahara_context { · 158 unmodified lines
159 struct sahara_packet *rx;159 struct sahara_packet *rx;
160 struct work_struct fw_work;160 struct work_struct fw_work;
161 struct work_struct dump_work;161 struct work_struct dump_work;
162 struct work_struct read_data_work;
162 struct mhi_device *mhi_dev;163 struct mhi_device *mhi_dev;
163 const char * const *image_table;164 const char * const *image_table;
164 u32 table_size;165 u32 table_size;
@@ -174,7 +175,10 @@ struct sahara_context { · 9 unmodified lines
174 u64 dump_image_offset;175 u64 dump_image_offset;
175 void *mem_dump_freespace;176 void *mem_dump_freespace;
176 u64 dump_images_left;177 u64 dump_images_left;
178 u32 read_data_offset;
179 u32 read_data_length;
177 bool is_mem_dump_mode;180 bool is_mem_dump_mode;
181 bool non_streaming;
178};182};
179 183
180static const char * const aic100_image_table[] = {184static const char * const aic100_image_table[] = {
@@ -194,6 +198,7 @@ static const char * const aic200_image_table[] = { · 13 unmodified lines
194 [23] = "qcom/aic200/aop.mbn",198 [23] = "qcom/aic200/aop.mbn",
195 [32] = "qcom/aic200/tz.mbn",199 [32] = "qcom/aic200/tz.mbn",
196 [33] = "qcom/aic200/hypvm.mbn",200 [33] = "qcom/aic200/hypvm.mbn",
201 [38] = "qcom/aic200/xbl_config.elf",
197 [39] = "qcom/aic200/aic200_abl.elf",202 [39] = "qcom/aic200/aic200_abl.elf",
198 [40] = "qcom/aic200/apdp.mbn",203 [40] = "qcom/aic200/apdp.mbn",
199 [41] = "qcom/aic200/devcfg.mbn",204 [41] = "qcom/aic200/devcfg.mbn",
@@ -202,6 +207,7 @@ static const char * const aic200_image_table[] = { · 2 unmodified lines
202 [49] = "qcom/aic200/shrm.elf",207 [49] = "qcom/aic200/shrm.elf",
203 [50] = "qcom/aic200/cpucp.elf",208 [50] = "qcom/aic200/cpucp.elf",
204 [51] = "qcom/aic200/aop_devcfg.mbn",209 [51] = "qcom/aic200/aop_devcfg.mbn",
210 [54] = "qcom/aic200/qupv3fw.elf",
205 [57] = "qcom/aic200/cpucp_dtbs.elf",211 [57] = "qcom/aic200/cpucp_dtbs.elf",
206 [62] = "qcom/aic200/uefi_dtbs.elf",212 [62] = "qcom/aic200/uefi_dtbs.elf",
207 [63] = "qcom/aic200/xbl_ac_config.mbn",213 [63] = "qcom/aic200/xbl_ac_config.mbn",
@@ -213,9 +219,15 @@ static const char * const aic200_image_table[] = { · 5 unmodified lines
213 [69] = "qcom/aic200/dcd.mbn",219 [69] = "qcom/aic200/dcd.mbn",
214 [73] = "qcom/aic200/gearvm.mbn",220 [73] = "qcom/aic200/gearvm.mbn",
215 [74] = "qcom/aic200/sti.bin",221 [74] = "qcom/aic200/sti.bin",
216 [75] = "qcom/aic200/pvs.bin",222 [76] = "qcom/aic200/tz_qti_config.mbn",
223 [78] = "qcom/aic200/pvs.bin",
217};224};
218 225
226static bool is_streaming(struct sahara_context *context)
227{
228 return !context->non_streaming;
229}
230
219static int sahara_find_image(struct sahara_context *context, u32 image_id)231static int sahara_find_image(struct sahara_context *context, u32 image_id)
220{232{
221 int ret;233 int ret;
@@ -265,6 +277,8 @@ static void sahara_send_reset(struct sahara_context *context) · 43 unmodified lines
265 int ret;277 int ret;
266 278
267 context->is_mem_dump_mode = false;279 context->is_mem_dump_mode = false;
280 context->read_data_offset = 0;
281 context->read_data_length = 0;
268 282
269 context->tx[0]->cmd = cpu_to_le32(SAHARA_RESET_CMD);283 context->tx[0]->cmd = cpu_to_le32(SAHARA_RESET_CMD);
270 context->tx[0]->length = cpu_to_le32(SAHARA_RESET_LENGTH);284 context->tx[0]->length = cpu_to_le32(SAHARA_RESET_LENGTH);
@@ -319,9 +333,39 @@ static void sahara_hello(struct sahara_context *context) · 48 unmodified lines
319 dev_err(&context->mhi_dev->dev, "Unable to send hello response %d\n", ret);333 dev_err(&context->mhi_dev->dev, "Unable to send hello response %d\n", ret);
320}334}
321 335
336static int read_data_helper(struct sahara_context *context, int buf_index)
337{
338 enum mhi_flags mhi_flag;
339 u32 pkt_data_len;
340 int ret;
341
342 pkt_data_len = min(context->read_data_length, SAHARA_PACKET_MAX_SIZE);
343
344 memcpy(context->tx[buf_index],
345 &context->firmware->data[context->read_data_offset],
346 pkt_data_len);
347
348 context->read_data_offset += pkt_data_len;
349 context->read_data_length -= pkt_data_len;
350
351 if (is_streaming(context) || !context->read_data_length)
352 mhi_flag = MHI_EOT;
353 else
354 mhi_flag = MHI_CHAIN;
355
356 ret = mhi_queue_buf(context->mhi_dev, DMA_TO_DEVICE,
357 context->tx[buf_index], pkt_data_len, mhi_flag);
358 if (ret) {
359 dev_err(&context->mhi_dev->dev, "Unable to send read_data response %d\n", ret);
360 return ret;
361 }
362
363 return 0;
364}
365
322static void sahara_read_data(struct sahara_context *context)366static void sahara_read_data(struct sahara_context *context)
323{367{
324 u32 image_id, data_offset, data_len, pkt_data_len;368 u32 image_id, data_offset, data_len;
325 int ret;369 int ret;
326 int i;370 int i;
327 371
@@ -357,7 +401,7 @@ static void sahara_read_data(struct sahara_context *context) · 29 unmodified lines
357 * and is not needed here on error.401 * and is not needed here on error.
358 */402 */
359 403
360 if (data_len > SAHARA_TRANSFER_MAX_SIZE) {404 if (context->non_streaming && data_len > SAHARA_TRANSFER_MAX_SIZE) {
361 dev_err(&context->mhi_dev->dev, "Malformed read_data packet - data len %d exceeds max xfer size %d\n",405 dev_err(&context->mhi_dev->dev, "Malformed read_data packet - data len %d exceeds max xfer size %d\n",
362 data_len, SAHARA_TRANSFER_MAX_SIZE);406 data_len, SAHARA_TRANSFER_MAX_SIZE);
363 sahara_send_reset(context);407 sahara_send_reset(context);
@@ -378,22 +422,18 @@ static void sahara_read_data(struct sahara_context *context) · 14 unmodified lines
378 return;422 return;
379 }423 }
380 424
381 for (i = 0; i < SAHARA_NUM_TX_BUF && data_len; ++i) {425 context->read_data_offset = data_offset;
382 pkt_data_len = min(data_len, SAHARA_PACKET_MAX_SIZE);426 context->read_data_length = data_len;
383
384 memcpy(context->tx[i], &context->firmware->data[data_offset], pkt_data_len);
385 427
386 data_offset += pkt_data_len;428 if (is_streaming(context)) {
387 data_len -= pkt_data_len;429 schedule_work(&context->read_data_work);
430 return;
431 }
388 432
389 ret = mhi_queue_buf(context->mhi_dev, DMA_TO_DEVICE,433 for (i = 0; i < SAHARA_NUM_TX_BUF && context->read_data_length; ++i) {
390 context->tx[i], pkt_data_len,434 ret = read_data_helper(context, i);
391 !data_len ? MHI_EOT : MHI_CHAIN);435 if (ret)
392 if (ret) {436 break;
393 dev_err(&context->mhi_dev->dev, "Unable to send read_data response %d\n",
394 ret);
395 return;
396 }
397 }437 }
398}438}
399 439
@@ -538,6 +578,7 @@ static void sahara_parse_dump_table(struct sahara_context *context) · 138 unmodified lines
538 struct sahara_memory_dump_meta_v1 *dump_meta;578 struct sahara_memory_dump_meta_v1 *dump_meta;
539 u64 table_nents;579 u64 table_nents;
540 u64 dump_length;580 u64 dump_length;
581 u64 mul_bytes;
541 int ret;582 int ret;
542 u64 i;583 u64 i;
543 584
@@ -551,8 +592,9 @@ static void sahara_parse_dump_table(struct sahara_context *context) · 7 unmodified lines
551 dev_table[i].description[SAHARA_TABLE_ENTRY_STR_LEN - 1] = 0;592 dev_table[i].description[SAHARA_TABLE_ENTRY_STR_LEN - 1] = 0;
552 dev_table[i].filename[SAHARA_TABLE_ENTRY_STR_LEN - 1] = 0;593 dev_table[i].filename[SAHARA_TABLE_ENTRY_STR_LEN - 1] = 0;
553 594
554 dump_length = size_add(dump_length, le64_to_cpu(dev_table[i].length));595 if (check_add_overflow(dump_length,
555 if (dump_length == SIZE_MAX) {596 le64_to_cpu(dev_table[i].length),
597 &dump_length)) {
556 /* Discard the dump */598 /* Discard the dump */
557 sahara_send_reset(context);599 sahara_send_reset(context);
558 return;600 return;
@@ -568,14 +610,17 @@ static void sahara_parse_dump_table(struct sahara_context *context) · 9 unmodified lines
568 dev_table[i].filename);610 dev_table[i].filename);
569 }611 }
570 612
571 dump_length = size_add(dump_length, sizeof(*dump_meta));613 if (check_add_overflow(dump_length, (u64)sizeof(*dump_meta), &dump_length)) {
572 if (dump_length == SIZE_MAX) {
573 /* Discard the dump */614 /* Discard the dump */
574 sahara_send_reset(context);615 sahara_send_reset(context);
575 return;616 return;
576 }617 }
577 dump_length = size_add(dump_length, size_mul(sizeof(*image_out_table), table_nents));618 if (check_mul_overflow((u64)sizeof(*image_out_table), table_nents, &mul_bytes)) {
578 if (dump_length == SIZE_MAX) {619 /* Discard the dump */
620 sahara_send_reset(context);
621 return;
622 }
623 if (check_add_overflow(dump_length, mul_bytes, &dump_length)) {
579 /* Discard the dump */624 /* Discard the dump */
580 sahara_send_reset(context);625 sahara_send_reset(context);
581 return;626 return;
@@ -615,7 +660,7 @@ static void sahara_parse_dump_table(struct sahara_context *context) · 33 unmodified lines
615 660
616 /* Request the first chunk of the first image */661 /* Request the first chunk of the first image */
617 context->dump_image = &image_out_table[0];662 context->dump_image = &image_out_table[0];
618 dump_length = min(context->dump_image->length, SAHARA_READ_MAX_SIZE);663 dump_length = min_t(u64, context->dump_image->length, SAHARA_READ_MAX_SIZE);
619 /* Avoid requesting EOI sized data so that we can identify errors */664 /* Avoid requesting EOI sized data so that we can identify errors */
620 if (dump_length == SAHARA_END_OF_IMAGE_LENGTH)665 if (dump_length == SAHARA_END_OF_IMAGE_LENGTH)
621 dump_length = SAHARA_END_OF_IMAGE_LENGTH / 2;666 dump_length = SAHARA_END_OF_IMAGE_LENGTH / 2;
@@ -663,7 +708,7 @@ static void sahara_parse_dump_image(struct sahara_context *context) · 41 unmodified lines
663 708
664 /* Get next image chunk */709 /* Get next image chunk */
665 dump_length = context->dump_image->length - context->dump_image_offset;710 dump_length = context->dump_image->length - context->dump_image_offset;
666 dump_length = min(dump_length, SAHARA_READ_MAX_SIZE);711 dump_length = min_t(u64, dump_length, SAHARA_READ_MAX_SIZE);
667 /* Avoid requesting EOI sized data so that we can identify errors */712 /* Avoid requesting EOI sized data so that we can identify errors */
668 if (dump_length == SAHARA_END_OF_IMAGE_LENGTH)713 if (dump_length == SAHARA_END_OF_IMAGE_LENGTH)
669 dump_length = SAHARA_END_OF_IMAGE_LENGTH / 2;714 dump_length = SAHARA_END_OF_IMAGE_LENGTH / 2;
@@ -742,6 +787,13 @@ error: · 72 unmodified lines
742 sahara_send_reset(context);787 sahara_send_reset(context);
743}788}
744 789
790static void sahara_read_data_processing(struct work_struct *work)
791{
792 struct sahara_context *context = container_of(work, struct sahara_context, read_data_work);
793
794 read_data_helper(context, 0);
795}
796
745static int sahara_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id)797static int sahara_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id)
746{798{
747 struct sahara_context *context;799 struct sahara_context *context;
@@ -756,34 +808,56 @@ static int sahara_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_ · 8 unmodified lines
756 if (!context->rx)808 if (!context->rx)
757 return -ENOMEM;809 return -ENOMEM;
758 810
811 if (!strcmp(mhi_dev->mhi_cntrl->name, "AIC200")) {
812 context->image_table = aic200_image_table;
813 context->table_size = ARRAY_SIZE(aic200_image_table);
814 } else {
815 context->image_table = aic100_image_table;
816 context->table_size = ARRAY_SIZE(aic100_image_table);
817 context->non_streaming = true;
818 }
819
759 /*820 /*
760 * AIC100 defines SAHARA_TRANSFER_MAX_SIZE as the largest value it821 * There are two firmware implementations for READ_DATA handling.
761 * will request for READ_DATA. This is larger than822 * The older "SBL" implementation defines a Sahara transfer size, and
762 * SAHARA_PACKET_MAX_SIZE, and we need 9x SAHARA_PACKET_MAX_SIZE to823 * expects that the response is a single transport transfer. If the
763 * cover SAHARA_TRANSFER_MAX_SIZE. When the remote side issues a824 * FW wants to transfer a file that is larger than the transfer size,
764 * READ_DATA, it requires a transfer of the exact size requested. We825 * the FW will issue multiple READ_DATA commands. For this
765 * can use MHI_CHAIN to link multiple buffers into a single transfer826 * implementation, we need to allocate enough buffers to contain the
766 * but the remote side will not consume the buffers until it sees an827 * entire Sahara transfer size.
767 * EOT, thus we need to allocate enough buffers to put in the tx fifo828 *
768 * to cover an entire READ_DATA request of the max size.829 * The newer "XBL" implementation does not define a maximum transfer
830 * size and instead expects the data to be streamed over using the
831 * transport level MTU. The FW will issue a single READ_DATA command
832 * of whatever size, and consume multiple transport level transfers
833 * until the expected amount of data is consumed. For this
834 * implementation we only need a single buffer of the transport MTU
835 * but we'll need to be able to use it multiple times for a single
836 * READ_DATA request.
837 *
838 * AIC100 is the SBL implementation and defines SAHARA_TRANSFER_MAX_SIZE
839 * and we need 9x SAHARA_PACKET_MAX_SIZE to cover that. We can use
840 * MHI_CHAIN to link multiple buffers into a single transfer but the
841 * remote side will not consume the buffers until it sees an EOT, thus
842 * we need to allocate enough buffers to put in the tx fifo to cover an
843 * entire READ_DATA request of the max size.
844 *
845 * AIC200 is the XBL implementation, and so a single buffer will work.
769 */846 */
770 for (i = 0; i < SAHARA_NUM_TX_BUF; ++i) {847 for (i = 0; i < SAHARA_NUM_TX_BUF; ++i) {
771 context->tx[i] = devm_kzalloc(&mhi_dev->dev, SAHARA_PACKET_MAX_SIZE, GFP_KERNEL);848 context->tx[i] = devm_kzalloc(&mhi_dev->dev,
849 SAHARA_PACKET_MAX_SIZE,
850 GFP_KERNEL);
772 if (!context->tx[i])851 if (!context->tx[i])
773 return -ENOMEM;852 return -ENOMEM;
853 if (is_streaming(context))
854 break;
774 }855 }
775 856
776 context->mhi_dev = mhi_dev;857 context->mhi_dev = mhi_dev;
777 INIT_WORK(&context->fw_work, sahara_processing);858 INIT_WORK(&context->fw_work, sahara_processing);
778 INIT_WORK(&context->dump_work, sahara_dump_processing);859 INIT_WORK(&context->dump_work, sahara_dump_processing);
779 860 INIT_WORK(&context->read_data_work, sahara_read_data_processing);
780 if (!strcmp(mhi_dev->mhi_cntrl->name, "AIC200")) {
781 context->image_table = aic200_image_table;
782 context->table_size = ARRAY_SIZE(aic200_image_table);
783 } else {
784 context->image_table = aic100_image_table;
785 context->table_size = ARRAY_SIZE(aic100_image_table);
786 }
787 861
788 context->active_image_id = SAHARA_IMAGE_ID_NONE;862 context->active_image_id = SAHARA_IMAGE_ID_NONE;
789 dev_set_drvdata(&mhi_dev->dev, context);863 dev_set_drvdata(&mhi_dev->dev, context);
@@ -814,6 +888,10 @@ static void sahara_mhi_remove(struct mhi_device *mhi_dev) · 24 unmodified lines
814 888
815static void sahara_mhi_ul_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result)889static void sahara_mhi_ul_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result)
816{890{
891 struct sahara_context *context = dev_get_drvdata(&mhi_dev->dev);
892
893 if (!mhi_result->transaction_status && context->read_data_length && is_streaming(context))
894 schedule_work(&context->read_data_work);
817}895}
818 896
819static void sahara_mhi_dl_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result)897static void sahara_mhi_dl_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result)
drivers/accel/rocket/rocket_gem.c+1−0
@@ -2,6 +2,7 @@ · 1 unmodified line
2/* Copyright 2024-2025 Tomeu Vizoso <tomeu@tomeuvizoso.net> */2/* Copyright 2024-2025 Tomeu Vizoso <tomeu@tomeuvizoso.net> */
3 3
4#include <drm/drm_device.h>4#include <drm/drm_device.h>
5#include <drm/drm_print.h>
5#include <drm/drm_utils.h>6#include <drm/drm_utils.h>
6#include <drm/rocket_accel.h>7#include <drm/rocket_accel.h>
7#include <linux/dma-mapping.h>8#include <linux/dma-mapping.h>
drivers/dma-buf/dma-fence.c+26−26
@@ -121,29 +121,27 @@ static const struct dma_fence_ops dma_fence_stub_ops = { · 120 unmodified lines
121 .get_timeline_name = dma_fence_stub_get_name,121 .get_timeline_name = dma_fence_stub_get_name,
122};122};
123 123
124static int __init dma_fence_init_stub(void)
125{
126 dma_fence_init(&dma_fence_stub, &dma_fence_stub_ops,
127 &dma_fence_stub_lock, 0, 0);
128
129 set_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
130 &dma_fence_stub.flags);
131
132 dma_fence_signal(&dma_fence_stub);
133 return 0;
134}
135subsys_initcall(dma_fence_init_stub);
136
124/**137/**
125 * dma_fence_get_stub - return a signaled fence138 * dma_fence_get_stub - return a signaled fence
126 *139 *
127 * Return a stub fence which is already signaled. The fence's140 * Return a stub fence which is already signaled. The fence's timestamp
128 * timestamp corresponds to the first time after boot this141 * corresponds to the initialisation time of the linux kernel.
129 * function is called.
130 */142 */
131struct dma_fence *dma_fence_get_stub(void)143struct dma_fence *dma_fence_get_stub(void)
132{144{
133 spin_lock(&dma_fence_stub_lock);
134 if (!dma_fence_stub.ops) {
135 dma_fence_init(&dma_fence_stub,
136 &dma_fence_stub_ops,
137 &dma_fence_stub_lock,
138 0, 0);
139
140 set_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
141 &dma_fence_stub.flags);
142
143 dma_fence_signal_locked(&dma_fence_stub);
144 }
145 spin_unlock(&dma_fence_stub_lock);
146
147 return dma_fence_get(&dma_fence_stub);145 return dma_fence_get(&dma_fence_stub);
148}146}
149EXPORT_SYMBOL(dma_fence_get_stub);147EXPORT_SYMBOL(dma_fence_get_stub);
@@ -999,19 +997,21 @@ EXPORT_SYMBOL(dma_fence_set_deadline); · 849 unmodified lines
999 */997 */
1000void dma_fence_describe(struct dma_fence *fence, struct seq_file *seq)998void dma_fence_describe(struct dma_fence *fence, struct seq_file *seq)
1001{999{
1002 const char __rcu *timeline;1000 const char __rcu *timeline = "";
1003 const char __rcu *driver;1001 const char __rcu *driver = "";
1002 const char *signaled = "";
1004 1003
1005 rcu_read_lock();1004 rcu_read_lock();
1006 1005
1007 timeline = dma_fence_timeline_name(fence);1006 if (!dma_fence_is_signaled(fence)) {
1008 driver = dma_fence_driver_name(fence);1007 timeline = dma_fence_timeline_name(fence);
1008 driver = dma_fence_driver_name(fence);
1009 signaled = "un";
1010 }
1009 1011
1010 seq_printf(seq, "%s %s seq %llu %ssignalled\n",1012 seq_printf(seq, "%llu:%llu %s %s %ssignalled\n",
1011 rcu_dereference(driver),1013 fence->context, fence->seqno, timeline, driver,
1012 rcu_dereference(timeline),1014 signaled);
1013 fence->seqno,
1014 dma_fence_is_signaled(fence) ? "" : "un");
1015 1015
1016 rcu_read_unlock();1016 rcu_read_unlock();
1017}1017}
drivers/dma-buf/heaps/Kconfig+0−10
@@ -12,13 +12,3 @@ config DMABUF_HEAPS_CMA · 11 unmodified lines
12 Choose this option to enable dma-buf CMA heap. This heap is backed12 Choose this option to enable dma-buf CMA heap. This heap is backed
13 by the Contiguous Memory Allocator (CMA). If your system has these13 by the Contiguous Memory Allocator (CMA). If your system has these
14 regions, you should say Y here.14 regions, you should say Y here.
15
16config DMABUF_HEAPS_CMA_LEGACY
17 bool "Legacy DMA-BUF CMA Heap"
18 default y
19 depends on DMABUF_HEAPS_CMA
20 help
21 Add a duplicate CMA-backed dma-buf heap with legacy naming derived
22 from the CMA area's devicetree node, or "reserved" if the area is not
23 defined in the devicetree. This uses the same underlying allocator as
24 CONFIG_DMABUF_HEAPS_CMA.
drivers/dma-buf/heaps/cma_heap.c+30−17
@@ -14,6 +14,7 @@ · 13 unmodified lines
14 14
15#include <linux/cma.h>15#include <linux/cma.h>
16#include <linux/dma-buf.h>16#include <linux/dma-buf.h>
17#include <linux/dma-buf/heaps/cma.h>
17#include <linux/dma-heap.h>18#include <linux/dma-heap.h>
18#include <linux/dma-map-ops.h>19#include <linux/dma-map-ops.h>
19#include <linux/err.h>20#include <linux/err.h>
@@ -21,12 +22,27 @@ · 1 unmodified line
21#include <linux/io.h>22#include <linux/io.h>
22#include <linux/mm.h>23#include <linux/mm.h>
23#include <linux/module.h>24#include <linux/module.h>
25#include <linux/of.h>
26#include <linux/of_reserved_mem.h>
24#include <linux/scatterlist.h>27#include <linux/scatterlist.h>
25#include <linux/slab.h>28#include <linux/slab.h>
26#include <linux/vmalloc.h>29#include <linux/vmalloc.h>
27 30
28#define DEFAULT_CMA_NAME "default_cma_region"31#define DEFAULT_CMA_NAME "default_cma_region"
29 32
33static struct cma *dma_areas[MAX_CMA_AREAS] __initdata;
34static unsigned int dma_areas_num __initdata;
35
36int __init dma_heap_cma_register_heap(struct cma *cma)
37{
38 if (dma_areas_num >= ARRAY_SIZE(dma_areas))
39 return -EINVAL;
40
41 dma_areas[dma_areas_num++] = cma;
42
43 return 0;
44}
45
30struct cma_heap {46struct cma_heap {
31 struct dma_heap *heap;47 struct dma_heap *heap;
32 struct cma *cma;48 struct cma *cma;
@@ -395,33 +411,30 @@ static int __init __add_cma_heap(struct cma *cma, const char *name) · 362 unmodified lines
395 return 0;411 return 0;
396}412}
397 413
398static int __init add_default_cma_heap(void)414static int __init add_cma_heaps(void)
399{415{
400 struct cma *default_cma = dev_get_cma_area(NULL);416 struct cma *default_cma = dev_get_cma_area(NULL);
401 const char *legacy_cma_name;417 unsigned int i;
402 int ret;418 int ret;
403 419
404 if (!default_cma)420 if (default_cma) {
405 return 0;421 ret = __add_cma_heap(default_cma, DEFAULT_CMA_NAME);
422 if (ret)
423 return ret;
424 }
406 425
407 ret = __add_cma_heap(default_cma, DEFAULT_CMA_NAME);426 for (i = 0; i < dma_areas_num; i++) {
408 if (ret)427 struct cma *cma = dma_areas[i];
409 return ret;
410 428
411 if (IS_ENABLED(CONFIG_DMABUF_HEAPS_CMA_LEGACY)) {429 ret = __add_cma_heap(cma, cma_get_name(cma));
412 legacy_cma_name = cma_get_name(default_cma);430 if (ret) {
413 if (!strcmp(legacy_cma_name, DEFAULT_CMA_NAME)) {431 pr_warn("Failed to add CMA heap %s", cma_get_name(cma));
414 pr_warn("legacy name and default name are the same, skipping legacy heap\n");432 continue;
415 return 0;
416 }433 }
417 434
418 ret = __add_cma_heap(default_cma, legacy_cma_name);
419 if (ret)
420 pr_warn("failed to add legacy heap: %pe\n",
421 ERR_PTR(ret));
422 }435 }
423 436
424 return 0;437 return 0;
425}438}
426module_init(add_default_cma_heap);439module_init(add_cma_heaps);
427MODULE_DESCRIPTION("DMA-BUF CMA Heap");440MODULE_DESCRIPTION("DMA-BUF CMA Heap");
drivers/dma-buf/heaps/system_heap.c+24−9
@@ -186,20 +186,35 @@ static int system_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) · 185 unmodified lines
186 struct system_heap_buffer *buffer = dmabuf->priv;186 struct system_heap_buffer *buffer = dmabuf->priv;
187 struct sg_table *table = &buffer->sg_table;187 struct sg_table *table = &buffer->sg_table;
188 unsigned long addr = vma->vm_start;188 unsigned long addr = vma->vm_start;
189 struct sg_page_iter piter;189 unsigned long pgoff = vma->vm_pgoff;
190 int ret;190 struct scatterlist *sg;
191 int i, ret;
192
193 for_each_sgtable_sg(table, sg, i) {
194 unsigned long n = sg->length >> PAGE_SHIFT;
191 195
192 for_each_sgtable_page(table, &piter, vma->vm_pgoff) {196 if (pgoff < n)
193 struct page *page = sg_page_iter_page(&piter);197 break;
198 pgoff -= n;
199 }
200
201 for (; sg && addr < vma->vm_end; sg = sg_next(sg)) {
202 unsigned long n = (sg->length >> PAGE_SHIFT) - pgoff;
203 struct page *page = sg_page(sg) + pgoff;
204 unsigned long size = n << PAGE_SHIFT;
205
206 if (addr + size > vma->vm_end)
207 size = vma->vm_end - addr;
194 208
195 ret = remap_pfn_range(vma, addr, page_to_pfn(page), PAGE_SIZE,209 ret = remap_pfn_range(vma, addr, page_to_pfn(page),
196 vma->vm_page_prot);210 size, vma->vm_page_prot);
197 if (ret)211 if (ret)
198 return ret;212 return ret;
199 addr += PAGE_SIZE;213
200 if (addr >= vma->vm_end)214 addr += size;
201 return 0;215 pgoff = 0;
202 }216 }
217
203 return 0;218 return 0;
204}219}
205 220
drivers/dma-buf/sw_sync.c+4−0
@@ -8,6 +8,7 @@ · 7 unmodified lines
8#include <linux/file.h>8#include <linux/file.h>
9#include <linux/fs.h>9#include <linux/fs.h>
10#include <linux/uaccess.h>10#include <linux/uaccess.h>
11#include <linux/panic.h>
11#include <linux/slab.h>12#include <linux/slab.h>
12#include <linux/sync_file.h>13#include <linux/sync_file.h>
13 14
@@ -349,6 +350,9 @@ static long sw_sync_ioctl_create_fence(struct sync_timeline *obj, · 335 unmodified lines
349 struct sync_file *sync_file;350 struct sync_file *sync_file;
350 struct sw_sync_create_fence_data data;351 struct sw_sync_create_fence_data data;
351 352
353 /* SW sync fence are inherently unsafe and can deadlock the kernel */
354 add_taint(TAINT_SOFTLOCKUP, LOCKDEP_STILL_OK);
355
352 if (fd < 0)356 if (fd < 0)
353 return fd;357 return fd;
354 358
drivers/gpu/drm/Makefile+5−2
@@ -41,6 +41,7 @@ drm-y := \ · 40 unmodified lines
41 drm_bridge.o \41 drm_bridge.o \
42 drm_cache.o \42 drm_cache.o \
43 drm_color_mgmt.o \43 drm_color_mgmt.o \
44 drm_colorop.o \
44 drm_connector.o \45 drm_connector.o \
45 drm_crtc.o \46 drm_crtc.o \
46 drm_displayid.o \47 drm_displayid.o \
@@ -76,7 +77,8 @@ drm-y := \ · 29 unmodified lines
76drm-$(CONFIG_DRM_CLIENT) += \77drm-$(CONFIG_DRM_CLIENT) += \
77 drm_client.o \78 drm_client.o \
78 drm_client_event.o \79 drm_client_event.o \
79 drm_client_modeset.o80 drm_client_modeset.o \
81 drm_client_sysrq.o
80drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o82drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o
81drm-$(CONFIG_COMPAT) += drm_ioc32.o83drm-$(CONFIG_COMPAT) += drm_ioc32.o
82drm-$(CONFIG_DRM_PANEL) += drm_panel.o84drm-$(CONFIG_DRM_PANEL) += drm_panel.o
@@ -150,7 +152,8 @@ drm_kms_helper-y := \ · 67 unmodified lines
150 drm_plane_helper.o \152 drm_plane_helper.o \
151 drm_probe_helper.o \153 drm_probe_helper.o \
152 drm_self_refresh_helper.o \154 drm_self_refresh_helper.o \
153 drm_simple_kms_helper.o155 drm_simple_kms_helper.o \
156 drm_vblank_helper.o
154drm_kms_helper-$(CONFIG_DRM_PANEL_BRIDGE) += bridge/panel.o157drm_kms_helper-$(CONFIG_DRM_PANEL_BRIDGE) += bridge/panel.o
155drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fb_helper.o158drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fb_helper.o
156obj-$(CONFIG_DRM_KMS_HELPER) += drm_kms_helper.o159obj-$(CONFIG_DRM_KMS_HELPER) += drm_kms_helper.o
drivers/gpu/drm/adp/adp_drv.c+1−0
@@ -16,6 +16,7 @@ · 15 unmodified lines
16#include <drm/drm_gem_dma_helper.h>16#include <drm/drm_gem_dma_helper.h>
17#include <drm/drm_gem_framebuffer_helper.h>17#include <drm/drm_gem_framebuffer_helper.h>
18#include <drm/drm_of.h>18#include <drm/drm_of.h>
19#include <drm/drm_print.h>
19#include <drm/drm_probe_helper.h>20#include <drm/drm_probe_helper.h>
20#include <drm/drm_vblank.h>21#include <drm/drm_vblank.h>
21 22
drivers/gpu/drm/amd/amdgpu/Kconfig+16−8
@@ -43,14 +43,16 @@ config DRM_AMDGPU_SI · 42 unmodified lines
43 bool "Enable amdgpu support for SI parts"43 bool "Enable amdgpu support for SI parts"
44 depends on DRM_AMDGPU44 depends on DRM_AMDGPU
45 help45 help
46 Choose this option if you want to enable experimental support46 Choose this option if you want to enable support
47 for SI (Southern Islands) asics.47 for SI (Southern Islands) asics.
48 48
49 SI is already supported in radeon. Experimental support for SI49 SI (Southern Islands) are first generation GCN GPUs,
50 in amdgpu will be disabled by default and is still provided by50 supported by both drivers: radeon (old) and amdgpu (new).
51 radeon. Use module options to override this:51 By default, SI dedicated GPUs are supported by amdgpu.
52 52
53 radeon.si_support=0 amdgpu.si_support=153 Use module options to override this:
54 To use radeon for SI,
55 radeon.si_support=1 amdgpu.si_support=0
54 56
55config DRM_AMDGPU_CIK57config DRM_AMDGPU_CIK
56 bool "Enable amdgpu support for CIK parts"58 bool "Enable amdgpu support for CIK parts"
@@ -59,11 +61,17 @@ config DRM_AMDGPU_CIK · 2 unmodified lines
59 Choose this option if you want to enable support for CIK (Sea61 Choose this option if you want to enable support for CIK (Sea
60 Islands) asics.62 Islands) asics.
61 63
62 CIK is already supported in radeon. Support for CIK in amdgpu64 CIK (Sea Islands) are second generation GCN GPUs,
63 will be disabled by default and is still provided by radeon.65 supported by both drivers: radeon (old) and amdgpu (new).
64 Use module options to override this:66 By default,
67 CIK dedicated GPUs are supported by amdgpu
68 CIK APUs are supported by radeon
65 69
70 Use module options to override this:
71 To use amdgpu for CIK,
66 radeon.cik_support=0 amdgpu.cik_support=172 radeon.cik_support=0 amdgpu.cik_support=1
73 To use radeon for CIK,
74 radeon.cik_support=1 amdgpu.cik_support=0
67 75
68config DRM_AMDGPU_USERPTR76config DRM_AMDGPU_USERPTR
69 bool "Always enable userptr write support"77 bool "Always enable userptr write support"
drivers/gpu/drm/amd/amdgpu/Makefile+8−2
@@ -37,7 +37,8 @@ ccflags-y := -I$(FULL_AMD_PATH)/include/asic_reg \ · 36 unmodified lines
37 -I$(FULL_AMD_DISPLAY_PATH)/modules/inc \37 -I$(FULL_AMD_DISPLAY_PATH)/modules/inc \
38 -I$(FULL_AMD_DISPLAY_PATH)/dc \38 -I$(FULL_AMD_DISPLAY_PATH)/dc \
39 -I$(FULL_AMD_DISPLAY_PATH)/amdgpu_dm \39 -I$(FULL_AMD_DISPLAY_PATH)/amdgpu_dm \
40 -I$(FULL_AMD_PATH)/amdkfd40 -I$(FULL_AMD_PATH)/amdkfd \
41 -I$(FULL_AMD_PATH)/ras/ras_mgr
41 42
42# Locally disable W=1 warnings enabled in drm subsystem Makefile43# Locally disable W=1 warnings enabled in drm subsystem Makefile
43subdir-ccflags-y += -Wno-override-init44subdir-ccflags-y += -Wno-override-init
@@ -77,7 +78,7 @@ amdgpu-$(CONFIG_DRM_AMDGPU_CIK)+= cik.o cik_ih.o \ · 33 unmodified lines
77 dce_v8_0.o gfx_v7_0.o cik_sdma.o uvd_v4_2.o vce_v2_0.o78 dce_v8_0.o gfx_v7_0.o cik_sdma.o uvd_v4_2.o vce_v2_0.o
78 79
79amdgpu-$(CONFIG_DRM_AMDGPU_SI)+= si.o gmc_v6_0.o gfx_v6_0.o si_ih.o si_dma.o dce_v6_0.o \80amdgpu-$(CONFIG_DRM_AMDGPU_SI)+= si.o gmc_v6_0.o gfx_v6_0.o si_ih.o si_dma.o dce_v6_0.o \
80 uvd_v3_1.o81 uvd_v3_1.o vce_v1_0.o
81 82
82amdgpu-y += \83amdgpu-y += \
83 vi.o mxgpu_vi.o nbio_v6_1.o soc15.o emu_soc.o mxgpu_ai.o nbio_v7_0.o vega10_reg_init.o \84 vi.o mxgpu_vi.o nbio_v6_1.o soc15.o emu_soc.o mxgpu_ai.o nbio_v7_0.o vega10_reg_init.o \
@@ -324,4 +325,9 @@ amdgpu-y += \ · 240 unmodified lines
324 isp_v4_1_1.o325 isp_v4_1_1.o
325endif326endif
326 327
328AMD_GPU_RAS_PATH := ../ras
329AMD_GPU_RAS_FULL_PATH := $(FULL_AMD_PATH)/ras
330include $(AMD_GPU_RAS_FULL_PATH)/Makefile
331amdgpu-y += $(AMD_GPU_RAS_FILES)
332
327obj-$(CONFIG_DRM_AMDGPU)+= amdgpu.o333obj-$(CONFIG_DRM_AMDGPU)+= amdgpu.o
drivers/gpu/drm/amd/amdgpu/aldebaran.c+4−0
@@ -88,6 +88,10 @@ static int aldebaran_mode2_suspend_ip(struct amdgpu_device *adev) · 87 unmodified lines
88 uint32_t ip_block;88 uint32_t ip_block;
89 int r, i;89 int r, i;
90 90
91 /* Skip suspend of SDMA IP versions >= 4.4.2. They are multi-aid */
92 if (adev->aid_mask)
93 ip_block_mask &= ~BIT(AMD_IP_BLOCK_TYPE_SDMA);
94
91 amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);95 amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
92 amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);96 amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
93 97
drivers/gpu/drm/amd/amdgpu/amdgpu.h+14−15
@@ -372,13 +372,15 @@ void amdgpu_device_ip_get_clockgating_state(struct amdgpu_device *adev, · 371 unmodified lines
372 u64 *flags);372 u64 *flags);
373int amdgpu_device_ip_wait_for_idle(struct amdgpu_device *adev,373int amdgpu_device_ip_wait_for_idle(struct amdgpu_device *adev,
374 enum amd_ip_block_type block_type);374 enum amd_ip_block_type block_type);
375bool amdgpu_device_ip_is_hw(struct amdgpu_device *adev,
376 enum amd_ip_block_type block_type);
375bool amdgpu_device_ip_is_valid(struct amdgpu_device *adev,377bool amdgpu_device_ip_is_valid(struct amdgpu_device *adev,
376 enum amd_ip_block_type block_type);378 enum amd_ip_block_type block_type);
377int amdgpu_ip_block_suspend(struct amdgpu_ip_block *ip_block);379int amdgpu_ip_block_suspend(struct amdgpu_ip_block *ip_block);
378 380
379int amdgpu_ip_block_resume(struct amdgpu_ip_block *ip_block);381int amdgpu_ip_block_resume(struct amdgpu_ip_block *ip_block);
380 382
381#define AMDGPU_MAX_IP_NUM 16383#define AMDGPU_MAX_IP_NUM AMD_IP_BLOCK_TYPE_NUM
382 384
383struct amdgpu_ip_block_status {385struct amdgpu_ip_block_status {
384 bool valid;386 bool valid;
@@ -839,8 +841,6 @@ struct amd_powerplay { · 454 unmodified lines
839 const struct amd_pm_funcs *pp_funcs;841 const struct amd_pm_funcs *pp_funcs;
840};842};
841 843
842struct ip_discovery_top;
843
844/* polaris10 kickers */844/* polaris10 kickers */
845#define ASICID_IS_P20(did, rid) (((did == 0x67DF) && \845#define ASICID_IS_P20(did, rid) (((did == 0x67DF) && \
846 ((rid == 0xE3) || \846 ((rid == 0xE3) || \
@@ -972,8 +972,7 @@ struct amdgpu_device { · 125 unmodified lines
972 struct notifier_block acpi_nb;972 struct notifier_block acpi_nb;
973 struct notifier_block pm_nb;973 struct notifier_block pm_nb;
974 struct amdgpu_i2c_chan *i2c_bus[AMDGPU_MAX_I2C_BUS];974 struct amdgpu_i2c_chan *i2c_bus[AMDGPU_MAX_I2C_BUS];
975 struct debugfs_blob_wrapper debugfs_vbios_blob;975 struct debugfs_blob_wrapper debugfs_vbios_blob;
976 struct debugfs_blob_wrapper debugfs_discovery_blob;
977 struct mutex srbm_mutex;976 struct mutex srbm_mutex;
978 /* GRBM index mutex. Protects concurrent access to GRBM index */977 /* GRBM index mutex. Protects concurrent access to GRBM index */
979 struct mutex grbm_idx_mutex;978 struct mutex grbm_idx_mutex;
@@ -1063,6 +1062,9 @@ struct amdgpu_device { · 83 unmodified lines
1063 u32 log2_max_MBps;1062 u32 log2_max_MBps;
1064 } mm_stats;1063 } mm_stats;
1065 1064
1065 /* discovery*/
1066 struct amdgpu_discovery_info discovery;
1067
1066 /* display */1068 /* display */
1067 bool enable_virtual_display;1069 bool enable_virtual_display;
1068 struct amdgpu_vkms_output *amdgpu_vkms_output;1070 struct amdgpu_vkms_output *amdgpu_vkms_output;
@@ -1174,6 +1176,12 @@ struct amdgpu_device { · 105 unmodified lines
1174 * queue fence.1176 * queue fence.
1175 */1177 */
1176 struct xarray userq_xa;1178 struct xarray userq_xa;
1179 /**
1180 * @userq_doorbell_xa: Global user queue map (doorbell index → queue)
1181 * Key: doorbell_index (unique global identifier for the queue)
1182 * Value: struct amdgpu_usermode_queue
1183 */
1184 struct xarray userq_doorbell_xa;
1177 1185
1178 /* df */1186 /* df */
1179 struct amdgpu_df df;1187 struct amdgpu_df df;
@@ -1265,8 +1273,6 @@ struct amdgpu_device { · 85 unmodified lines
1265 1273
1266 struct list_head ras_list;1274 struct list_head ras_list;
1267 1275
1268 struct ip_discovery_top *ip_top;
1269
1270 struct amdgpu_reset_domain *reset_domain;1276 struct amdgpu_reset_domain *reset_domain;
1271 1277
1272 struct mutex benchmark_mutex;1278 struct mutex benchmark_mutex;
@@ -1309,9 +1315,8 @@ struct amdgpu_device { · 36 unmodified lines
1309 */1315 */
1310 bool apu_prefer_gtt;1316 bool apu_prefer_gtt;
1311 1317
1312 struct list_head userq_mgr_list;
1313 struct mutex userq_mutex;
1314 bool userq_halt_for_enforce_isolation;1318 bool userq_halt_for_enforce_isolation;
1319 struct work_struct userq_reset_work;
1315 struct amdgpu_uid *uid_info;1320 struct amdgpu_uid *uid_info;
1316 1321
1317 /* KFD1322 /* KFD
@@ -1535,11 +1540,6 @@ int emu_soc_asic_init(struct amdgpu_device *adev); · 217 unmodified lines
1535#define amdgpu_asic_read_bios_from_rom(adev, b, l) (adev)->asic_funcs->read_bios_from_rom((adev), (b), (l))1540#define amdgpu_asic_read_bios_from_rom(adev, b, l) (adev)->asic_funcs->read_bios_from_rom((adev), (b), (l))
1536#define amdgpu_asic_read_register(adev, se, sh, offset, v)((adev)->asic_funcs->read_register((adev), (se), (sh), (offset), (v)))1541#define amdgpu_asic_read_register(adev, se, sh, offset, v)((adev)->asic_funcs->read_register((adev), (se), (sh), (offset), (v)))
1537#define amdgpu_asic_get_config_memsize(adev) (adev)->asic_funcs->get_config_memsize((adev))1542#define amdgpu_asic_get_config_memsize(adev) (adev)->asic_funcs->get_config_memsize((adev))
1538#define amdgpu_asic_flush_hdp(adev, r) \
1539 ((adev)->asic_funcs->flush_hdp ? (adev)->asic_funcs->flush_hdp((adev), (r)) : (adev)->hdp.funcs->flush_hdp((adev), (r)))
1540#define amdgpu_asic_invalidate_hdp(adev, r) \
1541 ((adev)->asic_funcs->invalidate_hdp ? (adev)->asic_funcs->invalidate_hdp((adev), (r)) : \
1542 ((adev)->hdp.funcs->invalidate_hdp ? (adev)->hdp.funcs->invalidate_hdp((adev), (r)) : (void)0))
1543#define amdgpu_asic_need_full_reset(adev) (adev)->asic_funcs->need_full_reset((adev))1543#define amdgpu_asic_need_full_reset(adev) (adev)->asic_funcs->need_full_reset((adev))
1544#define amdgpu_asic_init_doorbell_index(adev) (adev)->asic_funcs->init_doorbell_index((adev))1544#define amdgpu_asic_init_doorbell_index(adev) (adev)->asic_funcs->init_doorbell_index((adev))
1545#define amdgpu_asic_get_pcie_usage(adev, cnt0, cnt1) ((adev)->asic_funcs->get_pcie_usage((adev), (cnt0), (cnt1)))1545#define amdgpu_asic_get_pcie_usage(adev, cnt0, cnt1) ((adev)->asic_funcs->get_pcie_usage((adev), (cnt0), (cnt1)))
@@ -1638,7 +1638,6 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, · 92 unmodified lines
1638 struct drm_file *file_priv);1638 struct drm_file *file_priv);
1639void amdgpu_driver_release_kms(struct drm_device *dev);1639void amdgpu_driver_release_kms(struct drm_device *dev);
1640 1640
1641int amdgpu_device_ip_suspend(struct amdgpu_device *adev);
1642int amdgpu_device_prepare(struct drm_device *dev);1641int amdgpu_device_prepare(struct drm_device *dev);
1643void amdgpu_device_complete(struct drm_device *dev);1642void amdgpu_device_complete(struct drm_device *dev);
1644int amdgpu_device_suspend(struct drm_device *dev, bool fbcon);1643int amdgpu_device_suspend(struct drm_device *dev, bool fbcon);
drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c+8−2
@@ -302,17 +302,19 @@ static int acp_hw_init(struct amdgpu_ip_block *ip_block) · 301 unmodified lines
302 adev->acp.acp_res[2].end = adev->acp.acp_res[2].start;302 adev->acp.acp_res[2].end = adev->acp.acp_res[2].start;
303 303
304 adev->acp.acp_cell[0].name = "acp_audio_dma";304 adev->acp.acp_cell[0].name = "acp_audio_dma";
305 adev->acp.acp_cell[0].id = 0;
305 adev->acp.acp_cell[0].num_resources = 3;306 adev->acp.acp_cell[0].num_resources = 3;
306 adev->acp.acp_cell[0].resources = &adev->acp.acp_res[0];307 adev->acp.acp_cell[0].resources = &adev->acp.acp_res[0];
307 adev->acp.acp_cell[0].platform_data = &adev->asic_type;308 adev->acp.acp_cell[0].platform_data = &adev->asic_type;
308 adev->acp.acp_cell[0].pdata_size = sizeof(adev->asic_type);309 adev->acp.acp_cell[0].pdata_size = sizeof(adev->asic_type);
309 310
310 adev->acp.acp_cell[1].name = "designware-i2s";311 adev->acp.acp_cell[1].name = "designware-i2s";
312 adev->acp.acp_cell[1].id = 1;
311 adev->acp.acp_cell[1].num_resources = 1;313 adev->acp.acp_cell[1].num_resources = 1;
312 adev->acp.acp_cell[1].resources = &adev->acp.acp_res[1];314 adev->acp.acp_cell[1].resources = &adev->acp.acp_res[1];
313 adev->acp.acp_cell[1].platform_data = &i2s_pdata[0];315 adev->acp.acp_cell[1].platform_data = &i2s_pdata[0];
314 adev->acp.acp_cell[1].pdata_size = sizeof(struct i2s_platform_data);316 adev->acp.acp_cell[1].pdata_size = sizeof(struct i2s_platform_data);
315 r = mfd_add_hotplug_devices(adev->acp.parent, adev->acp.acp_cell, 2);317 r = mfd_add_devices(adev->acp.parent, 0, adev->acp.acp_cell, 2, NULL, 0, NULL);
316 if (r)318 if (r)
317 goto failure;319 goto failure;
318 r = device_for_each_child(adev->acp.parent, &adev->acp.acp_genpd->gpd,320 r = device_for_each_child(adev->acp.parent, &adev->acp.acp_genpd->gpd,
@@ -410,30 +412,34 @@ static int acp_hw_init(struct amdgpu_ip_block *ip_block) · 91 unmodified lines
410 adev->acp.acp_res[4].end = adev->acp.acp_res[4].start;412 adev->acp.acp_res[4].end = adev->acp.acp_res[4].start;
411 413
412 adev->acp.acp_cell[0].name = "acp_audio_dma";414 adev->acp.acp_cell[0].name = "acp_audio_dma";
415 adev->acp.acp_cell[0].id = 0;
413 adev->acp.acp_cell[0].num_resources = 5;416 adev->acp.acp_cell[0].num_resources = 5;
414 adev->acp.acp_cell[0].resources = &adev->acp.acp_res[0];417 adev->acp.acp_cell[0].resources = &adev->acp.acp_res[0];
415 adev->acp.acp_cell[0].platform_data = &adev->asic_type;418 adev->acp.acp_cell[0].platform_data = &adev->asic_type;
416 adev->acp.acp_cell[0].pdata_size = sizeof(adev->asic_type);419 adev->acp.acp_cell[0].pdata_size = sizeof(adev->asic_type);
417 420
418 adev->acp.acp_cell[1].name = "designware-i2s";421 adev->acp.acp_cell[1].name = "designware-i2s";
422 adev->acp.acp_cell[1].id = 1;
419 adev->acp.acp_cell[1].num_resources = 1;423 adev->acp.acp_cell[1].num_resources = 1;
420 adev->acp.acp_cell[1].resources = &adev->acp.acp_res[1];424 adev->acp.acp_cell[1].resources = &adev->acp.acp_res[1];
421 adev->acp.acp_cell[1].platform_data = &i2s_pdata[0];425 adev->acp.acp_cell[1].platform_data = &i2s_pdata[0];
422 adev->acp.acp_cell[1].pdata_size = sizeof(struct i2s_platform_data);426 adev->acp.acp_cell[1].pdata_size = sizeof(struct i2s_platform_data);
423 427
424 adev->acp.acp_cell[2].name = "designware-i2s";428 adev->acp.acp_cell[2].name = "designware-i2s";
429 adev->acp.acp_cell[2].id = 2;
425 adev->acp.acp_cell[2].num_resources = 1;430 adev->acp.acp_cell[2].num_resources = 1;
426 adev->acp.acp_cell[2].resources = &adev->acp.acp_res[2];431 adev->acp.acp_cell[2].resources = &adev->acp.acp_res[2];
427 adev->acp.acp_cell[2].platform_data = &i2s_pdata[1];432 adev->acp.acp_cell[2].platform_data = &i2s_pdata[1];
428 adev->acp.acp_cell[2].pdata_size = sizeof(struct i2s_platform_data);433 adev->acp.acp_cell[2].pdata_size = sizeof(struct i2s_platform_data);
429 434
430 adev->acp.acp_cell[3].name = "designware-i2s";435 adev->acp.acp_cell[3].name = "designware-i2s";
436 adev->acp.acp_cell[3].id = 3;
431 adev->acp.acp_cell[3].num_resources = 1;437 adev->acp.acp_cell[3].num_resources = 1;
432 adev->acp.acp_cell[3].resources = &adev->acp.acp_res[3];438 adev->acp.acp_cell[3].resources = &adev->acp.acp_res[3];
433 adev->acp.acp_cell[3].platform_data = &i2s_pdata[2];439 adev->acp.acp_cell[3].platform_data = &i2s_pdata[2];
434 adev->acp.acp_cell[3].pdata_size = sizeof(struct i2s_platform_data);440 adev->acp.acp_cell[3].pdata_size = sizeof(struct i2s_platform_data);
435 441
436 r = mfd_add_hotplug_devices(adev->acp.parent, adev->acp.acp_cell, ACP_DEVS);442 r = mfd_add_devices(adev->acp.parent, 0, adev->acp.acp_cell, ACP_DEVS, NULL, 0, NULL);
437 if (r)443 if (r)
438 goto failure;444 goto failure;
439 445
drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c+0−1
@@ -507,7 +507,6 @@ static int amdgpu_atif_handler(struct amdgpu_device *adev, · 506 unmodified lines
507 pm_runtime_get_sync(adev_to_drm(adev)->dev);507 pm_runtime_get_sync(adev_to_drm(adev)->dev);
508 /* Just fire off a uevent and let userspace tell us what to do */508 /* Just fire off a uevent and let userspace tell us what to do */
509 drm_helper_hpd_irq_event(adev_to_drm(adev));509 drm_helper_hpd_irq_event(adev_to_drm(adev));
510 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
511 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);510 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
512 }511 }
513 }512 }
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h+1−1
@@ -71,7 +71,7 @@ struct kgd_mem { · 70 unmodified lines
71 struct mutex lock;71 struct mutex lock;
72 struct amdgpu_bo *bo;72 struct amdgpu_bo *bo;
73 struct dma_buf *dmabuf;73 struct dma_buf *dmabuf;
74 struct hmm_range *range;74 struct amdgpu_hmm_range *range;
75 struct list_head attachments;75 struct list_head attachments;
76 /* protected by amdkfd_process_info.lock */76 /* protected by amdkfd_process_info.lock */
77 struct list_head validate_list;77 struct list_head validate_list;
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c+21−11
@@ -1057,7 +1057,7 @@ static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr, · 1056 unmodified lines
1057 struct amdkfd_process_info *process_info = mem->process_info;1057 struct amdkfd_process_info *process_info = mem->process_info;
1058 struct amdgpu_bo *bo = mem->bo;1058 struct amdgpu_bo *bo = mem->bo;
1059 struct ttm_operation_ctx ctx = { true, false };1059 struct ttm_operation_ctx ctx = { true, false };
1060 struct hmm_range *range;1060 struct amdgpu_hmm_range *range;
1061 int ret = 0;1061 int ret = 0;
1062 1062
1063 mutex_lock(&process_info->lock);1063 mutex_lock(&process_info->lock);
@@ -1089,8 +1089,15 @@ static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr, · 25 unmodified lines
1089 return 0;1089 return 0;
1090 }1090 }
1091 1091
1092 ret = amdgpu_ttm_tt_get_user_pages(bo, &range);1092 range = amdgpu_hmm_range_alloc(NULL);
1093 if (unlikely(!range)) {
1094 ret = -ENOMEM;
1095 goto unregister_out;
1096 }
1097
1098 ret = amdgpu_ttm_tt_get_user_pages(bo, range);
1093 if (ret) {1099 if (ret) {
1100 amdgpu_hmm_range_free(range);
1094 if (ret == -EAGAIN)1101 if (ret == -EAGAIN)
1095 pr_debug("Failed to get user pages, try again\n");1102 pr_debug("Failed to get user pages, try again\n");
1096 else1103 else
@@ -1113,7 +1120,7 @@ static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr, · 16 unmodified lines
1113 amdgpu_bo_unreserve(bo);1120 amdgpu_bo_unreserve(bo);
1114 1121
1115release_out:1122release_out:
1116 amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm, range);1123 amdgpu_hmm_range_free(range);
1117unregister_out:1124unregister_out:
1118 if (ret)1125 if (ret)
1119 amdgpu_hmm_unregister(bo);1126 amdgpu_hmm_unregister(bo);
@@ -1920,7 +1927,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( · 800 unmodified lines
1920 if (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm)) {1927 if (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm)) {
1921 amdgpu_hmm_unregister(mem->bo);1928 amdgpu_hmm_unregister(mem->bo);
1922 mutex_lock(&process_info->notifier_lock);1929 mutex_lock(&process_info->notifier_lock);
1923 amdgpu_ttm_tt_discard_user_pages(mem->bo->tbo.ttm, mem->range);1930 amdgpu_hmm_range_free(mem->range);
1924 mutex_unlock(&process_info->notifier_lock);1931 mutex_unlock(&process_info->notifier_lock);
1925 }1932 }
1926 1933
@@ -1958,9 +1965,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( · 31 unmodified lines
1958 */1965 */
1959 if (size) {1966 if (size) {
1960 if (!is_imported &&1967 if (!is_imported &&
1961 (mem->bo->preferred_domains == AMDGPU_GEM_DOMAIN_VRAM ||1968 mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM)
1962 (adev->apu_prefer_gtt &&
1963 mem->bo->preferred_domains == AMDGPU_GEM_DOMAIN_GTT)))
1964 *size = bo_size;1969 *size = bo_size;
1965 else1970 else
1966 *size = 0;1971 *size = 0;
@@ -2546,7 +2551,7 @@ static int update_invalid_user_pages(struct amdkfd_process_info *process_info, · 579 unmodified lines
2546 2551
2547 bo = mem->bo;2552 bo = mem->bo;
2548 2553
2549 amdgpu_ttm_tt_discard_user_pages(bo->tbo.ttm, mem->range);2554 amdgpu_hmm_range_free(mem->range);
2550 mem->range = NULL;2555 mem->range = NULL;
2551 2556
2552 /* BO reservations and getting user pages (hmm_range_fault)2557 /* BO reservations and getting user pages (hmm_range_fault)
@@ -2570,9 +2575,14 @@ static int update_invalid_user_pages(struct amdkfd_process_info *process_info, · 17 unmodified lines
2570 }2575 }
2571 }2576 }
2572 2577
2578 mem->range = amdgpu_hmm_range_alloc(NULL);
2579 if (unlikely(!mem->range))
2580 return -ENOMEM;
2573 /* Get updated user pages */2581 /* Get updated user pages */
2574 ret = amdgpu_ttm_tt_get_user_pages(bo, &mem->range);2582 ret = amdgpu_ttm_tt_get_user_pages(bo, mem->range);
2575 if (ret) {2583 if (ret) {
2584 amdgpu_hmm_range_free(mem->range);
2585 mem->range = NULL;
2576 pr_debug("Failed %d to get user pages\n", ret);2586 pr_debug("Failed %d to get user pages\n", ret);
2577 2587
2578 /* Return -EFAULT bad address error as success. It will2588 /* Return -EFAULT bad address error as success. It will
@@ -2745,8 +2755,8 @@ static int confirm_valid_user_pages_locked(struct amdkfd_process_info *process_i · 166 unmodified lines
2745 continue;2755 continue;
2746 2756
2747 /* Only check mem with hmm range associated */2757 /* Only check mem with hmm range associated */
2748 valid = amdgpu_ttm_tt_get_user_pages_done(2758 valid = amdgpu_hmm_range_valid(mem->range);
2749 mem->bo->tbo.ttm, mem->range);2759 amdgpu_hmm_range_free(mem->range);
2750 2760
2751 mem->range = NULL;2761 mem->range = NULL;
2752 if (!valid) {2762 if (!valid) {
drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c+16−13
@@ -181,19 +181,22 @@ int amdgpu_atomfirmware_allocate_fb_scratch(struct amdgpu_device *adev) · 180 unmodified lines
181 u8 frev, crev;181 u8 frev, crev;
182 int usage_bytes = 0;182 int usage_bytes = 0;
183 183
184 if (amdgpu_atom_parse_data_header(ctx, index, NULL, &frev, &crev, &data_offset)) {184 /* Skip atomfirmware allocation for SRIOV VFs when dynamic crit regn is enabled */
185 if (frev == 2 && crev == 1) {185 if (!(amdgpu_sriov_vf(adev) && adev->virt.is_dynamic_crit_regn_enabled)) {
186 fw_usage_v2_1 =186 if (amdgpu_atom_parse_data_header(ctx, index, NULL, &frev, &crev, &data_offset)) {
187 (struct vram_usagebyfirmware_v2_1 *)(ctx->bios + data_offset);187 if (frev == 2 && crev == 1) {
188 amdgpu_atomfirmware_allocate_fb_v2_1(adev,188 fw_usage_v2_1 =
189 fw_usage_v2_1,189 (struct vram_usagebyfirmware_v2_1 *)(ctx->bios + data_offset);
190 &usage_bytes);190 amdgpu_atomfirmware_allocate_fb_v2_1(adev,
191 } else if (frev >= 2 && crev >= 2) {191 fw_usage_v2_1,
192 fw_usage_v2_2 =192 &usage_bytes);
193 (struct vram_usagebyfirmware_v2_2 *)(ctx->bios + data_offset);193 } else if (frev >= 2 && crev >= 2) {
194 amdgpu_atomfirmware_allocate_fb_v2_2(adev,194 fw_usage_v2_2 =
195 fw_usage_v2_2,195 (struct vram_usagebyfirmware_v2_2 *)(ctx->bios + data_offset);
196 &usage_bytes);196 amdgpu_atomfirmware_allocate_fb_v2_2(adev,
197 fw_usage_v2_2,
198 &usage_bytes);
199 }
197 }200 }
198 }201 }
199 202
drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c+26−10
@@ -96,13 +96,14 @@ void amdgpu_bios_release(struct amdgpu_device *adev) · 95 unmodified lines
96 * part of the system bios. On boot, the system bios puts a96 * part of the system bios. On boot, the system bios puts a
97 * copy of the igp rom at the start of vram if a discrete card is97 * copy of the igp rom at the start of vram if a discrete card is
98 * present.98 * present.
99 * For SR-IOV, the vbios image is also put in VRAM in the VF.99 * For SR-IOV, if dynamic critical region is not enabled,
100 * the vbios image is also put at the start of VRAM in the VF.
100 */101 */
101static bool amdgpu_read_bios_from_vram(struct amdgpu_device *adev)102static bool amdgpu_read_bios_from_vram(struct amdgpu_device *adev)
102{103{
103 uint8_t __iomem *bios;104 uint8_t __iomem *bios = NULL;
104 resource_size_t vram_base;105 resource_size_t vram_base;
105 resource_size_t size = 256 * 1024; /* ??? */106 u32 size = 256U * 1024U; /* ??? */
106 107
107 if (!(adev->flags & AMD_IS_APU))108 if (!(adev->flags & AMD_IS_APU))
108 if (amdgpu_device_need_post(adev))109 if (amdgpu_device_need_post(adev))
@@ -114,18 +115,33 @@ static bool amdgpu_read_bios_from_vram(struct amdgpu_device *adev) · 5 unmodified lines
114 115
115 adev->bios = NULL;116 adev->bios = NULL;
116 vram_base = pci_resource_start(adev->pdev, 0);117 vram_base = pci_resource_start(adev->pdev, 0);
117 bios = ioremap_wc(vram_base, size);
118 if (!bios)
119 return false;
120 118
121 adev->bios = kmalloc(size, GFP_KERNEL);119 adev->bios = kmalloc(size, GFP_KERNEL);
122 if (!adev->bios) {120 if (!adev->bios)
123 iounmap(bios);
124 return false;121 return false;
122
123 /* For SRIOV with dynamic critical region is enabled,
124 * the vbios image is put at a dynamic offset of VRAM in the VF.
125 * If dynamic critical region is disabled, follow the existing logic as on baremetal.
126 */
127 if (amdgpu_sriov_vf(adev) && adev->virt.is_dynamic_crit_regn_enabled) {
128 if (amdgpu_virt_get_dynamic_data_info(adev,
129 AMD_SRIOV_MSG_VBIOS_IMG_TABLE_ID, adev->bios, &size)) {
130 amdgpu_bios_release(adev);
131 return false;
132 }
133 } else {
134 bios = ioremap_wc(vram_base, size);
135 if (!bios) {
136 amdgpu_bios_release(adev);
137 return false;
138 }
139
140 memcpy_fromio(adev->bios, bios, size);
141 iounmap(bios);
125 }142 }
143
126 adev->bios_size = size;144 adev->bios_size = size;
127 memcpy_fromio(adev->bios, bios, size);
128 iounmap(bios);
129 145
130 if (!check_atom_bios(adev, size)) {146 if (!check_atom_bios(adev, size)) {
131 amdgpu_bios_release(adev);147 amdgpu_bios_release(adev);
drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h+1−1
@@ -38,7 +38,7 @@ struct amdgpu_bo_list_entry { · 37 unmodified lines
38 struct amdgpu_bo *bo;38 struct amdgpu_bo *bo;
39 struct amdgpu_bo_va *bo_va;39 struct amdgpu_bo_va *bo_va;
40 uint32_t priority;40 uint32_t priority;
41 struct hmm_range *range;41 struct amdgpu_hmm_range *range;
42 bool user_invalidated;42 bool user_invalidated;
43};43};
44 44
drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c+4−12
@@ -734,10 +734,8 @@ amdgpu_connector_lvds_detect(struct drm_connector *connector, bool force) · 733 unmodified lines
734 734
735 amdgpu_connector_update_scratch_regs(connector, ret);735 amdgpu_connector_update_scratch_regs(connector, ret);
736 736
737 if (!drm_kms_helper_is_poll_worker()) {737 if (!drm_kms_helper_is_poll_worker())
738 pm_runtime_mark_last_busy(connector->dev->dev);
739 pm_runtime_put_autosuspend(connector->dev->dev);738 pm_runtime_put_autosuspend(connector->dev->dev);
740 }
741 739
742 return ret;740 return ret;
743}741}
@@ -919,10 +917,8 @@ amdgpu_connector_vga_detect(struct drm_connector *connector, bool force) · 175 unmodified lines
919 amdgpu_connector_update_scratch_regs(connector, ret);917 amdgpu_connector_update_scratch_regs(connector, ret);
920 918
921out:919out:
922 if (!drm_kms_helper_is_poll_worker()) {920 if (!drm_kms_helper_is_poll_worker())
923 pm_runtime_mark_last_busy(connector->dev->dev);
924 pm_runtime_put_autosuspend(connector->dev->dev);921 pm_runtime_put_autosuspend(connector->dev->dev);
925 }
926 922
927 return ret;923 return ret;
928}924}
@@ -1146,10 +1142,8 @@ out: · 217 unmodified lines
1146 amdgpu_connector_update_scratch_regs(connector, ret);1142 amdgpu_connector_update_scratch_regs(connector, ret);
1147 1143
1148exit:1144exit:
1149 if (!drm_kms_helper_is_poll_worker()) {1145 if (!drm_kms_helper_is_poll_worker())
1150 pm_runtime_mark_last_busy(connector->dev->dev);
1151 pm_runtime_put_autosuspend(connector->dev->dev);1146 pm_runtime_put_autosuspend(connector->dev->dev);
1152 }
1153 1147
1154 return ret;1148 return ret;
1155}1149}
@@ -1486,10 +1480,8 @@ amdgpu_connector_dp_detect(struct drm_connector *connector, bool force) · 330 unmodified lines
1486 1480
1487 amdgpu_connector_update_scratch_regs(connector, ret);1481 amdgpu_connector_update_scratch_regs(connector, ret);
1488out:1482out:
1489 if (!drm_kms_helper_is_poll_worker()) {1483 if (!drm_kms_helper_is_poll_worker())
1490 pm_runtime_mark_last_busy(connector->dev->dev);
1491 pm_runtime_put_autosuspend(connector->dev->dev);1484 pm_runtime_put_autosuspend(connector->dev->dev);
1492 }
1493 1485
1494 if (connector->connector_type == DRM_MODE_CONNECTOR_DisplayPort ||1486 if (connector->connector_type == DRM_MODE_CONNECTOR_DisplayPort ||
1495 connector->connector_type == DRM_MODE_CONNECTOR_eDP)1487 connector->connector_type == DRM_MODE_CONNECTOR_eDP)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c+11−8
@@ -29,7 +29,6 @@ · 28 unmodified lines
29#include <linux/pagemap.h>29#include <linux/pagemap.h>
30#include <linux/sync_file.h>30#include <linux/sync_file.h>
31#include <linux/dma-buf.h>31#include <linux/dma-buf.h>
32#include <linux/hmm.h>
33 32
34#include <drm/amdgpu_drm.h>33#include <drm/amdgpu_drm.h>
35#include <drm/drm_syncobj.h>34#include <drm/drm_syncobj.h>
@@ -41,6 +40,7 @@ · 5 unmodified lines
41#include "amdgpu_gmc.h"40#include "amdgpu_gmc.h"
42#include "amdgpu_gem.h"41#include "amdgpu_gem.h"
43#include "amdgpu_ras.h"42#include "amdgpu_ras.h"
43#include "amdgpu_hmm.h"
44 44
45static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p,45static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p,
46 struct amdgpu_device *adev,46 struct amdgpu_device *adev,
@@ -891,12 +891,17 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, · 844 unmodified lines
891 bool userpage_invalidated = false;891 bool userpage_invalidated = false;
892 struct amdgpu_bo *bo = e->bo;892 struct amdgpu_bo *bo = e->bo;
893 893
894 r = amdgpu_ttm_tt_get_user_pages(bo, &e->range);894 e->range = amdgpu_hmm_range_alloc(NULL);
895 if (unlikely(!e->range))
896 return -ENOMEM;
897
898 r = amdgpu_ttm_tt_get_user_pages(bo, e->range);
895 if (r)899 if (r)
896 goto out_free_user_pages;900 goto out_free_user_pages;
897 901
898 for (i = 0; i < bo->tbo.ttm->num_pages; i++) {902 for (i = 0; i < bo->tbo.ttm->num_pages; i++) {
899 if (bo->tbo.ttm->pages[i] != hmm_pfn_to_page(e->range->hmm_pfns[i])) {903 if (bo->tbo.ttm->pages[i] !=
904 hmm_pfn_to_page(e->range->hmm_range.hmm_pfns[i])) {
900 userpage_invalidated = true;905 userpage_invalidated = true;
901 break;906 break;
902 }907 }
@@ -990,9 +995,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, · 87 unmodified lines
990 995
991out_free_user_pages:996out_free_user_pages:
992 amdgpu_bo_list_for_each_userptr_entry(e, p->bo_list) {997 amdgpu_bo_list_for_each_userptr_entry(e, p->bo_list) {
993 struct amdgpu_bo *bo = e->bo;998 amdgpu_hmm_range_free(e->range);
994
995 amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm, e->range);
996 e->range = NULL;999 e->range = NULL;
997 }1000 }
998 mutex_unlock(&p->bo_list->bo_list_mutex);1001 mutex_unlock(&p->bo_list->bo_list_mutex);
@@ -1323,8 +1326,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, · 324 unmodified lines
1323 */1326 */
1324 r = 0;1327 r = 0;
1325 amdgpu_bo_list_for_each_userptr_entry(e, p->bo_list) {1328 amdgpu_bo_list_for_each_userptr_entry(e, p->bo_list) {
1326 r |= !amdgpu_ttm_tt_get_user_pages_done(e->bo->tbo.ttm,1329 r |= !amdgpu_hmm_range_valid(e->range);
1327 e->range);1330 amdgpu_hmm_range_free(e->range);
1328 e->range = NULL;1331 e->range = NULL;
1329 }1332 }
1330 if (r) {1333 if (r) {
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c+4−30
@@ -129,7 +129,6 @@ static int amdgpu_debugfs_process_reg_op(bool read, struct file *f, · 128 unmodified lines
129 if (use_bank) {129 if (use_bank) {
130 if ((sh_bank != 0xFFFFFFFF && sh_bank >= adev->gfx.config.max_sh_per_se) ||130 if ((sh_bank != 0xFFFFFFFF && sh_bank >= adev->gfx.config.max_sh_per_se) ||
131 (se_bank != 0xFFFFFFFF && se_bank >= adev->gfx.config.max_shader_engines)) {131 (se_bank != 0xFFFFFFFF && se_bank >= adev->gfx.config.max_shader_engines)) {
132 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
133 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);132 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
134 amdgpu_virt_disable_access_debugfs(adev);133 amdgpu_virt_disable_access_debugfs(adev);
135 return -EINVAL;134 return -EINVAL;
@@ -179,7 +178,6 @@ end: · 43 unmodified lines
179 if (pm_pg_lock)178 if (pm_pg_lock)
180 mutex_unlock(&adev->pm.mutex);179 mutex_unlock(&adev->pm.mutex);
181 180
182 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
183 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);181 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
184 182
185 amdgpu_virt_disable_access_debugfs(adev);183 amdgpu_virt_disable_access_debugfs(adev);
@@ -255,7 +253,6 @@ static ssize_t amdgpu_debugfs_regs2_op(struct file *f, char __user *buf, u32 off · 69 unmodified lines
255 if (rd->id.use_grbm) {253 if (rd->id.use_grbm) {
256 if ((rd->id.grbm.sh != 0xFFFFFFFF && rd->id.grbm.sh >= adev->gfx.config.max_sh_per_se) ||254 if ((rd->id.grbm.sh != 0xFFFFFFFF && rd->id.grbm.sh >= adev->gfx.config.max_sh_per_se) ||
257 (rd->id.grbm.se != 0xFFFFFFFF && rd->id.grbm.se >= adev->gfx.config.max_shader_engines)) {255 (rd->id.grbm.se != 0xFFFFFFFF && rd->id.grbm.se >= adev->gfx.config.max_shader_engines)) {
258 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
259 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);256 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
260 amdgpu_virt_disable_access_debugfs(adev);257 amdgpu_virt_disable_access_debugfs(adev);
261 mutex_unlock(&rd->lock);258 mutex_unlock(&rd->lock);
@@ -310,7 +307,6 @@ end: · 48 unmodified lines
310 307
311 mutex_unlock(&rd->lock);308 mutex_unlock(&rd->lock);
312 309
313 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
314 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);310 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
315 311
316 amdgpu_virt_disable_access_debugfs(adev);312 amdgpu_virt_disable_access_debugfs(adev);
@@ -446,7 +442,6 @@ static ssize_t amdgpu_debugfs_gprwave_read(struct file *f, char __user *buf, siz · 129 unmodified lines
446 amdgpu_gfx_select_se_sh(adev, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, rd->id.xcc_id);442 amdgpu_gfx_select_se_sh(adev, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, rd->id.xcc_id);
447 mutex_unlock(&adev->grbm_idx_mutex);443 mutex_unlock(&adev->grbm_idx_mutex);
448 444
449 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
450 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);445 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
451 446
452 if (!x) {447 if (!x) {
@@ -557,7 +552,6 @@ static ssize_t amdgpu_debugfs_regs_pcie_read(struct file *f, char __user *buf, · 104 unmodified lines
557 552
558 r = result;553 r = result;
559out:554out:
560 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
561 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);555 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
562 amdgpu_virt_disable_access_debugfs(adev);556 amdgpu_virt_disable_access_debugfs(adev);
563 return r;557 return r;
@@ -617,7 +611,6 @@ static ssize_t amdgpu_debugfs_regs_pcie_write(struct file *f, const char __user · 53 unmodified lines
617 611
618 r = result;612 r = result;
619out:613out:
620 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
621 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);614 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
622 amdgpu_virt_disable_access_debugfs(adev);615 amdgpu_virt_disable_access_debugfs(adev);
623 return r;616 return r;
@@ -676,7 +669,6 @@ static ssize_t amdgpu_debugfs_regs_didt_read(struct file *f, char __user *buf, · 52 unmodified lines
676 669
677 r = result;670 r = result;
678out:671out:
679 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
680 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);672 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
681 amdgpu_virt_disable_access_debugfs(adev);673 amdgpu_virt_disable_access_debugfs(adev);
682 return r;674 return r;
@@ -736,7 +728,6 @@ static ssize_t amdgpu_debugfs_regs_didt_write(struct file *f, const char __user · 53 unmodified lines
736 728
737 r = result;729 r = result;
738out:730out:
739 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
740 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);731 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
741 amdgpu_virt_disable_access_debugfs(adev);732 amdgpu_virt_disable_access_debugfs(adev);
742 return r;733 return r;
@@ -795,7 +786,6 @@ static ssize_t amdgpu_debugfs_regs_smc_read(struct file *f, char __user *buf, · 52 unmodified lines
795 786
796 r = result;787 r = result;
797out:788out:
798 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
799 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);789 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
800 amdgpu_virt_disable_access_debugfs(adev);790 amdgpu_virt_disable_access_debugfs(adev);
801 return r;791 return r;
@@ -855,7 +845,6 @@ static ssize_t amdgpu_debugfs_regs_smc_write(struct file *f, const char __user * · 53 unmodified lines
855 845
856 r = result;846 r = result;
857out:847out:
858 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
859 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);848 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
860 amdgpu_virt_disable_access_debugfs(adev);849 amdgpu_virt_disable_access_debugfs(adev);
861 return r;850 return r;
@@ -1003,7 +992,6 @@ static ssize_t amdgpu_debugfs_sensor_read(struct file *f, char __user *buf, · 141 unmodified lines
1003 992
1004 r = amdgpu_dpm_read_sensor(adev, idx, &values[0], &valuesize);993 r = amdgpu_dpm_read_sensor(adev, idx, &values[0], &valuesize);
1005 994
1006 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1007 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);995 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1008 996
1009 if (r) {997 if (r) {
@@ -1094,7 +1082,6 @@ static ssize_t amdgpu_debugfs_wave_read(struct file *f, char __user *buf, · 84 unmodified lines
1094 amdgpu_gfx_select_se_sh(adev, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0);1082 amdgpu_gfx_select_se_sh(adev, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0);
1095 mutex_unlock(&adev->grbm_idx_mutex);1083 mutex_unlock(&adev->grbm_idx_mutex);
1096 1084
1097 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1098 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1085 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1099 1086
1100 if (!x) {1087 if (!x) {
@@ -1192,7 +1179,6 @@ static ssize_t amdgpu_debugfs_gpr_read(struct file *f, char __user *buf, · 91 unmodified lines
1192 amdgpu_gfx_select_se_sh(adev, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0);1179 amdgpu_gfx_select_se_sh(adev, 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0);
1193 mutex_unlock(&adev->grbm_idx_mutex);1180 mutex_unlock(&adev->grbm_idx_mutex);
1194 1181
1195 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1196 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1182 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1197 1183
1198 while (size) {1184 while (size) {
@@ -1266,7 +1252,6 @@ static ssize_t amdgpu_debugfs_gfxoff_residency_read(struct file *f, char __user · 67 unmodified lines
1266 1252
1267 r = result;1253 r = result;
1268out:1254out:
1269 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1270 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1255 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1271 1256
1272 return r;1257 return r;
@@ -1315,7 +1300,6 @@ static ssize_t amdgpu_debugfs_gfxoff_residency_write(struct file *f, const char · 42 unmodified lines
1315 1300
1316 r = result;1301 r = result;
1317out:1302out:
1318 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1319 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1303 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1320 1304
1321 return r;1305 return r;
@@ -1365,7 +1349,6 @@ static ssize_t amdgpu_debugfs_gfxoff_count_read(struct file *f, char __user *buf · 43 unmodified lines
1365 1349
1366 r = result;1350 r = result;
1367out:1351out:
1368 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1369 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1352 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1370 1353
1371 return r;1354 return r;
@@ -1414,7 +1397,6 @@ static ssize_t amdgpu_debugfs_gfxoff_write(struct file *f, const char __user *bu · 42 unmodified lines
1414 1397
1415 r = result;1398 r = result;
1416out:1399out:
1417 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1418 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1400 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1419 1401
1420 return r;1402 return r;
@@ -1460,7 +1442,6 @@ static ssize_t amdgpu_debugfs_gfxoff_read(struct file *f, char __user *buf, · 39 unmodified lines
1460 1442
1461 r = result;1443 r = result;
1462out:1444out:
1463 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1464 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1445 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1465 1446
1466 return r;1447 return r;
@@ -1501,7 +1482,6 @@ static ssize_t amdgpu_debugfs_gfxoff_status_read(struct file *f, char __user *bu · 34 unmodified lines
1501 1482
1502 r = result;1483 r = result;
1503out:1484out:
1504 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
1505 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1485 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
1506 1486
1507 return r;1487 return r;
@@ -1701,7 +1681,6 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 193 unmodified lines
1701 1681
1702 up_write(&adev->reset_domain->sem);1682 up_write(&adev->reset_domain->sem);
1703 1683
1704 pm_runtime_mark_last_busy(dev->dev);
1705 pm_runtime_put_autosuspend(dev->dev);1684 pm_runtime_put_autosuspend(dev->dev);
1706 1685
1707 return 0;1686 return 0;
@@ -1721,7 +1700,6 @@ static int amdgpu_debugfs_evict_vram(void *data, u64 *val) · 13 unmodified lines
1721 1700
1722 *val = amdgpu_ttm_evict_resources(adev, TTM_PL_VRAM);1701 *val = amdgpu_ttm_evict_resources(adev, TTM_PL_VRAM);
1723 1702
1724 pm_runtime_mark_last_busy(dev->dev);
1725 pm_runtime_put_autosuspend(dev->dev);1703 pm_runtime_put_autosuspend(dev->dev);
1726 1704
1727 return 0;1705 return 0;
@@ -1742,7 +1720,6 @@ static int amdgpu_debugfs_evict_gtt(void *data, u64 *val) · 14 unmodified lines
1742 1720
1743 *val = amdgpu_ttm_evict_resources(adev, TTM_PL_TT);1721 *val = amdgpu_ttm_evict_resources(adev, TTM_PL_TT);
1744 1722
1745 pm_runtime_mark_last_busy(dev->dev);
1746 pm_runtime_put_autosuspend(dev->dev);1723 pm_runtime_put_autosuspend(dev->dev);
1747 1724
1748 return 0;1725 return 0;
@@ -1762,7 +1739,6 @@ static int amdgpu_debugfs_benchmark(void *data, u64 val) · 13 unmodified lines
1762 1739
1763 r = amdgpu_benchmark(adev, val);1740 r = amdgpu_benchmark(adev, val);
1764 1741
1765 pm_runtime_mark_last_busy(dev->dev);
1766 pm_runtime_put_autosuspend(dev->dev);1742 pm_runtime_put_autosuspend(dev->dev);
1767 1743
1768 return r;1744 return r;
@@ -1902,7 +1878,7 @@ no_preempt: · 133 unmodified lines
1902 continue;1878 continue;
1903 }1879 }
1904 job = to_amdgpu_job(s_job);1880 job = to_amdgpu_job(s_job);
1905 if (preempted && (&job->hw_fence.base) == fence)1881 if (preempted && (&job->hw_fence->base) == fence)
1906 /* mark the job as preempted */1882 /* mark the job as preempted */
1907 job->preemption_status |= AMDGPU_IB_PREEMPTED;1883 job->preemption_status |= AMDGPU_IB_PREEMPTED;
1908 }1884 }
@@ -2014,7 +1990,6 @@ static int amdgpu_debugfs_sclk_set(void *data, u64 val) · 105 unmodified lines
2014 ret = -EINVAL;1990 ret = -EINVAL;
2015 1991
2016out:1992out:
2017 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
2018 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);1993 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
2019 1994
2020 return ret;1995 return ret;
@@ -2123,10 +2098,9 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev) · 102 unmodified lines
2123 debugfs_create_blob("amdgpu_vbios", 0444, root,2098 debugfs_create_blob("amdgpu_vbios", 0444, root,
2124 &adev->debugfs_vbios_blob);2099 &adev->debugfs_vbios_blob);
2125 2100
2126 adev->debugfs_discovery_blob.data = adev->mman.discovery_bin;2101 if (adev->discovery.debugfs_blob.size)
2127 adev->debugfs_discovery_blob.size = adev->mman.discovery_tmr_size;2102 debugfs_create_blob("amdgpu_discovery", 0444, root,
2128 debugfs_create_blob("amdgpu_discovery", 0444, root,2103 &adev->discovery.debugfs_blob);
2129 &adev->debugfs_discovery_blob);
2130 2104
2131 return 0;2105 return 0;
2132}2106}
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c+284−134
@@ -71,6 +71,7 @@ · 70 unmodified lines
71 71
72#include "amdgpu_xgmi.h"72#include "amdgpu_xgmi.h"
73#include "amdgpu_ras.h"73#include "amdgpu_ras.h"
74#include "amdgpu_ras_mgr.h"
74#include "amdgpu_pmu.h"75#include "amdgpu_pmu.h"
75#include "amdgpu_fru_eeprom.h"76#include "amdgpu_fru_eeprom.h"
76#include "amdgpu_reset.h"77#include "amdgpu_reset.h"
@@ -179,6 +180,10 @@ struct amdgpu_init_level amdgpu_init_minimal_xgmi = { · 102 unmodified lines
179 BIT(AMD_IP_BLOCK_TYPE_PSP)180 BIT(AMD_IP_BLOCK_TYPE_PSP)
180};181};
181 182
183static int amdgpu_device_ip_resume_phase1(struct amdgpu_device *adev);
184static int amdgpu_device_ip_resume_phase2(struct amdgpu_device *adev);
185static int amdgpu_device_ip_resume_phase3(struct amdgpu_device *adev);
186
182static void amdgpu_device_load_switch_state(struct amdgpu_device *adev);187static void amdgpu_device_load_switch_state(struct amdgpu_device *adev);
183 188
184static inline bool amdgpu_ip_member_of_hwini(struct amdgpu_device *adev,189static inline bool amdgpu_ip_member_of_hwini(struct amdgpu_device *adev,
@@ -2387,7 +2392,7 @@ int amdgpu_device_ip_wait_for_idle(struct amdgpu_device *adev, · 2202 unmodified lines
2387}2392}
2388 2393
2389/**2394/**
2390 * amdgpu_device_ip_is_valid - is the hardware IP enabled2395 * amdgpu_device_ip_is_hw - is the hardware IP enabled
2391 *2396 *
2392 * @adev: amdgpu_device pointer2397 * @adev: amdgpu_device pointer
2393 * @block_type: Type of hardware IP (SMU, GFX, UVD, etc.)2398 * @block_type: Type of hardware IP (SMU, GFX, UVD, etc.)
@@ -2395,6 +2400,27 @@ int amdgpu_device_ip_wait_for_idle(struct amdgpu_device *adev, · 1 unmodified line
2395 * Check if the hardware IP is enable or not.2400 * Check if the hardware IP is enable or not.
2396 * Returns true if it the IP is enable, false if not.2401 * Returns true if it the IP is enable, false if not.
2397 */2402 */
2403bool amdgpu_device_ip_is_hw(struct amdgpu_device *adev,
2404 enum amd_ip_block_type block_type)
2405{
2406 int i;
2407
2408 for (i = 0; i < adev->num_ip_blocks; i++) {
2409 if (adev->ip_blocks[i].version->type == block_type)
2410 return adev->ip_blocks[i].status.hw;
2411 }
2412 return false;
2413}
2414
2415/**
2416 * amdgpu_device_ip_is_valid - is the hardware IP valid
2417 *
2418 * @adev: amdgpu_device pointer
2419 * @block_type: Type of hardware IP (SMU, GFX, UVD, etc.)
2420 *
2421 * Check if the hardware IP is valid or not.
2422 * Returns true if it the IP is valid, false if not.
2423 */
2398bool amdgpu_device_ip_is_valid(struct amdgpu_device *adev,2424bool amdgpu_device_ip_is_valid(struct amdgpu_device *adev,
2399 enum amd_ip_block_type block_type)2425 enum amd_ip_block_type block_type)
2400{2426{
@@ -2473,6 +2499,7 @@ static const char *ip_block_names[] = { · 72 unmodified lines
2473 [AMD_IP_BLOCK_TYPE_VPE] = "vpe",2499 [AMD_IP_BLOCK_TYPE_VPE] = "vpe",
2474 [AMD_IP_BLOCK_TYPE_UMSCH_MM] = "umsch_mm",2500 [AMD_IP_BLOCK_TYPE_UMSCH_MM] = "umsch_mm",
2475 [AMD_IP_BLOCK_TYPE_ISP] = "isp",2501 [AMD_IP_BLOCK_TYPE_ISP] = "isp",
2502 [AMD_IP_BLOCK_TYPE_RAS] = "ras",
2476};2503};
2477 2504
2478static const char *ip_block_name(struct amdgpu_device *adev, enum amd_ip_block_type type)2505static const char *ip_block_name(struct amdgpu_device *adev, enum amd_ip_block_type type)
@@ -2633,12 +2660,12 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev) · 154 unmodified lines
2633 chip_name = "arcturus";2660 chip_name = "arcturus";
2634 break;2661 break;
2635 case CHIP_NAVI12:2662 case CHIP_NAVI12:
2636 if (adev->mman.discovery_bin)2663 if (adev->discovery.bin)
2637 return 0;2664 return 0;
2638 chip_name = "navi12";2665 chip_name = "navi12";
2639 break;2666 break;
2640 case CHIP_CYAN_SKILLFISH:2667 case CHIP_CYAN_SKILLFISH:
2641 if (adev->mman.discovery_bin)2668 if (adev->discovery.bin)
2642 return 0;2669 return 0;
2643 chip_name = "cyan_skillfish";2670 chip_name = "cyan_skillfish";
2644 break;2671 break;
@@ -2763,6 +2790,10 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) · 118 unmodified lines
2763 r = amdgpu_virt_request_full_gpu(adev, true);2790 r = amdgpu_virt_request_full_gpu(adev, true);
2764 if (r)2791 if (r)
2765 return r;2792 return r;
2793
2794 r = amdgpu_virt_init_critical_region(adev);
2795 if (r)
2796 return r;
2766 }2797 }
2767 2798
2768 switch (adev->asic_type) {2799 switch (adev->asic_type) {
@@ -3652,6 +3683,20 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 883 unmodified lines
3652 "failed to release exclusive mode on fini\n");3683 "failed to release exclusive mode on fini\n");
3653 }3684 }
3654 3685
3686 /*
3687 * Driver reload on the APU can fail due to firmware validation because
3688 * the PSP is always running, as it is shared across the whole SoC.
3689 * This same issue does not occur on dGPU because it has a mechanism
3690 * that checks whether the PSP is running. A solution for those issues
3691 * in the APU is to trigger a GPU reset, but this should be done during
3692 * the unload phase to avoid adding boot latency and screen flicker.
3693 */
3694 if ((adev->flags & AMD_IS_APU) && !adev->gmc.is_app_apu) {
3695 r = amdgpu_asic_reset(adev);
3696 if (r)
3697 dev_err(adev->dev, "asic reset on %s failed\n", __func__);
3698 }
3699
3655 return 0;3700 return 0;
3656}3701}
3657 3702
@@ -3762,7 +3807,7 @@ static void amdgpu_device_delay_enable_gfx_off(struct work_struct *work) · 104 unmodified lines
3762 */3807 */
3763static int amdgpu_device_ip_suspend_phase1(struct amdgpu_device *adev)3808static int amdgpu_device_ip_suspend_phase1(struct amdgpu_device *adev)
3764{3809{
3765 int i, r;3810 int i, r, rec;
3766 3811
3767 amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);3812 amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
3768 amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);3813 amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
@@ -3783,13 +3828,25 @@ static int amdgpu_device_ip_suspend_phase1(struct amdgpu_device *adev) · 14 unmodified lines
3783 if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_DCE)3828 if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_DCE)
3784 continue;3829 continue;
3785 3830
3786 /* XXX handle errors */
3787 r = amdgpu_ip_block_suspend(&adev->ip_blocks[i]);3831 r = amdgpu_ip_block_suspend(&adev->ip_blocks[i]);
3788 if (r)3832 if (r)
3789 return r;3833 goto unwind;
3790 }3834 }
3791 3835
3792 return 0;3836 return 0;
3837unwind:
3838 rec = amdgpu_device_ip_resume_phase3(adev);
3839 if (rec)
3840 dev_err(adev->dev,
3841 "amdgpu_device_ip_resume_phase3 failed during unwind: %d\n",
3842 rec);
3843
3844 amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_ALLOW);
3845
3846 amdgpu_device_set_pg_state(adev, AMD_PG_STATE_GATE);
3847 amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);
3848
3849 return r;
3793}3850}
3794 3851
3795/**3852/**
@@ -3805,7 +3862,7 @@ static int amdgpu_device_ip_suspend_phase1(struct amdgpu_device *adev) · 9 unmodified lines
3805 */3862 */
3806static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev)3863static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev)
3807{3864{
3808 int i, r;3865 int i, r, rec;
3809 3866
3810 if (adev->in_s0ix)3867 if (adev->in_s0ix)
3811 amdgpu_dpm_gfx_state_change(adev, sGpuChangeState_D3Entry);3868 amdgpu_dpm_gfx_state_change(adev, sGpuChangeState_D3Entry);
@@ -3866,9 +3923,9 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) · 54 unmodified lines
3866 adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP)3923 adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP)
3867 continue;3924 continue;
3868 3925
3869 /* XXX handle errors */
3870 r = amdgpu_ip_block_suspend(&adev->ip_blocks[i]);3926 r = amdgpu_ip_block_suspend(&adev->ip_blocks[i]);
3871 adev->ip_blocks[i].status.hw = false;3927 if (r)
3928 goto unwind;
3872 3929
3873 /* handle putting the SMC in the appropriate state */3930 /* handle putting the SMC in the appropriate state */
3874 if (!amdgpu_sriov_vf(adev)) {3931 if (!amdgpu_sriov_vf(adev)) {
@@ -3878,13 +3935,40 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) · 3 unmodified lines
3878 dev_err(adev->dev,3935 dev_err(adev->dev,
3879 "SMC failed to set mp1 state %d, %d\n",3936 "SMC failed to set mp1 state %d, %d\n",
3880 adev->mp1_state, r);3937 adev->mp1_state, r);
3881 return r;3938 goto unwind;
3882 }3939 }
3883 }3940 }
3884 }3941 }
3885 }3942 }
3886 3943
3887 return 0;3944 return 0;
3945unwind:
3946 /* suspend phase 2 = resume phase 1 + resume phase 2 */
3947 rec = amdgpu_device_ip_resume_phase1(adev);
3948 if (rec) {
3949 dev_err(adev->dev,
3950 "amdgpu_device_ip_resume_phase1 failed during unwind: %d\n",
3951 rec);
3952 return r;
3953 }
3954
3955 rec = amdgpu_device_fw_loading(adev);
3956 if (rec) {
3957 dev_err(adev->dev,
3958 "amdgpu_device_fw_loading failed during unwind: %d\n",
3959 rec);
3960 return r;
3961 }
3962
3963 rec = amdgpu_device_ip_resume_phase2(adev);
3964 if (rec) {
3965 dev_err(adev->dev,
3966 "amdgpu_device_ip_resume_phase2 failed during unwind: %d\n",
3967 rec);
3968 return r;
3969 }
3970
3971 return r;
3888}3972}
3889 3973
3890/**3974/**
@@ -3898,7 +3982,7 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) · 7 unmodified lines
3898 * in each IP into a state suitable for suspend.3982 * in each IP into a state suitable for suspend.
3899 * Returns 0 on success, negative error code on failure.3983 * Returns 0 on success, negative error code on failure.
3900 */3984 */
3901int amdgpu_device_ip_suspend(struct amdgpu_device *adev)3985static int amdgpu_device_ip_suspend(struct amdgpu_device *adev)
3902{3986{
3903 int r;3987 int r;
3904 3988
@@ -4182,25 +4266,13 @@ bool amdgpu_device_asic_has_dc_support(struct pci_dev *pdev, · 277 unmodified lines
4182 case CHIP_PITCAIRN:4266 case CHIP_PITCAIRN:
4183 case CHIP_VERDE:4267 case CHIP_VERDE:
4184 case CHIP_OLAND:4268 case CHIP_OLAND:
4185 /*4269 return amdgpu_dc != 0 && IS_ENABLED(CONFIG_DRM_AMD_DC_SI);
4186 * We have systems in the wild with these ASICs that require
4187 * LVDS and VGA support which is not supported with DC.
4188 *
4189 * Fallback to the non-DC driver here by default so as not to
4190 * cause regressions.
4191 */
4192#if defined(CONFIG_DRM_AMD_DC_SI)
4193 return amdgpu_dc > 0;
4194#else
4195 return false;
4196#endif
4197 case CHIP_BONAIRE:
4198 case CHIP_KAVERI:4270 case CHIP_KAVERI:
4199 case CHIP_KABINI:4271 case CHIP_KABINI:
4200 case CHIP_MULLINS:4272 case CHIP_MULLINS:
4201 /*4273 /*
4202 * We have systems in the wild with these ASICs that require4274 * We have systems in the wild with these ASICs that require
4203 * VGA support which is not supported with DC.4275 * TRAVIS and NUTMEG support which is not supported with DC.
4204 *4276 *
4205 * Fallback to the non-DC driver here by default so as not to4277 * Fallback to the non-DC driver here by default so as not to
4206 * cause regressions.4278 * cause regressions.
@@ -4288,58 +4360,53 @@ static int amdgpu_device_get_job_timeout_settings(struct amdgpu_device *adev) · 81 unmodified lines
4288 long timeout;4360 long timeout;
4289 int ret = 0;4361 int ret = 0;
4290 4362
4291 /*4363 /* By default timeout for all queues is 2 sec */
4292 * By default timeout for jobs is 10 sec4364 adev->gfx_timeout = adev->compute_timeout = adev->sdma_timeout =
4293 */4365 adev->video_timeout = msecs_to_jiffies(2000);
4294 adev->compute_timeout = adev->gfx_timeout = msecs_to_jiffies(10000);
4295 adev->sdma_timeout = adev->video_timeout = adev->gfx_timeout;
4296 4366
4297 if (strnlen(input, AMDGPU_MAX_TIMEOUT_PARAM_LENGTH)) {4367 if (!strnlen(input, AMDGPU_MAX_TIMEOUT_PARAM_LENGTH))
4298 while ((timeout_setting = strsep(&input, ",")) &&4368 return 0;
4299 strnlen(timeout_setting, AMDGPU_MAX_TIMEOUT_PARAM_LENGTH)) {
4300 ret = kstrtol(timeout_setting, 0, &timeout);
4301 if (ret)
4302 return ret;
4303 4369
4304 if (timeout == 0) {4370 while ((timeout_setting = strsep(&input, ",")) &&
4305 index++;4371 strnlen(timeout_setting, AMDGPU_MAX_TIMEOUT_PARAM_LENGTH)) {
4306 continue;4372 ret = kstrtol(timeout_setting, 0, &timeout);
4307 } else if (timeout < 0) {4373 if (ret)
4308 timeout = MAX_SCHEDULE_TIMEOUT;4374 return ret;
4309 dev_warn(adev->dev, "lockup timeout disabled");
4310 add_taint(TAINT_SOFTLOCKUP, LOCKDEP_STILL_OK);
4311 } else {
4312 timeout = msecs_to_jiffies(timeout);
4313 }
4314 4375
4315 switch (index++) {4376 if (timeout == 0) {
4316 case 0:4377 index++;
4317 adev->gfx_timeout = timeout;4378 continue;
4318 break;4379 } else if (timeout < 0) {
4319 case 1:4380 timeout = MAX_SCHEDULE_TIMEOUT;
4320 adev->compute_timeout = timeout;4381 dev_warn(adev->dev, "lockup timeout disabled");
4321 break;4382 add_taint(TAINT_SOFTLOCKUP, LOCKDEP_STILL_OK);
4322 case 2:4383 } else {
4323 adev->sdma_timeout = timeout;4384 timeout = msecs_to_jiffies(timeout);
4324 break;
4325 case 3:
4326 adev->video_timeout = timeout;
4327 break;
4328 default:
4329 break;
4330 }
4331 }4385 }
4332 /*4386
4333 * There is only one value specified and4387 switch (index++) {
4334 * it should apply to all non-compute jobs.4388 case 0:
4335 */4389 adev->gfx_timeout = timeout;
4336 if (index == 1) {4390 break;
4337 adev->sdma_timeout = adev->video_timeout = adev->gfx_timeout;4391 case 1:
4338 if (amdgpu_sriov_vf(adev) || amdgpu_passthrough(adev))4392 adev->compute_timeout = timeout;
4339 adev->compute_timeout = adev->gfx_timeout;4393 break;
4394 case 2:
4395 adev->sdma_timeout = timeout;
4396 break;
4397 case 3:
4398 adev->video_timeout = timeout;
4399 break;
4400 default:
4401 break;
4340 }4402 }
4341 }4403 }
4342 4404
4405 /* When only one value specified apply it to all queues. */
4406 if (index == 1)
4407 adev->gfx_timeout = adev->compute_timeout = adev->sdma_timeout =
4408 adev->video_timeout = timeout;
4409
4343 return ret;4410 return ret;
4344}4411}
4345 4412
@@ -4394,6 +4461,55 @@ static void amdgpu_device_set_mcbp(struct amdgpu_device *adev) · 48 unmodified lines
4394 dev_info(adev->dev, "MCBP is enabled\n");4461 dev_info(adev->dev, "MCBP is enabled\n");
4395}4462}
4396 4463
4464static int amdgpu_device_sys_interface_init(struct amdgpu_device *adev)
4465{
4466 int r;
4467
4468 r = amdgpu_atombios_sysfs_init(adev);
4469 if (r)
4470 drm_err(&adev->ddev,
4471 "registering atombios sysfs failed (%d).\n", r);
4472
4473 r = amdgpu_pm_sysfs_init(adev);
4474 if (r)
4475 dev_err(adev->dev, "registering pm sysfs failed (%d).\n", r);
4476
4477 r = amdgpu_ucode_sysfs_init(adev);
4478 if (r) {
4479 adev->ucode_sysfs_en = false;
4480 dev_err(adev->dev, "Creating firmware sysfs failed (%d).\n", r);
4481 } else
4482 adev->ucode_sysfs_en = true;
4483
4484 r = amdgpu_device_attr_sysfs_init(adev);
4485 if (r)
4486 dev_err(adev->dev, "Could not create amdgpu device attr\n");
4487
4488 r = devm_device_add_group(adev->dev, &amdgpu_board_attrs_group);
4489 if (r)
4490 dev_err(adev->dev,
4491 "Could not create amdgpu board attributes\n");
4492
4493 amdgpu_fru_sysfs_init(adev);
4494 amdgpu_reg_state_sysfs_init(adev);
4495 amdgpu_xcp_sysfs_init(adev);
4496
4497 return r;
4498}
4499
4500static void amdgpu_device_sys_interface_fini(struct amdgpu_device *adev)
4501{
4502 if (adev->pm.sysfs_initialized)
4503 amdgpu_pm_sysfs_fini(adev);
4504 if (adev->ucode_sysfs_en)
4505 amdgpu_ucode_sysfs_fini(adev);
4506 amdgpu_device_attr_sysfs_fini(adev);
4507 amdgpu_fru_sysfs_fini(adev);
4508
4509 amdgpu_reg_state_sysfs_fini(adev);
4510 amdgpu_xcp_sysfs_fini(adev);
4511}
4512
4397/**4513/**
4398 * amdgpu_device_init - initialize the driver4514 * amdgpu_device_init - initialize the driver
4399 *4515 *
@@ -4493,7 +4609,6 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 93 unmodified lines
4493 mutex_init(&adev->gfx.userq_sch_mutex);4609 mutex_init(&adev->gfx.userq_sch_mutex);
4494 mutex_init(&adev->gfx.workload_profile_mutex);4610 mutex_init(&adev->gfx.workload_profile_mutex);
4495 mutex_init(&adev->vcn.workload_profile_mutex);4611 mutex_init(&adev->vcn.workload_profile_mutex);
4496 mutex_init(&adev->userq_mutex);
4497 4612
4498 amdgpu_device_init_apu_flags(adev);4613 amdgpu_device_init_apu_flags(adev);
4499 4614
@@ -4521,7 +4636,7 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 21 unmodified lines
4521 4636
4522 INIT_LIST_HEAD(&adev->pm.od_kobj_list);4637 INIT_LIST_HEAD(&adev->pm.od_kobj_list);
4523 4638
4524 INIT_LIST_HEAD(&adev->userq_mgr_list);4639 xa_init(&adev->userq_doorbell_xa);
4525 4640
4526 INIT_DELAYED_WORK(&adev->delayed_init_work,4641 INIT_DELAYED_WORK(&adev->delayed_init_work,
4527 amdgpu_device_delayed_init_work_handler);4642 amdgpu_device_delayed_init_work_handler);
@@ -4544,6 +4659,7 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 16 unmodified lines
4544 }4659 }
4545 4660
4546 INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);4661 INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
4662 INIT_WORK(&adev->userq_reset_work, amdgpu_userq_reset_work);
4547 4663
4548 adev->gfx.gfx_off_req_count = 1;4664 adev->gfx.gfx_off_req_count = 1;
4549 adev->gfx.gfx_off_residency = 0;4665 adev->gfx.gfx_off_residency = 0;
@@ -4817,39 +4933,14 @@ fence_driver_init: · 267 unmodified lines
4817 flush_delayed_work(&adev->delayed_init_work);4933 flush_delayed_work(&adev->delayed_init_work);
4818 }4934 }
4819 4935
4936 if (adev->init_lvl->level == AMDGPU_INIT_LEVEL_MINIMAL_XGMI)
4937 amdgpu_xgmi_reset_on_init(adev);
4820 /*4938 /*
4821 * Place those sysfs registering after `late_init`. As some of those4939 * Place those sysfs registering after `late_init`. As some of those
4822 * operations performed in `late_init` might affect the sysfs4940 * operations performed in `late_init` might affect the sysfs
4823 * interfaces creating.4941 * interfaces creating.
4824 */4942 */
4825 r = amdgpu_atombios_sysfs_init(adev);4943 r = amdgpu_device_sys_interface_init(adev);
4826 if (r)
4827 drm_err(&adev->ddev,
4828 "registering atombios sysfs failed (%d).\n", r);
4829
4830 r = amdgpu_pm_sysfs_init(adev);
4831 if (r)
4832 dev_err(adev->dev, "registering pm sysfs failed (%d).\n", r);
4833
4834 r = amdgpu_ucode_sysfs_init(adev);
4835 if (r) {
4836 adev->ucode_sysfs_en = false;
4837 dev_err(adev->dev, "Creating firmware sysfs failed (%d).\n", r);
4838 } else
4839 adev->ucode_sysfs_en = true;
4840
4841 r = amdgpu_device_attr_sysfs_init(adev);
4842 if (r)
4843 dev_err(adev->dev, "Could not create amdgpu device attr\n");
4844
4845 r = devm_device_add_group(adev->dev, &amdgpu_board_attrs_group);
4846 if (r)
4847 dev_err(adev->dev,
4848 "Could not create amdgpu board attributes\n");
4849
4850 amdgpu_fru_sysfs_init(adev);
4851 amdgpu_reg_state_sysfs_init(adev);
4852 amdgpu_xcp_sysfs_init(adev);
4853 4944
4854 if (IS_ENABLED(CONFIG_PERF_EVENTS))4945 if (IS_ENABLED(CONFIG_PERF_EVENTS))
4855 r = amdgpu_pmu_init(adev);4946 r = amdgpu_pmu_init(adev);
@@ -4877,9 +4968,6 @@ fence_driver_init: · 21 unmodified lines
4877 if (px)4968 if (px)
4878 vga_switcheroo_init_domain_pm_ops(adev->dev, &adev->vga_pm_domain);4969 vga_switcheroo_init_domain_pm_ops(adev->dev, &adev->vga_pm_domain);
4879 4970
4880 if (adev->init_lvl->level == AMDGPU_INIT_LEVEL_MINIMAL_XGMI)
4881 amdgpu_xgmi_reset_on_init(adev);
4882
4883 amdgpu_device_check_iommu_direct_map(adev);4971 amdgpu_device_check_iommu_direct_map(adev);
4884 4972
4885 adev->pm_nb.notifier_call = amdgpu_device_pm_notifier;4973 adev->pm_nb.notifier_call = amdgpu_device_pm_notifier;
@@ -4971,15 +5059,7 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) · 85 unmodified lines
4971 }5059 }
4972 amdgpu_fence_driver_hw_fini(adev);5060 amdgpu_fence_driver_hw_fini(adev);
4973 5061
4974 if (adev->pm.sysfs_initialized)5062 amdgpu_device_sys_interface_fini(adev);
4975 amdgpu_pm_sysfs_fini(adev);
4976 if (adev->ucode_sysfs_en)
4977 amdgpu_ucode_sysfs_fini(adev);
4978 amdgpu_device_attr_sysfs_fini(adev);
4979 amdgpu_fru_sysfs_fini(adev);
4980
4981 amdgpu_reg_state_sysfs_fini(adev);
4982 amdgpu_xcp_sysfs_fini(adev);
4983 5063
4984 /* disable ras feature must before hw fini */5064 /* disable ras feature must before hw fini */
4985 amdgpu_ras_pre_fini(adev);5065 amdgpu_ras_pre_fini(adev);
@@ -5054,7 +5134,7 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) · 68 unmodified lines
5054 5134
5055 if (IS_ENABLED(CONFIG_PERF_EVENTS))5135 if (IS_ENABLED(CONFIG_PERF_EVENTS))
5056 amdgpu_pmu_fini(adev);5136 amdgpu_pmu_fini(adev);
5057 if (adev->mman.discovery_bin)5137 if (adev->discovery.bin)
5058 amdgpu_discovery_fini(adev);5138 amdgpu_discovery_fini(adev);
5059 5139
5060 amdgpu_reset_put_reset_domain(adev->reset_domain);5140 amdgpu_reset_put_reset_domain(adev->reset_domain);
@@ -5202,7 +5282,7 @@ void amdgpu_device_complete(struct drm_device *dev) · 141 unmodified lines
5202int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients)5282int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients)
5203{5283{
5204 struct amdgpu_device *adev = drm_to_adev(dev);5284 struct amdgpu_device *adev = drm_to_adev(dev);
5205 int r = 0;5285 int r, rec;
5206 5286
5207 if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)5287 if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
5208 return 0;5288 return 0;
@@ -5218,35 +5298,92 @@ int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients) · 9 unmodified lines
5218 return r;5298 return r;
5219 }5299 }
5220 5300
5221 if (amdgpu_acpi_smart_shift_update(adev, AMDGPU_SS_DEV_D3))5301 r = amdgpu_acpi_smart_shift_update(adev, AMDGPU_SS_DEV_D3);
5222 dev_warn(adev->dev, "smart shift update failed\n");5302 if (r)
5303 goto unwind_sriov;
5223 5304
5224 if (notify_clients)5305 if (notify_clients)
5225 drm_client_dev_suspend(adev_to_drm(adev), false);5306 drm_client_dev_suspend(adev_to_drm(adev));
5226 5307
5227 cancel_delayed_work_sync(&adev->delayed_init_work);5308 cancel_delayed_work_sync(&adev->delayed_init_work);
5228 5309
5229 amdgpu_ras_suspend(adev);5310 amdgpu_ras_suspend(adev);
5230 5311
5231 amdgpu_device_ip_suspend_phase1(adev);5312 r = amdgpu_device_ip_suspend_phase1(adev);
5313 if (r)
5314 goto unwind_smartshift;
5232 5315
5233 amdgpu_amdkfd_suspend(adev, !amdgpu_sriov_vf(adev) && !adev->in_runpm);5316 amdgpu_amdkfd_suspend(adev, !amdgpu_sriov_vf(adev) && !adev->in_runpm);
5234 amdgpu_userq_suspend(adev);5317 r = amdgpu_userq_suspend(adev);
5318 if (r)
5319 goto unwind_ip_phase1;
5235 5320
5236 r = amdgpu_device_evict_resources(adev);5321 r = amdgpu_device_evict_resources(adev);
5237 if (r)5322 if (r)
5238 return r;5323 goto unwind_userq;
5239 5324
5240 amdgpu_ttm_set_buffer_funcs_status(adev, false);5325 amdgpu_ttm_set_buffer_funcs_status(adev, false);
5241 5326
5242 amdgpu_fence_driver_hw_fini(adev);5327 amdgpu_fence_driver_hw_fini(adev);
5243 5328
5244 amdgpu_device_ip_suspend_phase2(adev);5329 r = amdgpu_device_ip_suspend_phase2(adev);
5330 if (r)
5331 goto unwind_evict;
5245 5332
5246 if (amdgpu_sriov_vf(adev))5333 if (amdgpu_sriov_vf(adev))
5247 amdgpu_virt_release_full_gpu(adev, false);5334 amdgpu_virt_release_full_gpu(adev, false);
5248 5335
5249 return 0;5336 return 0;
5337
5338unwind_evict:
5339 if (adev->mman.buffer_funcs_ring->sched.ready)
5340 amdgpu_ttm_set_buffer_funcs_status(adev, true);
5341 amdgpu_fence_driver_hw_init(adev);
5342
5343unwind_userq:
5344 rec = amdgpu_userq_resume(adev);
5345 if (rec) {
5346 dev_warn(adev->dev, "failed to re-initialize user queues: %d\n", rec);
5347 return r;
5348 }
5349 rec = amdgpu_amdkfd_resume(adev, !amdgpu_sriov_vf(adev) && !adev->in_runpm);
5350 if (rec) {
5351 dev_warn(adev->dev, "failed to re-initialize kfd: %d\n", rec);
5352 return r;
5353 }
5354
5355unwind_ip_phase1:
5356 /* suspend phase 1 = resume phase 3 */
5357 rec = amdgpu_device_ip_resume_phase3(adev);
5358 if (rec) {
5359 dev_warn(adev->dev, "failed to re-initialize IPs phase1: %d\n", rec);
5360 return r;
5361 }
5362
5363unwind_smartshift:
5364 rec = amdgpu_acpi_smart_shift_update(adev, AMDGPU_SS_DEV_D0);
5365 if (rec) {
5366 dev_warn(adev->dev, "failed to re-update smart shift: %d\n", rec);
5367 return r;
5368 }
5369
5370 if (notify_clients)
5371 drm_client_dev_resume(adev_to_drm(adev));
5372
5373 amdgpu_ras_resume(adev);
5374
5375unwind_sriov:
5376 if (amdgpu_sriov_vf(adev)) {
5377 rec = amdgpu_virt_request_full_gpu(adev, true);
5378 if (rec) {
5379 dev_warn(adev->dev, "failed to reinitialize sriov: %d\n", rec);
5380 return r;
5381 }
5382 }
5383
5384 adev->in_suspend = adev->in_s0ix = adev->in_s3 = false;
5385
5386 return r;
5250}5387}
5251 5388
5252static inline int amdgpu_virt_resume(struct amdgpu_device *adev)5389static inline int amdgpu_virt_resume(struct amdgpu_device *adev)
@@ -5352,7 +5489,7 @@ exit: · 99 unmodified lines
5352 flush_delayed_work(&adev->delayed_init_work);5489 flush_delayed_work(&adev->delayed_init_work);
5353 5490
5354 if (notify_clients)5491 if (notify_clients)
5355 drm_client_dev_resume(adev_to_drm(adev), false);5492 drm_client_dev_resume(adev_to_drm(adev));
5356 5493
5357 amdgpu_ras_resume(adev);5494 amdgpu_ras_resume(adev);
5358 5495
@@ -5808,11 +5945,6 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, · 449 unmodified lines
5808 if (!amdgpu_ring_sched_ready(ring))5945 if (!amdgpu_ring_sched_ready(ring))
5809 continue;5946 continue;
5810 5947
5811 /* Clear job fence from fence drv to avoid force_completion
5812 * leave NULL and vm flush fence in fence drv
5813 */
5814 amdgpu_fence_driver_clear_job_fences(ring);
5815
5816 /* after all hw jobs are reset, hw fence is meaningless, so force_completion */5948 /* after all hw jobs are reset, hw fence is meaningless, so force_completion */
5817 amdgpu_fence_driver_force_completion(ring);5949 amdgpu_fence_driver_force_completion(ring);
5818 }5950 }
@@ -5957,7 +6089,11 @@ int amdgpu_device_reinit_after_reset(struct amdgpu_reset_context *reset_context) · 138 unmodified lines
5957 if (r)6089 if (r)
5958 goto out;6090 goto out;
5959 6091
5960 drm_client_dev_resume(adev_to_drm(tmp_adev), false);6092 r = amdgpu_userq_post_reset(tmp_adev, vram_lost);
6093 if (r)
6094 goto out;
6095
6096 drm_client_dev_resume(adev_to_drm(tmp_adev));
5961 6097
5962 /*6098 /*
5963 * The GPU enters bad state once faulty pages6099 * The GPU enters bad state once faulty pages
@@ -6179,6 +6315,7 @@ static inline void amdgpu_device_stop_pending_resets(struct amdgpu_device *adev) · 215 unmodified lines
6179 if (!amdgpu_sriov_vf(adev))6315 if (!amdgpu_sriov_vf(adev))
6180 cancel_work(&adev->reset_work);6316 cancel_work(&adev->reset_work);
6181#endif6317#endif
6318 cancel_work(&adev->userq_reset_work);
6182 6319
6183 if (adev->kfd.dev)6320 if (adev->kfd.dev)
6184 cancel_work(&adev->kfd.reset_work);6321 cancel_work(&adev->kfd.reset_work);
@@ -6292,13 +6429,15 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev, · 107 unmodified lines
6292 */6429 */
6293 amdgpu_unregister_gpu_instance(tmp_adev);6430 amdgpu_unregister_gpu_instance(tmp_adev);
6294 6431
6295 drm_client_dev_suspend(adev_to_drm(tmp_adev), false);6432 drm_client_dev_suspend(adev_to_drm(tmp_adev));
6296 6433
6297 /* disable ras on ALL IPs */6434 /* disable ras on ALL IPs */
6298 if (!need_emergency_restart && !amdgpu_reset_in_dpc(adev) &&6435 if (!need_emergency_restart && !amdgpu_reset_in_dpc(adev) &&
6299 amdgpu_device_ip_need_full_reset(tmp_adev))6436 amdgpu_device_ip_need_full_reset(tmp_adev))
6300 amdgpu_ras_suspend(tmp_adev);6437 amdgpu_ras_suspend(tmp_adev);
6301 6438
6439 amdgpu_userq_pre_reset(tmp_adev);
6440
6302 for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {6441 for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
6303 struct amdgpu_ring *ring = tmp_adev->rings[i];6442 struct amdgpu_ring *ring = tmp_adev->rings[i];
6304 6443
@@ -6528,6 +6667,9 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 223 unmodified lines
6528 goto end_reset;6667 goto end_reset;
6529 }6668 }
6530 6669
6670 /* Cannot be called after locking reset domain */
6671 amdgpu_ras_pre_reset(adev, &device_list);
6672
6531 /* We need to lock reset domain only once both for XGMI and single device */6673 /* We need to lock reset domain only once both for XGMI and single device */
6532 amdgpu_device_recovery_get_reset_lock(adev, &device_list);6674 amdgpu_device_recovery_get_reset_lock(adev, &device_list);
6533 6675
@@ -6541,7 +6683,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 7 unmodified lines
6541 *6683 *
6542 * job->base holds a reference to parent fence6684 * job->base holds a reference to parent fence
6543 */6685 */
6544 if (job && dma_fence_is_signaled(&job->hw_fence.base)) {6686 if (job && dma_fence_is_signaled(&job->hw_fence->base)) {
6545 job_signaled = true;6687 job_signaled = true;
6546 dev_info(adev->dev, "Guilty job already signaled, skipping HW reset");6688 dev_info(adev->dev, "Guilty job already signaled, skipping HW reset");
6547 goto skip_hw_reset;6689 goto skip_hw_reset;
@@ -6558,6 +6700,7 @@ skip_sched_resume: · 10 unmodified lines
6558 amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);6700 amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
6559reset_unlock:6701reset_unlock:
6560 amdgpu_device_recovery_put_reset_lock(adev, &device_list);6702 amdgpu_device_recovery_put_reset_lock(adev, &device_list);
6703 amdgpu_ras_post_reset(adev, &device_list);
6561end_reset:6704end_reset:
6562 if (hive) {6705 if (hive) {
6563 mutex_unlock(&hive->hive_lock);6706 mutex_unlock(&hive->hive_lock);
@@ -7285,10 +7428,17 @@ void amdgpu_device_flush_hdp(struct amdgpu_device *adev, · 721 unmodified lines
7285 if (adev->gmc.xgmi.connected_to_cpu)7428 if (adev->gmc.xgmi.connected_to_cpu)
7286 return;7429 return;
7287 7430
7288 if (ring && ring->funcs->emit_hdp_flush)7431 if (ring && ring->funcs->emit_hdp_flush) {
7289 amdgpu_ring_emit_hdp_flush(ring);7432 amdgpu_ring_emit_hdp_flush(ring);
7290 else7433 return;
7291 amdgpu_asic_flush_hdp(adev, ring);7434 }
7435
7436 if (!ring && amdgpu_sriov_runtime(adev)) {
7437 if (!amdgpu_kiq_hdp_flush(adev))
7438 return;
7439 }
7440
7441 amdgpu_hdp_flush(adev, ring);
7292}7442}
7293 7443
7294void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev,7444void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev,
@@ -7301,7 +7451,7 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, · 6 unmodified lines
7301 if (adev->gmc.xgmi.connected_to_cpu)7451 if (adev->gmc.xgmi.connected_to_cpu)
7302 return;7452 return;
7303 7453
7304 amdgpu_asic_invalidate_hdp(adev, ring);7454 amdgpu_hdp_invalidate(adev, ring);
7305}7455}
7306 7456
7307int amdgpu_in_reset(struct amdgpu_device *adev)7457int amdgpu_in_reset(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c+156−88
@@ -107,6 +107,7 @@ · 106 unmodified lines
107#include "vcn_v5_0_1.h"107#include "vcn_v5_0_1.h"
108#include "jpeg_v5_0_0.h"108#include "jpeg_v5_0_0.h"
109#include "jpeg_v5_0_1.h"109#include "jpeg_v5_0_1.h"
110#include "amdgpu_ras_mgr.h"
110 111
111#include "amdgpu_vpe.h"112#include "amdgpu_vpe.h"
112#if defined(CONFIG_DRM_AMD_ISP)113#if defined(CONFIG_DRM_AMD_ISP)
@@ -254,9 +255,9 @@ static int amdgpu_discovery_read_binary_from_sysmem(struct amdgpu_device *adev, · 141 unmodified lines
254 pos = tmr_offset + tmr_size - DISCOVERY_TMR_OFFSET;255 pos = tmr_offset + tmr_size - DISCOVERY_TMR_OFFSET;
255 256
256 /* This region is read-only and reserved from system use */257 /* This region is read-only and reserved from system use */
257 discv_regn = memremap(pos, adev->mman.discovery_tmr_size, MEMREMAP_WC);258 discv_regn = memremap(pos, adev->discovery.size, MEMREMAP_WC);
258 if (discv_regn) {259 if (discv_regn) {
259 memcpy(binary, discv_regn, adev->mman.discovery_tmr_size);260 memcpy(binary, discv_regn, adev->discovery.size);
260 memunmap(discv_regn);261 memunmap(discv_regn);
261 return 0;262 return 0;
262 }263 }
@@ -298,10 +299,31 @@ static int amdgpu_discovery_read_binary_from_mem(struct amdgpu_device *adev, · 35 unmodified lines
298 else299 else
299 vram_size <<= 20;300 vram_size <<= 20;
300 301
302 /*
303 * If in VRAM, discovery TMR is marked for reservation. If it is in system mem,
304 * then it is not required to be reserved.
305 */
301 if (sz_valid) {306 if (sz_valid) {
302 uint64_t pos = vram_size - DISCOVERY_TMR_OFFSET;307 if (amdgpu_sriov_vf(adev) && adev->virt.is_dynamic_crit_regn_enabled) {
303 amdgpu_device_vram_access(adev, pos, (uint32_t *)binary,308 /* For SRIOV VFs with dynamic critical region enabled,
304 adev->mman.discovery_tmr_size, false);309 * we will get the IPD binary via below call.
310 * If dynamic critical is disabled, fall through to normal seq.
311 */
312 if (amdgpu_virt_get_dynamic_data_info(adev,
313 AMD_SRIOV_MSG_IPD_TABLE_ID, binary,
314 &adev->discovery.size)) {
315 dev_err(adev->dev,
316 "failed to read discovery info from dynamic critical region.");
317 ret = -EINVAL;
318 goto exit;
319 }
320 } else {
321 uint64_t pos = vram_size - DISCOVERY_TMR_OFFSET;
322
323 amdgpu_device_vram_access(adev, pos, (uint32_t *)binary,
324 adev->discovery.size, false);
325 adev->discovery.reserve_tmr = true;
326 }
305 } else {327 } else {
306 ret = amdgpu_discovery_read_binary_from_sysmem(adev, binary);328 ret = amdgpu_discovery_read_binary_from_sysmem(adev, binary);
307 }329 }
@@ -310,7 +332,7 @@ static int amdgpu_discovery_read_binary_from_mem(struct amdgpu_device *adev, · 2 unmodified lines
310 dev_err(adev->dev,332 dev_err(adev->dev,
311 "failed to read discovery info from memory, vram size read: %llx",333 "failed to read discovery info from memory, vram size read: %llx",
312 vram_size);334 vram_size);
313 335exit:
314 return ret;336 return ret;
315}337}
316 338
@@ -389,6 +411,7 @@ static void amdgpu_discovery_harvest_config_quirk(struct amdgpu_device *adev) · 72 unmodified lines
389static int amdgpu_discovery_verify_npsinfo(struct amdgpu_device *adev,411static int amdgpu_discovery_verify_npsinfo(struct amdgpu_device *adev,
390 struct binary_header *bhdr)412 struct binary_header *bhdr)
391{413{
414 uint8_t *discovery_bin = adev->discovery.bin;
392 struct table_info *info;415 struct table_info *info;
393 uint16_t checksum;416 uint16_t checksum;
394 uint16_t offset;417 uint16_t offset;
@@ -398,14 +421,14 @@ static int amdgpu_discovery_verify_npsinfo(struct amdgpu_device *adev, · 3 unmodified lines
398 checksum = le16_to_cpu(info->checksum);421 checksum = le16_to_cpu(info->checksum);
399 422
400 struct nps_info_header *nhdr =423 struct nps_info_header *nhdr =
401 (struct nps_info_header *)(adev->mman.discovery_bin + offset);424 (struct nps_info_header *)(discovery_bin + offset);
402 425
403 if (le32_to_cpu(nhdr->table_id) != NPS_INFO_TABLE_ID) {426 if (le32_to_cpu(nhdr->table_id) != NPS_INFO_TABLE_ID) {
404 dev_dbg(adev->dev, "invalid ip discovery nps info table id\n");427 dev_dbg(adev->dev, "invalid ip discovery nps info table id\n");
405 return -EINVAL;428 return -EINVAL;
406 }429 }
407 430
408 if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,431 if (!amdgpu_discovery_verify_checksum(discovery_bin + offset,
409 le32_to_cpu(nhdr->size_bytes),432 le32_to_cpu(nhdr->size_bytes),
410 checksum)) {433 checksum)) {
411 dev_dbg(adev->dev, "invalid nps info data table checksum\n");434 dev_dbg(adev->dev, "invalid nps info data table checksum\n");
@@ -417,8 +440,11 @@ static int amdgpu_discovery_verify_npsinfo(struct amdgpu_device *adev, · 5 unmodified lines
417 440
418static const char *amdgpu_discovery_get_fw_name(struct amdgpu_device *adev)441static const char *amdgpu_discovery_get_fw_name(struct amdgpu_device *adev)
419{442{
420 if (amdgpu_discovery == 2)443 if (amdgpu_discovery == 2) {
444 /* Assume there is valid discovery TMR in VRAM even if binary is sideloaded */
445 adev->discovery.reserve_tmr = true;
421 return "amdgpu/ip_discovery.bin";446 return "amdgpu/ip_discovery.bin";
447 }
422 448
423 switch (adev->asic_type) {449 switch (adev->asic_type) {
424 case CHIP_VEGA10:450 case CHIP_VEGA10:
@@ -447,49 +473,53 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 22 unmodified lines
447{473{
448 struct table_info *info;474 struct table_info *info;
449 struct binary_header *bhdr;475 struct binary_header *bhdr;
476 uint8_t *discovery_bin;
450 const char *fw_name;477 const char *fw_name;
451 uint16_t offset;478 uint16_t offset;
452 uint16_t size;479 uint16_t size;
453 uint16_t checksum;480 uint16_t checksum;
454 int r;481 int r;
455 482
456 adev->mman.discovery_tmr_size = DISCOVERY_TMR_SIZE;483 adev->discovery.bin = kzalloc(DISCOVERY_TMR_SIZE, GFP_KERNEL);
457 adev->mman.discovery_bin = kzalloc(adev->mman.discovery_tmr_size, GFP_KERNEL);484 if (!adev->discovery.bin)
458 if (!adev->mman.discovery_bin)
459 return -ENOMEM;485 return -ENOMEM;
486 adev->discovery.size = DISCOVERY_TMR_SIZE;
487 adev->discovery.debugfs_blob.data = adev->discovery.bin;
488 adev->discovery.debugfs_blob.size = adev->discovery.size;
460 489
490 discovery_bin = adev->discovery.bin;
461 /* Read from file if it is the preferred option */491 /* Read from file if it is the preferred option */
462 fw_name = amdgpu_discovery_get_fw_name(adev);492 fw_name = amdgpu_discovery_get_fw_name(adev);
463 if (fw_name != NULL) {493 if (fw_name != NULL) {
464 drm_dbg(&adev->ddev, "use ip discovery information from file");494 drm_dbg(&adev->ddev, "use ip discovery information from file");
465 r = amdgpu_discovery_read_binary_from_file(adev, adev->mman.discovery_bin, fw_name);495 r = amdgpu_discovery_read_binary_from_file(adev, discovery_bin,
496 fw_name);
466 if (r)497 if (r)
467 goto out;498 goto out;
468 } else {499 } else {
469 drm_dbg(&adev->ddev, "use ip discovery information from memory");500 drm_dbg(&adev->ddev, "use ip discovery information from memory");
470 r = amdgpu_discovery_read_binary_from_mem(501 r = amdgpu_discovery_read_binary_from_mem(adev, discovery_bin);
471 adev, adev->mman.discovery_bin);
472 if (r)502 if (r)
473 goto out;503 goto out;
474 }504 }
475 505
476 /* check the ip discovery binary signature */506 /* check the ip discovery binary signature */
477 if (!amdgpu_discovery_verify_binary_signature(adev->mman.discovery_bin)) {507 if (!amdgpu_discovery_verify_binary_signature(discovery_bin)) {
478 dev_err(adev->dev,508 dev_err(adev->dev,
479 "get invalid ip discovery binary signature\n");509 "get invalid ip discovery binary signature\n");
480 r = -EINVAL;510 r = -EINVAL;
481 goto out;511 goto out;
482 }512 }
483 513
484 bhdr = (struct binary_header *)adev->mman.discovery_bin;514 bhdr = (struct binary_header *)discovery_bin;
485 515
486 offset = offsetof(struct binary_header, binary_checksum) +516 offset = offsetof(struct binary_header, binary_checksum) +
487 sizeof(bhdr->binary_checksum);517 sizeof(bhdr->binary_checksum);
488 size = le16_to_cpu(bhdr->binary_size) - offset;518 size = le16_to_cpu(bhdr->binary_size) - offset;
489 checksum = le16_to_cpu(bhdr->binary_checksum);519 checksum = le16_to_cpu(bhdr->binary_checksum);
490 520
491 if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,521 if (!amdgpu_discovery_verify_checksum(discovery_bin + offset, size,
492 size, checksum)) {522 checksum)) {
493 dev_err(adev->dev, "invalid ip discovery binary checksum\n");523 dev_err(adev->dev, "invalid ip discovery binary checksum\n");
494 r = -EINVAL;524 r = -EINVAL;
495 goto out;525 goto out;
@@ -501,15 +531,16 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 5 unmodified lines
501 531
502 if (offset) {532 if (offset) {
503 struct ip_discovery_header *ihdr =533 struct ip_discovery_header *ihdr =
504 (struct ip_discovery_header *)(adev->mman.discovery_bin + offset);534 (struct ip_discovery_header *)(discovery_bin + offset);
505 if (le32_to_cpu(ihdr->signature) != DISCOVERY_TABLE_SIGNATURE) {535 if (le32_to_cpu(ihdr->signature) != DISCOVERY_TABLE_SIGNATURE) {
506 dev_err(adev->dev, "invalid ip discovery data table signature\n");536 dev_err(adev->dev, "invalid ip discovery data table signature\n");
507 r = -EINVAL;537 r = -EINVAL;
508 goto out;538 goto out;
509 }539 }
510 540
511 if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,541 if (!amdgpu_discovery_verify_checksum(discovery_bin + offset,
512 le16_to_cpu(ihdr->size), checksum)) {542 le16_to_cpu(ihdr->size),
543 checksum)) {
513 dev_err(adev->dev, "invalid ip discovery data table checksum\n");544 dev_err(adev->dev, "invalid ip discovery data table checksum\n");
514 r = -EINVAL;545 r = -EINVAL;
515 goto out;546 goto out;
@@ -522,7 +553,7 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 6 unmodified lines
522 553
523 if (offset) {554 if (offset) {
524 struct gpu_info_header *ghdr =555 struct gpu_info_header *ghdr =
525 (struct gpu_info_header *)(adev->mman.discovery_bin + offset);556 (struct gpu_info_header *)(discovery_bin + offset);
526 557
527 if (le32_to_cpu(ghdr->table_id) != GC_TABLE_ID) {558 if (le32_to_cpu(ghdr->table_id) != GC_TABLE_ID) {
528 dev_err(adev->dev, "invalid ip discovery gc table id\n");559 dev_err(adev->dev, "invalid ip discovery gc table id\n");
@@ -530,8 +561,9 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 1 unmodified line
530 goto out;561 goto out;
531 }562 }
532 563
533 if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,564 if (!amdgpu_discovery_verify_checksum(discovery_bin + offset,
534 le32_to_cpu(ghdr->size), checksum)) {565 le32_to_cpu(ghdr->size),
566 checksum)) {
535 dev_err(adev->dev, "invalid gc data table checksum\n");567 dev_err(adev->dev, "invalid gc data table checksum\n");
536 r = -EINVAL;568 r = -EINVAL;
537 goto out;569 goto out;
@@ -544,7 +576,7 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 6 unmodified lines
544 576
545 if (offset) {577 if (offset) {
546 struct harvest_info_header *hhdr =578 struct harvest_info_header *hhdr =
547 (struct harvest_info_header *)(adev->mman.discovery_bin + offset);579 (struct harvest_info_header *)(discovery_bin + offset);
548 580
549 if (le32_to_cpu(hhdr->signature) != HARVEST_TABLE_SIGNATURE) {581 if (le32_to_cpu(hhdr->signature) != HARVEST_TABLE_SIGNATURE) {
550 dev_err(adev->dev, "invalid ip discovery harvest table signature\n");582 dev_err(adev->dev, "invalid ip discovery harvest table signature\n");
@@ -552,8 +584,9 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 1 unmodified line
552 goto out;584 goto out;
553 }585 }
554 586
555 if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,587 if (!amdgpu_discovery_verify_checksum(
556 sizeof(struct harvest_table), checksum)) {588 discovery_bin + offset,
589 sizeof(struct harvest_table), checksum)) {
557 dev_err(adev->dev, "invalid harvest data table checksum\n");590 dev_err(adev->dev, "invalid harvest data table checksum\n");
558 r = -EINVAL;591 r = -EINVAL;
559 goto out;592 goto out;
@@ -566,7 +599,7 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 6 unmodified lines
566 599
567 if (offset) {600 if (offset) {
568 struct vcn_info_header *vhdr =601 struct vcn_info_header *vhdr =
569 (struct vcn_info_header *)(adev->mman.discovery_bin + offset);602 (struct vcn_info_header *)(discovery_bin + offset);
570 603
571 if (le32_to_cpu(vhdr->table_id) != VCN_INFO_TABLE_ID) {604 if (le32_to_cpu(vhdr->table_id) != VCN_INFO_TABLE_ID) {
572 dev_err(adev->dev, "invalid ip discovery vcn table id\n");605 dev_err(adev->dev, "invalid ip discovery vcn table id\n");
@@ -574,8 +607,9 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 1 unmodified line
574 goto out;607 goto out;
575 }608 }
576 609
577 if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,610 if (!amdgpu_discovery_verify_checksum(
578 le32_to_cpu(vhdr->size_bytes), checksum)) {611 discovery_bin + offset,
612 le32_to_cpu(vhdr->size_bytes), checksum)) {
579 dev_err(adev->dev, "invalid vcn data table checksum\n");613 dev_err(adev->dev, "invalid vcn data table checksum\n");
580 r = -EINVAL;614 r = -EINVAL;
581 goto out;615 goto out;
@@ -588,7 +622,7 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 6 unmodified lines
588 622
589 if (0 && offset) {623 if (0 && offset) {
590 struct mall_info_header *mhdr =624 struct mall_info_header *mhdr =
591 (struct mall_info_header *)(adev->mman.discovery_bin + offset);625 (struct mall_info_header *)(discovery_bin + offset);
592 626
593 if (le32_to_cpu(mhdr->table_id) != MALL_INFO_TABLE_ID) {627 if (le32_to_cpu(mhdr->table_id) != MALL_INFO_TABLE_ID) {
594 dev_err(adev->dev, "invalid ip discovery mall table id\n");628 dev_err(adev->dev, "invalid ip discovery mall table id\n");
@@ -596,8 +630,9 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 1 unmodified line
596 goto out;630 goto out;
597 }631 }
598 632
599 if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset,633 if (!amdgpu_discovery_verify_checksum(
600 le32_to_cpu(mhdr->size_bytes), checksum)) {634 discovery_bin + offset,
635 le32_to_cpu(mhdr->size_bytes), checksum)) {
601 dev_err(adev->dev, "invalid mall data table checksum\n");636 dev_err(adev->dev, "invalid mall data table checksum\n");
602 r = -EINVAL;637 r = -EINVAL;
603 goto out;638 goto out;
@@ -607,8 +642,8 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 3 unmodified lines
607 return 0;642 return 0;
608 643
609out:644out:
610 kfree(adev->mman.discovery_bin);645 kfree(adev->discovery.bin);
611 adev->mman.discovery_bin = NULL;646 adev->discovery.bin = NULL;
612 if ((amdgpu_discovery != 2) &&647 if ((amdgpu_discovery != 2) &&
613 (RREG32(mmIP_DISCOVERY_VERSION) == 4))648 (RREG32(mmIP_DISCOVERY_VERSION) == 4))
614 amdgpu_ras_query_boot_status(adev, 4);649 amdgpu_ras_query_boot_status(adev, 4);
@@ -620,8 +655,8 @@ static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev); · 5 unmodified lines
620void amdgpu_discovery_fini(struct amdgpu_device *adev)655void amdgpu_discovery_fini(struct amdgpu_device *adev)
621{656{
622 amdgpu_discovery_sysfs_fini(adev);657 amdgpu_discovery_sysfs_fini(adev);
623 kfree(adev->mman.discovery_bin);658 kfree(adev->discovery.bin);
624 adev->mman.discovery_bin = NULL;659 adev->discovery.bin = NULL;
625}660}
626 661
627static int amdgpu_discovery_validate_ip(struct amdgpu_device *adev,662static int amdgpu_discovery_validate_ip(struct amdgpu_device *adev,
@@ -646,6 +681,7 @@ static int amdgpu_discovery_validate_ip(struct amdgpu_device *adev, · 18 unmodified lines
646static void amdgpu_discovery_read_harvest_bit_per_ip(struct amdgpu_device *adev,681static void amdgpu_discovery_read_harvest_bit_per_ip(struct amdgpu_device *adev,
647 uint32_t *vcn_harvest_count)682 uint32_t *vcn_harvest_count)
648{683{
684 uint8_t *discovery_bin = adev->discovery.bin;
649 struct binary_header *bhdr;685 struct binary_header *bhdr;
650 struct ip_discovery_header *ihdr;686 struct ip_discovery_header *ihdr;
651 struct die_header *dhdr;687 struct die_header *dhdr;
@@ -655,21 +691,21 @@ static void amdgpu_discovery_read_harvest_bit_per_ip(struct amdgpu_device *adev, · 3 unmodified lines
655 uint8_t inst;691 uint8_t inst;
656 int i, j;692 int i, j;
657 693
658 bhdr = (struct binary_header *)adev->mman.discovery_bin;694 bhdr = (struct binary_header *)discovery_bin;
659 ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin +695 ihdr = (struct ip_discovery_header
660 le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset));696 *)(discovery_bin +
697 le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset));
661 num_dies = le16_to_cpu(ihdr->num_dies);698 num_dies = le16_to_cpu(ihdr->num_dies);
662 699
663 /* scan harvest bit of all IP data structures */700 /* scan harvest bit of all IP data structures */
664 for (i = 0; i < num_dies; i++) {701 for (i = 0; i < num_dies; i++) {
665 die_offset = le16_to_cpu(ihdr->die_info[i].die_offset);702 die_offset = le16_to_cpu(ihdr->die_info[i].die_offset);
666 dhdr = (struct die_header *)(adev->mman.discovery_bin + die_offset);703 dhdr = (struct die_header *)(discovery_bin + die_offset);
667 num_ips = le16_to_cpu(dhdr->num_ips);704 num_ips = le16_to_cpu(dhdr->num_ips);
668 ip_offset = die_offset + sizeof(*dhdr);705 ip_offset = die_offset + sizeof(*dhdr);
669 706
670 for (j = 0; j < num_ips; j++) {707 for (j = 0; j < num_ips; j++) {
671 ip = (struct ip *)(adev->mman.discovery_bin +708 ip = (struct ip *)(discovery_bin + ip_offset);
672 ip_offset);
673 inst = ip->number_instance;709 inst = ip->number_instance;
674 hw_id = le16_to_cpu(ip->hw_id);710 hw_id = le16_to_cpu(ip->hw_id);
675 if (amdgpu_discovery_validate_ip(adev, inst, hw_id))711 if (amdgpu_discovery_validate_ip(adev, inst, hw_id))
@@ -711,13 +747,14 @@ static void amdgpu_discovery_read_from_harvest_table(struct amdgpu_device *adev, · 35 unmodified lines
711 uint32_t *vcn_harvest_count,747 uint32_t *vcn_harvest_count,
712 uint32_t *umc_harvest_count)748 uint32_t *umc_harvest_count)
713{749{
750 uint8_t *discovery_bin = adev->discovery.bin;
714 struct binary_header *bhdr;751 struct binary_header *bhdr;
715 struct harvest_table *harvest_info;752 struct harvest_table *harvest_info;
716 u16 offset;753 u16 offset;
717 int i;754 int i;
718 uint32_t umc_harvest_config = 0;755 uint32_t umc_harvest_config = 0;
719 756
720 bhdr = (struct binary_header *)adev->mman.discovery_bin;757 bhdr = (struct binary_header *)discovery_bin;
721 offset = le16_to_cpu(bhdr->table_list[HARVEST_INFO].offset);758 offset = le16_to_cpu(bhdr->table_list[HARVEST_INFO].offset);
722 759
723 if (!offset) {760 if (!offset) {
@@ -725,7 +762,7 @@ static void amdgpu_discovery_read_from_harvest_table(struct amdgpu_device *adev, · 1 unmodified line
725 return;762 return;
726 }763 }
727 764
728 harvest_info = (struct harvest_table *)(adev->mman.discovery_bin + offset);765 harvest_info = (struct harvest_table *)(discovery_bin + offset);
729 766
730 for (i = 0; i < 32; i++) {767 for (i = 0; i < 32; i++) {
731 if (le16_to_cpu(harvest_info->list[i].hw_id) == 0)768 if (le16_to_cpu(harvest_info->list[i].hw_id) == 0)
@@ -1021,8 +1058,8 @@ static void ip_disc_release(struct kobject *kobj) · 289 unmodified lines
1021 kobj);1058 kobj);
1022 struct amdgpu_device *adev = ip_top->adev;1059 struct amdgpu_device *adev = ip_top->adev;
1023 1060
1024 adev->ip_top = NULL;
1025 kfree(ip_top);1061 kfree(ip_top);
1062 adev->discovery.ip_top = NULL;
1026}1063}
1027 1064
1028static uint8_t amdgpu_discovery_get_harvest_info(struct amdgpu_device *adev,1065static uint8_t amdgpu_discovery_get_harvest_info(struct amdgpu_device *adev,
@@ -1062,6 +1099,7 @@ static int amdgpu_discovery_sysfs_ips(struct amdgpu_device *adev, · 33 unmodified lines
1062 const size_t _ip_offset, const int num_ips,1099 const size_t _ip_offset, const int num_ips,
1063 bool reg_base_64)1100 bool reg_base_64)
1064{1101{
1102 uint8_t *discovery_bin = adev->discovery.bin;
1065 int ii, jj, kk, res;1103 int ii, jj, kk, res;
1066 uint16_t hw_id;1104 uint16_t hw_id;
1067 uint8_t inst;1105 uint8_t inst;
@@ -1079,7 +1117,7 @@ static int amdgpu_discovery_sysfs_ips(struct amdgpu_device *adev, · 11 unmodified lines
1079 struct ip_v4 *ip;1117 struct ip_v4 *ip;
1080 struct ip_hw_instance *ip_hw_instance;1118 struct ip_hw_instance *ip_hw_instance;
1081 1119
1082 ip = (struct ip_v4 *)(adev->mman.discovery_bin + ip_offset);1120 ip = (struct ip_v4 *)(discovery_bin + ip_offset);
1083 inst = ip->instance_number;1121 inst = ip->instance_number;
1084 hw_id = le16_to_cpu(ip->hw_id);1122 hw_id = le16_to_cpu(ip->hw_id);
1085 if (amdgpu_discovery_validate_ip(adev, inst, hw_id) ||1123 if (amdgpu_discovery_validate_ip(adev, inst, hw_id) ||
@@ -1166,17 +1204,20 @@ next_ip: · 80 unmodified lines
1166 1204
1167static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev)1205static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev)
1168{1206{
1207 struct ip_discovery_top *ip_top = adev->discovery.ip_top;
1208 uint8_t *discovery_bin = adev->discovery.bin;
1169 struct binary_header *bhdr;1209 struct binary_header *bhdr;
1170 struct ip_discovery_header *ihdr;1210 struct ip_discovery_header *ihdr;
1171 struct die_header *dhdr;1211 struct die_header *dhdr;
1172 struct kset *die_kset = &adev->ip_top->die_kset;1212 struct kset *die_kset = &ip_top->die_kset;
1173 u16 num_dies, die_offset, num_ips;1213 u16 num_dies, die_offset, num_ips;
1174 size_t ip_offset;1214 size_t ip_offset;
1175 int ii, res;1215 int ii, res;
1176 1216
1177 bhdr = (struct binary_header *)adev->mman.discovery_bin;1217 bhdr = (struct binary_header *)discovery_bin;
1178 ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin +1218 ihdr = (struct ip_discovery_header
1179 le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset));1219 *)(discovery_bin +
1220 le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset));
1180 num_dies = le16_to_cpu(ihdr->num_dies);1221 num_dies = le16_to_cpu(ihdr->num_dies);
1181 1222
1182 DRM_DEBUG("number of dies: %d\n", num_dies);1223 DRM_DEBUG("number of dies: %d\n", num_dies);
@@ -1185,7 +1226,7 @@ static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev) · 2 unmodified lines
1185 struct ip_die_entry *ip_die_entry;1226 struct ip_die_entry *ip_die_entry;
1186 1227
1187 die_offset = le16_to_cpu(ihdr->die_info[ii].die_offset);1228 die_offset = le16_to_cpu(ihdr->die_info[ii].die_offset);
1188 dhdr = (struct die_header *)(adev->mman.discovery_bin + die_offset);1229 dhdr = (struct die_header *)(discovery_bin + die_offset);
1189 num_ips = le16_to_cpu(dhdr->num_ips);1230 num_ips = le16_to_cpu(dhdr->num_ips);
1190 ip_offset = die_offset + sizeof(*dhdr);1231 ip_offset = die_offset + sizeof(*dhdr);
1191 1232
@@ -1219,30 +1260,32 @@ static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev) · 27 unmodified lines
1219 1260
1220static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)1261static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)
1221{1262{
1263 uint8_t *discovery_bin = adev->discovery.bin;
1264 struct ip_discovery_top *ip_top;
1222 struct kset *die_kset;1265 struct kset *die_kset;
1223 int res, ii;1266 int res, ii;
1224 1267
1225 if (!adev->mman.discovery_bin)1268 if (!discovery_bin)
1226 return -EINVAL;1269 return -EINVAL;
1227 1270
1228 adev->ip_top = kzalloc(sizeof(*adev->ip_top), GFP_KERNEL);1271 ip_top = kzalloc(sizeof(*ip_top), GFP_KERNEL);
1229 if (!adev->ip_top)1272 if (!ip_top)
1230 return -ENOMEM;1273 return -ENOMEM;
1231 1274
1232 adev->ip_top->adev = adev;1275 ip_top->adev = adev;
1233 1276 adev->discovery.ip_top = ip_top;
1234 res = kobject_init_and_add(&adev->ip_top->kobj, &ip_discovery_ktype,1277 res = kobject_init_and_add(&ip_top->kobj, &ip_discovery_ktype,
1235 &adev->dev->kobj, "ip_discovery");1278 &adev->dev->kobj, "ip_discovery");
1236 if (res) {1279 if (res) {
1237 DRM_ERROR("Couldn't init and add ip_discovery/");1280 DRM_ERROR("Couldn't init and add ip_discovery/");
1238 goto Err;1281 goto Err;
1239 }1282 }
1240 1283
1241 die_kset = &adev->ip_top->die_kset;1284 die_kset = &ip_top->die_kset;
1242 kobject_set_name(&die_kset->kobj, "%s", "die");1285 kobject_set_name(&die_kset->kobj, "%s", "die");
1243 die_kset->kobj.parent = &adev->ip_top->kobj;1286 die_kset->kobj.parent = &ip_top->kobj;
1244 die_kset->kobj.ktype = &die_kobj_ktype;1287 die_kset->kobj.ktype = &die_kobj_ktype;
1245 res = kset_register(&adev->ip_top->die_kset);1288 res = kset_register(&ip_top->die_kset);
1246 if (res) {1289 if (res) {
1247 DRM_ERROR("Couldn't register die_kset");1290 DRM_ERROR("Couldn't register die_kset");
1248 goto Err;1291 goto Err;
@@ -1256,7 +1299,7 @@ static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev) · 7 unmodified lines
1256 1299
1257 return res;1300 return res;
1258Err:1301Err:
1259 kobject_put(&adev->ip_top->kobj);1302 kobject_put(&ip_top->kobj);
1260 return res;1303 return res;
1261}1304}
1262 1305
@@ -1301,10 +1344,11 @@ static void amdgpu_discovery_sysfs_die_free(struct ip_die_entry *ip_die_entry) · 38 unmodified lines
1301 1344
1302static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev)1345static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev)
1303{1346{
1347 struct ip_discovery_top *ip_top = adev->discovery.ip_top;
1304 struct list_head *el, *tmp;1348 struct list_head *el, *tmp;
1305 struct kset *die_kset;1349 struct kset *die_kset;
1306 1350
1307 die_kset = &adev->ip_top->die_kset;1351 die_kset = &ip_top->die_kset;
1308 spin_lock(&die_kset->list_lock);1352 spin_lock(&die_kset->list_lock);
1309 list_for_each_prev_safe(el, tmp, &die_kset->list) {1353 list_for_each_prev_safe(el, tmp, &die_kset->list) {
1310 list_del_init(el);1354 list_del_init(el);
@@ -1313,8 +1357,8 @@ static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev) · 2 unmodified lines
1313 spin_lock(&die_kset->list_lock);1357 spin_lock(&die_kset->list_lock);
1314 }1358 }
1315 spin_unlock(&die_kset->list_lock);1359 spin_unlock(&die_kset->list_lock);
1316 kobject_put(&adev->ip_top->die_kset.kobj);1360 kobject_put(&ip_top->die_kset.kobj);
1317 kobject_put(&adev->ip_top->kobj);1361 kobject_put(&ip_top->kobj);
1318}1362}
1319 1363
1320/* ================================================== */1364/* ================================================== */
@@ -1325,6 +1369,7 @@ static int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev) · 4 unmodified lines
1325 struct binary_header *bhdr;1369 struct binary_header *bhdr;
1326 struct ip_discovery_header *ihdr;1370 struct ip_discovery_header *ihdr;
1327 struct die_header *dhdr;1371 struct die_header *dhdr;
1372 uint8_t *discovery_bin;
1328 struct ip_v4 *ip;1373 struct ip_v4 *ip;
1329 uint16_t die_offset;1374 uint16_t die_offset;
1330 uint16_t ip_offset;1375 uint16_t ip_offset;
@@ -1340,22 +1385,23 @@ static int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev) · 9 unmodified lines
1340 r = amdgpu_discovery_init(adev);1385 r = amdgpu_discovery_init(adev);
1341 if (r)1386 if (r)
1342 return r;1387 return r;
1343 1388 discovery_bin = adev->discovery.bin;
1344 wafl_ver = 0;1389 wafl_ver = 0;
1345 adev->gfx.xcc_mask = 0;1390 adev->gfx.xcc_mask = 0;
1346 adev->sdma.sdma_mask = 0;1391 adev->sdma.sdma_mask = 0;
1347 adev->vcn.inst_mask = 0;1392 adev->vcn.inst_mask = 0;
1348 adev->jpeg.inst_mask = 0;1393 adev->jpeg.inst_mask = 0;
1349 bhdr = (struct binary_header *)adev->mman.discovery_bin;1394 bhdr = (struct binary_header *)discovery_bin;
1350 ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin +1395 ihdr = (struct ip_discovery_header
1351 le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset));1396 *)(discovery_bin +
1397 le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset));
1352 num_dies = le16_to_cpu(ihdr->num_dies);1398 num_dies = le16_to_cpu(ihdr->num_dies);
1353 1399
1354 DRM_DEBUG("number of dies: %d\n", num_dies);1400 DRM_DEBUG("number of dies: %d\n", num_dies);
1355 1401
1356 for (i = 0; i < num_dies; i++) {1402 for (i = 0; i < num_dies; i++) {
1357 die_offset = le16_to_cpu(ihdr->die_info[i].die_offset);1403 die_offset = le16_to_cpu(ihdr->die_info[i].die_offset);
1358 dhdr = (struct die_header *)(adev->mman.discovery_bin + die_offset);1404 dhdr = (struct die_header *)(discovery_bin + die_offset);
1359 num_ips = le16_to_cpu(dhdr->num_ips);1405 num_ips = le16_to_cpu(dhdr->num_ips);
1360 ip_offset = die_offset + sizeof(*dhdr);1406 ip_offset = die_offset + sizeof(*dhdr);
1361 1407
@@ -1369,7 +1415,7 @@ static int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev) · 7 unmodified lines
1369 le16_to_cpu(dhdr->die_id), num_ips);1415 le16_to_cpu(dhdr->die_id), num_ips);
1370 1416
1371 for (j = 0; j < num_ips; j++) {1417 for (j = 0; j < num_ips; j++) {
1372 ip = (struct ip_v4 *)(adev->mman.discovery_bin + ip_offset);1418 ip = (struct ip_v4 *)(discovery_bin + ip_offset);
1373 1419
1374 inst = ip->instance_number;1420 inst = ip->instance_number;
1375 hw_id = le16_to_cpu(ip->hw_id);1421 hw_id = le16_to_cpu(ip->hw_id);
@@ -1519,16 +1565,16 @@ next_ip: · 143 unmodified lines
1519 1565
1520static void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev)1566static void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev)
1521{1567{
1568 uint8_t *discovery_bin = adev->discovery.bin;
1522 struct ip_discovery_header *ihdr;1569 struct ip_discovery_header *ihdr;
1523 struct binary_header *bhdr;1570 struct binary_header *bhdr;
1524 int vcn_harvest_count = 0;1571 int vcn_harvest_count = 0;
1525 int umc_harvest_count = 0;1572 int umc_harvest_count = 0;
1526 uint16_t offset, ihdr_ver;1573 uint16_t offset, ihdr_ver;
1527 1574
1528 bhdr = (struct binary_header *)adev->mman.discovery_bin;1575 bhdr = (struct binary_header *)discovery_bin;
1529 offset = le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset);1576 offset = le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset);
1530 ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin +1577 ihdr = (struct ip_discovery_header *)(discovery_bin + offset);
1531 offset);
1532 ihdr_ver = le16_to_cpu(ihdr->version);1578 ihdr_ver = le16_to_cpu(ihdr->version);
1533 /*1579 /*
1534 * Harvest table does not fit Navi1x and legacy GPUs,1580 * Harvest table does not fit Navi1x and legacy GPUs,
@@ -1575,22 +1621,23 @@ union gc_info { · 40 unmodified lines
1575 1621
1576static int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev)1622static int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev)
1577{1623{
1624 uint8_t *discovery_bin = adev->discovery.bin;
1578 struct binary_header *bhdr;1625 struct binary_header *bhdr;
1579 union gc_info *gc_info;1626 union gc_info *gc_info;
1580 u16 offset;1627 u16 offset;
1581 1628
1582 if (!adev->mman.discovery_bin) {1629 if (!discovery_bin) {
1583 DRM_ERROR("ip discovery uninitialized\n");1630 DRM_ERROR("ip discovery uninitialized\n");
1584 return -EINVAL;1631 return -EINVAL;
1585 }1632 }
1586 1633
1587 bhdr = (struct binary_header *)adev->mman.discovery_bin;1634 bhdr = (struct binary_header *)discovery_bin;
1588 offset = le16_to_cpu(bhdr->table_list[GC].offset);1635 offset = le16_to_cpu(bhdr->table_list[GC].offset);
1589 1636
1590 if (!offset)1637 if (!offset)
1591 return 0;1638 return 0;
1592 1639
1593 gc_info = (union gc_info *)(adev->mman.discovery_bin + offset);1640 gc_info = (union gc_info *)(discovery_bin + offset);
1594 1641
1595 switch (le16_to_cpu(gc_info->v1.header.version_major)) {1642 switch (le16_to_cpu(gc_info->v1.header.version_major)) {
1596 case 1:1643 case 1:
@@ -1683,24 +1730,25 @@ union mall_info { · 86 unmodified lines
1683 1730
1684static int amdgpu_discovery_get_mall_info(struct amdgpu_device *adev)1731static int amdgpu_discovery_get_mall_info(struct amdgpu_device *adev)
1685{1732{
1733 uint8_t *discovery_bin = adev->discovery.bin;
1686 struct binary_header *bhdr;1734 struct binary_header *bhdr;
1687 union mall_info *mall_info;1735 union mall_info *mall_info;
1688 u32 u, mall_size_per_umc, m_s_present, half_use;1736 u32 u, mall_size_per_umc, m_s_present, half_use;
1689 u64 mall_size;1737 u64 mall_size;
1690 u16 offset;1738 u16 offset;
1691 1739
1692 if (!adev->mman.discovery_bin) {1740 if (!discovery_bin) {
1693 DRM_ERROR("ip discovery uninitialized\n");1741 DRM_ERROR("ip discovery uninitialized\n");
1694 return -EINVAL;1742 return -EINVAL;
1695 }1743 }
1696 1744
1697 bhdr = (struct binary_header *)adev->mman.discovery_bin;1745 bhdr = (struct binary_header *)discovery_bin;
1698 offset = le16_to_cpu(bhdr->table_list[MALL_INFO].offset);1746 offset = le16_to_cpu(bhdr->table_list[MALL_INFO].offset);
1699 1747
1700 if (!offset)1748 if (!offset)
1701 return 0;1749 return 0;
1702 1750
1703 mall_info = (union mall_info *)(adev->mman.discovery_bin + offset);1751 mall_info = (union mall_info *)(discovery_bin + offset);
1704 1752
1705 switch (le16_to_cpu(mall_info->v1.header.version_major)) {1753 switch (le16_to_cpu(mall_info->v1.header.version_major)) {
1706 case 1:1754 case 1:
@@ -1739,12 +1787,13 @@ union vcn_info { · 32 unmodified lines
1739 1787
1740static int amdgpu_discovery_get_vcn_info(struct amdgpu_device *adev)1788static int amdgpu_discovery_get_vcn_info(struct amdgpu_device *adev)
1741{1789{
1790 uint8_t *discovery_bin = adev->discovery.bin;
1742 struct binary_header *bhdr;1791 struct binary_header *bhdr;
1743 union vcn_info *vcn_info;1792 union vcn_info *vcn_info;
1744 u16 offset;1793 u16 offset;
1745 int v;1794 int v;
1746 1795
1747 if (!adev->mman.discovery_bin) {1796 if (!discovery_bin) {
1748 DRM_ERROR("ip discovery uninitialized\n");1797 DRM_ERROR("ip discovery uninitialized\n");
1749 return -EINVAL;1798 return -EINVAL;
1750 }1799 }
@@ -1759,13 +1808,13 @@ static int amdgpu_discovery_get_vcn_info(struct amdgpu_device *adev) · 8 unmodified lines
1759 return -EINVAL;1808 return -EINVAL;
1760 }1809 }
1761 1810
1762 bhdr = (struct binary_header *)adev->mman.discovery_bin;1811 bhdr = (struct binary_header *)discovery_bin;
1763 offset = le16_to_cpu(bhdr->table_list[VCN_INFO].offset);1812 offset = le16_to_cpu(bhdr->table_list[VCN_INFO].offset);
1764 1813
1765 if (!offset)1814 if (!offset)
1766 return 0;1815 return 0;
1767 1816
1768 vcn_info = (union vcn_info *)(adev->mman.discovery_bin + offset);1817 vcn_info = (union vcn_info *)(discovery_bin + offset);
1769 1818
1770 switch (le16_to_cpu(vcn_info->v1.header.version_major)) {1819 switch (le16_to_cpu(vcn_info->v1.header.version_major)) {
1771 case 1:1820 case 1:
@@ -1825,6 +1874,7 @@ int amdgpu_discovery_get_nps_info(struct amdgpu_device *adev, · 53 unmodified lines
1825 struct amdgpu_gmc_memrange **ranges,1874 struct amdgpu_gmc_memrange **ranges,
1826 int *range_cnt, bool refresh)1875 int *range_cnt, bool refresh)
1827{1876{
1877 uint8_t *discovery_bin = adev->discovery.bin;
1828 struct amdgpu_gmc_memrange *mem_ranges;1878 struct amdgpu_gmc_memrange *mem_ranges;
1829 struct binary_header *bhdr;1879 struct binary_header *bhdr;
1830 union nps_info *nps_info;1880 union nps_info *nps_info;
@@ -1841,13 +1891,13 @@ int amdgpu_discovery_get_nps_info(struct amdgpu_device *adev, · 10 unmodified lines
1841 return r;1891 return r;
1842 nps_info = &nps_data;1892 nps_info = &nps_data;
1843 } else {1893 } else {
1844 if (!adev->mman.discovery_bin) {1894 if (!discovery_bin) {
1845 dev_err(adev->dev,1895 dev_err(adev->dev,
1846 "fetch mem range failed, ip discovery uninitialized\n");1896 "fetch mem range failed, ip discovery uninitialized\n");
1847 return -EINVAL;1897 return -EINVAL;
1848 }1898 }
1849 1899
1850 bhdr = (struct binary_header *)adev->mman.discovery_bin;1900 bhdr = (struct binary_header *)discovery_bin;
1851 offset = le16_to_cpu(bhdr->table_list[NPS_INFO].offset);1901 offset = le16_to_cpu(bhdr->table_list[NPS_INFO].offset);
1852 1902
1853 if (!offset)1903 if (!offset)
@@ -1857,8 +1907,7 @@ int amdgpu_discovery_get_nps_info(struct amdgpu_device *adev, · 3 unmodified lines
1857 if (amdgpu_discovery_verify_npsinfo(adev, bhdr))1907 if (amdgpu_discovery_verify_npsinfo(adev, bhdr))
1858 return -ENOENT;1908 return -ENOENT;
1859 1909
1860 nps_info =1910 nps_info = (union nps_info *)(discovery_bin + offset);
1861 (union nps_info *)(adev->mman.discovery_bin + offset);
1862 }1911 }
1863 1912
1864 switch (le16_to_cpu(nps_info->v1.header.version_major)) {1913 switch (le16_to_cpu(nps_info->v1.header.version_major)) {
@@ -2361,6 +2410,21 @@ static int amdgpu_discovery_set_sdma_ip_blocks(struct amdgpu_device *adev) · 496 unmodified lines
2361 amdgpu_ip_version(adev, SDMA0_HWIP, 0));2410 amdgpu_ip_version(adev, SDMA0_HWIP, 0));
2362 return -EINVAL;2411 return -EINVAL;
2363 }2412 }
2413
2414 return 0;
2415}
2416
2417static int amdgpu_discovery_set_ras_ip_blocks(struct amdgpu_device *adev)
2418{
2419 switch (amdgpu_ip_version(adev, MP0_HWIP, 0)) {
2420 case IP_VERSION(13, 0, 6):
2421 case IP_VERSION(13, 0, 12):
2422 case IP_VERSION(13, 0, 14):
2423 amdgpu_device_ip_block_add(adev, &ras_v1_0_ip_block);
2424 break;
2425 default:
2426 break;
2427 }
2364 return 0;2428 return 0;
2365}2429}
2366 2430
@@ -3141,6 +3205,10 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 774 unmodified lines
3141 if (r)3205 if (r)
3142 return r;3206 return r;
3143 3207
3208 r = amdgpu_discovery_set_ras_ip_blocks(adev);
3209 if (r)
3210 return r;
3211
3144 if ((adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT &&3212 if ((adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT &&
3145 !amdgpu_sriov_vf(adev)) ||3213 !amdgpu_sriov_vf(adev)) ||
3146 (adev->firmware.load_type == AMDGPU_FW_LOAD_RLC_BACKDOOR_AUTO && amdgpu_dpm == 1)) {3214 (adev->firmware.load_type == AMDGPU_FW_LOAD_RLC_BACKDOOR_AUTO && amdgpu_dpm == 1)) {
drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h+12−0
@@ -24,9 +24,21 @@ · 23 unmodified lines
24#ifndef __AMDGPU_DISCOVERY__24#ifndef __AMDGPU_DISCOVERY__
25#define __AMDGPU_DISCOVERY__25#define __AMDGPU_DISCOVERY__
26 26
27#include <linux/debugfs.h>
28
27#define DISCOVERY_TMR_SIZE (10 << 10)29#define DISCOVERY_TMR_SIZE (10 << 10)
28#define DISCOVERY_TMR_OFFSET (64 << 10)30#define DISCOVERY_TMR_OFFSET (64 << 10)
29 31
32struct ip_discovery_top;
33
34struct amdgpu_discovery_info {
35 struct debugfs_blob_wrapper debugfs_blob;
36 struct ip_discovery_top *ip_top;
37 uint32_t size;
38 uint8_t *bin;
39 bool reserve_tmr;
40};
41
30void amdgpu_discovery_fini(struct amdgpu_device *adev);42void amdgpu_discovery_fini(struct amdgpu_device *adev);
31int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev);43int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev);
32 44
drivers/gpu/drm/amd/amdgpu/amdgpu_display.c+59−3
@@ -332,8 +332,6 @@ int amdgpu_display_crtc_set_config(struct drm_mode_set *set, · 331 unmodified lines
332 if (crtc->enabled)332 if (crtc->enabled)
333 active = true;333 active = true;
334 334
335 pm_runtime_mark_last_busy(dev->dev);
336
337 adev = drm_to_adev(dev);335 adev = drm_to_adev(dev);
338 /* if we have active crtcs and we don't have a power ref,336 /* if we have active crtcs and we don't have a power ref,
339 * take the current one337 * take the current one
@@ -1365,6 +1363,64 @@ static const struct drm_prop_enum_list amdgpu_dither_enum_list[] = { · 1025 unmodified lines
1365 { AMDGPU_FMT_DITHER_ENABLE, "on" },1363 { AMDGPU_FMT_DITHER_ENABLE, "on" },
1366};1364};
1367 1365
1366/**
1367 * DOC: property for adaptive backlight modulation
1368 *
1369 * The 'adaptive backlight modulation' property is used for the compositor to
1370 * directly control the adaptive backlight modulation power savings feature
1371 * that is part of DCN hardware.
1372 *
1373 * The property will be attached specifically to eDP panels that support it.
1374 *
1375 * The property is by default set to 'sysfs' to allow the sysfs file 'panel_power_savings'
1376 * to be able to control it.
1377 * If set to 'off' the compositor will ensure it stays off.
1378 * The other values 'min', 'bias min', 'bias max', and 'max' will control the
1379 * intensity of the power savings.
1380 *
1381 * Modifying this value can have implications on color accuracy, so tread
1382 * carefully.
1383 */
1384static int amdgpu_display_setup_abm_prop(struct amdgpu_device *adev)
1385{
1386 const struct drm_prop_enum_list props[] = {
1387 { ABM_SYSFS_CONTROL, "sysfs" },
1388 { ABM_LEVEL_OFF, "off" },
1389 { ABM_LEVEL_MIN, "min" },
1390 { ABM_LEVEL_BIAS_MIN, "bias min" },
1391 { ABM_LEVEL_BIAS_MAX, "bias max" },
1392 { ABM_LEVEL_MAX, "max" },
1393 };
1394 struct drm_property *prop;
1395 int i;
1396
1397 if (!adev->dc_enabled)
1398 return 0;
1399
1400 prop = drm_property_create(adev_to_drm(adev), DRM_MODE_PROP_ENUM,
1401 "adaptive backlight modulation",
1402 6);
1403 if (!prop)
1404 return -ENOMEM;
1405
1406 for (i = 0; i < ARRAY_SIZE(props); i++) {
1407 int ret;
1408
1409 ret = drm_property_add_enum(prop, props[i].type,
1410 props[i].name);
1411
1412 if (ret) {
1413 drm_property_destroy(adev_to_drm(adev), prop);
1414
1415 return ret;
1416 }
1417 }
1418
1419 adev->mode_info.abm_level_property = prop;
1420
1421 return 0;
1422}
1423
1368int amdgpu_display_modeset_create_props(struct amdgpu_device *adev)1424int amdgpu_display_modeset_create_props(struct amdgpu_device *adev)
1369{1425{
1370 int sz;1426 int sz;
@@ -1411,7 +1467,7 @@ int amdgpu_display_modeset_create_props(struct amdgpu_device *adev) · 40 unmodified lines
1411 "dither",1467 "dither",
1412 amdgpu_dither_enum_list, sz);1468 amdgpu_dither_enum_list, sz);
1413 1469
1414 return 0;1470 return amdgpu_display_setup_abm_prop(adev);
1415}1471}
1416 1472
1417void amdgpu_display_update_priority(struct amdgpu_device *adev)1473void amdgpu_display_update_priority(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/amdgpu_display.h+7−0
@@ -55,4 +55,11 @@ int amdgpu_display_resume_helper(struct amdgpu_device *adev); · 54 unmodified lines
55int amdgpu_display_get_scanout_buffer(struct drm_plane *plane,55int amdgpu_display_get_scanout_buffer(struct drm_plane *plane,
56 struct drm_scanout_buffer *sb);56 struct drm_scanout_buffer *sb);
57 57
58#define ABM_SYSFS_CONTROL -1
59#define ABM_LEVEL_OFF 0
60#define ABM_LEVEL_MIN 1
61#define ABM_LEVEL_BIAS_MIN 2
62#define ABM_LEVEL_BIAS_MAX 3
63#define ABM_LEVEL_MAX 4
64
58#endif65#endif
drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c+19−0
@@ -81,6 +81,19 @@ static int amdgpu_dma_buf_attach(struct dma_buf *dmabuf, · 80 unmodified lines
81 struct drm_gem_object *obj = dmabuf->priv;81 struct drm_gem_object *obj = dmabuf->priv;
82 struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);82 struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
83 struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);83 struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
84 int r;
85
86 /*
87 * Disable peer-to-peer access for DCC-enabled VRAM surfaces on GFX12+.
88 * Such buffers cannot be safely accessed over P2P due to device-local
89 * compression metadata. Fallback to system-memory path instead.
90 * Device supports GFX12 (GC 12.x or newer)
91 * BO was created with the AMDGPU_GEM_CREATE_GFX12_DCC flag
92 *
93 */
94 if (amdgpu_ip_version(adev, GC_HWIP, 0) >= IP_VERSION(12, 0, 0) &&
95 bo->flags & AMDGPU_GEM_CREATE_GFX12_DCC)
96 attach->peer2peer = false;
84 97
85 /*98 /*
86 * Disable peer-to-peer access for DCC-enabled VRAM surfaces on GFX12+.99 * Disable peer-to-peer access for DCC-enabled VRAM surfaces on GFX12+.
@@ -98,8 +111,14 @@ static int amdgpu_dma_buf_attach(struct dma_buf *dmabuf, · 11 unmodified lines
98 pci_p2pdma_distance(adev->pdev, attach->dev, false) < 0)111 pci_p2pdma_distance(adev->pdev, attach->dev, false) < 0)
99 attach->peer2peer = false;112 attach->peer2peer = false;
100 113
114 r = dma_resv_lock(bo->tbo.base.resv, NULL);
115 if (r)
116 return r;
117
101 amdgpu_vm_bo_update_shared(bo);118 amdgpu_vm_bo_update_shared(bo);
102 119
120 dma_resv_unlock(bo->tbo.base.resv);
121
103 return 0;122 return 0;
104}123}
105 124
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c+104−101
@@ -312,7 +312,7 @@ module_param_named(moverate, amdgpu_moverate, int, 0600); · 311 unmodified lines
312 * DOC: audio (int)312 * DOC: audio (int)
313 * Set HDMI/DPAudio. Only affects non-DC display handling. The default is -1 (Enabled), set 0 to disabled it.313 * Set HDMI/DPAudio. Only affects non-DC display handling. The default is -1 (Enabled), set 0 to disabled it.
314 */314 */
315MODULE_PARM_DESC(audio, "Audio enable (-1 = auto, 0 = disable, 1 = enable)");315MODULE_PARM_DESC(audio, "HDMI/DP Audio enable for non DC displays (-1 = auto, 0 = disable, 1 = enable)");
316module_param_named(audio, amdgpu_audio, int, 0444);316module_param_named(audio, amdgpu_audio, int, 0444);
317 317
318/**318/**
@@ -354,22 +354,16 @@ module_param_named(svm_default_granularity, amdgpu_svm_default_granularity, uint · 35 unmodified lines
354 * DOC: lockup_timeout (string)354 * DOC: lockup_timeout (string)
355 * Set GPU scheduler timeout value in ms.355 * Set GPU scheduler timeout value in ms.
356 *356 *
357 * The format can be [Non-Compute] or [GFX,Compute,SDMA,Video]. That is there can be one or357 * The format can be [single value] for setting all timeouts at once or
358 * multiple values specified. 0 and negative values are invalidated. They will be adjusted358 * [GFX,Compute,SDMA,Video] to set individual timeouts.
359 * to the default timeout.359 * Negative values mean infinity.
360 *360 *
361 * - With one value specified, the setting will apply to all non-compute jobs.361 * By default(with no lockup_timeout settings), the timeout for all queues is 2000.
362 * - With multiple values specified, the first one will be for GFX.
363 * The second one is for Compute. The third and fourth ones are
364 * for SDMA and Video.
365 *
366 * By default(with no lockup_timeout settings), the timeout for all jobs is 10000.
367 */362 */
368MODULE_PARM_DESC(lockup_timeout,363MODULE_PARM_DESC(lockup_timeout,
369 "GPU lockup timeout in ms (default: 10000 for all jobs. "364 "GPU lockup timeout in ms (default: 2000. 0: keep default value. negative: infinity timeout), format: [single value for all] or [GFX,Compute,SDMA,Video].");
370 "0: keep default value. negative: infinity timeout), format: for bare metal [Non-Compute] or [GFX,Compute,SDMA,Video]; "365module_param_string(lockup_timeout, amdgpu_lockup_timeout,
371 "for passthrough or sriov [all jobs] or [GFX,Compute,SDMA,Video].");366 sizeof(amdgpu_lockup_timeout), 0444);
372module_param_string(lockup_timeout, amdgpu_lockup_timeout, sizeof(amdgpu_lockup_timeout), 0444);
373 367
374/**368/**
375 * DOC: dpm (int)369 * DOC: dpm (int)
@@ -624,39 +618,39 @@ module_param_named(timeout_period, amdgpu_watchdog_timer.period, uint, 0644); · 248 unmodified lines
624 618
625/**619/**
626 * DOC: si_support (int)620 * DOC: si_support (int)
627 * Set SI support driver. This parameter works after set config CONFIG_DRM_AMDGPU_SI. For SI asic, when radeon driver is enabled,621 * 1 = enabled, 0 = disabled, -1 = default
628 * set value 0 to use radeon driver, while set value 1 to use amdgpu driver. The default is using radeon driver when it available,622 *
629 * otherwise using amdgpu driver.623 * SI (Southern Islands) are first generation GCN GPUs, supported by both
630 */624 * drivers: radeon (old) and amdgpu (new). This parameter controls whether
625 * amdgpu should support SI.
626 * By default, SI dedicated GPUs are supported by amdgpu.
627 * Only relevant when CONFIG_DRM_AMDGPU_SI is enabled to build SI support in amdgpu.
628 * See also radeon.si_support which should be disabled when amdgpu.si_support is
629 * enabled, and vice versa.
630 */
631int amdgpu_si_support = -1;
631#ifdef CONFIG_DRM_AMDGPU_SI632#ifdef CONFIG_DRM_AMDGPU_SI
632 633MODULE_PARM_DESC(si_support, "SI support (1 = enabled, 0 = disabled, -1 = default)");
633#if IS_ENABLED(CONFIG_DRM_RADEON) || IS_ENABLED(CONFIG_DRM_RADEON_MODULE)
634int amdgpu_si_support;
635MODULE_PARM_DESC(si_support, "SI support (1 = enabled, 0 = disabled (default))");
636#else
637int amdgpu_si_support = 1;
638MODULE_PARM_DESC(si_support, "SI support (1 = enabled (default), 0 = disabled)");
639#endif
640
641module_param_named(si_support, amdgpu_si_support, int, 0444);634module_param_named(si_support, amdgpu_si_support, int, 0444);
642#endif635#endif
643 636
644/**637/**
645 * DOC: cik_support (int)638 * DOC: cik_support (int)
646 * Set CIK support driver. This parameter works after set config CONFIG_DRM_AMDGPU_CIK. For CIK asic, when radeon driver is enabled,639 * 1 = enabled, 0 = disabled, -1 = default
647 * set value 0 to use radeon driver, while set value 1 to use amdgpu driver. The default is using radeon driver when it available,640 *
648 * otherwise using amdgpu driver.641 * CIK (Sea Islands) are second generation GCN GPUs, supported by both
649 */642 * drivers: radeon (old) and amdgpu (new). This parameter controls whether
643 * amdgpu should support CIK.
644 * By default:
645 * - CIK dedicated GPUs are supported by amdgpu.
646 * - CIK APUs are supported by radeon (except when radeon is not built).
647 * Only relevant when CONFIG_DRM_AMDGPU_CIK is enabled to build CIK support in amdgpu.
648 * See also radeon.cik_support which should be disabled when amdgpu.cik_support is
649 * enabled, and vice versa.
650 */
651int amdgpu_cik_support = -1;
650#ifdef CONFIG_DRM_AMDGPU_CIK652#ifdef CONFIG_DRM_AMDGPU_CIK
651 653MODULE_PARM_DESC(cik_support, "CIK support (1 = enabled, 0 = disabled, -1 = default)");
652#if IS_ENABLED(CONFIG_DRM_RADEON) || IS_ENABLED(CONFIG_DRM_RADEON_MODULE)
653int amdgpu_cik_support;
654MODULE_PARM_DESC(cik_support, "CIK support (1 = enabled, 0 = disabled (default))");
655#else
656int amdgpu_cik_support = 1;
657MODULE_PARM_DESC(cik_support, "CIK support (1 = enabled (default), 0 = disabled)");
658#endif
659
660module_param_named(cik_support, amdgpu_cik_support, int, 0444);654module_param_named(cik_support, amdgpu_cik_support, int, 0444);
661#endif655#endif
662 656
@@ -2234,7 +2228,6 @@ static void amdgpu_get_secondary_funcs(struct amdgpu_device *adev) · 1571 unmodified lines
2234 adev->pdev->bus->number, i);2228 adev->pdev->bus->number, i);
2235 if (p) {2229 if (p) {
2236 pm_runtime_get_sync(&p->dev);2230 pm_runtime_get_sync(&p->dev);
2237 pm_runtime_mark_last_busy(&p->dev);
2238 pm_runtime_put_autosuspend(&p->dev);2231 pm_runtime_put_autosuspend(&p->dev);
2239 pci_dev_put(p);2232 pci_dev_put(p);
2240 }2233 }
@@ -2313,6 +2306,72 @@ static unsigned long amdgpu_fix_asic_type(struct pci_dev *pdev, unsigned long fl · 72 unmodified lines
2313 return flags;2306 return flags;
2314}2307}
2315 2308
2309static bool amdgpu_support_enabled(struct device *dev,
2310 const enum amd_asic_type family)
2311{
2312 const char *gen;
2313 const char *param;
2314 int module_param = -1;
2315 bool radeon_support_built = IS_ENABLED(CONFIG_DRM_RADEON);
2316 bool amdgpu_support_built = false;
2317 bool support_by_default = false;
2318
2319 switch (family) {
2320 case CHIP_TAHITI:
2321 case CHIP_PITCAIRN:
2322 case CHIP_VERDE:
2323 case CHIP_OLAND:
2324 case CHIP_HAINAN:
2325 gen = "SI";
2326 param = "si_support";
2327 module_param = amdgpu_si_support;
2328 amdgpu_support_built = IS_ENABLED(CONFIG_DRM_AMDGPU_SI);
2329 support_by_default = true;
2330 break;
2331
2332 case CHIP_BONAIRE:
2333 case CHIP_HAWAII:
2334 support_by_default = true;
2335 fallthrough;
2336 case CHIP_KAVERI:
2337 case CHIP_KABINI:
2338 case CHIP_MULLINS:
2339 gen = "CIK";
2340 param = "cik_support";
2341 module_param = amdgpu_cik_support;
2342 amdgpu_support_built = IS_ENABLED(CONFIG_DRM_AMDGPU_CIK);
2343 break;
2344
2345 default:
2346 /* All other chips are supported by amdgpu only */
2347 return true;
2348 }
2349
2350 if (!amdgpu_support_built) {
2351 dev_info(dev, "amdgpu built without %s support\n", gen);
2352 return false;
2353 }
2354
2355 if ((module_param == -1 && (support_by_default || !radeon_support_built)) ||
2356 module_param == 1) {
2357 if (radeon_support_built)
2358 dev_info(dev, "%s support provided by amdgpu.\n"
2359 "Use radeon.%s=1 amdgpu.%s=0 to override.\n",
2360 gen, param, param);
2361
2362 return true;
2363 }
2364
2365 if (radeon_support_built)
2366 dev_info(dev, "%s support provided by radeon.\n"
2367 "Use radeon.%s=0 amdgpu.%s=1 to override.\n",
2368 gen, param, param);
2369 else if (module_param == 0)
2370 dev_info(dev, "%s support disabled by module param\n", gen);
2371
2372 return false;
2373}
2374
2316static int amdgpu_pci_probe(struct pci_dev *pdev,2375static int amdgpu_pci_probe(struct pci_dev *pdev,
2317 const struct pci_device_id *ent)2376 const struct pci_device_id *ent)
2318{2377{
@@ -2360,48 +2419,8 @@ static int amdgpu_pci_probe(struct pci_dev *pdev, · 41 unmodified lines
2360 return -ENOTSUPP;2419 return -ENOTSUPP;
2361 }2420 }
2362 2421
2363 switch (flags & AMD_ASIC_MASK) {2422 if (!amdgpu_support_enabled(&pdev->dev, flags & AMD_ASIC_MASK))
2364 case CHIP_TAHITI:
2365 case CHIP_PITCAIRN:
2366 case CHIP_VERDE:
2367 case CHIP_OLAND:
2368 case CHIP_HAINAN:
2369#ifdef CONFIG_DRM_AMDGPU_SI
2370 if (!amdgpu_si_support) {
2371 dev_info(&pdev->dev,
2372 "SI support provided by radeon.\n");
2373 dev_info(&pdev->dev,
2374 "Use radeon.si_support=0 amdgpu.si_support=1 to override.\n"
2375 );
2376 return -ENODEV;
2377 }
2378 break;
2379#else
2380 dev_info(&pdev->dev, "amdgpu is built without SI support.\n");
2381 return -ENODEV;
2382#endif
2383 case CHIP_KAVERI:
2384 case CHIP_BONAIRE:
2385 case CHIP_HAWAII:
2386 case CHIP_KABINI:
2387 case CHIP_MULLINS:
2388#ifdef CONFIG_DRM_AMDGPU_CIK
2389 if (!amdgpu_cik_support) {
2390 dev_info(&pdev->dev,
2391 "CIK support provided by radeon.\n");
2392 dev_info(&pdev->dev,
2393 "Use radeon.cik_support=0 amdgpu.cik_support=1 to override.\n"
2394 );
2395 return -ENODEV;
2396 }
2397 break;
2398#else
2399 dev_info(&pdev->dev, "amdgpu is built without CIK support.\n");
2400 return -ENODEV;2423 return -ENODEV;
2401#endif
2402 default:
2403 break;
2404 }
2405 2424
2406 adev = devm_drm_dev_alloc(&pdev->dev, &amdgpu_kms_driver, typeof(*adev), ddev);2425 adev = devm_drm_dev_alloc(&pdev->dev, &amdgpu_kms_driver, typeof(*adev), ddev);
2407 if (IS_ERR(adev))2426 if (IS_ERR(adev))
@@ -2480,7 +2499,6 @@ retry_init: · 72 unmodified lines
2480 2499
2481 pm_runtime_allow(ddev->dev);2500 pm_runtime_allow(ddev->dev);
2482 2501
2483 pm_runtime_mark_last_busy(ddev->dev);
2484 pm_runtime_put_autosuspend(ddev->dev);2502 pm_runtime_put_autosuspend(ddev->dev);
2485 2503
2486 pci_wake_from_d3(pdev, TRUE);2504 pci_wake_from_d3(pdev, TRUE);
@@ -2564,7 +2582,8 @@ amdgpu_pci_shutdown(struct pci_dev *pdev) · 77 unmodified lines
2564 */2582 */
2565 if (!amdgpu_passthrough(adev))2583 if (!amdgpu_passthrough(adev))
2566 adev->mp1_state = PP_MP1_STATE_UNLOAD;2584 adev->mp1_state = PP_MP1_STATE_UNLOAD;
2567 amdgpu_device_ip_suspend(adev);2585 amdgpu_device_prepare(dev);
2586 amdgpu_device_suspend(dev, true);
2568 adev->mp1_state = PP_MP1_STATE_NONE;2587 adev->mp1_state = PP_MP1_STATE_NONE;
2569}2588}
2570 2589
@@ -2782,22 +2801,8 @@ static int amdgpu_runtime_idle_check_userq(struct device *dev) · 211 unmodified lines
2782 struct pci_dev *pdev = to_pci_dev(dev);2801 struct pci_dev *pdev = to_pci_dev(dev);
2783 struct drm_device *drm_dev = pci_get_drvdata(pdev);2802 struct drm_device *drm_dev = pci_get_drvdata(pdev);
2784 struct amdgpu_device *adev = drm_to_adev(drm_dev);2803 struct amdgpu_device *adev = drm_to_adev(drm_dev);
2785 struct amdgpu_usermode_queue *queue;
2786 struct amdgpu_userq_mgr *uqm, *tmp;
2787 int queue_id;
2788 int ret = 0;
2789
2790 mutex_lock(&adev->userq_mutex);
2791 list_for_each_entry_safe(uqm, tmp, &adev->userq_mgr_list, list) {
2792 idr_for_each_entry(&uqm->userq_idr, queue, queue_id) {
2793 ret = -EBUSY;
2794 goto done;
2795 }
2796 }
2797done:
2798 mutex_unlock(&adev->userq_mutex);
2799 2804
2800 return ret;2805 return xa_empty(&adev->userq_doorbell_xa) ? 0 : -EBUSY;
2801}2806}
2802 2807
2803static int amdgpu_pmops_runtime_suspend(struct device *dev)2808static int amdgpu_pmops_runtime_suspend(struct device *dev)
@@ -2944,7 +2949,6 @@ static int amdgpu_pmops_runtime_idle(struct device *dev) · 140 unmodified lines
2944 2949
2945 ret = amdgpu_runtime_idle_check_userq(dev);2950 ret = amdgpu_runtime_idle_check_userq(dev);
2946done:2951done:
2947 pm_runtime_mark_last_busy(dev);
2948 pm_runtime_autosuspend(dev);2952 pm_runtime_autosuspend(dev);
2949 return ret;2953 return ret;
2950}2954}
@@ -2980,7 +2984,6 @@ long amdgpu_drm_ioctl(struct file *filp, · 29 unmodified lines
2980 2984
2981 ret = drm_ioctl(filp, cmd, arg);2985 ret = drm_ioctl(filp, cmd, arg);
2982 2986
2983 pm_runtime_mark_last_busy(dev->dev);
2984out:2987out:
2985 pm_runtime_put_autosuspend(dev->dev);2988 pm_runtime_put_autosuspend(dev->dev);
2986 return ret;2989 return ret;
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c+13−132
@@ -45,16 +45,11 @@ · 44 unmodified lines
45 * Cast helper45 * Cast helper
46 */46 */
47static const struct dma_fence_ops amdgpu_fence_ops;47static const struct dma_fence_ops amdgpu_fence_ops;
48static const struct dma_fence_ops amdgpu_job_fence_ops;
49static inline struct amdgpu_fence *to_amdgpu_fence(struct dma_fence *f)48static inline struct amdgpu_fence *to_amdgpu_fence(struct dma_fence *f)
50{49{
51 struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, base);50 struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, base);
52 51
53 if (__f->base.ops == &amdgpu_fence_ops ||52 return __f;
54 __f->base.ops == &amdgpu_job_fence_ops)
55 return __f;
56
57 return NULL;
58}53}
59 54
60/**55/**
@@ -98,51 +93,32 @@ static u32 amdgpu_fence_read(struct amdgpu_ring *ring) · 37 unmodified lines
98 * amdgpu_fence_emit - emit a fence on the requested ring93 * amdgpu_fence_emit - emit a fence on the requested ring
99 *94 *
100 * @ring: ring the fence is associated with95 * @ring: ring the fence is associated with
101 * @f: resulting fence object
102 * @af: amdgpu fence input96 * @af: amdgpu fence input
103 * @flags: flags to pass into the subordinate .emit_fence() call97 * @flags: flags to pass into the subordinate .emit_fence() call
104 *98 *
105 * Emits a fence command on the requested ring (all asics).99 * Emits a fence command on the requested ring (all asics).
106 * Returns 0 on success, -ENOMEM on failure.100 * Returns 0 on success, -ENOMEM on failure.
107 */101 */
108int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f,102int amdgpu_fence_emit(struct amdgpu_ring *ring, struct amdgpu_fence *af,
109 struct amdgpu_fence *af, unsigned int flags)103 unsigned int flags)
110{104{
111 struct amdgpu_device *adev = ring->adev;105 struct amdgpu_device *adev = ring->adev;
112 struct dma_fence *fence;106 struct dma_fence *fence;
113 struct amdgpu_fence *am_fence;
114 struct dma_fence __rcu **ptr;107 struct dma_fence __rcu **ptr;
115 uint32_t seq;108 uint32_t seq;
116 int r;109 int r;
117 110
118 if (!af) {111 fence = &af->base;
119 /* create a separate hw fence */112 af->ring = ring;
120 am_fence = kzalloc(sizeof(*am_fence), GFP_KERNEL);
121 if (!am_fence)
122 return -ENOMEM;
123 } else {
124 am_fence = af;
125 }
126 fence = &am_fence->base;
127 am_fence->ring = ring;
128 113
129 seq = ++ring->fence_drv.sync_seq;114 seq = ++ring->fence_drv.sync_seq;
130 am_fence->seq = seq;115 dma_fence_init(fence, &amdgpu_fence_ops,
131 if (af) {116 &ring->fence_drv.lock,
132 dma_fence_init(fence, &amdgpu_job_fence_ops,117 adev->fence_context + ring->idx, seq);
133 &ring->fence_drv.lock,
134 adev->fence_context + ring->idx, seq);
135 /* Against remove in amdgpu_job_{free, free_cb} */
136 dma_fence_get(fence);
137 } else {
138 dma_fence_init(fence, &amdgpu_fence_ops,
139 &ring->fence_drv.lock,
140 adev->fence_context + ring->idx, seq);
141 }
142 118
143 amdgpu_ring_emit_fence(ring, ring->fence_drv.gpu_addr,119 amdgpu_ring_emit_fence(ring, ring->fence_drv.gpu_addr,
144 seq, flags | AMDGPU_FENCE_FLAG_INT);120 seq, flags | AMDGPU_FENCE_FLAG_INT);
145 amdgpu_fence_save_wptr(fence);121 amdgpu_fence_save_wptr(af);
146 pm_runtime_get_noresume(adev_to_drm(adev)->dev);122 pm_runtime_get_noresume(adev_to_drm(adev)->dev);
147 ptr = &ring->fence_drv.fences[seq & ring->fence_drv.num_fences_mask];123 ptr = &ring->fence_drv.fences[seq & ring->fence_drv.num_fences_mask];
148 if (unlikely(rcu_dereference_protected(*ptr, 1))) {124 if (unlikely(rcu_dereference_protected(*ptr, 1))) {
@@ -167,8 +143,6 @@ int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f, · 18 unmodified lines
167 */143 */
168 rcu_assign_pointer(*ptr, dma_fence_get(fence));144 rcu_assign_pointer(*ptr, dma_fence_get(fence));
169 145
170 *f = fence;
171
172 return 0;146 return 0;
173}147}
174 148
@@ -276,7 +250,6 @@ bool amdgpu_fence_process(struct amdgpu_ring *ring) · 101 unmodified lines
276 drv->signalled_wptr = am_fence->wptr;250 drv->signalled_wptr = am_fence->wptr;
277 dma_fence_signal(fence);251 dma_fence_signal(fence);
278 dma_fence_put(fence);252 dma_fence_put(fence);
279 pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
280 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);253 pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
281 } while (last_seq != seq);254 } while (last_seq != seq);
282 255
@@ -669,36 +642,6 @@ void amdgpu_fence_driver_hw_init(struct amdgpu_device *adev) · 386 unmodified lines
669 }642 }
670}643}
671 644
672/**
673 * amdgpu_fence_driver_clear_job_fences - clear job embedded fences of ring
674 *
675 * @ring: fence of the ring to be cleared
676 *
677 */
678void amdgpu_fence_driver_clear_job_fences(struct amdgpu_ring *ring)
679{
680 int i;
681 struct dma_fence *old, **ptr;
682
683 for (i = 0; i <= ring->fence_drv.num_fences_mask; i++) {
684 ptr = &ring->fence_drv.fences[i];
685 old = rcu_dereference_protected(*ptr, 1);
686 if (old && old->ops == &amdgpu_job_fence_ops) {
687 struct amdgpu_job *job;
688
689 /* For non-scheduler bad job, i.e. failed ib test, we need to signal
690 * it right here or we won't be able to track them in fence_drv
691 * and they will remain unsignaled during sa_bo free.
692 */
693 job = container_of(old, struct amdgpu_job, hw_fence.base);
694 if (!job->base.s_fence && !dma_fence_is_signaled(old))
695 dma_fence_signal(old);
696 RCU_INIT_POINTER(*ptr, NULL);
697 dma_fence_put(old);
698 }
699 }
700}
701
702/**645/**
703 * amdgpu_fence_driver_set_error - set error code on fences646 * amdgpu_fence_driver_set_error - set error code on fences
704 * @ring: the ring which contains the fences647 * @ring: the ring which contains the fences
@@ -755,7 +698,7 @@ void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring) · 50 unmodified lines
755/**698/**
756 * amdgpu_fence_driver_guilty_force_completion - force signal of specified sequence699 * amdgpu_fence_driver_guilty_force_completion - force signal of specified sequence
757 *700 *
758 * @fence: fence of the ring to signal701 * @af: fence of the ring to signal
759 *702 *
760 */703 */
761void amdgpu_fence_driver_guilty_force_completion(struct amdgpu_fence *af)704void amdgpu_fence_driver_guilty_force_completion(struct amdgpu_fence *af)
@@ -792,15 +735,13 @@ void amdgpu_fence_driver_guilty_force_completion(struct amdgpu_fence *af) · 30 unmodified lines
792 } while (last_seq != seq);735 } while (last_seq != seq);
793 spin_unlock_irqrestore(&ring->fence_drv.lock, flags);736 spin_unlock_irqrestore(&ring->fence_drv.lock, flags);
794 /* signal the guilty fence */737 /* signal the guilty fence */
795 amdgpu_fence_write(ring, af->seq);738 amdgpu_fence_write(ring, (u32)af->base.seqno);
796 amdgpu_fence_process(ring);739 amdgpu_fence_process(ring);
797}740}
798 741
799void amdgpu_fence_save_wptr(struct dma_fence *fence)742void amdgpu_fence_save_wptr(struct amdgpu_fence *af)
800{743{
801 struct amdgpu_fence *am_fence = container_of(fence, struct amdgpu_fence, base);744 af->wptr = af->ring->wptr;
802
803 am_fence->wptr = am_fence->ring->wptr;
804}745}
805 746
806static void amdgpu_ring_backup_unprocessed_command(struct amdgpu_ring *ring,747static void amdgpu_ring_backup_unprocessed_command(struct amdgpu_ring *ring,
@@ -866,13 +807,6 @@ static const char *amdgpu_fence_get_timeline_name(struct dma_fence *f) · 59 unmodified lines
866 return (const char *)to_amdgpu_fence(f)->ring->name;807 return (const char *)to_amdgpu_fence(f)->ring->name;
867}808}
868 809
869static const char *amdgpu_job_fence_get_timeline_name(struct dma_fence *f)
870{
871 struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence.base);
872
873 return (const char *)to_amdgpu_ring(job->base.sched)->name;
874}
875
876/**810/**
877 * amdgpu_fence_enable_signaling - enable signalling on fence811 * amdgpu_fence_enable_signaling - enable signalling on fence
878 * @f: fence812 * @f: fence
@@ -889,23 +823,6 @@ static bool amdgpu_fence_enable_signaling(struct dma_fence *f) · 10 unmodified lines
889 return true;823 return true;
890}824}
891 825
892/**
893 * amdgpu_job_fence_enable_signaling - enable signalling on job fence
894 * @f: fence
895 *
896 * This is the simliar function with amdgpu_fence_enable_signaling above, it
897 * only handles the job embedded fence.
898 */
899static bool amdgpu_job_fence_enable_signaling(struct dma_fence *f)
900{
901 struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence.base);
902
903 if (!timer_pending(&to_amdgpu_ring(job->base.sched)->fence_drv.fallback_timer))
904 amdgpu_fence_schedule_fallback(to_amdgpu_ring(job->base.sched));
905
906 return true;
907}
908
909/**826/**
910 * amdgpu_fence_free - free up the fence memory827 * amdgpu_fence_free - free up the fence memory
911 *828 *
@@ -921,21 +838,6 @@ static void amdgpu_fence_free(struct rcu_head *rcu) · 9 unmodified lines
921 kfree(to_amdgpu_fence(f));838 kfree(to_amdgpu_fence(f));
922}839}
923 840
924/**
925 * amdgpu_job_fence_free - free up the job with embedded fence
926 *
927 * @rcu: RCU callback head
928 *
929 * Free up the job with embedded fence after the RCU grace period.
930 */
931static void amdgpu_job_fence_free(struct rcu_head *rcu)
932{
933 struct dma_fence *f = container_of(rcu, struct dma_fence, rcu);
934
935 /* free job if fence has a parent job */
936 kfree(container_of(f, struct amdgpu_job, hw_fence.base));
937}
938
939/**841/**
940 * amdgpu_fence_release - callback that fence can be freed842 * amdgpu_fence_release - callback that fence can be freed
941 *843 *
@@ -949,19 +851,6 @@ static void amdgpu_fence_release(struct dma_fence *f) · 7 unmodified lines
949 call_rcu(&f->rcu, amdgpu_fence_free);851 call_rcu(&f->rcu, amdgpu_fence_free);
950}852}
951 853
952/**
953 * amdgpu_job_fence_release - callback that job embedded fence can be freed
954 *
955 * @f: fence
956 *
957 * This is the simliar function with amdgpu_fence_release above, it
958 * only handles the job embedded fence.
959 */
960static void amdgpu_job_fence_release(struct dma_fence *f)
961{
962 call_rcu(&f->rcu, amdgpu_job_fence_free);
963}
964
965static const struct dma_fence_ops amdgpu_fence_ops = {854static const struct dma_fence_ops amdgpu_fence_ops = {
966 .get_driver_name = amdgpu_fence_get_driver_name,855 .get_driver_name = amdgpu_fence_get_driver_name,
967 .get_timeline_name = amdgpu_fence_get_timeline_name,856 .get_timeline_name = amdgpu_fence_get_timeline_name,
@@ -969,13 +858,6 @@ static const struct dma_fence_ops amdgpu_fence_ops = { · 1 unmodified line
969 .release = amdgpu_fence_release,858 .release = amdgpu_fence_release,
970};859};
971 860
972static const struct dma_fence_ops amdgpu_job_fence_ops = {
973 .get_driver_name = amdgpu_fence_get_driver_name,
974 .get_timeline_name = amdgpu_job_fence_get_timeline_name,
975 .enable_signaling = amdgpu_job_fence_enable_signaling,
976 .release = amdgpu_job_fence_release,
977};
978
979/*861/*
980 * Fence debugfs862 * Fence debugfs
981 */863 */
@@ -1045,7 +927,6 @@ static int gpu_recover_get(void *data, u64 *val) · 63 unmodified lines
1045 927
1046 *val = atomic_read(&adev->reset_domain->reset_res);928 *val = atomic_read(&adev->reset_domain->reset_res);
1047 929
1048 pm_runtime_mark_last_busy(dev->dev);
1049 pm_runtime_put_autosuspend(dev->dev);930 pm_runtime_put_autosuspend(dev->dev);
1050 931
1051 return 0;932 return 0;
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c+37−3
@@ -302,7 +302,6 @@ void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, · 301 unmodified lines
302 int pages)302 int pages)
303{303{
304 unsigned t;304 unsigned t;
305 unsigned p;
306 int i, j;305 int i, j;
307 u64 page_base;306 u64 page_base;
308 /* Starting from VEGA10, system bit must be 0 to mean invalid. */307 /* Starting from VEGA10, system bit must be 0 to mean invalid. */
@@ -316,8 +315,7 @@ void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, · 7 unmodified lines
316 return;315 return;
317 316
318 t = offset / AMDGPU_GPU_PAGE_SIZE;317 t = offset / AMDGPU_GPU_PAGE_SIZE;
319 p = t / AMDGPU_GPU_PAGES_IN_CPU_PAGE;318 for (i = 0; i < pages; i++) {
320 for (i = 0; i < pages; i++, p++) {
321 page_base = adev->dummy_page_addr;319 page_base = adev->dummy_page_addr;
322 if (!adev->gart.ptr)320 if (!adev->gart.ptr)
323 continue;321 continue;
@@ -369,6 +367,42 @@ void amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, · 45 unmodified lines
369 drm_dev_exit(idx);367 drm_dev_exit(idx);
370}368}
371 369
370/**
371 * amdgpu_gart_map_vram_range - map VRAM pages into the GART page table
372 *
373 * @adev: amdgpu_device pointer
374 * @pa: physical address of the first page to be mapped
375 * @start_page: first page to map in the GART aperture
376 * @num_pages: number of pages to be mapped
377 * @flags: page table entry flags
378 * @dst: CPU address of the GART table
379 *
380 * Binds a BO that is allocated in VRAM to the GART page table
381 * (all ASICs).
382 *
383 * Useful when a kernel BO is located in VRAM but
384 * needs to be accessed from the GART address space.
385 */
386void amdgpu_gart_map_vram_range(struct amdgpu_device *adev, uint64_t pa,
387 uint64_t start_page, uint64_t num_pages,
388 uint64_t flags, void *dst)
389{
390 u32 i, idx;
391
392 /* The SYSTEM flag indicates the pages aren't in VRAM. */
393 WARN_ON_ONCE(flags & AMDGPU_PTE_SYSTEM);
394
395 if (!drm_dev_enter(adev_to_drm(adev), &idx))
396 return;
397
398 for (i = 0; i < num_pages; ++i) {
399 amdgpu_gmc_set_pte_pde(adev, adev->gart.ptr,
400 start_page + i, pa + AMDGPU_GPU_PAGE_SIZE * i, flags);
401 }
402
403 drm_dev_exit(idx);
404}
405
372/**406/**
373 * amdgpu_gart_bind - bind pages into the gart page table407 * amdgpu_gart_bind - bind pages into the gart page table
374 *408 *
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h+3−0
@@ -64,5 +64,8 @@ void amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, · 63 unmodified lines
64 void *dst);64 void *dst);
65void amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset,65void amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset,
66 int pages, dma_addr_t *dma_addr, uint64_t flags);66 int pages, dma_addr_t *dma_addr, uint64_t flags);
67void amdgpu_gart_map_vram_range(struct amdgpu_device *adev, uint64_t pa,
68 uint64_t start_page, uint64_t num_pages,
69 uint64_t flags, void *dst);
67void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev);70void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev);
68#endif71#endif
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c+10−7
@@ -198,7 +198,7 @@ static void amdgpu_gem_object_free(struct drm_gem_object *gobj) · 197 unmodified lines
198 struct amdgpu_bo *aobj = gem_to_amdgpu_bo(gobj);198 struct amdgpu_bo *aobj = gem_to_amdgpu_bo(gobj);
199 199
200 amdgpu_hmm_unregister(aobj);200 amdgpu_hmm_unregister(aobj);
201 ttm_bo_put(&aobj->tbo);201 ttm_bo_fini(&aobj->tbo);
202}202}
203 203
204int amdgpu_gem_object_create(struct amdgpu_device *adev, unsigned long size,204int amdgpu_gem_object_create(struct amdgpu_device *adev, unsigned long size,
@@ -531,7 +531,7 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data, · 326 unmodified lines
531 struct drm_amdgpu_gem_userptr *args = data;531 struct drm_amdgpu_gem_userptr *args = data;
532 struct amdgpu_fpriv *fpriv = filp->driver_priv;532 struct amdgpu_fpriv *fpriv = filp->driver_priv;
533 struct drm_gem_object *gobj;533 struct drm_gem_object *gobj;
534 struct hmm_range *range;534 struct amdgpu_hmm_range *range;
535 struct amdgpu_bo *bo;535 struct amdgpu_bo *bo;
536 uint32_t handle;536 uint32_t handle;
537 int r;537 int r;
@@ -572,10 +572,14 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data, · 34 unmodified lines
572 goto release_object;572 goto release_object;
573 573
574 if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE) {574 if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE) {
575 r = amdgpu_ttm_tt_get_user_pages(bo, &range);575 range = amdgpu_hmm_range_alloc(NULL);
576 if (r)576 if (unlikely(!range))
577 return -ENOMEM;
578 r = amdgpu_ttm_tt_get_user_pages(bo, range);
579 if (r) {
580 amdgpu_hmm_range_free(range);
577 goto release_object;581 goto release_object;
578 582 }
579 r = amdgpu_bo_reserve(bo, true);583 r = amdgpu_bo_reserve(bo, true);
580 if (r)584 if (r)
581 goto user_pages_done;585 goto user_pages_done;
@@ -597,8 +601,7 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data, · 15 unmodified lines
597 601
598user_pages_done:602user_pages_done:
599 if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE)603 if (args->flags & AMDGPU_GEM_USERPTR_VALIDATE)
600 amdgpu_ttm_tt_get_user_pages_done(bo->tbo.ttm, range);604 amdgpu_hmm_range_free(range);
601
602release_object:605release_object:
603 drm_gem_object_put(gobj);606 drm_gem_object_put(gobj);
604 607
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c+71−1
@@ -33,6 +33,7 @@ · 32 unmodified lines
33#include "amdgpu_reset.h"33#include "amdgpu_reset.h"
34#include "amdgpu_xcp.h"34#include "amdgpu_xcp.h"
35#include "amdgpu_xgmi.h"35#include "amdgpu_xgmi.h"
36#include "amdgpu_mes.h"
36#include "nvd.h"37#include "nvd.h"
37 38
38/* delay 0.1 second to enable gfx off feature */39/* delay 0.1 second to enable gfx off feature */
@@ -1194,6 +1195,75 @@ failed_kiq_write: · 1155 unmodified lines
1194 dev_err(adev->dev, "failed to write reg:%x\n", reg);1195 dev_err(adev->dev, "failed to write reg:%x\n", reg);
1195}1196}
1196 1197
1198int amdgpu_kiq_hdp_flush(struct amdgpu_device *adev)
1199{
1200 signed long r, cnt = 0;
1201 unsigned long flags;
1202 uint32_t seq;
1203 struct amdgpu_kiq *kiq = &adev->gfx.kiq[0];
1204 struct amdgpu_ring *ring = &kiq->ring;
1205
1206 if (amdgpu_device_skip_hw_access(adev))
1207 return 0;
1208
1209 if (adev->enable_mes_kiq && adev->mes.ring[0].sched.ready)
1210 return amdgpu_mes_hdp_flush(adev);
1211
1212 if (!ring->funcs->emit_hdp_flush) {
1213 return -EOPNOTSUPP;
1214 }
1215
1216 spin_lock_irqsave(&kiq->ring_lock, flags);
1217 r = amdgpu_ring_alloc(ring, 32);
1218 if (r)
1219 goto failed_unlock;
1220
1221 amdgpu_ring_emit_hdp_flush(ring);
1222 r = amdgpu_fence_emit_polling(ring, &seq, MAX_KIQ_REG_WAIT);
1223 if (r)
1224 goto failed_undo;
1225
1226 amdgpu_ring_commit(ring);
1227 spin_unlock_irqrestore(&kiq->ring_lock, flags);
1228
1229 r = amdgpu_fence_wait_polling(ring, seq, MAX_KIQ_REG_WAIT);
1230
1231 /* don't wait anymore for gpu reset case because this way may
1232 * block gpu_recover() routine forever, e.g. this virt_kiq_rreg
1233 * is triggered in TTM and ttm_bo_lock_delayed_workqueue() will
1234 * never return if we keep waiting in virt_kiq_rreg, which cause
1235 * gpu_recover() hang there.
1236 *
1237 * also don't wait anymore for IRQ context
1238 * */
1239 if (r < 1 && (amdgpu_in_reset(adev) || in_interrupt()))
1240 goto failed_kiq_hdp_flush;
1241
1242 might_sleep();
1243 while (r < 1 && cnt++ < MAX_KIQ_REG_TRY) {
1244 if (amdgpu_in_reset(adev))
1245 goto failed_kiq_hdp_flush;
1246
1247 msleep(MAX_KIQ_REG_BAILOUT_INTERVAL);
1248 r = amdgpu_fence_wait_polling(ring, seq, MAX_KIQ_REG_WAIT);
1249 }
1250
1251 if (cnt > MAX_KIQ_REG_TRY) {
1252 dev_err(adev->dev, "failed to flush HDP via KIQ timeout\n");
1253 return -ETIMEDOUT;
1254 }
1255
1256 return 0;
1257
1258failed_undo:
1259 amdgpu_ring_undo(ring);
1260failed_unlock:
1261 spin_unlock_irqrestore(&kiq->ring_lock, flags);
1262failed_kiq_hdp_flush:
1263 dev_err(adev->dev, "failed to flush HDP via KIQ\n");
1264 return r < 0 ? r : -EIO;
1265}
1266
1197int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev)1267int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev)
1198{1268{
1199 if (amdgpu_num_kcq == -1) {1269 if (amdgpu_num_kcq == -1) {
@@ -1600,7 +1670,6 @@ static ssize_t amdgpu_gfx_set_run_cleaner_shader(struct device *dev, · 400 unmodified lines
1600 1670
1601 ret = amdgpu_gfx_run_cleaner_shader(adev, value);1671 ret = amdgpu_gfx_run_cleaner_shader(adev, value);
1602 1672
1603 pm_runtime_mark_last_busy(ddev->dev);
1604 pm_runtime_put_autosuspend(ddev->dev);1673 pm_runtime_put_autosuspend(ddev->dev);
1605 1674
1606 if (ret)1675 if (ret)
@@ -2485,3 +2554,4 @@ void amdgpu_debugfs_compute_sched_mask_init(struct amdgpu_device *adev) · 878 unmodified lines
2485 &amdgpu_debugfs_compute_sched_mask_fops);2554 &amdgpu_debugfs_compute_sched_mask_fops);
2486#endif2555#endif
2487}2556}
2557
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h+1−0
@@ -615,6 +615,7 @@ int amdgpu_gfx_cp_ecc_error_irq(struct amdgpu_device *adev, · 614 unmodified lines
615 struct amdgpu_iv_entry *entry);615 struct amdgpu_iv_entry *entry);
616uint32_t amdgpu_kiq_rreg(struct amdgpu_device *adev, uint32_t reg, uint32_t xcc_id);616uint32_t amdgpu_kiq_rreg(struct amdgpu_device *adev, uint32_t reg, uint32_t xcc_id);
617void amdgpu_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v, uint32_t xcc_id);617void amdgpu_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v, uint32_t xcc_id);
618int amdgpu_kiq_hdp_flush(struct amdgpu_device *adev);
618int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev);619int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev);
619void amdgpu_gfx_cp_init_microcode(struct amdgpu_device *adev, uint32_t ucode_id);620void amdgpu_gfx_cp_init_microcode(struct amdgpu_device *adev, uint32_t ucode_id);
620 621
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h+5−0
@@ -86,6 +86,11 @@ enum amdgpu_memory_partition { · 85 unmodified lines
86 86
87#define AMDGPU_MAX_MEM_RANGES 887#define AMDGPU_MAX_MEM_RANGES 8
88 88
89#define AMDGPU_GMC9_FAULT_SOURCE_DATA_RETRY 0x80
90#define AMDGPU_GMC9_FAULT_SOURCE_DATA_READ 0x40
91#define AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE 0x20
92#define AMDGPU_GMC9_FAULT_SOURCE_DATA_EXE 0x10
93
89/*94/*
90 * GMC page fault information95 * GMC page fault information
91 */96 */
drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c+1−0
@@ -284,6 +284,7 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) · 283 unmodified lines
284 ttm_resource_manager_init(man, &adev->mman.bdev, gtt_size);284 ttm_resource_manager_init(man, &adev->mman.bdev, gtt_size);
285 285
286 start = AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS;286 start = AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS;
287 start += amdgpu_vce_required_gart_pages(adev);
287 size = (adev->gmc.gart_size >> PAGE_SHIFT) - start;288 size = (adev->gmc.gart_size >> PAGE_SHIFT) - start;
288 drm_mm_init(&mgr->mm, start, size);289 drm_mm_init(&mgr->mm, start, size);
289 spin_lock_init(&mgr->lock);290 spin_lock_init(&mgr->lock);
drivers/gpu/drm/amd/amdgpu/amdgpu_hdp.c+16−0
@@ -66,3 +66,19 @@ void amdgpu_hdp_generic_flush(struct amdgpu_device *adev, · 65 unmodified lines
66 0);66 0);
67 }67 }
68}68}
69
70void amdgpu_hdp_invalidate(struct amdgpu_device *adev, struct amdgpu_ring *ring)
71{
72 if (adev->asic_funcs && adev->asic_funcs->invalidate_hdp)
73 adev->asic_funcs->invalidate_hdp(adev, ring);
74 else if (adev->hdp.funcs && adev->hdp.funcs->invalidate_hdp)
75 adev->hdp.funcs->invalidate_hdp(adev, ring);
76}
77
78void amdgpu_hdp_flush(struct amdgpu_device *adev, struct amdgpu_ring *ring)
79{
80 if (adev->asic_funcs && adev->asic_funcs->flush_hdp)
81 adev->asic_funcs->flush_hdp(adev, ring);
82 else if (adev->hdp.funcs && adev->hdp.funcs->flush_hdp)
83 adev->hdp.funcs->flush_hdp(adev, ring);
84}
drivers/gpu/drm/amd/amdgpu/amdgpu_hdp.h+4−0
@@ -46,4 +46,8 @@ struct amdgpu_hdp { · 45 unmodified lines
46int amdgpu_hdp_ras_sw_init(struct amdgpu_device *adev);46int amdgpu_hdp_ras_sw_init(struct amdgpu_device *adev);
47void amdgpu_hdp_generic_flush(struct amdgpu_device *adev,47void amdgpu_hdp_generic_flush(struct amdgpu_device *adev,
48 struct amdgpu_ring *ring);48 struct amdgpu_ring *ring);
49void amdgpu_hdp_invalidate(struct amdgpu_device *adev,
50 struct amdgpu_ring *ring);
51void amdgpu_hdp_flush(struct amdgpu_device *adev,
52 struct amdgpu_ring *ring);
49#endif /* __AMDGPU_HDP_H__ */53#endif /* __AMDGPU_HDP_H__ */
drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c+62−17
@@ -168,17 +168,13 @@ void amdgpu_hmm_unregister(struct amdgpu_bo *bo) · 167 unmodified lines
168int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier,168int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier,
169 uint64_t start, uint64_t npages, bool readonly,169 uint64_t start, uint64_t npages, bool readonly,
170 void *owner,170 void *owner,
171 struct hmm_range **phmm_range)171 struct amdgpu_hmm_range *range)
172{172{
173 struct hmm_range *hmm_range;
174 unsigned long end;173 unsigned long end;
175 unsigned long timeout;174 unsigned long timeout;
176 unsigned long *pfns;175 unsigned long *pfns;
177 int r = 0;176 int r = 0;
178 177 struct hmm_range *hmm_range = &range->hmm_range;
179 hmm_range = kzalloc(sizeof(*hmm_range), GFP_KERNEL);
180 if (unlikely(!hmm_range))
181 return -ENOMEM;
182 178
183 pfns = kvmalloc_array(npages, sizeof(*pfns), GFP_KERNEL);179 pfns = kvmalloc_array(npages, sizeof(*pfns), GFP_KERNEL);
184 if (unlikely(!pfns)) {180 if (unlikely(!pfns)) {
@@ -221,28 +217,77 @@ retry: · 36 unmodified lines
221 hmm_range->start = start;217 hmm_range->start = start;
222 hmm_range->hmm_pfns = pfns;218 hmm_range->hmm_pfns = pfns;
223 219
224 *phmm_range = hmm_range;
225
226 return 0;220 return 0;
227 221
228out_free_pfns:222out_free_pfns:
229 kvfree(pfns);223 kvfree(pfns);
224 hmm_range->hmm_pfns = NULL;
230out_free_range:225out_free_range:
231 kfree(hmm_range);
232
233 if (r == -EBUSY)226 if (r == -EBUSY)
234 r = -EAGAIN;227 r = -EAGAIN;
235 return r;228 return r;
236}229}
237 230
238bool amdgpu_hmm_range_get_pages_done(struct hmm_range *hmm_range)231/**
232 * amdgpu_hmm_range_valid - check if an HMM range is still valid
233 * @range: pointer to the &struct amdgpu_hmm_range to validate
234 *
235 * Determines whether the given HMM range @range is still valid by
236 * checking for invalidations via the MMU notifier sequence. This is
237 * typically used to verify that the range has not been invalidated
238 * by concurrent address space updates before it is accessed.
239 *
240 * Return:
241 * * true if @range is valid and can be used safely
242 * * false if @range is NULL or has been invalidated
243 */
244bool amdgpu_hmm_range_valid(struct amdgpu_hmm_range *range)
239{245{
240 bool r;246 if (!range)
247 return false;
241 248
242 r = mmu_interval_read_retry(hmm_range->notifier,249 return !mmu_interval_read_retry(range->hmm_range.notifier,
243 hmm_range->notifier_seq);250 range->hmm_range.notifier_seq);
244 kvfree(hmm_range->hmm_pfns);251}
245 kfree(hmm_range);
246 252
247 return r;253/**
254 * amdgpu_hmm_range_alloc - allocate and initialize an AMDGPU HMM range
255 * @bo: optional buffer object to associate with this HMM range
256 *
257 * Allocates memory for amdgpu_hmm_range and associates it with the @bo passed.
258 * The reference count of the @bo is incremented.
259 *
260 * Return:
261 * Pointer to a newly allocated struct amdgpu_hmm_range on success,
262 * or NULL if memory allocation fails.
263 */
264struct amdgpu_hmm_range *amdgpu_hmm_range_alloc(struct amdgpu_bo *bo)
265{
266 struct amdgpu_hmm_range *range;
267
268 range = kzalloc(sizeof(*range), GFP_KERNEL);
269 if (!range)
270 return NULL;
271
272 range->bo = amdgpu_bo_ref(bo);
273 return range;
274}
275
276/**
277 * amdgpu_hmm_range_free - release an AMDGPU HMM range
278 * @range: pointer to the range object to free
279 *
280 * Releases all resources held by @range, including the associated
281 * hmm_pfns and the dropping reference of associated bo if any.
282 *
283 * Return: void
284 */
285void amdgpu_hmm_range_free(struct amdgpu_hmm_range *range)
286{
287 if (!range)
288 return;
289
290 kvfree(range->hmm_range.hmm_pfns);
291 amdgpu_bo_unref(&range->bo);
292 kfree(range);
248}293}
drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.h+22−2
@@ -31,13 +31,20 @@ · 30 unmodified lines
31#include <linux/interval_tree.h>31#include <linux/interval_tree.h>
32#include <linux/mmu_notifier.h>32#include <linux/mmu_notifier.h>
33 33
34struct amdgpu_hmm_range {
35 struct hmm_range hmm_range;
36 struct amdgpu_bo *bo;
37};
38
34int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier,39int amdgpu_hmm_range_get_pages(struct mmu_interval_notifier *notifier,
35 uint64_t start, uint64_t npages, bool readonly,40 uint64_t start, uint64_t npages, bool readonly,
36 void *owner,41 void *owner,
37 struct hmm_range **phmm_range);42 struct amdgpu_hmm_range *range);
38bool amdgpu_hmm_range_get_pages_done(struct hmm_range *hmm_range);
39 43
40#if defined(CONFIG_HMM_MIRROR)44#if defined(CONFIG_HMM_MIRROR)
45bool amdgpu_hmm_range_valid(struct amdgpu_hmm_range *range);
46struct amdgpu_hmm_range *amdgpu_hmm_range_alloc(struct amdgpu_bo *bo);
47void amdgpu_hmm_range_free(struct amdgpu_hmm_range *range);
41int amdgpu_hmm_register(struct amdgpu_bo *bo, unsigned long addr);48int amdgpu_hmm_register(struct amdgpu_bo *bo, unsigned long addr);
42void amdgpu_hmm_unregister(struct amdgpu_bo *bo);49void amdgpu_hmm_unregister(struct amdgpu_bo *bo);
43#else50#else
@@ -47,7 +54,20 @@ static inline int amdgpu_hmm_register(struct amdgpu_bo *bo, unsigned long addr) · 3 unmodified lines
47 "add CONFIG_ZONE_DEVICE=y in config file to fix this\n");54 "add CONFIG_ZONE_DEVICE=y in config file to fix this\n");
48 return -ENODEV;55 return -ENODEV;
49}56}
57
50static inline void amdgpu_hmm_unregister(struct amdgpu_bo *bo) {}58static inline void amdgpu_hmm_unregister(struct amdgpu_bo *bo) {}
59
60static inline bool amdgpu_hmm_range_valid(struct amdgpu_hmm_range *range)
61{
62 return false;
63}
64
65static inline struct amdgpu_hmm_range *amdgpu_hmm_range_alloc(struct amdgpu_bo *bo)
66{
67 return NULL;
68}
69
70static inline void amdgpu_hmm_range_free(struct amdgpu_hmm_range *range) {}
51#endif71#endif
52 72
53#endif73#endif
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c+26−10
@@ -149,17 +149,19 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs, · 148 unmodified lines
149 if (job) {149 if (job) {
150 vm = job->vm;150 vm = job->vm;
151 fence_ctx = job->base.s_fence ?151 fence_ctx = job->base.s_fence ?
152 job->base.s_fence->scheduled.context : 0;152 job->base.s_fence->finished.context : 0;
153 shadow_va = job->shadow_va;153 shadow_va = job->shadow_va;
154 csa_va = job->csa_va;154 csa_va = job->csa_va;
155 gds_va = job->gds_va;155 gds_va = job->gds_va;
156 init_shadow = job->init_shadow;156 init_shadow = job->init_shadow;
157 af = &job->hw_fence;157 af = job->hw_fence;
158 /* Save the context of the job for reset handling.158 /* Save the context of the job for reset handling.
159 * The driver needs this so it can skip the ring159 * The driver needs this so it can skip the ring
160 * contents for guilty contexts.160 * contents for guilty contexts.
161 */161 */
162 af->context = job->base.s_fence ? job->base.s_fence->finished.context : 0;162 af->context = fence_ctx;
163 /* the vm fence is also part of the job's context */
164 job->hw_vm_fence->context = fence_ctx;
163 } else {165 } else {
164 vm = NULL;166 vm = NULL;
165 fence_ctx = 0;167 fence_ctx = 0;
@@ -167,23 +169,28 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs, · 1 unmodified line
167 csa_va = 0;169 csa_va = 0;
168 gds_va = 0;170 gds_va = 0;
169 init_shadow = false;171 init_shadow = false;
170 af = NULL;172 af = kzalloc(sizeof(*af), GFP_ATOMIC);
173 if (!af)
174 return -ENOMEM;
171 }175 }
172 176
173 if (!ring->sched.ready) {177 if (!ring->sched.ready) {
174 dev_err(adev->dev, "couldn't schedule ib on ring <%s>\n", ring->name);178 dev_err(adev->dev, "couldn't schedule ib on ring <%s>\n", ring->name);
175 return -EINVAL;179 r = -EINVAL;
180 goto free_fence;
176 }181 }
177 182
178 if (vm && !job->vmid) {183 if (vm && !job->vmid) {
179 dev_err(adev->dev, "VM IB without ID\n");184 dev_err(adev->dev, "VM IB without ID\n");
180 return -EINVAL;185 r = -EINVAL;
186 goto free_fence;
181 }187 }
182 188
183 if ((ib->flags & AMDGPU_IB_FLAGS_SECURE) &&189 if ((ib->flags & AMDGPU_IB_FLAGS_SECURE) &&
184 (!ring->funcs->secure_submission_supported)) {190 (!ring->funcs->secure_submission_supported)) {
185 dev_err(adev->dev, "secure submissions not supported on ring <%s>\n", ring->name);191 dev_err(adev->dev, "secure submissions not supported on ring <%s>\n", ring->name);
186 return -EINVAL;192 r = -EINVAL;
193 goto free_fence;
187 }194 }
188 195
189 alloc_size = ring->funcs->emit_frame_size + num_ibs *196 alloc_size = ring->funcs->emit_frame_size + num_ibs *
@@ -192,7 +199,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs, · 2 unmodified lines
192 r = amdgpu_ring_alloc(ring, alloc_size);199 r = amdgpu_ring_alloc(ring, alloc_size);
193 if (r) {200 if (r) {
194 dev_err(adev->dev, "scheduling IB failed (%d).\n", r);201 dev_err(adev->dev, "scheduling IB failed (%d).\n", r);
195 return r;202 goto free_fence;
196 }203 }
197 204
198 need_ctx_switch = ring->current_ctx != fence_ctx;205 need_ctx_switch = ring->current_ctx != fence_ctx;
@@ -289,7 +296,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs, · 90 unmodified lines
289 amdgpu_ring_init_cond_exec(ring, ring->cond_exe_gpu_addr);296 amdgpu_ring_init_cond_exec(ring, ring->cond_exe_gpu_addr);
290 }297 }
291 298
292 r = amdgpu_fence_emit(ring, f, af, fence_flags);299 r = amdgpu_fence_emit(ring, af, fence_flags);
293 if (r) {300 if (r) {
294 dev_err(adev->dev, "failed to emit fence (%d)\n", r);301 dev_err(adev->dev, "failed to emit fence (%d)\n", r);
295 if (job && job->vmid)302 if (job && job->vmid)
@@ -297,6 +304,10 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs, · 1 unmodified line
297 amdgpu_ring_undo(ring);304 amdgpu_ring_undo(ring);
298 return r;305 return r;
299 }306 }
307 *f = &af->base;
308 /* get a ref for the job */
309 if (job)
310 dma_fence_get(*f);
300 311
301 if (ring->funcs->insert_end)312 if (ring->funcs->insert_end)
302 ring->funcs->insert_end(ring);313 ring->funcs->insert_end(ring);
@@ -317,12 +328,17 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs, · 14 unmodified lines
317 * fence so we know what rings contents to backup328 * fence so we know what rings contents to backup
318 * after we reset the queue.329 * after we reset the queue.
319 */330 */
320 amdgpu_fence_save_wptr(*f);331 amdgpu_fence_save_wptr(af);
321 332
322 amdgpu_ring_ib_end(ring);333 amdgpu_ring_ib_end(ring);
323 amdgpu_ring_commit(ring);334 amdgpu_ring_commit(ring);
324 335
325 return 0;336 return 0;
337
338free_fence:
339 if (!job)
340 kfree(af);
341 return r;
326}342}
327 343
328/**344/**
drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c+17−41
@@ -201,58 +201,34 @@ static int amdgpu_vmid_grab_idle(struct amdgpu_ring *ring, · 200 unmodified lines
201 struct amdgpu_device *adev = ring->adev;201 struct amdgpu_device *adev = ring->adev;
202 unsigned vmhub = ring->vm_hub;202 unsigned vmhub = ring->vm_hub;
203 struct amdgpu_vmid_mgr *id_mgr = &adev->vm_manager.id_mgr[vmhub];203 struct amdgpu_vmid_mgr *id_mgr = &adev->vm_manager.id_mgr[vmhub];
204 struct dma_fence **fences;
205 unsigned i;
206 204
205 /* If anybody is waiting for a VMID let everybody wait for fairness */
207 if (!dma_fence_is_signaled(ring->vmid_wait)) {206 if (!dma_fence_is_signaled(ring->vmid_wait)) {
208 *fence = dma_fence_get(ring->vmid_wait);207 *fence = dma_fence_get(ring->vmid_wait);
209 return 0;208 return 0;
210 }209 }
211 210
212 fences = kmalloc_array(id_mgr->num_ids, sizeof(void *), GFP_NOWAIT);
213 if (!fences)
214 return -ENOMEM;
215
216 /* Check if we have an idle VMID */211 /* Check if we have an idle VMID */
217 i = 0;212 list_for_each_entry_reverse((*idle), &id_mgr->ids_lru, list) {
218 list_for_each_entry((*idle), &id_mgr->ids_lru, list) {
219 /* Don't use per engine and per process VMID at the same time */213 /* Don't use per engine and per process VMID at the same time */
220 struct amdgpu_ring *r = adev->vm_manager.concurrent_flush ?214 struct amdgpu_ring *r = adev->vm_manager.concurrent_flush ?
221 NULL : ring;215 NULL : ring;
222 216
223 fences[i] = amdgpu_sync_peek_fence(&(*idle)->active, r);217 *fence = amdgpu_sync_peek_fence(&(*idle)->active, r);
224 if (!fences[i])218 if (!(*fence))
225 break;219 return 0;
226 ++i;
227 }220 }
228 221
229 /* If we can't find a idle VMID to use, wait till one becomes available */222 /*
230 if (&(*idle)->list == &id_mgr->ids_lru) {223 * If we can't find a idle VMID to use, wait on a fence from the least
231 u64 fence_context = adev->vm_manager.fence_context + ring->idx;224 * recently used in the hope that it will be available soon.
232 unsigned seqno = ++adev->vm_manager.seqno[ring->idx];225 */
233 struct dma_fence_array *array;226 *idle = NULL;
234 unsigned j;227 dma_fence_put(ring->vmid_wait);
235 228 ring->vmid_wait = dma_fence_get(*fence);
236 *idle = NULL;
237 for (j = 0; j < i; ++j)
238 dma_fence_get(fences[j]);
239
240 array = dma_fence_array_create(i, fences, fence_context,
241 seqno, true);
242 if (!array) {
243 for (j = 0; j < i; ++j)
244 dma_fence_put(fences[j]);
245 kfree(fences);
246 return -ENOMEM;
247 }
248
249 *fence = dma_fence_get(&array->base);
250 dma_fence_put(ring->vmid_wait);
251 ring->vmid_wait = &array->base;
252 return 0;
253 }
254 kfree(fences);
255 229
230 /* This is the reference we return */
231 dma_fence_get(*fence);
256 return 0;232 return 0;
257}233}
258 234
@@ -313,7 +289,7 @@ static int amdgpu_vmid_grab_reserved(struct amdgpu_vm *vm, · 54 unmodified lines
313 * user of the VMID.289 * user of the VMID.
314 */290 */
315 r = amdgpu_sync_fence(&(*id)->active, &job->base.s_fence->finished,291 r = amdgpu_sync_fence(&(*id)->active, &job->base.s_fence->finished,
316 GFP_NOWAIT);292 GFP_ATOMIC);
317 if (r)293 if (r)
318 return r;294 return r;
319 295
@@ -373,7 +349,7 @@ static int amdgpu_vmid_grab_used(struct amdgpu_vm *vm, · 53 unmodified lines
373 */349 */
374 r = amdgpu_sync_fence(&(*id)->active,350 r = amdgpu_sync_fence(&(*id)->active,
375 &job->base.s_fence->finished,351 &job->base.s_fence->finished,
376 GFP_NOWAIT);352 GFP_ATOMIC);
377 if (r)353 if (r)
378 return r;354 return r;
379 355
@@ -426,7 +402,7 @@ int amdgpu_vmid_grab(struct amdgpu_vm *vm, struct amdgpu_ring *ring, · 46 unmodified lines
426 /* Remember this submission as user of the VMID */402 /* Remember this submission as user of the VMID */
427 r = amdgpu_sync_fence(&id->active,403 r = amdgpu_sync_fence(&id->active,
428 &job->base.s_fence->finished,404 &job->base.s_fence->finished,
429 GFP_NOWAIT);405 GFP_ATOMIC);
430 if (r)406 if (r)
431 goto error;407 goto error;
432 408
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c+53−16
@@ -130,14 +130,12 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) · 129 unmodified lines
130 }130 }
131 131
132 /* attempt a per ring reset */132 /* attempt a per ring reset */
133 if (unlikely(adev->debug_disable_gpu_ring_reset)) {133 if (amdgpu_gpu_recovery &&
134 dev_err(adev->dev, "Ring reset disabled by debug mask\n");134 amdgpu_ring_is_reset_type_supported(ring, AMDGPU_RESET_TYPE_PER_QUEUE) &&
135 } else if (amdgpu_gpu_recovery &&135 ring->funcs->reset) {
136 amdgpu_ring_is_reset_type_supported(ring, AMDGPU_RESET_TYPE_PER_QUEUE) &&
137 ring->funcs->reset) {
138 dev_err(adev->dev, "Starting %s ring reset\n",136 dev_err(adev->dev, "Starting %s ring reset\n",
139 s_job->sched->name);137 s_job->sched->name);
140 r = amdgpu_ring_reset(ring, job->vmid, &job->hw_fence);138 r = amdgpu_ring_reset(ring, job->vmid, job->hw_fence);
141 if (!r) {139 if (!r) {
142 atomic_inc(&ring->adev->gpu_reset_counter);140 atomic_inc(&ring->adev->gpu_reset_counter);
143 dev_err(adev->dev, "Ring %s reset succeeded\n",141 dev_err(adev->dev, "Ring %s reset succeeded\n",
@@ -186,6 +184,9 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 42 unmodified lines
186 unsigned int num_ibs, struct amdgpu_job **job,184 unsigned int num_ibs, struct amdgpu_job **job,
187 u64 drm_client_id)185 u64 drm_client_id)
188{186{
187 struct amdgpu_fence *af;
188 int r;
189
189 if (num_ibs == 0)190 if (num_ibs == 0)
190 return -EINVAL;191 return -EINVAL;
191 192
@@ -193,6 +194,20 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 1 unmodified line
193 if (!*job)194 if (!*job)
194 return -ENOMEM;195 return -ENOMEM;
195 196
197 af = kzalloc(sizeof(struct amdgpu_fence), GFP_KERNEL);
198 if (!af) {
199 r = -ENOMEM;
200 goto err_job;
201 }
202 (*job)->hw_fence = af;
203
204 af = kzalloc(sizeof(struct amdgpu_fence), GFP_KERNEL);
205 if (!af) {
206 r = -ENOMEM;
207 goto err_fence;
208 }
209 (*job)->hw_vm_fence = af;
210
196 (*job)->vm = vm;211 (*job)->vm = vm;
197 212
198 amdgpu_sync_create(&(*job)->explicit_sync);213 amdgpu_sync_create(&(*job)->explicit_sync);
@@ -204,6 +219,14 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 5 unmodified lines
204 219
205 return drm_sched_job_init(&(*job)->base, entity, 1, owner,220 return drm_sched_job_init(&(*job)->base, entity, 1, owner,
206 drm_client_id);221 drm_client_id);
222
223err_fence:
224 kfree((*job)->hw_fence);
225err_job:
226 kfree(*job);
227 *job = NULL;
228
229 return r;
207}230}
208 231
209int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev,232int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev,
@@ -223,7 +246,10 @@ int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, · 13 unmodified lines
223 if (r) {246 if (r) {
224 if (entity)247 if (entity)
225 drm_sched_job_cleanup(&(*job)->base);248 drm_sched_job_cleanup(&(*job)->base);
249 kfree((*job)->hw_vm_fence);
250 kfree((*job)->hw_fence);
226 kfree(*job);251 kfree(*job);
252 *job = NULL;
227 }253 }
228 254
229 return r;255 return r;
@@ -251,11 +277,11 @@ void amdgpu_job_free_resources(struct amdgpu_job *job) · 21 unmodified lines
251 struct dma_fence *f;277 struct dma_fence *f;
252 unsigned i;278 unsigned i;
253 279
254 /* Check if any fences where initialized */280 /* Check if any fences were initialized */
255 if (job->base.s_fence && job->base.s_fence->finished.ops)281 if (job->base.s_fence && job->base.s_fence->finished.ops)
256 f = &job->base.s_fence->finished;282 f = &job->base.s_fence->finished;
257 else if (job->hw_fence.base.ops)283 else if (job->hw_fence && job->hw_fence->base.ops)
258 f = &job->hw_fence.base;284 f = &job->hw_fence->base;
259 else285 else
260 f = NULL;286 f = NULL;
261 287
@@ -271,11 +297,16 @@ static void amdgpu_job_free_cb(struct drm_sched_job *s_job) · 9 unmodified lines
271 297
272 amdgpu_sync_free(&job->explicit_sync);298 amdgpu_sync_free(&job->explicit_sync);
273 299
274 /* only put the hw fence if has embedded fence */300 if (job->hw_fence->base.ops)
275 if (!job->hw_fence.base.ops)301 dma_fence_put(&job->hw_fence->base);
276 kfree(job);302 else
303 kfree(job->hw_fence);
304 if (job->hw_vm_fence->base.ops)
305 dma_fence_put(&job->hw_vm_fence->base);
277 else306 else
278 dma_fence_put(&job->hw_fence.base);307 kfree(job->hw_vm_fence);
308
309 kfree(job);
279}310}
280 311
281void amdgpu_job_set_gang_leader(struct amdgpu_job *job,312void amdgpu_job_set_gang_leader(struct amdgpu_job *job,
@@ -304,10 +335,16 @@ void amdgpu_job_free(struct amdgpu_job *job) · 22 unmodified lines
304 if (job->gang_submit != &job->base.s_fence->scheduled)335 if (job->gang_submit != &job->base.s_fence->scheduled)
305 dma_fence_put(job->gang_submit);336 dma_fence_put(job->gang_submit);
306 337
307 if (!job->hw_fence.base.ops)338 if (job->hw_fence->base.ops)
308 kfree(job);339 dma_fence_put(&job->hw_fence->base);
340 else
341 kfree(job->hw_fence);
342 if (job->hw_vm_fence->base.ops)
343 dma_fence_put(&job->hw_vm_fence->base);
309 else344 else
310 dma_fence_put(&job->hw_fence.base);345 kfree(job->hw_vm_fence);
346
347 kfree(job);
311}348}
312 349
313struct dma_fence *amdgpu_job_submit(struct amdgpu_job *job)350struct dma_fence *amdgpu_job_submit(struct amdgpu_job *job)
drivers/gpu/drm/amd/amdgpu/amdgpu_job.h+2−1
@@ -64,7 +64,8 @@ struct amdgpu_job { · 63 unmodified lines
64 struct drm_sched_job base;64 struct drm_sched_job base;
65 struct amdgpu_vm *vm;65 struct amdgpu_vm *vm;
66 struct amdgpu_sync explicit_sync;66 struct amdgpu_sync explicit_sync;
67 struct amdgpu_fence hw_fence;67 struct amdgpu_fence *hw_fence;
68 struct amdgpu_fence *hw_vm_fence;
68 struct dma_fence *gang_submit;69 struct dma_fence *gang_submit;
69 uint32_t preamble_status;70 uint32_t preamble_status;
70 uint32_t preemption_status;71 uint32_t preemption_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c+0−2
@@ -1471,7 +1471,6 @@ error_pasid: · 1470 unmodified lines
1471 kfree(fpriv);1471 kfree(fpriv);
1472 1472
1473out_suspend:1473out_suspend:
1474 pm_runtime_mark_last_busy(dev->dev);
1475pm_put:1474pm_put:
1476 pm_runtime_put_autosuspend(dev->dev);1475 pm_runtime_put_autosuspend(dev->dev);
1477 1476
@@ -1539,7 +1538,6 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, · 61 unmodified lines
1539 kfree(fpriv);1538 kfree(fpriv);
1540 file_priv->driver_priv = NULL;1539 file_priv->driver_priv = NULL;
1541 1540
1542 pm_runtime_mark_last_busy(dev->dev);
1543 pm_runtime_put_autosuspend(dev->dev);1541 pm_runtime_put_autosuspend(dev->dev);
1544}1542}
1545 1543
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c+14−2
@@ -105,8 +105,8 @@ int amdgpu_mes_init(struct amdgpu_device *adev) · 104 unmodified lines
105 spin_lock_init(&adev->mes.ring_lock[i]);105 spin_lock_init(&adev->mes.ring_lock[i]);
106 106
107 adev->mes.total_max_queue = AMDGPU_FENCE_MES_QUEUE_ID_MASK;107 adev->mes.total_max_queue = AMDGPU_FENCE_MES_QUEUE_ID_MASK;
108 adev->mes.vmid_mask_mmhub = 0xffffff00;108 adev->mes.vmid_mask_mmhub = 0xFF00;
109 adev->mes.vmid_mask_gfxhub = adev->gfx.disable_kq ? 0xfffffffe : 0xffffff00;109 adev->mes.vmid_mask_gfxhub = adev->gfx.disable_kq ? 0xFFFE : 0xFF00;
110 110
111 num_pipes = adev->gfx.me.num_pipe_per_me * adev->gfx.me.num_me;111 num_pipes = adev->gfx.me.num_pipe_per_me * adev->gfx.me.num_me;
112 if (num_pipes > AMDGPU_MES_MAX_GFX_PIPES)112 if (num_pipes > AMDGPU_MES_MAX_GFX_PIPES)
@@ -528,6 +528,18 @@ error: · 415 unmodified lines
528 return r;528 return r;
529}529}
530 530
531int amdgpu_mes_hdp_flush(struct amdgpu_device *adev)
532{
533 uint32_t hdp_flush_req_offset, hdp_flush_done_offset, ref_and_mask;
534
535 hdp_flush_req_offset = adev->nbio.funcs->get_hdp_flush_req_offset(adev);
536 hdp_flush_done_offset = adev->nbio.funcs->get_hdp_flush_done_offset(adev);
537 ref_and_mask = adev->nbio.hdp_flush_reg->ref_and_mask_cp0;
538
539 return amdgpu_mes_reg_write_reg_wait(adev, hdp_flush_req_offset, hdp_flush_done_offset,
540 ref_and_mask, ref_and_mask);
541}
542
531int amdgpu_mes_set_shader_debugger(struct amdgpu_device *adev,543int amdgpu_mes_set_shader_debugger(struct amdgpu_device *adev,
532 uint64_t process_context_addr,544 uint64_t process_context_addr,
533 uint32_t spi_gdbg_per_vmid_cntl,545 uint32_t spi_gdbg_per_vmid_cntl,
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h+2−0
@@ -239,6 +239,7 @@ struct mes_add_queue_input { · 238 unmodified lines
239struct mes_remove_queue_input {239struct mes_remove_queue_input {
240 uint32_t doorbell_offset;240 uint32_t doorbell_offset;
241 uint64_t gang_context_addr;241 uint64_t gang_context_addr;
242 bool remove_queue_after_reset;
242};243};
243 244
244struct mes_map_legacy_queue_input {245struct mes_map_legacy_queue_input {
@@ -428,6 +429,7 @@ int amdgpu_mes_wreg(struct amdgpu_device *adev, · 183 unmodified lines
428int amdgpu_mes_reg_write_reg_wait(struct amdgpu_device *adev,429int amdgpu_mes_reg_write_reg_wait(struct amdgpu_device *adev,
429 uint32_t reg0, uint32_t reg1,430 uint32_t reg0, uint32_t reg1,
430 uint32_t ref, uint32_t mask);431 uint32_t ref, uint32_t mask);
432int amdgpu_mes_hdp_flush(struct amdgpu_device *adev);
431int amdgpu_mes_set_shader_debugger(struct amdgpu_device *adev,433int amdgpu_mes_set_shader_debugger(struct amdgpu_device *adev,
432 uint64_t process_context_addr,434 uint64_t process_context_addr,
433 uint32_t spi_gdbg_per_vmid_cntl,435 uint32_t spi_gdbg_per_vmid_cntl,
drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h+2−0
@@ -326,6 +326,8 @@ struct amdgpu_mode_info { · 325 unmodified lines
326 struct drm_property *audio_property;326 struct drm_property *audio_property;
327 /* FMT dithering */327 /* FMT dithering */
328 struct drm_property *dither_property;328 struct drm_property *dither_property;
329 /* Adaptive Backlight Modulation (power feature) */
330 struct drm_property *abm_level_property;
329 /* hardcoded DFP edid from BIOS */331 /* hardcoded DFP edid from BIOS */
330 const struct drm_edid *bios_hardcoded_edid;332 const struct drm_edid *bios_hardcoded_edid;
331 333
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h+1−0
@@ -96,6 +96,7 @@ struct amdgpu_bo_va { · 95 unmodified lines
96 * if non-zero, cannot unmap from GPU because user queues may still access it96 * if non-zero, cannot unmap from GPU because user queues may still access it
97 */97 */
98 unsigned int queue_refcount;98 unsigned int queue_refcount;
99 atomic_t userq_va_mapped;
99};100};
100 101
101struct amdgpu_bo {102struct amdgpu_bo {
drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c+16−7
@@ -1539,6 +1539,7 @@ static void psp_xgmi_reflect_topology_info(struct psp_context *psp, · 1538 unmodified lines
1539 uint64_t src_node_id = psp->adev->gmc.xgmi.node_id;1539 uint64_t src_node_id = psp->adev->gmc.xgmi.node_id;
1540 uint64_t dst_node_id = node_info.node_id;1540 uint64_t dst_node_id = node_info.node_id;
1541 uint8_t dst_num_hops = node_info.num_hops;1541 uint8_t dst_num_hops = node_info.num_hops;
1542 uint8_t dst_is_sharing_enabled = node_info.is_sharing_enabled;
1542 uint8_t dst_num_links = node_info.num_links;1543 uint8_t dst_num_links = node_info.num_links;
1543 1544
1544 hive = amdgpu_get_xgmi_hive(psp->adev);1545 hive = amdgpu_get_xgmi_hive(psp->adev);
@@ -1558,13 +1559,20 @@ static void psp_xgmi_reflect_topology_info(struct psp_context *psp, · 13 unmodified lines
1558 continue;1559 continue;
1559 1560
1560 mirror_top_info->nodes[j].num_hops = dst_num_hops;1561 mirror_top_info->nodes[j].num_hops = dst_num_hops;
1561 /*1562 mirror_top_info->nodes[j].is_sharing_enabled = dst_is_sharing_enabled;
1562 * prevent 0 num_links value re-reflection since reflection1563 /* prevent 0 num_links value re-reflection since reflection
1563 * criteria is based on num_hops (direct or indirect).1564 * criteria is based on num_hops (direct or indirect).
1564 *
1565 */1565 */
1566 if (dst_num_links)1566 if (dst_num_links) {
1567 mirror_top_info->nodes[j].num_links = dst_num_links;1567 mirror_top_info->nodes[j].num_links = dst_num_links;
1568 /* swap src and dst due to frame of reference */
1569 for (int k = 0; k < dst_num_links; k++) {
1570 mirror_top_info->nodes[j].port_num[k].src_xgmi_port_num =
1571 node_info.port_num[k].dst_xgmi_port_num;
1572 mirror_top_info->nodes[j].port_num[k].dst_xgmi_port_num =
1573 node_info.port_num[k].src_xgmi_port_num;
1574 }
1575 }
1568 1576
1569 break;1577 break;
1570 }1578 }
@@ -1639,9 +1647,10 @@ int psp_xgmi_get_topology_info(struct psp_context *psp, · 68 unmodified lines
1639 amdgpu_ip_version(psp->adev, MP0_HWIP, 0) ==1647 amdgpu_ip_version(psp->adev, MP0_HWIP, 0) ==
1640 IP_VERSION(13, 0, 6) ||1648 IP_VERSION(13, 0, 6) ||
1641 amdgpu_ip_version(psp->adev, MP0_HWIP, 0) ==1649 amdgpu_ip_version(psp->adev, MP0_HWIP, 0) ==
1642 IP_VERSION(13, 0, 14);1650 IP_VERSION(13, 0, 14) ||
1643 bool ta_port_num_support = amdgpu_sriov_vf(psp->adev) ? 0 :1651 amdgpu_sriov_vf(psp->adev);
1644 psp->xgmi_context.xgmi_ta_caps & EXTEND_PEER_LINK_INFO_CMD_FLAG;1652 bool ta_port_num_support = psp->xgmi_context.xgmi_ta_caps & EXTEND_PEER_LINK_INFO_CMD_FLAG ||
1653 amdgpu_sriov_xgmi_ta_ext_peer_link_en(psp->adev);
1645 1654
1646 /* popluate the shared output buffer rather than the cmd input buffer1655 /* popluate the shared output buffer rather than the cmd input buffer
1647 * with node_ids as the input for GET_PEER_LINKS command execution.1656 * with node_ids as the input for GET_PEER_LINKS command execution.
drivers/gpu/drm/amd/amdgpu/amdgpu_rap.c+0−1
@@ -101,7 +101,6 @@ static ssize_t amdgpu_rap_debugfs_write(struct file *f, const char __user *buf, · 100 unmodified lines
101 }101 }
102 102
103 amdgpu_gfx_off_ctrl(adev, true);103 amdgpu_gfx_off_ctrl(adev, true);
104 pm_runtime_mark_last_busy(dev->dev);
105 pm_runtime_put_autosuspend(dev->dev);104 pm_runtime_put_autosuspend(dev->dev);
106 105
107 return size;106 return size;
drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c+322−66
@@ -41,6 +41,7 @@ · 40 unmodified lines
41#include "atom.h"41#include "atom.h"
42#include "amdgpu_reset.h"42#include "amdgpu_reset.h"
43#include "amdgpu_psp.h"43#include "amdgpu_psp.h"
44#include "amdgpu_ras_mgr.h"
44 45
45#ifdef CONFIG_X86_MCE_AMD46#ifdef CONFIG_X86_MCE_AMD
46#include <asm/mce.h>47#include <asm/mce.h>
@@ -149,6 +150,8 @@ static void amdgpu_ras_critical_region_fini(struct amdgpu_device *adev); · 102 unmodified lines
149 150
150#ifdef CONFIG_X86_MCE_AMD151#ifdef CONFIG_X86_MCE_AMD
151static void amdgpu_register_bad_pages_mca_notifier(struct amdgpu_device *adev);152static void amdgpu_register_bad_pages_mca_notifier(struct amdgpu_device *adev);
153static void
154amdgpu_unregister_bad_pages_mca_notifier(struct amdgpu_device *adev);
152struct mce_notifier_adev_list {155struct mce_notifier_adev_list {
153 struct amdgpu_device *devs[MAX_GPU_INSTANCE];156 struct amdgpu_device *devs[MAX_GPU_INSTANCE];
154 int num_gpu;157 int num_gpu;
@@ -611,6 +614,8 @@ static ssize_t amdgpu_ras_debugfs_ctrl_write(struct file *f, · 456 unmodified lines
611 return size;614 return size;
612}615}
613 616
617static int amdgpu_uniras_clear_badpages_info(struct amdgpu_device *adev);
618
614/**619/**
615 * DOC: AMDGPU RAS debugfs EEPROM table reset interface620 * DOC: AMDGPU RAS debugfs EEPROM table reset interface
616 *621 *
@@ -635,6 +640,11 @@ static ssize_t amdgpu_ras_debugfs_eeprom_write(struct file *f, · 18 unmodified lines
635 (struct amdgpu_device *)file_inode(f)->i_private;640 (struct amdgpu_device *)file_inode(f)->i_private;
636 int ret;641 int ret;
637 642
643 if (amdgpu_uniras_enabled(adev)) {
644 ret = amdgpu_uniras_clear_badpages_info(adev);
645 return ret ? ret : size;
646 }
647
638 ret = amdgpu_ras_eeprom_reset_table(648 ret = amdgpu_ras_eeprom_reset_table(
639 &(amdgpu_ras_get_context(adev)->eeprom_control));649 &(amdgpu_ras_get_context(adev)->eeprom_control));
640 650
@@ -1542,9 +1552,51 @@ out_fini_err_data: · 901 unmodified lines
1542 return ret;1552 return ret;
1543}1553}
1544 1554
1555static int amdgpu_uniras_clear_badpages_info(struct amdgpu_device *adev)
1556{
1557 struct ras_cmd_dev_handle req = {0};
1558 int ret;
1559
1560 ret = amdgpu_ras_mgr_handle_ras_cmd(adev, RAS_CMD__CLEAR_BAD_PAGE_INFO,
1561 &req, sizeof(req), NULL, 0);
1562 if (ret) {
1563 dev_err(adev->dev, "Failed to clear bad pages info, ret: %d\n", ret);
1564 return ret;
1565 }
1566
1567 return 0;
1568}
1569
1570static int amdgpu_uniras_query_block_ecc(struct amdgpu_device *adev,
1571 struct ras_query_if *info)
1572{
1573 struct ras_cmd_block_ecc_info_req req = {0};
1574 struct ras_cmd_block_ecc_info_rsp rsp = {0};
1575 int ret;
1576
1577 if (!info)
1578 return -EINVAL;
1579
1580 req.block_id = info->head.block;
1581 req.subblock_id = info->head.sub_block_index;
1582
1583 ret = amdgpu_ras_mgr_handle_ras_cmd(adev, RAS_CMD__GET_BLOCK_ECC_STATUS,
1584 &req, sizeof(req), &rsp, sizeof(rsp));
1585 if (!ret) {
1586 info->ce_count = rsp.ce_count;
1587 info->ue_count = rsp.ue_count;
1588 info->de_count = rsp.de_count;
1589 }
1590
1591 return ret;
1592}
1593
1545int amdgpu_ras_query_error_status(struct amdgpu_device *adev, struct ras_query_if *info)1594int amdgpu_ras_query_error_status(struct amdgpu_device *adev, struct ras_query_if *info)
1546{1595{
1547 return amdgpu_ras_query_error_status_with_event(adev, info, RAS_EVENT_TYPE_INVALID);1596 if (amdgpu_uniras_enabled(adev))
1597 return amdgpu_uniras_query_block_ecc(adev, info);
1598 else
1599 return amdgpu_ras_query_error_status_with_event(adev, info, RAS_EVENT_TYPE_INVALID);
1548}1600}
1549 1601
1550int amdgpu_ras_reset_error_count(struct amdgpu_device *adev,1602int amdgpu_ras_reset_error_count(struct amdgpu_device *adev,
@@ -1596,6 +1648,27 @@ int amdgpu_ras_reset_error_status(struct amdgpu_device *adev, · 45 unmodified lines
1596 return 0;1648 return 0;
1597}1649}
1598 1650
1651static int amdgpu_uniras_error_inject(struct amdgpu_device *adev,
1652 struct ras_inject_if *info)
1653{
1654 struct ras_cmd_inject_error_req inject_req;
1655 struct ras_cmd_inject_error_rsp rsp;
1656
1657 if (!info)
1658 return -EINVAL;
1659
1660 memset(&inject_req, 0, sizeof(inject_req));
1661 inject_req.block_id = info->head.block;
1662 inject_req.subblock_id = info->head.sub_block_index;
1663 inject_req.address = info->address;
1664 inject_req.error_type = info->head.type;
1665 inject_req.instance_mask = info->instance_mask;
1666 inject_req.method = info->value;
1667
1668 return amdgpu_ras_mgr_handle_ras_cmd(adev, RAS_CMD__INJECT_ERROR,
1669 &inject_req, sizeof(inject_req), &rsp, sizeof(rsp));
1670}
1671
1599/* wrapper of psp_ras_trigger_error */1672/* wrapper of psp_ras_trigger_error */
1600int amdgpu_ras_error_inject(struct amdgpu_device *adev,1673int amdgpu_ras_error_inject(struct amdgpu_device *adev,
1601 struct ras_inject_if *info)1674 struct ras_inject_if *info)
@@ -1613,6 +1686,9 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 11 unmodified lines
1613 info->head.block,1686 info->head.block,
1614 info->head.sub_block_index);1687 info->head.sub_block_index);
1615 1688
1689 if (amdgpu_uniras_enabled(adev))
1690 return amdgpu_uniras_error_inject(adev, info);
1691
1616 /* inject on guest isn't allowed, return success directly */1692 /* inject on guest isn't allowed, return success directly */
1617 if (amdgpu_sriov_vf(adev))1693 if (amdgpu_sriov_vf(adev))
1618 return 0;1694 return 0;
@@ -1757,7 +1833,9 @@ int amdgpu_ras_query_error_count(struct amdgpu_device *adev, · 138 unmodified lines
1757/* sysfs begin */1833/* sysfs begin */
1758 1834
1759static int amdgpu_ras_badpages_read(struct amdgpu_device *adev,1835static int amdgpu_ras_badpages_read(struct amdgpu_device *adev,
1760 struct ras_badpage **bps, unsigned int *count);1836 struct ras_badpage *bps, uint32_t count, uint32_t start);
1837static int amdgpu_uniras_badpages_read(struct amdgpu_device *adev,
1838 struct ras_badpage *bps, uint32_t count, uint32_t start);
1761 1839
1762static char *amdgpu_ras_badpage_flags_str(unsigned int flags)1840static char *amdgpu_ras_badpage_flags_str(unsigned int flags)
1763{1841{
@@ -1815,19 +1893,50 @@ static ssize_t amdgpu_ras_sysfs_badpages_read(struct file *f, · 51 unmodified lines
1815 unsigned int end = div64_ul(ppos + count - 1, element_size);1893 unsigned int end = div64_ul(ppos + count - 1, element_size);
1816 ssize_t s = 0;1894 ssize_t s = 0;
1817 struct ras_badpage *bps = NULL;1895 struct ras_badpage *bps = NULL;
1818 unsigned int bps_count = 0;1896 int bps_count = 0, i, status;
1897 uint64_t address;
1819 1898
1820 memset(buf, 0, count);1899 memset(buf, 0, count);
1821 1900
1822 if (amdgpu_ras_badpages_read(adev, &bps, &bps_count))1901 bps_count = end - start;
1902 bps = kmalloc_array(bps_count, sizeof(*bps), GFP_KERNEL);
1903 if (!bps)
1904 return 0;
1905
1906 memset(bps, 0, sizeof(*bps) * bps_count);
1907
1908 if (amdgpu_uniras_enabled(adev))
1909 bps_count = amdgpu_uniras_badpages_read(adev, bps, bps_count, start);
1910 else
1911 bps_count = amdgpu_ras_badpages_read(adev, bps, bps_count, start);
1912
1913 if (bps_count <= 0) {
1914 kfree(bps);
1823 return 0;1915 return 0;
1916 }
1917
1918 for (i = 0; i < bps_count; i++) {
1919 address = ((uint64_t)bps[i].bp) << AMDGPU_GPU_PAGE_SHIFT;
1920 if (amdgpu_ras_check_critical_address(adev, address))
1921 continue;
1922
1923 bps[i].size = AMDGPU_GPU_PAGE_SIZE;
1924
1925 status = amdgpu_vram_mgr_query_page_status(&adev->mman.vram_mgr,
1926 address);
1927 if (status == -EBUSY)
1928 bps[i].flags = AMDGPU_RAS_RETIRE_PAGE_PENDING;
1929 else if (status == -ENOENT)
1930 bps[i].flags = AMDGPU_RAS_RETIRE_PAGE_FAULT;
1931 else
1932 bps[i].flags = AMDGPU_RAS_RETIRE_PAGE_RESERVED;
1824 1933
1825 for (; start < end && start < bps_count; start++)
1826 s += scnprintf(&buf[s], element_size + 1,1934 s += scnprintf(&buf[s], element_size + 1,
1827 "0x%08x : 0x%08x : %1s\n",1935 "0x%08x : 0x%08x : %1s\n",
1828 bps[start].bp,1936 bps[i].bp,
1829 bps[start].size,1937 bps[i].size,
1830 amdgpu_ras_badpage_flags_str(bps[start].flags));1938 amdgpu_ras_badpage_flags_str(bps[i].flags));
1939 }
1831 1940
1832 kfree(bps);1941 kfree(bps);
1833 1942
@@ -1843,12 +1952,42 @@ static ssize_t amdgpu_ras_sysfs_features_read(struct device *dev, · 9 unmodified lines
1843 return sysfs_emit(buf, "feature mask: 0x%x\n", con->features);1952 return sysfs_emit(buf, "feature mask: 0x%x\n", con->features);
1844}1953}
1845 1954
1955static bool amdgpu_ras_get_version_info(struct amdgpu_device *adev, u32 *major,
1956 u32 *minor, u32 *rev)
1957{
1958 int i;
1959
1960 if (!adev || !major || !minor || !rev || !amdgpu_uniras_enabled(adev))
1961 return false;
1962
1963 for (i = 0; i < adev->num_ip_blocks; i++) {
1964 if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_RAS) {
1965 *major = adev->ip_blocks[i].version->major;
1966 *minor = adev->ip_blocks[i].version->minor;
1967 *rev = adev->ip_blocks[i].version->rev;
1968 return true;
1969 }
1970 }
1971
1972 return false;
1973}
1974
1846static ssize_t amdgpu_ras_sysfs_version_show(struct device *dev,1975static ssize_t amdgpu_ras_sysfs_version_show(struct device *dev,
1847 struct device_attribute *attr, char *buf)1976 struct device_attribute *attr, char *buf)
1848{1977{
1849 struct amdgpu_ras *con =1978 struct amdgpu_ras *con =
1850 container_of(attr, struct amdgpu_ras, version_attr);1979 container_of(attr, struct amdgpu_ras, version_attr);
1851 return sysfs_emit(buf, "table version: 0x%x\n", con->eeprom_control.tbl_hdr.version);1980 u32 major, minor, rev;
1981 ssize_t size = 0;
1982
1983 size += sysfs_emit_at(buf, size, "table version: 0x%x\n",
1984 con->eeprom_control.tbl_hdr.version);
1985
1986 if (amdgpu_ras_get_version_info(con->adev, &major, &minor, &rev))
1987 size += sysfs_emit_at(buf, size, "ras version: %u.%u.%u\n",
1988 major, minor, rev);
1989
1990 return size;
1852}1991}
1853 1992
1854static ssize_t amdgpu_ras_sysfs_schema_show(struct device *dev,1993static ssize_t amdgpu_ras_sysfs_schema_show(struct device *dev,
@@ -2241,6 +2380,11 @@ void amdgpu_ras_interrupt_fatal_error_handler(struct amdgpu_device *adev) · 386 unmodified lines
2241 amdgpu_ras_is_err_state(adev, AMDGPU_RAS_BLOCK__ANY))2380 amdgpu_ras_is_err_state(adev, AMDGPU_RAS_BLOCK__ANY))
2242 return;2381 return;
2243 2382
2383 if (amdgpu_uniras_enabled(adev)) {
2384 amdgpu_ras_mgr_handle_fatal_interrupt(adev, NULL);
2385 return;
2386 }
2387
2244 if (adev->nbio.ras &&2388 if (adev->nbio.ras &&
2245 adev->nbio.ras->handle_ras_controller_intr_no_bifring)2389 adev->nbio.ras->handle_ras_controller_intr_no_bifring)
2246 adev->nbio.ras->handle_ras_controller_intr_no_bifring(adev);2390 adev->nbio.ras->handle_ras_controller_intr_no_bifring(adev);
@@ -2411,6 +2555,16 @@ int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, · 164 unmodified lines
2411 struct ras_manager *obj;2555 struct ras_manager *obj;
2412 struct ras_ih_data *data;2556 struct ras_ih_data *data;
2413 2557
2558 if (amdgpu_uniras_enabled(adev)) {
2559 struct ras_ih_info ih_info;
2560
2561 memset(&ih_info, 0, sizeof(ih_info));
2562 ih_info.block = info->head.block;
2563 memcpy(&ih_info.iv_entry, info->entry, sizeof(struct amdgpu_iv_entry));
2564
2565 return amdgpu_ras_mgr_handle_controller_interrupt(adev, &ih_info);
2566 }
2567
2414 obj = amdgpu_ras_find_obj(adev, &info->head);2568 obj = amdgpu_ras_find_obj(adev, &info->head);
2415 if (!obj)2569 if (!obj)
2416 return -EINVAL;2570 return -EINVAL;
@@ -2605,62 +2759,83 @@ static void amdgpu_ras_query_err_status(struct amdgpu_device *adev) · 188 unmodified lines
2605 }2759 }
2606}2760}
2607 2761
2608/* recovery begin */
2609
2610/* return 0 on success.
2611 * caller need free bps.
2612 */
2613static int amdgpu_ras_badpages_read(struct amdgpu_device *adev,2762static int amdgpu_ras_badpages_read(struct amdgpu_device *adev,
2614 struct ras_badpage **bps, unsigned int *count)2763 struct ras_badpage *bps, uint32_t count, uint32_t start)
2615{2764{
2616 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);2765 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
2617 struct ras_err_handler_data *data;2766 struct ras_err_handler_data *data;
2618 int i = 0;2767 int r = 0;
2619 int ret = 0, status;2768 uint32_t i;
2620 2769
2621 if (!con || !con->eh_data || !bps || !count)2770 if (!con || !con->eh_data || !bps || !count)
2622 return -EINVAL;2771 return -EINVAL;
2623 2772
2624 mutex_lock(&con->recovery_lock);2773 mutex_lock(&con->recovery_lock);
2625 data = con->eh_data;2774 data = con->eh_data;
2626 if (!data || data->count == 0) {2775 if (start < data->count) {
2627 *bps = NULL;2776 for (i = start; i < data->count; i++) {
2628 ret = -EINVAL;2777 if (!data->bps[i].ts)
2629 goto out;2778 continue;
2779
2780 bps[r].bp = data->bps[i].retired_page;
2781 r++;
2782 if (r >= count)
2783 break;
2784 }
2630 }2785 }
2786 mutex_unlock(&con->recovery_lock);
2631 2787
2632 *bps = kmalloc_array(data->count, sizeof(struct ras_badpage), GFP_KERNEL);2788 return r;
2633 if (!*bps) {2789}
2634 ret = -ENOMEM;
2635 goto out;
2636 }
2637 2790
2638 for (; i < data->count; i++) {2791static int amdgpu_uniras_badpages_read(struct amdgpu_device *adev,
2639 if (!data->bps[i].ts)2792 struct ras_badpage *bps, uint32_t count, uint32_t start)
2640 continue;2793{
2794 struct ras_cmd_bad_pages_info_req cmd_input;
2795 struct ras_cmd_bad_pages_info_rsp *output;
2796 uint32_t group, start_group, end_group;
2797 uint32_t pos, pos_in_group;
2798 int r = 0, i;
2641 2799
2642 (*bps)[i] = (struct ras_badpage){2800 if (!bps || !count)
2643 .bp = data->bps[i].retired_page,2801 return -EINVAL;
2644 .size = AMDGPU_GPU_PAGE_SIZE,
2645 .flags = AMDGPU_RAS_RETIRE_PAGE_RESERVED,
2646 };
2647 2802
2648 if (amdgpu_ras_check_critical_address(adev,2803 output = kmalloc(sizeof(*output), GFP_KERNEL);
2649 data->bps[i].retired_page << AMDGPU_GPU_PAGE_SHIFT))2804 if (!output)
2650 continue;2805 return -ENOMEM;
2651 2806
2652 status = amdgpu_vram_mgr_query_page_status(&adev->mman.vram_mgr,2807 memset(&cmd_input, 0, sizeof(cmd_input));
2653 data->bps[i].retired_page << AMDGPU_GPU_PAGE_SHIFT);2808
2654 if (status == -EBUSY)2809 start_group = start / RAS_CMD_MAX_BAD_PAGES_PER_GROUP;
2655 (*bps)[i].flags = AMDGPU_RAS_RETIRE_PAGE_PENDING;2810 end_group = (start + count + RAS_CMD_MAX_BAD_PAGES_PER_GROUP - 1) /
2656 else if (status == -ENOENT)2811 RAS_CMD_MAX_BAD_PAGES_PER_GROUP;
2657 (*bps)[i].flags = AMDGPU_RAS_RETIRE_PAGE_FAULT;2812
2813 pos = start;
2814 for (group = start_group; group < end_group; group++) {
2815 memset(output, 0, sizeof(*output));
2816 cmd_input.group_index = group;
2817 if (amdgpu_ras_mgr_handle_ras_cmd(adev, RAS_CMD__GET_BAD_PAGES,
2818 &cmd_input, sizeof(cmd_input), output, sizeof(*output)))
2819 goto out;
2820
2821 if (pos >= output->bp_total_cnt)
2822 goto out;
2823
2824 pos_in_group = pos - group * RAS_CMD_MAX_BAD_PAGES_PER_GROUP;
2825 for (i = pos_in_group; i < output->bp_in_group; i++, pos++) {
2826 if (!output->records[i].ts)
2827 continue;
2828
2829 bps[r].bp = output->records[i].retired_page;
2830 r++;
2831 if (r >= count)
2832 goto out;
2833 }
2658 }2834 }
2659 2835
2660 *count = con->bad_page_num;
2661out:2836out:
2662 mutex_unlock(&con->recovery_lock);2837 kfree(output);
2663 return ret;2838 return r;
2664}2839}
2665 2840
2666static void amdgpu_ras_set_fed_all(struct amdgpu_device *adev,2841static void amdgpu_ras_set_fed_all(struct amdgpu_device *adev,
@@ -2748,8 +2923,12 @@ static void amdgpu_ras_do_recovery(struct work_struct *work) · 81 unmodified lines
2748 type = amdgpu_ras_get_fatal_error_event(adev);2923 type = amdgpu_ras_get_fatal_error_event(adev);
2749 list_for_each_entry(remote_adev,2924 list_for_each_entry(remote_adev,
2750 device_list_handle, gmc.xgmi.head) {2925 device_list_handle, gmc.xgmi.head) {
2751 amdgpu_ras_query_err_status(remote_adev);2926 if (amdgpu_uniras_enabled(remote_adev)) {
2752 amdgpu_ras_log_on_err_counter(remote_adev, type);2927 amdgpu_ras_mgr_update_ras_ecc(remote_adev);
2928 } else {
2929 amdgpu_ras_query_err_status(remote_adev);
2930 amdgpu_ras_log_on_err_counter(remote_adev, type);
2931 }
2753 }2932 }
2754 2933
2755 }2934 }
@@ -2837,8 +3016,13 @@ static int amdgpu_ras_mca2pa_by_idx(struct amdgpu_device *adev, · 81 unmodified lines
2837 addr_in.ma.err_addr = bps->address;3016 addr_in.ma.err_addr = bps->address;
2838 addr_in.ma.socket_id = socket;3017 addr_in.ma.socket_id = socket;
2839 addr_in.ma.ch_inst = bps->mem_channel;3018 addr_in.ma.ch_inst = bps->mem_channel;
2840 /* tell RAS TA the node instance is not used */3019 if (!amdgpu_ras_smu_eeprom_supported(adev)) {
2841 addr_in.ma.node_inst = TA_RAS_INV_NODE;3020 /* tell RAS TA the node instance is not used */
3021 addr_in.ma.node_inst = TA_RAS_INV_NODE;
3022 } else {
3023 addr_in.ma.umc_inst = bps->mcumc_id;
3024 addr_in.ma.node_inst = bps->cu;
3025 }
2842 3026
2843 if (adev->umc.ras && adev->umc.ras->convert_ras_err_addr)3027 if (adev->umc.ras && adev->umc.ras->convert_ras_err_addr)
2844 ret = adev->umc.ras->convert_ras_err_addr(adev, err_data,3028 ret = adev->umc.ras->convert_ras_err_addr(adev, err_data,
@@ -2981,8 +3165,16 @@ static int __amdgpu_ras_convert_rec_from_rom(struct amdgpu_device *adev, · 136 unmodified lines
2981 int i = 0;3165 int i = 0;
2982 enum amdgpu_memory_partition save_nps;3166 enum amdgpu_memory_partition save_nps;
2983 3167
2984 save_nps = (bps->retired_page >> UMC_NPS_SHIFT) & UMC_NPS_MASK;3168 if (!amdgpu_ras_smu_eeprom_supported(adev)) {
2985 bps->retired_page &= ~(UMC_NPS_MASK << UMC_NPS_SHIFT);3169 save_nps = (bps->retired_page >> UMC_NPS_SHIFT) & UMC_NPS_MASK;
3170 bps->retired_page &= ~(UMC_NPS_MASK << UMC_NPS_SHIFT);
3171 } else {
3172 /* if pmfw manages eeprom, save_nps is not stored on eeprom,
3173 * we should always convert mca address into physical address,
3174 * make save_nps different from nps
3175 */
3176 save_nps = nps + 1;
3177 }
2986 3178
2987 if (save_nps == nps) {3179 if (save_nps == nps) {
2988 if (amdgpu_umc_pages_in_a_row(adev, err_data,3180 if (amdgpu_umc_pages_in_a_row(adev, err_data,
@@ -3048,7 +3240,8 @@ int amdgpu_ras_add_bad_pages(struct amdgpu_device *adev, · 59 unmodified lines
3048 3240
3049 if (from_rom) {3241 if (from_rom) {
3050 /* there is no pa recs in V3, so skip pa recs processing */3242 /* there is no pa recs in V3, so skip pa recs processing */
3051 if (control->tbl_hdr.version < RAS_TABLE_VER_V3) {3243 if ((control->tbl_hdr.version < RAS_TABLE_VER_V3) &&
3244 !amdgpu_ras_smu_eeprom_supported(adev)) {
3052 for (i = 0; i < pages; i++) {3245 for (i = 0; i < pages; i++) {
3053 if (control->ras_num_recs - i >= adev->umc.retire_unit) {3246 if (control->ras_num_recs - i >= adev->umc.retire_unit) {
3054 if ((bps[i].address == bps[i + 1].address) &&3247 if ((bps[i].address == bps[i + 1].address) &&
@@ -3118,7 +3311,13 @@ int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev, · 63 unmodified lines
3118 mutex_lock(&con->recovery_lock);3311 mutex_lock(&con->recovery_lock);
3119 control = &con->eeprom_control;3312 control = &con->eeprom_control;
3120 data = con->eh_data;3313 data = con->eh_data;
3121 unit_num = data->count / adev->umc.retire_unit - control->ras_num_recs;3314 if (amdgpu_ras_smu_eeprom_supported(adev))
3315 unit_num = control->ras_num_recs -
3316 control->ras_num_recs_old;
3317 else
3318 unit_num = data->count / adev->umc.retire_unit -
3319 control->ras_num_recs;
3320
3122 save_count = con->bad_page_num - control->ras_num_bad_pages;3321 save_count = con->bad_page_num - control->ras_num_bad_pages;
3123 mutex_unlock(&con->recovery_lock);3322 mutex_unlock(&con->recovery_lock);
3124 3323
@@ -3126,7 +3325,7 @@ int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev, · 1 unmodified line
3126 *new_cnt = unit_num;3325 *new_cnt = unit_num;
3127 3326
3128 /* only new entries are saved */3327 /* only new entries are saved */
3129 if (unit_num > 0) {3328 if (unit_num && save_count) {
3130 /*old asics only save pa to eeprom like before*/3329 /*old asics only save pa to eeprom like before*/
3131 if (IP_VERSION_MAJ(amdgpu_ip_version(adev, UMC_HWIP, 0)) < 12) {3330 if (IP_VERSION_MAJ(amdgpu_ip_version(adev, UMC_HWIP, 0)) < 12) {
3132 if (amdgpu_ras_eeprom_append(control,3331 if (amdgpu_ras_eeprom_append(control,
@@ -3179,7 +3378,8 @@ static int amdgpu_ras_load_bad_pages(struct amdgpu_device *adev) · 46 unmodified lines
3179 /*In V3, there is no pa recs, and some cases(when address==0) may be parsed3378 /*In V3, there is no pa recs, and some cases(when address==0) may be parsed
3180 as pa recs, so add verion check to avoid it.3379 as pa recs, so add verion check to avoid it.
3181 */3380 */
3182 if (control->tbl_hdr.version < RAS_TABLE_VER_V3) {3381 if ((control->tbl_hdr.version < RAS_TABLE_VER_V3) &&
3382 !amdgpu_ras_smu_eeprom_supported(adev)) {
3183 for (i = 0; i < control->ras_num_recs; i++) {3383 for (i = 0; i < control->ras_num_recs; i++) {
3184 if ((control->ras_num_recs - i) >= adev->umc.retire_unit) {3384 if ((control->ras_num_recs - i) >= adev->umc.retire_unit) {
3185 if ((bps[i].address == bps[i + 1].address) &&3385 if ((bps[i].address == bps[i + 1].address) &&
@@ -3590,7 +3790,12 @@ int amdgpu_ras_init_badpage_info(struct amdgpu_device *adev) · 404 unmodified lines
3590 if (!con || amdgpu_sriov_vf(adev))3790 if (!con || amdgpu_sriov_vf(adev))
3591 return 0;3791 return 0;
3592 3792
3793 if (amdgpu_uniras_enabled(adev))
3794 return 0;
3795
3593 control = &con->eeprom_control;3796 control = &con->eeprom_control;
3797 con->ras_smu_drv = amdgpu_dpm_get_ras_smu_driver(adev);
3798
3594 ret = amdgpu_ras_eeprom_init(control);3799 ret = amdgpu_ras_eeprom_init(control);
3595 control->is_eeprom_valid = !ret;3800 control->is_eeprom_valid = !ret;
3596 3801
@@ -3751,7 +3956,9 @@ static int amdgpu_ras_recovery_fini(struct amdgpu_device *adev) · 154 unmodified lines
3751 mutex_unlock(&con->recovery_lock);3956 mutex_unlock(&con->recovery_lock);
3752 3957
3753 amdgpu_ras_critical_region_init(adev);3958 amdgpu_ras_critical_region_init(adev);
3754 3959#ifdef CONFIG_X86_MCE_AMD
3960 amdgpu_unregister_bad_pages_mca_notifier(adev);
3961#endif
3755 return 0;3962 return 0;
3756}3963}
3757/* recovery end */3964/* recovery end */
@@ -3975,7 +4182,6 @@ static void amdgpu_ras_counte_dw(struct work_struct *work) · 217 unmodified lines
3975 atomic_set(&con->ras_ue_count, ue_count);4182 atomic_set(&con->ras_ue_count, ue_count);
3976 }4183 }
3977 4184
3978 pm_runtime_mark_last_busy(dev->dev);
3979Out:4185Out:
3980 pm_runtime_put_autosuspend(dev->dev);4186 pm_runtime_put_autosuspend(dev->dev);
3981}4187}
@@ -4584,6 +4790,9 @@ int amdgpu_ras_mark_ras_event_caller(struct amdgpu_device *adev, enum ras_event_ · 602 unmodified lines
4584 struct ras_event_state *event_state;4790 struct ras_event_state *event_state;
4585 int ret = 0;4791 int ret = 0;
4586 4792
4793 if (amdgpu_uniras_enabled(adev))
4794 return 0;
4795
4587 if (type >= RAS_EVENT_TYPE_COUNT) {4796 if (type >= RAS_EVENT_TYPE_COUNT) {
4588 ret = -EINVAL;4797 ret = -EINVAL;
4589 goto out;4798 goto out;
@@ -4634,20 +4843,18 @@ u64 amdgpu_ras_acquire_event_id(struct amdgpu_device *adev, enum ras_event_type · 44 unmodified lines
4634 return id;4843 return id;
4635}4844}
4636 4845
4637void amdgpu_ras_global_ras_isr(struct amdgpu_device *adev)4846int amdgpu_ras_global_ras_isr(struct amdgpu_device *adev)
4638{4847{
4639 if (atomic_cmpxchg(&amdgpu_ras_in_intr, 0, 1) == 0) {4848 if (atomic_cmpxchg(&amdgpu_ras_in_intr, 0, 1) == 0) {
4640 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);4849 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
4641 enum ras_event_type type = RAS_EVENT_TYPE_FATAL;4850 enum ras_event_type type = RAS_EVENT_TYPE_FATAL;
4642 u64 event_id;4851 u64 event_id = RAS_EVENT_INVALID_ID;
4643 4852
4644 if (amdgpu_ras_mark_ras_event(adev, type)) {4853 if (amdgpu_uniras_enabled(adev))
4645 dev_err(adev->dev,4854 return 0;
4646 "uncorrectable hardware error (ERREVENT_ATHUB_INTERRUPT) detected!\n");
4647 return;
4648 }
4649 4855
4650 event_id = amdgpu_ras_acquire_event_id(adev, type);4856 if (!amdgpu_ras_mark_ras_event(adev, type))
4857 event_id = amdgpu_ras_acquire_event_id(adev, type);
4651 4858
4652 RAS_EVENT_LOG(adev, event_id, "uncorrectable hardware error"4859 RAS_EVENT_LOG(adev, event_id, "uncorrectable hardware error"
4653 "(ERREVENT_ATHUB_INTERRUPT) detected!\n");4860 "(ERREVENT_ATHUB_INTERRUPT) detected!\n");
@@ -4656,6 +4863,8 @@ void amdgpu_ras_global_ras_isr(struct amdgpu_device *adev) · 2 unmodified lines
4656 ras->gpu_reset_flags |= AMDGPU_RAS_GPU_RESET_MODE1_RESET;4863 ras->gpu_reset_flags |= AMDGPU_RAS_GPU_RESET_MODE1_RESET;
4657 amdgpu_ras_reset_gpu(adev);4864 amdgpu_ras_reset_gpu(adev);
4658 }4865 }
4866
4867 return -EBUSY;
4659}4868}
4660 4869
4661bool amdgpu_ras_need_emergency_restart(struct amdgpu_device *adev)4870bool amdgpu_ras_need_emergency_restart(struct amdgpu_device *adev)
@@ -4783,6 +4992,28 @@ static void amdgpu_register_bad_pages_mca_notifier(struct amdgpu_device *adev) · 121 unmodified lines
4783 notifier_registered = true;4992 notifier_registered = true;
4784 }4993 }
4785}4994}
4995static void amdgpu_unregister_bad_pages_mca_notifier(struct amdgpu_device *adev)
4996{
4997 int i, j;
4998
4999 if (!notifier_registered && !mce_adev_list.num_gpu)
5000 return;
5001 for (i = 0, j = 0; i < mce_adev_list.num_gpu; i++) {
5002 if (mce_adev_list.devs[i] == adev)
5003 mce_adev_list.devs[i] = NULL;
5004 if (!mce_adev_list.devs[i])
5005 ++j;
5006 }
5007
5008 if (j == mce_adev_list.num_gpu) {
5009 mce_adev_list.num_gpu = 0;
5010 /* Unregister x86 notifier with MCE subsystem. */
5011 if (notifier_registered) {
5012 mce_unregister_decode_chain(&amdgpu_bad_page_nb);
5013 notifier_registered = false;
5014 }
5015 }
5016}
4786#endif5017#endif
4787 5018
4788struct amdgpu_ras *amdgpu_ras_get_context(struct amdgpu_device *adev)5019struct amdgpu_ras *amdgpu_ras_get_context(struct amdgpu_device *adev)
@@ -5408,6 +5639,9 @@ bool amdgpu_ras_is_rma(struct amdgpu_device *adev) · 619 unmodified lines
5408{5639{
5409 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);5640 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
5410 5641
5642 if (amdgpu_uniras_enabled(adev))
5643 return amdgpu_ras_mgr_is_rma(adev);
5644
5411 if (!con)5645 if (!con)
5412 return false;5646 return false;
5413 5647
@@ -5490,3 +5724,25 @@ bool amdgpu_ras_check_critical_address(struct amdgpu_device *adev, uint64_t addr · 76 unmodified lines
5490 5724
5491 return ret;5725 return ret;
5492}5726}
5727
5728void amdgpu_ras_pre_reset(struct amdgpu_device *adev,
5729 struct list_head *device_list)
5730{
5731 struct amdgpu_device *tmp_adev = NULL;
5732
5733 list_for_each_entry(tmp_adev, device_list, reset_list) {
5734 if (amdgpu_uniras_enabled(tmp_adev))
5735 amdgpu_ras_mgr_pre_reset(tmp_adev);
5736 }
5737}
5738
5739void amdgpu_ras_post_reset(struct amdgpu_device *adev,
5740 struct list_head *device_list)
5741{
5742 struct amdgpu_device *tmp_adev = NULL;
5743
5744 list_for_each_entry(tmp_adev, device_list, reset_list) {
5745 if (amdgpu_uniras_enabled(tmp_adev))
5746 amdgpu_ras_mgr_post_reset(tmp_adev);
5747 }
5748}
drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h+37−1
@@ -503,7 +503,34 @@ struct ras_critical_region { · 502 unmodified lines
503 uint64_t size;503 uint64_t size;
504};504};
505 505
506struct ras_eeprom_table_version {
507 uint32_t minor : 16;
508 uint32_t major : 16;
509};
510
511struct ras_eeprom_smu_funcs {
512 int (*get_ras_table_version)(struct amdgpu_device *adev,
513 uint32_t *table_version);
514 int (*get_badpage_count)(struct amdgpu_device *adev, uint32_t *count, uint32_t timeout);
515 int (*get_badpage_mca_addr)(struct amdgpu_device *adev, uint16_t index, uint64_t *mca_addr);
516 int (*set_timestamp)(struct amdgpu_device *adev, uint64_t timestamp);
517 int (*get_timestamp)(struct amdgpu_device *adev,
518 uint16_t index, uint64_t *timestamp);
519 int (*get_badpage_ipid)(struct amdgpu_device *adev, uint16_t index, uint64_t *ipid);
520 int (*erase_ras_table)(struct amdgpu_device *adev, uint32_t *result);
521};
522
523enum ras_smu_feature_flags {
524 RAS_SMU_FEATURE_BIT__RAS_EEPROM = BIT_ULL(0),
525};
526
527struct ras_smu_drv {
528 const struct ras_eeprom_smu_funcs *smu_eeprom_funcs;
529 void (*ras_smu_feature_flags)(struct amdgpu_device *adev, uint64_t *flags);
530};
531
506struct amdgpu_ras {532struct amdgpu_ras {
533 void *ras_mgr;
507 /* ras infrastructure */534 /* ras infrastructure */
508 /* for ras itself. */535 /* for ras itself. */
509 uint32_t features;536 uint32_t features;
@@ -590,6 +617,10 @@ struct amdgpu_ras { · 80 unmodified lines
590 617
591 /* Protect poison injection */618 /* Protect poison injection */
592 struct mutex poison_lock;619 struct mutex poison_lock;
620
621 /* Disable/Enable uniras switch */
622 bool uniras_enabled;
623 const struct ras_smu_drv *ras_smu_drv;
593};624};
594 625
595struct ras_fs_data {626struct ras_fs_data {
@@ -909,7 +940,7 @@ static inline void amdgpu_ras_intr_cleared(void) · 313 unmodified lines
909 atomic_set(&amdgpu_ras_in_intr, 0);940 atomic_set(&amdgpu_ras_in_intr, 0);
910}941}
911 942
912void amdgpu_ras_global_ras_isr(struct amdgpu_device *adev);943int amdgpu_ras_global_ras_isr(struct amdgpu_device *adev);
913 944
914void amdgpu_ras_set_error_query_ready(struct amdgpu_device *adev, bool ready);945void amdgpu_ras_set_error_query_ready(struct amdgpu_device *adev, bool ready);
915 946
@@ -1008,4 +1039,9 @@ void amdgpu_ras_event_log_print(struct amdgpu_device *adev, u64 event_id, · 92 unmodified lines
1008 const char *fmt, ...);1039 const char *fmt, ...);
1009 1040
1010bool amdgpu_ras_is_rma(struct amdgpu_device *adev);1041bool amdgpu_ras_is_rma(struct amdgpu_device *adev);
1042
1043void amdgpu_ras_pre_reset(struct amdgpu_device *adev,
1044 struct list_head *device_list);
1045void amdgpu_ras_post_reset(struct amdgpu_device *adev,
1046 struct list_head *device_list);
1011#endif1047#endif
drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c+394−28
@@ -32,6 +32,7 @@ · 31 unmodified lines
32#include <linux/uaccess.h>32#include <linux/uaccess.h>
33 33
34#include "amdgpu_reset.h"34#include "amdgpu_reset.h"
35#include "amdgpu_ras_mgr.h"
35 36
36/* These are memory addresses as would be seen by one or more EEPROM37/* These are memory addresses as would be seen by one or more EEPROM
37 * chips strung on the I2C bus, usually by manipulating pins 1-3 of a38 * chips strung on the I2C bus, usually by manipulating pins 1-3 of a
@@ -123,6 +124,8 @@ · 85 unmodified lines
123 RAS_TABLE_V2_1_INFO_SIZE) \124 RAS_TABLE_V2_1_INFO_SIZE) \
124 / RAS_TABLE_RECORD_SIZE)125 / RAS_TABLE_RECORD_SIZE)
125 126
127#define RAS_SMU_MESSAGE_TIMEOUT_MS 1000 /* 1s */
128
126/* Given a zero-based index of an EEPROM RAS record, yields the EEPROM129/* Given a zero-based index of an EEPROM RAS record, yields the EEPROM
127 * offset off of RAS_TABLE_START. That is, this is something you can130 * offset off of RAS_TABLE_START. That is, this is something you can
128 * add to control->i2c_address, and then tell I2C layer to read131 * add to control->i2c_address, and then tell I2C layer to read
@@ -443,40 +446,57 @@ int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control) · 314 unmodified lines
443 struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr;446 struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr;
444 struct amdgpu_ras_eeprom_table_ras_info *rai = &control->tbl_rai;447 struct amdgpu_ras_eeprom_table_ras_info *rai = &control->tbl_rai;
445 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);448 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
449 u32 erase_res = 0;
446 u8 csum;450 u8 csum;
447 int res;451 int res;
448 452
449 mutex_lock(&control->ras_tbl_mutex);453 mutex_lock(&control->ras_tbl_mutex);
450 454
451 hdr->header = RAS_TABLE_HDR_VAL;455 if (!amdgpu_ras_smu_eeprom_supported(adev)) {
452 amdgpu_ras_set_eeprom_table_version(control);456 hdr->header = RAS_TABLE_HDR_VAL;
457 amdgpu_ras_set_eeprom_table_version(control);
453 458
454 if (hdr->version >= RAS_TABLE_VER_V2_1) {459 if (hdr->version >= RAS_TABLE_VER_V2_1) {
455 hdr->first_rec_offset = RAS_RECORD_START_V2_1;460 hdr->first_rec_offset = RAS_RECORD_START_V2_1;
456 hdr->tbl_size = RAS_TABLE_HEADER_SIZE +461 hdr->tbl_size = RAS_TABLE_HEADER_SIZE +
457 RAS_TABLE_V2_1_INFO_SIZE;462 RAS_TABLE_V2_1_INFO_SIZE;
458 rai->rma_status = GPU_HEALTH_USABLE;463 rai->rma_status = GPU_HEALTH_USABLE;
459 /**464
460 * GPU health represented as a percentage.465 control->ras_record_offset = RAS_RECORD_START_V2_1;
461 * 0 means worst health, 100 means fully health.466 control->ras_max_record_count = RAS_MAX_RECORD_COUNT_V2_1;
462 */467 /**
463 rai->health_percent = 100;468 * GPU health represented as a percentage.
464 /* ecc_page_threshold = 0 means disable bad page retirement */469 * 0 means worst health, 100 means fully health.
465 rai->ecc_page_threshold = con->bad_page_cnt_threshold;470 */
471 rai->health_percent = 100;
472 /* ecc_page_threshold = 0 means disable bad page retirement */
473 rai->ecc_page_threshold = con->bad_page_cnt_threshold;
474 } else {
475 hdr->first_rec_offset = RAS_RECORD_START;
476 hdr->tbl_size = RAS_TABLE_HEADER_SIZE;
477
478 control->ras_record_offset = RAS_RECORD_START;
479 control->ras_max_record_count = RAS_MAX_RECORD_COUNT;
480 }
481
482 csum = __calc_hdr_byte_sum(control);
483 if (hdr->version >= RAS_TABLE_VER_V2_1)
484 csum += __calc_ras_info_byte_sum(control);
485 csum = -csum;
486 hdr->checksum = csum;
487 res = __write_table_header(control);
488 if (!res && hdr->version > RAS_TABLE_VER_V1)
489 res = __write_table_ras_info(control);
466 } else {490 } else {
467 hdr->first_rec_offset = RAS_RECORD_START;491 res = amdgpu_ras_smu_erase_ras_table(adev, &erase_res);
468 hdr->tbl_size = RAS_TABLE_HEADER_SIZE;492 if (res || erase_res) {
493 dev_warn(adev->dev, "RAS EEPROM reset failed, res:%d result:%d",
494 res, erase_res);
495 if (!res)
496 res = -EIO;
497 }
469 }498 }
470 499
471 csum = __calc_hdr_byte_sum(control);
472 if (hdr->version >= RAS_TABLE_VER_V2_1)
473 csum += __calc_ras_info_byte_sum(control);
474 csum = -csum;
475 hdr->checksum = csum;
476 res = __write_table_header(control);
477 if (!res && hdr->version > RAS_TABLE_VER_V1)
478 res = __write_table_ras_info(control);
479
480 control->ras_num_recs = 0;500 control->ras_num_recs = 0;
481 control->ras_num_bad_pages = 0;501 control->ras_num_bad_pages = 0;
482 control->ras_num_mca_recs = 0;502 control->ras_num_mca_recs = 0;
@@ -556,6 +576,9 @@ bool amdgpu_ras_eeprom_check_err_threshold(struct amdgpu_device *adev) · 73 unmodified lines
556{576{
557 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);577 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
558 578
579 if (amdgpu_uniras_enabled(adev))
580 return amdgpu_ras_mgr_check_eeprom_safety_watermark(adev);
581
559 if (!__is_ras_eeprom_supported(adev) ||582 if (!__is_ras_eeprom_supported(adev) ||
560 !amdgpu_bad_page_threshold)583 !amdgpu_bad_page_threshold)
561 return false;584 return false;
@@ -766,7 +789,8 @@ amdgpu_ras_eeprom_update_header(struct amdgpu_ras_eeprom_control *control) · 204 unmodified lines
766 "Saved bad pages %d reaches threshold value %d\n",789 "Saved bad pages %d reaches threshold value %d\n",
767 control->ras_num_bad_pages, ras->bad_page_cnt_threshold);790 control->ras_num_bad_pages, ras->bad_page_cnt_threshold);
768 791
769 if (adev->cper.enabled && amdgpu_cper_generate_bp_threshold_record(adev))792 if (adev->cper.enabled && !amdgpu_uniras_enabled(adev) &&
793 amdgpu_cper_generate_bp_threshold_record(adev))
770 dev_warn(adev->dev, "fail to generate bad page threshold cper records\n");794 dev_warn(adev->dev, "fail to generate bad page threshold cper records\n");
771 795
772 if ((amdgpu_bad_page_threshold != -1) &&796 if ((amdgpu_bad_page_threshold != -1) &&
@@ -849,6 +873,71 @@ Out: · 76 unmodified lines
849 return res;873 return res;
850}874}
851 875
876int amdgpu_ras_eeprom_update_record_num(struct amdgpu_ras_eeprom_control *control)
877{
878 struct amdgpu_device *adev = to_amdgpu_device(control);
879 int ret, retry = 20;
880
881 if (!amdgpu_ras_smu_eeprom_supported(adev))
882 return 0;
883
884 control->ras_num_recs_old = control->ras_num_recs;
885
886 do {
887 /* 1000ms timeout is long enough, smu_get_badpage_count won't
888 * return -EBUSY before timeout.
889 */
890 ret = amdgpu_ras_smu_get_badpage_count(adev,
891 &(control->ras_num_recs), RAS_SMU_MESSAGE_TIMEOUT_MS);
892 if (!ret &&
893 (control->ras_num_recs_old == control->ras_num_recs)) {
894 /* record number update in PMFW needs some time,
895 * smu_get_badpage_count may return immediately without
896 * count update, sleep for a while and retry again.
897 */
898 msleep(50);
899 retry--;
900 } else {
901 break;
902 }
903 } while (retry);
904
905 /* no update of record number is not a real failure,
906 * don't print warning here
907 */
908 if (!ret && (control->ras_num_recs_old == control->ras_num_recs))
909 ret = -EINVAL;
910
911 return ret;
912}
913
914static int amdgpu_ras_smu_eeprom_append(struct amdgpu_ras_eeprom_control *control)
915{
916 struct amdgpu_device *adev = to_amdgpu_device(control);
917 struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
918
919 if (!amdgpu_ras_smu_eeprom_supported(adev) || !con)
920 return 0;
921
922 control->ras_num_bad_pages = con->bad_page_num;
923
924 if (amdgpu_bad_page_threshold != 0 &&
925 control->ras_num_bad_pages > con->bad_page_cnt_threshold) {
926 dev_warn(adev->dev,
927 "Saved bad pages %d reaches threshold value %d\n",
928 control->ras_num_bad_pages, con->bad_page_cnt_threshold);
929
930 if (adev->cper.enabled && amdgpu_cper_generate_bp_threshold_record(adev))
931 dev_warn(adev->dev, "fail to generate bad page threshold cper records\n");
932
933 if ((amdgpu_bad_page_threshold != -1) &&
934 (amdgpu_bad_page_threshold != -2))
935 con->is_rma = true;
936 }
937
938 return 0;
939}
940
852/**941/**
853 * amdgpu_ras_eeprom_append -- append records to the EEPROM RAS table942 * amdgpu_ras_eeprom_append -- append records to the EEPROM RAS table
854 * @control: pointer to control structure943 * @control: pointer to control structure
@@ -873,6 +962,9 @@ int amdgpu_ras_eeprom_append(struct amdgpu_ras_eeprom_control *control, · 18 unmodified lines
873 if (!__is_ras_eeprom_supported(adev))962 if (!__is_ras_eeprom_supported(adev))
874 return 0;963 return 0;
875 964
965 if (amdgpu_ras_smu_eeprom_supported(adev))
966 return amdgpu_ras_smu_eeprom_append(control);
967
876 if (num == 0) {968 if (num == 0) {
877 dev_err(adev->dev, "will not append 0 records\n");969 dev_err(adev->dev, "will not append 0 records\n");
878 return -EINVAL;970 return -EINVAL;
@@ -948,6 +1040,50 @@ static int __amdgpu_ras_eeprom_read(struct amdgpu_ras_eeprom_control *control, · 69 unmodified lines
948 return res;1040 return res;
949}1041}
950 1042
1043int amdgpu_ras_eeprom_read_idx(struct amdgpu_ras_eeprom_control *control,
1044 struct eeprom_table_record *record, u32 rec_idx,
1045 const u32 num)
1046{
1047 struct amdgpu_device *adev = to_amdgpu_device(control);
1048 uint64_t ts, end_idx;
1049 int i, ret;
1050 u64 mca, ipid;
1051
1052 if (!amdgpu_ras_smu_eeprom_supported(adev))
1053 return 0;
1054
1055 if (!adev->umc.ras || !adev->umc.ras->mca_ipid_parse)
1056 return -EOPNOTSUPP;
1057
1058 end_idx = rec_idx + num;
1059 for (i = rec_idx; i < end_idx; i++) {
1060 ret = amdgpu_ras_smu_get_badpage_mca_addr(adev, i, &mca);
1061 if (ret)
1062 return ret;
1063
1064 ret = amdgpu_ras_smu_get_badpage_ipid(adev, i, &ipid);
1065 if (ret)
1066 return ret;
1067
1068 ret = amdgpu_ras_smu_get_timestamp(adev, i, &ts);
1069 if (ret)
1070 return ret;
1071
1072 record[i - rec_idx].address = mca;
1073 /* retired_page (pa) is unused now */
1074 record[i - rec_idx].retired_page = 0x1ULL;
1075 record[i - rec_idx].ts = ts;
1076 record[i - rec_idx].err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
1077
1078 adev->umc.ras->mca_ipid_parse(adev, ipid,
1079 (uint32_t *)&(record[i - rec_idx].cu),
1080 (uint32_t *)&(record[i - rec_idx].mem_channel),
1081 (uint32_t *)&(record[i - rec_idx].mcumc_id), NULL);
1082 }
1083
1084 return 0;
1085}
1086
951/**1087/**
952 * amdgpu_ras_eeprom_read -- read EEPROM1088 * amdgpu_ras_eeprom_read -- read EEPROM
953 * @control: pointer to control structure1089 * @control: pointer to control structure
@@ -969,6 +1105,9 @@ int amdgpu_ras_eeprom_read(struct amdgpu_ras_eeprom_control *control, · 15 unmodified lines
969 u8 *buf, *pp;1105 u8 *buf, *pp;
970 u32 g0, g1;1106 u32 g0, g1;
971 1107
1108 if (amdgpu_ras_smu_eeprom_supported(adev))
1109 return amdgpu_ras_eeprom_read_idx(control, record, 0, num);
1110
972 if (!__is_ras_eeprom_supported(adev))1111 if (!__is_ras_eeprom_supported(adev))
973 return 0;1112 return 0;
974 1113
@@ -1140,6 +1279,10 @@ static ssize_t amdgpu_ras_debugfs_table_read(struct file *f, char __user *buf, · 165 unmodified lines
1140 int res = -EFAULT;1279 int res = -EFAULT;
1141 size_t data_len;1280 size_t data_len;
1142 1281
1282 /* pmfw manages eeprom data by itself */
1283 if (amdgpu_ras_smu_eeprom_supported(adev))
1284 return 0;
1285
1143 mutex_lock(&control->ras_tbl_mutex);1286 mutex_lock(&control->ras_tbl_mutex);
1144 1287
1145 /* We want *pos - data_len > 0, which means there's1288 /* We want *pos - data_len > 0, which means there's
@@ -1370,6 +1513,42 @@ Out: · 224 unmodified lines
1370 return res == RAS_TABLE_V2_1_INFO_SIZE ? 0 : res;1513 return res == RAS_TABLE_V2_1_INFO_SIZE ? 0 : res;
1371}1514}
1372 1515
1516static int amdgpu_ras_smu_eeprom_init(struct amdgpu_ras_eeprom_control *control)
1517{
1518 struct amdgpu_device *adev = to_amdgpu_device(control);
1519 struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr;
1520 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
1521 uint64_t local_time;
1522 int res;
1523
1524 ras->is_rma = false;
1525
1526 if (!__is_ras_eeprom_supported(adev))
1527 return 0;
1528 mutex_init(&control->ras_tbl_mutex);
1529
1530 res = amdgpu_ras_smu_get_table_version(adev, &(hdr->version));
1531 if (res)
1532 return res;
1533
1534 res = amdgpu_ras_smu_get_badpage_count(adev,
1535 &(control->ras_num_recs), 100);
1536 if (res)
1537 return res;
1538
1539 local_time = (uint64_t)ktime_get_real_seconds();
1540 res = amdgpu_ras_smu_set_timestamp(adev, local_time);
1541 if (res)
1542 return res;
1543
1544 control->ras_max_record_count = 4000;
1545
1546 control->ras_num_mca_recs = 0;
1547 control->ras_num_pa_recs = 0;
1548
1549 return 0;
1550}
1551
1373int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control)1552int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control)
1374{1553{
1375 struct amdgpu_device *adev = to_amdgpu_device(control);1554 struct amdgpu_device *adev = to_amdgpu_device(control);
@@ -1378,6 +1557,9 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control) · 2 unmodified lines
1378 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);1557 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
1379 int res;1558 int res;
1380 1559
1560 if (amdgpu_ras_smu_eeprom_supported(adev))
1561 return amdgpu_ras_smu_eeprom_init(control);
1562
1381 ras->is_rma = false;1563 ras->is_rma = false;
1382 1564
1383 if (!__is_ras_eeprom_supported(adev))1565 if (!__is_ras_eeprom_supported(adev))
@@ -1444,6 +1626,47 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control) · 60 unmodified lines
1444 return 0;1626 return 0;
1445}1627}
1446 1628
1629static int amdgpu_ras_smu_eeprom_check(struct amdgpu_ras_eeprom_control *control)
1630{
1631 struct amdgpu_device *adev = to_amdgpu_device(control);
1632 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
1633
1634 if (!__is_ras_eeprom_supported(adev))
1635 return 0;
1636
1637 control->ras_num_bad_pages = ras->bad_page_num;
1638
1639 if ((ras->bad_page_cnt_threshold < control->ras_num_bad_pages) &&
1640 amdgpu_bad_page_threshold != 0) {
1641 dev_warn(adev->dev,
1642 "RAS records:%d exceed threshold:%d\n",
1643 control->ras_num_bad_pages, ras->bad_page_cnt_threshold);
1644 if ((amdgpu_bad_page_threshold == -1) ||
1645 (amdgpu_bad_page_threshold == -2)) {
1646 dev_warn(adev->dev,
1647 "Please consult AMD Service Action Guide (SAG) for appropriate service procedures\n");
1648 } else {
1649 ras->is_rma = true;
1650 dev_warn(adev->dev,
1651 "User defined threshold is set, runtime service will be halt when threshold is reached\n");
1652 }
1653
1654 return 0;
1655 }
1656
1657 dev_dbg(adev->dev,
1658 "Found existing EEPROM table with %d records",
1659 control->ras_num_bad_pages);
1660
1661 /* Warn if we are at 90% of the threshold or above
1662 */
1663 if (10 * control->ras_num_bad_pages >= 9 * ras->bad_page_cnt_threshold)
1664 dev_warn(adev->dev, "RAS records:%u exceeds 90%% of threshold:%d",
1665 control->ras_num_bad_pages,
1666 ras->bad_page_cnt_threshold);
1667 return 0;
1668}
1669
1447int amdgpu_ras_eeprom_check(struct amdgpu_ras_eeprom_control *control)1670int amdgpu_ras_eeprom_check(struct amdgpu_ras_eeprom_control *control)
1448{1671{
1449 struct amdgpu_device *adev = to_amdgpu_device(control);1672 struct amdgpu_device *adev = to_amdgpu_device(control);
@@ -1451,6 +1674,9 @@ int amdgpu_ras_eeprom_check(struct amdgpu_ras_eeprom_control *control) · 1 unmodified line
1451 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);1674 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
1452 int res = 0;1675 int res = 0;
1453 1676
1677 if (amdgpu_ras_smu_eeprom_supported(adev))
1678 return amdgpu_ras_smu_eeprom_check(control);
1679
1454 if (!__is_ras_eeprom_supported(adev))1680 if (!__is_ras_eeprom_supported(adev))
1455 return 0;1681 return 0;
1456 1682
@@ -1541,7 +1767,8 @@ void amdgpu_ras_eeprom_check_and_recover(struct amdgpu_device *adev) · 84 unmodified lines
1541 struct amdgpu_ras_eeprom_control *control;1767 struct amdgpu_ras_eeprom_control *control;
1542 int res;1768 int res;
1543 1769
1544 if (!__is_ras_eeprom_supported(adev) || !ras)1770 if (!__is_ras_eeprom_supported(adev) || !ras ||
1771 amdgpu_ras_smu_eeprom_supported(adev))
1545 return;1772 return;
1546 control = &ras->eeprom_control;1773 control = &ras->eeprom_control;
1547 if (!control->is_eeprom_valid)1774 if (!control->is_eeprom_valid)
@@ -1561,4 +1788,143 @@ void amdgpu_ras_eeprom_check_and_recover(struct amdgpu_device *adev) · 13 unmodified lines
1561 control->is_eeprom_valid = false;1788 control->is_eeprom_valid = false;
1562 }1789 }
1563 return;1790 return;
1564}1791}
1792
1793static const struct ras_smu_drv *amdgpu_ras_get_smu_ras_drv(struct amdgpu_device *adev)
1794{
1795 struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
1796
1797 if (!ras)
1798 return NULL;
1799
1800 return ras->ras_smu_drv;
1801}
1802
1803static uint64_t amdgpu_ras_smu_get_feature_flags(struct amdgpu_device *adev)
1804{
1805 const struct ras_smu_drv *ras_smu_drv = amdgpu_ras_get_smu_ras_drv(adev);
1806 uint64_t flags = 0ULL;
1807
1808 if (!ras_smu_drv)
1809 goto out;
1810
1811 if (ras_smu_drv->ras_smu_feature_flags)
1812 ras_smu_drv->ras_smu_feature_flags(adev, &flags);
1813
1814out:
1815 return flags;
1816}
1817
1818bool amdgpu_ras_smu_eeprom_supported(struct amdgpu_device *adev)
1819{
1820 const struct ras_smu_drv *smu_ras_drv = amdgpu_ras_get_smu_ras_drv(adev);
1821 uint64_t flags = 0ULL;
1822
1823 if (!__is_ras_eeprom_supported(adev) || !smu_ras_drv)
1824 return false;
1825
1826 if (!smu_ras_drv->smu_eeprom_funcs)
1827 return false;
1828
1829 flags = amdgpu_ras_smu_get_feature_flags(adev);
1830
1831 return !!(flags & RAS_SMU_FEATURE_BIT__RAS_EEPROM);
1832}
1833
1834int amdgpu_ras_smu_get_table_version(struct amdgpu_device *adev,
1835 uint32_t *table_version)
1836{
1837 const struct ras_smu_drv *smu_ras_drv = amdgpu_ras_get_smu_ras_drv(adev);
1838
1839 if (!amdgpu_ras_smu_eeprom_supported(adev))
1840 return -EOPNOTSUPP;
1841
1842 if (smu_ras_drv->smu_eeprom_funcs->get_ras_table_version)
1843 return smu_ras_drv->smu_eeprom_funcs->get_ras_table_version(adev,
1844 table_version);
1845 return -EOPNOTSUPP;
1846}
1847
1848int amdgpu_ras_smu_get_badpage_count(struct amdgpu_device *adev,
1849 uint32_t *count, uint32_t timeout)
1850{
1851 const struct ras_smu_drv *smu_ras_drv = amdgpu_ras_get_smu_ras_drv(adev);
1852
1853 if (!amdgpu_ras_smu_eeprom_supported(adev))
1854 return -EOPNOTSUPP;
1855
1856 if (smu_ras_drv->smu_eeprom_funcs->get_badpage_count)
1857 return smu_ras_drv->smu_eeprom_funcs->get_badpage_count(adev,
1858 count, timeout);
1859 return -EOPNOTSUPP;
1860}
1861
1862int amdgpu_ras_smu_get_badpage_mca_addr(struct amdgpu_device *adev,
1863 uint16_t index, uint64_t *mca_addr)
1864{
1865 const struct ras_smu_drv *smu_ras_drv = amdgpu_ras_get_smu_ras_drv(adev);
1866
1867 if (!amdgpu_ras_smu_eeprom_supported(adev))
1868 return -EOPNOTSUPP;
1869
1870 if (smu_ras_drv->smu_eeprom_funcs->get_badpage_mca_addr)
1871 return smu_ras_drv->smu_eeprom_funcs->get_badpage_mca_addr(adev,
1872 index, mca_addr);
1873 return -EOPNOTSUPP;
1874}
1875
1876int amdgpu_ras_smu_set_timestamp(struct amdgpu_device *adev,
1877 uint64_t timestamp)
1878{
1879 const struct ras_smu_drv *smu_ras_drv = amdgpu_ras_get_smu_ras_drv(adev);
1880
1881 if (!amdgpu_ras_smu_eeprom_supported(adev))
1882 return -EOPNOTSUPP;
1883
1884 if (smu_ras_drv->smu_eeprom_funcs->set_timestamp)
1885 return smu_ras_drv->smu_eeprom_funcs->set_timestamp(adev,
1886 timestamp);
1887 return -EOPNOTSUPP;
1888}
1889
1890int amdgpu_ras_smu_get_timestamp(struct amdgpu_device *adev,
1891 uint16_t index, uint64_t *timestamp)
1892{
1893 const struct ras_smu_drv *smu_ras_drv = amdgpu_ras_get_smu_ras_drv(adev);
1894
1895 if (!amdgpu_ras_smu_eeprom_supported(adev))
1896 return -EOPNOTSUPP;
1897
1898 if (smu_ras_drv->smu_eeprom_funcs->get_timestamp)
1899 return smu_ras_drv->smu_eeprom_funcs->get_timestamp(adev,
1900 index, timestamp);
1901 return -EOPNOTSUPP;
1902}
1903
1904int amdgpu_ras_smu_get_badpage_ipid(struct amdgpu_device *adev,
1905 uint16_t index, uint64_t *ipid)
1906{
1907 const struct ras_smu_drv *smu_ras_drv = amdgpu_ras_get_smu_ras_drv(adev);
1908
1909 if (!amdgpu_ras_smu_eeprom_supported(adev))
1910 return -EOPNOTSUPP;
1911
1912 if (smu_ras_drv->smu_eeprom_funcs->get_badpage_ipid)
1913 return smu_ras_drv->smu_eeprom_funcs->get_badpage_ipid(adev,
1914 index, ipid);
1915 return -EOPNOTSUPP;
1916}
1917
1918int amdgpu_ras_smu_erase_ras_table(struct amdgpu_device *adev,
1919 uint32_t *result)
1920{
1921 const struct ras_smu_drv *smu_ras_drv = amdgpu_ras_get_smu_ras_drv(adev);
1922
1923 if (!amdgpu_ras_smu_eeprom_supported(adev))
1924 return -EOPNOTSUPP;
1925
1926 if (smu_ras_drv->smu_eeprom_funcs->erase_ras_table)
1927 return smu_ras_drv->smu_eeprom_funcs->erase_ras_table(adev,
1928 result);
1929 return -EOPNOTSUPP;
1930}
drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.h+30−0
@@ -82,6 +82,7 @@ struct amdgpu_ras_eeprom_control { · 81 unmodified lines
82 /* Number of records in the table.82 /* Number of records in the table.
83 */83 */
84 u32 ras_num_recs;84 u32 ras_num_recs;
85 u32 ras_num_recs_old;
85 86
86 /* the bad page number is ras_num_recs or87 /* the bad page number is ras_num_recs or
87 * ras_num_recs * umc.retire_unit88 * ras_num_recs * umc.retire_unit
@@ -163,6 +164,35 @@ int amdgpu_ras_eeprom_check(struct amdgpu_ras_eeprom_control *control); · 75 unmodified lines
163 164
164void amdgpu_ras_eeprom_check_and_recover(struct amdgpu_device *adev);165void amdgpu_ras_eeprom_check_and_recover(struct amdgpu_device *adev);
165 166
167bool amdgpu_ras_smu_eeprom_supported(struct amdgpu_device *adev);
168
169int amdgpu_ras_smu_get_table_version(struct amdgpu_device *adev,
170 uint32_t *table_version);
171
172int amdgpu_ras_smu_get_badpage_count(struct amdgpu_device *adev,
173 uint32_t *count, uint32_t timeout);
174
175int amdgpu_ras_smu_get_badpage_mca_addr(struct amdgpu_device *adev,
176 uint16_t index, uint64_t *mca_addr);
177
178int amdgpu_ras_smu_set_timestamp(struct amdgpu_device *adev,
179 uint64_t timestamp);
180
181int amdgpu_ras_smu_get_timestamp(struct amdgpu_device *adev,
182 uint16_t index, uint64_t *timestamp);
183
184int amdgpu_ras_smu_get_badpage_ipid(struct amdgpu_device *adev,
185 uint16_t index, uint64_t *ipid);
186
187int amdgpu_ras_smu_erase_ras_table(struct amdgpu_device *adev,
188 uint32_t *result);
189
190int amdgpu_ras_eeprom_read_idx(struct amdgpu_ras_eeprom_control *control,
191 struct eeprom_table_record *record, u32 rec_idx,
192 const u32 num);
193
194int amdgpu_ras_eeprom_update_record_num(struct amdgpu_ras_eeprom_control *control);
195
166extern const struct file_operations amdgpu_ras_debugfs_eeprom_size_ops;196extern const struct file_operations amdgpu_ras_debugfs_eeprom_size_ops;
167extern const struct file_operations amdgpu_ras_debugfs_eeprom_table_ops;197extern const struct file_operations amdgpu_ras_debugfs_eeprom_table_ops;
168 198
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c+74−5
@@ -33,6 +33,7 @@ · 32 unmodified lines
33 33
34#include <drm/amdgpu_drm.h>34#include <drm/amdgpu_drm.h>
35#include "amdgpu.h"35#include "amdgpu.h"
36#include "amdgpu_ras_mgr.h"
36#include "atom.h"37#include "atom.h"
37 38
38/*39/*
@@ -159,8 +160,16 @@ void amdgpu_ring_insert_nop(struct amdgpu_ring *ring, uint32_t count) · 120 unmodified lines
159 */160 */
160void amdgpu_ring_generic_pad_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib)161void amdgpu_ring_generic_pad_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib)
161{162{
162 while (ib->length_dw & ring->funcs->align_mask)163 u32 align_mask = ring->funcs->align_mask;
163 ib->ptr[ib->length_dw++] = ring->funcs->nop;164 u32 count = ib->length_dw & align_mask;
165
166 if (count) {
167 count = align_mask + 1 - count;
168
169 memset32(&ib->ptr[ib->length_dw], ring->funcs->nop, count);
170
171 ib->length_dw += count;
172 }
164}173}
165 174
166/**175/**
@@ -460,9 +469,6 @@ bool amdgpu_ring_soft_recovery(struct amdgpu_ring *ring, unsigned int vmid, · 293 unmodified lines
460 ktime_t deadline;469 ktime_t deadline;
461 bool ret;470 bool ret;
462 471
463 if (unlikely(ring->adev->debug_disable_soft_recovery))
464 return false;
465
466 deadline = ktime_add_us(ktime_get(), 10000);472 deadline = ktime_add_us(ktime_get(), 10000);
467 473
468 if (amdgpu_sriov_vf(ring->adev) || !ring->funcs->soft_recovery || !fence)474 if (amdgpu_sriov_vf(ring->adev) || !ring->funcs->soft_recovery || !fence)
@@ -490,6 +496,66 @@ bool amdgpu_ring_soft_recovery(struct amdgpu_ring *ring, unsigned int vmid, · 21 unmodified lines
490 */496 */
491#if defined(CONFIG_DEBUG_FS)497#if defined(CONFIG_DEBUG_FS)
492 498
499static ssize_t amdgpu_ras_cper_debugfs_read(struct file *f, char __user *buf,
500 size_t size, loff_t *offset)
501{
502 const uint8_t ring_header_size = 12;
503 struct amdgpu_ring *ring = file_inode(f)->i_private;
504 struct ras_cmd_cper_snapshot_req *snapshot_req __free(kfree) =
505 kzalloc(sizeof(struct ras_cmd_cper_snapshot_req), GFP_KERNEL);
506 struct ras_cmd_cper_snapshot_rsp *snapshot_rsp __free(kfree) =
507 kzalloc(sizeof(struct ras_cmd_cper_snapshot_rsp), GFP_KERNEL);
508 struct ras_cmd_cper_record_req *record_req __free(kfree) =
509 kzalloc(sizeof(struct ras_cmd_cper_record_req), GFP_KERNEL);
510 struct ras_cmd_cper_record_rsp *record_rsp __free(kfree) =
511 kzalloc(sizeof(struct ras_cmd_cper_record_rsp), GFP_KERNEL);
512 uint8_t *ring_header __free(kfree) =
513 kzalloc(ring_header_size, GFP_KERNEL);
514 uint32_t total_cper_num;
515 uint64_t start_cper_id;
516 int r;
517
518 if (!snapshot_req || !snapshot_rsp || !record_req || !record_rsp ||
519 !ring_header)
520 return -ENOMEM;
521
522 if (!(*offset)) {
523 /* Need at least 12 bytes for the header on the first read */
524 if (size < ring_header_size)
525 return -EINVAL;
526
527 if (copy_to_user(buf, ring_header, ring_header_size))
528 return -EFAULT;
529 buf += ring_header_size;
530 size -= ring_header_size;
531 }
532
533 r = amdgpu_ras_mgr_handle_ras_cmd(ring->adev,
534 RAS_CMD__GET_CPER_SNAPSHOT,
535 snapshot_req, sizeof(struct ras_cmd_cper_snapshot_req),
536 snapshot_rsp, sizeof(struct ras_cmd_cper_snapshot_rsp));
537 if (r || !snapshot_rsp->total_cper_num)
538 return r;
539
540 start_cper_id = snapshot_rsp->start_cper_id;
541 total_cper_num = snapshot_rsp->total_cper_num;
542
543 record_req->buf_ptr = (uint64_t)(uintptr_t)buf;
544 record_req->buf_size = size;
545 record_req->cper_start_id = start_cper_id + *offset;
546 record_req->cper_num = total_cper_num;
547 r = amdgpu_ras_mgr_handle_ras_cmd(ring->adev, RAS_CMD__GET_CPER_RECORD,
548 record_req, sizeof(struct ras_cmd_cper_record_req),
549 record_rsp, sizeof(struct ras_cmd_cper_record_rsp));
550 if (r)
551 return r;
552
553 r = *offset ? record_rsp->real_data_size : record_rsp->real_data_size + ring_header_size;
554 (*offset) += record_rsp->real_cper_num;
555
556 return r;
557}
558
493/* Layout of file is 12 bytes consisting of559/* Layout of file is 12 bytes consisting of
494 * - rptr560 * - rptr
495 * - wptr561 * - wptr
@@ -506,6 +572,9 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf, · 10 unmodified lines
506 loff_t i;572 loff_t i;
507 int r;573 int r;
508 574
575 if (ring->funcs->type == AMDGPU_RING_TYPE_CPER && amdgpu_uniras_enabled(ring->adev))
576 return amdgpu_ras_cper_debugfs_read(f, buf, size, pos);
577
509 if (*pos & 3 || size & 3)578 if (*pos & 3 || size & 3)
510 return -EINVAL;579 return -EINVAL;
511 580
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h+4−5
@@ -83,6 +83,7 @@ enum amdgpu_ring_type { · 82 unmodified lines
83 AMDGPU_RING_TYPE_MES,83 AMDGPU_RING_TYPE_MES,
84 AMDGPU_RING_TYPE_UMSCH_MM,84 AMDGPU_RING_TYPE_UMSCH_MM,
85 AMDGPU_RING_TYPE_CPER,85 AMDGPU_RING_TYPE_CPER,
86 AMDGPU_RING_TYPE_MAX,
86};87};
87 88
88enum amdgpu_ib_pool_type {89enum amdgpu_ib_pool_type {
@@ -147,16 +148,14 @@ struct amdgpu_fence { · 58 unmodified lines
147 u64 wptr;148 u64 wptr;
148 /* fence context for resets */149 /* fence context for resets */
149 u64 context;150 u64 context;
150 uint32_t seq;
151};151};
152 152
153extern const struct drm_sched_backend_ops amdgpu_sched_ops;153extern const struct drm_sched_backend_ops amdgpu_sched_ops;
154 154
155void amdgpu_fence_driver_clear_job_fences(struct amdgpu_ring *ring);
156void amdgpu_fence_driver_set_error(struct amdgpu_ring *ring, int error);155void amdgpu_fence_driver_set_error(struct amdgpu_ring *ring, int error);
157void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);156void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
158void amdgpu_fence_driver_guilty_force_completion(struct amdgpu_fence *af);157void amdgpu_fence_driver_guilty_force_completion(struct amdgpu_fence *af);
159void amdgpu_fence_save_wptr(struct dma_fence *fence);158void amdgpu_fence_save_wptr(struct amdgpu_fence *af);
160 159
161int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring);160int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring);
162int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring,161int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring,
@@ -166,8 +165,8 @@ void amdgpu_fence_driver_hw_init(struct amdgpu_device *adev); · 3 unmodified lines
166void amdgpu_fence_driver_hw_fini(struct amdgpu_device *adev);165void amdgpu_fence_driver_hw_fini(struct amdgpu_device *adev);
167int amdgpu_fence_driver_sw_init(struct amdgpu_device *adev);166int amdgpu_fence_driver_sw_init(struct amdgpu_device *adev);
168void amdgpu_fence_driver_sw_fini(struct amdgpu_device *adev);167void amdgpu_fence_driver_sw_fini(struct amdgpu_device *adev);
169int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f,168int amdgpu_fence_emit(struct amdgpu_ring *ring, struct amdgpu_fence *af,
170 struct amdgpu_fence *af, unsigned int flags);169 unsigned int flags);
171int amdgpu_fence_emit_polling(struct amdgpu_ring *ring, uint32_t *s,170int amdgpu_fence_emit_polling(struct amdgpu_ring *ring, uint32_t *s,
172 uint32_t timeout);171 uint32_t timeout);
173bool amdgpu_fence_process(struct amdgpu_ring *ring);172bool amdgpu_fence_process(struct amdgpu_ring *ring);
drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c+0−1
@@ -159,7 +159,6 @@ static ssize_t amdgpu_securedisplay_debugfs_write(struct file *f, const char __u · 158 unmodified lines
159 dev_err(adev->dev, "Invalid input: %s\n", str);159 dev_err(adev->dev, "Invalid input: %s\n", str);
160 }160 }
161 161
162 pm_runtime_mark_last_busy(dev->dev);
163 pm_runtime_put_autosuspend(dev->dev);162 pm_runtime_put_autosuspend(dev->dev);
164 163
165 return size;164 return size;
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c+41−81
@@ -188,7 +188,6 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 187 unmodified lines
188 struct amdgpu_job *job;188 struct amdgpu_job *job;
189 void *cpu_addr;189 void *cpu_addr;
190 uint64_t flags;190 uint64_t flags;
191 unsigned int i;
192 int r;191 int r;
193 192
194 BUG_ON(adev->mman.buffer_funcs->copy_max_bytes <193 BUG_ON(adev->mman.buffer_funcs->copy_max_bytes <
@@ -255,16 +254,9 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 60 unmodified lines
255 dma_addr = &bo->ttm->dma_address[mm_cur->start >> PAGE_SHIFT];254 dma_addr = &bo->ttm->dma_address[mm_cur->start >> PAGE_SHIFT];
256 amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags, cpu_addr);255 amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags, cpu_addr);
257 } else {256 } else {
258 dma_addr_t dma_address;257 u64 pa = mm_cur->start + adev->vm_manager.vram_base_offset;
259
260 dma_address = mm_cur->start;
261 dma_address += adev->vm_manager.vram_base_offset;
262 258
263 for (i = 0; i < num_pages; ++i) {259 amdgpu_gart_map_vram_range(adev, pa, 0, num_pages, flags, cpu_addr);
264 amdgpu_gart_map(adev, i << PAGE_SHIFT, 1, &dma_address,
265 flags, cpu_addr);
266 dma_address += PAGE_SIZE;
267 }
268 }260 }
269 261
270 dma_fence_put(amdgpu_job_submit(job));262 dma_fence_put(amdgpu_job_submit(job));
@@ -286,12 +278,13 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 15 unmodified lines
286 * move and different for a BO to BO copy.278 * move and different for a BO to BO copy.
287 *279 *
288 */280 */
289int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,281__attribute__((nonnull))
290 const struct amdgpu_copy_mem *src,282static int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
291 const struct amdgpu_copy_mem *dst,283 const struct amdgpu_copy_mem *src,
292 uint64_t size, bool tmz,284 const struct amdgpu_copy_mem *dst,
293 struct dma_resv *resv,285 uint64_t size, bool tmz,
294 struct dma_fence **f)286 struct dma_resv *resv,
287 struct dma_fence **f)
295{288{
296 struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;289 struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
297 struct amdgpu_res_cursor src_mm, dst_mm;290 struct amdgpu_res_cursor src_mm, dst_mm;
@@ -365,9 +358,7 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev, · 67 unmodified lines
365 }358 }
366error:359error:
367 mutex_unlock(&adev->mman.gtt_window_lock);360 mutex_unlock(&adev->mman.gtt_window_lock);
368 if (f)361 *f = fence;
369 *f = dma_fence_get(fence);
370 dma_fence_put(fence);
371 return r;362 return r;
372}363}
373 364
@@ -706,10 +697,11 @@ struct amdgpu_ttm_tt { · 332 unmodified lines
706 * memory and start HMM tracking CPU page table update697 * memory and start HMM tracking CPU page table update
707 *698 *
708 * Calling function must call amdgpu_ttm_tt_userptr_range_done() once and only699 * Calling function must call amdgpu_ttm_tt_userptr_range_done() once and only
709 * once afterwards to stop HMM tracking700 * once afterwards to stop HMM tracking. Its the caller responsibility to ensure
701 * that range is a valid memory and it is freed too.
710 */702 */
711int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,703int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,
712 struct hmm_range **range)704 struct amdgpu_hmm_range *range)
713{705{
714 struct ttm_tt *ttm = bo->tbo.ttm;706 struct ttm_tt *ttm = bo->tbo.ttm;
715 struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm);707 struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm);
@@ -719,9 +711,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, · 3 unmodified lines
719 bool readonly;711 bool readonly;
720 int r = 0;712 int r = 0;
721 713
722 /* Make sure get_user_pages_done() can cleanup gracefully */
723 *range = NULL;
724
725 mm = bo->notifier.mm;714 mm = bo->notifier.mm;
726 if (unlikely(!mm)) {715 if (unlikely(!mm)) {
727 DRM_DEBUG_DRIVER("BO is not registered?\n");716 DRM_DEBUG_DRIVER("BO is not registered?\n");
@@ -756,38 +745,6 @@ out_unlock: · 28 unmodified lines
756 return r;745 return r;
757}746}
758 747

155 more lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h+4−25

This diff is large: its 64 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c+93−54

This diff is large: its 183 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c+516−124

This diff is large: its 1004 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h+24−5

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c+3−2

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c+108−66

This diff is large: its 265 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_vce.h+3−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h+4−2

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c+316−27

This diff is large: its 466 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h+27−0

This diff is large: its 76 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c+23−16

This diff is large: its 104 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h+0−4

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c+10−27

This diff is large: its 72 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h+76−16

This diff is large: its 120 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/cik_ih.c+12−0

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/cz_ih.c+10−0

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c+7−4

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c+11−14

This diff is large: its 80 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c+4−2

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c+29−0

This diff is large: its 42 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c+29−2

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c+15−7

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c+6−0

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c+6−0

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c+8−2

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/iceland_ih.c+10−0

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/mes_userqueue.c+38−22

This diff is large: its 124 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/mes_v11_0.c+4−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/mes_v12_0.c+4−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/mxgpu_nv.c+28−12

This diff is large: its 76 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/nbio_v7_9.c+13−11

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c+6−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c+5−3

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c+3−2

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/si.c+11−11

This diff is large: its 64 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/si_ih.c+12−0

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/sid.h+0−40

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/soc15.c+0−4

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/tonga_ih.c+10−0

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/umc_v12_0.c+14−0

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/vce_v1_0.c+839−0

This diff is large: its 840 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/vce_v1_0.h+32−0

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/vce_v2_0.c+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/vce_v3_0.c+5−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/amd/amdgpu/vce_v4_0.c+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_chardev.c+8−4

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c+8−4

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_events.c+0−11

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c+6−1

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_migrate.c+0−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_migrate.h+0−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_process.c+8−4

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_svm.c+13−7

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_svm.h+0−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/amdkfd/kfd_topology.c+3−1

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/Makefile+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c+281−44

This diff is large: its 512 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c+748−20

This diff is large: its 860 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_colorop.c+209−0

This diff is large: its 210 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_colorop.h+36−0

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c+12−14

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c+3−2

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c+9−9

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c+17−0

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c+39−0

This diff is large: its 61 lines aren’t shown here.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_replay.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/Makefile+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/bios/bios_parser.c+89−6

This diff is large: its 159 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c+4−4

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/bios/command_table.c+286−0

This diff is large: its 316 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/bios/command_table.h+6−0

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn314/dcn314_clk_mgr.c+4−4

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.c+85−2

This diff is large: its 134 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn315/dcn315_clk_mgr.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/clk_mgr/dcn35/dcn35_clk_mgr.c+34−4

This diff is large: its 65 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc.c+918−205

This diff is large: its 1556 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c+3010−205

This diff is large: its 3339 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c+2−2

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_resource.c+112−17

This diff is large: its 263 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_state.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/core/dc_stream.c+45−11

This diff is large: its 116 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc.h+555−26

This diff is large: its 699 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_bios_types.h+9−0

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c+126−6

This diff is large: its 199 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h+43−0

This diff is large: its 61 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_dp_types.h+36−0

This diff is large: its 57 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_spl_translate.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_stream.h+5−6

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dc_types.h+23−1

This diff is large: its 80 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn20/dcn20_dccg.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn20/dcn20_dccg.h+63−1

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn31/dcn31_dccg.c+123−0

This diff is large: its 137 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn31/dcn31_dccg.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn314/dcn314_dccg.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn314/dcn314_dccg.h+1−2

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn35/dcn35_dccg.c+20−4

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn35/dcn35_dccg.h+11−2

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dccg/dcn401/dcn401_dccg.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_abm.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_audio.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c+3−3

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c+85−0

This diff is large: its 169 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.h+9−7

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_stream_encoder.c+14−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dce_stream_encoder.h+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dmub_hw_lock_mgr.c+27−6

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dmub_hw_lock_mgr.h+11−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dmub_replay.c+18−2

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dce/dmub_replay.h+3−2

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2/Makefile+0−141

This diff is large: its 142 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/Makefile+140−0

This diff is large: its 141 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/cmntypes.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/cmntypes.h+9−9

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_core.crenamed from drivers/gpu/drm/amd/display/dc/dml2/display_mode_core.c+2−0

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_core.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/display_mode_core.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_core_structs.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/display_mode_core_structs.h+1−2

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_lib_defines.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/display_mode_lib_defines.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_util.crenamed from drivers/gpu/drm/amd/display/dc/dml2/display_mode_util.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/display_mode_util.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/display_mode_util.h+0−2

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_translation_helper.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/dml21_translation_helper.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_translation_helper.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/dml21_translation_helper.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_utils.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/dml21_utils.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_utils.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/dml21_utils.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_wrapper.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/dml21_wrapper.c+0−4

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/dml21_wrapper.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/dml21_wrapper.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/bounding_boxes/dcn4_soc_bb.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/inc/bounding_boxes/dcn4_soc_bb.h+0−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml2_external_lib_deps.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/inc/dml2_external_lib_deps.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/inc/dml_top.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_dchub_registers.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/inc/dml_top_dchub_registers.h+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_display_cfg_types.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/inc/dml_top_display_cfg_types.h+20−3

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_policy_types.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/inc/dml_top_policy_types.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_soc_parameter_types.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/inc/dml_top_soc_parameter_types.h+8−3

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/inc/dml_top_types.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/inc/dml_top_types.h+7−0

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_dcn4.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_dcn4.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4_calcs.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_dcn4_calcs.c+75−48

This diff is large: its 375 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4_calcs.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_dcn4_calcs.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_factory.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_factory.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_factory.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_factory.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_shared_types.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_shared_types.h+30−15

This diff is large: its 212 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_utils.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_utils.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_utils.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_core/dml2_core_utils.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_dpmm/dml2_dpmm_dcn4.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_dpmm/dml2_dpmm_dcn4.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_dpmm/dml2_dpmm_dcn4.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_dpmm/dml2_dpmm_dcn4.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_dpmm/dml2_dpmm_factory.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_dpmm/dml2_dpmm_factory.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_dpmm/dml2_dpmm_factory.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_dpmm/dml2_dpmm_factory.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_mcg/dml2_mcg_dcn4.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_mcg/dml2_mcg_dcn4.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_mcg/dml2_mcg_dcn4.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_mcg/dml2_mcg_dcn4.h+1−1

This diff is large: its 6 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_mcg/dml2_mcg_factory.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_mcg/dml2_mcg_factory.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_mcg/dml2_mcg_factory.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_mcg/dml2_mcg_factory.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_dcn3.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_pmo/dml2_pmo_dcn3.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_dcn3.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_pmo/dml2_pmo_dcn3.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_dcn4_fams2.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_pmo/dml2_pmo_dcn4_fams2.c+248−229

This diff is large: its 850 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_dcn4_fams2.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_pmo/dml2_pmo_dcn4_fams2.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_factory.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_pmo/dml2_pmo_factory.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_pmo/dml2_pmo_factory.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_pmo/dml2_pmo_factory.h+1−1

This diff is large: its 6 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_standalone_libraries/lib_float_math.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_standalone_libraries/lib_float_math.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_standalone_libraries/lib_float_math.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_standalone_libraries/lib_float_math.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_interfaces.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_top/dml2_top_interfaces.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_legacy.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_top/dml2_top_legacy.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_legacy.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_top/dml2_top_legacy.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_soc15.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_top/dml2_top_soc15.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_top/dml2_top_soc15.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/dml2_top/dml2_top_soc15.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/inc/dml2_debug.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/inc/dml2_debug.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/inc/dml2_internal_shared_types.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml21/src/inc/dml2_internal_shared_types.h+49−27

This diff is large: its 159 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_dc_resource_mgmt.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_dc_resource_mgmt.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_dc_resource_mgmt.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_dc_resource_mgmt.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_dc_types.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_dc_types.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_internal_types.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_internal_types.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_mall_phantom.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_mall_phantom.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_mall_phantom.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_mall_phantom.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_policy.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_policy.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_policy.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_policy.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_translation_helper.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_translation_helper.c+3−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_translation_helper.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_translation_helper.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_utils.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_utils.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_utils.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_utils.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_wrapper.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_wrapper.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml2_wrapper.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml2_wrapper.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml_assert.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml_assert.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml_depedencies.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml_depedencies.h+1−0

This diff is large: its 5 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dml2_0/dml_display_rq_dlg_calc.crenamed from drivers/gpu/drm/amd/display/dc/dml2/dml_display_rq_dlg_calc.c+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml_display_rq_dlg_calc.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml_display_rq_dlg_calc.h+0−0

File renamed without changes.

drivers/gpu/drm/amd/display/dc/dml2_0/dml_logging.hrenamed from drivers/gpu/drm/amd/display/dc/dml2/dml_logging.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dpp/dcn10/dcn10_dpp.c+14−5

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dpp/dcn10/dcn10_dpp.h+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dpp/dcn30/dcn30_dpp.c+28−8

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dpp/dcn30/dcn30_dpp.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dpp/dcn32/dcn32_dpp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dpp/dcn35/dcn35_dpp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dpp/dcn401/dcn401_dpp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dpp/dcn401/dcn401_dpp_cm.c+28−15

This diff is large: its 68 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dsc/dcn20/dcn20_dsc.c+9−1

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dsc/dcn20/dcn20_dsc.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dsc/dcn35/dcn35_dsc.c+31−1

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dsc/dcn401/dcn401_dsc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/dsc/dsc.h+6−0

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubbub/dcn30/dcn30_hubbub.c+7−26

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubbub/dcn30/dcn30_hubbub.h+1−5

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubbub/dcn31/dcn31_hubbub.c+4−4

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubbub/dcn32/dcn32_hubbub.c+1−2

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubbub/dcn35/dcn35_hubbub.c+51−2

This diff is large: its 67 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubbub/dcn35/dcn35_hubbub.h+1−0

This diff is large: its 6 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubbub/dcn401/dcn401_hubbub.c+1−2

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn10/dcn10_hubp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn10/dcn10_hubp.h+133−3

This diff is large: its 164 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn20/dcn20_hubp.c+38−31

This diff is large: its 89 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn20/dcn20_hubp.h+6−2

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn21/dcn21_hubp.c+0−2

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn30/dcn30_hubp.c+121−26

This diff is large: its 168 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn30/dcn30_hubp.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn31/dcn31_hubp.c+1−3

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn32/dcn32_hubp.c+44−30

This diff is large: its 96 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn35/dcn35_hubp.c+1−3

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn401/dcn401_hubp.c+14−14

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hubp/dcn401/dcn401_hubp.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dce110/dce110_hwseq.c+86−3

This diff is large: its 160 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn10/dcn10_hwseq.c+4−1

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn20/dcn20_hwseq.c+4−1

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn30/dcn30_hwseq.c+40−36

This diff is large: its 93 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn31/dcn31_hwseq.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn314/dcn314_hwseq.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn32/dcn32_hwseq.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn35/dcn35_hwseq.c+139−5

This diff is large: its 183 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn35/dcn35_hwseq.h+8−0

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn35/dcn35_init.c+6−0

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn401/dcn401_hwseq.c+1390−17

This diff is large: its 1501 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn401/dcn401_hwseq.h+102−2

This diff is large: its 138 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/dcn401/dcn401_init.c+26−2

This diff is large: its 110 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h+1440−6

This diff is large: its 1556 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer_private.h+36−0

This diff is large: its 102 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/core_types.h+6−6

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/cursor_reg_cache.h+25−3

This diff is large: its 48 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/dccg.h+120−1

This diff is large: its 142 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/dchubbub.h+41−9

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/dpp.h+16−1

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h+5−2

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h+49−0

This diff is large: its 61 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h+2−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/mem_input.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/mpc.h+27−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/opp.h+13−0

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h+130−0

This diff is large: its 144 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/link_service.h+2−2

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/inc/resource.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/accessories/link_dp_cts.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/hwss/link_hwss_dio.c+12−7

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_detection.c+171−1

This diff is large: its 224 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_dpms.c+10−1

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/link_factory.c+59−7

This diff is large: its 125 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/protocols/link_ddc.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c+50−3

This diff is large: its 116 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.c+15−9

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_irq_handler.c+3−1

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.c+135−8

This diff is large: its 198 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/link/protocols/link_edp_panel_control.h+2−2

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/mpc/dcn30/dcn30_mpc.c+16−0

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/mpc/dcn30/dcn30_mpc.h+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/mpc/dcn32/dcn32_mpc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/mpc/dcn401/dcn401_mpc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/opp/dcn10/dcn10_opp.c+13−1

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/opp/dcn10/dcn10_opp.h+4−2

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/opp/dcn20/dcn20_opp.c+13−0

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/opp/dcn20/dcn20_opp.h+5−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/opp/dcn35/dcn35_opp.c+13−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/opp/dcn35/dcn35_opp.h+3−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/optc/dcn10/dcn10_optc.h+37−1

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.c+131−0

This diff is large: its 145 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dce100/dce100_resource.c+33−5

This diff is large: its 108 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dce110/dce110_resource.c+6−1

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dce112/dce112_resource.c+7−2

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dce120/dce120_resource.c+7−2

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dce60/dce60_resource.c+27−2

This diff is large: its 85 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dce80/dce80_resource.c+31−3

This diff is large: its 111 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn10/dcn10_resource.c+6−1

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn20/dcn20_resource.c+19−16

This diff is large: its 150 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn201/dcn201_resource.c+19−17

This diff is large: its 151 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn21/dcn21_resource.c+5−1

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn30/dcn30_resource.c+5−1

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn301/dcn301_resource.c+5−1

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn302/dcn302_resource.c+5−1

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn303/dcn303_resource.c+5−1

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn31/dcn31_resource.c+5−1

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn314/dcn314_resource.c+5−1

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn315/dcn315_resource.c+6−1

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn316/dcn316_resource.c+5−1

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c+9−4

This diff is large: its 51 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn321/dcn321_resource.c+5−1

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn35/dcn35_resource.c+6−2

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn351/dcn351_resource.c+6−2

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn36/dcn36_resource.c+6−2

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.c+7−3

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/soc_and_ip_translator/dcn401/dcn401_soc_and_ip_translator.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/sspl/dc_spl.c+15−0

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/amd/display/dc/sspl/dc_spl_types.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/dmub_srv.h+31−1

This diff is large: its 87 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h+708−11

This diff is large: its 879 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_dcn31.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_dcn32.c+31−19

This diff is large: its 106 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_dcn35.c+40−0

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_dcn35.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_dcn401.c+14−3

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c+23−10

This diff is large: its 85 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/bios_parser_types.h+6−5

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/dpcd_defs.h+16−0

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/grph_object_ctrl_defs.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/grph_object_id.h+7−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/amd/display/include/signal_types.h+12−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c+3−3

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c+11−2

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_execution.c+38−49

This diff is large: its 131 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_transition.c+32−29

This diff is large: its 107 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/hdcp/hdcp_ddc.c+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.c+2−120

This diff is large: its 132 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/hdcp/hdcp_log.h+6−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h+82−66

This diff is large: its 182 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/power/power_helpers.c+33−0

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/amd/display/modules/power/power_helpers.h+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/amd/include/amd_shared.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/include/asic_reg/vce/vce_1_0_d.h+5−0

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/amd/include/asic_reg/vce/vce_1_0_sh_mask.h+10−0

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/include/kgd_pp_interface.h+124−1

This diff is large: its 151 lines aren’t shown here.

drivers/gpu/drm/amd/include/mes_v11_api_def.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/include/mes_v12_api_def.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/amd/pm/amdgpu_dpm.c+18−4

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/amd/pm/amdgpu_pm.c+21−12

This diff is large: its 105 lines aren’t shown here.

drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h+2−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.c+14−14

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.h+0−557

This diff is large: its 675 lines aren’t shown here.

drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c+5−5

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/amd/pm/powerplay/smumgr/smu10_smumgr.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/pm/powerplay/smumgr/vega10_smumgr.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/pm/powerplay/smumgr/vega12_smumgr.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/amd/pm/powerplay/smumgr/vega20_smumgr.c+4−4

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c+56−39

This diff is large: its 187 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h+20−1

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v13_0_12_pmfw.h+7−2

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/inc/pmfw_if/smu_v13_0_12_ppsmc.h+9−11

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/inc/smu_types.h+12−2

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu11/cyan_skillfish_ppt.c+3−2

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c+9−8

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c+3−2

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c+6−1

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c+11−12

This diff is large: its 87 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu12/renoir_ppt.c+4−3

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c+4−3

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_12_ppt.c+193−56

This diff is large: its 337 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_4_ppt.c+3−2

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_5_ppt.c+3−2

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.c+235−91

This diff is large: its 581 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.h+164−1

This diff is large: its 194 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c+4−3

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu13/yellow_carp_ppt.c+3−2

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_0_ppt.c+3−2

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c+4−3

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c+9−5

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu_cmn.h+67−0

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/amd/pm/swsmu/smu_internal.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/amd/ras/Makefile+34−0

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/Makefile+33−0

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_cmd.c+285−0

This diff is large: its 286 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_cmd.h+54−0

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_eeprom_i2c.c+182−0

This diff is large: its 183 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_eeprom_i2c.h+27−0

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_mgr.c+648−0

This diff is large: its 649 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_mgr.h+83−0

This diff is large: its 84 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_mp1_v13_0.c+94−0

This diff is large: its 95 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_mp1_v13_0.h+30−0

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_nbio_v7_9.c+125−0

This diff is large: its 126 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_nbio_v7_9.h+30−0

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_process.c+190−0

This diff is large: its 191 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_process.h+41−0

This diff is large: its 42 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/amdgpu_ras_sys.c+279−0

This diff is large: its 280 lines aren’t shown here.

drivers/gpu/drm/amd/ras/ras_mgr/ras_sys.h+110−0

This diff is large: its 111 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/Makefile+44−0

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras.h+370−0

This diff is large: its 371 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_aca.c+672−0

This diff is large: its 673 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_aca.h+164−0

This diff is large: its 165 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_aca_v1_0.c+379−0

This diff is large: its 380 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_aca_v1_0.h+71−0

This diff is large: its 72 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_cmd.c+522−0

This diff is large: its 523 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_cmd.h+426−0

This diff is large: its 427 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_core.c+603−0

This diff is large: its 604 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_cper.c+315−0

This diff is large: its 316 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_cper.h+304−0

This diff is large: its 305 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_eeprom.c+1339−0

This diff is large: its 1340 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_eeprom.h+197−0

This diff is large: its 198 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_gfx.c+70−0

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_gfx.h+43−0

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_gfx_v9_0.c+426−0

This diff is large: its 427 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_gfx_v9_0.h+259−0

This diff is large: its 260 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_log_ring.c+317−0

This diff is large: its 318 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_log_ring.h+93−0

This diff is large: its 94 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_mp1.c+81−0

This diff is large: its 82 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_mp1.h+50−0

This diff is large: its 51 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_mp1_v13_0.c+105−0

This diff is large: its 106 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_mp1_v13_0.h+30−0

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_nbio.c+96−0

This diff is large: its 97 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_nbio.h+46−0

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_nbio_v7_9.c+123−0

This diff is large: its 124 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_nbio_v7_9.h+31−0

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_process.c+322−0

This diff is large: its 323 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_process.h+53−0

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_psp.c+750−0

This diff is large: its 751 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_psp.h+145−0

This diff is large: its 146 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_psp_v13_0.c+46−0

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_psp_v13_0.h+31−0

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_ta_if.h+231−0

This diff is large: its 232 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_umc.c+707−0

This diff is large: its 708 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_umc.h+166−0

This diff is large: its 167 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_umc_v12_0.c+511−0

This diff is large: its 512 lines aren’t shown here.

drivers/gpu/drm/amd/ras/rascore/ras_umc_v12_0.h+314−0

This diff is large: its 315 lines aren’t shown here.

drivers/gpu/drm/arm/display/komeda/komeda_crtc.c+18−13

This diff is large: its 130 lines aren’t shown here.

drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/arm/hdlcd_crtc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/arm/hdlcd_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/arm/malidp_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/arm/malidp_mw.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/arm/malidp_planes.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/armada/armada_crtc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/armada/armada_debugfs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/armada/armada_fb.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/armada/armada_fbdev.c+2−13

This diff is large: its 51 lines aren’t shown here.

drivers/gpu/drm/armada/armada_gem.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/armada/armada_overlay.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/armada/armada_plane.c+2−6

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/ast/Makefile+2−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/ast/ast_2000.c+108−0

This diff is large: its 119 lines aren’t shown here.

drivers/gpu/drm/ast/ast_2100.c+92−0

This diff is large: its 103 lines aren’t shown here.

drivers/gpu/drm/ast/ast_2200.c+92−0

This diff is large: its 93 lines aren’t shown here.

drivers/gpu/drm/ast/ast_2300.c+135−0

This diff is large: its 146 lines aren’t shown here.

drivers/gpu/drm/ast/ast_2400.c+100−0

This diff is large: its 101 lines aren’t shown here.

drivers/gpu/drm/ast/ast_2500.c+106−0

This diff is large: its 118 lines aren’t shown here.

drivers/gpu/drm/ast/ast_2600.c+72−0

This diff is large: its 83 lines aren’t shown here.

drivers/gpu/drm/ast/ast_drv.c+66−3

This diff is large: its 106 lines aren’t shown here.

drivers/gpu/drm/ast/ast_drv.h+92−8

This diff is large: its 146 lines aren’t shown here.

drivers/gpu/drm/ast/ast_main.c+0−268

This diff is large: its 269 lines aren’t shown here.

drivers/gpu/drm/ast/ast_mode.c+24−34

This diff is large: its 122 lines aren’t shown here.

drivers/gpu/drm/ast/ast_tables.h+0−60

This diff is large: its 67 lines aren’t shown here.

drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_crtc.c+11−10

This diff is large: its 84 lines aren’t shown here.

drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c+8−7

This diff is large: its 65 lines aren’t shown here.

drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_output.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_plane.c+43−10

This diff is large: its 127 lines aren’t shown here.

drivers/gpu/drm/bridge/imx/Kconfig+11−0

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/bridge/imx/Makefile+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/bridge/imx/imx8mp-hdmi-pai.c+158−0

This diff is large: its 159 lines aren’t shown here.

drivers/gpu/drm/bridge/imx/imx8mp-hdmi-tx.c+60−5

This diff is large: its 104 lines aren’t shown here.

drivers/gpu/drm/bridge/imx/imx8qxp-ldb.c+1−6

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/bridge/ite-it66121.c+33−35

This diff is large: its 194 lines aren’t shown here.

drivers/gpu/drm/bridge/simple-bridge.c+10−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/bridge/synopsys/Kconfig+8−0

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/bridge/synopsys/dw-dp.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/bridge/synopsys/dw-hdmi-gp-audio.c+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/bridge/synopsys/dw-hdmi-qp.c+230−5

This diff is large: its 320 lines aren’t shown here.

drivers/gpu/drm/bridge/synopsys/dw-hdmi-qp.h+14−0

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/bridge/synopsys/dw-hdmi.c+17−1

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/bridge/ti-sn65dsi86.c+112−0

This diff is large: its 200 lines aren’t shown here.

drivers/gpu/drm/clients/drm_fbdev_client.c+23−14

This diff is large: its 79 lines aren’t shown here.

drivers/gpu/drm/clients/drm_log.c+32−11

This diff is large: its 115 lines aren’t shown here.

drivers/gpu/drm/display/drm_bridge_connector.c+48−21

This diff is large: its 165 lines aren’t shown here.

drivers/gpu/drm/display/drm_dp_helper.c+213−19

This diff is large: its 299 lines aren’t shown here.

drivers/gpu/drm/drm_atomic.c+200−25

This diff is large: its 430 lines aren’t shown here.

drivers/gpu/drm/drm_atomic_helper.c+19−5

This diff is large: its 76 lines aren’t shown here.

drivers/gpu/drm/drm_atomic_state_helper.c+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/drm_atomic_uapi.c+160−0

This diff is large: its 230 lines aren’t shown here.

drivers/gpu/drm/drm_bridge.c+52−15

This diff is large: its 189 lines aren’t shown here.

drivers/gpu/drm/drm_buddy.c+239−156

This diff is large: its 669 lines aren’t shown here.

drivers/gpu/drm/drm_client.c+96−102

This diff is large: its 325 lines aren’t shown here.

drivers/gpu/drm/drm_client_event.c+15−14

This diff is large: its 104 lines aren’t shown here.

drivers/gpu/drm/drm_client_modeset.c+44−0

This diff is large: its 51 lines aren’t shown here.

drivers/gpu/drm/drm_client_sysrq.c+65−0

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/drm_color_mgmt.c+43−0

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/drm_colorop.c+599−0

This diff is large: its 600 lines aren’t shown here.

drivers/gpu/drm/drm_connector.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/drm_crtc.c+35−0

This diff is large: its 49 lines aren’t shown here.

drivers/gpu/drm/drm_crtc_internal.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/drm_displayid.c+43−15

This diff is large: its 110 lines aren’t shown here.

drivers/gpu/drm/drm_displayid_internal.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/drm_drv.c+3−0

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/drm_dumb_buffers.c+160−11

This diff is large: its 202 lines aren’t shown here.

drivers/gpu/drm/drm_edid.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/drm_fb_helper.c+25−113

This diff is large: its 262 lines aren’t shown here.

drivers/gpu/drm/drm_fbdev_dma.c+7−18

This diff is large: its 86 lines aren’t shown here.

drivers/gpu/drm/drm_fbdev_shmem.c+6−15

This diff is large: its 75 lines aren’t shown here.

drivers/gpu/drm/drm_fbdev_ttm.c+7−17

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/drm/drm_file.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/drm_format_helper.c+0−91

This diff is large: its 98 lines aren’t shown here.

drivers/gpu/drm/drm_framebuffer.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/drm_gem.c+9−19

This diff is large: its 64 lines aren’t shown here.

drivers/gpu/drm/drm_gem_atomic_helper.c+0−2

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/drm_gem_dma_helper.c+7−3

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/drm_gem_framebuffer_helper.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/drm_gem_shmem_helper.c+73−41

This diff is large: its 172 lines aren’t shown here.

drivers/gpu/drm/drm_gem_ttm_helper.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/drm_gem_vram_helper.c+6−5

This diff is large: its 49 lines aren’t shown here.

drivers/gpu/drm/drm_gpusvm.c+4−2

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/drm_gpuvm.c+191−0

This diff is large: its 254 lines aren’t shown here.

drivers/gpu/drm/drm_internal.h+11−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/drm_ioctl.c+7−0

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/drm_mipi_dbi.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/drm_mm.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/drm_mode_config.c+7−0

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/drm/drm_mode_object.c+18−0

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/drm_modeset_helper.c+3−3

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/drm_plane.c+59−0

This diff is large: its 63 lines aren’t shown here.

drivers/gpu/drm/drm_prime.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/drm_vblank.c+171−3

This diff is large: its 199 lines aren’t shown here.

drivers/gpu/drm/drm_vblank_helper.c+176−0

This diff is large: its 177 lines aren’t shown here.

drivers/gpu/drm/drm_vblank_work.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/etnaviv/etnaviv_buffer.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/etnaviv/etnaviv_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/etnaviv/etnaviv_gem.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/etnaviv/etnaviv_gpu.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/etnaviv/etnaviv_hwdb.c+32−0

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/etnaviv/etnaviv_mmu.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos5433_drm_decon.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos7_drm_decon.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_drm_fb.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_drm_fbdev.c+2−10

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_drm_fimd.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_drm_g2d.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_drm_gem.c+6−3

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_drm_ipp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_drm_plane.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_drm_vidi.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/exynos/exynos_mixer.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_plane.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/backlight.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/gma500/cdv_device.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/cdv_intel_display.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/cdv_intel_dp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/cdv_intel_hdmi.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/cdv_intel_lvds.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/fbdev.c+1−57

This diff is large: its 99 lines aren’t shown here.

drivers/gpu/drm/gma500/gem.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/intel_bios.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/intel_gmbus.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/gma500/mid_bios.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/oaktrail_crtc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/oaktrail_hdmi.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/oaktrail_hdmi_i2c.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/gma500/oaktrail_lvds.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/opregion.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/gma500/psb_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/psb_intel_display.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/psb_intel_lvds.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/psb_intel_sdvo.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/gma500/psb_irq.c+24−13

This diff is large: its 89 lines aren’t shown here.

drivers/gpu/drm/gud/gud_connector.c+4−4

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/gud/gud_drv.c+24−21

This diff is large: its 100 lines aren’t shown here.

drivers/gpu/drm/gud/gud_pipe.c+6−6

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/hisilicon/kirin/kirin_drm_ade.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/hyperv/hyperv_drm_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/hyperv/hyperv_drm_modeset.c+12−0

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/i915/Makefile+13−0

This diff is large: its 72 lines aren’t shown here.

drivers/gpu/drm/i915/display/g4x_dp.c+1−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/display/hsw_ips.c+29−34

This diff is large: its 117 lines aren’t shown here.

drivers/gpu/drm/i915/display/i9xx_plane.c+15−19

This diff is large: its 107 lines aren’t shown here.

drivers/gpu/drm/i915/display/i9xx_plane.h+3−2

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/display/i9xx_wm.c+5−4

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/i915/display/icl_dsi.c+18−21

This diff is large: its 109 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_acpi.c+1−1

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_alpm.c+39−52

This diff is large: its 277 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_alpm.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_backlight.c+1−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_bios.c+4−5

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_bo.c+30−10

This diff is large: its 58 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_bo.h+4−4

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_bw.c+45−310

This diff is large: its 476 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_bw.h+1−5

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_casf.c+290−0

This diff is large: its 291 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_casf.h+21−0

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_casf_regs.h+33−0

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_cdclk.c+373−139

This diff is large: its 911 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_cdclk.h+12−4

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_color.c+9−8

This diff is large: its 49 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_combo_phy.c+1−1

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_connector.c+2−1

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_crt.c+8−8

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_crtc.c+97−2

This diff is large: its 124 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_crtc.h+11−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_crtc_state_dump.c+10−6

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_cursor.c+26−6

This diff is large: its 70 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_cursor.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_cx0_phy.c+117−132

This diff is large: its 531 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_cx0_phy.h+21−0

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_cx0_phy_regs.h+20−12

This diff is large: its 79 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dbuf_bw.c+295−0

This diff is large: its 296 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dbuf_bw.h+37−0

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_ddi.c+53−24

This diff is large: its 185 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_ddi_buf_trans.c+81−2

This diff is large: its 114 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_ddi_buf_trans.h+9−0

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_de.h+54−53

This diff is large: its 171 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display.c+216−146

This diff is large: its 745 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display.h+8−8

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_conversion.c+13−7

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_core.h+30−4

This diff is large: its 81 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_debugfs.c+9−3

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_device.c+5−1

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_device.h+7−2

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_driver.c+10−13

This diff is large: its 68 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_irq.c+103−26

This diff is large: its 264 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_irq.h+5−3

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_jiffies.h+43−0

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_power.c+12−9

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_power_map.c+14−12

This diff is large: its 75 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_power_well.c+39−19

This diff is large: its 140 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_reset.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_rpm.c+13−20

This diff is large: its 83 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_types.h+61−12

This diff is large: its 169 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_utils.c+32−0

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_utils.h+31−0

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_wa.c+4−1

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_display_wa.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dmc.c+42−20

This diff is large: its 161 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dmc_wl.c+12−13

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dp.c+237−26

This diff is large: its 398 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dp.h+11−1

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dp_aux.c+4−4

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dp_hdcp.c+7−7

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dp_link_training.c+2−1

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dp_mst.c+33−16

This diff is large: its 116 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dpio_phy.c+6−6

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dpll.c+32−3

This diff is large: its 77 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dpll_mgr.c+14−15

This diff is large: its 96 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dpll_mgr.h+11−0

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dpt.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dsb.c+33−24

This diff is large: its 83 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dsb.h+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dsi_vbt.c+24−13

This diff is large: its 90 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_dvo.c+1−1

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fb.c+21−20

This diff is large: its 182 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fb_bo.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fb_bo.h+1−2

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fb_pin.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fbc.c+145−58

This diff is large: its 434 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fbc.h+3−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fbdev.c+60−12

This diff is large: its 114 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fbdev_fb.c+20−38

This diff is large: its 101 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fbdev_fb.h+8−6

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_fdi.c+1−1

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_flipq.c+7−7

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_frontbuffer.c+23−129

This diff is large: its 231 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_frontbuffer.h+6−12

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_gmbus.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_hdcp.c+24−27

This diff is large: its 116 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_hdcp_gsc.c+1−1

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_hdmi.c+17−3

This diff is large: its 48 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_hdmi.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_hotplug.c+3−2

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_hotplug_irq.c+4−1

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_link_bw.c+9−8

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_link_bw.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_lspcon.c+1−1

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_lt_phy.c+2327−0

This diff is large: its 2328 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_lt_phy.h+47−0

This diff is large: its 48 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_lt_phy_regs.h+90−0

This diff is large: its 91 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_lvds.c+3−3

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_modeset_setup.c+11−3

This diff is large: its 51 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_modeset_verify.c+2−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_overlay.c+2−5

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_pch.c+2−2

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_pch_display.c+6−6

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_pch_refclk.c+5−7

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_pfit.c+1−1

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_pipe_crc.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_plane.c+13−52

This diff is large: its 126 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_plane.h+0−3

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_plane_initial.c+9−4

This diff is large: its 49 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_pmdemand.c+11−10

This diff is large: its 48 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_pps.c+2−1

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_psr.c+329−110

This diff is large: its 699 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_psr.h+4−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_qp_tables.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_sbi.c+4−2

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_snps_hdmi_pll.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_snps_phy.c+5−5

This diff is large: its 43 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_sprite.c+5−7

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_tc.c+18−5

This diff is large: its 58 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_tc.h+3−0

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_vblank.c+14−2

This diff is large: its 42 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_vblank.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_vdsc.c+25−1

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_vdsc.h+3−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_vrr.c+312−186

This diff is large: its 752 lines aren’t shown here.

drivers/gpu/drm/i915/display/intel_vrr.h+3−2

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_prefill.c+157−0

This diff is large: its 158 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_prefill.h+46−0

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_scaler.c+221−13

This diff is large: its 319 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_scaler.h+17−0

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_universal_plane.c+109−84

This diff is large: its 421 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_universal_plane_regs.h+18−6

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_watermark.c+213−168

This diff is large: its 596 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_watermark.h+3−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/display/skl_watermark_regs.h+28−24

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/i915/display/vlv_clock.c+88−0

This diff is large: its 89 lines aren’t shown here.

drivers/gpu/drm/i915/display/vlv_clock.h+38−0

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/i915/display/vlv_dsi.c+28−28

This diff is large: its 165 lines aren’t shown here.

drivers/gpu/drm/i915/display/vlv_dsi_pll.c+4−4

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_context.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_create.c+3−2

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_mman.c+3−2

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_object.c+9−8

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_object.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_object_frontbuffer.c+103−0

This diff is large: its 104 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_object_frontbuffer.h+18−36

This diff is large: its 87 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_object_types.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_pages.c+3−1

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_phys.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_shmem.c+12−3

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_shrinker.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_stolen.c+78−27

This diff is large: its 191 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_stolen.h+16−18

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_tiling.c+3−2

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_ttm.c+5−3

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_userptr.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gem_wait.c+7−0

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/i915/gem/i915_gemfs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c+2−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/gen2_engine_cs.c+4−4

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/i915/gt/gen8_engine_cs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_breadcrumbs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c+3−0

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_engine_user.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_execlists_submission.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_ggtt.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_ggtt_fencing.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_ggtt_gmch.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_gt_debugfs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_lrc.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_mocs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_rc6.c+4−1

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_region_lmem.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_renderstate.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_rps.c+8−10

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_sa_media.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_sseu.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_timeline.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_tlb.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_wopcm.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/intel_workarounds.c+5−4

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/i915/gt/selftest_context.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gt/selftest_execlists.c+3−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/gt/selftest_migrate.c+8−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_gsc_uc_heci_cmd_submit.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/aperture_gm.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/cfg_space.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/cmd_parser.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/display.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/dmabuf.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/edid.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/gtt.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/handlers.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/interrupt.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/kvmgt.c+19−17

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/mmio.c+5−2

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/mmio_context.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/scheduler.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/vgpu.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_cmd_parser.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/i915_config.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_debugfs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/i915_driver.c+87−68

This diff is large: its 306 lines aren’t shown here.

drivers/gpu/drm/i915/i915_driver.h+2−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/i915_drv.h+6−17

This diff is large: its 61 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gem.c+3−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/i915_getparam.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_irq.c+27−81

This diff is large: its 216 lines aren’t shown here.

drivers/gpu/drm/i915/i915_jiffies.h+16−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/i915_mmio_range.c+18−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/i915_mmio_range.h+19−0

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/i915/i915_module.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/i915_perf.c+28−39

This diff is large: its 187 lines aren’t shown here.

drivers/gpu/drm/i915/i915_pmu.c+4−2

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/i915/i915_query.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_reg.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/i915_reg_defs.h+10−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/i915_request.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_switcheroo.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_sysfs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_ttm_buddy_manager.c+2−2

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/i915/i915_utils.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/i915_utils.h+4−37

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/drm/i915/i915_vgpu.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_vma.c+6−4

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/i915/intel_clock_gating.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_gvt.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_memory_region.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/intel_pcode.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_region_ttm.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_runtime_pm.c+77−0

This diff is large: its 91 lines aren’t shown here.

drivers/gpu/drm/i915/intel_runtime_pm.h+3−0

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/i915/intel_step.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_uncore.c+9−7

This diff is large: its 79 lines aren’t shown here.

drivers/gpu/drm/i915/intel_uncore.h+1−7

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/i915/intel_wakeref.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/pxp/intel_pxp.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/pxp/intel_pxp_gsccs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/pxp/intel_pxp_huc.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/pxp/intel_pxp_session.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_active.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_gem_gtt.c+4−0

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_request.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_selftest.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/intel_uncore.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/mock_gem_device.c+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/i915/soc/intel_dram.c+10−5

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/i915/soc/intel_dram.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/soc/intel_gmch.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/soc/intel_rom.c+4−3

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/i915/soc/intel_rom.h+3−3

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/i915/vlv_iosf_sb.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/imagination/pvr_ccb.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/imagination/pvr_device.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/imagination/pvr_fw.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/imagination/pvr_fw_meta.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/imagination/pvr_fw_trace.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/imagination/pvr_power.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/imagination/pvr_vm.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/imx/dc/dc-ed.c+4−4

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/imx/dc/dc-fg.c+2−2

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/imx/dc/dc-fu.c+5−5

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/imx/dc/dc-fu.h+2−2

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/imx/dc/dc-lb.c+14−14

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/imx/dc/dc-plane.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/imx/dcss/dcss-plane.c+3−2

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/imx/ipuv3/dw_hdmi-imx.c+0−1

This diff is large: its 5 lines aren’t shown here.

drivers/gpu/drm/imx/ipuv3/imx-drm-core.c+25−6

This diff is large: its 51 lines aren’t shown here.

drivers/gpu/drm/imx/ipuv3/imx-ldb.c+0−1

This diff is large: its 5 lines aren’t shown here.

drivers/gpu/drm/imx/ipuv3/imx-tve.c+10−8

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/imx/ipuv3/ipuv3-plane.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/imx/ipuv3/parallel-display.c+2−3

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/imx/lcdc/imx-lcdc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/ingenic/ingenic-drm-drv.c+9−4

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/ingenic/ingenic-ipu.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/kmb/kmb_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/kmb/kmb_plane.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/lima/lima_sched.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/logicvc/logicvc_layer.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_benchmark.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_crtc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_debugfs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_gem.c+10−22

This diff is large: its 74 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_i2c.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_irq.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_output_7a1000.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_output_7a2000.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_pixpll.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_plane.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/loongson/lsdc_ttm.c+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/mcde/mcde_clk_div.c+8−5

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/mcde/mcde_display.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mediatek/Kconfig+22−1

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/mediatek/Makefile+3−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_crtc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_ddp_comp.c+28−5

This diff is large: its 67 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_ddp_comp.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_disp_ccorr.c+1−22

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_disp_ovl_adaptor.c+12−0

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_dp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_drm_drv.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_gem.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_hdmi.c+21−518

This diff is large: its 639 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_hdmi_common.c+456−0

This diff is large: its 457 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_hdmi_common.h+198−0

This diff is large: its 199 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_hdmi_ddc_v2.c+396−0

This diff is large: its 397 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_hdmi_regs_v2.h+263−0

This diff is large: its 264 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_hdmi_v2.c+1521−0

This diff is large: its 1522 lines aren’t shown here.

drivers/gpu/drm/mediatek/mtk_plane.c+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/meson/meson_overlay.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/meson/meson_plane.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200eh.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200eh3.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200eh5.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200er.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200ev.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200ew3.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200se.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_g200wb.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_vga.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mgag200/mgag200_vga_bmc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/msm/Makefile+2−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a2xx_catalog.c+4−3

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a2xx_gpu.c+27−25

This diff is large: its 87 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a2xx_gpu.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a3xx_catalog.c+7−6

This diff is large: its 62 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a3xx_gpu.c+27−25

This diff is large: its 83 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a3xx_gpu.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a4xx_catalog.c+4−3

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a4xx_gpu.c+28−26

This diff is large: its 78 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a4xx_gpu.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a5xx_catalog.c+9−8

This diff is large: its 80 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a5xx_gpu.c+31−30

This diff is large: its 86 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a5xx_gpu.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_catalog.c+362−23

This diff is large: its 630 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_gmu.c+248−82

This diff is large: its 714 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_gmu.h+15−10

This diff is large: its 77 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_gpu.c+273−165

This diff is large: its 728 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_gpu.h+30−1

This diff is large: its 64 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c+3−2

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_gpu_state.h+37−37

This diff is large: its 123 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_hfi.c+71−3

This diff is large: its 101 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a6xx_hfi.h+17−0

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/a8xx_gpu.c+1201−0

This diff is large: its 1202 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/adreno_device.c+3−1

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/adreno_gen7_0_0_snapshot.h+210−210

This diff is large: its 820 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/adreno_gen7_2_0_snapshot.h+166−166

This diff is large: its 437 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/adreno_gen7_9_0_snapshot.h+235−235

This diff is large: its 958 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/adreno_gpu.c+12−0

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/msm/adreno/adreno_gpu.h+46−10

This diff is large: its 129 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/catalog/dpu_12_2_glymur.h+541−0

This diff is large: its 542 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c+73−42

This diff is large: its 234 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h+5−3

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c+20−27

This diff is large: its 110 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c+1−0

This diff is large: its 5 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_hw_dsc.h+0−6

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_hw_mdss.h+8−0

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c+264−164

This diff is large: its 652 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h+4−8

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_rm.c+5−1

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h+5−5

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/msm/disp/mdp4/mdp4_crtc.c+2−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/msm/disp/mdp5/mdp5_crtc.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c+3−4

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/msm/disp/msm_disp_snapshot.h+7−6

This diff is large: its 63 lines aren’t shown here.

drivers/gpu/drm/msm/dp/dp_ctrl.c+5−5

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/msm/dp/dp_display.c+9−0

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/msm/dp/dp_link.c+117−0

This diff is large: its 147 lines aren’t shown here.

drivers/gpu/drm/msm/dp/dp_link.h+5−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/msm/dp/dp_panel.c+9−69

This diff is large: its 133 lines aren’t shown here.

drivers/gpu/drm/msm/dp/dp_panel.h+0−3

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/msm/msm_fbdev.c+1−10

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/msm/msm_gem.c+28−2

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/msm/msm_gem_vma.c+22−6

This diff is large: its 113 lines aren’t shown here.

drivers/gpu/drm/msm/msm_gpu.c+13−8

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/msm/msm_mdss.c+2−0

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/msm/registers/adreno/a6xx.xml+1646−553

This diff is large: its 3321 lines aren’t shown here.

drivers/gpu/drm/msm/registers/adreno/a6xx_enums.xml+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/msm/registers/adreno/a6xx_gmu.xml+153−130

This diff is large: its 339 lines aren’t shown here.

drivers/gpu/drm/msm/registers/adreno/a7xx_enums.xml+0−7

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/msm/registers/adreno/a8xx_descriptors.xml+121−0

This diff is large: its 122 lines aren’t shown here.

drivers/gpu/drm/msm/registers/adreno/a8xx_enums.xml+299−0

This diff is large: its 300 lines aren’t shown here.

drivers/gpu/drm/msm/registers/adreno/adreno_common.xml+12−0

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/msm/registers/adreno/adreno_pm4.xml+247−114

This diff is large: its 488 lines aren’t shown here.

drivers/gpu/drm/msm/registers/gen_header.py+12−7

This diff is large: its 59 lines aren’t shown here.

drivers/gpu/drm/mxsfb/lcdif_kms.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/mxsfb/mxsfb_kms.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/nouveau/Kconfig+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/nouveau/include/nvfw/hs.h+3−1

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/nouveau/include/nvkm/core/tegra.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/nouveau/include/nvkm/subdev/clk.h+1−0

This diff is large: its 6 lines aren’t shown here.

drivers/gpu/drm/nouveau/nouveau_bo.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/nouveau/nouveau_display.c+6−5

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/nouveau/nouveau_drv.h+4−1

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/nouveau/nouveau_gem.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/nouveau/nouveau_platform.c+20−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/nouveau/nouveau_ttm.c+4−2

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/nouveau/nouveau_uvmm.c+85−17

This diff is large: its 246 lines aren’t shown here.

drivers/gpu/drm/nouveau/nouveau_uvmm.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/engine/device/base.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/engine/device/tegra.c+4−0

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/clk/Kbuild+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a.c+5−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a_devfreq.c+320−0

This diff is large: its 321 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a_devfreq.h+24−0

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/clk/gm20b.c+5−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/clk/gp10b.c+185−0

This diff is large: its 186 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/clk/gp10b.h+18−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/fb/gb100.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/fb/gb202.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/fb/gh100.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c+40−29

This diff is large: its 128 lines aren’t shown here.

drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/nova/Kconfig+2−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_crtc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_debugfs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_dmm_tiler.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_encoder.c+1−3

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_fb.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_fbdev.c+2−10

This diff is large: its 42 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_gem.c+8−8

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_irq.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_overlay.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/omapdrm/omap_plane.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/panel/Kconfig+56−3

This diff is large: its 101 lines aren’t shown here.

drivers/gpu/drm/panel/Makefile+4−0

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/panel/panel-edp.c+10−0

This diff is large: its 67 lines aren’t shown here.

drivers/gpu/drm/panel/panel-ilitek-ili9881c.c+883−444

This diff is large: its 1687 lines aren’t shown here.

drivers/gpu/drm/panel/panel-ilitek-ili9882t.c+69−0

This diff is large: its 104 lines aren’t shown here.

drivers/gpu/drm/panel/panel-jadard-jd9365da-h3.c+9−12

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/panel/panel-lg-ld070wx3.c+184−0

This diff is large: its 185 lines aren’t shown here.

drivers/gpu/drm/panel/panel-newvision-nv3052c.c+74−334

This diff is large: its 457 lines aren’t shown here.

drivers/gpu/drm/panel/panel-ronbo-rb070d30.c+4−4

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/panel/panel-samsung-s6e3fc2x01.c+385−0

This diff is large: its 386 lines aren’t shown here.

drivers/gpu/drm/panel/panel-samsung-sofef00.c+67−38

This diff is large: its 246 lines aren’t shown here.

drivers/gpu/drm/panel/panel-sharp-lq079l1sx01.c+225−0

This diff is large: its 226 lines aren’t shown here.

drivers/gpu/drm/panel/panel-simple.c+96−31

This diff is large: its 183 lines aren’t shown here.

drivers/gpu/drm/panel/panel-synaptics-tddi.c+277−0

This diff is large: its 278 lines aren’t shown here.

drivers/gpu/drm/panel/panel-visionox-rm69299.c+69−2

This diff is large: its 120 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_devfreq.c+4−2

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_device.c+36−32

This diff is large: its 230 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_device.h+16−8

This diff is large: its 81 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_drv.c+183−60

This diff is large: its 467 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_dump.c+4−4

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_gem.c+5−4

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_gpu.c+40−26

This diff is large: its 202 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_job.c+239−97

This diff is large: its 617 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_job.h+31−7

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_mmu.c+81−34

This diff is large: its 311 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_mmu.h+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/panfrost/panfrost_perfcnt.c+15−11

This diff is large: its 97 lines aren’t shown here.

drivers/gpu/drm/panthor/Makefile+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_devfreq.c+50−14

This diff is large: its 124 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_devfreq.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_device.c+37−6

This diff is large: its 151 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_device.h+22−3

This diff is large: its 60 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_drv.c+11−2

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_fw.c+115−19

This diff is large: its 224 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_fw.h+30−2

This diff is large: its 62 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_gem.c+7−14

This diff is large: its 58 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_gpu.c+34−4

This diff is large: its 84 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_gpu.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_heap.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_hw.c+104−5

This diff is large: its 155 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_hw.h+46−1

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_mmu.c+50−96

This diff is large: its 306 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_pwr.c+549−0

This diff is large: its 550 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_pwr.h+23−0

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_regs.h+83−0

This diff is large: its 109 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_sched.c+261−103

This diff is large: its 627 lines aren’t shown here.

drivers/gpu/drm/panthor/panthor_sched.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/pl111/pl111_display.c+9−5

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_cmd.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_debugfs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_display.c+30−0

This diff is large: its 74 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_gem.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_image.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_ioctl.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_irq.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_kms.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_release.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/qxl/qxl_ttm.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_acpi.c+0−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_connectors.c+5−15

This diff is large: its 60 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_device.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_display.c+0−2

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_drv.c+56−32

This diff is large: its 141 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_fbdev.c+1−16

This diff is large: its 59 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_gem.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_kms.c+0−4

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/radeon/radeon_ttm.c+4−2

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/renesas/rcar-du/rcar_du_crtc.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/renesas/rcar-du/rcar_du_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/renesas/rcar-du/rcar_du_kms.c+5−2

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/renesas/rcar-du/rcar_mipi_dsi.c+34−16

This diff is large: its 109 lines aren’t shown here.

drivers/gpu/drm/renesas/rcar-du/rcar_mipi_dsi_regs.h+203−184

This diff is large: its 496 lines aren’t shown here.

drivers/gpu/drm/renesas/rz-du/Kconfig+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/renesas/rz-du/rzg2l_du_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/rockchip/analogix_dp-rockchip.c+17−25

This diff is large: its 63 lines aren’t shown here.

drivers/gpu/drm/rockchip/cdn-dp-core.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/rockchip/cdn-dp-reg.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/rockchip/dw-mipi-dsi-rockchip.c+21−0

This diff is large: its 49 lines aren’t shown here.

drivers/gpu/drm/rockchip/dw_hdmi_qp-rockchip.c+119−76

This diff is large: its 328 lines aren’t shown here.

drivers/gpu/drm/rockchip/inno_hdmi.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/rockchip/rk3066_hdmi.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/rockchip/rockchip_drm_drv.c+4−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/rockchip/rockchip_drm_gem.c+7−6

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/rockchip/rockchip_drm_vop.c+4−3

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/rockchip/rockchip_drm_vop2.c+80−62

This diff is large: its 272 lines aren’t shown here.

drivers/gpu/drm/rockchip/rockchip_lvds.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/rockchip/rockchip_rgb.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/rockchip/rockchip_vop2_reg.c+42−7

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/rockchip/rockchip_vop_reg.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/scheduler/sched_main.c+16−4

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/drm/scheduler/tests/sched_tests.h+1−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/sitronix/st7571-i2c.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/sitronix/st7586.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sitronix/st7735r.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/solomon/ssd130x.c+36−51

This diff is large: its 237 lines aren’t shown here.

drivers/gpu/drm/sti/sti_cursor.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sti/sti_drv.c+6−13

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/sti/sti_gdp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sti/sti_hda.c+5−0

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/sti/sti_hdmi.c+2−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/sti/sti_hqvdp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sti/sti_plane.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/stm/drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/stm/dw_mipi_dsi-stm.c+8−6

This diff is large: its 48 lines aren’t shown here.

drivers/gpu/drm/stm/ltdc.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/stm/lvds.c+7−5

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun4i_backend.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun4i_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun4i_frontend.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun4i_hdmi_ddc_clk.c+7−5

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun4i_tcon_dclk.c+10−8

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_csc.c+61−52

This diff is large: its 194 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_csc.h+4−12

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_mixer.c+142−76

This diff is large: its 365 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_mixer.h+43−22

This diff is large: its 107 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_ui_layer.c+76−111

This diff is large: its 324 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_ui_layer.h+5−2

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_ui_scaler.c+19−25

This diff is large: its 98 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_ui_scaler.h+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_vi_layer.c+93−155

This diff is large: its 418 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_vi_layer.h+5−2

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_vi_scaler.c+26−25

This diff is large: its 120 lines aren’t shown here.

drivers/gpu/drm/sun4i/sun8i_vi_scaler.h+4−2

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/sysfb/drm_sysfb_helper.h+32−2

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/sysfb/drm_sysfb_modeset.c+149−4

This diff is large: its 193 lines aren’t shown here.

drivers/gpu/drm/sysfb/efidrm.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sysfb/ofdrm.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/sysfb/simpledrm.c+3−1

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/sysfb/vesadrm.c+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/tegra/Makefile+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tegra/dc.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/tegra/drm.c+3−0

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/tegra/drm.h+1−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/tegra/dsi.c+29−27

This diff is large: its 91 lines aren’t shown here.

drivers/gpu/drm/tegra/fb.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tegra/fbdev.c+1−10

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/tegra/gem.c+5−3

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/tegra/hdmi.c+3−2

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/tegra/hub.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tegra/nvjpg.c+330−0

This diff is large: its 331 lines aren’t shown here.

drivers/gpu/drm/tegra/sor.c+3−2

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/tests/.kunitconfig+2−0

This diff is large: its 6 lines aren’t shown here.

drivers/gpu/drm/tests/Makefile+2−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/tests/drm_buddy_test.c+105−0

This diff is large: its 119 lines aren’t shown here.

drivers/gpu/drm/tests/drm_fixp_test.c+71−0

This diff is large: its 72 lines aren’t shown here.

drivers/gpu/drm/tests/drm_mm_test.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tidss/tidss_crtc.c+24−18

This diff is large: its 115 lines aren’t shown here.

drivers/gpu/drm/tidss/tidss_dispc.c+49−77

This diff is large: its 261 lines aren’t shown here.

drivers/gpu/drm/tidss/tidss_dispc.h+1−5

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/tidss/tidss_drv.c+0−16

This diff is large: its 72 lines aren’t shown here.

drivers/gpu/drm/tidss/tidss_drv.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/tidss/tidss_kms.c+0−4

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/tidss/tidss_oldi.c+22−0

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/drm/tidss/tidss_plane.c+0−8

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/tilcdc/tilcdc_crtc.c+1−8

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/tilcdc/tilcdc_plane.c+1−2

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/tiny/bochs.c+11−0

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/tiny/cirrus-qemu.c+12−0

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/tiny/gm12u320.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/hx8357d.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/ili9163.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/ili9225.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/ili9341.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/ili9486.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/mi0283qt.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/panel-mipi-dbi.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/pixpaper.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/tiny/repaper.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/ttm/tests/ttm_bo_test.c+14−14

This diff is large: its 119 lines aren’t shown here.

drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c+37−36

This diff is large: its 257 lines aren’t shown here.

drivers/gpu/drm/ttm/tests/ttm_device_test.c+13−20

This diff is large: its 109 lines aren’t shown here.

drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.c+9−13

This diff is large: its 59 lines aren’t shown here.

drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.h+2−5

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/ttm/tests/ttm_mock_manager.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/ttm/tests/ttm_pool_test.c+12−12

This diff is large: its 97 lines aren’t shown here.

drivers/gpu/drm/ttm/tests/ttm_resource_test.c+3−2

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_bo.c+35−32

This diff is large: its 120 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_bo_internal.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_bo_util.c+31−7

This diff is large: its 60 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_device.c+5−4

This diff is large: its 44 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_module.c+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_pool.c+26−19

This diff is large: its 167 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_pool_internal.h+25−0

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_resource.c+24−13

This diff is large: its 80 lines aren’t shown here.

drivers/gpu/drm/ttm/ttm_tt.c+7−4

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/tve200/tve200_display.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/udl/udl_edid.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/v3d/v3d_bo.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/v3d/v3d_debugfs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/v3d/v3d_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/v3d/v3d_gem.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/v3d/v3d_gemfs.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/v3d/v3d_irq.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/v3d/v3d_sched.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/v3d/v3d_submit.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vboxvideo/vbox_irq.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vboxvideo/vbox_main.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vboxvideo/vbox_mode.c+5−4

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/vboxvideo/vbox_ttm.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/Kconfig+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_bo.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_debugfs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_dpi.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_dsi.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_gem.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_hdmi.c+55−83

This diff is large: its 270 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_hdmi.h+0−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_hvs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_irq.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_kms.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_perfmon.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_plane.c+3−4

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_render_cl.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_txp.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_v3d.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_validate.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_validate_shaders.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/vc4/vc4_vec.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vgem/vgem_fence.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/virtio/virtgpu_debugfs.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/virtio/virtgpu_display.c+33−4

This diff is large: its 103 lines aren’t shown here.

drivers/gpu/drm/virtio/virtgpu_drv.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/virtio/virtgpu_kms.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/virtio/virtgpu_object.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/virtio/virtgpu_plane.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/virtio/virtgpu_vq.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vkms/Kconfig+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vkms/Makefile+4−1

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/vkms/tests/Makefile+2−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/vkms/tests/vkms_color_test.c+414−0

This diff is large: its 415 lines aren’t shown here.

drivers/gpu/drm/vkms/tests/vkms_config_test.c+54−17

This diff is large: its 166 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_colorop.c+120−0

This diff is large: its 121 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_composer.c+119−17

This diff is large: its 188 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_composer.h+28−0

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_config.c+12−3

This diff is large: its 64 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_config.h+53−1

This diff is large: its 101 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_configfs.c+843−0

This diff is large: its 844 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_configfs.h+8−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_connector.c+35−0

This diff is large: its 49 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_connector.h+9−0

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_crtc.c+10−78

This diff is large: its 148 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_drv.c+24−3

This diff is large: its 81 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_drv.h+30−4

This diff is large: its 76 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_luts.c+811−0

This diff is large: its 812 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_luts.h+12−0

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_output.c+2−5

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_plane.c+8−2

This diff is large: its 43 lines aren’t shown here.

drivers/gpu/drm/vkms/vkms_writeback.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vmwgfx/vmwgfx_drv.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/vmwgfx/vmwgfx_drv.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/vmwgfx/vmwgfx_gem.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/vmwgfx/vmwgfx_kms.c+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/vmwgfx/vmwgfx_surface.c+4−17

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/vmwgfx/vmwgfx_vkms.c+2−4

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/xe/Kconfig.debug+8−8

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/xe/Makefile+17−3

This diff is large: its 99 lines aren’t shown here.

drivers/gpu/drm/xe/abi/guc_actions_abi.h+0−8

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/xe/compat-i915-headers/gem/i915_gem_object.h+1−3

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/xe/compat-i915-headers/gem/i915_gem_stolen.h+30−75

This diff is large: its 111 lines aren’t shown here.

drivers/gpu/drm/xe/compat-i915-headers/i915_drv.h+0−4

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/xe/compat-i915-headers/i915_scheduler_types.h+0−13

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/xe/compat-i915-headers/i915_utils.h+8−1

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/xe/compat-i915-headers/i915_vma.h+0−2

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/compat-i915-headers/intel_uncore.h+10−21

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/xe/display/ext/i915_utils.c+0−27

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/xe/display/intel_bo.c+51−9

This diff is large: its 85 lines aren’t shown here.

drivers/gpu/drm/xe/display/intel_fb_bo.c+1−2

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/xe/display/intel_fbdev_fb.c+24−46

This diff is large: its 104 lines aren’t shown here.

drivers/gpu/drm/xe/display/xe_display.c+16−7

This diff is large: its 87 lines aren’t shown here.

drivers/gpu/drm/xe/display/xe_display.h+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/display/xe_display_rpm.c+31−30

This diff is large: its 105 lines aren’t shown here.

drivers/gpu/drm/xe/display/xe_display_rpm.h+11−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/xe/display/xe_display_wa.c+2−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/display/xe_panic.c+36−14

This diff is large: its 86 lines aren’t shown here.

drivers/gpu/drm/xe/display/xe_plane_initial.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/display/xe_stolen.c+123−0

This diff is large: its 124 lines aren’t shown here.

drivers/gpu/drm/xe/instructions/xe_gpu_commands.h+6−0

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/xe/regs/xe_engine_regs.h+4−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/regs/xe_gt_regs.h+25−5

This diff is large: its 79 lines aren’t shown here.

drivers/gpu/drm/xe/regs/xe_i2c_regs.h+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/xe/regs/xe_irq_regs.h+6−2

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/xe/regs/xe_pmt.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xe/regs/xe_regs.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/tests/xe_dma_buf.c+15−2

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/xe/tests/xe_gt_sriov_pf_config_kunit.c+208−0

This diff is large: its 209 lines aren’t shown here.

drivers/gpu/drm/xe/tests/xe_pci.c+4−2

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/xe/tests/xe_pci_test.c+11−5

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/xe/tests/xe_rtp_test.c+4−2

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/xe/xe_bo.c+83−34

This diff is large: its 252 lines aren’t shown here.

drivers/gpu/drm/xe/xe_bo.h+3−1

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/xe/xe_bo_doc.h+4−4

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/xe/xe_bo_evict.c+9−0

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/xe/xe_configfs.c+282−1

This diff is large: its 432 lines aren’t shown here.

drivers/gpu/drm/xe/xe_configfs.h+8−0

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/xe/xe_debugfs.c+7−9

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/xe/xe_device.c+48−33

This diff is large: its 163 lines aren’t shown here.

drivers/gpu/drm/xe/xe_device_sysfs.c+1−9

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/xe/xe_device_types.h+36−19

This diff is large: its 145 lines aren’t shown here.

drivers/gpu/drm/xe/xe_device_wa_oob.rules+3−0

This diff is large: its 6 lines aren’t shown here.

drivers/gpu/drm/xe/xe_dma_buf.c+26−15

This diff is large: its 59 lines aren’t shown here.

drivers/gpu/drm/xe/xe_eu_stall.c+30−6

This diff is large: its 77 lines aren’t shown here.

drivers/gpu/drm/xe/xe_exec.c+12−16

This diff is large: its 95 lines aren’t shown here.

drivers/gpu/drm/xe/xe_exec_queue.c+160−69

This diff is large: its 326 lines aren’t shown here.

drivers/gpu/drm/xe/xe_exec_queue.h+22−6

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/xe/xe_exec_queue_types.h+13−0

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/xe/xe_execlist.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_force_wake_types.h+24−2

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/drm/xe/xe_ggtt.c+149−25

This diff is large: its 263 lines aren’t shown here.

drivers/gpu/drm/xe/xe_ggtt.h+3−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/xe/xe_ggtt_types.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gpu_scheduler.c+14−13

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gpu_scheduler.h+22−7

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gsc.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt.c+46−50

This diff is large: its 214 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt.h+19−0

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_clock.c+0−19

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_debugfs.c+45−114

This diff is large: its 270 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_debugfs.h+1−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_freq.c+17−20

This diff is large: its 56 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_mcr.c+74−6

This diff is large: its 136 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_pagefault.c+0−679

This diff is large: its 680 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_pagefault.h+0−19

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf.c+8−28

This diff is large: its 48 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c+308−41

This diff is large: its 546 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_config.h+16−0

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_control.c+705−45

This diff is large: its 952 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_control.h+12−0

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_control_types.h+32−4

This diff is large: its 85 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c+245−216

This diff is large: its 606 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.c+829−188

This diff is large: its 1225 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.h+39−9

This diff is large: its 56 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_migration_types.h+11−23

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c+20−1

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_pf_types.h+2−3

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_printk.h+5−2

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_vf.c+361−104

This diff is large: its 593 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_vf.h+6−5

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h+26−8

This diff is large: its 61 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_throttle.c+187−168

This diff is large: its 429 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_topology.c+21−3

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_topology.h+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/xe_gt_types.h+11−68

This diff is large: its 114 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guard.h+119−0

This diff is large: its 120 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc.c+176−152

This diff is large: its 438 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_ads.c+6−2

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_ads_types.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_buf.c+46−11

This diff is large: its 89 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_buf.h+2−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_capture.c+28−1

This diff is large: its 64 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_ct.c+136−241

This diff is large: its 568 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_ct.h+11−2

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_ct_types.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_exec_queue_types.h+15−0

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_fwif.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_log_types.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_pagefault.c+95−0

This diff is large: its 96 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_pagefault.h+15−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_pc.c+65−82

This diff is large: its 287 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_pc_types.h+0−4

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_relay.c+15−2

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_relay_types.h+4−0

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_submit.c+444−162

This diff is large: its 937 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_submit.h+2−5

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/xe/xe_guc_tlb_inval.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_heci_gsc.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_huc.c+7−3

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/xe/xe_hw_engine.c+48−14

This diff is large: its 102 lines aren’t shown here.

drivers/gpu/drm/xe/xe_hwmon.c+0−8

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/xe/xe_i2c.c+27−1

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/drm/xe/xe_i2c.h+4−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/xe/xe_irq.c+94−44

This diff is large: its 190 lines aren’t shown here.

drivers/gpu/drm/xe/xe_lmtt.c+5−6

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/xe/xe_lrc.c+13−7

This diff is large: its 57 lines aren’t shown here.

drivers/gpu/drm/xe/xe_lrc.h+14−2

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/xe/xe_map.h+2−20

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/xe/xe_memirq.c+48−9

This diff is large: its 110 lines aren’t shown here.

drivers/gpu/drm/xe/xe_memirq.h+2−0

This diff is large: its 7 lines aren’t shown here.

drivers/gpu/drm/xe/xe_migrate.c+323−46

This diff is large: its 563 lines aren’t shown here.

drivers/gpu/drm/xe/xe_migrate.h+16−0

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/xe/xe_migrate_doc.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_mmio.c+29−0

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/xe/xe_mmio.h+4−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_mocs.c+40−2

This diff is large: its 82 lines aren’t shown here.

drivers/gpu/drm/xe/xe_mocs.h+1−7

This diff is large: its 14 lines aren’t shown here.

drivers/gpu/drm/xe/xe_oa.c+11−9

This diff is large: its 64 lines aren’t shown here.

drivers/gpu/drm/xe/xe_oa_types.h+3−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pagefault.c+445−0

This diff is large: its 446 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pagefault.h+19−0

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pagefault_types.h+136−0

This diff is large: its 137 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pat.c+126−19

This diff is large: its 304 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pat.h+6−6

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pci.c+206−78

This diff is large: its 591 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pci_sriov.c+56−59

This diff is large: its 189 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pci_sriov.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pci_types.h+6−5

This diff is large: its 43 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pcode.c+26−14

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pcode_api.h+0−6

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/xe/xe_platform_types.h+3−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pm.c+66−0

This diff is large: its 131 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pm.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pmu.c+10−1

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/xe/xe_preempt_fence.c+11−0

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/xe/xe_preempt_fence_types.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_psmi.c+2−2

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/xe/xe_pt.c+42−68

This diff is large: its 260 lines aren’t shown here.

drivers/gpu/drm/xe/xe_query.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_range_fence.h+2−2

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/xe/xe_reg_whitelist.c+9−1

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/xe/xe_ring_ops.c+19−4

This diff is large: its 109 lines aren’t shown here.

drivers/gpu/drm/xe/xe_rtp.c+20−11

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/xe/xe_rtp.h+25−5

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/drm/xe/xe_rtp_types.h+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sa.c+21−0

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sa.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sched_job.c+5−20

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sched_job.h+6−7

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sched_job_types.h+10−1

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_packet.c+520−0

This diff is large: its 521 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_packet.h+30−0

This diff is large: its 31 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_packet_types.h+75−0

This diff is large: its 76 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf.c+134−41

This diff is large: its 226 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf.h+8−14

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_control.c+279−0

This diff is large: its 280 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_control.h+22−0

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_debugfs.c+395−0

This diff is large: its 396 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_debugfs.h+18−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_helpers.h+27−0

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_migration.c+340−0

This diff is large: its 341 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_migration.h+29−0

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_migration_types.h+37−0

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_provision.c+438−0

This diff is large: its 439 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_provision.h+45−0

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_provision_types.h+36−0

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_sysfs.c+647−0

This diff is large: its 648 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_sysfs.h+16−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_pf_types.h+25−0

This diff is large: its 53 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_printk.h+6−6

This diff is large: its 29 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_vf.c+11−275

This diff is large: its 334 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_vf.h+1−1

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_vf_ccs.c+72−2

This diff is large: its 110 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_vf_ccs.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sriov_vf_types.h+3−7

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/xe/xe_svm.c+2−3

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/xe/xe_sync.c+41−33

This diff is large: its 101 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile.c+9−5

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile_debugfs.c+13−6

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile_debugfs.h+3−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile_sriov_pf_debugfs.c+253−0

This diff is large: its 254 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile_sriov_pf_debugfs.h+15−0

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile_sriov_printk.h+33−0

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile_sriov_vf.c+104−8

This diff is large: its 147 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile_sriov_vf.h+7−2

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h+23−0

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tlb_inval.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tlb_inval_job.c+24−7

This diff is large: its 122 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tlb_inval_job.h+3−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/xe_trace.h+23−0

This diff is large: its 30 lines aren’t shown here.

drivers/gpu/drm/xe/xe_ttm_stolen_mgr.c+5−3

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/xe/xe_ttm_sys_mgr.c+3−3

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/xe/xe_ttm_vram_mgr.c+3−3

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tuning.c+21−6

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/drm/xe/xe_tuning.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xe/xe_uc_fw_types.h+3−3

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/xe/xe_uc_types.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_userptr.c+3−1

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/xe/xe_validation.h+3−3

This diff is large: its 27 lines aren’t shown here.

drivers/gpu/drm/xe/xe_vm.c+90−48

This diff is large: its 320 lines aren’t shown here.

drivers/gpu/drm/xe/xe_vm_doc.h+4−4

This diff is large: its 34 lines aren’t shown here.

drivers/gpu/drm/xe/xe_vm_types.h+3−7

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/xe/xe_vram.c+23−15

This diff is large: its 117 lines aren’t shown here.

drivers/gpu/drm/xe/xe_wa.c+19−8

This diff is large: its 77 lines aren’t shown here.

drivers/gpu/drm/xe/xe_wa.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/xe/xe_wa_oob.rules+6−11

This diff is large: its 58 lines aren’t shown here.

drivers/gpu/drm/xen/xen_drm_front.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xen/xen_drm_front_gem.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xen/xen_drm_front_kms.c+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/xlnx/zynqmp_kms.c+5−2

This diff is large: its 23 lines aren’t shown here.

drivers/gpu/host1x/bus.c+12−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/host1x/dev.c+12−8

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/host1x/dev.h+3−0

This diff is large: its 10 lines aren’t shown here.

drivers/gpu/host1x/hw/channel_hw.c+66−40

This diff is large: its 183 lines aren’t shown here.

drivers/gpu/host1x/hw/intr_hw.c+47−9

This diff is large: its 87 lines aren’t shown here.

drivers/gpu/host1x/syncpt.c+1−3

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/nova-core/bitfield.rs+330−0

This diff is large: its 331 lines aren’t shown here.

drivers/gpu/nova-core/dma.rs+16−20

This diff is large: its 50 lines aren’t shown here.

drivers/gpu/nova-core/driver.rs+24−3

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/nova-core/falcon.rs+166−115

This diff is large: its 455 lines aren’t shown here.

drivers/gpu/nova-core/falcon/gsp.rs+27−2

This diff is large: its 42 lines aren’t shown here.

drivers/gpu/nova-core/falcon/hal.rs+10−4

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/nova-core/falcon/hal/ga102.rs+24−20

This diff is large: its 68 lines aren’t shown here.

drivers/gpu/nova-core/falcon/sec2.rs+8−2

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/nova-core/fb.rs+86−16

This diff is large: its 152 lines aren’t shown here.

drivers/gpu/nova-core/fb/hal.rs+4−2

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/nova-core/fb/hal/ga100.rs+11−5

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/nova-core/fb/hal/ga102.rs+5−3

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/nova-core/fb/hal/tu102.rs+13−12

This diff is large: its 40 lines aren’t shown here.

drivers/gpu/nova-core/firmware.rs+20−13

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/nova-core/firmware/booter.rs+54−28

This diff is large: its 155 lines aren’t shown here.

drivers/gpu/nova-core/firmware/fwsec.rs+105−83

This diff is large: its 314 lines aren’t shown here.

drivers/gpu/nova-core/firmware/gsp.rs+34−19

This diff is large: its 100 lines aren’t shown here.

drivers/gpu/nova-core/firmware/riscv.rs+19−15

This diff is large: its 69 lines aren’t shown here.

drivers/gpu/nova-core/gfw.rs+24−24

This diff is large: its 61 lines aren’t shown here.

drivers/gpu/nova-core/gpu.rs+88−27

This diff is large: its 180 lines aren’t shown here.

drivers/gpu/nova-core/gsp.rs+145−6

This diff is large: its 167 lines aren’t shown here.

drivers/gpu/nova-core/gsp/boot.rs+134−19

This diff is large: its 189 lines aren’t shown here.

drivers/gpu/nova-core/gsp/cmdq.rs+679−0

This diff is large: its 680 lines aren’t shown here.

drivers/gpu/nova-core/gsp/commands.rs+227−0

This diff is large: its 228 lines aren’t shown here.

drivers/gpu/nova-core/gsp/fw.rs+922−1

This diff is large: its 930 lines aren’t shown here.

drivers/gpu/nova-core/gsp/fw/commands.rs+128−0

This diff is large: its 129 lines aren’t shown here.

drivers/gpu/nova-core/gsp/fw/r570_144.rs+4−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs+950−0

This diff is large: its 952 lines aren’t shown here.

drivers/gpu/nova-core/gsp/sequencer.rs+407−0

This diff is large: its 408 lines aren’t shown here.

drivers/gpu/nova-core/nova_core.rs+5−0

This diff is large: its 20 lines aren’t shown here.

drivers/gpu/nova-core/num.rs+217−0

This diff is large: its 218 lines aren’t shown here.

drivers/gpu/nova-core/regs.rs+78−13

This diff is large: its 155 lines aren’t shown here.

drivers/gpu/nova-core/regs/macros.rs+24−263

This diff is large: its 439 lines aren’t shown here.

drivers/gpu/nova-core/sbuffer.rs+227−0

This diff is large: its 228 lines aren’t shown here.

drivers/gpu/nova-core/util.rs+11−22

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/nova-core/vbios.rs+181−242

This diff is large: its 696 lines aren’t shown here.

drivers/video/fbdev/Kconfig+4−4

This diff is large: its 52 lines aren’t shown here.

drivers/video/fbdev/core/Kconfig+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/video/fbdev/core/bitblit.c+61−61

This diff is large: its 196 lines aren’t shown here.

drivers/video/fbdev/core/fbcon.c+236−223

This diff is large: its 1183 lines aren’t shown here.

drivers/video/fbdev/core/fbcon.h+8−9

This diff is large: its 58 lines aren’t shown here.

drivers/video/fbdev/core/fbcon_ccw.c+78−73

This diff is large: its 276 lines aren’t shown here.

drivers/video/fbdev/core/fbcon_cw.c+78−73

This diff is large: its 276 lines aren’t shown here.

drivers/video/fbdev/core/fbcon_rotate.c+15−32

This diff is large: its 87 lines aren’t shown here.

drivers/video/fbdev/core/fbcon_rotate.h+15−3

This diff is large: its 23 lines aren’t shown here.

drivers/video/fbdev/core/fbcon_ud.c+86−81

This diff is large: its 308 lines aren’t shown here.

drivers/video/fbdev/core/softcursor.c+9−9

This diff is large: its 37 lines aren’t shown here.

drivers/video/fbdev/core/tileblit.c+18−14

This diff is large: its 53 lines aren’t shown here.

drivers/video/fbdev/simplefb.c+3−3

This diff is large: its 22 lines aren’t shown here.

include/drm/bridge/dw_hdmi.h+10−1

This diff is large: its 32 lines aren’t shown here.

include/drm/bridge/dw_hdmi_qp.h+6−0

This diff is large: its 13 lines aren’t shown here.

include/drm/display/drm_dp.h+6−0

This diff is large: its 41 lines aren’t shown here.

include/drm/display/drm_dp_helper.h+22−0

This diff is large: its 50 lines aren’t shown here.

include/drm/drm_atomic.h+193−62

This diff is large: its 358 lines aren’t shown here.

include/drm/drm_atomic_uapi.h+3−0

This diff is large: its 17 lines aren’t shown here.

include/drm/drm_bridge.h+57−4

This diff is large: its 88 lines aren’t shown here.

include/drm/drm_buddy.h+9−4

This diff is large: its 42 lines aren’t shown here.

include/drm/drm_client.h+24−29

This diff is large: its 110 lines aren’t shown here.

include/drm/drm_client_event.h+6−6

This diff is large: its 26 lines aren’t shown here.

include/drm/drm_color_mgmt.h+29−0

This diff is large: its 48 lines aren’t shown here.

include/drm/drm_colorop.h+464−0

This diff is large: its 465 lines aren’t shown here.

include/drm/drm_crtc.h+19−1

This diff is large: its 46 lines aren’t shown here.

include/drm/drm_device.h+8−0

This diff is large: its 15 lines aren’t shown here.

include/drm/drm_dumb_buffers.h+14−0

This diff is large: its 15 lines aren’t shown here.

include/drm/drm_edid.h+6−0

This diff is large: its 13 lines aren’t shown here.

include/drm/drm_fb_helper.h+2−18

This diff is large: its 49 lines aren’t shown here.

include/drm/drm_file.h+7−0

This diff is large: its 14 lines aren’t shown here.

include/drm/drm_fixed.h+17−0

This diff is large: its 24 lines aren’t shown here.

include/drm/drm_format_helper.h+0−4

This diff is large: its 11 lines aren’t shown here.

include/drm/drm_gem_shmem_helper.h+2−0

This diff is large: its 13 lines aren’t shown here.

include/drm/drm_gpusvm.h+4−0

This diff is large: its 18 lines aren’t shown here.

include/drm/drm_gpuvm.h+16−0

This diff is large: its 51 lines aren’t shown here.

include/drm/drm_mm.h+1−1

This diff is large: its 9 lines aren’t shown here.

include/drm/drm_mode_config.h+18−0

This diff is large: its 25 lines aren’t shown here.

include/drm/drm_modeset_helper_vtables.h+12−0

This diff is large: its 19 lines aren’t shown here.

include/drm/drm_plane.h+19−0

This diff is large: its 38 lines aren’t shown here.

include/drm/drm_vblank.h+32−0

This diff is large: its 60 lines aren’t shown here.

include/drm/drm_vblank_helper.h+56−0

This diff is large: its 57 lines aren’t shown here.

include/drm/gpu_scheduler.h+1−1

This diff is large: its 9 lines aren’t shown here.

include/drm/intel/display_member.h+42−0

This diff is large: its 43 lines aren’t shown here.

include/drm/intel/display_parent_interface.h+45−0

This diff is large: its 46 lines aren’t shown here.

include/drm/intel/pciids.h+18−2

This diff is large: its 39 lines aren’t shown here.

include/drm/ttm/ttm_allocation.h+12−0

This diff is large: its 13 lines aren’t shown here.

include/drm/ttm/ttm_bo.h+1−1

This diff is large: its 9 lines aren’t shown here.

include/drm/ttm/ttm_device.h+7−1

This diff is large: its 29 lines aren’t shown here.

include/drm/ttm/ttm_pool.h+3−5

This diff is large: its 28 lines aren’t shown here.

include/drm/ttm/ttm_resource.h+23−9

This diff is large: its 76 lines aren’t shown here.

include/linux/dma-buf/heaps/cma.h+16−0

This diff is large: its 17 lines aren’t shown here.

include/sound/asoundef.h+9−0

This diff is large: its 16 lines aren’t shown here.

include/uapi/drm/amdgpu_drm.h+0−9

This diff is large: its 16 lines aren’t shown here.

include/uapi/drm/amdxdna_accel.h+61−0

This diff is large: its 83 lines aren’t shown here.

include/uapi/drm/drm.h+15−0

This diff is large: its 22 lines aren’t shown here.

include/uapi/drm/drm_mode.h+183−1

This diff is large: its 219 lines aren’t shown here.

include/uapi/drm/ethosu_accel.h+261−0

This diff is large: its 262 lines aren’t shown here.

include/uapi/drm/ivpu_accel.h+63−0

This diff is large: its 126 lines aren’t shown here.

include/uapi/drm/panfrost_drm.h+123−27

This diff is large: its 238 lines aren’t shown here.

include/uapi/drm/xe_drm.h+5−1

This diff is large: its 13 lines aren’t shown here.

init/Kconfig+3−0

This diff is large: its 10 lines aren’t shown here.

kernel/dma/contiguous.c+11−0

This diff is large: its 39 lines aren’t shown here.

rust/kernel/drm/gem/mod.rs+24−29

This diff is large: its 94 lines aren’t shown here.

rust/kernel/lib.rs+4−0

This diff is large: its 18 lines aren’t shown here.

rust/kernel/prelude.rs+3−0

This diff is large: its 7 lines aren’t shown here.

rust/kernel/slice.rs+49−0

This diff is large: its 50 lines aren’t shown here.

rust/kernel/transmute.rs+63−0

This diff is large: its 84 lines aren’t shown here.

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t