History

Merge tag 'drm-intel-next-2019-07-30' of git://anongit.freedesktop.org/drm/drm-intel into drm-next

- More changes on simplifying locking mechanisms (Chris) - Selftests fixes and improvements (Chris) - More work around engine tracking for better handling (Chris, Tvrtko) - HDCP debug and info improvements (Ram, Ashuman) - Add DSI properties (Vandita) - Rework on sdvo support for better debuggability before fixing bugs (Ville) - Display PLLs fixes and improvements, specially targeting Ice Lake (Imre, Matt, Ville) - Perf fixes and improvements (Lionel) - Enumerate scratch buffers (Lionel) - Add infra to hold off preemption on a request (Lionel) - Ice Lake color space fixes (Uma) - Type-C fixes and improvements (Lucas) - Fix and improvements around workarounds (Chris, John, Tvrtko) - GuC related fixes and improvements (Chris, Daniele, Michal, Tvrtko) - Fix on VLV/CHV display power domain (Ville) - Improvements around Watermark (Ville) - Favor intel_ types on intel_atomic functions (Ville) - Don’t pass stack garbage to pcode (Ville) - Improve display tracepoints (Steven) - Don’t overestimate 4:2:0 link symbol clock (Ville) - Add support for 4th pipe and transcoder (Lucas) - Introduce initial support for Tiger Lake platform (Daniele, Lucas, Mahesh, Jose, Imre, Mika, Vandita, Rodrigo, Michel) - PPGTT allocation simplification (Chris) - Standardize function names and suffixes to make clean, symmetric and let checkpatch happy (Janusz) - Skip SINK_COUNT read on CH7511 (Ville) - Fix on kernel documentation (Chris, Michal) - Add modular FIA (Anusha, Lucas) - Fix EHL display (Matt, Vivek) - Enable hotplug retry (Imre, Jose) - Disable preemption under GVT (Chris) - OA; Reconfigure context on the fly (Chris) - Fixes and improvements around engine reset. (Chris) - Small clean up on display pipe fault mask (Ville) - Make sure cdclk is high enough for DP audio on VLV/CHV (Ville) - Drop some wmb() and improve pwrite flush (Chris) - Fix critical PSR regression (DK) - Remove unused variables (YueHaibing) - Use dev_get_drvdata for simplification (Chunhong) - Use upstream version of header tests (Jani) drm-intel-next-2019-07-08: - Signal fence completion from i915_request_wait (Chris) - Fixes and improvements around rings pin/unpin (Chris) - Display uncore prep patches (Daniele) - Execlists preemption improvements (Chris) - Selftests fixes and improvements (Chris) - More Elkhartlake enabling work (Vandita, Jose, Matt, Vivek) - Defer address space cleanup to an RCU worker (Chris) - Implicit dev_priv removal and GT compartmentalization and other related follow-ups (Tvrtko, Chris) - Prevent dereference of engine before NULL check in error capture (Chris) - GuC related fixes (Daniele, Robert) - Many changes on active tracking, timelines and locking mechanisms (Chris) - Disable SAMPLER_STATE prefetching on Gen11 (HW W/a) (Kenneth) - I915_perf fixes (Lionel) - Add Ice Lake PCI ID (Mika) - eDP backlight fix (Lee) - Fix various gen2 tracepoints (Ville) - Some irq vfunc clean-up and improvements (Ville) - Move OA files to separated folder (Michal) - Display self contained headers clean-up (Jani) - Preparation for 4th pile (Lucas) - Move atomic commit, watermark and other places to use more intel_crtc_state (Maarten) - Many Ice Lake Type C and Thunderbolt fixes (Imre) - Fix some Ice Lake hw w/a whitelist regs (Lionel) - Fix memleak in runtime wakeref tracking (Mika) - Remove unused Private PPAT manager (Michal) - Don't check PPGTT presence on PPGTT-only platforms (Michal) - Fix ICL DSI suspend/resume (Chris) - Fix ICL Bandwidth issues (Ville) - Add N & CTS values for 10/12 bit deep color (Aditya) - Moving more GT related stuff under gt folder (Chris) - Forcewake related fixes (Chris) - Show support for accurate sw PMU busyness tracking (Chris) - Handle gtt double alloc failures (Chris) - Upgrade to new GuC version (Michal) - Improve w/a debug dumps and pull engine w/a initialization into a common (Chris) - Look for instdone on all engines at hangcheck (Tvrtko) - Engine lookup simplification (Chris) - Many plane color formats fixes and improvements (Ville) - Fix some compilation issues (YueHaibing) - GTT page directory clean up and improvements (Mika) Signed-off-by: Dave Airlie <airlied@redhat.com>

Dave Airlie committed 7 years ago

Commit dce14e32 parents e21a712 e0e712f248 files · +12996 −10415Browse files

Unified · Split

248 files
  1. Documentation/gpu/i915.rst +10 −10
  2. arch/x86/kernel/early-quirks.c +1 −0
  3. drivers/gpu/drm/i915/Kconfig.debug +15 −0
  4. drivers/gpu/drm/i915/Makefile +43 −38
  5. drivers/gpu/drm/i915/Makefile.header-test +0 −22
  6. drivers/gpu/drm/i915/display/Makefile +5 −1
  7. drivers/gpu/drm/i915/display/Makefile.header-test +0 −16
  8. drivers/gpu/drm/i915/display/icl_dsi.c +112 −73
  9. drivers/gpu/drm/i915/display/intel_atomic_plane.c +36 −20
  10. drivers/gpu/drm/i915/display/intel_atomic_plane.h +2 −3
  11. drivers/gpu/drm/i915/display/intel_audio.c +68 −13
  12. drivers/gpu/drm/i915/display/intel_bios.c +24 −4
  13. drivers/gpu/drm/i915/display/intel_bios.h +2 −1
  14. drivers/gpu/drm/i915/display/intel_bw.c +1 −1
  15. drivers/gpu/drm/i915/display/intel_cdclk.c +56 −28
  16. drivers/gpu/drm/i915/display/intel_combo_phy.c +125 −68
  17. drivers/gpu/drm/i915/display/intel_combo_phy.h +2 −2
  18. drivers/gpu/drm/i915/display/intel_connector.c +1 −1
  19. drivers/gpu/drm/i915/display/intel_ddi.c +229 −174
  20. drivers/gpu/drm/i915/display/intel_display.c +734 −511
  21. drivers/gpu/drm/i915/display/intel_display.h +46 −4
  22. drivers/gpu/drm/i915/display/intel_display_power.c +614 −44
  23. drivers/gpu/drm/i915/display/intel_display_power.h +32 −3
  24. drivers/gpu/drm/i915/display/intel_dp.c +45 −247
  25. drivers/gpu/drm/i915/display/intel_dp.h +0 −2
  26. drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c +4 −1
  27. drivers/gpu/drm/i915/display/intel_dp_mst.h +7 −1
  28. drivers/gpu/drm/i915/display/intel_dpll_mgr.c +478 −174
  29. drivers/gpu/drm/i915/display/intel_dpll_mgr.h +44 −13
  30. drivers/gpu/drm/i915/display/intel_dsi.h +9 −3
  31. drivers/gpu/drm/i915/display/intel_gmbus.c +18 −2
  32. drivers/gpu/drm/i915/display/intel_hdcp.c +9 −3
  33. drivers/gpu/drm/i915/display/intel_hdmi.c +42 −33
  34. drivers/gpu/drm/i915/display/intel_hotplug.c +46 −13
  35. drivers/gpu/drm/i915/display/intel_hotplug.h +3 −2
  36. drivers/gpu/drm/i915/display/intel_overlay.c +7 −3
  37. drivers/gpu/drm/i915/display/intel_pipe_crc.c +1 −1
  38. drivers/gpu/drm/i915/display/intel_sdvo.c +172 −142
  39. drivers/gpu/drm/i915/display/intel_sprite.c +252 −83
  40. drivers/gpu/drm/i915/display/intel_tc.c +537 −0
  41. drivers/gpu/drm/i915/display/intel_tc.h +35 −0
  42. drivers/gpu/drm/i915/display/intel_vbt_defs.h +3 −0
  43. drivers/gpu/drm/i915/display/intel_vdsc.c +10 −4
  44. drivers/gpu/drm/i915/display/vlv_dsi.c +2 −2
  45. drivers/gpu/drm/i915/gem/Makefile +5 −1
  46. drivers/gpu/drm/i915/gem/Makefile.header-test +0 −16
  47. drivers/gpu/drm/i915/gem/i915_gem_client_blt.c +10 −11
  48. drivers/gpu/drm/i915/gem/i915_gem_context.c +30 −53
  49. drivers/gpu/drm/i915/gem/i915_gem_context.h +0 −6
  50. drivers/gpu/drm/i915/gem/i915_gem_context_types.h +2 −2
  51. drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +9 −13
  52. drivers/gpu/drm/i915/gem/i915_gem_mman.c +5 −3
  53. drivers/gpu/drm/i915/gem/i915_gem_object.c +46 −51
  54. drivers/gpu/drm/i915/gem/i915_gem_object.h +1 −7
  55. drivers/gpu/drm/i915/gem/i915_gem_object_blt.c +1 −5
  56. drivers/gpu/drm/i915/gem/i915_gem_object_types.h +0 −1
  57. drivers/gpu/drm/i915/gem/i915_gem_phys.c +3 −9
  58. drivers/gpu/drm/i915/gem/i915_gem_pm.c +19 −18
  59. drivers/gpu/drm/i915/gem/i915_gem_shmem.c +7 −0
  60. drivers/gpu/drm/i915/gem/i915_gem_shrinker.c +12 −15
  61. drivers/gpu/drm/i915/gem/i915_gem_stolen.c +0 −2
  62. drivers/gpu/drm/i915/gem/i915_gem_throttle.c +1 −1
  63. drivers/gpu/drm/i915/gem/i915_gem_userptr.c +10 −1
  64. drivers/gpu/drm/i915/gem/selftests/huge_pages.c +48 −21
  65. drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c +6 −20
  66. drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c +7 −2
  67. drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +36 −49
  68. drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +36 −23
  69. drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c +6 −9
  70. drivers/gpu/drm/i915/gt/Makefile +4 −1
  71. drivers/gpu/drm/i915/gt/Makefile.header-test +0 −16
  72. drivers/gpu/drm/i915/gt/gen6_renderstate.c +0 −0
  73. drivers/gpu/drm/i915/gt/gen7_renderstate.c +0 −0
  74. drivers/gpu/drm/i915/gt/gen8_renderstate.c +0 −0
  75. drivers/gpu/drm/i915/gt/gen9_renderstate.c +0 −0
  76. drivers/gpu/drm/i915/gt/intel_context.c +88 −35
  77. drivers/gpu/drm/i915/gt/intel_context.h +16 −2
  78. drivers/gpu/drm/i915/gt/intel_context_types.h +8 −1
  79. drivers/gpu/drm/i915/gt/intel_engine.h +15 −60
  80. drivers/gpu/drm/i915/gt/intel_engine_cs.c +85 −118
  81. drivers/gpu/drm/i915/gt/intel_engine_pm.c +4 −3
  82. drivers/gpu/drm/i915/gt/intel_engine_pm.h +6 −0
  83. drivers/gpu/drm/i915/gt/intel_engine_types.h +26 −48
  84. drivers/gpu/drm/i915/gt/intel_gpu_commands.h +7 −0
  85. drivers/gpu/drm/i915/gt/intel_gt.c +250 −0
  86. drivers/gpu/drm/i915/gt/intel_gt.h +60 −0
  87. drivers/gpu/drm/i915/gt/intel_gt_pm.c +33 −29
  88. drivers/gpu/drm/i915/gt/intel_gt_pm.h +6 −6
  89. drivers/gpu/drm/i915/gt/intel_gt_types.h +96 −0
  90. drivers/gpu/drm/i915/gt/intel_hangcheck.c +42 −29
  91. drivers/gpu/drm/i915/gt/intel_lrc.c +580 −437
  92. drivers/gpu/drm/i915/gt/intel_mocs.c +40 −22
  93. drivers/gpu/drm/i915/gt/intel_mocs.h +4 −2
  94. drivers/gpu/drm/i915/gt/intel_renderstate.c +6 −7
  95. drivers/gpu/drm/i915/gt/intel_renderstate.h +7 −3
  96. drivers/gpu/drm/i915/gt/intel_reset.c +218 −406
  97. drivers/gpu/drm/i915/gt/intel_reset.h +42 −33
  98. drivers/gpu/drm/i915/gt/intel_reset_types.h +50 −0
  99. drivers/gpu/drm/i915/gt/intel_ringbuffer.c +141 −77
  100. drivers/gpu/drm/i915/gt/intel_timeline.c +120 −110
  101. drivers/gpu/drm/i915/gt/intel_timeline.h +93 −0
  102. drivers/gpu/drm/i915/gt/intel_timeline_types.h +4 −4
  103. drivers/gpu/drm/i915/gt/intel_workarounds.c +133 −94
  104. drivers/gpu/drm/i915/gt/intel_workarounds.h +3 −3
  105. drivers/gpu/drm/i915/gt/intel_workarounds_types.h +1 −0
  106. drivers/gpu/drm/i915/gt/mock_engine.c +8 −6
  107. drivers/gpu/drm/i915/gt/selftest_hangcheck.c +252 −259
  108. drivers/gpu/drm/i915/gt/selftest_lrc.c +361 −30
  109. drivers/gpu/drm/i915/gt/selftest_reset.c +96 −37
  110. drivers/gpu/drm/i915/gt/selftest_timeline.c +65 −66
  111. drivers/gpu/drm/i915/gt/selftest_workarounds.c +89 −66
  112. drivers/gpu/drm/i915/gt/selftests/mock_timeline.c +4 −4
  113. drivers/gpu/drm/i915/gt/selftests/mock_timeline.h +3 −3
  114. drivers/gpu/drm/i915/gt/uc/Makefile +5 −0
  115. drivers/gpu/drm/i915/gt/uc/intel_guc.c +118 −172
  116. drivers/gpu/drm/i915/gt/uc/intel_guc.h +17 −16
  117. drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c +15 −14
  118. drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h +0 −0
  119. drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +4 −18
  120. drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h +9 −2
  121. drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c +181 −0
  122. drivers/gpu/drm/i915/gt/uc/intel_guc_fw.h +0 −0
  123. drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h +5 −76
  124. drivers/gpu/drm/i915/gt/uc/intel_guc_log.c +38 −10
  125. drivers/gpu/drm/i915/gt/uc/intel_guc_log.h +0 −1
  126. drivers/gpu/drm/i915/gt/uc/intel_guc_reg.h +22 −16
  127. drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +103 −393
  128. drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h +0 −2
  129. drivers/gpu/drm/i915/gt/uc/intel_huc.c +41 −28
  130. drivers/gpu/drm/i915/gt/uc/intel_huc.h +5 −7
  131. drivers/gpu/drm/i915/gt/uc/intel_huc_fw.c +53 −0
  132. drivers/gpu/drm/i915/gt/uc/intel_huc_fw.h +0 −0
  133. drivers/gpu/drm/i915/gt/uc/intel_uc.c +570 −0
  134. drivers/gpu/drm/i915/gt/uc/intel_uc.h +22 −17
  135. drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +540 −0
  136. drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h +66 −36
  137. drivers/gpu/drm/i915/gt/uc/intel_uc_fw_abi.h +82 −0
  138. drivers/gpu/drm/i915/gt/uc/selftest_guc.c +13 −36
  139. drivers/gpu/drm/i915/gvt/gtt.h +7 −6
  140. drivers/gpu/drm/i915/gvt/scheduler.c +1 −1
  141. drivers/gpu/drm/i915/i915_active.c +215 −106
  142. drivers/gpu/drm/i915/i915_active.h +16 −12
  143. drivers/gpu/drm/i915/i915_active_types.h +11 −2
  144. drivers/gpu/drm/i915/i915_debugfs.c +107 −110
  145. drivers/gpu/drm/i915/i915_drv.c +105 −95
  146. drivers/gpu/drm/i915/i915_drv.h +66 −179
  147. drivers/gpu/drm/i915/i915_fixed.h +5 −0
  148. drivers/gpu/drm/i915/i915_gem.c +119 −199
  149. drivers/gpu/drm/i915/i915_gem_batch_pool.c +17 −25
  150. drivers/gpu/drm/i915/i915_gem_fence_reg.c +32 −0
  151. drivers/gpu/drm/i915/i915_gem_fence_reg.h +3 −0
  152. drivers/gpu/drm/i915/i915_gem_gtt.c +844 −1209
  153. drivers/gpu/drm/i915/i915_gem_gtt.h +53 −147
  154. drivers/gpu/drm/i915/i915_gem_render_state.h +0 −31
  155. drivers/gpu/drm/i915/i915_globals.h +2 −0
  156. drivers/gpu/drm/i915/i915_gpu_error.c +262 −326
  157. drivers/gpu/drm/i915/i915_gpu_error.h +4 −67
  158. drivers/gpu/drm/i915/i915_irq.c +411 −392
  159. drivers/gpu/drm/i915/i915_irq.h +35 −10
  160. drivers/gpu/drm/i915/i915_params.c +3 −2
  161. drivers/gpu/drm/i915/i915_params.h +1 −1
  162. drivers/gpu/drm/i915/i915_pci.c +37 −6
  163. drivers/gpu/drm/i915/i915_perf.c +220 −84
  164. drivers/gpu/drm/i915/i915_pmu.c +1 −3
  165. drivers/gpu/drm/i915/i915_priolist_types.h +10 −0
  166. drivers/gpu/drm/i915/i915_pvinfo.h +5 −2
  167. drivers/gpu/drm/i915/i915_reg.h +158 −73
  168. drivers/gpu/drm/i915/i915_request.c +69 −57
  169. drivers/gpu/drm/i915/i915_request.h +19 −5
  170. drivers/gpu/drm/i915/i915_scheduler.c +2 −2
  171. drivers/gpu/drm/i915/i915_scheduler_types.h +1 −0
  172. drivers/gpu/drm/i915/i915_selftest.h +26 −1
  173. drivers/gpu/drm/i915/i915_timeline.h +0 −94
  174. drivers/gpu/drm/i915/i915_trace.h +6 −6
  175. drivers/gpu/drm/i915/i915_utils.h +12 −0
  176. drivers/gpu/drm/i915/i915_vgpu.c +43 −20
  177. drivers/gpu/drm/i915/i915_vgpu.h +4 −3
  178. drivers/gpu/drm/i915/i915_vma.c +28 −45
  179. drivers/gpu/drm/i915/intel_device_info.c +17 −23
  180. drivers/gpu/drm/i915/intel_device_info.h +4 −1
  181. drivers/gpu/drm/i915/intel_drv.h +32 −5
  182. drivers/gpu/drm/i915/intel_guc_fw.c +0 −308
  183. drivers/gpu/drm/i915/intel_gvt.c +4 −3
  184. drivers/gpu/drm/i915/intel_gvt.h +4 −3
  185. drivers/gpu/drm/i915/intel_huc_fw.c +0 −215
  186. drivers/gpu/drm/i915/intel_pm.c +214 −227
  187. drivers/gpu/drm/i915/intel_pm.h +2 −2
  188. drivers/gpu/drm/i915/intel_runtime_pm.c +1 −1
  189. drivers/gpu/drm/i915/intel_runtime_pm.h +1 −1
  190. drivers/gpu/drm/i915/intel_uc.c +0 −561
  191. drivers/gpu/drm/i915/intel_uc_fw.c +0 −357
  192. drivers/gpu/drm/i915/intel_uncore.c +261 −204
  193. drivers/gpu/drm/i915/intel_uncore.h +13 −11
  194. drivers/gpu/drm/i915/intel_wakeref.c +5 −4
  195. drivers/gpu/drm/i915/intel_wakeref.h +9 −2
  196. drivers/gpu/drm/i915/intel_wopcm.c +22 −18
  197. drivers/gpu/drm/i915/intel_wopcm.h +3 −1
  198. drivers/gpu/drm/i915/oa/Makefile +0 −0
  199. drivers/gpu/drm/i915/oa/i915_oa_bdw.c +0 −0
  200. drivers/gpu/drm/i915/oa/i915_oa_bdw.h +1 −1
  201. drivers/gpu/drm/i915/oa/i915_oa_bxt.c +0 −0
  202. drivers/gpu/drm/i915/oa/i915_oa_bxt.h +1 −1
  203. drivers/gpu/drm/i915/oa/i915_oa_cflgt2.c +0 −0
  204. drivers/gpu/drm/i915/oa/i915_oa_cflgt2.h +1 −1
  205. drivers/gpu/drm/i915/oa/i915_oa_cflgt3.c +0 −0
  206. drivers/gpu/drm/i915/oa/i915_oa_cflgt3.h +1 −1
  207. drivers/gpu/drm/i915/oa/i915_oa_chv.c +0 −0
  208. drivers/gpu/drm/i915/oa/i915_oa_chv.h +1 −1
  209. drivers/gpu/drm/i915/oa/i915_oa_cnl.c +0 −0
  210. drivers/gpu/drm/i915/oa/i915_oa_cnl.h +1 −1
  211. drivers/gpu/drm/i915/oa/i915_oa_glk.c +0 −0
  212. drivers/gpu/drm/i915/oa/i915_oa_glk.h +1 −1
  213. drivers/gpu/drm/i915/oa/i915_oa_hsw.c +0 −0
  214. drivers/gpu/drm/i915/oa/i915_oa_hsw.h +1 −1
  215. drivers/gpu/drm/i915/oa/i915_oa_icl.c +0 −0
  216. drivers/gpu/drm/i915/oa/i915_oa_icl.h +1 −1
  217. drivers/gpu/drm/i915/oa/i915_oa_kblgt2.c +0 −0
  218. drivers/gpu/drm/i915/oa/i915_oa_kblgt2.h +1 −1
  219. drivers/gpu/drm/i915/oa/i915_oa_kblgt3.c +0 −0
  220. drivers/gpu/drm/i915/oa/i915_oa_kblgt3.h +1 −1
  221. drivers/gpu/drm/i915/oa/i915_oa_sklgt2.c +0 −0
  222. drivers/gpu/drm/i915/oa/i915_oa_sklgt2.h +1 −1
  223. drivers/gpu/drm/i915/oa/i915_oa_sklgt3.c +0 −0
  224. drivers/gpu/drm/i915/oa/i915_oa_sklgt3.h +1 −1
  225. drivers/gpu/drm/i915/oa/i915_oa_sklgt4.c +0 −0
  226. drivers/gpu/drm/i915/oa/i915_oa_sklgt4.h +1 −1
  227. drivers/gpu/drm/i915/selftests/i915_active.c +97 −27
  228. drivers/gpu/drm/i915/selftests/i915_gem.c +4 −7
  229. drivers/gpu/drm/i915/selftests/i915_gem_evict.c +2 −1
  230. drivers/gpu/drm/i915/selftests/i915_gem_gtt.c +1 −3
  231. drivers/gpu/drm/i915/selftests/i915_live_selftests.h +1 −1
  232. drivers/gpu/drm/i915/selftests/i915_mock_selftests.h +1 −1
  233. drivers/gpu/drm/i915/selftests/i915_request.c +25 −18
  234. drivers/gpu/drm/i915/selftests/i915_selftest.c +64 −1
  235. drivers/gpu/drm/i915/selftests/i915_vma.c +10 −0
  236. drivers/gpu/drm/i915/selftests/igt_flush_test.c +3 −2
  237. drivers/gpu/drm/i915/selftests/igt_reset.c +18 −20
  238. drivers/gpu/drm/i915/selftests/igt_reset.h +6 −4
  239. drivers/gpu/drm/i915/selftests/igt_spinner.c +6 −2
  240. drivers/gpu/drm/i915/selftests/igt_spinner.h +3 −0
  241. drivers/gpu/drm/i915/selftests/igt_wedge_me.h +0 −58
  242. drivers/gpu/drm/i915/selftests/mock_gem_device.c +5 −13
  243. drivers/gpu/drm/i915/selftests/mock_gtt.c +3 −0
  244. drivers/gpu/drm/i915/selftests/mock_uncore.c +2 −2
  245. include/drm/i915_component.h +1 −1
  246. include/drm/i915_drm.h +8 −5
  247. include/drm/i915_pciids.h +12 −1
  248. include/uapi/drm/i915_drm.h +1 −0
Documentation/gpu/i915.rst+10−10
@@ -430,31 +430,31 @@ WOPCM Layout · 429 unmodified lines
430430 GuC
431431 ===
432432
433+Firmware Layout
434+-------------------
435+
436+.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_uc_fw_abi.h
437+ :doc: Firmware Layout
438+
433439 GuC-specific firmware loader
434440 ----------------------------
435441
436−.. kernel-doc:: drivers/gpu/drm/i915/intel_guc_fw.c
442+.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c
437443 :internal:
438444
439445 GuC-based command submission
440446 ----------------------------
441447
442−.. kernel-doc:: drivers/gpu/drm/i915/intel_guc_submission.c
448+.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
443449 :doc: GuC-based command submission
444450
445−.. kernel-doc:: drivers/gpu/drm/i915/intel_guc_submission.c
451+.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
446452 :internal:
447453
448−GuC Firmware Layout
449−-------------------
450−
451−.. kernel-doc:: drivers/gpu/drm/i915/intel_guc_fwif.h
452− :doc: GuC Firmware Layout
453−
454454 GuC Address Space
455455 -----------------
456456
457−.. kernel-doc:: drivers/gpu/drm/i915/intel_guc.c
457+.. kernel-doc:: drivers/gpu/drm/i915/gt/uc/intel_guc.c
458458 :doc: GuC Address Space
459459
460460 Tracing
arch/x86/kernel/early-quirks.c+1−0
@@ -549,6 +549,7 @@ static const struct pci_device_id intel_early_ids[] __initconst = { · 548 unmodified lines
549549 INTEL_CNL_IDS(&gen9_early_ops),
550550 INTEL_ICL_11_IDS(&gen11_early_ops),
551551 INTEL_EHL_IDS(&gen11_early_ops),
552+ INTEL_TGL_12_IDS(&gen11_early_ops),
552553 };
553554
554555 struct resource intel_graphics_stolen_res __ro_after_init = DEFINE_RES_MEM(0, 0);
drivers/gpu/drm/i915/Kconfig.debug+15−0
@@ -7,6 +7,7 @@ config DRM_I915_WERROR · 6 unmodified lines
77 # We use the dependency on !COMPILE_TEST to not be enabled in
88 # allmodconfig or allyesconfig configurations
99 depends on !COMPILE_TEST
10+ select HEADER_TEST
1011 default n
1112 help
1213 Add -Werror to the build flags for (and only for) i915.ko.
@@ -94,6 +95,20 @@ config DRM_I915_TRACE_GEM · 81 unmodified lines
9495
9596 If in doubt, say "N".
9697
98+config DRM_I915_TRACE_GTT
99+ bool "Insert extra ftrace output from the GTT internals"
100+ depends on DRM_I915_DEBUG_GEM
101+ select TRACING
102+ default n
103+ help
104+ Enable additional and verbose debugging output that will spam
105+ ordinary tests, but may be vital for post-mortem debugging when
106+ used with /proc/sys/kernel/ftrace_dump_on_oops
107+
108+ Recommended for driver developers only.
109+
110+ If in doubt, say "N".
111+
97112 config DRM_I915_SW_FENCE_DEBUG_OBJECTS
98113 bool "Enable additional driver debugging for fence objects"
99114 depends on DRM_I915
drivers/gpu/drm/i915/Makefile+43−38
@@ -32,9 +32,9 @@ subdir-ccflags-y += \ · 31 unmodified lines
3232 $(call as-instr,movntdqa (%eax)$(comma)%xmm0,-DCONFIG_AS_MOVNTDQA)
3333
3434 # Extra header tests
35−include $(src)/Makefile.header-test
35+header-test-pattern-$(CONFIG_DRM_I915_WERROR) := *.h
3636
37−subdir-ccflags-y += -I$(src)
37+subdir-ccflags-y += -I$(srctree)/$(src)
3838
3939 # Please keep these build lists sorted!
4040
@@ -73,14 +73,23 @@ gt-y += \ · 32 unmodified lines
7373 gt/intel_context.o \
7474 gt/intel_engine_cs.o \
7575 gt/intel_engine_pm.o \
76+ gt/intel_gt.o \
7677 gt/intel_gt_pm.o \
7778 gt/intel_hangcheck.o \
7879 gt/intel_lrc.o \
80+ gt/intel_renderstate.o \
7981 gt/intel_reset.o \
8082 gt/intel_ringbuffer.o \
8183 gt/intel_mocs.o \
8284 gt/intel_sseu.o \
85+ gt/intel_timeline.o \
8386 gt/intel_workarounds.o
87+# autogenerated null render state
88+gt-y += \
89+ gt/gen6_renderstate.o \
90+ gt/gen7_renderstate.o \
91+ gt/gen8_renderstate.o \
92+ gt/gen9_renderstate.o
8493 gt-$(CONFIG_DRM_I915_SELFTEST) += \
8594 gt/mock_engine.o
8695 i915-y += $(gt-y)
@@ -120,33 +129,26 @@ i915-y += \ · 33 unmodified lines
120129 i915_gem_fence_reg.o \
121130 i915_gem_gtt.o \
122131 i915_gem.o \
123− i915_gem_render_state.o \
124132 i915_globals.o \
125133 i915_query.o \
126134 i915_request.o \
127135 i915_scheduler.o \
128− i915_timeline.o \
129136 i915_trace_points.o \
130137 i915_vma.o \
131138 intel_wopcm.o
132139
133140 # general-purpose microcontroller (GuC) support
134−i915-y += intel_uc.o \
135− intel_uc_fw.o \
136− intel_guc.o \
137− intel_guc_ads.o \
138− intel_guc_ct.o \
139− intel_guc_fw.o \
140− intel_guc_log.o \
141− intel_guc_submission.o \
142− intel_huc.o \
143− intel_huc_fw.o
144−
145−# autogenerated null render state
146−i915-y += intel_renderstate_gen6.o \
147− intel_renderstate_gen7.o \
148− intel_renderstate_gen8.o \
149− intel_renderstate_gen9.o
141+obj-y += gt/uc/
142+i915-y += gt/uc/intel_uc.o \
143+ gt/uc/intel_uc_fw.o \
144+ gt/uc/intel_guc.o \
145+ gt/uc/intel_guc_ads.o \
146+ gt/uc/intel_guc_ct.o \
147+ gt/uc/intel_guc_fw.o \
148+ gt/uc/intel_guc_log.o \
149+ gt/uc/intel_guc_submission.o \
150+ gt/uc/intel_huc.o \
151+ gt/uc/intel_huc_fw.o
150152
151153 # modesetting core code
152154 obj-y += display/
@@ -173,7 +175,8 @@ i915-y += \ · 20 unmodified lines
173175 display/intel_overlay.o \
174176 display/intel_psr.o \
175177 display/intel_quirks.o \
176− display/intel_sprite.o
178+ display/intel_sprite.o \
179+ display/intel_tc.o
177180 i915-$(CONFIG_ACPI) += \
178181 display/intel_acpi.o \
179182 display/intel_opregion.o
@@ -210,6 +213,25 @@ i915-y += \ · 30 unmodified lines
210213 display/vlv_dsi.o \
211214 display/vlv_dsi_pll.o
212215
216+# perf code
217+obj-y += oa/
218+i915-y += \
219+ oa/i915_oa_hsw.o \
220+ oa/i915_oa_bdw.o \
221+ oa/i915_oa_chv.o \
222+ oa/i915_oa_sklgt2.o \
223+ oa/i915_oa_sklgt3.o \
224+ oa/i915_oa_sklgt4.o \
225+ oa/i915_oa_bxt.o \
226+ oa/i915_oa_kblgt2.o \
227+ oa/i915_oa_kblgt3.o \
228+ oa/i915_oa_glk.o \
229+ oa/i915_oa_cflgt2.o \
230+ oa/i915_oa_cflgt3.o \
231+ oa/i915_oa_cnl.o \
232+ oa/i915_oa_icl.o
233+i915-y += i915_perf.o
234+
213235 # Post-mortem debug and GPU hang state capture
214236 i915-$(CONFIG_DRM_I915_CAPTURE_ERROR) += i915_gpu_error.o
215237 i915-$(CONFIG_DRM_I915_SELFTEST) += \
@@ -224,23 +246,6 @@ i915-$(CONFIG_DRM_I915_SELFTEST) += \ · 8 unmodified lines
224246 # virtual gpu code
225247 i915-y += i915_vgpu.o
226248
227−# perf code
228−i915-y += i915_perf.o \
229− i915_oa_hsw.o \
230− i915_oa_bdw.o \
231− i915_oa_chv.o \
232− i915_oa_sklgt2.o \
233− i915_oa_sklgt3.o \
234− i915_oa_sklgt4.o \
235− i915_oa_bxt.o \
236− i915_oa_kblgt2.o \
237− i915_oa_kblgt3.o \
238− i915_oa_glk.o \
239− i915_oa_cflgt2.o \
240− i915_oa_cflgt3.o \
241− i915_oa_cnl.o \
242− i915_oa_icl.o
243−
244249 ifeq ($(CONFIG_DRM_I915_GVT),y)
245250 i915-y += intel_gvt.o
246251 include $(src)/gvt/Makefile
drivers/gpu/drm/i915/Makefile.header-test+0−22
@@ -1,22 +0,0 @@
1−# SPDX-License-Identifier: MIT
2−# Copyright © 2019 Intel Corporation
3−
4−# Test the headers are compilable as standalone units
5−header-test-$(CONFIG_DRM_I915_WERROR) := \
6− i915_active_types.h \
7− i915_debugfs.h \
8− i915_drv.h \
9− i915_irq.h \
10− i915_params.h \
11− i915_priolist_types.h \
12− i915_reg.h \
13− i915_scheduler_types.h \
14− i915_timeline_types.h \
15− i915_utils.h \
16− intel_csr.h \
17− intel_drv.h \
18− intel_pm.h \
19− intel_runtime_pm.h \
20− intel_sideband.h \
21− intel_uncore.h \
22− intel_wakeref.h
drivers/gpu/drm/i915/display/Makefile+5−1
@@ -1,2 +1,6 @@
1+# For building individual subdir files on the command line
2+subdir-ccflags-y += -I$(srctree)/$(src)/..
3+
14 # Extra header tests
2−include $(src)/Makefile.header-test
5+header-test-pattern-$(CONFIG_DRM_I915_WERROR) := *.h
6+header-test- := intel_vbt_defs.h
drivers/gpu/drm/i915/display/Makefile.header-test+0−16
@@ -1,16 +0,0 @@
1−# SPDX-License-Identifier: MIT
2−# Copyright © 2019 Intel Corporation
3−
4−# Test the headers are compilable as standalone units
5−header_test := $(notdir $(filter-out %/intel_vbt_defs.h,$(wildcard $(src)/*.h)))
6−
7−quiet_cmd_header_test = HDRTEST $@
8− cmd_header_test = echo "\#include \"$(<F)\"" > $@
9−
10−header_test_%.c: %.h
11− $(call cmd,header_test)
12−
13−extra-$(CONFIG_DRM_I915_WERROR) += \
14− $(foreach h,$(header_test),$(patsubst %.h,header_test_%.o,$(h)))
15−
16−clean-files += $(foreach h,$(header_test),$(patsubst %.h,header_test_%.c,$(h)))
drivers/gpu/drm/i915/display/icl_dsi.c+112−73
@@ -202,63 +202,62 @@ static void dsi_program_swing_and_deemphasis(struct intel_encoder *encoder) · 201 unmodified lines
202202 {
203203 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
204204 struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
205− enum port port;
205+ enum phy phy;
206206 u32 tmp;
207207 int lane;
208208
209− for_each_dsi_port(port, intel_dsi->ports) {
210−
209+ for_each_dsi_phy(phy, intel_dsi->phys) {
211210 /*
212211 * Program voltage swing and pre-emphasis level values as per
213212 * table in BSPEC under DDI buffer programing
214213 */
215− tmp = I915_READ(ICL_PORT_TX_DW5_LN0(port));
214+ tmp = I915_READ(ICL_PORT_TX_DW5_LN0(phy));
216215 tmp &= ~(SCALING_MODE_SEL_MASK | RTERM_SELECT_MASK);
217216 tmp |= SCALING_MODE_SEL(0x2);
218217 tmp |= TAP2_DISABLE | TAP3_DISABLE;
219218 tmp |= RTERM_SELECT(0x6);
220− I915_WRITE(ICL_PORT_TX_DW5_GRP(port), tmp);
219+ I915_WRITE(ICL_PORT_TX_DW5_GRP(phy), tmp);
221220
222− tmp = I915_READ(ICL_PORT_TX_DW5_AUX(port));
221+ tmp = I915_READ(ICL_PORT_TX_DW5_AUX(phy));
223222 tmp &= ~(SCALING_MODE_SEL_MASK | RTERM_SELECT_MASK);
224223 tmp |= SCALING_MODE_SEL(0x2);
225224 tmp |= TAP2_DISABLE | TAP3_DISABLE;
226225 tmp |= RTERM_SELECT(0x6);
227− I915_WRITE(ICL_PORT_TX_DW5_AUX(port), tmp);
226+ I915_WRITE(ICL_PORT_TX_DW5_AUX(phy), tmp);
228227
229− tmp = I915_READ(ICL_PORT_TX_DW2_LN0(port));
228+ tmp = I915_READ(ICL_PORT_TX_DW2_LN0(phy));
230229 tmp &= ~(SWING_SEL_LOWER_MASK | SWING_SEL_UPPER_MASK |
231230 RCOMP_SCALAR_MASK);
232231 tmp |= SWING_SEL_UPPER(0x2);
233232 tmp |= SWING_SEL_LOWER(0x2);
234233 tmp |= RCOMP_SCALAR(0x98);
235− I915_WRITE(ICL_PORT_TX_DW2_GRP(port), tmp);
234+ I915_WRITE(ICL_PORT_TX_DW2_GRP(phy), tmp);
236235
237− tmp = I915_READ(ICL_PORT_TX_DW2_AUX(port));
236+ tmp = I915_READ(ICL_PORT_TX_DW2_AUX(phy));
238237 tmp &= ~(SWING_SEL_LOWER_MASK | SWING_SEL_UPPER_MASK |
239238 RCOMP_SCALAR_MASK);
240239 tmp |= SWING_SEL_UPPER(0x2);
241240 tmp |= SWING_SEL_LOWER(0x2);
242241 tmp |= RCOMP_SCALAR(0x98);
243− I915_WRITE(ICL_PORT_TX_DW2_AUX(port), tmp);
242+ I915_WRITE(ICL_PORT_TX_DW2_AUX(phy), tmp);
244243
245− tmp = I915_READ(ICL_PORT_TX_DW4_AUX(port));
244+ tmp = I915_READ(ICL_PORT_TX_DW4_AUX(phy));
246245 tmp &= ~(POST_CURSOR_1_MASK | POST_CURSOR_2_MASK |
247246 CURSOR_COEFF_MASK);
248247 tmp |= POST_CURSOR_1(0x0);
249248 tmp |= POST_CURSOR_2(0x0);
250249 tmp |= CURSOR_COEFF(0x3f);
251− I915_WRITE(ICL_PORT_TX_DW4_AUX(port), tmp);
250+ I915_WRITE(ICL_PORT_TX_DW4_AUX(phy), tmp);
252251
253252 for (lane = 0; lane <= 3; lane++) {
254253 /* Bspec: must not use GRP register for write */
255− tmp = I915_READ(ICL_PORT_TX_DW4_LN(lane, port));
254+ tmp = I915_READ(ICL_PORT_TX_DW4_LN(lane, phy));
256255 tmp &= ~(POST_CURSOR_1_MASK | POST_CURSOR_2_MASK |
257256 CURSOR_COEFF_MASK);
258257 tmp |= POST_CURSOR_1(0x0);
259258 tmp |= POST_CURSOR_2(0x0);
260259 tmp |= CURSOR_COEFF(0x3f);
261− I915_WRITE(ICL_PORT_TX_DW4_LN(lane, port), tmp);
260+ I915_WRITE(ICL_PORT_TX_DW4_LN(lane, phy), tmp);
262261 }
263262 }
264263 }
@@ -364,10 +363,10 @@ static void gen11_dsi_power_up_lanes(struct intel_encoder *encoder) · 99 unmodified lines
364363 {
365364 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
366365 struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
367− enum port port;
366+ enum phy phy;
368367
369− for_each_dsi_port(port, intel_dsi->ports)
370− intel_combo_phy_power_up_lanes(dev_priv, port, true,
368+ for_each_dsi_phy(phy, intel_dsi->phys)
369+ intel_combo_phy_power_up_lanes(dev_priv, phy, true,
371370 intel_dsi->lane_count, false);
372371 }
373372
@@ -375,34 +374,47 @@ static void gen11_dsi_config_phy_lanes_sequence(struct intel_encoder *encoder) · 1 unmodified line
375374 {
376375 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
377376 struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
378− enum port port;
377+ enum phy phy;
379378 u32 tmp;
380379 int lane;
381380
382381 /* Step 4b(i) set loadgen select for transmit and aux lanes */
383− for_each_dsi_port(port, intel_dsi->ports) {
384− tmp = I915_READ(ICL_PORT_TX_DW4_AUX(port));
382+ for_each_dsi_phy(phy, intel_dsi->phys) {
383+ tmp = I915_READ(ICL_PORT_TX_DW4_AUX(phy));
385384 tmp &= ~LOADGEN_SELECT;
386− I915_WRITE(ICL_PORT_TX_DW4_AUX(port), tmp);
385+ I915_WRITE(ICL_PORT_TX_DW4_AUX(phy), tmp);
387386 for (lane = 0; lane <= 3; lane++) {
388− tmp = I915_READ(ICL_PORT_TX_DW4_LN(lane, port));
387+ tmp = I915_READ(ICL_PORT_TX_DW4_LN(lane, phy));
389388 tmp &= ~LOADGEN_SELECT;
390389 if (lane != 2)
391390 tmp |= LOADGEN_SELECT;
392− I915_WRITE(ICL_PORT_TX_DW4_LN(lane, port), tmp);
391+ I915_WRITE(ICL_PORT_TX_DW4_LN(lane, phy), tmp);
393392 }
394393 }
395394
396395 /* Step 4b(ii) set latency optimization for transmit and aux lanes */
397− for_each_dsi_port(port, intel_dsi->ports) {
398− tmp = I915_READ(ICL_PORT_TX_DW2_AUX(port));
396+ for_each_dsi_phy(phy, intel_dsi->phys) {
397+ tmp = I915_READ(ICL_PORT_TX_DW2_AUX(phy));
399398 tmp &= ~FRC_LATENCY_OPTIM_MASK;
400399 tmp |= FRC_LATENCY_OPTIM_VAL(0x5);
401− I915_WRITE(ICL_PORT_TX_DW2_AUX(port), tmp);
402− tmp = I915_READ(ICL_PORT_TX_DW2_LN0(port));
400+ I915_WRITE(ICL_PORT_TX_DW2_AUX(phy), tmp);
401+ tmp = I915_READ(ICL_PORT_TX_DW2_LN0(phy));
403402 tmp &= ~FRC_LATENCY_OPTIM_MASK;
404403 tmp |= FRC_LATENCY_OPTIM_VAL(0x5);
405− I915_WRITE(ICL_PORT_TX_DW2_GRP(port), tmp);
404+ I915_WRITE(ICL_PORT_TX_DW2_GRP(phy), tmp);
405+
406+ /* For EHL set latency optimization for PCS_DW1 lanes */
407+ if (IS_ELKHARTLAKE(dev_priv)) {
408+ tmp = I915_READ(ICL_PORT_PCS_DW1_AUX(phy));
409+ tmp &= ~LATENCY_OPTIM_MASK;
410+ tmp |= LATENCY_OPTIM_VAL(0);
411+ I915_WRITE(ICL_PORT_PCS_DW1_AUX(phy), tmp);
412+
413+ tmp = I915_READ(ICL_PORT_PCS_DW1_LN0(phy));
414+ tmp &= ~LATENCY_OPTIM_MASK;
415+ tmp |= LATENCY_OPTIM_VAL(0x1);
416+ I915_WRITE(ICL_PORT_PCS_DW1_GRP(phy), tmp);
417+ }
406418 }
407419
408420 }
@@ -412,16 +424,16 @@ static void gen11_dsi_voltage_swing_program_seq(struct intel_encoder *encoder) · 3 unmodified lines
412424 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
413425 struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
414426 u32 tmp;
415− enum port port;
427+ enum phy phy;
416428
417429 /* clear common keeper enable bit */
418− for_each_dsi_port(port, intel_dsi->ports) {
419− tmp = I915_READ(ICL_PORT_PCS_DW1_LN0(port));
430+ for_each_dsi_phy(phy, intel_dsi->phys) {
431+ tmp = I915_READ(ICL_PORT_PCS_DW1_LN0(phy));
420432 tmp &= ~COMMON_KEEPER_EN;
421− I915_WRITE(ICL_PORT_PCS_DW1_GRP(port), tmp);
422− tmp = I915_READ(ICL_PORT_PCS_DW1_AUX(port));
433+ I915_WRITE(ICL_PORT_PCS_DW1_GRP(phy), tmp);
434+ tmp = I915_READ(ICL_PORT_PCS_DW1_AUX(phy));
423435 tmp &= ~COMMON_KEEPER_EN;
424− I915_WRITE(ICL_PORT_PCS_DW1_AUX(port), tmp);
436+ I915_WRITE(ICL_PORT_PCS_DW1_AUX(phy), tmp);
425437 }
426438
427439 /*
@@ -429,33 +441,33 @@ static void gen11_dsi_voltage_swing_program_seq(struct intel_encoder *encoder) · 1 unmodified line
429441 * Note: loadgen select program is done
430442 * as part of lane phy sequence configuration
431443 */
432− for_each_dsi_port(port, intel_dsi->ports) {
433− tmp = I915_READ(ICL_PORT_CL_DW5(port));
444+ for_each_dsi_phy(phy, intel_dsi->phys) {
445+ tmp = I915_READ(ICL_PORT_CL_DW5(phy));
434446 tmp |= SUS_CLOCK_CONFIG;
435− I915_WRITE(ICL_PORT_CL_DW5(port), tmp);
447+ I915_WRITE(ICL_PORT_CL_DW5(phy), tmp);
436448 }
437449
438450 /* Clear training enable to change swing values */
439− for_each_dsi_port(port, intel_dsi->ports) {
440− tmp = I915_READ(ICL_PORT_TX_DW5_LN0(port));
451+ for_each_dsi_phy(phy, intel_dsi->phys) {
452+ tmp = I915_READ(ICL_PORT_TX_DW5_LN0(phy));
441453 tmp &= ~TX_TRAINING_EN;
442− I915_WRITE(ICL_PORT_TX_DW5_GRP(port), tmp);
443− tmp = I915_READ(ICL_PORT_TX_DW5_AUX(port));
454+ I915_WRITE(ICL_PORT_TX_DW5_GRP(phy), tmp);
455+ tmp = I915_READ(ICL_PORT_TX_DW5_AUX(phy));
444456 tmp &= ~TX_TRAINING_EN;
445− I915_WRITE(ICL_PORT_TX_DW5_AUX(port), tmp);
457+ I915_WRITE(ICL_PORT_TX_DW5_AUX(phy), tmp);
446458 }
447459
448460 /* Program swing and de-emphasis */
449461 dsi_program_swing_and_deemphasis(encoder);
450462
451463 /* Set training enable to trigger update */
452− for_each_dsi_port(port, intel_dsi->ports) {
453− tmp = I915_READ(ICL_PORT_TX_DW5_LN0(port));
464+ for_each_dsi_phy(phy, intel_dsi->phys) {
465+ tmp = I915_READ(ICL_PORT_TX_DW5_LN0(phy));
454466 tmp |= TX_TRAINING_EN;
455− I915_WRITE(ICL_PORT_TX_DW5_GRP(port), tmp);
456− tmp = I915_READ(ICL_PORT_TX_DW5_AUX(port));
467+ I915_WRITE(ICL_PORT_TX_DW5_GRP(phy), tmp);
468+ tmp = I915_READ(ICL_PORT_TX_DW5_AUX(phy));
457469 tmp |= TX_TRAINING_EN;
458− I915_WRITE(ICL_PORT_TX_DW5_AUX(port), tmp);
470+ I915_WRITE(ICL_PORT_TX_DW5_AUX(phy), tmp);
459471 }
460472 }
461473
@@ -484,6 +496,7 @@ static void gen11_dsi_setup_dphy_timings(struct intel_encoder *encoder) · 22 unmodified lines
484496 struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
485497 u32 tmp;
486498 enum port port;
499+ enum phy phy;
487500
488501 /* Program T-INIT master registers */
489502 for_each_dsi_port(port, intel_dsi->ports) {
@@ -531,6 +544,14 @@ static void gen11_dsi_setup_dphy_timings(struct intel_encoder *encoder) · 41 unmodified lines
531544 I915_WRITE(DSI_TA_TIMING_PARAM(port), tmp);
532545 }
533546 }
547+
548+ if (IS_ELKHARTLAKE(dev_priv)) {
549+ for_each_dsi_phy(phy, intel_dsi->phys) {
550+ tmp = I915_READ(ICL_DPHY_CHKN(phy));
551+ tmp |= ICL_DPHY_CHKN_AFE_OVER_PPI_STRAP;
552+ I915_WRITE(ICL_DPHY_CHKN(phy), tmp);
553+ }
554+ }
534555 }
535556
536557 static void gen11_dsi_gate_clocks(struct intel_encoder *encoder)
@@ -538,15 +559,14 @@ static void gen11_dsi_gate_clocks(struct intel_encoder *encoder) · 1 unmodified line
538559 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
539560 struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
540561 u32 tmp;
541− enum port port;
562+ enum phy phy;
542563
543564 mutex_lock(&dev_priv->dpll_lock);
544− tmp = I915_READ(DPCLKA_CFGCR0_ICL);
545− for_each_dsi_port(port, intel_dsi->ports) {
546− tmp |= DPCLKA_CFGCR0_DDI_CLK_OFF(port);
547− }
565+ tmp = I915_READ(ICL_DPCLKA_CFGCR0);
566+ for_each_dsi_phy(phy, intel_dsi->phys)
567+ tmp |= ICL_DPCLKA_CFGCR0_DDI_CLK_OFF(phy);
548568
549− I915_WRITE(DPCLKA_CFGCR0_ICL, tmp);
569+ I915_WRITE(ICL_DPCLKA_CFGCR0, tmp);
550570 mutex_unlock(&dev_priv->dpll_lock);
551571 }
552572
@@ -555,15 +575,14 @@ static void gen11_dsi_ungate_clocks(struct intel_encoder *encoder) · 2 unmodified lines
555575 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
556576 struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
557577 u32 tmp;
558− enum port port;
578+ enum phy phy;
559579
560580 mutex_lock(&dev_priv->dpll_lock);
561− tmp = I915_READ(DPCLKA_CFGCR0_ICL);
562− for_each_dsi_port(port, intel_dsi->ports) {
563− tmp &= ~DPCLKA_CFGCR0_DDI_CLK_OFF(port);
564− }
581+ tmp = I915_READ(ICL_DPCLKA_CFGCR0);
582+ for_each_dsi_phy(phy, intel_dsi->phys)
583+ tmp &= ~ICL_DPCLKA_CFGCR0_DDI_CLK_OFF(phy);
565584
566− I915_WRITE(DPCLKA_CFGCR0_ICL, tmp);
585+ I915_WRITE(ICL_DPCLKA_CFGCR0, tmp);
567586 mutex_unlock(&dev_priv->dpll_lock);
568587 }
569588
@@ -573,24 +592,24 @@ static void gen11_dsi_map_pll(struct intel_encoder *encoder, · 3 unmodified lines
573592 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
574593 struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
575594 struct intel_shared_dpll *pll = crtc_state->shared_dpll;
576− enum port port;
595+ enum phy phy;
577596 u32 val;
578597
579598 mutex_lock(&dev_priv->dpll_lock);
580599
581− val = I915_READ(DPCLKA_CFGCR0_ICL);
582− for_each_dsi_port(port, intel_dsi->ports) {
583− val &= ~DPCLKA_CFGCR0_DDI_CLK_SEL_MASK(port);
584− val |= DPCLKA_CFGCR0_DDI_CLK_SEL(pll->info->id, port);
600+ val = I915_READ(ICL_DPCLKA_CFGCR0);
601+ for_each_dsi_phy(phy, intel_dsi->phys) {
602+ val &= ~ICL_DPCLKA_CFGCR0_DDI_CLK_SEL_MASK(phy);
603+ val |= ICL_DPCLKA_CFGCR0_DDI_CLK_SEL(pll->info->id, phy);
585604 }
586− I915_WRITE(DPCLKA_CFGCR0_ICL, val);
605+ I915_WRITE(ICL_DPCLKA_CFGCR0, val);
587606
588− for_each_dsi_port(port, intel_dsi->ports) {
589− val &= ~DPCLKA_CFGCR0_DDI_CLK_OFF(port);
607+ for_each_dsi_phy(phy, intel_dsi->phys) {
608+ val &= ~ICL_DPCLKA_CFGCR0_DDI_CLK_OFF(phy);
590609 }
591− I915_WRITE(DPCLKA_CFGCR0_ICL, val);
610+ I915_WRITE(ICL_DPCLKA_CFGCR0, val);
592611
593− POSTING_READ(DPCLKA_CFGCR0_ICL);
612+ POSTING_READ(ICL_DPCLKA_CFGCR0);
594613
595614 mutex_unlock(&dev_priv->dpll_lock);
596615 }
@@ -744,7 +763,7 @@ gen11_dsi_set_transcoder_timings(struct intel_encoder *encoder, · 147 unmodified lines
744763 enum transcoder dsi_trans;
745764 /* horizontal timings */
746765 u16 htotal, hactive, hsync_start, hsync_end, hsync_size;
747− u16 hfront_porch, hback_porch;
766+ u16 hback_porch;
748767 /* vertical timings */
749768 u16 vtotal, vactive, vsync_start, vsync_end, vsync_shift;
750769
@@ -753,8 +772,6 @@ gen11_dsi_set_transcoder_timings(struct intel_encoder *encoder, · 2 unmodified lines
753772 hsync_start = adjusted_mode->crtc_hsync_start;
754773 hsync_end = adjusted_mode->crtc_hsync_end;
755774 hsync_size = hsync_end - hsync_start;
756− hfront_porch = (adjusted_mode->crtc_hsync_start -
757− adjusted_mode->crtc_hdisplay);
758775 hback_porch = (adjusted_mode->crtc_htotal -
759776 adjusted_mode->crtc_hsync_end);
760777 vactive = adjusted_mode->crtc_vdisplay;
@@ -1487,6 +1504,26 @@ static void icl_dphy_param_init(struct intel_dsi *intel_dsi) · 726 unmodified lines
14871504 intel_dsi_log_params(intel_dsi);
14881505 }
14891506
1507+static void icl_dsi_add_properties(struct intel_connector *connector)
1508+{
1509+ u32 allowed_scalers;
1510+
1511+ allowed_scalers = BIT(DRM_MODE_SCALE_ASPECT) |
1512+ BIT(DRM_MODE_SCALE_FULLSCREEN) |
1513+ BIT(DRM_MODE_SCALE_CENTER);
1514+
1515+ drm_connector_attach_scaling_mode_property(&connector->base,
1516+ allowed_scalers);
1517+
1518+ connector->base.state->scaling_mode = DRM_MODE_SCALE_ASPECT;
1519+
1520+ connector->base.display_info.panel_orientation =
1521+ intel_dsi_get_panel_orientation(connector);
1522+ drm_connector_init_panel_orientation_property(&connector->base,
1523+ connector->panel.fixed_mode->hdisplay,
1524+ connector->panel.fixed_mode->vdisplay);
1525+}
1526+
14901527 void icl_dsi_init(struct drm_i915_private *dev_priv)
14911528 {
14921529 struct drm_device *dev = &dev_priv->drm;
@@ -1580,6 +1617,8 @@ void icl_dsi_init(struct drm_i915_private *dev_priv) · 87 unmodified lines
15801617 }
15811618
15821619 icl_dphy_param_init(intel_dsi);
1620+
1621+ icl_dsi_add_properties(intel_connector);
15831622 return;
15841623
15851624 err:
drivers/gpu/drm/i915/display/intel_atomic_plane.c+36−20
@@ -176,33 +176,49 @@ int intel_plane_atomic_check_with_state(const struct intel_crtc_state *old_crtc_ · 175 unmodified lines
176176 new_crtc_state->data_rate[plane->id] =
177177 intel_plane_data_rate(new_crtc_state, new_plane_state);
178178
179− return intel_plane_atomic_calc_changes(old_crtc_state,
180− &new_crtc_state->base,
181− old_plane_state,
182− &new_plane_state->base);
179+ return intel_plane_atomic_calc_changes(old_crtc_state, new_crtc_state,
180+ old_plane_state, new_plane_state);
183181 }
184182
185−static int intel_plane_atomic_check(struct drm_plane *plane,
186− struct drm_plane_state *new_plane_state)
183+static struct intel_crtc *
184+get_crtc_from_states(const struct intel_plane_state *old_plane_state,
185+ const struct intel_plane_state *new_plane_state)
187186 {
188− struct drm_atomic_state *state = new_plane_state->state;
189− const struct drm_plane_state *old_plane_state =
190− drm_atomic_get_old_plane_state(state, plane);
191− struct drm_crtc *crtc = new_plane_state->crtc ?: old_plane_state->crtc;
192− const struct drm_crtc_state *old_crtc_state;
193− struct drm_crtc_state *new_crtc_state;
194−
195− new_plane_state->visible = false;
187+ if (new_plane_state->base.crtc)
188+ return to_intel_crtc(new_plane_state->base.crtc);
189+
190+ if (old_plane_state->base.crtc)
191+ return to_intel_crtc(old_plane_state->base.crtc);
192+
193+ return NULL;
194+}
195+
196+static int intel_plane_atomic_check(struct drm_plane *_plane,
197+ struct drm_plane_state *_new_plane_state)
198+{
199+ struct intel_plane *plane = to_intel_plane(_plane);
200+ struct intel_atomic_state *state =
201+ to_intel_atomic_state(_new_plane_state->state);
202+ struct intel_plane_state *new_plane_state =
203+ to_intel_plane_state(_new_plane_state);
204+ const struct intel_plane_state *old_plane_state =
205+ intel_atomic_get_old_plane_state(state, plane);
206+ struct intel_crtc *crtc =
207+ get_crtc_from_states(old_plane_state, new_plane_state);
208+ const struct intel_crtc_state *old_crtc_state;
209+ struct intel_crtc_state *new_crtc_state;
210+
211+ new_plane_state->base.visible = false;
196212 if (!crtc)
197213 return 0;
198214
199− old_crtc_state = drm_atomic_get_old_crtc_state(state, crtc);
200− new_crtc_state = drm_atomic_get_new_crtc_state(state, crtc);
215+ old_crtc_state = intel_atomic_get_old_crtc_state(state, crtc);
216+ new_crtc_state = intel_atomic_get_new_crtc_state(state, crtc);
201217
202− return intel_plane_atomic_check_with_state(to_intel_crtc_state(old_crtc_state),
203− to_intel_crtc_state(new_crtc_state),
204− to_intel_plane_state(old_plane_state),
205− to_intel_plane_state(new_plane_state));
218+ return intel_plane_atomic_check_with_state(old_crtc_state,
219+ new_crtc_state,
220+ old_plane_state,
221+ new_plane_state);
206222 }
207223
208224 static struct intel_plane *
drivers/gpu/drm/i915/display/intel_atomic_plane.h+2−3
@@ -8,7 +8,6 @@ · 7 unmodified lines
88
99 #include <linux/types.h>
1010
11−struct drm_crtc_state;
1211 struct drm_plane;
1312 struct drm_property;
1413 struct intel_atomic_state;
@@ -43,8 +42,8 @@ int intel_plane_atomic_check_with_state(const struct intel_crtc_state *old_crtc_ · 28 unmodified lines
4342 const struct intel_plane_state *old_plane_state,
4443 struct intel_plane_state *intel_state);
4544 int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_crtc_state,
46− struct drm_crtc_state *crtc_state,
45+ struct intel_crtc_state *crtc_state,
4746 const struct intel_plane_state *old_plane_state,
48− struct drm_plane_state *plane_state);
47+ struct intel_plane_state *plane_state);
4948
5049 #endif /* __INTEL_ATOMIC_PLANE_H__ */
drivers/gpu/drm/i915/display/intel_audio.c+68−13
@@ -72,6 +72,13 @@ struct dp_aud_n_m { · 71 unmodified lines
7272 u16 n;
7373 };
7474
75+struct hdmi_aud_ncts {
76+ int sample_rate;
77+ int clock;
78+ int n;
79+ int cts;
80+};
81+
7582 /* Values according to DP 1.4 Table 2-104 */
7683 static const struct dp_aud_n_m dp_aud_n_m[] = {
7784 { 32000, LC_162M, 1024, 10125 },
@@ -148,12 +155,7 @@ static const struct { · 70 unmodified lines
148155 #define TMDS_594M 594000
149156 #define TMDS_593M 593407
150157
151−static const struct {
152− int sample_rate;
153− int clock;
154− int n;
155− int cts;
156−} hdmi_aud_ncts[] = {
158+static const struct hdmi_aud_ncts hdmi_aud_ncts_24bpp[] = {
157159 { 32000, TMDS_296M, 5824, 421875 },
158160 { 32000, TMDS_297M, 3072, 222750 },
159161 { 32000, TMDS_593M, 5824, 843750 },
@@ -184,6 +186,49 @@ static const struct { · 24 unmodified lines
184186 { 192000, TMDS_594M, 24576, 594000 },
185187 };
186188
189+/* Appendix C - N & CTS values for deep color from HDMI 2.0 spec*/
190+/* HDMI N/CTS table for 10 bit deep color(30 bpp)*/
191+#define TMDS_371M 371250
192+#define TMDS_370M 370878
193+
194+static const struct hdmi_aud_ncts hdmi_aud_ncts_30bpp[] = {
195+ { 32000, TMDS_370M, 5824, 527344 },
196+ { 32000, TMDS_371M, 6144, 556875 },
197+ { 44100, TMDS_370M, 8918, 585938 },
198+ { 44100, TMDS_371M, 4704, 309375 },
199+ { 88200, TMDS_370M, 17836, 585938 },
200+ { 88200, TMDS_371M, 9408, 309375 },
201+ { 176400, TMDS_370M, 35672, 585938 },
202+ { 176400, TMDS_371M, 18816, 309375 },
203+ { 48000, TMDS_370M, 11648, 703125 },
204+ { 48000, TMDS_371M, 5120, 309375 },
205+ { 96000, TMDS_370M, 23296, 703125 },
206+ { 96000, TMDS_371M, 10240, 309375 },
207+ { 192000, TMDS_370M, 46592, 703125 },
208+ { 192000, TMDS_371M, 20480, 309375 },
209+};
210+
211+/* HDMI N/CTS table for 12 bit deep color(36 bpp)*/
212+#define TMDS_445_5M 445500
213+#define TMDS_445M 445054
214+
215+static const struct hdmi_aud_ncts hdmi_aud_ncts_36bpp[] = {
216+ { 32000, TMDS_445M, 5824, 632813 },
217+ { 32000, TMDS_445_5M, 4096, 445500 },
218+ { 44100, TMDS_445M, 8918, 703125 },
219+ { 44100, TMDS_445_5M, 4704, 371250 },
220+ { 88200, TMDS_445M, 17836, 703125 },
221+ { 88200, TMDS_445_5M, 9408, 371250 },
222+ { 176400, TMDS_445M, 35672, 703125 },
223+ { 176400, TMDS_445_5M, 18816, 371250 },
224+ { 48000, TMDS_445M, 5824, 421875 },
225+ { 48000, TMDS_445_5M, 5120, 371250 },
226+ { 96000, TMDS_445M, 11648, 421875 },
227+ { 96000, TMDS_445_5M, 10240, 371250 },
228+ { 192000, TMDS_445M, 23296, 421875 },
229+ { 192000, TMDS_445_5M, 20480, 371250 },
230+};
231+
187232 /* get AUD_CONFIG_PIXEL_CLOCK_HDMI_* value for mode */
188233 static u32 audio_config_hdmi_pixel_clock(const struct intel_crtc_state *crtc_state)
189234 {
@@ -212,14 +257,24 @@ static u32 audio_config_hdmi_pixel_clock(const struct intel_crtc_state *crtc_sta · 22 unmodified lines
212257 static int audio_config_hdmi_get_n(const struct intel_crtc_state *crtc_state,
213258 int rate)
214259 {
215− const struct drm_display_mode *adjusted_mode =
216− &crtc_state->base.adjusted_mode;
217− int i;
260+ const struct hdmi_aud_ncts *hdmi_ncts_table;
261+ int i, size;
262+
263+ if (crtc_state->pipe_bpp == 36) {
264+ hdmi_ncts_table = hdmi_aud_ncts_36bpp;
265+ size = ARRAY_SIZE(hdmi_aud_ncts_36bpp);
266+ } else if (crtc_state->pipe_bpp == 30) {
267+ hdmi_ncts_table = hdmi_aud_ncts_30bpp;
268+ size = ARRAY_SIZE(hdmi_aud_ncts_30bpp);
269+ } else {
270+ hdmi_ncts_table = hdmi_aud_ncts_24bpp;
271+ size = ARRAY_SIZE(hdmi_aud_ncts_24bpp);
272+ }
218273
219− for (i = 0; i < ARRAY_SIZE(hdmi_aud_ncts); i++) {
220− if (rate == hdmi_aud_ncts[i].sample_rate &&
221− adjusted_mode->crtc_clock == hdmi_aud_ncts[i].clock) {
222− return hdmi_aud_ncts[i].n;
274+ for (i = 0; i < size; i++) {
275+ if (rate == hdmi_ncts_table[i].sample_rate &&
276+ crtc_state->port_clock == hdmi_ncts_table[i].clock) {
277+ return hdmi_ncts_table[i].n;
223278 }
224279 }
225280 return 0;
drivers/gpu/drm/i915/display/intel_bios.c+24−4
@@ -28,6 +28,7 @@ · 27 unmodified lines
2828 #include <drm/drm_dp_helper.h>
2929 #include <drm/i915_drm.h>
3030
31+#include "display/intel_display.h"
3132 #include "display/intel_gmbus.h"
3233
3334 #include "i915_drv.h"
@@ -1354,12 +1355,27 @@ static const u8 mcc_ddc_pin_map[] = { · 1320 unmodified lines
13541355 [MCC_DDC_BUS_DDI_C] = GMBUS_PIN_9_TC1_ICP,
13551356 };
13561357
1358+static const u8 tgp_ddc_pin_map[] = {
1359+ [ICL_DDC_BUS_DDI_A] = GMBUS_PIN_1_BXT,
1360+ [ICL_DDC_BUS_DDI_B] = GMBUS_PIN_2_BXT,
1361+ [TGL_DDC_BUS_DDI_C] = GMBUS_PIN_3_BXT,
1362+ [ICL_DDC_BUS_PORT_1] = GMBUS_PIN_9_TC1_ICP,
1363+ [ICL_DDC_BUS_PORT_2] = GMBUS_PIN_10_TC2_ICP,
1364+ [ICL_DDC_BUS_PORT_3] = GMBUS_PIN_11_TC3_ICP,
1365+ [ICL_DDC_BUS_PORT_4] = GMBUS_PIN_12_TC4_ICP,
1366+ [TGL_DDC_BUS_PORT_5] = GMBUS_PIN_13_TC5_TGP,
1367+ [TGL_DDC_BUS_PORT_6] = GMBUS_PIN_14_TC6_TGP,
1368+};
1369+
13571370 static u8 map_ddc_pin(struct drm_i915_private *dev_priv, u8 vbt_pin)
13581371 {
13591372 const u8 *ddc_pin_map;
13601373 int n_entries;
13611374
1362− if (HAS_PCH_MCC(dev_priv)) {
1375+ if (HAS_PCH_TGP(dev_priv)) {
1376+ ddc_pin_map = tgp_ddc_pin_map;
1377+ n_entries = ARRAY_SIZE(tgp_ddc_pin_map);
1378+ } else if (HAS_PCH_MCC(dev_priv)) {
13631379 ddc_pin_map = mcc_ddc_pin_map;
13641380 n_entries = ARRAY_SIZE(mcc_ddc_pin_map);
13651381 } else if (HAS_PCH_ICP(dev_priv)) {
@@ -1668,6 +1684,9 @@ parse_general_definitions(struct drm_i915_private *dev_priv, · 302 unmodified lines
16681684 if (!child->device_type)
16691685 continue;
16701686
1687+ DRM_DEBUG_KMS("Found VBT child device with type 0x%x\n",
1688+ child->device_type);
1689+
16711690 /*
16721691 * Copy as much as we know (sizeof) and is available
16731692 * (child_dev_size) of the child device. Accessing the data must
@@ -1730,12 +1749,13 @@ init_vbt_missing_defaults(struct drm_i915_private *dev_priv) · 56 unmodified lines
17301749 for (port = PORT_A; port < I915_MAX_PORTS; port++) {
17311750 struct ddi_vbt_port_info *info =
17321751 &dev_priv->vbt.ddi_port_info[port];
1752+ enum phy phy = intel_port_to_phy(dev_priv, port);
17331753
17341754 /*
17351755 * VBT has the TypeC mode (native,TBT/USB) and we don't want
17361756 * to detect it.
17371757 */
1738− if (intel_port_is_tc(dev_priv, port))
1758+ if (intel_phy_is_tc(dev_priv, phy))
17391759 continue;
17401760
17411761 info->supports_dvi = (port != PORT_A && port != PORT_E);
@@ -1888,10 +1908,10 @@ out: · 146 unmodified lines
18881908 }
18891909
18901910 /**
1891− * intel_bios_cleanup - Free any resources allocated by intel_bios_init()
1911+ * intel_bios_driver_remove - Free any resources allocated by intel_bios_init()
18921912 * @dev_priv: i915 device instance
18931913 */
1894−void intel_bios_cleanup(struct drm_i915_private *dev_priv)
1914+void intel_bios_driver_remove(struct drm_i915_private *dev_priv)
18951915 {
18961916 kfree(dev_priv->vbt.child_dev);
18971917 dev_priv->vbt.child_dev = NULL;
drivers/gpu/drm/i915/display/intel_bios.h+2−1
@@ -42,6 +42,7 @@ enum intel_backlight_type { · 41 unmodified lines
4242 INTEL_BACKLIGHT_DISPLAY_DDI,
4343 INTEL_BACKLIGHT_DSI_DCS,
4444 INTEL_BACKLIGHT_PANEL_DRIVER_INTERFACE,
45+ INTEL_BACKLIGHT_VESA_EDP_AUX_INTERFACE,
4546 };
4647
4748 struct edp_power_seq {
@@ -227,7 +228,7 @@ struct mipi_pps_data { · 179 unmodified lines
227228 } __packed;
228229
229230 void intel_bios_init(struct drm_i915_private *dev_priv);
230−void intel_bios_cleanup(struct drm_i915_private *dev_priv);
231+void intel_bios_driver_remove(struct drm_i915_private *dev_priv);
231232 bool intel_bios_is_valid_vbt(const void *buf, size_t size);
232233 bool intel_bios_is_tv_present(struct drm_i915_private *dev_priv);
233234 bool intel_bios_is_lvds_present(struct drm_i915_private *dev_priv, u8 *i2c_pin);
drivers/gpu/drm/i915/display/intel_bw.c+1−1
@@ -65,7 +65,7 @@ static int icl_pcode_read_qgv_point_info(struct drm_i915_private *dev_priv, · 64 unmodified lines
6565 struct intel_qgv_point *sp,
6666 int point)
6767 {
68− u32 val = 0, val2;
68+ u32 val = 0, val2 = 0;
6969 int ret;
7070
7171 ret = sandybridge_pcode_read(dev_priv,
drivers/gpu/drm/i915/display/intel_cdclk.c+56−28
@@ -545,10 +545,10 @@ static void vlv_set_cdclk(struct drm_i915_private *dev_priv, · 544 unmodified lines
545545 /* There are cases where we can end up here with power domains
546546 * off and a CDCLK frequency other than the minimum, like when
547547 * issuing a modeset without actually changing any display after
548− * a system suspend. So grab the PIPE-A domain, which covers
548+ * a system suspend. So grab the display core domain, which covers
549549 * the HW blocks needed for the following programming.
550550 */
551− wakeref = intel_display_power_get(dev_priv, POWER_DOMAIN_PIPE_A);
551+ wakeref = intel_display_power_get(dev_priv, POWER_DOMAIN_DISPLAY_CORE);
552552
553553 vlv_iosf_sb_get(dev_priv,
554554 BIT(VLV_IOSF_SB_CCK) |
@@ -606,7 +606,7 @@ static void vlv_set_cdclk(struct drm_i915_private *dev_priv, · 51 unmodified lines
606606
607607 vlv_program_pfi_credits(dev_priv);
608608
609− intel_display_power_put(dev_priv, POWER_DOMAIN_PIPE_A, wakeref);
609+ intel_display_power_put(dev_priv, POWER_DOMAIN_DISPLAY_CORE, wakeref);
610610 }
611611
612612 static void chv_set_cdclk(struct drm_i915_private *dev_priv,
@@ -631,10 +631,10 @@ static void chv_set_cdclk(struct drm_i915_private *dev_priv, · 18 unmodified lines
631631 /* There are cases where we can end up here with power domains
632632 * off and a CDCLK frequency other than the minimum, like when
633633 * issuing a modeset without actually changing any display after
634− * a system suspend. So grab the PIPE-A domain, which covers
634+ * a system suspend. So grab the display core domain, which covers
635635 * the HW blocks needed for the following programming.
636636 */
637− wakeref = intel_display_power_get(dev_priv, POWER_DOMAIN_PIPE_A);
637+ wakeref = intel_display_power_get(dev_priv, POWER_DOMAIN_DISPLAY_CORE);
638638
639639 vlv_punit_get(dev_priv);
640640 val = vlv_punit_read(dev_priv, PUNIT_REG_DSPSSPM);
@@ -653,7 +653,7 @@ static void chv_set_cdclk(struct drm_i915_private *dev_priv, · 12 unmodified lines
653653
654654 vlv_program_pfi_credits(dev_priv);
655655
656− intel_display_power_put(dev_priv, POWER_DOMAIN_PIPE_A, wakeref);
656+ intel_display_power_put(dev_priv, POWER_DOMAIN_DISPLAY_CORE, wakeref);
657657 }
658658
659659 static int bdw_calc_cdclk(int min_cdclk)
@@ -1756,9 +1756,10 @@ sanitize: · 1096 unmodified lines
17561756
17571757 static int icl_calc_cdclk(int min_cdclk, unsigned int ref)
17581758 {
1759− int ranges_24[] = { 312000, 552000, 648000 };
1760− int ranges_19_38[] = { 307200, 556800, 652800 };
1761− int *ranges;
1759+ static const int ranges_24[] = { 180000, 192000, 312000, 552000, 648000 };
1760+ static const int ranges_19_38[] = { 172800, 192000, 307200, 556800, 652800 };
1761+ const int *ranges;
1762+ int len, i;
17621763
17631764 switch (ref) {
17641765 default:
@@ -1766,19 +1767,22 @@ static int icl_calc_cdclk(int min_cdclk, unsigned int ref) · 1 unmodified line
17661767 /* fall through */
17671768 case 24000:
17681769 ranges = ranges_24;
1770+ len = ARRAY_SIZE(ranges_24);
17691771 break;
17701772 case 19200:
17711773 case 38400:
17721774 ranges = ranges_19_38;
1775+ len = ARRAY_SIZE(ranges_19_38);
17731776 break;
17741777 }
17751778
1776− if (min_cdclk > ranges[1])
1777− return ranges[2];
1778− else if (min_cdclk > ranges[0])
1779− return ranges[1];
1780− else
1781− return ranges[0];
1779+ for (i = 0; i < len; i++) {
1780+ if (min_cdclk <= ranges[i])
1781+ return ranges[i];
1782+ }
1783+
1784+ WARN_ON(min_cdclk > ranges[len - 1]);
1785+ return ranges[len - 1];
17821786 }
17831787
17841788 static int icl_calc_cdclk_pll_vco(struct drm_i915_private *dev_priv, int cdclk)
@@ -1792,16 +1796,24 @@ static int icl_calc_cdclk_pll_vco(struct drm_i915_private *dev_priv, int cdclk) · 7 unmodified lines
17921796 default:
17931797 MISSING_CASE(cdclk);
17941798 /* fall through */
1799+ case 172800:
17951800 case 307200:
17961801 case 556800:
17971802 case 652800:
17981803 WARN_ON(dev_priv->cdclk.hw.ref != 19200 &&
17991804 dev_priv->cdclk.hw.ref != 38400);
18001805 break;
1806+ case 180000:
18011807 case 312000:
18021808 case 552000:
18031809 case 648000:
18041810 WARN_ON(dev_priv->cdclk.hw.ref != 24000);
1811+ break;
1812+ case 192000:
1813+ WARN_ON(dev_priv->cdclk.hw.ref != 19200 &&
1814+ dev_priv->cdclk.hw.ref != 38400 &&
1815+ dev_priv->cdclk.hw.ref != 24000);
1816+ break;
18051817 }
18061818
18071819 ratio = cdclk / (dev_priv->cdclk.hw.ref / 2);
@@ -1854,14 +1866,23 @@ static void icl_set_cdclk(struct drm_i915_private *dev_priv, · 46 unmodified lines
18541866 dev_priv->cdclk.hw.voltage_level = cdclk_state->voltage_level;
18551867 }
18561868
1857−static u8 icl_calc_voltage_level(int cdclk)
1869+static u8 icl_calc_voltage_level(struct drm_i915_private *dev_priv, int cdclk)
18581870 {
1859− if (cdclk > 556800)
1860− return 2;
1861− else if (cdclk > 312000)
1862− return 1;
1863− else
1864− return 0;
1871+ if (IS_ELKHARTLAKE(dev_priv)) {
1872+ if (cdclk > 312000)
1873+ return 2;
1874+ else if (cdclk > 180000)
1875+ return 1;
1876+ else
1877+ return 0;
1878+ } else {
1879+ if (cdclk > 556800)
1880+ return 2;
1881+ else if (cdclk > 312000)
1882+ return 1;
1883+ else
1884+ return 0;
1885+ }
18651886 }
18661887
18671888 static void icl_get_cdclk(struct drm_i915_private *dev_priv,
@@ -1912,7 +1933,7 @@ out: · 44 unmodified lines
19121933 * at least what the CDCLK frequency requires.
19131934 */
19141935 cdclk_state->voltage_level =
1915− icl_calc_voltage_level(cdclk_state->cdclk);
1936+ icl_calc_voltage_level(dev_priv, cdclk_state->cdclk);
19161937 }
19171938
19181939 static void icl_init_cdclk(struct drm_i915_private *dev_priv)
@@ -1947,7 +1968,8 @@ sanitize: · 28 unmodified lines
19471968 sanitized_state.vco = icl_calc_cdclk_pll_vco(dev_priv,
19481969 sanitized_state.cdclk);
19491970 sanitized_state.voltage_level =
1950− icl_calc_voltage_level(sanitized_state.cdclk);
1971+ icl_calc_voltage_level(dev_priv,
1972+ sanitized_state.cdclk);
19511973
19521974 icl_set_cdclk(dev_priv, &sanitized_state, INVALID_PIPE);
19531975 }
@@ -1958,7 +1980,8 @@ static void icl_uninit_cdclk(struct drm_i915_private *dev_priv) · 4 unmodified lines
19581980
19591981 cdclk_state.cdclk = cdclk_state.bypass;
19601982 cdclk_state.vco = 0;
1961− cdclk_state.voltage_level = icl_calc_voltage_level(cdclk_state.cdclk);
1983+ cdclk_state.voltage_level = icl_calc_voltage_level(dev_priv,
1984+ cdclk_state.cdclk);
19621985
19631986 icl_set_cdclk(dev_priv, &cdclk_state, INVALID_PIPE);
19641987 }
@@ -2560,7 +2583,7 @@ static int icl_modeset_calc_cdclk(struct intel_atomic_state *state) · 595 unmodified lines
25602583 state->cdclk.logical.vco = vco;
25612584 state->cdclk.logical.cdclk = cdclk;
25622585 state->cdclk.logical.voltage_level =
2563− max(icl_calc_voltage_level(cdclk),
2586+ max(icl_calc_voltage_level(dev_priv, cdclk),
25642587 cnl_compute_min_voltage_level(state));
25652588
25662589 if (!state->active_crtcs) {
@@ -2570,7 +2593,7 @@ static int icl_modeset_calc_cdclk(struct intel_atomic_state *state) · 3 unmodified lines
25702593 state->cdclk.actual.vco = vco;
25712594 state->cdclk.actual.cdclk = cdclk;
25722595 state->cdclk.actual.voltage_level =
2573− icl_calc_voltage_level(cdclk);
2596+ icl_calc_voltage_level(dev_priv, cdclk);
25742597 } else {
25752598 state->cdclk.actual = state->cdclk.logical;
25762599 }
@@ -2605,7 +2628,12 @@ static int intel_compute_max_dotclk(struct drm_i915_private *dev_priv) · 28 unmodified lines
26052628 */
26062629 void intel_update_max_cdclk(struct drm_i915_private *dev_priv)
26072630 {
2608− if (INTEL_GEN(dev_priv) >= 11) {
2631+ if (IS_ELKHARTLAKE(dev_priv)) {
2632+ if (dev_priv->cdclk.hw.ref == 24000)
2633+ dev_priv->max_cdclk_freq = 552000;
2634+ else
2635+ dev_priv->max_cdclk_freq = 556800;
2636+ } else if (INTEL_GEN(dev_priv) >= 11) {
26092637 if (dev_priv->cdclk.hw.ref == 24000)
26102638 dev_priv->max_cdclk_freq = 648000;
26112639 else
drivers/gpu/drm/i915/display/intel_combo_phy.c+125−68
@@ -6,13 +6,13 @@ · 5 unmodified lines
66 #include "intel_combo_phy.h"
77 #include "intel_drv.h"
88
9−#define for_each_combo_port(__dev_priv, __port) \
10− for ((__port) = PORT_A; (__port) < I915_MAX_PORTS; (__port)++) \
11− for_each_if(intel_port_is_combophy(__dev_priv, __port))
9+#define for_each_combo_phy(__dev_priv, __phy) \
10+ for ((__phy) = PHY_A; (__phy) < I915_MAX_PHYS; (__phy)++) \
11+ for_each_if(intel_phy_is_combo(__dev_priv, __phy))
1212
13−#define for_each_combo_port_reverse(__dev_priv, __port) \
14− for ((__port) = I915_MAX_PORTS; (__port)-- > PORT_A;) \
15− for_each_if(intel_port_is_combophy(__dev_priv, __port))
13+#define for_each_combo_phy_reverse(__dev_priv, __phy) \
14+ for ((__phy) = I915_MAX_PHYS; (__phy)-- > PHY_A;) \
15+ for_each_if(intel_phy_is_combo(__dev_priv, __phy))
1616
1717 enum {
1818 PROCMON_0_85V_DOT_0,
@@ -38,18 +38,17 @@ static const struct cnl_procmon { · 19 unmodified lines
3838 };
3939
4040 /*
41− * CNL has just one set of registers, while ICL has two sets: one for port A and
42− * the other for port B. The CNL registers are equivalent to the ICL port A
43− * registers, that's why we call the ICL macros even though the function has CNL
44− * on its name.
41+ * CNL has just one set of registers, while gen11 has a set for each combo PHY.
42+ * The CNL registers are equivalent to the gen11 PHY A registers, that's why we
43+ * call the ICL macros even though the function has CNL on its name.
4544 */
4645 static const struct cnl_procmon *
47−cnl_get_procmon_ref_values(struct drm_i915_private *dev_priv, enum port port)
46+cnl_get_procmon_ref_values(struct drm_i915_private *dev_priv, enum phy phy)
4847 {
4948 const struct cnl_procmon *procmon;
5049 u32 val;
5150
52− val = I915_READ(ICL_PORT_COMP_DW3(port));
51+ val = I915_READ(ICL_PORT_COMP_DW3(phy));
5352 switch (val & (PROCESS_INFO_MASK | VOLTAGE_INFO_MASK)) {
5453 default:
5554 MISSING_CASE(val);
@@ -75,32 +74,32 @@ cnl_get_procmon_ref_values(struct drm_i915_private *dev_priv, enum port port) · 19 unmodified lines
7574 }
7675
7776 static void cnl_set_procmon_ref_values(struct drm_i915_private *dev_priv,
78− enum port port)
77+ enum phy phy)
7978 {
8079 const struct cnl_procmon *procmon;
8180 u32 val;
8281
83− procmon = cnl_get_procmon_ref_values(dev_priv, port);
82+ procmon = cnl_get_procmon_ref_values(dev_priv, phy);
8483
85− val = I915_READ(ICL_PORT_COMP_DW1(port));
84+ val = I915_READ(ICL_PORT_COMP_DW1(phy));
8685 val &= ~((0xff << 16) | 0xff);
8786 val |= procmon->dw1;
88− I915_WRITE(ICL_PORT_COMP_DW1(port), val);
87+ I915_WRITE(ICL_PORT_COMP_DW1(phy), val);
8988
90− I915_WRITE(ICL_PORT_COMP_DW9(port), procmon->dw9);
91− I915_WRITE(ICL_PORT_COMP_DW10(port), procmon->dw10);
89+ I915_WRITE(ICL_PORT_COMP_DW9(phy), procmon->dw9);
90+ I915_WRITE(ICL_PORT_COMP_DW10(phy), procmon->dw10);
9291 }
9392
9493 static bool check_phy_reg(struct drm_i915_private *dev_priv,
95− enum port port, i915_reg_t reg, u32 mask,
94+ enum phy phy, i915_reg_t reg, u32 mask,
9695 u32 expected_val)
9796 {
9897 u32 val = I915_READ(reg);
9998
10099 if ((val & mask) != expected_val) {
101− DRM_DEBUG_DRIVER("Port %c combo PHY reg %08x state mismatch: "
100+ DRM_DEBUG_DRIVER("Combo PHY %c reg %08x state mismatch: "
102101 "current %08x mask %08x expected %08x\n",
103− port_name(port),
102+ phy_name(phy),
104103 reg.reg, val, mask, expected_val);
105104 return false;
106105 }
@@ -109,18 +108,18 @@ static bool check_phy_reg(struct drm_i915_private *dev_priv, · 2 unmodified lines
109108 }
110109
111110 static bool cnl_verify_procmon_ref_values(struct drm_i915_private *dev_priv,
112− enum port port)
111+ enum phy phy)
113112 {
114113 const struct cnl_procmon *procmon;
115114 bool ret;
116115
117− procmon = cnl_get_procmon_ref_values(dev_priv, port);
116+ procmon = cnl_get_procmon_ref_values(dev_priv, phy);
118117
119− ret = check_phy_reg(dev_priv, port, ICL_PORT_COMP_DW1(port),
118+ ret = check_phy_reg(dev_priv, phy, ICL_PORT_COMP_DW1(phy),
120119 (0xff << 16) | 0xff, procmon->dw1);
121− ret &= check_phy_reg(dev_priv, port, ICL_PORT_COMP_DW9(port),
120+ ret &= check_phy_reg(dev_priv, phy, ICL_PORT_COMP_DW9(phy),
122121 -1U, procmon->dw9);
123− ret &= check_phy_reg(dev_priv, port, ICL_PORT_COMP_DW10(port),
122+ ret &= check_phy_reg(dev_priv, phy, ICL_PORT_COMP_DW10(phy),
124123 -1U, procmon->dw10);
125124
126125 return ret;
@@ -134,15 +133,15 @@ static bool cnl_combo_phy_enabled(struct drm_i915_private *dev_priv) · 7 unmodified lines
134133
135134 static bool cnl_combo_phy_verify_state(struct drm_i915_private *dev_priv)
136135 {
137− enum port port = PORT_A;
136+ enum phy phy = PHY_A;
138137 bool ret;
139138
140139 if (!cnl_combo_phy_enabled(dev_priv))
141140 return false;
142141
143− ret = cnl_verify_procmon_ref_values(dev_priv, port);
142+ ret = cnl_verify_procmon_ref_values(dev_priv, phy);
144143
145− ret &= check_phy_reg(dev_priv, port, CNL_PORT_CL1CM_DW5,
144+ ret &= check_phy_reg(dev_priv, phy, CNL_PORT_CL1CM_DW5,
146145 CL_POWER_DOWN_ENABLE, CL_POWER_DOWN_ENABLE);
147146
148147 return ret;
@@ -157,7 +156,7 @@ static void cnl_combo_phys_init(struct drm_i915_private *dev_priv) · 8 unmodified lines
157156 I915_WRITE(CHICKEN_MISC_2, val);
158157
159158 /* Dummy PORT_A to get the correct CNL register from the ICL macro */
160− cnl_set_procmon_ref_values(dev_priv, PORT_A);
159+ cnl_set_procmon_ref_values(dev_priv, PHY_A);
161160
162161 val = I915_READ(CNL_PORT_COMP_DW0);
163162 val |= COMP_INIT;
@@ -181,35 +180,39 @@ static void cnl_combo_phys_uninit(struct drm_i915_private *dev_priv) · 17 unmodified lines
181180 }
182181
183182 static bool icl_combo_phy_enabled(struct drm_i915_private *dev_priv,
184− enum port port)
183+ enum phy phy)
185184 {
186− return !(I915_READ(ICL_PHY_MISC(port)) &
187− ICL_PHY_MISC_DE_IO_COMP_PWR_DOWN) &&
188− (I915_READ(ICL_PORT_COMP_DW0(port)) & COMP_INIT);
185+ /* The PHY C added by EHL has no PHY_MISC register */
186+ if (IS_ELKHARTLAKE(dev_priv) && phy == PHY_C)
187+ return I915_READ(ICL_PORT_COMP_DW0(phy)) & COMP_INIT;
188+ else
189+ return !(I915_READ(ICL_PHY_MISC(phy)) &
190+ ICL_PHY_MISC_DE_IO_COMP_PWR_DOWN) &&
191+ (I915_READ(ICL_PORT_COMP_DW0(phy)) & COMP_INIT);
189192 }
190193
191194 static bool icl_combo_phy_verify_state(struct drm_i915_private *dev_priv,
192− enum port port)
195+ enum phy phy)
193196 {
194197 bool ret;
195198
196− if (!icl_combo_phy_enabled(dev_priv, port))
199+ if (!icl_combo_phy_enabled(dev_priv, phy))
197200 return false;
198201
199− ret = cnl_verify_procmon_ref_values(dev_priv, port);
202+ ret = cnl_verify_procmon_ref_values(dev_priv, phy);
200203
201− if (port == PORT_A)
202− ret &= check_phy_reg(dev_priv, port, ICL_PORT_COMP_DW8(port),
204+ if (phy == PHY_A)
205+ ret &= check_phy_reg(dev_priv, phy, ICL_PORT_COMP_DW8(phy),
203206 IREFGEN, IREFGEN);
204207
205− ret &= check_phy_reg(dev_priv, port, ICL_PORT_CL_DW5(port),
208+ ret &= check_phy_reg(dev_priv, phy, ICL_PORT_CL_DW5(phy),
206209 CL_POWER_DOWN_ENABLE, CL_POWER_DOWN_ENABLE);
207210
208211 return ret;
209212 }
210213
211214 void intel_combo_phy_power_up_lanes(struct drm_i915_private *dev_priv,
212− enum port port, bool is_dsi,
215+ enum phy phy, bool is_dsi,
213216 int lane_count, bool lane_reversal)
214217 {
215218 u8 lane_mask;
@@ -254,66 +257,120 @@ void intel_combo_phy_power_up_lanes(struct drm_i915_private *dev_priv, · 38 unmodified lines
254257 }
255258 }
256259
257− val = I915_READ(ICL_PORT_CL_DW10(port));
260+ val = I915_READ(ICL_PORT_CL_DW10(phy));
258261 val &= ~PWR_DOWN_LN_MASK;
259262 val |= lane_mask << PWR_DOWN_LN_SHIFT;
260− I915_WRITE(ICL_PORT_CL_DW10(port), val);
263+ I915_WRITE(ICL_PORT_CL_DW10(phy), val);
264+}
265+
266+static u32 ehl_combo_phy_a_mux(struct drm_i915_private *i915, u32 val)
267+{
268+ bool ddi_a_present = i915->vbt.ddi_port_info[PORT_A].child != NULL;
269+ bool ddi_d_present = i915->vbt.ddi_port_info[PORT_D].child != NULL;
270+ bool dsi_present = intel_bios_is_dsi_present(i915, NULL);
271+
272+ /*
273+ * VBT's 'dvo port' field for child devices references the DDI, not
274+ * the PHY. So if combo PHY A is wired up to drive an external
275+ * display, we should see a child device present on PORT_D and
276+ * nothing on PORT_A and no DSI.
277+ */
278+ if (ddi_d_present && !ddi_a_present && !dsi_present)
279+ return val | ICL_PHY_MISC_MUX_DDID;
280+
281+ /*
282+ * If we encounter a VBT that claims to have an external display on
283+ * DDI-D _and_ an internal display on DDI-A/DSI leave an error message
284+ * in the log and let the internal display win.
285+ */
286+ if (ddi_d_present)
287+ DRM_ERROR("VBT claims to have both internal and external displays on PHY A. Configuring for internal.\n");
288+
289+ return val & ~ICL_PHY_MISC_MUX_DDID;
261290 }
262291
263292 static void icl_combo_phys_init(struct drm_i915_private *dev_priv)
264293 {
265− enum port port;
294+ enum phy phy;
266295
267− for_each_combo_port(dev_priv, port) {
296+ for_each_combo_phy(dev_priv, phy) {
268297 u32 val;
269298
270− if (icl_combo_phy_verify_state(dev_priv, port)) {
271− DRM_DEBUG_DRIVER("Port %c combo PHY already enabled, won't reprogram it.\n",
272− port_name(port));
299+ if (icl_combo_phy_verify_state(dev_priv, phy)) {
300+ DRM_DEBUG_DRIVER("Combo PHY %c already enabled, won't reprogram it.\n",
301+ phy_name(phy));
273302 continue;
274303 }
275304
276− val = I915_READ(ICL_PHY_MISC(port));
305+ /*
306+ * Although EHL adds a combo PHY C, there's no PHY_MISC
307+ * register for it and no need to program the
308+ * DE_IO_COMP_PWR_DOWN setting on PHY C.
309+ */
310+ if (IS_ELKHARTLAKE(dev_priv) && phy == PHY_C)
311+ goto skip_phy_misc;
312+
313+ /*
314+ * EHL's combo PHY A can be hooked up to either an external
315+ * display (via DDI-D) or an internal display (via DDI-A or
316+ * the DSI DPHY). This is a motherboard design decision that
317+ * can't be changed on the fly, so initialize the PHY's mux
318+ * based on whether our VBT indicates the presence of any
319+ * "internal" child devices.
320+ */
321+ val = I915_READ(ICL_PHY_MISC(phy));
322+ if (IS_ELKHARTLAKE(dev_priv) && phy == PHY_A)
323+ val = ehl_combo_phy_a_mux(dev_priv, val);
277324 val &= ~ICL_PHY_MISC_DE_IO_COMP_PWR_DOWN;
278− I915_WRITE(ICL_PHY_MISC(port), val);
325+ I915_WRITE(ICL_PHY_MISC(phy), val);
279326
280− cnl_set_procmon_ref_values(dev_priv, port);
327+skip_phy_misc:
328+ cnl_set_procmon_ref_values(dev_priv, phy);
281329
282− if (port == PORT_A) {
283− val = I915_READ(ICL_PORT_COMP_DW8(port));
330+ if (phy == PHY_A) {
331+ val = I915_READ(ICL_PORT_COMP_DW8(phy));
284332 val |= IREFGEN;
285− I915_WRITE(ICL_PORT_COMP_DW8(port), val);
333+ I915_WRITE(ICL_PORT_COMP_DW8(phy), val);
286334 }
287335
288− val = I915_READ(ICL_PORT_COMP_DW0(port));
336+ val = I915_READ(ICL_PORT_COMP_DW0(phy));
289337 val |= COMP_INIT;
290− I915_WRITE(ICL_PORT_COMP_DW0(port), val);
338+ I915_WRITE(ICL_PORT_COMP_DW0(phy), val);
291339
292− val = I915_READ(ICL_PORT_CL_DW5(port));
340+ val = I915_READ(ICL_PORT_CL_DW5(phy));
293341 val |= CL_POWER_DOWN_ENABLE;
294− I915_WRITE(ICL_PORT_CL_DW5(port), val);
342+ I915_WRITE(ICL_PORT_CL_DW5(phy), val);
295343 }
296344 }
297345
298346 static void icl_combo_phys_uninit(struct drm_i915_private *dev_priv)
299347 {
300− enum port port;
348+ enum phy phy;
301349
302− for_each_combo_port_reverse(dev_priv, port) {
350+ for_each_combo_phy_reverse(dev_priv, phy) {
303351 u32 val;
304352
305− if (port == PORT_A &&
306− !icl_combo_phy_verify_state(dev_priv, port))
307− DRM_WARN("Port %c combo PHY HW state changed unexpectedly\n",
308− port_name(port));
353+ if (phy == PHY_A &&
354+ !icl_combo_phy_verify_state(dev_priv, phy))
355+ DRM_WARN("Combo PHY %c HW state changed unexpectedly\n",
356+ phy_name(phy));
357+
358+ /*
359+ * Although EHL adds a combo PHY C, there's no PHY_MISC
360+ * register for it and no need to program the
361+ * DE_IO_COMP_PWR_DOWN setting on PHY C.
362+ */
363+ if (IS_ELKHARTLAKE(dev_priv) && phy == PHY_C)
364+ goto skip_phy_misc;
309365
310− val = I915_READ(ICL_PHY_MISC(port));
366+ val = I915_READ(ICL_PHY_MISC(phy));
311367 val |= ICL_PHY_MISC_DE_IO_COMP_PWR_DOWN;
312− I915_WRITE(ICL_PHY_MISC(port), val);
368+ I915_WRITE(ICL_PHY_MISC(phy), val);
313369
314− val = I915_READ(ICL_PORT_COMP_DW0(port));
370+skip_phy_misc:
371+ val = I915_READ(ICL_PORT_COMP_DW0(phy));
315372 val &= ~COMP_INIT;
316− I915_WRITE(ICL_PORT_COMP_DW0(port), val);
373+ I915_WRITE(ICL_PORT_COMP_DW0(phy), val);
317374 }
318375 }
319376
drivers/gpu/drm/i915/display/intel_combo_phy.h+2−2
@@ -7,14 +7,14 @@ · 6 unmodified lines
77 #define __INTEL_COMBO_PHY_H__
88
99 #include <linux/types.h>
10−#include <drm/i915_drm.h>
1110
1211 struct drm_i915_private;
12+enum phy;
1313
1414 void intel_combo_phy_init(struct drm_i915_private *dev_priv);
1515 void intel_combo_phy_uninit(struct drm_i915_private *dev_priv);
1616 void intel_combo_phy_power_up_lanes(struct drm_i915_private *dev_priv,
17− enum port port, bool is_dsi,
17+ enum phy phy, bool is_dsi,
1818 int lane_count, bool lane_reversal);
1919
2020 #endif /* __INTEL_COMBO_PHY_H__ */
drivers/gpu/drm/i915/display/intel_connector.c+1−1
@@ -118,7 +118,7 @@ int intel_connector_register(struct drm_connector *connector) · 117 unmodified lines
118118 if (ret)
119119 goto err;
120120
121− if (i915_inject_load_failure()) {
121+ if (i915_inject_probe_failure()) {
122122 ret = -EFAULT;
123123 goto err_backlight;
124124 }
drivers/gpu/drm/i915/display/intel_ddi.c+229−174
@@ -45,6 +45,7 @@ · 44 unmodified lines
4545 #include "intel_lspcon.h"
4646 #include "intel_panel.h"
4747 #include "intel_psr.h"
48+#include "intel_tc.h"
4849 #include "intel_vdsc.h"
4950
5051 struct ddi_buf_trans {
@@ -846,8 +847,8 @@ cnl_get_buf_trans_edp(struct drm_i915_private *dev_priv, int *n_entries) · 795 unmodified lines
846847 }
847848
848849 static const struct cnl_ddi_buf_trans *
849−icl_get_combo_buf_trans(struct drm_i915_private *dev_priv, enum port port,
850− int type, int rate, int *n_entries)
850+icl_get_combo_buf_trans(struct drm_i915_private *dev_priv, int type, int rate,
851+ int *n_entries)
851852 {
852853 if (type == INTEL_OUTPUT_HDMI) {
853854 *n_entries = ARRAY_SIZE(icl_combo_phy_ddi_translations_hdmi);
@@ -867,12 +868,13 @@ icl_get_combo_buf_trans(struct drm_i915_private *dev_priv, enum port port, · 13 unmodified lines
867868 static int intel_ddi_hdmi_level(struct drm_i915_private *dev_priv, enum port port)
868869 {
869870 int n_entries, level, default_entry;
871+ enum phy phy = intel_port_to_phy(dev_priv, port);
870872
871873 level = dev_priv->vbt.ddi_port_info[port].hdmi_level_shift;
872874
873875 if (INTEL_GEN(dev_priv) >= 11) {
874− if (intel_port_is_combophy(dev_priv, port))
875− icl_get_combo_buf_trans(dev_priv, port, INTEL_OUTPUT_HDMI,
876+ if (intel_phy_is_combo(dev_priv, phy))
877+ icl_get_combo_buf_trans(dev_priv, INTEL_OUTPUT_HDMI,
876878 0, &n_entries);
877879 else
878880 n_entries = ARRAY_SIZE(icl_mg_phy_ddi_translations);
@@ -1486,9 +1488,10 @@ static void icl_ddi_clock_get(struct intel_encoder *encoder, · 607 unmodified lines
14861488 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
14871489 struct intel_dpll_hw_state *pll_state = &pipe_config->dpll_hw_state;
14881490 enum port port = encoder->port;
1491+ enum phy phy = intel_port_to_phy(dev_priv, port);
14891492 int link_clock;
14901493
1491− if (intel_port_is_combophy(dev_priv, port)) {
1494+ if (intel_phy_is_combo(dev_priv, phy)) {
14921495 link_clock = cnl_calc_wrpll_link(dev_priv, pll_state);
14931496 } else {
14941497 enum intel_dpll_id pll_id = intel_get_shared_dpll_id(dev_priv,
@@ -1770,7 +1773,10 @@ void intel_ddi_enable_transcoder_func(const struct intel_crtc_state *crtc_state) · 275 unmodified lines
17701773
17711774 /* Enable TRANS_DDI_FUNC_CTL for the pipe to work in HDMI mode */
17721775 temp = TRANS_DDI_FUNC_ENABLE;
1773− temp |= TRANS_DDI_SELECT_PORT(port);
1776+ if (INTEL_GEN(dev_priv) >= 12)
1777+ temp |= TGL_TRANS_DDI_SELECT_PORT(port);
1778+ else
1779+ temp |= TRANS_DDI_SELECT_PORT(port);
17741780
17751781 switch (crtc_state->pipe_bpp) {
17761782 case 18:
@@ -1850,8 +1856,13 @@ void intel_ddi_disable_transcoder_func(const struct intel_crtc_state *crtc_state · 73 unmodified lines
18501856 i915_reg_t reg = TRANS_DDI_FUNC_CTL(cpu_transcoder);
18511857 u32 val = I915_READ(reg);
18521858
1853− val &= ~(TRANS_DDI_FUNC_ENABLE | TRANS_DDI_PORT_MASK | TRANS_DDI_DP_VC_PAYLOAD_ALLOC);
1854− val |= TRANS_DDI_PORT_NONE;
1859+ if (INTEL_GEN(dev_priv) >= 12) {
1860+ val &= ~(TRANS_DDI_FUNC_ENABLE | TGL_TRANS_DDI_PORT_MASK |
1861+ TRANS_DDI_DP_VC_PAYLOAD_ALLOC);
1862+ } else {
1863+ val &= ~(TRANS_DDI_FUNC_ENABLE | TRANS_DDI_PORT_MASK |
1864+ TRANS_DDI_DP_VC_PAYLOAD_ALLOC);
1865+ }
18551866 I915_WRITE(reg, val);
18561867
18571868 if (dev_priv->quirks & QUIRK_INCREASE_DDI_DISABLED_TIME &&
@@ -2003,10 +2014,19 @@ static void intel_ddi_get_encoder_pipes(struct intel_encoder *encoder, · 145 unmodified lines
20032014 mst_pipe_mask = 0;
20042015 for_each_pipe(dev_priv, p) {
20052016 enum transcoder cpu_transcoder = (enum transcoder)p;
2017+ unsigned int port_mask, ddi_select;
2018+
2019+ if (INTEL_GEN(dev_priv) >= 12) {
2020+ port_mask = TGL_TRANS_DDI_PORT_MASK;
2021+ ddi_select = TGL_TRANS_DDI_SELECT_PORT(port);
2022+ } else {
2023+ port_mask = TRANS_DDI_PORT_MASK;
2024+ ddi_select = TRANS_DDI_SELECT_PORT(port);
2025+ }
20062026
20072027 tmp = I915_READ(TRANS_DDI_FUNC_CTL(cpu_transcoder));
20082028
2009− if ((tmp & TRANS_DDI_PORT_MASK) != TRANS_DDI_SELECT_PORT(port))
2029+ if ((tmp & port_mask) != ddi_select)
20102030 continue;
20112031
20122032 if ((tmp & TRANS_DDI_MODE_SELECT_MASK) ==
@@ -2085,6 +2105,7 @@ static void intel_ddi_get_power_domains(struct intel_encoder *encoder, · 72 unmodified lines
20852105 {
20862106 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
20872107 struct intel_digital_port *dig_port;
2108+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
20882109
20892110 /*
20902111 * TODO: Add support for MST encoders. Atm, the following should never
@@ -2102,7 +2123,7 @@ static void intel_ddi_get_power_domains(struct intel_encoder *encoder, · 11 unmodified lines
21022123 * ports.
21032124 */
21042125 if (intel_crtc_has_dp_encoder(crtc_state) ||
2105− intel_port_is_tc(dev_priv, encoder->port))
2126+ intel_phy_is_tc(dev_priv, phy))
21062127 intel_display_power_get(dev_priv,
21072128 intel_ddi_main_link_aux_domain(dig_port));
21082129
@@ -2122,9 +2143,14 @@ void intel_ddi_enable_pipe_clock(const struct intel_crtc_state *crtc_state) · 13 unmodified lines
21222143 enum port port = encoder->port;
21232144 enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
21242145
2125− if (cpu_transcoder != TRANSCODER_EDP)
2126− I915_WRITE(TRANS_CLK_SEL(cpu_transcoder),
2127− TRANS_CLK_SEL_PORT(port));
2146+ if (cpu_transcoder != TRANSCODER_EDP) {
2147+ if (INTEL_GEN(dev_priv) >= 12)
2148+ I915_WRITE(TRANS_CLK_SEL(cpu_transcoder),
2149+ TGL_TRANS_CLK_SEL_PORT(port));
2150+ else
2151+ I915_WRITE(TRANS_CLK_SEL(cpu_transcoder),
2152+ TRANS_CLK_SEL_PORT(port));
2153+ }
21282154 }
21292155
21302156 void intel_ddi_disable_pipe_clock(const struct intel_crtc_state *crtc_state)
@@ -2132,9 +2158,14 @@ void intel_ddi_disable_pipe_clock(const struct intel_crtc_state *crtc_state) · 1 unmodified line
21322158 struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
21332159 enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
21342160
2135− if (cpu_transcoder != TRANSCODER_EDP)
2136− I915_WRITE(TRANS_CLK_SEL(cpu_transcoder),
2137− TRANS_CLK_SEL_DISABLED);
2161+ if (cpu_transcoder != TRANSCODER_EDP) {
2162+ if (INTEL_GEN(dev_priv) >= 12)
2163+ I915_WRITE(TRANS_CLK_SEL(cpu_transcoder),
2164+ TGL_TRANS_CLK_SEL_DISABLED);
2165+ else
2166+ I915_WRITE(TRANS_CLK_SEL(cpu_transcoder),
2167+ TRANS_CLK_SEL_DISABLED);
2168+ }
21382169 }
21392170
21402171 static void _skl_ddi_set_iboost(struct drm_i915_private *dev_priv,
@@ -2227,11 +2258,12 @@ u8 intel_ddi_dp_voltage_max(struct intel_encoder *encoder) · 86 unmodified lines
22272258 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
22282259 struct intel_dp *intel_dp = enc_to_intel_dp(&encoder->base);
22292260 enum port port = encoder->port;
2261+ enum phy phy = intel_port_to_phy(dev_priv, port);
22302262 int n_entries;
22312263
22322264 if (INTEL_GEN(dev_priv) >= 11) {
2233− if (intel_port_is_combophy(dev_priv, port))
2234− icl_get_combo_buf_trans(dev_priv, port, encoder->type,
2265+ if (intel_phy_is_combo(dev_priv, phy))
2266+ icl_get_combo_buf_trans(dev_priv, encoder->type,
22352267 intel_dp->link_rate, &n_entries);
22362268 else
22372269 n_entries = ARRAY_SIZE(icl_mg_phy_ddi_translations);
@@ -2413,15 +2445,15 @@ static void cnl_ddi_vswing_sequence(struct intel_encoder *encoder, · 175 unmodified lines
24132445 }
24142446
24152447 static void icl_ddi_combo_vswing_program(struct drm_i915_private *dev_priv,
2416− u32 level, enum port port, int type,
2448+ u32 level, enum phy phy, int type,
24172449 int rate)
24182450 {
24192451 const struct cnl_ddi_buf_trans *ddi_translations = NULL;
24202452 u32 n_entries, val;
24212453 int ln;
24222454
2423− ddi_translations = icl_get_combo_buf_trans(dev_priv, port, type,
2424− rate, &n_entries);
2455+ ddi_translations = icl_get_combo_buf_trans(dev_priv, type, rate,
2456+ &n_entries);
24252457 if (!ddi_translations)
24262458 return;
24272459
@@ -2431,41 +2463,41 @@ static void icl_ddi_combo_vswing_program(struct drm_i915_private *dev_priv, · 3 unmodified lines
24312463 }
24322464
24332465 /* Set PORT_TX_DW5 */
2434− val = I915_READ(ICL_PORT_TX_DW5_LN0(port));
2466+ val = I915_READ(ICL_PORT_TX_DW5_LN0(phy));
24352467 val &= ~(SCALING_MODE_SEL_MASK | RTERM_SELECT_MASK |
24362468 TAP2_DISABLE | TAP3_DISABLE);
24372469 val |= SCALING_MODE_SEL(0x2);
24382470 val |= RTERM_SELECT(0x6);
24392471 val |= TAP3_DISABLE;
2440− I915_WRITE(ICL_PORT_TX_DW5_GRP(port), val);
2472+ I915_WRITE(ICL_PORT_TX_DW5_GRP(phy), val);
24412473
24422474 /* Program PORT_TX_DW2 */
2443− val = I915_READ(ICL_PORT_TX_DW2_LN0(port));
2475+ val = I915_READ(ICL_PORT_TX_DW2_LN0(phy));
24442476 val &= ~(SWING_SEL_LOWER_MASK | SWING_SEL_UPPER_MASK |
24452477 RCOMP_SCALAR_MASK);
24462478 val |= SWING_SEL_UPPER(ddi_translations[level].dw2_swing_sel);
24472479 val |= SWING_SEL_LOWER(ddi_translations[level].dw2_swing_sel);
24482480 /* Program Rcomp scalar for every table entry */
24492481 val |= RCOMP_SCALAR(0x98);
2450− I915_WRITE(ICL_PORT_TX_DW2_GRP(port), val);
2482+ I915_WRITE(ICL_PORT_TX_DW2_GRP(phy), val);
24512483
24522484 /* Program PORT_TX_DW4 */
24532485 /* We cannot write to GRP. It would overwrite individual loadgen. */
24542486 for (ln = 0; ln <= 3; ln++) {
2455− val = I915_READ(ICL_PORT_TX_DW4_LN(ln, port));
2487+ val = I915_READ(ICL_PORT_TX_DW4_LN(ln, phy));
24562488 val &= ~(POST_CURSOR_1_MASK | POST_CURSOR_2_MASK |
24572489 CURSOR_COEFF_MASK);
24582490 val |= POST_CURSOR_1(ddi_translations[level].dw4_post_cursor_1);
24592491 val |= POST_CURSOR_2(ddi_translations[level].dw4_post_cursor_2);
24602492 val |= CURSOR_COEFF(ddi_translations[level].dw4_cursor_coeff);
2461− I915_WRITE(ICL_PORT_TX_DW4_LN(ln, port), val);
2493+ I915_WRITE(ICL_PORT_TX_DW4_LN(ln, phy), val);
24622494 }
24632495
24642496 /* Program PORT_TX_DW7 */
2465− val = I915_READ(ICL_PORT_TX_DW7_LN0(port));
2497+ val = I915_READ(ICL_PORT_TX_DW7_LN0(phy));
24662498 val &= ~N_SCALAR_MASK;
24672499 val |= N_SCALAR(ddi_translations[level].dw7_n_scalar);
2468− I915_WRITE(ICL_PORT_TX_DW7_GRP(port), val);
2500+ I915_WRITE(ICL_PORT_TX_DW7_GRP(phy), val);
24692501 }
24702502
24712503 static void icl_combo_phy_ddi_vswing_sequence(struct intel_encoder *encoder,
@@ -2473,7 +2505,7 @@ static void icl_combo_phy_ddi_vswing_sequence(struct intel_encoder *encoder, · 1 unmodified line
24732505 enum intel_output_type type)
24742506 {
24752507 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
2476− enum port port = encoder->port;
2508+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
24772509 int width = 0;
24782510 int rate = 0;
24792511 u32 val;
@@ -2494,12 +2526,12 @@ static void icl_combo_phy_ddi_vswing_sequence(struct intel_encoder *encoder, · 14 unmodified lines
24942526 * set PORT_PCS_DW1 cmnkeeper_enable to 1b,
24952527 * else clear to 0b.
24962528 */
2497− val = I915_READ(ICL_PORT_PCS_DW1_LN0(port));
2529+ val = I915_READ(ICL_PORT_PCS_DW1_LN0(phy));
24982530 if (type == INTEL_OUTPUT_HDMI)
24992531 val &= ~COMMON_KEEPER_EN;
25002532 else
25012533 val |= COMMON_KEEPER_EN;
2502− I915_WRITE(ICL_PORT_PCS_DW1_GRP(port), val);
2534+ I915_WRITE(ICL_PORT_PCS_DW1_GRP(phy), val);
25032535
25042536 /* 2. Program loadgen select */
25052537 /*
@@ -2509,33 +2541,33 @@ static void icl_combo_phy_ddi_vswing_sequence(struct intel_encoder *encoder, · 3 unmodified lines
25092541 * > 6 GHz (LN0=0, LN1=0, LN2=0, LN3=0)
25102542 */
25112543 for (ln = 0; ln <= 3; ln++) {
2512− val = I915_READ(ICL_PORT_TX_DW4_LN(ln, port));
2544+ val = I915_READ(ICL_PORT_TX_DW4_LN(ln, phy));
25132545 val &= ~LOADGEN_SELECT;
25142546
25152547 if ((rate <= 600000 && width == 4 && ln >= 1) ||
25162548 (rate <= 600000 && width < 4 && (ln == 1 || ln == 2))) {
25172549 val |= LOADGEN_SELECT;
25182550 }
2519− I915_WRITE(ICL_PORT_TX_DW4_LN(ln, port), val);
2551+ I915_WRITE(ICL_PORT_TX_DW4_LN(ln, phy), val);
25202552 }
25212553
25222554 /* 3. Set PORT_CL_DW5 SUS Clock Config to 11b */
2523− val = I915_READ(ICL_PORT_CL_DW5(port));
2555+ val = I915_READ(ICL_PORT_CL_DW5(phy));
25242556 val |= SUS_CLOCK_CONFIG;
2525− I915_WRITE(ICL_PORT_CL_DW5(port), val);
2557+ I915_WRITE(ICL_PORT_CL_DW5(phy), val);
25262558
25272559 /* 4. Clear training enable to change swing values */
2528− val = I915_READ(ICL_PORT_TX_DW5_LN0(port));
2560+ val = I915_READ(ICL_PORT_TX_DW5_LN0(phy));
25292561 val &= ~TX_TRAINING_EN;
2530− I915_WRITE(ICL_PORT_TX_DW5_GRP(port), val);
2562+ I915_WRITE(ICL_PORT_TX_DW5_GRP(phy), val);
25312563
25322564 /* 5. Program swing and de-emphasis */
2533− icl_ddi_combo_vswing_program(dev_priv, level, port, type, rate);
2565+ icl_ddi_combo_vswing_program(dev_priv, level, phy, type, rate);
25342566
25352567 /* 6. Set training enable to trigger update */
2536− val = I915_READ(ICL_PORT_TX_DW5_LN0(port));
2568+ val = I915_READ(ICL_PORT_TX_DW5_LN0(phy));
25372569 val |= TX_TRAINING_EN;
2538− I915_WRITE(ICL_PORT_TX_DW5_GRP(port), val);
2570+ I915_WRITE(ICL_PORT_TX_DW5_GRP(phy), val);
25392571 }
25402572
25412573 static void icl_mg_phy_ddi_vswing_sequence(struct intel_encoder *encoder,
@@ -2663,9 +2695,9 @@ static void icl_ddi_vswing_sequence(struct intel_encoder *encoder, · 121 unmodified lines
26632695 enum intel_output_type type)
26642696 {
26652697 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
2666− enum port port = encoder->port;
2698+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
26672699
2668− if (intel_port_is_combophy(dev_priv, port))
2700+ if (intel_phy_is_combo(dev_priv, phy))
26692701 icl_combo_phy_ddi_vswing_sequence(encoder, level, type);
26702702 else
26712703 icl_mg_phy_ddi_vswing_sequence(encoder, link_clock, level);
@@ -2728,12 +2760,13 @@ u32 ddi_signal_levels(struct intel_dp *intel_dp) · 56 unmodified lines
27282760
27292761 static inline
27302762 u32 icl_dpclka_cfgcr0_clk_off(struct drm_i915_private *dev_priv,
2731− enum port port)
2763+ enum phy phy)
27322764 {
2733− if (intel_port_is_combophy(dev_priv, port)) {
2734− return ICL_DPCLKA_CFGCR0_DDI_CLK_OFF(port);
2735− } else if (intel_port_is_tc(dev_priv, port)) {
2736− enum tc_port tc_port = intel_port_to_tc(dev_priv, port);
2765+ if (intel_phy_is_combo(dev_priv, phy)) {
2766+ return ICL_DPCLKA_CFGCR0_DDI_CLK_OFF(phy);
2767+ } else if (intel_phy_is_tc(dev_priv, phy)) {
2768+ enum tc_port tc_port = intel_port_to_tc(dev_priv,
2769+ (enum port)phy);
27372770
27382771 return ICL_DPCLKA_CFGCR0_TC_CLK_OFF(tc_port);
27392772 }
@@ -2746,23 +2779,33 @@ static void icl_map_plls_to_ports(struct intel_encoder *encoder, · 6 unmodified lines
27462779 {
27472780 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
27482781 struct intel_shared_dpll *pll = crtc_state->shared_dpll;
2749− enum port port = encoder->port;
2782+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
27502783 u32 val;
27512784
27522785 mutex_lock(&dev_priv->dpll_lock);
27532786
2754− val = I915_READ(DPCLKA_CFGCR0_ICL);
2755− WARN_ON((val & icl_dpclka_cfgcr0_clk_off(dev_priv, port)) == 0);
2787+ val = I915_READ(ICL_DPCLKA_CFGCR0);
2788+ WARN_ON((val & icl_dpclka_cfgcr0_clk_off(dev_priv, phy)) == 0);
27562789
2757− if (intel_port_is_combophy(dev_priv, port)) {
2758− val &= ~DPCLKA_CFGCR0_DDI_CLK_SEL_MASK(port);
2759− val |= DPCLKA_CFGCR0_DDI_CLK_SEL(pll->info->id, port);
2760− I915_WRITE(DPCLKA_CFGCR0_ICL, val);
2761− POSTING_READ(DPCLKA_CFGCR0_ICL);
2790+ if (intel_phy_is_combo(dev_priv, phy)) {
2791+ /*
2792+ * Even though this register references DDIs, note that we
2793+ * want to pass the PHY rather than the port (DDI). For
2794+ * ICL, port=phy in all cases so it doesn't matter, but for
2795+ * EHL the bspec notes the following:
2796+ *
2797+ * "DDID clock tied to DDIA clock, so DPCLKA_CFGCR0 DDIA
2798+ * Clock Select chooses the PLL for both DDIA and DDID and
2799+ * drives port A in all cases."
2800+ */
2801+ val &= ~ICL_DPCLKA_CFGCR0_DDI_CLK_SEL_MASK(phy);
2802+ val |= ICL_DPCLKA_CFGCR0_DDI_CLK_SEL(pll->info->id, phy);
2803+ I915_WRITE(ICL_DPCLKA_CFGCR0, val);
2804+ POSTING_READ(ICL_DPCLKA_CFGCR0);
27622805 }
27632806
2764− val &= ~icl_dpclka_cfgcr0_clk_off(dev_priv, port);
2765− I915_WRITE(DPCLKA_CFGCR0_ICL, val);
2807+ val &= ~icl_dpclka_cfgcr0_clk_off(dev_priv, phy);
2808+ I915_WRITE(ICL_DPCLKA_CFGCR0, val);
27662809
27672810 mutex_unlock(&dev_priv->dpll_lock);
27682811 }
@@ -2770,14 +2813,14 @@ static void icl_map_plls_to_ports(struct intel_encoder *encoder, · 1 unmodified line
27702813 static void icl_unmap_plls_to_ports(struct intel_encoder *encoder)
27712814 {
27722815 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
2773− enum port port = encoder->port;
2816+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
27742817 u32 val;
27752818
27762819 mutex_lock(&dev_priv->dpll_lock);
27772820
2778− val = I915_READ(DPCLKA_CFGCR0_ICL);
2779− val |= icl_dpclka_cfgcr0_clk_off(dev_priv, port);
2780− I915_WRITE(DPCLKA_CFGCR0_ICL, val);
2821+ val = I915_READ(ICL_DPCLKA_CFGCR0);
2822+ val |= icl_dpclka_cfgcr0_clk_off(dev_priv, phy);
2823+ I915_WRITE(ICL_DPCLKA_CFGCR0, val);
27812824
27822825 mutex_unlock(&dev_priv->dpll_lock);
27832826 }
@@ -2835,11 +2878,13 @@ void icl_sanitize_encoder_pll_mapping(struct intel_encoder *encoder) · 51 unmodified lines
28352878 ddi_clk_needed = false;
28362879 }
28372880
2838− val = I915_READ(DPCLKA_CFGCR0_ICL);
2881+ val = I915_READ(ICL_DPCLKA_CFGCR0);
28392882 for_each_port_masked(port, port_mask) {
2883+ enum phy phy = intel_port_to_phy(dev_priv, port);
2884+
28402885 bool ddi_clk_ungated = !(val &
28412886 icl_dpclka_cfgcr0_clk_off(dev_priv,
2842− port));
2887+ phy));
28432888
28442889 if (ddi_clk_needed == ddi_clk_ungated)
28452890 continue;
@@ -2851,10 +2896,10 @@ void icl_sanitize_encoder_pll_mapping(struct intel_encoder *encoder) · 5 unmodified lines
28512896 if (WARN_ON(ddi_clk_needed))
28522897 continue;
28532898
2854− DRM_NOTE("Port %c is disabled/in DSI mode with an ungated DDI clock, gate it\n",
2855− port_name(port));
2856− val |= icl_dpclka_cfgcr0_clk_off(dev_priv, port);
2857− I915_WRITE(DPCLKA_CFGCR0_ICL, val);
2899+ DRM_NOTE("PHY %c is disabled/in DSI mode with an ungated DDI clock, gate it\n",
2900+ phy_name(port));
2901+ val |= icl_dpclka_cfgcr0_clk_off(dev_priv, phy);
2902+ I915_WRITE(ICL_DPCLKA_CFGCR0, val);
28582903 }
28592904 }
28602905
@@ -2863,6 +2908,7 @@ static void intel_ddi_clk_select(struct intel_encoder *encoder, · 2 unmodified lines
28632908 {
28642909 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
28652910 enum port port = encoder->port;
2911+ enum phy phy = intel_port_to_phy(dev_priv, port);
28662912 u32 val;
28672913 const struct intel_shared_dpll *pll = crtc_state->shared_dpll;
28682914
@@ -2872,7 +2918,7 @@ static void intel_ddi_clk_select(struct intel_encoder *encoder, · 3 unmodified lines
28722918 mutex_lock(&dev_priv->dpll_lock);
28732919
28742920 if (INTEL_GEN(dev_priv) >= 11) {
2875− if (!intel_port_is_combophy(dev_priv, port))
2921+ if (!intel_phy_is_combo(dev_priv, phy))
28762922 I915_WRITE(DDI_CLK_SEL(port),
28772923 icl_pll_to_ddi_clk_sel(encoder, crtc_state));
28782924 } else if (IS_CANNONLAKE(dev_priv)) {
@@ -2912,9 +2958,10 @@ static void intel_ddi_clk_disable(struct intel_encoder *encoder) · 33 unmodified lines
29122958 {
29132959 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
29142960 enum port port = encoder->port;
2961+ enum phy phy = intel_port_to_phy(dev_priv, port);
29152962
29162963 if (INTEL_GEN(dev_priv) >= 11) {
2917− if (!intel_port_is_combophy(dev_priv, port))
2964+ if (!intel_phy_is_combo(dev_priv, phy))
29182965 I915_WRITE(DDI_CLK_SEL(port), DDI_CLK_SEL_NONE);
29192966 } else if (IS_CANNONLAKE(dev_priv)) {
29202967 I915_WRITE(DPCLKA_CFGCR0, I915_READ(DPCLKA_CFGCR0) |
@@ -2995,25 +3042,22 @@ static void icl_program_mg_dp_mode(struct intel_digital_port *intel_dig_port) · 74 unmodified lines
29953042 {
29963043 struct drm_i915_private *dev_priv = to_i915(intel_dig_port->base.base.dev);
29973044 enum port port = intel_dig_port->base.port;
2998− enum tc_port tc_port = intel_port_to_tc(dev_priv, port);
2999− u32 ln0, ln1, lane_info;
3045+ u32 ln0, ln1, lane_mask;
30003046
3001− if (tc_port == PORT_TC_NONE || intel_dig_port->tc_type == TC_PORT_TBT)
3047+ if (intel_dig_port->tc_mode == TC_PORT_TBT_ALT)
30023048 return;
30033049
30043050 ln0 = I915_READ(MG_DP_MODE(0, port));
30053051 ln1 = I915_READ(MG_DP_MODE(1, port));
30063052
3007− switch (intel_dig_port->tc_type) {
3008− case TC_PORT_TYPEC:
3053+ switch (intel_dig_port->tc_mode) {
3054+ case TC_PORT_DP_ALT:
30093055 ln0 &= ~(MG_DP_MODE_CFG_DP_X1_MODE | MG_DP_MODE_CFG_DP_X2_MODE);
30103056 ln1 &= ~(MG_DP_MODE_CFG_DP_X1_MODE | MG_DP_MODE_CFG_DP_X2_MODE);
30113057
3012− lane_info = (I915_READ(PORT_TX_DFLEXDPSP) &
3013− DP_LANE_ASSIGNMENT_MASK(tc_port)) >>
3014− DP_LANE_ASSIGNMENT_SHIFT(tc_port);
3058+ lane_mask = intel_tc_port_get_lane_mask(intel_dig_port);
30153059
3016− switch (lane_info) {
3060+ switch (lane_mask) {
30173061 case 0x1:
30183062 case 0x4:
30193063 break;
@@ -3038,7 +3082,7 @@ static void icl_program_mg_dp_mode(struct intel_digital_port *intel_dig_port) · 18 unmodified lines
30383082 MG_DP_MODE_CFG_DP_X2_MODE;
30393083 break;
30403084 default:
3041− MISSING_CASE(lane_info);
3085+ MISSING_CASE(lane_mask);
30423086 }
30433087 break;
30443088
@@ -3048,7 +3092,7 @@ static void icl_program_mg_dp_mode(struct intel_digital_port *intel_dig_port) · 3 unmodified lines
30483092 break;
30493093
30503094 default:
3051− MISSING_CASE(intel_dig_port->tc_type);
3095+ MISSING_CASE(intel_dig_port->tc_mode);
30523096 return;
30533097 }
30543098
@@ -3110,6 +3154,7 @@ static void intel_ddi_pre_enable_dp(struct intel_encoder *encoder, · 55 unmodified lines
31103154 struct intel_dp *intel_dp = enc_to_intel_dp(&encoder->base);
31113155 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
31123156 enum port port = encoder->port;
3157+ enum phy phy = intel_port_to_phy(dev_priv, port);
31133158 struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
31143159 bool is_mst = intel_crtc_has_type(crtc_state, INTEL_OUTPUT_DP_MST);
31153160 int level = intel_ddi_dp_level(intel_dp);
@@ -3123,7 +3168,10 @@ static void intel_ddi_pre_enable_dp(struct intel_encoder *encoder, · 7 unmodified lines
31233168
31243169 intel_ddi_clk_select(encoder, crtc_state);
31253170
3126− intel_display_power_get(dev_priv, dig_port->ddi_io_power_domain);
3171+ if (!intel_phy_is_tc(dev_priv, phy) ||
3172+ dig_port->tc_mode != TC_PORT_TBT_ALT)
3173+ intel_display_power_get(dev_priv,
3174+ dig_port->ddi_io_power_domain);
31273175
31283176 icl_program_mg_dp_mode(dig_port);
31293177 icl_disable_phy_clock_gating(dig_port);
@@ -3138,11 +3186,11 @@ static void intel_ddi_pre_enable_dp(struct intel_encoder *encoder, · 8 unmodified lines
31383186 else
31393187 intel_prepare_dp_ddi_buffers(encoder, crtc_state);
31403188
3141− if (intel_port_is_combophy(dev_priv, port)) {
3189+ if (intel_phy_is_combo(dev_priv, phy)) {
31423190 bool lane_reversal =
31433191 dig_port->saved_port_bits & DDI_BUF_PORT_REVERSAL;
31443192
3145− intel_combo_phy_power_up_lanes(dev_priv, port, false,
3193+ intel_combo_phy_power_up_lanes(dev_priv, phy, false,
31463194 crtc_state->lane_count,
31473195 lane_reversal);
31483196 }
@@ -3290,6 +3338,7 @@ static void intel_ddi_post_disable_dp(struct intel_encoder *encoder, · 141 unmodified lines
32903338 struct intel_dp *intel_dp = &dig_port->dp;
32913339 bool is_mst = intel_crtc_has_type(old_crtc_state,
32923340 INTEL_OUTPUT_DP_MST);
3341+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
32933342
32943343 if (!is_mst) {
32953344 intel_ddi_disable_pipe_clock(old_crtc_state);
@@ -3305,8 +3354,10 @@ static void intel_ddi_post_disable_dp(struct intel_encoder *encoder, · 9 unmodified lines
33053354 intel_edp_panel_vdd_on(intel_dp);
33063355 intel_edp_panel_off(intel_dp);
33073356
3308− intel_display_power_put_unchecked(dev_priv,
3309− dig_port->ddi_io_power_domain);
3357+ if (!intel_phy_is_tc(dev_priv, phy) ||
3358+ dig_port->tc_mode != TC_PORT_TBT_ALT)
3359+ intel_display_power_put_unchecked(dev_priv,
3360+ dig_port->ddi_io_power_domain);
33103361
33113362 intel_ddi_clk_disable(encoder);
33123363 }
@@ -3591,33 +3642,28 @@ static void intel_ddi_update_pipe(struct intel_encoder *encoder, · 278 unmodified lines
35913642 intel_hdcp_disable(to_intel_connector(conn_state->connector));
35923643 }
35933644
3594−static void intel_ddi_set_fia_lane_count(struct intel_encoder *encoder,
3595− const struct intel_crtc_state *pipe_config,
3596− enum port port)
3645+static void
3646+intel_ddi_update_prepare(struct intel_atomic_state *state,
3647+ struct intel_encoder *encoder,
3648+ struct intel_crtc *crtc)
35973649 {
3598− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
3599− struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
3600− enum tc_port tc_port = intel_port_to_tc(dev_priv, port);
3601− u32 val = I915_READ(PORT_TX_DFLEXDPMLE1);
3602− bool lane_reversal = dig_port->saved_port_bits & DDI_BUF_PORT_REVERSAL;
3603−
3604− val &= ~DFLEXDPMLE1_DPMLETC_MASK(tc_port);
3605− switch (pipe_config->lane_count) {
3606− case 1:
3607− val |= (lane_reversal) ? DFLEXDPMLE1_DPMLETC_ML3(tc_port) :
3608− DFLEXDPMLE1_DPMLETC_ML0(tc_port);
3609− break;
3610− case 2:
3611− val |= (lane_reversal) ? DFLEXDPMLE1_DPMLETC_ML3_2(tc_port) :
3612− DFLEXDPMLE1_DPMLETC_ML1_0(tc_port);
3613− break;
3614− case 4:
3615− val |= DFLEXDPMLE1_DPMLETC_ML3_0(tc_port);
3616− break;
3617− default:
3618− MISSING_CASE(pipe_config->lane_count);
3619− }
3620− I915_WRITE(PORT_TX_DFLEXDPMLE1, val);
3650+ struct intel_crtc_state *crtc_state =
3651+ crtc ? intel_atomic_get_new_crtc_state(state, crtc) : NULL;
3652+ int required_lanes = crtc_state ? crtc_state->lane_count : 1;
3653+
3654+ WARN_ON(crtc && crtc->active);
3655+
3656+ intel_tc_port_get_link(enc_to_dig_port(&encoder->base), required_lanes);
3657+ if (crtc_state && crtc_state->base.active)
3658+ intel_update_active_dpll(state, crtc, encoder);
3659+}
3660+
3661+static void
3662+intel_ddi_update_complete(struct intel_atomic_state *state,
3663+ struct intel_encoder *encoder,
3664+ struct intel_crtc *crtc)
3665+{
3666+ intel_tc_port_put_link(enc_to_dig_port(&encoder->base));
36213667 }
36223668
36233669 static void
@@ -3627,26 +3673,25 @@ intel_ddi_pre_pll_enable(struct intel_encoder *encoder, · 3 unmodified lines
36273673 {
36283674 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
36293675 struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
3630− enum port port = encoder->port;
3676+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
3677+ bool is_tc_port = intel_phy_is_tc(dev_priv, phy);
36313678
3632− if (intel_crtc_has_dp_encoder(crtc_state) ||
3633− intel_port_is_tc(dev_priv, encoder->port))
3679+ if (is_tc_port)
3680+ intel_tc_port_get_link(dig_port, crtc_state->lane_count);
3681+
3682+ if (intel_crtc_has_dp_encoder(crtc_state) || is_tc_port)
36343683 intel_display_power_get(dev_priv,
36353684 intel_ddi_main_link_aux_domain(dig_port));
36363685
3637− if (IS_GEN9_LP(dev_priv))
3686+ if (is_tc_port && dig_port->tc_mode != TC_PORT_TBT_ALT)
3687+ /*
3688+ * Program the lane count for static/dynamic connections on
3689+ * Type-C ports. Skip this step for TBT.
3690+ */
3691+ intel_tc_port_set_fia_lane_count(dig_port, crtc_state->lane_count);
3692+ else if (IS_GEN9_LP(dev_priv))
36383693 bxt_ddi_phy_set_lane_optim_mask(encoder,
36393694 crtc_state->lane_lat_optim_mask);
3640−
3641− /*
3642− * Program the lane count for static/dynamic connections on Type-C ports.
3643− * Skip this step for TBT.
3644− */
3645− if (dig_port->tc_type == TC_PORT_UNKNOWN ||
3646− dig_port->tc_type == TC_PORT_TBT)
3647− return;
3648−
3649− intel_ddi_set_fia_lane_count(encoder, crtc_state, port);
36503695 }
36513696
36523697 static void
@@ -3656,11 +3701,15 @@ intel_ddi_post_pll_disable(struct intel_encoder *encoder, · 3 unmodified lines
36563701 {
36573702 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
36583703 struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
3704+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
3705+ bool is_tc_port = intel_phy_is_tc(dev_priv, phy);
36593706
3660− if (intel_crtc_has_dp_encoder(crtc_state) ||
3661− intel_port_is_tc(dev_priv, encoder->port))
3707+ if (intel_crtc_has_dp_encoder(crtc_state) || is_tc_port)
36623708 intel_display_power_put_unchecked(dev_priv,
36633709 intel_ddi_main_link_aux_domain(dig_port));
3710+
3711+ if (is_tc_port)
3712+ intel_tc_port_put_link(dig_port);
36643713 }
36653714
36663715 static void intel_ddi_prepare_link_retrain(struct intel_dp *intel_dp)
@@ -3737,7 +3786,6 @@ void intel_ddi_get_config(struct intel_encoder *encoder, · 70 unmodified lines
37373786 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
37383787 struct intel_crtc *intel_crtc = to_intel_crtc(pipe_config->base.crtc);
37393788 enum transcoder cpu_transcoder = pipe_config->cpu_transcoder;
3740− struct intel_digital_port *intel_dig_port;
37413789 u32 temp, flags = 0;
37423790
37433791 /* XXX: DSI transcoder paranoia */
@@ -3776,7 +3824,6 @@ void intel_ddi_get_config(struct intel_encoder *encoder, · 32 unmodified lines
37763824 switch (temp & TRANS_DDI_MODE_SELECT_MASK) {
37773825 case TRANS_DDI_MODE_SELECT_HDMI:
37783826 pipe_config->has_hdmi_sink = true;
3779− intel_dig_port = enc_to_dig_port(&encoder->base);
37803827
37813828 pipe_config->infoframes.enable |=
37823829 intel_hdmi_infoframes_enabled(encoder, pipe_config);
@@ -3914,49 +3961,18 @@ static int intel_ddi_compute_config(struct intel_encoder *encoder, · 131 unmodified lines
39143961 return 0;
39153962 }
39163963
3917−static void intel_ddi_encoder_suspend(struct intel_encoder *encoder)
3918−{
3919− struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
3920− struct drm_i915_private *i915 = to_i915(encoder->base.dev);
3921−
3922− intel_dp_encoder_suspend(encoder);
3923−
3924− /*
3925− * TODO: disconnect also from USB DP alternate mode once we have a
3926− * way to handle the modeset restore in that mode during resume
3927− * even if the sink has disappeared while being suspended.
3928− */
3929− if (dig_port->tc_legacy_port)
3930− icl_tc_phy_disconnect(i915, dig_port);
3931−}
3932−
3933−static void intel_ddi_encoder_reset(struct drm_encoder *drm_encoder)
3934−{
3935− struct intel_digital_port *dig_port = enc_to_dig_port(drm_encoder);
3936− struct drm_i915_private *i915 = to_i915(drm_encoder->dev);
3937−
3938− if (intel_port_is_tc(i915, dig_port->base.port))
3939− intel_digital_port_connected(&dig_port->base);
3940−
3941− intel_dp_encoder_reset(drm_encoder);
3942−}
3943−
39443964 static void intel_ddi_encoder_destroy(struct drm_encoder *encoder)
39453965 {
39463966 struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
3947− struct drm_i915_private *i915 = to_i915(encoder->dev);
39483967
39493968 intel_dp_encoder_flush_work(encoder);
39503969
3951− if (intel_port_is_tc(i915, dig_port->base.port))
3952− icl_tc_phy_disconnect(i915, dig_port);
3953−
39543970 drm_encoder_cleanup(encoder);
39553971 kfree(dig_port);
39563972 }
39573973
39583974 static const struct drm_encoder_funcs intel_ddi_funcs = {
3959− .reset = intel_ddi_encoder_reset,
3975+ .reset = intel_dp_encoder_reset,
39603976 .destroy = intel_ddi_encoder_destroy,
39613977 };
39623978
@@ -4081,14 +4097,17 @@ static int intel_hdmi_reset_link(struct intel_encoder *encoder, · 118 unmodified lines
40814097 return modeset_pipe(&crtc->base, ctx);
40824098 }
40834099
4084−static bool intel_ddi_hotplug(struct intel_encoder *encoder,
4085− struct intel_connector *connector)
4100+static enum intel_hotplug_state
4101+intel_ddi_hotplug(struct intel_encoder *encoder,
4102+ struct intel_connector *connector,
4103+ bool irq_received)
40864104 {
4105+ struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
40874106 struct drm_modeset_acquire_ctx ctx;
4088− bool changed;
4107+ enum intel_hotplug_state state;
40894108 int ret;
40904109
4091− changed = intel_encoder_hotplug(encoder, connector);
4110+ state = intel_encoder_hotplug(encoder, connector, irq_received);
40924111
40934112 drm_modeset_acquire_init(&ctx, 0);
40944113
@@ -4110,7 +4129,27 @@ static bool intel_ddi_hotplug(struct intel_encoder *encoder, · 15 unmodified lines
41104129 drm_modeset_acquire_fini(&ctx);
41114130 WARN(ret, "Acquiring modeset locks failed with %i\n", ret);
41124131
4113− return changed;
4132+ /*
4133+ * Unpowered type-c dongles can take some time to boot and be
4134+ * responsible, so here giving some time to those dongles to power up
4135+ * and then retrying the probe.
4136+ *
4137+ * On many platforms the HDMI live state signal is known to be
4138+ * unreliable, so we can't use it to detect if a sink is connected or
4139+ * not. Instead we detect if it's connected based on whether we can
4140+ * read the EDID or not. That in turn has a problem during disconnect,
4141+ * since the HPD interrupt may be raised before the DDC lines get
4142+ * disconnected (due to how the required length of DDC vs. HPD
4143+ * connector pins are specified) and so we'll still be able to get a
4144+ * valid EDID. To solve this schedule another detection cycle if this
4145+ * time around we didn't detect any change in the sink's connection
4146+ * status.
4147+ */
4148+ if (state == INTEL_HOTPLUG_UNCHANGED && irq_received &&
4149+ !dig_port->dp.is_mst)
4150+ state = INTEL_HOTPLUG_RETRY;
4151+
4152+ return state;
41144153 }
41154154
41164155 static struct intel_connector *
@@ -4198,6 +4237,7 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port) · 81 unmodified lines
41984237 struct drm_encoder *encoder;
41994238 bool init_hdmi, init_dp, init_lspcon = false;
42004239 enum pipe pipe;
4240+ enum phy phy = intel_port_to_phy(dev_priv, port);
42014241
42024242 init_hdmi = port_info->supports_dvi || port_info->supports_hdmi;
42034243 init_dp = port_info->supports_dp;
@@ -4242,7 +4282,7 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port) · 38 unmodified lines
42424282 intel_encoder->update_pipe = intel_ddi_update_pipe;
42434283 intel_encoder->get_hw_state = intel_ddi_get_hw_state;
42444284 intel_encoder->get_config = intel_ddi_get_config;
4245− intel_encoder->suspend = intel_ddi_encoder_suspend;
4285+ intel_encoder->suspend = intel_dp_encoder_suspend;
42464286 intel_encoder->get_power_domains = intel_ddi_get_power_domains;
42474287 intel_encoder->type = INTEL_OUTPUT_DDI;
42484288 intel_encoder->power_domain = intel_port_to_power_domain(port);
@@ -4261,9 +4301,15 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port) · 12 unmodified lines
42614301 intel_dig_port->max_lanes = intel_ddi_max_lanes(intel_dig_port);
42624302 intel_dig_port->aux_ch = intel_bios_port_aux_ch(dev_priv, port);
42634303
4264− intel_dig_port->tc_legacy_port = intel_port_is_tc(dev_priv, port) &&
4265− !port_info->supports_typec_usb &&
4266− !port_info->supports_tbt;
4304+ if (intel_phy_is_tc(dev_priv, phy)) {
4305+ bool is_legacy = !port_info->supports_typec_usb &&
4306+ !port_info->supports_tbt;
4307+
4308+ intel_tc_port_init(intel_dig_port, is_legacy);
4309+
4310+ intel_encoder->update_prepare = intel_ddi_update_prepare;
4311+ intel_encoder->update_complete = intel_ddi_update_complete;
4312+ }
42674313
42684314 switch (port) {
42694315 case PORT_A:
@@ -4290,6 +4336,18 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port) · 20 unmodified lines
42904336 intel_dig_port->ddi_io_power_domain =
42914337 POWER_DOMAIN_PORT_DDI_F_IO;
42924338 break;
4339+ case PORT_G:
4340+ intel_dig_port->ddi_io_power_domain =
4341+ POWER_DOMAIN_PORT_DDI_G_IO;
4342+ break;
4343+ case PORT_H:
4344+ intel_dig_port->ddi_io_power_domain =
4345+ POWER_DOMAIN_PORT_DDI_H_IO;
4346+ break;
4347+ case PORT_I:
4348+ intel_dig_port->ddi_io_power_domain =
4349+ POWER_DOMAIN_PORT_DDI_I_IO;
4350+ break;
42934351 default:
42944352 MISSING_CASE(port);
42954353 }
@@ -4324,9 +4382,6 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port) · 28 unmodified lines
43244382
43254383 intel_infoframe_init(intel_dig_port);
43264384
4327− if (intel_port_is_tc(dev_priv, port))
4328− intel_digital_port_connected(intel_encoder);
4329−
43304385 return;
43314386
43324387 err:
drivers/gpu/drm/i915/display/intel_display.c+734−511
@@ -78,6 +78,7 @@ · 77 unmodified lines
7878 #include "intel_quirks.h"
7979 #include "intel_sideband.h"
8080 #include "intel_sprite.h"
81+#include "intel_tc.h"
8182
8283 /* Primary plane formats for gen <= 3 */
8384 static const u32 i8xx_primary_formats[] = {
@@ -515,9 +516,9 @@ icl_wa_scalerclkgating(struct drm_i915_private *dev_priv, enum pipe pipe, · 431 unmodified lines
515516 }
516517
517518 static bool
518−needs_modeset(const struct drm_crtc_state *state)
519+needs_modeset(const struct intel_crtc_state *state)
519520 {
520− return drm_atomic_crtc_needs_modeset(state);
521+ return drm_atomic_crtc_needs_modeset(&state->base);
521522 }
522523
523524 /*
@@ -3715,10 +3716,27 @@ int i9xx_check_plane_surface(struct intel_plane_state *plane_state) · 3191 unmodified lines
37153716 return 0;
37163717 }
37173718
3719+static bool i9xx_plane_has_windowing(struct intel_plane *plane)
3720+{
3721+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
3722+ enum i9xx_plane_id i9xx_plane = plane->i9xx_plane;
3723+
3724+ if (IS_CHERRYVIEW(dev_priv))
3725+ return i9xx_plane == PLANE_B;
3726+ else if (INTEL_GEN(dev_priv) >= 5 || IS_G4X(dev_priv))
3727+ return false;
3728+ else if (IS_GEN(dev_priv, 4))
3729+ return i9xx_plane == PLANE_C;
3730+ else
3731+ return i9xx_plane == PLANE_B ||
3732+ i9xx_plane == PLANE_C;
3733+}
3734+
37183735 static int
37193736 i9xx_plane_check(struct intel_crtc_state *crtc_state,
37203737 struct intel_plane_state *plane_state)
37213738 {
3739+ struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
37223740 int ret;
37233741
37243742 ret = chv_plane_check_rotation(plane_state);
@@ -3729,7 +3747,8 @@ i9xx_plane_check(struct intel_crtc_state *crtc_state, · 4 unmodified lines
37293747 &crtc_state->base,
37303748 DRM_PLANE_HELPER_NO_SCALING,
37313749 DRM_PLANE_HELPER_NO_SCALING,
3732− false, true);
3750+ i9xx_plane_has_windowing(plane),
3751+ true);
37333752 if (ret)
37343753 return ret;
37353754
@@ -3758,6 +3777,10 @@ static void i9xx_update_plane(struct intel_plane *plane, · 22 unmodified lines
37583777 u32 linear_offset;
37593778 int x = plane_state->color_plane[0].x;
37603779 int y = plane_state->color_plane[0].y;
3780+ int crtc_x = plane_state->base.dst.x1;
3781+ int crtc_y = plane_state->base.dst.y1;
3782+ int crtc_w = drm_rect_width(&plane_state->base.dst);
3783+ int crtc_h = drm_rect_height(&plane_state->base.dst);
37613784 unsigned long irqflags;
37623785 u32 dspaddr_offset;
37633786 u32 dspcntr;
@@ -3776,18 +3799,18 @@ static void i9xx_update_plane(struct intel_plane *plane, · 12 unmodified lines
37763799 I915_WRITE_FW(DSPSTRIDE(i9xx_plane), plane_state->color_plane[0].stride);
37773800
37783801 if (INTEL_GEN(dev_priv) < 4) {
3779− /* pipesrc and dspsize control the size that is scaled from,
3780− * which should always be the user's requested size.
3802+ /*
3803+ * PLANE_A doesn't actually have a full window
3804+ * generator but let's assume we still need to
3805+ * program whatever is there.
37813806 */
3782− I915_WRITE_FW(DSPPOS(i9xx_plane), 0);
3807+ I915_WRITE_FW(DSPPOS(i9xx_plane), (crtc_y << 16) | crtc_x);
37833808 I915_WRITE_FW(DSPSIZE(i9xx_plane),
3784− ((crtc_state->pipe_src_h - 1) << 16) |
3785− (crtc_state->pipe_src_w - 1));
3809+ ((crtc_h - 1) << 16) | (crtc_w - 1));
37863810 } else if (IS_CHERRYVIEW(dev_priv) && i9xx_plane == PLANE_B) {
3787− I915_WRITE_FW(PRIMPOS(i9xx_plane), 0);
3811+ I915_WRITE_FW(PRIMPOS(i9xx_plane), (crtc_y << 16) | crtc_x);
37883812 I915_WRITE_FW(PRIMSIZE(i9xx_plane),
3789− ((crtc_state->pipe_src_h - 1) << 16) |
3790− (crtc_state->pipe_src_w - 1));
3813+ ((crtc_h - 1) << 16) | (crtc_w - 1));
37913814 I915_WRITE_FW(PRIMCNSTALPHA(i9xx_plane), 0);
37923815 }
37933816
@@ -3950,10 +3973,10 @@ static u32 skl_plane_ctl_format(u32 pixel_format) · 156 unmodified lines
39503973 case DRM_FORMAT_XRGB8888:
39513974 case DRM_FORMAT_ARGB8888:
39523975 return PLANE_CTL_FORMAT_XRGB_8888;
3976+ case DRM_FORMAT_XBGR2101010:
3977+ return PLANE_CTL_FORMAT_XRGB_2101010 | PLANE_CTL_ORDER_RGBX;
39533978 case DRM_FORMAT_XRGB2101010:
39543979 return PLANE_CTL_FORMAT_XRGB_2101010;
3955− case DRM_FORMAT_XBGR2101010:
3956− return PLANE_CTL_ORDER_RGBX | PLANE_CTL_FORMAT_XRGB_2101010;
39573980 case DRM_FORMAT_XBGR16161616F:
39583981 case DRM_FORMAT_ABGR16161616F:
39593982 return PLANE_CTL_FORMAT_XRGB_16161616F | PLANE_CTL_ORDER_RGBX;
@@ -4248,12 +4271,13 @@ void intel_prepare_reset(struct drm_i915_private *dev_priv) · 288 unmodified lines
42484271 return;
42494272
42504273 /* We have a modeset vs reset deadlock, defensively unbreak it. */
4251− set_bit(I915_RESET_MODESET, &dev_priv->gpu_error.flags);
4252− wake_up_all(&dev_priv->gpu_error.wait_queue);
4274+ set_bit(I915_RESET_MODESET, &dev_priv->gt.reset.flags);
4275+ smp_mb__after_atomic();
4276+ wake_up_bit(&dev_priv->gt.reset.flags, I915_RESET_MODESET);
42534277
42544278 if (atomic_read(&dev_priv->gpu_error.pending_fb_pin)) {
42554279 DRM_DEBUG_KMS("Modeset potentially stuck, unbreaking through wedging\n");
4256− i915_gem_set_wedged(dev_priv);
4280+ intel_gt_set_wedged(&dev_priv->gt);
42574281 }
42584282
42594283 /*
@@ -4299,7 +4323,7 @@ void intel_finish_reset(struct drm_i915_private *dev_priv) · 39 unmodified lines
42994323 int ret;
43004324
43014325 /* reset doesn't touch the display */
4302− if (!test_bit(I915_RESET_MODESET, &dev_priv->gpu_error.flags))
4326+ if (!test_bit(I915_RESET_MODESET, &dev_priv->gt.reset.flags))
43034327 return;
43044328
43054329 state = fetch_and_zero(&dev_priv->modeset_restore_state);
@@ -4339,7 +4363,7 @@ unlock: · 33 unmodified lines
43394363 drm_modeset_acquire_fini(ctx);
43404364 mutex_unlock(&dev->mode_config.mutex);
43414365
4342− clear_bit(I915_RESET_MODESET, &dev_priv->gpu_error.flags);
4366+ clear_bit_unlock(I915_RESET_MODESET, &dev_priv->gt.reset.flags);
43434367 }
43444368
43454369 static void icl_set_pipe_chicken(struct intel_crtc *crtc)
@@ -5796,7 +5820,7 @@ static bool hsw_pre_update_disable_ips(const struct intel_crtc_state *old_crtc_s · 1450 unmodified lines
57965820 if (!old_crtc_state->ips_enabled)
57975821 return false;
57985822
5799− if (needs_modeset(&new_crtc_state->base))
5823+ if (needs_modeset(new_crtc_state))
58005824 return true;
58015825
58025826 /*
@@ -5823,7 +5847,7 @@ static bool hsw_post_update_enable_ips(const struct intel_crtc_state *old_crtc_s · 20 unmodified lines
58235847 if (!new_crtc_state->ips_enabled)
58245848 return false;
58255849
5826− if (needs_modeset(&new_crtc_state->base))
5850+ if (needs_modeset(new_crtc_state))
58275851 return true;
58285852
58295853 /*
@@ -5877,13 +5901,13 @@ static void intel_post_plane_update(struct intel_crtc_state *old_crtc_state) · 47 unmodified lines
58775901 struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->base.crtc);
58785902 struct drm_device *dev = crtc->base.dev;
58795903 struct drm_i915_private *dev_priv = to_i915(dev);
5880− struct drm_atomic_state *old_state = old_crtc_state->base.state;
5904+ struct drm_atomic_state *state = old_crtc_state->base.state;
58815905 struct intel_crtc_state *pipe_config =
5882− intel_atomic_get_new_crtc_state(to_intel_atomic_state(old_state),
5906+ intel_atomic_get_new_crtc_state(to_intel_atomic_state(state),
58835907 crtc);
58845908 struct drm_plane *primary = crtc->base.primary;
58855909 struct drm_plane_state *old_primary_state =
5886− drm_atomic_get_old_plane_state(old_state, primary);
5910+ drm_atomic_get_old_plane_state(state, primary);
58875911
58885912 intel_frontbuffer_flip(to_i915(crtc->base.dev), pipe_config->fb_bits);
58895913
@@ -5895,12 +5919,12 @@ static void intel_post_plane_update(struct intel_crtc_state *old_crtc_state) · 5 unmodified lines
58955919
58965920 if (old_primary_state) {
58975921 struct drm_plane_state *new_primary_state =
5898− drm_atomic_get_new_plane_state(old_state, primary);
5922+ drm_atomic_get_new_plane_state(state, primary);
58995923
59005924 intel_fbc_post_update(crtc);
59015925
59025926 if (new_primary_state->visible &&
5903− (needs_modeset(&pipe_config->base) ||
5927+ (needs_modeset(pipe_config) ||
59045928 !old_primary_state->visible))
59055929 intel_post_enable_primary(&crtc->base, pipe_config);
59065930 }
@@ -5920,20 +5944,20 @@ static void intel_pre_plane_update(struct intel_crtc_state *old_crtc_state, · 13 unmodified lines
59205944 struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->base.crtc);
59215945 struct drm_device *dev = crtc->base.dev;
59225946 struct drm_i915_private *dev_priv = to_i915(dev);
5923− struct drm_atomic_state *old_state = old_crtc_state->base.state;
5947+ struct drm_atomic_state *state = old_crtc_state->base.state;
59245948 struct drm_plane *primary = crtc->base.primary;
59255949 struct drm_plane_state *old_primary_state =
5926− drm_atomic_get_old_plane_state(old_state, primary);
5927− bool modeset = needs_modeset(&pipe_config->base);
5928− struct intel_atomic_state *old_intel_state =
5929− to_intel_atomic_state(old_state);
5950+ drm_atomic_get_old_plane_state(state, primary);
5951+ bool modeset = needs_modeset(pipe_config);
5952+ struct intel_atomic_state *intel_state =
5953+ to_intel_atomic_state(state);
59305954
59315955 if (hsw_pre_update_disable_ips(old_crtc_state, pipe_config))
59325956 hsw_disable_ips(old_crtc_state);
59335957
59345958 if (old_primary_state) {
59355959 struct intel_plane_state *new_primary_state =
5936− intel_atomic_get_new_plane_state(old_intel_state,
5960+ intel_atomic_get_new_plane_state(intel_state,
59375961 to_intel_plane(primary));
59385962
59395963 intel_fbc_pre_update(crtc, pipe_config, new_primary_state);
@@ -5984,7 +6008,7 @@ static void intel_pre_plane_update(struct intel_crtc_state *old_crtc_state, · 44 unmodified lines
59846008 * If we're doing a modeset, we're done. No need to do any pre-vblank
59856009 * watermark programming here.
59866010 */
5987− if (needs_modeset(&pipe_config->base))
6011+ if (needs_modeset(pipe_config))
59886012 return;
59896013
59906014 /*
@@ -6002,7 +6026,7 @@ static void intel_pre_plane_update(struct intel_crtc_state *old_crtc_state, · 11 unmodified lines
60026026 * us to.
60036027 */
60046028 if (dev_priv->display.initial_watermarks != NULL)
6005− dev_priv->display.initial_watermarks(old_intel_state,
6029+ dev_priv->display.initial_watermarks(intel_state,
60066030 pipe_config);
60076031 else if (pipe_config->update_wm_pre)
60086032 intel_update_watermarks(crtc);
@@ -6036,19 +6060,111 @@ static void intel_crtc_disable_planes(struct intel_atomic_state *state, · 27 unmodified lines
60366060 intel_frontbuffer_flip(dev_priv, fb_bits);
60376061 }
60386062
6039−static void intel_encoders_pre_pll_enable(struct drm_crtc *crtc,
6063+/*
6064+ * intel_connector_primary_encoder - get the primary encoder for a connector
6065+ * @connector: connector for which to return the encoder
6066+ *
6067+ * Returns the primary encoder for a connector. There is a 1:1 mapping from
6068+ * all connectors to their encoder, except for DP-MST connectors which have
6069+ * both a virtual and a primary encoder. These DP-MST primary encoders can be
6070+ * pointed to by as many DP-MST connectors as there are pipes.
6071+ */
6072+static struct intel_encoder *
6073+intel_connector_primary_encoder(struct intel_connector *connector)
6074+{
6075+ struct intel_encoder *encoder;
6076+
6077+ if (connector->mst_port)
6078+ return &dp_to_dig_port(connector->mst_port)->base;
6079+
6080+ encoder = intel_attached_encoder(&connector->base);
6081+ WARN_ON(!encoder);
6082+
6083+ return encoder;
6084+}
6085+
6086+static bool
6087+intel_connector_needs_modeset(struct intel_atomic_state *state,
6088+ const struct drm_connector_state *old_conn_state,
6089+ const struct drm_connector_state *new_conn_state)
6090+{
6091+ struct intel_crtc *old_crtc = old_conn_state->crtc ?
6092+ to_intel_crtc(old_conn_state->crtc) : NULL;
6093+ struct intel_crtc *new_crtc = new_conn_state->crtc ?
6094+ to_intel_crtc(new_conn_state->crtc) : NULL;
6095+
6096+ return new_crtc != old_crtc ||
6097+ (new_crtc &&
6098+ needs_modeset(intel_atomic_get_new_crtc_state(state, new_crtc)));
6099+}
6100+
6101+static void intel_encoders_update_prepare(struct intel_atomic_state *state)
6102+{
6103+ struct drm_connector_state *old_conn_state;
6104+ struct drm_connector_state *new_conn_state;
6105+ struct drm_connector *conn;
6106+ int i;
6107+
6108+ for_each_oldnew_connector_in_state(&state->base, conn,
6109+ old_conn_state, new_conn_state, i) {
6110+ struct intel_encoder *encoder;
6111+ struct intel_crtc *crtc;
6112+
6113+ if (!intel_connector_needs_modeset(state,
6114+ old_conn_state,
6115+ new_conn_state))
6116+ continue;
6117+
6118+ encoder = intel_connector_primary_encoder(to_intel_connector(conn));
6119+ if (!encoder->update_prepare)
6120+ continue;
6121+
6122+ crtc = new_conn_state->crtc ?
6123+ to_intel_crtc(new_conn_state->crtc) : NULL;
6124+ encoder->update_prepare(state, encoder, crtc);
6125+ }
6126+}
6127+
6128+static void intel_encoders_update_complete(struct intel_atomic_state *state)
6129+{
6130+ struct drm_connector_state *old_conn_state;
6131+ struct drm_connector_state *new_conn_state;
6132+ struct drm_connector *conn;
6133+ int i;
6134+
6135+ for_each_oldnew_connector_in_state(&state->base, conn,
6136+ old_conn_state, new_conn_state, i) {
6137+ struct intel_encoder *encoder;
6138+ struct intel_crtc *crtc;
6139+
6140+ if (!intel_connector_needs_modeset(state,
6141+ old_conn_state,
6142+ new_conn_state))
6143+ continue;
6144+
6145+ encoder = intel_connector_primary_encoder(to_intel_connector(conn));
6146+ if (!encoder->update_complete)
6147+ continue;
6148+
6149+ crtc = new_conn_state->crtc ?
6150+ to_intel_crtc(new_conn_state->crtc) : NULL;
6151+ encoder->update_complete(state, encoder, crtc);
6152+ }
6153+}
6154+
6155+static void intel_encoders_pre_pll_enable(struct intel_crtc *crtc,
60406156 struct intel_crtc_state *crtc_state,
6041− struct drm_atomic_state *old_state)
6157+ struct intel_atomic_state *state)
60426158 {
60436159 struct drm_connector_state *conn_state;
60446160 struct drm_connector *conn;
60456161 int i;
60466162
6047− for_each_new_connector_in_state(old_state, conn, conn_state, i) {
6163+ for_each_new_connector_in_state(&state->base, conn, conn_state, i) {
60486164 struct intel_encoder *encoder =
60496165 to_intel_encoder(conn_state->best_encoder);
60506166
6051− if (conn_state->crtc != crtc)
6167+ if (conn_state->crtc != &crtc->base)
60526168 continue;
60536169
60546170 if (encoder->pre_pll_enable)
@@ -6056,19 +6172,19 @@ static void intel_encoders_pre_pll_enable(struct drm_crtc *crtc, · 1 unmodified line
60566172 }
60576173 }
60586174
6059−static void intel_encoders_pre_enable(struct drm_crtc *crtc,
6175+static void intel_encoders_pre_enable(struct intel_crtc *crtc,
60606176 struct intel_crtc_state *crtc_state,
6061− struct drm_atomic_state *old_state)
6177+ struct intel_atomic_state *state)
60626178 {
60636179 struct drm_connector_state *conn_state;
60646180 struct drm_connector *conn;
60656181 int i;
60666182
6067− for_each_new_connector_in_state(old_state, conn, conn_state, i) {
6183+ for_each_new_connector_in_state(&state->base, conn, conn_state, i) {
60686184 struct intel_encoder *encoder =
60696185 to_intel_encoder(conn_state->best_encoder);
60706186
6071− if (conn_state->crtc != crtc)
6187+ if (conn_state->crtc != &crtc->base)
60726188 continue;
60736189
60746190 if (encoder->pre_enable)
@@ -6076,19 +6192,19 @@ static void intel_encoders_pre_enable(struct drm_crtc *crtc, · 1 unmodified line
60766192 }
60776193 }
60786194
6079−static void intel_encoders_enable(struct drm_crtc *crtc,
6195+static void intel_encoders_enable(struct intel_crtc *crtc,
60806196 struct intel_crtc_state *crtc_state,
6081− struct drm_atomic_state *old_state)
6197+ struct intel_atomic_state *state)
60826198 {
60836199 struct drm_connector_state *conn_state;
60846200 struct drm_connector *conn;
60856201 int i;
60866202
6087− for_each_new_connector_in_state(old_state, conn, conn_state, i) {
6203+ for_each_new_connector_in_state(&state->base, conn, conn_state, i) {
60886204 struct intel_encoder *encoder =
60896205 to_intel_encoder(conn_state->best_encoder);
60906206
6091− if (conn_state->crtc != crtc)
6207+ if (conn_state->crtc != &crtc->base)
60926208 continue;
60936209
60946210 if (encoder->enable)
@@ -6097,19 +6213,19 @@ static void intel_encoders_enable(struct drm_crtc *crtc, · 2 unmodified lines
60976213 }
60986214 }
60996215
6100−static void intel_encoders_disable(struct drm_crtc *crtc,
6216+static void intel_encoders_disable(struct intel_crtc *crtc,
61016217 struct intel_crtc_state *old_crtc_state,
6102− struct drm_atomic_state *old_state)
6218+ struct intel_atomic_state *state)
61036219 {
61046220 struct drm_connector_state *old_conn_state;
61056221 struct drm_connector *conn;
61066222 int i;
61076223
6108− for_each_old_connector_in_state(old_state, conn, old_conn_state, i) {
6224+ for_each_old_connector_in_state(&state->base, conn, old_conn_state, i) {
61096225 struct intel_encoder *encoder =
61106226 to_intel_encoder(old_conn_state->best_encoder);
61116227
6112− if (old_conn_state->crtc != crtc)
6228+ if (old_conn_state->crtc != &crtc->base)
61136229 continue;
61146230
61156231 intel_opregion_notify_encoder(encoder, false);
@@ -6118,19 +6234,19 @@ static void intel_encoders_disable(struct drm_crtc *crtc, · 2 unmodified lines
61186234 }
61196235 }
61206236
6121−static void intel_encoders_post_disable(struct drm_crtc *crtc,
6237+static void intel_encoders_post_disable(struct intel_crtc *crtc,
61226238 struct intel_crtc_state *old_crtc_state,
6123− struct drm_atomic_state *old_state)
6239+ struct intel_atomic_state *state)
61246240 {
61256241 struct drm_connector_state *old_conn_state;
61266242 struct drm_connector *conn;
61276243 int i;
61286244
6129− for_each_old_connector_in_state(old_state, conn, old_conn_state, i) {
6245+ for_each_old_connector_in_state(&state->base, conn, old_conn_state, i) {
61306246 struct intel_encoder *encoder =
61316247 to_intel_encoder(old_conn_state->best_encoder);
61326248
6133− if (old_conn_state->crtc != crtc)
6249+ if (old_conn_state->crtc != &crtc->base)
61346250 continue;
61356251
61366252 if (encoder->post_disable)
@@ -6138,19 +6254,19 @@ static void intel_encoders_post_disable(struct drm_crtc *crtc, · 1 unmodified line
61386254 }
61396255 }
61406256
6141−static void intel_encoders_post_pll_disable(struct drm_crtc *crtc,
6257+static void intel_encoders_post_pll_disable(struct intel_crtc *crtc,
61426258 struct intel_crtc_state *old_crtc_state,
6143− struct drm_atomic_state *old_state)
6259+ struct intel_atomic_state *state)
61446260 {
61456261 struct drm_connector_state *old_conn_state;
61466262 struct drm_connector *conn;
61476263 int i;
61486264
6149− for_each_old_connector_in_state(old_state, conn, old_conn_state, i) {
6265+ for_each_old_connector_in_state(&state->base, conn, old_conn_state, i) {
61506266 struct intel_encoder *encoder =
61516267 to_intel_encoder(old_conn_state->best_encoder);
61526268
6153− if (old_conn_state->crtc != crtc)
6269+ if (old_conn_state->crtc != &crtc->base)
61546270 continue;
61556271
61566272 if (encoder->post_pll_disable)
@@ -6158,19 +6274,19 @@ static void intel_encoders_post_pll_disable(struct drm_crtc *crtc, · 1 unmodified line
61586274 }
61596275 }
61606276
6161−static void intel_encoders_update_pipe(struct drm_crtc *crtc,
6277+static void intel_encoders_update_pipe(struct intel_crtc *crtc,
61626278 struct intel_crtc_state *crtc_state,
6163− struct drm_atomic_state *old_state)
6279+ struct intel_atomic_state *state)
61646280 {
61656281 struct drm_connector_state *conn_state;
61666282 struct drm_connector *conn;
61676283 int i;
61686284
6169− for_each_new_connector_in_state(old_state, conn, conn_state, i) {
6285+ for_each_new_connector_in_state(&state->base, conn, conn_state, i) {
61706286 struct intel_encoder *encoder =
61716287 to_intel_encoder(conn_state->best_encoder);
61726288
6173− if (conn_state->crtc != crtc)
6289+ if (conn_state->crtc != &crtc->base)
61746290 continue;
61756291
61766292 if (encoder->update_pipe)
@@ -6187,15 +6303,13 @@ static void intel_disable_primary_plane(const struct intel_crtc_state *crtc_stat · 10 unmodified lines
61876303 }
61886304
61896305 static void ironlake_crtc_enable(struct intel_crtc_state *pipe_config,
6190− struct drm_atomic_state *old_state)
6306+ struct intel_atomic_state *state)
61916307 {
61926308 struct drm_crtc *crtc = pipe_config->base.crtc;
61936309 struct drm_device *dev = crtc->dev;
61946310 struct drm_i915_private *dev_priv = to_i915(dev);
61956311 struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
61966312 int pipe = intel_crtc->pipe;
6197− struct intel_atomic_state *old_intel_state =
6198− to_intel_atomic_state(old_state);
61996313
62006314 if (WARN_ON(intel_crtc->active))
62016315 return;
@@ -6231,7 +6345,7 @@ static void ironlake_crtc_enable(struct intel_crtc_state *pipe_config, · 29 unmodified lines
62316345
62326346 intel_crtc->active = true;
62336347
6234− intel_encoders_pre_enable(crtc, pipe_config, old_state);
6348+ intel_encoders_pre_enable(intel_crtc, pipe_config, state);
62356349
62366350 if (pipe_config->has_pch_encoder) {
62376351 /* Note: FDI PLL enabling _must_ be done before we enable the
@@ -6255,16 +6369,16 @@ static void ironlake_crtc_enable(struct intel_crtc_state *pipe_config, · 17 unmodified lines
62556369 intel_disable_primary_plane(pipe_config);
62566370
62576371 if (dev_priv->display.initial_watermarks != NULL)
6258− dev_priv->display.initial_watermarks(old_intel_state, pipe_config);
6372+ dev_priv->display.initial_watermarks(state, pipe_config);
62596373 intel_enable_pipe(pipe_config);
62606374
62616375 if (pipe_config->has_pch_encoder)
6262− ironlake_pch_enable(old_intel_state, pipe_config);
6376+ ironlake_pch_enable(state, pipe_config);
62636377
62646378 assert_vblank_disabled(crtc);
62656379 intel_crtc_vblank_on(pipe_config);
62666380
6267− intel_encoders_enable(crtc, pipe_config, old_state);
6381+ intel_encoders_enable(intel_crtc, pipe_config, state);
62686382
62696383 if (HAS_PCH_CPT(dev_priv))
62706384 cpt_verify_modeset(dev, intel_crtc->pipe);
@@ -6310,33 +6424,37 @@ static void icl_pipe_mbus_enable(struct intel_crtc *crtc) · 39 unmodified lines
63106424 u32 val;
63116425
63126426 val = MBUS_DBOX_A_CREDIT(2);
6313− val |= MBUS_DBOX_BW_CREDIT(1);
6314− val |= MBUS_DBOX_B_CREDIT(8);
6427+
6428+ if (INTEL_GEN(dev_priv) >= 12) {
6429+ val |= MBUS_DBOX_BW_CREDIT(2);
6430+ val |= MBUS_DBOX_B_CREDIT(12);
6431+ } else {
6432+ val |= MBUS_DBOX_BW_CREDIT(1);
6433+ val |= MBUS_DBOX_B_CREDIT(8);
6434+ }
63156435
63166436 I915_WRITE(PIPE_MBUS_DBOX_CTL(pipe), val);
63176437 }
63186438
63196439 static void haswell_crtc_enable(struct intel_crtc_state *pipe_config,
6320− struct drm_atomic_state *old_state)
6440+ struct intel_atomic_state *state)
63216441 {
63226442 struct drm_crtc *crtc = pipe_config->base.crtc;
63236443 struct drm_i915_private *dev_priv = to_i915(crtc->dev);
63246444 struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
63256445 int pipe = intel_crtc->pipe, hsw_workaround_pipe;
63266446 enum transcoder cpu_transcoder = pipe_config->cpu_transcoder;
6327− struct intel_atomic_state *old_intel_state =
6328− to_intel_atomic_state(old_state);
63296447 bool psl_clkgate_wa;
63306448
63316449 if (WARN_ON(intel_crtc->active))
63326450 return;
63336451
6334− intel_encoders_pre_pll_enable(crtc, pipe_config, old_state);
6452+ intel_encoders_pre_pll_enable(intel_crtc, pipe_config, state);
63356453
63366454 if (pipe_config->shared_dpll)
63376455 intel_enable_shared_dpll(pipe_config);
63386456
6339− intel_encoders_pre_enable(crtc, pipe_config, old_state);
6457+ intel_encoders_pre_enable(intel_crtc, pipe_config, state);
63406458
63416459 if (intel_crtc_has_dp_encoder(pipe_config))
63426460 intel_dp_set_m_n(pipe_config, M1_N1);
@@ -6394,7 +6512,7 @@ static void haswell_crtc_enable(struct intel_crtc_state *pipe_config, · 51 unmodified lines
63946512 intel_ddi_enable_transcoder_func(pipe_config);
63956513
63966514 if (dev_priv->display.initial_watermarks != NULL)
6397− dev_priv->display.initial_watermarks(old_intel_state, pipe_config);
6515+ dev_priv->display.initial_watermarks(state, pipe_config);
63986516
63996517 if (INTEL_GEN(dev_priv) >= 11)
64006518 icl_pipe_mbus_enable(intel_crtc);
@@ -6404,7 +6522,7 @@ static void haswell_crtc_enable(struct intel_crtc_state *pipe_config, · 3 unmodified lines
64046522 intel_enable_pipe(pipe_config);
64056523
64066524 if (pipe_config->has_pch_encoder)
6407− lpt_pch_enable(old_intel_state, pipe_config);
6525+ lpt_pch_enable(state, pipe_config);
64086526
64096527 if (intel_crtc_has_type(pipe_config, INTEL_OUTPUT_DP_MST))
64106528 intel_ddi_set_vc_payload_alloc(pipe_config, true);
@@ -6412,7 +6530,7 @@ static void haswell_crtc_enable(struct intel_crtc_state *pipe_config, · 1 unmodified line
64126530 assert_vblank_disabled(crtc);
64136531 intel_crtc_vblank_on(pipe_config);
64146532
6415− intel_encoders_enable(crtc, pipe_config, old_state);
6533+ intel_encoders_enable(intel_crtc, pipe_config, state);
64166534
64176535 if (psl_clkgate_wa) {
64186536 intel_wait_for_vblank(dev_priv, pipe);
@@ -6444,7 +6562,7 @@ static void ironlake_pfit_disable(const struct intel_crtc_state *old_crtc_state) · 25 unmodified lines
64446562 }
64456563
64466564 static void ironlake_crtc_disable(struct intel_crtc_state *old_crtc_state,
6447− struct drm_atomic_state *old_state)
6565+ struct intel_atomic_state *state)
64486566 {
64496567 struct drm_crtc *crtc = old_crtc_state->base.crtc;
64506568 struct drm_device *dev = crtc->dev;
@@ -6460,7 +6578,7 @@ static void ironlake_crtc_disable(struct intel_crtc_state *old_crtc_state, · 9 unmodified lines
64606578 intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
64616579 intel_set_pch_fifo_underrun_reporting(dev_priv, pipe, false);
64626580
6463− intel_encoders_disable(crtc, old_crtc_state, old_state);
6581+ intel_encoders_disable(intel_crtc, old_crtc_state, state);
64646582
64656583 drm_crtc_vblank_off(crtc);
64666584 assert_vblank_disabled(crtc);
@@ -6472,7 +6590,7 @@ static void ironlake_crtc_disable(struct intel_crtc_state *old_crtc_state, · 5 unmodified lines
64726590 if (old_crtc_state->has_pch_encoder)
64736591 ironlake_fdi_disable(crtc);
64746592
6475− intel_encoders_post_disable(crtc, old_crtc_state, old_state);
6593+ intel_encoders_post_disable(intel_crtc, old_crtc_state, state);
64766594
64776595 if (old_crtc_state->has_pch_encoder) {
64786596 ironlake_disable_pch_transcoder(dev_priv, pipe);
@@ -6503,14 +6621,14 @@ static void ironlake_crtc_disable(struct intel_crtc_state *old_crtc_state, · 24 unmodified lines
65036621 }
65046622
65056623 static void haswell_crtc_disable(struct intel_crtc_state *old_crtc_state,
6506− struct drm_atomic_state *old_state)
6624+ struct intel_atomic_state *state)
65076625 {
65086626 struct drm_crtc *crtc = old_crtc_state->base.crtc;
65096627 struct drm_i915_private *dev_priv = to_i915(crtc->dev);
65106628 struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
65116629 enum transcoder cpu_transcoder = old_crtc_state->cpu_transcoder;
65126630
6513− intel_encoders_disable(crtc, old_crtc_state, old_state);
6631+ intel_encoders_disable(intel_crtc, old_crtc_state, state);
65146632
65156633 drm_crtc_vblank_off(crtc);
65166634 assert_vblank_disabled(crtc);
@@ -6532,9 +6650,9 @@ static void haswell_crtc_disable(struct intel_crtc_state *old_crtc_state, · 15 unmodified lines
65326650 else
65336651 ironlake_pfit_disable(old_crtc_state);
65346652
6535− intel_encoders_post_disable(crtc, old_crtc_state, old_state);
6653+ intel_encoders_post_disable(intel_crtc, old_crtc_state, state);
65366654
6537− intel_encoders_post_pll_disable(crtc, old_crtc_state, old_state);
6655+ intel_encoders_post_pll_disable(intel_crtc, old_crtc_state, state);
65386656 }
65396657
65406658 static void i9xx_pfit_enable(const struct intel_crtc_state *crtc_state)
@@ -6560,33 +6678,47 @@ static void i9xx_pfit_enable(const struct intel_crtc_state *crtc_state) · 19 unmodified lines
65606678 I915_WRITE(BCLRPAT(crtc->pipe), 0);
65616679 }
65626680
6563−bool intel_port_is_combophy(struct drm_i915_private *dev_priv, enum port port)
6681+bool intel_phy_is_combo(struct drm_i915_private *dev_priv, enum phy phy)
65646682 {
6565− if (port == PORT_NONE)
6683+ if (phy == PHY_NONE)
65666684 return false;
65676685
6568− if (IS_ELKHARTLAKE(dev_priv))
6569− return port <= PORT_C;
6686+ if (IS_ELKHARTLAKE(dev_priv) || INTEL_GEN(dev_priv) >= 12)
6687+ return phy <= PHY_C;
65706688
65716689 if (INTEL_GEN(dev_priv) >= 11)
6572− return port <= PORT_B;
6690+ return phy <= PHY_B;
65736691
65746692 return false;
65756693 }
65766694
6577−bool intel_port_is_tc(struct drm_i915_private *dev_priv, enum port port)
6695+bool intel_phy_is_tc(struct drm_i915_private *dev_priv, enum phy phy)
65786696 {
6697+ if (INTEL_GEN(dev_priv) >= 12)
6698+ return phy >= PHY_D && phy <= PHY_I;
6699+
65796700 if (INTEL_GEN(dev_priv) >= 11 && !IS_ELKHARTLAKE(dev_priv))
6580− return port >= PORT_C && port <= PORT_F;
6701+ return phy >= PHY_C && phy <= PHY_F;
65816702
65826703 return false;
65836704 }
65846705
6706+enum phy intel_port_to_phy(struct drm_i915_private *i915, enum port port)
6707+{
6708+ if (IS_ELKHARTLAKE(i915) && port == PORT_D)
6709+ return PHY_A;
6710+
6711+ return (enum phy)port;
6712+}
6713+
65856714 enum tc_port intel_port_to_tc(struct drm_i915_private *dev_priv, enum port port)
65866715 {
6587− if (!intel_port_is_tc(dev_priv, port))
6716+ if (!intel_phy_is_tc(dev_priv, intel_port_to_phy(dev_priv, port)))
65886717 return PORT_TC_NONE;
65896718
6719+ if (INTEL_GEN(dev_priv) >= 12)
6720+ return port - PORT_D;
6721+
65906722 return port - PORT_C;
65916723 }
65926724
@@ -6614,6 +6746,26 @@ enum intel_display_power_domain intel_port_to_power_domain(enum port port) · 21 unmodified lines
66146746 enum intel_display_power_domain
66156747 intel_aux_power_domain(struct intel_digital_port *dig_port)
66166748 {
6749+ struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
6750+ enum phy phy = intel_port_to_phy(dev_priv, dig_port->base.port);
6751+
6752+ if (intel_phy_is_tc(dev_priv, phy) &&
6753+ dig_port->tc_mode == TC_PORT_TBT_ALT) {
6754+ switch (dig_port->aux_ch) {
6755+ case AUX_CH_C:
6756+ return POWER_DOMAIN_AUX_TBT1;
6757+ case AUX_CH_D:
6758+ return POWER_DOMAIN_AUX_TBT2;
6759+ case AUX_CH_E:
6760+ return POWER_DOMAIN_AUX_TBT3;
6761+ case AUX_CH_F:
6762+ return POWER_DOMAIN_AUX_TBT4;
6763+ default:
6764+ MISSING_CASE(dig_port->aux_ch);
6765+ return POWER_DOMAIN_AUX_TBT1;
6766+ }
6767+ }
6768+
66176769 switch (dig_port->aux_ch) {
66186770 case AUX_CH_A:
66196771 return POWER_DOMAIN_AUX_A;
@@ -6633,14 +6785,12 @@ intel_aux_power_domain(struct intel_digital_port *dig_port) · 13 unmodified lines
66336785 }
66346786 }
66356787
6636−static u64 get_crtc_power_domains(struct drm_crtc *crtc,
6637− struct intel_crtc_state *crtc_state)
6788+static u64 get_crtc_power_domains(struct intel_crtc_state *crtc_state)
66386789 {
6639− struct drm_device *dev = crtc->dev;
6640− struct drm_i915_private *dev_priv = to_i915(dev);
6790+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
6791+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
66416792 struct drm_encoder *encoder;
6642− struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
6643− enum pipe pipe = intel_crtc->pipe;
6793+ enum pipe pipe = crtc->pipe;
66446794 u64 mask;
66456795 enum transcoder transcoder = crtc_state->cpu_transcoder;
66466796
@@ -6653,7 +6803,8 @@ static u64 get_crtc_power_domains(struct drm_crtc *crtc, · 6 unmodified lines
66536803 crtc_state->pch_pfit.force_thru)
66546804 mask |= BIT_ULL(POWER_DOMAIN_PIPE_PANEL_FITTER(pipe));
66556805
6656− drm_for_each_encoder_mask(encoder, dev, crtc_state->base.encoder_mask) {
6806+ drm_for_each_encoder_mask(encoder, &dev_priv->drm,
6807+ crtc_state->base.encoder_mask) {
66576808 struct intel_encoder *intel_encoder = to_intel_encoder(encoder);
66586809
66596810 mask |= BIT_ULL(intel_encoder->power_domain);
@@ -6669,17 +6820,16 @@ static u64 get_crtc_power_domains(struct drm_crtc *crtc, · 9 unmodified lines
66696820 }
66706821
66716822 static u64
6672−modeset_get_crtc_power_domains(struct drm_crtc *crtc,
6673− struct intel_crtc_state *crtc_state)
6823+modeset_get_crtc_power_domains(struct intel_crtc_state *crtc_state)
66746824 {
6675− struct drm_i915_private *dev_priv = to_i915(crtc->dev);
6676− struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
6825+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
6826+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
66776827 enum intel_display_power_domain domain;
66786828 u64 domains, new_domains, old_domains;
66796829
6680− old_domains = intel_crtc->enabled_power_domains;
6681− intel_crtc->enabled_power_domains = new_domains =
6682− get_crtc_power_domains(crtc, crtc_state);
6830+ old_domains = crtc->enabled_power_domains;
6831+ crtc->enabled_power_domains = new_domains =
6832+ get_crtc_power_domains(crtc_state);
66836833
66846834 domains = new_domains & ~old_domains;
66856835
@@ -6699,10 +6849,8 @@ static void modeset_put_power_domains(struct drm_i915_private *dev_priv, · 13 unmodified lines
66996849 }
67006850
67016851 static void valleyview_crtc_enable(struct intel_crtc_state *pipe_config,
6702− struct drm_atomic_state *old_state)
6852+ struct intel_atomic_state *state)
67036853 {
6704− struct intel_atomic_state *old_intel_state =
6705− to_intel_atomic_state(old_state);
67066854 struct drm_crtc *crtc = pipe_config->base.crtc;
67076855 struct drm_device *dev = crtc->dev;
67086856 struct drm_i915_private *dev_priv = to_i915(dev);
@@ -6729,7 +6877,7 @@ static void valleyview_crtc_enable(struct intel_crtc_state *pipe_config, · 20 unmodified lines
67296877
67306878 intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
67316879
6732− intel_encoders_pre_pll_enable(crtc, pipe_config, old_state);
6880+ intel_encoders_pre_pll_enable(intel_crtc, pipe_config, state);
67336881
67346882 if (IS_CHERRYVIEW(dev_priv)) {
67356883 chv_prepare_pll(intel_crtc, pipe_config);
@@ -6739,7 +6887,7 @@ static void valleyview_crtc_enable(struct intel_crtc_state *pipe_config, · 3 unmodified lines
67396887 vlv_enable_pll(intel_crtc, pipe_config);
67406888 }
67416889
6742− intel_encoders_pre_enable(crtc, pipe_config, old_state);
6890+ intel_encoders_pre_enable(intel_crtc, pipe_config, state);
67436891
67446892 i9xx_pfit_enable(pipe_config);
67456893
@@ -6748,14 +6896,13 @@ static void valleyview_crtc_enable(struct intel_crtc_state *pipe_config, · 2 unmodified lines
67486896 /* update DSPCNTR to configure gamma for pipe bottom color */
67496897 intel_disable_primary_plane(pipe_config);
67506898
6751− dev_priv->display.initial_watermarks(old_intel_state,
6752− pipe_config);
6899+ dev_priv->display.initial_watermarks(state, pipe_config);
67536900 intel_enable_pipe(pipe_config);
67546901
67556902 assert_vblank_disabled(crtc);
67566903 intel_crtc_vblank_on(pipe_config);
67576904
6758− intel_encoders_enable(crtc, pipe_config, old_state);
6905+ intel_encoders_enable(intel_crtc, pipe_config, state);
67596906 }
67606907
67616908 static void i9xx_set_pll_dividers(const struct intel_crtc_state *crtc_state)
@@ -6768,10 +6915,8 @@ static void i9xx_set_pll_dividers(const struct intel_crtc_state *crtc_state) · 6 unmodified lines
67686915 }
67696916
67706917 static void i9xx_crtc_enable(struct intel_crtc_state *pipe_config,
6771− struct drm_atomic_state *old_state)
6918+ struct intel_atomic_state *state)
67726919 {
6773− struct intel_atomic_state *old_intel_state =
6774− to_intel_atomic_state(old_state);
67756920 struct drm_crtc *crtc = pipe_config->base.crtc;
67766921 struct drm_device *dev = crtc->dev;
67776922 struct drm_i915_private *dev_priv = to_i915(dev);
@@ -6796,7 +6941,7 @@ static void i9xx_crtc_enable(struct intel_crtc_state *pipe_config, · 18 unmodified lines
67966941 if (!IS_GEN(dev_priv, 2))
67976942 intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
67986943
6799− intel_encoders_pre_enable(crtc, pipe_config, old_state);
6944+ intel_encoders_pre_enable(intel_crtc, pipe_config, state);
68006945
68016946 i9xx_enable_pll(intel_crtc, pipe_config);
68026947
@@ -6808,7 +6953,7 @@ static void i9xx_crtc_enable(struct intel_crtc_state *pipe_config, · 5 unmodified lines
68086953 intel_disable_primary_plane(pipe_config);
68096954
68106955 if (dev_priv->display.initial_watermarks != NULL)
6811− dev_priv->display.initial_watermarks(old_intel_state,
6956+ dev_priv->display.initial_watermarks(state,
68126957 pipe_config);
68136958 else
68146959 intel_update_watermarks(intel_crtc);
@@ -6817,7 +6962,7 @@ static void i9xx_crtc_enable(struct intel_crtc_state *pipe_config, · 2 unmodified lines
68176962 assert_vblank_disabled(crtc);
68186963 intel_crtc_vblank_on(pipe_config);
68196964
6820− intel_encoders_enable(crtc, pipe_config, old_state);
6965+ intel_encoders_enable(intel_crtc, pipe_config, state);
68216966 }
68226967
68236968 static void i9xx_pfit_disable(const struct intel_crtc_state *old_crtc_state)
@@ -6836,7 +6981,7 @@ static void i9xx_pfit_disable(const struct intel_crtc_state *old_crtc_state) · 12 unmodified lines
68366981 }
68376982
68386983 static void i9xx_crtc_disable(struct intel_crtc_state *old_crtc_state,
6839− struct drm_atomic_state *old_state)
6984+ struct intel_atomic_state *state)
68406985 {
68416986 struct drm_crtc *crtc = old_crtc_state->base.crtc;
68426987 struct drm_device *dev = crtc->dev;
@@ -6851,7 +6996,7 @@ static void i9xx_crtc_disable(struct intel_crtc_state *old_crtc_state, · 8 unmodified lines
68516996 if (IS_GEN(dev_priv, 2))
68526997 intel_wait_for_vblank(dev_priv, pipe);
68536998
6854− intel_encoders_disable(crtc, old_crtc_state, old_state);
6999+ intel_encoders_disable(intel_crtc, old_crtc_state, state);
68557000
68567001 drm_crtc_vblank_off(crtc);
68577002 assert_vblank_disabled(crtc);
@@ -6860,7 +7005,7 @@ static void i9xx_crtc_disable(struct intel_crtc_state *old_crtc_state, · 2 unmodified lines
68607005
68617006 i9xx_pfit_disable(old_crtc_state);
68627007
6863− intel_encoders_post_disable(crtc, old_crtc_state, old_state);
7008+ intel_encoders_post_disable(intel_crtc, old_crtc_state, state);
68647009
68657010 if (!intel_crtc_has_type(old_crtc_state, INTEL_OUTPUT_DSI)) {
68667011 if (IS_CHERRYVIEW(dev_priv))
@@ -6871,7 +7016,7 @@ static void i9xx_crtc_disable(struct intel_crtc_state *old_crtc_state, · 4 unmodified lines
68717016 i9xx_disable_pll(old_crtc_state);
68727017 }
68737018
6874− intel_encoders_post_pll_disable(crtc, old_crtc_state, old_state);
7019+ intel_encoders_post_pll_disable(intel_crtc, old_crtc_state, state);
68757020
68767021 if (!IS_GEN(dev_priv, 2))
68777022 intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
@@ -6925,7 +7070,7 @@ static void intel_crtc_disable_noatomic(struct drm_crtc *crtc, · 47 unmodified lines
69257070
69267071 WARN_ON(IS_ERR(crtc_state) || ret);
69277072
6928− dev_priv->display.crtc_disable(crtc_state, state);
7073+ dev_priv->display.crtc_disable(crtc_state, to_intel_atomic_state(state));
69297074
69307075 drm_atomic_state_put(state);
69317076
@@ -6988,7 +7133,7 @@ void intel_encoder_destroy(struct drm_encoder *encoder) · 56 unmodified lines
69887133
69897134 /* Cross check the actual hw state with our own modeset state tracking (and it's
69907135 * internal consistency). */
6991−static void intel_connector_verify_state(struct drm_crtc_state *crtc_state,
7136+static void intel_connector_verify_state(struct intel_crtc_state *crtc_state,
69927137 struct drm_connector_state *conn_state)
69937138 {
69947139 struct intel_connector *connector = to_intel_connector(conn_state->connector);
@@ -7006,7 +7151,7 @@ static void intel_connector_verify_state(struct drm_crtc_state *crtc_state, · 11 unmodified lines
70067151 if (!crtc_state)
70077152 return;
70087153
7009− I915_STATE_WARN(!crtc_state->active,
7154+ I915_STATE_WARN(!crtc_state->base.active,
70107155 "connector is active, but attached crtc isn't\n");
70117156
70127157 if (!encoder || encoder->type == INTEL_OUTPUT_DP_MST)
@@ -7018,7 +7163,7 @@ static void intel_connector_verify_state(struct drm_crtc_state *crtc_state, · 5 unmodified lines
70187163 I915_STATE_WARN(conn_state->crtc != encoder->base.crtc,
70197164 "attached encoder crtc differs from connector crtc\n");
70207165 } else {
7021− I915_STATE_WARN(crtc_state && crtc_state->active,
7166+ I915_STATE_WARN(crtc_state && crtc_state->base.active,
70227167 "attached crtc is active, but connector isn't\n");
70237168 I915_STATE_WARN(!crtc_state && conn_state->best_encoder,
70247169 "best encoder set without crtc!\n");
@@ -9484,6 +9629,8 @@ static int ironlake_crtc_compute_clock(struct intel_crtc *crtc, · 2459 unmodified lines
94849629 struct intel_crtc_state *crtc_state)
94859630 {
94869631 struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
9632+ struct intel_atomic_state *state =
9633+ to_intel_atomic_state(crtc_state->base.state);
94879634 const struct intel_limit *limit;
94889635 int refclk = 120000;
94899636
@@ -9525,7 +9672,7 @@ static int ironlake_crtc_compute_clock(struct intel_crtc *crtc, · 35 unmodified lines
95259672
95269673 ironlake_compute_dpll(crtc, crtc_state, NULL);
95279674
9528− if (!intel_get_shared_dpll(crtc_state, NULL)) {
9675+ if (!intel_reserve_shared_dplls(state, crtc, NULL)) {
95299676 DRM_DEBUG_KMS("failed to find PLL for pipe %c\n",
95309677 pipe_name(crtc->pipe));
95319678 return -EINVAL;
@@ -9906,7 +10053,7 @@ static int haswell_crtc_compute_clock(struct intel_crtc *crtc, · 374 unmodified lines
990610053 struct intel_encoder *encoder =
990710054 intel_get_crtc_new_encoder(state, crtc_state);
990810055
9909− if (!intel_get_shared_dpll(crtc_state, encoder)) {
10056+ if (!intel_reserve_shared_dplls(state, crtc, encoder)) {
991010057 DRM_DEBUG_KMS("failed to find PLL for pipe %c\n",
991110058 pipe_name(crtc->pipe));
991210059 return -EINVAL;
@@ -9936,22 +10083,37 @@ static void icelake_get_ddi_pll(struct drm_i915_private *dev_priv, · 23 unmodified lines
993610083 enum port port,
993710084 struct intel_crtc_state *pipe_config)
993810085 {
10086+ enum phy phy = intel_port_to_phy(dev_priv, port);
10087+ enum icl_port_dpll_id port_dpll_id;
993910088 enum intel_dpll_id id;
994010089 u32 temp;
994110090
9942− /* TODO: TBT pll not implemented. */
9943− if (intel_port_is_combophy(dev_priv, port)) {
9944− temp = I915_READ(DPCLKA_CFGCR0_ICL) &
9945− DPCLKA_CFGCR0_DDI_CLK_SEL_MASK(port);
9946− id = temp >> DPCLKA_CFGCR0_DDI_CLK_SEL_SHIFT(port);
9947− } else if (intel_port_is_tc(dev_priv, port)) {
9948− id = icl_tc_port_to_pll_id(intel_port_to_tc(dev_priv, port));
10091+ if (intel_phy_is_combo(dev_priv, phy)) {
10092+ temp = I915_READ(ICL_DPCLKA_CFGCR0) &
10093+ ICL_DPCLKA_CFGCR0_DDI_CLK_SEL_MASK(phy);
10094+ id = temp >> ICL_DPCLKA_CFGCR0_DDI_CLK_SEL_SHIFT(phy);
10095+ port_dpll_id = ICL_PORT_DPLL_DEFAULT;
10096+ } else if (intel_phy_is_tc(dev_priv, phy)) {
10097+ u32 clk_sel = I915_READ(DDI_CLK_SEL(port)) & DDI_CLK_SEL_MASK;
10098+
10099+ if (clk_sel == DDI_CLK_SEL_MG) {
10100+ id = icl_tc_port_to_pll_id(intel_port_to_tc(dev_priv,
10101+ port));
10102+ port_dpll_id = ICL_PORT_DPLL_MG_PHY;
10103+ } else {
10104+ WARN_ON(clk_sel < DDI_CLK_SEL_TBT_162);
10105+ id = DPLL_ID_ICL_TBTPLL;
10106+ port_dpll_id = ICL_PORT_DPLL_DEFAULT;
10107+ }
994910108 } else {
995010109 WARN(1, "Invalid port %x\n", port);
995110110 return;
995210111 }
995310112
9954− pipe_config->shared_dpll = intel_get_shared_dpll_by_id(dev_priv, id);
10113+ pipe_config->icl_port_dplls[port_dpll_id].pll =
10114+ intel_get_shared_dpll_by_id(dev_priv, id);
10115+
10116+ icl_set_active_port_dpll(pipe_config, port_dpll_id);
995510117 }
995610118
995710119 static void bxt_get_ddi_pll(struct drm_i915_private *dev_priv,
@@ -10191,7 +10353,11 @@ static void haswell_get_ddi_port_state(struct intel_crtc *crtc, · 233 unmodified lines
1019110353
1019210354 tmp = I915_READ(TRANS_DDI_FUNC_CTL(pipe_config->cpu_transcoder));
1019310355
10194− port = (tmp & TRANS_DDI_PORT_MASK) >> TRANS_DDI_PORT_SHIFT;
10356+ if (INTEL_GEN(dev_priv) >= 12)
10357+ port = (tmp & TGL_TRANS_DDI_PORT_MASK) >>
10358+ TGL_TRANS_DDI_PORT_SHIFT;
10359+ else
10360+ port = (tmp & TRANS_DDI_PORT_MASK) >> TRANS_DDI_PORT_SHIFT;
1019510361
1019610362 if (INTEL_GEN(dev_priv) >= 11)
1019710363 icelake_get_ddi_pll(dev_priv, port, pipe_config);
@@ -11297,7 +11463,7 @@ static void intel_crtc_destroy(struct drm_crtc *crtc) · 1099 unmodified lines
1129711463 *
1129811464 * Returns true or false.
1129911465 */
11300−static bool intel_wm_need_update(struct intel_plane_state *cur,
11466+static bool intel_wm_need_update(const struct intel_plane_state *cur,
1130111467 struct intel_plane_state *new)
1130211468 {
1130311469 /* Update watermarks on tiling or size changes. */
@@ -11329,33 +11495,28 @@ static bool needs_scaling(const struct intel_plane_state *state) · 25 unmodified lines
1132911495 }
1133011496
1133111497 int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_crtc_state,
11332− struct drm_crtc_state *crtc_state,
11498+ struct intel_crtc_state *crtc_state,
1133311499 const struct intel_plane_state *old_plane_state,
11334− struct drm_plane_state *plane_state)
11500+ struct intel_plane_state *plane_state)
1133511501 {
11336− struct intel_crtc_state *pipe_config = to_intel_crtc_state(crtc_state);
11337− struct drm_crtc *crtc = crtc_state->crtc;
11338− struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
11339− struct intel_plane *plane = to_intel_plane(plane_state->plane);
11340− struct drm_device *dev = crtc->dev;
11341− struct drm_i915_private *dev_priv = to_i915(dev);
11502+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
11503+ struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
11504+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
1134211505 bool mode_changed = needs_modeset(crtc_state);
1134311506 bool was_crtc_enabled = old_crtc_state->base.active;
11344− bool is_crtc_enabled = crtc_state->active;
11507+ bool is_crtc_enabled = crtc_state->base.active;
1134511508 bool turn_off, turn_on, visible, was_visible;
11346− struct drm_framebuffer *fb = plane_state->fb;
11509+ struct drm_framebuffer *fb = plane_state->base.fb;
1134711510 int ret;
1134811511
1134911512 if (INTEL_GEN(dev_priv) >= 9 && plane->id != PLANE_CURSOR) {
11350− ret = skl_update_scaler_plane(
11351− to_intel_crtc_state(crtc_state),
11352− to_intel_plane_state(plane_state));
11513+ ret = skl_update_scaler_plane(crtc_state, plane_state);
1135311514 if (ret)
1135411515 return ret;
1135511516 }
1135611517
1135711518 was_visible = old_plane_state->base.visible;
11358− visible = plane_state->visible;
11519+ visible = plane_state->base.visible;
1135911520
1136011521 if (!was_crtc_enabled && WARN_ON(was_visible))
1136111522 was_visible = false;
@@ -11371,22 +11532,22 @@ int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_crtc_stat · 9 unmodified lines
1137111532 * only combine the results from all planes in the current place?
1137211533 */
1137311534 if (!is_crtc_enabled) {
11374− plane_state->visible = visible = false;
11375− to_intel_crtc_state(crtc_state)->active_planes &= ~BIT(plane->id);
11376− to_intel_crtc_state(crtc_state)->data_rate[plane->id] = 0;
11535+ plane_state->base.visible = visible = false;
11536+ crtc_state->active_planes &= ~BIT(plane->id);
11537+ crtc_state->data_rate[plane->id] = 0;
1137711538 }
1137811539
1137911540 if (!was_visible && !visible)
1138011541 return 0;
1138111542
1138211543 if (fb != old_plane_state->base.fb)
11383− pipe_config->fb_changed = true;
11544+ crtc_state->fb_changed = true;
1138411545
1138511546 turn_off = was_visible && (!visible || mode_changed);
1138611547 turn_on = visible && (!was_visible || mode_changed);
1138711548
1138811549 DRM_DEBUG_ATOMIC("[CRTC:%d:%s] has [PLANE:%d:%s] with fb %i\n",
11389− intel_crtc->base.base.id, intel_crtc->base.name,
11550+ crtc->base.base.id, crtc->base.name,
1139011551 plane->base.base.id, plane->base.name,
1139111552 fb ? fb->base.id : -1);
1139211553
@@ -11397,29 +11558,28 @@ int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_crtc_stat · 4 unmodified lines
1139711558
1139811559 if (turn_on) {
1139911560 if (INTEL_GEN(dev_priv) < 5 && !IS_G4X(dev_priv))
11400− pipe_config->update_wm_pre = true;
11561+ crtc_state->update_wm_pre = true;
1140111562
1140211563 /* must disable cxsr around plane enable/disable */
1140311564 if (plane->id != PLANE_CURSOR)
11404− pipe_config->disable_cxsr = true;
11565+ crtc_state->disable_cxsr = true;
1140511566 } else if (turn_off) {
1140611567 if (INTEL_GEN(dev_priv) < 5 && !IS_G4X(dev_priv))
11407− pipe_config->update_wm_post = true;
11568+ crtc_state->update_wm_post = true;
1140811569
1140911570 /* must disable cxsr around plane enable/disable */
1141011571 if (plane->id != PLANE_CURSOR)
11411− pipe_config->disable_cxsr = true;
11412− } else if (intel_wm_need_update(to_intel_plane_state(plane->base.state),
11413− to_intel_plane_state(plane_state))) {
11572+ crtc_state->disable_cxsr = true;
11573+ } else if (intel_wm_need_update(old_plane_state, plane_state)) {
1141411574 if (INTEL_GEN(dev_priv) < 5 && !IS_G4X(dev_priv)) {
1141511575 /* FIXME bollocks */
11416− pipe_config->update_wm_pre = true;
11417− pipe_config->update_wm_post = true;
11576+ crtc_state->update_wm_pre = true;
11577+ crtc_state->update_wm_post = true;
1141811578 }
1141911579 }
1142011580
1142111581 if (visible || was_visible)
11422− pipe_config->fb_bits |= plane->frontbuffer_bit;
11582+ crtc_state->fb_bits |= plane->frontbuffer_bit;
1142311583
1142411584 /*
1142511585 * ILK/SNB DVSACNTR/Sprite Enable
@@ -11458,8 +11618,8 @@ int intel_plane_atomic_calc_changes(const struct intel_crtc_state *old_crtc_stat · 32 unmodified lines
1145811618 (IS_GEN_RANGE(dev_priv, 5, 6) ||
1145911619 IS_IVYBRIDGE(dev_priv)) &&
1146011620 (turn_on || (!needs_scaling(old_plane_state) &&
11461− needs_scaling(to_intel_plane_state(plane_state)))))
11462− pipe_config->disable_lp_wm = true;
11621+ needs_scaling(plane_state))))
11622+ crtc_state->disable_lp_wm = true;
1146311623
1146411624 return 0;
1146511625 }
@@ -11608,7 +11768,7 @@ static int intel_crtc_atomic_check(struct drm_crtc *crtc, · 142 unmodified lines
1160811768 struct intel_crtc_state *pipe_config =
1160911769 to_intel_crtc_state(crtc_state);
1161011770 int ret;
11611− bool mode_changed = needs_modeset(crtc_state);
11771+ bool mode_changed = needs_modeset(pipe_config);
1161211772
1161311773 if (INTEL_GEN(dev_priv) < 5 && !IS_G4X(dev_priv) &&
1161411774 mode_changed && !crtc_state->active)
@@ -12090,6 +12250,8 @@ clear_intel_crtc_state(struct intel_crtc_state *crtc_state) · 475 unmodified lines
1209012250 saved_state->scaler_state = crtc_state->scaler_state;
1209112251 saved_state->shared_dpll = crtc_state->shared_dpll;
1209212252 saved_state->dpll_hw_state = crtc_state->dpll_hw_state;
12253+ memcpy(saved_state->icl_port_dplls, crtc_state->icl_port_dplls,
12254+ sizeof(saved_state->icl_port_dplls));
1209312255 saved_state->crc_enabled = crtc_state->crc_enabled;
1209412256 if (IS_G4X(dev_priv) ||
1209512257 IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
@@ -12706,10 +12868,10 @@ static void intel_pipe_config_sanity_check(struct drm_i915_private *dev_priv, · 610 unmodified lines
1270612868 }
1270712869 }
1270812870
12709−static void verify_wm_state(struct drm_crtc *crtc,
12710− struct drm_crtc_state *new_state)
12871+static void verify_wm_state(struct intel_crtc *crtc,
12872+ struct intel_crtc_state *new_crtc_state)
1271112873 {
12712− struct drm_i915_private *dev_priv = to_i915(crtc->dev);
12874+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
1271312875 struct skl_hw_state {
1271412876 struct skl_ddb_entry ddb_y[I915_MAX_PLANES];
1271512877 struct skl_ddb_entry ddb_uv[I915_MAX_PLANES];
@@ -12719,21 +12881,20 @@ static void verify_wm_state(struct drm_crtc *crtc, · 3 unmodified lines
1271912881 struct skl_ddb_allocation *sw_ddb;
1272012882 struct skl_pipe_wm *sw_wm;
1272112883 struct skl_ddb_entry *hw_ddb_entry, *sw_ddb_entry;
12722− struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
12723− const enum pipe pipe = intel_crtc->pipe;
12884+ const enum pipe pipe = crtc->pipe;
1272412885 int plane, level, max_level = ilk_wm_max_level(dev_priv);
1272512886
12726− if (INTEL_GEN(dev_priv) < 9 || !new_state->active)
12887+ if (INTEL_GEN(dev_priv) < 9 || !new_crtc_state->base.active)
1272712888 return;
1272812889
1272912890 hw = kzalloc(sizeof(*hw), GFP_KERNEL);
1273012891 if (!hw)
1273112892 return;
1273212893
12733− skl_pipe_wm_get_hw_state(intel_crtc, &hw->wm);
12734− sw_wm = &to_intel_crtc_state(new_state)->wm.skl.optimal;
12894+ skl_pipe_wm_get_hw_state(crtc, &hw->wm);
12895+ sw_wm = &new_crtc_state->wm.skl.optimal;
1273512896
12736− skl_pipe_ddb_get_hw_state(intel_crtc, hw->ddb_y, hw->ddb_uv);
12897+ skl_pipe_ddb_get_hw_state(crtc, hw->ddb_y, hw->ddb_uv);
1273712898
1273812899 skl_ddb_get_hw_state(dev_priv, &hw->ddb);
1273912900 sw_ddb = &dev_priv->wm.skl_hw.ddb;
@@ -12781,7 +12942,7 @@ static void verify_wm_state(struct drm_crtc *crtc, · 41 unmodified lines
1278112942
1278212943 /* DDB */
1278312944 hw_ddb_entry = &hw->ddb_y[plane];
12784− sw_ddb_entry = &to_intel_crtc_state(new_state)->wm.skl.plane_ddb_y[plane];
12945+ sw_ddb_entry = &new_crtc_state->wm.skl.plane_ddb_y[plane];
1278512946
1278612947 if (!skl_ddb_entry_equal(hw_ddb_entry, sw_ddb_entry)) {
1278712948 DRM_ERROR("mismatch in DDB state pipe %c plane %d (expected (%u,%u), found (%u,%u))\n",
@@ -12833,7 +12994,7 @@ static void verify_wm_state(struct drm_crtc *crtc, · 45 unmodified lines
1283312994
1283412995 /* DDB */
1283512996 hw_ddb_entry = &hw->ddb_y[PLANE_CURSOR];
12836− sw_ddb_entry = &to_intel_crtc_state(new_state)->wm.skl.plane_ddb_y[PLANE_CURSOR];
12997+ sw_ddb_entry = &new_crtc_state->wm.skl.plane_ddb_y[PLANE_CURSOR];
1283712998
1283812999 if (!skl_ddb_entry_equal(hw_ddb_entry, sw_ddb_entry)) {
1283913000 DRM_ERROR("mismatch in DDB state pipe %c cursor (expected (%u,%u), found (%u,%u))\n",
@@ -12847,23 +13008,22 @@ static void verify_wm_state(struct drm_crtc *crtc, · 7 unmodified lines
1284713008 }
1284813009
1284913010 static void
12850−verify_connector_state(struct drm_device *dev,
12851− struct drm_atomic_state *state,
12852− struct drm_crtc *crtc)
13011+verify_connector_state(struct intel_atomic_state *state,
13012+ struct intel_crtc *crtc)
1285313013 {
1285413014 struct drm_connector *connector;
1285513015 struct drm_connector_state *new_conn_state;
1285613016 int i;
1285713017
12858− for_each_new_connector_in_state(state, connector, new_conn_state, i) {
13018+ for_each_new_connector_in_state(&state->base, connector, new_conn_state, i) {
1285913019 struct drm_encoder *encoder = connector->encoder;
12860− struct drm_crtc_state *crtc_state = NULL;
13020+ struct intel_crtc_state *crtc_state = NULL;
1286113021
12862− if (new_conn_state->crtc != crtc)
13022+ if (new_conn_state->crtc != &crtc->base)
1286313023 continue;
1286413024
1286513025 if (crtc)
12866− crtc_state = drm_atomic_get_new_crtc_state(state, new_conn_state->crtc);
13026+ crtc_state = intel_atomic_get_new_crtc_state(state, crtc);
1286713027
1286813028 intel_connector_verify_state(crtc_state, new_conn_state);
1286913029
@@ -12873,14 +13033,14 @@ verify_connector_state(struct drm_device *dev, · 3 unmodified lines
1287313033 }
1287413034
1287513035 static void
12876−verify_encoder_state(struct drm_device *dev, struct drm_atomic_state *state)
13036+verify_encoder_state(struct drm_i915_private *dev_priv, struct intel_atomic_state *state)
1287713037 {
1287813038 struct intel_encoder *encoder;
1287913039 struct drm_connector *connector;
1288013040 struct drm_connector_state *old_conn_state, *new_conn_state;
1288113041 int i;
1288213042
12883− for_each_intel_encoder(dev, encoder) {
13043+ for_each_intel_encoder(&dev_priv->drm, encoder) {
1288413044 bool enabled = false, found = false;
1288513045 enum pipe pipe;
1288613046
@@ -12888,7 +13048,7 @@ verify_encoder_state(struct drm_device *dev, struct drm_atomic_state *state) · 1 unmodified line
1288813048 encoder->base.base.id,
1288913049 encoder->base.name);
1289013050
12891− for_each_oldnew_connector_in_state(state, connector, old_conn_state,
13051+ for_each_oldnew_connector_in_state(&state->base, connector, old_conn_state,
1289213052 new_conn_state, i) {
1289313053 if (old_conn_state->best_encoder == &encoder->base)
1289413054 found = true;
@@ -12922,50 +13082,49 @@ verify_encoder_state(struct drm_device *dev, struct drm_atomic_state *state) · 27 unmodified lines
1292213082 }
1292313083
1292413084 static void
12925−verify_crtc_state(struct drm_crtc *crtc,
12926− struct drm_crtc_state *old_crtc_state,
12927− struct drm_crtc_state *new_crtc_state)
13085+verify_crtc_state(struct intel_crtc *crtc,
13086+ struct intel_crtc_state *old_crtc_state,
13087+ struct intel_crtc_state *new_crtc_state)
1292813088 {
12929− struct drm_device *dev = crtc->dev;
13089+ struct drm_device *dev = crtc->base.dev;
1293013090 struct drm_i915_private *dev_priv = to_i915(dev);
1293113091 struct intel_encoder *encoder;
12932− struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
12933− struct intel_crtc_state *pipe_config, *sw_config;
12934− struct drm_atomic_state *old_state;
13092+ struct intel_crtc_state *pipe_config;
13093+ struct drm_atomic_state *state;
1293513094 bool active;
1293613095
12937− old_state = old_crtc_state->state;
12938− __drm_atomic_helper_crtc_destroy_state(old_crtc_state);
12939− pipe_config = to_intel_crtc_state(old_crtc_state);
13096+ state = old_crtc_state->base.state;
13097+ __drm_atomic_helper_crtc_destroy_state(&old_crtc_state->base);
13098+ pipe_config = old_crtc_state;
1294013099 memset(pipe_config, 0, sizeof(*pipe_config));
12941− pipe_config->base.crtc = crtc;
12942− pipe_config->base.state = old_state;
13100+ pipe_config->base.crtc = &crtc->base;
13101+ pipe_config->base.state = state;
1294313102
12944− DRM_DEBUG_KMS("[CRTC:%d:%s]\n", crtc->base.id, crtc->name);
13103+ DRM_DEBUG_KMS("[CRTC:%d:%s]\n", crtc->base.base.id, crtc->base.name);
1294513104
12946− active = dev_priv->display.get_pipe_config(intel_crtc, pipe_config);
13105+ active = dev_priv->display.get_pipe_config(crtc, pipe_config);
1294713106
1294813107 /* we keep both pipes enabled on 830 */
1294913108 if (IS_I830(dev_priv))
12950− active = new_crtc_state->active;
13109+ active = new_crtc_state->base.active;
1295113110
12952− I915_STATE_WARN(new_crtc_state->active != active,
13111+ I915_STATE_WARN(new_crtc_state->base.active != active,
1295313112 "crtc active state doesn't match with hw state "
12954− "(expected %i, found %i)\n", new_crtc_state->active, active);
13113+ "(expected %i, found %i)\n", new_crtc_state->base.active, active);
1295513114
12956− I915_STATE_WARN(intel_crtc->active != new_crtc_state->active,
13115+ I915_STATE_WARN(crtc->active != new_crtc_state->base.active,
1295713116 "transitional active state does not match atomic hw state "
12958− "(expected %i, found %i)\n", new_crtc_state->active, intel_crtc->active);
13117+ "(expected %i, found %i)\n", new_crtc_state->base.active, crtc->active);
1295913118
12960− for_each_encoder_on_crtc(dev, crtc, encoder) {
13119+ for_each_encoder_on_crtc(dev, &crtc->base, encoder) {
1296113120 enum pipe pipe;
1296213121
1296313122 active = encoder->get_hw_state(encoder, &pipe);
12964− I915_STATE_WARN(active != new_crtc_state->active,
13123+ I915_STATE_WARN(active != new_crtc_state->base.active,
1296513124 "[ENCODER:%i] active %i with crtc active %i\n",
12966− encoder->base.base.id, active, new_crtc_state->active);
13125+ encoder->base.base.id, active, new_crtc_state->base.active);
1296713126
12968− I915_STATE_WARN(active && intel_crtc->pipe != pipe,
13127+ I915_STATE_WARN(active && crtc->pipe != pipe,
1296913128 "Encoder connected to wrong pipe %c\n",
1297013129 pipe_name(pipe));
1297113130
@@ -12975,16 +13134,16 @@ verify_crtc_state(struct drm_crtc *crtc, · 3 unmodified lines
1297513134
1297613135 intel_crtc_compute_pixel_rate(pipe_config);
1297713136
12978− if (!new_crtc_state->active)
13137+ if (!new_crtc_state->base.active)
1297913138 return;
1298013139
1298113140 intel_pipe_config_sanity_check(dev_priv, pipe_config);
1298213141
12983− sw_config = to_intel_crtc_state(new_crtc_state);
12984− if (!intel_pipe_config_compare(sw_config, pipe_config, false)) {
13142+ if (!intel_pipe_config_compare(new_crtc_state,
13143+ pipe_config, false)) {
1298513144 I915_STATE_WARN(1, "pipe state doesn't match!\n");
1298613145 intel_dump_pipe_config(pipe_config, NULL, "[hw state]");
12987− intel_dump_pipe_config(sw_config, NULL, "[sw state]");
13146+ intel_dump_pipe_config(new_crtc_state, NULL, "[sw state]");
1298813147 }
1298913148 }
1299013149
@@ -13004,8 +13163,8 @@ intel_verify_planes(struct intel_atomic_state *state) · 13 unmodified lines
1300413163 static void
1300513164 verify_single_dpll_state(struct drm_i915_private *dev_priv,
1300613165 struct intel_shared_dpll *pll,
13007− struct drm_crtc *crtc,
13008− struct drm_crtc_state *new_state)
13166+ struct intel_crtc *crtc,
13167+ struct intel_crtc_state *new_crtc_state)
1300913168 {
1301013169 struct intel_dpll_hw_state dpll_hw_state;
1301113170 unsigned int crtc_mask;
@@ -13035,16 +13194,16 @@ verify_single_dpll_state(struct drm_i915_private *dev_priv, · 23 unmodified lines
1303513194 return;
1303613195 }
1303713196
13038− crtc_mask = drm_crtc_mask(crtc);
13197+ crtc_mask = drm_crtc_mask(&crtc->base);
1303913198
13040− if (new_state->active)
13199+ if (new_crtc_state->base.active)
1304113200 I915_STATE_WARN(!(pll->active_mask & crtc_mask),
1304213201 "pll active mismatch (expected pipe %c in active mask 0x%02x)\n",
13043− pipe_name(drm_crtc_index(crtc)), pll->active_mask);
13202+ pipe_name(drm_crtc_index(&crtc->base)), pll->active_mask);
1304413203 else
1304513204 I915_STATE_WARN(pll->active_mask & crtc_mask,
1304613205 "pll active mismatch (didn't expect pipe %c in active mask 0x%02x)\n",
13047− pipe_name(drm_crtc_index(crtc)), pll->active_mask);
13206+ pipe_name(drm_crtc_index(&crtc->base)), pll->active_mask);
1304813207
1304913208 I915_STATE_WARN(!(pll->state.crtc_mask & crtc_mask),
1305013209 "pll enabled crtcs mismatch (expected 0x%x in 0x%02x)\n",
@@ -13057,51 +13216,47 @@ verify_single_dpll_state(struct drm_i915_private *dev_priv, · 6 unmodified lines
1305713216 }
1305813217
1305913218 static void
13060−verify_shared_dpll_state(struct drm_device *dev, struct drm_crtc *crtc,
13061− struct drm_crtc_state *old_crtc_state,
13062− struct drm_crtc_state *new_crtc_state)
13219+verify_shared_dpll_state(struct intel_crtc *crtc,
13220+ struct intel_crtc_state *old_crtc_state,
13221+ struct intel_crtc_state *new_crtc_state)
1306313222 {
13064− struct drm_i915_private *dev_priv = to_i915(dev);
13065− struct intel_crtc_state *old_state = to_intel_crtc_state(old_crtc_state);
13066− struct intel_crtc_state *new_state = to_intel_crtc_state(new_crtc_state);
13223+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
1306713224
13068− if (new_state->shared_dpll)
13069− verify_single_dpll_state(dev_priv, new_state->shared_dpll, crtc, new_crtc_state);
13225+ if (new_crtc_state->shared_dpll)
13226+ verify_single_dpll_state(dev_priv, new_crtc_state->shared_dpll, crtc, new_crtc_state);
1307013227
13071− if (old_state->shared_dpll &&
13072− old_state->shared_dpll != new_state->shared_dpll) {
13073− unsigned int crtc_mask = drm_crtc_mask(crtc);
13074− struct intel_shared_dpll *pll = old_state->shared_dpll;
13228+ if (old_crtc_state->shared_dpll &&
13229+ old_crtc_state->shared_dpll != new_crtc_state->shared_dpll) {
13230+ unsigned int crtc_mask = drm_crtc_mask(&crtc->base);
13231+ struct intel_shared_dpll *pll = old_crtc_state->shared_dpll;
1307513232
1307613233 I915_STATE_WARN(pll->active_mask & crtc_mask,
1307713234 "pll active mismatch (didn't expect pipe %c in active mask)\n",
13078− pipe_name(drm_crtc_index(crtc)));
13235+ pipe_name(drm_crtc_index(&crtc->base)));
1307913236 I915_STATE_WARN(pll->state.crtc_mask & crtc_mask,
1308013237 "pll enabled crtcs mismatch (found %x in enabled mask)\n",
13081− pipe_name(drm_crtc_index(crtc)));
13238+ pipe_name(drm_crtc_index(&crtc->base)));
1308213239 }
1308313240 }
1308413241
1308513242 static void
13086−intel_modeset_verify_crtc(struct drm_crtc *crtc,
13087− struct drm_atomic_state *state,
13088− struct drm_crtc_state *old_state,
13089− struct drm_crtc_state *new_state)
13243+intel_modeset_verify_crtc(struct intel_crtc *crtc,
13244+ struct intel_atomic_state *state,
13245+ struct intel_crtc_state *old_crtc_state,
13246+ struct intel_crtc_state *new_crtc_state)
1309013247 {
13091− if (!needs_modeset(new_state) &&
13092− !to_intel_crtc_state(new_state)->update_pipe)
13248+ if (!needs_modeset(new_crtc_state) && !new_crtc_state->update_pipe)
1309313249 return;
1309413250
13095− verify_wm_state(crtc, new_state);
13096− verify_connector_state(crtc->dev, state, crtc);
13097− verify_crtc_state(crtc, old_state, new_state);
13098− verify_shared_dpll_state(crtc->dev, crtc, old_state, new_state);
13251+ verify_wm_state(crtc, new_crtc_state);
13252+ verify_connector_state(state, crtc);
13253+ verify_crtc_state(crtc, old_crtc_state, new_crtc_state);
13254+ verify_shared_dpll_state(crtc, old_crtc_state, new_crtc_state);
1309913255 }
1310013256
1310113257 static void
13102−verify_disabled_dpll_state(struct drm_device *dev)
13258+verify_disabled_dpll_state(struct drm_i915_private *dev_priv)
1310313259 {
13104− struct drm_i915_private *dev_priv = to_i915(dev);
1310513260 int i;
1310613261
1310713262 for (i = 0; i < dev_priv->num_shared_dpll; i++)
@@ -13109,12 +13264,12 @@ verify_disabled_dpll_state(struct drm_device *dev) · 1 unmodified line
1310913264 }
1311013265
1311113266 static void
13112−intel_modeset_verify_disabled(struct drm_device *dev,
13113− struct drm_atomic_state *state)
13267+intel_modeset_verify_disabled(struct drm_i915_private *dev_priv,
13268+ struct intel_atomic_state *state)
1311413269 {
13115− verify_encoder_state(dev, state);
13116− verify_connector_state(dev, state, NULL);
13117− verify_disabled_dpll_state(dev);
13270+ verify_encoder_state(dev_priv, state);
13271+ verify_connector_state(state, NULL);
13272+ verify_disabled_dpll_state(dev_priv);
1311813273 }
1311913274
1312013275 static void update_scanline_offset(const struct intel_crtc_state *crtc_state)
@@ -13168,27 +13323,18 @@ static void update_scanline_offset(const struct intel_crtc_state *crtc_state) · 47 unmodified lines
1316813323 static void intel_modeset_clear_plls(struct intel_atomic_state *state)
1316913324 {
1317013325 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
13171− struct intel_crtc_state *old_crtc_state, *new_crtc_state;
13326+ struct intel_crtc_state *new_crtc_state;
1317213327 struct intel_crtc *crtc;
1317313328 int i;
1317413329
1317513330 if (!dev_priv->display.crtc_compute_clock)
1317613331 return;
1317713332
13178− for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
13179− new_crtc_state, i) {
13180− struct intel_shared_dpll *old_dpll =
13181− old_crtc_state->shared_dpll;
13182−
13183− if (!needs_modeset(&new_crtc_state->base))
13184− continue;
13185−
13186− new_crtc_state->shared_dpll = NULL;
13187−
13188− if (!old_dpll)
13333+ for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {
13334+ if (!needs_modeset(new_crtc_state))
1318913335 continue;
1319013336
13191− intel_release_shared_dpll(old_dpll, crtc, &state->base);
13337+ intel_release_shared_dplls(state, crtc);
1319213338 }
1319313339 }
1319413340
@@ -13210,7 +13356,7 @@ static int haswell_mode_set_planes_workaround(struct intel_atomic_state *state) · 15 unmodified lines
1321013356 /* look at all crtc's that are going to be enabled in during modeset */
1321113357 for_each_new_intel_crtc_in_state(state, crtc, crtc_state, i) {
1321213358 if (!crtc_state->base.active ||
13213− !needs_modeset(&crtc_state->base))
13359+ !needs_modeset(crtc_state))
1321413360 continue;
1321513361
1321613362 if (first_crtc_state) {
@@ -13235,7 +13381,7 @@ static int haswell_mode_set_planes_workaround(struct intel_atomic_state *state) · 18 unmodified lines
1323513381 crtc_state->hsw_workaround_pipe = INVALID_PIPE;
1323613382
1323713383 if (!crtc_state->base.active ||
13238− needs_modeset(&crtc_state->base))
13384+ needs_modeset(crtc_state))
1323913385 continue;
1324013386
1324113387 /* 2 or more enabled crtcs means no need for w/a */
@@ -13253,15 +13399,16 @@ static int haswell_mode_set_planes_workaround(struct intel_atomic_state *state) · 11 unmodified lines
1325313399 return 0;
1325413400 }
1325513401
13256−static int intel_lock_all_pipes(struct drm_atomic_state *state)
13402+static int intel_lock_all_pipes(struct intel_atomic_state *state)
1325713403 {
13258− struct drm_crtc *crtc;
13404+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
13405+ struct intel_crtc *crtc;
1325913406
1326013407 /* Add all pipes to the state */
13261− for_each_crtc(state->dev, crtc) {
13262− struct drm_crtc_state *crtc_state;
13408+ for_each_intel_crtc(&dev_priv->drm, crtc) {
13409+ struct intel_crtc_state *crtc_state;
1326313410
13264− crtc_state = drm_atomic_get_crtc_state(state, crtc);
13411+ crtc_state = intel_atomic_get_crtc_state(&state->base, crtc);
1326513412 if (IS_ERR(crtc_state))
1326613413 return PTR_ERR(crtc_state);
1326713414 }
@@ -13269,32 +13416,35 @@ static int intel_lock_all_pipes(struct drm_atomic_state *state) · 1 unmodified line
1326913416 return 0;
1327013417 }
1327113418
13272−static int intel_modeset_all_pipes(struct drm_atomic_state *state)
13419+static int intel_modeset_all_pipes(struct intel_atomic_state *state)
1327313420 {
13274− struct drm_crtc *crtc;
13421+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
13422+ struct intel_crtc *crtc;
1327513423
1327613424 /*
1327713425 * Add all pipes to the state, and force
1327813426 * a modeset on all the active ones.
1327913427 */
13280− for_each_crtc(state->dev, crtc) {
13281− struct drm_crtc_state *crtc_state;
13428+ for_each_intel_crtc(&dev_priv->drm, crtc) {
13429+ struct intel_crtc_state *crtc_state;
1328213430 int ret;
1328313431
13284− crtc_state = drm_atomic_get_crtc_state(state, crtc);
13432+ crtc_state = intel_atomic_get_crtc_state(&state->base, crtc);
1328513433 if (IS_ERR(crtc_state))
1328613434 return PTR_ERR(crtc_state);
1328713435
13288− if (!crtc_state->active || needs_modeset(crtc_state))
13436+ if (!crtc_state->base.active || needs_modeset(crtc_state))
1328913437 continue;
1329013438
13291− crtc_state->mode_changed = true;
13439+ crtc_state->base.mode_changed = true;
1329213440
13293− ret = drm_atomic_add_affected_connectors(state, crtc);
13441+ ret = drm_atomic_add_affected_connectors(&state->base,
13442+ &crtc->base);
1329413443 if (ret)
1329513444 return ret;
1329613445
13297− ret = drm_atomic_add_affected_planes(state, crtc);
13446+ ret = drm_atomic_add_affected_planes(&state->base,
13447+ &crtc->base);
1329813448 if (ret)
1329913449 return ret;
1330013450 }
@@ -13356,18 +13506,18 @@ static int intel_modeset_checks(struct intel_atomic_state *state) · 55 unmodified lines
1335613506 */
1335713507 if (intel_cdclk_changed(&dev_priv->cdclk.logical,
1335813508 &state->cdclk.logical)) {
13359− ret = intel_lock_all_pipes(&state->base);
13509+ ret = intel_lock_all_pipes(state);
1336013510 if (ret < 0)
1336113511 return ret;
1336213512 }
1336313513
1336413514 if (is_power_of_2(state->active_crtcs)) {
13365− struct drm_crtc *crtc;
13366− struct drm_crtc_state *crtc_state;
13515+ struct intel_crtc *crtc;
13516+ struct intel_crtc_state *crtc_state;
1336713517
1336813518 pipe = ilog2(state->active_crtcs);
13369− crtc = &intel_get_crtc_for_pipe(dev_priv, pipe)->base;
13370− crtc_state = drm_atomic_get_new_crtc_state(&state->base, crtc);
13519+ crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
13520+ crtc_state = intel_atomic_get_new_crtc_state(state, crtc);
1337113521 if (crtc_state && needs_modeset(crtc_state))
1337213522 pipe = INVALID_PIPE;
1337313523 } else {
@@ -13379,14 +13529,14 @@ static int intel_modeset_checks(struct intel_atomic_state *state) · 5 unmodified lines
1337913529 intel_cdclk_needs_cd2x_update(dev_priv,
1338013530 &dev_priv->cdclk.actual,
1338113531 &state->cdclk.actual)) {
13382− ret = intel_lock_all_pipes(&state->base);
13532+ ret = intel_lock_all_pipes(state);
1338313533 if (ret < 0)
1338413534 return ret;
1338513535
1338613536 state->cdclk.pipe = pipe;
1338713537 } else if (intel_cdclk_needs_modeset(&dev_priv->cdclk.actual,
1338813538 &state->cdclk.actual)) {
13389− ret = intel_modeset_all_pipes(&state->base);
13539+ ret = intel_modeset_all_pipes(state);
1339013540 if (ret < 0)
1339113541 return ret;
1339213542
@@ -13478,7 +13628,7 @@ static int intel_atomic_check(struct drm_device *dev, · 85 unmodified lines
1347813628
1347913629 for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
1348013630 new_crtc_state, i) {
13481− if (!needs_modeset(&new_crtc_state->base))
13631+ if (!needs_modeset(new_crtc_state))
1348213632 continue;
1348313633
1348413634 if (!new_crtc_state->base.enable) {
@@ -13492,7 +13642,7 @@ static int intel_atomic_check(struct drm_device *dev, · 7 unmodified lines
1349213642
1349313643 intel_crtc_check_fastset(old_crtc_state, new_crtc_state);
1349413644
13495− if (needs_modeset(&new_crtc_state->base))
13645+ if (needs_modeset(new_crtc_state))
1349613646 any_ms = true;
1349713647 }
1349813648
@@ -13527,12 +13677,12 @@ static int intel_atomic_check(struct drm_device *dev, · 28 unmodified lines
1352713677
1352813678 for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state,
1352913679 new_crtc_state, i) {
13530− if (!needs_modeset(&new_crtc_state->base) &&
13680+ if (!needs_modeset(new_crtc_state) &&
1353113681 !new_crtc_state->update_pipe)
1353213682 continue;
1353313683
1353413684 intel_dump_pipe_config(new_crtc_state, state,
13535− needs_modeset(&new_crtc_state->base) ?
13685+ needs_modeset(new_crtc_state) ?
1353613686 "[modeset]" : "[fastset]");
1353713687 }
1353813688
@@ -13553,10 +13703,10 @@ static int intel_atomic_check(struct drm_device *dev, · 14 unmodified lines
1355313703 return ret;
1355413704 }
1355513705
13556−static int intel_atomic_prepare_commit(struct drm_device *dev,
13557− struct drm_atomic_state *state)
13706+static int intel_atomic_prepare_commit(struct intel_atomic_state *state)
1355813707 {
13559− return drm_atomic_helper_prepare_planes(dev, state);
13708+ return drm_atomic_helper_prepare_planes(state->base.dev,
13709+ &state->base);
1356013710 }
1356113711
1356213712 u32 intel_crtc_get_vblank_counter(struct intel_crtc *crtc)
@@ -13567,60 +13717,57 @@ u32 intel_crtc_get_vblank_counter(struct intel_crtc *crtc) · 4 unmodified lines
1356713717 if (!vblank->max_vblank_count)
1356813718 return (u32)drm_crtc_accurate_vblank_count(&crtc->base);
1356913719
13570− return dev->driver->get_vblank_counter(dev, crtc->pipe);
13720+ return crtc->base.funcs->get_vblank_counter(&crtc->base);
1357113721 }
1357213722
13573−static void intel_update_crtc(struct drm_crtc *crtc,
13574− struct drm_atomic_state *state,
13575− struct drm_crtc_state *old_crtc_state,
13576− struct drm_crtc_state *new_crtc_state)
13723+static void intel_update_crtc(struct intel_crtc *crtc,
13724+ struct intel_atomic_state *state,
13725+ struct intel_crtc_state *old_crtc_state,
13726+ struct intel_crtc_state *new_crtc_state)
1357713727 {
13578− struct drm_device *dev = crtc->dev;
13728+ struct drm_device *dev = state->base.dev;
1357913729 struct drm_i915_private *dev_priv = to_i915(dev);
13580− struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
13581− struct intel_crtc_state *pipe_config = to_intel_crtc_state(new_crtc_state);
1358213730 bool modeset = needs_modeset(new_crtc_state);
1358313731 struct intel_plane_state *new_plane_state =
13584− intel_atomic_get_new_plane_state(to_intel_atomic_state(state),
13585− to_intel_plane(crtc->primary));
13732+ intel_atomic_get_new_plane_state(state,
13733+ to_intel_plane(crtc->base.primary));
1358613734
1358713735 if (modeset) {
13588− update_scanline_offset(pipe_config);
13589− dev_priv->display.crtc_enable(pipe_config, state);
13736+ update_scanline_offset(new_crtc_state);
13737+ dev_priv->display.crtc_enable(new_crtc_state, state);
1359013738
1359113739 /* vblanks work again, re-enable pipe CRC. */
13592− intel_crtc_enable_pipe_crc(intel_crtc);
13740+ intel_crtc_enable_pipe_crc(crtc);
1359313741 } else {
13594− intel_pre_plane_update(to_intel_crtc_state(old_crtc_state),
13595− pipe_config);
13742+ intel_pre_plane_update(old_crtc_state, new_crtc_state);
1359613743
13597− if (pipe_config->update_pipe)
13598− intel_encoders_update_pipe(crtc, pipe_config, state);
13744+ if (new_crtc_state->update_pipe)
13745+ intel_encoders_update_pipe(crtc, new_crtc_state, state);
1359913746 }
1360013747
13601− if (pipe_config->update_pipe && !pipe_config->enable_fbc)
13602− intel_fbc_disable(intel_crtc);
13748+ if (new_crtc_state->update_pipe && !new_crtc_state->enable_fbc)
13749+ intel_fbc_disable(crtc);
1360313750 else if (new_plane_state)
13604− intel_fbc_enable(intel_crtc, pipe_config, new_plane_state);
13751+ intel_fbc_enable(crtc, new_crtc_state, new_plane_state);
1360513752
13606− intel_begin_crtc_commit(to_intel_atomic_state(state), intel_crtc);
13753+ intel_begin_crtc_commit(state, crtc);
1360713754
1360813755 if (INTEL_GEN(dev_priv) >= 9)
13609− skl_update_planes_on_crtc(to_intel_atomic_state(state), intel_crtc);
13756+ skl_update_planes_on_crtc(state, crtc);
1361013757 else
13611− i9xx_update_planes_on_crtc(to_intel_atomic_state(state), intel_crtc);
13758+ i9xx_update_planes_on_crtc(state, crtc);
1361213759
13613− intel_finish_crtc_commit(to_intel_atomic_state(state), intel_crtc);
13760+ intel_finish_crtc_commit(state, crtc);
1361413761 }
1361513762
13616−static void intel_update_crtcs(struct drm_atomic_state *state)
13763+static void intel_update_crtcs(struct intel_atomic_state *state)
1361713764 {
13618− struct drm_crtc *crtc;
13619− struct drm_crtc_state *old_crtc_state, *new_crtc_state;
13765+ struct intel_crtc *crtc;
13766+ struct intel_crtc_state *old_crtc_state, *new_crtc_state;
1362013767 int i;
1362113768
13622− for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
13623− if (!new_crtc_state->active)
13769+ for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
13770+ if (!new_crtc_state->base.active)
1362413771 continue;
1362513772
1362613773 intel_update_crtc(crtc, state, old_crtc_state,
@@ -13628,26 +13775,23 @@ static void intel_update_crtcs(struct drm_atomic_state *state) · 1 unmodified line
1362813775 }
1362913776 }
1363013777
13631−static void skl_update_crtcs(struct drm_atomic_state *state)
13778+static void skl_update_crtcs(struct intel_atomic_state *state)
1363213779 {
13633− struct drm_i915_private *dev_priv = to_i915(state->dev);
13634− struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
13635− struct drm_crtc *crtc;
13636− struct intel_crtc *intel_crtc;
13637− struct drm_crtc_state *old_crtc_state, *new_crtc_state;
13638− struct intel_crtc_state *cstate;
13780+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
13781+ struct intel_crtc *crtc;
13782+ struct intel_crtc_state *old_crtc_state, *new_crtc_state;
1363913783 unsigned int updated = 0;
1364013784 bool progress;
1364113785 enum pipe pipe;
1364213786 int i;
1364313787 u8 hw_enabled_slices = dev_priv->wm.skl_hw.ddb.enabled_slices;
13644− u8 required_slices = intel_state->wm_results.ddb.enabled_slices;
13788+ u8 required_slices = state->wm_results.ddb.enabled_slices;
1364513789 struct skl_ddb_entry entries[I915_MAX_PIPES] = {};
1364613790
13647− for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i)
13791+ for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i)
1364813792 /* ignore allocations for crtc's that have been turned off. */
13649− if (new_crtc_state->active)
13650− entries[i] = to_intel_crtc_state(old_crtc_state)->wm.skl.ddb;
13793+ if (new_crtc_state->base.active)
13794+ entries[i] = old_crtc_state->wm.skl.ddb;
1365113795
1365213796 /* If 2nd DBuf slice required, enable it here */
1365313797 if (INTEL_GEN(dev_priv) >= 11 && required_slices > hw_enabled_slices)
@@ -13662,24 +13806,22 @@ static void skl_update_crtcs(struct drm_atomic_state *state) · 8 unmodified lines
1366213806 do {
1366313807 progress = false;
1366413808
13665− for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
13809+ for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
1366613810 bool vbl_wait = false;
13667− unsigned int cmask = drm_crtc_mask(crtc);
13811+ unsigned int cmask = drm_crtc_mask(&crtc->base);
1366813812
13669− intel_crtc = to_intel_crtc(crtc);
13670− cstate = to_intel_crtc_state(new_crtc_state);
13671− pipe = intel_crtc->pipe;
13813+ pipe = crtc->pipe;
1367213814
13673− if (updated & cmask || !cstate->base.active)
13815+ if (updated & cmask || !new_crtc_state->base.active)
1367413816 continue;
1367513817
13676− if (skl_ddb_allocation_overlaps(&cstate->wm.skl.ddb,
13818+ if (skl_ddb_allocation_overlaps(&new_crtc_state->wm.skl.ddb,
1367713819 entries,
1367813820 INTEL_INFO(dev_priv)->num_pipes, i))
1367913821 continue;
1368013822
1368113823 updated |= cmask;
13682− entries[i] = cstate->wm.skl.ddb;
13824+ entries[i] = new_crtc_state->wm.skl.ddb;
1368313825
1368413826 /*
1368513827 * If this is an already active pipe, it's DDB changed,
@@ -13687,10 +13829,10 @@ static void skl_update_crtcs(struct drm_atomic_state *state) · 1 unmodified line
1368713829 * then we need to wait for a vblank to pass for the
1368813830 * new ddb allocation to take effect.
1368913831 */
13690− if (!skl_ddb_entry_equal(&cstate->wm.skl.ddb,
13691− &to_intel_crtc_state(old_crtc_state)->wm.skl.ddb) &&
13692− !new_crtc_state->active_changed &&
13693− intel_state->wm_results.dirty_pipes != updated)
13832+ if (!skl_ddb_entry_equal(&new_crtc_state->wm.skl.ddb,
13833+ &old_crtc_state->wm.skl.ddb) &&
13834+ !new_crtc_state->base.active_changed &&
13835+ state->wm_results.dirty_pipes != updated)
1369413836 vbl_wait = true;
1369513837
1369613838 intel_update_crtc(crtc, state, old_crtc_state,
@@ -13736,18 +13878,21 @@ static void intel_atomic_commit_fence_wait(struct intel_atomic_state *intel_stat · 39 unmodified lines
1373613878 for (;;) {
1373713879 prepare_to_wait(&intel_state->commit_ready.wait,
1373813880 &wait_fence, TASK_UNINTERRUPTIBLE);
13739− prepare_to_wait(&dev_priv->gpu_error.wait_queue,
13881+ prepare_to_wait(bit_waitqueue(&dev_priv->gt.reset.flags,
13882+ I915_RESET_MODESET),
1374013883 &wait_reset, TASK_UNINTERRUPTIBLE);
1374113884
1374213885
13743− if (i915_sw_fence_done(&intel_state->commit_ready)
13744− || test_bit(I915_RESET_MODESET, &dev_priv->gpu_error.flags))
13886+ if (i915_sw_fence_done(&intel_state->commit_ready) ||
13887+ test_bit(I915_RESET_MODESET, &dev_priv->gt.reset.flags))
1374513888 break;
1374613889
1374713890 schedule();
1374813891 }
1374913892 finish_wait(&intel_state->commit_ready.wait, &wait_fence);
13750− finish_wait(&dev_priv->gpu_error.wait_queue, &wait_reset);
13893+ finish_wait(bit_waitqueue(&dev_priv->gt.reset.flags,
13894+ I915_RESET_MODESET),
13895+ &wait_reset);
1375113896 }
1375213897
1375313898 static void intel_atomic_cleanup_work(struct work_struct *work)
@@ -13763,57 +13908,49 @@ static void intel_atomic_cleanup_work(struct work_struct *work) · 9 unmodified lines
1376313908 intel_atomic_helper_free_state(i915);
1376413909 }
1376513910
13766−static void intel_atomic_commit_tail(struct drm_atomic_state *state)
13911+static void intel_atomic_commit_tail(struct intel_atomic_state *state)
1376713912 {
13768− struct drm_device *dev = state->dev;
13769− struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
13913+ struct drm_device *dev = state->base.dev;
1377013914 struct drm_i915_private *dev_priv = to_i915(dev);
13771− struct drm_crtc_state *old_crtc_state, *new_crtc_state;
13772− struct intel_crtc_state *new_intel_crtc_state, *old_intel_crtc_state;
13773− struct drm_crtc *crtc;
13774− struct intel_crtc *intel_crtc;
13915+ struct intel_crtc_state *new_crtc_state, *old_crtc_state;
13916+ struct intel_crtc *crtc;
1377513917 u64 put_domains[I915_MAX_PIPES] = {};
1377613918 intel_wakeref_t wakeref = 0;
1377713919 int i;
1377813920
13779− intel_atomic_commit_fence_wait(intel_state);
13921+ intel_atomic_commit_fence_wait(state);
1378013922
13781− drm_atomic_helper_wait_for_dependencies(state);
13923+ drm_atomic_helper_wait_for_dependencies(&state->base);
1378213924
13783− if (intel_state->modeset)
13925+ if (state->modeset)
1378413926 wakeref = intel_display_power_get(dev_priv, POWER_DOMAIN_MODESET);
1378513927
13786− for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
13787− old_intel_crtc_state = to_intel_crtc_state(old_crtc_state);
13788− new_intel_crtc_state = to_intel_crtc_state(new_crtc_state);
13789− intel_crtc = to_intel_crtc(crtc);
13790−
13928+ for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
1379113929 if (needs_modeset(new_crtc_state) ||
13792− to_intel_crtc_state(new_crtc_state)->update_pipe) {
13930+ new_crtc_state->update_pipe) {
1379313931
13794− put_domains[intel_crtc->pipe] =
13795− modeset_get_crtc_power_domains(crtc,
13796− new_intel_crtc_state);
13932+ put_domains[crtc->pipe] =
13933+ modeset_get_crtc_power_domains(new_crtc_state);
1379713934 }
1379813935
1379913936 if (!needs_modeset(new_crtc_state))
1380013937 continue;
1380113938
13802− intel_pre_plane_update(old_intel_crtc_state, new_intel_crtc_state);
13939+ intel_pre_plane_update(old_crtc_state, new_crtc_state);
1380313940
13804− if (old_crtc_state->active) {
13805− intel_crtc_disable_planes(intel_state, intel_crtc);
13941+ if (old_crtc_state->base.active) {
13942+ intel_crtc_disable_planes(state, crtc);
1380613943
1380713944 /*
1380813945 * We need to disable pipe CRC before disabling the pipe,
1380913946 * or we race against vblank off.
1381013947 */
13811− intel_crtc_disable_pipe_crc(intel_crtc);
13948+ intel_crtc_disable_pipe_crc(crtc);
1381213949
13813− dev_priv->display.crtc_disable(old_intel_crtc_state, state);
13814− intel_crtc->active = false;
13815− intel_fbc_disable(intel_crtc);
13816− intel_disable_shared_dpll(old_intel_crtc_state);
13950+ dev_priv->display.crtc_disable(old_crtc_state, state);
13951+ crtc->active = false;
13952+ intel_fbc_disable(crtc);
13953+ intel_disable_shared_dpll(old_crtc_state);
1381713954
1381813955 /*
1381913956 * Underruns don't always raise
@@ -13823,25 +13960,25 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state) · 3 unmodified lines
1382313960 intel_check_pch_fifo_underruns(dev_priv);
1382413961
1382513962 /* FIXME unify this for all platforms */
13826− if (!new_crtc_state->active &&
13963+ if (!new_crtc_state->base.active &&
1382713964 !HAS_GMCH(dev_priv) &&
1382813965 dev_priv->display.initial_watermarks)
13829− dev_priv->display.initial_watermarks(intel_state,
13830− new_intel_crtc_state);
13966+ dev_priv->display.initial_watermarks(state,
13967+ new_crtc_state);
1383113968 }
1383213969 }
1383313970
13834− /* FIXME: Eventually get rid of our intel_crtc->config pointer */
13835− for_each_new_crtc_in_state(state, crtc, new_crtc_state, i)
13836− to_intel_crtc(crtc)->config = to_intel_crtc_state(new_crtc_state);
13971+ /* FIXME: Eventually get rid of our crtc->config pointer */
13972+ for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i)
13973+ crtc->config = new_crtc_state;
1383713974
13838− if (intel_state->modeset) {
13839− drm_atomic_helper_update_legacy_modeset_state(state->dev, state);
13975+ if (state->modeset) {
13976+ drm_atomic_helper_update_legacy_modeset_state(dev, &state->base);
1384013977
1384113978 intel_set_cdclk_pre_plane_update(dev_priv,
13842− &intel_state->cdclk.actual,
13979+ &state->cdclk.actual,
1384313980 &dev_priv->cdclk.actual,
13844− intel_state->cdclk.pipe);
13981+ state->cdclk.pipe);
1384513982
1384613983 /*
1384713984 * SKL workaround: bspec recommends we disable the SAGV when we
@@ -13850,31 +13987,37 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state) · 2 unmodified lines
1385013987 if (!intel_can_enable_sagv(state))
1385113988 intel_disable_sagv(dev_priv);
1385213989
13853− intel_modeset_verify_disabled(dev, state);
13990+ intel_modeset_verify_disabled(dev_priv, state);
1385413991 }
1385513992
1385613993 /* Complete the events for pipes that have now been disabled */
13857− for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
13994+ for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {
1385813995 bool modeset = needs_modeset(new_crtc_state);
1385913996
1386013997 /* Complete events for now disable pipes here. */
13861− if (modeset && !new_crtc_state->active && new_crtc_state->event) {
13998+ if (modeset && !new_crtc_state->base.active && new_crtc_state->base.event) {
1386213999 spin_lock_irq(&dev->event_lock);
13863− drm_crtc_send_vblank_event(crtc, new_crtc_state->event);
14000+ drm_crtc_send_vblank_event(&crtc->base, new_crtc_state->base.event);
1386414001 spin_unlock_irq(&dev->event_lock);
1386514002
13866− new_crtc_state->event = NULL;
14003+ new_crtc_state->base.event = NULL;
1386714004 }
1386814005 }
1386914006
14007+ if (state->modeset)
14008+ intel_encoders_update_prepare(state);
14009+
1387014010 /* Now enable the clocks, plane, pipe, and connectors that we set up. */
1387114011 dev_priv->display.update_crtcs(state);
1387214012
13873− if (intel_state->modeset)
14013+ if (state->modeset) {
14014+ intel_encoders_update_complete(state);
14015+
1387414016 intel_set_cdclk_post_plane_update(dev_priv,
13875− &intel_state->cdclk.actual,
14017+ &state->cdclk.actual,
1387614018 &dev_priv->cdclk.actual,
13877− intel_state->cdclk.pipe);
14019+ state->cdclk.pipe);
14020+ }
1387814021
1387914022 /* FIXME: We should call drm_atomic_helper_commit_hw_done() here
1388014023 * already, but still need the state for the delayed optimization. To
@@ -13885,16 +14028,14 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state) · 4 unmodified lines
1388514028 * - switch over to the vblank wait helper in the core after that since
1388614029 * we don't need out special handling any more.
1388714030 */
13888− drm_atomic_helper_wait_for_flip_done(dev, state);
14031+ drm_atomic_helper_wait_for_flip_done(dev, &state->base);
1388914032
13890− for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
13891− new_intel_crtc_state = to_intel_crtc_state(new_crtc_state);
13892−
13893− if (new_crtc_state->active &&
14033+ for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {
14034+ if (new_crtc_state->base.active &&
1389414035 !needs_modeset(new_crtc_state) &&
13895− (new_intel_crtc_state->base.color_mgmt_changed ||
13896− new_intel_crtc_state->update_pipe))
13897− intel_color_load_luts(new_intel_crtc_state);
14036+ (new_crtc_state->base.color_mgmt_changed ||
14037+ new_crtc_state->update_pipe))
14038+ intel_color_load_luts(new_crtc_state);
1389814039 }
1389914040
1390014041 /*
@@ -13904,16 +14045,14 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state) · 3 unmodified lines
1390414045 *
1390514046 * TODO: Move this (and other cleanup) to an async worker eventually.
1390614047 */
13907− for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
13908− new_intel_crtc_state = to_intel_crtc_state(new_crtc_state);
13909−
14048+ for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {
1391014049 if (dev_priv->display.optimize_watermarks)
13911− dev_priv->display.optimize_watermarks(intel_state,
13912− new_intel_crtc_state);
14050+ dev_priv->display.optimize_watermarks(state,
14051+ new_crtc_state);
1391314052 }
1391414053
13915− for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
13916− intel_post_plane_update(to_intel_crtc_state(old_crtc_state));
14054+ for_each_oldnew_intel_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
14055+ intel_post_plane_update(old_crtc_state);
1391714056
1391814057 if (put_domains[i])
1391914058 modeset_put_power_domains(dev_priv, put_domains[i]);
@@ -13921,15 +14060,15 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state) · 1 unmodified line
1392114060 intel_modeset_verify_crtc(crtc, state, old_crtc_state, new_crtc_state);
1392214061 }
1392314062
13924− if (intel_state->modeset)
13925− intel_verify_planes(intel_state);
14063+ if (state->modeset)
14064+ intel_verify_planes(state);
1392614065
13927− if (intel_state->modeset && intel_can_enable_sagv(state))
14066+ if (state->modeset && intel_can_enable_sagv(state))
1392814067 intel_enable_sagv(dev_priv);
1392914068
13930− drm_atomic_helper_commit_hw_done(state);
14069+ drm_atomic_helper_commit_hw_done(&state->base);
1393114070
13932− if (intel_state->modeset) {
14071+ if (state->modeset) {
1393314072 /* As one of the primary mmio accessors, KMS has a high
1393414073 * likelihood of triggering bugs in unclaimed access. After we
1393514074 * finish modesetting, see if an error has been flagged, and if
@@ -13939,7 +14078,7 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state) · 3 unmodified lines
1393914078 intel_uncore_arm_unclaimed_mmio_detection(&dev_priv->uncore);
1394014079 intel_display_power_put(dev_priv, POWER_DOMAIN_MODESET, wakeref);
1394114080 }
13942− intel_runtime_pm_put(&dev_priv->runtime_pm, intel_state->wakeref);
14081+ intel_runtime_pm_put(&dev_priv->runtime_pm, state->wakeref);
1394314082
1394414083 /*
1394514084 * Defer the cleanup of the old state to a separate worker to not
@@ -13949,14 +14088,14 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state) · 3 unmodified lines
1394914088 * schedule point (cond_resched()) here anyway to keep latencies
1395014089 * down.
1395114090 */
13952− INIT_WORK(&state->commit_work, intel_atomic_cleanup_work);
13953− queue_work(system_highpri_wq, &state->commit_work);
14091+ INIT_WORK(&state->base.commit_work, intel_atomic_cleanup_work);
14092+ queue_work(system_highpri_wq, &state->base.commit_work);
1395414093 }
1395514094
1395614095 static void intel_atomic_commit_work(struct work_struct *work)
1395714096 {
13958− struct drm_atomic_state *state =
13959− container_of(work, struct drm_atomic_state, commit_work);
14097+ struct intel_atomic_state *state =
14098+ container_of(work, struct intel_atomic_state, base.commit_work);
1396014099
1396114100 intel_atomic_commit_tail(state);
1396214101 }
@@ -13986,42 +14125,31 @@ intel_atomic_commit_ready(struct i915_sw_fence *fence, · 23 unmodified lines
1398614125 return NOTIFY_DONE;
1398714126 }
1398814127
13989−static void intel_atomic_track_fbs(struct drm_atomic_state *state)
14128+static void intel_atomic_track_fbs(struct intel_atomic_state *state)
1399014129 {
13991− struct drm_plane_state *old_plane_state, *new_plane_state;
13992− struct drm_plane *plane;
14130+ struct intel_plane_state *old_plane_state, *new_plane_state;
14131+ struct intel_plane *plane;
1399314132 int i;
1399414133
13995− for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i)
13996− i915_gem_track_fb(intel_fb_obj(old_plane_state->fb),
13997− intel_fb_obj(new_plane_state->fb),
13998− to_intel_plane(plane)->frontbuffer_bit);
14134+ for_each_oldnew_intel_plane_in_state(state, plane, old_plane_state,
14135+ new_plane_state, i)
14136+ i915_gem_track_fb(intel_fb_obj(old_plane_state->base.fb),
14137+ intel_fb_obj(new_plane_state->base.fb),
14138+ plane->frontbuffer_bit);
1399914139 }
1400014140
14001−/**
14002− * intel_atomic_commit - commit validated state object
14003− * @dev: DRM device
14004− * @state: the top-level driver state object
14005− * @nonblock: nonblocking commit
14006− *
14007− * This function commits a top-level state object that has been validated
14008− * with drm_atomic_helper_check().
14009− *
14010− * RETURNS
14011− * Zero for success or -errno.
14012− */
1401314141 static int intel_atomic_commit(struct drm_device *dev,
14014− struct drm_atomic_state *state,
14142+ struct drm_atomic_state *_state,
1401514143 bool nonblock)
1401614144 {
14017− struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
14145+ struct intel_atomic_state *state = to_intel_atomic_state(_state);
1401814146 struct drm_i915_private *dev_priv = to_i915(dev);
1401914147 int ret = 0;
1402014148
14021− intel_state->wakeref = intel_runtime_pm_get(&dev_priv->runtime_pm);
14149+ state->wakeref = intel_runtime_pm_get(&dev_priv->runtime_pm);
1402214150
14023− drm_atomic_state_get(state);
14024− i915_sw_fence_init(&intel_state->commit_ready,
14151+ drm_atomic_state_get(&state->base);
14152+ i915_sw_fence_init(&state->commit_ready,
1402514153 intel_atomic_commit_ready);
1402614154
1402714155 /*
@@ -14041,63 +14169,61 @@ static int intel_atomic_commit(struct drm_device *dev, · 13 unmodified lines
1404114169 * FIXME doing watermarks and fb cleanup from a vblank worker
1404214170 * (assuming we had any) would solve these problems.
1404314171 */
14044− if (INTEL_GEN(dev_priv) < 9 && state->legacy_cursor_update) {
14172+ if (INTEL_GEN(dev_priv) < 9 && state->base.legacy_cursor_update) {
1404514173 struct intel_crtc_state *new_crtc_state;
1404614174 struct intel_crtc *crtc;
1404714175 int i;
1404814176
14049− for_each_new_intel_crtc_in_state(intel_state, crtc, new_crtc_state, i)
14177+ for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i)
1405014178 if (new_crtc_state->wm.need_postvbl_update ||
1405114179 new_crtc_state->update_wm_post)
14052− state->legacy_cursor_update = false;
14180+ state->base.legacy_cursor_update = false;
1405314181 }
1405414182
14055− ret = intel_atomic_prepare_commit(dev, state);
14183+ ret = intel_atomic_prepare_commit(state);
1405614184 if (ret) {
1405714185 DRM_DEBUG_ATOMIC("Preparing state failed with %i\n", ret);
14058− i915_sw_fence_commit(&intel_state->commit_ready);
14059− intel_runtime_pm_put(&dev_priv->runtime_pm, intel_state->wakeref);
14186+ i915_sw_fence_commit(&state->commit_ready);
14187+ intel_runtime_pm_put(&dev_priv->runtime_pm, state->wakeref);
1406014188 return ret;
1406114189 }
1406214190
14063− ret = drm_atomic_helper_setup_commit(state, nonblock);
14191+ ret = drm_atomic_helper_setup_commit(&state->base, nonblock);
1406414192 if (!ret)
14065− ret = drm_atomic_helper_swap_state(state, true);
14193+ ret = drm_atomic_helper_swap_state(&state->base, true);
1406614194
1406714195 if (ret) {
14068− i915_sw_fence_commit(&intel_state->commit_ready);
14196+ i915_sw_fence_commit(&state->commit_ready);
1406914197
14070− drm_atomic_helper_cleanup_planes(dev, state);
14071− intel_runtime_pm_put(&dev_priv->runtime_pm, intel_state->wakeref);
14198+ drm_atomic_helper_cleanup_planes(dev, &state->base);
14199+ intel_runtime_pm_put(&dev_priv->runtime_pm, state->wakeref);
1407214200 return ret;
1407314201 }
1407414202 dev_priv->wm.distrust_bios_wm = false;
1407514203 intel_shared_dpll_swap_state(state);
1407614204 intel_atomic_track_fbs(state);
1407714205
14078− if (intel_state->modeset) {
14079− memcpy(dev_priv->min_cdclk, intel_state->min_cdclk,
14080− sizeof(intel_state->min_cdclk));
14081− memcpy(dev_priv->min_voltage_level,
14082− intel_state->min_voltage_level,
14083− sizeof(intel_state->min_voltage_level));
14084− dev_priv->active_crtcs = intel_state->active_crtcs;
14085− dev_priv->cdclk.force_min_cdclk =
14086− intel_state->cdclk.force_min_cdclk;
14206+ if (state->modeset) {
14207+ memcpy(dev_priv->min_cdclk, state->min_cdclk,
14208+ sizeof(state->min_cdclk));
14209+ memcpy(dev_priv->min_voltage_level, state->min_voltage_level,
14210+ sizeof(state->min_voltage_level));
14211+ dev_priv->active_crtcs = state->active_crtcs;
14212+ dev_priv->cdclk.force_min_cdclk = state->cdclk.force_min_cdclk;
1408714213
14088− intel_cdclk_swap_state(intel_state);
14214+ intel_cdclk_swap_state(state);
1408914215 }
1409014216
14091− drm_atomic_state_get(state);
14092− INIT_WORK(&state->commit_work, intel_atomic_commit_work);
14217+ drm_atomic_state_get(&state->base);
14218+ INIT_WORK(&state->base.commit_work, intel_atomic_commit_work);
1409314219
14094− i915_sw_fence_commit(&intel_state->commit_ready);
14095− if (nonblock && intel_state->modeset) {
14096− queue_work(dev_priv->modeset_wq, &state->commit_work);
14220+ i915_sw_fence_commit(&state->commit_ready);
14221+ if (nonblock && state->modeset) {
14222+ queue_work(dev_priv->modeset_wq, &state->base.commit_work);
1409714223 } else if (nonblock) {
14098− queue_work(system_unbound_wq, &state->commit_work);
14224+ queue_work(system_unbound_wq, &state->base.commit_work);
1409914225 } else {
14100− if (intel_state->modeset)
14226+ if (state->modeset)
1410114227 flush_workqueue(dev_priv->modeset_wq);
1410214228 intel_atomic_commit_tail(state);
1410314229 }
@@ -14105,18 +14231,6 @@ static int intel_atomic_commit(struct drm_device *dev, · 1 unmodified line
1410514231 return 0;
1410614232 }
1410714233
14108−static const struct drm_crtc_funcs intel_crtc_funcs = {
14109− .gamma_set = drm_atomic_helper_legacy_gamma_set,
14110− .set_config = drm_atomic_helper_set_config,
14111− .destroy = intel_crtc_destroy,
14112− .page_flip = drm_atomic_helper_page_flip,
14113− .atomic_duplicate_state = intel_crtc_duplicate_state,
14114− .atomic_destroy_state = intel_crtc_destroy_state,
14115− .set_crc_source = intel_crtc_set_crc_source,
14116− .verify_crc_source = intel_crtc_verify_crc_source,
14117− .get_crc_sources = intel_crtc_get_crc_sources,
14118−};
14119−
1412014234 struct wait_rps_boost {
1412114235 struct wait_queue_entry wait;
1412214236
@@ -14250,9 +14364,9 @@ intel_prepare_plane_fb(struct drm_plane *plane, · 127 unmodified lines
1425014364 int ret;
1425114365
1425214366 if (old_obj) {
14253− struct drm_crtc_state *crtc_state =
14254− drm_atomic_get_new_crtc_state(new_state->state,
14255− plane->state->crtc);
14367+ struct intel_crtc_state *crtc_state =
14368+ intel_atomic_get_new_crtc_state(intel_state,
14369+ to_intel_crtc(plane->state->crtc));
1425614370
1425714371 /* Big Hammer, we also need to ensure that any pending
1425814372 * MI_WAIT_FOR_EVENT inside a user batch buffer on the
@@ -14413,7 +14527,7 @@ static void intel_begin_crtc_commit(struct intel_atomic_state *state, · 154 unmodified lines
1441314527 intel_atomic_get_old_crtc_state(state, crtc);
1441414528 struct intel_crtc_state *new_crtc_state =
1441514529 intel_atomic_get_new_crtc_state(state, crtc);
14416− bool modeset = needs_modeset(&new_crtc_state->base);
14530+ bool modeset = needs_modeset(new_crtc_state);
1441714531
1441814532 /* Perform vblank evasion around commit operation */
1441914533 intel_pipe_update_start(new_crtc_state);
@@ -14466,7 +14580,7 @@ static void intel_finish_crtc_commit(struct intel_atomic_state *state, · 46 unmodified lines
1446614580 intel_pipe_update_end(new_crtc_state);
1446714581
1446814582 if (new_crtc_state->update_pipe &&
14469− !needs_modeset(&new_crtc_state->base) &&
14583+ !needs_modeset(new_crtc_state) &&
1447014584 old_crtc_state->base.mode.private_flags & I915_MODE_FLAG_INHERITED)
1447114585 intel_crtc_arm_fifo_underrun(crtc, new_crtc_state);
1447214586 }
@@ -14580,7 +14694,7 @@ intel_legacy_cursor_update(struct drm_plane *plane, · 107 unmodified lines
1458014694 * When crtc is inactive or there is a modeset pending,
1458114695 * wait for it to complete in the slowpath
1458214696 */
14583− if (!crtc_state->base.active || needs_modeset(&crtc_state->base) ||
14697+ if (!crtc_state->base.active || needs_modeset(crtc_state) ||
1458414698 crtc_state->update_pipe)
1458514699 goto slow;
1458614700
@@ -14910,8 +15024,76 @@ static void intel_crtc_init_scalers(struct intel_crtc *crtc, · 323 unmodified lines
1491015024 scaler_state->scaler_id = -1;
1491115025 }
1491215026
15027+#define INTEL_CRTC_FUNCS \
15028+ .gamma_set = drm_atomic_helper_legacy_gamma_set, \
15029+ .set_config = drm_atomic_helper_set_config, \
15030+ .destroy = intel_crtc_destroy, \
15031+ .page_flip = drm_atomic_helper_page_flip, \
15032+ .atomic_duplicate_state = intel_crtc_duplicate_state, \
15033+ .atomic_destroy_state = intel_crtc_destroy_state, \
15034+ .set_crc_source = intel_crtc_set_crc_source, \
15035+ .verify_crc_source = intel_crtc_verify_crc_source, \
15036+ .get_crc_sources = intel_crtc_get_crc_sources
15037+
15038+static const struct drm_crtc_funcs bdw_crtc_funcs = {
15039+ INTEL_CRTC_FUNCS,
15040+
15041+ .get_vblank_counter = g4x_get_vblank_counter,
15042+ .enable_vblank = bdw_enable_vblank,
15043+ .disable_vblank = bdw_disable_vblank,
15044+};
15045+
15046+static const struct drm_crtc_funcs ilk_crtc_funcs = {
15047+ INTEL_CRTC_FUNCS,
15048+
15049+ .get_vblank_counter = g4x_get_vblank_counter,
15050+ .enable_vblank = ilk_enable_vblank,
15051+ .disable_vblank = ilk_disable_vblank,
15052+};
15053+
15054+static const struct drm_crtc_funcs g4x_crtc_funcs = {
15055+ INTEL_CRTC_FUNCS,
15056+
15057+ .get_vblank_counter = g4x_get_vblank_counter,
15058+ .enable_vblank = i965_enable_vblank,
15059+ .disable_vblank = i965_disable_vblank,
15060+};
15061+
15062+static const struct drm_crtc_funcs i965_crtc_funcs = {
15063+ INTEL_CRTC_FUNCS,
15064+
15065+ .get_vblank_counter = i915_get_vblank_counter,
15066+ .enable_vblank = i965_enable_vblank,
15067+ .disable_vblank = i965_disable_vblank,
15068+};
15069+
15070+static const struct drm_crtc_funcs i945gm_crtc_funcs = {
15071+ INTEL_CRTC_FUNCS,
15072+
15073+ .get_vblank_counter = i915_get_vblank_counter,
15074+ .enable_vblank = i945gm_enable_vblank,
15075+ .disable_vblank = i945gm_disable_vblank,
15076+};
15077+
15078+static const struct drm_crtc_funcs i915_crtc_funcs = {
15079+ INTEL_CRTC_FUNCS,
15080+
15081+ .get_vblank_counter = i915_get_vblank_counter,
15082+ .enable_vblank = i8xx_enable_vblank,
15083+ .disable_vblank = i8xx_disable_vblank,
15084+};
15085+
15086+static const struct drm_crtc_funcs i8xx_crtc_funcs = {
15087+ INTEL_CRTC_FUNCS,
15088+
15089+ /* no hw vblank counter */
15090+ .enable_vblank = i8xx_enable_vblank,
15091+ .disable_vblank = i8xx_disable_vblank,
15092+};
15093+
1491315094 static int intel_crtc_init(struct drm_i915_private *dev_priv, enum pipe pipe)
1491415095 {
15096+ const struct drm_crtc_funcs *funcs;
1491515097 struct intel_crtc *intel_crtc;
1491615098 struct intel_crtc_state *crtc_state = NULL;
1491715099 struct intel_plane *primary = NULL;
@@ -14955,10 +15137,28 @@ static int intel_crtc_init(struct drm_i915_private *dev_priv, enum pipe pipe) · 37 unmodified lines
1495515137 }
1495615138 intel_crtc->plane_ids_mask |= BIT(cursor->id);
1495715139
15140+ if (HAS_GMCH(dev_priv)) {
15141+ if (IS_CHERRYVIEW(dev_priv) ||
15142+ IS_VALLEYVIEW(dev_priv) || IS_G4X(dev_priv))
15143+ funcs = &g4x_crtc_funcs;
15144+ else if (IS_GEN(dev_priv, 4))
15145+ funcs = &i965_crtc_funcs;
15146+ else if (IS_I945GM(dev_priv))
15147+ funcs = &i945gm_crtc_funcs;
15148+ else if (IS_GEN(dev_priv, 3))
15149+ funcs = &i915_crtc_funcs;
15150+ else
15151+ funcs = &i8xx_crtc_funcs;
15152+ } else {
15153+ if (INTEL_GEN(dev_priv) >= 8)
15154+ funcs = &bdw_crtc_funcs;
15155+ else
15156+ funcs = &ilk_crtc_funcs;
15157+ }
15158+
1495815159 ret = drm_crtc_init_with_planes(&dev_priv->drm, &intel_crtc->base,
1495915160 &primary->base, &cursor->base,
14960− &intel_crtc_funcs,
14961− "pipe %c", pipe_name(pipe));
15161+ funcs, "pipe %c", pipe_name(pipe));
1496215162 if (ret)
1496315163 goto fail;
1496415164
@@ -15114,12 +15314,18 @@ static void intel_setup_outputs(struct drm_i915_private *dev_priv) · 149 unmodified lines
1511415314 if (!HAS_DISPLAY(dev_priv))
1511515315 return;
1511615316
15117− if (IS_ELKHARTLAKE(dev_priv)) {
15317+ if (INTEL_GEN(dev_priv) >= 12) {
15318+ /* TODO: initialize TC ports as well */
15319+ intel_ddi_init(dev_priv, PORT_A);
15320+ intel_ddi_init(dev_priv, PORT_B);
15321+ intel_ddi_init(dev_priv, PORT_C);
15322+ } else if (IS_ELKHARTLAKE(dev_priv)) {
1511815323 intel_ddi_init(dev_priv, PORT_A);
1511915324 intel_ddi_init(dev_priv, PORT_B);
1512015325 intel_ddi_init(dev_priv, PORT_C);
15326+ intel_ddi_init(dev_priv, PORT_D);
1512115327 icl_dsi_init(dev_priv);
15122− } else if (INTEL_GEN(dev_priv) >= 11) {
15328+ } else if (IS_GEN(dev_priv, 11)) {
1512315329 intel_ddi_init(dev_priv, PORT_A);
1512415330 intel_ddi_init(dev_priv, PORT_B);
1512515331 intel_ddi_init(dev_priv, PORT_C);
@@ -15775,8 +15981,8 @@ static void sanitize_watermarks(struct drm_device *dev) · 649 unmodified lines
1577515981 struct drm_i915_private *dev_priv = to_i915(dev);
1577615982 struct drm_atomic_state *state;
1577715983 struct intel_atomic_state *intel_state;
15778− struct drm_crtc *crtc;
15779− struct drm_crtc_state *cstate;
15984+ struct intel_crtc *crtc;
15985+ struct intel_crtc_state *crtc_state;
1578015986 struct drm_modeset_acquire_ctx ctx;
1578115987 int ret;
1578215988 int i;
@@ -15831,13 +16037,11 @@ retry: · 48 unmodified lines
1583116037 }
1583216038
1583316039 /* Write calculated watermark values back */
15834− for_each_new_crtc_in_state(state, crtc, cstate, i) {
15835− struct intel_crtc_state *cs = to_intel_crtc_state(cstate);
15836−
15837− cs->wm.need_postvbl_update = true;
15838− dev_priv->display.optimize_watermarks(intel_state, cs);
16040+ for_each_new_intel_crtc_in_state(intel_state, crtc, crtc_state, i) {
16041+ crtc_state->wm.need_postvbl_update = true;
16042+ dev_priv->display.optimize_watermarks(intel_state, crtc_state);
1583916043
15840− to_intel_crtc_state(crtc->state)->wm = cs->wm;
16044+ to_intel_crtc_state(crtc->base.state)->wm = crtc_state->wm;
1584116045 }
1584216046
1584316047 put_state:
@@ -16495,6 +16699,13 @@ static void intel_modeset_readout_hw_state(struct drm_device *dev) · 651 unmodified lines
1649516699
1649616700 pll->on = pll->info->funcs->get_hw_state(dev_priv, pll,
1649716701 &pll->state.hw_state);
16702+
16703+ if (IS_ELKHARTLAKE(dev_priv) && pll->on &&
16704+ pll->info->id == DPLL_ID_EHL_DPLL4) {
16705+ pll->wakeref = intel_display_power_get(dev_priv,
16706+ POWER_DOMAIN_DPLL_DC_OFF);
16707+ }
16708+
1649816709 pll->state.crtc_mask = 0;
1649916710 for_each_intel_crtc(dev, crtc) {
1650016711 struct intel_crtc_state *crtc_state =
@@ -16744,6 +16955,17 @@ intel_modeset_setup_hw_state(struct drm_device *dev, · 243 unmodified lines
1674416955 intel_modeset_readout_hw_state(dev);
1674516956
1674616957 /* HW state is read out, now we need to sanitize this mess. */
16958+
16959+ /* Sanitize the TypeC port mode upfront, encoders depend on this */
16960+ for_each_intel_encoder(dev, encoder) {
16961+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
16962+
16963+ /* We need to sanitize only the MST primary port. */
16964+ if (encoder->type != INTEL_OUTPUT_DP_MST &&
16965+ intel_phy_is_tc(dev_priv, phy))
16966+ intel_tc_port_sanitize(enc_to_dig_port(&encoder->base));
16967+ }
16968+
1674716969 get_encoder_power_domains(dev_priv);
1674816970
1674916971 if (HAS_PCH_IBX(dev_priv))
@@ -16804,7 +17026,7 @@ intel_modeset_setup_hw_state(struct drm_device *dev, · 54 unmodified lines
1680417026 u64 put_domains;
1680517027
1680617028 crtc_state = to_intel_crtc_state(crtc->base.state);
16807− put_domains = modeset_get_crtc_power_domains(&crtc->base, crtc_state);
17029+ put_domains = modeset_get_crtc_power_domains(crtc_state);
1680817030 if (WARN_ON(put_domains))
1680917031 modeset_put_power_domains(dev_priv, put_domains);
1681017032 }
@@ -16866,7 +17088,7 @@ static void intel_hpd_poll_fini(struct drm_device *dev) · 55 unmodified lines
1686617088 drm_connector_list_iter_end(&conn_iter);
1686717089 }
1686817090
16869−void intel_modeset_cleanup(struct drm_device *dev)
17091+void intel_modeset_driver_remove(struct drm_device *dev)
1687017092 {
1687117093 struct drm_i915_private *dev_priv = to_i915(dev);
1687217094
@@ -16982,7 +17204,7 @@ struct intel_display_error_state { · 109 unmodified lines
1698217204 u32 vtotal;
1698317205 u32 vblank;
1698417206 u32 vsync;
16985− } transcoder[4];
17207+ } transcoder[5];
1698617208 };
1698717209
1698817210 struct intel_display_error_state *
@@ -16993,6 +17215,7 @@ intel_display_capture_error_state(struct drm_i915_private *dev_priv) · 4 unmodified lines
1699317215 TRANSCODER_A,
1699417216 TRANSCODER_B,
1699517217 TRANSCODER_C,
17218+ TRANSCODER_D,
1699617219 TRANSCODER_EDP,
1699717220 };
1699817221 int i;
drivers/gpu/drm/i915/display/intel_display.h+46−4
@@ -45,6 +45,8 @@ enum i915_gpio { · 44 unmodified lines
4545 GPIOK,
4646 GPIOL,
4747 GPIOM,
48+ GPION,
49+ GPIOO,
4850 };
4951
5052 /*
@@ -58,6 +60,7 @@ enum pipe { · 7 unmodified lines
5860 PIPE_A = 0,
5961 PIPE_B,
6062 PIPE_C,
63+ PIPE_D,
6164 _PIPE_EDP,
6265
6366 I915_MAX_PIPES = _PIPE_EDP
@@ -75,6 +78,7 @@ enum transcoder { · 11 unmodified lines
7578 TRANSCODER_A = PIPE_A,
7679 TRANSCODER_B = PIPE_B,
7780 TRANSCODER_C = PIPE_C,
81+ TRANSCODER_D = PIPE_D,
7882
7983 /*
8084 * The following transcoders can map to any pipe, their enum value
@@ -98,6 +102,8 @@ static inline const char *transcoder_name(enum transcoder transcoder) · 17 unmodified lines
98102 return "B";
99103 case TRANSCODER_C:
100104 return "C";
105+ case TRANSCODER_D:
106+ return "D";
101107 case TRANSCODER_EDP:
102108 return "EDP";
103109 case TRANSCODER_DSI_A:
@@ -173,6 +179,12 @@ static inline const char *port_identifier(enum port port) · 69 unmodified lines
173179 return "Port E";
174180 case PORT_F:
175181 return "Port F";
182+ case PORT_G:
183+ return "Port G";
184+ case PORT_H:
185+ return "Port H";
186+ case PORT_I:
187+ return "Port I";
176188 default:
177189 return "<invalid>";
178190 }
@@ -185,14 +197,15 @@ enum tc_port { · 6 unmodified lines
185197 PORT_TC2,
186198 PORT_TC3,
187199 PORT_TC4,
200+ PORT_TC5,
201+ PORT_TC6,
188202
189203 I915_MAX_TC_PORTS
190204 };
191205
192−enum tc_port_type {
193− TC_PORT_UNKNOWN = 0,
194− TC_PORT_TYPEC,
195− TC_PORT_TBT,
206+enum tc_port_mode {
207+ TC_PORT_TBT_ALT,
208+ TC_PORT_DP_ALT,
196209 TC_PORT_LEGACY,
197210 };
198211
@@ -229,6 +242,30 @@ struct intel_link_m_n { · 30 unmodified lines
229242 u32 link_n;
230243 };
231244
245+enum phy {
246+ PHY_NONE = -1,
247+
248+ PHY_A = 0,
249+ PHY_B,
250+ PHY_C,
251+ PHY_D,
252+ PHY_E,
253+ PHY_F,
254+ PHY_G,
255+ PHY_H,
256+ PHY_I,
257+
258+ I915_MAX_PHYS
259+};
260+
261+#define phy_name(a) ((a) + 'A')
262+
263+enum phy_fia {
264+ FIA1,
265+ FIA2,
266+ FIA3,
267+};
268+
232269 #define for_each_pipe(__dev_priv, __p) \
233270 for ((__p) = 0; (__p) < INTEL_INFO(__dev_priv)->num_pipes; (__p)++)
234271
@@ -254,6 +291,10 @@ struct intel_link_m_n { · 19 unmodified lines
254291 for ((__port) = PORT_A; (__port) < I915_MAX_PORTS; (__port)++) \
255292 for_each_if((__ports_mask) & BIT(__port))
256293
294+#define for_each_phy_masked(__phy, __phys_mask) \
295+ for ((__phy) = PHY_A; (__phy) < I915_MAX_PHYS; (__phy)++) \
296+ for_each_if((__phys_mask) & BIT(__phy))
297+
257298 #define for_each_crtc(dev, crtc) \
258299 list_for_each_entry(crtc, &(dev)->mode_config.crtc_list, head)
259300
@@ -357,5 +398,6 @@ void lpt_disable_clkout_dp(struct drm_i915_private *dev_priv); · 97 unmodified lines
357398 u32 intel_plane_fb_max_stride(struct drm_i915_private *dev_priv,
358399 u32 pixel_format, u64 modifier);
359400 bool intel_plane_can_remap(const struct intel_plane_state *plane_state);
401+enum phy intel_port_to_phy(struct drm_i915_private *i915, enum port port);
360402
361403 #endif
drivers/gpu/drm/i915/display/intel_display_power.c+614−44
@@ -17,13 +17,17 @@ · 16 unmodified lines
1717 #include "intel_drv.h"
1818 #include "intel_hotplug.h"
1919 #include "intel_sideband.h"
20+#include "intel_tc.h"
2021
2122 bool intel_display_power_well_is_enabled(struct drm_i915_private *dev_priv,
2223 enum i915_power_well_id power_well_id);
2324
2425 const char *
25−intel_display_power_domain_str(enum intel_display_power_domain domain)
26+intel_display_power_domain_str(struct drm_i915_private *i915,
27+ enum intel_display_power_domain domain)
2628 {
29+ bool ddi_tc_ports = IS_GEN(i915, 12);
30+
2731 switch (domain) {
2832 case POWER_DOMAIN_DISPLAY_CORE:
2933 return "DISPLAY_CORE";
@@ -33,22 +37,28 @@ intel_display_power_domain_str(enum intel_display_power_domain domain) · 3 unmodified lines
3337 return "PIPE_B";
3438 case POWER_DOMAIN_PIPE_C:
3539 return "PIPE_C";
40+ case POWER_DOMAIN_PIPE_D:
41+ return "PIPE_D";
3642 case POWER_DOMAIN_PIPE_A_PANEL_FITTER:
3743 return "PIPE_A_PANEL_FITTER";
3844 case POWER_DOMAIN_PIPE_B_PANEL_FITTER:
3945 return "PIPE_B_PANEL_FITTER";
4046 case POWER_DOMAIN_PIPE_C_PANEL_FITTER:
4147 return "PIPE_C_PANEL_FITTER";
48+ case POWER_DOMAIN_PIPE_D_PANEL_FITTER:
49+ return "PIPE_D_PANEL_FITTER";
4250 case POWER_DOMAIN_TRANSCODER_A:
4351 return "TRANSCODER_A";
4452 case POWER_DOMAIN_TRANSCODER_B:
4553 return "TRANSCODER_B";
4654 case POWER_DOMAIN_TRANSCODER_C:
4755 return "TRANSCODER_C";
56+ case POWER_DOMAIN_TRANSCODER_D:
57+ return "TRANSCODER_D";
4858 case POWER_DOMAIN_TRANSCODER_EDP:
4959 return "TRANSCODER_EDP";
50− case POWER_DOMAIN_TRANSCODER_EDP_VDSC:
51− return "TRANSCODER_EDP_VDSC";
60+ case POWER_DOMAIN_TRANSCODER_VDSC_PW2:
61+ return "TRANSCODER_VDSC_PW2";
5262 case POWER_DOMAIN_TRANSCODER_DSI_A:
5363 return "TRANSCODER_DSI_A";
5464 case POWER_DOMAIN_TRANSCODER_DSI_C:
@@ -60,11 +70,23 @@ intel_display_power_domain_str(enum intel_display_power_domain domain) · 5 unmodified lines
6070 case POWER_DOMAIN_PORT_DDI_C_LANES:
6171 return "PORT_DDI_C_LANES";
6272 case POWER_DOMAIN_PORT_DDI_D_LANES:
63− return "PORT_DDI_D_LANES";
73+ BUILD_BUG_ON(POWER_DOMAIN_PORT_DDI_D_LANES !=
74+ POWER_DOMAIN_PORT_DDI_TC1_LANES);
75+ return ddi_tc_ports ? "PORT_DDI_TC1_LANES" : "PORT_DDI_D_LANES";
6476 case POWER_DOMAIN_PORT_DDI_E_LANES:
65− return "PORT_DDI_E_LANES";
77+ BUILD_BUG_ON(POWER_DOMAIN_PORT_DDI_E_LANES !=
78+ POWER_DOMAIN_PORT_DDI_TC2_LANES);
79+ return ddi_tc_ports ? "PORT_DDI_TC2_LANES" : "PORT_DDI_E_LANES";
6680 case POWER_DOMAIN_PORT_DDI_F_LANES:
67− return "PORT_DDI_F_LANES";
81+ BUILD_BUG_ON(POWER_DOMAIN_PORT_DDI_F_LANES !=
82+ POWER_DOMAIN_PORT_DDI_TC3_LANES);
83+ return ddi_tc_ports ? "PORT_DDI_TC3_LANES" : "PORT_DDI_F_LANES";
84+ case POWER_DOMAIN_PORT_DDI_TC4_LANES:
85+ return "PORT_DDI_TC4_LANES";
86+ case POWER_DOMAIN_PORT_DDI_TC5_LANES:
87+ return "PORT_DDI_TC5_LANES";
88+ case POWER_DOMAIN_PORT_DDI_TC6_LANES:
89+ return "PORT_DDI_TC6_LANES";
6890 case POWER_DOMAIN_PORT_DDI_A_IO:
6991 return "PORT_DDI_A_IO";
7092 case POWER_DOMAIN_PORT_DDI_B_IO:
@@ -72,11 +94,23 @@ intel_display_power_domain_str(enum intel_display_power_domain domain) · 1 unmodified line
7294 case POWER_DOMAIN_PORT_DDI_C_IO:
7395 return "PORT_DDI_C_IO";
7496 case POWER_DOMAIN_PORT_DDI_D_IO:
75− return "PORT_DDI_D_IO";
97+ BUILD_BUG_ON(POWER_DOMAIN_PORT_DDI_D_IO !=
98+ POWER_DOMAIN_PORT_DDI_TC1_IO);
99+ return ddi_tc_ports ? "PORT_DDI_TC1_IO" : "PORT_DDI_D_IO";
76100 case POWER_DOMAIN_PORT_DDI_E_IO:
77− return "PORT_DDI_E_IO";
101+ BUILD_BUG_ON(POWER_DOMAIN_PORT_DDI_E_IO !=
102+ POWER_DOMAIN_PORT_DDI_TC2_IO);
103+ return ddi_tc_ports ? "PORT_DDI_TC2_IO" : "PORT_DDI_E_IO";
78104 case POWER_DOMAIN_PORT_DDI_F_IO:
79− return "PORT_DDI_F_IO";
105+ BUILD_BUG_ON(POWER_DOMAIN_PORT_DDI_F_IO !=
106+ POWER_DOMAIN_PORT_DDI_TC3_IO);
107+ return ddi_tc_ports ? "PORT_DDI_TC3_IO" : "PORT_DDI_F_IO";
108+ case POWER_DOMAIN_PORT_DDI_TC4_IO:
109+ return "PORT_DDI_TC4_IO";
110+ case POWER_DOMAIN_PORT_DDI_TC5_IO:
111+ return "PORT_DDI_TC5_IO";
112+ case POWER_DOMAIN_PORT_DDI_TC6_IO:
113+ return "PORT_DDI_TC6_IO";
80114 case POWER_DOMAIN_PORT_DSI:
81115 return "PORT_DSI";
82116 case POWER_DOMAIN_PORT_CRT:
@@ -94,11 +128,20 @@ intel_display_power_domain_str(enum intel_display_power_domain domain) · 11 unmodified lines
94128 case POWER_DOMAIN_AUX_C:
95129 return "AUX_C";
96130 case POWER_DOMAIN_AUX_D:
97− return "AUX_D";
131+ BUILD_BUG_ON(POWER_DOMAIN_AUX_D != POWER_DOMAIN_AUX_TC1);
132+ return ddi_tc_ports ? "AUX_TC1" : "AUX_D";
98133 case POWER_DOMAIN_AUX_E:
99− return "AUX_E";
134+ BUILD_BUG_ON(POWER_DOMAIN_AUX_E != POWER_DOMAIN_AUX_TC2);
135+ return ddi_tc_ports ? "AUX_TC2" : "AUX_E";
100136 case POWER_DOMAIN_AUX_F:
101− return "AUX_F";
137+ BUILD_BUG_ON(POWER_DOMAIN_AUX_F != POWER_DOMAIN_AUX_TC3);
138+ return ddi_tc_ports ? "AUX_TC3" : "AUX_F";
139+ case POWER_DOMAIN_AUX_TC4:
140+ return "AUX_TC4";
141+ case POWER_DOMAIN_AUX_TC5:
142+ return "AUX_TC5";
143+ case POWER_DOMAIN_AUX_TC6:
144+ return "AUX_TC6";
102145 case POWER_DOMAIN_AUX_IO_A:
103146 return "AUX_IO_A";
104147 case POWER_DOMAIN_AUX_TBT1:
@@ -109,6 +152,10 @@ intel_display_power_domain_str(enum intel_display_power_domain domain) · 4 unmodified lines
109152 return "AUX_TBT3";
110153 case POWER_DOMAIN_AUX_TBT4:
111154 return "AUX_TBT4";
155+ case POWER_DOMAIN_AUX_TBT5:
156+ return "AUX_TBT5";
157+ case POWER_DOMAIN_AUX_TBT6:
158+ return "AUX_TBT6";
112159 case POWER_DOMAIN_GMBUS:
113160 return "GMBUS";
114161 case POWER_DOMAIN_INIT:
@@ -117,6 +164,8 @@ intel_display_power_domain_str(enum intel_display_power_domain domain) · 2 unmodified lines
117164 return "MODESET";
118165 case POWER_DOMAIN_GT_IRQ:
119166 return "GT_IRQ";
167+ case POWER_DOMAIN_DPLL_DC_OFF:
168+ return "DPLL_DC_OFF";
120169 default:
121170 MISSING_CASE(domain);
122171 return "?";
@@ -269,11 +318,17 @@ static void hsw_wait_for_power_well_enable(struct drm_i915_private *dev_priv, · 146 unmodified lines
269318 int pw_idx = power_well->desc->hsw.idx;
270319
271320 /* Timeout for PW1:10 us, AUX:not specified, other PWs:20 us. */
272− WARN_ON(intel_wait_for_register(&dev_priv->uncore,
273− regs->driver,
274− HSW_PWR_WELL_CTL_STATE(pw_idx),
275− HSW_PWR_WELL_CTL_STATE(pw_idx),
276− 1));
321+ if (intel_wait_for_register(&dev_priv->uncore,
322+ regs->driver,
323+ HSW_PWR_WELL_CTL_STATE(pw_idx),
324+ HSW_PWR_WELL_CTL_STATE(pw_idx),
325+ 1)) {
326+ DRM_DEBUG_KMS("%s power well enable timeout\n",
327+ power_well->desc->name);
328+
329+ /* An AUX timeout is expected if the TBT DP tunnel is down. */
330+ WARN_ON(!power_well->desc->hsw.is_tc_tbt);
331+ }
277332 }
278333
279334 static u32 hsw_power_well_requesters(struct drm_i915_private *dev_priv,
@@ -388,7 +443,7 @@ static void hsw_power_well_disable(struct drm_i915_private *dev_priv, · 108 unmodified lines
388443 hsw_wait_for_power_well_disable(dev_priv, power_well);
389444 }
390445
391−#define ICL_AUX_PW_TO_PORT(pw_idx) ((pw_idx) - ICL_PW_CTL_IDX_AUX_A)
446+#define ICL_AUX_PW_TO_PHY(pw_idx) ((pw_idx) - ICL_PW_CTL_IDX_AUX_A)
392447
393448 static void
394449 icl_combo_phy_aux_power_well_enable(struct drm_i915_private *dev_priv,
@@ -396,21 +451,29 @@ icl_combo_phy_aux_power_well_enable(struct drm_i915_private *dev_priv, · 1 unmodified line
396451 {
397452 const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
398453 int pw_idx = power_well->desc->hsw.idx;
399− enum port port = ICL_AUX_PW_TO_PORT(pw_idx);
454+ enum phy phy = ICL_AUX_PW_TO_PHY(pw_idx);
400455 u32 val;
456+ int wa_idx_max;
401457
402458 val = I915_READ(regs->driver);
403459 I915_WRITE(regs->driver, val | HSW_PWR_WELL_CTL_REQ(pw_idx));
404460
405− val = I915_READ(ICL_PORT_CL_DW12(port));
406− I915_WRITE(ICL_PORT_CL_DW12(port), val | ICL_LANE_ENABLE_AUX);
461+ if (INTEL_GEN(dev_priv) < 12) {
462+ val = I915_READ(ICL_PORT_CL_DW12(phy));
463+ I915_WRITE(ICL_PORT_CL_DW12(phy), val | ICL_LANE_ENABLE_AUX);
464+ }
407465
408466 hsw_wait_for_power_well_enable(dev_priv, power_well);
409467
410− /* Display WA #1178: icl */
411− if (IS_ICELAKE(dev_priv) &&
412− pw_idx >= ICL_PW_CTL_IDX_AUX_A && pw_idx <= ICL_PW_CTL_IDX_AUX_B &&
413− !intel_bios_is_port_edp(dev_priv, port)) {
468+ /* Display WA #1178: icl, tgl */
469+ if (IS_TIGERLAKE(dev_priv))
470+ wa_idx_max = ICL_PW_CTL_IDX_AUX_C;
471+ else
472+ wa_idx_max = ICL_PW_CTL_IDX_AUX_B;
473+
474+ if (!IS_ELKHARTLAKE(dev_priv) &&
475+ pw_idx >= ICL_PW_CTL_IDX_AUX_A && pw_idx <= wa_idx_max &&
476+ !intel_bios_is_port_edp(dev_priv, (enum port)phy)) {
414477 val = I915_READ(ICL_AUX_ANAOVRD1(pw_idx));
415478 val |= ICL_AUX_ANAOVRD1_ENABLE | ICL_AUX_ANAOVRD1_LDO_BYPASS;
416479 I915_WRITE(ICL_AUX_ANAOVRD1(pw_idx), val);
@@ -423,11 +486,13 @@ icl_combo_phy_aux_power_well_disable(struct drm_i915_private *dev_priv, · 6 unmodified lines
423486 {
424487 const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
425488 int pw_idx = power_well->desc->hsw.idx;
426− enum port port = ICL_AUX_PW_TO_PORT(pw_idx);
489+ enum phy phy = ICL_AUX_PW_TO_PHY(pw_idx);
427490 u32 val;
428491
429− val = I915_READ(ICL_PORT_CL_DW12(port));
430− I915_WRITE(ICL_PORT_CL_DW12(port), val & ~ICL_LANE_ENABLE_AUX);
492+ if (INTEL_GEN(dev_priv) < 12) {
493+ val = I915_READ(ICL_PORT_CL_DW12(phy));
494+ I915_WRITE(ICL_PORT_CL_DW12(phy), val & ~ICL_LANE_ENABLE_AUX);
495+ }
431496
432497 val = I915_READ(regs->driver);
433498 I915_WRITE(regs->driver, val & ~HSW_PWR_WELL_CTL_REQ(pw_idx));
@@ -441,26 +506,108 @@ icl_combo_phy_aux_power_well_disable(struct drm_i915_private *dev_priv, · 7 unmodified lines
441506 #define ICL_TBT_AUX_PW_TO_CH(pw_idx) \
442507 ((pw_idx) - ICL_PW_CTL_IDX_AUX_TBT1 + AUX_CH_C)
443508
509+static enum aux_ch icl_tc_phy_aux_ch(struct drm_i915_private *dev_priv,
510+ struct i915_power_well *power_well)
511+{
512+ int pw_idx = power_well->desc->hsw.idx;
513+
514+ return power_well->desc->hsw.is_tc_tbt ? ICL_TBT_AUX_PW_TO_CH(pw_idx) :
515+ ICL_AUX_PW_TO_CH(pw_idx);
516+}
517+
518+#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_RUNTIME_PM)
519+
520+static u64 async_put_domains_mask(struct i915_power_domains *power_domains);
521+
522+static int power_well_async_ref_count(struct drm_i915_private *dev_priv,
523+ struct i915_power_well *power_well)
524+{
525+ int refs = hweight64(power_well->desc->domains &
526+ async_put_domains_mask(&dev_priv->power_domains));
527+
528+ WARN_ON(refs > power_well->count);
529+
530+ return refs;
531+}
532+
533+static void icl_tc_port_assert_ref_held(struct drm_i915_private *dev_priv,
534+ struct i915_power_well *power_well)
535+{
536+ enum aux_ch aux_ch = icl_tc_phy_aux_ch(dev_priv, power_well);
537+ struct intel_digital_port *dig_port = NULL;
538+ struct intel_encoder *encoder;
539+
540+ /* Bypass the check if all references are released asynchronously */
541+ if (power_well_async_ref_count(dev_priv, power_well) ==
542+ power_well->count)
543+ return;
544+
545+ aux_ch = icl_tc_phy_aux_ch(dev_priv, power_well);
546+
547+ for_each_intel_encoder(&dev_priv->drm, encoder) {
548+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
549+
550+ if (!intel_phy_is_tc(dev_priv, phy))
551+ continue;
552+
553+ /* We'll check the MST primary port */
554+ if (encoder->type == INTEL_OUTPUT_DP_MST)
555+ continue;
556+
557+ dig_port = enc_to_dig_port(&encoder->base);
558+ if (WARN_ON(!dig_port))
559+ continue;
560+
561+ if (dig_port->aux_ch != aux_ch) {
562+ dig_port = NULL;
563+ continue;
564+ }
565+
566+ break;
567+ }
568+
569+ if (WARN_ON(!dig_port))
570+ return;
571+
572+ WARN_ON(!intel_tc_port_ref_held(dig_port));
573+}
574+
575+#else
576+
577+static void icl_tc_port_assert_ref_held(struct drm_i915_private *dev_priv,
578+ struct i915_power_well *power_well)
579+{
580+}
581+
582+#endif
583+
444584 static void
445585 icl_tc_phy_aux_power_well_enable(struct drm_i915_private *dev_priv,
446586 struct i915_power_well *power_well)
447587 {
448− int pw_idx = power_well->desc->hsw.idx;
449− bool is_tbt = power_well->desc->hsw.is_tc_tbt;
450− enum aux_ch aux_ch;
588+ enum aux_ch aux_ch = icl_tc_phy_aux_ch(dev_priv, power_well);
451589 u32 val;
452590
453− aux_ch = is_tbt ? ICL_TBT_AUX_PW_TO_CH(pw_idx) :
454− ICL_AUX_PW_TO_CH(pw_idx);
591+ icl_tc_port_assert_ref_held(dev_priv, power_well);
592+
455593 val = I915_READ(DP_AUX_CH_CTL(aux_ch));
456594 val &= ~DP_AUX_CH_CTL_TBT_IO;
457− if (is_tbt)
595+ if (power_well->desc->hsw.is_tc_tbt)
458596 val |= DP_AUX_CH_CTL_TBT_IO;
459597 I915_WRITE(DP_AUX_CH_CTL(aux_ch), val);
460598
461599 hsw_power_well_enable(dev_priv, power_well);
462600 }
463601
602+static void
603+icl_tc_phy_aux_power_well_disable(struct drm_i915_private *dev_priv,
604+ struct i915_power_well *power_well)
605+{
606+ icl_tc_port_assert_ref_held(dev_priv, power_well);
607+
608+ hsw_power_well_disable(dev_priv, power_well);
609+}
610+
464611 /*
465612 * We should only use the power well if we explicitly asked the hardware to
466613 * enable it, so check if it's enabled and also check if we've requested it to
@@ -1071,7 +1218,7 @@ static void vlv_display_power_well_deinit(struct drm_i915_private *dev_priv) · 604 unmodified lines
10711218 spin_unlock_irq(&dev_priv->irq_lock);
10721219
10731220 /* make sure we're done processing display irqs */
1074− synchronize_irq(dev_priv->drm.irq);
1221+ intel_synchronize_irq(dev_priv);
10751222
10761223 intel_power_sequencer_reset(dev_priv);
10771224
@@ -1575,12 +1722,15 @@ __async_put_domains_state_ok(struct i915_power_domains *power_domains) · 497 unmodified lines
15751722 static void print_power_domains(struct i915_power_domains *power_domains,
15761723 const char *prefix, u64 mask)
15771724 {
1725+ struct drm_i915_private *i915 =
1726+ container_of(power_domains, struct drm_i915_private,
1727+ power_domains);
15781728 enum intel_display_power_domain domain;
15791729
15801730 DRM_DEBUG_DRIVER("%s (%lu):\n", prefix, hweight64(mask));
15811731 for_each_power_domain(domain, mask)
15821732 DRM_DEBUG_DRIVER("%s use_count %d\n",
1583− intel_display_power_domain_str(domain),
1733+ intel_display_power_domain_str(i915, domain),
15841734 power_domains->domain_use_count[domain]);
15851735 }
15861736
@@ -1750,7 +1900,7 @@ __intel_display_power_put_domain(struct drm_i915_private *dev_priv, · 163 unmodified lines
17501900 {
17511901 struct i915_power_domains *power_domains;
17521902 struct i915_power_well *power_well;
1753− const char *name = intel_display_power_domain_str(domain);
1903+ const char *name = intel_display_power_domain_str(dev_priv, domain);
17541904
17551905 power_domains = &dev_priv->power_domains;
17561906
@@ -2359,7 +2509,7 @@ void intel_display_power_put(struct drm_i915_private *dev_priv, · 602 unmodified lines
23592509 */
23602510 #define ICL_PW_2_POWER_DOMAINS ( \
23612511 ICL_PW_3_POWER_DOMAINS | \
2362− BIT_ULL(POWER_DOMAIN_TRANSCODER_EDP_VDSC) | \
2512+ BIT_ULL(POWER_DOMAIN_TRANSCODER_VDSC_PW2) | \
23632513 BIT_ULL(POWER_DOMAIN_INIT))
23642514 /*
23652515 * - KVMR (HW control)
@@ -2368,6 +2518,7 @@ void intel_display_power_put(struct drm_i915_private *dev_priv, · 2 unmodified lines
23682518 ICL_PW_2_POWER_DOMAINS | \
23692519 BIT_ULL(POWER_DOMAIN_MODESET) | \
23702520 BIT_ULL(POWER_DOMAIN_AUX_A) | \
2521+ BIT_ULL(POWER_DOMAIN_DPLL_DC_OFF) | \
23712522 BIT_ULL(POWER_DOMAIN_INIT))
23722523
23732524 #define ICL_DDI_IO_A_POWER_DOMAINS ( \
@@ -2405,6 +2556,93 @@ void intel_display_power_put(struct drm_i915_private *dev_priv, · 31 unmodified lines
24052556 #define ICL_AUX_TBT4_IO_POWER_DOMAINS ( \
24062557 BIT_ULL(POWER_DOMAIN_AUX_TBT4))
24072558
2559+#define TGL_PW_5_POWER_DOMAINS ( \
2560+ BIT_ULL(POWER_DOMAIN_PIPE_D) | \
2561+ BIT_ULL(POWER_DOMAIN_PIPE_D_PANEL_FITTER) | \
2562+ BIT_ULL(POWER_DOMAIN_INIT))
2563+
2564+#define TGL_PW_4_POWER_DOMAINS ( \
2565+ TGL_PW_5_POWER_DOMAINS | \
2566+ BIT_ULL(POWER_DOMAIN_PIPE_C) | \
2567+ BIT_ULL(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
2568+ BIT_ULL(POWER_DOMAIN_INIT))
2569+
2570+#define TGL_PW_3_POWER_DOMAINS ( \
2571+ TGL_PW_4_POWER_DOMAINS | \
2572+ BIT_ULL(POWER_DOMAIN_PIPE_B) | \
2573+ BIT_ULL(POWER_DOMAIN_TRANSCODER_B) | \
2574+ BIT_ULL(POWER_DOMAIN_TRANSCODER_C) | \
2575+ BIT_ULL(POWER_DOMAIN_TRANSCODER_D) | \
2576+ BIT_ULL(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
2577+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC1_LANES) | \
2578+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC1_IO) | \
2579+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC2_LANES) | \
2580+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC2_IO) | \
2581+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC3_LANES) | \
2582+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC3_IO) | \
2583+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC4_LANES) | \
2584+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC4_IO) | \
2585+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC5_LANES) | \
2586+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC5_IO) | \
2587+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC6_LANES) | \
2588+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC6_IO) | \
2589+ BIT_ULL(POWER_DOMAIN_AUX_TC1) | \
2590+ BIT_ULL(POWER_DOMAIN_AUX_TC2) | \
2591+ BIT_ULL(POWER_DOMAIN_AUX_TC3) | \
2592+ BIT_ULL(POWER_DOMAIN_AUX_TC4) | \
2593+ BIT_ULL(POWER_DOMAIN_AUX_TC5) | \
2594+ BIT_ULL(POWER_DOMAIN_AUX_TC6) | \
2595+ BIT_ULL(POWER_DOMAIN_AUX_TBT1) | \
2596+ BIT_ULL(POWER_DOMAIN_AUX_TBT2) | \
2597+ BIT_ULL(POWER_DOMAIN_AUX_TBT3) | \
2598+ BIT_ULL(POWER_DOMAIN_AUX_TBT4) | \
2599+ BIT_ULL(POWER_DOMAIN_AUX_TBT5) | \
2600+ BIT_ULL(POWER_DOMAIN_AUX_TBT6) | \
2601+ BIT_ULL(POWER_DOMAIN_VGA) | \
2602+ BIT_ULL(POWER_DOMAIN_AUDIO) | \
2603+ BIT_ULL(POWER_DOMAIN_INIT))
2604+
2605+#define TGL_PW_2_POWER_DOMAINS ( \
2606+ TGL_PW_3_POWER_DOMAINS | \
2607+ BIT_ULL(POWER_DOMAIN_TRANSCODER_VDSC_PW2) | \
2608+ BIT_ULL(POWER_DOMAIN_INIT))
2609+
2610+#define TGL_DISPLAY_DC_OFF_POWER_DOMAINS ( \
2611+ TGL_PW_2_POWER_DOMAINS | \
2612+ BIT_ULL(POWER_DOMAIN_MODESET) | \
2613+ BIT_ULL(POWER_DOMAIN_AUX_A) | \
2614+ BIT_ULL(POWER_DOMAIN_INIT))
2615+
2616+#define TGL_DDI_IO_TC1_POWER_DOMAINS ( \
2617+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC1_IO))
2618+#define TGL_DDI_IO_TC2_POWER_DOMAINS ( \
2619+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC2_IO))
2620+#define TGL_DDI_IO_TC3_POWER_DOMAINS ( \
2621+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC3_IO))
2622+#define TGL_DDI_IO_TC4_POWER_DOMAINS ( \
2623+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC4_IO))
2624+#define TGL_DDI_IO_TC5_POWER_DOMAINS ( \
2625+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC5_IO))
2626+#define TGL_DDI_IO_TC6_POWER_DOMAINS ( \
2627+ BIT_ULL(POWER_DOMAIN_PORT_DDI_TC6_IO))
2628+
2629+#define TGL_AUX_TC1_IO_POWER_DOMAINS ( \
2630+ BIT_ULL(POWER_DOMAIN_AUX_TC1))
2631+#define TGL_AUX_TC2_IO_POWER_DOMAINS ( \
2632+ BIT_ULL(POWER_DOMAIN_AUX_TC2))
2633+#define TGL_AUX_TC3_IO_POWER_DOMAINS ( \
2634+ BIT_ULL(POWER_DOMAIN_AUX_TC3))
2635+#define TGL_AUX_TC4_IO_POWER_DOMAINS ( \
2636+ BIT_ULL(POWER_DOMAIN_AUX_TC4))
2637+#define TGL_AUX_TC5_IO_POWER_DOMAINS ( \
2638+ BIT_ULL(POWER_DOMAIN_AUX_TC5))
2639+#define TGL_AUX_TC6_IO_POWER_DOMAINS ( \
2640+ BIT_ULL(POWER_DOMAIN_AUX_TC6))
2641+#define TGL_AUX_TBT5_IO_POWER_DOMAINS ( \
2642+ BIT_ULL(POWER_DOMAIN_AUX_TBT5))
2643+#define TGL_AUX_TBT6_IO_POWER_DOMAINS ( \
2644+ BIT_ULL(POWER_DOMAIN_AUX_TBT6))
2645+
24082646 static const struct i915_power_well_ops i9xx_always_on_power_well_ops = {
24092647 .sync_hw = i9xx_power_well_sync_hw_noop,
24102648 .enable = i9xx_always_on_power_well_noop,
@@ -3113,7 +3351,7 @@ static const struct i915_power_well_ops icl_combo_phy_aux_power_well_ops = { · 702 unmodified lines
31133351 static const struct i915_power_well_ops icl_tc_phy_aux_power_well_ops = {
31143352 .sync_hw = hsw_power_well_sync_hw,
31153353 .enable = icl_tc_phy_aux_power_well_enable,
3116− .disable = hsw_power_well_disable,
3354+ .disable = icl_tc_phy_aux_power_well_disable,
31173355 .is_enabled = hsw_power_well_enabled,
31183356 };
31193357
@@ -3362,6 +3600,335 @@ static const struct i915_power_well_desc icl_power_wells[] = { · 242 unmodified lines
33623600 },
33633601 };
33643602
3603+static const struct i915_power_well_desc tgl_power_wells[] = {
3604+ {
3605+ .name = "always-on",
3606+ .always_on = true,
3607+ .domains = POWER_DOMAIN_MASK,
3608+ .ops = &i9xx_always_on_power_well_ops,
3609+ .id = DISP_PW_ID_NONE,
3610+ },
3611+ {
3612+ .name = "power well 1",
3613+ /* Handled by the DMC firmware */
3614+ .always_on = true,
3615+ .domains = 0,
3616+ .ops = &hsw_power_well_ops,
3617+ .id = SKL_DISP_PW_1,
3618+ {
3619+ .hsw.regs = &hsw_power_well_regs,
3620+ .hsw.idx = ICL_PW_CTL_IDX_PW_1,
3621+ .hsw.has_fuses = true,
3622+ },
3623+ },
3624+ {
3625+ .name = "DC off",
3626+ .domains = TGL_DISPLAY_DC_OFF_POWER_DOMAINS,
3627+ .ops = &gen9_dc_off_power_well_ops,
3628+ .id = DISP_PW_ID_NONE,
3629+ },
3630+ {
3631+ .name = "power well 2",
3632+ .domains = TGL_PW_2_POWER_DOMAINS,
3633+ .ops = &hsw_power_well_ops,
3634+ .id = SKL_DISP_PW_2,
3635+ {
3636+ .hsw.regs = &hsw_power_well_regs,
3637+ .hsw.idx = ICL_PW_CTL_IDX_PW_2,
3638+ .hsw.has_fuses = true,
3639+ },
3640+ },
3641+ {
3642+ .name = "power well 3",
3643+ .domains = TGL_PW_3_POWER_DOMAINS,
3644+ .ops = &hsw_power_well_ops,
3645+ .id = DISP_PW_ID_NONE,
3646+ {
3647+ .hsw.regs = &hsw_power_well_regs,
3648+ .hsw.idx = ICL_PW_CTL_IDX_PW_3,
3649+ .hsw.irq_pipe_mask = BIT(PIPE_B),
3650+ .hsw.has_vga = true,
3651+ .hsw.has_fuses = true,
3652+ },
3653+ },
3654+ {
3655+ .name = "DDI A IO",
3656+ .domains = ICL_DDI_IO_A_POWER_DOMAINS,
3657+ .ops = &hsw_power_well_ops,
3658+ .id = DISP_PW_ID_NONE,
3659+ {
3660+ .hsw.regs = &icl_ddi_power_well_regs,
3661+ .hsw.idx = ICL_PW_CTL_IDX_DDI_A,
3662+ }
3663+ },
3664+ {
3665+ .name = "DDI B IO",
3666+ .domains = ICL_DDI_IO_B_POWER_DOMAINS,
3667+ .ops = &hsw_power_well_ops,
3668+ .id = DISP_PW_ID_NONE,
3669+ {
3670+ .hsw.regs = &icl_ddi_power_well_regs,
3671+ .hsw.idx = ICL_PW_CTL_IDX_DDI_B,
3672+ }
3673+ },
3674+ {
3675+ .name = "DDI C IO",
3676+ .domains = ICL_DDI_IO_C_POWER_DOMAINS,
3677+ .ops = &hsw_power_well_ops,
3678+ .id = DISP_PW_ID_NONE,
3679+ {
3680+ .hsw.regs = &icl_ddi_power_well_regs,
3681+ .hsw.idx = ICL_PW_CTL_IDX_DDI_C,
3682+ }
3683+ },
3684+ {
3685+ .name = "DDI TC1 IO",
3686+ .domains = TGL_DDI_IO_TC1_POWER_DOMAINS,
3687+ .ops = &hsw_power_well_ops,
3688+ .id = DISP_PW_ID_NONE,
3689+ {
3690+ .hsw.regs = &icl_ddi_power_well_regs,
3691+ .hsw.idx = TGL_PW_CTL_IDX_DDI_TC1,
3692+ },
3693+ },
3694+ {
3695+ .name = "DDI TC2 IO",
3696+ .domains = TGL_DDI_IO_TC2_POWER_DOMAINS,
3697+ .ops = &hsw_power_well_ops,
3698+ .id = DISP_PW_ID_NONE,
3699+ {
3700+ .hsw.regs = &icl_ddi_power_well_regs,
3701+ .hsw.idx = TGL_PW_CTL_IDX_DDI_TC2,
3702+ },
3703+ },
3704+ {
3705+ .name = "DDI TC3 IO",
3706+ .domains = TGL_DDI_IO_TC3_POWER_DOMAINS,
3707+ .ops = &hsw_power_well_ops,
3708+ .id = DISP_PW_ID_NONE,
3709+ {
3710+ .hsw.regs = &icl_ddi_power_well_regs,
3711+ .hsw.idx = TGL_PW_CTL_IDX_DDI_TC3,
3712+ },
3713+ },
3714+ {
3715+ .name = "DDI TC4 IO",
3716+ .domains = TGL_DDI_IO_TC4_POWER_DOMAINS,
3717+ .ops = &hsw_power_well_ops,
3718+ .id = DISP_PW_ID_NONE,
3719+ {
3720+ .hsw.regs = &icl_ddi_power_well_regs,
3721+ .hsw.idx = TGL_PW_CTL_IDX_DDI_TC4,
3722+ },
3723+ },
3724+ {
3725+ .name = "DDI TC5 IO",
3726+ .domains = TGL_DDI_IO_TC5_POWER_DOMAINS,
3727+ .ops = &hsw_power_well_ops,
3728+ .id = DISP_PW_ID_NONE,
3729+ {
3730+ .hsw.regs = &icl_ddi_power_well_regs,
3731+ .hsw.idx = TGL_PW_CTL_IDX_DDI_TC5,
3732+ },
3733+ },
3734+ {
3735+ .name = "DDI TC6 IO",
3736+ .domains = TGL_DDI_IO_TC6_POWER_DOMAINS,
3737+ .ops = &hsw_power_well_ops,
3738+ .id = DISP_PW_ID_NONE,
3739+ {
3740+ .hsw.regs = &icl_ddi_power_well_regs,
3741+ .hsw.idx = TGL_PW_CTL_IDX_DDI_TC6,
3742+ },
3743+ },
3744+ {
3745+ .name = "AUX A",
3746+ .domains = ICL_AUX_A_IO_POWER_DOMAINS,
3747+ .ops = &icl_combo_phy_aux_power_well_ops,
3748+ .id = DISP_PW_ID_NONE,
3749+ {
3750+ .hsw.regs = &icl_aux_power_well_regs,
3751+ .hsw.idx = ICL_PW_CTL_IDX_AUX_A,
3752+ },
3753+ },
3754+ {
3755+ .name = "AUX B",
3756+ .domains = ICL_AUX_B_IO_POWER_DOMAINS,
3757+ .ops = &icl_combo_phy_aux_power_well_ops,
3758+ .id = DISP_PW_ID_NONE,
3759+ {
3760+ .hsw.regs = &icl_aux_power_well_regs,
3761+ .hsw.idx = ICL_PW_CTL_IDX_AUX_B,
3762+ },
3763+ },
3764+ {
3765+ .name = "AUX C",
3766+ .domains = ICL_AUX_C_IO_POWER_DOMAINS,
3767+ .ops = &icl_combo_phy_aux_power_well_ops,
3768+ .id = DISP_PW_ID_NONE,
3769+ {
3770+ .hsw.regs = &icl_aux_power_well_regs,
3771+ .hsw.idx = ICL_PW_CTL_IDX_AUX_C,
3772+ },
3773+ },
3774+ {
3775+ .name = "AUX TC1",
3776+ .domains = TGL_AUX_TC1_IO_POWER_DOMAINS,
3777+ .ops = &icl_tc_phy_aux_power_well_ops,
3778+ .id = DISP_PW_ID_NONE,
3779+ {
3780+ .hsw.regs = &icl_aux_power_well_regs,
3781+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TC1,
3782+ .hsw.is_tc_tbt = false,
3783+ },
3784+ },
3785+ {
3786+ .name = "AUX TC2",
3787+ .domains = TGL_AUX_TC2_IO_POWER_DOMAINS,
3788+ .ops = &icl_tc_phy_aux_power_well_ops,
3789+ .id = DISP_PW_ID_NONE,
3790+ {
3791+ .hsw.regs = &icl_aux_power_well_regs,
3792+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TC2,
3793+ .hsw.is_tc_tbt = false,
3794+ },
3795+ },
3796+ {
3797+ .name = "AUX TC3",
3798+ .domains = TGL_AUX_TC3_IO_POWER_DOMAINS,
3799+ .ops = &icl_tc_phy_aux_power_well_ops,
3800+ .id = DISP_PW_ID_NONE,
3801+ {
3802+ .hsw.regs = &icl_aux_power_well_regs,
3803+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TC3,
3804+ .hsw.is_tc_tbt = false,
3805+ },
3806+ },
3807+ {
3808+ .name = "AUX TC4",
3809+ .domains = TGL_AUX_TC4_IO_POWER_DOMAINS,
3810+ .ops = &icl_tc_phy_aux_power_well_ops,
3811+ .id = DISP_PW_ID_NONE,
3812+ {
3813+ .hsw.regs = &icl_aux_power_well_regs,
3814+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TC4,
3815+ .hsw.is_tc_tbt = false,
3816+ },
3817+ },
3818+ {
3819+ .name = "AUX TC5",
3820+ .domains = TGL_AUX_TC5_IO_POWER_DOMAINS,
3821+ .ops = &icl_tc_phy_aux_power_well_ops,
3822+ .id = DISP_PW_ID_NONE,
3823+ {
3824+ .hsw.regs = &icl_aux_power_well_regs,
3825+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TC5,
3826+ .hsw.is_tc_tbt = false,
3827+ },
3828+ },
3829+ {
3830+ .name = "AUX TC6",
3831+ .domains = TGL_AUX_TC6_IO_POWER_DOMAINS,
3832+ .ops = &icl_tc_phy_aux_power_well_ops,
3833+ .id = DISP_PW_ID_NONE,
3834+ {
3835+ .hsw.regs = &icl_aux_power_well_regs,
3836+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TC6,
3837+ .hsw.is_tc_tbt = false,
3838+ },
3839+ },
3840+ {
3841+ .name = "AUX TBT1",
3842+ .domains = ICL_AUX_TBT1_IO_POWER_DOMAINS,
3843+ .ops = &hsw_power_well_ops,
3844+ .id = DISP_PW_ID_NONE,
3845+ {
3846+ .hsw.regs = &icl_aux_power_well_regs,
3847+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TBT1,
3848+ .hsw.is_tc_tbt = true,
3849+ },
3850+ },
3851+ {
3852+ .name = "AUX TBT2",
3853+ .domains = ICL_AUX_TBT2_IO_POWER_DOMAINS,
3854+ .ops = &hsw_power_well_ops,
3855+ .id = DISP_PW_ID_NONE,
3856+ {
3857+ .hsw.regs = &icl_aux_power_well_regs,
3858+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TBT2,
3859+ .hsw.is_tc_tbt = true,
3860+ },
3861+ },
3862+ {
3863+ .name = "AUX TBT3",
3864+ .domains = ICL_AUX_TBT3_IO_POWER_DOMAINS,
3865+ .ops = &hsw_power_well_ops,
3866+ .id = DISP_PW_ID_NONE,
3867+ {
3868+ .hsw.regs = &icl_aux_power_well_regs,
3869+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TBT3,
3870+ .hsw.is_tc_tbt = true,
3871+ },
3872+ },
3873+ {
3874+ .name = "AUX TBT4",
3875+ .domains = ICL_AUX_TBT4_IO_POWER_DOMAINS,
3876+ .ops = &hsw_power_well_ops,
3877+ .id = DISP_PW_ID_NONE,
3878+ {
3879+ .hsw.regs = &icl_aux_power_well_regs,
3880+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TBT4,
3881+ .hsw.is_tc_tbt = true,
3882+ },
3883+ },
3884+ {
3885+ .name = "AUX TBT5",
3886+ .domains = TGL_AUX_TBT5_IO_POWER_DOMAINS,
3887+ .ops = &hsw_power_well_ops,
3888+ .id = DISP_PW_ID_NONE,
3889+ {
3890+ .hsw.regs = &icl_aux_power_well_regs,
3891+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TBT5,
3892+ .hsw.is_tc_tbt = true,
3893+ },
3894+ },
3895+ {
3896+ .name = "AUX TBT6",
3897+ .domains = TGL_AUX_TBT6_IO_POWER_DOMAINS,
3898+ .ops = &hsw_power_well_ops,
3899+ .id = DISP_PW_ID_NONE,
3900+ {
3901+ .hsw.regs = &icl_aux_power_well_regs,
3902+ .hsw.idx = TGL_PW_CTL_IDX_AUX_TBT6,
3903+ .hsw.is_tc_tbt = true,
3904+ },
3905+ },
3906+ {
3907+ .name = "power well 4",
3908+ .domains = TGL_PW_4_POWER_DOMAINS,
3909+ .ops = &hsw_power_well_ops,
3910+ .id = DISP_PW_ID_NONE,
3911+ {
3912+ .hsw.regs = &hsw_power_well_regs,
3913+ .hsw.idx = ICL_PW_CTL_IDX_PW_4,
3914+ .hsw.has_fuses = true,
3915+ .hsw.irq_pipe_mask = BIT(PIPE_C),
3916+ }
3917+ },
3918+ {
3919+ .name = "power well 5",
3920+ .domains = TGL_PW_5_POWER_DOMAINS,
3921+ .ops = &hsw_power_well_ops,
3922+ .id = DISP_PW_ID_NONE,
3923+ {
3924+ .hsw.regs = &hsw_power_well_regs,
3925+ .hsw.idx = TGL_PW_CTL_IDX_PW_5,
3926+ .hsw.has_fuses = true,
3927+ .hsw.irq_pipe_mask = BIT(PIPE_D),
3928+ },
3929+ },
3930+};
3931+
33653932 static int
33663933 sanitize_disable_power_well_option(const struct drm_i915_private *dev_priv,
33673934 int disable_power_well)
@@ -3489,7 +4056,9 @@ int intel_power_domains_init(struct drm_i915_private *dev_priv) · 121 unmodified lines
34894056 * The enabling order will be from lower to higher indexed wells,
34904057 * the disabling order is reversed.
34914058 */
3492− if (IS_GEN(dev_priv, 11)) {
4059+ if (IS_GEN(dev_priv, 12)) {
4060+ err = set_power_wells(power_domains, tgl_power_wells);
4061+ } else if (IS_GEN(dev_priv, 11)) {
34934062 err = set_power_wells(power_domains, icl_power_wells);
34944063 } else if (IS_CANNONLAKE(dev_priv)) {
34954064 err = set_power_wells(power_domains, cnl_power_wells);
@@ -4337,7 +4906,7 @@ static void intel_power_domains_verify_state(struct drm_i915_private *dev_priv); · 841 unmodified lines
43374906 *
43384907 * It will return with power domains disabled (to be enabled later by
43394908 * intel_power_domains_enable()) and must be paired with
4340− * intel_power_domains_fini_hw().
4909+ * intel_power_domains_driver_remove().
43414910 */
43424911 void intel_power_domains_init_hw(struct drm_i915_private *i915, bool resume)
43434912 {
@@ -4389,7 +4958,7 @@ void intel_power_domains_init_hw(struct drm_i915_private *i915, bool resume) · 45 unmodified lines
43894958 }
43904959
43914960 /**
4392− * intel_power_domains_fini_hw - deinitialize hw power domain state
4961+ * intel_power_domains_driver_remove - deinitialize hw power domain state
43934962 * @i915: i915 device instance
43944963 *
43954964 * De-initializes the display power domain HW state. It also ensures that the
@@ -4399,7 +4968,7 @@ void intel_power_domains_init_hw(struct drm_i915_private *i915, bool resume) · 3 unmodified lines
43994968 * intel_power_domains_disable()) and must be paired with
44004969 * intel_power_domains_init_hw().
44014970 */
4402−void intel_power_domains_fini_hw(struct drm_i915_private *i915)
4971+void intel_power_domains_driver_remove(struct drm_i915_private *i915)
44034972 {
44044973 intel_wakeref_t wakeref __maybe_unused =
44054974 fetch_and_zero(&i915->power_domains.wakeref);
@@ -4553,7 +5122,8 @@ static void intel_power_domains_dump_info(struct drm_i915_private *i915) · 147 unmodified lines
45535122
45545123 for_each_power_domain(domain, power_well->desc->domains)
45555124 DRM_DEBUG_DRIVER(" %-23s %d\n",
4556− intel_display_power_domain_str(domain),
5125+ intel_display_power_domain_str(i915,
5126+ domain),
45575127 power_domains->domain_use_count[domain]);
45585128 }
45595129 }
drivers/gpu/drm/i915/display/intel_display_power.h+32−3
@@ -18,28 +18,47 @@ enum intel_display_power_domain { · 17 unmodified lines
1818 POWER_DOMAIN_PIPE_A,
1919 POWER_DOMAIN_PIPE_B,
2020 POWER_DOMAIN_PIPE_C,
21+ POWER_DOMAIN_PIPE_D,
2122 POWER_DOMAIN_PIPE_A_PANEL_FITTER,
2223 POWER_DOMAIN_PIPE_B_PANEL_FITTER,
2324 POWER_DOMAIN_PIPE_C_PANEL_FITTER,
25+ POWER_DOMAIN_PIPE_D_PANEL_FITTER,
2426 POWER_DOMAIN_TRANSCODER_A,
2527 POWER_DOMAIN_TRANSCODER_B,
2628 POWER_DOMAIN_TRANSCODER_C,
29+ POWER_DOMAIN_TRANSCODER_D,
2730 POWER_DOMAIN_TRANSCODER_EDP,
28− POWER_DOMAIN_TRANSCODER_EDP_VDSC,
31+ /* VDSC/joining for TRANSCODER_EDP (ICL) or TRANSCODER_A (TGL) */
32+ POWER_DOMAIN_TRANSCODER_VDSC_PW2,
2933 POWER_DOMAIN_TRANSCODER_DSI_A,
3034 POWER_DOMAIN_TRANSCODER_DSI_C,
3135 POWER_DOMAIN_PORT_DDI_A_LANES,
3236 POWER_DOMAIN_PORT_DDI_B_LANES,
3337 POWER_DOMAIN_PORT_DDI_C_LANES,
3438 POWER_DOMAIN_PORT_DDI_D_LANES,
39+ POWER_DOMAIN_PORT_DDI_TC1_LANES = POWER_DOMAIN_PORT_DDI_D_LANES,
3540 POWER_DOMAIN_PORT_DDI_E_LANES,
41+ POWER_DOMAIN_PORT_DDI_TC2_LANES = POWER_DOMAIN_PORT_DDI_E_LANES,
3642 POWER_DOMAIN_PORT_DDI_F_LANES,
43+ POWER_DOMAIN_PORT_DDI_TC3_LANES = POWER_DOMAIN_PORT_DDI_F_LANES,
44+ POWER_DOMAIN_PORT_DDI_TC4_LANES,
45+ POWER_DOMAIN_PORT_DDI_TC5_LANES,
46+ POWER_DOMAIN_PORT_DDI_TC6_LANES,
3747 POWER_DOMAIN_PORT_DDI_A_IO,
3848 POWER_DOMAIN_PORT_DDI_B_IO,
3949 POWER_DOMAIN_PORT_DDI_C_IO,
4050 POWER_DOMAIN_PORT_DDI_D_IO,
51+ POWER_DOMAIN_PORT_DDI_TC1_IO = POWER_DOMAIN_PORT_DDI_D_IO,
4152 POWER_DOMAIN_PORT_DDI_E_IO,
53+ POWER_DOMAIN_PORT_DDI_TC2_IO = POWER_DOMAIN_PORT_DDI_E_IO,
4254 POWER_DOMAIN_PORT_DDI_F_IO,
55+ POWER_DOMAIN_PORT_DDI_TC3_IO = POWER_DOMAIN_PORT_DDI_F_IO,
56+ POWER_DOMAIN_PORT_DDI_G_IO,
57+ POWER_DOMAIN_PORT_DDI_TC4_IO = POWER_DOMAIN_PORT_DDI_G_IO,
58+ POWER_DOMAIN_PORT_DDI_H_IO,
59+ POWER_DOMAIN_PORT_DDI_TC5_IO = POWER_DOMAIN_PORT_DDI_H_IO,
60+ POWER_DOMAIN_PORT_DDI_I_IO,
61+ POWER_DOMAIN_PORT_DDI_TC6_IO = POWER_DOMAIN_PORT_DDI_I_IO,
4362 POWER_DOMAIN_PORT_DSI,
4463 POWER_DOMAIN_PORT_CRT,
4564 POWER_DOMAIN_PORT_OTHER,
@@ -49,16 +68,25 @@ enum intel_display_power_domain { · 3 unmodified lines
4968 POWER_DOMAIN_AUX_B,
5069 POWER_DOMAIN_AUX_C,
5170 POWER_DOMAIN_AUX_D,
71+ POWER_DOMAIN_AUX_TC1 = POWER_DOMAIN_AUX_D,
5272 POWER_DOMAIN_AUX_E,
73+ POWER_DOMAIN_AUX_TC2 = POWER_DOMAIN_AUX_E,
5374 POWER_DOMAIN_AUX_F,
75+ POWER_DOMAIN_AUX_TC3 = POWER_DOMAIN_AUX_F,
76+ POWER_DOMAIN_AUX_TC4,
77+ POWER_DOMAIN_AUX_TC5,
78+ POWER_DOMAIN_AUX_TC6,
5479 POWER_DOMAIN_AUX_IO_A,
5580 POWER_DOMAIN_AUX_TBT1,
5681 POWER_DOMAIN_AUX_TBT2,
5782 POWER_DOMAIN_AUX_TBT3,
5883 POWER_DOMAIN_AUX_TBT4,
84+ POWER_DOMAIN_AUX_TBT5,
85+ POWER_DOMAIN_AUX_TBT6,
5986 POWER_DOMAIN_GMBUS,
6087 POWER_DOMAIN_MODESET,
6188 POWER_DOMAIN_GT_IRQ,
89+ POWER_DOMAIN_DPLL_DC_OFF,
6290 POWER_DOMAIN_INIT,
6391
6492 POWER_DOMAIN_NUM,
@@ -213,7 +241,7 @@ void gen9_enable_dc5(struct drm_i915_private *dev_priv); · 148 unmodified lines
213241 int intel_power_domains_init(struct drm_i915_private *dev_priv);
214242 void intel_power_domains_cleanup(struct drm_i915_private *dev_priv);
215243 void intel_power_domains_init_hw(struct drm_i915_private *dev_priv, bool resume);
216−void intel_power_domains_fini_hw(struct drm_i915_private *dev_priv);
244+void intel_power_domains_driver_remove(struct drm_i915_private *dev_priv);
217245 void icl_display_core_init(struct drm_i915_private *dev_priv, bool resume);
218246 void icl_display_core_uninit(struct drm_i915_private *dev_priv);
219247 void intel_power_domains_enable(struct drm_i915_private *dev_priv);
@@ -227,7 +255,8 @@ void bxt_display_core_init(struct drm_i915_private *dev_priv, bool resume); · 7 unmodified lines
227255 void bxt_display_core_uninit(struct drm_i915_private *dev_priv);
228256
229257 const char *
230−intel_display_power_domain_str(enum intel_display_power_domain domain);
258+intel_display_power_domain_str(struct drm_i915_private *i915,
259+ enum intel_display_power_domain domain);
231260
232261 bool intel_display_power_is_enabled(struct drm_i915_private *dev_priv,
233262 enum intel_display_power_domain domain);
drivers/gpu/drm/i915/display/intel_dp.c+45−247
@@ -62,6 +62,7 @@ · 61 unmodified lines
6262 #include "intel_panel.h"
6363 #include "intel_psr.h"
6464 #include "intel_sideband.h"
65+#include "intel_tc.h"
6566 #include "intel_vdsc.h"
6667
6768 #define DP_DPRX_ESI_LEN 14
@@ -211,47 +212,13 @@ static int intel_dp_max_common_rate(struct intel_dp *intel_dp) · 143 unmodified lines
211212 return intel_dp->common_rates[intel_dp->num_common_rates - 1];
212213 }
213214
214−static int intel_dp_get_fia_supported_lane_count(struct intel_dp *intel_dp)
215−{
216− struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
217− struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
218− enum tc_port tc_port = intel_port_to_tc(dev_priv, dig_port->base.port);
219− intel_wakeref_t wakeref;
220− u32 lane_info;
221−
222− if (tc_port == PORT_TC_NONE || dig_port->tc_type != TC_PORT_TYPEC)
223− return 4;
224−
225− lane_info = 0;
226− with_intel_display_power(dev_priv, POWER_DOMAIN_DISPLAY_CORE, wakeref)
227− lane_info = (I915_READ(PORT_TX_DFLEXDPSP) &
228− DP_LANE_ASSIGNMENT_MASK(tc_port)) >>
229− DP_LANE_ASSIGNMENT_SHIFT(tc_port);
230−
231− switch (lane_info) {
232− default:
233− MISSING_CASE(lane_info);
234− /* fall through */
235− case 1:
236− case 2:
237− case 4:
238− case 8:
239− return 1;
240− case 3:
241− case 12:
242− return 2;
243− case 15:
244− return 4;
245− }
246−}
247−
248215 /* Theoretical max between source and sink */
249216 static int intel_dp_max_common_lane_count(struct intel_dp *intel_dp)
250217 {
251218 struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
252219 int source_max = intel_dig_port->max_lanes;
253220 int sink_max = drm_dp_max_lane_count(intel_dp->dpcd);
254− int fia_max = intel_dp_get_fia_supported_lane_count(intel_dp);
221+ int fia_max = intel_tc_port_fia_max_lane_count(intel_dig_port);
255222
256223 return min3(source_max, sink_max, fia_max);
257224 }
@@ -330,9 +297,9 @@ static int icl_max_source_rate(struct intel_dp *intel_dp) · 72 unmodified lines
330297 {
331298 struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
332299 struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
333− enum port port = dig_port->base.port;
300+ enum phy phy = intel_port_to_phy(dev_priv, dig_port->base.port);
334301
335− if (intel_port_is_combophy(dev_priv, port) &&
302+ if (intel_phy_is_combo(dev_priv, phy) &&
336303 !IS_ELKHARTLAKE(dev_priv) &&
337304 !intel_dp_is_edp(intel_dp))
338305 return 540000;
@@ -1209,7 +1176,7 @@ static u32 skl_get_aux_send_ctl(struct intel_dp *intel_dp, · 870 unmodified lines
12091176 DP_AUX_CH_CTL_FW_SYNC_PULSE_SKL(32) |
12101177 DP_AUX_CH_CTL_SYNC_PULSE_SKL(32);
12111178
1212− if (intel_dig_port->tc_type == TC_PORT_TBT)
1179+ if (intel_dig_port->tc_mode == TC_PORT_TBT_ALT)
12131180 ret |= DP_AUX_CH_CTL_TBT_IO;
12141181
12151182 return ret;
@@ -1225,6 +1192,8 @@ intel_dp_aux_xfer(struct intel_dp *intel_dp, · 9 unmodified lines
12251192 struct drm_i915_private *i915 =
12261193 to_i915(intel_dig_port->base.base.dev);
12271194 struct intel_uncore *uncore = &i915->uncore;
1195+ enum phy phy = intel_port_to_phy(i915, intel_dig_port->base.port);
1196+ bool is_tc_port = intel_phy_is_tc(i915, phy);
12281197 i915_reg_t ch_ctl, ch_data[5];
12291198 u32 aux_clock_divider;
12301199 enum intel_display_power_domain aux_domain =
@@ -1240,6 +1209,9 @@ intel_dp_aux_xfer(struct intel_dp *intel_dp, · 9 unmodified lines
12401209 for (i = 0; i < ARRAY_SIZE(ch_data); i++)
12411210 ch_data[i] = intel_dp->aux_ch_data_reg(intel_dp, i);
12421211
1212+ if (is_tc_port)
1213+ intel_tc_port_lock(intel_dig_port);
1214+
12431215 aux_wakeref = intel_display_power_get(i915, aux_domain);
12441216 pps_wakeref = pps_lock(intel_dp);
12451217
@@ -1392,6 +1364,9 @@ out: · 146 unmodified lines
13921364 pps_unlock(intel_dp, pps_wakeref);
13931365 intel_display_power_put_async(i915, aux_domain, aux_wakeref);
13941366
1367+ if (is_tc_port)
1368+ intel_tc_port_unlock(intel_dig_port);
1369+
13951370 return ret;
13961371 }
13971372
@@ -1879,8 +1854,10 @@ intel_dp_compute_link_config_wide(struct intel_dp *intel_dp, · 481 unmodified lines
18791854 int mode_rate, link_clock, link_avail;
18801855
18811856 for (bpp = limits->max_bpp; bpp >= limits->min_bpp; bpp -= 2 * 3) {
1857+ int output_bpp = intel_dp_output_bpp(pipe_config, bpp);
1858+
18821859 mode_rate = intel_dp_link_required(adjusted_mode->crtc_clock,
1883− bpp);
1860+ output_bpp);
18841861
18851862 for (clock = limits->min_clock; clock <= limits->max_clock; clock++) {
18861863 for (lane_count = limits->min_lane_count;
@@ -4244,8 +4221,14 @@ intel_dp_get_dpcd(struct intel_dp *intel_dp) · 2357 unmodified lines
42444221 if (!intel_dp_read_dpcd(intel_dp))
42454222 return false;
42464223
4247− /* Don't clobber cached eDP rates. */
4224+ /*
4225+ * Don't clobber cached eDP rates. Also skip re-reading
4226+ * the OUI/ID since we know it won't change.
4227+ */
42484228 if (!intel_dp_is_edp(intel_dp)) {
4229+ drm_dp_read_desc(&intel_dp->aux, &intel_dp->desc,
4230+ drm_dp_is_branch(intel_dp->dpcd));
4231+
42494232 intel_dp_set_sink_rates(intel_dp);
42504233 intel_dp_set_common_rates(intel_dp);
42514234 }
@@ -4254,7 +4237,8 @@ intel_dp_get_dpcd(struct intel_dp *intel_dp) · 2 unmodified lines
42544237 * Some eDP panels do not set a valid value for sink count, that is why
42554238 * it don't care about read it here and in intel_edp_init_dpcd().
42564239 */
4257− if (!intel_dp_is_edp(intel_dp)) {
4240+ if (!intel_dp_is_edp(intel_dp) &&
4241+ !drm_dp_has_quirk(&intel_dp->desc, DP_DPCD_QUIRK_NO_SINK_COUNT)) {
42584242 u8 count;
42594243 ssize_t r;
42604244
@@ -4879,14 +4863,16 @@ int intel_dp_retrain_link(struct intel_encoder *encoder, · 618 unmodified lines
48794863 * retrain the link to get a picture. That's in case no
48804864 * userspace component reacted to intermittent HPD dip.
48814865 */
4882−static bool intel_dp_hotplug(struct intel_encoder *encoder,
4883− struct intel_connector *connector)
4866+static enum intel_hotplug_state
4867+intel_dp_hotplug(struct intel_encoder *encoder,
4868+ struct intel_connector *connector,
4869+ bool irq_received)
48844870 {
48854871 struct drm_modeset_acquire_ctx ctx;
4886− bool changed;
4872+ enum intel_hotplug_state state;
48874873 int ret;
48884874
4889− changed = intel_encoder_hotplug(encoder, connector);
4875+ state = intel_encoder_hotplug(encoder, connector, irq_received);
48904876
48914877 drm_modeset_acquire_init(&ctx, 0);
48924878
@@ -4905,7 +4891,14 @@ static bool intel_dp_hotplug(struct intel_encoder *encoder, · 12 unmodified lines
49054891 drm_modeset_acquire_fini(&ctx);
49064892 WARN(ret, "Acquiring modeset locks failed with %i\n", ret);
49074893
4908− return changed;
4894+ /*
4895+ * Keeping it consistent with intel_ddi_hotplug() and
4896+ * intel_hdmi_hotplug().
4897+ */
4898+ if (state == INTEL_HOTPLUG_UNCHANGED && irq_received)
4899+ state = INTEL_HOTPLUG_RETRY;
4900+
4901+ return state;
49094902 }
49104903
49114904 static void intel_dp_check_service_irq(struct intel_dp *intel_dp)
@@ -5233,204 +5226,16 @@ static bool icl_combo_port_connected(struct drm_i915_private *dev_priv, · 321 unmodified lines
52335226 return I915_READ(SDEISR) & SDE_DDI_HOTPLUG_ICP(port);
52345227 }
52355228
5236−static const char *tc_type_name(enum tc_port_type type)
5237−{
5238− static const char * const names[] = {
5239− [TC_PORT_UNKNOWN] = "unknown",
5240− [TC_PORT_LEGACY] = "legacy",
5241− [TC_PORT_TYPEC] = "typec",
5242− [TC_PORT_TBT] = "tbt",
5243− };
5244−
5245− if (WARN_ON(type >= ARRAY_SIZE(names)))
5246− type = TC_PORT_UNKNOWN;
5247−
5248− return names[type];
5249−}
5250−
5251−static void icl_update_tc_port_type(struct drm_i915_private *dev_priv,
5252− struct intel_digital_port *intel_dig_port,
5253− bool is_legacy, bool is_typec, bool is_tbt)
5254−{
5255− enum port port = intel_dig_port->base.port;
5256− enum tc_port_type old_type = intel_dig_port->tc_type;
5257−
5258− WARN_ON(is_legacy + is_typec + is_tbt != 1);
5259−
5260− if (is_legacy)
5261− intel_dig_port->tc_type = TC_PORT_LEGACY;
5262− else if (is_typec)
5263− intel_dig_port->tc_type = TC_PORT_TYPEC;
5264− else if (is_tbt)
5265− intel_dig_port->tc_type = TC_PORT_TBT;
5266− else
5267− return;
5268−
5269− /* Types are not supposed to be changed at runtime. */
5270− WARN_ON(old_type != TC_PORT_UNKNOWN &&
5271− old_type != intel_dig_port->tc_type);
5272−
5273− if (old_type != intel_dig_port->tc_type)
5274− DRM_DEBUG_KMS("Port %c has TC type %s\n", port_name(port),
5275− tc_type_name(intel_dig_port->tc_type));
5276−}
5277−
5278−/*
5279− * This function implements the first part of the Connect Flow described by our
5280− * specification, Gen11 TypeC Programming chapter. The rest of the flow (reading
5281− * lanes, EDID, etc) is done as needed in the typical places.
5282− *
5283− * Unlike the other ports, type-C ports are not available to use as soon as we
5284− * get a hotplug. The type-C PHYs can be shared between multiple controllers:
5285− * display, USB, etc. As a result, handshaking through FIA is required around
5286− * connect and disconnect to cleanly transfer ownership with the controller and
5287− * set the type-C power state.
5288− *
5289− * We could opt to only do the connect flow when we actually try to use the AUX
5290− * channels or do a modeset, then immediately run the disconnect flow after
5291− * usage, but there are some implications on this for a dynamic environment:
5292− * things may go away or change behind our backs. So for now our driver is
5293− * always trying to acquire ownership of the controller as soon as it gets an
5294− * interrupt (or polls state and sees a port is connected) and only gives it
5295− * back when it sees a disconnect. Implementation of a more fine-grained model
5296− * will require a lot of coordination with user space and thorough testing for
5297− * the extra possible cases.
5298− */
5299−static bool icl_tc_phy_connect(struct drm_i915_private *dev_priv,
5300− struct intel_digital_port *dig_port)
5301−{
5302− enum tc_port tc_port = intel_port_to_tc(dev_priv, dig_port->base.port);
5303− u32 val;
5304−
5305− if (dig_port->tc_type != TC_PORT_LEGACY &&
5306− dig_port->tc_type != TC_PORT_TYPEC)
5307− return true;
5308−
5309− val = I915_READ(PORT_TX_DFLEXDPPMS);
5310− if (!(val & DP_PHY_MODE_STATUS_COMPLETED(tc_port))) {
5311− DRM_DEBUG_KMS("DP PHY for TC port %d not ready\n", tc_port);
5312− WARN_ON(dig_port->tc_legacy_port);
5313− return false;
5314− }
5315−
5316− /*
5317− * This function may be called many times in a row without an HPD event
5318− * in between, so try to avoid the write when we can.
5319− */
5320− val = I915_READ(PORT_TX_DFLEXDPCSSS);
5321− if (!(val & DP_PHY_MODE_STATUS_NOT_SAFE(tc_port))) {
5322− val |= DP_PHY_MODE_STATUS_NOT_SAFE(tc_port);
5323− I915_WRITE(PORT_TX_DFLEXDPCSSS, val);
5324− }
5325−
5326− /*
5327− * Now we have to re-check the live state, in case the port recently
5328− * became disconnected. Not necessary for legacy mode.
5329− */
5330− if (dig_port->tc_type == TC_PORT_TYPEC &&
5331− !(I915_READ(PORT_TX_DFLEXDPSP) & TC_LIVE_STATE_TC(tc_port))) {
5332− DRM_DEBUG_KMS("TC PHY %d sudden disconnect.\n", tc_port);
5333− icl_tc_phy_disconnect(dev_priv, dig_port);
5334− return false;
5335− }
5336−
5337− return true;
5338−}
5339−
5340−/*
5341− * See the comment at the connect function. This implements the Disconnect
5342− * Flow.
5343− */
5344−void icl_tc_phy_disconnect(struct drm_i915_private *dev_priv,
5345− struct intel_digital_port *dig_port)
5346−{
5347− enum tc_port tc_port = intel_port_to_tc(dev_priv, dig_port->base.port);
5348−
5349− if (dig_port->tc_type == TC_PORT_UNKNOWN)
5350− return;
5351−
5352− /*
5353− * TBT disconnection flow is read the live status, what was done in
5354− * caller.
5355− */
5356− if (dig_port->tc_type == TC_PORT_TYPEC ||
5357− dig_port->tc_type == TC_PORT_LEGACY) {
5358− u32 val;
5359−
5360− val = I915_READ(PORT_TX_DFLEXDPCSSS);
5361− val &= ~DP_PHY_MODE_STATUS_NOT_SAFE(tc_port);
5362− I915_WRITE(PORT_TX_DFLEXDPCSSS, val);
5363− }
5364−
5365− DRM_DEBUG_KMS("Port %c TC type %s disconnected\n",
5366− port_name(dig_port->base.port),
5367− tc_type_name(dig_port->tc_type));
5368−
5369− dig_port->tc_type = TC_PORT_UNKNOWN;
5370−}
5371−
5372−/*
5373− * The type-C ports are different because even when they are connected, they may
5374− * not be available/usable by the graphics driver: see the comment on
5375− * icl_tc_phy_connect(). So in our driver instead of adding the additional
5376− * concept of "usable" and make everything check for "connected and usable" we
5377− * define a port as "connected" when it is not only connected, but also when it
5378− * is usable by the rest of the driver. That maintains the old assumption that
5379− * connected ports are usable, and avoids exposing to the users objects they
5380− * can't really use.
5381− */
5382−static bool icl_tc_port_connected(struct drm_i915_private *dev_priv,
5383− struct intel_digital_port *intel_dig_port)
5384−{
5385− enum port port = intel_dig_port->base.port;
5386− enum tc_port tc_port = intel_port_to_tc(dev_priv, port);
5387− bool is_legacy, is_typec, is_tbt;
5388− u32 dpsp;
5389−
5390− /*
5391− * Complain if we got a legacy port HPD, but VBT didn't mark the port as
5392− * legacy. Treat the port as legacy from now on.
5393− */
5394− if (!intel_dig_port->tc_legacy_port &&
5395− I915_READ(SDEISR) & SDE_TC_HOTPLUG_ICP(tc_port)) {
5396− DRM_ERROR("VBT incorrectly claims port %c is not TypeC legacy\n",
5397− port_name(port));
5398− intel_dig_port->tc_legacy_port = true;
5399− }
5400− is_legacy = intel_dig_port->tc_legacy_port;
5401−
5402− /*
5403− * The spec says we shouldn't be using the ISR bits for detecting
5404− * between TC and TBT. We should use DFLEXDPSP.
5405− */
5406− dpsp = I915_READ(PORT_TX_DFLEXDPSP);
5407− is_typec = dpsp & TC_LIVE_STATE_TC(tc_port);
5408− is_tbt = dpsp & TC_LIVE_STATE_TBT(tc_port);
5409−
5410− if (!is_legacy && !is_typec && !is_tbt) {
5411− icl_tc_phy_disconnect(dev_priv, intel_dig_port);
5412−
5413− return false;
5414− }
5415−
5416− icl_update_tc_port_type(dev_priv, intel_dig_port, is_legacy, is_typec,
5417− is_tbt);
5418−
5419− if (!icl_tc_phy_connect(dev_priv, intel_dig_port))
5420− return false;
5421−
5422− return true;
5423−}
5424−
54255229 static bool icl_digital_port_connected(struct intel_encoder *encoder)
54265230 {
54275231 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
54285232 struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
5233+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
54295234
5430− if (intel_port_is_combophy(dev_priv, encoder->port))
5235+ if (intel_phy_is_combo(dev_priv, phy))
54315236 return icl_combo_port_connected(dev_priv, dig_port);
5432− else if (intel_port_is_tc(dev_priv, encoder->port))
5433− return icl_tc_port_connected(dev_priv, dig_port);
5237+ else if (intel_phy_is_tc(dev_priv, phy))
5238+ return intel_tc_port_connected(dig_port);
54345239 else
54355240 MISSING_CASE(encoder->hpd_pin);
54365241
@@ -5588,9 +5393,6 @@ intel_dp_detect(struct drm_connector *connector, · 151 unmodified lines
55885393 if (INTEL_GEN(dev_priv) >= 11)
55895394 intel_dp_get_dsc_sink_cap(intel_dp);
55905395
5591− drm_dp_read_desc(&intel_dp->aux, &intel_dp->desc,
5592− drm_dp_is_branch(intel_dp->dpcd));
5593−
55945396 intel_dp_configure_mst(intel_dp);
55955397
55965398 if (intel_dp->is_mst) {
@@ -6835,8 +6637,6 @@ static void intel_dp_set_drrs_state(struct drm_i915_private *dev_priv, · 1238 unmodified lines
68356637 const struct intel_crtc_state *crtc_state,
68366638 int refresh_rate)
68376639 {
6838− struct intel_encoder *encoder;
6839− struct intel_digital_port *dig_port = NULL;
68406640 struct intel_dp *intel_dp = dev_priv->drrs.dp;
68416641 struct intel_crtc *intel_crtc = to_intel_crtc(crtc_state->base.crtc);
68426642 enum drrs_refresh_rate_type index = DRRS_HIGH_RR;
@@ -6851,9 +6651,6 @@ static void intel_dp_set_drrs_state(struct drm_i915_private *dev_priv, · 8 unmodified lines
68516651 return;
68526652 }
68536653
6854− dig_port = dp_to_dig_port(intel_dp);
6855− encoder = &dig_port->base;
6856−
68576654 if (!intel_crtc) {
68586655 DRM_DEBUG_KMS("DRRS: intel_crtc not initialized\n");
68596656 return;
@@ -7333,6 +7130,7 @@ intel_dp_init_connector(struct intel_digital_port *intel_dig_port, · 473 unmodified lines
73337130 struct drm_device *dev = intel_encoder->base.dev;
73347131 struct drm_i915_private *dev_priv = to_i915(dev);
73357132 enum port port = intel_encoder->port;
7133+ enum phy phy = intel_port_to_phy(dev_priv, port);
73367134 int type;
73377135
73387136 /* Initialize the work for modeset in case of link train failure */
@@ -7359,7 +7157,7 @@ intel_dp_init_connector(struct intel_digital_port *intel_dig_port, · 20 unmodified lines
73597157 * Currently we don't support eDP on TypeC ports, although in
73607158 * theory it could work on TypeC legacy ports.
73617159 */
7362− WARN_ON(intel_port_is_tc(dev_priv, port));
7160+ WARN_ON(intel_phy_is_tc(dev_priv, phy));
73637161 type = DRM_MODE_CONNECTOR_eDP;
73647162 } else {
73657163 type = DRM_MODE_CONNECTOR_DisplayPort;
drivers/gpu/drm/i915/display/intel_dp.h+0−2
@@ -112,8 +112,6 @@ bool intel_dp_get_colorimetry_status(struct intel_dp *intel_dp); · 111 unmodified lines
112112 int intel_dp_link_required(int pixel_clock, int bpp);
113113 int intel_dp_max_data_rate(int max_link_clock, int max_lanes);
114114 bool intel_digital_port_connected(struct intel_encoder *encoder);
115−void icl_tc_phy_disconnect(struct drm_i915_private *dev_priv,
116− struct intel_digital_port *dig_port);
117115
118116 static inline unsigned int intel_dp_unused_lane_mask(int lane_count)
119117 {
drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c+4−1
@@ -264,8 +264,11 @@ intel_dp_aux_display_control_capable(struct intel_connector *connector) · 263 unmodified lines
264264 int intel_dp_aux_init_backlight_funcs(struct intel_connector *intel_connector)
265265 {
266266 struct intel_panel *panel = &intel_connector->panel;
267+ struct drm_i915_private *dev_priv = to_i915(intel_connector->base.dev);
267268
268− if (!i915_modparams.enable_dpcd_backlight)
269+ if (i915_modparams.enable_dpcd_backlight == 0 ||
270+ (i915_modparams.enable_dpcd_backlight == -1 &&
271+ dev_priv->vbt.backlight.type != INTEL_BACKLIGHT_VESA_EDP_AUX_INTERFACE))
269272 return -ENODEV;
270273
271274 if (!intel_dp_aux_display_control_capable(intel_connector))
drivers/gpu/drm/i915/display/intel_dp_mst.h+7−1
@@ -6,9 +6,15 @@ · 5 unmodified lines
66 #ifndef __INTEL_DP_MST_H__
77 #define __INTEL_DP_MST_H__
88
9−struct intel_digital_port;
9+#include "intel_drv.h"
1010
1111 int intel_dp_mst_encoder_init(struct intel_digital_port *intel_dig_port, int conn_id);
1212 void intel_dp_mst_encoder_cleanup(struct intel_digital_port *intel_dig_port);
13+static inline int
14+intel_dp_mst_encoder_active_links(struct intel_digital_port *intel_dig_port)
15+{
16+ return intel_dig_port->dp.active_mst_links;
17+}
18+
1319
1420 #endif /* __INTEL_DP_MST_H__ */
drivers/gpu/drm/i915/display/intel_dpll_mgr.c+478−174
@@ -36,9 +36,10 @@ · 35 unmodified lines
3636 * This file provides an abstraction over display PLLs. The function
3737 * intel_shared_dpll_init() initializes the PLLs for the given platform. The
3838 * users of a PLL are tracked and that tracking is integrated with the atomic
39− * modest interface. During an atomic operation, a PLL can be requested for a
40− * given CRTC and encoder configuration by calling intel_get_shared_dpll() and
41− * a previously used PLL can be released with intel_release_shared_dpll().
39+ * modset interface. During an atomic operation, required PLLs can be reserved
40+ * for a given CRTC and encoder configuration by calling
41+ * intel_reserve_shared_dplls() and previously reserved PLLs can be released
42+ * with intel_release_shared_dplls().
4243 * Changes to the users are first staged in the atomic state, and then made
4344 * effective by calling intel_shared_dpll_swap_state() during the atomic
4445 * commit phase.
@@ -243,17 +244,18 @@ out: · 198 unmodified lines
243244 }
244245
245246 static struct intel_shared_dpll *
246−intel_find_shared_dpll(struct intel_crtc_state *crtc_state,
247+intel_find_shared_dpll(struct intel_atomic_state *state,
248+ const struct intel_crtc *crtc,
249+ const struct intel_dpll_hw_state *pll_state,
247250 enum intel_dpll_id range_min,
248251 enum intel_dpll_id range_max)
249252 {
250− struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
251253 struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
252254 struct intel_shared_dpll *pll, *unused_pll = NULL;
253255 struct intel_shared_dpll_state *shared_dpll;
254256 enum intel_dpll_id i;
255257
256− shared_dpll = intel_atomic_get_shared_dpll_state(crtc_state->base.state);
258+ shared_dpll = intel_atomic_get_shared_dpll_state(&state->base);
257259
258260 for (i = range_min; i <= range_max; i++) {
259261 pll = &dev_priv->shared_dplls[i];
@@ -265,9 +267,9 @@ intel_find_shared_dpll(struct intel_crtc_state *crtc_state, · 5 unmodified lines
265267 continue;
266268 }
267269
268− if (memcmp(&crtc_state->dpll_hw_state,
270+ if (memcmp(pll_state,
269271 &shared_dpll[i].hw_state,
270− sizeof(crtc_state->dpll_hw_state)) == 0) {
272+ sizeof(*pll_state)) == 0) {
271273 DRM_DEBUG_KMS("[CRTC:%d:%s] sharing existing %s (crtc mask 0x%08x, active %x)\n",
272274 crtc->base.base.id, crtc->base.name,
273275 pll->info->name,
@@ -289,26 +291,51 @@ intel_find_shared_dpll(struct intel_crtc_state *crtc_state, · 15 unmodified lines
289291 }
290292
291293 static void
292−intel_reference_shared_dpll(struct intel_shared_dpll *pll,
293− struct intel_crtc_state *crtc_state)
294+intel_reference_shared_dpll(struct intel_atomic_state *state,
295+ const struct intel_crtc *crtc,
296+ const struct intel_shared_dpll *pll,
297+ const struct intel_dpll_hw_state *pll_state)
294298 {
295299 struct intel_shared_dpll_state *shared_dpll;
296− struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
297300 const enum intel_dpll_id id = pll->info->id;
298301
299− shared_dpll = intel_atomic_get_shared_dpll_state(crtc_state->base.state);
302+ shared_dpll = intel_atomic_get_shared_dpll_state(&state->base);
300303
301304 if (shared_dpll[id].crtc_mask == 0)
302− shared_dpll[id].hw_state =
303− crtc_state->dpll_hw_state;
305+ shared_dpll[id].hw_state = *pll_state;
304306
305− crtc_state->shared_dpll = pll;
306307 DRM_DEBUG_DRIVER("using %s for pipe %c\n", pll->info->name,
307308 pipe_name(crtc->pipe));
308309
309310 shared_dpll[id].crtc_mask |= 1 << crtc->pipe;
310311 }
311312
313+static void intel_unreference_shared_dpll(struct intel_atomic_state *state,
314+ const struct intel_crtc *crtc,
315+ const struct intel_shared_dpll *pll)
316+{
317+ struct intel_shared_dpll_state *shared_dpll;
318+
319+ shared_dpll = intel_atomic_get_shared_dpll_state(&state->base);
320+ shared_dpll[pll->info->id].crtc_mask &= ~(1 << crtc->pipe);
321+}
322+
323+static void intel_put_dpll(struct intel_atomic_state *state,
324+ struct intel_crtc *crtc)
325+{
326+ const struct intel_crtc_state *old_crtc_state =
327+ intel_atomic_get_old_crtc_state(state, crtc);
328+ struct intel_crtc_state *new_crtc_state =
329+ intel_atomic_get_new_crtc_state(state, crtc);
330+
331+ new_crtc_state->shared_dpll = NULL;
332+
333+ if (!old_crtc_state->shared_dpll)
334+ return;
335+
336+ intel_unreference_shared_dpll(state, crtc, old_crtc_state->shared_dpll);
337+}
338+
312339 /**
313340 * intel_shared_dpll_swap_state - make atomic DPLL configuration effective
314341 * @state: atomic state
@@ -320,25 +347,20 @@ intel_reference_shared_dpll(struct intel_shared_dpll *pll, · 5 unmodified lines
320347 * i.e. it also puts the current state into @state, even though there is no
321348 * need for that at this moment.
322349 */
323−void intel_shared_dpll_swap_state(struct drm_atomic_state *state)
350+void intel_shared_dpll_swap_state(struct intel_atomic_state *state)
324351 {
325− struct drm_i915_private *dev_priv = to_i915(state->dev);
326− struct intel_shared_dpll_state *shared_dpll;
327− struct intel_shared_dpll *pll;
352+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
353+ struct intel_shared_dpll_state *shared_dpll = state->shared_dpll;
328354 enum intel_dpll_id i;
329355
330− if (!to_intel_atomic_state(state)->dpll_set)
356+ if (!state->dpll_set)
331357 return;
332358
333− shared_dpll = to_intel_atomic_state(state)->shared_dpll;
334359 for (i = 0; i < dev_priv->num_shared_dpll; i++) {
335− struct intel_shared_dpll_state tmp;
360+ struct intel_shared_dpll *pll =
361+ &dev_priv->shared_dplls[i];
336362
337− pll = &dev_priv->shared_dplls[i];
338−
339− tmp = pll->state;
340− pll->state = shared_dpll[i];
341− shared_dpll[i] = tmp;
363+ swap(pll->state, shared_dpll[i]);
342364 }
343365 }
344366
@@ -421,11 +443,12 @@ static void ibx_pch_dpll_disable(struct drm_i915_private *dev_priv, · 76 unmodified lines
421443 udelay(200);
422444 }
423445
424−static struct intel_shared_dpll *
425−ibx_get_dpll(struct intel_crtc_state *crtc_state,
426− struct intel_encoder *encoder)
446+static bool ibx_get_dpll(struct intel_atomic_state *state,
447+ struct intel_crtc *crtc,
448+ struct intel_encoder *encoder)
427449 {
428− struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
450+ struct intel_crtc_state *crtc_state =
451+ intel_atomic_get_new_crtc_state(state, crtc);
429452 struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
430453 struct intel_shared_dpll *pll;
431454 enum intel_dpll_id i;
@@ -439,18 +462,22 @@ ibx_get_dpll(struct intel_crtc_state *crtc_state, · 7 unmodified lines
439462 crtc->base.base.id, crtc->base.name,
440463 pll->info->name);
441464 } else {
442− pll = intel_find_shared_dpll(crtc_state,
465+ pll = intel_find_shared_dpll(state, crtc,
466+ &crtc_state->dpll_hw_state,
443467 DPLL_ID_PCH_PLL_A,
444468 DPLL_ID_PCH_PLL_B);
445469 }
446470
447471 if (!pll)
448− return NULL;
472+ return false;
449473
450474 /* reference the pll */
451− intel_reference_shared_dpll(pll, crtc_state);
475+ intel_reference_shared_dpll(state, crtc,
476+ pll, &crtc_state->dpll_hw_state);
452477
453− return pll;
478+ crtc_state->shared_dpll = pll;
479+
480+ return true;
454481 }
455482
456483 static void ibx_dump_hw_state(struct drm_i915_private *dev_priv,
@@ -767,8 +794,12 @@ hsw_ddi_calculate_wrpll(int clock /* in Hz */, · 310 unmodified lines
767794 *r2_out = best.r2;
768795 }
769796
770−static struct intel_shared_dpll *hsw_ddi_hdmi_get_dpll(struct intel_crtc_state *crtc_state)
797+static struct intel_shared_dpll *
798+hsw_ddi_hdmi_get_dpll(struct intel_atomic_state *state,
799+ struct intel_crtc *crtc)
771800 {
801+ struct intel_crtc_state *crtc_state =
802+ intel_atomic_get_new_crtc_state(state, crtc);
772803 struct intel_shared_dpll *pll;
773804 u32 val;
774805 unsigned int p, n2, r2;
@@ -781,7 +812,8 @@ static struct intel_shared_dpll *hsw_ddi_hdmi_get_dpll(struct intel_crtc_state * · 6 unmodified lines
781812
782813 crtc_state->dpll_hw_state.wrpll = val;
783814
784− pll = intel_find_shared_dpll(crtc_state,
815+ pll = intel_find_shared_dpll(state, crtc,
816+ &crtc_state->dpll_hw_state,
785817 DPLL_ID_WRPLL1, DPLL_ID_WRPLL2);
786818
787819 if (!pll)
@@ -821,38 +853,44 @@ hsw_ddi_dp_get_dpll(struct intel_crtc_state *crtc_state) · 33 unmodified lines
821853 return pll;
822854 }
823855
824−static struct intel_shared_dpll *
825−hsw_get_dpll(struct intel_crtc_state *crtc_state,
826− struct intel_encoder *encoder)
856+static bool hsw_get_dpll(struct intel_atomic_state *state,
857+ struct intel_crtc *crtc,
858+ struct intel_encoder *encoder)
827859 {
860+ struct intel_crtc_state *crtc_state =
861+ intel_atomic_get_new_crtc_state(state, crtc);
828862 struct intel_shared_dpll *pll;
829863
830864 memset(&crtc_state->dpll_hw_state, 0,
831865 sizeof(crtc_state->dpll_hw_state));
832866
833867 if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI)) {
834− pll = hsw_ddi_hdmi_get_dpll(crtc_state);
868+ pll = hsw_ddi_hdmi_get_dpll(state, crtc);
835869 } else if (intel_crtc_has_dp_encoder(crtc_state)) {
836870 pll = hsw_ddi_dp_get_dpll(crtc_state);
837871 } else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_ANALOG)) {
838872 if (WARN_ON(crtc_state->port_clock / 2 != 135000))
839− return NULL;
873+ return false;
840874
841875 crtc_state->dpll_hw_state.spll =
842876 SPLL_PLL_ENABLE | SPLL_FREQ_1350MHz | SPLL_REF_MUXED_SSC;
843877
844− pll = intel_find_shared_dpll(crtc_state,
878+ pll = intel_find_shared_dpll(state, crtc,
879+ &crtc_state->dpll_hw_state,
845880 DPLL_ID_SPLL, DPLL_ID_SPLL);
846881 } else {
847− return NULL;
882+ return false;
848883 }
849884
850885 if (!pll)
851− return NULL;
886+ return false;
852887
853− intel_reference_shared_dpll(pll, crtc_state);
888+ intel_reference_shared_dpll(state, crtc,
889+ pll, &crtc_state->dpll_hw_state);
854890
855− return pll;
891+ crtc_state->shared_dpll = pll;
892+
893+ return true;
856894 }
857895
858896 static void hsw_dump_hw_state(struct drm_i915_private *dev_priv,
@@ -1385,10 +1423,12 @@ skl_ddi_dp_set_dpll_hw_state(struct intel_crtc_state *crtc_state) · 526 unmodified lines
13851423 return true;
13861424 }
13871425
1388−static struct intel_shared_dpll *
1389−skl_get_dpll(struct intel_crtc_state *crtc_state,
1390− struct intel_encoder *encoder)
1426+static bool skl_get_dpll(struct intel_atomic_state *state,
1427+ struct intel_crtc *crtc,
1428+ struct intel_encoder *encoder)
13911429 {
1430+ struct intel_crtc_state *crtc_state =
1431+ intel_atomic_get_new_crtc_state(state, crtc);
13921432 struct intel_shared_dpll *pll;
13931433 bool bret;
13941434
@@ -1396,32 +1436,37 @@ skl_get_dpll(struct intel_crtc_state *crtc_state, · 1 unmodified line
13961436 bret = skl_ddi_hdmi_pll_dividers(crtc_state);
13971437 if (!bret) {
13981438 DRM_DEBUG_KMS("Could not get HDMI pll dividers.\n");
1399− return NULL;
1439+ return false;
14001440 }
14011441 } else if (intel_crtc_has_dp_encoder(crtc_state)) {
14021442 bret = skl_ddi_dp_set_dpll_hw_state(crtc_state);
14031443 if (!bret) {
14041444 DRM_DEBUG_KMS("Could not set DP dpll HW state.\n");
1405− return NULL;
1445+ return false;
14061446 }
14071447 } else {
1408− return NULL;
1448+ return false;
14091449 }
14101450
14111451 if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_EDP))
1412− pll = intel_find_shared_dpll(crtc_state,
1452+ pll = intel_find_shared_dpll(state, crtc,
1453+ &crtc_state->dpll_hw_state,
14131454 DPLL_ID_SKL_DPLL0,
14141455 DPLL_ID_SKL_DPLL0);
14151456 else
1416− pll = intel_find_shared_dpll(crtc_state,
1457+ pll = intel_find_shared_dpll(state, crtc,
1458+ &crtc_state->dpll_hw_state,
14171459 DPLL_ID_SKL_DPLL1,
14181460 DPLL_ID_SKL_DPLL3);
14191461 if (!pll)
1420− return NULL;
1462+ return false;
14211463
1422− intel_reference_shared_dpll(pll, crtc_state);
1464+ intel_reference_shared_dpll(state, crtc,
1465+ pll, &crtc_state->dpll_hw_state);
14231466
1424− return pll;
1467+ crtc_state->shared_dpll = pll;
1468+
1469+ return true;
14251470 }
14261471
14271472 static void skl_dump_hw_state(struct drm_i915_private *dev_priv,
@@ -1827,22 +1872,23 @@ bxt_ddi_hdmi_set_dpll_hw_state(struct intel_crtc_state *crtc_state) · 399 unmodified lines
18271872 return bxt_ddi_set_dpll_hw_state(crtc_state, &clk_div);
18281873 }
18291874
1830−static struct intel_shared_dpll *
1831−bxt_get_dpll(struct intel_crtc_state *crtc_state,
1832− struct intel_encoder *encoder)
1875+static bool bxt_get_dpll(struct intel_atomic_state *state,
1876+ struct intel_crtc *crtc,
1877+ struct intel_encoder *encoder)
18331878 {
1834− struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
1879+ struct intel_crtc_state *crtc_state =
1880+ intel_atomic_get_new_crtc_state(state, crtc);
18351881 struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
18361882 struct intel_shared_dpll *pll;
18371883 enum intel_dpll_id id;
18381884
18391885 if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI) &&
18401886 !bxt_ddi_hdmi_set_dpll_hw_state(crtc_state))
1841− return NULL;
1887+ return false;
18421888
18431889 if (intel_crtc_has_dp_encoder(crtc_state) &&
18441890 !bxt_ddi_dp_set_dpll_hw_state(crtc_state))
1845− return NULL;
1891+ return false;
18461892
18471893 /* 1:1 mapping between ports and PLLs */
18481894 id = (enum intel_dpll_id) encoder->port;
@@ -1851,9 +1897,12 @@ bxt_get_dpll(struct intel_crtc_state *crtc_state, · 2 unmodified lines
18511897 DRM_DEBUG_KMS("[CRTC:%d:%s] using pre-allocated %s\n",
18521898 crtc->base.base.id, crtc->base.name, pll->info->name);
18531899
1854− intel_reference_shared_dpll(pll, crtc_state);
1900+ intel_reference_shared_dpll(state, crtc,
1901+ pll, &crtc_state->dpll_hw_state);
18551902
1856− return pll;
1903+ crtc_state->shared_dpll = pll;
1904+
1905+ return true;
18571906 }
18581907
18591908 static void bxt_dump_hw_state(struct drm_i915_private *dev_priv,
@@ -1884,9 +1933,14 @@ static const struct intel_shared_dpll_funcs bxt_ddi_pll_funcs = { · 24 unmodified lines
18841933 struct intel_dpll_mgr {
18851934 const struct dpll_info *dpll_info;
18861935
1887− struct intel_shared_dpll *(*get_dpll)(struct intel_crtc_state *crtc_state,
1888− struct intel_encoder *encoder);
1889−
1936+ bool (*get_dplls)(struct intel_atomic_state *state,
1937+ struct intel_crtc *crtc,
1938+ struct intel_encoder *encoder);
1939+ void (*put_dplls)(struct intel_atomic_state *state,
1940+ struct intel_crtc *crtc);
1941+ void (*update_active_dpll)(struct intel_atomic_state *state,
1942+ struct intel_crtc *crtc,
1943+ struct intel_encoder *encoder);
18901944 void (*dump_hw_state)(struct drm_i915_private *dev_priv,
18911945 const struct intel_dpll_hw_state *hw_state);
18921946 };
@@ -1899,7 +1953,8 @@ static const struct dpll_info pch_plls[] = { · 6 unmodified lines
18991953
19001954 static const struct intel_dpll_mgr pch_pll_mgr = {
19011955 .dpll_info = pch_plls,
1902− .get_dpll = ibx_get_dpll,
1956+ .get_dplls = ibx_get_dpll,
1957+ .put_dplls = intel_put_dpll,
19031958 .dump_hw_state = ibx_dump_hw_state,
19041959 };
19051960
@@ -1915,7 +1970,8 @@ static const struct dpll_info hsw_plls[] = { · 9 unmodified lines
19151970
19161971 static const struct intel_dpll_mgr hsw_pll_mgr = {
19171972 .dpll_info = hsw_plls,
1918− .get_dpll = hsw_get_dpll,
1973+ .get_dplls = hsw_get_dpll,
1974+ .put_dplls = intel_put_dpll,
19191975 .dump_hw_state = hsw_dump_hw_state,
19201976 };
19211977
@@ -1929,7 +1985,8 @@ static const struct dpll_info skl_plls[] = { · 7 unmodified lines
19291985
19301986 static const struct intel_dpll_mgr skl_pll_mgr = {
19311987 .dpll_info = skl_plls,
1932− .get_dpll = skl_get_dpll,
1988+ .get_dplls = skl_get_dpll,
1989+ .put_dplls = intel_put_dpll,
19331990 .dump_hw_state = skl_dump_hw_state,
19341991 };
19351992
@@ -1942,7 +1999,8 @@ static const struct dpll_info bxt_plls[] = { · 6 unmodified lines
19421999
19432000 static const struct intel_dpll_mgr bxt_pll_mgr = {
19442001 .dpll_info = bxt_plls,
1945− .get_dpll = bxt_get_dpll,
2002+ .get_dplls = bxt_get_dpll,
2003+ .put_dplls = intel_put_dpll,
19462004 .dump_hw_state = bxt_dump_hw_state,
19472005 };
19482006
@@ -2332,10 +2390,12 @@ cnl_ddi_dp_set_dpll_hw_state(struct intel_crtc_state *crtc_state) · 383 unmodified lines
23322390 return true;
23332391 }
23342392
2335−static struct intel_shared_dpll *
2336−cnl_get_dpll(struct intel_crtc_state *crtc_state,
2337− struct intel_encoder *encoder)
2393+static bool cnl_get_dpll(struct intel_atomic_state *state,
2394+ struct intel_crtc *crtc,
2395+ struct intel_encoder *encoder)
23382396 {
2397+ struct intel_crtc_state *crtc_state =
2398+ intel_atomic_get_new_crtc_state(state, crtc);
23392399 struct intel_shared_dpll *pll;
23402400 bool bret;
23412401
@@ -2343,31 +2403,35 @@ cnl_get_dpll(struct intel_crtc_state *crtc_state, · 1 unmodified line
23432403 bret = cnl_ddi_hdmi_pll_dividers(crtc_state);
23442404 if (!bret) {
23452405 DRM_DEBUG_KMS("Could not get HDMI pll dividers.\n");
2346− return NULL;
2406+ return false;
23472407 }
23482408 } else if (intel_crtc_has_dp_encoder(crtc_state)) {
23492409 bret = cnl_ddi_dp_set_dpll_hw_state(crtc_state);
23502410 if (!bret) {
23512411 DRM_DEBUG_KMS("Could not set DP dpll HW state.\n");
2352− return NULL;
2412+ return false;
23532413 }
23542414 } else {
23552415 DRM_DEBUG_KMS("Skip DPLL setup for output_types 0x%x\n",
23562416 crtc_state->output_types);
2357− return NULL;
2417+ return false;
23582418 }
23592419
2360− pll = intel_find_shared_dpll(crtc_state,
2420+ pll = intel_find_shared_dpll(state, crtc,
2421+ &crtc_state->dpll_hw_state,
23612422 DPLL_ID_SKL_DPLL0,
23622423 DPLL_ID_SKL_DPLL2);
23632424 if (!pll) {
23642425 DRM_DEBUG_KMS("No PLL selected\n");
2365− return NULL;
2426+ return false;
23662427 }
23672428
2368− intel_reference_shared_dpll(pll, crtc_state);
2429+ intel_reference_shared_dpll(state, crtc,
2430+ pll, &crtc_state->dpll_hw_state);
23692431
2370− return pll;
2432+ crtc_state->shared_dpll = pll;
2433+
2434+ return true;
23712435 }
23722436
23732437 static void cnl_dump_hw_state(struct drm_i915_private *dev_priv,
@@ -2394,7 +2458,8 @@ static const struct dpll_info cnl_plls[] = { · 20 unmodified lines
23942458
23952459 static const struct intel_dpll_mgr cnl_pll_mgr = {
23962460 .dpll_info = cnl_plls,
2397− .get_dpll = cnl_get_dpll,
2461+ .get_dplls = cnl_get_dpll,
2462+ .put_dplls = intel_put_dpll,
23982463 .dump_hw_state = cnl_dump_hw_state,
23992464 };
24002465
@@ -2506,14 +2571,16 @@ static bool icl_calc_tbt_pll(struct intel_crtc_state *crtc_state, · 105 unmodified lines
25062571 }
25072572
25082573 static bool icl_calc_dpll_state(struct intel_crtc_state *crtc_state,
2509− struct intel_encoder *encoder)
2574+ struct intel_encoder *encoder,
2575+ struct intel_dpll_hw_state *pll_state)
25102576 {
25112577 struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
25122578 u32 cfgcr0, cfgcr1;
25132579 struct skl_wrpll_params pll_params = { 0 };
25142580 bool ret;
25152581
2516− if (intel_port_is_tc(dev_priv, encoder->port))
2582+ if (intel_phy_is_tc(dev_priv, intel_port_to_phy(dev_priv,
2583+ encoder->port)))
25172584 ret = icl_calc_tbt_pll(crtc_state, &pll_params);
25182585 else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI) ||
25192586 intel_crtc_has_type(crtc_state, INTEL_OUTPUT_DSI))
@@ -2530,14 +2597,17 @@ static bool icl_calc_dpll_state(struct intel_crtc_state *crtc_state, · 10 unmodified lines
25302597 cfgcr1 = DPLL_CFGCR1_QDIV_RATIO(pll_params.qdiv_ratio) |
25312598 DPLL_CFGCR1_QDIV_MODE(pll_params.qdiv_mode) |
25322599 DPLL_CFGCR1_KDIV(pll_params.kdiv) |
2533− DPLL_CFGCR1_PDIV(pll_params.pdiv) |
2534− DPLL_CFGCR1_CENTRAL_FREQ_8400;
2600+ DPLL_CFGCR1_PDIV(pll_params.pdiv);
25352601
2536− memset(&crtc_state->dpll_hw_state, 0,
2537− sizeof(crtc_state->dpll_hw_state));
2602+ if (INTEL_GEN(dev_priv) >= 12)
2603+ cfgcr1 |= TGL_DPLL_CFGCR1_CFSELOVRD_NORMAL_XTAL;
2604+ else
2605+ cfgcr1 |= DPLL_CFGCR1_CENTRAL_FREQ_8400;
25382606
2539− crtc_state->dpll_hw_state.cfgcr0 = cfgcr0;
2540− crtc_state->dpll_hw_state.cfgcr1 = cfgcr1;
2607+ memset(pll_state, 0, sizeof(*pll_state));
2608+
2609+ pll_state->cfgcr0 = cfgcr0;
2610+ pll_state->cfgcr1 = cfgcr1;
25412611
25422612 return true;
25432613 }
@@ -2627,10 +2697,10 @@ static bool icl_mg_pll_find_divisors(int clock_khz, bool is_dp, bool use_ssc, · 83 unmodified lines
26272697 * The specification for this function uses real numbers, so the math had to be
26282698 * adapted to integer-only calculation, that's why it looks so different.
26292699 */
2630−static bool icl_calc_mg_pll_state(struct intel_crtc_state *crtc_state)
2700+static bool icl_calc_mg_pll_state(struct intel_crtc_state *crtc_state,
2701+ struct intel_dpll_hw_state *pll_state)
26312702 {
26322703 struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
2633− struct intel_dpll_hw_state *pll_state = &crtc_state->dpll_hw_state;
26342704 int refclk_khz = dev_priv->cdclk.hw.ref;
26352705 int clock = crtc_state->port_clock;
26362706 u32 dco_khz, m1div, m2div_int, m2div_rem, m2div_frac;
@@ -2792,63 +2862,184 @@ static bool icl_calc_mg_pll_state(struct intel_crtc_state *crtc_state) · 155 unmodified lines
27922862 return true;
27932863 }
27942864
2795−static struct intel_shared_dpll *
2796−icl_get_dpll(struct intel_crtc_state *crtc_state,
2797− struct intel_encoder *encoder)
2865+/**
2866+ * icl_set_active_port_dpll - select the active port DPLL for a given CRTC
2867+ * @crtc_state: state for the CRTC to select the DPLL for
2868+ * @port_dpll_id: the active @port_dpll_id to select
2869+ *
2870+ * Select the given @port_dpll_id instance from the DPLLs reserved for the
2871+ * CRTC.
2872+ */
2873+void icl_set_active_port_dpll(struct intel_crtc_state *crtc_state,
2874+ enum icl_port_dpll_id port_dpll_id)
27982875 {
2799− struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
2800− struct intel_digital_port *intel_dig_port;
2801− struct intel_shared_dpll *pll;
2876+ struct icl_port_dpll *port_dpll =
2877+ &crtc_state->icl_port_dplls[port_dpll_id];
2878+
2879+ crtc_state->shared_dpll = port_dpll->pll;
2880+ crtc_state->dpll_hw_state = port_dpll->hw_state;
2881+}
2882+
2883+static void icl_update_active_dpll(struct intel_atomic_state *state,
2884+ struct intel_crtc *crtc,
2885+ struct intel_encoder *encoder)
2886+{
2887+ struct intel_crtc_state *crtc_state =
2888+ intel_atomic_get_new_crtc_state(state, crtc);
2889+ struct intel_digital_port *primary_port;
2890+ enum icl_port_dpll_id port_dpll_id = ICL_PORT_DPLL_DEFAULT;
2891+
2892+ primary_port = encoder->type == INTEL_OUTPUT_DP_MST ?
2893+ enc_to_mst(&encoder->base)->primary :
2894+ enc_to_dig_port(&encoder->base);
2895+
2896+ if (primary_port &&
2897+ (primary_port->tc_mode == TC_PORT_DP_ALT ||
2898+ primary_port->tc_mode == TC_PORT_LEGACY))
2899+ port_dpll_id = ICL_PORT_DPLL_MG_PHY;
2900+
2901+ icl_set_active_port_dpll(crtc_state, port_dpll_id);
2902+}
2903+
2904+static bool icl_get_combo_phy_dpll(struct intel_atomic_state *state,
2905+ struct intel_crtc *crtc,
2906+ struct intel_encoder *encoder)
2907+{
2908+ struct intel_crtc_state *crtc_state =
2909+ intel_atomic_get_new_crtc_state(state, crtc);
2910+ struct icl_port_dpll *port_dpll =
2911+ &crtc_state->icl_port_dplls[ICL_PORT_DPLL_DEFAULT];
2912+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
28022913 enum port port = encoder->port;
2803− enum intel_dpll_id min, max;
2804− bool ret;
2914+ bool has_dpll4 = false;
28052915
2806− if (intel_port_is_combophy(dev_priv, port)) {
2807− min = DPLL_ID_ICL_DPLL0;
2808− max = DPLL_ID_ICL_DPLL1;
2809− ret = icl_calc_dpll_state(crtc_state, encoder);
2810− } else if (intel_port_is_tc(dev_priv, port)) {
2811− if (encoder->type == INTEL_OUTPUT_DP_MST) {
2812− struct intel_dp_mst_encoder *mst_encoder;
2916+ if (!icl_calc_dpll_state(crtc_state, encoder, &port_dpll->hw_state)) {
2917+ DRM_DEBUG_KMS("Could not calculate combo PHY PLL state.\n");
28132918
2814− mst_encoder = enc_to_mst(&encoder->base);
2815− intel_dig_port = mst_encoder->primary;
2816− } else {
2817− intel_dig_port = enc_to_dig_port(&encoder->base);
2818− }
2919+ return false;
2920+ }
28192921
2820− if (intel_dig_port->tc_type == TC_PORT_TBT) {
2821− min = DPLL_ID_ICL_TBTPLL;
2822− max = min;
2823− ret = icl_calc_dpll_state(crtc_state, encoder);
2824− } else {
2825− enum tc_port tc_port;
2922+ if (IS_ELKHARTLAKE(dev_priv) && port != PORT_A)
2923+ has_dpll4 = true;
2924+
2925+ port_dpll->pll = intel_find_shared_dpll(state, crtc,
2926+ &port_dpll->hw_state,
2927+ DPLL_ID_ICL_DPLL0,
2928+ has_dpll4 ? DPLL_ID_EHL_DPLL4
2929+ : DPLL_ID_ICL_DPLL1);
2930+ if (!port_dpll->pll) {
2931+ DRM_DEBUG_KMS("No combo PHY PLL found for port %c\n",
2932+ port_name(encoder->port));
2933+ return false;
2934+ }
28262935
2827− tc_port = intel_port_to_tc(dev_priv, port);
2828− min = icl_tc_port_to_pll_id(tc_port);
2829− max = min;
2830− ret = icl_calc_mg_pll_state(crtc_state);
2831− }
2832− } else {
2833− MISSING_CASE(port);
2834− return NULL;
2936+ intel_reference_shared_dpll(state, crtc,
2937+ port_dpll->pll, &port_dpll->hw_state);
2938+
2939+ icl_update_active_dpll(state, crtc, encoder);
2940+
2941+ return true;
2942+}
2943+
2944+static bool icl_get_tc_phy_dplls(struct intel_atomic_state *state,
2945+ struct intel_crtc *crtc,
2946+ struct intel_encoder *encoder)
2947+{
2948+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
2949+ struct intel_crtc_state *crtc_state =
2950+ intel_atomic_get_new_crtc_state(state, crtc);
2951+ struct icl_port_dpll *port_dpll;
2952+ enum intel_dpll_id dpll_id;
2953+
2954+ port_dpll = &crtc_state->icl_port_dplls[ICL_PORT_DPLL_DEFAULT];
2955+ if (!icl_calc_dpll_state(crtc_state, encoder, &port_dpll->hw_state)) {
2956+ DRM_DEBUG_KMS("Could not calculate TBT PLL state.\n");
2957+ return false;
28352958 }
28362959
2837− if (!ret) {
2838− DRM_DEBUG_KMS("Could not calculate PLL state.\n");
2839− return NULL;
2960+ port_dpll->pll = intel_find_shared_dpll(state, crtc,
2961+ &port_dpll->hw_state,
2962+ DPLL_ID_ICL_TBTPLL,
2963+ DPLL_ID_ICL_TBTPLL);
2964+ if (!port_dpll->pll) {
2965+ DRM_DEBUG_KMS("No TBT-ALT PLL found\n");
2966+ return false;
28402967 }
2968+ intel_reference_shared_dpll(state, crtc,
2969+ port_dpll->pll, &port_dpll->hw_state);
28412970
28422971
2843− pll = intel_find_shared_dpll(crtc_state, min, max);
2844− if (!pll) {
2845− DRM_DEBUG_KMS("No PLL selected\n");
2846− return NULL;
2972+ port_dpll = &crtc_state->icl_port_dplls[ICL_PORT_DPLL_MG_PHY];
2973+ if (!icl_calc_mg_pll_state(crtc_state, &port_dpll->hw_state)) {
2974+ DRM_DEBUG_KMS("Could not calculate MG PHY PLL state.\n");
2975+ goto err_unreference_tbt_pll;
28472976 }
28482977
2849− intel_reference_shared_dpll(pll, crtc_state);
2978+ dpll_id = icl_tc_port_to_pll_id(intel_port_to_tc(dev_priv,
2979+ encoder->port));
2980+ port_dpll->pll = intel_find_shared_dpll(state, crtc,
2981+ &port_dpll->hw_state,
2982+ dpll_id,
2983+ dpll_id);
2984+ if (!port_dpll->pll) {
2985+ DRM_DEBUG_KMS("No MG PHY PLL found\n");
2986+ goto err_unreference_tbt_pll;
2987+ }
2988+ intel_reference_shared_dpll(state, crtc,
2989+ port_dpll->pll, &port_dpll->hw_state);
28502990
2851− return pll;
2991+ icl_update_active_dpll(state, crtc, encoder);
2992+
2993+ return true;
2994+
2995+err_unreference_tbt_pll:
2996+ port_dpll = &crtc_state->icl_port_dplls[ICL_PORT_DPLL_DEFAULT];
2997+ intel_unreference_shared_dpll(state, crtc, port_dpll->pll);
2998+
2999+ return false;
3000+}
3001+
3002+static bool icl_get_dplls(struct intel_atomic_state *state,
3003+ struct intel_crtc *crtc,
3004+ struct intel_encoder *encoder)
3005+{
3006+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
3007+ enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
3008+
3009+ if (intel_phy_is_combo(dev_priv, phy))
3010+ return icl_get_combo_phy_dpll(state, crtc, encoder);
3011+ else if (intel_phy_is_tc(dev_priv, phy))
3012+ return icl_get_tc_phy_dplls(state, crtc, encoder);
3013+
3014+ MISSING_CASE(phy);
3015+
3016+ return false;
3017+}
3018+
3019+static void icl_put_dplls(struct intel_atomic_state *state,
3020+ struct intel_crtc *crtc)
3021+{
3022+ const struct intel_crtc_state *old_crtc_state =
3023+ intel_atomic_get_old_crtc_state(state, crtc);
3024+ struct intel_crtc_state *new_crtc_state =
3025+ intel_atomic_get_new_crtc_state(state, crtc);
3026+ enum icl_port_dpll_id id;
3027+
3028+ new_crtc_state->shared_dpll = NULL;
3029+
3030+ for (id = ICL_PORT_DPLL_DEFAULT; id < ICL_PORT_DPLL_COUNT; id++) {
3031+ const struct icl_port_dpll *old_port_dpll =
3032+ &old_crtc_state->icl_port_dplls[id];
3033+ struct icl_port_dpll *new_port_dpll =
3034+ &new_crtc_state->icl_port_dplls[id];
3035+
3036+ new_port_dpll->pll = NULL;
3037+
3038+ if (!old_port_dpll->pll)
3039+ continue;
3040+
3041+ intel_unreference_shared_dpll(state, crtc, old_port_dpll->pll);
3042+ }
28523043 }
28533044
28543045 static bool mg_pll_get_hw_state(struct drm_i915_private *dev_priv,
@@ -2932,8 +3123,18 @@ static bool icl_pll_get_hw_state(struct drm_i915_private *dev_priv, · 77 unmodified lines
29323123 if (!(val & PLL_ENABLE))
29333124 goto out;
29343125
2935− hw_state->cfgcr0 = I915_READ(ICL_DPLL_CFGCR0(id));
2936− hw_state->cfgcr1 = I915_READ(ICL_DPLL_CFGCR1(id));
3126+ if (INTEL_GEN(dev_priv) >= 12) {
3127+ hw_state->cfgcr0 = I915_READ(TGL_DPLL_CFGCR0(id));
3128+ hw_state->cfgcr1 = I915_READ(TGL_DPLL_CFGCR1(id));
3129+ } else {
3130+ if (IS_ELKHARTLAKE(dev_priv) && id == DPLL_ID_EHL_DPLL4) {
3131+ hw_state->cfgcr0 = I915_READ(ICL_DPLL_CFGCR0(4));
3132+ hw_state->cfgcr1 = I915_READ(ICL_DPLL_CFGCR1(4));
3133+ } else {
3134+ hw_state->cfgcr0 = I915_READ(ICL_DPLL_CFGCR0(id));
3135+ hw_state->cfgcr1 = I915_READ(ICL_DPLL_CFGCR1(id));
3136+ }
3137+ }
29373138
29383139 ret = true;
29393140 out:
@@ -2945,8 +3146,14 @@ static bool combo_pll_get_hw_state(struct drm_i915_private *dev_priv, · 5 unmodified lines
29453146 struct intel_shared_dpll *pll,
29463147 struct intel_dpll_hw_state *hw_state)
29473148 {
2948− return icl_pll_get_hw_state(dev_priv, pll, hw_state,
2949− CNL_DPLL_ENABLE(pll->info->id));
3149+ i915_reg_t enable_reg = CNL_DPLL_ENABLE(pll->info->id);
3150+
3151+ if (IS_ELKHARTLAKE(dev_priv) &&
3152+ pll->info->id == DPLL_ID_EHL_DPLL4) {
3153+ enable_reg = MG_PLL_ENABLE(0);
3154+ }
3155+
3156+ return icl_pll_get_hw_state(dev_priv, pll, hw_state, enable_reg);
29503157 }
29513158
29523159 static bool tbt_pll_get_hw_state(struct drm_i915_private *dev_priv,
@@ -2961,10 +3168,24 @@ static void icl_dpll_write(struct drm_i915_private *dev_priv, · 8 unmodified lines
29613168 {
29623169 struct intel_dpll_hw_state *hw_state = &pll->state.hw_state;
29633170 const enum intel_dpll_id id = pll->info->id;
3171+ i915_reg_t cfgcr0_reg, cfgcr1_reg;
29643172
2965− I915_WRITE(ICL_DPLL_CFGCR0(id), hw_state->cfgcr0);
2966− I915_WRITE(ICL_DPLL_CFGCR1(id), hw_state->cfgcr1);
2967− POSTING_READ(ICL_DPLL_CFGCR1(id));
3173+ if (INTEL_GEN(dev_priv) >= 12) {
3174+ cfgcr0_reg = TGL_DPLL_CFGCR0(id);
3175+ cfgcr1_reg = TGL_DPLL_CFGCR1(id);
3176+ } else {
3177+ if (IS_ELKHARTLAKE(dev_priv) && id == DPLL_ID_EHL_DPLL4) {
3178+ cfgcr0_reg = ICL_DPLL_CFGCR0(4);
3179+ cfgcr1_reg = ICL_DPLL_CFGCR1(4);
3180+ } else {
3181+ cfgcr0_reg = ICL_DPLL_CFGCR0(id);
3182+ cfgcr1_reg = ICL_DPLL_CFGCR1(id);
3183+ }
3184+ }
3185+
3186+ I915_WRITE(cfgcr0_reg, hw_state->cfgcr0);
3187+ I915_WRITE(cfgcr1_reg, hw_state->cfgcr1);
3188+ POSTING_READ(cfgcr1_reg);
29683189 }
29693190
29703191 static void icl_mg_pll_write(struct drm_i915_private *dev_priv,
@@ -3057,6 +3278,19 @@ static void combo_pll_enable(struct drm_i915_private *dev_priv, · 86 unmodified lines
30573278 {
30583279 i915_reg_t enable_reg = CNL_DPLL_ENABLE(pll->info->id);
30593280
3281+ if (IS_ELKHARTLAKE(dev_priv) &&
3282+ pll->info->id == DPLL_ID_EHL_DPLL4) {
3283+ enable_reg = MG_PLL_ENABLE(0);
3284+
3285+ /*
3286+ * We need to disable DC states when this DPLL is enabled.
3287+ * This can be done by taking a reference on DPLL4 power
3288+ * domain.
3289+ */
3290+ pll->wakeref = intel_display_power_get(dev_priv,
3291+ POWER_DOMAIN_DPLL_DC_OFF);
3292+ }
3293+
30603294 icl_pll_power_enable(dev_priv, pll, enable_reg);
30613295
30623296 icl_dpll_write(dev_priv, pll);
@@ -3152,7 +3386,19 @@ static void icl_pll_disable(struct drm_i915_private *dev_priv, · 89 unmodified lines
31523386 static void combo_pll_disable(struct drm_i915_private *dev_priv,
31533387 struct intel_shared_dpll *pll)
31543388 {
3155− icl_pll_disable(dev_priv, pll, CNL_DPLL_ENABLE(pll->info->id));
3389+ i915_reg_t enable_reg = CNL_DPLL_ENABLE(pll->info->id);
3390+
3391+ if (IS_ELKHARTLAKE(dev_priv) &&
3392+ pll->info->id == DPLL_ID_EHL_DPLL4) {
3393+ enable_reg = MG_PLL_ENABLE(0);
3394+ icl_pll_disable(dev_priv, pll, enable_reg);
3395+
3396+ intel_display_power_put(dev_priv, POWER_DOMAIN_DPLL_DC_OFF,
3397+ pll->wakeref);
3398+ return;
3399+ }
3400+
3401+ icl_pll_disable(dev_priv, pll, enable_reg);
31563402 }
31573403
31583404 static void tbt_pll_disable(struct drm_i915_private *dev_priv,
@@ -3223,19 +3469,38 @@ static const struct dpll_info icl_plls[] = { · 64 unmodified lines
32233469
32243470 static const struct intel_dpll_mgr icl_pll_mgr = {
32253471 .dpll_info = icl_plls,
3226− .get_dpll = icl_get_dpll,
3472+ .get_dplls = icl_get_dplls,
3473+ .put_dplls = icl_put_dplls,
3474+ .update_active_dpll = icl_update_active_dpll,
32273475 .dump_hw_state = icl_dump_hw_state,
32283476 };
32293477
32303478 static const struct dpll_info ehl_plls[] = {
32313479 { "DPLL 0", &combo_pll_funcs, DPLL_ID_ICL_DPLL0, 0 },
32323480 { "DPLL 1", &combo_pll_funcs, DPLL_ID_ICL_DPLL1, 0 },
3481+ { "DPLL 4", &combo_pll_funcs, DPLL_ID_EHL_DPLL4, 0 },
32333482 { },
32343483 };
32353484
32363485 static const struct intel_dpll_mgr ehl_pll_mgr = {
32373486 .dpll_info = ehl_plls,
3238− .get_dpll = icl_get_dpll,
3487+ .get_dplls = icl_get_dplls,
3488+ .put_dplls = icl_put_dplls,
3489+ .dump_hw_state = icl_dump_hw_state,
3490+};
3491+
3492+static const struct dpll_info tgl_plls[] = {
3493+ { "DPLL 0", &combo_pll_funcs, DPLL_ID_ICL_DPLL0, 0 },
3494+ { "DPLL 1", &combo_pll_funcs, DPLL_ID_ICL_DPLL1, 0 },
3495+ { "TBT PLL", &tbt_pll_funcs, DPLL_ID_ICL_TBTPLL, 0 },
3496+ /* TODO: Add typeC plls */
3497+ { },
3498+};
3499+
3500+static const struct intel_dpll_mgr tgl_pll_mgr = {
3501+ .dpll_info = tgl_plls,
3502+ .get_dplls = icl_get_dplls,
3503+ .put_dplls = icl_put_dplls,
32393504 .dump_hw_state = icl_dump_hw_state,
32403505 };
32413506
@@ -3252,7 +3517,9 @@ void intel_shared_dpll_init(struct drm_device *dev) · 10 unmodified lines
32523517 const struct dpll_info *dpll_info;
32533518 int i;
32543519
3255− if (IS_ELKHARTLAKE(dev_priv))
3520+ if (INTEL_GEN(dev_priv) >= 12)
3521+ dpll_mgr = &tgl_pll_mgr;
3522+ else if (IS_ELKHARTLAKE(dev_priv))
32563523 dpll_mgr = &ehl_pll_mgr;
32573524 else if (INTEL_GEN(dev_priv) >= 11)
32583525 dpll_mgr = &icl_pll_mgr;
@@ -3287,50 +3554,87 @@ void intel_shared_dpll_init(struct drm_device *dev) · 28 unmodified lines
32873554 }
32883555
32893556 /**
3290− * intel_get_shared_dpll - get a shared DPLL for CRTC and encoder combination
3291− * @crtc_state: atomic state for the crtc
3557+ * intel_reserve_shared_dplls - reserve DPLLs for CRTC and encoder combination
3558+ * @state: atomic state
3559+ * @crtc: CRTC to reserve DPLLs for
32923560 * @encoder: encoder
32933561 *
3294− * Find an appropriate DPLL for the given CRTC and encoder combination. A
3295− * reference from the @crtc_state to the returned pll is registered in the
3296− * atomic state. That configuration is made effective by calling
3297− * intel_shared_dpll_swap_state(). The reference should be released by calling
3298− * intel_release_shared_dpll().
3562+ * This function reserves all required DPLLs for the given CRTC and encoder
3563+ * combination in the current atomic commit @state and the new @crtc atomic
3564+ * state.
3565+ *
3566+ * The new configuration in the atomic commit @state is made effective by
3567+ * calling intel_shared_dpll_swap_state().
3568+ *
3569+ * The reserved DPLLs should be released by calling
3570+ * intel_release_shared_dplls().
32993571 *
33003572 * Returns:
3301− * A shared DPLL to be used by @crtc_state and @encoder.
3573+ * True if all required DPLLs were successfully reserved.
33023574 */
3303−struct intel_shared_dpll *
3304−intel_get_shared_dpll(struct intel_crtc_state *crtc_state,
3305− struct intel_encoder *encoder)
3575+bool intel_reserve_shared_dplls(struct intel_atomic_state *state,
3576+ struct intel_crtc *crtc,
3577+ struct intel_encoder *encoder)
33063578 {
3307− struct drm_i915_private *dev_priv = to_i915(crtc_state->base.crtc->dev);
3579+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
33083580 const struct intel_dpll_mgr *dpll_mgr = dev_priv->dpll_mgr;
33093581
33103582 if (WARN_ON(!dpll_mgr))
3311− return NULL;
3583+ return false;
33123584
3313− return dpll_mgr->get_dpll(crtc_state, encoder);
3585+ return dpll_mgr->get_dplls(state, crtc, encoder);
33143586 }
33153587
33163588 /**
3317− * intel_release_shared_dpll - end use of DPLL by CRTC in atomic state
3318− * @dpll: dpll in use by @crtc
3319− * @crtc: crtc
3589+ * intel_release_shared_dplls - end use of DPLLs by CRTC in atomic state
33203590 * @state: atomic state
3591+ * @crtc: crtc from which the DPLLs are to be released
33213592 *
3322− * This function releases the reference from @crtc to @dpll from the
3323− * atomic @state. The new configuration is made effective by calling
3324− * intel_shared_dpll_swap_state().
3593+ * This function releases all DPLLs reserved by intel_reserve_shared_dplls()
3594+ * from the current atomic commit @state and the old @crtc atomic state.
3595+ *
3596+ * The new configuration in the atomic commit @state is made effective by
3597+ * calling intel_shared_dpll_swap_state().
33253598 */
3326−void intel_release_shared_dpll(struct intel_shared_dpll *dpll,
3327− struct intel_crtc *crtc,
3328− struct drm_atomic_state *state)
3599+void intel_release_shared_dplls(struct intel_atomic_state *state,
3600+ struct intel_crtc *crtc)
33293601 {
3330− struct intel_shared_dpll_state *shared_dpll_state;
3602+ struct drm_i915_private *dev_priv = to_i915(state->base.dev);
3603+ const struct intel_dpll_mgr *dpll_mgr = dev_priv->dpll_mgr;
3604+
3605+ /*
3606+ * FIXME: this function is called for every platform having a
3607+ * compute_clock hook, even though the platform doesn't yet support
3608+ * the shared DPLL framework and intel_reserve_shared_dplls() is not
3609+ * called on those.
3610+ */
3611+ if (!dpll_mgr)
3612+ return;
3613+
3614+ dpll_mgr->put_dplls(state, crtc);
3615+}
3616+
3617+/**
3618+ * intel_update_active_dpll - update the active DPLL for a CRTC/encoder
3619+ * @state: atomic state
3620+ * @crtc: the CRTC for which to update the active DPLL
3621+ * @encoder: encoder determining the type of port DPLL
3622+ *
3623+ * Update the active DPLL for the given @crtc/@encoder in @crtc's atomic state,
3624+ * from the port DPLLs reserved previously by intel_reserve_shared_dplls(). The
3625+ * DPLL selected will be based on the current mode of the encoder's port.
3626+ */
3627+void intel_update_active_dpll(struct intel_atomic_state *state,
3628+ struct intel_crtc *crtc,
3629+ struct intel_encoder *encoder)
3630+{
3631+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
3632+ const struct intel_dpll_mgr *dpll_mgr = dev_priv->dpll_mgr;
3633+
3634+ if (WARN_ON(!dpll_mgr))
3635+ return;
33313636
3332− shared_dpll_state = intel_atomic_get_shared_dpll_state(state);
3333− shared_dpll_state[dpll->info->id].crtc_mask &= ~(1 << crtc->pipe);
3637+ dpll_mgr->update_active_dpll(state, crtc, encoder);
33343638 }
33353639
33363640 /**
drivers/gpu/drm/i915/display/intel_dpll_mgr.h+44−13
@@ -28,6 +28,7 @@ · 27 unmodified lines
2828 #include <linux/types.h>
2929
3030 #include "intel_display.h"
31+#include "intel_wakeref.h"
3132
3233 /*FIXME: Move this to a more appropriate place. */
3334 #define abs_diff(a, b) ({ \
@@ -36,9 +37,9 @@ · 2 unmodified lines
3637 (void) (&__a == &__b); \
3738 __a > __b ? (__a - __b) : (__b - __a); })
3839
39−struct drm_atomic_state;
4040 struct drm_device;
4141 struct drm_i915_private;
42+struct intel_atomic_state;
4243 struct intel_crtc;
4344 struct intel_crtc_state;
4445 struct intel_encoder;
@@ -110,35 +111,59 @@ enum intel_dpll_id { · 65 unmodified lines
110111
111112
112113 /**
113− * @DPLL_ID_ICL_DPLL0: ICL combo PHY DPLL0
114+ * @DPLL_ID_ICL_DPLL0: ICL/TGL combo PHY DPLL0
114115 */
115116 DPLL_ID_ICL_DPLL0 = 0,
116117 /**
117− * @DPLL_ID_ICL_DPLL1: ICL combo PHY DPLL1
118+ * @DPLL_ID_ICL_DPLL1: ICL/TGL combo PHY DPLL1
118119 */
119120 DPLL_ID_ICL_DPLL1 = 1,
120121 /**
121− * @DPLL_ID_ICL_TBTPLL: ICL TBT PLL
122+ * @DPLL_ID_EHL_DPLL4: EHL combo PHY DPLL4
123+ */
124+ DPLL_ID_EHL_DPLL4 = 2,
125+ /**
126+ * @DPLL_ID_ICL_TBTPLL: ICL/TGL TBT PLL
122127 */
123128 DPLL_ID_ICL_TBTPLL = 2,
124129 /**
125− * @DPLL_ID_ICL_MGPLL1: ICL MG PLL 1 port 1 (C)
130+ * @DPLL_ID_ICL_MGPLL1: ICL MG PLL 1 port 1 (C),
131+ * TGL TC PLL 1 port 1 (TC1)
126132 */
127133 DPLL_ID_ICL_MGPLL1 = 3,
128134 /**
129135 * @DPLL_ID_ICL_MGPLL2: ICL MG PLL 1 port 2 (D)
136+ * TGL TC PLL 1 port 2 (TC2)
130137 */
131138 DPLL_ID_ICL_MGPLL2 = 4,
132139 /**
133140 * @DPLL_ID_ICL_MGPLL3: ICL MG PLL 1 port 3 (E)
141+ * TGL TC PLL 1 port 3 (TC3)
134142 */
135143 DPLL_ID_ICL_MGPLL3 = 5,
136144 /**
137145 * @DPLL_ID_ICL_MGPLL4: ICL MG PLL 1 port 4 (F)
146+ * TGL TC PLL 1 port 4 (TC4)
138147 */
139148 DPLL_ID_ICL_MGPLL4 = 6,
149+ /**
150+ * @DPLL_ID_TGL_TCPLL5: TGL TC PLL port 5 (TC5)
151+ */
152+ DPLL_ID_TGL_MGPLL5 = 7,
153+ /**
154+ * @DPLL_ID_TGL_TCPLL6: TGL TC PLL port 6 (TC6)
155+ */
156+ DPLL_ID_TGL_MGPLL6 = 8,
157+};
158+
159+#define I915_NUM_PLLS 9
160+
161+enum icl_port_dpll_id {
162+ ICL_PORT_DPLL_DEFAULT,
163+ ICL_PORT_DPLL_MG_PHY,
164+
165+ ICL_PORT_DPLL_COUNT,
140166 };
141−#define I915_NUM_PLLS 7
142167
143168 struct intel_dpll_hw_state {
144169 /* i9xx, pch plls */
@@ -195,7 +220,7 @@ struct intel_dpll_hw_state { · 50 unmodified lines
195220 * future state which would be applied by an atomic mode set (stored in
196221 * a struct &intel_atomic_state).
197222 *
198− * See also intel_get_shared_dpll() and intel_release_shared_dpll().
223+ * See also intel_reserve_shared_dplls() and intel_release_shared_dplls().
199224 */
200225 struct intel_shared_dpll_state {
201226 /**
@@ -312,6 +337,7 @@ struct intel_shared_dpll { · 110 unmodified lines
312337 * @info: platform specific info
313338 */
314339 const struct dpll_info *info;
340+ intel_wakeref_t wakeref;
315341 };
316342
317343 #define SKL_DPLL0 0
@@ -331,15 +357,20 @@ void assert_shared_dpll(struct drm_i915_private *dev_priv, · 13 unmodified lines
331357 bool state);
332358 #define assert_shared_dpll_enabled(d, p) assert_shared_dpll(d, p, true)
333359 #define assert_shared_dpll_disabled(d, p) assert_shared_dpll(d, p, false)
334−struct intel_shared_dpll *intel_get_shared_dpll(struct intel_crtc_state *state,
335− struct intel_encoder *encoder);
336−void intel_release_shared_dpll(struct intel_shared_dpll *dpll,
337− struct intel_crtc *crtc,
338− struct drm_atomic_state *state);
360+bool intel_reserve_shared_dplls(struct intel_atomic_state *state,
361+ struct intel_crtc *crtc,
362+ struct intel_encoder *encoder);
363+void intel_release_shared_dplls(struct intel_atomic_state *state,
364+ struct intel_crtc *crtc);
365+void icl_set_active_port_dpll(struct intel_crtc_state *crtc_state,
366+ enum icl_port_dpll_id port_dpll_id);
367+void intel_update_active_dpll(struct intel_atomic_state *state,
368+ struct intel_crtc *crtc,
369+ struct intel_encoder *encoder);
339370 void intel_prepare_shared_dpll(const struct intel_crtc_state *crtc_state);
340371 void intel_enable_shared_dpll(const struct intel_crtc_state *crtc_state);
341372 void intel_disable_shared_dpll(const struct intel_crtc_state *crtc_state);
342−void intel_shared_dpll_swap_state(struct drm_atomic_state *state);
373+void intel_shared_dpll_swap_state(struct intel_atomic_state *state);
343374 void intel_shared_dpll_init(struct drm_device *dev);
344375
345376 void intel_dpll_dump_hw_state(struct drm_i915_private *dev_priv,
drivers/gpu/drm/i915/display/intel_dsi.h+9−3
@@ -49,8 +49,11 @@ struct intel_dsi { · 48 unmodified lines
4949
5050 struct intel_connector *attached_connector;
5151
52− /* bit mask of ports being driven */
53− u16 ports;
52+ /* bit mask of ports (vlv dsi) or phys (icl dsi) being driven */
53+ union {
54+ u16 ports; /* VLV DSI */
55+ u16 phys; /* ICL DSI */
56+ };
5457
5558 /* if true, use HS mode, otherwise LP */
5659 bool hs;
@@ -132,7 +135,10 @@ static inline struct intel_dsi_host *to_intel_dsi_host(struct mipi_dsi_host *h) · 75 unmodified lines
132135 return container_of(h, struct intel_dsi_host, base);
133136 }
134137
135−#define for_each_dsi_port(__port, __ports_mask) for_each_port_masked(__port, __ports_mask)
138+#define for_each_dsi_port(__port, __ports_mask) \
139+ for_each_port_masked(__port, __ports_mask)
140+#define for_each_dsi_phy(__phy, __phys_mask) \
141+ for_each_phy_masked(__phy, __phys_mask)
136142
137143 static inline struct intel_dsi *enc_to_intel_dsi(struct drm_encoder *encoder)
138144 {
drivers/gpu/drm/i915/display/intel_gmbus.c+18−2
@@ -94,11 +94,25 @@ static const struct gmbus_pin gmbus_pins_mcc[] = { · 93 unmodified lines
9494 [GMBUS_PIN_9_TC1_ICP] = { "dpc", GPIOJ },
9595 };
9696
97+static const struct gmbus_pin gmbus_pins_tgp[] = {
98+ [GMBUS_PIN_1_BXT] = { "dpa", GPIOB },
99+ [GMBUS_PIN_2_BXT] = { "dpb", GPIOC },
100+ [GMBUS_PIN_3_BXT] = { "dpc", GPIOD },
101+ [GMBUS_PIN_9_TC1_ICP] = { "tc1", GPIOJ },
102+ [GMBUS_PIN_10_TC2_ICP] = { "tc2", GPIOK },
103+ [GMBUS_PIN_11_TC3_ICP] = { "tc3", GPIOL },
104+ [GMBUS_PIN_12_TC4_ICP] = { "tc4", GPIOM },
105+ [GMBUS_PIN_13_TC5_TGP] = { "tc5", GPION },
106+ [GMBUS_PIN_14_TC6_TGP] = { "tc6", GPIOO },
107+};
108+
97109 /* pin is expected to be valid */
98110 static const struct gmbus_pin *get_gmbus_pin(struct drm_i915_private *dev_priv,
99111 unsigned int pin)
100112 {
101− if (HAS_PCH_MCC(dev_priv))
113+ if (HAS_PCH_TGP(dev_priv))
114+ return &gmbus_pins_tgp[pin];
115+ else if (HAS_PCH_MCC(dev_priv))
102116 return &gmbus_pins_mcc[pin];
103117 else if (HAS_PCH_ICP(dev_priv))
104118 return &gmbus_pins_icp[pin];
@@ -119,7 +133,9 @@ bool intel_gmbus_is_valid_pin(struct drm_i915_private *dev_priv, · 14 unmodified lines
119133 {
120134 unsigned int size;
121135
122− if (HAS_PCH_MCC(dev_priv))
136+ if (HAS_PCH_TGP(dev_priv))
137+ size = ARRAY_SIZE(gmbus_pins_tgp);
138+ else if (HAS_PCH_MCC(dev_priv))
123139 size = ARRAY_SIZE(gmbus_pins_mcc);
124140 else if (HAS_PCH_ICP(dev_priv))
125141 size = ARRAY_SIZE(gmbus_pins_icp);
drivers/gpu/drm/i915/display/intel_hdcp.c+9−3
@@ -523,12 +523,16 @@ int intel_hdcp_auth_downstream(struct intel_connector *connector) · 522 unmodified lines
523523 * authentication.
524524 */
525525 num_downstream = DRM_HDCP_NUM_DOWNSTREAM(bstatus[0]);
526− if (num_downstream == 0)
526+ if (num_downstream == 0) {
527+ DRM_DEBUG_KMS("Repeater with zero downstream devices\n");
527528 return -EINVAL;
529+ }
528530
529531 ksv_fifo = kcalloc(DRM_HDCP_KSV_LEN, num_downstream, GFP_KERNEL);
530− if (!ksv_fifo)
532+ if (!ksv_fifo) {
533+ DRM_DEBUG_KMS("Out of mem: ksv_fifo\n");
531534 return -ENOMEM;
535+ }
532536
533537 ret = shim->read_ksv_fifo(intel_dig_port, num_downstream, ksv_fifo);
534538 if (ret)
@@ -1206,8 +1210,10 @@ static int hdcp2_authentication_key_exchange(struct intel_connector *connector) · 671 unmodified lines
12061210 if (ret < 0)
12071211 return ret;
12081212
1209− if (msgs.send_cert.rx_caps[0] != HDCP_2_2_RX_CAPS_VERSION_VAL)
1213+ if (msgs.send_cert.rx_caps[0] != HDCP_2_2_RX_CAPS_VERSION_VAL) {
1214+ DRM_DEBUG_KMS("cert.rx_caps dont claim HDCP2.2\n");
12101215 return -EINVAL;
1216+ }
12111217
12121218 hdcp->is_repeater = HDCP_2_2_RX_REPEATER(msgs.send_cert.rx_caps[2]);
12131219
drivers/gpu/drm/i915/display/intel_hdmi.c+42−33
@@ -2930,51 +2930,34 @@ static u8 cnp_port_to_ddc_pin(struct drm_i915_private *dev_priv, · 2929 unmodified lines
29302930
29312931 static u8 icl_port_to_ddc_pin(struct drm_i915_private *dev_priv, enum port port)
29322932 {
2933− u8 ddc_pin;
2933+ enum phy phy = intel_port_to_phy(dev_priv, port);
29342934
2935− switch (port) {
2936− case PORT_A:
2937− ddc_pin = GMBUS_PIN_1_BXT;
2938− break;
2939− case PORT_B:
2940− ddc_pin = GMBUS_PIN_2_BXT;
2941− break;
2942− case PORT_C:
2943− ddc_pin = GMBUS_PIN_9_TC1_ICP;
2944− break;
2945− case PORT_D:
2946− ddc_pin = GMBUS_PIN_10_TC2_ICP;
2947− break;
2948− case PORT_E:
2949− ddc_pin = GMBUS_PIN_11_TC3_ICP;
2950− break;
2951− case PORT_F:
2952− ddc_pin = GMBUS_PIN_12_TC4_ICP;
2953− break;
2954− default:
2955− MISSING_CASE(port);
2956− ddc_pin = GMBUS_PIN_2_BXT;
2957− break;
2958− }
2959− return ddc_pin;
2935+ if (intel_phy_is_combo(dev_priv, phy))
2936+ return GMBUS_PIN_1_BXT + port;
2937+ else if (intel_phy_is_tc(dev_priv, phy))
2938+ return GMBUS_PIN_9_TC1_ICP + intel_port_to_tc(dev_priv, port);
2939+
2940+ WARN(1, "Unknown port:%c\n", port_name(port));
2941+ return GMBUS_PIN_2_BXT;
29602942 }
29612943
29622944 static u8 mcc_port_to_ddc_pin(struct drm_i915_private *dev_priv, enum port port)
29632945 {
2946+ enum phy phy = intel_port_to_phy(dev_priv, port);
29642947 u8 ddc_pin;
29652948
2966− switch (port) {
2967− case PORT_A:
2949+ switch (phy) {
2950+ case PHY_A:
29682951 ddc_pin = GMBUS_PIN_1_BXT;
29692952 break;
2970− case PORT_B:
2953+ case PHY_B:
29712954 ddc_pin = GMBUS_PIN_2_BXT;
29722955 break;
2973− case PORT_C:
2956+ case PHY_C:
29742957 ddc_pin = GMBUS_PIN_9_TC1_ICP;
29752958 break;
29762959 default:
2977− MISSING_CASE(port);
2960+ MISSING_CASE(phy);
29782961 ddc_pin = GMBUS_PIN_1_BXT;
29792962 break;
29802963 }
@@ -3019,7 +3002,7 @@ static u8 intel_hdmi_ddc_pin(struct drm_i915_private *dev_priv, · 38 unmodified lines
30193002
30203003 if (HAS_PCH_MCC(dev_priv))
30213004 ddc_pin = mcc_port_to_ddc_pin(dev_priv, port);
3022− else if (HAS_PCH_ICP(dev_priv))
3005+ else if (HAS_PCH_TGP(dev_priv) || HAS_PCH_ICP(dev_priv))
30233006 ddc_pin = icl_port_to_ddc_pin(dev_priv, port);
30243007 else if (HAS_PCH_CNP(dev_priv))
30253008 ddc_pin = cnp_port_to_ddc_pin(dev_priv, port);
@@ -3143,6 +3126,32 @@ void intel_hdmi_init_connector(struct intel_digital_port *intel_dig_port, · 117 unmodified lines
31433126 DRM_DEBUG_KMS("CEC notifier get failed\n");
31443127 }
31453128
3129+static enum intel_hotplug_state
3130+intel_hdmi_hotplug(struct intel_encoder *encoder,
3131+ struct intel_connector *connector, bool irq_received)
3132+{
3133+ enum intel_hotplug_state state;
3134+
3135+ state = intel_encoder_hotplug(encoder, connector, irq_received);
3136+
3137+ /*
3138+ * On many platforms the HDMI live state signal is known to be
3139+ * unreliable, so we can't use it to detect if a sink is connected or
3140+ * not. Instead we detect if it's connected based on whether we can
3141+ * read the EDID or not. That in turn has a problem during disconnect,
3142+ * since the HPD interrupt may be raised before the DDC lines get
3143+ * disconnected (due to how the required length of DDC vs. HPD
3144+ * connector pins are specified) and so we'll still be able to get a
3145+ * valid EDID. To solve this schedule another detection cycle if this
3146+ * time around we didn't detect any change in the sink's connection
3147+ * status.
3148+ */
3149+ if (state == INTEL_HOTPLUG_UNCHANGED && irq_received)
3150+ state = INTEL_HOTPLUG_RETRY;
3151+
3152+ return state;
3153+}
3154+
31463155 void intel_hdmi_init(struct drm_i915_private *dev_priv,
31473156 i915_reg_t hdmi_reg, enum port port)
31483157 {
@@ -3166,7 +3175,7 @@ void intel_hdmi_init(struct drm_i915_private *dev_priv, · 17 unmodified lines
31663175 &intel_hdmi_enc_funcs, DRM_MODE_ENCODER_TMDS,
31673176 "HDMI %c", port_name(port));
31683177
3169− intel_encoder->hotplug = intel_encoder_hotplug;
3178+ intel_encoder->hotplug = intel_hdmi_hotplug;
31703179 intel_encoder->compute_config = intel_hdmi_compute_config;
31713180 if (HAS_PCH_SPLIT(dev_priv)) {
31723181 intel_encoder->disable = pch_disable_hdmi;
drivers/gpu/drm/i915/display/intel_hotplug.c+46−13
@@ -112,6 +112,7 @@ enum hpd_pin intel_hpd_pin_default(struct drm_i915_private *dev_priv, · 111 unmodified lines
112112
113113 #define HPD_STORM_DETECT_PERIOD 1000
114114 #define HPD_STORM_REENABLE_DELAY (2 * 60 * 1000)
115+#define HPD_RETRY_DELAY 1000
115116
116117 /**
117118 * intel_hpd_irq_storm_detect - gather stats and detect HPD IRQ storm on a pin
@@ -266,8 +267,10 @@ static void intel_hpd_irq_storm_reenable_work(struct work_struct *work) · 148 unmodified lines
266267 intel_runtime_pm_put(&dev_priv->runtime_pm, wakeref);
267268 }
268269
269−bool intel_encoder_hotplug(struct intel_encoder *encoder,
270− struct intel_connector *connector)
270+enum intel_hotplug_state
271+intel_encoder_hotplug(struct intel_encoder *encoder,
272+ struct intel_connector *connector,
273+ bool irq_received)
271274 {
272275 struct drm_device *dev = connector->base.dev;
273276 enum drm_connector_status old_status;
@@ -279,7 +282,7 @@ bool intel_encoder_hotplug(struct intel_encoder *encoder, · 5 unmodified lines
279282 drm_helper_probe_detect(&connector->base, NULL, false);
280283
281284 if (old_status == connector->base.status)
282− return false;
285+ return INTEL_HOTPLUG_UNCHANGED;
283286
284287 DRM_DEBUG_KMS("[CONNECTOR:%d:%s] status updated from %s to %s\n",
285288 connector->base.base.id,
@@ -287,7 +290,7 @@ bool intel_encoder_hotplug(struct intel_encoder *encoder, · 1 unmodified line
287290 drm_get_connector_status_name(old_status),
288291 drm_get_connector_status_name(connector->base.status));
289292
290− return true;
293+ return INTEL_HOTPLUG_CHANGED;
291294 }
292295
293296 static bool intel_encoder_has_hpd_pulse(struct intel_encoder *encoder)
@@ -339,7 +342,7 @@ static void i915_digport_work_func(struct work_struct *work) · 45 unmodified lines
339342 spin_lock_irq(&dev_priv->irq_lock);
340343 dev_priv->hotplug.event_bits |= old_bits;
341344 spin_unlock_irq(&dev_priv->irq_lock);
342− schedule_work(&dev_priv->hotplug.hotplug_work);
345+ queue_delayed_work(system_wq, &dev_priv->hotplug.hotplug_work, 0);
343346 }
344347 }
345348
@@ -349,14 +352,16 @@ static void i915_digport_work_func(struct work_struct *work) · 3 unmodified lines
349352 static void i915_hotplug_work_func(struct work_struct *work)
350353 {
351354 struct drm_i915_private *dev_priv =
352− container_of(work, struct drm_i915_private, hotplug.hotplug_work);
355+ container_of(work, struct drm_i915_private,
356+ hotplug.hotplug_work.work);
353357 struct drm_device *dev = &dev_priv->drm;
354358 struct intel_connector *intel_connector;
355359 struct intel_encoder *intel_encoder;
356360 struct drm_connector *connector;
357361 struct drm_connector_list_iter conn_iter;
358− bool changed = false;
362+ u32 changed = 0, retry = 0;
359363 u32 hpd_event_bits;
364+ u32 hpd_retry_bits;
360365
361366 mutex_lock(&dev->mode_config.mutex);
362367 DRM_DEBUG_KMS("running encoder hotplug functions\n");
@@ -365,6 +370,8 @@ static void i915_hotplug_work_func(struct work_struct *work) · 2 unmodified lines
365370
366371 hpd_event_bits = dev_priv->hotplug.event_bits;
367372 dev_priv->hotplug.event_bits = 0;
373+ hpd_retry_bits = dev_priv->hotplug.retry_bits;
374+ dev_priv->hotplug.retry_bits = 0;
368375
369376 /* Enable polling for connectors which had HPD IRQ storms */
370377 intel_hpd_irq_storm_switch_to_polling(dev_priv);
@@ -373,16 +380,29 @@ static void i915_hotplug_work_func(struct work_struct *work) · 2 unmodified lines
373380
374381 drm_connector_list_iter_begin(dev, &conn_iter);
375382 drm_for_each_connector_iter(connector, &conn_iter) {
383+ u32 hpd_bit;
384+
376385 intel_connector = to_intel_connector(connector);
377386 if (!intel_connector->encoder)
378387 continue;
379388 intel_encoder = intel_connector->encoder;
380− if (hpd_event_bits & (1 << intel_encoder->hpd_pin)) {
389+ hpd_bit = BIT(intel_encoder->hpd_pin);
390+ if ((hpd_event_bits | hpd_retry_bits) & hpd_bit) {
381391 DRM_DEBUG_KMS("Connector %s (pin %i) received hotplug event.\n",
382392 connector->name, intel_encoder->hpd_pin);
383393
384− changed |= intel_encoder->hotplug(intel_encoder,
385− intel_connector);
394+ switch (intel_encoder->hotplug(intel_encoder,
395+ intel_connector,
396+ hpd_event_bits & hpd_bit)) {
397+ case INTEL_HOTPLUG_UNCHANGED:
398+ break;
399+ case INTEL_HOTPLUG_CHANGED:
400+ changed |= hpd_bit;
401+ break;
402+ case INTEL_HOTPLUG_RETRY:
403+ retry |= hpd_bit;
404+ break;
405+ }
386406 }
387407 }
388408 drm_connector_list_iter_end(&conn_iter);
@@ -390,6 +410,17 @@ static void i915_hotplug_work_func(struct work_struct *work) · 1 unmodified line
390410
391411 if (changed)
392412 drm_kms_helper_hotplug_event(dev);
413+
414+ /* Remove shared HPD pins that have changed */
415+ retry &= ~changed;
416+ if (retry) {
417+ spin_lock_irq(&dev_priv->irq_lock);
418+ dev_priv->hotplug.retry_bits |= retry;
419+ spin_unlock_irq(&dev_priv->irq_lock);
420+
421+ mod_delayed_work(system_wq, &dev_priv->hotplug.hotplug_work,
422+ msecs_to_jiffies(HPD_RETRY_DELAY));
423+ }
393424 }
394425
395426
@@ -516,7 +547,7 @@ void intel_hpd_irq_handler(struct drm_i915_private *dev_priv, · 120 unmodified lines
516547 if (queue_dig)
517548 queue_work(dev_priv->hotplug.dp_wq, &dev_priv->hotplug.dig_port_work);
518549 if (queue_hp)
519− schedule_work(&dev_priv->hotplug.hotplug_work);
550+ queue_delayed_work(system_wq, &dev_priv->hotplug.hotplug_work, 0);
520551 }
521552
522553 /**
@@ -636,7 +667,8 @@ void intel_hpd_poll_init(struct drm_i915_private *dev_priv) · 113 unmodified lines
636667
637668 void intel_hpd_init_work(struct drm_i915_private *dev_priv)
638669 {
639− INIT_WORK(&dev_priv->hotplug.hotplug_work, i915_hotplug_work_func);
670+ INIT_DELAYED_WORK(&dev_priv->hotplug.hotplug_work,
671+ i915_hotplug_work_func);
640672 INIT_WORK(&dev_priv->hotplug.dig_port_work, i915_digport_work_func);
641673 INIT_WORK(&dev_priv->hotplug.poll_init_work, i915_hpd_poll_init_work);
642674 INIT_DELAYED_WORK(&dev_priv->hotplug.reenable_work,
@@ -650,11 +682,12 @@ void intel_hpd_cancel_work(struct drm_i915_private *dev_priv) · 7 unmodified lines
650682 dev_priv->hotplug.long_port_mask = 0;
651683 dev_priv->hotplug.short_port_mask = 0;
652684 dev_priv->hotplug.event_bits = 0;
685+ dev_priv->hotplug.retry_bits = 0;
653686
654687 spin_unlock_irq(&dev_priv->irq_lock);
655688
656689 cancel_work_sync(&dev_priv->hotplug.dig_port_work);
657− cancel_work_sync(&dev_priv->hotplug.hotplug_work);
690+ cancel_delayed_work_sync(&dev_priv->hotplug.hotplug_work);
658691 cancel_work_sync(&dev_priv->hotplug.poll_init_work);
659692 cancel_delayed_work_sync(&dev_priv->hotplug.reenable_work);
660693 }
drivers/gpu/drm/i915/display/intel_hotplug.h+3−2
@@ -15,8 +15,9 @@ struct intel_connector; · 14 unmodified lines
1515 struct intel_encoder;
1616
1717 void intel_hpd_poll_init(struct drm_i915_private *dev_priv);
18−bool intel_encoder_hotplug(struct intel_encoder *encoder,
19− struct intel_connector *connector);
18+enum intel_hotplug_state intel_encoder_hotplug(struct intel_encoder *encoder,
19+ struct intel_connector *connector,
20+ bool irq_received);
2021 void intel_hpd_irq_handler(struct drm_i915_private *dev_priv,
2122 u32 pin_mask, u32 long_mask);
2223 void intel_hpd_init(struct drm_i915_private *dev_priv);
drivers/gpu/drm/i915/display/intel_overlay.c+7−3
@@ -175,6 +175,7 @@ struct overlay_registers { · 174 unmodified lines
175175
176176 struct intel_overlay {
177177 struct drm_i915_private *i915;
178+ struct intel_context *context;
178179 struct intel_crtc *crtc;
179180 struct i915_vma *vma;
180181 struct i915_vma *old_vma;
@@ -239,9 +240,7 @@ static int intel_overlay_do_wait_request(struct intel_overlay *overlay, · 58 unmodified lines
239240
240241 static struct i915_request *alloc_request(struct intel_overlay *overlay)
241242 {
242− struct intel_engine_cs *engine = overlay->i915->engine[RCS0];
243−
244− return i915_request_create(engine->kernel_context);
243+ return i915_request_create(overlay->context);
245244 }
246245
247246 /* overlay needs to be disable in OCMD reg */
@@ -1359,11 +1358,16 @@ void intel_overlay_setup(struct drm_i915_private *dev_priv) · 1111 unmodified lines
13591358 if (!HAS_OVERLAY(dev_priv))
13601359 return;
13611360
1361+ if (!HAS_ENGINE(dev_priv, RCS0))
1362+ return;
1363+
13621364 overlay = kzalloc(sizeof(*overlay), GFP_KERNEL);
13631365 if (!overlay)
13641366 return;
13651367
13661368 overlay->i915 = dev_priv;
1369+ overlay->context = dev_priv->engine[RCS0]->kernel_context;
1370+ GEM_BUG_ON(!overlay->context);
13671371
13681372 overlay->color_key = 0x0101fe;
13691373 overlay->color_key_enabled = true;
drivers/gpu/drm/i915/display/intel_pipe_crc.c+1−1
@@ -667,5 +667,5 @@ void intel_crtc_disable_pipe_crc(struct intel_crtc *intel_crtc) · 666 unmodified lines
667667
668668 I915_WRITE(PIPE_CRC_CTL(crtc->index), 0);
669669 POSTING_READ(PIPE_CRC_CTL(crtc->index));
670− synchronize_irq(dev_priv->drm.irq);
670+ intel_synchronize_irq(dev_priv);
671671 }
drivers/gpu/drm/i915/display/intel_sdvo.c+172−142
@@ -274,130 +274,145 @@ static bool intel_sdvo_read_byte(struct intel_sdvo *intel_sdvo, u8 addr, u8 *ch) · 273 unmodified lines
274274 return false;
275275 }
276276
277−#define SDVO_CMD_NAME_ENTRY(cmd) {cmd, #cmd}
277+#define SDVO_CMD_NAME_ENTRY(cmd_) { .cmd = SDVO_CMD_ ## cmd_, .name = #cmd_ }
278+
278279 /** Mapping of command numbers to names, for debug output */
279−static const struct _sdvo_cmd_name {
280+static const struct {
280281 u8 cmd;
281282 const char *name;
282283 } __attribute__ ((packed)) sdvo_cmd_names[] = {
283− SDVO_CMD_NAME_ENTRY(SDVO_CMD_RESET),
284− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_DEVICE_CAPS),
285− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_FIRMWARE_REV),
286− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_TRAINED_INPUTS),
287− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_ACTIVE_OUTPUTS),
288− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_ACTIVE_OUTPUTS),
289− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_IN_OUT_MAP),
290− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_IN_OUT_MAP),
291− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_ATTACHED_DISPLAYS),
292− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_HOT_PLUG_SUPPORT),
293− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_ACTIVE_HOT_PLUG),
294− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_ACTIVE_HOT_PLUG),
295− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_INTERRUPT_EVENT_SOURCE),
296− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_TARGET_INPUT),
297− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_TARGET_OUTPUT),
298− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_INPUT_TIMINGS_PART1),
299− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_INPUT_TIMINGS_PART2),
300− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_INPUT_TIMINGS_PART1),
301− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_INPUT_TIMINGS_PART2),
302− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_INPUT_TIMINGS_PART1),
303− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_OUTPUT_TIMINGS_PART1),
304− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_OUTPUT_TIMINGS_PART2),
305− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_OUTPUT_TIMINGS_PART1),
306− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_OUTPUT_TIMINGS_PART2),
307− SDVO_CMD_NAME_ENTRY(SDVO_CMD_CREATE_PREFERRED_INPUT_TIMING),
308− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_PREFERRED_INPUT_TIMING_PART1),
309− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_PREFERRED_INPUT_TIMING_PART2),
310− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_INPUT_PIXEL_CLOCK_RANGE),
311− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_OUTPUT_PIXEL_CLOCK_RANGE),
312− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SUPPORTED_CLOCK_RATE_MULTS),
313− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_CLOCK_RATE_MULT),
314− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_CLOCK_RATE_MULT),
315− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SUPPORTED_TV_FORMATS),
316− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_TV_FORMAT),
317− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_TV_FORMAT),
318− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SUPPORTED_POWER_STATES),
319− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_POWER_STATE),
320− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_ENCODER_POWER_STATE),
321− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_DISPLAY_POWER_STATE),
322− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_CONTROL_BUS_SWITCH),
323− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SDTV_RESOLUTION_SUPPORT),
324− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SCALED_HDTV_RESOLUTION_SUPPORT),
325− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SUPPORTED_ENHANCEMENTS),
284+ SDVO_CMD_NAME_ENTRY(RESET),
285+ SDVO_CMD_NAME_ENTRY(GET_DEVICE_CAPS),
286+ SDVO_CMD_NAME_ENTRY(GET_FIRMWARE_REV),
287+ SDVO_CMD_NAME_ENTRY(GET_TRAINED_INPUTS),
288+ SDVO_CMD_NAME_ENTRY(GET_ACTIVE_OUTPUTS),
289+ SDVO_CMD_NAME_ENTRY(SET_ACTIVE_OUTPUTS),
290+ SDVO_CMD_NAME_ENTRY(GET_IN_OUT_MAP),
291+ SDVO_CMD_NAME_ENTRY(SET_IN_OUT_MAP),
292+ SDVO_CMD_NAME_ENTRY(GET_ATTACHED_DISPLAYS),
293+ SDVO_CMD_NAME_ENTRY(GET_HOT_PLUG_SUPPORT),
294+ SDVO_CMD_NAME_ENTRY(SET_ACTIVE_HOT_PLUG),
295+ SDVO_CMD_NAME_ENTRY(GET_ACTIVE_HOT_PLUG),
296+ SDVO_CMD_NAME_ENTRY(GET_INTERRUPT_EVENT_SOURCE),
297+ SDVO_CMD_NAME_ENTRY(SET_TARGET_INPUT),
298+ SDVO_CMD_NAME_ENTRY(SET_TARGET_OUTPUT),
299+ SDVO_CMD_NAME_ENTRY(GET_INPUT_TIMINGS_PART1),
300+ SDVO_CMD_NAME_ENTRY(GET_INPUT_TIMINGS_PART2),
301+ SDVO_CMD_NAME_ENTRY(SET_INPUT_TIMINGS_PART1),
302+ SDVO_CMD_NAME_ENTRY(SET_INPUT_TIMINGS_PART2),
303+ SDVO_CMD_NAME_ENTRY(SET_OUTPUT_TIMINGS_PART1),
304+ SDVO_CMD_NAME_ENTRY(SET_OUTPUT_TIMINGS_PART2),
305+ SDVO_CMD_NAME_ENTRY(GET_OUTPUT_TIMINGS_PART1),
306+ SDVO_CMD_NAME_ENTRY(GET_OUTPUT_TIMINGS_PART2),
307+ SDVO_CMD_NAME_ENTRY(CREATE_PREFERRED_INPUT_TIMING),
308+ SDVO_CMD_NAME_ENTRY(GET_PREFERRED_INPUT_TIMING_PART1),
309+ SDVO_CMD_NAME_ENTRY(GET_PREFERRED_INPUT_TIMING_PART2),
310+ SDVO_CMD_NAME_ENTRY(GET_INPUT_PIXEL_CLOCK_RANGE),
311+ SDVO_CMD_NAME_ENTRY(GET_OUTPUT_PIXEL_CLOCK_RANGE),
312+ SDVO_CMD_NAME_ENTRY(GET_SUPPORTED_CLOCK_RATE_MULTS),
313+ SDVO_CMD_NAME_ENTRY(GET_CLOCK_RATE_MULT),
314+ SDVO_CMD_NAME_ENTRY(SET_CLOCK_RATE_MULT),
315+ SDVO_CMD_NAME_ENTRY(GET_SUPPORTED_TV_FORMATS),
316+ SDVO_CMD_NAME_ENTRY(GET_TV_FORMAT),
317+ SDVO_CMD_NAME_ENTRY(SET_TV_FORMAT),
318+ SDVO_CMD_NAME_ENTRY(GET_SUPPORTED_POWER_STATES),
319+ SDVO_CMD_NAME_ENTRY(GET_POWER_STATE),
320+ SDVO_CMD_NAME_ENTRY(SET_ENCODER_POWER_STATE),
321+ SDVO_CMD_NAME_ENTRY(SET_DISPLAY_POWER_STATE),
322+ SDVO_CMD_NAME_ENTRY(SET_CONTROL_BUS_SWITCH),
323+ SDVO_CMD_NAME_ENTRY(GET_SDTV_RESOLUTION_SUPPORT),
324+ SDVO_CMD_NAME_ENTRY(GET_SCALED_HDTV_RESOLUTION_SUPPORT),
325+ SDVO_CMD_NAME_ENTRY(GET_SUPPORTED_ENHANCEMENTS),
326326
327327 /* Add the op code for SDVO enhancements */
328− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_HPOS),
329− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_HPOS),
330− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_HPOS),
331− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_VPOS),
332− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_VPOS),
333− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_VPOS),
334− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_SATURATION),
335− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SATURATION),
336− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_SATURATION),
337− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_HUE),
338− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_HUE),
339− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_HUE),
340− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_CONTRAST),
341− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_CONTRAST),
342− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_CONTRAST),
343− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_BRIGHTNESS),
344− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_BRIGHTNESS),
345− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_BRIGHTNESS),
346− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_OVERSCAN_H),
347− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_OVERSCAN_H),
348− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_OVERSCAN_H),
349− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_OVERSCAN_V),
350− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_OVERSCAN_V),
351− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_OVERSCAN_V),
352− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_FLICKER_FILTER),
353− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_FLICKER_FILTER),
354− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_FLICKER_FILTER),
355− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_FLICKER_FILTER_ADAPTIVE),
356− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_FLICKER_FILTER_ADAPTIVE),
357− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_FLICKER_FILTER_ADAPTIVE),
358− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_FLICKER_FILTER_2D),
359− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_FLICKER_FILTER_2D),
360− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_FLICKER_FILTER_2D),
361− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_SHARPNESS),
362− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SHARPNESS),
363− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_SHARPNESS),
364− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_DOT_CRAWL),
365− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_DOT_CRAWL),
366− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_TV_CHROMA_FILTER),
367− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_TV_CHROMA_FILTER),
368− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_TV_CHROMA_FILTER),
369− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_MAX_TV_LUMA_FILTER),
370− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_TV_LUMA_FILTER),
371− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_TV_LUMA_FILTER),
328+ SDVO_CMD_NAME_ENTRY(GET_MAX_HPOS),
329+ SDVO_CMD_NAME_ENTRY(GET_HPOS),
330+ SDVO_CMD_NAME_ENTRY(SET_HPOS),
331+ SDVO_CMD_NAME_ENTRY(GET_MAX_VPOS),
332+ SDVO_CMD_NAME_ENTRY(GET_VPOS),
333+ SDVO_CMD_NAME_ENTRY(SET_VPOS),
334+ SDVO_CMD_NAME_ENTRY(GET_MAX_SATURATION),
335+ SDVO_CMD_NAME_ENTRY(GET_SATURATION),
336+ SDVO_CMD_NAME_ENTRY(SET_SATURATION),
337+ SDVO_CMD_NAME_ENTRY(GET_MAX_HUE),
338+ SDVO_CMD_NAME_ENTRY(GET_HUE),
339+ SDVO_CMD_NAME_ENTRY(SET_HUE),
340+ SDVO_CMD_NAME_ENTRY(GET_MAX_CONTRAST),
341+ SDVO_CMD_NAME_ENTRY(GET_CONTRAST),
342+ SDVO_CMD_NAME_ENTRY(SET_CONTRAST),
343+ SDVO_CMD_NAME_ENTRY(GET_MAX_BRIGHTNESS),
344+ SDVO_CMD_NAME_ENTRY(GET_BRIGHTNESS),
345+ SDVO_CMD_NAME_ENTRY(SET_BRIGHTNESS),
346+ SDVO_CMD_NAME_ENTRY(GET_MAX_OVERSCAN_H),
347+ SDVO_CMD_NAME_ENTRY(GET_OVERSCAN_H),
348+ SDVO_CMD_NAME_ENTRY(SET_OVERSCAN_H),
349+ SDVO_CMD_NAME_ENTRY(GET_MAX_OVERSCAN_V),
350+ SDVO_CMD_NAME_ENTRY(GET_OVERSCAN_V),
351+ SDVO_CMD_NAME_ENTRY(SET_OVERSCAN_V),
352+ SDVO_CMD_NAME_ENTRY(GET_MAX_FLICKER_FILTER),
353+ SDVO_CMD_NAME_ENTRY(GET_FLICKER_FILTER),
354+ SDVO_CMD_NAME_ENTRY(SET_FLICKER_FILTER),
355+ SDVO_CMD_NAME_ENTRY(GET_MAX_FLICKER_FILTER_ADAPTIVE),
356+ SDVO_CMD_NAME_ENTRY(GET_FLICKER_FILTER_ADAPTIVE),
357+ SDVO_CMD_NAME_ENTRY(SET_FLICKER_FILTER_ADAPTIVE),
358+ SDVO_CMD_NAME_ENTRY(GET_MAX_FLICKER_FILTER_2D),
359+ SDVO_CMD_NAME_ENTRY(GET_FLICKER_FILTER_2D),
360+ SDVO_CMD_NAME_ENTRY(SET_FLICKER_FILTER_2D),
361+ SDVO_CMD_NAME_ENTRY(GET_MAX_SHARPNESS),
362+ SDVO_CMD_NAME_ENTRY(GET_SHARPNESS),
363+ SDVO_CMD_NAME_ENTRY(SET_SHARPNESS),
364+ SDVO_CMD_NAME_ENTRY(GET_DOT_CRAWL),
365+ SDVO_CMD_NAME_ENTRY(SET_DOT_CRAWL),
366+ SDVO_CMD_NAME_ENTRY(GET_MAX_TV_CHROMA_FILTER),
367+ SDVO_CMD_NAME_ENTRY(GET_TV_CHROMA_FILTER),
368+ SDVO_CMD_NAME_ENTRY(SET_TV_CHROMA_FILTER),
369+ SDVO_CMD_NAME_ENTRY(GET_MAX_TV_LUMA_FILTER),
370+ SDVO_CMD_NAME_ENTRY(GET_TV_LUMA_FILTER),
371+ SDVO_CMD_NAME_ENTRY(SET_TV_LUMA_FILTER),
372372
373373 /* HDMI op code */
374− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_SUPP_ENCODE),
375− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_ENCODE),
376− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_ENCODE),
377− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_PIXEL_REPLI),
378− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_PIXEL_REPLI),
379− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_COLORIMETRY_CAP),
380− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_COLORIMETRY),
381− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_COLORIMETRY),
382− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_AUDIO_ENCRYPT_PREFER),
383− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_AUDIO_STAT),
384− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_AUDIO_STAT),
385− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_HBUF_INDEX),
386− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_HBUF_INDEX),
387− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_HBUF_INFO),
388− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_HBUF_AV_SPLIT),
389− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_HBUF_AV_SPLIT),
390− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_HBUF_TXRATE),
391− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_HBUF_TXRATE),
392− SDVO_CMD_NAME_ENTRY(SDVO_CMD_SET_HBUF_DATA),
393− SDVO_CMD_NAME_ENTRY(SDVO_CMD_GET_HBUF_DATA),
374+ SDVO_CMD_NAME_ENTRY(GET_SUPP_ENCODE),
375+ SDVO_CMD_NAME_ENTRY(GET_ENCODE),
376+ SDVO_CMD_NAME_ENTRY(SET_ENCODE),
377+ SDVO_CMD_NAME_ENTRY(SET_PIXEL_REPLI),
378+ SDVO_CMD_NAME_ENTRY(GET_PIXEL_REPLI),
379+ SDVO_CMD_NAME_ENTRY(GET_COLORIMETRY_CAP),
380+ SDVO_CMD_NAME_ENTRY(SET_COLORIMETRY),
381+ SDVO_CMD_NAME_ENTRY(GET_COLORIMETRY),
382+ SDVO_CMD_NAME_ENTRY(GET_AUDIO_ENCRYPT_PREFER),
383+ SDVO_CMD_NAME_ENTRY(SET_AUDIO_STAT),
384+ SDVO_CMD_NAME_ENTRY(GET_AUDIO_STAT),
385+ SDVO_CMD_NAME_ENTRY(GET_HBUF_INDEX),
386+ SDVO_CMD_NAME_ENTRY(SET_HBUF_INDEX),
387+ SDVO_CMD_NAME_ENTRY(GET_HBUF_INFO),
388+ SDVO_CMD_NAME_ENTRY(GET_HBUF_AV_SPLIT),
389+ SDVO_CMD_NAME_ENTRY(SET_HBUF_AV_SPLIT),
390+ SDVO_CMD_NAME_ENTRY(GET_HBUF_TXRATE),
391+ SDVO_CMD_NAME_ENTRY(SET_HBUF_TXRATE),
392+ SDVO_CMD_NAME_ENTRY(SET_HBUF_DATA),
393+ SDVO_CMD_NAME_ENTRY(GET_HBUF_DATA),
394394 };
395395
396+#undef SDVO_CMD_NAME_ENTRY
397+
398+static const char *sdvo_cmd_name(u8 cmd)
399+{
400+ int i;
401+
402+ for (i = 0; i < ARRAY_SIZE(sdvo_cmd_names); i++) {
403+ if (cmd == sdvo_cmd_names[i].cmd)
404+ return sdvo_cmd_names[i].name;
405+ }
406+
407+ return NULL;
408+}
409+
396410 #define SDVO_NAME(svdo) ((svdo)->port == PORT_B ? "SDVOB" : "SDVOC")
397411
398412 static void intel_sdvo_debug_write(struct intel_sdvo *intel_sdvo, u8 cmd,
399413 const void *args, int args_len)
400414 {
415+ const char *cmd_name;
401416 int i, pos = 0;
402417 #define BUF_LEN 256
403418 char buffer[BUF_LEN];
@@ -412,15 +427,12 @@ static void intel_sdvo_debug_write(struct intel_sdvo *intel_sdvo, u8 cmd, · 8 unmodified lines
412427 for (; i < 8; i++) {
413428 BUF_PRINT(" ");
414429 }
415− for (i = 0; i < ARRAY_SIZE(sdvo_cmd_names); i++) {
416− if (cmd == sdvo_cmd_names[i].cmd) {
417− BUF_PRINT("(%s)", sdvo_cmd_names[i].name);
418− break;
419− }
420− }
421− if (i == ARRAY_SIZE(sdvo_cmd_names)) {
430+
431+ cmd_name = sdvo_cmd_name(cmd);
432+ if (cmd_name)
433+ BUF_PRINT("(%s)", cmd_name);
434+ else
422435 BUF_PRINT("(%02X)", cmd);
423− }
424436 BUG_ON(pos >= BUF_LEN - 1);
425437 #undef BUF_PRINT
426438 #undef BUF_LEN
@@ -429,15 +441,23 @@ static void intel_sdvo_debug_write(struct intel_sdvo *intel_sdvo, u8 cmd, · 2 unmodified lines
429441 }
430442
431443 static const char * const cmd_status_names[] = {
432− "Power on",
433− "Success",
434− "Not supported",
435− "Invalid arg",
436− "Pending",
437− "Target not specified",
438− "Scaling not supported"
444+ [SDVO_CMD_STATUS_POWER_ON] = "Power on",
445+ [SDVO_CMD_STATUS_SUCCESS] = "Success",
446+ [SDVO_CMD_STATUS_NOTSUPP] = "Not supported",
447+ [SDVO_CMD_STATUS_INVALID_ARG] = "Invalid arg",
448+ [SDVO_CMD_STATUS_PENDING] = "Pending",
449+ [SDVO_CMD_STATUS_TARGET_NOT_SPECIFIED] = "Target not specified",
450+ [SDVO_CMD_STATUS_SCALING_NOT_SUPP] = "Scaling not supported",
439451 };
440452
453+static const char *sdvo_cmd_status(u8 status)
454+{
455+ if (status < ARRAY_SIZE(cmd_status_names))
456+ return cmd_status_names[status];
457+ else
458+ return NULL;
459+}
460+
441461 static bool __intel_sdvo_write_cmd(struct intel_sdvo *intel_sdvo, u8 cmd,
442462 const void *args, int args_len,
443463 bool unlocked)
@@ -516,6 +536,7 @@ static bool intel_sdvo_write_cmd(struct intel_sdvo *intel_sdvo, u8 cmd, · 72 unmodified lines
516536 static bool intel_sdvo_read_response(struct intel_sdvo *intel_sdvo,
517537 void *response, int response_len)
518538 {
539+ const char *cmd_status;
519540 u8 retry = 15; /* 5 quick checks, followed by 10 long checks */
520541 u8 status;
521542 int i, pos = 0;
@@ -562,8 +583,9 @@ static bool intel_sdvo_read_response(struct intel_sdvo *intel_sdvo, · 40 unmodified lines
562583 #define BUF_PRINT(args...) \
563584 pos += snprintf(buffer + pos, max_t(int, BUF_LEN - pos, 0), args)
564585
565− if (status <= SDVO_CMD_STATUS_SCALING_NOT_SUPP)
566− BUF_PRINT("(%s)", cmd_status_names[status]);
586+ cmd_status = sdvo_cmd_status(status);
587+ if (cmd_status)
588+ BUF_PRINT("(%s)", cmd_status);
567589 else
568590 BUF_PRINT("(??? %d)", status);
569591
@@ -929,6 +951,20 @@ static bool intel_sdvo_set_audio_state(struct intel_sdvo *intel_sdvo, · 359 unmodified lines
929951 &audio_state, 1);
930952 }
931953
954+static bool intel_sdvo_get_hbuf_size(struct intel_sdvo *intel_sdvo,
955+ u8 *hbuf_size)
956+{
957+ if (!intel_sdvo_get_value(intel_sdvo, SDVO_CMD_GET_HBUF_INFO,
958+ hbuf_size, 1))
959+ return false;
960+
961+ /* Buffer size is 0 based, hooray! However zero means zero. */
962+ if (*hbuf_size)
963+ (*hbuf_size)++;
964+
965+ return true;
966+}
967+
932968 #if 0
933969 static void intel_sdvo_dump_hdmi_buf(struct intel_sdvo *intel_sdvo)
934970 {
@@ -972,14 +1008,10 @@ static bool intel_sdvo_write_infoframe(struct intel_sdvo *intel_sdvo, · 37 unmodified lines
9721008 set_buf_index, 2))
9731009 return false;
9741010
975− if (!intel_sdvo_get_value(intel_sdvo, SDVO_CMD_GET_HBUF_INFO,
976− &hbuf_size, 1))
1011+ if (!intel_sdvo_get_hbuf_size(intel_sdvo, &hbuf_size))
9771012 return false;
9781013
979− /* Buffer size is 0 based, hooray! */
980− hbuf_size++;
981−
982− DRM_DEBUG_KMS("writing sdvo hbuf: %i, hbuf_size %i, hbuf_size: %i\n",
1014+ DRM_DEBUG_KMS("writing sdvo hbuf: %i, length %u, hbuf_size: %i\n",
9831015 if_index, length, hbuf_size);
9841016
9851017 if (hbuf_size < length)
@@ -1030,14 +1062,10 @@ static ssize_t intel_sdvo_read_infoframe(struct intel_sdvo *intel_sdvo, · 44 unmodified lines
10301062 if (tx_rate == SDVO_HBUF_TX_DISABLED)
10311063 return 0;
10321064
1033− if (!intel_sdvo_get_value(intel_sdvo, SDVO_CMD_GET_HBUF_INFO,
1034− &hbuf_size, 1))
1035− return -ENXIO;
1036−
1037− /* Buffer size is 0 based, hooray! */
1038− hbuf_size++;
1065+ if (!intel_sdvo_get_hbuf_size(intel_sdvo, &hbuf_size))
1066+ return false;
10391067
1040− DRM_DEBUG_KMS("reading sdvo hbuf: %i, hbuf_size %i, hbuf_size: %i\n",
1068+ DRM_DEBUG_KMS("reading sdvo hbuf: %i, length %u, hbuf_size: %i\n",
10411069 if_index, length, hbuf_size);
10421070
10431071 hbuf_size = min_t(unsigned int, length, hbuf_size);
@@ -1893,12 +1921,14 @@ static void intel_sdvo_enable_hotplug(struct intel_encoder *encoder) · 849 unmodified lines
18931921 &intel_sdvo->hotplug_active, 2);
18941922 }
18951923
1896−static bool intel_sdvo_hotplug(struct intel_encoder *encoder,
1897− struct intel_connector *connector)
1924+static enum intel_hotplug_state
1925+intel_sdvo_hotplug(struct intel_encoder *encoder,
1926+ struct intel_connector *connector,
1927+ bool irq_received)
18981928 {
18991929 intel_sdvo_enable_hotplug(encoder);
19001930
1901− return intel_encoder_hotplug(encoder, connector);
1931+ return intel_encoder_hotplug(encoder, connector, irq_received);
19021932 }
19031933
19041934 static bool
drivers/gpu/drm/i915/display/intel_sprite.c+252−83
@@ -441,9 +441,21 @@ icl_program_input_csc(struct intel_plane *plane, · 440 unmodified lines
441441 */
442442 [DRM_COLOR_YCBCR_BT709] = {
443443 0x7C98, 0x7800, 0x0,
444− 0x9EF8, 0x7800, 0xABF8,
444+ 0x9EF8, 0x7800, 0xAC00,
445445 0x0, 0x7800, 0x7ED8,
446446 },
447+ /*
448+ * BT.2020 full range YCbCr -> full range RGB
449+ * The matrix required is :
450+ * [1.000, 0.000, 1.474,
451+ * 1.000, -0.1645, -0.5713,
452+ * 1.000, 1.8814, 0.0000]
453+ */
454+ [DRM_COLOR_YCBCR_BT2020] = {
455+ 0x7BC8, 0x7800, 0x0,
456+ 0x8928, 0x7800, 0xAA88,
457+ 0x0, 0x7800, 0x7F10,
458+ },
447459 };
448460
449461 /* Matrix for Limited Range to Full Range Conversion */
@@ -451,26 +463,38 @@ icl_program_input_csc(struct intel_plane *plane, · 1 unmodified line
451463 /*
452464 * BT.601 Limted range YCbCr -> full range RGB
453465 * The matrix required is :
454− * [1.164384, 0.000, 1.596370,
455− * 1.138393, -0.382500, -0.794598,
456− * 1.138393, 1.971696, 0.0000]
466+ * [1.164384, 0.000, 1.596027,
467+ * 1.164384, -0.39175, -0.812813,
468+ * 1.164384, 2.017232, 0.0000]
457469 */
458470 [DRM_COLOR_YCBCR_BT601] = {
459471 0x7CC8, 0x7950, 0x0,
460− 0x8CB8, 0x7918, 0x9C40,
461− 0x0, 0x7918, 0x7FC8,
472+ 0x8D00, 0x7950, 0x9C88,
473+ 0x0, 0x7950, 0x6810,
462474 },
463475 /*
464476 * BT.709 Limited range YCbCr -> full range RGB
465477 * The matrix required is :
466− * [1.164, 0.000, 1.833671,
467− * 1.138393, -0.213249, -0.532909,
468− * 1.138393, 2.112402, 0.0000]
478+ * [1.164384, 0.000, 1.792741,
479+ * 1.164384, -0.213249, -0.532909,
480+ * 1.164384, 2.112402, 0.0000]
469481 */
470482 [DRM_COLOR_YCBCR_BT709] = {
471− 0x7EA8, 0x7950, 0x0,
472− 0x8888, 0x7918, 0xADA8,
473− 0x0, 0x7918, 0x6870,
483+ 0x7E58, 0x7950, 0x0,
484+ 0x8888, 0x7950, 0xADA8,
485+ 0x0, 0x7950, 0x6870,
486+ },
487+ /*
488+ * BT.2020 Limited range YCbCr -> full range RGB
489+ * The matrix required is :
490+ * [1.164, 0.000, 1.678,
491+ * 1.164, -0.1873, -0.6504,
492+ * 1.164, 2.1417, 0.0000]
493+ */
494+ [DRM_COLOR_YCBCR_BT2020] = {
495+ 0x7D70, 0x7950, 0x0,
496+ 0x8A68, 0x7950, 0xAC00,
497+ 0x0, 0x7950, 0x6890,
474498 },
475499 };
476500 const u16 *csc;
@@ -492,8 +516,11 @@ icl_program_input_csc(struct intel_plane *plane, · 15 unmodified lines
492516
493517 I915_WRITE_FW(PLANE_INPUT_CSC_PREOFF(pipe, plane_id, 0),
494518 PREOFF_YUV_TO_RGB_HI);
495− I915_WRITE_FW(PLANE_INPUT_CSC_PREOFF(pipe, plane_id, 1),
496− PREOFF_YUV_TO_RGB_ME);
519+ if (plane_state->base.color_range == DRM_COLOR_YCBCR_FULL_RANGE)
520+ I915_WRITE_FW(PLANE_INPUT_CSC_PREOFF(pipe, plane_id, 1), 0);
521+ else
522+ I915_WRITE_FW(PLANE_INPUT_CSC_PREOFF(pipe, plane_id, 1),
523+ PREOFF_YUV_TO_RGB_ME);
497524 I915_WRITE_FW(PLANE_INPUT_CSC_PREOFF(pipe, plane_id, 2),
498525 PREOFF_YUV_TO_RGB_LO);
499526 I915_WRITE_FW(PLANE_INPUT_CSC_POSTOFF(pipe, plane_id, 0), 0x0);
@@ -683,6 +710,16 @@ skl_plane_get_hw_state(struct intel_plane *plane, · 183 unmodified lines
683710 return ret;
684711 }
685712
713+static void i9xx_plane_linear_gamma(u16 gamma[8])
714+{
715+ /* The points are not evenly spaced. */
716+ static const u8 in[8] = { 0, 1, 2, 4, 8, 16, 24, 32 };
717+ int i;
718+
719+ for (i = 0; i < 8; i++)
720+ gamma[i] = (in[i] << 8) / 32;
721+}
722+
686723 static void
687724 chv_update_csc(const struct intel_plane_state *plane_state)
688725 {
@@ -858,6 +895,31 @@ static u32 vlv_sprite_ctl(const struct intel_crtc_state *crtc_state, · 169 unmodified lines
858895 return sprctl;
859896 }
860897
898+static void vlv_update_gamma(const struct intel_plane_state *plane_state)
899+{
900+ struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
901+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
902+ const struct drm_framebuffer *fb = plane_state->base.fb;
903+ enum pipe pipe = plane->pipe;
904+ enum plane_id plane_id = plane->id;
905+ u16 gamma[8];
906+ int i;
907+
908+ /* Seems RGB data bypasses the gamma always */
909+ if (!fb->format->is_yuv)
910+ return;
911+
912+ i9xx_plane_linear_gamma(gamma);
913+
914+ /* FIXME these register are single buffered :( */
915+ /* The two end points are implicit (0.0 and 1.0) */
916+ for (i = 1; i < 8 - 1; i++)
917+ I915_WRITE_FW(SPGAMC(pipe, plane_id, i - 1),
918+ gamma[i] << 16 |
919+ gamma[i] << 8 |
920+ gamma[i]);
921+}
922+
861923 static void
862924 vlv_update_plane(struct intel_plane *plane,
863925 const struct intel_crtc_state *crtc_state,
@@ -916,6 +978,7 @@ vlv_update_plane(struct intel_plane *plane, · 52 unmodified lines
916978 intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
917979
918980 vlv_update_clrc(plane_state);
981+ vlv_update_gamma(plane_state);
919982
920983 spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
921984 }
@@ -1013,6 +1076,8 @@ static u32 ivb_sprite_ctl(const struct intel_crtc_state *crtc_state, · 91 unmodified lines
10131076 return 0;
10141077 }
10151078
1079+ sprctl |= SPRITE_INT_GAMMA_DISABLE;
1080+
10161081 if (plane_state->base.color_encoding == DRM_COLOR_YCBCR_BT709)
10171082 sprctl |= SPRITE_YUV_TO_RGB_CSC_FORMAT_BT709;
10181083
@@ -1033,6 +1098,45 @@ static u32 ivb_sprite_ctl(const struct intel_crtc_state *crtc_state, · 14 unmodified lines
10331098 return sprctl;
10341099 }
10351100
1101+static void ivb_sprite_linear_gamma(u16 gamma[18])
1102+{
1103+ int i;
1104+
1105+ for (i = 0; i < 17; i++)
1106+ gamma[i] = (i << 10) / 16;
1107+
1108+ gamma[i] = 3 << 10;
1109+ i++;
1110+}
1111+
1112+static void ivb_update_gamma(const struct intel_plane_state *plane_state)
1113+{
1114+ struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
1115+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
1116+ enum pipe pipe = plane->pipe;
1117+ u16 gamma[18];
1118+ int i;
1119+
1120+ ivb_sprite_linear_gamma(gamma);
1121+
1122+ /* FIXME these register are single buffered :( */
1123+ for (i = 0; i < 16; i++)
1124+ I915_WRITE_FW(SPRGAMC(pipe, i),
1125+ gamma[i] << 20 |
1126+ gamma[i] << 10 |
1127+ gamma[i]);
1128+
1129+ I915_WRITE_FW(SPRGAMC16(pipe, 0), gamma[i]);
1130+ I915_WRITE_FW(SPRGAMC16(pipe, 1), gamma[i]);
1131+ I915_WRITE_FW(SPRGAMC16(pipe, 2), gamma[i]);
1132+ i++;
1133+
1134+ I915_WRITE_FW(SPRGAMC17(pipe, 0), gamma[i]);
1135+ I915_WRITE_FW(SPRGAMC17(pipe, 1), gamma[i]);
1136+ I915_WRITE_FW(SPRGAMC17(pipe, 2), gamma[i]);
1137+ i++;
1138+}
1139+
10361140 static void
10371141 ivb_update_plane(struct intel_plane *plane,
10381142 const struct intel_crtc_state *crtc_state,
@@ -1099,6 +1203,8 @@ ivb_update_plane(struct intel_plane *plane, · 60 unmodified lines
10991203 I915_WRITE_FW(SPRSURF(pipe),
11001204 intel_plane_ggtt_offset(plane_state) + sprsurf_offset);
11011205
1206+ ivb_update_gamma(plane_state);
1207+
11021208 spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
11031209 }
11041210
@@ -1224,6 +1330,66 @@ static u32 g4x_sprite_ctl(const struct intel_crtc_state *crtc_state, · 119 unmodified lines
12241330 return dvscntr;
12251331 }
12261332
1333+static void g4x_update_gamma(const struct intel_plane_state *plane_state)
1334+{
1335+ struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
1336+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
1337+ const struct drm_framebuffer *fb = plane_state->base.fb;
1338+ enum pipe pipe = plane->pipe;
1339+ u16 gamma[8];
1340+ int i;
1341+
1342+ /* Seems RGB data bypasses the gamma always */
1343+ if (!fb->format->is_yuv)
1344+ return;
1345+
1346+ i9xx_plane_linear_gamma(gamma);
1347+
1348+ /* FIXME these register are single buffered :( */
1349+ /* The two end points are implicit (0.0 and 1.0) */
1350+ for (i = 1; i < 8 - 1; i++)
1351+ I915_WRITE_FW(DVSGAMC_G4X(pipe, i - 1),
1352+ gamma[i] << 16 |
1353+ gamma[i] << 8 |
1354+ gamma[i]);
1355+}
1356+
1357+static void ilk_sprite_linear_gamma(u16 gamma[17])
1358+{
1359+ int i;
1360+
1361+ for (i = 0; i < 17; i++)
1362+ gamma[i] = (i << 10) / 16;
1363+}
1364+
1365+static void ilk_update_gamma(const struct intel_plane_state *plane_state)
1366+{
1367+ struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
1368+ struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
1369+ const struct drm_framebuffer *fb = plane_state->base.fb;
1370+ enum pipe pipe = plane->pipe;
1371+ u16 gamma[17];
1372+ int i;
1373+
1374+ /* Seems RGB data bypasses the gamma always */
1375+ if (!fb->format->is_yuv)
1376+ return;
1377+
1378+ ilk_sprite_linear_gamma(gamma);
1379+
1380+ /* FIXME these register are single buffered :( */
1381+ for (i = 0; i < 16; i++)
1382+ I915_WRITE_FW(DVSGAMC_ILK(pipe, i),
1383+ gamma[i] << 20 |
1384+ gamma[i] << 10 |
1385+ gamma[i]);
1386+
1387+ I915_WRITE_FW(DVSGAMCMAX_ILK(pipe, 0), gamma[i]);
1388+ I915_WRITE_FW(DVSGAMCMAX_ILK(pipe, 1), gamma[i]);
1389+ I915_WRITE_FW(DVSGAMCMAX_ILK(pipe, 2), gamma[i]);
1390+ i++;
1391+}
1392+
12271393 static void
12281394 g4x_update_plane(struct intel_plane *plane,
12291395 const struct intel_crtc_state *crtc_state,
@@ -1283,6 +1449,11 @@ g4x_update_plane(struct intel_plane *plane, · 53 unmodified lines
12831449 I915_WRITE_FW(DVSSURF(pipe),
12841450 intel_plane_ggtt_offset(plane_state) + dvssurf_offset);
12851451
1452+ if (IS_G4X(dev_priv))
1453+ g4x_update_gamma(plane_state);
1454+ else
1455+ ilk_update_gamma(plane_state);
1456+
12861457 spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
12871458 }
12881459
@@ -1347,7 +1518,7 @@ g4x_sprite_check_scaling(struct intel_crtc_state *crtc_state, · 58 unmodified lines
13471518 const struct drm_framebuffer *fb = plane_state->base.fb;
13481519 const struct drm_rect *src = &plane_state->base.src;
13491520 const struct drm_rect *dst = &plane_state->base.dst;
1350− int src_x, src_y, src_w, src_h, crtc_w, crtc_h;
1521+ int src_x, src_w, src_h, crtc_w, crtc_h;
13511522 const struct drm_display_mode *adjusted_mode =
13521523 &crtc_state->base.adjusted_mode;
13531524 unsigned int cpp = fb->format->cpp[0];
@@ -1358,7 +1529,6 @@ g4x_sprite_check_scaling(struct intel_crtc_state *crtc_state, · 4 unmodified lines
13581529 crtc_h = drm_rect_height(dst);
13591530
13601531 src_x = src->x1 >> 16;
1361− src_y = src->y1 >> 16;
13621532 src_w = drm_rect_width(src) >> 16;
13631533 src_h = drm_rect_height(src) >> 16;
13641534
@@ -1852,28 +2022,7 @@ static const u32 skl_plane_formats[] = { · 487 unmodified lines
18522022 DRM_FORMAT_VYUY,
18532023 };
18542024
1855−static const u32 icl_plane_formats[] = {
1856− DRM_FORMAT_C8,
1857− DRM_FORMAT_RGB565,
1858− DRM_FORMAT_XRGB8888,
1859− DRM_FORMAT_XBGR8888,
1860− DRM_FORMAT_ARGB8888,
1861− DRM_FORMAT_ABGR8888,
1862− DRM_FORMAT_XRGB2101010,
1863− DRM_FORMAT_XBGR2101010,
1864− DRM_FORMAT_YUYV,
1865− DRM_FORMAT_YVYU,
1866− DRM_FORMAT_UYVY,
1867− DRM_FORMAT_VYUY,
1868− DRM_FORMAT_Y210,
1869− DRM_FORMAT_Y212,
1870− DRM_FORMAT_Y216,
1871− DRM_FORMAT_XVYU2101010,
1872− DRM_FORMAT_XVYU12_16161616,
1873− DRM_FORMAT_XVYU16161616,
1874−};
1875−
1876−static const u32 icl_hdr_plane_formats[] = {
2025+static const u32 skl_planar_formats[] = {
18772026 DRM_FORMAT_C8,
18782027 DRM_FORMAT_RGB565,
18792028 DRM_FORMAT_XRGB8888,
@@ -1882,23 +2031,14 @@ static const u32 icl_hdr_plane_formats[] = { · 2 unmodified lines
18822031 DRM_FORMAT_ABGR8888,
18832032 DRM_FORMAT_XRGB2101010,
18842033 DRM_FORMAT_XBGR2101010,
1885− DRM_FORMAT_XRGB16161616F,
1886− DRM_FORMAT_XBGR16161616F,
1887− DRM_FORMAT_ARGB16161616F,
1888− DRM_FORMAT_ABGR16161616F,
18892034 DRM_FORMAT_YUYV,
18902035 DRM_FORMAT_YVYU,
18912036 DRM_FORMAT_UYVY,
18922037 DRM_FORMAT_VYUY,
1893− DRM_FORMAT_Y210,
1894− DRM_FORMAT_Y212,
1895− DRM_FORMAT_Y216,
1896− DRM_FORMAT_XVYU2101010,
1897− DRM_FORMAT_XVYU12_16161616,
1898− DRM_FORMAT_XVYU16161616,
2038+ DRM_FORMAT_NV12,
18992039 };
19002040
1901−static const u32 skl_planar_formats[] = {
2041+static const u32 glk_planar_formats[] = {
19022042 DRM_FORMAT_C8,
19032043 DRM_FORMAT_RGB565,
19042044 DRM_FORMAT_XRGB8888,
@@ -1912,9 +2052,12 @@ static const u32 skl_planar_formats[] = { · 7 unmodified lines
19122052 DRM_FORMAT_UYVY,
19132053 DRM_FORMAT_VYUY,
19142054 DRM_FORMAT_NV12,
2055+ DRM_FORMAT_P010,
2056+ DRM_FORMAT_P012,
2057+ DRM_FORMAT_P016,
19152058 };
19162059
1917−static const u32 glk_planar_formats[] = {
2060+static const u32 icl_sdr_y_plane_formats[] = {
19182061 DRM_FORMAT_C8,
19192062 DRM_FORMAT_RGB565,
19202063 DRM_FORMAT_XRGB8888,
@@ -1927,13 +2070,15 @@ static const u32 glk_planar_formats[] = { · 6 unmodified lines
19272070 DRM_FORMAT_YVYU,
19282071 DRM_FORMAT_UYVY,
19292072 DRM_FORMAT_VYUY,
1930− DRM_FORMAT_NV12,
1931− DRM_FORMAT_P010,
1932− DRM_FORMAT_P012,
1933− DRM_FORMAT_P016,
2073+ DRM_FORMAT_Y210,
2074+ DRM_FORMAT_Y212,
2075+ DRM_FORMAT_Y216,
2076+ DRM_FORMAT_XVYU2101010,
2077+ DRM_FORMAT_XVYU12_16161616,
2078+ DRM_FORMAT_XVYU16161616,
19342079 };
19352080
1936−static const u32 icl_planar_formats[] = {
2081+static const u32 icl_sdr_uv_plane_formats[] = {
19372082 DRM_FORMAT_C8,
19382083 DRM_FORMAT_RGB565,
19392084 DRM_FORMAT_XRGB8888,
@@ -1958,7 +2103,7 @@ static const u32 icl_planar_formats[] = { · 18 unmodified lines
19582103 DRM_FORMAT_XVYU16161616,
19592104 };
19602105
1961−static const u32 icl_hdr_planar_formats[] = {
2106+static const u32 icl_hdr_plane_formats[] = {
19622107 DRM_FORMAT_C8,
19632108 DRM_FORMAT_RGB565,
19642109 DRM_FORMAT_XRGB8888,
@@ -2201,9 +2346,6 @@ static bool skl_plane_has_fbc(struct drm_i915_private *dev_priv, · 236 unmodified lines
22012346 static bool skl_plane_has_planar(struct drm_i915_private *dev_priv,
22022347 enum pipe pipe, enum plane_id plane_id)
22032348 {
2204− if (INTEL_GEN(dev_priv) >= 11)
2205− return plane_id <= PLANE_SPRITE3;
2206−
22072349 /* Display WA #0870: skl, bxt */
22082350 if (IS_SKYLAKE(dev_priv) || IS_BROXTON(dev_priv))
22092351 return false;
@@ -2217,6 +2359,48 @@ static bool skl_plane_has_planar(struct drm_i915_private *dev_priv, · 7 unmodified lines
22172359 return true;
22182360 }
22192361
2362+static const u32 *skl_get_plane_formats(struct drm_i915_private *dev_priv,
2363+ enum pipe pipe, enum plane_id plane_id,
2364+ int *num_formats)
2365+{
2366+ if (skl_plane_has_planar(dev_priv, pipe, plane_id)) {
2367+ *num_formats = ARRAY_SIZE(skl_planar_formats);
2368+ return skl_planar_formats;
2369+ } else {
2370+ *num_formats = ARRAY_SIZE(skl_plane_formats);
2371+ return skl_plane_formats;
2372+ }
2373+}
2374+
2375+static const u32 *glk_get_plane_formats(struct drm_i915_private *dev_priv,
2376+ enum pipe pipe, enum plane_id plane_id,
2377+ int *num_formats)
2378+{
2379+ if (skl_plane_has_planar(dev_priv, pipe, plane_id)) {
2380+ *num_formats = ARRAY_SIZE(glk_planar_formats);
2381+ return glk_planar_formats;
2382+ } else {
2383+ *num_formats = ARRAY_SIZE(skl_plane_formats);
2384+ return skl_plane_formats;
2385+ }
2386+}
2387+
2388+static const u32 *icl_get_plane_formats(struct drm_i915_private *dev_priv,
2389+ enum pipe pipe, enum plane_id plane_id,
2390+ int *num_formats)
2391+{
2392+ if (icl_is_hdr_plane(dev_priv, plane_id)) {
2393+ *num_formats = ARRAY_SIZE(icl_hdr_plane_formats);
2394+ return icl_hdr_plane_formats;
2395+ } else if (icl_is_nv12_y_plane(plane_id)) {
2396+ *num_formats = ARRAY_SIZE(icl_sdr_y_plane_formats);
2397+ return icl_sdr_y_plane_formats;
2398+ } else {
2399+ *num_formats = ARRAY_SIZE(icl_sdr_uv_plane_formats);
2400+ return icl_sdr_uv_plane_formats;
2401+ }
2402+}
2403+
22202404 static bool skl_plane_has_ccs(struct drm_i915_private *dev_priv,
22212405 enum pipe pipe, enum plane_id plane_id)
22222406 {
@@ -2270,30 +2454,15 @@ skl_universal_plane_create(struct drm_i915_private *dev_priv, · 47 unmodified lines
22702454 if (icl_is_nv12_y_plane(plane_id))
22712455 plane->update_slave = icl_update_slave;
22722456
2273− if (skl_plane_has_planar(dev_priv, pipe, plane_id)) {
2274− if (icl_is_hdr_plane(dev_priv, plane_id)) {
2275− formats = icl_hdr_planar_formats;
2276− num_formats = ARRAY_SIZE(icl_hdr_planar_formats);
2277− } else if (INTEL_GEN(dev_priv) >= 11) {
2278− formats = icl_planar_formats;
2279− num_formats = ARRAY_SIZE(icl_planar_formats);
2280− } else if (INTEL_GEN(dev_priv) == 10 || IS_GEMINILAKE(dev_priv)) {
2281− formats = glk_planar_formats;
2282− num_formats = ARRAY_SIZE(glk_planar_formats);
2283− } else {
2284− formats = skl_planar_formats;
2285− num_formats = ARRAY_SIZE(skl_planar_formats);
2286− }
2287− } else if (icl_is_hdr_plane(dev_priv, plane_id)) {
2288− formats = icl_hdr_plane_formats;
2289− num_formats = ARRAY_SIZE(icl_hdr_plane_formats);
2290− } else if (INTEL_GEN(dev_priv) >= 11) {
2291− formats = icl_plane_formats;
2292− num_formats = ARRAY_SIZE(icl_plane_formats);
2293− } else {
2294− formats = skl_plane_formats;
2295− num_formats = ARRAY_SIZE(skl_plane_formats);
2296− }
2457+ if (INTEL_GEN(dev_priv) >= 11)
2458+ formats = icl_get_plane_formats(dev_priv, pipe,
2459+ plane_id, &num_formats);
2460+ else if (INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv))
2461+ formats = glk_get_plane_formats(dev_priv, pipe,
2462+ plane_id, &num_formats);
2463+ else
2464+ formats = skl_get_plane_formats(dev_priv, pipe,
2465+ plane_id, &num_formats);
22972466
22982467 plane->has_ccs = skl_plane_has_ccs(dev_priv, pipe, plane_id);
22992468 if (plane->has_ccs)
drivers/gpu/drm/i915/display/intel_tc.c+537−0
@@ -0,0 +1,537 @@
1+// SPDX-License-Identifier: MIT
2+/*
3+ * Copyright © 2019 Intel Corporation
4+ */
5+
6+#include "i915_drv.h"
7+#include "intel_display.h"
8+#include "intel_dp_mst.h"
9+#include "intel_tc.h"
10+
11+static const char *tc_port_mode_name(enum tc_port_mode mode)
12+{
13+ static const char * const names[] = {
14+ [TC_PORT_TBT_ALT] = "tbt-alt",
15+ [TC_PORT_DP_ALT] = "dp-alt",
16+ [TC_PORT_LEGACY] = "legacy",
17+ };
18+
19+ if (WARN_ON(mode >= ARRAY_SIZE(names)))
20+ mode = TC_PORT_TBT_ALT;
21+
22+ return names[mode];
23+}
24+
25+static bool has_modular_fia(struct drm_i915_private *i915)
26+{
27+ if (!INTEL_INFO(i915)->display.has_modular_fia)
28+ return false;
29+
30+ return intel_uncore_read(&i915->uncore,
31+ PORT_TX_DFLEXDPSP(FIA1)) & MODULAR_FIA_MASK;
32+}
33+
34+static enum phy_fia tc_port_to_fia(struct drm_i915_private *i915,
35+ enum tc_port tc_port)
36+{
37+ if (!has_modular_fia(i915))
38+ return FIA1;
39+
40+ /*
41+ * Each Modular FIA instance houses 2 TC ports. In SOC that has more
42+ * than two TC ports, there are multiple instances of Modular FIA.
43+ */
44+ return tc_port / 2;
45+}
46+
47+u32 intel_tc_port_get_lane_mask(struct intel_digital_port *dig_port)
48+{
49+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
50+ enum tc_port tc_port = intel_port_to_tc(i915, dig_port->base.port);
51+ struct intel_uncore *uncore = &i915->uncore;
52+ u32 lane_mask;
53+
54+ lane_mask = intel_uncore_read(uncore,
55+ PORT_TX_DFLEXDPSP(dig_port->tc_phy_fia));
56+
57+ WARN_ON(lane_mask == 0xffffffff);
58+
59+ return (lane_mask & DP_LANE_ASSIGNMENT_MASK(tc_port)) >>
60+ DP_LANE_ASSIGNMENT_SHIFT(tc_port);
61+}
62+
63+int intel_tc_port_fia_max_lane_count(struct intel_digital_port *dig_port)
64+{
65+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
66+ intel_wakeref_t wakeref;
67+ u32 lane_mask;
68+
69+ if (dig_port->tc_mode != TC_PORT_DP_ALT)
70+ return 4;
71+
72+ lane_mask = 0;
73+ with_intel_display_power(i915, POWER_DOMAIN_DISPLAY_CORE, wakeref)
74+ lane_mask = intel_tc_port_get_lane_mask(dig_port);
75+
76+ switch (lane_mask) {
77+ default:
78+ MISSING_CASE(lane_mask);
79+ /* fall-through */
80+ case 0x1:
81+ case 0x2:
82+ case 0x4:
83+ case 0x8:
84+ return 1;
85+ case 0x3:
86+ case 0xc:
87+ return 2;
88+ case 0xf:
89+ return 4;
90+ }
91+}
92+
93+void intel_tc_port_set_fia_lane_count(struct intel_digital_port *dig_port,
94+ int required_lanes)
95+{
96+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
97+ enum tc_port tc_port = intel_port_to_tc(i915, dig_port->base.port);
98+ bool lane_reversal = dig_port->saved_port_bits & DDI_BUF_PORT_REVERSAL;
99+ struct intel_uncore *uncore = &i915->uncore;
100+ u32 val;
101+
102+ WARN_ON(lane_reversal && dig_port->tc_mode != TC_PORT_LEGACY);
103+
104+ val = intel_uncore_read(uncore,
105+ PORT_TX_DFLEXDPMLE1(dig_port->tc_phy_fia));
106+ val &= ~DFLEXDPMLE1_DPMLETC_MASK(tc_port);
107+
108+ switch (required_lanes) {
109+ case 1:
110+ val |= lane_reversal ? DFLEXDPMLE1_DPMLETC_ML3(tc_port) :
111+ DFLEXDPMLE1_DPMLETC_ML0(tc_port);
112+ break;
113+ case 2:
114+ val |= lane_reversal ? DFLEXDPMLE1_DPMLETC_ML3_2(tc_port) :
115+ DFLEXDPMLE1_DPMLETC_ML1_0(tc_port);
116+ break;
117+ case 4:
118+ val |= DFLEXDPMLE1_DPMLETC_ML3_0(tc_port);
119+ break;
120+ default:
121+ MISSING_CASE(required_lanes);
122+ }
123+
124+ intel_uncore_write(uncore,
125+ PORT_TX_DFLEXDPMLE1(dig_port->tc_phy_fia), val);
126+}
127+
128+static void tc_port_fixup_legacy_flag(struct intel_digital_port *dig_port,
129+ u32 live_status_mask)
130+{
131+ u32 valid_hpd_mask;
132+
133+ if (dig_port->tc_legacy_port)
134+ valid_hpd_mask = BIT(TC_PORT_LEGACY);
135+ else
136+ valid_hpd_mask = BIT(TC_PORT_DP_ALT) |
137+ BIT(TC_PORT_TBT_ALT);
138+
139+ if (!(live_status_mask & ~valid_hpd_mask))
140+ return;
141+
142+ /* If live status mismatches the VBT flag, trust the live status. */
143+ DRM_ERROR("Port %s: live status %08x mismatches the legacy port flag, fix flag\n",
144+ dig_port->tc_port_name, live_status_mask);
145+
146+ dig_port->tc_legacy_port = !dig_port->tc_legacy_port;
147+}
148+
149+static u32 tc_port_live_status_mask(struct intel_digital_port *dig_port)
150+{
151+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
152+ enum tc_port tc_port = intel_port_to_tc(i915, dig_port->base.port);
153+ struct intel_uncore *uncore = &i915->uncore;
154+ u32 mask = 0;
155+ u32 val;
156+
157+ val = intel_uncore_read(uncore,
158+ PORT_TX_DFLEXDPSP(dig_port->tc_phy_fia));
159+
160+ if (val == 0xffffffff) {
161+ DRM_DEBUG_KMS("Port %s: PHY in TCCOLD, nothing connected\n",
162+ dig_port->tc_port_name);
163+ return mask;
164+ }
165+
166+ if (val & TC_LIVE_STATE_TBT(tc_port))
167+ mask |= BIT(TC_PORT_TBT_ALT);
168+ if (val & TC_LIVE_STATE_TC(tc_port))
169+ mask |= BIT(TC_PORT_DP_ALT);
170+
171+ if (intel_uncore_read(uncore, SDEISR) & SDE_TC_HOTPLUG_ICP(tc_port))
172+ mask |= BIT(TC_PORT_LEGACY);
173+
174+ /* The sink can be connected only in a single mode. */
175+ if (!WARN_ON(hweight32(mask) > 1))
176+ tc_port_fixup_legacy_flag(dig_port, mask);
177+
178+ return mask;
179+}
180+
181+static bool icl_tc_phy_status_complete(struct intel_digital_port *dig_port)
182+{
183+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
184+ enum tc_port tc_port = intel_port_to_tc(i915, dig_port->base.port);
185+ struct intel_uncore *uncore = &i915->uncore;
186+ u32 val;
187+
188+ val = intel_uncore_read(uncore,
189+ PORT_TX_DFLEXDPPMS(dig_port->tc_phy_fia));
190+ if (val == 0xffffffff) {
191+ DRM_DEBUG_KMS("Port %s: PHY in TCCOLD, assuming not complete\n",
192+ dig_port->tc_port_name);
193+ return false;
194+ }
195+
196+ return val & DP_PHY_MODE_STATUS_COMPLETED(tc_port);
197+}
198+
199+static bool icl_tc_phy_set_safe_mode(struct intel_digital_port *dig_port,
200+ bool enable)
201+{
202+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
203+ enum tc_port tc_port = intel_port_to_tc(i915, dig_port->base.port);
204+ struct intel_uncore *uncore = &i915->uncore;
205+ u32 val;
206+
207+ val = intel_uncore_read(uncore,
208+ PORT_TX_DFLEXDPCSSS(dig_port->tc_phy_fia));
209+ if (val == 0xffffffff) {
210+ DRM_DEBUG_KMS("Port %s: PHY in TCCOLD, can't set safe-mode to %s\n",
211+ dig_port->tc_port_name,
212+ enableddisabled(enable));
213+
214+ return false;
215+ }
216+
217+ val &= ~DP_PHY_MODE_STATUS_NOT_SAFE(tc_port);
218+ if (!enable)
219+ val |= DP_PHY_MODE_STATUS_NOT_SAFE(tc_port);
220+
221+ intel_uncore_write(uncore,
222+ PORT_TX_DFLEXDPCSSS(dig_port->tc_phy_fia), val);
223+
224+ if (enable && wait_for(!icl_tc_phy_status_complete(dig_port), 10))
225+ DRM_DEBUG_KMS("Port %s: PHY complete clear timed out\n",
226+ dig_port->tc_port_name);
227+
228+ return true;
229+}
230+
231+static bool icl_tc_phy_is_in_safe_mode(struct intel_digital_port *dig_port)
232+{
233+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
234+ enum tc_port tc_port = intel_port_to_tc(i915, dig_port->base.port);
235+ struct intel_uncore *uncore = &i915->uncore;
236+ u32 val;
237+
238+ val = intel_uncore_read(uncore,
239+ PORT_TX_DFLEXDPCSSS(dig_port->tc_phy_fia));
240+ if (val == 0xffffffff) {
241+ DRM_DEBUG_KMS("Port %s: PHY in TCCOLD, assume safe mode\n",
242+ dig_port->tc_port_name);
243+ return true;
244+ }
245+
246+ return !(val & DP_PHY_MODE_STATUS_NOT_SAFE(tc_port));
247+}
248+
249+/*
250+ * This function implements the first part of the Connect Flow described by our
251+ * specification, Gen11 TypeC Programming chapter. The rest of the flow (reading
252+ * lanes, EDID, etc) is done as needed in the typical places.
253+ *
254+ * Unlike the other ports, type-C ports are not available to use as soon as we
255+ * get a hotplug. The type-C PHYs can be shared between multiple controllers:
256+ * display, USB, etc. As a result, handshaking through FIA is required around
257+ * connect and disconnect to cleanly transfer ownership with the controller and
258+ * set the type-C power state.
259+ */
260+static void icl_tc_phy_connect(struct intel_digital_port *dig_port,
261+ int required_lanes)
262+{
263+ int max_lanes;
264+
265+ if (!icl_tc_phy_status_complete(dig_port)) {
266+ DRM_DEBUG_KMS("Port %s: PHY not ready\n",
267+ dig_port->tc_port_name);
268+ goto out_set_tbt_alt_mode;
269+ }
270+
271+ if (!icl_tc_phy_set_safe_mode(dig_port, false) &&
272+ !WARN_ON(dig_port->tc_legacy_port))
273+ goto out_set_tbt_alt_mode;
274+
275+ max_lanes = intel_tc_port_fia_max_lane_count(dig_port);
276+ if (dig_port->tc_legacy_port) {
277+ WARN_ON(max_lanes != 4);
278+ dig_port->tc_mode = TC_PORT_LEGACY;
279+
280+ return;
281+ }
282+
283+ /*
284+ * Now we have to re-check the live state, in case the port recently
285+ * became disconnected. Not necessary for legacy mode.
286+ */
287+ if (!(tc_port_live_status_mask(dig_port) & BIT(TC_PORT_DP_ALT))) {
288+ DRM_DEBUG_KMS("Port %s: PHY sudden disconnect\n",
289+ dig_port->tc_port_name);
290+ goto out_set_safe_mode;
291+ }
292+
293+ if (max_lanes < required_lanes) {
294+ DRM_DEBUG_KMS("Port %s: PHY max lanes %d < required lanes %d\n",
295+ dig_port->tc_port_name,
296+ max_lanes, required_lanes);
297+ goto out_set_safe_mode;
298+ }
299+
300+ dig_port->tc_mode = TC_PORT_DP_ALT;
301+
302+ return;
303+
304+out_set_safe_mode:
305+ icl_tc_phy_set_safe_mode(dig_port, true);
306+out_set_tbt_alt_mode:
307+ dig_port->tc_mode = TC_PORT_TBT_ALT;
308+}
309+
310+/*
311+ * See the comment at the connect function. This implements the Disconnect
312+ * Flow.
313+ */
314+static void icl_tc_phy_disconnect(struct intel_digital_port *dig_port)
315+{
316+ switch (dig_port->tc_mode) {
317+ case TC_PORT_LEGACY:
318+ /* Nothing to do, we never disconnect from legacy mode */
319+ break;
320+ case TC_PORT_DP_ALT:
321+ icl_tc_phy_set_safe_mode(dig_port, true);
322+ dig_port->tc_mode = TC_PORT_TBT_ALT;
323+ break;
324+ case TC_PORT_TBT_ALT:
325+ /* Nothing to do, we stay in TBT-alt mode */
326+ break;
327+ default:
328+ MISSING_CASE(dig_port->tc_mode);
329+ }
330+}
331+
332+static bool icl_tc_phy_is_connected(struct intel_digital_port *dig_port)
333+{
334+ if (!icl_tc_phy_status_complete(dig_port)) {
335+ DRM_DEBUG_KMS("Port %s: PHY status not complete\n",
336+ dig_port->tc_port_name);
337+ return dig_port->tc_mode == TC_PORT_TBT_ALT;
338+ }
339+
340+ if (icl_tc_phy_is_in_safe_mode(dig_port)) {
341+ DRM_DEBUG_KMS("Port %s: PHY still in safe mode\n",
342+ dig_port->tc_port_name);
343+
344+ return false;
345+ }
346+
347+ return dig_port->tc_mode == TC_PORT_DP_ALT ||
348+ dig_port->tc_mode == TC_PORT_LEGACY;
349+}
350+
351+static enum tc_port_mode
352+intel_tc_port_get_current_mode(struct intel_digital_port *dig_port)
353+{
354+ u32 live_status_mask = tc_port_live_status_mask(dig_port);
355+ bool in_safe_mode = icl_tc_phy_is_in_safe_mode(dig_port);
356+ enum tc_port_mode mode;
357+
358+ if (in_safe_mode || WARN_ON(!icl_tc_phy_status_complete(dig_port)))
359+ return TC_PORT_TBT_ALT;
360+
361+ mode = dig_port->tc_legacy_port ? TC_PORT_LEGACY : TC_PORT_DP_ALT;
362+ if (live_status_mask) {
363+ enum tc_port_mode live_mode = fls(live_status_mask) - 1;
364+
365+ if (!WARN_ON(live_mode == TC_PORT_TBT_ALT))
366+ mode = live_mode;
367+ }
368+
369+ return mode;
370+}
371+
372+static enum tc_port_mode
373+intel_tc_port_get_target_mode(struct intel_digital_port *dig_port)
374+{
375+ u32 live_status_mask = tc_port_live_status_mask(dig_port);
376+
377+ if (live_status_mask)
378+ return fls(live_status_mask) - 1;
379+
380+ return icl_tc_phy_status_complete(dig_port) &&
381+ dig_port->tc_legacy_port ? TC_PORT_LEGACY :
382+ TC_PORT_TBT_ALT;
383+}
384+
385+static void intel_tc_port_reset_mode(struct intel_digital_port *dig_port,
386+ int required_lanes)
387+{
388+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
389+ enum tc_port_mode old_tc_mode = dig_port->tc_mode;
390+
391+ intel_display_power_flush_work(i915);
392+ WARN_ON(intel_display_power_is_enabled(i915,
393+ intel_aux_power_domain(dig_port)));
394+
395+ icl_tc_phy_disconnect(dig_port);
396+ icl_tc_phy_connect(dig_port, required_lanes);
397+
398+ DRM_DEBUG_KMS("Port %s: TC port mode reset (%s -> %s)\n",
399+ dig_port->tc_port_name,
400+ tc_port_mode_name(old_tc_mode),
401+ tc_port_mode_name(dig_port->tc_mode));
402+}
403+
404+static void
405+intel_tc_port_link_init_refcount(struct intel_digital_port *dig_port,
406+ int refcount)
407+{
408+ WARN_ON(dig_port->tc_link_refcount);
409+ dig_port->tc_link_refcount = refcount;
410+}
411+
412+void intel_tc_port_sanitize(struct intel_digital_port *dig_port)
413+{
414+ struct intel_encoder *encoder = &dig_port->base;
415+ int active_links = 0;
416+
417+ mutex_lock(&dig_port->tc_lock);
418+
419+ dig_port->tc_mode = intel_tc_port_get_current_mode(dig_port);
420+ if (dig_port->dp.is_mst)
421+ active_links = intel_dp_mst_encoder_active_links(dig_port);
422+ else if (encoder->base.crtc)
423+ active_links = to_intel_crtc(encoder->base.crtc)->active;
424+
425+ if (active_links) {
426+ if (!icl_tc_phy_is_connected(dig_port))
427+ DRM_DEBUG_KMS("Port %s: PHY disconnected with %d active link(s)\n",
428+ dig_port->tc_port_name, active_links);
429+ intel_tc_port_link_init_refcount(dig_port, active_links);
430+
431+ goto out;
432+ }
433+
434+ if (dig_port->tc_legacy_port)
435+ icl_tc_phy_connect(dig_port, 1);
436+
437+out:
438+ DRM_DEBUG_KMS("Port %s: sanitize mode (%s)\n",
439+ dig_port->tc_port_name,
440+ tc_port_mode_name(dig_port->tc_mode));
441+
442+ mutex_unlock(&dig_port->tc_lock);
443+}
444+
445+static bool intel_tc_port_needs_reset(struct intel_digital_port *dig_port)
446+{
447+ return intel_tc_port_get_target_mode(dig_port) != dig_port->tc_mode;
448+}
449+
450+/*
451+ * The type-C ports are different because even when they are connected, they may
452+ * not be available/usable by the graphics driver: see the comment on
453+ * icl_tc_phy_connect(). So in our driver instead of adding the additional
454+ * concept of "usable" and make everything check for "connected and usable" we
455+ * define a port as "connected" when it is not only connected, but also when it
456+ * is usable by the rest of the driver. That maintains the old assumption that
457+ * connected ports are usable, and avoids exposing to the users objects they
458+ * can't really use.
459+ */
460+bool intel_tc_port_connected(struct intel_digital_port *dig_port)
461+{
462+ bool is_connected;
463+
464+ intel_tc_port_lock(dig_port);
465+ is_connected = tc_port_live_status_mask(dig_port) &
466+ BIT(dig_port->tc_mode);
467+ intel_tc_port_unlock(dig_port);
468+
469+ return is_connected;
470+}
471+
472+static void __intel_tc_port_lock(struct intel_digital_port *dig_port,
473+ int required_lanes)
474+{
475+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
476+ intel_wakeref_t wakeref;
477+
478+ wakeref = intel_display_power_get(i915, POWER_DOMAIN_DISPLAY_CORE);
479+
480+ mutex_lock(&dig_port->tc_lock);
481+
482+ if (!dig_port->tc_link_refcount &&
483+ intel_tc_port_needs_reset(dig_port))
484+ intel_tc_port_reset_mode(dig_port, required_lanes);
485+
486+ WARN_ON(dig_port->tc_lock_wakeref);
487+ dig_port->tc_lock_wakeref = wakeref;
488+}
489+
490+void intel_tc_port_lock(struct intel_digital_port *dig_port)
491+{
492+ __intel_tc_port_lock(dig_port, 1);
493+}
494+
495+void intel_tc_port_unlock(struct intel_digital_port *dig_port)
496+{
497+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
498+ intel_wakeref_t wakeref = fetch_and_zero(&dig_port->tc_lock_wakeref);
499+
500+ mutex_unlock(&dig_port->tc_lock);
501+
502+ intel_display_power_put_async(i915, POWER_DOMAIN_DISPLAY_CORE,
503+ wakeref);
504+}
505+
506+void intel_tc_port_get_link(struct intel_digital_port *dig_port,
507+ int required_lanes)
508+{
509+ __intel_tc_port_lock(dig_port, required_lanes);
510+ dig_port->tc_link_refcount++;
511+ intel_tc_port_unlock(dig_port);
512+}
513+
514+void intel_tc_port_put_link(struct intel_digital_port *dig_port)
515+{
516+ mutex_lock(&dig_port->tc_lock);
517+ dig_port->tc_link_refcount--;
518+ mutex_unlock(&dig_port->tc_lock);
519+}
520+
521+void intel_tc_port_init(struct intel_digital_port *dig_port, bool is_legacy)
522+{
523+ struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
524+ enum port port = dig_port->base.port;
525+ enum tc_port tc_port = intel_port_to_tc(i915, port);
526+
527+ if (WARN_ON(tc_port == PORT_TC_NONE))
528+ return;
529+
530+ snprintf(dig_port->tc_port_name, sizeof(dig_port->tc_port_name),
531+ "%c/TC#%d", port_name(port), tc_port + 1);
532+
533+ mutex_init(&dig_port->tc_lock);
534+ dig_port->tc_legacy_port = is_legacy;
535+ dig_port->tc_link_refcount = 0;
536+ dig_port->tc_phy_fia = tc_port_to_fia(i915, tc_port);
537+}
drivers/gpu/drm/i915/display/intel_tc.h+35−0
@@ -0,0 +1,35 @@
1+/* SPDX-License-Identifier: MIT */
2+/*
3+ * Copyright © 2019 Intel Corporation
4+ */
5+
6+#ifndef __INTEL_TC_H__
7+#define __INTEL_TC_H__
8+
9+#include <linux/mutex.h>
10+#include <linux/types.h>
11+
12+#include "intel_drv.h"
13+
14+bool intel_tc_port_connected(struct intel_digital_port *dig_port);
15+u32 intel_tc_port_get_lane_mask(struct intel_digital_port *dig_port);
16+int intel_tc_port_fia_max_lane_count(struct intel_digital_port *dig_port);
17+void intel_tc_port_set_fia_lane_count(struct intel_digital_port *dig_port,
18+ int required_lanes);
19+
20+void intel_tc_port_sanitize(struct intel_digital_port *dig_port);
21+void intel_tc_port_lock(struct intel_digital_port *dig_port);
22+void intel_tc_port_unlock(struct intel_digital_port *dig_port);
23+void intel_tc_port_get_link(struct intel_digital_port *dig_port,
24+ int required_lanes);
25+void intel_tc_port_put_link(struct intel_digital_port *dig_port);
26+
27+static inline int intel_tc_port_ref_held(struct intel_digital_port *dig_port)
28+{
29+ return mutex_is_locked(&dig_port->tc_lock) ||
30+ dig_port->tc_link_refcount;
31+}
32+
33+void intel_tc_port_init(struct intel_digital_port *dig_port, bool is_legacy);
34+
35+#endif /* __INTEL_TC_H__ */
drivers/gpu/drm/i915/display/intel_vbt_defs.h+3−0
@@ -310,10 +310,13 @@ enum vbt_gmbus_ddi { · 309 unmodified lines
310310 DDC_BUS_DDI_F,
311311 ICL_DDC_BUS_DDI_A = 0x1,
312312 ICL_DDC_BUS_DDI_B,
313+ TGL_DDC_BUS_DDI_C,
313314 ICL_DDC_BUS_PORT_1 = 0x4,
314315 ICL_DDC_BUS_PORT_2,
315316 ICL_DDC_BUS_PORT_3,
316317 ICL_DDC_BUS_PORT_4,
318+ TGL_DDC_BUS_PORT_5,
319+ TGL_DDC_BUS_PORT_6,
317320 MCC_DDC_BUS_DDI_A = 0x1,
318321 MCC_DDC_BUS_DDI_B,
319322 MCC_DDC_BUS_DDI_C = 0x4,
drivers/gpu/drm/i915/display/intel_vdsc.c+10−4
@@ -459,17 +459,23 @@ int intel_dp_compute_dsc_params(struct intel_dp *intel_dp, · 458 unmodified lines
459459 enum intel_display_power_domain
460460 intel_dsc_power_domain(const struct intel_crtc_state *crtc_state)
461461 {
462+ struct drm_i915_private *i915 = to_i915(crtc_state->base.crtc->dev);
462463 enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
463464
464465 /*
465− * On ICL VDSC/joining for eDP transcoder uses a separate power well PW2
466− * This requires POWER_DOMAIN_TRANSCODER_EDP_VDSC power domain.
466+ * On ICL VDSC/joining for eDP transcoder uses a separate power well,
467+ * PW2. This requires POWER_DOMAIN_TRANSCODER_VDSC_PW2 power domain.
467468 * For any other transcoder, VDSC/joining uses the power well associated
468469 * with the pipe/transcoder in use. Hence another reference on the
469470 * transcoder power domain will suffice.
471+ *
472+ * On TGL we have the same mapping, but for transcoder A (the special
473+ * TRANSCODER_EDP is gone).
470474 */
471− if (cpu_transcoder == TRANSCODER_EDP)
472− return POWER_DOMAIN_TRANSCODER_EDP_VDSC;
475+ if (INTEL_GEN(i915) >= 12 && cpu_transcoder == TRANSCODER_A)
476+ return POWER_DOMAIN_TRANSCODER_VDSC_PW2;
477+ else if (cpu_transcoder == TRANSCODER_EDP)
478+ return POWER_DOMAIN_TRANSCODER_VDSC_PW2;
473479 else
474480 return POWER_DOMAIN_TRANSCODER(cpu_transcoder);
475481 }
drivers/gpu/drm/i915/display/vlv_dsi.c+2−2
@@ -1644,7 +1644,7 @@ vlv_dsi_get_panel_orientation(struct intel_connector *connector) · 1643 unmodified lines
16441644 return intel_dsi_get_panel_orientation(connector);
16451645 }
16461646
1647−static void intel_dsi_add_properties(struct intel_connector *connector)
1647+static void vlv_dsi_add_properties(struct intel_connector *connector)
16481648 {
16491649 struct drm_i915_private *dev_priv = to_i915(connector->base.dev);
16501650
@@ -1983,7 +1983,7 @@ void vlv_dsi_init(struct drm_i915_private *dev_priv) · 332 unmodified lines
19831983 intel_panel_init(&intel_connector->panel, fixed_mode, NULL);
19841984 intel_panel_setup_backlight(connector, INVALID_PIPE);
19851985
1986− intel_dsi_add_properties(intel_connector);
1986+ vlv_dsi_add_properties(intel_connector);
19871987
19881988 return;
19891989
drivers/gpu/drm/i915/gem/Makefile+5−1
@@ -1 +1,5 @@
1−include $(src)/Makefile.header-test # Extra header tests
1+# For building individual subdir files on the command line
2+subdir-ccflags-y += -I$(srctree)/$(src)/..
3+
4+# Extra header tests
5+header-test-pattern-$(CONFIG_DRM_I915_WERROR) := *.h
drivers/gpu/drm/i915/gem/Makefile.header-test+0−16
@@ -1,16 +0,0 @@
1−# SPDX-License-Identifier: MIT
2−# Copyright © 2019 Intel Corporation
3−
4−# Test the headers are compilable as standalone units
5−header_test := $(notdir $(wildcard $(src)/*.h))
6−
7−quiet_cmd_header_test = HDRTEST $@
8− cmd_header_test = echo "\#include \"$(<F)\"" > $@
9−
10−header_test_%.c: %.h
11− $(call cmd,header_test)
12−
13−extra-$(CONFIG_DRM_I915_WERROR) += \
14− $(foreach h,$(header_test),$(patsubst %.h,header_test_%.o,$(h)))
15−
16−clean-files += $(foreach h,$(header_test),$(patsubst %.h,header_test_%.c,$(h)))
drivers/gpu/drm/i915/gem/i915_gem_client_blt.c+10−11
@@ -72,7 +72,6 @@ static struct i915_sleeve *create_sleeve(struct i915_address_space *vm, · 71 unmodified lines
7272 vma->ops = &proxy_vma_ops;
7373
7474 sleeve->vma = vma;
75− sleeve->obj = i915_gem_object_get(obj);
7675 sleeve->pages = pages;
7776 sleeve->page_sizes = *page_sizes;
7877
@@ -85,7 +84,6 @@ err_free: · 6 unmodified lines
8584
8685 static void destroy_sleeve(struct i915_sleeve *sleeve)
8786 {
88− i915_gem_object_put(sleeve->obj);
8987 kfree(sleeve);
9088 }
9189
@@ -155,7 +153,7 @@ static void clear_pages_worker(struct work_struct *work) · 63 unmodified lines
155153 {
156154 struct clear_pages_work *w = container_of(work, typeof(*w), work);
157155 struct drm_i915_private *i915 = w->ce->gem_context->i915;
158− struct drm_i915_gem_object *obj = w->sleeve->obj;
156+ struct drm_i915_gem_object *obj = w->sleeve->vma->obj;
159157 struct i915_vma *vma = w->sleeve->vma;
160158 struct i915_request *rq;
161159 int err = w->dma.error;
@@ -164,11 +162,12 @@ static void clear_pages_worker(struct work_struct *work) · 2 unmodified lines
164162 goto out_signal;
165163
166164 if (obj->cache_dirty) {
167− obj->write_domain = 0;
168165 if (i915_gem_object_has_struct_page(obj))
169166 drm_clflush_sg(w->sleeve->pages);
170167 obj->cache_dirty = false;
171168 }
169+ obj->read_domains = I915_GEM_GPU_DOMAINS;
170+ obj->write_domain = 0;
172171
173172 /* XXX: we need to kill this */
174173 mutex_lock(&i915->drm.struct_mutex);
@@ -193,10 +192,12 @@ static void clear_pages_worker(struct work_struct *work) · 18 unmodified lines
193192 goto out_request;
194193 }
195194
196− /* XXX: more feverish nightmares await */
197− i915_vma_lock(vma);
198− err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
199− i915_vma_unlock(vma);
195+ /*
196+ * w->dma is already exported via (vma|obj)->resv we need only
197+ * keep track of the GPU activity within this vma/request, and
198+ * propagate the signal from the request to w->dma.
199+ */
200+ err = i915_active_ref(&vma->active, rq->fence.context, rq);
200201 if (err)
201202 goto out_request;
202203
@@ -249,13 +250,11 @@ int i915_gem_schedule_fill_pages_blt(struct drm_i915_gem_object *obj, · 46 unmodified lines
249250 u32 value)
250251 {
251252 struct drm_i915_private *i915 = to_i915(obj->base.dev);
252− struct i915_gem_context *ctx = ce->gem_context;
253− struct i915_address_space *vm = ctx->vm ?: &i915->ggtt.vm;
254253 struct clear_pages_work *work;
255254 struct i915_sleeve *sleeve;
256255 int err;
257256
258− sleeve = create_sleeve(vm, obj, pages, page_sizes);
257+ sleeve = create_sleeve(ce->vm, obj, pages, page_sizes);
259258 if (IS_ERR(sleeve))
260259 return PTR_ERR(sleeve);
261260
drivers/gpu/drm/i915/gem/i915_gem_context.c+30−53
@@ -316,7 +316,7 @@ static void i915_gem_context_free(struct i915_gem_context *ctx) · 315 unmodified lines
316316 mutex_destroy(&ctx->engines_mutex);
317317
318318 if (ctx->timeline)
319− i915_timeline_put(ctx->timeline);
319+ intel_timeline_put(ctx->timeline);
320320
321321 kfree(ctx->name);
322322 put_pid(ctx->pid);
@@ -459,8 +459,7 @@ __create_context(struct drm_i915_private *i915) · 136 unmodified lines
459459 i915_gem_context_set_recoverable(ctx);
460460
461461 ctx->ring_size = 4 * PAGE_SIZE;
462− ctx->desc_template =
463− default_desc_template(i915, &i915->mm.aliasing_ppgtt->vm);
462+ ctx->desc_template = default_desc_template(i915, NULL);
464463
465464 for (i = 0; i < ARRAY_SIZE(ctx->hang_timestamp); i++)
466465 ctx->hang_timestamp[i] = jiffies - CONTEXT_FAST_HANG_JIFFIES;
@@ -476,10 +475,18 @@ static struct i915_address_space * · 9 unmodified lines
476475 __set_ppgtt(struct i915_gem_context *ctx, struct i915_address_space *vm)
477476 {
478477 struct i915_address_space *old = ctx->vm;
478+ struct i915_gem_engines_iter it;
479+ struct intel_context *ce;
479480
480481 ctx->vm = i915_vm_get(vm);
481482 ctx->desc_template = default_desc_template(ctx->i915, vm);
482483
484+ for_each_gem_engine(ce, i915_gem_context_lock_engines(ctx), it) {
485+ i915_vm_put(ce->vm);
486+ ce->vm = i915_vm_get(vm);
487+ }
488+ i915_gem_context_unlock_engines(ctx);
489+
483490 return old;
484491 }
485492
@@ -528,9 +535,9 @@ i915_gem_create_context(struct drm_i915_private *dev_priv, unsigned int flags) · 42 unmodified lines
528535 }
529536
530537 if (flags & I915_CONTEXT_CREATE_FLAGS_SINGLE_TIMELINE) {
531− struct i915_timeline *timeline;
538+ struct intel_timeline *timeline;
532539
533− timeline = i915_timeline_create(dev_priv, NULL);
540+ timeline = intel_timeline_create(&dev_priv->gt, NULL);
534541 if (IS_ERR(timeline)) {
535542 context_close(ctx);
536543 return ERR_CAST(timeline);
@@ -644,20 +651,13 @@ static void init_contexts(struct drm_i915_private *i915) · 107 unmodified lines
644651 init_llist_head(&i915->contexts.free_list);
645652 }
646653
647−static bool needs_preempt_context(struct drm_i915_private *i915)
648−{
649− return HAS_EXECLISTS(i915);
650−}
651−
652654 int i915_gem_contexts_init(struct drm_i915_private *dev_priv)
653655 {
654656 struct i915_gem_context *ctx;
655657
656658 /* Reassure ourselves we are only called once */
657659 GEM_BUG_ON(dev_priv->kernel_context);
658− GEM_BUG_ON(dev_priv->preempt_context);
659660
660− intel_engine_init_ctx_wa(dev_priv->engine[RCS0]);
661661 init_contexts(dev_priv);
662662
663663 /* lowest priority; idle task */
@@ -677,15 +677,6 @@ int i915_gem_contexts_init(struct drm_i915_private *dev_priv) · 13 unmodified lines
677677 GEM_BUG_ON(!atomic_read(&ctx->hw_id_pin_count));
678678 dev_priv->kernel_context = ctx;
679679
680− /* highest priority; preempting task */
681− if (needs_preempt_context(dev_priv)) {
682− ctx = i915_gem_context_create_kernel(dev_priv, INT_MAX);
683− if (!IS_ERR(ctx))
684− dev_priv->preempt_context = ctx;
685− else
686− DRM_ERROR("Failed to create preempt context; disabling preemption\n");
687− }
688−
689680 DRM_DEBUG_DRIVER("%s context support initialized\n",
690681 DRIVER_CAPS(dev_priv)->has_logical_contexts ?
691682 "logical" : "fake");
@@ -696,8 +687,6 @@ void i915_gem_contexts_fini(struct drm_i915_private *i915) · 4 unmodified lines
696687 {
697688 lockdep_assert_held(&i915->drm.struct_mutex);
698689
699− if (i915->preempt_context)
700− destroy_kernel_context(&i915->preempt_context);
701690 destroy_kernel_context(&i915->kernel_context);
702691
703692 /* Must free all deferred contexts (via flush_workqueue) first */
@@ -923,8 +912,12 @@ static int context_barrier_task(struct i915_gem_context *ctx, · 219 unmodified lines
923912 if (!cb)
924913 return -ENOMEM;
925914
926− i915_active_init(i915, &cb->base, cb_retire);
927− i915_active_acquire(&cb->base);
915+ i915_active_init(i915, &cb->base, NULL, cb_retire);
916+ err = i915_active_acquire(&cb->base);
917+ if (err) {
918+ kfree(cb);
919+ return err;
920+ }
928921
929922 for_each_gem_engine(ce, i915_gem_context_lock_engines(ctx), it) {
930923 struct i915_request *rq;
@@ -1019,7 +1012,7 @@ static void set_ppgtt_barrier(void *data) · 88 unmodified lines
10191012
10201013 static int emit_ppgtt_update(struct i915_request *rq, void *data)
10211014 {
1022− struct i915_address_space *vm = rq->gem_context->vm;
1015+ struct i915_address_space *vm = rq->hw_context->vm;
10231016 struct intel_engine_cs *engine = rq->engine;
10241017 u32 base = engine->mmio_base;
10251018 u32 *cs;
@@ -1128,9 +1121,8 @@ static int set_ppgtt(struct drm_i915_file_private *file_priv, · 102 unmodified lines
11281121 set_ppgtt_barrier,
11291122 old);
11301123 if (err) {
1131− ctx->vm = old;
1132− ctx->desc_template = default_desc_template(ctx->i915, old);
1133− i915_vm_put(vm);
1124+ i915_vm_put(__set_ppgtt(ctx, old));
1125+ i915_vm_put(old);
11341126 }
11351127
11361128 unlock:
@@ -1187,26 +1179,11 @@ gen8_modify_rpcs(struct intel_context *ce, struct intel_sseu sseu) · 50 unmodified lines
11871179 if (IS_ERR(rq))
11881180 return PTR_ERR(rq);
11891181
1190− /* Queue this switch after all other activity by this context. */
1191− ret = i915_active_request_set(&ce->ring->timeline->last_request, rq);
1192− if (ret)
1193− goto out_add;
1194−
1195− /*
1196− * Guarantee context image and the timeline remains pinned until the
1197− * modifying request is retired by setting the ce activity tracker.
1198− *
1199− * But we only need to take one pin on the account of it. Or in other
1200− * words transfer the pinned ce object to tracked active request.
1201− */
1202− GEM_BUG_ON(i915_active_is_idle(&ce->active));
1203− ret = i915_active_ref(&ce->active, rq->fence.context, rq);
1204− if (ret)
1205− goto out_add;
1206−
1207− ret = gen8_emit_rpcs_config(rq, ce, sseu);
1182+ /* Serialise with the remote context */
1183+ ret = intel_context_prepare_remote_request(ce, rq);
1184+ if (ret == 0)
1185+ ret = gen8_emit_rpcs_config(rq, ce, sseu);
12081186
1209−out_add:
12101187 i915_request_add(rq);
12111188 return ret;
12121189 }
@@ -2015,8 +1992,8 @@ static int clone_timeline(struct i915_gem_context *dst, · 802 unmodified lines
20151992 GEM_BUG_ON(src->timeline == dst->timeline);
20161993
20171994 if (dst->timeline)
2018− i915_timeline_put(dst->timeline);
2019− dst->timeline = i915_timeline_get(src->timeline);
1995+ intel_timeline_put(dst->timeline);
1996+ dst->timeline = intel_timeline_get(src->timeline);
20201997 }
20211998
20221999 return 0;
@@ -2141,7 +2118,7 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data, · 118 unmodified lines
21412118 if (args->flags & I915_CONTEXT_CREATE_FLAGS_UNKNOWN)
21422119 return -EINVAL;
21432120
2144− ret = i915_terminally_wedged(i915);
2121+ ret = intel_gt_terminally_wedged(&i915->gt);
21452122 if (ret)
21462123 return ret;
21472124
@@ -2287,8 +2264,8 @@ int i915_gem_context_getparam_ioctl(struct drm_device *dev, void *data, · 139 unmodified lines
22872264 args->size = 0;
22882265 if (ctx->vm)
22892266 args->value = ctx->vm->total;
2290− else if (to_i915(dev)->mm.aliasing_ppgtt)
2291− args->value = to_i915(dev)->mm.aliasing_ppgtt->vm.total;
2267+ else if (to_i915(dev)->ggtt.alias)
2268+ args->value = to_i915(dev)->ggtt.alias->vm.total;
22922269 else
22932270 args->value = to_i915(dev)->ggtt.vm.total;
22942271 break;
drivers/gpu/drm/i915/gem/i915_gem_context.h+0−6
@@ -197,12 +197,6 @@ i915_gem_context_unlock_engines(struct i915_gem_context *ctx) · 196 unmodified lines
197197 mutex_unlock(&ctx->engines_mutex);
198198 }
199199
200−static inline struct intel_context *
201−i915_gem_context_lookup_engine(struct i915_gem_context *ctx, unsigned int idx)
202−{
203− return i915_gem_context_engines(ctx)->engines[idx];
204−}
205−
206200 static inline struct intel_context *
207201 i915_gem_context_get_engine(struct i915_gem_context *ctx, unsigned int idx)
208202 {
drivers/gpu/drm/i915/gem/i915_gem_context_types.h+2−2
@@ -26,7 +26,7 @@ struct pid; · 25 unmodified lines
2626 struct drm_i915_private;
2727 struct drm_i915_file_private;
2828 struct i915_address_space;
29−struct i915_timeline;
29+struct intel_timeline;
3030 struct intel_ring;
3131
3232 struct i915_gem_engines {
@@ -77,7 +77,7 @@ struct i915_gem_context { · 44 unmodified lines
7777 struct i915_gem_engines __rcu *engines;
7878 struct mutex engines_mutex; /* guards writes to engines */
7979
80− struct i915_timeline *timeline;
80+ struct intel_timeline *timeline;
8181
8282 /**
8383 * @vm: unique address space (GTT)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c+9−13
@@ -16,6 +16,7 @@ · 15 unmodified lines
1616
1717 #include "gem/i915_gem_ioctls.h"
1818 #include "gt/intel_context.h"
19+#include "gt/intel_gt.h"
1920 #include "gt/intel_gt_pm.h"
2021
2122 #include "i915_gem_ioctls.h"
@@ -222,7 +223,6 @@ struct i915_execbuffer { · 200 unmodified lines
222223 struct intel_engine_cs *engine; /** engine to queue the request to */
223224 struct intel_context *context; /* logical state for the request */
224225 struct i915_gem_context *gem_context; /** caller's context */
225− struct i915_address_space *vm; /** GTT and vma for the request */
226226
227227 struct i915_request *request; /** our request to build */
228228 struct i915_vma *batch; /** identity of the batch obj/vma */
@@ -696,7 +696,7 @@ static int eb_reserve(struct i915_execbuffer *eb) · 467 unmodified lines
696696
697697 case 1:
698698 /* Too fragmented, unbind everything and retry */
699− err = i915_gem_evict_vm(eb->vm);
699+ err = i915_gem_evict_vm(eb->context->vm);
700700 if (err)
701701 return err;
702702 break;
@@ -724,12 +724,8 @@ static int eb_select_context(struct i915_execbuffer *eb) · 21 unmodified lines
724724 return -ENOENT;
725725
726726 eb->gem_context = ctx;
727− if (ctx->vm) {
728− eb->vm = ctx->vm;
727+ if (ctx->vm)
729728 eb->invalid_flags |= EXEC_OBJECT_NEEDS_GTT;
730− } else {
731− eb->vm = &eb->i915->ggtt.vm;
732− }
733729
734730 eb->context_flags = 0;
735731 if (test_bit(UCONTEXT_NO_ZEROMAP, &ctx->user_flags))
@@ -831,7 +827,7 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb) · 95 unmodified lines
831827 goto err_vma;
832828 }
833829
834− vma = i915_vma_instance(obj, eb->vm, NULL);
830+ vma = i915_vma_instance(obj, eb->context->vm, NULL);
835831 if (IS_ERR(vma)) {
836832 err = PTR_ERR(vma);
837833 goto err_obj;
@@ -994,7 +990,7 @@ static void reloc_gpu_flush(struct reloc_cache *cache) · 156 unmodified lines
994990 __i915_gem_object_flush_map(cache->rq->batch->obj, 0, cache->rq_size);
995991 i915_gem_object_unpin_map(cache->rq->batch->obj);
996992
997− i915_gem_chipset_flush(cache->rq->i915);
993+ intel_gt_chipset_flush(cache->rq->engine->gt);
998994
999995 i915_request_add(cache->rq);
1000996 cache->rq = NULL;
@@ -1954,7 +1950,7 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb) · 953 unmodified lines
19541950 eb->exec = NULL;
19551951
19561952 /* Unconditionally flush any chipset caches (for streaming writes). */
1957− i915_gem_chipset_flush(eb->i915);
1953+ intel_gt_chipset_flush(eb->engine->gt);
19581954 return 0;
19591955
19601956 err_skip:
@@ -2129,7 +2125,7 @@ static int eb_pin_context(struct i915_execbuffer *eb, struct intel_context *ce) · 168 unmodified lines
21292125 * ABI: Before userspace accesses the GPU (e.g. execbuffer), report
21302126 * EIO if the GPU is already wedged.
21312127 */
2132− err = i915_terminally_wedged(eb->i915);
2128+ err = intel_gt_terminally_wedged(ce->engine->gt);
21332129 if (err)
21342130 return err;
21352131
@@ -2436,7 +2432,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 300 unmodified lines
24362432 * wakeref that we hold until the GPU has been idle for at least
24372433 * 100ms.
24382434 */
2439− intel_gt_pm_get(eb.i915);
2435+ intel_gt_pm_get(&eb.i915->gt);
24402436
24412437 err = i915_mutex_lock_interruptible(dev);
24422438 if (err)
@@ -2606,7 +2602,7 @@ err_engine: · 163 unmodified lines
26062602 err_unlock:
26072603 mutex_unlock(&dev->struct_mutex);
26082604 err_rpm:
2609− intel_gt_pm_put(eb.i915);
2605+ intel_gt_pm_put(&eb.i915->gt);
26102606 i915_gem_context_put(eb.gem_context);
26112607 err_destroy:
26122608 eb_destroy(&eb);
drivers/gpu/drm/i915/gem/i915_gem_mman.c+5−3
@@ -7,6 +7,8 @@ · 6 unmodified lines
77 #include <linux/mman.h>
88 #include <linux/sizes.h>
99
10+#include "gt/intel_gt.h"
11+
1012 #include "i915_drv.h"
1113 #include "i915_gem_gtt.h"
1214 #include "i915_gem_ioctls.h"
@@ -246,7 +248,7 @@ vm_fault_t i915_gem_fault(struct vm_fault *vmf) · 233 unmodified lines
246248
247249 wakeref = intel_runtime_pm_get(rpm);
248250
249− srcu = i915_reset_trylock(i915);
251+ srcu = intel_gt_reset_trylock(ggtt->vm.gt);
250252 if (srcu < 0) {
251253 ret = srcu;
252254 goto err_rpm;
@@ -326,7 +328,7 @@ err_unpin: · 73 unmodified lines
326328 err_unlock:
327329 mutex_unlock(&dev->struct_mutex);
328330 err_reset:
329− i915_reset_unlock(i915, srcu);
331+ intel_gt_reset_unlock(ggtt->vm.gt, srcu);
330332 err_rpm:
331333 intel_runtime_pm_put(rpm, wakeref);
332334 i915_gem_object_unpin_pages(obj);
@@ -339,7 +341,7 @@ err: · 6 unmodified lines
339341 * fail). But any other -EIO isn't ours (e.g. swap in failure)
340342 * and so needs to be reported.
341343 */
342− if (!i915_terminally_wedged(i915))
344+ if (!intel_gt_is_wedged(ggtt->vm.gt))
343345 return VM_FAULT_SIGBUS;
344346 /* else, fall through */
345347 case -EAGAIN:
drivers/gpu/drm/i915/gem/i915_gem_object.c+46−51
@@ -23,7 +23,7 @@ · 22 unmodified lines
2323 */
2424
2525 #include "display/intel_frontbuffer.h"
26−
26+#include "gt/intel_gt.h"
2727 #include "i915_drv.h"
2828 #include "i915_gem_clflush.h"
2929 #include "i915_gem_context.h"
@@ -146,6 +146,19 @@ void i915_gem_close_object(struct drm_gem_object *gem, struct drm_file *file) · 116 unmodified lines
146146 }
147147 }
148148
149+static void __i915_gem_free_object_rcu(struct rcu_head *head)
150+{
151+ struct drm_i915_gem_object *obj =
152+ container_of(head, typeof(*obj), rcu);
153+ struct drm_i915_private *i915 = to_i915(obj->base.dev);
154+
155+ reservation_object_fini(&obj->base._resv);
156+ i915_gem_object_free(obj);
157+
158+ GEM_BUG_ON(!atomic_read(&i915->mm.free_count));
159+ atomic_dec(&i915->mm.free_count);
160+}
161+
149162 static void __i915_gem_free_objects(struct drm_i915_private *i915,
150163 struct llist_node *freed)
151164 {
@@ -160,7 +173,6 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, · 8 unmodified lines
160173
161174 mutex_lock(&i915->drm.struct_mutex);
162175
163− GEM_BUG_ON(i915_gem_object_is_active(obj));
164176 list_for_each_entry_safe(vma, vn, &obj->vma.list, obj_link) {
165177 GEM_BUG_ON(i915_vma_is_active(vma));
166178 vma->flags &= ~I915_VMA_PIN_MASK;
@@ -169,22 +181,6 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, · 2 unmodified lines
169181 GEM_BUG_ON(!list_empty(&obj->vma.list));
170182 GEM_BUG_ON(!RB_EMPTY_ROOT(&obj->vma.tree));
171183
172− /*
173− * This serializes freeing with the shrinker. Since the free
174− * is delayed, first by RCU then by the workqueue, we want the
175− * shrinker to be able to free pages of unreferenced objects,
176− * or else we may oom whilst there are plenty of deferred
177− * freed objects.
178− */
179− if (i915_gem_object_has_pages(obj) &&
180− i915_gem_object_is_shrinkable(obj)) {
181− unsigned long flags;
182−
183− spin_lock_irqsave(&i915->mm.obj_lock, flags);
184− list_del_init(&obj->mm.link);
185− spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
186− }
187−
188184 mutex_unlock(&i915->drm.struct_mutex);
189185
190186 GEM_BUG_ON(atomic_read(&obj->bind_count));
@@ -192,25 +188,21 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, · 1 unmodified line
192188 GEM_BUG_ON(atomic_read(&obj->frontbuffer_bits));
193189 GEM_BUG_ON(!list_empty(&obj->lut_list));
194190
195− if (obj->ops->release)
196− obj->ops->release(obj);
197−
198191 atomic_set(&obj->mm.pages_pin_count, 0);
199192 __i915_gem_object_put_pages(obj, I915_MM_NORMAL);
200193 GEM_BUG_ON(i915_gem_object_has_pages(obj));
194+ bitmap_free(obj->bit_17);
201195
202196 if (obj->base.import_attach)
203197 drm_prime_gem_destroy(&obj->base, NULL);
204198
205− drm_gem_object_release(&obj->base);
199+ drm_gem_free_mmap_offset(&obj->base);
206200
207− bitmap_free(obj->bit_17);
208− i915_gem_object_free(obj);
209−
210− GEM_BUG_ON(!atomic_read(&i915->mm.free_count));
211− atomic_dec(&i915->mm.free_count);
201+ if (obj->ops->release)
202+ obj->ops->release(obj);
212203
213− cond_resched();
204+ /* But keep the pointer alive for RCU-protected lookups */
205+ call_rcu(&obj->rcu, __i915_gem_free_object_rcu);
214206 }
215207 intel_runtime_pm_put(&i915->runtime_pm, wakeref);
216208 }
@@ -261,18 +253,34 @@ static void __i915_gem_free_work(struct work_struct *work) · 44 unmodified lines
261253 spin_unlock(&i915->mm.free_lock);
262254 }
263255
264−static void __i915_gem_free_object_rcu(struct rcu_head *head)
256+void i915_gem_free_object(struct drm_gem_object *gem_obj)
265257 {
266− struct drm_i915_gem_object *obj =
267− container_of(head, typeof(*obj), rcu);
258+ struct drm_i915_gem_object *obj = to_intel_bo(gem_obj);
268259 struct drm_i915_private *i915 = to_i915(obj->base.dev);
269260
270261 /*
271− * We reuse obj->rcu for the freed list, so we had better not treat
272− * it like a rcu_head from this point forwards. And we expect all
273− * objects to be freed via this path.
262+ * Before we free the object, make sure any pure RCU-only
263+ * read-side critical sections are complete, e.g.
264+ * i915_gem_busy_ioctl(). For the corresponding synchronized
265+ * lookup see i915_gem_object_lookup_rcu().
274266 */
275− destroy_rcu_head(&obj->rcu);
267+ atomic_inc(&i915->mm.free_count);
268+
269+ /*
270+ * This serializes freeing with the shrinker. Since the free
271+ * is delayed, first by RCU then by the workqueue, we want the
272+ * shrinker to be able to free pages of unreferenced objects,
273+ * or else we may oom whilst there are plenty of deferred
274+ * freed objects.
275+ */
276+ if (i915_gem_object_has_pages(obj) &&
277+ i915_gem_object_is_shrinkable(obj)) {
278+ unsigned long flags;
279+
280+ spin_lock_irqsave(&i915->mm.obj_lock, flags);
281+ list_del_init(&obj->mm.link);
282+ spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
283+ }
276284
277285 /*
278286 * Since we require blocking on struct_mutex to unbind the freed
@@ -288,20 +296,6 @@ static void __i915_gem_free_object_rcu(struct rcu_head *head) · 9 unmodified lines
288296 queue_work(i915->wq, &i915->mm.free_work);
289297 }
290298
291−void i915_gem_free_object(struct drm_gem_object *gem_obj)
292−{
293− struct drm_i915_gem_object *obj = to_intel_bo(gem_obj);
294−
295− /*
296− * Before we free the object, make sure any pure RCU-only
297− * read-side critical sections are complete, e.g.
298− * i915_gem_busy_ioctl(). For the corresponding synchronized
299− * lookup see i915_gem_object_lookup_rcu().
300− */
301− atomic_inc(&to_i915(obj->base.dev)->mm.free_count);
302− call_rcu(&obj->rcu, __i915_gem_free_object_rcu);
303−}
304−
305299 static inline enum fb_op_origin
306300 fb_write_origin(struct drm_i915_gem_object *obj, unsigned int domain)
307301 {
@@ -319,7 +313,6 @@ void · 11 unmodified lines
319313 i915_gem_object_flush_write_domain(struct drm_i915_gem_object *obj,
320314 unsigned int flush_domains)
321315 {
322− struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
323316 struct i915_vma *vma;
324317
325318 assert_object_held(obj);
@@ -329,7 +322,8 @@ i915_gem_object_flush_write_domain(struct drm_i915_gem_object *obj, · 3 unmodified lines
329322
330323 switch (obj->write_domain) {
331324 case I915_GEM_DOMAIN_GTT:
332− i915_gem_flush_ggtt_writes(dev_priv);
325+ for_each_ggtt_vma(vma, obj)
326+ intel_gt_flush_ggtt_writes(vma->vm->gt);
333327
334328 intel_fb_obj_flush(obj,
335329 fb_write_origin(obj, I915_GEM_DOMAIN_GTT));
@@ -340,6 +334,7 @@ i915_gem_object_flush_write_domain(struct drm_i915_gem_object *obj, · 4 unmodified lines
340334
341335 i915_vma_unset_ggtt_write(vma);
342336 }
337+
343338 break;
344339
345340 case I915_GEM_DOMAIN_WC:
drivers/gpu/drm/i915/gem/i915_gem_object.h+1−7
@@ -81,7 +81,7 @@ i915_gem_object_lookup(struct drm_file *file, u32 handle) · 80 unmodified lines
8181 }
8282
8383 __deprecated
84−extern struct drm_gem_object *
84+struct drm_gem_object *
8585 drm_gem_object_lookup(struct drm_file *file, u32 handle);
8686
8787 __attribute__((nonnull))
@@ -158,12 +158,6 @@ i915_gem_object_needs_async_cancel(const struct drm_i915_gem_object *obj) · 70 unmodified lines
158158 return obj->ops->flags & I915_GEM_OBJECT_ASYNC_CANCEL;
159159 }
160160
161−static inline bool
162−i915_gem_object_is_active(const struct drm_i915_gem_object *obj)
163−{
164− return READ_ONCE(obj->active_count);
165−}
166−
167161 static inline bool
168162 i915_gem_object_is_framebuffer(const struct drm_i915_gem_object *obj)
169163 {
drivers/gpu/drm/i915/gem/i915_gem_object_blt.c+1−5
@@ -47,15 +47,11 @@ int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj, · 46 unmodified lines
4747 struct intel_context *ce,
4848 u32 value)
4949 {
50− struct drm_i915_private *i915 = to_i915(obj->base.dev);
51− struct i915_gem_context *ctx = ce->gem_context;
52− struct i915_address_space *vm = ctx->vm ?: &i915->ggtt.vm;
5350 struct i915_request *rq;
5451 struct i915_vma *vma;
5552 int err;
5653
57− /* XXX: ce->vm please */
58− vma = i915_vma_instance(obj, vm, NULL);
54+ vma = i915_vma_instance(obj, ce->vm, NULL);
5955 if (IS_ERR(vma))
6056 return PTR_ERR(vma);
6157
drivers/gpu/drm/i915/gem/i915_gem_object_types.h+0−1
@@ -154,7 +154,6 @@ struct drm_i915_gem_object { · 153 unmodified lines
154154
155155 /** Count of VMA actually bound by this object */
156156 atomic_t bind_count;
157− unsigned int active_count;
158157 /** Count of how many global VMA are currently pinned for use by HW */
159158 unsigned int pin_global;
160159
drivers/gpu/drm/i915/gem/i915_gem_phys.c+3−9
@@ -13,6 +13,7 @@ · 12 unmodified lines
1313 #include <drm/drm_legacy.h> /* for drm_pci.h! */
1414 #include <drm/drm_pci.h>
1515
16+#include "gt/intel_gt.h"
1617 #include "i915_drv.h"
1718 #include "i915_gem_object.h"
1819 #include "i915_scatterlist.h"
@@ -60,7 +61,7 @@ static int i915_gem_object_get_pages_phys(struct drm_i915_gem_object *obj) · 41 unmodified lines
6061 vaddr += PAGE_SIZE;
6162 }
6263
63− i915_gem_chipset_flush(to_i915(obj->base.dev));
64+ intel_gt_chipset_flush(&to_i915(obj->base.dev)->gt);
6465
6566 st = kmalloc(sizeof(*st), GFP_KERNEL);
6667 if (!st) {
@@ -132,16 +133,9 @@ i915_gem_object_put_pages_phys(struct drm_i915_gem_object *obj, · 65 unmodified lines
132133 drm_pci_free(obj->base.dev, obj->phys_handle);
133134 }
134135
135−static void
136−i915_gem_object_release_phys(struct drm_i915_gem_object *obj)
137−{
138− i915_gem_object_unpin_pages(obj);
139−}
140−
141136 static const struct drm_i915_gem_object_ops i915_gem_phys_ops = {
142137 .get_pages = i915_gem_object_get_pages_phys,
143138 .put_pages = i915_gem_object_put_pages_phys,
144− .release = i915_gem_object_release_phys,
145139 };
146140
147141 int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
@@ -158,7 +152,7 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align) · 10 unmodified lines
158152 if (obj->ops != &i915_gem_shmem_ops)
159153 return -EINVAL;
160154
161− err = i915_gem_object_unbind(obj);
155+ err = i915_gem_object_unbind(obj, I915_GEM_OBJECT_UNBIND_ACTIVE);
162156 if (err)
163157 return err;
164158
drivers/gpu/drm/i915/gem/i915_gem_pm.c+19−18
@@ -5,6 +5,7 @@ · 4 unmodified lines
55 */
66
77 #include "gem/i915_gem_pm.h"
8+#include "gt/intel_gt.h"
89 #include "gt/intel_gt_pm.h"
910
1011 #include "i915_drv.h"
@@ -38,7 +39,7 @@ static void i915_gem_park(struct drm_i915_private *i915) · 27 unmodified lines
3839 i915_gem_batch_pool_fini(&engine->batch_pool);
3940 }
4041
41− i915_timelines_park(i915);
42+ intel_timelines_park(i915);
4243 i915_vma_parked(i915);
4344
4445 i915_globals_park();
@@ -54,7 +55,8 @@ static void idle_work_handler(struct work_struct *work) · 9 unmodified lines
5455 mutex_lock(&i915->drm.struct_mutex);
5556
5657 intel_wakeref_lock(&i915->gt.wakeref);
57− park = !intel_wakeref_active(&i915->gt.wakeref) && !work_pending(work);
58+ park = (!intel_wakeref_is_active(&i915->gt.wakeref) &&
59+ !work_pending(work));
5860 intel_wakeref_unlock(&i915->gt.wakeref);
5961 if (park)
6062 i915_gem_park(i915);
@@ -105,18 +107,18 @@ static int pm_notifier(struct notifier_block *nb, · 44 unmodified lines
105107 return NOTIFY_OK;
106108 }
107109
108−static bool switch_to_kernel_context_sync(struct drm_i915_private *i915)
110+static bool switch_to_kernel_context_sync(struct intel_gt *gt)
109111 {
110− bool result = !i915_terminally_wedged(i915);
112+ bool result = !intel_gt_is_wedged(gt);
111113
112114 do {
113− if (i915_gem_wait_for_idle(i915,
115+ if (i915_gem_wait_for_idle(gt->i915,
114116 I915_WAIT_LOCKED |
115117 I915_WAIT_FOR_IDLE_BOOST,
116118 I915_GEM_IDLE_TIMEOUT) == -ETIME) {
117119 /* XXX hide warning from gem_eio */
118120 if (i915_modparams.reset) {
119− dev_err(i915->drm.dev,
121+ dev_err(gt->i915->drm.dev,
120122 "Failed to idle engines, declaring wedged!\n");
121123 GEM_TRACE_DUMP();
122124 }
@@ -125,18 +127,18 @@ static bool switch_to_kernel_context_sync(struct drm_i915_private *i915) · 2 unmodified lines
125127 * Forcibly cancel outstanding work and leave
126128 * the gpu quiet.
127129 */
128− i915_gem_set_wedged(i915);
130+ intel_gt_set_wedged(gt);
129131 result = false;
130132 }
131− } while (i915_retire_requests(i915) && result);
133+ } while (i915_retire_requests(gt->i915) && result);
132134
133− GEM_BUG_ON(i915->gt.awake);
135+ GEM_BUG_ON(gt->awake);
134136 return result;
135137 }
136138
137139 bool i915_gem_load_power_context(struct drm_i915_private *i915)
138140 {
139− return switch_to_kernel_context_sync(i915);
141+ return switch_to_kernel_context_sync(&i915->gt);
140142 }
141143
142144 void i915_gem_suspend(struct drm_i915_private *i915)
@@ -157,7 +159,7 @@ void i915_gem_suspend(struct drm_i915_private *i915) · 14 unmodified lines
157159 * state. Fortunately, the kernel_context is disposable and we do
158160 * not rely on its state.
159161 */
160− switch_to_kernel_context_sync(i915);
162+ switch_to_kernel_context_sync(&i915->gt);
161163
162164 mutex_unlock(&i915->drm.struct_mutex);
163165
@@ -168,11 +170,11 @@ void i915_gem_suspend(struct drm_i915_private *i915) · 4 unmodified lines
168170 GEM_BUG_ON(i915->gt.awake);
169171 flush_work(&i915->gem.idle_work);
170172
171− cancel_delayed_work_sync(&i915->gpu_error.hangcheck_work);
173+ cancel_delayed_work_sync(&i915->gt.hangcheck.work);
172174
173175 i915_gem_drain_freed_objects(i915);
174176
175− intel_uc_suspend(i915);
177+ intel_uc_suspend(&i915->gt.uc);
176178 }
177179
178180 static struct drm_i915_gem_object *first_mm_object(struct list_head *list)
@@ -237,7 +239,6 @@ void i915_gem_suspend_late(struct drm_i915_private *i915) · 58 unmodified lines
237239 }
238240 spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
239241
240− intel_uc_sanitize(i915);
241242 i915_gem_sanitize(i915);
242243 }
243244
@@ -261,10 +262,10 @@ void i915_gem_resume(struct drm_i915_private *i915) · 17 unmodified lines
261262 * guarantee that the context image is complete. So let's just reset
262263 * it and start again.
263264 */
264− if (intel_gt_resume(i915))
265+ if (intel_gt_resume(&i915->gt))
265266 goto err_wedged;
266267
267− intel_uc_resume(i915);
268+ intel_uc_resume(&i915->gt.uc);
268269
269270 /* Always reload a context for powersaving. */
270271 if (!i915_gem_load_power_context(i915))
@@ -276,10 +277,10 @@ out_unlock: · 5 unmodified lines
276277 return;
277278
278279 err_wedged:
279− if (!i915_reset_failed(i915)) {
280+ if (!intel_gt_is_wedged(&i915->gt)) {
280281 dev_err(i915->drm.dev,
281282 "Failed to re-initialize GPU, declaring it wedged!\n");
282− i915_gem_set_wedged(i915);
283+ intel_gt_set_wedged(&i915->gt);
283284 }
284285 goto out_unlock;
285286 }
drivers/gpu/drm/i915/gem/i915_gem_shmem.c+7−0
@@ -414,6 +414,11 @@ shmem_pwrite(struct drm_i915_gem_object *obj, · 413 unmodified lines
414414 return 0;
415415 }
416416
417+static void shmem_release(struct drm_i915_gem_object *obj)
418+{
419+ fput(obj->base.filp);
420+}
421+
417422 const struct drm_i915_gem_object_ops i915_gem_shmem_ops = {
418423 .flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE |
419424 I915_GEM_OBJECT_IS_SHRINKABLE,
@@ -424,6 +429,8 @@ const struct drm_i915_gem_object_ops i915_gem_shmem_ops = { · 4 unmodified lines
424429 .writeback = shmem_writeback,
425430
426431 .pwrite = shmem_pwrite,
432+
433+ .release = shmem_release,
427434 };
428435
429436 static int create_shmem(struct drm_i915_private *i915,
drivers/gpu/drm/i915/gem/i915_gem_shrinker.c+12−15
@@ -88,10 +88,18 @@ static bool can_release_pages(struct drm_i915_gem_object *obj) · 87 unmodified lines
8888 return swap_available() || obj->mm.madv == I915_MADV_DONTNEED;
8989 }
9090
91−static bool unsafe_drop_pages(struct drm_i915_gem_object *obj)
91+static bool unsafe_drop_pages(struct drm_i915_gem_object *obj,
92+ unsigned long shrink)
9293 {
93− if (i915_gem_object_unbind(obj) == 0)
94+ unsigned long flags;
95+
96+ flags = 0;
97+ if (shrink & I915_SHRINK_ACTIVE)
98+ flags = I915_GEM_OBJECT_UNBIND_ACTIVE;
99+
100+ if (i915_gem_object_unbind(obj, flags) == 0)
94101 __i915_gem_object_put_pages(obj, I915_MM_SHRINKER);
102+
95103 return !i915_gem_object_has_pages(obj);
96104 }
97105
@@ -169,7 +177,6 @@ i915_gem_shrink(struct drm_i915_private *i915, · 71 unmodified lines
169177 */
170178
171179 trace_i915_gem_shrink(i915, target, shrink);
172− i915_retire_requests(i915);
173180
174181 /*
175182 * Unbinding of objects will require HW access; Let us not wake the
@@ -230,8 +237,7 @@ i915_gem_shrink(struct drm_i915_private *i915, · 54 unmodified lines
230237 continue;
231238
232239 if (!(shrink & I915_SHRINK_ACTIVE) &&
233− (i915_gem_object_is_active(obj) ||
234− i915_gem_object_is_framebuffer(obj)))
240+ i915_gem_object_is_framebuffer(obj))
235241 continue;
236242
237243 if (!(shrink & I915_SHRINK_BOUND) &&
@@ -246,7 +252,7 @@ i915_gem_shrink(struct drm_i915_private *i915, · 8 unmodified lines
246252
247253 spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
248254
249− if (unsafe_drop_pages(obj)) {
255+ if (unsafe_drop_pages(obj, shrink)) {
250256 /* May arrive from get_pages on another bo */
251257 mutex_lock_nested(&obj->mm.lock,
252258 I915_MM_SHRINKER);
@@ -269,8 +275,6 @@ i915_gem_shrink(struct drm_i915_private *i915, · 16 unmodified lines
269275 if (shrink & I915_SHRINK_BOUND)
270276 intel_runtime_pm_put(&i915->runtime_pm, wakeref);
271277
272− i915_retire_requests(i915);
273−
274278 shrinker_unlock(i915, unlock);
275279
276280 if (nr_scanned)
@@ -427,12 +431,6 @@ i915_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr · 150 unmodified lines
427431 if (!shrinker_lock(i915, 0, &unlock))
428432 return NOTIFY_DONE;
429433
430− /* Force everything onto the inactive lists */
431− if (i915_gem_wait_for_idle(i915,
432− I915_WAIT_LOCKED,
433− MAX_SCHEDULE_TIMEOUT))
434− goto out;
435−
436434 with_intel_runtime_pm(&i915->runtime_pm, wakeref)
437435 freed_pages += i915_gem_shrink(i915, -1UL, NULL,
438436 I915_SHRINK_BOUND |
@@ -455,7 +453,6 @@ i915_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr · 16 unmodified lines
455453 }
456454 mutex_unlock(&i915->ggtt.vm.mutex);
457455
458−out:
459456 shrinker_unlock(i915, unlock);
460457
461458 *(unsigned long *)ptr += freed_pages;
drivers/gpu/drm/i915/gem/i915_gem_stolen.c+0−2
@@ -529,8 +529,6 @@ i915_gem_object_release_stolen(struct drm_i915_gem_object *obj) · 528 unmodified lines
529529
530530 GEM_BUG_ON(!stolen);
531531
532− __i915_gem_object_unpin_pages(obj);
533−
534532 i915_gem_stolen_remove_node(dev_priv, stolen);
535533 kfree(stolen);
536534 }
drivers/gpu/drm/i915/gem/i915_gem_throttle.c+1−1
@@ -41,7 +41,7 @@ i915_gem_throttle_ioctl(struct drm_device *dev, void *data, · 40 unmodified lines
4141 long ret;
4242
4343 /* ABI: return -EIO if already wedged */
44− ret = i915_terminally_wedged(to_i915(dev));
44+ ret = intel_gt_terminally_wedged(&to_i915(dev)->gt);
4545 if (ret)
4646 return ret;
4747
drivers/gpu/drm/i915/gem/i915_gem_userptr.c+10−1
@@ -150,7 +150,8 @@ userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, · 149 unmodified lines
150150 }
151151 }
152152
153− ret = i915_gem_object_unbind(obj);
153+ ret = i915_gem_object_unbind(obj,
154+ I915_GEM_OBJECT_UNBIND_ACTIVE);
154155 if (ret == 0)
155156 ret = __i915_gem_object_put_pages(obj, I915_MM_SHRINKER);
156157 i915_gem_object_put(obj);
@@ -662,6 +663,14 @@ i915_gem_userptr_put_pages(struct drm_i915_gem_object *obj, · 505 unmodified lines
662663 __i915_gem_object_release_shmem(obj, pages, true);
663664 i915_gem_gtt_finish_pages(obj, pages);
664665
666+ /*
667+ * We always mark objects as dirty when they are used by the GPU,
668+ * just in case. However, if we set the vma as being read-only we know
669+ * that the object will never have been written to.
670+ */
671+ if (i915_gem_object_is_readonly(obj))
672+ obj->mm.dirty = false;
673+
665674 for_each_sgt_page(page, sgt_iter, pages) {
666675 if (obj->mm.dirty)
667676 /*
drivers/gpu/drm/i915/gem/selftests/huge_pages.c+48−21
@@ -10,6 +10,8 @@ · 9 unmodified lines
1010
1111 #include "gem/i915_gem_pm.h"
1212
13+#include "gt/intel_gt.h"
14+
1315 #include "igt_gem_utils.h"
1416 #include "mock_context.h"
1517
@@ -926,7 +928,7 @@ gpu_write_dw(struct i915_vma *vma, u64 offset, u32 val) · 910 unmodified lines
926928 }
927929
928930 *cmd = MI_BATCH_BUFFER_END;
929− i915_gem_chipset_flush(i915);
931+ intel_gt_chipset_flush(vma->vm->gt);
930932
931933 i915_gem_object_unpin_map(obj);
932934
@@ -1037,8 +1039,7 @@ static int __igt_write_huge(struct i915_gem_context *ctx, · 104 unmodified lines
10371039 u64 size, u64 offset,
10381040 u32 dword, u32 val)
10391041 {
1040− struct drm_i915_private *i915 = to_i915(obj->base.dev);
1041− struct i915_address_space *vm = ctx->vm ?: &i915->ggtt.vm;
1042+ struct i915_address_space *vm = ctx->vm ?: &engine->gt->ggtt->vm;
10421043 unsigned int flags = PIN_USER | PIN_OFFSET_FIXED;
10431044 struct i915_vma *vma;
10441045 int err;
@@ -1421,6 +1422,9 @@ static int igt_ppgtt_pin_update(void *arg) · 376 unmodified lines
14211422 struct drm_i915_gem_object *obj;
14221423 struct i915_vma *vma;
14231424 unsigned int flags = PIN_USER | PIN_OFFSET_FIXED;
1425+ struct intel_engine_cs *engine;
1426+ enum intel_engine_id id;
1427+ unsigned int n;
14241428 int first, last;
14251429 int err;
14261430
@@ -1518,11 +1522,20 @@ static int igt_ppgtt_pin_update(void *arg) · 91 unmodified lines
15181522 * land in the now stale 2M page.
15191523 */
15201524
1521− err = gpu_write(vma, ctx, dev_priv->engine[RCS0], 0, 0xdeadbeaf);
1522− if (err)
1523− goto out_unpin;
1525+ n = 0;
1526+ for_each_engine(engine, dev_priv, id) {
1527+ if (!intel_engine_can_store_dword(engine))
1528+ continue;
15241529
1525− err = cpu_check(obj, 0, 0xdeadbeaf);
1530+ err = gpu_write(vma, ctx, engine, n++, 0xdeadbeaf);
1531+ if (err)
1532+ goto out_unpin;
1533+ }
1534+ while (n--) {
1535+ err = cpu_check(obj, n, 0xdeadbeaf);
1536+ if (err)
1537+ goto out_unpin;
1538+ }
15261539
15271540 out_unpin:
15281541 i915_vma_unpin(vma);
@@ -1598,8 +1611,11 @@ static int igt_shrink_thp(void *arg) · 69 unmodified lines
15981611 struct drm_i915_private *i915 = ctx->i915;
15991612 struct i915_address_space *vm = ctx->vm ?: &i915->ggtt.vm;
16001613 struct drm_i915_gem_object *obj;
1614+ struct intel_engine_cs *engine;
1615+ enum intel_engine_id id;
16011616 struct i915_vma *vma;
16021617 unsigned int flags = PIN_USER;
1618+ unsigned int n;
16031619 int err;
16041620
16051621 /*
@@ -1635,9 +1651,15 @@ static int igt_shrink_thp(void *arg) · 29 unmodified lines
16351651 if (err)
16361652 goto out_unpin;
16371653
1638− err = gpu_write(vma, ctx, i915->engine[RCS0], 0, 0xdeadbeaf);
1639− if (err)
1640− goto out_unpin;
1654+ n = 0;
1655+ for_each_engine(engine, i915, id) {
1656+ if (!intel_engine_can_store_dword(engine))
1657+ continue;
1658+
1659+ err = gpu_write(vma, ctx, engine, n++, 0xdeadbeaf);
1660+ if (err)
1661+ goto out_unpin;
1662+ }
16411663
16421664 i915_vma_unpin(vma);
16431665
@@ -1662,7 +1684,12 @@ static int igt_shrink_thp(void *arg) · 18 unmodified lines
16621684 if (err)
16631685 goto out_close;
16641686
1665− err = cpu_check(obj, 0, 0xdeadbeaf);
1687+ while (n--) {
1688+ err = cpu_check(obj, n, 0xdeadbeaf);
1689+ if (err)
1690+ goto out_unpin;
1691+ }
1692+
16661693
16671694 out_unpin:
16681695 i915_vma_unpin(vma);
@@ -1726,7 +1753,7 @@ out_unlock: · 57 unmodified lines
17261753 return err;
17271754 }
17281755
1729−int i915_gem_huge_page_live_selftests(struct drm_i915_private *dev_priv)
1756+int i915_gem_huge_page_live_selftests(struct drm_i915_private *i915)
17301757 {
17311758 static const struct i915_subtest tests[] = {
17321759 SUBTEST(igt_shrink_thp),
@@ -1741,22 +1768,22 @@ int i915_gem_huge_page_live_selftests(struct drm_i915_private *dev_priv) · 8 unmodified lines
17411768 intel_wakeref_t wakeref;
17421769 int err;
17431770
1744− if (!HAS_PPGTT(dev_priv)) {
1771+ if (!HAS_PPGTT(i915)) {
17451772 pr_info("PPGTT not supported, skipping live-selftests\n");
17461773 return 0;
17471774 }
17481775
1749− if (i915_terminally_wedged(dev_priv))
1776+ if (intel_gt_is_wedged(&i915->gt))
17501777 return 0;
17511778
1752− file = mock_file(dev_priv);
1779+ file = mock_file(i915);
17531780 if (IS_ERR(file))
17541781 return PTR_ERR(file);
17551782
1756− mutex_lock(&dev_priv->drm.struct_mutex);
1757− wakeref = intel_runtime_pm_get(&dev_priv->runtime_pm);
1783+ mutex_lock(&i915->drm.struct_mutex);
1784+ wakeref = intel_runtime_pm_get(&i915->runtime_pm);
17581785
1759− ctx = live_context(dev_priv, file);
1786+ ctx = live_context(i915, file);
17601787 if (IS_ERR(ctx)) {
17611788 err = PTR_ERR(ctx);
17621789 goto out_unlock;
@@ -1768,10 +1795,10 @@ int i915_gem_huge_page_live_selftests(struct drm_i915_private *dev_priv) · 5 unmodified lines
17681795 err = i915_subtests(tests, ctx);
17691796
17701797 out_unlock:
1771− intel_runtime_pm_put(&dev_priv->runtime_pm, wakeref);
1772− mutex_unlock(&dev_priv->drm.struct_mutex);
1798+ intel_runtime_pm_put(&i915->runtime_pm, wakeref);
1799+ mutex_unlock(&i915->drm.struct_mutex);
17731800
1774− mock_file_free(dev_priv, file);
1801+ mock_file_free(i915, file);
17751802
17761803 return err;
17771804 }
drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c+6−20
@@ -5,14 +5,16 @@ · 4 unmodified lines
55
66 #include "i915_selftest.h"
77
8+#include "gt/intel_gt.h"
9+
810 #include "selftests/igt_flush_test.h"
911 #include "selftests/mock_drm.h"
1012 #include "mock_context.h"
1113
1214 static int igt_client_fill(void *arg)
1315 {
14− struct intel_context *ce = arg;
15− struct drm_i915_private *i915 = ce->gem_context->i915;
16+ struct drm_i915_private *i915 = arg;
17+ struct intel_context *ce = i915->engine[BCS0]->kernel_context;
1618 struct drm_i915_gem_object *obj;
1719 struct rnd_state prng;
1820 IGT_TIMEOUT(end);
@@ -63,17 +65,6 @@ static int igt_client_fill(void *arg) · 44 unmodified lines
6365 if (err)
6466 goto err_unpin;
6567
66− /*
67− * XXX: For now do the wait without the object resv lock to
68− * ensure we don't deadlock.
69− */
70− err = i915_gem_object_wait(obj,
71− I915_WAIT_INTERRUPTIBLE |
72− I915_WAIT_ALL,
73− MAX_SCHEDULE_TIMEOUT);
74− if (err)
75− goto err_unpin;
76−
7768 i915_gem_object_lock(obj);
7869 err = i915_gem_object_set_to_cpu_domain(obj, false);
7970 i915_gem_object_unlock(obj);
@@ -100,11 +91,6 @@ err_unpin: · 20 unmodified lines
10091 err_put:
10192 i915_gem_object_put(obj);
10293 err_flush:
103− mutex_lock(&i915->drm.struct_mutex);
104− if (igt_flush_test(i915, I915_WAIT_LOCKED))
105− err = -EIO;
106− mutex_unlock(&i915->drm.struct_mutex);
107−
10894 if (err == -ENOMEM)
10995 err = 0;
11096
@@ -117,11 +103,11 @@ int i915_gem_client_blt_live_selftests(struct drm_i915_private *i915) · 6 unmodified lines
117103 SUBTEST(igt_client_fill),
118104 };
119105
120− if (i915_terminally_wedged(i915))
106+ if (intel_gt_is_wedged(&i915->gt))
121107 return 0;
122108
123109 if (!HAS_ENGINE(i915, BCS0))
124110 return 0;
125111
126− return i915_subtests(tests, i915->engine[BCS0]->kernel_context);
112+ return i915_live_subtests(tests, i915);
127113 }
drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c+7−2
@@ -6,6 +6,8 @@ · 5 unmodified lines
66
77 #include <linux/prime_numbers.h>
88
9+#include "gt/intel_gt.h"
10+
911 #include "i915_selftest.h"
1012 #include "selftests/i915_random.h"
1113
@@ -242,12 +244,15 @@ static bool always_valid(struct drm_i915_private *i915) · 230 unmodified lines
242244
243245 static bool needs_fence_registers(struct drm_i915_private *i915)
244246 {
245− return !i915_terminally_wedged(i915);
247+ return !intel_gt_is_wedged(&i915->gt);
246248 }
247249
248250 static bool needs_mi_store_dword(struct drm_i915_private *i915)
249251 {
250− if (i915_terminally_wedged(i915))
252+ if (intel_gt_is_wedged(&i915->gt))
253+ return false;
254+
255+ if (!HAS_ENGINE(i915, RCS0))
251256 return false;
252257
253258 return intel_engine_can_store_dword(i915->engine[RCS0]);
drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c+36−49
@@ -7,6 +7,7 @@ · 6 unmodified lines
77 #include <linux/prime_numbers.h>
88
99 #include "gem/i915_gem_pm.h"
10+#include "gt/intel_gt.h"
1011 #include "gt/intel_reset.h"
1112 #include "i915_selftest.h"
1213
@@ -31,7 +32,6 @@ static int live_nop_switch(void *arg) · 18 unmodified lines
3132 struct intel_engine_cs *engine;
3233 struct i915_gem_context **ctx;
3334 enum intel_engine_id id;
34− intel_wakeref_t wakeref;
3535 struct igt_live_test t;
3636 struct drm_file *file;
3737 unsigned long n;
@@ -53,7 +53,6 @@ static int live_nop_switch(void *arg) · 15 unmodified lines
5353 return PTR_ERR(file);
5454
5555 mutex_lock(&i915->drm.struct_mutex);
56− wakeref = intel_runtime_pm_get(&i915->runtime_pm);
5756
5857 ctx = kcalloc(nctx, sizeof(*ctx), GFP_KERNEL);
5958 if (!ctx) {
@@ -85,7 +84,7 @@ static int live_nop_switch(void *arg) · 25 unmodified lines
8584 }
8685 if (i915_request_wait(rq, 0, HZ / 5) < 0) {
8786 pr_err("Failed to populated %d contexts\n", nctx);
88− i915_gem_set_wedged(i915);
87+ intel_gt_set_wedged(&i915->gt);
8988 err = -EIO;
9089 goto out_unlock;
9190 }
@@ -129,7 +128,7 @@ static int live_nop_switch(void *arg) · 37 unmodified lines
129128 if (i915_request_wait(rq, 0, HZ / 5) < 0) {
130129 pr_err("Switching between %ld contexts timed out\n",
131130 prime);
132− i915_gem_set_wedged(i915);
131+ intel_gt_set_wedged(&i915->gt);
133132 break;
134133 }
135134
@@ -152,7 +151,6 @@ static int live_nop_switch(void *arg) · 16 unmodified lines
152151 }
153152
154153 out_unlock:
155− intel_runtime_pm_put(&i915->runtime_pm, wakeref);
156154 mutex_unlock(&i915->drm.struct_mutex);
157155 mock_file_free(i915, file);
158156 return err;
@@ -237,8 +235,7 @@ static int gpu_fill(struct drm_i915_gem_object *obj, · 78 unmodified lines
237235 struct intel_engine_cs *engine,
238236 unsigned int dw)
239237 {
240− struct drm_i915_private *i915 = to_i915(obj->base.dev);
241− struct i915_address_space *vm = ctx->vm ?: &i915->ggtt.vm;
238+ struct i915_address_space *vm = ctx->vm ?: &engine->gt->ggtt->vm;
242239 struct i915_request *rq;
243240 struct i915_vma *vma;
244241 struct i915_vma *batch;
@@ -431,6 +428,9 @@ create_test_object(struct i915_gem_context *ctx, · 186 unmodified lines
431428 u64 size;
432429 int err;
433430
431+ /* Keep in GEM's good graces */
432+ i915_retire_requests(ctx->i915);
433+
434434 size = min(vm->total / 2, 1024ull * DW_PER_PAGE * PAGE_SIZE);
435435 size = round_down(size, DW_PER_PAGE * PAGE_SIZE);
436436
@@ -507,7 +507,6 @@ static int igt_ctx_exec(void *arg) · 70 unmodified lines
507507 dw = 0;
508508 while (!time_after(jiffies, end_time)) {
509509 struct i915_gem_context *ctx;
510− intel_wakeref_t wakeref;
511510
512511 ctx = live_context(i915, file);
513512 if (IS_ERR(ctx)) {
@@ -523,8 +522,7 @@ static int igt_ctx_exec(void *arg) · 9 unmodified lines
523522 }
524523 }
525524
526− with_intel_runtime_pm(&i915->runtime_pm, wakeref)
527− err = gpu_fill(obj, ctx, engine, dw);
525+ err = gpu_fill(obj, ctx, engine, dw);
528526 if (err) {
529527 pr_err("Failed to fill dword %lu [%lu/%lu] with gpu (%s) in ctx %u [full-ppgtt? %s], err=%d\n",
530528 ndwords, dw, max_dwords(obj),
@@ -565,6 +563,8 @@ out_unlock: · 34 unmodified lines
565563 mock_file_free(i915, file);
566564 if (err)
567565 return err;
566+
567+ i915_gem_drain_freed_objects(i915);
568568 }
569569
570570 return 0;
@@ -623,7 +623,6 @@ static int igt_shared_ctx_exec(void *arg) · 52 unmodified lines
623623 ncontexts = 0;
624624 while (!time_after(jiffies, end_time)) {
625625 struct i915_gem_context *ctx;
626− intel_wakeref_t wakeref;
627626
628627 ctx = kernel_context(i915);
629628 if (IS_ERR(ctx)) {
@@ -642,9 +641,7 @@ static int igt_shared_ctx_exec(void *arg) · 12 unmodified lines
642641 }
643642 }
644643
645− err = 0;
646− with_intel_runtime_pm(&i915->runtime_pm, wakeref)
647− err = gpu_fill(obj, ctx, engine, dw);
644+ err = gpu_fill(obj, ctx, engine, dw);
648645 if (err) {
649646 pr_err("Failed to fill dword %lu [%lu/%lu] with gpu (%s) in ctx %u [full-ppgtt? %s], err=%d\n",
650647 ndwords, dw, max_dwords(obj),
@@ -678,6 +675,10 @@ static int igt_shared_ctx_exec(void *arg) · 27 unmodified lines
678675
679676 dw += rem;
680677 }
678+
679+ mutex_unlock(&i915->drm.struct_mutex);
680+ i915_gem_drain_freed_objects(i915);
681+ mutex_lock(&i915->drm.struct_mutex);
681682 }
682683 out_test:
683684 if (igt_live_test_end(&t))
@@ -746,7 +747,7 @@ emit_rpcs_query(struct drm_i915_gem_object *obj, · 62 unmodified lines
746747
747748 GEM_BUG_ON(!intel_engine_can_store_dword(ce->engine));
748749
749− vma = i915_vma_instance(obj, ce->gem_context->vm, NULL);
750+ vma = i915_vma_instance(obj, ce->vm, NULL);
750751 if (IS_ERR(vma))
751752 return PTR_ERR(vma);
752753
@@ -956,7 +957,7 @@ __sseu_finish(struct drm_i915_private *i915, · 203 unmodified lines
956957 int ret = 0;
957958
958959 if (flags & TEST_RESET) {
959− ret = i915_reset_engine(ce->engine, "sseu");
960+ ret = intel_engine_reset(ce->engine, "sseu");
960961 if (ret)
961962 goto out;
962963 }
@@ -1025,35 +1026,33 @@ __igt_ctx_sseu(struct drm_i915_private *i915, · 62 unmodified lines
10251026 unsigned int flags)
10261027 {
10271028 struct intel_engine_cs *engine = i915->engine[RCS0];
1028− struct intel_sseu default_sseu = engine->sseu;
10291029 struct drm_i915_gem_object *obj;
10301030 struct i915_gem_context *ctx;
10311031 struct intel_context *ce;
10321032 struct intel_sseu pg_sseu;
1033− intel_wakeref_t wakeref;
10341033 struct drm_file *file;
10351034 int ret;
10361035
1037− if (INTEL_GEN(i915) < 9)
1036+ if (INTEL_GEN(i915) < 9 || !engine)
10381037 return 0;
10391038
10401039 if (!RUNTIME_INFO(i915)->sseu.has_slice_pg)
10411040 return 0;
10421041
1043− if (hweight32(default_sseu.slice_mask) < 2)
1042+ if (hweight32(engine->sseu.slice_mask) < 2)
10441043 return 0;
10451044
10461045 /*
10471046 * Gen11 VME friendly power-gated configuration with half enabled
10481047 * sub-slices.
10491048 */
1050− pg_sseu = default_sseu;
1049+ pg_sseu = engine->sseu;
10511050 pg_sseu.slice_mask = 1;
10521051 pg_sseu.subslice_mask =
1053− ~(~0 << (hweight32(default_sseu.subslice_mask) / 2));
1052+ ~(~0 << (hweight32(engine->sseu.subslice_mask) / 2));
10541053
10551054 pr_info("SSEU subtest '%s', flags=%x, def_slices=%u, pg_slices=%u\n",
1056− name, flags, hweight32(default_sseu.slice_mask),
1055+ name, flags, hweight32(engine->sseu.slice_mask),
10571056 hweight32(pg_sseu.slice_mask));
10581057
10591058 file = mock_file(i915);
@@ -1061,7 +1060,7 @@ __igt_ctx_sseu(struct drm_i915_private *i915, · 1 unmodified line
10611060 return PTR_ERR(file);
10621061
10631062 if (flags & TEST_RESET)
1064− igt_global_reset_lock(i915);
1063+ igt_global_reset_lock(&i915->gt);
10651064
10661065 mutex_lock(&i915->drm.struct_mutex);
10671066
@@ -1078,12 +1077,10 @@ __igt_ctx_sseu(struct drm_i915_private *i915, · 10 unmodified lines
10781077 goto out_unlock;
10791078 }
10801079
1081− wakeref = intel_runtime_pm_get(&i915->runtime_pm);
1082−
10831080 ce = i915_gem_context_get_engine(ctx, RCS0);
10841081 if (IS_ERR(ce)) {
10851082 ret = PTR_ERR(ce);
1086− goto out_rpm;
1083+ goto out_put;
10871084 }
10881085
10891086 ret = intel_context_pin(ce);
@@ -1091,7 +1088,7 @@ __igt_ctx_sseu(struct drm_i915_private *i915, · 1 unmodified line
10911088 goto out_context;
10921089
10931090 /* First set the default mask. */
1094− ret = __sseu_test(i915, name, flags, ce, obj, default_sseu);
1091+ ret = __sseu_test(i915, name, flags, ce, obj, engine->sseu);
10951092 if (ret)
10961093 goto out_fail;
10971094
@@ -1101,7 +1098,7 @@ __igt_ctx_sseu(struct drm_i915_private *i915, · 3 unmodified lines
11011098 goto out_fail;
11021099
11031100 /* Back to defaults. */
1104− ret = __sseu_test(i915, name, flags, ce, obj, default_sseu);
1101+ ret = __sseu_test(i915, name, flags, ce, obj, engine->sseu);
11051102 if (ret)
11061103 goto out_fail;
11071104
@@ -1117,15 +1114,14 @@ out_fail: · 9 unmodified lines
11171114 intel_context_unpin(ce);
11181115 out_context:
11191116 intel_context_put(ce);
1120−out_rpm:
1121− intel_runtime_pm_put(&i915->runtime_pm, wakeref);
1117+out_put:
11221118 i915_gem_object_put(obj);
11231119
11241120 out_unlock:
11251121 mutex_unlock(&i915->drm.struct_mutex);
11261122
11271123 if (flags & TEST_RESET)
1128− igt_global_reset_unlock(i915);
1124+ igt_global_reset_unlock(&i915->gt);
11291125
11301126 mock_file_free(i915, file);
11311127
@@ -1194,7 +1190,7 @@ static int igt_ctx_readonly(void *arg) · 62 unmodified lines
11941190 goto out_unlock;
11951191 }
11961192
1197− vm = ctx->vm ?: &i915->mm.aliasing_ppgtt->vm;
1193+ vm = ctx->vm ?: &i915->ggtt.alias->vm;
11981194 if (!vm || !vm->has_read_only) {
11991195 err = 0;
12001196 goto out_unlock;
@@ -1207,8 +1203,6 @@ static int igt_ctx_readonly(void *arg) · 6 unmodified lines
12071203 unsigned int id;
12081204
12091205 for_each_engine(engine, i915, id) {
1210− intel_wakeref_t wakeref;
1211−
12121206 if (!intel_engine_can_store_dword(engine))
12131207 continue;
12141208
@@ -1223,9 +1217,7 @@ static int igt_ctx_readonly(void *arg) · 8 unmodified lines
12231217 i915_gem_object_set_readonly(obj);
12241218 }
12251219
1226− err = 0;
1227− with_intel_runtime_pm(&i915->runtime_pm, wakeref)
1228− err = gpu_fill(obj, ctx, engine, dw);
1220+ err = gpu_fill(obj, ctx, engine, dw);
12291221 if (err) {
12301222 pr_err("Failed to fill dword %lu [%lu/%lu] with gpu (%s) in ctx %u [full-ppgtt? %s], err=%d\n",
12311223 ndwords, dw, max_dwords(obj),
@@ -1488,7 +1480,6 @@ static int igt_vm_isolation(void *arg) · 256 unmodified lines
14881480 struct drm_i915_private *i915 = arg;
14891481 struct i915_gem_context *ctx_a, *ctx_b;
14901482 struct intel_engine_cs *engine;
1491− intel_wakeref_t wakeref;
14921483 struct igt_live_test t;
14931484 struct drm_file *file;
14941485 I915_RND_STATE(prng);
@@ -1535,8 +1526,6 @@ static int igt_vm_isolation(void *arg) · 40 unmodified lines
15351526 GEM_BUG_ON(ctx_b->vm->total != vm_total);
15361527 vm_total -= I915_GTT_PAGE_SIZE;
15371528
1538− wakeref = intel_runtime_pm_get(&i915->runtime_pm);
1539−
15401529 count = 0;
15411530 for_each_engine(engine, i915, id) {
15421531 IGT_TIMEOUT(end_time);
@@ -1551,7 +1540,7 @@ static int igt_vm_isolation(void *arg) · 8 unmodified lines
15511540
15521541 div64_u64_rem(i915_prandom_u64_state(&prng),
15531542 vm_total, &offset);
1554− offset &= -sizeof(u32);
1543+ offset = round_down(offset, alignof_dword);
15551544 offset += I915_GTT_PAGE_SIZE;
15561545
15571546 err = write_to_scratch(ctx_a, engine,
@@ -1560,7 +1549,7 @@ static int igt_vm_isolation(void *arg) · 2 unmodified lines
15601549 err = read_from_scratch(ctx_b, engine,
15611550 offset, &value);
15621551 if (err)
1563− goto out_rpm;
1552+ goto out_unlock;
15641553
15651554 if (value) {
15661555 pr_err("%s: Read %08x from scratch (offset 0x%08x_%08x), after %lu reads!\n",
@@ -1569,7 +1558,7 @@ static int igt_vm_isolation(void *arg) · 2 unmodified lines
15691558 lower_32_bits(offset),
15701559 this);
15711560 err = -EINVAL;
1572− goto out_rpm;
1561+ goto out_unlock;
15731562 }
15741563
15751564 this++;
@@ -1579,8 +1568,6 @@ static int igt_vm_isolation(void *arg) · 3 unmodified lines
15791568 pr_info("Checked %lu scratch offsets across %d engines\n",
15801569 count, RUNTIME_INFO(i915)->num_engines);
15811570
1582−out_rpm:
1583− intel_runtime_pm_put(&i915->runtime_pm, wakeref);
15841571 out_unlock:
15851572 if (igt_live_test_end(&t))
15861573 err = -EIO;
@@ -1736,7 +1723,7 @@ int i915_gem_context_mock_selftests(void) · 149 unmodified lines
17361723 return err;
17371724 }
17381725
1739−int i915_gem_context_live_selftests(struct drm_i915_private *dev_priv)
1726+int i915_gem_context_live_selftests(struct drm_i915_private *i915)
17401727 {
17411728 static const struct i915_subtest tests[] = {
17421729 SUBTEST(live_nop_switch),
@@ -1747,8 +1734,8 @@ int i915_gem_context_live_selftests(struct drm_i915_private *dev_priv) · 4 unmodified lines
17471734 SUBTEST(igt_vm_isolation),
17481735 };
17491736
1750− if (i915_terminally_wedged(dev_priv))
1737+ if (intel_gt_is_wedged(&i915->gt))
17511738 return 0;
17521739
1753− return i915_subtests(tests, dev_priv);
1740+ return i915_live_subtests(tests, i915);
17541741 }
drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c+36−23
@@ -6,6 +6,7 @@ · 5 unmodified lines
66
77 #include <linux/prime_numbers.h>
88
9+#include "gt/intel_gt.h"
910 #include "gt/intel_gt_pm.h"
1011 #include "huge_gem_object.h"
1112 #include "i915_selftest.h"
@@ -143,7 +144,7 @@ static int check_partial_mapping(struct drm_i915_gem_object *obj, · 131 unmodified lines
143144 if (offset >= obj->base.size)
144145 continue;
145146
146− i915_gem_flush_ggtt_writes(to_i915(obj->base.dev));
147+ intel_gt_flush_ggtt_writes(&to_i915(obj->base.dev)->gt);
147148
148149 p = i915_gem_object_get_page(obj, offset >> PAGE_SHIFT);
149150 cpu = kmap(p) + offset_in_page(offset);
@@ -327,7 +328,8 @@ out: · 177 unmodified lines
327328 static int make_obj_busy(struct drm_i915_gem_object *obj)
328329 {
329330 struct drm_i915_private *i915 = to_i915(obj->base.dev);
330− struct i915_request *rq;
331+ struct intel_engine_cs *engine;
332+ enum intel_engine_id id;
331333 struct i915_vma *vma;
332334 int err;
333335
@@ -339,17 +341,21 @@ static int make_obj_busy(struct drm_i915_gem_object *obj) · 5 unmodified lines
339341 if (err)
340342 return err;
341343
342− rq = i915_request_create(i915->engine[RCS0]->kernel_context);
343− if (IS_ERR(rq)) {
344− i915_vma_unpin(vma);
345− return PTR_ERR(rq);
346− }
344+ for_each_engine(engine, i915, id) {
345+ struct i915_request *rq;
346+
347+ rq = i915_request_create(engine->kernel_context);
348+ if (IS_ERR(rq)) {
349+ i915_vma_unpin(vma);
350+ return PTR_ERR(rq);
351+ }
347352
348− i915_vma_lock(vma);
349− err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
350− i915_vma_unlock(vma);
353+ i915_vma_lock(vma);
354+ err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
355+ i915_vma_unlock(vma);
351356
352− i915_request_add(rq);
357+ i915_request_add(rq);
358+ }
353359
354360 i915_vma_unpin(vma);
355361 i915_gem_object_put(obj); /* leave it only alive via its active ref */
@@ -378,7 +384,7 @@ static void disable_retire_worker(struct drm_i915_private *i915) · 22 unmodified lines
378384 {
379385 i915_gem_shrinker_unregister(i915);
380386
381− intel_gt_pm_get(i915);
387+ intel_gt_pm_get(&i915->gt);
382388
383389 cancel_delayed_work_sync(&i915->gem.retire_work);
384390 flush_work(&i915->gem.idle_work);
@@ -386,7 +392,7 @@ static void disable_retire_worker(struct drm_i915_private *i915) · 1 unmodified line
386392
387393 static void restore_retire_worker(struct drm_i915_private *i915)
388394 {
389− intel_gt_pm_put(i915);
395+ intel_gt_pm_put(&i915->gt);
390396
391397 mutex_lock(&i915->drm.struct_mutex);
392398 igt_flush_test(i915, I915_WAIT_LOCKED);
@@ -395,6 +401,18 @@ static void restore_retire_worker(struct drm_i915_private *i915) · 2 unmodified lines
395401 i915_gem_shrinker_register(i915);
396402 }
397403
404+static void mmap_offset_lock(struct drm_i915_private *i915)
405+ __acquires(&i915->drm.vma_offset_manager->vm_lock)
406+{
407+ write_lock(&i915->drm.vma_offset_manager->vm_lock);
408+}
409+
410+static void mmap_offset_unlock(struct drm_i915_private *i915)
411+ __releases(&i915->drm.vma_offset_manager->vm_lock)
412+{
413+ write_unlock(&i915->drm.vma_offset_manager->vm_lock);
414+}
415+
398416 static int igt_mmap_offset_exhaustion(void *arg)
399417 {
400418 struct drm_i915_private *i915 = arg;
@@ -413,7 +431,9 @@ static int igt_mmap_offset_exhaustion(void *arg) · 12 unmodified lines
413431 drm_mm_for_each_hole(hole, mm, hole_start, hole_end) {
414432 resv.start = hole_start;
415433 resv.size = hole_end - hole_start - 1; /* PAGE_SIZE units */
434+ mmap_offset_lock(i915);
416435 err = drm_mm_reserve_node(mm, &resv);
436+ mmap_offset_unlock(i915);
417437 if (err) {
418438 pr_err("Failed to trim VMA manager, err=%d\n", err);
419439 goto out_park;
@@ -458,7 +478,7 @@ static int igt_mmap_offset_exhaustion(void *arg) · 38 unmodified lines
458478
459479 /* Now fill with busy dead objects that we expect to reap */
460480 for (loop = 0; loop < 3; loop++) {
461− if (i915_terminally_wedged(i915))
481+ if (intel_gt_is_wedged(&i915->gt))
462482 break;
463483
464484 obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
@@ -474,19 +494,12 @@ static int igt_mmap_offset_exhaustion(void *arg) · 9 unmodified lines
474494 pr_err("[loop %d] Failed to busy the object\n", loop);
475495 goto err_obj;
476496 }
477−
478− /* NB we rely on the _active_ reference to access obj now */
479− GEM_BUG_ON(!i915_gem_object_is_active(obj));
480− err = create_mmap_offset(obj);
481− if (err) {
482− pr_err("[loop %d] create_mmap_offset failed with err=%d\n",
483− loop, err);
484− goto out;
485− }
486497 }
487498
488499 out:
500+ mmap_offset_lock(i915);
489501 drm_mm_remove_node(&resv);
502+ mmap_offset_unlock(i915);
490503 out_park:
491504 restore_retire_worker(i915);
492505 return err;
drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c+6−9
@@ -3,6 +3,8 @@ · 2 unmodified lines
33 * Copyright © 2019 Intel Corporation
44 */
55
6+#include "gt/intel_gt.h"
7+
68 #include "i915_selftest.h"
79
810 #include "selftests/igt_flush_test.h"
@@ -11,8 +13,8 @@ · 2 unmodified lines
1113
1214 static int igt_fill_blt(void *arg)
1315 {
14− struct intel_context *ce = arg;
15− struct drm_i915_private *i915 = ce->gem_context->i915;
16+ struct drm_i915_private *i915 = arg;
17+ struct intel_context *ce = i915->engine[BCS0]->kernel_context;
1618 struct drm_i915_gem_object *obj;
1719 struct rnd_state prng;
1820 IGT_TIMEOUT(end);
@@ -83,11 +85,6 @@ err_unpin: · 64 unmodified lines
8385 err_put:
8486 i915_gem_object_put(obj);
8587 err_flush:
86− mutex_lock(&i915->drm.struct_mutex);
87− if (igt_flush_test(i915, I915_WAIT_LOCKED))
88− err = -EIO;
89− mutex_unlock(&i915->drm.struct_mutex);
90−
9188 if (err == -ENOMEM)
9289 err = 0;
9390
@@ -100,11 +97,11 @@ int i915_gem_object_blt_live_selftests(struct drm_i915_private *i915) · 6 unmodified lines
10097 SUBTEST(igt_fill_blt),
10198 };
10299
103− if (i915_terminally_wedged(i915))
100+ if (intel_gt_is_wedged(&i915->gt))
104101 return 0;
105102
106103 if (!HAS_ENGINE(i915, BCS0))
107104 return 0;
108105
109− return i915_subtests(tests, i915->engine[BCS0]->kernel_context);
106+ return i915_live_subtests(tests, i915);
110107 }
drivers/gpu/drm/i915/gt/Makefile+4−1
@@ -1,2 +1,5 @@
1+# For building individual subdir files on the command line
2+subdir-ccflags-y += -I$(srctree)/$(src)/..
3+
14 # Extra header tests
2−include $(src)/Makefile.header-test
5+header-test-pattern-$(CONFIG_DRM_I915_WERROR) := *.h
drivers/gpu/drm/i915/gt/Makefile.header-test+0−16
@@ -1,16 +0,0 @@
1−# SPDX-License-Identifier: MIT
2−# Copyright © 2019 Intel Corporation
3−
4−# Test the headers are compilable as standalone units
5−header_test := $(notdir $(wildcard $(src)/*.h))
6−
7−quiet_cmd_header_test = HDRTEST $@
8− cmd_header_test = echo "\#include \"$(<F)\"" > $@
9−
10−header_test_%.c: %.h
11− $(call cmd,header_test)
12−
13−extra-$(CONFIG_DRM_I915_WERROR) += \
14− $(foreach h,$(header_test),$(patsubst %.h,header_test_%.o,$(h)))
15−
16−clean-files += $(foreach h,$(header_test),$(patsubst %.h,header_test_%.c,$(h)))
drivers/gpu/drm/i915/gt/gen6_renderstate.crenamed from drivers/gpu/drm/i915/intel_renderstate_gen6.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/gt/gen7_renderstate.crenamed from drivers/gpu/drm/i915/intel_renderstate_gen7.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/gt/gen8_renderstate.crenamed from drivers/gpu/drm/i915/intel_renderstate_gen8.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/gt/gen9_renderstate.crenamed from drivers/gpu/drm/i915/intel_renderstate_gen9.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/gt/intel_context.c+88−35
@@ -59,6 +59,10 @@ int __intel_context_do_pin(struct intel_context *ce) · 58 unmodified lines
5959 if (err)
6060 goto err;
6161
62+ GEM_TRACE("%s context:%llx pin ring:{head:%04x, tail:%04x}\n",
63+ ce->engine->name, ce->ring->timeline->fence_context,
64+ ce->ring->head, ce->ring->tail);
65+
6266 i915_gem_context_get(ce->gem_context); /* for ctx->ppgtt */
6367
6468 smp_mb__before_atomic(); /* flush pin before it is visible */
@@ -85,6 +89,9 @@ void intel_context_unpin(struct intel_context *ce) · 20 unmodified lines
8589 mutex_lock_nested(&ce->pin_mutex, SINGLE_DEPTH_NESTING);
8690
8791 if (likely(atomic_dec_and_test(&ce->pin_count))) {
92+ GEM_TRACE("%s context:%llx retire\n",
93+ ce->engine->name, ce->ring->timeline->fence_context);
94+
8895 ce->ops->unpin(ce);
8996
9097 i915_gem_context_put(ce->gem_context);
@@ -95,11 +102,15 @@ void intel_context_unpin(struct intel_context *ce) · 4 unmodified lines
95102 intel_context_put(ce);
96103 }
97104
98−static int __context_pin_state(struct i915_vma *vma, unsigned long flags)
105+static int __context_pin_state(struct i915_vma *vma)
99106 {
107+ u64 flags;
100108 int err;
101109
102− err = i915_vma_pin(vma, 0, 0, flags | PIN_GLOBAL);
110+ flags = i915_ggtt_pin_bias(vma) | PIN_OFFSET_BIAS;
111+ flags |= PIN_HIGH | PIN_GLOBAL;
112+
113+ err = i915_vma_pin(vma, 0, 0, flags);
103114 if (err)
104115 return err;
105116
@@ -119,10 +130,13 @@ static void __context_unpin_state(struct i915_vma *vma) · 13 unmodified lines
119130 __i915_vma_unpin(vma);
120131 }
121132
122−static void intel_context_retire(struct i915_active *active)
133+static void __intel_context_retire(struct i915_active *active)
123134 {
124135 struct intel_context *ce = container_of(active, typeof(*ce), active);
125136
137+ GEM_TRACE("%s context:%llx retire\n",
138+ ce->engine->name, ce->ring->timeline->fence_context);
139+
126140 if (ce->state)
127141 __context_unpin_state(ce->state);
128142
@@ -130,35 +144,11 @@ static void intel_context_retire(struct i915_active *active) · 1 unmodified line
130144 intel_context_put(ce);
131145 }
132146
133−void
134−intel_context_init(struct intel_context *ce,
135− struct i915_gem_context *ctx,
136− struct intel_engine_cs *engine)
137−{
138− GEM_BUG_ON(!engine->cops);
139−
140− kref_init(&ce->ref);
141−
142− ce->gem_context = ctx;
143− ce->engine = engine;
144− ce->ops = engine->cops;
145− ce->sseu = engine->sseu;
146−
147− INIT_LIST_HEAD(&ce->signal_link);
148− INIT_LIST_HEAD(&ce->signals);
149−
150− mutex_init(&ce->pin_mutex);
151−
152− i915_active_init(ctx->i915, &ce->active, intel_context_retire);
153−}
154−
155−int intel_context_active_acquire(struct intel_context *ce, unsigned long flags)
147+static int __intel_context_active(struct i915_active *active)
156148 {
149+ struct intel_context *ce = container_of(active, typeof(*ce), active);
157150 int err;
158151
159− if (!i915_active_acquire(&ce->active))
160− return 0;
161−
162152 intel_context_get(ce);
163153
164154 err = intel_ring_pin(ce->ring);
@@ -168,7 +158,7 @@ int intel_context_active_acquire(struct intel_context *ce, unsigned long flags) · 3 unmodified lines
168158 if (!ce->state)
169159 return 0;
170160
171− err = __context_pin_state(ce->state, flags);
161+ err = __context_pin_state(ce->state);
172162 if (err)
173163 goto err_ring;
174164
@@ -188,15 +178,40 @@ err_ring: · 13 unmodified lines
188178 intel_ring_unpin(ce->ring);
189179 err_put:
190180 intel_context_put(ce);
191− i915_active_cancel(&ce->active);
192181 return err;
193182 }
194183
195−void intel_context_active_release(struct intel_context *ce)
184+void
185+intel_context_init(struct intel_context *ce,
186+ struct i915_gem_context *ctx,
187+ struct intel_engine_cs *engine)
188+{
189+ GEM_BUG_ON(!engine->cops);
190+
191+ kref_init(&ce->ref);
192+
193+ ce->gem_context = ctx;
194+ ce->vm = i915_vm_get(ctx->vm ?: &engine->gt->ggtt->vm);
195+
196+ ce->engine = engine;
197+ ce->ops = engine->cops;
198+ ce->sseu = engine->sseu;
199+
200+ INIT_LIST_HEAD(&ce->signal_link);
201+ INIT_LIST_HEAD(&ce->signals);
202+
203+ mutex_init(&ce->pin_mutex);
204+
205+ i915_active_init(ctx->i915, &ce->active,
206+ __intel_context_active, __intel_context_retire);
207+}
208+
209+void intel_context_fini(struct intel_context *ce)
196210 {
197− /* Nodes preallocated in intel_context_active() */
198− i915_active_acquire_barrier(&ce->active);
199− i915_active_release(&ce->active);
211+ i915_vm_put(ce->vm);
212+
213+ mutex_destroy(&ce->pin_mutex);
214+ i915_active_fini(&ce->active);
200215 }
201216
202217 static void i915_global_context_shrink(void)
@@ -234,6 +249,44 @@ void intel_context_exit_engine(struct intel_context *ce) · 31 unmodified lines
234249 intel_engine_pm_put(ce->engine);
235250 }
236251
252+int intel_context_prepare_remote_request(struct intel_context *ce,
253+ struct i915_request *rq)
254+{
255+ struct intel_timeline *tl = ce->ring->timeline;
256+ int err;
257+
258+ /* Only suitable for use in remotely modifying this context */
259+ GEM_BUG_ON(rq->hw_context == ce);
260+
261+ if (rq->timeline != tl) { /* beware timeline sharing */
262+ err = mutex_lock_interruptible_nested(&tl->mutex,
263+ SINGLE_DEPTH_NESTING);
264+ if (err)
265+ return err;
266+
267+ /* Queue this switch after current activity by this context. */
268+ err = i915_active_request_set(&tl->last_request, rq);
269+ if (err)
270+ goto unlock;
271+ }
272+ lockdep_assert_held(&tl->mutex);
273+
274+ /*
275+ * Guarantee context image and the timeline remains pinned until the
276+ * modifying request is retired by setting the ce activity tracker.
277+ *
278+ * But we only need to take one pin on the account of it. Or in other
279+ * words transfer the pinned ce object to tracked active request.
280+ */
281+ GEM_BUG_ON(i915_active_is_idle(&ce->active));
282+ err = i915_active_ref(&ce->active, rq->fence.context, rq);
283+
284+unlock:
285+ if (rq->timeline != tl)
286+ mutex_unlock(&tl->mutex);
287+ return err;
288+}
289+
237290 struct i915_request *intel_context_create_request(struct intel_context *ce)
238291 {
239292 struct i915_request *rq;
drivers/gpu/drm/i915/gt/intel_context.h+16−2
@@ -9,12 +9,14 @@ · 8 unmodified lines
99
1010 #include <linux/lockdep.h>
1111
12+#include "i915_active.h"
1213 #include "intel_context_types.h"
1314 #include "intel_engine_types.h"
1415
1516 void intel_context_init(struct intel_context *ce,
1617 struct i915_gem_context *ctx,
1718 struct intel_engine_cs *engine);
19+void intel_context_fini(struct intel_context *ce);
1820
1921 struct intel_context *
2022 intel_context_create(struct i915_gem_context *ctx,
@@ -102,8 +104,17 @@ static inline void intel_context_exit(struct intel_context *ce) · 81 unmodified lines
102104 ce->ops->exit(ce);
103105 }
104106
105−int intel_context_active_acquire(struct intel_context *ce, unsigned long flags);
106−void intel_context_active_release(struct intel_context *ce);
107+static inline int intel_context_active_acquire(struct intel_context *ce)
108+{
109+ return i915_active_acquire(&ce->active);
110+}
111+
112+static inline void intel_context_active_release(struct intel_context *ce)
113+{
114+ /* Nodes preallocated in intel_context_active() */
115+ i915_active_acquire_barrier(&ce->active);
116+ i915_active_release(&ce->active);
117+}
107118
108119 static inline struct intel_context *intel_context_get(struct intel_context *ce)
109120 {
@@ -129,6 +140,9 @@ static inline void intel_context_timeline_unlock(struct intel_context *ce) · 19 unmodified lines
129140 mutex_unlock(&ce->ring->timeline->mutex);
130141 }
131142
143+int intel_context_prepare_remote_request(struct intel_context *ce,
144+ struct i915_request *rq);
145+
132146 struct i915_request *intel_context_create_request(struct intel_context *ce);
133147
134148 #endif /* __INTEL_CONTEXT_H__ */
drivers/gpu/drm/i915/gt/intel_context_types.h+8−1
@@ -13,6 +13,7 @@ · 12 unmodified lines
1313 #include <linux/types.h>
1414
1515 #include "i915_active_types.h"
16+#include "i915_utils.h"
1617 #include "intel_engine_types.h"
1718 #include "intel_sseu.h"
1819
@@ -35,9 +36,15 @@ struct intel_context_ops { · 16 unmodified lines
3536 struct intel_context {
3637 struct kref ref;
3738
38− struct i915_gem_context *gem_context;
3939 struct intel_engine_cs *engine;
4040 struct intel_engine_cs *inflight;
41+#define intel_context_inflight(ce) ptr_mask_bits((ce)->inflight, 2)
42+#define intel_context_inflight_count(ce) ptr_unmask_bits((ce)->inflight, 2)
43+#define intel_context_inflight_inc(ce) ptr_count_inc(&(ce)->inflight)
44+#define intel_context_inflight_dec(ce) ptr_count_dec(&(ce)->inflight)
45+
46+ struct i915_address_space *vm;
47+ struct i915_gem_context *gem_context;
4148
4249 struct list_head signal_link;
4350 struct list_head signals;
drivers/gpu/drm/i915/gt/intel_engine.h+15−60
@@ -14,7 +14,7 @@ · 13 unmodified lines
1414 #include "i915_reg.h"
1515 #include "i915_request.h"
1616 #include "i915_selftest.h"
17−#include "i915_timeline.h"
17+#include "gt/intel_timeline.h"
1818 #include "intel_engine_types.h"
1919 #include "intel_gpu_commands.h"
2020 #include "intel_workarounds.h"
@@ -51,7 +51,7 @@ struct drm_printer; · 30 unmodified lines
5151 #define ENGINE_READ16(...) __ENGINE_READ_OP(read16, __VA_ARGS__)
5252 #define ENGINE_READ(...) __ENGINE_READ_OP(read, __VA_ARGS__)
5353 #define ENGINE_READ_FW(...) __ENGINE_READ_OP(read_fw, __VA_ARGS__)
54−#define ENGINE_POSTING_READ(...) __ENGINE_READ_OP(posting_read, __VA_ARGS__)
54+#define ENGINE_POSTING_READ(...) __ENGINE_READ_OP(posting_read_fw, __VA_ARGS__)
5555 #define ENGINE_POSTING_READ16(...) __ENGINE_READ_OP(posting_read16, __VA_ARGS__)
5656
5757 #define ENGINE_READ64(engine__, lower_reg__, upper_reg__) \
@@ -125,71 +125,26 @@ hangcheck_action_to_str(const enum intel_engine_hangcheck_action a) · 67 unmodified lines
125125
126126 void intel_engines_set_scheduler_caps(struct drm_i915_private *i915);
127127
128−static inline void
129−execlists_set_active(struct intel_engine_execlists *execlists,
130− unsigned int bit)
131−{
132− __set_bit(bit, (unsigned long *)&execlists->active);
133−}
134−
135−static inline bool
136−execlists_set_active_once(struct intel_engine_execlists *execlists,
137− unsigned int bit)
138−{
139− return !__test_and_set_bit(bit, (unsigned long *)&execlists->active);
140−}
141−
142−static inline void
143−execlists_clear_active(struct intel_engine_execlists *execlists,
144− unsigned int bit)
145−{
146− __clear_bit(bit, (unsigned long *)&execlists->active);
147−}
148−
149−static inline void
150−execlists_clear_all_active(struct intel_engine_execlists *execlists)
128+static inline unsigned int
129+execlists_num_ports(const struct intel_engine_execlists * const execlists)
151130 {
152− execlists->active = 0;
131+ return execlists->port_mask + 1;
153132 }
154133
155−static inline bool
156−execlists_is_active(const struct intel_engine_execlists *execlists,
157− unsigned int bit)
134+static inline struct i915_request *
135+execlists_active(const struct intel_engine_execlists *execlists)
158136 {
159− return test_bit(bit, (unsigned long *)&execlists->active);
137+ GEM_BUG_ON(execlists->active - execlists->inflight >
138+ execlists_num_ports(execlists));
139+ return READ_ONCE(*execlists->active);
160140 }
161141
162−void execlists_user_begin(struct intel_engine_execlists *execlists,
163− const struct execlist_port *port);
164−void execlists_user_end(struct intel_engine_execlists *execlists);
165−
166142 void
167143 execlists_cancel_port_requests(struct intel_engine_execlists * const execlists);
168144
169145 struct i915_request *
170146 execlists_unwind_incomplete_requests(struct intel_engine_execlists *execlists);
171147
172−static inline unsigned int
173−execlists_num_ports(const struct intel_engine_execlists * const execlists)
174−{
175− return execlists->port_mask + 1;
176−}
177−
178−static inline struct execlist_port *
179−execlists_port_complete(struct intel_engine_execlists * const execlists,
180− struct execlist_port * const port)
181−{
182− const unsigned int m = execlists->port_mask;
183−
184− GEM_BUG_ON(port_index(port, execlists) != 0);
185− GEM_BUG_ON(!execlists_is_active(execlists, EXECLISTS_ACTIVE_USER));
186−
187− memmove(port, port + 1, m * sizeof(struct execlist_port));
188− memset(port + m, 0, sizeof(struct execlist_port));
189−
190− return port;
191−}
192−
193148 static inline u32
194149 intel_read_status_page(const struct intel_engine_cs *engine, int reg)
195150 {
@@ -245,7 +200,7 @@ intel_write_status_page(struct intel_engine_cs *engine, int reg, u32 value) · 49 unmodified lines
245200
246201 struct intel_ring *
247202 intel_engine_create_ring(struct intel_engine_cs *engine,
248− struct i915_timeline *timeline,
203+ struct intel_timeline *timeline,
249204 int size);
250205 int intel_ring_pin(struct intel_ring *ring);
251206 void intel_ring_reset(struct intel_ring *ring, u32 tail);
@@ -456,8 +411,8 @@ gen8_emit_ggtt_write(u32 *cs, u32 value, u32 gtt_offset, u32 flags) · 204 unmodified lines
456411 return cs;
457412 }
458413
459−static inline void intel_engine_reset(struct intel_engine_cs *engine,
460− bool stalled)
414+static inline void __intel_engine_reset(struct intel_engine_cs *engine,
415+ bool stalled)
461416 {
462417 if (engine->reset.reset)
463418 engine->reset.reset(engine, stalled);
@@ -465,9 +420,9 @@ static inline void intel_engine_reset(struct intel_engine_cs *engine, · 1 unmodified line
465420 }
466421
467422 bool intel_engine_is_idle(struct intel_engine_cs *engine);
468−bool intel_engines_are_idle(struct drm_i915_private *dev_priv);
423+bool intel_engines_are_idle(struct intel_gt *gt);
469424
470−void intel_engines_reset_default_submission(struct drm_i915_private *i915);
425+void intel_engines_reset_default_submission(struct intel_gt *gt);
471426 unsigned int intel_engines_has_context_isolation(struct drm_i915_private *i915);
472427
473428 bool intel_engine_can_store_dword(struct intel_engine_cs *engine);
drivers/gpu/drm/i915/gt/intel_engine_cs.c+85−118
@@ -28,6 +28,8 @@ · 27 unmodified lines
2828
2929 #include "i915_drv.h"
3030
31+#include "gt/intel_gt.h"
32+
3133 #include "intel_engine.h"
3234 #include "intel_engine_pm.h"
3335 #include "intel_context.h"
@@ -314,6 +316,7 @@ intel_engine_setup(struct drm_i915_private *dev_priv, · 280 unmodified lines
314316 engine->id = id;
315317 engine->mask = BIT(id);
316318 engine->i915 = dev_priv;
319+ engine->gt = &dev_priv->gt;
317320 engine->uncore = &dev_priv->uncore;
318321 __sprint_engine_name(engine->name, info);
319322 engine->hw_id = engine->guc_id = info->hw_id;
@@ -423,7 +426,7 @@ int intel_engines_init_mmio(struct drm_i915_private *i915) · 103 unmodified lines
423426 WARN_ON(engine_mask &
424427 GENMASK(BITS_PER_TYPE(mask) - 1, I915_NUM_ENGINES));
425428
426− if (i915_inject_load_failure())
429+ if (i915_inject_probe_failure())
427430 return -ENODEV;
428431
429432 for (i = 0; i < ARRAY_SIZE(intel_engines); i++) {
@@ -445,15 +448,9 @@ int intel_engines_init_mmio(struct drm_i915_private *i915) · 15 unmodified lines
445448 if (WARN_ON(mask != engine_mask))
446449 device_info->engine_mask = mask;
447450
448− /* We always presume we have at least RCS available for later probing */
449− if (WARN_ON(!HAS_ENGINE(i915, RCS0))) {
450− err = -ENODEV;
451− goto cleanup;
452− }
453−
454451 RUNTIME_INFO(i915)->num_engines = hweight32(mask);
455452
456− i915_check_and_clear_faults(i915);
453+ intel_gt_check_and_clear_faults(&i915->gt);
457454
458455 intel_setup_engine_capabilities(i915);
459456
@@ -508,6 +505,10 @@ void intel_engine_init_execlists(struct intel_engine_cs *engine) · 48 unmodified lines
508505 GEM_BUG_ON(!is_power_of_2(execlists_num_ports(execlists)));
509506 GEM_BUG_ON(execlists_num_ports(execlists) > EXECLIST_MAX_PORTS);
510507
508+ memset(execlists->pending, 0, sizeof(execlists->pending));
509+ execlists->active =
510+ memset(execlists->inflight, 0, sizeof(execlists->inflight));
511+
511512 execlists->queue_priority_hint = INT_MIN;
512513 execlists->queue = RB_ROOT_CACHED;
513514 }
@@ -577,7 +578,7 @@ static int init_status_page(struct intel_engine_cs *engine) · 63 unmodified lines
577578
578579 i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);
579580
580− vma = i915_vma_instance(obj, &engine->i915->ggtt.vm, NULL);
581+ vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);
581582 if (IS_ERR(vma)) {
582583 ret = PTR_ERR(vma);
583584 goto err;
@@ -629,6 +630,10 @@ static int intel_engine_setup_common(struct intel_engine_cs *engine) · 45 unmodified lines
629630 engine->sseu =
630631 intel_sseu_from_device_info(&RUNTIME_INFO(engine->i915)->sseu);
631632
633+ intel_engine_init_workarounds(engine);
634+ intel_engine_init_whitelist(engine);
635+ intel_engine_init_ctx_wa(engine);
636+
632637 return 0;
633638 }
634639
@@ -681,9 +686,10 @@ void intel_engines_set_scheduler_caps(struct drm_i915_private *i915) · 46 unmodified lines
681686 u8 engine;
682687 u8 sched;
683688 } map[] = {
684−#define MAP(x, y) { ilog2(I915_ENGINE_HAS_##x), ilog2(I915_SCHEDULER_CAP_##y) }
685− MAP(PREEMPTION, PREEMPTION),
686− MAP(SEMAPHORES, SEMAPHORES),
689+#define MAP(x, y) { ilog2(I915_ENGINE_##x), ilog2(I915_SCHEDULER_CAP_##y) }
690+ MAP(HAS_PREEMPTION, PREEMPTION),
691+ MAP(HAS_SEMAPHORES, SEMAPHORES),
692+ MAP(SUPPORTS_STATS, ENGINE_BUSY_STATS),
687693 #undef MAP
688694 };
689695 struct intel_engine_cs *engine;
@@ -717,7 +723,7 @@ void intel_engines_set_scheduler_caps(struct drm_i915_private *i915) · 27 unmodified lines
717723
718724 struct measure_breadcrumb {
719725 struct i915_request rq;
720− struct i915_timeline timeline;
726+ struct intel_timeline timeline;
721727 struct intel_ring ring;
722728 u32 cs[1024];
723729 };
@@ -727,15 +733,15 @@ static int measure_breadcrumb_dw(struct intel_engine_cs *engine) · 3 unmodified lines
727733 struct measure_breadcrumb *frame;
728734 int dw = -ENOMEM;
729735
730− GEM_BUG_ON(!engine->i915->gt.scratch);
736+ GEM_BUG_ON(!engine->gt->scratch);
731737
732738 frame = kzalloc(sizeof(*frame), GFP_KERNEL);
733739 if (!frame)
734740 return -ENOMEM;
735741
736− if (i915_timeline_init(engine->i915,
737− &frame->timeline,
738− engine->status_page.vma))
742+ if (intel_timeline_init(&frame->timeline,
743+ engine->gt,
744+ engine->status_page.vma))
739745 goto out_frame;
740746
741747 INIT_LIST_HEAD(&frame->ring.request_list);
@@ -750,17 +756,17 @@ static int measure_breadcrumb_dw(struct intel_engine_cs *engine) · 8 unmodified lines
750756 frame->rq.ring = &frame->ring;
751757 frame->rq.timeline = &frame->timeline;
752758
753− dw = i915_timeline_pin(&frame->timeline);
759+ dw = intel_timeline_pin(&frame->timeline);
754760 if (dw < 0)
755761 goto out_timeline;
756762
757763 dw = engine->emit_fini_breadcrumb(&frame->rq, frame->cs) - frame->cs;
758764 GEM_BUG_ON(dw & 1); /* RING_TAIL must be qword aligned */
759765
760− i915_timeline_unpin(&frame->timeline);
766+ intel_timeline_unpin(&frame->timeline);
761767
762768 out_timeline:
763− i915_timeline_fini(&frame->timeline);
769+ intel_timeline_fini(&frame->timeline);
764770 out_frame:
765771 kfree(frame);
766772 return dw;
@@ -823,6 +829,8 @@ int intel_engine_init_common(struct intel_engine_cs *engine) · 56 unmodified lines
823829 struct drm_i915_private *i915 = engine->i915;
824830 int ret;
825831
832+ engine->set_default_submission(engine);
833+
826834 /* We may need to do things with the shrinker which
827835 * require us to immediately switch back to the default
828836 * context. This can cause a problem as pinning the
@@ -835,28 +843,15 @@ int intel_engine_init_common(struct intel_engine_cs *engine) · 6 unmodified lines
835843 if (ret)
836844 return ret;
837845
838− /*
839− * Similarly the preempt context must always be available so that
840− * we can interrupt the engine at any time. However, as preemption
841− * is optional, we allow it to fail.
842− */
843− if (i915->preempt_context)
844− pin_context(i915->preempt_context, engine,
845− &engine->preempt_context);
846−
847846 ret = measure_breadcrumb_dw(engine);
848847 if (ret < 0)
849848 goto err_unpin;
850849
851850 engine->emit_fini_breadcrumb_dw = ret;
852851
853− engine->set_default_submission(engine);
854−
855852 return 0;
856853
857854 err_unpin:
858− if (engine->preempt_context)
859− intel_context_unpin(engine->preempt_context);
860855 intel_context_unpin(engine->kernel_context);
861856 return ret;
862857 }
@@ -881,8 +876,6 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine) · 18 unmodified lines
881876 if (engine->default_state)
882877 i915_gem_object_put(engine->default_state);
883878
884− if (engine->preempt_context)
885− intel_context_unpin(engine->preempt_context);
886879 intel_context_unpin(engine->kernel_context);
887880 GEM_BUG_ON(!llist_empty(&engine->barrier_tasks));
888881
@@ -966,57 +959,23 @@ const char *i915_cache_level_str(struct drm_i915_private *i915, int type) · 77 unmodified lines
966959 }
967960 }
968961
969−u32 intel_calculate_mcr_s_ss_select(struct drm_i915_private *dev_priv)
970−{
971− const struct sseu_dev_info *sseu = &RUNTIME_INFO(dev_priv)->sseu;
972− unsigned int slice = fls(sseu->slice_mask) - 1;
973− unsigned int subslice;
974− u32 mcr_s_ss_select;
975−
976− GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask));
977− subslice = fls(sseu->subslice_mask[slice]);
978− GEM_BUG_ON(!subslice);
979− subslice--;
980−
981− if (IS_GEN(dev_priv, 10))
982− mcr_s_ss_select = GEN8_MCR_SLICE(slice) |
983− GEN8_MCR_SUBSLICE(subslice);
984− else if (INTEL_GEN(dev_priv) >= 11)
985− mcr_s_ss_select = GEN11_MCR_SLICE(slice) |
986− GEN11_MCR_SUBSLICE(subslice);
987− else
988− mcr_s_ss_select = 0;
989−
990− return mcr_s_ss_select;
991−}
992−
993962 static u32
994963 read_subslice_reg(struct intel_engine_cs *engine, int slice, int subslice,
995964 i915_reg_t reg)
996965 {
997966 struct drm_i915_private *i915 = engine->i915;
998967 struct intel_uncore *uncore = engine->uncore;
999− u32 mcr_slice_subslice_mask;
1000− u32 mcr_slice_subslice_select;
1001− u32 default_mcr_s_ss_select;
1002− u32 mcr;
1003− u32 ret;
968+ u32 mcr_mask, mcr_ss, mcr, old_mcr, val;
1004969 enum forcewake_domains fw_domains;
1005970
1006971 if (INTEL_GEN(i915) >= 11) {
1007− mcr_slice_subslice_mask = GEN11_MCR_SLICE_MASK |
1008− GEN11_MCR_SUBSLICE_MASK;
1009− mcr_slice_subslice_select = GEN11_MCR_SLICE(slice) |
1010− GEN11_MCR_SUBSLICE(subslice);
972+ mcr_mask = GEN11_MCR_SLICE_MASK | GEN11_MCR_SUBSLICE_MASK;
973+ mcr_ss = GEN11_MCR_SLICE(slice) | GEN11_MCR_SUBSLICE(subslice);
1011974 } else {
1012− mcr_slice_subslice_mask = GEN8_MCR_SLICE_MASK |
1013− GEN8_MCR_SUBSLICE_MASK;
1014− mcr_slice_subslice_select = GEN8_MCR_SLICE(slice) |
1015− GEN8_MCR_SUBSLICE(subslice);
975+ mcr_mask = GEN8_MCR_SLICE_MASK | GEN8_MCR_SUBSLICE_MASK;
976+ mcr_ss = GEN8_MCR_SLICE(slice) | GEN8_MCR_SUBSLICE(subslice);
1016977 }
1017978
1018− default_mcr_s_ss_select = intel_calculate_mcr_s_ss_select(i915);
1019−
1020979 fw_domains = intel_uncore_forcewake_for_reg(uncore, reg,
1021980 FW_REG_READ);
1022981 fw_domains |= intel_uncore_forcewake_for_reg(uncore,
@@ -1026,26 +985,23 @@ read_subslice_reg(struct intel_engine_cs *engine, int slice, int subslice, · 3 unmodified lines
1026985 spin_lock_irq(&uncore->lock);
1027986 intel_uncore_forcewake_get__locked(uncore, fw_domains);
1028987
1029− mcr = intel_uncore_read_fw(uncore, GEN8_MCR_SELECTOR);
1030−
1031− WARN_ON_ONCE((mcr & mcr_slice_subslice_mask) !=
1032− default_mcr_s_ss_select);
988+ old_mcr = mcr = intel_uncore_read_fw(uncore, GEN8_MCR_SELECTOR);
1033989
1034− mcr &= ~mcr_slice_subslice_mask;
1035− mcr |= mcr_slice_subslice_select;
990+ mcr &= ~mcr_mask;
991+ mcr |= mcr_ss;
1036992 intel_uncore_write_fw(uncore, GEN8_MCR_SELECTOR, mcr);
1037993
1038− ret = intel_uncore_read_fw(uncore, reg);
994+ val = intel_uncore_read_fw(uncore, reg);
1039995
1040− mcr &= ~mcr_slice_subslice_mask;
1041− mcr |= default_mcr_s_ss_select;
996+ mcr &= ~mcr_mask;
997+ mcr |= old_mcr & mcr_mask;
1042998
1043999 intel_uncore_write_fw(uncore, GEN8_MCR_SELECTOR, mcr);
10441000
10451001 intel_uncore_forcewake_put__locked(uncore, fw_domains);
10461002 spin_unlock_irq(&uncore->lock);
10471003
1048− return ret;
1004+ return val;
10491005 }
10501006
10511007 /* NB: please notice the memset */
@@ -1150,17 +1106,17 @@ static bool ring_is_idle(struct intel_engine_cs *engine) · 98 unmodified lines
11501106 bool intel_engine_is_idle(struct intel_engine_cs *engine)
11511107 {
11521108 /* More white lies, if wedged, hw state is inconsistent */
1153− if (i915_reset_failed(engine->i915))
1109+ if (intel_gt_is_wedged(engine->gt))
11541110 return true;
11551111
1156− if (!intel_wakeref_active(&engine->wakeref))
1112+ if (!intel_engine_pm_is_awake(engine))
11571113 return true;
11581114
11591115 /* Waiting to drain ELSP? */
1160− if (READ_ONCE(engine->execlists.active)) {
1116+ if (execlists_active(&engine->execlists)) {
11611117 struct tasklet_struct *t = &engine->execlists.tasklet;
11621118
1163− synchronize_hardirq(engine->i915->drm.irq);
1119+ synchronize_hardirq(engine->i915->drm.pdev->irq);
11641120
11651121 local_bh_disable();
11661122 if (tasklet_trylock(t)) {
@@ -1174,7 +1130,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine) · 7 unmodified lines
11741130 /* Otherwise flush the tasklet if it was on another cpu */
11751131 tasklet_unlock_wait(t);
11761132
1177− if (READ_ONCE(engine->execlists.active))
1133+ if (execlists_active(&engine->execlists))
11781134 return false;
11791135 }
11801136
@@ -1186,7 +1142,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine) · 5 unmodified lines
11861142 return ring_is_idle(engine);
11871143 }
11881144
1189−bool intel_engines_are_idle(struct drm_i915_private *i915)
1145+bool intel_engines_are_idle(struct intel_gt *gt)
11901146 {
11911147 struct intel_engine_cs *engine;
11921148 enum intel_engine_id id;
@@ -1195,14 +1151,14 @@ bool intel_engines_are_idle(struct drm_i915_private *i915) · 2 unmodified lines
11951151 * If the driver is wedged, HW state may be very inconsistent and
11961152 * report that it is still busy, even though we have stopped using it.
11971153 */
1198− if (i915_reset_failed(i915))
1154+ if (intel_gt_is_wedged(gt))
11991155 return true;
12001156
12011157 /* Already parked (and passed an idleness test); must still be idle */
1202− if (!READ_ONCE(i915->gt.awake))
1158+ if (!READ_ONCE(gt->awake))
12031159 return true;
12041160
1205− for_each_engine(engine, i915, id) {
1161+ for_each_engine(engine, gt->i915, id) {
12061162 if (!intel_engine_is_idle(engine))
12071163 return false;
12081164 }
@@ -1210,12 +1166,12 @@ bool intel_engines_are_idle(struct drm_i915_private *i915) · 1 unmodified line
12101166 return true;
12111167 }
12121168
1213−void intel_engines_reset_default_submission(struct drm_i915_private *i915)
1169+void intel_engines_reset_default_submission(struct intel_gt *gt)
12141170 {
12151171 struct intel_engine_cs *engine;
12161172 enum intel_engine_id id;
12171173
1218− for_each_engine(engine, i915, id)
1174+ for_each_engine(engine, gt->i915, id)
12191175 engine->set_default_submission(engine);
12201176 }
12211177
@@ -1372,6 +1328,7 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine, · 150 unmodified lines
13721328 }
13731329
13741330 if (HAS_EXECLISTS(dev_priv)) {
1331+ struct i915_request * const *port, *rq;
13751332 const u32 *hws =
13761333 &engine->status_page.addr[I915_HWS_CSB_BUF0_INDEX];
13771334 const u8 num_entries = execlists->csb_size;
@@ -1404,27 +1361,33 @@ static void intel_engine_print_registers(struct intel_engine_cs *engine, · 26 unmodified lines
14041361 }
14051362
14061363 spin_lock_irqsave(&engine->active.lock, flags);
1407− for (idx = 0; idx < execlists_num_ports(execlists); idx++) {
1408− struct i915_request *rq;
1409− unsigned int count;
1364+ for (port = execlists->active; (rq = *port); port++) {
1365+ char hdr[80];
1366+ int len;
1367+
1368+ len = snprintf(hdr, sizeof(hdr),
1369+ "\t\tActive[%d: ",
1370+ (int)(port - execlists->active));
1371+ if (!i915_request_signaled(rq))
1372+ len += snprintf(hdr + len, sizeof(hdr) - len,
1373+ "ring:{start:%08x, hwsp:%08x, seqno:%08x}, ",
1374+ i915_ggtt_offset(rq->ring->vma),
1375+ rq->timeline->hwsp_offset,
1376+ hwsp_seqno(rq));
1377+ snprintf(hdr + len, sizeof(hdr) - len, "rq: ");
1378+ print_request(m, rq, hdr);
1379+ }
1380+ for (port = execlists->pending; (rq = *port); port++) {
14101381 char hdr[80];
14111382
1412− rq = port_unpack(&execlists->port[idx], &count);
1413− if (!rq) {
1414− drm_printf(m, "\t\tELSP[%d] idle\n", idx);
1415− } else if (!i915_request_signaled(rq)) {
1416− snprintf(hdr, sizeof(hdr),
1417− "\t\tELSP[%d] count=%d, ring:{start:%08x, hwsp:%08x, seqno:%08x}, rq: ",
1418− idx, count,
1419− i915_ggtt_offset(rq->ring->vma),
1420− rq->timeline->hwsp_offset,
1421− hwsp_seqno(rq));
1422− print_request(m, rq, hdr);
1423− } else {
1424− print_request(m, rq, "\t\tELSP[%d] rq: ");
1425− }
1383+ snprintf(hdr, sizeof(hdr),
1384+ "\t\tPending[%d] ring:{start:%08x, hwsp:%08x, seqno:%08x}, rq: ",
1385+ (int)(port - execlists->pending),
1386+ i915_ggtt_offset(rq->ring->vma),
1387+ rq->timeline->hwsp_offset,
1388+ hwsp_seqno(rq));
1389+ print_request(m, rq, hdr);
14261390 }
1427− drm_printf(m, "\t\tHW active? 0x%x\n", execlists->active);
14281391 spin_unlock_irqrestore(&engine->active.lock, flags);
14291392 } else if (INTEL_GEN(dev_priv) > 6) {
14301393 drm_printf(m, "\tPP_DIR_BASE: 0x%08x\n",
@@ -1486,7 +1449,7 @@ void intel_engine_dump(struct intel_engine_cs *engine, · 55 unmodified lines
14861449 va_end(ap);
14871450 }
14881451
1489− if (i915_reset_failed(engine->i915))
1452+ if (intel_gt_is_wedged(engine->gt))
14901453 drm_printf(m, "*** WEDGED ***\n");
14911454
14921455 drm_printf(m, "\tAwake? %d\n", atomic_read(&engine->wakeref.count));
@@ -1587,15 +1550,19 @@ int intel_enable_engine_stats(struct intel_engine_cs *engine) · 94 unmodified lines
15871550 }
15881551
15891552 if (engine->stats.enabled++ == 0) {
1590− const struct execlist_port *port = execlists->port;
1591− unsigned int num_ports = execlists_num_ports(execlists);
1553+ struct i915_request * const *port;
1554+ struct i915_request *rq;
15921555
15931556 engine->stats.enabled_at = ktime_get();
15941557
15951558 /* XXX submission method oblivious? */
1596− while (num_ports-- && port_isset(port)) {
1559+ for (port = execlists->active; (rq = *port); port++)
15971560 engine->stats.active++;
1598− port++;
1561+
1562+ for (port = execlists->pending; (rq = *port); port++) {
1563+ /* Exclude any contexts already counted in active */
1564+ if (intel_context_inflight_count(rq->hw_context) == 1)
1565+ engine->stats.active++;
15991566 }
16001567
16011568 if (engine->stats.active)
drivers/gpu/drm/i915/gt/intel_engine_pm.c+4−3
@@ -8,6 +8,7 @@ · 7 unmodified lines
88
99 #include "intel_engine.h"
1010 #include "intel_engine_pm.h"
11+#include "intel_gt.h"
1112 #include "intel_gt_pm.h"
1213
1314 static int __engine_unpark(struct intel_wakeref *wf)
@@ -18,7 +19,7 @@ static int __engine_unpark(struct intel_wakeref *wf) · 4 unmodified lines
1819
1920 GEM_TRACE("%s\n", engine->name);
2021
21− intel_gt_pm_get(engine->i915);
22+ intel_gt_pm_get(engine->gt);
2223
2324 /* Pin the default state for fast resets from atomic context. */
2425 map = NULL;
@@ -66,7 +67,7 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine) · 41 unmodified lines
6667 return true;
6768
6869 /* GPU is pointing to the void, as good as in the kernel context. */
69− if (i915_reset_failed(engine->i915))
70+ if (intel_gt_is_wedged(engine->gt))
7071 return true;
7172
7273 /*
@@ -129,7 +130,7 @@ static int __engine_park(struct intel_wakeref *wf) · 56 unmodified lines
129130
130131 engine->execlists.no_priolist = false;
131132
132− intel_gt_pm_put(engine->i915);
133+ intel_gt_pm_put(engine->gt);
133134 return 0;
134135 }
135136
drivers/gpu/drm/i915/gt/intel_engine_pm.h+6−0
@@ -15,6 +15,12 @@ struct drm_i915_private; · 14 unmodified lines
1515 void intel_engine_pm_get(struct intel_engine_cs *engine);
1616 void intel_engine_pm_put(struct intel_engine_cs *engine);
1717
18+static inline bool
19+intel_engine_pm_is_awake(const struct intel_engine_cs *engine)
20+{
21+ return intel_wakeref_is_active(&engine->wakeref);
22+}
23+
1824 static inline bool
1925 intel_engine_pm_get_if_awake(struct intel_engine_cs *engine)
2026 {
drivers/gpu/drm/i915/gt/intel_engine_types.h+26−48
@@ -12,6 +12,7 @@ · 11 unmodified lines
1212 #include <linux/kref.h>
1313 #include <linux/list.h>
1414 #include <linux/llist.h>
15+#include <linux/timer.h>
1516 #include <linux/types.h>
1617
1718 #include "i915_gem.h"
@@ -19,7 +20,7 @@ · 1 unmodified line
1920 #include "i915_pmu.h"
2021 #include "i915_priolist_types.h"
2122 #include "i915_selftest.h"
22−#include "i915_timeline_types.h"
23+#include "gt/intel_timeline_types.h"
2324 #include "intel_sseu.h"
2425 #include "intel_wakeref.h"
2526 #include "intel_workarounds_types.h"
@@ -35,6 +36,7 @@ struct drm_i915_reg_table; · 9 unmodified lines
3536 struct i915_gem_context;
3637 struct i915_request;
3738 struct i915_sched_attr;
39+struct intel_gt;
3840 struct intel_uncore;
3941
4042 typedef u8 intel_engine_mask_t;
@@ -66,7 +68,7 @@ struct intel_ring { · 25 unmodified lines
6668 struct i915_vma *vma;
6769 void *vaddr;
6870
69− struct i915_timeline *timeline;
71+ struct intel_timeline *timeline;
7072 struct list_head request_list;
7173 struct list_head active_link;
7274
@@ -149,6 +151,11 @@ struct intel_engine_execlists { · 76 unmodified lines
149151 */
150152 struct tasklet_struct tasklet;
151153
154+ /**
155+ * @timer: kick the current context if its timeslice expires
156+ */
157+ struct timer_list timer;
158+
152159 /**
153160 * @default_priolist: priority list for I915_PRIORITY_NORMAL
154161 */
@@ -172,51 +179,28 @@ struct intel_engine_execlists { · 17 unmodified lines
172179 */
173180 u32 __iomem *ctrl_reg;
174181
182+#define EXECLIST_MAX_PORTS 2
175183 /**
176− * @port: execlist port states
184+ * @active: the currently known context executing on HW
185+ */
186+ struct i915_request * const *active;
187+ /**
188+ * @inflight: the set of contexts submitted and acknowleged by HW
177189 *
178− * For each hardware ELSP (ExecList Submission Port) we keep
179− * track of the last request and the number of times we submitted
180− * that port to hw. We then count the number of times the hw reports
181− * a context completion or preemption. As only one context can
182− * be active on hw, we limit resubmission of context to port[0]. This
183− * is called Lite Restore, of the context.
190+ * The set of inflight contexts is managed by reading CS events
191+ * from the HW. On a context-switch event (not preemption), we
192+ * know the HW has transitioned from port0 to port1, and we
193+ * advance our inflight/active tracking accordingly.
184194 */
185− struct execlist_port {
186− /**
187− * @request_count: combined request and submission count
188− */
189− struct i915_request *request_count;
190−#define EXECLIST_COUNT_BITS 2
191−#define port_request(p) ptr_mask_bits((p)->request_count, EXECLIST_COUNT_BITS)
192−#define port_count(p) ptr_unmask_bits((p)->request_count, EXECLIST_COUNT_BITS)
193−#define port_pack(rq, count) ptr_pack_bits(rq, count, EXECLIST_COUNT_BITS)
194−#define port_unpack(p, count) ptr_unpack_bits((p)->request_count, count, EXECLIST_COUNT_BITS)
195−#define port_set(p, packed) ((p)->request_count = (packed))
196−#define port_isset(p) ((p)->request_count)
197−#define port_index(p, execlists) ((p) - (execlists)->port)
198−
199− /**
200− * @context_id: context ID for port
201− */
202− GEM_DEBUG_DECL(u32 context_id);
203−
204−#define EXECLIST_MAX_PORTS 2
205− } port[EXECLIST_MAX_PORTS];
206−
195+ struct i915_request *inflight[EXECLIST_MAX_PORTS + 1 /* sentinel */];
207196 /**
208− * @active: is the HW active? We consider the HW as active after
209− * submitting any context for execution and until we have seen the
210− * last context completion event. After that, we do not expect any
211− * more events until we submit, and so can park the HW.
197+ * @pending: the next set of contexts submitted to ELSP
212198 *
213− * As we have a small number of different sources from which we feed
214− * the HW, we track the state of each inside a single bitfield.
199+ * We store the array of contexts that we submit to HW (via ELSP) and
200+ * promote them to the inflight array once HW has signaled the
201+ * preemption or idle-to-active event.
215202 */
216− unsigned int active;
217−#define EXECLISTS_ACTIVE_USER 0
218−#define EXECLISTS_ACTIVE_PREEMPT 1
219−#define EXECLISTS_ACTIVE_HWACK 2
203+ struct i915_request *pending[EXECLIST_MAX_PORTS + 1];
220204
221205 /**
222206 * @port_mask: number of execlist ports - 1
@@ -257,11 +241,6 @@ struct intel_engine_execlists { · 34 unmodified lines
257241 */
258242 u32 *csb_status;
259243
260− /**
261− * @preempt_complete_status: expected CSB upon completing preemption
262− */
263− u32 preempt_complete_status;
264−
265244 /**
266245 * @csb_size: context status buffer FIFO size
267246 */
@@ -279,6 +258,7 @@ struct intel_engine_execlists { · 11 unmodified lines
279258
280259 struct intel_engine_cs {
281260 struct drm_i915_private *i915;
261+ struct intel_gt *gt;
282262 struct intel_uncore *uncore;
283263 char name[INTEL_ENGINE_CS_MAX_NAME];
284264
@@ -308,7 +288,6 @@ struct intel_engine_cs { · 23 unmodified lines
308288 struct llist_head barrier_tasks;
309289
310290 struct intel_context *kernel_context; /* pinned */
311− struct intel_context *preempt_context; /* pinned; optional */
312291
313292 intel_engine_mask_t saturated; /* submitting semaphores too late? */
314293
@@ -404,7 +383,6 @@ struct intel_engine_cs { · 89 unmodified lines
404383 const struct intel_context_ops *cops;
405384
406385 int (*request_alloc)(struct i915_request *rq);
407− int (*init_context)(struct i915_request *rq);
408386
409387 int (*emit_flush)(struct i915_request *request, u32 mode);
410388 #define EMIT_INVALIDATE BIT(0)
drivers/gpu/drm/i915/gt/intel_gpu_commands.h+7−0
@@ -7,6 +7,13 @@ · 6 unmodified lines
77 #ifndef _INTEL_GPU_COMMANDS_H_
88 #define _INTEL_GPU_COMMANDS_H_
99
10+/*
11+ * Target address alignments required for GPU access e.g.
12+ * MI_STORE_DWORD_IMM.
13+ */
14+#define alignof_dword 4
15+#define alignof_qword 8
16+
1017 /*
1118 * Instruction field definitions used by the command parser
1219 */
drivers/gpu/drm/i915/gt/intel_gt.c+250−0
@@ -0,0 +1,250 @@
1+// SPDX-License-Identifier: MIT
2+/*
3+ * Copyright © 2019 Intel Corporation
4+ */
5+
6+#include "i915_drv.h"
7+
8+#include "intel_gt.h"
9+#include "intel_gt_pm.h"
10+#include "intel_uncore.h"
11+
12+void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915)
13+{
14+ gt->i915 = i915;
15+ gt->uncore = &i915->uncore;
16+
17+ INIT_LIST_HEAD(&gt->active_rings);
18+ INIT_LIST_HEAD(&gt->closed_vma);
19+
20+ spin_lock_init(&gt->closed_lock);
21+
22+ intel_gt_init_hangcheck(gt);
23+ intel_gt_init_reset(gt);
24+ intel_gt_pm_init_early(gt);
25+}
26+
27+void intel_gt_init_hw(struct drm_i915_private *i915)
28+{
29+ i915->gt.ggtt = &i915->ggtt;
30+}
31+
32+static void rmw_set(struct intel_uncore *uncore, i915_reg_t reg, u32 set)
33+{
34+ intel_uncore_rmw(uncore, reg, 0, set);
35+}
36+
37+static void rmw_clear(struct intel_uncore *uncore, i915_reg_t reg, u32 clr)
38+{
39+ intel_uncore_rmw(uncore, reg, clr, 0);
40+}
41+
42+static void clear_register(struct intel_uncore *uncore, i915_reg_t reg)
43+{
44+ intel_uncore_rmw(uncore, reg, 0, 0);
45+}
46+
47+static void gen8_clear_engine_error_register(struct intel_engine_cs *engine)
48+{
49+ GEN6_RING_FAULT_REG_RMW(engine, RING_FAULT_VALID, 0);
50+ GEN6_RING_FAULT_REG_POSTING_READ(engine);
51+}
52+
53+void
54+intel_gt_clear_error_registers(struct intel_gt *gt,
55+ intel_engine_mask_t engine_mask)
56+{
57+ struct drm_i915_private *i915 = gt->i915;
58+ struct intel_uncore *uncore = gt->uncore;
59+ u32 eir;
60+
61+ if (!IS_GEN(i915, 2))
62+ clear_register(uncore, PGTBL_ER);
63+
64+ if (INTEL_GEN(i915) < 4)
65+ clear_register(uncore, IPEIR(RENDER_RING_BASE));
66+ else
67+ clear_register(uncore, IPEIR_I965);
68+
69+ clear_register(uncore, EIR);
70+ eir = intel_uncore_read(uncore, EIR);
71+ if (eir) {
72+ /*
73+ * some errors might have become stuck,
74+ * mask them.
75+ */
76+ DRM_DEBUG_DRIVER("EIR stuck: 0x%08x, masking\n", eir);
77+ rmw_set(uncore, EMR, eir);
78+ intel_uncore_write(uncore, GEN2_IIR,
79+ I915_MASTER_ERROR_INTERRUPT);
80+ }
81+
82+ if (INTEL_GEN(i915) >= 8) {
83+ rmw_clear(uncore, GEN8_RING_FAULT_REG, RING_FAULT_VALID);
84+ intel_uncore_posting_read(uncore, GEN8_RING_FAULT_REG);
85+ } else if (INTEL_GEN(i915) >= 6) {
86+ struct intel_engine_cs *engine;
87+ enum intel_engine_id id;
88+
89+ for_each_engine_masked(engine, i915, engine_mask, id)
90+ gen8_clear_engine_error_register(engine);
91+ }
92+}
93+
94+static void gen6_check_faults(struct intel_gt *gt)
95+{
96+ struct intel_engine_cs *engine;
97+ enum intel_engine_id id;
98+ u32 fault;
99+
100+ for_each_engine(engine, gt->i915, id) {
101+ fault = GEN6_RING_FAULT_REG_READ(engine);
102+ if (fault & RING_FAULT_VALID) {
103+ DRM_DEBUG_DRIVER("Unexpected fault\n"
104+ "\tAddr: 0x%08lx\n"
105+ "\tAddress space: %s\n"
106+ "\tSource ID: %d\n"
107+ "\tType: %d\n",
108+ fault & PAGE_MASK,
109+ fault & RING_FAULT_GTTSEL_MASK ?
110+ "GGTT" : "PPGTT",
111+ RING_FAULT_SRCID(fault),
112+ RING_FAULT_FAULT_TYPE(fault));
113+ }
114+ }
115+}
116+
117+static void gen8_check_faults(struct intel_gt *gt)
118+{
119+ struct intel_uncore *uncore = gt->uncore;
120+ u32 fault = intel_uncore_read(uncore, GEN8_RING_FAULT_REG);
121+
122+ if (fault & RING_FAULT_VALID) {
123+ u32 fault_data0, fault_data1;
124+ u64 fault_addr;
125+
126+ fault_data0 = intel_uncore_read(uncore, GEN8_FAULT_TLB_DATA0);
127+ fault_data1 = intel_uncore_read(uncore, GEN8_FAULT_TLB_DATA1);
128+ fault_addr = ((u64)(fault_data1 & FAULT_VA_HIGH_BITS) << 44) |
129+ ((u64)fault_data0 << 12);
130+
131+ DRM_DEBUG_DRIVER("Unexpected fault\n"
132+ "\tAddr: 0x%08x_%08x\n"
133+ "\tAddress space: %s\n"
134+ "\tEngine ID: %d\n"
135+ "\tSource ID: %d\n"
136+ "\tType: %d\n",
137+ upper_32_bits(fault_addr),
138+ lower_32_bits(fault_addr),
139+ fault_data1 & FAULT_GTT_SEL ? "GGTT" : "PPGTT",
140+ GEN8_RING_FAULT_ENGINE_ID(fault),
141+ RING_FAULT_SRCID(fault),
142+ RING_FAULT_FAULT_TYPE(fault));
143+ }
144+}
145+
146+void intel_gt_check_and_clear_faults(struct intel_gt *gt)
147+{
148+ struct drm_i915_private *i915 = gt->i915;
149+
150+ /* From GEN8 onwards we only have one 'All Engine Fault Register' */
151+ if (INTEL_GEN(i915) >= 8)
152+ gen8_check_faults(gt);
153+ else if (INTEL_GEN(i915) >= 6)
154+ gen6_check_faults(gt);
155+ else
156+ return;
157+
158+ intel_gt_clear_error_registers(gt, ALL_ENGINES);
159+}
160+
161+void intel_gt_flush_ggtt_writes(struct intel_gt *gt)
162+{
163+ struct drm_i915_private *i915 = gt->i915;
164+ intel_wakeref_t wakeref;
165+
166+ /*
167+ * No actual flushing is required for the GTT write domain for reads
168+ * from the GTT domain. Writes to it "immediately" go to main memory
169+ * as far as we know, so there's no chipset flush. It also doesn't
170+ * land in the GPU render cache.
171+ *
172+ * However, we do have to enforce the order so that all writes through
173+ * the GTT land before any writes to the device, such as updates to
174+ * the GATT itself.
175+ *
176+ * We also have to wait a bit for the writes to land from the GTT.
177+ * An uncached read (i.e. mmio) seems to be ideal for the round-trip
178+ * timing. This issue has only been observed when switching quickly
179+ * between GTT writes and CPU reads from inside the kernel on recent hw,
180+ * and it appears to only affect discrete GTT blocks (i.e. on LLC
181+ * system agents we cannot reproduce this behaviour, until Cannonlake
182+ * that was!).
183+ */
184+
185+ wmb();
186+
187+ if (INTEL_INFO(i915)->has_coherent_ggtt)
188+ return;
189+
190+ intel_gt_chipset_flush(gt);
191+
192+ with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
193+ struct intel_uncore *uncore = gt->uncore;
194+
195+ spin_lock_irq(&uncore->lock);
196+ intel_uncore_posting_read_fw(uncore,
197+ RING_HEAD(RENDER_RING_BASE));
198+ spin_unlock_irq(&uncore->lock);
199+ }
200+}
201+
202+void intel_gt_chipset_flush(struct intel_gt *gt)
203+{
204+ wmb();
205+ if (INTEL_GEN(gt->i915) < 6)
206+ intel_gtt_chipset_flush();
207+}
208+
209+int intel_gt_init_scratch(struct intel_gt *gt, unsigned int size)
210+{
211+ struct drm_i915_private *i915 = gt->i915;
212+ struct drm_i915_gem_object *obj;
213+ struct i915_vma *vma;
214+ int ret;
215+
216+ obj = i915_gem_object_create_stolen(i915, size);
217+ if (!obj)
218+ obj = i915_gem_object_create_internal(i915, size);
219+ if (IS_ERR(obj)) {
220+ DRM_ERROR("Failed to allocate scratch page\n");
221+ return PTR_ERR(obj);
222+ }
223+
224+ vma = i915_vma_instance(obj, &gt->ggtt->vm, NULL);
225+ if (IS_ERR(vma)) {
226+ ret = PTR_ERR(vma);
227+ goto err_unref;
228+ }
229+
230+ ret = i915_vma_pin(vma, 0, 0, PIN_GLOBAL | PIN_HIGH);
231+ if (ret)
232+ goto err_unref;
233+
234+ gt->scratch = vma;
235+ return 0;
236+
237+err_unref:
238+ i915_gem_object_put(obj);
239+ return ret;
240+}
241+
242+void intel_gt_fini_scratch(struct intel_gt *gt)
243+{
244+ i915_vma_unpin_and_release(&gt->scratch, 0);
245+}
246+
247+void intel_gt_cleanup_early(struct intel_gt *gt)
248+{
249+ intel_gt_fini_reset(gt);
250+}
drivers/gpu/drm/i915/gt/intel_gt.h+60−0
@@ -0,0 +1,60 @@
1+/* SPDX-License-Identifier: MIT */
2+/*
3+ * Copyright © 2019 Intel Corporation
4+ */
5+
6+#ifndef __INTEL_GT__
7+#define __INTEL_GT__
8+
9+#include "intel_engine_types.h"
10+#include "intel_gt_types.h"
11+#include "intel_reset.h"
12+
13+struct drm_i915_private;
14+
15+static inline struct intel_gt *uc_to_gt(struct intel_uc *uc)
16+{
17+ return container_of(uc, struct intel_gt, uc);
18+}
19+
20+static inline struct intel_gt *guc_to_gt(struct intel_guc *guc)
21+{
22+ return container_of(guc, struct intel_gt, uc.guc);
23+}
24+
25+static inline struct intel_gt *huc_to_gt(struct intel_huc *huc)
26+{
27+ return container_of(huc, struct intel_gt, uc.huc);
28+}
29+
30+void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915);
31+void intel_gt_init_hw(struct drm_i915_private *i915);
32+
33+void intel_gt_cleanup_early(struct intel_gt *gt);
34+
35+void intel_gt_check_and_clear_faults(struct intel_gt *gt);
36+void intel_gt_clear_error_registers(struct intel_gt *gt,
37+ intel_engine_mask_t engine_mask);
38+
39+void intel_gt_flush_ggtt_writes(struct intel_gt *gt);
40+void intel_gt_chipset_flush(struct intel_gt *gt);
41+
42+void intel_gt_init_hangcheck(struct intel_gt *gt);
43+
44+int intel_gt_init_scratch(struct intel_gt *gt, unsigned int size);
45+void intel_gt_fini_scratch(struct intel_gt *gt);
46+
47+static inline u32 intel_gt_scratch_offset(const struct intel_gt *gt,
48+ enum intel_gt_scratch_field field)
49+{
50+ return i915_ggtt_offset(gt->scratch) + field;
51+}
52+
53+static inline bool intel_gt_is_wedged(struct intel_gt *gt)
54+{
55+ return __intel_reset_failed(&gt->reset);
56+}
57+
58+void intel_gt_queue_hangcheck(struct intel_gt *gt);
59+
60+#endif /* __INTEL_GT_H__ */
drivers/gpu/drm/i915/gt/intel_gt_pm.c+33−29
@@ -5,7 +5,9 @@ · 4 unmodified lines
55 */
66
77 #include "i915_drv.h"
8+#include "i915_params.h"
89 #include "intel_engine_pm.h"
10+#include "intel_gt.h"
911 #include "intel_gt_pm.h"
1012 #include "intel_pm.h"
1113 #include "intel_wakeref.h"
@@ -17,8 +19,8 @@ static void pm_notify(struct drm_i915_private *i915, int state) · 5 unmodified lines
1719
1820 static int intel_gt_unpark(struct intel_wakeref *wf)
1921 {
20− struct drm_i915_private *i915 =
21− container_of(wf, typeof(*i915), gt.wakeref);
22+ struct intel_gt *gt = container_of(wf, typeof(*gt), wakeref);
23+ struct drm_i915_private *i915 = gt->i915;
2224
2325 GEM_TRACE("\n");
2426
@@ -33,8 +35,8 @@ static int intel_gt_unpark(struct intel_wakeref *wf) · 8 unmodified lines
3335 * Work around it by grabbing a GT IRQ power domain whilst there is any
3436 * GT activity, preventing any DC state transitions.
3537 */
36− i915->gt.awake = intel_display_power_get(i915, POWER_DOMAIN_GT_IRQ);
37− GEM_BUG_ON(!i915->gt.awake);
38+ gt->awake = intel_display_power_get(i915, POWER_DOMAIN_GT_IRQ);
39+ GEM_BUG_ON(!gt->awake);
3840
3941 intel_enable_gt_powersave(i915);
4042
@@ -44,16 +46,18 @@ static int intel_gt_unpark(struct intel_wakeref *wf) · 3 unmodified lines
4446
4547 i915_pmu_gt_unparked(i915);
4648
47− i915_queue_hangcheck(i915);
49+ intel_gt_queue_hangcheck(gt);
4850
4951 pm_notify(i915, INTEL_GT_UNPARK);
5052
5153 return 0;
5254 }
5355
54−void intel_gt_pm_get(struct drm_i915_private *i915)
56+void intel_gt_pm_get(struct intel_gt *gt)
5557 {
56− intel_wakeref_get(&i915->runtime_pm, &i915->gt.wakeref, intel_gt_unpark);
58+ struct intel_runtime_pm *rpm = &gt->i915->runtime_pm;
59+
60+ intel_wakeref_get(rpm, &gt->wakeref, intel_gt_unpark);
5761 }
5862
5963 static int intel_gt_park(struct intel_wakeref *wf)
@@ -76,28 +80,30 @@ static int intel_gt_park(struct intel_wakeref *wf) · 16 unmodified lines
7680 return 0;
7781 }
7882
79−void intel_gt_pm_put(struct drm_i915_private *i915)
83+void intel_gt_pm_put(struct intel_gt *gt)
8084 {
81− intel_wakeref_put(&i915->runtime_pm, &i915->gt.wakeref, intel_gt_park);
85+ struct intel_runtime_pm *rpm = &gt->i915->runtime_pm;
86+
87+ intel_wakeref_put(rpm, &gt->wakeref, intel_gt_park);
8288 }
8389
84−void intel_gt_pm_init(struct drm_i915_private *i915)
90+void intel_gt_pm_init_early(struct intel_gt *gt)
8591 {
86− intel_wakeref_init(&i915->gt.wakeref);
87− BLOCKING_INIT_NOTIFIER_HEAD(&i915->gt.pm_notifications);
92+ intel_wakeref_init(&gt->wakeref);
93+ BLOCKING_INIT_NOTIFIER_HEAD(&gt->pm_notifications);
8894 }
8995
90−static bool reset_engines(struct drm_i915_private *i915)
96+static bool reset_engines(struct intel_gt *gt)
9197 {
92− if (INTEL_INFO(i915)->gpu_reset_clobbers_display)
98+ if (INTEL_INFO(gt->i915)->gpu_reset_clobbers_display)
9399 return false;
94100
95− return intel_gpu_reset(i915, ALL_ENGINES) == 0;
101+ return __intel_gt_reset(gt, ALL_ENGINES) == 0;
96102 }
97103
98104 /**
99105 * intel_gt_sanitize: called after the GPU has lost power
100− * @i915: the i915 device
106+ * @gt: the i915 GT container
101107 * @force: ignore a failed reset and sanitize engine state anyway
102108 *
103109 * Anytime we reset the GPU, either with an explicit GPU reset or through a
@@ -105,21 +111,23 @@ static bool reset_engines(struct drm_i915_private *i915) · 1 unmodified line
105111 * to match. Note that calling intel_gt_sanitize() if the GPU has not
106112 * been reset results in much confusion!
107113 */
108−void intel_gt_sanitize(struct drm_i915_private *i915, bool force)
114+void intel_gt_sanitize(struct intel_gt *gt, bool force)
109115 {
110116 struct intel_engine_cs *engine;
111117 enum intel_engine_id id;
112118
113119 GEM_TRACE("\n");
114120
115− if (!reset_engines(i915) && !force)
121+ intel_uc_sanitize(&gt->uc);
122+
123+ if (!reset_engines(gt) && !force)
116124 return;
117125
118− for_each_engine(engine, i915, id)
119− intel_engine_reset(engine, false);
126+ for_each_engine(engine, gt->i915, id)
127+ __intel_engine_reset(engine, false);
120128 }
121129
122−int intel_gt_resume(struct drm_i915_private *i915)
130+int intel_gt_resume(struct intel_gt *gt)
123131 {
124132 struct intel_engine_cs *engine;
125133 enum intel_engine_id id;
@@ -131,8 +139,8 @@ int intel_gt_resume(struct drm_i915_private *i915) · 5 unmodified lines
131139 * Only the kernel contexts should remain pinned over suspend,
132140 * allowing us to fixup the user contexts on their first pin.
133141 */
134− intel_gt_pm_get(i915);
135− for_each_engine(engine, i915, id) {
142+ intel_gt_pm_get(gt);
143+ for_each_engine(engine, gt->i915, id) {
136144 struct intel_context *ce;
137145
138146 intel_engine_pm_get(engine);
@@ -141,22 +149,18 @@ int intel_gt_resume(struct drm_i915_private *i915) · 2 unmodified lines
141149 if (ce)
142150 ce->ops->reset(ce);
143151
144− ce = engine->preempt_context;
145− if (ce)
146− ce->ops->reset(ce);
147−
148152 engine->serial++; /* kernel context lost */
149153 err = engine->resume(engine);
150154
151155 intel_engine_pm_put(engine);
152156 if (err) {
153− dev_err(i915->drm.dev,
157+ dev_err(gt->i915->drm.dev,
154158 "Failed to restart %s (%d)\n",
155159 engine->name, err);
156160 break;
157161 }
158162 }
159− intel_gt_pm_put(i915);
163+ intel_gt_pm_put(gt);
160164
161165 return err;
162166 }
drivers/gpu/drm/i915/gt/intel_gt_pm.h+6−6
@@ -9,19 +9,19 @@ · 8 unmodified lines
99
1010 #include <linux/types.h>
1111
12−struct drm_i915_private;
12+struct intel_gt;
1313
1414 enum {
1515 INTEL_GT_UNPARK,
1616 INTEL_GT_PARK,
1717 };
1818
19−void intel_gt_pm_get(struct drm_i915_private *i915);
20−void intel_gt_pm_put(struct drm_i915_private *i915);
19+void intel_gt_pm_get(struct intel_gt *gt);
20+void intel_gt_pm_put(struct intel_gt *gt);
2121
22−void intel_gt_pm_init(struct drm_i915_private *i915);
22+void intel_gt_pm_init_early(struct intel_gt *gt);
2323
24−void intel_gt_sanitize(struct drm_i915_private *i915, bool force);
25−int intel_gt_resume(struct drm_i915_private *i915);
24+void intel_gt_sanitize(struct intel_gt *gt, bool force);
25+int intel_gt_resume(struct intel_gt *gt);
2626
2727 #endif /* INTEL_GT_PM_H */
drivers/gpu/drm/i915/gt/intel_gt_types.h+96−0
@@ -0,0 +1,96 @@
1+/* SPDX-License-Identifier: MIT */
2+/*
3+ * Copyright © 2019 Intel Corporation
4+ */
5+
6+#ifndef __INTEL_GT_TYPES__
7+#define __INTEL_GT_TYPES__
8+
9+#include <linux/ktime.h>
10+#include <linux/list.h>
11+#include <linux/mutex.h>
12+#include <linux/notifier.h>
13+#include <linux/spinlock.h>
14+#include <linux/types.h>
15+
16+#include "uc/intel_uc.h"
17+
18+#include "i915_vma.h"
19+#include "intel_reset_types.h"
20+#include "intel_wakeref.h"
21+
22+struct drm_i915_private;
23+struct i915_ggtt;
24+struct intel_uncore;
25+
26+struct intel_hangcheck {
27+ /* For hangcheck timer */
28+#define DRM_I915_HANGCHECK_PERIOD 1500 /* in ms */
29+#define DRM_I915_HANGCHECK_JIFFIES msecs_to_jiffies(DRM_I915_HANGCHECK_PERIOD)
30+
31+ struct delayed_work work;
32+};
33+
34+struct intel_gt {
35+ struct drm_i915_private *i915;
36+ struct intel_uncore *uncore;
37+ struct i915_ggtt *ggtt;
38+
39+ struct intel_uc uc;
40+
41+ struct intel_gt_timelines {
42+ struct mutex mutex; /* protects list */
43+ struct list_head active_list;
44+
45+ /* Pack multiple timelines' seqnos into the same page */
46+ spinlock_t hwsp_lock;
47+ struct list_head hwsp_free_list;
48+ } timelines;
49+
50+ struct list_head active_rings;
51+
52+ struct intel_wakeref wakeref;
53+
54+ struct list_head closed_vma;
55+ spinlock_t closed_lock; /* guards the list of closed_vma */
56+
57+ struct intel_hangcheck hangcheck;
58+ struct intel_reset reset;
59+
60+ /**
61+ * Is the GPU currently considered idle, or busy executing
62+ * userspace requests? Whilst idle, we allow runtime power
63+ * management to power down the hardware and display clocks.
64+ * In order to reduce the effect on performance, there
65+ * is a slight delay before we do so.
66+ */
67+ intel_wakeref_t awake;
68+
69+ struct blocking_notifier_head pm_notifications;
70+
71+ ktime_t last_init_time;
72+
73+ struct i915_vma *scratch;
74+
75+ u32 pm_imr;
76+ u32 pm_ier;
77+
78+ u32 pm_guc_events;
79+};
80+
81+enum intel_gt_scratch_field {
82+ /* 8 bytes */
83+ INTEL_GT_SCRATCH_FIELD_DEFAULT = 0,
84+
85+ /* 8 bytes */
86+ INTEL_GT_SCRATCH_FIELD_CLEAR_SLM_WA = 128,
87+
88+ /* 8 bytes */
89+ INTEL_GT_SCRATCH_FIELD_RENDER_FLUSH = 128,
90+
91+ /* 8 bytes */
92+ INTEL_GT_SCRATCH_FIELD_COHERENTL3_WA = 256,
93+
94+};
95+
96+#endif /* __INTEL_GT_TYPES_H__ */
drivers/gpu/drm/i915/gt/intel_hangcheck.c+42−29
@@ -22,8 +22,10 @@ · 21 unmodified lines
2222 *
2323 */
2424
25−#include "intel_reset.h"
2625 #include "i915_drv.h"
26+#include "intel_engine.h"
27+#include "intel_gt.h"
28+#include "intel_reset.h"
2729
2830 struct hangcheck {
2931 u64 acthd;
@@ -57,9 +59,6 @@ static bool subunits_stuck(struct intel_engine_cs *engine) · 27 unmodified lines
5759 int slice;
5860 int subslice;
5961
60− if (engine->id != RCS0)
61− return true;
62−
6362 intel_engine_get_instdone(engine, &instdone);
6463
6564 /* There might be unstable subunit states even when
@@ -103,7 +102,6 @@ head_stuck(struct intel_engine_cs *engine, u64 acthd) · 37 unmodified lines
103102 static enum intel_engine_hangcheck_action
104103 engine_stuck(struct intel_engine_cs *engine, u64 acthd)
105104 {
106− struct drm_i915_private *dev_priv = engine->i915;
107105 enum intel_engine_hangcheck_action ha;
108106 u32 tmp;
109107
@@ -111,7 +109,7 @@ engine_stuck(struct intel_engine_cs *engine, u64 acthd) · 1 unmodified line
111109 if (ha != ENGINE_DEAD)
112110 return ha;
113111
114− if (IS_GEN(dev_priv, 2))
112+ if (IS_GEN(engine->i915, 2))
115113 return ENGINE_DEAD;
116114
117115 /* Is the chip hanging on a WAIT_FOR_EVENT?
@@ -121,8 +119,8 @@ engine_stuck(struct intel_engine_cs *engine, u64 acthd) · 3 unmodified lines
121119 */
122120 tmp = ENGINE_READ(engine, RING_CTL);
123121 if (tmp & RING_WAIT) {
124− i915_handle_error(dev_priv, engine->mask, 0,
125− "stuck wait on %s", engine->name);
122+ intel_gt_handle_error(engine->gt, engine->mask, 0,
123+ "stuck wait on %s", engine->name);
126124 ENGINE_WRITE(engine, RING_CTL, tmp);
127125 return ENGINE_WAIT_KICK;
128126 }
@@ -222,7 +220,7 @@ static void hangcheck_accumulate_sample(struct intel_engine_cs *engine, · 93 unmodified lines
222220 I915_ENGINE_WEDGED_TIMEOUT);
223221 }
224222
225−static void hangcheck_declare_hang(struct drm_i915_private *i915,
223+static void hangcheck_declare_hang(struct intel_gt *gt,
226224 intel_engine_mask_t hung,
227225 intel_engine_mask_t stuck)
228226 {
@@ -238,12 +236,12 @@ static void hangcheck_declare_hang(struct drm_i915_private *i915, · 9 unmodified lines
238236 hung &= ~stuck;
239237 len = scnprintf(msg, sizeof(msg),
240238 "%s on ", stuck == hung ? "no progress" : "hang");
241− for_each_engine_masked(engine, i915, hung, tmp)
239+ for_each_engine_masked(engine, gt->i915, hung, tmp)
242240 len += scnprintf(msg + len, sizeof(msg) - len,
243241 "%s, ", engine->name);
244242 msg[len-2] = '\0';
245243
246− return i915_handle_error(i915, hung, I915_ERROR_CAPTURE, "%s", msg);
244+ return intel_gt_handle_error(gt, hung, I915_ERROR_CAPTURE, "%s", msg);
247245 }
248246
249247 /*
@@ -254,11 +252,10 @@ static void hangcheck_declare_hang(struct drm_i915_private *i915, · 4 unmodified lines
254252 * we kick the ring. If we see no progress on three subsequent calls
255253 * we assume chip is wedged and try to fix it by resetting the chip.
256254 */
257−static void i915_hangcheck_elapsed(struct work_struct *work)
255+static void hangcheck_elapsed(struct work_struct *work)
258256 {
259− struct drm_i915_private *dev_priv =
260− container_of(work, typeof(*dev_priv),
261− gpu_error.hangcheck_work.work);
257+ struct intel_gt *gt =
258+ container_of(work, typeof(*gt), hangcheck.work.work);
262259 intel_engine_mask_t hung = 0, stuck = 0, wedged = 0;
263260 struct intel_engine_cs *engine;
264261 enum intel_engine_id id;
@@ -267,13 +264,13 @@ static void i915_hangcheck_elapsed(struct work_struct *work) · 2 unmodified lines
267264 if (!i915_modparams.enable_hangcheck)
268265 return;
269266
270− if (!READ_ONCE(dev_priv->gt.awake))
267+ if (!READ_ONCE(gt->awake))
271268 return;
272269
273− if (i915_terminally_wedged(dev_priv))
270+ if (intel_gt_is_wedged(gt))
274271 return;
275272
276− wakeref = intel_runtime_pm_get_if_in_use(&dev_priv->runtime_pm);
273+ wakeref = intel_runtime_pm_get_if_in_use(&gt->i915->runtime_pm);
277274 if (!wakeref)
278275 return;
279276
@@ -281,9 +278,9 @@ static void i915_hangcheck_elapsed(struct work_struct *work) · 1 unmodified line
281278 * periodically arm the mmio checker to see if we are triggering
282279 * any invalid access.
283280 */
284− intel_uncore_arm_unclaimed_mmio_detection(&dev_priv->uncore);
281+ intel_uncore_arm_unclaimed_mmio_detection(gt->uncore);
285282
286− for_each_engine(engine, dev_priv, id) {
283+ for_each_engine(engine, gt->i915, id) {
287284 struct hangcheck hc;
288285
289286 intel_engine_signal_breadcrumbs(engine);
@@ -305,7 +302,7 @@ static void i915_hangcheck_elapsed(struct work_struct *work) · 15 unmodified lines
305302 if (GEM_SHOW_DEBUG() && (hung | stuck)) {
306303 struct drm_printer p = drm_debug_printer("hangcheck");
307304
308− for_each_engine(engine, dev_priv, id) {
305+ for_each_engine(engine, gt->i915, id) {
309306 if (intel_engine_is_idle(engine))
310307 continue;
311308
@@ -314,20 +311,37 @@ static void i915_hangcheck_elapsed(struct work_struct *work) · 2 unmodified lines
314311 }
315312
316313 if (wedged) {
317− dev_err(dev_priv->drm.dev,
314+ dev_err(gt->i915->drm.dev,
318315 "GPU recovery timed out,"
319316 " cancelling all in-flight rendering.\n");
320317 GEM_TRACE_DUMP();
321− i915_gem_set_wedged(dev_priv);
318+ intel_gt_set_wedged(gt);
322319 }
323320
324321 if (hung)
325− hangcheck_declare_hang(dev_priv, hung, stuck);
322+ hangcheck_declare_hang(gt, hung, stuck);
326323
327− intel_runtime_pm_put(&dev_priv->runtime_pm, wakeref);
324+ intel_runtime_pm_put(&gt->i915->runtime_pm, wakeref);
328325
329326 /* Reset timer in case GPU hangs without another request being added */
330− i915_queue_hangcheck(dev_priv);
327+ intel_gt_queue_hangcheck(gt);
328+}
329+
330+void intel_gt_queue_hangcheck(struct intel_gt *gt)
331+{
332+ unsigned long delay;
333+
334+ if (unlikely(!i915_modparams.enable_hangcheck))
335+ return;
336+
337+ /*
338+ * Don't continually defer the hangcheck so that it is always run at
339+ * least once after work has been scheduled on any ring. Otherwise,
340+ * we will ignore a hung ring if a second ring is kept busy.
341+ */
342+
343+ delay = round_jiffies_up_relative(DRM_I915_HANGCHECK_JIFFIES);
344+ queue_delayed_work(system_long_wq, &gt->hangcheck.work, delay);
331345 }
332346
333347 void intel_engine_init_hangcheck(struct intel_engine_cs *engine)
@@ -336,10 +350,9 @@ void intel_engine_init_hangcheck(struct intel_engine_cs *engine) · 2 unmodified lines
336350 engine->hangcheck.action_timestamp = jiffies;
337351 }
338352
339−void intel_hangcheck_init(struct drm_i915_private *i915)
353+void intel_gt_init_hangcheck(struct intel_gt *gt)
340354 {
341− INIT_DELAYED_WORK(&i915->gpu_error.hangcheck_work,
342− i915_hangcheck_elapsed);
355+ INIT_DELAYED_WORK(&gt->hangcheck.work, hangcheck_elapsed);
343356 }
344357
345358 #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
drivers/gpu/drm/i915/gt/intel_lrc.c+580−437
@@ -136,9 +136,9 @@ · 135 unmodified lines
136136 #include "gem/i915_gem_context.h"
137137
138138 #include "i915_drv.h"
139−#include "i915_gem_render_state.h"
140139 #include "i915_vgpu.h"
141140 #include "intel_engine_pm.h"
141+#include "intel_gt.h"
142142 #include "intel_lrc_reg.h"
143143 #include "intel_mocs.h"
144144 #include "intel_reset.h"
@@ -161,6 +161,8 @@ · 16 unmodified lines
161161 #define GEN8_CTX_STATUS_COMPLETED_MASK \
162162 (GEN8_CTX_STATUS_COMPLETE | GEN8_CTX_STATUS_PREEMPTED)
163163
164+#define CTX_DESC_FORCE_RESTORE BIT_ULL(2)
165+
164166 /* Typical size of the average request (2 pipecontrols and a MI_BB) */
165167 #define EXECLISTS_REQUEST_SIZE 64 /* bytes */
166168 #define WA_TAIL_DWORDS 2
@@ -221,6 +223,26 @@ static void execlists_init_reg_state(u32 *reg_state, · 54 unmodified lines
221223 struct intel_engine_cs *engine,
222224 struct intel_ring *ring);
223225
226+static inline u32 intel_hws_preempt_address(struct intel_engine_cs *engine)
227+{
228+ return (i915_ggtt_offset(engine->status_page.vma) +
229+ I915_GEM_HWS_PREEMPT_ADDR);
230+}
231+
232+static inline void
233+ring_set_paused(const struct intel_engine_cs *engine, int state)
234+{
235+ /*
236+ * We inspect HWS_PREEMPT with a semaphore inside
237+ * engine->emit_fini_breadcrumb. If the dword is true,
238+ * the ring is paused as the semaphore will busywait
239+ * until the dword is false.
240+ */
241+ engine->status_page.addr[I915_GEM_HWS_PREEMPT] = state;
242+ if (state)
243+ wmb();
244+}
245+
224246 static inline struct i915_priolist *to_priolist(struct rb_node *rb)
225247 {
226248 return rb_entry(rb, struct i915_priolist, node);
@@ -235,6 +257,17 @@ static int effective_prio(const struct i915_request *rq) · 8 unmodified lines
235257 {
236258 int prio = rq_prio(rq);
237259
260+ /*
261+ * If this request is special and must not be interrupted at any
262+ * cost, so be it. Note we are only checking the most recent request
263+ * in the context and so may be masking an earlier vip request. It
264+ * is hoped that under the conditions where nopreempt is used, this
265+ * will not matter (i.e. all requests to that context will be
266+ * nopreempt for as long as desired).
267+ */
268+ if (i915_request_has_nopreempt(rq))
269+ prio = I915_PRIORITY_UNPREEMPTABLE;
270+
238271 /*
239272 * On unwinding the active request, we give it a priority bump
240273 * if it has completed waiting on any semaphore. If we know that
@@ -245,6 +278,7 @@ static int effective_prio(const struct i915_request *rq) · 4 unmodified lines
245278 prio |= I915_PRIORITY_NOSEMAPHORE;
246279
247280 /* Restrict mere WAIT boosts from triggering preemption */
281+ BUILD_BUG_ON(__NO_PREEMPTION & ~I915_PRIORITY_MASK); /* only internal */
248282 return prio | __NO_PREEMPTION;
249283 }
250284
@@ -271,10 +305,7 @@ static inline bool need_preempt(const struct intel_engine_cs *engine, · 20 unmodified lines
271305 {
272306 int last_prio;
273307
274− if (!engine->preempt_context)
275− return false;
276−
277− if (i915_request_completed(rq))
308+ if (!intel_engine_has_semaphores(engine))
278309 return false;
279310
280311 /*
@@ -338,9 +369,6 @@ __maybe_unused static inline bool · 57 unmodified lines
338369 assert_priority_queue(const struct i915_request *prev,
339370 const struct i915_request *next)
340371 {
341− const struct intel_engine_execlists *execlists =
342− &prev->engine->execlists;
343−
344372 /*
345373 * Without preemption, the prev may refer to the still active element
346374 * which we refuse to let go.
@@ -348,7 +376,7 @@ assert_priority_queue(const struct i915_request *prev, · 1 unmodified line
348376 * Even with preemption, there are times when we think it is better not
349377 * to preempt and leave an ostensibly lower priority request in flight.
350378 */
351− if (port_request(execlists->port) == prev)
379+ if (i915_request_is_active(prev))
352380 return true;
353381
354382 return rq_prio(prev) >= rq_prio(next);
@@ -442,13 +470,11 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine) · 87 unmodified lines
442470 struct intel_engine_cs *owner;
443471
444472 if (i915_request_completed(rq))
445− break;
473+ continue; /* XXX */
446474
447475 __i915_request_unsubmit(rq);
448476 unwind_wa_tail(rq);
449477
450− GEM_BUG_ON(rq->hw_context->inflight);
451−
452478 /*
453479 * Push the request back into the queue for later resubmission.
454480 * If this request is not native to this physical engine (i.e.
@@ -468,6 +494,19 @@ __unwind_incomplete_requests(struct intel_engine_cs *engine) · 13 unmodified lines
468494 list_move(&rq->sched.link, pl);
469495 active = rq;
470496 } else {
497+ /*
498+ * Decouple the virtual breadcrumb before moving it
499+ * back to the virtual engine -- we don't want the
500+ * request to complete in the background and try
501+ * and cancel the breadcrumb on the virtual engine
502+ * (instead of the old engine where it is linked)!
503+ */
504+ if (test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
505+ &rq->fence.flags)) {
506+ spin_lock(&rq->lock);
507+ i915_request_cancel_breadcrumb(rq);
508+ spin_unlock(&rq->lock);
509+ }
471510 rq->engine = owner;
472511 owner->submit_request(rq);
473512 active = NULL;
@@ -500,32 +539,32 @@ execlists_context_status_change(struct i915_request *rq, unsigned long status) · 26 unmodified lines
500539 status, rq);
501540 }
502541
503−inline void
504−execlists_user_begin(struct intel_engine_execlists *execlists,
505− const struct execlist_port *port)
542+static inline struct i915_request *
543+execlists_schedule_in(struct i915_request *rq, int idx)
506544 {
507− execlists_set_active_once(execlists, EXECLISTS_ACTIVE_USER);
508−}
545+ struct intel_context *ce = rq->hw_context;
546+ int count;
509547
510−inline void
511−execlists_user_end(struct intel_engine_execlists *execlists)
512−{
513− execlists_clear_active(execlists, EXECLISTS_ACTIVE_USER);
514−}
548+ trace_i915_request_in(rq, idx);
515549
516−static inline void
517−execlists_context_schedule_in(struct i915_request *rq)
518−{
519− GEM_BUG_ON(rq->hw_context->inflight);
550+ count = intel_context_inflight_count(ce);
551+ if (!count) {
552+ intel_context_get(ce);
553+ ce->inflight = rq->engine;
554+
555+ execlists_context_status_change(rq, INTEL_CONTEXT_SCHEDULE_IN);
556+ intel_engine_context_in(ce->inflight);
557+ }
558+
559+ intel_context_inflight_inc(ce);
560+ GEM_BUG_ON(intel_context_inflight(ce) != rq->engine);
520561
521− execlists_context_status_change(rq, INTEL_CONTEXT_SCHEDULE_IN);
522− intel_engine_context_in(rq->engine);
523− rq->hw_context->inflight = rq->engine;
562+ return i915_request_get(rq);
524563 }
525564
526−static void kick_siblings(struct i915_request *rq)
565+static void kick_siblings(struct i915_request *rq, struct intel_context *ce)
527566 {
528− struct virtual_engine *ve = to_virtual_engine(rq->hw_context->engine);
567+ struct virtual_engine *ve = container_of(ce, typeof(*ve), context);
529568 struct i915_request *next = READ_ONCE(ve->request);
530569
531570 if (next && next->execution_mask & ~rq->execution_mask)
@@ -533,29 +572,42 @@ static void kick_siblings(struct i915_request *rq) · 1 unmodified line
533572 }
534573
535574 static inline void
536−execlists_context_schedule_out(struct i915_request *rq, unsigned long status)
575+execlists_schedule_out(struct i915_request *rq)
537576 {
538− rq->hw_context->inflight = NULL;
539− intel_engine_context_out(rq->engine);
540− execlists_context_status_change(rq, status);
577+ struct intel_context *ce = rq->hw_context;
578+
579+ GEM_BUG_ON(!intel_context_inflight_count(ce));
580+
541581 trace_i915_request_out(rq);
542582
543− /*
544− * If this is part of a virtual engine, its next request may have
545− * been blocked waiting for access to the active context. We have
546− * to kick all the siblings again in case we need to switch (e.g.
547− * the next request is not runnable on this engine). Hopefully,
548− * we will already have submitted the next request before the
549− * tasklet runs and do not need to rebuild each virtual tree
550− * and kick everyone again.
551− */
552− if (rq->engine != rq->hw_context->engine)
553− kick_siblings(rq);
583+ intel_context_inflight_dec(ce);
584+ if (!intel_context_inflight_count(ce)) {
585+ intel_engine_context_out(ce->inflight);
586+ execlists_context_status_change(rq, INTEL_CONTEXT_SCHEDULE_OUT);
587+
588+ /*
589+ * If this is part of a virtual engine, its next request may
590+ * have been blocked waiting for access to the active context.
591+ * We have to kick all the siblings again in case we need to
592+ * switch (e.g. the next request is not runnable on this
593+ * engine). Hopefully, we will already have submitted the next
594+ * request before the tasklet runs and do not need to rebuild
595+ * each virtual tree and kick everyone again.
596+ */
597+ ce->inflight = NULL;
598+ if (rq->engine != ce->engine)
599+ kick_siblings(rq, ce);
600+
601+ intel_context_put(ce);
602+ }
603+
604+ i915_request_put(rq);
554605 }
555606
556−static u64 execlists_update_context(struct i915_request *rq)
607+static u64 execlists_update_context(const struct i915_request *rq)
557608 {
558609 struct intel_context *ce = rq->hw_context;
610+ u64 desc;
559611
560612 ce->lrc_reg_state[CTX_RING_TAIL + 1] =
561613 intel_ring_set_tail(rq->ring, rq->tail);
@@ -576,7 +628,11 @@ static u64 execlists_update_context(struct i915_request *rq) · 14 unmodified lines
576628 * wmb).
577629 */
578630 mb();
579− return ce->lrc_desc;
631+
632+ desc = ce->lrc_desc;
633+ ce->lrc_desc &= ~CTX_DESC_FORCE_RESTORE;
634+
635+ return desc;
580636 }
581637
582638 static inline void write_desc(struct intel_engine_execlists *execlists, u64 desc, u32 port)
@@ -590,12 +646,62 @@ static inline void write_desc(struct intel_engine_execlists *execlists, u64 desc · 7 unmodified lines
590646 }
591647 }
592648
649+static __maybe_unused void
650+trace_ports(const struct intel_engine_execlists *execlists,
651+ const char *msg,
652+ struct i915_request * const *ports)
653+{
654+ const struct intel_engine_cs *engine =
655+ container_of(execlists, typeof(*engine), execlists);
656+
657+ GEM_TRACE("%s: %s { %llx:%lld%s, %llx:%lld }\n",
658+ engine->name, msg,
659+ ports[0]->fence.context,
660+ ports[0]->fence.seqno,
661+ i915_request_completed(ports[0]) ? "!" :
662+ i915_request_started(ports[0]) ? "*" :
663+ "",
664+ ports[1] ? ports[1]->fence.context : 0,
665+ ports[1] ? ports[1]->fence.seqno : 0);
666+}
667+
668+static __maybe_unused bool
669+assert_pending_valid(const struct intel_engine_execlists *execlists,
670+ const char *msg)
671+{
672+ struct i915_request * const *port, *rq;
673+ struct intel_context *ce = NULL;
674+
675+ trace_ports(execlists, msg, execlists->pending);
676+
677+ if (execlists->pending[execlists_num_ports(execlists)])
678+ return false;
679+
680+ for (port = execlists->pending; (rq = *port); port++) {
681+ if (ce == rq->hw_context)
682+ return false;
683+
684+ ce = rq->hw_context;
685+ if (i915_request_completed(rq))
686+ continue;
687+
688+ if (i915_active_is_idle(&ce->active))
689+ return false;
690+
691+ if (!i915_vma_is_pinned(ce->state))
692+ return false;
693+ }
694+
695+ return ce;
696+}
697+
593698 static void execlists_submit_ports(struct intel_engine_cs *engine)
594699 {
595700 struct intel_engine_execlists *execlists = &engine->execlists;
596− struct execlist_port *port = execlists->port;
597701 unsigned int n;
598702
703+ GEM_BUG_ON(!assert_pending_valid(execlists, "submit"));
704+
599705 /*
600706 * We can skip acquiring intel_runtime_pm_get() here as it was taken
601707 * on our behalf by the request (see i915_gem_mark_busy()) and it will
@@ -604,7 +710,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine) · 2 unmodified lines
604710 * that all ELSP are drained i.e. we have processed the CSB,
605711 * before allowing ourselves to idle and calling intel_runtime_pm_put().
606712 */
607− GEM_BUG_ON(!intel_wakeref_active(&engine->wakeref));
713+ GEM_BUG_ON(!intel_engine_pm_is_awake(engine));
608714
609715 /*
610716 * ELSQ note: the submit queue is not cleared after being submitted
@@ -613,38 +719,16 @@ static void execlists_submit_ports(struct intel_engine_cs *engine) · 2 unmodified lines
613719 * of elsq entries, keep this in mind before changing the loop below.
614720 */
615721 for (n = execlists_num_ports(execlists); n--; ) {
616− struct i915_request *rq;
617− unsigned int count;
618− u64 desc;
722+ struct i915_request *rq = execlists->pending[n];
619723
620− rq = port_unpack(&port[n], &count);
621− if (rq) {
622− GEM_BUG_ON(count > !n);
623− if (!count++)
624− execlists_context_schedule_in(rq);
625− port_set(&port[n], port_pack(rq, count));
626− desc = execlists_update_context(rq);
627− GEM_DEBUG_EXEC(port[n].context_id = upper_32_bits(desc));
628−
629− GEM_TRACE("%s in[%d]: ctx=%d.%d, fence %llx:%lld (current %d), prio=%d\n",
630− engine->name, n,
631− port[n].context_id, count,
632− rq->fence.context, rq->fence.seqno,
633− hwsp_seqno(rq),
634− rq_prio(rq));
635− } else {
636− GEM_BUG_ON(!n);
637− desc = 0;
638− }
639−
640− write_desc(execlists, desc, n);
724+ write_desc(execlists,
725+ rq ? execlists_update_context(rq) : 0,
726+ n);
641727 }
642728
643729 /* we need to manually load the submit queue */
644730 if (execlists->ctrl_reg)
645731 writel(EL_CTRL_LOAD, execlists->ctrl_reg);
646−
647− execlists_clear_active(execlists, EXECLISTS_ACTIVE_HWACK);
648732 }
649733
650734 static bool ctx_single_port_submission(const struct intel_context *ce)
@@ -668,6 +752,7 @@ static bool can_merge_ctx(const struct intel_context *prev, · 17 unmodified lines
668752 static bool can_merge_rq(const struct i915_request *prev,
669753 const struct i915_request *next)
670754 {
755+ GEM_BUG_ON(prev == next);
671756 GEM_BUG_ON(!assert_priority_queue(prev, next));
672757
673758 if (!can_merge_ctx(prev->hw_context, next->hw_context))
@@ -676,58 +761,6 @@ static bool can_merge_rq(const struct i915_request *prev, · 2 unmodified lines
676761 return true;
677762 }
678763
679−static void port_assign(struct execlist_port *port, struct i915_request *rq)
680−{
681− GEM_BUG_ON(rq == port_request(port));
682−
683− if (port_isset(port))
684− i915_request_put(port_request(port));
685−
686− port_set(port, port_pack(i915_request_get(rq), port_count(port)));
687−}
688−
689−static void inject_preempt_context(struct intel_engine_cs *engine)
690−{
691− struct intel_engine_execlists *execlists = &engine->execlists;
692− struct intel_context *ce = engine->preempt_context;
693− unsigned int n;
694−
695− GEM_BUG_ON(execlists->preempt_complete_status !=
696− upper_32_bits(ce->lrc_desc));
697−
698− /*
699− * Switch to our empty preempt context so
700− * the state of the GPU is known (idle).
701− */
702− GEM_TRACE("%s\n", engine->name);
703− for (n = execlists_num_ports(execlists); --n; )
704− write_desc(execlists, 0, n);
705−
706− write_desc(execlists, ce->lrc_desc, n);
707−
708− /* we need to manually load the submit queue */
709− if (execlists->ctrl_reg)
710− writel(EL_CTRL_LOAD, execlists->ctrl_reg);
711−
712− execlists_clear_active(execlists, EXECLISTS_ACTIVE_HWACK);
713− execlists_set_active(execlists, EXECLISTS_ACTIVE_PREEMPT);
714−
715− (void)I915_SELFTEST_ONLY(execlists->preempt_hang.count++);
716−}
717−
718−static void complete_preempt_context(struct intel_engine_execlists *execlists)
719−{
720− GEM_BUG_ON(!execlists_is_active(execlists, EXECLISTS_ACTIVE_PREEMPT));
721−
722− if (inject_preempt_hang(execlists))
723− return;
724−
725− execlists_cancel_port_requests(execlists);
726− __unwind_incomplete_requests(container_of(execlists,
727− struct intel_engine_cs,
728− execlists));
729−}
730−
731764 static void virtual_update_register_offsets(u32 *regs,
732765 struct intel_engine_cs *engine)
733766 {
@@ -792,7 +825,7 @@ static bool virtual_matches(const struct virtual_engine *ve, · 58 unmodified lines
792825 * we reuse the register offsets). This is a very small
793826 * hystersis on the greedy seelction algorithm.
794827 */
795− inflight = READ_ONCE(ve->context.inflight);
828+ inflight = intel_context_inflight(&ve->context);
796829 if (inflight && inflight != engine)
797830 return false;
798831
@@ -815,13 +848,108 @@ static void virtual_xfer_breadcrumbs(struct virtual_engine *ve, · 16 unmodified lines
815848 spin_unlock(&old->breadcrumbs.irq_lock);
816849 }
817850
851+static struct i915_request *
852+last_active(const struct intel_engine_execlists *execlists)
853+{
854+ struct i915_request * const *last = execlists->active;
855+
856+ while (*last && i915_request_completed(*last))
857+ last++;
858+
859+ return *last;
860+}
861+
862+static void defer_request(struct i915_request *rq, struct list_head * const pl)
863+{
864+ LIST_HEAD(list);
865+
866+ /*
867+ * We want to move the interrupted request to the back of
868+ * the round-robin list (i.e. its priority level), but
869+ * in doing so, we must then move all requests that were in
870+ * flight and were waiting for the interrupted request to
871+ * be run after it again.
872+ */
873+ do {
874+ struct i915_dependency *p;
875+
876+ GEM_BUG_ON(i915_request_is_active(rq));
877+ list_move_tail(&rq->sched.link, pl);
878+
879+ list_for_each_entry(p, &rq->sched.waiters_list, wait_link) {
880+ struct i915_request *w =
881+ container_of(p->waiter, typeof(*w), sched);
882+
883+ /* Leave semaphores spinning on the other engines */
884+ if (w->engine != rq->engine)
885+ continue;
886+
887+ /* No waiter should start before its signaler */
888+ GEM_BUG_ON(i915_request_started(w) &&
889+ !i915_request_completed(rq));
890+
891+ GEM_BUG_ON(i915_request_is_active(w));
892+ if (list_empty(&w->sched.link))
893+ continue; /* Not yet submitted; unready */
894+
895+ if (rq_prio(w) < rq_prio(rq))
896+ continue;
897+
898+ GEM_BUG_ON(rq_prio(w) > rq_prio(rq));
899+ list_move_tail(&w->sched.link, &list);
900+ }
901+
902+ rq = list_first_entry_or_null(&list, typeof(*rq), sched.link);
903+ } while (rq);
904+}
905+
906+static void defer_active(struct intel_engine_cs *engine)
907+{
908+ struct i915_request *rq;
909+
910+ rq = __unwind_incomplete_requests(engine);
911+ if (!rq)
912+ return;
913+
914+ defer_request(rq, i915_sched_lookup_priolist(engine, rq_prio(rq)));
915+}
916+
917+static bool
918+need_timeslice(struct intel_engine_cs *engine, const struct i915_request *rq)
919+{
920+ int hint;
921+
922+ if (!intel_engine_has_semaphores(engine))
923+ return false;
924+
925+ if (list_is_last(&rq->sched.link, &engine->active.requests))
926+ return false;
927+
928+ hint = max(rq_prio(list_next_entry(rq, sched.link)),
929+ engine->execlists.queue_priority_hint);
930+
931+ return hint >= effective_prio(rq);
932+}
933+
934+static bool
935+enable_timeslice(struct intel_engine_cs *engine)
936+{
937+ struct i915_request *last = last_active(&engine->execlists);
938+
939+ return last && need_timeslice(engine, last);
940+}
941+
942+static void record_preemption(struct intel_engine_execlists *execlists)
943+{
944+ (void)I915_SELFTEST_ONLY(execlists->preempt_hang.count++);
945+}
946+
818947 static void execlists_dequeue(struct intel_engine_cs *engine)
819948 {
820949 struct intel_engine_execlists * const execlists = &engine->execlists;
821− struct execlist_port *port = execlists->port;
822− const struct execlist_port * const last_port =
823− &execlists->port[execlists->port_mask];
824− struct i915_request *last = port_request(port);
950+ struct i915_request **port = execlists->pending;
951+ struct i915_request ** const last_port = port + execlists->port_mask;
952+ struct i915_request *last;
825953 struct rb_node *rb;
826954 bool submit = false;
827955
@@ -867,65 +995,100 @@ static void execlists_dequeue(struct intel_engine_cs *engine) · 39 unmodified lines
867995 break;
868996 }
869997
998+ /*
999+ * If the queue is higher priority than the last
1000+ * request in the currently active context, submit afresh.
1001+ * We will resubmit again afterwards in case we need to split
1002+ * the active context to interject the preemption request,
1003+ * i.e. we will retrigger preemption following the ack in case
1004+ * of trouble.
1005+ */
1006+ last = last_active(execlists);
8701007 if (last) {
871− /*
872− * Don't resubmit or switch until all outstanding
873− * preemptions (lite-restore) are seen. Then we
874− * know the next preemption status we see corresponds
875− * to this ELSP update.
876− */
877− GEM_BUG_ON(!execlists_is_active(execlists,
878− EXECLISTS_ACTIVE_USER));
879− GEM_BUG_ON(!port_count(&port[0]));
1008+ if (need_preempt(engine, last, rb)) {
1009+ GEM_TRACE("%s: preempting last=%llx:%lld, prio=%d, hint=%d\n",
1010+ engine->name,
1011+ last->fence.context,
1012+ last->fence.seqno,
1013+ last->sched.attr.priority,
1014+ execlists->queue_priority_hint);
1015+ record_preemption(execlists);
8801016
881− /*
882− * If we write to ELSP a second time before the HW has had
883− * a chance to respond to the previous write, we can confuse
884− * the HW and hit "undefined behaviour". After writing to ELSP,
885− * we must then wait until we see a context-switch event from
886− * the HW to indicate that it has had a chance to respond.
887− */
888− if (!execlists_is_active(execlists, EXECLISTS_ACTIVE_HWACK))
889− return;
1017+ /*
1018+ * Don't let the RING_HEAD advance past the breadcrumb
1019+ * as we unwind (and until we resubmit) so that we do
1020+ * not accidentally tell it to go backwards.
1021+ */
1022+ ring_set_paused(engine, 1);
8901023
891− if (need_preempt(engine, last, rb)) {
892− inject_preempt_context(engine);
893− return;
894− }
1024+ /*
1025+ * Note that we have not stopped the GPU at this point,
1026+ * so we are unwinding the incomplete requests as they
1027+ * remain inflight and so by the time we do complete
1028+ * the preemption, some of the unwound requests may
1029+ * complete!
1030+ */
1031+ __unwind_incomplete_requests(engine);
8951032
896− /*
897− * In theory, we could coalesce more requests onto
898− * the second port (the first port is active, with
899− * no preemptions pending). However, that means we
900− * then have to deal with the possible lite-restore
901− * of the second port (as we submit the ELSP, there
902− * may be a context-switch) but also we may complete
903− * the resubmission before the context-switch. Ergo,
904− * coalescing onto the second port will cause a
905− * preemption event, but we cannot predict whether
906− * that will affect port[0] or port[1].
907− *
908− * If the second port is already active, we can wait
909− * until the next context-switch before contemplating
910− * new requests. The GPU will be busy and we should be
911− * able to resubmit the new ELSP before it idles,
912− * avoiding pipeline bubbles (momentary pauses where
913− * the driver is unable to keep up the supply of new
914− * work). However, we have to double check that the
915− * priorities of the ports haven't been switch.
916− */
917− if (port_count(&port[1]))
918− return;
1033+ /*
1034+ * If we need to return to the preempted context, we
1035+ * need to skip the lite-restore and force it to
1036+ * reload the RING_TAIL. Otherwise, the HW has a
1037+ * tendency to ignore us rewinding the TAIL to the
1038+ * end of an earlier request.
1039+ */
1040+ last->hw_context->lrc_desc |= CTX_DESC_FORCE_RESTORE;
1041+ last = NULL;
1042+ } else if (need_timeslice(engine, last) &&
1043+ !timer_pending(&engine->execlists.timer)) {
1044+ GEM_TRACE("%s: expired last=%llx:%lld, prio=%d, hint=%d\n",
1045+ engine->name,
1046+ last->fence.context,
1047+ last->fence.seqno,
1048+ last->sched.attr.priority,
1049+ execlists->queue_priority_hint);
9191050
920− /*
921− * WaIdleLiteRestore:bdw,skl
922− * Apply the wa NOOPs to prevent
923− * ring:HEAD == rq:TAIL as we resubmit the
924− * request. See gen8_emit_fini_breadcrumb() for
925− * where we prepare the padding after the
926− * end of the request.
927− */
928− last->tail = last->wa_tail;
1051+ ring_set_paused(engine, 1);
1052+ defer_active(engine);
1053+
1054+ /*
1055+ * Unlike for preemption, if we rewind and continue
1056+ * executing the same context as previously active,
1057+ * the order of execution will remain the same and
1058+ * the tail will only advance. We do not need to
1059+ * force a full context restore, as a lite-restore
1060+ * is sufficient to resample the monotonic TAIL.
1061+ *
1062+ * If we switch to any other context, similarly we
1063+ * will not rewind TAIL of current context, and
1064+ * normal save/restore will preserve state and allow
1065+ * us to later continue executing the same request.
1066+ */
1067+ last = NULL;
1068+ } else {
1069+ /*
1070+ * Otherwise if we already have a request pending
1071+ * for execution after the current one, we can
1072+ * just wait until the next CS event before
1073+ * queuing more. In either case we will force a
1074+ * lite-restore preemption event, but if we wait
1075+ * we hopefully coalesce several updates into a single
1076+ * submission.
1077+ */
1078+ if (!list_is_last(&last->sched.link,
1079+ &engine->active.requests))
1080+ return;
1081+
1082+ /*
1083+ * WaIdleLiteRestore:bdw,skl
1084+ * Apply the wa NOOPs to prevent
1085+ * ring:HEAD == rq:TAIL as we resubmit the
1086+ * request. See gen8_emit_fini_breadcrumb() for
1087+ * where we prepare the padding after the
1088+ * end of the request.
1089+ */
1090+ last->tail = last->wa_tail;
1091+ }
9291092 }
9301093
9311094 while (rb) { /* XXX virtual is always taking precedence */
@@ -955,9 +1118,24 @@ static void execlists_dequeue(struct intel_engine_cs *engine) · 23 unmodified lines
9551118 continue;
9561119 }
9571120
1121+ if (i915_request_completed(rq)) {
1122+ ve->request = NULL;
1123+ ve->base.execlists.queue_priority_hint = INT_MIN;
1124+ rb_erase_cached(rb, &execlists->virtual);
1125+ RB_CLEAR_NODE(rb);
1126+
1127+ rq->engine = engine;
1128+ __i915_request_submit(rq);
1129+
1130+ spin_unlock(&ve->base.active.lock);
1131+
1132+ rb = rb_first_cached(&execlists->virtual);
1133+ continue;
1134+ }
1135+
9581136 if (last && !can_merge_rq(last, rq)) {
9591137 spin_unlock(&ve->base.active.lock);
960− return; /* leave this rq for another engine */
1138+ return; /* leave this for another */
9611139 }
9621140
9631141 GEM_TRACE("%s: virtual rq=%llx:%lld%s, new engine? %s\n",
@@ -1006,9 +1184,10 @@ static void execlists_dequeue(struct intel_engine_cs *engine) · 42 unmodified lines
10061184 }
10071185
10081186 __i915_request_submit(rq);
1009− trace_i915_request_in(rq, port_index(port, execlists));
1010− submit = true;
1011− last = rq;
1187+ if (!i915_request_completed(rq)) {
1188+ submit = true;
1189+ last = rq;
1190+ }
10121191 }
10131192
10141193 spin_unlock(&ve->base.active.lock);
@@ -1021,6 +1200,9 @@ static void execlists_dequeue(struct intel_engine_cs *engine) · 6 unmodified lines
10211200 int i;
10221201
10231202 priolist_for_each_request_consume(rq, rn, p, i) {
1203+ if (i915_request_completed(rq))
1204+ goto skip;
1205+
10241206 /*
10251207 * Can we combine this request with the current port?
10261208 * It has to be the same context/ringbuffer and not
@@ -1060,19 +1242,14 @@ static void execlists_dequeue(struct intel_engine_cs *engine) · 33 unmodified lines
10601242 ctx_single_port_submission(rq->hw_context))
10611243 goto done;
10621244
1063−
1064− if (submit)
1065− port_assign(port, last);
1245+ *port = execlists_schedule_in(last, port - execlists->pending);
10661246 port++;
1067−
1068− GEM_BUG_ON(port_isset(port));
10691247 }
10701248
1071− __i915_request_submit(rq);
1072− trace_i915_request_in(rq, port_index(port, execlists));
1073−
10741249 last = rq;
10751250 submit = true;
1251+skip:
1252+ __i915_request_submit(rq);
10761253 }
10771254
10781255 rb_erase_cached(&p->node, &execlists->queue);
@@ -1097,54 +1274,32 @@ done: · 18 unmodified lines
10971274 * interrupt for secondary ports).
10981275 */
10991276 execlists->queue_priority_hint = queue_prio(execlists);
1277+ GEM_TRACE("%s: queue_priority_hint:%d, submit:%s\n",
1278+ engine->name, execlists->queue_priority_hint,
1279+ yesno(submit));
11001280
11011281 if (submit) {
1102− port_assign(port, last);
1282+ *port = execlists_schedule_in(last, port - execlists->pending);
1283+ memset(port + 1, 0, (last_port - port) * sizeof(*port));
11031284 execlists_submit_ports(engine);
1285+ } else {
1286+ ring_set_paused(engine, 0);
11041287 }
1105−
1106− /* We must always keep the beast fed if we have work piled up */
1107− GEM_BUG_ON(rb_first_cached(&execlists->queue) &&
1108− !port_isset(execlists->port));
1109−
1110− /* Re-evaluate the executing context setup after each preemptive kick */
1111− if (last)
1112− execlists_user_begin(execlists, execlists->port);
1113−
1114− /* If the engine is now idle, so should be the flag; and vice versa. */
1115− GEM_BUG_ON(execlists_is_active(&engine->execlists,
1116− EXECLISTS_ACTIVE_USER) ==
1117− !port_isset(engine->execlists.port));
11181288 }
11191289
11201290 void
11211291 execlists_cancel_port_requests(struct intel_engine_execlists * const execlists)
11221292 {
1123− struct execlist_port *port = execlists->port;
1124− unsigned int num_ports = execlists_num_ports(execlists);
1293+ struct i915_request * const *port, *rq;
11251294
1126− while (num_ports-- && port_isset(port)) {
1127− struct i915_request *rq = port_request(port);
1295+ for (port = execlists->pending; (rq = *port); port++)
1296+ execlists_schedule_out(rq);
1297+ memset(execlists->pending, 0, sizeof(execlists->pending));
11281298
1129− GEM_TRACE("%s:port%u fence %llx:%lld, (current %d)\n",
1130− rq->engine->name,
1131− (unsigned int)(port - execlists->port),
1132− rq->fence.context, rq->fence.seqno,
1133− hwsp_seqno(rq));
1134−
1135− GEM_BUG_ON(!execlists->active);
1136− execlists_context_schedule_out(rq,
1137− i915_request_completed(rq) ?
1138− INTEL_CONTEXT_SCHEDULE_OUT :
1139− INTEL_CONTEXT_SCHEDULE_PREEMPTED);
1140−
1141− i915_request_put(rq);
1142−
1143− memset(port, 0, sizeof(*port));
1144− port++;
1145− }
1146−
1147− execlists_clear_all_active(execlists);
1299+ for (port = execlists->active; (rq = *port); port++)
1300+ execlists_schedule_out(rq);
1301+ execlists->active =
1302+ memset(execlists->inflight, 0, sizeof(execlists->inflight));
11481303 }
11491304
11501305 static inline void
@@ -1160,10 +1315,33 @@ reset_in_progress(const struct intel_engine_execlists *execlists) · 9 unmodified lines
11601315 return unlikely(!__tasklet_is_enabled(&execlists->tasklet));
11611316 }
11621317
1318+enum csb_step {
1319+ CSB_NOP,
1320+ CSB_PROMOTE,
1321+ CSB_PREEMPT,
1322+ CSB_COMPLETE,
1323+};
1324+
1325+static inline enum csb_step
1326+csb_parse(const struct intel_engine_execlists *execlists, const u32 *csb)
1327+{
1328+ unsigned int status = *csb;
1329+
1330+ if (status & GEN8_CTX_STATUS_IDLE_ACTIVE)
1331+ return CSB_PROMOTE;
1332+
1333+ if (status & GEN8_CTX_STATUS_PREEMPTED)
1334+ return CSB_PREEMPT;
1335+
1336+ if (*execlists->active)
1337+ return CSB_COMPLETE;
1338+
1339+ return CSB_NOP;
1340+}
1341+
11631342 static void process_csb(struct intel_engine_cs *engine)
11641343 {
11651344 struct intel_engine_execlists * const execlists = &engine->execlists;
1166− struct execlist_port *port = execlists->port;
11671345 const u32 * const buf = execlists->csb_status;
11681346 const u8 num_entries = execlists->csb_size;
11691347 u8 head, tail;
@@ -1198,10 +1376,6 @@ static void process_csb(struct intel_engine_cs *engine) · 28 unmodified lines
11981376 rmb();
11991377
12001378 do {
1201− struct i915_request *rq;
1202− unsigned int status;
1203− unsigned int count;
1204−
12051379 if (++head == num_entries)
12061380 head = 0;
12071381
@@ -1223,68 +1397,39 @@ static void process_csb(struct intel_engine_cs *engine) · 15 unmodified lines
12231397 * status notifier.
12241398 */
12251399
1226− GEM_TRACE("%s csb[%d]: status=0x%08x:0x%08x, active=0x%x\n",
1400+ GEM_TRACE("%s csb[%d]: status=0x%08x:0x%08x\n",
12271401 engine->name, head,
1228− buf[2 * head + 0], buf[2 * head + 1],
1229− execlists->active);
1230−
1231− status = buf[2 * head];
1232− if (status & (GEN8_CTX_STATUS_IDLE_ACTIVE |
1233− GEN8_CTX_STATUS_PREEMPTED))
1234− execlists_set_active(execlists,
1235− EXECLISTS_ACTIVE_HWACK);
1236− if (status & GEN8_CTX_STATUS_ACTIVE_IDLE)
1237− execlists_clear_active(execlists,
1238− EXECLISTS_ACTIVE_HWACK);
1239−
1240− if (!(status & GEN8_CTX_STATUS_COMPLETED_MASK))
1241− continue;
1402+ buf[2 * head + 0], buf[2 * head + 1]);
12421403
1243− /* We should never get a COMPLETED | IDLE_ACTIVE! */
1244− GEM_BUG_ON(status & GEN8_CTX_STATUS_IDLE_ACTIVE);
1404+ switch (csb_parse(execlists, buf + 2 * head)) {
1405+ case CSB_PREEMPT: /* cancel old inflight, prepare for switch */
1406+ trace_ports(execlists, "preempted", execlists->active);
12451407
1246− if (status & GEN8_CTX_STATUS_COMPLETE &&
1247− buf[2*head + 1] == execlists->preempt_complete_status) {
1248− GEM_TRACE("%s preempt-idle\n", engine->name);
1249− complete_preempt_context(execlists);
1250− continue;
1251− }
1408+ while (*execlists->active)
1409+ execlists_schedule_out(*execlists->active++);
12521410
1253− if (status & GEN8_CTX_STATUS_PREEMPTED &&
1254− execlists_is_active(execlists,
1255− EXECLISTS_ACTIVE_PREEMPT))
1256− continue;
1411+ /* fallthrough */
1412+ case CSB_PROMOTE: /* switch pending to inflight */
1413+ GEM_BUG_ON(*execlists->active);
1414+ GEM_BUG_ON(!assert_pending_valid(execlists, "promote"));
1415+ execlists->active =
1416+ memcpy(execlists->inflight,
1417+ execlists->pending,
1418+ execlists_num_ports(execlists) *
1419+ sizeof(*execlists->pending));
1420+ execlists->pending[0] = NULL;
12571421
1258− GEM_BUG_ON(!execlists_is_active(execlists,
1259− EXECLISTS_ACTIVE_USER));
1422+ trace_ports(execlists, "promoted", execlists->active);
12601423
1261− rq = port_unpack(port, &count);
1262− GEM_TRACE("%s out[0]: ctx=%d.%d, fence %llx:%lld (current %d), prio=%d\n",
1263− engine->name,
1264− port->context_id, count,
1265− rq ? rq->fence.context : 0,
1266− rq ? rq->fence.seqno : 0,
1267− rq ? hwsp_seqno(rq) : 0,
1268− rq ? rq_prio(rq) : 0);
1424+ if (enable_timeslice(engine))
1425+ mod_timer(&execlists->timer, jiffies + 1);
12691426
1270− /* Check the context/desc id for this event matches */
1271− GEM_DEBUG_BUG_ON(buf[2 * head + 1] != port->context_id);
1427+ if (!inject_preempt_hang(execlists))
1428+ ring_set_paused(engine, 0);
1429+ break;
12721430
1273− GEM_BUG_ON(count == 0);
1274− if (--count == 0) {
1275− /*
1276− * On the final event corresponding to the
1277− * submission of this context, we expect either
1278− * an element-switch event or a completion
1279− * event (and on completion, the active-idle
1280− * marker). No more preemptions, lite-restore
1281− * or otherwise.
1282− */
1283− GEM_BUG_ON(status & GEN8_CTX_STATUS_PREEMPTED);
1284− GEM_BUG_ON(port_isset(&port[1]) &&
1285− !(status & GEN8_CTX_STATUS_ELEMENT_SWITCH));
1286− GEM_BUG_ON(!port_isset(&port[1]) &&
1287− !(status & GEN8_CTX_STATUS_ACTIVE_IDLE));
1431+ case CSB_COMPLETE: /* port0 completed, advanced to port1 */
1432+ trace_ports(execlists, "completed", execlists->active);
12881433
12891434 /*
12901435 * We rely on the hardware being strongly
@@ -1292,22 +1437,16 @@ static void process_csb(struct intel_engine_cs *engine) · 1 unmodified line
12921437 * coherent (visible from the CPU) before the
12931438 * user interrupt and CSB is processed.
12941439 */
1295− GEM_BUG_ON(!i915_request_completed(rq));
1440+ GEM_BUG_ON(!i915_request_completed(*execlists->active) &&
1441+ !reset_in_progress(execlists));
1442+ execlists_schedule_out(*execlists->active++);
12961443
1297− execlists_context_schedule_out(rq,
1298− INTEL_CONTEXT_SCHEDULE_OUT);
1299− i915_request_put(rq);
1300−
1301− GEM_TRACE("%s completed ctx=%d\n",
1302− engine->name, port->context_id);
1444+ GEM_BUG_ON(execlists->active - execlists->inflight >
1445+ execlists_num_ports(execlists));
1446+ break;
13031447
1304− port = execlists_port_complete(execlists, port);
1305− if (port_isset(port))
1306− execlists_user_begin(execlists, port);
1307− else
1308− execlists_user_end(execlists);
1309− } else {
1310− port_set(port, port_pack(rq, count));
1448+ case CSB_NOP:
1449+ break;
13111450 }
13121451 } while (head != tail);
13131452
@@ -1332,7 +1471,7 @@ static void __execlists_submission_tasklet(struct intel_engine_cs *const engine) · 18 unmodified lines
13321471 lockdep_assert_held(&engine->active.lock);
13331472
13341473 process_csb(engine);
1335− if (!execlists_is_active(&engine->execlists, EXECLISTS_ACTIVE_PREEMPT))
1474+ if (!engine->execlists.pending[0])
13361475 execlists_dequeue(engine);
13371476 }
13381477
@@ -1345,16 +1484,20 @@ static void execlists_submission_tasklet(unsigned long data) · 6 unmodified lines
13451484 struct intel_engine_cs * const engine = (struct intel_engine_cs *)data;
13461485 unsigned long flags;
13471486
1348− GEM_TRACE("%s awake?=%d, active=%x\n",
1349− engine->name,
1350− !!intel_wakeref_active(&engine->wakeref),
1351− engine->execlists.active);
1352−
13531487 spin_lock_irqsave(&engine->active.lock, flags);
13541488 __execlists_submission_tasklet(engine);
13551489 spin_unlock_irqrestore(&engine->active.lock, flags);
13561490 }
13571491
1492+static void execlists_submission_timer(struct timer_list *timer)
1493+{
1494+ struct intel_engine_cs *engine =
1495+ from_timer(engine, timer, execlists.timer);
1496+
1497+ /* Kick the tasklet for some interrupt coalescing and reset handling */
1498+ tasklet_hi_schedule(&engine->execlists.tasklet);
1499+}
1500+
13581501 static void queue_request(struct intel_engine_cs *engine,
13591502 struct i915_sched_node *node,
13601503 int prio)
@@ -1376,12 +1519,16 @@ static void __submit_queue_imm(struct intel_engine_cs *engine) · 15 unmodified lines
13761519 tasklet_hi_schedule(&execlists->tasklet);
13771520 }
13781521
1379−static void submit_queue(struct intel_engine_cs *engine, int prio)
1522+static void submit_queue(struct intel_engine_cs *engine,
1523+ const struct i915_request *rq)
13801524 {
1381− if (prio > engine->execlists.queue_priority_hint) {
1382− engine->execlists.queue_priority_hint = prio;
1383− __submit_queue_imm(engine);
1384− }
1525+ struct intel_engine_execlists *execlists = &engine->execlists;
1526+
1527+ if (rq_prio(rq) <= execlists->queue_priority_hint)
1528+ return;
1529+
1530+ execlists->queue_priority_hint = rq_prio(rq);
1531+ __submit_queue_imm(engine);
13851532 }
13861533
13871534 static void execlists_submit_request(struct i915_request *request)
@@ -1397,7 +1544,7 @@ static void execlists_submit_request(struct i915_request *request) · 9 unmodified lines
13971544 GEM_BUG_ON(RB_EMPTY_ROOT(&engine->execlists.queue.rb_root));
13981545 GEM_BUG_ON(list_empty(&request->sched.link));
13991546
1400− submit_queue(engine, rq_prio(request));
1547+ submit_queue(engine, request);
14011548
14021549 spin_unlock_irqrestore(&engine->active.lock, flags);
14031550 }
@@ -1405,9 +1552,7 @@ static void execlists_submit_request(struct i915_request *request) · 1 unmodified line
14051552 static void __execlists_context_fini(struct intel_context *ce)
14061553 {
14071554 intel_ring_put(ce->ring);
1408−
1409− GEM_BUG_ON(i915_gem_object_is_active(ce->state->obj));
1410− i915_gem_object_put(ce->state->obj);
1555+ i915_vma_put(ce->state);
14111556 }
14121557
14131558 static void execlists_context_destroy(struct kref *kref)
@@ -1420,6 +1565,7 @@ static void execlists_context_destroy(struct kref *kref) · 6 unmodified lines
14201565 if (ce->state)
14211566 __execlists_context_fini(ce);
14221567
1568+ intel_context_fini(ce);
14231569 intel_context_free(ce);
14241570 }
14251571
@@ -1444,9 +1590,12 @@ __execlists_update_reg_state(struct intel_context *ce, · 18 unmodified lines
14441590 regs[CTX_RING_TAIL + 1] = ring->tail;
14451591
14461592 /* RPCS */
1447− if (engine->class == RENDER_CLASS)
1593+ if (engine->class == RENDER_CLASS) {
14481594 regs[CTX_R_PWR_CLK_STATE + 1] =
14491595 intel_sseu_make_rpcs(engine->i915, &ce->sseu);
1596+
1597+ i915_oa_init_reg_state(engine, ce, regs);
1598+ }
14501599 }
14511600
14521601 static int
@@ -1456,19 +1605,15 @@ __execlists_context_pin(struct intel_context *ce, · 3 unmodified lines
14561605 void *vaddr;
14571606 int ret;
14581607
1459− GEM_BUG_ON(!ce->gem_context->vm);
1460−
14611608 ret = execlists_context_deferred_alloc(ce, engine);
14621609 if (ret)
14631610 goto err;
14641611 GEM_BUG_ON(!ce->state);
14651612
1466− ret = intel_context_active_acquire(ce,
1467− engine->i915->ggtt.pin_bias |
1468− PIN_OFFSET_BIAS |
1469− PIN_HIGH);
1613+ ret = intel_context_active_acquire(ce);
14701614 if (ret)
14711615 goto err;
1616+ GEM_BUG_ON(!i915_vma_is_pinned(ce->state));
14721617
14731618 vaddr = i915_gem_object_pin_map(ce->state->obj,
14741619 i915_coherent_map_type(engine->i915) |
@@ -1569,8 +1714,7 @@ static int gen8_emit_init_breadcrumb(struct i915_request *rq) · 94 unmodified lines
15691714 static int emit_pdps(struct i915_request *rq)
15701715 {
15711716 const struct intel_engine_cs * const engine = rq->engine;
1572− struct i915_ppgtt * const ppgtt =
1573− i915_vm_to_ppgtt(rq->gem_context->vm);
1717+ struct i915_ppgtt * const ppgtt = i915_vm_to_ppgtt(rq->hw_context->vm);
15741718 int err, i;
15751719 u32 *cs;
15761720
@@ -1643,7 +1787,7 @@ static int execlists_request_alloc(struct i915_request *request) · 66 unmodified lines
16431787 */
16441788
16451789 /* Unconditionally invalidate GPU caches and TLBs. */
1646− if (i915_vm_is_4lvl(request->gem_context->vm))
1790+ if (i915_vm_is_4lvl(request->hw_context->vm))
16471791 ret = request->engine->emit_flush(request, EMIT_INVALIDATE);
16481792 else
16491793 ret = emit_pdps(request);
@@ -1676,7 +1820,8 @@ gen8_emit_flush_coherentl3_wa(struct intel_engine_cs *engine, u32 *batch) · 26 unmodified lines
16761820 /* NB no one else is allowed to scribble over scratch + 256! */
16771821 *batch++ = MI_STORE_REGISTER_MEM_GEN8 | MI_SRM_LRM_GLOBAL_GTT;
16781822 *batch++ = i915_mmio_reg_offset(GEN8_L3SQCREG4);
1679− *batch++ = i915_scratch_offset(engine->i915) + 256;
1823+ *batch++ = intel_gt_scratch_offset(engine->gt,
1824+ INTEL_GT_SCRATCH_FIELD_COHERENTL3_WA);
16801825 *batch++ = 0;
16811826
16821827 *batch++ = MI_LOAD_REGISTER_IMM(1);
@@ -1690,12 +1835,19 @@ gen8_emit_flush_coherentl3_wa(struct intel_engine_cs *engine, u32 *batch) · 7 unmodified lines
16901835
16911836 *batch++ = MI_LOAD_REGISTER_MEM_GEN8 | MI_SRM_LRM_GLOBAL_GTT;
16921837 *batch++ = i915_mmio_reg_offset(GEN8_L3SQCREG4);
1693− *batch++ = i915_scratch_offset(engine->i915) + 256;
1838+ *batch++ = intel_gt_scratch_offset(engine->gt,
1839+ INTEL_GT_SCRATCH_FIELD_COHERENTL3_WA);
16941840 *batch++ = 0;
16951841
16961842 return batch;
16971843 }
16981844
1845+static u32 slm_offset(struct intel_engine_cs *engine)
1846+{
1847+ return intel_gt_scratch_offset(engine->gt,
1848+ INTEL_GT_SCRATCH_FIELD_CLEAR_SLM_WA);
1849+}
1850+
16991851 /*
17001852 * Typically we only have one indirect_ctx and per_ctx batch buffer which are
17011853 * initialized at the beginning and shared across all contexts but this field
@@ -1727,8 +1879,7 @@ static u32 *gen8_init_indirectctx_bb(struct intel_engine_cs *engine, u32 *batch) · 25 unmodified lines
17271879 PIPE_CONTROL_GLOBAL_GTT_IVB |
17281880 PIPE_CONTROL_CS_STALL |
17291881 PIPE_CONTROL_QW_WRITE,
1730− i915_scratch_offset(engine->i915) +
1731− 2 * CACHELINE_BYTES);
1882+ slm_offset(engine));
17321883
17331884 *batch++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
17341885
@@ -1874,7 +2025,7 @@ static int lrc_setup_wa_ctx(struct intel_engine_cs *engine) · 139 unmodified lines
18742025 if (IS_ERR(obj))
18752026 return PTR_ERR(obj);
18762027
1877− vma = i915_vma_instance(obj, &engine->i915->ggtt.vm, NULL);
2028+ vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);
18782029 if (IS_ERR(vma)) {
18792030 err = PTR_ERR(vma);
18802031 goto err;
@@ -1970,22 +2121,23 @@ static int intel_init_workaround_bb(struct intel_engine_cs *engine) · 89 unmodified lines
19702121
19712122 static void enable_execlists(struct intel_engine_cs *engine)
19722123 {
2124+ u32 mode;
2125+
2126+ assert_forcewakes_active(engine->uncore, FORCEWAKE_ALL);
2127+
19732128 intel_engine_set_hwsp_writemask(engine, ~0u); /* HWSTAM */
19742129
19752130 if (INTEL_GEN(engine->i915) >= 11)
1976− ENGINE_WRITE(engine,
1977− RING_MODE_GEN7,
1978− _MASKED_BIT_ENABLE(GEN11_GFX_DISABLE_LEGACY_MODE));
2131+ mode = _MASKED_BIT_ENABLE(GEN11_GFX_DISABLE_LEGACY_MODE);
19792132 else
1980− ENGINE_WRITE(engine,
1981− RING_MODE_GEN7,
1982− _MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE));
2133+ mode = _MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE);
2134+ ENGINE_WRITE_FW(engine, RING_MODE_GEN7, mode);
19832135
1984− ENGINE_WRITE(engine, RING_MI_MODE, _MASKED_BIT_DISABLE(STOP_RING));
2136+ ENGINE_WRITE_FW(engine, RING_MI_MODE, _MASKED_BIT_DISABLE(STOP_RING));
19852137
1986− ENGINE_WRITE(engine,
1987− RING_HWS_PGA,
1988− i915_ggtt_offset(engine->status_page.vma));
2138+ ENGINE_WRITE_FW(engine,
2139+ RING_HWS_PGA,
2140+ i915_ggtt_offset(engine->status_page.vma));
19892141 ENGINE_POSTING_READ(engine, RING_HWS_PGA);
19902142 }
19912143
@@ -1993,7 +2145,7 @@ static bool unexpected_starting_state(struct intel_engine_cs *engine) · 1 unmodified line
19932145 {
19942146 bool unexpected = false;
19952147
1996− if (ENGINE_READ(engine, RING_MI_MODE) & STOP_RING) {
2148+ if (ENGINE_READ_FW(engine, RING_MI_MODE) & STOP_RING) {
19972149 DRM_DEBUG_DRIVER("STOP_RING still set in RING_MI_MODE\n");
19982150 unexpected = true;
19992151 }
@@ -2041,34 +2193,32 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine) · 41 unmodified lines
20412193 __tasklet_disable_sync_once(&execlists->tasklet);
20422194 GEM_BUG_ON(!reset_in_progress(execlists));
20432195
2044− intel_engine_stop_cs(engine);
2045−
20462196 /* And flush any current direct submission. */
20472197 spin_lock_irqsave(&engine->active.lock, flags);
20482198 spin_unlock_irqrestore(&engine->active.lock, flags);
2049−}
2050−
2051−static bool lrc_regs_ok(const struct i915_request *rq)
2052−{
2053− const struct intel_ring *ring = rq->ring;
2054− const u32 *regs = rq->hw_context->lrc_reg_state;
2055−
2056− /* Quick spot check for the common signs of context corruption */
20572199
2058− if (regs[CTX_RING_BUFFER_CONTROL + 1] !=
2059− (RING_CTL_SIZE(ring->size) | RING_VALID))
2060− return false;
2061−
2062− if (regs[CTX_RING_BUFFER_START + 1] != i915_ggtt_offset(ring->vma))
2063− return false;
2064−
2065− return true;
2200+ /*
2201+ * We stop engines, otherwise we might get failed reset and a
2202+ * dead gpu (on elk). Also as modern gpu as kbl can suffer
2203+ * from system hang if batchbuffer is progressing when
2204+ * the reset is issued, regardless of READY_TO_RESET ack.
2205+ * Thus assume it is best to stop engines on all gens
2206+ * where we have a gpu reset.
2207+ *
2208+ * WaKBLVECSSemaphoreWaitPoll:kbl (on ALL_ENGINES)
2209+ *
2210+ * FIXME: Wa for more modern gens needs to be validated
2211+ */
2212+ intel_engine_stop_cs(engine);
20662213 }
20672214
2068−static void reset_csb_pointers(struct intel_engine_execlists *execlists)
2215+static void reset_csb_pointers(struct intel_engine_cs *engine)
20692216 {
2217+ struct intel_engine_execlists * const execlists = &engine->execlists;
20702218 const unsigned int reset_value = execlists->csb_size - 1;
20712219
2220+ ring_set_paused(engine, 0);
2221+
20722222 /*
20732223 * After a reset, the HW starts writing into CSB entry [0]. We
20742224 * therefore have to set our HEAD pointer back one entry so that
@@ -2115,18 +2265,21 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 40 unmodified lines
21152265 process_csb(engine); /* drain preemption events */
21162266
21172267 /* Following the reset, we need to reload the CSB read/write pointers */
2118− reset_csb_pointers(&engine->execlists);
2268+ reset_csb_pointers(engine);
21192269
21202270 /*
21212271 * Save the currently executing context, even if we completed
21222272 * its request, it was still running at the time of the
21232273 * reset and will have been clobbered.
21242274 */
2125− if (!port_isset(execlists->port))
2126− goto out_clear;
2275+ rq = execlists_active(execlists);
2276+ if (!rq)
2277+ goto unwind;
21272278
2128− rq = port_request(execlists->port);
21292279 ce = rq->hw_context;
2280+ GEM_BUG_ON(i915_active_is_idle(&ce->active));
2281+ GEM_BUG_ON(!i915_vma_is_pinned(ce->state));
2282+ rq = active_request(rq);
21302283
21312284 /*
21322285 * Catch up with any missed context-switch interrupts.
@@ -2139,9 +2292,12 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 6 unmodified lines
21392292 */
21402293 execlists_cancel_port_requests(execlists);
21412294
2142− rq = active_request(rq);
2143− if (!rq)
2295+ if (!rq) {
2296+ ce->ring->head = ce->ring->tail;
21442297 goto out_replay;
2298+ }
2299+
2300+ ce->ring->head = intel_ring_wrap(ce->ring, rq->head);
21452301
21462302 /*
21472303 * If this request hasn't started yet, e.g. it is waiting on a
@@ -2155,7 +2311,7 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 7 unmodified lines
21552311 * Otherwise, if we have not started yet, the request should replay
21562312 * perfectly and we do not need to flag the result as being erroneous.
21572313 */
2158− if (!i915_request_started(rq) && lrc_regs_ok(rq))
2314+ if (!i915_request_started(rq))
21592315 goto out_replay;
21602316
21612317 /*
@@ -2169,8 +2325,8 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 7 unmodified lines
21692325 * and have to at least restore the RING register in the context
21702326 * image back to the expected values to skip over the guilty request.
21712327 */
2172− i915_reset_request(rq, stalled);
2173− if (!stalled && lrc_regs_ok(rq))
2328+ __i915_request_reset(rq, stalled);
2329+ if (!stalled)
21742330 goto out_replay;
21752331
21762332 /*
@@ -2190,17 +2346,14 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 13 unmodified lines
21902346 execlists_init_reg_state(regs, ce, engine, ce->ring);
21912347
21922348 out_replay:
2193− /* Rerun the request; its payload has been neutered (if guilty). */
2194− ce->ring->head =
2195− rq ? intel_ring_wrap(ce->ring, rq->head) : ce->ring->tail;
2349+ GEM_TRACE("%s replay {head:%04x, tail:%04x\n",
2350+ engine->name, ce->ring->head, ce->ring->tail);
21962351 intel_ring_update_space(ce->ring);
21972352 __execlists_update_reg_state(ce, engine);
21982353
2354+unwind:
21992355 /* Push back any incomplete requests for replay after the reset. */
22002356 __unwind_incomplete_requests(engine);
2201−
2202−out_clear:
2203− execlists_clear_all_active(execlists);
22042357 }
22052358
22062359 static void execlists_reset(struct intel_engine_cs *engine, bool stalled)
@@ -2296,7 +2449,6 @@ static void execlists_cancel_requests(struct intel_engine_cs *engine) · 89 unmodified lines
22962449
22972450 execlists->queue_priority_hint = INT_MIN;
22982451 execlists->queue = RB_ROOT_CACHED;
2299− GEM_BUG_ON(port_isset(execlists->port));
23002452
23012453 GEM_BUG_ON(__tasklet_is_enabled(&execlists->tasklet));
23022454 execlists->tasklet.func = nop_submission_tasklet;
@@ -2434,7 +2586,8 @@ static int gen8_emit_flush_render(struct i915_request *request, · 131 unmodified lines
24342586 {
24352587 struct intel_engine_cs *engine = request->engine;
24362588 u32 scratch_addr =
2437− i915_scratch_offset(engine->i915) + 2 * CACHELINE_BYTES;
2589+ intel_gt_scratch_offset(engine->gt,
2590+ INTEL_GT_SCRATCH_FIELD_RENDER_FLUSH);
24382591 bool vf_flush_wa = false, dc_flush_wa = false;
24392592 u32 *cs, flags = 0;
24402593 int len;
@@ -2514,15 +2667,30 @@ static u32 *gen8_emit_wa_tail(struct i915_request *request, u32 *cs) · 73 unmodified lines
25142667 return cs;
25152668 }
25162669
2670+static u32 *emit_preempt_busywait(struct i915_request *request, u32 *cs)
2671+{
2672+ *cs++ = MI_SEMAPHORE_WAIT |
2673+ MI_SEMAPHORE_GLOBAL_GTT |
2674+ MI_SEMAPHORE_POLL |
2675+ MI_SEMAPHORE_SAD_EQ_SDD;
2676+ *cs++ = 0;
2677+ *cs++ = intel_hws_preempt_address(request->engine);
2678+ *cs++ = 0;
2679+
2680+ return cs;
2681+}
2682+
25172683 static u32 *gen8_emit_fini_breadcrumb(struct i915_request *request, u32 *cs)
25182684 {
25192685 cs = gen8_emit_ggtt_write(cs,
25202686 request->fence.seqno,
25212687 request->timeline->hwsp_offset,
25222688 0);
2523−
25242689 *cs++ = MI_USER_INTERRUPT;
2690+
25252691 *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
2692+ if (intel_engine_has_semaphores(request->engine))
2693+ cs = emit_preempt_busywait(request, cs);
25262694
25272695 request->tail = intel_ring_offset(request, cs);
25282696 assert_ring_tail_valid(request->ring, request->tail);
@@ -2543,9 +2711,11 @@ static u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs) · 14 unmodified lines
25432711 PIPE_CONTROL_FLUSH_ENABLE |
25442712 PIPE_CONTROL_CS_STALL,
25452713 0);
2546−
25472714 *cs++ = MI_USER_INTERRUPT;
2715+
25482716 *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
2717+ if (intel_engine_has_semaphores(request->engine))
2718+ cs = emit_preempt_busywait(request, cs);
25492719
25502720 request->tail = intel_ring_offset(request, cs);
25512721 assert_ring_tail_valid(request->ring, request->tail);
@@ -2553,27 +2723,9 @@ static u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs) · 1 unmodified line
25532723 return gen8_emit_wa_tail(request, cs);
25542724 }
25552725
2556−static int gen8_init_rcs_context(struct i915_request *rq)
2557−{
2558− int ret;
2559−
2560− ret = intel_engine_emit_ctx_wa(rq);
2561− if (ret)
2562− return ret;
2563−
2564− ret = intel_rcs_context_init_mocs(rq);
2565− /*
2566− * Failing to program the MOCS is non-fatal.The system will not
2567− * run at peak performance. So generate an error and carry on.
2568− */
2569− if (ret)
2570− DRM_ERROR("MOCS failed to program: expect performance issues.\n");
2571−
2572− return i915_gem_render_state_emit(rq);
2573−}
2574−
25752726 static void execlists_park(struct intel_engine_cs *engine)
25762727 {
2728+ del_timer_sync(&engine->execlists.timer);
25772729 intel_engine_park(engine);
25782730 }
25792731
@@ -2592,11 +2744,11 @@ void intel_execlists_set_default_submission(struct intel_engine_cs *engine) · 12 unmodified lines
25922744 engine->unpark = NULL;
25932745
25942746 engine->flags |= I915_ENGINE_SUPPORTS_STATS;
2595− if (!intel_vgpu_active(engine->i915))
2747+ if (!intel_vgpu_active(engine->i915)) {
25962748 engine->flags |= I915_ENGINE_HAS_SEMAPHORES;
2597− if (engine->preempt_context &&
2598− HAS_LOGICAL_RING_PREEMPTION(engine->i915))
2599− engine->flags |= I915_ENGINE_HAS_PREEMPTION;
2749+ if (HAS_LOGICAL_RING_PREEMPTION(engine->i915))
2750+ engine->flags |= I915_ENGINE_HAS_PREEMPTION;
2751+ }
26002752 }
26012753
26022754 static void execlists_destroy(struct intel_engine_cs *engine)
@@ -2672,12 +2824,12 @@ int intel_execlists_submission_setup(struct intel_engine_cs *engine) · 69 unmodified lines
26722824
26732825 tasklet_init(&engine->execlists.tasklet,
26742826 execlists_submission_tasklet, (unsigned long)engine);
2827+ timer_setup(&engine->execlists.timer, execlists_submission_timer, 0);
26752828
26762829 logical_ring_default_vfuncs(engine);
26772830 logical_ring_default_irqs(engine);
26782831
26792832 if (engine->class == RENDER_CLASS) {
2680− engine->init_context = gen8_init_rcs_context;
26812833 engine->emit_flush = gen8_emit_flush_render;
26822834 engine->emit_fini_breadcrumb = gen8_emit_fini_breadcrumb_rcs;
26832835 }
@@ -2697,9 +2849,6 @@ int intel_execlists_submission_init(struct intel_engine_cs *engine) · 13 unmodified lines
26972849 if (ret)
26982850 return ret;
26992851
2700− intel_engine_init_workarounds(engine);
2701− intel_engine_init_whitelist(engine);
2702−
27032852 if (intel_init_workaround_bb(engine))
27042853 /*
27052854 * We continue even if we fail to initialize WA batch
@@ -2718,11 +2867,6 @@ int intel_execlists_submission_init(struct intel_engine_cs *engine) · 12 unmodified lines
27182867 i915_mmio_reg_offset(RING_ELSP(base));
27192868 }
27202869
2721− execlists->preempt_complete_status = ~0u;
2722− if (engine->preempt_context)
2723− execlists->preempt_complete_status =
2724− upper_32_bits(engine->preempt_context->lrc_desc);
2725−
27262870 execlists->csb_status =
27272871 &engine->status_page.addr[I915_HWS_CSB_BUF0_INDEX];
27282872
@@ -2734,7 +2878,7 @@ int intel_execlists_submission_init(struct intel_engine_cs *engine) · 5 unmodified lines
27342878 else
27352879 execlists->csb_size = GEN11_CSB_ENTRIES;
27362880
2737− reset_csb_pointers(execlists);
2881+ reset_csb_pointers(engine);
27382882
27392883 return 0;
27402884 }
@@ -2773,7 +2917,7 @@ static void execlists_init_reg_state(u32 *regs, · 32 unmodified lines
27732917 struct intel_engine_cs *engine,
27742918 struct intel_ring *ring)
27752919 {
2776− struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(ce->gem_context->vm);
2920+ struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(ce->vm);
27772921 bool rcs = engine->class == RENDER_CLASS;
27782922 u32 base = engine->mmio_base;
27792923
@@ -2864,8 +3008,6 @@ static void execlists_init_reg_state(u32 *regs, · 84 unmodified lines
28643008 if (rcs) {
28653009 regs[CTX_LRI_HEADER_2] = MI_LOAD_REGISTER_IMM(1);
28663010 CTX_REG(regs, CTX_R_PWR_CLK_STATE, GEN8_R_PWR_CLK_STATE, 0);
2867−
2868− i915_oa_init_reg_state(engine, ce, regs);
28693011 }
28703012
28713013 regs[CTX_END] = MI_BATCH_BUFFER_END;
@@ -2917,11 +3059,6 @@ populate_lr_context(struct intel_context *ce, · 45 unmodified lines
29173059 if (!engine->default_state)
29183060 regs[CTX_CONTEXT_CONTROL + 1] |=
29193061 _MASKED_BIT_ENABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT);
2920− if (ce->gem_context == engine->i915->preempt_context &&
2921− INTEL_GEN(engine->i915) < 11)
2922− regs[CTX_CONTEXT_CONTROL + 1] |=
2923− _MASKED_BIT_ENABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT |
2924− CTX_CTRL_ENGINE_CTX_SAVE_INHIBIT);
29253062
29263063 ret = 0;
29273064 err_unpin_ctx:
@@ -2932,12 +3069,13 @@ err_unpin_ctx: · 4 unmodified lines
29323069 return ret;
29333070 }
29343071
2935−static struct i915_timeline *get_timeline(struct i915_gem_context *ctx)
3072+static struct intel_timeline *
3073+get_timeline(struct i915_gem_context *ctx, struct intel_gt *gt)
29363074 {
29373075 if (ctx->timeline)
2938− return i915_timeline_get(ctx->timeline);
3076+ return intel_timeline_get(ctx->timeline);
29393077 else
2940− return i915_timeline_create(ctx->i915, NULL);
3078+ return intel_timeline_create(gt, NULL);
29413079 }
29423080
29433081 static int execlists_context_deferred_alloc(struct intel_context *ce,
@@ -2947,7 +3085,7 @@ static int execlists_context_deferred_alloc(struct intel_context *ce, · 3 unmodified lines
29473085 struct i915_vma *vma;
29483086 u32 context_size;
29493087 struct intel_ring *ring;
2950− struct i915_timeline *timeline;
3088+ struct intel_timeline *timeline;
29513089 int ret;
29523090
29533091 if (ce->state)
@@ -2965,13 +3103,13 @@ static int execlists_context_deferred_alloc(struct intel_context *ce, · 11 unmodified lines
29653103 if (IS_ERR(ctx_obj))
29663104 return PTR_ERR(ctx_obj);
29673105
2968− vma = i915_vma_instance(ctx_obj, &engine->i915->ggtt.vm, NULL);
3106+ vma = i915_vma_instance(ctx_obj, &engine->gt->ggtt->vm, NULL);
29693107 if (IS_ERR(vma)) {
29703108 ret = PTR_ERR(vma);
29713109 goto error_deref_obj;
29723110 }
29733111
2974− timeline = get_timeline(ce->gem_context);
3112+ timeline = get_timeline(ce->gem_context, engine->gt);
29753113 if (IS_ERR(timeline)) {
29763114 ret = PTR_ERR(timeline);
29773115 goto error_deref_obj;
@@ -2980,7 +3118,7 @@ static int execlists_context_deferred_alloc(struct intel_context *ce, · 2 unmodified lines
29803118 ring = intel_engine_create_ring(engine,
29813119 timeline,
29823120 ce->gem_context->ring_size);
2983− i915_timeline_put(timeline);
3121+ intel_timeline_put(timeline);
29843122 if (IS_ERR(ring)) {
29853123 ret = PTR_ERR(ring);
29863124 goto error_deref_obj;
@@ -3038,6 +3176,7 @@ static void virtual_context_destroy(struct kref *kref) · 51 unmodified lines
30383176
30393177 if (ve->context.state)
30403178 __execlists_context_fini(&ve->context);
3179+ intel_context_fini(&ve->context);
30413180
30423181 kfree(ve->bonds);
30433182 kfree(ve);
@@ -3290,11 +3429,11 @@ intel_execlists_create_virtual(struct i915_gem_context *ctx, · 246 unmodified lines
32903429 return ERR_PTR(-ENOMEM);
32913430
32923431 ve->base.i915 = ctx->i915;
3432+ ve->base.gt = siblings[0]->gt;
32933433 ve->base.id = -1;
32943434 ve->base.class = OTHER_CLASS;
32953435 ve->base.uabi_class = I915_ENGINE_CLASS_INVALID;
32963436 ve->base.instance = I915_ENGINE_CLASS_INVALID_VIRTUAL;
3297− ve->base.flags = I915_ENGINE_IS_VIRTUAL;
32983437
32993438 /*
33003439 * The decision on whether to submit a request using semaphores
@@ -3391,8 +3530,12 @@ intel_execlists_create_virtual(struct i915_gem_context *ctx, · 90 unmodified lines
33913530 ve->base.emit_fini_breadcrumb = sibling->emit_fini_breadcrumb;
33923531 ve->base.emit_fini_breadcrumb_dw =
33933532 sibling->emit_fini_breadcrumb_dw;
3533+
3534+ ve->base.flags = sibling->flags;
33943535 }
33953536
3537+ ve->base.flags |= I915_ENGINE_IS_VIRTUAL;
3538+
33963539 return &ve->context;
33973540
33983541 err_put:
drivers/gpu/drm/i915/gt/intel_mocs.c+40−22
@@ -23,6 +23,7 @@ · 22 unmodified lines
2323 #include "i915_drv.h"
2424
2525 #include "intel_engine.h"
26+#include "intel_gt.h"
2627 #include "intel_mocs.h"
2728 #include "intel_lrc.h"
2829
@@ -247,7 +248,7 @@ static const struct drm_i915_mocs_entry icelake_mocs_table[] = { · 218 unmodified lines
247248
248249 /**
249250 * get_mocs_settings()
250− * @dev_priv: i915 device.
251+ * @gt: gt device
251252 * @table: Output table that will be made to point at appropriate
252253 * MOCS values for the device.
253254 *
@@ -257,33 +258,34 @@ static const struct drm_i915_mocs_entry icelake_mocs_table[] = { · 3 unmodified lines
257258 *
258259 * Return: true if there are applicable MOCS settings for the device.
259260 */
260−static bool get_mocs_settings(struct drm_i915_private *dev_priv,
261+static bool get_mocs_settings(struct intel_gt *gt,
261262 struct drm_i915_mocs_table *table)
262263 {
264+ struct drm_i915_private *i915 = gt->i915;
263265 bool result = false;
264266
265− if (INTEL_GEN(dev_priv) >= 11) {
267+ if (INTEL_GEN(i915) >= 11) {
266268 table->size = ARRAY_SIZE(icelake_mocs_table);
267269 table->table = icelake_mocs_table;
268270 table->n_entries = GEN11_NUM_MOCS_ENTRIES;
269271 result = true;
270− } else if (IS_GEN9_BC(dev_priv) || IS_CANNONLAKE(dev_priv)) {
272+ } else if (IS_GEN9_BC(i915) || IS_CANNONLAKE(i915)) {
271273 table->size = ARRAY_SIZE(skylake_mocs_table);
272274 table->n_entries = GEN9_NUM_MOCS_ENTRIES;
273275 table->table = skylake_mocs_table;
274276 result = true;
275− } else if (IS_GEN9_LP(dev_priv)) {
277+ } else if (IS_GEN9_LP(i915)) {
276278 table->size = ARRAY_SIZE(broxton_mocs_table);
277279 table->n_entries = GEN9_NUM_MOCS_ENTRIES;
278280 table->table = broxton_mocs_table;
279281 result = true;
280282 } else {
281− WARN_ONCE(INTEL_GEN(dev_priv) >= 9,
283+ WARN_ONCE(INTEL_GEN(i915) >= 9,
282284 "Platform that should have a MOCS table does not.\n");
283285 }
284286
285287 /* WaDisableSkipCaching:skl,bxt,kbl,glk */
286− if (IS_GEN(dev_priv, 9)) {
288+ if (IS_GEN(i915, 9)) {
287289 int i;
288290
289291 for (i = 0; i < table->size; i++)
@@ -338,12 +340,16 @@ static u32 get_entry_control(const struct drm_i915_mocs_table *table, · 48 unmodified lines
338340 */
339341 void intel_mocs_init_engine(struct intel_engine_cs *engine)
340342 {
341− struct drm_i915_private *dev_priv = engine->i915;
343+ struct intel_gt *gt = engine->gt;
344+ struct intel_uncore *uncore = gt->uncore;
342345 struct drm_i915_mocs_table table;
343346 unsigned int index;
344347 u32 unused_value;
345348
346− if (!get_mocs_settings(dev_priv, &table))
349+ /* Called under a blanket forcewake */
350+ assert_forcewakes_active(uncore, FORCEWAKE_ALL);
351+
352+ if (!get_mocs_settings(gt, &table))
347353 return;
348354
349355 /* Set unused values to PTE */
@@ -352,12 +358,16 @@ void intel_mocs_init_engine(struct intel_engine_cs *engine) · 2 unmodified lines
352358 for (index = 0; index < table.size; index++) {
353359 u32 value = get_entry_control(&table, index);
354360
355− I915_WRITE(mocs_register(engine->id, index), value);
361+ intel_uncore_write_fw(uncore,
362+ mocs_register(engine->id, index),
363+ value);
356364 }
357365
358366 /* All remaining entries are also unused */
359367 for (; index < table.n_entries; index++)
360− I915_WRITE(mocs_register(engine->id, index), unused_value);
368+ intel_uncore_write_fw(uncore,
369+ mocs_register(engine->id, index),
370+ unused_value);
361371 }
362372
363373 /**
@@ -490,7 +500,7 @@ static int emit_mocs_l3cc_table(struct i915_request *rq, · 126 unmodified lines
490500
491501 /**
492502 * intel_mocs_init_l3cc_table() - program the mocs control table
493− * @dev_priv: i915 device private
503+ * @gt: the intel_gt container
494504 *
495505 * This function simply programs the mocs registers for the given table
496506 * starting at the given address. This register set is programmed in pairs.
@@ -502,13 +512,14 @@ static int emit_mocs_l3cc_table(struct i915_request *rq, · 5 unmodified lines
502512 *
503513 * Return: Nothing.
504514 */
505−void intel_mocs_init_l3cc_table(struct drm_i915_private *dev_priv)
515+void intel_mocs_init_l3cc_table(struct intel_gt *gt)
506516 {
517+ struct intel_uncore *uncore = gt->uncore;
507518 struct drm_i915_mocs_table table;
508519 unsigned int i;
509520 u16 unused_value;
510521
511− if (!get_mocs_settings(dev_priv, &table))
522+ if (!get_mocs_settings(gt, &table))
512523 return;
513524
514525 /* Set unused values to PTE */
@@ -518,23 +529,27 @@ void intel_mocs_init_l3cc_table(struct drm_i915_private *dev_priv) · 3 unmodified lines
518529 u16 low = get_entry_l3cc(&table, 2 * i);
519530 u16 high = get_entry_l3cc(&table, 2 * i + 1);
520531
521− I915_WRITE(GEN9_LNCFCMOCS(i),
522− l3cc_combine(&table, low, high));
532+ intel_uncore_write(uncore,
533+ GEN9_LNCFCMOCS(i),
534+ l3cc_combine(&table, low, high));
523535 }
524536
525537 /* Odd table size - 1 left over */
526538 if (table.size & 0x01) {
527539 u16 low = get_entry_l3cc(&table, 2 * i);
528540
529− I915_WRITE(GEN9_LNCFCMOCS(i),
530− l3cc_combine(&table, low, unused_value));
541+ intel_uncore_write(uncore,
542+ GEN9_LNCFCMOCS(i),
543+ l3cc_combine(&table, low, unused_value));
531544 i++;
532545 }
533546
534547 /* All remaining entries are also unused */
535548 for (; i < table.n_entries / 2; i++)
536− I915_WRITE(GEN9_LNCFCMOCS(i),
537− l3cc_combine(&table, unused_value, unused_value));
549+ intel_uncore_write(uncore,
550+ GEN9_LNCFCMOCS(i),
551+ l3cc_combine(&table, unused_value,
552+ unused_value));
538553 }
539554
540555 /**
@@ -553,12 +568,15 @@ void intel_mocs_init_l3cc_table(struct drm_i915_private *dev_priv) · 12 unmodified lines
553568 *
554569 * Return: 0 on success, otherwise the error status.
555570 */
556−int intel_rcs_context_init_mocs(struct i915_request *rq)
571+int intel_mocs_emit(struct i915_request *rq)
557572 {
558573 struct drm_i915_mocs_table t;
559574 int ret;
560575
561− if (get_mocs_settings(rq->i915, &t)) {
576+ if (rq->engine->class != RENDER_CLASS)
577+ return 0;
578+
579+ if (get_mocs_settings(rq->engine->gt, &t)) {
562580 /* Program the RCS control registers */
563581 ret = emit_mocs_control_table(rq, &t);
564582 if (ret)
drivers/gpu/drm/i915/gt/intel_mocs.h+4−2
@@ -52,9 +52,11 @@ · 51 unmodified lines
5252 struct drm_i915_private;
5353 struct i915_request;
5454 struct intel_engine_cs;
55+struct intel_gt;
5556
56−int intel_rcs_context_init_mocs(struct i915_request *rq);
57−void intel_mocs_init_l3cc_table(struct drm_i915_private *dev_priv);
57+void intel_mocs_init_l3cc_table(struct intel_gt *gt);
5858 void intel_mocs_init_engine(struct intel_engine_cs *engine);
5959
60+int intel_mocs_emit(struct i915_request *rq);
61+
6062 #endif
drivers/gpu/drm/i915/gt/intel_renderstate.crenamed from drivers/gpu/drm/i915/i915_gem_render_state.c+6−7
@@ -26,10 +26,9 @@ · 25 unmodified lines
2626 */
2727
2828 #include "i915_drv.h"
29−#include "i915_gem_render_state.h"
3029 #include "intel_renderstate.h"
3130
32−struct intel_render_state {
31+struct intel_renderstate {
3332 const struct intel_renderstate_rodata *rodata;
3433 struct drm_i915_gem_object *obj;
3534 struct i915_vma *vma;
@@ -42,7 +41,7 @@ struct intel_render_state { · 6 unmodified lines
4241 static const struct intel_renderstate_rodata *
4342 render_state_get_rodata(const struct intel_engine_cs *engine)
4443 {
45− if (engine->id != RCS0)
44+ if (engine->class != RENDER_CLASS)
4645 return NULL;
4746
4847 switch (INTEL_GEN(engine->i915)) {
@@ -75,7 +74,7 @@ render_state_get_rodata(const struct intel_engine_cs *engine) · 26 unmodified lines
7574 (batch)[(i)++] = (val); \
7675 } while(0)
7776
78−static int render_state_setup(struct intel_render_state *so,
77+static int render_state_setup(struct intel_renderstate *so,
7978 struct drm_i915_private *i915)
8079 {
8180 const struct intel_renderstate_rodata *rodata = so->rodata;
@@ -177,10 +176,10 @@ err: · 95 unmodified lines
177176
178177 #undef OUT_BATCH
179178
180−int i915_gem_render_state_emit(struct i915_request *rq)
179+int intel_renderstate_emit(struct i915_request *rq)
181180 {
182181 struct intel_engine_cs *engine = rq->engine;
183− struct intel_render_state so = {}; /* keep the compiler happy */
182+ struct intel_renderstate so = {}; /* keep the compiler happy */
184183 int err;
185184
186185 so.rodata = render_state_get_rodata(engine);
@@ -194,7 +193,7 @@ int i915_gem_render_state_emit(struct i915_request *rq) · 7 unmodified lines
194193 if (IS_ERR(so.obj))
195194 return PTR_ERR(so.obj);
196195
197− so.vma = i915_vma_instance(so.obj, &engine->i915->ggtt.vm, NULL);
196+ so.vma = i915_vma_instance(so.obj, &engine->gt->ggtt->vm, NULL);
198197 if (IS_ERR(so.vma)) {
199198 err = PTR_ERR(so.vma);
200199 goto err_obj;
drivers/gpu/drm/i915/gt/intel_renderstate.hrenamed from drivers/gpu/drm/i915/intel_renderstate.h+7−3
@@ -21,11 +21,13 @@ · 20 unmodified lines
2121 * DEALINGS IN THE SOFTWARE.
2222 */
2323
24−#ifndef _INTEL_RENDERSTATE_H
25−#define _INTEL_RENDERSTATE_H
24+#ifndef _INTEL_RENDERSTATE_H_
25+#define _INTEL_RENDERSTATE_H_
2626
2727 #include <linux/types.h>
2828
29+struct i915_request;
30+
2931 struct intel_renderstate_rodata {
3032 const u32 *reloc;
3133 const u32 *batch;
@@ -44,4 +46,6 @@ extern const struct intel_renderstate_rodata gen7_null_state; · 12 unmodified lines
4446 extern const struct intel_renderstate_rodata gen8_null_state;
4547 extern const struct intel_renderstate_rodata gen9_null_state;
4648
47−#endif /* INTEL_RENDERSTATE_H */
49+int intel_renderstate_emit(struct i915_request *rq);
50+
51+#endif /* _INTEL_RENDERSTATE_H_ */
drivers/gpu/drm/i915/gt/intel_reset.c+218−406
@@ -15,26 +15,17 @@ · 14 unmodified lines
1515 #include "i915_gpu_error.h"
1616 #include "i915_irq.h"
1717 #include "intel_engine_pm.h"
18+#include "intel_gt.h"
1819 #include "intel_gt_pm.h"
1920 #include "intel_reset.h"
2021
21−#include "intel_guc.h"
22+#include "uc/intel_guc.h"
2223
2324 #define RESET_MAX_RETRIES 3
2425
2526 /* XXX How to handle concurrent GGTT updates using tiling registers? */
2627 #define RESET_UNDER_STOP_MACHINE 0
2728
28−static void rmw_set(struct intel_uncore *uncore, i915_reg_t reg, u32 set)
29−{
30− intel_uncore_rmw(uncore, reg, 0, set);
31−}
32−
33−static void rmw_clear(struct intel_uncore *uncore, i915_reg_t reg, u32 clr)
34−{
35− intel_uncore_rmw(uncore, reg, clr, 0);
36−}
37−
3829 static void rmw_set_fw(struct intel_uncore *uncore, i915_reg_t reg, u32 set)
3930 {
4031 intel_uncore_rmw_fw(uncore, reg, 0, set);
@@ -123,7 +114,7 @@ static void context_mark_innocent(struct i915_gem_context *ctx) · 82 unmodified lines
123114 atomic_inc(&ctx->active_count);
124115 }
125116
126−void i915_reset_request(struct i915_request *rq, bool guilty)
117+void __i915_request_reset(struct i915_request *rq, bool guilty)
127118 {
128119 GEM_TRACE("%s rq=%llx:%lld, guilty? %s\n",
129120 rq->engine->name,
@@ -144,48 +135,6 @@ void i915_reset_request(struct i915_request *rq, bool guilty) · 14 unmodified lines
144135 }
145136 }
146137
147−static void gen3_stop_engine(struct intel_engine_cs *engine)
148−{
149− struct intel_uncore *uncore = engine->uncore;
150− const u32 base = engine->mmio_base;
151−
152− GEM_TRACE("%s\n", engine->name);
153−
154− if (intel_engine_stop_cs(engine))
155− GEM_TRACE("%s: timed out on STOP_RING\n", engine->name);
156−
157− intel_uncore_write_fw(uncore,
158− RING_HEAD(base),
159− intel_uncore_read_fw(uncore, RING_TAIL(base)));
160− intel_uncore_posting_read_fw(uncore, RING_HEAD(base)); /* paranoia */
161−
162− intel_uncore_write_fw(uncore, RING_HEAD(base), 0);
163− intel_uncore_write_fw(uncore, RING_TAIL(base), 0);
164− intel_uncore_posting_read_fw(uncore, RING_TAIL(base));
165−
166− /* The ring must be empty before it is disabled */
167− intel_uncore_write_fw(uncore, RING_CTL(base), 0);
168−
169− /* Check acts as a post */
170− if (intel_uncore_read_fw(uncore, RING_HEAD(base)))
171− GEM_TRACE("%s: ring head [%x] not parked\n",
172− engine->name,
173− intel_uncore_read_fw(uncore, RING_HEAD(base)));
174−}
175−
176−static void i915_stop_engines(struct drm_i915_private *i915,
177− intel_engine_mask_t engine_mask)
178−{
179− struct intel_engine_cs *engine;
180− intel_engine_mask_t tmp;
181−
182− if (INTEL_GEN(i915) < 3)
183− return;
184−
185− for_each_engine_masked(engine, i915, engine_mask, tmp)
186− gen3_stop_engine(engine);
187−}
188−
189138 static bool i915_in_reset(struct pci_dev *pdev)
190139 {
191140 u8 gdrst;
@@ -194,11 +143,11 @@ static bool i915_in_reset(struct pci_dev *pdev) · 2 unmodified lines
194143 return gdrst & GRDOM_RESET_STATUS;
195144 }
196145
197−static int i915_do_reset(struct drm_i915_private *i915,
146+static int i915_do_reset(struct intel_gt *gt,
198147 intel_engine_mask_t engine_mask,
199148 unsigned int retry)
200149 {
201− struct pci_dev *pdev = i915->drm.pdev;
150+ struct pci_dev *pdev = gt->i915->drm.pdev;
202151 int err;
203152
204153 /* Assert reset for at least 20 usec, and wait for acknowledgement. */
@@ -223,22 +172,22 @@ static bool g4x_reset_complete(struct pci_dev *pdev) · 18 unmodified lines
223172 return (gdrst & GRDOM_RESET_ENABLE) == 0;
224173 }
225174
226−static int g33_do_reset(struct drm_i915_private *i915,
175+static int g33_do_reset(struct intel_gt *gt,
227176 intel_engine_mask_t engine_mask,
228177 unsigned int retry)
229178 {
230− struct pci_dev *pdev = i915->drm.pdev;
179+ struct pci_dev *pdev = gt->i915->drm.pdev;
231180
232181 pci_write_config_byte(pdev, I915_GDRST, GRDOM_RESET_ENABLE);
233182 return wait_for_atomic(g4x_reset_complete(pdev), 50);
234183 }
235184
236−static int g4x_do_reset(struct drm_i915_private *i915,
185+static int g4x_do_reset(struct intel_gt *gt,
237186 intel_engine_mask_t engine_mask,
238187 unsigned int retry)
239188 {
240− struct pci_dev *pdev = i915->drm.pdev;
241− struct intel_uncore *uncore = &i915->uncore;
189+ struct pci_dev *pdev = gt->i915->drm.pdev;
190+ struct intel_uncore *uncore = gt->uncore;
242191 int ret;
243192
244193 /* WaVcpClkGateDisableForMediaReset:ctg,elk */
@@ -270,11 +219,11 @@ out: · 25 unmodified lines
270219 return ret;
271220 }
272221
273−static int ironlake_do_reset(struct drm_i915_private *i915,
222+static int ironlake_do_reset(struct intel_gt *gt,
274223 intel_engine_mask_t engine_mask,
275224 unsigned int retry)
276225 {
277− struct intel_uncore *uncore = &i915->uncore;
226+ struct intel_uncore *uncore = gt->uncore;
278227 int ret;
279228
280229 intel_uncore_write_fw(uncore, ILK_GDSR,
@@ -306,10 +255,9 @@ out: · 25 unmodified lines
306255 }
307256
308257 /* Reset the hardware domains (GENX_GRDOM_*) specified by mask */
309−static int gen6_hw_domain_reset(struct drm_i915_private *i915,
310− u32 hw_domain_mask)
258+static int gen6_hw_domain_reset(struct intel_gt *gt, u32 hw_domain_mask)
311259 {
312− struct intel_uncore *uncore = &i915->uncore;
260+ struct intel_uncore *uncore = gt->uncore;
313261 int err;
314262
315263 /*
@@ -331,7 +279,7 @@ static int gen6_hw_domain_reset(struct drm_i915_private *i915, · 15 unmodified lines
331279 return err;
332280 }
333281
334−static int gen6_reset_engines(struct drm_i915_private *i915,
282+static int gen6_reset_engines(struct intel_gt *gt,
335283 intel_engine_mask_t engine_mask,
336284 unsigned int retry)
337285 {
@@ -351,13 +299,13 @@ static int gen6_reset_engines(struct drm_i915_private *i915, · 13 unmodified lines
351299 intel_engine_mask_t tmp;
352300
353301 hw_mask = 0;
354− for_each_engine_masked(engine, i915, engine_mask, tmp) {
302+ for_each_engine_masked(engine, gt->i915, engine_mask, tmp) {
355303 GEM_BUG_ON(engine->id >= ARRAY_SIZE(hw_engine_mask));
356304 hw_mask |= hw_engine_mask[engine->id];
357305 }
358306 }
359307
360− return gen6_hw_domain_reset(i915, hw_mask);
308+ return gen6_hw_domain_reset(gt, hw_mask);
361309 }
362310
363311 static u32 gen11_lock_sfc(struct intel_engine_cs *engine)
@@ -455,7 +403,7 @@ static void gen11_unlock_sfc(struct intel_engine_cs *engine) · 91 unmodified lines
455403 rmw_clear_fw(uncore, sfc_forced_lock, sfc_forced_lock_bit);
456404 }
457405
458−static int gen11_reset_engines(struct drm_i915_private *i915,
406+static int gen11_reset_engines(struct intel_gt *gt,
459407 intel_engine_mask_t engine_mask,
460408 unsigned int retry)
461409 {
@@ -478,17 +426,17 @@ static int gen11_reset_engines(struct drm_i915_private *i915, · 16 unmodified lines
478426 hw_mask = GEN11_GRDOM_FULL;
479427 } else {
480428 hw_mask = 0;
481− for_each_engine_masked(engine, i915, engine_mask, tmp) {
429+ for_each_engine_masked(engine, gt->i915, engine_mask, tmp) {
482430 GEM_BUG_ON(engine->id >= ARRAY_SIZE(hw_engine_mask));
483431 hw_mask |= hw_engine_mask[engine->id];
484432 hw_mask |= gen11_lock_sfc(engine);
485433 }
486434 }
487435
488− ret = gen6_hw_domain_reset(i915, hw_mask);
436+ ret = gen6_hw_domain_reset(gt, hw_mask);
489437
490438 if (engine_mask != ALL_ENGINES)
491− for_each_engine_masked(engine, i915, engine_mask, tmp)
439+ for_each_engine_masked(engine, gt->i915, engine_mask, tmp)
492440 gen11_unlock_sfc(engine);
493441
494442 return ret;
@@ -538,7 +486,7 @@ static void gen8_engine_reset_cancel(struct intel_engine_cs *engine) · 43 unmodified lines
538486 _MASKED_BIT_DISABLE(RESET_CTL_REQUEST_RESET));
539487 }
540488
541−static int gen8_reset_engines(struct drm_i915_private *i915,
489+static int gen8_reset_engines(struct intel_gt *gt,
542490 intel_engine_mask_t engine_mask,
543491 unsigned int retry)
544492 {
@@ -547,7 +495,7 @@ static int gen8_reset_engines(struct drm_i915_private *i915, · 2 unmodified lines
547495 intel_engine_mask_t tmp;
548496 int ret;
549497
550− for_each_engine_masked(engine, i915, engine_mask, tmp) {
498+ for_each_engine_masked(engine, gt->i915, engine_mask, tmp) {
551499 ret = gen8_engine_reset_prepare(engine);
552500 if (ret && !reset_non_ready)
553501 goto skip_reset;
@@ -563,23 +511,23 @@ static int gen8_reset_engines(struct drm_i915_private *i915, · 9 unmodified lines
563511 * We rather take context corruption instead of
564512 * failed reset with a wedged driver/gpu. And
565513 * active bb execution case should be covered by
566− * i915_stop_engines we have before the reset.
514+ * stop_engines() we have before the reset.
567515 */
568516 }
569517
570− if (INTEL_GEN(i915) >= 11)
571− ret = gen11_reset_engines(i915, engine_mask, retry);
518+ if (INTEL_GEN(gt->i915) >= 11)
519+ ret = gen11_reset_engines(gt, engine_mask, retry);
572520 else
573− ret = gen6_reset_engines(i915, engine_mask, retry);
521+ ret = gen6_reset_engines(gt, engine_mask, retry);
574522
575523 skip_reset:
576− for_each_engine_masked(engine, i915, engine_mask, tmp)
524+ for_each_engine_masked(engine, gt->i915, engine_mask, tmp)
577525 gen8_engine_reset_cancel(engine);
578526
579527 return ret;
580528 }
581529
582−typedef int (*reset_func)(struct drm_i915_private *,
530+typedef int (*reset_func)(struct intel_gt *,
583531 intel_engine_mask_t engine_mask,
584532 unsigned int retry);
585533
@@ -601,15 +549,14 @@ static reset_func intel_get_gpu_reset(struct drm_i915_private *i915) · 15 unmodified lines
601549 return NULL;
602550 }
603551
604−int intel_gpu_reset(struct drm_i915_private *i915,
605− intel_engine_mask_t engine_mask)
552+int __intel_gt_reset(struct intel_gt *gt, intel_engine_mask_t engine_mask)
606553 {
607554 const int retries = engine_mask == ALL_ENGINES ? RESET_MAX_RETRIES : 1;
608555 reset_func reset;
609556 int ret = -ETIMEDOUT;
610557 int retry;
611558
612− reset = intel_get_gpu_reset(i915);
559+ reset = intel_get_gpu_reset(gt->i915);
613560 if (!reset)
614561 return -ENODEV;
615562
@@ -617,31 +564,14 @@ int intel_gpu_reset(struct drm_i915_private *i915, · 1 unmodified line
617564 * If the power well sleeps during the reset, the reset
618565 * request may be dropped and never completes (causing -EIO).
619566 */
620− intel_uncore_forcewake_get(&i915->uncore, FORCEWAKE_ALL);
567+ intel_uncore_forcewake_get(gt->uncore, FORCEWAKE_ALL);
621568 for (retry = 0; ret == -ETIMEDOUT && retry < retries; retry++) {
622− /*
623− * We stop engines, otherwise we might get failed reset and a
624− * dead gpu (on elk). Also as modern gpu as kbl can suffer
625− * from system hang if batchbuffer is progressing when
626− * the reset is issued, regardless of READY_TO_RESET ack.
627− * Thus assume it is best to stop engines on all gens
628− * where we have a gpu reset.
629− *
630− * WaKBLVECSSemaphoreWaitPoll:kbl (on ALL_ENGINES)
631− *
632− * WaMediaResetMainRingCleanup:ctg,elk (presumably)
633− *
634− * FIXME: Wa for more modern gens needs to be validated
635− */
636− if (retry)
637− i915_stop_engines(i915, engine_mask);
638−
639569 GEM_TRACE("engine_mask=%x\n", engine_mask);
640570 preempt_disable();
641− ret = reset(i915, engine_mask, retry);
571+ ret = reset(gt, engine_mask, retry);
642572 preempt_enable();
643573 }
644− intel_uncore_forcewake_put(&i915->uncore, FORCEWAKE_ALL);
574+ intel_uncore_forcewake_put(gt->uncore, FORCEWAKE_ALL);
645575
646576 return ret;
647577 }
@@ -659,17 +589,17 @@ bool intel_has_reset_engine(struct drm_i915_private *i915) · 11 unmodified lines
659589 return INTEL_INFO(i915)->has_reset_engine && i915_modparams.reset >= 2;
660590 }
661591
662−int intel_reset_guc(struct drm_i915_private *i915)
592+int intel_reset_guc(struct intel_gt *gt)
663593 {
664594 u32 guc_domain =
665− INTEL_GEN(i915) >= 11 ? GEN11_GRDOM_GUC : GEN9_GRDOM_GUC;
595+ INTEL_GEN(gt->i915) >= 11 ? GEN11_GRDOM_GUC : GEN9_GRDOM_GUC;
666596 int ret;
667597
668− GEM_BUG_ON(!HAS_GUC(i915));
598+ GEM_BUG_ON(!HAS_GT_UC(gt->i915));
669599
670− intel_uncore_forcewake_get(&i915->uncore, FORCEWAKE_ALL);
671− ret = gen6_hw_domain_reset(i915, guc_domain);
672− intel_uncore_forcewake_put(&i915->uncore, FORCEWAKE_ALL);
600+ intel_uncore_forcewake_get(gt->uncore, FORCEWAKE_ALL);
601+ ret = gen6_hw_domain_reset(gt, guc_domain);
602+ intel_uncore_forcewake_put(gt->uncore, FORCEWAKE_ALL);
673603
674604 return ret;
675605 }
@@ -691,56 +621,55 @@ static void reset_prepare_engine(struct intel_engine_cs *engine) · 15 unmodified lines
691621 engine->reset.prepare(engine);
692622 }
693623
694−static void revoke_mmaps(struct drm_i915_private *i915)
624+static void revoke_mmaps(struct intel_gt *gt)
695625 {
696626 int i;
697627
698− for (i = 0; i < i915->ggtt.num_fences; i++) {
628+ for (i = 0; i < gt->ggtt->num_fences; i++) {
699629 struct drm_vma_offset_node *node;
700630 struct i915_vma *vma;
701631 u64 vma_offset;
702632
703− vma = READ_ONCE(i915->ggtt.fence_regs[i].vma);
633+ vma = READ_ONCE(gt->ggtt->fence_regs[i].vma);
704634 if (!vma)
705635 continue;
706636
707637 if (!i915_vma_has_userfault(vma))
708638 continue;
709639
710− GEM_BUG_ON(vma->fence != &i915->ggtt.fence_regs[i]);
640+ GEM_BUG_ON(vma->fence != &gt->ggtt->fence_regs[i]);
711641 node = &vma->obj->base.vma_node;
712642 vma_offset = vma->ggtt_view.partial.offset << PAGE_SHIFT;
713− unmap_mapping_range(i915->drm.anon_inode->i_mapping,
643+ unmap_mapping_range(gt->i915->drm.anon_inode->i_mapping,
714644 drm_vma_node_offset_addr(node) + vma_offset,
715645 vma->size,
716646 1);
717647 }
718648 }
719649
720−static intel_engine_mask_t reset_prepare(struct drm_i915_private *i915)
650+static intel_engine_mask_t reset_prepare(struct intel_gt *gt)
721651 {
722652 struct intel_engine_cs *engine;
723653 intel_engine_mask_t awake = 0;
724654 enum intel_engine_id id;
725655
726− for_each_engine(engine, i915, id) {
656+ for_each_engine(engine, gt->i915, id) {
727657 if (intel_engine_pm_get_if_awake(engine))
728658 awake |= engine->mask;
729659 reset_prepare_engine(engine);
730660 }
731661
732− intel_uc_reset_prepare(i915);
662+ intel_uc_reset_prepare(&gt->uc);
733663
734664 return awake;
735665 }
736666
737−static void gt_revoke(struct drm_i915_private *i915)
667+static void gt_revoke(struct intel_gt *gt)
738668 {
739− revoke_mmaps(i915);
669+ revoke_mmaps(gt);
740670 }
741671
742−static int gt_reset(struct drm_i915_private *i915,
743− intel_engine_mask_t stalled_mask)
672+static int gt_reset(struct intel_gt *gt, intel_engine_mask_t stalled_mask)
744673 {
745674 struct intel_engine_cs *engine;
746675 enum intel_engine_id id;
@@ -750,14 +679,14 @@ static int gt_reset(struct drm_i915_private *i915, · 3 unmodified lines
750679 * Everything depends on having the GTT running, so we need to start
751680 * there.
752681 */
753− err = i915_ggtt_enable_hw(i915);
682+ err = i915_ggtt_enable_hw(gt->i915);
754683 if (err)
755684 return err;
756685
757− for_each_engine(engine, i915, id)
758− intel_engine_reset(engine, stalled_mask & engine->mask);
686+ for_each_engine(engine, gt->i915, id)
687+ __intel_engine_reset(engine, stalled_mask & engine->mask);
759688
760− i915_gem_restore_fences(i915);
689+ i915_gem_restore_fences(gt->i915);
761690
762691 return err;
763692 }
@@ -770,13 +699,12 @@ static void reset_finish_engine(struct intel_engine_cs *engine) · 6 unmodified lines
770699 intel_engine_signal_breadcrumbs(engine);
771700 }
772701
773−static void reset_finish(struct drm_i915_private *i915,
774− intel_engine_mask_t awake)
702+static void reset_finish(struct intel_gt *gt, intel_engine_mask_t awake)
775703 {
776704 struct intel_engine_cs *engine;
777705 enum intel_engine_id id;
778706
779− for_each_engine(engine, i915, id) {
707+ for_each_engine(engine, gt->i915, id) {
780708 reset_finish_engine(engine);
781709 if (awake & engine->mask)
782710 intel_engine_pm_put(engine);
@@ -800,20 +728,19 @@ static void nop_submit_request(struct i915_request *request) · 17 unmodified lines
800728 intel_engine_queue_breadcrumbs(engine);
801729 }
802730
803−static void __i915_gem_set_wedged(struct drm_i915_private *i915)
731+static void __intel_gt_set_wedged(struct intel_gt *gt)
804732 {
805− struct i915_gpu_error *error = &i915->gpu_error;
806733 struct intel_engine_cs *engine;
807734 intel_engine_mask_t awake;
808735 enum intel_engine_id id;
809736
810− if (test_bit(I915_WEDGED, &error->flags))
737+ if (test_bit(I915_WEDGED, &gt->reset.flags))
811738 return;
812739
813− if (GEM_SHOW_DEBUG() && !intel_engines_are_idle(i915)) {
740+ if (GEM_SHOW_DEBUG() && !intel_engines_are_idle(gt)) {
814741 struct drm_printer p = drm_debug_printer(__func__);
815742
816− for_each_engine(engine, i915, id)
743+ for_each_engine(engine, gt->i915, id)
817744 intel_engine_dump(engine, &p, "%s\n", engine->name);
818745 }
819746
@@ -824,17 +751,17 @@ static void __i915_gem_set_wedged(struct drm_i915_private *i915) · 4 unmodified lines
824751 * rolling the global seqno forward (since this would complete requests
825752 * for which we haven't set the fence error to EIO yet).
826753 */
827− awake = reset_prepare(i915);
754+ awake = reset_prepare(gt);
828755
829756 /* Even if the GPU reset fails, it should still stop the engines */
830− if (!INTEL_INFO(i915)->gpu_reset_clobbers_display)
831− intel_gpu_reset(i915, ALL_ENGINES);
757+ if (!INTEL_INFO(gt->i915)->gpu_reset_clobbers_display)
758+ __intel_gt_reset(gt, ALL_ENGINES);
832759
833− for_each_engine(engine, i915, id) {
760+ for_each_engine(engine, gt->i915, id) {
834761 engine->submit_request = nop_submit_request;
835762 engine->schedule = NULL;
836763 }
837− i915->caps.scheduler = 0;
764+ gt->i915->caps.scheduler = 0;
838765
839766 /*
840767 * Make sure no request can slip through without getting completed by
@@ -842,37 +769,36 @@ static void __i915_gem_set_wedged(struct drm_i915_private *i915) · 1 unmodified line
842769 * in nop_submit_request.
843770 */
844771 synchronize_rcu_expedited();
845− set_bit(I915_WEDGED, &error->flags);
772+ set_bit(I915_WEDGED, &gt->reset.flags);
846773
847774 /* Mark all executing requests as skipped */
848− for_each_engine(engine, i915, id)
775+ for_each_engine(engine, gt->i915, id)
849776 engine->cancel_requests(engine);
850777
851− reset_finish(i915, awake);
778+ reset_finish(gt, awake);
852779
853780 GEM_TRACE("end\n");
854781 }
855782
856−void i915_gem_set_wedged(struct drm_i915_private *i915)
783+void intel_gt_set_wedged(struct intel_gt *gt)
857784 {
858− struct i915_gpu_error *error = &i915->gpu_error;
859785 intel_wakeref_t wakeref;
860786
861− mutex_lock(&error->wedge_mutex);
862− with_intel_runtime_pm(&i915->runtime_pm, wakeref)
863− __i915_gem_set_wedged(i915);
864− mutex_unlock(&error->wedge_mutex);
787+ mutex_lock(&gt->reset.mutex);
788+ with_intel_runtime_pm(&gt->i915->runtime_pm, wakeref)
789+ __intel_gt_set_wedged(gt);
790+ mutex_unlock(&gt->reset.mutex);
865791 }
866792
867−static bool __i915_gem_unset_wedged(struct drm_i915_private *i915)
793+static bool __intel_gt_unset_wedged(struct intel_gt *gt)
868794 {
869− struct i915_gpu_error *error = &i915->gpu_error;
870− struct i915_timeline *tl;
795+ struct intel_gt_timelines *timelines = &gt->timelines;
796+ struct intel_timeline *tl;
871797
872− if (!test_bit(I915_WEDGED, &error->flags))
798+ if (!test_bit(I915_WEDGED, &gt->reset.flags))
873799 return true;
874800
875− if (!i915->gt.scratch) /* Never full initialised, recovery impossible */
801+ if (!gt->scratch) /* Never full initialised, recovery impossible */
876802 return false;
877803
878804 GEM_TRACE("start\n");
@@ -887,8 +813,8 @@ static bool __i915_gem_unset_wedged(struct drm_i915_private *i915) · 8 unmodified lines
887813 *
888814 * No more can be submitted until we reset the wedged bit.
889815 */
890− mutex_lock(&i915->gt.timelines.mutex);
891− list_for_each_entry(tl, &i915->gt.timelines.active_list, link) {
816+ mutex_lock(&timelines->mutex);
817+ list_for_each_entry(tl, &timelines->active_list, link) {
892818 struct i915_request *rq;
893819
894820 rq = i915_active_request_get_unlocked(&tl->last_request);
@@ -905,9 +831,9 @@ static bool __i915_gem_unset_wedged(struct drm_i915_private *i915) · 10 unmodified lines
905831 dma_fence_default_wait(&rq->fence, false, MAX_SCHEDULE_TIMEOUT);
906832 i915_request_put(rq);
907833 }
908− mutex_unlock(&i915->gt.timelines.mutex);
834+ mutex_unlock(&timelines->mutex);
909835
910− intel_gt_sanitize(i915, false);
836+ intel_gt_sanitize(gt, false);
911837
912838 /*
913839 * Undo nop_submit_request. We prevent all new i915 requests from
@@ -918,53 +844,51 @@ static bool __i915_gem_unset_wedged(struct drm_i915_private *i915) · 4 unmodified lines
918844 * the nop_submit_request on reset, we can do this from normal
919845 * context and do not require stop_machine().
920846 */
921− intel_engines_reset_default_submission(i915);
847+ intel_engines_reset_default_submission(gt);
922848
923849 GEM_TRACE("end\n");
924850
925851 smp_mb__before_atomic(); /* complete takeover before enabling execbuf */
926− clear_bit(I915_WEDGED, &i915->gpu_error.flags);
852+ clear_bit(I915_WEDGED, &gt->reset.flags);
927853
928854 return true;
929855 }
930856
931−bool i915_gem_unset_wedged(struct drm_i915_private *i915)
857+bool intel_gt_unset_wedged(struct intel_gt *gt)
932858 {
933− struct i915_gpu_error *error = &i915->gpu_error;
934859 bool result;
935860
936− mutex_lock(&error->wedge_mutex);
937− result = __i915_gem_unset_wedged(i915);
938− mutex_unlock(&error->wedge_mutex);
861+ mutex_lock(&gt->reset.mutex);
862+ result = __intel_gt_unset_wedged(gt);
863+ mutex_unlock(&gt->reset.mutex);
939864
940865 return result;
941866 }
942867
943−static int do_reset(struct drm_i915_private *i915,
944− intel_engine_mask_t stalled_mask)
868+static int do_reset(struct intel_gt *gt, intel_engine_mask_t stalled_mask)
945869 {
946870 int err, i;
947871
948− gt_revoke(i915);
872+ gt_revoke(gt);
949873
950− err = intel_gpu_reset(i915, ALL_ENGINES);
874+ err = __intel_gt_reset(gt, ALL_ENGINES);
951875 for (i = 0; err && i < RESET_MAX_RETRIES; i++) {
952876 msleep(10 * (i + 1));
953− err = intel_gpu_reset(i915, ALL_ENGINES);
877+ err = __intel_gt_reset(gt, ALL_ENGINES);
954878 }
955879 if (err)
956880 return err;
957881
958− return gt_reset(i915, stalled_mask);
882+ return gt_reset(gt, stalled_mask);
959883 }
960884
961−static int resume(struct drm_i915_private *i915)
885+static int resume(struct intel_gt *gt)
962886 {
963887 struct intel_engine_cs *engine;
964888 enum intel_engine_id id;
965889 int ret;
966890
967− for_each_engine(engine, i915, id) {
891+ for_each_engine(engine, gt->i915, id) {
968892 ret = engine->resume(engine);
969893 if (ret)
970894 return ret;
@@ -974,8 +898,8 @@ static int resume(struct drm_i915_private *i915) · 3 unmodified lines
974898 }
975899
976900 /**
977− * i915_reset - reset chip after a hang
978− * @i915: #drm_i915_private to reset
901+ * intel_gt_reset - reset chip after a hang
902+ * @gt: #intel_gt to reset
979903 * @stalled_mask: mask of the stalled engines with the guilty requests
980904 * @reason: user error message for why we are resetting
981905 *
@@ -990,50 +914,50 @@ static int resume(struct drm_i915_private *i915) · 8 unmodified lines
990914 * - re-init interrupt state
991915 * - re-init display
992916 */
993−void i915_reset(struct drm_i915_private *i915,
994− intel_engine_mask_t stalled_mask,
995− const char *reason)
917+void intel_gt_reset(struct intel_gt *gt,
918+ intel_engine_mask_t stalled_mask,
919+ const char *reason)
996920 {
997− struct i915_gpu_error *error = &i915->gpu_error;
998921 intel_engine_mask_t awake;
999922 int ret;
1000923
1001− GEM_TRACE("flags=%lx\n", error->flags);
924+ GEM_TRACE("flags=%lx\n", gt->reset.flags);
1002925
1003926 might_sleep();
1004− GEM_BUG_ON(!test_bit(I915_RESET_BACKOFF, &error->flags));
1005− mutex_lock(&error->wedge_mutex);
927+ GEM_BUG_ON(!test_bit(I915_RESET_BACKOFF, &gt->reset.flags));
928+ mutex_lock(&gt->reset.mutex);
1006929
1007930 /* Clear any previous failed attempts at recovery. Time to try again. */
1008− if (!__i915_gem_unset_wedged(i915))
931+ if (!__intel_gt_unset_wedged(gt))
1009932 goto unlock;
1010933
1011934 if (reason)
1012− dev_notice(i915->drm.dev, "Resetting chip for %s\n", reason);
1013− error->reset_count++;
935+ dev_notice(gt->i915->drm.dev,
936+ "Resetting chip for %s\n", reason);
937+ atomic_inc(&gt->i915->gpu_error.reset_count);
1014938
1015− awake = reset_prepare(i915);
939+ awake = reset_prepare(gt);
1016940
1017− if (!intel_has_gpu_reset(i915)) {
941+ if (!intel_has_gpu_reset(gt->i915)) {
1018942 if (i915_modparams.reset)
1019− dev_err(i915->drm.dev, "GPU reset not supported\n");
943+ dev_err(gt->i915->drm.dev, "GPU reset not supported\n");
1020944 else
1021945 DRM_DEBUG_DRIVER("GPU reset disabled\n");
1022946 goto error;
1023947 }
1024948
1025− if (INTEL_INFO(i915)->gpu_reset_clobbers_display)
1026− intel_runtime_pm_disable_interrupts(i915);
949+ if (INTEL_INFO(gt->i915)->gpu_reset_clobbers_display)
950+ intel_runtime_pm_disable_interrupts(gt->i915);
1027951
1028− if (do_reset(i915, stalled_mask)) {
1029− dev_err(i915->drm.dev, "Failed to reset chip\n");
952+ if (do_reset(gt, stalled_mask)) {
953+ dev_err(gt->i915->drm.dev, "Failed to reset chip\n");
1030954 goto taint;
1031955 }
1032956
1033− if (INTEL_INFO(i915)->gpu_reset_clobbers_display)
1034− intel_runtime_pm_enable_interrupts(i915);
957+ if (INTEL_INFO(gt->i915)->gpu_reset_clobbers_display)
958+ intel_runtime_pm_enable_interrupts(gt->i915);
1035959
1036− intel_overlay_reset(i915);
960+ intel_overlay_reset(gt->i915);
1037961
1038962 /*
1039963 * Next we need to restore the context, but we don't use those
@@ -1043,23 +967,23 @@ void i915_reset(struct drm_i915_private *i915, · 3 unmodified lines
1043967 * was running at the time of the reset (i.e. we weren't VT
1044968 * switched away).
1045969 */
1046− ret = i915_gem_init_hw(i915);
970+ ret = i915_gem_init_hw(gt->i915);
1047971 if (ret) {
1048972 DRM_ERROR("Failed to initialise HW following reset (%d)\n",
1049973 ret);
1050974 goto taint;
1051975 }
1052976
1053− ret = resume(i915);
977+ ret = resume(gt);
1054978 if (ret)
1055979 goto taint;
1056980
1057− i915_queue_hangcheck(i915);
981+ intel_gt_queue_hangcheck(gt);
1058982
1059983 finish:
1060− reset_finish(i915, awake);
984+ reset_finish(gt, awake);
1061985 unlock:
1062− mutex_unlock(&error->wedge_mutex);
986+ mutex_unlock(&gt->reset.mutex);
1063987 return;
1064988
1065989 taint:
@@ -1077,18 +1001,17 @@ taint: · 11 unmodified lines
10771001 */
10781002 add_taint_for_CI(TAINT_WARN);
10791003 error:
1080− __i915_gem_set_wedged(i915);
1004+ __intel_gt_set_wedged(gt);
10811005 goto finish;
10821006 }
10831007
1084−static inline int intel_gt_reset_engine(struct drm_i915_private *i915,
1085− struct intel_engine_cs *engine)
1008+static inline int intel_gt_reset_engine(struct intel_engine_cs *engine)
10861009 {
1087− return intel_gpu_reset(i915, engine->mask);
1010+ return __intel_gt_reset(engine->gt, engine->mask);
10881011 }
10891012
10901013 /**
1091− * i915_reset_engine - reset GPU engine to recover from a hang
1014+ * intel_engine_reset - reset GPU engine to recover from a hang
10921015 * @engine: engine to reset
10931016 * @msg: reason for GPU reset; or NULL for no dev_notice()
10941017 *
@@ -1100,13 +1023,13 @@ static inline int intel_gt_reset_engine(struct drm_i915_private *i915, · 5 unmodified lines
11001023 * - reset engine (which will force the engine to idle)
11011024 * - re-init/configure engine
11021025 */
1103−int i915_reset_engine(struct intel_engine_cs *engine, const char *msg)
1026+int intel_engine_reset(struct intel_engine_cs *engine, const char *msg)
11041027 {
1105− struct i915_gpu_error *error = &engine->i915->gpu_error;
1028+ struct intel_gt *gt = engine->gt;
11061029 int ret;
11071030
1108− GEM_TRACE("%s flags=%lx\n", engine->name, error->flags);
1109− GEM_BUG_ON(!test_bit(I915_RESET_ENGINE + engine->id, &error->flags));
1031+ GEM_TRACE("%s flags=%lx\n", engine->name, gt->reset.flags);
1032+ GEM_BUG_ON(!test_bit(I915_RESET_ENGINE + engine->id, &gt->reset.flags));
11101033
11111034 if (!intel_engine_pm_get_if_awake(engine))
11121035 return 0;
@@ -1116,16 +1039,16 @@ int i915_reset_engine(struct intel_engine_cs *engine, const char *msg) · 3 unmodified lines
11161039 if (msg)
11171040 dev_notice(engine->i915->drm.dev,
11181041 "Resetting %s for %s\n", engine->name, msg);
1119− error->reset_engine_count[engine->id]++;
1042+ atomic_inc(&engine->i915->gpu_error.reset_engine_count[engine->uabi_class]);
11201043
1121− if (!engine->i915->guc.execbuf_client)
1122− ret = intel_gt_reset_engine(engine->i915, engine);
1044+ if (!engine->gt->uc.guc.execbuf_client)
1045+ ret = intel_gt_reset_engine(engine);
11231046 else
1124− ret = intel_guc_reset_engine(&engine->i915->guc, engine);
1047+ ret = intel_guc_reset_engine(&engine->gt->uc.guc, engine);
11251048 if (ret) {
11261049 /* If we fail here, we expect to fallback to a global reset */
11271050 DRM_DEBUG_DRIVER("%sFailed to reset %s, ret=%d\n",
1128− engine->i915->guc.execbuf_client ? "GuC " : "",
1051+ engine->gt->uc.guc.execbuf_client ? "GuC " : "",
11291052 engine->name, ret);
11301053 goto out;
11311054 }
@@ -1135,7 +1058,7 @@ int i915_reset_engine(struct intel_engine_cs *engine, const char *msg) · 3 unmodified lines
11351058 * active request and can drop it, adjust head to skip the offending
11361059 * request to resume executing remaining requests in the queue.
11371060 */
1138− intel_engine_reset(engine, true);
1061+ __intel_engine_reset(engine, true);
11391062
11401063 /*
11411064 * The engine and its registers (and workarounds in case of render)
@@ -1151,16 +1074,15 @@ out: · 9 unmodified lines
11511074 return ret;
11521075 }
11531076
1154−static void i915_reset_device(struct drm_i915_private *i915,
1155− u32 engine_mask,
1156− const char *reason)
1077+static void intel_gt_reset_global(struct intel_gt *gt,
1078+ u32 engine_mask,
1079+ const char *reason)
11571080 {
1158− struct i915_gpu_error *error = &i915->gpu_error;
1159− struct kobject *kobj = &i915->drm.primary->kdev->kobj;
1081+ struct kobject *kobj = &gt->i915->drm.primary->kdev->kobj;
11601082 char *error_event[] = { I915_ERROR_UEVENT "=1", NULL };
11611083 char *reset_event[] = { I915_RESET_UEVENT "=1", NULL };
11621084 char *reset_done_event[] = { I915_ERROR_UEVENT "=0", NULL };
1163− struct i915_wedge_me w;
1085+ struct intel_wedge_me w;
11641086
11651087 kobject_uevent_env(kobj, KOBJ_CHANGE, error_event);
11661088
@@ -1168,137 +1090,24 @@ static void i915_reset_device(struct drm_i915_private *i915, · 1 unmodified line
11681090 kobject_uevent_env(kobj, KOBJ_CHANGE, reset_event);
11691091
11701092 /* Use a watchdog to ensure that our reset completes */
1171− i915_wedge_on_timeout(&w, i915, 5 * HZ) {
1172− intel_prepare_reset(i915);
1093+ intel_wedge_on_timeout(&w, gt, 5 * HZ) {
1094+ intel_prepare_reset(gt->i915);
11731095
11741096 /* Flush everyone using a resource about to be clobbered */
1175− synchronize_srcu_expedited(&error->reset_backoff_srcu);
1097+ synchronize_srcu_expedited(&gt->reset.backoff_srcu);
11761098
1177− i915_reset(i915, engine_mask, reason);
1099+ intel_gt_reset(gt, engine_mask, reason);
11781100
1179− intel_finish_reset(i915);
1101+ intel_finish_reset(gt->i915);
11801102 }
11811103
1182− if (!test_bit(I915_WEDGED, &error->flags))
1104+ if (!test_bit(I915_WEDGED, &gt->reset.flags))
11831105 kobject_uevent_env(kobj, KOBJ_CHANGE, reset_done_event);
11841106 }
11851107
1186−static void clear_register(struct intel_uncore *uncore, i915_reg_t reg)
1187−{
1188− intel_uncore_rmw(uncore, reg, 0, 0);
1189−}
1190−
1191−static void gen8_clear_engine_error_register(struct intel_engine_cs *engine)
1192−{
1193− GEN6_RING_FAULT_REG_RMW(engine, RING_FAULT_VALID, 0);
1194− GEN6_RING_FAULT_REG_POSTING_READ(engine);
1195−}
1196−
1197−static void clear_error_registers(struct drm_i915_private *i915,
1198− intel_engine_mask_t engine_mask)
1199−{
1200− struct intel_uncore *uncore = &i915->uncore;
1201− u32 eir;
1202−
1203− if (!IS_GEN(i915, 2))
1204− clear_register(uncore, PGTBL_ER);
1205−
1206− if (INTEL_GEN(i915) < 4)
1207− clear_register(uncore, IPEIR(RENDER_RING_BASE));
1208− else
1209− clear_register(uncore, IPEIR_I965);
1210−
1211− clear_register(uncore, EIR);
1212− eir = intel_uncore_read(uncore, EIR);
1213− if (eir) {
1214− /*
1215− * some errors might have become stuck,
1216− * mask them.
1217− */
1218− DRM_DEBUG_DRIVER("EIR stuck: 0x%08x, masking\n", eir);
1219− rmw_set(uncore, EMR, eir);
1220− intel_uncore_write(uncore, GEN2_IIR,
1221− I915_MASTER_ERROR_INTERRUPT);
1222− }
1223−
1224− if (INTEL_GEN(i915) >= 8) {
1225− rmw_clear(uncore, GEN8_RING_FAULT_REG, RING_FAULT_VALID);
1226− intel_uncore_posting_read(uncore, GEN8_RING_FAULT_REG);
1227− } else if (INTEL_GEN(i915) >= 6) {
1228− struct intel_engine_cs *engine;
1229− enum intel_engine_id id;
1230−
1231− for_each_engine_masked(engine, i915, engine_mask, id)
1232− gen8_clear_engine_error_register(engine);
1233− }
1234−}
1235−
1236−static void gen6_check_faults(struct drm_i915_private *dev_priv)
1237−{
1238− struct intel_engine_cs *engine;
1239− enum intel_engine_id id;
1240− u32 fault;
1241−
1242− for_each_engine(engine, dev_priv, id) {
1243− fault = GEN6_RING_FAULT_REG_READ(engine);
1244− if (fault & RING_FAULT_VALID) {
1245− DRM_DEBUG_DRIVER("Unexpected fault\n"
1246− "\tAddr: 0x%08lx\n"
1247− "\tAddress space: %s\n"
1248− "\tSource ID: %d\n"
1249− "\tType: %d\n",
1250− fault & PAGE_MASK,
1251− fault & RING_FAULT_GTTSEL_MASK ? "GGTT" : "PPGTT",
1252− RING_FAULT_SRCID(fault),
1253− RING_FAULT_FAULT_TYPE(fault));
1254− }
1255− }
1256−}
1257−
1258−static void gen8_check_faults(struct drm_i915_private *dev_priv)
1259−{
1260− u32 fault = I915_READ(GEN8_RING_FAULT_REG);
1261−
1262− if (fault & RING_FAULT_VALID) {
1263− u32 fault_data0, fault_data1;
1264− u64 fault_addr;
1265−
1266− fault_data0 = I915_READ(GEN8_FAULT_TLB_DATA0);
1267− fault_data1 = I915_READ(GEN8_FAULT_TLB_DATA1);
1268− fault_addr = ((u64)(fault_data1 & FAULT_VA_HIGH_BITS) << 44) |
1269− ((u64)fault_data0 << 12);
1270−
1271− DRM_DEBUG_DRIVER("Unexpected fault\n"
1272− "\tAddr: 0x%08x_%08x\n"
1273− "\tAddress space: %s\n"
1274− "\tEngine ID: %d\n"
1275− "\tSource ID: %d\n"
1276− "\tType: %d\n",
1277− upper_32_bits(fault_addr),
1278− lower_32_bits(fault_addr),
1279− fault_data1 & FAULT_GTT_SEL ? "GGTT" : "PPGTT",
1280− GEN8_RING_FAULT_ENGINE_ID(fault),
1281− RING_FAULT_SRCID(fault),
1282− RING_FAULT_FAULT_TYPE(fault));
1283− }
1284−}
1285−
1286−void i915_check_and_clear_faults(struct drm_i915_private *i915)
1287−{
1288− /* From GEN8 onwards we only have one 'All Engine Fault Register' */
1289− if (INTEL_GEN(i915) >= 8)
1290− gen8_check_faults(i915);
1291− else if (INTEL_GEN(i915) >= 6)
1292− gen6_check_faults(i915);
1293− else
1294− return;
1295−
1296− clear_error_registers(i915, ALL_ENGINES);
1297−}
1298−
12991108 /**
1300− * i915_handle_error - handle a gpu error
1301− * @i915: i915 device private
1109+ * intel_gt_handle_error - handle a gpu error
1110+ * @gt: the intel_gt
13021111 * @engine_mask: mask representing engines that are hung
13031112 * @flags: control flags
13041113 * @fmt: Error message format string
@@ -1309,12 +1118,11 @@ void i915_check_and_clear_faults(struct drm_i915_private *i915) · 4 unmodified lines
13091118 * so userspace knows something bad happened (should trigger collection
13101119 * of a ring dump etc.).
13111120 */
1312−void i915_handle_error(struct drm_i915_private *i915,
1313− intel_engine_mask_t engine_mask,
1314− unsigned long flags,
1315− const char *fmt, ...)
1121+void intel_gt_handle_error(struct intel_gt *gt,
1122+ intel_engine_mask_t engine_mask,
1123+ unsigned long flags,
1124+ const char *fmt, ...)
13161125 {
1317− struct i915_gpu_error *error = &i915->gpu_error;
13181126 struct intel_engine_cs *engine;
13191127 intel_wakeref_t wakeref;
13201128 intel_engine_mask_t tmp;
@@ -1338,33 +1146,31 @@ void i915_handle_error(struct drm_i915_private *i915, · 17 unmodified lines
13381146 * isn't the case at least when we get here by doing a
13391147 * simulated reset via debugfs, so get an RPM reference.
13401148 */
1341− wakeref = intel_runtime_pm_get(&i915->runtime_pm);
1149+ wakeref = intel_runtime_pm_get(&gt->i915->runtime_pm);
13421150
1343− engine_mask &= INTEL_INFO(i915)->engine_mask;
1151+ engine_mask &= INTEL_INFO(gt->i915)->engine_mask;
13441152
13451153 if (flags & I915_ERROR_CAPTURE) {
1346− i915_capture_error_state(i915, engine_mask, msg);
1347− clear_error_registers(i915, engine_mask);
1154+ i915_capture_error_state(gt->i915, engine_mask, msg);
1155+ intel_gt_clear_error_registers(gt, engine_mask);
13481156 }
13491157
13501158 /*
13511159 * Try engine reset when available. We fall back to full reset if
13521160 * single reset fails.
13531161 */
1354− if (intel_has_reset_engine(i915) && !__i915_wedged(error)) {
1355− for_each_engine_masked(engine, i915, engine_mask, tmp) {
1162+ if (intel_has_reset_engine(gt->i915) && !intel_gt_is_wedged(gt)) {
1163+ for_each_engine_masked(engine, gt->i915, engine_mask, tmp) {
13561164 BUILD_BUG_ON(I915_RESET_MODESET >= I915_RESET_ENGINE);
13571165 if (test_and_set_bit(I915_RESET_ENGINE + engine->id,
1358− &error->flags))
1166+ &gt->reset.flags))
13591167 continue;
13601168
1361− if (i915_reset_engine(engine, msg) == 0)
1169+ if (intel_engine_reset(engine, msg) == 0)
13621170 engine_mask &= ~engine->mask;
13631171
1364− clear_bit(I915_RESET_ENGINE + engine->id,
1365− &error->flags);
1366− wake_up_bit(&error->flags,
1367− I915_RESET_ENGINE + engine->id);
1172+ clear_and_wake_up_bit(I915_RESET_ENGINE + engine->id,
1173+ &gt->reset.flags);
13681174 }
13691175 }
13701176
@@ -1372,9 +1178,9 @@ void i915_handle_error(struct drm_i915_private *i915, · 1 unmodified line
13721178 goto out;
13731179
13741180 /* Full reset needs the mutex, stop any other user trying to do so. */
1375− if (test_and_set_bit(I915_RESET_BACKOFF, &error->flags)) {
1376− wait_event(error->reset_queue,
1377− !test_bit(I915_RESET_BACKOFF, &error->flags));
1181+ if (test_and_set_bit(I915_RESET_BACKOFF, &gt->reset.flags)) {
1182+ wait_event(gt->reset.queue,
1183+ !test_bit(I915_RESET_BACKOFF, &gt->reset.flags));
13781184 goto out; /* piggy-back on the other reset */
13791185 }
13801186
@@ -1382,113 +1188,119 @@ void i915_handle_error(struct drm_i915_private *i915, · 1 unmodified line
13821188 synchronize_rcu_expedited();
13831189
13841190 /* Prevent any other reset-engine attempt. */
1385− for_each_engine(engine, i915, tmp) {
1191+ for_each_engine(engine, gt->i915, tmp) {
13861192 while (test_and_set_bit(I915_RESET_ENGINE + engine->id,
1387− &error->flags))
1388− wait_on_bit(&error->flags,
1193+ &gt->reset.flags))
1194+ wait_on_bit(&gt->reset.flags,
13891195 I915_RESET_ENGINE + engine->id,
13901196 TASK_UNINTERRUPTIBLE);
13911197 }
13921198
1393− i915_reset_device(i915, engine_mask, msg);
1199+ intel_gt_reset_global(gt, engine_mask, msg);
13941200
1395− for_each_engine(engine, i915, tmp) {
1396− clear_bit(I915_RESET_ENGINE + engine->id,
1397− &error->flags);
1398− }
1399−
1400− clear_bit(I915_RESET_BACKOFF, &error->flags);
1401− wake_up_all(&error->reset_queue);
1201+ for_each_engine(engine, gt->i915, tmp)
1202+ clear_bit_unlock(I915_RESET_ENGINE + engine->id,
1203+ &gt->reset.flags);
1204+ clear_bit_unlock(I915_RESET_BACKOFF, &gt->reset.flags);
1205+ smp_mb__after_atomic();
1206+ wake_up_all(&gt->reset.queue);
14021207
14031208 out:
1404− intel_runtime_pm_put(&i915->runtime_pm, wakeref);
1209+ intel_runtime_pm_put(&gt->i915->runtime_pm, wakeref);
14051210 }
14061211
1407−int i915_reset_trylock(struct drm_i915_private *i915)
1212+int intel_gt_reset_trylock(struct intel_gt *gt)
14081213 {
1409− struct i915_gpu_error *error = &i915->gpu_error;
14101214 int srcu;
14111215
1412− might_lock(&error->reset_backoff_srcu);
1216+ might_lock(&gt->reset.backoff_srcu);
14131217 might_sleep();
14141218
14151219 rcu_read_lock();
1416− while (test_bit(I915_RESET_BACKOFF, &error->flags)) {
1220+ while (test_bit(I915_RESET_BACKOFF, &gt->reset.flags)) {
14171221 rcu_read_unlock();
14181222
1419− if (wait_event_interruptible(error->reset_queue,
1223+ if (wait_event_interruptible(gt->reset.queue,
14201224 !test_bit(I915_RESET_BACKOFF,
1421− &error->flags)))
1225+ &gt->reset.flags)))
14221226 return -EINTR;
14231227
14241228 rcu_read_lock();
14251229 }
1426− srcu = srcu_read_lock(&error->reset_backoff_srcu);
1230+ srcu = srcu_read_lock(&gt->reset.backoff_srcu);
14271231 rcu_read_unlock();
14281232
14291233 return srcu;
14301234 }
14311235
1432−void i915_reset_unlock(struct drm_i915_private *i915, int tag)
1433−__releases(&i915->gpu_error.reset_backoff_srcu)
1236+void intel_gt_reset_unlock(struct intel_gt *gt, int tag)
1237+__releases(&gt->reset.backoff_srcu)
14341238 {
1435− struct i915_gpu_error *error = &i915->gpu_error;
1436−
1437− srcu_read_unlock(&error->reset_backoff_srcu, tag);
1239+ srcu_read_unlock(&gt->reset.backoff_srcu, tag);
14381240 }
14391241
1440−int i915_terminally_wedged(struct drm_i915_private *i915)
1242+int intel_gt_terminally_wedged(struct intel_gt *gt)
14411243 {
1442− struct i915_gpu_error *error = &i915->gpu_error;
1443−
14441244 might_sleep();
14451245
1446− if (!__i915_wedged(error))
1246+ if (!intel_gt_is_wedged(gt))
14471247 return 0;
14481248
14491249 /* Reset still in progress? Maybe we will recover? */
1450− if (!test_bit(I915_RESET_BACKOFF, &error->flags))
1250+ if (!test_bit(I915_RESET_BACKOFF, &gt->reset.flags))
14511251 return -EIO;
14521252
14531253 /* XXX intel_reset_finish() still takes struct_mutex!!! */
1454− if (mutex_is_locked(&i915->drm.struct_mutex))
1254+ if (mutex_is_locked(&gt->i915->drm.struct_mutex))
14551255 return -EAGAIN;
14561256
1457− if (wait_event_interruptible(error->reset_queue,
1257+ if (wait_event_interruptible(gt->reset.queue,
14581258 !test_bit(I915_RESET_BACKOFF,
1459− &error->flags)))
1259+ &gt->reset.flags)))
14601260 return -EINTR;
14611261
1462− return __i915_wedged(error) ? -EIO : 0;
1262+ return intel_gt_is_wedged(gt) ? -EIO : 0;
1263+}
1264+
1265+void intel_gt_init_reset(struct intel_gt *gt)
1266+{
1267+ init_waitqueue_head(&gt->reset.queue);
1268+ mutex_init(&gt->reset.mutex);
1269+ init_srcu_struct(&gt->reset.backoff_srcu);
1270+}
1271+
1272+void intel_gt_fini_reset(struct intel_gt *gt)
1273+{
1274+ cleanup_srcu_struct(&gt->reset.backoff_srcu);
14631275 }
14641276
1465−static void i915_wedge_me(struct work_struct *work)
1277+static void intel_wedge_me(struct work_struct *work)
14661278 {
1467− struct i915_wedge_me *w = container_of(work, typeof(*w), work.work);
1279+ struct intel_wedge_me *w = container_of(work, typeof(*w), work.work);
14681280
1469− dev_err(w->i915->drm.dev,
1281+ dev_err(w->gt->i915->drm.dev,
14701282 "%s timed out, cancelling all in-flight rendering.\n",
14711283 w->name);
1472− i915_gem_set_wedged(w->i915);
1284+ intel_gt_set_wedged(w->gt);
14731285 }
14741286
1475−void __i915_init_wedge(struct i915_wedge_me *w,
1476− struct drm_i915_private *i915,
1477− long timeout,
1478− const char *name)
1287+void __intel_init_wedge(struct intel_wedge_me *w,
1288+ struct intel_gt *gt,
1289+ long timeout,
1290+ const char *name)
14791291 {
1480− w->i915 = i915;
1292+ w->gt = gt;
14811293 w->name = name;
14821294
1483− INIT_DELAYED_WORK_ONSTACK(&w->work, i915_wedge_me);
1295+ INIT_DELAYED_WORK_ONSTACK(&w->work, intel_wedge_me);
14841296 schedule_delayed_work(&w->work, timeout);
14851297 }
14861298
1487−void __i915_fini_wedge(struct i915_wedge_me *w)
1299+void __intel_fini_wedge(struct intel_wedge_me *w)
14881300 {
14891301 cancel_delayed_work_sync(&w->work);
14901302 destroy_delayed_work_on_stack(&w->work);
1491− w->i915 = NULL;
1303+ w->gt = NULL;
14921304 }
14931305
14941306 #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
drivers/gpu/drm/i915/gt/intel_reset.h+42−33
@@ -11,58 +11,67 @@ · 10 unmodified lines
1111 #include <linux/types.h>
1212 #include <linux/srcu.h>
1313
14−#include "gt/intel_engine_types.h"
14+#include "intel_engine_types.h"
15+#include "intel_reset_types.h"
1516
1617 struct drm_i915_private;
1718 struct i915_request;
1819 struct intel_engine_cs;
20+struct intel_gt;
1921 struct intel_guc;
2022
23+void intel_gt_init_reset(struct intel_gt *gt);
24+void intel_gt_fini_reset(struct intel_gt *gt);
25+
2126 __printf(4, 5)
22−void i915_handle_error(struct drm_i915_private *i915,
23− intel_engine_mask_t engine_mask,
24− unsigned long flags,
25− const char *fmt, ...);
27+void intel_gt_handle_error(struct intel_gt *gt,
28+ intel_engine_mask_t engine_mask,
29+ unsigned long flags,
30+ const char *fmt, ...);
2631 #define I915_ERROR_CAPTURE BIT(0)
2732
28−void i915_check_and_clear_faults(struct drm_i915_private *i915);
29−
30−void i915_reset(struct drm_i915_private *i915,
31− intel_engine_mask_t stalled_mask,
32− const char *reason);
33−int i915_reset_engine(struct intel_engine_cs *engine,
34− const char *reason);
35−
36−void i915_reset_request(struct i915_request *rq, bool guilty);
33+void intel_gt_reset(struct intel_gt *gt,
34+ intel_engine_mask_t stalled_mask,
35+ const char *reason);
36+int intel_engine_reset(struct intel_engine_cs *engine,
37+ const char *reason);
3738
38−int __must_check i915_reset_trylock(struct drm_i915_private *i915);
39−void i915_reset_unlock(struct drm_i915_private *i915, int tag);
39+void __i915_request_reset(struct i915_request *rq, bool guilty);
4040
41−int i915_terminally_wedged(struct drm_i915_private *i915);
41+int __must_check intel_gt_reset_trylock(struct intel_gt *gt);
42+void intel_gt_reset_unlock(struct intel_gt *gt, int tag);
4243
43−bool intel_has_gpu_reset(struct drm_i915_private *i915);
44−bool intel_has_reset_engine(struct drm_i915_private *i915);
44+void intel_gt_set_wedged(struct intel_gt *gt);
45+bool intel_gt_unset_wedged(struct intel_gt *gt);
46+int intel_gt_terminally_wedged(struct intel_gt *gt);
4547
46−int intel_gpu_reset(struct drm_i915_private *i915,
47− intel_engine_mask_t engine_mask);
48+int __intel_gt_reset(struct intel_gt *gt, intel_engine_mask_t engine_mask);
4849
49−int intel_reset_guc(struct drm_i915_private *i915);
50+int intel_reset_guc(struct intel_gt *gt);
5051
51−struct i915_wedge_me {
52+struct intel_wedge_me {
5253 struct delayed_work work;
53− struct drm_i915_private *i915;
54+ struct intel_gt *gt;
5455 const char *name;
5556 };
5657
57−void __i915_init_wedge(struct i915_wedge_me *w,
58− struct drm_i915_private *i915,
59− long timeout,
60− const char *name);
61−void __i915_fini_wedge(struct i915_wedge_me *w);
58+void __intel_init_wedge(struct intel_wedge_me *w,
59+ struct intel_gt *gt,
60+ long timeout,
61+ const char *name);
62+void __intel_fini_wedge(struct intel_wedge_me *w);
6263
63−#define i915_wedge_on_timeout(W, DEV, TIMEOUT) \
64− for (__i915_init_wedge((W), (DEV), (TIMEOUT), __func__); \
65− (W)->i915; \
66− __i915_fini_wedge((W)))
64+#define intel_wedge_on_timeout(W, GT, TIMEOUT) \
65+ for (__intel_init_wedge((W), (GT), (TIMEOUT), __func__); \
66+ (W)->gt; \
67+ __intel_fini_wedge((W)))
68+
69+static inline bool __intel_reset_failed(const struct intel_reset *reset)
70+{
71+ return unlikely(test_bit(I915_WEDGED, &reset->flags));
72+}
73+
74+bool intel_has_gpu_reset(struct drm_i915_private *i915);
75+bool intel_has_reset_engine(struct drm_i915_private *i915);
6776
6877 #endif /* I915_RESET_H */
drivers/gpu/drm/i915/gt/intel_reset_types.h+50−0
@@ -0,0 +1,50 @@
1+/* SPDX-License-Identifier: MIT */
2+/*
3+ * Copyright © 2019 Intel Corporation
4+ */
5+
6+#ifndef __INTEL_RESET_TYPES_H_
7+#define __INTEL_RESET_TYPES_H_
8+
9+#include <linux/mutex.h>
10+#include <linux/wait.h>
11+#include <linux/srcu.h>
12+
13+struct intel_reset {
14+ /**
15+ * flags: Control various stages of the GPU reset
16+ *
17+ * #I915_RESET_BACKOFF - When we start a global reset, we need to
18+ * serialise with any other users attempting to do the same, and
19+ * any global resources that may be clobber by the reset (such as
20+ * FENCE registers).
21+ *
22+ * #I915_RESET_ENGINE[num_engines] - Since the driver doesn't need to
23+ * acquire the struct_mutex to reset an engine, we need an explicit
24+ * flag to prevent two concurrent reset attempts in the same engine.
25+ * As the number of engines continues to grow, allocate the flags from
26+ * the most significant bits.
27+ *
28+ * #I915_WEDGED - If reset fails and we can no longer use the GPU,
29+ * we set the #I915_WEDGED bit. Prior to command submission, e.g.
30+ * i915_request_alloc(), this bit is checked and the sequence
31+ * aborted (with -EIO reported to userspace) if set.
32+ */
33+ unsigned long flags;
34+#define I915_RESET_BACKOFF 0
35+#define I915_RESET_MODESET 1
36+#define I915_RESET_ENGINE 2
37+#define I915_WEDGED (BITS_PER_LONG - 1)
38+
39+ struct mutex mutex; /* serialises wedging/unwedging */
40+
41+ /**
42+ * Waitqueue to signal when the reset has completed. Used by clients
43+ * that wait for dev_priv->mm.wedged to settle.
44+ */
45+ wait_queue_head_t queue;
46+
47+ struct srcu_struct backoff_srcu;
48+};
49+
50+#endif /* _INTEL_RESET_TYPES_H_ */
drivers/gpu/drm/i915/gt/intel_ringbuffer.c+141−77
@@ -34,9 +34,9 @@ · 33 unmodified lines
3434 #include "gem/i915_gem_context.h"
3535
3636 #include "i915_drv.h"
37−#include "i915_gem_render_state.h"
3837 #include "i915_trace.h"
3938 #include "intel_context.h"
39+#include "intel_gt.h"
4040 #include "intel_reset.h"
4141 #include "intel_workarounds.h"
4242
@@ -75,7 +75,8 @@ gen2_render_ring_flush(struct i915_request *rq, u32 mode) · 32 unmodified lines
7575 *cs++ = cmd;
7676 while (num_store_dw--) {
7777 *cs++ = MI_STORE_DWORD_IMM | MI_MEM_VIRTUAL;
78− *cs++ = i915_scratch_offset(rq->i915);
78+ *cs++ = intel_gt_scratch_offset(rq->engine->gt,
79+ INTEL_GT_SCRATCH_FIELD_DEFAULT);
7980 *cs++ = 0;
8081 }
8182 *cs++ = MI_FLUSH | MI_NO_WRITE_FLUSH;
@@ -148,7 +149,9 @@ gen4_render_ring_flush(struct i915_request *rq, u32 mode) · 66 unmodified lines
148149 */
149150 if (mode & EMIT_INVALIDATE) {
150151 *cs++ = GFX_OP_PIPE_CONTROL(4) | PIPE_CONTROL_QW_WRITE;
151− *cs++ = i915_scratch_offset(rq->i915) | PIPE_CONTROL_GLOBAL_GTT;
152+ *cs++ = intel_gt_scratch_offset(rq->engine->gt,
153+ INTEL_GT_SCRATCH_FIELD_DEFAULT) |
154+ PIPE_CONTROL_GLOBAL_GTT;
152155 *cs++ = 0;
153156 *cs++ = 0;
154157
@@ -156,7 +159,9 @@ gen4_render_ring_flush(struct i915_request *rq, u32 mode) · 1 unmodified line
156159 *cs++ = MI_FLUSH;
157160
158161 *cs++ = GFX_OP_PIPE_CONTROL(4) | PIPE_CONTROL_QW_WRITE;
159− *cs++ = i915_scratch_offset(rq->i915) | PIPE_CONTROL_GLOBAL_GTT;
162+ *cs++ = intel_gt_scratch_offset(rq->engine->gt,
163+ INTEL_GT_SCRATCH_FIELD_DEFAULT) |
164+ PIPE_CONTROL_GLOBAL_GTT;
160165 *cs++ = 0;
161166 *cs++ = 0;
162167 }
@@ -208,7 +213,9 @@ gen4_render_ring_flush(struct i915_request *rq, u32 mode) · 45 unmodified lines
208213 static int
209214 gen6_emit_post_sync_nonzero_flush(struct i915_request *rq)
210215 {
211− u32 scratch_addr = i915_scratch_offset(rq->i915) + 2 * CACHELINE_BYTES;
216+ u32 scratch_addr =
217+ intel_gt_scratch_offset(rq->engine->gt,
218+ INTEL_GT_SCRATCH_FIELD_RENDER_FLUSH);
212219 u32 *cs;
213220
214221 cs = intel_ring_begin(rq, 6);
@@ -241,7 +248,9 @@ gen6_emit_post_sync_nonzero_flush(struct i915_request *rq) · 26 unmodified lines
241248 static int
242249 gen6_render_ring_flush(struct i915_request *rq, u32 mode)
243250 {
244− u32 scratch_addr = i915_scratch_offset(rq->i915) + 2 * CACHELINE_BYTES;
251+ u32 scratch_addr =
252+ intel_gt_scratch_offset(rq->engine->gt,
253+ INTEL_GT_SCRATCH_FIELD_RENDER_FLUSH);
245254 u32 *cs, flags = 0;
246255 int ret;
247256
@@ -299,7 +308,9 @@ static u32 *gen6_rcs_emit_breadcrumb(struct i915_request *rq, u32 *cs) · 51 unmodified lines
299308
300309 *cs++ = GFX_OP_PIPE_CONTROL(4);
301310 *cs++ = PIPE_CONTROL_QW_WRITE;
302− *cs++ = i915_scratch_offset(rq->i915) | PIPE_CONTROL_GLOBAL_GTT;
311+ *cs++ = intel_gt_scratch_offset(rq->engine->gt,
312+ INTEL_GT_SCRATCH_FIELD_DEFAULT) |
313+ PIPE_CONTROL_GLOBAL_GTT;
303314 *cs++ = 0;
304315
305316 /* Finally we can flush and with it emit the breadcrumb */
@@ -342,7 +353,9 @@ gen7_render_ring_cs_stall_wa(struct i915_request *rq) · 36 unmodified lines
342353 static int
343354 gen7_render_ring_flush(struct i915_request *rq, u32 mode)
344355 {
345− u32 scratch_addr = i915_scratch_offset(rq->i915) + 2 * CACHELINE_BYTES;
356+ u32 scratch_addr =
357+ intel_gt_scratch_offset(rq->engine->gt,
358+ INTEL_GT_SCRATCH_FIELD_RENDER_FLUSH);
346359 u32 *cs, flags = 0;
347360
348361 /*
@@ -725,7 +738,45 @@ out: · 376 unmodified lines
725738
726739 static void reset_prepare(struct intel_engine_cs *engine)
727740 {
728− intel_engine_stop_cs(engine);
741+ struct intel_uncore *uncore = engine->uncore;
742+ const u32 base = engine->mmio_base;
743+
744+ /*
745+ * We stop engines, otherwise we might get failed reset and a
746+ * dead gpu (on elk). Also as modern gpu as kbl can suffer
747+ * from system hang if batchbuffer is progressing when
748+ * the reset is issued, regardless of READY_TO_RESET ack.
749+ * Thus assume it is best to stop engines on all gens
750+ * where we have a gpu reset.
751+ *
752+ * WaKBLVECSSemaphoreWaitPoll:kbl (on ALL_ENGINES)
753+ *
754+ * WaMediaResetMainRingCleanup:ctg,elk (presumably)
755+ *
756+ * FIXME: Wa for more modern gens needs to be validated
757+ */
758+ GEM_TRACE("%s\n", engine->name);
759+
760+ if (intel_engine_stop_cs(engine))
761+ GEM_TRACE("%s: timed out on STOP_RING\n", engine->name);
762+
763+ intel_uncore_write_fw(uncore,
764+ RING_HEAD(base),
765+ intel_uncore_read_fw(uncore, RING_TAIL(base)));
766+ intel_uncore_posting_read_fw(uncore, RING_HEAD(base)); /* paranoia */
767+
768+ intel_uncore_write_fw(uncore, RING_HEAD(base), 0);
769+ intel_uncore_write_fw(uncore, RING_TAIL(base), 0);
770+ intel_uncore_posting_read_fw(uncore, RING_TAIL(base));
771+
772+ /* The ring must be empty before it is disabled */
773+ intel_uncore_write_fw(uncore, RING_CTL(base), 0);
774+
775+ /* Check acts as a post */
776+ if (intel_uncore_read_fw(uncore, RING_HEAD(base)))
777+ GEM_TRACE("%s: ring head [%x] not parked\n",
778+ engine->name,
779+ intel_uncore_read_fw(uncore, RING_HEAD(base)));
729780 }
730781
731782 static void reset_ring(struct intel_engine_cs *engine, bool stalled)
@@ -781,7 +832,7 @@ static void reset_ring(struct intel_engine_cs *engine, bool stalled) · 49 unmodified lines
781832 * If the request was innocent, we try to replay the request
782833 * with the restored context.
783834 */
784− i915_reset_request(rq, stalled);
835+ __i915_request_reset(rq, stalled);
785836
786837 GEM_BUG_ON(rq->ring != engine->buffer);
787838 head = rq->head;
@@ -797,21 +848,6 @@ static void reset_finish(struct intel_engine_cs *engine) · 9 unmodified lines
797848 {
798849 }
799850
800−static int intel_rcs_ctx_init(struct i915_request *rq)
801−{
802− int ret;
803−
804− ret = intel_engine_emit_ctx_wa(rq);
805− if (ret != 0)
806− return ret;
807−
808− ret = i915_gem_render_state_emit(rq);
809− if (ret)
810− return ret;
811−
812− return 0;
813−}
814−
815851 static int rcs_resume(struct intel_engine_cs *engine)
816852 {
817853 struct drm_i915_private *dev_priv = engine->i915;
@@ -1033,14 +1069,14 @@ hsw_vebox_irq_enable(struct intel_engine_cs *engine) · 215 unmodified lines
10331069 /* Flush/delay to ensure the RING_IMR is active before the GT IMR */
10341070 ENGINE_POSTING_READ(engine, RING_IMR);
10351071
1036− gen6_unmask_pm_irq(engine->i915, engine->irq_enable_mask);
1072+ gen6_unmask_pm_irq(engine->gt, engine->irq_enable_mask);
10371073 }
10381074
10391075 static void
10401076 hsw_vebox_irq_disable(struct intel_engine_cs *engine)
10411077 {
10421078 ENGINE_WRITE(engine, RING_IMR, ~0);
1043− gen6_mask_pm_irq(engine->i915, engine->irq_enable_mask);
1079+ gen6_mask_pm_irq(engine->gt, engine->irq_enable_mask);
10441080 }
10451081
10461082 static int
@@ -1071,9 +1107,11 @@ i830_emit_bb_start(struct i915_request *rq, · 24 unmodified lines
10711107 u64 offset, u32 len,
10721108 unsigned int dispatch_flags)
10731109 {
1074− u32 *cs, cs_offset = i915_scratch_offset(rq->i915);
1110+ u32 *cs, cs_offset =
1111+ intel_gt_scratch_offset(rq->engine->gt,
1112+ INTEL_GT_SCRATCH_FIELD_DEFAULT);
10751113
1076− GEM_BUG_ON(rq->i915->gt.scratch->size < I830_WA_SIZE);
1114+ GEM_BUG_ON(rq->engine->gt->scratch->size < I830_WA_SIZE);
10771115
10781116 cs = intel_ring_begin(rq, 6);
10791117 if (IS_ERR(cs))
@@ -1156,7 +1194,7 @@ int intel_ring_pin(struct intel_ring *ring) · 76 unmodified lines
11561194 if (atomic_fetch_inc(&ring->pin_count))
11571195 return 0;
11581196
1159− ret = i915_timeline_pin(ring->timeline);
1197+ ret = intel_timeline_pin(ring->timeline);
11601198 if (ret)
11611199 goto err_unpin;
11621200
@@ -1189,12 +1227,13 @@ int intel_ring_pin(struct intel_ring *ring) · 26 unmodified lines
11891227 GEM_BUG_ON(ring->vaddr);
11901228 ring->vaddr = addr;
11911229
1230+ GEM_TRACE("ring:%llx pin\n", ring->timeline->fence_context);
11921231 return 0;
11931232
11941233 err_ring:
11951234 i915_vma_unpin(vma);
11961235 err_timeline:
1197− i915_timeline_unpin(ring->timeline);
1236+ intel_timeline_unpin(ring->timeline);
11981237 err_unpin:
11991238 atomic_dec(&ring->pin_count);
12001239 return ret;
@@ -1215,10 +1254,13 @@ void intel_ring_unpin(struct intel_ring *ring) · 14 unmodified lines
12151254 if (!atomic_dec_and_test(&ring->pin_count))
12161255 return;
12171256
1257+ GEM_TRACE("ring:%llx unpin\n", ring->timeline->fence_context);
1258+
12181259 /* Discard any unused bytes beyond that submitted to hw. */
12191260 intel_ring_reset(ring, ring->tail);
12201261
12211262 GEM_BUG_ON(!ring->vma);
1263+ i915_vma_unset_ggtt_write(ring->vma);
12221264 if (i915_vma_is_map_and_fenceable(ring->vma))
12231265 i915_vma_unpin_iomap(ring->vma);
12241266 else
@@ -1230,19 +1272,19 @@ void intel_ring_unpin(struct intel_ring *ring) · 5 unmodified lines
12301272 ring->vma->obj->pin_global--;
12311273 i915_vma_unpin(ring->vma);
12321274
1233− i915_timeline_unpin(ring->timeline);
1275+ intel_timeline_unpin(ring->timeline);
12341276 }
12351277
1236−static struct i915_vma *
1237−intel_ring_create_vma(struct drm_i915_private *dev_priv, int size)
1278+static struct i915_vma *create_ring_vma(struct i915_ggtt *ggtt, int size)
12381279 {
1239− struct i915_address_space *vm = &dev_priv->ggtt.vm;
1280+ struct i915_address_space *vm = &ggtt->vm;
1281+ struct drm_i915_private *i915 = vm->i915;
12401282 struct drm_i915_gem_object *obj;
12411283 struct i915_vma *vma;
12421284
1243− obj = i915_gem_object_create_stolen(dev_priv, size);
1285+ obj = i915_gem_object_create_stolen(i915, size);
12441286 if (!obj)
1245− obj = i915_gem_object_create_internal(dev_priv, size);
1287+ obj = i915_gem_object_create_internal(i915, size);
12461288 if (IS_ERR(obj))
12471289 return ERR_CAST(obj);
12481290
@@ -1266,9 +1308,10 @@ err: · 17 unmodified lines
12661308
12671309 struct intel_ring *
12681310 intel_engine_create_ring(struct intel_engine_cs *engine,
1269− struct i915_timeline *timeline,
1311+ struct intel_timeline *timeline,
12701312 int size)
12711313 {
1314+ struct drm_i915_private *i915 = engine->i915;
12721315 struct intel_ring *ring;
12731316 struct i915_vma *vma;
12741317
@@ -1281,7 +1324,7 @@ intel_engine_create_ring(struct intel_engine_cs *engine, · 6 unmodified lines
12811324
12821325 kref_init(&ring->ref);
12831326 INIT_LIST_HEAD(&ring->request_list);
1284− ring->timeline = i915_timeline_get(timeline);
1327+ ring->timeline = intel_timeline_get(timeline);
12851328
12861329 ring->size = size;
12871330 /* Workaround an erratum on the i830 which causes a hang if
@@ -1289,12 +1332,12 @@ intel_engine_create_ring(struct intel_engine_cs *engine, · 1 unmodified line
12891332 * of the buffer.
12901333 */
12911334 ring->effective_size = size;
1292− if (IS_I830(engine->i915) || IS_I845G(engine->i915))
1335+ if (IS_I830(i915) || IS_I845G(i915))
12931336 ring->effective_size -= 2 * CACHELINE_BYTES;
12941337
12951338 intel_ring_update_space(ring);
12961339
1297− vma = intel_ring_create_vma(engine->i915, size);
1340+ vma = create_ring_vma(engine->gt->ggtt, size);
12981341 if (IS_ERR(vma)) {
12991342 kfree(ring);
13001343 return ERR_CAST(vma);
@@ -1311,13 +1354,12 @@ void intel_ring_free(struct kref *ref) · 10 unmodified lines
13111354 i915_vma_close(ring->vma);
13121355 i915_vma_put(ring->vma);
13131356
1314− i915_timeline_put(ring->timeline);
1357+ intel_timeline_put(ring->timeline);
13151358 kfree(ring);
13161359 }
13171360
13181361 static void __ring_context_fini(struct intel_context *ce)
13191362 {
1320− GEM_BUG_ON(i915_gem_object_is_active(ce->state->obj));
13211363 i915_gem_object_put(ce->state->obj);
13221364 }
13231365
@@ -1330,33 +1372,45 @@ static void ring_context_destroy(struct kref *ref) · 6 unmodified lines
13301372 if (ce->state)
13311373 __ring_context_fini(ce);
13321374
1375+ intel_context_fini(ce);
13331376 intel_context_free(ce);
13341377 }
13351378
1336−static int __context_pin_ppgtt(struct i915_gem_context *ctx)
1379+static struct i915_address_space *vm_alias(struct intel_context *ce)
1380+{
1381+ struct i915_address_space *vm;
1382+
1383+ vm = ce->vm;
1384+ if (i915_is_ggtt(vm))
1385+ vm = &i915_vm_to_ggtt(vm)->alias->vm;
1386+
1387+ return vm;
1388+}
1389+
1390+static int __context_pin_ppgtt(struct intel_context *ce)
13371391 {
13381392 struct i915_address_space *vm;
13391393 int err = 0;
13401394
1341− vm = ctx->vm ?: &ctx->i915->mm.aliasing_ppgtt->vm;
1395+ vm = vm_alias(ce);
13421396 if (vm)
13431397 err = gen6_ppgtt_pin(i915_vm_to_ppgtt((vm)));
13441398
13451399 return err;
13461400 }
13471401
1348−static void __context_unpin_ppgtt(struct i915_gem_context *ctx)
1402+static void __context_unpin_ppgtt(struct intel_context *ce)
13491403 {
13501404 struct i915_address_space *vm;
13511405
1352− vm = ctx->vm ?: &ctx->i915->mm.aliasing_ppgtt->vm;
1406+ vm = vm_alias(ce);
13531407 if (vm)
13541408 gen6_ppgtt_unpin(i915_vm_to_ppgtt(vm));
13551409 }
13561410
13571411 static void ring_context_unpin(struct intel_context *ce)
13581412 {
1359− __context_unpin_ppgtt(ce->gem_context);
1413+ __context_unpin_ppgtt(ce);
13601414 }
13611415
13621416 static struct i915_vma *
@@ -1412,7 +1466,7 @@ alloc_context_vma(struct intel_engine_cs *engine) · 49 unmodified lines
14121466 i915_gem_object_unpin_map(obj);
14131467 }
14141468
1415− vma = i915_vma_instance(obj, &i915->ggtt.vm, NULL);
1469+ vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);
14161470 if (IS_ERR(vma)) {
14171471 err = PTR_ERR(vma);
14181472 goto err_obj;
@@ -1446,11 +1500,11 @@ static int ring_context_pin(struct intel_context *ce) · 27 unmodified lines
14461500 ce->state = vma;
14471501 }
14481502
1449− err = intel_context_active_acquire(ce, PIN_HIGH);
1503+ err = intel_context_active_acquire(ce);
14501504 if (err)
14511505 return err;
14521506
1453− err = __context_pin_ppgtt(ce->gem_context);
1507+ err = __context_pin_ppgtt(ce);
14541508 if (err)
14551509 goto err_active;
14561510
@@ -1492,7 +1546,7 @@ static int load_pd_dir(struct i915_request *rq, const struct i915_ppgtt *ppgtt) · 35 unmodified lines
14921546
14931547 *cs++ = MI_LOAD_REGISTER_IMM(1);
14941548 *cs++ = i915_mmio_reg_offset(RING_PP_DIR_BASE(engine->mmio_base));
1495− *cs++ = ppgtt->pd->base.ggtt_offset << 10;
1549+ *cs++ = px_base(ppgtt->pd)->ggtt_offset << 10;
14961550
14971551 intel_ring_advance(rq, cs);
14981552
@@ -1511,7 +1565,8 @@ static int flush_pd_dir(struct i915_request *rq) · 12 unmodified lines
15111565 /* Stall until the page table load is complete */
15121566 *cs++ = MI_STORE_REGISTER_MEM | MI_SRM_LRM_GLOBAL_GTT;
15131567 *cs++ = i915_mmio_reg_offset(RING_PP_DIR_BASE(engine->mmio_base));
1514− *cs++ = i915_scratch_offset(rq->i915);
1568+ *cs++ = intel_gt_scratch_offset(rq->engine->gt,
1569+ INTEL_GT_SCRATCH_FIELD_DEFAULT);
15151570 *cs++ = MI_NOOP;
15161571
15171572 intel_ring_advance(rq, cs);
@@ -1627,7 +1682,8 @@ static inline int mi_set_context(struct i915_request *rq, u32 flags) · 109 unmodified lines
16271682 /* Insert a delay before the next switch! */
16281683 *cs++ = MI_STORE_REGISTER_MEM | MI_SRM_LRM_GLOBAL_GTT;
16291684 *cs++ = i915_mmio_reg_offset(last_reg);
1630− *cs++ = i915_scratch_offset(rq->i915);
1685+ *cs++ = intel_gt_scratch_offset(rq->engine->gt,
1686+ INTEL_GT_SCRATCH_FIELD_DEFAULT);
16311687 *cs++ = MI_NOOP;
16321688 }
16331689 *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
@@ -1640,7 +1696,7 @@ static inline int mi_set_context(struct i915_request *rq, u32 flags) · 6 unmodified lines
16401696 return 0;
16411697 }
16421698
1643−static int remap_l3(struct i915_request *rq, int slice)
1699+static int remap_l3_slice(struct i915_request *rq, int slice)
16441700 {
16451701 u32 *cs, *remap_info = rq->i915->l3_parity.remap_info[slice];
16461702 int i;
@@ -1668,15 +1724,34 @@ static int remap_l3(struct i915_request *rq, int slice) · 21 unmodified lines
16681724 return 0;
16691725 }
16701726
1727+static int remap_l3(struct i915_request *rq)
1728+{
1729+ struct i915_gem_context *ctx = rq->gem_context;
1730+ int i, err;
1731+
1732+ if (!ctx->remap_slice)
1733+ return 0;
1734+
1735+ for (i = 0; i < MAX_L3_SLICES; i++) {
1736+ if (!(ctx->remap_slice & BIT(i)))
1737+ continue;
1738+
1739+ err = remap_l3_slice(rq, i);
1740+ if (err)
1741+ return err;
1742+ }
1743+
1744+ ctx->remap_slice = 0;
1745+ return 0;
1746+}
1747+
16711748 static int switch_context(struct i915_request *rq)
16721749 {
16731750 struct intel_engine_cs *engine = rq->engine;
1674− struct i915_gem_context *ctx = rq->gem_context;
1675− struct i915_address_space *vm =
1676− ctx->vm ?: &rq->i915->mm.aliasing_ppgtt->vm;
1751+ struct i915_address_space *vm = vm_alias(rq->hw_context);
16771752 unsigned int unwind_mm = 0;
16781753 u32 hw_flags = 0;
1679− int ret, i;
1754+ int ret;
16801755
16811756 GEM_BUG_ON(HAS_EXECLISTS(rq->i915));
16821757
@@ -1720,7 +1795,7 @@ static int switch_context(struct i915_request *rq) · 37 unmodified lines
17201795 * as nothing actually executes using the kernel context; it
17211796 * is purely used for flushing user contexts.
17221797 */
1723− if (i915_gem_context_is_kernel(ctx))
1798+ if (i915_gem_context_is_kernel(rq->gem_context))
17241799 hw_flags = MI_RESTORE_INHIBIT;
17251800
17261801 ret = mi_set_context(rq, hw_flags);
@@ -1754,18 +1829,9 @@ static int switch_context(struct i915_request *rq) · 27 unmodified lines
17541829 goto err_mm;
17551830 }
17561831
1757− if (ctx->remap_slice) {
1758− for (i = 0; i < MAX_L3_SLICES; i++) {
1759− if (!(ctx->remap_slice & BIT(i)))
1760− continue;
1761−
1762− ret = remap_l3(rq, i);
1763− if (ret)
1764− goto err_mm;
1765− }
1766−
1767− ctx->remap_slice = 0;
1768− }
1832+ ret = remap_l3(rq);
1833+ if (ret)
1834+ goto err_mm;
17691835
17701836 return 0;
17711837
@@ -2166,11 +2232,9 @@ static void setup_rcs(struct intel_engine_cs *engine) · 394 unmodified lines
21662232 engine->irq_enable_mask = GT_RENDER_USER_INTERRUPT;
21672233
21682234 if (INTEL_GEN(i915) >= 7) {
2169− engine->init_context = intel_rcs_ctx_init;
21702235 engine->emit_flush = gen7_render_ring_flush;
21712236 engine->emit_fini_breadcrumb = gen7_rcs_emit_breadcrumb;
21722237 } else if (IS_GEN(i915, 6)) {
2173− engine->init_context = intel_rcs_ctx_init;
21742238 engine->emit_flush = gen6_render_ring_flush;
21752239 engine->emit_fini_breadcrumb = gen6_rcs_emit_breadcrumb;
21762240 } else if (IS_GEN(i915, 5)) {
@@ -2267,11 +2331,11 @@ int intel_ring_submission_setup(struct intel_engine_cs *engine) · 90 unmodified lines
22672331
22682332 int intel_ring_submission_init(struct intel_engine_cs *engine)
22692333 {
2270− struct i915_timeline *timeline;
2334+ struct intel_timeline *timeline;
22712335 struct intel_ring *ring;
22722336 int err;
22732337
2274− timeline = i915_timeline_create(engine->i915, engine->status_page.vma);
2338+ timeline = intel_timeline_create(engine->gt, engine->status_page.vma);
22752339 if (IS_ERR(timeline)) {
22762340 err = PTR_ERR(timeline);
22772341 goto err;
@@ -2279,7 +2343,7 @@ int intel_ring_submission_init(struct intel_engine_cs *engine) · 1 unmodified line
22792343 GEM_BUG_ON(timeline->has_initial_breadcrumb);
22802344
22812345 ring = intel_engine_create_ring(engine, timeline, 32 * PAGE_SIZE);
2282− i915_timeline_put(timeline);
2346+ intel_timeline_put(timeline);
22832347 if (IS_ERR(ring)) {
22842348 err = PTR_ERR(ring);
22852349 goto err;
drivers/gpu/drm/i915/gt/intel_timeline.crenamed from drivers/gpu/drm/i915/i915_timeline.c+120−110
@@ -4,38 +4,36 @@ · 3 unmodified lines
44 * Copyright © 2016-2018 Intel Corporation
55 */
66
7+#include "gt/intel_gt_types.h"
8+
79 #include "i915_drv.h"
810
911 #include "i915_active.h"
1012 #include "i915_syncmap.h"
11−#include "i915_timeline.h"
13+#include "gt/intel_timeline.h"
1214
1315 #define ptr_set_bit(ptr, bit) ((typeof(ptr))((unsigned long)(ptr) | BIT(bit)))
1416 #define ptr_test_bit(ptr, bit) ((unsigned long)(ptr) & BIT(bit))
1517
16−struct i915_timeline_hwsp {
17− struct i915_gt_timelines *gt;
18+struct intel_timeline_hwsp {
19+ struct intel_gt *gt;
20+ struct intel_gt_timelines *gt_timelines;
1821 struct list_head free_link;
1922 struct i915_vma *vma;
2023 u64 free_bitmap;
2124 };
2225
23−struct i915_timeline_cacheline {
26+struct intel_timeline_cacheline {
2427 struct i915_active active;
25− struct i915_timeline_hwsp *hwsp;
28+ struct intel_timeline_hwsp *hwsp;
2629 void *vaddr;
2730 #define CACHELINE_BITS 6
2831 #define CACHELINE_FREE CACHELINE_BITS
2932 };
3033
31−static inline struct drm_i915_private *
32−hwsp_to_i915(struct i915_timeline_hwsp *hwsp)
33−{
34− return container_of(hwsp->gt, struct drm_i915_private, gt.timelines);
35−}
36−
37−static struct i915_vma *__hwsp_alloc(struct drm_i915_private *i915)
34+static struct i915_vma *__hwsp_alloc(struct intel_gt *gt)
3835 {
36+ struct drm_i915_private *i915 = gt->i915;
3937 struct drm_i915_gem_object *obj;
4038 struct i915_vma *vma;
4139
@@ -45,7 +43,7 @@ static struct i915_vma *__hwsp_alloc(struct drm_i915_private *i915) · 3 unmodified lines
4543
4644 i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);
4745
48− vma = i915_vma_instance(obj, &i915->ggtt.vm, NULL);
46+ vma = i915_vma_instance(obj, &gt->ggtt->vm, NULL);
4947 if (IS_ERR(vma))
5048 i915_gem_object_put(obj);
5149
@@ -53,11 +51,10 @@ static struct i915_vma *__hwsp_alloc(struct drm_i915_private *i915) · 1 unmodified line
5351 }
5452
5553 static struct i915_vma *
56−hwsp_alloc(struct i915_timeline *timeline, unsigned int *cacheline)
54+hwsp_alloc(struct intel_timeline *timeline, unsigned int *cacheline)
5755 {
58− struct drm_i915_private *i915 = timeline->i915;
59− struct i915_gt_timelines *gt = &i915->gt.timelines;
60− struct i915_timeline_hwsp *hwsp;
56+ struct intel_gt_timelines *gt = &timeline->gt->timelines;
57+ struct intel_timeline_hwsp *hwsp;
6158
6259 BUILD_BUG_ON(BITS_PER_TYPE(u64) * CACHELINE_BYTES > PAGE_SIZE);
6360
@@ -75,16 +72,17 @@ hwsp_alloc(struct i915_timeline *timeline, unsigned int *cacheline) · 11 unmodified lines
7572 if (!hwsp)
7673 return ERR_PTR(-ENOMEM);
7774
78− vma = __hwsp_alloc(i915);
75+ vma = __hwsp_alloc(timeline->gt);
7976 if (IS_ERR(vma)) {
8077 kfree(hwsp);
8178 return vma;
8279 }
8380
8481 vma->private = hwsp;
82+ hwsp->gt = timeline->gt;
8583 hwsp->vma = vma;
8684 hwsp->free_bitmap = ~0ull;
87− hwsp->gt = gt;
85+ hwsp->gt_timelines = gt;
8886
8987 spin_lock_irq(&gt->hwsp_lock);
9088 list_add(&hwsp->free_link, &gt->hwsp_free_list);
@@ -102,9 +100,9 @@ hwsp_alloc(struct i915_timeline *timeline, unsigned int *cacheline) · 11 unmodified lines
102100 return hwsp->vma;
103101 }
104102
105−static void __idle_hwsp_free(struct i915_timeline_hwsp *hwsp, int cacheline)
103+static void __idle_hwsp_free(struct intel_timeline_hwsp *hwsp, int cacheline)
106104 {
107− struct i915_gt_timelines *gt = hwsp->gt;
105+ struct intel_gt_timelines *gt = hwsp->gt_timelines;
108106 unsigned long flags;
109107
110108 spin_lock_irqsave(&gt->hwsp_lock, flags);
@@ -126,7 +124,7 @@ static void __idle_hwsp_free(struct i915_timeline_hwsp *hwsp, int cacheline) · 15 unmodified lines
126124 spin_unlock_irqrestore(&gt->hwsp_lock, flags);
127125 }
128126
129−static void __idle_cacheline_free(struct i915_timeline_cacheline *cl)
127+static void __idle_cacheline_free(struct intel_timeline_cacheline *cl)
130128 {
131129 GEM_BUG_ON(!i915_active_is_idle(&cl->active));
132130
@@ -140,7 +138,7 @@ static void __idle_cacheline_free(struct i915_timeline_cacheline *cl) · 7 unmodified lines
140138
141139 static void __cacheline_retire(struct i915_active *active)
142140 {
143− struct i915_timeline_cacheline *cl =
141+ struct intel_timeline_cacheline *cl =
144142 container_of(active, typeof(*cl), active);
145143
146144 i915_vma_unpin(cl->hwsp->vma);
@@ -148,10 +146,19 @@ static void __cacheline_retire(struct i915_active *active) · 1 unmodified line
148146 __idle_cacheline_free(cl);
149147 }
150148
151−static struct i915_timeline_cacheline *
152−cacheline_alloc(struct i915_timeline_hwsp *hwsp, unsigned int cacheline)
149+static int __cacheline_active(struct i915_active *active)
150+{
151+ struct intel_timeline_cacheline *cl =
152+ container_of(active, typeof(*cl), active);
153+
154+ __i915_vma_pin(cl->hwsp->vma);
155+ return 0;
156+}
157+
158+static struct intel_timeline_cacheline *
159+cacheline_alloc(struct intel_timeline_hwsp *hwsp, unsigned int cacheline)
153160 {
154− struct i915_timeline_cacheline *cl;
161+ struct intel_timeline_cacheline *cl;
155162 void *vaddr;
156163
157164 GEM_BUG_ON(cacheline >= BIT(CACHELINE_BITS));
@@ -170,24 +177,25 @@ cacheline_alloc(struct i915_timeline_hwsp *hwsp, unsigned int cacheline) · 12 unmodified lines
170177 cl->hwsp = hwsp;
171178 cl->vaddr = page_pack_bits(vaddr, cacheline);
172179
173− i915_active_init(hwsp_to_i915(hwsp), &cl->active, __cacheline_retire);
180+ i915_active_init(hwsp->gt->i915, &cl->active,
181+ __cacheline_active, __cacheline_retire);
174182
175183 return cl;
176184 }
177185
178−static void cacheline_acquire(struct i915_timeline_cacheline *cl)
186+static void cacheline_acquire(struct intel_timeline_cacheline *cl)
179187 {
180− if (cl && i915_active_acquire(&cl->active))
181− __i915_vma_pin(cl->hwsp->vma);
188+ if (cl)
189+ i915_active_acquire(&cl->active);
182190 }
183191
184−static void cacheline_release(struct i915_timeline_cacheline *cl)
192+static void cacheline_release(struct intel_timeline_cacheline *cl)
185193 {
186194 if (cl)
187195 i915_active_release(&cl->active);
188196 }
189197
190−static void cacheline_free(struct i915_timeline_cacheline *cl)
198+static void cacheline_free(struct intel_timeline_cacheline *cl)
191199 {
192200 GEM_BUG_ON(ptr_test_bit(cl->vaddr, CACHELINE_FREE));
193201 cl->vaddr = ptr_set_bit(cl->vaddr, CACHELINE_FREE);
@@ -196,29 +204,22 @@ static void cacheline_free(struct i915_timeline_cacheline *cl) · 2 unmodified lines
196204 __idle_cacheline_free(cl);
197205 }
198206
199−int i915_timeline_init(struct drm_i915_private *i915,
200− struct i915_timeline *timeline,
201− struct i915_vma *hwsp)
207+int intel_timeline_init(struct intel_timeline *timeline,
208+ struct intel_gt *gt,
209+ struct i915_vma *hwsp)
202210 {
203211 void *vaddr;
204212
205− /*
206− * Ideally we want a set of engines on a single leaf as we expect
207− * to mostly be tracking synchronisation between engines. It is not
208− * a huge issue if this is not the case, but we may want to mitigate
209− * any page crossing penalties if they become an issue.
210− *
211− * Called during early_init before we know how many engines there are.
212− */
213− BUILD_BUG_ON(KSYNCMAP < I915_NUM_ENGINES);
213+ kref_init(&timeline->kref);
214214
215− timeline->i915 = i915;
215+ timeline->gt = gt;
216216 timeline->pin_count = 0;
217+
217218 timeline->has_initial_breadcrumb = !hwsp;
218219 timeline->hwsp_cacheline = NULL;
219220
220221 if (!hwsp) {
221− struct i915_timeline_cacheline *cl;
222+ struct intel_timeline_cacheline *cl;
222223 unsigned int cacheline;
223224
224225 hwsp = hwsp_alloc(timeline, &cacheline);
@@ -261,55 +262,47 @@ int i915_timeline_init(struct drm_i915_private *i915, · 36 unmodified lines
261262 return 0;
262263 }
263264
264−void i915_timelines_init(struct drm_i915_private *i915)
265+static void timelines_init(struct intel_gt *gt)
265266 {
266− struct i915_gt_timelines *gt = &i915->gt.timelines;
267+ struct intel_gt_timelines *timelines = &gt->timelines;
267268
268− mutex_init(&gt->mutex);
269− INIT_LIST_HEAD(&gt->active_list);
269+ mutex_init(&timelines->mutex);
270+ INIT_LIST_HEAD(&timelines->active_list);
270271
271− spin_lock_init(&gt->hwsp_lock);
272− INIT_LIST_HEAD(&gt->hwsp_free_list);
272+ spin_lock_init(&timelines->hwsp_lock);
273+ INIT_LIST_HEAD(&timelines->hwsp_free_list);
274+}
273275
274− /* via i915_gem_wait_for_idle() */
275− i915_gem_shrinker_taints_mutex(i915, &gt->mutex);
276+void intel_timelines_init(struct drm_i915_private *i915)
277+{
278+ timelines_init(&i915->gt);
276279 }
277280
278−static void timeline_add_to_active(struct i915_timeline *tl)
281+static void timeline_add_to_active(struct intel_timeline *tl)
279282 {
280− struct i915_gt_timelines *gt = &tl->i915->gt.timelines;
283+ struct intel_gt_timelines *gt = &tl->gt->timelines;
281284
282285 mutex_lock(&gt->mutex);
283286 list_add(&tl->link, &gt->active_list);
284287 mutex_unlock(&gt->mutex);
285288 }
286289
287−static void timeline_remove_from_active(struct i915_timeline *tl)
290+static void timeline_remove_from_active(struct intel_timeline *tl)
288291 {
289− struct i915_gt_timelines *gt = &tl->i915->gt.timelines;
292+ struct intel_gt_timelines *gt = &tl->gt->timelines;
290293
291294 mutex_lock(&gt->mutex);
292295 list_del(&tl->link);
293296 mutex_unlock(&gt->mutex);
294297 }
295298
296−/**
297− * i915_timelines_park - called when the driver idles
298− * @i915: the drm_i915_private device
299− *
300− * When the driver is completely idle, we know that all of our sync points
301− * have been signaled and our tracking is then entirely redundant. Any request
302− * to wait upon an older sync point will be completed instantly as we know
303− * the fence is signaled and therefore we will not even look them up in the
304− * sync point map.
305− */
306−void i915_timelines_park(struct drm_i915_private *i915)
299+static void timelines_park(struct intel_gt *gt)
307300 {
308− struct i915_gt_timelines *gt = &i915->gt.timelines;
309− struct i915_timeline *timeline;
301+ struct intel_gt_timelines *timelines = &gt->timelines;
302+ struct intel_timeline *timeline;
310303
311− mutex_lock(&gt->mutex);
312− list_for_each_entry(timeline, &gt->active_list, link) {
304+ mutex_lock(&timelines->mutex);
305+ list_for_each_entry(timeline, &timelines->active_list, link) {
313306 /*
314307 * All known fences are completed so we can scrap
315308 * the current sync point tracking and start afresh,
@@ -318,10 +311,25 @@ void i915_timelines_park(struct drm_i915_private *i915) · 2 unmodified lines
318311 */
319312 i915_syncmap_free(&timeline->sync);
320313 }
321− mutex_unlock(&gt->mutex);
314+ mutex_unlock(&timelines->mutex);
322315 }
323316
324−void i915_timeline_fini(struct i915_timeline *timeline)
317+/**
318+ * intel_timelines_park - called when the driver idles
319+ * @i915: the drm_i915_private device
320+ *
321+ * When the driver is completely idle, we know that all of our sync points
322+ * have been signaled and our tracking is then entirely redundant. Any request
323+ * to wait upon an older sync point will be completed instantly as we know
324+ * the fence is signaled and therefore we will not even look them up in the
325+ * sync point map.
326+ */
327+void intel_timelines_park(struct drm_i915_private *i915)
328+{
329+ timelines_park(&i915->gt);
330+}
331+
332+void intel_timeline_fini(struct intel_timeline *timeline)
325333 {
326334 GEM_BUG_ON(timeline->pin_count);
327335 GEM_BUG_ON(!list_empty(&timeline->requests));
@@ -336,29 +344,26 @@ void i915_timeline_fini(struct i915_timeline *timeline) · 8 unmodified lines
336344 i915_vma_put(timeline->hwsp_ggtt);
337345 }
338346
339−struct i915_timeline *
340−i915_timeline_create(struct drm_i915_private *i915,
341− struct i915_vma *global_hwsp)
347+struct intel_timeline *
348+intel_timeline_create(struct intel_gt *gt, struct i915_vma *global_hwsp)
342349 {
343− struct i915_timeline *timeline;
350+ struct intel_timeline *timeline;
344351 int err;
345352
346353 timeline = kzalloc(sizeof(*timeline), GFP_KERNEL);
347354 if (!timeline)
348355 return ERR_PTR(-ENOMEM);
349356
350− err = i915_timeline_init(i915, timeline, global_hwsp);
357+ err = intel_timeline_init(timeline, gt, global_hwsp);
351358 if (err) {
352359 kfree(timeline);
353360 return ERR_PTR(err);
354361 }
355362
356− kref_init(&timeline->kref);
357−
358363 return timeline;
359364 }
360365
361−int i915_timeline_pin(struct i915_timeline *tl)
366+int intel_timeline_pin(struct intel_timeline *tl)
362367 {
363368 int err;
364369
@@ -384,7 +389,7 @@ unpin: · 19 unmodified lines
384389 return err;
385390 }
386391
387−static u32 timeline_advance(struct i915_timeline *tl)
392+static u32 timeline_advance(struct intel_timeline *tl)
388393 {
389394 GEM_BUG_ON(!tl->pin_count);
390395 GEM_BUG_ON(tl->seqno & tl->has_initial_breadcrumb);
@@ -392,17 +397,17 @@ static u32 timeline_advance(struct i915_timeline *tl) · 1 unmodified line
392397 return tl->seqno += 1 + tl->has_initial_breadcrumb;
393398 }
394399
395−static void timeline_rollback(struct i915_timeline *tl)
400+static void timeline_rollback(struct intel_timeline *tl)
396401 {
397402 tl->seqno -= 1 + tl->has_initial_breadcrumb;
398403 }
399404
400405 static noinline int
401−__i915_timeline_get_seqno(struct i915_timeline *tl,
402− struct i915_request *rq,
403− u32 *seqno)
406+__intel_timeline_get_seqno(struct intel_timeline *tl,
407+ struct i915_request *rq,
408+ u32 *seqno)
404409 {
405− struct i915_timeline_cacheline *cl;
410+ struct intel_timeline_cacheline *cl;
406411 unsigned int cacheline;
407412 struct i915_vma *vma;
408413 void *vaddr;
@@ -488,31 +493,31 @@ err_rollback: · 79 unmodified lines
488493 return err;
489494 }
490495
491−int i915_timeline_get_seqno(struct i915_timeline *tl,
492− struct i915_request *rq,
493− u32 *seqno)
496+int intel_timeline_get_seqno(struct intel_timeline *tl,
497+ struct i915_request *rq,
498+ u32 *seqno)
494499 {
495500 *seqno = timeline_advance(tl);
496501
497502 /* Replace the HWSP on wraparound for HW semaphores */
498503 if (unlikely(!*seqno && tl->hwsp_cacheline))
499− return __i915_timeline_get_seqno(tl, rq, seqno);
504+ return __intel_timeline_get_seqno(tl, rq, seqno);
500505
501506 return 0;
502507 }
503508
504−static int cacheline_ref(struct i915_timeline_cacheline *cl,
509+static int cacheline_ref(struct intel_timeline_cacheline *cl,
505510 struct i915_request *rq)
506511 {
507512 return i915_active_ref(&cl->active, rq->fence.context, rq);
508513 }
509514
510−int i915_timeline_read_hwsp(struct i915_request *from,
511− struct i915_request *to,
512− u32 *hwsp)
515+int intel_timeline_read_hwsp(struct i915_request *from,
516+ struct i915_request *to,
517+ u32 *hwsp)
513518 {
514− struct i915_timeline_cacheline *cl = from->hwsp_cacheline;
515− struct i915_timeline *tl = from->timeline;
519+ struct intel_timeline_cacheline *cl = from->hwsp_cacheline;
520+ struct intel_timeline *tl = from->timeline;
516521 int err;
517522
518523 GEM_BUG_ON(to->timeline == tl);
@@ -535,7 +540,7 @@ int i915_timeline_read_hwsp(struct i915_request *from, · 16 unmodified lines
535540 return err;
536541 }
537542
538−void i915_timeline_unpin(struct i915_timeline *tl)
543+void intel_timeline_unpin(struct intel_timeline *tl)
539544 {
540545 GEM_BUG_ON(!tl->pin_count);
541546 if (--tl->pin_count)
@@ -554,26 +559,31 @@ void i915_timeline_unpin(struct i915_timeline *tl) · 12 unmodified lines
554559 __i915_vma_unpin(tl->hwsp_ggtt);
555560 }
556561
557−void __i915_timeline_free(struct kref *kref)
562+void __intel_timeline_free(struct kref *kref)
558563 {
559− struct i915_timeline *timeline =
564+ struct intel_timeline *timeline =
560565 container_of(kref, typeof(*timeline), kref);
561566
562− i915_timeline_fini(timeline);
567+ intel_timeline_fini(timeline);
563568 kfree(timeline);
564569 }
565570
566−void i915_timelines_fini(struct drm_i915_private *i915)
571+static void timelines_fini(struct intel_gt *gt)
567572 {
568− struct i915_gt_timelines *gt = &i915->gt.timelines;
573+ struct intel_gt_timelines *timelines = &gt->timelines;
569574
570− GEM_BUG_ON(!list_empty(&gt->active_list));
571− GEM_BUG_ON(!list_empty(&gt->hwsp_free_list));
575+ GEM_BUG_ON(!list_empty(&timelines->active_list));
576+ GEM_BUG_ON(!list_empty(&timelines->hwsp_free_list));
572577
573− mutex_destroy(&gt->mutex);
578+ mutex_destroy(&timelines->mutex);
579+}
580+
581+void intel_timelines_fini(struct drm_i915_private *i915)
582+{
583+ timelines_fini(&i915->gt);
574584 }
575585
576586 #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
577−#include "selftests/mock_timeline.c"
578−#include "selftests/i915_timeline.c"
587+#include "gt/selftests/mock_timeline.c"
588+#include "gt/selftest_timeline.c"
579589 #endif
drivers/gpu/drm/i915/gt/intel_timeline.h+93−0
@@ -0,0 +1,93 @@
1+/*
2+ * Copyright © 2016 Intel Corporation
3+ *
4+ * Permission is hereby granted, free of charge, to any person obtaining a
5+ * copy of this software and associated documentation files (the "Software"),
6+ * to deal in the Software without restriction, including without limitation
7+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
8+ * and/or sell copies of the Software, and to permit persons to whom the
9+ * Software is furnished to do so, subject to the following conditions:
10+ *
11+ * The above copyright notice and this permission notice (including the next
12+ * paragraph) shall be included in all copies or substantial portions of the
13+ * Software.
14+ *
15+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
18+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
20+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
21+ * IN THE SOFTWARE.
22+ *
23+ */
24+
25+#ifndef I915_TIMELINE_H
26+#define I915_TIMELINE_H
27+
28+#include <linux/lockdep.h>
29+
30+#include "i915_active.h"
31+#include "i915_syncmap.h"
32+#include "gt/intel_timeline_types.h"
33+
34+int intel_timeline_init(struct intel_timeline *tl,
35+ struct intel_gt *gt,
36+ struct i915_vma *hwsp);
37+void intel_timeline_fini(struct intel_timeline *tl);
38+
39+struct intel_timeline *
40+intel_timeline_create(struct intel_gt *gt, struct i915_vma *global_hwsp);
41+
42+static inline struct intel_timeline *
43+intel_timeline_get(struct intel_timeline *timeline)
44+{
45+ kref_get(&timeline->kref);
46+ return timeline;
47+}
48+
49+void __intel_timeline_free(struct kref *kref);
50+static inline void intel_timeline_put(struct intel_timeline *timeline)
51+{
52+ kref_put(&timeline->kref, __intel_timeline_free);
53+}
54+
55+static inline int __intel_timeline_sync_set(struct intel_timeline *tl,
56+ u64 context, u32 seqno)
57+{
58+ return i915_syncmap_set(&tl->sync, context, seqno);
59+}
60+
61+static inline int intel_timeline_sync_set(struct intel_timeline *tl,
62+ const struct dma_fence *fence)
63+{
64+ return __intel_timeline_sync_set(tl, fence->context, fence->seqno);
65+}
66+
67+static inline bool __intel_timeline_sync_is_later(struct intel_timeline *tl,
68+ u64 context, u32 seqno)
69+{
70+ return i915_syncmap_is_later(&tl->sync, context, seqno);
71+}
72+
73+static inline bool intel_timeline_sync_is_later(struct intel_timeline *tl,
74+ const struct dma_fence *fence)
75+{
76+ return __intel_timeline_sync_is_later(tl, fence->context, fence->seqno);
77+}
78+
79+int intel_timeline_pin(struct intel_timeline *tl);
80+int intel_timeline_get_seqno(struct intel_timeline *tl,
81+ struct i915_request *rq,
82+ u32 *seqno);
83+void intel_timeline_unpin(struct intel_timeline *tl);
84+
85+int intel_timeline_read_hwsp(struct i915_request *from,
86+ struct i915_request *until,
87+ u32 *hwsp_offset);
88+
89+void intel_timelines_init(struct drm_i915_private *i915);
90+void intel_timelines_park(struct drm_i915_private *i915);
91+void intel_timelines_fini(struct drm_i915_private *i915);
92+
93+#endif
drivers/gpu/drm/i915/gt/intel_timeline_types.hrenamed from drivers/gpu/drm/i915/i915_timeline_types.h+4−4
@@ -16,10 +16,10 @@ · 15 unmodified lines
1616
1717 struct drm_i915_private;
1818 struct i915_vma;
19−struct i915_timeline_cacheline;
19+struct intel_timeline_cacheline;
2020 struct i915_syncmap;
2121
22−struct i915_timeline {
22+struct intel_timeline {
2323 u64 fence_context;
2424 u32 seqno;
2525
@@ -30,7 +30,7 @@ struct i915_timeline { · 4 unmodified lines
3030 struct i915_vma *hwsp_ggtt;
3131 u32 hwsp_offset;
3232
33− struct i915_timeline_cacheline *hwsp_cacheline;
33+ struct intel_timeline_cacheline *hwsp_cacheline;
3434
3535 bool has_initial_breadcrumb;
3636
@@ -59,7 +59,7 @@ struct i915_timeline { · 22 unmodified lines
5959 struct i915_syncmap *sync;
6060
6161 struct list_head link;
62− struct drm_i915_private *i915;
62+ struct intel_gt *gt;
6363
6464 struct kref kref;
6565 };
drivers/gpu/drm/i915/gt/intel_workarounds.c+133−94
@@ -6,6 +6,7 @@ · 5 unmodified lines
66
77 #include "i915_drv.h"
88 #include "intel_context.h"
9+#include "intel_gt.h"
910 #include "intel_workarounds.h"
1011
1112 /**
@@ -49,9 +50,10 @@ · 37 unmodified lines
4950 * - Public functions to init or apply the given workaround type.
5051 */
5152
52−static void wa_init_start(struct i915_wa_list *wal, const char *name)
53+static void wa_init_start(struct i915_wa_list *wal, const char *name, const char *engine_name)
5354 {
5455 wal->name = name;
56+ wal->engine_name = engine_name;
5557 }
5658
5759 #define WA_LIST_CHUNK (1 << 4)
@@ -73,8 +75,8 @@ static void wa_init_finish(struct i915_wa_list *wal) · 15 unmodified lines
7375 if (!wal->count)
7476 return;
7577
76− DRM_DEBUG_DRIVER("Initialized %u %s workarounds\n",
77− wal->wa_count, wal->name);
78+ DRM_DEBUG_DRIVER("Initialized %u %s workarounds on %s\n",
79+ wal->wa_count, wal->name, wal->engine_name);
7880 }
7981
8082 static void _wa_add(struct i915_wa_list *wal, const struct i915_wa *wa)
@@ -175,19 +177,6 @@ wa_write_or(struct i915_wa_list *wal, i915_reg_t reg, u32 val) · 94 unmodified lines
175177 wa_write_masked_or(wal, reg, val, val);
176178 }
177179
178−static void
179−ignore_wa_write_or(struct i915_wa_list *wal, i915_reg_t reg, u32 mask, u32 val)
180−{
181− struct i915_wa wa = {
182− .reg = reg,
183− .mask = mask,
184− .val = val,
185− /* Bonkers HW, skip verifying */
186− };
187−
188− _wa_add(wal, &wa);
189−}
190−
191180 #define WA_SET_BIT_MASKED(addr, mask) \
192181 wa_write_masked_or(wal, (addr), (mask), _MASKED_BIT_ENABLE(mask))
193182
@@ -530,12 +519,6 @@ static void icl_ctx_workarounds_init(struct intel_engine_cs *engine, · 336 unmodified lines
530519 {
531520 struct drm_i915_private *i915 = engine->i915;
532521
533− /* WaDisableBankHangMode:icl */
534− wa_write(wal,
535− GEN8_L3CNTLREG,
536− intel_uncore_read(engine->uncore, GEN8_L3CNTLREG) |
537− GEN8_ERRDETBCTRL);
538−
539522 /* WaDisableBankHangMode:icl */
540523 wa_write(wal,
541524 GEN8_L3CNTLREG,
@@ -596,7 +579,7 @@ __intel_engine_init_ctx_wa(struct intel_engine_cs *engine, · 54 unmodified lines
596579 if (engine->class != RENDER_CLASS)
597580 return;
598581
599− wa_init_start(wal, name);
582+ wa_init_start(wal, name, engine->name);
600583
601584 if (IS_GEN(i915, 11))
602585 icl_ctx_workarounds_init(engine, wal);
@@ -766,7 +749,10 @@ static void · 163 unmodified lines
766749 wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal)
767750 {
768751 const struct sseu_dev_info *sseu = &RUNTIME_INFO(i915)->sseu;
769− u32 mcr_slice_subslice_mask;
752+ unsigned int slice, subslice;
753+ u32 l3_en, mcr, mcr_mask;
754+
755+ GEM_BUG_ON(INTEL_GEN(i915) < 10);
770756
771757 /*
772758 * WaProgramMgsrForL3BankSpecificMmioReads: cnl,icl
@@ -774,42 +760,7 @@ wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal) · 1 unmodified line
774760 * the case, we might need to program MCR select to a valid L3Bank
775761 * by default, to make sure we correctly read certain registers
776762 * later on (in the range 0xB100 - 0xB3FF).
777− * This might be incompatible with
778− * WaProgramMgsrForCorrectSliceSpecificMmioReads.
779− * Fortunately, this should not happen in production hardware, so
780− * we only assert that this is the case (instead of implementing
781− * something more complex that requires checking the range of every
782− * MMIO read).
783− */
784− if (INTEL_GEN(i915) >= 10 &&
785− is_power_of_2(sseu->slice_mask)) {
786− /*
787− * read FUSE3 for enabled L3 Bank IDs, if L3 Bank matches
788− * enabled subslice, no need to redirect MCR packet
789− */
790− u32 slice = fls(sseu->slice_mask);
791− u32 fuse3 =
792− intel_uncore_read(&i915->uncore, GEN10_MIRROR_FUSE3);
793− u8 ss_mask = sseu->subslice_mask[slice];
794−
795− u8 enabled_mask = (ss_mask | ss_mask >>
796− GEN10_L3BANK_PAIR_COUNT) & GEN10_L3BANK_MASK;
797− u8 disabled_mask = fuse3 & GEN10_L3BANK_MASK;
798−
799− /*
800− * Production silicon should have matched L3Bank and
801− * subslice enabled
802− */
803− WARN_ON((enabled_mask & disabled_mask) != enabled_mask);
804− }
805−
806− if (INTEL_GEN(i915) >= 11)
807− mcr_slice_subslice_mask = GEN11_MCR_SLICE_MASK |
808− GEN11_MCR_SUBSLICE_MASK;
809− else
810− mcr_slice_subslice_mask = GEN8_MCR_SLICE_MASK |
811− GEN8_MCR_SUBSLICE_MASK;
812− /*
763+ *
813764 * WaProgramMgsrForCorrectSliceSpecificMmioReads:cnl,icl
814765 * Before any MMIO read into slice/subslice specific registers, MCR
815766 * packet control register needs to be programmed to point to any
@@ -819,11 +770,51 @@ wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal) · 3 unmodified lines
819770 * are consistent across s/ss in almost all cases. In the rare
820771 * occasions, such as INSTDONE, where this value is dependent
821772 * on s/ss combo, the read should be done with read_subslice_reg.
773+ *
774+ * Since GEN8_MCR_SELECTOR contains dual-purpose bits which select both
775+ * to which subslice, or to which L3 bank, the respective mmio reads
776+ * will go, we have to find a common index which works for both
777+ * accesses.
778+ *
779+ * Case where we cannot find a common index fortunately should not
780+ * happen in production hardware, so we only emit a warning instead of
781+ * implementing something more complex that requires checking the range
782+ * of every MMIO read.
822783 */
823− wa_write_masked_or(wal,
824− GEN8_MCR_SELECTOR,
825− mcr_slice_subslice_mask,
826− intel_calculate_mcr_s_ss_select(i915));
784+
785+ if (INTEL_GEN(i915) >= 10 && is_power_of_2(sseu->slice_mask)) {
786+ u32 l3_fuse =
787+ intel_uncore_read(&i915->uncore, GEN10_MIRROR_FUSE3) &
788+ GEN10_L3BANK_MASK;
789+
790+ DRM_DEBUG_DRIVER("L3 fuse = %x\n", l3_fuse);
791+ l3_en = ~(l3_fuse << GEN10_L3BANK_PAIR_COUNT | l3_fuse);
792+ } else {
793+ l3_en = ~0;
794+ }
795+
796+ slice = fls(sseu->slice_mask) - 1;
797+ GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask));
798+ subslice = fls(l3_en & sseu->subslice_mask[slice]);
799+ if (!subslice) {
800+ DRM_WARN("No common index found between subslice mask %x and L3 bank mask %x!\n",
801+ sseu->subslice_mask[slice], l3_en);
802+ subslice = fls(l3_en);
803+ WARN_ON(!subslice);
804+ }
805+ subslice--;
806+
807+ if (INTEL_GEN(i915) >= 11) {
808+ mcr = GEN11_MCR_SLICE(slice) | GEN11_MCR_SUBSLICE(subslice);
809+ mcr_mask = GEN11_MCR_SLICE_MASK | GEN11_MCR_SUBSLICE_MASK;
810+ } else {
811+ mcr = GEN8_MCR_SLICE(slice) | GEN8_MCR_SUBSLICE(subslice);
812+ mcr_mask = GEN8_MCR_SLICE_MASK | GEN8_MCR_SUBSLICE_MASK;
813+ }
814+
815+ DRM_DEBUG_DRIVER("MCR slice/subslice = %x\n", mcr);
816+
817+ wa_write_masked_or(wal, GEN8_MCR_SELECTOR, mcr_mask, mcr);
827818 }
828819
829820 static void
@@ -926,7 +917,7 @@ void intel_gt_init_workarounds(struct drm_i915_private *i915) · 96 unmodified lines
926917 {
927918 struct i915_wa_list *wal = &i915->gt_wa_list;
928919
929− wa_init_start(wal, "GT");
920+ wa_init_start(wal, "GT", "global");
930921 gt_init_workarounds(i915, wal);
931922 wa_init_finish(wal);
932923 }
@@ -990,9 +981,9 @@ wa_list_apply(struct intel_uncore *uncore, const struct i915_wa_list *wal) · 57 unmodified lines
990981 spin_unlock_irqrestore(&uncore->lock, flags);
991982 }
992983
993−void intel_gt_apply_workarounds(struct drm_i915_private *i915)
984+void intel_gt_apply_workarounds(struct intel_gt *gt)
994985 {
995− wa_list_apply(&i915->uncore, &i915->gt_wa_list);
986+ wa_list_apply(gt->uncore, &gt->i915->gt_wa_list);
996987 }
997988
998989 static bool wa_list_verify(struct intel_uncore *uncore,
@@ -1011,10 +1002,23 @@ static bool wa_list_verify(struct intel_uncore *uncore, · 12 unmodified lines
10111002 return ok;
10121003 }
10131004
1014−bool intel_gt_verify_workarounds(struct drm_i915_private *i915,
1015− const char *from)
1005+bool intel_gt_verify_workarounds(struct intel_gt *gt, const char *from)
1006+{
1007+ return wa_list_verify(gt->uncore, &gt->i915->gt_wa_list, from);
1008+}
1009+
1010+static inline bool is_nonpriv_flags_valid(u32 flags)
10161011 {
1017− return wa_list_verify(&i915->uncore, &i915->gt_wa_list, from);
1012+ /* Check only valid flag bits are set */
1013+ if (flags & ~RING_FORCE_TO_NONPRIV_MASK_VALID)
1014+ return false;
1015+
1016+ /* NB: Only 3 out of 4 enum values are valid for access field */
1017+ if ((flags & RING_FORCE_TO_NONPRIV_ACCESS_MASK) ==
1018+ RING_FORCE_TO_NONPRIV_ACCESS_INVALID)
1019+ return false;
1020+
1021+ return true;
10181022 }
10191023
10201024 static void
@@ -1027,6 +1031,9 @@ whitelist_reg_ext(struct i915_wa_list *wal, i915_reg_t reg, u32 flags) · 6 unmodified lines
10271031 if (GEM_DEBUG_WARN_ON(wal->count >= RING_MAX_NONPRIV_SLOTS))
10281032 return;
10291033
1034+ if (GEM_DEBUG_WARN_ON(!is_nonpriv_flags_valid(flags)))
1035+ return;
1036+
10301037 wa.reg.reg |= flags;
10311038 _wa_add(wal, &wa);
10321039 }
@@ -1034,7 +1041,7 @@ whitelist_reg_ext(struct i915_wa_list *wal, i915_reg_t reg, u32 flags) · 1 unmodified line
10341041 static void
10351042 whitelist_reg(struct i915_wa_list *wal, i915_reg_t reg)
10361043 {
1037− whitelist_reg_ext(wal, reg, RING_FORCE_TO_NONPRIV_RW);
1044+ whitelist_reg_ext(wal, reg, RING_FORCE_TO_NONPRIV_ACCESS_RW);
10381045 }
10391046
10401047 static void gen9_whitelist_build(struct i915_wa_list *w)
@@ -1115,7 +1122,7 @@ static void cfl_whitelist_build(struct intel_engine_cs *engine) · 74 unmodified lines
11151122 * - PS_DEPTH_COUNT_UDW
11161123 */
11171124 whitelist_reg_ext(w, PS_INVOCATION_COUNT,
1118− RING_FORCE_TO_NONPRIV_RD |
1125+ RING_FORCE_TO_NONPRIV_ACCESS_RD |
11191126 RING_FORCE_TO_NONPRIV_RANGE_4);
11201127 }
11211128
@@ -1155,20 +1162,20 @@ static void icl_whitelist_build(struct intel_engine_cs *engine) · 33 unmodified lines
11551162 * - PS_DEPTH_COUNT_UDW
11561163 */
11571164 whitelist_reg_ext(w, PS_INVOCATION_COUNT,
1158− RING_FORCE_TO_NONPRIV_RD |
1165+ RING_FORCE_TO_NONPRIV_ACCESS_RD |
11591166 RING_FORCE_TO_NONPRIV_RANGE_4);
11601167 break;
11611168
11621169 case VIDEO_DECODE_CLASS:
11631170 /* hucStatusRegOffset */
11641171 whitelist_reg_ext(w, _MMIO(0x2000 + engine->mmio_base),
1165− RING_FORCE_TO_NONPRIV_RD);
1172+ RING_FORCE_TO_NONPRIV_ACCESS_RD);
11661173 /* hucUKernelHdrInfoRegOffset */
11671174 whitelist_reg_ext(w, _MMIO(0x2014 + engine->mmio_base),
1168− RING_FORCE_TO_NONPRIV_RD);
1175+ RING_FORCE_TO_NONPRIV_ACCESS_RD);
11691176 /* hucStatus2RegOffset */
11701177 whitelist_reg_ext(w, _MMIO(0x23B0 + engine->mmio_base),
1171− RING_FORCE_TO_NONPRIV_RD);
1178+ RING_FORCE_TO_NONPRIV_ACCESS_RD);
11721179 break;
11731180
11741181 default:
@@ -1181,7 +1188,7 @@ void intel_engine_init_whitelist(struct intel_engine_cs *engine) · 6 unmodified lines
11811188 struct drm_i915_private *i915 = engine->i915;
11821189 struct i915_wa_list *w = &engine->whitelist;
11831190
1184− wa_init_start(w, "whitelist");
1191+ wa_init_start(w, "whitelist", engine->name);
11851192
11861193 if (IS_GEN(i915, 11))
11871194 icl_whitelist_build(engine);
@@ -1240,10 +1247,9 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal) · 52 unmodified lines
12401247 _3D_CHICKEN3_AA_LINE_QUALITY_FIX_ENABLE);
12411248
12421249 /* WaPipelineFlushCoherentLines:icl */
1243− ignore_wa_write_or(wal,
1244− GEN8_L3SQCREG4,
1245− GEN8_LQSC_FLUSH_COHERENT_LINES,
1246− GEN8_LQSC_FLUSH_COHERENT_LINES);
1250+ wa_write_or(wal,
1251+ GEN8_L3SQCREG4,
1252+ GEN8_LQSC_FLUSH_COHERENT_LINES);
12471253
12481254 /*
12491255 * Wa_1405543622:icl
@@ -1270,10 +1276,9 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal) · 20 unmodified lines
12701276 * Wa_1405733216:icl
12711277 * Formerly known as WaDisableCleanEvicts
12721278 */
1273− ignore_wa_write_or(wal,
1274− GEN8_L3SQCREG4,
1275− GEN11_LQSC_CLEAN_EVICT_DISABLE,
1276− GEN11_LQSC_CLEAN_EVICT_DISABLE);
1279+ wa_write_or(wal,
1280+ GEN8_L3SQCREG4,
1281+ GEN11_LQSC_CLEAN_EVICT_DISABLE);
12771282
12781283 /* WaForwardProgressSoftReset:icl */
12791284 wa_write_or(wal,
@@ -1292,6 +1297,12 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal) · 12 unmodified lines
12921297 wa_write_or(wal,
12931298 GEN7_SARCHKMD,
12941299 GEN7_DISABLE_SAMPLER_PREFETCH);
1300+
1301+ /* Wa_1409178092:icl */
1302+ wa_write_masked_or(wal,
1303+ GEN11_SCRATCH2,
1304+ GEN11_COHERENT_PARTIAL_WRITE_MERGE_ENABLE,
1305+ 0);
12951306 }
12961307
12971308 if (IS_GEN_RANGE(i915, 9, 11)) {
@@ -1360,7 +1371,7 @@ engine_init_workarounds(struct intel_engine_cs *engine, struct i915_wa_list *wal · 62 unmodified lines
13601371 if (I915_SELFTEST_ONLY(INTEL_GEN(engine->i915) < 8))
13611372 return;
13621373
1363− if (engine->id == RCS0)
1374+ if (engine->class == RENDER_CLASS)
13641375 rcs_engine_wa_init(engine, wal);
13651376 else
13661377 xcs_engine_wa_init(engine, wal);
@@ -1370,10 +1381,10 @@ void intel_engine_init_workarounds(struct intel_engine_cs *engine) · 3 unmodified lines
13701381 {
13711382 struct i915_wa_list *wal = &engine->wa_list;
13721383
1373− if (GEM_WARN_ON(INTEL_GEN(engine->i915) < 8))
1384+ if (INTEL_GEN(engine->i915) < 8)
13741385 return;
13751386
1376− wa_init_start(wal, engine->name);
1387+ wa_init_start(wal, "engine", engine->name);
13771388 engine_init_workarounds(engine, wal);
13781389 wa_init_finish(wal);
13791390 }
@@ -1416,26 +1427,50 @@ err_obj: · 36 unmodified lines
14161427 return ERR_PTR(err);
14171428 }
14181429
1430+static bool mcr_range(struct drm_i915_private *i915, u32 offset)
1431+{
1432+ /*
1433+ * Registers in this range are affected by the MCR selector
1434+ * which only controls CPU initiated MMIO. Routing does not
1435+ * work for CS access so we cannot verify them on this path.
1436+ */
1437+ if (INTEL_GEN(i915) >= 8 && (offset >= 0xb100 && offset <= 0xb3ff))
1438+ return true;
1439+
1440+ return false;
1441+}
1442+
14191443 static int
14201444 wa_list_srm(struct i915_request *rq,
14211445 const struct i915_wa_list *wal,
14221446 struct i915_vma *vma)
14231447 {
1448+ struct drm_i915_private *i915 = rq->i915;
1449+ unsigned int i, count = 0;
14241450 const struct i915_wa *wa;
1425− unsigned int i;
14261451 u32 srm, *cs;
14271452
14281453 srm = MI_STORE_REGISTER_MEM | MI_SRM_LRM_GLOBAL_GTT;
1429− if (INTEL_GEN(rq->i915) >= 8)
1454+ if (INTEL_GEN(i915) >= 8)
14301455 srm++;
14311456
1432− cs = intel_ring_begin(rq, 4 * wal->count);
1457+ for (i = 0, wa = wal->list; i < wal->count; i++, wa++) {
1458+ if (!mcr_range(i915, i915_mmio_reg_offset(wa->reg)))
1459+ count++;
1460+ }
1461+
1462+ cs = intel_ring_begin(rq, 4 * count);
14331463 if (IS_ERR(cs))
14341464 return PTR_ERR(cs);
14351465
14361466 for (i = 0, wa = wal->list; i < wal->count; i++, wa++) {
1467+ u32 offset = i915_mmio_reg_offset(wa->reg);
1468+
1469+ if (mcr_range(i915, offset))
1470+ continue;
1471+
14371472 *cs++ = srm;
1438− *cs++ = i915_mmio_reg_offset(wa->reg);
1473+ *cs++ = offset;
14391474 *cs++ = i915_ggtt_offset(vma) + sizeof(u32) * i;
14401475 *cs++ = 0;
14411476 }
@@ -1458,7 +1493,7 @@ static int engine_wa_list_verify(struct intel_context *ce, · 16 unmodified lines
14581493 if (!wal->count)
14591494 return 0;
14601495
1461− vma = create_scratch(&ce->engine->i915->ggtt.vm, wal->count);
1496+ vma = create_scratch(&ce->engine->gt->ggtt->vm, wal->count);
14621497 if (IS_ERR(vma))
14631498 return PTR_ERR(vma);
14641499
@@ -1485,9 +1520,13 @@ static int engine_wa_list_verify(struct intel_context *ce, · 20 unmodified lines
14851520 }
14861521
14871522 err = 0;
1488− for (i = 0, wa = wal->list; i < wal->count; i++, wa++)
1523+ for (i = 0, wa = wal->list; i < wal->count; i++, wa++) {
1524+ if (mcr_range(rq->i915, i915_mmio_reg_offset(wa->reg)))
1525+ continue;
1526+
14891527 if (!wa_verify(wa, results[i], wal->name, from))
14901528 err = -ENXIO;
1529+ }
14911530
14921531 i915_gem_object_unpin_map(vma->obj);
14931532
drivers/gpu/drm/i915/gt/intel_workarounds.h+3−3
@@ -14,6 +14,7 @@ · 13 unmodified lines
1414 struct drm_i915_private;
1515 struct i915_request;
1616 struct intel_engine_cs;
17+struct intel_gt;
1718
1819 static inline void intel_wa_list_free(struct i915_wa_list *wal)
1920 {
@@ -25,9 +26,8 @@ void intel_engine_init_ctx_wa(struct intel_engine_cs *engine); · 5 unmodified lines
2526 int intel_engine_emit_ctx_wa(struct i915_request *rq);
2627
2728 void intel_gt_init_workarounds(struct drm_i915_private *i915);
28−void intel_gt_apply_workarounds(struct drm_i915_private *i915);
29−bool intel_gt_verify_workarounds(struct drm_i915_private *i915,
30− const char *from);
29+void intel_gt_apply_workarounds(struct intel_gt *gt);
30+bool intel_gt_verify_workarounds(struct intel_gt *gt, const char *from);
3131
3232 void intel_engine_init_whitelist(struct intel_engine_cs *engine);
3333 void intel_engine_apply_whitelist(struct intel_engine_cs *engine);
drivers/gpu/drm/i915/gt/intel_workarounds_types.h+1−0
@@ -20,6 +20,7 @@ struct i915_wa { · 19 unmodified lines
2020
2121 struct i915_wa_list {
2222 const char *name;
23+ const char *engine_name;
2324 struct i915_wa *list;
2425 unsigned int count;
2526 unsigned int wa_count;
drivers/gpu/drm/i915/gt/mock_engine.c+8−6
@@ -33,15 +33,15 @@ · 32 unmodified lines
3333
3434 struct mock_ring {
3535 struct intel_ring base;
36− struct i915_timeline timeline;
36+ struct intel_timeline timeline;
3737 };
3838
39−static void mock_timeline_pin(struct i915_timeline *tl)
39+static void mock_timeline_pin(struct intel_timeline *tl)
4040 {
4141 tl->pin_count++;
4242 }
4343
44−static void mock_timeline_unpin(struct i915_timeline *tl)
44+static void mock_timeline_unpin(struct intel_timeline *tl)
4545 {
4646 GEM_BUG_ON(!tl->pin_count);
4747 tl->pin_count--;
@@ -56,7 +56,7 @@ static struct intel_ring *mock_ring(struct intel_engine_cs *engine) · 8 unmodified lines
5656 if (!ring)
5757 return NULL;
5858
59− if (i915_timeline_init(engine->i915, &ring->timeline, NULL)) {
59+ if (intel_timeline_init(&ring->timeline, engine->gt, NULL)) {
6060 kfree(ring);
6161 return NULL;
6262 }
@@ -78,7 +78,7 @@ static void mock_ring_free(struct intel_ring *base) · 15 unmodified lines
7878 {
7979 struct mock_ring *ring = container_of(base, typeof(*ring), base);
8080
81− i915_timeline_fini(&ring->timeline);
81+ intel_timeline_fini(&ring->timeline);
8282 kfree(ring);
8383 }
8484
@@ -142,6 +142,7 @@ static void mock_context_destroy(struct kref *ref) · 57 unmodified lines
142142 if (ce->ring)
143143 mock_ring_free(ce->ring);
144144
145+ intel_context_fini(ce);
145146 intel_context_free(ce);
146147 }
147148
@@ -155,7 +156,7 @@ static int mock_context_pin(struct intel_context *ce) · 7 unmodified lines
155156 return -ENOMEM;
156157 }
157158
158− ret = intel_context_active_acquire(ce, PIN_HIGH);
159+ ret = intel_context_active_acquire(ce);
159160 if (ret)
160161 return ret;
161162
@@ -257,6 +258,7 @@ struct intel_engine_cs *mock_engine(struct drm_i915_private *i915, · 95 unmodified lines
257258
258259 /* minimal engine setup for requests */
259260 engine->base.i915 = i915;
261+ engine->base.gt = &i915->gt;
260262 snprintf(engine->base.name, sizeof(engine->base.name), "%s", name);
261263 engine->base.id = id;
262264 engine->base.mask = BIT(id);
drivers/gpu/drm/i915/gt/selftest_hangcheck.c+252−259
@@ -25,13 +25,13 @@ · 24 unmodified lines
2525 #include <linux/kthread.h>
2626
2727 #include "gem/i915_gem_context.h"
28+#include "gt/intel_gt.h"
2829 #include "intel_engine_pm.h"
2930
3031 #include "i915_selftest.h"
3132 #include "selftests/i915_random.h"
3233 #include "selftests/igt_flush_test.h"
3334 #include "selftests/igt_reset.h"
34−#include "selftests/igt_wedge_me.h"
3535 #include "selftests/igt_atomic.h"
3636
3737 #include "selftests/mock_drm.h"
@@ -42,7 +42,7 @@ · 4 unmodified lines
4242 #define IGT_IDLE_TIMEOUT 50 /* ms; time to wait after flushing between tests */
4343
4444 struct hang {
45− struct drm_i915_private *i915;
45+ struct intel_gt *gt;
4646 struct drm_i915_gem_object *hws;
4747 struct drm_i915_gem_object *obj;
4848 struct i915_gem_context *ctx;
@@ -50,27 +50,27 @@ struct hang { · 1 unmodified line
5050 u32 *batch;
5151 };
5252
53−static int hang_init(struct hang *h, struct drm_i915_private *i915)
53+static int hang_init(struct hang *h, struct intel_gt *gt)
5454 {
5555 void *vaddr;
5656 int err;
5757
5858 memset(h, 0, sizeof(*h));
59− h->i915 = i915;
59+ h->gt = gt;
6060
61− h->ctx = kernel_context(i915);
61+ h->ctx = kernel_context(gt->i915);
6262 if (IS_ERR(h->ctx))
6363 return PTR_ERR(h->ctx);
6464
6565 GEM_BUG_ON(i915_gem_context_is_bannable(h->ctx));
6666
67− h->hws = i915_gem_object_create_internal(i915, PAGE_SIZE);
67+ h->hws = i915_gem_object_create_internal(gt->i915, PAGE_SIZE);
6868 if (IS_ERR(h->hws)) {
6969 err = PTR_ERR(h->hws);
7070 goto err_ctx;
7171 }
7272
73− h->obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
73+ h->obj = i915_gem_object_create_internal(gt->i915, PAGE_SIZE);
7474 if (IS_ERR(h->obj)) {
7575 err = PTR_ERR(h->obj);
7676 goto err_hws;
@@ -85,7 +85,7 @@ static int hang_init(struct hang *h, struct drm_i915_private *i915) · 8 unmodified lines
8585 h->seqno = memset(vaddr, 0xff, PAGE_SIZE);
8686
8787 vaddr = i915_gem_object_pin_map(h->obj,
88− i915_coherent_map_type(i915));
88+ i915_coherent_map_type(gt->i915));
8989 if (IS_ERR(vaddr)) {
9090 err = PTR_ERR(vaddr);
9191 goto err_unpin_hws;
@@ -127,35 +127,31 @@ static int move_to_active(struct i915_vma *vma, · 35 unmodified lines
127127 static struct i915_request *
128128 hang_create_request(struct hang *h, struct intel_engine_cs *engine)
129129 {
130− struct drm_i915_private *i915 = h->i915;
131− struct i915_address_space *vm = h->ctx->vm ?: &i915->ggtt.vm;
130+ struct intel_gt *gt = h->gt;
131+ struct i915_address_space *vm = h->ctx->vm ?: &engine->gt->ggtt->vm;
132+ struct drm_i915_gem_object *obj;
132133 struct i915_request *rq = NULL;
133134 struct i915_vma *hws, *vma;
134135 unsigned int flags;
136+ void *vaddr;
135137 u32 *batch;
136138 int err;
137139
138− if (i915_gem_object_is_active(h->obj)) {
139− struct drm_i915_gem_object *obj;
140− void *vaddr;
141−
142− obj = i915_gem_object_create_internal(h->i915, PAGE_SIZE);
143− if (IS_ERR(obj))
144− return ERR_CAST(obj);
140+ obj = i915_gem_object_create_internal(gt->i915, PAGE_SIZE);
141+ if (IS_ERR(obj))
142+ return ERR_CAST(obj);
145143
146− vaddr = i915_gem_object_pin_map(obj,
147− i915_coherent_map_type(h->i915));
148− if (IS_ERR(vaddr)) {
149− i915_gem_object_put(obj);
150− return ERR_CAST(vaddr);
151− }
144+ vaddr = i915_gem_object_pin_map(obj, i915_coherent_map_type(gt->i915));
145+ if (IS_ERR(vaddr)) {
146+ i915_gem_object_put(obj);
147+ return ERR_CAST(vaddr);
148+ }
152149
153− i915_gem_object_unpin_map(h->obj);
154− i915_gem_object_put(h->obj);
150+ i915_gem_object_unpin_map(h->obj);
151+ i915_gem_object_put(h->obj);
155152
156− h->obj = obj;
157− h->batch = vaddr;
158− }
153+ h->obj = obj;
154+ h->batch = vaddr;
159155
160156 vma = i915_vma_instance(h->obj, vm, NULL);
161157 if (IS_ERR(vma))
@@ -188,7 +184,7 @@ hang_create_request(struct hang *h, struct intel_engine_cs *engine) · 26 unmodified lines
188184 goto cancel_rq;
189185
190186 batch = h->batch;
191− if (INTEL_GEN(i915) >= 8) {
187+ if (INTEL_GEN(gt->i915) >= 8) {
192188 *batch++ = MI_STORE_DWORD_IMM_GEN4;
193189 *batch++ = lower_32_bits(hws_address(hws, rq));
194190 *batch++ = upper_32_bits(hws_address(hws, rq));
@@ -202,7 +198,7 @@ hang_create_request(struct hang *h, struct intel_engine_cs *engine) · 7 unmodified lines
202198 *batch++ = MI_BATCH_BUFFER_START | 1 << 8 | 1;
203199 *batch++ = lower_32_bits(vma->node.start);
204200 *batch++ = upper_32_bits(vma->node.start);
205− } else if (INTEL_GEN(i915) >= 6) {
201+ } else if (INTEL_GEN(gt->i915) >= 6) {
206202 *batch++ = MI_STORE_DWORD_IMM_GEN4;
207203 *batch++ = 0;
208204 *batch++ = lower_32_bits(hws_address(hws, rq));
@@ -215,7 +211,7 @@ hang_create_request(struct hang *h, struct intel_engine_cs *engine) · 6 unmodified lines
215211 *batch++ = MI_ARB_CHECK;
216212 *batch++ = MI_BATCH_BUFFER_START | 1 << 8;
217213 *batch++ = lower_32_bits(vma->node.start);
218− } else if (INTEL_GEN(i915) >= 4) {
214+ } else if (INTEL_GEN(gt->i915) >= 4) {
219215 *batch++ = MI_STORE_DWORD_IMM_GEN4 | MI_USE_GGTT;
220216 *batch++ = 0;
221217 *batch++ = lower_32_bits(hws_address(hws, rq));
@@ -242,7 +238,7 @@ hang_create_request(struct hang *h, struct intel_engine_cs *engine) · 20 unmodified lines
242238 *batch++ = lower_32_bits(vma->node.start);
243239 }
244240 *batch++ = MI_BATCH_BUFFER_END; /* not reached */
245− i915_gem_chipset_flush(h->i915);
241+ intel_gt_chipset_flush(engine->gt);
246242
247243 if (rq->engine->emit_init_breadcrumb) {
248244 err = rq->engine->emit_init_breadcrumb(rq);
@@ -251,7 +247,7 @@ hang_create_request(struct hang *h, struct intel_engine_cs *engine) · 2 unmodified lines
251247 }
252248
253249 flags = 0;
254− if (INTEL_GEN(vm->i915) <= 5)
250+ if (INTEL_GEN(gt->i915) <= 5)
255251 flags |= I915_DISPATCH_SECURE;
256252
257253 err = rq->engine->emit_bb_start(rq, vma->node.start, PAGE_SIZE, flags);
@@ -276,7 +272,7 @@ static u32 hws_seqno(const struct hang *h, const struct i915_request *rq) · 18 unmodified lines
276272 static void hang_fini(struct hang *h)
277273 {
278274 *h->batch = MI_BATCH_BUFFER_END;
279− i915_gem_chipset_flush(h->i915);
275+ intel_gt_chipset_flush(h->gt);
280276
281277 i915_gem_object_unpin_map(h->obj);
282278 i915_gem_object_put(h->obj);
@@ -286,7 +282,7 @@ static void hang_fini(struct hang *h) · 3 unmodified lines
286282
287283 kernel_context_close(h->ctx);
288284
289− igt_flush_test(h->i915, I915_WAIT_LOCKED);
285+ igt_flush_test(h->gt->i915, I915_WAIT_LOCKED);
290286 }
291287
292288 static bool wait_until_running(struct hang *h, struct i915_request *rq)
@@ -301,7 +297,7 @@ static bool wait_until_running(struct hang *h, struct i915_request *rq) · 8 unmodified lines
301297
302298 static int igt_hang_sanitycheck(void *arg)
303299 {
304− struct drm_i915_private *i915 = arg;
300+ struct intel_gt *gt = arg;
305301 struct i915_request *rq;
306302 struct intel_engine_cs *engine;
307303 enum intel_engine_id id;
@@ -310,13 +306,13 @@ static int igt_hang_sanitycheck(void *arg) · 2 unmodified lines
310306
311307 /* Basic check that we can execute our hanging batch */
312308
313− mutex_lock(&i915->drm.struct_mutex);
314− err = hang_init(&h, i915);
309+ mutex_lock(&gt->i915->drm.struct_mutex);
310+ err = hang_init(&h, gt);
315311 if (err)
316312 goto unlock;
317313
318− for_each_engine(engine, i915, id) {
319− struct igt_wedge_me w;
314+ for_each_engine(engine, gt->i915, id) {
315+ struct intel_wedge_me w;
320316 long timeout;
321317
322318 if (!intel_engine_can_store_dword(engine))
@@ -333,15 +329,15 @@ static int igt_hang_sanitycheck(void *arg) · 10 unmodified lines
333329 i915_request_get(rq);
334330
335331 *h.batch = MI_BATCH_BUFFER_END;
336− i915_gem_chipset_flush(i915);
332+ intel_gt_chipset_flush(engine->gt);
337333
338334 i915_request_add(rq);
339335
340336 timeout = 0;
341− igt_wedge_on_timeout(&w, i915, HZ / 10 /* 100ms timeout*/)
337+ intel_wedge_on_timeout(&w, gt, HZ / 10 /* 100ms */)
342338 timeout = i915_request_wait(rq, 0,
343339 MAX_SCHEDULE_TIMEOUT);
344− if (i915_reset_failed(i915))
340+ if (intel_gt_is_wedged(gt))
345341 timeout = -EIO;
346342
347343 i915_request_put(rq);
@@ -357,7 +353,7 @@ static int igt_hang_sanitycheck(void *arg) · 9 unmodified lines
357353 fini:
358354 hang_fini(&h);
359355 unlock:
360− mutex_unlock(&i915->drm.struct_mutex);
356+ mutex_unlock(&gt->i915->drm.struct_mutex);
361357 return err;
362358 }
363359
@@ -368,37 +364,37 @@ static bool wait_for_idle(struct intel_engine_cs *engine) · 4 unmodified lines
368364
369365 static int igt_reset_nop(void *arg)
370366 {
371− struct drm_i915_private *i915 = arg;
367+ struct intel_gt *gt = arg;
368+ struct i915_gpu_error *global = &gt->i915->gpu_error;
372369 struct intel_engine_cs *engine;
373370 struct i915_gem_context *ctx;
374371 unsigned int reset_count, count;
375372 enum intel_engine_id id;
376− intel_wakeref_t wakeref;
377373 struct drm_file *file;
378374 IGT_TIMEOUT(end_time);
379375 int err = 0;
380376
381377 /* Check that we can reset during non-user portions of requests */
382378
383− file = mock_file(i915);
379+ file = mock_file(gt->i915);
384380 if (IS_ERR(file))
385381 return PTR_ERR(file);
386382
387− mutex_lock(&i915->drm.struct_mutex);
388− ctx = live_context(i915, file);
389− mutex_unlock(&i915->drm.struct_mutex);
383+ mutex_lock(&gt->i915->drm.struct_mutex);
384+ ctx = live_context(gt->i915, file);
385+ mutex_unlock(&gt->i915->drm.struct_mutex);
390386 if (IS_ERR(ctx)) {
391387 err = PTR_ERR(ctx);
392388 goto out;
393389 }
394390
395391 i915_gem_context_clear_bannable(ctx);
396− wakeref = intel_runtime_pm_get(&i915->runtime_pm);
397− reset_count = i915_reset_count(&i915->gpu_error);
392+ reset_count = i915_reset_count(global);
398393 count = 0;
399394 do {
400− mutex_lock(&i915->drm.struct_mutex);
401− for_each_engine(engine, i915, id) {
395+ mutex_lock(&gt->i915->drm.struct_mutex);
396+
397+ for_each_engine(engine, gt->i915, id) {
402398 int i;
403399
404400 for (i = 0; i < 16; i++) {
@@ -413,82 +409,78 @@ static int igt_reset_nop(void *arg) · 8 unmodified lines
413409 i915_request_add(rq);
414410 }
415411 }
416− mutex_unlock(&i915->drm.struct_mutex);
417412
418− igt_global_reset_lock(i915);
419− i915_reset(i915, ALL_ENGINES, NULL);
420− igt_global_reset_unlock(i915);
421− if (i915_reset_failed(i915)) {
413+ igt_global_reset_lock(gt);
414+ intel_gt_reset(gt, ALL_ENGINES, NULL);
415+ igt_global_reset_unlock(gt);
416+
417+ mutex_unlock(&gt->i915->drm.struct_mutex);
418+ if (intel_gt_is_wedged(gt)) {
422419 err = -EIO;
423420 break;
424421 }
425422
426− if (i915_reset_count(&i915->gpu_error) !=
427− reset_count + ++count) {
423+ if (i915_reset_count(global) != reset_count + ++count) {
428424 pr_err("Full GPU reset not recorded!\n");
429425 err = -EINVAL;
430426 break;
431427 }
432428
433− err = igt_flush_test(i915, 0);
429+ err = igt_flush_test(gt->i915, 0);
434430 if (err)
435431 break;
436432 } while (time_before(jiffies, end_time));
437433 pr_info("%s: %d resets\n", __func__, count);
438434
439− mutex_lock(&i915->drm.struct_mutex);
440− err = igt_flush_test(i915, I915_WAIT_LOCKED);
441− mutex_unlock(&i915->drm.struct_mutex);
442−
443− intel_runtime_pm_put(&i915->runtime_pm, wakeref);
435+ mutex_lock(&gt->i915->drm.struct_mutex);
436+ err = igt_flush_test(gt->i915, I915_WAIT_LOCKED);
437+ mutex_unlock(&gt->i915->drm.struct_mutex);
444438
445439 out:
446− mock_file_free(i915, file);
447− if (i915_reset_failed(i915))
440+ mock_file_free(gt->i915, file);
441+ if (intel_gt_is_wedged(gt))
448442 err = -EIO;
449443 return err;
450444 }
451445
452446 static int igt_reset_nop_engine(void *arg)
453447 {
454− struct drm_i915_private *i915 = arg;
448+ struct intel_gt *gt = arg;
449+ struct i915_gpu_error *global = &gt->i915->gpu_error;
455450 struct intel_engine_cs *engine;
456451 struct i915_gem_context *ctx;
457452 enum intel_engine_id id;
458− intel_wakeref_t wakeref;
459453 struct drm_file *file;
460454 int err = 0;
461455
462456 /* Check that we can engine-reset during non-user portions */
463457
464− if (!intel_has_reset_engine(i915))
458+ if (!intel_has_reset_engine(gt->i915))
465459 return 0;
466460
467− file = mock_file(i915);
461+ file = mock_file(gt->i915);
468462 if (IS_ERR(file))
469463 return PTR_ERR(file);
470464
471− mutex_lock(&i915->drm.struct_mutex);
472− ctx = live_context(i915, file);
473− mutex_unlock(&i915->drm.struct_mutex);
465+ mutex_lock(&gt->i915->drm.struct_mutex);
466+ ctx = live_context(gt->i915, file);
467+ mutex_unlock(&gt->i915->drm.struct_mutex);
474468 if (IS_ERR(ctx)) {
475469 err = PTR_ERR(ctx);
476470 goto out;
477471 }
478472
479473 i915_gem_context_clear_bannable(ctx);
480− wakeref = intel_runtime_pm_get(&i915->runtime_pm);
481− for_each_engine(engine, i915, id) {
474+ for_each_engine(engine, gt->i915, id) {
482475 unsigned int reset_count, reset_engine_count;
483476 unsigned int count;
484477 IGT_TIMEOUT(end_time);
485478
486− reset_count = i915_reset_count(&i915->gpu_error);
487− reset_engine_count = i915_reset_engine_count(&i915->gpu_error,
488− engine);
479+ reset_count = i915_reset_count(global);
480+ reset_engine_count = i915_reset_engine_count(global, engine);
489481 count = 0;
490482
491− set_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
483+ set_bit(I915_RESET_ENGINE + id, &gt->reset.flags);
492484 do {
493485 int i;
494486
@@ -499,7 +491,7 @@ static int igt_reset_nop_engine(void *arg) · 4 unmodified lines
499491 break;
500492 }
501493
502− mutex_lock(&i915->drm.struct_mutex);
494+ mutex_lock(&gt->i915->drm.struct_mutex);
503495 for (i = 0; i < 16; i++) {
504496 struct i915_request *rq;
505497
@@ -511,21 +503,20 @@ static int igt_reset_nop_engine(void *arg) · 5 unmodified lines
511503
512504 i915_request_add(rq);
513505 }
514− mutex_unlock(&i915->drm.struct_mutex);
515−
516− err = i915_reset_engine(engine, NULL);
506+ err = intel_engine_reset(engine, NULL);
507+ mutex_unlock(&gt->i915->drm.struct_mutex);
517508 if (err) {
518509 pr_err("i915_reset_engine failed\n");
519510 break;
520511 }
521512
522− if (i915_reset_count(&i915->gpu_error) != reset_count) {
513+ if (i915_reset_count(global) != reset_count) {
523514 pr_err("Full GPU reset recorded! (engine reset expected)\n");
524515 err = -EINVAL;
525516 break;
526517 }
527518
528− if (i915_reset_engine_count(&i915->gpu_error, engine) !=
519+ if (i915_reset_engine_count(global, engine) !=
529520 reset_engine_count + ++count) {
530521 pr_err("%s engine reset not recorded!\n",
531522 engine->name);
@@ -533,31 +524,31 @@ static int igt_reset_nop_engine(void *arg) · 1 unmodified line
533524 break;
534525 }
535526 } while (time_before(jiffies, end_time));
536− clear_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
527+ clear_bit(I915_RESET_ENGINE + id, &gt->reset.flags);
537528 pr_info("%s(%s): %d resets\n", __func__, engine->name, count);
538529
539530 if (err)
540531 break;
541532
542− err = igt_flush_test(i915, 0);
533+ err = igt_flush_test(gt->i915, 0);
543534 if (err)
544535 break;
545536 }
546537
547− mutex_lock(&i915->drm.struct_mutex);
548− err = igt_flush_test(i915, I915_WAIT_LOCKED);
549− mutex_unlock(&i915->drm.struct_mutex);
538+ mutex_lock(&gt->i915->drm.struct_mutex);
539+ err = igt_flush_test(gt->i915, I915_WAIT_LOCKED);
540+ mutex_unlock(&gt->i915->drm.struct_mutex);
550541
551− intel_runtime_pm_put(&i915->runtime_pm, wakeref);
552542 out:
553− mock_file_free(i915, file);
554− if (i915_reset_failed(i915))
543+ mock_file_free(gt->i915, file);
544+ if (intel_gt_is_wedged(gt))
555545 err = -EIO;
556546 return err;
557547 }
558548
559−static int __igt_reset_engine(struct drm_i915_private *i915, bool active)
549+static int __igt_reset_engine(struct intel_gt *gt, bool active)
560550 {
551+ struct i915_gpu_error *global = &gt->i915->gpu_error;
561552 struct intel_engine_cs *engine;
562553 enum intel_engine_id id;
563554 struct hang h;
@@ -565,18 +556,18 @@ static int __igt_reset_engine(struct drm_i915_private *i915, bool active) · 1 unmodified line
565556
566557 /* Check that we can issue an engine reset on an idle engine (no-op) */
567558
568− if (!intel_has_reset_engine(i915))
559+ if (!intel_has_reset_engine(gt->i915))
569560 return 0;
570561
571562 if (active) {
572− mutex_lock(&i915->drm.struct_mutex);
573− err = hang_init(&h, i915);
574− mutex_unlock(&i915->drm.struct_mutex);
563+ mutex_lock(&gt->i915->drm.struct_mutex);
564+ err = hang_init(&h, gt);
565+ mutex_unlock(&gt->i915->drm.struct_mutex);
575566 if (err)
576567 return err;
577568 }
578569
579− for_each_engine(engine, i915, id) {
570+ for_each_engine(engine, gt->i915, id) {
580571 unsigned int reset_count, reset_engine_count;
581572 IGT_TIMEOUT(end_time);
582573
@@ -590,30 +581,29 @@ static int __igt_reset_engine(struct drm_i915_private *i915, bool active) · 7 unmodified lines
590581 break;
591582 }
592583
593− reset_count = i915_reset_count(&i915->gpu_error);
594− reset_engine_count = i915_reset_engine_count(&i915->gpu_error,
595− engine);
584+ reset_count = i915_reset_count(global);
585+ reset_engine_count = i915_reset_engine_count(global, engine);
596586
597587 intel_engine_pm_get(engine);
598− set_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
588+ set_bit(I915_RESET_ENGINE + id, &gt->reset.flags);
599589 do {
600590 if (active) {
601591 struct i915_request *rq;
602592
603− mutex_lock(&i915->drm.struct_mutex);
593+ mutex_lock(&gt->i915->drm.struct_mutex);
604594 rq = hang_create_request(&h, engine);
605595 if (IS_ERR(rq)) {
606596 err = PTR_ERR(rq);
607− mutex_unlock(&i915->drm.struct_mutex);
597+ mutex_unlock(&gt->i915->drm.struct_mutex);
608598 break;
609599 }
610600
611601 i915_request_get(rq);
612602 i915_request_add(rq);
613− mutex_unlock(&i915->drm.struct_mutex);
603+ mutex_unlock(&gt->i915->drm.struct_mutex);
614604
615605 if (!wait_until_running(&h, rq)) {
616− struct drm_printer p = drm_info_printer(i915->drm.dev);
606+ struct drm_printer p = drm_info_printer(gt->i915->drm.dev);
617607
618608 pr_err("%s: Failed to start request %llx, at %x\n",
619609 __func__, rq->fence.seqno, hws_seqno(&h, rq));
@@ -628,19 +618,19 @@ static int __igt_reset_engine(struct drm_i915_private *i915, bool active) · 8 unmodified lines
628618 i915_request_put(rq);
629619 }
630620
631− err = i915_reset_engine(engine, NULL);
621+ err = intel_engine_reset(engine, NULL);
632622 if (err) {
633623 pr_err("i915_reset_engine failed\n");
634624 break;
635625 }
636626
637− if (i915_reset_count(&i915->gpu_error) != reset_count) {
627+ if (i915_reset_count(global) != reset_count) {
638628 pr_err("Full GPU reset recorded! (engine reset expected)\n");
639629 err = -EINVAL;
640630 break;
641631 }
642632
643− if (i915_reset_engine_count(&i915->gpu_error, engine) !=
633+ if (i915_reset_engine_count(global, engine) !=
644634 ++reset_engine_count) {
645635 pr_err("%s engine reset not recorded!\n",
646636 engine->name);
@@ -648,24 +638,24 @@ static int __igt_reset_engine(struct drm_i915_private *i915, bool active) · 1 unmodified line
648638 break;
649639 }
650640 } while (time_before(jiffies, end_time));
651− clear_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
641+ clear_bit(I915_RESET_ENGINE + id, &gt->reset.flags);
652642 intel_engine_pm_put(engine);
653643
654644 if (err)
655645 break;
656646
657− err = igt_flush_test(i915, 0);
647+ err = igt_flush_test(gt->i915, 0);
658648 if (err)
659649 break;
660650 }
661651
662− if (i915_reset_failed(i915))
652+ if (intel_gt_is_wedged(gt))
663653 err = -EIO;
664654
665655 if (active) {
666− mutex_lock(&i915->drm.struct_mutex);
656+ mutex_lock(&gt->i915->drm.struct_mutex);
667657 hang_fini(&h);
668− mutex_unlock(&i915->drm.struct_mutex);
658+ mutex_unlock(&gt->i915->drm.struct_mutex);
669659 }
670660
671661 return err;
@@ -707,7 +697,7 @@ static int active_request_put(struct i915_request *rq) · 35 unmodified lines
707697 rq->fence.seqno);
708698 GEM_TRACE_DUMP();
709699
710− i915_gem_set_wedged(rq->i915);
700+ intel_gt_set_wedged(rq->engine->gt);
711701 err = -EIO;
712702 }
713703
@@ -784,10 +774,11 @@ err_file: · 70 unmodified lines
784774 return err;
785775 }
786776
787−static int __igt_reset_engines(struct drm_i915_private *i915,
777+static int __igt_reset_engines(struct intel_gt *gt,
788778 const char *test_name,
789779 unsigned int flags)
790780 {
781+ struct i915_gpu_error *global = &gt->i915->gpu_error;
791782 struct intel_engine_cs *engine, *other;
792783 enum intel_engine_id id, tmp;
793784 struct hang h;
@@ -797,13 +788,13 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 3 unmodified lines
797788 * with any other engine.
798789 */
799790
800− if (!intel_has_reset_engine(i915))
791+ if (!intel_has_reset_engine(gt->i915))
801792 return 0;
802793
803794 if (flags & TEST_ACTIVE) {
804− mutex_lock(&i915->drm.struct_mutex);
805− err = hang_init(&h, i915);
806− mutex_unlock(&i915->drm.struct_mutex);
795+ mutex_lock(&gt->i915->drm.struct_mutex);
796+ err = hang_init(&h, gt);
797+ mutex_unlock(&gt->i915->drm.struct_mutex);
807798 if (err)
808799 return err;
809800
@@ -811,9 +802,9 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 1 unmodified line
811802 h.ctx->sched.priority = 1024;
812803 }
813804
814− for_each_engine(engine, i915, id) {
805+ for_each_engine(engine, gt->i915, id) {
815806 struct active_engine threads[I915_NUM_ENGINES] = {};
816− unsigned long global = i915_reset_count(&i915->gpu_error);
807+ unsigned long device = i915_reset_count(global);
817808 unsigned long count = 0, reported;
818809 IGT_TIMEOUT(end_time);
819810
@@ -829,12 +820,11 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 9 unmodified lines
829820 }
830821
831822 memset(threads, 0, sizeof(threads));
832− for_each_engine(other, i915, tmp) {
823+ for_each_engine(other, gt->i915, tmp) {
833824 struct task_struct *tsk;
834825
835826 threads[tmp].resets =
836− i915_reset_engine_count(&i915->gpu_error,
837− other);
827+ i915_reset_engine_count(global, other);
838828
839829 if (!(flags & TEST_OTHERS))
840830 continue;
@@ -857,25 +847,25 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 16 unmodified lines
857847 }
858848
859849 intel_engine_pm_get(engine);
860− set_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
850+ set_bit(I915_RESET_ENGINE + id, &gt->reset.flags);
861851 do {
862852 struct i915_request *rq = NULL;
863853
864854 if (flags & TEST_ACTIVE) {
865− mutex_lock(&i915->drm.struct_mutex);
855+ mutex_lock(&gt->i915->drm.struct_mutex);
866856 rq = hang_create_request(&h, engine);
867857 if (IS_ERR(rq)) {
868858 err = PTR_ERR(rq);
869− mutex_unlock(&i915->drm.struct_mutex);
859+ mutex_unlock(&gt->i915->drm.struct_mutex);
870860 break;
871861 }
872862
873863 i915_request_get(rq);
874864 i915_request_add(rq);
875− mutex_unlock(&i915->drm.struct_mutex);
865+ mutex_unlock(&gt->i915->drm.struct_mutex);
876866
877867 if (!wait_until_running(&h, rq)) {
878− struct drm_printer p = drm_info_printer(i915->drm.dev);
868+ struct drm_printer p = drm_info_printer(gt->i915->drm.dev);
879869
880870 pr_err("%s: Failed to start request %llx, at %x\n",
881871 __func__, rq->fence.seqno, hws_seqno(&h, rq));
@@ -888,7 +878,7 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 6 unmodified lines
888878 }
889879 }
890880
891− err = i915_reset_engine(engine, NULL);
881+ err = intel_engine_reset(engine, NULL);
892882 if (err) {
893883 pr_err("i915_reset_engine(%s:%s): failed, err=%d\n",
894884 engine->name, test_name, err);
@@ -900,7 +890,7 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 5 unmodified lines
900890 if (rq) {
901891 if (i915_request_wait(rq, 0, HZ / 5) < 0) {
902892 struct drm_printer p =
903− drm_info_printer(i915->drm.dev);
893+ drm_info_printer(gt->i915->drm.dev);
904894
905895 pr_err("i915_reset_engine(%s:%s):"
906896 " failed to complete request after reset\n",
@@ -910,7 +900,7 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 3 unmodified lines
910900 i915_request_put(rq);
911901
912902 GEM_TRACE_DUMP();
913− i915_gem_set_wedged(i915);
903+ intel_gt_set_wedged(gt);
914904 err = -EIO;
915905 break;
916906 }
@@ -920,7 +910,7 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 3 unmodified lines
920910
921911 if (!(flags & TEST_SELF) && !wait_for_idle(engine)) {
922912 struct drm_printer p =
923− drm_info_printer(i915->drm.dev);
913+ drm_info_printer(gt->i915->drm.dev);
924914
925915 pr_err("i915_reset_engine(%s:%s):"
926916 " failed to idle after reset\n",
@@ -932,12 +922,12 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 5 unmodified lines
932922 break;
933923 }
934924 } while (time_before(jiffies, end_time));
935− clear_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
925+ clear_bit(I915_RESET_ENGINE + id, &gt->reset.flags);
936926 intel_engine_pm_put(engine);
937927 pr_info("i915_reset_engine(%s:%s): %lu resets\n",
938928 engine->name, test_name, count);
939929
940− reported = i915_reset_engine_count(&i915->gpu_error, engine);
930+ reported = i915_reset_engine_count(global, engine);
941931 reported -= threads[engine->id].resets;
942932 if (reported != count) {
943933 pr_err("i915_reset_engine(%s:%s): reset %lu times, but reported %lu\n",
@@ -947,7 +937,7 @@ static int __igt_reset_engines(struct drm_i915_private *i915, · 3 unmodified lines
947937 }
948938
949939 unwind:
950− for_each_engine(other, i915, tmp) {
940+ for_each_engine(other, gt->i915, tmp) {
951941 int ret;
952942
953943 if (!threads[tmp].task)
@@ -962,22 +952,21 @@ unwind: · 8 unmodified lines
962952 }
963953 put_task_struct(threads[tmp].task);
964954
965− if (other != engine &&
955+ if (other->uabi_class != engine->uabi_class &&
966956 threads[tmp].resets !=
967− i915_reset_engine_count(&i915->gpu_error, other)) {
957+ i915_reset_engine_count(global, other)) {
968958 pr_err("Innocent engine %s was reset (count=%ld)\n",
969959 other->name,
970− i915_reset_engine_count(&i915->gpu_error,
971− other) -
960+ i915_reset_engine_count(global, other) -
972961 threads[tmp].resets);
973962 if (!err)
974963 err = -EINVAL;
975964 }
976965 }
977966
978− if (global != i915_reset_count(&i915->gpu_error)) {
967+ if (device != i915_reset_count(global)) {
979968 pr_err("Global reset (count=%ld)!\n",
980− i915_reset_count(&i915->gpu_error) - global);
969+ i915_reset_count(global) - device);
981970 if (!err)
982971 err = -EINVAL;
983972 }
@@ -985,20 +974,20 @@ unwind: · 1 unmodified line
985974 if (err)
986975 break;
987976
988− mutex_lock(&i915->drm.struct_mutex);
989− err = igt_flush_test(i915, I915_WAIT_LOCKED);
990− mutex_unlock(&i915->drm.struct_mutex);
977+ mutex_lock(&gt->i915->drm.struct_mutex);
978+ err = igt_flush_test(gt->i915, I915_WAIT_LOCKED);
979+ mutex_unlock(&gt->i915->drm.struct_mutex);
991980 if (err)
992981 break;
993982 }
994983
995− if (i915_reset_failed(i915))
984+ if (intel_gt_is_wedged(gt))
996985 err = -EIO;
997986
998987 if (flags & TEST_ACTIVE) {
999− mutex_lock(&i915->drm.struct_mutex);
988+ mutex_lock(&gt->i915->drm.struct_mutex);
1000989 hang_fini(&h);
1001− mutex_unlock(&i915->drm.struct_mutex);
990+ mutex_unlock(&gt->i915->drm.struct_mutex);
1002991 }
1003992
1004993 return err;
@@ -1024,13 +1013,13 @@ static int igt_reset_engines(void *arg) · 19 unmodified lines
10241013 },
10251014 { }
10261015 };
1027− struct drm_i915_private *i915 = arg;
1016+ struct intel_gt *gt = arg;
10281017 typeof(*phases) *p;
10291018 int err;
10301019
10311020 for (p = phases; p->name; p++) {
10321021 if (p->flags & TEST_PRIORITY) {
1033− if (!(i915->caps.scheduler & I915_SCHEDULER_CAP_PRIORITY))
1022+ if (!(gt->i915->caps.scheduler & I915_SCHEDULER_CAP_PRIORITY))
10341023 continue;
10351024 }
10361025
@@ -1042,38 +1031,39 @@ static int igt_reset_engines(void *arg) · 5 unmodified lines
10421031 return 0;
10431032 }
10441033
1045−static u32 fake_hangcheck(struct drm_i915_private *i915,
1046− intel_engine_mask_t mask)
1034+static u32 fake_hangcheck(struct intel_gt *gt, intel_engine_mask_t mask)
10471035 {
1048− u32 count = i915_reset_count(&i915->gpu_error);
1036+ u32 count = i915_reset_count(&gt->i915->gpu_error);
10491037
1050− i915_reset(i915, mask, NULL);
1038+ intel_gt_reset(gt, mask, NULL);
10511039
10521040 return count;
10531041 }
10541042
10551043 static int igt_reset_wait(void *arg)
10561044 {
1057− struct drm_i915_private *i915 = arg;
1045+ struct intel_gt *gt = arg;
1046+ struct i915_gpu_error *global = &gt->i915->gpu_error;
1047+ struct intel_engine_cs *engine = gt->i915->engine[RCS0];
10581048 struct i915_request *rq;
10591049 unsigned int reset_count;
10601050 struct hang h;
10611051 long timeout;
10621052 int err;
10631053
1064− if (!intel_engine_can_store_dword(i915->engine[RCS0]))
1054+ if (!engine || !intel_engine_can_store_dword(engine))
10651055 return 0;
10661056
10671057 /* Check that we detect a stuck waiter and issue a reset */
10681058
1069− igt_global_reset_lock(i915);
1059+ igt_global_reset_lock(gt);
10701060
1071− mutex_lock(&i915->drm.struct_mutex);
1072− err = hang_init(&h, i915);
1061+ mutex_lock(&gt->i915->drm.struct_mutex);
1062+ err = hang_init(&h, gt);
10731063 if (err)
10741064 goto unlock;
10751065
1076− rq = hang_create_request(&h, i915->engine[RCS0]);
1066+ rq = hang_create_request(&h, engine);
10771067 if (IS_ERR(rq)) {
10781068 err = PTR_ERR(rq);
10791069 goto fini;
@@ -1083,19 +1073,19 @@ static int igt_reset_wait(void *arg) · 3 unmodified lines
10831073 i915_request_add(rq);
10841074
10851075 if (!wait_until_running(&h, rq)) {
1086− struct drm_printer p = drm_info_printer(i915->drm.dev);
1076+ struct drm_printer p = drm_info_printer(gt->i915->drm.dev);
10871077
10881078 pr_err("%s: Failed to start request %llx, at %x\n",
10891079 __func__, rq->fence.seqno, hws_seqno(&h, rq));
10901080 intel_engine_dump(rq->engine, &p, "%s\n", rq->engine->name);
10911081
1092− i915_gem_set_wedged(i915);
1082+ intel_gt_set_wedged(gt);
10931083
10941084 err = -EIO;
10951085 goto out_rq;
10961086 }
10971087
1098− reset_count = fake_hangcheck(i915, ALL_ENGINES);
1088+ reset_count = fake_hangcheck(gt, ALL_ENGINES);
10991089
11001090 timeout = i915_request_wait(rq, 0, 10);
11011091 if (timeout < 0) {
@@ -1105,7 +1095,7 @@ static int igt_reset_wait(void *arg) · 3 unmodified lines
11051095 goto out_rq;
11061096 }
11071097
1108− if (i915_reset_count(&i915->gpu_error) == reset_count) {
1098+ if (i915_reset_count(global) == reset_count) {
11091099 pr_err("No GPU reset recorded!\n");
11101100 err = -EINVAL;
11111101 goto out_rq;
@@ -1116,10 +1106,10 @@ out_rq: · 4 unmodified lines
11161106 fini:
11171107 hang_fini(&h);
11181108 unlock:
1119− mutex_unlock(&i915->drm.struct_mutex);
1120− igt_global_reset_unlock(i915);
1109+ mutex_unlock(&gt->i915->drm.struct_mutex);
1110+ igt_global_reset_unlock(gt);
11211111
1122− if (i915_reset_failed(i915))
1112+ if (intel_gt_is_wedged(gt))
11231113 return -EIO;
11241114
11251115 return err;
@@ -1178,11 +1168,12 @@ out_unlock: · 52 unmodified lines
11781168 return err;
11791169 }
11801170
1181−static int __igt_reset_evict_vma(struct drm_i915_private *i915,
1171+static int __igt_reset_evict_vma(struct intel_gt *gt,
11821172 struct i915_address_space *vm,
11831173 int (*fn)(void *),
11841174 unsigned int flags)
11851175 {
1176+ struct intel_engine_cs *engine = gt->i915->engine[RCS0];
11861177 struct drm_i915_gem_object *obj;
11871178 struct task_struct *tsk = NULL;
11881179 struct i915_request *rq;
@@ -1190,17 +1181,17 @@ static int __igt_reset_evict_vma(struct drm_i915_private *i915, · 1 unmodified line
11901181 struct hang h;
11911182 int err;
11921183
1193− if (!intel_engine_can_store_dword(i915->engine[RCS0]))
1184+ if (!engine || !intel_engine_can_store_dword(engine))
11941185 return 0;
11951186
11961187 /* Check that we can recover an unbind stuck on a hanging request */
11971188
1198− mutex_lock(&i915->drm.struct_mutex);
1199− err = hang_init(&h, i915);
1189+ mutex_lock(&gt->i915->drm.struct_mutex);
1190+ err = hang_init(&h, gt);
12001191 if (err)
12011192 goto unlock;
12021193
1203− obj = i915_gem_object_create_internal(i915, SZ_1M);
1194+ obj = i915_gem_object_create_internal(gt->i915, SZ_1M);
12041195 if (IS_ERR(obj)) {
12051196 err = PTR_ERR(obj);
12061197 goto fini;
@@ -1220,7 +1211,7 @@ static int __igt_reset_evict_vma(struct drm_i915_private *i915, · 13 unmodified lines
12201211 goto out_obj;
12211212 }
12221213
1223− rq = hang_create_request(&h, i915->engine[RCS0]);
1214+ rq = hang_create_request(&h, engine);
12241215 if (IS_ERR(rq)) {
12251216 err = PTR_ERR(rq);
12261217 goto out_obj;
@@ -1258,16 +1249,16 @@ static int __igt_reset_evict_vma(struct drm_i915_private *i915, · 31 unmodified lines
12581249 if (err)
12591250 goto out_rq;
12601251
1261− mutex_unlock(&i915->drm.struct_mutex);
1252+ mutex_unlock(&gt->i915->drm.struct_mutex);
12621253
12631254 if (!wait_until_running(&h, rq)) {
1264− struct drm_printer p = drm_info_printer(i915->drm.dev);
1255+ struct drm_printer p = drm_info_printer(gt->i915->drm.dev);
12651256
12661257 pr_err("%s: Failed to start request %llx, at %x\n",
12671258 __func__, rq->fence.seqno, hws_seqno(&h, rq));
12681259 intel_engine_dump(rq->engine, &p, "%s\n", rq->engine->name);
12691260
1270− i915_gem_set_wedged(i915);
1261+ intel_gt_set_wedged(gt);
12711262 goto out_reset;
12721263 }
12731264
@@ -1284,31 +1275,31 @@ static int __igt_reset_evict_vma(struct drm_i915_private *i915, · 10 unmodified lines
12841275 wait_for_completion(&arg.completion);
12851276
12861277 if (wait_for(!list_empty(&rq->fence.cb_list), 10)) {
1287− struct drm_printer p = drm_info_printer(i915->drm.dev);
1278+ struct drm_printer p = drm_info_printer(gt->i915->drm.dev);
12881279
12891280 pr_err("igt/evict_vma kthread did not wait\n");
12901281 intel_engine_dump(rq->engine, &p, "%s\n", rq->engine->name);
12911282
1292− i915_gem_set_wedged(i915);
1283+ intel_gt_set_wedged(gt);
12931284 goto out_reset;
12941285 }
12951286
12961287 out_reset:
1297− igt_global_reset_lock(i915);
1298− fake_hangcheck(rq->i915, rq->engine->mask);
1299− igt_global_reset_unlock(i915);
1288+ igt_global_reset_lock(gt);
1289+ fake_hangcheck(gt, rq->engine->mask);
1290+ igt_global_reset_unlock(gt);
13001291
13011292 if (tsk) {
1302− struct igt_wedge_me w;
1293+ struct intel_wedge_me w;
13031294
13041295 /* The reset, even indirectly, should take less than 10ms. */
1305− igt_wedge_on_timeout(&w, i915, HZ / 10 /* 100ms timeout*/)
1296+ intel_wedge_on_timeout(&w, gt, HZ / 10 /* 100ms */)
13061297 err = kthread_stop(tsk);
13071298
13081299 put_task_struct(tsk);
13091300 }
13101301
1311− mutex_lock(&i915->drm.struct_mutex);
1302+ mutex_lock(&gt->i915->drm.struct_mutex);
13121303 out_rq:
13131304 i915_request_put(rq);
13141305 out_obj:
@@ -1316,9 +1307,9 @@ out_obj: · 1 unmodified line
13161307 fini:
13171308 hang_fini(&h);
13181309 unlock:
1319− mutex_unlock(&i915->drm.struct_mutex);
1310+ mutex_unlock(&gt->i915->drm.struct_mutex);
13201311
1321− if (i915_reset_failed(i915))
1312+ if (intel_gt_is_wedged(gt))
13221313 return -EIO;
13231314
13241315 return err;
@@ -1326,26 +1317,26 @@ unlock: · 1 unmodified line
13261317
13271318 static int igt_reset_evict_ggtt(void *arg)
13281319 {
1329− struct drm_i915_private *i915 = arg;
1320+ struct intel_gt *gt = arg;
13301321
1331− return __igt_reset_evict_vma(i915, &i915->ggtt.vm,
1322+ return __igt_reset_evict_vma(gt, &gt->ggtt->vm,
13321323 evict_vma, EXEC_OBJECT_WRITE);
13331324 }
13341325
13351326 static int igt_reset_evict_ppgtt(void *arg)
13361327 {
1337− struct drm_i915_private *i915 = arg;
1328+ struct intel_gt *gt = arg;
13381329 struct i915_gem_context *ctx;
13391330 struct drm_file *file;
13401331 int err;
13411332
1342− file = mock_file(i915);
1333+ file = mock_file(gt->i915);
13431334 if (IS_ERR(file))
13441335 return PTR_ERR(file);
13451336
1346− mutex_lock(&i915->drm.struct_mutex);
1347− ctx = live_context(i915, file);
1348− mutex_unlock(&i915->drm.struct_mutex);
1337+ mutex_lock(&gt->i915->drm.struct_mutex);
1338+ ctx = live_context(gt->i915, file);
1339+ mutex_unlock(&gt->i915->drm.struct_mutex);
13491340 if (IS_ERR(ctx)) {
13501341 err = PTR_ERR(ctx);
13511342 goto out;
@@ -1353,29 +1344,29 @@ static int igt_reset_evict_ppgtt(void *arg) · 1 unmodified line
13531344
13541345 err = 0;
13551346 if (ctx->vm) /* aliasing == global gtt locking, covered above */
1356− err = __igt_reset_evict_vma(i915, ctx->vm,
1347+ err = __igt_reset_evict_vma(gt, ctx->vm,
13571348 evict_vma, EXEC_OBJECT_WRITE);
13581349
13591350 out:
1360− mock_file_free(i915, file);
1351+ mock_file_free(gt->i915, file);
13611352 return err;
13621353 }
13631354
13641355 static int igt_reset_evict_fence(void *arg)
13651356 {
1366− struct drm_i915_private *i915 = arg;
1357+ struct intel_gt *gt = arg;
13671358
1368− return __igt_reset_evict_vma(i915, &i915->ggtt.vm,
1359+ return __igt_reset_evict_vma(gt, &gt->ggtt->vm,
13691360 evict_fence, EXEC_OBJECT_NEEDS_FENCE);
13701361 }
13711362
1372−static int wait_for_others(struct drm_i915_private *i915,
1363+static int wait_for_others(struct intel_gt *gt,
13731364 struct intel_engine_cs *exclude)
13741365 {
13751366 struct intel_engine_cs *engine;
13761367 enum intel_engine_id id;
13771368
1378− for_each_engine(engine, i915, id) {
1369+ for_each_engine(engine, gt->i915, id) {
13791370 if (engine == exclude)
13801371 continue;
13811372
@@ -1388,7 +1379,8 @@ static int wait_for_others(struct drm_i915_private *i915, · 6 unmodified lines
13881379
13891380 static int igt_reset_queue(void *arg)
13901381 {
1391− struct drm_i915_private *i915 = arg;
1382+ struct intel_gt *gt = arg;
1383+ struct i915_gpu_error *global = &gt->i915->gpu_error;
13921384 struct intel_engine_cs *engine;
13931385 enum intel_engine_id id;
13941386 struct hang h;
@@ -1396,14 +1388,14 @@ static int igt_reset_queue(void *arg) · 1 unmodified line
13961388
13971389 /* Check that we replay pending requests following a hang */
13981390
1399− igt_global_reset_lock(i915);
1391+ igt_global_reset_lock(gt);
14001392
1401− mutex_lock(&i915->drm.struct_mutex);
1402− err = hang_init(&h, i915);
1393+ mutex_lock(&gt->i915->drm.struct_mutex);
1394+ err = hang_init(&h, gt);
14031395 if (err)
14041396 goto unlock;
14051397
1406− for_each_engine(engine, i915, id) {
1398+ for_each_engine(engine, gt->i915, id) {
14071399 struct i915_request *prev;
14081400 IGT_TIMEOUT(end_time);
14091401 unsigned int count;
@@ -1444,7 +1436,7 @@ static int igt_reset_queue(void *arg) · 34 unmodified lines
14441436 * (hangcheck), or we focus on resetting just one
14451437 * engine and so avoid repeatedly resetting innocents.
14461438 */
1447− err = wait_for_others(i915, engine);
1439+ err = wait_for_others(gt, engine);
14481440 if (err) {
14491441 pr_err("%s(%s): Failed to idle other inactive engines after device reset\n",
14501442 __func__, engine->name);
@@ -1452,12 +1444,12 @@ static int igt_reset_queue(void *arg) · 1 unmodified line
14521444 i915_request_put(prev);
14531445
14541446 GEM_TRACE_DUMP();
1455− i915_gem_set_wedged(i915);
1447+ intel_gt_set_wedged(gt);
14561448 goto fini;
14571449 }
14581450
14591451 if (!wait_until_running(&h, prev)) {
1460− struct drm_printer p = drm_info_printer(i915->drm.dev);
1452+ struct drm_printer p = drm_info_printer(gt->i915->drm.dev);
14611453
14621454 pr_err("%s(%s): Failed to start request %llx, at %x\n",
14631455 __func__, engine->name,
@@ -1468,13 +1460,13 @@ static int igt_reset_queue(void *arg) · 4 unmodified lines
14681460 i915_request_put(rq);
14691461 i915_request_put(prev);
14701462
1471− i915_gem_set_wedged(i915);
1463+ intel_gt_set_wedged(gt);
14721464
14731465 err = -EIO;
14741466 goto fini;
14751467 }
14761468
1477− reset_count = fake_hangcheck(i915, BIT(id));
1469+ reset_count = fake_hangcheck(gt, BIT(id));
14781470
14791471 if (prev->fence.error != -EIO) {
14801472 pr_err("GPU reset not recorded on hanging request [fence.error=%d]!\n",
@@ -1494,7 +1486,7 @@ static int igt_reset_queue(void *arg) · 13 unmodified lines
14941486 goto fini;
14951487 }
14961488
1497− if (i915_reset_count(&i915->gpu_error) == reset_count) {
1489+ if (i915_reset_count(global) == reset_count) {
14981490 pr_err("No GPU reset recorded!\n");
14991491 i915_request_put(rq);
15001492 i915_request_put(prev);
@@ -1509,11 +1501,11 @@ static int igt_reset_queue(void *arg) · 8 unmodified lines
15091501 pr_info("%s: Completed %d resets\n", engine->name, count);
15101502
15111503 *h.batch = MI_BATCH_BUFFER_END;
1512− i915_gem_chipset_flush(i915);
1504+ intel_gt_chipset_flush(engine->gt);
15131505
15141506 i915_request_put(prev);
15151507
1516− err = igt_flush_test(i915, I915_WAIT_LOCKED);
1508+ err = igt_flush_test(gt->i915, I915_WAIT_LOCKED);
15171509 if (err)
15181510 break;
15191511 }
@@ -1521,10 +1513,10 @@ static int igt_reset_queue(void *arg) · 1 unmodified line
15211513 fini:
15221514 hang_fini(&h);
15231515 unlock:
1524− mutex_unlock(&i915->drm.struct_mutex);
1525− igt_global_reset_unlock(i915);
1516+ mutex_unlock(&gt->i915->drm.struct_mutex);
1517+ igt_global_reset_unlock(gt);
15261518
1527− if (i915_reset_failed(i915))
1519+ if (intel_gt_is_wedged(gt))
15281520 return -EIO;
15291521
15301522 return err;
@@ -1532,8 +1524,9 @@ unlock: · 1 unmodified line
15321524
15331525 static int igt_handle_error(void *arg)
15341526 {
1535− struct drm_i915_private *i915 = arg;
1536− struct intel_engine_cs *engine = i915->engine[RCS0];
1527+ struct intel_gt *gt = arg;
1528+ struct i915_gpu_error *global = &gt->i915->gpu_error;
1529+ struct intel_engine_cs *engine = gt->i915->engine[RCS0];
15371530 struct hang h;
15381531 struct i915_request *rq;
15391532 struct i915_gpu_state *error;
@@ -1541,15 +1534,15 @@ static int igt_handle_error(void *arg) · 1 unmodified line
15411534
15421535 /* Check that we can issue a global GPU and engine reset */
15431536
1544− if (!intel_has_reset_engine(i915))
1537+ if (!intel_has_reset_engine(gt->i915))
15451538 return 0;
15461539
15471540 if (!engine || !intel_engine_can_store_dword(engine))
15481541 return 0;
15491542
1550− mutex_lock(&i915->drm.struct_mutex);
1543+ mutex_lock(&gt->i915->drm.struct_mutex);
15511544
1552− err = hang_init(&h, i915);
1545+ err = hang_init(&h, gt);
15531546 if (err)
15541547 goto err_unlock;
15551548
@@ -1563,28 +1556,28 @@ static int igt_handle_error(void *arg) · 7 unmodified lines
15631556 i915_request_add(rq);
15641557
15651558 if (!wait_until_running(&h, rq)) {
1566− struct drm_printer p = drm_info_printer(i915->drm.dev);
1559+ struct drm_printer p = drm_info_printer(gt->i915->drm.dev);
15671560
15681561 pr_err("%s: Failed to start request %llx, at %x\n",
15691562 __func__, rq->fence.seqno, hws_seqno(&h, rq));
15701563 intel_engine_dump(rq->engine, &p, "%s\n", rq->engine->name);
15711564
1572− i915_gem_set_wedged(i915);
1565+ intel_gt_set_wedged(gt);
15731566
15741567 err = -EIO;
15751568 goto err_request;
15761569 }
15771570
1578− mutex_unlock(&i915->drm.struct_mutex);
1571+ mutex_unlock(&gt->i915->drm.struct_mutex);
15791572
15801573 /* Temporarily disable error capture */
1581− error = xchg(&i915->gpu_error.first_error, (void *)-1);
1574+ error = xchg(&global->first_error, (void *)-1);
15821575
1583− i915_handle_error(i915, engine->mask, 0, NULL);
1576+ intel_gt_handle_error(gt, engine->mask, 0, NULL);
15841577
1585− xchg(&i915->gpu_error.first_error, error);
1578+ xchg(&global->first_error, error);
15861579
1587− mutex_lock(&i915->drm.struct_mutex);
1580+ mutex_lock(&gt->i915->drm.struct_mutex);
15881581
15891582 if (rq->fence.error != -EIO) {
15901583 pr_err("Guilty request not identified!\n");
@@ -1597,7 +1590,7 @@ err_request: · 6 unmodified lines
15971590 err_fini:
15981591 hang_fini(&h);
15991592 err_unlock:
1600− mutex_unlock(&i915->drm.struct_mutex);
1593+ mutex_unlock(&gt->i915->drm.struct_mutex);
16011594 return err;
16021595 }
16031596
@@ -1614,7 +1607,7 @@ static int __igt_atomic_reset_engine(struct intel_engine_cs *engine, · 10 unmodified lines
16141607 tasklet_disable_nosync(t);
16151608 p->critical_section_begin();
16161609
1617− err = i915_reset_engine(engine, NULL);
1610+ err = intel_engine_reset(engine, NULL);
16181611
16191612 p->critical_section_end();
16201613 tasklet_enable(t);
@@ -1629,7 +1622,6 @@ static int __igt_atomic_reset_engine(struct intel_engine_cs *engine, · 8 unmodified lines
16291622 static int igt_atomic_reset_engine(struct intel_engine_cs *engine,
16301623 const struct igt_atomic_section *p)
16311624 {
1632− struct drm_i915_private *i915 = engine->i915;
16331625 struct i915_request *rq;
16341626 struct hang h;
16351627 int err;
@@ -1638,7 +1630,7 @@ static int igt_atomic_reset_engine(struct intel_engine_cs *engine, · 2 unmodified lines
16381630 if (err)
16391631 return err;
16401632
1641− err = hang_init(&h, i915);
1633+ err = hang_init(&h, engine->gt);
16421634 if (err)
16431635 return err;
16441636
@@ -1657,16 +1649,16 @@ static int igt_atomic_reset_engine(struct intel_engine_cs *engine, · 12 unmodified lines
16571649 pr_err("%s(%s): Failed to start request %llx, at %x\n",
16581650 __func__, engine->name,
16591651 rq->fence.seqno, hws_seqno(&h, rq));
1660− i915_gem_set_wedged(i915);
1652+ intel_gt_set_wedged(engine->gt);
16611653 err = -EIO;
16621654 }
16631655
16641656 if (err == 0) {
1665− struct igt_wedge_me w;
1657+ struct intel_wedge_me w;
16661658
1667− igt_wedge_on_timeout(&w, i915, HZ / 20 /* 50ms timeout*/)
1659+ intel_wedge_on_timeout(&w, engine->gt, HZ / 20 /* 50ms */)
16681660 i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT);
1669− if (i915_reset_failed(i915))
1661+ if (intel_gt_is_wedged(engine->gt))
16701662 err = -EIO;
16711663 }
16721664
@@ -1678,30 +1670,30 @@ out: · 5 unmodified lines
16781670
16791671 static int igt_reset_engines_atomic(void *arg)
16801672 {
1681− struct drm_i915_private *i915 = arg;
1673+ struct intel_gt *gt = arg;
16821674 const typeof(*igt_atomic_phases) *p;
16831675 int err = 0;
16841676
16851677 /* Check that the engines resets are usable from atomic context */
16861678
1687− if (!intel_has_reset_engine(i915))
1679+ if (!intel_has_reset_engine(gt->i915))
16881680 return 0;
16891681
1690− if (USES_GUC_SUBMISSION(i915))
1682+ if (USES_GUC_SUBMISSION(gt->i915))
16911683 return 0;
16921684
1693− igt_global_reset_lock(i915);
1694− mutex_lock(&i915->drm.struct_mutex);
1685+ igt_global_reset_lock(gt);
1686+ mutex_lock(&gt->i915->drm.struct_mutex);
16951687
16961688 /* Flush any requests before we get started and check basics */
1697− if (!igt_force_reset(i915))
1689+ if (!igt_force_reset(gt))
16981690 goto unlock;
16991691
17001692 for (p = igt_atomic_phases; p->name; p++) {
17011693 struct intel_engine_cs *engine;
17021694 enum intel_engine_id id;
17031695
1704− for_each_engine(engine, i915, id) {
1696+ for_each_engine(engine, gt->i915, id) {
17051697 err = igt_atomic_reset_engine(engine, p);
17061698 if (err)
17071699 goto out;
@@ -1710,11 +1702,11 @@ static int igt_reset_engines_atomic(void *arg) · 2 unmodified lines
17101702
17111703 out:
17121704 /* As we poke around the guts, do a full reset before continuing. */
1713− igt_force_reset(i915);
1705+ igt_force_reset(gt);
17141706
17151707 unlock:
1716− mutex_unlock(&i915->drm.struct_mutex);
1717− igt_global_reset_unlock(i915);
1708+ mutex_unlock(&gt->i915->drm.struct_mutex);
1709+ igt_global_reset_unlock(gt);
17181710
17191711 return err;
17201712 }
@@ -1736,28 +1728,29 @@ int intel_hangcheck_live_selftests(struct drm_i915_private *i915) · 15 unmodified lines
17361728 SUBTEST(igt_reset_evict_fence),
17371729 SUBTEST(igt_handle_error),
17381730 };
1731+ struct intel_gt *gt = &i915->gt;
17391732 intel_wakeref_t wakeref;
17401733 bool saved_hangcheck;
17411734 int err;
17421735
1743− if (!intel_has_gpu_reset(i915))
1736+ if (!intel_has_gpu_reset(gt->i915))
17441737 return 0;
17451738
1746− if (i915_terminally_wedged(i915))
1739+ if (intel_gt_is_wedged(gt))
17471740 return -EIO; /* we're long past hope of a successful reset */
17481741
1749− wakeref = intel_runtime_pm_get(&i915->runtime_pm);
1742+ wakeref = intel_runtime_pm_get(&gt->i915->runtime_pm);
17501743 saved_hangcheck = fetch_and_zero(&i915_modparams.enable_hangcheck);
1751− drain_delayed_work(&i915->gpu_error.hangcheck_work); /* flush param */
1744+ drain_delayed_work(&gt->hangcheck.work); /* flush param */
17521745
1753− err = i915_subtests(tests, i915);
1746+ err = intel_gt_live_subtests(tests, gt);
17541747
1755− mutex_lock(&i915->drm.struct_mutex);
1756− igt_flush_test(i915, I915_WAIT_LOCKED);
1757− mutex_unlock(&i915->drm.struct_mutex);
1748+ mutex_lock(&gt->i915->drm.struct_mutex);
1749+ igt_flush_test(gt->i915, I915_WAIT_LOCKED);
1750+ mutex_unlock(&gt->i915->drm.struct_mutex);
17581751
17591752 i915_modparams.enable_hangcheck = saved_hangcheck;
1760− intel_runtime_pm_put(&i915->runtime_pm, wakeref);
1753+ intel_runtime_pm_put(&gt->i915->runtime_pm, wakeref);
17611754
17621755 return err;
17631756 }
drivers/gpu/drm/i915/gt/selftest_lrc.c+361−30
@@ -55,7 +55,7 @@ static int live_sanitycheck(void *arg) · 54 unmodified lines
5555 if (!igt_wait_for_spinner(&spin, rq)) {
5656 GEM_TRACE("spinner failed to start\n");
5757 GEM_TRACE_DUMP();
58− i915_gem_set_wedged(i915);
58+ intel_gt_set_wedged(&i915->gt);
5959 err = -EIO;
6060 goto err_ctx;
6161 }
@@ -73,12 +73,231 @@ err_ctx: · 11 unmodified lines
7373 err_spin:
7474 igt_spinner_fini(&spin);
7575 err_unlock:
76− igt_flush_test(i915, I915_WAIT_LOCKED);
7776 intel_runtime_pm_put(&i915->runtime_pm, wakeref);
7877 mutex_unlock(&i915->drm.struct_mutex);
7978 return err;
8079 }
8180
81+static int
82+emit_semaphore_chain(struct i915_request *rq, struct i915_vma *vma, int idx)
83+{
84+ u32 *cs;
85+
86+ cs = intel_ring_begin(rq, 10);
87+ if (IS_ERR(cs))
88+ return PTR_ERR(cs);
89+
90+ *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
91+
92+ *cs++ = MI_SEMAPHORE_WAIT |
93+ MI_SEMAPHORE_GLOBAL_GTT |
94+ MI_SEMAPHORE_POLL |
95+ MI_SEMAPHORE_SAD_NEQ_SDD;
96+ *cs++ = 0;
97+ *cs++ = i915_ggtt_offset(vma) + 4 * idx;
98+ *cs++ = 0;
99+
100+ if (idx > 0) {
101+ *cs++ = MI_STORE_DWORD_IMM_GEN4 | MI_USE_GGTT;
102+ *cs++ = i915_ggtt_offset(vma) + 4 * (idx - 1);
103+ *cs++ = 0;
104+ *cs++ = 1;
105+ } else {
106+ *cs++ = MI_NOOP;
107+ *cs++ = MI_NOOP;
108+ *cs++ = MI_NOOP;
109+ *cs++ = MI_NOOP;
110+ }
111+
112+ *cs++ = MI_ARB_ON_OFF | MI_ARB_DISABLE;
113+
114+ intel_ring_advance(rq, cs);
115+ return 0;
116+}
117+
118+static struct i915_request *
119+semaphore_queue(struct intel_engine_cs *engine, struct i915_vma *vma, int idx)
120+{
121+ struct i915_gem_context *ctx;
122+ struct i915_request *rq;
123+ int err;
124+
125+ ctx = kernel_context(engine->i915);
126+ if (!ctx)
127+ return ERR_PTR(-ENOMEM);
128+
129+ rq = igt_request_alloc(ctx, engine);
130+ if (IS_ERR(rq))
131+ goto out_ctx;
132+
133+ err = emit_semaphore_chain(rq, vma, idx);
134+ i915_request_add(rq);
135+ if (err)
136+ rq = ERR_PTR(err);
137+
138+out_ctx:
139+ kernel_context_close(ctx);
140+ return rq;
141+}
142+
143+static int
144+release_queue(struct intel_engine_cs *engine,
145+ struct i915_vma *vma,
146+ int idx)
147+{
148+ struct i915_sched_attr attr = {
149+ .priority = I915_USER_PRIORITY(I915_PRIORITY_MAX),
150+ };
151+ struct i915_request *rq;
152+ u32 *cs;
153+
154+ rq = i915_request_create(engine->kernel_context);
155+ if (IS_ERR(rq))
156+ return PTR_ERR(rq);
157+
158+ cs = intel_ring_begin(rq, 4);
159+ if (IS_ERR(cs)) {
160+ i915_request_add(rq);
161+ return PTR_ERR(cs);
162+ }
163+
164+ *cs++ = MI_STORE_DWORD_IMM_GEN4 | MI_USE_GGTT;
165+ *cs++ = i915_ggtt_offset(vma) + 4 * (idx - 1);
166+ *cs++ = 0;
167+ *cs++ = 1;
168+
169+ intel_ring_advance(rq, cs);
170+ i915_request_add(rq);
171+
172+ engine->schedule(rq, &attr);
173+
174+ return 0;
175+}
176+
177+static int
178+slice_semaphore_queue(struct intel_engine_cs *outer,
179+ struct i915_vma *vma,
180+ int count)
181+{
182+ struct intel_engine_cs *engine;
183+ struct i915_request *head;
184+ enum intel_engine_id id;
185+ int err, i, n = 0;
186+
187+ head = semaphore_queue(outer, vma, n++);
188+ if (IS_ERR(head))
189+ return PTR_ERR(head);
190+
191+ i915_request_get(head);
192+ for_each_engine(engine, outer->i915, id) {
193+ for (i = 0; i < count; i++) {
194+ struct i915_request *rq;
195+
196+ rq = semaphore_queue(engine, vma, n++);
197+ if (IS_ERR(rq)) {
198+ err = PTR_ERR(rq);
199+ goto out;
200+ }
201+ }
202+ }
203+
204+ err = release_queue(outer, vma, n);
205+ if (err)
206+ goto out;
207+
208+ if (i915_request_wait(head,
209+ I915_WAIT_LOCKED,
210+ 2 * RUNTIME_INFO(outer->i915)->num_engines * (count + 2) * (count + 3)) < 0) {
211+ pr_err("Failed to slice along semaphore chain of length (%d, %d)!\n",
212+ count, n);
213+ GEM_TRACE_DUMP();
214+ intel_gt_set_wedged(outer->gt);
215+ err = -EIO;
216+ }
217+
218+out:
219+ i915_request_put(head);
220+ return err;
221+}
222+
223+static int live_timeslice_preempt(void *arg)
224+{
225+ struct drm_i915_private *i915 = arg;
226+ struct drm_i915_gem_object *obj;
227+ intel_wakeref_t wakeref;
228+ struct i915_vma *vma;
229+ void *vaddr;
230+ int err = 0;
231+ int count;
232+
233+ /*
234+ * If a request takes too long, we would like to give other users
235+ * a fair go on the GPU. In particular, users may create batches
236+ * that wait upon external input, where that input may even be
237+ * supplied by another GPU job. To avoid blocking forever, we
238+ * need to preempt the current task and replace it with another
239+ * ready task.
240+ */
241+
242+ mutex_lock(&i915->drm.struct_mutex);
243+ wakeref = intel_runtime_pm_get(&i915->runtime_pm);
244+
245+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
246+ if (IS_ERR(obj)) {
247+ err = PTR_ERR(obj);
248+ goto err_unlock;
249+ }
250+
251+ vma = i915_vma_instance(obj, &i915->ggtt.vm, NULL);
252+ if (IS_ERR(vma)) {
253+ err = PTR_ERR(vma);
254+ goto err_obj;
255+ }
256+
257+ vaddr = i915_gem_object_pin_map(obj, I915_MAP_WC);
258+ if (IS_ERR(vaddr)) {
259+ err = PTR_ERR(vaddr);
260+ goto err_obj;
261+ }
262+
263+ err = i915_vma_pin(vma, 0, 0, PIN_GLOBAL);
264+ if (err)
265+ goto err_map;
266+
267+ for_each_prime_number_from(count, 1, 16) {
268+ struct intel_engine_cs *engine;
269+ enum intel_engine_id id;
270+
271+ for_each_engine(engine, i915, id) {
272+ if (!intel_engine_has_preemption(engine))
273+ continue;
274+
275+ memset(vaddr, 0, PAGE_SIZE);
276+
277+ err = slice_semaphore_queue(engine, vma, count);
278+ if (err)
279+ goto err_pin;
280+
281+ if (igt_flush_test(i915, I915_WAIT_LOCKED)) {
282+ err = -EIO;
283+ goto err_pin;
284+ }
285+ }
286+ }
287+
288+err_pin:
289+ i915_vma_unpin(vma);
290+err_map:
291+ i915_gem_object_unpin_map(obj);
292+err_obj:
293+ i915_gem_object_put(obj);
294+err_unlock:
295+ intel_runtime_pm_put(&i915->runtime_pm, wakeref);
296+ mutex_unlock(&i915->drm.struct_mutex);
297+
298+ return err;
299+}
300+
82301 static int live_busywait_preempt(void *arg)
83302 {
84303 struct drm_i915_private *i915 = arg;
@@ -138,6 +357,9 @@ static int live_busywait_preempt(void *arg) · 53 unmodified lines
138357 struct igt_live_test t;
139358 u32 *cs;
140359
360+ if (!intel_engine_has_preemption(engine))
361+ continue;
362+
141363 if (!intel_engine_can_store_dword(engine))
142364 continue;
143365
@@ -229,7 +451,7 @@ static int live_busywait_preempt(void *arg) · 85 unmodified lines
229451 intel_engine_dump(engine, &p, "%s\n", engine->name);
230452 GEM_TRACE_DUMP();
231453
232− i915_gem_set_wedged(i915);
454+ intel_gt_set_wedged(&i915->gt);
233455 err = -EIO;
234456 goto err_vma;
235457 }
@@ -253,8 +475,6 @@ err_ctx_lo: · 17 unmodified lines
253475 err_ctx_hi:
254476 kernel_context_close(ctx_hi);
255477 err_unlock:
256− if (igt_flush_test(i915, I915_WAIT_LOCKED))
257− err = -EIO;
258478 intel_runtime_pm_put(&i915->runtime_pm, wakeref);
259479 mutex_unlock(&i915->drm.struct_mutex);
260480 return err;
@@ -320,7 +540,7 @@ static int live_preempt(void *arg) · 59 unmodified lines
320540 if (!igt_wait_for_spinner(&spin_lo, rq)) {
321541 GEM_TRACE("lo spinner failed to start\n");
322542 GEM_TRACE_DUMP();
323− i915_gem_set_wedged(i915);
543+ intel_gt_set_wedged(&i915->gt);
324544 err = -EIO;
325545 goto err_ctx_lo;
326546 }
@@ -337,7 +557,7 @@ static int live_preempt(void *arg) · 10 unmodified lines
337557 if (!igt_wait_for_spinner(&spin_hi, rq)) {
338558 GEM_TRACE("hi spinner failed to start\n");
339559 GEM_TRACE_DUMP();
340− i915_gem_set_wedged(i915);
560+ intel_gt_set_wedged(&i915->gt);
341561 err = -EIO;
342562 goto err_ctx_lo;
343563 }
@@ -361,7 +581,6 @@ err_spin_lo: · 17 unmodified lines
361581 err_spin_hi:
362582 igt_spinner_fini(&spin_hi);
363583 err_unlock:
364− igt_flush_test(i915, I915_WAIT_LOCKED);
365584 intel_runtime_pm_put(&i915->runtime_pm, wakeref);
366585 mutex_unlock(&i915->drm.struct_mutex);
367586 return err;

302 more lines aren’t shown here.

drivers/gpu/drm/i915/gt/selftest_reset.c+96−37

This diff is large: its 207 lines aren’t shown here.

drivers/gpu/drm/i915/gt/selftest_timeline.crenamed from drivers/gpu/drm/i915/selftests/i915_timeline.c+65−66

This diff is large: its 470 lines aren’t shown here.

drivers/gpu/drm/i915/gt/selftest_workarounds.c+89−66

This diff is large: its 445 lines aren’t shown here.

drivers/gpu/drm/i915/gt/selftests/mock_timeline.crenamed from drivers/gpu/drm/i915/selftests/mock_timeline.c+4−4

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/i915/gt/selftests/mock_timeline.hrenamed from drivers/gpu/drm/i915/selftests/mock_timeline.h+3−3

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/Makefile+5−0

This diff is large: its 6 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc.crenamed from drivers/gpu/drm/i915/intel_guc.c+118−172

This diff is large: its 444 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc.hrenamed from drivers/gpu/drm/i915/intel_guc.h+17−16

This diff is large: its 91 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_ads.crenamed from drivers/gpu/drm/i915/intel_guc_ads.c+15−14

This diff is large: its 78 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_ads.hrenamed from drivers/gpu/drm/i915/intel_guc_ads.h+0−0

File renamed without changes.

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.crenamed from drivers/gpu/drm/i915/intel_guc_ct.c+4−18

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_ct.hrenamed from drivers/gpu/drm/i915/intel_guc_ct.h+9−2

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c+181−0

This diff is large: its 182 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_fw.hrenamed from drivers/gpu/drm/i915/intel_guc_fw.h+0−0

File renamed without changes.

drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.hrenamed from drivers/gpu/drm/i915/intel_guc_fwif.h+5−76

This diff is large: its 103 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_log.crenamed from drivers/gpu/drm/i915/intel_guc_log.c+38−10

This diff is large: its 125 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_log.hrenamed from drivers/gpu/drm/i915/intel_guc_log.h+0−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_reg.hrenamed from drivers/gpu/drm/i915/intel_guc_reg.h+22−16

This diff is large: its 58 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_submission.crenamed from drivers/gpu/drm/i915/intel_guc_submission.c+103−393

This diff is large: its 814 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_guc_submission.hrenamed from drivers/gpu/drm/i915/intel_guc_submission.h+0−2

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_huc.crenamed from drivers/gpu/drm/i915/intel_huc.c+41−28

This diff is large: its 157 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_huc.hrenamed from drivers/gpu/drm/i915/intel_huc.h+5−7

This diff is large: its 35 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_huc_fw.c+53−0

This diff is large: its 54 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_huc_fw.hrenamed from drivers/gpu/drm/i915/intel_huc_fw.h+0−0

File renamed without changes.

drivers/gpu/drm/i915/gt/uc/intel_uc.c+570−0

This diff is large: its 571 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_uc.hrenamed from drivers/gpu/drm/i915/intel_uc.h+22−17

This diff is large: its 57 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c+540−0

This diff is large: its 541 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_uc_fw.hrenamed from drivers/gpu/drm/i915/intel_uc_fw.h+66−36

This diff is large: its 180 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/intel_uc_fw_abi.h+82−0

This diff is large: its 83 lines aren’t shown here.

drivers/gpu/drm/i915/gt/uc/selftest_guc.crenamed from drivers/gpu/drm/i915/selftests/intel_guc.c+13−36

This diff is large: its 117 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/gtt.h+7−6

This diff is large: its 25 lines aren’t shown here.

drivers/gpu/drm/i915/gvt/scheduler.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_active.c+215−106

This diff is large: its 492 lines aren’t shown here.

drivers/gpu/drm/i915/i915_active.h+16−12

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/i915/i915_active_types.h+11−2

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/i915/i915_debugfs.c+107−110

This diff is large: its 471 lines aren’t shown here.

drivers/gpu/drm/i915/i915_drv.c+105−95

This diff is large: its 608 lines aren’t shown here.

drivers/gpu/drm/i915/i915_drv.h+66−179

This diff is large: its 507 lines aren’t shown here.

drivers/gpu/drm/i915/i915_fixed.h+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gem.c+119−199

This diff is large: its 637 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gem_batch_pool.c+17−25

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gem_fence_reg.c+32−0

This diff is large: its 36 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gem_fence_reg.h+3−0

This diff is large: its 15 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gem_gtt.c+844−1209

This diff is large: its 3004 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gem_gtt.h+53−147

This diff is large: its 323 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gem_render_state.h+0−31

This diff is large: its 32 lines aren’t shown here.

drivers/gpu/drm/i915/i915_globals.h+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gpu_error.c+262−326

This diff is large: its 959 lines aren’t shown here.

drivers/gpu/drm/i915/i915_gpu_error.h+4−67

This diff is large: its 115 lines aren’t shown here.

drivers/gpu/drm/i915/i915_irq.c+411−392

This diff is large: its 1686 lines aren’t shown here.

drivers/gpu/drm/i915/i915_irq.h+35−10

This diff is large: its 71 lines aren’t shown here.

drivers/gpu/drm/i915/i915_params.c+3−2

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/i915_params.h+1−1

This diff is large: its 16 lines aren’t shown here.

drivers/gpu/drm/i915/i915_pci.c+37−6

This diff is large: its 91 lines aren’t shown here.

drivers/gpu/drm/i915/i915_perf.c+220−84

This diff is large: its 357 lines aren’t shown here.

drivers/gpu/drm/i915/i915_pmu.c+1−3

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/i915_priolist_types.h+10−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/i915_pvinfo.h+5−2

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/i915/i915_reg.h+158−73

This diff is large: its 614 lines aren’t shown here.

drivers/gpu/drm/i915/i915_request.c+69−57

This diff is large: its 235 lines aren’t shown here.

drivers/gpu/drm/i915/i915_request.h+19−5

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/i915/i915_scheduler.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/i915/i915_scheduler_types.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/i915_selftest.h+26−1

This diff is large: its 39 lines aren’t shown here.

drivers/gpu/drm/i915/i915_timeline.h+0−94

This diff is large: its 95 lines aren’t shown here.

drivers/gpu/drm/i915/i915_trace.h+6−6

This diff is large: its 52 lines aren’t shown here.

drivers/gpu/drm/i915/i915_utils.h+12−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/i915_vgpu.c+43−20

This diff is large: its 132 lines aren’t shown here.

drivers/gpu/drm/i915/i915_vgpu.h+4−3

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/i915/i915_vma.c+28−45

This diff is large: its 147 lines aren’t shown here.

drivers/gpu/drm/i915/intel_device_info.c+17−23

This diff is large: its 60 lines aren’t shown here.

drivers/gpu/drm/i915/intel_device_info.h+4−1

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/i915/intel_drv.h+32−5

This diff is large: its 85 lines aren’t shown here.

drivers/gpu/drm/i915/intel_guc_fw.c+0−308

This diff is large: its 309 lines aren’t shown here.

drivers/gpu/drm/i915/intel_gvt.c+4−3

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/i915/intel_gvt.h+4−3

This diff is large: its 24 lines aren’t shown here.

drivers/gpu/drm/i915/intel_huc_fw.c+0−215

This diff is large: its 216 lines aren’t shown here.

drivers/gpu/drm/i915/intel_pm.c+214−227

This diff is large: its 1164 lines aren’t shown here.

drivers/gpu/drm/i915/intel_pm.h+2−2

This diff is large: its 21 lines aren’t shown here.

drivers/gpu/drm/i915/intel_runtime_pm.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_runtime_pm.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_uc.c+0−561

This diff is large: its 562 lines aren’t shown here.

drivers/gpu/drm/i915/intel_uc_fw.c+0−357

This diff is large: its 358 lines aren’t shown here.

drivers/gpu/drm/i915/intel_uncore.c+261−204

This diff is large: its 776 lines aren’t shown here.

drivers/gpu/drm/i915/intel_uncore.h+13−11

This diff is large: its 73 lines aren’t shown here.

drivers/gpu/drm/i915/intel_wakeref.c+5−4

This diff is large: its 42 lines aren’t shown here.

drivers/gpu/drm/i915/intel_wakeref.h+9−2

This diff is large: its 37 lines aren’t shown here.

drivers/gpu/drm/i915/intel_wopcm.c+22−18

This diff is large: its 111 lines aren’t shown here.

drivers/gpu/drm/i915/intel_wopcm.h+3−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/oa/Makefile+0−0

Empty file added.

drivers/gpu/drm/i915/oa/i915_oa_bdw.crenamed from drivers/gpu/drm/i915/i915_oa_bdw.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_bdw.hrenamed from drivers/gpu/drm/i915/i915_oa_bdw.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_bxt.crenamed from drivers/gpu/drm/i915/i915_oa_bxt.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_bxt.hrenamed from drivers/gpu/drm/i915/i915_oa_bxt.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_cflgt2.crenamed from drivers/gpu/drm/i915/i915_oa_cflgt2.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_cflgt2.hrenamed from drivers/gpu/drm/i915/i915_oa_cflgt2.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_cflgt3.crenamed from drivers/gpu/drm/i915/i915_oa_cflgt3.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_cflgt3.hrenamed from drivers/gpu/drm/i915/i915_oa_cflgt3.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_chv.crenamed from drivers/gpu/drm/i915/i915_oa_chv.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_chv.hrenamed from drivers/gpu/drm/i915/i915_oa_chv.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_cnl.crenamed from drivers/gpu/drm/i915/i915_oa_cnl.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_cnl.hrenamed from drivers/gpu/drm/i915/i915_oa_cnl.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_glk.crenamed from drivers/gpu/drm/i915/i915_oa_glk.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_glk.hrenamed from drivers/gpu/drm/i915/i915_oa_glk.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_hsw.crenamed from drivers/gpu/drm/i915/i915_oa_hsw.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_hsw.hrenamed from drivers/gpu/drm/i915/i915_oa_hsw.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_icl.crenamed from drivers/gpu/drm/i915/i915_oa_icl.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_icl.hrenamed from drivers/gpu/drm/i915/i915_oa_icl.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_kblgt2.crenamed from drivers/gpu/drm/i915/i915_oa_kblgt2.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_kblgt2.hrenamed from drivers/gpu/drm/i915/i915_oa_kblgt2.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_kblgt3.crenamed from drivers/gpu/drm/i915/i915_oa_kblgt3.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_kblgt3.hrenamed from drivers/gpu/drm/i915/i915_oa_kblgt3.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_sklgt2.crenamed from drivers/gpu/drm/i915/i915_oa_sklgt2.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_sklgt2.hrenamed from drivers/gpu/drm/i915/i915_oa_sklgt2.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_sklgt3.crenamed from drivers/gpu/drm/i915/i915_oa_sklgt3.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_sklgt3.hrenamed from drivers/gpu/drm/i915/i915_oa_sklgt3.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/oa/i915_oa_sklgt4.crenamed from drivers/gpu/drm/i915/i915_oa_sklgt4.c+0−0

File renamed without changes.

drivers/gpu/drm/i915/oa/i915_oa_sklgt4.hrenamed from drivers/gpu/drm/i915/i915_oa_sklgt4.h+1−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_active.c+97−27

This diff is large: its 226 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_gem.c+4−7

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_gem_evict.c+2−1

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_gem_gtt.c+1−3

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_live_selftests.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_mock_selftests.h+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_request.c+25−18

This diff is large: its 143 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_selftest.c+64−1

This diff is large: its 90 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_vma.c+10−0

This diff is large: its 45 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/igt_flush_test.c+3−2

This diff is large: its 26 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/igt_reset.c+18−20

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/igt_reset.h+6−4

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/igt_spinner.c+6−2

This diff is large: its 43 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/igt_spinner.h+3−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/igt_wedge_me.h+0−58

This diff is large: its 59 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/mock_gem_device.c+5−13

This diff is large: its 55 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/mock_gtt.c+3−0

This diff is large: its 17 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/mock_uncore.c+2−2

This diff is large: its 9 lines aren’t shown here.

include/drm/i915_component.h+1−1

This diff is large: its 9 lines aren’t shown here.

include/drm/i915_drm.h+8−5

This diff is large: its 27 lines aren’t shown here.

include/drm/i915_pciids.h+12−1

This diff is large: its 25 lines aren’t shown here.

include/uapi/drm/i915_drm.h+1−0

This diff is large: its 8 lines aren’t shown here.