History

Merge tag 'drm-intel-next-2020-05-15' of git://anongit.freedesktop.org/drm/drm-intel into drm-next

UAPI Changes: - drm/i915: Show per-engine default property values in sysfs By providing the default values configured into the kernel via sysfs, it is much more convenient for userspace to restore those sane defaults, or at least know what are considered good baseline. This is useful, for example, to cleanup after any failed userspace prior to commencing new jobs. Cross-subsystem Changes: - video/hdmi: Add Unpack only function for DRM infoframe - Includes pull request gvt-next-2020-05-12 Driver Changes: - Restore Cherryview back to full-ppgtt (Chris, Mika) - Document locking guidelines for i915 (Chris, Daniel, Joonas) - Fix GitLab #1746: Handle idling during i915_gem_evict_something busy loops (Chris) - Display WA #1105: Require linear fb stride to be multiple of 512 bytes on gen9/glk (Ville) - Add Wa_14010685332 for ICP/ICL (Matt R) - Restrict w/a 1607087056 for EHL/JSL (Swathi) - Fix interrupt handling for DP AUX transactions on Tigerlake (Imre) - Revert "drm/i915/tgl: Include ro parts of l3 to invalidate" (Mika) - Fix HDC pipeline flush hardware bit on Gen12 (Mika) - Flush L3 when flushing render on Gen12 (Mika) - Invalidate aux table entries forcibly between BB on Gen12 (Mika) - Add aux table invalidate for all engines on Gen12 (Mika) - Force pte cacheline to main memory Gen8+ (Mika) - Add and enable TGL+ SAGV support (Stanislav) - Implement vm_ops->access on i915 mmaps for GDB (Chris, Kristian) - Replace zero-length array with flexible-array (Gustavo) - Improve batch buffer pool effectiveness to mitigate soft-rc6 hit (Chris) - Remove wait priority boosting (Chris) - Keep driver module referenced when PMU is active (Chris) - Sanitize RPS interrupts upon resume (Chris) - Extend pcode read timeout to 20 ms (Chris) - Wait for ACT sent before enabling MST pipe (Ville) - Extend support to async relocations to SNB (Chris) - Remove CNL pre-prod workarounds (Ville) - Don't enable WaIncreaseLatencyIPCEnabled when IPC is disabled (Sultan) - Record the active CCID from before reset (Chris) - Mark concurrent submissions with a weak-dependency (Chris) - Peel dma-fence-chains for await to allow engine-to-engine sync (Lionel) - Prevent using semaphores to chain up to external fences (Chris) - Fix GLK watermark calculations (Ville) - Emit await(batch) before MI_BB_START (Chris) - Reset execlists registers before HWSP (Chris) - Drop no-semaphore boosting in favor of fast timeslicing (Chris) - Fix enabled infoframe states of lspcon (Gwan-gyeong) - Program DP SDPs on pipe updates (Gwan-gyeong) - Stop sending DP SDPs on ddi disable (Gwan-gyeong) - Store CS timestamp frequency in Hz (Ville) - Remove unused HAS_FWTABLE macro (Pascal) - Use batchbuffer chaining for relocations to save ring space (Chris) - Try different engines for relocs if MI ops not supported (Chris, Tvrtko) - Lazily acquire the device wakeref for freeing objects (Chris) - Streamline display code arithmetics around rounding etc. (Ville) - Use bw state for per crtc SAGV evaluation (Stanislav) - Track active_pipes in bw_state (Stanislav) - Nuke mode.vrefresh usage (Ville) - Warn if the FBC is still writing to stolen on removal (Chris) - Added new PCode commands prepping for QGV rescricting (Stansilav) - Stop holding onto the pinned_default_state (Chris) - Propagate error from completed fences (Chris) - Ignore submit-fences on the same timeline (Chris) - Pull waiting on an external dma-fence into its routine (Chris) - Replace the hardcoded I915_FENCE_TIMEOUT with Kconfig (Chris) - Mark up the racy read of execlists->context_tag (Chris) - Tidy up the return handling for completed dma-fences (Chris) - Introduce skl_plane_wm_level accessor (Stanislav) - Extract SKL SAGV checking (Stanislav) - Make active_pipes check skl specific (Stanislav) - Suspend tasklets before resume sanitization (Chris) - Remove redundant exec_fence (Chris) - Mark the addition of the initial-breadcrumb in the request (Chris) - Transfer old virtual breadcrumbs to irq_worker (Chris) - Read the DP SDPs from the video DIP (Gwan-gyeong) - Program DP SDPs with computed configs (Gwan-gyeong) - Add state readout for DP VSC and DP HDR Metadata Infoframe SDP (Gwan-gyeong) - Add compute routine for DP PSR VSC SDP (Gwan-gyeong) - Use new DP VSC SDP compute routine on PSR (Gwan-gyeong) - Restrict qgv points which don't have enough bandwidth. (Stanislav) - Nuke pointless div by 64bit (Ville) - Static checker code fixes (Nathan, Mika, Chris) - Add logging function for DP VSC SDP (Gwan-gyeong) - Include HDMI DRM infoframe, DP HDR metadata and DP VSC SDP in the crtc state dump (Gwan-gyeong) - Make timeslicing explicit engine property (Chris, Tvrtko) - Selftest and debugging improvements (Chris) - Align variable names with BSpec (Ville) - Tidy up gen8+ breadcrumb emission code (Chris) - Turn intel_digital_port_connected() in a vfunc (Ville) - Use stashed away hpd isr bits in intel_digital_port_connected() (Ville) - Extract i915_cs_timestamp_{ns_to_ticks,tick_to_ns}() (Ville)

Dave Airlie committed 6 years ago

Commit 6cf99162 parents bfbe174 3a36aa2105 files · +3097 −1505Browse files

Unified · Split

105 files
  1. Documentation/gpu/i915.rst +46 −0
  2. drivers/gpu/drm/drm_dp_helper.c +174 −0
  3. drivers/gpu/drm/i915/Kconfig.profile +12 −0
  4. drivers/gpu/drm/i915/Makefile +2 −1
  5. drivers/gpu/drm/i915/display/intel_audio.c +29 −48
  6. drivers/gpu/drm/i915/display/intel_bw.c +114 −47
  7. drivers/gpu/drm/i915/display/intel_bw.h +18 −0
  8. drivers/gpu/drm/i915/display/intel_ddi.c +56 −4
  9. drivers/gpu/drm/i915/display/intel_display.c +76 −9
  10. drivers/gpu/drm/i915/display/intel_display_types.h +3 −0
  11. drivers/gpu/drm/i915/display/intel_dp.c +230 −364
  12. drivers/gpu/drm/i915/display/intel_dp.h +10 −5
  13. drivers/gpu/drm/i915/display/intel_dp_mst.c +5 −4
  14. drivers/gpu/drm/i915/display/intel_fbc.c +10 −2
  15. drivers/gpu/drm/i915/display/intel_lspcon.c +1 −1
  16. drivers/gpu/drm/i915/display/intel_psr.c +19 −39
  17. drivers/gpu/drm/i915/display/intel_psr.h +4 −2
  18. drivers/gpu/drm/i915/display/intel_tc.c +4 −3
  19. drivers/gpu/drm/i915/display/intel_tc.h +2 −1
  20. drivers/gpu/drm/i915/display/intel_vbt_defs.h +2 −2
  21. drivers/gpu/drm/i915/gem/i915_gem_clflush.c +1 −1
  22. drivers/gpu/drm/i915/gem/i915_gem_client_blt.c +1 −3
  23. drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +313 −165
  24. drivers/gpu/drm/i915/gem/i915_gem_fence.c +2 −2
  25. drivers/gpu/drm/i915/gem/i915_gem_mman.c +34 −0
  26. drivers/gpu/drm/i915/gem/i915_gem_object.c +0 −3
  27. drivers/gpu/drm/i915/gem/i915_gem_object_blt.c +15 −11
  28. drivers/gpu/drm/i915/gem/i915_gem_object_blt.h +0 −1
  29. drivers/gpu/drm/i915/gem/i915_gem_pages.c +1 −0
  30. drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c +1 −30
  31. drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c +2 −0
  32. drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +32 −17
  33. drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c +171 −0
  34. drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +124 −0
  35. drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c +10 −16
  36. drivers/gpu/drm/i915/gt/gen8_ppgtt.c +13 −2
  37. drivers/gpu/drm/i915/gt/intel_breadcrumbs.c +42 −10
  38. drivers/gpu/drm/i915/gt/intel_engine.h +31 −15
  39. drivers/gpu/drm/i915/gt/intel_engine_cs.c +2 −4
  40. drivers/gpu/drm/i915/gt/intel_engine_pm.c +1 −15
  41. drivers/gpu/drm/i915/gt/intel_engine_pool.h +0 −34
  42. drivers/gpu/drm/i915/gt/intel_engine_types.h +22 −14
  43. drivers/gpu/drm/i915/gt/intel_gpu_commands.h +1 −2
  44. drivers/gpu/drm/i915/gt/intel_gt.c +3 −0
  45. drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.c +76 −38
  46. drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h +37 −0
  47. drivers/gpu/drm/i915/gt/intel_gt_buffer_pool_types.h +9 −6
  48. drivers/gpu/drm/i915/gt/intel_gt_pm.c +6 −4
  49. drivers/gpu/drm/i915/gt/intel_gt_types.h +11 −0
  50. drivers/gpu/drm/i915/gt/intel_lrc.c +224 −96
  51. drivers/gpu/drm/i915/gt/intel_rc6.c +1 −7
  52. drivers/gpu/drm/i915/gt/intel_renderstate.c +9 −7
  53. drivers/gpu/drm/i915/gt/intel_rps.c +4 −1
  54. drivers/gpu/drm/i915/gt/intel_rps.h +1 −0
  55. drivers/gpu/drm/i915/gt/intel_timeline.c +4 −4
  56. drivers/gpu/drm/i915/gt/intel_timeline.h +0 −5
  57. drivers/gpu/drm/i915/gt/intel_workarounds.c +7 −21
  58. drivers/gpu/drm/i915/gt/mock_engine.c +0 −2
  59. drivers/gpu/drm/i915/gt/selftest_context.c +4 −8
  60. drivers/gpu/drm/i915/gt/selftest_lrc.c +42 −44
  61. drivers/gpu/drm/i915/gt/selftest_ring_submission.c +2 −0
  62. drivers/gpu/drm/i915/gt/selftest_rps.c +42 −14
  63. drivers/gpu/drm/i915/gt/sysfs_engines.c +94 −0
  64. drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +1 −1
  65. drivers/gpu/drm/i915/gvt/cmd_parser.c +45 −0
  66. drivers/gpu/drm/i915/gvt/execlist.c +0 −2
  67. drivers/gpu/drm/i915/gvt/gtt.c +1 −0
  68. drivers/gpu/drm/i915/gvt/gtt.h +1 −0
  69. drivers/gpu/drm/i915/gvt/handlers.c +1 −1
  70. drivers/gpu/drm/i915/gvt/scheduler.c +106 −9
  71. drivers/gpu/drm/i915/gvt/scheduler.h +1 −0
  72. drivers/gpu/drm/i915/i915_config.c +15 −0
  73. drivers/gpu/drm/i915/i915_debugfs.c +7 −4
  74. drivers/gpu/drm/i915/i915_drv.h +29 −8
  75. drivers/gpu/drm/i915/i915_gem_evict.c +12 −14
  76. drivers/gpu/drm/i915/i915_getparam.c +1 −1
  77. drivers/gpu/drm/i915/i915_gpu_error.h +1 −1
  78. drivers/gpu/drm/i915/i915_irq.c +125 −102
  79. drivers/gpu/drm/i915/i915_pci.c +1 −1
  80. drivers/gpu/drm/i915/i915_perf.c +4 −8
  81. drivers/gpu/drm/i915/i915_pmu.c +4 −1
  82. drivers/gpu/drm/i915/i915_priolist_types.h +1 −6
  83. drivers/gpu/drm/i915/i915_reg.h +13 −0
  84. drivers/gpu/drm/i915/i915_request.c +82 −45
  85. drivers/gpu/drm/i915/i915_request.h +20 −8
  86. drivers/gpu/drm/i915/i915_scheduler.c +8 −21
  87. drivers/gpu/drm/i915/i915_scheduler.h +2 −1
  88. drivers/gpu/drm/i915/i915_scheduler_types.h +2 −1
  89. drivers/gpu/drm/i915/i915_selftest.h +2 −0
  90. drivers/gpu/drm/i915/i915_sw_fence.c +4 −6
  91. drivers/gpu/drm/i915/intel_device_info.c +17 −17
  92. drivers/gpu/drm/i915/intel_device_info.h +1 −1
  93. drivers/gpu/drm/i915/intel_pm.c +266 −62
  94. drivers/gpu/drm/i915/intel_pm.h +3 −1
  95. drivers/gpu/drm/i915/intel_sideband.c +3 −1
  96. drivers/gpu/drm/i915/intel_uncore.c +0 −5
  97. drivers/gpu/drm/i915/intel_wopcm.c +1 −2
  98. drivers/gpu/drm/i915/selftests/i915_live_selftests.h +1 −0
  99. drivers/gpu/drm/i915/selftests/i915_perf.c +2 −4
  100. drivers/gpu/drm/i915/selftests/i915_request.c +21 −16
  101. drivers/gpu/drm/i915/selftests/i915_selftest.c +29 −0
  102. drivers/gpu/drm/i915/selftests/igt_spinner.c +1 −2
  103. drivers/video/hdmi.c +46 −19
  104. include/drm/drm_dp_helper.h +3 −0
  105. include/linux/hdmi.h +2 −0
Documentation/gpu/i915.rst+46−0
@@ -329,6 +329,52 @@ for execution also include a list of all locations within buffers that · 328 unmodified lines
329329 refer to GPU-addresses so that the kernel can edit the buffer correctly.
330330 This process is dubbed relocation.
331331
332+Locking Guidelines
333+------------------
334+
335+.. note::
336+ This is a description of how the locking should be after
337+ refactoring is done. Does not necessarily reflect what the locking
338+ looks like while WIP.
339+
340+#. All locking rules and interface contracts with cross-driver interfaces
341+ (dma-buf, dma_fence) need to be followed.
342+
343+#. No struct_mutex anywhere in the code
344+
345+#. dma_resv will be the outermost lock (when needed) and ww_acquire_ctx
346+ is to be hoisted at highest level and passed down within i915_gem_ctx
347+ in the call chain
348+
349+#. While holding lru/memory manager (buddy, drm_mm, whatever) locks
350+ system memory allocations are not allowed
351+
352+ * Enforce this by priming lockdep (with fs_reclaim). If we
353+ allocate memory while holding these looks we get a rehash
354+ of the shrinker vs. struct_mutex saga, and that would be
355+ real bad.
356+
357+#. Do not nest different lru/memory manager locks within each other.
358+ Take them in turn to update memory allocations, relying on the object’s
359+ dma_resv ww_mutex to serialize against other operations.
360+
361+#. The suggestion for lru/memory managers locks is that they are small
362+ enough to be spinlocks.
363+
364+#. All features need to come with exhaustive kernel selftests and/or
365+ IGT tests when appropriate
366+
367+#. All LMEM uAPI paths need to be fully restartable (_interruptible()
368+ for all locks/waits/sleeps)
369+
370+ * Error handling validation through signal injection.
371+ Still the best strategy we have for validating GEM uAPI
372+ corner cases.
373+ Must be excessively used in the IGT, and we need to check
374+ that we really have full path coverage of all error cases.
375+
376+ * -EDEADLK handling with ww_mutex
377+
332378 GEM BO Management Implementation Details
333379 ----------------------------------------
334380
drivers/gpu/drm/drm_dp_helper.c+174−0
@@ -1629,3 +1629,177 @@ int drm_dp_set_phy_test_pattern(struct drm_dp_aux *aux, · 1628 unmodified lines
16291629 return 0;
16301630 }
16311631 EXPORT_SYMBOL(drm_dp_set_phy_test_pattern);
1632+
1633+static const char *dp_pixelformat_get_name(enum dp_pixelformat pixelformat)
1634+{
1635+ if (pixelformat < 0 || pixelformat > DP_PIXELFORMAT_RESERVED)
1636+ return "Invalid";
1637+
1638+ switch (pixelformat) {
1639+ case DP_PIXELFORMAT_RGB:
1640+ return "RGB";
1641+ case DP_PIXELFORMAT_YUV444:
1642+ return "YUV444";
1643+ case DP_PIXELFORMAT_YUV422:
1644+ return "YUV422";
1645+ case DP_PIXELFORMAT_YUV420:
1646+ return "YUV420";
1647+ case DP_PIXELFORMAT_Y_ONLY:
1648+ return "Y_ONLY";
1649+ case DP_PIXELFORMAT_RAW:
1650+ return "RAW";
1651+ default:
1652+ return "Reserved";
1653+ }
1654+}
1655+
1656+static const char *dp_colorimetry_get_name(enum dp_pixelformat pixelformat,
1657+ enum dp_colorimetry colorimetry)
1658+{
1659+ if (pixelformat < 0 || pixelformat > DP_PIXELFORMAT_RESERVED)
1660+ return "Invalid";
1661+
1662+ switch (colorimetry) {
1663+ case DP_COLORIMETRY_DEFAULT:
1664+ switch (pixelformat) {
1665+ case DP_PIXELFORMAT_RGB:
1666+ return "sRGB";
1667+ case DP_PIXELFORMAT_YUV444:
1668+ case DP_PIXELFORMAT_YUV422:
1669+ case DP_PIXELFORMAT_YUV420:
1670+ return "BT.601";
1671+ case DP_PIXELFORMAT_Y_ONLY:
1672+ return "DICOM PS3.14";
1673+ case DP_PIXELFORMAT_RAW:
1674+ return "Custom Color Profile";
1675+ default:
1676+ return "Reserved";
1677+ }
1678+ case DP_COLORIMETRY_RGB_WIDE_FIXED: /* and DP_COLORIMETRY_BT709_YCC */
1679+ switch (pixelformat) {
1680+ case DP_PIXELFORMAT_RGB:
1681+ return "Wide Fixed";
1682+ case DP_PIXELFORMAT_YUV444:
1683+ case DP_PIXELFORMAT_YUV422:
1684+ case DP_PIXELFORMAT_YUV420:
1685+ return "BT.709";
1686+ default:
1687+ return "Reserved";
1688+ }
1689+ case DP_COLORIMETRY_RGB_WIDE_FLOAT: /* and DP_COLORIMETRY_XVYCC_601 */
1690+ switch (pixelformat) {
1691+ case DP_PIXELFORMAT_RGB:
1692+ return "Wide Float";
1693+ case DP_PIXELFORMAT_YUV444:
1694+ case DP_PIXELFORMAT_YUV422:
1695+ case DP_PIXELFORMAT_YUV420:
1696+ return "xvYCC 601";
1697+ default:
1698+ return "Reserved";
1699+ }
1700+ case DP_COLORIMETRY_OPRGB: /* and DP_COLORIMETRY_XVYCC_709 */
1701+ switch (pixelformat) {
1702+ case DP_PIXELFORMAT_RGB:
1703+ return "OpRGB";
1704+ case DP_PIXELFORMAT_YUV444:
1705+ case DP_PIXELFORMAT_YUV422:
1706+ case DP_PIXELFORMAT_YUV420:
1707+ return "xvYCC 709";
1708+ default:
1709+ return "Reserved";
1710+ }
1711+ case DP_COLORIMETRY_DCI_P3_RGB: /* and DP_COLORIMETRY_SYCC_601 */
1712+ switch (pixelformat) {
1713+ case DP_PIXELFORMAT_RGB:
1714+ return "DCI-P3";
1715+ case DP_PIXELFORMAT_YUV444:
1716+ case DP_PIXELFORMAT_YUV422:
1717+ case DP_PIXELFORMAT_YUV420:
1718+ return "sYCC 601";
1719+ default:
1720+ return "Reserved";
1721+ }
1722+ case DP_COLORIMETRY_RGB_CUSTOM: /* and DP_COLORIMETRY_OPYCC_601 */
1723+ switch (pixelformat) {
1724+ case DP_PIXELFORMAT_RGB:
1725+ return "Custom Profile";
1726+ case DP_PIXELFORMAT_YUV444:
1727+ case DP_PIXELFORMAT_YUV422:
1728+ case DP_PIXELFORMAT_YUV420:
1729+ return "OpYCC 601";
1730+ default:
1731+ return "Reserved";
1732+ }
1733+ case DP_COLORIMETRY_BT2020_RGB: /* and DP_COLORIMETRY_BT2020_CYCC */
1734+ switch (pixelformat) {
1735+ case DP_PIXELFORMAT_RGB:
1736+ return "BT.2020 RGB";
1737+ case DP_PIXELFORMAT_YUV444:
1738+ case DP_PIXELFORMAT_YUV422:
1739+ case DP_PIXELFORMAT_YUV420:
1740+ return "BT.2020 CYCC";
1741+ default:
1742+ return "Reserved";
1743+ }
1744+ case DP_COLORIMETRY_BT2020_YCC:
1745+ switch (pixelformat) {
1746+ case DP_PIXELFORMAT_YUV444:
1747+ case DP_PIXELFORMAT_YUV422:
1748+ case DP_PIXELFORMAT_YUV420:
1749+ return "BT.2020 YCC";
1750+ default:
1751+ return "Reserved";
1752+ }
1753+ default:
1754+ return "Invalid";
1755+ }
1756+}
1757+
1758+static const char *dp_dynamic_range_get_name(enum dp_dynamic_range dynamic_range)
1759+{
1760+ switch (dynamic_range) {
1761+ case DP_DYNAMIC_RANGE_VESA:
1762+ return "VESA range";
1763+ case DP_DYNAMIC_RANGE_CTA:
1764+ return "CTA range";
1765+ default:
1766+ return "Invalid";
1767+ }
1768+}
1769+
1770+static const char *dp_content_type_get_name(enum dp_content_type content_type)
1771+{
1772+ switch (content_type) {
1773+ case DP_CONTENT_TYPE_NOT_DEFINED:
1774+ return "Not defined";
1775+ case DP_CONTENT_TYPE_GRAPHICS:
1776+ return "Graphics";
1777+ case DP_CONTENT_TYPE_PHOTO:
1778+ return "Photo";
1779+ case DP_CONTENT_TYPE_VIDEO:
1780+ return "Video";
1781+ case DP_CONTENT_TYPE_GAME:
1782+ return "Game";
1783+ default:
1784+ return "Reserved";
1785+ }
1786+}
1787+
1788+void drm_dp_vsc_sdp_log(const char *level, struct device *dev,
1789+ const struct drm_dp_vsc_sdp *vsc)
1790+{
1791+#define DP_SDP_LOG(fmt, ...) dev_printk(level, dev, fmt, ##__VA_ARGS__)
1792+ DP_SDP_LOG("DP SDP: %s, revision %u, length %u\n", "VSC",
1793+ vsc->revision, vsc->length);
1794+ DP_SDP_LOG(" pixelformat: %s\n",
1795+ dp_pixelformat_get_name(vsc->pixelformat));
1796+ DP_SDP_LOG(" colorimetry: %s\n",
1797+ dp_colorimetry_get_name(vsc->pixelformat, vsc->colorimetry));
1798+ DP_SDP_LOG(" bpc: %u\n", vsc->bpc);
1799+ DP_SDP_LOG(" dynamic range: %s\n",
1800+ dp_dynamic_range_get_name(vsc->dynamic_range));
1801+ DP_SDP_LOG(" content type: %s\n",
1802+ dp_content_type_get_name(vsc->content_type));
1803+#undef DP_SDP_LOG
1804+}
1805+EXPORT_SYMBOL(drm_dp_vsc_sdp_log);
drivers/gpu/drm/i915/Kconfig.profile+12−0
@@ -1,3 +1,15 @@
1+config DRM_I915_FENCE_TIMEOUT
2+ int "Timeout for unsignaled foreign fences (ms, jiffy granularity)"
3+ default 10000 # milliseconds
4+ help
5+ When listening to a foreign fence, we install a supplementary timer
6+ to ensure that we are always signaled and our userspace is able to
7+ make forward progress. This value specifies the timeout used for an
8+ unsignaled foreign fence.
9+
10+ May be 0 to disable the timeout, and rely on the foreign fence being
11+ eventually signaled.
12+
113 config DRM_I915_USERFAULT_AUTOSUSPEND
214 int "Runtime autosuspend delay for userspace GGTT mmaps (ms)"
315 default 250 # milliseconds
drivers/gpu/drm/i915/Makefile+2−1
@@ -35,6 +35,7 @@ subdir-ccflags-y += -I$(srctree)/$(src) · 34 unmodified lines
3535
3636 # core driver code
3737 i915-y += i915_drv.o \
38+ i915_config.o \
3839 i915_irq.o \
3940 i915_getparam.o \
4041 i915_params.o \
@@ -87,11 +88,11 @@ gt-y += \ · 46 unmodified lines
8788 gt/intel_engine_cs.o \
8889 gt/intel_engine_heartbeat.o \
8990 gt/intel_engine_pm.o \
90− gt/intel_engine_pool.o \
9191 gt/intel_engine_user.o \
9292 gt/intel_ggtt.o \
9393 gt/intel_ggtt_fencing.o \
9494 gt/intel_gt.o \
95+ gt/intel_gt_buffer_pool.o \
9596 gt/intel_gt_clock_utils.o \
9697 gt/intel_gt_irq.o \
9798 gt/intel_gt_pm.o \
drivers/gpu/drm/i915/display/intel_audio.c+29−48
@@ -514,85 +514,67 @@ static void hsw_audio_codec_disable(struct intel_encoder *encoder, · 513 unmodified lines
514514 mutex_unlock(&dev_priv->av_mutex);
515515 }
516516
517−/* Add a factor to take care of rounding and truncations */
518−#define ROUNDING_FACTOR 10000
519−
520−static unsigned int get_hblank_early_enable_config(struct intel_encoder *encoder,
521− const struct intel_crtc_state *crtc_state)
517+static unsigned int calc_hblank_early_prog(struct intel_encoder *encoder,
518+ const struct intel_crtc_state *crtc_state)
522519 {
523520 struct drm_i915_private *i915 = to_i915(encoder->base.dev);
524521 unsigned int link_clks_available, link_clks_required;
525522 unsigned int tu_data, tu_line, link_clks_active;
526− unsigned int hblank_rise, hblank_early_prog;
527− unsigned int h_active, h_total, hblank_delta, pixel_clk, v_total;
528− unsigned int fec_coeff, refresh_rate, cdclk, vdsc_bpp;
523+ unsigned int h_active, h_total, hblank_delta, pixel_clk;
524+ unsigned int fec_coeff, cdclk, vdsc_bpp;
525+ unsigned int link_clk, lanes;
526+ unsigned int hblank_rise;
529527
530528 h_active = crtc_state->hw.adjusted_mode.crtc_hdisplay;
531529 h_total = crtc_state->hw.adjusted_mode.crtc_htotal;
532− v_total = crtc_state->hw.adjusted_mode.crtc_vtotal;
533530 pixel_clk = crtc_state->hw.adjusted_mode.crtc_clock;
534− refresh_rate = crtc_state->hw.adjusted_mode.vrefresh;
535531 vdsc_bpp = crtc_state->dsc.compressed_bpp;
536532 cdclk = i915->cdclk.hw.cdclk;
537533 /* fec= 0.972261, using rounding multiplier of 1000000 */
538534 fec_coeff = 972261;
535+ link_clk = crtc_state->port_clock;
536+ lanes = crtc_state->lane_count;
539537
540538 drm_dbg_kms(&i915->drm, "h_active = %u link_clk = %u :"
541539 "lanes = %u vdsc_bpp = %u cdclk = %u\n",
542− h_active, crtc_state->port_clock, crtc_state->lane_count,
543− vdsc_bpp, cdclk);
540+ h_active, link_clk, lanes, vdsc_bpp, cdclk);
544541
545− if (WARN_ON(!crtc_state->port_clock || !crtc_state->lane_count ||
546− !crtc_state->dsc.compressed_bpp || !i915->cdclk.hw.cdclk))
542+ if (WARN_ON(!link_clk || !pixel_clk || !lanes || !vdsc_bpp || !cdclk))
547543 return 0;
548544
549− link_clks_available = ((((h_total - h_active) *
550− ((crtc_state->port_clock * ROUNDING_FACTOR) /
551− pixel_clk)) / ROUNDING_FACTOR) - 28);
552−
553− link_clks_required = DIV_ROUND_UP(192000, (refresh_rate *
554− v_total)) * ((48 /
555− crtc_state->lane_count) + 2);
545+ link_clks_available = (h_total - h_active) * link_clk / pixel_clk - 28;
546+ link_clks_required = DIV_ROUND_UP(192000 * h_total, 1000 * pixel_clk) * (48 / lanes + 2);
556547
557548 if (link_clks_available > link_clks_required)
558549 hblank_delta = 32;
559550 else
560− hblank_delta = DIV_ROUND_UP(((((5 * ROUNDING_FACTOR) /
561− crtc_state->port_clock) + ((5 *
562− ROUNDING_FACTOR) /
563− cdclk)) * pixel_clk),
564− ROUNDING_FACTOR);
565−
566− tu_data = (pixel_clk * vdsc_bpp * 8) / ((crtc_state->port_clock *
567− crtc_state->lane_count * fec_coeff) / 1000000);
568− tu_line = (((h_active * crtc_state->port_clock * fec_coeff) /
569− 1000000) / (64 * pixel_clk));
570− link_clks_active = (tu_line - 1) * 64 + tu_data;
551+ hblank_delta = DIV64_U64_ROUND_UP(mul_u32_u32(5 * (link_clk + cdclk), pixel_clk),
552+ mul_u32_u32(link_clk, cdclk));
571553
572− hblank_rise = ((link_clks_active + 6 * DIV_ROUND_UP(link_clks_active,
573− 250) + 4) * ((pixel_clk * ROUNDING_FACTOR) /
574− crtc_state->port_clock)) / ROUNDING_FACTOR;
554+ tu_data = div64_u64(mul_u32_u32(pixel_clk * vdsc_bpp * 8, 1000000),
555+ mul_u32_u32(link_clk * lanes, fec_coeff));
556+ tu_line = div64_u64(h_active * mul_u32_u32(link_clk, fec_coeff),
557+ mul_u32_u32(64 * pixel_clk, 1000000));
558+ link_clks_active = (tu_line - 1) * 64 + tu_data;
575559
576− hblank_early_prog = h_active - hblank_rise + hblank_delta;
560+ hblank_rise = (link_clks_active + 6 * DIV_ROUND_UP(link_clks_active, 250) + 4) * pixel_clk / link_clk;
577561
578− return hblank_early_prog;
562+ return h_active - hblank_rise + hblank_delta;
579563 }
580564
581−static unsigned int get_sample_room_req_config(const struct intel_crtc_state *crtc_state)
565+static unsigned int calc_samples_room(const struct intel_crtc_state *crtc_state)
582566 {
583567 unsigned int h_active, h_total, pixel_clk;
584− unsigned int samples_room;
568+ unsigned int link_clk, lanes;
585569
586570 h_active = crtc_state->hw.adjusted_mode.hdisplay;
587571 h_total = crtc_state->hw.adjusted_mode.htotal;
588572 pixel_clk = crtc_state->hw.adjusted_mode.clock;
573+ link_clk = crtc_state->port_clock;
574+ lanes = crtc_state->lane_count;
589575
590− samples_room = ((((h_total - h_active) * ((crtc_state->port_clock *
591− ROUNDING_FACTOR) / pixel_clk)) /
592− ROUNDING_FACTOR) - 12) / ((48 /
593− crtc_state->lane_count) + 2);
594−
595− return samples_room;
576+ return ((h_total - h_active) * link_clk - 12 * pixel_clk) /
577+ (pixel_clk * (48 / lanes + 2));
596578 }
597579
598580 static void enable_audio_dsc_wa(struct intel_encoder *encoder,
@@ -618,8 +600,7 @@ static void enable_audio_dsc_wa(struct intel_encoder *encoder, · 19 unmodified lines
618600 (crtc_state->hw.adjusted_mode.hdisplay >= 3840 &&
619601 crtc_state->hw.adjusted_mode.vdisplay >= 2160)) {
620602 /* Get hblank early enable value required */
621− hblank_early_prog = get_hblank_early_enable_config(encoder,
622− crtc_state);
603+ hblank_early_prog = calc_hblank_early_prog(encoder, crtc_state);
623604 if (hblank_early_prog < 32) {
624605 val &= ~HBLANK_START_COUNT_MASK(pipe);
625606 val |= HBLANK_START_COUNT(pipe, HBLANK_START_COUNT_32);
@@ -635,7 +616,7 @@ static void enable_audio_dsc_wa(struct intel_encoder *encoder, · 9 unmodified lines
635616 }
636617
637618 /* Get samples room value required */
638− samples_room = get_sample_room_req_config(crtc_state);
619+ samples_room = calc_samples_room(crtc_state);
639620 if (samples_room < 3) {
640621 val &= ~NUMBER_SAMPLES_PER_LINE_MASK(pipe);
641622 val |= NUMBER_SAMPLES_PER_LINE(pipe, samples_room);
drivers/gpu/drm/i915/display/intel_bw.c+114−47
@@ -8,6 +8,9 @@ · 7 unmodified lines
88 #include "intel_bw.h"
99 #include "intel_display_types.h"
1010 #include "intel_sideband.h"
11+#include "intel_atomic.h"
12+#include "intel_pm.h"
13+
1114
1215 /* Parameters for Qclk Geyserville (QGV) */
1316 struct intel_qgv_point {
@@ -113,6 +116,26 @@ static int icl_pcode_read_qgv_point_info(struct drm_i915_private *dev_priv, · 99 unmodified lines
113116 return 0;
114117 }
115118
119+int icl_pcode_restrict_qgv_points(struct drm_i915_private *dev_priv,
120+ u32 points_mask)
121+{
122+ int ret;
123+
124+ /* bspec says to keep retrying for at least 1 ms */
125+ ret = skl_pcode_request(dev_priv, ICL_PCODE_SAGV_DE_MEM_SS_CONFIG,
126+ points_mask,
127+ ICL_PCODE_POINTS_RESTRICTED_MASK,
128+ ICL_PCODE_POINTS_RESTRICTED,
129+ 1);
130+
131+ if (ret < 0) {
132+ drm_err(&dev_priv->drm, "Failed to disable qgv points (%d)\n", ret);
133+ return ret;
134+ }
135+
136+ return 0;
137+}
138+
116139 static int icl_get_qgv_points(struct drm_i915_private *dev_priv,
117140 struct intel_qgv_info *qi)
118141 {
@@ -240,6 +263,16 @@ static int icl_get_bw_info(struct drm_i915_private *dev_priv, const struct intel · 121 unmodified lines
240263 break;
241264 }
242265
266+ /*
267+ * In case if SAGV is disabled in BIOS, we always get 1
268+ * SAGV point, but we can't send PCode commands to restrict it
269+ * as it will fail and pointless anyway.
270+ */
271+ if (qi.num_points == 1)
272+ dev_priv->sagv_status = I915_SAGV_NOT_CONTROLLED;
273+ else
274+ dev_priv->sagv_status = I915_SAGV_ENABLED;
275+
243276 return 0;
244277 }
245278
@@ -248,6 +281,11 @@ static unsigned int icl_max_bw(struct drm_i915_private *dev_priv, · 2 unmodified lines
248281 {
249282 int i;
250283
284+ /*
285+ * Let's return max bw for 0 planes
286+ */
287+ num_planes = max(1, num_planes);
288+
251289 for (i = 0; i < ARRAY_SIZE(dev_priv->max_bw); i++) {
252290 const struct intel_bw_info *bi =
253291 &dev_priv->max_bw[i];
@@ -277,34 +315,6 @@ void intel_bw_init_hw(struct drm_i915_private *dev_priv) · 23 unmodified lines
277315 icl_get_bw_info(dev_priv, &icl_sa_info);
278316 }
279317
280−static unsigned int intel_max_data_rate(struct drm_i915_private *dev_priv,
281− int num_planes)
282−{
283− if (INTEL_GEN(dev_priv) >= 11) {
284− /*
285− * Any bw group has same amount of QGV points
286− */
287− const struct intel_bw_info *bi =
288− &dev_priv->max_bw[0];
289− unsigned int min_bw = UINT_MAX;
290− int i;
291−
292− /*
293− * FIXME with SAGV disabled maybe we can assume
294− * point 1 will always be used? Seems to match
295− * the behaviour observed in the wild.
296− */
297− for (i = 0; i < bi->num_qgv_points; i++) {
298− unsigned int bw = icl_max_bw(dev_priv, num_planes, i);
299−
300− min_bw = min(bw, min_bw);
301− }
302− return min_bw;
303− } else {
304− return UINT_MAX;
305− }
306−}
307−
308318 static unsigned int intel_bw_crtc_num_active_planes(const struct intel_crtc_state *crtc_state)
309319 {
310320 /*
@@ -414,11 +424,16 @@ int intel_bw_atomic_check(struct intel_atomic_state *state) · 103 unmodified lines
414424 {
415425 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
416426 struct intel_crtc_state *new_crtc_state, *old_crtc_state;
417− struct intel_bw_state *bw_state = NULL;
418− unsigned int data_rate, max_data_rate;
427+ struct intel_bw_state *new_bw_state = NULL;
428+ const struct intel_bw_state *old_bw_state = NULL;
429+ unsigned int data_rate;
419430 unsigned int num_active_planes;
420431 struct intel_crtc *crtc;
421432 int i, ret;
433+ u32 allowed_points = 0;
434+ unsigned int max_bw_point = 0, max_bw = 0;
435+ unsigned int num_qgv_points = dev_priv->max_bw[0].num_qgv_points;
436+ u32 mask = (1 << num_qgv_points) - 1;
422437
423438 /* FIXME earlier gens need some checks too */
424439 if (INTEL_GEN(dev_priv) < 11)
@@ -443,41 +458,93 @@ int intel_bw_atomic_check(struct intel_atomic_state *state) · 18 unmodified lines
443458 old_active_planes == new_active_planes)
444459 continue;
445460
446− bw_state = intel_atomic_get_bw_state(state);
447− if (IS_ERR(bw_state))
448− return PTR_ERR(bw_state);
461+ new_bw_state = intel_atomic_get_bw_state(state);
462+ if (IS_ERR(new_bw_state))
463+ return PTR_ERR(new_bw_state);
449464
450− bw_state->data_rate[crtc->pipe] = new_data_rate;
451− bw_state->num_active_planes[crtc->pipe] = new_active_planes;
465+ new_bw_state->data_rate[crtc->pipe] = new_data_rate;
466+ new_bw_state->num_active_planes[crtc->pipe] = new_active_planes;
452467
453468 drm_dbg_kms(&dev_priv->drm,
454469 "pipe %c data rate %u num active planes %u\n",
455470 pipe_name(crtc->pipe),
456− bw_state->data_rate[crtc->pipe],
457− bw_state->num_active_planes[crtc->pipe]);
471+ new_bw_state->data_rate[crtc->pipe],
472+ new_bw_state->num_active_planes[crtc->pipe]);
458473 }
459474
460− if (!bw_state)
475+ if (!new_bw_state)
461476 return 0;
462477
463− ret = intel_atomic_lock_global_state(&bw_state->base);
478+ ret = intel_atomic_lock_global_state(&new_bw_state->base);
464479 if (ret)
465480 return ret;
466481
467− data_rate = intel_bw_data_rate(dev_priv, bw_state);
468− num_active_planes = intel_bw_num_active_planes(dev_priv, bw_state);
482+ data_rate = intel_bw_data_rate(dev_priv, new_bw_state);
483+ data_rate = DIV_ROUND_UP(data_rate, 1000);
469484
470− max_data_rate = intel_max_data_rate(dev_priv, num_active_planes);
485+ num_active_planes = intel_bw_num_active_planes(dev_priv, new_bw_state);
471486
472− data_rate = DIV_ROUND_UP(data_rate, 1000);
487+ for (i = 0; i < num_qgv_points; i++) {
488+ unsigned int max_data_rate;
473489
474− if (data_rate > max_data_rate) {
475− drm_dbg_kms(&dev_priv->drm,
476− "Bandwidth %u MB/s exceeds max available %d MB/s (%d active planes)\n",
477− data_rate, max_data_rate, num_active_planes);
490+ max_data_rate = icl_max_bw(dev_priv, num_active_planes, i);
491+ /*
492+ * We need to know which qgv point gives us
493+ * maximum bandwidth in order to disable SAGV
494+ * if we find that we exceed SAGV block time
495+ * with watermarks. By that moment we already
496+ * have those, as it is calculated earlier in
497+ * intel_atomic_check,
498+ */
499+ if (max_data_rate > max_bw) {
500+ max_bw_point = i;
501+ max_bw = max_data_rate;
502+ }
503+ if (max_data_rate >= data_rate)
504+ allowed_points |= BIT(i);
505+ drm_dbg_kms(&dev_priv->drm, "QGV point %d: max bw %d required %d\n",
506+ i, max_data_rate, data_rate);
507+ }
508+
509+ /*
510+ * BSpec states that we always should have at least one allowed point
511+ * left, so if we couldn't - simply reject the configuration for obvious
512+ * reasons.
513+ */
514+ if (allowed_points == 0) {
515+ drm_dbg_kms(&dev_priv->drm, "No QGV points provide sufficient memory"
516+ " bandwidth %d for display configuration(%d active planes).\n",
517+ data_rate, num_active_planes);
478518 return -EINVAL;
479519 }
480520
521+ /*
522+ * Leave only single point with highest bandwidth, if
523+ * we can't enable SAGV due to the increased memory latency it may
524+ * cause.
525+ */
526+ if (!intel_can_enable_sagv(dev_priv, new_bw_state)) {
527+ allowed_points = BIT(max_bw_point);
528+ drm_dbg_kms(&dev_priv->drm, "No SAGV, using single QGV point %d\n",
529+ max_bw_point);
530+ }
531+ /*
532+ * We store the ones which need to be masked as that is what PCode
533+ * actually accepts as a parameter.
534+ */
535+ new_bw_state->qgv_points_mask = ~allowed_points & mask;
536+
537+ old_bw_state = intel_atomic_get_old_bw_state(state);
538+ /*
539+ * If the actual mask had changed we need to make sure that
540+ * the commits are serialized(in case this is a nomodeset, nonblocking)
541+ */
542+ if (new_bw_state->qgv_points_mask != old_bw_state->qgv_points_mask) {
543+ ret = intel_atomic_serialize_global_state(&new_bw_state->base);
544+ if (ret)
545+ return ret;
546+ }
547+
481548 return 0;
482549 }
483550
drivers/gpu/drm/i915/display/intel_bw.h+18−0
@@ -18,8 +18,24 @@ struct intel_crtc_state; · 17 unmodified lines
1818 struct intel_bw_state {
1919 struct intel_global_state base;
2020
21+ /*
22+ * Contains a bit mask, used to determine, whether correspondent
23+ * pipe allows SAGV or not.
24+ */
25+ u8 pipe_sagv_reject;
26+
27+ /*
28+ * Current QGV points mask, which restricts
29+ * some particular SAGV states, not to confuse
30+ * with pipe_sagv_mask.
31+ */
32+ u8 qgv_points_mask;
33+
2134 unsigned int data_rate[I915_MAX_PIPES];
2235 u8 num_active_planes[I915_MAX_PIPES];
36+
37+ /* bitmask of active pipes */
38+ u8 active_pipes;
2339 };
2440
2541 #define to_intel_bw_state(x) container_of((x), struct intel_bw_state, base)
@@ -38,5 +54,7 @@ int intel_bw_init(struct drm_i915_private *dev_priv); · 12 unmodified lines
3854 int intel_bw_atomic_check(struct intel_atomic_state *state);
3955 void intel_bw_crtc_update(struct intel_bw_state *bw_state,
4056 const struct intel_crtc_state *crtc_state);
57+int icl_pcode_restrict_qgv_points(struct drm_i915_private *dev_priv,
58+ u32 points_mask);
4159
4260 #endif /* __INTEL_BW_H__ */
drivers/gpu/drm/i915/display/intel_ddi.c+56−4
@@ -3472,6 +3472,8 @@ static void intel_ddi_post_disable_dp(struct intel_atomic_state *state, · 3471 unmodified lines
34723472 INTEL_OUTPUT_DP_MST);
34733473 enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
34743474
3475+ intel_dp_set_infoframes(encoder, false, old_crtc_state, old_conn_state);
3476+
34753477 /*
34763478 * Power down sink before disabling the port, otherwise we end
34773479 * up getting interrupts from the sink on detecting link loss.
@@ -3680,9 +3682,8 @@ static void intel_enable_ddi_dp(struct intel_atomic_state *state, · 202 unmodified lines
36803682 intel_dp_stop_link_train(intel_dp);
36813683
36823684 intel_edp_backlight_on(crtc_state, conn_state);
3683− intel_psr_enable(intel_dp, crtc_state);
3684− intel_dp_vsc_enable(intel_dp, crtc_state, conn_state);
3685− intel_dp_hdr_metadata_enable(intel_dp, crtc_state, conn_state);
3685+ intel_psr_enable(intel_dp, crtc_state, conn_state);
3686+ intel_dp_set_infoframes(encoder, true, crtc_state, conn_state);
36863687 intel_edp_drrs_enable(intel_dp, crtc_state);
36873688
36883689 if (crtc_state->has_audio)
@@ -3864,7 +3865,8 @@ static void intel_ddi_update_pipe_dp(struct intel_atomic_state *state, · 175 unmodified lines
38643865
38653866 intel_ddi_set_dp_msa(crtc_state, conn_state);
38663867
3867− intel_psr_update(intel_dp, crtc_state);
3868+ intel_psr_update(intel_dp, crtc_state, conn_state);
3869+ intel_dp_set_infoframes(encoder, true, crtc_state, conn_state);
38683870 intel_edp_drrs_enable(intel_dp, crtc_state);
38693871
38703872 intel_panel_update_backlight(state, encoder, crtc_state, conn_state);
@@ -4235,6 +4237,9 @@ void intel_ddi_get_config(struct intel_encoder *encoder, · 364 unmodified lines
42354237 pipe_config->fec_enable);
42364238 }
42374239
4240+ pipe_config->infoframes.enable |=
4241+ intel_hdmi_infoframes_enabled(encoder, pipe_config);
4242+
42384243 break;
42394244 case TRANS_DDI_MODE_SELECT_DP_MST:
42404245 pipe_config->output_types |= BIT(INTEL_OUTPUT_DP_MST);
@@ -4246,6 +4251,9 @@ void intel_ddi_get_config(struct intel_encoder *encoder, · 5 unmodified lines
42464251 REG_FIELD_GET(TRANS_DDI_MST_TRANSPORT_SELECT_MASK, temp);
42474252
42484253 intel_dp_get_m_n(intel_crtc, pipe_config);
4254+
4255+ pipe_config->infoframes.enable |=
4256+ intel_hdmi_infoframes_enabled(encoder, pipe_config);
42494257 break;
42504258 default:
42514259 break;
@@ -4300,6 +4308,9 @@ void intel_ddi_get_config(struct intel_encoder *encoder, · 48 unmodified lines
43004308
43014309 if (INTEL_GEN(dev_priv) >= 8)
43024310 bdw_get_trans_port_sync_config(pipe_config);
4311+
4312+ intel_read_dp_sdp(encoder, pipe_config, HDMI_PACKET_TYPE_GAMUT_METADATA);
4313+ intel_read_dp_sdp(encoder, pipe_config, DP_SDP_VSC);
43034314 }
43044315
43054316 static enum intel_output_type
@@ -4689,6 +4700,30 @@ intel_ddi_hotplug(struct intel_encoder *encoder, · 383 unmodified lines
46894700 return state;
46904701 }
46914702
4703+static bool lpt_digital_port_connected(struct intel_encoder *encoder)
4704+{
4705+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
4706+ u32 bit = dev_priv->hotplug.pch_hpd[encoder->hpd_pin];
4707+
4708+ return intel_de_read(dev_priv, SDEISR) & bit;
4709+}
4710+
4711+static bool hsw_digital_port_connected(struct intel_encoder *encoder)
4712+{
4713+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
4714+ u32 bit = dev_priv->hotplug.hpd[encoder->hpd_pin];
4715+
4716+ return intel_de_read(dev_priv, DEISR) & bit;
4717+}
4718+
4719+static bool bdw_digital_port_connected(struct intel_encoder *encoder)
4720+{
4721+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
4722+ u32 bit = dev_priv->hotplug.hpd[encoder->hpd_pin];
4723+
4724+ return intel_de_read(dev_priv, GEN8_DE_PORT_ISR) & bit;
4725+}
4726+
46924727 static struct intel_connector *
46934728 intel_ddi_init_hdmi_connector(struct intel_digital_port *intel_dig_port)
46944729 {
@@ -4885,6 +4920,23 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port) · 190 unmodified lines
48854920 port_name(port));
48864921 }
48874922
4923+ if (INTEL_GEN(dev_priv) >= 11) {
4924+ if (intel_phy_is_tc(dev_priv, phy))
4925+ intel_dig_port->connected = intel_tc_port_connected;
4926+ else
4927+ intel_dig_port->connected = lpt_digital_port_connected;
4928+ } else if (INTEL_GEN(dev_priv) >= 8) {
4929+ if (port == PORT_A || IS_GEN9_LP(dev_priv))
4930+ intel_dig_port->connected = bdw_digital_port_connected;
4931+ else
4932+ intel_dig_port->connected = lpt_digital_port_connected;
4933+ } else {
4934+ if (port == PORT_A)
4935+ intel_dig_port->connected = hsw_digital_port_connected;
4936+ else
4937+ intel_dig_port->connected = lpt_digital_port_connected;
4938+ }
4939+
48884940 intel_infoframe_init(intel_dig_port);
48894941
48904942 return;
drivers/gpu/drm/i915/display/intel_display.c+76−9
@@ -12866,6 +12866,16 @@ intel_dump_infoframe(struct drm_i915_private *dev_priv, · 12865 unmodified lines
1286612866 hdmi_infoframe_log(KERN_DEBUG, dev_priv->drm.dev, frame);
1286712867 }
1286812868
12869+static void
12870+intel_dump_dp_vsc_sdp(struct drm_i915_private *dev_priv,
12871+ const struct drm_dp_vsc_sdp *vsc)
12872+{
12873+ if (!drm_debug_enabled(DRM_UT_KMS))
12874+ return;
12875+
12876+ drm_dp_vsc_sdp_log(KERN_DEBUG, dev_priv->drm.dev, vsc);
12877+}
12878+
1286912879 #define OUTPUT_TYPE(x) [INTEL_OUTPUT_ ## x] = #x
1287012880
1287112881 static const char * const output_type_str[] = {
@@ -13023,6 +13033,15 @@ static void intel_dump_pipe_config(const struct intel_crtc_state *pipe_config, · 151 unmodified lines
1302313033 if (pipe_config->infoframes.enable &
1302413034 intel_hdmi_infoframe_enable(HDMI_INFOFRAME_TYPE_VENDOR))
1302513035 intel_dump_infoframe(dev_priv, &pipe_config->infoframes.hdmi);
13036+ if (pipe_config->infoframes.enable &
13037+ intel_hdmi_infoframe_enable(HDMI_INFOFRAME_TYPE_DRM))
13038+ intel_dump_infoframe(dev_priv, &pipe_config->infoframes.drm);
13039+ if (pipe_config->infoframes.enable &
13040+ intel_hdmi_infoframe_enable(HDMI_PACKET_TYPE_GAMUT_METADATA))
13041+ intel_dump_infoframe(dev_priv, &pipe_config->infoframes.drm);
13042+ if (pipe_config->infoframes.enable &
13043+ intel_hdmi_infoframe_enable(DP_SDP_VSC))
13044+ intel_dump_dp_vsc_sdp(dev_priv, &pipe_config->infoframes.vsc);
1302613045
1302713046 drm_dbg_kms(&dev_priv->drm, "requested mode:\n");
1302813047 drm_mode_debug_printmodeline(&pipe_config->hw.mode);
@@ -13470,6 +13489,13 @@ intel_compare_infoframe(const union hdmi_infoframe *a, · 441 unmodified lines
1347013489 return memcmp(a, b, sizeof(*a)) == 0;
1347113490 }
1347213491
13492+static bool
13493+intel_compare_dp_vsc_sdp(const struct drm_dp_vsc_sdp *a,
13494+ const struct drm_dp_vsc_sdp *b)
13495+{
13496+ return memcmp(a, b, sizeof(*a)) == 0;
13497+}
13498+
1347313499 static void
1347413500 pipe_config_infoframe_mismatch(struct drm_i915_private *dev_priv,
1347513501 bool fastset, const char *name,
@@ -13495,6 +13521,31 @@ pipe_config_infoframe_mismatch(struct drm_i915_private *dev_priv, · 19 unmodified lines
1349513521 }
1349613522 }
1349713523
13524+static void
13525+pipe_config_dp_vsc_sdp_mismatch(struct drm_i915_private *dev_priv,
13526+ bool fastset, const char *name,
13527+ const struct drm_dp_vsc_sdp *a,
13528+ const struct drm_dp_vsc_sdp *b)
13529+{
13530+ if (fastset) {
13531+ if (!drm_debug_enabled(DRM_UT_KMS))
13532+ return;
13533+
13534+ drm_dbg_kms(&dev_priv->drm,
13535+ "fastset mismatch in %s dp sdp\n", name);
13536+ drm_dbg_kms(&dev_priv->drm, "expected:\n");
13537+ drm_dp_vsc_sdp_log(KERN_DEBUG, dev_priv->drm.dev, a);
13538+ drm_dbg_kms(&dev_priv->drm, "found:\n");
13539+ drm_dp_vsc_sdp_log(KERN_DEBUG, dev_priv->drm.dev, b);
13540+ } else {
13541+ drm_err(&dev_priv->drm, "mismatch in %s dp sdp\n", name);
13542+ drm_err(&dev_priv->drm, "expected:\n");
13543+ drm_dp_vsc_sdp_log(KERN_ERR, dev_priv->drm.dev, a);
13544+ drm_err(&dev_priv->drm, "found:\n");
13545+ drm_dp_vsc_sdp_log(KERN_ERR, dev_priv->drm.dev, b);
13546+ }
13547+}
13548+
1349813549 static void __printf(4, 5)
1349913550 pipe_config_mismatch(bool fastset, const struct intel_crtc *crtc,
1350013551 const char *name, const char *format, ...)
@@ -13696,6 +13747,17 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config, · 195 unmodified lines
1369613747 } \
1369713748 } while (0)
1369813749
13750+#define PIPE_CONF_CHECK_DP_VSC_SDP(name) do { \
13751+ if (!current_config->has_psr && !pipe_config->has_psr && \
13752+ !intel_compare_dp_vsc_sdp(&current_config->infoframes.name, \
13753+ &pipe_config->infoframes.name)) { \
13754+ pipe_config_dp_vsc_sdp_mismatch(dev_priv, fastset, __stringify(name), \
13755+ &current_config->infoframes.name, \
13756+ &pipe_config->infoframes.name); \
13757+ ret = false; \
13758+ } \
13759+} while (0)
13760+
1369913761 #define PIPE_CONF_CHECK_COLOR_LUT(name1, name2, bit_precision) do { \
1370013762 if (current_config->name1 != pipe_config->name1) { \
1370113763 pipe_config_mismatch(fastset, crtc, __stringify(name1), \
@@ -13873,6 +13935,7 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config, · 171 unmodified lines
1387313935 PIPE_CONF_CHECK_INFOFRAME(spd);
1387413936 PIPE_CONF_CHECK_INFOFRAME(hdmi);
1387513937 PIPE_CONF_CHECK_INFOFRAME(drm);
13938+ PIPE_CONF_CHECK_DP_VSC_SDP(vsc);
1387613939
1387713940 PIPE_CONF_CHECK_X(sync_mode_slaves_mask);
1387813941 PIPE_CONF_CHECK_I(master_transcoder);
@@ -13961,7 +14024,9 @@ static void verify_wm_state(struct intel_crtc *crtc, · 82 unmodified lines
1396114024 /* Watermarks */
1396214025 for (level = 0; level <= max_level; level++) {
1396314026 if (skl_wm_level_equals(&hw_plane_wm->wm[level],
13964− &sw_plane_wm->wm[level]))
14027+ &sw_plane_wm->wm[level]) ||
14028+ (level == 0 && skl_wm_level_equals(&hw_plane_wm->wm[level],
14029+ &sw_plane_wm->sagv_wm0)))
1396514030 continue;
1396614031
1396714032 drm_err(&dev_priv->drm,
@@ -14016,7 +14081,9 @@ static void verify_wm_state(struct intel_crtc *crtc, · 48 unmodified lines
1401614081 /* Watermarks */
1401714082 for (level = 0; level <= max_level; level++) {
1401814083 if (skl_wm_level_equals(&hw_plane_wm->wm[level],
14019− &sw_plane_wm->wm[level]))
14084+ &sw_plane_wm->wm[level]) ||
14085+ (level == 0 && skl_wm_level_equals(&hw_plane_wm->wm[level],
14086+ &sw_plane_wm->sagv_wm0)))
1402014087 continue;
1402114088
1402214089 drm_err(&dev_priv->drm,
@@ -15378,11 +15445,11 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state) · 1355 unmodified lines
1537815445
1537915446 intel_set_cdclk_pre_plane_update(state);
1538015447
15381− intel_sagv_pre_plane_update(state);
15382−
1538315448 intel_modeset_verify_disabled(dev_priv, state);
1538415449 }
1538515450
15451+ intel_sagv_pre_plane_update(state);
15452+
1538615453 /* Complete the events for pipes that have now been disabled */
1538715454 for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {
1538815455 bool modeset = needs_modeset(new_crtc_state);
@@ -15475,11 +15542,10 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state) · 86 unmodified lines
1547515542 intel_check_cpu_fifo_underruns(dev_priv);
1547615543 intel_check_pch_fifo_underruns(dev_priv);
1547715544
15478− if (state->modeset) {
15545+ if (state->modeset)
1547915546 intel_verify_planes(state);
1548015547
15481− intel_sagv_post_plane_update(state);
15482− }
15548+ intel_sagv_post_plane_update(state);
1548315549
1548415550 drm_atomic_helper_commit_hw_done(&state->base);
1548515551
@@ -15815,7 +15881,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane, · 329 unmodified lines
1581515881 if (new_plane_state->uapi.fence) { /* explicit fencing */
1581615882 ret = i915_sw_fence_await_dma_fence(&state->commit_ready,
1581715883 new_plane_state->uapi.fence,
15818− I915_FENCE_TIMEOUT,
15884+ i915_fence_timeout(dev_priv),
1581915885 GFP_KERNEL);
1582015886 if (ret < 0)
1582115887 return ret;
@@ -15842,7 +15908,8 @@ intel_prepare_plane_fb(struct drm_plane *_plane, · 20 unmodified lines
1584215908
1584315909 ret = i915_sw_fence_await_reservation(&state->commit_ready,
1584415910 obj->base.resv, NULL,
15845− false, I915_FENCE_TIMEOUT,
15911+ false,
15912+ i915_fence_timeout(dev_priv),
1584615913 GFP_KERNEL);
1584715914 if (ret < 0)
1584815915 goto unpin_fb;
drivers/gpu/drm/i915/display/intel_display_types.h+3−0
@@ -688,11 +688,13 @@ struct skl_plane_wm { · 687 unmodified lines
688688 struct skl_wm_level wm[8];
689689 struct skl_wm_level uv_wm[8];
690690 struct skl_wm_level trans_wm;
691+ struct skl_wm_level sagv_wm0;
691692 bool is_planar;
692693 };
693694
694695 struct skl_pipe_wm {
695696 struct skl_plane_wm planes[I915_MAX_PLANES];
697+ bool use_sagv_wm;
696698 };
697699
698700 enum vlv_wm_level {
@@ -1424,6 +1426,7 @@ struct intel_digital_port { · 725 unmodified lines
14241426 const struct drm_connector_state *conn_state);
14251427 u32 (*infoframes_enabled)(struct intel_encoder *encoder,
14261428 const struct intel_crtc_state *pipe_config);
1429+ bool (*connected)(struct intel_encoder *encoder);
14271430 };
14281431
14291432 struct intel_dp_mst_encoder {
drivers/gpu/drm/i915/display/intel_dp.c+230−364
@@ -2487,8 +2487,8 @@ static void intel_dp_compute_vsc_sdp(struct intel_dp *intel_dp, · 2486 unmodified lines
24872487 {
24882488 struct drm_dp_vsc_sdp *vsc = &crtc_state->infoframes.vsc;
24892489
2490− /* When PSR is enabled, VSC SDP is handled by PSR routine */
2491− if (intel_psr_enabled(intel_dp))
2490+ /* When a crtc state has PSR, VSC SDP will be handled by PSR routine */
2491+ if (crtc_state->has_psr)
24922492 return;
24932493
24942494 if (!intel_dp_needs_vsc_sdp(crtc_state, conn_state))
@@ -2500,6 +2500,42 @@ static void intel_dp_compute_vsc_sdp(struct intel_dp *intel_dp, · 5 unmodified lines
25002500 &crtc_state->infoframes.vsc);
25012501 }
25022502
2503+void intel_dp_compute_psr_vsc_sdp(struct intel_dp *intel_dp,
2504+ const struct intel_crtc_state *crtc_state,
2505+ const struct drm_connector_state *conn_state,
2506+ struct drm_dp_vsc_sdp *vsc)
2507+{
2508+ struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2509+
2510+ vsc->sdp_type = DP_SDP_VSC;
2511+
2512+ if (dev_priv->psr.psr2_enabled) {
2513+ if (dev_priv->psr.colorimetry_support &&
2514+ intel_dp_needs_vsc_sdp(crtc_state, conn_state)) {
2515+ /* [PSR2, +Colorimetry] */
2516+ intel_dp_compute_vsc_colorimetry(crtc_state, conn_state,
2517+ vsc);
2518+ } else {
2519+ /*
2520+ * [PSR2, -Colorimetry]
2521+ * Prepare VSC Header for SU as per eDP 1.4 spec, Table 6-11
2522+ * 3D stereo + PSR/PSR2 + Y-coordinate.
2523+ */
2524+ vsc->revision = 0x4;
2525+ vsc->length = 0xe;
2526+ }
2527+ } else {
2528+ /*
2529+ * [PSR1]
2530+ * Prepare VSC Header for SU as per DP 1.4 spec, Table 2-118
2531+ * VSC SDP supporting 3D stereo + PSR (applies to eDP v1.3 or
2532+ * higher).
2533+ */
2534+ vsc->revision = 0x2;
2535+ vsc->length = 0x8;
2536+ }
2537+}
2538+
25032539 static void
25042540 intel_dp_compute_hdr_metadata_infoframe_sdp(struct intel_dp *intel_dp,
25052541 struct intel_crtc_state *crtc_state,
@@ -4791,6 +4827,13 @@ static ssize_t intel_dp_vsc_sdp_pack(const struct drm_dp_vsc_sdp *vsc, · 2285 unmodified lines
47914827 sdp->sdp_header.HB2 = vsc->revision; /* Revision Number */
47924828 sdp->sdp_header.HB3 = vsc->length; /* Number of Valid Data Bytes */
47934829
4830+ /*
4831+ * Only revision 0x5 supports Pixel Encoding/Colorimetry Format as
4832+ * per DP 1.4a spec.
4833+ */
4834+ if (vsc->revision != 0x5)
4835+ goto out;
4836+
47944837 /* VSC SDP Payload for DB16 through DB18 */
47954838 /* Pixel Encoding and Colorimetry Formats */
47964839 sdp->db[16] = (vsc->pixelformat & 0xf) << 4; /* DB16[7:4] */
@@ -4823,6 +4866,7 @@ static ssize_t intel_dp_vsc_sdp_pack(const struct drm_dp_vsc_sdp *vsc, · 26 unmodified lines
48234866 /* Content Type */
48244867 sdp->db[18] = vsc->content_type & 0x7;
48254868
4869+out:
48264870 return length;
48274871 }
48284872
@@ -4935,6 +4979,24 @@ static void intel_write_dp_sdp(struct intel_encoder *encoder, · 106 unmodified lines
49354979 intel_dig_port->write_infoframe(encoder, crtc_state, type, &sdp, len);
49364980 }
49374981
4982+void intel_write_dp_vsc_sdp(struct intel_encoder *encoder,
4983+ const struct intel_crtc_state *crtc_state,
4984+ struct drm_dp_vsc_sdp *vsc)
4985+{
4986+ struct intel_digital_port *intel_dig_port = enc_to_dig_port(encoder);
4987+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
4988+ struct dp_sdp sdp = {};
4989+ ssize_t len;
4990+
4991+ len = intel_dp_vsc_sdp_pack(vsc, &sdp, sizeof(sdp));
4992+
4993+ if (drm_WARN_ON(&dev_priv->drm, len < 0))
4994+ return;
4995+
4996+ intel_dig_port->write_infoframe(encoder, crtc_state, DP_SDP_VSC,
4997+ &sdp, len);
4998+}
4999+
49385000 void intel_dp_set_infoframes(struct intel_encoder *encoder,
49395001 bool enable,
49405002 const struct intel_crtc_state *crtc_state,
@@ -4971,233 +5033,191 @@ void intel_dp_set_infoframes(struct intel_encoder *encoder, · 30 unmodified lines
49715033 intel_write_dp_sdp(encoder, crtc_state, HDMI_PACKET_TYPE_GAMUT_METADATA);
49725034 }
49735035
4974−static void
4975−intel_dp_setup_vsc_sdp(struct intel_dp *intel_dp,
4976− const struct intel_crtc_state *crtc_state,
4977− const struct drm_connector_state *conn_state)
5036+static int intel_dp_vsc_sdp_unpack(struct drm_dp_vsc_sdp *vsc,
5037+ const void *buffer, size_t size)
49785038 {
4979− struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
4980− struct dp_sdp vsc_sdp = {};
4981−
4982− /* Prepare VSC Header for SU as per DP 1.4a spec, Table 2-119 */
4983− vsc_sdp.sdp_header.HB0 = 0;
4984− vsc_sdp.sdp_header.HB1 = 0x7;
5039+ const struct dp_sdp *sdp = buffer;
49855040
4986− /*
4987− * VSC SDP supporting 3D stereo, PSR2, and Pixel Encoding/
4988− * Colorimetry Format indication.
4989− */
4990− vsc_sdp.sdp_header.HB2 = 0x5;
5041+ if (size < sizeof(struct dp_sdp))
5042+ return -EINVAL;
49915043
4992− /*
4993− * VSC SDP supporting 3D stereo, + PSR2, + Pixel Encoding/
4994− * Colorimetry Format indication (HB2 = 05h).
4995− */
4996− vsc_sdp.sdp_header.HB3 = 0x13;
5044+ memset(vsc, 0, size);
49975045
4998− /* DP 1.4a spec, Table 2-120 */
4999− switch (crtc_state->output_format) {
5000− case INTEL_OUTPUT_FORMAT_YCBCR444:
5001− vsc_sdp.db[16] = 0x1 << 4; /* YCbCr 444 : DB16[7:4] = 1h */
5002− break;
5003− case INTEL_OUTPUT_FORMAT_YCBCR420:
5004− vsc_sdp.db[16] = 0x3 << 4; /* YCbCr 420 : DB16[7:4] = 3h */
5005− break;
5006− case INTEL_OUTPUT_FORMAT_RGB:
5007− default:
5008− /* RGB: DB16[7:4] = 0h */
5009− break;
5010− }
5046+ if (sdp->sdp_header.HB0 != 0)
5047+ return -EINVAL;
50115048
5012− switch (conn_state->colorspace) {
5013− case DRM_MODE_COLORIMETRY_BT709_YCC:
5014− vsc_sdp.db[16] |= 0x1;
5015− break;
5016− case DRM_MODE_COLORIMETRY_XVYCC_601:
5017− vsc_sdp.db[16] |= 0x2;
5018− break;
5019− case DRM_MODE_COLORIMETRY_XVYCC_709:
5020− vsc_sdp.db[16] |= 0x3;
5021− break;
5022− case DRM_MODE_COLORIMETRY_SYCC_601:
5023− vsc_sdp.db[16] |= 0x4;
5024− break;
5025− case DRM_MODE_COLORIMETRY_OPYCC_601:
5026− vsc_sdp.db[16] |= 0x5;
5027− break;
5028− case DRM_MODE_COLORIMETRY_BT2020_CYCC:
5029− case DRM_MODE_COLORIMETRY_BT2020_RGB:
5030− vsc_sdp.db[16] |= 0x6;
5031− break;
5032− case DRM_MODE_COLORIMETRY_BT2020_YCC:
5033− vsc_sdp.db[16] |= 0x7;
5034− break;
5035− case DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65:
5036− case DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER:
5037− vsc_sdp.db[16] |= 0x4; /* DCI-P3 (SMPTE RP 431-2) */
5038− break;
5039− default:
5040− /* sRGB (IEC 61966-2-1) / ITU-R BT.601: DB16[0:3] = 0h */
5049+ if (sdp->sdp_header.HB1 != DP_SDP_VSC)
5050+ return -EINVAL;
50415051
5042− /* RGB->YCBCR color conversion uses the BT.709 color space. */
5043− if (crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420)
5044− vsc_sdp.db[16] |= 0x1; /* 0x1, ITU-R BT.709 */
5045− break;
5046− }
5052+ vsc->sdp_type = sdp->sdp_header.HB1;
5053+ vsc->revision = sdp->sdp_header.HB2;
5054+ vsc->length = sdp->sdp_header.HB3;
50475055
5048− /*
5049− * For pixel encoding formats YCbCr444, YCbCr422, YCbCr420, and Y Only,
5050− * the following Component Bit Depth values are defined:
5051− * 001b = 8bpc.
5052− * 010b = 10bpc.
5053− * 011b = 12bpc.
5054− * 100b = 16bpc.
5055− */
5056− switch (crtc_state->pipe_bpp) {
5057− case 24: /* 8bpc */
5058− vsc_sdp.db[17] = 0x1;
5059− break;
5060− case 30: /* 10bpc */
5061− vsc_sdp.db[17] = 0x2;
5062− break;
5063− case 36: /* 12bpc */
5064− vsc_sdp.db[17] = 0x3;
5065− break;
5066− case 48: /* 16bpc */
5067− vsc_sdp.db[17] = 0x4;
5068− break;
5069− default:
5070− MISSING_CASE(crtc_state->pipe_bpp);
5071− break;
5072− }
5056+ if ((sdp->sdp_header.HB2 == 0x2 && sdp->sdp_header.HB3 == 0x8) ||
5057+ (sdp->sdp_header.HB2 == 0x4 && sdp->sdp_header.HB3 == 0xe)) {
5058+ /*
5059+ * - HB2 = 0x2, HB3 = 0x8
5060+ * VSC SDP supporting 3D stereo + PSR
5061+ * - HB2 = 0x4, HB3 = 0xe
5062+ * VSC SDP supporting 3D stereo + PSR2 with Y-coordinate of
5063+ * first scan line of the SU region (applies to eDP v1.4b
5064+ * and higher).
5065+ */
5066+ return 0;
5067+ } else if (sdp->sdp_header.HB2 == 0x5 && sdp->sdp_header.HB3 == 0x13) {
5068+ /*
5069+ * - HB2 = 0x5, HB3 = 0x13
5070+ * VSC SDP supporting 3D stereo + PSR2 + Pixel Encoding/Colorimetry
5071+ * Format.
5072+ */
5073+ vsc->pixelformat = (sdp->db[16] >> 4) & 0xf;
5074+ vsc->colorimetry = sdp->db[16] & 0xf;
5075+ vsc->dynamic_range = (sdp->db[17] >> 7) & 0x1;
50735076
5074− /*
5075− * Dynamic Range (Bit 7)
5076− * 0 = VESA range, 1 = CTA range.
5077− * all YCbCr are always limited range
5078− */
5079− vsc_sdp.db[17] |= 0x80;
5077+ switch (sdp->db[17] & 0x7) {
5078+ case 0x0:
5079+ vsc->bpc = 6;
5080+ break;
5081+ case 0x1:
5082+ vsc->bpc = 8;
5083+ break;
5084+ case 0x2:
5085+ vsc->bpc = 10;
5086+ break;
5087+ case 0x3:
5088+ vsc->bpc = 12;
5089+ break;
5090+ case 0x4:
5091+ vsc->bpc = 16;
5092+ break;
5093+ default:
5094+ MISSING_CASE(sdp->db[17] & 0x7);
5095+ return -EINVAL;
5096+ }
50805097
5081− /*
5082− * Content Type (Bits 2:0)
5083− * 000b = Not defined.
5084− * 001b = Graphics.
5085− * 010b = Photo.
5086− * 011b = Video.
5087− * 100b = Game
5088− * All other values are RESERVED.
5089− * Note: See CTA-861-G for the definition and expected
5090− * processing by a stream sink for the above contect types.
5091− */
5092− vsc_sdp.db[18] = 0;
5098+ vsc->content_type = sdp->db[18] & 0x7;
5099+ } else {
5100+ return -EINVAL;
5101+ }
50935102
5094− intel_dig_port->write_infoframe(&intel_dig_port->base,
5095− crtc_state, DP_SDP_VSC, &vsc_sdp, sizeof(vsc_sdp));
5103+ return 0;
50965104 }
50975105
5098−static void
5099−intel_dp_setup_hdr_metadata_infoframe_sdp(struct intel_dp *intel_dp,
5100− const struct intel_crtc_state *crtc_state,
5101− const struct drm_connector_state *conn_state)
5106+static int
5107+intel_dp_hdr_metadata_infoframe_sdp_unpack(struct hdmi_drm_infoframe *drm_infoframe,
5108+ const void *buffer, size_t size)
51025109 {
5103− struct drm_i915_private *i915 = dp_to_i915(intel_dp);
5104− struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
5105− struct dp_sdp infoframe_sdp = {};
5106− struct hdmi_drm_infoframe drm_infoframe = {};
5107− const int infoframe_size = HDMI_INFOFRAME_HEADER_SIZE + HDMI_DRM_INFOFRAME_SIZE;
5108− unsigned char buf[HDMI_INFOFRAME_HEADER_SIZE + HDMI_DRM_INFOFRAME_SIZE];
5109− ssize_t len;
51105110 int ret;
51115111
5112− ret = drm_hdmi_infoframe_set_hdr_metadata(&drm_infoframe, conn_state);
5113− if (ret) {
5114− drm_dbg_kms(&i915->drm,
5115− "couldn't set HDR metadata in infoframe\n");
5116− return;
5117− }
5112+ const struct dp_sdp *sdp = buffer;
51185113
5119− len = hdmi_drm_infoframe_pack_only(&drm_infoframe, buf, sizeof(buf));
5120− if (len < 0) {
5121− drm_dbg_kms(&i915->drm,
5122− "buffer size is smaller than hdr metadata infoframe\n");
5123− return;
5124− }
5114+ if (size < sizeof(struct dp_sdp))
5115+ return -EINVAL;
51255116
5126− if (len != infoframe_size) {
5127− drm_dbg_kms(&i915->drm, "wrong static hdr metadata size\n");
5128− return;
5129− }
5117+ if (sdp->sdp_header.HB0 != 0)
5118+ return -EINVAL;
51305119
5131− /*
5132− * Set up the infoframe sdp packet for HDR static metadata.
5133− * Prepare VSC Header for SU as per DP 1.4a spec,
5134− * Table 2-100 and Table 2-101
5135− */
5120+ if (sdp->sdp_header.HB1 != HDMI_INFOFRAME_TYPE_DRM)
5121+ return -EINVAL;
51365122
5137− /* Packet ID, 00h for non-Audio INFOFRAME */
5138− infoframe_sdp.sdp_header.HB0 = 0;
5139− /*
5140− * Packet Type 80h + Non-audio INFOFRAME Type value
5141− * HDMI_INFOFRAME_TYPE_DRM: 0x87,
5142− */
5143− infoframe_sdp.sdp_header.HB1 = drm_infoframe.type;
51445123 /*
51455124 * Least Significant Eight Bits of (Data Byte Count – 1)
5146− * infoframe_size - 1,
5125+ * 1Dh (i.e., Data Byte Count = 30 bytes).
51475126 */
5148− infoframe_sdp.sdp_header.HB2 = 0x1D;
5127+ if (sdp->sdp_header.HB2 != 0x1D)
5128+ return -EINVAL;
5129+
5130+ /* Most Significant Two Bits of (Data Byte Count – 1), Clear to 00b. */
5131+ if ((sdp->sdp_header.HB3 & 0x3) != 0)
5132+ return -EINVAL;
5133+
51495134 /* INFOFRAME SDP Version Number */
5150− infoframe_sdp.sdp_header.HB3 = (0x13 << 2);
5135+ if (((sdp->sdp_header.HB3 >> 2) & 0x3f) != 0x13)
5136+ return -EINVAL;
5137+
51515138 /* CTA Header Byte 2 (INFOFRAME Version Number) */
5152− infoframe_sdp.db[0] = drm_infoframe.version;
5139+ if (sdp->db[0] != 1)
5140+ return -EINVAL;
5141+
51535142 /* CTA Header Byte 3 (Length of INFOFRAME): HDMI_DRM_INFOFRAME_SIZE */
5154− infoframe_sdp.db[1] = drm_infoframe.length;
5155− /*
5156− * Copy HDMI_DRM_INFOFRAME_SIZE size from a buffer after
5157− * HDMI_INFOFRAME_HEADER_SIZE
5158− */
5159− BUILD_BUG_ON(sizeof(infoframe_sdp.db) < HDMI_DRM_INFOFRAME_SIZE + 2);
5160− memcpy(&infoframe_sdp.db[2], &buf[HDMI_INFOFRAME_HEADER_SIZE],
5161− HDMI_DRM_INFOFRAME_SIZE);
5143+ if (sdp->db[1] != HDMI_DRM_INFOFRAME_SIZE)
5144+ return -EINVAL;
51625145
5163− /*
5164− * Size of DP infoframe sdp packet for HDR static metadata is consist of
5165− * - DP SDP Header(struct dp_sdp_header): 4 bytes
5166− * - Two Data Blocks: 2 bytes
5167− * CTA Header Byte2 (INFOFRAME Version Number)
5168− * CTA Header Byte3 (Length of INFOFRAME)
5169− * - HDMI_DRM_INFOFRAME_SIZE: 26 bytes
5170− *
5171− * Prior to GEN11's GMP register size is identical to DP HDR static metadata
5172− * infoframe size. But GEN11+ has larger than that size, write_infoframe
5173− * will pad rest of the size.
5174− */
5175− intel_dig_port->write_infoframe(&intel_dig_port->base, crtc_state,
5176− HDMI_PACKET_TYPE_GAMUT_METADATA,
5177− &infoframe_sdp,
5178− sizeof(struct dp_sdp_header) + 2 + HDMI_DRM_INFOFRAME_SIZE);
5146+ ret = hdmi_drm_infoframe_unpack_only(drm_infoframe, &sdp->db[2],
5147+ HDMI_DRM_INFOFRAME_SIZE);
5148+
5149+ return ret;
51795150 }
51805151
5181−void intel_dp_vsc_enable(struct intel_dp *intel_dp,
5182− const struct intel_crtc_state *crtc_state,
5183− const struct drm_connector_state *conn_state)
5152+static void intel_read_dp_vsc_sdp(struct intel_encoder *encoder,
5153+ struct intel_crtc_state *crtc_state,
5154+ struct drm_dp_vsc_sdp *vsc)
51845155 {
5185− if (!intel_dp_needs_vsc_sdp(crtc_state, conn_state))
5156+ struct intel_digital_port *intel_dig_port = enc_to_dig_port(encoder);
5157+ struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
5158+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
5159+ unsigned int type = DP_SDP_VSC;
5160+ struct dp_sdp sdp = {};
5161+ int ret;
5162+
5163+ /* When PSR is enabled, VSC SDP is handled by PSR routine */
5164+ if (intel_psr_enabled(intel_dp))
5165+ return;
5166+
5167+ if ((crtc_state->infoframes.enable &
5168+ intel_hdmi_infoframe_enable(type)) == 0)
51865169 return;
51875170
5188− intel_dp_setup_vsc_sdp(intel_dp, crtc_state, conn_state);
5171+ intel_dig_port->read_infoframe(encoder, crtc_state, type, &sdp, sizeof(sdp));
5172+
5173+ ret = intel_dp_vsc_sdp_unpack(vsc, &sdp, sizeof(sdp));
5174+
5175+ if (ret)
5176+ drm_dbg_kms(&dev_priv->drm, "Failed to unpack DP VSC SDP\n");
51895177 }
51905178
5191−void intel_dp_hdr_metadata_enable(struct intel_dp *intel_dp,
5192− const struct intel_crtc_state *crtc_state,
5193− const struct drm_connector_state *conn_state)
5179+static void intel_read_dp_hdr_metadata_infoframe_sdp(struct intel_encoder *encoder,
5180+ struct intel_crtc_state *crtc_state,
5181+ struct hdmi_drm_infoframe *drm_infoframe)
51945182 {
5195− if (!conn_state->hdr_output_metadata)
5183+ struct intel_digital_port *intel_dig_port = enc_to_dig_port(encoder);
5184+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
5185+ unsigned int type = HDMI_PACKET_TYPE_GAMUT_METADATA;
5186+ struct dp_sdp sdp = {};
5187+ int ret;
5188+
5189+ if ((crtc_state->infoframes.enable &
5190+ intel_hdmi_infoframe_enable(type)) == 0)
51965191 return;
51975192
5198− intel_dp_setup_hdr_metadata_infoframe_sdp(intel_dp,
5199− crtc_state,
5200− conn_state);
5193+ intel_dig_port->read_infoframe(encoder, crtc_state, type, &sdp,
5194+ sizeof(sdp));
5195+
5196+ ret = intel_dp_hdr_metadata_infoframe_sdp_unpack(drm_infoframe, &sdp,
5197+ sizeof(sdp));
5198+
5199+ if (ret)
5200+ drm_dbg_kms(&dev_priv->drm,
5201+ "Failed to unpack DP HDR Metadata Infoframe SDP\n");
5202+}
5203+
5204+void intel_read_dp_sdp(struct intel_encoder *encoder,
5205+ struct intel_crtc_state *crtc_state,
5206+ unsigned int type)
5207+{
5208+ switch (type) {
5209+ case DP_SDP_VSC:
5210+ intel_read_dp_vsc_sdp(encoder, crtc_state,
5211+ &crtc_state->infoframes.vsc);
5212+ break;
5213+ case HDMI_PACKET_TYPE_GAMUT_METADATA:
5214+ intel_read_dp_hdr_metadata_infoframe_sdp(encoder, crtc_state,
5215+ &crtc_state->infoframes.drm.drm);
5216+ break;
5217+ default:
5218+ MISSING_CASE(type);
5219+ break;
5220+ }
52015221 }
52025222
52035223 static u8 intel_dp_autotest_link_training(struct intel_dp *intel_dp)
@@ -5998,64 +6018,7 @@ edp_detect(struct intel_dp *intel_dp) · 794 unmodified lines
59986018 static bool ibx_digital_port_connected(struct intel_encoder *encoder)
59996019 {
60006020 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6001− u32 bit;
6002−
6003− switch (encoder->hpd_pin) {
6004− case HPD_PORT_B:
6005− bit = SDE_PORTB_HOTPLUG;
6006− break;
6007− case HPD_PORT_C:
6008− bit = SDE_PORTC_HOTPLUG;
6009− break;
6010− case HPD_PORT_D:
6011− bit = SDE_PORTD_HOTPLUG;
6012− break;
6013− default:
6014− MISSING_CASE(encoder->hpd_pin);
6015− return false;
6016− }
6017−
6018− return intel_de_read(dev_priv, SDEISR) & bit;
6019−}
6020−
6021−static bool cpt_digital_port_connected(struct intel_encoder *encoder)
6022−{
6023− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6024− u32 bit;
6025−
6026− switch (encoder->hpd_pin) {
6027− case HPD_PORT_B:
6028− bit = SDE_PORTB_HOTPLUG_CPT;
6029− break;
6030− case HPD_PORT_C:
6031− bit = SDE_PORTC_HOTPLUG_CPT;
6032− break;
6033− case HPD_PORT_D:
6034− bit = SDE_PORTD_HOTPLUG_CPT;
6035− break;
6036− default:
6037− MISSING_CASE(encoder->hpd_pin);
6038− return false;
6039− }
6040−
6041− return intel_de_read(dev_priv, SDEISR) & bit;
6042−}
6043−
6044−static bool spt_digital_port_connected(struct intel_encoder *encoder)
6045−{
6046− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6047− u32 bit;
6048−
6049− switch (encoder->hpd_pin) {
6050− case HPD_PORT_A:
6051− bit = SDE_PORTA_HOTPLUG_SPT;
6052− break;
6053− case HPD_PORT_E:
6054− bit = SDE_PORTE_HOTPLUG_SPT;
6055− break;
6056− default:
6057− return cpt_digital_port_connected(encoder);
6058− }
6021+ u32 bit = dev_priv->hotplug.pch_hpd[encoder->hpd_pin];
60596022
60606023 return intel_de_read(dev_priv, SDEISR) & bit;
60616024 }
@@ -6109,89 +6072,9 @@ static bool gm45_digital_port_connected(struct intel_encoder *encoder) · 47 unmodified lines
61096072 static bool ilk_digital_port_connected(struct intel_encoder *encoder)
61106073 {
61116074 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6075+ u32 bit = dev_priv->hotplug.hpd[encoder->hpd_pin];
61126076
6113− if (encoder->hpd_pin == HPD_PORT_A)
6114− return intel_de_read(dev_priv, DEISR) & DE_DP_A_HOTPLUG;
6115− else
6116− return ibx_digital_port_connected(encoder);
6117−}
6118−
6119−static bool snb_digital_port_connected(struct intel_encoder *encoder)
6120−{
6121− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6122−
6123− if (encoder->hpd_pin == HPD_PORT_A)
6124− return intel_de_read(dev_priv, DEISR) & DE_DP_A_HOTPLUG;
6125− else
6126− return cpt_digital_port_connected(encoder);
6127−}
6128−
6129−static bool ivb_digital_port_connected(struct intel_encoder *encoder)
6130−{
6131− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6132−
6133− if (encoder->hpd_pin == HPD_PORT_A)
6134− return intel_de_read(dev_priv, DEISR) & DE_DP_A_HOTPLUG_IVB;
6135− else
6136− return cpt_digital_port_connected(encoder);
6137−}
6138−
6139−static bool bdw_digital_port_connected(struct intel_encoder *encoder)
6140−{
6141− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6142−
6143− if (encoder->hpd_pin == HPD_PORT_A)
6144− return intel_de_read(dev_priv, GEN8_DE_PORT_ISR) & GEN8_PORT_DP_A_HOTPLUG;
6145− else
6146− return cpt_digital_port_connected(encoder);
6147−}
6148−
6149−static bool bxt_digital_port_connected(struct intel_encoder *encoder)
6150−{
6151− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6152− u32 bit;
6153−
6154− switch (encoder->hpd_pin) {
6155− case HPD_PORT_A:
6156− bit = BXT_DE_PORT_HP_DDIA;
6157− break;
6158− case HPD_PORT_B:
6159− bit = BXT_DE_PORT_HP_DDIB;
6160− break;
6161− case HPD_PORT_C:
6162− bit = BXT_DE_PORT_HP_DDIC;
6163− break;
6164− default:
6165− MISSING_CASE(encoder->hpd_pin);
6166− return false;
6167− }
6168−
6169− return intel_de_read(dev_priv, GEN8_DE_PORT_ISR) & bit;
6170−}
6171−
6172−static bool intel_combo_phy_connected(struct drm_i915_private *dev_priv,
6173− enum phy phy)
6174−{
6175− if (HAS_PCH_MCC(dev_priv) && phy == PHY_C)
6176− return intel_de_read(dev_priv, SDEISR) & SDE_TC_HOTPLUG_ICP(PORT_TC1);
6177−
6178− return intel_de_read(dev_priv, SDEISR) & SDE_DDI_HOTPLUG_ICP(phy);
6179−}
6180−
6181−static bool icp_digital_port_connected(struct intel_encoder *encoder)
6182−{
6183− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6184− struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
6185− enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
6186−
6187− if (intel_phy_is_combo(dev_priv, phy))
6188− return intel_combo_phy_connected(dev_priv, phy);
6189− else if (intel_phy_is_tc(dev_priv, phy))
6190− return intel_tc_port_connected(dig_port);
6191− else
6192− MISSING_CASE(encoder->hpd_pin);
6193−
6194− return false;
6077+ return intel_de_read(dev_priv, DEISR) & bit;
61956078 }
61966079
61976080 /*
@@ -6205,44 +6088,15 @@ static bool icp_digital_port_connected(struct intel_encoder *encoder) · 7 unmodified lines
62056088 *
62066089 * Return %true if port is connected, %false otherwise.
62076090 */
6208−static bool __intel_digital_port_connected(struct intel_encoder *encoder)
6209−{
6210− struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6211−
6212− if (HAS_GMCH(dev_priv)) {
6213− if (IS_GM45(dev_priv))
6214− return gm45_digital_port_connected(encoder);
6215− else
6216− return g4x_digital_port_connected(encoder);
6217− }
6218−
6219− if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)
6220− return icp_digital_port_connected(encoder);
6221− else if (INTEL_PCH_TYPE(dev_priv) >= PCH_SPT)
6222− return spt_digital_port_connected(encoder);
6223− else if (IS_GEN9_LP(dev_priv))
6224− return bxt_digital_port_connected(encoder);
6225− else if (IS_GEN(dev_priv, 8))
6226− return bdw_digital_port_connected(encoder);
6227− else if (IS_GEN(dev_priv, 7))
6228− return ivb_digital_port_connected(encoder);
6229− else if (IS_GEN(dev_priv, 6))
6230− return snb_digital_port_connected(encoder);
6231− else if (IS_GEN(dev_priv, 5))
6232− return ilk_digital_port_connected(encoder);
6233−
6234− MISSING_CASE(INTEL_GEN(dev_priv));
6235− return false;
6236−}
6237−
62386091 bool intel_digital_port_connected(struct intel_encoder *encoder)
62396092 {
62406093 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6094+ struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
62416095 bool is_connected = false;
62426096 intel_wakeref_t wakeref;
62436097
62446098 with_intel_display_power(dev_priv, POWER_DOMAIN_DISPLAY_CORE, wakeref)
6245− is_connected = __intel_digital_port_connected(encoder);
6099+ is_connected = dig_port->connected(encoder);
62466100
62476101 return is_connected;
62486102 }
@@ -8522,6 +8376,18 @@ bool intel_dp_init(struct drm_i915_private *dev_priv, · 2273 unmodified lines
85228376
85238377 intel_dig_port->hpd_pulse = intel_dp_hpd_pulse;
85248378
8379+ if (HAS_GMCH(dev_priv)) {
8380+ if (IS_GM45(dev_priv))
8381+ intel_dig_port->connected = gm45_digital_port_connected;
8382+ else
8383+ intel_dig_port->connected = g4x_digital_port_connected;
8384+ } else {
8385+ if (port == PORT_A)
8386+ intel_dig_port->connected = ilk_digital_port_connected;
8387+ else
8388+ intel_dig_port->connected = ibx_digital_port_connected;
8389+ }
8390+
85258391 if (port != PORT_A)
85268392 intel_infoframe_init(intel_dig_port);
85278393
drivers/gpu/drm/i915/display/intel_dp.h+10−5
@@ -16,6 +16,7 @@ struct drm_connector_state; · 15 unmodified lines
1616 struct drm_encoder;
1717 struct drm_i915_private;
1818 struct drm_modeset_acquire_ctx;
19+struct drm_dp_vsc_sdp;
1920 struct intel_connector;
2021 struct intel_crtc_state;
2122 struct intel_digital_port;
@@ -108,15 +109,19 @@ int intel_dp_link_required(int pixel_clock, int bpp); · 86 unmodified lines
108109 int intel_dp_max_data_rate(int max_link_clock, int max_lanes);
109110 bool intel_dp_needs_vsc_sdp(const struct intel_crtc_state *crtc_state,
110111 const struct drm_connector_state *conn_state);
111−void intel_dp_vsc_enable(struct intel_dp *intel_dp,
112− const struct intel_crtc_state *crtc_state,
113− const struct drm_connector_state *conn_state);
114−void intel_dp_hdr_metadata_enable(struct intel_dp *intel_dp,
112+void intel_dp_compute_psr_vsc_sdp(struct intel_dp *intel_dp,
115113 const struct intel_crtc_state *crtc_state,
116− const struct drm_connector_state *conn_state);
114+ const struct drm_connector_state *conn_state,
115+ struct drm_dp_vsc_sdp *vsc);
116+void intel_write_dp_vsc_sdp(struct intel_encoder *encoder,
117+ const struct intel_crtc_state *crtc_state,
118+ struct drm_dp_vsc_sdp *vsc);
117119 void intel_dp_set_infoframes(struct intel_encoder *encoder, bool enable,
118120 const struct intel_crtc_state *crtc_state,
119121 const struct drm_connector_state *conn_state);
122+void intel_read_dp_sdp(struct intel_encoder *encoder,
123+ struct intel_crtc_state *crtc_state,
124+ unsigned int type);
120125 bool intel_digital_port_connected(struct intel_encoder *encoder);
121126 void intel_dp_process_phy_request(struct intel_dp *intel_dp);
122127
drivers/gpu/drm/i915/display/intel_dp_mst.c+5−4
@@ -510,10 +510,6 @@ static void intel_mst_enable_dp(struct intel_atomic_state *state, · 509 unmodified lines
510510
511511 intel_ddi_enable_transcoder_func(encoder, pipe_config);
512512
513− intel_enable_pipe(pipe_config);
514−
515− intel_crtc_vblank_on(pipe_config);
516−
517513 drm_dbg_kms(&dev_priv->drm, "active links %d\n",
518514 intel_dp->active_mst_links);
519515
@@ -524,6 +520,11 @@ static void intel_mst_enable_dp(struct intel_atomic_state *state, · 4 unmodified lines
524520 drm_dp_check_act_status(&intel_dp->mst_mgr);
525521
526522 drm_dp_update_payload_part2(&intel_dp->mst_mgr);
523+
524+ intel_enable_pipe(pipe_config);
525+
526+ intel_crtc_vblank_on(pipe_config);
527+
527528 if (pipe_config->has_audio)
528529 intel_audio_codec_enable(encoder, pipe_config, conn_state);
529530 }
drivers/gpu/drm/i915/display/intel_fbc.c+10−2
@@ -540,6 +540,9 @@ static void __intel_fbc_cleanup_cfb(struct drm_i915_private *dev_priv) · 539 unmodified lines
540540 {
541541 struct intel_fbc *fbc = &dev_priv->fbc;
542542
543+ if (WARN_ON(intel_fbc_hw_is_active(dev_priv)))
544+ return;
545+
543546 if (!drm_mm_node_allocated(&fbc->compressed_fb))
544547 return;
545548
@@ -564,7 +567,7 @@ void intel_fbc_cleanup_cfb(struct drm_i915_private *dev_priv) · 18 unmodified lines
564567 }
565568
566569 static bool stride_is_valid(struct drm_i915_private *dev_priv,
567− unsigned int stride)
570+ u64 modifier, unsigned int stride)
568571 {
569572 /* This should have been caught earlier. */
570573 if (drm_WARN_ON_ONCE(&dev_priv->drm, (stride & (64 - 1)) != 0))
@@ -580,6 +583,11 @@ static bool stride_is_valid(struct drm_i915_private *dev_priv, · 9 unmodified lines
580583 if (IS_GEN(dev_priv, 4) && !IS_G4X(dev_priv) && stride < 2048)
581584 return false;
582585
586+ /* Display WA #1105: skl,bxt,kbl,cfl,glk */
587+ if (IS_GEN(dev_priv, 9) &&
588+ modifier == DRM_FORMAT_MOD_LINEAR && stride & 511)
589+ return false;
590+
583591 if (stride > 16384)
584592 return false;
585593
@@ -810,7 +818,7 @@ static bool intel_fbc_can_activate(struct intel_crtc *crtc) · 224 unmodified lines
810818 return false;
811819 }
812820
813− if (!stride_is_valid(dev_priv, cache->fb.stride)) {
821+ if (!stride_is_valid(dev_priv, cache->fb.modifier, cache->fb.stride)) {
814822 fbc->no_fbc_reason = "framebuffer stride not supported";
815823 return false;
816824 }
drivers/gpu/drm/i915/display/intel_lspcon.c+1−1
@@ -522,7 +522,7 @@ u32 lspcon_infoframes_enabled(struct intel_encoder *encoder, · 521 unmodified lines
522522 const struct intel_crtc_state *pipe_config)
523523 {
524524 /* FIXME actually read this from the hw */
525− return enc_to_intel_lspcon(encoder)->active;
525+ return 0;
526526 }
527527
528528 void lspcon_resume(struct intel_lspcon *lspcon)
drivers/gpu/drm/i915/display/intel_psr.c+19−39
@@ -30,6 +30,7 @@ · 29 unmodified lines
3030 #include "intel_display_types.h"
3131 #include "intel_psr.h"
3232 #include "intel_sprite.h"
33+#include "intel_hdmi.h"
3334
3435 /**
3536 * DOC: Panel Self Refresh (PSR/SRD)
@@ -357,39 +358,6 @@ void intel_psr_init_dpcd(struct intel_dp *intel_dp) · 321 unmodified lines
357358 }
358359 }
359360
360−static void intel_psr_setup_vsc(struct intel_dp *intel_dp,
361− const struct intel_crtc_state *crtc_state)
362−{
363− struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
364− struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
365− struct dp_sdp psr_vsc;
366−
367− if (dev_priv->psr.psr2_enabled) {
368− /* Prepare VSC Header for SU as per EDP 1.4 spec, Table 6.11 */
369− memset(&psr_vsc, 0, sizeof(psr_vsc));
370− psr_vsc.sdp_header.HB0 = 0;
371− psr_vsc.sdp_header.HB1 = 0x7;
372− if (dev_priv->psr.colorimetry_support) {
373− psr_vsc.sdp_header.HB2 = 0x5;
374− psr_vsc.sdp_header.HB3 = 0x13;
375− } else {
376− psr_vsc.sdp_header.HB2 = 0x4;
377− psr_vsc.sdp_header.HB3 = 0xe;
378− }
379− } else {
380− /* Prepare VSC packet as per EDP 1.3 spec, Table 3.10 */
381− memset(&psr_vsc, 0, sizeof(psr_vsc));
382− psr_vsc.sdp_header.HB0 = 0;
383− psr_vsc.sdp_header.HB1 = 0x7;
384− psr_vsc.sdp_header.HB2 = 0x2;
385− psr_vsc.sdp_header.HB3 = 0x8;
386− }
387−
388− intel_dig_port->write_infoframe(&intel_dig_port->base,
389− crtc_state,
390− DP_SDP_VSC, &psr_vsc, sizeof(psr_vsc));
391−}
392−
393361 static void hsw_psr_setup_aux(struct intel_dp *intel_dp)
394362 {
395363 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
@@ -756,6 +724,8 @@ void intel_psr_compute_config(struct intel_dp *intel_dp, · 360 unmodified lines
756724 if (intel_dp != dev_priv->psr.dp)
757725 return;
758726
727+ if (!psr_global_enabled(dev_priv))
728+ return;
759729 /*
760730 * HSW spec explicitly says PSR is tied to port A.
761731 * BDW+ platforms have a instance of PSR registers per transcoder but
@@ -798,6 +768,7 @@ void intel_psr_compute_config(struct intel_dp *intel_dp, · 36 unmodified lines
798768
799769 crtc_state->has_psr = true;
800770 crtc_state->has_psr2 = intel_psr2_config_valid(intel_dp, crtc_state);
771+ crtc_state->infoframes.enable |= intel_hdmi_infoframe_enable(DP_SDP_VSC);
801772 }
802773
803774 static void intel_psr_activate(struct intel_dp *intel_dp)
@@ -880,9 +851,12 @@ static void intel_psr_enable_source(struct intel_dp *intel_dp, · 76 unmodified lines
880851 }
881852
882853 static void intel_psr_enable_locked(struct drm_i915_private *dev_priv,
883− const struct intel_crtc_state *crtc_state)
854+ const struct intel_crtc_state *crtc_state,
855+ const struct drm_connector_state *conn_state)
884856 {
885857 struct intel_dp *intel_dp = dev_priv->psr.dp;
858+ struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
859+ struct intel_encoder *encoder = &intel_dig_port->base;
886860 u32 val;
887861
888862 drm_WARN_ON(&dev_priv->drm, dev_priv->psr.enabled);
@@ -921,7 +895,9 @@ static void intel_psr_enable_locked(struct drm_i915_private *dev_priv, · 32 unmodified lines
921895
922896 drm_dbg_kms(&dev_priv->drm, "Enabling PSR%s\n",
923897 dev_priv->psr.psr2_enabled ? "2" : "1");
924− intel_psr_setup_vsc(intel_dp, crtc_state);
898+ intel_dp_compute_psr_vsc_sdp(intel_dp, crtc_state, conn_state,
899+ &dev_priv->psr.vsc);
900+ intel_write_dp_vsc_sdp(encoder, crtc_state, &dev_priv->psr.vsc);
925901 intel_psr_enable_sink(intel_dp);
926902 intel_psr_enable_source(intel_dp, crtc_state);
927903 dev_priv->psr.enabled = true;
@@ -933,11 +909,13 @@ static void intel_psr_enable_locked(struct drm_i915_private *dev_priv, · 5 unmodified lines
933909 * intel_psr_enable - Enable PSR
934910 * @intel_dp: Intel DP
935911 * @crtc_state: new CRTC state
912+ * @conn_state: new CONNECTOR state
936913 *
937914 * This function can only be called after the pipe is fully trained and enabled.
938915 */
939916 void intel_psr_enable(struct intel_dp *intel_dp,
940− const struct intel_crtc_state *crtc_state)
917+ const struct intel_crtc_state *crtc_state,
918+ const struct drm_connector_state *conn_state)
941919 {
942920 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
943921
@@ -958,7 +936,7 @@ void intel_psr_enable(struct intel_dp *intel_dp, · 14 unmodified lines
958936 goto unlock;
959937 }
960938
961− intel_psr_enable_locked(dev_priv, crtc_state);
939+ intel_psr_enable_locked(dev_priv, crtc_state, conn_state);
962940
963941 unlock:
964942 mutex_unlock(&dev_priv->psr.lock);
@@ -1091,13 +1069,15 @@ static void psr_force_hw_tracking_exit(struct drm_i915_private *dev_priv) · 126 unmodified lines
10911069 * intel_psr_update - Update PSR state
10921070 * @intel_dp: Intel DP
10931071 * @crtc_state: new CRTC state
1072+ * @conn_state: new CONNECTOR state
10941073 *
10951074 * This functions will update PSR states, disabling, enabling or switching PSR
10961075 * version when executing fastsets. For full modeset, intel_psr_disable() and
10971076 * intel_psr_enable() should be called instead.
10981077 */
10991078 void intel_psr_update(struct intel_dp *intel_dp,
1100− const struct intel_crtc_state *crtc_state)
1079+ const struct intel_crtc_state *crtc_state,
1080+ const struct drm_connector_state *conn_state)
11011081 {
11021082 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
11031083 struct i915_psr *psr = &dev_priv->psr;
@@ -1134,7 +1114,7 @@ void intel_psr_update(struct intel_dp *intel_dp, · 30 unmodified lines
11341114 intel_psr_disable_locked(intel_dp);
11351115
11361116 if (enable)
1137− intel_psr_enable_locked(dev_priv, crtc_state);
1117+ intel_psr_enable_locked(dev_priv, crtc_state, conn_state);
11381118
11391119 unlock:
11401120 mutex_unlock(&dev_priv->psr.lock);
drivers/gpu/drm/i915/display/intel_psr.h+4−2
@@ -17,11 +17,13 @@ struct intel_dp; · 16 unmodified lines
1717 #define CAN_PSR(dev_priv) (HAS_PSR(dev_priv) && dev_priv->psr.sink_support)
1818 void intel_psr_init_dpcd(struct intel_dp *intel_dp);
1919 void intel_psr_enable(struct intel_dp *intel_dp,
20− const struct intel_crtc_state *crtc_state);
20+ const struct intel_crtc_state *crtc_state,
21+ const struct drm_connector_state *conn_state);
2122 void intel_psr_disable(struct intel_dp *intel_dp,
2223 const struct intel_crtc_state *old_crtc_state);
2324 void intel_psr_update(struct intel_dp *intel_dp,
24− const struct intel_crtc_state *crtc_state);
25+ const struct intel_crtc_state *crtc_state,
26+ const struct drm_connector_state *conn_state);
2527 int intel_psr_debug_set(struct drm_i915_private *dev_priv, u64 value);
2628 void intel_psr_invalidate(struct drm_i915_private *dev_priv,
2729 unsigned frontbuffer_bits,
drivers/gpu/drm/i915/display/intel_tc.c+4−3
@@ -238,8 +238,8 @@ static void tc_port_fixup_legacy_flag(struct intel_digital_port *dig_port, · 237 unmodified lines
238238 static u32 tc_port_live_status_mask(struct intel_digital_port *dig_port)
239239 {
240240 struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
241− enum tc_port tc_port = intel_port_to_tc(i915, dig_port->base.port);
242241 struct intel_uncore *uncore = &i915->uncore;
242+ u32 isr_bit = i915->hotplug.pch_hpd[dig_port->base.hpd_pin];
243243 u32 mask = 0;
244244 u32 val;
245245
@@ -258,7 +258,7 @@ static u32 tc_port_live_status_mask(struct intel_digital_port *dig_port) · 12 unmodified lines
258258 if (val & TC_LIVE_STATE_TC(dig_port->tc_phy_fia_idx))
259259 mask |= BIT(TC_PORT_DP_ALT);
260260
261− if (intel_uncore_read(uncore, SDEISR) & SDE_TC_HOTPLUG_ICP(tc_port))
261+ if (intel_uncore_read(uncore, SDEISR) & isr_bit)
262262 mask |= BIT(TC_PORT_LEGACY);
263263
264264 /* The sink can be connected only in a single mode. */
@@ -562,8 +562,9 @@ static bool intel_tc_port_needs_reset(struct intel_digital_port *dig_port) · 297 unmodified lines
562562 * connected ports are usable, and avoids exposing to the users objects they
563563 * can't really use.
564564 */
565−bool intel_tc_port_connected(struct intel_digital_port *dig_port)
565+bool intel_tc_port_connected(struct intel_encoder *encoder)
566566 {
567+ struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
567568 bool is_connected;
568569 intel_wakeref_t tc_cold_wref;
569570
drivers/gpu/drm/i915/display/intel_tc.h+2−1
@@ -10,8 +10,9 @@ · 9 unmodified lines
1010 #include <linux/types.h>
1111
1212 struct intel_digital_port;
13+struct intel_encoder;
1314
14−bool intel_tc_port_connected(struct intel_digital_port *dig_port);
15+bool intel_tc_port_connected(struct intel_encoder *encoder);
1516 u32 intel_tc_port_get_lane_mask(struct intel_digital_port *dig_port);
1617 u32 intel_tc_port_get_pin_assignment_mask(struct intel_digital_port *dig_port);
1718 int intel_tc_port_fia_max_lane_count(struct intel_digital_port *dig_port);
drivers/gpu/drm/i915/display/intel_vbt_defs.h+2−2
@@ -462,7 +462,7 @@ struct bdb_general_definitions { · 461 unmodified lines
462462 * number = (block_size - sizeof(bdb_general_definitions))/
463463 * defs->child_dev_size;
464464 */
465− u8 devices[0];
465+ u8 devices[];
466466 } __packed;
467467
468468 /*
@@ -839,7 +839,7 @@ struct bdb_mipi_config { · 370 unmodified lines
839839
840840 struct bdb_mipi_sequence {
841841 u8 version;
842− u8 data[0]; /* up to 6 variable length blocks */
842+ u8 data[]; /* up to 6 variable length blocks */
843843 } __packed;
844844
845845 /*
drivers/gpu/drm/i915/gem/i915_gem_clflush.c+1−1
@@ -108,7 +108,7 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj, · 107 unmodified lines
108108 if (clflush) {
109109 i915_sw_fence_await_reservation(&clflush->base.chain,
110110 obj->base.resv, NULL, true,
111− I915_FENCE_TIMEOUT,
111+ i915_fence_timeout(to_i915(obj->base.dev)),
112112 I915_FENCE_GFP);
113113 dma_resv_add_excl_fence(obj->base.resv, &clflush->base.dma);
114114 dma_fence_work_commit(&clflush->base);
drivers/gpu/drm/i915/gem/i915_gem_client_blt.c+1−3
@@ -6,7 +6,6 @@ · 5 unmodified lines
66 #include "i915_drv.h"
77 #include "gt/intel_context.h"
88 #include "gt/intel_engine_pm.h"
9−#include "gt/intel_engine_pool.h"
109 #include "i915_gem_client_blt.h"
1110 #include "i915_gem_object_blt.h"
1211
@@ -289,8 +288,7 @@ int i915_gem_schedule_fill_pages_blt(struct drm_i915_gem_object *obj, · 276 unmodified lines
289288
290289 i915_gem_object_lock(obj);
291290 err = i915_sw_fence_await_reservation(&work->wait,
292− obj->base.resv, NULL,
293− true, I915_FENCE_TIMEOUT,
291+ obj->base.resv, NULL, true, 0,
294292 I915_FENCE_GFP);
295293 if (err < 0) {
296294 dma_fence_set_error(&work->dma, err);
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c+313−165
@@ -15,8 +15,8 @@ · 14 unmodified lines
1515
1616 #include "gem/i915_gem_ioctls.h"
1717 #include "gt/intel_context.h"
18−#include "gt/intel_engine_pool.h"
1918 #include "gt/intel_gt.h"
19+#include "gt/intel_gt_buffer_pool.h"
2020 #include "gt/intel_gt_pm.h"
2121 #include "gt/intel_ring.h"
2222
@@ -268,7 +268,9 @@ struct i915_execbuffer { · 245 unmodified lines
268268 bool has_fence : 1;
269269 bool needs_unfenced : 1;
270270
271+ struct i915_vma *target;
271272 struct i915_request *rq;
273+ struct i915_vma *rq_vma;
272274 u32 *rq_cmd;
273275 unsigned int rq_size;
274276 } reloc_cache;
@@ -953,7 +955,7 @@ static void reloc_cache_init(struct reloc_cache *cache, · 678 unmodified lines
953955 cache->needs_unfenced = INTEL_INFO(i915)->unfenced_needs_alignment;
954956 cache->node.flags = 0;
955957 cache->rq = NULL;
956− cache->rq_size = 0;
958+ cache->target = NULL;
957959 }
958960
959961 static inline void *unmask_page(unsigned long p)
@@ -975,29 +977,122 @@ static inline struct i915_ggtt *cache_to_ggtt(struct reloc_cache *cache) · 15 unmodified lines
975977 return &i915->ggtt;
976978 }
977979
978−static void reloc_gpu_flush(struct reloc_cache *cache)
980+#define RELOC_TAIL 4
981+
982+static int reloc_gpu_chain(struct reloc_cache *cache)
979983 {
980− struct drm_i915_gem_object *obj = cache->rq->batch->obj;
984+ struct intel_gt_buffer_pool_node *pool;
985+ struct i915_request *rq = cache->rq;
986+ struct i915_vma *batch;
987+ u32 *cmd;
988+ int err;
981989
982− GEM_BUG_ON(cache->rq_size >= obj->base.size / sizeof(u32));
983− cache->rq_cmd[cache->rq_size] = MI_BATCH_BUFFER_END;
990+ pool = intel_gt_get_buffer_pool(rq->engine->gt, PAGE_SIZE);
991+ if (IS_ERR(pool))
992+ return PTR_ERR(pool);
984993
985− __i915_gem_object_flush_map(obj, 0, sizeof(u32) * (cache->rq_size + 1));
986− i915_gem_object_unpin_map(obj);
994+ batch = i915_vma_instance(pool->obj, rq->context->vm, NULL);
995+ if (IS_ERR(batch)) {
996+ err = PTR_ERR(batch);
997+ goto out_pool;
998+ }
987999
988− intel_gt_chipset_flush(cache->rq->engine->gt);
1000+ err = i915_vma_pin(batch, 0, 0, PIN_USER | PIN_NONBLOCK);
1001+ if (err)
1002+ goto out_pool;
9891003
990− i915_request_add(cache->rq);
991− cache->rq = NULL;
1004+ GEM_BUG_ON(cache->rq_size + RELOC_TAIL > PAGE_SIZE / sizeof(u32));
1005+ cmd = cache->rq_cmd + cache->rq_size;
1006+ *cmd++ = MI_ARB_CHECK;
1007+ if (cache->gen >= 8)
1008+ *cmd++ = MI_BATCH_BUFFER_START_GEN8;
1009+ else if (cache->gen >= 6)
1010+ *cmd++ = MI_BATCH_BUFFER_START;
1011+ else
1012+ *cmd++ = MI_BATCH_BUFFER_START | MI_BATCH_GTT;
1013+ *cmd++ = lower_32_bits(batch->node.start);
1014+ *cmd++ = upper_32_bits(batch->node.start); /* Always 0 for gen<8 */
1015+ i915_gem_object_flush_map(cache->rq_vma->obj);
1016+ i915_gem_object_unpin_map(cache->rq_vma->obj);
1017+ cache->rq_vma = NULL;
1018+
1019+ err = intel_gt_buffer_pool_mark_active(pool, rq);
1020+ if (err == 0) {
1021+ i915_vma_lock(batch);
1022+ err = i915_request_await_object(rq, batch->obj, false);
1023+ if (err == 0)
1024+ err = i915_vma_move_to_active(batch, rq, 0);
1025+ i915_vma_unlock(batch);
1026+ }
1027+ i915_vma_unpin(batch);
1028+ if (err)
1029+ goto out_pool;
1030+
1031+ cmd = i915_gem_object_pin_map(batch->obj,
1032+ cache->has_llc ?
1033+ I915_MAP_FORCE_WB :
1034+ I915_MAP_FORCE_WC);
1035+ if (IS_ERR(cmd)) {
1036+ err = PTR_ERR(cmd);
1037+ goto out_pool;
1038+ }
1039+
1040+ /* Return with batch mapping (cmd) still pinned */
1041+ cache->rq_cmd = cmd;
1042+ cache->rq_size = 0;
1043+ cache->rq_vma = batch;
1044+
1045+out_pool:
1046+ intel_gt_buffer_pool_put(pool);
1047+ return err;
1048+}
1049+
1050+static unsigned int reloc_bb_flags(const struct reloc_cache *cache)
1051+{
1052+ return cache->gen > 5 ? 0 : I915_DISPATCH_SECURE;
1053+}
1054+
1055+static int reloc_gpu_flush(struct reloc_cache *cache)
1056+{
1057+ struct i915_request *rq;
1058+ int err;
1059+
1060+ rq = fetch_and_zero(&cache->rq);
1061+ if (!rq)
1062+ return 0;
1063+
1064+ if (cache->rq_vma) {
1065+ struct drm_i915_gem_object *obj = cache->rq_vma->obj;
1066+
1067+ GEM_BUG_ON(cache->rq_size >= obj->base.size / sizeof(u32));
1068+ cache->rq_cmd[cache->rq_size++] = MI_BATCH_BUFFER_END;
1069+
1070+ __i915_gem_object_flush_map(obj,
1071+ 0, sizeof(u32) * cache->rq_size);
1072+ i915_gem_object_unpin_map(obj);
1073+ }
1074+
1075+ err = 0;
1076+ if (rq->engine->emit_init_breadcrumb)
1077+ err = rq->engine->emit_init_breadcrumb(rq);
1078+ if (!err)
1079+ err = rq->engine->emit_bb_start(rq,
1080+ rq->batch->node.start,
1081+ PAGE_SIZE,
1082+ reloc_bb_flags(cache));
1083+ if (err)
1084+ i915_request_set_error_once(rq, err);
1085+
1086+ intel_gt_chipset_flush(rq->engine->gt);
1087+ i915_request_add(rq);
1088+
1089+ return err;
9921090 }
9931091
9941092 static void reloc_cache_reset(struct reloc_cache *cache)
9951093 {
9961094 void *vaddr;
9971095
998− if (cache->rq)
999− reloc_gpu_flush(cache);
1000−
10011096 if (!cache->vaddr)
10021097 return;
10031098
@@ -1190,17 +1285,17 @@ static int reloc_move_to_gpu(struct i915_request *rq, struct i915_vma *vma) · 186 unmodified lines
11901285 }
11911286
11921287 static int __reloc_gpu_alloc(struct i915_execbuffer *eb,
1193− struct i915_vma *vma,
1288+ struct intel_engine_cs *engine,
11941289 unsigned int len)
11951290 {
11961291 struct reloc_cache *cache = &eb->reloc_cache;
1197− struct intel_engine_pool_node *pool;
1292+ struct intel_gt_buffer_pool_node *pool;
11981293 struct i915_request *rq;
11991294 struct i915_vma *batch;
12001295 u32 *cmd;
12011296 int err;
12021297
1203− pool = intel_engine_get_pool(eb->engine, PAGE_SIZE);
1298+ pool = intel_gt_get_buffer_pool(engine->gt, PAGE_SIZE);
12041299 if (IS_ERR(pool))
12051300 return PTR_ERR(pool);
12061301
@@ -1213,7 +1308,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, · 6 unmodified lines
12131308 goto out_pool;
12141309 }
12151310
1216− batch = i915_vma_instance(pool->obj, vma->vm, NULL);
1311+ batch = i915_vma_instance(pool->obj, eb->context->vm, NULL);
12171312 if (IS_ERR(batch)) {
12181313 err = PTR_ERR(batch);
12191314 goto err_unmap;
@@ -1223,26 +1318,32 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, · 3 unmodified lines
12231318 if (err)
12241319 goto err_unmap;
12251320
1226− rq = i915_request_create(eb->context);
1321+ if (engine == eb->context->engine) {
1322+ rq = i915_request_create(eb->context);
1323+ } else {
1324+ struct intel_context *ce;
1325+
1326+ ce = intel_context_create(engine);
1327+ if (IS_ERR(ce)) {
1328+ err = PTR_ERR(ce);
1329+ goto err_unpin;
1330+ }
1331+
1332+ i915_vm_put(ce->vm);
1333+ ce->vm = i915_vm_get(eb->context->vm);
1334+
1335+ rq = intel_context_create_request(ce);
1336+ intel_context_put(ce);
1337+ }
12271338 if (IS_ERR(rq)) {
12281339 err = PTR_ERR(rq);
12291340 goto err_unpin;
12301341 }
12311342
1232− err = intel_engine_pool_mark_active(pool, rq);
1233− if (err)
1234− goto err_request;
1235−
1236− err = reloc_move_to_gpu(rq, vma);
1343+ err = intel_gt_buffer_pool_mark_active(pool, rq);
12371344 if (err)
12381345 goto err_request;
12391346
1240− err = eb->engine->emit_bb_start(rq,
1241− batch->node.start, PAGE_SIZE,
1242− cache->gen > 5 ? 0 : I915_DISPATCH_SECURE);
1243− if (err)
1244− goto skip_request;
1245−
12461347 i915_vma_lock(batch);
12471348 err = i915_request_await_object(rq, batch->obj, false);
12481349 if (err == 0)
@@ -1257,6 +1358,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, · 8 unmodified lines
12571358 cache->rq = rq;
12581359 cache->rq_cmd = cmd;
12591360 cache->rq_size = 0;
1361+ cache->rq_vma = batch;
12601362
12611363 /* Return with batch mapping (cmd) still pinned */
12621364 goto out_pool;
@@ -1270,31 +1372,57 @@ err_unpin: · 7 unmodified lines
12701372 err_unmap:
12711373 i915_gem_object_unpin_map(pool->obj);
12721374 out_pool:
1273− intel_engine_pool_put(pool);
1375+ intel_gt_buffer_pool_put(pool);
12741376 return err;
12751377 }
12761378
1379+static bool reloc_can_use_engine(const struct intel_engine_cs *engine)
1380+{
1381+ return engine->class != VIDEO_DECODE_CLASS || !IS_GEN(engine->i915, 6);
1382+}
1383+
12771384 static u32 *reloc_gpu(struct i915_execbuffer *eb,
12781385 struct i915_vma *vma,
12791386 unsigned int len)
12801387 {
12811388 struct reloc_cache *cache = &eb->reloc_cache;
12821389 u32 *cmd;
1283−
1284− if (cache->rq_size > PAGE_SIZE/sizeof(u32) - (len + 1))
1285− reloc_gpu_flush(cache);
1390+ int err;
12861391
12871392 if (unlikely(!cache->rq)) {
1288− int err;
1393+ struct intel_engine_cs *engine = eb->engine;
12891394
1290− if (!intel_engine_can_store_dword(eb->engine))
1291− return ERR_PTR(-ENODEV);
1395+ if (!reloc_can_use_engine(engine)) {
1396+ engine = engine->gt->engine_class[COPY_ENGINE_CLASS][0];
1397+ if (!engine)
1398+ return ERR_PTR(-ENODEV);
1399+ }
12921400
1293− err = __reloc_gpu_alloc(eb, vma, len);
1401+ err = __reloc_gpu_alloc(eb, engine, len);
12941402 if (unlikely(err))
12951403 return ERR_PTR(err);
12961404 }
12971405
1406+ if (vma != cache->target) {
1407+ err = reloc_move_to_gpu(cache->rq, vma);
1408+ if (unlikely(err)) {
1409+ i915_request_set_error_once(cache->rq, err);
1410+ return ERR_PTR(err);
1411+ }
1412+
1413+ cache->target = vma;
1414+ }
1415+
1416+ if (unlikely(cache->rq_size + len >
1417+ PAGE_SIZE / sizeof(u32) - RELOC_TAIL)) {
1418+ err = reloc_gpu_chain(cache);
1419+ if (unlikely(err)) {
1420+ i915_request_set_error_once(cache->rq, err);
1421+ return ERR_PTR(err);
1422+ }
1423+ }
1424+
1425+ GEM_BUG_ON(cache->rq_size + len >= PAGE_SIZE / sizeof(u32));
12981426 cmd = cache->rq_cmd + cache->rq_size;
12991427 cache->rq_size += len;
13001428
@@ -1312,91 +1440,138 @@ static inline bool use_reloc_gpu(struct i915_vma *vma) · 11 unmodified lines
13121440 return !dma_resv_test_signaled_rcu(vma->resv, true);
13131441 }
13141442
1315−static u64
1316−relocate_entry(struct i915_vma *vma,
1317− const struct drm_i915_gem_relocation_entry *reloc,
1318− struct i915_execbuffer *eb,
1319− const struct i915_vma *target)
1443+static unsigned long vma_phys_addr(struct i915_vma *vma, u32 offset)
13201444 {
1321− u64 offset = reloc->offset;
1322− u64 target_offset = relocation_target(reloc, target);
1323− bool wide = eb->reloc_cache.use_64bit_reloc;
1324− void *vaddr;
1445+ struct page *page;
1446+ unsigned long addr;
13251447
1326− if (!eb->reloc_cache.vaddr && use_reloc_gpu(vma)) {
1327− const unsigned int gen = eb->reloc_cache.gen;
1328− unsigned int len;
1329− u32 *batch;
1330− u64 addr;
1448+ GEM_BUG_ON(vma->pages != vma->obj->mm.pages);
13311449
1332− if (wide)
1333− len = offset & 7 ? 8 : 5;
1334− else if (gen >= 4)
1335− len = 4;
1336− else
1337− len = 3;
1450+ page = i915_gem_object_get_page(vma->obj, offset >> PAGE_SHIFT);
1451+ addr = PFN_PHYS(page_to_pfn(page));
1452+ GEM_BUG_ON(overflows_type(addr, u32)); /* expected dma32 */
13381453
1339− batch = reloc_gpu(eb, vma, len);
1340− if (IS_ERR(batch))
1341− goto repeat;
1454+ return addr + offset_in_page(offset);
1455+}
1456+
1457+static bool __reloc_entry_gpu(struct i915_execbuffer *eb,
1458+ struct i915_vma *vma,
1459+ u64 offset,
1460+ u64 target_addr)
1461+{
1462+ const unsigned int gen = eb->reloc_cache.gen;
1463+ unsigned int len;
1464+ u32 *batch;
1465+ u64 addr;
1466+
1467+ if (gen >= 8)
1468+ len = offset & 7 ? 8 : 5;
1469+ else if (gen >= 4)
1470+ len = 4;
1471+ else
1472+ len = 3;
1473+
1474+ batch = reloc_gpu(eb, vma, len);
1475+ if (IS_ERR(batch))
1476+ return false;
1477+
1478+ addr = gen8_canonical_addr(vma->node.start + offset);
1479+ if (gen >= 8) {
1480+ if (offset & 7) {
1481+ *batch++ = MI_STORE_DWORD_IMM_GEN4;
1482+ *batch++ = lower_32_bits(addr);
1483+ *batch++ = upper_32_bits(addr);
1484+ *batch++ = lower_32_bits(target_addr);
1485+
1486+ addr = gen8_canonical_addr(addr + 4);
13421487
1343− addr = gen8_canonical_addr(vma->node.start + offset);
1344− if (wide) {
1345− if (offset & 7) {
1346− *batch++ = MI_STORE_DWORD_IMM_GEN4;
1347− *batch++ = lower_32_bits(addr);
1348− *batch++ = upper_32_bits(addr);
1349− *batch++ = lower_32_bits(target_offset);
1350−
1351− addr = gen8_canonical_addr(addr + 4);
1352−
1353− *batch++ = MI_STORE_DWORD_IMM_GEN4;
1354− *batch++ = lower_32_bits(addr);
1355− *batch++ = upper_32_bits(addr);
1356− *batch++ = upper_32_bits(target_offset);
1357− } else {
1358− *batch++ = (MI_STORE_DWORD_IMM_GEN4 | (1 << 21)) + 1;
1359− *batch++ = lower_32_bits(addr);
1360− *batch++ = upper_32_bits(addr);
1361− *batch++ = lower_32_bits(target_offset);
1362− *batch++ = upper_32_bits(target_offset);
1363− }
1364− } else if (gen >= 6) {
13651488 *batch++ = MI_STORE_DWORD_IMM_GEN4;
1366− *batch++ = 0;
1367− *batch++ = addr;
1368− *batch++ = target_offset;
1369− } else if (gen >= 4) {
1370− *batch++ = MI_STORE_DWORD_IMM_GEN4 | MI_USE_GGTT;
1371− *batch++ = 0;
1372− *batch++ = addr;
1373− *batch++ = target_offset;
1489+ *batch++ = lower_32_bits(addr);
1490+ *batch++ = upper_32_bits(addr);
1491+ *batch++ = upper_32_bits(target_addr);
13741492 } else {
1375− *batch++ = MI_STORE_DWORD_IMM | MI_MEM_VIRTUAL;
1376− *batch++ = addr;
1377− *batch++ = target_offset;
1493+ *batch++ = (MI_STORE_DWORD_IMM_GEN4 | (1 << 21)) + 1;
1494+ *batch++ = lower_32_bits(addr);
1495+ *batch++ = upper_32_bits(addr);
1496+ *batch++ = lower_32_bits(target_addr);
1497+ *batch++ = upper_32_bits(target_addr);
13781498 }
1379−
1380− goto out;
1499+ } else if (gen >= 6) {
1500+ *batch++ = MI_STORE_DWORD_IMM_GEN4;
1501+ *batch++ = 0;
1502+ *batch++ = addr;
1503+ *batch++ = target_addr;
1504+ } else if (IS_I965G(eb->i915)) {
1505+ *batch++ = MI_STORE_DWORD_IMM_GEN4;
1506+ *batch++ = 0;
1507+ *batch++ = vma_phys_addr(vma, offset);
1508+ *batch++ = target_addr;
1509+ } else if (gen >= 4) {
1510+ *batch++ = MI_STORE_DWORD_IMM_GEN4 | MI_USE_GGTT;
1511+ *batch++ = 0;
1512+ *batch++ = addr;
1513+ *batch++ = target_addr;
1514+ } else if (gen >= 3 &&
1515+ !(IS_I915G(eb->i915) || IS_I915GM(eb->i915))) {
1516+ *batch++ = MI_STORE_DWORD_IMM | MI_MEM_VIRTUAL;
1517+ *batch++ = addr;
1518+ *batch++ = target_addr;
1519+ } else {
1520+ *batch++ = MI_STORE_DWORD_IMM;
1521+ *batch++ = vma_phys_addr(vma, offset);
1522+ *batch++ = target_addr;
13811523 }
13821524
1525+ return true;
1526+}
1527+
1528+static bool reloc_entry_gpu(struct i915_execbuffer *eb,
1529+ struct i915_vma *vma,
1530+ u64 offset,
1531+ u64 target_addr)
1532+{
1533+ if (eb->reloc_cache.vaddr)
1534+ return false;
1535+
1536+ if (!use_reloc_gpu(vma))
1537+ return false;
1538+
1539+ return __reloc_entry_gpu(eb, vma, offset, target_addr);
1540+}
1541+
1542+static u64
1543+relocate_entry(struct i915_vma *vma,
1544+ const struct drm_i915_gem_relocation_entry *reloc,
1545+ struct i915_execbuffer *eb,
1546+ const struct i915_vma *target)
1547+{
1548+ u64 target_addr = relocation_target(reloc, target);
1549+ u64 offset = reloc->offset;
1550+
1551+ if (!reloc_entry_gpu(eb, vma, offset, target_addr)) {
1552+ bool wide = eb->reloc_cache.use_64bit_reloc;
1553+ void *vaddr;
1554+
13831555 repeat:
1384− vaddr = reloc_vaddr(vma->obj, &eb->reloc_cache, offset >> PAGE_SHIFT);
1385− if (IS_ERR(vaddr))
1386− return PTR_ERR(vaddr);
1556+ vaddr = reloc_vaddr(vma->obj,
1557+ &eb->reloc_cache,
1558+ offset >> PAGE_SHIFT);
1559+ if (IS_ERR(vaddr))
1560+ return PTR_ERR(vaddr);
13871561
1388− clflush_write32(vaddr + offset_in_page(offset),
1389− lower_32_bits(target_offset),
1390− eb->reloc_cache.vaddr);
1562+ GEM_BUG_ON(!IS_ALIGNED(offset, sizeof(u32)));
1563+ clflush_write32(vaddr + offset_in_page(offset),
1564+ lower_32_bits(target_addr),
1565+ eb->reloc_cache.vaddr);
13911566
1392− if (wide) {
1393− offset += sizeof(u32);
1394− target_offset >>= 32;
1395− wide = false;
1396− goto repeat;
1567+ if (wide) {
1568+ offset += sizeof(u32);
1569+ target_addr >>= 32;
1570+ wide = false;
1571+ goto repeat;
1572+ }
13971573 }
13981574
1399−out:
14001575 return target->node.start | UPDATE;
14011576 }
14021577
@@ -1596,15 +1771,20 @@ static int eb_relocate(struct i915_execbuffer *eb) · 193 unmodified lines
15961771 /* The objects are in their final locations, apply the relocations. */
15971772 if (eb->args->flags & __EXEC_HAS_RELOC) {
15981773 struct eb_vma *ev;
1774+ int flush;
15991775
16001776 list_for_each_entry(ev, &eb->relocs, reloc_link) {
16011777 err = eb_relocate_vma(eb, ev);
16021778 if (err)
1603− return err;
1779+ break;
16041780 }
1781+
1782+ flush = reloc_gpu_flush(&eb->reloc_cache);
1783+ if (!err)
1784+ err = flush;
16051785 }
16061786
1607− return 0;
1787+ return err;
16081788 }
16091789
16101790 static int eb_move_to_gpu(struct i915_execbuffer *eb)
@@ -1887,7 +2067,7 @@ err_free: · 276 unmodified lines
18872067 static int eb_parse(struct i915_execbuffer *eb)
18882068 {
18892069 struct drm_i915_private *i915 = eb->i915;
1890− struct intel_engine_pool_node *pool;
2070+ struct intel_gt_buffer_pool_node *pool;
18912071 struct i915_vma *shadow, *trampoline;
18922072 unsigned int len;
18932073 int err;
@@ -1910,7 +2090,7 @@ static int eb_parse(struct i915_execbuffer *eb) · 16 unmodified lines
19102090 len += I915_CMD_PARSER_TRAMPOLINE_SIZE;
19112091 }
19122092
1913− pool = intel_engine_get_pool(eb->engine, len);
2093+ pool = intel_gt_get_buffer_pool(eb->engine->gt, len);
19142094 if (IS_ERR(pool))
19152095 return PTR_ERR(pool);
19162096
@@ -1958,7 +2138,7 @@ err_trampoline: · 41 unmodified lines
19582138 err_shadow:
19592139 i915_vma_unpin(shadow);
19602140 err:
1961− intel_engine_pool_put(pool);
2141+ intel_gt_buffer_pool_put(pool);
19622142 return err;
19632143 }
19642144
@@ -2402,30 +2582,6 @@ static void eb_request_add(struct i915_execbuffer *eb) · 437 unmodified lines
24022582 /* Check that the context wasn't destroyed before submission */
24032583 if (likely(!intel_context_is_closed(eb->context))) {
24042584 attr = eb->gem_context->sched;
2405−
2406− /*
2407− * Boost actual workloads past semaphores!
2408− *
2409− * With semaphores we spin on one engine waiting for another,
2410− * simply to reduce the latency of starting our work when
2411− * the signaler completes. However, if there is any other
2412− * work that we could be doing on this engine instead, that
2413− * is better utilisation and will reduce the overall duration
2414− * of the current work. To avoid PI boosting a semaphore
2415− * far in the distance past over useful work, we keep a history
2416− * of any semaphore use along our dependency chain.
2417− */
2418− if (!(rq->sched.flags & I915_SCHED_HAS_SEMAPHORE_CHAIN))
2419− attr.priority |= I915_PRIORITY_NOSEMAPHORE;
2420−
2421− /*
2422− * Boost priorities to new clients (new request flows).
2423− *
2424− * Allow interactive/synchronous clients to jump ahead of
2425− * the bulk clients. (FQ_CODEL)
2426− */
2427− if (list_empty(&rq->sched.signalers_list))
2428− attr.priority |= I915_PRIORITY_WAIT;
24292585 } else {
24302586 /* Serialise with context_close via the add_to_timeline */
24312587 i915_request_set_error_once(rq, -ENOENT);
@@ -2451,7 +2607,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 19 unmodified lines
24512607 struct drm_i915_private *i915 = to_i915(dev);
24522608 struct i915_execbuffer eb;
24532609 struct dma_fence *in_fence = NULL;
2454− struct dma_fence *exec_fence = NULL;
24552610 struct sync_file *out_fence = NULL;
24562611 struct i915_vma *batch;
24572612 int out_fence_fd = -1;
@@ -2494,30 +2649,22 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 36 unmodified lines
24942649 if (args->flags & I915_EXEC_IS_PINNED)
24952650 eb.batch_flags |= I915_DISPATCH_PINNED;
24962651
2497− if (args->flags & I915_EXEC_FENCE_IN) {
2652+#define IN_FENCES (I915_EXEC_FENCE_IN | I915_EXEC_FENCE_SUBMIT)
2653+ if (args->flags & IN_FENCES) {
2654+ if ((args->flags & IN_FENCES) == IN_FENCES)
2655+ return -EINVAL;
2656+
24982657 in_fence = sync_file_get_fence(lower_32_bits(args->rsvd2));
24992658 if (!in_fence)
25002659 return -EINVAL;
25012660 }
2502−
2503− if (args->flags & I915_EXEC_FENCE_SUBMIT) {
2504− if (in_fence) {
2505− err = -EINVAL;
2506− goto err_in_fence;
2507− }
2508−
2509− exec_fence = sync_file_get_fence(lower_32_bits(args->rsvd2));
2510− if (!exec_fence) {
2511− err = -EINVAL;
2512− goto err_in_fence;
2513− }
2514− }
2661+#undef IN_FENCES
25152662
25162663 if (args->flags & I915_EXEC_FENCE_OUT) {
25172664 out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
25182665 if (out_fence_fd < 0) {
25192666 err = out_fence_fd;
2520− goto err_exec_fence;
2667+ goto err_in_fence;
25212668 }
25222669 }
25232670
@@ -2608,14 +2755,13 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 84 unmodified lines
26082755 }
26092756
26102757 if (in_fence) {
2611− err = i915_request_await_dma_fence(eb.request, in_fence);
2612− if (err < 0)
2613− goto err_request;
2614− }
2615−
2616− if (exec_fence) {
2617− err = i915_request_await_execution(eb.request, exec_fence,
2618− eb.engine->bond_execute);
2758+ if (args->flags & I915_EXEC_FENCE_SUBMIT)
2759+ err = i915_request_await_execution(eb.request,
2760+ in_fence,
2761+ eb.engine->bond_execute);
2762+ else
2763+ err = i915_request_await_dma_fence(eb.request,
2764+ in_fence);
26192765 if (err < 0)
26202766 goto err_request;
26212767 }
@@ -2643,7 +2789,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 21 unmodified lines
26432789 */
26442790 eb.request->batch = batch;
26452791 if (batch->private)
2646− intel_engine_pool_mark_active(batch->private, eb.request);
2792+ intel_gt_buffer_pool_mark_active(batch->private, eb.request);
26472793
26482794 trace_i915_request_queue(eb.request, eb.batch_flags);
26492795 err = eb_submit(&eb, batch);
@@ -2672,7 +2818,7 @@ err_batch_unpin: · 22 unmodified lines
26722818 i915_vma_unpin(batch);
26732819 err_parse:
26742820 if (batch->private)
2675− intel_engine_pool_put(batch->private);
2821+ intel_gt_buffer_pool_put(batch->private);
26762822 err_vma:
26772823 if (eb.trampoline)
26782824 i915_vma_unpin(eb.trampoline);
@@ -2684,8 +2830,6 @@ err_destroy: · 5 unmodified lines
26842830 err_out_fence:
26852831 if (out_fence_fd != -1)
26862832 put_unused_fd(out_fence_fd);
2687−err_exec_fence:
2688− dma_fence_put(exec_fence);
26892833 err_in_fence:
26902834 dma_fence_put(in_fence);
26912835 return err;
@@ -2894,3 +3038,7 @@ end:; · 202 unmodified lines
28943038 kvfree(exec2_list);
28953039 return err;
28963040 }
3041+
3042+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
3043+#include "selftests/i915_gem_execbuffer.c"
3044+#endif
drivers/gpu/drm/i915/gem/i915_gem_fence.c+2−2
@@ -72,8 +72,8 @@ i915_gem_object_lock_fence(struct drm_i915_gem_object *obj) · 71 unmodified lines
7272 0, 0);
7373
7474 if (i915_sw_fence_await_reservation(&stub->chain,
75− obj->base.resv, NULL,
76− true, I915_FENCE_TIMEOUT,
75+ obj->base.resv, NULL, true,
76+ i915_fence_timeout(to_i915(obj->base.dev)),
7777 I915_FENCE_GFP) < 0)
7878 goto err;
7979
drivers/gpu/drm/i915/gem/i915_gem_mman.c+34−0
@@ -396,6 +396,38 @@ err: · 395 unmodified lines
396396 return i915_error_to_vmf_fault(ret);
397397 }
398398
399+static int
400+vm_access(struct vm_area_struct *area, unsigned long addr,
401+ void *buf, int len, int write)
402+{
403+ struct i915_mmap_offset *mmo = area->vm_private_data;
404+ struct drm_i915_gem_object *obj = mmo->obj;
405+ void *vaddr;
406+
407+ if (i915_gem_object_is_readonly(obj) && write)
408+ return -EACCES;
409+
410+ addr -= area->vm_start;
411+ if (addr >= obj->base.size)
412+ return -EINVAL;
413+
414+ /* As this is primarily for debugging, let's focus on simplicity */
415+ vaddr = i915_gem_object_pin_map(obj, I915_MAP_FORCE_WC);
416+ if (IS_ERR(vaddr))
417+ return PTR_ERR(vaddr);
418+
419+ if (write) {
420+ memcpy(vaddr + addr, buf, len);
421+ __i915_gem_object_flush_map(obj, addr, len);
422+ } else {
423+ memcpy(buf, vaddr + addr, len);
424+ }
425+
426+ i915_gem_object_unpin_map(obj);
427+
428+ return len;
429+}
430+
399431 void __i915_gem_object_release_mmap_gtt(struct drm_i915_gem_object *obj)
400432 {
401433 struct i915_vma *vma;
@@ -745,12 +777,14 @@ static void vm_close(struct vm_area_struct *vma) · 343 unmodified lines
745777
746778 static const struct vm_operations_struct vm_ops_gtt = {
747779 .fault = vm_fault_gtt,
780+ .access = vm_access,
748781 .open = vm_open,
749782 .close = vm_close,
750783 };
751784
752785 static const struct vm_operations_struct vm_ops_cpu = {
753786 .fault = vm_fault_cpu,
787+ .access = vm_access,
754788 .open = vm_open,
755789 .close = vm_close,
756790 };
drivers/gpu/drm/i915/gem/i915_gem_object.c+0−3
@@ -162,9 +162,7 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, · 161 unmodified lines
162162 struct llist_node *freed)
163163 {
164164 struct drm_i915_gem_object *obj, *on;
165− intel_wakeref_t wakeref;
166165
167− wakeref = intel_runtime_pm_get(&i915->runtime_pm);
168166 llist_for_each_entry_safe(obj, on, freed, freed) {
169167 struct i915_mmap_offset *mmo, *mn;
170168
@@ -224,7 +222,6 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, · 53 unmodified lines
224222 call_rcu(&obj->rcu, __i915_gem_free_object_rcu);
225223 cond_resched();
226224 }
227− intel_runtime_pm_put(&i915->runtime_pm, wakeref);
228225 }
229226
230227 void i915_gem_flush_free_objects(struct drm_i915_private *i915)
drivers/gpu/drm/i915/gem/i915_gem_object_blt.c+15−11
@@ -6,8 +6,8 @@ · 5 unmodified lines
66 #include "i915_drv.h"
77 #include "gt/intel_context.h"
88 #include "gt/intel_engine_pm.h"
9−#include "gt/intel_engine_pool.h"
109 #include "gt/intel_gt.h"
10+#include "gt/intel_gt_buffer_pool.h"
1111 #include "gt/intel_ring.h"
1212 #include "i915_gem_clflush.h"
1313 #include "i915_gem_object_blt.h"
@@ -18,7 +18,7 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce, · 4 unmodified lines
1818 {
1919 struct drm_i915_private *i915 = ce->vm->i915;
2020 const u32 block_size = SZ_8M; /* ~1ms at 8GiB/s preemption delay */
21− struct intel_engine_pool_node *pool;
21+ struct intel_gt_buffer_pool_node *pool;
2222 struct i915_vma *batch;
2323 u64 offset;
2424 u64 count;
@@ -33,7 +33,7 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce, · 8 unmodified lines
3333 count = div_u64(round_up(vma->size, block_size), block_size);
3434 size = (1 + 8 * count) * sizeof(u32);
3535 size = round_up(size, PAGE_SIZE);
36− pool = intel_engine_get_pool(ce->engine, size);
36+ pool = intel_gt_get_buffer_pool(ce->engine->gt, size);
3737 if (IS_ERR(pool)) {
3838 err = PTR_ERR(pool);
3939 goto out_pm;
@@ -78,10 +78,12 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce, · 38 unmodified lines
7878 } while (rem);
7979
8080 *cmd = MI_BATCH_BUFFER_END;
81− intel_gt_chipset_flush(ce->vm->gt);
8281
82+ i915_gem_object_flush_map(pool->obj);
8383 i915_gem_object_unpin_map(pool->obj);
8484
85+ intel_gt_chipset_flush(ce->vm->gt);
86+
8587 batch = i915_vma_instance(pool->obj, ce->vm, NULL);
8688 if (IS_ERR(batch)) {
8789 err = PTR_ERR(batch);
@@ -96,7 +98,7 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce, · 8 unmodified lines
9698 return batch;
9799
98100 out_put:
99− intel_engine_pool_put(pool);
101+ intel_gt_buffer_pool_put(pool);
100102 out_pm:
101103 intel_engine_pm_put(ce->engine);
102104 return ERR_PTR(err);
@@ -114,13 +116,13 @@ int intel_emit_vma_mark_active(struct i915_vma *vma, struct i915_request *rq) · 11 unmodified lines
114116 if (unlikely(err))
115117 return err;
116118
117− return intel_engine_pool_mark_active(vma->private, rq);
119+ return intel_gt_buffer_pool_mark_active(vma->private, rq);
118120 }
119121
120122 void intel_emit_vma_release(struct intel_context *ce, struct i915_vma *vma)
121123 {
122124 i915_vma_unpin(vma);
123− intel_engine_pool_put(vma->private);
125+ intel_gt_buffer_pool_put(vma->private);
124126 intel_engine_pm_put(ce->engine);
125127 }
126128
@@ -213,7 +215,7 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce, · 86 unmodified lines
213215 {
214216 struct drm_i915_private *i915 = ce->vm->i915;
215217 const u32 block_size = SZ_8M; /* ~1ms at 8GiB/s preemption delay */
216− struct intel_engine_pool_node *pool;
218+ struct intel_gt_buffer_pool_node *pool;
217219 struct i915_vma *batch;
218220 u64 src_offset, dst_offset;
219221 u64 count, rem;
@@ -228,7 +230,7 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce, · 8 unmodified lines
228230 count = div_u64(round_up(dst->size, block_size), block_size);
229231 size = (1 + 11 * count) * sizeof(u32);
230232 size = round_up(size, PAGE_SIZE);
231− pool = intel_engine_get_pool(ce->engine, size);
233+ pool = intel_gt_get_buffer_pool(ce->engine->gt, size);
232234 if (IS_ERR(pool)) {
233235 err = PTR_ERR(pool);
234236 goto out_pm;
@@ -289,10 +291,12 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce, · 54 unmodified lines
289291 } while (rem);
290292
291293 *cmd = MI_BATCH_BUFFER_END;
292− intel_gt_chipset_flush(ce->vm->gt);
293294
295+ i915_gem_object_flush_map(pool->obj);
294296 i915_gem_object_unpin_map(pool->obj);
295297
298+ intel_gt_chipset_flush(ce->vm->gt);
299+
296300 batch = i915_vma_instance(pool->obj, ce->vm, NULL);
297301 if (IS_ERR(batch)) {
298302 err = PTR_ERR(batch);
@@ -307,7 +311,7 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce, · 8 unmodified lines
307311 return batch;
308312
309313 out_put:
310− intel_engine_pool_put(pool);
314+ intel_gt_buffer_pool_put(pool);
311315 out_pm:
312316 intel_engine_pm_put(ce->engine);
313317 return ERR_PTR(err);
drivers/gpu/drm/i915/gem/i915_gem_object_blt.h+0−1
@@ -10,7 +10,6 @@ · 9 unmodified lines
1010
1111 #include "gt/intel_context.h"
1212 #include "gt/intel_engine_pm.h"
13−#include "gt/intel_engine_pool.h"
1413 #include "i915_vma.h"
1514
1615 struct drm_i915_gem_object;
drivers/gpu/drm/i915/gem/i915_gem_pages.c+1−0
@@ -391,6 +391,7 @@ void __i915_gem_object_flush_map(struct drm_i915_gem_object *obj, · 390 unmodified lines
391391 GEM_BUG_ON(range_overflows_t(typeof(obj->base.size),
392392 offset, size, obj->base.size));
393393
394+ wmb(); /* let all previous writes be visible to coherent partners */
394395 obj->mm.dirty = true;
395396
396397 if (obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE)
drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c+1−30
@@ -302,35 +302,6 @@ static void fill_scratch(struct tiled_blits *t, u32 *vaddr, u32 val) · 301 unmodified lines
302302 i915_gem_object_flush_map(t->scratch.vma->obj);
303303 }
304304
305−static void hexdump(const void *buf, size_t len)
306−{
307− const size_t rowsize = 8 * sizeof(u32);
308− const void *prev = NULL;
309− bool skip = false;
310− size_t pos;
311−
312− for (pos = 0; pos < len; pos += rowsize) {
313− char line[128];
314−
315− if (prev && !memcmp(prev, buf + pos, rowsize)) {
316− if (!skip) {
317− pr_info("*\n");
318− skip = true;
319− }
320− continue;
321− }
322−
323− WARN_ON_ONCE(hex_dump_to_buffer(buf + pos, len - pos,
324− rowsize, sizeof(u32),
325− line, sizeof(line),
326− false) >= sizeof(line));
327− pr_info("[%04zx] %s\n", pos, line);
328−
329− prev = buf + pos;
330− skip = false;
331− }
332−}
333−
334305 static u64 swizzle_bit(unsigned int bit, u64 offset)
335306 {
336307 return (offset & BIT_ULL(bit)) >> (bit - 6);
@@ -426,7 +397,7 @@ static int verify_buffer(const struct tiled_blits *t, · 89 unmodified lines
426397 pr_err("Invalid %s tiling detected at (%d, %d), start_val %x\n",
427398 repr_tiling(buf->tiling),
428399 x, y, buf->start_val);
429− hexdump(vaddr, 4096);
400+ igt_hexdump(vaddr, 4096);
430401 }
431402
432403 i915_gem_object_unpin_map(buf->vma->obj);
drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c+2−0
@@ -158,6 +158,8 @@ static int wc_set(struct context *ctx, unsigned long offset, u32 v) · 157 unmodified lines
158158 return PTR_ERR(map);
159159
160160 map[offset / sizeof(*map)] = v;
161+
162+ __i915_gem_object_flush_map(ctx->obj, offset, sizeof(*map));
161163 i915_gem_object_unpin_map(ctx->obj);
162164
163165 return 0;
drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c+32−17
@@ -972,12 +972,6 @@ emit_rpcs_query(struct drm_i915_gem_object *obj, · 971 unmodified lines
972972 goto err_batch;
973973 }
974974
975− err = rq->engine->emit_bb_start(rq,
976− batch->node.start, batch->node.size,
977− 0);
978− if (err)
979− goto err_request;
980−
981975 i915_vma_lock(batch);
982976 err = i915_request_await_object(rq, batch->obj, false);
983977 if (err == 0)
@@ -994,6 +988,18 @@ emit_rpcs_query(struct drm_i915_gem_object *obj, · 10 unmodified lines
994988 if (err)
995989 goto skip_request;
996990
991+ if (rq->engine->emit_init_breadcrumb) {
992+ err = rq->engine->emit_init_breadcrumb(rq);
993+ if (err)
994+ goto skip_request;
995+ }
996+
997+ err = rq->engine->emit_bb_start(rq,
998+ batch->node.start, batch->node.size,
999+ 0);
1000+ if (err)
1001+ goto skip_request;
1002+
9971003 i915_vma_unpin_and_release(&batch, 0);
9981004 i915_vma_unpin(vma);
9991005
@@ -1005,7 +1011,6 @@ emit_rpcs_query(struct drm_i915_gem_object *obj, · 5 unmodified lines
10051011
10061012 skip_request:
10071013 i915_request_set_error_once(rq, err);
1008−err_request:
10091014 i915_request_add(rq);
10101015 err_batch:
10111016 i915_vma_unpin_and_release(&batch, 0);
@@ -1541,10 +1546,6 @@ static int write_to_scratch(struct i915_gem_context *ctx, · 529 unmodified lines
15411546 goto err_unpin;
15421547 }
15431548
1544− err = engine->emit_bb_start(rq, vma->node.start, vma->node.size, 0);
1545− if (err)
1546− goto err_request;
1547−
15481549 i915_vma_lock(vma);
15491550 err = i915_request_await_object(rq, vma->obj, false);
15501551 if (err == 0)
@@ -1553,6 +1554,16 @@ static int write_to_scratch(struct i915_gem_context *ctx, · 2 unmodified lines
15531554 if (err)
15541555 goto skip_request;
15551556
1557+ if (rq->engine->emit_init_breadcrumb) {
1558+ err = rq->engine->emit_init_breadcrumb(rq);
1559+ if (err)
1560+ goto skip_request;
1561+ }
1562+
1563+ err = engine->emit_bb_start(rq, vma->node.start, vma->node.size, 0);
1564+ if (err)
1565+ goto skip_request;
1566+
15561567 i915_vma_unpin(vma);
15571568
15581569 i915_request_add(rq);
@@ -1560,7 +1571,6 @@ static int write_to_scratch(struct i915_gem_context *ctx, · 1 unmodified line
15601571 goto out_vm;
15611572 skip_request:
15621573 i915_request_set_error_once(rq, err);
1563−err_request:
15641574 i915_request_add(rq);
15651575 err_unpin:
15661576 i915_vma_unpin(vma);
@@ -1674,10 +1684,6 @@ static int read_from_scratch(struct i915_gem_context *ctx, · 107 unmodified lines
16741684 goto err_unpin;
16751685 }
16761686
1677− err = engine->emit_bb_start(rq, vma->node.start, vma->node.size, flags);
1678− if (err)
1679− goto err_request;
1680−
16811687 i915_vma_lock(vma);
16821688 err = i915_request_await_object(rq, vma->obj, true);
16831689 if (err == 0)
@@ -1686,6 +1692,16 @@ static int read_from_scratch(struct i915_gem_context *ctx, · 2 unmodified lines
16861692 if (err)
16871693 goto skip_request;
16881694
1695+ if (rq->engine->emit_init_breadcrumb) {
1696+ err = rq->engine->emit_init_breadcrumb(rq);
1697+ if (err)
1698+ goto skip_request;
1699+ }
1700+
1701+ err = engine->emit_bb_start(rq, vma->node.start, vma->node.size, flags);
1702+ if (err)
1703+ goto skip_request;
1704+
16891705 i915_vma_unpin(vma);
16901706
16911707 i915_request_add(rq);
@@ -1708,7 +1724,6 @@ static int read_from_scratch(struct i915_gem_context *ctx, · 16 unmodified lines
17081724 goto out_vm;
17091725 skip_request:
17101726 i915_request_set_error_once(rq, err);
1711−err_request:
17121727 i915_request_add(rq);
17131728 err_unpin:
17141729 i915_vma_unpin(vma);
drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c+171−0
@@ -0,0 +1,171 @@
1+// SPDX-License-Identifier: MIT
2+/*
3+ * Copyright © 2020 Intel Corporation
4+ */
5+
6+#include "i915_selftest.h"
7+
8+#include "gt/intel_engine_pm.h"
9+#include "selftests/igt_flush_test.h"
10+
11+static u64 read_reloc(const u32 *map, int x, const u64 mask)
12+{
13+ u64 reloc;
14+
15+ memcpy(&reloc, &map[x], sizeof(reloc));
16+ return reloc & mask;
17+}
18+
19+static int __igt_gpu_reloc(struct i915_execbuffer *eb,
20+ struct drm_i915_gem_object *obj)
21+{
22+ const unsigned int offsets[] = { 8, 3, 0 };
23+ const u64 mask =
24+ GENMASK_ULL(eb->reloc_cache.use_64bit_reloc ? 63 : 31, 0);
25+ const u32 *map = page_mask_bits(obj->mm.mapping);
26+ struct i915_request *rq;
27+ struct i915_vma *vma;
28+ int err;
29+ int i;
30+
31+ vma = i915_vma_instance(obj, eb->context->vm, NULL);
32+ if (IS_ERR(vma))
33+ return PTR_ERR(vma);
34+
35+ err = i915_vma_pin(vma, 0, 0, PIN_USER | PIN_HIGH);
36+ if (err)
37+ return err;
38+
39+ /* 8-Byte aligned */
40+ if (!__reloc_entry_gpu(eb, vma,
41+ offsets[0] * sizeof(u32),
42+ 0)) {
43+ err = -EIO;
44+ goto unpin_vma;
45+ }
46+
47+ /* !8-Byte aligned */
48+ if (!__reloc_entry_gpu(eb, vma,
49+ offsets[1] * sizeof(u32),
50+ 1)) {
51+ err = -EIO;
52+ goto unpin_vma;
53+ }
54+
55+ /* Skip to the end of the cmd page */
56+ i = PAGE_SIZE / sizeof(u32) - RELOC_TAIL - 1;
57+ i -= eb->reloc_cache.rq_size;
58+ memset32(eb->reloc_cache.rq_cmd + eb->reloc_cache.rq_size,
59+ MI_NOOP, i);
60+ eb->reloc_cache.rq_size += i;
61+
62+ /* Force batch chaining */
63+ if (!__reloc_entry_gpu(eb, vma,
64+ offsets[2] * sizeof(u32),
65+ 2)) {
66+ err = -EIO;
67+ goto unpin_vma;
68+ }
69+
70+ GEM_BUG_ON(!eb->reloc_cache.rq);
71+ rq = i915_request_get(eb->reloc_cache.rq);
72+ err = reloc_gpu_flush(&eb->reloc_cache);
73+ if (err)
74+ goto put_rq;
75+ GEM_BUG_ON(eb->reloc_cache.rq);
76+
77+ err = i915_gem_object_wait(obj, I915_WAIT_INTERRUPTIBLE, HZ / 2);
78+ if (err) {
79+ intel_gt_set_wedged(eb->engine->gt);
80+ goto put_rq;
81+ }
82+
83+ if (!i915_request_completed(rq)) {
84+ pr_err("%s: did not wait for relocations!\n", eb->engine->name);
85+ err = -EINVAL;
86+ goto put_rq;
87+ }
88+
89+ for (i = 0; i < ARRAY_SIZE(offsets); i++) {
90+ u64 reloc = read_reloc(map, offsets[i], mask);
91+
92+ if (reloc != i) {
93+ pr_err("%s[%d]: map[%d] %llx != %x\n",
94+ eb->engine->name, i, offsets[i], reloc, i);
95+ err = -EINVAL;
96+ }
97+ }
98+ if (err)
99+ igt_hexdump(map, 4096);
100+
101+put_rq:
102+ i915_request_put(rq);
103+unpin_vma:
104+ i915_vma_unpin(vma);
105+ return err;
106+}
107+
108+static int igt_gpu_reloc(void *arg)
109+{
110+ struct i915_execbuffer eb;
111+ struct drm_i915_gem_object *scratch;
112+ int err = 0;
113+ u32 *map;
114+
115+ eb.i915 = arg;
116+
117+ scratch = i915_gem_object_create_internal(eb.i915, 4096);
118+ if (IS_ERR(scratch))
119+ return PTR_ERR(scratch);
120+
121+ map = i915_gem_object_pin_map(scratch, I915_MAP_WC);
122+ if (IS_ERR(map)) {
123+ err = PTR_ERR(map);
124+ goto err_scratch;
125+ }
126+
127+ for_each_uabi_engine(eb.engine, eb.i915) {
128+ reloc_cache_init(&eb.reloc_cache, eb.i915);
129+ memset(map, POISON_INUSE, 4096);
130+
131+ intel_engine_pm_get(eb.engine);
132+ eb.context = intel_context_create(eb.engine);
133+ if (IS_ERR(eb.context)) {
134+ err = PTR_ERR(eb.context);
135+ goto err_pm;
136+ }
137+
138+ err = intel_context_pin(eb.context);
139+ if (err)
140+ goto err_put;
141+
142+ err = __igt_gpu_reloc(&eb, scratch);
143+
144+ intel_context_unpin(eb.context);
145+err_put:
146+ intel_context_put(eb.context);
147+err_pm:
148+ intel_engine_pm_put(eb.engine);
149+ if (err)
150+ break;
151+ }
152+
153+ if (igt_flush_test(eb.i915))
154+ err = -EIO;
155+
156+err_scratch:
157+ i915_gem_object_put(scratch);
158+ return err;
159+}
160+
161+int i915_gem_execbuffer_live_selftests(struct drm_i915_private *i915)
162+{
163+ static const struct i915_subtest tests[] = {
164+ SUBTEST(igt_gpu_reloc),
165+ };
166+
167+ if (intel_gt_is_wedged(&i915->gt))
168+ return 0;
169+
170+ return i915_live_subtests(tests, i915);
171+}
drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c+124−0
@@ -952,6 +952,129 @@ static int igt_mmap(void *arg) · 951 unmodified lines
952952 return 0;
953953 }
954954
955+static const char *repr_mmap_type(enum i915_mmap_type type)
956+{
957+ switch (type) {
958+ case I915_MMAP_TYPE_GTT: return "gtt";
959+ case I915_MMAP_TYPE_WB: return "wb";
960+ case I915_MMAP_TYPE_WC: return "wc";
961+ case I915_MMAP_TYPE_UC: return "uc";
962+ default: return "unknown";
963+ }
964+}
965+
966+static bool can_access(const struct drm_i915_gem_object *obj)
967+{
968+ unsigned int flags =
969+ I915_GEM_OBJECT_HAS_STRUCT_PAGE | I915_GEM_OBJECT_HAS_IOMEM;
970+
971+ return i915_gem_object_type_has(obj, flags);
972+}
973+
974+static int __igt_mmap_access(struct drm_i915_private *i915,
975+ struct drm_i915_gem_object *obj,
976+ enum i915_mmap_type type)
977+{
978+ struct i915_mmap_offset *mmo;
979+ unsigned long __user *ptr;
980+ unsigned long A, B;
981+ unsigned long x, y;
982+ unsigned long addr;
983+ int err;
984+
985+ memset(&A, 0xAA, sizeof(A));
986+ memset(&B, 0xBB, sizeof(B));
987+
988+ if (!can_mmap(obj, type) || !can_access(obj))
989+ return 0;
990+
991+ mmo = mmap_offset_attach(obj, type, NULL);
992+ if (IS_ERR(mmo))
993+ return PTR_ERR(mmo);
994+
995+ addr = igt_mmap_node(i915, &mmo->vma_node, 0, PROT_WRITE, MAP_SHARED);
996+ if (IS_ERR_VALUE(addr))
997+ return addr;
998+ ptr = (unsigned long __user *)addr;
999+
1000+ err = __put_user(A, ptr);
1001+ if (err) {
1002+ pr_err("%s(%s): failed to write into user mmap\n",
1003+ obj->mm.region->name, repr_mmap_type(type));
1004+ goto out_unmap;
1005+ }
1006+
1007+ intel_gt_flush_ggtt_writes(&i915->gt);
1008+
1009+ err = access_process_vm(current, addr, &x, sizeof(x), 0);
1010+ if (err != sizeof(x)) {
1011+ pr_err("%s(%s): access_process_vm() read failed\n",
1012+ obj->mm.region->name, repr_mmap_type(type));
1013+ goto out_unmap;
1014+ }
1015+
1016+ err = access_process_vm(current, addr, &B, sizeof(B), FOLL_WRITE);
1017+ if (err != sizeof(B)) {
1018+ pr_err("%s(%s): access_process_vm() write failed\n",
1019+ obj->mm.region->name, repr_mmap_type(type));
1020+ goto out_unmap;
1021+ }
1022+
1023+ intel_gt_flush_ggtt_writes(&i915->gt);
1024+
1025+ err = __get_user(y, ptr);
1026+ if (err) {
1027+ pr_err("%s(%s): failed to read from user mmap\n",
1028+ obj->mm.region->name, repr_mmap_type(type));
1029+ goto out_unmap;
1030+ }
1031+
1032+ if (x != A || y != B) {
1033+ pr_err("%s(%s): failed to read/write values, found (%lx, %lx)\n",
1034+ obj->mm.region->name, repr_mmap_type(type),
1035+ x, y);
1036+ err = -EINVAL;
1037+ goto out_unmap;
1038+ }
1039+
1040+out_unmap:
1041+ vm_munmap(addr, obj->base.size);
1042+ return err;
1043+}
1044+
1045+static int igt_mmap_access(void *arg)
1046+{
1047+ struct drm_i915_private *i915 = arg;
1048+ struct intel_memory_region *mr;
1049+ enum intel_region_id id;
1050+
1051+ for_each_memory_region(mr, i915, id) {
1052+ struct drm_i915_gem_object *obj;
1053+ int err;
1054+
1055+ obj = i915_gem_object_create_region(mr, PAGE_SIZE, 0);
1056+ if (obj == ERR_PTR(-ENODEV))
1057+ continue;
1058+
1059+ if (IS_ERR(obj))
1060+ return PTR_ERR(obj);
1061+
1062+ err = __igt_mmap_access(i915, obj, I915_MMAP_TYPE_GTT);
1063+ if (err == 0)
1064+ err = __igt_mmap_access(i915, obj, I915_MMAP_TYPE_WB);
1065+ if (err == 0)
1066+ err = __igt_mmap_access(i915, obj, I915_MMAP_TYPE_WC);
1067+ if (err == 0)
1068+ err = __igt_mmap_access(i915, obj, I915_MMAP_TYPE_UC);
1069+
1070+ i915_gem_object_put(obj);
1071+ if (err)
1072+ return err;
1073+ }
1074+
1075+ return 0;
1076+}
1077+
9551078 static int __igt_mmap_gpu(struct drm_i915_private *i915,
9561079 struct drm_i915_gem_object *obj,
9571080 enum i915_mmap_type type)
@@ -1229,6 +1352,7 @@ int i915_gem_mman_live_selftests(struct drm_i915_private *i915) · 271 unmodified lines
12291352 SUBTEST(igt_smoke_tiling),
12301353 SUBTEST(igt_mmap_offset_exhaustion),
12311354 SUBTEST(igt_mmap),
1355+ SUBTEST(igt_mmap_access),
12321356 SUBTEST(igt_mmap_revoke),
12331357 SUBTEST(igt_mmap_gpu),
12341358 };
drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c+10−16
@@ -83,6 +83,8 @@ igt_emit_store_dw(struct i915_vma *vma, · 82 unmodified lines
8383 offset += PAGE_SIZE;
8484 }
8585 *cmd = MI_BATCH_BUFFER_END;
86+
87+ i915_gem_object_flush_map(obj);
8688 i915_gem_object_unpin_map(obj);
8789
8890 intel_gt_chipset_flush(vma->vm->gt);
@@ -126,16 +128,6 @@ int igt_gpu_fill_dw(struct intel_context *ce, · 37 unmodified lines
126128 goto err_batch;
127129 }
128130
129− flags = 0;
130− if (INTEL_GEN(ce->vm->i915) <= 5)
131− flags |= I915_DISPATCH_SECURE;
132−
133− err = rq->engine->emit_bb_start(rq,
134− batch->node.start, batch->node.size,
135− flags);
136− if (err)
137− goto err_request;
138−
139131 i915_vma_lock(batch);
140132 err = i915_request_await_object(rq, batch->obj, false);
141133 if (err == 0)
@@ -152,15 +144,17 @@ int igt_gpu_fill_dw(struct intel_context *ce, · 10 unmodified lines
152144 if (err)
153145 goto skip_request;
154146
155− i915_request_add(rq);
156−
157− i915_vma_unpin_and_release(&batch, 0);
147+ flags = 0;
148+ if (INTEL_GEN(ce->vm->i915) <= 5)
149+ flags |= I915_DISPATCH_SECURE;
158150
159− return 0;
151+ err = rq->engine->emit_bb_start(rq,
152+ batch->node.start, batch->node.size,
153+ flags);
160154
161155 skip_request:
162− i915_request_set_error_once(rq, err);
163−err_request:
156+ if (err)
157+ i915_request_set_error_once(rq, err);
164158 i915_request_add(rq);
165159 err_batch:
166160 i915_vma_unpin_and_release(&batch, 0);
drivers/gpu/drm/i915/gt/gen8_ppgtt.c+13−2
@@ -389,6 +389,16 @@ static int gen8_ppgtt_alloc(struct i915_address_space *vm, · 388 unmodified lines
389389 return err;
390390 }
391391
392+static __always_inline void
393+write_pte(gen8_pte_t *pte, const gen8_pte_t val)
394+{
395+ /* Magic delays? Or can we refine these to flush all in one pass? */
396+ *pte = val;
397+ wmb(); /* cpu to cache */
398+ clflush(pte); /* cache to memory */
399+ wmb(); /* visible to all */
400+}
401+
392402 static __always_inline u64
393403 gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt,
394404 struct i915_page_directory *pdp,
@@ -405,7 +415,8 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt, · 10 unmodified lines
405415 vaddr = kmap_atomic_px(i915_pt_entry(pd, gen8_pd_index(idx, 1)));
406416 do {
407417 GEM_BUG_ON(iter->sg->length < I915_GTT_PAGE_SIZE);
408− vaddr[gen8_pd_index(idx, 0)] = pte_encode | iter->dma;
418+ write_pte(&vaddr[gen8_pd_index(idx, 0)],
419+ pte_encode | iter->dma);
409420
410421 iter->dma += I915_GTT_PAGE_SIZE;
411422 if (iter->dma >= iter->max) {
@@ -487,7 +498,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma, · 75 unmodified lines
487498
488499 do {
489500 GEM_BUG_ON(iter->sg->length < page_size);
490− vaddr[index++] = encode | iter->dma;
501+ write_pte(&vaddr[index++], encode | iter->dma);
491502
492503 start += page_size;
493504 iter->dma += page_size;
drivers/gpu/drm/i915/gt/intel_breadcrumbs.c+42−10
@@ -142,6 +142,18 @@ static void add_retire(struct intel_breadcrumbs *b, struct intel_timeline *tl) · 141 unmodified lines
142142 intel_engine_add_retire(engine, tl);
143143 }
144144
145+static void __signal_request(struct i915_request *rq, struct list_head *signals)
146+{
147+ GEM_BUG_ON(!test_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags));
148+ clear_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags);
149+
150+ if (!__dma_fence_signal(&rq->fence))
151+ return;
152+
153+ i915_request_get(rq);
154+ list_add_tail(&rq->signal_link, signals);
155+}
156+
145157 static void signal_irq_work(struct irq_work *work)
146158 {
147159 struct intel_breadcrumbs *b = container_of(work, typeof(*b), irq_work);
@@ -155,6 +167,8 @@ static void signal_irq_work(struct irq_work *work) · 7 unmodified lines
155167 if (b->irq_armed && list_empty(&b->signalers))
156168 __intel_breadcrumbs_disarm_irq(b);
157169
170+ list_splice_init(&b->signaled_requests, &signal);
171+
158172 list_for_each_entry_safe(ce, cn, &b->signalers, signal_link) {
159173 GEM_BUG_ON(list_empty(&ce->signals));
160174
@@ -163,24 +177,15 @@ static void signal_irq_work(struct irq_work *work) · 2 unmodified lines
163177 list_entry(pos, typeof(*rq), signal_link);
164178
165179 GEM_BUG_ON(!check_signal_order(ce, rq));
166−
167180 if (!__request_completed(rq))
168181 break;
169182
170− GEM_BUG_ON(!test_bit(I915_FENCE_FLAG_SIGNAL,
171− &rq->fence.flags));
172− clear_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags);
173−
174− if (!__dma_fence_signal(&rq->fence))
175− continue;
176−
177183 /*
178184 * Queue for execution after dropping the signaling
179185 * spinlock as the callback chain may end up adding
180186 * more signalers to the same context or engine.
181187 */
182− i915_request_get(rq);
183− list_add_tail(&rq->signal_link, &signal);
188+ __signal_request(rq, &signal);
184189 }
185190
186191 /*
@@ -255,6 +260,7 @@ void intel_engine_init_breadcrumbs(struct intel_engine_cs *engine) · 68 unmodified lines
255260
256261 spin_lock_init(&b->irq_lock);
257262 INIT_LIST_HEAD(&b->signalers);
263+ INIT_LIST_HEAD(&b->signaled_requests);
258264
259265 init_irq_work(&b->irq_work, signal_irq_work);
260266 }
@@ -274,6 +280,32 @@ void intel_engine_reset_breadcrumbs(struct intel_engine_cs *engine) · 13 unmodified lines
274280 spin_unlock_irqrestore(&b->irq_lock, flags);
275281 }
276282
283+void intel_engine_transfer_stale_breadcrumbs(struct intel_engine_cs *engine,
284+ struct intel_context *ce)
285+{
286+ struct intel_breadcrumbs *b = &engine->breadcrumbs;
287+ unsigned long flags;
288+
289+ spin_lock_irqsave(&b->irq_lock, flags);
290+ if (!list_empty(&ce->signals)) {
291+ struct i915_request *rq, *next;
292+
293+ /* Queue for executing the signal callbacks in the irq_work */
294+ list_for_each_entry_safe(rq, next, &ce->signals, signal_link) {
295+ GEM_BUG_ON(rq->engine != engine);
296+ GEM_BUG_ON(!__request_completed(rq));
297+
298+ __signal_request(rq, &b->signaled_requests);
299+ }
300+
301+ INIT_LIST_HEAD(&ce->signals);
302+ list_del_init(&ce->signal_link);
303+
304+ irq_work_queue(&b->irq_work);
305+ }
306+ spin_unlock_irqrestore(&b->irq_lock, flags);
307+}
308+
277309 void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine)
278310 {
279311 }
drivers/gpu/drm/i915/gt/intel_engine.h+31−15
@@ -238,22 +238,35 @@ intel_engine_signal_breadcrumbs(struct intel_engine_cs *engine) · 237 unmodified lines
238238 void intel_engine_reset_breadcrumbs(struct intel_engine_cs *engine);
239239 void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine);
240240
241+void intel_engine_transfer_stale_breadcrumbs(struct intel_engine_cs *engine,
242+ struct intel_context *ce);
243+
241244 void intel_engine_print_breadcrumbs(struct intel_engine_cs *engine,
242245 struct drm_printer *p);
243246
244−static inline u32 *gen8_emit_pipe_control(u32 *batch, u32 flags, u32 offset)
247+static inline u32 *__gen8_emit_pipe_control(u32 *batch, u32 flags0, u32 flags1, u32 offset)
245248 {
246249 memset(batch, 0, 6 * sizeof(u32));
247250
248− batch[0] = GFX_OP_PIPE_CONTROL(6);
249− batch[1] = flags;
251+ batch[0] = GFX_OP_PIPE_CONTROL(6) | flags0;
252+ batch[1] = flags1;
250253 batch[2] = offset;
251254
252255 return batch + 6;
253256 }
254257
258+static inline u32 *gen8_emit_pipe_control(u32 *batch, u32 flags, u32 offset)
259+{
260+ return __gen8_emit_pipe_control(batch, 0, flags, offset);
261+}
262+
263+static inline u32 *gen12_emit_pipe_control(u32 *batch, u32 flags0, u32 flags1, u32 offset)
264+{
265+ return __gen8_emit_pipe_control(batch, flags0, flags1, offset);
266+}
267+
255268 static inline u32 *
256−gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags)
269+__gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags0, u32 flags1)
257270 {
258271 /* We're using qword write, offset should be aligned to 8 bytes. */
259272 GEM_BUG_ON(!IS_ALIGNED(gtt_offset, 8));
@@ -262,8 +275,8 @@ gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags) · 2 unmodified lines
262275 * need a prior CS_STALL, which is emitted by the flush
263276 * following the batch.
264277 */
265− *cs++ = GFX_OP_PIPE_CONTROL(6);
266− *cs++ = flags | PIPE_CONTROL_QW_WRITE | PIPE_CONTROL_GLOBAL_GTT_IVB;
278+ *cs++ = GFX_OP_PIPE_CONTROL(6) | flags0;
279+ *cs++ = flags1 | PIPE_CONTROL_QW_WRITE | PIPE_CONTROL_GLOBAL_GTT_IVB;
267280 *cs++ = gtt_offset;
268281 *cs++ = 0;
269282 *cs++ = value;
@@ -273,6 +286,18 @@ gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags) · 3 unmodified lines
273286 return cs;
274287 }
275288
289+static inline u32*
290+gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags)
291+{
292+ return __gen8_emit_ggtt_write_rcs(cs, value, gtt_offset, 0, flags);
293+}
294+
295+static inline u32*
296+gen12_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags0, u32 flags1)
297+{
298+ return __gen8_emit_ggtt_write_rcs(cs, value, gtt_offset, flags0, flags1);
299+}
300+
276301 static inline u32 *
277302 gen8_emit_ggtt_write(u32 *cs, u32 value, u32 gtt_offset, u32 flags)
278303 {
@@ -332,13 +357,4 @@ intel_engine_has_preempt_reset(const struct intel_engine_cs *engine) · 53 unmodified lines
332357 return intel_engine_has_preemption(engine);
333358 }
334359
335−static inline bool
336−intel_engine_has_timeslices(const struct intel_engine_cs *engine)
337−{
338− if (!IS_ACTIVE(CONFIG_DRM_I915_TIMESLICE_DURATION))
339− return false;
340−
341− return intel_engine_has_semaphores(engine);
342−}
343−
344360 #endif /* _INTEL_RINGBUFFER_H_ */
drivers/gpu/drm/i915/gt/intel_engine_cs.c+2−4
@@ -31,7 +31,6 @@ · 30 unmodified lines
3131 #include "intel_context.h"
3232 #include "intel_engine.h"
3333 #include "intel_engine_pm.h"
34−#include "intel_engine_pool.h"
3534 #include "intel_engine_user.h"
3635 #include "intel_gt.h"
3736 #include "intel_gt_requests.h"
@@ -327,6 +326,8 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id) · 289 unmodified lines
327326 if (INTEL_GEN(i915) == 12 && engine->class == RENDER_CLASS)
328327 engine->props.preempt_timeout_ms = 0;
329328
329+ engine->defaults = engine->props; /* never to change again */
330+
330331 engine->context_size = intel_engine_context_size(gt, engine->class);
331332 if (WARN_ON(engine->context_size > BIT(20)))
332333 engine->context_size = 0;
@@ -631,8 +632,6 @@ static int engine_setup_common(struct intel_engine_cs *engine) · 298 unmodified lines
631632 intel_engine_init__pm(engine);
632633 intel_engine_init_retire(engine);
633634
634− intel_engine_pool_init(&engine->pool);
635−
636635 /* Use the whole device by default */
637636 engine->sseu =
638637 intel_sseu_from_device_info(&RUNTIME_INFO(engine->i915)->sseu);
@@ -829,7 +828,6 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine) · 190 unmodified lines
829828 cleanup_status_page(engine);
830829
831830 intel_engine_fini_retire(engine);
832− intel_engine_pool_fini(&engine->pool);
833831 intel_engine_fini_breadcrumbs(engine);
834832 intel_engine_cleanup_cmd_parser(engine);
835833
drivers/gpu/drm/i915/gt/intel_engine_pm.c+1−15
@@ -10,7 +10,6 @@ · 9 unmodified lines
1010 #include "intel_engine.h"
1111 #include "intel_engine_heartbeat.h"
1212 #include "intel_engine_pm.h"
13−#include "intel_engine_pool.h"
1413 #include "intel_gt.h"
1514 #include "intel_gt_pm.h"
1615 #include "intel_rc6.h"
@@ -22,18 +21,11 @@ static int __engine_unpark(struct intel_wakeref *wf) · 5 unmodified lines
2221 struct intel_engine_cs *engine =
2322 container_of(wf, typeof(*engine), wakeref);
2423 struct intel_context *ce;
25− void *map;
2624
2725 ENGINE_TRACE(engine, "\n");
2826
2927 intel_gt_pm_get(engine->gt);
3028
31− /* Pin the default state for fast resets from atomic context. */
32− map = NULL;
33− if (engine->default_state)
34− map = shmem_pin_map(engine->default_state);
35− engine->pinned_default_state = map;
36−
3729 /* Discard stale context state from across idling */
3830 ce = engine->kernel_context;
3931 if (ce) {
@@ -43,6 +35,7 @@ static int __engine_unpark(struct intel_wakeref *wf) · 3 unmodified lines
4335 if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) && ce->state) {
4436 struct drm_i915_gem_object *obj = ce->state->obj;
4537 int type = i915_coherent_map_type(engine->i915);
38+ void *map;
4639
4740 map = i915_gem_object_pin_map(obj, type);
4841 if (!IS_ERR(map)) {
@@ -254,7 +247,6 @@ static int __engine_park(struct intel_wakeref *wf) · 205 unmodified lines
254247
255248 intel_engine_park_heartbeat(engine);
256249 intel_engine_disarm_breadcrumbs(engine);
257− intel_engine_pool_park(&engine->pool);
258250
259251 /* Must be reset upon idling, or we may miss the busy wakeup. */
260252 GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
@@ -262,12 +254,6 @@ static int __engine_park(struct intel_wakeref *wf) · 1 unmodified line
262254 if (engine->park)
263255 engine->park(engine);
264256
265− if (engine->pinned_default_state) {
266− shmem_unpin_map(engine->default_state,
267− engine->pinned_default_state);
268− engine->pinned_default_state = NULL;
269− }
270−
271257 engine->execlists.no_priolist = false;
272258
273259 /* While gt calls i915_vma_parked(), we have to break the lock cycle */
drivers/gpu/drm/i915/gt/intel_engine_pool.h+0−34
@@ -1,34 +0,0 @@
1−/*
2− * SPDX-License-Identifier: MIT
3− *
4− * Copyright © 2014-2018 Intel Corporation
5− */
6−
7−#ifndef INTEL_ENGINE_POOL_H
8−#define INTEL_ENGINE_POOL_H
9−
10−#include "intel_engine_pool_types.h"
11−#include "i915_active.h"
12−#include "i915_request.h"
13−
14−struct intel_engine_pool_node *
15−intel_engine_get_pool(struct intel_engine_cs *engine, size_t size);
16−
17−static inline int
18−intel_engine_pool_mark_active(struct intel_engine_pool_node *node,
19− struct i915_request *rq)
20−{
21− return i915_active_add_request(&node->active, rq);
22−}
23−
24−static inline void
25−intel_engine_pool_put(struct intel_engine_pool_node *node)
26−{
27− i915_active_release(&node->active);
28−}
29−
30−void intel_engine_pool_init(struct intel_engine_pool *pool);
31−void intel_engine_pool_park(struct intel_engine_pool *pool);
32−void intel_engine_pool_fini(struct intel_engine_pool *pool);
33−
34−#endif /* INTEL_ENGINE_POOL_H */
drivers/gpu/drm/i915/gt/intel_engine_types.h+22−14
@@ -22,7 +22,6 @@ · 21 unmodified lines
2222 #include "i915_pmu.h"
2323 #include "i915_priolist_types.h"
2424 #include "i915_selftest.h"
25−#include "intel_engine_pool_types.h"
2625 #include "intel_sseu.h"
2726 #include "intel_timeline_types.h"
2827 #include "intel_wakeref.h"
@@ -180,6 +179,11 @@ struct intel_engine_execlists { · 151 unmodified lines
180179 */
181180 u32 error_interrupt;
182181
182+ /**
183+ * @reset_ccid: Active CCID [EXECLISTS_STATUS_HI] at the time of reset
184+ */
185+ u32 reset_ccid;
186+
183187 /**
184188 * @no_priolist: priority lists disabled
185189 */
@@ -340,7 +344,6 @@ struct intel_engine_cs { · 154 unmodified lines
340344 unsigned long wakeref_serial;
341345 struct intel_wakeref wakeref;
342346 struct file *default_state;
343− void *pinned_default_state;
344347
345348 struct {
346349 struct intel_ring *ring;
@@ -374,6 +377,8 @@ struct intel_engine_cs { · 27 unmodified lines
374377 spinlock_t irq_lock;
375378 struct list_head signalers;
376379
380+ struct list_head signaled_requests;
381+
377382 struct irq_work irq_work; /* for use from inside irq_lock */
378383
379384 unsigned int irq_enabled;
@@ -405,13 +410,6 @@ struct intel_engine_cs { · 25 unmodified lines
405410 struct i915_pmu_sample sample[I915_ENGINE_SAMPLE_COUNT];
406411 } pmu;
407412
408− /*
409− * A pool of objects to use as shadow copies of client batch buffers
410− * when the command parser is enabled. Prevents the client from
411− * modifying the batch contents after software parsing.
412− */
413− struct intel_engine_pool pool;
414−
415413 struct intel_hw_status_page status_page;
416414 struct i915_ctx_workarounds wa_ctx;
417415 struct i915_wa_list ctx_wa_list;
@@ -500,10 +498,11 @@ struct intel_engine_cs { · 82 unmodified lines
500498 #define I915_ENGINE_SUPPORTS_STATS BIT(1)
501499 #define I915_ENGINE_HAS_PREEMPTION BIT(2)
502500 #define I915_ENGINE_HAS_SEMAPHORES BIT(3)
503−#define I915_ENGINE_NEEDS_BREADCRUMB_TASKLET BIT(4)
504−#define I915_ENGINE_IS_VIRTUAL BIT(5)
505−#define I915_ENGINE_HAS_RELATIVE_MMIO BIT(6)
506−#define I915_ENGINE_REQUIRES_CMD_PARSER BIT(7)
501+#define I915_ENGINE_HAS_TIMESLICES BIT(4)
502+#define I915_ENGINE_NEEDS_BREADCRUMB_TASKLET BIT(5)
503+#define I915_ENGINE_IS_VIRTUAL BIT(6)
504+#define I915_ENGINE_HAS_RELATIVE_MMIO BIT(7)
505+#define I915_ENGINE_REQUIRES_CMD_PARSER BIT(8)
507506 unsigned int flags;
508507
509508 /*
@@ -568,7 +567,7 @@ struct intel_engine_cs { · 58 unmodified lines
568567 unsigned long preempt_timeout_ms;
569568 unsigned long stop_timeout_ms;
570569 unsigned long timeslice_duration_ms;
571− } props;
570+ } props, defaults;
572571 };
573572
574573 static inline bool
@@ -601,6 +600,15 @@ intel_engine_has_semaphores(const struct intel_engine_cs *engine) · 26 unmodified lines
601600 return engine->flags & I915_ENGINE_HAS_SEMAPHORES;
602601 }
603602
603+static inline bool
604+intel_engine_has_timeslices(const struct intel_engine_cs *engine)
605+{
606+ if (!IS_ACTIVE(CONFIG_DRM_I915_TIMESLICE_DURATION))
607+ return false;
608+
609+ return engine->flags & I915_ENGINE_HAS_TIMESLICES;
610+}
611+
604612 static inline bool
605613 intel_engine_needs_breadcrumb_tasklet(const struct intel_engine_cs *engine)
606614 {
drivers/gpu/drm/i915/gt/intel_gpu_commands.h+1−2
@@ -236,9 +236,8 @@ · 235 unmodified lines
236236 #define PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH (1<<12) /* gen6+ */
237237 #define PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE (1<<11) /* MBZ on ILK */
238238 #define PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE (1<<10) /* GM45+ only */
239−#define PIPE_CONTROL_L3_RO_CACHE_INVALIDATE REG_BIT(10) /* gen12 */
240239 #define PIPE_CONTROL_INDIRECT_STATE_DISABLE (1<<9)
241−#define PIPE_CONTROL_HDC_PIPELINE_FLUSH REG_BIT(9) /* gen12 */
240+#define PIPE_CONTROL0_HDC_PIPELINE_FLUSH REG_BIT(9) /* gen12 */
242241 #define PIPE_CONTROL_NOTIFY (1<<8)
243242 #define PIPE_CONTROL_FLUSH_ENABLE (1<<7) /* gen7+ */
244243 #define PIPE_CONTROL_DC_FLUSH_ENABLE (1<<5)
drivers/gpu/drm/i915/gt/intel_gt.c+3−0
@@ -7,6 +7,7 @@ · 6 unmodified lines
77 #include "i915_drv.h"
88 #include "intel_context.h"
99 #include "intel_gt.h"
10+#include "intel_gt_buffer_pool.h"
1011 #include "intel_gt_clock_utils.h"
1112 #include "intel_gt_pm.h"
1213 #include "intel_gt_requests.h"
@@ -28,6 +29,7 @@ void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915) · 15 unmodified lines
2829 INIT_LIST_HEAD(&gt->closed_vma);
2930 spin_lock_init(&gt->closed_lock);
3031
32+ intel_gt_init_buffer_pool(gt);
3133 intel_gt_init_reset(gt);
3234 intel_gt_init_requests(gt);
3335 intel_gt_init_timelines(gt);
@@ -621,6 +623,7 @@ void intel_gt_driver_release(struct intel_gt *gt) · 587 unmodified lines
621623
622624 intel_gt_pm_fini(gt);
623625 intel_gt_fini_scratch(gt);
626+ intel_gt_fini_buffer_pool(gt);
624627 }
625628
626629 void intel_gt_driver_late_release(struct intel_gt *gt)
drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.crenamed from drivers/gpu/drm/i915/gt/intel_engine_pool.c+76−38
@@ -1,6 +1,5 @@
1+// SPDX-License-Identifier: MIT
12 /*
2− * SPDX-License-Identifier: MIT
3− *
43 * Copyright © 2014-2018 Intel Corporation
54 */
65
@@ -8,15 +7,15 @@ · 1 unmodified line
87
98 #include "i915_drv.h"
109 #include "intel_engine_pm.h"
11−#include "intel_engine_pool.h"
10+#include "intel_gt_buffer_pool.h"
1211
13−static struct intel_engine_cs *to_engine(struct intel_engine_pool *pool)
12+static struct intel_gt *to_gt(struct intel_gt_buffer_pool *pool)
1413 {
15− return container_of(pool, struct intel_engine_cs, pool);
14+ return container_of(pool, struct intel_gt, buffer_pool);
1615 }
1716
1817 static struct list_head *
19−bucket_for_size(struct intel_engine_pool *pool, size_t sz)
18+bucket_for_size(struct intel_gt_buffer_pool *pool, size_t sz)
2019 {
2120 int n;
2221
@@ -32,16 +31,50 @@ bucket_for_size(struct intel_engine_pool *pool, size_t sz) · 9 unmodified lines
3231 return &pool->cache_list[n];
3332 }
3433
35−static void node_free(struct intel_engine_pool_node *node)
34+static void node_free(struct intel_gt_buffer_pool_node *node)
3635 {
3736 i915_gem_object_put(node->obj);
3837 i915_active_fini(&node->active);
3938 kfree(node);
4039 }
4140
41+static void pool_free_work(struct work_struct *wrk)
42+{
43+ struct intel_gt_buffer_pool *pool =
44+ container_of(wrk, typeof(*pool), work.work);
45+ struct intel_gt_buffer_pool_node *node, *next;
46+ unsigned long old = jiffies - HZ;
47+ bool active = false;
48+ LIST_HEAD(stale);
49+ int n;
50+
51+ /* Free buffers that have not been used in the past second */
52+ spin_lock_irq(&pool->lock);
53+ for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++) {
54+ struct list_head *list = &pool->cache_list[n];
55+
56+ /* Most recent at head; oldest at tail */
57+ list_for_each_entry_safe_reverse(node, next, list, link) {
58+ if (time_before(node->age, old))
59+ break;
60+
61+ list_move(&node->link, &stale);
62+ }
63+ active |= !list_empty(list);
64+ }
65+ spin_unlock_irq(&pool->lock);
66+
67+ list_for_each_entry_safe(node, next, &stale, link)
68+ node_free(node);
69+
70+ if (active)
71+ schedule_delayed_work(&pool->work,
72+ round_jiffies_up_relative(HZ));
73+}
74+
4275 static int pool_active(struct i915_active *ref)
4376 {
44− struct intel_engine_pool_node *node =
77+ struct intel_gt_buffer_pool_node *node =
4578 container_of(ref, typeof(*node), active);
4679 struct dma_resv *resv = node->obj->base.resv;
4780 int err;
@@ -64,29 +97,31 @@ static int pool_active(struct i915_active *ref) · 16 unmodified lines
6497 __i915_active_call
6598 static void pool_retire(struct i915_active *ref)
6699 {
67− struct intel_engine_pool_node *node =
100+ struct intel_gt_buffer_pool_node *node =
68101 container_of(ref, typeof(*node), active);
69− struct intel_engine_pool *pool = node->pool;
102+ struct intel_gt_buffer_pool *pool = node->pool;
70103 struct list_head *list = bucket_for_size(pool, node->obj->base.size);
71104 unsigned long flags;
72105
73− GEM_BUG_ON(!intel_engine_pm_is_awake(to_engine(pool)));
74−
75106 i915_gem_object_unpin_pages(node->obj);
76107
77108 /* Return this object to the shrinker pool */
78109 i915_gem_object_make_purgeable(node->obj);
79110
80111 spin_lock_irqsave(&pool->lock, flags);
112+ node->age = jiffies;
81113 list_add(&node->link, list);
82114 spin_unlock_irqrestore(&pool->lock, flags);
115+
116+ schedule_delayed_work(&pool->work,
117+ round_jiffies_up_relative(HZ));
83118 }
84119
85−static struct intel_engine_pool_node *
86−node_create(struct intel_engine_pool *pool, size_t sz)
120+static struct intel_gt_buffer_pool_node *
121+node_create(struct intel_gt_buffer_pool *pool, size_t sz)
87122 {
88− struct intel_engine_cs *engine = to_engine(pool);
89− struct intel_engine_pool_node *node;
123+ struct intel_gt *gt = to_gt(pool);
124+ struct intel_gt_buffer_pool_node *node;
90125 struct drm_i915_gem_object *obj;
91126
92127 node = kmalloc(sizeof(*node),
@@ -97,7 +132,7 @@ node_create(struct intel_engine_pool *pool, size_t sz) · 4 unmodified lines
97132 node->pool = pool;
98133 i915_active_init(&node->active, pool_active, pool_retire);
99134
100− obj = i915_gem_object_create_internal(engine->i915, sz);
135+ obj = i915_gem_object_create_internal(gt->i915, sz);
101136 if (IS_ERR(obj)) {
102137 i915_active_fini(&node->active);
103138 kfree(node);
@@ -110,26 +145,15 @@ node_create(struct intel_engine_pool *pool, size_t sz) · 6 unmodified lines
110145 return node;
111146 }
112147
113−static struct intel_engine_pool *lookup_pool(struct intel_engine_cs *engine)
148+struct intel_gt_buffer_pool_node *
149+intel_gt_get_buffer_pool(struct intel_gt *gt, size_t size)
114150 {
115− if (intel_engine_is_virtual(engine))
116− engine = intel_virtual_engine_get_sibling(engine, 0);
117−
118− GEM_BUG_ON(!engine);
119− return &engine->pool;
120−}
121−
122−struct intel_engine_pool_node *
123−intel_engine_get_pool(struct intel_engine_cs *engine, size_t size)
124−{
125− struct intel_engine_pool *pool = lookup_pool(engine);
126− struct intel_engine_pool_node *node;
151+ struct intel_gt_buffer_pool *pool = &gt->buffer_pool;
152+ struct intel_gt_buffer_pool_node *node;
127153 struct list_head *list;
128154 unsigned long flags;
129155 int ret;
130156
131− GEM_BUG_ON(!intel_engine_pm_is_awake(to_engine(pool)));
132−
133157 size = PAGE_ALIGN(size);
134158 list = bucket_for_size(pool, size);
135159
@@ -157,34 +181,48 @@ intel_engine_get_pool(struct intel_engine_cs *engine, size_t size) · 21 unmodified lines
157181 return node;
158182 }
159183
160−void intel_engine_pool_init(struct intel_engine_pool *pool)
184+void intel_gt_init_buffer_pool(struct intel_gt *gt)
161185 {
186+ struct intel_gt_buffer_pool *pool = &gt->buffer_pool;
162187 int n;
163188
164189 spin_lock_init(&pool->lock);
165190 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++)
166191 INIT_LIST_HEAD(&pool->cache_list[n]);
192+ INIT_DELAYED_WORK(&pool->work, pool_free_work);
167193 }
168194
169−void intel_engine_pool_park(struct intel_engine_pool *pool)
195+static void pool_free_imm(struct intel_gt_buffer_pool *pool)
170196 {
171197 int n;
172198
199+ spin_lock_irq(&pool->lock);
173200 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++) {
201+ struct intel_gt_buffer_pool_node *node, *next;
174202 struct list_head *list = &pool->cache_list[n];
175− struct intel_engine_pool_node *node, *nn;
176203
177− list_for_each_entry_safe(node, nn, list, link)
204+ list_for_each_entry_safe(node, next, list, link)
178205 node_free(node);
179−
180206 INIT_LIST_HEAD(list);
181207 }
208+ spin_unlock_irq(&pool->lock);
209+}
210+
211+void intel_gt_flush_buffer_pool(struct intel_gt *gt)
212+{
213+ struct intel_gt_buffer_pool *pool = &gt->buffer_pool;
214+
215+ if (cancel_delayed_work_sync(&pool->work))
216+ pool_free_imm(pool);
182217 }
183218
184−void intel_engine_pool_fini(struct intel_engine_pool *pool)
219+void intel_gt_fini_buffer_pool(struct intel_gt *gt)
185220 {
221+ struct intel_gt_buffer_pool *pool = &gt->buffer_pool;
186222 int n;
187223
224+ intel_gt_flush_buffer_pool(gt);
225+
188226 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++)
189227 GEM_BUG_ON(!list_empty(&pool->cache_list[n]));
190228 }
drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h+37−0
@@ -0,0 +1,37 @@
1+/* SPDX-License-Identifier: MIT */
2+/*
3+ * Copyright © 2014-2018 Intel Corporation
4+ */
5+
6+#ifndef INTEL_GT_BUFFER_POOL_H
7+#define INTEL_GT_BUFFER_POOL_H
8+
9+#include <linux/types.h>
10+
11+#include "i915_active.h"
12+#include "intel_gt_buffer_pool_types.h"
13+
14+struct intel_gt;
15+struct i915_request;
16+
17+struct intel_gt_buffer_pool_node *
18+intel_gt_get_buffer_pool(struct intel_gt *gt, size_t size);
19+
20+static inline int
21+intel_gt_buffer_pool_mark_active(struct intel_gt_buffer_pool_node *node,
22+ struct i915_request *rq)
23+{
24+ return i915_active_add_request(&node->active, rq);
25+}
26+
27+static inline void
28+intel_gt_buffer_pool_put(struct intel_gt_buffer_pool_node *node)
29+{
30+ i915_active_release(&node->active);
31+}
32+
33+void intel_gt_init_buffer_pool(struct intel_gt *gt);
34+void intel_gt_flush_buffer_pool(struct intel_gt *gt);
35+void intel_gt_fini_buffer_pool(struct intel_gt *gt);
36+
37+#endif /* INTEL_GT_BUFFER_POOL_H */
drivers/gpu/drm/i915/gt/intel_gt_buffer_pool_types.hrenamed from drivers/gpu/drm/i915/gt/intel_engine_pool_types.h+9−6
@@ -4,26 +4,29 @@ · 3 unmodified lines
44 * Copyright © 2014-2018 Intel Corporation
55 */
66
7−#ifndef INTEL_ENGINE_POOL_TYPES_H
8−#define INTEL_ENGINE_POOL_TYPES_H
7+#ifndef INTEL_GT_BUFFER_POOL_TYPES_H
8+#define INTEL_GT_BUFFER_POOL_TYPES_H
99
1010 #include <linux/list.h>
1111 #include <linux/spinlock.h>
12+#include <linux/workqueue.h>
1213
1314 #include "i915_active_types.h"
1415
1516 struct drm_i915_gem_object;
1617
17−struct intel_engine_pool {
18+struct intel_gt_buffer_pool {
1819 spinlock_t lock;
1920 struct list_head cache_list[4];
21+ struct delayed_work work;
2022 };
2123
22−struct intel_engine_pool_node {
24+struct intel_gt_buffer_pool_node {
2325 struct i915_active active;
2426 struct drm_i915_gem_object *obj;
2527 struct list_head link;
26− struct intel_engine_pool *pool;
28+ struct intel_gt_buffer_pool *pool;
29+ unsigned long age;
2730 };
2831
29−#endif /* INTEL_ENGINE_POOL_TYPES_H */
32+#endif /* INTEL_GT_BUFFER_POOL_TYPES_H */
drivers/gpu/drm/i915/gt/intel_gt_pm.c+6−4
@@ -150,10 +150,6 @@ static void gt_sanitize(struct intel_gt *gt, bool force) · 149 unmodified lines
150150 if (intel_gt_is_wedged(gt))
151151 intel_gt_unset_wedged(gt);
152152
153− for_each_engine(engine, gt, id)
154− if (engine->sanitize)
155− engine->sanitize(engine);
156−
157153 intel_uc_sanitize(&gt->uc);
158154
159155 for_each_engine(engine, gt, id)
@@ -162,6 +158,10 @@ static void gt_sanitize(struct intel_gt *gt, bool force) · 2 unmodified lines
162158
163159 intel_uc_reset_prepare(&gt->uc);
164160
161+ for_each_engine(engine, gt, id)
162+ if (engine->sanitize)
163+ engine->sanitize(engine);
164+
165165 if (reset_engines(gt) || force) {
166166 for_each_engine(engine, gt, id)
167167 __intel_engine_reset(engine, false);
@@ -171,6 +171,8 @@ static void gt_sanitize(struct intel_gt *gt, bool force) · 3 unmodified lines
171171 if (engine->reset.finish)
172172 engine->reset.finish(engine);
173173
174+ intel_rps_sanitize(&gt->rps);
175+
174176 intel_uncore_forcewake_put(gt->uncore, FORCEWAKE_ALL);
175177 intel_runtime_pm_put(gt->uncore->rpm, wakeref);
176178 }
drivers/gpu/drm/i915/gt/intel_gt_types.h+11−0
@@ -17,6 +17,7 @@ · 16 unmodified lines
1717
1818 #include "i915_vma.h"
1919 #include "intel_engine_types.h"
20+#include "intel_gt_buffer_pool_types.h"
2021 #include "intel_llc_types.h"
2122 #include "intel_reset_types.h"
2223 #include "intel_rc6_types.h"
@@ -97,6 +98,16 @@ struct intel_gt { · 74 unmodified lines
9798 */
9899 struct i915_address_space *vm;
99100
101+ /*
102+ * A pool of objects to use as shadow copies of client batch buffers
103+ * when the command parser is enabled. Prevents the client from
104+ * modifying the batch contents after software parsing.
105+ *
106+ * Buffers older than 1s are periodically reaped from the pool,
107+ * or may be reclaimed by the shrinker before then.
108+ */
109+ struct intel_gt_buffer_pool buffer_pool;
110+
100111 struct i915_vma *scratch;
101112 };
102113
drivers/gpu/drm/i915/gt/intel_lrc.c+224−96
@@ -217,7 +217,7 @@ struct virtual_engine { · 216 unmodified lines
217217
218218 /* And finally, which physical engines this virtual engine maps onto. */
219219 unsigned int num_siblings;
220− struct intel_engine_cs *siblings[0];
220+ struct intel_engine_cs *siblings[];
221221 };
222222
223223 static struct virtual_engine *to_virtual_engine(struct intel_engine_cs *engine)
@@ -429,18 +429,7 @@ static int effective_prio(const struct i915_request *rq) · 205 unmodified lines
429429 if (i915_request_has_nopreempt(rq))
430430 prio = I915_PRIORITY_UNPREEMPTABLE;
431431
432− /*
433− * On unwinding the active request, we give it a priority bump
434− * if it has completed waiting on any semaphore. If we know that
435− * the request has already started, we can prevent an unwanted
436− * preempt-to-idle cycle by taking that into account now.
437− */
438− if (__i915_request_has_started(rq))
439− prio |= I915_PRIORITY_NOSEMAPHORE;
440−
441− /* Restrict mere WAIT boosts from triggering preemption */
442− BUILD_BUG_ON(__NO_PREEMPTION & ~I915_PRIORITY_MASK); /* only internal */
443− return prio | __NO_PREEMPTION;
432+ return prio;
444433 }
445434
446435 static int queue_prio(const struct intel_engine_execlists *execlists)
@@ -1271,14 +1260,11 @@ execlists_check_context(const struct intel_context *ce, · 824 unmodified lines
12711260 static void restore_default_state(struct intel_context *ce,
12721261 struct intel_engine_cs *engine)
12731262 {
1274− u32 *regs = ce->lrc_reg_state;
1263+ u32 *regs;
12751264
1276− if (engine->pinned_default_state)
1277− memcpy(regs, /* skip restoring the vanilla PPHWSP */
1278− engine->pinned_default_state + LRC_STATE_OFFSET,
1279− engine->context_size - PAGE_SIZE);
1265+ regs = memset(ce->lrc_reg_state, 0, engine->context_size - PAGE_SIZE);
1266+ execlists_init_reg_state(regs, ce, engine, ce->ring, true);
12801267
1281− execlists_init_reg_state(regs, ce, engine, ce->ring, false);
12821268 ce->runtime.last = intel_context_get_runtime(ce);
12831269 }
12841270
@@ -1372,7 +1358,7 @@ __execlists_schedule_in(struct i915_request *rq) · 87 unmodified lines
13721358 ce->lrc.ccid = ce->tag;
13731359 } else {
13741360 /* We don't need a strict matching tag, just different values */
1375− unsigned int tag = ffs(engine->context_tag);
1361+ unsigned int tag = ffs(READ_ONCE(engine->context_tag));
13761362
13771363 GEM_BUG_ON(tag == 0 || tag >= BITS_PER_LONG);
13781364 clear_bit(tag - 1, &engine->context_tag);
@@ -1826,30 +1812,16 @@ static bool virtual_matches(const struct virtual_engine *ve, · 447 unmodified lines
18261812 return true;
18271813 }
18281814
1829−static void virtual_xfer_breadcrumbs(struct virtual_engine *ve,
1830− struct i915_request *rq)
1815+static void virtual_xfer_breadcrumbs(struct virtual_engine *ve)
18311816 {
1832− struct intel_engine_cs *old = ve->siblings[0];
1833−
1834− /* All unattached (rq->engine == old) must already be completed */
1835−
1836− spin_lock(&old->breadcrumbs.irq_lock);
1837− if (!list_empty(&ve->context.signal_link)) {
1838− list_del_init(&ve->context.signal_link);
1839−
1840− /*
1841− * We cannot acquire the new engine->breadcrumbs.irq_lock
1842− * (as we are holding a breadcrumbs.irq_lock already),
1843− * so attach this request to the signaler on submission.
1844− * The queued irq_work will occur when we finally drop
1845− * the engine->active.lock after dequeue.
1846− */
1847− set_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT, &rq->fence.flags);
1848−
1849− /* Also transfer the pending irq_work for the old breadcrumb. */
1850− intel_engine_signal_breadcrumbs(rq->engine);
1851− }
1852− spin_unlock(&old->breadcrumbs.irq_lock);
1817+ /*
1818+ * All the outstanding signals on ve->siblings[0] must have
1819+ * been completed, just pending the interrupt handler. As those
1820+ * signals still refer to the old sibling (via rq->engine), we must
1821+ * transfer those to the old irq_worker to keep our locking
1822+ * consistent.
1823+ */
1824+ intel_engine_transfer_stale_breadcrumbs(ve->siblings[0], &ve->context);
18531825 }
18541826
18551827 #define for_each_waiter(p__, rq__) \
@@ -1883,12 +1855,16 @@ static void defer_request(struct i915_request *rq, struct list_head * const pl) · 27 unmodified lines
18831855 struct i915_request *w =
18841856 container_of(p->waiter, typeof(*w), sched);
18851857
1858+ if (p->flags & I915_DEPENDENCY_WEAK)
1859+ continue;
1860+
18861861 /* Leave semaphores spinning on the other engines */
18871862 if (w->engine != rq->engine)
18881863 continue;
18891864
18901865 /* No waiter should start before its signaler */
1891− GEM_BUG_ON(i915_request_started(w) &&
1866+ GEM_BUG_ON(i915_request_has_initial_breadcrumb(w) &&
1867+ i915_request_started(w) &&
18921868 !i915_request_completed(rq));
18931869
18941870 GEM_BUG_ON(i915_request_is_active(w));
@@ -2280,7 +2256,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine) · 385 unmodified lines
22802256 engine);
22812257
22822258 if (!list_empty(&ve->context.signals))
2283− virtual_xfer_breadcrumbs(ve, rq);
2259+ virtual_xfer_breadcrumbs(ve);
22842260
22852261 /*
22862262 * Move the bound engine to the top of the list
@@ -3494,6 +3470,7 @@ static int gen8_emit_init_breadcrumb(struct i915_request *rq) · 1207 unmodified lines
34943470 {
34953471 u32 *cs;
34963472
3473+ GEM_BUG_ON(i915_request_has_initial_breadcrumb(rq));
34973474 if (!i915_request_timeline(rq)->has_initial_breadcrumb)
34983475 return 0;
34993476
@@ -3520,6 +3497,56 @@ static int gen8_emit_init_breadcrumb(struct i915_request *rq) · 20 unmodified lines
35203497 /* Record the updated position of the request's payload */
35213498 rq->infix = intel_ring_offset(rq, cs);
35223499
3500+ __set_bit(I915_FENCE_FLAG_INITIAL_BREADCRUMB, &rq->fence.flags);
3501+
3502+ return 0;
3503+}
3504+
3505+static int emit_pdps(struct i915_request *rq)
3506+{
3507+ const struct intel_engine_cs * const engine = rq->engine;
3508+ struct i915_ppgtt * const ppgtt = i915_vm_to_ppgtt(rq->context->vm);
3509+ int err, i;
3510+ u32 *cs;
3511+
3512+ GEM_BUG_ON(intel_vgpu_active(rq->i915));
3513+
3514+ /*
3515+ * Beware ye of the dragons, this sequence is magic!
3516+ *
3517+ * Small changes to this sequence can cause anything from
3518+ * GPU hangs to forcewake errors and machine lockups!
3519+ */
3520+
3521+ /* Flush any residual operations from the context load */
3522+ err = engine->emit_flush(rq, EMIT_FLUSH);
3523+ if (err)
3524+ return err;
3525+
3526+ /* Magic required to prevent forcewake errors! */
3527+ err = engine->emit_flush(rq, EMIT_INVALIDATE);
3528+ if (err)
3529+ return err;
3530+
3531+ cs = intel_ring_begin(rq, 4 * GEN8_3LVL_PDPES + 2);
3532+ if (IS_ERR(cs))
3533+ return PTR_ERR(cs);
3534+
3535+ /* Ensure the LRI have landed before we invalidate & continue */
3536+ *cs++ = MI_LOAD_REGISTER_IMM(2 * GEN8_3LVL_PDPES) | MI_LRI_FORCE_POSTED;
3537+ for (i = GEN8_3LVL_PDPES; i--; ) {
3538+ const dma_addr_t pd_daddr = i915_page_dir_dma_addr(ppgtt, i);
3539+ u32 base = engine->mmio_base;
3540+
3541+ *cs++ = i915_mmio_reg_offset(GEN8_RING_PDP_UDW(base, i));
3542+ *cs++ = upper_32_bits(pd_daddr);
3543+ *cs++ = i915_mmio_reg_offset(GEN8_RING_PDP_LDW(base, i));
3544+ *cs++ = lower_32_bits(pd_daddr);
3545+ }
3546+ *cs++ = MI_NOOP;
3547+
3548+ intel_ring_advance(rq, cs);
3549+
35233550 return 0;
35243551 }
35253552
@@ -3544,6 +3571,12 @@ static int execlists_request_alloc(struct i915_request *request) · 18 unmodified lines
35443571 * to cancel/unwind this request now.
35453572 */
35463573
3574+ if (!i915_vm_is_4lvl(request->context->vm)) {
3575+ ret = emit_pdps(request);
3576+ if (ret)
3577+ return ret;
3578+ }
3579+
35473580 /* Unconditionally invalidate GPU caches and TLBs. */
35483581 ret = request->engine->emit_flush(request, EMIT_INVALIDATE);
35493582 if (ret)
@@ -3885,6 +3918,14 @@ static void reset_csb_pointers(struct intel_engine_cs *engine) · 335 unmodified lines
38853918
38863919 ring_set_paused(engine, 0);
38873920
3921+ /*
3922+ * Sometimes Icelake forgets to reset its pointers on a GPU reset.
3923+ * Bludgeon them with a mmio update to be sure.
3924+ */
3925+ ENGINE_WRITE(engine, RING_CONTEXT_STATUS_PTR,
3926+ 0xffff << 16 | reset_value << 8 | reset_value);
3927+ ENGINE_POSTING_READ(engine, RING_CONTEXT_STATUS_PTR);
3928+
38883929 /*
38893930 * After a reset, the HW starts writing into CSB entry [0]. We
38903931 * therefore have to set our HEAD pointer back one entry so that
@@ -3898,16 +3939,15 @@ static void reset_csb_pointers(struct intel_engine_cs *engine) · 7 unmodified lines
38983939 WRITE_ONCE(*execlists->csb_write, reset_value);
38993940 wmb(); /* Make sure this is visible to HW (paranoia?) */
39003941
3901− /*
3902− * Sometimes Icelake forgets to reset its pointers on a GPU reset.
3903− * Bludgeon them with a mmio update to be sure.
3904− */
3942+ invalidate_csb_entries(&execlists->csb_status[0],
3943+ &execlists->csb_status[reset_value]);
3944+
3945+ /* Once more for luck and our trusty paranoia */
39053946 ENGINE_WRITE(engine, RING_CONTEXT_STATUS_PTR,
3906− reset_value << 8 | reset_value);
3947+ 0xffff << 16 | reset_value << 8 | reset_value);
39073948 ENGINE_POSTING_READ(engine, RING_CONTEXT_STATUS_PTR);
39083949
3909− invalidate_csb_entries(&execlists->csb_status[0],
3910− &execlists->csb_status[reset_value]);
3950+ GEM_BUG_ON(READ_ONCE(*execlists->csb_write) != reset_value);
39113951 }
39123952
39133953 static void execlists_sanitize(struct intel_engine_cs *engine)
@@ -4074,6 +4114,8 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine) · 160 unmodified lines
40744114 */
40754115 ring_set_paused(engine, 1);
40764116 intel_engine_stop_cs(engine);
4117+
4118+ engine->execlists.reset_ccid = active_ccid(engine);
40774119 }
40784120
40794121 static void __reset_stop_ring(u32 *regs, const struct intel_engine_cs *engine)
@@ -4116,7 +4158,7 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 36 unmodified lines
41164158 * its request, it was still running at the time of the
41174159 * reset and will have been clobbered.
41184160 */
4119− rq = execlists_active(execlists);
4161+ rq = active_context(engine, engine->execlists.reset_ccid);
41204162 if (!rq)
41214163 goto unwind;
41224164
@@ -4166,8 +4208,6 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 43 unmodified lines
41664208 * image back to the expected values to skip over the guilty request.
41674209 */
41684210 __i915_request_reset(rq, stalled);
4169− if (!stalled)
4170− goto out_replay;
41714211
41724212 /*
41734213 * We want a simple context + ring to execute the breadcrumb update.
@@ -4177,9 +4217,6 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 3 unmodified lines
41774217 * future request will be after userspace has had the opportunity
41784218 * to recreate its own state.
41794219 */
4180− GEM_BUG_ON(!intel_context_is_pinned(ce));
4181− restore_default_state(ce, engine);
4182−
41834220 out_replay:
41844221 ENGINE_TRACE(engine, "replay {head:%04x, tail:%04x}\n",
41854222 head, ce->ring->tail);
@@ -4545,6 +4582,42 @@ static u32 preparser_disable(bool state) · 359 unmodified lines
45454582 return MI_ARB_CHECK | 1 << 8 | state;
45464583 }
45474584
4585+static i915_reg_t aux_inv_reg(const struct intel_engine_cs *engine)
4586+{
4587+ static const i915_reg_t vd[] = {
4588+ GEN12_VD0_AUX_NV,
4589+ GEN12_VD1_AUX_NV,
4590+ GEN12_VD2_AUX_NV,
4591+ GEN12_VD3_AUX_NV,
4592+ };
4593+
4594+ static const i915_reg_t ve[] = {
4595+ GEN12_VE0_AUX_NV,
4596+ GEN12_VE1_AUX_NV,
4597+ };
4598+
4599+ if (engine->class == VIDEO_DECODE_CLASS)
4600+ return vd[engine->instance];
4601+
4602+ if (engine->class == VIDEO_ENHANCEMENT_CLASS)
4603+ return ve[engine->instance];
4604+
4605+ GEM_BUG_ON("unknown aux_inv_reg\n");
4606+
4607+ return INVALID_MMIO_REG;
4608+}
4609+
4610+static u32 *
4611+gen12_emit_aux_table_inv(const i915_reg_t inv_reg, u32 *cs)
4612+{
4613+ *cs++ = MI_LOAD_REGISTER_IMM(1);
4614+ *cs++ = i915_mmio_reg_offset(inv_reg);
4615+ *cs++ = AUX_INV;
4616+ *cs++ = MI_NOOP;
4617+
4618+ return cs;
4619+}
4620+
45484621 static int gen12_emit_flush_render(struct i915_request *request,
45494622 u32 mode)
45504623 {
@@ -4553,13 +4626,13 @@ static int gen12_emit_flush_render(struct i915_request *request, · 2 unmodified lines
45534626 u32 *cs;
45544627
45554628 flags |= PIPE_CONTROL_TILE_CACHE_FLUSH;
4629+ flags |= PIPE_CONTROL_FLUSH_L3;
45564630 flags |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
45574631 flags |= PIPE_CONTROL_DEPTH_CACHE_FLUSH;
45584632 /* Wa_1409600907:tgl */
45594633 flags |= PIPE_CONTROL_DEPTH_STALL;
45604634 flags |= PIPE_CONTROL_DC_FLUSH_ENABLE;
45614635 flags |= PIPE_CONTROL_FLUSH_ENABLE;
4562− flags |= PIPE_CONTROL_HDC_PIPELINE_FLUSH;
45634636
45644637 flags |= PIPE_CONTROL_STORE_DATA_INDEX;
45654638 flags |= PIPE_CONTROL_QW_WRITE;
@@ -4570,7 +4643,9 @@ static int gen12_emit_flush_render(struct i915_request *request, · 4 unmodified lines
45704643 if (IS_ERR(cs))
45714644 return PTR_ERR(cs);
45724645
4573− cs = gen8_emit_pipe_control(cs, flags, LRC_PPHWSP_SCRATCH_ADDR);
4646+ cs = gen12_emit_pipe_control(cs,
4647+ PIPE_CONTROL0_HDC_PIPELINE_FLUSH,
4648+ flags, LRC_PPHWSP_SCRATCH_ADDR);
45744649 intel_ring_advance(request, cs);
45754650 }
45764651
@@ -4585,14 +4660,13 @@ static int gen12_emit_flush_render(struct i915_request *request, · 8 unmodified lines
45854660 flags |= PIPE_CONTROL_VF_CACHE_INVALIDATE;
45864661 flags |= PIPE_CONTROL_CONST_CACHE_INVALIDATE;
45874662 flags |= PIPE_CONTROL_STATE_CACHE_INVALIDATE;
4588− flags |= PIPE_CONTROL_L3_RO_CACHE_INVALIDATE;
45894663
45904664 flags |= PIPE_CONTROL_STORE_DATA_INDEX;
45914665 flags |= PIPE_CONTROL_QW_WRITE;
45924666
45934667 flags |= PIPE_CONTROL_CS_STALL;
45944668
4595− cs = intel_ring_begin(request, 8);
4669+ cs = intel_ring_begin(request, 8 + 4);
45964670 if (IS_ERR(cs))
45974671 return PTR_ERR(cs);
45984672
@@ -4605,6 +4679,9 @@ static int gen12_emit_flush_render(struct i915_request *request, · 6 unmodified lines
46054679
46064680 cs = gen8_emit_pipe_control(cs, flags, LRC_PPHWSP_SCRATCH_ADDR);
46074681
4682+ /* hsdes: 1809175790 */
4683+ cs = gen12_emit_aux_table_inv(GEN12_GFX_CCS_AUX_NV, cs);
4684+
46084685 *cs++ = preparser_disable(false);
46094686 intel_ring_advance(request, cs);
46104687 }
@@ -4612,6 +4689,56 @@ static int gen12_emit_flush_render(struct i915_request *request, · 1 unmodified line
46124689 return 0;
46134690 }
46144691
4692+static int gen12_emit_flush(struct i915_request *request, u32 mode)
4693+{
4694+ intel_engine_mask_t aux_inv = 0;
4695+ u32 cmd, *cs;
4696+
4697+ if (mode & EMIT_INVALIDATE)
4698+ aux_inv = request->engine->mask & ~BIT(BCS0);
4699+
4700+ cs = intel_ring_begin(request,
4701+ 4 + (aux_inv ? 2 * hweight8(aux_inv) + 2 : 0));
4702+ if (IS_ERR(cs))
4703+ return PTR_ERR(cs);
4704+
4705+ cmd = MI_FLUSH_DW + 1;
4706+
4707+ /* We always require a command barrier so that subsequent
4708+ * commands, such as breadcrumb interrupts, are strictly ordered
4709+ * wrt the contents of the write cache being flushed to memory
4710+ * (and thus being coherent from the CPU).
4711+ */
4712+ cmd |= MI_FLUSH_DW_STORE_INDEX | MI_FLUSH_DW_OP_STOREDW;
4713+
4714+ if (mode & EMIT_INVALIDATE) {
4715+ cmd |= MI_INVALIDATE_TLB;
4716+ if (request->engine->class == VIDEO_DECODE_CLASS)
4717+ cmd |= MI_INVALIDATE_BSD;
4718+ }
4719+
4720+ *cs++ = cmd;
4721+ *cs++ = LRC_PPHWSP_SCRATCH_ADDR;
4722+ *cs++ = 0; /* upper addr */
4723+ *cs++ = 0; /* value */
4724+
4725+ if (aux_inv) { /* hsdes: 1809175790 */
4726+ struct intel_engine_cs *engine;
4727+ unsigned int tmp;
4728+
4729+ *cs++ = MI_LOAD_REGISTER_IMM(hweight8(aux_inv));
4730+ for_each_engine_masked(engine, request->engine->gt,
4731+ aux_inv, tmp) {
4732+ *cs++ = i915_mmio_reg_offset(aux_inv_reg(engine));
4733+ *cs++ = AUX_INV;
4734+ }
4735+ *cs++ = MI_NOOP;
4736+ }
4737+ intel_ring_advance(request, cs);
4738+
4739+ return 0;
4740+}
4741+
46154742 /*
46164743 * Reserve space for 2 NOOPs at the end of each request to be
46174744 * used as a workaround for not being allowed to do lite
@@ -4641,8 +4768,7 @@ static u32 *emit_preempt_busywait(struct i915_request *request, u32 *cs) · 23 unmodified lines
46414768 }
46424769
46434770 static __always_inline u32*
4644−gen8_emit_fini_breadcrumb_footer(struct i915_request *request,
4645− u32 *cs)
4771+gen8_emit_fini_breadcrumb_tail(struct i915_request *request, u32 *cs)
46464772 {
46474773 *cs++ = MI_USER_INTERRUPT;
46484774
@@ -4656,14 +4782,16 @@ gen8_emit_fini_breadcrumb_footer(struct i915_request *request, · 7 unmodified lines
46564782 return gen8_emit_wa_tail(request, cs);
46574783 }
46584784
4659−static u32 *gen8_emit_fini_breadcrumb(struct i915_request *request, u32 *cs)
4785+static u32 *emit_xcs_breadcrumb(struct i915_request *request, u32 *cs)
46604786 {
4661− cs = gen8_emit_ggtt_write(cs,
4662− request->fence.seqno,
4663− i915_request_active_timeline(request)->hwsp_offset,
4664− 0);
4787+ u32 addr = i915_request_active_timeline(request)->hwsp_offset;
4788+
4789+ return gen8_emit_ggtt_write(cs, request->fence.seqno, addr, 0);
4790+}
46654791
4666− return gen8_emit_fini_breadcrumb_footer(request, cs);
4792+static u32 *gen8_emit_fini_breadcrumb(struct i915_request *rq, u32 *cs)
4793+{
4794+ return gen8_emit_fini_breadcrumb_tail(rq, emit_xcs_breadcrumb(rq, cs));
46674795 }
46684796
46694797 static u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs)
@@ -4681,7 +4809,7 @@ static u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs) · 11 unmodified lines
46814809 PIPE_CONTROL_FLUSH_ENABLE |
46824810 PIPE_CONTROL_CS_STALL);
46834811
4684− return gen8_emit_fini_breadcrumb_footer(request, cs);
4812+ return gen8_emit_fini_breadcrumb_tail(request, cs);
46854813 }
46864814
46874815 static u32 *
@@ -4697,7 +4825,7 @@ gen11_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs) · 9 unmodified lines
46974825 PIPE_CONTROL_DC_FLUSH_ENABLE |
46984826 PIPE_CONTROL_FLUSH_ENABLE);
46994827
4700− return gen8_emit_fini_breadcrumb_footer(request, cs);
4828+ return gen8_emit_fini_breadcrumb_tail(request, cs);
47014829 }
47024830
47034831 /*
@@ -4735,7 +4863,7 @@ static u32 *gen12_emit_preempt_busywait(struct i915_request *request, u32 *cs) · 31 unmodified lines
47354863 }
47364864
47374865 static __always_inline u32*
4738−gen12_emit_fini_breadcrumb_footer(struct i915_request *request, u32 *cs)
4866+gen12_emit_fini_breadcrumb_tail(struct i915_request *request, u32 *cs)
47394867 {
47404868 *cs++ = MI_USER_INTERRUPT;
47414869
@@ -4749,33 +4877,29 @@ gen12_emit_fini_breadcrumb_footer(struct i915_request *request, u32 *cs) · 7 unmodified lines
47494877 return gen8_emit_wa_tail(request, cs);
47504878 }
47514879
4752−static u32 *gen12_emit_fini_breadcrumb(struct i915_request *request, u32 *cs)
4880+static u32 *gen12_emit_fini_breadcrumb(struct i915_request *rq, u32 *cs)
47534881 {
4754− cs = gen8_emit_ggtt_write(cs,
4755− request->fence.seqno,
4756− i915_request_active_timeline(request)->hwsp_offset,
4757− 0);
4758−
4759− return gen12_emit_fini_breadcrumb_footer(request, cs);
4882+ return gen12_emit_fini_breadcrumb_tail(rq, emit_xcs_breadcrumb(rq, cs));
47604883 }
47614884
47624885 static u32 *
47634886 gen12_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs)
47644887 {
4765− cs = gen8_emit_ggtt_write_rcs(cs,
4766− request->fence.seqno,
4767− i915_request_active_timeline(request)->hwsp_offset,
4768− PIPE_CONTROL_CS_STALL |
4769− PIPE_CONTROL_TILE_CACHE_FLUSH |
4770− PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
4771− PIPE_CONTROL_DEPTH_CACHE_FLUSH |
4772− /* Wa_1409600907:tgl */
4773− PIPE_CONTROL_DEPTH_STALL |
4774− PIPE_CONTROL_DC_FLUSH_ENABLE |
4775− PIPE_CONTROL_FLUSH_ENABLE |
4776− PIPE_CONTROL_HDC_PIPELINE_FLUSH);
4888+ cs = gen12_emit_ggtt_write_rcs(cs,
4889+ request->fence.seqno,
4890+ i915_request_active_timeline(request)->hwsp_offset,
4891+ PIPE_CONTROL0_HDC_PIPELINE_FLUSH,
4892+ PIPE_CONTROL_CS_STALL |
4893+ PIPE_CONTROL_TILE_CACHE_FLUSH |
4894+ PIPE_CONTROL_FLUSH_L3 |
4895+ PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
4896+ PIPE_CONTROL_DEPTH_CACHE_FLUSH |
4897+ /* Wa_1409600907:tgl */
4898+ PIPE_CONTROL_DEPTH_STALL |
4899+ PIPE_CONTROL_DC_FLUSH_ENABLE |
4900+ PIPE_CONTROL_FLUSH_ENABLE);
47774901
4778− return gen12_emit_fini_breadcrumb_footer(request, cs);
4902+ return gen12_emit_fini_breadcrumb_tail(request, cs);
47794903 }
47804904
47814905 static void execlists_park(struct intel_engine_cs *engine)
@@ -4801,8 +4925,11 @@ void intel_execlists_set_default_submission(struct intel_engine_cs *engine) · 19 unmodified lines
48014925 engine->flags |= I915_ENGINE_SUPPORTS_STATS;
48024926 if (!intel_vgpu_active(engine->i915)) {
48034927 engine->flags |= I915_ENGINE_HAS_SEMAPHORES;
4804− if (HAS_LOGICAL_RING_PREEMPTION(engine->i915))
4928+ if (HAS_LOGICAL_RING_PREEMPTION(engine->i915)) {
48054929 engine->flags |= I915_ENGINE_HAS_PREEMPTION;
4930+ if (IS_ACTIVE(CONFIG_DRM_I915_TIMESLICE_DURATION))
4931+ engine->flags |= I915_ENGINE_HAS_TIMESLICES;
4932+ }
48064933 }
48074934
48084935 if (INTEL_GEN(engine->i915) >= 12)
@@ -4845,9 +4972,10 @@ logical_ring_default_vfuncs(struct intel_engine_cs *engine) · 36 unmodified lines
48454972 engine->emit_flush = gen8_emit_flush;
48464973 engine->emit_init_breadcrumb = gen8_emit_init_breadcrumb;
48474974 engine->emit_fini_breadcrumb = gen8_emit_fini_breadcrumb;
4848− if (INTEL_GEN(engine->i915) >= 12)
4975+ if (INTEL_GEN(engine->i915) >= 12) {
48494976 engine->emit_fini_breadcrumb = gen12_emit_fini_breadcrumb;
4850−
4977+ engine->emit_flush = gen12_emit_flush;
4978+ }
48514979 engine->set_default_submission = intel_execlists_set_default_submission;
48524980
48534981 if (INTEL_GEN(engine->i915) < 11) {
drivers/gpu/drm/i915/gt/intel_rc6.c+1−7
@@ -113,7 +113,6 @@ static void gen9_rc6_enable(struct intel_rc6 *rc6) · 112 unmodified lines
113113 struct intel_uncore *uncore = rc6_to_uncore(rc6);
114114 struct intel_engine_cs *engine;
115115 enum intel_engine_id id;
116− u32 rc6_mode;
117116
118117 /* 2b: Program RC6 thresholds.*/
119118 if (INTEL_GEN(rc6_to_i915(rc6)) >= 10) {
@@ -165,16 +164,11 @@ static void gen9_rc6_enable(struct intel_rc6 *rc6) · 45 unmodified lines
165164 /* 3a: Enable RC6 */
166165 set(uncore, GEN6_RC6_THRESHOLD, 37500); /* 37.5/125ms per EI */
167166
168− /* WaRsUseTimeoutMode:cnl (pre-prod) */
169− if (IS_CNL_REVID(rc6_to_i915(rc6), CNL_REVID_A0, CNL_REVID_C0))
170− rc6_mode = GEN7_RC_CTL_TO_MODE;
171− else
172− rc6_mode = GEN6_RC_CTL_EI_MODE(1);
173167
174168 rc6->ctl_enable =
175169 GEN6_RC_CTL_HW_ENABLE |
176170 GEN6_RC_CTL_RC6_ENABLE |
177− rc6_mode;
171+ GEN6_RC_CTL_EI_MODE(1);
178172
179173 /*
180174 * WaRsDisableCoarsePowerGating:skl,cnl
drivers/gpu/drm/i915/gt/intel_renderstate.c+9−7
@@ -219,6 +219,14 @@ int intel_renderstate_emit(struct intel_renderstate *so, · 218 unmodified lines
219219 if (!so->vma)
220220 return 0;
221221
222+ i915_vma_lock(so->vma);
223+ err = i915_request_await_object(rq, so->vma->obj, false);
224+ if (err == 0)
225+ err = i915_vma_move_to_active(so->vma, rq, 0);
226+ i915_vma_unlock(so->vma);
227+ if (err)
228+ return err;
229+
222230 err = engine->emit_bb_start(rq,
223231 so->batch_offset, so->batch_size,
224232 I915_DISPATCH_SECURE);
@@ -233,13 +241,7 @@ int intel_renderstate_emit(struct intel_renderstate *so, · 8 unmodified lines
233241 return err;
234242 }
235243
236− i915_vma_lock(so->vma);
237− err = i915_request_await_object(rq, so->vma->obj, false);
238− if (err == 0)
239− err = i915_vma_move_to_active(so->vma, rq, 0);
240− i915_vma_unlock(so->vma);
241−
242− return err;
244+ return 0;
243245 }
244246
245247 void intel_renderstate_fini(struct intel_renderstate *so)
drivers/gpu/drm/i915/gt/intel_rps.c+4−1
@@ -1844,8 +1844,11 @@ void intel_rps_init(struct intel_rps *rps) · 1843 unmodified lines
18441844
18451845 if (INTEL_GEN(i915) >= 8 && INTEL_GEN(i915) < 11)
18461846 rps->pm_intrmsk_mbz |= GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC;
1847+}
18471848
1848− if (INTEL_GEN(i915) >= 6)
1849+void intel_rps_sanitize(struct intel_rps *rps)
1850+{
1851+ if (INTEL_GEN(rps_to_i915(rps)) >= 6)
18491852 rps_disable_interrupts(rps);
18501853 }
18511854
drivers/gpu/drm/i915/gt/intel_rps.h+1−0
@@ -13,6 +13,7 @@ struct i915_request; · 12 unmodified lines
1313
1414 void intel_rps_init_early(struct intel_rps *rps);
1515 void intel_rps_init(struct intel_rps *rps);
16+void intel_rps_sanitize(struct intel_rps *rps);
1617
1718 void intel_rps_driver_register(struct intel_rps *rps);
1819 void intel_rps_driver_unregister(struct intel_rps *rps);
drivers/gpu/drm/i915/gt/intel_timeline.c+4−4
@@ -211,9 +211,9 @@ static void cacheline_free(struct intel_timeline_cacheline *cl) · 210 unmodified lines
211211 i915_active_release(&cl->active);
212212 }
213213
214−int intel_timeline_init(struct intel_timeline *timeline,
215− struct intel_gt *gt,
216− struct i915_vma *hwsp)
214+static int intel_timeline_init(struct intel_timeline *timeline,
215+ struct intel_gt *gt,
216+ struct i915_vma *hwsp)
217217 {
218218 void *vaddr;
219219
@@ -280,7 +280,7 @@ void intel_gt_init_timelines(struct intel_gt *gt) · 60 unmodified lines
280280 INIT_LIST_HEAD(&timelines->hwsp_free_list);
281281 }
282282
283−void intel_timeline_fini(struct intel_timeline *timeline)
283+static void intel_timeline_fini(struct intel_timeline *timeline)
284284 {
285285 GEM_BUG_ON(atomic_read(&timeline->pin_count));
286286 GEM_BUG_ON(!list_empty(&timeline->requests));
drivers/gpu/drm/i915/gt/intel_timeline.h+0−5
@@ -31,11 +31,6 @@ · 30 unmodified lines
3131 #include "i915_syncmap.h"
3232 #include "gt/intel_timeline_types.h"
3333
34−int intel_timeline_init(struct intel_timeline *tl,
35− struct intel_gt *gt,
36− struct i915_vma *hwsp);
37−void intel_timeline_fini(struct intel_timeline *tl);
38−
3934 struct intel_timeline *
4035 intel_timeline_create(struct intel_gt *gt, struct i915_vma *global_hwsp);
4136
drivers/gpu/drm/i915/gt/intel_workarounds.c+7−21
@@ -485,25 +485,14 @@ static void cfl_ctx_workarounds_init(struct intel_engine_cs *engine, · 484 unmodified lines
485485 static void cnl_ctx_workarounds_init(struct intel_engine_cs *engine,
486486 struct i915_wa_list *wal)
487487 {
488− struct drm_i915_private *i915 = engine->i915;
489−
490488 /* WaForceContextSaveRestoreNonCoherent:cnl */
491489 WA_SET_BIT_MASKED(CNL_HDC_CHICKEN0,
492490 HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT);
493491
494− /* WaThrottleEUPerfToAvoidTDBackPressure:cnl(pre-prod) */
495− if (IS_CNL_REVID(i915, CNL_REVID_B0, CNL_REVID_B0))
496− WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, THROTTLE_12_5);
497−
498492 /* WaDisableReplayBufferBankArbitrationOptimization:cnl */
499493 WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
500494 GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
501495
502− /* WaDisableEnhancedSBEVertexCaching:cnl (pre-prod) */
503− if (IS_CNL_REVID(i915, 0, CNL_REVID_B0))
504− WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
505− GEN8_CSC2_SBE_VUE_CACHE_CONSERVATIVE);
506−
507496 /* WaPushConstantDereferenceHoldDisable:cnl */
508497 WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2, PUSH_CONSTANT_DEREF_DISABLE);
509498
@@ -872,12 +861,6 @@ cnl_gt_workarounds_init(struct drm_i915_private *i915, struct i915_wa_list *wal) · 362 unmodified lines
872861 {
873862 wa_init_mcr(i915, wal);
874863
875− /* WaDisableI2mCycleOnWRPort:cnl (pre-prod) */
876− if (IS_CNL_REVID(i915, CNL_REVID_B0, CNL_REVID_B0))
877− wa_write_or(wal,
878− GAMT_CHKN_BIT_REG,
879− GAMT_CHKN_DISABLE_I2M_CYCLE_ON_WR_PORT);
880−
881864 /* WaInPlaceDecompressionHang:cnl */
882865 wa_write_or(wal,
883866 GEN9_GAMT_ECO_REG_RW_IA,
@@ -934,10 +917,13 @@ icl_gt_workarounds_init(struct drm_i915_private *i915, struct i915_wa_list *wal) · 50 unmodified lines
934917 GAMT_CHKN_BIT_REG,
935918 GAMT_CHKN_DISABLE_L3_COH_PIPE);
936919
937− /* Wa_1607087056:icl */
938− wa_write_or(wal,
939− SLICE_UNIT_LEVEL_CLKGATE,
940− L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
920+ /* Wa_1607087056:icl,ehl,jsl */
921+ if (IS_ICELAKE(i915) ||
922+ IS_EHL_REVID(i915, EHL_REVID_A0, EHL_REVID_A0)) {
923+ wa_write_or(wal,
924+ SLICE_UNIT_LEVEL_CLKGATE,
925+ L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
926+ }
941927 }
942928
943929 static void
drivers/gpu/drm/i915/gt/mock_engine.c+0−2
@@ -28,7 +28,6 @@ · 27 unmodified lines
2828 #include "i915_drv.h"
2929 #include "intel_context.h"
3030 #include "intel_engine_pm.h"
31−#include "intel_engine_pool.h"
3231
3332 #include "mock_engine.h"
3433 #include "selftests/mock_request.h"
@@ -328,7 +327,6 @@ int mock_engine_init(struct intel_engine_cs *engine) · 293 unmodified lines
328327 intel_engine_init_execlists(engine);
329328 intel_engine_init__pm(engine);
330329 intel_engine_init_retire(engine);
331− intel_engine_pool_init(&engine->pool);
332330
333331 ce = create_kernel_context(engine);
334332 if (IS_ERR(ce))
drivers/gpu/drm/i915/gt/selftest_context.c+4−8
@@ -24,6 +24,7 @@ static int request_sync(struct i915_request *rq) · 23 unmodified lines
2424
2525 /* Opencode i915_request_add() so we can keep the timeline locked. */
2626 __i915_request_commit(rq);
27+ rq->sched.attr.priority = I915_PRIORITY_BARRIER;
2728 __i915_request_queue(rq, NULL);
2829
2930 timeout = i915_request_wait(rq, 0, HZ / 10);
@@ -154,10 +155,7 @@ static int live_context_size(void *arg) · 124 unmodified lines
154155 */
155156
156157 for_each_engine(engine, gt, id) {
157− struct {
158− struct file *state;
159− void *pinned;
160− } saved;
158+ struct file *saved;
161159
162160 if (!engine->context_size)
163161 continue;
@@ -171,8 +169,7 @@ static int live_context_size(void *arg) · 7 unmodified lines
171169 * active state is sufficient, we are only checking that we
172170 * don't use more than we planned.
173171 */
174− saved.state = fetch_and_zero(&engine->default_state);
175− saved.pinned = fetch_and_zero(&engine->pinned_default_state);
172+ saved = fetch_and_zero(&engine->default_state);
176173
177174 /* Overlaps with the execlists redzone */
178175 engine->context_size += I915_GTT_PAGE_SIZE;
@@ -181,8 +178,7 @@ static int live_context_size(void *arg) · 2 unmodified lines
181178
182179 engine->context_size -= I915_GTT_PAGE_SIZE;
183180
184− engine->pinned_default_state = saved.pinned;
185− engine->default_state = saved.state;
181+ engine->default_state = saved;
186182
187183 intel_engine_pm_put(engine);
188184
drivers/gpu/drm/i915/gt/selftest_lrc.c+42−44
@@ -4342,35 +4342,6 @@ int intel_execlists_live_selftests(struct drm_i915_private *i915) · 4341 unmodified lines
43424342 return intel_gt_live_subtests(tests, &i915->gt);
43434343 }
43444344
4345−static void hexdump(const void *buf, size_t len)
4346−{
4347− const size_t rowsize = 8 * sizeof(u32);
4348− const void *prev = NULL;
4349− bool skip = false;
4350− size_t pos;
4351−
4352− for (pos = 0; pos < len; pos += rowsize) {
4353− char line[128];
4354−
4355− if (prev && !memcmp(prev, buf + pos, rowsize)) {
4356− if (!skip) {
4357− pr_info("*\n");
4358− skip = true;
4359− }
4360− continue;
4361− }
4362−
4363− WARN_ON_ONCE(hex_dump_to_buffer(buf + pos, len - pos,
4364− rowsize, sizeof(u32),
4365− line, sizeof(line),
4366− false) >= sizeof(line));
4367− pr_info("[%04zx] %s\n", pos, line);
4368−
4369− prev = buf + pos;
4370− skip = false;
4371− }
4372−}
4373−
43744345 static int emit_semaphore_signal(struct intel_context *ce, void *slot)
43754346 {
43764347 const u32 offset =
@@ -4518,10 +4489,10 @@ static int live_lrc_layout(void *arg) · 141 unmodified lines
45184489
45194490 if (err) {
45204491 pr_info("%s: HW register image:\n", engine->name);
4521− hexdump(hw, PAGE_SIZE);
4492+ igt_hexdump(hw, PAGE_SIZE);
45224493
45234494 pr_info("%s: SW register image:\n", engine->name);
4524− hexdump(lrc, PAGE_SIZE);
4495+ igt_hexdump(lrc, PAGE_SIZE);
45254496 }
45264497
45274498 shmem_unpin_map(engine->default_state, hw);
@@ -5206,6 +5177,7 @@ store_context(struct intel_context *ce, struct i915_vma *scratch) · 678 unmodified lines
52065177 {
52075178 struct i915_vma *batch;
52085179 u32 dw, x, *cs, *hw;
5180+ u32 *defaults;
52095181
52105182 batch = create_user_vma(ce->vm, SZ_64K);
52115183 if (IS_ERR(batch))
@@ -5217,9 +5189,16 @@ store_context(struct intel_context *ce, struct i915_vma *scratch) · 5 unmodified lines
52175189 return ERR_CAST(cs);
52185190 }
52195191
5192+ defaults = shmem_pin_map(ce->engine->default_state);
5193+ if (!defaults) {
5194+ i915_gem_object_unpin_map(batch->obj);
5195+ i915_vma_put(batch);
5196+ return ERR_PTR(-ENOMEM);
5197+ }
5198+
52205199 x = 0;
52215200 dw = 0;
5222− hw = ce->engine->pinned_default_state;
5201+ hw = defaults;
52235202 hw += LRC_STATE_OFFSET / sizeof(*hw);
52245203 do {
52255204 u32 len = hw[dw] & 0x7f;
@@ -5250,6 +5229,8 @@ store_context(struct intel_context *ce, struct i915_vma *scratch) · 24 unmodified lines
52505229
52515230 *cs++ = MI_BATCH_BUFFER_END;
52525231
5232+ shmem_unpin_map(ce->engine->default_state, defaults);
5233+
52535234 i915_gem_object_flush_map(batch->obj);
52545235 i915_gem_object_unpin_map(batch->obj);
52555236
@@ -5360,6 +5341,7 @@ static struct i915_vma *load_context(struct intel_context *ce, u32 poison) · 104 unmodified lines
53605341 {
53615342 struct i915_vma *batch;
53625343 u32 dw, *cs, *hw;
5344+ u32 *defaults;
53635345
53645346 batch = create_user_vma(ce->vm, SZ_64K);
53655347 if (IS_ERR(batch))
@@ -5371,8 +5353,15 @@ static struct i915_vma *load_context(struct intel_context *ce, u32 poison) · 5 unmodified lines
53715353 return ERR_CAST(cs);
53725354 }
53735355
5356+ defaults = shmem_pin_map(ce->engine->default_state);
5357+ if (!defaults) {
5358+ i915_gem_object_unpin_map(batch->obj);
5359+ i915_vma_put(batch);
5360+ return ERR_PTR(-ENOMEM);
5361+ }
5362+
53745363 dw = 0;
5375− hw = ce->engine->pinned_default_state;
5364+ hw = defaults;
53765365 hw += LRC_STATE_OFFSET / sizeof(*hw);
53775366 do {
53785367 u32 len = hw[dw] & 0x7f;
@@ -5400,6 +5389,8 @@ static struct i915_vma *load_context(struct intel_context *ce, u32 poison) · 21 unmodified lines
54005389
54015390 *cs++ = MI_BATCH_BUFFER_END;
54025391
5392+ shmem_unpin_map(ce->engine->default_state, defaults);
5393+
54035394 i915_gem_object_flush_map(batch->obj);
54045395 i915_gem_object_unpin_map(batch->obj);
54055396
@@ -5467,6 +5458,7 @@ static int compare_isolation(struct intel_engine_cs *engine, · 61 unmodified lines
54675458 {
54685459 u32 x, dw, *hw, *lrc;
54695460 u32 *A[2], *B[2];
5461+ u32 *defaults;
54705462 int err = 0;
54715463
54725464 A[0] = i915_gem_object_pin_map(ref[0]->obj, I915_MAP_WC);
@@ -5499,9 +5491,15 @@ static int compare_isolation(struct intel_engine_cs *engine, · 26 unmodified lines
54995491 }
55005492 lrc += LRC_STATE_OFFSET / sizeof(*hw);
55015493
5494+ defaults = shmem_pin_map(ce->engine->default_state);
5495+ if (!defaults) {
5496+ err = -ENOMEM;
5497+ goto err_lrc;
5498+ }
5499+
55025500 x = 0;
55035501 dw = 0;
5504− hw = engine->pinned_default_state;
5502+ hw = defaults;
55055503 hw += LRC_STATE_OFFSET / sizeof(*hw);
55065504 do {
55075505 u32 len = hw[dw] & 0x7f;
@@ -5541,6 +5539,8 @@ static int compare_isolation(struct intel_engine_cs *engine, · 33 unmodified lines
55415539 } while (dw < PAGE_SIZE / sizeof(u32) &&
55425540 (hw[dw] & ~BIT(0)) != MI_BATCH_BUFFER_END);
55435541
5542+ shmem_unpin_map(ce->engine->default_state, defaults);
5543+err_lrc:
55445544 i915_gem_object_unpin_map(ce->state->obj);
55455545 err_B1:
55465546 i915_gem_object_unpin_map(result[1]->obj);
@@ -5690,18 +5690,16 @@ static int live_lrc_isolation(void *arg) · 143 unmodified lines
56905690 continue;
56915691
56925692 intel_engine_pm_get(engine);
5693− if (engine->pinned_default_state) {
5694− for (i = 0; i < ARRAY_SIZE(poison); i++) {
5695− int result;
5693+ for (i = 0; i < ARRAY_SIZE(poison); i++) {
5694+ int result;
56965695
5697− result = __lrc_isolation(engine, poison[i]);
5698− if (result && !err)
5699− err = result;
5696+ result = __lrc_isolation(engine, poison[i]);
5697+ if (result && !err)
5698+ err = result;
57005699
5701− result = __lrc_isolation(engine, ~poison[i]);
5702− if (result && !err)
5703− err = result;
5704− }
5700+ result = __lrc_isolation(engine, ~poison[i]);
5701+ if (result && !err)
5702+ err = result;
57055703 }
57065704 intel_engine_pm_put(engine);
57075705 if (igt_flush_test(gt->i915)) {
drivers/gpu/drm/i915/gt/selftest_ring_submission.c+2−0
@@ -54,6 +54,8 @@ static struct i915_vma *create_wally(struct intel_engine_cs *engine) · 53 unmodified lines
5454 *cs++ = STACK_MAGIC;
5555
5656 *cs++ = MI_BATCH_BUFFER_END;
57+
58+ i915_gem_object_flush_map(obj);
5759 i915_gem_object_unpin_map(obj);
5860
5961 vma->private = intel_context_create(engine); /* dummy residuals */
drivers/gpu/drm/i915/gt/selftest_rps.c+42−14
@@ -56,6 +56,18 @@ static int cmp_u64(const void *A, const void *B) · 55 unmodified lines
5656 return 0;
5757 }
5858
59+static int cmp_u32(const void *A, const void *B)
60+{
61+ const u32 *a = A, *b = B;
62+
63+ if (a < b)
64+ return -1;
65+ else if (a > b)
66+ return 1;
67+ else
68+ return 0;
69+}
70+
5971 static struct i915_vma *
6072 create_spin_counter(struct intel_engine_cs *engine,
6173 struct i915_address_space *vm,
@@ -236,8 +248,8 @@ int live_rps_clock_interval(void *arg) · 174 unmodified lines
236248 for_each_engine(engine, gt, id) {
237249 unsigned long saved_heartbeat;
238250 struct i915_request *rq;
239− ktime_t dt;
240251 u32 cycles;
252+ u64 dt;
241253
242254 if (!intel_engine_can_store_dword(engine))
243255 continue;
@@ -286,15 +298,29 @@ int live_rps_clock_interval(void *arg) · 42 unmodified lines
286298 engine->name);
287299 err = -ENODEV;
288300 } else {
289− preempt_disable();
290− dt = ktime_get();
291− cycles = -intel_uncore_read_fw(gt->uncore,
292− GEN6_RP_CUR_UP_EI);
293− udelay(1000);
294− dt = ktime_sub(ktime_get(), dt);
295− cycles += intel_uncore_read_fw(gt->uncore,
296− GEN6_RP_CUR_UP_EI);
297− preempt_enable();
301+ ktime_t dt_[5];
302+ u32 cycles_[5];
303+ int i;
304+
305+ for (i = 0; i < 5; i++) {
306+ preempt_disable();
307+
308+ dt_[i] = ktime_get();
309+ cycles_[i] = -intel_uncore_read_fw(gt->uncore, GEN6_RP_CUR_UP_EI);
310+
311+ udelay(1000);
312+
313+ dt_[i] = ktime_sub(ktime_get(), dt_[i]);
314+ cycles_[i] += intel_uncore_read_fw(gt->uncore, GEN6_RP_CUR_UP_EI);
315+
316+ preempt_enable();
317+ }
318+
319+ /* Use the median of both cycle/dt; close enough */
320+ sort(cycles_, 5, sizeof(*cycles_), cmp_u32, NULL);
321+ cycles = (cycles_[1] + 2 * cycles_[2] + cycles_[3]) / 4;
322+ sort(dt_, 5, sizeof(*dt_), cmp_u64, NULL);
323+ dt = div_u64(dt_[1] + 2 * dt_[2] + dt_[3], 4);
298324 }
299325
300326 intel_uncore_write_fw(gt->uncore, GEN6_RP_CONTROL, 0);
@@ -306,14 +332,14 @@ int live_rps_clock_interval(void *arg) · 5 unmodified lines
306332 if (err == 0) {
307333 u64 time = intel_gt_pm_interval_to_ns(gt, cycles);
308334 u32 expected =
309− intel_gt_ns_to_pm_interval(gt, ktime_to_ns(dt));
335+ intel_gt_ns_to_pm_interval(gt, dt);
310336
311337 pr_info("%s: rps counted %d C0 cycles [%lldns] in %lldns [%d cycles], using GT clock frequency of %uKHz\n",
312− engine->name, cycles, time, ktime_to_ns(dt), expected,
338+ engine->name, cycles, time, dt, expected,
313339 gt->clock_frequency / 1000);
314340
315− if (10 * time < 8 * ktime_to_ns(dt) ||
316− 8 * time > 10 * ktime_to_ns(dt)) {
341+ if (10 * time < 8 * dt ||
342+ 8 * time > 10 * dt) {
317343 pr_err("%s: rps clock time does not match walltime!\n",
318344 engine->name);
319345 err = -EINVAL;
@@ -701,6 +727,7 @@ int live_rps_frequency_cs(void *arg) · 381 unmodified lines
701727
702728 err_vma:
703729 *cancel = MI_BATCH_BUFFER_END;
730+ i915_gem_object_flush_map(vma->obj);
704731 i915_gem_object_unpin_map(vma->obj);
705732 i915_vma_unpin(vma);
706733 i915_vma_put(vma);
@@ -842,6 +869,7 @@ int live_rps_frequency_srm(void *arg) · 135 unmodified lines
842869
843870 err_vma:
844871 *cancel = MI_BATCH_BUFFER_END;
872+ i915_gem_object_flush_map(vma->obj);
845873 i915_gem_object_unpin_map(vma->obj);
846874 i915_vma_unpin(vma);
847875 i915_vma_put(vma);
drivers/gpu/drm/i915/gt/sysfs_engines.c+94−0
@@ -191,6 +191,17 @@ max_spin_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) · 190 unmodified lines
191191 static struct kobj_attribute max_spin_attr =
192192 __ATTR(max_busywait_duration_ns, 0644, max_spin_show, max_spin_store);
193193
194+static ssize_t
195+max_spin_default(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
196+{
197+ struct intel_engine_cs *engine = kobj_to_engine(kobj);
198+
199+ return sprintf(buf, "%lu\n", engine->defaults.max_busywait_duration_ns);
200+}
201+
202+static struct kobj_attribute max_spin_def =
203+__ATTR(max_busywait_duration_ns, 0444, max_spin_default, NULL);
204+
194205 static ssize_t
195206 timeslice_store(struct kobject *kobj, struct kobj_attribute *attr,
196207 const char *buf, size_t count)
@@ -233,6 +244,17 @@ timeslice_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) · 36 unmodified lines
233244 static struct kobj_attribute timeslice_duration_attr =
234245 __ATTR(timeslice_duration_ms, 0644, timeslice_show, timeslice_store);
235246
247+static ssize_t
248+timeslice_default(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
249+{
250+ struct intel_engine_cs *engine = kobj_to_engine(kobj);
251+
252+ return sprintf(buf, "%lu\n", engine->defaults.timeslice_duration_ms);
253+}
254+
255+static struct kobj_attribute timeslice_duration_def =
256+__ATTR(timeslice_duration_ms, 0444, timeslice_default, NULL);
257+
236258 static ssize_t
237259 stop_store(struct kobject *kobj, struct kobj_attribute *attr,
238260 const char *buf, size_t count)
@@ -272,6 +294,17 @@ stop_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) · 33 unmodified lines
272294 static struct kobj_attribute stop_timeout_attr =
273295 __ATTR(stop_timeout_ms, 0644, stop_show, stop_store);
274296
297+static ssize_t
298+stop_default(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
299+{
300+ struct intel_engine_cs *engine = kobj_to_engine(kobj);
301+
302+ return sprintf(buf, "%lu\n", engine->defaults.stop_timeout_ms);
303+}
304+
305+static struct kobj_attribute stop_timeout_def =
306+__ATTR(stop_timeout_ms, 0444, stop_default, NULL);
307+
275308 static ssize_t
276309 preempt_timeout_store(struct kobject *kobj, struct kobj_attribute *attr,
277310 const char *buf, size_t count)
@@ -316,6 +349,18 @@ preempt_timeout_show(struct kobject *kobj, struct kobj_attribute *attr, · 38 unmodified lines
316349 static struct kobj_attribute preempt_timeout_attr =
317350 __ATTR(preempt_timeout_ms, 0644, preempt_timeout_show, preempt_timeout_store);
318351
352+static ssize_t
353+preempt_timeout_default(struct kobject *kobj, struct kobj_attribute *attr,
354+ char *buf)
355+{
356+ struct intel_engine_cs *engine = kobj_to_engine(kobj);
357+
358+ return sprintf(buf, "%lu\n", engine->defaults.preempt_timeout_ms);
359+}
360+
361+static struct kobj_attribute preempt_timeout_def =
362+__ATTR(preempt_timeout_ms, 0444, preempt_timeout_default, NULL);
363+
319364 static ssize_t
320365 heartbeat_store(struct kobject *kobj, struct kobj_attribute *attr,
321366 const char *buf, size_t count)
@@ -359,6 +404,17 @@ heartbeat_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) · 37 unmodified lines
359404 static struct kobj_attribute heartbeat_interval_attr =
360405 __ATTR(heartbeat_interval_ms, 0644, heartbeat_show, heartbeat_store);
361406
407+static ssize_t
408+heartbeat_default(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
409+{
410+ struct intel_engine_cs *engine = kobj_to_engine(kobj);
411+
412+ return sprintf(buf, "%lu\n", engine->defaults.heartbeat_interval_ms);
413+}
414+
415+static struct kobj_attribute heartbeat_interval_def =
416+__ATTR(heartbeat_interval_ms, 0444, heartbeat_default, NULL);
417+
362418 static void kobj_engine_release(struct kobject *kobj)
363419 {
364420 kfree(kobj);
@@ -390,6 +446,42 @@ kobj_engine(struct kobject *dir, struct intel_engine_cs *engine) · 25 unmodified lines
390446 return &ke->base;
391447 }
392448
449+static void add_defaults(struct kobj_engine *parent)
450+{
451+ static const struct attribute *files[] = {
452+ &max_spin_def.attr,
453+ &stop_timeout_def.attr,
454+#if CONFIG_DRM_I915_HEARTBEAT_INTERVAL
455+ &heartbeat_interval_def.attr,
456+#endif
457+ NULL
458+ };
459+ struct kobj_engine *ke;
460+
461+ ke = kzalloc(sizeof(*ke), GFP_KERNEL);
462+ if (!ke)
463+ return;
464+
465+ kobject_init(&ke->base, &kobj_engine_type);
466+ ke->engine = parent->engine;
467+
468+ if (kobject_add(&ke->base, &parent->base, "%s", ".defaults")) {
469+ kobject_put(&ke->base);
470+ return;
471+ }
472+
473+ if (sysfs_create_files(&ke->base, files))
474+ return;
475+
476+ if (intel_engine_has_timeslices(ke->engine) &&
477+ sysfs_create_file(&ke->base, &timeslice_duration_def.attr))
478+ return;
479+
480+ if (intel_engine_has_preempt_reset(ke->engine) &&
481+ sysfs_create_file(&ke->base, &preempt_timeout_def.attr))
482+ return;
483+}
484+
393485 void intel_engines_add_sysfs(struct drm_i915_private *i915)
394486 {
395487 static const struct attribute *files[] = {
@@ -433,6 +525,8 @@ void intel_engines_add_sysfs(struct drm_i915_private *i915) · 37 unmodified lines
433525 sysfs_create_file(kobj, &preempt_timeout_attr.attr))
434526 goto err_engine;
435527
528+ add_defaults(container_of(kobj, struct kobj_engine, base));
529+
436530 if (0) {
437531 err_object:
438532 kobject_put(kobj);
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c+1−1
@@ -258,7 +258,7 @@ static void guc_submit(struct intel_engine_cs *engine, · 257 unmodified lines
258258
259259 static inline int rq_prio(const struct i915_request *rq)
260260 {
261− return rq->sched.attr.priority | __NO_PREEMPTION;
261+ return rq->sched.attr.priority;
262262 }
263263
264264 static struct i915_request *schedule_in(struct i915_request *rq, int idx)
drivers/gpu/drm/i915/gvt/cmd_parser.c+45−0
@@ -882,6 +882,47 @@ static int mocs_cmd_reg_handler(struct parser_exec_state *s, · 881 unmodified lines
882882 return 0;
883883 }
884884
885+static int is_cmd_update_pdps(unsigned int offset,
886+ struct parser_exec_state *s)
887+{
888+ u32 base = s->workload->engine->mmio_base;
889+ return i915_mmio_reg_equal(_MMIO(offset), GEN8_RING_PDP_UDW(base, 0));
890+}
891+
892+static int cmd_pdp_mmio_update_handler(struct parser_exec_state *s,
893+ unsigned int offset, unsigned int index)
894+{
895+ struct intel_vgpu *vgpu = s->vgpu;
896+ struct intel_vgpu_mm *shadow_mm = s->workload->shadow_mm;
897+ struct intel_vgpu_mm *mm;
898+ u64 pdps[GEN8_3LVL_PDPES];
899+
900+ if (shadow_mm->ppgtt_mm.root_entry_type ==
901+ GTT_TYPE_PPGTT_ROOT_L4_ENTRY) {
902+ pdps[0] = (u64)cmd_val(s, 2) << 32;
903+ pdps[0] |= cmd_val(s, 4);
904+
905+ mm = intel_vgpu_find_ppgtt_mm(vgpu, pdps);
906+ if (!mm) {
907+ gvt_vgpu_err("failed to get the 4-level shadow vm\n");
908+ return -EINVAL;
909+ }
910+ intel_vgpu_mm_get(mm);
911+ list_add_tail(&mm->ppgtt_mm.link,
912+ &s->workload->lri_shadow_mm);
913+ *cmd_ptr(s, 2) = upper_32_bits(mm->ppgtt_mm.shadow_pdps[0]);
914+ *cmd_ptr(s, 4) = lower_32_bits(mm->ppgtt_mm.shadow_pdps[0]);
915+ } else {
916+ /* Currently all guests use PML4 table and now can't
917+ * have a guest with 3-level table but uses LRI for
918+ * PPGTT update. So this is simply un-testable. */
919+ GEM_BUG_ON(1);
920+ gvt_vgpu_err("invalid shared shadow vm type\n");
921+ return -EINVAL;
922+ }
923+ return 0;
924+}
925+
885926 static int cmd_reg_handler(struct parser_exec_state *s,
886927 unsigned int offset, unsigned int index, char *cmd)
887928 {
@@ -920,6 +961,10 @@ static int cmd_reg_handler(struct parser_exec_state *s, · 32 unmodified lines
920961 patch_value(s, cmd_ptr(s, index), VGT_PVINFO_PAGE);
921962 }
922963
964+ if (is_cmd_update_pdps(offset, s) &&
965+ cmd_pdp_mmio_update_handler(s, offset, index))
966+ return -EINVAL;
967+
923968 /* TODO
924969 * In order to let workload with inhibit context to generate
925970 * correct image data into memory, vregs values will be loaded to
drivers/gpu/drm/i915/gvt/execlist.c+0−2
@@ -424,8 +424,6 @@ static int complete_execlist_workload(struct intel_vgpu_workload *workload) · 423 unmodified lines
424424
425425 ret = emulate_execlist_ctx_schedule_out(execlist, &workload->ctx_desc);
426426 out:
427− intel_vgpu_unpin_mm(workload->shadow_mm);
428− intel_vgpu_destroy_workload(workload);
429427 return ret;
430428 }
431429
drivers/gpu/drm/i915/gvt/gtt.c+1−0
@@ -1900,6 +1900,7 @@ struct intel_vgpu_mm *intel_vgpu_create_ppgtt_mm(struct intel_vgpu *vgpu, · 1899 unmodified lines
19001900
19011901 INIT_LIST_HEAD(&mm->ppgtt_mm.list);
19021902 INIT_LIST_HEAD(&mm->ppgtt_mm.lru_list);
1903+ INIT_LIST_HEAD(&mm->ppgtt_mm.link);
19031904
19041905 if (root_entry_type == GTT_TYPE_PPGTT_ROOT_L4_ENTRY)
19051906 mm->ppgtt_mm.guest_pdps[0] = pdps[0];
drivers/gpu/drm/i915/gvt/gtt.h+1−0
@@ -160,6 +160,7 @@ struct intel_vgpu_mm { · 159 unmodified lines
160160
161161 struct list_head list;
162162 struct list_head lru_list;
163+ struct list_head link; /* possible LRI shadow mm list */
163164 } ppgtt_mm;
164165 struct {
165166 void *virtual_ggtt;
drivers/gpu/drm/i915/gvt/handlers.c+1−1
@@ -2812,7 +2812,7 @@ static int init_bdw_mmio_info(struct intel_gvt *gvt) · 2811 unmodified lines
28122812 MMIO_D(GAMTARBMODE, D_BDW_PLUS);
28132813
28142814 #define RING_REG(base) _MMIO((base) + 0x270)
2815− MMIO_RING_F(RING_REG, 32, 0, 0, 0, D_BDW_PLUS, NULL, NULL);
2815+ MMIO_RING_F(RING_REG, 32, F_CMD_ACCESS, 0, 0, D_BDW_PLUS, NULL, NULL);
28162816 #undef RING_REG
28172817
28182818 MMIO_RING_GM_RDR(RING_HWS_PGA, D_BDW_PLUS, NULL, hws_pga_write);
drivers/gpu/drm/i915/gvt/scheduler.c+106−9
@@ -58,10 +58,8 @@ static void set_context_pdp_root_pointer( · 57 unmodified lines
5858
5959 static void update_shadow_pdps(struct intel_vgpu_workload *workload)
6060 {
61− struct drm_i915_gem_object *ctx_obj =
62− workload->req->context->state->obj;
6361 struct execlist_ring_context *shadow_ring_context;
64− struct page *page;
62+ struct intel_context *ctx = workload->req->context;
6563
6664 if (WARN_ON(!workload->shadow_mm))
6765 return;
@@ -69,11 +67,9 @@ static void update_shadow_pdps(struct intel_vgpu_workload *workload) · 1 unmodified line
6967 if (WARN_ON(!atomic_read(&workload->shadow_mm->pincount)))
7068 return;
7169
72− page = i915_gem_object_get_page(ctx_obj, LRC_STATE_PN);
73− shadow_ring_context = kmap(page);
70+ shadow_ring_context = (struct execlist_ring_context *)ctx->lrc_reg_state;
7471 set_context_pdp_root_pointer(shadow_ring_context,
7572 (void *)workload->shadow_mm->ppgtt_mm.shadow_pdps);
76− kunmap(page);
7773 }
7874
7975 /*
@@ -646,10 +642,11 @@ static void release_shadow_batch_buffer(struct intel_vgpu_workload *workload) · 566 unmodified lines
646642 }
647643 }
648644
649−static int prepare_workload(struct intel_vgpu_workload *workload)
645+static int
646+intel_vgpu_shadow_mm_pin(struct intel_vgpu_workload *workload)
650647 {
651648 struct intel_vgpu *vgpu = workload->vgpu;
652− struct intel_vgpu_submission *s = &vgpu->submission;
649+ struct intel_vgpu_mm *m;
653650 int ret = 0;
654651
655652 ret = intel_vgpu_pin_mm(workload->shadow_mm);
@@ -664,6 +661,52 @@ static int prepare_workload(struct intel_vgpu_workload *workload) · 8 unmodified lines
664661 return -EINVAL;
665662 }
666663
664+ if (!list_empty(&workload->lri_shadow_mm)) {
665+ list_for_each_entry(m, &workload->lri_shadow_mm,
666+ ppgtt_mm.link) {
667+ ret = intel_vgpu_pin_mm(m);
668+ if (ret) {
669+ list_for_each_entry_from_reverse(m,
670+ &workload->lri_shadow_mm,
671+ ppgtt_mm.link)
672+ intel_vgpu_unpin_mm(m);
673+ gvt_vgpu_err("LRI shadow ppgtt fail to pin\n");
674+ break;
675+ }
676+ }
677+ }
678+
679+ if (ret)
680+ intel_vgpu_unpin_mm(workload->shadow_mm);
681+
682+ return ret;
683+}
684+
685+static void
686+intel_vgpu_shadow_mm_unpin(struct intel_vgpu_workload *workload)
687+{
688+ struct intel_vgpu_mm *m;
689+
690+ if (!list_empty(&workload->lri_shadow_mm)) {
691+ list_for_each_entry(m, &workload->lri_shadow_mm,
692+ ppgtt_mm.link)
693+ intel_vgpu_unpin_mm(m);
694+ }
695+ intel_vgpu_unpin_mm(workload->shadow_mm);
696+}
697+
698+static int prepare_workload(struct intel_vgpu_workload *workload)
699+{
700+ struct intel_vgpu *vgpu = workload->vgpu;
701+ struct intel_vgpu_submission *s = &vgpu->submission;
702+ int ret = 0;
703+
704+ ret = intel_vgpu_shadow_mm_pin(workload);
705+ if (ret) {
706+ gvt_vgpu_err("fail to pin shadow mm\n");
707+ return ret;
708+ }
709+
667710 update_shadow_pdps(workload);
668711
669712 set_context_ppgtt_from_shadow(workload, s->shadow[workload->engine->id]);
@@ -710,7 +753,7 @@ err_shadow_wa_ctx: · 40 unmodified lines
710753 err_shadow_batch:
711754 release_shadow_batch_buffer(workload);
712755 err_unpin_mm:
713− intel_vgpu_unpin_mm(workload->shadow_mm);
756+ intel_vgpu_shadow_mm_unpin(workload);
714757 return ret;
715758 }
716759
@@ -820,6 +863,37 @@ out: · 103 unmodified lines
820863 return workload;
821864 }
822865
866+static void update_guest_pdps(struct intel_vgpu *vgpu,
867+ u64 ring_context_gpa, u32 pdp[8])
868+{
869+ u64 gpa;
870+ int i;
871+
872+ gpa = ring_context_gpa + RING_CTX_OFF(pdps[0].val);
873+
874+ for (i = 0; i < 8; i++)
875+ intel_gvt_hypervisor_write_gpa(vgpu,
876+ gpa + i * 8, &pdp[7 - i], 4);
877+}
878+
879+static bool
880+check_shadow_context_ppgtt(struct execlist_ring_context *c, struct intel_vgpu_mm *m)
881+{
882+ if (m->ppgtt_mm.root_entry_type == GTT_TYPE_PPGTT_ROOT_L4_ENTRY) {
883+ u64 shadow_pdp = c->pdps[7].val | (u64) c->pdps[6].val << 32;
884+
885+ if (shadow_pdp != m->ppgtt_mm.shadow_pdps[0]) {
886+ gvt_dbg_mm("4-level context ppgtt not match LRI command\n");
887+ return false;
888+ }
889+ return true;
890+ } else {
891+ /* see comment in LRI handler in cmd_parser.c */
892+ gvt_dbg_mm("invalid shadow mm type\n");
893+ return false;
894+ }
895+}
896+
823897 static void update_guest_context(struct intel_vgpu_workload *workload)
824898 {
825899 struct i915_request *rq = workload->req;
@@ -905,6 +979,15 @@ write: · 79 unmodified lines
905979
906980 shadow_ring_context = (void *) ctx->lrc_reg_state;
907981
982+ if (!list_empty(&workload->lri_shadow_mm)) {
983+ struct intel_vgpu_mm *m = list_last_entry(&workload->lri_shadow_mm,
984+ struct intel_vgpu_mm,
985+ ppgtt_mm.link);
986+ GEM_BUG_ON(!check_shadow_context_ppgtt(shadow_ring_context, m));
987+ update_guest_pdps(vgpu, workload->ring_context_gpa,
988+ (void *)m->ppgtt_mm.guest_pdps);
989+ }
990+
908991 #define COPY_REG(name) \
909992 intel_gvt_hypervisor_write_gpa(vgpu, workload->ring_context_gpa + \
910993 RING_CTX_OFF(name.val), &shadow_ring_context->name.val, 4)
@@ -1013,6 +1096,9 @@ static void complete_current_workload(struct intel_gvt *gvt, int ring_id) · 102 unmodified lines
10131096
10141097 workload->complete(workload);
10151098
1099+ intel_vgpu_shadow_mm_unpin(workload);
1100+ intel_vgpu_destroy_workload(workload);
1101+
10161102 atomic_dec(&s->running_workload_num);
10171103 wake_up(&scheduler->workload_complete_wq);
10181104
@@ -1406,6 +1492,16 @@ void intel_vgpu_destroy_workload(struct intel_vgpu_workload *workload) · 387 unmodified lines
14061492 release_shadow_batch_buffer(workload);
14071493 release_shadow_wa_ctx(&workload->wa_ctx);
14081494
1495+ if (!list_empty(&workload->lri_shadow_mm)) {
1496+ struct intel_vgpu_mm *m, *mm;
1497+ list_for_each_entry_safe(m, mm, &workload->lri_shadow_mm,
1498+ ppgtt_mm.link) {
1499+ list_del(&m->ppgtt_mm.link);
1500+ intel_vgpu_mm_put(m);
1501+ }
1502+ }
1503+
1504+ GEM_BUG_ON(!list_empty(&workload->lri_shadow_mm));
14091505 if (workload->shadow_mm)
14101506 intel_vgpu_mm_put(workload->shadow_mm);
14111507
@@ -1424,6 +1520,7 @@ alloc_workload(struct intel_vgpu *vgpu) · 12 unmodified lines
14241520
14251521 INIT_LIST_HEAD(&workload->list);
14261522 INIT_LIST_HEAD(&workload->shadow_bb);
1523+ INIT_LIST_HEAD(&workload->lri_shadow_mm);
14271524
14281525 init_waitqueue_head(&workload->shadow_ctx_status_wq);
14291526 atomic_set(&workload->shadow_ctx_active, 0);
drivers/gpu/drm/i915/gvt/scheduler.h+1−0
@@ -87,6 +87,7 @@ struct intel_vgpu_workload { · 86 unmodified lines
8787 int status;
8888
8989 struct intel_vgpu_mm *shadow_mm;
90+ struct list_head lri_shadow_mm; /* For PPGTT load cmd */
9091
9192 /* different submission model may need different handler */
9293 int (*prepare)(struct intel_vgpu_workload *);
drivers/gpu/drm/i915/i915_config.c+15−0
@@ -0,0 +1,15 @@
1+// SPDX-License-Identifier: MIT
2+/*
3+ * Copyright © 2020 Intel Corporation
4+ */
5+
6+#include "i915_drv.h"
7+
8+unsigned long
9+i915_fence_context_timeout(const struct drm_i915_private *i915, u64 context)
10+{
11+ if (context && IS_ACTIVE(CONFIG_DRM_I915_FENCE_TIMEOUT))
12+ return msecs_to_jiffies_timeout(CONFIG_DRM_I915_FENCE_TIMEOUT);
13+
14+ return 0;
15+}
drivers/gpu/drm/i915/i915_debugfs.c+7−4
@@ -32,6 +32,7 @@ · 31 unmodified lines
3232 #include <drm/drm_debugfs.h>
3333
3434 #include "gem/i915_gem_context.h"
35+#include "gt/intel_gt_buffer_pool.h"
3536 #include "gt/intel_gt_clock_utils.h"
3637 #include "gt/intel_gt_pm.h"
3738 #include "gt/intel_gt_requests.h"
@@ -1303,8 +1304,8 @@ static int i915_engine_info(struct seq_file *m, void *unused) · 1265 unmodified lines
13031304 seq_printf(m, "GT awake? %s [%d]\n",
13041305 yesno(dev_priv->gt.awake),
13051306 atomic_read(&dev_priv->gt.wakeref.count));
1306− seq_printf(m, "CS timestamp frequency: %u kHz\n",
1307− RUNTIME_INFO(dev_priv)->cs_timestamp_frequency_khz);
1307+ seq_printf(m, "CS timestamp frequency: %u Hz\n",
1308+ RUNTIME_INFO(dev_priv)->cs_timestamp_frequency_hz);
13081309
13091310 p = drm_seq_file_printer(m);
13101311 for_each_uabi_engine(engine, dev_priv)
@@ -1403,13 +1404,12 @@ static int · 92 unmodified lines
14031404 i915_perf_noa_delay_set(void *data, u64 val)
14041405 {
14051406 struct drm_i915_private *i915 = data;
1406− const u32 clk = RUNTIME_INFO(i915)->cs_timestamp_frequency_khz;
14071407
14081408 /*
14091409 * This would lead to infinite waits as we're doing timestamp
14101410 * difference on the CS with only 32bits.
14111411 */
1412− if (val > mul_u32_u32(U32_MAX, clk))
1412+ if (i915_cs_timestamp_ns_to_ticks(i915, val) > U32_MAX)
14131413 return -EINVAL;
14141414
14151415 atomic64_set(&i915->perf.noa_programming_delay, val);
@@ -1484,6 +1484,9 @@ gt_drop_caches(struct intel_gt *gt, u64 val) · 68 unmodified lines
14841484 if (val & DROP_RESET_ACTIVE && intel_gt_terminally_wedged(gt))
14851485 intel_gt_handle_error(gt, ALL_ENGINES, 0, NULL);
14861486
1487+ if (val & DROP_FREED)
1488+ intel_gt_flush_buffer_pool(gt);
1489+
14871490 return 0;
14881491 }
14891492
drivers/gpu/drm/i915/i915_drv.h+29−8
@@ -108,8 +108,8 @@ · 107 unmodified lines
108108
109109 #define DRIVER_NAME "i915"
110110 #define DRIVER_DESC "Intel Graphics"
111−#define DRIVER_DATE "20200430"
112−#define DRIVER_TIMESTAMP 1588234401
111+#define DRIVER_DATE "20200515"
112+#define DRIVER_TIMESTAMP 1589543364
113113
114114 struct drm_i915_gem_object;
115115
@@ -148,6 +148,8 @@ enum hpd_pin { · 32 unmodified lines
148148 struct i915_hotplug {
149149 struct delayed_work hotplug_work;
150150
151+ const u32 *hpd, *pch_hpd;
152+
151153 struct {
152154 unsigned long last_jiffies;
153155 int count;
@@ -510,6 +512,7 @@ struct i915_psr { · 356 unmodified lines
510512 u32 dc3co_exit_delay;
511513 struct delayed_work dc3co_work;
512514 bool force_mode_changed;
515+ struct drm_dp_vsc_sdp vsc;
513516 };
514517
515518 #define QUIRK_LVDS_SSC_DISABLE (1<<1)
@@ -614,13 +617,14 @@ struct i915_gem_mm { · 98 unmodified lines
614617
615618 #define I915_IDLE_ENGINES_TIMEOUT (200) /* in ms */
616619
617−#define I915_RESET_TIMEOUT (10 * HZ) /* 10s */
618−#define I915_FENCE_TIMEOUT (10 * HZ) /* 10s */
619−
620−#define I915_ENGINE_DEAD_TIMEOUT (4 * HZ) /* Seqno, head and subunits dead */
621−#define I915_SEQNO_DEAD_TIMEOUT (12 * HZ) /* Seqno dead with active head */
620+unsigned long i915_fence_context_timeout(const struct drm_i915_private *i915,
621+ u64 context);
622622
623−#define I915_ENGINE_WEDGED_TIMEOUT (60 * HZ) /* Reset but no recovery? */
623+static inline unsigned long
624+i915_fence_timeout(const struct drm_i915_private *i915)
625+{
626+ return i915_fence_context_timeout(i915, U64_MAX);
627+}
624628
625629 /* Amount of SAGV/QGV points, BSpec precisely defines this */
626630 #define I915_NUM_QGV_POINTS 8
@@ -1507,6 +1511,11 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915, · 880 unmodified lines
15071511 #define IS_ICL_REVID(p, since, until) \
15081512 (IS_ICELAKE(p) && IS_REVID(p, since, until))
15091513
1514+#define EHL_REVID_A0 0x0
1515+
1516+#define IS_EHL_REVID(p, since, until) \
1517+ (IS_ELKHARTLAKE(p) && IS_REVID(p, since, until))
1518+
15101519 #define TGL_REVID_A0 0x0
15111520 #define TGL_REVID_B0 0x1
15121521 #define TGL_REVID_C0 0x2
@@ -1915,4 +1924,16 @@ i915_coherent_map_type(struct drm_i915_private *i915) · 402 unmodified lines
19151924 return HAS_LLC(i915) ? I915_MAP_WB : I915_MAP_WC;
19161925 }
19171926
1927+static inline u64 i915_cs_timestamp_ns_to_ticks(struct drm_i915_private *i915, u64 val)
1928+{
1929+ return DIV_ROUND_UP_ULL(val * RUNTIME_INFO(i915)->cs_timestamp_frequency_hz,
1930+ 1000000000);
1931+}
1932+
1933+static inline u64 i915_cs_timestamp_ticks_to_ns(struct drm_i915_private *i915, u64 val)
1934+{
1935+ return div_u64(val * 1000000000,
1936+ RUNTIME_INFO(i915)->cs_timestamp_frequency_hz);
1937+}
1938+
19181939 #endif
drivers/gpu/drm/i915/i915_gem_evict.c+12−14
@@ -128,6 +128,13 @@ search_again: · 127 unmodified lines
128128 active = NULL;
129129 INIT_LIST_HEAD(&eviction_list);
130130 list_for_each_entry_safe(vma, next, &vm->bound_list, vm_link) {
131+ if (vma == active) { /* now seen this vma twice */
132+ if (flags & PIN_NONBLOCK)
133+ break;
134+
135+ active = ERR_PTR(-EAGAIN);
136+ }
137+
131138 /*
132139 * We keep this list in a rough least-recently scanned order
133140 * of active elements (inactive elements are cheap to reap).
@@ -143,21 +150,12 @@ search_again: · 9 unmodified lines
143150 * To notice when we complete one full cycle, we record the
144151 * first active element seen, before moving it to the tail.
145152 */
146− if (i915_vma_is_active(vma)) {
147− if (vma == active) {
148− if (flags & PIN_NONBLOCK)
149− break;
150−
151− active = ERR_PTR(-EAGAIN);
152− }
153−
154− if (active != ERR_PTR(-EAGAIN)) {
155− if (!active)
156− active = vma;
153+ if (active != ERR_PTR(-EAGAIN) && i915_vma_is_active(vma)) {
154+ if (!active)
155+ active = vma;
157156
158− list_move_tail(&vma->vm_link, &vm->bound_list);
159− continue;
160− }
157+ list_move_tail(&vma->vm_link, &vm->bound_list);
158+ continue;
161159 }
162160
163161 if (mark_free(&scan, vma, flags, &eviction_list))
drivers/gpu/drm/i915/i915_getparam.c+1−1
@@ -153,7 +153,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data, · 152 unmodified lines
153153 return -ENODEV;
154154 break;
155155 case I915_PARAM_CS_TIMESTAMP_FREQUENCY:
156− value = 1000 * RUNTIME_INFO(i915)->cs_timestamp_frequency_khz;
156+ value = RUNTIME_INFO(i915)->cs_timestamp_frequency_hz;
157157 break;
158158 case I915_PARAM_MMAP_GTT_COHERENT:
159159 value = INTEL_INFO(i915)->has_coherent_ggtt;
drivers/gpu/drm/i915/i915_gpu_error.h+1−1
@@ -42,7 +42,7 @@ struct i915_vma_coredump { · 41 unmodified lines
4242 int num_pages;
4343 int page_count;
4444 int unused;
45− u32 *pages[0];
45+ u32 *pages[];
4646 };
4747
4848 struct i915_request_coredump {
drivers/gpu/drm/i915/i915_irq.c+125−102
@@ -124,7 +124,6 @@ static const u32 hpd_status_i915[HPD_NUM_PINS] = { · 123 unmodified lines
124124 [HPD_PORT_D] = PORTD_HOTPLUG_INT_STATUS,
125125 };
126126
127−/* BXT hpd list */
128127 static const u32 hpd_bxt[HPD_NUM_PINS] = {
129128 [HPD_PORT_A] = BXT_DE_PORT_HP_DDIA,
130129 [HPD_PORT_B] = BXT_DE_PORT_HP_DDIB,
@@ -168,6 +167,49 @@ static const u32 hpd_tgp[HPD_NUM_PINS] = { · 37 unmodified lines
168167 [HPD_PORT_I] = SDE_TC_HOTPLUG_ICP(PORT_TC6),
169168 };
170169
170+static void intel_hpd_init_pins(struct drm_i915_private *dev_priv)
171+{
172+ struct i915_hotplug *hpd = &dev_priv->hotplug;
173+
174+ if (HAS_GMCH(dev_priv)) {
175+ if (IS_G4X(dev_priv) || IS_VALLEYVIEW(dev_priv) ||
176+ IS_CHERRYVIEW(dev_priv))
177+ hpd->hpd = hpd_status_g4x;
178+ else
179+ hpd->hpd = hpd_status_i915;
180+ return;
181+ }
182+
183+ if (INTEL_GEN(dev_priv) >= 12)
184+ hpd->hpd = hpd_gen12;
185+ else if (INTEL_GEN(dev_priv) >= 11)
186+ hpd->hpd = hpd_gen11;
187+ else if (IS_GEN9_LP(dev_priv))
188+ hpd->hpd = hpd_bxt;
189+ else if (INTEL_GEN(dev_priv) >= 8)
190+ hpd->hpd = hpd_bdw;
191+ else if (INTEL_GEN(dev_priv) >= 7)
192+ hpd->hpd = hpd_ivb;
193+ else
194+ hpd->hpd = hpd_ilk;
195+
196+ if (!HAS_PCH_SPLIT(dev_priv) || HAS_PCH_NOP(dev_priv))
197+ return;
198+
199+ if (HAS_PCH_TGP(dev_priv) || HAS_PCH_JSP(dev_priv))
200+ hpd->pch_hpd = hpd_tgp;
201+ else if (HAS_PCH_ICP(dev_priv) || HAS_PCH_MCC(dev_priv))
202+ hpd->pch_hpd = hpd_icp;
203+ else if (HAS_PCH_CNP(dev_priv) || HAS_PCH_SPT(dev_priv))
204+ hpd->pch_hpd = hpd_spt;
205+ else if (HAS_PCH_LPT(dev_priv) || HAS_PCH_CPT(dev_priv))
206+ hpd->pch_hpd = hpd_cpt;
207+ else if (HAS_PCH_IBX(dev_priv))
208+ hpd->pch_hpd = hpd_ibx;
209+ else
210+ MISSING_CASE(INTEL_PCH_TYPE(dev_priv));
211+}
212+
171213 static void
172214 intel_handle_vblank(struct drm_i915_private *dev_priv, enum pipe pipe)
173215 {
@@ -1504,33 +1546,27 @@ static void i9xx_hpd_irq_handler(struct drm_i915_private *dev_priv, · 1330 unmodified lines
15041546 u32 hotplug_status)
15051547 {
15061548 u32 pin_mask = 0, long_mask = 0;
1549+ u32 hotplug_trigger;
15071550
1508− if (IS_G4X(dev_priv) || IS_VALLEYVIEW(dev_priv) ||
1509− IS_CHERRYVIEW(dev_priv)) {
1510− u32 hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_G4X;
1511−
1512− if (hotplug_trigger) {
1513− intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1514− hotplug_trigger, hotplug_trigger,
1515− hpd_status_g4x,
1516− i9xx_port_hotplug_long_detect);
1551+ if (IS_G4X(dev_priv) ||
1552+ IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
1553+ hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_G4X;
1554+ else
1555+ hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_I915;
15171556
1518− intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
1519− }
1557+ if (hotplug_trigger) {
1558+ intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1559+ hotplug_trigger, hotplug_trigger,
1560+ dev_priv->hotplug.hpd,
1561+ i9xx_port_hotplug_long_detect);
15201562
1521− if (hotplug_status & DP_AUX_CHANNEL_MASK_INT_STATUS_G4X)
1522− dp_aux_irq_handler(dev_priv);
1523− } else {
1524− u32 hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_I915;
1525−
1526− if (hotplug_trigger) {
1527− intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1528− hotplug_trigger, hotplug_trigger,
1529− hpd_status_i915,
1530− i9xx_port_hotplug_long_detect);
1531− intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
1532− }
1563+ intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
15331564 }
1565+
1566+ if ((IS_G4X(dev_priv) ||
1567+ IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) &&
1568+ hotplug_status & DP_AUX_CHANNEL_MASK_INT_STATUS_G4X)
1569+ dp_aux_irq_handler(dev_priv);
15341570 }
15351571
15361572 static irqreturn_t valleyview_irq_handler(int irq, void *arg)
@@ -1696,8 +1732,7 @@ static irqreturn_t cherryview_irq_handler(int irq, void *arg) · 159 unmodified lines
16961732 }
16971733
16981734 static void ibx_hpd_irq_handler(struct drm_i915_private *dev_priv,
1699− u32 hotplug_trigger,
1700− const u32 hpd[HPD_NUM_PINS])
1735+ u32 hotplug_trigger)
17011736 {
17021737 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;
17031738
@@ -1720,8 +1755,9 @@ static void ibx_hpd_irq_handler(struct drm_i915_private *dev_priv, · 16 unmodified lines
17201755 if (!hotplug_trigger)
17211756 return;
17221757
1723− intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, hotplug_trigger,
1724− dig_hotplug_reg, hpd,
1758+ intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1759+ hotplug_trigger, dig_hotplug_reg,
1760+ dev_priv->hotplug.pch_hpd,
17251761 pch_port_hotplug_long_detect);
17261762
17271763 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
@@ -1732,7 +1768,7 @@ static void ibx_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 4 unmodified lines
17321768 enum pipe pipe;
17331769 u32 hotplug_trigger = pch_iir & SDE_HOTPLUG_MASK;
17341770
1735− ibx_hpd_irq_handler(dev_priv, hotplug_trigger, hpd_ibx);
1771+ ibx_hpd_irq_handler(dev_priv, hotplug_trigger);
17361772
17371773 if (pch_iir & SDE_AUDIO_POWER_MASK) {
17381774 int port = ffs((pch_iir & SDE_AUDIO_POWER_MASK) >>
@@ -1820,7 +1856,7 @@ static void cpt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 81 unmodified lines
18201856 enum pipe pipe;
18211857 u32 hotplug_trigger = pch_iir & SDE_HOTPLUG_MASK_CPT;
18221858
1823− ibx_hpd_irq_handler(dev_priv, hotplug_trigger, hpd_cpt);
1859+ ibx_hpd_irq_handler(dev_priv, hotplug_trigger);
18241860
18251861 if (pch_iir & SDE_AUDIO_POWER_MASK_CPT) {
18261862 int port = ffs((pch_iir & SDE_AUDIO_POWER_MASK_CPT) >>
@@ -1857,22 +1893,18 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 30 unmodified lines
18571893 u32 ddi_hotplug_trigger, tc_hotplug_trigger;
18581894 u32 pin_mask = 0, long_mask = 0;
18591895 bool (*tc_port_hotplug_long_detect)(enum hpd_pin pin, u32 val);
1860− const u32 *pins;
18611896
18621897 if (HAS_PCH_TGP(dev_priv)) {
18631898 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_TGP;
18641899 tc_hotplug_trigger = pch_iir & SDE_TC_MASK_TGP;
18651900 tc_port_hotplug_long_detect = tgp_tc_port_hotplug_long_detect;
1866− pins = hpd_tgp;
18671901 } else if (HAS_PCH_JSP(dev_priv)) {
18681902 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_TGP;
18691903 tc_hotplug_trigger = 0;
1870− pins = hpd_tgp;
18711904 } else if (HAS_PCH_MCC(dev_priv)) {
18721905 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_ICP;
18731906 tc_hotplug_trigger = pch_iir & SDE_TC_HOTPLUG_ICP(PORT_TC1);
18741907 tc_port_hotplug_long_detect = icp_tc_port_hotplug_long_detect;
1875− pins = hpd_icp;
18761908 } else {
18771909 drm_WARN(&dev_priv->drm, !HAS_PCH_ICP(dev_priv),
18781910 "Unrecognized PCH type 0x%x\n",
@@ -1881,7 +1913,6 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 2 unmodified lines
18811913 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_ICP;
18821914 tc_hotplug_trigger = pch_iir & SDE_TC_MASK_ICP;
18831915 tc_port_hotplug_long_detect = icp_tc_port_hotplug_long_detect;
1884− pins = hpd_icp;
18851916 }
18861917
18871918 if (ddi_hotplug_trigger) {
@@ -1891,8 +1922,8 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 3 unmodified lines
18911922 I915_WRITE(SHOTPLUG_CTL_DDI, dig_hotplug_reg);
18921923
18931924 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1894− ddi_hotplug_trigger,
1895− dig_hotplug_reg, pins,
1925+ ddi_hotplug_trigger, dig_hotplug_reg,
1926+ dev_priv->hotplug.pch_hpd,
18961927 icp_ddi_port_hotplug_long_detect);
18971928 }
18981929
@@ -1903,8 +1934,8 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 4 unmodified lines
19031934 I915_WRITE(SHOTPLUG_CTL_TC, dig_hotplug_reg);
19041935
19051936 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1906− tc_hotplug_trigger,
1907− dig_hotplug_reg, pins,
1937+ tc_hotplug_trigger, dig_hotplug_reg,
1938+ dev_priv->hotplug.pch_hpd,
19081939 tc_port_hotplug_long_detect);
19091940 }
19101941
@@ -1929,7 +1960,8 @@ static void spt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 18 unmodified lines
19291960 I915_WRITE(PCH_PORT_HOTPLUG, dig_hotplug_reg);
19301961
19311962 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1932− hotplug_trigger, dig_hotplug_reg, hpd_spt,
1963+ hotplug_trigger, dig_hotplug_reg,
1964+ dev_priv->hotplug.pch_hpd,
19331965 spt_port_hotplug_long_detect);
19341966 }
19351967
@@ -1940,7 +1972,8 @@ static void spt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 4 unmodified lines
19401972 I915_WRITE(PCH_PORT_HOTPLUG2, dig_hotplug_reg);
19411973
19421974 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1943− hotplug2_trigger, dig_hotplug_reg, hpd_spt,
1975+ hotplug2_trigger, dig_hotplug_reg,
1976+ dev_priv->hotplug.pch_hpd,
19441977 spt_port_hotplug2_long_detect);
19451978 }
19461979
@@ -1952,16 +1985,16 @@ static void spt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 5 unmodified lines
19521985 }
19531986
19541987 static void ilk_hpd_irq_handler(struct drm_i915_private *dev_priv,
1955− u32 hotplug_trigger,
1956− const u32 hpd[HPD_NUM_PINS])
1988+ u32 hotplug_trigger)
19571989 {
19581990 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;
19591991
19601992 dig_hotplug_reg = I915_READ(DIGITAL_PORT_HOTPLUG_CNTRL);
19611993 I915_WRITE(DIGITAL_PORT_HOTPLUG_CNTRL, dig_hotplug_reg);
19621994
1963− intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, hotplug_trigger,
1964− dig_hotplug_reg, hpd,
1995+ intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1996+ hotplug_trigger, dig_hotplug_reg,
1997+ dev_priv->hotplug.hpd,
19651998 ilk_port_hotplug_long_detect);
19661999
19672000 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
@@ -1974,7 +2007,7 @@ static void ilk_display_irq_handler(struct drm_i915_private *dev_priv, · 6 unmodified lines
19742007 u32 hotplug_trigger = de_iir & DE_DP_A_HOTPLUG;
19752008
19762009 if (hotplug_trigger)
1977− ilk_hpd_irq_handler(dev_priv, hotplug_trigger, hpd_ilk);
2010+ ilk_hpd_irq_handler(dev_priv, hotplug_trigger);
19782011
19792012 if (de_iir & DE_AUX_CHANNEL_A)
19802013 dp_aux_irq_handler(dev_priv);
@@ -2020,7 +2053,7 @@ static void ivb_display_irq_handler(struct drm_i915_private *dev_priv, · 39 unmodified lines
20202053 u32 hotplug_trigger = de_iir & DE_DP_A_HOTPLUG_IVB;
20212054
20222055 if (hotplug_trigger)
2023− ilk_hpd_irq_handler(dev_priv, hotplug_trigger, hpd_ivb);
2056+ ilk_hpd_irq_handler(dev_priv, hotplug_trigger);
20242057
20252058 if (de_iir & DE_ERR_INT_IVB)
20262059 ivb_err_int_handler(dev_priv);
@@ -2130,16 +2163,16 @@ static irqreturn_t ilk_irq_handler(int irq, void *arg) · 103 unmodified lines
21302163 }
21312164
21322165 static void bxt_hpd_irq_handler(struct drm_i915_private *dev_priv,
2133− u32 hotplug_trigger,
2134− const u32 hpd[HPD_NUM_PINS])
2166+ u32 hotplug_trigger)
21352167 {
21362168 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;
21372169
21382170 dig_hotplug_reg = I915_READ(PCH_PORT_HOTPLUG);
21392171 I915_WRITE(PCH_PORT_HOTPLUG, dig_hotplug_reg);
21402172
2141− intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, hotplug_trigger,
2142− dig_hotplug_reg, hpd,
2173+ intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
2174+ hotplug_trigger, dig_hotplug_reg,
2175+ dev_priv->hotplug.hpd,
21432176 bxt_port_hotplug_long_detect);
21442177
21452178 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
@@ -2151,15 +2184,11 @@ static void gen11_hpd_irq_handler(struct drm_i915_private *dev_priv, u32 iir) · 5 unmodified lines
21512184 u32 trigger_tc = iir & GEN11_DE_TC_HOTPLUG_MASK;
21522185 u32 trigger_tbt = iir & GEN11_DE_TBT_HOTPLUG_MASK;
21532186 long_pulse_detect_func long_pulse_detect;
2154− const u32 *hpd;
21552187
2156− if (INTEL_GEN(dev_priv) >= 12) {
2188+ if (INTEL_GEN(dev_priv) >= 12)
21572189 long_pulse_detect = gen12_port_hotplug_long_detect;
2158− hpd = hpd_gen12;
2159− } else {
2190+ else
21602191 long_pulse_detect = gen11_port_hotplug_long_detect;
2161− hpd = hpd_gen11;
2162− }
21632192
21642193 if (trigger_tc) {
21652194 u32 dig_hotplug_reg;
@@ -2167,8 +2196,10 @@ static void gen11_hpd_irq_handler(struct drm_i915_private *dev_priv, u32 iir) · 1 unmodified line
21672196 dig_hotplug_reg = I915_READ(GEN11_TC_HOTPLUG_CTL);
21682197 I915_WRITE(GEN11_TC_HOTPLUG_CTL, dig_hotplug_reg);
21692198
2170− intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, trigger_tc,
2171− dig_hotplug_reg, hpd, long_pulse_detect);
2199+ intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
2200+ trigger_tc, dig_hotplug_reg,
2201+ dev_priv->hotplug.hpd,
2202+ long_pulse_detect);
21722203 }
21732204
21742205 if (trigger_tbt) {
@@ -2177,8 +2208,10 @@ static void gen11_hpd_irq_handler(struct drm_i915_private *dev_priv, u32 iir) · 2 unmodified lines
21772208 dig_hotplug_reg = I915_READ(GEN11_TBT_HOTPLUG_CTL);
21782209 I915_WRITE(GEN11_TBT_HOTPLUG_CTL, dig_hotplug_reg);
21792210
2180− intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, trigger_tbt,
2181− dig_hotplug_reg, hpd, long_pulse_detect);
2211+ intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
2212+ trigger_tbt, dig_hotplug_reg,
2213+ dev_priv->hotplug.hpd,
2214+ long_pulse_detect);
21822215 }
21832216
21842217 if (pin_mask)
@@ -2309,15 +2342,13 @@ gen8_de_irq_handler(struct drm_i915_private *dev_priv, u32 master_ctl) · 124 unmodified lines
23092342 if (IS_GEN9_LP(dev_priv)) {
23102343 tmp_mask = iir & BXT_DE_PORT_HOTPLUG_MASK;
23112344 if (tmp_mask) {
2312− bxt_hpd_irq_handler(dev_priv, tmp_mask,
2313− hpd_bxt);
2345+ bxt_hpd_irq_handler(dev_priv, tmp_mask);
23142346 found = true;
23152347 }
23162348 } else if (IS_BROADWELL(dev_priv)) {
23172349 tmp_mask = iir & GEN8_PORT_DP_A_HOTPLUG;
23182350 if (tmp_mask) {
2319− ilk_hpd_irq_handler(dev_priv,
2320− tmp_mask, hpd_bdw);
2351+ ilk_hpd_irq_handler(dev_priv, tmp_mask);
23212352 found = true;
23222353 }
23232354 }
@@ -2870,6 +2901,14 @@ static void gen11_display_irq_reset(struct drm_i915_private *dev_priv) · 546 unmodified lines
28702901
28712902 if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)
28722903 GEN3_IRQ_RESET(uncore, SDE);
2904+
2905+ /* Wa_14010685332:icl */
2906+ if (INTEL_PCH_TYPE(dev_priv) == PCH_ICP) {
2907+ intel_uncore_rmw(uncore, SOUTH_CHICKEN1,
2908+ SBCLK_RUN_REFCLK_DIS, SBCLK_RUN_REFCLK_DIS);
2909+ intel_uncore_rmw(uncore, SOUTH_CHICKEN1,
2910+ SBCLK_RUN_REFCLK_DIS, 0);
2911+ }
28732912 }
28742913
28752914 static void gen11_irq_reset(struct drm_i915_private *dev_priv)
@@ -2989,13 +3028,12 @@ static void ibx_hpd_irq_setup(struct drm_i915_private *dev_priv) · 113 unmodified lines
29893028 {
29903029 u32 hotplug_irqs, enabled_irqs;
29913030
2992− if (HAS_PCH_IBX(dev_priv)) {
3031+ if (HAS_PCH_IBX(dev_priv))
29933032 hotplug_irqs = SDE_HOTPLUG_MASK;
2994− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_ibx);
2995− } else {
3033+ else
29963034 hotplug_irqs = SDE_HOTPLUG_MASK_CPT;
2997− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_cpt);
2998− }
3035+
3036+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.pch_hpd);
29993037
30003038 ibx_display_interrupt_update(dev_priv, hotplug_irqs, enabled_irqs);
30013039
@@ -3021,13 +3059,12 @@ static void icp_hpd_detection_setup(struct drm_i915_private *dev_priv, · 19 unmodified lines
30213059
30223060 static void icp_hpd_irq_setup(struct drm_i915_private *dev_priv,
30233061 u32 sde_ddi_mask, u32 sde_tc_mask,
3024− u32 ddi_enable_mask, u32 tc_enable_mask,
3025− const u32 *pins)
3062+ u32 ddi_enable_mask, u32 tc_enable_mask)
30263063 {
30273064 u32 hotplug_irqs, enabled_irqs;
30283065
30293066 hotplug_irqs = sde_ddi_mask | sde_tc_mask;
3030− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, pins);
3067+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.pch_hpd);
30313068
30323069 I915_WRITE(SHPD_FILTER_CNT, SHPD_FILTER_CNT_500_ADJ);
30333070
@@ -3044,8 +3081,7 @@ static void mcc_hpd_irq_setup(struct drm_i915_private *dev_priv) · 10 unmodified lines
30443081 {
30453082 icp_hpd_irq_setup(dev_priv,
30463083 SDE_DDI_MASK_ICP, SDE_TC_HOTPLUG_ICP(PORT_TC1),
3047− ICP_DDI_HPD_ENABLE_MASK, ICP_TC_HPD_ENABLE(PORT_TC1),
3048− hpd_icp);
3084+ ICP_DDI_HPD_ENABLE_MASK, ICP_TC_HPD_ENABLE(PORT_TC1));
30493085 }
30503086
30513087 /*
@@ -3057,8 +3093,7 @@ static void jsp_hpd_irq_setup(struct drm_i915_private *dev_priv) · 5 unmodified lines
30573093 {
30583094 icp_hpd_irq_setup(dev_priv,
30593095 SDE_DDI_MASK_TGP, 0,
3060− TGP_DDI_HPD_ENABLE_MASK, 0,
3061− hpd_tgp);
3096+ TGP_DDI_HPD_ENABLE_MASK, 0);
30623097 }
30633098
30643099 static void gen11_hpd_detection_setup(struct drm_i915_private *dev_priv)
@@ -3083,11 +3118,9 @@ static void gen11_hpd_detection_setup(struct drm_i915_private *dev_priv) · 18 unmodified lines
30833118 static void gen11_hpd_irq_setup(struct drm_i915_private *dev_priv)
30843119 {
30853120 u32 hotplug_irqs, enabled_irqs;
3086− const u32 *hpd;
30873121 u32 val;
30883122
3089− hpd = INTEL_GEN(dev_priv) >= 12 ? hpd_gen12 : hpd_gen11;
3090− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd);
3123+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
30913124 hotplug_irqs = GEN11_DE_TC_HOTPLUG_MASK | GEN11_DE_TBT_HOTPLUG_MASK;
30923125
30933126 val = I915_READ(GEN11_DE_HPD_IMR);
@@ -3099,12 +3132,10 @@ static void gen11_hpd_irq_setup(struct drm_i915_private *dev_priv) · 5 unmodified lines
30993132
31003133 if (INTEL_PCH_TYPE(dev_priv) >= PCH_TGP)
31013134 icp_hpd_irq_setup(dev_priv, SDE_DDI_MASK_TGP, SDE_TC_MASK_TGP,
3102− TGP_DDI_HPD_ENABLE_MASK,
3103− TGP_TC_HPD_ENABLE_MASK, hpd_tgp);
3135+ TGP_DDI_HPD_ENABLE_MASK, TGP_TC_HPD_ENABLE_MASK);
31043136 else if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)
31053137 icp_hpd_irq_setup(dev_priv, SDE_DDI_MASK_ICP, SDE_TC_MASK_ICP,
3106− ICP_DDI_HPD_ENABLE_MASK,
3107− ICP_TC_HPD_ENABLE_MASK, hpd_icp);
3138+ ICP_DDI_HPD_ENABLE_MASK, ICP_TC_HPD_ENABLE_MASK);
31083139 }
31093140
31103141 static void spt_hpd_detection_setup(struct drm_i915_private *dev_priv)
@@ -3140,7 +3171,7 @@ static void spt_hpd_irq_setup(struct drm_i915_private *dev_priv) · 29 unmodified lines
31403171 I915_WRITE(SHPD_FILTER_CNT, SHPD_FILTER_CNT_500_ADJ);
31413172
31423173 hotplug_irqs = SDE_HOTPLUG_MASK_SPT;
3143− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_spt);
3174+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.pch_hpd);
31443175
31453176 ibx_display_interrupt_update(dev_priv, hotplug_irqs, enabled_irqs);
31463177
@@ -3169,17 +3200,17 @@ static void ilk_hpd_irq_setup(struct drm_i915_private *dev_priv) · 22 unmodified lines
31693200
31703201 if (INTEL_GEN(dev_priv) >= 8) {
31713202 hotplug_irqs = GEN8_PORT_DP_A_HOTPLUG;
3172− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_bdw);
3203+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
31733204
31743205 bdw_update_port_irq(dev_priv, hotplug_irqs, enabled_irqs);
31753206 } else if (INTEL_GEN(dev_priv) >= 7) {
31763207 hotplug_irqs = DE_DP_A_HOTPLUG_IVB;
3177− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_ivb);
3208+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
31783209
31793210 ilk_update_display_irq(dev_priv, hotplug_irqs, enabled_irqs);
31803211 } else {
31813212 hotplug_irqs = DE_DP_A_HOTPLUG;
3182− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_ilk);
3213+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
31833214
31843215 ilk_update_display_irq(dev_priv, hotplug_irqs, enabled_irqs);
31853216 }
@@ -3230,7 +3261,7 @@ static void bxt_hpd_irq_setup(struct drm_i915_private *dev_priv) · 44 unmodified lines
32303261 {
32313262 u32 hotplug_irqs, enabled_irqs;
32323263
3233− enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_bxt);
3264+ enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
32343265 hotplug_irqs = BXT_DE_PORT_HOTPLUG_MASK;
32353266
32363267 bdw_update_port_irq(dev_priv, hotplug_irqs, enabled_irqs);
@@ -3361,7 +3392,7 @@ static void gen8_de_irq_postinstall(struct drm_i915_private *dev_priv) · 124 unmodified lines
33613392 u32 de_pipe_masked = gen8_de_pipe_fault_mask(dev_priv) |
33623393 GEN8_PIPE_CDCLK_CRC_DONE;
33633394 u32 de_pipe_enables;
3364− u32 de_port_masked = GEN8_AUX_CHANNEL_A;
3395+ u32 de_port_masked = gen8_de_port_aux_mask(dev_priv);
33653396 u32 de_port_enables;
33663397 u32 de_misc_masked = GEN8_DE_EDP_PSR;
33673398 enum pipe pipe;
@@ -3369,18 +3400,8 @@ static void gen8_de_irq_postinstall(struct drm_i915_private *dev_priv) · 1 unmodified line
33693400 if (INTEL_GEN(dev_priv) <= 10)
33703401 de_misc_masked |= GEN8_DE_MISC_GSE;
33713402
3372− if (INTEL_GEN(dev_priv) >= 9) {
3373− de_port_masked |= GEN9_AUX_CHANNEL_B | GEN9_AUX_CHANNEL_C |
3374− GEN9_AUX_CHANNEL_D;
3375− if (IS_GEN9_LP(dev_priv))
3376− de_port_masked |= BXT_DE_PORT_GMBUS;
3377− }
3378−
3379− if (INTEL_GEN(dev_priv) >= 11)
3380− de_port_masked |= ICL_AUX_CHANNEL_E;
3381−
3382− if (IS_CNL_WITH_PORT_F(dev_priv) || INTEL_GEN(dev_priv) >= 11)
3383− de_port_masked |= CNL_AUX_CHANNEL_F;
3403+ if (IS_GEN9_LP(dev_priv))
3404+ de_port_masked |= BXT_DE_PORT_GMBUS;
33843405
33853406 de_pipe_enables = de_pipe_masked | GEN8_PIPE_VBLANK |
33863407 GEN8_PIPE_FIFO_UNDERRUN;
@@ -3934,6 +3955,8 @@ void intel_irq_init(struct drm_i915_private *dev_priv) · 547 unmodified lines
39343955 struct drm_device *dev = &dev_priv->drm;
39353956 int i;
39363957
3958+ intel_hpd_init_pins(dev_priv);
3959+
39373960 intel_hpd_init_work(dev_priv);
39383961
39393962 INIT_WORK(&dev_priv->l3_parity.error_work, ivb_parity_work);
drivers/gpu/drm/i915/i915_pci.c+1−1
@@ -615,7 +615,7 @@ static const struct intel_device_info chv_info = { · 614 unmodified lines
615615 .has_logical_ring_contexts = 1,
616616 .display.has_gmch = 1,
617617 .dma_mask_size = 39,
618− .ppgtt_type = INTEL_PPGTT_ALIASING,
618+ .ppgtt_type = INTEL_PPGTT_FULL,
619619 .ppgtt_size = 32,
620620 .has_reset_engine = 1,
621621 .has_snoop = true,
drivers/gpu/drm/i915/i915_perf.c+4−8
@@ -1612,10 +1612,7 @@ static int alloc_noa_wait(struct i915_perf_stream *stream) · 1611 unmodified lines
16121612 struct drm_i915_gem_object *bo;
16131613 struct i915_vma *vma;
16141614 const u64 delay_ticks = 0xffffffffffffffff -
1615− DIV64_U64_ROUND_UP(
1616− atomic64_read(&stream->perf->noa_programming_delay) *
1617− RUNTIME_INFO(i915)->cs_timestamp_frequency_khz,
1618− 1000000ull);
1615+ i915_cs_timestamp_ns_to_ticks(i915, atomic64_read(&stream->perf->noa_programming_delay));
16191616 const u32 base = stream->engine->mmio_base;
16201617 #define CS_GPR(x) GEN8_RING_CS_GPR(base, x)
16211618 u32 *batch, *ts0, *cs, *jump;
@@ -3485,8 +3482,7 @@ err: · 1863 unmodified lines
34853482
34863483 static u64 oa_exponent_to_ns(struct i915_perf *perf, int exponent)
34873484 {
3488− return div64_u64(1000000000ULL * (2ULL << exponent),
3489− 1000ULL * RUNTIME_INFO(perf->i915)->cs_timestamp_frequency_khz);
3485+ return i915_cs_timestamp_ticks_to_ns(perf->i915, 2ULL << exponent);
34903486 }
34913487
34923488 /**
@@ -4344,8 +4340,8 @@ void i915_perf_init(struct drm_i915_private *i915) · 851 unmodified lines
43444340 if (perf->ops.enable_metric_set) {
43454341 mutex_init(&perf->lock);
43464342
4347− oa_sample_rate_hard_limit = 1000 *
4348− (RUNTIME_INFO(i915)->cs_timestamp_frequency_khz / 2);
4343+ oa_sample_rate_hard_limit =
4344+ RUNTIME_INFO(i915)->cs_timestamp_frequency_hz / 2;
43494345
43504346 mutex_init(&perf->metrics_lock);
43514347 idr_init(&perf->metrics_idr);
drivers/gpu/drm/i915/i915_pmu.c+4−1
@@ -442,6 +442,7 @@ static u64 count_interrupts(struct drm_i915_private *i915) · 441 unmodified lines
442442 static void i915_pmu_event_destroy(struct perf_event *event)
443443 {
444444 WARN_ON(event->parent);
445+ module_put(THIS_MODULE);
445446 }
446447
447448 static int
@@ -533,8 +534,10 @@ static int i915_pmu_event_init(struct perf_event *event) · 85 unmodified lines
533534 if (ret)
534535 return ret;
535536
536− if (!event->parent)
537+ if (!event->parent) {
538+ __module_get(THIS_MODULE);
537539 event->destroy = i915_pmu_event_destroy;
540+ }
538541
539542 return 0;
540543 }
drivers/gpu/drm/i915/i915_priolist_types.h+1−6
@@ -24,15 +24,12 @@ enum { · 23 unmodified lines
2424 I915_PRIORITY_DISPLAY,
2525 };
2626
27−#define I915_USER_PRIORITY_SHIFT 2
27+#define I915_USER_PRIORITY_SHIFT 0
2828 #define I915_USER_PRIORITY(x) ((x) << I915_USER_PRIORITY_SHIFT)
2929
3030 #define I915_PRIORITY_COUNT BIT(I915_USER_PRIORITY_SHIFT)
3131 #define I915_PRIORITY_MASK (I915_PRIORITY_COUNT - 1)
3232
33−#define I915_PRIORITY_WAIT ((u8)BIT(0))
34−#define I915_PRIORITY_NOSEMAPHORE ((u8)BIT(1))
35−
3633 /* Smallest priority value that cannot be bumped. */
3734 #define I915_PRIORITY_INVALID (INT_MIN | (u8)I915_PRIORITY_MASK)
3835
@@ -47,8 +44,6 @@ enum { · 8 unmodified lines
4744 #define I915_PRIORITY_UNPREEMPTABLE INT_MAX
4845 #define I915_PRIORITY_BARRIER INT_MAX
4946
50−#define __NO_PREEMPTION (I915_PRIORITY_WAIT)
51−
5247 struct i915_priolist {
5348 struct list_head requests[I915_PRIORITY_COUNT];
5449 struct rb_node node;
drivers/gpu/drm/i915/i915_reg.h+13−0
@@ -2557,6 +2557,14 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg) · 2556 unmodified lines
25572557 #define GEN10_PAT_INDEX(index) _MMIO(0x40e0 + (index) * 4)
25582558 #define GEN12_PAT_INDEX(index) _MMIO(0x4800 + (index) * 4)
25592559 #define BSD_HWS_PGA_GEN7 _MMIO(0x04180)
2560+#define GEN12_GFX_CCS_AUX_NV _MMIO(0x4208)
2561+#define GEN12_VD0_AUX_NV _MMIO(0x4218)
2562+#define GEN12_VD1_AUX_NV _MMIO(0x4228)
2563+#define GEN12_VD2_AUX_NV _MMIO(0x4298)
2564+#define GEN12_VD3_AUX_NV _MMIO(0x42A8)
2565+#define GEN12_VE0_AUX_NV _MMIO(0x4238)
2566+#define GEN12_VE1_AUX_NV _MMIO(0x42B8)
2567+#define AUX_INV REG_BIT(0)
25602568 #define BLT_HWS_PGA_GEN7 _MMIO(0x04280)
25612569 #define VEBOX_HWS_PGA_GEN7 _MMIO(0x04380)
25622570 #define RING_ACTHD(base) _MMIO((base) + 0x74)
@@ -8573,6 +8581,7 @@ enum { · 6010 unmodified lines
85738581 #define FDI_BC_BIFURCATION_SELECT (1 << 12)
85748582 #define CHASSIS_CLK_REQ_DURATION_MASK (0xf << 8)
85758583 #define CHASSIS_CLK_REQ_DURATION(x) ((x) << 8)
8584+#define SBCLK_RUN_REFCLK_DIS (1 << 7)
85768585 #define SPT_PWM_GRANULARITY (1 << 0)
85778586 #define SOUTH_CHICKEN2 _MMIO(0xc2004)
85788587 #define FDI_MPHY_IOSFSB_RESET_STATUS (1 << 13)
@@ -9064,6 +9073,7 @@ enum { · 485 unmodified lines
90649073 #define GEN7_PCODE_ILLEGAL_DATA 0x3
90659074 #define GEN11_PCODE_ILLEGAL_SUBCOMMAND 0x4
90669075 #define GEN11_PCODE_LOCKED 0x6
9076+#define GEN11_PCODE_REJECTED 0x11
90679077 #define GEN7_PCODE_MIN_FREQ_TABLE_GT_RATIO_OUT_OF_RANGE 0x10
90689078 #define GEN6_PCODE_WRITE_RC6VIDS 0x4
90699079 #define GEN6_PCODE_READ_RC6VIDS 0x5
@@ -9085,6 +9095,9 @@ enum { · 15 unmodified lines
90859095 #define ICL_PCODE_MEM_SUBSYSYSTEM_INFO 0xd
90869096 #define ICL_PCODE_MEM_SS_READ_GLOBAL_INFO (0x0 << 8)
90879097 #define ICL_PCODE_MEM_SS_READ_QGV_POINT_INFO(point) (((point) << 16) | (0x1 << 8))
9098+#define ICL_PCODE_SAGV_DE_MEM_SS_CONFIG 0xe
9099+#define ICL_PCODE_POINTS_RESTRICTED 0x0
9100+#define ICL_PCODE_POINTS_RESTRICTED_MASK 0x1
90889101 #define GEN6_PCODE_READ_D_COMP 0x10
90899102 #define GEN6_PCODE_WRITE_D_COMP 0x11
90909103 #define ICL_PCODE_EXIT_TCCOLD 0x12
drivers/gpu/drm/i915/i915_request.c+82−45
@@ -23,6 +23,7 @@ · 22 unmodified lines
2323 */
2424
2525 #include <linux/dma-fence-array.h>
26+#include <linux/dma-fence-chain.h>
2627 #include <linux/irq_work.h>
2728 #include <linux/prefetch.h>
2829 #include <linux/sched.h>
@@ -367,8 +368,6 @@ __await_execution(struct i915_request *rq, · 338 unmodified lines
367368 }
368369 spin_unlock_irq(&signal->lock);
369370
370− /* Copy across semaphore status as we need the same behaviour */
371− rq->sched.flags |= signal->sched.flags;
372371 return 0;
373372 }
374373
@@ -536,10 +535,8 @@ void __i915_request_unsubmit(struct i915_request *request) · 161 unmodified lines
536535 spin_unlock(&request->lock);
537536
538537 /* We've already spun, don't charge on resubmitting. */
539− if (request->sched.semaphores && i915_request_started(request)) {
540− request->sched.attr.priority |= I915_PRIORITY_NOSEMAPHORE;
538+ if (request->sched.semaphores && i915_request_started(request))
541539 request->sched.semaphores = 0;
542− }
543540
544541 /*
545542 * We don't need to wake_up any waiters on request->execute, they
@@ -597,15 +594,6 @@ submit_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state) · 51 unmodified lines
597594 return NOTIFY_DONE;
598595 }
599596
600−static void irq_semaphore_cb(struct irq_work *wrk)
601−{
602− struct i915_request *rq =
603− container_of(wrk, typeof(*rq), semaphore_work);
604−
605− i915_schedule_bump_priority(rq, I915_PRIORITY_NOSEMAPHORE);
606− i915_request_put(rq);
607−}
608−
609597 static int __i915_sw_fence_call
610598 semaphore_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
611599 {
@@ -613,11 +601,6 @@ semaphore_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state) · 1 unmodified line
613601
614602 switch (state) {
615603 case FENCE_COMPLETE:
616− if (!(READ_ONCE(rq->sched.attr.priority) & I915_PRIORITY_NOSEMAPHORE)) {
617− i915_request_get(rq);
618− init_irq_work(&rq->semaphore_work, irq_semaphore_cb);
619− irq_work_queue(&rq->semaphore_work);
620− }
621604 break;
622605
623606 case FENCE_FREE:
@@ -950,6 +933,7 @@ __emit_semaphore_wait(struct i915_request *to, · 326 unmodified lines
950933 u32 *cs;
951934
952935 GEM_BUG_ON(INTEL_GEN(to->i915) < 8);
936+ GEM_BUG_ON(i915_request_has_initial_breadcrumb(to));
953937
954938 /* We need to pin the signaler's HWSP until we are finished reading. */
955939 err = intel_timeline_read_hwsp(from, to, &hwsp_offset);
@@ -995,13 +979,26 @@ emit_semaphore_wait(struct i915_request *to, · 39 unmodified lines
995979 gfp_t gfp)
996980 {
997981 const intel_engine_mask_t mask = READ_ONCE(from->engine)->mask;
982+ struct i915_sw_fence *wait = &to->submit;
998983
999984 if (!intel_context_use_semaphores(to->context))
1000985 goto await_fence;
1001986
987+ if (i915_request_has_initial_breadcrumb(to))
988+ goto await_fence;
989+
1002990 if (!rcu_access_pointer(from->hwsp_cacheline))
1003991 goto await_fence;
1004992
993+ /*
994+ * If this or its dependents are waiting on an external fence
995+ * that may fail catastrophically, then we want to avoid using
996+ * sempahores as they bypass the fence signaling metadata, and we
997+ * lose the fence->error propagation.
998+ */
999+ if (from->sched.flags & I915_SCHED_HAS_EXTERNAL_CHAIN)
1000+ goto await_fence;
1001+
10051002 /* Just emit the first semaphore we see as request space is limited. */
10061003 if (already_busywaiting(to) & mask)
10071004 goto await_fence;
@@ -1017,11 +1014,10 @@ emit_semaphore_wait(struct i915_request *to, · 9 unmodified lines
10171014 goto await_fence;
10181015
10191016 to->sched.semaphores |= mask;
1020− to->sched.flags |= I915_SCHED_HAS_SEMAPHORE_CHAIN;
1021− return 0;
1017+ wait = &to->semaphore;
10221018
10231019 await_fence:
1024− return i915_sw_fence_await_dma_fence(&to->submit,
1020+ return i915_sw_fence_await_dma_fence(wait,
10251021 &from->fence, 0,
10261022 I915_FENCE_GFP);
10271023 }
@@ -1034,11 +1030,15 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from) · 6 unmodified lines
10341030 GEM_BUG_ON(to == from);
10351031 GEM_BUG_ON(to->timeline == from->timeline);
10361032
1037− if (i915_request_completed(from))
1033+ if (i915_request_completed(from)) {
1034+ i915_sw_fence_set_error_once(&to->submit, from->fence.error);
10381035 return 0;
1036+ }
10391037
10401038 if (to->engine->schedule) {
1041− ret = i915_sched_node_add_dependency(&to->sched, &from->sched);
1039+ ret = i915_sched_node_add_dependency(&to->sched,
1040+ &from->sched,
1041+ I915_DEPENDENCY_EXTERNAL);
10421042 if (ret < 0)
10431043 return ret;
10441044 }
@@ -1052,15 +1052,56 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from) · 7 unmodified lines
10521052 if (ret < 0)
10531053 return ret;
10541054
1055− if (to->sched.flags & I915_SCHED_HAS_SEMAPHORE_CHAIN) {
1056− ret = i915_sw_fence_await_dma_fence(&to->semaphore,
1057− &from->fence, 0,
1058− I915_FENCE_GFP);
1059− if (ret < 0)
1060− return ret;
1055+ return 0;
1056+}
1057+
1058+static void mark_external(struct i915_request *rq)
1059+{
1060+ /*
1061+ * The downside of using semaphores is that we lose metadata passing
1062+ * along the signaling chain. This is particularly nasty when we
1063+ * need to pass along a fatal error such as EFAULT or EDEADLK. For
1064+ * fatal errors we want to scrub the request before it is executed,
1065+ * which means that we cannot preload the request onto HW and have
1066+ * it wait upon a semaphore.
1067+ */
1068+ rq->sched.flags |= I915_SCHED_HAS_EXTERNAL_CHAIN;
1069+}
1070+
1071+static int
1072+__i915_request_await_external(struct i915_request *rq, struct dma_fence *fence)
1073+{
1074+ mark_external(rq);
1075+ return i915_sw_fence_await_dma_fence(&rq->submit, fence,
1076+ i915_fence_context_timeout(rq->i915,
1077+ fence->context),
1078+ I915_FENCE_GFP);
1079+}
1080+
1081+static int
1082+i915_request_await_external(struct i915_request *rq, struct dma_fence *fence)
1083+{
1084+ struct dma_fence *iter;
1085+ int err = 0;
1086+
1087+ if (!to_dma_fence_chain(fence))
1088+ return __i915_request_await_external(rq, fence);
1089+
1090+ dma_fence_chain_for_each(iter, fence) {
1091+ struct dma_fence_chain *chain = to_dma_fence_chain(iter);
1092+
1093+ if (!dma_fence_is_i915(chain->fence)) {
1094+ err = __i915_request_await_external(rq, iter);
1095+ break;
1096+ }
1097+
1098+ err = i915_request_await_dma_fence(rq, chain->fence);
1099+ if (err < 0)
1100+ break;
10611101 }
10621102
1063− return 0;
1103+ dma_fence_put(iter);
1104+ return err;
10641105 }
10651106
10661107 int
@@ -1110,9 +1151,7 @@ i915_request_await_dma_fence(struct i915_request *rq, struct dma_fence *fence) · 43 unmodified lines
11101151 if (dma_fence_is_i915(fence))
11111152 ret = i915_request_await_request(rq, to_request(fence));
11121153 else
1113− ret = i915_sw_fence_await_dma_fence(&rq->submit, fence,
1114− fence->context ? I915_FENCE_TIMEOUT : 0,
1115− I915_FENCE_GFP);
1154+ ret = i915_request_await_external(rq, fence);
11161155 if (ret < 0)
11171156 return ret;
11181157
@@ -1192,7 +1231,8 @@ __i915_request_await_execution(struct i915_request *to, · 73 unmodified lines
11921231 * immediate execution, and so we must wait until it reaches the
11931232 * active slot.
11941233 */
1195− if (intel_engine_has_semaphores(to->engine)) {
1234+ if (intel_engine_has_semaphores(to->engine) &&
1235+ !i915_request_has_initial_breadcrumb(to)) {
11961236 err = __emit_semaphore_wait(to, from, from->fence.seqno - 1);
11971237 if (err < 0)
11981238 return err;
@@ -1200,7 +1240,9 @@ __i915_request_await_execution(struct i915_request *to, · 1 unmodified line
12001240
12011241 /* Couple the dependency tree for PI on this exposed to->fence */
12021242 if (to->engine->schedule) {
1203− err = i915_sched_node_add_dependency(&to->sched, &from->sched);
1243+ err = i915_sched_node_add_dependency(&to->sched,
1244+ &from->sched,
1245+ I915_DEPENDENCY_WEAK);
12041246 if (err < 0)
12051247 return err;
12061248 }
@@ -1236,6 +1278,9 @@ i915_request_await_execution(struct i915_request *rq, · 29 unmodified lines
12361278 continue;
12371279 }
12381280
1281+ if (fence->context == rq->fence.context)
1282+ continue;
1283+
12391284 /*
12401285 * We don't squash repeated fence dependencies here as we
12411286 * want to run our callback in all cases.
@@ -1246,9 +1291,7 @@ i915_request_await_execution(struct i915_request *rq, · 4 unmodified lines
12461291 to_request(fence),
12471292 hook);
12481293 else
1249− ret = i915_sw_fence_await_dma_fence(&rq->submit, fence,
1250− I915_FENCE_TIMEOUT,
1251− GFP_KERNEL);
1294+ ret = i915_request_await_external(rq, fence);
12521295 if (ret < 0)
12531296 return ret;
12541297 } while (--nchild);
@@ -1456,11 +1499,6 @@ void i915_request_add(struct i915_request *rq) · 201 unmodified lines
14561499 attr = ctx->sched;
14571500 rcu_read_unlock();
14581501
1459− if (!(rq->sched.flags & I915_SCHED_HAS_SEMAPHORE_CHAIN))
1460− attr.priority |= I915_PRIORITY_NOSEMAPHORE;
1461− if (list_empty(&rq->sched.signalers_list))
1462− attr.priority |= I915_PRIORITY_WAIT;
1463−
14641502 __i915_request_queue(rq, &attr);
14651503
14661504 mutex_unlock(&tl->mutex);
@@ -1645,7 +1683,6 @@ long i915_request_wait(struct i915_request *rq, · 178 unmodified lines
16451683 if (flags & I915_WAIT_PRIORITY) {
16461684 if (!i915_request_started(rq) && INTEL_GEN(rq->i915) >= 6)
16471685 intel_rps_boost(rq);
1648− i915_schedule_bump_priority(rq, I915_PRIORITY_WAIT);
16491686 }
16501687
16511688 wait.tsk = current;
drivers/gpu/drm/i915/i915_request.h+20−8
@@ -84,19 +84,26 @@ enum { · 83 unmodified lines
8484 I915_FENCE_FLAG_PQUEUE,
8585
8686 /*
87− * I915_FENCE_FLAG_SIGNAL - this request is currently on signal_list
87+ * I915_FENCE_FLAG_HOLD - this request is currently on hold
8888 *
89− * Internal bookkeeping used by the breadcrumb code to track when
90− * a request is on the various signal_list.
89+ * This request has been suspended, pending an ongoing investigation.
9190 */
92− I915_FENCE_FLAG_SIGNAL,
91+ I915_FENCE_FLAG_HOLD,
9392
9493 /*
95− * I915_FENCE_FLAG_HOLD - this request is currently on hold
94+ * I915_FENCE_FLAG_INITIAL_BREADCRUMB - this request has the initial
95+ * breadcrumb that marks the end of semaphore waits and start of the
96+ * user payload.
97+ */
98+ I915_FENCE_FLAG_INITIAL_BREADCRUMB,
99+
100+ /*
101+ * I915_FENCE_FLAG_SIGNAL - this request is currently on signal_list
96102 *
97− * This request has been suspended, pending an ongoing investigation.
103+ * Internal bookkeeping used by the breadcrumb code to track when
104+ * a request is on the various signal_list.
98105 */
99− I915_FENCE_FLAG_HOLD,
106+ I915_FENCE_FLAG_SIGNAL,
100107
101108 /*
102109 * I915_FENCE_FLAG_NOPREEMPT - this request should not be preempted
@@ -209,7 +216,6 @@ struct i915_request { · 106 unmodified lines
209216 };
210217 struct list_head execute_cb;
211218 struct i915_sw_fence semaphore;
212− struct irq_work semaphore_work;
213219
214220 /*
215221 * A list of everyone we wait upon, and everyone who waits upon us.
@@ -390,6 +396,12 @@ static inline bool i915_request_in_priority_queue(const struct i915_request *rq) · 174 unmodified lines
390396 return test_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
391397 }
392398
399+static inline bool
400+i915_request_has_initial_breadcrumb(const struct i915_request *rq)
401+{
402+ return test_bit(I915_FENCE_FLAG_INITIAL_BREADCRUMB, &rq->fence.flags);
403+}
404+
393405 /**
394406 * Returns true if seq1 is later than seq2.
395407 */
drivers/gpu/drm/i915/i915_scheduler.c+8−21
@@ -51,11 +51,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists) · 50 unmodified lines
5151 GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
5252 rb_first(&execlists->queue.rb_root));
5353
54− last_prio = (INT_MAX >> I915_USER_PRIORITY_SHIFT) + 1;
54+ last_prio = INT_MAX;
5555 for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
5656 const struct i915_priolist *p = to_priolist(rb);
5757
58− GEM_BUG_ON(p->priority >= last_prio);
58+ GEM_BUG_ON(p->priority > last_prio);
5959 last_prio = p->priority;
6060
6161 GEM_BUG_ON(!p->used);
@@ -174,7 +174,7 @@ sched_lock_engine(const struct i915_sched_node *node, · 112 unmodified lines
174174
175175 static inline int rq_prio(const struct i915_request *rq)
176176 {
177− return rq->sched.attr.priority | __NO_PREEMPTION;
177+ return rq->sched.attr.priority;
178178 }
179179
180180 static inline bool need_preempt(int prio, int active)
@@ -434,25 +434,12 @@ bool __i915_sched_node_add_dependency(struct i915_sched_node *node, · 253 unmodified lines
434434 dep->waiter = node;
435435 dep->flags = flags;
436436
437− /* Keep track of whether anyone on this chain has a semaphore */
438− if (signal->flags & I915_SCHED_HAS_SEMAPHORE_CHAIN &&
439− !node_started(signal))
440− node->flags |= I915_SCHED_HAS_SEMAPHORE_CHAIN;
441−
442437 /* All set, now publish. Beware the lockless walkers. */
443438 list_add_rcu(&dep->signal_link, &node->signalers_list);
444439 list_add_rcu(&dep->wait_link, &signal->waiters_list);
445440
446− /*
447− * As we do not allow WAIT to preempt inflight requests,
448− * once we have executed a request, along with triggering
449− * any execution callbacks, we must preserve its ordering
450− * within the non-preemptible FIFO.
451− */
452− BUILD_BUG_ON(__NO_PREEMPTION & ~I915_PRIORITY_MASK);
453− if (flags & I915_DEPENDENCY_EXTERNAL)
454− __bump_priority(signal, __NO_PREEMPTION);
455−
441+ /* Propagate the chains */
442+ node->flags |= signal->flags;
456443 ret = true;
457444 }
458445
@@ -462,7 +449,8 @@ bool __i915_sched_node_add_dependency(struct i915_sched_node *node, · 3 unmodified lines
462449 }
463450
464451 int i915_sched_node_add_dependency(struct i915_sched_node *node,
465− struct i915_sched_node *signal)
452+ struct i915_sched_node *signal,
453+ unsigned long flags)
466454 {
467455 struct i915_dependency *dep;
468456
@@ -473,8 +461,7 @@ int i915_sched_node_add_dependency(struct i915_sched_node *node, · 4 unmodified lines
473461 local_bh_disable();
474462
475463 if (!__i915_sched_node_add_dependency(node, signal, dep,
476− I915_DEPENDENCY_EXTERNAL |
477− I915_DEPENDENCY_ALLOC))
464+ flags | I915_DEPENDENCY_ALLOC))
478465 i915_dependency_free(dep);
479466
480467 local_bh_enable(); /* kick submission tasklet */
drivers/gpu/drm/i915/i915_scheduler.h+2−1
@@ -34,7 +34,8 @@ bool __i915_sched_node_add_dependency(struct i915_sched_node *node, · 33 unmodified lines
3434 unsigned long flags);
3535
3636 int i915_sched_node_add_dependency(struct i915_sched_node *node,
37− struct i915_sched_node *signal);
37+ struct i915_sched_node *signal,
38+ unsigned long flags);
3839
3940 void i915_sched_node_fini(struct i915_sched_node *node);
4041
drivers/gpu/drm/i915/i915_scheduler_types.h+2−1
@@ -65,7 +65,7 @@ struct i915_sched_node { · 64 unmodified lines
6565 struct list_head link;
6666 struct i915_sched_attr attr;
6767 unsigned int flags;
68−#define I915_SCHED_HAS_SEMAPHORE_CHAIN BIT(0)
68+#define I915_SCHED_HAS_EXTERNAL_CHAIN BIT(0)
6969 intel_engine_mask_t semaphores;
7070 };
7171
@@ -78,6 +78,7 @@ struct i915_dependency { · 6 unmodified lines
7878 unsigned long flags;
7979 #define I915_DEPENDENCY_ALLOC BIT(0)
8080 #define I915_DEPENDENCY_EXTERNAL BIT(1)
81+#define I915_DEPENDENCY_WEAK BIT(2)
8182 };
8283
8384 #endif /* _I915_SCHEDULER_TYPES_H_ */
drivers/gpu/drm/i915/i915_selftest.h+2−0
@@ -133,4 +133,6 @@ bool __igt_timeout(unsigned long timeout, const char *fmt, ...); · 132 unmodified lines
133133 #define igt_timeout(t, fmt, ...) \
134134 __igt_timeout((t), KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
135135
136+void igt_hexdump(const void *buf, size_t len);
137+
136138 #endif /* !__I915_SELFTEST_H__ */
drivers/gpu/drm/i915/i915_sw_fence.c+4−6
@@ -546,13 +546,11 @@ int __i915_sw_fence_await_dma_fence(struct i915_sw_fence *fence, · 545 unmodified lines
546546 cb->fence = fence;
547547 i915_sw_fence_await(fence);
548548
549− ret = dma_fence_add_callback(dma, &cb->base, __dma_i915_sw_fence_wake);
550− if (ret == 0) {
551− ret = 1;
552− } else {
549+ ret = 1;
550+ if (dma_fence_add_callback(dma, &cb->base, __dma_i915_sw_fence_wake)) {
551+ /* fence already signaled */
553552 __dma_i915_sw_fence_wake(dma, &cb->base);
554− if (ret == -ENOENT) /* fence already signaled */
555− ret = 0;
553+ ret = 0;
556554 }
557555
558556 return ret;
drivers/gpu/drm/i915/intel_device_info.c+17−17
@@ -136,8 +136,8 @@ void intel_device_info_print_runtime(const struct intel_runtime_info *info, · 135 unmodified lines
136136 sseu_dump(&info->sseu, p);
137137
138138 drm_printf(p, "rawclk rate: %u kHz\n", info->rawclk_freq);
139− drm_printf(p, "CS timestamp frequency: %u kHz\n",
140− info->cs_timestamp_frequency_khz);
139+ drm_printf(p, "CS timestamp frequency: %u Hz\n",
140+ info->cs_timestamp_frequency_hz);
141141 }
142142
143143 static int sseu_eu_idx(const struct sseu_dev_info *sseu, int slice,
@@ -678,12 +678,12 @@ static u32 read_reference_ts_freq(struct drm_i915_private *dev_priv) · 534 unmodified lines
678678
679679 base_freq = ((ts_override & GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DIVIDER_MASK) >>
680680 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DIVIDER_SHIFT) + 1;
681− base_freq *= 1000;
681+ base_freq *= 1000000;
682682
683683 frac_freq = ((ts_override &
684684 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DENOMINATOR_MASK) >>
685685 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DENOMINATOR_SHIFT);
686− frac_freq = 1000 / (frac_freq + 1);
686+ frac_freq = 1000000 / (frac_freq + 1);
687687
688688 return base_freq + frac_freq;
689689 }
@@ -691,8 +691,8 @@ static u32 read_reference_ts_freq(struct drm_i915_private *dev_priv) · 1 unmodified line
691691 static u32 gen10_get_crystal_clock_freq(struct drm_i915_private *dev_priv,
692692 u32 rpm_config_reg)
693693 {
694− u32 f19_2_mhz = 19200;
695− u32 f24_mhz = 24000;
694+ u32 f19_2_mhz = 19200000;
695+ u32 f24_mhz = 24000000;
696696 u32 crystal_clock = (rpm_config_reg &
697697 GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>
698698 GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;
@@ -711,10 +711,10 @@ static u32 gen10_get_crystal_clock_freq(struct drm_i915_private *dev_priv, · 12 unmodified lines
711711 static u32 gen11_get_crystal_clock_freq(struct drm_i915_private *dev_priv,
712712 u32 rpm_config_reg)
713713 {
714− u32 f19_2_mhz = 19200;
715− u32 f24_mhz = 24000;
716− u32 f25_mhz = 25000;
717− u32 f38_4_mhz = 38400;
714+ u32 f19_2_mhz = 19200000;
715+ u32 f24_mhz = 24000000;
716+ u32 f25_mhz = 25000000;
717+ u32 f38_4_mhz = 38400000;
718718 u32 crystal_clock = (rpm_config_reg &
719719 GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>
720720 GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;
@@ -736,9 +736,9 @@ static u32 gen11_get_crystal_clock_freq(struct drm_i915_private *dev_priv, · 15 unmodified lines
736736
737737 static u32 read_timestamp_frequency(struct drm_i915_private *dev_priv)
738738 {
739− u32 f12_5_mhz = 12500;
740− u32 f19_2_mhz = 19200;
741− u32 f24_mhz = 24000;
739+ u32 f12_5_mhz = 12500000;
740+ u32 f19_2_mhz = 19200000;
741+ u32 f24_mhz = 24000000;
742742
743743 if (INTEL_GEN(dev_priv) <= 4) {
744744 /* PRMs say:
@@ -747,7 +747,7 @@ static u32 read_timestamp_frequency(struct drm_i915_private *dev_priv) · 2 unmodified lines
747747 * hclks." (through the “Clocking Configuration”
748748 * (“CLKCFG”) MCHBAR register)
749749 */
750− return RUNTIME_INFO(dev_priv)->rawclk_freq / 16;
750+ return RUNTIME_INFO(dev_priv)->rawclk_freq * 1000 / 16;
751751 } else if (INTEL_GEN(dev_priv) <= 8) {
752752 /* PRMs say:
753753 *
@@ -1048,11 +1048,11 @@ void intel_device_info_runtime_init(struct drm_i915_private *dev_priv) · 294 unmodified lines
10481048 drm_dbg(&dev_priv->drm, "rawclk rate: %d kHz\n", runtime->rawclk_freq);
10491049
10501050 /* Initialize command stream timestamp frequency */
1051− runtime->cs_timestamp_frequency_khz =
1051+ runtime->cs_timestamp_frequency_hz =
10521052 read_timestamp_frequency(dev_priv);
1053− if (runtime->cs_timestamp_frequency_khz) {
1053+ if (runtime->cs_timestamp_frequency_hz) {
10541054 runtime->cs_timestamp_period_ns =
1055− div_u64(1e6, runtime->cs_timestamp_frequency_khz);
1055+ i915_cs_timestamp_ticks_to_ns(dev_priv, 1);
10561056 drm_dbg(&dev_priv->drm,
10571057 "CS timestamp wraparound in %lldms\n",
10581058 div_u64(mul_u32_u32(runtime->cs_timestamp_period_ns,
drivers/gpu/drm/i915/intel_device_info.h+1−1
@@ -221,7 +221,7 @@ struct intel_runtime_info { · 220 unmodified lines
221221
222222 u32 rawclk_freq;
223223
224− u32 cs_timestamp_frequency_khz;
224+ u32 cs_timestamp_frequency_hz;
225225 u32 cs_timestamp_period_ns;
226226
227227 /* Media engine access to SFC per instance */
drivers/gpu/drm/i915/intel_pm.c+266−62
@@ -43,6 +43,7 @@ · 42 unmodified lines
4343 #include "i915_fixed.h"
4444 #include "i915_irq.h"
4545 #include "i915_trace.h"
46+#include "display/intel_bw.h"
4647 #include "intel_pm.h"
4748 #include "intel_sideband.h"
4849 #include "../../../platform/x86/intel_ips.h"
@@ -3637,10 +3638,6 @@ static bool skl_needs_memory_bw_wa(struct drm_i915_private *dev_priv) · 3588 unmodified lines
36373638 static bool
36383639 intel_has_sagv(struct drm_i915_private *dev_priv)
36393640 {
3640− /* HACK! */
3641− if (IS_GEN(dev_priv, 12))
3642− return false;
3643−
36443641 return (IS_GEN9_BC(dev_priv) || INTEL_GEN(dev_priv) >= 10) &&
36453642 dev_priv->sagv_status != I915_SAGV_NOT_CONTROLLED;
36463643 }
@@ -3760,34 +3757,116 @@ intel_disable_sagv(struct drm_i915_private *dev_priv) · 113 unmodified lines
37603757 void intel_sagv_pre_plane_update(struct intel_atomic_state *state)
37613758 {
37623759 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
3760+ const struct intel_bw_state *new_bw_state;
3761+ const struct intel_bw_state *old_bw_state;
3762+ u32 new_mask = 0;
3763+
3764+ /*
3765+ * Just return if we can't control SAGV or don't have it.
3766+ * This is different from situation when we have SAGV but just can't
3767+ * afford it due to DBuf limitation - in case if SAGV is completely
3768+ * disabled in a BIOS, we are not even allowed to send a PCode request,
3769+ * as it will throw an error. So have to check it here.
3770+ */
3771+ if (!intel_has_sagv(dev_priv))
3772+ return;
3773+
3774+ new_bw_state = intel_atomic_get_new_bw_state(state);
3775+ if (!new_bw_state)
3776+ return;
37633777
3764− if (!intel_can_enable_sagv(state))
3778+ if (INTEL_GEN(dev_priv) < 11 && !intel_can_enable_sagv(dev_priv, new_bw_state)) {
37653779 intel_disable_sagv(dev_priv);
3780+ return;
3781+ }
3782+
3783+ old_bw_state = intel_atomic_get_old_bw_state(state);
3784+ /*
3785+ * Nothing to mask
3786+ */
3787+ if (new_bw_state->qgv_points_mask == old_bw_state->qgv_points_mask)
3788+ return;
3789+
3790+ new_mask = old_bw_state->qgv_points_mask | new_bw_state->qgv_points_mask;
3791+
3792+ /*
3793+ * If new mask is zero - means there is nothing to mask,
3794+ * we can only unmask, which should be done in unmask.
3795+ */
3796+ if (!new_mask)
3797+ return;
3798+
3799+ /*
3800+ * Restrict required qgv points before updating the configuration.
3801+ * According to BSpec we can't mask and unmask qgv points at the same
3802+ * time. Also masking should be done before updating the configuration
3803+ * and unmasking afterwards.
3804+ */
3805+ icl_pcode_restrict_qgv_points(dev_priv, new_mask);
37663806 }
37673807
37683808 void intel_sagv_post_plane_update(struct intel_atomic_state *state)
37693809 {
37703810 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
3811+ const struct intel_bw_state *new_bw_state;
3812+ const struct intel_bw_state *old_bw_state;
3813+ u32 new_mask = 0;
3814+
3815+ /*
3816+ * Just return if we can't control SAGV or don't have it.
3817+ * This is different from situation when we have SAGV but just can't
3818+ * afford it due to DBuf limitation - in case if SAGV is completely
3819+ * disabled in a BIOS, we are not even allowed to send a PCode request,
3820+ * as it will throw an error. So have to check it here.
3821+ */
3822+ if (!intel_has_sagv(dev_priv))
3823+ return;
3824+
3825+ new_bw_state = intel_atomic_get_new_bw_state(state);
3826+ if (!new_bw_state)
3827+ return;
37713828
3772− if (intel_can_enable_sagv(state))
3829+ if (INTEL_GEN(dev_priv) < 11 && intel_can_enable_sagv(dev_priv, new_bw_state)) {
37733830 intel_enable_sagv(dev_priv);
3831+ return;
3832+ }
3833+
3834+ old_bw_state = intel_atomic_get_old_bw_state(state);
3835+ /*
3836+ * Nothing to unmask
3837+ */
3838+ if (new_bw_state->qgv_points_mask == old_bw_state->qgv_points_mask)
3839+ return;
3840+
3841+ new_mask = new_bw_state->qgv_points_mask;
3842+
3843+ /*
3844+ * Allow required qgv points after updating the configuration.
3845+ * According to BSpec we can't mask and unmask qgv points at the same
3846+ * time. Also masking should be done before updating the configuration
3847+ * and unmasking afterwards.
3848+ */
3849+ icl_pcode_restrict_qgv_points(dev_priv, new_mask);
37743850 }
37753851
3776−static bool intel_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state)
3852+static bool skl_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state)
37773853 {
3778− struct drm_device *dev = crtc_state->uapi.crtc->dev;
3779− struct drm_i915_private *dev_priv = to_i915(dev);
37803854 struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
3855+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
37813856 struct intel_plane *plane;
3857+ const struct intel_plane_state *plane_state;
37823858 int level, latency;
37833859
3860+ if (!intel_has_sagv(dev_priv))
3861+ return false;
3862+
37843863 if (!crtc_state->hw.active)
37853864 return true;
37863865
37873866 if (crtc_state->hw.adjusted_mode.flags & DRM_MODE_FLAG_INTERLACE)
37883867 return false;
37893868
3790− for_each_intel_plane_on_crtc(dev, crtc, plane) {
3869+ intel_atomic_crtc_state_for_each_plane_state(plane, plane_state, crtc_state) {
37913870 const struct skl_plane_wm *wm =
37923871 &crtc_state->wm.skl.optimal.planes[plane->id];
37933872
@@ -3803,7 +3882,7 @@ static bool intel_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state · 9 unmodified lines
38033882 latency = dev_priv->wm.skl_latency[level];
38043883
38053884 if (skl_needs_memory_bw_wa(dev_priv) &&
3806− plane->base.state->fb->modifier ==
3885+ plane_state->uapi.fb->modifier ==
38073886 I915_FORMAT_MOD_X_TILED)
38083887 latency += 15;
38093888
@@ -3819,35 +3898,110 @@ static bool intel_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state · 9 unmodified lines
38193898 return true;
38203899 }
38213900
3822−bool intel_can_enable_sagv(struct intel_atomic_state *state)
3901+static bool tgl_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state)
3902+{
3903+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
3904+ enum plane_id plane_id;
3905+
3906+ if (!crtc_state->hw.active)
3907+ return true;
3908+
3909+ for_each_plane_id_on_crtc(crtc, plane_id) {
3910+ const struct skl_ddb_entry *plane_alloc =
3911+ &crtc_state->wm.skl.plane_ddb_y[plane_id];
3912+ const struct skl_plane_wm *wm =
3913+ &crtc_state->wm.skl.optimal.planes[plane_id];
3914+
3915+ if (skl_ddb_entry_size(plane_alloc) < wm->sagv_wm0.min_ddb_alloc)
3916+ return false;
3917+ }
3918+
3919+ return true;
3920+}
3921+
3922+static bool intel_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state)
3923+{
3924+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
3925+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
3926+
3927+ if (INTEL_GEN(dev_priv) >= 12)
3928+ return tgl_crtc_can_enable_sagv(crtc_state);
3929+ else
3930+ return skl_crtc_can_enable_sagv(crtc_state);
3931+}
3932+
3933+bool intel_can_enable_sagv(struct drm_i915_private *dev_priv,
3934+ const struct intel_bw_state *bw_state)
3935+{
3936+ if (INTEL_GEN(dev_priv) < 11 &&
3937+ bw_state->active_pipes && !is_power_of_2(bw_state->active_pipes))
3938+ return false;
3939+
3940+ return bw_state->pipe_sagv_reject == 0;
3941+}
3942+
3943+static int intel_compute_sagv_mask(struct intel_atomic_state *state)
38233944 {
38243945 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
3946+ int ret;
38253947 struct intel_crtc *crtc;
3826− const struct intel_crtc_state *crtc_state;
3827− enum pipe pipe;
3948+ struct intel_crtc_state *new_crtc_state;
3949+ struct intel_bw_state *new_bw_state = NULL;
3950+ const struct intel_bw_state *old_bw_state = NULL;
3951+ int i;
38283952
3829− if (!intel_has_sagv(dev_priv))
3830− return false;
3953+ for_each_new_intel_crtc_in_state(state, crtc,
3954+ new_crtc_state, i) {
3955+ new_bw_state = intel_atomic_get_bw_state(state);
3956+ if (IS_ERR(new_bw_state))
3957+ return PTR_ERR(new_bw_state);
38313958
3832− /*
3833− * If there are no active CRTCs, no additional checks need be performed
3834− */
3835− if (hweight8(state->active_pipes) == 0)
3836− return true;
3959+ old_bw_state = intel_atomic_get_old_bw_state(state);
38373960
3838− /*
3839− * SKL+ workaround: bspec recommends we disable SAGV when we have
3840− * more then one pipe enabled
3841− */
3842− if (hweight8(state->active_pipes) > 1)
3843− return false;
3961+ if (intel_crtc_can_enable_sagv(new_crtc_state))
3962+ new_bw_state->pipe_sagv_reject &= ~BIT(crtc->pipe);
3963+ else
3964+ new_bw_state->pipe_sagv_reject |= BIT(crtc->pipe);
3965+ }
3966+
3967+ if (!new_bw_state)
3968+ return 0;
3969+
3970+ new_bw_state->active_pipes =
3971+ intel_calc_active_pipes(state, old_bw_state->active_pipes);
3972+
3973+ if (new_bw_state->active_pipes != old_bw_state->active_pipes) {
3974+ ret = intel_atomic_lock_global_state(&new_bw_state->base);
3975+ if (ret)
3976+ return ret;
3977+ }
3978+
3979+ for_each_new_intel_crtc_in_state(state, crtc,
3980+ new_crtc_state, i) {
3981+ struct skl_pipe_wm *pipe_wm = &new_crtc_state->wm.skl.optimal;
38443982
3845− /* Since we're now guaranteed to only have one active CRTC... */
3846− pipe = ffs(state->active_pipes) - 1;
3847− crtc = intel_get_crtc_for_pipe(dev_priv, pipe);
3848− crtc_state = to_intel_crtc_state(crtc->base.state);
3983+ /*
3984+ * We store use_sagv_wm in the crtc state rather than relying on
3985+ * that bw state since we have no convenient way to get at the
3986+ * latter from the plane commit hooks (especially in the legacy
3987+ * cursor case)
3988+ */
3989+ pipe_wm->use_sagv_wm = INTEL_GEN(dev_priv) >= 12 &&
3990+ intel_can_enable_sagv(dev_priv, new_bw_state);
3991+ }
3992+
3993+ if (intel_can_enable_sagv(dev_priv, new_bw_state) !=
3994+ intel_can_enable_sagv(dev_priv, old_bw_state)) {
3995+ ret = intel_atomic_serialize_global_state(&new_bw_state->base);
3996+ if (ret)
3997+ return ret;
3998+ } else if (new_bw_state->pipe_sagv_reject != old_bw_state->pipe_sagv_reject) {
3999+ ret = intel_atomic_lock_global_state(&new_bw_state->base);
4000+ if (ret)
4001+ return ret;
4002+ }
38494003
3850− return intel_crtc_can_enable_sagv(crtc_state);
4004+ return 0;
38514005 }
38524006
38534007 /*
@@ -4574,6 +4728,20 @@ icl_get_total_relative_data_rate(struct intel_crtc_state *crtc_state, · 720 unmodified lines
45744728 return total_data_rate;
45754729 }
45764730
4731+static const struct skl_wm_level *
4732+skl_plane_wm_level(const struct intel_crtc_state *crtc_state,
4733+ enum plane_id plane_id,
4734+ int level)
4735+{
4736+ const struct skl_pipe_wm *pipe_wm = &crtc_state->wm.skl.optimal;
4737+ const struct skl_plane_wm *wm = &pipe_wm->planes[plane_id];
4738+
4739+ if (level == 0 && pipe_wm->use_sagv_wm)
4740+ return &wm->sagv_wm0;
4741+
4742+ return &wm->wm[level];
4743+}
4744+
45774745 static int
45784746 skl_allocate_pipe_ddb(struct intel_crtc_state *crtc_state)
45794747 {
@@ -4610,7 +4778,6 @@ skl_allocate_pipe_ddb(struct intel_crtc_state *crtc_state) · 30 unmodified lines
46104778 plane_data_rate,
46114779 uv_plane_data_rate);
46124780
4613−
46144781 skl_ddb_get_pipe_allocation_limits(dev_priv, crtc_state, total_data_rate,
46154782 alloc, &num_active);
46164783 alloc_size = skl_ddb_entry_size(alloc);
@@ -4810,7 +4977,7 @@ skl_wm_method1(const struct drm_i915_private *dev_priv, u32 pixel_rate, · 193 unmodified lines
48104977 wm_intermediate_val = latency * pixel_rate * cpp;
48114978 ret = div_fixed16(wm_intermediate_val, 1000 * dbuf_block_size);
48124979
4813− if (INTEL_GEN(dev_priv) >= 10)
4980+ if (INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv))
48144981 ret = add_fixed16_u32(ret, 1);
48154982
48164983 return ret;
@@ -4945,18 +5112,19 @@ skl_compute_wm_params(const struct intel_crtc_state *crtc_state, · 128 unmodified lines
49455112 wp->y_min_scanlines,
49465113 wp->dbuf_block_size);
49475114
4948− if (INTEL_GEN(dev_priv) >= 10)
5115+ if (INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv))
49495116 interm_pbpl++;
49505117
49515118 wp->plane_blocks_per_line = div_fixed16(interm_pbpl,
49525119 wp->y_min_scanlines);
4953− } else if (wp->x_tiled && IS_GEN(dev_priv, 9)) {
4954− interm_pbpl = DIV_ROUND_UP(wp->plane_bytes_per_line,
4955− wp->dbuf_block_size);
4956− wp->plane_blocks_per_line = u32_to_fixed16(interm_pbpl);
49575120 } else {
49585121 interm_pbpl = DIV_ROUND_UP(wp->plane_bytes_per_line,
4959− wp->dbuf_block_size) + 1;
5122+ wp->dbuf_block_size);
5123+
5124+ if (!wp->x_tiled ||
5125+ INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv))
5126+ interm_pbpl++;
5127+
49605128 wp->plane_blocks_per_line = u32_to_fixed16(interm_pbpl);
49615129 }
49625130
@@ -5022,7 +5190,7 @@ static void skl_compute_plane_wm(const struct intel_crtc_state *crtc_state, · 59 unmodified lines
50225190 * WaIncreaseLatencyIPCEnabled: kbl,cfl
50235191 * Display WA #1141: kbl,cfl
50245192 */
5025− if ((IS_KABYLAKE(dev_priv) || IS_COFFEELAKE(dev_priv)) ||
5193+ if ((IS_KABYLAKE(dev_priv) || IS_COFFEELAKE(dev_priv)) &&
50265194 dev_priv->ipc_enabled)
50275195 latency += 4;
50285196
@@ -5145,6 +5313,20 @@ skl_compute_wm_levels(const struct intel_crtc_state *crtc_state, · 116 unmodified lines
51455313 }
51465314 }
51475315
5316+static void tgl_compute_sagv_wm(const struct intel_crtc_state *crtc_state,
5317+ const struct skl_wm_params *wm_params,
5318+ struct skl_plane_wm *plane_wm)
5319+{
5320+ struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
5321+ struct skl_wm_level *sagv_wm = &plane_wm->sagv_wm0;
5322+ struct skl_wm_level *levels = plane_wm->wm;
5323+ unsigned int latency = dev_priv->wm.skl_latency[0] + dev_priv->sagv_block_time_us;
5324+
5325+ skl_compute_plane_wm(crtc_state, 0, latency,
5326+ wm_params, &levels[0],
5327+ sagv_wm);
5328+}
5329+
51485330 static void skl_compute_transition_wm(const struct intel_crtc_state *crtc_state,
51495331 const struct skl_wm_params *wp,
51505332 struct skl_plane_wm *wm)
@@ -5197,10 +5379,6 @@ static void skl_compute_transition_wm(const struct intel_crtc_state *crtc_state, · 46 unmodified lines
51975379 trans_offset_b;
51985380 } else {
51995381 res_blocks = wm0_sel_res_b + trans_offset_b;
5200−
5201− /* WA BUG:1938466 add one block for non y-tile planes */
5202− if (IS_CNL_REVID(dev_priv, CNL_REVID_A0, CNL_REVID_A0))
5203− res_blocks += 1;
52045382 }
52055383
52065384 /*
@@ -5216,6 +5394,8 @@ static int skl_build_plane_wm_single(struct intel_crtc_state *crtc_state, · 9 unmodified lines
52165394 const struct intel_plane_state *plane_state,
52175395 enum plane_id plane_id, int color_plane)
52185396 {
5397+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
5398+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
52195399 struct skl_plane_wm *wm = &crtc_state->wm.skl.optimal.planes[plane_id];
52205400 struct skl_wm_params wm_params;
52215401 int ret;
@@ -5226,6 +5406,10 @@ static int skl_build_plane_wm_single(struct intel_crtc_state *crtc_state, · 4 unmodified lines
52265406 return ret;
52275407
52285408 skl_compute_wm_levels(crtc_state, &wm_params, wm->wm);
5409+
5410+ if (INTEL_GEN(dev_priv) >= 12)
5411+ tgl_compute_sagv_wm(crtc_state, &wm_params, wm);
5412+
52295413 skl_compute_transition_wm(crtc_state, &wm_params, wm);
52305414
52315415 return 0;
@@ -5385,8 +5569,12 @@ void skl_write_plane_wm(struct intel_plane *plane, · 153 unmodified lines
53855569 &crtc_state->wm.skl.plane_ddb_uv[plane_id];
53865570
53875571 for (level = 0; level <= max_level; level++) {
5572+ const struct skl_wm_level *wm_level;
5573+
5574+ wm_level = skl_plane_wm_level(crtc_state, plane_id, level);
5575+
53885576 skl_write_wm_level(dev_priv, PLANE_WM(pipe, plane_id, level),
5389− &wm->wm[level]);
5577+ wm_level);
53905578 }
53915579 skl_write_wm_level(dev_priv, PLANE_WM_TRANS(pipe, plane_id),
53925580 &wm->trans_wm);
@@ -5419,8 +5607,12 @@ void skl_write_cursor_wm(struct intel_plane *plane, · 26 unmodified lines
54195607 &crtc_state->wm.skl.plane_ddb_y[plane_id];
54205608
54215609 for (level = 0; level <= max_level; level++) {
5610+ const struct skl_wm_level *wm_level;
5611+
5612+ wm_level = skl_plane_wm_level(crtc_state, plane_id, level);
5613+
54225614 skl_write_wm_level(dev_priv, CUR_WM(pipe, level),
5423− &wm->wm[level]);
5615+ wm_level);
54245616 }
54255617 skl_write_wm_level(dev_priv, CUR_WM_TRANS(pipe), &wm->trans_wm);
54265618
@@ -5584,23 +5776,25 @@ skl_print_wm_changes(struct intel_atomic_state *state) · 157 unmodified lines
55845776 continue;
55855777
55865778 drm_dbg_kms(&dev_priv->drm,
5587− "[PLANE:%d:%s] level %cwm0,%cwm1,%cwm2,%cwm3,%cwm4,%cwm5,%cwm6,%cwm7,%ctwm"
5588− " -> %cwm0,%cwm1,%cwm2,%cwm3,%cwm4,%cwm5,%cwm6,%cwm7,%ctwm\n",
5779+ "[PLANE:%d:%s] level %cwm0,%cwm1,%cwm2,%cwm3,%cwm4,%cwm5,%cwm6,%cwm7,%ctwm,%cswm"
5780+ " -> %cwm0,%cwm1,%cwm2,%cwm3,%cwm4,%cwm5,%cwm6,%cwm7,%ctwm,%cswm\n",
55895781 plane->base.base.id, plane->base.name,
55905782 enast(old_wm->wm[0].plane_en), enast(old_wm->wm[1].plane_en),
55915783 enast(old_wm->wm[2].plane_en), enast(old_wm->wm[3].plane_en),
55925784 enast(old_wm->wm[4].plane_en), enast(old_wm->wm[5].plane_en),
55935785 enast(old_wm->wm[6].plane_en), enast(old_wm->wm[7].plane_en),
55945786 enast(old_wm->trans_wm.plane_en),
5787+ enast(old_wm->sagv_wm0.plane_en),
55955788 enast(new_wm->wm[0].plane_en), enast(new_wm->wm[1].plane_en),
55965789 enast(new_wm->wm[2].plane_en), enast(new_wm->wm[3].plane_en),
55975790 enast(new_wm->wm[4].plane_en), enast(new_wm->wm[5].plane_en),
55985791 enast(new_wm->wm[6].plane_en), enast(new_wm->wm[7].plane_en),
5599− enast(new_wm->trans_wm.plane_en));
5792+ enast(new_wm->trans_wm.plane_en),
5793+ enast(new_wm->sagv_wm0.plane_en));
56005794
56015795 drm_dbg_kms(&dev_priv->drm,
5602− "[PLANE:%d:%s] lines %c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d"
5603− " -> %c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d\n",
5796+ "[PLANE:%d:%s] lines %c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d"
5797+ " -> %c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d\n",
56045798 plane->base.base.id, plane->base.name,
56055799 enast(old_wm->wm[0].ignore_lines), old_wm->wm[0].plane_res_l,
56065800 enast(old_wm->wm[1].ignore_lines), old_wm->wm[1].plane_res_l,
@@ -5611,6 +5805,7 @@ skl_print_wm_changes(struct intel_atomic_state *state) · 4 unmodified lines
56115805 enast(old_wm->wm[6].ignore_lines), old_wm->wm[6].plane_res_l,
56125806 enast(old_wm->wm[7].ignore_lines), old_wm->wm[7].plane_res_l,
56135807 enast(old_wm->trans_wm.ignore_lines), old_wm->trans_wm.plane_res_l,
5808+ enast(old_wm->sagv_wm0.ignore_lines), old_wm->sagv_wm0.plane_res_l,
56145809
56155810 enast(new_wm->wm[0].ignore_lines), new_wm->wm[0].plane_res_l,
56165811 enast(new_wm->wm[1].ignore_lines), new_wm->wm[1].plane_res_l,
@@ -5620,37 +5815,42 @@ skl_print_wm_changes(struct intel_atomic_state *state) · 3 unmodified lines
56205815 enast(new_wm->wm[5].ignore_lines), new_wm->wm[5].plane_res_l,
56215816 enast(new_wm->wm[6].ignore_lines), new_wm->wm[6].plane_res_l,
56225817 enast(new_wm->wm[7].ignore_lines), new_wm->wm[7].plane_res_l,
5623− enast(new_wm->trans_wm.ignore_lines), new_wm->trans_wm.plane_res_l);
5818+ enast(new_wm->trans_wm.ignore_lines), new_wm->trans_wm.plane_res_l,
5819+ enast(new_wm->sagv_wm0.ignore_lines), new_wm->sagv_wm0.plane_res_l);
56245820
56255821 drm_dbg_kms(&dev_priv->drm,
5626− "[PLANE:%d:%s] blocks %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d"
5627− " -> %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d\n",
5822+ "[PLANE:%d:%s] blocks %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d"
5823+ " -> %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d\n",
56285824 plane->base.base.id, plane->base.name,
56295825 old_wm->wm[0].plane_res_b, old_wm->wm[1].plane_res_b,
56305826 old_wm->wm[2].plane_res_b, old_wm->wm[3].plane_res_b,
56315827 old_wm->wm[4].plane_res_b, old_wm->wm[5].plane_res_b,
56325828 old_wm->wm[6].plane_res_b, old_wm->wm[7].plane_res_b,
56335829 old_wm->trans_wm.plane_res_b,
5830+ old_wm->sagv_wm0.plane_res_b,
56345831 new_wm->wm[0].plane_res_b, new_wm->wm[1].plane_res_b,
56355832 new_wm->wm[2].plane_res_b, new_wm->wm[3].plane_res_b,
56365833 new_wm->wm[4].plane_res_b, new_wm->wm[5].plane_res_b,
56375834 new_wm->wm[6].plane_res_b, new_wm->wm[7].plane_res_b,
5638− new_wm->trans_wm.plane_res_b);
5835+ new_wm->trans_wm.plane_res_b,
5836+ new_wm->sagv_wm0.plane_res_b);
56395837
56405838 drm_dbg_kms(&dev_priv->drm,
5641− "[PLANE:%d:%s] min_ddb %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d"
5642− " -> %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d\n",
5839+ "[PLANE:%d:%s] min_ddb %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d"
5840+ " -> %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d\n",
56435841 plane->base.base.id, plane->base.name,
56445842 old_wm->wm[0].min_ddb_alloc, old_wm->wm[1].min_ddb_alloc,
56455843 old_wm->wm[2].min_ddb_alloc, old_wm->wm[3].min_ddb_alloc,
56465844 old_wm->wm[4].min_ddb_alloc, old_wm->wm[5].min_ddb_alloc,
56475845 old_wm->wm[6].min_ddb_alloc, old_wm->wm[7].min_ddb_alloc,
56485846 old_wm->trans_wm.min_ddb_alloc,
5847+ old_wm->sagv_wm0.min_ddb_alloc,
56495848 new_wm->wm[0].min_ddb_alloc, new_wm->wm[1].min_ddb_alloc,
56505849 new_wm->wm[2].min_ddb_alloc, new_wm->wm[3].min_ddb_alloc,
56515850 new_wm->wm[4].min_ddb_alloc, new_wm->wm[5].min_ddb_alloc,
56525851 new_wm->wm[6].min_ddb_alloc, new_wm->wm[7].min_ddb_alloc,
5653− new_wm->trans_wm.min_ddb_alloc);
5852+ new_wm->trans_wm.min_ddb_alloc,
5853+ new_wm->sagv_wm0.min_ddb_alloc);
56545854 }
56555855 }
56565856 }
@@ -5811,6 +6011,10 @@ skl_compute_wm(struct intel_atomic_state *state) · 154 unmodified lines
58116011 if (ret)
58126012 return ret;
58136013
6014+ ret = intel_compute_sagv_mask(state);
6015+ if (ret)
6016+ return ret;
6017+
58146018 /*
58156019 * skl_compute_ddb() will have adjusted the final watermarks
58166020 * based on how much ddb is available. Now we can actually
@@ -5939,6 +6143,9 @@ void skl_pipe_wm_get_hw_state(struct intel_crtc *crtc, · 122 unmodified lines
59396143 skl_wm_level_from_reg_val(val, &wm->wm[level]);
59406144 }
59416145
6146+ if (INTEL_GEN(dev_priv) >= 12)
6147+ wm->sagv_wm0 = wm->wm[0];
6148+
59426149 if (plane_id != PLANE_CURSOR)
59436150 val = I915_READ(PLANE_WM_TRANS(pipe, plane_id));
59446151 else
@@ -6916,9 +7123,6 @@ static void cnl_init_clock_gating(struct drm_i915_private *dev_priv) · 971 unmodified lines
69167123 val = I915_READ(SLICE_UNIT_LEVEL_CLKGATE);
69177124 /* ReadHitWriteOnlyDisable:cnl */
69187125 val |= RCCUNIT_CLKGATE_DIS;
6919− /* WaSarbUnitClockGatingDisable:cnl (pre-prod) */
6920− if (IS_CNL_REVID(dev_priv, CNL_REVID_A0, CNL_REVID_B0))
6921− val |= SARBUNIT_CLKGATE_DIS;
69227126 I915_WRITE(SLICE_UNIT_LEVEL_CLKGATE, val);
69237127
69247128 /* Wa_2201832410:cnl */
drivers/gpu/drm/i915/intel_pm.h+3−1
@@ -9,6 +9,7 @@ · 8 unmodified lines
99 #include <linux/types.h>
1010
1111 #include "i915_reg.h"
12+#include "display/intel_bw.h"
1213
1314 struct drm_device;
1415 struct drm_i915_private;
@@ -41,7 +42,8 @@ void skl_pipe_wm_get_hw_state(struct intel_crtc *crtc, · 26 unmodified lines
4142 struct skl_pipe_wm *out);
4243 void g4x_wm_sanitize(struct drm_i915_private *dev_priv);
4344 void vlv_wm_sanitize(struct drm_i915_private *dev_priv);
44−bool intel_can_enable_sagv(struct intel_atomic_state *state);
45+bool intel_can_enable_sagv(struct drm_i915_private *dev_priv,
46+ const struct intel_bw_state *bw_state);
4547 int intel_enable_sagv(struct drm_i915_private *dev_priv);
4648 int intel_disable_sagv(struct drm_i915_private *dev_priv);
4749 void intel_sagv_pre_plane_update(struct intel_atomic_state *state);
drivers/gpu/drm/i915/intel_sideband.c+3−1
@@ -371,6 +371,8 @@ static int gen7_check_mailbox_status(u32 mbox) · 370 unmodified lines
371371 return -ENXIO;
372372 case GEN11_PCODE_LOCKED:
373373 return -EBUSY;
374+ case GEN11_PCODE_REJECTED:
375+ return -EACCES;
374376 case GEN7_PCODE_MIN_FREQ_TABLE_GT_RATIO_OUT_OF_RANGE:
375377 return -EOVERFLOW;
376378 default:
@@ -429,7 +431,7 @@ int sandybridge_pcode_read(struct drm_i915_private *i915, u32 mbox, · 52 unmodified lines
429431
430432 mutex_lock(&i915->sb_lock);
431433 err = __sandybridge_pcode_rw(i915, mbox, val, val1,
432− 500, 0,
434+ 500, 20,
433435 true);
434436 mutex_unlock(&i915->sb_lock);
435437
drivers/gpu/drm/i915/intel_uncore.c+0−5
@@ -899,11 +899,6 @@ find_fw_domain(struct intel_uncore *uncore, u32 offset) · 898 unmodified lines
899899 #define GEN_FW_RANGE(s, e, d) \
900900 { .start = (s), .end = (e), .domains = (d) }
901901
902−#define HAS_FWTABLE(dev_priv) \
903− (INTEL_GEN(dev_priv) >= 9 || \
904− IS_CHERRYVIEW(dev_priv) || \
905− IS_VALLEYVIEW(dev_priv))
906−
907902 /* *Must* be sorted by offset ranges! See intel_fw_table_check(). */
908903 static const struct intel_forcewake_range __vlv_fw_ranges[] = {
909904 GEN_FW_RANGE(0x2000, 0x3fff, FORCEWAKE_RENDER),
drivers/gpu/drm/i915/intel_wopcm.c+1−2
@@ -149,8 +149,7 @@ static bool check_hw_restrictions(struct drm_i915_private *i915, · 148 unmodified lines
149149 guc_wopcm_size))
150150 return false;
151151
152− if ((IS_GEN(i915, 9) ||
153− IS_CNL_REVID(i915, CNL_REVID_A0, CNL_REVID_A0)) &&
152+ if (IS_GEN(i915, 9) &&
154153 !gen9_check_huc_fw_fits(i915, guc_wopcm_size, huc_fw_size))
155154 return false;
156155
drivers/gpu/drm/i915/selftests/i915_live_selftests.h+1−0
@@ -37,6 +37,7 @@ selftest(gem, i915_gem_live_selftests) · 36 unmodified lines
3737 selftest(evict, i915_gem_evict_live_selftests)
3838 selftest(hugepages, i915_gem_huge_page_live_selftests)
3939 selftest(gem_contexts, i915_gem_context_live_selftests)
40+selftest(gem_execbuf, i915_gem_execbuffer_live_selftests)
4041 selftest(blt, i915_gem_object_blt_live_selftests)
4142 selftest(client, i915_gem_client_blt_live_selftests)
4243 selftest(reset, intel_reset_live_selftests)
drivers/gpu/drm/i915/selftests/i915_perf.c+2−4
@@ -221,8 +221,7 @@ static int live_noa_delay(void *arg) · 220 unmodified lines
221221 goto out;
222222 }
223223
224− if (rq->engine->emit_init_breadcrumb &&
225− i915_request_timeline(rq)->has_initial_breadcrumb) {
224+ if (rq->engine->emit_init_breadcrumb) {
226225 err = rq->engine->emit_init_breadcrumb(rq);
227226 if (err) {
228227 i915_request_add(rq);
@@ -263,8 +262,7 @@ static int live_noa_delay(void *arg) · 34 unmodified lines
263262
264263 delay = intel_read_status_page(stream->engine, 0x102);
265264 delay -= intel_read_status_page(stream->engine, 0x100);
266− delay = div_u64(mul_u32_u32(delay, 1000 * 1000),
267− RUNTIME_INFO(i915)->cs_timestamp_frequency_khz);
265+ delay = i915_cs_timestamp_ticks_to_ns(i915, delay);
268266 pr_info("GPU delay: %uns, expected %lluns\n",
269267 delay, expected);
270268
drivers/gpu/drm/i915/selftests/i915_request.c+21−16
@@ -816,10 +816,12 @@ static int recursive_batch_resolve(struct i915_vma *batch) · 815 unmodified lines
816816 return PTR_ERR(cmd);
817817
818818 *cmd = MI_BATCH_BUFFER_END;
819− intel_gt_chipset_flush(batch->vm->gt);
820819
820+ __i915_gem_object_flush_map(batch->obj, 0, sizeof(*cmd));
821821 i915_gem_object_unpin_map(batch->obj);
822822
823+ intel_gt_chipset_flush(batch->vm->gt);
824+
823825 return 0;
824826 }
825827
@@ -865,13 +867,6 @@ static int live_all_engines(void *arg) · 39 unmodified lines
865867 goto out_request;
866868 }
867869
868− err = engine->emit_bb_start(request[idx],
869− batch->node.start,
870− batch->node.size,
871− 0);
872− GEM_BUG_ON(err);
873− request[idx]->batch = batch;
874−
875870 i915_vma_lock(batch);
876871 err = i915_request_await_object(request[idx], batch->obj, 0);
877872 if (err == 0)
@@ -879,6 +874,13 @@ static int live_all_engines(void *arg) · 1 unmodified line
879874 i915_vma_unlock(batch);
880875 GEM_BUG_ON(err);
881876
877+ err = engine->emit_bb_start(request[idx],
878+ batch->node.start,
879+ batch->node.size,
880+ 0);
881+ GEM_BUG_ON(err);
882+ request[idx]->batch = batch;
883+
882884 i915_request_get(request[idx]);
883885 i915_request_add(request[idx]);
884886 idx++;
@@ -993,13 +995,6 @@ static int live_sequential_engines(void *arg) · 108 unmodified lines
993995 }
994996 }
995997
996− err = engine->emit_bb_start(request[idx],
997− batch->node.start,
998− batch->node.size,
999− 0);
1000− GEM_BUG_ON(err);
1001− request[idx]->batch = batch;
1002−
1003998 i915_vma_lock(batch);
1004999 err = i915_request_await_object(request[idx],
10051000 batch->obj, false);
@@ -1008,6 +1003,13 @@ static int live_sequential_engines(void *arg) · 2 unmodified lines
10081003 i915_vma_unlock(batch);
10091004 GEM_BUG_ON(err);
10101005
1006+ err = engine->emit_bb_start(request[idx],
1007+ batch->node.start,
1008+ batch->node.size,
1009+ 0);
1010+ GEM_BUG_ON(err);
1011+ request[idx]->batch = batch;
1012+
10111013 i915_request_get(request[idx]);
10121014 i915_request_add(request[idx]);
10131015
@@ -1060,9 +1062,12 @@ out_request: · 46 unmodified lines
10601062 I915_MAP_WC);
10611063 if (!IS_ERR(cmd)) {
10621064 *cmd = MI_BATCH_BUFFER_END;
1063− intel_gt_chipset_flush(engine->gt);
10641065
1066+ __i915_gem_object_flush_map(request[idx]->batch->obj,
1067+ 0, sizeof(*cmd));
10651068 i915_gem_object_unpin_map(request[idx]->batch->obj);
1069+
1070+ intel_gt_chipset_flush(engine->gt);
10661071 }
10671072
10681073 i915_vma_put(request[idx]->batch);
drivers/gpu/drm/i915/selftests/i915_selftest.c+29−0
@@ -396,6 +396,35 @@ bool __igt_timeout(unsigned long timeout, const char *fmt, ...) · 395 unmodified lines
396396 return true;
397397 }
398398
399+void igt_hexdump(const void *buf, size_t len)
400+{
401+ const size_t rowsize = 8 * sizeof(u32);
402+ const void *prev = NULL;
403+ bool skip = false;
404+ size_t pos;
405+
406+ for (pos = 0; pos < len; pos += rowsize) {
407+ char line[128];
408+
409+ if (prev && !memcmp(prev, buf + pos, rowsize)) {
410+ if (!skip) {
411+ pr_info("*\n");
412+ skip = true;
413+ }
414+ continue;
415+ }
416+
417+ WARN_ON_ONCE(hex_dump_to_buffer(buf + pos, len - pos,
418+ rowsize, sizeof(u32),
419+ line, sizeof(line),
420+ false) >= sizeof(line));
421+ pr_info("[%04zx] %s\n", pos, line);
422+
423+ prev = buf + pos;
424+ skip = false;
425+ }
426+}
427+
399428 module_param_named(st_random_seed, i915_selftest.random_seed, uint, 0400);
400429 module_param_named(st_timeout, i915_selftest.timeout_ms, uint, 0400);
401430 module_param_named(st_filter, i915_selftest.filter, charp, 0400);
drivers/gpu/drm/i915/selftests/igt_spinner.c+1−2
@@ -169,8 +169,7 @@ igt_spinner_create_request(struct igt_spinner *spin, · 168 unmodified lines
169169
170170 intel_gt_chipset_flush(engine->gt);
171171
172− if (engine->emit_init_breadcrumb &&
173− i915_request_timeline(rq)->has_initial_breadcrumb) {
172+ if (engine->emit_init_breadcrumb) {
174173 err = engine->emit_init_breadcrumb(rq);
175174 if (err)
176175 goto cancel_rq;
drivers/video/hdmi.c+46−19
@@ -1768,20 +1768,21 @@ hdmi_vendor_any_infoframe_unpack(union hdmi_vendor_any_infoframe *frame, · 1767 unmodified lines
17681768 }
17691769
17701770 /**
1771− * hdmi_drm_infoframe_unpack() - unpack binary buffer to a HDMI DRM infoframe
1771+ * hdmi_drm_infoframe_unpack_only() - unpack binary buffer of CTA-861-G DRM
1772+ * infoframe DataBytes to a HDMI DRM
1773+ * infoframe
17721774 * @frame: HDMI DRM infoframe
17731775 * @buffer: source buffer
17741776 * @size: size of buffer
17751777 *
1776− * Unpacks the information contained in binary @buffer into a structured
1777− * @frame of the HDMI Dynamic Range and Mastering (DRM) information frame.
1778− * Also verifies the checksum as required by section 5.3.5 of the HDMI 1.4
1779− * specification.
1778+ * Unpacks CTA-861-G DRM infoframe DataBytes contained in the binary @buffer
1779+ * into a structured @frame of the HDMI Dynamic Range and Mastering (DRM)
1780+ * infoframe.
17801781 *
17811782 * Returns 0 on success or a negative error code on failure.
17821783 */
1783−static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame,
1784− const void *buffer, size_t size)
1784+int hdmi_drm_infoframe_unpack_only(struct hdmi_drm_infoframe *frame,
1785+ const void *buffer, size_t size)
17851786 {
17861787 const u8 *ptr = buffer;
17871788 const u8 *temp;
@@ -1790,23 +1791,13 @@ static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame, · 2 unmodified lines
17901791 int ret;
17911792 int i;
17921793
1793− if (size < HDMI_INFOFRAME_SIZE(DRM))
1794− return -EINVAL;
1795−
1796− if (ptr[0] != HDMI_INFOFRAME_TYPE_DRM ||
1797− ptr[1] != 1 ||
1798− ptr[2] != HDMI_DRM_INFOFRAME_SIZE)
1799− return -EINVAL;
1800−
1801− if (hdmi_infoframe_checksum(buffer, HDMI_INFOFRAME_SIZE(DRM)) != 0)
1794+ if (size < HDMI_DRM_INFOFRAME_SIZE)
18021795 return -EINVAL;
18031796
18041797 ret = hdmi_drm_infoframe_init(frame);
18051798 if (ret)
18061799 return ret;
18071800
1808− ptr += HDMI_INFOFRAME_HEADER_SIZE;
1809−
18101801 frame->eotf = ptr[0] & 0x7;
18111802 frame->metadata_type = ptr[1] & 0x7;
18121803
@@ -1814,7 +1805,7 @@ static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame, · 1 unmodified line
18141805 for (i = 0; i < 3; i++) {
18151806 x_lsb = *temp++;
18161807 x_msb = *temp++;
1817− frame->display_primaries[i].x = (x_msb << 8) | x_lsb;
1808+ frame->display_primaries[i].x = (x_msb << 8) | x_lsb;
18181809 y_lsb = *temp++;
18191810 y_msb = *temp++;
18201811 frame->display_primaries[i].y = (y_msb << 8) | y_lsb;
@@ -1830,6 +1821,42 @@ static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame, · 9 unmodified lines
18301821
18311822 return 0;
18321823 }
1824+EXPORT_SYMBOL(hdmi_drm_infoframe_unpack_only);
1825+
1826+/**
1827+ * hdmi_drm_infoframe_unpack() - unpack binary buffer to a HDMI DRM infoframe
1828+ * @frame: HDMI DRM infoframe
1829+ * @buffer: source buffer
1830+ * @size: size of buffer
1831+ *
1832+ * Unpacks the CTA-861-G DRM infoframe contained in the binary @buffer into
1833+ * a structured @frame of the HDMI Dynamic Range and Mastering (DRM)
1834+ * infoframe. It also verifies the checksum as required by section 5.3.5 of
1835+ * the HDMI 1.4 specification.
1836+ *
1837+ * Returns 0 on success or a negative error code on failure.
1838+ */
1839+static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame,
1840+ const void *buffer, size_t size)
1841+{
1842+ const u8 *ptr = buffer;
1843+ int ret;
1844+
1845+ if (size < HDMI_INFOFRAME_SIZE(DRM))
1846+ return -EINVAL;
1847+
1848+ if (ptr[0] != HDMI_INFOFRAME_TYPE_DRM ||
1849+ ptr[1] != 1 ||
1850+ ptr[2] != HDMI_DRM_INFOFRAME_SIZE)
1851+ return -EINVAL;
1852+
1853+ if (hdmi_infoframe_checksum(buffer, HDMI_INFOFRAME_SIZE(DRM)) != 0)
1854+ return -EINVAL;
1855+
1856+ ret = hdmi_drm_infoframe_unpack_only(frame, ptr + HDMI_INFOFRAME_HEADER_SIZE,
1857+ size - HDMI_INFOFRAME_HEADER_SIZE);
1858+ return ret;
1859+}
18331860
18341861 /**
18351862 * hdmi_infoframe_unpack() - unpack binary buffer to a HDMI infoframe
include/drm/drm_dp_helper.h+3−0
@@ -1348,6 +1348,9 @@ struct drm_dp_vsc_sdp { · 1347 unmodified lines
13481348 enum dp_content_type content_type;
13491349 };
13501350
1351+void drm_dp_vsc_sdp_log(const char *level, struct device *dev,
1352+ const struct drm_dp_vsc_sdp *vsc);
1353+
13511354 int drm_dp_psr_setup_time(const u8 psr_cap[EDP_PSR_RECEIVER_CAP_SIZE]);
13521355
13531356 static inline int
include/linux/hdmi.h+2−0
@@ -219,6 +219,8 @@ ssize_t hdmi_drm_infoframe_pack(struct hdmi_drm_infoframe *frame, void *buffer, · 218 unmodified lines
219219 ssize_t hdmi_drm_infoframe_pack_only(const struct hdmi_drm_infoframe *frame,
220220 void *buffer, size_t size);
221221 int hdmi_drm_infoframe_check(struct hdmi_drm_infoframe *frame);
222+int hdmi_drm_infoframe_unpack_only(struct hdmi_drm_infoframe *frame,
223+ const void *buffer, size_t size);
222224
223225 enum hdmi_spd_sdi {
224226 HDMI_SPD_SDI_UNKNOWN,

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t