History

Merge tag 'drm-intel-next-2020-05-15' of git://anongit.freedesktop.org/drm/drm-intel into drm-next

UAPI Changes: - drm/i915: Show per-engine default property values in sysfs By providing the default values configured into the kernel via sysfs, it is much more convenient for userspace to restore those sane defaults, or at least know what are considered good baseline. This is useful, for example, to cleanup after any failed userspace prior to commencing new jobs. Cross-subsystem Changes: - video/hdmi: Add Unpack only function for DRM infoframe - Includes pull request gvt-next-2020-05-12 Driver Changes: - Restore Cherryview back to full-ppgtt (Chris, Mika) - Document locking guidelines for i915 (Chris, Daniel, Joonas) - Fix GitLab #1746: Handle idling during i915_gem_evict_something busy loops (Chris) - Display WA #1105: Require linear fb stride to be multiple of 512 bytes on gen9/glk (Ville) - Add Wa_14010685332 for ICP/ICL (Matt R) - Restrict w/a 1607087056 for EHL/JSL (Swathi) - Fix interrupt handling for DP AUX transactions on Tigerlake (Imre) - Revert "drm/i915/tgl: Include ro parts of l3 to invalidate" (Mika) - Fix HDC pipeline flush hardware bit on Gen12 (Mika) - Flush L3 when flushing render on Gen12 (Mika) - Invalidate aux table entries forcibly between BB on Gen12 (Mika) - Add aux table invalidate for all engines on Gen12 (Mika) - Force pte cacheline to main memory Gen8+ (Mika) - Add and enable TGL+ SAGV support (Stanislav) - Implement vm_ops->access on i915 mmaps for GDB (Chris, Kristian) - Replace zero-length array with flexible-array (Gustavo) - Improve batch buffer pool effectiveness to mitigate soft-rc6 hit (Chris) - Remove wait priority boosting (Chris) - Keep driver module referenced when PMU is active (Chris) - Sanitize RPS interrupts upon resume (Chris) - Extend pcode read timeout to 20 ms (Chris) - Wait for ACT sent before enabling MST pipe (Ville) - Extend support to async relocations to SNB (Chris) - Remove CNL pre-prod workarounds (Ville) - Don't enable WaIncreaseLatencyIPCEnabled when IPC is disabled (Sultan) - Record the active CCID from before reset (Chris) - Mark concurrent submissions with a weak-dependency (Chris) - Peel dma-fence-chains for await to allow engine-to-engine sync (Lionel) - Prevent using semaphores to chain up to external fences (Chris) - Fix GLK watermark calculations (Ville) - Emit await(batch) before MI_BB_START (Chris) - Reset execlists registers before HWSP (Chris) - Drop no-semaphore boosting in favor of fast timeslicing (Chris) - Fix enabled infoframe states of lspcon (Gwan-gyeong) - Program DP SDPs on pipe updates (Gwan-gyeong) - Stop sending DP SDPs on ddi disable (Gwan-gyeong) - Store CS timestamp frequency in Hz (Ville) - Remove unused HAS_FWTABLE macro (Pascal) - Use batchbuffer chaining for relocations to save ring space (Chris) - Try different engines for relocs if MI ops not supported (Chris, Tvrtko) - Lazily acquire the device wakeref for freeing objects (Chris) - Streamline display code arithmetics around rounding etc. (Ville) - Use bw state for per crtc SAGV evaluation (Stanislav) - Track active_pipes in bw_state (Stanislav) - Nuke mode.vrefresh usage (Ville) - Warn if the FBC is still writing to stolen on removal (Chris) - Added new PCode commands prepping for QGV rescricting (Stansilav) - Stop holding onto the pinned_default_state (Chris) - Propagate error from completed fences (Chris) - Ignore submit-fences on the same timeline (Chris) - Pull waiting on an external dma-fence into its routine (Chris) - Replace the hardcoded I915_FENCE_TIMEOUT with Kconfig (Chris) - Mark up the racy read of execlists->context_tag (Chris) - Tidy up the return handling for completed dma-fences (Chris) - Introduce skl_plane_wm_level accessor (Stanislav) - Extract SKL SAGV checking (Stanislav) - Make active_pipes check skl specific (Stanislav) - Suspend tasklets before resume sanitization (Chris) - Remove redundant exec_fence (Chris) - Mark the addition of the initial-breadcrumb in the request (Chris) - Transfer old virtual breadcrumbs to irq_worker (Chris) - Read the DP SDPs from the video DIP (Gwan-gyeong) - Program DP SDPs with computed configs (Gwan-gyeong) - Add state readout for DP VSC and DP HDR Metadata Infoframe SDP (Gwan-gyeong) - Add compute routine for DP PSR VSC SDP (Gwan-gyeong) - Use new DP VSC SDP compute routine on PSR (Gwan-gyeong) - Restrict qgv points which don't have enough bandwidth. (Stanislav) - Nuke pointless div by 64bit (Ville) - Static checker code fixes (Nathan, Mika, Chris) - Add logging function for DP VSC SDP (Gwan-gyeong) - Include HDMI DRM infoframe, DP HDR metadata and DP VSC SDP in the crtc state dump (Gwan-gyeong) - Make timeslicing explicit engine property (Chris, Tvrtko) - Selftest and debugging improvements (Chris) - Align variable names with BSpec (Ville) - Tidy up gen8+ breadcrumb emission code (Chris) - Turn intel_digital_port_connected() in a vfunc (Ville) - Use stashed away hpd isr bits in intel_digital_port_connected() (Ville) - Extract i915_cs_timestamp_{ns_to_ticks,tick_to_ns}() (Ville)

Dave Airlie committed 6 years ago

Commit 6cf99162 parents bfbe174 3a36aa2105 files · +3097 −1505Browse files

Unified · Split

105 files
  1. Documentation/gpu/i915.rst +46 −0
  2. drivers/gpu/drm/drm_dp_helper.c +174 −0
  3. drivers/gpu/drm/i915/Kconfig.profile +12 −0
  4. drivers/gpu/drm/i915/Makefile +2 −1
  5. drivers/gpu/drm/i915/display/intel_audio.c +29 −48
  6. drivers/gpu/drm/i915/display/intel_bw.c +114 −47
  7. drivers/gpu/drm/i915/display/intel_bw.h +18 −0
  8. drivers/gpu/drm/i915/display/intel_ddi.c +56 −4
  9. drivers/gpu/drm/i915/display/intel_display.c +76 −9
  10. drivers/gpu/drm/i915/display/intel_display_types.h +3 −0
  11. drivers/gpu/drm/i915/display/intel_dp.c +230 −364
  12. drivers/gpu/drm/i915/display/intel_dp.h +10 −5
  13. drivers/gpu/drm/i915/display/intel_dp_mst.c +5 −4
  14. drivers/gpu/drm/i915/display/intel_fbc.c +10 −2
  15. drivers/gpu/drm/i915/display/intel_lspcon.c +1 −1
  16. drivers/gpu/drm/i915/display/intel_psr.c +19 −39
  17. drivers/gpu/drm/i915/display/intel_psr.h +4 −2
  18. drivers/gpu/drm/i915/display/intel_tc.c +4 −3
  19. drivers/gpu/drm/i915/display/intel_tc.h +2 −1
  20. drivers/gpu/drm/i915/display/intel_vbt_defs.h +2 −2
  21. drivers/gpu/drm/i915/gem/i915_gem_clflush.c +1 −1
  22. drivers/gpu/drm/i915/gem/i915_gem_client_blt.c +1 −3
  23. drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +313 −165
  24. drivers/gpu/drm/i915/gem/i915_gem_fence.c +2 −2
  25. drivers/gpu/drm/i915/gem/i915_gem_mman.c +34 −0
  26. drivers/gpu/drm/i915/gem/i915_gem_object.c +0 −3
  27. drivers/gpu/drm/i915/gem/i915_gem_object_blt.c +15 −11
  28. drivers/gpu/drm/i915/gem/i915_gem_object_blt.h +0 −1
  29. drivers/gpu/drm/i915/gem/i915_gem_pages.c +1 −0
  30. drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c +1 −30
  31. drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c +2 −0
  32. drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +32 −17
  33. drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c +171 −0
  34. drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +124 −0
  35. drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c +10 −16
  36. drivers/gpu/drm/i915/gt/gen8_ppgtt.c +13 −2
  37. drivers/gpu/drm/i915/gt/intel_breadcrumbs.c +42 −10
  38. drivers/gpu/drm/i915/gt/intel_engine.h +31 −15
  39. drivers/gpu/drm/i915/gt/intel_engine_cs.c +2 −4
  40. drivers/gpu/drm/i915/gt/intel_engine_pm.c +1 −15
  41. drivers/gpu/drm/i915/gt/intel_engine_pool.h +0 −34
  42. drivers/gpu/drm/i915/gt/intel_engine_types.h +22 −14
  43. drivers/gpu/drm/i915/gt/intel_gpu_commands.h +1 −2
  44. drivers/gpu/drm/i915/gt/intel_gt.c +3 −0
  45. drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.c +76 −38
  46. drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h +37 −0
  47. drivers/gpu/drm/i915/gt/intel_gt_buffer_pool_types.h +9 −6
  48. drivers/gpu/drm/i915/gt/intel_gt_pm.c +6 −4
  49. drivers/gpu/drm/i915/gt/intel_gt_types.h +11 −0
  50. drivers/gpu/drm/i915/gt/intel_lrc.c +224 −96
  51. drivers/gpu/drm/i915/gt/intel_rc6.c +1 −7
  52. drivers/gpu/drm/i915/gt/intel_renderstate.c +9 −7
  53. drivers/gpu/drm/i915/gt/intel_rps.c +4 −1
  54. drivers/gpu/drm/i915/gt/intel_rps.h +1 −0
  55. drivers/gpu/drm/i915/gt/intel_timeline.c +4 −4
  56. drivers/gpu/drm/i915/gt/intel_timeline.h +0 −5
  57. drivers/gpu/drm/i915/gt/intel_workarounds.c +7 −21
  58. drivers/gpu/drm/i915/gt/mock_engine.c +0 −2
  59. drivers/gpu/drm/i915/gt/selftest_context.c +4 −8
  60. drivers/gpu/drm/i915/gt/selftest_lrc.c +42 −44
  61. drivers/gpu/drm/i915/gt/selftest_ring_submission.c +2 −0
  62. drivers/gpu/drm/i915/gt/selftest_rps.c +42 −14
  63. drivers/gpu/drm/i915/gt/sysfs_engines.c +94 −0
  64. drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +1 −1
  65. drivers/gpu/drm/i915/gvt/cmd_parser.c +45 −0
  66. drivers/gpu/drm/i915/gvt/execlist.c +0 −2
  67. drivers/gpu/drm/i915/gvt/gtt.c +1 −0
  68. drivers/gpu/drm/i915/gvt/gtt.h +1 −0
  69. drivers/gpu/drm/i915/gvt/handlers.c +1 −1
  70. drivers/gpu/drm/i915/gvt/scheduler.c +106 −9
  71. drivers/gpu/drm/i915/gvt/scheduler.h +1 −0
  72. drivers/gpu/drm/i915/i915_config.c +15 −0
  73. drivers/gpu/drm/i915/i915_debugfs.c +7 −4
  74. drivers/gpu/drm/i915/i915_drv.h +29 −8
  75. drivers/gpu/drm/i915/i915_gem_evict.c +12 −14
  76. drivers/gpu/drm/i915/i915_getparam.c +1 −1
  77. drivers/gpu/drm/i915/i915_gpu_error.h +1 −1
  78. drivers/gpu/drm/i915/i915_irq.c +125 −102
  79. drivers/gpu/drm/i915/i915_pci.c +1 −1
  80. drivers/gpu/drm/i915/i915_perf.c +4 −8
  81. drivers/gpu/drm/i915/i915_pmu.c +4 −1
  82. drivers/gpu/drm/i915/i915_priolist_types.h +1 −6
  83. drivers/gpu/drm/i915/i915_reg.h +13 −0
  84. drivers/gpu/drm/i915/i915_request.c +82 −45
  85. drivers/gpu/drm/i915/i915_request.h +20 −8
  86. drivers/gpu/drm/i915/i915_scheduler.c +8 −21
  87. drivers/gpu/drm/i915/i915_scheduler.h +2 −1
  88. drivers/gpu/drm/i915/i915_scheduler_types.h +2 −1
  89. drivers/gpu/drm/i915/i915_selftest.h +2 −0
  90. drivers/gpu/drm/i915/i915_sw_fence.c +4 −6
  91. drivers/gpu/drm/i915/intel_device_info.c +17 −17
  92. drivers/gpu/drm/i915/intel_device_info.h +1 −1
  93. drivers/gpu/drm/i915/intel_pm.c +266 −62
  94. drivers/gpu/drm/i915/intel_pm.h +3 −1
  95. drivers/gpu/drm/i915/intel_sideband.c +3 −1
  96. drivers/gpu/drm/i915/intel_uncore.c +0 −5
  97. drivers/gpu/drm/i915/intel_wopcm.c +1 −2
  98. drivers/gpu/drm/i915/selftests/i915_live_selftests.h +1 −0
  99. drivers/gpu/drm/i915/selftests/i915_perf.c +2 −4
  100. drivers/gpu/drm/i915/selftests/i915_request.c +21 −16
  101. drivers/gpu/drm/i915/selftests/i915_selftest.c +29 −0
  102. drivers/gpu/drm/i915/selftests/igt_spinner.c +1 −2
  103. drivers/video/hdmi.c +46 −19
  104. include/drm/drm_dp_helper.h +3 −0
  105. include/linux/hdmi.h +2 −0
Documentation/gpu/i915.rst+46−0
@@ -329,6 +329,52 @@ for execution also include a list of all locations within buffers that · 328 unmodified lines
329refer to GPU-addresses so that the kernel can edit the buffer correctly.329refer to GPU-addresses so that the kernel can edit the buffer correctly.
330This process is dubbed relocation.330This process is dubbed relocation.
331 331
332Locking Guidelines
333------------------
334
335.. note::
336 This is a description of how the locking should be after
337 refactoring is done. Does not necessarily reflect what the locking
338 looks like while WIP.
339
340#. All locking rules and interface contracts with cross-driver interfaces
341 (dma-buf, dma_fence) need to be followed.
342
343#. No struct_mutex anywhere in the code
344
345#. dma_resv will be the outermost lock (when needed) and ww_acquire_ctx
346 is to be hoisted at highest level and passed down within i915_gem_ctx
347 in the call chain
348
349#. While holding lru/memory manager (buddy, drm_mm, whatever) locks
350 system memory allocations are not allowed
351
352 * Enforce this by priming lockdep (with fs_reclaim). If we
353 allocate memory while holding these looks we get a rehash
354 of the shrinker vs. struct_mutex saga, and that would be
355 real bad.
356
357#. Do not nest different lru/memory manager locks within each other.
358 Take them in turn to update memory allocations, relying on the object’s
359 dma_resv ww_mutex to serialize against other operations.
360
361#. The suggestion for lru/memory managers locks is that they are small
362 enough to be spinlocks.
363
364#. All features need to come with exhaustive kernel selftests and/or
365 IGT tests when appropriate
366
367#. All LMEM uAPI paths need to be fully restartable (_interruptible()
368 for all locks/waits/sleeps)
369
370 * Error handling validation through signal injection.
371 Still the best strategy we have for validating GEM uAPI
372 corner cases.
373 Must be excessively used in the IGT, and we need to check
374 that we really have full path coverage of all error cases.
375
376 * -EDEADLK handling with ww_mutex
377
332GEM BO Management Implementation Details378GEM BO Management Implementation Details
333----------------------------------------379----------------------------------------
334 380
drivers/gpu/drm/drm_dp_helper.c+174−0
@@ -1629,3 +1629,177 @@ int drm_dp_set_phy_test_pattern(struct drm_dp_aux *aux, · 1628 unmodified lines
1629 return 0;1629 return 0;
1630}1630}
1631EXPORT_SYMBOL(drm_dp_set_phy_test_pattern);1631EXPORT_SYMBOL(drm_dp_set_phy_test_pattern);
1632
1633static const char *dp_pixelformat_get_name(enum dp_pixelformat pixelformat)
1634{
1635 if (pixelformat < 0 || pixelformat > DP_PIXELFORMAT_RESERVED)
1636 return "Invalid";
1637
1638 switch (pixelformat) {
1639 case DP_PIXELFORMAT_RGB:
1640 return "RGB";
1641 case DP_PIXELFORMAT_YUV444:
1642 return "YUV444";
1643 case DP_PIXELFORMAT_YUV422:
1644 return "YUV422";
1645 case DP_PIXELFORMAT_YUV420:
1646 return "YUV420";
1647 case DP_PIXELFORMAT_Y_ONLY:
1648 return "Y_ONLY";
1649 case DP_PIXELFORMAT_RAW:
1650 return "RAW";
1651 default:
1652 return "Reserved";
1653 }
1654}
1655
1656static const char *dp_colorimetry_get_name(enum dp_pixelformat pixelformat,
1657 enum dp_colorimetry colorimetry)
1658{
1659 if (pixelformat < 0 || pixelformat > DP_PIXELFORMAT_RESERVED)
1660 return "Invalid";
1661
1662 switch (colorimetry) {
1663 case DP_COLORIMETRY_DEFAULT:
1664 switch (pixelformat) {
1665 case DP_PIXELFORMAT_RGB:
1666 return "sRGB";
1667 case DP_PIXELFORMAT_YUV444:
1668 case DP_PIXELFORMAT_YUV422:
1669 case DP_PIXELFORMAT_YUV420:
1670 return "BT.601";
1671 case DP_PIXELFORMAT_Y_ONLY:
1672 return "DICOM PS3.14";
1673 case DP_PIXELFORMAT_RAW:
1674 return "Custom Color Profile";
1675 default:
1676 return "Reserved";
1677 }
1678 case DP_COLORIMETRY_RGB_WIDE_FIXED: /* and DP_COLORIMETRY_BT709_YCC */
1679 switch (pixelformat) {
1680 case DP_PIXELFORMAT_RGB:
1681 return "Wide Fixed";
1682 case DP_PIXELFORMAT_YUV444:
1683 case DP_PIXELFORMAT_YUV422:
1684 case DP_PIXELFORMAT_YUV420:
1685 return "BT.709";
1686 default:
1687 return "Reserved";
1688 }
1689 case DP_COLORIMETRY_RGB_WIDE_FLOAT: /* and DP_COLORIMETRY_XVYCC_601 */
1690 switch (pixelformat) {
1691 case DP_PIXELFORMAT_RGB:
1692 return "Wide Float";
1693 case DP_PIXELFORMAT_YUV444:
1694 case DP_PIXELFORMAT_YUV422:
1695 case DP_PIXELFORMAT_YUV420:
1696 return "xvYCC 601";
1697 default:
1698 return "Reserved";
1699 }
1700 case DP_COLORIMETRY_OPRGB: /* and DP_COLORIMETRY_XVYCC_709 */
1701 switch (pixelformat) {
1702 case DP_PIXELFORMAT_RGB:
1703 return "OpRGB";
1704 case DP_PIXELFORMAT_YUV444:
1705 case DP_PIXELFORMAT_YUV422:
1706 case DP_PIXELFORMAT_YUV420:
1707 return "xvYCC 709";
1708 default:
1709 return "Reserved";
1710 }
1711 case DP_COLORIMETRY_DCI_P3_RGB: /* and DP_COLORIMETRY_SYCC_601 */
1712 switch (pixelformat) {
1713 case DP_PIXELFORMAT_RGB:
1714 return "DCI-P3";
1715 case DP_PIXELFORMAT_YUV444:
1716 case DP_PIXELFORMAT_YUV422:
1717 case DP_PIXELFORMAT_YUV420:
1718 return "sYCC 601";
1719 default:
1720 return "Reserved";
1721 }
1722 case DP_COLORIMETRY_RGB_CUSTOM: /* and DP_COLORIMETRY_OPYCC_601 */
1723 switch (pixelformat) {
1724 case DP_PIXELFORMAT_RGB:
1725 return "Custom Profile";
1726 case DP_PIXELFORMAT_YUV444:
1727 case DP_PIXELFORMAT_YUV422:
1728 case DP_PIXELFORMAT_YUV420:
1729 return "OpYCC 601";
1730 default:
1731 return "Reserved";
1732 }
1733 case DP_COLORIMETRY_BT2020_RGB: /* and DP_COLORIMETRY_BT2020_CYCC */
1734 switch (pixelformat) {
1735 case DP_PIXELFORMAT_RGB:
1736 return "BT.2020 RGB";
1737 case DP_PIXELFORMAT_YUV444:
1738 case DP_PIXELFORMAT_YUV422:
1739 case DP_PIXELFORMAT_YUV420:
1740 return "BT.2020 CYCC";
1741 default:
1742 return "Reserved";
1743 }
1744 case DP_COLORIMETRY_BT2020_YCC:
1745 switch (pixelformat) {
1746 case DP_PIXELFORMAT_YUV444:
1747 case DP_PIXELFORMAT_YUV422:
1748 case DP_PIXELFORMAT_YUV420:
1749 return "BT.2020 YCC";
1750 default:
1751 return "Reserved";
1752 }
1753 default:
1754 return "Invalid";
1755 }
1756}
1757
1758static const char *dp_dynamic_range_get_name(enum dp_dynamic_range dynamic_range)
1759{
1760 switch (dynamic_range) {
1761 case DP_DYNAMIC_RANGE_VESA:
1762 return "VESA range";
1763 case DP_DYNAMIC_RANGE_CTA:
1764 return "CTA range";
1765 default:
1766 return "Invalid";
1767 }
1768}
1769
1770static const char *dp_content_type_get_name(enum dp_content_type content_type)
1771{
1772 switch (content_type) {
1773 case DP_CONTENT_TYPE_NOT_DEFINED:
1774 return "Not defined";
1775 case DP_CONTENT_TYPE_GRAPHICS:
1776 return "Graphics";
1777 case DP_CONTENT_TYPE_PHOTO:
1778 return "Photo";
1779 case DP_CONTENT_TYPE_VIDEO:
1780 return "Video";
1781 case DP_CONTENT_TYPE_GAME:
1782 return "Game";
1783 default:
1784 return "Reserved";
1785 }
1786}
1787
1788void drm_dp_vsc_sdp_log(const char *level, struct device *dev,
1789 const struct drm_dp_vsc_sdp *vsc)
1790{
1791#define DP_SDP_LOG(fmt, ...) dev_printk(level, dev, fmt, ##__VA_ARGS__)
1792 DP_SDP_LOG("DP SDP: %s, revision %u, length %u\n", "VSC",
1793 vsc->revision, vsc->length);
1794 DP_SDP_LOG(" pixelformat: %s\n",
1795 dp_pixelformat_get_name(vsc->pixelformat));
1796 DP_SDP_LOG(" colorimetry: %s\n",
1797 dp_colorimetry_get_name(vsc->pixelformat, vsc->colorimetry));
1798 DP_SDP_LOG(" bpc: %u\n", vsc->bpc);
1799 DP_SDP_LOG(" dynamic range: %s\n",
1800 dp_dynamic_range_get_name(vsc->dynamic_range));
1801 DP_SDP_LOG(" content type: %s\n",
1802 dp_content_type_get_name(vsc->content_type));
1803#undef DP_SDP_LOG
1804}
1805EXPORT_SYMBOL(drm_dp_vsc_sdp_log);
drivers/gpu/drm/i915/Kconfig.profile+12−0
@@ -1,3 +1,15 @@
1config DRM_I915_FENCE_TIMEOUT
2 int "Timeout for unsignaled foreign fences (ms, jiffy granularity)"
3 default 10000 # milliseconds
4 help
5 When listening to a foreign fence, we install a supplementary timer
6 to ensure that we are always signaled and our userspace is able to
7 make forward progress. This value specifies the timeout used for an
8 unsignaled foreign fence.
9
10 May be 0 to disable the timeout, and rely on the foreign fence being
11 eventually signaled.
12
1config DRM_I915_USERFAULT_AUTOSUSPEND13config DRM_I915_USERFAULT_AUTOSUSPEND
2 int "Runtime autosuspend delay for userspace GGTT mmaps (ms)"14 int "Runtime autosuspend delay for userspace GGTT mmaps (ms)"
3 default 250 # milliseconds15 default 250 # milliseconds
drivers/gpu/drm/i915/Makefile+2−1
@@ -35,6 +35,7 @@ subdir-ccflags-y += -I$(srctree)/$(src) · 34 unmodified lines
35 35
36# core driver code36# core driver code
37i915-y += i915_drv.o \37i915-y += i915_drv.o \
38 i915_config.o \
38 i915_irq.o \39 i915_irq.o \
39 i915_getparam.o \40 i915_getparam.o \
40 i915_params.o \41 i915_params.o \
@@ -87,11 +88,11 @@ gt-y += \ · 46 unmodified lines
87 gt/intel_engine_cs.o \88 gt/intel_engine_cs.o \
88 gt/intel_engine_heartbeat.o \89 gt/intel_engine_heartbeat.o \
89 gt/intel_engine_pm.o \90 gt/intel_engine_pm.o \
90 gt/intel_engine_pool.o \
91 gt/intel_engine_user.o \91 gt/intel_engine_user.o \
92 gt/intel_ggtt.o \92 gt/intel_ggtt.o \
93 gt/intel_ggtt_fencing.o \93 gt/intel_ggtt_fencing.o \
94 gt/intel_gt.o \94 gt/intel_gt.o \
95 gt/intel_gt_buffer_pool.o \
95 gt/intel_gt_clock_utils.o \96 gt/intel_gt_clock_utils.o \
96 gt/intel_gt_irq.o \97 gt/intel_gt_irq.o \
97 gt/intel_gt_pm.o \98 gt/intel_gt_pm.o \
drivers/gpu/drm/i915/display/intel_audio.c+29−48
@@ -514,85 +514,67 @@ static void hsw_audio_codec_disable(struct intel_encoder *encoder, · 513 unmodified lines
514 mutex_unlock(&dev_priv->av_mutex);514 mutex_unlock(&dev_priv->av_mutex);
515}515}
516 516
517/* Add a factor to take care of rounding and truncations */517static unsigned int calc_hblank_early_prog(struct intel_encoder *encoder,
518#define ROUNDING_FACTOR 10000518 const struct intel_crtc_state *crtc_state)
519
520static unsigned int get_hblank_early_enable_config(struct intel_encoder *encoder,
521 const struct intel_crtc_state *crtc_state)
522{519{
523 struct drm_i915_private *i915 = to_i915(encoder->base.dev);520 struct drm_i915_private *i915 = to_i915(encoder->base.dev);
524 unsigned int link_clks_available, link_clks_required;521 unsigned int link_clks_available, link_clks_required;
525 unsigned int tu_data, tu_line, link_clks_active;522 unsigned int tu_data, tu_line, link_clks_active;
526 unsigned int hblank_rise, hblank_early_prog;523 unsigned int h_active, h_total, hblank_delta, pixel_clk;
527 unsigned int h_active, h_total, hblank_delta, pixel_clk, v_total;524 unsigned int fec_coeff, cdclk, vdsc_bpp;
528 unsigned int fec_coeff, refresh_rate, cdclk, vdsc_bpp;525 unsigned int link_clk, lanes;
526 unsigned int hblank_rise;
529 527
530 h_active = crtc_state->hw.adjusted_mode.crtc_hdisplay;528 h_active = crtc_state->hw.adjusted_mode.crtc_hdisplay;
531 h_total = crtc_state->hw.adjusted_mode.crtc_htotal;529 h_total = crtc_state->hw.adjusted_mode.crtc_htotal;
532 v_total = crtc_state->hw.adjusted_mode.crtc_vtotal;
533 pixel_clk = crtc_state->hw.adjusted_mode.crtc_clock;530 pixel_clk = crtc_state->hw.adjusted_mode.crtc_clock;
534 refresh_rate = crtc_state->hw.adjusted_mode.vrefresh;
535 vdsc_bpp = crtc_state->dsc.compressed_bpp;531 vdsc_bpp = crtc_state->dsc.compressed_bpp;
536 cdclk = i915->cdclk.hw.cdclk;532 cdclk = i915->cdclk.hw.cdclk;
537 /* fec= 0.972261, using rounding multiplier of 1000000 */533 /* fec= 0.972261, using rounding multiplier of 1000000 */
538 fec_coeff = 972261;534 fec_coeff = 972261;
535 link_clk = crtc_state->port_clock;
536 lanes = crtc_state->lane_count;
539 537
540 drm_dbg_kms(&i915->drm, "h_active = %u link_clk = %u :"538 drm_dbg_kms(&i915->drm, "h_active = %u link_clk = %u :"
541 "lanes = %u vdsc_bpp = %u cdclk = %u\n",539 "lanes = %u vdsc_bpp = %u cdclk = %u\n",
542 h_active, crtc_state->port_clock, crtc_state->lane_count,540 h_active, link_clk, lanes, vdsc_bpp, cdclk);
543 vdsc_bpp, cdclk);
544 541
545 if (WARN_ON(!crtc_state->port_clock || !crtc_state->lane_count ||542 if (WARN_ON(!link_clk || !pixel_clk || !lanes || !vdsc_bpp || !cdclk))
546 !crtc_state->dsc.compressed_bpp || !i915->cdclk.hw.cdclk))
547 return 0;543 return 0;
548 544
549 link_clks_available = ((((h_total - h_active) *545 link_clks_available = (h_total - h_active) * link_clk / pixel_clk - 28;
550 ((crtc_state->port_clock * ROUNDING_FACTOR) /546 link_clks_required = DIV_ROUND_UP(192000 * h_total, 1000 * pixel_clk) * (48 / lanes + 2);
551 pixel_clk)) / ROUNDING_FACTOR) - 28);
552
553 link_clks_required = DIV_ROUND_UP(192000, (refresh_rate *
554 v_total)) * ((48 /
555 crtc_state->lane_count) + 2);
556 547
557 if (link_clks_available > link_clks_required)548 if (link_clks_available > link_clks_required)
558 hblank_delta = 32;549 hblank_delta = 32;
559 else550 else
560 hblank_delta = DIV_ROUND_UP(((((5 * ROUNDING_FACTOR) /551 hblank_delta = DIV64_U64_ROUND_UP(mul_u32_u32(5 * (link_clk + cdclk), pixel_clk),
561 crtc_state->port_clock) + ((5 *552 mul_u32_u32(link_clk, cdclk));
562 ROUNDING_FACTOR) /
563 cdclk)) * pixel_clk),
564 ROUNDING_FACTOR);
565
566 tu_data = (pixel_clk * vdsc_bpp * 8) / ((crtc_state->port_clock *
567 crtc_state->lane_count * fec_coeff) / 1000000);
568 tu_line = (((h_active * crtc_state->port_clock * fec_coeff) /
569 1000000) / (64 * pixel_clk));
570 link_clks_active = (tu_line - 1) * 64 + tu_data;
571 553
572 hblank_rise = ((link_clks_active + 6 * DIV_ROUND_UP(link_clks_active,554 tu_data = div64_u64(mul_u32_u32(pixel_clk * vdsc_bpp * 8, 1000000),
573 250) + 4) * ((pixel_clk * ROUNDING_FACTOR) /555 mul_u32_u32(link_clk * lanes, fec_coeff));
574 crtc_state->port_clock)) / ROUNDING_FACTOR;556 tu_line = div64_u64(h_active * mul_u32_u32(link_clk, fec_coeff),
557 mul_u32_u32(64 * pixel_clk, 1000000));
558 link_clks_active = (tu_line - 1) * 64 + tu_data;
575 559
576 hblank_early_prog = h_active - hblank_rise + hblank_delta;560 hblank_rise = (link_clks_active + 6 * DIV_ROUND_UP(link_clks_active, 250) + 4) * pixel_clk / link_clk;
577 561
578 return hblank_early_prog;562 return h_active - hblank_rise + hblank_delta;
579}563}
580 564
581static unsigned int get_sample_room_req_config(const struct intel_crtc_state *crtc_state)565static unsigned int calc_samples_room(const struct intel_crtc_state *crtc_state)
582{566{
583 unsigned int h_active, h_total, pixel_clk;567 unsigned int h_active, h_total, pixel_clk;
584 unsigned int samples_room;568 unsigned int link_clk, lanes;
585 569
586 h_active = crtc_state->hw.adjusted_mode.hdisplay;570 h_active = crtc_state->hw.adjusted_mode.hdisplay;
587 h_total = crtc_state->hw.adjusted_mode.htotal;571 h_total = crtc_state->hw.adjusted_mode.htotal;
588 pixel_clk = crtc_state->hw.adjusted_mode.clock;572 pixel_clk = crtc_state->hw.adjusted_mode.clock;
573 link_clk = crtc_state->port_clock;
574 lanes = crtc_state->lane_count;
589 575
590 samples_room = ((((h_total - h_active) * ((crtc_state->port_clock *576 return ((h_total - h_active) * link_clk - 12 * pixel_clk) /
591 ROUNDING_FACTOR) / pixel_clk)) /577 (pixel_clk * (48 / lanes + 2));
592 ROUNDING_FACTOR) - 12) / ((48 /
593 crtc_state->lane_count) + 2);
594
595 return samples_room;
596}578}
597 579
598static void enable_audio_dsc_wa(struct intel_encoder *encoder,580static void enable_audio_dsc_wa(struct intel_encoder *encoder,
@@ -618,8 +600,7 @@ static void enable_audio_dsc_wa(struct intel_encoder *encoder, · 19 unmodified lines
618 (crtc_state->hw.adjusted_mode.hdisplay >= 3840 &&600 (crtc_state->hw.adjusted_mode.hdisplay >= 3840 &&
619 crtc_state->hw.adjusted_mode.vdisplay >= 2160)) {601 crtc_state->hw.adjusted_mode.vdisplay >= 2160)) {
620 /* Get hblank early enable value required */602 /* Get hblank early enable value required */
621 hblank_early_prog = get_hblank_early_enable_config(encoder,603 hblank_early_prog = calc_hblank_early_prog(encoder, crtc_state);
622 crtc_state);
623 if (hblank_early_prog < 32) {604 if (hblank_early_prog < 32) {
624 val &= ~HBLANK_START_COUNT_MASK(pipe);605 val &= ~HBLANK_START_COUNT_MASK(pipe);
625 val |= HBLANK_START_COUNT(pipe, HBLANK_START_COUNT_32);606 val |= HBLANK_START_COUNT(pipe, HBLANK_START_COUNT_32);
@@ -635,7 +616,7 @@ static void enable_audio_dsc_wa(struct intel_encoder *encoder, · 9 unmodified lines
635 }616 }
636 617
637 /* Get samples room value required */618 /* Get samples room value required */
638 samples_room = get_sample_room_req_config(crtc_state);619 samples_room = calc_samples_room(crtc_state);
639 if (samples_room < 3) {620 if (samples_room < 3) {
640 val &= ~NUMBER_SAMPLES_PER_LINE_MASK(pipe);621 val &= ~NUMBER_SAMPLES_PER_LINE_MASK(pipe);
641 val |= NUMBER_SAMPLES_PER_LINE(pipe, samples_room);622 val |= NUMBER_SAMPLES_PER_LINE(pipe, samples_room);
drivers/gpu/drm/i915/display/intel_bw.c+114−47
@@ -8,6 +8,9 @@ · 7 unmodified lines
8#include "intel_bw.h"8#include "intel_bw.h"
9#include "intel_display_types.h"9#include "intel_display_types.h"
10#include "intel_sideband.h"10#include "intel_sideband.h"
11#include "intel_atomic.h"
12#include "intel_pm.h"
13
11 14
12/* Parameters for Qclk Geyserville (QGV) */15/* Parameters for Qclk Geyserville (QGV) */
13struct intel_qgv_point {16struct intel_qgv_point {
@@ -113,6 +116,26 @@ static int icl_pcode_read_qgv_point_info(struct drm_i915_private *dev_priv, · 99 unmodified lines
113 return 0;116 return 0;
114}117}
115 118
119int icl_pcode_restrict_qgv_points(struct drm_i915_private *dev_priv,
120 u32 points_mask)
121{
122 int ret;
123
124 /* bspec says to keep retrying for at least 1 ms */
125 ret = skl_pcode_request(dev_priv, ICL_PCODE_SAGV_DE_MEM_SS_CONFIG,
126 points_mask,
127 ICL_PCODE_POINTS_RESTRICTED_MASK,
128 ICL_PCODE_POINTS_RESTRICTED,
129 1);
130
131 if (ret < 0) {
132 drm_err(&dev_priv->drm, "Failed to disable qgv points (%d)\n", ret);
133 return ret;
134 }
135
136 return 0;
137}
138
116static int icl_get_qgv_points(struct drm_i915_private *dev_priv,139static int icl_get_qgv_points(struct drm_i915_private *dev_priv,
117 struct intel_qgv_info *qi)140 struct intel_qgv_info *qi)
118{141{
@@ -240,6 +263,16 @@ static int icl_get_bw_info(struct drm_i915_private *dev_priv, const struct intel · 121 unmodified lines
240 break;263 break;
241 }264 }
242 265
266 /*
267 * In case if SAGV is disabled in BIOS, we always get 1
268 * SAGV point, but we can't send PCode commands to restrict it
269 * as it will fail and pointless anyway.
270 */
271 if (qi.num_points == 1)
272 dev_priv->sagv_status = I915_SAGV_NOT_CONTROLLED;
273 else
274 dev_priv->sagv_status = I915_SAGV_ENABLED;
275
243 return 0;276 return 0;
244}277}
245 278
@@ -248,6 +281,11 @@ static unsigned int icl_max_bw(struct drm_i915_private *dev_priv, · 2 unmodified lines
248{281{
249 int i;282 int i;
250 283
284 /*
285 * Let's return max bw for 0 planes
286 */
287 num_planes = max(1, num_planes);
288
251 for (i = 0; i < ARRAY_SIZE(dev_priv->max_bw); i++) {289 for (i = 0; i < ARRAY_SIZE(dev_priv->max_bw); i++) {
252 const struct intel_bw_info *bi =290 const struct intel_bw_info *bi =
253 &dev_priv->max_bw[i];291 &dev_priv->max_bw[i];
@@ -277,34 +315,6 @@ void intel_bw_init_hw(struct drm_i915_private *dev_priv) · 23 unmodified lines
277 icl_get_bw_info(dev_priv, &icl_sa_info);315 icl_get_bw_info(dev_priv, &icl_sa_info);
278}316}
279 317
280static unsigned int intel_max_data_rate(struct drm_i915_private *dev_priv,
281 int num_planes)
282{
283 if (INTEL_GEN(dev_priv) >= 11) {
284 /*
285 * Any bw group has same amount of QGV points
286 */
287 const struct intel_bw_info *bi =
288 &dev_priv->max_bw[0];
289 unsigned int min_bw = UINT_MAX;
290 int i;
291
292 /*
293 * FIXME with SAGV disabled maybe we can assume
294 * point 1 will always be used? Seems to match
295 * the behaviour observed in the wild.
296 */
297 for (i = 0; i < bi->num_qgv_points; i++) {
298 unsigned int bw = icl_max_bw(dev_priv, num_planes, i);
299
300 min_bw = min(bw, min_bw);
301 }
302 return min_bw;
303 } else {
304 return UINT_MAX;
305 }
306}
307
308static unsigned int intel_bw_crtc_num_active_planes(const struct intel_crtc_state *crtc_state)318static unsigned int intel_bw_crtc_num_active_planes(const struct intel_crtc_state *crtc_state)
309{319{
310 /*320 /*
@@ -414,11 +424,16 @@ int intel_bw_atomic_check(struct intel_atomic_state *state) · 103 unmodified lines
414{424{
415 struct drm_i915_private *dev_priv = to_i915(state->base.dev);425 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
416 struct intel_crtc_state *new_crtc_state, *old_crtc_state;426 struct intel_crtc_state *new_crtc_state, *old_crtc_state;
417 struct intel_bw_state *bw_state = NULL;427 struct intel_bw_state *new_bw_state = NULL;
418 unsigned int data_rate, max_data_rate;428 const struct intel_bw_state *old_bw_state = NULL;
429 unsigned int data_rate;
419 unsigned int num_active_planes;430 unsigned int num_active_planes;
420 struct intel_crtc *crtc;431 struct intel_crtc *crtc;
421 int i, ret;432 int i, ret;
433 u32 allowed_points = 0;
434 unsigned int max_bw_point = 0, max_bw = 0;
435 unsigned int num_qgv_points = dev_priv->max_bw[0].num_qgv_points;
436 u32 mask = (1 << num_qgv_points) - 1;
422 437
423 /* FIXME earlier gens need some checks too */438 /* FIXME earlier gens need some checks too */
424 if (INTEL_GEN(dev_priv) < 11)439 if (INTEL_GEN(dev_priv) < 11)
@@ -443,41 +458,93 @@ int intel_bw_atomic_check(struct intel_atomic_state *state) · 18 unmodified lines
443 old_active_planes == new_active_planes)458 old_active_planes == new_active_planes)
444 continue;459 continue;
445 460
446 bw_state = intel_atomic_get_bw_state(state);461 new_bw_state = intel_atomic_get_bw_state(state);
447 if (IS_ERR(bw_state))462 if (IS_ERR(new_bw_state))
448 return PTR_ERR(bw_state);463 return PTR_ERR(new_bw_state);
449 464
450 bw_state->data_rate[crtc->pipe] = new_data_rate;465 new_bw_state->data_rate[crtc->pipe] = new_data_rate;
451 bw_state->num_active_planes[crtc->pipe] = new_active_planes;466 new_bw_state->num_active_planes[crtc->pipe] = new_active_planes;
452 467
453 drm_dbg_kms(&dev_priv->drm,468 drm_dbg_kms(&dev_priv->drm,
454 "pipe %c data rate %u num active planes %u\n",469 "pipe %c data rate %u num active planes %u\n",
455 pipe_name(crtc->pipe),470 pipe_name(crtc->pipe),
456 bw_state->data_rate[crtc->pipe],471 new_bw_state->data_rate[crtc->pipe],
457 bw_state->num_active_planes[crtc->pipe]);472 new_bw_state->num_active_planes[crtc->pipe]);
458 }473 }
459 474
460 if (!bw_state)475 if (!new_bw_state)
461 return 0;476 return 0;
462 477
463 ret = intel_atomic_lock_global_state(&bw_state->base);478 ret = intel_atomic_lock_global_state(&new_bw_state->base);
464 if (ret)479 if (ret)
465 return ret;480 return ret;
466 481
467 data_rate = intel_bw_data_rate(dev_priv, bw_state);482 data_rate = intel_bw_data_rate(dev_priv, new_bw_state);
468 num_active_planes = intel_bw_num_active_planes(dev_priv, bw_state);483 data_rate = DIV_ROUND_UP(data_rate, 1000);
469 484
470 max_data_rate = intel_max_data_rate(dev_priv, num_active_planes);485 num_active_planes = intel_bw_num_active_planes(dev_priv, new_bw_state);
471 486
472 data_rate = DIV_ROUND_UP(data_rate, 1000);487 for (i = 0; i < num_qgv_points; i++) {
488 unsigned int max_data_rate;
473 489
474 if (data_rate > max_data_rate) {490 max_data_rate = icl_max_bw(dev_priv, num_active_planes, i);
475 drm_dbg_kms(&dev_priv->drm,491 /*
476 "Bandwidth %u MB/s exceeds max available %d MB/s (%d active planes)\n",492 * We need to know which qgv point gives us
477 data_rate, max_data_rate, num_active_planes);493 * maximum bandwidth in order to disable SAGV
494 * if we find that we exceed SAGV block time
495 * with watermarks. By that moment we already
496 * have those, as it is calculated earlier in
497 * intel_atomic_check,
498 */
499 if (max_data_rate > max_bw) {
500 max_bw_point = i;
501 max_bw = max_data_rate;
502 }
503 if (max_data_rate >= data_rate)
504 allowed_points |= BIT(i);
505 drm_dbg_kms(&dev_priv->drm, "QGV point %d: max bw %d required %d\n",
506 i, max_data_rate, data_rate);
507 }
508
509 /*
510 * BSpec states that we always should have at least one allowed point
511 * left, so if we couldn't - simply reject the configuration for obvious
512 * reasons.
513 */
514 if (allowed_points == 0) {
515 drm_dbg_kms(&dev_priv->drm, "No QGV points provide sufficient memory"
516 " bandwidth %d for display configuration(%d active planes).\n",
517 data_rate, num_active_planes);
478 return -EINVAL;518 return -EINVAL;
479 }519 }
480 520
521 /*
522 * Leave only single point with highest bandwidth, if
523 * we can't enable SAGV due to the increased memory latency it may
524 * cause.
525 */
526 if (!intel_can_enable_sagv(dev_priv, new_bw_state)) {
527 allowed_points = BIT(max_bw_point);
528 drm_dbg_kms(&dev_priv->drm, "No SAGV, using single QGV point %d\n",
529 max_bw_point);
530 }
531 /*
532 * We store the ones which need to be masked as that is what PCode
533 * actually accepts as a parameter.
534 */
535 new_bw_state->qgv_points_mask = ~allowed_points & mask;
536
537 old_bw_state = intel_atomic_get_old_bw_state(state);
538 /*
539 * If the actual mask had changed we need to make sure that
540 * the commits are serialized(in case this is a nomodeset, nonblocking)
541 */
542 if (new_bw_state->qgv_points_mask != old_bw_state->qgv_points_mask) {
543 ret = intel_atomic_serialize_global_state(&new_bw_state->base);
544 if (ret)
545 return ret;
546 }
547
481 return 0;548 return 0;
482}549}
483 550
drivers/gpu/drm/i915/display/intel_bw.h+18−0
@@ -18,8 +18,24 @@ struct intel_crtc_state; · 17 unmodified lines
18struct intel_bw_state {18struct intel_bw_state {
19 struct intel_global_state base;19 struct intel_global_state base;
20 20
21 /*
22 * Contains a bit mask, used to determine, whether correspondent
23 * pipe allows SAGV or not.
24 */
25 u8 pipe_sagv_reject;
26
27 /*
28 * Current QGV points mask, which restricts
29 * some particular SAGV states, not to confuse
30 * with pipe_sagv_mask.
31 */
32 u8 qgv_points_mask;
33
21 unsigned int data_rate[I915_MAX_PIPES];34 unsigned int data_rate[I915_MAX_PIPES];
22 u8 num_active_planes[I915_MAX_PIPES];35 u8 num_active_planes[I915_MAX_PIPES];
36
37 /* bitmask of active pipes */
38 u8 active_pipes;
23};39};
24 40
25#define to_intel_bw_state(x) container_of((x), struct intel_bw_state, base)41#define to_intel_bw_state(x) container_of((x), struct intel_bw_state, base)
@@ -38,5 +54,7 @@ int intel_bw_init(struct drm_i915_private *dev_priv); · 12 unmodified lines
38int intel_bw_atomic_check(struct intel_atomic_state *state);54int intel_bw_atomic_check(struct intel_atomic_state *state);
39void intel_bw_crtc_update(struct intel_bw_state *bw_state,55void intel_bw_crtc_update(struct intel_bw_state *bw_state,
40 const struct intel_crtc_state *crtc_state);56 const struct intel_crtc_state *crtc_state);
57int icl_pcode_restrict_qgv_points(struct drm_i915_private *dev_priv,
58 u32 points_mask);
41 59
42#endif /* __INTEL_BW_H__ */60#endif /* __INTEL_BW_H__ */
drivers/gpu/drm/i915/display/intel_ddi.c+56−4
@@ -3472,6 +3472,8 @@ static void intel_ddi_post_disable_dp(struct intel_atomic_state *state, · 3471 unmodified lines
3472 INTEL_OUTPUT_DP_MST);3472 INTEL_OUTPUT_DP_MST);
3473 enum phy phy = intel_port_to_phy(dev_priv, encoder->port);3473 enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
3474 3474
3475 intel_dp_set_infoframes(encoder, false, old_crtc_state, old_conn_state);
3476
3475 /*3477 /*
3476 * Power down sink before disabling the port, otherwise we end3478 * Power down sink before disabling the port, otherwise we end
3477 * up getting interrupts from the sink on detecting link loss.3479 * up getting interrupts from the sink on detecting link loss.
@@ -3680,9 +3682,8 @@ static void intel_enable_ddi_dp(struct intel_atomic_state *state, · 202 unmodified lines
3680 intel_dp_stop_link_train(intel_dp);3682 intel_dp_stop_link_train(intel_dp);
3681 3683
3682 intel_edp_backlight_on(crtc_state, conn_state);3684 intel_edp_backlight_on(crtc_state, conn_state);
3683 intel_psr_enable(intel_dp, crtc_state);3685 intel_psr_enable(intel_dp, crtc_state, conn_state);
3684 intel_dp_vsc_enable(intel_dp, crtc_state, conn_state);3686 intel_dp_set_infoframes(encoder, true, crtc_state, conn_state);
3685 intel_dp_hdr_metadata_enable(intel_dp, crtc_state, conn_state);
3686 intel_edp_drrs_enable(intel_dp, crtc_state);3687 intel_edp_drrs_enable(intel_dp, crtc_state);
3687 3688
3688 if (crtc_state->has_audio)3689 if (crtc_state->has_audio)
@@ -3864,7 +3865,8 @@ static void intel_ddi_update_pipe_dp(struct intel_atomic_state *state, · 175 unmodified lines
3864 3865
3865 intel_ddi_set_dp_msa(crtc_state, conn_state);3866 intel_ddi_set_dp_msa(crtc_state, conn_state);
3866 3867
3867 intel_psr_update(intel_dp, crtc_state);3868 intel_psr_update(intel_dp, crtc_state, conn_state);
3869 intel_dp_set_infoframes(encoder, true, crtc_state, conn_state);
3868 intel_edp_drrs_enable(intel_dp, crtc_state);3870 intel_edp_drrs_enable(intel_dp, crtc_state);
3869 3871
3870 intel_panel_update_backlight(state, encoder, crtc_state, conn_state);3872 intel_panel_update_backlight(state, encoder, crtc_state, conn_state);
@@ -4235,6 +4237,9 @@ void intel_ddi_get_config(struct intel_encoder *encoder, · 364 unmodified lines
4235 pipe_config->fec_enable);4237 pipe_config->fec_enable);
4236 }4238 }
4237 4239
4240 pipe_config->infoframes.enable |=
4241 intel_hdmi_infoframes_enabled(encoder, pipe_config);
4242
4238 break;4243 break;
4239 case TRANS_DDI_MODE_SELECT_DP_MST:4244 case TRANS_DDI_MODE_SELECT_DP_MST:
4240 pipe_config->output_types |= BIT(INTEL_OUTPUT_DP_MST);4245 pipe_config->output_types |= BIT(INTEL_OUTPUT_DP_MST);
@@ -4246,6 +4251,9 @@ void intel_ddi_get_config(struct intel_encoder *encoder, · 5 unmodified lines
4246 REG_FIELD_GET(TRANS_DDI_MST_TRANSPORT_SELECT_MASK, temp);4251 REG_FIELD_GET(TRANS_DDI_MST_TRANSPORT_SELECT_MASK, temp);
4247 4252
4248 intel_dp_get_m_n(intel_crtc, pipe_config);4253 intel_dp_get_m_n(intel_crtc, pipe_config);
4254
4255 pipe_config->infoframes.enable |=
4256 intel_hdmi_infoframes_enabled(encoder, pipe_config);
4249 break;4257 break;
4250 default:4258 default:
4251 break;4259 break;
@@ -4300,6 +4308,9 @@ void intel_ddi_get_config(struct intel_encoder *encoder, · 48 unmodified lines
4300 4308
4301 if (INTEL_GEN(dev_priv) >= 8)4309 if (INTEL_GEN(dev_priv) >= 8)
4302 bdw_get_trans_port_sync_config(pipe_config);4310 bdw_get_trans_port_sync_config(pipe_config);
4311
4312 intel_read_dp_sdp(encoder, pipe_config, HDMI_PACKET_TYPE_GAMUT_METADATA);
4313 intel_read_dp_sdp(encoder, pipe_config, DP_SDP_VSC);
4303}4314}
4304 4315
4305static enum intel_output_type4316static enum intel_output_type
@@ -4689,6 +4700,30 @@ intel_ddi_hotplug(struct intel_encoder *encoder, · 383 unmodified lines
4689 return state;4700 return state;
4690}4701}
4691 4702
4703static bool lpt_digital_port_connected(struct intel_encoder *encoder)
4704{
4705 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
4706 u32 bit = dev_priv->hotplug.pch_hpd[encoder->hpd_pin];
4707
4708 return intel_de_read(dev_priv, SDEISR) & bit;
4709}
4710
4711static bool hsw_digital_port_connected(struct intel_encoder *encoder)
4712{
4713 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
4714 u32 bit = dev_priv->hotplug.hpd[encoder->hpd_pin];
4715
4716 return intel_de_read(dev_priv, DEISR) & bit;
4717}
4718
4719static bool bdw_digital_port_connected(struct intel_encoder *encoder)
4720{
4721 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
4722 u32 bit = dev_priv->hotplug.hpd[encoder->hpd_pin];
4723
4724 return intel_de_read(dev_priv, GEN8_DE_PORT_ISR) & bit;
4725}
4726
4692static struct intel_connector *4727static struct intel_connector *
4693intel_ddi_init_hdmi_connector(struct intel_digital_port *intel_dig_port)4728intel_ddi_init_hdmi_connector(struct intel_digital_port *intel_dig_port)
4694{4729{
@@ -4885,6 +4920,23 @@ void intel_ddi_init(struct drm_i915_private *dev_priv, enum port port) · 190 unmodified lines
4885 port_name(port));4920 port_name(port));
4886 }4921 }
4887 4922
4923 if (INTEL_GEN(dev_priv) >= 11) {
4924 if (intel_phy_is_tc(dev_priv, phy))
4925 intel_dig_port->connected = intel_tc_port_connected;
4926 else
4927 intel_dig_port->connected = lpt_digital_port_connected;
4928 } else if (INTEL_GEN(dev_priv) >= 8) {
4929 if (port == PORT_A || IS_GEN9_LP(dev_priv))
4930 intel_dig_port->connected = bdw_digital_port_connected;
4931 else
4932 intel_dig_port->connected = lpt_digital_port_connected;
4933 } else {
4934 if (port == PORT_A)
4935 intel_dig_port->connected = hsw_digital_port_connected;
4936 else
4937 intel_dig_port->connected = lpt_digital_port_connected;
4938 }
4939
4888 intel_infoframe_init(intel_dig_port);4940 intel_infoframe_init(intel_dig_port);
4889 4941
4890 return;4942 return;
drivers/gpu/drm/i915/display/intel_display.c+76−9
@@ -12866,6 +12866,16 @@ intel_dump_infoframe(struct drm_i915_private *dev_priv, · 12865 unmodified lines
12866 hdmi_infoframe_log(KERN_DEBUG, dev_priv->drm.dev, frame);12866 hdmi_infoframe_log(KERN_DEBUG, dev_priv->drm.dev, frame);
12867}12867}
12868 12868
12869static void
12870intel_dump_dp_vsc_sdp(struct drm_i915_private *dev_priv,
12871 const struct drm_dp_vsc_sdp *vsc)
12872{
12873 if (!drm_debug_enabled(DRM_UT_KMS))
12874 return;
12875
12876 drm_dp_vsc_sdp_log(KERN_DEBUG, dev_priv->drm.dev, vsc);
12877}
12878
12869#define OUTPUT_TYPE(x) [INTEL_OUTPUT_ ## x] = #x12879#define OUTPUT_TYPE(x) [INTEL_OUTPUT_ ## x] = #x
12870 12880
12871static const char * const output_type_str[] = {12881static const char * const output_type_str[] = {
@@ -13023,6 +13033,15 @@ static void intel_dump_pipe_config(const struct intel_crtc_state *pipe_config, · 151 unmodified lines
13023 if (pipe_config->infoframes.enable &13033 if (pipe_config->infoframes.enable &
13024 intel_hdmi_infoframe_enable(HDMI_INFOFRAME_TYPE_VENDOR))13034 intel_hdmi_infoframe_enable(HDMI_INFOFRAME_TYPE_VENDOR))
13025 intel_dump_infoframe(dev_priv, &pipe_config->infoframes.hdmi);13035 intel_dump_infoframe(dev_priv, &pipe_config->infoframes.hdmi);
13036 if (pipe_config->infoframes.enable &
13037 intel_hdmi_infoframe_enable(HDMI_INFOFRAME_TYPE_DRM))
13038 intel_dump_infoframe(dev_priv, &pipe_config->infoframes.drm);
13039 if (pipe_config->infoframes.enable &
13040 intel_hdmi_infoframe_enable(HDMI_PACKET_TYPE_GAMUT_METADATA))
13041 intel_dump_infoframe(dev_priv, &pipe_config->infoframes.drm);
13042 if (pipe_config->infoframes.enable &
13043 intel_hdmi_infoframe_enable(DP_SDP_VSC))
13044 intel_dump_dp_vsc_sdp(dev_priv, &pipe_config->infoframes.vsc);
13026 13045
13027 drm_dbg_kms(&dev_priv->drm, "requested mode:\n");13046 drm_dbg_kms(&dev_priv->drm, "requested mode:\n");
13028 drm_mode_debug_printmodeline(&pipe_config->hw.mode);13047 drm_mode_debug_printmodeline(&pipe_config->hw.mode);
@@ -13470,6 +13489,13 @@ intel_compare_infoframe(const union hdmi_infoframe *a, · 441 unmodified lines
13470 return memcmp(a, b, sizeof(*a)) == 0;13489 return memcmp(a, b, sizeof(*a)) == 0;
13471}13490}
13472 13491
13492static bool
13493intel_compare_dp_vsc_sdp(const struct drm_dp_vsc_sdp *a,
13494 const struct drm_dp_vsc_sdp *b)
13495{
13496 return memcmp(a, b, sizeof(*a)) == 0;
13497}
13498
13473static void13499static void
13474pipe_config_infoframe_mismatch(struct drm_i915_private *dev_priv,13500pipe_config_infoframe_mismatch(struct drm_i915_private *dev_priv,
13475 bool fastset, const char *name,13501 bool fastset, const char *name,
@@ -13495,6 +13521,31 @@ pipe_config_infoframe_mismatch(struct drm_i915_private *dev_priv, · 19 unmodified lines
13495 }13521 }
13496}13522}
13497 13523
13524static void
13525pipe_config_dp_vsc_sdp_mismatch(struct drm_i915_private *dev_priv,
13526 bool fastset, const char *name,
13527 const struct drm_dp_vsc_sdp *a,
13528 const struct drm_dp_vsc_sdp *b)
13529{
13530 if (fastset) {
13531 if (!drm_debug_enabled(DRM_UT_KMS))
13532 return;
13533
13534 drm_dbg_kms(&dev_priv->drm,
13535 "fastset mismatch in %s dp sdp\n", name);
13536 drm_dbg_kms(&dev_priv->drm, "expected:\n");
13537 drm_dp_vsc_sdp_log(KERN_DEBUG, dev_priv->drm.dev, a);
13538 drm_dbg_kms(&dev_priv->drm, "found:\n");
13539 drm_dp_vsc_sdp_log(KERN_DEBUG, dev_priv->drm.dev, b);
13540 } else {
13541 drm_err(&dev_priv->drm, "mismatch in %s dp sdp\n", name);
13542 drm_err(&dev_priv->drm, "expected:\n");
13543 drm_dp_vsc_sdp_log(KERN_ERR, dev_priv->drm.dev, a);
13544 drm_err(&dev_priv->drm, "found:\n");
13545 drm_dp_vsc_sdp_log(KERN_ERR, dev_priv->drm.dev, b);
13546 }
13547}
13548
13498static void __printf(4, 5)13549static void __printf(4, 5)
13499pipe_config_mismatch(bool fastset, const struct intel_crtc *crtc,13550pipe_config_mismatch(bool fastset, const struct intel_crtc *crtc,
13500 const char *name, const char *format, ...)13551 const char *name, const char *format, ...)
@@ -13696,6 +13747,17 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config, · 195 unmodified lines
13696 } \13747 } \
13697} while (0)13748} while (0)
13698 13749
13750#define PIPE_CONF_CHECK_DP_VSC_SDP(name) do { \
13751 if (!current_config->has_psr && !pipe_config->has_psr && \
13752 !intel_compare_dp_vsc_sdp(&current_config->infoframes.name, \
13753 &pipe_config->infoframes.name)) { \
13754 pipe_config_dp_vsc_sdp_mismatch(dev_priv, fastset, __stringify(name), \
13755 &current_config->infoframes.name, \
13756 &pipe_config->infoframes.name); \
13757 ret = false; \
13758 } \
13759} while (0)
13760
13699#define PIPE_CONF_CHECK_COLOR_LUT(name1, name2, bit_precision) do { \13761#define PIPE_CONF_CHECK_COLOR_LUT(name1, name2, bit_precision) do { \
13700 if (current_config->name1 != pipe_config->name1) { \13762 if (current_config->name1 != pipe_config->name1) { \
13701 pipe_config_mismatch(fastset, crtc, __stringify(name1), \13763 pipe_config_mismatch(fastset, crtc, __stringify(name1), \
@@ -13873,6 +13935,7 @@ intel_pipe_config_compare(const struct intel_crtc_state *current_config, · 171 unmodified lines
13873 PIPE_CONF_CHECK_INFOFRAME(spd);13935 PIPE_CONF_CHECK_INFOFRAME(spd);
13874 PIPE_CONF_CHECK_INFOFRAME(hdmi);13936 PIPE_CONF_CHECK_INFOFRAME(hdmi);
13875 PIPE_CONF_CHECK_INFOFRAME(drm);13937 PIPE_CONF_CHECK_INFOFRAME(drm);
13938 PIPE_CONF_CHECK_DP_VSC_SDP(vsc);
13876 13939
13877 PIPE_CONF_CHECK_X(sync_mode_slaves_mask);13940 PIPE_CONF_CHECK_X(sync_mode_slaves_mask);
13878 PIPE_CONF_CHECK_I(master_transcoder);13941 PIPE_CONF_CHECK_I(master_transcoder);
@@ -13961,7 +14024,9 @@ static void verify_wm_state(struct intel_crtc *crtc, · 82 unmodified lines
13961 /* Watermarks */14024 /* Watermarks */
13962 for (level = 0; level <= max_level; level++) {14025 for (level = 0; level <= max_level; level++) {
13963 if (skl_wm_level_equals(&hw_plane_wm->wm[level],14026 if (skl_wm_level_equals(&hw_plane_wm->wm[level],
13964 &sw_plane_wm->wm[level]))14027 &sw_plane_wm->wm[level]) ||
14028 (level == 0 && skl_wm_level_equals(&hw_plane_wm->wm[level],
14029 &sw_plane_wm->sagv_wm0)))
13965 continue;14030 continue;
13966 14031
13967 drm_err(&dev_priv->drm,14032 drm_err(&dev_priv->drm,
@@ -14016,7 +14081,9 @@ static void verify_wm_state(struct intel_crtc *crtc, · 48 unmodified lines
14016 /* Watermarks */14081 /* Watermarks */
14017 for (level = 0; level <= max_level; level++) {14082 for (level = 0; level <= max_level; level++) {
14018 if (skl_wm_level_equals(&hw_plane_wm->wm[level],14083 if (skl_wm_level_equals(&hw_plane_wm->wm[level],
14019 &sw_plane_wm->wm[level]))14084 &sw_plane_wm->wm[level]) ||
14085 (level == 0 && skl_wm_level_equals(&hw_plane_wm->wm[level],
14086 &sw_plane_wm->sagv_wm0)))
14020 continue;14087 continue;
14021 14088
14022 drm_err(&dev_priv->drm,14089 drm_err(&dev_priv->drm,
@@ -15378,11 +15445,11 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state) · 1355 unmodified lines
15378 15445
15379 intel_set_cdclk_pre_plane_update(state);15446 intel_set_cdclk_pre_plane_update(state);
15380 15447
15381 intel_sagv_pre_plane_update(state);
15382
15383 intel_modeset_verify_disabled(dev_priv, state);15448 intel_modeset_verify_disabled(dev_priv, state);
15384 }15449 }
15385 15450
15451 intel_sagv_pre_plane_update(state);
15452
15386 /* Complete the events for pipes that have now been disabled */15453 /* Complete the events for pipes that have now been disabled */
15387 for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {15454 for_each_new_intel_crtc_in_state(state, crtc, new_crtc_state, i) {
15388 bool modeset = needs_modeset(new_crtc_state);15455 bool modeset = needs_modeset(new_crtc_state);
@@ -15475,11 +15542,10 @@ static void intel_atomic_commit_tail(struct intel_atomic_state *state) · 86 unmodified lines
15475 intel_check_cpu_fifo_underruns(dev_priv);15542 intel_check_cpu_fifo_underruns(dev_priv);
15476 intel_check_pch_fifo_underruns(dev_priv);15543 intel_check_pch_fifo_underruns(dev_priv);
15477 15544
15478 if (state->modeset) {15545 if (state->modeset)
15479 intel_verify_planes(state);15546 intel_verify_planes(state);
15480 15547
15481 intel_sagv_post_plane_update(state);15548 intel_sagv_post_plane_update(state);
15482 }
15483 15549
15484 drm_atomic_helper_commit_hw_done(&state->base);15550 drm_atomic_helper_commit_hw_done(&state->base);
15485 15551
@@ -15815,7 +15881,7 @@ intel_prepare_plane_fb(struct drm_plane *_plane, · 329 unmodified lines
15815 if (new_plane_state->uapi.fence) { /* explicit fencing */15881 if (new_plane_state->uapi.fence) { /* explicit fencing */
15816 ret = i915_sw_fence_await_dma_fence(&state->commit_ready,15882 ret = i915_sw_fence_await_dma_fence(&state->commit_ready,
15817 new_plane_state->uapi.fence,15883 new_plane_state->uapi.fence,
15818 I915_FENCE_TIMEOUT,15884 i915_fence_timeout(dev_priv),
15819 GFP_KERNEL);15885 GFP_KERNEL);
15820 if (ret < 0)15886 if (ret < 0)
15821 return ret;15887 return ret;
@@ -15842,7 +15908,8 @@ intel_prepare_plane_fb(struct drm_plane *_plane, · 20 unmodified lines
15842 15908
15843 ret = i915_sw_fence_await_reservation(&state->commit_ready,15909 ret = i915_sw_fence_await_reservation(&state->commit_ready,
15844 obj->base.resv, NULL,15910 obj->base.resv, NULL,
15845 false, I915_FENCE_TIMEOUT,15911 false,
15912 i915_fence_timeout(dev_priv),
15846 GFP_KERNEL);15913 GFP_KERNEL);
15847 if (ret < 0)15914 if (ret < 0)
15848 goto unpin_fb;15915 goto unpin_fb;
drivers/gpu/drm/i915/display/intel_display_types.h+3−0
@@ -688,11 +688,13 @@ struct skl_plane_wm { · 687 unmodified lines
688 struct skl_wm_level wm[8];688 struct skl_wm_level wm[8];
689 struct skl_wm_level uv_wm[8];689 struct skl_wm_level uv_wm[8];
690 struct skl_wm_level trans_wm;690 struct skl_wm_level trans_wm;
691 struct skl_wm_level sagv_wm0;
691 bool is_planar;692 bool is_planar;
692};693};
693 694
694struct skl_pipe_wm {695struct skl_pipe_wm {
695 struct skl_plane_wm planes[I915_MAX_PLANES];696 struct skl_plane_wm planes[I915_MAX_PLANES];
697 bool use_sagv_wm;
696};698};
697 699
698enum vlv_wm_level {700enum vlv_wm_level {
@@ -1424,6 +1426,7 @@ struct intel_digital_port { · 725 unmodified lines
1424 const struct drm_connector_state *conn_state);1426 const struct drm_connector_state *conn_state);
1425 u32 (*infoframes_enabled)(struct intel_encoder *encoder,1427 u32 (*infoframes_enabled)(struct intel_encoder *encoder,
1426 const struct intel_crtc_state *pipe_config);1428 const struct intel_crtc_state *pipe_config);
1429 bool (*connected)(struct intel_encoder *encoder);
1427};1430};
1428 1431
1429struct intel_dp_mst_encoder {1432struct intel_dp_mst_encoder {
drivers/gpu/drm/i915/display/intel_dp.c+230−364
@@ -2487,8 +2487,8 @@ static void intel_dp_compute_vsc_sdp(struct intel_dp *intel_dp, · 2486 unmodified lines
2487{2487{
2488 struct drm_dp_vsc_sdp *vsc = &crtc_state->infoframes.vsc;2488 struct drm_dp_vsc_sdp *vsc = &crtc_state->infoframes.vsc;
2489 2489
2490 /* When PSR is enabled, VSC SDP is handled by PSR routine */2490 /* When a crtc state has PSR, VSC SDP will be handled by PSR routine */
2491 if (intel_psr_enabled(intel_dp))2491 if (crtc_state->has_psr)
2492 return;2492 return;
2493 2493
2494 if (!intel_dp_needs_vsc_sdp(crtc_state, conn_state))2494 if (!intel_dp_needs_vsc_sdp(crtc_state, conn_state))
@@ -2500,6 +2500,42 @@ static void intel_dp_compute_vsc_sdp(struct intel_dp *intel_dp, · 5 unmodified lines
2500 &crtc_state->infoframes.vsc);2500 &crtc_state->infoframes.vsc);
2501}2501}
2502 2502
2503void intel_dp_compute_psr_vsc_sdp(struct intel_dp *intel_dp,
2504 const struct intel_crtc_state *crtc_state,
2505 const struct drm_connector_state *conn_state,
2506 struct drm_dp_vsc_sdp *vsc)
2507{
2508 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2509
2510 vsc->sdp_type = DP_SDP_VSC;
2511
2512 if (dev_priv->psr.psr2_enabled) {
2513 if (dev_priv->psr.colorimetry_support &&
2514 intel_dp_needs_vsc_sdp(crtc_state, conn_state)) {
2515 /* [PSR2, +Colorimetry] */
2516 intel_dp_compute_vsc_colorimetry(crtc_state, conn_state,
2517 vsc);
2518 } else {
2519 /*
2520 * [PSR2, -Colorimetry]
2521 * Prepare VSC Header for SU as per eDP 1.4 spec, Table 6-11
2522 * 3D stereo + PSR/PSR2 + Y-coordinate.
2523 */
2524 vsc->revision = 0x4;
2525 vsc->length = 0xe;
2526 }
2527 } else {
2528 /*
2529 * [PSR1]
2530 * Prepare VSC Header for SU as per DP 1.4 spec, Table 2-118
2531 * VSC SDP supporting 3D stereo + PSR (applies to eDP v1.3 or
2532 * higher).
2533 */
2534 vsc->revision = 0x2;
2535 vsc->length = 0x8;
2536 }
2537}
2538
2503static void2539static void
2504intel_dp_compute_hdr_metadata_infoframe_sdp(struct intel_dp *intel_dp,2540intel_dp_compute_hdr_metadata_infoframe_sdp(struct intel_dp *intel_dp,
2505 struct intel_crtc_state *crtc_state,2541 struct intel_crtc_state *crtc_state,
@@ -4791,6 +4827,13 @@ static ssize_t intel_dp_vsc_sdp_pack(const struct drm_dp_vsc_sdp *vsc, · 2285 unmodified lines
4791 sdp->sdp_header.HB2 = vsc->revision; /* Revision Number */4827 sdp->sdp_header.HB2 = vsc->revision; /* Revision Number */
4792 sdp->sdp_header.HB3 = vsc->length; /* Number of Valid Data Bytes */4828 sdp->sdp_header.HB3 = vsc->length; /* Number of Valid Data Bytes */
4793 4829
4830 /*
4831 * Only revision 0x5 supports Pixel Encoding/Colorimetry Format as
4832 * per DP 1.4a spec.
4833 */
4834 if (vsc->revision != 0x5)
4835 goto out;
4836
4794 /* VSC SDP Payload for DB16 through DB18 */4837 /* VSC SDP Payload for DB16 through DB18 */
4795 /* Pixel Encoding and Colorimetry Formats */4838 /* Pixel Encoding and Colorimetry Formats */
4796 sdp->db[16] = (vsc->pixelformat & 0xf) << 4; /* DB16[7:4] */4839 sdp->db[16] = (vsc->pixelformat & 0xf) << 4; /* DB16[7:4] */
@@ -4823,6 +4866,7 @@ static ssize_t intel_dp_vsc_sdp_pack(const struct drm_dp_vsc_sdp *vsc, · 26 unmodified lines
4823 /* Content Type */4866 /* Content Type */
4824 sdp->db[18] = vsc->content_type & 0x7;4867 sdp->db[18] = vsc->content_type & 0x7;
4825 4868
4869out:
4826 return length;4870 return length;
4827}4871}
4828 4872
@@ -4935,6 +4979,24 @@ static void intel_write_dp_sdp(struct intel_encoder *encoder, · 106 unmodified lines
4935 intel_dig_port->write_infoframe(encoder, crtc_state, type, &sdp, len);4979 intel_dig_port->write_infoframe(encoder, crtc_state, type, &sdp, len);
4936}4980}
4937 4981
4982void intel_write_dp_vsc_sdp(struct intel_encoder *encoder,
4983 const struct intel_crtc_state *crtc_state,
4984 struct drm_dp_vsc_sdp *vsc)
4985{
4986 struct intel_digital_port *intel_dig_port = enc_to_dig_port(encoder);
4987 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
4988 struct dp_sdp sdp = {};
4989 ssize_t len;
4990
4991 len = intel_dp_vsc_sdp_pack(vsc, &sdp, sizeof(sdp));
4992
4993 if (drm_WARN_ON(&dev_priv->drm, len < 0))
4994 return;
4995
4996 intel_dig_port->write_infoframe(encoder, crtc_state, DP_SDP_VSC,
4997 &sdp, len);
4998}
4999
4938void intel_dp_set_infoframes(struct intel_encoder *encoder,5000void intel_dp_set_infoframes(struct intel_encoder *encoder,
4939 bool enable,5001 bool enable,
4940 const struct intel_crtc_state *crtc_state,5002 const struct intel_crtc_state *crtc_state,
@@ -4971,233 +5033,191 @@ void intel_dp_set_infoframes(struct intel_encoder *encoder, · 30 unmodified lines
4971 intel_write_dp_sdp(encoder, crtc_state, HDMI_PACKET_TYPE_GAMUT_METADATA);5033 intel_write_dp_sdp(encoder, crtc_state, HDMI_PACKET_TYPE_GAMUT_METADATA);
4972}5034}
4973 5035
4974static void5036static int intel_dp_vsc_sdp_unpack(struct drm_dp_vsc_sdp *vsc,
4975intel_dp_setup_vsc_sdp(struct intel_dp *intel_dp,5037 const void *buffer, size_t size)
4976 const struct intel_crtc_state *crtc_state,
4977 const struct drm_connector_state *conn_state)
4978{5038{
4979 struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);5039 const struct dp_sdp *sdp = buffer;
4980 struct dp_sdp vsc_sdp = {};
4981
4982 /* Prepare VSC Header for SU as per DP 1.4a spec, Table 2-119 */
4983 vsc_sdp.sdp_header.HB0 = 0;
4984 vsc_sdp.sdp_header.HB1 = 0x7;
4985 5040
4986 /*5041 if (size < sizeof(struct dp_sdp))
4987 * VSC SDP supporting 3D stereo, PSR2, and Pixel Encoding/5042 return -EINVAL;
4988 * Colorimetry Format indication.
4989 */
4990 vsc_sdp.sdp_header.HB2 = 0x5;
4991 5043
4992 /*5044 memset(vsc, 0, size);
4993 * VSC SDP supporting 3D stereo, + PSR2, + Pixel Encoding/
4994 * Colorimetry Format indication (HB2 = 05h).
4995 */
4996 vsc_sdp.sdp_header.HB3 = 0x13;
4997 5045
4998 /* DP 1.4a spec, Table 2-120 */5046 if (sdp->sdp_header.HB0 != 0)
4999 switch (crtc_state->output_format) {5047 return -EINVAL;
5000 case INTEL_OUTPUT_FORMAT_YCBCR444:
5001 vsc_sdp.db[16] = 0x1 << 4; /* YCbCr 444 : DB16[7:4] = 1h */
5002 break;
5003 case INTEL_OUTPUT_FORMAT_YCBCR420:
5004 vsc_sdp.db[16] = 0x3 << 4; /* YCbCr 420 : DB16[7:4] = 3h */
5005 break;
5006 case INTEL_OUTPUT_FORMAT_RGB:
5007 default:
5008 /* RGB: DB16[7:4] = 0h */
5009 break;
5010 }
5011 5048
5012 switch (conn_state->colorspace) {5049 if (sdp->sdp_header.HB1 != DP_SDP_VSC)
5013 case DRM_MODE_COLORIMETRY_BT709_YCC:5050 return -EINVAL;
5014 vsc_sdp.db[16] |= 0x1;
5015 break;
5016 case DRM_MODE_COLORIMETRY_XVYCC_601:
5017 vsc_sdp.db[16] |= 0x2;
5018 break;
5019 case DRM_MODE_COLORIMETRY_XVYCC_709:
5020 vsc_sdp.db[16] |= 0x3;
5021 break;
5022 case DRM_MODE_COLORIMETRY_SYCC_601:
5023 vsc_sdp.db[16] |= 0x4;
5024 break;
5025 case DRM_MODE_COLORIMETRY_OPYCC_601:
5026 vsc_sdp.db[16] |= 0x5;
5027 break;
5028 case DRM_MODE_COLORIMETRY_BT2020_CYCC:
5029 case DRM_MODE_COLORIMETRY_BT2020_RGB:
5030 vsc_sdp.db[16] |= 0x6;
5031 break;
5032 case DRM_MODE_COLORIMETRY_BT2020_YCC:
5033 vsc_sdp.db[16] |= 0x7;
5034 break;
5035 case DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65:
5036 case DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER:
5037 vsc_sdp.db[16] |= 0x4; /* DCI-P3 (SMPTE RP 431-2) */
5038 break;
5039 default:
5040 /* sRGB (IEC 61966-2-1) / ITU-R BT.601: DB16[0:3] = 0h */
5041 5051
5042 /* RGB->YCBCR color conversion uses the BT.709 color space. */5052 vsc->sdp_type = sdp->sdp_header.HB1;
5043 if (crtc_state->output_format == INTEL_OUTPUT_FORMAT_YCBCR420)5053 vsc->revision = sdp->sdp_header.HB2;
5044 vsc_sdp.db[16] |= 0x1; /* 0x1, ITU-R BT.709 */5054 vsc->length = sdp->sdp_header.HB3;
5045 break;
5046 }
5047 5055
5048 /*5056 if ((sdp->sdp_header.HB2 == 0x2 && sdp->sdp_header.HB3 == 0x8) ||
5049 * For pixel encoding formats YCbCr444, YCbCr422, YCbCr420, and Y Only,5057 (sdp->sdp_header.HB2 == 0x4 && sdp->sdp_header.HB3 == 0xe)) {
5050 * the following Component Bit Depth values are defined:5058 /*
5051 * 001b = 8bpc.5059 * - HB2 = 0x2, HB3 = 0x8
5052 * 010b = 10bpc.5060 * VSC SDP supporting 3D stereo + PSR
5053 * 011b = 12bpc.5061 * - HB2 = 0x4, HB3 = 0xe
5054 * 100b = 16bpc.5062 * VSC SDP supporting 3D stereo + PSR2 with Y-coordinate of
5055 */5063 * first scan line of the SU region (applies to eDP v1.4b
5056 switch (crtc_state->pipe_bpp) {5064 * and higher).
5057 case 24: /* 8bpc */5065 */
5058 vsc_sdp.db[17] = 0x1;5066 return 0;
5059 break;5067 } else if (sdp->sdp_header.HB2 == 0x5 && sdp->sdp_header.HB3 == 0x13) {
5060 case 30: /* 10bpc */5068 /*
5061 vsc_sdp.db[17] = 0x2;5069 * - HB2 = 0x5, HB3 = 0x13
5062 break;5070 * VSC SDP supporting 3D stereo + PSR2 + Pixel Encoding/Colorimetry
5063 case 36: /* 12bpc */5071 * Format.
5064 vsc_sdp.db[17] = 0x3;5072 */
5065 break;5073 vsc->pixelformat = (sdp->db[16] >> 4) & 0xf;
5066 case 48: /* 16bpc */5074 vsc->colorimetry = sdp->db[16] & 0xf;
5067 vsc_sdp.db[17] = 0x4;5075 vsc->dynamic_range = (sdp->db[17] >> 7) & 0x1;
5068 break;
5069 default:
5070 MISSING_CASE(crtc_state->pipe_bpp);
5071 break;
5072 }
5073 5076
5074 /*5077 switch (sdp->db[17] & 0x7) {
5075 * Dynamic Range (Bit 7)5078 case 0x0:
5076 * 0 = VESA range, 1 = CTA range.5079 vsc->bpc = 6;
5077 * all YCbCr are always limited range5080 break;
5078 */5081 case 0x1:
5079 vsc_sdp.db[17] |= 0x80;5082 vsc->bpc = 8;
5083 break;
5084 case 0x2:
5085 vsc->bpc = 10;
5086 break;
5087 case 0x3:
5088 vsc->bpc = 12;
5089 break;
5090 case 0x4:
5091 vsc->bpc = 16;
5092 break;
5093 default:
5094 MISSING_CASE(sdp->db[17] & 0x7);
5095 return -EINVAL;
5096 }
5080 5097
5081 /*5098 vsc->content_type = sdp->db[18] & 0x7;
5082 * Content Type (Bits 2:0)5099 } else {
5083 * 000b = Not defined.5100 return -EINVAL;
5084 * 001b = Graphics.5101 }
5085 * 010b = Photo.
5086 * 011b = Video.
5087 * 100b = Game
5088 * All other values are RESERVED.
5089 * Note: See CTA-861-G for the definition and expected
5090 * processing by a stream sink for the above contect types.
5091 */
5092 vsc_sdp.db[18] = 0;
5093 5102
5094 intel_dig_port->write_infoframe(&intel_dig_port->base,5103 return 0;
5095 crtc_state, DP_SDP_VSC, &vsc_sdp, sizeof(vsc_sdp));
5096}5104}
5097 5105
5098static void5106static int
5099intel_dp_setup_hdr_metadata_infoframe_sdp(struct intel_dp *intel_dp,5107intel_dp_hdr_metadata_infoframe_sdp_unpack(struct hdmi_drm_infoframe *drm_infoframe,
5100 const struct intel_crtc_state *crtc_state,5108 const void *buffer, size_t size)
5101 const struct drm_connector_state *conn_state)
5102{5109{
5103 struct drm_i915_private *i915 = dp_to_i915(intel_dp);
5104 struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
5105 struct dp_sdp infoframe_sdp = {};
5106 struct hdmi_drm_infoframe drm_infoframe = {};
5107 const int infoframe_size = HDMI_INFOFRAME_HEADER_SIZE + HDMI_DRM_INFOFRAME_SIZE;
5108 unsigned char buf[HDMI_INFOFRAME_HEADER_SIZE + HDMI_DRM_INFOFRAME_SIZE];
5109 ssize_t len;
5110 int ret;5110 int ret;
5111 5111
5112 ret = drm_hdmi_infoframe_set_hdr_metadata(&drm_infoframe, conn_state);5112 const struct dp_sdp *sdp = buffer;
5113 if (ret) {
5114 drm_dbg_kms(&i915->drm,
5115 "couldn't set HDR metadata in infoframe\n");
5116 return;
5117 }
5118 5113
5119 len = hdmi_drm_infoframe_pack_only(&drm_infoframe, buf, sizeof(buf));5114 if (size < sizeof(struct dp_sdp))
5120 if (len < 0) {5115 return -EINVAL;
5121 drm_dbg_kms(&i915->drm,
5122 "buffer size is smaller than hdr metadata infoframe\n");
5123 return;
5124 }
5125 5116
5126 if (len != infoframe_size) {5117 if (sdp->sdp_header.HB0 != 0)
5127 drm_dbg_kms(&i915->drm, "wrong static hdr metadata size\n");5118 return -EINVAL;
5128 return;
5129 }
5130 5119
5131 /*5120 if (sdp->sdp_header.HB1 != HDMI_INFOFRAME_TYPE_DRM)
5132 * Set up the infoframe sdp packet for HDR static metadata.5121 return -EINVAL;
5133 * Prepare VSC Header for SU as per DP 1.4a spec,
5134 * Table 2-100 and Table 2-101
5135 */
5136 5122
5137 /* Packet ID, 00h for non-Audio INFOFRAME */
5138 infoframe_sdp.sdp_header.HB0 = 0;
5139 /*
5140 * Packet Type 80h + Non-audio INFOFRAME Type value
5141 * HDMI_INFOFRAME_TYPE_DRM: 0x87,
5142 */
5143 infoframe_sdp.sdp_header.HB1 = drm_infoframe.type;
5144 /*5123 /*
5145 * Least Significant Eight Bits of (Data Byte Count – 1)5124 * Least Significant Eight Bits of (Data Byte Count – 1)
5146 * infoframe_size - 1,5125 * 1Dh (i.e., Data Byte Count = 30 bytes).
5147 */5126 */
5148 infoframe_sdp.sdp_header.HB2 = 0x1D;5127 if (sdp->sdp_header.HB2 != 0x1D)
5128 return -EINVAL;
5129
5130 /* Most Significant Two Bits of (Data Byte Count – 1), Clear to 00b. */
5131 if ((sdp->sdp_header.HB3 & 0x3) != 0)
5132 return -EINVAL;
5133
5149 /* INFOFRAME SDP Version Number */5134 /* INFOFRAME SDP Version Number */
5150 infoframe_sdp.sdp_header.HB3 = (0x13 << 2);5135 if (((sdp->sdp_header.HB3 >> 2) & 0x3f) != 0x13)
5136 return -EINVAL;
5137
5151 /* CTA Header Byte 2 (INFOFRAME Version Number) */5138 /* CTA Header Byte 2 (INFOFRAME Version Number) */
5152 infoframe_sdp.db[0] = drm_infoframe.version;5139 if (sdp->db[0] != 1)
5140 return -EINVAL;
5141
5153 /* CTA Header Byte 3 (Length of INFOFRAME): HDMI_DRM_INFOFRAME_SIZE */5142 /* CTA Header Byte 3 (Length of INFOFRAME): HDMI_DRM_INFOFRAME_SIZE */
5154 infoframe_sdp.db[1] = drm_infoframe.length;5143 if (sdp->db[1] != HDMI_DRM_INFOFRAME_SIZE)
5155 /*5144 return -EINVAL;
5156 * Copy HDMI_DRM_INFOFRAME_SIZE size from a buffer after
5157 * HDMI_INFOFRAME_HEADER_SIZE
5158 */
5159 BUILD_BUG_ON(sizeof(infoframe_sdp.db) < HDMI_DRM_INFOFRAME_SIZE + 2);
5160 memcpy(&infoframe_sdp.db[2], &buf[HDMI_INFOFRAME_HEADER_SIZE],
5161 HDMI_DRM_INFOFRAME_SIZE);
5162 5145
5163 /*5146 ret = hdmi_drm_infoframe_unpack_only(drm_infoframe, &sdp->db[2],
5164 * Size of DP infoframe sdp packet for HDR static metadata is consist of5147 HDMI_DRM_INFOFRAME_SIZE);
5165 * - DP SDP Header(struct dp_sdp_header): 4 bytes5148
5166 * - Two Data Blocks: 2 bytes5149 return ret;
5167 * CTA Header Byte2 (INFOFRAME Version Number)
5168 * CTA Header Byte3 (Length of INFOFRAME)
5169 * - HDMI_DRM_INFOFRAME_SIZE: 26 bytes
5170 *
5171 * Prior to GEN11's GMP register size is identical to DP HDR static metadata
5172 * infoframe size. But GEN11+ has larger than that size, write_infoframe
5173 * will pad rest of the size.
5174 */
5175 intel_dig_port->write_infoframe(&intel_dig_port->base, crtc_state,
5176 HDMI_PACKET_TYPE_GAMUT_METADATA,
5177 &infoframe_sdp,
5178 sizeof(struct dp_sdp_header) + 2 + HDMI_DRM_INFOFRAME_SIZE);
5179}5150}
5180 5151
5181void intel_dp_vsc_enable(struct intel_dp *intel_dp,5152static void intel_read_dp_vsc_sdp(struct intel_encoder *encoder,
5182 const struct intel_crtc_state *crtc_state,5153 struct intel_crtc_state *crtc_state,
5183 const struct drm_connector_state *conn_state)5154 struct drm_dp_vsc_sdp *vsc)
5184{5155{
5185 if (!intel_dp_needs_vsc_sdp(crtc_state, conn_state))5156 struct intel_digital_port *intel_dig_port = enc_to_dig_port(encoder);
5157 struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
5158 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
5159 unsigned int type = DP_SDP_VSC;
5160 struct dp_sdp sdp = {};
5161 int ret;
5162
5163 /* When PSR is enabled, VSC SDP is handled by PSR routine */
5164 if (intel_psr_enabled(intel_dp))
5165 return;
5166
5167 if ((crtc_state->infoframes.enable &
5168 intel_hdmi_infoframe_enable(type)) == 0)
5186 return;5169 return;
5187 5170
5188 intel_dp_setup_vsc_sdp(intel_dp, crtc_state, conn_state);5171 intel_dig_port->read_infoframe(encoder, crtc_state, type, &sdp, sizeof(sdp));
5172
5173 ret = intel_dp_vsc_sdp_unpack(vsc, &sdp, sizeof(sdp));
5174
5175 if (ret)
5176 drm_dbg_kms(&dev_priv->drm, "Failed to unpack DP VSC SDP\n");
5189}5177}
5190 5178
5191void intel_dp_hdr_metadata_enable(struct intel_dp *intel_dp,5179static void intel_read_dp_hdr_metadata_infoframe_sdp(struct intel_encoder *encoder,
5192 const struct intel_crtc_state *crtc_state,5180 struct intel_crtc_state *crtc_state,
5193 const struct drm_connector_state *conn_state)5181 struct hdmi_drm_infoframe *drm_infoframe)
5194{5182{
5195 if (!conn_state->hdr_output_metadata)5183 struct intel_digital_port *intel_dig_port = enc_to_dig_port(encoder);
5184 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
5185 unsigned int type = HDMI_PACKET_TYPE_GAMUT_METADATA;
5186 struct dp_sdp sdp = {};
5187 int ret;
5188
5189 if ((crtc_state->infoframes.enable &
5190 intel_hdmi_infoframe_enable(type)) == 0)
5196 return;5191 return;
5197 5192
5198 intel_dp_setup_hdr_metadata_infoframe_sdp(intel_dp,5193 intel_dig_port->read_infoframe(encoder, crtc_state, type, &sdp,
5199 crtc_state,5194 sizeof(sdp));
5200 conn_state);5195
5196 ret = intel_dp_hdr_metadata_infoframe_sdp_unpack(drm_infoframe, &sdp,
5197 sizeof(sdp));
5198
5199 if (ret)
5200 drm_dbg_kms(&dev_priv->drm,
5201 "Failed to unpack DP HDR Metadata Infoframe SDP\n");
5202}
5203
5204void intel_read_dp_sdp(struct intel_encoder *encoder,
5205 struct intel_crtc_state *crtc_state,
5206 unsigned int type)
5207{
5208 switch (type) {
5209 case DP_SDP_VSC:
5210 intel_read_dp_vsc_sdp(encoder, crtc_state,
5211 &crtc_state->infoframes.vsc);
5212 break;
5213 case HDMI_PACKET_TYPE_GAMUT_METADATA:
5214 intel_read_dp_hdr_metadata_infoframe_sdp(encoder, crtc_state,
5215 &crtc_state->infoframes.drm.drm);
5216 break;
5217 default:
5218 MISSING_CASE(type);
5219 break;
5220 }
5201}5221}
5202 5222
5203static u8 intel_dp_autotest_link_training(struct intel_dp *intel_dp)5223static u8 intel_dp_autotest_link_training(struct intel_dp *intel_dp)
@@ -5998,64 +6018,7 @@ edp_detect(struct intel_dp *intel_dp) · 794 unmodified lines
5998static bool ibx_digital_port_connected(struct intel_encoder *encoder)6018static bool ibx_digital_port_connected(struct intel_encoder *encoder)
5999{6019{
6000 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);6020 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6001 u32 bit;6021 u32 bit = dev_priv->hotplug.pch_hpd[encoder->hpd_pin];
6002
6003 switch (encoder->hpd_pin) {
6004 case HPD_PORT_B:
6005 bit = SDE_PORTB_HOTPLUG;
6006 break;
6007 case HPD_PORT_C:
6008 bit = SDE_PORTC_HOTPLUG;
6009 break;
6010 case HPD_PORT_D:
6011 bit = SDE_PORTD_HOTPLUG;
6012 break;
6013 default:
6014 MISSING_CASE(encoder->hpd_pin);
6015 return false;
6016 }
6017
6018 return intel_de_read(dev_priv, SDEISR) & bit;
6019}
6020
6021static bool cpt_digital_port_connected(struct intel_encoder *encoder)
6022{
6023 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6024 u32 bit;
6025
6026 switch (encoder->hpd_pin) {
6027 case HPD_PORT_B:
6028 bit = SDE_PORTB_HOTPLUG_CPT;
6029 break;
6030 case HPD_PORT_C:
6031 bit = SDE_PORTC_HOTPLUG_CPT;
6032 break;
6033 case HPD_PORT_D:
6034 bit = SDE_PORTD_HOTPLUG_CPT;
6035 break;
6036 default:
6037 MISSING_CASE(encoder->hpd_pin);
6038 return false;
6039 }
6040
6041 return intel_de_read(dev_priv, SDEISR) & bit;
6042}
6043
6044static bool spt_digital_port_connected(struct intel_encoder *encoder)
6045{
6046 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6047 u32 bit;
6048
6049 switch (encoder->hpd_pin) {
6050 case HPD_PORT_A:
6051 bit = SDE_PORTA_HOTPLUG_SPT;
6052 break;
6053 case HPD_PORT_E:
6054 bit = SDE_PORTE_HOTPLUG_SPT;
6055 break;
6056 default:
6057 return cpt_digital_port_connected(encoder);
6058 }
6059 6022
6060 return intel_de_read(dev_priv, SDEISR) & bit;6023 return intel_de_read(dev_priv, SDEISR) & bit;
6061}6024}
@@ -6109,89 +6072,9 @@ static bool gm45_digital_port_connected(struct intel_encoder *encoder) · 47 unmodified lines
6109static bool ilk_digital_port_connected(struct intel_encoder *encoder)6072static bool ilk_digital_port_connected(struct intel_encoder *encoder)
6110{6073{
6111 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);6074 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6075 u32 bit = dev_priv->hotplug.hpd[encoder->hpd_pin];
6112 6076
6113 if (encoder->hpd_pin == HPD_PORT_A)6077 return intel_de_read(dev_priv, DEISR) & bit;
6114 return intel_de_read(dev_priv, DEISR) & DE_DP_A_HOTPLUG;
6115 else
6116 return ibx_digital_port_connected(encoder);
6117}
6118
6119static bool snb_digital_port_connected(struct intel_encoder *encoder)
6120{
6121 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6122
6123 if (encoder->hpd_pin == HPD_PORT_A)
6124 return intel_de_read(dev_priv, DEISR) & DE_DP_A_HOTPLUG;
6125 else
6126 return cpt_digital_port_connected(encoder);
6127}
6128
6129static bool ivb_digital_port_connected(struct intel_encoder *encoder)
6130{
6131 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6132
6133 if (encoder->hpd_pin == HPD_PORT_A)
6134 return intel_de_read(dev_priv, DEISR) & DE_DP_A_HOTPLUG_IVB;
6135 else
6136 return cpt_digital_port_connected(encoder);
6137}
6138
6139static bool bdw_digital_port_connected(struct intel_encoder *encoder)
6140{
6141 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6142
6143 if (encoder->hpd_pin == HPD_PORT_A)
6144 return intel_de_read(dev_priv, GEN8_DE_PORT_ISR) & GEN8_PORT_DP_A_HOTPLUG;
6145 else
6146 return cpt_digital_port_connected(encoder);
6147}
6148
6149static bool bxt_digital_port_connected(struct intel_encoder *encoder)
6150{
6151 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6152 u32 bit;
6153
6154 switch (encoder->hpd_pin) {
6155 case HPD_PORT_A:
6156 bit = BXT_DE_PORT_HP_DDIA;
6157 break;
6158 case HPD_PORT_B:
6159 bit = BXT_DE_PORT_HP_DDIB;
6160 break;
6161 case HPD_PORT_C:
6162 bit = BXT_DE_PORT_HP_DDIC;
6163 break;
6164 default:
6165 MISSING_CASE(encoder->hpd_pin);
6166 return false;
6167 }
6168
6169 return intel_de_read(dev_priv, GEN8_DE_PORT_ISR) & bit;
6170}
6171
6172static bool intel_combo_phy_connected(struct drm_i915_private *dev_priv,
6173 enum phy phy)
6174{
6175 if (HAS_PCH_MCC(dev_priv) && phy == PHY_C)
6176 return intel_de_read(dev_priv, SDEISR) & SDE_TC_HOTPLUG_ICP(PORT_TC1);
6177
6178 return intel_de_read(dev_priv, SDEISR) & SDE_DDI_HOTPLUG_ICP(phy);
6179}
6180
6181static bool icp_digital_port_connected(struct intel_encoder *encoder)
6182{
6183 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6184 struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
6185 enum phy phy = intel_port_to_phy(dev_priv, encoder->port);
6186
6187 if (intel_phy_is_combo(dev_priv, phy))
6188 return intel_combo_phy_connected(dev_priv, phy);
6189 else if (intel_phy_is_tc(dev_priv, phy))
6190 return intel_tc_port_connected(dig_port);
6191 else
6192 MISSING_CASE(encoder->hpd_pin);
6193
6194 return false;
6195}6078}
6196 6079
6197/*6080/*
@@ -6205,44 +6088,15 @@ static bool icp_digital_port_connected(struct intel_encoder *encoder) · 7 unmodified lines
6205 *6088 *
6206 * Return %true if port is connected, %false otherwise.6089 * Return %true if port is connected, %false otherwise.
6207 */6090 */
6208static bool __intel_digital_port_connected(struct intel_encoder *encoder)
6209{
6210 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6211
6212 if (HAS_GMCH(dev_priv)) {
6213 if (IS_GM45(dev_priv))
6214 return gm45_digital_port_connected(encoder);
6215 else
6216 return g4x_digital_port_connected(encoder);
6217 }
6218
6219 if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)
6220 return icp_digital_port_connected(encoder);
6221 else if (INTEL_PCH_TYPE(dev_priv) >= PCH_SPT)
6222 return spt_digital_port_connected(encoder);
6223 else if (IS_GEN9_LP(dev_priv))
6224 return bxt_digital_port_connected(encoder);
6225 else if (IS_GEN(dev_priv, 8))
6226 return bdw_digital_port_connected(encoder);
6227 else if (IS_GEN(dev_priv, 7))
6228 return ivb_digital_port_connected(encoder);
6229 else if (IS_GEN(dev_priv, 6))
6230 return snb_digital_port_connected(encoder);
6231 else if (IS_GEN(dev_priv, 5))
6232 return ilk_digital_port_connected(encoder);
6233
6234 MISSING_CASE(INTEL_GEN(dev_priv));
6235 return false;
6236}
6237
6238bool intel_digital_port_connected(struct intel_encoder *encoder)6091bool intel_digital_port_connected(struct intel_encoder *encoder)
6239{6092{
6240 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);6093 struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
6094 struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
6241 bool is_connected = false;6095 bool is_connected = false;
6242 intel_wakeref_t wakeref;6096 intel_wakeref_t wakeref;
6243 6097
6244 with_intel_display_power(dev_priv, POWER_DOMAIN_DISPLAY_CORE, wakeref)6098 with_intel_display_power(dev_priv, POWER_DOMAIN_DISPLAY_CORE, wakeref)
6245 is_connected = __intel_digital_port_connected(encoder);6099 is_connected = dig_port->connected(encoder);
6246 6100
6247 return is_connected;6101 return is_connected;
6248}6102}
@@ -8522,6 +8376,18 @@ bool intel_dp_init(struct drm_i915_private *dev_priv, · 2273 unmodified lines
8522 8376
8523 intel_dig_port->hpd_pulse = intel_dp_hpd_pulse;8377 intel_dig_port->hpd_pulse = intel_dp_hpd_pulse;
8524 8378
8379 if (HAS_GMCH(dev_priv)) {
8380 if (IS_GM45(dev_priv))
8381 intel_dig_port->connected = gm45_digital_port_connected;
8382 else
8383 intel_dig_port->connected = g4x_digital_port_connected;
8384 } else {
8385 if (port == PORT_A)
8386 intel_dig_port->connected = ilk_digital_port_connected;
8387 else
8388 intel_dig_port->connected = ibx_digital_port_connected;
8389 }
8390
8525 if (port != PORT_A)8391 if (port != PORT_A)
8526 intel_infoframe_init(intel_dig_port);8392 intel_infoframe_init(intel_dig_port);
8527 8393
drivers/gpu/drm/i915/display/intel_dp.h+10−5
@@ -16,6 +16,7 @@ struct drm_connector_state; · 15 unmodified lines
16struct drm_encoder;16struct drm_encoder;
17struct drm_i915_private;17struct drm_i915_private;
18struct drm_modeset_acquire_ctx;18struct drm_modeset_acquire_ctx;
19struct drm_dp_vsc_sdp;
19struct intel_connector;20struct intel_connector;
20struct intel_crtc_state;21struct intel_crtc_state;
21struct intel_digital_port;22struct intel_digital_port;
@@ -108,15 +109,19 @@ int intel_dp_link_required(int pixel_clock, int bpp); · 86 unmodified lines
108int intel_dp_max_data_rate(int max_link_clock, int max_lanes);109int intel_dp_max_data_rate(int max_link_clock, int max_lanes);
109bool intel_dp_needs_vsc_sdp(const struct intel_crtc_state *crtc_state,110bool intel_dp_needs_vsc_sdp(const struct intel_crtc_state *crtc_state,
110 const struct drm_connector_state *conn_state);111 const struct drm_connector_state *conn_state);
111void intel_dp_vsc_enable(struct intel_dp *intel_dp,112void intel_dp_compute_psr_vsc_sdp(struct intel_dp *intel_dp,
112 const struct intel_crtc_state *crtc_state,
113 const struct drm_connector_state *conn_state);
114void intel_dp_hdr_metadata_enable(struct intel_dp *intel_dp,
115 const struct intel_crtc_state *crtc_state,113 const struct intel_crtc_state *crtc_state,
116 const struct drm_connector_state *conn_state);114 const struct drm_connector_state *conn_state,
115 struct drm_dp_vsc_sdp *vsc);
116void intel_write_dp_vsc_sdp(struct intel_encoder *encoder,
117 const struct intel_crtc_state *crtc_state,
118 struct drm_dp_vsc_sdp *vsc);
117void intel_dp_set_infoframes(struct intel_encoder *encoder, bool enable,119void intel_dp_set_infoframes(struct intel_encoder *encoder, bool enable,
118 const struct intel_crtc_state *crtc_state,120 const struct intel_crtc_state *crtc_state,
119 const struct drm_connector_state *conn_state);121 const struct drm_connector_state *conn_state);
122void intel_read_dp_sdp(struct intel_encoder *encoder,
123 struct intel_crtc_state *crtc_state,
124 unsigned int type);
120bool intel_digital_port_connected(struct intel_encoder *encoder);125bool intel_digital_port_connected(struct intel_encoder *encoder);
121void intel_dp_process_phy_request(struct intel_dp *intel_dp);126void intel_dp_process_phy_request(struct intel_dp *intel_dp);
122 127
drivers/gpu/drm/i915/display/intel_dp_mst.c+5−4
@@ -510,10 +510,6 @@ static void intel_mst_enable_dp(struct intel_atomic_state *state, · 509 unmodified lines
510 510
511 intel_ddi_enable_transcoder_func(encoder, pipe_config);511 intel_ddi_enable_transcoder_func(encoder, pipe_config);
512 512
513 intel_enable_pipe(pipe_config);
514
515 intel_crtc_vblank_on(pipe_config);
516
517 drm_dbg_kms(&dev_priv->drm, "active links %d\n",513 drm_dbg_kms(&dev_priv->drm, "active links %d\n",
518 intel_dp->active_mst_links);514 intel_dp->active_mst_links);
519 515
@@ -524,6 +520,11 @@ static void intel_mst_enable_dp(struct intel_atomic_state *state, · 4 unmodified lines
524 drm_dp_check_act_status(&intel_dp->mst_mgr);520 drm_dp_check_act_status(&intel_dp->mst_mgr);
525 521
526 drm_dp_update_payload_part2(&intel_dp->mst_mgr);522 drm_dp_update_payload_part2(&intel_dp->mst_mgr);
523
524 intel_enable_pipe(pipe_config);
525
526 intel_crtc_vblank_on(pipe_config);
527
527 if (pipe_config->has_audio)528 if (pipe_config->has_audio)
528 intel_audio_codec_enable(encoder, pipe_config, conn_state);529 intel_audio_codec_enable(encoder, pipe_config, conn_state);
529}530}
drivers/gpu/drm/i915/display/intel_fbc.c+10−2
@@ -540,6 +540,9 @@ static void __intel_fbc_cleanup_cfb(struct drm_i915_private *dev_priv) · 539 unmodified lines
540{540{
541 struct intel_fbc *fbc = &dev_priv->fbc;541 struct intel_fbc *fbc = &dev_priv->fbc;
542 542
543 if (WARN_ON(intel_fbc_hw_is_active(dev_priv)))
544 return;
545
543 if (!drm_mm_node_allocated(&fbc->compressed_fb))546 if (!drm_mm_node_allocated(&fbc->compressed_fb))
544 return;547 return;
545 548
@@ -564,7 +567,7 @@ void intel_fbc_cleanup_cfb(struct drm_i915_private *dev_priv) · 18 unmodified lines
564}567}
565 568
566static bool stride_is_valid(struct drm_i915_private *dev_priv,569static bool stride_is_valid(struct drm_i915_private *dev_priv,
567 unsigned int stride)570 u64 modifier, unsigned int stride)
568{571{
569 /* This should have been caught earlier. */572 /* This should have been caught earlier. */
570 if (drm_WARN_ON_ONCE(&dev_priv->drm, (stride & (64 - 1)) != 0))573 if (drm_WARN_ON_ONCE(&dev_priv->drm, (stride & (64 - 1)) != 0))
@@ -580,6 +583,11 @@ static bool stride_is_valid(struct drm_i915_private *dev_priv, · 9 unmodified lines
580 if (IS_GEN(dev_priv, 4) && !IS_G4X(dev_priv) && stride < 2048)583 if (IS_GEN(dev_priv, 4) && !IS_G4X(dev_priv) && stride < 2048)
581 return false;584 return false;
582 585
586 /* Display WA #1105: skl,bxt,kbl,cfl,glk */
587 if (IS_GEN(dev_priv, 9) &&
588 modifier == DRM_FORMAT_MOD_LINEAR && stride & 511)
589 return false;
590
583 if (stride > 16384)591 if (stride > 16384)
584 return false;592 return false;
585 593
@@ -810,7 +818,7 @@ static bool intel_fbc_can_activate(struct intel_crtc *crtc) · 224 unmodified lines
810 return false;818 return false;
811 }819 }
812 820
813 if (!stride_is_valid(dev_priv, cache->fb.stride)) {821 if (!stride_is_valid(dev_priv, cache->fb.modifier, cache->fb.stride)) {
814 fbc->no_fbc_reason = "framebuffer stride not supported";822 fbc->no_fbc_reason = "framebuffer stride not supported";
815 return false;823 return false;
816 }824 }
drivers/gpu/drm/i915/display/intel_lspcon.c+1−1
@@ -522,7 +522,7 @@ u32 lspcon_infoframes_enabled(struct intel_encoder *encoder, · 521 unmodified lines
522 const struct intel_crtc_state *pipe_config)522 const struct intel_crtc_state *pipe_config)
523{523{
524 /* FIXME actually read this from the hw */524 /* FIXME actually read this from the hw */
525 return enc_to_intel_lspcon(encoder)->active;525 return 0;
526}526}
527 527
528void lspcon_resume(struct intel_lspcon *lspcon)528void lspcon_resume(struct intel_lspcon *lspcon)
drivers/gpu/drm/i915/display/intel_psr.c+19−39
@@ -30,6 +30,7 @@ · 29 unmodified lines
30#include "intel_display_types.h"30#include "intel_display_types.h"
31#include "intel_psr.h"31#include "intel_psr.h"
32#include "intel_sprite.h"32#include "intel_sprite.h"
33#include "intel_hdmi.h"
33 34
34/**35/**
35 * DOC: Panel Self Refresh (PSR/SRD)36 * DOC: Panel Self Refresh (PSR/SRD)
@@ -357,39 +358,6 @@ void intel_psr_init_dpcd(struct intel_dp *intel_dp) · 321 unmodified lines
357 }358 }
358}359}
359 360
360static void intel_psr_setup_vsc(struct intel_dp *intel_dp,
361 const struct intel_crtc_state *crtc_state)
362{
363 struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
364 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
365 struct dp_sdp psr_vsc;
366
367 if (dev_priv->psr.psr2_enabled) {
368 /* Prepare VSC Header for SU as per EDP 1.4 spec, Table 6.11 */
369 memset(&psr_vsc, 0, sizeof(psr_vsc));
370 psr_vsc.sdp_header.HB0 = 0;
371 psr_vsc.sdp_header.HB1 = 0x7;
372 if (dev_priv->psr.colorimetry_support) {
373 psr_vsc.sdp_header.HB2 = 0x5;
374 psr_vsc.sdp_header.HB3 = 0x13;
375 } else {
376 psr_vsc.sdp_header.HB2 = 0x4;
377 psr_vsc.sdp_header.HB3 = 0xe;
378 }
379 } else {
380 /* Prepare VSC packet as per EDP 1.3 spec, Table 3.10 */
381 memset(&psr_vsc, 0, sizeof(psr_vsc));
382 psr_vsc.sdp_header.HB0 = 0;
383 psr_vsc.sdp_header.HB1 = 0x7;
384 psr_vsc.sdp_header.HB2 = 0x2;
385 psr_vsc.sdp_header.HB3 = 0x8;
386 }
387
388 intel_dig_port->write_infoframe(&intel_dig_port->base,
389 crtc_state,
390 DP_SDP_VSC, &psr_vsc, sizeof(psr_vsc));
391}
392
393static void hsw_psr_setup_aux(struct intel_dp *intel_dp)361static void hsw_psr_setup_aux(struct intel_dp *intel_dp)
394{362{
395 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);363 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
@@ -756,6 +724,8 @@ void intel_psr_compute_config(struct intel_dp *intel_dp, · 360 unmodified lines
756 if (intel_dp != dev_priv->psr.dp)724 if (intel_dp != dev_priv->psr.dp)
757 return;725 return;
758 726
727 if (!psr_global_enabled(dev_priv))
728 return;
759 /*729 /*
760 * HSW spec explicitly says PSR is tied to port A.730 * HSW spec explicitly says PSR is tied to port A.
761 * BDW+ platforms have a instance of PSR registers per transcoder but731 * BDW+ platforms have a instance of PSR registers per transcoder but
@@ -798,6 +768,7 @@ void intel_psr_compute_config(struct intel_dp *intel_dp, · 36 unmodified lines
798 768
799 crtc_state->has_psr = true;769 crtc_state->has_psr = true;
800 crtc_state->has_psr2 = intel_psr2_config_valid(intel_dp, crtc_state);770 crtc_state->has_psr2 = intel_psr2_config_valid(intel_dp, crtc_state);
771 crtc_state->infoframes.enable |= intel_hdmi_infoframe_enable(DP_SDP_VSC);
801}772}
802 773
803static void intel_psr_activate(struct intel_dp *intel_dp)774static void intel_psr_activate(struct intel_dp *intel_dp)
@@ -880,9 +851,12 @@ static void intel_psr_enable_source(struct intel_dp *intel_dp, · 76 unmodified lines
880}851}
881 852
882static void intel_psr_enable_locked(struct drm_i915_private *dev_priv,853static void intel_psr_enable_locked(struct drm_i915_private *dev_priv,
883 const struct intel_crtc_state *crtc_state)854 const struct intel_crtc_state *crtc_state,
855 const struct drm_connector_state *conn_state)
884{856{
885 struct intel_dp *intel_dp = dev_priv->psr.dp;857 struct intel_dp *intel_dp = dev_priv->psr.dp;
858 struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
859 struct intel_encoder *encoder = &intel_dig_port->base;
886 u32 val;860 u32 val;
887 861
888 drm_WARN_ON(&dev_priv->drm, dev_priv->psr.enabled);862 drm_WARN_ON(&dev_priv->drm, dev_priv->psr.enabled);
@@ -921,7 +895,9 @@ static void intel_psr_enable_locked(struct drm_i915_private *dev_priv, · 32 unmodified lines
921 895
922 drm_dbg_kms(&dev_priv->drm, "Enabling PSR%s\n",896 drm_dbg_kms(&dev_priv->drm, "Enabling PSR%s\n",
923 dev_priv->psr.psr2_enabled ? "2" : "1");897 dev_priv->psr.psr2_enabled ? "2" : "1");
924 intel_psr_setup_vsc(intel_dp, crtc_state);898 intel_dp_compute_psr_vsc_sdp(intel_dp, crtc_state, conn_state,
899 &dev_priv->psr.vsc);
900 intel_write_dp_vsc_sdp(encoder, crtc_state, &dev_priv->psr.vsc);
925 intel_psr_enable_sink(intel_dp);901 intel_psr_enable_sink(intel_dp);
926 intel_psr_enable_source(intel_dp, crtc_state);902 intel_psr_enable_source(intel_dp, crtc_state);
927 dev_priv->psr.enabled = true;903 dev_priv->psr.enabled = true;
@@ -933,11 +909,13 @@ static void intel_psr_enable_locked(struct drm_i915_private *dev_priv, · 5 unmodified lines
933 * intel_psr_enable - Enable PSR909 * intel_psr_enable - Enable PSR
934 * @intel_dp: Intel DP910 * @intel_dp: Intel DP
935 * @crtc_state: new CRTC state911 * @crtc_state: new CRTC state
912 * @conn_state: new CONNECTOR state
936 *913 *
937 * This function can only be called after the pipe is fully trained and enabled.914 * This function can only be called after the pipe is fully trained and enabled.
938 */915 */
939void intel_psr_enable(struct intel_dp *intel_dp,916void intel_psr_enable(struct intel_dp *intel_dp,
940 const struct intel_crtc_state *crtc_state)917 const struct intel_crtc_state *crtc_state,
918 const struct drm_connector_state *conn_state)
941{919{
942 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);920 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
943 921
@@ -958,7 +936,7 @@ void intel_psr_enable(struct intel_dp *intel_dp, · 14 unmodified lines
958 goto unlock;936 goto unlock;
959 }937 }
960 938
961 intel_psr_enable_locked(dev_priv, crtc_state);939 intel_psr_enable_locked(dev_priv, crtc_state, conn_state);
962 940
963unlock:941unlock:
964 mutex_unlock(&dev_priv->psr.lock);942 mutex_unlock(&dev_priv->psr.lock);
@@ -1091,13 +1069,15 @@ static void psr_force_hw_tracking_exit(struct drm_i915_private *dev_priv) · 126 unmodified lines
1091 * intel_psr_update - Update PSR state1069 * intel_psr_update - Update PSR state
1092 * @intel_dp: Intel DP1070 * @intel_dp: Intel DP
1093 * @crtc_state: new CRTC state1071 * @crtc_state: new CRTC state
1072 * @conn_state: new CONNECTOR state
1094 *1073 *
1095 * This functions will update PSR states, disabling, enabling or switching PSR1074 * This functions will update PSR states, disabling, enabling or switching PSR
1096 * version when executing fastsets. For full modeset, intel_psr_disable() and1075 * version when executing fastsets. For full modeset, intel_psr_disable() and
1097 * intel_psr_enable() should be called instead.1076 * intel_psr_enable() should be called instead.
1098 */1077 */
1099void intel_psr_update(struct intel_dp *intel_dp,1078void intel_psr_update(struct intel_dp *intel_dp,
1100 const struct intel_crtc_state *crtc_state)1079 const struct intel_crtc_state *crtc_state,
1080 const struct drm_connector_state *conn_state)
1101{1081{
1102 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);1082 struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1103 struct i915_psr *psr = &dev_priv->psr;1083 struct i915_psr *psr = &dev_priv->psr;
@@ -1134,7 +1114,7 @@ void intel_psr_update(struct intel_dp *intel_dp, · 30 unmodified lines
1134 intel_psr_disable_locked(intel_dp);1114 intel_psr_disable_locked(intel_dp);
1135 1115
1136 if (enable)1116 if (enable)
1137 intel_psr_enable_locked(dev_priv, crtc_state);1117 intel_psr_enable_locked(dev_priv, crtc_state, conn_state);
1138 1118
1139unlock:1119unlock:
1140 mutex_unlock(&dev_priv->psr.lock);1120 mutex_unlock(&dev_priv->psr.lock);
drivers/gpu/drm/i915/display/intel_psr.h+4−2
@@ -17,11 +17,13 @@ struct intel_dp; · 16 unmodified lines
17#define CAN_PSR(dev_priv) (HAS_PSR(dev_priv) && dev_priv->psr.sink_support)17#define CAN_PSR(dev_priv) (HAS_PSR(dev_priv) && dev_priv->psr.sink_support)
18void intel_psr_init_dpcd(struct intel_dp *intel_dp);18void intel_psr_init_dpcd(struct intel_dp *intel_dp);
19void intel_psr_enable(struct intel_dp *intel_dp,19void intel_psr_enable(struct intel_dp *intel_dp,
20 const struct intel_crtc_state *crtc_state);20 const struct intel_crtc_state *crtc_state,
21 const struct drm_connector_state *conn_state);
21void intel_psr_disable(struct intel_dp *intel_dp,22void intel_psr_disable(struct intel_dp *intel_dp,
22 const struct intel_crtc_state *old_crtc_state);23 const struct intel_crtc_state *old_crtc_state);
23void intel_psr_update(struct intel_dp *intel_dp,24void intel_psr_update(struct intel_dp *intel_dp,
24 const struct intel_crtc_state *crtc_state);25 const struct intel_crtc_state *crtc_state,
26 const struct drm_connector_state *conn_state);
25int intel_psr_debug_set(struct drm_i915_private *dev_priv, u64 value);27int intel_psr_debug_set(struct drm_i915_private *dev_priv, u64 value);
26void intel_psr_invalidate(struct drm_i915_private *dev_priv,28void intel_psr_invalidate(struct drm_i915_private *dev_priv,
27 unsigned frontbuffer_bits,29 unsigned frontbuffer_bits,
drivers/gpu/drm/i915/display/intel_tc.c+4−3
@@ -238,8 +238,8 @@ static void tc_port_fixup_legacy_flag(struct intel_digital_port *dig_port, · 237 unmodified lines
238static u32 tc_port_live_status_mask(struct intel_digital_port *dig_port)238static u32 tc_port_live_status_mask(struct intel_digital_port *dig_port)
239{239{
240 struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);240 struct drm_i915_private *i915 = to_i915(dig_port->base.base.dev);
241 enum tc_port tc_port = intel_port_to_tc(i915, dig_port->base.port);
242 struct intel_uncore *uncore = &i915->uncore;241 struct intel_uncore *uncore = &i915->uncore;
242 u32 isr_bit = i915->hotplug.pch_hpd[dig_port->base.hpd_pin];
243 u32 mask = 0;243 u32 mask = 0;
244 u32 val;244 u32 val;
245 245
@@ -258,7 +258,7 @@ static u32 tc_port_live_status_mask(struct intel_digital_port *dig_port) · 12 unmodified lines
258 if (val & TC_LIVE_STATE_TC(dig_port->tc_phy_fia_idx))258 if (val & TC_LIVE_STATE_TC(dig_port->tc_phy_fia_idx))
259 mask |= BIT(TC_PORT_DP_ALT);259 mask |= BIT(TC_PORT_DP_ALT);
260 260
261 if (intel_uncore_read(uncore, SDEISR) & SDE_TC_HOTPLUG_ICP(tc_port))261 if (intel_uncore_read(uncore, SDEISR) & isr_bit)
262 mask |= BIT(TC_PORT_LEGACY);262 mask |= BIT(TC_PORT_LEGACY);
263 263
264 /* The sink can be connected only in a single mode. */264 /* The sink can be connected only in a single mode. */
@@ -562,8 +562,9 @@ static bool intel_tc_port_needs_reset(struct intel_digital_port *dig_port) · 297 unmodified lines
562 * connected ports are usable, and avoids exposing to the users objects they562 * connected ports are usable, and avoids exposing to the users objects they
563 * can't really use.563 * can't really use.
564 */564 */
565bool intel_tc_port_connected(struct intel_digital_port *dig_port)565bool intel_tc_port_connected(struct intel_encoder *encoder)
566{566{
567 struct intel_digital_port *dig_port = enc_to_dig_port(encoder);
567 bool is_connected;568 bool is_connected;
568 intel_wakeref_t tc_cold_wref;569 intel_wakeref_t tc_cold_wref;
569 570
drivers/gpu/drm/i915/display/intel_tc.h+2−1
@@ -10,8 +10,9 @@ · 9 unmodified lines
10#include <linux/types.h>10#include <linux/types.h>
11 11
12struct intel_digital_port;12struct intel_digital_port;
13struct intel_encoder;
13 14
14bool intel_tc_port_connected(struct intel_digital_port *dig_port);15bool intel_tc_port_connected(struct intel_encoder *encoder);
15u32 intel_tc_port_get_lane_mask(struct intel_digital_port *dig_port);16u32 intel_tc_port_get_lane_mask(struct intel_digital_port *dig_port);
16u32 intel_tc_port_get_pin_assignment_mask(struct intel_digital_port *dig_port);17u32 intel_tc_port_get_pin_assignment_mask(struct intel_digital_port *dig_port);
17int intel_tc_port_fia_max_lane_count(struct intel_digital_port *dig_port);18int intel_tc_port_fia_max_lane_count(struct intel_digital_port *dig_port);
drivers/gpu/drm/i915/display/intel_vbt_defs.h+2−2
@@ -462,7 +462,7 @@ struct bdb_general_definitions { · 461 unmodified lines
462 * number = (block_size - sizeof(bdb_general_definitions))/462 * number = (block_size - sizeof(bdb_general_definitions))/
463 * defs->child_dev_size;463 * defs->child_dev_size;
464 */464 */
465 u8 devices[0];465 u8 devices[];
466} __packed;466} __packed;
467 467
468/*468/*
@@ -839,7 +839,7 @@ struct bdb_mipi_config { · 370 unmodified lines
839 839
840struct bdb_mipi_sequence {840struct bdb_mipi_sequence {
841 u8 version;841 u8 version;
842 u8 data[0]; /* up to 6 variable length blocks */842 u8 data[]; /* up to 6 variable length blocks */
843} __packed;843} __packed;
844 844
845/*845/*
drivers/gpu/drm/i915/gem/i915_gem_clflush.c+1−1
@@ -108,7 +108,7 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj, · 107 unmodified lines
108 if (clflush) {108 if (clflush) {
109 i915_sw_fence_await_reservation(&clflush->base.chain,109 i915_sw_fence_await_reservation(&clflush->base.chain,
110 obj->base.resv, NULL, true,110 obj->base.resv, NULL, true,
111 I915_FENCE_TIMEOUT,111 i915_fence_timeout(to_i915(obj->base.dev)),
112 I915_FENCE_GFP);112 I915_FENCE_GFP);
113 dma_resv_add_excl_fence(obj->base.resv, &clflush->base.dma);113 dma_resv_add_excl_fence(obj->base.resv, &clflush->base.dma);
114 dma_fence_work_commit(&clflush->base);114 dma_fence_work_commit(&clflush->base);
drivers/gpu/drm/i915/gem/i915_gem_client_blt.c+1−3
@@ -6,7 +6,6 @@ · 5 unmodified lines
6#include "i915_drv.h"6#include "i915_drv.h"
7#include "gt/intel_context.h"7#include "gt/intel_context.h"
8#include "gt/intel_engine_pm.h"8#include "gt/intel_engine_pm.h"
9#include "gt/intel_engine_pool.h"
10#include "i915_gem_client_blt.h"9#include "i915_gem_client_blt.h"
11#include "i915_gem_object_blt.h"10#include "i915_gem_object_blt.h"
12 11
@@ -289,8 +288,7 @@ int i915_gem_schedule_fill_pages_blt(struct drm_i915_gem_object *obj, · 276 unmodified lines
289 288
290 i915_gem_object_lock(obj);289 i915_gem_object_lock(obj);
291 err = i915_sw_fence_await_reservation(&work->wait,290 err = i915_sw_fence_await_reservation(&work->wait,
292 obj->base.resv, NULL,291 obj->base.resv, NULL, true, 0,
293 true, I915_FENCE_TIMEOUT,
294 I915_FENCE_GFP);292 I915_FENCE_GFP);
295 if (err < 0) {293 if (err < 0) {
296 dma_fence_set_error(&work->dma, err);294 dma_fence_set_error(&work->dma, err);
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c+313−165
@@ -15,8 +15,8 @@ · 14 unmodified lines
15 15
16#include "gem/i915_gem_ioctls.h"16#include "gem/i915_gem_ioctls.h"
17#include "gt/intel_context.h"17#include "gt/intel_context.h"
18#include "gt/intel_engine_pool.h"
19#include "gt/intel_gt.h"18#include "gt/intel_gt.h"
19#include "gt/intel_gt_buffer_pool.h"
20#include "gt/intel_gt_pm.h"20#include "gt/intel_gt_pm.h"
21#include "gt/intel_ring.h"21#include "gt/intel_ring.h"
22 22
@@ -268,7 +268,9 @@ struct i915_execbuffer { · 245 unmodified lines
268 bool has_fence : 1;268 bool has_fence : 1;
269 bool needs_unfenced : 1;269 bool needs_unfenced : 1;
270 270
271 struct i915_vma *target;
271 struct i915_request *rq;272 struct i915_request *rq;
273 struct i915_vma *rq_vma;
272 u32 *rq_cmd;274 u32 *rq_cmd;
273 unsigned int rq_size;275 unsigned int rq_size;
274 } reloc_cache;276 } reloc_cache;
@@ -953,7 +955,7 @@ static void reloc_cache_init(struct reloc_cache *cache, · 678 unmodified lines
953 cache->needs_unfenced = INTEL_INFO(i915)->unfenced_needs_alignment;955 cache->needs_unfenced = INTEL_INFO(i915)->unfenced_needs_alignment;
954 cache->node.flags = 0;956 cache->node.flags = 0;
955 cache->rq = NULL;957 cache->rq = NULL;
956 cache->rq_size = 0;958 cache->target = NULL;
957}959}
958 960
959static inline void *unmask_page(unsigned long p)961static inline void *unmask_page(unsigned long p)
@@ -975,29 +977,122 @@ static inline struct i915_ggtt *cache_to_ggtt(struct reloc_cache *cache) · 15 unmodified lines
975 return &i915->ggtt;977 return &i915->ggtt;
976}978}
977 979
978static void reloc_gpu_flush(struct reloc_cache *cache)980#define RELOC_TAIL 4
981
982static int reloc_gpu_chain(struct reloc_cache *cache)
979{983{
980 struct drm_i915_gem_object *obj = cache->rq->batch->obj;984 struct intel_gt_buffer_pool_node *pool;
985 struct i915_request *rq = cache->rq;
986 struct i915_vma *batch;
987 u32 *cmd;
988 int err;
981 989
982 GEM_BUG_ON(cache->rq_size >= obj->base.size / sizeof(u32));990 pool = intel_gt_get_buffer_pool(rq->engine->gt, PAGE_SIZE);
983 cache->rq_cmd[cache->rq_size] = MI_BATCH_BUFFER_END;991 if (IS_ERR(pool))
992 return PTR_ERR(pool);
984 993
985 __i915_gem_object_flush_map(obj, 0, sizeof(u32) * (cache->rq_size + 1));994 batch = i915_vma_instance(pool->obj, rq->context->vm, NULL);
986 i915_gem_object_unpin_map(obj);995 if (IS_ERR(batch)) {
996 err = PTR_ERR(batch);
997 goto out_pool;
998 }
987 999
988 intel_gt_chipset_flush(cache->rq->engine->gt);1000 err = i915_vma_pin(batch, 0, 0, PIN_USER | PIN_NONBLOCK);
1001 if (err)
1002 goto out_pool;
989 1003
990 i915_request_add(cache->rq);1004 GEM_BUG_ON(cache->rq_size + RELOC_TAIL > PAGE_SIZE / sizeof(u32));
991 cache->rq = NULL;1005 cmd = cache->rq_cmd + cache->rq_size;
1006 *cmd++ = MI_ARB_CHECK;
1007 if (cache->gen >= 8)
1008 *cmd++ = MI_BATCH_BUFFER_START_GEN8;
1009 else if (cache->gen >= 6)
1010 *cmd++ = MI_BATCH_BUFFER_START;
1011 else
1012 *cmd++ = MI_BATCH_BUFFER_START | MI_BATCH_GTT;
1013 *cmd++ = lower_32_bits(batch->node.start);
1014 *cmd++ = upper_32_bits(batch->node.start); /* Always 0 for gen<8 */
1015 i915_gem_object_flush_map(cache->rq_vma->obj);
1016 i915_gem_object_unpin_map(cache->rq_vma->obj);
1017 cache->rq_vma = NULL;
1018
1019 err = intel_gt_buffer_pool_mark_active(pool, rq);
1020 if (err == 0) {
1021 i915_vma_lock(batch);
1022 err = i915_request_await_object(rq, batch->obj, false);
1023 if (err == 0)
1024 err = i915_vma_move_to_active(batch, rq, 0);
1025 i915_vma_unlock(batch);
1026 }
1027 i915_vma_unpin(batch);
1028 if (err)
1029 goto out_pool;
1030
1031 cmd = i915_gem_object_pin_map(batch->obj,
1032 cache->has_llc ?
1033 I915_MAP_FORCE_WB :
1034 I915_MAP_FORCE_WC);
1035 if (IS_ERR(cmd)) {
1036 err = PTR_ERR(cmd);
1037 goto out_pool;
1038 }
1039
1040 /* Return with batch mapping (cmd) still pinned */
1041 cache->rq_cmd = cmd;
1042 cache->rq_size = 0;
1043 cache->rq_vma = batch;
1044
1045out_pool:
1046 intel_gt_buffer_pool_put(pool);
1047 return err;
1048}
1049
1050static unsigned int reloc_bb_flags(const struct reloc_cache *cache)
1051{
1052 return cache->gen > 5 ? 0 : I915_DISPATCH_SECURE;
1053}
1054
1055static int reloc_gpu_flush(struct reloc_cache *cache)
1056{
1057 struct i915_request *rq;
1058 int err;
1059
1060 rq = fetch_and_zero(&cache->rq);
1061 if (!rq)
1062 return 0;
1063
1064 if (cache->rq_vma) {
1065 struct drm_i915_gem_object *obj = cache->rq_vma->obj;
1066
1067 GEM_BUG_ON(cache->rq_size >= obj->base.size / sizeof(u32));
1068 cache->rq_cmd[cache->rq_size++] = MI_BATCH_BUFFER_END;
1069
1070 __i915_gem_object_flush_map(obj,
1071 0, sizeof(u32) * cache->rq_size);
1072 i915_gem_object_unpin_map(obj);
1073 }
1074
1075 err = 0;
1076 if (rq->engine->emit_init_breadcrumb)
1077 err = rq->engine->emit_init_breadcrumb(rq);
1078 if (!err)
1079 err = rq->engine->emit_bb_start(rq,
1080 rq->batch->node.start,
1081 PAGE_SIZE,
1082 reloc_bb_flags(cache));
1083 if (err)
1084 i915_request_set_error_once(rq, err);
1085
1086 intel_gt_chipset_flush(rq->engine->gt);
1087 i915_request_add(rq);
1088
1089 return err;
992}1090}
993 1091
994static void reloc_cache_reset(struct reloc_cache *cache)1092static void reloc_cache_reset(struct reloc_cache *cache)
995{1093{
996 void *vaddr;1094 void *vaddr;
997 1095
998 if (cache->rq)
999 reloc_gpu_flush(cache);
1000
1001 if (!cache->vaddr)1096 if (!cache->vaddr)
1002 return;1097 return;
1003 1098
@@ -1190,17 +1285,17 @@ static int reloc_move_to_gpu(struct i915_request *rq, struct i915_vma *vma) · 186 unmodified lines
1190}1285}
1191 1286
1192static int __reloc_gpu_alloc(struct i915_execbuffer *eb,1287static int __reloc_gpu_alloc(struct i915_execbuffer *eb,
1193 struct i915_vma *vma,1288 struct intel_engine_cs *engine,
1194 unsigned int len)1289 unsigned int len)
1195{1290{
1196 struct reloc_cache *cache = &eb->reloc_cache;1291 struct reloc_cache *cache = &eb->reloc_cache;
1197 struct intel_engine_pool_node *pool;1292 struct intel_gt_buffer_pool_node *pool;
1198 struct i915_request *rq;1293 struct i915_request *rq;
1199 struct i915_vma *batch;1294 struct i915_vma *batch;
1200 u32 *cmd;1295 u32 *cmd;
1201 int err;1296 int err;
1202 1297
1203 pool = intel_engine_get_pool(eb->engine, PAGE_SIZE);1298 pool = intel_gt_get_buffer_pool(engine->gt, PAGE_SIZE);
1204 if (IS_ERR(pool))1299 if (IS_ERR(pool))
1205 return PTR_ERR(pool);1300 return PTR_ERR(pool);
1206 1301
@@ -1213,7 +1308,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, · 6 unmodified lines
1213 goto out_pool;1308 goto out_pool;
1214 }1309 }
1215 1310
1216 batch = i915_vma_instance(pool->obj, vma->vm, NULL);1311 batch = i915_vma_instance(pool->obj, eb->context->vm, NULL);
1217 if (IS_ERR(batch)) {1312 if (IS_ERR(batch)) {
1218 err = PTR_ERR(batch);1313 err = PTR_ERR(batch);
1219 goto err_unmap;1314 goto err_unmap;
@@ -1223,26 +1318,32 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, · 3 unmodified lines
1223 if (err)1318 if (err)
1224 goto err_unmap;1319 goto err_unmap;
1225 1320
1226 rq = i915_request_create(eb->context);1321 if (engine == eb->context->engine) {
1322 rq = i915_request_create(eb->context);
1323 } else {
1324 struct intel_context *ce;
1325
1326 ce = intel_context_create(engine);
1327 if (IS_ERR(ce)) {
1328 err = PTR_ERR(ce);
1329 goto err_unpin;
1330 }
1331
1332 i915_vm_put(ce->vm);
1333 ce->vm = i915_vm_get(eb->context->vm);
1334
1335 rq = intel_context_create_request(ce);
1336 intel_context_put(ce);
1337 }
1227 if (IS_ERR(rq)) {1338 if (IS_ERR(rq)) {
1228 err = PTR_ERR(rq);1339 err = PTR_ERR(rq);
1229 goto err_unpin;1340 goto err_unpin;
1230 }1341 }
1231 1342
1232 err = intel_engine_pool_mark_active(pool, rq);1343 err = intel_gt_buffer_pool_mark_active(pool, rq);
1233 if (err)
1234 goto err_request;
1235
1236 err = reloc_move_to_gpu(rq, vma);
1237 if (err)1344 if (err)
1238 goto err_request;1345 goto err_request;
1239 1346
1240 err = eb->engine->emit_bb_start(rq,
1241 batch->node.start, PAGE_SIZE,
1242 cache->gen > 5 ? 0 : I915_DISPATCH_SECURE);
1243 if (err)
1244 goto skip_request;
1245
1246 i915_vma_lock(batch);1347 i915_vma_lock(batch);
1247 err = i915_request_await_object(rq, batch->obj, false);1348 err = i915_request_await_object(rq, batch->obj, false);
1248 if (err == 0)1349 if (err == 0)
@@ -1257,6 +1358,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb, · 8 unmodified lines
1257 cache->rq = rq;1358 cache->rq = rq;
1258 cache->rq_cmd = cmd;1359 cache->rq_cmd = cmd;
1259 cache->rq_size = 0;1360 cache->rq_size = 0;
1361 cache->rq_vma = batch;
1260 1362
1261 /* Return with batch mapping (cmd) still pinned */1363 /* Return with batch mapping (cmd) still pinned */
1262 goto out_pool;1364 goto out_pool;
@@ -1270,31 +1372,57 @@ err_unpin: · 7 unmodified lines
1270err_unmap:1372err_unmap:
1271 i915_gem_object_unpin_map(pool->obj);1373 i915_gem_object_unpin_map(pool->obj);
1272out_pool:1374out_pool:
1273 intel_engine_pool_put(pool);1375 intel_gt_buffer_pool_put(pool);
1274 return err;1376 return err;
1275}1377}
1276 1378
1379static bool reloc_can_use_engine(const struct intel_engine_cs *engine)
1380{
1381 return engine->class != VIDEO_DECODE_CLASS || !IS_GEN(engine->i915, 6);
1382}
1383
1277static u32 *reloc_gpu(struct i915_execbuffer *eb,1384static u32 *reloc_gpu(struct i915_execbuffer *eb,
1278 struct i915_vma *vma,1385 struct i915_vma *vma,
1279 unsigned int len)1386 unsigned int len)
1280{1387{
1281 struct reloc_cache *cache = &eb->reloc_cache;1388 struct reloc_cache *cache = &eb->reloc_cache;
1282 u32 *cmd;1389 u32 *cmd;
1283 1390 int err;
1284 if (cache->rq_size > PAGE_SIZE/sizeof(u32) - (len + 1))
1285 reloc_gpu_flush(cache);
1286 1391
1287 if (unlikely(!cache->rq)) {1392 if (unlikely(!cache->rq)) {
1288 int err;1393 struct intel_engine_cs *engine = eb->engine;
1289 1394
1290 if (!intel_engine_can_store_dword(eb->engine))1395 if (!reloc_can_use_engine(engine)) {
1291 return ERR_PTR(-ENODEV);1396 engine = engine->gt->engine_class[COPY_ENGINE_CLASS][0];
1397 if (!engine)
1398 return ERR_PTR(-ENODEV);
1399 }
1292 1400
1293 err = __reloc_gpu_alloc(eb, vma, len);1401 err = __reloc_gpu_alloc(eb, engine, len);
1294 if (unlikely(err))1402 if (unlikely(err))
1295 return ERR_PTR(err);1403 return ERR_PTR(err);
1296 }1404 }
1297 1405
1406 if (vma != cache->target) {
1407 err = reloc_move_to_gpu(cache->rq, vma);
1408 if (unlikely(err)) {
1409 i915_request_set_error_once(cache->rq, err);
1410 return ERR_PTR(err);
1411 }
1412
1413 cache->target = vma;
1414 }
1415
1416 if (unlikely(cache->rq_size + len >
1417 PAGE_SIZE / sizeof(u32) - RELOC_TAIL)) {
1418 err = reloc_gpu_chain(cache);
1419 if (unlikely(err)) {
1420 i915_request_set_error_once(cache->rq, err);
1421 return ERR_PTR(err);
1422 }
1423 }
1424
1425 GEM_BUG_ON(cache->rq_size + len >= PAGE_SIZE / sizeof(u32));
1298 cmd = cache->rq_cmd + cache->rq_size;1426 cmd = cache->rq_cmd + cache->rq_size;
1299 cache->rq_size += len;1427 cache->rq_size += len;
1300 1428
@@ -1312,91 +1440,138 @@ static inline bool use_reloc_gpu(struct i915_vma *vma) · 11 unmodified lines
1312 return !dma_resv_test_signaled_rcu(vma->resv, true);1440 return !dma_resv_test_signaled_rcu(vma->resv, true);
1313}1441}
1314 1442
1315static u641443static unsigned long vma_phys_addr(struct i915_vma *vma, u32 offset)
1316relocate_entry(struct i915_vma *vma,
1317 const struct drm_i915_gem_relocation_entry *reloc,
1318 struct i915_execbuffer *eb,
1319 const struct i915_vma *target)
1320{1444{
1321 u64 offset = reloc->offset;1445 struct page *page;
1322 u64 target_offset = relocation_target(reloc, target);1446 unsigned long addr;
1323 bool wide = eb->reloc_cache.use_64bit_reloc;
1324 void *vaddr;
1325 1447
1326 if (!eb->reloc_cache.vaddr && use_reloc_gpu(vma)) {1448 GEM_BUG_ON(vma->pages != vma->obj->mm.pages);
1327 const unsigned int gen = eb->reloc_cache.gen;
1328 unsigned int len;
1329 u32 *batch;
1330 u64 addr;
1331 1449
1332 if (wide)1450 page = i915_gem_object_get_page(vma->obj, offset >> PAGE_SHIFT);
1333 len = offset & 7 ? 8 : 5;1451 addr = PFN_PHYS(page_to_pfn(page));
1334 else if (gen >= 4)1452 GEM_BUG_ON(overflows_type(addr, u32)); /* expected dma32 */
1335 len = 4;
1336 else
1337 len = 3;
1338 1453
1339 batch = reloc_gpu(eb, vma, len);1454 return addr + offset_in_page(offset);
1340 if (IS_ERR(batch))1455}
1341 goto repeat;1456
1457static bool __reloc_entry_gpu(struct i915_execbuffer *eb,
1458 struct i915_vma *vma,
1459 u64 offset,
1460 u64 target_addr)
1461{
1462 const unsigned int gen = eb->reloc_cache.gen;
1463 unsigned int len;
1464 u32 *batch;
1465 u64 addr;
1466
1467 if (gen >= 8)
1468 len = offset & 7 ? 8 : 5;
1469 else if (gen >= 4)
1470 len = 4;
1471 else
1472 len = 3;
1473
1474 batch = reloc_gpu(eb, vma, len);
1475 if (IS_ERR(batch))
1476 return false;
1477
1478 addr = gen8_canonical_addr(vma->node.start + offset);
1479 if (gen >= 8) {
1480 if (offset & 7) {
1481 *batch++ = MI_STORE_DWORD_IMM_GEN4;
1482 *batch++ = lower_32_bits(addr);
1483 *batch++ = upper_32_bits(addr);
1484 *batch++ = lower_32_bits(target_addr);
1485
1486 addr = gen8_canonical_addr(addr + 4);
1342 1487
1343 addr = gen8_canonical_addr(vma->node.start + offset);
1344 if (wide) {
1345 if (offset & 7) {
1346 *batch++ = MI_STORE_DWORD_IMM_GEN4;
1347 *batch++ = lower_32_bits(addr);
1348 *batch++ = upper_32_bits(addr);
1349 *batch++ = lower_32_bits(target_offset);
1350
1351 addr = gen8_canonical_addr(addr + 4);
1352
1353 *batch++ = MI_STORE_DWORD_IMM_GEN4;
1354 *batch++ = lower_32_bits(addr);
1355 *batch++ = upper_32_bits(addr);
1356 *batch++ = upper_32_bits(target_offset);
1357 } else {
1358 *batch++ = (MI_STORE_DWORD_IMM_GEN4 | (1 << 21)) + 1;
1359 *batch++ = lower_32_bits(addr);
1360 *batch++ = upper_32_bits(addr);
1361 *batch++ = lower_32_bits(target_offset);
1362 *batch++ = upper_32_bits(target_offset);
1363 }
1364 } else if (gen >= 6) {
1365 *batch++ = MI_STORE_DWORD_IMM_GEN4;1488 *batch++ = MI_STORE_DWORD_IMM_GEN4;
1366 *batch++ = 0;1489 *batch++ = lower_32_bits(addr);
1367 *batch++ = addr;1490 *batch++ = upper_32_bits(addr);
1368 *batch++ = target_offset;1491 *batch++ = upper_32_bits(target_addr);
1369 } else if (gen >= 4) {
1370 *batch++ = MI_STORE_DWORD_IMM_GEN4 | MI_USE_GGTT;
1371 *batch++ = 0;
1372 *batch++ = addr;
1373 *batch++ = target_offset;
1374 } else {1492 } else {
1375 *batch++ = MI_STORE_DWORD_IMM | MI_MEM_VIRTUAL;1493 *batch++ = (MI_STORE_DWORD_IMM_GEN4 | (1 << 21)) + 1;
1376 *batch++ = addr;1494 *batch++ = lower_32_bits(addr);
1377 *batch++ = target_offset;1495 *batch++ = upper_32_bits(addr);
1496 *batch++ = lower_32_bits(target_addr);
1497 *batch++ = upper_32_bits(target_addr);
1378 }1498 }
1379 1499 } else if (gen >= 6) {
1380 goto out;1500 *batch++ = MI_STORE_DWORD_IMM_GEN4;
1501 *batch++ = 0;
1502 *batch++ = addr;
1503 *batch++ = target_addr;
1504 } else if (IS_I965G(eb->i915)) {
1505 *batch++ = MI_STORE_DWORD_IMM_GEN4;
1506 *batch++ = 0;
1507 *batch++ = vma_phys_addr(vma, offset);
1508 *batch++ = target_addr;
1509 } else if (gen >= 4) {
1510 *batch++ = MI_STORE_DWORD_IMM_GEN4 | MI_USE_GGTT;
1511 *batch++ = 0;
1512 *batch++ = addr;
1513 *batch++ = target_addr;
1514 } else if (gen >= 3 &&
1515 !(IS_I915G(eb->i915) || IS_I915GM(eb->i915))) {
1516 *batch++ = MI_STORE_DWORD_IMM | MI_MEM_VIRTUAL;
1517 *batch++ = addr;
1518 *batch++ = target_addr;
1519 } else {
1520 *batch++ = MI_STORE_DWORD_IMM;
1521 *batch++ = vma_phys_addr(vma, offset);
1522 *batch++ = target_addr;
1381 }1523 }
1382 1524
1525 return true;
1526}
1527
1528static bool reloc_entry_gpu(struct i915_execbuffer *eb,
1529 struct i915_vma *vma,
1530 u64 offset,
1531 u64 target_addr)
1532{
1533 if (eb->reloc_cache.vaddr)
1534 return false;
1535
1536 if (!use_reloc_gpu(vma))
1537 return false;
1538
1539 return __reloc_entry_gpu(eb, vma, offset, target_addr);
1540}
1541
1542static u64
1543relocate_entry(struct i915_vma *vma,
1544 const struct drm_i915_gem_relocation_entry *reloc,
1545 struct i915_execbuffer *eb,
1546 const struct i915_vma *target)
1547{
1548 u64 target_addr = relocation_target(reloc, target);
1549 u64 offset = reloc->offset;
1550
1551 if (!reloc_entry_gpu(eb, vma, offset, target_addr)) {
1552 bool wide = eb->reloc_cache.use_64bit_reloc;
1553 void *vaddr;
1554
1383repeat:1555repeat:
1384 vaddr = reloc_vaddr(vma->obj, &eb->reloc_cache, offset >> PAGE_SHIFT);1556 vaddr = reloc_vaddr(vma->obj,
1385 if (IS_ERR(vaddr))1557 &eb->reloc_cache,
1386 return PTR_ERR(vaddr);1558 offset >> PAGE_SHIFT);
1559 if (IS_ERR(vaddr))
1560 return PTR_ERR(vaddr);
1387 1561
1388 clflush_write32(vaddr + offset_in_page(offset),1562 GEM_BUG_ON(!IS_ALIGNED(offset, sizeof(u32)));
1389 lower_32_bits(target_offset),1563 clflush_write32(vaddr + offset_in_page(offset),
1390 eb->reloc_cache.vaddr);1564 lower_32_bits(target_addr),
1565 eb->reloc_cache.vaddr);
1391 1566
1392 if (wide) {1567 if (wide) {
1393 offset += sizeof(u32);1568 offset += sizeof(u32);
1394 target_offset >>= 32;1569 target_addr >>= 32;
1395 wide = false;1570 wide = false;
1396 goto repeat;1571 goto repeat;
1572 }
1397 }1573 }
1398 1574
1399out:
1400 return target->node.start | UPDATE;1575 return target->node.start | UPDATE;
1401}1576}
1402 1577
@@ -1596,15 +1771,20 @@ static int eb_relocate(struct i915_execbuffer *eb) · 193 unmodified lines
1596 /* The objects are in their final locations, apply the relocations. */1771 /* The objects are in their final locations, apply the relocations. */
1597 if (eb->args->flags & __EXEC_HAS_RELOC) {1772 if (eb->args->flags & __EXEC_HAS_RELOC) {
1598 struct eb_vma *ev;1773 struct eb_vma *ev;
1774 int flush;
1599 1775
1600 list_for_each_entry(ev, &eb->relocs, reloc_link) {1776 list_for_each_entry(ev, &eb->relocs, reloc_link) {
1601 err = eb_relocate_vma(eb, ev);1777 err = eb_relocate_vma(eb, ev);
1602 if (err)1778 if (err)
1603 return err;1779 break;
1604 }1780 }
1781
1782 flush = reloc_gpu_flush(&eb->reloc_cache);
1783 if (!err)
1784 err = flush;
1605 }1785 }
1606 1786
1607 return 0;1787 return err;
1608}1788}
1609 1789
1610static int eb_move_to_gpu(struct i915_execbuffer *eb)1790static int eb_move_to_gpu(struct i915_execbuffer *eb)
@@ -1887,7 +2067,7 @@ err_free: · 276 unmodified lines
1887static int eb_parse(struct i915_execbuffer *eb)2067static int eb_parse(struct i915_execbuffer *eb)
1888{2068{
1889 struct drm_i915_private *i915 = eb->i915;2069 struct drm_i915_private *i915 = eb->i915;
1890 struct intel_engine_pool_node *pool;2070 struct intel_gt_buffer_pool_node *pool;
1891 struct i915_vma *shadow, *trampoline;2071 struct i915_vma *shadow, *trampoline;
1892 unsigned int len;2072 unsigned int len;
1893 int err;2073 int err;
@@ -1910,7 +2090,7 @@ static int eb_parse(struct i915_execbuffer *eb) · 16 unmodified lines
1910 len += I915_CMD_PARSER_TRAMPOLINE_SIZE;2090 len += I915_CMD_PARSER_TRAMPOLINE_SIZE;
1911 }2091 }
1912 2092
1913 pool = intel_engine_get_pool(eb->engine, len);2093 pool = intel_gt_get_buffer_pool(eb->engine->gt, len);
1914 if (IS_ERR(pool))2094 if (IS_ERR(pool))
1915 return PTR_ERR(pool);2095 return PTR_ERR(pool);
1916 2096
@@ -1958,7 +2138,7 @@ err_trampoline: · 41 unmodified lines
1958err_shadow:2138err_shadow:
1959 i915_vma_unpin(shadow);2139 i915_vma_unpin(shadow);
1960err:2140err:
1961 intel_engine_pool_put(pool);2141 intel_gt_buffer_pool_put(pool);
1962 return err;2142 return err;
1963}2143}
1964 2144
@@ -2402,30 +2582,6 @@ static void eb_request_add(struct i915_execbuffer *eb) · 437 unmodified lines
2402 /* Check that the context wasn't destroyed before submission */2582 /* Check that the context wasn't destroyed before submission */
2403 if (likely(!intel_context_is_closed(eb->context))) {2583 if (likely(!intel_context_is_closed(eb->context))) {
2404 attr = eb->gem_context->sched;2584 attr = eb->gem_context->sched;
2405
2406 /*
2407 * Boost actual workloads past semaphores!
2408 *
2409 * With semaphores we spin on one engine waiting for another,
2410 * simply to reduce the latency of starting our work when
2411 * the signaler completes. However, if there is any other
2412 * work that we could be doing on this engine instead, that
2413 * is better utilisation and will reduce the overall duration
2414 * of the current work. To avoid PI boosting a semaphore
2415 * far in the distance past over useful work, we keep a history
2416 * of any semaphore use along our dependency chain.
2417 */
2418 if (!(rq->sched.flags & I915_SCHED_HAS_SEMAPHORE_CHAIN))
2419 attr.priority |= I915_PRIORITY_NOSEMAPHORE;
2420
2421 /*
2422 * Boost priorities to new clients (new request flows).
2423 *
2424 * Allow interactive/synchronous clients to jump ahead of
2425 * the bulk clients. (FQ_CODEL)
2426 */
2427 if (list_empty(&rq->sched.signalers_list))
2428 attr.priority |= I915_PRIORITY_WAIT;
2429 } else {2585 } else {
2430 /* Serialise with context_close via the add_to_timeline */2586 /* Serialise with context_close via the add_to_timeline */
2431 i915_request_set_error_once(rq, -ENOENT);2587 i915_request_set_error_once(rq, -ENOENT);
@@ -2451,7 +2607,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 19 unmodified lines
2451 struct drm_i915_private *i915 = to_i915(dev);2607 struct drm_i915_private *i915 = to_i915(dev);
2452 struct i915_execbuffer eb;2608 struct i915_execbuffer eb;
2453 struct dma_fence *in_fence = NULL;2609 struct dma_fence *in_fence = NULL;
2454 struct dma_fence *exec_fence = NULL;
2455 struct sync_file *out_fence = NULL;2610 struct sync_file *out_fence = NULL;
2456 struct i915_vma *batch;2611 struct i915_vma *batch;
2457 int out_fence_fd = -1;2612 int out_fence_fd = -1;
@@ -2494,30 +2649,22 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 36 unmodified lines
2494 if (args->flags & I915_EXEC_IS_PINNED)2649 if (args->flags & I915_EXEC_IS_PINNED)
2495 eb.batch_flags |= I915_DISPATCH_PINNED;2650 eb.batch_flags |= I915_DISPATCH_PINNED;
2496 2651
2497 if (args->flags & I915_EXEC_FENCE_IN) {2652#define IN_FENCES (I915_EXEC_FENCE_IN | I915_EXEC_FENCE_SUBMIT)
2653 if (args->flags & IN_FENCES) {
2654 if ((args->flags & IN_FENCES) == IN_FENCES)
2655 return -EINVAL;
2656
2498 in_fence = sync_file_get_fence(lower_32_bits(args->rsvd2));2657 in_fence = sync_file_get_fence(lower_32_bits(args->rsvd2));
2499 if (!in_fence)2658 if (!in_fence)
2500 return -EINVAL;2659 return -EINVAL;
2501 }2660 }
2502 2661#undef IN_FENCES
2503 if (args->flags & I915_EXEC_FENCE_SUBMIT) {
2504 if (in_fence) {
2505 err = -EINVAL;
2506 goto err_in_fence;
2507 }
2508
2509 exec_fence = sync_file_get_fence(lower_32_bits(args->rsvd2));
2510 if (!exec_fence) {
2511 err = -EINVAL;
2512 goto err_in_fence;
2513 }
2514 }
2515 2662
2516 if (args->flags & I915_EXEC_FENCE_OUT) {2663 if (args->flags & I915_EXEC_FENCE_OUT) {
2517 out_fence_fd = get_unused_fd_flags(O_CLOEXEC);2664 out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
2518 if (out_fence_fd < 0) {2665 if (out_fence_fd < 0) {
2519 err = out_fence_fd;2666 err = out_fence_fd;
2520 goto err_exec_fence;2667 goto err_in_fence;
2521 }2668 }
2522 }2669 }
2523 2670
@@ -2608,14 +2755,13 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 84 unmodified lines
2608 }2755 }
2609 2756
2610 if (in_fence) {2757 if (in_fence) {
2611 err = i915_request_await_dma_fence(eb.request, in_fence);2758 if (args->flags & I915_EXEC_FENCE_SUBMIT)
2612 if (err < 0)2759 err = i915_request_await_execution(eb.request,
2613 goto err_request;2760 in_fence,
2614 }2761 eb.engine->bond_execute);
2615 2762 else
2616 if (exec_fence) {2763 err = i915_request_await_dma_fence(eb.request,
2617 err = i915_request_await_execution(eb.request, exec_fence,2764 in_fence);
2618 eb.engine->bond_execute);
2619 if (err < 0)2765 if (err < 0)
2620 goto err_request;2766 goto err_request;
2621 }2767 }
@@ -2643,7 +2789,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, · 21 unmodified lines
2643 */2789 */
2644 eb.request->batch = batch;2790 eb.request->batch = batch;
2645 if (batch->private)2791 if (batch->private)
2646 intel_engine_pool_mark_active(batch->private, eb.request);2792 intel_gt_buffer_pool_mark_active(batch->private, eb.request);
2647 2793
2648 trace_i915_request_queue(eb.request, eb.batch_flags);2794 trace_i915_request_queue(eb.request, eb.batch_flags);
2649 err = eb_submit(&eb, batch);2795 err = eb_submit(&eb, batch);
@@ -2672,7 +2818,7 @@ err_batch_unpin: · 22 unmodified lines
2672 i915_vma_unpin(batch);2818 i915_vma_unpin(batch);
2673err_parse:2819err_parse:
2674 if (batch->private)2820 if (batch->private)
2675 intel_engine_pool_put(batch->private);2821 intel_gt_buffer_pool_put(batch->private);
2676err_vma:2822err_vma:
2677 if (eb.trampoline)2823 if (eb.trampoline)
2678 i915_vma_unpin(eb.trampoline);2824 i915_vma_unpin(eb.trampoline);
@@ -2684,8 +2830,6 @@ err_destroy: · 5 unmodified lines
2684err_out_fence:2830err_out_fence:
2685 if (out_fence_fd != -1)2831 if (out_fence_fd != -1)
2686 put_unused_fd(out_fence_fd);2832 put_unused_fd(out_fence_fd);
2687err_exec_fence:
2688 dma_fence_put(exec_fence);
2689err_in_fence:2833err_in_fence:
2690 dma_fence_put(in_fence);2834 dma_fence_put(in_fence);
2691 return err;2835 return err;
@@ -2894,3 +3038,7 @@ end:; · 202 unmodified lines
2894 kvfree(exec2_list);3038 kvfree(exec2_list);
2895 return err;3039 return err;
2896}3040}
3041
3042#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
3043#include "selftests/i915_gem_execbuffer.c"
3044#endif
drivers/gpu/drm/i915/gem/i915_gem_fence.c+2−2
@@ -72,8 +72,8 @@ i915_gem_object_lock_fence(struct drm_i915_gem_object *obj) · 71 unmodified lines
72 0, 0);72 0, 0);
73 73
74 if (i915_sw_fence_await_reservation(&stub->chain,74 if (i915_sw_fence_await_reservation(&stub->chain,
75 obj->base.resv, NULL,75 obj->base.resv, NULL, true,
76 true, I915_FENCE_TIMEOUT,76 i915_fence_timeout(to_i915(obj->base.dev)),
77 I915_FENCE_GFP) < 0)77 I915_FENCE_GFP) < 0)
78 goto err;78 goto err;
79 79
drivers/gpu/drm/i915/gem/i915_gem_mman.c+34−0
@@ -396,6 +396,38 @@ err: · 395 unmodified lines
396 return i915_error_to_vmf_fault(ret);396 return i915_error_to_vmf_fault(ret);
397}397}
398 398
399static int
400vm_access(struct vm_area_struct *area, unsigned long addr,
401 void *buf, int len, int write)
402{
403 struct i915_mmap_offset *mmo = area->vm_private_data;
404 struct drm_i915_gem_object *obj = mmo->obj;
405 void *vaddr;
406
407 if (i915_gem_object_is_readonly(obj) && write)
408 return -EACCES;
409
410 addr -= area->vm_start;
411 if (addr >= obj->base.size)
412 return -EINVAL;
413
414 /* As this is primarily for debugging, let's focus on simplicity */
415 vaddr = i915_gem_object_pin_map(obj, I915_MAP_FORCE_WC);
416 if (IS_ERR(vaddr))
417 return PTR_ERR(vaddr);
418
419 if (write) {
420 memcpy(vaddr + addr, buf, len);
421 __i915_gem_object_flush_map(obj, addr, len);
422 } else {
423 memcpy(buf, vaddr + addr, len);
424 }
425
426 i915_gem_object_unpin_map(obj);
427
428 return len;
429}
430
399void __i915_gem_object_release_mmap_gtt(struct drm_i915_gem_object *obj)431void __i915_gem_object_release_mmap_gtt(struct drm_i915_gem_object *obj)
400{432{
401 struct i915_vma *vma;433 struct i915_vma *vma;
@@ -745,12 +777,14 @@ static void vm_close(struct vm_area_struct *vma) · 343 unmodified lines
745 777
746static const struct vm_operations_struct vm_ops_gtt = {778static const struct vm_operations_struct vm_ops_gtt = {
747 .fault = vm_fault_gtt,779 .fault = vm_fault_gtt,
780 .access = vm_access,
748 .open = vm_open,781 .open = vm_open,
749 .close = vm_close,782 .close = vm_close,
750};783};
751 784
752static const struct vm_operations_struct vm_ops_cpu = {785static const struct vm_operations_struct vm_ops_cpu = {
753 .fault = vm_fault_cpu,786 .fault = vm_fault_cpu,
787 .access = vm_access,
754 .open = vm_open,788 .open = vm_open,
755 .close = vm_close,789 .close = vm_close,
756};790};
drivers/gpu/drm/i915/gem/i915_gem_object.c+0−3
@@ -162,9 +162,7 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, · 161 unmodified lines
162 struct llist_node *freed)162 struct llist_node *freed)
163{163{
164 struct drm_i915_gem_object *obj, *on;164 struct drm_i915_gem_object *obj, *on;
165 intel_wakeref_t wakeref;
166 165
167 wakeref = intel_runtime_pm_get(&i915->runtime_pm);
168 llist_for_each_entry_safe(obj, on, freed, freed) {166 llist_for_each_entry_safe(obj, on, freed, freed) {
169 struct i915_mmap_offset *mmo, *mn;167 struct i915_mmap_offset *mmo, *mn;
170 168
@@ -224,7 +222,6 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, · 53 unmodified lines
224 call_rcu(&obj->rcu, __i915_gem_free_object_rcu);222 call_rcu(&obj->rcu, __i915_gem_free_object_rcu);
225 cond_resched();223 cond_resched();
226 }224 }
227 intel_runtime_pm_put(&i915->runtime_pm, wakeref);
228}225}
229 226
230void i915_gem_flush_free_objects(struct drm_i915_private *i915)227void i915_gem_flush_free_objects(struct drm_i915_private *i915)
drivers/gpu/drm/i915/gem/i915_gem_object_blt.c+15−11
@@ -6,8 +6,8 @@ · 5 unmodified lines
6#include "i915_drv.h"6#include "i915_drv.h"
7#include "gt/intel_context.h"7#include "gt/intel_context.h"
8#include "gt/intel_engine_pm.h"8#include "gt/intel_engine_pm.h"
9#include "gt/intel_engine_pool.h"
10#include "gt/intel_gt.h"9#include "gt/intel_gt.h"
10#include "gt/intel_gt_buffer_pool.h"
11#include "gt/intel_ring.h"11#include "gt/intel_ring.h"
12#include "i915_gem_clflush.h"12#include "i915_gem_clflush.h"
13#include "i915_gem_object_blt.h"13#include "i915_gem_object_blt.h"
@@ -18,7 +18,7 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce, · 4 unmodified lines
18{18{
19 struct drm_i915_private *i915 = ce->vm->i915;19 struct drm_i915_private *i915 = ce->vm->i915;
20 const u32 block_size = SZ_8M; /* ~1ms at 8GiB/s preemption delay */20 const u32 block_size = SZ_8M; /* ~1ms at 8GiB/s preemption delay */
21 struct intel_engine_pool_node *pool;21 struct intel_gt_buffer_pool_node *pool;
22 struct i915_vma *batch;22 struct i915_vma *batch;
23 u64 offset;23 u64 offset;
24 u64 count;24 u64 count;
@@ -33,7 +33,7 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce, · 8 unmodified lines
33 count = div_u64(round_up(vma->size, block_size), block_size);33 count = div_u64(round_up(vma->size, block_size), block_size);
34 size = (1 + 8 * count) * sizeof(u32);34 size = (1 + 8 * count) * sizeof(u32);
35 size = round_up(size, PAGE_SIZE);35 size = round_up(size, PAGE_SIZE);
36 pool = intel_engine_get_pool(ce->engine, size);36 pool = intel_gt_get_buffer_pool(ce->engine->gt, size);
37 if (IS_ERR(pool)) {37 if (IS_ERR(pool)) {
38 err = PTR_ERR(pool);38 err = PTR_ERR(pool);
39 goto out_pm;39 goto out_pm;
@@ -78,10 +78,12 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce, · 38 unmodified lines
78 } while (rem);78 } while (rem);
79 79
80 *cmd = MI_BATCH_BUFFER_END;80 *cmd = MI_BATCH_BUFFER_END;
81 intel_gt_chipset_flush(ce->vm->gt);
82 81
82 i915_gem_object_flush_map(pool->obj);
83 i915_gem_object_unpin_map(pool->obj);83 i915_gem_object_unpin_map(pool->obj);
84 84
85 intel_gt_chipset_flush(ce->vm->gt);
86
85 batch = i915_vma_instance(pool->obj, ce->vm, NULL);87 batch = i915_vma_instance(pool->obj, ce->vm, NULL);
86 if (IS_ERR(batch)) {88 if (IS_ERR(batch)) {
87 err = PTR_ERR(batch);89 err = PTR_ERR(batch);
@@ -96,7 +98,7 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce, · 8 unmodified lines
96 return batch;98 return batch;
97 99
98out_put:100out_put:
99 intel_engine_pool_put(pool);101 intel_gt_buffer_pool_put(pool);
100out_pm:102out_pm:
101 intel_engine_pm_put(ce->engine);103 intel_engine_pm_put(ce->engine);
102 return ERR_PTR(err);104 return ERR_PTR(err);
@@ -114,13 +116,13 @@ int intel_emit_vma_mark_active(struct i915_vma *vma, struct i915_request *rq) · 11 unmodified lines
114 if (unlikely(err))116 if (unlikely(err))
115 return err;117 return err;
116 118
117 return intel_engine_pool_mark_active(vma->private, rq);119 return intel_gt_buffer_pool_mark_active(vma->private, rq);
118}120}
119 121
120void intel_emit_vma_release(struct intel_context *ce, struct i915_vma *vma)122void intel_emit_vma_release(struct intel_context *ce, struct i915_vma *vma)
121{123{
122 i915_vma_unpin(vma);124 i915_vma_unpin(vma);
123 intel_engine_pool_put(vma->private);125 intel_gt_buffer_pool_put(vma->private);
124 intel_engine_pm_put(ce->engine);126 intel_engine_pm_put(ce->engine);
125}127}
126 128
@@ -213,7 +215,7 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce, · 86 unmodified lines
213{215{
214 struct drm_i915_private *i915 = ce->vm->i915;216 struct drm_i915_private *i915 = ce->vm->i915;
215 const u32 block_size = SZ_8M; /* ~1ms at 8GiB/s preemption delay */217 const u32 block_size = SZ_8M; /* ~1ms at 8GiB/s preemption delay */
216 struct intel_engine_pool_node *pool;218 struct intel_gt_buffer_pool_node *pool;
217 struct i915_vma *batch;219 struct i915_vma *batch;
218 u64 src_offset, dst_offset;220 u64 src_offset, dst_offset;
219 u64 count, rem;221 u64 count, rem;
@@ -228,7 +230,7 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce, · 8 unmodified lines
228 count = div_u64(round_up(dst->size, block_size), block_size);230 count = div_u64(round_up(dst->size, block_size), block_size);
229 size = (1 + 11 * count) * sizeof(u32);231 size = (1 + 11 * count) * sizeof(u32);
230 size = round_up(size, PAGE_SIZE);232 size = round_up(size, PAGE_SIZE);
231 pool = intel_engine_get_pool(ce->engine, size);233 pool = intel_gt_get_buffer_pool(ce->engine->gt, size);
232 if (IS_ERR(pool)) {234 if (IS_ERR(pool)) {
233 err = PTR_ERR(pool);235 err = PTR_ERR(pool);
234 goto out_pm;236 goto out_pm;
@@ -289,10 +291,12 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce, · 54 unmodified lines
289 } while (rem);291 } while (rem);
290 292
291 *cmd = MI_BATCH_BUFFER_END;293 *cmd = MI_BATCH_BUFFER_END;
292 intel_gt_chipset_flush(ce->vm->gt);
293 294
295 i915_gem_object_flush_map(pool->obj);
294 i915_gem_object_unpin_map(pool->obj);296 i915_gem_object_unpin_map(pool->obj);
295 297
298 intel_gt_chipset_flush(ce->vm->gt);
299
296 batch = i915_vma_instance(pool->obj, ce->vm, NULL);300 batch = i915_vma_instance(pool->obj, ce->vm, NULL);
297 if (IS_ERR(batch)) {301 if (IS_ERR(batch)) {
298 err = PTR_ERR(batch);302 err = PTR_ERR(batch);
@@ -307,7 +311,7 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce, · 8 unmodified lines
307 return batch;311 return batch;
308 312
309out_put:313out_put:
310 intel_engine_pool_put(pool);314 intel_gt_buffer_pool_put(pool);
311out_pm:315out_pm:
312 intel_engine_pm_put(ce->engine);316 intel_engine_pm_put(ce->engine);
313 return ERR_PTR(err);317 return ERR_PTR(err);
drivers/gpu/drm/i915/gem/i915_gem_object_blt.h+0−1
@@ -10,7 +10,6 @@ · 9 unmodified lines
10 10
11#include "gt/intel_context.h"11#include "gt/intel_context.h"
12#include "gt/intel_engine_pm.h"12#include "gt/intel_engine_pm.h"
13#include "gt/intel_engine_pool.h"
14#include "i915_vma.h"13#include "i915_vma.h"
15 14
16struct drm_i915_gem_object;15struct drm_i915_gem_object;
drivers/gpu/drm/i915/gem/i915_gem_pages.c+1−0
@@ -391,6 +391,7 @@ void __i915_gem_object_flush_map(struct drm_i915_gem_object *obj, · 390 unmodified lines
391 GEM_BUG_ON(range_overflows_t(typeof(obj->base.size),391 GEM_BUG_ON(range_overflows_t(typeof(obj->base.size),
392 offset, size, obj->base.size));392 offset, size, obj->base.size));
393 393
394 wmb(); /* let all previous writes be visible to coherent partners */
394 obj->mm.dirty = true;395 obj->mm.dirty = true;
395 396
396 if (obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE)397 if (obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE)
drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c+1−30
@@ -302,35 +302,6 @@ static void fill_scratch(struct tiled_blits *t, u32 *vaddr, u32 val) · 301 unmodified lines
302 i915_gem_object_flush_map(t->scratch.vma->obj);302 i915_gem_object_flush_map(t->scratch.vma->obj);
303}303}
304 304
305static void hexdump(const void *buf, size_t len)
306{
307 const size_t rowsize = 8 * sizeof(u32);
308 const void *prev = NULL;
309 bool skip = false;
310 size_t pos;
311
312 for (pos = 0; pos < len; pos += rowsize) {
313 char line[128];
314
315 if (prev && !memcmp(prev, buf + pos, rowsize)) {
316 if (!skip) {
317 pr_info("*\n");
318 skip = true;
319 }
320 continue;
321 }
322
323 WARN_ON_ONCE(hex_dump_to_buffer(buf + pos, len - pos,
324 rowsize, sizeof(u32),
325 line, sizeof(line),
326 false) >= sizeof(line));
327 pr_info("[%04zx] %s\n", pos, line);
328
329 prev = buf + pos;
330 skip = false;
331 }
332}
333
334static u64 swizzle_bit(unsigned int bit, u64 offset)305static u64 swizzle_bit(unsigned int bit, u64 offset)
335{306{
336 return (offset & BIT_ULL(bit)) >> (bit - 6);307 return (offset & BIT_ULL(bit)) >> (bit - 6);
@@ -426,7 +397,7 @@ static int verify_buffer(const struct tiled_blits *t, · 89 unmodified lines
426 pr_err("Invalid %s tiling detected at (%d, %d), start_val %x\n",397 pr_err("Invalid %s tiling detected at (%d, %d), start_val %x\n",
427 repr_tiling(buf->tiling),398 repr_tiling(buf->tiling),
428 x, y, buf->start_val);399 x, y, buf->start_val);
429 hexdump(vaddr, 4096);400 igt_hexdump(vaddr, 4096);
430 }401 }
431 402
432 i915_gem_object_unpin_map(buf->vma->obj);403 i915_gem_object_unpin_map(buf->vma->obj);
drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c+2−0
@@ -158,6 +158,8 @@ static int wc_set(struct context *ctx, unsigned long offset, u32 v) · 157 unmodified lines
158 return PTR_ERR(map);158 return PTR_ERR(map);
159 159
160 map[offset / sizeof(*map)] = v;160 map[offset / sizeof(*map)] = v;
161
162 __i915_gem_object_flush_map(ctx->obj, offset, sizeof(*map));
161 i915_gem_object_unpin_map(ctx->obj);163 i915_gem_object_unpin_map(ctx->obj);
162 164
163 return 0;165 return 0;
drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c+32−17
@@ -972,12 +972,6 @@ emit_rpcs_query(struct drm_i915_gem_object *obj, · 971 unmodified lines
972 goto err_batch;972 goto err_batch;
973 }973 }
974 974
975 err = rq->engine->emit_bb_start(rq,
976 batch->node.start, batch->node.size,
977 0);
978 if (err)
979 goto err_request;
980
981 i915_vma_lock(batch);975 i915_vma_lock(batch);
982 err = i915_request_await_object(rq, batch->obj, false);976 err = i915_request_await_object(rq, batch->obj, false);
983 if (err == 0)977 if (err == 0)
@@ -994,6 +988,18 @@ emit_rpcs_query(struct drm_i915_gem_object *obj, · 10 unmodified lines
994 if (err)988 if (err)
995 goto skip_request;989 goto skip_request;
996 990
991 if (rq->engine->emit_init_breadcrumb) {
992 err = rq->engine->emit_init_breadcrumb(rq);
993 if (err)
994 goto skip_request;
995 }
996
997 err = rq->engine->emit_bb_start(rq,
998 batch->node.start, batch->node.size,
999 0);
1000 if (err)
1001 goto skip_request;
1002
997 i915_vma_unpin_and_release(&batch, 0);1003 i915_vma_unpin_and_release(&batch, 0);
998 i915_vma_unpin(vma);1004 i915_vma_unpin(vma);
999 1005
@@ -1005,7 +1011,6 @@ emit_rpcs_query(struct drm_i915_gem_object *obj, · 5 unmodified lines
1005 1011
1006skip_request:1012skip_request:
1007 i915_request_set_error_once(rq, err);1013 i915_request_set_error_once(rq, err);
1008err_request:
1009 i915_request_add(rq);1014 i915_request_add(rq);
1010err_batch:1015err_batch:
1011 i915_vma_unpin_and_release(&batch, 0);1016 i915_vma_unpin_and_release(&batch, 0);
@@ -1541,10 +1546,6 @@ static int write_to_scratch(struct i915_gem_context *ctx, · 529 unmodified lines
1541 goto err_unpin;1546 goto err_unpin;
1542 }1547 }
1543 1548
1544 err = engine->emit_bb_start(rq, vma->node.start, vma->node.size, 0);
1545 if (err)
1546 goto err_request;
1547
1548 i915_vma_lock(vma);1549 i915_vma_lock(vma);
1549 err = i915_request_await_object(rq, vma->obj, false);1550 err = i915_request_await_object(rq, vma->obj, false);
1550 if (err == 0)1551 if (err == 0)
@@ -1553,6 +1554,16 @@ static int write_to_scratch(struct i915_gem_context *ctx, · 2 unmodified lines
1553 if (err)1554 if (err)
1554 goto skip_request;1555 goto skip_request;
1555 1556
1557 if (rq->engine->emit_init_breadcrumb) {
1558 err = rq->engine->emit_init_breadcrumb(rq);
1559 if (err)
1560 goto skip_request;
1561 }
1562
1563 err = engine->emit_bb_start(rq, vma->node.start, vma->node.size, 0);
1564 if (err)
1565 goto skip_request;
1566
1556 i915_vma_unpin(vma);1567 i915_vma_unpin(vma);
1557 1568
1558 i915_request_add(rq);1569 i915_request_add(rq);
@@ -1560,7 +1571,6 @@ static int write_to_scratch(struct i915_gem_context *ctx, · 1 unmodified line
1560 goto out_vm;1571 goto out_vm;
1561skip_request:1572skip_request:
1562 i915_request_set_error_once(rq, err);1573 i915_request_set_error_once(rq, err);
1563err_request:
1564 i915_request_add(rq);1574 i915_request_add(rq);
1565err_unpin:1575err_unpin:
1566 i915_vma_unpin(vma);1576 i915_vma_unpin(vma);
@@ -1674,10 +1684,6 @@ static int read_from_scratch(struct i915_gem_context *ctx, · 107 unmodified lines
1674 goto err_unpin;1684 goto err_unpin;
1675 }1685 }
1676 1686
1677 err = engine->emit_bb_start(rq, vma->node.start, vma->node.size, flags);
1678 if (err)
1679 goto err_request;
1680
1681 i915_vma_lock(vma);1687 i915_vma_lock(vma);
1682 err = i915_request_await_object(rq, vma->obj, true);1688 err = i915_request_await_object(rq, vma->obj, true);
1683 if (err == 0)1689 if (err == 0)
@@ -1686,6 +1692,16 @@ static int read_from_scratch(struct i915_gem_context *ctx, · 2 unmodified lines
1686 if (err)1692 if (err)
1687 goto skip_request;1693 goto skip_request;
1688 1694
1695 if (rq->engine->emit_init_breadcrumb) {
1696 err = rq->engine->emit_init_breadcrumb(rq);
1697 if (err)
1698 goto skip_request;
1699 }
1700
1701 err = engine->emit_bb_start(rq, vma->node.start, vma->node.size, flags);
1702 if (err)
1703 goto skip_request;
1704
1689 i915_vma_unpin(vma);1705 i915_vma_unpin(vma);
1690 1706
1691 i915_request_add(rq);1707 i915_request_add(rq);
@@ -1708,7 +1724,6 @@ static int read_from_scratch(struct i915_gem_context *ctx, · 16 unmodified lines
1708 goto out_vm;1724 goto out_vm;
1709skip_request:1725skip_request:
1710 i915_request_set_error_once(rq, err);1726 i915_request_set_error_once(rq, err);
1711err_request:
1712 i915_request_add(rq);1727 i915_request_add(rq);
1713err_unpin:1728err_unpin:
1714 i915_vma_unpin(vma);1729 i915_vma_unpin(vma);
drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c+171−0
@@ -0,0 +1,171 @@
1// SPDX-License-Identifier: MIT
2/*
3 * Copyright © 2020 Intel Corporation
4 */
5
6#include "i915_selftest.h"
7
8#include "gt/intel_engine_pm.h"
9#include "selftests/igt_flush_test.h"
10
11static u64 read_reloc(const u32 *map, int x, const u64 mask)
12{
13 u64 reloc;
14
15 memcpy(&reloc, &map[x], sizeof(reloc));
16 return reloc & mask;
17}
18
19static int __igt_gpu_reloc(struct i915_execbuffer *eb,
20 struct drm_i915_gem_object *obj)
21{
22 const unsigned int offsets[] = { 8, 3, 0 };
23 const u64 mask =
24 GENMASK_ULL(eb->reloc_cache.use_64bit_reloc ? 63 : 31, 0);
25 const u32 *map = page_mask_bits(obj->mm.mapping);
26 struct i915_request *rq;
27 struct i915_vma *vma;
28 int err;
29 int i;
30
31 vma = i915_vma_instance(obj, eb->context->vm, NULL);
32 if (IS_ERR(vma))
33 return PTR_ERR(vma);
34
35 err = i915_vma_pin(vma, 0, 0, PIN_USER | PIN_HIGH);
36 if (err)
37 return err;
38
39 /* 8-Byte aligned */
40 if (!__reloc_entry_gpu(eb, vma,
41 offsets[0] * sizeof(u32),
42 0)) {
43 err = -EIO;
44 goto unpin_vma;
45 }
46
47 /* !8-Byte aligned */
48 if (!__reloc_entry_gpu(eb, vma,
49 offsets[1] * sizeof(u32),
50 1)) {
51 err = -EIO;
52 goto unpin_vma;
53 }
54
55 /* Skip to the end of the cmd page */
56 i = PAGE_SIZE / sizeof(u32) - RELOC_TAIL - 1;
57 i -= eb->reloc_cache.rq_size;
58 memset32(eb->reloc_cache.rq_cmd + eb->reloc_cache.rq_size,
59 MI_NOOP, i);
60 eb->reloc_cache.rq_size += i;
61
62 /* Force batch chaining */
63 if (!__reloc_entry_gpu(eb, vma,
64 offsets[2] * sizeof(u32),
65 2)) {
66 err = -EIO;
67 goto unpin_vma;
68 }
69
70 GEM_BUG_ON(!eb->reloc_cache.rq);
71 rq = i915_request_get(eb->reloc_cache.rq);
72 err = reloc_gpu_flush(&eb->reloc_cache);
73 if (err)
74 goto put_rq;
75 GEM_BUG_ON(eb->reloc_cache.rq);
76
77 err = i915_gem_object_wait(obj, I915_WAIT_INTERRUPTIBLE, HZ / 2);
78 if (err) {
79 intel_gt_set_wedged(eb->engine->gt);
80 goto put_rq;
81 }
82
83 if (!i915_request_completed(rq)) {
84 pr_err("%s: did not wait for relocations!\n", eb->engine->name);
85 err = -EINVAL;
86 goto put_rq;
87 }
88
89 for (i = 0; i < ARRAY_SIZE(offsets); i++) {
90 u64 reloc = read_reloc(map, offsets[i], mask);
91
92 if (reloc != i) {
93 pr_err("%s[%d]: map[%d] %llx != %x\n",
94 eb->engine->name, i, offsets[i], reloc, i);
95 err = -EINVAL;
96 }
97 }
98 if (err)
99 igt_hexdump(map, 4096);
100
101put_rq:
102 i915_request_put(rq);
103unpin_vma:
104 i915_vma_unpin(vma);
105 return err;
106}
107
108static int igt_gpu_reloc(void *arg)
109{
110 struct i915_execbuffer eb;
111 struct drm_i915_gem_object *scratch;
112 int err = 0;
113 u32 *map;
114
115 eb.i915 = arg;
116
117 scratch = i915_gem_object_create_internal(eb.i915, 4096);
118 if (IS_ERR(scratch))
119 return PTR_ERR(scratch);
120
121 map = i915_gem_object_pin_map(scratch, I915_MAP_WC);
122 if (IS_ERR(map)) {
123 err = PTR_ERR(map);
124 goto err_scratch;
125 }
126
127 for_each_uabi_engine(eb.engine, eb.i915) {
128 reloc_cache_init(&eb.reloc_cache, eb.i915);
129 memset(map, POISON_INUSE, 4096);
130
131 intel_engine_pm_get(eb.engine);
132 eb.context = intel_context_create(eb.engine);
133 if (IS_ERR(eb.context)) {
134 err = PTR_ERR(eb.context);
135 goto err_pm;
136 }
137
138 err = intel_context_pin(eb.context);
139 if (err)
140 goto err_put;
141
142 err = __igt_gpu_reloc(&eb, scratch);
143
144 intel_context_unpin(eb.context);
145err_put:
146 intel_context_put(eb.context);
147err_pm:
148 intel_engine_pm_put(eb.engine);
149 if (err)
150 break;
151 }
152
153 if (igt_flush_test(eb.i915))
154 err = -EIO;
155
156err_scratch:
157 i915_gem_object_put(scratch);
158 return err;
159}
160
161int i915_gem_execbuffer_live_selftests(struct drm_i915_private *i915)
162{
163 static const struct i915_subtest tests[] = {
164 SUBTEST(igt_gpu_reloc),
165 };
166
167 if (intel_gt_is_wedged(&i915->gt))
168 return 0;
169
170 return i915_live_subtests(tests, i915);
171}
drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c+124−0
@@ -952,6 +952,129 @@ static int igt_mmap(void *arg) · 951 unmodified lines
952 return 0;952 return 0;
953}953}
954 954
955static const char *repr_mmap_type(enum i915_mmap_type type)
956{
957 switch (type) {
958 case I915_MMAP_TYPE_GTT: return "gtt";
959 case I915_MMAP_TYPE_WB: return "wb";
960 case I915_MMAP_TYPE_WC: return "wc";
961 case I915_MMAP_TYPE_UC: return "uc";
962 default: return "unknown";
963 }
964}
965
966static bool can_access(const struct drm_i915_gem_object *obj)
967{
968 unsigned int flags =
969 I915_GEM_OBJECT_HAS_STRUCT_PAGE | I915_GEM_OBJECT_HAS_IOMEM;
970
971 return i915_gem_object_type_has(obj, flags);
972}
973
974static int __igt_mmap_access(struct drm_i915_private *i915,
975 struct drm_i915_gem_object *obj,
976 enum i915_mmap_type type)
977{
978 struct i915_mmap_offset *mmo;
979 unsigned long __user *ptr;
980 unsigned long A, B;
981 unsigned long x, y;
982 unsigned long addr;
983 int err;
984
985 memset(&A, 0xAA, sizeof(A));
986 memset(&B, 0xBB, sizeof(B));
987
988 if (!can_mmap(obj, type) || !can_access(obj))
989 return 0;
990
991 mmo = mmap_offset_attach(obj, type, NULL);
992 if (IS_ERR(mmo))
993 return PTR_ERR(mmo);
994
995 addr = igt_mmap_node(i915, &mmo->vma_node, 0, PROT_WRITE, MAP_SHARED);
996 if (IS_ERR_VALUE(addr))
997 return addr;
998 ptr = (unsigned long __user *)addr;
999
1000 err = __put_user(A, ptr);
1001 if (err) {
1002 pr_err("%s(%s): failed to write into user mmap\n",
1003 obj->mm.region->name, repr_mmap_type(type));
1004 goto out_unmap;
1005 }
1006
1007 intel_gt_flush_ggtt_writes(&i915->gt);
1008
1009 err = access_process_vm(current, addr, &x, sizeof(x), 0);
1010 if (err != sizeof(x)) {
1011 pr_err("%s(%s): access_process_vm() read failed\n",
1012 obj->mm.region->name, repr_mmap_type(type));
1013 goto out_unmap;
1014 }
1015
1016 err = access_process_vm(current, addr, &B, sizeof(B), FOLL_WRITE);
1017 if (err != sizeof(B)) {
1018 pr_err("%s(%s): access_process_vm() write failed\n",
1019 obj->mm.region->name, repr_mmap_type(type));
1020 goto out_unmap;
1021 }
1022
1023 intel_gt_flush_ggtt_writes(&i915->gt);
1024
1025 err = __get_user(y, ptr);
1026 if (err) {
1027 pr_err("%s(%s): failed to read from user mmap\n",
1028 obj->mm.region->name, repr_mmap_type(type));
1029 goto out_unmap;
1030 }
1031
1032 if (x != A || y != B) {
1033 pr_err("%s(%s): failed to read/write values, found (%lx, %lx)\n",
1034 obj->mm.region->name, repr_mmap_type(type),
1035 x, y);
1036 err = -EINVAL;
1037 goto out_unmap;
1038 }
1039
1040out_unmap:
1041 vm_munmap(addr, obj->base.size);
1042 return err;
1043}
1044
1045static int igt_mmap_access(void *arg)
1046{
1047 struct drm_i915_private *i915 = arg;
1048 struct intel_memory_region *mr;
1049 enum intel_region_id id;
1050
1051 for_each_memory_region(mr, i915, id) {
1052 struct drm_i915_gem_object *obj;
1053 int err;
1054
1055 obj = i915_gem_object_create_region(mr, PAGE_SIZE, 0);
1056 if (obj == ERR_PTR(-ENODEV))
1057 continue;
1058
1059 if (IS_ERR(obj))
1060 return PTR_ERR(obj);
1061
1062 err = __igt_mmap_access(i915, obj, I915_MMAP_TYPE_GTT);
1063 if (err == 0)
1064 err = __igt_mmap_access(i915, obj, I915_MMAP_TYPE_WB);
1065 if (err == 0)
1066 err = __igt_mmap_access(i915, obj, I915_MMAP_TYPE_WC);
1067 if (err == 0)
1068 err = __igt_mmap_access(i915, obj, I915_MMAP_TYPE_UC);
1069
1070 i915_gem_object_put(obj);
1071 if (err)
1072 return err;
1073 }
1074
1075 return 0;
1076}
1077
955static int __igt_mmap_gpu(struct drm_i915_private *i915,1078static int __igt_mmap_gpu(struct drm_i915_private *i915,
956 struct drm_i915_gem_object *obj,1079 struct drm_i915_gem_object *obj,
957 enum i915_mmap_type type)1080 enum i915_mmap_type type)
@@ -1229,6 +1352,7 @@ int i915_gem_mman_live_selftests(struct drm_i915_private *i915) · 271 unmodified lines
1229 SUBTEST(igt_smoke_tiling),1352 SUBTEST(igt_smoke_tiling),
1230 SUBTEST(igt_mmap_offset_exhaustion),1353 SUBTEST(igt_mmap_offset_exhaustion),
1231 SUBTEST(igt_mmap),1354 SUBTEST(igt_mmap),
1355 SUBTEST(igt_mmap_access),
1232 SUBTEST(igt_mmap_revoke),1356 SUBTEST(igt_mmap_revoke),
1233 SUBTEST(igt_mmap_gpu),1357 SUBTEST(igt_mmap_gpu),
1234 };1358 };
drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c+10−16
@@ -83,6 +83,8 @@ igt_emit_store_dw(struct i915_vma *vma, · 82 unmodified lines
83 offset += PAGE_SIZE;83 offset += PAGE_SIZE;
84 }84 }
85 *cmd = MI_BATCH_BUFFER_END;85 *cmd = MI_BATCH_BUFFER_END;
86
87 i915_gem_object_flush_map(obj);
86 i915_gem_object_unpin_map(obj);88 i915_gem_object_unpin_map(obj);
87 89
88 intel_gt_chipset_flush(vma->vm->gt);90 intel_gt_chipset_flush(vma->vm->gt);
@@ -126,16 +128,6 @@ int igt_gpu_fill_dw(struct intel_context *ce, · 37 unmodified lines
126 goto err_batch;128 goto err_batch;
127 }129 }
128 130
129 flags = 0;
130 if (INTEL_GEN(ce->vm->i915) <= 5)
131 flags |= I915_DISPATCH_SECURE;
132
133 err = rq->engine->emit_bb_start(rq,
134 batch->node.start, batch->node.size,
135 flags);
136 if (err)
137 goto err_request;
138
139 i915_vma_lock(batch);131 i915_vma_lock(batch);
140 err = i915_request_await_object(rq, batch->obj, false);132 err = i915_request_await_object(rq, batch->obj, false);
141 if (err == 0)133 if (err == 0)
@@ -152,15 +144,17 @@ int igt_gpu_fill_dw(struct intel_context *ce, · 10 unmodified lines
152 if (err)144 if (err)
153 goto skip_request;145 goto skip_request;
154 146
155 i915_request_add(rq);147 flags = 0;
156 148 if (INTEL_GEN(ce->vm->i915) <= 5)
157 i915_vma_unpin_and_release(&batch, 0);149 flags |= I915_DISPATCH_SECURE;
158 150
159 return 0;151 err = rq->engine->emit_bb_start(rq,
152 batch->node.start, batch->node.size,
153 flags);
160 154
161skip_request:155skip_request:
162 i915_request_set_error_once(rq, err);156 if (err)
163err_request:157 i915_request_set_error_once(rq, err);
164 i915_request_add(rq);158 i915_request_add(rq);
165err_batch:159err_batch:
166 i915_vma_unpin_and_release(&batch, 0);160 i915_vma_unpin_and_release(&batch, 0);
drivers/gpu/drm/i915/gt/gen8_ppgtt.c+13−2
@@ -389,6 +389,16 @@ static int gen8_ppgtt_alloc(struct i915_address_space *vm, · 388 unmodified lines
389 return err;389 return err;
390}390}
391 391
392static __always_inline void
393write_pte(gen8_pte_t *pte, const gen8_pte_t val)
394{
395 /* Magic delays? Or can we refine these to flush all in one pass? */
396 *pte = val;
397 wmb(); /* cpu to cache */
398 clflush(pte); /* cache to memory */
399 wmb(); /* visible to all */
400}
401
392static __always_inline u64402static __always_inline u64
393gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt,403gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt,
394 struct i915_page_directory *pdp,404 struct i915_page_directory *pdp,
@@ -405,7 +415,8 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt, · 10 unmodified lines
405 vaddr = kmap_atomic_px(i915_pt_entry(pd, gen8_pd_index(idx, 1)));415 vaddr = kmap_atomic_px(i915_pt_entry(pd, gen8_pd_index(idx, 1)));
406 do {416 do {
407 GEM_BUG_ON(iter->sg->length < I915_GTT_PAGE_SIZE);417 GEM_BUG_ON(iter->sg->length < I915_GTT_PAGE_SIZE);
408 vaddr[gen8_pd_index(idx, 0)] = pte_encode | iter->dma;418 write_pte(&vaddr[gen8_pd_index(idx, 0)],
419 pte_encode | iter->dma);
409 420
410 iter->dma += I915_GTT_PAGE_SIZE;421 iter->dma += I915_GTT_PAGE_SIZE;
411 if (iter->dma >= iter->max) {422 if (iter->dma >= iter->max) {
@@ -487,7 +498,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma, · 75 unmodified lines
487 498
488 do {499 do {
489 GEM_BUG_ON(iter->sg->length < page_size);500 GEM_BUG_ON(iter->sg->length < page_size);
490 vaddr[index++] = encode | iter->dma;501 write_pte(&vaddr[index++], encode | iter->dma);
491 502
492 start += page_size;503 start += page_size;
493 iter->dma += page_size;504 iter->dma += page_size;
drivers/gpu/drm/i915/gt/intel_breadcrumbs.c+42−10
@@ -142,6 +142,18 @@ static void add_retire(struct intel_breadcrumbs *b, struct intel_timeline *tl) · 141 unmodified lines
142 intel_engine_add_retire(engine, tl);142 intel_engine_add_retire(engine, tl);
143}143}
144 144
145static void __signal_request(struct i915_request *rq, struct list_head *signals)
146{
147 GEM_BUG_ON(!test_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags));
148 clear_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags);
149
150 if (!__dma_fence_signal(&rq->fence))
151 return;
152
153 i915_request_get(rq);
154 list_add_tail(&rq->signal_link, signals);
155}
156
145static void signal_irq_work(struct irq_work *work)157static void signal_irq_work(struct irq_work *work)
146{158{
147 struct intel_breadcrumbs *b = container_of(work, typeof(*b), irq_work);159 struct intel_breadcrumbs *b = container_of(work, typeof(*b), irq_work);
@@ -155,6 +167,8 @@ static void signal_irq_work(struct irq_work *work) · 7 unmodified lines
155 if (b->irq_armed && list_empty(&b->signalers))167 if (b->irq_armed && list_empty(&b->signalers))
156 __intel_breadcrumbs_disarm_irq(b);168 __intel_breadcrumbs_disarm_irq(b);
157 169
170 list_splice_init(&b->signaled_requests, &signal);
171
158 list_for_each_entry_safe(ce, cn, &b->signalers, signal_link) {172 list_for_each_entry_safe(ce, cn, &b->signalers, signal_link) {
159 GEM_BUG_ON(list_empty(&ce->signals));173 GEM_BUG_ON(list_empty(&ce->signals));
160 174
@@ -163,24 +177,15 @@ static void signal_irq_work(struct irq_work *work) · 2 unmodified lines
163 list_entry(pos, typeof(*rq), signal_link);177 list_entry(pos, typeof(*rq), signal_link);
164 178
165 GEM_BUG_ON(!check_signal_order(ce, rq));179 GEM_BUG_ON(!check_signal_order(ce, rq));
166
167 if (!__request_completed(rq))180 if (!__request_completed(rq))
168 break;181 break;
169 182
170 GEM_BUG_ON(!test_bit(I915_FENCE_FLAG_SIGNAL,
171 &rq->fence.flags));
172 clear_bit(I915_FENCE_FLAG_SIGNAL, &rq->fence.flags);
173
174 if (!__dma_fence_signal(&rq->fence))
175 continue;
176
177 /*183 /*
178 * Queue for execution after dropping the signaling184 * Queue for execution after dropping the signaling
179 * spinlock as the callback chain may end up adding185 * spinlock as the callback chain may end up adding
180 * more signalers to the same context or engine.186 * more signalers to the same context or engine.
181 */187 */
182 i915_request_get(rq);188 __signal_request(rq, &signal);
183 list_add_tail(&rq->signal_link, &signal);
184 }189 }
185 190
186 /*191 /*
@@ -255,6 +260,7 @@ void intel_engine_init_breadcrumbs(struct intel_engine_cs *engine) · 68 unmodified lines
255 260
256 spin_lock_init(&b->irq_lock);261 spin_lock_init(&b->irq_lock);
257 INIT_LIST_HEAD(&b->signalers);262 INIT_LIST_HEAD(&b->signalers);
263 INIT_LIST_HEAD(&b->signaled_requests);
258 264
259 init_irq_work(&b->irq_work, signal_irq_work);265 init_irq_work(&b->irq_work, signal_irq_work);
260}266}
@@ -274,6 +280,32 @@ void intel_engine_reset_breadcrumbs(struct intel_engine_cs *engine) · 13 unmodified lines
274 spin_unlock_irqrestore(&b->irq_lock, flags);280 spin_unlock_irqrestore(&b->irq_lock, flags);
275}281}
276 282
283void intel_engine_transfer_stale_breadcrumbs(struct intel_engine_cs *engine,
284 struct intel_context *ce)
285{
286 struct intel_breadcrumbs *b = &engine->breadcrumbs;
287 unsigned long flags;
288
289 spin_lock_irqsave(&b->irq_lock, flags);
290 if (!list_empty(&ce->signals)) {
291 struct i915_request *rq, *next;
292
293 /* Queue for executing the signal callbacks in the irq_work */
294 list_for_each_entry_safe(rq, next, &ce->signals, signal_link) {
295 GEM_BUG_ON(rq->engine != engine);
296 GEM_BUG_ON(!__request_completed(rq));
297
298 __signal_request(rq, &b->signaled_requests);
299 }
300
301 INIT_LIST_HEAD(&ce->signals);
302 list_del_init(&ce->signal_link);
303
304 irq_work_queue(&b->irq_work);
305 }
306 spin_unlock_irqrestore(&b->irq_lock, flags);
307}
308
277void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine)309void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine)
278{310{
279}311}
drivers/gpu/drm/i915/gt/intel_engine.h+31−15
@@ -238,22 +238,35 @@ intel_engine_signal_breadcrumbs(struct intel_engine_cs *engine) · 237 unmodified lines
238void intel_engine_reset_breadcrumbs(struct intel_engine_cs *engine);238void intel_engine_reset_breadcrumbs(struct intel_engine_cs *engine);
239void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine);239void intel_engine_fini_breadcrumbs(struct intel_engine_cs *engine);
240 240
241void intel_engine_transfer_stale_breadcrumbs(struct intel_engine_cs *engine,
242 struct intel_context *ce);
243
241void intel_engine_print_breadcrumbs(struct intel_engine_cs *engine,244void intel_engine_print_breadcrumbs(struct intel_engine_cs *engine,
242 struct drm_printer *p);245 struct drm_printer *p);
243 246
244static inline u32 *gen8_emit_pipe_control(u32 *batch, u32 flags, u32 offset)247static inline u32 *__gen8_emit_pipe_control(u32 *batch, u32 flags0, u32 flags1, u32 offset)
245{248{
246 memset(batch, 0, 6 * sizeof(u32));249 memset(batch, 0, 6 * sizeof(u32));
247 250
248 batch[0] = GFX_OP_PIPE_CONTROL(6);251 batch[0] = GFX_OP_PIPE_CONTROL(6) | flags0;
249 batch[1] = flags;252 batch[1] = flags1;
250 batch[2] = offset;253 batch[2] = offset;
251 254
252 return batch + 6;255 return batch + 6;
253}256}
254 257
258static inline u32 *gen8_emit_pipe_control(u32 *batch, u32 flags, u32 offset)
259{
260 return __gen8_emit_pipe_control(batch, 0, flags, offset);
261}
262
263static inline u32 *gen12_emit_pipe_control(u32 *batch, u32 flags0, u32 flags1, u32 offset)
264{
265 return __gen8_emit_pipe_control(batch, flags0, flags1, offset);
266}
267
255static inline u32 *268static inline u32 *
256gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags)269__gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags0, u32 flags1)
257{270{
258 /* We're using qword write, offset should be aligned to 8 bytes. */271 /* We're using qword write, offset should be aligned to 8 bytes. */
259 GEM_BUG_ON(!IS_ALIGNED(gtt_offset, 8));272 GEM_BUG_ON(!IS_ALIGNED(gtt_offset, 8));
@@ -262,8 +275,8 @@ gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags) · 2 unmodified lines
262 * need a prior CS_STALL, which is emitted by the flush275 * need a prior CS_STALL, which is emitted by the flush
263 * following the batch.276 * following the batch.
264 */277 */
265 *cs++ = GFX_OP_PIPE_CONTROL(6);278 *cs++ = GFX_OP_PIPE_CONTROL(6) | flags0;
266 *cs++ = flags | PIPE_CONTROL_QW_WRITE | PIPE_CONTROL_GLOBAL_GTT_IVB;279 *cs++ = flags1 | PIPE_CONTROL_QW_WRITE | PIPE_CONTROL_GLOBAL_GTT_IVB;
267 *cs++ = gtt_offset;280 *cs++ = gtt_offset;
268 *cs++ = 0;281 *cs++ = 0;
269 *cs++ = value;282 *cs++ = value;
@@ -273,6 +286,18 @@ gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags) · 3 unmodified lines
273 return cs;286 return cs;
274}287}
275 288
289static inline u32*
290gen8_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags)
291{
292 return __gen8_emit_ggtt_write_rcs(cs, value, gtt_offset, 0, flags);
293}
294
295static inline u32*
296gen12_emit_ggtt_write_rcs(u32 *cs, u32 value, u32 gtt_offset, u32 flags0, u32 flags1)
297{
298 return __gen8_emit_ggtt_write_rcs(cs, value, gtt_offset, flags0, flags1);
299}
300
276static inline u32 *301static inline u32 *
277gen8_emit_ggtt_write(u32 *cs, u32 value, u32 gtt_offset, u32 flags)302gen8_emit_ggtt_write(u32 *cs, u32 value, u32 gtt_offset, u32 flags)
278{303{
@@ -332,13 +357,4 @@ intel_engine_has_preempt_reset(const struct intel_engine_cs *engine) · 53 unmodified lines
332 return intel_engine_has_preemption(engine);357 return intel_engine_has_preemption(engine);
333}358}
334 359
335static inline bool
336intel_engine_has_timeslices(const struct intel_engine_cs *engine)
337{
338 if (!IS_ACTIVE(CONFIG_DRM_I915_TIMESLICE_DURATION))
339 return false;
340
341 return intel_engine_has_semaphores(engine);
342}
343
344#endif /* _INTEL_RINGBUFFER_H_ */360#endif /* _INTEL_RINGBUFFER_H_ */
drivers/gpu/drm/i915/gt/intel_engine_cs.c+2−4
@@ -31,7 +31,6 @@ · 30 unmodified lines
31#include "intel_context.h"31#include "intel_context.h"
32#include "intel_engine.h"32#include "intel_engine.h"
33#include "intel_engine_pm.h"33#include "intel_engine_pm.h"
34#include "intel_engine_pool.h"
35#include "intel_engine_user.h"34#include "intel_engine_user.h"
36#include "intel_gt.h"35#include "intel_gt.h"
37#include "intel_gt_requests.h"36#include "intel_gt_requests.h"
@@ -327,6 +326,8 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id) · 289 unmodified lines
327 if (INTEL_GEN(i915) == 12 && engine->class == RENDER_CLASS)326 if (INTEL_GEN(i915) == 12 && engine->class == RENDER_CLASS)
328 engine->props.preempt_timeout_ms = 0;327 engine->props.preempt_timeout_ms = 0;
329 328
329 engine->defaults = engine->props; /* never to change again */
330
330 engine->context_size = intel_engine_context_size(gt, engine->class);331 engine->context_size = intel_engine_context_size(gt, engine->class);
331 if (WARN_ON(engine->context_size > BIT(20)))332 if (WARN_ON(engine->context_size > BIT(20)))
332 engine->context_size = 0;333 engine->context_size = 0;
@@ -631,8 +632,6 @@ static int engine_setup_common(struct intel_engine_cs *engine) · 298 unmodified lines
631 intel_engine_init__pm(engine);632 intel_engine_init__pm(engine);
632 intel_engine_init_retire(engine);633 intel_engine_init_retire(engine);
633 634
634 intel_engine_pool_init(&engine->pool);
635
636 /* Use the whole device by default */635 /* Use the whole device by default */
637 engine->sseu =636 engine->sseu =
638 intel_sseu_from_device_info(&RUNTIME_INFO(engine->i915)->sseu);637 intel_sseu_from_device_info(&RUNTIME_INFO(engine->i915)->sseu);
@@ -829,7 +828,6 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine) · 190 unmodified lines
829 cleanup_status_page(engine);828 cleanup_status_page(engine);
830 829
831 intel_engine_fini_retire(engine);830 intel_engine_fini_retire(engine);
832 intel_engine_pool_fini(&engine->pool);
833 intel_engine_fini_breadcrumbs(engine);831 intel_engine_fini_breadcrumbs(engine);
834 intel_engine_cleanup_cmd_parser(engine);832 intel_engine_cleanup_cmd_parser(engine);
835 833
drivers/gpu/drm/i915/gt/intel_engine_pm.c+1−15
@@ -10,7 +10,6 @@ · 9 unmodified lines
10#include "intel_engine.h"10#include "intel_engine.h"
11#include "intel_engine_heartbeat.h"11#include "intel_engine_heartbeat.h"
12#include "intel_engine_pm.h"12#include "intel_engine_pm.h"
13#include "intel_engine_pool.h"
14#include "intel_gt.h"13#include "intel_gt.h"
15#include "intel_gt_pm.h"14#include "intel_gt_pm.h"
16#include "intel_rc6.h"15#include "intel_rc6.h"
@@ -22,18 +21,11 @@ static int __engine_unpark(struct intel_wakeref *wf) · 5 unmodified lines
22 struct intel_engine_cs *engine =21 struct intel_engine_cs *engine =
23 container_of(wf, typeof(*engine), wakeref);22 container_of(wf, typeof(*engine), wakeref);
24 struct intel_context *ce;23 struct intel_context *ce;
25 void *map;
26 24
27 ENGINE_TRACE(engine, "\n");25 ENGINE_TRACE(engine, "\n");
28 26
29 intel_gt_pm_get(engine->gt);27 intel_gt_pm_get(engine->gt);
30 28
31 /* Pin the default state for fast resets from atomic context. */
32 map = NULL;
33 if (engine->default_state)
34 map = shmem_pin_map(engine->default_state);
35 engine->pinned_default_state = map;
36
37 /* Discard stale context state from across idling */29 /* Discard stale context state from across idling */
38 ce = engine->kernel_context;30 ce = engine->kernel_context;
39 if (ce) {31 if (ce) {
@@ -43,6 +35,7 @@ static int __engine_unpark(struct intel_wakeref *wf) · 3 unmodified lines
43 if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) && ce->state) {35 if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM) && ce->state) {
44 struct drm_i915_gem_object *obj = ce->state->obj;36 struct drm_i915_gem_object *obj = ce->state->obj;
45 int type = i915_coherent_map_type(engine->i915);37 int type = i915_coherent_map_type(engine->i915);
38 void *map;
46 39
47 map = i915_gem_object_pin_map(obj, type);40 map = i915_gem_object_pin_map(obj, type);
48 if (!IS_ERR(map)) {41 if (!IS_ERR(map)) {
@@ -254,7 +247,6 @@ static int __engine_park(struct intel_wakeref *wf) · 205 unmodified lines
254 247
255 intel_engine_park_heartbeat(engine);248 intel_engine_park_heartbeat(engine);
256 intel_engine_disarm_breadcrumbs(engine);249 intel_engine_disarm_breadcrumbs(engine);
257 intel_engine_pool_park(&engine->pool);
258 250
259 /* Must be reset upon idling, or we may miss the busy wakeup. */251 /* Must be reset upon idling, or we may miss the busy wakeup. */
260 GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);252 GEM_BUG_ON(engine->execlists.queue_priority_hint != INT_MIN);
@@ -262,12 +254,6 @@ static int __engine_park(struct intel_wakeref *wf) · 1 unmodified line
262 if (engine->park)254 if (engine->park)
263 engine->park(engine);255 engine->park(engine);
264 256
265 if (engine->pinned_default_state) {
266 shmem_unpin_map(engine->default_state,
267 engine->pinned_default_state);
268 engine->pinned_default_state = NULL;
269 }
270
271 engine->execlists.no_priolist = false;257 engine->execlists.no_priolist = false;
272 258
273 /* While gt calls i915_vma_parked(), we have to break the lock cycle */259 /* While gt calls i915_vma_parked(), we have to break the lock cycle */
drivers/gpu/drm/i915/gt/intel_engine_pool.h+0−34
@@ -1,34 +0,0 @@
1/*
2 * SPDX-License-Identifier: MIT
3 *
4 * Copyright © 2014-2018 Intel Corporation
5 */
6
7#ifndef INTEL_ENGINE_POOL_H
8#define INTEL_ENGINE_POOL_H
9
10#include "intel_engine_pool_types.h"
11#include "i915_active.h"
12#include "i915_request.h"
13
14struct intel_engine_pool_node *
15intel_engine_get_pool(struct intel_engine_cs *engine, size_t size);
16
17static inline int
18intel_engine_pool_mark_active(struct intel_engine_pool_node *node,
19 struct i915_request *rq)
20{
21 return i915_active_add_request(&node->active, rq);
22}
23
24static inline void
25intel_engine_pool_put(struct intel_engine_pool_node *node)
26{
27 i915_active_release(&node->active);
28}
29
30void intel_engine_pool_init(struct intel_engine_pool *pool);
31void intel_engine_pool_park(struct intel_engine_pool *pool);
32void intel_engine_pool_fini(struct intel_engine_pool *pool);
33
34#endif /* INTEL_ENGINE_POOL_H */
drivers/gpu/drm/i915/gt/intel_engine_types.h+22−14
@@ -22,7 +22,6 @@ · 21 unmodified lines
22#include "i915_pmu.h"22#include "i915_pmu.h"
23#include "i915_priolist_types.h"23#include "i915_priolist_types.h"
24#include "i915_selftest.h"24#include "i915_selftest.h"
25#include "intel_engine_pool_types.h"
26#include "intel_sseu.h"25#include "intel_sseu.h"
27#include "intel_timeline_types.h"26#include "intel_timeline_types.h"
28#include "intel_wakeref.h"27#include "intel_wakeref.h"
@@ -180,6 +179,11 @@ struct intel_engine_execlists { · 151 unmodified lines
180 */179 */
181 u32 error_interrupt;180 u32 error_interrupt;
182 181
182 /**
183 * @reset_ccid: Active CCID [EXECLISTS_STATUS_HI] at the time of reset
184 */
185 u32 reset_ccid;
186
183 /**187 /**
184 * @no_priolist: priority lists disabled188 * @no_priolist: priority lists disabled
185 */189 */
@@ -340,7 +344,6 @@ struct intel_engine_cs { · 154 unmodified lines
340 unsigned long wakeref_serial;344 unsigned long wakeref_serial;
341 struct intel_wakeref wakeref;345 struct intel_wakeref wakeref;
342 struct file *default_state;346 struct file *default_state;
343 void *pinned_default_state;
344 347
345 struct {348 struct {
346 struct intel_ring *ring;349 struct intel_ring *ring;
@@ -374,6 +377,8 @@ struct intel_engine_cs { · 27 unmodified lines
374 spinlock_t irq_lock;377 spinlock_t irq_lock;
375 struct list_head signalers;378 struct list_head signalers;
376 379
380 struct list_head signaled_requests;
381
377 struct irq_work irq_work; /* for use from inside irq_lock */382 struct irq_work irq_work; /* for use from inside irq_lock */
378 383
379 unsigned int irq_enabled;384 unsigned int irq_enabled;
@@ -405,13 +410,6 @@ struct intel_engine_cs { · 25 unmodified lines
405 struct i915_pmu_sample sample[I915_ENGINE_SAMPLE_COUNT];410 struct i915_pmu_sample sample[I915_ENGINE_SAMPLE_COUNT];
406 } pmu;411 } pmu;
407 412
408 /*
409 * A pool of objects to use as shadow copies of client batch buffers
410 * when the command parser is enabled. Prevents the client from
411 * modifying the batch contents after software parsing.
412 */
413 struct intel_engine_pool pool;
414
415 struct intel_hw_status_page status_page;413 struct intel_hw_status_page status_page;
416 struct i915_ctx_workarounds wa_ctx;414 struct i915_ctx_workarounds wa_ctx;
417 struct i915_wa_list ctx_wa_list;415 struct i915_wa_list ctx_wa_list;
@@ -500,10 +498,11 @@ struct intel_engine_cs { · 82 unmodified lines
500#define I915_ENGINE_SUPPORTS_STATS BIT(1)498#define I915_ENGINE_SUPPORTS_STATS BIT(1)
501#define I915_ENGINE_HAS_PREEMPTION BIT(2)499#define I915_ENGINE_HAS_PREEMPTION BIT(2)
502#define I915_ENGINE_HAS_SEMAPHORES BIT(3)500#define I915_ENGINE_HAS_SEMAPHORES BIT(3)
503#define I915_ENGINE_NEEDS_BREADCRUMB_TASKLET BIT(4)501#define I915_ENGINE_HAS_TIMESLICES BIT(4)
504#define I915_ENGINE_IS_VIRTUAL BIT(5)502#define I915_ENGINE_NEEDS_BREADCRUMB_TASKLET BIT(5)
505#define I915_ENGINE_HAS_RELATIVE_MMIO BIT(6)503#define I915_ENGINE_IS_VIRTUAL BIT(6)
506#define I915_ENGINE_REQUIRES_CMD_PARSER BIT(7)504#define I915_ENGINE_HAS_RELATIVE_MMIO BIT(7)
505#define I915_ENGINE_REQUIRES_CMD_PARSER BIT(8)
507 unsigned int flags;506 unsigned int flags;
508 507
509 /*508 /*
@@ -568,7 +567,7 @@ struct intel_engine_cs { · 58 unmodified lines
568 unsigned long preempt_timeout_ms;567 unsigned long preempt_timeout_ms;
569 unsigned long stop_timeout_ms;568 unsigned long stop_timeout_ms;
570 unsigned long timeslice_duration_ms;569 unsigned long timeslice_duration_ms;
571 } props;570 } props, defaults;
572};571};
573 572
574static inline bool573static inline bool
@@ -601,6 +600,15 @@ intel_engine_has_semaphores(const struct intel_engine_cs *engine) · 26 unmodified lines
601 return engine->flags & I915_ENGINE_HAS_SEMAPHORES;600 return engine->flags & I915_ENGINE_HAS_SEMAPHORES;
602}601}
603 602
603static inline bool
604intel_engine_has_timeslices(const struct intel_engine_cs *engine)
605{
606 if (!IS_ACTIVE(CONFIG_DRM_I915_TIMESLICE_DURATION))
607 return false;
608
609 return engine->flags & I915_ENGINE_HAS_TIMESLICES;
610}
611
604static inline bool612static inline bool
605intel_engine_needs_breadcrumb_tasklet(const struct intel_engine_cs *engine)613intel_engine_needs_breadcrumb_tasklet(const struct intel_engine_cs *engine)
606{614{
drivers/gpu/drm/i915/gt/intel_gpu_commands.h+1−2
@@ -236,9 +236,8 @@ · 235 unmodified lines
236#define PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH (1<<12) /* gen6+ */236#define PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH (1<<12) /* gen6+ */
237#define PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE (1<<11) /* MBZ on ILK */237#define PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE (1<<11) /* MBZ on ILK */
238#define PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE (1<<10) /* GM45+ only */238#define PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE (1<<10) /* GM45+ only */
239#define PIPE_CONTROL_L3_RO_CACHE_INVALIDATE REG_BIT(10) /* gen12 */
240#define PIPE_CONTROL_INDIRECT_STATE_DISABLE (1<<9)239#define PIPE_CONTROL_INDIRECT_STATE_DISABLE (1<<9)
241#define PIPE_CONTROL_HDC_PIPELINE_FLUSH REG_BIT(9) /* gen12 */240#define PIPE_CONTROL0_HDC_PIPELINE_FLUSH REG_BIT(9) /* gen12 */
242#define PIPE_CONTROL_NOTIFY (1<<8)241#define PIPE_CONTROL_NOTIFY (1<<8)
243#define PIPE_CONTROL_FLUSH_ENABLE (1<<7) /* gen7+ */242#define PIPE_CONTROL_FLUSH_ENABLE (1<<7) /* gen7+ */
244#define PIPE_CONTROL_DC_FLUSH_ENABLE (1<<5)243#define PIPE_CONTROL_DC_FLUSH_ENABLE (1<<5)
drivers/gpu/drm/i915/gt/intel_gt.c+3−0
@@ -7,6 +7,7 @@ · 6 unmodified lines
7#include "i915_drv.h"7#include "i915_drv.h"
8#include "intel_context.h"8#include "intel_context.h"
9#include "intel_gt.h"9#include "intel_gt.h"
10#include "intel_gt_buffer_pool.h"
10#include "intel_gt_clock_utils.h"11#include "intel_gt_clock_utils.h"
11#include "intel_gt_pm.h"12#include "intel_gt_pm.h"
12#include "intel_gt_requests.h"13#include "intel_gt_requests.h"
@@ -28,6 +29,7 @@ void intel_gt_init_early(struct intel_gt *gt, struct drm_i915_private *i915) · 15 unmodified lines
28 INIT_LIST_HEAD(&gt->closed_vma);29 INIT_LIST_HEAD(&gt->closed_vma);
29 spin_lock_init(&gt->closed_lock);30 spin_lock_init(&gt->closed_lock);
30 31
32 intel_gt_init_buffer_pool(gt);
31 intel_gt_init_reset(gt);33 intel_gt_init_reset(gt);
32 intel_gt_init_requests(gt);34 intel_gt_init_requests(gt);
33 intel_gt_init_timelines(gt);35 intel_gt_init_timelines(gt);
@@ -621,6 +623,7 @@ void intel_gt_driver_release(struct intel_gt *gt) · 587 unmodified lines
621 623
622 intel_gt_pm_fini(gt);624 intel_gt_pm_fini(gt);
623 intel_gt_fini_scratch(gt);625 intel_gt_fini_scratch(gt);
626 intel_gt_fini_buffer_pool(gt);
624}627}
625 628
626void intel_gt_driver_late_release(struct intel_gt *gt)629void intel_gt_driver_late_release(struct intel_gt *gt)
drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.crenamed from drivers/gpu/drm/i915/gt/intel_engine_pool.c+76−38
@@ -1,6 +1,5 @@
1// SPDX-License-Identifier: MIT
1/*2/*
2 * SPDX-License-Identifier: MIT
3 *
4 * Copyright © 2014-2018 Intel Corporation3 * Copyright © 2014-2018 Intel Corporation
5 */4 */
6 5
@@ -8,15 +7,15 @@ · 1 unmodified line
8 7
9#include "i915_drv.h"8#include "i915_drv.h"
10#include "intel_engine_pm.h"9#include "intel_engine_pm.h"
11#include "intel_engine_pool.h"10#include "intel_gt_buffer_pool.h"
12 11
13static struct intel_engine_cs *to_engine(struct intel_engine_pool *pool)12static struct intel_gt *to_gt(struct intel_gt_buffer_pool *pool)
14{13{
15 return container_of(pool, struct intel_engine_cs, pool);14 return container_of(pool, struct intel_gt, buffer_pool);
16}15}
17 16
18static struct list_head *17static struct list_head *
19bucket_for_size(struct intel_engine_pool *pool, size_t sz)18bucket_for_size(struct intel_gt_buffer_pool *pool, size_t sz)
20{19{
21 int n;20 int n;
22 21
@@ -32,16 +31,50 @@ bucket_for_size(struct intel_engine_pool *pool, size_t sz) · 9 unmodified lines
32 return &pool->cache_list[n];31 return &pool->cache_list[n];
33}32}
34 33
35static void node_free(struct intel_engine_pool_node *node)34static void node_free(struct intel_gt_buffer_pool_node *node)
36{35{
37 i915_gem_object_put(node->obj);36 i915_gem_object_put(node->obj);
38 i915_active_fini(&node->active);37 i915_active_fini(&node->active);
39 kfree(node);38 kfree(node);
40}39}
41 40
41static void pool_free_work(struct work_struct *wrk)
42{
43 struct intel_gt_buffer_pool *pool =
44 container_of(wrk, typeof(*pool), work.work);
45 struct intel_gt_buffer_pool_node *node, *next;
46 unsigned long old = jiffies - HZ;
47 bool active = false;
48 LIST_HEAD(stale);
49 int n;
50
51 /* Free buffers that have not been used in the past second */
52 spin_lock_irq(&pool->lock);
53 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++) {
54 struct list_head *list = &pool->cache_list[n];
55
56 /* Most recent at head; oldest at tail */
57 list_for_each_entry_safe_reverse(node, next, list, link) {
58 if (time_before(node->age, old))
59 break;
60
61 list_move(&node->link, &stale);
62 }
63 active |= !list_empty(list);
64 }
65 spin_unlock_irq(&pool->lock);
66
67 list_for_each_entry_safe(node, next, &stale, link)
68 node_free(node);
69
70 if (active)
71 schedule_delayed_work(&pool->work,
72 round_jiffies_up_relative(HZ));
73}
74
42static int pool_active(struct i915_active *ref)75static int pool_active(struct i915_active *ref)
43{76{
44 struct intel_engine_pool_node *node =77 struct intel_gt_buffer_pool_node *node =
45 container_of(ref, typeof(*node), active);78 container_of(ref, typeof(*node), active);
46 struct dma_resv *resv = node->obj->base.resv;79 struct dma_resv *resv = node->obj->base.resv;
47 int err;80 int err;
@@ -64,29 +97,31 @@ static int pool_active(struct i915_active *ref) · 16 unmodified lines
64__i915_active_call97__i915_active_call
65static void pool_retire(struct i915_active *ref)98static void pool_retire(struct i915_active *ref)
66{99{
67 struct intel_engine_pool_node *node =100 struct intel_gt_buffer_pool_node *node =
68 container_of(ref, typeof(*node), active);101 container_of(ref, typeof(*node), active);
69 struct intel_engine_pool *pool = node->pool;102 struct intel_gt_buffer_pool *pool = node->pool;
70 struct list_head *list = bucket_for_size(pool, node->obj->base.size);103 struct list_head *list = bucket_for_size(pool, node->obj->base.size);
71 unsigned long flags;104 unsigned long flags;
72 105
73 GEM_BUG_ON(!intel_engine_pm_is_awake(to_engine(pool)));
74
75 i915_gem_object_unpin_pages(node->obj);106 i915_gem_object_unpin_pages(node->obj);
76 107
77 /* Return this object to the shrinker pool */108 /* Return this object to the shrinker pool */
78 i915_gem_object_make_purgeable(node->obj);109 i915_gem_object_make_purgeable(node->obj);
79 110
80 spin_lock_irqsave(&pool->lock, flags);111 spin_lock_irqsave(&pool->lock, flags);
112 node->age = jiffies;
81 list_add(&node->link, list);113 list_add(&node->link, list);
82 spin_unlock_irqrestore(&pool->lock, flags);114 spin_unlock_irqrestore(&pool->lock, flags);
115
116 schedule_delayed_work(&pool->work,
117 round_jiffies_up_relative(HZ));
83}118}
84 119
85static struct intel_engine_pool_node *120static struct intel_gt_buffer_pool_node *
86node_create(struct intel_engine_pool *pool, size_t sz)121node_create(struct intel_gt_buffer_pool *pool, size_t sz)
87{122{
88 struct intel_engine_cs *engine = to_engine(pool);123 struct intel_gt *gt = to_gt(pool);
89 struct intel_engine_pool_node *node;124 struct intel_gt_buffer_pool_node *node;
90 struct drm_i915_gem_object *obj;125 struct drm_i915_gem_object *obj;
91 126
92 node = kmalloc(sizeof(*node),127 node = kmalloc(sizeof(*node),
@@ -97,7 +132,7 @@ node_create(struct intel_engine_pool *pool, size_t sz) · 4 unmodified lines
97 node->pool = pool;132 node->pool = pool;
98 i915_active_init(&node->active, pool_active, pool_retire);133 i915_active_init(&node->active, pool_active, pool_retire);
99 134
100 obj = i915_gem_object_create_internal(engine->i915, sz);135 obj = i915_gem_object_create_internal(gt->i915, sz);
101 if (IS_ERR(obj)) {136 if (IS_ERR(obj)) {
102 i915_active_fini(&node->active);137 i915_active_fini(&node->active);
103 kfree(node);138 kfree(node);
@@ -110,26 +145,15 @@ node_create(struct intel_engine_pool *pool, size_t sz) · 6 unmodified lines
110 return node;145 return node;
111}146}
112 147
113static struct intel_engine_pool *lookup_pool(struct intel_engine_cs *engine)148struct intel_gt_buffer_pool_node *
149intel_gt_get_buffer_pool(struct intel_gt *gt, size_t size)
114{150{
115 if (intel_engine_is_virtual(engine))151 struct intel_gt_buffer_pool *pool = &gt->buffer_pool;
116 engine = intel_virtual_engine_get_sibling(engine, 0);152 struct intel_gt_buffer_pool_node *node;
117
118 GEM_BUG_ON(!engine);
119 return &engine->pool;
120}
121
122struct intel_engine_pool_node *
123intel_engine_get_pool(struct intel_engine_cs *engine, size_t size)
124{
125 struct intel_engine_pool *pool = lookup_pool(engine);
126 struct intel_engine_pool_node *node;
127 struct list_head *list;153 struct list_head *list;
128 unsigned long flags;154 unsigned long flags;
129 int ret;155 int ret;
130 156
131 GEM_BUG_ON(!intel_engine_pm_is_awake(to_engine(pool)));
132
133 size = PAGE_ALIGN(size);157 size = PAGE_ALIGN(size);
134 list = bucket_for_size(pool, size);158 list = bucket_for_size(pool, size);
135 159
@@ -157,34 +181,48 @@ intel_engine_get_pool(struct intel_engine_cs *engine, size_t size) · 21 unmodified lines
157 return node;181 return node;
158}182}
159 183
160void intel_engine_pool_init(struct intel_engine_pool *pool)184void intel_gt_init_buffer_pool(struct intel_gt *gt)
161{185{
186 struct intel_gt_buffer_pool *pool = &gt->buffer_pool;
162 int n;187 int n;
163 188
164 spin_lock_init(&pool->lock);189 spin_lock_init(&pool->lock);
165 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++)190 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++)
166 INIT_LIST_HEAD(&pool->cache_list[n]);191 INIT_LIST_HEAD(&pool->cache_list[n]);
192 INIT_DELAYED_WORK(&pool->work, pool_free_work);
167}193}
168 194
169void intel_engine_pool_park(struct intel_engine_pool *pool)195static void pool_free_imm(struct intel_gt_buffer_pool *pool)
170{196{
171 int n;197 int n;
172 198
199 spin_lock_irq(&pool->lock);
173 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++) {200 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++) {
201 struct intel_gt_buffer_pool_node *node, *next;
174 struct list_head *list = &pool->cache_list[n];202 struct list_head *list = &pool->cache_list[n];
175 struct intel_engine_pool_node *node, *nn;
176 203
177 list_for_each_entry_safe(node, nn, list, link)204 list_for_each_entry_safe(node, next, list, link)
178 node_free(node);205 node_free(node);
179
180 INIT_LIST_HEAD(list);206 INIT_LIST_HEAD(list);
181 }207 }
208 spin_unlock_irq(&pool->lock);
209}
210
211void intel_gt_flush_buffer_pool(struct intel_gt *gt)
212{
213 struct intel_gt_buffer_pool *pool = &gt->buffer_pool;
214
215 if (cancel_delayed_work_sync(&pool->work))
216 pool_free_imm(pool);
182}217}
183 218
184void intel_engine_pool_fini(struct intel_engine_pool *pool)219void intel_gt_fini_buffer_pool(struct intel_gt *gt)
185{220{
221 struct intel_gt_buffer_pool *pool = &gt->buffer_pool;
186 int n;222 int n;
187 223
224 intel_gt_flush_buffer_pool(gt);
225
188 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++)226 for (n = 0; n < ARRAY_SIZE(pool->cache_list); n++)
189 GEM_BUG_ON(!list_empty(&pool->cache_list[n]));227 GEM_BUG_ON(!list_empty(&pool->cache_list[n]));
190}228}
drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h+37−0
@@ -0,0 +1,37 @@
1/* SPDX-License-Identifier: MIT */
2/*
3 * Copyright © 2014-2018 Intel Corporation
4 */
5
6#ifndef INTEL_GT_BUFFER_POOL_H
7#define INTEL_GT_BUFFER_POOL_H
8
9#include <linux/types.h>
10
11#include "i915_active.h"
12#include "intel_gt_buffer_pool_types.h"
13
14struct intel_gt;
15struct i915_request;
16
17struct intel_gt_buffer_pool_node *
18intel_gt_get_buffer_pool(struct intel_gt *gt, size_t size);
19
20static inline int
21intel_gt_buffer_pool_mark_active(struct intel_gt_buffer_pool_node *node,
22 struct i915_request *rq)
23{
24 return i915_active_add_request(&node->active, rq);
25}
26
27static inline void
28intel_gt_buffer_pool_put(struct intel_gt_buffer_pool_node *node)
29{
30 i915_active_release(&node->active);
31}
32
33void intel_gt_init_buffer_pool(struct intel_gt *gt);
34void intel_gt_flush_buffer_pool(struct intel_gt *gt);
35void intel_gt_fini_buffer_pool(struct intel_gt *gt);
36
37#endif /* INTEL_GT_BUFFER_POOL_H */
drivers/gpu/drm/i915/gt/intel_gt_buffer_pool_types.hrenamed from drivers/gpu/drm/i915/gt/intel_engine_pool_types.h+9−6
@@ -4,26 +4,29 @@ · 3 unmodified lines
4 * Copyright © 2014-2018 Intel Corporation4 * Copyright © 2014-2018 Intel Corporation
5 */5 */
6 6
7#ifndef INTEL_ENGINE_POOL_TYPES_H7#ifndef INTEL_GT_BUFFER_POOL_TYPES_H
8#define INTEL_ENGINE_POOL_TYPES_H8#define INTEL_GT_BUFFER_POOL_TYPES_H
9 9
10#include <linux/list.h>10#include <linux/list.h>
11#include <linux/spinlock.h>11#include <linux/spinlock.h>
12#include <linux/workqueue.h>
12 13
13#include "i915_active_types.h"14#include "i915_active_types.h"
14 15
15struct drm_i915_gem_object;16struct drm_i915_gem_object;
16 17
17struct intel_engine_pool {18struct intel_gt_buffer_pool {
18 spinlock_t lock;19 spinlock_t lock;
19 struct list_head cache_list[4];20 struct list_head cache_list[4];
21 struct delayed_work work;
20};22};
21 23
22struct intel_engine_pool_node {24struct intel_gt_buffer_pool_node {
23 struct i915_active active;25 struct i915_active active;
24 struct drm_i915_gem_object *obj;26 struct drm_i915_gem_object *obj;
25 struct list_head link;27 struct list_head link;
26 struct intel_engine_pool *pool;28 struct intel_gt_buffer_pool *pool;
29 unsigned long age;
27};30};
28 31
29#endif /* INTEL_ENGINE_POOL_TYPES_H */32#endif /* INTEL_GT_BUFFER_POOL_TYPES_H */
drivers/gpu/drm/i915/gt/intel_gt_pm.c+6−4
@@ -150,10 +150,6 @@ static void gt_sanitize(struct intel_gt *gt, bool force) · 149 unmodified lines
150 if (intel_gt_is_wedged(gt))150 if (intel_gt_is_wedged(gt))
151 intel_gt_unset_wedged(gt);151 intel_gt_unset_wedged(gt);
152 152
153 for_each_engine(engine, gt, id)
154 if (engine->sanitize)
155 engine->sanitize(engine);
156
157 intel_uc_sanitize(&gt->uc);153 intel_uc_sanitize(&gt->uc);
158 154
159 for_each_engine(engine, gt, id)155 for_each_engine(engine, gt, id)
@@ -162,6 +158,10 @@ static void gt_sanitize(struct intel_gt *gt, bool force) · 2 unmodified lines
162 158
163 intel_uc_reset_prepare(&gt->uc);159 intel_uc_reset_prepare(&gt->uc);
164 160
161 for_each_engine(engine, gt, id)
162 if (engine->sanitize)
163 engine->sanitize(engine);
164
165 if (reset_engines(gt) || force) {165 if (reset_engines(gt) || force) {
166 for_each_engine(engine, gt, id)166 for_each_engine(engine, gt, id)
167 __intel_engine_reset(engine, false);167 __intel_engine_reset(engine, false);
@@ -171,6 +171,8 @@ static void gt_sanitize(struct intel_gt *gt, bool force) · 3 unmodified lines
171 if (engine->reset.finish)171 if (engine->reset.finish)
172 engine->reset.finish(engine);172 engine->reset.finish(engine);
173 173
174 intel_rps_sanitize(&gt->rps);
175
174 intel_uncore_forcewake_put(gt->uncore, FORCEWAKE_ALL);176 intel_uncore_forcewake_put(gt->uncore, FORCEWAKE_ALL);
175 intel_runtime_pm_put(gt->uncore->rpm, wakeref);177 intel_runtime_pm_put(gt->uncore->rpm, wakeref);
176}178}
drivers/gpu/drm/i915/gt/intel_gt_types.h+11−0
@@ -17,6 +17,7 @@ · 16 unmodified lines
17 17
18#include "i915_vma.h"18#include "i915_vma.h"
19#include "intel_engine_types.h"19#include "intel_engine_types.h"
20#include "intel_gt_buffer_pool_types.h"
20#include "intel_llc_types.h"21#include "intel_llc_types.h"
21#include "intel_reset_types.h"22#include "intel_reset_types.h"
22#include "intel_rc6_types.h"23#include "intel_rc6_types.h"
@@ -97,6 +98,16 @@ struct intel_gt { · 74 unmodified lines
97 */98 */
98 struct i915_address_space *vm;99 struct i915_address_space *vm;
99 100
101 /*
102 * A pool of objects to use as shadow copies of client batch buffers
103 * when the command parser is enabled. Prevents the client from
104 * modifying the batch contents after software parsing.
105 *
106 * Buffers older than 1s are periodically reaped from the pool,
107 * or may be reclaimed by the shrinker before then.
108 */
109 struct intel_gt_buffer_pool buffer_pool;
110
100 struct i915_vma *scratch;111 struct i915_vma *scratch;
101};112};
102 113
drivers/gpu/drm/i915/gt/intel_lrc.c+224−96
@@ -217,7 +217,7 @@ struct virtual_engine { · 216 unmodified lines
217 217
218 /* And finally, which physical engines this virtual engine maps onto. */218 /* And finally, which physical engines this virtual engine maps onto. */
219 unsigned int num_siblings;219 unsigned int num_siblings;
220 struct intel_engine_cs *siblings[0];220 struct intel_engine_cs *siblings[];
221};221};
222 222
223static struct virtual_engine *to_virtual_engine(struct intel_engine_cs *engine)223static struct virtual_engine *to_virtual_engine(struct intel_engine_cs *engine)
@@ -429,18 +429,7 @@ static int effective_prio(const struct i915_request *rq) · 205 unmodified lines
429 if (i915_request_has_nopreempt(rq))429 if (i915_request_has_nopreempt(rq))
430 prio = I915_PRIORITY_UNPREEMPTABLE;430 prio = I915_PRIORITY_UNPREEMPTABLE;
431 431
432 /*432 return prio;
433 * On unwinding the active request, we give it a priority bump
434 * if it has completed waiting on any semaphore. If we know that
435 * the request has already started, we can prevent an unwanted
436 * preempt-to-idle cycle by taking that into account now.
437 */
438 if (__i915_request_has_started(rq))
439 prio |= I915_PRIORITY_NOSEMAPHORE;
440
441 /* Restrict mere WAIT boosts from triggering preemption */
442 BUILD_BUG_ON(__NO_PREEMPTION & ~I915_PRIORITY_MASK); /* only internal */
443 return prio | __NO_PREEMPTION;
444}433}
445 434
446static int queue_prio(const struct intel_engine_execlists *execlists)435static int queue_prio(const struct intel_engine_execlists *execlists)
@@ -1271,14 +1260,11 @@ execlists_check_context(const struct intel_context *ce, · 824 unmodified lines
1271static void restore_default_state(struct intel_context *ce,1260static void restore_default_state(struct intel_context *ce,
1272 struct intel_engine_cs *engine)1261 struct intel_engine_cs *engine)
1273{1262{
1274 u32 *regs = ce->lrc_reg_state;1263 u32 *regs;
1275 1264
1276 if (engine->pinned_default_state)1265 regs = memset(ce->lrc_reg_state, 0, engine->context_size - PAGE_SIZE);
1277 memcpy(regs, /* skip restoring the vanilla PPHWSP */1266 execlists_init_reg_state(regs, ce, engine, ce->ring, true);
1278 engine->pinned_default_state + LRC_STATE_OFFSET,
1279 engine->context_size - PAGE_SIZE);
1280 1267
1281 execlists_init_reg_state(regs, ce, engine, ce->ring, false);
1282 ce->runtime.last = intel_context_get_runtime(ce);1268 ce->runtime.last = intel_context_get_runtime(ce);
1283}1269}
1284 1270
@@ -1372,7 +1358,7 @@ __execlists_schedule_in(struct i915_request *rq) · 87 unmodified lines
1372 ce->lrc.ccid = ce->tag;1358 ce->lrc.ccid = ce->tag;
1373 } else {1359 } else {
1374 /* We don't need a strict matching tag, just different values */1360 /* We don't need a strict matching tag, just different values */
1375 unsigned int tag = ffs(engine->context_tag);1361 unsigned int tag = ffs(READ_ONCE(engine->context_tag));
1376 1362
1377 GEM_BUG_ON(tag == 0 || tag >= BITS_PER_LONG);1363 GEM_BUG_ON(tag == 0 || tag >= BITS_PER_LONG);
1378 clear_bit(tag - 1, &engine->context_tag);1364 clear_bit(tag - 1, &engine->context_tag);
@@ -1826,30 +1812,16 @@ static bool virtual_matches(const struct virtual_engine *ve, · 447 unmodified lines
1826 return true;1812 return true;
1827}1813}
1828 1814
1829static void virtual_xfer_breadcrumbs(struct virtual_engine *ve,1815static void virtual_xfer_breadcrumbs(struct virtual_engine *ve)
1830 struct i915_request *rq)
1831{1816{
1832 struct intel_engine_cs *old = ve->siblings[0];1817 /*
1833 1818 * All the outstanding signals on ve->siblings[0] must have
1834 /* All unattached (rq->engine == old) must already be completed */1819 * been completed, just pending the interrupt handler. As those
1835 1820 * signals still refer to the old sibling (via rq->engine), we must
1836 spin_lock(&old->breadcrumbs.irq_lock);1821 * transfer those to the old irq_worker to keep our locking
1837 if (!list_empty(&ve->context.signal_link)) {1822 * consistent.
1838 list_del_init(&ve->context.signal_link);1823 */
1839 1824 intel_engine_transfer_stale_breadcrumbs(ve->siblings[0], &ve->context);
1840 /*
1841 * We cannot acquire the new engine->breadcrumbs.irq_lock
1842 * (as we are holding a breadcrumbs.irq_lock already),
1843 * so attach this request to the signaler on submission.
1844 * The queued irq_work will occur when we finally drop
1845 * the engine->active.lock after dequeue.
1846 */
1847 set_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT, &rq->fence.flags);
1848
1849 /* Also transfer the pending irq_work for the old breadcrumb. */
1850 intel_engine_signal_breadcrumbs(rq->engine);
1851 }
1852 spin_unlock(&old->breadcrumbs.irq_lock);
1853}1825}
1854 1826
1855#define for_each_waiter(p__, rq__) \1827#define for_each_waiter(p__, rq__) \
@@ -1883,12 +1855,16 @@ static void defer_request(struct i915_request *rq, struct list_head * const pl) · 27 unmodified lines
1883 struct i915_request *w =1855 struct i915_request *w =
1884 container_of(p->waiter, typeof(*w), sched);1856 container_of(p->waiter, typeof(*w), sched);
1885 1857
1858 if (p->flags & I915_DEPENDENCY_WEAK)
1859 continue;
1860
1886 /* Leave semaphores spinning on the other engines */1861 /* Leave semaphores spinning on the other engines */
1887 if (w->engine != rq->engine)1862 if (w->engine != rq->engine)
1888 continue;1863 continue;
1889 1864
1890 /* No waiter should start before its signaler */1865 /* No waiter should start before its signaler */
1891 GEM_BUG_ON(i915_request_started(w) &&1866 GEM_BUG_ON(i915_request_has_initial_breadcrumb(w) &&
1867 i915_request_started(w) &&
1892 !i915_request_completed(rq));1868 !i915_request_completed(rq));
1893 1869
1894 GEM_BUG_ON(i915_request_is_active(w));1870 GEM_BUG_ON(i915_request_is_active(w));
@@ -2280,7 +2256,7 @@ static void execlists_dequeue(struct intel_engine_cs *engine) · 385 unmodified lines
2280 engine);2256 engine);
2281 2257
2282 if (!list_empty(&ve->context.signals))2258 if (!list_empty(&ve->context.signals))
2283 virtual_xfer_breadcrumbs(ve, rq);2259 virtual_xfer_breadcrumbs(ve);
2284 2260
2285 /*2261 /*
2286 * Move the bound engine to the top of the list2262 * Move the bound engine to the top of the list
@@ -3494,6 +3470,7 @@ static int gen8_emit_init_breadcrumb(struct i915_request *rq) · 1207 unmodified lines
3494{3470{
3495 u32 *cs;3471 u32 *cs;
3496 3472
3473 GEM_BUG_ON(i915_request_has_initial_breadcrumb(rq));
3497 if (!i915_request_timeline(rq)->has_initial_breadcrumb)3474 if (!i915_request_timeline(rq)->has_initial_breadcrumb)
3498 return 0;3475 return 0;
3499 3476
@@ -3520,6 +3497,56 @@ static int gen8_emit_init_breadcrumb(struct i915_request *rq) · 20 unmodified lines
3520 /* Record the updated position of the request's payload */3497 /* Record the updated position of the request's payload */
3521 rq->infix = intel_ring_offset(rq, cs);3498 rq->infix = intel_ring_offset(rq, cs);
3522 3499
3500 __set_bit(I915_FENCE_FLAG_INITIAL_BREADCRUMB, &rq->fence.flags);
3501
3502 return 0;
3503}
3504
3505static int emit_pdps(struct i915_request *rq)
3506{
3507 const struct intel_engine_cs * const engine = rq->engine;
3508 struct i915_ppgtt * const ppgtt = i915_vm_to_ppgtt(rq->context->vm);
3509 int err, i;
3510 u32 *cs;
3511
3512 GEM_BUG_ON(intel_vgpu_active(rq->i915));
3513
3514 /*
3515 * Beware ye of the dragons, this sequence is magic!
3516 *
3517 * Small changes to this sequence can cause anything from
3518 * GPU hangs to forcewake errors and machine lockups!
3519 */
3520
3521 /* Flush any residual operations from the context load */
3522 err = engine->emit_flush(rq, EMIT_FLUSH);
3523 if (err)
3524 return err;
3525
3526 /* Magic required to prevent forcewake errors! */
3527 err = engine->emit_flush(rq, EMIT_INVALIDATE);
3528 if (err)
3529 return err;
3530
3531 cs = intel_ring_begin(rq, 4 * GEN8_3LVL_PDPES + 2);
3532 if (IS_ERR(cs))
3533 return PTR_ERR(cs);
3534
3535 /* Ensure the LRI have landed before we invalidate & continue */
3536 *cs++ = MI_LOAD_REGISTER_IMM(2 * GEN8_3LVL_PDPES) | MI_LRI_FORCE_POSTED;
3537 for (i = GEN8_3LVL_PDPES; i--; ) {
3538 const dma_addr_t pd_daddr = i915_page_dir_dma_addr(ppgtt, i);
3539 u32 base = engine->mmio_base;
3540
3541 *cs++ = i915_mmio_reg_offset(GEN8_RING_PDP_UDW(base, i));
3542 *cs++ = upper_32_bits(pd_daddr);
3543 *cs++ = i915_mmio_reg_offset(GEN8_RING_PDP_LDW(base, i));
3544 *cs++ = lower_32_bits(pd_daddr);
3545 }
3546 *cs++ = MI_NOOP;
3547
3548 intel_ring_advance(rq, cs);
3549
3523 return 0;3550 return 0;
3524}3551}
3525 3552
@@ -3544,6 +3571,12 @@ static int execlists_request_alloc(struct i915_request *request) · 18 unmodified lines
3544 * to cancel/unwind this request now.3571 * to cancel/unwind this request now.
3545 */3572 */
3546 3573
3574 if (!i915_vm_is_4lvl(request->context->vm)) {
3575 ret = emit_pdps(request);
3576 if (ret)
3577 return ret;
3578 }
3579
3547 /* Unconditionally invalidate GPU caches and TLBs. */3580 /* Unconditionally invalidate GPU caches and TLBs. */
3548 ret = request->engine->emit_flush(request, EMIT_INVALIDATE);3581 ret = request->engine->emit_flush(request, EMIT_INVALIDATE);
3549 if (ret)3582 if (ret)
@@ -3885,6 +3918,14 @@ static void reset_csb_pointers(struct intel_engine_cs *engine) · 335 unmodified lines
3885 3918
3886 ring_set_paused(engine, 0);3919 ring_set_paused(engine, 0);
3887 3920
3921 /*
3922 * Sometimes Icelake forgets to reset its pointers on a GPU reset.
3923 * Bludgeon them with a mmio update to be sure.
3924 */
3925 ENGINE_WRITE(engine, RING_CONTEXT_STATUS_PTR,
3926 0xffff << 16 | reset_value << 8 | reset_value);
3927 ENGINE_POSTING_READ(engine, RING_CONTEXT_STATUS_PTR);
3928
3888 /*3929 /*
3889 * After a reset, the HW starts writing into CSB entry [0]. We3930 * After a reset, the HW starts writing into CSB entry [0]. We
3890 * therefore have to set our HEAD pointer back one entry so that3931 * therefore have to set our HEAD pointer back one entry so that
@@ -3898,16 +3939,15 @@ static void reset_csb_pointers(struct intel_engine_cs *engine) · 7 unmodified lines
3898 WRITE_ONCE(*execlists->csb_write, reset_value);3939 WRITE_ONCE(*execlists->csb_write, reset_value);
3899 wmb(); /* Make sure this is visible to HW (paranoia?) */3940 wmb(); /* Make sure this is visible to HW (paranoia?) */
3900 3941
3901 /*3942 invalidate_csb_entries(&execlists->csb_status[0],
3902 * Sometimes Icelake forgets to reset its pointers on a GPU reset.3943 &execlists->csb_status[reset_value]);
3903 * Bludgeon them with a mmio update to be sure.3944
3904 */3945 /* Once more for luck and our trusty paranoia */
3905 ENGINE_WRITE(engine, RING_CONTEXT_STATUS_PTR,3946 ENGINE_WRITE(engine, RING_CONTEXT_STATUS_PTR,
3906 reset_value << 8 | reset_value);3947 0xffff << 16 | reset_value << 8 | reset_value);
3907 ENGINE_POSTING_READ(engine, RING_CONTEXT_STATUS_PTR);3948 ENGINE_POSTING_READ(engine, RING_CONTEXT_STATUS_PTR);
3908 3949
3909 invalidate_csb_entries(&execlists->csb_status[0],3950 GEM_BUG_ON(READ_ONCE(*execlists->csb_write) != reset_value);
3910 &execlists->csb_status[reset_value]);
3911}3951}
3912 3952
3913static void execlists_sanitize(struct intel_engine_cs *engine)3953static void execlists_sanitize(struct intel_engine_cs *engine)
@@ -4074,6 +4114,8 @@ static void execlists_reset_prepare(struct intel_engine_cs *engine) · 160 unmodified lines
4074 */4114 */
4075 ring_set_paused(engine, 1);4115 ring_set_paused(engine, 1);
4076 intel_engine_stop_cs(engine);4116 intel_engine_stop_cs(engine);
4117
4118 engine->execlists.reset_ccid = active_ccid(engine);
4077}4119}
4078 4120
4079static void __reset_stop_ring(u32 *regs, const struct intel_engine_cs *engine)4121static void __reset_stop_ring(u32 *regs, const struct intel_engine_cs *engine)
@@ -4116,7 +4158,7 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 36 unmodified lines
4116 * its request, it was still running at the time of the4158 * its request, it was still running at the time of the
4117 * reset and will have been clobbered.4159 * reset and will have been clobbered.
4118 */4160 */
4119 rq = execlists_active(execlists);4161 rq = active_context(engine, engine->execlists.reset_ccid);
4120 if (!rq)4162 if (!rq)
4121 goto unwind;4163 goto unwind;
4122 4164
@@ -4166,8 +4208,6 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 43 unmodified lines
4166 * image back to the expected values to skip over the guilty request.4208 * image back to the expected values to skip over the guilty request.
4167 */4209 */
4168 __i915_request_reset(rq, stalled);4210 __i915_request_reset(rq, stalled);
4169 if (!stalled)
4170 goto out_replay;
4171 4211
4172 /*4212 /*
4173 * We want a simple context + ring to execute the breadcrumb update.4213 * We want a simple context + ring to execute the breadcrumb update.
@@ -4177,9 +4217,6 @@ static void __execlists_reset(struct intel_engine_cs *engine, bool stalled) · 3 unmodified lines
4177 * future request will be after userspace has had the opportunity4217 * future request will be after userspace has had the opportunity
4178 * to recreate its own state.4218 * to recreate its own state.
4179 */4219 */
4180 GEM_BUG_ON(!intel_context_is_pinned(ce));
4181 restore_default_state(ce, engine);
4182
4183out_replay:4220out_replay:
4184 ENGINE_TRACE(engine, "replay {head:%04x, tail:%04x}\n",4221 ENGINE_TRACE(engine, "replay {head:%04x, tail:%04x}\n",
4185 head, ce->ring->tail);4222 head, ce->ring->tail);
@@ -4545,6 +4582,42 @@ static u32 preparser_disable(bool state) · 359 unmodified lines
4545 return MI_ARB_CHECK | 1 << 8 | state;4582 return MI_ARB_CHECK | 1 << 8 | state;
4546}4583}
4547 4584
4585static i915_reg_t aux_inv_reg(const struct intel_engine_cs *engine)
4586{
4587 static const i915_reg_t vd[] = {
4588 GEN12_VD0_AUX_NV,
4589 GEN12_VD1_AUX_NV,
4590 GEN12_VD2_AUX_NV,
4591 GEN12_VD3_AUX_NV,
4592 };
4593
4594 static const i915_reg_t ve[] = {
4595 GEN12_VE0_AUX_NV,
4596 GEN12_VE1_AUX_NV,
4597 };
4598
4599 if (engine->class == VIDEO_DECODE_CLASS)
4600 return vd[engine->instance];
4601
4602 if (engine->class == VIDEO_ENHANCEMENT_CLASS)
4603 return ve[engine->instance];
4604
4605 GEM_BUG_ON("unknown aux_inv_reg\n");
4606
4607 return INVALID_MMIO_REG;
4608}
4609
4610static u32 *
4611gen12_emit_aux_table_inv(const i915_reg_t inv_reg, u32 *cs)
4612{
4613 *cs++ = MI_LOAD_REGISTER_IMM(1);
4614 *cs++ = i915_mmio_reg_offset(inv_reg);
4615 *cs++ = AUX_INV;
4616 *cs++ = MI_NOOP;
4617
4618 return cs;
4619}
4620
4548static int gen12_emit_flush_render(struct i915_request *request,4621static int gen12_emit_flush_render(struct i915_request *request,
4549 u32 mode)4622 u32 mode)
4550{4623{
@@ -4553,13 +4626,13 @@ static int gen12_emit_flush_render(struct i915_request *request, · 2 unmodified lines
4553 u32 *cs;4626 u32 *cs;
4554 4627
4555 flags |= PIPE_CONTROL_TILE_CACHE_FLUSH;4628 flags |= PIPE_CONTROL_TILE_CACHE_FLUSH;
4629 flags |= PIPE_CONTROL_FLUSH_L3;
4556 flags |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;4630 flags |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
4557 flags |= PIPE_CONTROL_DEPTH_CACHE_FLUSH;4631 flags |= PIPE_CONTROL_DEPTH_CACHE_FLUSH;
4558 /* Wa_1409600907:tgl */4632 /* Wa_1409600907:tgl */
4559 flags |= PIPE_CONTROL_DEPTH_STALL;4633 flags |= PIPE_CONTROL_DEPTH_STALL;
4560 flags |= PIPE_CONTROL_DC_FLUSH_ENABLE;4634 flags |= PIPE_CONTROL_DC_FLUSH_ENABLE;
4561 flags |= PIPE_CONTROL_FLUSH_ENABLE;4635 flags |= PIPE_CONTROL_FLUSH_ENABLE;
4562 flags |= PIPE_CONTROL_HDC_PIPELINE_FLUSH;
4563 4636
4564 flags |= PIPE_CONTROL_STORE_DATA_INDEX;4637 flags |= PIPE_CONTROL_STORE_DATA_INDEX;
4565 flags |= PIPE_CONTROL_QW_WRITE;4638 flags |= PIPE_CONTROL_QW_WRITE;
@@ -4570,7 +4643,9 @@ static int gen12_emit_flush_render(struct i915_request *request, · 4 unmodified lines
4570 if (IS_ERR(cs))4643 if (IS_ERR(cs))
4571 return PTR_ERR(cs);4644 return PTR_ERR(cs);
4572 4645
4573 cs = gen8_emit_pipe_control(cs, flags, LRC_PPHWSP_SCRATCH_ADDR);4646 cs = gen12_emit_pipe_control(cs,
4647 PIPE_CONTROL0_HDC_PIPELINE_FLUSH,
4648 flags, LRC_PPHWSP_SCRATCH_ADDR);
4574 intel_ring_advance(request, cs);4649 intel_ring_advance(request, cs);
4575 }4650 }
4576 4651
@@ -4585,14 +4660,13 @@ static int gen12_emit_flush_render(struct i915_request *request, · 8 unmodified lines
4585 flags |= PIPE_CONTROL_VF_CACHE_INVALIDATE;4660 flags |= PIPE_CONTROL_VF_CACHE_INVALIDATE;
4586 flags |= PIPE_CONTROL_CONST_CACHE_INVALIDATE;4661 flags |= PIPE_CONTROL_CONST_CACHE_INVALIDATE;
4587 flags |= PIPE_CONTROL_STATE_CACHE_INVALIDATE;4662 flags |= PIPE_CONTROL_STATE_CACHE_INVALIDATE;
4588 flags |= PIPE_CONTROL_L3_RO_CACHE_INVALIDATE;
4589 4663
4590 flags |= PIPE_CONTROL_STORE_DATA_INDEX;4664 flags |= PIPE_CONTROL_STORE_DATA_INDEX;
4591 flags |= PIPE_CONTROL_QW_WRITE;4665 flags |= PIPE_CONTROL_QW_WRITE;
4592 4666
4593 flags |= PIPE_CONTROL_CS_STALL;4667 flags |= PIPE_CONTROL_CS_STALL;
4594 4668
4595 cs = intel_ring_begin(request, 8);4669 cs = intel_ring_begin(request, 8 + 4);
4596 if (IS_ERR(cs))4670 if (IS_ERR(cs))
4597 return PTR_ERR(cs);4671 return PTR_ERR(cs);
4598 4672
@@ -4605,6 +4679,9 @@ static int gen12_emit_flush_render(struct i915_request *request, · 6 unmodified lines
4605 4679
4606 cs = gen8_emit_pipe_control(cs, flags, LRC_PPHWSP_SCRATCH_ADDR);4680 cs = gen8_emit_pipe_control(cs, flags, LRC_PPHWSP_SCRATCH_ADDR);
4607 4681
4682 /* hsdes: 1809175790 */
4683 cs = gen12_emit_aux_table_inv(GEN12_GFX_CCS_AUX_NV, cs);
4684
4608 *cs++ = preparser_disable(false);4685 *cs++ = preparser_disable(false);
4609 intel_ring_advance(request, cs);4686 intel_ring_advance(request, cs);
4610 }4687 }
@@ -4612,6 +4689,56 @@ static int gen12_emit_flush_render(struct i915_request *request, · 1 unmodified line
4612 return 0;4689 return 0;
4613}4690}
4614 4691
4692static int gen12_emit_flush(struct i915_request *request, u32 mode)
4693{
4694 intel_engine_mask_t aux_inv = 0;
4695 u32 cmd, *cs;
4696
4697 if (mode & EMIT_INVALIDATE)
4698 aux_inv = request->engine->mask & ~BIT(BCS0);
4699
4700 cs = intel_ring_begin(request,
4701 4 + (aux_inv ? 2 * hweight8(aux_inv) + 2 : 0));
4702 if (IS_ERR(cs))
4703 return PTR_ERR(cs);
4704
4705 cmd = MI_FLUSH_DW + 1;
4706
4707 /* We always require a command barrier so that subsequent
4708 * commands, such as breadcrumb interrupts, are strictly ordered
4709 * wrt the contents of the write cache being flushed to memory
4710 * (and thus being coherent from the CPU).
4711 */
4712 cmd |= MI_FLUSH_DW_STORE_INDEX | MI_FLUSH_DW_OP_STOREDW;
4713
4714 if (mode & EMIT_INVALIDATE) {
4715 cmd |= MI_INVALIDATE_TLB;
4716 if (request->engine->class == VIDEO_DECODE_CLASS)
4717 cmd |= MI_INVALIDATE_BSD;
4718 }
4719
4720 *cs++ = cmd;
4721 *cs++ = LRC_PPHWSP_SCRATCH_ADDR;
4722 *cs++ = 0; /* upper addr */
4723 *cs++ = 0; /* value */
4724
4725 if (aux_inv) { /* hsdes: 1809175790 */
4726 struct intel_engine_cs *engine;
4727 unsigned int tmp;
4728
4729 *cs++ = MI_LOAD_REGISTER_IMM(hweight8(aux_inv));
4730 for_each_engine_masked(engine, request->engine->gt,
4731 aux_inv, tmp) {
4732 *cs++ = i915_mmio_reg_offset(aux_inv_reg(engine));
4733 *cs++ = AUX_INV;
4734 }
4735 *cs++ = MI_NOOP;
4736 }
4737 intel_ring_advance(request, cs);
4738
4739 return 0;
4740}
4741
4615/*4742/*
4616 * Reserve space for 2 NOOPs at the end of each request to be4743 * Reserve space for 2 NOOPs at the end of each request to be
4617 * used as a workaround for not being allowed to do lite4744 * used as a workaround for not being allowed to do lite
@@ -4641,8 +4768,7 @@ static u32 *emit_preempt_busywait(struct i915_request *request, u32 *cs) · 23 unmodified lines
4641}4768}
4642 4769
4643static __always_inline u32*4770static __always_inline u32*
4644gen8_emit_fini_breadcrumb_footer(struct i915_request *request,4771gen8_emit_fini_breadcrumb_tail(struct i915_request *request, u32 *cs)
4645 u32 *cs)
4646{4772{
4647 *cs++ = MI_USER_INTERRUPT;4773 *cs++ = MI_USER_INTERRUPT;
4648 4774
@@ -4656,14 +4782,16 @@ gen8_emit_fini_breadcrumb_footer(struct i915_request *request, · 7 unmodified lines
4656 return gen8_emit_wa_tail(request, cs);4782 return gen8_emit_wa_tail(request, cs);
4657}4783}
4658 4784
4659static u32 *gen8_emit_fini_breadcrumb(struct i915_request *request, u32 *cs)4785static u32 *emit_xcs_breadcrumb(struct i915_request *request, u32 *cs)
4660{4786{
4661 cs = gen8_emit_ggtt_write(cs,4787 u32 addr = i915_request_active_timeline(request)->hwsp_offset;
4662 request->fence.seqno,4788
4663 i915_request_active_timeline(request)->hwsp_offset,4789 return gen8_emit_ggtt_write(cs, request->fence.seqno, addr, 0);
4664 0);4790}
4665 4791
4666 return gen8_emit_fini_breadcrumb_footer(request, cs);4792static u32 *gen8_emit_fini_breadcrumb(struct i915_request *rq, u32 *cs)
4793{
4794 return gen8_emit_fini_breadcrumb_tail(rq, emit_xcs_breadcrumb(rq, cs));
4667}4795}
4668 4796
4669static u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs)4797static u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs)
@@ -4681,7 +4809,7 @@ static u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs) · 11 unmodified lines
4681 PIPE_CONTROL_FLUSH_ENABLE |4809 PIPE_CONTROL_FLUSH_ENABLE |
4682 PIPE_CONTROL_CS_STALL);4810 PIPE_CONTROL_CS_STALL);
4683 4811
4684 return gen8_emit_fini_breadcrumb_footer(request, cs);4812 return gen8_emit_fini_breadcrumb_tail(request, cs);
4685}4813}
4686 4814
4687static u32 *4815static u32 *
@@ -4697,7 +4825,7 @@ gen11_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs) · 9 unmodified lines
4697 PIPE_CONTROL_DC_FLUSH_ENABLE |4825 PIPE_CONTROL_DC_FLUSH_ENABLE |
4698 PIPE_CONTROL_FLUSH_ENABLE);4826 PIPE_CONTROL_FLUSH_ENABLE);
4699 4827
4700 return gen8_emit_fini_breadcrumb_footer(request, cs);4828 return gen8_emit_fini_breadcrumb_tail(request, cs);
4701}4829}
4702 4830
4703/*4831/*
@@ -4735,7 +4863,7 @@ static u32 *gen12_emit_preempt_busywait(struct i915_request *request, u32 *cs) · 31 unmodified lines
4735}4863}
4736 4864
4737static __always_inline u32*4865static __always_inline u32*
4738gen12_emit_fini_breadcrumb_footer(struct i915_request *request, u32 *cs)4866gen12_emit_fini_breadcrumb_tail(struct i915_request *request, u32 *cs)
4739{4867{
4740 *cs++ = MI_USER_INTERRUPT;4868 *cs++ = MI_USER_INTERRUPT;
4741 4869
@@ -4749,33 +4877,29 @@ gen12_emit_fini_breadcrumb_footer(struct i915_request *request, u32 *cs) · 7 unmodified lines
4749 return gen8_emit_wa_tail(request, cs);4877 return gen8_emit_wa_tail(request, cs);
4750}4878}
4751 4879
4752static u32 *gen12_emit_fini_breadcrumb(struct i915_request *request, u32 *cs)4880static u32 *gen12_emit_fini_breadcrumb(struct i915_request *rq, u32 *cs)
4753{4881{
4754 cs = gen8_emit_ggtt_write(cs,4882 return gen12_emit_fini_breadcrumb_tail(rq, emit_xcs_breadcrumb(rq, cs));
4755 request->fence.seqno,
4756 i915_request_active_timeline(request)->hwsp_offset,
4757 0);
4758
4759 return gen12_emit_fini_breadcrumb_footer(request, cs);
4760}4883}
4761 4884
4762static u32 *4885static u32 *
4763gen12_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs)4886gen12_emit_fini_breadcrumb_rcs(struct i915_request *request, u32 *cs)
4764{4887{
4765 cs = gen8_emit_ggtt_write_rcs(cs,4888 cs = gen12_emit_ggtt_write_rcs(cs,
4766 request->fence.seqno,4889 request->fence.seqno,
4767 i915_request_active_timeline(request)->hwsp_offset,4890 i915_request_active_timeline(request)->hwsp_offset,
4768 PIPE_CONTROL_CS_STALL |4891 PIPE_CONTROL0_HDC_PIPELINE_FLUSH,
4769 PIPE_CONTROL_TILE_CACHE_FLUSH |4892 PIPE_CONTROL_CS_STALL |
4770 PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |4893 PIPE_CONTROL_TILE_CACHE_FLUSH |
4771 PIPE_CONTROL_DEPTH_CACHE_FLUSH |4894 PIPE_CONTROL_FLUSH_L3 |
4772 /* Wa_1409600907:tgl */4895 PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
4773 PIPE_CONTROL_DEPTH_STALL |4896 PIPE_CONTROL_DEPTH_CACHE_FLUSH |
4774 PIPE_CONTROL_DC_FLUSH_ENABLE |4897 /* Wa_1409600907:tgl */
4775 PIPE_CONTROL_FLUSH_ENABLE |4898 PIPE_CONTROL_DEPTH_STALL |
4776 PIPE_CONTROL_HDC_PIPELINE_FLUSH);4899 PIPE_CONTROL_DC_FLUSH_ENABLE |
4900 PIPE_CONTROL_FLUSH_ENABLE);
4777 4901
4778 return gen12_emit_fini_breadcrumb_footer(request, cs);4902 return gen12_emit_fini_breadcrumb_tail(request, cs);
4779}4903}
4780 4904
4781static void execlists_park(struct intel_engine_cs *engine)4905static void execlists_park(struct intel_engine_cs *engine)
@@ -4801,8 +4925,11 @@ void intel_execlists_set_default_submission(struct intel_engine_cs *engine) · 19 unmodified lines
4801 engine->flags |= I915_ENGINE_SUPPORTS_STATS;4925 engine->flags |= I915_ENGINE_SUPPORTS_STATS;
4802 if (!intel_vgpu_active(engine->i915)) {4926 if (!intel_vgpu_active(engine->i915)) {
4803 engine->flags |= I915_ENGINE_HAS_SEMAPHORES;4927 engine->flags |= I915_ENGINE_HAS_SEMAPHORES;
4804 if (HAS_LOGICAL_RING_PREEMPTION(engine->i915))4928 if (HAS_LOGICAL_RING_PREEMPTION(engine->i915)) {
4805 engine->flags |= I915_ENGINE_HAS_PREEMPTION;4929 engine->flags |= I915_ENGINE_HAS_PREEMPTION;
4930 if (IS_ACTIVE(CONFIG_DRM_I915_TIMESLICE_DURATION))
4931 engine->flags |= I915_ENGINE_HAS_TIMESLICES;
4932 }
4806 }4933 }
4807 4934
4808 if (INTEL_GEN(engine->i915) >= 12)4935 if (INTEL_GEN(engine->i915) >= 12)
@@ -4845,9 +4972,10 @@ logical_ring_default_vfuncs(struct intel_engine_cs *engine) · 36 unmodified lines
4845 engine->emit_flush = gen8_emit_flush;4972 engine->emit_flush = gen8_emit_flush;
4846 engine->emit_init_breadcrumb = gen8_emit_init_breadcrumb;4973 engine->emit_init_breadcrumb = gen8_emit_init_breadcrumb;
4847 engine->emit_fini_breadcrumb = gen8_emit_fini_breadcrumb;4974 engine->emit_fini_breadcrumb = gen8_emit_fini_breadcrumb;
4848 if (INTEL_GEN(engine->i915) >= 12)4975 if (INTEL_GEN(engine->i915) >= 12) {
4849 engine->emit_fini_breadcrumb = gen12_emit_fini_breadcrumb;4976 engine->emit_fini_breadcrumb = gen12_emit_fini_breadcrumb;
4850 4977 engine->emit_flush = gen12_emit_flush;
4978 }
4851 engine->set_default_submission = intel_execlists_set_default_submission;4979 engine->set_default_submission = intel_execlists_set_default_submission;
4852 4980
4853 if (INTEL_GEN(engine->i915) < 11) {4981 if (INTEL_GEN(engine->i915) < 11) {
drivers/gpu/drm/i915/gt/intel_rc6.c+1−7
@@ -113,7 +113,6 @@ static void gen9_rc6_enable(struct intel_rc6 *rc6) · 112 unmodified lines
113 struct intel_uncore *uncore = rc6_to_uncore(rc6);113 struct intel_uncore *uncore = rc6_to_uncore(rc6);
114 struct intel_engine_cs *engine;114 struct intel_engine_cs *engine;
115 enum intel_engine_id id;115 enum intel_engine_id id;
116 u32 rc6_mode;
117 116
118 /* 2b: Program RC6 thresholds.*/117 /* 2b: Program RC6 thresholds.*/
119 if (INTEL_GEN(rc6_to_i915(rc6)) >= 10) {118 if (INTEL_GEN(rc6_to_i915(rc6)) >= 10) {
@@ -165,16 +164,11 @@ static void gen9_rc6_enable(struct intel_rc6 *rc6) · 45 unmodified lines
165 /* 3a: Enable RC6 */164 /* 3a: Enable RC6 */
166 set(uncore, GEN6_RC6_THRESHOLD, 37500); /* 37.5/125ms per EI */165 set(uncore, GEN6_RC6_THRESHOLD, 37500); /* 37.5/125ms per EI */
167 166
168 /* WaRsUseTimeoutMode:cnl (pre-prod) */
169 if (IS_CNL_REVID(rc6_to_i915(rc6), CNL_REVID_A0, CNL_REVID_C0))
170 rc6_mode = GEN7_RC_CTL_TO_MODE;
171 else
172 rc6_mode = GEN6_RC_CTL_EI_MODE(1);
173 167
174 rc6->ctl_enable =168 rc6->ctl_enable =
175 GEN6_RC_CTL_HW_ENABLE |169 GEN6_RC_CTL_HW_ENABLE |
176 GEN6_RC_CTL_RC6_ENABLE |170 GEN6_RC_CTL_RC6_ENABLE |
177 rc6_mode;171 GEN6_RC_CTL_EI_MODE(1);
178 172
179 /*173 /*
180 * WaRsDisableCoarsePowerGating:skl,cnl174 * WaRsDisableCoarsePowerGating:skl,cnl
drivers/gpu/drm/i915/gt/intel_renderstate.c+9−7
@@ -219,6 +219,14 @@ int intel_renderstate_emit(struct intel_renderstate *so, · 218 unmodified lines
219 if (!so->vma)219 if (!so->vma)
220 return 0;220 return 0;
221 221
222 i915_vma_lock(so->vma);
223 err = i915_request_await_object(rq, so->vma->obj, false);
224 if (err == 0)
225 err = i915_vma_move_to_active(so->vma, rq, 0);
226 i915_vma_unlock(so->vma);
227 if (err)
228 return err;
229
222 err = engine->emit_bb_start(rq,230 err = engine->emit_bb_start(rq,
223 so->batch_offset, so->batch_size,231 so->batch_offset, so->batch_size,
224 I915_DISPATCH_SECURE);232 I915_DISPATCH_SECURE);
@@ -233,13 +241,7 @@ int intel_renderstate_emit(struct intel_renderstate *so, · 8 unmodified lines
233 return err;241 return err;
234 }242 }
235 243
236 i915_vma_lock(so->vma);244 return 0;
237 err = i915_request_await_object(rq, so->vma->obj, false);
238 if (err == 0)
239 err = i915_vma_move_to_active(so->vma, rq, 0);
240 i915_vma_unlock(so->vma);
241
242 return err;
243}245}
244 246
245void intel_renderstate_fini(struct intel_renderstate *so)247void intel_renderstate_fini(struct intel_renderstate *so)
drivers/gpu/drm/i915/gt/intel_rps.c+4−1
@@ -1844,8 +1844,11 @@ void intel_rps_init(struct intel_rps *rps) · 1843 unmodified lines
1844 1844
1845 if (INTEL_GEN(i915) >= 8 && INTEL_GEN(i915) < 11)1845 if (INTEL_GEN(i915) >= 8 && INTEL_GEN(i915) < 11)
1846 rps->pm_intrmsk_mbz |= GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC;1846 rps->pm_intrmsk_mbz |= GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC;
1847}
1847 1848
1848 if (INTEL_GEN(i915) >= 6)1849void intel_rps_sanitize(struct intel_rps *rps)
1850{
1851 if (INTEL_GEN(rps_to_i915(rps)) >= 6)
1849 rps_disable_interrupts(rps);1852 rps_disable_interrupts(rps);
1850}1853}
1851 1854
drivers/gpu/drm/i915/gt/intel_rps.h+1−0
@@ -13,6 +13,7 @@ struct i915_request; · 12 unmodified lines
13 13
14void intel_rps_init_early(struct intel_rps *rps);14void intel_rps_init_early(struct intel_rps *rps);
15void intel_rps_init(struct intel_rps *rps);15void intel_rps_init(struct intel_rps *rps);
16void intel_rps_sanitize(struct intel_rps *rps);
16 17
17void intel_rps_driver_register(struct intel_rps *rps);18void intel_rps_driver_register(struct intel_rps *rps);
18void intel_rps_driver_unregister(struct intel_rps *rps);19void intel_rps_driver_unregister(struct intel_rps *rps);
drivers/gpu/drm/i915/gt/intel_timeline.c+4−4
@@ -211,9 +211,9 @@ static void cacheline_free(struct intel_timeline_cacheline *cl) · 210 unmodified lines
211 i915_active_release(&cl->active);211 i915_active_release(&cl->active);
212}212}
213 213
214int intel_timeline_init(struct intel_timeline *timeline,214static int intel_timeline_init(struct intel_timeline *timeline,
215 struct intel_gt *gt,215 struct intel_gt *gt,
216 struct i915_vma *hwsp)216 struct i915_vma *hwsp)
217{217{
218 void *vaddr;218 void *vaddr;
219 219
@@ -280,7 +280,7 @@ void intel_gt_init_timelines(struct intel_gt *gt) · 60 unmodified lines
280 INIT_LIST_HEAD(&timelines->hwsp_free_list);280 INIT_LIST_HEAD(&timelines->hwsp_free_list);
281}281}
282 282
283void intel_timeline_fini(struct intel_timeline *timeline)283static void intel_timeline_fini(struct intel_timeline *timeline)
284{284{
285 GEM_BUG_ON(atomic_read(&timeline->pin_count));285 GEM_BUG_ON(atomic_read(&timeline->pin_count));
286 GEM_BUG_ON(!list_empty(&timeline->requests));286 GEM_BUG_ON(!list_empty(&timeline->requests));
drivers/gpu/drm/i915/gt/intel_timeline.h+0−5
@@ -31,11 +31,6 @@ · 30 unmodified lines
31#include "i915_syncmap.h"31#include "i915_syncmap.h"
32#include "gt/intel_timeline_types.h"32#include "gt/intel_timeline_types.h"
33 33
34int intel_timeline_init(struct intel_timeline *tl,
35 struct intel_gt *gt,
36 struct i915_vma *hwsp);
37void intel_timeline_fini(struct intel_timeline *tl);
38
39struct intel_timeline *34struct intel_timeline *
40intel_timeline_create(struct intel_gt *gt, struct i915_vma *global_hwsp);35intel_timeline_create(struct intel_gt *gt, struct i915_vma *global_hwsp);
41 36
drivers/gpu/drm/i915/gt/intel_workarounds.c+7−21
@@ -485,25 +485,14 @@ static void cfl_ctx_workarounds_init(struct intel_engine_cs *engine, · 484 unmodified lines
485static void cnl_ctx_workarounds_init(struct intel_engine_cs *engine,485static void cnl_ctx_workarounds_init(struct intel_engine_cs *engine,
486 struct i915_wa_list *wal)486 struct i915_wa_list *wal)
487{487{
488 struct drm_i915_private *i915 = engine->i915;
489
490 /* WaForceContextSaveRestoreNonCoherent:cnl */488 /* WaForceContextSaveRestoreNonCoherent:cnl */
491 WA_SET_BIT_MASKED(CNL_HDC_CHICKEN0,489 WA_SET_BIT_MASKED(CNL_HDC_CHICKEN0,
492 HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT);490 HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT);
493 491
494 /* WaThrottleEUPerfToAvoidTDBackPressure:cnl(pre-prod) */
495 if (IS_CNL_REVID(i915, CNL_REVID_B0, CNL_REVID_B0))
496 WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, THROTTLE_12_5);
497
498 /* WaDisableReplayBufferBankArbitrationOptimization:cnl */492 /* WaDisableReplayBufferBankArbitrationOptimization:cnl */
499 WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,493 WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
500 GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);494 GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
501 495
502 /* WaDisableEnhancedSBEVertexCaching:cnl (pre-prod) */
503 if (IS_CNL_REVID(i915, 0, CNL_REVID_B0))
504 WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
505 GEN8_CSC2_SBE_VUE_CACHE_CONSERVATIVE);
506
507 /* WaPushConstantDereferenceHoldDisable:cnl */496 /* WaPushConstantDereferenceHoldDisable:cnl */
508 WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2, PUSH_CONSTANT_DEREF_DISABLE);497 WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2, PUSH_CONSTANT_DEREF_DISABLE);
509 498
@@ -872,12 +861,6 @@ cnl_gt_workarounds_init(struct drm_i915_private *i915, struct i915_wa_list *wal) · 362 unmodified lines
872{861{
873 wa_init_mcr(i915, wal);862 wa_init_mcr(i915, wal);
874 863
875 /* WaDisableI2mCycleOnWRPort:cnl (pre-prod) */
876 if (IS_CNL_REVID(i915, CNL_REVID_B0, CNL_REVID_B0))
877 wa_write_or(wal,
878 GAMT_CHKN_BIT_REG,
879 GAMT_CHKN_DISABLE_I2M_CYCLE_ON_WR_PORT);
880
881 /* WaInPlaceDecompressionHang:cnl */864 /* WaInPlaceDecompressionHang:cnl */
882 wa_write_or(wal,865 wa_write_or(wal,
883 GEN9_GAMT_ECO_REG_RW_IA,866 GEN9_GAMT_ECO_REG_RW_IA,
@@ -934,10 +917,13 @@ icl_gt_workarounds_init(struct drm_i915_private *i915, struct i915_wa_list *wal) · 50 unmodified lines
934 GAMT_CHKN_BIT_REG,917 GAMT_CHKN_BIT_REG,
935 GAMT_CHKN_DISABLE_L3_COH_PIPE);918 GAMT_CHKN_DISABLE_L3_COH_PIPE);
936 919
937 /* Wa_1607087056:icl */920 /* Wa_1607087056:icl,ehl,jsl */
938 wa_write_or(wal,921 if (IS_ICELAKE(i915) ||
939 SLICE_UNIT_LEVEL_CLKGATE,922 IS_EHL_REVID(i915, EHL_REVID_A0, EHL_REVID_A0)) {
940 L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);923 wa_write_or(wal,
924 SLICE_UNIT_LEVEL_CLKGATE,
925 L3_CLKGATE_DIS | L3_CR2X_CLKGATE_DIS);
926 }
941}927}
942 928
943static void929static void
drivers/gpu/drm/i915/gt/mock_engine.c+0−2
@@ -28,7 +28,6 @@ · 27 unmodified lines
28#include "i915_drv.h"28#include "i915_drv.h"
29#include "intel_context.h"29#include "intel_context.h"
30#include "intel_engine_pm.h"30#include "intel_engine_pm.h"
31#include "intel_engine_pool.h"
32 31
33#include "mock_engine.h"32#include "mock_engine.h"
34#include "selftests/mock_request.h"33#include "selftests/mock_request.h"
@@ -328,7 +327,6 @@ int mock_engine_init(struct intel_engine_cs *engine) · 293 unmodified lines
328 intel_engine_init_execlists(engine);327 intel_engine_init_execlists(engine);
329 intel_engine_init__pm(engine);328 intel_engine_init__pm(engine);
330 intel_engine_init_retire(engine);329 intel_engine_init_retire(engine);
331 intel_engine_pool_init(&engine->pool);
332 330
333 ce = create_kernel_context(engine);331 ce = create_kernel_context(engine);
334 if (IS_ERR(ce))332 if (IS_ERR(ce))
drivers/gpu/drm/i915/gt/selftest_context.c+4−8
@@ -24,6 +24,7 @@ static int request_sync(struct i915_request *rq) · 23 unmodified lines
24 24
25 /* Opencode i915_request_add() so we can keep the timeline locked. */25 /* Opencode i915_request_add() so we can keep the timeline locked. */
26 __i915_request_commit(rq);26 __i915_request_commit(rq);
27 rq->sched.attr.priority = I915_PRIORITY_BARRIER;
27 __i915_request_queue(rq, NULL);28 __i915_request_queue(rq, NULL);
28 29
29 timeout = i915_request_wait(rq, 0, HZ / 10);30 timeout = i915_request_wait(rq, 0, HZ / 10);
@@ -154,10 +155,7 @@ static int live_context_size(void *arg) · 124 unmodified lines
154 */155 */
155 156
156 for_each_engine(engine, gt, id) {157 for_each_engine(engine, gt, id) {
157 struct {158 struct file *saved;
158 struct file *state;
159 void *pinned;
160 } saved;
161 159
162 if (!engine->context_size)160 if (!engine->context_size)
163 continue;161 continue;
@@ -171,8 +169,7 @@ static int live_context_size(void *arg) · 7 unmodified lines
171 * active state is sufficient, we are only checking that we169 * active state is sufficient, we are only checking that we
172 * don't use more than we planned.170 * don't use more than we planned.
173 */171 */
174 saved.state = fetch_and_zero(&engine->default_state);172 saved = fetch_and_zero(&engine->default_state);
175 saved.pinned = fetch_and_zero(&engine->pinned_default_state);
176 173
177 /* Overlaps with the execlists redzone */174 /* Overlaps with the execlists redzone */
178 engine->context_size += I915_GTT_PAGE_SIZE;175 engine->context_size += I915_GTT_PAGE_SIZE;
@@ -181,8 +178,7 @@ static int live_context_size(void *arg) · 2 unmodified lines
181 178
182 engine->context_size -= I915_GTT_PAGE_SIZE;179 engine->context_size -= I915_GTT_PAGE_SIZE;
183 180
184 engine->pinned_default_state = saved.pinned;181 engine->default_state = saved;
185 engine->default_state = saved.state;
186 182
187 intel_engine_pm_put(engine);183 intel_engine_pm_put(engine);
188 184
drivers/gpu/drm/i915/gt/selftest_lrc.c+42−44
@@ -4342,35 +4342,6 @@ int intel_execlists_live_selftests(struct drm_i915_private *i915) · 4341 unmodified lines
4342 return intel_gt_live_subtests(tests, &i915->gt);4342 return intel_gt_live_subtests(tests, &i915->gt);
4343}4343}
4344 4344
4345static void hexdump(const void *buf, size_t len)
4346{
4347 const size_t rowsize = 8 * sizeof(u32);
4348 const void *prev = NULL;
4349 bool skip = false;
4350 size_t pos;
4351
4352 for (pos = 0; pos < len; pos += rowsize) {
4353 char line[128];
4354
4355 if (prev && !memcmp(prev, buf + pos, rowsize)) {
4356 if (!skip) {
4357 pr_info("*\n");
4358 skip = true;
4359 }
4360 continue;
4361 }
4362
4363 WARN_ON_ONCE(hex_dump_to_buffer(buf + pos, len - pos,
4364 rowsize, sizeof(u32),
4365 line, sizeof(line),
4366 false) >= sizeof(line));
4367 pr_info("[%04zx] %s\n", pos, line);
4368
4369 prev = buf + pos;
4370 skip = false;
4371 }
4372}
4373
4374static int emit_semaphore_signal(struct intel_context *ce, void *slot)4345static int emit_semaphore_signal(struct intel_context *ce, void *slot)
4375{4346{
4376 const u32 offset =4347 const u32 offset =
@@ -4518,10 +4489,10 @@ static int live_lrc_layout(void *arg) · 141 unmodified lines
4518 4489
4519 if (err) {4490 if (err) {
4520 pr_info("%s: HW register image:\n", engine->name);4491 pr_info("%s: HW register image:\n", engine->name);
4521 hexdump(hw, PAGE_SIZE);4492 igt_hexdump(hw, PAGE_SIZE);
4522 4493
4523 pr_info("%s: SW register image:\n", engine->name);4494 pr_info("%s: SW register image:\n", engine->name);
4524 hexdump(lrc, PAGE_SIZE);4495 igt_hexdump(lrc, PAGE_SIZE);
4525 }4496 }
4526 4497
4527 shmem_unpin_map(engine->default_state, hw);4498 shmem_unpin_map(engine->default_state, hw);
@@ -5206,6 +5177,7 @@ store_context(struct intel_context *ce, struct i915_vma *scratch) · 678 unmodified lines
5206{5177{
5207 struct i915_vma *batch;5178 struct i915_vma *batch;
5208 u32 dw, x, *cs, *hw;5179 u32 dw, x, *cs, *hw;
5180 u32 *defaults;
5209 5181
5210 batch = create_user_vma(ce->vm, SZ_64K);5182 batch = create_user_vma(ce->vm, SZ_64K);
5211 if (IS_ERR(batch))5183 if (IS_ERR(batch))
@@ -5217,9 +5189,16 @@ store_context(struct intel_context *ce, struct i915_vma *scratch) · 5 unmodified lines
5217 return ERR_CAST(cs);5189 return ERR_CAST(cs);
5218 }5190 }
5219 5191
5192 defaults = shmem_pin_map(ce->engine->default_state);
5193 if (!defaults) {
5194 i915_gem_object_unpin_map(batch->obj);
5195 i915_vma_put(batch);
5196 return ERR_PTR(-ENOMEM);
5197 }
5198
5220 x = 0;5199 x = 0;
5221 dw = 0;5200 dw = 0;
5222 hw = ce->engine->pinned_default_state;5201 hw = defaults;
5223 hw += LRC_STATE_OFFSET / sizeof(*hw);5202 hw += LRC_STATE_OFFSET / sizeof(*hw);
5224 do {5203 do {
5225 u32 len = hw[dw] & 0x7f;5204 u32 len = hw[dw] & 0x7f;
@@ -5250,6 +5229,8 @@ store_context(struct intel_context *ce, struct i915_vma *scratch) · 24 unmodified lines
5250 5229
5251 *cs++ = MI_BATCH_BUFFER_END;5230 *cs++ = MI_BATCH_BUFFER_END;
5252 5231
5232 shmem_unpin_map(ce->engine->default_state, defaults);
5233
5253 i915_gem_object_flush_map(batch->obj);5234 i915_gem_object_flush_map(batch->obj);
5254 i915_gem_object_unpin_map(batch->obj);5235 i915_gem_object_unpin_map(batch->obj);
5255 5236
@@ -5360,6 +5341,7 @@ static struct i915_vma *load_context(struct intel_context *ce, u32 poison) · 104 unmodified lines
5360{5341{
5361 struct i915_vma *batch;5342 struct i915_vma *batch;
5362 u32 dw, *cs, *hw;5343 u32 dw, *cs, *hw;
5344 u32 *defaults;
5363 5345
5364 batch = create_user_vma(ce->vm, SZ_64K);5346 batch = create_user_vma(ce->vm, SZ_64K);
5365 if (IS_ERR(batch))5347 if (IS_ERR(batch))
@@ -5371,8 +5353,15 @@ static struct i915_vma *load_context(struct intel_context *ce, u32 poison) · 5 unmodified lines
5371 return ERR_CAST(cs);5353 return ERR_CAST(cs);
5372 }5354 }
5373 5355
5356 defaults = shmem_pin_map(ce->engine->default_state);
5357 if (!defaults) {
5358 i915_gem_object_unpin_map(batch->obj);
5359 i915_vma_put(batch);
5360 return ERR_PTR(-ENOMEM);
5361 }
5362
5374 dw = 0;5363 dw = 0;
5375 hw = ce->engine->pinned_default_state;5364 hw = defaults;
5376 hw += LRC_STATE_OFFSET / sizeof(*hw);5365 hw += LRC_STATE_OFFSET / sizeof(*hw);
5377 do {5366 do {
5378 u32 len = hw[dw] & 0x7f;5367 u32 len = hw[dw] & 0x7f;
@@ -5400,6 +5389,8 @@ static struct i915_vma *load_context(struct intel_context *ce, u32 poison) · 21 unmodified lines
5400 5389
5401 *cs++ = MI_BATCH_BUFFER_END;5390 *cs++ = MI_BATCH_BUFFER_END;
5402 5391
5392 shmem_unpin_map(ce->engine->default_state, defaults);
5393
5403 i915_gem_object_flush_map(batch->obj);5394 i915_gem_object_flush_map(batch->obj);
5404 i915_gem_object_unpin_map(batch->obj);5395 i915_gem_object_unpin_map(batch->obj);
5405 5396
@@ -5467,6 +5458,7 @@ static int compare_isolation(struct intel_engine_cs *engine, · 61 unmodified lines
5467{5458{
5468 u32 x, dw, *hw, *lrc;5459 u32 x, dw, *hw, *lrc;
5469 u32 *A[2], *B[2];5460 u32 *A[2], *B[2];
5461 u32 *defaults;
5470 int err = 0;5462 int err = 0;
5471 5463
5472 A[0] = i915_gem_object_pin_map(ref[0]->obj, I915_MAP_WC);5464 A[0] = i915_gem_object_pin_map(ref[0]->obj, I915_MAP_WC);
@@ -5499,9 +5491,15 @@ static int compare_isolation(struct intel_engine_cs *engine, · 26 unmodified lines
5499 }5491 }
5500 lrc += LRC_STATE_OFFSET / sizeof(*hw);5492 lrc += LRC_STATE_OFFSET / sizeof(*hw);
5501 5493
5494 defaults = shmem_pin_map(ce->engine->default_state);
5495 if (!defaults) {
5496 err = -ENOMEM;
5497 goto err_lrc;
5498 }
5499
5502 x = 0;5500 x = 0;
5503 dw = 0;5501 dw = 0;
5504 hw = engine->pinned_default_state;5502 hw = defaults;
5505 hw += LRC_STATE_OFFSET / sizeof(*hw);5503 hw += LRC_STATE_OFFSET / sizeof(*hw);
5506 do {5504 do {
5507 u32 len = hw[dw] & 0x7f;5505 u32 len = hw[dw] & 0x7f;
@@ -5541,6 +5539,8 @@ static int compare_isolation(struct intel_engine_cs *engine, · 33 unmodified lines
5541 } while (dw < PAGE_SIZE / sizeof(u32) &&5539 } while (dw < PAGE_SIZE / sizeof(u32) &&
5542 (hw[dw] & ~BIT(0)) != MI_BATCH_BUFFER_END);5540 (hw[dw] & ~BIT(0)) != MI_BATCH_BUFFER_END);
5543 5541
5542 shmem_unpin_map(ce->engine->default_state, defaults);
5543err_lrc:
5544 i915_gem_object_unpin_map(ce->state->obj);5544 i915_gem_object_unpin_map(ce->state->obj);
5545err_B1:5545err_B1:
5546 i915_gem_object_unpin_map(result[1]->obj);5546 i915_gem_object_unpin_map(result[1]->obj);
@@ -5690,18 +5690,16 @@ static int live_lrc_isolation(void *arg) · 143 unmodified lines
5690 continue;5690 continue;
5691 5691
5692 intel_engine_pm_get(engine);5692 intel_engine_pm_get(engine);
5693 if (engine->pinned_default_state) {5693 for (i = 0; i < ARRAY_SIZE(poison); i++) {
5694 for (i = 0; i < ARRAY_SIZE(poison); i++) {5694 int result;
5695 int result;
5696 5695
5697 result = __lrc_isolation(engine, poison[i]);5696 result = __lrc_isolation(engine, poison[i]);
5698 if (result && !err)5697 if (result && !err)
5699 err = result;5698 err = result;
5700 5699
5701 result = __lrc_isolation(engine, ~poison[i]);5700 result = __lrc_isolation(engine, ~poison[i]);
5702 if (result && !err)5701 if (result && !err)
5703 err = result;5702 err = result;
5704 }
5705 }5703 }
5706 intel_engine_pm_put(engine);5704 intel_engine_pm_put(engine);
5707 if (igt_flush_test(gt->i915)) {5705 if (igt_flush_test(gt->i915)) {
drivers/gpu/drm/i915/gt/selftest_ring_submission.c+2−0
@@ -54,6 +54,8 @@ static struct i915_vma *create_wally(struct intel_engine_cs *engine) · 53 unmodified lines
54 *cs++ = STACK_MAGIC;54 *cs++ = STACK_MAGIC;
55 55
56 *cs++ = MI_BATCH_BUFFER_END;56 *cs++ = MI_BATCH_BUFFER_END;
57
58 i915_gem_object_flush_map(obj);
57 i915_gem_object_unpin_map(obj);59 i915_gem_object_unpin_map(obj);
58 60
59 vma->private = intel_context_create(engine); /* dummy residuals */61 vma->private = intel_context_create(engine); /* dummy residuals */
drivers/gpu/drm/i915/gt/selftest_rps.c+42−14
@@ -56,6 +56,18 @@ static int cmp_u64(const void *A, const void *B) · 55 unmodified lines
56 return 0;56 return 0;
57}57}
58 58
59static int cmp_u32(const void *A, const void *B)
60{
61 const u32 *a = A, *b = B;
62
63 if (a < b)
64 return -1;
65 else if (a > b)
66 return 1;
67 else
68 return 0;
69}
70
59static struct i915_vma *71static struct i915_vma *
60create_spin_counter(struct intel_engine_cs *engine,72create_spin_counter(struct intel_engine_cs *engine,
61 struct i915_address_space *vm,73 struct i915_address_space *vm,
@@ -236,8 +248,8 @@ int live_rps_clock_interval(void *arg) · 174 unmodified lines
236 for_each_engine(engine, gt, id) {248 for_each_engine(engine, gt, id) {
237 unsigned long saved_heartbeat;249 unsigned long saved_heartbeat;
238 struct i915_request *rq;250 struct i915_request *rq;
239 ktime_t dt;
240 u32 cycles;251 u32 cycles;
252 u64 dt;
241 253
242 if (!intel_engine_can_store_dword(engine))254 if (!intel_engine_can_store_dword(engine))
243 continue;255 continue;
@@ -286,15 +298,29 @@ int live_rps_clock_interval(void *arg) · 42 unmodified lines
286 engine->name);298 engine->name);
287 err = -ENODEV;299 err = -ENODEV;
288 } else {300 } else {
289 preempt_disable();301 ktime_t dt_[5];
290 dt = ktime_get();302 u32 cycles_[5];
291 cycles = -intel_uncore_read_fw(gt->uncore,303 int i;
292 GEN6_RP_CUR_UP_EI);304
293 udelay(1000);305 for (i = 0; i < 5; i++) {
294 dt = ktime_sub(ktime_get(), dt);306 preempt_disable();
295 cycles += intel_uncore_read_fw(gt->uncore,307
296 GEN6_RP_CUR_UP_EI);308 dt_[i] = ktime_get();
297 preempt_enable();309 cycles_[i] = -intel_uncore_read_fw(gt->uncore, GEN6_RP_CUR_UP_EI);
310
311 udelay(1000);
312
313 dt_[i] = ktime_sub(ktime_get(), dt_[i]);
314 cycles_[i] += intel_uncore_read_fw(gt->uncore, GEN6_RP_CUR_UP_EI);
315
316 preempt_enable();
317 }
318
319 /* Use the median of both cycle/dt; close enough */
320 sort(cycles_, 5, sizeof(*cycles_), cmp_u32, NULL);
321 cycles = (cycles_[1] + 2 * cycles_[2] + cycles_[3]) / 4;
322 sort(dt_, 5, sizeof(*dt_), cmp_u64, NULL);
323 dt = div_u64(dt_[1] + 2 * dt_[2] + dt_[3], 4);
298 }324 }
299 325
300 intel_uncore_write_fw(gt->uncore, GEN6_RP_CONTROL, 0);326 intel_uncore_write_fw(gt->uncore, GEN6_RP_CONTROL, 0);
@@ -306,14 +332,14 @@ int live_rps_clock_interval(void *arg) · 5 unmodified lines
306 if (err == 0) {332 if (err == 0) {
307 u64 time = intel_gt_pm_interval_to_ns(gt, cycles);333 u64 time = intel_gt_pm_interval_to_ns(gt, cycles);
308 u32 expected =334 u32 expected =
309 intel_gt_ns_to_pm_interval(gt, ktime_to_ns(dt));335 intel_gt_ns_to_pm_interval(gt, dt);
310 336
311 pr_info("%s: rps counted %d C0 cycles [%lldns] in %lldns [%d cycles], using GT clock frequency of %uKHz\n",337 pr_info("%s: rps counted %d C0 cycles [%lldns] in %lldns [%d cycles], using GT clock frequency of %uKHz\n",
312 engine->name, cycles, time, ktime_to_ns(dt), expected,338 engine->name, cycles, time, dt, expected,
313 gt->clock_frequency / 1000);339 gt->clock_frequency / 1000);
314 340
315 if (10 * time < 8 * ktime_to_ns(dt) ||341 if (10 * time < 8 * dt ||
316 8 * time > 10 * ktime_to_ns(dt)) {342 8 * time > 10 * dt) {
317 pr_err("%s: rps clock time does not match walltime!\n",343 pr_err("%s: rps clock time does not match walltime!\n",
318 engine->name);344 engine->name);
319 err = -EINVAL;345 err = -EINVAL;
@@ -701,6 +727,7 @@ int live_rps_frequency_cs(void *arg) · 381 unmodified lines
701 727
702err_vma:728err_vma:
703 *cancel = MI_BATCH_BUFFER_END;729 *cancel = MI_BATCH_BUFFER_END;
730 i915_gem_object_flush_map(vma->obj);
704 i915_gem_object_unpin_map(vma->obj);731 i915_gem_object_unpin_map(vma->obj);
705 i915_vma_unpin(vma);732 i915_vma_unpin(vma);
706 i915_vma_put(vma);733 i915_vma_put(vma);
@@ -842,6 +869,7 @@ int live_rps_frequency_srm(void *arg) · 135 unmodified lines
842 869
843err_vma:870err_vma:
844 *cancel = MI_BATCH_BUFFER_END;871 *cancel = MI_BATCH_BUFFER_END;
872 i915_gem_object_flush_map(vma->obj);
845 i915_gem_object_unpin_map(vma->obj);873 i915_gem_object_unpin_map(vma->obj);
846 i915_vma_unpin(vma);874 i915_vma_unpin(vma);
847 i915_vma_put(vma);875 i915_vma_put(vma);
drivers/gpu/drm/i915/gt/sysfs_engines.c+94−0
@@ -191,6 +191,17 @@ max_spin_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) · 190 unmodified lines
191static struct kobj_attribute max_spin_attr =191static struct kobj_attribute max_spin_attr =
192__ATTR(max_busywait_duration_ns, 0644, max_spin_show, max_spin_store);192__ATTR(max_busywait_duration_ns, 0644, max_spin_show, max_spin_store);
193 193
194static ssize_t
195max_spin_default(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
196{
197 struct intel_engine_cs *engine = kobj_to_engine(kobj);
198
199 return sprintf(buf, "%lu\n", engine->defaults.max_busywait_duration_ns);
200}
201
202static struct kobj_attribute max_spin_def =
203__ATTR(max_busywait_duration_ns, 0444, max_spin_default, NULL);
204
194static ssize_t205static ssize_t
195timeslice_store(struct kobject *kobj, struct kobj_attribute *attr,206timeslice_store(struct kobject *kobj, struct kobj_attribute *attr,
196 const char *buf, size_t count)207 const char *buf, size_t count)
@@ -233,6 +244,17 @@ timeslice_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) · 36 unmodified lines
233static struct kobj_attribute timeslice_duration_attr =244static struct kobj_attribute timeslice_duration_attr =
234__ATTR(timeslice_duration_ms, 0644, timeslice_show, timeslice_store);245__ATTR(timeslice_duration_ms, 0644, timeslice_show, timeslice_store);
235 246
247static ssize_t
248timeslice_default(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
249{
250 struct intel_engine_cs *engine = kobj_to_engine(kobj);
251
252 return sprintf(buf, "%lu\n", engine->defaults.timeslice_duration_ms);
253}
254
255static struct kobj_attribute timeslice_duration_def =
256__ATTR(timeslice_duration_ms, 0444, timeslice_default, NULL);
257
236static ssize_t258static ssize_t
237stop_store(struct kobject *kobj, struct kobj_attribute *attr,259stop_store(struct kobject *kobj, struct kobj_attribute *attr,
238 const char *buf, size_t count)260 const char *buf, size_t count)
@@ -272,6 +294,17 @@ stop_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) · 33 unmodified lines
272static struct kobj_attribute stop_timeout_attr =294static struct kobj_attribute stop_timeout_attr =
273__ATTR(stop_timeout_ms, 0644, stop_show, stop_store);295__ATTR(stop_timeout_ms, 0644, stop_show, stop_store);
274 296
297static ssize_t
298stop_default(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
299{
300 struct intel_engine_cs *engine = kobj_to_engine(kobj);
301
302 return sprintf(buf, "%lu\n", engine->defaults.stop_timeout_ms);
303}
304
305static struct kobj_attribute stop_timeout_def =
306__ATTR(stop_timeout_ms, 0444, stop_default, NULL);
307
275static ssize_t308static ssize_t
276preempt_timeout_store(struct kobject *kobj, struct kobj_attribute *attr,309preempt_timeout_store(struct kobject *kobj, struct kobj_attribute *attr,
277 const char *buf, size_t count)310 const char *buf, size_t count)
@@ -316,6 +349,18 @@ preempt_timeout_show(struct kobject *kobj, struct kobj_attribute *attr, · 38 unmodified lines
316static struct kobj_attribute preempt_timeout_attr =349static struct kobj_attribute preempt_timeout_attr =
317__ATTR(preempt_timeout_ms, 0644, preempt_timeout_show, preempt_timeout_store);350__ATTR(preempt_timeout_ms, 0644, preempt_timeout_show, preempt_timeout_store);
318 351
352static ssize_t
353preempt_timeout_default(struct kobject *kobj, struct kobj_attribute *attr,
354 char *buf)
355{
356 struct intel_engine_cs *engine = kobj_to_engine(kobj);
357
358 return sprintf(buf, "%lu\n", engine->defaults.preempt_timeout_ms);
359}
360
361static struct kobj_attribute preempt_timeout_def =
362__ATTR(preempt_timeout_ms, 0444, preempt_timeout_default, NULL);
363
319static ssize_t364static ssize_t
320heartbeat_store(struct kobject *kobj, struct kobj_attribute *attr,365heartbeat_store(struct kobject *kobj, struct kobj_attribute *attr,
321 const char *buf, size_t count)366 const char *buf, size_t count)
@@ -359,6 +404,17 @@ heartbeat_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) · 37 unmodified lines
359static struct kobj_attribute heartbeat_interval_attr =404static struct kobj_attribute heartbeat_interval_attr =
360__ATTR(heartbeat_interval_ms, 0644, heartbeat_show, heartbeat_store);405__ATTR(heartbeat_interval_ms, 0644, heartbeat_show, heartbeat_store);
361 406
407static ssize_t
408heartbeat_default(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
409{
410 struct intel_engine_cs *engine = kobj_to_engine(kobj);
411
412 return sprintf(buf, "%lu\n", engine->defaults.heartbeat_interval_ms);
413}
414
415static struct kobj_attribute heartbeat_interval_def =
416__ATTR(heartbeat_interval_ms, 0444, heartbeat_default, NULL);
417
362static void kobj_engine_release(struct kobject *kobj)418static void kobj_engine_release(struct kobject *kobj)
363{419{
364 kfree(kobj);420 kfree(kobj);
@@ -390,6 +446,42 @@ kobj_engine(struct kobject *dir, struct intel_engine_cs *engine) · 25 unmodified lines
390 return &ke->base;446 return &ke->base;
391}447}
392 448
449static void add_defaults(struct kobj_engine *parent)
450{
451 static const struct attribute *files[] = {
452 &max_spin_def.attr,
453 &stop_timeout_def.attr,
454#if CONFIG_DRM_I915_HEARTBEAT_INTERVAL
455 &heartbeat_interval_def.attr,
456#endif
457 NULL
458 };
459 struct kobj_engine *ke;
460
461 ke = kzalloc(sizeof(*ke), GFP_KERNEL);
462 if (!ke)
463 return;
464
465 kobject_init(&ke->base, &kobj_engine_type);
466 ke->engine = parent->engine;
467
468 if (kobject_add(&ke->base, &parent->base, "%s", ".defaults")) {
469 kobject_put(&ke->base);
470 return;
471 }
472
473 if (sysfs_create_files(&ke->base, files))
474 return;
475
476 if (intel_engine_has_timeslices(ke->engine) &&
477 sysfs_create_file(&ke->base, &timeslice_duration_def.attr))
478 return;
479
480 if (intel_engine_has_preempt_reset(ke->engine) &&
481 sysfs_create_file(&ke->base, &preempt_timeout_def.attr))
482 return;
483}
484
393void intel_engines_add_sysfs(struct drm_i915_private *i915)485void intel_engines_add_sysfs(struct drm_i915_private *i915)
394{486{
395 static const struct attribute *files[] = {487 static const struct attribute *files[] = {
@@ -433,6 +525,8 @@ void intel_engines_add_sysfs(struct drm_i915_private *i915) · 37 unmodified lines
433 sysfs_create_file(kobj, &preempt_timeout_attr.attr))525 sysfs_create_file(kobj, &preempt_timeout_attr.attr))
434 goto err_engine;526 goto err_engine;
435 527
528 add_defaults(container_of(kobj, struct kobj_engine, base));
529
436 if (0) {530 if (0) {
437err_object:531err_object:
438 kobject_put(kobj);532 kobject_put(kobj);
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c+1−1
@@ -258,7 +258,7 @@ static void guc_submit(struct intel_engine_cs *engine, · 257 unmodified lines
258 258
259static inline int rq_prio(const struct i915_request *rq)259static inline int rq_prio(const struct i915_request *rq)
260{260{
261 return rq->sched.attr.priority | __NO_PREEMPTION;261 return rq->sched.attr.priority;
262}262}
263 263
264static struct i915_request *schedule_in(struct i915_request *rq, int idx)264static struct i915_request *schedule_in(struct i915_request *rq, int idx)
drivers/gpu/drm/i915/gvt/cmd_parser.c+45−0
@@ -882,6 +882,47 @@ static int mocs_cmd_reg_handler(struct parser_exec_state *s, · 881 unmodified lines
882 return 0;882 return 0;
883}883}
884 884
885static int is_cmd_update_pdps(unsigned int offset,
886 struct parser_exec_state *s)
887{
888 u32 base = s->workload->engine->mmio_base;
889 return i915_mmio_reg_equal(_MMIO(offset), GEN8_RING_PDP_UDW(base, 0));
890}
891
892static int cmd_pdp_mmio_update_handler(struct parser_exec_state *s,
893 unsigned int offset, unsigned int index)
894{
895 struct intel_vgpu *vgpu = s->vgpu;
896 struct intel_vgpu_mm *shadow_mm = s->workload->shadow_mm;
897 struct intel_vgpu_mm *mm;
898 u64 pdps[GEN8_3LVL_PDPES];
899
900 if (shadow_mm->ppgtt_mm.root_entry_type ==
901 GTT_TYPE_PPGTT_ROOT_L4_ENTRY) {
902 pdps[0] = (u64)cmd_val(s, 2) << 32;
903 pdps[0] |= cmd_val(s, 4);
904
905 mm = intel_vgpu_find_ppgtt_mm(vgpu, pdps);
906 if (!mm) {
907 gvt_vgpu_err("failed to get the 4-level shadow vm\n");
908 return -EINVAL;
909 }
910 intel_vgpu_mm_get(mm);
911 list_add_tail(&mm->ppgtt_mm.link,
912 &s->workload->lri_shadow_mm);
913 *cmd_ptr(s, 2) = upper_32_bits(mm->ppgtt_mm.shadow_pdps[0]);
914 *cmd_ptr(s, 4) = lower_32_bits(mm->ppgtt_mm.shadow_pdps[0]);
915 } else {
916 /* Currently all guests use PML4 table and now can't
917 * have a guest with 3-level table but uses LRI for
918 * PPGTT update. So this is simply un-testable. */
919 GEM_BUG_ON(1);
920 gvt_vgpu_err("invalid shared shadow vm type\n");
921 return -EINVAL;
922 }
923 return 0;
924}
925
885static int cmd_reg_handler(struct parser_exec_state *s,926static int cmd_reg_handler(struct parser_exec_state *s,
886 unsigned int offset, unsigned int index, char *cmd)927 unsigned int offset, unsigned int index, char *cmd)
887{928{
@@ -920,6 +961,10 @@ static int cmd_reg_handler(struct parser_exec_state *s, · 32 unmodified lines
920 patch_value(s, cmd_ptr(s, index), VGT_PVINFO_PAGE);961 patch_value(s, cmd_ptr(s, index), VGT_PVINFO_PAGE);
921 }962 }
922 963
964 if (is_cmd_update_pdps(offset, s) &&
965 cmd_pdp_mmio_update_handler(s, offset, index))
966 return -EINVAL;
967
923 /* TODO968 /* TODO
924 * In order to let workload with inhibit context to generate969 * In order to let workload with inhibit context to generate
925 * correct image data into memory, vregs values will be loaded to970 * correct image data into memory, vregs values will be loaded to
drivers/gpu/drm/i915/gvt/execlist.c+0−2
@@ -424,8 +424,6 @@ static int complete_execlist_workload(struct intel_vgpu_workload *workload) · 423 unmodified lines
424 424
425 ret = emulate_execlist_ctx_schedule_out(execlist, &workload->ctx_desc);425 ret = emulate_execlist_ctx_schedule_out(execlist, &workload->ctx_desc);
426out:426out:
427 intel_vgpu_unpin_mm(workload->shadow_mm);
428 intel_vgpu_destroy_workload(workload);
429 return ret;427 return ret;
430}428}
431 429
drivers/gpu/drm/i915/gvt/gtt.c+1−0
@@ -1900,6 +1900,7 @@ struct intel_vgpu_mm *intel_vgpu_create_ppgtt_mm(struct intel_vgpu *vgpu, · 1899 unmodified lines
1900 1900
1901 INIT_LIST_HEAD(&mm->ppgtt_mm.list);1901 INIT_LIST_HEAD(&mm->ppgtt_mm.list);
1902 INIT_LIST_HEAD(&mm->ppgtt_mm.lru_list);1902 INIT_LIST_HEAD(&mm->ppgtt_mm.lru_list);
1903 INIT_LIST_HEAD(&mm->ppgtt_mm.link);
1903 1904
1904 if (root_entry_type == GTT_TYPE_PPGTT_ROOT_L4_ENTRY)1905 if (root_entry_type == GTT_TYPE_PPGTT_ROOT_L4_ENTRY)
1905 mm->ppgtt_mm.guest_pdps[0] = pdps[0];1906 mm->ppgtt_mm.guest_pdps[0] = pdps[0];
drivers/gpu/drm/i915/gvt/gtt.h+1−0
@@ -160,6 +160,7 @@ struct intel_vgpu_mm { · 159 unmodified lines
160 160
161 struct list_head list;161 struct list_head list;
162 struct list_head lru_list;162 struct list_head lru_list;
163 struct list_head link; /* possible LRI shadow mm list */
163 } ppgtt_mm;164 } ppgtt_mm;
164 struct {165 struct {
165 void *virtual_ggtt;166 void *virtual_ggtt;
drivers/gpu/drm/i915/gvt/handlers.c+1−1
@@ -2812,7 +2812,7 @@ static int init_bdw_mmio_info(struct intel_gvt *gvt) · 2811 unmodified lines
2812 MMIO_D(GAMTARBMODE, D_BDW_PLUS);2812 MMIO_D(GAMTARBMODE, D_BDW_PLUS);
2813 2813
2814#define RING_REG(base) _MMIO((base) + 0x270)2814#define RING_REG(base) _MMIO((base) + 0x270)
2815 MMIO_RING_F(RING_REG, 32, 0, 0, 0, D_BDW_PLUS, NULL, NULL);2815 MMIO_RING_F(RING_REG, 32, F_CMD_ACCESS, 0, 0, D_BDW_PLUS, NULL, NULL);
2816#undef RING_REG2816#undef RING_REG
2817 2817
2818 MMIO_RING_GM_RDR(RING_HWS_PGA, D_BDW_PLUS, NULL, hws_pga_write);2818 MMIO_RING_GM_RDR(RING_HWS_PGA, D_BDW_PLUS, NULL, hws_pga_write);
drivers/gpu/drm/i915/gvt/scheduler.c+106−9
@@ -58,10 +58,8 @@ static void set_context_pdp_root_pointer( · 57 unmodified lines
58 58
59static void update_shadow_pdps(struct intel_vgpu_workload *workload)59static void update_shadow_pdps(struct intel_vgpu_workload *workload)
60{60{
61 struct drm_i915_gem_object *ctx_obj =
62 workload->req->context->state->obj;
63 struct execlist_ring_context *shadow_ring_context;61 struct execlist_ring_context *shadow_ring_context;
64 struct page *page;62 struct intel_context *ctx = workload->req->context;
65 63
66 if (WARN_ON(!workload->shadow_mm))64 if (WARN_ON(!workload->shadow_mm))
67 return;65 return;
@@ -69,11 +67,9 @@ static void update_shadow_pdps(struct intel_vgpu_workload *workload) · 1 unmodified line
69 if (WARN_ON(!atomic_read(&workload->shadow_mm->pincount)))67 if (WARN_ON(!atomic_read(&workload->shadow_mm->pincount)))
70 return;68 return;
71 69
72 page = i915_gem_object_get_page(ctx_obj, LRC_STATE_PN);70 shadow_ring_context = (struct execlist_ring_context *)ctx->lrc_reg_state;
73 shadow_ring_context = kmap(page);
74 set_context_pdp_root_pointer(shadow_ring_context,71 set_context_pdp_root_pointer(shadow_ring_context,
75 (void *)workload->shadow_mm->ppgtt_mm.shadow_pdps);72 (void *)workload->shadow_mm->ppgtt_mm.shadow_pdps);
76 kunmap(page);
77}73}
78 74
79/*75/*
@@ -646,10 +642,11 @@ static void release_shadow_batch_buffer(struct intel_vgpu_workload *workload) · 566 unmodified lines
646 }642 }
647}643}
648 644
649static int prepare_workload(struct intel_vgpu_workload *workload)645static int
646intel_vgpu_shadow_mm_pin(struct intel_vgpu_workload *workload)
650{647{
651 struct intel_vgpu *vgpu = workload->vgpu;648 struct intel_vgpu *vgpu = workload->vgpu;
652 struct intel_vgpu_submission *s = &vgpu->submission;649 struct intel_vgpu_mm *m;
653 int ret = 0;650 int ret = 0;
654 651
655 ret = intel_vgpu_pin_mm(workload->shadow_mm);652 ret = intel_vgpu_pin_mm(workload->shadow_mm);
@@ -664,6 +661,52 @@ static int prepare_workload(struct intel_vgpu_workload *workload) · 8 unmodified lines
664 return -EINVAL;661 return -EINVAL;
665 }662 }
666 663
664 if (!list_empty(&workload->lri_shadow_mm)) {
665 list_for_each_entry(m, &workload->lri_shadow_mm,
666 ppgtt_mm.link) {
667 ret = intel_vgpu_pin_mm(m);
668 if (ret) {
669 list_for_each_entry_from_reverse(m,
670 &workload->lri_shadow_mm,
671 ppgtt_mm.link)
672 intel_vgpu_unpin_mm(m);
673 gvt_vgpu_err("LRI shadow ppgtt fail to pin\n");
674 break;
675 }
676 }
677 }
678
679 if (ret)
680 intel_vgpu_unpin_mm(workload->shadow_mm);
681
682 return ret;
683}
684
685static void
686intel_vgpu_shadow_mm_unpin(struct intel_vgpu_workload *workload)
687{
688 struct intel_vgpu_mm *m;
689
690 if (!list_empty(&workload->lri_shadow_mm)) {
691 list_for_each_entry(m, &workload->lri_shadow_mm,
692 ppgtt_mm.link)
693 intel_vgpu_unpin_mm(m);
694 }
695 intel_vgpu_unpin_mm(workload->shadow_mm);
696}
697
698static int prepare_workload(struct intel_vgpu_workload *workload)
699{
700 struct intel_vgpu *vgpu = workload->vgpu;
701 struct intel_vgpu_submission *s = &vgpu->submission;
702 int ret = 0;
703
704 ret = intel_vgpu_shadow_mm_pin(workload);
705 if (ret) {
706 gvt_vgpu_err("fail to pin shadow mm\n");
707 return ret;
708 }
709
667 update_shadow_pdps(workload);710 update_shadow_pdps(workload);
668 711
669 set_context_ppgtt_from_shadow(workload, s->shadow[workload->engine->id]);712 set_context_ppgtt_from_shadow(workload, s->shadow[workload->engine->id]);
@@ -710,7 +753,7 @@ err_shadow_wa_ctx: · 40 unmodified lines
710err_shadow_batch:753err_shadow_batch:
711 release_shadow_batch_buffer(workload);754 release_shadow_batch_buffer(workload);
712err_unpin_mm:755err_unpin_mm:
713 intel_vgpu_unpin_mm(workload->shadow_mm);756 intel_vgpu_shadow_mm_unpin(workload);
714 return ret;757 return ret;
715}758}
716 759
@@ -820,6 +863,37 @@ out: · 103 unmodified lines
820 return workload;863 return workload;
821}864}
822 865
866static void update_guest_pdps(struct intel_vgpu *vgpu,
867 u64 ring_context_gpa, u32 pdp[8])
868{
869 u64 gpa;
870 int i;
871
872 gpa = ring_context_gpa + RING_CTX_OFF(pdps[0].val);
873
874 for (i = 0; i < 8; i++)
875 intel_gvt_hypervisor_write_gpa(vgpu,
876 gpa + i * 8, &pdp[7 - i], 4);
877}
878
879static bool
880check_shadow_context_ppgtt(struct execlist_ring_context *c, struct intel_vgpu_mm *m)
881{
882 if (m->ppgtt_mm.root_entry_type == GTT_TYPE_PPGTT_ROOT_L4_ENTRY) {
883 u64 shadow_pdp = c->pdps[7].val | (u64) c->pdps[6].val << 32;
884
885 if (shadow_pdp != m->ppgtt_mm.shadow_pdps[0]) {
886 gvt_dbg_mm("4-level context ppgtt not match LRI command\n");
887 return false;
888 }
889 return true;
890 } else {
891 /* see comment in LRI handler in cmd_parser.c */
892 gvt_dbg_mm("invalid shadow mm type\n");
893 return false;
894 }
895}
896
823static void update_guest_context(struct intel_vgpu_workload *workload)897static void update_guest_context(struct intel_vgpu_workload *workload)
824{898{
825 struct i915_request *rq = workload->req;899 struct i915_request *rq = workload->req;
@@ -905,6 +979,15 @@ write: · 79 unmodified lines
905 979
906 shadow_ring_context = (void *) ctx->lrc_reg_state;980 shadow_ring_context = (void *) ctx->lrc_reg_state;
907 981
982 if (!list_empty(&workload->lri_shadow_mm)) {
983 struct intel_vgpu_mm *m = list_last_entry(&workload->lri_shadow_mm,
984 struct intel_vgpu_mm,
985 ppgtt_mm.link);
986 GEM_BUG_ON(!check_shadow_context_ppgtt(shadow_ring_context, m));
987 update_guest_pdps(vgpu, workload->ring_context_gpa,
988 (void *)m->ppgtt_mm.guest_pdps);
989 }
990
908#define COPY_REG(name) \991#define COPY_REG(name) \
909 intel_gvt_hypervisor_write_gpa(vgpu, workload->ring_context_gpa + \992 intel_gvt_hypervisor_write_gpa(vgpu, workload->ring_context_gpa + \
910 RING_CTX_OFF(name.val), &shadow_ring_context->name.val, 4)993 RING_CTX_OFF(name.val), &shadow_ring_context->name.val, 4)
@@ -1013,6 +1096,9 @@ static void complete_current_workload(struct intel_gvt *gvt, int ring_id) · 102 unmodified lines
1013 1096
1014 workload->complete(workload);1097 workload->complete(workload);
1015 1098
1099 intel_vgpu_shadow_mm_unpin(workload);
1100 intel_vgpu_destroy_workload(workload);
1101
1016 atomic_dec(&s->running_workload_num);1102 atomic_dec(&s->running_workload_num);
1017 wake_up(&scheduler->workload_complete_wq);1103 wake_up(&scheduler->workload_complete_wq);
1018 1104
@@ -1406,6 +1492,16 @@ void intel_vgpu_destroy_workload(struct intel_vgpu_workload *workload) · 387 unmodified lines
1406 release_shadow_batch_buffer(workload);1492 release_shadow_batch_buffer(workload);
1407 release_shadow_wa_ctx(&workload->wa_ctx);1493 release_shadow_wa_ctx(&workload->wa_ctx);
1408 1494
1495 if (!list_empty(&workload->lri_shadow_mm)) {
1496 struct intel_vgpu_mm *m, *mm;
1497 list_for_each_entry_safe(m, mm, &workload->lri_shadow_mm,
1498 ppgtt_mm.link) {
1499 list_del(&m->ppgtt_mm.link);
1500 intel_vgpu_mm_put(m);
1501 }
1502 }
1503
1504 GEM_BUG_ON(!list_empty(&workload->lri_shadow_mm));
1409 if (workload->shadow_mm)1505 if (workload->shadow_mm)
1410 intel_vgpu_mm_put(workload->shadow_mm);1506 intel_vgpu_mm_put(workload->shadow_mm);
1411 1507
@@ -1424,6 +1520,7 @@ alloc_workload(struct intel_vgpu *vgpu) · 12 unmodified lines
1424 1520
1425 INIT_LIST_HEAD(&workload->list);1521 INIT_LIST_HEAD(&workload->list);
1426 INIT_LIST_HEAD(&workload->shadow_bb);1522 INIT_LIST_HEAD(&workload->shadow_bb);
1523 INIT_LIST_HEAD(&workload->lri_shadow_mm);
1427 1524
1428 init_waitqueue_head(&workload->shadow_ctx_status_wq);1525 init_waitqueue_head(&workload->shadow_ctx_status_wq);
1429 atomic_set(&workload->shadow_ctx_active, 0);1526 atomic_set(&workload->shadow_ctx_active, 0);
drivers/gpu/drm/i915/gvt/scheduler.h+1−0
@@ -87,6 +87,7 @@ struct intel_vgpu_workload { · 86 unmodified lines
87 int status;87 int status;
88 88
89 struct intel_vgpu_mm *shadow_mm;89 struct intel_vgpu_mm *shadow_mm;
90 struct list_head lri_shadow_mm; /* For PPGTT load cmd */
90 91
91 /* different submission model may need different handler */92 /* different submission model may need different handler */
92 int (*prepare)(struct intel_vgpu_workload *);93 int (*prepare)(struct intel_vgpu_workload *);
drivers/gpu/drm/i915/i915_config.c+15−0
@@ -0,0 +1,15 @@
1// SPDX-License-Identifier: MIT
2/*
3 * Copyright © 2020 Intel Corporation
4 */
5
6#include "i915_drv.h"
7
8unsigned long
9i915_fence_context_timeout(const struct drm_i915_private *i915, u64 context)
10{
11 if (context && IS_ACTIVE(CONFIG_DRM_I915_FENCE_TIMEOUT))
12 return msecs_to_jiffies_timeout(CONFIG_DRM_I915_FENCE_TIMEOUT);
13
14 return 0;
15}
drivers/gpu/drm/i915/i915_debugfs.c+7−4
@@ -32,6 +32,7 @@ · 31 unmodified lines
32#include <drm/drm_debugfs.h>32#include <drm/drm_debugfs.h>
33 33
34#include "gem/i915_gem_context.h"34#include "gem/i915_gem_context.h"
35#include "gt/intel_gt_buffer_pool.h"
35#include "gt/intel_gt_clock_utils.h"36#include "gt/intel_gt_clock_utils.h"
36#include "gt/intel_gt_pm.h"37#include "gt/intel_gt_pm.h"
37#include "gt/intel_gt_requests.h"38#include "gt/intel_gt_requests.h"
@@ -1303,8 +1304,8 @@ static int i915_engine_info(struct seq_file *m, void *unused) · 1265 unmodified lines
1303 seq_printf(m, "GT awake? %s [%d]\n",1304 seq_printf(m, "GT awake? %s [%d]\n",
1304 yesno(dev_priv->gt.awake),1305 yesno(dev_priv->gt.awake),
1305 atomic_read(&dev_priv->gt.wakeref.count));1306 atomic_read(&dev_priv->gt.wakeref.count));
1306 seq_printf(m, "CS timestamp frequency: %u kHz\n",1307 seq_printf(m, "CS timestamp frequency: %u Hz\n",
1307 RUNTIME_INFO(dev_priv)->cs_timestamp_frequency_khz);1308 RUNTIME_INFO(dev_priv)->cs_timestamp_frequency_hz);
1308 1309
1309 p = drm_seq_file_printer(m);1310 p = drm_seq_file_printer(m);
1310 for_each_uabi_engine(engine, dev_priv)1311 for_each_uabi_engine(engine, dev_priv)
@@ -1403,13 +1404,12 @@ static int · 92 unmodified lines
1403i915_perf_noa_delay_set(void *data, u64 val)1404i915_perf_noa_delay_set(void *data, u64 val)
1404{1405{
1405 struct drm_i915_private *i915 = data;1406 struct drm_i915_private *i915 = data;
1406 const u32 clk = RUNTIME_INFO(i915)->cs_timestamp_frequency_khz;
1407 1407
1408 /*1408 /*
1409 * This would lead to infinite waits as we're doing timestamp1409 * This would lead to infinite waits as we're doing timestamp
1410 * difference on the CS with only 32bits.1410 * difference on the CS with only 32bits.
1411 */1411 */
1412 if (val > mul_u32_u32(U32_MAX, clk))1412 if (i915_cs_timestamp_ns_to_ticks(i915, val) > U32_MAX)
1413 return -EINVAL;1413 return -EINVAL;
1414 1414
1415 atomic64_set(&i915->perf.noa_programming_delay, val);1415 atomic64_set(&i915->perf.noa_programming_delay, val);
@@ -1484,6 +1484,9 @@ gt_drop_caches(struct intel_gt *gt, u64 val) · 68 unmodified lines
1484 if (val & DROP_RESET_ACTIVE && intel_gt_terminally_wedged(gt))1484 if (val & DROP_RESET_ACTIVE && intel_gt_terminally_wedged(gt))
1485 intel_gt_handle_error(gt, ALL_ENGINES, 0, NULL);1485 intel_gt_handle_error(gt, ALL_ENGINES, 0, NULL);
1486 1486
1487 if (val & DROP_FREED)
1488 intel_gt_flush_buffer_pool(gt);
1489
1487 return 0;1490 return 0;
1488}1491}
1489 1492
drivers/gpu/drm/i915/i915_drv.h+29−8
@@ -108,8 +108,8 @@ · 107 unmodified lines
108 108
109#define DRIVER_NAME "i915"109#define DRIVER_NAME "i915"
110#define DRIVER_DESC "Intel Graphics"110#define DRIVER_DESC "Intel Graphics"
111#define DRIVER_DATE "20200430"111#define DRIVER_DATE "20200515"
112#define DRIVER_TIMESTAMP 1588234401112#define DRIVER_TIMESTAMP 1589543364
113 113
114struct drm_i915_gem_object;114struct drm_i915_gem_object;
115 115
@@ -148,6 +148,8 @@ enum hpd_pin { · 32 unmodified lines
148struct i915_hotplug {148struct i915_hotplug {
149 struct delayed_work hotplug_work;149 struct delayed_work hotplug_work;
150 150
151 const u32 *hpd, *pch_hpd;
152
151 struct {153 struct {
152 unsigned long last_jiffies;154 unsigned long last_jiffies;
153 int count;155 int count;
@@ -510,6 +512,7 @@ struct i915_psr { · 356 unmodified lines
510 u32 dc3co_exit_delay;512 u32 dc3co_exit_delay;
511 struct delayed_work dc3co_work;513 struct delayed_work dc3co_work;
512 bool force_mode_changed;514 bool force_mode_changed;
515 struct drm_dp_vsc_sdp vsc;
513};516};
514 517
515#define QUIRK_LVDS_SSC_DISABLE (1<<1)518#define QUIRK_LVDS_SSC_DISABLE (1<<1)
@@ -614,13 +617,14 @@ struct i915_gem_mm { · 98 unmodified lines
614 617
615#define I915_IDLE_ENGINES_TIMEOUT (200) /* in ms */618#define I915_IDLE_ENGINES_TIMEOUT (200) /* in ms */
616 619
617#define I915_RESET_TIMEOUT (10 * HZ) /* 10s */620unsigned long i915_fence_context_timeout(const struct drm_i915_private *i915,
618#define I915_FENCE_TIMEOUT (10 * HZ) /* 10s */621 u64 context);
619
620#define I915_ENGINE_DEAD_TIMEOUT (4 * HZ) /* Seqno, head and subunits dead */
621#define I915_SEQNO_DEAD_TIMEOUT (12 * HZ) /* Seqno dead with active head */
622 622
623#define I915_ENGINE_WEDGED_TIMEOUT (60 * HZ) /* Reset but no recovery? */623static inline unsigned long
624i915_fence_timeout(const struct drm_i915_private *i915)
625{
626 return i915_fence_context_timeout(i915, U64_MAX);
627}
624 628
625/* Amount of SAGV/QGV points, BSpec precisely defines this */629/* Amount of SAGV/QGV points, BSpec precisely defines this */
626#define I915_NUM_QGV_POINTS 8630#define I915_NUM_QGV_POINTS 8
@@ -1507,6 +1511,11 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915, · 880 unmodified lines
1507#define IS_ICL_REVID(p, since, until) \1511#define IS_ICL_REVID(p, since, until) \
1508 (IS_ICELAKE(p) && IS_REVID(p, since, until))1512 (IS_ICELAKE(p) && IS_REVID(p, since, until))
1509 1513
1514#define EHL_REVID_A0 0x0
1515
1516#define IS_EHL_REVID(p, since, until) \
1517 (IS_ELKHARTLAKE(p) && IS_REVID(p, since, until))
1518
1510#define TGL_REVID_A0 0x01519#define TGL_REVID_A0 0x0
1511#define TGL_REVID_B0 0x11520#define TGL_REVID_B0 0x1
1512#define TGL_REVID_C0 0x21521#define TGL_REVID_C0 0x2
@@ -1915,4 +1924,16 @@ i915_coherent_map_type(struct drm_i915_private *i915) · 402 unmodified lines
1915 return HAS_LLC(i915) ? I915_MAP_WB : I915_MAP_WC;1924 return HAS_LLC(i915) ? I915_MAP_WB : I915_MAP_WC;
1916}1925}
1917 1926
1927static inline u64 i915_cs_timestamp_ns_to_ticks(struct drm_i915_private *i915, u64 val)
1928{
1929 return DIV_ROUND_UP_ULL(val * RUNTIME_INFO(i915)->cs_timestamp_frequency_hz,
1930 1000000000);
1931}
1932
1933static inline u64 i915_cs_timestamp_ticks_to_ns(struct drm_i915_private *i915, u64 val)
1934{
1935 return div_u64(val * 1000000000,
1936 RUNTIME_INFO(i915)->cs_timestamp_frequency_hz);
1937}
1938
1918#endif1939#endif
drivers/gpu/drm/i915/i915_gem_evict.c+12−14
@@ -128,6 +128,13 @@ search_again: · 127 unmodified lines
128 active = NULL;128 active = NULL;
129 INIT_LIST_HEAD(&eviction_list);129 INIT_LIST_HEAD(&eviction_list);
130 list_for_each_entry_safe(vma, next, &vm->bound_list, vm_link) {130 list_for_each_entry_safe(vma, next, &vm->bound_list, vm_link) {
131 if (vma == active) { /* now seen this vma twice */
132 if (flags & PIN_NONBLOCK)
133 break;
134
135 active = ERR_PTR(-EAGAIN);
136 }
137
131 /*138 /*
132 * We keep this list in a rough least-recently scanned order139 * We keep this list in a rough least-recently scanned order
133 * of active elements (inactive elements are cheap to reap).140 * of active elements (inactive elements are cheap to reap).
@@ -143,21 +150,12 @@ search_again: · 9 unmodified lines
143 * To notice when we complete one full cycle, we record the150 * To notice when we complete one full cycle, we record the
144 * first active element seen, before moving it to the tail.151 * first active element seen, before moving it to the tail.
145 */152 */
146 if (i915_vma_is_active(vma)) {153 if (active != ERR_PTR(-EAGAIN) && i915_vma_is_active(vma)) {
147 if (vma == active) {154 if (!active)
148 if (flags & PIN_NONBLOCK)155 active = vma;
149 break;
150
151 active = ERR_PTR(-EAGAIN);
152 }
153
154 if (active != ERR_PTR(-EAGAIN)) {
155 if (!active)
156 active = vma;
157 156
158 list_move_tail(&vma->vm_link, &vm->bound_list);157 list_move_tail(&vma->vm_link, &vm->bound_list);
159 continue;158 continue;
160 }
161 }159 }
162 160
163 if (mark_free(&scan, vma, flags, &eviction_list))161 if (mark_free(&scan, vma, flags, &eviction_list))
drivers/gpu/drm/i915/i915_getparam.c+1−1
@@ -153,7 +153,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data, · 152 unmodified lines
153 return -ENODEV;153 return -ENODEV;
154 break;154 break;
155 case I915_PARAM_CS_TIMESTAMP_FREQUENCY:155 case I915_PARAM_CS_TIMESTAMP_FREQUENCY:
156 value = 1000 * RUNTIME_INFO(i915)->cs_timestamp_frequency_khz;156 value = RUNTIME_INFO(i915)->cs_timestamp_frequency_hz;
157 break;157 break;
158 case I915_PARAM_MMAP_GTT_COHERENT:158 case I915_PARAM_MMAP_GTT_COHERENT:
159 value = INTEL_INFO(i915)->has_coherent_ggtt;159 value = INTEL_INFO(i915)->has_coherent_ggtt;
drivers/gpu/drm/i915/i915_gpu_error.h+1−1
@@ -42,7 +42,7 @@ struct i915_vma_coredump { · 41 unmodified lines
42 int num_pages;42 int num_pages;
43 int page_count;43 int page_count;
44 int unused;44 int unused;
45 u32 *pages[0];45 u32 *pages[];
46};46};
47 47
48struct i915_request_coredump {48struct i915_request_coredump {
drivers/gpu/drm/i915/i915_irq.c+125−102
@@ -124,7 +124,6 @@ static const u32 hpd_status_i915[HPD_NUM_PINS] = { · 123 unmodified lines
124 [HPD_PORT_D] = PORTD_HOTPLUG_INT_STATUS,124 [HPD_PORT_D] = PORTD_HOTPLUG_INT_STATUS,
125};125};
126 126
127/* BXT hpd list */
128static const u32 hpd_bxt[HPD_NUM_PINS] = {127static const u32 hpd_bxt[HPD_NUM_PINS] = {
129 [HPD_PORT_A] = BXT_DE_PORT_HP_DDIA,128 [HPD_PORT_A] = BXT_DE_PORT_HP_DDIA,
130 [HPD_PORT_B] = BXT_DE_PORT_HP_DDIB,129 [HPD_PORT_B] = BXT_DE_PORT_HP_DDIB,
@@ -168,6 +167,49 @@ static const u32 hpd_tgp[HPD_NUM_PINS] = { · 37 unmodified lines
168 [HPD_PORT_I] = SDE_TC_HOTPLUG_ICP(PORT_TC6),167 [HPD_PORT_I] = SDE_TC_HOTPLUG_ICP(PORT_TC6),
169};168};
170 169
170static void intel_hpd_init_pins(struct drm_i915_private *dev_priv)
171{
172 struct i915_hotplug *hpd = &dev_priv->hotplug;
173
174 if (HAS_GMCH(dev_priv)) {
175 if (IS_G4X(dev_priv) || IS_VALLEYVIEW(dev_priv) ||
176 IS_CHERRYVIEW(dev_priv))
177 hpd->hpd = hpd_status_g4x;
178 else
179 hpd->hpd = hpd_status_i915;
180 return;
181 }
182
183 if (INTEL_GEN(dev_priv) >= 12)
184 hpd->hpd = hpd_gen12;
185 else if (INTEL_GEN(dev_priv) >= 11)
186 hpd->hpd = hpd_gen11;
187 else if (IS_GEN9_LP(dev_priv))
188 hpd->hpd = hpd_bxt;
189 else if (INTEL_GEN(dev_priv) >= 8)
190 hpd->hpd = hpd_bdw;
191 else if (INTEL_GEN(dev_priv) >= 7)
192 hpd->hpd = hpd_ivb;
193 else
194 hpd->hpd = hpd_ilk;
195
196 if (!HAS_PCH_SPLIT(dev_priv) || HAS_PCH_NOP(dev_priv))
197 return;
198
199 if (HAS_PCH_TGP(dev_priv) || HAS_PCH_JSP(dev_priv))
200 hpd->pch_hpd = hpd_tgp;
201 else if (HAS_PCH_ICP(dev_priv) || HAS_PCH_MCC(dev_priv))
202 hpd->pch_hpd = hpd_icp;
203 else if (HAS_PCH_CNP(dev_priv) || HAS_PCH_SPT(dev_priv))
204 hpd->pch_hpd = hpd_spt;
205 else if (HAS_PCH_LPT(dev_priv) || HAS_PCH_CPT(dev_priv))
206 hpd->pch_hpd = hpd_cpt;
207 else if (HAS_PCH_IBX(dev_priv))
208 hpd->pch_hpd = hpd_ibx;
209 else
210 MISSING_CASE(INTEL_PCH_TYPE(dev_priv));
211}
212
171static void213static void
172intel_handle_vblank(struct drm_i915_private *dev_priv, enum pipe pipe)214intel_handle_vblank(struct drm_i915_private *dev_priv, enum pipe pipe)
173{215{
@@ -1504,33 +1546,27 @@ static void i9xx_hpd_irq_handler(struct drm_i915_private *dev_priv, · 1330 unmodified lines
1504 u32 hotplug_status)1546 u32 hotplug_status)
1505{1547{
1506 u32 pin_mask = 0, long_mask = 0;1548 u32 pin_mask = 0, long_mask = 0;
1549 u32 hotplug_trigger;
1507 1550
1508 if (IS_G4X(dev_priv) || IS_VALLEYVIEW(dev_priv) ||1551 if (IS_G4X(dev_priv) ||
1509 IS_CHERRYVIEW(dev_priv)) {1552 IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
1510 u32 hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_G4X;1553 hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_G4X;
1511 1554 else
1512 if (hotplug_trigger) {1555 hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_I915;
1513 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1514 hotplug_trigger, hotplug_trigger,
1515 hpd_status_g4x,
1516 i9xx_port_hotplug_long_detect);
1517 1556
1518 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);1557 if (hotplug_trigger) {
1519 }1558 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1559 hotplug_trigger, hotplug_trigger,
1560 dev_priv->hotplug.hpd,
1561 i9xx_port_hotplug_long_detect);
1520 1562
1521 if (hotplug_status & DP_AUX_CHANNEL_MASK_INT_STATUS_G4X)1563 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
1522 dp_aux_irq_handler(dev_priv);
1523 } else {
1524 u32 hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_I915;
1525
1526 if (hotplug_trigger) {
1527 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1528 hotplug_trigger, hotplug_trigger,
1529 hpd_status_i915,
1530 i9xx_port_hotplug_long_detect);
1531 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
1532 }
1533 }1564 }
1565
1566 if ((IS_G4X(dev_priv) ||
1567 IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) &&
1568 hotplug_status & DP_AUX_CHANNEL_MASK_INT_STATUS_G4X)
1569 dp_aux_irq_handler(dev_priv);
1534}1570}
1535 1571
1536static irqreturn_t valleyview_irq_handler(int irq, void *arg)1572static irqreturn_t valleyview_irq_handler(int irq, void *arg)
@@ -1696,8 +1732,7 @@ static irqreturn_t cherryview_irq_handler(int irq, void *arg) · 159 unmodified lines
1696}1732}
1697 1733
1698static void ibx_hpd_irq_handler(struct drm_i915_private *dev_priv,1734static void ibx_hpd_irq_handler(struct drm_i915_private *dev_priv,
1699 u32 hotplug_trigger,1735 u32 hotplug_trigger)
1700 const u32 hpd[HPD_NUM_PINS])
1701{1736{
1702 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;1737 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;
1703 1738
@@ -1720,8 +1755,9 @@ static void ibx_hpd_irq_handler(struct drm_i915_private *dev_priv, · 16 unmodified lines
1720 if (!hotplug_trigger)1755 if (!hotplug_trigger)
1721 return;1756 return;
1722 1757
1723 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, hotplug_trigger,1758 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1724 dig_hotplug_reg, hpd,1759 hotplug_trigger, dig_hotplug_reg,
1760 dev_priv->hotplug.pch_hpd,
1725 pch_port_hotplug_long_detect);1761 pch_port_hotplug_long_detect);
1726 1762
1727 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);1763 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
@@ -1732,7 +1768,7 @@ static void ibx_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 4 unmodified lines
1732 enum pipe pipe;1768 enum pipe pipe;
1733 u32 hotplug_trigger = pch_iir & SDE_HOTPLUG_MASK;1769 u32 hotplug_trigger = pch_iir & SDE_HOTPLUG_MASK;
1734 1770
1735 ibx_hpd_irq_handler(dev_priv, hotplug_trigger, hpd_ibx);1771 ibx_hpd_irq_handler(dev_priv, hotplug_trigger);
1736 1772
1737 if (pch_iir & SDE_AUDIO_POWER_MASK) {1773 if (pch_iir & SDE_AUDIO_POWER_MASK) {
1738 int port = ffs((pch_iir & SDE_AUDIO_POWER_MASK) >>1774 int port = ffs((pch_iir & SDE_AUDIO_POWER_MASK) >>
@@ -1820,7 +1856,7 @@ static void cpt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 81 unmodified lines
1820 enum pipe pipe;1856 enum pipe pipe;
1821 u32 hotplug_trigger = pch_iir & SDE_HOTPLUG_MASK_CPT;1857 u32 hotplug_trigger = pch_iir & SDE_HOTPLUG_MASK_CPT;
1822 1858
1823 ibx_hpd_irq_handler(dev_priv, hotplug_trigger, hpd_cpt);1859 ibx_hpd_irq_handler(dev_priv, hotplug_trigger);
1824 1860
1825 if (pch_iir & SDE_AUDIO_POWER_MASK_CPT) {1861 if (pch_iir & SDE_AUDIO_POWER_MASK_CPT) {
1826 int port = ffs((pch_iir & SDE_AUDIO_POWER_MASK_CPT) >>1862 int port = ffs((pch_iir & SDE_AUDIO_POWER_MASK_CPT) >>
@@ -1857,22 +1893,18 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 30 unmodified lines
1857 u32 ddi_hotplug_trigger, tc_hotplug_trigger;1893 u32 ddi_hotplug_trigger, tc_hotplug_trigger;
1858 u32 pin_mask = 0, long_mask = 0;1894 u32 pin_mask = 0, long_mask = 0;
1859 bool (*tc_port_hotplug_long_detect)(enum hpd_pin pin, u32 val);1895 bool (*tc_port_hotplug_long_detect)(enum hpd_pin pin, u32 val);
1860 const u32 *pins;
1861 1896
1862 if (HAS_PCH_TGP(dev_priv)) {1897 if (HAS_PCH_TGP(dev_priv)) {
1863 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_TGP;1898 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_TGP;
1864 tc_hotplug_trigger = pch_iir & SDE_TC_MASK_TGP;1899 tc_hotplug_trigger = pch_iir & SDE_TC_MASK_TGP;
1865 tc_port_hotplug_long_detect = tgp_tc_port_hotplug_long_detect;1900 tc_port_hotplug_long_detect = tgp_tc_port_hotplug_long_detect;
1866 pins = hpd_tgp;
1867 } else if (HAS_PCH_JSP(dev_priv)) {1901 } else if (HAS_PCH_JSP(dev_priv)) {
1868 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_TGP;1902 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_TGP;
1869 tc_hotplug_trigger = 0;1903 tc_hotplug_trigger = 0;
1870 pins = hpd_tgp;
1871 } else if (HAS_PCH_MCC(dev_priv)) {1904 } else if (HAS_PCH_MCC(dev_priv)) {
1872 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_ICP;1905 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_ICP;
1873 tc_hotplug_trigger = pch_iir & SDE_TC_HOTPLUG_ICP(PORT_TC1);1906 tc_hotplug_trigger = pch_iir & SDE_TC_HOTPLUG_ICP(PORT_TC1);
1874 tc_port_hotplug_long_detect = icp_tc_port_hotplug_long_detect;1907 tc_port_hotplug_long_detect = icp_tc_port_hotplug_long_detect;
1875 pins = hpd_icp;
1876 } else {1908 } else {
1877 drm_WARN(&dev_priv->drm, !HAS_PCH_ICP(dev_priv),1909 drm_WARN(&dev_priv->drm, !HAS_PCH_ICP(dev_priv),
1878 "Unrecognized PCH type 0x%x\n",1910 "Unrecognized PCH type 0x%x\n",
@@ -1881,7 +1913,6 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 2 unmodified lines
1881 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_ICP;1913 ddi_hotplug_trigger = pch_iir & SDE_DDI_MASK_ICP;
1882 tc_hotplug_trigger = pch_iir & SDE_TC_MASK_ICP;1914 tc_hotplug_trigger = pch_iir & SDE_TC_MASK_ICP;
1883 tc_port_hotplug_long_detect = icp_tc_port_hotplug_long_detect;1915 tc_port_hotplug_long_detect = icp_tc_port_hotplug_long_detect;
1884 pins = hpd_icp;
1885 }1916 }
1886 1917
1887 if (ddi_hotplug_trigger) {1918 if (ddi_hotplug_trigger) {
@@ -1891,8 +1922,8 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 3 unmodified lines
1891 I915_WRITE(SHOTPLUG_CTL_DDI, dig_hotplug_reg);1922 I915_WRITE(SHOTPLUG_CTL_DDI, dig_hotplug_reg);
1892 1923
1893 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,1924 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1894 ddi_hotplug_trigger,1925 ddi_hotplug_trigger, dig_hotplug_reg,
1895 dig_hotplug_reg, pins,1926 dev_priv->hotplug.pch_hpd,
1896 icp_ddi_port_hotplug_long_detect);1927 icp_ddi_port_hotplug_long_detect);
1897 }1928 }
1898 1929
@@ -1903,8 +1934,8 @@ static void icp_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 4 unmodified lines
1903 I915_WRITE(SHOTPLUG_CTL_TC, dig_hotplug_reg);1934 I915_WRITE(SHOTPLUG_CTL_TC, dig_hotplug_reg);
1904 1935
1905 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,1936 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1906 tc_hotplug_trigger,1937 tc_hotplug_trigger, dig_hotplug_reg,
1907 dig_hotplug_reg, pins,1938 dev_priv->hotplug.pch_hpd,
1908 tc_port_hotplug_long_detect);1939 tc_port_hotplug_long_detect);
1909 }1940 }
1910 1941
@@ -1929,7 +1960,8 @@ static void spt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 18 unmodified lines
1929 I915_WRITE(PCH_PORT_HOTPLUG, dig_hotplug_reg);1960 I915_WRITE(PCH_PORT_HOTPLUG, dig_hotplug_reg);
1930 1961
1931 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,1962 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1932 hotplug_trigger, dig_hotplug_reg, hpd_spt,1963 hotplug_trigger, dig_hotplug_reg,
1964 dev_priv->hotplug.pch_hpd,
1933 spt_port_hotplug_long_detect);1965 spt_port_hotplug_long_detect);
1934 }1966 }
1935 1967
@@ -1940,7 +1972,8 @@ static void spt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 4 unmodified lines
1940 I915_WRITE(PCH_PORT_HOTPLUG2, dig_hotplug_reg);1972 I915_WRITE(PCH_PORT_HOTPLUG2, dig_hotplug_reg);
1941 1973
1942 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,1974 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1943 hotplug2_trigger, dig_hotplug_reg, hpd_spt,1975 hotplug2_trigger, dig_hotplug_reg,
1976 dev_priv->hotplug.pch_hpd,
1944 spt_port_hotplug2_long_detect);1977 spt_port_hotplug2_long_detect);
1945 }1978 }
1946 1979
@@ -1952,16 +1985,16 @@ static void spt_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir) · 5 unmodified lines
1952}1985}
1953 1986
1954static void ilk_hpd_irq_handler(struct drm_i915_private *dev_priv,1987static void ilk_hpd_irq_handler(struct drm_i915_private *dev_priv,
1955 u32 hotplug_trigger,1988 u32 hotplug_trigger)
1956 const u32 hpd[HPD_NUM_PINS])
1957{1989{
1958 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;1990 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;
1959 1991
1960 dig_hotplug_reg = I915_READ(DIGITAL_PORT_HOTPLUG_CNTRL);1992 dig_hotplug_reg = I915_READ(DIGITAL_PORT_HOTPLUG_CNTRL);
1961 I915_WRITE(DIGITAL_PORT_HOTPLUG_CNTRL, dig_hotplug_reg);1993 I915_WRITE(DIGITAL_PORT_HOTPLUG_CNTRL, dig_hotplug_reg);
1962 1994
1963 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, hotplug_trigger,1995 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
1964 dig_hotplug_reg, hpd,1996 hotplug_trigger, dig_hotplug_reg,
1997 dev_priv->hotplug.hpd,
1965 ilk_port_hotplug_long_detect);1998 ilk_port_hotplug_long_detect);
1966 1999
1967 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);2000 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
@@ -1974,7 +2007,7 @@ static void ilk_display_irq_handler(struct drm_i915_private *dev_priv, · 6 unmodified lines
1974 u32 hotplug_trigger = de_iir & DE_DP_A_HOTPLUG;2007 u32 hotplug_trigger = de_iir & DE_DP_A_HOTPLUG;
1975 2008
1976 if (hotplug_trigger)2009 if (hotplug_trigger)
1977 ilk_hpd_irq_handler(dev_priv, hotplug_trigger, hpd_ilk);2010 ilk_hpd_irq_handler(dev_priv, hotplug_trigger);
1978 2011
1979 if (de_iir & DE_AUX_CHANNEL_A)2012 if (de_iir & DE_AUX_CHANNEL_A)
1980 dp_aux_irq_handler(dev_priv);2013 dp_aux_irq_handler(dev_priv);
@@ -2020,7 +2053,7 @@ static void ivb_display_irq_handler(struct drm_i915_private *dev_priv, · 39 unmodified lines
2020 u32 hotplug_trigger = de_iir & DE_DP_A_HOTPLUG_IVB;2053 u32 hotplug_trigger = de_iir & DE_DP_A_HOTPLUG_IVB;
2021 2054
2022 if (hotplug_trigger)2055 if (hotplug_trigger)
2023 ilk_hpd_irq_handler(dev_priv, hotplug_trigger, hpd_ivb);2056 ilk_hpd_irq_handler(dev_priv, hotplug_trigger);
2024 2057
2025 if (de_iir & DE_ERR_INT_IVB)2058 if (de_iir & DE_ERR_INT_IVB)
2026 ivb_err_int_handler(dev_priv);2059 ivb_err_int_handler(dev_priv);
@@ -2130,16 +2163,16 @@ static irqreturn_t ilk_irq_handler(int irq, void *arg) · 103 unmodified lines
2130}2163}
2131 2164
2132static void bxt_hpd_irq_handler(struct drm_i915_private *dev_priv,2165static void bxt_hpd_irq_handler(struct drm_i915_private *dev_priv,
2133 u32 hotplug_trigger,2166 u32 hotplug_trigger)
2134 const u32 hpd[HPD_NUM_PINS])
2135{2167{
2136 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;2168 u32 dig_hotplug_reg, pin_mask = 0, long_mask = 0;
2137 2169
2138 dig_hotplug_reg = I915_READ(PCH_PORT_HOTPLUG);2170 dig_hotplug_reg = I915_READ(PCH_PORT_HOTPLUG);
2139 I915_WRITE(PCH_PORT_HOTPLUG, dig_hotplug_reg);2171 I915_WRITE(PCH_PORT_HOTPLUG, dig_hotplug_reg);
2140 2172
2141 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, hotplug_trigger,2173 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
2142 dig_hotplug_reg, hpd,2174 hotplug_trigger, dig_hotplug_reg,
2175 dev_priv->hotplug.hpd,
2143 bxt_port_hotplug_long_detect);2176 bxt_port_hotplug_long_detect);
2144 2177
2145 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);2178 intel_hpd_irq_handler(dev_priv, pin_mask, long_mask);
@@ -2151,15 +2184,11 @@ static void gen11_hpd_irq_handler(struct drm_i915_private *dev_priv, u32 iir) · 5 unmodified lines
2151 u32 trigger_tc = iir & GEN11_DE_TC_HOTPLUG_MASK;2184 u32 trigger_tc = iir & GEN11_DE_TC_HOTPLUG_MASK;
2152 u32 trigger_tbt = iir & GEN11_DE_TBT_HOTPLUG_MASK;2185 u32 trigger_tbt = iir & GEN11_DE_TBT_HOTPLUG_MASK;
2153 long_pulse_detect_func long_pulse_detect;2186 long_pulse_detect_func long_pulse_detect;
2154 const u32 *hpd;
2155 2187
2156 if (INTEL_GEN(dev_priv) >= 12) {2188 if (INTEL_GEN(dev_priv) >= 12)
2157 long_pulse_detect = gen12_port_hotplug_long_detect;2189 long_pulse_detect = gen12_port_hotplug_long_detect;
2158 hpd = hpd_gen12;2190 else
2159 } else {
2160 long_pulse_detect = gen11_port_hotplug_long_detect;2191 long_pulse_detect = gen11_port_hotplug_long_detect;
2161 hpd = hpd_gen11;
2162 }
2163 2192
2164 if (trigger_tc) {2193 if (trigger_tc) {
2165 u32 dig_hotplug_reg;2194 u32 dig_hotplug_reg;
@@ -2167,8 +2196,10 @@ static void gen11_hpd_irq_handler(struct drm_i915_private *dev_priv, u32 iir) · 1 unmodified line
2167 dig_hotplug_reg = I915_READ(GEN11_TC_HOTPLUG_CTL);2196 dig_hotplug_reg = I915_READ(GEN11_TC_HOTPLUG_CTL);
2168 I915_WRITE(GEN11_TC_HOTPLUG_CTL, dig_hotplug_reg);2197 I915_WRITE(GEN11_TC_HOTPLUG_CTL, dig_hotplug_reg);
2169 2198
2170 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, trigger_tc,2199 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
2171 dig_hotplug_reg, hpd, long_pulse_detect);2200 trigger_tc, dig_hotplug_reg,
2201 dev_priv->hotplug.hpd,
2202 long_pulse_detect);
2172 }2203 }
2173 2204
2174 if (trigger_tbt) {2205 if (trigger_tbt) {
@@ -2177,8 +2208,10 @@ static void gen11_hpd_irq_handler(struct drm_i915_private *dev_priv, u32 iir) · 2 unmodified lines
2177 dig_hotplug_reg = I915_READ(GEN11_TBT_HOTPLUG_CTL);2208 dig_hotplug_reg = I915_READ(GEN11_TBT_HOTPLUG_CTL);
2178 I915_WRITE(GEN11_TBT_HOTPLUG_CTL, dig_hotplug_reg);2209 I915_WRITE(GEN11_TBT_HOTPLUG_CTL, dig_hotplug_reg);
2179 2210
2180 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask, trigger_tbt,2211 intel_get_hpd_pins(dev_priv, &pin_mask, &long_mask,
2181 dig_hotplug_reg, hpd, long_pulse_detect);2212 trigger_tbt, dig_hotplug_reg,
2213 dev_priv->hotplug.hpd,
2214 long_pulse_detect);
2182 }2215 }
2183 2216
2184 if (pin_mask)2217 if (pin_mask)
@@ -2309,15 +2342,13 @@ gen8_de_irq_handler(struct drm_i915_private *dev_priv, u32 master_ctl) · 124 unmodified lines
2309 if (IS_GEN9_LP(dev_priv)) {2342 if (IS_GEN9_LP(dev_priv)) {
2310 tmp_mask = iir & BXT_DE_PORT_HOTPLUG_MASK;2343 tmp_mask = iir & BXT_DE_PORT_HOTPLUG_MASK;
2311 if (tmp_mask) {2344 if (tmp_mask) {
2312 bxt_hpd_irq_handler(dev_priv, tmp_mask,2345 bxt_hpd_irq_handler(dev_priv, tmp_mask);
2313 hpd_bxt);
2314 found = true;2346 found = true;
2315 }2347 }
2316 } else if (IS_BROADWELL(dev_priv)) {2348 } else if (IS_BROADWELL(dev_priv)) {
2317 tmp_mask = iir & GEN8_PORT_DP_A_HOTPLUG;2349 tmp_mask = iir & GEN8_PORT_DP_A_HOTPLUG;
2318 if (tmp_mask) {2350 if (tmp_mask) {
2319 ilk_hpd_irq_handler(dev_priv,2351 ilk_hpd_irq_handler(dev_priv, tmp_mask);
2320 tmp_mask, hpd_bdw);
2321 found = true;2352 found = true;
2322 }2353 }
2323 }2354 }
@@ -2870,6 +2901,14 @@ static void gen11_display_irq_reset(struct drm_i915_private *dev_priv) · 546 unmodified lines
2870 2901
2871 if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)2902 if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)
2872 GEN3_IRQ_RESET(uncore, SDE);2903 GEN3_IRQ_RESET(uncore, SDE);
2904
2905 /* Wa_14010685332:icl */
2906 if (INTEL_PCH_TYPE(dev_priv) == PCH_ICP) {
2907 intel_uncore_rmw(uncore, SOUTH_CHICKEN1,
2908 SBCLK_RUN_REFCLK_DIS, SBCLK_RUN_REFCLK_DIS);
2909 intel_uncore_rmw(uncore, SOUTH_CHICKEN1,
2910 SBCLK_RUN_REFCLK_DIS, 0);
2911 }
2873}2912}
2874 2913
2875static void gen11_irq_reset(struct drm_i915_private *dev_priv)2914static void gen11_irq_reset(struct drm_i915_private *dev_priv)
@@ -2989,13 +3028,12 @@ static void ibx_hpd_irq_setup(struct drm_i915_private *dev_priv) · 113 unmodified lines
2989{3028{
2990 u32 hotplug_irqs, enabled_irqs;3029 u32 hotplug_irqs, enabled_irqs;
2991 3030
2992 if (HAS_PCH_IBX(dev_priv)) {3031 if (HAS_PCH_IBX(dev_priv))
2993 hotplug_irqs = SDE_HOTPLUG_MASK;3032 hotplug_irqs = SDE_HOTPLUG_MASK;
2994 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_ibx);3033 else
2995 } else {
2996 hotplug_irqs = SDE_HOTPLUG_MASK_CPT;3034 hotplug_irqs = SDE_HOTPLUG_MASK_CPT;
2997 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_cpt);3035
2998 }3036 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.pch_hpd);
2999 3037
3000 ibx_display_interrupt_update(dev_priv, hotplug_irqs, enabled_irqs);3038 ibx_display_interrupt_update(dev_priv, hotplug_irqs, enabled_irqs);
3001 3039
@@ -3021,13 +3059,12 @@ static void icp_hpd_detection_setup(struct drm_i915_private *dev_priv, · 19 unmodified lines
3021 3059
3022static void icp_hpd_irq_setup(struct drm_i915_private *dev_priv,3060static void icp_hpd_irq_setup(struct drm_i915_private *dev_priv,
3023 u32 sde_ddi_mask, u32 sde_tc_mask,3061 u32 sde_ddi_mask, u32 sde_tc_mask,
3024 u32 ddi_enable_mask, u32 tc_enable_mask,3062 u32 ddi_enable_mask, u32 tc_enable_mask)
3025 const u32 *pins)
3026{3063{
3027 u32 hotplug_irqs, enabled_irqs;3064 u32 hotplug_irqs, enabled_irqs;
3028 3065
3029 hotplug_irqs = sde_ddi_mask | sde_tc_mask;3066 hotplug_irqs = sde_ddi_mask | sde_tc_mask;
3030 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, pins);3067 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.pch_hpd);
3031 3068
3032 I915_WRITE(SHPD_FILTER_CNT, SHPD_FILTER_CNT_500_ADJ);3069 I915_WRITE(SHPD_FILTER_CNT, SHPD_FILTER_CNT_500_ADJ);
3033 3070
@@ -3044,8 +3081,7 @@ static void mcc_hpd_irq_setup(struct drm_i915_private *dev_priv) · 10 unmodified lines
3044{3081{
3045 icp_hpd_irq_setup(dev_priv,3082 icp_hpd_irq_setup(dev_priv,
3046 SDE_DDI_MASK_ICP, SDE_TC_HOTPLUG_ICP(PORT_TC1),3083 SDE_DDI_MASK_ICP, SDE_TC_HOTPLUG_ICP(PORT_TC1),
3047 ICP_DDI_HPD_ENABLE_MASK, ICP_TC_HPD_ENABLE(PORT_TC1),3084 ICP_DDI_HPD_ENABLE_MASK, ICP_TC_HPD_ENABLE(PORT_TC1));
3048 hpd_icp);
3049}3085}
3050 3086
3051/*3087/*
@@ -3057,8 +3093,7 @@ static void jsp_hpd_irq_setup(struct drm_i915_private *dev_priv) · 5 unmodified lines
3057{3093{
3058 icp_hpd_irq_setup(dev_priv,3094 icp_hpd_irq_setup(dev_priv,
3059 SDE_DDI_MASK_TGP, 0,3095 SDE_DDI_MASK_TGP, 0,
3060 TGP_DDI_HPD_ENABLE_MASK, 0,3096 TGP_DDI_HPD_ENABLE_MASK, 0);
3061 hpd_tgp);
3062}3097}
3063 3098
3064static void gen11_hpd_detection_setup(struct drm_i915_private *dev_priv)3099static void gen11_hpd_detection_setup(struct drm_i915_private *dev_priv)
@@ -3083,11 +3118,9 @@ static void gen11_hpd_detection_setup(struct drm_i915_private *dev_priv) · 18 unmodified lines
3083static void gen11_hpd_irq_setup(struct drm_i915_private *dev_priv)3118static void gen11_hpd_irq_setup(struct drm_i915_private *dev_priv)
3084{3119{
3085 u32 hotplug_irqs, enabled_irqs;3120 u32 hotplug_irqs, enabled_irqs;
3086 const u32 *hpd;
3087 u32 val;3121 u32 val;
3088 3122
3089 hpd = INTEL_GEN(dev_priv) >= 12 ? hpd_gen12 : hpd_gen11;3123 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
3090 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd);
3091 hotplug_irqs = GEN11_DE_TC_HOTPLUG_MASK | GEN11_DE_TBT_HOTPLUG_MASK;3124 hotplug_irqs = GEN11_DE_TC_HOTPLUG_MASK | GEN11_DE_TBT_HOTPLUG_MASK;
3092 3125
3093 val = I915_READ(GEN11_DE_HPD_IMR);3126 val = I915_READ(GEN11_DE_HPD_IMR);
@@ -3099,12 +3132,10 @@ static void gen11_hpd_irq_setup(struct drm_i915_private *dev_priv) · 5 unmodified lines
3099 3132
3100 if (INTEL_PCH_TYPE(dev_priv) >= PCH_TGP)3133 if (INTEL_PCH_TYPE(dev_priv) >= PCH_TGP)
3101 icp_hpd_irq_setup(dev_priv, SDE_DDI_MASK_TGP, SDE_TC_MASK_TGP,3134 icp_hpd_irq_setup(dev_priv, SDE_DDI_MASK_TGP, SDE_TC_MASK_TGP,
3102 TGP_DDI_HPD_ENABLE_MASK,3135 TGP_DDI_HPD_ENABLE_MASK, TGP_TC_HPD_ENABLE_MASK);
3103 TGP_TC_HPD_ENABLE_MASK, hpd_tgp);
3104 else if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)3136 else if (INTEL_PCH_TYPE(dev_priv) >= PCH_ICP)
3105 icp_hpd_irq_setup(dev_priv, SDE_DDI_MASK_ICP, SDE_TC_MASK_ICP,3137 icp_hpd_irq_setup(dev_priv, SDE_DDI_MASK_ICP, SDE_TC_MASK_ICP,
3106 ICP_DDI_HPD_ENABLE_MASK,3138 ICP_DDI_HPD_ENABLE_MASK, ICP_TC_HPD_ENABLE_MASK);
3107 ICP_TC_HPD_ENABLE_MASK, hpd_icp);
3108}3139}
3109 3140
3110static void spt_hpd_detection_setup(struct drm_i915_private *dev_priv)3141static void spt_hpd_detection_setup(struct drm_i915_private *dev_priv)
@@ -3140,7 +3171,7 @@ static void spt_hpd_irq_setup(struct drm_i915_private *dev_priv) · 29 unmodified lines
3140 I915_WRITE(SHPD_FILTER_CNT, SHPD_FILTER_CNT_500_ADJ);3171 I915_WRITE(SHPD_FILTER_CNT, SHPD_FILTER_CNT_500_ADJ);
3141 3172
3142 hotplug_irqs = SDE_HOTPLUG_MASK_SPT;3173 hotplug_irqs = SDE_HOTPLUG_MASK_SPT;
3143 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_spt);3174 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.pch_hpd);
3144 3175
3145 ibx_display_interrupt_update(dev_priv, hotplug_irqs, enabled_irqs);3176 ibx_display_interrupt_update(dev_priv, hotplug_irqs, enabled_irqs);
3146 3177
@@ -3169,17 +3200,17 @@ static void ilk_hpd_irq_setup(struct drm_i915_private *dev_priv) · 22 unmodified lines
3169 3200
3170 if (INTEL_GEN(dev_priv) >= 8) {3201 if (INTEL_GEN(dev_priv) >= 8) {
3171 hotplug_irqs = GEN8_PORT_DP_A_HOTPLUG;3202 hotplug_irqs = GEN8_PORT_DP_A_HOTPLUG;
3172 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_bdw);3203 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
3173 3204
3174 bdw_update_port_irq(dev_priv, hotplug_irqs, enabled_irqs);3205 bdw_update_port_irq(dev_priv, hotplug_irqs, enabled_irqs);
3175 } else if (INTEL_GEN(dev_priv) >= 7) {3206 } else if (INTEL_GEN(dev_priv) >= 7) {
3176 hotplug_irqs = DE_DP_A_HOTPLUG_IVB;3207 hotplug_irqs = DE_DP_A_HOTPLUG_IVB;
3177 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_ivb);3208 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
3178 3209
3179 ilk_update_display_irq(dev_priv, hotplug_irqs, enabled_irqs);3210 ilk_update_display_irq(dev_priv, hotplug_irqs, enabled_irqs);
3180 } else {3211 } else {
3181 hotplug_irqs = DE_DP_A_HOTPLUG;3212 hotplug_irqs = DE_DP_A_HOTPLUG;
3182 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_ilk);3213 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
3183 3214
3184 ilk_update_display_irq(dev_priv, hotplug_irqs, enabled_irqs);3215 ilk_update_display_irq(dev_priv, hotplug_irqs, enabled_irqs);
3185 }3216 }
@@ -3230,7 +3261,7 @@ static void bxt_hpd_irq_setup(struct drm_i915_private *dev_priv) · 44 unmodified lines
3230{3261{
3231 u32 hotplug_irqs, enabled_irqs;3262 u32 hotplug_irqs, enabled_irqs;
3232 3263
3233 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, hpd_bxt);3264 enabled_irqs = intel_hpd_enabled_irqs(dev_priv, dev_priv->hotplug.hpd);
3234 hotplug_irqs = BXT_DE_PORT_HOTPLUG_MASK;3265 hotplug_irqs = BXT_DE_PORT_HOTPLUG_MASK;
3235 3266
3236 bdw_update_port_irq(dev_priv, hotplug_irqs, enabled_irqs);3267 bdw_update_port_irq(dev_priv, hotplug_irqs, enabled_irqs);
@@ -3361,7 +3392,7 @@ static void gen8_de_irq_postinstall(struct drm_i915_private *dev_priv) · 124 unmodified lines
3361 u32 de_pipe_masked = gen8_de_pipe_fault_mask(dev_priv) |3392 u32 de_pipe_masked = gen8_de_pipe_fault_mask(dev_priv) |
3362 GEN8_PIPE_CDCLK_CRC_DONE;3393 GEN8_PIPE_CDCLK_CRC_DONE;
3363 u32 de_pipe_enables;3394 u32 de_pipe_enables;
3364 u32 de_port_masked = GEN8_AUX_CHANNEL_A;3395 u32 de_port_masked = gen8_de_port_aux_mask(dev_priv);
3365 u32 de_port_enables;3396 u32 de_port_enables;
3366 u32 de_misc_masked = GEN8_DE_EDP_PSR;3397 u32 de_misc_masked = GEN8_DE_EDP_PSR;
3367 enum pipe pipe;3398 enum pipe pipe;
@@ -3369,18 +3400,8 @@ static void gen8_de_irq_postinstall(struct drm_i915_private *dev_priv) · 1 unmodified line
3369 if (INTEL_GEN(dev_priv) <= 10)3400 if (INTEL_GEN(dev_priv) <= 10)
3370 de_misc_masked |= GEN8_DE_MISC_GSE;3401 de_misc_masked |= GEN8_DE_MISC_GSE;
3371 3402
3372 if (INTEL_GEN(dev_priv) >= 9) {3403 if (IS_GEN9_LP(dev_priv))
3373 de_port_masked |= GEN9_AUX_CHANNEL_B | GEN9_AUX_CHANNEL_C |3404 de_port_masked |= BXT_DE_PORT_GMBUS;
3374 GEN9_AUX_CHANNEL_D;
3375 if (IS_GEN9_LP(dev_priv))
3376 de_port_masked |= BXT_DE_PORT_GMBUS;
3377 }
3378
3379 if (INTEL_GEN(dev_priv) >= 11)
3380 de_port_masked |= ICL_AUX_CHANNEL_E;
3381
3382 if (IS_CNL_WITH_PORT_F(dev_priv) || INTEL_GEN(dev_priv) >= 11)
3383 de_port_masked |= CNL_AUX_CHANNEL_F;
3384 3405
3385 de_pipe_enables = de_pipe_masked | GEN8_PIPE_VBLANK |3406 de_pipe_enables = de_pipe_masked | GEN8_PIPE_VBLANK |
3386 GEN8_PIPE_FIFO_UNDERRUN;3407 GEN8_PIPE_FIFO_UNDERRUN;
@@ -3934,6 +3955,8 @@ void intel_irq_init(struct drm_i915_private *dev_priv) · 547 unmodified lines
3934 struct drm_device *dev = &dev_priv->drm;3955 struct drm_device *dev = &dev_priv->drm;
3935 int i;3956 int i;
3936 3957
3958 intel_hpd_init_pins(dev_priv);
3959
3937 intel_hpd_init_work(dev_priv);3960 intel_hpd_init_work(dev_priv);
3938 3961
3939 INIT_WORK(&dev_priv->l3_parity.error_work, ivb_parity_work);3962 INIT_WORK(&dev_priv->l3_parity.error_work, ivb_parity_work);
drivers/gpu/drm/i915/i915_pci.c+1−1
@@ -615,7 +615,7 @@ static const struct intel_device_info chv_info = { · 614 unmodified lines
615 .has_logical_ring_contexts = 1,615 .has_logical_ring_contexts = 1,
616 .display.has_gmch = 1,616 .display.has_gmch = 1,
617 .dma_mask_size = 39,617 .dma_mask_size = 39,
618 .ppgtt_type = INTEL_PPGTT_ALIASING,618 .ppgtt_type = INTEL_PPGTT_FULL,
619 .ppgtt_size = 32,619 .ppgtt_size = 32,
620 .has_reset_engine = 1,620 .has_reset_engine = 1,
621 .has_snoop = true,621 .has_snoop = true,
drivers/gpu/drm/i915/i915_perf.c+4−8
@@ -1612,10 +1612,7 @@ static int alloc_noa_wait(struct i915_perf_stream *stream) · 1611 unmodified lines
1612 struct drm_i915_gem_object *bo;1612 struct drm_i915_gem_object *bo;
1613 struct i915_vma *vma;1613 struct i915_vma *vma;
1614 const u64 delay_ticks = 0xffffffffffffffff -1614 const u64 delay_ticks = 0xffffffffffffffff -
1615 DIV64_U64_ROUND_UP(1615 i915_cs_timestamp_ns_to_ticks(i915, atomic64_read(&stream->perf->noa_programming_delay));
1616 atomic64_read(&stream->perf->noa_programming_delay) *
1617 RUNTIME_INFO(i915)->cs_timestamp_frequency_khz,
1618 1000000ull);
1619 const u32 base = stream->engine->mmio_base;1616 const u32 base = stream->engine->mmio_base;
1620#define CS_GPR(x) GEN8_RING_CS_GPR(base, x)1617#define CS_GPR(x) GEN8_RING_CS_GPR(base, x)
1621 u32 *batch, *ts0, *cs, *jump;1618 u32 *batch, *ts0, *cs, *jump;
@@ -3485,8 +3482,7 @@ err: · 1863 unmodified lines
3485 3482
3486static u64 oa_exponent_to_ns(struct i915_perf *perf, int exponent)3483static u64 oa_exponent_to_ns(struct i915_perf *perf, int exponent)
3487{3484{
3488 return div64_u64(1000000000ULL * (2ULL << exponent),3485 return i915_cs_timestamp_ticks_to_ns(perf->i915, 2ULL << exponent);
3489 1000ULL * RUNTIME_INFO(perf->i915)->cs_timestamp_frequency_khz);
3490}3486}
3491 3487
3492/**3488/**
@@ -4344,8 +4340,8 @@ void i915_perf_init(struct drm_i915_private *i915) · 851 unmodified lines
4344 if (perf->ops.enable_metric_set) {4340 if (perf->ops.enable_metric_set) {
4345 mutex_init(&perf->lock);4341 mutex_init(&perf->lock);
4346 4342
4347 oa_sample_rate_hard_limit = 1000 *4343 oa_sample_rate_hard_limit =
4348 (RUNTIME_INFO(i915)->cs_timestamp_frequency_khz / 2);4344 RUNTIME_INFO(i915)->cs_timestamp_frequency_hz / 2;
4349 4345
4350 mutex_init(&perf->metrics_lock);4346 mutex_init(&perf->metrics_lock);
4351 idr_init(&perf->metrics_idr);4347 idr_init(&perf->metrics_idr);
drivers/gpu/drm/i915/i915_pmu.c+4−1
@@ -442,6 +442,7 @@ static u64 count_interrupts(struct drm_i915_private *i915) · 441 unmodified lines
442static void i915_pmu_event_destroy(struct perf_event *event)442static void i915_pmu_event_destroy(struct perf_event *event)
443{443{
444 WARN_ON(event->parent);444 WARN_ON(event->parent);
445 module_put(THIS_MODULE);
445}446}
446 447
447static int448static int
@@ -533,8 +534,10 @@ static int i915_pmu_event_init(struct perf_event *event) · 85 unmodified lines
533 if (ret)534 if (ret)
534 return ret;535 return ret;
535 536
536 if (!event->parent)537 if (!event->parent) {
538 __module_get(THIS_MODULE);
537 event->destroy = i915_pmu_event_destroy;539 event->destroy = i915_pmu_event_destroy;
540 }
538 541
539 return 0;542 return 0;
540}543}
drivers/gpu/drm/i915/i915_priolist_types.h+1−6
@@ -24,15 +24,12 @@ enum { · 23 unmodified lines
24 I915_PRIORITY_DISPLAY,24 I915_PRIORITY_DISPLAY,
25};25};
26 26
27#define I915_USER_PRIORITY_SHIFT 227#define I915_USER_PRIORITY_SHIFT 0
28#define I915_USER_PRIORITY(x) ((x) << I915_USER_PRIORITY_SHIFT)28#define I915_USER_PRIORITY(x) ((x) << I915_USER_PRIORITY_SHIFT)
29 29
30#define I915_PRIORITY_COUNT BIT(I915_USER_PRIORITY_SHIFT)30#define I915_PRIORITY_COUNT BIT(I915_USER_PRIORITY_SHIFT)
31#define I915_PRIORITY_MASK (I915_PRIORITY_COUNT - 1)31#define I915_PRIORITY_MASK (I915_PRIORITY_COUNT - 1)
32 32
33#define I915_PRIORITY_WAIT ((u8)BIT(0))
34#define I915_PRIORITY_NOSEMAPHORE ((u8)BIT(1))
35
36/* Smallest priority value that cannot be bumped. */33/* Smallest priority value that cannot be bumped. */
37#define I915_PRIORITY_INVALID (INT_MIN | (u8)I915_PRIORITY_MASK)34#define I915_PRIORITY_INVALID (INT_MIN | (u8)I915_PRIORITY_MASK)
38 35
@@ -47,8 +44,6 @@ enum { · 8 unmodified lines
47#define I915_PRIORITY_UNPREEMPTABLE INT_MAX44#define I915_PRIORITY_UNPREEMPTABLE INT_MAX
48#define I915_PRIORITY_BARRIER INT_MAX45#define I915_PRIORITY_BARRIER INT_MAX
49 46
50#define __NO_PREEMPTION (I915_PRIORITY_WAIT)
51
52struct i915_priolist {47struct i915_priolist {
53 struct list_head requests[I915_PRIORITY_COUNT];48 struct list_head requests[I915_PRIORITY_COUNT];
54 struct rb_node node;49 struct rb_node node;
drivers/gpu/drm/i915/i915_reg.h+13−0
@@ -2557,6 +2557,14 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg) · 2556 unmodified lines
2557#define GEN10_PAT_INDEX(index) _MMIO(0x40e0 + (index) * 4)2557#define GEN10_PAT_INDEX(index) _MMIO(0x40e0 + (index) * 4)
2558#define GEN12_PAT_INDEX(index) _MMIO(0x4800 + (index) * 4)2558#define GEN12_PAT_INDEX(index) _MMIO(0x4800 + (index) * 4)
2559#define BSD_HWS_PGA_GEN7 _MMIO(0x04180)2559#define BSD_HWS_PGA_GEN7 _MMIO(0x04180)
2560#define GEN12_GFX_CCS_AUX_NV _MMIO(0x4208)
2561#define GEN12_VD0_AUX_NV _MMIO(0x4218)
2562#define GEN12_VD1_AUX_NV _MMIO(0x4228)
2563#define GEN12_VD2_AUX_NV _MMIO(0x4298)
2564#define GEN12_VD3_AUX_NV _MMIO(0x42A8)
2565#define GEN12_VE0_AUX_NV _MMIO(0x4238)
2566#define GEN12_VE1_AUX_NV _MMIO(0x42B8)
2567#define AUX_INV REG_BIT(0)
2560#define BLT_HWS_PGA_GEN7 _MMIO(0x04280)2568#define BLT_HWS_PGA_GEN7 _MMIO(0x04280)
2561#define VEBOX_HWS_PGA_GEN7 _MMIO(0x04380)2569#define VEBOX_HWS_PGA_GEN7 _MMIO(0x04380)
2562#define RING_ACTHD(base) _MMIO((base) + 0x74)2570#define RING_ACTHD(base) _MMIO((base) + 0x74)
@@ -8573,6 +8581,7 @@ enum { · 6010 unmodified lines
8573#define FDI_BC_BIFURCATION_SELECT (1 << 12)8581#define FDI_BC_BIFURCATION_SELECT (1 << 12)
8574#define CHASSIS_CLK_REQ_DURATION_MASK (0xf << 8)8582#define CHASSIS_CLK_REQ_DURATION_MASK (0xf << 8)
8575#define CHASSIS_CLK_REQ_DURATION(x) ((x) << 8)8583#define CHASSIS_CLK_REQ_DURATION(x) ((x) << 8)
8584#define SBCLK_RUN_REFCLK_DIS (1 << 7)
8576#define SPT_PWM_GRANULARITY (1 << 0)8585#define SPT_PWM_GRANULARITY (1 << 0)
8577#define SOUTH_CHICKEN2 _MMIO(0xc2004)8586#define SOUTH_CHICKEN2 _MMIO(0xc2004)
8578#define FDI_MPHY_IOSFSB_RESET_STATUS (1 << 13)8587#define FDI_MPHY_IOSFSB_RESET_STATUS (1 << 13)
@@ -9064,6 +9073,7 @@ enum { · 485 unmodified lines
9064#define GEN7_PCODE_ILLEGAL_DATA 0x39073#define GEN7_PCODE_ILLEGAL_DATA 0x3
9065#define GEN11_PCODE_ILLEGAL_SUBCOMMAND 0x49074#define GEN11_PCODE_ILLEGAL_SUBCOMMAND 0x4
9066#define GEN11_PCODE_LOCKED 0x69075#define GEN11_PCODE_LOCKED 0x6
9076#define GEN11_PCODE_REJECTED 0x11
9067#define GEN7_PCODE_MIN_FREQ_TABLE_GT_RATIO_OUT_OF_RANGE 0x109077#define GEN7_PCODE_MIN_FREQ_TABLE_GT_RATIO_OUT_OF_RANGE 0x10
9068#define GEN6_PCODE_WRITE_RC6VIDS 0x49078#define GEN6_PCODE_WRITE_RC6VIDS 0x4
9069#define GEN6_PCODE_READ_RC6VIDS 0x59079#define GEN6_PCODE_READ_RC6VIDS 0x5
@@ -9085,6 +9095,9 @@ enum { · 15 unmodified lines
9085#define ICL_PCODE_MEM_SUBSYSYSTEM_INFO 0xd9095#define ICL_PCODE_MEM_SUBSYSYSTEM_INFO 0xd
9086#define ICL_PCODE_MEM_SS_READ_GLOBAL_INFO (0x0 << 8)9096#define ICL_PCODE_MEM_SS_READ_GLOBAL_INFO (0x0 << 8)
9087#define ICL_PCODE_MEM_SS_READ_QGV_POINT_INFO(point) (((point) << 16) | (0x1 << 8))9097#define ICL_PCODE_MEM_SS_READ_QGV_POINT_INFO(point) (((point) << 16) | (0x1 << 8))
9098#define ICL_PCODE_SAGV_DE_MEM_SS_CONFIG 0xe
9099#define ICL_PCODE_POINTS_RESTRICTED 0x0
9100#define ICL_PCODE_POINTS_RESTRICTED_MASK 0x1
9088#define GEN6_PCODE_READ_D_COMP 0x109101#define GEN6_PCODE_READ_D_COMP 0x10
9089#define GEN6_PCODE_WRITE_D_COMP 0x119102#define GEN6_PCODE_WRITE_D_COMP 0x11
9090#define ICL_PCODE_EXIT_TCCOLD 0x129103#define ICL_PCODE_EXIT_TCCOLD 0x12
drivers/gpu/drm/i915/i915_request.c+82−45
@@ -23,6 +23,7 @@ · 22 unmodified lines
23 */23 */
24 24
25#include <linux/dma-fence-array.h>25#include <linux/dma-fence-array.h>
26#include <linux/dma-fence-chain.h>
26#include <linux/irq_work.h>27#include <linux/irq_work.h>
27#include <linux/prefetch.h>28#include <linux/prefetch.h>
28#include <linux/sched.h>29#include <linux/sched.h>
@@ -367,8 +368,6 @@ __await_execution(struct i915_request *rq, · 338 unmodified lines
367 }368 }
368 spin_unlock_irq(&signal->lock);369 spin_unlock_irq(&signal->lock);
369 370
370 /* Copy across semaphore status as we need the same behaviour */
371 rq->sched.flags |= signal->sched.flags;
372 return 0;371 return 0;
373}372}
374 373
@@ -536,10 +535,8 @@ void __i915_request_unsubmit(struct i915_request *request) · 161 unmodified lines
536 spin_unlock(&request->lock);535 spin_unlock(&request->lock);
537 536
538 /* We've already spun, don't charge on resubmitting. */537 /* We've already spun, don't charge on resubmitting. */
539 if (request->sched.semaphores && i915_request_started(request)) {538 if (request->sched.semaphores && i915_request_started(request))
540 request->sched.attr.priority |= I915_PRIORITY_NOSEMAPHORE;
541 request->sched.semaphores = 0;539 request->sched.semaphores = 0;
542 }
543 540
544 /*541 /*
545 * We don't need to wake_up any waiters on request->execute, they542 * We don't need to wake_up any waiters on request->execute, they
@@ -597,15 +594,6 @@ submit_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state) · 51 unmodified lines
597 return NOTIFY_DONE;594 return NOTIFY_DONE;
598}595}
599 596
600static void irq_semaphore_cb(struct irq_work *wrk)
601{
602 struct i915_request *rq =
603 container_of(wrk, typeof(*rq), semaphore_work);
604
605 i915_schedule_bump_priority(rq, I915_PRIORITY_NOSEMAPHORE);
606 i915_request_put(rq);
607}
608
609static int __i915_sw_fence_call597static int __i915_sw_fence_call
610semaphore_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)598semaphore_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
611{599{
@@ -613,11 +601,6 @@ semaphore_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state) · 1 unmodified line
613 601
614 switch (state) {602 switch (state) {
615 case FENCE_COMPLETE:603 case FENCE_COMPLETE:
616 if (!(READ_ONCE(rq->sched.attr.priority) & I915_PRIORITY_NOSEMAPHORE)) {
617 i915_request_get(rq);
618 init_irq_work(&rq->semaphore_work, irq_semaphore_cb);
619 irq_work_queue(&rq->semaphore_work);
620 }
621 break;604 break;
622 605
623 case FENCE_FREE:606 case FENCE_FREE:
@@ -950,6 +933,7 @@ __emit_semaphore_wait(struct i915_request *to, · 326 unmodified lines
950 u32 *cs;933 u32 *cs;
951 934
952 GEM_BUG_ON(INTEL_GEN(to->i915) < 8);935 GEM_BUG_ON(INTEL_GEN(to->i915) < 8);
936 GEM_BUG_ON(i915_request_has_initial_breadcrumb(to));
953 937
954 /* We need to pin the signaler's HWSP until we are finished reading. */938 /* We need to pin the signaler's HWSP until we are finished reading. */
955 err = intel_timeline_read_hwsp(from, to, &hwsp_offset);939 err = intel_timeline_read_hwsp(from, to, &hwsp_offset);
@@ -995,13 +979,26 @@ emit_semaphore_wait(struct i915_request *to, · 39 unmodified lines
995 gfp_t gfp)979 gfp_t gfp)
996{980{
997 const intel_engine_mask_t mask = READ_ONCE(from->engine)->mask;981 const intel_engine_mask_t mask = READ_ONCE(from->engine)->mask;
982 struct i915_sw_fence *wait = &to->submit;
998 983
999 if (!intel_context_use_semaphores(to->context))984 if (!intel_context_use_semaphores(to->context))
1000 goto await_fence;985 goto await_fence;
1001 986
987 if (i915_request_has_initial_breadcrumb(to))
988 goto await_fence;
989
1002 if (!rcu_access_pointer(from->hwsp_cacheline))990 if (!rcu_access_pointer(from->hwsp_cacheline))
1003 goto await_fence;991 goto await_fence;
1004 992
993 /*
994 * If this or its dependents are waiting on an external fence
995 * that may fail catastrophically, then we want to avoid using
996 * sempahores as they bypass the fence signaling metadata, and we
997 * lose the fence->error propagation.
998 */
999 if (from->sched.flags & I915_SCHED_HAS_EXTERNAL_CHAIN)
1000 goto await_fence;
1001
1005 /* Just emit the first semaphore we see as request space is limited. */1002 /* Just emit the first semaphore we see as request space is limited. */
1006 if (already_busywaiting(to) & mask)1003 if (already_busywaiting(to) & mask)
1007 goto await_fence;1004 goto await_fence;
@@ -1017,11 +1014,10 @@ emit_semaphore_wait(struct i915_request *to, · 9 unmodified lines
1017 goto await_fence;1014 goto await_fence;
1018 1015
1019 to->sched.semaphores |= mask;1016 to->sched.semaphores |= mask;
1020 to->sched.flags |= I915_SCHED_HAS_SEMAPHORE_CHAIN;1017 wait = &to->semaphore;
1021 return 0;
1022 1018
1023await_fence:1019await_fence:
1024 return i915_sw_fence_await_dma_fence(&to->submit,1020 return i915_sw_fence_await_dma_fence(wait,
1025 &from->fence, 0,1021 &from->fence, 0,
1026 I915_FENCE_GFP);1022 I915_FENCE_GFP);
1027}1023}
@@ -1034,11 +1030,15 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from) · 6 unmodified lines
1034 GEM_BUG_ON(to == from);1030 GEM_BUG_ON(to == from);
1035 GEM_BUG_ON(to->timeline == from->timeline);1031 GEM_BUG_ON(to->timeline == from->timeline);
1036 1032
1037 if (i915_request_completed(from))1033 if (i915_request_completed(from)) {
1034 i915_sw_fence_set_error_once(&to->submit, from->fence.error);
1038 return 0;1035 return 0;
1036 }
1039 1037
1040 if (to->engine->schedule) {1038 if (to->engine->schedule) {
1041 ret = i915_sched_node_add_dependency(&to->sched, &from->sched);1039 ret = i915_sched_node_add_dependency(&to->sched,
1040 &from->sched,
1041 I915_DEPENDENCY_EXTERNAL);
1042 if (ret < 0)1042 if (ret < 0)
1043 return ret;1043 return ret;
1044 }1044 }
@@ -1052,15 +1052,56 @@ i915_request_await_request(struct i915_request *to, struct i915_request *from) · 7 unmodified lines
1052 if (ret < 0)1052 if (ret < 0)
1053 return ret;1053 return ret;
1054 1054
1055 if (to->sched.flags & I915_SCHED_HAS_SEMAPHORE_CHAIN) {1055 return 0;
1056 ret = i915_sw_fence_await_dma_fence(&to->semaphore,1056}
1057 &from->fence, 0,1057
1058 I915_FENCE_GFP);1058static void mark_external(struct i915_request *rq)
1059 if (ret < 0)1059{
1060 return ret;1060 /*
1061 * The downside of using semaphores is that we lose metadata passing
1062 * along the signaling chain. This is particularly nasty when we
1063 * need to pass along a fatal error such as EFAULT or EDEADLK. For
1064 * fatal errors we want to scrub the request before it is executed,
1065 * which means that we cannot preload the request onto HW and have
1066 * it wait upon a semaphore.
1067 */
1068 rq->sched.flags |= I915_SCHED_HAS_EXTERNAL_CHAIN;
1069}
1070
1071static int
1072__i915_request_await_external(struct i915_request *rq, struct dma_fence *fence)
1073{
1074 mark_external(rq);
1075 return i915_sw_fence_await_dma_fence(&rq->submit, fence,
1076 i915_fence_context_timeout(rq->i915,
1077 fence->context),
1078 I915_FENCE_GFP);
1079}
1080
1081static int
1082i915_request_await_external(struct i915_request *rq, struct dma_fence *fence)
1083{
1084 struct dma_fence *iter;
1085 int err = 0;
1086
1087 if (!to_dma_fence_chain(fence))
1088 return __i915_request_await_external(rq, fence);
1089
1090 dma_fence_chain_for_each(iter, fence) {
1091 struct dma_fence_chain *chain = to_dma_fence_chain(iter);
1092
1093 if (!dma_fence_is_i915(chain->fence)) {
1094 err = __i915_request_await_external(rq, iter);
1095 break;
1096 }
1097
1098 err = i915_request_await_dma_fence(rq, chain->fence);
1099 if (err < 0)
1100 break;
1061 }1101 }
1062 1102
1063 return 0;1103 dma_fence_put(iter);
1104 return err;
1064}1105}
1065 1106
1066int1107int
@@ -1110,9 +1151,7 @@ i915_request_await_dma_fence(struct i915_request *rq, struct dma_fence *fence) · 43 unmodified lines
1110 if (dma_fence_is_i915(fence))1151 if (dma_fence_is_i915(fence))
1111 ret = i915_request_await_request(rq, to_request(fence));1152 ret = i915_request_await_request(rq, to_request(fence));
1112 else1153 else
1113 ret = i915_sw_fence_await_dma_fence(&rq->submit, fence,1154 ret = i915_request_await_external(rq, fence);
1114 fence->context ? I915_FENCE_TIMEOUT : 0,
1115 I915_FENCE_GFP);
1116 if (ret < 0)1155 if (ret < 0)
1117 return ret;1156 return ret;
1118 1157
@@ -1192,7 +1231,8 @@ __i915_request_await_execution(struct i915_request *to, · 73 unmodified lines
1192 * immediate execution, and so we must wait until it reaches the1231 * immediate execution, and so we must wait until it reaches the
1193 * active slot.1232 * active slot.
1194 */1233 */
1195 if (intel_engine_has_semaphores(to->engine)) {1234 if (intel_engine_has_semaphores(to->engine) &&
1235 !i915_request_has_initial_breadcrumb(to)) {
1196 err = __emit_semaphore_wait(to, from, from->fence.seqno - 1);1236 err = __emit_semaphore_wait(to, from, from->fence.seqno - 1);
1197 if (err < 0)1237 if (err < 0)
1198 return err;1238 return err;
@@ -1200,7 +1240,9 @@ __i915_request_await_execution(struct i915_request *to, · 1 unmodified line
1200 1240
1201 /* Couple the dependency tree for PI on this exposed to->fence */1241 /* Couple the dependency tree for PI on this exposed to->fence */
1202 if (to->engine->schedule) {1242 if (to->engine->schedule) {
1203 err = i915_sched_node_add_dependency(&to->sched, &from->sched);1243 err = i915_sched_node_add_dependency(&to->sched,
1244 &from->sched,
1245 I915_DEPENDENCY_WEAK);
1204 if (err < 0)1246 if (err < 0)
1205 return err;1247 return err;
1206 }1248 }
@@ -1236,6 +1278,9 @@ i915_request_await_execution(struct i915_request *rq, · 29 unmodified lines
1236 continue;1278 continue;
1237 }1279 }
1238 1280
1281 if (fence->context == rq->fence.context)
1282 continue;
1283
1239 /*1284 /*
1240 * We don't squash repeated fence dependencies here as we1285 * We don't squash repeated fence dependencies here as we
1241 * want to run our callback in all cases.1286 * want to run our callback in all cases.
@@ -1246,9 +1291,7 @@ i915_request_await_execution(struct i915_request *rq, · 4 unmodified lines
1246 to_request(fence),1291 to_request(fence),
1247 hook);1292 hook);
1248 else1293 else
1249 ret = i915_sw_fence_await_dma_fence(&rq->submit, fence,1294 ret = i915_request_await_external(rq, fence);
1250 I915_FENCE_TIMEOUT,
1251 GFP_KERNEL);
1252 if (ret < 0)1295 if (ret < 0)
1253 return ret;1296 return ret;
1254 } while (--nchild);1297 } while (--nchild);
@@ -1456,11 +1499,6 @@ void i915_request_add(struct i915_request *rq) · 201 unmodified lines
1456 attr = ctx->sched;1499 attr = ctx->sched;
1457 rcu_read_unlock();1500 rcu_read_unlock();
1458 1501
1459 if (!(rq->sched.flags & I915_SCHED_HAS_SEMAPHORE_CHAIN))
1460 attr.priority |= I915_PRIORITY_NOSEMAPHORE;
1461 if (list_empty(&rq->sched.signalers_list))
1462 attr.priority |= I915_PRIORITY_WAIT;
1463
1464 __i915_request_queue(rq, &attr);1502 __i915_request_queue(rq, &attr);
1465 1503
1466 mutex_unlock(&tl->mutex);1504 mutex_unlock(&tl->mutex);
@@ -1645,7 +1683,6 @@ long i915_request_wait(struct i915_request *rq, · 178 unmodified lines
1645 if (flags & I915_WAIT_PRIORITY) {1683 if (flags & I915_WAIT_PRIORITY) {
1646 if (!i915_request_started(rq) && INTEL_GEN(rq->i915) >= 6)1684 if (!i915_request_started(rq) && INTEL_GEN(rq->i915) >= 6)
1647 intel_rps_boost(rq);1685 intel_rps_boost(rq);
1648 i915_schedule_bump_priority(rq, I915_PRIORITY_WAIT);
1649 }1686 }
1650 1687
1651 wait.tsk = current;1688 wait.tsk = current;
drivers/gpu/drm/i915/i915_request.h+20−8
@@ -84,19 +84,26 @@ enum { · 83 unmodified lines
84 I915_FENCE_FLAG_PQUEUE,84 I915_FENCE_FLAG_PQUEUE,
85 85
86 /*86 /*
87 * I915_FENCE_FLAG_SIGNAL - this request is currently on signal_list87 * I915_FENCE_FLAG_HOLD - this request is currently on hold
88 *88 *
89 * Internal bookkeeping used by the breadcrumb code to track when89 * This request has been suspended, pending an ongoing investigation.
90 * a request is on the various signal_list.
91 */90 */
92 I915_FENCE_FLAG_SIGNAL,91 I915_FENCE_FLAG_HOLD,
93 92
94 /*93 /*
95 * I915_FENCE_FLAG_HOLD - this request is currently on hold94 * I915_FENCE_FLAG_INITIAL_BREADCRUMB - this request has the initial
95 * breadcrumb that marks the end of semaphore waits and start of the
96 * user payload.
97 */
98 I915_FENCE_FLAG_INITIAL_BREADCRUMB,
99
100 /*
101 * I915_FENCE_FLAG_SIGNAL - this request is currently on signal_list
96 *102 *
97 * This request has been suspended, pending an ongoing investigation.103 * Internal bookkeeping used by the breadcrumb code to track when
104 * a request is on the various signal_list.
98 */105 */
99 I915_FENCE_FLAG_HOLD,106 I915_FENCE_FLAG_SIGNAL,
100 107
101 /*108 /*
102 * I915_FENCE_FLAG_NOPREEMPT - this request should not be preempted109 * I915_FENCE_FLAG_NOPREEMPT - this request should not be preempted
@@ -209,7 +216,6 @@ struct i915_request { · 106 unmodified lines
209 };216 };
210 struct list_head execute_cb;217 struct list_head execute_cb;
211 struct i915_sw_fence semaphore;218 struct i915_sw_fence semaphore;
212 struct irq_work semaphore_work;
213 219
214 /*220 /*
215 * A list of everyone we wait upon, and everyone who waits upon us.221 * A list of everyone we wait upon, and everyone who waits upon us.
@@ -390,6 +396,12 @@ static inline bool i915_request_in_priority_queue(const struct i915_request *rq) · 174 unmodified lines
390 return test_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);396 return test_bit(I915_FENCE_FLAG_PQUEUE, &rq->fence.flags);
391}397}
392 398
399static inline bool
400i915_request_has_initial_breadcrumb(const struct i915_request *rq)
401{
402 return test_bit(I915_FENCE_FLAG_INITIAL_BREADCRUMB, &rq->fence.flags);
403}
404
393/**405/**
394 * Returns true if seq1 is later than seq2.406 * Returns true if seq1 is later than seq2.
395 */407 */
drivers/gpu/drm/i915/i915_scheduler.c+8−21
@@ -51,11 +51,11 @@ static void assert_priolists(struct intel_engine_execlists * const execlists) · 50 unmodified lines
51 GEM_BUG_ON(rb_first_cached(&execlists->queue) !=51 GEM_BUG_ON(rb_first_cached(&execlists->queue) !=
52 rb_first(&execlists->queue.rb_root));52 rb_first(&execlists->queue.rb_root));
53 53
54 last_prio = (INT_MAX >> I915_USER_PRIORITY_SHIFT) + 1;54 last_prio = INT_MAX;
55 for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {55 for (rb = rb_first_cached(&execlists->queue); rb; rb = rb_next(rb)) {
56 const struct i915_priolist *p = to_priolist(rb);56 const struct i915_priolist *p = to_priolist(rb);
57 57
58 GEM_BUG_ON(p->priority >= last_prio);58 GEM_BUG_ON(p->priority > last_prio);
59 last_prio = p->priority;59 last_prio = p->priority;
60 60
61 GEM_BUG_ON(!p->used);61 GEM_BUG_ON(!p->used);
@@ -174,7 +174,7 @@ sched_lock_engine(const struct i915_sched_node *node, · 112 unmodified lines
174 174
175static inline int rq_prio(const struct i915_request *rq)175static inline int rq_prio(const struct i915_request *rq)
176{176{
177 return rq->sched.attr.priority | __NO_PREEMPTION;177 return rq->sched.attr.priority;
178}178}
179 179
180static inline bool need_preempt(int prio, int active)180static inline bool need_preempt(int prio, int active)
@@ -434,25 +434,12 @@ bool __i915_sched_node_add_dependency(struct i915_sched_node *node, · 253 unmodified lines
434 dep->waiter = node;434 dep->waiter = node;
435 dep->flags = flags;435 dep->flags = flags;
436 436
437 /* Keep track of whether anyone on this chain has a semaphore */
438 if (signal->flags & I915_SCHED_HAS_SEMAPHORE_CHAIN &&
439 !node_started(signal))
440 node->flags |= I915_SCHED_HAS_SEMAPHORE_CHAIN;
441
442 /* All set, now publish. Beware the lockless walkers. */437 /* All set, now publish. Beware the lockless walkers. */
443 list_add_rcu(&dep->signal_link, &node->signalers_list);438 list_add_rcu(&dep->signal_link, &node->signalers_list);
444 list_add_rcu(&dep->wait_link, &signal->waiters_list);439 list_add_rcu(&dep->wait_link, &signal->waiters_list);
445 440
446 /*441 /* Propagate the chains */
447 * As we do not allow WAIT to preempt inflight requests,442 node->flags |= signal->flags;
448 * once we have executed a request, along with triggering
449 * any execution callbacks, we must preserve its ordering
450 * within the non-preemptible FIFO.
451 */
452 BUILD_BUG_ON(__NO_PREEMPTION & ~I915_PRIORITY_MASK);
453 if (flags & I915_DEPENDENCY_EXTERNAL)
454 __bump_priority(signal, __NO_PREEMPTION);
455
456 ret = true;443 ret = true;
457 }444 }
458 445
@@ -462,7 +449,8 @@ bool __i915_sched_node_add_dependency(struct i915_sched_node *node, · 3 unmodified lines
462}449}
463 450
464int i915_sched_node_add_dependency(struct i915_sched_node *node,451int i915_sched_node_add_dependency(struct i915_sched_node *node,
465 struct i915_sched_node *signal)452 struct i915_sched_node *signal,
453 unsigned long flags)
466{454{
467 struct i915_dependency *dep;455 struct i915_dependency *dep;
468 456
@@ -473,8 +461,7 @@ int i915_sched_node_add_dependency(struct i915_sched_node *node, · 4 unmodified lines
473 local_bh_disable();461 local_bh_disable();
474 462
475 if (!__i915_sched_node_add_dependency(node, signal, dep,463 if (!__i915_sched_node_add_dependency(node, signal, dep,
476 I915_DEPENDENCY_EXTERNAL |464 flags | I915_DEPENDENCY_ALLOC))
477 I915_DEPENDENCY_ALLOC))
478 i915_dependency_free(dep);465 i915_dependency_free(dep);
479 466
480 local_bh_enable(); /* kick submission tasklet */467 local_bh_enable(); /* kick submission tasklet */
drivers/gpu/drm/i915/i915_scheduler.h+2−1
@@ -34,7 +34,8 @@ bool __i915_sched_node_add_dependency(struct i915_sched_node *node, · 33 unmodified lines
34 unsigned long flags);34 unsigned long flags);
35 35
36int i915_sched_node_add_dependency(struct i915_sched_node *node,36int i915_sched_node_add_dependency(struct i915_sched_node *node,
37 struct i915_sched_node *signal);37 struct i915_sched_node *signal,
38 unsigned long flags);
38 39
39void i915_sched_node_fini(struct i915_sched_node *node);40void i915_sched_node_fini(struct i915_sched_node *node);
40 41
drivers/gpu/drm/i915/i915_scheduler_types.h+2−1
@@ -65,7 +65,7 @@ struct i915_sched_node { · 64 unmodified lines
65 struct list_head link;65 struct list_head link;
66 struct i915_sched_attr attr;66 struct i915_sched_attr attr;
67 unsigned int flags;67 unsigned int flags;
68#define I915_SCHED_HAS_SEMAPHORE_CHAIN BIT(0)68#define I915_SCHED_HAS_EXTERNAL_CHAIN BIT(0)
69 intel_engine_mask_t semaphores;69 intel_engine_mask_t semaphores;
70};70};
71 71
@@ -78,6 +78,7 @@ struct i915_dependency { · 6 unmodified lines
78 unsigned long flags;78 unsigned long flags;
79#define I915_DEPENDENCY_ALLOC BIT(0)79#define I915_DEPENDENCY_ALLOC BIT(0)
80#define I915_DEPENDENCY_EXTERNAL BIT(1)80#define I915_DEPENDENCY_EXTERNAL BIT(1)
81#define I915_DEPENDENCY_WEAK BIT(2)
81};82};
82 83
83#endif /* _I915_SCHEDULER_TYPES_H_ */84#endif /* _I915_SCHEDULER_TYPES_H_ */
drivers/gpu/drm/i915/i915_selftest.h+2−0
@@ -133,4 +133,6 @@ bool __igt_timeout(unsigned long timeout, const char *fmt, ...); · 132 unmodified lines
133#define igt_timeout(t, fmt, ...) \133#define igt_timeout(t, fmt, ...) \
134 __igt_timeout((t), KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)134 __igt_timeout((t), KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
135 135
136void igt_hexdump(const void *buf, size_t len);
137
136#endif /* !__I915_SELFTEST_H__ */138#endif /* !__I915_SELFTEST_H__ */
drivers/gpu/drm/i915/i915_sw_fence.c+4−6
@@ -546,13 +546,11 @@ int __i915_sw_fence_await_dma_fence(struct i915_sw_fence *fence, · 545 unmodified lines
546 cb->fence = fence;546 cb->fence = fence;
547 i915_sw_fence_await(fence);547 i915_sw_fence_await(fence);
548 548
549 ret = dma_fence_add_callback(dma, &cb->base, __dma_i915_sw_fence_wake);549 ret = 1;
550 if (ret == 0) {550 if (dma_fence_add_callback(dma, &cb->base, __dma_i915_sw_fence_wake)) {
551 ret = 1;551 /* fence already signaled */
552 } else {
553 __dma_i915_sw_fence_wake(dma, &cb->base);552 __dma_i915_sw_fence_wake(dma, &cb->base);
554 if (ret == -ENOENT) /* fence already signaled */553 ret = 0;
555 ret = 0;
556 }554 }
557 555
558 return ret;556 return ret;
drivers/gpu/drm/i915/intel_device_info.c+17−17
@@ -136,8 +136,8 @@ void intel_device_info_print_runtime(const struct intel_runtime_info *info, · 135 unmodified lines
136 sseu_dump(&info->sseu, p);136 sseu_dump(&info->sseu, p);
137 137
138 drm_printf(p, "rawclk rate: %u kHz\n", info->rawclk_freq);138 drm_printf(p, "rawclk rate: %u kHz\n", info->rawclk_freq);
139 drm_printf(p, "CS timestamp frequency: %u kHz\n",139 drm_printf(p, "CS timestamp frequency: %u Hz\n",
140 info->cs_timestamp_frequency_khz);140 info->cs_timestamp_frequency_hz);
141}141}
142 142
143static int sseu_eu_idx(const struct sseu_dev_info *sseu, int slice,143static int sseu_eu_idx(const struct sseu_dev_info *sseu, int slice,
@@ -678,12 +678,12 @@ static u32 read_reference_ts_freq(struct drm_i915_private *dev_priv) · 534 unmodified lines
678 678
679 base_freq = ((ts_override & GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DIVIDER_MASK) >>679 base_freq = ((ts_override & GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DIVIDER_MASK) >>
680 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DIVIDER_SHIFT) + 1;680 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DIVIDER_SHIFT) + 1;
681 base_freq *= 1000;681 base_freq *= 1000000;
682 682
683 frac_freq = ((ts_override &683 frac_freq = ((ts_override &
684 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DENOMINATOR_MASK) >>684 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DENOMINATOR_MASK) >>
685 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DENOMINATOR_SHIFT);685 GEN9_TIMESTAMP_OVERRIDE_US_COUNTER_DENOMINATOR_SHIFT);
686 frac_freq = 1000 / (frac_freq + 1);686 frac_freq = 1000000 / (frac_freq + 1);
687 687
688 return base_freq + frac_freq;688 return base_freq + frac_freq;
689}689}
@@ -691,8 +691,8 @@ static u32 read_reference_ts_freq(struct drm_i915_private *dev_priv) · 1 unmodified line
691static u32 gen10_get_crystal_clock_freq(struct drm_i915_private *dev_priv,691static u32 gen10_get_crystal_clock_freq(struct drm_i915_private *dev_priv,
692 u32 rpm_config_reg)692 u32 rpm_config_reg)
693{693{
694 u32 f19_2_mhz = 19200;694 u32 f19_2_mhz = 19200000;
695 u32 f24_mhz = 24000;695 u32 f24_mhz = 24000000;
696 u32 crystal_clock = (rpm_config_reg &696 u32 crystal_clock = (rpm_config_reg &
697 GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>697 GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>
698 GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;698 GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;
@@ -711,10 +711,10 @@ static u32 gen10_get_crystal_clock_freq(struct drm_i915_private *dev_priv, · 12 unmodified lines
711static u32 gen11_get_crystal_clock_freq(struct drm_i915_private *dev_priv,711static u32 gen11_get_crystal_clock_freq(struct drm_i915_private *dev_priv,
712 u32 rpm_config_reg)712 u32 rpm_config_reg)
713{713{
714 u32 f19_2_mhz = 19200;714 u32 f19_2_mhz = 19200000;
715 u32 f24_mhz = 24000;715 u32 f24_mhz = 24000000;
716 u32 f25_mhz = 25000;716 u32 f25_mhz = 25000000;
717 u32 f38_4_mhz = 38400;717 u32 f38_4_mhz = 38400000;
718 u32 crystal_clock = (rpm_config_reg &718 u32 crystal_clock = (rpm_config_reg &
719 GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>719 GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>
720 GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;720 GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;
@@ -736,9 +736,9 @@ static u32 gen11_get_crystal_clock_freq(struct drm_i915_private *dev_priv, · 15 unmodified lines
736 736
737static u32 read_timestamp_frequency(struct drm_i915_private *dev_priv)737static u32 read_timestamp_frequency(struct drm_i915_private *dev_priv)
738{738{
739 u32 f12_5_mhz = 12500;739 u32 f12_5_mhz = 12500000;
740 u32 f19_2_mhz = 19200;740 u32 f19_2_mhz = 19200000;
741 u32 f24_mhz = 24000;741 u32 f24_mhz = 24000000;
742 742
743 if (INTEL_GEN(dev_priv) <= 4) {743 if (INTEL_GEN(dev_priv) <= 4) {
744 /* PRMs say:744 /* PRMs say:
@@ -747,7 +747,7 @@ static u32 read_timestamp_frequency(struct drm_i915_private *dev_priv) · 2 unmodified lines
747 * hclks." (through the “Clocking Configuration”747 * hclks." (through the “Clocking Configuration”
748 * (“CLKCFG”) MCHBAR register)748 * (“CLKCFG”) MCHBAR register)
749 */749 */
750 return RUNTIME_INFO(dev_priv)->rawclk_freq / 16;750 return RUNTIME_INFO(dev_priv)->rawclk_freq * 1000 / 16;
751 } else if (INTEL_GEN(dev_priv) <= 8) {751 } else if (INTEL_GEN(dev_priv) <= 8) {
752 /* PRMs say:752 /* PRMs say:
753 *753 *
@@ -1048,11 +1048,11 @@ void intel_device_info_runtime_init(struct drm_i915_private *dev_priv) · 294 unmodified lines
1048 drm_dbg(&dev_priv->drm, "rawclk rate: %d kHz\n", runtime->rawclk_freq);1048 drm_dbg(&dev_priv->drm, "rawclk rate: %d kHz\n", runtime->rawclk_freq);
1049 1049
1050 /* Initialize command stream timestamp frequency */1050 /* Initialize command stream timestamp frequency */
1051 runtime->cs_timestamp_frequency_khz =1051 runtime->cs_timestamp_frequency_hz =
1052 read_timestamp_frequency(dev_priv);1052 read_timestamp_frequency(dev_priv);
1053 if (runtime->cs_timestamp_frequency_khz) {1053 if (runtime->cs_timestamp_frequency_hz) {
1054 runtime->cs_timestamp_period_ns =1054 runtime->cs_timestamp_period_ns =
1055 div_u64(1e6, runtime->cs_timestamp_frequency_khz);1055 i915_cs_timestamp_ticks_to_ns(dev_priv, 1);
1056 drm_dbg(&dev_priv->drm,1056 drm_dbg(&dev_priv->drm,
1057 "CS timestamp wraparound in %lldms\n",1057 "CS timestamp wraparound in %lldms\n",
1058 div_u64(mul_u32_u32(runtime->cs_timestamp_period_ns,1058 div_u64(mul_u32_u32(runtime->cs_timestamp_period_ns,
drivers/gpu/drm/i915/intel_device_info.h+1−1
@@ -221,7 +221,7 @@ struct intel_runtime_info { · 220 unmodified lines
221 221
222 u32 rawclk_freq;222 u32 rawclk_freq;
223 223
224 u32 cs_timestamp_frequency_khz;224 u32 cs_timestamp_frequency_hz;
225 u32 cs_timestamp_period_ns;225 u32 cs_timestamp_period_ns;
226 226
227 /* Media engine access to SFC per instance */227 /* Media engine access to SFC per instance */
drivers/gpu/drm/i915/intel_pm.c+266−62
@@ -43,6 +43,7 @@ · 42 unmodified lines
43#include "i915_fixed.h"43#include "i915_fixed.h"
44#include "i915_irq.h"44#include "i915_irq.h"
45#include "i915_trace.h"45#include "i915_trace.h"
46#include "display/intel_bw.h"
46#include "intel_pm.h"47#include "intel_pm.h"
47#include "intel_sideband.h"48#include "intel_sideband.h"
48#include "../../../platform/x86/intel_ips.h"49#include "../../../platform/x86/intel_ips.h"
@@ -3637,10 +3638,6 @@ static bool skl_needs_memory_bw_wa(struct drm_i915_private *dev_priv) · 3588 unmodified lines
3637static bool3638static bool
3638intel_has_sagv(struct drm_i915_private *dev_priv)3639intel_has_sagv(struct drm_i915_private *dev_priv)
3639{3640{
3640 /* HACK! */
3641 if (IS_GEN(dev_priv, 12))
3642 return false;
3643
3644 return (IS_GEN9_BC(dev_priv) || INTEL_GEN(dev_priv) >= 10) &&3641 return (IS_GEN9_BC(dev_priv) || INTEL_GEN(dev_priv) >= 10) &&
3645 dev_priv->sagv_status != I915_SAGV_NOT_CONTROLLED;3642 dev_priv->sagv_status != I915_SAGV_NOT_CONTROLLED;
3646}3643}
@@ -3760,34 +3757,116 @@ intel_disable_sagv(struct drm_i915_private *dev_priv) · 113 unmodified lines
3760void intel_sagv_pre_plane_update(struct intel_atomic_state *state)3757void intel_sagv_pre_plane_update(struct intel_atomic_state *state)
3761{3758{
3762 struct drm_i915_private *dev_priv = to_i915(state->base.dev);3759 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
3760 const struct intel_bw_state *new_bw_state;
3761 const struct intel_bw_state *old_bw_state;
3762 u32 new_mask = 0;
3763
3764 /*
3765 * Just return if we can't control SAGV or don't have it.
3766 * This is different from situation when we have SAGV but just can't
3767 * afford it due to DBuf limitation - in case if SAGV is completely
3768 * disabled in a BIOS, we are not even allowed to send a PCode request,
3769 * as it will throw an error. So have to check it here.
3770 */
3771 if (!intel_has_sagv(dev_priv))
3772 return;
3773
3774 new_bw_state = intel_atomic_get_new_bw_state(state);
3775 if (!new_bw_state)
3776 return;
3763 3777
3764 if (!intel_can_enable_sagv(state))3778 if (INTEL_GEN(dev_priv) < 11 && !intel_can_enable_sagv(dev_priv, new_bw_state)) {
3765 intel_disable_sagv(dev_priv);3779 intel_disable_sagv(dev_priv);
3780 return;
3781 }
3782
3783 old_bw_state = intel_atomic_get_old_bw_state(state);
3784 /*
3785 * Nothing to mask
3786 */
3787 if (new_bw_state->qgv_points_mask == old_bw_state->qgv_points_mask)
3788 return;
3789
3790 new_mask = old_bw_state->qgv_points_mask | new_bw_state->qgv_points_mask;
3791
3792 /*
3793 * If new mask is zero - means there is nothing to mask,
3794 * we can only unmask, which should be done in unmask.
3795 */
3796 if (!new_mask)
3797 return;
3798
3799 /*
3800 * Restrict required qgv points before updating the configuration.
3801 * According to BSpec we can't mask and unmask qgv points at the same
3802 * time. Also masking should be done before updating the configuration
3803 * and unmasking afterwards.
3804 */
3805 icl_pcode_restrict_qgv_points(dev_priv, new_mask);
3766}3806}
3767 3807
3768void intel_sagv_post_plane_update(struct intel_atomic_state *state)3808void intel_sagv_post_plane_update(struct intel_atomic_state *state)
3769{3809{
3770 struct drm_i915_private *dev_priv = to_i915(state->base.dev);3810 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
3811 const struct intel_bw_state *new_bw_state;
3812 const struct intel_bw_state *old_bw_state;
3813 u32 new_mask = 0;
3814
3815 /*
3816 * Just return if we can't control SAGV or don't have it.
3817 * This is different from situation when we have SAGV but just can't
3818 * afford it due to DBuf limitation - in case if SAGV is completely
3819 * disabled in a BIOS, we are not even allowed to send a PCode request,
3820 * as it will throw an error. So have to check it here.
3821 */
3822 if (!intel_has_sagv(dev_priv))
3823 return;
3824
3825 new_bw_state = intel_atomic_get_new_bw_state(state);
3826 if (!new_bw_state)
3827 return;
3771 3828
3772 if (intel_can_enable_sagv(state))3829 if (INTEL_GEN(dev_priv) < 11 && intel_can_enable_sagv(dev_priv, new_bw_state)) {
3773 intel_enable_sagv(dev_priv);3830 intel_enable_sagv(dev_priv);
3831 return;
3832 }
3833
3834 old_bw_state = intel_atomic_get_old_bw_state(state);
3835 /*
3836 * Nothing to unmask
3837 */
3838 if (new_bw_state->qgv_points_mask == old_bw_state->qgv_points_mask)
3839 return;
3840
3841 new_mask = new_bw_state->qgv_points_mask;
3842
3843 /*
3844 * Allow required qgv points after updating the configuration.
3845 * According to BSpec we can't mask and unmask qgv points at the same
3846 * time. Also masking should be done before updating the configuration
3847 * and unmasking afterwards.
3848 */
3849 icl_pcode_restrict_qgv_points(dev_priv, new_mask);
3774}3850}
3775 3851
3776static bool intel_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state)3852static bool skl_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state)
3777{3853{
3778 struct drm_device *dev = crtc_state->uapi.crtc->dev;
3779 struct drm_i915_private *dev_priv = to_i915(dev);
3780 struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);3854 struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
3855 struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
3781 struct intel_plane *plane;3856 struct intel_plane *plane;
3857 const struct intel_plane_state *plane_state;
3782 int level, latency;3858 int level, latency;
3783 3859
3860 if (!intel_has_sagv(dev_priv))
3861 return false;
3862
3784 if (!crtc_state->hw.active)3863 if (!crtc_state->hw.active)
3785 return true;3864 return true;
3786 3865
3787 if (crtc_state->hw.adjusted_mode.flags & DRM_MODE_FLAG_INTERLACE)3866 if (crtc_state->hw.adjusted_mode.flags & DRM_MODE_FLAG_INTERLACE)
3788 return false;3867 return false;
3789 3868
3790 for_each_intel_plane_on_crtc(dev, crtc, plane) {3869 intel_atomic_crtc_state_for_each_plane_state(plane, plane_state, crtc_state) {
3791 const struct skl_plane_wm *wm =3870 const struct skl_plane_wm *wm =
3792 &crtc_state->wm.skl.optimal.planes[plane->id];3871 &crtc_state->wm.skl.optimal.planes[plane->id];
3793 3872
@@ -3803,7 +3882,7 @@ static bool intel_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state · 9 unmodified lines
3803 latency = dev_priv->wm.skl_latency[level];3882 latency = dev_priv->wm.skl_latency[level];
3804 3883
3805 if (skl_needs_memory_bw_wa(dev_priv) &&3884 if (skl_needs_memory_bw_wa(dev_priv) &&
3806 plane->base.state->fb->modifier ==3885 plane_state->uapi.fb->modifier ==
3807 I915_FORMAT_MOD_X_TILED)3886 I915_FORMAT_MOD_X_TILED)
3808 latency += 15;3887 latency += 15;
3809 3888
@@ -3819,35 +3898,110 @@ static bool intel_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state · 9 unmodified lines
3819 return true;3898 return true;
3820}3899}
3821 3900
3822bool intel_can_enable_sagv(struct intel_atomic_state *state)3901static bool tgl_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state)
3902{
3903 struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
3904 enum plane_id plane_id;
3905
3906 if (!crtc_state->hw.active)
3907 return true;
3908
3909 for_each_plane_id_on_crtc(crtc, plane_id) {
3910 const struct skl_ddb_entry *plane_alloc =
3911 &crtc_state->wm.skl.plane_ddb_y[plane_id];
3912 const struct skl_plane_wm *wm =
3913 &crtc_state->wm.skl.optimal.planes[plane_id];
3914
3915 if (skl_ddb_entry_size(plane_alloc) < wm->sagv_wm0.min_ddb_alloc)
3916 return false;
3917 }
3918
3919 return true;
3920}
3921
3922static bool intel_crtc_can_enable_sagv(const struct intel_crtc_state *crtc_state)
3923{
3924 struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
3925 struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
3926
3927 if (INTEL_GEN(dev_priv) >= 12)
3928 return tgl_crtc_can_enable_sagv(crtc_state);
3929 else
3930 return skl_crtc_can_enable_sagv(crtc_state);
3931}
3932
3933bool intel_can_enable_sagv(struct drm_i915_private *dev_priv,
3934 const struct intel_bw_state *bw_state)
3935{
3936 if (INTEL_GEN(dev_priv) < 11 &&
3937 bw_state->active_pipes && !is_power_of_2(bw_state->active_pipes))
3938 return false;
3939
3940 return bw_state->pipe_sagv_reject == 0;
3941}
3942
3943static int intel_compute_sagv_mask(struct intel_atomic_state *state)
3823{3944{
3824 struct drm_i915_private *dev_priv = to_i915(state->base.dev);3945 struct drm_i915_private *dev_priv = to_i915(state->base.dev);
3946 int ret;
3825 struct intel_crtc *crtc;3947 struct intel_crtc *crtc;
3826 const struct intel_crtc_state *crtc_state;3948 struct intel_crtc_state *new_crtc_state;
3827 enum pipe pipe;3949 struct intel_bw_state *new_bw_state = NULL;
3950 const struct intel_bw_state *old_bw_state = NULL;
3951 int i;
3828 3952
3829 if (!intel_has_sagv(dev_priv))3953 for_each_new_intel_crtc_in_state(state, crtc,
3830 return false;3954 new_crtc_state, i) {
3955 new_bw_state = intel_atomic_get_bw_state(state);
3956 if (IS_ERR(new_bw_state))
3957 return PTR_ERR(new_bw_state);
3831 3958
3832 /*3959 old_bw_state = intel_atomic_get_old_bw_state(state);
3833 * If there are no active CRTCs, no additional checks need be performed
3834 */
3835 if (hweight8(state->active_pipes) == 0)
3836 return true;
3837 3960
3838 /*3961 if (intel_crtc_can_enable_sagv(new_crtc_state))
3839 * SKL+ workaround: bspec recommends we disable SAGV when we have3962 new_bw_state->pipe_sagv_reject &= ~BIT(crtc->pipe);
3840 * more then one pipe enabled3963 else
3841 */3964 new_bw_state->pipe_sagv_reject |= BIT(crtc->pipe);
3842 if (hweight8(state->active_pipes) > 1)3965 }
3843 return false;3966
3967 if (!new_bw_state)
3968 return 0;
3969
3970 new_bw_state->active_pipes =
3971 intel_calc_active_pipes(state, old_bw_state->active_pipes);
3972
3973 if (new_bw_state->active_pipes != old_bw_state->active_pipes) {
3974 ret = intel_atomic_lock_global_state(&new_bw_state->base);
3975 if (ret)
3976 return ret;
3977 }
3978
3979 for_each_new_intel_crtc_in_state(state, crtc,
3980 new_crtc_state, i) {
3981 struct skl_pipe_wm *pipe_wm = &new_crtc_state->wm.skl.optimal;
3844 3982
3845 /* Since we're now guaranteed to only have one active CRTC... */3983 /*
3846 pipe = ffs(state->active_pipes) - 1;3984 * We store use_sagv_wm in the crtc state rather than relying on
3847 crtc = intel_get_crtc_for_pipe(dev_priv, pipe);3985 * that bw state since we have no convenient way to get at the
3848 crtc_state = to_intel_crtc_state(crtc->base.state);3986 * latter from the plane commit hooks (especially in the legacy
3987 * cursor case)
3988 */
3989 pipe_wm->use_sagv_wm = INTEL_GEN(dev_priv) >= 12 &&
3990 intel_can_enable_sagv(dev_priv, new_bw_state);
3991 }
3992
3993 if (intel_can_enable_sagv(dev_priv, new_bw_state) !=
3994 intel_can_enable_sagv(dev_priv, old_bw_state)) {
3995 ret = intel_atomic_serialize_global_state(&new_bw_state->base);
3996 if (ret)
3997 return ret;
3998 } else if (new_bw_state->pipe_sagv_reject != old_bw_state->pipe_sagv_reject) {
3999 ret = intel_atomic_lock_global_state(&new_bw_state->base);
4000 if (ret)
4001 return ret;
4002 }
3849 4003
3850 return intel_crtc_can_enable_sagv(crtc_state);4004 return 0;
3851}4005}
3852 4006
3853/*4007/*
@@ -4574,6 +4728,20 @@ icl_get_total_relative_data_rate(struct intel_crtc_state *crtc_state, · 720 unmodified lines
4574 return total_data_rate;4728 return total_data_rate;
4575}4729}
4576 4730
4731static const struct skl_wm_level *
4732skl_plane_wm_level(const struct intel_crtc_state *crtc_state,
4733 enum plane_id plane_id,
4734 int level)
4735{
4736 const struct skl_pipe_wm *pipe_wm = &crtc_state->wm.skl.optimal;
4737 const struct skl_plane_wm *wm = &pipe_wm->planes[plane_id];
4738
4739 if (level == 0 && pipe_wm->use_sagv_wm)
4740 return &wm->sagv_wm0;
4741
4742 return &wm->wm[level];
4743}
4744
4577static int4745static int
4578skl_allocate_pipe_ddb(struct intel_crtc_state *crtc_state)4746skl_allocate_pipe_ddb(struct intel_crtc_state *crtc_state)
4579{4747{
@@ -4610,7 +4778,6 @@ skl_allocate_pipe_ddb(struct intel_crtc_state *crtc_state) · 30 unmodified lines
4610 plane_data_rate,4778 plane_data_rate,
4611 uv_plane_data_rate);4779 uv_plane_data_rate);
4612 4780
4613
4614 skl_ddb_get_pipe_allocation_limits(dev_priv, crtc_state, total_data_rate,4781 skl_ddb_get_pipe_allocation_limits(dev_priv, crtc_state, total_data_rate,
4615 alloc, &num_active);4782 alloc, &num_active);
4616 alloc_size = skl_ddb_entry_size(alloc);4783 alloc_size = skl_ddb_entry_size(alloc);
@@ -4810,7 +4977,7 @@ skl_wm_method1(const struct drm_i915_private *dev_priv, u32 pixel_rate, · 193 unmodified lines
4810 wm_intermediate_val = latency * pixel_rate * cpp;4977 wm_intermediate_val = latency * pixel_rate * cpp;
4811 ret = div_fixed16(wm_intermediate_val, 1000 * dbuf_block_size);4978 ret = div_fixed16(wm_intermediate_val, 1000 * dbuf_block_size);
4812 4979
4813 if (INTEL_GEN(dev_priv) >= 10)4980 if (INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv))
4814 ret = add_fixed16_u32(ret, 1);4981 ret = add_fixed16_u32(ret, 1);
4815 4982
4816 return ret;4983 return ret;
@@ -4945,18 +5112,19 @@ skl_compute_wm_params(const struct intel_crtc_state *crtc_state, · 128 unmodified lines
4945 wp->y_min_scanlines,5112 wp->y_min_scanlines,
4946 wp->dbuf_block_size);5113 wp->dbuf_block_size);
4947 5114
4948 if (INTEL_GEN(dev_priv) >= 10)5115 if (INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv))
4949 interm_pbpl++;5116 interm_pbpl++;
4950 5117
4951 wp->plane_blocks_per_line = div_fixed16(interm_pbpl,5118 wp->plane_blocks_per_line = div_fixed16(interm_pbpl,
4952 wp->y_min_scanlines);5119 wp->y_min_scanlines);
4953 } else if (wp->x_tiled && IS_GEN(dev_priv, 9)) {
4954 interm_pbpl = DIV_ROUND_UP(wp->plane_bytes_per_line,
4955 wp->dbuf_block_size);
4956 wp->plane_blocks_per_line = u32_to_fixed16(interm_pbpl);
4957 } else {5120 } else {
4958 interm_pbpl = DIV_ROUND_UP(wp->plane_bytes_per_line,5121 interm_pbpl = DIV_ROUND_UP(wp->plane_bytes_per_line,
4959 wp->dbuf_block_size) + 1;5122 wp->dbuf_block_size);
5123
5124 if (!wp->x_tiled ||
5125 INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv))
5126 interm_pbpl++;
5127
4960 wp->plane_blocks_per_line = u32_to_fixed16(interm_pbpl);5128 wp->plane_blocks_per_line = u32_to_fixed16(interm_pbpl);
4961 }5129 }
4962 5130
@@ -5022,7 +5190,7 @@ static void skl_compute_plane_wm(const struct intel_crtc_state *crtc_state, · 59 unmodified lines
5022 * WaIncreaseLatencyIPCEnabled: kbl,cfl5190 * WaIncreaseLatencyIPCEnabled: kbl,cfl
5023 * Display WA #1141: kbl,cfl5191 * Display WA #1141: kbl,cfl
5024 */5192 */
5025 if ((IS_KABYLAKE(dev_priv) || IS_COFFEELAKE(dev_priv)) ||5193 if ((IS_KABYLAKE(dev_priv) || IS_COFFEELAKE(dev_priv)) &&
5026 dev_priv->ipc_enabled)5194 dev_priv->ipc_enabled)
5027 latency += 4;5195 latency += 4;
5028 5196
@@ -5145,6 +5313,20 @@ skl_compute_wm_levels(const struct intel_crtc_state *crtc_state, · 116 unmodified lines
5145 }5313 }
5146}5314}
5147 5315
5316static void tgl_compute_sagv_wm(const struct intel_crtc_state *crtc_state,
5317 const struct skl_wm_params *wm_params,
5318 struct skl_plane_wm *plane_wm)
5319{
5320 struct drm_i915_private *dev_priv = to_i915(crtc_state->uapi.crtc->dev);
5321 struct skl_wm_level *sagv_wm = &plane_wm->sagv_wm0;
5322 struct skl_wm_level *levels = plane_wm->wm;
5323 unsigned int latency = dev_priv->wm.skl_latency[0] + dev_priv->sagv_block_time_us;
5324
5325 skl_compute_plane_wm(crtc_state, 0, latency,
5326 wm_params, &levels[0],
5327 sagv_wm);
5328}
5329
5148static void skl_compute_transition_wm(const struct intel_crtc_state *crtc_state,5330static void skl_compute_transition_wm(const struct intel_crtc_state *crtc_state,
5149 const struct skl_wm_params *wp,5331 const struct skl_wm_params *wp,
5150 struct skl_plane_wm *wm)5332 struct skl_plane_wm *wm)
@@ -5197,10 +5379,6 @@ static void skl_compute_transition_wm(const struct intel_crtc_state *crtc_state, · 46 unmodified lines
5197 trans_offset_b;5379 trans_offset_b;
5198 } else {5380 } else {
5199 res_blocks = wm0_sel_res_b + trans_offset_b;5381 res_blocks = wm0_sel_res_b + trans_offset_b;
5200
5201 /* WA BUG:1938466 add one block for non y-tile planes */
5202 if (IS_CNL_REVID(dev_priv, CNL_REVID_A0, CNL_REVID_A0))
5203 res_blocks += 1;
5204 }5382 }
5205 5383
5206 /*5384 /*
@@ -5216,6 +5394,8 @@ static int skl_build_plane_wm_single(struct intel_crtc_state *crtc_state, · 9 unmodified lines
5216 const struct intel_plane_state *plane_state,5394 const struct intel_plane_state *plane_state,
5217 enum plane_id plane_id, int color_plane)5395 enum plane_id plane_id, int color_plane)
5218{5396{
5397 struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc);
5398 struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
5219 struct skl_plane_wm *wm = &crtc_state->wm.skl.optimal.planes[plane_id];5399 struct skl_plane_wm *wm = &crtc_state->wm.skl.optimal.planes[plane_id];
5220 struct skl_wm_params wm_params;5400 struct skl_wm_params wm_params;
5221 int ret;5401 int ret;
@@ -5226,6 +5406,10 @@ static int skl_build_plane_wm_single(struct intel_crtc_state *crtc_state, · 4 unmodified lines
5226 return ret;5406 return ret;
5227 5407
5228 skl_compute_wm_levels(crtc_state, &wm_params, wm->wm);5408 skl_compute_wm_levels(crtc_state, &wm_params, wm->wm);
5409
5410 if (INTEL_GEN(dev_priv) >= 12)
5411 tgl_compute_sagv_wm(crtc_state, &wm_params, wm);
5412
5229 skl_compute_transition_wm(crtc_state, &wm_params, wm);5413 skl_compute_transition_wm(crtc_state, &wm_params, wm);
5230 5414
5231 return 0;5415 return 0;
@@ -5385,8 +5569,12 @@ void skl_write_plane_wm(struct intel_plane *plane, · 153 unmodified lines
5385 &crtc_state->wm.skl.plane_ddb_uv[plane_id];5569 &crtc_state->wm.skl.plane_ddb_uv[plane_id];
5386 5570
5387 for (level = 0; level <= max_level; level++) {5571 for (level = 0; level <= max_level; level++) {
5572 const struct skl_wm_level *wm_level;
5573
5574 wm_level = skl_plane_wm_level(crtc_state, plane_id, level);
5575
5388 skl_write_wm_level(dev_priv, PLANE_WM(pipe, plane_id, level),5576 skl_write_wm_level(dev_priv, PLANE_WM(pipe, plane_id, level),
5389 &wm->wm[level]);5577 wm_level);
5390 }5578 }
5391 skl_write_wm_level(dev_priv, PLANE_WM_TRANS(pipe, plane_id),5579 skl_write_wm_level(dev_priv, PLANE_WM_TRANS(pipe, plane_id),
5392 &wm->trans_wm);5580 &wm->trans_wm);
@@ -5419,8 +5607,12 @@ void skl_write_cursor_wm(struct intel_plane *plane, · 26 unmodified lines
5419 &crtc_state->wm.skl.plane_ddb_y[plane_id];5607 &crtc_state->wm.skl.plane_ddb_y[plane_id];
5420 5608
5421 for (level = 0; level <= max_level; level++) {5609 for (level = 0; level <= max_level; level++) {
5610 const struct skl_wm_level *wm_level;
5611
5612 wm_level = skl_plane_wm_level(crtc_state, plane_id, level);
5613
5422 skl_write_wm_level(dev_priv, CUR_WM(pipe, level),5614 skl_write_wm_level(dev_priv, CUR_WM(pipe, level),
5423 &wm->wm[level]);5615 wm_level);
5424 }5616 }
5425 skl_write_wm_level(dev_priv, CUR_WM_TRANS(pipe), &wm->trans_wm);5617 skl_write_wm_level(dev_priv, CUR_WM_TRANS(pipe), &wm->trans_wm);
5426 5618
@@ -5584,23 +5776,25 @@ skl_print_wm_changes(struct intel_atomic_state *state) · 157 unmodified lines
5584 continue;5776 continue;
5585 5777
5586 drm_dbg_kms(&dev_priv->drm,5778 drm_dbg_kms(&dev_priv->drm,
5587 "[PLANE:%d:%s] level %cwm0,%cwm1,%cwm2,%cwm3,%cwm4,%cwm5,%cwm6,%cwm7,%ctwm"5779 "[PLANE:%d:%s] level %cwm0,%cwm1,%cwm2,%cwm3,%cwm4,%cwm5,%cwm6,%cwm7,%ctwm,%cswm"
5588 " -> %cwm0,%cwm1,%cwm2,%cwm3,%cwm4,%cwm5,%cwm6,%cwm7,%ctwm\n",5780 " -> %cwm0,%cwm1,%cwm2,%cwm3,%cwm4,%cwm5,%cwm6,%cwm7,%ctwm,%cswm\n",
5589 plane->base.base.id, plane->base.name,5781 plane->base.base.id, plane->base.name,
5590 enast(old_wm->wm[0].plane_en), enast(old_wm->wm[1].plane_en),5782 enast(old_wm->wm[0].plane_en), enast(old_wm->wm[1].plane_en),
5591 enast(old_wm->wm[2].plane_en), enast(old_wm->wm[3].plane_en),5783 enast(old_wm->wm[2].plane_en), enast(old_wm->wm[3].plane_en),
5592 enast(old_wm->wm[4].plane_en), enast(old_wm->wm[5].plane_en),5784 enast(old_wm->wm[4].plane_en), enast(old_wm->wm[5].plane_en),
5593 enast(old_wm->wm[6].plane_en), enast(old_wm->wm[7].plane_en),5785 enast(old_wm->wm[6].plane_en), enast(old_wm->wm[7].plane_en),
5594 enast(old_wm->trans_wm.plane_en),5786 enast(old_wm->trans_wm.plane_en),
5787 enast(old_wm->sagv_wm0.plane_en),
5595 enast(new_wm->wm[0].plane_en), enast(new_wm->wm[1].plane_en),5788 enast(new_wm->wm[0].plane_en), enast(new_wm->wm[1].plane_en),
5596 enast(new_wm->wm[2].plane_en), enast(new_wm->wm[3].plane_en),5789 enast(new_wm->wm[2].plane_en), enast(new_wm->wm[3].plane_en),
5597 enast(new_wm->wm[4].plane_en), enast(new_wm->wm[5].plane_en),5790 enast(new_wm->wm[4].plane_en), enast(new_wm->wm[5].plane_en),
5598 enast(new_wm->wm[6].plane_en), enast(new_wm->wm[7].plane_en),5791 enast(new_wm->wm[6].plane_en), enast(new_wm->wm[7].plane_en),
5599 enast(new_wm->trans_wm.plane_en));5792 enast(new_wm->trans_wm.plane_en),
5793 enast(new_wm->sagv_wm0.plane_en));
5600 5794
5601 drm_dbg_kms(&dev_priv->drm,5795 drm_dbg_kms(&dev_priv->drm,
5602 "[PLANE:%d:%s] lines %c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d"5796 "[PLANE:%d:%s] lines %c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d"
5603 " -> %c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d\n",5797 " -> %c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d,%c%3d\n",
5604 plane->base.base.id, plane->base.name,5798 plane->base.base.id, plane->base.name,
5605 enast(old_wm->wm[0].ignore_lines), old_wm->wm[0].plane_res_l,5799 enast(old_wm->wm[0].ignore_lines), old_wm->wm[0].plane_res_l,
5606 enast(old_wm->wm[1].ignore_lines), old_wm->wm[1].plane_res_l,5800 enast(old_wm->wm[1].ignore_lines), old_wm->wm[1].plane_res_l,
@@ -5611,6 +5805,7 @@ skl_print_wm_changes(struct intel_atomic_state *state) · 4 unmodified lines
5611 enast(old_wm->wm[6].ignore_lines), old_wm->wm[6].plane_res_l,5805 enast(old_wm->wm[6].ignore_lines), old_wm->wm[6].plane_res_l,
5612 enast(old_wm->wm[7].ignore_lines), old_wm->wm[7].plane_res_l,5806 enast(old_wm->wm[7].ignore_lines), old_wm->wm[7].plane_res_l,
5613 enast(old_wm->trans_wm.ignore_lines), old_wm->trans_wm.plane_res_l,5807 enast(old_wm->trans_wm.ignore_lines), old_wm->trans_wm.plane_res_l,
5808 enast(old_wm->sagv_wm0.ignore_lines), old_wm->sagv_wm0.plane_res_l,
5614 5809
5615 enast(new_wm->wm[0].ignore_lines), new_wm->wm[0].plane_res_l,5810 enast(new_wm->wm[0].ignore_lines), new_wm->wm[0].plane_res_l,
5616 enast(new_wm->wm[1].ignore_lines), new_wm->wm[1].plane_res_l,5811 enast(new_wm->wm[1].ignore_lines), new_wm->wm[1].plane_res_l,
@@ -5620,37 +5815,42 @@ skl_print_wm_changes(struct intel_atomic_state *state) · 3 unmodified lines
5620 enast(new_wm->wm[5].ignore_lines), new_wm->wm[5].plane_res_l,5815 enast(new_wm->wm[5].ignore_lines), new_wm->wm[5].plane_res_l,
5621 enast(new_wm->wm[6].ignore_lines), new_wm->wm[6].plane_res_l,5816 enast(new_wm->wm[6].ignore_lines), new_wm->wm[6].plane_res_l,
5622 enast(new_wm->wm[7].ignore_lines), new_wm->wm[7].plane_res_l,5817 enast(new_wm->wm[7].ignore_lines), new_wm->wm[7].plane_res_l,
5623 enast(new_wm->trans_wm.ignore_lines), new_wm->trans_wm.plane_res_l);5818 enast(new_wm->trans_wm.ignore_lines), new_wm->trans_wm.plane_res_l,
5819 enast(new_wm->sagv_wm0.ignore_lines), new_wm->sagv_wm0.plane_res_l);
5624 5820
5625 drm_dbg_kms(&dev_priv->drm,5821 drm_dbg_kms(&dev_priv->drm,
5626 "[PLANE:%d:%s] blocks %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d"5822 "[PLANE:%d:%s] blocks %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d"
5627 " -> %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d\n",5823 " -> %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d\n",
5628 plane->base.base.id, plane->base.name,5824 plane->base.base.id, plane->base.name,
5629 old_wm->wm[0].plane_res_b, old_wm->wm[1].plane_res_b,5825 old_wm->wm[0].plane_res_b, old_wm->wm[1].plane_res_b,
5630 old_wm->wm[2].plane_res_b, old_wm->wm[3].plane_res_b,5826 old_wm->wm[2].plane_res_b, old_wm->wm[3].plane_res_b,
5631 old_wm->wm[4].plane_res_b, old_wm->wm[5].plane_res_b,5827 old_wm->wm[4].plane_res_b, old_wm->wm[5].plane_res_b,
5632 old_wm->wm[6].plane_res_b, old_wm->wm[7].plane_res_b,5828 old_wm->wm[6].plane_res_b, old_wm->wm[7].plane_res_b,
5633 old_wm->trans_wm.plane_res_b,5829 old_wm->trans_wm.plane_res_b,
5830 old_wm->sagv_wm0.plane_res_b,
5634 new_wm->wm[0].plane_res_b, new_wm->wm[1].plane_res_b,5831 new_wm->wm[0].plane_res_b, new_wm->wm[1].plane_res_b,
5635 new_wm->wm[2].plane_res_b, new_wm->wm[3].plane_res_b,5832 new_wm->wm[2].plane_res_b, new_wm->wm[3].plane_res_b,
5636 new_wm->wm[4].plane_res_b, new_wm->wm[5].plane_res_b,5833 new_wm->wm[4].plane_res_b, new_wm->wm[5].plane_res_b,
5637 new_wm->wm[6].plane_res_b, new_wm->wm[7].plane_res_b,5834 new_wm->wm[6].plane_res_b, new_wm->wm[7].plane_res_b,
5638 new_wm->trans_wm.plane_res_b);5835 new_wm->trans_wm.plane_res_b,
5836 new_wm->sagv_wm0.plane_res_b);
5639 5837
5640 drm_dbg_kms(&dev_priv->drm,5838 drm_dbg_kms(&dev_priv->drm,
5641 "[PLANE:%d:%s] min_ddb %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d"5839 "[PLANE:%d:%s] min_ddb %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d"
5642 " -> %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d\n",5840 " -> %4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d,%4d\n",
5643 plane->base.base.id, plane->base.name,5841 plane->base.base.id, plane->base.name,
5644 old_wm->wm[0].min_ddb_alloc, old_wm->wm[1].min_ddb_alloc,5842 old_wm->wm[0].min_ddb_alloc, old_wm->wm[1].min_ddb_alloc,
5645 old_wm->wm[2].min_ddb_alloc, old_wm->wm[3].min_ddb_alloc,5843 old_wm->wm[2].min_ddb_alloc, old_wm->wm[3].min_ddb_alloc,
5646 old_wm->wm[4].min_ddb_alloc, old_wm->wm[5].min_ddb_alloc,5844 old_wm->wm[4].min_ddb_alloc, old_wm->wm[5].min_ddb_alloc,
5647 old_wm->wm[6].min_ddb_alloc, old_wm->wm[7].min_ddb_alloc,5845 old_wm->wm[6].min_ddb_alloc, old_wm->wm[7].min_ddb_alloc,
5648 old_wm->trans_wm.min_ddb_alloc,5846 old_wm->trans_wm.min_ddb_alloc,
5847 old_wm->sagv_wm0.min_ddb_alloc,
5649 new_wm->wm[0].min_ddb_alloc, new_wm->wm[1].min_ddb_alloc,5848 new_wm->wm[0].min_ddb_alloc, new_wm->wm[1].min_ddb_alloc,
5650 new_wm->wm[2].min_ddb_alloc, new_wm->wm[3].min_ddb_alloc,5849 new_wm->wm[2].min_ddb_alloc, new_wm->wm[3].min_ddb_alloc,
5651 new_wm->wm[4].min_ddb_alloc, new_wm->wm[5].min_ddb_alloc,5850 new_wm->wm[4].min_ddb_alloc, new_wm->wm[5].min_ddb_alloc,
5652 new_wm->wm[6].min_ddb_alloc, new_wm->wm[7].min_ddb_alloc,5851 new_wm->wm[6].min_ddb_alloc, new_wm->wm[7].min_ddb_alloc,
5653 new_wm->trans_wm.min_ddb_alloc);5852 new_wm->trans_wm.min_ddb_alloc,
5853 new_wm->sagv_wm0.min_ddb_alloc);
5654 }5854 }
5655 }5855 }
5656}5856}
@@ -5811,6 +6011,10 @@ skl_compute_wm(struct intel_atomic_state *state) · 154 unmodified lines
5811 if (ret)6011 if (ret)
5812 return ret;6012 return ret;
5813 6013
6014 ret = intel_compute_sagv_mask(state);
6015 if (ret)
6016 return ret;
6017
5814 /*6018 /*
5815 * skl_compute_ddb() will have adjusted the final watermarks6019 * skl_compute_ddb() will have adjusted the final watermarks
5816 * based on how much ddb is available. Now we can actually6020 * based on how much ddb is available. Now we can actually
@@ -5939,6 +6143,9 @@ void skl_pipe_wm_get_hw_state(struct intel_crtc *crtc, · 122 unmodified lines
5939 skl_wm_level_from_reg_val(val, &wm->wm[level]);6143 skl_wm_level_from_reg_val(val, &wm->wm[level]);
5940 }6144 }
5941 6145
6146 if (INTEL_GEN(dev_priv) >= 12)
6147 wm->sagv_wm0 = wm->wm[0];
6148
5942 if (plane_id != PLANE_CURSOR)6149 if (plane_id != PLANE_CURSOR)
5943 val = I915_READ(PLANE_WM_TRANS(pipe, plane_id));6150 val = I915_READ(PLANE_WM_TRANS(pipe, plane_id));
5944 else6151 else
@@ -6916,9 +7123,6 @@ static void cnl_init_clock_gating(struct drm_i915_private *dev_priv) · 971 unmodified lines
6916 val = I915_READ(SLICE_UNIT_LEVEL_CLKGATE);7123 val = I915_READ(SLICE_UNIT_LEVEL_CLKGATE);
6917 /* ReadHitWriteOnlyDisable:cnl */7124 /* ReadHitWriteOnlyDisable:cnl */
6918 val |= RCCUNIT_CLKGATE_DIS;7125 val |= RCCUNIT_CLKGATE_DIS;
6919 /* WaSarbUnitClockGatingDisable:cnl (pre-prod) */
6920 if (IS_CNL_REVID(dev_priv, CNL_REVID_A0, CNL_REVID_B0))
6921 val |= SARBUNIT_CLKGATE_DIS;
6922 I915_WRITE(SLICE_UNIT_LEVEL_CLKGATE, val);7126 I915_WRITE(SLICE_UNIT_LEVEL_CLKGATE, val);
6923 7127
6924 /* Wa_2201832410:cnl */7128 /* Wa_2201832410:cnl */
drivers/gpu/drm/i915/intel_pm.h+3−1
@@ -9,6 +9,7 @@ · 8 unmodified lines
9#include <linux/types.h>9#include <linux/types.h>
10 10
11#include "i915_reg.h"11#include "i915_reg.h"
12#include "display/intel_bw.h"
12 13
13struct drm_device;14struct drm_device;
14struct drm_i915_private;15struct drm_i915_private;
@@ -41,7 +42,8 @@ void skl_pipe_wm_get_hw_state(struct intel_crtc *crtc, · 26 unmodified lines
41 struct skl_pipe_wm *out);42 struct skl_pipe_wm *out);
42void g4x_wm_sanitize(struct drm_i915_private *dev_priv);43void g4x_wm_sanitize(struct drm_i915_private *dev_priv);
43void vlv_wm_sanitize(struct drm_i915_private *dev_priv);44void vlv_wm_sanitize(struct drm_i915_private *dev_priv);
44bool intel_can_enable_sagv(struct intel_atomic_state *state);45bool intel_can_enable_sagv(struct drm_i915_private *dev_priv,
46 const struct intel_bw_state *bw_state);
45int intel_enable_sagv(struct drm_i915_private *dev_priv);47int intel_enable_sagv(struct drm_i915_private *dev_priv);
46int intel_disable_sagv(struct drm_i915_private *dev_priv);48int intel_disable_sagv(struct drm_i915_private *dev_priv);
47void intel_sagv_pre_plane_update(struct intel_atomic_state *state);49void intel_sagv_pre_plane_update(struct intel_atomic_state *state);
drivers/gpu/drm/i915/intel_sideband.c+3−1
@@ -371,6 +371,8 @@ static int gen7_check_mailbox_status(u32 mbox) · 370 unmodified lines
371 return -ENXIO;371 return -ENXIO;
372 case GEN11_PCODE_LOCKED:372 case GEN11_PCODE_LOCKED:
373 return -EBUSY;373 return -EBUSY;
374 case GEN11_PCODE_REJECTED:
375 return -EACCES;
374 case GEN7_PCODE_MIN_FREQ_TABLE_GT_RATIO_OUT_OF_RANGE:376 case GEN7_PCODE_MIN_FREQ_TABLE_GT_RATIO_OUT_OF_RANGE:
375 return -EOVERFLOW;377 return -EOVERFLOW;
376 default:378 default:
@@ -429,7 +431,7 @@ int sandybridge_pcode_read(struct drm_i915_private *i915, u32 mbox, · 52 unmodified lines
429 431
430 mutex_lock(&i915->sb_lock);432 mutex_lock(&i915->sb_lock);
431 err = __sandybridge_pcode_rw(i915, mbox, val, val1,433 err = __sandybridge_pcode_rw(i915, mbox, val, val1,
432 500, 0,434 500, 20,
433 true);435 true);
434 mutex_unlock(&i915->sb_lock);436 mutex_unlock(&i915->sb_lock);
435 437
drivers/gpu/drm/i915/intel_uncore.c+0−5
@@ -899,11 +899,6 @@ find_fw_domain(struct intel_uncore *uncore, u32 offset) · 898 unmodified lines
899#define GEN_FW_RANGE(s, e, d) \899#define GEN_FW_RANGE(s, e, d) \
900 { .start = (s), .end = (e), .domains = (d) }900 { .start = (s), .end = (e), .domains = (d) }
901 901
902#define HAS_FWTABLE(dev_priv) \
903 (INTEL_GEN(dev_priv) >= 9 || \
904 IS_CHERRYVIEW(dev_priv) || \
905 IS_VALLEYVIEW(dev_priv))
906
907/* *Must* be sorted by offset ranges! See intel_fw_table_check(). */902/* *Must* be sorted by offset ranges! See intel_fw_table_check(). */
908static const struct intel_forcewake_range __vlv_fw_ranges[] = {903static const struct intel_forcewake_range __vlv_fw_ranges[] = {
909 GEN_FW_RANGE(0x2000, 0x3fff, FORCEWAKE_RENDER),904 GEN_FW_RANGE(0x2000, 0x3fff, FORCEWAKE_RENDER),
drivers/gpu/drm/i915/intel_wopcm.c+1−2
@@ -149,8 +149,7 @@ static bool check_hw_restrictions(struct drm_i915_private *i915, · 148 unmodified lines
149 guc_wopcm_size))149 guc_wopcm_size))
150 return false;150 return false;
151 151
152 if ((IS_GEN(i915, 9) ||152 if (IS_GEN(i915, 9) &&
153 IS_CNL_REVID(i915, CNL_REVID_A0, CNL_REVID_A0)) &&
154 !gen9_check_huc_fw_fits(i915, guc_wopcm_size, huc_fw_size))153 !gen9_check_huc_fw_fits(i915, guc_wopcm_size, huc_fw_size))
155 return false;154 return false;
156 155
drivers/gpu/drm/i915/selftests/i915_live_selftests.h+1−0
@@ -37,6 +37,7 @@ selftest(gem, i915_gem_live_selftests) · 36 unmodified lines
37selftest(evict, i915_gem_evict_live_selftests)37selftest(evict, i915_gem_evict_live_selftests)
38selftest(hugepages, i915_gem_huge_page_live_selftests)38selftest(hugepages, i915_gem_huge_page_live_selftests)
39selftest(gem_contexts, i915_gem_context_live_selftests)39selftest(gem_contexts, i915_gem_context_live_selftests)
40selftest(gem_execbuf, i915_gem_execbuffer_live_selftests)
40selftest(blt, i915_gem_object_blt_live_selftests)41selftest(blt, i915_gem_object_blt_live_selftests)
41selftest(client, i915_gem_client_blt_live_selftests)42selftest(client, i915_gem_client_blt_live_selftests)
42selftest(reset, intel_reset_live_selftests)43selftest(reset, intel_reset_live_selftests)
drivers/gpu/drm/i915/selftests/i915_perf.c+2−4
@@ -221,8 +221,7 @@ static int live_noa_delay(void *arg) · 220 unmodified lines
221 goto out;221 goto out;
222 }222 }
223 223
224 if (rq->engine->emit_init_breadcrumb &&224 if (rq->engine->emit_init_breadcrumb) {
225 i915_request_timeline(rq)->has_initial_breadcrumb) {
226 err = rq->engine->emit_init_breadcrumb(rq);225 err = rq->engine->emit_init_breadcrumb(rq);
227 if (err) {226 if (err) {
228 i915_request_add(rq);227 i915_request_add(rq);
@@ -263,8 +262,7 @@ static int live_noa_delay(void *arg) · 34 unmodified lines
263 262
264 delay = intel_read_status_page(stream->engine, 0x102);263 delay = intel_read_status_page(stream->engine, 0x102);
265 delay -= intel_read_status_page(stream->engine, 0x100);264 delay -= intel_read_status_page(stream->engine, 0x100);
266 delay = div_u64(mul_u32_u32(delay, 1000 * 1000),265 delay = i915_cs_timestamp_ticks_to_ns(i915, delay);
267 RUNTIME_INFO(i915)->cs_timestamp_frequency_khz);
268 pr_info("GPU delay: %uns, expected %lluns\n",266 pr_info("GPU delay: %uns, expected %lluns\n",
269 delay, expected);267 delay, expected);
270 268
drivers/gpu/drm/i915/selftests/i915_request.c+21−16
@@ -816,10 +816,12 @@ static int recursive_batch_resolve(struct i915_vma *batch) · 815 unmodified lines
816 return PTR_ERR(cmd);816 return PTR_ERR(cmd);
817 817
818 *cmd = MI_BATCH_BUFFER_END;818 *cmd = MI_BATCH_BUFFER_END;
819 intel_gt_chipset_flush(batch->vm->gt);
820 819
820 __i915_gem_object_flush_map(batch->obj, 0, sizeof(*cmd));
821 i915_gem_object_unpin_map(batch->obj);821 i915_gem_object_unpin_map(batch->obj);
822 822
823 intel_gt_chipset_flush(batch->vm->gt);
824
823 return 0;825 return 0;
824}826}
825 827
@@ -865,13 +867,6 @@ static int live_all_engines(void *arg) · 39 unmodified lines
865 goto out_request;867 goto out_request;
866 }868 }
867 869
868 err = engine->emit_bb_start(request[idx],
869 batch->node.start,
870 batch->node.size,
871 0);
872 GEM_BUG_ON(err);
873 request[idx]->batch = batch;
874
875 i915_vma_lock(batch);870 i915_vma_lock(batch);
876 err = i915_request_await_object(request[idx], batch->obj, 0);871 err = i915_request_await_object(request[idx], batch->obj, 0);
877 if (err == 0)872 if (err == 0)
@@ -879,6 +874,13 @@ static int live_all_engines(void *arg) · 1 unmodified line
879 i915_vma_unlock(batch);874 i915_vma_unlock(batch);
880 GEM_BUG_ON(err);875 GEM_BUG_ON(err);
881 876
877 err = engine->emit_bb_start(request[idx],
878 batch->node.start,
879 batch->node.size,
880 0);
881 GEM_BUG_ON(err);
882 request[idx]->batch = batch;
883
882 i915_request_get(request[idx]);884 i915_request_get(request[idx]);
883 i915_request_add(request[idx]);885 i915_request_add(request[idx]);
884 idx++;886 idx++;
@@ -993,13 +995,6 @@ static int live_sequential_engines(void *arg) · 108 unmodified lines
993 }995 }
994 }996 }
995 997
996 err = engine->emit_bb_start(request[idx],
997 batch->node.start,
998 batch->node.size,
999 0);
1000 GEM_BUG_ON(err);
1001 request[idx]->batch = batch;
1002
1003 i915_vma_lock(batch);998 i915_vma_lock(batch);
1004 err = i915_request_await_object(request[idx],999 err = i915_request_await_object(request[idx],
1005 batch->obj, false);1000 batch->obj, false);
@@ -1008,6 +1003,13 @@ static int live_sequential_engines(void *arg) · 2 unmodified lines
1008 i915_vma_unlock(batch);1003 i915_vma_unlock(batch);
1009 GEM_BUG_ON(err);1004 GEM_BUG_ON(err);
1010 1005
1006 err = engine->emit_bb_start(request[idx],
1007 batch->node.start,
1008 batch->node.size,
1009 0);
1010 GEM_BUG_ON(err);
1011 request[idx]->batch = batch;
1012
1011 i915_request_get(request[idx]);1013 i915_request_get(request[idx]);
1012 i915_request_add(request[idx]);1014 i915_request_add(request[idx]);
1013 1015
@@ -1060,9 +1062,12 @@ out_request: · 46 unmodified lines
1060 I915_MAP_WC);1062 I915_MAP_WC);
1061 if (!IS_ERR(cmd)) {1063 if (!IS_ERR(cmd)) {
1062 *cmd = MI_BATCH_BUFFER_END;1064 *cmd = MI_BATCH_BUFFER_END;
1063 intel_gt_chipset_flush(engine->gt);
1064 1065
1066 __i915_gem_object_flush_map(request[idx]->batch->obj,
1067 0, sizeof(*cmd));
1065 i915_gem_object_unpin_map(request[idx]->batch->obj);1068 i915_gem_object_unpin_map(request[idx]->batch->obj);
1069
1070 intel_gt_chipset_flush(engine->gt);
1066 }1071 }
1067 1072
1068 i915_vma_put(request[idx]->batch);1073 i915_vma_put(request[idx]->batch);
drivers/gpu/drm/i915/selftests/i915_selftest.c+29−0
@@ -396,6 +396,35 @@ bool __igt_timeout(unsigned long timeout, const char *fmt, ...) · 395 unmodified lines
396 return true;396 return true;
397}397}
398 398
399void igt_hexdump(const void *buf, size_t len)
400{
401 const size_t rowsize = 8 * sizeof(u32);
402 const void *prev = NULL;
403 bool skip = false;
404 size_t pos;
405
406 for (pos = 0; pos < len; pos += rowsize) {
407 char line[128];
408
409 if (prev && !memcmp(prev, buf + pos, rowsize)) {
410 if (!skip) {
411 pr_info("*\n");
412 skip = true;
413 }
414 continue;
415 }
416
417 WARN_ON_ONCE(hex_dump_to_buffer(buf + pos, len - pos,
418 rowsize, sizeof(u32),
419 line, sizeof(line),
420 false) >= sizeof(line));
421 pr_info("[%04zx] %s\n", pos, line);
422
423 prev = buf + pos;
424 skip = false;
425 }
426}
427
399module_param_named(st_random_seed, i915_selftest.random_seed, uint, 0400);428module_param_named(st_random_seed, i915_selftest.random_seed, uint, 0400);
400module_param_named(st_timeout, i915_selftest.timeout_ms, uint, 0400);429module_param_named(st_timeout, i915_selftest.timeout_ms, uint, 0400);
401module_param_named(st_filter, i915_selftest.filter, charp, 0400);430module_param_named(st_filter, i915_selftest.filter, charp, 0400);
drivers/gpu/drm/i915/selftests/igt_spinner.c+1−2
@@ -169,8 +169,7 @@ igt_spinner_create_request(struct igt_spinner *spin, · 168 unmodified lines
169 169
170 intel_gt_chipset_flush(engine->gt);170 intel_gt_chipset_flush(engine->gt);
171 171
172 if (engine->emit_init_breadcrumb &&172 if (engine->emit_init_breadcrumb) {
173 i915_request_timeline(rq)->has_initial_breadcrumb) {
174 err = engine->emit_init_breadcrumb(rq);173 err = engine->emit_init_breadcrumb(rq);
175 if (err)174 if (err)
176 goto cancel_rq;175 goto cancel_rq;
drivers/video/hdmi.c+46−19
@@ -1768,20 +1768,21 @@ hdmi_vendor_any_infoframe_unpack(union hdmi_vendor_any_infoframe *frame, · 1767 unmodified lines
1768}1768}
1769 1769
1770/**1770/**
1771 * hdmi_drm_infoframe_unpack() - unpack binary buffer to a HDMI DRM infoframe1771 * hdmi_drm_infoframe_unpack_only() - unpack binary buffer of CTA-861-G DRM
1772 * infoframe DataBytes to a HDMI DRM
1773 * infoframe
1772 * @frame: HDMI DRM infoframe1774 * @frame: HDMI DRM infoframe
1773 * @buffer: source buffer1775 * @buffer: source buffer
1774 * @size: size of buffer1776 * @size: size of buffer
1775 *1777 *
1776 * Unpacks the information contained in binary @buffer into a structured1778 * Unpacks CTA-861-G DRM infoframe DataBytes contained in the binary @buffer
1777 * @frame of the HDMI Dynamic Range and Mastering (DRM) information frame.1779 * into a structured @frame of the HDMI Dynamic Range and Mastering (DRM)
1778 * Also verifies the checksum as required by section 5.3.5 of the HDMI 1.41780 * infoframe.
1779 * specification.
1780 *1781 *
1781 * Returns 0 on success or a negative error code on failure.1782 * Returns 0 on success or a negative error code on failure.
1782 */1783 */
1783static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame,1784int hdmi_drm_infoframe_unpack_only(struct hdmi_drm_infoframe *frame,
1784 const void *buffer, size_t size)1785 const void *buffer, size_t size)
1785{1786{
1786 const u8 *ptr = buffer;1787 const u8 *ptr = buffer;
1787 const u8 *temp;1788 const u8 *temp;
@@ -1790,23 +1791,13 @@ static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame, · 2 unmodified lines
1790 int ret;1791 int ret;
1791 int i;1792 int i;
1792 1793
1793 if (size < HDMI_INFOFRAME_SIZE(DRM))1794 if (size < HDMI_DRM_INFOFRAME_SIZE)
1794 return -EINVAL;
1795
1796 if (ptr[0] != HDMI_INFOFRAME_TYPE_DRM ||
1797 ptr[1] != 1 ||
1798 ptr[2] != HDMI_DRM_INFOFRAME_SIZE)
1799 return -EINVAL;
1800
1801 if (hdmi_infoframe_checksum(buffer, HDMI_INFOFRAME_SIZE(DRM)) != 0)
1802 return -EINVAL;1795 return -EINVAL;
1803 1796
1804 ret = hdmi_drm_infoframe_init(frame);1797 ret = hdmi_drm_infoframe_init(frame);
1805 if (ret)1798 if (ret)
1806 return ret;1799 return ret;
1807 1800
1808 ptr += HDMI_INFOFRAME_HEADER_SIZE;
1809
1810 frame->eotf = ptr[0] & 0x7;1801 frame->eotf = ptr[0] & 0x7;
1811 frame->metadata_type = ptr[1] & 0x7;1802 frame->metadata_type = ptr[1] & 0x7;
1812 1803
@@ -1814,7 +1805,7 @@ static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame, · 1 unmodified line
1814 for (i = 0; i < 3; i++) {1805 for (i = 0; i < 3; i++) {
1815 x_lsb = *temp++;1806 x_lsb = *temp++;
1816 x_msb = *temp++;1807 x_msb = *temp++;
1817 frame->display_primaries[i].x = (x_msb << 8) | x_lsb;1808 frame->display_primaries[i].x = (x_msb << 8) | x_lsb;
1818 y_lsb = *temp++;1809 y_lsb = *temp++;
1819 y_msb = *temp++;1810 y_msb = *temp++;
1820 frame->display_primaries[i].y = (y_msb << 8) | y_lsb;1811 frame->display_primaries[i].y = (y_msb << 8) | y_lsb;
@@ -1830,6 +1821,42 @@ static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame, · 9 unmodified lines
1830 1821
1831 return 0;1822 return 0;
1832}1823}
1824EXPORT_SYMBOL(hdmi_drm_infoframe_unpack_only);
1825
1826/**
1827 * hdmi_drm_infoframe_unpack() - unpack binary buffer to a HDMI DRM infoframe
1828 * @frame: HDMI DRM infoframe
1829 * @buffer: source buffer
1830 * @size: size of buffer
1831 *
1832 * Unpacks the CTA-861-G DRM infoframe contained in the binary @buffer into
1833 * a structured @frame of the HDMI Dynamic Range and Mastering (DRM)
1834 * infoframe. It also verifies the checksum as required by section 5.3.5 of
1835 * the HDMI 1.4 specification.
1836 *
1837 * Returns 0 on success or a negative error code on failure.
1838 */
1839static int hdmi_drm_infoframe_unpack(struct hdmi_drm_infoframe *frame,
1840 const void *buffer, size_t size)
1841{
1842 const u8 *ptr = buffer;
1843 int ret;
1844
1845 if (size < HDMI_INFOFRAME_SIZE(DRM))
1846 return -EINVAL;
1847
1848 if (ptr[0] != HDMI_INFOFRAME_TYPE_DRM ||
1849 ptr[1] != 1 ||
1850 ptr[2] != HDMI_DRM_INFOFRAME_SIZE)
1851 return -EINVAL;
1852
1853 if (hdmi_infoframe_checksum(buffer, HDMI_INFOFRAME_SIZE(DRM)) != 0)
1854 return -EINVAL;
1855
1856 ret = hdmi_drm_infoframe_unpack_only(frame, ptr + HDMI_INFOFRAME_HEADER_SIZE,
1857 size - HDMI_INFOFRAME_HEADER_SIZE);
1858 return ret;
1859}
1833 1860
1834/**1861/**
1835 * hdmi_infoframe_unpack() - unpack binary buffer to a HDMI infoframe1862 * hdmi_infoframe_unpack() - unpack binary buffer to a HDMI infoframe
include/drm/drm_dp_helper.h+3−0
@@ -1348,6 +1348,9 @@ struct drm_dp_vsc_sdp { · 1347 unmodified lines
1348 enum dp_content_type content_type;1348 enum dp_content_type content_type;
1349};1349};
1350 1350
1351void drm_dp_vsc_sdp_log(const char *level, struct device *dev,
1352 const struct drm_dp_vsc_sdp *vsc);
1353
1351int drm_dp_psr_setup_time(const u8 psr_cap[EDP_PSR_RECEIVER_CAP_SIZE]);1354int drm_dp_psr_setup_time(const u8 psr_cap[EDP_PSR_RECEIVER_CAP_SIZE]);
1352 1355
1353static inline int1356static inline int
include/linux/hdmi.h+2−0
@@ -219,6 +219,8 @@ ssize_t hdmi_drm_infoframe_pack(struct hdmi_drm_infoframe *frame, void *buffer, · 218 unmodified lines
219ssize_t hdmi_drm_infoframe_pack_only(const struct hdmi_drm_infoframe *frame,219ssize_t hdmi_drm_infoframe_pack_only(const struct hdmi_drm_infoframe *frame,
220 void *buffer, size_t size);220 void *buffer, size_t size);
221int hdmi_drm_infoframe_check(struct hdmi_drm_infoframe *frame);221int hdmi_drm_infoframe_check(struct hdmi_drm_infoframe *frame);
222int hdmi_drm_infoframe_unpack_only(struct hdmi_drm_infoframe *frame,
223 const void *buffer, size_t size);
222 224
223enum hdmi_spd_sdi {225enum hdmi_spd_sdi {
224 HDMI_SPD_SDI_UNKNOWN,226 HDMI_SPD_SDI_UNKNOWN,

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t