Merge tag 'drm-intel-next-2018-09-06-2' of git://anongit.freedesktop.org/drm/drm-intel into drm-next

Merge tag 'gvt-next-2018-09-04' drm-intel-next-2018-09-06-1: UAPI Changes: - GGTT coherency GETPARAM: GGTT has turned out to be non-coherent for some platforms, which we've failed to communicate to userspace so far. SNA was modified to do extra flushing on non-coherent GGTT access, while Mesa will mitigate by always requiring WC mapping (which is non-coherent anyway). - Neuter Resource Streamer uAPI: There never really were users for the feature, so neuter it while keeping the interface bits for compatibility. This is a long due item from past. Cross-subsystem Changes: - Backmerge of branch drm-next-4.19 for DP_DPCD_REV_14 changes Core Changes: - None Driver Changes: - A load of Icelake (ICL) enabling patches (Paulo, Manasi) - Enabled full PPGTT for IVB,VLV and HSW (Chris) - Bugzilla #107113: Distribute DDB based on display resolutions (Mahesh) - Bugzillas #100023,#107476,#94921: Support limited range DP displays (Jani) - Bugzilla #107503: Increase LSPCON timeout (Fredrik) - Avoid boosting GPU due to an occasional stall in interactive workloads (Chris) - Apply GGTT coherency W/A only for affected systems instead of all (Chris) - Fix for infinite link training loop for faulty USB-C MST hubs (Nathan) - Keep KMS functional on Gen4 and earlier when GPU is wedged (Chris) - Stop holding ppGTT reference from closed VMAs (Chris) - Clear error registers after error capture (Lionel) - Various Icelake fixes (Anusha, Jyoti, Ville, Tvrtko) - Add missing Coffeelake (CFL) PCI IDs (Rodrigo) - Flush execlists tasklet directly from reset-finish (Chris) - Fix LPE audio runtime PM (Chris) - Fix detection of out of range surface positions (GLK/CNL) (Ville) - Remove wait-for-idle for PSR2 (Dhinakaran) - Power down existing display hardware resources when display is disabled (Chris) - Don't allow runtime power management if RC6 doesn't exist (Chris) - Add debugging checks for runtime power management paths (Imre) - Increase symmetry in display power init/fini paths (Imre) - Isolate GVT specific macros from i915_reg.h (Lucas) - Increase symmetry in power management enable/disable paths (Chris) - Increase IP disable timeout to 100 ms to avoid DRM_ERROR (Imre) - Fix memory leak from HDMI HDCP write function (Brian, Rodrigo) - Reject Y/Yf tiling on interlaced modes (Ville) - Use a cached mapping for the physical HWS on older gens (Chris) - Force slow path of writing relocations to buffer if unable to write to userspace (Chris) - Do a full device reset after being wedged (Chris) - Keep forcewake counts over reset (in case of debugfs user) (Imre, Chris) - Avoid false-positive errors from power wells during init (Imre) - Reset engines forcibly in exchange of declaring whole device wedged (Mika) - Reduce context HW ID lifetime in preparation for Icelake (Chris) - Attempt to recover from module load failures (Chris) - Keep select interrupts over a reset to avoid missing/losing them (Chris) - GuC submission backend improvements (Jakub) - Terminate context images with BB_END (Chris, Lionel) - Make GCC evaluate GGTT view struct size assertions again (Ville) - Add selftest to exercise suspend/hibernate code-paths for GEM (Chris) - Use a full emulation of a user ppgtt context in selftests (Chris) - Exercise resetting in the middle of a wait-on-fence in selftests (Chris) - Fix coherency issues on selftests for Baytrail (Chris) - Various other GEM fixes / self-test updates (Chris, Matt) - GuC doorbell self-tests (Daniele) - PSR mode control through debugfs for IGTs (Maarten) - Degrade expected WM latency errors to DRM_DEBUG_KMS (Chris) - Cope with errors better in MST link training (Dhinakaran) - Fix WARN on KBL external displays (Azhar) - Power well code cleanups (Imre) - Fixes to PSR debugging (Dhinakaran) - Make forcewake errors louder for easier catching in CI (WARNs) (Chris) - Fortify tiling code against programmer errors (Chris) - Bunch of fixes for CI exposed corner cases (multiple authors, mostly Chris) Signed-off-by: Dave Airlie <airlied@redhat.com>

Dave Airlie committed 8 years ago

Commit b1c15662 parents 1f3eb34 a28957b87 files · +3857 −1806Browse files

Unified · Split

87 files

drivers/gpu/drm/i915/Kconfig.debug +12 −0
drivers/gpu/drm/i915/gvt/cfg_space.c +12 −0
drivers/gpu/drm/i915/gvt/cmd_parser.c +10 −1
drivers/gpu/drm/i915/gvt/display.c +1 −0
drivers/gpu/drm/i915/gvt/edid.c +9 −0
drivers/gpu/drm/i915/gvt/gtt.c +6 −3
drivers/gpu/drm/i915/gvt/gvt.c +1 −2
drivers/gpu/drm/i915/gvt/handlers.c +12 −22
drivers/gpu/drm/i915/gvt/kvmgt.c +2 −2
drivers/gpu/drm/i915/gvt/mmio.c +2 −1
drivers/gpu/drm/i915/gvt/mmio_context.c +0 −13
drivers/gpu/drm/i915/gvt/mmio_context.h +3 −0
drivers/gpu/drm/i915/gvt/opregion.c +0 −1
drivers/gpu/drm/i915/gvt/page_track.c +2 −0
drivers/gpu/drm/i915/gvt/reg.h +18 −0
drivers/gpu/drm/i915/gvt/scheduler.c +34 −30
drivers/gpu/drm/i915/i915_debugfs.c +64 −24
drivers/gpu/drm/i915/i915_drv.c +104 −90
drivers/gpu/drm/i915/i915_drv.h +49 −14
drivers/gpu/drm/i915/i915_gem.c +52 −13
drivers/gpu/drm/i915/i915_gem.h +0 −6
drivers/gpu/drm/i915/i915_gem_context.c +161 −71
drivers/gpu/drm/i915/i915_gem_context.h +23 −3
drivers/gpu/drm/i915/i915_gem_execbuffer.c +21 −22
drivers/gpu/drm/i915/i915_gem_gtt.c +25 −27
drivers/gpu/drm/i915/i915_gem_gtt.h +10 −13
drivers/gpu/drm/i915/i915_gem_object.h +5 −5
drivers/gpu/drm/i915/i915_irq.c +26 −10
drivers/gpu/drm/i915/i915_pci.c +11 −5
drivers/gpu/drm/i915/i915_perf.c +24 −31
drivers/gpu/drm/i915/i915_reg.h +387 −340
drivers/gpu/drm/i915/i915_request.c +4 −5
drivers/gpu/drm/i915/i915_request.h +25 −14
drivers/gpu/drm/i915/i915_vma.c +4 −1
drivers/gpu/drm/i915/i915_vma.h +9 −1
drivers/gpu/drm/i915/intel_atomic_plane.c +4 −2
drivers/gpu/drm/i915/intel_breadcrumbs.c +2 −4
drivers/gpu/drm/i915/intel_csr.c +24 −9
drivers/gpu/drm/i915/intel_ddi.c +223 −17
drivers/gpu/drm/i915/intel_device_info.h +1 −1
drivers/gpu/drm/i915/intel_display.c +115 −57
drivers/gpu/drm/i915/intel_display.h +26 −2
drivers/gpu/drm/i915/intel_dp.c +451 −83
drivers/gpu/drm/i915/intel_dp_link_training.c +20 −5
drivers/gpu/drm/i915/intel_dp_mst.c +7 −7
drivers/gpu/drm/i915/intel_dpll_mgr.c +46 −16
drivers/gpu/drm/i915/intel_dpll_mgr.h +1 −0
drivers/gpu/drm/i915/intel_drv.h +33 −10
drivers/gpu/drm/i915/intel_engine_cs.c +41 −52
drivers/gpu/drm/i915/intel_guc.c +46 −56
drivers/gpu/drm/i915/intel_guc.h +5 −7
drivers/gpu/drm/i915/intel_guc_ads.c +1 −1
drivers/gpu/drm/i915/intel_guc_ct.c +2 −5
drivers/gpu/drm/i915/intel_guc_fwif.h +1 −0
drivers/gpu/drm/i915/intel_guc_log.c +1 −1
drivers/gpu/drm/i915/intel_guc_submission.c +20 −15
drivers/gpu/drm/i915/intel_guc_submission.h +4 −0
drivers/gpu/drm/i915/intel_hangcheck.c +1 −1
drivers/gpu/drm/i915/intel_hdcp.c +3 −3
drivers/gpu/drm/i915/intel_hdmi.c +7 −3
drivers/gpu/drm/i915/intel_huc.c +1 −1
drivers/gpu/drm/i915/intel_i2c.c +8 −8
drivers/gpu/drm/i915/intel_lrc.c +110 −49
drivers/gpu/drm/i915/intel_lrc.h +0 −2
drivers/gpu/drm/i915/intel_lrc_reg.h +1 −1
drivers/gpu/drm/i915/intel_mocs.c +3 −8
drivers/gpu/drm/i915/intel_mocs.h +1 −1
drivers/gpu/drm/i915/intel_pm.c +62 −39
drivers/gpu/drm/i915/intel_psr.c +192 −85
drivers/gpu/drm/i915/intel_ringbuffer.c +56 −54
drivers/gpu/drm/i915/intel_ringbuffer.h +29 −10
drivers/gpu/drm/i915/intel_runtime_pm.c +674 −329
drivers/gpu/drm/i915/intel_sprite.c +7 −6
drivers/gpu/drm/i915/intel_uc_fw.c +1 −1
drivers/gpu/drm/i915/intel_uncore.c +92 −38
drivers/gpu/drm/i915/intel_wopcm.c +6 −0
drivers/gpu/drm/i915/selftests/huge_pages.c +6 −5
drivers/gpu/drm/i915/selftests/i915_gem.c +221 −0
drivers/gpu/drm/i915/selftests/i915_gem_coherency.c +17 −21
drivers/gpu/drm/i915/selftests/i915_gem_object.c +1 −1
drivers/gpu/drm/i915/selftests/i915_live_selftests.h +1 −0
drivers/gpu/drm/i915/selftests/intel_guc.c +38 −0
drivers/gpu/drm/i915/selftests/intel_hangcheck.c +89 −12
drivers/gpu/drm/i915/selftests/mock_context.c +3 −8
drivers/gpu/drm/i915/selftests/mock_gtt.c +2 −0
include/drm/i915_pciids.h +1 −0
include/uapi/drm/i915_drm.h +22 −0

drivers/gpu/drm/i915/Kconfig.debug+12−0

@@ -30,6 +30,7 @@ config DRM_I915_DEBUG · 29 unmodified lines
30	select SW_SYNC # signaling validation framework (igt/syncobj*)	30	select SW_SYNC # signaling validation framework (igt/syncobj*)
31	select DRM_I915_SW_FENCE_DEBUG_OBJECTS	31	select DRM_I915_SW_FENCE_DEBUG_OBJECTS
32	select DRM_I915_SELFTEST	32	select DRM_I915_SELFTEST
		33	select DRM_I915_DEBUG_RUNTIME_PM
33	default n	34	default n
34	help	35	help
35	Choose this option to turn on extra driver debugging that may affect	36	Choose this option to turn on extra driver debugging that may affect
@@ -167,3 +168,14 @@ config DRM_I915_DEBUG_VBLANK_EVADE · 131 unmodified lines
167	the vblank.	168	the vblank.
168		169
169	If in doubt, say "N".	170	If in doubt, say "N".
		171
		172	config DRM_I915_DEBUG_RUNTIME_PM
		173	bool "Enable extra state checking for runtime PM"
		174	depends on DRM_I915
		175	default n
		176	help
		177	Choose this option to turn on extra state checking for the
		178	runtime PM functionality. This may introduce overhead during
		179	driver loading, suspend and resume operations.
		180
		181	If in doubt, say "N"

drivers/gpu/drm/i915/gvt/cfg_space.c+12−0

 /**
  * vgpu_pci_cfg_mem_write - write virtual cfg space memory
+ * @vgpu: target vgpu
+ * @off: offset
+ * @src: src ptr to write
+ * @bytes: number of bytes
  *
  * Use this function to write virtual cfg space memory.
  * For standard cfg space, only RW bits can be changed,
 /**
  * intel_vgpu_emulate_cfg_read - emulate vGPU configuration space read
+ * @vgpu: target vgpu
+ * @offset: offset
+ * @p_data: return data ptr
+ * @bytes: number of bytes to read
  *
  * Returns:
  * Zero on success, negative error code if failed.
 /**
  * intel_vgpu_emulate_cfg_read - emulate vGPU configuration space write
+ * @vgpu: target vgpu
+ * @offset: offset
+ * @p_data: write data ptr
+ * @bytes: number of bytes to write
  *
  * Returns:
  * Zero on success, negative error code if failed.

drivers/gpu/drm/i915/gvt/cmd_parser.c+10−1

 	return ret;
 }
+static int mi_noop_index;
 static struct cmd_info cmd_info[] = {
 	{"MI_NOOP", OP_MI_NOOP, F_LEN_CONST, R_ALL, D_ALL, 0, 1, NULL},
 	cmd = cmd_val(s, 0);
-	info = get_cmd_info(s->vgpu->gvt, cmd, s->ring_id);
+	/* fastpath for MI_NOOP */
+	if (cmd == MI_NOOP)
+		info = &cmd_info[mi_noop_index];
+	else
+		info = get_cmd_info(s->vgpu->gvt, cmd, s->ring_id);
 	if (info == NULL) {
 		gvt_vgpu_err("unknown cmd 0x%x, opcode=0x%x, addr_type=%s, ring %d, workload=%p\n",
 				cmd, get_opcode(cmd, s->ring_id),
 			kfree(e);
 			return -EEXIST;
 		}
+		if (cmd_info[i].opcode == OP_MI_NOOP)
+			mi_noop_index = i;
 		INIT_HLIST_NODE(&e->hlist);
 		add_cmd_entry(gvt, e);

drivers/gpu/drm/i915/gvt/display.c+1−0

@@ -462,6 +462,7 @@ void intel_vgpu_clean_display(struct intel_vgpu *vgpu) · 461 unmodified lines
462	/**	462	/**
463	* intel_vgpu_init_display- initialize vGPU virtual display emulation	463	* intel_vgpu_init_display- initialize vGPU virtual display emulation
464	* @vgpu: a vGPU	464	* @vgpu: a vGPU
		465	* @resolution: resolution index for intel_vgpu_edid
465	*	466	*
466	* This function is used to initialize vGPU virtual display emulation stuffs	467	* This function is used to initialize vGPU virtual display emulation stuffs
467	*	468	*

drivers/gpu/drm/i915/gvt/edid.c+9−0

 /**
  * intel_gvt_i2c_handle_gmbus_read - emulate gmbus register mmio read
  * @vgpu: a vGPU
+ * @offset: reg offset
+ * @p_data: data return buffer
+ * @bytes: access data length
  *
  * This function is used to emulate gmbus register mmio read
  *
 /**
  * intel_gvt_i2c_handle_gmbus_write - emulate gmbus register mmio write
  * @vgpu: a vGPU
+ * @offset: reg offset
+ * @p_data: data return buffer
+ * @bytes: access data length
  *
  * This function is used to emulate gmbus register mmio write
  *
 /**
  * intel_gvt_i2c_handle_aux_ch_write - emulate AUX channel register write
  * @vgpu: a vGPU
+ * @port_idx: port index
+ * @offset: reg offset
+ * @p_data: write ptr
  *
  * This function is used to emulate AUX channel register write
  *

drivers/gpu/drm/i915/gvt/gtt.c+6−3

 }
 /**
+ * Check if can do 2M page
+ * @vgpu: target vgpu
+ * @entry: target pfn's gtt entry
+ *
  * Return 1 if 2MB huge gtt shadowing is possilbe, 0 if miscondition,
  * negtive if found err.
  */
 /**
  * intel_vgpu_pin_mm - increase the pin count of a vGPU mm object
- * @vgpu: a vGPU
+ * @mm: target vgpu mm
  *
  * This function is called when user wants to use a vGPU mm object. If this
  * mm object hasn't been shadowed yet, the shadow will be populated at this
 /**
  * intel_vgpu_find_ppgtt_mm - find a PPGTT mm object
  * @vgpu: a vGPU
- * @page_table_level: PPGTT page table level
+ * @pdps: pdp root array
- * @root_entry: PPGTT page table root pointers
  *
  * This function is used to find a PPGTT mm object from mm object pool
  *

drivers/gpu/drm/i915/gvt/gvt.c+1−2

 /**
  * intel_gvt_init_host - Load MPT modules and detect if we're running in host
- * @gvt: intel gvt device
  *
  * This function is called at the driver loading stage. If failed to find a
  * loadable MPT module or detect currently we're running in a VM, then GVT-g
 /**
  * intel_gvt_clean_device - clean a GVT device
- * @gvt: intel gvt device
+ * @dev_priv: i915 private
  *
  * This function is called at the driver unloading stage, to free the
  * resources owned by a GVT device.

drivers/gpu/drm/i915/gvt/handlers.c+12−22

 {
 	write_vreg(vgpu, offset, p_data, bytes);
-	if (vgpu_vreg(vgpu, offset) & HSW_PWR_WELL_CTL_REQ(HSW_DISP_PW_GLOBAL))
+	if (vgpu_vreg(vgpu, offset) &
+	    HSW_PWR_WELL_CTL_REQ(HSW_PW_CTL_IDX_GLOBAL))
 		vgpu_vreg(vgpu, offset) |=
-			HSW_PWR_WELL_CTL_STATE(HSW_DISP_PW_GLOBAL);
+			HSW_PWR_WELL_CTL_STATE(HSW_PW_CTL_IDX_GLOBAL);
 	else
 		vgpu_vreg(vgpu, offset) &=
-			~HSW_PWR_WELL_CTL_STATE(HSW_DISP_PW_GLOBAL);
+			~HSW_PWR_WELL_CTL_STATE(HSW_PW_CTL_IDX_GLOBAL);
 	return 0;
 }
 	MMIO_F(PCH_GMBUS0, 4 * 4, 0, 0, 0, D_ALL, gmbus_mmio_read,
 		gmbus_mmio_write);
-	MMIO_F(PCH_GPIOA, 6 * 4, F_UNALIGN, 0, 0, D_ALL, NULL, NULL);
+	MMIO_F(PCH_GPIO_BASE, 6 * 4, F_UNALIGN, 0, 0, D_ALL, NULL, NULL);
 	MMIO_F(_MMIO(0xe4f00), 0x28, 0, 0, 0, D_ALL, NULL, NULL);
 	MMIO_F(_MMIO(_PCH_DPB_AUX_CH_CTL), 6 * 4, 0, 0, 0, D_PRE_SKL, NULL,
 	MMIO_D(GEN6_RC6p_THRESHOLD, D_ALL);
 	MMIO_D(GEN6_RC6pp_THRESHOLD, D_ALL);
 	MMIO_D(GEN6_PMINTRMSK, D_ALL);
-	/*
+	MMIO_DH(HSW_PWR_WELL_CTL1, D_BDW, NULL, power_well_ctl_mmio_write);
-	 * Use an arbitrary power well controlled by the PWR_WELL_CTL
+	MMIO_DH(HSW_PWR_WELL_CTL2, D_BDW, NULL, power_well_ctl_mmio_write);
-	 * register.
+	MMIO_DH(HSW_PWR_WELL_CTL3, D_BDW, NULL, power_well_ctl_mmio_write);
-	 */
+	MMIO_DH(HSW_PWR_WELL_CTL4, D_BDW, NULL, power_well_ctl_mmio_write);
-	MMIO_DH(HSW_PWR_WELL_CTL_BIOS(HSW_DISP_PW_GLOBAL), D_BDW, NULL,
-		power_well_ctl_mmio_write);
-	MMIO_DH(HSW_PWR_WELL_CTL_DRIVER(HSW_DISP_PW_GLOBAL), D_BDW, NULL,
-		power_well_ctl_mmio_write);
-	MMIO_DH(HSW_PWR_WELL_CTL_KVMR, D_BDW, NULL, power_well_ctl_mmio_write);
-	MMIO_DH(HSW_PWR_WELL_CTL_DEBUG(HSW_DISP_PW_GLOBAL), D_BDW, NULL,
-		power_well_ctl_mmio_write);
 	MMIO_DH(HSW_PWR_WELL_CTL5, D_BDW, NULL, power_well_ctl_mmio_write);
 	MMIO_DH(HSW_PWR_WELL_CTL6, D_BDW, NULL, power_well_ctl_mmio_write);
 	MMIO_F(_MMIO(_DPD_AUX_CH_CTL), 6 * 4, 0, 0, 0, D_SKL_PLUS, NULL,
 						dp_aux_ch_ctl_mmio_write);
-	/*
+	MMIO_D(HSW_PWR_WELL_CTL1, D_SKL_PLUS);
-	 * Use an arbitrary power well controlled by the PWR_WELL_CTL
+	MMIO_DH(HSW_PWR_WELL_CTL2, D_SKL_PLUS, NULL, skl_power_well_ctl_write);
-	 * register.
-	 */
-	MMIO_D(HSW_PWR_WELL_CTL_BIOS(SKL_DISP_PW_MISC_IO), D_SKL_PLUS);
-	MMIO_DH(HSW_PWR_WELL_CTL_DRIVER(SKL_DISP_PW_MISC_IO), D_SKL_PLUS, NULL,
-		skl_power_well_ctl_write);
 	MMIO_D(_MMIO(0xa210), D_SKL_PLUS);
 	MMIO_D(GEN9_MEDIA_PG_IDLE_HYSTERESIS, D_SKL_PLUS);
  * @offset: register offset
  * @pdata: data buffer
  * @bytes: data length
+ * @is_read: read or write
  *
  * Returns:
  * Zero on success, negative error code if failed.

drivers/gpu/drm/i915/gvt/kvmgt.c+2−2

@@ -1712,7 +1712,7 @@ static unsigned long kvmgt_gfn_to_pfn(unsigned long handle, unsigned long gfn) · 1711 unmodified lines
1712	return pfn;	1712	return pfn;
1713	}	1713	}
1714		1714
1715	int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,	1715	static int kvmgt_dma_map_guest_page(unsigned long handle, unsigned long gfn,
1716	unsigned long size, dma_addr_t *dma_addr)	1716	unsigned long size, dma_addr_t *dma_addr)
1717	{	1717	{
1718	struct kvmgt_guest_info *info;	1718	struct kvmgt_guest_info *info;
@@ -1761,7 +1761,7 @@ static void __gvt_dma_release(struct kref *ref) · 42 unmodified lines
1761	__gvt_cache_remove_entry(entry->vgpu, entry);	1761	__gvt_cache_remove_entry(entry->vgpu, entry);
1762	}	1762	}
1763		1763
1764	void kvmgt_dma_unmap_guest_page(unsigned long handle, dma_addr_t dma_addr)	1764	static void kvmgt_dma_unmap_guest_page(unsigned long handle, dma_addr_t dma_addr)
1765	{	1765	{
1766	struct kvmgt_guest_info *info;	1766	struct kvmgt_guest_info *info;
1767	struct gvt_dma *entry;	1767	struct gvt_dma *entry;

drivers/gpu/drm/i915/gvt/mmio.c+2−1

 /**
  * intel_vgpu_gpa_to_mmio_offset - translate a GPA to MMIO offset
  * @vgpu: a vGPU
+ * @gpa: guest physical address
  *
  * Returns:
  * Zero on success, negative error code if failed
 /**
  * intel_vgpu_reset_mmio - reset virtual MMIO space
  * @vgpu: a vGPU
- *
+ * @dmlr: whether this is device model level reset
  */
 void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu, bool dmlr)
 {

drivers/gpu/drm/i915/gvt/mmio_context.c+0−13

@@ -37,19 +37,6 @@ · 36 unmodified lines
37	#include "gvt.h"	37	#include "gvt.h"
38	#include "trace.h"	38	#include "trace.h"
39		39
40	/**
41	* Defined in Intel Open Source PRM.
42	* Ref: https://01.org/linuxgraphics/documentation/hardware-specification-prms
43	*/
44	#define TRVATTL3PTRDW(i) _MMIO(0x4de0 + (i)*4)
45	#define TRNULLDETCT _MMIO(0x4de8)
46	#define TRINVTILEDETCT _MMIO(0x4dec)
47	#define TRVADR _MMIO(0x4df0)
48	#define TRTTE _MMIO(0x4df4)
49	#define RING_EXCC(base) _MMIO((base) + 0x28)
50	#define RING_GFX_MODE(base) _MMIO((base) + 0x29c)
51	#define VF_GUARDBAND _MMIO(0x83a4)
52
53	#define GEN9_MOCS_SIZE 64	40	#define GEN9_MOCS_SIZE 64
54		41
55	/* Raw offset is appened to each line for convenience. */	42	/* Raw offset is appened to each line for convenience. */

drivers/gpu/drm/i915/gvt/mmio_context.h+3−0

@@ -53,5 +53,8 @@ bool is_inhibit_context(struct intel_context *ce); · 52 unmodified lines
53		53
54	int intel_vgpu_restore_inhibit_context(struct intel_vgpu *vgpu,	54	int intel_vgpu_restore_inhibit_context(struct intel_vgpu *vgpu,
55	struct i915_request *req);	55	struct i915_request *req);
		56	#define IS_RESTORE_INHIBIT(a) \
		57	(_MASKED_BIT_ENABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT) == \
		58	((a) & _MASKED_BIT_ENABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT)))
56		59
57	#endif	60	#endif

drivers/gpu/drm/i915/gvt/opregion.c+0−1

@@ -216,7 +216,6 @@ static void virt_vbt_generation(struct vbt *v) · 215 unmodified lines
216	/**	216	/**
217	* intel_vgpu_init_opregion - initialize the stuff used to emulate opregion	217	* intel_vgpu_init_opregion - initialize the stuff used to emulate opregion
218	* @vgpu: a vGPU	218	* @vgpu: a vGPU
219	* @gpa: guest physical address of opregion
220	*	219	*
221	* Returns:	220	* Returns:
222	* Zero on success, negative error code if failed.	221	* Zero on success, negative error code if failed.

drivers/gpu/drm/i915/gvt/page_track.c+2−0

  * intel_vgpu_register_page_track - register a guest page to be tacked
  * @vgpu: a vGPU
  * @gfn: the gfn of guest page
+ * @handler: page track handler
+ * @priv: tracker private
  *
  * Returns:
  * zero on success, negative error code if failed.

drivers/gpu/drm/i915/gvt/reg.h+18−0

@@ -77,4 +77,22 @@ · 76 unmodified lines
77	#define _RING_CTL_BUF_SIZE(ctl) (((ctl) & RB_TAIL_SIZE_MASK) + \	77	#define _RING_CTL_BUF_SIZE(ctl) (((ctl) & RB_TAIL_SIZE_MASK) + \
78	I915_GTT_PAGE_SIZE)	78	I915_GTT_PAGE_SIZE)
79		79
		80	#define PCH_GPIO_BASE _MMIO(0xc5010)
		81
		82	#define PCH_GMBUS0 _MMIO(0xc5100)
		83	#define PCH_GMBUS1 _MMIO(0xc5104)
		84	#define PCH_GMBUS2 _MMIO(0xc5108)
		85	#define PCH_GMBUS3 _MMIO(0xc510c)
		86	#define PCH_GMBUS4 _MMIO(0xc5110)
		87	#define PCH_GMBUS5 _MMIO(0xc5120)
		88
		89	#define TRVATTL3PTRDW(i) _MMIO(0x4de0 + (i) * 4)
		90	#define TRNULLDETCT _MMIO(0x4de8)
		91	#define TRINVTILEDETCT _MMIO(0x4dec)
		92	#define TRVADR _MMIO(0x4df0)
		93	#define TRTTE _MMIO(0x4df4)
		94	#define RING_EXCC(base) _MMIO((base) + 0x28)
		95	#define RING_GFX_MODE(base) _MMIO((base) + 0x29c)
		96	#define VF_GUARDBAND _MMIO(0x83a4)
		97
80	#endif	98	#endif

drivers/gpu/drm/i915/gvt/scheduler.c+34−30

 	unsigned long context_gpa, context_page_num;
 	int i;
-	gvt_dbg_sched("ring id %d workload lrca %x", ring_id,
-			workload->ctx_desc.lrca);
-	context_page_num = gvt->dev_priv->engine[ring_id]->context_size;
-	context_page_num = context_page_num >> PAGE_SHIFT;
-	if (IS_BROADWELL(gvt->dev_priv) && ring_id == RCS)
-		context_page_num = 19;
-	i = 2;
-	while (i < context_page_num) {
-		context_gpa = intel_vgpu_gma_to_gpa(vgpu->gtt.ggtt_mm,
-				(u32)((workload->ctx_desc.lrca + i) <<
-				I915_GTT_PAGE_SHIFT));
-		if (context_gpa == INTEL_GVT_INVALID_ADDR) {
-			gvt_vgpu_err("Invalid guest context descriptor\n");
-			return -EFAULT;
-		}
-		page = i915_gem_object_get_page(ctx_obj, LRC_HEADER_PAGES + i);
-		dst = kmap(page);
-		intel_gvt_hypervisor_read_gpa(vgpu, context_gpa, dst,
-				I915_GTT_PAGE_SIZE);
-		kunmap(page);
-		i++;
-	}
 	page = i915_gem_object_get_page(ctx_obj, LRC_STATE_PN);
 	shadow_ring_context = kmap(page);
 	sr_oa_regs(workload, (u32 *)shadow_ring_context, false);
 	kunmap(page);
+	if (IS_RESTORE_INHIBIT(shadow_ring_context->ctx_ctrl.val))
+		return 0;
+	gvt_dbg_sched("ring id %d workload lrca %x", ring_id,
+			workload->ctx_desc.lrca);
+	context_page_num = gvt->dev_priv->engine[ring_id]->context_size;
+	context_page_num = context_page_num >> PAGE_SHIFT;
+	if (IS_BROADWELL(gvt->dev_priv) && ring_id == RCS)
+		context_page_num = 19;
+	i = 2;
+	while (i < context_page_num) {
+		context_gpa = intel_vgpu_gma_to_gpa(vgpu->gtt.ggtt_mm,
+				(u32)((workload->ctx_desc.lrca + i) <<
+				I915_GTT_PAGE_SHIFT));
+		if (context_gpa == INTEL_GVT_INVALID_ADDR) {
+			gvt_vgpu_err("Invalid guest context descriptor\n");
+			return -EFAULT;
+		}
+		page = i915_gem_object_get_page(ctx_obj, LRC_HEADER_PAGES + i);
+		dst = kmap(page);
+		intel_gvt_hypervisor_read_gpa(vgpu, context_gpa, dst,
+				I915_GTT_PAGE_SIZE);
+		kunmap(page);
+		i++;
+	}
 	return 0;
 }
 /**
  * intel_vgpu_select_submission_ops - select virtual submission interface
  * @vgpu: a vGPU
+ * @engine_mask: either ALL_ENGINES or target engine mask
  * @interface: expected vGPU virtual submission interface
  *
  * This function is called when guest configures submission interface.
 /**
  * intel_vgpu_destroy_workload - destroy a vGPU workload
- * @vgpu: a vGPU
+ * @workload: workload to destroy
  *
  * This function is called when destroy a vGPU workload.
  *
 /**
  * intel_vgpu_create_workload - create a vGPU workload
  * @vgpu: a vGPU
+ * @ring_id: ring index
  * @desc: a guest context descriptor
  *
  * This function is called when creating a vGPU workload.

drivers/gpu/drm/i915/i915_debugfs.c+64−24

@@ -1953,7 +1953,10 @@ static int i915_context_status(struct seq_file m, void unused) · 1952 unmodified lines
1953	return ret;	1953	return ret;
1954		1954
1955	list_for_each_entry(ctx, &dev_priv->contexts.list, link) {	1955	list_for_each_entry(ctx, &dev_priv->contexts.list, link) {
1956	seq_printf(m, "HW context %u ", ctx->hw_id);	1956	seq_puts(m, "HW context ");
		1957	if (!list_empty(&ctx->hw_id_link))
		1958	seq_printf(m, "%x [pin %u]", ctx->hw_id,
		1959	atomic_read(&ctx->hw_id_pin_count));
1957	if (ctx->pid) {	1960	if (ctx->pid) {
1958	struct task_struct *task;	1961	struct task_struct *task;
1959		1962
@@ -2708,7 +2711,9 @@ static int i915_edp_psr_status(struct seq_file m, void data) · 748 unmodified lines
2708	intel_runtime_pm_get(dev_priv);	2711	intel_runtime_pm_get(dev_priv);
2709		2712
2710	mutex_lock(&dev_priv->psr.lock);	2713	mutex_lock(&dev_priv->psr.lock);
2711	seq_printf(m, "Enabled: %s\n", yesno((bool)dev_priv->psr.enabled));	2714	seq_printf(m, "PSR mode: %s\n",
		2715	dev_priv->psr.psr2_enabled ? "PSR2" : "PSR1");
		2716	seq_printf(m, "Enabled: %s\n", yesno(dev_priv->psr.enabled));
2712	seq_printf(m, "Busy frontbuffer bits: 0x%03x\n",	2717	seq_printf(m, "Busy frontbuffer bits: 0x%03x\n",
2713	dev_priv->psr.busy_frontbuffer_bits);	2718	dev_priv->psr.busy_frontbuffer_bits);
2714		2719
@@ -2735,7 +2740,7 @@ static int i915_edp_psr_status(struct seq_file m, void data) · 20 unmodified lines
2735	psr_source_status(dev_priv, m);	2740	psr_source_status(dev_priv, m);
2736	mutex_unlock(&dev_priv->psr.lock);	2741	mutex_unlock(&dev_priv->psr.lock);
2737		2742
2738	if (READ_ONCE(dev_priv->psr.debug)) {	2743	if (READ_ONCE(dev_priv->psr.debug) & I915_PSR_DEBUG_IRQ) {
2739	seq_printf(m, "Last attempted entry at: %lld\n",	2744	seq_printf(m, "Last attempted entry at: %lld\n",
2740	dev_priv->psr.last_entry_attempt);	2745	dev_priv->psr.last_entry_attempt);
2741	seq_printf(m, "Last exit at: %lld\n",	2746	seq_printf(m, "Last exit at: %lld\n",
@@ -2750,17 +2755,32 @@ static int · 8 unmodified lines
2750	i915_edp_psr_debug_set(void *data, u64 val)	2755	i915_edp_psr_debug_set(void *data, u64 val)
2751	{	2756	{
2752	struct drm_i915_private *dev_priv = data;	2757	struct drm_i915_private *dev_priv = data;
		2758	struct drm_modeset_acquire_ctx ctx;
		2759	int ret;
2753		2760
2754	if (!CAN_PSR(dev_priv))	2761	if (!CAN_PSR(dev_priv))
2755	return -ENODEV;	2762	return -ENODEV;
2756		2763
2757	DRM_DEBUG_KMS("PSR debug %s\n", enableddisabled(val));	2764	DRM_DEBUG_KMS("Setting PSR debug to %llx\n", val);
2758		2765
2759	intel_runtime_pm_get(dev_priv);	2766	intel_runtime_pm_get(dev_priv);
2760	intel_psr_irq_control(dev_priv, !!val);	2767
		2768	drm_modeset_acquire_init(&ctx, DRM_MODESET_ACQUIRE_INTERRUPTIBLE);
		2769
		2770	retry:
		2771	ret = intel_psr_set_debugfs_mode(dev_priv, &ctx, val);
		2772	if (ret == -EDEADLK) {
		2773	ret = drm_modeset_backoff(&ctx);
		2774	if (!ret)
		2775	goto retry;
		2776	}
		2777
		2778	drm_modeset_drop_locks(&ctx);
		2779	drm_modeset_acquire_fini(&ctx);
		2780
2761	intel_runtime_pm_put(dev_priv);	2781	intel_runtime_pm_put(dev_priv);
2762		2782
2763	return 0;	2783	return ret;
2764	}	2784	}
2765		2785
2766	static int	2786	static int
@@ -2845,10 +2865,10 @@ static int i915_power_domain_info(struct seq_file m, void unused) · 78 unmodified lines
2845	enum intel_display_power_domain power_domain;	2865	enum intel_display_power_domain power_domain;
2846		2866
2847	power_well = &power_domains->power_wells[i];	2867	power_well = &power_domains->power_wells[i];
2848	seq_printf(m, "%-25s %d\n", power_well->name,	2868	seq_printf(m, "%-25s %d\n", power_well->desc->name,
2849	power_well->count);	2869	power_well->count);
2850		2870
2851	for_each_power_domain(power_domain, power_well->domains)	2871	for_each_power_domain(power_domain, power_well->desc->domains)
2852	seq_printf(m, " %-23s %d\n",	2872	seq_printf(m, " %-23s %d\n",
2853	intel_display_power_domain_str(power_domain),	2873	intel_display_power_domain_str(power_domain),
2854	power_domains->domain_use_count[power_domain]);	2874	power_domains->domain_use_count[power_domain]);
@@ -4114,13 +4134,17 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_ring_test_irq_fops, · 1259 unmodified lines
4114	#define DROP_FREED BIT(4)	4134	#define DROP_FREED BIT(4)
4115	#define DROP_SHRINK_ALL BIT(5)	4135	#define DROP_SHRINK_ALL BIT(5)
4116	#define DROP_IDLE BIT(6)	4136	#define DROP_IDLE BIT(6)
		4137	#define DROP_RESET_ACTIVE BIT(7)
		4138	#define DROP_RESET_SEQNO BIT(8)
4117	#define DROP_ALL (DROP_UNBOUND \| \	4139	#define DROP_ALL (DROP_UNBOUND \| \
4118	DROP_BOUND \| \	4140	DROP_BOUND \| \
4119	DROP_RETIRE \| \	4141	DROP_RETIRE \| \
4120	DROP_ACTIVE \| \	4142	DROP_ACTIVE \| \
4121	DROP_FREED \| \	4143	DROP_FREED \| \
4122	DROP_SHRINK_ALL \|\	4144	DROP_SHRINK_ALL \|\
4123	DROP_IDLE)	4145	DROP_IDLE \| \
		4146	DROP_RESET_ACTIVE \| \
		4147	DROP_RESET_SEQNO)
4124	static int	4148	static int
4125	i915_drop_caches_get(void data, u64 val)	4149	i915_drop_caches_get(void data, u64 val)
4126	{	4150	{
@@ -4132,53 +4156,69 @@ i915_drop_caches_get(void data, u64 val) · 5 unmodified lines
4132	static int	4156	static int
4133	i915_drop_caches_set(void *data, u64 val)	4157	i915_drop_caches_set(void *data, u64 val)
4134	{	4158	{
4135	struct drm_i915_private *dev_priv = data;	4159	struct drm_i915_private *i915 = data;
4136	struct drm_device *dev = &dev_priv->drm;
4137	int ret = 0;	4160	int ret = 0;
4138		4161
4139	DRM_DEBUG("Dropping caches: 0x%08llx [0x%08llx]\n",	4162	DRM_DEBUG("Dropping caches: 0x%08llx [0x%08llx]\n",
4140	val, val & DROP_ALL);	4163	val, val & DROP_ALL);
4141		4164
		4165	if (val & DROP_RESET_ACTIVE && !intel_engines_are_idle(i915))
		4166	i915_gem_set_wedged(i915);
		4167
4142	/* No need to check and wait for gpu resets, only libdrm auto-restarts	4168	/* No need to check and wait for gpu resets, only libdrm auto-restarts
4143	* on ioctls on -EAGAIN. */	4169	* on ioctls on -EAGAIN. */
4144	if (val & (DROP_ACTIVE \| DROP_RETIRE)) {	4170	if (val & (DROP_ACTIVE \| DROP_RETIRE \| DROP_RESET_SEQNO)) {
4145	ret = mutex_lock_interruptible(&dev->struct_mutex);	4171	ret = mutex_lock_interruptible(&i915->drm.struct_mutex);
4146	if (ret)	4172	if (ret)
4147	return ret;	4173	return ret;
4148		4174
4149	if (val & DROP_ACTIVE)	4175	if (val & DROP_ACTIVE)
4150	ret = i915_gem_wait_for_idle(dev_priv,	4176	ret = i915_gem_wait_for_idle(i915,
4151	I915_WAIT_INTERRUPTIBLE \|	4177	I915_WAIT_INTERRUPTIBLE \|
4152	I915_WAIT_LOCKED,	4178	I915_WAIT_LOCKED,
4153	MAX_SCHEDULE_TIMEOUT);	4179	MAX_SCHEDULE_TIMEOUT);
4154		4180
		4181	if (val & DROP_RESET_SEQNO) {
		4182	intel_runtime_pm_get(i915);
		4183	ret = i915_gem_set_global_seqno(&i915->drm, 1);
		4184	intel_runtime_pm_put(i915);
		4185	}
		4186
4155	if (val & DROP_RETIRE)	4187	if (val & DROP_RETIRE)
4156	i915_retire_requests(dev_priv);	4188	i915_retire_requests(i915);
4157		4189
4158	mutex_unlock(&dev->struct_mutex);	4190	mutex_unlock(&i915->drm.struct_mutex);
		4191	}
		4192
		4193	if (val & DROP_RESET_ACTIVE &&
		4194	i915_terminally_wedged(&i915->gpu_error)) {
		4195	i915_handle_error(i915, ALL_ENGINES, 0, NULL);
		4196	wait_on_bit(&i915->gpu_error.flags,
		4197	I915_RESET_HANDOFF,
		4198	TASK_UNINTERRUPTIBLE);
4159	}	4199	}
4160		4200
4161	fs_reclaim_acquire(GFP_KERNEL);	4201	fs_reclaim_acquire(GFP_KERNEL);
4162	if (val & DROP_BOUND)	4202	if (val & DROP_BOUND)
4163	i915_gem_shrink(dev_priv, LONG_MAX, NULL, I915_SHRINK_BOUND);	4203	i915_gem_shrink(i915, LONG_MAX, NULL, I915_SHRINK_BOUND);
4164		4204
4165	if (val & DROP_UNBOUND)	4205	if (val & DROP_UNBOUND)
4166	i915_gem_shrink(dev_priv, LONG_MAX, NULL, I915_SHRINK_UNBOUND);	4206	i915_gem_shrink(i915, LONG_MAX, NULL, I915_SHRINK_UNBOUND);
4167		4207
4168	if (val & DROP_SHRINK_ALL)	4208	if (val & DROP_SHRINK_ALL)
4169	i915_gem_shrink_all(dev_priv);	4209	i915_gem_shrink_all(i915);
4170	fs_reclaim_release(GFP_KERNEL);	4210	fs_reclaim_release(GFP_KERNEL);
4171		4211
4172	if (val & DROP_IDLE) {	4212	if (val & DROP_IDLE) {
4173	do {	4213	do {
4174	if (READ_ONCE(dev_priv->gt.active_requests))	4214	if (READ_ONCE(i915->gt.active_requests))
4175	flush_delayed_work(&dev_priv->gt.retire_work);	4215	flush_delayed_work(&i915->gt.retire_work);
4176	drain_delayed_work(&dev_priv->gt.idle_work);	4216	drain_delayed_work(&i915->gt.idle_work);
4177	} while (READ_ONCE(dev_priv->gt.awake));	4217	} while (READ_ONCE(i915->gt.awake));
4178	}	4218	}
4179		4219
4180	if (val & DROP_FREED)	4220	if (val & DROP_FREED)
4181	i915_gem_drain_freed_objects(dev_priv);	4221	i915_gem_drain_freed_objects(i915);
4182		4222
4183	return ret;	4223	return ret;
4184	}	4224	}

drivers/gpu/drm/i915/i915_drv.c+104−90

 			value = 2;
 		break;
 	case I915_PARAM_HAS_RESOURCE_STREAMER:
-		value = HAS_RESOURCE_STREAMER(dev_priv);
+		value = 0;
 		break;
 	case I915_PARAM_HAS_POOLED_EU:
 		value = HAS_POOLED_EU(dev_priv);
 	case I915_PARAM_CS_TIMESTAMP_FREQUENCY:
 		value = 1000 * INTEL_INFO(dev_priv)->cs_timestamp_frequency_khz;
 		break;
+	case I915_PARAM_MMAP_GTT_COHERENT:
+		value = INTEL_INFO(dev_priv)->has_coherent_ggtt;
+		break;
 	default:
 		DRM_DEBUG("Unknown parameter %d\n", param->param);
 		return -EINVAL;
 	intel_teardown_gmbus(dev_priv);
 cleanup_csr:
 	intel_csr_ucode_fini(dev_priv);
-	intel_power_domains_fini(dev_priv);
+	intel_power_domains_fini_hw(dev_priv);
 	vga_switcheroo_unregister_client(pdev);
 cleanup_vga_client:
 	vga_client_register(pdev, NULL, NULL, NULL);
 /**
  * i915_driver_init_early - setup state not requiring device access
  * @dev_priv: device private
- * @ent: the matching pci_device_id
  *
  * Initialize everything that is a "SW-only" state, that is state not
  * requiring accessing the device or exposing the driver via kernel internal
  * system memory allocation, setting up device specific attributes and
  * function hooks not requiring accessing the device.
  */
-static int i915_driver_init_early(struct drm_i915_private *dev_priv,
+static int i915_driver_init_early(struct drm_i915_private *dev_priv)
-				  const struct pci_device_id *ent)
 {
-	const struct intel_device_info *match_info =
-		(struct intel_device_info *)ent->driver_data;
-	struct intel_device_info *device_info;
 	int ret = 0;
 	if (i915_inject_load_failure())
 		return -ENODEV;
-	/* Setup the write-once "constant" device info */
-	device_info = mkwrite_device_info(dev_priv);
-	memcpy(device_info, match_info, sizeof(*device_info));
-	device_info->device_id = dev_priv->drm.pdev->device;
-	BUILD_BUG_ON(INTEL_MAX_PLATFORMS >
-		     sizeof(device_info->platform_mask) * BITS_PER_BYTE);
-	BUG_ON(device_info->gen > sizeof(device_info->gen_mask) * BITS_PER_BYTE);
 	spin_lock_init(&dev_priv->irq_lock);
 	spin_lock_init(&dev_priv->gpu_error.lock);
 	mutex_init(&dev_priv->backlight_lock);
 	intel_uc_init_early(dev_priv);
 	intel_pm_setup(dev_priv);
 	intel_init_dpio(dev_priv);
-	intel_power_domains_init(dev_priv);
+	ret = intel_power_domains_init(dev_priv);
+	if (ret < 0)
+		goto err_uc;
 	intel_irq_init(dev_priv);
 	intel_hangcheck_init(dev_priv);
 	intel_init_display_hooks(dev_priv);
 	return 0;
+err_uc:
+	intel_uc_cleanup_early(dev_priv);
+	i915_gem_cleanup_early(dev_priv);
 err_workqueues:
 	i915_workqueues_cleanup(dev_priv);
 err_engines:
 static void i915_driver_cleanup_early(struct drm_i915_private *dev_priv)
 {
 	intel_irq_fini(dev_priv);
+	intel_power_domains_cleanup(dev_priv);
 	intel_uc_cleanup_early(dev_priv);
 	i915_gem_cleanup_early(dev_priv);
 	i915_workqueues_cleanup(dev_priv);
 	 */
 	if (INTEL_INFO(dev_priv)->num_pipes)
 		drm_kms_helper_poll_init(dev);
+	intel_power_domains_enable(dev_priv);
+	intel_runtime_pm_enable(dev_priv);
 }
 /**
  */
 static void i915_driver_unregister(struct drm_i915_private *dev_priv)
 {
+	intel_runtime_pm_disable(dev_priv);
+	intel_power_domains_disable(dev_priv);
 	intel_fbdev_unregister(dev_priv);
 	intel_audio_deinit(dev_priv);
 		DRM_INFO("DRM_I915_DEBUG enabled\n");
 	if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
 		DRM_INFO("DRM_I915_DEBUG_GEM enabled\n");
+	if (IS_ENABLED(CONFIG_DRM_I915_DEBUG_RUNTIME_PM))
+		DRM_INFO("DRM_I915_DEBUG_RUNTIME_PM enabled\n");
+}
+static struct drm_i915_private *
+i915_driver_create(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+	const struct intel_device_info *match_info =
+		(struct intel_device_info *)ent->driver_data;
+	struct intel_device_info *device_info;
+	struct drm_i915_private *i915;
+	i915 = kzalloc(sizeof(*i915), GFP_KERNEL);
+	if (!i915)
+		return NULL;
+	if (drm_dev_init(&i915->drm, &driver, &pdev->dev)) {
+		kfree(i915);
+		return NULL;
+	}
+	i915->drm.pdev = pdev;
+	i915->drm.dev_private = i915;
+	pci_set_drvdata(pdev, &i915->drm);
+	/* Setup the write-once "constant" device info */
+	device_info = mkwrite_device_info(i915);
+	memcpy(device_info, match_info, sizeof(*device_info));
+	device_info->device_id = pdev->device;
+	BUILD_BUG_ON(INTEL_MAX_PLATFORMS >
+		     sizeof(device_info->platform_mask) * BITS_PER_BYTE);
+	BUG_ON(device_info->gen > sizeof(device_info->gen_mask) * BITS_PER_BYTE);
+	return i915;
+}
+static void i915_driver_destroy(struct drm_i915_private *i915)
+{
+	struct pci_dev *pdev = i915->drm.pdev;
+	drm_dev_fini(&i915->drm);
+	kfree(i915);
+	/* And make sure we never chase our dangling pointer from pci_dev */
+	pci_set_drvdata(pdev, NULL);
 }
 /**
 	if (!i915_modparams.nuclear_pageflip && match_info->gen < 5)
 		driver.driver_features &= ~DRIVER_ATOMIC;
-	ret = -ENOMEM;
+	dev_priv = i915_driver_create(pdev, ent);
-	dev_priv = kzalloc(sizeof(*dev_priv), GFP_KERNEL);
+	if (!dev_priv)
-	if (dev_priv)
+		return -ENOMEM;
-		ret = drm_dev_init(&dev_priv->drm, &driver, &pdev->dev);
-	if (ret) {
-		DRM_DEV_ERROR(&pdev->dev, "allocation failed\n");
-		goto out_free;
-	}
-	dev_priv->drm.pdev = pdev;
-	dev_priv->drm.dev_private = dev_priv;
 	ret = pci_enable_device(pdev);
 	if (ret)
 		goto out_fini;
-	pci_set_drvdata(pdev, &dev_priv->drm);
+	ret = i915_driver_init_early(dev_priv);
-	/*
-	 * Disable the system suspend direct complete optimization, which can
-	 * leave the device suspended skipping the driver's suspend handlers
-	 * if the device was already runtime suspended. This is needed due to
-	 * the difference in our runtime and system suspend sequence and
-	 * becaue the HDA driver may require us to enable the audio power
-	 * domain during system suspend.
-	 */
-	dev_pm_set_driver_flags(&pdev->dev, DPM_FLAG_NEVER_SKIP);
-	ret = i915_driver_init_early(dev_priv, ent);
 	if (ret < 0)
 		goto out_pci_disable;
-	intel_runtime_pm_get(dev_priv);
+	disable_rpm_wakeref_asserts(dev_priv);
 	ret = i915_driver_init_mmio(dev_priv);
 	if (ret < 0)
 	i915_driver_register(dev_priv);
-	intel_runtime_pm_enable(dev_priv);
 	intel_init_ipc(dev_priv);
-	intel_runtime_pm_put(dev_priv);
+	enable_rpm_wakeref_asserts(dev_priv);
 	i915_welcome_messages(dev_priv);
 out_cleanup_mmio:
 	i915_driver_cleanup_mmio(dev_priv);
 out_runtime_pm_put:
-	intel_runtime_pm_put(dev_priv);
+	enable_rpm_wakeref_asserts(dev_priv);
 	i915_driver_cleanup_early(dev_priv);
 out_pci_disable:
 	pci_disable_device(pdev);
 out_fini:
 	i915_load_error(dev_priv, "Device initialization failed (%d)\n", ret);
-	drm_dev_fini(&dev_priv->drm);
+	i915_driver_destroy(dev_priv);
-out_free:
-	kfree(dev_priv);
-	pci_set_drvdata(pdev, NULL);
 	return ret;
 }
 	struct drm_i915_private *dev_priv = to_i915(dev);
 	struct pci_dev *pdev = dev_priv->drm.pdev;
+	disable_rpm_wakeref_asserts(dev_priv);
 	i915_driver_unregister(dev_priv);
 	if (i915_gem_suspend(dev_priv))
 		DRM_ERROR("failed to idle hardware; continuing to unload!\n");
-	intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
 	drm_atomic_helper_shutdown(dev);
 	intel_gvt_cleanup(dev_priv);
 	i915_gem_fini(dev_priv);
 	intel_fbc_cleanup_cfb(dev_priv);
-	intel_power_domains_fini(dev_priv);
+	intel_power_domains_fini_hw(dev_priv);
 	i915_driver_cleanup_hw(dev_priv);
 	i915_driver_cleanup_mmio(dev_priv);
-	intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
+	enable_rpm_wakeref_asserts(dev_priv);
+	WARN_ON(atomic_read(&dev_priv->runtime_pm.wakeref_count));
 }
 static void i915_driver_release(struct drm_device *dev)
 	struct drm_i915_private *dev_priv = to_i915(dev);
 	i915_driver_cleanup_early(dev_priv);
-	drm_dev_fini(&dev_priv->drm);
+	i915_driver_destroy(dev_priv);
-	kfree(dev_priv);
 }
 static int i915_driver_open(struct drm_device *dev, struct drm_file *file)
 	/* We do a lot of poking in a lot of registers, make sure they work
 	 * properly. */
-	intel_display_set_init_power(dev_priv, true);
+	intel_power_domains_disable(dev_priv);
 	drm_kms_helper_poll_disable(dev);
 	return 0;
 }
+static enum i915_drm_suspend_mode
+get_suspend_mode(struct drm_i915_private *dev_priv, bool hibernate)
+{
+	if (hibernate)
+		return I915_DRM_SUSPEND_HIBERNATE;
+	if (suspend_to_idle(dev_priv))
+		return I915_DRM_SUSPEND_IDLE;
+	return I915_DRM_SUSPEND_MEM;
+}
 static int i915_drm_suspend_late(struct drm_device *dev, bool hibernation)
 {
 	struct drm_i915_private *dev_priv = to_i915(dev);
 	i915_gem_suspend_late(dev_priv);
-	intel_display_set_init_power(dev_priv, false);
 	intel_uncore_suspend(dev_priv);
-	/*
+	intel_power_domains_suspend(dev_priv,
-	 * In case of firmware assisted context save/restore don't manually
+				    get_suspend_mode(dev_priv, hibernation));
-	 * deinit the power domains. This also means the CSR/DMC firmware will
-	 * stay active, it will power down any HW resources as required and
-	 * also enable deeper system power states that would be blocked if the
-	 * firmware was inactive.
-	 */
-	if (IS_GEN9_LP(dev_priv) || hibernation || !suspend_to_idle(dev_priv) ||
-	    dev_priv->csr.dmc_payload == NULL) {
-		intel_power_domains_suspend(dev_priv);
-		dev_priv->power_domains_suspended = true;
-	}
 	ret = 0;
 	if (IS_GEN9_LP(dev_priv))
 	if (ret) {
 		DRM_ERROR("Suspend complete failed: %d\n", ret);
-		if (dev_priv->power_domains_suspended) {
+		intel_power_domains_resume(dev_priv);
-			intel_power_domains_init_hw(dev_priv, true);
-			dev_priv->power_domains_suspended = false;
-		}
 		goto out;
 	}
 	/*
 	 * ... but also need to make sure that hotplug processing
 	 * doesn't cause havoc. Like in the driver load code we don't
-	 * bother with the tiny race here where we might loose hotplug
+	 * bother with the tiny race here where we might lose hotplug
 	 * notifications.
 	 * */
 	intel_hpd_init(dev_priv);
 	intel_opregion_notify_adapter(dev_priv, PCI_D0);
+	intel_power_domains_enable(dev_priv);
 	enable_rpm_wakeref_asserts(dev_priv);
 	return 0;
 	ret = pci_set_power_state(pdev, PCI_D0);
 	if (ret) {
 		DRM_ERROR("failed to set PCI D0 power state (%d)\n", ret);
-		goto out;
+		return ret;
 	}
 	/*
 	 * depend on the device enable refcount we can't anyway depend on them
 	 * disabling/enabling the device.
 	 */
-	if (pci_enable_device(pdev)) {
+	if (pci_enable_device(pdev))
-		ret = -EIO;
+		return -EIO;
-		goto out;
-	}
 	pci_set_master(pdev);
 	intel_uncore_sanitize(dev_priv);
-	if (dev_priv->power_domains_suspended)
+	intel_power_domains_resume(dev_priv);
-		intel_power_domains_init_hw(dev_priv, true);
-	else
-		intel_display_set_init_power(dev_priv, true);
 	intel_engines_sanitize(dev_priv);
 	enable_rpm_wakeref_asserts(dev_priv);
-out:
-	dev_priv->power_domains_suspended = false;
 	return ret;
 }
 		dev_notice(i915->drm.dev, "Resetting chip for %s\n", reason);
 	error->reset_count++;
-	disable_irq(i915->drm.irq);
 	ret = i915_gem_reset_prepare(i915);
 	if (ret) {
 		dev_err(i915->drm.dev, "GPU recovery failed\n");
 finish:
 	i915_gem_reset_finish(i915);
-	enable_irq(i915->drm.irq);
 wakeup:
 	clear_bit(I915_RESET_HANDOFF, &error->flags);
 	wake_up_bit(&error->flags, I915_RESET_HANDOFF);
 		goto out;
 out:
+	intel_engine_cancel_stop_cs(engine);
 	i915_gem_reset_finish_engine(engine);
 	return ret;
 }

drivers/gpu/drm/i915/i915_drv.h+49−14

 #define DRIVER_NAME		"i915"
 #define DRIVER_DESC		"Intel Graphics"
-#define DRIVER_DATE		"20180719"
+#define DRIVER_DATE		"20180906"
-#define DRIVER_TIMESTAMP	1532015279
+#define DRIVER_TIMESTAMP	1536242083
 /* Use I915_STATE_WARN(x) and I915_STATE_WARN_ON() (rather than WARN() and
  * WARN_ON()) for hw state sanity checks to check for unexpected conditions
 struct i915_psr {
 	struct mutex lock;
+#define I915_PSR_DEBUG_MODE_MASK	0x0f
+#define I915_PSR_DEBUG_DEFAULT		0x00
+#define I915_PSR_DEBUG_DISABLE		0x01
+#define I915_PSR_DEBUG_ENABLE		0x02
+#define I915_PSR_DEBUG_FORCE_PSR1	0x03
+#define I915_PSR_DEBUG_IRQ		0x10
+	u32 debug;
 	bool sink_support;
-	struct intel_dp *enabled;
+	bool prepared, enabled;
+	struct intel_dp *dp;
 	bool active;
 	struct work_struct work;
 	unsigned busy_frontbuffer_bits;
 	bool alpm;
 	bool psr2_enabled;
 	u8 sink_sync_latency;
-	bool debug;
 	ktime_t last_entry_attempt;
 	ktime_t last_exit;
 };
 			   struct i915_power_well *power_well);
 };
+struct i915_power_well_regs {
+	i915_reg_t bios;
+	i915_reg_t driver;
+	i915_reg_t kvmr;
+	i915_reg_t debug;
+};
 /* Power well structure for haswell */
-struct i915_power_well {
+struct i915_power_well_desc {
 	const char *name;
 	bool always_on;
-	/* power well enable/disable usage count */
-	int count;
-	/* cached hw enabled state */
-	bool hw_enabled;
 	u64 domains;
 	/* unique identifier for this power well */
 	enum i915_power_well_id id;
 	 * well specific.
 	 */
 	union {
+		struct {
+			/*
+			 * request/status flag index in the PUNIT power well
+			 * control/status registers.
+			 */
+			u8 idx;
+		} vlv;
 		struct {
 			enum dpio_phy phy;
 		} bxt;
 		struct {
+			const struct i915_power_well_regs *regs;
+			/*
+			 * request/status flag index in the power well
+			 * constrol/status registers.
+			 */
+			u8 idx;
 			/* Mask of pipes whose IRQ logic is backed by the pw */
 			u8 irq_pipe_mask;
 			/* The pw is backing the VGA functionality */
 	const struct i915_power_well_ops *ops;
 };
+struct i915_power_well {
+	const struct i915_power_well_desc *desc;
+	/* power well enable/disable usage count */
+	int count;
+	/* cached hw enabled state */
+	bool hw_enabled;
+};
 struct i915_power_domains {
 	/*
 	 * Power wells needed for initialization at driver init and suspend
 	 * time are on. They are kept on until after the first modeset.
 	 */
-	bool init_power_on;
 	bool initializing;
+	bool display_core_suspended;
 	int power_well_count;
 	struct mutex lock;
 	struct mutex gmbus_mutex;
 	/**
-	 * Base address of the gmbus and gpio block.
+	 * Base address of where the gmbus and gpio blocks are located (either
+	 * on PCH or on SoC for platforms without PCH).
 	 */
 	uint32_t gpio_mmio_base;
 	struct intel_engine_cs *engine_class[MAX_ENGINE_CLASS + 1]
 					    [MAX_ENGINE_INSTANCE + 1];
-	struct drm_dma_handle *status_page_dmah;
 	struct resource mch_res;
 	/* protects the irq masks */
 	struct mutex av_mutex;
 	struct {
+		struct mutex mutex;
 		struct list_head list;
 		struct llist_head free_list;
 		struct work_struct free_work;
 #define MAX_CONTEXT_HW_ID (1<<21) /* exclusive */
 #define MAX_GUC_CONTEXT_HW_ID (1 << 20) /* exclusive */
 #define GEN11_MAX_CONTEXT_HW_ID (1<<11) /* exclusive */
+		struct list_head hw_id_list;
 	} contexts;
 	u32 fdi_rx_config;
 #define USES_GUC_SUBMISSION(dev_priv)	intel_uc_is_using_guc_submission()
 #define USES_HUC(dev_priv)		intel_uc_is_using_huc()
-#define HAS_RESOURCE_STREAMER(dev_priv) ((dev_priv)->info.has_resource_streamer)
 #define HAS_POOLED_EU(dev_priv)	((dev_priv)->info.has_pooled_eu)
 #define INTEL_PCH_DEVICE_ID_MASK		0xff80
 int intel_irq_install(struct drm_i915_private *dev_priv);
 void intel_irq_uninstall(struct drm_i915_private *dev_priv);
+void i915_clear_error_registers(struct drm_i915_private *dev_priv);
 static inline bool intel_gvt_active(struct drm_i915_private *dev_priv)
 {
 	return dev_priv->gvt;

drivers/gpu/drm/i915/i915_gem.c+52−13

@@ -802,6 +802,11 @@ void i915_gem_flush_ggtt_writes(struct drm_i915_private *dev_priv) · 801 unmodified lines
802	* that was!).	802	* that was!).
803	*/	803	*/
804		804
		805	wmb();
		806
		807	if (INTEL_INFO(dev_priv)->has_coherent_ggtt)
		808	return;
		809
805	i915_gem_chipset_flush(dev_priv);	810	i915_gem_chipset_flush(dev_priv);
806		811
807	intel_runtime_pm_get(dev_priv);	812	intel_runtime_pm_get(dev_priv);
@@ -1906,7 +1911,7 @@ i915_gem_mmap_ioctl(struct drm_device dev, void data, · 1098 unmodified lines
1906	return 0;	1911	return 0;
1907	}	1912	}
1908		1913
1909	static unsigned int tile_row_pages(struct drm_i915_gem_object *obj)	1914	static unsigned int tile_row_pages(const struct drm_i915_gem_object *obj)
1910	{	1915	{
1911	return i915_gem_object_get_tile_row_size(obj) >> PAGE_SHIFT;	1916	return i915_gem_object_get_tile_row_size(obj) >> PAGE_SHIFT;
1912	}	1917	}
@@ -1965,7 +1970,7 @@ int i915_gem_mmap_gtt_version(void) · 52 unmodified lines
1965	}	1970	}
1966		1971
1967	static inline struct i915_ggtt_view	1972	static inline struct i915_ggtt_view
1968	compute_partial_view(struct drm_i915_gem_object *obj,	1973	compute_partial_view(const struct drm_i915_gem_object *obj,
1969	pgoff_t page_offset,	1974	pgoff_t page_offset,
1970	unsigned int chunk)	1975	unsigned int chunk)
1971	{	1976	{
@@ -2013,7 +2018,7 @@ vm_fault_t i915_gem_fault(struct vm_fault *vmf) · 41 unmodified lines
2013	struct drm_device *dev = obj->base.dev;	2018	struct drm_device *dev = obj->base.dev;
2014	struct drm_i915_private *dev_priv = to_i915(dev);	2019	struct drm_i915_private *dev_priv = to_i915(dev);
2015	struct i915_ggtt *ggtt = &dev_priv->ggtt;	2020	struct i915_ggtt *ggtt = &dev_priv->ggtt;
2016	bool write = !!(vmf->flags & FAULT_FLAG_WRITE);	2021	bool write = area->vm_flags & VM_WRITE;
2017	struct i915_vma *vma;	2022	struct i915_vma *vma;
2018	pgoff_t page_offset;	2023	pgoff_t page_offset;
2019	int ret;	2024	int ret;
@@ -2528,13 +2533,21 @@ static int i915_gem_object_get_pages_gtt(struct drm_i915_gem_object *obj) · 508 unmodified lines
2528	gfp_t noreclaim;	2533	gfp_t noreclaim;
2529	int ret;	2534	int ret;
2530		2535
2531	/* Assert that the object is not currently in any GPU domain. As it	2536	/*
		2537	* Assert that the object is not currently in any GPU domain. As it
2532	* wasn't in the GTT, there shouldn't be any way it could have been in	2538	* wasn't in the GTT, there shouldn't be any way it could have been in
2533	* a GPU cache	2539	* a GPU cache
2534	*/	2540	*/
2535	GEM_BUG_ON(obj->read_domains & I915_GEM_GPU_DOMAINS);	2541	GEM_BUG_ON(obj->read_domains & I915_GEM_GPU_DOMAINS);
2536	GEM_BUG_ON(obj->write_domain & I915_GEM_GPU_DOMAINS);	2542	GEM_BUG_ON(obj->write_domain & I915_GEM_GPU_DOMAINS);
2537		2543
		2544	/*
		2545	* If there's no chance of allocating enough pages for the whole
		2546	* object, bail early.
		2547	*/
		2548	if (page_count > totalram_pages)
		2549	return -ENOMEM;
		2550
2538	st = kmalloc(sizeof(*st), GFP_KERNEL);	2551	st = kmalloc(sizeof(*st), GFP_KERNEL);
2539	if (st == NULL)	2552	if (st == NULL)
2540	return -ENOMEM;	2553	return -ENOMEM;
@@ -2545,7 +2558,8 @@ rebuild_st: · 4 unmodified lines
2545	return -ENOMEM;	2558	return -ENOMEM;
2546	}	2559	}
2547		2560
2548	/* Get the list of pages out of our struct file. They'll be pinned	2561	/*
		2562	* Get the list of pages out of our struct file. They'll be pinned
2549	* at this point until we release them.	2563	* at this point until we release them.
2550	*	2564	*
2551	* Fail silently without starting the shrinker	2565	* Fail silently without starting the shrinker
@@ -2577,7 +2591,8 @@ rebuild_st: · 25 unmodified lines
2577	i915_gem_shrink(dev_priv, 2 * page_count, NULL, *s++);	2591	i915_gem_shrink(dev_priv, 2 * page_count, NULL, *s++);
2578	cond_resched();	2592	cond_resched();
2579		2593
2580	/* We've tried hard to allocate the memory by reaping	2594	/*
		2595	* We've tried hard to allocate the memory by reaping
2581	* our own buffer, now let the real VM do its job and	2596	* our own buffer, now let the real VM do its job and
2582	* go down in flames if truly OOM.	2597	* go down in flames if truly OOM.
2583	*	2598	*
@@ -2589,7 +2604,8 @@ rebuild_st: · 5 unmodified lines
2589	/* reclaim and warn, but no oom */	2604	/* reclaim and warn, but no oom */
2590	gfp = mapping_gfp_mask(mapping);	2605	gfp = mapping_gfp_mask(mapping);
2591		2606
2592	/* Our bo are always dirty and so we require	2607	/*
		2608	* Our bo are always dirty and so we require
2593	* kswapd to reclaim our pages (direct reclaim	2609	* kswapd to reclaim our pages (direct reclaim
2594	* does not effectively begin pageout of our	2610	* does not effectively begin pageout of our
2595	* buffers on its own). However, direct reclaim	2611	* buffers on its own). However, direct reclaim
@@ -2633,7 +2649,8 @@ rebuild_st: · 37 unmodified lines
2633		2649
2634	ret = i915_gem_gtt_prepare_pages(obj, st);	2650	ret = i915_gem_gtt_prepare_pages(obj, st);
2635	if (ret) {	2651	if (ret) {
2636	/* DMA remapping failed? One possible cause is that	2652	/*
		2653	* DMA remapping failed? One possible cause is that
2637	* it could not reserve enough large entries, asking	2654	* it could not reserve enough large entries, asking
2638	* for PAGE_SIZE chunks instead may be helpful.	2655	* for PAGE_SIZE chunks instead may be helpful.
2639	*/	2656	*/
@@ -2667,7 +2684,8 @@ err_pages: · 27 unmodified lines
2667	sg_free_table(st);	2684	sg_free_table(st);
2668	kfree(st);	2685	kfree(st);
2669		2686
2670	/* shmemfs first checks if there is enough memory to allocate the page	2687	/*
		2688	* shmemfs first checks if there is enough memory to allocate the page
2671	* and reports ENOSPC should there be insufficient, along with the usual	2689	* and reports ENOSPC should there be insufficient, along with the usual
2672	* ENOMEM for a genuine allocation failure.	2690	* ENOMEM for a genuine allocation failure.
2673	*	2691	*
@@ -3307,8 +3325,8 @@ void i915_gem_set_wedged(struct drm_i915_private *i915) · 633 unmodified lines
3307	intel_engine_dump(engine, &p, "%s\n", engine->name);	3325	intel_engine_dump(engine, &p, "%s\n", engine->name);
3308	}	3326	}
3309		3327
3310	set_bit(I915_WEDGED, &i915->gpu_error.flags);	3328	if (test_and_set_bit(I915_WEDGED, &i915->gpu_error.flags))
3311	smp_mb__after_atomic();	3329	goto out;
3312		3330
3313	/*	3331	/*
3314	* First, stop submission to hw, but do not yet complete requests by	3332	* First, stop submission to hw, but do not yet complete requests by
@@ -3324,7 +3342,8 @@ void i915_gem_set_wedged(struct drm_i915_private *i915) · 9 unmodified lines
3324	i915->caps.scheduler = 0;	3342	i915->caps.scheduler = 0;
3325		3343
3326	/* Even if the GPU reset fails, it should still stop the engines */	3344	/* Even if the GPU reset fails, it should still stop the engines */
3327	intel_gpu_reset(i915, ALL_ENGINES);	3345	if (INTEL_GEN(i915) >= 5)
		3346	intel_gpu_reset(i915, ALL_ENGINES);
3328		3347
3329	/*	3348	/*
3330	* Make sure no one is running the old callback before we proceed with	3349	* Make sure no one is running the old callback before we proceed with
@@ -3367,6 +3386,7 @@ void i915_gem_set_wedged(struct drm_i915_private *i915) · 36 unmodified lines
3367	i915_gem_reset_finish_engine(engine);	3386	i915_gem_reset_finish_engine(engine);
3368	}	3387	}
3369		3388
		3389	out:
3370	GEM_TRACE("end\n");	3390	GEM_TRACE("end\n");
3371		3391
3372	wake_up_all(&i915->gpu_error.reset_queue);	3392	wake_up_all(&i915->gpu_error.reset_queue);
@@ -3816,6 +3836,12 @@ int i915_gem_wait_for_idle(struct drm_i915_private *i915, · 443 unmodified lines
3816	if (timeout < 0)	3836	if (timeout < 0)
3817	return timeout;	3837	return timeout;
3818	}	3838	}
		3839	if (GEM_SHOW_DEBUG() && !timeout) {
		3840	/* Presume that timeout was non-zero to begin with! */
		3841	dev_warn(&i915->drm.pdev->dev,
		3842	"Missed idle-completion interrupt!\n");
		3843	GEM_TRACE_DUMP();
		3844	}
3819		3845
3820	err = wait_for_engines(i915);	3846	err = wait_for_engines(i915);
3821	if (err)	3847	if (err)
@@ -5592,6 +5618,8 @@ err_uc_misc: · 1770 unmodified lines
5592	i915_gem_cleanup_userptr(dev_priv);	5618	i915_gem_cleanup_userptr(dev_priv);
5593		5619
5594	if (ret == -EIO) {	5620	if (ret == -EIO) {
		5621	mutex_lock(&dev_priv->drm.struct_mutex);
		5622
5595	/*	5623	/*
5596	* Allow engine initialisation to fail by marking the GPU as	5624	* Allow engine initialisation to fail by marking the GPU as
5597	* wedged. But we only want to do this where the GPU is angry,	5625	* wedged. But we only want to do this where the GPU is angry,
@@ -5602,7 +5630,14 @@ err_uc_misc: · 4 unmodified lines
5602	"Failed to initialize GPU, declaring it wedged!\n");	5630	"Failed to initialize GPU, declaring it wedged!\n");
5603	i915_gem_set_wedged(dev_priv);	5631	i915_gem_set_wedged(dev_priv);
5604	}	5632	}
5605	ret = 0;	5633
		5634	/* Minimal basic recovery for KMS */
		5635	ret = i915_ggtt_enable_hw(dev_priv);
		5636	i915_gem_restore_gtt_mappings(dev_priv);
		5637	i915_gem_restore_fences(dev_priv);
		5638	intel_init_clock_gating(dev_priv);
		5639
		5640	mutex_unlock(&dev_priv->drm.struct_mutex);
5606	}	5641	}
5607		5642
5608	i915_gem_drain_freed_objects(dev_priv);	5643	i915_gem_drain_freed_objects(dev_priv);
@@ -5612,6 +5647,7 @@ err_uc_misc: · 3 unmodified lines
5612	void i915_gem_fini(struct drm_i915_private *dev_priv)	5647	void i915_gem_fini(struct drm_i915_private *dev_priv)
5613	{	5648	{
5614	i915_gem_suspend_late(dev_priv);	5649	i915_gem_suspend_late(dev_priv);
		5650	intel_disable_gt_powersave(dev_priv);
5615		5651
5616	/* Flush any outstanding unpin_work. */	5652	/* Flush any outstanding unpin_work. */
5617	i915_gem_drain_workqueue(dev_priv);	5653	i915_gem_drain_workqueue(dev_priv);
@@ -5623,6 +5659,8 @@ void i915_gem_fini(struct drm_i915_private *dev_priv) · 5 unmodified lines
5623	i915_gem_contexts_fini(dev_priv);	5659	i915_gem_contexts_fini(dev_priv);
5624	mutex_unlock(&dev_priv->drm.struct_mutex);	5660	mutex_unlock(&dev_priv->drm.struct_mutex);
5625		5661
		5662	intel_cleanup_gt_powersave(dev_priv);
		5663
5626	intel_uc_fini_misc(dev_priv);	5664	intel_uc_fini_misc(dev_priv);
5627	i915_gem_cleanup_userptr(dev_priv);	5665	i915_gem_cleanup_userptr(dev_priv);
5628		5666
@@ -6182,4 +6220,5 @@ err_unlock: · 553 unmodified lines
6182	#include "selftests/huge_pages.c"	6220	#include "selftests/huge_pages.c"
6183	#include "selftests/i915_gem_object.c"	6221	#include "selftests/i915_gem_object.c"
6184	#include "selftests/i915_gem_coherency.c"	6222	#include "selftests/i915_gem_coherency.c"
		6223	#include "selftests/i915_gem.c"
6185	#endif	6224	#endif

drivers/gpu/drm/i915/i915_gem.h+0−6

 		tasklet_unlock_wait(t);
 }
-static inline void __tasklet_enable_sync_once(struct tasklet_struct *t)
-{
-	if (atomic_dec_return(&t->count) == 0)
-		tasklet_kill(t);
-}
 static inline bool __tasklet_is_enabled(const struct tasklet_struct *t)
 {
 	return !atomic_read(&t->count);

drivers/gpu/drm/i915/i915_gem_context.c+161−71

 	rcu_read_unlock();
 }
+static inline int new_hw_id(struct drm_i915_private *i915, gfp_t gfp)
+{
+	unsigned int max;
+	lockdep_assert_held(&i915->contexts.mutex);
+	if (INTEL_GEN(i915) >= 11)
+		max = GEN11_MAX_CONTEXT_HW_ID;
+	else if (USES_GUC_SUBMISSION(i915))
+		/*
+		 * When using GuC in proxy submission, GuC consumes the
+		 * highest bit in the context id to indicate proxy submission.
+		 */
+		max = MAX_GUC_CONTEXT_HW_ID;
+	else
+		max = MAX_CONTEXT_HW_ID;
+	return ida_simple_get(&i915->contexts.hw_ida, 0, max, gfp);
+}
+static int steal_hw_id(struct drm_i915_private *i915)
+{
+	struct i915_gem_context *ctx, *cn;
+	LIST_HEAD(pinned);
+	int id = -ENOSPC;
+	lockdep_assert_held(&i915->contexts.mutex);
+	list_for_each_entry_safe(ctx, cn,
+				 &i915->contexts.hw_id_list, hw_id_link) {
+		if (atomic_read(&ctx->hw_id_pin_count)) {
+			list_move_tail(&ctx->hw_id_link, &pinned);
+			continue;
+		}
+		GEM_BUG_ON(!ctx->hw_id); /* perma-pinned kernel context */
+		list_del_init(&ctx->hw_id_link);
+		id = ctx->hw_id;
+		break;
+	}
+	/*
+	 * Remember how far we got up on the last repossesion scan, so the
+	 * list is kept in a "least recently scanned" order.
+	 */
+	list_splice_tail(&pinned, &i915->contexts.hw_id_list);
+	return id;
+}
+static int assign_hw_id(struct drm_i915_private *i915, unsigned int *out)
+{
+	int ret;
+	lockdep_assert_held(&i915->contexts.mutex);
+	/*
+	 * We prefer to steal/stall ourselves and our users over that of the
+	 * entire system. That may be a little unfair to our users, and
+	 * even hurt high priority clients. The choice is whether to oomkill
+	 * something else, or steal a context id.
+	 */
+	ret = new_hw_id(i915, GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN);
+	if (unlikely(ret < 0)) {
+		ret = steal_hw_id(i915);
+		if (ret < 0) /* once again for the correct errno code */
+			ret = new_hw_id(i915, GFP_KERNEL);
+		if (ret < 0)
+			return ret;
+	}
+	*out = ret;
+	return 0;
+}
+static void release_hw_id(struct i915_gem_context *ctx)
+{
+	struct drm_i915_private *i915 = ctx->i915;
+	if (list_empty(&ctx->hw_id_link))
+		return;
+	mutex_lock(&i915->contexts.mutex);
+	if (!list_empty(&ctx->hw_id_link)) {
+		ida_simple_remove(&i915->contexts.hw_ida, ctx->hw_id);
+		list_del_init(&ctx->hw_id_link);
+	}
+	mutex_unlock(&i915->contexts.mutex);
+}
 static void i915_gem_context_free(struct i915_gem_context *ctx)
 {
 	unsigned int n;
 	lockdep_assert_held(&ctx->i915->drm.struct_mutex);
 	GEM_BUG_ON(!i915_gem_context_is_closed(ctx));
+	release_hw_id(ctx);
 	i915_ppgtt_put(ctx->ppgtt);
 	for (n = 0; n < ARRAY_SIZE(ctx->__engine); n++) {
 	list_del(&ctx->link);
-	ida_simple_remove(&ctx->i915->contexts.hw_ida, ctx->hw_id);
 	kfree_rcu(ctx, rcu);
 }
 {
 	i915_gem_context_set_closed(ctx);
+	/*
+	 * This context will never again be assinged to HW, so we can
+	 * reuse its ID for the next context.
+	 */
+	release_hw_id(ctx);
 	/*
 	 * The LUT uses the VMA as a backpointer to unref the object,
 	 * so we need to clear the LUT before we close all the VMA (inside
 	i915_gem_context_put(ctx);
 }
-static int assign_hw_id(struct drm_i915_private *dev_priv, unsigned *out)
-{
-	int ret;
-	unsigned int max;
-	if (INTEL_GEN(dev_priv) >= 11) {
-		max = GEN11_MAX_CONTEXT_HW_ID;
-	} else {
-		/*
-		 * When using GuC in proxy submission, GuC consumes the
-		 * highest bit in the context id to indicate proxy submission.
-		 */
-		if (USES_GUC_SUBMISSION(dev_priv))
-			max = MAX_GUC_CONTEXT_HW_ID;
-		else
-			max = MAX_CONTEXT_HW_ID;
-	}
-	ret = ida_simple_get(&dev_priv->contexts.hw_ida,
-			     0, max, GFP_KERNEL);
-	if (ret < 0) {
-		/* Contexts are only released when no longer active.
-		 * Flush any pending retires to hopefully release some
-		 * stale contexts and try again.
-		 */
-		i915_retire_requests(dev_priv);
-		ret = ida_simple_get(&dev_priv->contexts.hw_ida,
-				     0, max, GFP_KERNEL);
-		if (ret < 0)
-			return ret;
-	}
-	*out = ret;
-	return 0;
-}
 static u32 default_desc_template(const struct drm_i915_private *i915,
 				 const struct i915_hw_ppgtt *ppgtt)
 {
 	if (ctx == NULL)
 		return ERR_PTR(-ENOMEM);
-	ret = assign_hw_id(dev_priv, &ctx->hw_id);
-	if (ret) {
-		kfree(ctx);
-		return ERR_PTR(ret);
-	}
 	kref_init(&ctx->ref);
 	list_add_tail(&ctx->link, &dev_priv->contexts.list);
 	ctx->i915 = dev_priv;
 	INIT_RADIX_TREE(&ctx->handles_vma, GFP_KERNEL);
 	INIT_LIST_HEAD(&ctx->handles_list);
+	INIT_LIST_HEAD(&ctx->hw_id_link);
 	/* Default context will never have a file_priv */
 	ret = DEFAULT_CONTEXT_HANDLE;
 	ctx->desc_template =
 		default_desc_template(dev_priv, dev_priv->mm.aliasing_ppgtt);
-	/*
-	 * GuC requires the ring to be placed in Non-WOPCM memory. If GuC is not
-	 * present or not in use we still need a small bias as ring wraparound
-	 * at offset 0 sometimes hangs. No idea why.
-	 */
-	if (USES_GUC(dev_priv))
-		ctx->ggtt_offset_bias = dev_priv->guc.ggtt_pin_bias;
-	else
-		ctx->ggtt_offset_bias = I915_GTT_PAGE_SIZE;
 	return ctx;
 err_pid:
 	return ctx;
 }
+static void
+destroy_kernel_context(struct i915_gem_context **ctxp)
+{
+	struct i915_gem_context *ctx;
+	/* Keep the context ref so that we can free it immediately ourselves */
+	ctx = i915_gem_context_get(fetch_and_zero(ctxp));
+	GEM_BUG_ON(!i915_gem_context_is_kernel(ctx));
+	context_close(ctx);
+	i915_gem_context_free(ctx);
+}
 struct i915_gem_context *
 i915_gem_context_create_kernel(struct drm_i915_private *i915, int prio)
 {
 	struct i915_gem_context *ctx;
+	int err;
 	ctx = i915_gem_create_context(i915, NULL);
 	if (IS_ERR(ctx))
 		return ctx;
+	err = i915_gem_context_pin_hw_id(ctx);
+	if (err) {
+		destroy_kernel_context(&ctx);
+		return ERR_PTR(err);
+	}
 	i915_gem_context_clear_bannable(ctx);
 	ctx->sched.priority = prio;
 	ctx->ring_size = PAGE_SIZE;
 	return ctx;
 }
-static void
+static void init_contexts(struct drm_i915_private *i915)
-destroy_kernel_context(struct i915_gem_context **ctxp)
 {
-	struct i915_gem_context *ctx;
+	mutex_init(&i915->contexts.mutex);
+	INIT_LIST_HEAD(&i915->contexts.list);
-	/* Keep the context ref so that we can free it immediately ourselves */
+	/* Using the simple ida interface, the max is limited by sizeof(int) */
-	ctx = i915_gem_context_get(fetch_and_zero(ctxp));
+	BUILD_BUG_ON(MAX_CONTEXT_HW_ID > INT_MAX);
-	GEM_BUG_ON(!i915_gem_context_is_kernel(ctx));
+	BUILD_BUG_ON(GEN11_MAX_CONTEXT_HW_ID > INT_MAX);
+	ida_init(&i915->contexts.hw_ida);
+	INIT_LIST_HEAD(&i915->contexts.hw_id_list);
-	context_close(ctx);
+	INIT_WORK(&i915->contexts.free_work, contexts_free_worker);
-	i915_gem_context_free(ctx);
+	init_llist_head(&i915->contexts.free_list);
 }
 static bool needs_preempt_context(struct drm_i915_private *i915)
 	if (ret)
 		return ret;
-	INIT_LIST_HEAD(&dev_priv->contexts.list);
+	init_contexts(dev_priv);
-	INIT_WORK(&dev_priv->contexts.free_work, contexts_free_worker);
-	init_llist_head(&dev_priv->contexts.free_list);
-	/* Using the simple ida interface, the max is limited by sizeof(int) */
-	BUILD_BUG_ON(MAX_CONTEXT_HW_ID > INT_MAX);
-	BUILD_BUG_ON(GEN11_MAX_CONTEXT_HW_ID > INT_MAX);
-	ida_init(&dev_priv->contexts.hw_ida);
 	/* lowest priority; idle task */
 	ctx = i915_gem_context_create_kernel(dev_priv, I915_PRIORITY_MIN);
 	}
 	/*
 	 * For easy recognisablity, we want the kernel context to be 0 and then
-	 * all user contexts will have non-zero hw_id.
+	 * all user contexts will have non-zero hw_id. Kernel contexts are
+	 * permanently pinned, so that we never suffer a stall and can
+	 * use them from any allocation context (e.g. for evicting other
+	 * contexts and from inside the shrinker).
 	 */
 	GEM_BUG_ON(ctx->hw_id);
+	GEM_BUG_ON(!atomic_read(&ctx->hw_id_pin_count));
 	dev_priv->kernel_context = ctx;
 	/* highest priority; preempting task */
 	destroy_kernel_context(&i915->kernel_context);
 	/* Must free all deferred contexts (via flush_workqueue) first */
+	GEM_BUG_ON(!list_empty(&i915->contexts.hw_id_list));
 	ida_destroy(&i915->contexts.hw_ida);
 }
 	return ret;
 }
+int __i915_gem_context_pin_hw_id(struct i915_gem_context *ctx)
+{
+	struct drm_i915_private *i915 = ctx->i915;
+	int err = 0;
+	mutex_lock(&i915->contexts.mutex);
+	GEM_BUG_ON(i915_gem_context_is_closed(ctx));
+	if (list_empty(&ctx->hw_id_link)) {
+		GEM_BUG_ON(atomic_read(&ctx->hw_id_pin_count));
+		err = assign_hw_id(i915, &ctx->hw_id);
+		if (err)
+			goto out_unlock;
+		list_add_tail(&ctx->hw_id_link, &i915->contexts.hw_id_list);
+	}
+	GEM_BUG_ON(atomic_read(&ctx->hw_id_pin_count) == ~0u);
+	atomic_inc(&ctx->hw_id_pin_count);
+out_unlock:
+	mutex_unlock(&i915->contexts.mutex);
+	return err;
+}
 #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
 #include "selftests/mock_context.c"
 #include "selftests/i915_gem_context.c"

drivers/gpu/drm/i915/i915_gem_context.h+23−3

 	 * functions like fault reporting, PASID, scheduling. The
 	 * &drm_i915_private.context_hw_ida is used to assign a unqiue
 	 * id for the lifetime of the context.
+	 *
+	 * @hw_id_pin_count: - number of times this context had been pinned
+	 * for use (should be, at most, once per engine).
+	 *
+	 * @hw_id_link: - all contexts with an assigned id are tracked
+	 * for possible repossession.
 	 */
 	unsigned int hw_id;
+	atomic_t hw_id_pin_count;
+	struct list_head hw_id_link;
 	/**
 	 * @user_handle: userspace identifier
 	struct i915_sched_attr sched;
-	/** ggtt_offset_bias: placement restriction for context objects */
-	u32 ggtt_offset_bias;
 	/** engine: per-engine logical HW state */
 	struct intel_context {
 		struct i915_gem_context *gem_context;
 	__set_bit(CONTEXT_FORCE_SINGLE_SUBMISSION, &ctx->flags);
 }
+int __i915_gem_context_pin_hw_id(struct i915_gem_context *ctx);
+static inline int i915_gem_context_pin_hw_id(struct i915_gem_context *ctx)
+{
+	if (atomic_inc_not_zero(&ctx->hw_id_pin_count))
+		return 0;
+	return __i915_gem_context_pin_hw_id(ctx);
+}
+static inline void i915_gem_context_unpin_hw_id(struct i915_gem_context *ctx)
+{
+	GEM_BUG_ON(atomic_read(&ctx->hw_id_pin_count) == 0u);
+	atomic_dec(&ctx->hw_id_pin_count);
+}
 static inline bool i915_gem_context_is_default(const struct i915_gem_context *c)
 {
 	return c->user_handle == DEFAULT_CONTEXT_HANDLE;

drivers/gpu/drm/i915/i915_gem_execbuffer.c+21−22

 #define BATCH_OFFSET_BIAS (256*1024)
 #define __I915_EXEC_ILLEGAL_FLAGS \
-	(__I915_EXEC_UNKNOWN_FLAGS | I915_EXEC_CONSTANTS_MASK)
+	(__I915_EXEC_UNKNOWN_FLAGS | \
+	 I915_EXEC_CONSTANTS_MASK  | \
+	 I915_EXEC_RESOURCE_STREAMER)
 /* Catch emission of unexpected errors for CI! */
 #if IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)
 		return -ENOENT;
 	eb->ctx = ctx;
-	eb->vm = ctx->ppgtt ? &ctx->ppgtt->vm : &eb->i915->ggtt.vm;
+	if (ctx->ppgtt) {
+		eb->vm = &ctx->ppgtt->vm;
+		eb->invalid_flags |= EXEC_OBJECT_NEEDS_GTT;
+	} else {
+		eb->vm = &eb->i915->ggtt.vm;
+	}
 	eb->context_flags = 0;
 	if (ctx->flags & CONTEXT_NO_ZEROMAP)
 	u32 *cmd;
 	int err;
+	if (DBG_FORCE_RELOC == FORCE_GPU_RELOC) {
+		obj = vma->obj;
+		if (obj->cache_dirty & ~obj->cache_coherent)
+			i915_gem_clflush_object(obj, 0);
+		obj->write_domain = 0;
+	}
 	GEM_BUG_ON(vma->obj->write_domain & I915_GEM_DOMAIN_CPU);
 	obj = i915_gem_batch_pool_get(&eb->engine->batch_pool, PAGE_SIZE);
 				 * can read from this userspace address.
 				 */
 				offset = gen8_canonical_addr(offset & ~UPDATE);
-				__put_user(offset,
+				if (unlikely(__put_user(offset, &urelocs[r-stack].presumed_offset))) {
-					   &urelocs[r-stack].presumed_offset);
+					remain = -EFAULT;
+					goto out;
+				}
 			}
 		} while (r++, --count);
 		urelocs += ARRAY_SIZE(stack);
 		relocs = kvmalloc_array(size, 1, GFP_KERNEL);
 		if (!relocs) {
-			kvfree(relocs);
 			err = -ENOMEM;
 			goto err;
 		}
 			if (__copy_from_user((char *)relocs + copied,
 					     (char __user *)urelocs + copied,
 					     len)) {
+end_user:
 				kvfree(relocs);
 				err = -EFAULT;
 				goto err;
 			unsafe_put_user(-1,
 					&urelocs[copied].presumed_offset,
 					end_user);
-end_user:
 		user_access_end();
 		eb->exec[i].relocs_ptr = (uintptr_t)relocs;
 	eb.flags = (unsigned int *)(eb.vma + args->buffer_count + 1);
 	eb.invalid_flags = __EXEC_OBJECT_UNKNOWN_FLAGS;
-	if (USES_FULL_PPGTT(eb.i915))
-		eb.invalid_flags |= EXEC_OBJECT_NEEDS_GTT;
 	reloc_cache_init(&eb.reloc_cache, eb.i915);
 	eb.buffer_count = args->buffer_count;
 	if (!eb.engine)
 		return -EINVAL;
-	if (args->flags & I915_EXEC_RESOURCE_STREAMER) {
-		if (!HAS_RESOURCE_STREAMER(eb.i915)) {
-			DRM_DEBUG("RS is only allowed for Haswell, Gen8 and above\n");
-			return -EINVAL;
-		}
-		if (eb.engine->id != RCS) {
-			DRM_DEBUG("RS is not available on %s\n",
-				 eb.engine->name);
-			return -EINVAL;
-		}
-		eb.batch_flags |= I915_DISPATCH_RS;
-	}
 	if (args->flags & I915_EXEC_FENCE_IN) {
 		in_fence = sync_file_get_fence(lower_32_bits(args->rsvd2));
 		if (!in_fence)

drivers/gpu/drm/i915/i915_gem_gtt.c+25−27

 		return 0;
 	}
-	/* Early VLV doesn't have this */
+	if (has_full_48bit_ppgtt)
-	if (IS_VALLEYVIEW(dev_priv) && dev_priv->drm.pdev->revision < 0xb) {
+		return 3;
-		DRM_DEBUG_DRIVER("disabling PPGTT on pre-B3 step VLV\n");
-		return 0;
-	}
-	if (HAS_LOGICAL_RING_CONTEXTS(dev_priv)) {
-		if (has_full_48bit_ppgtt)
-			return 3;
-		if (has_full_ppgtt)
+	if (has_full_ppgtt)
-			return 2;
+		return 2;
-	}
 	return 1;
 }
 {
 	int i;
-	if (!px_page(pd))
-		return;
 	for (i = 0; i < I915_PDES; i++) {
 		if (pd->page_table[i] != vm->scratch_pt)
 			free_pt(vm, pd->page_table[i]);
 	return IS_GEN5(dev_priv) && IS_MOBILE(dev_priv) && intel_vtd_active();
 }
-static void gen6_check_and_clear_faults(struct drm_i915_private *dev_priv)
+static void gen6_check_faults(struct drm_i915_private *dev_priv)
 {
 	struct intel_engine_cs *engine;
 	enum intel_engine_id id;
 					 fault & RING_FAULT_GTTSEL_MASK ? "GGTT" : "PPGTT",
 					 RING_FAULT_SRCID(fault),
 					 RING_FAULT_FAULT_TYPE(fault));
-			I915_WRITE(RING_FAULT_REG(engine),
-				   fault & ~RING_FAULT_VALID);
 		}
 	}
-	POSTING_READ(RING_FAULT_REG(dev_priv->engine[RCS]));
 }
-static void gen8_check_and_clear_faults(struct drm_i915_private *dev_priv)
+static void gen8_check_faults(struct drm_i915_private *dev_priv)
 {
 	u32 fault = I915_READ(GEN8_RING_FAULT_REG);
 				 GEN8_RING_FAULT_ENGINE_ID(fault),
 				 RING_FAULT_SRCID(fault),
 				 RING_FAULT_FAULT_TYPE(fault));
-		I915_WRITE(GEN8_RING_FAULT_REG,
-			   fault & ~RING_FAULT_VALID);
 	}
-	POSTING_READ(GEN8_RING_FAULT_REG);
 }
 void i915_check_and_clear_faults(struct drm_i915_private *dev_priv)
 {
 	/* From GEN8 onwards we only have one 'All Engine Fault Register' */
 	if (INTEL_GEN(dev_priv) >= 8)
-		gen8_check_and_clear_faults(dev_priv);
+		gen8_check_faults(dev_priv);
 	else if (INTEL_GEN(dev_priv) >= 6)
-		gen6_check_and_clear_faults(dev_priv);
+		gen6_check_faults(dev_priv);
 	else
 		return;
+	i915_clear_error_registers(dev_priv);
 }
 void i915_gem_suspend_gtt_mappings(struct drm_i915_private *dev_priv)
 	struct drm_mm_node *entry;
 	int ret;
+	/*
+	 * GuC requires all resources that we're sharing with it to be placed in
+	 * non-WOPCM memory. If GuC is not present or not in use we still need a
+	 * small bias as ring wraparound at offset 0 sometimes hangs. No idea
+	 * why.
+	 */
+	ggtt->pin_bias = max_t(u32, I915_GTT_PAGE_SIZE,
+			       intel_guc_reserved_gtt_size(&dev_priv->guc));
 	ret = intel_vgt_balloon(dev_priv);
 	if (ret)
 		return ret;
 	mutex_lock(&dev_priv->drm.struct_mutex);
 	i915_address_space_init(&ggtt->vm, dev_priv);
+	ggtt->vm.is_ggtt = true;
 	/* Only VLV supports read-only GGTT mappings */
 	ggtt->vm.has_read_only = IS_VALLEYVIEW(dev_priv);
 void i915_ggtt_disable_guc(struct drm_i915_private *i915)
 {
+	/* XXX Temporary pardon for error unload */
+	if (i915->ggtt.invalidate == gen6_ggtt_invalidate)
+		return;
 	/* We should only be called after i915_ggtt_enable_guc() */
 	GEM_BUG_ON(i915->ggtt.invalidate != guc_ggtt_invalidate);

drivers/gpu/drm/i915/i915_gem_gtt.h+10−13

 	} plane[2];
 } __packed;
-static inline void assert_intel_rotation_info_is_packed(void)
-{
-	BUILD_BUG_ON(sizeof(struct intel_rotation_info) != 8*sizeof(unsigned int));
-}
 struct intel_partial_info {
 	u64 offset;
 	unsigned int size;
 } __packed;
-static inline void assert_intel_partial_info_is_packed(void)
-{
-	BUILD_BUG_ON(sizeof(struct intel_partial_info) != sizeof(u64) + sizeof(unsigned int));
-}
 enum i915_ggtt_view_type {
 	I915_GGTT_VIEW_NORMAL = 0,
 	I915_GGTT_VIEW_ROTATED = sizeof(struct intel_rotation_info),
 	I915_GGTT_VIEW_PARTIAL = sizeof(struct intel_partial_info),
 };
-static inline void assert_i915_ggtt_view_type_is_unique(void)
+static inline void assert_i915_gem_gtt_types(void)
 {
+	BUILD_BUG_ON(sizeof(struct intel_rotation_info) != 8*sizeof(unsigned int));
+	BUILD_BUG_ON(sizeof(struct intel_partial_info) != sizeof(u64) + sizeof(unsigned int));
 	/* As we encode the size of each branch inside the union into its type,
 	 * we have to be careful that each branch has a unique size.
 	 */
 };
 #define px_base(px) (&(px)->base)
-#define px_page(px) (px_base(px)->page)
 #define px_dma(px) (px_base(px)->daddr)
 struct i915_page_table {
 	struct pagestash free_pages;
+	/* Global GTT */
+	bool is_ggtt:1;
 	/* Some systems require uncached updates of the page directories */
 	bool pt_kmap_wc:1;
 	I915_SELFTEST_DECLARE(bool scrub_64K);
 };
-#define i915_is_ggtt(V) (!(V)->file)
+#define i915_is_ggtt(vm) ((vm)->is_ggtt)
 static inline bool
 i915_vm_is_48bit(const struct i915_address_space *vm)
 	int mtrr;
+	u32 pin_bias;
 	struct drm_mm_node error_capture;
 };

drivers/gpu/drm/i915/i915_gem_object.h+5−5

@@ -421,19 +421,19 @@ i915_gem_object_is_framebuffer(const struct drm_i915_gem_object *obj) · 420 unmodified lines
421	}	421	}
422		422
423	static inline unsigned int	423	static inline unsigned int
424	i915_gem_object_get_tiling(struct drm_i915_gem_object *obj)	424	i915_gem_object_get_tiling(const struct drm_i915_gem_object *obj)
425	{	425	{
426	return obj->tiling_and_stride & TILING_MASK;	426	return obj->tiling_and_stride & TILING_MASK;
427	}	427	}
428		428
429	static inline bool	429	static inline bool
430	i915_gem_object_is_tiled(struct drm_i915_gem_object *obj)	430	i915_gem_object_is_tiled(const struct drm_i915_gem_object *obj)
431	{	431	{
432	return i915_gem_object_get_tiling(obj) != I915_TILING_NONE;	432	return i915_gem_object_get_tiling(obj) != I915_TILING_NONE;
433	}	433	}
434		434
435	static inline unsigned int	435	static inline unsigned int
436	i915_gem_object_get_stride(struct drm_i915_gem_object *obj)	436	i915_gem_object_get_stride(const struct drm_i915_gem_object *obj)
437	{	437	{
438	return obj->tiling_and_stride & STRIDE_MASK;	438	return obj->tiling_and_stride & STRIDE_MASK;
439	}	439	}
@@ -446,13 +446,13 @@ i915_gem_tile_height(unsigned int tiling) · 6 unmodified lines
446	}	446	}
447		447
448	static inline unsigned int	448	static inline unsigned int
449	i915_gem_object_get_tile_height(struct drm_i915_gem_object *obj)	449	i915_gem_object_get_tile_height(const struct drm_i915_gem_object *obj)
450	{	450	{
451	return i915_gem_tile_height(i915_gem_object_get_tiling(obj));	451	return i915_gem_tile_height(i915_gem_object_get_tiling(obj));
452	}	452	}
453		453
454	static inline unsigned int	454	static inline unsigned int
455	i915_gem_object_get_tile_row_size(struct drm_i915_gem_object *obj)	455	i915_gem_object_get_tile_row_size(const struct drm_i915_gem_object *obj)
456	{	456	{
457	return (i915_gem_object_get_stride(obj) *	457	return (i915_gem_object_get_stride(obj) *
458	i915_gem_object_get_tile_height(obj));	458	i915_gem_object_get_tile_height(obj));

drivers/gpu/drm/i915/i915_irq.c+26−10

 void gen6_reset_rps_interrupts(struct drm_i915_private *dev_priv)
 {
 	spin_lock_irq(&dev_priv->irq_lock);
-	gen6_reset_pm_iir(dev_priv, dev_priv->pm_rps_events);
+	gen6_reset_pm_iir(dev_priv, GEN6_PM_RPS_EVENTS);
 	dev_priv->gt_pm.rps.pm_iir = 0;
 	spin_unlock_irq(&dev_priv->irq_lock);
 }
 	I915_WRITE(GEN6_PMINTRMSK, gen6_sanitize_rps_pm_mask(dev_priv, ~0u));
-	gen6_disable_pm_irq(dev_priv, dev_priv->pm_rps_events);
+	gen6_disable_pm_irq(dev_priv, GEN6_PM_RPS_EVENTS);
 	spin_unlock_irq(&dev_priv->irq_lock);
 	synchronize_irq(dev_priv->drm.irq);
 	if (master_ctl & (GEN8_GT_PM_IRQ | GEN8_GT_GUC_IRQ)) {
 		gt_iir[2] = raw_reg_read(regs, GEN8_GT_IIR(2));
-		if (likely(gt_iir[2] & (i915->pm_rps_events |
+		if (likely(gt_iir[2]))
-					i915->pm_guc_events)))
+			raw_reg_write(regs, GEN8_GT_IIR(2), gt_iir[2]);
-			raw_reg_write(regs, GEN8_GT_IIR(2),
-				      gt_iir[2] & (i915->pm_rps_events |
-						   i915->pm_guc_events));
 	}
 	if (master_ctl & GEN8_GT_VECS_IRQ) {
 		kobject_uevent_env(kobj, KOBJ_CHANGE, reset_done_event);
 }
-static void i915_clear_error_registers(struct drm_i915_private *dev_priv)
+void i915_clear_error_registers(struct drm_i915_private *dev_priv)
 {
 	u32 eir;
 		I915_WRITE(EMR, I915_READ(EMR) | eir);
 		I915_WRITE(IIR, I915_MASTER_ERROR_INTERRUPT);
 	}
+	if (INTEL_GEN(dev_priv) >= 8) {
+		I915_WRITE(GEN8_RING_FAULT_REG,
+			   I915_READ(GEN8_RING_FAULT_REG) & ~RING_FAULT_VALID);
+		POSTING_READ(GEN8_RING_FAULT_REG);
+	} else if (INTEL_GEN(dev_priv) >= 6) {
+		struct intel_engine_cs *engine;
+		enum intel_engine_id id;
+		for_each_engine(engine, dev_priv, id) {
+			I915_WRITE(RING_FAULT_REG(engine),
+				   I915_READ(RING_FAULT_REG(engine)) &
+				   ~RING_FAULT_VALID);
+		}
+		POSTING_READ(RING_FAULT_REG(dev_priv->engine[RCS]));
+	}
 }
 /**
 	 * Try engine reset when available. We fall back to full reset if
 	 * single reset fails.
 	 */
-	if (intel_has_reset_engine(dev_priv)) {
+	if (intel_has_reset_engine(dev_priv) &&
+	    !i915_terminally_wedged(&dev_priv->gpu_error)) {
 		for_each_engine_masked(engine, dev_priv, engine_mask, tmp) {
 			BUILD_BUG_ON(I915_RESET_MODESET >= I915_RESET_ENGINE);
 			if (test_and_set_bit(I915_RESET_ENGINE + engine->id,
 		/* WaGsvRC0ResidencyMethod:vlv */
 		dev_priv->pm_rps_events = GEN6_PM_RP_UP_EI_EXPIRED;
 	else
-		dev_priv->pm_rps_events = GEN6_PM_RPS_EVENTS;
+		dev_priv->pm_rps_events = (GEN6_PM_RP_UP_THRESHOLD |
+					   GEN6_PM_RP_DOWN_THRESHOLD |
+					   GEN6_PM_RP_DOWN_TIMEOUT);
 	rps->pm_intrmsk_mbz = 0;

drivers/gpu/drm/i915/i915_pci.c+11−5

 	.unfenced_needs_alignment = 1, \
 	.ring_mask = RENDER_RING, \
 	.has_snoop = true, \
+	.has_coherent_ggtt = false, \
 	GEN_DEFAULT_PIPEOFFSETS, \
 	GEN_DEFAULT_PAGE_SIZES, \
 	CURSOR_OFFSETS
 	.has_gmch_display = 1, \
 	.ring_mask = RENDER_RING, \
 	.has_snoop = true, \
+	.has_coherent_ggtt = true, \
 	GEN_DEFAULT_PIPEOFFSETS, \
 	GEN_DEFAULT_PAGE_SIZES, \
 	CURSOR_OFFSETS
 static const struct intel_device_info intel_i915g_info = {
 	GEN3_FEATURES,
 	PLATFORM(INTEL_I915G),
+	.has_coherent_ggtt = false,
 	.cursor_needs_physical = 1,
 	.has_overlay = 1, .overlay_needs_physical = 1,
 	.hws_needs_physical = 1,
 	.has_gmch_display = 1, \
 	.ring_mask = RENDER_RING, \
 	.has_snoop = true, \
+	.has_coherent_ggtt = true, \
 	GEN_DEFAULT_PIPEOFFSETS, \
 	GEN_DEFAULT_PAGE_SIZES, \
 	CURSOR_OFFSETS
 	.has_hotplug = 1, \
 	.ring_mask = RENDER_RING | BSD_RING, \
 	.has_snoop = true, \
+	.has_coherent_ggtt = true, \
 	/* ilk does support rc6, but we do not implement [power] contexts */ \
 	.has_rc6 = 0, \
 	GEN_DEFAULT_PIPEOFFSETS, \
 	.has_hotplug = 1, \
 	.has_fbc = 1, \
 	.ring_mask = RENDER_RING | BSD_RING | BLT_RING, \
+	.has_coherent_ggtt = true, \
 	.has_llc = 1, \
 	.has_rc6 = 1, \
 	.has_rc6p = 1, \
 	.has_hotplug = 1, \
 	.has_fbc = 1, \
 	.ring_mask = RENDER_RING | BSD_RING | BLT_RING, \
+	.has_coherent_ggtt = true, \
 	.has_llc = 1, \
 	.has_rc6 = 1, \
 	.has_rc6p = 1, \
 	.has_aliasing_ppgtt = 1,
 	.has_full_ppgtt = 1,
 	.has_snoop = true,
+	.has_coherent_ggtt = false,
 	.ring_mask = RENDER_RING | BSD_RING | BLT_RING,
 	.display_mmio_offset = VLV_DISPLAY_BASE,
 	GEN_DEFAULT_PAGE_SIZES,
 	.has_ddi = 1, \
 	.has_fpga_dbg = 1, \
 	.has_psr = 1, \
-	.has_resource_streamer = 1, \
 	.has_dp_mst = 1, \
 	.has_rc6p = 0 /* RC6p removed-by HSW */, \
 	.has_runtime_pm = 1
 	.ring_mask = RENDER_RING | BSD_RING | BLT_RING | VEBOX_RING,
 	.has_64bit_reloc = 1,
 	.has_runtime_pm = 1,
-	.has_resource_streamer = 1,
 	.has_rc6 = 1,
 	.has_logical_ring_contexts = 1,
 	.has_gmch_display = 1,
 	.has_full_ppgtt = 1,
 	.has_reset_engine = 1,
 	.has_snoop = true,
+	.has_coherent_ggtt = false,
 	.display_mmio_offset = VLV_DISPLAY_BASE,
 	GEN_DEFAULT_PAGE_SIZES,
 	GEN_CHV_PIPEOFFSETS,
 	.has_runtime_pm = 1, \
 	.has_pooled_eu = 0, \
 	.has_csr = 1, \
-	.has_resource_streamer = 1, \
 	.has_rc6 = 1, \
 	.has_dp_mst = 1, \
 	.has_logical_ring_contexts = 1, \
 	.has_full_48bit_ppgtt = 1, \
 	.has_reset_engine = 1, \
 	.has_snoop = true, \
+	.has_coherent_ggtt = false, \
 	.has_ipc = 1, \
 	GEN9_DEFAULT_PAGE_SIZES, \
 	GEN_DEFAULT_PIPEOFFSETS, \
 	GEN9_FEATURES, \
 	GEN(10), \
 	.ddb_size = 1024, \
+	.has_coherent_ggtt = false, \
 	GLK_COLORS
 static const struct intel_device_info intel_cannonlake_info = {
 	GEN10_FEATURES, \
 	GEN(11), \
 	.ddb_size = 2048, \
-	.has_csr = 0, \
 	.has_logical_ring_elsq = 1
 static const struct intel_device_info intel_icelake_11_info = {
 	GEN11_FEATURES,
 	PLATFORM(INTEL_ICELAKE),
 	.is_alpha_support = 1,
-	.has_resource_streamer = 0,
 	.ring_mask = RENDER_RING | BLT_RING | VEBOX_RING | BSD_RING | BSD3_RING,
 };

drivers/gpu/drm/i915/i915_perf.c+24−31

 #include "i915_oa_cflgt3.h"
 #include "i915_oa_cnl.h"
 #include "i915_oa_icl.h"
+#include "intel_lrc_reg.h"
 /* HW requires this to be a power of two, between 128k and 16M, though driver
  * is currently generally designed assuming the largest 16M size is used such
 {
 	mutex_lock(&i915->drm.struct_mutex);
-	i915_gem_object_unpin_map(i915->perf.oa.oa_buffer.vma->obj);
+	i915_vma_unpin_and_release(&i915->perf.oa.oa_buffer.vma,
-	i915_vma_unpin(i915->perf.oa.oa_buffer.vma);
+				   I915_VMA_RELEASE_MAP);
-	i915_gem_object_put(i915->perf.oa.oa_buffer.vma->obj);
-	i915->perf.oa.oa_buffer.vma = NULL;
-	i915->perf.oa.oa_buffer.vaddr = NULL;
 	mutex_unlock(&i915->drm.struct_mutex);
+	i915->perf.oa.oa_buffer.vaddr = NULL;
 }
 static void i915_oa_stream_destroy(struct i915_perf_stream *stream)
 	u32 ctx_oactxctrl = dev_priv->perf.oa.ctx_oactxctrl_offset;
 	u32 ctx_flexeu0 = dev_priv->perf.oa.ctx_flexeu0_offset;
 	/* The MMIO offsets for Flex EU registers aren't contiguous */
-	u32 flex_mmio[] = {
+	i915_reg_t flex_regs[] = {
-		i915_mmio_reg_offset(EU_PERF_CNTL0),
+		EU_PERF_CNTL0,
-		i915_mmio_reg_offset(EU_PERF_CNTL1),
+		EU_PERF_CNTL1,
-		i915_mmio_reg_offset(EU_PERF_CNTL2),
+		EU_PERF_CNTL2,
-		i915_mmio_reg_offset(EU_PERF_CNTL3),
+		EU_PERF_CNTL3,
-		i915_mmio_reg_offset(EU_PERF_CNTL4),
+		EU_PERF_CNTL4,
-		i915_mmio_reg_offset(EU_PERF_CNTL5),
+		EU_PERF_CNTL5,
-		i915_mmio_reg_offset(EU_PERF_CNTL6),
+		EU_PERF_CNTL6,
 	};
 	int i;
-	reg_state[ctx_oactxctrl] = i915_mmio_reg_offset(GEN8_OACTXCONTROL);
+	CTX_REG(reg_state, ctx_oactxctrl, GEN8_OACTXCONTROL,
-	reg_state[ctx_oactxctrl+1] = (dev_priv->perf.oa.period_exponent <<
+		(dev_priv->perf.oa.period_exponent << GEN8_OA_TIMER_PERIOD_SHIFT) |
-				      GEN8_OA_TIMER_PERIOD_SHIFT) |
+		(dev_priv->perf.oa.periodic ? GEN8_OA_TIMER_ENABLE : 0) |
-				     (dev_priv->perf.oa.periodic ?
+		GEN8_OA_COUNTER_RESUME);
-				      GEN8_OA_TIMER_ENABLE : 0) |
-				     GEN8_OA_COUNTER_RESUME;
-	for (i = 0; i < ARRAY_SIZE(flex_mmio); i++) {
+	for (i = 0; i < ARRAY_SIZE(flex_regs); i++) {
 		u32 state_offset = ctx_flexeu0 + i * 2;
-		u32 mmio = flex_mmio[i];
+		u32 mmio = i915_mmio_reg_offset(flex_regs[i]);
 		/*
 		 * This arbitrary default will select the 'EU FPU0 Pipeline
 			}
 		}
-		reg_state[state_offset] = mmio;
+		CTX_REG(reg_state, state_offset, flex_regs[i], value);
-		reg_state[state_offset+1] = value;
 	}
 }
 	/* Switch away from any user context. */
 	ret = gen8_switch_to_updated_kernel_context(dev_priv, oa_config);
 	if (ret)
-		goto out;
+		return ret;
 	/*
 	 * The OA register config is setup through the context image. This image
 				     wait_flags,
 				     MAX_SCHEDULE_TIMEOUT);
 	if (ret)
-		goto out;
+		return ret;
 	/* Update all contexts now that we've stalled the submission. */
 	list_for_each_entry(ctx, &dev_priv->contexts.list, link) {
 			continue;
 		regs = i915_gem_object_pin_map(ce->state->obj, I915_MAP_WB);
-		if (IS_ERR(regs)) {
+		if (IS_ERR(regs))
-			ret = PTR_ERR(regs);
+			return PTR_ERR(regs);
-			goto out;
-		}
 		ce->state->obj->mm.dirty = true;
 		regs += LRC_STATE_PN * PAGE_SIZE / sizeof(*regs);
 		i915_gem_object_unpin_map(ce->state->obj);
 	}
- out:
 	return ret;
 }

drivers/gpu/drm/i915/i915_reg.h+387−340

@@ -344,6 +344,8 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg) · 343 unmodified lines
344	#define GEN8_RPCS_S_CNT_ENABLE (1 << 18)	344	#define GEN8_RPCS_S_CNT_ENABLE (1 << 18)
345	#define GEN8_RPCS_S_CNT_SHIFT 15	345	#define GEN8_RPCS_S_CNT_SHIFT 15
346	#define GEN8_RPCS_S_CNT_MASK (0x7 << GEN8_RPCS_S_CNT_SHIFT)	346	#define GEN8_RPCS_S_CNT_MASK (0x7 << GEN8_RPCS_S_CNT_SHIFT)
		347	#define GEN11_RPCS_S_CNT_SHIFT 12
		348	#define GEN11_RPCS_S_CNT_MASK (0x3f << GEN11_RPCS_S_CNT_SHIFT)
347	#define GEN8_RPCS_SS_CNT_ENABLE (1 << 11)	349	#define GEN8_RPCS_SS_CNT_ENABLE (1 << 11)
348	#define GEN8_RPCS_SS_CNT_SHIFT 8	350	#define GEN8_RPCS_SS_CNT_SHIFT 8
349	#define GEN8_RPCS_SS_CNT_MASK (0x7 << GEN8_RPCS_SS_CNT_SHIFT)	351	#define GEN8_RPCS_SS_CNT_MASK (0x7 << GEN8_RPCS_SS_CNT_SHIFT)
@@ -1029,126 +1031,43 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg) · 679 unmodified lines
1029	/*	1031	/*
1030	* i915_power_well_id:	1032	* i915_power_well_id:
1031	*	1033	*
1032	* Platform specific IDs used to look up power wells and - except for custom	1034	* IDs used to look up power wells. Power wells accessed directly bypassing
1033	* power wells - to define request/status register flag bit positions. As such	1035	* the power domains framework must be assigned a unique ID. The rest of power
1034	* the set of IDs on a given platform must be unique and except for custom	1036	* wells must be assigned DISP_PW_ID_NONE.
1035	* power wells their value must stay fixed.
1036	*/	1037	*/
1037	enum i915_power_well_id {	1038	enum i915_power_well_id {
1038	/*	1039	DISP_PW_ID_NONE,
1039	* I830	1040
1040	* - custom power well	1041	VLV_DISP_PW_DISP2D,
1041	*/	1042	BXT_DISP_PW_DPIO_CMN_A,
1042	I830_DISP_PW_PIPES = 0,	1043	VLV_DISP_PW_DPIO_CMN_BC,
1043		1044	GLK_DISP_PW_DPIO_CMN_C,
1044	/*	1045	CHV_DISP_PW_DPIO_CMN_D,
1045	* VLV/CHV	1046	HSW_DISP_PW_GLOBAL,
1046	* - PUNIT_REG_PWRGT_CTRL (bit: id*2),	1047	SKL_DISP_PW_MISC_IO,
1047	* PUNIT_REG_PWRGT_STATUS (bit: id*2) (PUNIT HAS v0.8)	1048	SKL_DISP_PW_1,
1048	*/
1049	PUNIT_POWER_WELL_RENDER = 0,
1050	PUNIT_POWER_WELL_MEDIA = 1,
1051	PUNIT_POWER_WELL_DISP2D = 3,
1052	PUNIT_POWER_WELL_DPIO_CMN_BC = 5,
1053	PUNIT_POWER_WELL_DPIO_TX_B_LANES_01 = 6,
1054	PUNIT_POWER_WELL_DPIO_TX_B_LANES_23 = 7,
1055	PUNIT_POWER_WELL_DPIO_TX_C_LANES_01 = 8,
1056	PUNIT_POWER_WELL_DPIO_TX_C_LANES_23 = 9,
1057	PUNIT_POWER_WELL_DPIO_RX0 = 10,
1058	PUNIT_POWER_WELL_DPIO_RX1 = 11,
1059	PUNIT_POWER_WELL_DPIO_CMN_D = 12,
1060	/* - custom power well */
1061	CHV_DISP_PW_PIPE_A, /* 13 */
1062
1063	/*
1064	* HSW/BDW
1065	* - _HSW_PWR_WELL_CTL1-4 (status bit: id2, req bit: id2+1)
1066	*/
1067	HSW_DISP_PW_GLOBAL = 15,
1068
1069	/*
1070	* GEN9+
1071	* - _HSW_PWR_WELL_CTL1-4 (status bit: id2, req bit: id2+1)
1072	*/
1073	SKL_DISP_PW_MISC_IO = 0,
1074	SKL_DISP_PW_DDI_A_E,
1075	GLK_DISP_PW_DDI_A = SKL_DISP_PW_DDI_A_E,
1076	CNL_DISP_PW_DDI_A = SKL_DISP_PW_DDI_A_E,
1077	SKL_DISP_PW_DDI_B,
1078	SKL_DISP_PW_DDI_C,
1079	SKL_DISP_PW_DDI_D,
1080	CNL_DISP_PW_DDI_F = 6,
1081
1082	GLK_DISP_PW_AUX_A = 8,
1083	GLK_DISP_PW_AUX_B,
1084	GLK_DISP_PW_AUX_C,
1085	CNL_DISP_PW_AUX_A = GLK_DISP_PW_AUX_A,
1086	CNL_DISP_PW_AUX_B = GLK_DISP_PW_AUX_B,
1087	CNL_DISP_PW_AUX_C = GLK_DISP_PW_AUX_C,
1088	CNL_DISP_PW_AUX_D,
1089	CNL_DISP_PW_AUX_F,
1090
1091	SKL_DISP_PW_1 = 14,
1092	SKL_DISP_PW_2,	1049	SKL_DISP_PW_2,
1093
1094	/* - custom power wells */
1095	BXT_DPIO_CMN_A,
1096	BXT_DPIO_CMN_BC,
1097	GLK_DPIO_CMN_C, /* 18 */
1098
1099	/*
1100	* GEN11+
1101	* - _HSW_PWR_WELL_CTL1-4
1102	* (status bit: (id&15)2, req bit:(id&15)2+1)
1103	*/
1104	ICL_DISP_PW_1 = 0,
1105	ICL_DISP_PW_2,
1106	ICL_DISP_PW_3,
1107	ICL_DISP_PW_4,
1108
1109	/*
1110	* - _HSW_PWR_WELL_CTL_AUX1/2/4
1111	* (status bit: (id&15)2, req bit:(id&15)2+1)
1112	*/
1113	ICL_DISP_PW_AUX_A = 16,
1114	ICL_DISP_PW_AUX_B,
1115	ICL_DISP_PW_AUX_C,
1116	ICL_DISP_PW_AUX_D,
1117	ICL_DISP_PW_AUX_E,
1118	ICL_DISP_PW_AUX_F,
1119
1120	ICL_DISP_PW_AUX_TBT1 = 24,
1121	ICL_DISP_PW_AUX_TBT2,
1122	ICL_DISP_PW_AUX_TBT3,
1123	ICL_DISP_PW_AUX_TBT4,
1124
1125	/*
1126	* - _HSW_PWR_WELL_CTL_DDI1/2/4
1127	* (status bit: (id&15)2, req bit:(id&15)2+1)
1128	*/
1129	ICL_DISP_PW_DDI_A = 32,
1130	ICL_DISP_PW_DDI_B,
1131	ICL_DISP_PW_DDI_C,
1132	ICL_DISP_PW_DDI_D,
1133	ICL_DISP_PW_DDI_E,
1134	ICL_DISP_PW_DDI_F, /* 37 */
1135
1136	/*
1137	* Multiple platforms.
1138	* Must start following the highest ID of any platform.
1139	* - custom power wells
1140	*/
1141	SKL_DISP_PW_DC_OFF = 38,
1142	I915_DISP_PW_ALWAYS_ON,
1143	};	1050	};
1144		1051
1145	#define PUNIT_REG_PWRGT_CTRL 0x60	1052	#define PUNIT_REG_PWRGT_CTRL 0x60
1146	#define PUNIT_REG_PWRGT_STATUS 0x61	1053	#define PUNIT_REG_PWRGT_STATUS 0x61
1147	#define PUNIT_PWRGT_MASK(power_well) (3 << ((power_well) * 2))	1054	#define PUNIT_PWRGT_MASK(pw_idx) (3 << ((pw_idx) * 2))
1148	#define PUNIT_PWRGT_PWR_ON(power_well) (0 << ((power_well) * 2))	1055	#define PUNIT_PWRGT_PWR_ON(pw_idx) (0 << ((pw_idx) * 2))
1149	#define PUNIT_PWRGT_CLK_GATE(power_well) (1 << ((power_well) * 2))	1056	#define PUNIT_PWRGT_CLK_GATE(pw_idx) (1 << ((pw_idx) * 2))
1150	#define PUNIT_PWRGT_RESET(power_well) (2 << ((power_well) * 2))	1057	#define PUNIT_PWRGT_RESET(pw_idx) (2 << ((pw_idx) * 2))
1151	#define PUNIT_PWRGT_PWR_GATE(power_well) (3 << ((power_well) * 2))	1058	#define PUNIT_PWRGT_PWR_GATE(pw_idx) (3 << ((pw_idx) * 2))
		1059
		1060	#define PUNIT_PWGT_IDX_RENDER 0
		1061	#define PUNIT_PWGT_IDX_MEDIA 1
		1062	#define PUNIT_PWGT_IDX_DISP2D 3
		1063	#define PUNIT_PWGT_IDX_DPIO_CMN_BC 5
		1064	#define PUNIT_PWGT_IDX_DPIO_TX_B_LANES_01 6
		1065	#define PUNIT_PWGT_IDX_DPIO_TX_B_LANES_23 7
		1066	#define PUNIT_PWGT_IDX_DPIO_TX_C_LANES_01 8
		1067	#define PUNIT_PWGT_IDX_DPIO_TX_C_LANES_23 9
		1068	#define PUNIT_PWGT_IDX_DPIO_RX0 10
		1069	#define PUNIT_PWGT_IDX_DPIO_RX1 11
		1070	#define PUNIT_PWGT_IDX_DPIO_CMN_D 12
1152		1071
1153	#define PUNIT_REG_GPU_LFM 0xd3	1072	#define PUNIT_REG_GPU_LFM 0xd3
1154	#define PUNIT_REG_GPU_FREQ_REQ 0xd4	1073	#define PUNIT_REG_GPU_FREQ_REQ 0xd4
@@ -1932,121 +1851,200 @@ enum i915_power_well_id { · 777 unmodified lines
1932	#define N_SCALAR(x) ((x) << 24)	1851	#define N_SCALAR(x) ((x) << 24)
1933	#define N_SCALAR_MASK (0x7F << 24)	1852	#define N_SCALAR_MASK (0x7F << 24)
1934		1853
1935	#define _ICL_MG_PHY_PORT_LN(port, ln, ln0p1, ln0p2, ln1p1) \	1854	#define MG_PHY_PORT_LN(port, ln, ln0p1, ln0p2, ln1p1) \
1936	_MMIO(_PORT((port) - PORT_C, ln0p1, ln0p2) + (ln) * ((ln1p1) - (ln0p1)))	1855	_MMIO(_PORT((port) - PORT_C, ln0p1, ln0p2) + (ln) * ((ln1p1) - (ln0p1)))
1937		1856
1938	#define _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT1 0x16812C	1857	#define MG_TX_LINK_PARAMS_TX1LN0_PORT1 0x16812C
1939	#define _ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT1 0x16852C	1858	#define MG_TX_LINK_PARAMS_TX1LN1_PORT1 0x16852C
1940	#define _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT2 0x16912C	1859	#define MG_TX_LINK_PARAMS_TX1LN0_PORT2 0x16912C
1941	#define _ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT2 0x16952C	1860	#define MG_TX_LINK_PARAMS_TX1LN1_PORT2 0x16952C
1942	#define _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT3 0x16A12C	1861	#define MG_TX_LINK_PARAMS_TX1LN0_PORT3 0x16A12C
1943	#define _ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT3 0x16A52C	1862	#define MG_TX_LINK_PARAMS_TX1LN1_PORT3 0x16A52C
1944	#define _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT4 0x16B12C	1863	#define MG_TX_LINK_PARAMS_TX1LN0_PORT4 0x16B12C
1945	#define _ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT4 0x16B52C	1864	#define MG_TX_LINK_PARAMS_TX1LN1_PORT4 0x16B52C
1946	#define ICL_PORT_MG_TX1_LINK_PARAMS(port, ln) \	1865	#define MG_TX1_LINK_PARAMS(port, ln) \
1947	_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT1, \	1866	MG_PHY_PORT_LN(port, ln, MG_TX_LINK_PARAMS_TX1LN0_PORT1, \
1948	_ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT2, \	1867	MG_TX_LINK_PARAMS_TX1LN0_PORT2, \
1949	_ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT1)	1868	MG_TX_LINK_PARAMS_TX1LN1_PORT1)
1950		1869
1951	#define _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT1 0x1680AC	1870	#define MG_TX_LINK_PARAMS_TX2LN0_PORT1 0x1680AC
1952	#define _ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT1 0x1684AC	1871	#define MG_TX_LINK_PARAMS_TX2LN1_PORT1 0x1684AC
1953	#define _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT2 0x1690AC	1872	#define MG_TX_LINK_PARAMS_TX2LN0_PORT2 0x1690AC
1954	#define _ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT2 0x1694AC	1873	#define MG_TX_LINK_PARAMS_TX2LN1_PORT2 0x1694AC
1955	#define _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT3 0x16A0AC	1874	#define MG_TX_LINK_PARAMS_TX2LN0_PORT3 0x16A0AC
1956	#define _ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT3 0x16A4AC	1875	#define MG_TX_LINK_PARAMS_TX2LN1_PORT3 0x16A4AC
1957	#define _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT4 0x16B0AC	1876	#define MG_TX_LINK_PARAMS_TX2LN0_PORT4 0x16B0AC
1958	#define _ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT4 0x16B4AC	1877	#define MG_TX_LINK_PARAMS_TX2LN1_PORT4 0x16B4AC
1959	#define ICL_PORT_MG_TX2_LINK_PARAMS(port, ln) \	1878	#define MG_TX2_LINK_PARAMS(port, ln) \
1960	_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT1, \	1879	MG_PHY_PORT_LN(port, ln, MG_TX_LINK_PARAMS_TX2LN0_PORT1, \
1961	_ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT2, \	1880	MG_TX_LINK_PARAMS_TX2LN0_PORT2, \
1962	_ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT1)	1881	MG_TX_LINK_PARAMS_TX2LN1_PORT1)
1963	#define CRI_USE_FS32 (1 << 5)	1882	#define CRI_USE_FS32 (1 << 5)
1964		1883
1965	#define _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT1 0x16814C	1884	#define MG_TX_PISO_READLOAD_TX1LN0_PORT1 0x16814C
1966	#define _ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT1 0x16854C	1885	#define MG_TX_PISO_READLOAD_TX1LN1_PORT1 0x16854C
1967	#define _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT2 0x16914C	1886	#define MG_TX_PISO_READLOAD_TX1LN0_PORT2 0x16914C
1968	#define _ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT2 0x16954C	1887	#define MG_TX_PISO_READLOAD_TX1LN1_PORT2 0x16954C
1969	#define _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT3 0x16A14C	1888	#define MG_TX_PISO_READLOAD_TX1LN0_PORT3 0x16A14C
1970	#define _ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT3 0x16A54C	1889	#define MG_TX_PISO_READLOAD_TX1LN1_PORT3 0x16A54C
1971	#define _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT4 0x16B14C	1890	#define MG_TX_PISO_READLOAD_TX1LN0_PORT4 0x16B14C
1972	#define _ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT4 0x16B54C	1891	#define MG_TX_PISO_READLOAD_TX1LN1_PORT4 0x16B54C
1973	#define ICL_PORT_MG_TX1_PISO_READLOAD(port, ln) \	1892	#define MG_TX1_PISO_READLOAD(port, ln) \
1974	_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT1, \	1893	MG_PHY_PORT_LN(port, ln, MG_TX_PISO_READLOAD_TX1LN0_PORT1, \
1975	_ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT2, \	1894	MG_TX_PISO_READLOAD_TX1LN0_PORT2, \
1976	_ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT1)	1895	MG_TX_PISO_READLOAD_TX1LN1_PORT1)
1977		1896
1978	#define _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT1 0x1680CC	1897	#define MG_TX_PISO_READLOAD_TX2LN0_PORT1 0x1680CC
1979	#define _ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT1 0x1684CC	1898	#define MG_TX_PISO_READLOAD_TX2LN1_PORT1 0x1684CC
1980	#define _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT2 0x1690CC	1899	#define MG_TX_PISO_READLOAD_TX2LN0_PORT2 0x1690CC
1981	#define _ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT2 0x1694CC	1900	#define MG_TX_PISO_READLOAD_TX2LN1_PORT2 0x1694CC
1982	#define _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT3 0x16A0CC	1901	#define MG_TX_PISO_READLOAD_TX2LN0_PORT3 0x16A0CC
1983	#define _ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT3 0x16A4CC	1902	#define MG_TX_PISO_READLOAD_TX2LN1_PORT3 0x16A4CC
1984	#define _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT4 0x16B0CC	1903	#define MG_TX_PISO_READLOAD_TX2LN0_PORT4 0x16B0CC
1985	#define _ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT4 0x16B4CC	1904	#define MG_TX_PISO_READLOAD_TX2LN1_PORT4 0x16B4CC
1986	#define ICL_PORT_MG_TX2_PISO_READLOAD(port, ln) \	1905	#define MG_TX2_PISO_READLOAD(port, ln) \
1987	_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT1, \	1906	MG_PHY_PORT_LN(port, ln, MG_TX_PISO_READLOAD_TX2LN0_PORT1, \
1988	_ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT2, \	1907	MG_TX_PISO_READLOAD_TX2LN0_PORT2, \
1989	_ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT1)	1908	MG_TX_PISO_READLOAD_TX2LN1_PORT1)
1990	#define CRI_CALCINIT (1 << 1)	1909	#define CRI_CALCINIT (1 << 1)
1991		1910
1992	#define _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT1 0x168148	1911	#define MG_TX_SWINGCTRL_TX1LN0_PORT1 0x168148
1993	#define _ICL_MG_TX_SWINGCTRL_TX1LN1_PORT1 0x168548	1912	#define MG_TX_SWINGCTRL_TX1LN1_PORT1 0x168548
1994	#define _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT2 0x169148	1913	#define MG_TX_SWINGCTRL_TX1LN0_PORT2 0x169148
1995	#define _ICL_MG_TX_SWINGCTRL_TX1LN1_PORT2 0x169548	1914	#define MG_TX_SWINGCTRL_TX1LN1_PORT2 0x169548
1996	#define _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT3 0x16A148	1915	#define MG_TX_SWINGCTRL_TX1LN0_PORT3 0x16A148
1997	#define _ICL_MG_TX_SWINGCTRL_TX1LN1_PORT3 0x16A548	1916	#define MG_TX_SWINGCTRL_TX1LN1_PORT3 0x16A548
1998	#define _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT4 0x16B148	1917	#define MG_TX_SWINGCTRL_TX1LN0_PORT4 0x16B148
1999	#define _ICL_MG_TX_SWINGCTRL_TX1LN1_PORT4 0x16B548	1918	#define MG_TX_SWINGCTRL_TX1LN1_PORT4 0x16B548
2000	#define ICL_PORT_MG_TX1_SWINGCTRL(port, ln) \	1919	#define MG_TX1_SWINGCTRL(port, ln) \
2001	_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT1, \	1920	MG_PHY_PORT_LN(port, ln, MG_TX_SWINGCTRL_TX1LN0_PORT1, \
2002	_ICL_MG_TX_SWINGCTRL_TX1LN0_PORT2, \	1921	MG_TX_SWINGCTRL_TX1LN0_PORT2, \
2003	_ICL_MG_TX_SWINGCTRL_TX1LN1_PORT1)	1922	MG_TX_SWINGCTRL_TX1LN1_PORT1)
2004		1923
2005	#define _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT1 0x1680C8	1924	#define MG_TX_SWINGCTRL_TX2LN0_PORT1 0x1680C8
2006	#define _ICL_MG_TX_SWINGCTRL_TX2LN1_PORT1 0x1684C8	1925	#define MG_TX_SWINGCTRL_TX2LN1_PORT1 0x1684C8
2007	#define _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT2 0x1690C8	1926	#define MG_TX_SWINGCTRL_TX2LN0_PORT2 0x1690C8
2008	#define _ICL_MG_TX_SWINGCTRL_TX2LN1_PORT2 0x1694C8	1927	#define MG_TX_SWINGCTRL_TX2LN1_PORT2 0x1694C8
2009	#define _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT3 0x16A0C8	1928	#define MG_TX_SWINGCTRL_TX2LN0_PORT3 0x16A0C8
2010	#define _ICL_MG_TX_SWINGCTRL_TX2LN1_PORT3 0x16A4C8	1929	#define MG_TX_SWINGCTRL_TX2LN1_PORT3 0x16A4C8
2011	#define _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT4 0x16B0C8	1930	#define MG_TX_SWINGCTRL_TX2LN0_PORT4 0x16B0C8
2012	#define _ICL_MG_TX_SWINGCTRL_TX2LN1_PORT4 0x16B4C8	1931	#define MG_TX_SWINGCTRL_TX2LN1_PORT4 0x16B4C8
2013	#define ICL_PORT_MG_TX2_SWINGCTRL(port, ln) \	1932	#define MG_TX2_SWINGCTRL(port, ln) \
2014	_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT1, \	1933	MG_PHY_PORT_LN(port, ln, MG_TX_SWINGCTRL_TX2LN0_PORT1, \
2015	_ICL_MG_TX_SWINGCTRL_TX2LN0_PORT2, \	1934	MG_TX_SWINGCTRL_TX2LN0_PORT2, \
2016	_ICL_MG_TX_SWINGCTRL_TX2LN1_PORT1)	1935	MG_TX_SWINGCTRL_TX2LN1_PORT1)
2017	#define CRI_TXDEEMPH_OVERRIDE_17_12(x) ((x) << 0)	1936	#define CRI_TXDEEMPH_OVERRIDE_17_12(x) ((x) << 0)
2018	#define CRI_TXDEEMPH_OVERRIDE_17_12_MASK (0x3F << 0)	1937	#define CRI_TXDEEMPH_OVERRIDE_17_12_MASK (0x3F << 0)
2019		1938
2020	#define _ICL_MG_TX_DRVCTRL_TX1LN0_PORT1 0x168144	1939	#define MG_TX_DRVCTRL_TX1LN0_TXPORT1 0x168144
2021	#define _ICL_MG_TX_DRVCTRL_TX1LN1_PORT1 0x168544	1940	#define MG_TX_DRVCTRL_TX1LN1_TXPORT1 0x168544
2022	#define _ICL_MG_TX_DRVCTRL_TX1LN0_PORT2 0x169144	1941	#define MG_TX_DRVCTRL_TX1LN0_TXPORT2 0x169144
2023	#define _ICL_MG_TX_DRVCTRL_TX1LN1_PORT2 0x169544	1942	#define MG_TX_DRVCTRL_TX1LN1_TXPORT2 0x169544
2024	#define _ICL_MG_TX_DRVCTRL_TX1LN0_PORT3 0x16A144	1943	#define MG_TX_DRVCTRL_TX1LN0_TXPORT3 0x16A144
2025	#define _ICL_MG_TX_DRVCTRL_TX1LN1_PORT3 0x16A544	1944	#define MG_TX_DRVCTRL_TX1LN1_TXPORT3 0x16A544
2026	#define _ICL_MG_TX_DRVCTRL_TX1LN0_PORT4 0x16B144	1945	#define MG_TX_DRVCTRL_TX1LN0_TXPORT4 0x16B144
2027	#define _ICL_MG_TX_DRVCTRL_TX1LN1_PORT4 0x16B544	1946	#define MG_TX_DRVCTRL_TX1LN1_TXPORT4 0x16B544
2028	#define ICL_PORT_MG_TX1_DRVCTRL(port, ln) \	1947	#define MG_TX1_DRVCTRL(port, ln) \
2029	_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_DRVCTRL_TX1LN0_PORT1, \	1948	MG_PHY_PORT_LN(port, ln, MG_TX_DRVCTRL_TX1LN0_TXPORT1, \
2030	_ICL_MG_TX_DRVCTRL_TX1LN0_PORT2, \	1949	MG_TX_DRVCTRL_TX1LN0_TXPORT2, \
2031	_ICL_MG_TX_DRVCTRL_TX1LN1_PORT1)	1950	MG_TX_DRVCTRL_TX1LN1_TXPORT1)
2032		1951
2033	#define _ICL_MG_TX_DRVCTRL_TX2LN0_PORT1 0x1680C4	1952	#define MG_TX_DRVCTRL_TX2LN0_PORT1 0x1680C4
2034	#define _ICL_MG_TX_DRVCTRL_TX2LN1_PORT1 0x1684C4	1953	#define MG_TX_DRVCTRL_TX2LN1_PORT1 0x1684C4
2035	#define _ICL_MG_TX_DRVCTRL_TX2LN0_PORT2 0x1690C4	1954	#define MG_TX_DRVCTRL_TX2LN0_PORT2 0x1690C4
2036	#define _ICL_MG_TX_DRVCTRL_TX2LN1_PORT2 0x1694C4	1955	#define MG_TX_DRVCTRL_TX2LN1_PORT2 0x1694C4
2037	#define _ICL_MG_TX_DRVCTRL_TX2LN0_PORT3 0x16A0C4	1956	#define MG_TX_DRVCTRL_TX2LN0_PORT3 0x16A0C4
2038	#define _ICL_MG_TX_DRVCTRL_TX2LN1_PORT3 0x16A4C4	1957	#define MG_TX_DRVCTRL_TX2LN1_PORT3 0x16A4C4
2039	#define _ICL_MG_TX_DRVCTRL_TX2LN0_PORT4 0x16B0C4	1958	#define MG_TX_DRVCTRL_TX2LN0_PORT4 0x16B0C4
2040	#define _ICL_MG_TX_DRVCTRL_TX2LN1_PORT4 0x16B4C4	1959	#define MG_TX_DRVCTRL_TX2LN1_PORT4 0x16B4C4
2041	#define ICL_PORT_MG_TX2_DRVCTRL(port, ln) \	1960	#define MG_TX2_DRVCTRL(port, ln) \
2042	_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_DRVCTRL_TX2LN0_PORT1, \	1961	MG_PHY_PORT_LN(port, ln, MG_TX_DRVCTRL_TX2LN0_PORT1, \
2043	_ICL_MG_TX_DRVCTRL_TX2LN0_PORT2, \	1962	MG_TX_DRVCTRL_TX2LN0_PORT2, \
2044	_ICL_MG_TX_DRVCTRL_TX2LN1_PORT1)	1963	MG_TX_DRVCTRL_TX2LN1_PORT1)
2045	#define CRI_TXDEEMPH_OVERRIDE_11_6(x) ((x) << 24)	1964	#define CRI_TXDEEMPH_OVERRIDE_11_6(x) ((x) << 24)
2046	#define CRI_TXDEEMPH_OVERRIDE_11_6_MASK (0x3F << 24)	1965	#define CRI_TXDEEMPH_OVERRIDE_11_6_MASK (0x3F << 24)
2047	#define CRI_TXDEEMPH_OVERRIDE_EN (1 << 22)	1966	#define CRI_TXDEEMPH_OVERRIDE_EN (1 << 22)
2048	#define CRI_TXDEEMPH_OVERRIDE_5_0(x) ((x) << 16)	1967	#define CRI_TXDEEMPH_OVERRIDE_5_0(x) ((x) << 16)
2049	#define CRI_TXDEEMPH_OVERRIDE_5_0_MASK (0x3F << 16)	1968	#define CRI_TXDEEMPH_OVERRIDE_5_0_MASK (0x3F << 16)
		1969	#define CRI_LOADGEN_SEL(x) ((x) << 12)
		1970	#define CRI_LOADGEN_SEL_MASK (0x3 << 12)
		1971
		1972	#define MG_CLKHUB_LN0_PORT1 0x16839C
		1973	#define MG_CLKHUB_LN1_PORT1 0x16879C
		1974	#define MG_CLKHUB_LN0_PORT2 0x16939C
		1975	#define MG_CLKHUB_LN1_PORT2 0x16979C
		1976	#define MG_CLKHUB_LN0_PORT3 0x16A39C
		1977	#define MG_CLKHUB_LN1_PORT3 0x16A79C
		1978	#define MG_CLKHUB_LN0_PORT4 0x16B39C
		1979	#define MG_CLKHUB_LN1_PORT4 0x16B79C
		1980	#define MG_CLKHUB(port, ln) \
		1981	MG_PHY_PORT_LN(port, ln, MG_CLKHUB_LN0_PORT1, \
		1982	MG_CLKHUB_LN0_PORT2, \
		1983	MG_CLKHUB_LN1_PORT1)
		1984	#define CFG_LOW_RATE_LKREN_EN (1 << 11)
		1985
		1986	#define MG_TX_DCC_TX1LN0_PORT1 0x168110
		1987	#define MG_TX_DCC_TX1LN1_PORT1 0x168510
		1988	#define MG_TX_DCC_TX1LN0_PORT2 0x169110
		1989	#define MG_TX_DCC_TX1LN1_PORT2 0x169510
		1990	#define MG_TX_DCC_TX1LN0_PORT3 0x16A110
		1991	#define MG_TX_DCC_TX1LN1_PORT3 0x16A510
		1992	#define MG_TX_DCC_TX1LN0_PORT4 0x16B110
		1993	#define MG_TX_DCC_TX1LN1_PORT4 0x16B510
		1994	#define MG_TX1_DCC(port, ln) \
		1995	MG_PHY_PORT_LN(port, ln, MG_TX_DCC_TX1LN0_PORT1, \
		1996	MG_TX_DCC_TX1LN0_PORT2, \
		1997	MG_TX_DCC_TX1LN1_PORT1)
		1998	#define MG_TX_DCC_TX2LN0_PORT1 0x168090
		1999	#define MG_TX_DCC_TX2LN1_PORT1 0x168490
		2000	#define MG_TX_DCC_TX2LN0_PORT2 0x169090
		2001	#define MG_TX_DCC_TX2LN1_PORT2 0x169490
		2002	#define MG_TX_DCC_TX2LN0_PORT3 0x16A090
		2003	#define MG_TX_DCC_TX2LN1_PORT3 0x16A490
		2004	#define MG_TX_DCC_TX2LN0_PORT4 0x16B090
		2005	#define MG_TX_DCC_TX2LN1_PORT4 0x16B490
		2006	#define MG_TX2_DCC(port, ln) \
		2007	MG_PHY_PORT_LN(port, ln, MG_TX_DCC_TX2LN0_PORT1, \
		2008	MG_TX_DCC_TX2LN0_PORT2, \
		2009	MG_TX_DCC_TX2LN1_PORT1)
		2010	#define CFG_AMI_CK_DIV_OVERRIDE_VAL(x) ((x) << 25)
		2011	#define CFG_AMI_CK_DIV_OVERRIDE_VAL_MASK (0x3 << 25)
		2012	#define CFG_AMI_CK_DIV_OVERRIDE_EN (1 << 24)
		2013
		2014	#define MG_DP_MODE_LN0_ACU_PORT1 0x1683A0
		2015	#define MG_DP_MODE_LN1_ACU_PORT1 0x1687A0
		2016	#define MG_DP_MODE_LN0_ACU_PORT2 0x1693A0
		2017	#define MG_DP_MODE_LN1_ACU_PORT2 0x1697A0
		2018	#define MG_DP_MODE_LN0_ACU_PORT3 0x16A3A0
		2019	#define MG_DP_MODE_LN1_ACU_PORT3 0x16A7A0
		2020	#define MG_DP_MODE_LN0_ACU_PORT4 0x16B3A0
		2021	#define MG_DP_MODE_LN1_ACU_PORT4 0x16B7A0
		2022	#define MG_DP_MODE(port, ln) \
		2023	MG_PHY_PORT_LN(port, ln, MG_DP_MODE_LN0_ACU_PORT1, \
		2024	MG_DP_MODE_LN0_ACU_PORT2, \
		2025	MG_DP_MODE_LN1_ACU_PORT1)
		2026	#define MG_DP_MODE_CFG_DP_X2_MODE (1 << 7)
		2027	#define MG_DP_MODE_CFG_DP_X1_MODE (1 << 6)
		2028	#define MG_DP_MODE_CFG_TR2PWR_GATING (1 << 5)
		2029	#define MG_DP_MODE_CFG_TRPWR_GATING (1 << 4)
		2030	#define MG_DP_MODE_CFG_CLNPWR_GATING (1 << 3)
		2031	#define MG_DP_MODE_CFG_DIGPWR_GATING (1 << 2)
		2032	#define MG_DP_MODE_CFG_GAONPWR_GATING (1 << 1)
		2033
		2034	#define MG_MISC_SUS0_PORT1 0x168814
		2035	#define MG_MISC_SUS0_PORT2 0x169814
		2036	#define MG_MISC_SUS0_PORT3 0x16A814
		2037	#define MG_MISC_SUS0_PORT4 0x16B814
		2038	#define MG_MISC_SUS0(tc_port) \
		2039	_MMIO(_PORT(tc_port, MG_MISC_SUS0_PORT1, MG_MISC_SUS0_PORT2))
		2040	#define MG_MISC_SUS0_SUSCLK_DYNCLKGATE_MODE_MASK (3 << 14)
		2041	#define MG_MISC_SUS0_SUSCLK_DYNCLKGATE_MODE(x) ((x) << 14)
		2042	#define MG_MISC_SUS0_CFG_TR2PWR_GATING (1 << 12)
		2043	#define MG_MISC_SUS0_CFG_CL2PWR_GATING (1 << 11)
		2044	#define MG_MISC_SUS0_CFG_GAONPWR_GATING (1 << 10)
		2045	#define MG_MISC_SUS0_CFG_TRPWR_GATING (1 << 7)
		2046	#define MG_MISC_SUS0_CFG_CL1PWR_GATING (1 << 6)
		2047	#define MG_MISC_SUS0_CFG_DGPWR_GATING (1 << 5)
2050		2048
2051	/* The spec defines this only for BXT PHY0, but lets assume that this	2049	/* The spec defines this only for BXT PHY0, but lets assume that this
2052	* would exist for PHY1 too if it had a second channel.	2050	* would exist for PHY1 too if it had a second channel.
@@ -3086,18 +3084,9 @@ enum i915_power_well_id { · 1033 unmodified lines
3086	/*	3084	/*
3087	* GPIO regs	3085	* GPIO regs
3088	*/	3086	*/
3089	#define GPIOA _MMIO(0x5010)	3087	#define GPIO(gpio) _MMIO(dev_priv->gpio_mmio_base + 0x5010 + \
3090	#define GPIOB _MMIO(0x5014)	3088	4 * (gpio))
3091	#define GPIOC _MMIO(0x5018)	3089
3092	#define GPIOD _MMIO(0x501c)
3093	#define GPIOE _MMIO(0x5020)
3094	#define GPIOF _MMIO(0x5024)
3095	#define GPIOG _MMIO(0x5028)
3096	#define GPIOH _MMIO(0x502c)
3097	#define GPIOJ _MMIO(0x5034)
3098	#define GPIOK _MMIO(0x5038)
3099	#define GPIOL _MMIO(0x503C)
3100	#define GPIOM _MMIO(0x5040)
3101	# define GPIO_CLOCK_DIR_MASK (1 << 0)	3090	# define GPIO_CLOCK_DIR_MASK (1 << 0)
3102	# define GPIO_CLOCK_DIR_IN (0 << 1)	3091	# define GPIO_CLOCK_DIR_IN (0 << 1)
3103	# define GPIO_CLOCK_DIR_OUT (1 << 1)	3092	# define GPIO_CLOCK_DIR_OUT (1 << 1)
@@ -5476,6 +5465,7 @@ enum { · 2372 unmodified lines
5476	#define DP_AUX_CH_CTL_PSR_DATA_AUX_REG_SKL (1 << 14)	5465	#define DP_AUX_CH_CTL_PSR_DATA_AUX_REG_SKL (1 << 14)
5477	#define DP_AUX_CH_CTL_FS_DATA_AUX_REG_SKL (1 << 13)	5466	#define DP_AUX_CH_CTL_FS_DATA_AUX_REG_SKL (1 << 13)
5478	#define DP_AUX_CH_CTL_GTC_DATA_AUX_REG_SKL (1 << 12)	5467	#define DP_AUX_CH_CTL_GTC_DATA_AUX_REG_SKL (1 << 12)
		5468	#define DP_AUX_CH_CTL_TBT_IO (1 << 11)
5479	#define DP_AUX_CH_CTL_FW_SYNC_PULSE_SKL_MASK (0x1f << 5)	5469	#define DP_AUX_CH_CTL_FW_SYNC_PULSE_SKL_MASK (0x1f << 5)
5480	#define DP_AUX_CH_CTL_FW_SYNC_PULSE_SKL(c) (((c) - 1) << 5)	5470	#define DP_AUX_CH_CTL_FW_SYNC_PULSE_SKL(c) (((c) - 1) << 5)
5481	#define DP_AUX_CH_CTL_SYNC_PULSE_SKL(c) ((c) - 1)	5471	#define DP_AUX_CH_CTL_SYNC_PULSE_SKL(c) ((c) - 1)
@@ -6527,7 +6517,7 @@ enum { · 1045 unmodified lines
6527	#define PLANE_CTL_YUV422_UYVY (1 << 16)	6517	#define PLANE_CTL_YUV422_UYVY (1 << 16)
6528	#define PLANE_CTL_YUV422_YVYU (2 << 16)	6518	#define PLANE_CTL_YUV422_YVYU (2 << 16)
6529	#define PLANE_CTL_YUV422_VYUY (3 << 16)	6519	#define PLANE_CTL_YUV422_VYUY (3 << 16)
6530	#define PLANE_CTL_DECOMPRESSION_ENABLE (1 << 15)	6520	#define PLANE_CTL_RENDER_DECOMPRESSION_ENABLE (1 << 15)
6531	#define PLANE_CTL_TRICKLE_FEED_DISABLE (1 << 14)	6521	#define PLANE_CTL_TRICKLE_FEED_DISABLE (1 << 14)
6532	#define PLANE_CTL_PLANE_GAMMA_DISABLE (1 << 13) /* Pre-GLK */	6522	#define PLANE_CTL_PLANE_GAMMA_DISABLE (1 << 13) /* Pre-GLK */
6533	#define PLANE_CTL_TILED_MASK (0x7 << 10)	6523	#define PLANE_CTL_TILED_MASK (0x7 << 10)
@@ -7207,6 +7197,7 @@ enum { · 673 unmodified lines
7207	#define GEN11_TC3_HOTPLUG (1 << 18)	7197	#define GEN11_TC3_HOTPLUG (1 << 18)
7208	#define GEN11_TC2_HOTPLUG (1 << 17)	7198	#define GEN11_TC2_HOTPLUG (1 << 17)
7209	#define GEN11_TC1_HOTPLUG (1 << 16)	7199	#define GEN11_TC1_HOTPLUG (1 << 16)
		7200	#define GEN11_TC_HOTPLUG(tc_port) (1 << ((tc_port) + 16))
7210	#define GEN11_DE_TC_HOTPLUG_MASK (GEN11_TC4_HOTPLUG \| \	7201	#define GEN11_DE_TC_HOTPLUG_MASK (GEN11_TC4_HOTPLUG \| \
7211	GEN11_TC3_HOTPLUG \| \	7202	GEN11_TC3_HOTPLUG \| \
7212	GEN11_TC2_HOTPLUG \| \	7203	GEN11_TC2_HOTPLUG \| \
@@ -7215,6 +7206,7 @@ enum { · 2 unmodified lines
7215	#define GEN11_TBT3_HOTPLUG (1 << 2)	7206	#define GEN11_TBT3_HOTPLUG (1 << 2)
7216	#define GEN11_TBT2_HOTPLUG (1 << 1)	7207	#define GEN11_TBT2_HOTPLUG (1 << 1)
7217	#define GEN11_TBT1_HOTPLUG (1 << 0)	7208	#define GEN11_TBT1_HOTPLUG (1 << 0)
		7209	#define GEN11_TBT_HOTPLUG(tc_port) (1 << (tc_port))
7218	#define GEN11_DE_TBT_HOTPLUG_MASK (GEN11_TBT4_HOTPLUG \| \	7210	#define GEN11_DE_TBT_HOTPLUG_MASK (GEN11_TBT4_HOTPLUG \| \
7219	GEN11_TBT3_HOTPLUG \| \	7211	GEN11_TBT3_HOTPLUG \| \
7220	GEN11_TBT2_HOTPLUG \| \	7212	GEN11_TBT2_HOTPLUG \| \
@@ -7490,6 +7482,8 @@ enum { · 269 unmodified lines
7490		7482
7491	/* PCH */	7483	/* PCH */
7492		7484
		7485	#define PCH_DISPLAY_BASE 0xc0000u
		7486
7493	/* south display engine interrupt: IBX */	7487	/* south display engine interrupt: IBX */
7494	#define SDE_AUDIO_POWER_D (1 << 27)	7488	#define SDE_AUDIO_POWER_D (1 << 27)
7495	#define SDE_AUDIO_POWER_C (1 << 26)	7489	#define SDE_AUDIO_POWER_C (1 << 26)
@@ -7587,6 +7581,8 @@ enum { · 91 unmodified lines
7587	#define SDE_GMBUS_ICP (1 << 23)	7581	#define SDE_GMBUS_ICP (1 << 23)
7588	#define SDE_DDIB_HOTPLUG_ICP (1 << 17)	7582	#define SDE_DDIB_HOTPLUG_ICP (1 << 17)
7589	#define SDE_DDIA_HOTPLUG_ICP (1 << 16)	7583	#define SDE_DDIA_HOTPLUG_ICP (1 << 16)
		7584	#define SDE_TC_HOTPLUG_ICP(tc_port) (1 << ((tc_port) + 24))
		7585	#define SDE_DDI_HOTPLUG_ICP(port) (1 << ((port) + 16))
7590	#define SDE_DDI_MASK_ICP (SDE_DDIB_HOTPLUG_ICP \| \	7586	#define SDE_DDI_MASK_ICP (SDE_DDIB_HOTPLUG_ICP \| \
7591	SDE_DDIA_HOTPLUG_ICP)	7587	SDE_DDIA_HOTPLUG_ICP)
7592	#define SDE_TC_MASK_ICP (SDE_TC4_HOTPLUG_ICP \| \	7588	#define SDE_TC_MASK_ICP (SDE_TC4_HOTPLUG_ICP \| \
@@ -7782,20 +7778,6 @@ enum { · 189 unmodified lines
7782	#define ICP_TC_HPD_LONG_DETECT(tc_port) (2 << (tc_port) * 4)	7778	#define ICP_TC_HPD_LONG_DETECT(tc_port) (2 << (tc_port) * 4)
7783	#define ICP_TC_HPD_SHORT_DETECT(tc_port) (1 << (tc_port) * 4)	7779	#define ICP_TC_HPD_SHORT_DETECT(tc_port) (1 << (tc_port) * 4)
7784		7780
7785	#define PCH_GPIOA _MMIO(0xc5010)
7786	#define PCH_GPIOB _MMIO(0xc5014)
7787	#define PCH_GPIOC _MMIO(0xc5018)
7788	#define PCH_GPIOD _MMIO(0xc501c)
7789	#define PCH_GPIOE _MMIO(0xc5020)
7790	#define PCH_GPIOF _MMIO(0xc5024)
7791
7792	#define PCH_GMBUS0 _MMIO(0xc5100)
7793	#define PCH_GMBUS1 _MMIO(0xc5104)
7794	#define PCH_GMBUS2 _MMIO(0xc5108)
7795	#define PCH_GMBUS3 _MMIO(0xc510c)
7796	#define PCH_GMBUS4 _MMIO(0xc5110)
7797	#define PCH_GMBUS5 _MMIO(0xc5120)
7798
7799	#define _PCH_DPLL_A 0xc6014	7781	#define _PCH_DPLL_A 0xc6014
7800	#define _PCH_DPLL_B 0xc6018	7782	#define _PCH_DPLL_B 0xc6018
7801	#define PCH_DPLL(pll) _MMIO((pll) == 0 ? _PCH_DPLL_A : _PCH_DPLL_B)	7783	#define PCH_DPLL(pll) _MMIO((pll) == 0 ? _PCH_DPLL_A : _PCH_DPLL_B)
@@ -8498,8 +8480,10 @@ enum { · 696 unmodified lines
8498	#define GEN6_PM_RP_DOWN_THRESHOLD (1 << 4)	8480	#define GEN6_PM_RP_DOWN_THRESHOLD (1 << 4)
8499	#define GEN6_PM_RP_UP_EI_EXPIRED (1 << 2)	8481	#define GEN6_PM_RP_UP_EI_EXPIRED (1 << 2)
8500	#define GEN6_PM_RP_DOWN_EI_EXPIRED (1 << 1)	8482	#define GEN6_PM_RP_DOWN_EI_EXPIRED (1 << 1)
8501	#define GEN6_PM_RPS_EVENTS (GEN6_PM_RP_UP_THRESHOLD \| \	8483	#define GEN6_PM_RPS_EVENTS (GEN6_PM_RP_UP_EI_EXPIRED \| \
8502	GEN6_PM_RP_DOWN_THRESHOLD \| \	8484	GEN6_PM_RP_UP_THRESHOLD \| \
		8485	GEN6_PM_RP_DOWN_EI_EXPIRED \| \
		8486	GEN6_PM_RP_DOWN_THRESHOLD \| \
8503	GEN6_PM_RP_DOWN_TIMEOUT)	8487	GEN6_PM_RP_DOWN_TIMEOUT)
8504		8488
8505	#define GEN7_GT_SCRATCH(i) _MMIO(0x4F100 + (i) * 4)	8489	#define GEN7_GT_SCRATCH(i) _MMIO(0x4F100 + (i) * 4)
@@ -8827,46 +8811,78 @@ enum { · 321 unmodified lines
8827	#define HSW_AUD_CHICKENBIT _MMIO(0x65f10)	8811	#define HSW_AUD_CHICKENBIT _MMIO(0x65f10)
8828	#define SKL_AUD_CODEC_WAKE_SIGNAL (1 << 15)	8812	#define SKL_AUD_CODEC_WAKE_SIGNAL (1 << 15)
8829		8813
8830	/* HSW Power Wells */
8831	#define _HSW_PWR_WELL_CTL1 0x45400
8832	#define _HSW_PWR_WELL_CTL2 0x45404
8833	#define _HSW_PWR_WELL_CTL3 0x45408
8834	#define _HSW_PWR_WELL_CTL4 0x4540C
8835
8836	#define _ICL_PWR_WELL_CTL_AUX1 0x45440
8837	#define _ICL_PWR_WELL_CTL_AUX2 0x45444
8838	#define _ICL_PWR_WELL_CTL_AUX4 0x4544C
8839
8840	#define _ICL_PWR_WELL_CTL_DDI1 0x45450
8841	#define _ICL_PWR_WELL_CTL_DDI2 0x45454
8842	#define _ICL_PWR_WELL_CTL_DDI4 0x4545C
8843
8844	/*	8814	/*
8845	* Each power well control register contains up to 16 (request, status) HW	8815	* HSW - ICL power wells
8846	* flag tuples. The register index and HW flag shift is determined by the	8816	*
8847	* power well ID (see i915_power_well_id). There are 4 possible sources of	8817	* Platforms have up to 3 power well control register sets, each set
8848	* power well requests each source having its own set of control registers:	8818	* controlling up to 16 power wells via a request/status HW flag tuple:
8849	* BIOS, DRIVER, KVMR, DEBUG.	8819	* - main (HSW_PWR_WELL_CTL[1-4])
		8820	* - AUX (ICL_PWR_WELL_CTL_AUX[1-4])
		8821	* - DDI (ICL_PWR_WELL_CTL_DDI[1-4])
		8822	* Each control register set consists of up to 4 registers used by different
		8823	* sources that can request a power well to be enabled:
		8824	* - BIOS (HSW_PWR_WELL_CTL1/ICL_PWR_WELL_CTL_AUX1/ICL_PWR_WELL_CTL_DDI1)
		8825	* - DRIVER (HSW_PWR_WELL_CTL2/ICL_PWR_WELL_CTL_AUX2/ICL_PWR_WELL_CTL_DDI2)
		8826	* - KVMR (HSW_PWR_WELL_CTL3) (only in the main register set)
		8827	* - DEBUG (HSW_PWR_WELL_CTL4/ICL_PWR_WELL_CTL_AUX4/ICL_PWR_WELL_CTL_DDI4)
8850	*/	8828	*/
8851	#define _HSW_PW_REG_IDX(pw) ((pw) >> 4)	8829	#define HSW_PWR_WELL_CTL1 _MMIO(0x45400)
8852	#define _HSW_PW_SHIFT(pw) (((pw) & 0xf) * 2)	8830	#define HSW_PWR_WELL_CTL2 _MMIO(0x45404)
8853	#define HSW_PWR_WELL_CTL_BIOS(pw) _MMIO(_PICK(_HSW_PW_REG_IDX(pw), \	8831	#define HSW_PWR_WELL_CTL3 _MMIO(0x45408)
8854	_HSW_PWR_WELL_CTL1, \	8832	#define HSW_PWR_WELL_CTL4 _MMIO(0x4540C)
8855	_ICL_PWR_WELL_CTL_AUX1, \	8833	#define HSW_PWR_WELL_CTL_REQ(pw_idx) (0x2 << ((pw_idx) * 2))
8856	_ICL_PWR_WELL_CTL_DDI1))	8834	#define HSW_PWR_WELL_CTL_STATE(pw_idx) (0x1 << ((pw_idx) * 2))
8857	#define HSW_PWR_WELL_CTL_DRIVER(pw) _MMIO(_PICK(_HSW_PW_REG_IDX(pw), \	8835
8858	_HSW_PWR_WELL_CTL2, \	8836	/* HSW/BDW power well */
8859	_ICL_PWR_WELL_CTL_AUX2, \	8837	#define HSW_PW_CTL_IDX_GLOBAL 15
8860	_ICL_PWR_WELL_CTL_DDI2))	8838
8861	/* KVMR doesn't have a reg for AUX or DDI power well control */	8839	/* SKL/BXT/GLK/CNL power wells */
8862	#define HSW_PWR_WELL_CTL_KVMR _MMIO(_HSW_PWR_WELL_CTL3)	8840	#define SKL_PW_CTL_IDX_PW_2 15
8863	#define HSW_PWR_WELL_CTL_DEBUG(pw) _MMIO(_PICK(_HSW_PW_REG_IDX(pw), \	8841	#define SKL_PW_CTL_IDX_PW_1 14
8864	_HSW_PWR_WELL_CTL4, \	8842	#define CNL_PW_CTL_IDX_AUX_F 12
8865	_ICL_PWR_WELL_CTL_AUX4, \	8843	#define CNL_PW_CTL_IDX_AUX_D 11
8866	_ICL_PWR_WELL_CTL_DDI4))	8844	#define GLK_PW_CTL_IDX_AUX_C 10
8867		8845	#define GLK_PW_CTL_IDX_AUX_B 9
8868	#define HSW_PWR_WELL_CTL_REQ(pw) (1 << (_HSW_PW_SHIFT(pw) + 1))	8846	#define GLK_PW_CTL_IDX_AUX_A 8
8869	#define HSW_PWR_WELL_CTL_STATE(pw) (1 << _HSW_PW_SHIFT(pw))	8847	#define CNL_PW_CTL_IDX_DDI_F 6
		8848	#define SKL_PW_CTL_IDX_DDI_D 4
		8849	#define SKL_PW_CTL_IDX_DDI_C 3
		8850	#define SKL_PW_CTL_IDX_DDI_B 2
		8851	#define SKL_PW_CTL_IDX_DDI_A_E 1
		8852	#define GLK_PW_CTL_IDX_DDI_A 1
		8853	#define SKL_PW_CTL_IDX_MISC_IO 0
		8854
		8855	/* ICL - power wells */
		8856	#define ICL_PW_CTL_IDX_PW_4 3
		8857	#define ICL_PW_CTL_IDX_PW_3 2
		8858	#define ICL_PW_CTL_IDX_PW_2 1
		8859	#define ICL_PW_CTL_IDX_PW_1 0
		8860
		8861	#define ICL_PWR_WELL_CTL_AUX1 _MMIO(0x45440)
		8862	#define ICL_PWR_WELL_CTL_AUX2 _MMIO(0x45444)
		8863	#define ICL_PWR_WELL_CTL_AUX4 _MMIO(0x4544C)
		8864	#define ICL_PW_CTL_IDX_AUX_TBT4 11
		8865	#define ICL_PW_CTL_IDX_AUX_TBT3 10
		8866	#define ICL_PW_CTL_IDX_AUX_TBT2 9
		8867	#define ICL_PW_CTL_IDX_AUX_TBT1 8
		8868	#define ICL_PW_CTL_IDX_AUX_F 5
		8869	#define ICL_PW_CTL_IDX_AUX_E 4
		8870	#define ICL_PW_CTL_IDX_AUX_D 3
		8871	#define ICL_PW_CTL_IDX_AUX_C 2
		8872	#define ICL_PW_CTL_IDX_AUX_B 1
		8873	#define ICL_PW_CTL_IDX_AUX_A 0
		8874
		8875	#define ICL_PWR_WELL_CTL_DDI1 _MMIO(0x45450)
		8876	#define ICL_PWR_WELL_CTL_DDI2 _MMIO(0x45454)
		8877	#define ICL_PWR_WELL_CTL_DDI4 _MMIO(0x4545C)
		8878	#define ICL_PW_CTL_IDX_DDI_F 5
		8879	#define ICL_PW_CTL_IDX_DDI_E 4
		8880	#define ICL_PW_CTL_IDX_DDI_D 3
		8881	#define ICL_PW_CTL_IDX_DDI_C 2
		8882	#define ICL_PW_CTL_IDX_DDI_B 1
		8883	#define ICL_PW_CTL_IDX_DDI_A 0
		8884
		8885	/* HSW - power well misc debug registers */
8870	#define HSW_PWR_WELL_CTL5 _MMIO(0x45410)	8886	#define HSW_PWR_WELL_CTL5 _MMIO(0x45410)
8871	#define HSW_PWR_WELL_ENABLE_SINGLE_STEP (1 << 31)	8887	#define HSW_PWR_WELL_ENABLE_SINGLE_STEP (1 << 31)
8872	#define HSW_PWR_WELL_PWR_GATE_OVERRIDE (1 << 20)	8888	#define HSW_PWR_WELL_PWR_GATE_OVERRIDE (1 << 20)
@@ -8878,22 +8894,32 @@ enum skl_power_gate { · 5 unmodified lines
8878	SKL_PG0,	8894	SKL_PG0,
8879	SKL_PG1,	8895	SKL_PG1,
8880	SKL_PG2,	8896	SKL_PG2,
		8897	ICL_PG3,
		8898	ICL_PG4,
8881	};	8899	};
8882		8900
8883	#define SKL_FUSE_STATUS _MMIO(0x42000)	8901	#define SKL_FUSE_STATUS _MMIO(0x42000)
8884	#define SKL_FUSE_DOWNLOAD_STATUS (1 << 31)	8902	#define SKL_FUSE_DOWNLOAD_STATUS (1 << 31)
8885	/* PG0 (HW control->no power well ID), PG1..PG2 (SKL_DISP_PW1..SKL_DISP_PW2) */	8903	/*
8886	#define SKL_PW_TO_PG(pw) ((pw) - SKL_DISP_PW_1 + SKL_PG1)	8904	* PG0 is HW controlled, so doesn't have a corresponding power well control knob
8887	/* PG0 (HW control->no power well ID), PG1..PG4 (ICL_DISP_PW1..ICL_DISP_PW4) */	8905	* SKL_DISP_PW1_IDX..SKL_DISP_PW2_IDX -> PG1..PG2
8888	#define ICL_PW_TO_PG(pw) ((pw) - ICL_DISP_PW_1 + SKL_PG1)	8906	*/
		8907	#define SKL_PW_CTL_IDX_TO_PG(pw_idx) \
		8908	((pw_idx) - SKL_PW_CTL_IDX_PW_1 + SKL_PG1)
		8909	/*
		8910	* PG0 is HW controlled, so doesn't have a corresponding power well control knob
		8911	* ICL_DISP_PW1_IDX..ICL_DISP_PW4_IDX -> PG1..PG4
		8912	*/
		8913	#define ICL_PW_CTL_IDX_TO_PG(pw_idx) \
		8914	((pw_idx) - ICL_PW_CTL_IDX_PW_1 + SKL_PG1)
8889	#define SKL_FUSE_PG_DIST_STATUS(pg) (1 << (27 - (pg)))	8915	#define SKL_FUSE_PG_DIST_STATUS(pg) (1 << (27 - (pg)))
8890		8916
8891	#define _CNL_AUX_REG_IDX(pw) ((pw) - 9)	8917	#define _CNL_AUX_REG_IDX(pw_idx) ((pw_idx) - GLK_PW_CTL_IDX_AUX_B)
8892	#define _CNL_AUX_ANAOVRD1_B 0x162250	8918	#define _CNL_AUX_ANAOVRD1_B 0x162250
8893	#define _CNL_AUX_ANAOVRD1_C 0x162210	8919	#define _CNL_AUX_ANAOVRD1_C 0x162210
8894	#define _CNL_AUX_ANAOVRD1_D 0x1622D0	8920	#define _CNL_AUX_ANAOVRD1_D 0x1622D0
8895	#define _CNL_AUX_ANAOVRD1_F 0x162A90	8921	#define _CNL_AUX_ANAOVRD1_F 0x162A90
8896	#define CNL_AUX_ANAOVRD1(pw) _MMIO(_PICK(_CNL_AUX_REG_IDX(pw), \	8922	#define CNL_AUX_ANAOVRD1(pw_idx) _MMIO(_PICK(_CNL_AUX_REG_IDX(pw_idx), \
8897	_CNL_AUX_ANAOVRD1_B, \	8923	_CNL_AUX_ANAOVRD1_B, \
8898	_CNL_AUX_ANAOVRD1_C, \	8924	_CNL_AUX_ANAOVRD1_C, \
8899	_CNL_AUX_ANAOVRD1_D, \	8925	_CNL_AUX_ANAOVRD1_D, \
@@ -9367,9 +9393,13 @@ enum skl_power_gate { · 467 unmodified lines
9367	#define MG_CLKTOP2_HSCLKCTL_CORE_INPUTSEL_MASK (0x1 << 16)	9393	#define MG_CLKTOP2_HSCLKCTL_CORE_INPUTSEL_MASK (0x1 << 16)
9368	#define MG_CLKTOP2_HSCLKCTL_TLINEDRV_CLKSEL(x) ((x) << 14)	9394	#define MG_CLKTOP2_HSCLKCTL_TLINEDRV_CLKSEL(x) ((x) << 14)
9369	#define MG_CLKTOP2_HSCLKCTL_TLINEDRV_CLKSEL_MASK (0x3 << 14)	9395	#define MG_CLKTOP2_HSCLKCTL_TLINEDRV_CLKSEL_MASK (0x3 << 14)
9370	#define MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO(x) ((x) << 12)
9371	#define MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_MASK (0x3 << 12)	9396	#define MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_MASK (0x3 << 12)
		9397	#define MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_2 (0 << 12)
		9398	#define MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_3 (1 << 12)
		9399	#define MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_5 (2 << 12)
		9400	#define MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_7 (3 << 12)
9372	#define MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO(x) ((x) << 8)	9401	#define MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO(x) ((x) << 8)
		9402	#define MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO_SHIFT 8
9373	#define MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO_MASK (0xf << 8)	9403	#define MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO_MASK (0xf << 8)
9374	#define MG_CLKTOP2_HSCLKCTL(port) _MMIO_PORT((port) - PORT_C, \	9404	#define MG_CLKTOP2_HSCLKCTL(port) _MMIO_PORT((port) - PORT_C, \
9375	_MG_CLKTOP2_HSCLKCTL_PORT1, \	9405	_MG_CLKTOP2_HSCLKCTL_PORT1, \
@@ -9380,7 +9410,10 @@ enum skl_power_gate { · 4 unmodified lines
9380	#define _MG_PLL_DIV0_PORT3 0x16AA00	9410	#define _MG_PLL_DIV0_PORT3 0x16AA00
9381	#define _MG_PLL_DIV0_PORT4 0x16BA00	9411	#define _MG_PLL_DIV0_PORT4 0x16BA00
9382	#define MG_PLL_DIV0_FRACNEN_H (1 << 30)	9412	#define MG_PLL_DIV0_FRACNEN_H (1 << 30)
		9413	#define MG_PLL_DIV0_FBDIV_FRAC_MASK (0x3fffff << 8)
		9414	#define MG_PLL_DIV0_FBDIV_FRAC_SHIFT 8
9383	#define MG_PLL_DIV0_FBDIV_FRAC(x) ((x) << 8)	9415	#define MG_PLL_DIV0_FBDIV_FRAC(x) ((x) << 8)
		9416	#define MG_PLL_DIV0_FBDIV_INT_MASK (0xff << 0)
9384	#define MG_PLL_DIV0_FBDIV_INT(x) ((x) << 0)	9417	#define MG_PLL_DIV0_FBDIV_INT(x) ((x) << 0)
9385	#define MG_PLL_DIV0(port) _MMIO_PORT((port) - PORT_C, _MG_PLL_DIV0_PORT1, \	9418	#define MG_PLL_DIV0(port) _MMIO_PORT((port) - PORT_C, _MG_PLL_DIV0_PORT1, \
9386	_MG_PLL_DIV0_PORT2)	9419	_MG_PLL_DIV0_PORT2)
@@ -9395,6 +9428,7 @@ enum skl_power_gate { · 8 unmodified lines
9395	#define MG_PLL_DIV1_DITHER_DIV_4 (2 << 12)	9428	#define MG_PLL_DIV1_DITHER_DIV_4 (2 << 12)
9396	#define MG_PLL_DIV1_DITHER_DIV_8 (3 << 12)	9429	#define MG_PLL_DIV1_DITHER_DIV_8 (3 << 12)
9397	#define MG_PLL_DIV1_NDIVRATIO(x) ((x) << 4)	9430	#define MG_PLL_DIV1_NDIVRATIO(x) ((x) << 4)
		9431	#define MG_PLL_DIV1_FBPREDIV_MASK (0xf << 0)
9398	#define MG_PLL_DIV1_FBPREDIV(x) ((x) << 0)	9432	#define MG_PLL_DIV1_FBPREDIV(x) ((x) << 0)
9399	#define MG_PLL_DIV1(port) _MMIO_PORT((port) - PORT_C, _MG_PLL_DIV1_PORT1, \	9433	#define MG_PLL_DIV1(port) _MMIO_PORT((port) - PORT_C, _MG_PLL_DIV1_PORT1, \
9400	_MG_PLL_DIV1_PORT2)	9434	_MG_PLL_DIV1_PORT2)
@@ -10347,8 +10381,8 @@ enum skl_power_gate { · 946 unmodified lines
10347	#define ICL_PHY_MISC_DE_IO_COMP_PWR_DOWN (1 << 23)	10381	#define ICL_PHY_MISC_DE_IO_COMP_PWR_DOWN (1 << 23)
10348		10382
10349	/* Icelake Display Stream Compression Registers */	10383	/* Icelake Display Stream Compression Registers */
10350	#define DSCA_PICTURE_PARAMETER_SET_0 0x6B200	10384	#define DSCA_PICTURE_PARAMETER_SET_0 _MMIO(0x6B200)
10351	#define DSCC_PICTURE_PARAMETER_SET_0 0x6BA00	10385	#define DSCC_PICTURE_PARAMETER_SET_0 _MMIO(0x6BA00)
10352	#define _ICL_DSC0_PICTURE_PARAMETER_SET_0_PB 0x78270	10386	#define _ICL_DSC0_PICTURE_PARAMETER_SET_0_PB 0x78270
10353	#define _ICL_DSC1_PICTURE_PARAMETER_SET_0_PB 0x78370	10387	#define _ICL_DSC1_PICTURE_PARAMETER_SET_0_PB 0x78370
10354	#define _ICL_DSC0_PICTURE_PARAMETER_SET_0_PC 0x78470	10388	#define _ICL_DSC0_PICTURE_PARAMETER_SET_0_PC 0x78470
@@ -10368,8 +10402,8 @@ enum skl_power_gate { · 13 unmodified lines
10368	#define DSC_VER_MIN_SHIFT 4	10402	#define DSC_VER_MIN_SHIFT 4
10369	#define DSC_VER_MAJ (0x1 << 0)	10403	#define DSC_VER_MAJ (0x1 << 0)
10370		10404
10371	#define DSCA_PICTURE_PARAMETER_SET_1 0x6B204	10405	#define DSCA_PICTURE_PARAMETER_SET_1 _MMIO(0x6B204)
10372	#define DSCC_PICTURE_PARAMETER_SET_1 0x6BA04	10406	#define DSCC_PICTURE_PARAMETER_SET_1 _MMIO(0x6BA04)
10373	#define _ICL_DSC0_PICTURE_PARAMETER_SET_1_PB 0x78274	10407	#define _ICL_DSC0_PICTURE_PARAMETER_SET_1_PB 0x78274
10374	#define _ICL_DSC1_PICTURE_PARAMETER_SET_1_PB 0x78374	10408	#define _ICL_DSC1_PICTURE_PARAMETER_SET_1_PB 0x78374
10375	#define _ICL_DSC0_PICTURE_PARAMETER_SET_1_PC 0x78474	10409	#define _ICL_DSC0_PICTURE_PARAMETER_SET_1_PC 0x78474
@@ -10382,8 +10416,8 @@ enum skl_power_gate { · 6 unmodified lines
10382	_ICL_DSC1_PICTURE_PARAMETER_SET_1_PC)	10416	_ICL_DSC1_PICTURE_PARAMETER_SET_1_PC)
10383	#define DSC_BPP(bpp) ((bpp) << 0)	10417	#define DSC_BPP(bpp) ((bpp) << 0)
10384		10418
10385	#define DSCA_PICTURE_PARAMETER_SET_2 0x6B208	10419	#define DSCA_PICTURE_PARAMETER_SET_2 _MMIO(0x6B208)
10386	#define DSCC_PICTURE_PARAMETER_SET_2 0x6BA08	10420	#define DSCC_PICTURE_PARAMETER_SET_2 _MMIO(0x6BA08)
10387	#define _ICL_DSC0_PICTURE_PARAMETER_SET_2_PB 0x78278	10421	#define _ICL_DSC0_PICTURE_PARAMETER_SET_2_PB 0x78278
10388	#define _ICL_DSC1_PICTURE_PARAMETER_SET_2_PB 0x78378	10422	#define _ICL_DSC1_PICTURE_PARAMETER_SET_2_PB 0x78378
10389	#define _ICL_DSC0_PICTURE_PARAMETER_SET_2_PC 0x78478	10423	#define _ICL_DSC0_PICTURE_PARAMETER_SET_2_PC 0x78478
@@ -10397,8 +10431,8 @@ enum skl_power_gate { · 7 unmodified lines
10397	#define DSC_PIC_WIDTH(pic_width) ((pic_width) << 16)	10431	#define DSC_PIC_WIDTH(pic_width) ((pic_width) << 16)
10398	#define DSC_PIC_HEIGHT(pic_height) ((pic_height) << 0)	10432	#define DSC_PIC_HEIGHT(pic_height) ((pic_height) << 0)
10399		10433
10400	#define DSCA_PICTURE_PARAMETER_SET_3 0x6B20C	10434	#define DSCA_PICTURE_PARAMETER_SET_3 _MMIO(0x6B20C)
10401	#define DSCC_PICTURE_PARAMETER_SET_3 0x6BA0C	10435	#define DSCC_PICTURE_PARAMETER_SET_3 _MMIO(0x6BA0C)
10402	#define _ICL_DSC0_PICTURE_PARAMETER_SET_3_PB 0x7827C	10436	#define _ICL_DSC0_PICTURE_PARAMETER_SET_3_PB 0x7827C
10403	#define _ICL_DSC1_PICTURE_PARAMETER_SET_3_PB 0x7837C	10437	#define _ICL_DSC1_PICTURE_PARAMETER_SET_3_PB 0x7837C
10404	#define _ICL_DSC0_PICTURE_PARAMETER_SET_3_PC 0x7847C	10438	#define _ICL_DSC0_PICTURE_PARAMETER_SET_3_PC 0x7847C
@@ -10412,8 +10446,8 @@ enum skl_power_gate { · 7 unmodified lines
10412	#define DSC_SLICE_WIDTH(slice_width) ((slice_width) << 16)	10446	#define DSC_SLICE_WIDTH(slice_width) ((slice_width) << 16)
10413	#define DSC_SLICE_HEIGHT(slice_height) ((slice_height) << 0)	10447	#define DSC_SLICE_HEIGHT(slice_height) ((slice_height) << 0)
10414		10448
10415	#define DSCA_PICTURE_PARAMETER_SET_4 0x6B210	10449	#define DSCA_PICTURE_PARAMETER_SET_4 _MMIO(0x6B210)
10416	#define DSCC_PICTURE_PARAMETER_SET_4 0x6BA10	10450	#define DSCC_PICTURE_PARAMETER_SET_4 _MMIO(0x6BA10)
10417	#define _ICL_DSC0_PICTURE_PARAMETER_SET_4_PB 0x78280	10451	#define _ICL_DSC0_PICTURE_PARAMETER_SET_4_PB 0x78280
10418	#define _ICL_DSC1_PICTURE_PARAMETER_SET_4_PB 0x78380	10452	#define _ICL_DSC1_PICTURE_PARAMETER_SET_4_PB 0x78380
10419	#define _ICL_DSC0_PICTURE_PARAMETER_SET_4_PC 0x78480	10453	#define _ICL_DSC0_PICTURE_PARAMETER_SET_4_PC 0x78480
@@ -10422,13 +10456,13 @@ enum skl_power_gate { · 2 unmodified lines
10422	_ICL_DSC0_PICTURE_PARAMETER_SET_4_PB, \	10456	_ICL_DSC0_PICTURE_PARAMETER_SET_4_PB, \
10423	_ICL_DSC0_PICTURE_PARAMETER_SET_4_PC)	10457	_ICL_DSC0_PICTURE_PARAMETER_SET_4_PC)
10424	#define ICL_DSC1_PICTURE_PARAMETER_SET_4(pipe) _MMIO_PIPE((pipe) - PIPE_B, \	10458	#define ICL_DSC1_PICTURE_PARAMETER_SET_4(pipe) _MMIO_PIPE((pipe) - PIPE_B, \
10425	_ICL_DSC0_PICTURE_PARAMETER_SET_4_PB, \	10459	_ICL_DSC1_PICTURE_PARAMETER_SET_4_PB, \
10426	_ICL_DSC1_PICTURE_PARAMETER_SET_4_PC)	10460	_ICL_DSC1_PICTURE_PARAMETER_SET_4_PC)
10427	#define DSC_INITIAL_DEC_DELAY(dec_delay) ((dec_delay) << 16)	10461	#define DSC_INITIAL_DEC_DELAY(dec_delay) ((dec_delay) << 16)
10428	#define DSC_INITIAL_XMIT_DELAY(xmit_delay) ((xmit_delay) << 0)	10462	#define DSC_INITIAL_XMIT_DELAY(xmit_delay) ((xmit_delay) << 0)
10429		10463
10430	#define DSCA_PICTURE_PARAMETER_SET_5 0x6B214	10464	#define DSCA_PICTURE_PARAMETER_SET_5 _MMIO(0x6B214)
10431	#define DSCC_PICTURE_PARAMETER_SET_5 0x6BA14	10465	#define DSCC_PICTURE_PARAMETER_SET_5 _MMIO(0x6BA14)
10432	#define _ICL_DSC0_PICTURE_PARAMETER_SET_5_PB 0x78284	10466	#define _ICL_DSC0_PICTURE_PARAMETER_SET_5_PB 0x78284
10433	#define _ICL_DSC1_PICTURE_PARAMETER_SET_5_PB 0x78384	10467	#define _ICL_DSC1_PICTURE_PARAMETER_SET_5_PB 0x78384
10434	#define _ICL_DSC0_PICTURE_PARAMETER_SET_5_PC 0x78484	10468	#define _ICL_DSC0_PICTURE_PARAMETER_SET_5_PC 0x78484
@@ -10437,13 +10471,13 @@ enum skl_power_gate { · 2 unmodified lines
10437	_ICL_DSC0_PICTURE_PARAMETER_SET_5_PB, \	10471	_ICL_DSC0_PICTURE_PARAMETER_SET_5_PB, \
10438	_ICL_DSC0_PICTURE_PARAMETER_SET_5_PC)	10472	_ICL_DSC0_PICTURE_PARAMETER_SET_5_PC)
10439	#define ICL_DSC1_PICTURE_PARAMETER_SET_5(pipe) _MMIO_PIPE((pipe) - PIPE_B, \	10473	#define ICL_DSC1_PICTURE_PARAMETER_SET_5(pipe) _MMIO_PIPE((pipe) - PIPE_B, \
10440	_ICL_DSC1_PICTURE_PARAMETER_SET_5_PC, \	10474	_ICL_DSC1_PICTURE_PARAMETER_SET_5_PB, \
10441	_ICL_DSC1_PICTURE_PARAMETER_SET_5_PC)	10475	_ICL_DSC1_PICTURE_PARAMETER_SET_5_PC)
10442	#define DSC_SCALE_DEC_INTINT(scale_dec) ((scale_dec) << 16)	10476	#define DSC_SCALE_DEC_INT(scale_dec) ((scale_dec) << 16)
10443	#define DSC_SCALE_INC_INT(scale_inc) ((scale_inc) << 0)	10477	#define DSC_SCALE_INC_INT(scale_inc) ((scale_inc) << 0)
10444		10478
10445	#define DSCA_PICTURE_PARAMETER_SET_6 0x6B218	10479	#define DSCA_PICTURE_PARAMETER_SET_6 _MMIO(0x6B218)
10446	#define DSCC_PICTURE_PARAMETER_SET_6 0x6BA18	10480	#define DSCC_PICTURE_PARAMETER_SET_6 _MMIO(0x6BA18)
10447	#define _ICL_DSC0_PICTURE_PARAMETER_SET_6_PB 0x78288	10481	#define _ICL_DSC0_PICTURE_PARAMETER_SET_6_PB 0x78288
10448	#define _ICL_DSC1_PICTURE_PARAMETER_SET_6_PB 0x78388	10482	#define _ICL_DSC1_PICTURE_PARAMETER_SET_6_PB 0x78388
10449	#define _ICL_DSC0_PICTURE_PARAMETER_SET_6_PC 0x78488	10483	#define _ICL_DSC0_PICTURE_PARAMETER_SET_6_PC 0x78488
@@ -10454,13 +10488,13 @@ enum skl_power_gate { · 4 unmodified lines
10454	#define ICL_DSC1_PICTURE_PARAMETER_SET_6(pipe) _MMIO_PIPE((pipe) - PIPE_B, \	10488	#define ICL_DSC1_PICTURE_PARAMETER_SET_6(pipe) _MMIO_PIPE((pipe) - PIPE_B, \
10455	_ICL_DSC1_PICTURE_PARAMETER_SET_6_PB, \	10489	_ICL_DSC1_PICTURE_PARAMETER_SET_6_PB, \
10456	_ICL_DSC1_PICTURE_PARAMETER_SET_6_PC)	10490	_ICL_DSC1_PICTURE_PARAMETER_SET_6_PC)
10457	#define DSC_FLATNESS_MAX_QP(max_qp) (qp << 24)	10491	#define DSC_FLATNESS_MAX_QP(max_qp) ((max_qp) << 24)
10458	#define DSC_FLATNESS_MIN_QP(min_qp) (qp << 16)	10492	#define DSC_FLATNESS_MIN_QP(min_qp) ((min_qp) << 16)
10459	#define DSC_FIRST_LINE_BPG_OFFSET(offset) ((offset) << 8)	10493	#define DSC_FIRST_LINE_BPG_OFFSET(offset) ((offset) << 8)
10460	#define DSC_INITIAL_SCALE_VALUE(value) ((value) << 0)	10494	#define DSC_INITIAL_SCALE_VALUE(value) ((value) << 0)
10461		10495
10462	#define DSCA_PICTURE_PARAMETER_SET_7 0x6B21C	10496	#define DSCA_PICTURE_PARAMETER_SET_7 _MMIO(0x6B21C)
10463	#define DSCC_PICTURE_PARAMETER_SET_7 0x6BA1C	10497	#define DSCC_PICTURE_PARAMETER_SET_7 _MMIO(0x6BA1C)
10464	#define _ICL_DSC0_PICTURE_PARAMETER_SET_7_PB 0x7828C	10498	#define _ICL_DSC0_PICTURE_PARAMETER_SET_7_PB 0x7828C
10465	#define _ICL_DSC1_PICTURE_PARAMETER_SET_7_PB 0x7838C	10499	#define _ICL_DSC1_PICTURE_PARAMETER_SET_7_PB 0x7838C
10466	#define _ICL_DSC0_PICTURE_PARAMETER_SET_7_PC 0x7848C	10500	#define _ICL_DSC0_PICTURE_PARAMETER_SET_7_PC 0x7848C
@@ -10474,8 +10508,8 @@ enum skl_power_gate { · 7 unmodified lines
10474	#define DSC_NFL_BPG_OFFSET(bpg_offset) ((bpg_offset) << 16)	10508	#define DSC_NFL_BPG_OFFSET(bpg_offset) ((bpg_offset) << 16)
10475	#define DSC_SLICE_BPG_OFFSET(bpg_offset) ((bpg_offset) << 0)	10509	#define DSC_SLICE_BPG_OFFSET(bpg_offset) ((bpg_offset) << 0)
10476		10510
10477	#define DSCA_PICTURE_PARAMETER_SET_8 0x6B220	10511	#define DSCA_PICTURE_PARAMETER_SET_8 _MMIO(0x6B220)
10478	#define DSCC_PICTURE_PARAMETER_SET_8 0x6BA20	10512	#define DSCC_PICTURE_PARAMETER_SET_8 _MMIO(0x6BA20)
10479	#define _ICL_DSC0_PICTURE_PARAMETER_SET_8_PB 0x78290	10513	#define _ICL_DSC0_PICTURE_PARAMETER_SET_8_PB 0x78290
10480	#define _ICL_DSC1_PICTURE_PARAMETER_SET_8_PB 0x78390	10514	#define _ICL_DSC1_PICTURE_PARAMETER_SET_8_PB 0x78390
10481	#define _ICL_DSC0_PICTURE_PARAMETER_SET_8_PC 0x78490	10515	#define _ICL_DSC0_PICTURE_PARAMETER_SET_8_PC 0x78490
@@ -10489,8 +10523,8 @@ enum skl_power_gate { · 7 unmodified lines
10489	#define DSC_INITIAL_OFFSET(initial_offset) ((initial_offset) << 16)	10523	#define DSC_INITIAL_OFFSET(initial_offset) ((initial_offset) << 16)
10490	#define DSC_FINAL_OFFSET(final_offset) ((final_offset) << 0)	10524	#define DSC_FINAL_OFFSET(final_offset) ((final_offset) << 0)
10491		10525
10492	#define DSCA_PICTURE_PARAMETER_SET_9 0x6B224	10526	#define DSCA_PICTURE_PARAMETER_SET_9 _MMIO(0x6B224)
10493	#define DSCC_PICTURE_PARAMETER_SET_9 0x6BA24	10527	#define DSCC_PICTURE_PARAMETER_SET_9 _MMIO(0x6BA24)
10494	#define _ICL_DSC0_PICTURE_PARAMETER_SET_9_PB 0x78294	10528	#define _ICL_DSC0_PICTURE_PARAMETER_SET_9_PB 0x78294
10495	#define _ICL_DSC1_PICTURE_PARAMETER_SET_9_PB 0x78394	10529	#define _ICL_DSC1_PICTURE_PARAMETER_SET_9_PB 0x78394
10496	#define _ICL_DSC0_PICTURE_PARAMETER_SET_9_PC 0x78494	10530	#define _ICL_DSC0_PICTURE_PARAMETER_SET_9_PC 0x78494
@@ -10504,8 +10538,8 @@ enum skl_power_gate { · 7 unmodified lines
10504	#define DSC_RC_EDGE_FACTOR(rc_edge_fact) ((rc_edge_fact) << 16)	10538	#define DSC_RC_EDGE_FACTOR(rc_edge_fact) ((rc_edge_fact) << 16)
10505	#define DSC_RC_MODEL_SIZE(rc_model_size) ((rc_model_size) << 0)	10539	#define DSC_RC_MODEL_SIZE(rc_model_size) ((rc_model_size) << 0)
10506		10540
10507	#define DSCA_PICTURE_PARAMETER_SET_10 0x6B228	10541	#define DSCA_PICTURE_PARAMETER_SET_10 _MMIO(0x6B228)
10508	#define DSCC_PICTURE_PARAMETER_SET_10 0x6BA28	10542	#define DSCC_PICTURE_PARAMETER_SET_10 _MMIO(0x6BA28)
10509	#define _ICL_DSC0_PICTURE_PARAMETER_SET_10_PB 0x78298	10543	#define _ICL_DSC0_PICTURE_PARAMETER_SET_10_PB 0x78298
10510	#define _ICL_DSC1_PICTURE_PARAMETER_SET_10_PB 0x78398	10544	#define _ICL_DSC1_PICTURE_PARAMETER_SET_10_PB 0x78398
10511	#define _ICL_DSC0_PICTURE_PARAMETER_SET_10_PC 0x78498	10545	#define _ICL_DSC0_PICTURE_PARAMETER_SET_10_PC 0x78498
@@ -10521,8 +10555,8 @@ enum skl_power_gate { · 9 unmodified lines
10521	#define DSC_RC_QUANT_INC_LIMIT1(lim) ((lim) << 8)	10555	#define DSC_RC_QUANT_INC_LIMIT1(lim) ((lim) << 8)
10522	#define DSC_RC_QUANT_INC_LIMIT0(lim) ((lim) << 0)	10556	#define DSC_RC_QUANT_INC_LIMIT0(lim) ((lim) << 0)
10523		10557
10524	#define DSCA_PICTURE_PARAMETER_SET_11 0x6B22C	10558	#define DSCA_PICTURE_PARAMETER_SET_11 _MMIO(0x6B22C)
10525	#define DSCC_PICTURE_PARAMETER_SET_11 0x6BA2C	10559	#define DSCC_PICTURE_PARAMETER_SET_11 _MMIO(0x6BA2C)
10526	#define _ICL_DSC0_PICTURE_PARAMETER_SET_11_PB 0x7829C	10560	#define _ICL_DSC0_PICTURE_PARAMETER_SET_11_PB 0x7829C
10527	#define _ICL_DSC1_PICTURE_PARAMETER_SET_11_PB 0x7839C	10561	#define _ICL_DSC1_PICTURE_PARAMETER_SET_11_PB 0x7839C
10528	#define _ICL_DSC0_PICTURE_PARAMETER_SET_11_PC 0x7849C	10562	#define _ICL_DSC0_PICTURE_PARAMETER_SET_11_PC 0x7849C
@@ -10534,8 +10568,8 @@ enum skl_power_gate { · 5 unmodified lines
10534	_ICL_DSC1_PICTURE_PARAMETER_SET_11_PB, \	10568	_ICL_DSC1_PICTURE_PARAMETER_SET_11_PB, \
10535	_ICL_DSC1_PICTURE_PARAMETER_SET_11_PC)	10569	_ICL_DSC1_PICTURE_PARAMETER_SET_11_PC)
10536		10570
10537	#define DSCA_PICTURE_PARAMETER_SET_12 0x6B260	10571	#define DSCA_PICTURE_PARAMETER_SET_12 _MMIO(0x6B260)
10538	#define DSCC_PICTURE_PARAMETER_SET_12 0x6BA60	10572	#define DSCC_PICTURE_PARAMETER_SET_12 _MMIO(0x6BA60)
10539	#define _ICL_DSC0_PICTURE_PARAMETER_SET_12_PB 0x782A0	10573	#define _ICL_DSC0_PICTURE_PARAMETER_SET_12_PB 0x782A0
10540	#define _ICL_DSC1_PICTURE_PARAMETER_SET_12_PB 0x783A0	10574	#define _ICL_DSC1_PICTURE_PARAMETER_SET_12_PB 0x783A0
10541	#define _ICL_DSC0_PICTURE_PARAMETER_SET_12_PC 0x784A0	10575	#define _ICL_DSC0_PICTURE_PARAMETER_SET_12_PC 0x784A0
@@ -10547,8 +10581,8 @@ enum skl_power_gate { · 5 unmodified lines
10547	_ICL_DSC1_PICTURE_PARAMETER_SET_12_PB, \	10581	_ICL_DSC1_PICTURE_PARAMETER_SET_12_PB, \
10548	_ICL_DSC1_PICTURE_PARAMETER_SET_12_PC)	10582	_ICL_DSC1_PICTURE_PARAMETER_SET_12_PC)
10549		10583
10550	#define DSCA_PICTURE_PARAMETER_SET_13 0x6B264	10584	#define DSCA_PICTURE_PARAMETER_SET_13 _MMIO(0x6B264)
10551	#define DSCC_PICTURE_PARAMETER_SET_13 0x6BA64	10585	#define DSCC_PICTURE_PARAMETER_SET_13 _MMIO(0x6BA64)
10552	#define _ICL_DSC0_PICTURE_PARAMETER_SET_13_PB 0x782A4	10586	#define _ICL_DSC0_PICTURE_PARAMETER_SET_13_PB 0x782A4
10553	#define _ICL_DSC1_PICTURE_PARAMETER_SET_13_PB 0x783A4	10587	#define _ICL_DSC1_PICTURE_PARAMETER_SET_13_PB 0x783A4
10554	#define _ICL_DSC0_PICTURE_PARAMETER_SET_13_PC 0x784A4	10588	#define _ICL_DSC0_PICTURE_PARAMETER_SET_13_PC 0x784A4
@@ -10560,8 +10594,8 @@ enum skl_power_gate { · 5 unmodified lines
10560	_ICL_DSC1_PICTURE_PARAMETER_SET_13_PB, \	10594	_ICL_DSC1_PICTURE_PARAMETER_SET_13_PB, \
10561	_ICL_DSC1_PICTURE_PARAMETER_SET_13_PC)	10595	_ICL_DSC1_PICTURE_PARAMETER_SET_13_PC)
10562		10596
10563	#define DSCA_PICTURE_PARAMETER_SET_14 0x6B268	10597	#define DSCA_PICTURE_PARAMETER_SET_14 _MMIO(0x6B268)
10564	#define DSCC_PICTURE_PARAMETER_SET_14 0x6BA68	10598	#define DSCC_PICTURE_PARAMETER_SET_14 _MMIO(0x6BA68)
10565	#define _ICL_DSC0_PICTURE_PARAMETER_SET_14_PB 0x782A8	10599	#define _ICL_DSC0_PICTURE_PARAMETER_SET_14_PB 0x782A8
10566	#define _ICL_DSC1_PICTURE_PARAMETER_SET_14_PB 0x783A8	10600	#define _ICL_DSC1_PICTURE_PARAMETER_SET_14_PB 0x783A8
10567	#define _ICL_DSC0_PICTURE_PARAMETER_SET_14_PC 0x784A8	10601	#define _ICL_DSC0_PICTURE_PARAMETER_SET_14_PC 0x784A8
@@ -10573,8 +10607,8 @@ enum skl_power_gate { · 5 unmodified lines
10573	_ICL_DSC1_PICTURE_PARAMETER_SET_14_PB, \	10607	_ICL_DSC1_PICTURE_PARAMETER_SET_14_PB, \
10574	_ICL_DSC1_PICTURE_PARAMETER_SET_14_PC)	10608	_ICL_DSC1_PICTURE_PARAMETER_SET_14_PC)
10575		10609
10576	#define DSCA_PICTURE_PARAMETER_SET_15 0x6B26C	10610	#define DSCA_PICTURE_PARAMETER_SET_15 _MMIO(0x6B26C)
10577	#define DSCC_PICTURE_PARAMETER_SET_15 0x6BA6C	10611	#define DSCC_PICTURE_PARAMETER_SET_15 _MMIO(0x6BA6C)
10578	#define _ICL_DSC0_PICTURE_PARAMETER_SET_15_PB 0x782AC	10612	#define _ICL_DSC0_PICTURE_PARAMETER_SET_15_PB 0x782AC
10579	#define _ICL_DSC1_PICTURE_PARAMETER_SET_15_PB 0x783AC	10613	#define _ICL_DSC1_PICTURE_PARAMETER_SET_15_PB 0x783AC
10580	#define _ICL_DSC0_PICTURE_PARAMETER_SET_15_PC 0x784AC	10614	#define _ICL_DSC0_PICTURE_PARAMETER_SET_15_PC 0x784AC
@@ -10586,8 +10620,8 @@ enum skl_power_gate { · 5 unmodified lines
10586	_ICL_DSC1_PICTURE_PARAMETER_SET_15_PB, \	10620	_ICL_DSC1_PICTURE_PARAMETER_SET_15_PB, \
10587	_ICL_DSC1_PICTURE_PARAMETER_SET_15_PC)	10621	_ICL_DSC1_PICTURE_PARAMETER_SET_15_PC)
10588		10622
10589	#define DSCA_PICTURE_PARAMETER_SET_16 0x6B270	10623	#define DSCA_PICTURE_PARAMETER_SET_16 _MMIO(0x6B270)
10590	#define DSCC_PICTURE_PARAMETER_SET_16 0x6BA70	10624	#define DSCC_PICTURE_PARAMETER_SET_16 _MMIO(0x6BA70)
10591	#define _ICL_DSC0_PICTURE_PARAMETER_SET_16_PB 0x782B0	10625	#define _ICL_DSC0_PICTURE_PARAMETER_SET_16_PB 0x782B0
10592	#define _ICL_DSC1_PICTURE_PARAMETER_SET_16_PB 0x783B0	10626	#define _ICL_DSC1_PICTURE_PARAMETER_SET_16_PB 0x783B0
10593	#define _ICL_DSC0_PICTURE_PARAMETER_SET_16_PC 0x784B0	10627	#define _ICL_DSC0_PICTURE_PARAMETER_SET_16_PC 0x784B0
@@ -10599,7 +10633,7 @@ enum skl_power_gate { · 5 unmodified lines
10599	_ICL_DSC1_PICTURE_PARAMETER_SET_16_PB, \	10633	_ICL_DSC1_PICTURE_PARAMETER_SET_16_PB, \
10600	_ICL_DSC1_PICTURE_PARAMETER_SET_16_PC)	10634	_ICL_DSC1_PICTURE_PARAMETER_SET_16_PC)
10601	#define DSC_SLICE_PER_LINE(slice_per_line) ((slice_per_line) << 16)	10635	#define DSC_SLICE_PER_LINE(slice_per_line) ((slice_per_line) << 16)
10602	#define DSC_SLICE_CHUNK_SIZE(slice_chunk_aize) (slice_chunk_size << 0)	10636	#define DSC_SLICE_CHUNK_SIZE(slice_chunk_size) ((slice_chunk_size) << 0)
10603		10637
10604	/* Icelake Rate Control Buffer Threshold Registers */	10638	/* Icelake Rate Control Buffer Threshold Registers */
10605	#define DSCA_RC_BUF_THRESH_0 _MMIO(0x6B230)	10639	#define DSCA_RC_BUF_THRESH_0 _MMIO(0x6B230)
@@ -10652,4 +10686,17 @@ enum skl_power_gate { · 46 unmodified lines
10652	_ICL_DSC1_RC_BUF_THRESH_1_UDW_PB, \	10686	_ICL_DSC1_RC_BUF_THRESH_1_UDW_PB, \
10653	_ICL_DSC1_RC_BUF_THRESH_1_UDW_PC)	10687	_ICL_DSC1_RC_BUF_THRESH_1_UDW_PC)
10654		10688
		10689	#define PORT_TX_DFLEXDPSP _MMIO(0x1638A0)
		10690	#define TC_LIVE_STATE_TBT(tc_port) (1 << ((tc_port) * 8 + 6))
		10691	#define TC_LIVE_STATE_TC(tc_port) (1 << ((tc_port) * 8 + 5))
		10692	#define DP_LANE_ASSIGNMENT_SHIFT(tc_port) ((tc_port) * 8)
		10693	#define DP_LANE_ASSIGNMENT_MASK(tc_port) (0xf << ((tc_port) * 8))
		10694	#define DP_LANE_ASSIGNMENT(tc_port, x) ((x) << ((tc_port) * 8))
		10695
		10696	#define PORT_TX_DFLEXDPPMS _MMIO(0x163890)
		10697	#define DP_PHY_MODE_STATUS_COMPLETED(tc_port) (1 << (tc_port))
		10698
		10699	#define PORT_TX_DFLEXDPCSSS _MMIO(0x163894)
		10700	#define DP_PHY_MODE_STATUS_NOT_SAFE(tc_port) (1 << (tc_port))
		10701
10655	#endif /* _I915_REG_H_ */	10702	#endif /* _I915_REG_H_ */

drivers/gpu/drm/i915/i915_request.c+4−5

 	seqno = timeline_get_seqno(&engine->timeline);
 	GEM_BUG_ON(!seqno);
-	GEM_BUG_ON(i915_seqno_passed(intel_engine_get_seqno(engine), seqno));
+	GEM_BUG_ON(intel_engine_signaled(engine, seqno));
 	/* We may be recursing from the signal callback of another i915 fence */
 	spin_lock_nested(&request->lock, SINGLE_DEPTH_NESTING);
 	 */
 	GEM_BUG_ON(!request->global_seqno);
 	GEM_BUG_ON(request->global_seqno != engine->timeline.seqno);
-	GEM_BUG_ON(i915_seqno_passed(intel_engine_get_seqno(engine),
+	GEM_BUG_ON(intel_engine_has_completed(engine, request->global_seqno));
-				     request->global_seqno));
 	engine->timeline.seqno--;
 	/* We may be recursing from the signal callback of another i915 fence */
 	 * it is a fair assumption that it will not complete within our
 	 * relatively short timeout.
 	 */
-	if (!i915_seqno_passed(intel_engine_get_seqno(engine), seqno - 1))
+	if (!intel_engine_has_started(engine, seqno))
 		return false;
 	/*
 	irq = READ_ONCE(engine->breadcrumbs.irq_count);
 	timeout_us += local_clock_us(&cpu);
 	do {
-		if (i915_seqno_passed(intel_engine_get_seqno(engine), seqno))
+		if (intel_engine_has_completed(engine, seqno))
 			return seqno == i915_request_global_seqno(rq);
 		/*

drivers/gpu/drm/i915/i915_request.h+25−14

 #define I915_WAIT_ALL		BIT(2) /* used by i915_gem_object_wait() */
 #define I915_WAIT_FOR_IDLE_BOOST BIT(3)
-static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine);
+static inline bool intel_engine_has_started(struct intel_engine_cs *engine,
+					    u32 seqno);
+static inline bool intel_engine_has_completed(struct intel_engine_cs *engine,
+					      u32 seqno);
 /**
  * Returns true if seq1 is later than seq2.
 	return (s32)(seq1 - seq2) >= 0;
 }
+/**
+ * i915_request_started - check if the request has begun being executed
+ * @rq: the request
+ *
+ * Returns true if the request has been submitted to hardware, and the hardware
+ * has advanced passed the end of the previous request and so should be either
+ * currently processing the request (though it may be preempted and so
+ * not necessarily the next request to complete) or have completed the request.
+ */
+static inline bool i915_request_started(const struct i915_request *rq)
+{
+	u32 seqno;
+	seqno = i915_request_global_seqno(rq);
+	if (!seqno) /* not yet submitted to HW */
+		return false;
+	return intel_engine_has_started(rq->engine, seqno);
+}
 static inline bool
 __i915_request_completed(const struct i915_request *rq, u32 seqno)
 {
 	GEM_BUG_ON(!seqno);
-	return i915_seqno_passed(intel_engine_get_seqno(rq->engine), seqno) &&
+	return intel_engine_has_completed(rq->engine, seqno) &&
 		seqno == i915_request_global_seqno(rq);
 }
 	return __i915_request_completed(rq, seqno);
 }
-static inline bool i915_request_started(const struct i915_request *rq)
-{
-	u32 seqno;
-	seqno = i915_request_global_seqno(rq);
-	if (!seqno)
-		return false;
-	return i915_seqno_passed(intel_engine_get_seqno(rq->engine),
-				 seqno - 1);
-}
 static inline bool i915_sched_node_signaled(const struct i915_sched_node *node)
 {
 	const struct i915_request *rq =

drivers/gpu/drm/i915/i915_vma.c+4−1

 	i915_vma_unpin(vma);
 }
-void i915_vma_unpin_and_release(struct i915_vma **p_vma)
+void i915_vma_unpin_and_release(struct i915_vma **p_vma, unsigned int flags)
 {
 	struct i915_vma *vma;
 	struct drm_i915_gem_object *obj;
 	i915_vma_unpin(vma);
 	i915_vma_close(vma);
+	if (flags & I915_VMA_RELEASE_MAP)
+		i915_gem_object_unpin_map(obj);
 	__i915_gem_object_release_unless_active(obj);
 }

drivers/gpu/drm/i915/i915_vma.h+9−1

 		  struct i915_address_space *vm,
 		  const struct i915_ggtt_view *view);
-void i915_vma_unpin_and_release(struct i915_vma **p_vma);
+void i915_vma_unpin_and_release(struct i915_vma **p_vma, unsigned int flags);
+#define I915_VMA_RELEASE_MAP BIT(0)
 static inline bool i915_vma_is_active(struct i915_vma *vma)
 {
 	return lower_32_bits(vma->node.start);
 }
+static inline u32 i915_ggtt_pin_bias(struct i915_vma *vma)
+{
+	return i915_vm_to_ggtt(vma->vm)->pin_bias;
+}
 static inline struct i915_vma *i915_vma_get(struct i915_vma *vma)
 {
 	i915_gem_object_get(vma->obj);
 	if (cmp)
 		return cmp;
+	assert_i915_gem_gtt_types();
 	/* ggtt_view.type also encodes its size so that we both distinguish
 	 * different views using it as a "type" and also use a compact (no
 	 * accessing of uninitialised padding bytes) memcmp without storing

drivers/gpu/drm/i915/intel_atomic_plane.c+4−2

 	}
 	intel_state->base.visible = false;
-	ret = intel_plane->check_plane(intel_plane, crtc_state, intel_state);
+	ret = intel_plane->check_plane(crtc_state, intel_state);
 	if (ret)
 		return ret;
 	if (state->fb && INTEL_GEN(dev_priv) >= 9 && crtc_state->base.enable &&
 	    adjusted_mode->flags & DRM_MODE_FLAG_INTERLACE) {
 		if (state->fb->modifier == I915_FORMAT_MOD_Y_TILED ||
-		    state->fb->modifier == I915_FORMAT_MOD_Yf_TILED) {
+		    state->fb->modifier == I915_FORMAT_MOD_Yf_TILED ||
+		    state->fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
+		    state->fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS) {
 			DRM_DEBUG_KMS("Y/Yf tiling not supported in IF-ID mode\n");
 			return -EINVAL;
 		}

drivers/gpu/drm/i915/intel_breadcrumbs.c+2−4

 	spin_unlock(&b->irq_lock);
 	rbtree_postorder_for_each_entry_safe(wait, n, &b->waiters, node) {
-		GEM_BUG_ON(!i915_seqno_passed(intel_engine_get_seqno(engine),
+		GEM_BUG_ON(!intel_engine_signaled(engine, wait->seqno));
-					      wait->seqno));
 		RB_CLEAR_NODE(&wait->node);
 		wake_up_process(wait->tsk);
 	}
 		return armed;
 	/* Make the caller recheck if its request has already started. */
-	return i915_seqno_passed(intel_engine_get_seqno(engine),
+	return intel_engine_has_started(engine, wait->seqno);
-				 wait->seqno - 1);
 }
 static inline bool chain_wakeup(struct rb_node *rb, int priority)

drivers/gpu/drm/i915/intel_csr.c+24−9

 #define BXT_CSR_VERSION_REQUIRED	CSR_VERSION(1, 7)
-#define CSR_MAX_FW_SIZE			0x2FFF
+#define BXT_CSR_MAX_FW_SIZE		0x3000
+#define GLK_CSR_MAX_FW_SIZE		0x4000
+#define ICL_CSR_MAX_FW_SIZE		0x6000
 #define CSR_DEFAULT_FW_OFFSET		0xFFFFFFFF
 struct intel_css_header {
 	struct intel_csr *csr = &dev_priv->csr;
 	const struct stepping_info *si = intel_get_stepping_info(dev_priv);
 	uint32_t dmc_offset = CSR_DEFAULT_FW_OFFSET, readcount = 0, nbytes;
+	uint32_t max_fw_size = 0;
 	uint32_t i;
 	uint32_t *dmc_payload;
 	uint32_t required_version;
 			  si->stepping);
 		return NULL;
 	}
+	/* Convert dmc_offset into number of bytes. By default it is in dwords*/
+	dmc_offset *= 4;
 	readcount += dmc_offset;
 	/* Extract dmc_header information. */
 	/* fw_size is in dwords, so multiplied by 4 to convert into bytes. */
 	nbytes = dmc_header->fw_size * 4;
-	if (nbytes > CSR_MAX_FW_SIZE) {
+	if (INTEL_GEN(dev_priv) >= 11)
-		DRM_ERROR("DMC firmware too big (%u bytes)\n", nbytes);
+		max_fw_size = ICL_CSR_MAX_FW_SIZE;
+	else if (IS_CANNONLAKE(dev_priv) || IS_GEMINILAKE(dev_priv))
+		max_fw_size = GLK_CSR_MAX_FW_SIZE;
+	else if (IS_GEN9(dev_priv))
+		max_fw_size = BXT_CSR_MAX_FW_SIZE;
+	else
+		MISSING_CASE(INTEL_REVID(dev_priv));
+	if (nbytes > max_fw_size) {
+		DRM_ERROR("DMC FW too big (%u bytes)\n", nbytes);
 		return NULL;
 	}
 	csr->dmc_fw_size = dmc_header->fw_size;
 		csr->fw_path = I915_CSR_SKL;
 	else if (IS_BROXTON(dev_priv))
 		csr->fw_path = I915_CSR_BXT;
-	else {
-		DRM_ERROR("Unexpected: no known CSR firmware for platform\n");
-		return;
-	}
-	DRM_DEBUG_KMS("Loading %s\n", csr->fw_path);
 	/*
 	 * Obtain a runtime pm reference, until CSR is loaded,
 	 */
 	intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
+	if (csr->fw_path == NULL) {
+		DRM_DEBUG_KMS("No known CSR firmware for platform, disabling runtime PM\n");
+		WARN_ON(!IS_ALPHA_SUPPORT(INTEL_INFO(dev_priv)));
+		return;
+	}
+	DRM_DEBUG_KMS("Loading %s\n", csr->fw_path);
 	schedule_work(&dev_priv->csr.work);
 }

drivers/gpu/drm/i915/intel_ddi.c+223−17

@@ -1414,7 +1414,7 @@ static int cnl_calc_wrpll_link(struct drm_i915_private *dev_priv, · 1413 unmodified lines
1414	break;	1414	break;
1415	}	1415	}
1416		1416
1417	ref_clock = dev_priv->cdclk.hw.ref;	1417	ref_clock = cnl_hdmi_pll_ref_clock(dev_priv);
1418		1418
1419	dco_freq = (cfgcr0 & DPLL_CFGCR0_DCO_INTEGER_MASK) * ref_clock;	1419	dco_freq = (cfgcr0 & DPLL_CFGCR0_DCO_INTEGER_MASK) * ref_clock;
1420		1420
@@ -1427,6 +1427,81 @@ static int cnl_calc_wrpll_link(struct drm_i915_private *dev_priv, · 6 unmodified lines
1427	return dco_freq / (p0 * p1 * p2 * 5);	1427	return dco_freq / (p0 * p1 * p2 * 5);
1428	}	1428	}
1429		1429
		1430	static int icl_calc_tbt_pll_link(struct drm_i915_private *dev_priv,
		1431	enum port port)
		1432	{
		1433	u32 val = I915_READ(DDI_CLK_SEL(port)) & DDI_CLK_SEL_MASK;
		1434
		1435	switch (val) {
		1436	case DDI_CLK_SEL_NONE:
		1437	return 0;
		1438	case DDI_CLK_SEL_TBT_162:
		1439	return 162000;
		1440	case DDI_CLK_SEL_TBT_270:
		1441	return 270000;
		1442	case DDI_CLK_SEL_TBT_540:
		1443	return 540000;
		1444	case DDI_CLK_SEL_TBT_810:
		1445	return 810000;
		1446	default:
		1447	MISSING_CASE(val);
		1448	return 0;
		1449	}
		1450	}
		1451
		1452	static int icl_calc_mg_pll_link(struct drm_i915_private *dev_priv,
		1453	enum port port)
		1454	{
		1455	u32 mg_pll_div0, mg_clktop_hsclkctl;
		1456	u32 m1, m2_int, m2_frac, div1, div2, refclk;
		1457	u64 tmp;
		1458
		1459	refclk = dev_priv->cdclk.hw.ref;
		1460
		1461	mg_pll_div0 = I915_READ(MG_PLL_DIV0(port));
		1462	mg_clktop_hsclkctl = I915_READ(MG_CLKTOP2_HSCLKCTL(port));
		1463
		1464	m1 = I915_READ(MG_PLL_DIV1(port)) & MG_PLL_DIV1_FBPREDIV_MASK;
		1465	m2_int = mg_pll_div0 & MG_PLL_DIV0_FBDIV_INT_MASK;
		1466	m2_frac = (mg_pll_div0 & MG_PLL_DIV0_FRACNEN_H) ?
		1467	(mg_pll_div0 & MG_PLL_DIV0_FBDIV_FRAC_MASK) >>
		1468	MG_PLL_DIV0_FBDIV_FRAC_SHIFT : 0;
		1469
		1470	switch (mg_clktop_hsclkctl & MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_MASK) {
		1471	case MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_2:
		1472	div1 = 2;
		1473	break;
		1474	case MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_3:
		1475	div1 = 3;
		1476	break;
		1477	case MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_5:
		1478	div1 = 5;
		1479	break;
		1480	case MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_7:
		1481	div1 = 7;
		1482	break;
		1483	default:
		1484	MISSING_CASE(mg_clktop_hsclkctl);
		1485	return 0;
		1486	}
		1487
		1488	div2 = (mg_clktop_hsclkctl & MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO_MASK) >>
		1489	MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO_SHIFT;
		1490	/* div2 value of 0 is same as 1 means no div */
		1491	if (div2 == 0)
		1492	div2 = 1;
		1493
		1494	/*
		1495	* Adjust the original formula to delay the division by 2^22 in order to
		1496	* minimize possible rounding errors.
		1497	*/
		1498	tmp = (u64)m1 * m2_int * refclk +
		1499	(((u64)m1 * m2_frac * refclk) >> 22);
		1500	tmp = div_u64(tmp, 5 * div1 * div2);
		1501
		1502	return tmp;
		1503	}
		1504
1430	static void ddi_dotclock_get(struct intel_crtc_state *pipe_config)	1505	static void ddi_dotclock_get(struct intel_crtc_state *pipe_config)
1431	{	1506	{
1432	int dotclock;	1507	int dotclock;
@@ -1467,8 +1542,10 @@ static void icl_ddi_clock_get(struct intel_encoder *encoder, · 34 unmodified lines
1467	link_clock = icl_calc_dp_combo_pll_link(dev_priv,	1542	link_clock = icl_calc_dp_combo_pll_link(dev_priv,
1468	pll_id);	1543	pll_id);
1469	} else {	1544	} else {
1470	/* FIXME - Add for MG PLL */	1545	if (pll_id == DPLL_ID_ICL_TBTPLL)
1471	WARN(1, "MG PLL clock_get code not implemented yet\n");	1546	link_clock = icl_calc_tbt_pll_link(dev_priv, port);
		1547	else
		1548	link_clock = icl_calc_mg_pll_link(dev_priv, port);
1472	}	1549	}
1473		1550
1474	pipe_config->port_clock = link_clock;	1551	pipe_config->port_clock = link_clock;
@@ -2468,7 +2545,128 @@ static void icl_combo_phy_ddi_vswing_sequence(struct intel_encoder *encoder, · 993 unmodified lines
2468	I915_WRITE(ICL_PORT_TX_DW5_GRP(port), val);	2545	I915_WRITE(ICL_PORT_TX_DW5_GRP(port), val);
2469	}	2546	}
2470		2547
2471	static void icl_ddi_vswing_sequence(struct intel_encoder *encoder, u32 level,	2548	static void icl_mg_phy_ddi_vswing_sequence(struct intel_encoder *encoder,
		2549	int link_clock,
		2550	u32 level)
		2551	{
		2552	struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
		2553	enum port port = encoder->port;
		2554	const struct icl_mg_phy_ddi_buf_trans *ddi_translations;
		2555	u32 n_entries, val;
		2556	int ln;
		2557
		2558	n_entries = ARRAY_SIZE(icl_mg_phy_ddi_translations);
		2559	ddi_translations = icl_mg_phy_ddi_translations;
		2560	/* The table does not have values for level 3 and level 9. */
		2561	if (level >= n_entries \|\| level == 3 \|\| level == 9) {
		2562	DRM_DEBUG_KMS("DDI translation not found for level %d. Using %d instead.",
		2563	level, n_entries - 2);
		2564	level = n_entries - 2;
		2565	}
		2566
		2567	/* Set MG_TX_LINK_PARAMS cri_use_fs32 to 0. */
		2568	for (ln = 0; ln < 2; ln++) {
		2569	val = I915_READ(MG_TX1_LINK_PARAMS(port, ln));
		2570	val &= ~CRI_USE_FS32;
		2571	I915_WRITE(MG_TX1_LINK_PARAMS(port, ln), val);
		2572
		2573	val = I915_READ(MG_TX2_LINK_PARAMS(port, ln));
		2574	val &= ~CRI_USE_FS32;
		2575	I915_WRITE(MG_TX2_LINK_PARAMS(port, ln), val);
		2576	}
		2577
		2578	/* Program MG_TX_SWINGCTRL with values from vswing table */
		2579	for (ln = 0; ln < 2; ln++) {
		2580	val = I915_READ(MG_TX1_SWINGCTRL(port, ln));
		2581	val &= ~CRI_TXDEEMPH_OVERRIDE_17_12_MASK;
		2582	val \|= CRI_TXDEEMPH_OVERRIDE_17_12(
		2583	ddi_translations[level].cri_txdeemph_override_17_12);
		2584	I915_WRITE(MG_TX1_SWINGCTRL(port, ln), val);
		2585
		2586	val = I915_READ(MG_TX2_SWINGCTRL(port, ln));
		2587	val &= ~CRI_TXDEEMPH_OVERRIDE_17_12_MASK;
		2588	val \|= CRI_TXDEEMPH_OVERRIDE_17_12(
		2589	ddi_translations[level].cri_txdeemph_override_17_12);
		2590	I915_WRITE(MG_TX2_SWINGCTRL(port, ln), val);
		2591	}
		2592
		2593	/* Program MG_TX_DRVCTRL with values from vswing table */
		2594	for (ln = 0; ln < 2; ln++) {
		2595	val = I915_READ(MG_TX1_DRVCTRL(port, ln));
		2596	val &= ~(CRI_TXDEEMPH_OVERRIDE_11_6_MASK \|
		2597	CRI_TXDEEMPH_OVERRIDE_5_0_MASK);
		2598	val \|= CRI_TXDEEMPH_OVERRIDE_5_0(
		2599	ddi_translations[level].cri_txdeemph_override_5_0) \|
		2600	CRI_TXDEEMPH_OVERRIDE_11_6(
		2601	ddi_translations[level].cri_txdeemph_override_11_6) \|
		2602	CRI_TXDEEMPH_OVERRIDE_EN;
		2603	I915_WRITE(MG_TX1_DRVCTRL(port, ln), val);
		2604
		2605	val = I915_READ(MG_TX2_DRVCTRL(port, ln));
		2606	val &= ~(CRI_TXDEEMPH_OVERRIDE_11_6_MASK \|
		2607	CRI_TXDEEMPH_OVERRIDE_5_0_MASK);
		2608	val \|= CRI_TXDEEMPH_OVERRIDE_5_0(
		2609	ddi_translations[level].cri_txdeemph_override_5_0) \|
		2610	CRI_TXDEEMPH_OVERRIDE_11_6(
		2611	ddi_translations[level].cri_txdeemph_override_11_6) \|
		2612	CRI_TXDEEMPH_OVERRIDE_EN;
		2613	I915_WRITE(MG_TX2_DRVCTRL(port, ln), val);
		2614
		2615	/* FIXME: Program CRI_LOADGEN_SEL after the spec is updated */
		2616	}
		2617
		2618	/*
		2619	* Program MG_CLKHUB<LN, port being used> with value from frequency table
		2620	* In case of Legacy mode on MG PHY, both TX1 and TX2 enabled so use the
		2621	* values from table for which TX1 and TX2 enabled.
		2622	*/
		2623	for (ln = 0; ln < 2; ln++) {
		2624	val = I915_READ(MG_CLKHUB(port, ln));
		2625	if (link_clock < 300000)
		2626	val \|= CFG_LOW_RATE_LKREN_EN;
		2627	else
		2628	val &= ~CFG_LOW_RATE_LKREN_EN;
		2629	I915_WRITE(MG_CLKHUB(port, ln), val);
		2630	}
		2631
		2632	/* Program the MG_TX_DCC<LN, port being used> based on the link frequency */
		2633	for (ln = 0; ln < 2; ln++) {
		2634	val = I915_READ(MG_TX1_DCC(port, ln));
		2635	val &= ~CFG_AMI_CK_DIV_OVERRIDE_VAL_MASK;
		2636	if (link_clock <= 500000) {
		2637	val &= ~CFG_AMI_CK_DIV_OVERRIDE_EN;
		2638	} else {
		2639	val \|= CFG_AMI_CK_DIV_OVERRIDE_EN \|
		2640	CFG_AMI_CK_DIV_OVERRIDE_VAL(1);
		2641	}
		2642	I915_WRITE(MG_TX1_DCC(port, ln), val);
		2643
		2644	val = I915_READ(MG_TX2_DCC(port, ln));
		2645	val &= ~CFG_AMI_CK_DIV_OVERRIDE_VAL_MASK;
		2646	if (link_clock <= 500000) {
		2647	val &= ~CFG_AMI_CK_DIV_OVERRIDE_EN;
		2648	} else {
		2649	val \|= CFG_AMI_CK_DIV_OVERRIDE_EN \|
		2650	CFG_AMI_CK_DIV_OVERRIDE_VAL(1);
		2651	}
		2652	I915_WRITE(MG_TX2_DCC(port, ln), val);
		2653	}
		2654
		2655	/* Program MG_TX_PISO_READLOAD with values from vswing table */
		2656	for (ln = 0; ln < 2; ln++) {
		2657	val = I915_READ(MG_TX1_PISO_READLOAD(port, ln));
		2658	val \|= CRI_CALCINIT;
		2659	I915_WRITE(MG_TX1_PISO_READLOAD(port, ln), val);
		2660
		2661	val = I915_READ(MG_TX2_PISO_READLOAD(port, ln));
		2662	val \|= CRI_CALCINIT;
		2663	I915_WRITE(MG_TX2_PISO_READLOAD(port, ln), val);
		2664	}
		2665	}
		2666
		2667	static void icl_ddi_vswing_sequence(struct intel_encoder *encoder,
		2668	int link_clock,
		2669	u32 level,
2472	enum intel_output_type type)	2670	enum intel_output_type type)
2473	{	2671	{
2474	enum port port = encoder->port;	2672	enum port port = encoder->port;
@@ -2476,8 +2674,7 @@ static void icl_ddi_vswing_sequence(struct intel_encoder *encoder, u32 level, · 1 unmodified line
2476	if (port == PORT_A \|\| port == PORT_B)	2674	if (port == PORT_A \|\| port == PORT_B)
2477	icl_combo_phy_ddi_vswing_sequence(encoder, level, type);	2675	icl_combo_phy_ddi_vswing_sequence(encoder, level, type);
2478	else	2676	else
2479	/* Not Implemented Yet */	2677	icl_mg_phy_ddi_vswing_sequence(encoder, link_clock, level);
2480	WARN_ON(1);
2481	}	2678	}
2482		2679
2483	static uint32_t translate_signal_level(int signal_levels)	2680	static uint32_t translate_signal_level(int signal_levels)
@@ -2512,7 +2709,8 @@ u32 bxt_signal_levels(struct intel_dp *intel_dp) · 28 unmodified lines
2512	int level = intel_ddi_dp_level(intel_dp);	2709	int level = intel_ddi_dp_level(intel_dp);
2513		2710
2514	if (IS_ICELAKE(dev_priv))	2711	if (IS_ICELAKE(dev_priv))
2515	icl_ddi_vswing_sequence(encoder, level, encoder->type);	2712	icl_ddi_vswing_sequence(encoder, intel_dp->link_rate,
		2713	level, encoder->type);
2516	else if (IS_CANNONLAKE(dev_priv))	2714	else if (IS_CANNONLAKE(dev_priv))
2517	cnl_ddi_vswing_sequence(encoder, level, encoder->type);	2715	cnl_ddi_vswing_sequence(encoder, level, encoder->type);
2518	else	2716	else
@@ -2692,8 +2890,12 @@ static void intel_ddi_pre_enable_dp(struct intel_encoder *encoder, · 173 unmodified lines
2692		2890
2693	intel_display_power_get(dev_priv, dig_port->ddi_io_power_domain);	2891	intel_display_power_get(dev_priv, dig_port->ddi_io_power_domain);
2694		2892
		2893	icl_program_mg_dp_mode(intel_dp);
		2894	icl_disable_phy_clock_gating(dig_port);
		2895
2695	if (IS_ICELAKE(dev_priv))	2896	if (IS_ICELAKE(dev_priv))
2696	icl_ddi_vswing_sequence(encoder, level, encoder->type);	2897	icl_ddi_vswing_sequence(encoder, crtc_state->port_clock,
		2898	level, encoder->type);
2697	else if (IS_CANNONLAKE(dev_priv))	2899	else if (IS_CANNONLAKE(dev_priv))
2698	cnl_ddi_vswing_sequence(encoder, level, encoder->type);	2900	cnl_ddi_vswing_sequence(encoder, level, encoder->type);
2699	else if (IS_GEN9_LP(dev_priv))	2901	else if (IS_GEN9_LP(dev_priv))
@@ -2708,7 +2910,10 @@ static void intel_ddi_pre_enable_dp(struct intel_encoder *encoder, · 8 unmodified lines
2708	if (port != PORT_A \|\| INTEL_GEN(dev_priv) >= 9)	2910	if (port != PORT_A \|\| INTEL_GEN(dev_priv) >= 9)
2709	intel_dp_stop_link_train(intel_dp);	2911	intel_dp_stop_link_train(intel_dp);
2710		2912
2711	intel_ddi_enable_pipe_clock(crtc_state);	2913	icl_enable_phy_clock_gating(dig_port);
		2914
		2915	if (!is_mst)
		2916	intel_ddi_enable_pipe_clock(crtc_state);
2712	}	2917	}
2713		2918
2714	static void intel_ddi_pre_enable_hdmi(struct intel_encoder *encoder,	2919	static void intel_ddi_pre_enable_hdmi(struct intel_encoder *encoder,
@@ -2728,7 +2933,8 @@ static void intel_ddi_pre_enable_hdmi(struct intel_encoder *encoder, · 13 unmodified lines
2728	intel_display_power_get(dev_priv, dig_port->ddi_io_power_domain);	2933	intel_display_power_get(dev_priv, dig_port->ddi_io_power_domain);
2729		2934
2730	if (IS_ICELAKE(dev_priv))	2935	if (IS_ICELAKE(dev_priv))
2731	icl_ddi_vswing_sequence(encoder, level, INTEL_OUTPUT_HDMI);	2936	icl_ddi_vswing_sequence(encoder, crtc_state->port_clock,
		2937	level, INTEL_OUTPUT_HDMI);
2732	else if (IS_CANNONLAKE(dev_priv))	2938	else if (IS_CANNONLAKE(dev_priv))
2733	cnl_ddi_vswing_sequence(encoder, level, INTEL_OUTPUT_HDMI);	2939	cnl_ddi_vswing_sequence(encoder, level, INTEL_OUTPUT_HDMI);
2734	else if (IS_GEN9_LP(dev_priv))	2940	else if (IS_GEN9_LP(dev_priv))
@@ -2810,14 +3016,14 @@ static void intel_ddi_post_disable_dp(struct intel_encoder *encoder, · 75 unmodified lines
2810	bool is_mst = intel_crtc_has_type(old_crtc_state,	3016	bool is_mst = intel_crtc_has_type(old_crtc_state,
2811	INTEL_OUTPUT_DP_MST);	3017	INTEL_OUTPUT_DP_MST);
2812		3018
2813	intel_ddi_disable_pipe_clock(old_crtc_state);	3019	if (!is_mst) {
2814		3020	intel_ddi_disable_pipe_clock(old_crtc_state);
2815	/*	3021	/*
2816	* Power down sink before disabling the port, otherwise we end	3022	* Power down sink before disabling the port, otherwise we end
2817	* up getting interrupts from the sink on detecting link loss.	3023	* up getting interrupts from the sink on detecting link loss.
2818	*/	3024	*/
2819	if (!is_mst)
2820	intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_OFF);	3025	intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_OFF);
		3026	}
2821		3027
2822	intel_disable_ddi_buf(encoder);	3028	intel_disable_ddi_buf(encoder);
2823		3029

drivers/gpu/drm/i915/intel_device_info.h+1−1

 	func(has_psr); \
 	func(has_rc6); \
 	func(has_rc6p); \
-	func(has_resource_streamer); \
 	func(has_runtime_pm); \
 	func(has_snoop); \
+	func(has_coherent_ggtt); \
 	func(unfenced_needs_alignment); \
 	func(cursor_needs_physical); \
 	func(hws_needs_physical); \

drivers/gpu/drm/i915/intel_display.c+115−57

@@ -2474,6 +2474,12 @@ intel_get_format_info(const struct drm_mode_fb_cmd2 *cmd) · 2473 unmodified lines
2474	}	2474	}
2475	}	2475	}
2476		2476
		2477	bool is_ccs_modifier(u64 modifier)
		2478	{
		2479	return modifier == I915_FORMAT_MOD_Y_TILED_CCS \|\|
		2480	modifier == I915_FORMAT_MOD_Yf_TILED_CCS;
		2481	}
		2482
2477	static int	2483	static int
2478	intel_fill_fb_info(struct drm_i915_private *dev_priv,	2484	intel_fill_fb_info(struct drm_i915_private *dev_priv,
2479	struct drm_framebuffer *fb)	2485	struct drm_framebuffer *fb)
@@ -2504,8 +2510,7 @@ intel_fill_fb_info(struct drm_i915_private *dev_priv, · 24 unmodified lines
2504	return ret;	2510	return ret;
2505	}	2511	}
2506		2512
2507	if ((fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS \|\|	2513	if (is_ccs_modifier(fb->modifier) && i == 1) {
2508	fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS) && i == 1) {
2509	int hsub = fb->format->hsub;	2514	int hsub = fb->format->hsub;
2510	int vsub = fb->format->vsub;	2515	int vsub = fb->format->vsub;
2511	int tile_width, tile_height;	2516	int tile_width, tile_height;
@@ -3055,8 +3060,7 @@ static int skl_check_main_surface(const struct intel_crtc_state *crtc_state, · 543 unmodified lines
3055	* CCS AUX surface doesn't have its own x/y offsets, we must make sure	3060	* CCS AUX surface doesn't have its own x/y offsets, we must make sure
3056	* they match with the main surface x/y offsets.	3061	* they match with the main surface x/y offsets.
3057	*/	3062	*/
3058	if (fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS \|\|	3063	if (is_ccs_modifier(fb->modifier)) {
3059	fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS) {
3060	while (!skl_check_main_ccs_coordinates(plane_state, x, y, offset)) {	3064	while (!skl_check_main_ccs_coordinates(plane_state, x, y, offset)) {
3061	if (offset == 0)	3065	if (offset == 0)
3062	break;	3066	break;
@@ -3190,8 +3194,7 @@ int skl_check_plane_surface(const struct intel_crtc_state *crtc_state, · 127 unmodified lines
3190	ret = skl_check_nv12_aux_surface(plane_state);	3194	ret = skl_check_nv12_aux_surface(plane_state);
3191	if (ret)	3195	if (ret)
3192	return ret;	3196	return ret;
3193	} else if (fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS \|\|	3197	} else if (is_ccs_modifier(fb->modifier)) {
3194	fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS) {
3195	ret = skl_check_ccs_aux_surface(plane_state);	3198	ret = skl_check_ccs_aux_surface(plane_state);
3196	if (ret)	3199	if (ret)
3197	return ret;	3200	return ret;
@@ -3552,11 +3555,11 @@ static u32 skl_plane_ctl_tiling(uint64_t fb_modifier) · 354 unmodified lines
3552	case I915_FORMAT_MOD_Y_TILED:	3555	case I915_FORMAT_MOD_Y_TILED:
3553	return PLANE_CTL_TILED_Y;	3556	return PLANE_CTL_TILED_Y;
3554	case I915_FORMAT_MOD_Y_TILED_CCS:	3557	case I915_FORMAT_MOD_Y_TILED_CCS:
3555	return PLANE_CTL_TILED_Y \| PLANE_CTL_DECOMPRESSION_ENABLE;	3558	return PLANE_CTL_TILED_Y \| PLANE_CTL_RENDER_DECOMPRESSION_ENABLE;
3556	case I915_FORMAT_MOD_Yf_TILED:	3559	case I915_FORMAT_MOD_Yf_TILED:
3557	return PLANE_CTL_TILED_YF;	3560	return PLANE_CTL_TILED_YF;
3558	case I915_FORMAT_MOD_Yf_TILED_CCS:	3561	case I915_FORMAT_MOD_Yf_TILED_CCS:
3559	return PLANE_CTL_TILED_YF \| PLANE_CTL_DECOMPRESSION_ENABLE;	3562	return PLANE_CTL_TILED_YF \| PLANE_CTL_RENDER_DECOMPRESSION_ENABLE;
3560	default:	3563	default:
3561	MISSING_CASE(fb_modifier);	3564	MISSING_CASE(fb_modifier);
3562	}	3565	}
@@ -5079,10 +5082,14 @@ void hsw_disable_ips(const struct intel_crtc_state *crtc_state) · 1516 unmodified lines
5079	mutex_lock(&dev_priv->pcu_lock);	5082	mutex_lock(&dev_priv->pcu_lock);
5080	WARN_ON(sandybridge_pcode_write(dev_priv, DISPLAY_IPS_CONTROL, 0));	5083	WARN_ON(sandybridge_pcode_write(dev_priv, DISPLAY_IPS_CONTROL, 0));
5081	mutex_unlock(&dev_priv->pcu_lock);	5084	mutex_unlock(&dev_priv->pcu_lock);
5082	/* wait for pcode to finish disabling IPS, which may take up to 42ms */	5085	/*
		5086	* Wait for PCODE to finish disabling IPS. The BSpec specified
		5087	* 42ms timeout value leads to occasional timeouts so use 100ms
		5088	* instead.
		5089	*/
5083	if (intel_wait_for_register(dev_priv,	5090	if (intel_wait_for_register(dev_priv,
5084	IPS_CTL, IPS_ENABLE, 0,	5091	IPS_CTL, IPS_ENABLE, 0,
5085	42))	5092	100))
5086	DRM_ERROR("Timed out waiting for IPS disable\n");	5093	DRM_ERROR("Timed out waiting for IPS disable\n");
5087	} else {	5094	} else {
5088	I915_WRITE(IPS_CTL, 0);	5095	I915_WRITE(IPS_CTL, 0);
@@ -8799,13 +8806,13 @@ skylake_get_initial_plane_config(struct intel_crtc *crtc, · 3710 unmodified lines
8799	fb->modifier = I915_FORMAT_MOD_X_TILED;	8806	fb->modifier = I915_FORMAT_MOD_X_TILED;
8800	break;	8807	break;
8801	case PLANE_CTL_TILED_Y:	8808	case PLANE_CTL_TILED_Y:
8802	if (val & PLANE_CTL_DECOMPRESSION_ENABLE)	8809	if (val & PLANE_CTL_RENDER_DECOMPRESSION_ENABLE)
8803	fb->modifier = I915_FORMAT_MOD_Y_TILED_CCS;	8810	fb->modifier = I915_FORMAT_MOD_Y_TILED_CCS;
8804	else	8811	else
8805	fb->modifier = I915_FORMAT_MOD_Y_TILED;	8812	fb->modifier = I915_FORMAT_MOD_Y_TILED;
8806	break;	8813	break;
8807	case PLANE_CTL_TILED_YF:	8814	case PLANE_CTL_TILED_YF:
8808	if (val & PLANE_CTL_DECOMPRESSION_ENABLE)	8815	if (val & PLANE_CTL_RENDER_DECOMPRESSION_ENABLE)
8809	fb->modifier = I915_FORMAT_MOD_Yf_TILED_CCS;	8816	fb->modifier = I915_FORMAT_MOD_Yf_TILED_CCS;
8810	else	8817	else
8811	fb->modifier = I915_FORMAT_MOD_Yf_TILED;	8818	fb->modifier = I915_FORMAT_MOD_Yf_TILED;
@@ -8974,7 +8981,7 @@ static void assert_can_disable_lcpll(struct drm_i915_private *dev_priv) · 162 unmodified lines
8974	I915_STATE_WARN(crtc->active, "CRTC for pipe %c enabled\n",	8981	I915_STATE_WARN(crtc->active, "CRTC for pipe %c enabled\n",
8975	pipe_name(crtc->pipe));	8982	pipe_name(crtc->pipe));
8976		8983
8977	I915_STATE_WARN(I915_READ(HSW_PWR_WELL_CTL_DRIVER(HSW_DISP_PW_GLOBAL)),	8984	I915_STATE_WARN(I915_READ(HSW_PWR_WELL_CTL2),
8978	"Display power well on\n");	8985	"Display power well on\n");
8979	I915_STATE_WARN(I915_READ(SPLL_CTL) & SPLL_PLL_ENABLE, "SPLL enabled\n");	8986	I915_STATE_WARN(I915_READ(SPLL_CTL) & SPLL_PLL_ENABLE, "SPLL enabled\n");
8980	I915_STATE_WARN(I915_READ(WRPLL_CTL(0)) & WRPLL_PLL_ENABLE, "WRPLL1 enabled\n");	8987	I915_STATE_WARN(I915_READ(WRPLL_CTL(0)) & WRPLL_PLL_ENABLE, "WRPLL1 enabled\n");
@@ -9691,8 +9698,7 @@ static bool i845_cursor_size_ok(const struct intel_plane_state *plane_state) · 710 unmodified lines
9691	return intel_cursor_size_ok(plane_state) && IS_ALIGNED(width, 64);	9698	return intel_cursor_size_ok(plane_state) && IS_ALIGNED(width, 64);
9692	}	9699	}
9693		9700
9694	static int i845_check_cursor(struct intel_plane *plane,	9701	static int i845_check_cursor(struct intel_crtc_state *crtc_state,
9695	struct intel_crtc_state *crtc_state,
9696	struct intel_plane_state *plane_state)	9702	struct intel_plane_state *plane_state)
9697	{	9703	{
9698	const struct drm_framebuffer *fb = plane_state->base.fb;	9704	const struct drm_framebuffer *fb = plane_state->base.fb;
@@ -9882,10 +9888,10 @@ static bool i9xx_cursor_size_ok(const struct intel_plane_state *plane_state) · 183 unmodified lines
9882	return true;	9888	return true;
9883	}	9889	}
9884		9890
9885	static int i9xx_check_cursor(struct intel_plane *plane,	9891	static int i9xx_check_cursor(struct intel_crtc_state *crtc_state,
9886	struct intel_crtc_state *crtc_state,
9887	struct intel_plane_state *plane_state)	9892	struct intel_plane_state *plane_state)
9888	{	9893	{
		9894	struct intel_plane *plane = to_intel_plane(plane_state->base.plane);
9889	struct drm_i915_private *dev_priv = to_i915(plane->base.dev);	9895	struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
9890	const struct drm_framebuffer *fb = plane_state->base.fb;	9896	const struct drm_framebuffer *fb = plane_state->base.fb;
9891	enum pipe pipe = plane->pipe;	9897	enum pipe pipe = plane->pipe;
@@ -12739,7 +12745,7 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state) · 2847 unmodified lines
12739	* down.	12745	* down.
12740	*/	12746	*/
12741	INIT_WORK(&state->commit_work, intel_atomic_cleanup_work);	12747	INIT_WORK(&state->commit_work, intel_atomic_cleanup_work);
12742	schedule_work(&state->commit_work);	12748	queue_work(system_highpri_wq, &state->commit_work);
12743	}	12749	}
12744		12750
12745	static void intel_atomic_commit_work(struct work_struct *work)	12751	static void intel_atomic_commit_work(struct work_struct *work)
@@ -12969,8 +12975,11 @@ static int intel_plane_pin_fb(struct intel_plane_state *plane_state) · 223 unmodified lines
12969	INTEL_INFO(dev_priv)->cursor_needs_physical) {	12975	INTEL_INFO(dev_priv)->cursor_needs_physical) {
12970	struct drm_i915_gem_object *obj = intel_fb_obj(fb);	12976	struct drm_i915_gem_object *obj = intel_fb_obj(fb);
12971	const int align = intel_cursor_alignment(dev_priv);	12977	const int align = intel_cursor_alignment(dev_priv);
		12978	int err;
12972		12979
12973	return i915_gem_object_attach_phys(obj, align);	12980	err = i915_gem_object_attach_phys(obj, align);
		12981	if (err)
		12982	return err;
12974	}	12983	}
12975		12984
12976	vma = intel_pin_and_fence_fb_obj(fb,	12985	vma = intel_pin_and_fence_fb_obj(fb,
@@ -13189,10 +13198,10 @@ skl_max_scale(struct intel_crtc *intel_crtc, · 212 unmodified lines
13189	}	13198	}
13190		13199
13191	static int	13200	static int
13192	intel_check_primary_plane(struct intel_plane *plane,	13201	intel_check_primary_plane(struct intel_crtc_state *crtc_state,
13193	struct intel_crtc_state *crtc_state,
13194	struct intel_plane_state *state)	13202	struct intel_plane_state *state)
13195	{	13203	{
		13204	struct intel_plane *plane = to_intel_plane(state->base.plane);
13196	struct drm_i915_private *dev_priv = to_i915(plane->base.dev);	13205	struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
13197	struct drm_crtc *crtc = state->base.crtc;	13206	struct drm_crtc *crtc = state->base.crtc;
13198	int min_scale = DRM_PLANE_HELPER_NO_SCALING;	13207	int min_scale = DRM_PLANE_HELPER_NO_SCALING;
@@ -13400,8 +13409,7 @@ static bool skl_plane_format_mod_supported(struct drm_plane *_plane, · 201 unmodified lines
13400	case DRM_FORMAT_XBGR8888:	13409	case DRM_FORMAT_XBGR8888:
13401	case DRM_FORMAT_ARGB8888:	13410	case DRM_FORMAT_ARGB8888:
13402	case DRM_FORMAT_ABGR8888:	13411	case DRM_FORMAT_ABGR8888:
13403	if (modifier == I915_FORMAT_MOD_Yf_TILED_CCS \|\|	13412	if (is_ccs_modifier(modifier))
13404	modifier == I915_FORMAT_MOD_Y_TILED_CCS)
13405	return true;	13413	return true;
13406	/* fall through */	13414	/* fall through */
13407	case DRM_FORMAT_RGB565:	13415	case DRM_FORMAT_RGB565:
@@ -13620,24 +13628,22 @@ static bool skl_plane_has_fbc(struct drm_i915_private *dev_priv, · 212 unmodified lines
13620	bool skl_plane_has_planar(struct drm_i915_private *dev_priv,	13628	bool skl_plane_has_planar(struct drm_i915_private *dev_priv,
13621	enum pipe pipe, enum plane_id plane_id)	13629	enum pipe pipe, enum plane_id plane_id)
13622	{	13630	{
13623	if (plane_id == PLANE_PRIMARY) {	13631	/*
13624	if (IS_SKYLAKE(dev_priv) \|\| IS_BROXTON(dev_priv))	13632	* FIXME: ICL requires two hardware planes for scanning out NV12
13625	return false;	13633	* framebuffers. Do not advertize support until this is implemented.
13626	else if ((INTEL_GEN(dev_priv) == 9 && pipe == PIPE_C) &&	13634	*/
13627	!IS_GEMINILAKE(dev_priv))	13635	if (INTEL_GEN(dev_priv) >= 11)
13628	return false;	13636	return false;
13629	} else if (plane_id >= PLANE_SPRITE0) {	13637
13630	if (plane_id == PLANE_CURSOR)	13638	if (IS_SKYLAKE(dev_priv) \|\| IS_BROXTON(dev_priv))
13631	return false;	13639	return false;
13632	if (IS_GEMINILAKE(dev_priv) \|\| INTEL_GEN(dev_priv) == 10) {	13640
13633	if (plane_id != PLANE_SPRITE0)	13641	if (INTEL_GEN(dev_priv) == 9 && !IS_GEMINILAKE(dev_priv) && pipe == PIPE_C)
13634	return false;	13642	return false;
13635	} else {	13643
13636	if (plane_id != PLANE_SPRITE0 \|\| pipe == PIPE_C \|\|	13644	if (plane_id != PLANE_PRIMARY && plane_id != PLANE_SPRITE0)
13637	IS_SKYLAKE(dev_priv) \|\| IS_BROXTON(dev_priv))	13645	return false;
13638	return false;	13646
13639	}
13640	}
13641	return true;	13647	return true;
13642	}	13648	}
13643		13649
@@ -14131,6 +14137,9 @@ static void intel_setup_outputs(struct drm_i915_private *dev_priv) · 487 unmodified lines
14131		14137
14132	intel_pps_init(dev_priv);	14138	intel_pps_init(dev_priv);
14133		14139
		14140	if (INTEL_INFO(dev_priv)->num_pipes == 0)
		14141	return;
		14142
14134	/*	14143	/*
14135	* intel_edp_init_connector() depends on this completing first, to	14144	* intel_edp_init_connector() depends on this completing first, to
14136	* prevent the registeration of both eDP and LVDS and the incorrect	14145	* prevent the registeration of both eDP and LVDS and the incorrect
@@ -14547,7 +14556,7 @@ static int intel_framebuffer_init(struct intel_framebuffer *intel_fb, · 410 unmodified lines
14547	break;	14556	break;
14548	case DRM_FORMAT_NV12:	14557	case DRM_FORMAT_NV12:
14549	if (INTEL_GEN(dev_priv) < 9 \|\| IS_SKYLAKE(dev_priv) \|\|	14558	if (INTEL_GEN(dev_priv) < 9 \|\| IS_SKYLAKE(dev_priv) \|\|
14550	IS_BROXTON(dev_priv)) {	14559	IS_BROXTON(dev_priv) \|\| INTEL_GEN(dev_priv) >= 11) {
14551	DRM_DEBUG_KMS("unsupported pixel format: %s\n",	14560	DRM_DEBUG_KMS("unsupported pixel format: %s\n",
14552	drm_get_format_name(mode_cmd->pixel_format,	14561	drm_get_format_name(mode_cmd->pixel_format,
14553	&format_name));	14562	&format_name));
@@ -14594,8 +14603,7 @@ static int intel_framebuffer_init(struct intel_framebuffer *intel_fb, · 40 unmodified lines
14594	* potential runtime errors at plane configuration time.	14603	* potential runtime errors at plane configuration time.
14595	*/	14604	*/
14596	if (IS_GEN9(dev_priv) && i == 0 && fb->width > 3840 &&	14605	if (IS_GEN9(dev_priv) && i == 0 && fb->width > 3840 &&
14597	(fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS \|\|	14606	is_ccs_modifier(fb->modifier))
14598	fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS))
14599	stride_alignment *= 4;	14607	stride_alignment *= 4;
14600		14608
14601	if (fb->pitches[i] & (stride_alignment - 1)) {	14609	if (fb->pitches[i] & (stride_alignment - 1)) {
@@ -15131,12 +15139,61 @@ static void intel_update_fdi_pll_freq(struct drm_i915_private *dev_priv) · 529 unmodified lines
15131	DRM_DEBUG_DRIVER("FDI PLL freq=%d\n", dev_priv->fdi_pll_freq);	15139	DRM_DEBUG_DRIVER("FDI PLL freq=%d\n", dev_priv->fdi_pll_freq);
15132	}	15140	}
15133		15141
		15142	static int intel_initial_commit(struct drm_device *dev)
		15143	{
		15144	struct drm_atomic_state *state = NULL;
		15145	struct drm_modeset_acquire_ctx ctx;
		15146	struct drm_crtc *crtc;
		15147	struct drm_crtc_state *crtc_state;
		15148	int ret = 0;
		15149
		15150	state = drm_atomic_state_alloc(dev);
		15151	if (!state)
		15152	return -ENOMEM;
		15153
		15154	drm_modeset_acquire_init(&ctx, 0);
		15155
		15156	retry:
		15157	state->acquire_ctx = &ctx;
		15158
		15159	drm_for_each_crtc(crtc, dev) {
		15160	crtc_state = drm_atomic_get_crtc_state(state, crtc);
		15161	if (IS_ERR(crtc_state)) {
		15162	ret = PTR_ERR(crtc_state);
		15163	goto out;
		15164	}
		15165
		15166	if (crtc_state->active) {
		15167	ret = drm_atomic_add_affected_planes(state, crtc);
		15168	if (ret)
		15169	goto out;
		15170	}
		15171	}
		15172
		15173	ret = drm_atomic_commit(state);
		15174
		15175	out:
		15176	if (ret == -EDEADLK) {
		15177	drm_atomic_state_clear(state);
		15178	drm_modeset_backoff(&ctx);
		15179	goto retry;
		15180	}
		15181
		15182	drm_atomic_state_put(state);
		15183
		15184	drm_modeset_drop_locks(&ctx);
		15185	drm_modeset_acquire_fini(&ctx);
		15186
		15187	return ret;
		15188	}
		15189
15134	int intel_modeset_init(struct drm_device *dev)	15190	int intel_modeset_init(struct drm_device *dev)
15135	{	15191	{
15136	struct drm_i915_private *dev_priv = to_i915(dev);	15192	struct drm_i915_private *dev_priv = to_i915(dev);
15137	struct i915_ggtt *ggtt = &dev_priv->ggtt;	15193	struct i915_ggtt *ggtt = &dev_priv->ggtt;
15138	enum pipe pipe;	15194	enum pipe pipe;
15139	struct intel_crtc *crtc;	15195	struct intel_crtc *crtc;
		15196	int ret;
15140		15197
15141	dev_priv->modeset_wq = alloc_ordered_workqueue("i915_modeset", 0);	15198	dev_priv->modeset_wq = alloc_ordered_workqueue("i915_modeset", 0);
15142		15199
@@ -15160,9 +15217,6 @@ int intel_modeset_init(struct drm_device *dev) · 17 unmodified lines
15160		15217
15161	intel_init_pm(dev_priv);	15218	intel_init_pm(dev_priv);
15162		15219
15163	if (INTEL_INFO(dev_priv)->num_pipes == 0)
15164	return 0;
15165
15166	/*	15220	/*
15167	* There may be no VBT; and if the BIOS enabled SSC we can	15221	* There may be no VBT; and if the BIOS enabled SSC we can
15168	* just keep using it to avoid unnecessary flicker. Whereas if the	15222	* just keep using it to avoid unnecessary flicker. Whereas if the
@@ -15211,8 +15265,6 @@ int intel_modeset_init(struct drm_device *dev) · 42 unmodified lines
15211	INTEL_INFO(dev_priv)->num_pipes > 1 ? "s" : "");	15265	INTEL_INFO(dev_priv)->num_pipes > 1 ? "s" : "");
15212		15266
15213	for_each_pipe(dev_priv, pipe) {	15267	for_each_pipe(dev_priv, pipe) {
15214	int ret;
15215
15216	ret = intel_crtc_init(dev_priv, pipe);	15268	ret = intel_crtc_init(dev_priv, pipe);
15217	if (ret) {	15269	if (ret) {
15218	drm_mode_config_cleanup(dev);	15270	drm_mode_config_cleanup(dev);
@@ -15268,6 +15320,16 @@ int intel_modeset_init(struct drm_device *dev) · 49 unmodified lines
15268	if (!HAS_GMCH_DISPLAY(dev_priv))	15320	if (!HAS_GMCH_DISPLAY(dev_priv))
15269	sanitize_watermarks(dev);	15321	sanitize_watermarks(dev);
15270		15322
		15323	/*
		15324	* Force all active planes to recompute their states. So that on
		15325	* mode_setcrtc after probe, all the intel_plane_state variables
		15326	* are already calculated and there is no assert_plane warnings
		15327	* during bootup.
		15328	*/
		15329	ret = intel_initial_commit(dev);
		15330	if (ret)
		15331	DRM_DEBUG_KMS("Initial commit in probe failed.\n");
		15332
15271	return 0;	15333	return 0;
15272	}	15334	}
15273		15335
@@ -15792,6 +15854,8 @@ intel_modeset_setup_hw_state(struct drm_device *dev, · 518 unmodified lines
15792	struct intel_encoder *encoder;	15854	struct intel_encoder *encoder;
15793	int i;	15855	int i;
15794		15856
		15857	intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
		15858
15795	intel_early_display_was(dev_priv);	15859	intel_early_display_was(dev_priv);
15796	intel_modeset_readout_hw_state(dev);	15860	intel_modeset_readout_hw_state(dev);
15797		15861
@@ -15846,9 +15910,8 @@ intel_modeset_setup_hw_state(struct drm_device *dev, · 48 unmodified lines
15846	if (WARN_ON(put_domains))	15910	if (WARN_ON(put_domains))
15847	modeset_put_power_domains(dev_priv, put_domains);	15911	modeset_put_power_domains(dev_priv, put_domains);
15848	}	15912	}
15849	intel_display_set_init_power(dev_priv, false);
15850		15913
15851	intel_power_domains_verify_state(dev_priv);	15914	intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
15852		15915
15853	intel_fbc_init_pipe_state(dev_priv);	15916	intel_fbc_init_pipe_state(dev_priv);
15854	}	15917	}
@@ -15937,8 +16000,6 @@ void intel_modeset_cleanup(struct drm_device *dev) · 82 unmodified lines
15937	flush_work(&dev_priv->atomic_helper.free_work);	16000	flush_work(&dev_priv->atomic_helper.free_work);
15938	WARN_ON(!llist_empty(&dev_priv->atomic_helper.free_list));	16001	WARN_ON(!llist_empty(&dev_priv->atomic_helper.free_list));
15939		16002
15940	intel_disable_gt_powersave(dev_priv);
15941
15942	/*	16003	/*
15943	* Interrupts and polling as the first thing to avoid creating havoc.	16004	* Interrupts and polling as the first thing to avoid creating havoc.
15944	* Too much stuff here (turning of connectors, ...) would	16005	* Too much stuff here (turning of connectors, ...) would
@@ -15966,8 +16027,6 @@ void intel_modeset_cleanup(struct drm_device *dev) · 21 unmodified lines
15966		16027
15967	intel_cleanup_overlay(dev_priv);	16028	intel_cleanup_overlay(dev_priv);
15968		16029
15969	intel_cleanup_gt_powersave(dev_priv);
15970
15971	intel_teardown_gmbus(dev_priv);	16030	intel_teardown_gmbus(dev_priv);
15972		16031
15973	destroy_workqueue(dev_priv->modeset_wq);	16032	destroy_workqueue(dev_priv->modeset_wq);
@@ -16075,8 +16134,7 @@ intel_display_capture_error_state(struct drm_i915_private *dev_priv) · 101 unmodified lines
16075	return NULL;	16134	return NULL;
16076		16135
16077	if (IS_HASWELL(dev_priv) \|\| IS_BROADWELL(dev_priv))	16136	if (IS_HASWELL(dev_priv) \|\| IS_BROADWELL(dev_priv))
16078	error->power_well_driver =	16137	error->power_well_driver = I915_READ(HSW_PWR_WELL_CTL2);
16079	I915_READ(HSW_PWR_WELL_CTL_DRIVER(HSW_DISP_PW_GLOBAL));
16080		16138
16081	for_each_pipe(dev_priv, i) {	16139	for_each_pipe(dev_priv, i) {
16082	error->pipe[i].power_domain_on =	16140	error->pipe[i].power_domain_on =

drivers/gpu/drm/i915/intel_display.h+26−2

 #ifndef _INTEL_DISPLAY_H_
 #define _INTEL_DISPLAY_H_
+enum i915_gpio {
+	GPIOA,
+	GPIOB,
+	GPIOC,
+	GPIOD,
+	GPIOE,
+	GPIOF,
+	GPIOG,
+	GPIOH,
+	__GPIOI_UNUSED,
+	GPIOJ,
+	GPIOK,
+	GPIOL,
+	GPIOM,
+};
 enum pipe {
 	INVALID_PIPE = -1,
 	I915_MAX_TC_PORTS
 };
+enum tc_port_type {
+	TC_PORT_UNKNOWN = 0,
+	TC_PORT_TYPEC,
+	TC_PORT_TBT,
+	TC_PORT_LEGACY,
+};
 enum dpio_channel {
 	DPIO_CH0,
 	DPIO_CH1
 #define for_each_power_domain_well(__dev_priv, __power_well, __domain_mask)	\
 	for_each_power_well(__dev_priv, __power_well)				\
-		for_each_if((__power_well)->domains & (__domain_mask))
+		for_each_if((__power_well)->desc->domains & (__domain_mask))
 #define for_each_power_domain_well_rev(__dev_priv, __power_well, __domain_mask) \
 	for_each_power_well_rev(__dev_priv, __power_well)		        \
-		for_each_if((__power_well)->domains & (__domain_mask))
+		for_each_if((__power_well)->desc->domains & (__domain_mask))
 #define for_each_new_intel_plane_in_state(__state, plane, new_plane_state, __i) \
 	for ((__i) = 0; \
 			    struct intel_link_m_n *m_n,
 			    bool reduce_m_n);
+bool is_ccs_modifier(u64 modifier);
 #endif

drivers/gpu/drm/i915/intel_dp.c+451−83

@@ -107,13 +107,6 @@ bool intel_dp_is_edp(struct intel_dp *intel_dp) · 106 unmodified lines
107	return intel_dig_port->base.type == INTEL_OUTPUT_EDP;	107	return intel_dig_port->base.type == INTEL_OUTPUT_EDP;
108	}	108	}
109		109
110	static struct drm_device intel_dp_to_dev(struct intel_dp intel_dp)
111	{
112	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
113
114	return intel_dig_port->base.base.dev;
115	}
116
117	static struct intel_dp intel_attached_dp(struct drm_connector connector)	110	static struct intel_dp intel_attached_dp(struct drm_connector connector)
118	{	111	{
119	return enc_to_intel_dp(&intel_attached_encoder(connector)->base);	112	return enc_to_intel_dp(&intel_attached_encoder(connector)->base);
@@ -176,14 +169,45 @@ static int intel_dp_max_common_rate(struct intel_dp *intel_dp) · 56 unmodified lines
176	return intel_dp->common_rates[intel_dp->num_common_rates - 1];	169	return intel_dp->common_rates[intel_dp->num_common_rates - 1];
177	}	170	}
178		171
		172	static int intel_dp_get_fia_supported_lane_count(struct intel_dp *intel_dp)
		173	{
		174	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
		175	struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
		176	enum tc_port tc_port = intel_port_to_tc(dev_priv, dig_port->base.port);
		177	u32 lane_info;
		178
		179	if (tc_port == PORT_TC_NONE \|\| dig_port->tc_type != TC_PORT_TYPEC)
		180	return 4;
		181
		182	lane_info = (I915_READ(PORT_TX_DFLEXDPSP) &
		183	DP_LANE_ASSIGNMENT_MASK(tc_port)) >>
		184	DP_LANE_ASSIGNMENT_SHIFT(tc_port);
		185
		186	switch (lane_info) {
		187	default:
		188	MISSING_CASE(lane_info);
		189	case 1:
		190	case 2:
		191	case 4:
		192	case 8:
		193	return 1;
		194	case 3:
		195	case 12:
		196	return 2;
		197	case 15:
		198	return 4;
		199	}
		200	}
		201
179	/* Theoretical max between source and sink */	202	/* Theoretical max between source and sink */
180	static int intel_dp_max_common_lane_count(struct intel_dp *intel_dp)	203	static int intel_dp_max_common_lane_count(struct intel_dp *intel_dp)
181	{	204	{
182	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	205	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
183	int source_max = intel_dig_port->max_lanes;	206	int source_max = intel_dig_port->max_lanes;
184	int sink_max = drm_dp_max_lane_count(intel_dp->dpcd);	207	int sink_max = drm_dp_max_lane_count(intel_dp->dpcd);
		208	int fia_max = intel_dp_get_fia_supported_lane_count(intel_dp);
185		209
186	return min(source_max, sink_max);	210	return min3(source_max, sink_max, fia_max);
187	}	211	}
188		212
189	int intel_dp_max_lane_count(struct intel_dp *intel_dp)	213	int intel_dp_max_lane_count(struct intel_dp *intel_dp)
@@ -198,6 +222,138 @@ intel_dp_link_required(int pixel_clock, int bpp) · 8 unmodified lines
198	return DIV_ROUND_UP(pixel_clock * bpp, 8);	222	return DIV_ROUND_UP(pixel_clock * bpp, 8);
199	}	223	}
200		224
		225	void icl_program_mg_dp_mode(struct intel_dp *intel_dp)
		226	{
		227	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
		228	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
		229	enum port port = intel_dig_port->base.port;
		230	enum tc_port tc_port = intel_port_to_tc(dev_priv, port);
		231	u32 ln0, ln1, lane_info;
		232
		233	if (tc_port == PORT_TC_NONE \|\| intel_dig_port->tc_type == TC_PORT_TBT)
		234	return;
		235
		236	ln0 = I915_READ(MG_DP_MODE(port, 0));
		237	ln1 = I915_READ(MG_DP_MODE(port, 1));
		238
		239	switch (intel_dig_port->tc_type) {
		240	case TC_PORT_TYPEC:
		241	ln0 &= ~(MG_DP_MODE_CFG_DP_X1_MODE \| MG_DP_MODE_CFG_DP_X2_MODE);
		242	ln1 &= ~(MG_DP_MODE_CFG_DP_X1_MODE \| MG_DP_MODE_CFG_DP_X2_MODE);
		243
		244	lane_info = (I915_READ(PORT_TX_DFLEXDPSP) &
		245	DP_LANE_ASSIGNMENT_MASK(tc_port)) >>
		246	DP_LANE_ASSIGNMENT_SHIFT(tc_port);
		247
		248	switch (lane_info) {
		249	case 0x1:
		250	case 0x4:
		251	break;
		252	case 0x2:
		253	ln0 \|= MG_DP_MODE_CFG_DP_X1_MODE;
		254	break;
		255	case 0x3:
		256	ln0 \|= MG_DP_MODE_CFG_DP_X1_MODE \|
		257	MG_DP_MODE_CFG_DP_X2_MODE;
		258	break;
		259	case 0x8:
		260	ln1 \|= MG_DP_MODE_CFG_DP_X1_MODE;
		261	break;
		262	case 0xC:
		263	ln1 \|= MG_DP_MODE_CFG_DP_X1_MODE \|
		264	MG_DP_MODE_CFG_DP_X2_MODE;
		265	break;
		266	case 0xF:
		267	ln0 \|= MG_DP_MODE_CFG_DP_X1_MODE \|
		268	MG_DP_MODE_CFG_DP_X2_MODE;
		269	ln1 \|= MG_DP_MODE_CFG_DP_X1_MODE \|
		270	MG_DP_MODE_CFG_DP_X2_MODE;
		271	break;
		272	default:
		273	MISSING_CASE(lane_info);
		274	}
		275	break;
		276
		277	case TC_PORT_LEGACY:
		278	ln0 \|= MG_DP_MODE_CFG_DP_X1_MODE \| MG_DP_MODE_CFG_DP_X2_MODE;
		279	ln1 \|= MG_DP_MODE_CFG_DP_X1_MODE \| MG_DP_MODE_CFG_DP_X2_MODE;
		280	break;
		281
		282	default:
		283	MISSING_CASE(intel_dig_port->tc_type);
		284	return;
		285	}
		286
		287	I915_WRITE(MG_DP_MODE(port, 0), ln0);
		288	I915_WRITE(MG_DP_MODE(port, 1), ln1);
		289	}
		290
		291	void icl_enable_phy_clock_gating(struct intel_digital_port *dig_port)
		292	{
		293	struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
		294	enum port port = dig_port->base.port;
		295	enum tc_port tc_port = intel_port_to_tc(dev_priv, port);
		296	i915_reg_t mg_regs[2] = { MG_DP_MODE(port, 0), MG_DP_MODE(port, 1) };
		297	u32 val;
		298	int i;
		299
		300	if (tc_port == PORT_TC_NONE)
		301	return;
		302
		303	for (i = 0; i < ARRAY_SIZE(mg_regs); i++) {
		304	val = I915_READ(mg_regs[i]);
		305	val \|= MG_DP_MODE_CFG_TR2PWR_GATING \|
		306	MG_DP_MODE_CFG_TRPWR_GATING \|
		307	MG_DP_MODE_CFG_CLNPWR_GATING \|
		308	MG_DP_MODE_CFG_DIGPWR_GATING \|
		309	MG_DP_MODE_CFG_GAONPWR_GATING;
		310	I915_WRITE(mg_regs[i], val);
		311	}
		312
		313	val = I915_READ(MG_MISC_SUS0(tc_port));
		314	val \|= MG_MISC_SUS0_SUSCLK_DYNCLKGATE_MODE(3) \|
		315	MG_MISC_SUS0_CFG_TR2PWR_GATING \|
		316	MG_MISC_SUS0_CFG_CL2PWR_GATING \|
		317	MG_MISC_SUS0_CFG_GAONPWR_GATING \|
		318	MG_MISC_SUS0_CFG_TRPWR_GATING \|
		319	MG_MISC_SUS0_CFG_CL1PWR_GATING \|
		320	MG_MISC_SUS0_CFG_DGPWR_GATING;
		321	I915_WRITE(MG_MISC_SUS0(tc_port), val);
		322	}
		323
		324	void icl_disable_phy_clock_gating(struct intel_digital_port *dig_port)
		325	{
		326	struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
		327	enum port port = dig_port->base.port;
		328	enum tc_port tc_port = intel_port_to_tc(dev_priv, port);
		329	i915_reg_t mg_regs[2] = { MG_DP_MODE(port, 0), MG_DP_MODE(port, 1) };
		330	u32 val;
		331	int i;
		332
		333	if (tc_port == PORT_TC_NONE)
		334	return;
		335
		336	for (i = 0; i < ARRAY_SIZE(mg_regs); i++) {
		337	val = I915_READ(mg_regs[i]);
		338	val &= ~(MG_DP_MODE_CFG_TR2PWR_GATING \|
		339	MG_DP_MODE_CFG_TRPWR_GATING \|
		340	MG_DP_MODE_CFG_CLNPWR_GATING \|
		341	MG_DP_MODE_CFG_DIGPWR_GATING \|
		342	MG_DP_MODE_CFG_GAONPWR_GATING);
		343	I915_WRITE(mg_regs[i], val);
		344	}
		345
		346	val = I915_READ(MG_MISC_SUS0(tc_port));
		347	val &= ~(MG_MISC_SUS0_SUSCLK_DYNCLKGATE_MODE_MASK \|
		348	MG_MISC_SUS0_CFG_TR2PWR_GATING \|
		349	MG_MISC_SUS0_CFG_CL2PWR_GATING \|
		350	MG_MISC_SUS0_CFG_GAONPWR_GATING \|
		351	MG_MISC_SUS0_CFG_TRPWR_GATING \|
		352	MG_MISC_SUS0_CFG_CL1PWR_GATING \|
		353	MG_MISC_SUS0_CFG_DGPWR_GATING);
		354	I915_WRITE(MG_MISC_SUS0(tc_port), val);
		355	}
		356
201	int	357	int
202	intel_dp_max_data_rate(int max_link_clock, int max_lanes)	358	intel_dp_max_data_rate(int max_link_clock, int max_lanes)
203	{	359	{
@@ -498,7 +654,7 @@ intel_dp_pps_init(struct intel_dp *intel_dp); · 294 unmodified lines
498		654
499	static void pps_lock(struct intel_dp *intel_dp)	655	static void pps_lock(struct intel_dp *intel_dp)
500	{	656	{
501	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	657	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
502		658
503	/*	659	/*
504	* See intel_power_sequencer_reset() why we need	660	* See intel_power_sequencer_reset() why we need
@@ -511,7 +667,7 @@ static void pps_lock(struct intel_dp *intel_dp) · 6 unmodified lines
511		667
512	static void pps_unlock(struct intel_dp *intel_dp)	668	static void pps_unlock(struct intel_dp *intel_dp)
513	{	669	{
514	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	670	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
515		671
516	mutex_unlock(&dev_priv->pps_mutex);	672	mutex_unlock(&dev_priv->pps_mutex);
517		673
@@ -521,7 +677,7 @@ static void pps_unlock(struct intel_dp *intel_dp) · 3 unmodified lines
521	static void	677	static void
522	vlv_power_sequencer_kick(struct intel_dp *intel_dp)	678	vlv_power_sequencer_kick(struct intel_dp *intel_dp)
523	{	679	{
524	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	680	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
525	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	681	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
526	enum pipe pipe = intel_dp->pps_pipe;	682	enum pipe pipe = intel_dp->pps_pipe;
527	bool pll_enabled, release_cl_override = false;	683	bool pll_enabled, release_cl_override = false;
@@ -626,7 +782,7 @@ static enum pipe vlv_find_free_pps(struct drm_i915_private *dev_priv) · 98 unmodified lines
626	static enum pipe	782	static enum pipe
627	vlv_power_sequencer_pipe(struct intel_dp *intel_dp)	783	vlv_power_sequencer_pipe(struct intel_dp *intel_dp)
628	{	784	{
629	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	785	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
630	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	786	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
631	enum pipe pipe;	787	enum pipe pipe;
632		788
@@ -673,7 +829,7 @@ vlv_power_sequencer_pipe(struct intel_dp *intel_dp) · 40 unmodified lines
673	static int	829	static int
674	bxt_power_sequencer_idx(struct intel_dp *intel_dp)	830	bxt_power_sequencer_idx(struct intel_dp *intel_dp)
675	{	831	{
676	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	832	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
677	int backlight_controller = dev_priv->vbt.backlight.controller;	833	int backlight_controller = dev_priv->vbt.backlight.controller;
678		834
679	lockdep_assert_held(&dev_priv->pps_mutex);	835	lockdep_assert_held(&dev_priv->pps_mutex);
@@ -742,7 +898,7 @@ vlv_initial_pps_pipe(struct drm_i915_private *dev_priv, · 62 unmodified lines
742	static void	898	static void
743	vlv_initial_power_sequencer_setup(struct intel_dp *intel_dp)	899	vlv_initial_power_sequencer_setup(struct intel_dp *intel_dp)
744	{	900	{
745	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	901	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
746	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	902	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
747	enum port port = intel_dig_port->base.port;	903	enum port port = intel_dig_port->base.port;
748		904
@@ -819,7 +975,7 @@ struct pps_registers { · 70 unmodified lines
819	static void intel_pps_get_registers(struct intel_dp *intel_dp,	975	static void intel_pps_get_registers(struct intel_dp *intel_dp,
820	struct pps_registers *regs)	976	struct pps_registers *regs)
821	{	977	{
822	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	978	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
823	int pps_idx = 0;	979	int pps_idx = 0;
824		980
825	memset(regs, 0, sizeof(*regs));	981	memset(regs, 0, sizeof(*regs));
@@ -865,7 +1021,7 @@ static int edp_notify_handler(struct notifier_block *this, unsigned long code, · 39 unmodified lines
865	{	1021	{
866	struct intel_dp intel_dp = container_of(this, typeof( intel_dp),	1022	struct intel_dp intel_dp = container_of(this, typeof( intel_dp),
867	edp_notifier);	1023	edp_notifier);
868	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1024	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
869		1025
870	if (!intel_dp_is_edp(intel_dp) \|\| code != SYS_RESTART)	1026	if (!intel_dp_is_edp(intel_dp) \|\| code != SYS_RESTART)
871	return 0;	1027	return 0;
@@ -895,7 +1051,7 @@ static int edp_notify_handler(struct notifier_block *this, unsigned long code, · 23 unmodified lines
895		1051
896	static bool edp_have_panel_power(struct intel_dp *intel_dp)	1052	static bool edp_have_panel_power(struct intel_dp *intel_dp)
897	{	1053	{
898	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1054	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
899		1055
900	lockdep_assert_held(&dev_priv->pps_mutex);	1056	lockdep_assert_held(&dev_priv->pps_mutex);
901		1057
@@ -908,7 +1064,7 @@ static bool edp_have_panel_power(struct intel_dp *intel_dp) · 6 unmodified lines
908		1064
909	static bool edp_have_panel_vdd(struct intel_dp *intel_dp)	1065	static bool edp_have_panel_vdd(struct intel_dp *intel_dp)
910	{	1066	{
911	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1067	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
912		1068
913	lockdep_assert_held(&dev_priv->pps_mutex);	1069	lockdep_assert_held(&dev_priv->pps_mutex);
914		1070
@@ -922,7 +1078,7 @@ static bool edp_have_panel_vdd(struct intel_dp *intel_dp) · 7 unmodified lines
922	static void	1078	static void
923	intel_dp_check_edp(struct intel_dp *intel_dp)	1079	intel_dp_check_edp(struct intel_dp *intel_dp)
924	{	1080	{
925	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1081	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
926		1082
927	if (!intel_dp_is_edp(intel_dp))	1083	if (!intel_dp_is_edp(intel_dp))
928	return;	1084	return;
@@ -938,7 +1094,7 @@ intel_dp_check_edp(struct intel_dp *intel_dp) · 9 unmodified lines
938	static uint32_t	1094	static uint32_t
939	intel_dp_aux_wait_done(struct intel_dp *intel_dp)	1095	intel_dp_aux_wait_done(struct intel_dp *intel_dp)
940	{	1096	{
941	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1097	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
942	i915_reg_t ch_ctl = intel_dp->aux_ch_ctl_reg(intel_dp);	1098	i915_reg_t ch_ctl = intel_dp->aux_ch_ctl_reg(intel_dp);
943	uint32_t status;	1099	uint32_t status;
944	bool done;	1100	bool done;
@@ -955,7 +1111,7 @@ intel_dp_aux_wait_done(struct intel_dp *intel_dp) · 10 unmodified lines
955		1111
956	static uint32_t g4x_get_aux_clock_divider(struct intel_dp *intel_dp, int index)	1112	static uint32_t g4x_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
957	{	1113	{
958	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1114	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
959		1115
960	if (index)	1116	if (index)
961	return 0;	1117	return 0;
@@ -969,7 +1125,7 @@ static uint32_t g4x_get_aux_clock_divider(struct intel_dp *intel_dp, int index) · 7 unmodified lines
969		1125
970	static uint32_t ilk_get_aux_clock_divider(struct intel_dp *intel_dp, int index)	1126	static uint32_t ilk_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
971	{	1127	{
972	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1128	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
973		1129
974	if (index)	1130	if (index)
975	return 0;	1131	return 0;
@@ -987,7 +1143,7 @@ static uint32_t ilk_get_aux_clock_divider(struct intel_dp *intel_dp, int index) · 11 unmodified lines
987		1143
988	static uint32_t hsw_get_aux_clock_divider(struct intel_dp *intel_dp, int index)	1144	static uint32_t hsw_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
989	{	1145	{
990	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1146	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
991		1147
992	if (intel_dp->aux_ch != AUX_CH_A && HAS_PCH_LPT_H(dev_priv)) {	1148	if (intel_dp->aux_ch != AUX_CH_A && HAS_PCH_LPT_H(dev_priv)) {
993	/* Workaround for non-ULT HSW */	1149	/* Workaround for non-ULT HSW */
@@ -1045,15 +1201,23 @@ static uint32_t skl_get_aux_send_ctl(struct intel_dp *intel_dp, · 51 unmodified lines
1045	int send_bytes,	1201	int send_bytes,
1046	uint32_t unused)	1202	uint32_t unused)
1047	{	1203	{
1048	return DP_AUX_CH_CTL_SEND_BUSY \|	1204	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
1049	DP_AUX_CH_CTL_DONE \|	1205	uint32_t ret;
1050	DP_AUX_CH_CTL_INTERRUPT \|	1206
1051	DP_AUX_CH_CTL_TIME_OUT_ERROR \|	1207	ret = DP_AUX_CH_CTL_SEND_BUSY \|
1052	DP_AUX_CH_CTL_TIME_OUT_MAX \|	1208	DP_AUX_CH_CTL_DONE \|
1053	DP_AUX_CH_CTL_RECEIVE_ERROR \|	1209	DP_AUX_CH_CTL_INTERRUPT \|
1054	(send_bytes << DP_AUX_CH_CTL_MESSAGE_SIZE_SHIFT) \|	1210	DP_AUX_CH_CTL_TIME_OUT_ERROR \|
1055	DP_AUX_CH_CTL_FW_SYNC_PULSE_SKL(32) \|	1211	DP_AUX_CH_CTL_TIME_OUT_MAX \|
1056	DP_AUX_CH_CTL_SYNC_PULSE_SKL(32);	1212	DP_AUX_CH_CTL_RECEIVE_ERROR \|
		1213	(send_bytes << DP_AUX_CH_CTL_MESSAGE_SIZE_SHIFT) \|
		1214	DP_AUX_CH_CTL_FW_SYNC_PULSE_SKL(32) \|
		1215	DP_AUX_CH_CTL_SYNC_PULSE_SKL(32);
		1216
		1217	if (intel_dig_port->tc_type == TC_PORT_TBT)
		1218	ret \|= DP_AUX_CH_CTL_TBT_IO;
		1219
		1220	return ret;
1057	}	1221	}
1058		1222
1059	static int	1223	static int
@@ -1381,7 +1545,7 @@ intel_aux_power_domain(struct intel_dp *intel_dp) · 321 unmodified lines
1381		1545
1382	static i915_reg_t g4x_aux_ctl_reg(struct intel_dp *intel_dp)	1546	static i915_reg_t g4x_aux_ctl_reg(struct intel_dp *intel_dp)
1383	{	1547	{
1384	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1548	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1385	enum aux_ch aux_ch = intel_dp->aux_ch;	1549	enum aux_ch aux_ch = intel_dp->aux_ch;
1386		1550
1387	switch (aux_ch) {	1551	switch (aux_ch) {
@@ -1397,7 +1561,7 @@ static i915_reg_t g4x_aux_ctl_reg(struct intel_dp *intel_dp) · 9 unmodified lines
1397		1561
1398	static i915_reg_t g4x_aux_data_reg(struct intel_dp *intel_dp, int index)	1562	static i915_reg_t g4x_aux_data_reg(struct intel_dp *intel_dp, int index)
1399	{	1563	{
1400	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1564	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1401	enum aux_ch aux_ch = intel_dp->aux_ch;	1565	enum aux_ch aux_ch = intel_dp->aux_ch;
1402		1566
1403	switch (aux_ch) {	1567	switch (aux_ch) {
@@ -1413,7 +1577,7 @@ static i915_reg_t g4x_aux_data_reg(struct intel_dp *intel_dp, int index) · 9 unmodified lines
1413		1577
1414	static i915_reg_t ilk_aux_ctl_reg(struct intel_dp *intel_dp)	1578	static i915_reg_t ilk_aux_ctl_reg(struct intel_dp *intel_dp)
1415	{	1579	{
1416	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1580	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1417	enum aux_ch aux_ch = intel_dp->aux_ch;	1581	enum aux_ch aux_ch = intel_dp->aux_ch;
1418		1582
1419	switch (aux_ch) {	1583	switch (aux_ch) {
@@ -1431,7 +1595,7 @@ static i915_reg_t ilk_aux_ctl_reg(struct intel_dp *intel_dp) · 11 unmodified lines
1431		1595
1432	static i915_reg_t ilk_aux_data_reg(struct intel_dp *intel_dp, int index)	1596	static i915_reg_t ilk_aux_data_reg(struct intel_dp *intel_dp, int index)
1433	{	1597	{
1434	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1598	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1435	enum aux_ch aux_ch = intel_dp->aux_ch;	1599	enum aux_ch aux_ch = intel_dp->aux_ch;
1436		1600
1437	switch (aux_ch) {	1601	switch (aux_ch) {
@@ -1449,7 +1613,7 @@ static i915_reg_t ilk_aux_data_reg(struct intel_dp *intel_dp, int index) · 11 unmodified lines
1449		1613
1450	static i915_reg_t skl_aux_ctl_reg(struct intel_dp *intel_dp)	1614	static i915_reg_t skl_aux_ctl_reg(struct intel_dp *intel_dp)
1451	{	1615	{
1452	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1616	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1453	enum aux_ch aux_ch = intel_dp->aux_ch;	1617	enum aux_ch aux_ch = intel_dp->aux_ch;
1454		1618
1455	switch (aux_ch) {	1619	switch (aux_ch) {
@@ -1468,7 +1632,7 @@ static i915_reg_t skl_aux_ctl_reg(struct intel_dp *intel_dp) · 12 unmodified lines
1468		1632
1469	static i915_reg_t skl_aux_data_reg(struct intel_dp *intel_dp, int index)	1633	static i915_reg_t skl_aux_data_reg(struct intel_dp *intel_dp, int index)
1470	{	1634	{
1471	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1635	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1472	enum aux_ch aux_ch = intel_dp->aux_ch;	1636	enum aux_ch aux_ch = intel_dp->aux_ch;
1473		1637
1474	switch (aux_ch) {	1638	switch (aux_ch) {
@@ -1494,7 +1658,7 @@ intel_dp_aux_fini(struct intel_dp *intel_dp) · 19 unmodified lines
1494	static void	1658	static void
1495	intel_dp_aux_init(struct intel_dp *intel_dp)	1659	intel_dp_aux_init(struct intel_dp *intel_dp)
1496	{	1660	{
1497	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1661	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1498	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;	1662	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;
1499		1663
1500	intel_dp->aux_ch = intel_aux_ch(intel_dp);	1664	intel_dp->aux_ch = intel_aux_ch(intel_dp);
@@ -1662,7 +1826,7 @@ struct link_config_limits { · 161 unmodified lines
1662	static int intel_dp_compute_bpp(struct intel_dp *intel_dp,	1826	static int intel_dp_compute_bpp(struct intel_dp *intel_dp,
1663	struct intel_crtc_state *pipe_config)	1827	struct intel_crtc_state *pipe_config)
1664	{	1828	{
1665	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	1829	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
1666	struct intel_connector *intel_connector = intel_dp->attached_connector;	1830	struct intel_connector *intel_connector = intel_dp->attached_connector;
1667	int bpp, bpc;	1831	int bpp, bpc;
1668		1832
@@ -2030,7 +2194,7 @@ static void wait_panel_status(struct intel_dp *intel_dp, · 361 unmodified lines
2030	u32 mask,	2194	u32 mask,
2031	u32 value)	2195	u32 value)
2032	{	2196	{
2033	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2197	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2034	i915_reg_t pp_stat_reg, pp_ctrl_reg;	2198	i915_reg_t pp_stat_reg, pp_ctrl_reg;
2035		2199
2036	lockdep_assert_held(&dev_priv->pps_mutex);	2200	lockdep_assert_held(&dev_priv->pps_mutex);
@@ -2106,7 +2270,7 @@ static void edp_wait_backlight_off(struct intel_dp *intel_dp) · 69 unmodified lines
2106		2270
2107	static u32 ironlake_get_pp_control(struct intel_dp *intel_dp)	2271	static u32 ironlake_get_pp_control(struct intel_dp *intel_dp)
2108	{	2272	{
2109	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2273	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2110	u32 control;	2274	u32 control;
2111		2275
2112	lockdep_assert_held(&dev_priv->pps_mutex);	2276	lockdep_assert_held(&dev_priv->pps_mutex);
@@ -2127,7 +2291,7 @@ static u32 ironlake_get_pp_control(struct intel_dp *intel_dp) · 14 unmodified lines
2127	*/	2291	*/
2128	static bool edp_panel_vdd_on(struct intel_dp *intel_dp)	2292	static bool edp_panel_vdd_on(struct intel_dp *intel_dp)
2129	{	2293	{
2130	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2294	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2131	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	2295	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
2132	u32 pp;	2296	u32 pp;
2133	i915_reg_t pp_stat_reg, pp_ctrl_reg;	2297	i915_reg_t pp_stat_reg, pp_ctrl_reg;
@@ -2198,7 +2362,7 @@ void intel_edp_panel_vdd_on(struct intel_dp *intel_dp) · 64 unmodified lines
2198		2362
2199	static void edp_panel_vdd_off_sync(struct intel_dp *intel_dp)	2363	static void edp_panel_vdd_off_sync(struct intel_dp *intel_dp)
2200	{	2364	{
2201	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2365	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2202	struct intel_digital_port *intel_dig_port =	2366	struct intel_digital_port *intel_dig_port =
2203	dp_to_dig_port(intel_dp);	2367	dp_to_dig_port(intel_dp);
2204	u32 pp;	2368	u32 pp;
@@ -2264,7 +2428,7 @@ static void edp_panel_vdd_schedule_off(struct intel_dp *intel_dp) · 59 unmodified lines
2264	*/	2428	*/
2265	static void edp_panel_vdd_off(struct intel_dp *intel_dp, bool sync)	2429	static void edp_panel_vdd_off(struct intel_dp *intel_dp, bool sync)
2266	{	2430	{
2267	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2431	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2268		2432
2269	lockdep_assert_held(&dev_priv->pps_mutex);	2433	lockdep_assert_held(&dev_priv->pps_mutex);
2270		2434
@@ -2284,7 +2448,7 @@ static void edp_panel_vdd_off(struct intel_dp *intel_dp, bool sync) · 13 unmodified lines
2284		2448
2285	static void edp_panel_on(struct intel_dp *intel_dp)	2449	static void edp_panel_on(struct intel_dp *intel_dp)
2286	{	2450	{
2287	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2451	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2288	u32 pp;	2452	u32 pp;
2289	i915_reg_t pp_ctrl_reg;	2453	i915_reg_t pp_ctrl_reg;
2290		2454
@@ -2342,7 +2506,7 @@ void intel_edp_panel_on(struct intel_dp *intel_dp) · 51 unmodified lines
2342		2506
2343	static void edp_panel_off(struct intel_dp *intel_dp)	2507	static void edp_panel_off(struct intel_dp *intel_dp)
2344	{	2508	{
2345	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2509	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2346	u32 pp;	2510	u32 pp;
2347	i915_reg_t pp_ctrl_reg;	2511	i915_reg_t pp_ctrl_reg;
2348		2512
@@ -2390,7 +2554,7 @@ void intel_edp_panel_off(struct intel_dp *intel_dp) · 41 unmodified lines
2390	/* Enable backlight in the panel power control. */	2554	/* Enable backlight in the panel power control. */
2391	static void _intel_edp_backlight_on(struct intel_dp *intel_dp)	2555	static void _intel_edp_backlight_on(struct intel_dp *intel_dp)
2392	{	2556	{
2393	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2557	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2394	u32 pp;	2558	u32 pp;
2395	i915_reg_t pp_ctrl_reg;	2559	i915_reg_t pp_ctrl_reg;
2396		2560
@@ -2433,7 +2597,7 @@ void intel_edp_backlight_on(const struct intel_crtc_state *crtc_state, · 36 unmodified lines
2433	/* Disable backlight in the panel power control. */	2597	/* Disable backlight in the panel power control. */
2434	static void _intel_edp_backlight_off(struct intel_dp *intel_dp)	2598	static void _intel_edp_backlight_off(struct intel_dp *intel_dp)
2435	{	2599	{
2436	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	2600	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2437	u32 pp;	2601	u32 pp;
2438	i915_reg_t pp_ctrl_reg;	2602	i915_reg_t pp_ctrl_reg;
2439		2603
@@ -2864,7 +3028,7 @@ _intel_dp_set_link_train(struct intel_dp *intel_dp, · 424 unmodified lines
2864	uint32_t *DP,	3028	uint32_t *DP,
2865	uint8_t dp_train_pat)	3029	uint8_t dp_train_pat)
2866	{	3030	{
2867	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	3031	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2868	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	3032	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
2869	enum port port = intel_dig_port->base.port;	3033	enum port port = intel_dig_port->base.port;
2870	uint8_t train_pat_mask = drm_dp_training_pattern_mask(intel_dp->dpcd);	3034	uint8_t train_pat_mask = drm_dp_training_pattern_mask(intel_dp->dpcd);
@@ -2946,7 +3110,7 @@ _intel_dp_set_link_train(struct intel_dp *intel_dp, · 75 unmodified lines
2946	static void intel_dp_enable_port(struct intel_dp *intel_dp,	3110	static void intel_dp_enable_port(struct intel_dp *intel_dp,
2947	const struct intel_crtc_state *old_crtc_state)	3111	const struct intel_crtc_state *old_crtc_state)
2948	{	3112	{
2949	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	3113	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
2950		3114
2951	/* enable with pattern 1 (as per spec) */	3115	/* enable with pattern 1 (as per spec) */
2952		3116
@@ -3203,7 +3367,7 @@ intel_dp_get_link_status(struct intel_dp *intel_dp, uint8_t link_status[DP_LINK_ · 250 unmodified lines
3203	uint8_t	3367	uint8_t
3204	intel_dp_voltage_max(struct intel_dp *intel_dp)	3368	intel_dp_voltage_max(struct intel_dp *intel_dp)
3205	{	3369	{
3206	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	3370	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
3207	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;	3371	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;
3208	enum port port = encoder->port;	3372	enum port port = encoder->port;
3209		3373
@@ -3222,7 +3386,7 @@ intel_dp_voltage_max(struct intel_dp *intel_dp) · 12 unmodified lines
3222	uint8_t	3386	uint8_t
3223	intel_dp_pre_emphasis_max(struct intel_dp *intel_dp, uint8_t voltage_swing)	3387	intel_dp_pre_emphasis_max(struct intel_dp *intel_dp, uint8_t voltage_swing)
3224	{	3388	{
3225	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	3389	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
3226	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;	3390	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;
3227	enum port port = encoder->port;	3391	enum port port = encoder->port;
3228		3392
@@ -3534,7 +3698,7 @@ ivb_cpu_edp_signal_levels(uint8_t train_set) · 305 unmodified lines
3534	void	3698	void
3535	intel_dp_set_signal_levels(struct intel_dp *intel_dp)	3699	intel_dp_set_signal_levels(struct intel_dp *intel_dp)
3536	{	3700	{
3537	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	3701	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
3538	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	3702	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
3539	enum port port = intel_dig_port->base.port;	3703	enum port port = intel_dig_port->base.port;
3540	uint32_t signal_levels, mask = 0;	3704	uint32_t signal_levels, mask = 0;
@@ -3591,7 +3755,7 @@ intel_dp_program_link_training_pattern(struct intel_dp *intel_dp, · 50 unmodified lines
3591		3755
3592	void intel_dp_set_idle_link_train(struct intel_dp *intel_dp)	3756	void intel_dp_set_idle_link_train(struct intel_dp *intel_dp)
3593	{	3757	{
3594	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	3758	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
3595	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	3759	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
3596	enum port port = intel_dig_port->base.port;	3760	enum port port = intel_dig_port->base.port;
3597	uint32_t val;	3761	uint32_t val;
@@ -4090,12 +4254,14 @@ intel_dp_check_mst_status(struct intel_dp *intel_dp) · 492 unmodified lines
4090	int ret = 0;	4254	int ret = 0;
4091	int retry;	4255	int retry;
4092	bool handled;	4256	bool handled;
		4257
		4258	WARN_ON_ONCE(intel_dp->active_mst_links < 0);
4093	bret = intel_dp_get_sink_irq_esi(intel_dp, esi);	4259	bret = intel_dp_get_sink_irq_esi(intel_dp, esi);
4094	go_again:	4260	go_again:
4095	if (bret == true) {	4261	if (bret == true) {
4096		4262
4097	/* check link status - esi[10] = 0x200c */	4263	/* check link status - esi[10] = 0x200c */
4098	if (intel_dp->active_mst_links &&	4264	if (intel_dp->active_mst_links > 0 &&
4099	!drm_dp_channel_eq_ok(&esi[10], intel_dp->lane_count)) {	4265	!drm_dp_channel_eq_ok(&esi[10], intel_dp->lane_count)) {
4100	DRM_DEBUG_KMS("channel EQ not ok, retraining\n");	4266	DRM_DEBUG_KMS("channel EQ not ok, retraining\n");
4101	intel_dp_start_link_train(intel_dp);	4267	intel_dp_start_link_train(intel_dp);
@@ -4160,18 +4326,6 @@ intel_dp_needs_link_retrain(struct intel_dp *intel_dp) · 58 unmodified lines
4160	return !drm_dp_channel_eq_ok(link_status, intel_dp->lane_count);	4326	return !drm_dp_channel_eq_ok(link_status, intel_dp->lane_count);
4161	}	4327	}
4162		4328
4163	/*
4164	* If display is now connected check links status,
4165	* there has been known issues of link loss triggering
4166	* long pulse.
4167	*
4168	* Some sinks (eg. ASUS PB287Q) seem to perform some
4169	* weird HPD ping pong during modesets. So we can apparently
4170	* end up with HPD going low during a modeset, and then
4171	* going back up soon after. And once that happens we must
4172	* retrain the link to get a picture. That's in case no
4173	* userspace component reacted to intermittent HPD dip.
4174	*/
4175	int intel_dp_retrain_link(struct intel_encoder *encoder,	4329	int intel_dp_retrain_link(struct intel_encoder *encoder,
4176	struct drm_modeset_acquire_ctx *ctx)	4330	struct drm_modeset_acquire_ctx *ctx)
4177	{	4331	{
@@ -4294,7 +4448,7 @@ static bool intel_dp_hotplug(struct intel_encoder *encoder, · 116 unmodified lines
4294	static bool	4448	static bool
4295	intel_dp_short_pulse(struct intel_dp *intel_dp)	4449	intel_dp_short_pulse(struct intel_dp *intel_dp)
4296	{	4450	{
4297	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	4451	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
4298	u8 sink_irq_vector = 0;	4452	u8 sink_irq_vector = 0;
4299	u8 old_sink_count = intel_dp->sink_count;	4453	u8 old_sink_count = intel_dp->sink_count;
4300	bool ret;	4454	bool ret;
@@ -4586,10 +4740,205 @@ static bool bxt_digital_port_connected(struct intel_encoder *encoder) · 285 unmodified lines
4586	return I915_READ(GEN8_DE_PORT_ISR) & bit;	4740	return I915_READ(GEN8_DE_PORT_ISR) & bit;
4587	}	4741	}
4588		4742
		4743	static bool icl_combo_port_connected(struct drm_i915_private *dev_priv,
		4744	struct intel_digital_port *intel_dig_port)
		4745	{
		4746	enum port port = intel_dig_port->base.port;
		4747
		4748	return I915_READ(SDEISR) & SDE_DDI_HOTPLUG_ICP(port);
		4749	}
		4750
		4751	static void icl_update_tc_port_type(struct drm_i915_private *dev_priv,
		4752	struct intel_digital_port *intel_dig_port,
		4753	bool is_legacy, bool is_typec, bool is_tbt)
		4754	{
		4755	enum port port = intel_dig_port->base.port;
		4756	enum tc_port_type old_type = intel_dig_port->tc_type;
		4757	const char *type_str;
		4758
		4759	WARN_ON(is_legacy + is_typec + is_tbt != 1);
		4760
		4761	if (is_legacy) {
		4762	intel_dig_port->tc_type = TC_PORT_LEGACY;
		4763	type_str = "legacy";
		4764	} else if (is_typec) {
		4765	intel_dig_port->tc_type = TC_PORT_TYPEC;
		4766	type_str = "typec";
		4767	} else if (is_tbt) {
		4768	intel_dig_port->tc_type = TC_PORT_TBT;
		4769	type_str = "tbt";
		4770	} else {
		4771	return;
		4772	}
		4773
		4774	/* Types are not supposed to be changed at runtime. */
		4775	WARN_ON(old_type != TC_PORT_UNKNOWN &&
		4776	old_type != intel_dig_port->tc_type);
		4777
		4778	if (old_type != intel_dig_port->tc_type)
		4779	DRM_DEBUG_KMS("Port %c has TC type %s\n", port_name(port),
		4780	type_str);
		4781	}
		4782
		4783	/*
		4784	* This function implements the first part of the Connect Flow described by our
		4785	* specification, Gen11 TypeC Programming chapter. The rest of the flow (reading
		4786	* lanes, EDID, etc) is done as needed in the typical places.
		4787	*
		4788	* Unlike the other ports, type-C ports are not available to use as soon as we
		4789	* get a hotplug. The type-C PHYs can be shared between multiple controllers:
		4790	* display, USB, etc. As a result, handshaking through FIA is required around
		4791	* connect and disconnect to cleanly transfer ownership with the controller and
		4792	* set the type-C power state.
		4793	*
		4794	* We could opt to only do the connect flow when we actually try to use the AUX
		4795	* channels or do a modeset, then immediately run the disconnect flow after
		4796	* usage, but there are some implications on this for a dynamic environment:
		4797	* things may go away or change behind our backs. So for now our driver is
		4798	* always trying to acquire ownership of the controller as soon as it gets an
		4799	* interrupt (or polls state and sees a port is connected) and only gives it
		4800	* back when it sees a disconnect. Implementation of a more fine-grained model
		4801	* will require a lot of coordination with user space and thorough testing for
		4802	* the extra possible cases.
		4803	*/
		4804	static bool icl_tc_phy_connect(struct drm_i915_private *dev_priv,
		4805	struct intel_digital_port *dig_port)
		4806	{
		4807	enum tc_port tc_port = intel_port_to_tc(dev_priv, dig_port->base.port);
		4808	u32 val;
		4809
		4810	if (dig_port->tc_type != TC_PORT_LEGACY &&
		4811	dig_port->tc_type != TC_PORT_TYPEC)
		4812	return true;
		4813
		4814	val = I915_READ(PORT_TX_DFLEXDPPMS);
		4815	if (!(val & DP_PHY_MODE_STATUS_COMPLETED(tc_port))) {
		4816	DRM_DEBUG_KMS("DP PHY for TC port %d not ready\n", tc_port);
		4817	return false;
		4818	}
		4819
		4820	/*
		4821	* This function may be called many times in a row without an HPD event
		4822	* in between, so try to avoid the write when we can.
		4823	*/
		4824	val = I915_READ(PORT_TX_DFLEXDPCSSS);
		4825	if (!(val & DP_PHY_MODE_STATUS_NOT_SAFE(tc_port))) {
		4826	val \|= DP_PHY_MODE_STATUS_NOT_SAFE(tc_port);
		4827	I915_WRITE(PORT_TX_DFLEXDPCSSS, val);
		4828	}
		4829
		4830	/*
		4831	* Now we have to re-check the live state, in case the port recently
		4832	* became disconnected. Not necessary for legacy mode.
		4833	*/
		4834	if (dig_port->tc_type == TC_PORT_TYPEC &&
		4835	!(I915_READ(PORT_TX_DFLEXDPSP) & TC_LIVE_STATE_TC(tc_port))) {
		4836	DRM_DEBUG_KMS("TC PHY %d sudden disconnect.\n", tc_port);
		4837	val = I915_READ(PORT_TX_DFLEXDPCSSS);
		4838	val &= ~DP_PHY_MODE_STATUS_NOT_SAFE(tc_port);
		4839	I915_WRITE(PORT_TX_DFLEXDPCSSS, val);
		4840	return false;
		4841	}
		4842
		4843	return true;
		4844	}
		4845
		4846	/*
		4847	* See the comment at the connect function. This implements the Disconnect
		4848	* Flow.
		4849	*/
		4850	static void icl_tc_phy_disconnect(struct drm_i915_private *dev_priv,
		4851	struct intel_digital_port *dig_port)
		4852	{
		4853	enum tc_port tc_port = intel_port_to_tc(dev_priv, dig_port->base.port);
		4854	u32 val;
		4855
		4856	if (dig_port->tc_type != TC_PORT_LEGACY &&
		4857	dig_port->tc_type != TC_PORT_TYPEC)
		4858	return;
		4859
		4860	/*
		4861	* This function may be called many times in a row without an HPD event
		4862	* in between, so try to avoid the write when we can.
		4863	*/
		4864	val = I915_READ(PORT_TX_DFLEXDPCSSS);
		4865	if (val & DP_PHY_MODE_STATUS_NOT_SAFE(tc_port)) {
		4866	val &= ~DP_PHY_MODE_STATUS_NOT_SAFE(tc_port);
		4867	I915_WRITE(PORT_TX_DFLEXDPCSSS, val);
		4868	}
		4869	}
		4870
		4871	/*
		4872	* The type-C ports are different because even when they are connected, they may
		4873	* not be available/usable by the graphics driver: see the comment on
		4874	* icl_tc_phy_connect(). So in our driver instead of adding the additional
		4875	* concept of "usable" and make everything check for "connected and usable" we
		4876	* define a port as "connected" when it is not only connected, but also when it
		4877	* is usable by the rest of the driver. That maintains the old assumption that
		4878	* connected ports are usable, and avoids exposing to the users objects they
		4879	* can't really use.
		4880	*/
		4881	static bool icl_tc_port_connected(struct drm_i915_private *dev_priv,
		4882	struct intel_digital_port *intel_dig_port)
		4883	{
		4884	enum port port = intel_dig_port->base.port;
		4885	enum tc_port tc_port = intel_port_to_tc(dev_priv, port);
		4886	bool is_legacy, is_typec, is_tbt;
		4887	u32 dpsp;
		4888
		4889	is_legacy = I915_READ(SDEISR) & SDE_TC_HOTPLUG_ICP(tc_port);
		4890
		4891	/*
		4892	* The spec says we shouldn't be using the ISR bits for detecting
		4893	* between TC and TBT. We should use DFLEXDPSP.
		4894	*/
		4895	dpsp = I915_READ(PORT_TX_DFLEXDPSP);
		4896	is_typec = dpsp & TC_LIVE_STATE_TC(tc_port);
		4897	is_tbt = dpsp & TC_LIVE_STATE_TBT(tc_port);
		4898
		4899	if (!is_legacy && !is_typec && !is_tbt) {
		4900	icl_tc_phy_disconnect(dev_priv, intel_dig_port);
		4901	return false;
		4902	}
		4903
		4904	icl_update_tc_port_type(dev_priv, intel_dig_port, is_legacy, is_typec,
		4905	is_tbt);
		4906
		4907	if (!icl_tc_phy_connect(dev_priv, intel_dig_port))
		4908	return false;
		4909
		4910	return true;
		4911	}
		4912
		4913	static bool icl_digital_port_connected(struct intel_encoder *encoder)
		4914	{
		4915	struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
		4916	struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
		4917
		4918	switch (encoder->hpd_pin) {
		4919	case HPD_PORT_A:
		4920	case HPD_PORT_B:
		4921	return icl_combo_port_connected(dev_priv, dig_port);
		4922	case HPD_PORT_C:
		4923	case HPD_PORT_D:
		4924	case HPD_PORT_E:
		4925	case HPD_PORT_F:
		4926	return icl_tc_port_connected(dev_priv, dig_port);
		4927	default:
		4928	MISSING_CASE(encoder->hpd_pin);
		4929	return false;
		4930	}
		4931	}
		4932
4589	/*	4933	/*
4590	* intel_digital_port_connected - is the specified port connected?	4934	* intel_digital_port_connected - is the specified port connected?
4591	* @encoder: intel_encoder	4935	* @encoder: intel_encoder
4592	*	4936	*
		4937	* In cases where there's a connector physically connected but it can't be used
		4938	* by our hardware we also return false, since the rest of the driver should
		4939	* pretty much treat the port as disconnected. This is relevant for type-C
		4940	* (starting on ICL) where there's ownership involved.
		4941	*
4593	* Return %true if port is connected, %false otherwise.	4942	* Return %true if port is connected, %false otherwise.
4594	*/	4943	*/
4595	bool intel_digital_port_connected(struct intel_encoder *encoder)	4944	bool intel_digital_port_connected(struct intel_encoder *encoder)
@@ -4613,8 +4962,10 @@ bool intel_digital_port_connected(struct intel_encoder *encoder) · 17 unmodified lines
4613	return bdw_digital_port_connected(encoder);	4962	return bdw_digital_port_connected(encoder);
4614	else if (IS_GEN9_LP(dev_priv))	4963	else if (IS_GEN9_LP(dev_priv))
4615	return bxt_digital_port_connected(encoder);	4964	return bxt_digital_port_connected(encoder);
4616	else	4965	else if (IS_GEN9_BC(dev_priv) \|\| IS_GEN10(dev_priv))
4617	return spt_digital_port_connected(encoder);	4966	return spt_digital_port_connected(encoder);
		4967	else
		4968	return icl_digital_port_connected(encoder);
4618	}	4969	}
4619		4970
4620	static struct edid *	4971	static struct edid *
@@ -4661,7 +5012,8 @@ intel_dp_unset_edid(struct intel_dp *intel_dp) · 40 unmodified lines
4661	}	5012	}
4662		5013
4663	static int	5014	static int
4664	intel_dp_long_pulse(struct intel_connector *connector)	5015	intel_dp_long_pulse(struct intel_connector *connector,
		5016	struct drm_modeset_acquire_ctx *ctx)
4665	{	5017	{
4666	struct drm_i915_private *dev_priv = to_i915(connector->base.dev);	5018	struct drm_i915_private *dev_priv = to_i915(connector->base.dev);
4667	struct intel_dp *intel_dp = intel_attached_dp(&connector->base);	5019	struct intel_dp *intel_dp = intel_attached_dp(&connector->base);
@@ -4720,6 +5072,22 @@ intel_dp_long_pulse(struct intel_connector *connector) · 52 unmodified lines
4720	*/	5072	*/
4721	status = connector_status_disconnected;	5073	status = connector_status_disconnected;
4722	goto out;	5074	goto out;
		5075	} else {
		5076	/*
		5077	* If display is now connected check links status,
		5078	* there has been known issues of link loss triggering
		5079	* long pulse.
		5080	*
		5081	* Some sinks (eg. ASUS PB287Q) seem to perform some
		5082	* weird HPD ping pong during modesets. So we can apparently
		5083	* end up with HPD going low during a modeset, and then
		5084	* going back up soon after. And once that happens we must
		5085	* retrain the link to get a picture. That's in case no
		5086	* userspace component reacted to intermittent HPD dip.
		5087	*/
		5088	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;
		5089
		5090	intel_dp_retrain_link(encoder, ctx);
4723	}	5091	}
4724		5092
4725	/*	5093	/*
@@ -4781,7 +5149,7 @@ intel_dp_detect(struct drm_connector *connector, · 55 unmodified lines
4781	return ret;	5149	return ret;
4782	}	5150	}
4783		5151
4784	status = intel_dp_long_pulse(intel_dp->attached_connector);	5152	status = intel_dp_long_pulse(intel_dp->attached_connector, ctx);
4785	}	5153	}
4786		5154
4787	intel_dp->detect_done = false;	5155	intel_dp->detect_done = false;
@@ -5172,7 +5540,7 @@ static const struct intel_hdcp_shim intel_dp_hdcp_shim = { · 384 unmodified lines
5172		5540
5173	static void intel_edp_panel_vdd_sanitize(struct intel_dp *intel_dp)	5541	static void intel_edp_panel_vdd_sanitize(struct intel_dp *intel_dp)
5174	{	5542	{
5175	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	5543	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5176		5544
5177	lockdep_assert_held(&dev_priv->pps_mutex);	5545	lockdep_assert_held(&dev_priv->pps_mutex);
5178		5546
@@ -5193,7 +5561,7 @@ static void intel_edp_panel_vdd_sanitize(struct intel_dp *intel_dp) · 14 unmodified lines
5193		5561
5194	static enum pipe vlv_active_pipe(struct intel_dp *intel_dp)	5562	static enum pipe vlv_active_pipe(struct intel_dp *intel_dp)
5195	{	5563	{
5196	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	5564	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5197	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;	5565	struct intel_encoder *encoder = &dp_to_dig_port(intel_dp)->base;
5198	enum pipe pipe;	5566	enum pipe pipe;
5199		5567
@@ -5260,7 +5628,7 @@ enum irqreturn · 60 unmodified lines
5260	intel_dp_hpd_pulse(struct intel_digital_port *intel_dig_port, bool long_hpd)	5628	intel_dp_hpd_pulse(struct intel_digital_port *intel_dig_port, bool long_hpd)
5261	{	5629	{
5262	struct intel_dp *intel_dp = &intel_dig_port->dp;	5630	struct intel_dp *intel_dp = &intel_dig_port->dp;
5263	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	5631	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5264	enum irqreturn ret = IRQ_NONE;	5632	enum irqreturn ret = IRQ_NONE;
5265		5633
5266	if (long_hpd && intel_dig_port->base.type == INTEL_OUTPUT_EDP) {	5634	if (long_hpd && intel_dig_port->base.type == INTEL_OUTPUT_EDP) {
@@ -5376,7 +5744,7 @@ static void intel_dp_init_panel_power_timestamps(struct intel_dp *intel_dp) · 109 unmodified lines
5376	static void	5744	static void
5377	intel_pps_readout_hw_state(struct intel_dp intel_dp, struct edp_power_seq seq)	5745	intel_pps_readout_hw_state(struct intel_dp intel_dp, struct edp_power_seq seq)
5378	{	5746	{
5379	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	5747	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5380	u32 pp_on, pp_off, pp_div = 0, pp_ctl = 0;	5748	u32 pp_on, pp_off, pp_div = 0, pp_ctl = 0;
5381	struct pps_registers regs;	5749	struct pps_registers regs;
5382		5750
@@ -5444,7 +5812,7 @@ intel_pps_verify_state(struct intel_dp *intel_dp) · 61 unmodified lines
5444	static void	5812	static void
5445	intel_dp_init_panel_power_sequencer(struct intel_dp *intel_dp)	5813	intel_dp_init_panel_power_sequencer(struct intel_dp *intel_dp)
5446	{	5814	{
5447	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	5815	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5448	struct edp_power_seq cur, vbt, spec,	5816	struct edp_power_seq cur, vbt, spec,
5449	*final = &intel_dp->pps_delays;	5817	*final = &intel_dp->pps_delays;
5450		5818
@@ -5537,7 +5905,7 @@ static void · 86 unmodified lines
5537	intel_dp_init_panel_power_sequencer_registers(struct intel_dp *intel_dp,	5905	intel_dp_init_panel_power_sequencer_registers(struct intel_dp *intel_dp,
5538	bool force_disable_vdd)	5906	bool force_disable_vdd)
5539	{	5907	{
5540	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	5908	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5541	u32 pp_on, pp_off, pp_div, port_sel = 0;	5909	u32 pp_on, pp_off, pp_div, port_sel = 0;
5542	int div = dev_priv->rawclk_freq / 1000;	5910	int div = dev_priv->rawclk_freq / 1000;
5543	struct pps_registers regs;	5911	struct pps_registers regs;
@@ -5633,7 +6001,7 @@ intel_dp_init_panel_power_sequencer_registers(struct intel_dp *intel_dp, · 89 unmodified lines
5633		6001
5634	static void intel_dp_pps_init(struct intel_dp *intel_dp)	6002	static void intel_dp_pps_init(struct intel_dp *intel_dp)
5635	{	6003	{
5636	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	6004	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5637		6005
5638	if (IS_VALLEYVIEW(dev_priv) \|\| IS_CHERRYVIEW(dev_priv)) {	6006	if (IS_VALLEYVIEW(dev_priv) \|\| IS_CHERRYVIEW(dev_priv)) {
5639	vlv_initial_power_sequencer_setup(intel_dp);	6007	vlv_initial_power_sequencer_setup(intel_dp);
@@ -5750,7 +6118,7 @@ static void intel_dp_set_drrs_state(struct drm_i915_private *dev_priv, · 110 unmodified lines
5750	void intel_edp_drrs_enable(struct intel_dp *intel_dp,	6118	void intel_edp_drrs_enable(struct intel_dp *intel_dp,
5751	const struct intel_crtc_state *crtc_state)	6119	const struct intel_crtc_state *crtc_state)
5752	{	6120	{
5753	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	6121	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5754		6122
5755	if (!crtc_state->has_drrs) {	6123	if (!crtc_state->has_drrs) {
5756	DRM_DEBUG_KMS("Panel doesn't support DRRS\n");	6124	DRM_DEBUG_KMS("Panel doesn't support DRRS\n");
@@ -5785,7 +6153,7 @@ unlock: · 28 unmodified lines
5785	void intel_edp_drrs_disable(struct intel_dp *intel_dp,	6153	void intel_edp_drrs_disable(struct intel_dp *intel_dp,
5786	const struct intel_crtc_state *old_crtc_state)	6154	const struct intel_crtc_state *old_crtc_state)
5787	{	6155	{
5788	struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));	6156	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
5789		6157
5790	if (!old_crtc_state->has_drrs)	6158	if (!old_crtc_state->has_drrs)
5791	return;	6159	return;
@@ -6017,8 +6385,8 @@ intel_dp_drrs_init(struct intel_connector *connector, · 225 unmodified lines
6017	static bool intel_edp_init_connector(struct intel_dp *intel_dp,	6385	static bool intel_edp_init_connector(struct intel_dp *intel_dp,
6018	struct intel_connector *intel_connector)	6386	struct intel_connector *intel_connector)
6019	{	6387	{
6020	struct drm_device *dev = intel_dp_to_dev(intel_dp);	6388	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
6021	struct drm_i915_private *dev_priv = to_i915(dev);	6389	struct drm_device *dev = &dev_priv->drm;
6022	struct drm_connector *connector = &intel_connector->base;	6390	struct drm_connector *connector = &intel_connector->base;
6023	struct drm_display_mode *fixed_mode = NULL;	6391	struct drm_display_mode *fixed_mode = NULL;
6024	struct drm_display_mode *downclock_mode = NULL;	6392	struct drm_display_mode *downclock_mode = NULL;

drivers/gpu/drm/i915/intel_dp_link_training.c+20−5

 intel_dp_link_training_clock_recovery(struct intel_dp *intel_dp)
 {
 	uint8_t voltage;
-	int voltage_tries, max_vswing_tries;
+	int voltage_tries, cr_tries, max_cr_tries;
+	bool max_vswing_reached = false;
 	uint8_t link_config[2];
 	uint8_t link_bw, rate_select;
 		return false;
 	}
+	/*
+	 * The DP 1.4 spec defines the max clock recovery retries value
+	 * as 10 but for pre-DP 1.4 devices we set a very tolerant
+	 * retry limit of 80 (4 voltage levels x 4 preemphasis levels x
+	 * x 5 identical voltage retries). Since the previous specs didn't
+	 * define a limit and created the possibility of an infinite loop
+	 * we want to prevent any sync from triggering that corner case.
+	 */
+	if (intel_dp->dpcd[DP_DPCD_REV] >= DP_DPCD_REV_14)
+		max_cr_tries = 10;
+	else
+		max_cr_tries = 80;
 	voltage_tries = 1;
-	max_vswing_tries = 0;
+	for (cr_tries = 0; cr_tries < max_cr_tries; ++cr_tries) {
-	for (;;) {
 		uint8_t link_status[DP_LINK_STATUS_SIZE];
 		drm_dp_link_train_clock_recovery_delay(intel_dp->dpcd);
 			return false;
 		}
-		if (max_vswing_tries == 1) {
+		if (max_vswing_reached) {
 			DRM_DEBUG_KMS("Max Voltage Swing reached\n");
 			return false;
 		}
 			voltage_tries = 1;
 		if (intel_dp_link_max_vswing_reached(intel_dp))
-			++max_vswing_tries;
+			max_vswing_reached = true;
 	}
+	DRM_ERROR("Failed clock recovery %d times, giving up!\n", max_cr_tries);
+	return false;
 }
 /*

drivers/gpu/drm/i915/intel_dp_mst.c+7−7

 	struct intel_connector *connector =
 		to_intel_connector(old_conn_state->connector);
+	intel_ddi_disable_pipe_clock(old_crtc_state);
 	/* this can fail */
 	drm_dp_check_act_status(&intel_dp->mst_mgr);
 	/* and this can also fail */
 				       connector->port,
 				       pipe_config->pbn,
 				       pipe_config->dp_m_n.tu);
-	if (ret == false) {
+	if (!ret)
 		DRM_ERROR("failed to allocate vcpi\n");
-		return;
-	}
 	intel_dp->active_mst_links++;
 	temp = I915_READ(DP_TP_STATUS(port));
 	I915_WRITE(DP_TP_STATUS(port), temp);
 	ret = drm_dp_update_payload_part1(&intel_dp->mst_mgr);
+	intel_ddi_enable_pipe_clock(pipe_config);
 }
 static void intel_mst_enable_dp(struct intel_encoder *encoder,
 	struct intel_dp *intel_dp = &intel_dig_port->dp;
 	struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
 	enum port port = intel_dig_port->base.port;
-	int ret;
 	DRM_DEBUG_KMS("active links %d\n", intel_dp->active_mst_links);
 				    1))
 		DRM_ERROR("Timed out waiting for ACT sent\n");
-	ret = drm_dp_check_act_status(&intel_dp->mst_mgr);
+	drm_dp_check_act_status(&intel_dp->mst_mgr);
-	ret = drm_dp_update_payload_part2(&intel_dp->mst_mgr);
+	drm_dp_update_payload_part2(&intel_dp->mst_mgr);
 	if (pipe_config->has_audio)
 		intel_audio_codec_enable(encoder, pipe_config, conn_state);
 }

drivers/gpu/drm/i915/intel_dpll_mgr.c+46−16

 	params->dco_fraction = dco & 0x7fff;
 }
+int cnl_hdmi_pll_ref_clock(struct drm_i915_private *dev_priv)
+{
+	int ref_clock = dev_priv->cdclk.hw.ref;
+	/*
+	 * For ICL+, the spec states: if reference frequency is 38.4,
+	 * use 19.2 because the DPLL automatically divides that by 2.
+	 */
+	if (INTEL_GEN(dev_priv) >= 11 && ref_clock == 38400)
+		ref_clock = 19200;
+	return ref_clock;
+}
 static bool
 cnl_ddi_calculate_wrpll(int clock,
 			struct drm_i915_private *dev_priv,
 	cnl_wrpll_get_multipliers(best_div, &pdiv, &qdiv, &kdiv);
-	ref_clock = dev_priv->cdclk.hw.ref;
+	ref_clock = cnl_hdmi_pll_ref_clock(dev_priv);
-	/*
-	 * For ICL, the spec states: if reference frequency is 38.4, use 19.2
-	 * because the DPLL automatically divides that by 2.
-	 */
-	if (IS_ICELAKE(dev_priv) && ref_clock == 38400)
-		ref_clock = 19200;
 	cnl_wrpll_params_populate(wrpll_params, best_dco, ref_clock, pdiv, qdiv,
 				  kdiv);
 	  .pdiv = 0x1 /* 2 */, .kdiv = 1, .qdiv_mode = 0, .qdiv_ratio = 0},
 };
+static const struct skl_wrpll_params icl_tbt_pll_24MHz_values = {
+	.dco_integer = 0x151, .dco_fraction = 0x4000,
+	.pdiv = 0x4 /* 5 */, .kdiv = 1, .qdiv_mode = 0, .qdiv_ratio = 0,
+};
+static const struct skl_wrpll_params icl_tbt_pll_19_2MHz_values = {
+	.dco_integer = 0x1A5, .dco_fraction = 0x7000,
+	.pdiv = 0x4 /* 5 */, .kdiv = 1, .qdiv_mode = 0, .qdiv_ratio = 0,
+};
 static bool icl_calc_dp_combo_pll(struct drm_i915_private *dev_priv, int clock,
 				  struct skl_wrpll_params *pll_params)
 {
 	return true;
 }
+static bool icl_calc_tbt_pll(struct drm_i915_private *dev_priv, int clock,
+			     struct skl_wrpll_params *pll_params)
+{
+	*pll_params = dev_priv->cdclk.hw.ref == 24000 ?
+			icl_tbt_pll_24MHz_values : icl_tbt_pll_19_2MHz_values;
+	return true;
+}
 static bool icl_calc_dpll_state(struct intel_crtc_state *crtc_state,
 				struct intel_encoder *encoder, int clock,
 				struct intel_dpll_hw_state *pll_state)
 	struct skl_wrpll_params pll_params = { 0 };
 	bool ret;
-	if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
+	if (intel_port_is_tc(dev_priv, encoder->port))
+		ret = icl_calc_tbt_pll(dev_priv, clock, &pll_params);
+	else if (intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI))
 		ret = cnl_ddi_calculate_wrpll(clock, dev_priv, &pll_params);
 	else
 		ret = icl_calc_dp_combo_pll(dev_priv, clock, &pll_params);
 		for (div2 = 10; div2 > 0; div2--) {
 			int dco = div1 * div2 * clock_khz * 5;
-			int a_divratio, tlinedrv, inputsel, hsdiv;
+			int a_divratio, tlinedrv, inputsel;
+			u32 hsdiv;
 			if (dco < dco_min_freq || dco > dco_max_freq)
 				continue;
 				MISSING_CASE(div1);
 				/* fall through */
 			case 2:
-				hsdiv = 0;
+				hsdiv = MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_2;
 				break;
 			case 3:
-				hsdiv = 1;
+				hsdiv = MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_3;
 				break;
 			case 5:
-				hsdiv = 2;
+				hsdiv = MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_5;
 				break;
 			case 7:
-				hsdiv = 3;
+				hsdiv = MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO_7;
 				break;
 			}
 			state->mg_clktop2_hsclkctl =
 				MG_CLKTOP2_HSCLKCTL_TLINEDRV_CLKSEL(tlinedrv) |
 				MG_CLKTOP2_HSCLKCTL_CORE_INPUTSEL(inputsel) |
-				MG_CLKTOP2_HSCLKCTL_HSDIV_RATIO(hsdiv) |
+				hsdiv |
 				MG_CLKTOP2_HSCLKCTL_DSDIV_RATIO(div2);
 			return true;
 icl_get_dpll(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state,
 	     struct intel_encoder *encoder)
 {
+	struct intel_digital_port *intel_dig_port =
+			enc_to_dig_port(&encoder->base);
 	struct intel_shared_dpll *pll;
 	struct intel_dpll_hw_state pll_state = {};
 	enum port port = encoder->port;
 	case PORT_D:
 	case PORT_E:
 	case PORT_F:
-		if (0 /* TODO: TBT PLLs */) {
+		if (intel_dig_port->tc_type == TC_PORT_TBT) {
 			min = DPLL_ID_ICL_TBTPLL;
 			max = min;
 			ret = icl_calc_dpll_state(crtc_state, encoder, clock,

drivers/gpu/drm/i915/intel_dpll_mgr.h+1−0

@@ -344,5 +344,6 @@ void intel_dpll_dump_hw_state(struct drm_i915_private *dev_priv, · 343 unmodified lines
344	struct intel_dpll_hw_state *hw_state);	344	struct intel_dpll_hw_state *hw_state);
345	int icl_calc_dp_combo_pll_link(struct drm_i915_private *dev_priv,	345	int icl_calc_dp_combo_pll_link(struct drm_i915_private *dev_priv,
346	uint32_t pll_id);	346	uint32_t pll_id);
		347	int cnl_hdmi_pll_ref_clock(struct drm_i915_private *dev_priv);
347		348
348	#endif /* _INTEL_DPLL_MGR_H_ */	349	#endif /* _INTEL_DPLL_MGR_H_ */

drivers/gpu/drm/i915/intel_drv.h+33−10

 	void (*disable_plane)(struct intel_plane *plane,
 			      struct intel_crtc *crtc);
 	bool (*get_hw_state)(struct intel_plane *plane, enum pipe *pipe);
-	int (*check_plane)(struct intel_plane *plane,
+	int (*check_plane)(struct intel_crtc_state *crtc_state,
-			   struct intel_crtc_state *crtc_state,
+			   struct intel_plane_state *plane_state);
-			   struct intel_plane_state *state);
 };
 struct intel_watermark_params {
 	bool release_cl2_override;
 	uint8_t max_lanes;
 	enum intel_display_power_domain ddi_io_power_domain;
+	enum tc_port_type tc_type;
 	void (*write_infoframe)(struct drm_encoder *encoder,
 				const struct intel_crtc_state *crtc_state,
 	return &dp_to_dig_port(intel_dp)->lspcon;
 }
+static inline struct drm_i915_private *
+dp_to_i915(struct intel_dp *intel_dp)
+{
+	return to_i915(dp_to_dig_port(intel_dp)->base.base.dev);
+}
 static inline struct intel_digital_port *
 hdmi_to_dig_port(struct intel_hdmi *intel_hdmi)
 {
 			       unsigned int frontbuffer_bits);
 void intel_edp_drrs_flush(struct drm_i915_private *dev_priv,
 			  unsigned int frontbuffer_bits);
+void icl_program_mg_dp_mode(struct intel_dp *intel_dp);
+void icl_enable_phy_clock_gating(struct intel_digital_port *dig_port);
+void icl_disable_phy_clock_gating(struct intel_digital_port *dig_port);
 void
 intel_dp_program_link_training_pattern(struct intel_dp *intel_dp,
 		      const struct intel_crtc_state *crtc_state);
 void intel_psr_disable(struct intel_dp *intel_dp,
 		      const struct intel_crtc_state *old_crtc_state);
+int intel_psr_set_debugfs_mode(struct drm_i915_private *dev_priv,
+			       struct drm_modeset_acquire_ctx *ctx,
+			       u64 value);
 void intel_psr_invalidate(struct drm_i915_private *dev_priv,
 			  unsigned frontbuffer_bits,
 			  enum fb_op_origin origin);
 void intel_psr_init(struct drm_i915_private *dev_priv);
 void intel_psr_compute_config(struct intel_dp *intel_dp,
 			      struct intel_crtc_state *crtc_state);
-void intel_psr_irq_control(struct drm_i915_private *dev_priv, bool debug);
+void intel_psr_irq_control(struct drm_i915_private *dev_priv, u32 debug);
 void intel_psr_irq_handler(struct drm_i915_private *dev_priv, u32 psr_iir);
 void intel_psr_short_pulse(struct intel_dp *intel_dp);
-int intel_psr_wait_for_idle(const struct intel_crtc_state *new_crtc_state);
+int intel_psr_wait_for_idle(const struct intel_crtc_state *new_crtc_state,
+			    u32 *out_value);
 /* intel_runtime_pm.c */
 int intel_power_domains_init(struct drm_i915_private *);
-void intel_power_domains_fini(struct drm_i915_private *);
+void intel_power_domains_cleanup(struct drm_i915_private *dev_priv);
 void intel_power_domains_init_hw(struct drm_i915_private *dev_priv, bool resume);
-void intel_power_domains_suspend(struct drm_i915_private *dev_priv);
+void intel_power_domains_fini_hw(struct drm_i915_private *dev_priv);
-void intel_power_domains_verify_state(struct drm_i915_private *dev_priv);
+void intel_power_domains_enable(struct drm_i915_private *dev_priv);
+void intel_power_domains_disable(struct drm_i915_private *dev_priv);
+enum i915_drm_suspend_mode {
+	I915_DRM_SUSPEND_IDLE,
+	I915_DRM_SUSPEND_MEM,
+	I915_DRM_SUSPEND_HIBERNATE,
+};
+void intel_power_domains_suspend(struct drm_i915_private *dev_priv,
+				 enum i915_drm_suspend_mode);
+void intel_power_domains_resume(struct drm_i915_private *dev_priv);
 void bxt_display_core_init(struct drm_i915_private *dev_priv, bool resume);
 void bxt_display_core_uninit(struct drm_i915_private *dev_priv);
 void intel_runtime_pm_enable(struct drm_i915_private *dev_priv);
+void intel_runtime_pm_disable(struct drm_i915_private *dev_priv);
 const char *
 intel_display_power_domain_str(enum intel_display_power_domain domain);
 void intel_runtime_pm_get_noresume(struct drm_i915_private *dev_priv);
 void intel_runtime_pm_put(struct drm_i915_private *dev_priv);
-void intel_display_set_init_power(struct drm_i915_private *dev, bool enable);
 void chv_phy_powergate_lanes(struct intel_encoder *encoder,
 			     bool override, unsigned int mask);
 bool chv_phy_powergate_ch(struct drm_i915_private *dev_priv, enum dpio_phy phy,

drivers/gpu/drm/i915/intel_engine_cs.c+41−52

@@ -513,7 +513,7 @@ int intel_engine_create_scratch(struct intel_engine_cs *engine, · 512 unmodified lines
513	goto err_unref;	513	goto err_unref;
514	}	514	}
515		515
516	ret = i915_vma_pin(vma, 0, 4096, PIN_GLOBAL \| PIN_HIGH);	516	ret = i915_vma_pin(vma, 0, 0, PIN_GLOBAL \| PIN_HIGH);
517	if (ret)	517	if (ret)
518	goto err_unref;	518	goto err_unref;
519		519
@@ -527,36 +527,19 @@ err_unref: · 7 unmodified lines
527		527
528	void intel_engine_cleanup_scratch(struct intel_engine_cs *engine)	528	void intel_engine_cleanup_scratch(struct intel_engine_cs *engine)
529	{	529	{
530	i915_vma_unpin_and_release(&engine->scratch);	530	i915_vma_unpin_and_release(&engine->scratch, 0);
531	}
532
533	static void cleanup_phys_status_page(struct intel_engine_cs *engine)
534	{
535	struct drm_i915_private *dev_priv = engine->i915;
536
537	if (!dev_priv->status_page_dmah)
538	return;
539
540	drm_pci_free(&dev_priv->drm, dev_priv->status_page_dmah);
541	engine->status_page.page_addr = NULL;
542	}	531	}
543		532
544	static void cleanup_status_page(struct intel_engine_cs *engine)	533	static void cleanup_status_page(struct intel_engine_cs *engine)
545	{	534	{
546	struct i915_vma *vma;	535	if (HWS_NEEDS_PHYSICAL(engine->i915)) {
547	struct drm_i915_gem_object *obj;	536	void *addr = fetch_and_zero(&engine->status_page.page_addr);
548
549	vma = fetch_and_zero(&engine->status_page.vma);
550	if (!vma)
551	return;
552
553	obj = vma->obj;
554		537
555	i915_vma_unpin(vma);	538	__free_page(virt_to_page(addr));
556	i915_vma_close(vma);	539	}
557		540
558	i915_gem_object_unpin_map(obj);	541	i915_vma_unpin_and_release(&engine->status_page.vma,
559	__i915_gem_object_release_unless_active(obj);	542	I915_VMA_RELEASE_MAP);
560	}	543	}
561		544
562	static int init_status_page(struct intel_engine_cs *engine)	545	static int init_status_page(struct intel_engine_cs *engine)
@@ -598,7 +581,7 @@ static int init_status_page(struct intel_engine_cs *engine) · 35 unmodified lines
598	flags \|= PIN_MAPPABLE;	581	flags \|= PIN_MAPPABLE;
599	else	582	else
600	flags \|= PIN_HIGH;	583	flags \|= PIN_HIGH;
601	ret = i915_vma_pin(vma, 0, 4096, flags);	584	ret = i915_vma_pin(vma, 0, 0, flags);
602	if (ret)	585	if (ret)
603	goto err;	586	goto err;
604		587
@@ -622,17 +605,18 @@ err: · 17 unmodified lines
622		605
623	static int init_phys_status_page(struct intel_engine_cs *engine)	606	static int init_phys_status_page(struct intel_engine_cs *engine)
624	{	607	{
625	struct drm_i915_private *dev_priv = engine->i915;	608	struct page *page;
626		609
627	GEM_BUG_ON(engine->id != RCS);	610	/*
628		611	* Though the HWS register does support 36bit addresses, historically
629	dev_priv->status_page_dmah =	612	* we have had hangs and corruption reported due to wild writes if
630	drm_pci_alloc(&dev_priv->drm, PAGE_SIZE, PAGE_SIZE);	613	* the HWS is placed above 4G.
631	if (!dev_priv->status_page_dmah)	614	*/
		615	page = alloc_page(GFP_KERNEL \| __GFP_DMA32 \| __GFP_ZERO);
		616	if (!page)
632	return -ENOMEM;	617	return -ENOMEM;
633		618
634	engine->status_page.page_addr = dev_priv->status_page_dmah->vaddr;	619	engine->status_page.page_addr = page_address(page);
635	memset(engine->status_page.page_addr, 0, PAGE_SIZE);
636		620
637	return 0;	621	return 0;
638	}	622	}
@@ -722,10 +706,7 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine) · 83 unmodified lines
722		706
723	intel_engine_cleanup_scratch(engine);	707	intel_engine_cleanup_scratch(engine);
724		708
725	if (HWS_NEEDS_PHYSICAL(engine->i915))	709	cleanup_status_page(engine);
726	cleanup_phys_status_page(engine);
727	else
728	cleanup_status_page(engine);
729		710
730	intel_engine_fini_breadcrumbs(engine);	711	intel_engine_fini_breadcrumbs(engine);
731	intel_engine_cleanup_cmd_parser(engine);	712	intel_engine_cleanup_cmd_parser(engine);
@@ -800,6 +781,16 @@ int intel_engine_stop_cs(struct intel_engine_cs *engine) · 68 unmodified lines
800	return err;	781	return err;
801	}	782	}
802		783
		784	void intel_engine_cancel_stop_cs(struct intel_engine_cs *engine)
		785	{
		786	struct drm_i915_private *dev_priv = engine->i915;
		787
		788	GEM_TRACE("%s\n", engine->name);
		789
		790	I915_WRITE_FW(RING_MI_MODE(engine->mmio_base),
		791	_MASKED_BIT_DISABLE(STOP_RING));
		792	}
		793
803	const char i915_cache_level_str(struct drm_i915_private i915, int type)	794	const char i915_cache_level_str(struct drm_i915_private i915, int type)
804	{	795	{
805	switch (type) {	796	switch (type) {
@@ -980,8 +971,7 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine) · 174 unmodified lines
980	return true;	971	return true;
981		972
982	/* Any inflight/incomplete requests? */	973	/* Any inflight/incomplete requests? */
983	if (!i915_seqno_passed(intel_engine_get_seqno(engine),	974	if (!intel_engine_signaled(engine, intel_engine_last_submit(engine)))
984	intel_engine_last_submit(engine)))
985	return false;	975	return false;
986		976
987	if (I915_SELFTEST_ONLY(engine->breadcrumbs.mock))	977	if (I915_SELFTEST_ONLY(engine->breadcrumbs.mock))
@@ -1348,20 +1338,19 @@ static void intel_engine_print_registers(const struct intel_engine_cs *engine, · 360 unmodified lines
1348		1338
1349	if (HAS_EXECLISTS(dev_priv)) {	1339	if (HAS_EXECLISTS(dev_priv)) {
1350	const u32 *hws = &engine->status_page.page_addr[I915_HWS_CSB_BUF0_INDEX];	1340	const u32 *hws = &engine->status_page.page_addr[I915_HWS_CSB_BUF0_INDEX];
1351	u32 ptr, read, write;
1352	unsigned int idx;	1341	unsigned int idx;
		1342	u8 read, write;
1353		1343
1354	drm_printf(m, "\tExeclist status: 0x%08x %08x\n",	1344	drm_printf(m, "\tExeclist status: 0x%08x %08x\n",
1355	I915_READ(RING_EXECLIST_STATUS_LO(engine)),	1345	I915_READ(RING_EXECLIST_STATUS_LO(engine)),
1356	I915_READ(RING_EXECLIST_STATUS_HI(engine)));	1346	I915_READ(RING_EXECLIST_STATUS_HI(engine)));
1357		1347
1358	ptr = I915_READ(RING_CONTEXT_STATUS_PTR(engine));	1348	read = execlists->csb_head;
1359	read = GEN8_CSB_READ_PTR(ptr);	1349	write = READ_ONCE(*execlists->csb_write);
1360	write = GEN8_CSB_WRITE_PTR(ptr);	1350
1361	drm_printf(m, "\tExeclist CSB read %d [%d cached], write %d [%d from hws], tasklet queued? %s (%s)\n",	1351	drm_printf(m, "\tExeclist CSB read %d, write %d [mmio:%d], tasklet queued? %s (%s)\n",
1362	read, execlists->csb_head,	1352	read, write,
1363	write,	1353	GEN8_CSB_WRITE_PTR(I915_READ(RING_CONTEXT_STATUS_PTR(engine))),
1364	intel_read_status_page(engine, intel_hws_csb_write_index(engine->i915)),
1365	yesno(test_bit(TASKLET_STATE_SCHED,	1354	yesno(test_bit(TASKLET_STATE_SCHED,
1366	&engine->execlists.tasklet.state)),	1355	&engine->execlists.tasklet.state)),
1367	enableddisabled(!atomic_read(&engine->execlists.tasklet.count)));	1356	enableddisabled(!atomic_read(&engine->execlists.tasklet.count)));
@@ -1373,12 +1362,12 @@ static void intel_engine_print_registers(const struct intel_engine_cs *engine, · 5 unmodified lines
1373	write += GEN8_CSB_ENTRIES;	1362	write += GEN8_CSB_ENTRIES;
1374	while (read < write) {	1363	while (read < write) {
1375	idx = ++read % GEN8_CSB_ENTRIES;	1364	idx = ++read % GEN8_CSB_ENTRIES;
1376	drm_printf(m, "\tExeclist CSB[%d]: 0x%08x [0x%08x in hwsp], context: %d [%d in hwsp]\n",	1365	drm_printf(m, "\tExeclist CSB[%d]: 0x%08x [mmio:0x%08x], context: %d [mmio:%d]\n",
1377	idx,	1366	idx,
1378	I915_READ(RING_CONTEXT_STATUS_BUF_LO(engine, idx)),
1379	hws[idx * 2],	1367	hws[idx * 2],
1380	I915_READ(RING_CONTEXT_STATUS_BUF_HI(engine, idx)),	1368	I915_READ(RING_CONTEXT_STATUS_BUF_LO(engine, idx)),
1381	hws[idx * 2 + 1]);	1369	hws[idx * 2 + 1],
		1370	I915_READ(RING_CONTEXT_STATUS_BUF_HI(engine, idx)));
1382	}	1371	}
1383		1372
1384	rcu_read_lock();	1373	rcu_read_lock();

drivers/gpu/drm/i915/intel_guc.c+46−56

 #include "intel_guc_submission.h"
 #include "i915_drv.h"
-static void guc_init_ggtt_pin_bias(struct intel_guc *guc);
 static void gen8_guc_raise_irq(struct intel_guc *guc)
 {
 	struct drm_i915_private *dev_priv = guc_to_i915(guc);
 static void guc_fini_wq(struct intel_guc *guc)
 {
-	struct drm_i915_private *dev_priv = guc_to_i915(guc);
+	struct workqueue_struct *wq;
-	if (HAS_LOGICAL_RING_PREEMPTION(dev_priv) &&
+	wq = fetch_and_zero(&guc->preempt_wq);
-	    USES_GUC_SUBMISSION(dev_priv))
+	if (wq)
-		destroy_workqueue(guc->preempt_wq);
+		destroy_workqueue(wq);
-	destroy_workqueue(guc->log.relay.flush_wq);
+	wq = fetch_and_zero(&guc->log.relay.flush_wq);
+	if (wq)
+		destroy_workqueue(wq);
 }
 int intel_guc_init_misc(struct intel_guc *guc)
 	struct drm_i915_private *i915 = guc_to_i915(guc);
 	int ret;
-	guc_init_ggtt_pin_bias(guc);
 	ret = guc_init_wq(guc);
 	if (ret)
 		return ret;
 	vaddr = i915_gem_object_pin_map(vma->obj, I915_MAP_WB);
 	if (IS_ERR(vaddr)) {
-		i915_vma_unpin_and_release(&vma);
+		i915_vma_unpin_and_release(&vma, 0);
 		return PTR_ERR(vaddr);
 	}
 static void guc_shared_data_destroy(struct intel_guc *guc)
 {
-	i915_gem_object_unpin_map(guc->shared_data->obj);
+	i915_vma_unpin_and_release(&guc->shared_data, I915_VMA_RELEASE_MAP);
-	i915_vma_unpin_and_release(&guc->shared_data);
 }
 int intel_guc_init(struct intel_guc *guc)
  *
  * ::
  *
- *     +==============> +====================+ <== GUC_GGTT_TOP
+ *     +===========> +====================+ <== FFFF_FFFF
- *     ^                |                    |
+ *     ^             |      Reserved      |
- *     |                |                    |
+ *     |             +====================+ <== GUC_GGTT_TOP
- *     |                |        DRAM        |
+ *     |             |                    |
- *     |                |       Memory       |
+ *     |             |        DRAM        |
- *     |                |                    |
+ *    GuC            |                    |
- *    GuC               |                    |
+ *  Address    +===> +====================+ <== GuC ggtt_pin_bias
- *  Address  +========> +====================+ <== WOPCM Top
+ *   Space     ^     |                    |
- *   Space   ^          |   HW contexts RSVD |
+ *     |       |     |                    |
- *     |     |          |        WOPCM       |
+ *     |      GuC    |        GuC         |
- *     |     |     +==> +--------------------+ <== GuC WOPCM Top
+ *     |     WOPCM   |       WOPCM        |
- *     |    GuC    ^    |                    |
+ *     |      Size   |                    |
- *     |    GGTT   |    |                    |
+ *     |       |     |                    |
- *     |    Pin   GuC   |        GuC         |
+ *     v       v     |                    |
- *     |    Bias WOPCM  |       WOPCM        |
+ *     +=======+===> +====================+ <== 0000_0000
- *     |     |    Size  |                    |
- *     |     |     |    |                    |
- *     v     v     v    |                    |
- *     +=====+=====+==> +====================+ <== GuC WOPCM Base
- *                      |   Non-GuC WOPCM    |
- *                      |   (HuC/Reserved)   |
- *                      +====================+ <== WOPCM Base
  *
- * The lower part of GuC Address Space [0, ggtt_pin_bias) is mapped to WOPCM
+ * The lower part of GuC Address Space [0, ggtt_pin_bias) is mapped to GuC WOPCM
  * while upper part of GuC Address Space [ggtt_pin_bias, GUC_GGTT_TOP) is mapped
- * to DRAM. The value of the GuC ggtt_pin_bias is determined by WOPCM size and
+ * to DRAM. The value of the GuC ggtt_pin_bias is the GuC WOPCM size.
- * actual GuC WOPCM size.
  */
-/**
- * guc_init_ggtt_pin_bias() - Initialize the GuC ggtt_pin_bias value.
- * @guc: intel_guc structure.
- *
- * This function will calculate and initialize the ggtt_pin_bias value based on
- * overall WOPCM size and GuC WOPCM size.
- */
-static void guc_init_ggtt_pin_bias(struct intel_guc *guc)
-{
-	struct drm_i915_private *i915 = guc_to_i915(guc);
-	GEM_BUG_ON(!i915->wopcm.size);
-	GEM_BUG_ON(i915->wopcm.size < i915->wopcm.guc.base);
-	guc->ggtt_pin_bias = i915->wopcm.size - i915->wopcm.guc.base;
-}
 /**
  * intel_guc_allocate_vma() - Allocate a GGTT VMA for GuC usage
  * @guc:	the guc
 	struct drm_i915_private *dev_priv = guc_to_i915(guc);
 	struct drm_i915_gem_object *obj;
 	struct i915_vma *vma;
+	u64 flags;
 	int ret;
 	obj = i915_gem_object_create(dev_priv, size);
 	if (IS_ERR(vma))
 		goto err;
-	ret = i915_vma_pin(vma, 0, PAGE_SIZE,
+	flags = PIN_GLOBAL | PIN_OFFSET_BIAS | i915_ggtt_pin_bias(vma);
-			   PIN_GLOBAL | PIN_OFFSET_BIAS | guc->ggtt_pin_bias);
+	ret = i915_vma_pin(vma, 0, 0, flags);
 	if (ret) {
 		vma = ERR_PTR(ret);
 		goto err;
 	i915_gem_object_put(obj);
 	return vma;
 }
+/**
+ * intel_guc_reserved_gtt_size()
+ * @guc:	intel_guc structure
+ *
+ * The GuC WOPCM mapping shadows the lower part of the GGTT, so if we are using
+ * GuC we can't have any objects pinned in that region. This function returns
+ * the size of the shadowed region.
+ *
+ * Returns:
+ * 0 if GuC is not present or not in use.
+ * Otherwise, the GuC WOPCM size.
+ */
+u32 intel_guc_reserved_gtt_size(struct intel_guc *guc)
+{
+	return guc_to_i915(guc)->wopcm.guc.size;
+}

drivers/gpu/drm/i915/intel_guc.h+5−7

 	struct intel_guc_log log;
 	struct intel_guc_ct ct;
-	/* Offset where Non-WOPCM memory starts. */
-	u32 ggtt_pin_bias;
 	/* Log snapshot if GuC errors during load */
 	struct drm_i915_gem_object *load_err_log;
  * @vma: i915 graphics virtual memory area.
  *
  * GuC does not allow any gfx GGTT address that falls into range
- * [0, GuC ggtt_pin_bias), which is reserved for Boot ROM, SRAM and WOPCM.
+ * [0, ggtt.pin_bias), which is reserved for Boot ROM, SRAM and WOPCM.
- * Currently, in order to exclude [0, GuC ggtt_pin_bias) address space from
+ * Currently, in order to exclude [0, ggtt.pin_bias) address space from
  * GGTT, all gfx objects used by GuC are allocated with intel_guc_allocate_vma()
- * and pinned with PIN_OFFSET_BIAS along with the value of GuC ggtt_pin_bias.
+ * and pinned with PIN_OFFSET_BIAS along with the value of ggtt.pin_bias.
  *
  * Return: GGTT offset of the @vma.
  */
 {
 	u32 offset = i915_ggtt_offset(vma);
-	GEM_BUG_ON(offset < guc->ggtt_pin_bias);
+	GEM_BUG_ON(offset < i915_ggtt_pin_bias(vma));
 	GEM_BUG_ON(range_overflows_t(u64, offset, vma->size, GUC_GGTT_TOP));
 	return offset;
 int intel_guc_suspend(struct intel_guc *guc);
 int intel_guc_resume(struct intel_guc *guc);
 struct i915_vma *intel_guc_allocate_vma(struct intel_guc *guc, u32 size);
+u32 intel_guc_reserved_gtt_size(struct intel_guc *guc);
 static inline int intel_guc_sanitize(struct intel_guc *guc)
 {

drivers/gpu/drm/i915/intel_guc_ads.c+1−1

@@ -148,5 +148,5 @@ int intel_guc_ads_create(struct intel_guc *guc) · 147 unmodified lines
148		148
149	void intel_guc_ads_destroy(struct intel_guc *guc)	149	void intel_guc_ads_destroy(struct intel_guc *guc)
150	{	150	{
151	i915_vma_unpin_and_release(&guc->ads_vma);	151	i915_vma_unpin_and_release(&guc->ads_vma, 0);
152	}	152	}

drivers/gpu/drm/i915/intel_guc_ct.c+2−5

 	return 0;
 err_vma:
-	i915_vma_unpin_and_release(&ctch->vma);
+	i915_vma_unpin_and_release(&ctch->vma, 0);
 err_out:
 	CT_DEBUG_DRIVER("CT: channel %d initialization failed; err=%d\n",
 			ctch->owner, err);
 static void ctch_fini(struct intel_guc *guc,
 		      struct intel_guc_ct_channel *ctch)
 {
-	GEM_BUG_ON(!ctch->vma);
+	i915_vma_unpin_and_release(&ctch->vma, I915_VMA_RELEASE_MAP);
-	i915_gem_object_unpin_map(ctch->vma->obj);
-	i915_vma_unpin_and_release(&ctch->vma);
 }
 static int ctch_open(struct intel_guc *guc,

drivers/gpu/drm/i915/intel_guc_fwif.h+1−0

@@ -49,6 +49,7 @@ · 48 unmodified lines
49	#define WQ_TYPE_BATCH_BUF (0x1 << WQ_TYPE_SHIFT)	49	#define WQ_TYPE_BATCH_BUF (0x1 << WQ_TYPE_SHIFT)
50	#define WQ_TYPE_PSEUDO (0x2 << WQ_TYPE_SHIFT)	50	#define WQ_TYPE_PSEUDO (0x2 << WQ_TYPE_SHIFT)
51	#define WQ_TYPE_INORDER (0x3 << WQ_TYPE_SHIFT)	51	#define WQ_TYPE_INORDER (0x3 << WQ_TYPE_SHIFT)
		52	#define WQ_TYPE_NOOP (0x4 << WQ_TYPE_SHIFT)
52	#define WQ_TARGET_SHIFT 10	53	#define WQ_TARGET_SHIFT 10
53	#define WQ_LEN_SHIFT 16	54	#define WQ_LEN_SHIFT 16
54	#define WQ_NO_WCFLUSH_WAIT (1 << 27)	55	#define WQ_NO_WCFLUSH_WAIT (1 << 27)

drivers/gpu/drm/i915/intel_guc_log.c+1−1

@@ -498,7 +498,7 @@ err: · 497 unmodified lines
498		498
499	void intel_guc_log_destroy(struct intel_guc_log *log)	499	void intel_guc_log_destroy(struct intel_guc_log *log)
500	{	500	{
501	i915_vma_unpin_and_release(&log->vma);	501	i915_vma_unpin_and_release(&log->vma, 0);
502	}	502	}
503		503
504	int intel_guc_log_set_level(struct intel_guc_log *log, u32 level)	504	int intel_guc_log_set_level(struct intel_guc_log *log, u32 level)

drivers/gpu/drm/i915/intel_guc_submission.c+20−15

 	vaddr = i915_gem_object_pin_map(vma->obj, I915_MAP_WB);
 	if (IS_ERR(vaddr)) {
-		i915_vma_unpin_and_release(&vma);
+		i915_vma_unpin_and_release(&vma, 0);
 		return PTR_ERR(vaddr);
 	}
 static void guc_stage_desc_pool_destroy(struct intel_guc *guc)
 {
 	ida_destroy(&guc->stage_ids);
-	i915_gem_object_unpin_map(guc->stage_desc_pool->obj);
+	i915_vma_unpin_and_release(&guc->stage_desc_pool, I915_VMA_RELEASE_MAP);
-	i915_vma_unpin_and_release(&guc->stage_desc_pool);
 }
 /*
 	 */
 	BUILD_BUG_ON(wqi_size != 16);
+	/* We expect the WQ to be active if we're appending items to it */
+	GEM_BUG_ON(desc->wq_status != WQ_STATUS_ACTIVE);
 	/* Free space is guaranteed. */
 	wq_off = READ_ONCE(desc->tail);
 	GEM_BUG_ON(CIRC_SPACE(wq_off, READ_ONCE(desc->head),
 	/* WQ starts from the page after doorbell / process_desc */
 	wqi = client->vaddr + wq_off + GUC_DB_SIZE;
-	/* Now fill in the 4-word work queue item */
+	if (I915_SELFTEST_ONLY(client->use_nop_wqi)) {
-	wqi->header = WQ_TYPE_INORDER |
+		wqi->header = WQ_TYPE_NOOP | (wqi_len << WQ_LEN_SHIFT);
-		      (wqi_len << WQ_LEN_SHIFT) |
+	} else {
-		      (target_engine << WQ_TARGET_SHIFT) |
+		/* Now fill in the 4-word work queue item */
-		      WQ_NO_WCFLUSH_WAIT;
+		wqi->header = WQ_TYPE_INORDER |
-	wqi->context_desc = context_desc;
+			      (wqi_len << WQ_LEN_SHIFT) |
-	wqi->submit_element_info = ring_tail << WQ_RING_TAIL_SHIFT;
+			      (target_engine << WQ_TARGET_SHIFT) |
-	GEM_BUG_ON(ring_tail > WQ_RING_TAIL_MAX);
+			      WQ_NO_WCFLUSH_WAIT;
-	wqi->fence_id = fence_id;
+		wqi->context_desc = context_desc;
+		wqi->submit_element_info = ring_tail << WQ_RING_TAIL_SHIFT;
+		GEM_BUG_ON(ring_tail > WQ_RING_TAIL_MAX);
+		wqi->fence_id = fence_id;
+	}
 	/* Make the update visible to GuC */
 	WRITE_ONCE(desc->tail, (wq_off + wqi_size) & (GUC_WQ_SIZE - 1));
 err_vaddr:
 	i915_gem_object_unpin_map(client->vma->obj);
 err_vma:
-	i915_vma_unpin_and_release(&client->vma);
+	i915_vma_unpin_and_release(&client->vma, 0);
 err_id:
 	ida_simple_remove(&guc->stage_ids, client->stage_id);
 err_client:
 {
 	unreserve_doorbell(client);
 	guc_stage_desc_fini(client->guc, client);
-	i915_gem_object_unpin_map(client->vma->obj);
+	i915_vma_unpin_and_release(&client->vma, I915_VMA_RELEASE_MAP);
-	i915_vma_unpin_and_release(&client->vma);
 	ida_simple_remove(&client->guc->stage_ids, client->stage_id);
 	kfree(client);
 }

drivers/gpu/drm/i915/intel_guc_submission.h+4−0

 #include <linux/spinlock.h>
 #include "i915_gem.h"
+#include "i915_selftest.h"
 struct drm_i915_private;
 	spinlock_t wq_lock;
 	/* Per-engine counts of GuC submissions */
 	u64 submissions[I915_NUM_ENGINES];
+	/* For testing purposes, use nop WQ items instead of real ones */
+	I915_SELFTEST_DECLARE(bool use_nop_wqi);
 };
 int intel_guc_submission_init(struct intel_guc *guc);

drivers/gpu/drm/i915/intel_hangcheck.c+1−1

@@ -142,7 +142,7 @@ static int semaphore_passed(struct intel_engine_cs *engine) · 141 unmodified lines
142	if (signaller->hangcheck.deadlock >= I915_NUM_ENGINES)	142	if (signaller->hangcheck.deadlock >= I915_NUM_ENGINES)
143	return -1;	143	return -1;
144		144
145	if (i915_seqno_passed(intel_engine_get_seqno(signaller), seqno))	145	if (intel_engine_signaled(signaller, seqno))
146	return 1;	146	return 1;
147		147
148	/* cursory check for an unkickable deadlock */	148	/* cursory check for an unkickable deadlock */

drivers/gpu/drm/i915/intel_hdcp.c+3−3

@@ -57,9 +57,9 @@ static bool hdcp_key_loadable(struct drm_i915_private *dev_priv) · 56 unmodified lines
57		57
58	/* PG1 (power well #1) needs to be enabled */	58	/* PG1 (power well #1) needs to be enabled */
59	for_each_power_well(dev_priv, power_well) {	59	for_each_power_well(dev_priv, power_well) {
60	if (power_well->id == id) {	60	if (power_well->desc->id == id) {
61	enabled = power_well->ops->is_enabled(dev_priv,	61	enabled = power_well->desc->ops->is_enabled(dev_priv,
62	power_well);	62	power_well);
63	break;	63	break;
64	}	64	}
65	}	65	}

drivers/gpu/drm/i915/intel_hdmi.c+7−3

 static enum drm_connector_status
 intel_hdmi_detect(struct drm_connector *connector, bool force)
 {
-	enum drm_connector_status status;
+	enum drm_connector_status status = connector_status_disconnected;
 	struct drm_i915_private *dev_priv = to_i915(connector->dev);
 	struct intel_hdmi *intel_hdmi = intel_attached_hdmi(connector);
+	struct intel_encoder *encoder = &hdmi_to_dig_port(intel_hdmi)->base;
 	DRM_DEBUG_KMS("[CONNECTOR:%d:%s]\n",
 		      connector->base.id, connector->name);
 	intel_display_power_get(dev_priv, POWER_DOMAIN_GMBUS);
+	if (IS_ICELAKE(dev_priv) &&
+	    !intel_digital_port_connected(encoder))
+		goto out;
 	intel_hdmi_unset_edid(connector);
 	if (intel_hdmi_set_edid(connector))
 		status = connector_status_connected;
-	else
-		status = connector_status_disconnected;
+out:
 	intel_display_power_put(dev_priv, POWER_DOMAIN_GMBUS);
 	if (status != connector_status_connected)

drivers/gpu/drm/i915/intel_huc.c+1−1

@@ -63,7 +63,7 @@ int intel_huc_auth(struct intel_huc *huc) · 62 unmodified lines
63	return -ENOEXEC;	63	return -ENOEXEC;
64		64
65	vma = i915_gem_object_ggtt_pin(huc->fw.obj, NULL, 0, 0,	65	vma = i915_gem_object_ggtt_pin(huc->fw.obj, NULL, 0, 0,
66	PIN_OFFSET_BIAS \| guc->ggtt_pin_bias);	66	PIN_OFFSET_BIAS \| i915->ggtt.pin_bias);
67	if (IS_ERR(vma)) {	67	if (IS_ERR(vma)) {
68	ret = PTR_ERR(vma);	68	ret = PTR_ERR(vma);
69	DRM_ERROR("HuC: Failed to pin huc fw object %d\n", ret);	69	DRM_ERROR("HuC: Failed to pin huc fw object %d\n", ret);

drivers/gpu/drm/i915/intel_i2c.c+8−8

 struct gmbus_pin {
 	const char *name;
-	i915_reg_t reg;
+	enum i915_gpio gpio;
 };
 /* Map gmbus pin pairs to names and registers. */
 	else
 		size = ARRAY_SIZE(gmbus_pins);
-	return pin < size &&
+	return pin < size && get_gmbus_pin(dev_priv, pin)->name;
-		i915_mmio_reg_valid(get_gmbus_pin(dev_priv, pin)->reg);
 }
 /* Intel GPIO access functions */
 	algo = &bus->bit_algo;
-	bus->gpio_reg = _MMIO(dev_priv->gpio_mmio_base +
+	bus->gpio_reg = GPIO(get_gmbus_pin(dev_priv, pin)->gpio);
-			      i915_mmio_reg_offset(get_gmbus_pin(dev_priv, pin)->reg));
 	bus->adapter.algo_data = algo;
 	algo->setsda = set_data;
 	algo->setscl = set_clock;
 	if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
 		dev_priv->gpio_mmio_base = VLV_DISPLAY_BASE;
 	else if (!HAS_GMCH_DISPLAY(dev_priv))
-		dev_priv->gpio_mmio_base =
+		/*
-			i915_mmio_reg_offset(PCH_GPIOA) -
+		 * Broxton uses the same PCH offsets for South Display Engine,
-			i915_mmio_reg_offset(GPIOA);
+		 * even though it doesn't have a PCH.
+		 */
+		dev_priv->gpio_mmio_base = PCH_DISPLAY_BASE;
 	mutex_init(&dev_priv->gmbus_mutex);
 	init_waitqueue_head(&dev_priv->gmbus_wait_queue);

drivers/gpu/drm/i915/intel_lrc.c+110−49

@@ -541,11 +541,6 @@ static void inject_preempt_context(struct intel_engine_cs *engine) · 540 unmodified lines
541		541
542	GEM_BUG_ON(execlists->preempt_complete_status !=	542	GEM_BUG_ON(execlists->preempt_complete_status !=
543	upper_32_bits(ce->lrc_desc));	543	upper_32_bits(ce->lrc_desc));
544	GEM_BUG_ON((ce->lrc_reg_state[CTX_CONTEXT_CONTROL + 1] &
545	_MASKED_BIT_ENABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT \|
546	CTX_CTRL_ENGINE_CTX_SAVE_INHIBIT)) !=
547	_MASKED_BIT_ENABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT \|
548	CTX_CTRL_ENGINE_CTX_SAVE_INHIBIT));
549		544
550	/*	545	/*
551	* Switch to our empty preempt context so	546	* Switch to our empty preempt context so
@@ -1277,6 +1272,8 @@ static void execlists_context_destroy(struct intel_context *ce) · 725 unmodified lines
1277		1272
1278	static void execlists_context_unpin(struct intel_context *ce)	1273	static void execlists_context_unpin(struct intel_context *ce)
1279	{	1274	{
		1275	i915_gem_context_unpin_hw_id(ce->gem_context);
		1276
1280	intel_ring_unpin(ce->ring);	1277	intel_ring_unpin(ce->ring);
1281		1278
1282	ce->state->obj->pin_global--;	1279	ce->state->obj->pin_global--;
@@ -1303,10 +1300,9 @@ static int __context_pin(struct i915_gem_context ctx, struct i915_vma vma) · 20 unmodified lines
1303	}	1300	}
1304		1301
1305	flags = PIN_GLOBAL \| PIN_HIGH;	1302	flags = PIN_GLOBAL \| PIN_HIGH;
1306	if (ctx->ggtt_offset_bias)	1303	flags \|= PIN_OFFSET_BIAS \| i915_ggtt_pin_bias(vma);
1307	flags \|= PIN_OFFSET_BIAS \| ctx->ggtt_offset_bias;
1308		1304
1309	return i915_vma_pin(vma, 0, GEN8_LR_CONTEXT_ALIGN, flags);	1305	return i915_vma_pin(vma, 0, 0, flags);
1310	}	1306	}
1311		1307
1312	static struct intel_context *	1308	static struct intel_context *
@@ -1332,10 +1328,14 @@ __execlists_context_pin(struct intel_engine_cs *engine, · 19 unmodified lines
1332	goto unpin_vma;	1328	goto unpin_vma;
1333	}	1329	}
1334		1330
1335	ret = intel_ring_pin(ce->ring, ctx->i915, ctx->ggtt_offset_bias);	1331	ret = intel_ring_pin(ce->ring);
1336	if (ret)	1332	if (ret)
1337	goto unpin_map;	1333	goto unpin_map;
1338		1334
		1335	ret = i915_gem_context_pin_hw_id(ctx);
		1336	if (ret)
		1337	goto unpin_ring;
		1338
1339	intel_lr_context_descriptor_update(ctx, engine, ce);	1339	intel_lr_context_descriptor_update(ctx, engine, ce);
1340		1340
1341	ce->lrc_reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;	1341	ce->lrc_reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
@@ -1348,6 +1348,8 @@ __execlists_context_pin(struct intel_engine_cs *engine, · 6 unmodified lines
1348	i915_gem_context_get(ctx);	1348	i915_gem_context_get(ctx);
1349	return ce;	1349	return ce;
1350		1350
		1351	unpin_ring:
		1352	intel_ring_unpin(ce->ring);
1351	unpin_map:	1353	unpin_map:
1352	i915_gem_object_unpin_map(ce->state->obj);	1354	i915_gem_object_unpin_map(ce->state->obj);
1353	unpin_vma:	1355	unpin_vma:
@@ -1643,7 +1645,7 @@ static int lrc_setup_wa_ctx(struct intel_engine_cs *engine) · 289 unmodified lines
1643	goto err;	1645	goto err;
1644	}	1646	}
1645		1647
1646	err = i915_vma_pin(vma, 0, PAGE_SIZE, PIN_GLOBAL \| PIN_HIGH);	1648	err = i915_vma_pin(vma, 0, 0, PIN_GLOBAL \| PIN_HIGH);
1647	if (err)	1649	if (err)
1648	goto err;	1650	goto err;
1649		1651
@@ -1657,7 +1659,7 @@ err: · 7 unmodified lines
1657		1659
1658	static void lrc_destroy_wa_ctx(struct intel_engine_cs *engine)	1660	static void lrc_destroy_wa_ctx(struct intel_engine_cs *engine)
1659	{	1661	{
1660	i915_vma_unpin_and_release(&engine->wa_ctx.vma);	1662	i915_vma_unpin_and_release(&engine->wa_ctx.vma, 0);
1661	}	1663	}
1662		1664
1663	typedef u32 (wa_bb_func_t)(struct intel_engine_cs engine, u32 batch);	1665	typedef u32 (wa_bb_func_t)(struct intel_engine_cs engine, u32 batch);
@@ -1775,11 +1777,7 @@ static bool unexpected_starting_state(struct intel_engine_cs *engine) · 111 unmodified lines
1775		1777
1776	static int gen8_init_common_ring(struct intel_engine_cs *engine)	1778	static int gen8_init_common_ring(struct intel_engine_cs *engine)
1777	{	1779	{
1778	int ret;	1780	intel_mocs_init_engine(engine);
1779
1780	ret = intel_mocs_init_engine(engine);
1781	if (ret)
1782	return ret;
1783		1781
1784	intel_engine_reset_breadcrumbs(engine);	1782	intel_engine_reset_breadcrumbs(engine);
1785		1783
@@ -1838,7 +1836,8 @@ execlists_reset_prepare(struct intel_engine_cs *engine) · 52 unmodified lines
1838	struct i915_request request, active;	1836	struct i915_request request, active;
1839	unsigned long flags;	1837	unsigned long flags;
1840		1838
1841	GEM_TRACE("%s\n", engine->name);	1839	GEM_TRACE("%s: depth<-%d\n", engine->name,
		1840	atomic_read(&execlists->tasklet.count));
1842		1841
1843	/*	1842	/*
1844	* Prevent request submission to the hardware until we have	1843	* Prevent request submission to the hardware until we have
@@ -1971,22 +1970,18 @@ static void execlists_reset_finish(struct intel_engine_cs *engine) · 126 unmodified lines
1971	{	1970	{
1972	struct intel_engine_execlists * const execlists = &engine->execlists;	1971	struct intel_engine_execlists * const execlists = &engine->execlists;
1973		1972
1974	/* After a GPU reset, we may have requests to replay */
1975	if (!RB_EMPTY_ROOT(&execlists->queue.rb_root))
1976	tasklet_schedule(&execlists->tasklet);
1977
1978	/*	1973	/*
1979	* Flush the tasklet while we still have the forcewake to be sure	1974	* After a GPU reset, we may have requests to replay. Do so now while
1980	* that it is not allowed to sleep before we restart and reload a	1975	* we still have the forcewake to be sure that the GPU is not allowed
1981	* context.	1976	* to sleep before we restart and reload a context.
1982	*	1977	*
1983	* As before (with execlists_reset_prepare) we rely on the caller
1984	* serialising multiple attempts to reset so that we know that we
1985	* are the only one manipulating tasklet state.
1986	*/	1978	*/
1987	__tasklet_enable_sync_once(&execlists->tasklet);	1979	if (!RB_EMPTY_ROOT(&execlists->queue.rb_root))
		1980	execlists->tasklet.func(execlists->tasklet.data);
1988		1981
1989	GEM_TRACE("%s\n", engine->name);	1982	tasklet_enable(&execlists->tasklet);
		1983	GEM_TRACE("%s: depth->%d\n", engine->name,
		1984	atomic_read(&execlists->tasklet.count));
1990	}	1985	}
1991		1986
1992	static int intel_logical_ring_emit_pdps(struct i915_request *rq)	1987	static int intel_logical_ring_emit_pdps(struct i915_request *rq)
@@ -2066,8 +2061,7 @@ static int gen8_emit_bb_start(struct i915_request *rq, · 73 unmodified lines
2066		2061
2067	/* FIXME(BDW): Address space and security selectors. */	2062	/* FIXME(BDW): Address space and security selectors. */
2068	*cs++ = MI_BATCH_BUFFER_START_GEN8 \|	2063	*cs++ = MI_BATCH_BUFFER_START_GEN8 \|
2069	(flags & I915_DISPATCH_SECURE ? 0 : BIT(8)) \|	2064	(flags & I915_DISPATCH_SECURE ? 0 : BIT(8));
2070	(flags & I915_DISPATCH_RS ? MI_BATCH_RESOURCE_STREAMER : 0);
2071	*cs++ = lower_32_bits(offset);	2065	*cs++ = lower_32_bits(offset);
2072	*cs++ = upper_32_bits(offset);	2066	*cs++ = upper_32_bits(offset);
2073		2067
@@ -2494,6 +2488,9 @@ int logical_xcs_ring_init(struct intel_engine_cs *engine) · 420 unmodified lines
2494	static u32	2488	static u32
2495	make_rpcs(struct drm_i915_private *dev_priv)	2489	make_rpcs(struct drm_i915_private *dev_priv)
2496	{	2490	{
		2491	bool subslice_pg = INTEL_INFO(dev_priv)->sseu.has_subslice_pg;
		2492	u8 slices = hweight8(INTEL_INFO(dev_priv)->sseu.slice_mask);
		2493	u8 subslices = hweight8(INTEL_INFO(dev_priv)->sseu.subslice_mask[0]);
2497	u32 rpcs = 0;	2494	u32 rpcs = 0;
2498		2495
2499	/*	2496	/*
@@ -2503,6 +2500,38 @@ make_rpcs(struct drm_i915_private *dev_priv) · 3 unmodified lines
2503	if (INTEL_GEN(dev_priv) < 9)	2500	if (INTEL_GEN(dev_priv) < 9)
2504	return 0;	2501	return 0;
2505		2502
		2503	/*
		2504	* Since the SScount bitfield in GEN8_R_PWR_CLK_STATE is only three bits
		2505	* wide and Icelake has up to eight subslices, specfial programming is
		2506	* needed in order to correctly enable all subslices.
		2507	*
		2508	* According to documentation software must consider the configuration
		2509	* as 2x4x8 and hardware will translate this to 1x8x8.
		2510	*
		2511	* Furthemore, even though SScount is three bits, maximum documented
		2512	* value for it is four. From this some rules/restrictions follow:
		2513	*
		2514	* 1.
		2515	* If enabled subslice count is greater than four, two whole slices must
		2516	* be enabled instead.
		2517	*
		2518	* 2.
		2519	* When more than one slice is enabled, hardware ignores the subslice
		2520	* count altogether.
		2521	*
		2522	* From these restrictions it follows that it is not possible to enable
		2523	* a count of subslices between the SScount maximum of four restriction,
		2524	* and the maximum available number on a particular SKU. Either all
		2525	* subslices are enabled, or a count between one and four on the first
		2526	* slice.
		2527	*/
		2528	if (IS_GEN11(dev_priv) && slices == 1 && subslices >= 4) {
		2529	GEM_BUG_ON(subslices & 1);
		2530
		2531	subslice_pg = false;
		2532	slices *= 2;
		2533	}
		2534
2506	/*	2535	/*
2507	* Starting in Gen9, render power gating can leave	2536	* Starting in Gen9, render power gating can leave
2508	* slice/subslice/EU in a partially enabled state. We	2537	* slice/subslice/EU in a partially enabled state. We
@@ -2510,24 +2539,50 @@ make_rpcs(struct drm_i915_private *dev_priv) · 1 unmodified line
2510	* enablement.	2539	* enablement.
2511	*/	2540	*/
2512	if (INTEL_INFO(dev_priv)->sseu.has_slice_pg) {	2541	if (INTEL_INFO(dev_priv)->sseu.has_slice_pg) {
2513	rpcs \|= GEN8_RPCS_S_CNT_ENABLE;	2542	u32 mask, val = slices;
2514	rpcs \|= hweight8(INTEL_INFO(dev_priv)->sseu.slice_mask) <<	2543
2515	GEN8_RPCS_S_CNT_SHIFT;	2544	if (INTEL_GEN(dev_priv) >= 11) {
2516	rpcs \|= GEN8_RPCS_ENABLE;	2545	mask = GEN11_RPCS_S_CNT_MASK;
		2546	val <<= GEN11_RPCS_S_CNT_SHIFT;
		2547	} else {
		2548	mask = GEN8_RPCS_S_CNT_MASK;
		2549	val <<= GEN8_RPCS_S_CNT_SHIFT;
		2550	}
		2551
		2552	GEM_BUG_ON(val & ~mask);
		2553	val &= mask;
		2554
		2555	rpcs \|= GEN8_RPCS_ENABLE \| GEN8_RPCS_S_CNT_ENABLE \| val;
2517	}	2556	}
2518		2557
2519	if (INTEL_INFO(dev_priv)->sseu.has_subslice_pg) {	2558	if (subslice_pg) {
2520	rpcs \|= GEN8_RPCS_SS_CNT_ENABLE;	2559	u32 val = subslices;
2521	rpcs \|= hweight8(INTEL_INFO(dev_priv)->sseu.subslice_mask[0]) <<	2560
2522	GEN8_RPCS_SS_CNT_SHIFT;	2561	val <<= GEN8_RPCS_SS_CNT_SHIFT;
2523	rpcs \|= GEN8_RPCS_ENABLE;	2562
		2563	GEM_BUG_ON(val & ~GEN8_RPCS_SS_CNT_MASK);
		2564	val &= GEN8_RPCS_SS_CNT_MASK;
		2565
		2566	rpcs \|= GEN8_RPCS_ENABLE \| GEN8_RPCS_SS_CNT_ENABLE \| val;
2524	}	2567	}
2525		2568
2526	if (INTEL_INFO(dev_priv)->sseu.has_eu_pg) {	2569	if (INTEL_INFO(dev_priv)->sseu.has_eu_pg) {
2527	rpcs \|= INTEL_INFO(dev_priv)->sseu.eu_per_subslice <<	2570	u32 val;
2528	GEN8_RPCS_EU_MIN_SHIFT;	2571
2529	rpcs \|= INTEL_INFO(dev_priv)->sseu.eu_per_subslice <<	2572	val = INTEL_INFO(dev_priv)->sseu.eu_per_subslice <<
2530	GEN8_RPCS_EU_MAX_SHIFT;	2573	GEN8_RPCS_EU_MIN_SHIFT;
		2574	GEM_BUG_ON(val & ~GEN8_RPCS_EU_MIN_MASK);
		2575	val &= GEN8_RPCS_EU_MIN_MASK;
		2576
		2577	rpcs \|= val;
		2578
		2579	val = INTEL_INFO(dev_priv)->sseu.eu_per_subslice <<
		2580	GEN8_RPCS_EU_MAX_SHIFT;
		2581	GEM_BUG_ON(val & ~GEN8_RPCS_EU_MAX_MASK);
		2582	val &= GEN8_RPCS_EU_MAX_MASK;
		2583
		2584	rpcs \|= val;
		2585
2531	rpcs \|= GEN8_RPCS_ENABLE;	2586	rpcs \|= GEN8_RPCS_ENABLE;
2532	}	2587	}
2533		2588
@@ -2584,11 +2639,13 @@ static void execlists_init_reg_state(u32 *regs, · 50 unmodified lines
2584	MI_LRI_FORCE_POSTED;	2639	MI_LRI_FORCE_POSTED;
2585		2640
2586	CTX_REG(regs, CTX_CONTEXT_CONTROL, RING_CONTEXT_CONTROL(engine),	2641	CTX_REG(regs, CTX_CONTEXT_CONTROL, RING_CONTEXT_CONTROL(engine),
2587	_MASKED_BIT_DISABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT \|	2642	_MASKED_BIT_DISABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT) \|
2588	CTX_CTRL_ENGINE_CTX_SAVE_INHIBIT) \|	2643	_MASKED_BIT_ENABLE(CTX_CTRL_INHIBIT_SYN_CTX_SWITCH));
2589	_MASKED_BIT_ENABLE(CTX_CTRL_INHIBIT_SYN_CTX_SWITCH \|	2644	if (INTEL_GEN(dev_priv) < 11) {
2590	(HAS_RESOURCE_STREAMER(dev_priv) ?	2645	regs[CTX_CONTEXT_CONTROL + 1] \|=
2591	CTX_CTRL_RS_CTX_ENABLE : 0)));	2646	_MASKED_BIT_DISABLE(CTX_CTRL_ENGINE_CTX_SAVE_INHIBIT \|
		2647	CTX_CTRL_RS_CTX_ENABLE);
		2648	}
2592	CTX_REG(regs, CTX_RING_HEAD, RING_HEAD(base), 0);	2649	CTX_REG(regs, CTX_RING_HEAD, RING_HEAD(base), 0);
2593	CTX_REG(regs, CTX_RING_TAIL, RING_TAIL(base), 0);	2650	CTX_REG(regs, CTX_RING_TAIL, RING_TAIL(base), 0);
2594	CTX_REG(regs, CTX_RING_BUFFER_START, RING_START(base), 0);	2651	CTX_REG(regs, CTX_RING_BUFFER_START, RING_START(base), 0);
@@ -2654,6 +2711,10 @@ static void execlists_init_reg_state(u32 *regs, · 59 unmodified lines
2654		2711
2655	i915_oa_init_reg_state(engine, ctx, regs);	2712	i915_oa_init_reg_state(engine, ctx, regs);
2656	}	2713	}
		2714
		2715	regs[CTX_END] = MI_BATCH_BUFFER_END;
		2716	if (INTEL_GEN(dev_priv) >= 10)
		2717	regs[CTX_END] \|= BIT(0);
2657	}	2718	}
2658		2719
2659	static int	2720	static int

drivers/gpu/drm/i915/intel_lrc.h+0−2

@@ -27,8 +27,6 @@ · 26 unmodified lines
27	#include "intel_ringbuffer.h"	27	#include "intel_ringbuffer.h"
28	#include "i915_gem_context.h"	28	#include "i915_gem_context.h"
29		29
30	#define GEN8_LR_CONTEXT_ALIGN I915_GTT_MIN_ALIGNMENT
31
32	/* Execlists regs */	30	/* Execlists regs */
33	#define RING_ELSP(engine) _MMIO((engine)->mmio_base + 0x230)	31	#define RING_ELSP(engine) _MMIO((engine)->mmio_base + 0x230)
34	#define RING_EXECLIST_STATUS_LO(engine) _MMIO((engine)->mmio_base + 0x234)	32	#define RING_EXECLIST_STATUS_LO(engine) _MMIO((engine)->mmio_base + 0x234)

drivers/gpu/drm/i915/intel_lrc_reg.h+1−1

@@ -37,7 +37,7 @@ · 36 unmodified lines
37	#define CTX_PDP0_LDW 0x32	37	#define CTX_PDP0_LDW 0x32
38	#define CTX_LRI_HEADER_2 0x41	38	#define CTX_LRI_HEADER_2 0x41
39	#define CTX_R_PWR_CLK_STATE 0x42	39	#define CTX_R_PWR_CLK_STATE 0x42
40	#define CTX_GPGPU_CSR_BASE_ADDRESS 0x44	40	#define CTX_END 0x44
41		41
42	#define CTX_REG(reg_state, pos, reg, val) do { \	42	#define CTX_REG(reg_state, pos, reg, val) do { \
43	u32 *reg_state__ = (reg_state); \	43	u32 *reg_state__ = (reg_state); \

drivers/gpu/drm/i915/intel_mocs.c+3−8

  *
  * This function simply emits a MI_LOAD_REGISTER_IMM command for the
  * given table starting at the given address.
- *
- * Return: 0 on success, otherwise the error status.
  */
-int intel_mocs_init_engine(struct intel_engine_cs *engine)
+void intel_mocs_init_engine(struct intel_engine_cs *engine)
 {
 	struct drm_i915_private *dev_priv = engine->i915;
 	struct drm_i915_mocs_table table;
 	unsigned int index;
 	if (!get_mocs_settings(dev_priv, &table))
-		return 0;
+		return;
-	if (WARN_ON(table.size > GEN9_NUM_MOCS_ENTRIES))
+	GEM_BUG_ON(table.size > GEN9_NUM_MOCS_ENTRIES);
-		return -ENODEV;
 	for (index = 0; index < table.size; index++)
 		I915_WRITE(mocs_register(engine->id, index),
 	for (; index < GEN9_NUM_MOCS_ENTRIES; index++)
 		I915_WRITE(mocs_register(engine->id, index),
 			   table.table[0].control_value);
-	return 0;
 }
 /**

drivers/gpu/drm/i915/intel_mocs.h+1−1

@@ -54,6 +54,6 @@ · 53 unmodified lines
54		54
55	int intel_rcs_context_init_mocs(struct i915_request *rq);	55	int intel_rcs_context_init_mocs(struct i915_request *rq);
56	void intel_mocs_init_l3cc_table(struct drm_i915_private *dev_priv);	56	void intel_mocs_init_l3cc_table(struct drm_i915_private *dev_priv);
57	int intel_mocs_init_engine(struct intel_engine_cs *engine);	57	void intel_mocs_init_engine(struct intel_engine_cs *engine);
58		58
59	#endif	59	#endif

drivers/gpu/drm/i915/intel_pm.c+62−39

  */
 #include <linux/cpufreq.h>
+#include <linux/pm_runtime.h>
 #include <drm/drm_plane_helper.h>
 #include "i915_drv.h"
 #include "intel_drv.h"
 		unsigned int latency = wm[level];
 		if (latency == 0) {
-			DRM_ERROR("%s WM%d latency not provided\n",
+			DRM_DEBUG_KMS("%s WM%d latency not provided\n",
-				  name, level);
+				      name, level);
 			continue;
 		}
 	return true;
 }
-static unsigned int intel_get_ddb_size(struct drm_i915_private *dev_priv,
+static u16 intel_get_ddb_size(struct drm_i915_private *dev_priv,
-				       const struct intel_crtc_state *cstate,
+			      const struct intel_crtc_state *cstate,
-				       const unsigned int total_data_rate,
+			      const unsigned int total_data_rate,
-				       const int num_active,
+			      const int num_active,
-				       struct skl_ddb_allocation *ddb)
+			      struct skl_ddb_allocation *ddb)
 {
 	const struct drm_display_mode *adjusted_mode;
 	u64 total_data_bw;
 	struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
 	struct drm_i915_private *dev_priv = to_i915(dev);
 	struct drm_crtc *for_crtc = cstate->base.crtc;
-	unsigned int pipe_size, ddb_size;
+	const struct drm_crtc_state *crtc_state;
-	int nth_active_pipe;
+	const struct drm_crtc *crtc;
+	u32 pipe_width = 0, total_width = 0, width_before_pipe = 0;
+	enum pipe for_pipe = to_intel_crtc(for_crtc)->pipe;
+	u16 ddb_size;
+	u32 i;
 	if (WARN_ON(!state) || !cstate->base.active) {
 		alloc->start = 0;
 				      *num_active, ddb);
 	/*
-	 * If the state doesn't change the active CRTC's, then there's
+	 * If the state doesn't change the active CRTC's or there is no
-	 * no need to recalculate; the existing pipe allocation limits
+	 * modeset request, then there's no need to recalculate;
-	 * should remain unchanged.  Note that we're safe from racing
+	 * the existing pipe allocation limits should remain unchanged.
-	 * commits since any racing commit that changes the active CRTC
+	 * Note that we're safe from racing commits since any racing commit
-	 * list would need to grab _all_ crtc locks, including the one
+	 * that changes the active CRTC list or do modeset would need to
-	 * we currently hold.
+	 * grab _all_ crtc locks, including the one we currently hold.
 	 */
-	if (!intel_state->active_pipe_changes) {
+	if (!intel_state->active_pipe_changes && !intel_state->modeset) {
 		/*
 		 * alloc may be cleared by clear_intel_crtc_state,
 		 * copy from old state to be sure
 		return;
 	}
-	nth_active_pipe = hweight32(intel_state->active_crtcs &
+	/*
-				    (drm_crtc_mask(for_crtc) - 1));
+	 * Watermark/ddb requirement highly depends upon width of the
-	pipe_size = ddb_size / hweight32(intel_state->active_crtcs);
+	 * framebuffer, So instead of allocating DDB equally among pipes
-	alloc->start = nth_active_pipe * ddb_size / *num_active;
+	 * distribute DDB based on resolution/width of the display.
-	alloc->end = alloc->start + pipe_size;
+	 */
+	for_each_new_crtc_in_state(state, crtc, crtc_state, i) {
+		const struct drm_display_mode *adjusted_mode;
+		int hdisplay, vdisplay;
+		enum pipe pipe;
+		if (!crtc_state->enable)
+			continue;
+		pipe = to_intel_crtc(crtc)->pipe;
+		adjusted_mode = &crtc_state->adjusted_mode;
+		drm_mode_get_hv_timing(adjusted_mode, &hdisplay, &vdisplay);
+		total_width += hdisplay;
+		if (pipe < for_pipe)
+			width_before_pipe += hdisplay;
+		else if (pipe == for_pipe)
+			pipe_width = hdisplay;
+	}
+	alloc->start = ddb_size * width_before_pipe / total_width;
+	alloc->end = ddb_size * (width_before_pipe + pipe_width) / total_width;
 }
 static unsigned int skl_cursor_allocation(int num_active)
 				      val & PLANE_CTL_ALPHA_MASK);
 	val = I915_READ(PLANE_BUF_CFG(pipe, plane_id));
-	val2 = I915_READ(PLANE_NV12_BUF_CFG(pipe, plane_id));
+	/*
+	 * FIXME: add proper NV12 support for ICL. Avoid reading unclaimed
+	 * registers for now.
+	 */
+	if (INTEL_GEN(dev_priv) < 11)
+		val2 = I915_READ(PLANE_NV12_BUF_CFG(pipe, plane_id));
 	if (fourcc == DRM_FORMAT_NV12) {
 		skl_ddb_entry_init_from_hw(dev_priv,
 	skl_ddb_entry_write(dev_priv, PLANE_BUF_CFG(pipe, plane_id),
 			    &ddb->plane[pipe][plane_id]);
+	/* FIXME: add proper NV12 support for ICL. */
 	if (INTEL_GEN(dev_priv) >= 11)
 		return skl_ddb_entry_write(dev_priv,
 					   PLANE_BUF_CFG(pipe, plane_id),
 	return 0;
 }
-static void
-skl_copy_ddb_for_pipe(struct skl_ddb_values *dst,
-		      struct skl_ddb_values *src,
-		      enum pipe pipe)
-{
-	memcpy(dst->ddb.uv_plane[pipe], src->ddb.uv_plane[pipe],
-	       sizeof(dst->ddb.uv_plane[pipe]));
-	memcpy(dst->ddb.plane[pipe], src->ddb.plane[pipe],
-	       sizeof(dst->ddb.plane[pipe]));
-}
 static void
 skl_print_wm_changes(const struct drm_atomic_state *state)
 {
 	 * any other display updates race with this transaction, so we need
 	 * to grab the lock on *all* CRTC's.
 	 */
-	if (intel_state->active_pipe_changes) {
+	if (intel_state->active_pipe_changes || intel_state->modeset) {
 		realloc_pipes = ~0;
 		intel_state->wm_results.dirty_pipes = ~0;
 	}
 	if (cstate->base.active_changed)
 		skl_atomic_update_crtc_wm(state, cstate);
-	skl_copy_ddb_for_pipe(hw_vals, results, pipe);
+	memcpy(hw_vals->ddb.uv_plane[pipe], results->ddb.uv_plane[pipe],
+	       sizeof(hw_vals->ddb.uv_plane[pipe]));
+	memcpy(hw_vals->ddb.plane[pipe], results->ddb.plane[pipe],
+	       sizeof(hw_vals->ddb.plane[pipe]));
 	mutex_unlock(&dev_priv->wm.wm_mutex);
 }
 		new_power = HIGH_POWER;
 	rps_set_power(dev_priv, new_power);
 	mutex_unlock(&rps->power.mutex);
-	rps->last_adj = 0;
 }
 void intel_rps_mark_interactive(struct drm_i915_private *i915, bool interactive)
 	 */
 	if (!sanitize_rc6(dev_priv)) {
 		DRM_INFO("RC6 disabled, disabling runtime PM support\n");
-		intel_runtime_pm_get(dev_priv);
+		pm_runtime_get(&dev_priv->drm.pdev->dev);
 	}
 	mutex_lock(&dev_priv->pcu_lock);
 		valleyview_cleanup_gt_powersave(dev_priv);
 	if (!HAS_RC6(dev_priv))
-		intel_runtime_pm_put(dev_priv);
+		pm_runtime_put(&dev_priv->drm.pdev->dev);
 }
 /**
 	if (INTEL_GEN(dev_priv) >= 11)
 		gen11_reset_rps_interrupts(dev_priv);
-	else
+	else if (INTEL_GEN(dev_priv) >= 6)
 		gen6_reset_rps_interrupts(dev_priv);
 }

drivers/gpu/drm/i915/intel_psr.c+192−85

@@ -56,7 +56,30 @@ · 55 unmodified lines
56	#include "intel_drv.h"	56	#include "intel_drv.h"
57	#include "i915_drv.h"	57	#include "i915_drv.h"
58		58
59	void intel_psr_irq_control(struct drm_i915_private *dev_priv, bool debug)	59	static bool psr_global_enabled(u32 debug)
		60	{
		61	switch (debug & I915_PSR_DEBUG_MODE_MASK) {
		62	case I915_PSR_DEBUG_DEFAULT:
		63	return i915_modparams.enable_psr;
		64	case I915_PSR_DEBUG_DISABLE:
		65	return false;
		66	default:
		67	return true;
		68	}
		69	}
		70
		71	static bool intel_psr2_enabled(struct drm_i915_private *dev_priv,
		72	const struct intel_crtc_state *crtc_state)
		73	{
		74	switch (dev_priv->psr.debug & I915_PSR_DEBUG_MODE_MASK) {
		75	case I915_PSR_DEBUG_FORCE_PSR1:
		76	return false;
		77	default:
		78	return crtc_state->has_psr2;
		79	}
		80	}
		81
		82	void intel_psr_irq_control(struct drm_i915_private *dev_priv, u32 debug)
60	{	83	{
61	u32 debug_mask, mask;	84	u32 debug_mask, mask;
62		85
@@ -77,10 +100,9 @@ void intel_psr_irq_control(struct drm_i915_private *dev_priv, bool debug) · 14 unmodified lines
77	EDP_PSR_PRE_ENTRY(TRANSCODER_C);	100	EDP_PSR_PRE_ENTRY(TRANSCODER_C);
78	}	101	}
79		102
80	if (debug)	103	if (debug & I915_PSR_DEBUG_IRQ)
81	mask \|= debug_mask;	104	mask \|= debug_mask;
82		105
83	WRITE_ONCE(dev_priv->psr.debug, debug);
84	I915_WRITE(EDP_PSR_IMR, ~mask);	106	I915_WRITE(EDP_PSR_IMR, ~mask);
85	}	107	}
86		108
@@ -213,6 +235,9 @@ void intel_psr_init_dpcd(struct intel_dp *intel_dp) · 126 unmodified lines
213	dev_priv->psr.sink_sync_latency =	235	dev_priv->psr.sink_sync_latency =
214	intel_dp_get_sink_sync_latency(intel_dp);	236	intel_dp_get_sink_sync_latency(intel_dp);
215		237
		238	WARN_ON(dev_priv->psr.dp);
		239	dev_priv->psr.dp = intel_dp;
		240
216	if (INTEL_GEN(dev_priv) >= 9 &&	241	if (INTEL_GEN(dev_priv) >= 9 &&
217	(intel_dp->psr_dpcd[0] == DP_PSR2_WITH_Y_COORD_IS_SUPPORTED)) {	242	(intel_dp->psr_dpcd[0] == DP_PSR2_WITH_Y_COORD_IS_SUPPORTED)) {
218	bool y_req = intel_dp->psr_dpcd[1] &	243	bool y_req = intel_dp->psr_dpcd[1] &
@@ -245,7 +270,7 @@ static void intel_psr_setup_vsc(struct intel_dp *intel_dp, · 26 unmodified lines
245	const struct intel_crtc_state *crtc_state)	270	const struct intel_crtc_state *crtc_state)
246	{	271	{
247	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	272	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
248	struct drm_i915_private *dev_priv = to_i915(intel_dig_port->base.base.dev);	273	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
249	struct edp_vsc_psr psr_vsc;	274	struct edp_vsc_psr psr_vsc;
250		275
251	if (dev_priv->psr.psr2_enabled) {	276	if (dev_priv->psr.psr2_enabled) {
@@ -275,8 +300,7 @@ static void intel_psr_setup_vsc(struct intel_dp *intel_dp, · 23 unmodified lines
275		300
276	static void hsw_psr_setup_aux(struct intel_dp *intel_dp)	301	static void hsw_psr_setup_aux(struct intel_dp *intel_dp)
277	{	302	{
278	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);	303	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
279	struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
280	u32 aux_clock_divider, aux_ctl;	304	u32 aux_clock_divider, aux_ctl;
281	int i;	305	int i;
282	static const uint8_t aux_msg[] = {	306	static const uint8_t aux_msg[] = {
@@ -309,9 +333,7 @@ static void hsw_psr_setup_aux(struct intel_dp *intel_dp) · 26 unmodified lines
309		333
310	static void intel_psr_enable_sink(struct intel_dp *intel_dp)	334	static void intel_psr_enable_sink(struct intel_dp *intel_dp)
311	{	335	{
312	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);	336	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
313	struct drm_device *dev = dig_port->base.base.dev;
314	struct drm_i915_private *dev_priv = to_i915(dev);
315	u8 dpcd_val = DP_PSR_ENABLE;	337	u8 dpcd_val = DP_PSR_ENABLE;
316		338
317	/* Enable ALPM at sink for psr2 */	339	/* Enable ALPM at sink for psr2 */
@@ -332,9 +354,7 @@ static void intel_psr_enable_sink(struct intel_dp *intel_dp) · 14 unmodified lines
332		354
333	static void hsw_activate_psr1(struct intel_dp *intel_dp)	355	static void hsw_activate_psr1(struct intel_dp *intel_dp)
334	{	356	{
335	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);	357	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
336	struct drm_device *dev = dig_port->base.base.dev;
337	struct drm_i915_private *dev_priv = to_i915(dev);
338	u32 max_sleep_time = 0x1f;	358	u32 max_sleep_time = 0x1f;
339	u32 val = EDP_PSR_ENABLE;	359	u32 val = EDP_PSR_ENABLE;
340		360
@@ -389,9 +409,7 @@ static void hsw_activate_psr1(struct intel_dp *intel_dp) · 48 unmodified lines
389		409
390	static void hsw_activate_psr2(struct intel_dp *intel_dp)	410	static void hsw_activate_psr2(struct intel_dp *intel_dp)
391	{	411	{
392	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);	412	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
393	struct drm_device *dev = dig_port->base.base.dev;
394	struct drm_i915_private *dev_priv = to_i915(dev);
395	u32 val;	413	u32 val;
396		414
397	/* Let's use 6 as the minimum to cover all known cases including the	415	/* Let's use 6 as the minimum to cover all known cases including the
@@ -427,8 +445,7 @@ static void hsw_activate_psr2(struct intel_dp *intel_dp) · 29 unmodified lines
427	static bool intel_psr2_config_valid(struct intel_dp *intel_dp,	445	static bool intel_psr2_config_valid(struct intel_dp *intel_dp,
428	struct intel_crtc_state *crtc_state)	446	struct intel_crtc_state *crtc_state)
429	{	447	{
430	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);	448	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
431	struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
432	int crtc_hdisplay = crtc_state->base.adjusted_mode.crtc_hdisplay;	449	int crtc_hdisplay = crtc_state->base.adjusted_mode.crtc_hdisplay;
433	int crtc_vdisplay = crtc_state->base.adjusted_mode.crtc_vdisplay;	450	int crtc_vdisplay = crtc_state->base.adjusted_mode.crtc_vdisplay;
434	int psr_max_h = 0, psr_max_v = 0;	451	int psr_max_h = 0, psr_max_v = 0;
@@ -463,7 +480,7 @@ void intel_psr_compute_config(struct intel_dp *intel_dp, · 28 unmodified lines
463	struct intel_crtc_state *crtc_state)	480	struct intel_crtc_state *crtc_state)
464	{	481	{
465	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);	482	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
466	struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);	483	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
467	const struct drm_display_mode *adjusted_mode =	484	const struct drm_display_mode *adjusted_mode =
468	&crtc_state->base.adjusted_mode;	485	&crtc_state->base.adjusted_mode;
469	int psr_setup_time;	486	int psr_setup_time;
@@ -471,10 +488,8 @@ void intel_psr_compute_config(struct intel_dp *intel_dp, · 1 unmodified line
471	if (!CAN_PSR(dev_priv))	488	if (!CAN_PSR(dev_priv))
472	return;	489	return;
473		490
474	if (!i915_modparams.enable_psr) {	491	if (intel_dp != dev_priv->psr.dp)
475	DRM_DEBUG_KMS("PSR disable by flag\n");
476	return;	492	return;
477	}
478		493
479	/*	494	/*
480	* HSW spec explicitly says PSR is tied to port A.	495	* HSW spec explicitly says PSR is tied to port A.
@@ -517,14 +532,11 @@ void intel_psr_compute_config(struct intel_dp *intel_dp, · 36 unmodified lines
517		532
518	crtc_state->has_psr = true;	533	crtc_state->has_psr = true;
519	crtc_state->has_psr2 = intel_psr2_config_valid(intel_dp, crtc_state);	534	crtc_state->has_psr2 = intel_psr2_config_valid(intel_dp, crtc_state);
520	DRM_DEBUG_KMS("Enabling PSR%s\n", crtc_state->has_psr2 ? "2" : "");
521	}	535	}
522		536
523	static void intel_psr_activate(struct intel_dp *intel_dp)	537	static void intel_psr_activate(struct intel_dp *intel_dp)
524	{	538	{
525	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	539	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
526	struct drm_device *dev = intel_dig_port->base.base.dev;
527	struct drm_i915_private *dev_priv = to_i915(dev);
528		540
529	if (INTEL_GEN(dev_priv) >= 9)	541	if (INTEL_GEN(dev_priv) >= 9)
530	WARN_ON(I915_READ(EDP_PSR2_CTL) & EDP_PSR2_ENABLE);	542	WARN_ON(I915_READ(EDP_PSR2_CTL) & EDP_PSR2_ENABLE);
@@ -544,9 +556,7 @@ static void intel_psr_activate(struct intel_dp *intel_dp) · 13 unmodified lines
544	static void intel_psr_enable_source(struct intel_dp *intel_dp,	556	static void intel_psr_enable_source(struct intel_dp *intel_dp,
545	const struct intel_crtc_state *crtc_state)	557	const struct intel_crtc_state *crtc_state)
546	{	558	{
547	struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);	559	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
548	struct drm_device *dev = dig_port->base.base.dev;
549	struct drm_i915_private *dev_priv = to_i915(dev);
550	enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;	560	enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
551		561
552	/* Only HSW and BDW have PSR AUX registers that need to be setup. SKL+	562	/* Only HSW and BDW have PSR AUX registers that need to be setup. SKL+
@@ -589,6 +599,24 @@ static void intel_psr_enable_source(struct intel_dp *intel_dp, · 36 unmodified lines
589	}	599	}
590	}	600	}
591		601
		602	static void intel_psr_enable_locked(struct drm_i915_private *dev_priv,
		603	const struct intel_crtc_state *crtc_state)
		604	{
		605	struct intel_dp *intel_dp = dev_priv->psr.dp;
		606
		607	if (dev_priv->psr.enabled)
		608	return;
		609
		610	DRM_DEBUG_KMS("Enabling PSR%s\n",
		611	dev_priv->psr.psr2_enabled ? "2" : "1");
		612	intel_psr_setup_vsc(intel_dp, crtc_state);
		613	intel_psr_enable_sink(intel_dp);
		614	intel_psr_enable_source(intel_dp, crtc_state);
		615	dev_priv->psr.enabled = true;
		616
		617	intel_psr_activate(intel_dp);
		618	}
		619
592	/**	620	/**
593	* intel_psr_enable - Enable PSR	621	* intel_psr_enable - Enable PSR
594	* @intel_dp: Intel DP	622	* @intel_dp: Intel DP
@@ -599,9 +627,7 @@ static void intel_psr_enable_source(struct intel_dp *intel_dp, · 4 unmodified lines
599	void intel_psr_enable(struct intel_dp *intel_dp,	627	void intel_psr_enable(struct intel_dp *intel_dp,
600	const struct intel_crtc_state *crtc_state)	628	const struct intel_crtc_state *crtc_state)
601	{	629	{
602	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	630	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
603	struct drm_device *dev = intel_dig_port->base.base.dev;
604	struct drm_i915_private *dev_priv = to_i915(dev);
605		631
606	if (!crtc_state->has_psr)	632	if (!crtc_state->has_psr)
607	return;	633	return;
@@ -610,21 +636,21 @@ void intel_psr_enable(struct intel_dp *intel_dp, · 2 unmodified lines
610	return;	636	return;
611		637
612	WARN_ON(dev_priv->drrs.dp);	638	WARN_ON(dev_priv->drrs.dp);
		639
613	mutex_lock(&dev_priv->psr.lock);	640	mutex_lock(&dev_priv->psr.lock);
614	if (dev_priv->psr.enabled) {	641	if (dev_priv->psr.prepared) {
615	DRM_DEBUG_KMS("PSR already in use\n");	642	DRM_DEBUG_KMS("PSR already in use\n");
616	goto unlock;	643	goto unlock;
617	}	644	}
618		645
619	dev_priv->psr.psr2_enabled = crtc_state->has_psr2;	646	dev_priv->psr.psr2_enabled = intel_psr2_enabled(dev_priv, crtc_state);
620	dev_priv->psr.busy_frontbuffer_bits = 0;	647	dev_priv->psr.busy_frontbuffer_bits = 0;
		648	dev_priv->psr.prepared = true;
621		649
622	intel_psr_setup_vsc(intel_dp, crtc_state);	650	if (psr_global_enabled(dev_priv->psr.debug))
623	intel_psr_enable_sink(intel_dp);	651	intel_psr_enable_locked(dev_priv, crtc_state);
624	intel_psr_enable_source(intel_dp, crtc_state);	652	else
625	dev_priv->psr.enabled = intel_dp;	653	DRM_DEBUG_KMS("PSR disabled by flag\n");
626
627	intel_psr_activate(intel_dp);
628		654
629	unlock:	655	unlock:
630	mutex_unlock(&dev_priv->psr.lock);	656	mutex_unlock(&dev_priv->psr.lock);
@@ -633,9 +659,7 @@ unlock: · 2 unmodified lines
633	static void	659	static void
634	intel_psr_disable_source(struct intel_dp *intel_dp)	660	intel_psr_disable_source(struct intel_dp *intel_dp)
635	{	661	{
636	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	662	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
637	struct drm_device *dev = intel_dig_port->base.base.dev;
638	struct drm_i915_private *dev_priv = to_i915(dev);
639		663
640	if (dev_priv->psr.active) {	664	if (dev_priv->psr.active) {
641	i915_reg_t psr_status;	665	i915_reg_t psr_status;
@@ -674,21 +698,21 @@ intel_psr_disable_source(struct intel_dp *intel_dp) · 32 unmodified lines
674		698
675	static void intel_psr_disable_locked(struct intel_dp *intel_dp)	699	static void intel_psr_disable_locked(struct intel_dp *intel_dp)
676	{	700	{
677	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	701	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
678	struct drm_device *dev = intel_dig_port->base.base.dev;
679	struct drm_i915_private *dev_priv = to_i915(dev);
680		702
681	lockdep_assert_held(&dev_priv->psr.lock);	703	lockdep_assert_held(&dev_priv->psr.lock);
682		704
683	if (!dev_priv->psr.enabled)	705	if (!dev_priv->psr.enabled)
684	return;	706	return;
685		707
		708	DRM_DEBUG_KMS("Disabling PSR%s\n",
		709	dev_priv->psr.psr2_enabled ? "2" : "1");
686	intel_psr_disable_source(intel_dp);	710	intel_psr_disable_source(intel_dp);
687		711
688	/* Disable PSR on Sink */	712	/* Disable PSR on Sink */
689	drm_dp_dpcd_writeb(&intel_dp->aux, DP_PSR_EN_CFG, 0);	713	drm_dp_dpcd_writeb(&intel_dp->aux, DP_PSR_EN_CFG, 0);
690		714
691	dev_priv->psr.enabled = NULL;	715	dev_priv->psr.enabled = false;
692	}	716	}
693		717
694	/**	718	/**
@@ -701,9 +725,7 @@ static void intel_psr_disable_locked(struct intel_dp *intel_dp) · 6 unmodified lines
701	void intel_psr_disable(struct intel_dp *intel_dp,	725	void intel_psr_disable(struct intel_dp *intel_dp,
702	const struct intel_crtc_state *old_crtc_state)	726	const struct intel_crtc_state *old_crtc_state)
703	{	727	{
704	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	728	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
705	struct drm_device *dev = intel_dig_port->base.base.dev;
706	struct drm_i915_private *dev_priv = to_i915(dev);
707		729
708	if (!old_crtc_state->has_psr)	730	if (!old_crtc_state->has_psr)
709	return;	731	return;
@@ -712,57 +734,61 @@ void intel_psr_disable(struct intel_dp *intel_dp, · 2 unmodified lines
712	return;	734	return;
713		735
714	mutex_lock(&dev_priv->psr.lock);	736	mutex_lock(&dev_priv->psr.lock);
		737	if (!dev_priv->psr.prepared) {
		738	mutex_unlock(&dev_priv->psr.lock);
		739	return;
		740	}
		741
715	intel_psr_disable_locked(intel_dp);	742	intel_psr_disable_locked(intel_dp);
		743
		744	dev_priv->psr.prepared = false;
716	mutex_unlock(&dev_priv->psr.lock);	745	mutex_unlock(&dev_priv->psr.lock);
717	cancel_work_sync(&dev_priv->psr.work);	746	cancel_work_sync(&dev_priv->psr.work);
718	}	747	}
719		748
720	int intel_psr_wait_for_idle(const struct intel_crtc_state *new_crtc_state)	749	/**
		750	* intel_psr_wait_for_idle - wait for PSR1 to idle
		751	* @new_crtc_state: new CRTC state
		752	* @out_value: PSR status in case of failure
		753	*
		754	* This function is expected to be called from pipe_update_start() where it is
		755	* not expected to race with PSR enable or disable.
		756	*
		757	* Returns: 0 on success or -ETIMEOUT if PSR status does not idle.
		758	*/
		759	int intel_psr_wait_for_idle(const struct intel_crtc_state *new_crtc_state,
		760	u32 *out_value)
721	{	761	{
722	struct intel_crtc *crtc = to_intel_crtc(new_crtc_state->base.crtc);	762	struct intel_crtc *crtc = to_intel_crtc(new_crtc_state->base.crtc);
723	struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);	763	struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
724	i915_reg_t reg;
725	u32 mask;
726		764
727	if (!new_crtc_state->has_psr)	765	if (!dev_priv->psr.enabled \|\| !new_crtc_state->has_psr)
728	return 0;	766	return 0;
729		767
730	/*	768	/* FIXME: Update this for PSR2 if we need to wait for idle */
731	* The sole user right now is intel_pipe_update_start(),	769	if (READ_ONCE(dev_priv->psr.psr2_enabled))
732	* which won't race with psr_enable/disable, which is	770	return 0;
733	* where psr2_enabled is written to. So, we don't need
734	* to acquire the psr.lock. More importantly, we want the
735	* latency inside intel_pipe_update_start() to be as low
736	* as possible, so no need to acquire psr.lock when it is
737	* not needed and will induce latencies in the atomic
738	* update path.
739	*/
740	if (dev_priv->psr.psr2_enabled) {
741	reg = EDP_PSR2_STATUS;
742	mask = EDP_PSR2_STATUS_STATE_MASK;
743	} else {
744	reg = EDP_PSR_STATUS;
745	mask = EDP_PSR_STATUS_STATE_MASK;
746	}
747		771
748	/*	772	/*
749	* Max time for PSR to idle = Inverse of the refresh rate +	773	* From bspec: Panel Self Refresh (BDW+)
750	* 6 ms of exit training time + 1.5 ms of aux channel	774	* Max. time for PSR to idle = Inverse of the refresh rate + 6 ms of
751	* handshake. 50 msec is defesive enough to cover everything.	775	* exit training time + 1.5 ms of aux channel handshake. 50 ms is
		776	* defensive enough to cover everything.
752	*/	777	*/
753	return intel_wait_for_register(dev_priv, reg, mask,	778
754	EDP_PSR_STATUS_STATE_IDLE, 50);	779	return __intel_wait_for_register(dev_priv, EDP_PSR_STATUS,
		780	EDP_PSR_STATUS_STATE_MASK,
		781	EDP_PSR_STATUS_STATE_IDLE, 2, 50,
		782	out_value);
755	}	783	}
756		784
757	static bool __psr_wait_for_idle_locked(struct drm_i915_private *dev_priv)	785	static bool __psr_wait_for_idle_locked(struct drm_i915_private *dev_priv)
758	{	786	{
759	struct intel_dp *intel_dp;
760	i915_reg_t reg;	787	i915_reg_t reg;
761	u32 mask;	788	u32 mask;
762	int err;	789	int err;
763		790
764	intel_dp = dev_priv->psr.enabled;	791	if (!dev_priv->psr.enabled)
765	if (!intel_dp)
766	return false;	792	return false;
767		793
768	if (dev_priv->psr.psr2_enabled) {	794	if (dev_priv->psr.psr2_enabled) {
@@ -784,6 +810,89 @@ static bool __psr_wait_for_idle_locked(struct drm_i915_private *dev_priv) · 15 unmodified lines
784	return err == 0 && dev_priv->psr.enabled;	810	return err == 0 && dev_priv->psr.enabled;
785	}	811	}
786		812
		813	static bool switching_psr(struct drm_i915_private *dev_priv,
		814	struct intel_crtc_state *crtc_state,
		815	u32 mode)
		816	{
		817	/* Can't switch psr state anyway if PSR2 is not supported. */
		818	if (!crtc_state \|\| !crtc_state->has_psr2)
		819	return false;
		820
		821	if (dev_priv->psr.psr2_enabled && mode == I915_PSR_DEBUG_FORCE_PSR1)
		822	return true;
		823
		824	if (!dev_priv->psr.psr2_enabled && mode != I915_PSR_DEBUG_FORCE_PSR1)
		825	return true;
		826
		827	return false;
		828	}
		829
		830	int intel_psr_set_debugfs_mode(struct drm_i915_private *dev_priv,
		831	struct drm_modeset_acquire_ctx *ctx,
		832	u64 val)
		833	{
		834	struct drm_device *dev = &dev_priv->drm;
		835	struct drm_connector_state *conn_state;
		836	struct intel_crtc_state *crtc_state = NULL;
		837	struct drm_crtc_commit *commit;
		838	struct drm_crtc *crtc;
		839	struct intel_dp *dp;
		840	int ret;
		841	bool enable;
		842	u32 mode = val & I915_PSR_DEBUG_MODE_MASK;
		843
		844	if (val & ~(I915_PSR_DEBUG_IRQ \| I915_PSR_DEBUG_MODE_MASK) \|\|
		845	mode > I915_PSR_DEBUG_FORCE_PSR1) {
		846	DRM_DEBUG_KMS("Invalid debug mask %llx\n", val);
		847	return -EINVAL;
		848	}
		849
		850	ret = drm_modeset_lock(&dev->mode_config.connection_mutex, ctx);
		851	if (ret)
		852	return ret;
		853
		854	/* dev_priv->psr.dp should be set once and then never touched again. */
		855	dp = READ_ONCE(dev_priv->psr.dp);
		856	conn_state = dp->attached_connector->base.state;
		857	crtc = conn_state->crtc;
		858	if (crtc) {
		859	ret = drm_modeset_lock(&crtc->mutex, ctx);
		860	if (ret)
		861	return ret;
		862
		863	crtc_state = to_intel_crtc_state(crtc->state);
		864	commit = crtc_state->base.commit;
		865	} else {
		866	commit = conn_state->commit;
		867	}
		868	if (commit) {
		869	ret = wait_for_completion_interruptible(&commit->hw_done);
		870	if (ret)
		871	return ret;
		872	}
		873
		874	ret = mutex_lock_interruptible(&dev_priv->psr.lock);
		875	if (ret)
		876	return ret;
		877
		878	enable = psr_global_enabled(val);
		879
		880	if (!enable \|\| switching_psr(dev_priv, crtc_state, mode))
		881	intel_psr_disable_locked(dev_priv->psr.dp);
		882
		883	dev_priv->psr.debug = val;
		884	if (crtc)
		885	dev_priv->psr.psr2_enabled = intel_psr2_enabled(dev_priv, crtc_state);
		886
		887	intel_psr_irq_control(dev_priv, dev_priv->psr.debug);
		888
		889	if (dev_priv->psr.prepared && enable)
		890	intel_psr_enable_locked(dev_priv, crtc_state);
		891
		892	mutex_unlock(&dev_priv->psr.lock);
		893	return ret;
		894	}
		895
787	static void intel_psr_work(struct work_struct *work)	896	static void intel_psr_work(struct work_struct *work)
788	{	897	{
789	struct drm_i915_private *dev_priv =	898	struct drm_i915_private *dev_priv =
@@ -811,7 +920,7 @@ static void intel_psr_work(struct work_struct *work) · 21 unmodified lines
811	if (dev_priv->psr.busy_frontbuffer_bits \|\| dev_priv->psr.active)	920	if (dev_priv->psr.busy_frontbuffer_bits \|\| dev_priv->psr.active)
812	goto unlock;	921	goto unlock;
813		922
814	intel_psr_activate(dev_priv->psr.enabled);	923	intel_psr_activate(dev_priv->psr.dp);
815	unlock:	924	unlock:
816	mutex_unlock(&dev_priv->psr.lock);	925	mutex_unlock(&dev_priv->psr.lock);
817	}	926	}
@@ -866,7 +975,7 @@ void intel_psr_invalidate(struct drm_i915_private *dev_priv, · 48 unmodified lines
866	return;	975	return;
867	}	976	}
868		977
869	crtc = dp_to_dig_port(dev_priv->psr.enabled)->base.base.crtc;	978	crtc = dp_to_dig_port(dev_priv->psr.dp)->base.base.crtc;
870	pipe = to_intel_crtc(crtc)->pipe;	979	pipe = to_intel_crtc(crtc)->pipe;
871		980
872	frontbuffer_bits &= INTEL_FRONTBUFFER_ALL_MASK(pipe);	981	frontbuffer_bits &= INTEL_FRONTBUFFER_ALL_MASK(pipe);
@@ -909,7 +1018,7 @@ void intel_psr_flush(struct drm_i915_private *dev_priv, · 36 unmodified lines
909	return;	1018	return;
910	}	1019	}
911		1020
912	crtc = dp_to_dig_port(dev_priv->psr.enabled)->base.base.crtc;	1021	crtc = dp_to_dig_port(dev_priv->psr.dp)->base.base.crtc;
913	pipe = to_intel_crtc(crtc)->pipe;	1022	pipe = to_intel_crtc(crtc)->pipe;
914		1023
915	frontbuffer_bits &= INTEL_FRONTBUFFER_ALL_MASK(pipe);	1024	frontbuffer_bits &= INTEL_FRONTBUFFER_ALL_MASK(pipe);
@@ -977,9 +1086,7 @@ void intel_psr_init(struct drm_i915_private *dev_priv) · 61 unmodified lines
977		1086
978	void intel_psr_short_pulse(struct intel_dp *intel_dp)	1087	void intel_psr_short_pulse(struct intel_dp *intel_dp)
979	{	1088	{
980	struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);	1089	struct drm_i915_private *dev_priv = dp_to_i915(intel_dp);
981	struct drm_device *dev = intel_dig_port->base.base.dev;
982	struct drm_i915_private *dev_priv = to_i915(dev);
983	struct i915_psr *psr = &dev_priv->psr;	1090	struct i915_psr *psr = &dev_priv->psr;
984	u8 val;	1091	u8 val;
985	const u8 errors = DP_PSR_RFB_STORAGE_ERROR \|	1092	const u8 errors = DP_PSR_RFB_STORAGE_ERROR \|
@@ -991,7 +1098,7 @@ void intel_psr_short_pulse(struct intel_dp *intel_dp) · 5 unmodified lines
991		1098
992	mutex_lock(&psr->lock);	1099	mutex_lock(&psr->lock);
993		1100
994	if (psr->enabled != intel_dp)	1101	if (!psr->enabled \|\| psr->dp != intel_dp)
995	goto exit;	1102	goto exit;
996		1103
997	if (drm_dp_dpcd_readb(&intel_dp->aux, DP_PSR_STATUS, &val) != 1) {	1104	if (drm_dp_dpcd_readb(&intel_dp->aux, DP_PSR_STATUS, &val) != 1) {

drivers/gpu/drm/i915/intel_ringbuffer.c+56−54

@@ -344,11 +344,14 @@ gen7_render_ring_flush(struct i915_request *rq, u32 mode) · 343 unmodified lines
344	static void ring_setup_phys_status_page(struct intel_engine_cs *engine)	344	static void ring_setup_phys_status_page(struct intel_engine_cs *engine)
345	{	345	{
346	struct drm_i915_private *dev_priv = engine->i915;	346	struct drm_i915_private *dev_priv = engine->i915;
		347	struct page *page = virt_to_page(engine->status_page.page_addr);
		348	phys_addr_t phys = PFN_PHYS(page_to_pfn(page));
347	u32 addr;	349	u32 addr;
348		350
349	addr = dev_priv->status_page_dmah->busaddr;	351	addr = lower_32_bits(phys);
350	if (INTEL_GEN(dev_priv) >= 4)	352	if (INTEL_GEN(dev_priv) >= 4)
351	addr \|= (dev_priv->status_page_dmah->busaddr >> 28) & 0xf0;	353	addr \|= (phys >> 28) & 0xf0;
		354
352	I915_WRITE(HWS_PGA, addr);	355	I915_WRITE(HWS_PGA, addr);
353	}	356	}
354		357
@@ -537,6 +540,8 @@ static int init_ring_common(struct intel_engine_cs *engine) · 182 unmodified lines
537	if (INTEL_GEN(dev_priv) > 2)	540	if (INTEL_GEN(dev_priv) > 2)
538	I915_WRITE_MODE(engine, _MASKED_BIT_DISABLE(STOP_RING));	541	I915_WRITE_MODE(engine, _MASKED_BIT_DISABLE(STOP_RING));
539		542
		543	/* Papering over lost _interrupts_ immediately following the restart */
		544	intel_engine_wakeup(engine);
540	out:	545	out:
541	intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);	546	intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
542		547
@@ -1013,24 +1018,22 @@ i915_emit_bb_start(struct i915_request *rq, · 470 unmodified lines
1013	return 0;	1018	return 0;
1014	}	1019	}
1015		1020
1016		1021	int intel_ring_pin(struct intel_ring *ring)
1017
1018	int intel_ring_pin(struct intel_ring *ring,
1019	struct drm_i915_private *i915,
1020	unsigned int offset_bias)
1021	{	1022	{
1022	enum i915_map_type map = HAS_LLC(i915) ? I915_MAP_WB : I915_MAP_WC;
1023	struct i915_vma *vma = ring->vma;	1023	struct i915_vma *vma = ring->vma;
		1024	enum i915_map_type map =
		1025	HAS_LLC(vma->vm->i915) ? I915_MAP_WB : I915_MAP_WC;
1024	unsigned int flags;	1026	unsigned int flags;
1025	void *addr;	1027	void *addr;
1026	int ret;	1028	int ret;
1027		1029
1028	GEM_BUG_ON(ring->vaddr);	1030	GEM_BUG_ON(ring->vaddr);
1029		1031
1030
1031	flags = PIN_GLOBAL;	1032	flags = PIN_GLOBAL;
1032	if (offset_bias)	1033
1033	flags \|= PIN_OFFSET_BIAS \| offset_bias;	1034	/* Ring wraparound at offset 0 sometimes hangs. No idea why. */
		1035	flags \|= PIN_OFFSET_BIAS \| i915_ggtt_pin_bias(vma);
		1036
1034	if (vma->obj->stolen)	1037	if (vma->obj->stolen)
1035	flags \|= PIN_MAPPABLE;	1038	flags \|= PIN_MAPPABLE;
1036	else	1039	else
@@ -1045,7 +1048,7 @@ int intel_ring_pin(struct intel_ring *ring, · 8 unmodified lines
1045	return ret;	1048	return ret;
1046	}	1049	}
1047		1050
1048	ret = i915_vma_pin(vma, 0, PAGE_SIZE, flags);	1051	ret = i915_vma_pin(vma, 0, 0, flags);
1049	if (unlikely(ret))	1052	if (unlikely(ret))
1050	return ret;	1053	return ret;
1051		1054
@@ -1230,8 +1233,7 @@ static int __context_pin(struct intel_context *ce) · 178 unmodified lines
1230	return err;	1233	return err;
1231	}	1234	}
1232		1235
1233	err = i915_vma_pin(vma, 0, I915_GTT_MIN_ALIGNMENT,	1236	err = i915_vma_pin(vma, 0, 0, PIN_GLOBAL \| PIN_HIGH);
1234	PIN_GLOBAL \| PIN_HIGH);
1235	if (err)	1237	if (err)
1236	return err;	1238	return err;
1237		1239
@@ -1419,8 +1421,7 @@ static int intel_init_ring_buffer(struct intel_engine_cs *engine) · 181 unmodified lines
1419	goto err;	1421	goto err;
1420	}	1422	}
1421		1423
1422	/* Ring wraparound at offset 0 sometimes hangs. No idea why. */	1424	err = intel_ring_pin(ring);
1423	err = intel_ring_pin(ring, engine->i915, I915_GTT_PAGE_SIZE);
1424	if (err)	1425	if (err)
1425	goto err_ring;	1426	goto err_ring;
1426		1427
@@ -1706,9 +1707,29 @@ static int switch_context(struct i915_request *rq) · 279 unmodified lines
1706	}	1707	}
1707		1708
1708	if (ppgtt) {	1709	if (ppgtt) {
		1710	ret = engine->emit_flush(rq, EMIT_INVALIDATE);
		1711	if (ret)
		1712	goto err_mm;
		1713
1709	ret = flush_pd_dir(rq);	1714	ret = flush_pd_dir(rq);
1710	if (ret)	1715	if (ret)
1711	goto err_mm;	1716	goto err_mm;
		1717
		1718	/*
		1719	* Not only do we need a full barrier (post-sync write) after
		1720	* invalidating the TLBs, but we need to wait a little bit
		1721	* longer. Whether this is merely delaying us, or the
		1722	* subsequent flush is a key part of serialising with the
		1723	* post-sync op, this extra pass appears vital before a
		1724	* mm switch!
		1725	*/
		1726	ret = engine->emit_flush(rq, EMIT_INVALIDATE);
		1727	if (ret)
		1728	goto err_mm;
		1729
		1730	ret = engine->emit_flush(rq, EMIT_FLUSH);
		1731	if (ret)
		1732	goto err_mm;
1712	}	1733	}
1713		1734
1714	if (ctx->remap_slice) {	1735	if (ctx->remap_slice) {
@@ -1946,7 +1967,7 @@ static void gen6_bsd_submit_request(struct i915_request *request) · 231 unmodified lines
1946	intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);	1967	intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
1947	}	1968	}
1948		1969
1949	static int gen6_bsd_ring_flush(struct i915_request *rq, u32 mode)	1970	static int mi_flush_dw(struct i915_request *rq, u32 flags)
1950	{	1971	{
1951	u32 cmd, *cs;	1972	u32 cmd, *cs;
1952		1973
@@ -1956,7 +1977,8 @@ static int gen6_bsd_ring_flush(struct i915_request *rq, u32 mode) · 3 unmodified lines
1956		1977
1957	cmd = MI_FLUSH_DW;	1978	cmd = MI_FLUSH_DW;
1958		1979
1959	/* We always require a command barrier so that subsequent	1980	/*
		1981	* We always require a command barrier so that subsequent
1960	* commands, such as breadcrumb interrupts, are strictly ordered	1982	* commands, such as breadcrumb interrupts, are strictly ordered
1961	* wrt the contents of the write cache being flushed to memory	1983	* wrt the contents of the write cache being flushed to memory
1962	* (and thus being coherent from the CPU).	1984	* (and thus being coherent from the CPU).
@@ -1964,22 +1986,33 @@ static int gen6_bsd_ring_flush(struct i915_request *rq, u32 mode) · 1 unmodified line
1964	cmd \|= MI_FLUSH_DW_STORE_INDEX \| MI_FLUSH_DW_OP_STOREDW;	1986	cmd \|= MI_FLUSH_DW_STORE_INDEX \| MI_FLUSH_DW_OP_STOREDW;
1965		1987
1966	/*	1988	/*
1967	* Bspec vol 1c.5 - video engine command streamer:	1989	* Bspec vol 1c.3 - blitter engine command streamer:
1968	* "If ENABLED, all TLBs will be invalidated once the flush	1990	* "If ENABLED, all TLBs will be invalidated once the flush
1969	* operation is complete. This bit is only valid when the	1991	* operation is complete. This bit is only valid when the
1970	* Post-Sync Operation field is a value of 1h or 3h."	1992	* Post-Sync Operation field is a value of 1h or 3h."
1971	*/	1993	*/
1972	if (mode & EMIT_INVALIDATE)	1994	cmd \|= flags;
1973	cmd \|= MI_INVALIDATE_TLB \| MI_INVALIDATE_BSD;
1974		1995
1975	*cs++ = cmd;	1996	*cs++ = cmd;
1976	*cs++ = I915_GEM_HWS_SCRATCH_ADDR \| MI_FLUSH_DW_USE_GTT;	1997	*cs++ = I915_GEM_HWS_SCRATCH_ADDR \| MI_FLUSH_DW_USE_GTT;
1977	*cs++ = 0;	1998	*cs++ = 0;
1978	*cs++ = MI_NOOP;	1999	*cs++ = MI_NOOP;
		2000
1979	intel_ring_advance(rq, cs);	2001	intel_ring_advance(rq, cs);
		2002
1980	return 0;	2003	return 0;
1981	}	2004	}
1982		2005
		2006	static int gen6_flush_dw(struct i915_request *rq, u32 mode, u32 invflags)
		2007	{
		2008	return mi_flush_dw(rq, mode & EMIT_INVALIDATE ? invflags : 0);
		2009	}
		2010
		2011	static int gen6_bsd_ring_flush(struct i915_request *rq, u32 mode)
		2012	{
		2013	return gen6_flush_dw(rq, mode, MI_INVALIDATE_TLB \| MI_INVALIDATE_BSD);
		2014	}
		2015
1983	static int	2016	static int
1984	hsw_emit_bb_start(struct i915_request *rq,	2017	hsw_emit_bb_start(struct i915_request *rq,
1985	u64 offset, u32 len,	2018	u64 offset, u32 len,
@@ -1992,9 +2025,7 @@ hsw_emit_bb_start(struct i915_request *rq, · 6 unmodified lines
1992	return PTR_ERR(cs);	2025	return PTR_ERR(cs);
1993		2026
1994	*cs++ = MI_BATCH_BUFFER_START \| (dispatch_flags & I915_DISPATCH_SECURE ?	2027	*cs++ = MI_BATCH_BUFFER_START \| (dispatch_flags & I915_DISPATCH_SECURE ?
1995	0 : MI_BATCH_PPGTT_HSW \| MI_BATCH_NON_SECURE_HSW) \|	2028	0 : MI_BATCH_PPGTT_HSW \| MI_BATCH_NON_SECURE_HSW);
1996	(dispatch_flags & I915_DISPATCH_RS ?
1997	MI_BATCH_RESOURCE_STREAMER : 0);
1998	/* bit0-7 is the length on GEN6+ */	2029	/* bit0-7 is the length on GEN6+ */
1999	*cs++ = offset;	2030	*cs++ = offset;
2000	intel_ring_advance(rq, cs);	2031	intel_ring_advance(rq, cs);
@@ -2026,36 +2057,7 @@ gen6_emit_bb_start(struct i915_request *rq, · 25 unmodified lines
2026		2057
2027	static int gen6_ring_flush(struct i915_request *rq, u32 mode)	2058	static int gen6_ring_flush(struct i915_request *rq, u32 mode)
2028	{	2059	{
2029	u32 cmd, *cs;	2060	return gen6_flush_dw(rq, mode, MI_INVALIDATE_TLB);
2030
2031	cs = intel_ring_begin(rq, 4);
2032	if (IS_ERR(cs))
2033	return PTR_ERR(cs);
2034
2035	cmd = MI_FLUSH_DW;
2036
2037	/* We always require a command barrier so that subsequent
2038	* commands, such as breadcrumb interrupts, are strictly ordered
2039	* wrt the contents of the write cache being flushed to memory
2040	* (and thus being coherent from the CPU).
2041	*/
2042	cmd \|= MI_FLUSH_DW_STORE_INDEX \| MI_FLUSH_DW_OP_STOREDW;
2043
2044	/*
2045	* Bspec vol 1c.3 - blitter engine command streamer:
2046	* "If ENABLED, all TLBs will be invalidated once the flush
2047	* operation is complete. This bit is only valid when the
2048	* Post-Sync Operation field is a value of 1h or 3h."
2049	*/
2050	if (mode & EMIT_INVALIDATE)
2051	cmd \|= MI_INVALIDATE_TLB;
2052	*cs++ = cmd;
2053	*cs++ = I915_GEM_HWS_SCRATCH_ADDR \| MI_FLUSH_DW_USE_GTT;
2054	*cs++ = 0;
2055	*cs++ = MI_NOOP;
2056	intel_ring_advance(rq, cs);
2057
2058	return 0;
2059	}	2061	}
2060		2062
2061	static void intel_ring_init_semaphores(struct drm_i915_private *dev_priv,	2063	static void intel_ring_init_semaphores(struct drm_i915_private *dev_priv,

drivers/gpu/drm/i915/intel_ringbuffer.h+29−10

 					 unsigned int dispatch_flags);
 #define I915_DISPATCH_SECURE BIT(0)
 #define I915_DISPATCH_PINNED BIT(1)
-#define I915_DISPATCH_RS     BIT(2)
 	void		(*emit_breadcrumb)(struct i915_request *rq, u32 *cs);
 	int		emit_breadcrumb_sz;
 intel_engine_create_ring(struct intel_engine_cs *engine,
 			 struct i915_timeline *timeline,
 			 int size);
-int intel_ring_pin(struct intel_ring *ring,
+int intel_ring_pin(struct intel_ring *ring);
-		   struct drm_i915_private *i915,
-		   unsigned int offset_bias);
 void intel_ring_reset(struct intel_ring *ring, u32 tail);
 unsigned int intel_ring_update_space(struct intel_ring *ring);
 void intel_ring_unpin(struct intel_ring *ring);
 int intel_init_vebox_ring_buffer(struct intel_engine_cs *engine);
 int intel_engine_stop_cs(struct intel_engine_cs *engine);
+void intel_engine_cancel_stop_cs(struct intel_engine_cs *engine);
 u64 intel_engine_get_active_head(const struct intel_engine_cs *engine);
 u64 intel_engine_get_last_batch_head(const struct intel_engine_cs *engine);
-static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine)
-{
-	return intel_read_status_page(engine, I915_GEM_HWS_INDEX);
-}
 static inline u32 intel_engine_last_submit(struct intel_engine_cs *engine)
 {
-	/* We are only peeking at the tail of the submit queue (and not the
+	/*
+	 * We are only peeking at the tail of the submit queue (and not the
 	 * queue itself) in order to gain a hint as to the current active
 	 * state of the engine. Callers are not expected to be taking
 	 * engine->timeline->lock, nor are they expected to be concerned
 	return READ_ONCE(engine->timeline.seqno);
 }
+static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine)
+{
+	return intel_read_status_page(engine, I915_GEM_HWS_INDEX);
+}
+static inline bool intel_engine_signaled(struct intel_engine_cs *engine,
+					 u32 seqno)
+{
+	return i915_seqno_passed(intel_engine_get_seqno(engine), seqno);
+}
+static inline bool intel_engine_has_completed(struct intel_engine_cs *engine,
+					      u32 seqno)
+{
+	GEM_BUG_ON(!seqno);
+	return intel_engine_signaled(engine, seqno);
+}
+static inline bool intel_engine_has_started(struct intel_engine_cs *engine,
+					    u32 seqno)
+{
+	GEM_BUG_ON(!seqno);
+	return intel_engine_signaled(engine, seqno - 1);
+}
 void intel_engine_get_instdone(struct intel_engine_cs *engine,
 			       struct intel_instdone *instdone);

drivers/gpu/drm/i915/intel_runtime_pm.c+674−329

 bool intel_display_power_well_is_enabled(struct drm_i915_private *dev_priv,
 					 enum i915_power_well_id power_well_id);
-static struct i915_power_well *
-lookup_power_well(struct drm_i915_private *dev_priv,
-		  enum i915_power_well_id power_well_id);
 const char *
 intel_display_power_domain_str(enum intel_display_power_domain domain)
 {
 static void intel_power_well_enable(struct drm_i915_private *dev_priv,
 				    struct i915_power_well *power_well)
 {
-	DRM_DEBUG_KMS("enabling %s\n", power_well->name);
+	DRM_DEBUG_KMS("enabling %s\n", power_well->desc->name);
-	power_well->ops->enable(dev_priv, power_well);
+	power_well->desc->ops->enable(dev_priv, power_well);
 	power_well->hw_enabled = true;
 }
 static void intel_power_well_disable(struct drm_i915_private *dev_priv,
 				     struct i915_power_well *power_well)
 {
-	DRM_DEBUG_KMS("disabling %s\n", power_well->name);
+	DRM_DEBUG_KMS("disabling %s\n", power_well->desc->name);
 	power_well->hw_enabled = false;
-	power_well->ops->disable(dev_priv, power_well);
+	power_well->desc->ops->disable(dev_priv, power_well);
 }
 static void intel_power_well_get(struct drm_i915_private *dev_priv,
 				 struct i915_power_well *power_well)
 {
 	WARN(!power_well->count, "Use count on power well %s is already zero",
-	     power_well->name);
+	     power_well->desc->name);
 	if (!--power_well->count)
 		intel_power_well_disable(dev_priv, power_well);
 	is_enabled = true;
 	for_each_power_domain_well_rev(dev_priv, power_well, BIT_ULL(domain)) {
-		if (power_well->always_on)
+		if (power_well->desc->always_on)
 			continue;
 		if (!power_well->hw_enabled) {
 	return ret;
 }
-/**
- * intel_display_set_init_power - set the initial power domain state
- * @dev_priv: i915 device instance
- * @enable: whether to enable or disable the initial power domain state
- *
- * For simplicity our driver load/unload and system suspend/resume code assumes
- * that all power domains are always enabled. This functions controls the state
- * of this little hack. While the initial power domain state is enabled runtime
- * pm is effectively disabled.
- */
-void intel_display_set_init_power(struct drm_i915_private *dev_priv,
-				  bool enable)
-{
-	if (dev_priv->power_domains.init_power_on == enable)
-		return;
-	if (enable)
-		intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
-	else
-		intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
-	dev_priv->power_domains.init_power_on = enable;
-}
 /*
  * Starting with Haswell, we have a "Power Down Well" that can be turned off
  * when not needed anymore. We have 4 registers that can request the power well
 static void hsw_wait_for_power_well_enable(struct drm_i915_private *dev_priv,
 					   struct i915_power_well *power_well)
 {
-	enum i915_power_well_id id = power_well->id;
+	const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
+	int pw_idx = power_well->desc->hsw.idx;
 	/* Timeout for PW1:10 us, AUX:not specified, other PWs:20 us. */
 	WARN_ON(intel_wait_for_register(dev_priv,
-					HSW_PWR_WELL_CTL_DRIVER(id),
+					regs->driver,
-					HSW_PWR_WELL_CTL_STATE(id),
+					HSW_PWR_WELL_CTL_STATE(pw_idx),
-					HSW_PWR_WELL_CTL_STATE(id),
+					HSW_PWR_WELL_CTL_STATE(pw_idx),
 					1));
 }
 static u32 hsw_power_well_requesters(struct drm_i915_private *dev_priv,
-				     enum i915_power_well_id id)
+				     const struct i915_power_well_regs *regs,
+				     int pw_idx)
 {
-	u32 req_mask = HSW_PWR_WELL_CTL_REQ(id);
+	u32 req_mask = HSW_PWR_WELL_CTL_REQ(pw_idx);
 	u32 ret;
-	ret = I915_READ(HSW_PWR_WELL_CTL_BIOS(id)) & req_mask ? 1 : 0;
+	ret = I915_READ(regs->bios) & req_mask ? 1 : 0;
-	ret |= I915_READ(HSW_PWR_WELL_CTL_DRIVER(id)) & req_mask ? 2 : 0;
+	ret |= I915_READ(regs->driver) & req_mask ? 2 : 0;
-	ret |= I915_READ(HSW_PWR_WELL_CTL_KVMR) & req_mask ? 4 : 0;
+	if (regs->kvmr.reg)
-	ret |= I915_READ(HSW_PWR_WELL_CTL_DEBUG(id)) & req_mask ? 8 : 0;
+		ret |= I915_READ(regs->kvmr) & req_mask ? 4 : 0;
+	ret |= I915_READ(regs->debug) & req_mask ? 8 : 0;
 	return ret;
 }
 static void hsw_wait_for_power_well_disable(struct drm_i915_private *dev_priv,
 					    struct i915_power_well *power_well)
 {
-	enum i915_power_well_id id = power_well->id;
+	const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
+	int pw_idx = power_well->desc->hsw.idx;
 	bool disabled;
 	u32 reqs;
 	 * Skip the wait in case any of the request bits are set and print a
 	 * diagnostic message.
 	 */
-	wait_for((disabled = !(I915_READ(HSW_PWR_WELL_CTL_DRIVER(id)) &
+	wait_for((disabled = !(I915_READ(regs->driver) &
-			       HSW_PWR_WELL_CTL_STATE(id))) ||
+			       HSW_PWR_WELL_CTL_STATE(pw_idx))) ||
-		 (reqs = hsw_power_well_requesters(dev_priv, id)), 1);
+		 (reqs = hsw_power_well_requesters(dev_priv, regs, pw_idx)), 1);
 	if (disabled)
 		return;
 	DRM_DEBUG_KMS("%s forced on (bios:%d driver:%d kvmr:%d debug:%d)\n",
-		      power_well->name,
+		      power_well->desc->name,
 		      !!(reqs & 1), !!(reqs & 2), !!(reqs & 4), !!(reqs & 8));
 }
 static void hsw_power_well_enable(struct drm_i915_private *dev_priv,
 				  struct i915_power_well *power_well)
 {
-	enum i915_power_well_id id = power_well->id;
+	const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
-	bool wait_fuses = power_well->hsw.has_fuses;
+	int pw_idx = power_well->desc->hsw.idx;
+	bool wait_fuses = power_well->desc->hsw.has_fuses;
 	enum skl_power_gate uninitialized_var(pg);
 	u32 val;
 	if (wait_fuses) {
-		pg = INTEL_GEN(dev_priv) >= 11 ? ICL_PW_TO_PG(id) :
+		pg = INTEL_GEN(dev_priv) >= 11 ? ICL_PW_CTL_IDX_TO_PG(pw_idx) :
-						 SKL_PW_TO_PG(id);
+						 SKL_PW_CTL_IDX_TO_PG(pw_idx);
 		/*
 		 * For PW1 we have to wait both for the PW0/PG0 fuse state
 		 * before enabling the power well and PW1/PG1's own fuse
 			gen9_wait_for_power_well_fuses(dev_priv, SKL_PG0);
 	}
-	val = I915_READ(HSW_PWR_WELL_CTL_DRIVER(id));
+	val = I915_READ(regs->driver);
-	I915_WRITE(HSW_PWR_WELL_CTL_DRIVER(id), val | HSW_PWR_WELL_CTL_REQ(id));
+	I915_WRITE(regs->driver, val | HSW_PWR_WELL_CTL_REQ(pw_idx));
 	hsw_wait_for_power_well_enable(dev_priv, power_well);
 	/* Display WA #1178: cnl */
 	if (IS_CANNONLAKE(dev_priv) &&
-	    (id == CNL_DISP_PW_AUX_B || id == CNL_DISP_PW_AUX_C ||
+	    pw_idx >= GLK_PW_CTL_IDX_AUX_B &&
-	     id == CNL_DISP_PW_AUX_D || id == CNL_DISP_PW_AUX_F)) {
+	    pw_idx <= CNL_PW_CTL_IDX_AUX_F) {
-		val = I915_READ(CNL_AUX_ANAOVRD1(id));
+		val = I915_READ(CNL_AUX_ANAOVRD1(pw_idx));
 		val |= CNL_AUX_ANAOVRD1_ENABLE | CNL_AUX_ANAOVRD1_LDO_BYPASS;
-		I915_WRITE(CNL_AUX_ANAOVRD1(id), val);
+		I915_WRITE(CNL_AUX_ANAOVRD1(pw_idx), val);
 	}
 	if (wait_fuses)
 		gen9_wait_for_power_well_fuses(dev_priv, pg);
-	hsw_power_well_post_enable(dev_priv, power_well->hsw.irq_pipe_mask,
+	hsw_power_well_post_enable(dev_priv,
-				   power_well->hsw.has_vga);
+				   power_well->desc->hsw.irq_pipe_mask,
+				   power_well->desc->hsw.has_vga);
 }
 static void hsw_power_well_disable(struct drm_i915_private *dev_priv,
 				   struct i915_power_well *power_well)
 {
-	enum i915_power_well_id id = power_well->id;
+	const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
+	int pw_idx = power_well->desc->hsw.idx;
 	u32 val;
-	hsw_power_well_pre_disable(dev_priv, power_well->hsw.irq_pipe_mask);
+	hsw_power_well_pre_disable(dev_priv,
+				   power_well->desc->hsw.irq_pipe_mask);
-	val = I915_READ(HSW_PWR_WELL_CTL_DRIVER(id));
+	val = I915_READ(regs->driver);
-	I915_WRITE(HSW_PWR_WELL_CTL_DRIVER(id),
+	I915_WRITE(regs->driver, val & ~HSW_PWR_WELL_CTL_REQ(pw_idx));
-		   val & ~HSW_PWR_WELL_CTL_REQ(id));
 	hsw_wait_for_power_well_disable(dev_priv, power_well);
 }
-#define ICL_AUX_PW_TO_PORT(pw)	((pw) - ICL_DISP_PW_AUX_A)
+#define ICL_AUX_PW_TO_PORT(pw_idx)	((pw_idx) - ICL_PW_CTL_IDX_AUX_A)
 static void
 icl_combo_phy_aux_power_well_enable(struct drm_i915_private *dev_priv,
 				    struct i915_power_well *power_well)
 {
-	enum i915_power_well_id id = power_well->id;
+	const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
-	enum port port = ICL_AUX_PW_TO_PORT(id);
+	int pw_idx = power_well->desc->hsw.idx;
+	enum port port = ICL_AUX_PW_TO_PORT(pw_idx);
 	u32 val;
-	val = I915_READ(HSW_PWR_WELL_CTL_DRIVER(id));
+	val = I915_READ(regs->driver);
-	I915_WRITE(HSW_PWR_WELL_CTL_DRIVER(id), val | HSW_PWR_WELL_CTL_REQ(id));
+	I915_WRITE(regs->driver, val | HSW_PWR_WELL_CTL_REQ(pw_idx));
 	val = I915_READ(ICL_PORT_CL_DW12(port));
 	I915_WRITE(ICL_PORT_CL_DW12(port), val | ICL_LANE_ENABLE_AUX);
 icl_combo_phy_aux_power_well_disable(struct drm_i915_private *dev_priv,
 				     struct i915_power_well *power_well)
 {
-	enum i915_power_well_id id = power_well->id;
+	const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
-	enum port port = ICL_AUX_PW_TO_PORT(id);
+	int pw_idx = power_well->desc->hsw.idx;
+	enum port port = ICL_AUX_PW_TO_PORT(pw_idx);
 	u32 val;
 	val = I915_READ(ICL_PORT_CL_DW12(port));
 	I915_WRITE(ICL_PORT_CL_DW12(port), val & ~ICL_LANE_ENABLE_AUX);
-	val = I915_READ(HSW_PWR_WELL_CTL_DRIVER(id));
+	val = I915_READ(regs->driver);
-	I915_WRITE(HSW_PWR_WELL_CTL_DRIVER(id),
+	I915_WRITE(regs->driver, val & ~HSW_PWR_WELL_CTL_REQ(pw_idx));
-		   val & ~HSW_PWR_WELL_CTL_REQ(id));
 	hsw_wait_for_power_well_disable(dev_priv, power_well);
 }
 static bool hsw_power_well_enabled(struct drm_i915_private *dev_priv,
 				   struct i915_power_well *power_well)
 {
-	enum i915_power_well_id id = power_well->id;
+	const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
-	u32 mask = HSW_PWR_WELL_CTL_REQ(id) | HSW_PWR_WELL_CTL_STATE(id);
+	int pw_idx = power_well->desc->hsw.idx;
+	u32 mask = HSW_PWR_WELL_CTL_REQ(pw_idx) |
+		   HSW_PWR_WELL_CTL_STATE(pw_idx);
-	return (I915_READ(HSW_PWR_WELL_CTL_DRIVER(id)) & mask) == mask;
+	return (I915_READ(regs->driver) & mask) == mask;
 }
 static void assert_can_enable_dc9(struct drm_i915_private *dev_priv)
 {
-	enum i915_power_well_id id = SKL_DISP_PW_2;
 	WARN_ONCE((I915_READ(DC_STATE_EN) & DC_STATE_EN_DC9),
 		  "DC9 already programmed to be enabled.\n");
 	WARN_ONCE(I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC5,
 		  "DC5 still not disabled to enable DC9.\n");
-	WARN_ONCE(I915_READ(HSW_PWR_WELL_CTL_DRIVER(id)) &
+	WARN_ONCE(I915_READ(HSW_PWR_WELL_CTL2) &
-		  HSW_PWR_WELL_CTL_REQ(id),
+		  HSW_PWR_WELL_CTL_REQ(SKL_PW_CTL_IDX_PW_2),
 		  "Power well 2 on.\n");
 	WARN_ONCE(intel_irqs_enabled(dev_priv),
 		  "Interrupts not disabled yet.\n");
 	WARN_ONCE(!I915_READ(CSR_HTP_SKL), "CSR HTP Not fine\n");
 }
+static struct i915_power_well *
+lookup_power_well(struct drm_i915_private *dev_priv,
+		  enum i915_power_well_id power_well_id)
+{
+	struct i915_power_well *power_well;
+	for_each_power_well(dev_priv, power_well)
+		if (power_well->desc->id == power_well_id)
+			return power_well;
+	/*
+	 * It's not feasible to add error checking code to the callers since
+	 * this condition really shouldn't happen and it doesn't even make sense
+	 * to abort things like display initialization sequences. Just return
+	 * the first power well and hope the WARN gets reported so we can fix
+	 * our driver.
+	 */
+	WARN(1, "Power well %d not defined for this platform\n", power_well_id);
+	return &dev_priv->power_domains.power_wells[0];
+}
 static void assert_can_enable_dc5(struct drm_i915_private *dev_priv)
 {
 	bool pg2_enabled = intel_display_power_well_is_enabled(dev_priv,
 static void hsw_power_well_sync_hw(struct drm_i915_private *dev_priv,
 				   struct i915_power_well *power_well)
 {
-	enum i915_power_well_id id = power_well->id;
+	const struct i915_power_well_regs *regs = power_well->desc->hsw.regs;
-	u32 mask = HSW_PWR_WELL_CTL_REQ(id);
+	int pw_idx = power_well->desc->hsw.idx;
-	u32 bios_req = I915_READ(HSW_PWR_WELL_CTL_BIOS(id));
+	u32 mask = HSW_PWR_WELL_CTL_REQ(pw_idx);
+	u32 bios_req = I915_READ(regs->bios);
 	/* Take over the request bit if set by BIOS. */
 	if (bios_req & mask) {
-		u32 drv_req = I915_READ(HSW_PWR_WELL_CTL_DRIVER(id));
+		u32 drv_req = I915_READ(regs->driver);
 		if (!(drv_req & mask))
-			I915_WRITE(HSW_PWR_WELL_CTL_DRIVER(id), drv_req | mask);
+			I915_WRITE(regs->driver, drv_req | mask);
-		I915_WRITE(HSW_PWR_WELL_CTL_BIOS(id), bios_req & ~mask);
+		I915_WRITE(regs->bios, bios_req & ~mask);
 	}
 }
 static void bxt_dpio_cmn_power_well_enable(struct drm_i915_private *dev_priv,
 					   struct i915_power_well *power_well)
 {
-	bxt_ddi_phy_init(dev_priv, power_well->bxt.phy);
+	bxt_ddi_phy_init(dev_priv, power_well->desc->bxt.phy);
 }
 static void bxt_dpio_cmn_power_well_disable(struct drm_i915_private *dev_priv,
 					    struct i915_power_well *power_well)
 {
-	bxt_ddi_phy_uninit(dev_priv, power_well->bxt.phy);
+	bxt_ddi_phy_uninit(dev_priv, power_well->desc->bxt.phy);
 }
 static bool bxt_dpio_cmn_power_well_enabled(struct drm_i915_private *dev_priv,
 					    struct i915_power_well *power_well)
 {
-	return bxt_ddi_phy_is_enabled(dev_priv, power_well->bxt.phy);
+	return bxt_ddi_phy_is_enabled(dev_priv, power_well->desc->bxt.phy);
 }
 static void bxt_verify_ddi_phy_power_wells(struct drm_i915_private *dev_priv)
 {
 	struct i915_power_well *power_well;
-	power_well = lookup_power_well(dev_priv, BXT_DPIO_CMN_A);
+	power_well = lookup_power_well(dev_priv, BXT_DISP_PW_DPIO_CMN_A);
 	if (power_well->count > 0)
-		bxt_ddi_phy_verify_state(dev_priv, power_well->bxt.phy);
+		bxt_ddi_phy_verify_state(dev_priv, power_well->desc->bxt.phy);
-	power_well = lookup_power_well(dev_priv, BXT_DPIO_CMN_BC);
+	power_well = lookup_power_well(dev_priv, VLV_DISP_PW_DPIO_CMN_BC);
 	if (power_well->count > 0)
-		bxt_ddi_phy_verify_state(dev_priv, power_well->bxt.phy);
+		bxt_ddi_phy_verify_state(dev_priv, power_well->desc->bxt.phy);
 	if (IS_GEMINILAKE(dev_priv)) {
-		power_well = lookup_power_well(dev_priv, GLK_DPIO_CMN_C);
+		power_well = lookup_power_well(dev_priv,
+					       GLK_DISP_PW_DPIO_CMN_C);
 		if (power_well->count > 0)
-			bxt_ddi_phy_verify_state(dev_priv, power_well->bxt.phy);
+			bxt_ddi_phy_verify_state(dev_priv,
+						 power_well->desc->bxt.phy);
 	}
 }
 static void vlv_set_power_well(struct drm_i915_private *dev_priv,
 			       struct i915_power_well *power_well, bool enable)
 {
-	enum i915_power_well_id power_well_id = power_well->id;
+	int pw_idx = power_well->desc->vlv.idx;
 	u32 mask;
 	u32 state;
 	u32 ctrl;
-	mask = PUNIT_PWRGT_MASK(power_well_id);
+	mask = PUNIT_PWRGT_MASK(pw_idx);
-	state = enable ? PUNIT_PWRGT_PWR_ON(power_well_id) :
+	state = enable ? PUNIT_PWRGT_PWR_ON(pw_idx) :
-			 PUNIT_PWRGT_PWR_GATE(power_well_id);
+			 PUNIT_PWRGT_PWR_GATE(pw_idx);
 	mutex_lock(&dev_priv->pcu_lock);
 static bool vlv_power_well_enabled(struct drm_i915_private *dev_priv,
 				   struct i915_power_well *power_well)
 {
-	enum i915_power_well_id power_well_id = power_well->id;
+	int pw_idx = power_well->desc->vlv.idx;
 	bool enabled = false;
 	u32 mask;
 	u32 state;
 	u32 ctrl;
-	mask = PUNIT_PWRGT_MASK(power_well_id);
+	mask = PUNIT_PWRGT_MASK(pw_idx);
-	ctrl = PUNIT_PWRGT_PWR_ON(power_well_id);
+	ctrl = PUNIT_PWRGT_PWR_ON(pw_idx);
 	mutex_lock(&dev_priv->pcu_lock);
 	 * We only ever set the power-on and power-gate states, anything
 	 * else is unexpected.
 	 */
-	WARN_ON(state != PUNIT_PWRGT_PWR_ON(power_well_id) &&
+	WARN_ON(state != PUNIT_PWRGT_PWR_ON(pw_idx) &&
-		state != PUNIT_PWRGT_PWR_GATE(power_well_id));
+		state != PUNIT_PWRGT_PWR_GATE(pw_idx));
 	if (state == ctrl)
 		enabled = true;
 static void vlv_display_power_well_enable(struct drm_i915_private *dev_priv,
 					  struct i915_power_well *power_well)
 {
-	WARN_ON_ONCE(power_well->id != PUNIT_POWER_WELL_DISP2D);
 	vlv_set_power_well(dev_priv, power_well, true);
 	vlv_display_power_well_init(dev_priv);
 static void vlv_display_power_well_disable(struct drm_i915_private *dev_priv,
 					   struct i915_power_well *power_well)
 {
-	WARN_ON_ONCE(power_well->id != PUNIT_POWER_WELL_DISP2D);
 	vlv_display_power_well_deinit(dev_priv);
 	vlv_set_power_well(dev_priv, power_well, false);
 static void vlv_dpio_cmn_power_well_enable(struct drm_i915_private *dev_priv,
 					   struct i915_power_well *power_well)
 {
-	WARN_ON_ONCE(power_well->id != PUNIT_POWER_WELL_DPIO_CMN_BC);
 	/* since ref/cri clock was enabled */
 	udelay(1); /* >10ns for cmnreset, >0ns for sidereset */
 {
 	enum pipe pipe;
-	WARN_ON_ONCE(power_well->id != PUNIT_POWER_WELL_DPIO_CMN_BC);
 	for_each_pipe(dev_priv, pipe)
 		assert_pll_disabled(dev_priv, pipe);
 #define POWER_DOMAIN_MASK (GENMASK_ULL(POWER_DOMAIN_NUM - 1, 0))
-static struct i915_power_well *
-lookup_power_well(struct drm_i915_private *dev_priv,
-		  enum i915_power_well_id power_well_id)
-{
-	struct i915_power_domains *power_domains = &dev_priv->power_domains;
-	int i;
-	for (i = 0; i < power_domains->power_well_count; i++) {
-		struct i915_power_well *power_well;
-		power_well = &power_domains->power_wells[i];
-		if (power_well->id == power_well_id)
-			return power_well;
-	}
-	return NULL;
-}
 #define BITS_SET(val, bits) (((val) & (bits)) == (bits))
 static void assert_chv_phy_status(struct drm_i915_private *dev_priv)
 {
 	struct i915_power_well *cmn_bc =
-		lookup_power_well(dev_priv, PUNIT_POWER_WELL_DPIO_CMN_BC);
+		lookup_power_well(dev_priv, VLV_DISP_PW_DPIO_CMN_BC);
 	struct i915_power_well *cmn_d =
-		lookup_power_well(dev_priv, PUNIT_POWER_WELL_DPIO_CMN_D);
+		lookup_power_well(dev_priv, CHV_DISP_PW_DPIO_CMN_D);
 	u32 phy_control = dev_priv->chv_phy_control;
 	u32 phy_status = 0;
 	u32 phy_status_mask = 0xffffffff;
 				     PHY_STATUS_SPLINE_LDO(DPIO_PHY1, DPIO_CH0, 0) |
 				     PHY_STATUS_SPLINE_LDO(DPIO_PHY1, DPIO_CH0, 1));
-	if (cmn_bc->ops->is_enabled(dev_priv, cmn_bc)) {
+	if (cmn_bc->desc->ops->is_enabled(dev_priv, cmn_bc)) {
 		phy_status |= PHY_POWERGOOD(DPIO_PHY0);
 		/* this assumes override is only used to enable lanes */
 			phy_status |= PHY_STATUS_SPLINE_LDO(DPIO_PHY0, DPIO_CH1, 1);
 	}
-	if (cmn_d->ops->is_enabled(dev_priv, cmn_d)) {
+	if (cmn_d->desc->ops->is_enabled(dev_priv, cmn_d)) {
 		phy_status |= PHY_POWERGOOD(DPIO_PHY1);
 		/* this assumes override is only used to enable lanes */
 	enum pipe pipe;
 	uint32_t tmp;
-	WARN_ON_ONCE(power_well->id != PUNIT_POWER_WELL_DPIO_CMN_BC &&
+	WARN_ON_ONCE(power_well->desc->id != VLV_DISP_PW_DPIO_CMN_BC &&
-		     power_well->id != PUNIT_POWER_WELL_DPIO_CMN_D);
+		     power_well->desc->id != CHV_DISP_PW_DPIO_CMN_D);
-	if (power_well->id == PUNIT_POWER_WELL_DPIO_CMN_BC) {
+	if (power_well->desc->id == VLV_DISP_PW_DPIO_CMN_BC) {
 		pipe = PIPE_A;
 		phy = DPIO_PHY0;
 	} else {
 		DPIO_SUS_CLK_CONFIG_GATE_CLKREQ;
 	vlv_dpio_write(dev_priv, pipe, CHV_CMN_DW28, tmp);
-	if (power_well->id == PUNIT_POWER_WELL_DPIO_CMN_BC) {
+	if (power_well->desc->id == VLV_DISP_PW_DPIO_CMN_BC) {
 		tmp = vlv_dpio_read(dev_priv, pipe, _CHV_CMN_DW6_CH1);
 		tmp |= DPIO_DYNPWRDOWNEN_CH1;
 		vlv_dpio_write(dev_priv, pipe, _CHV_CMN_DW6_CH1, tmp);
 {
 	enum dpio_phy phy;
-	WARN_ON_ONCE(power_well->id != PUNIT_POWER_WELL_DPIO_CMN_BC &&
+	WARN_ON_ONCE(power_well->desc->id != VLV_DISP_PW_DPIO_CMN_BC &&
-		     power_well->id != PUNIT_POWER_WELL_DPIO_CMN_D);
+		     power_well->desc->id != CHV_DISP_PW_DPIO_CMN_D);
-	if (power_well->id == PUNIT_POWER_WELL_DPIO_CMN_BC) {
+	if (power_well->desc->id == VLV_DISP_PW_DPIO_CMN_BC) {
 		phy = DPIO_PHY0;
 		assert_pll_disabled(dev_priv, PIPE_A);
 		assert_pll_disabled(dev_priv, PIPE_B);
 static void chv_pipe_power_well_enable(struct drm_i915_private *dev_priv,
 				       struct i915_power_well *power_well)
 {
-	WARN_ON_ONCE(power_well->id != CHV_DISP_PW_PIPE_A);
 	chv_set_pipe_power_well(dev_priv, power_well, true);
 	vlv_display_power_well_init(dev_priv);
 static void chv_pipe_power_well_disable(struct drm_i915_private *dev_priv,
 					struct i915_power_well *power_well)
 {
-	WARN_ON_ONCE(power_well->id != CHV_DISP_PW_PIPE_A);
 	vlv_display_power_well_deinit(dev_priv);
 	chv_set_pipe_power_well(dev_priv, power_well, false);
 	.is_enabled = vlv_power_well_enabled,
 };
-static struct i915_power_well i9xx_always_on_power_well[] = {
+static const struct i915_power_well_desc i9xx_always_on_power_well[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 };
 	.is_enabled = i830_pipes_power_well_enabled,
 };
-static struct i915_power_well i830_power_wells[] = {
+static const struct i915_power_well_desc i830_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "pipes",
 		.domains = I830_PIPES_POWER_DOMAINS,
 		.ops = &i830_pipes_power_well_ops,
-		.id = I830_DISP_PW_PIPES,
+		.id = DISP_PW_ID_NONE,
 	},
 };
 	.is_enabled = bxt_dpio_cmn_power_well_enabled,
 };
-static struct i915_power_well hsw_power_wells[] = {
+static const struct i915_power_well_regs hsw_power_well_regs = {
+	.bios	= HSW_PWR_WELL_CTL1,
+	.driver	= HSW_PWR_WELL_CTL2,
+	.kvmr	= HSW_PWR_WELL_CTL3,
+	.debug	= HSW_PWR_WELL_CTL4,
+};
+static const struct i915_power_well_desc hsw_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "display",
 		.ops = &hsw_power_well_ops,
 		.id = HSW_DISP_PW_GLOBAL,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = HSW_PW_CTL_IDX_GLOBAL,
 			.hsw.has_vga = true,
 		},
 	},
 };
-static struct i915_power_well bdw_power_wells[] = {
+static const struct i915_power_well_desc bdw_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "display",
 		.ops = &hsw_power_well_ops,
 		.id = HSW_DISP_PW_GLOBAL,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = HSW_PW_CTL_IDX_GLOBAL,
 			.hsw.irq_pipe_mask = BIT(PIPE_B) | BIT(PIPE_C),
 			.hsw.has_vga = true,
 		},
 	.is_enabled = vlv_power_well_enabled,
 };
-static struct i915_power_well vlv_power_wells[] = {
+static const struct i915_power_well_desc vlv_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "display",
 		.domains = VLV_DISPLAY_POWER_DOMAINS,
-		.id = PUNIT_POWER_WELL_DISP2D,
 		.ops = &vlv_display_power_well_ops,
+		.id = VLV_DISP_PW_DISP2D,
+		{
+			.vlv.idx = PUNIT_PWGT_IDX_DISP2D,
+		},
 	},
 	{
 		.name = "dpio-tx-b-01",
 			   VLV_DPIO_TX_C_LANES_01_POWER_DOMAINS |
 			   VLV_DPIO_TX_C_LANES_23_POWER_DOMAINS,
 		.ops = &vlv_dpio_power_well_ops,
-		.id = PUNIT_POWER_WELL_DPIO_TX_B_LANES_01,
+		.id = DISP_PW_ID_NONE,
+		{
+			.vlv.idx = PUNIT_PWGT_IDX_DPIO_TX_B_LANES_01,
+		},
 	},
 	{
 		.name = "dpio-tx-b-23",
 			   VLV_DPIO_TX_C_LANES_01_POWER_DOMAINS |
 			   VLV_DPIO_TX_C_LANES_23_POWER_DOMAINS,
 		.ops = &vlv_dpio_power_well_ops,
-		.id = PUNIT_POWER_WELL_DPIO_TX_B_LANES_23,
+		.id = DISP_PW_ID_NONE,
+		{
+			.vlv.idx = PUNIT_PWGT_IDX_DPIO_TX_B_LANES_23,
+		},
 	},
 	{
 		.name = "dpio-tx-c-01",
 			   VLV_DPIO_TX_C_LANES_01_POWER_DOMAINS |
 			   VLV_DPIO_TX_C_LANES_23_POWER_DOMAINS,
 		.ops = &vlv_dpio_power_well_ops,
-		.id = PUNIT_POWER_WELL_DPIO_TX_C_LANES_01,
+		.id = DISP_PW_ID_NONE,
+		{
+			.vlv.idx = PUNIT_PWGT_IDX_DPIO_TX_C_LANES_01,
+		},
 	},
 	{
 		.name = "dpio-tx-c-23",
 			   VLV_DPIO_TX_C_LANES_01_POWER_DOMAINS |
 			   VLV_DPIO_TX_C_LANES_23_POWER_DOMAINS,
 		.ops = &vlv_dpio_power_well_ops,
-		.id = PUNIT_POWER_WELL_DPIO_TX_C_LANES_23,
+		.id = DISP_PW_ID_NONE,
+		{
+			.vlv.idx = PUNIT_PWGT_IDX_DPIO_TX_C_LANES_23,
+		},
 	},
 	{
 		.name = "dpio-common",
 		.domains = VLV_DPIO_CMN_BC_POWER_DOMAINS,
-		.id = PUNIT_POWER_WELL_DPIO_CMN_BC,
 		.ops = &vlv_dpio_cmn_power_well_ops,
+		.id = VLV_DISP_PW_DPIO_CMN_BC,
+		{
+			.vlv.idx = PUNIT_PWGT_IDX_DPIO_CMN_BC,
+		},
 	},
 };
-static struct i915_power_well chv_power_wells[] = {
+static const struct i915_power_well_desc chv_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "display",
 		 * required for any pipe to work.
 		 */
 		.domains = CHV_DISPLAY_POWER_DOMAINS,
-		.id = CHV_DISP_PW_PIPE_A,
 		.ops = &chv_pipe_power_well_ops,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "dpio-common-bc",
 		.domains = CHV_DPIO_CMN_BC_POWER_DOMAINS,
-		.id = PUNIT_POWER_WELL_DPIO_CMN_BC,
 		.ops = &chv_dpio_cmn_power_well_ops,
+		.id = VLV_DISP_PW_DPIO_CMN_BC,
+		{
+			.vlv.idx = PUNIT_PWGT_IDX_DPIO_CMN_BC,
+		},
 	},
 	{
 		.name = "dpio-common-d",
 		.domains = CHV_DPIO_CMN_D_POWER_DOMAINS,
-		.id = PUNIT_POWER_WELL_DPIO_CMN_D,
 		.ops = &chv_dpio_cmn_power_well_ops,
+		.id = CHV_DISP_PW_DPIO_CMN_D,
+		{
+			.vlv.idx = PUNIT_PWGT_IDX_DPIO_CMN_D,
+		},
 	},
 };
 	bool ret;
 	power_well = lookup_power_well(dev_priv, power_well_id);
-	ret = power_well->ops->is_enabled(dev_priv, power_well);
+	ret = power_well->desc->ops->is_enabled(dev_priv, power_well);
 	return ret;
 }
-static struct i915_power_well skl_power_wells[] = {
+static const struct i915_power_well_desc skl_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 1",
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_1,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_PW_1,
 			.hsw.has_fuses = true,
 		},
 	},
 		.domains = 0,
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_MISC_IO,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_MISC_IO,
+		},
 	},
 	{
 		.name = "DC off",
 		.domains = SKL_DISPLAY_DC_OFF_POWER_DOMAINS,
 		.ops = &gen9_dc_off_power_well_ops,
-		.id = SKL_DISP_PW_DC_OFF,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 2",
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_2,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_PW_2,
 			.hsw.irq_pipe_mask = BIT(PIPE_B) | BIT(PIPE_C),
 			.hsw.has_vga = true,
 			.hsw.has_fuses = true,
 		.name = "DDI A/E IO power well",
 		.domains = SKL_DISPLAY_DDI_IO_A_E_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_A_E,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_A_E,
+		},
 	},
 	{
 		.name = "DDI B IO power well",
 		.domains = SKL_DISPLAY_DDI_IO_B_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_B,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_B,
+		},
 	},
 	{
 		.name = "DDI C IO power well",
 		.domains = SKL_DISPLAY_DDI_IO_C_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_C,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_C,
+		},
 	},
 	{
 		.name = "DDI D IO power well",
 		.domains = SKL_DISPLAY_DDI_IO_D_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_D,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_D,
+		},
 	},
 };
-static struct i915_power_well bxt_power_wells[] = {
+static const struct i915_power_well_desc bxt_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 1",
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_1,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_PW_1,
 			.hsw.has_fuses = true,
 		},
 	},
 		.name = "DC off",
 		.domains = BXT_DISPLAY_DC_OFF_POWER_DOMAINS,
 		.ops = &gen9_dc_off_power_well_ops,
-		.id = SKL_DISP_PW_DC_OFF,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 2",
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_2,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_PW_2,
 			.hsw.irq_pipe_mask = BIT(PIPE_B) | BIT(PIPE_C),
 			.hsw.has_vga = true,
 			.hsw.has_fuses = true,
 		.name = "dpio-common-a",
 		.domains = BXT_DPIO_CMN_A_POWER_DOMAINS,
 		.ops = &bxt_dpio_cmn_power_well_ops,
-		.id = BXT_DPIO_CMN_A,
+		.id = BXT_DISP_PW_DPIO_CMN_A,
 		{
 			.bxt.phy = DPIO_PHY1,
 		},
 		.name = "dpio-common-bc",
 		.domains = BXT_DPIO_CMN_BC_POWER_DOMAINS,
 		.ops = &bxt_dpio_cmn_power_well_ops,
-		.id = BXT_DPIO_CMN_BC,
+		.id = VLV_DISP_PW_DPIO_CMN_BC,
 		{
 			.bxt.phy = DPIO_PHY0,
 		},
 	},
 };
-static struct i915_power_well glk_power_wells[] = {
+static const struct i915_power_well_desc glk_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 1",
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_1,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_PW_1,
 			.hsw.has_fuses = true,
 		},
 	},
 		.name = "DC off",
 		.domains = GLK_DISPLAY_DC_OFF_POWER_DOMAINS,
 		.ops = &gen9_dc_off_power_well_ops,
-		.id = SKL_DISP_PW_DC_OFF,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 2",
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_2,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_PW_2,
 			.hsw.irq_pipe_mask = BIT(PIPE_B) | BIT(PIPE_C),
 			.hsw.has_vga = true,
 			.hsw.has_fuses = true,
 		.name = "dpio-common-a",
 		.domains = GLK_DPIO_CMN_A_POWER_DOMAINS,
 		.ops = &bxt_dpio_cmn_power_well_ops,
-		.id = BXT_DPIO_CMN_A,
+		.id = BXT_DISP_PW_DPIO_CMN_A,
 		{
 			.bxt.phy = DPIO_PHY1,
 		},
 		.name = "dpio-common-b",
 		.domains = GLK_DPIO_CMN_B_POWER_DOMAINS,
 		.ops = &bxt_dpio_cmn_power_well_ops,
-		.id = BXT_DPIO_CMN_BC,
+		.id = VLV_DISP_PW_DPIO_CMN_BC,
 		{
 			.bxt.phy = DPIO_PHY0,
 		},
 		.name = "dpio-common-c",
 		.domains = GLK_DPIO_CMN_C_POWER_DOMAINS,
 		.ops = &bxt_dpio_cmn_power_well_ops,
-		.id = GLK_DPIO_CMN_C,
+		.id = GLK_DISP_PW_DPIO_CMN_C,
 		{
 			.bxt.phy = DPIO_PHY2,
 		},
 		.name = "AUX A",
 		.domains = GLK_DISPLAY_AUX_A_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = GLK_DISP_PW_AUX_A,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = GLK_PW_CTL_IDX_AUX_A,
+		},
 	},
 	{
 		.name = "AUX B",
 		.domains = GLK_DISPLAY_AUX_B_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = GLK_DISP_PW_AUX_B,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = GLK_PW_CTL_IDX_AUX_B,
+		},
 	},
 	{
 		.name = "AUX C",
 		.domains = GLK_DISPLAY_AUX_C_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = GLK_DISP_PW_AUX_C,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = GLK_PW_CTL_IDX_AUX_C,
+		},
 	},
 	{
 		.name = "DDI A IO power well",
 		.domains = GLK_DISPLAY_DDI_IO_A_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = GLK_DISP_PW_DDI_A,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = GLK_PW_CTL_IDX_DDI_A,
+		},
 	},
 	{
 		.name = "DDI B IO power well",
 		.domains = GLK_DISPLAY_DDI_IO_B_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_B,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_B,
+		},
 	},
 	{
 		.name = "DDI C IO power well",
 		.domains = GLK_DISPLAY_DDI_IO_C_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_C,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_C,
+		},
 	},
 };
-static struct i915_power_well cnl_power_wells[] = {
+static const struct i915_power_well_desc cnl_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 1",
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_1,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_PW_1,
 			.hsw.has_fuses = true,
 		},
 	},
 		.name = "AUX A",
 		.domains = CNL_DISPLAY_AUX_A_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = CNL_DISP_PW_AUX_A,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = GLK_PW_CTL_IDX_AUX_A,
+		},
 	},
 	{
 		.name = "AUX B",
 		.domains = CNL_DISPLAY_AUX_B_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = CNL_DISP_PW_AUX_B,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = GLK_PW_CTL_IDX_AUX_B,
+		},
 	},
 	{
 		.name = "AUX C",
 		.domains = CNL_DISPLAY_AUX_C_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = CNL_DISP_PW_AUX_C,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = GLK_PW_CTL_IDX_AUX_C,
+		},
 	},
 	{
 		.name = "AUX D",
 		.domains = CNL_DISPLAY_AUX_D_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = CNL_DISP_PW_AUX_D,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = CNL_PW_CTL_IDX_AUX_D,
+		},
 	},
 	{
 		.name = "DC off",
 		.domains = CNL_DISPLAY_DC_OFF_POWER_DOMAINS,
 		.ops = &gen9_dc_off_power_well_ops,
-		.id = SKL_DISP_PW_DC_OFF,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 2",
 		.ops = &hsw_power_well_ops,
 		.id = SKL_DISP_PW_2,
 		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_PW_2,
 			.hsw.irq_pipe_mask = BIT(PIPE_B) | BIT(PIPE_C),
 			.hsw.has_vga = true,
 			.hsw.has_fuses = true,
 		.name = "DDI A IO power well",
 		.domains = CNL_DISPLAY_DDI_A_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = CNL_DISP_PW_DDI_A,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = GLK_PW_CTL_IDX_DDI_A,
+		},
 	},
 	{
 		.name = "DDI B IO power well",
 		.domains = CNL_DISPLAY_DDI_B_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_B,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_B,
+		},
 	},
 	{
 		.name = "DDI C IO power well",
 		.domains = CNL_DISPLAY_DDI_C_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_C,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_C,
+		},
 	},
 	{
 		.name = "DDI D IO power well",
 		.domains = CNL_DISPLAY_DDI_D_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = SKL_DISP_PW_DDI_D,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = SKL_PW_CTL_IDX_DDI_D,
+		},
 	},
 	{
 		.name = "DDI F IO power well",
 		.domains = CNL_DISPLAY_DDI_F_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = CNL_DISP_PW_DDI_F,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = CNL_PW_CTL_IDX_DDI_F,
+		},
 	},
 	{
 		.name = "AUX F",
 		.domains = CNL_DISPLAY_AUX_F_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = CNL_DISP_PW_AUX_F,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = CNL_PW_CTL_IDX_AUX_F,
+		},
 	},
 };
 	.is_enabled = hsw_power_well_enabled,
 };
-static struct i915_power_well icl_power_wells[] = {
+static const struct i915_power_well_regs icl_aux_power_well_regs = {
+	.bios	= ICL_PWR_WELL_CTL_AUX1,
+	.driver	= ICL_PWR_WELL_CTL_AUX2,
+	.debug	= ICL_PWR_WELL_CTL_AUX4,
+};
+static const struct i915_power_well_regs icl_ddi_power_well_regs = {
+	.bios	= ICL_PWR_WELL_CTL_DDI1,
+	.driver	= ICL_PWR_WELL_CTL_DDI2,
+	.debug	= ICL_PWR_WELL_CTL_DDI4,
+};
+static const struct i915_power_well_desc icl_power_wells[] = {
 	{
 		.name = "always-on",
 		.always_on = 1,
 		.domains = POWER_DOMAIN_MASK,
 		.ops = &i9xx_always_on_power_well_ops,
-		.id = I915_DISP_PW_ALWAYS_ON,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 1",
 		/* Handled by the DMC firmware */
 		.domains = 0,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_1,
+		.id = SKL_DISP_PW_1,
-		.hsw.has_fuses = true,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_PW_1,
+			.hsw.has_fuses = true,
+		},
 	},
 	{
 		.name = "power well 2",
 		.domains = ICL_PW_2_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_2,
+		.id = SKL_DISP_PW_2,
-		.hsw.has_fuses = true,
+		{
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_PW_2,
+			.hsw.has_fuses = true,
+		},
 	},
 	{
 		.name = "DC off",
 		.domains = ICL_DISPLAY_DC_OFF_POWER_DOMAINS,
 		.ops = &gen9_dc_off_power_well_ops,
-		.id = SKL_DISP_PW_DC_OFF,
+		.id = DISP_PW_ID_NONE,
 	},
 	{
 		.name = "power well 3",
 		.domains = ICL_PW_3_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_3,
+		.id = DISP_PW_ID_NONE,
-		.hsw.irq_pipe_mask = BIT(PIPE_B),
+		{
-		.hsw.has_vga = true,
+			.hsw.regs = &hsw_power_well_regs,
-		.hsw.has_fuses = true,
+			.hsw.idx = ICL_PW_CTL_IDX_PW_3,
+			.hsw.irq_pipe_mask = BIT(PIPE_B),
+			.hsw.has_vga = true,
+			.hsw.has_fuses = true,
+		},
 	},
 	{
 		.name = "DDI A IO",
 		.domains = ICL_DDI_IO_A_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_DDI_A,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_ddi_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_DDI_A,
+		},
 	},
 	{
 		.name = "DDI B IO",
 		.domains = ICL_DDI_IO_B_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_DDI_B,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_ddi_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_DDI_B,
+		},
 	},
 	{
 		.name = "DDI C IO",
 		.domains = ICL_DDI_IO_C_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_DDI_C,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_ddi_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_DDI_C,
+		},
 	},
 	{
 		.name = "DDI D IO",
 		.domains = ICL_DDI_IO_D_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_DDI_D,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_ddi_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_DDI_D,
+		},
 	},
 	{
 		.name = "DDI E IO",
 		.domains = ICL_DDI_IO_E_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_DDI_E,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_ddi_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_DDI_E,
+		},
 	},
 	{
 		.name = "DDI F IO",
 		.domains = ICL_DDI_IO_F_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_DDI_F,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_ddi_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_DDI_F,
+		},
 	},
 	{
 		.name = "AUX A",
 		.domains = ICL_AUX_A_IO_POWER_DOMAINS,
 		.ops = &icl_combo_phy_aux_power_well_ops,
-		.id = ICL_DISP_PW_AUX_A,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_A,
+		},
 	},
 	{
 		.name = "AUX B",
 		.domains = ICL_AUX_B_IO_POWER_DOMAINS,
 		.ops = &icl_combo_phy_aux_power_well_ops,
-		.id = ICL_DISP_PW_AUX_B,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_B,
+		},
 	},
 	{
 		.name = "AUX C",
 		.domains = ICL_AUX_C_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_AUX_C,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_C,
+		},
 	},
 	{
 		.name = "AUX D",
 		.domains = ICL_AUX_D_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_AUX_D,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_D,
+		},
 	},
 	{
 		.name = "AUX E",
 		.domains = ICL_AUX_E_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_AUX_E,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_E,
+		},
 	},
 	{
 		.name = "AUX F",
 		.domains = ICL_AUX_F_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_AUX_F,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_F,
+		},
 	},
 	{
 		.name = "AUX TBT1",
 		.domains = ICL_AUX_TBT1_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_AUX_TBT1,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_TBT1,
+		},
 	},
 	{
 		.name = "AUX TBT2",
 		.domains = ICL_AUX_TBT2_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_AUX_TBT2,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_TBT2,
+		},
 	},
 	{
 		.name = "AUX TBT3",
 		.domains = ICL_AUX_TBT3_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_AUX_TBT3,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_TBT3,
+		},
 	},
 	{
 		.name = "AUX TBT4",
 		.domains = ICL_AUX_TBT4_IO_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_AUX_TBT4,
+		.id = DISP_PW_ID_NONE,
+		{
+			.hsw.regs = &icl_aux_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_AUX_TBT4,
+		},
 	},
 	{
 		.name = "power well 4",
 		.domains = ICL_PW_4_POWER_DOMAINS,
 		.ops = &hsw_power_well_ops,
-		.id = ICL_DISP_PW_4,
+		.id = DISP_PW_ID_NONE,
-		.hsw.has_fuses = true,
+		{
-		.hsw.irq_pipe_mask = BIT(PIPE_C),
+			.hsw.regs = &hsw_power_well_regs,
+			.hsw.idx = ICL_PW_CTL_IDX_PW_4,
+			.hsw.has_fuses = true,
+			.hsw.irq_pipe_mask = BIT(PIPE_C),
+		},
 	},
 };
 	return mask;
 }
-static void assert_power_well_ids_unique(struct drm_i915_private *dev_priv)
+static int
+__set_power_wells(struct i915_power_domains *power_domains,
+		  const struct i915_power_well_desc *power_well_descs,
+		  int power_well_count)
 {
-	struct i915_power_domains *power_domains = &dev_priv->power_domains;
+	u64 power_well_ids = 0;
-	u64 power_well_ids;
 	int i;
-	power_well_ids = 0;
+	power_domains->power_well_count = power_well_count;
-	for (i = 0; i < power_domains->power_well_count; i++) {
+	power_domains->power_wells =
-		enum i915_power_well_id id = power_domains->power_wells[i].id;
+				kcalloc(power_well_count,
+					sizeof(*power_domains->power_wells),
+					GFP_KERNEL);
+	if (!power_domains->power_wells)
+		return -ENOMEM;
+	for (i = 0; i < power_well_count; i++) {
+		enum i915_power_well_id id = power_well_descs[i].id;
+		power_domains->power_wells[i].desc = &power_well_descs[i];
+		if (id == DISP_PW_ID_NONE)
+			continue;
 		WARN_ON(id >= sizeof(power_well_ids) * 8);
 		WARN_ON(power_well_ids & BIT_ULL(id));
 		power_well_ids |= BIT_ULL(id);
 	}
+	return 0;
 }
-#define set_power_wells(power_domains, __power_wells) ({		\
+#define set_power_wells(power_domains, __power_well_descs) \
-	(power_domains)->power_wells = (__power_wells);			\
+	__set_power_wells(power_domains, __power_well_descs, \
-	(power_domains)->power_well_count = ARRAY_SIZE(__power_wells);	\
+			  ARRAY_SIZE(__power_well_descs))
-})
 /**
  * intel_power_domains_init - initializes the power domain structures
 int intel_power_domains_init(struct drm_i915_private *dev_priv)
 {
 	struct i915_power_domains *power_domains = &dev_priv->power_domains;
+	int err;
 	i915_modparams.disable_power_well =
 		sanitize_disable_power_well_option(dev_priv,
 	 * the disabling order is reversed.
 	 */
 	if (IS_ICELAKE(dev_priv)) {
-		set_power_wells(power_domains, icl_power_wells);
+		err = set_power_wells(power_domains, icl_power_wells);
 	} else if (IS_HASWELL(dev_priv)) {
-		set_power_wells(power_domains, hsw_power_wells);
+		err = set_power_wells(power_domains, hsw_power_wells);
 	} else if (IS_BROADWELL(dev_priv)) {
-		set_power_wells(power_domains, bdw_power_wells);
+		err = set_power_wells(power_domains, bdw_power_wells);
 	} else if (IS_GEN9_BC(dev_priv)) {
-		set_power_wells(power_domains, skl_power_wells);
+		err = set_power_wells(power_domains, skl_power_wells);
 	} else if (IS_CANNONLAKE(dev_priv)) {
-		set_power_wells(power_domains, cnl_power_wells);
+		err = set_power_wells(power_domains, cnl_power_wells);
 		/*
 		 * DDI and Aux IO are getting enabled for all ports
 			power_domains->power_well_count -= 2;
 	} else if (IS_BROXTON(dev_priv)) {
-		set_power_wells(power_domains, bxt_power_wells);
+		err = set_power_wells(power_domains, bxt_power_wells);
 	} else if (IS_GEMINILAKE(dev_priv)) {
-		set_power_wells(power_domains, glk_power_wells);
+		err = set_power_wells(power_domains, glk_power_wells);
 	} else if (IS_CHERRYVIEW(dev_priv)) {
-		set_power_wells(power_domains, chv_power_wells);
+		err = set_power_wells(power_domains, chv_power_wells);
 	} else if (IS_VALLEYVIEW(dev_priv)) {
-		set_power_wells(power_domains, vlv_power_wells);
+		err = set_power_wells(power_domains, vlv_power_wells);
 	} else if (IS_I830(dev_priv)) {
-		set_power_wells(power_domains, i830_power_wells);
+		err = set_power_wells(power_domains, i830_power_wells);
 	} else {
-		set_power_wells(power_domains, i9xx_always_on_power_well);
+		err = set_power_wells(power_domains, i9xx_always_on_power_well);
 	}
-	assert_power_well_ids_unique(dev_priv);
+	return err;
-	return 0;
 }
 /**
- * intel_power_domains_fini - finalizes the power domain structures
+ * intel_power_domains_cleanup - clean up power domains resources
  * @dev_priv: i915 device instance
  *
- * Finalizes the power domain structures for @dev_priv depending upon the
+ * Release any resources acquired by intel_power_domains_init()
- * supported platform. This function also disables runtime pm and ensures that
- * the device stays powered up so that the driver can be reloaded.
  */
-void intel_power_domains_fini(struct drm_i915_private *dev_priv)
+void intel_power_domains_cleanup(struct drm_i915_private *dev_priv)
 {
-	struct device *kdev = &dev_priv->drm.pdev->dev;
+	kfree(dev_priv->power_domains.power_wells);
-	/*
-	 * The i915.ko module is still not prepared to be loaded when
-	 * the power well is not enabled, so just enable it in case
-	 * we're going to unload/reload.
-	 * The following also reacquires the RPM reference the core passed
-	 * to the driver during loading, which is dropped in
-	 * intel_runtime_pm_enable(). We have to hand back the control of the
-	 * device to the core with this reference held.
-	 */
-	intel_display_set_init_power(dev_priv, true);
-	/* Remove the refcount we took to keep power well support disabled. */
-	if (!i915_modparams.disable_power_well)
-		intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
-	/*
-	 * Remove the refcount we took in intel_runtime_pm_enable() in case
-	 * the platform doesn't support runtime PM.
-	 */
-	if (!HAS_RUNTIME_PM(dev_priv))
-		pm_runtime_put(kdev);
 }
 static void intel_power_domains_sync_hw(struct drm_i915_private *dev_priv)
 	mutex_lock(&power_domains->lock);
 	for_each_power_well(dev_priv, power_well) {
-		power_well->ops->sync_hw(dev_priv, power_well);
+		power_well->desc->ops->sync_hw(dev_priv, power_well);
-		power_well->hw_enabled = power_well->ops->is_enabled(dev_priv,
+		power_well->hw_enabled =
-								     power_well);
+			power_well->desc->ops->is_enabled(dev_priv, power_well);
 	}
 	mutex_unlock(&power_domains->lock);
 }
 	 *    The AUX IO power wells will be enabled on demand.
 	 */
 	mutex_lock(&power_domains->lock);
-	well = lookup_power_well(dev_priv, ICL_DISP_PW_1);
+	well = lookup_power_well(dev_priv, SKL_DISP_PW_1);
 	intel_power_well_enable(dev_priv, well);
 	mutex_unlock(&power_domains->lock);
 	/* 7. Setup MBUS. */
 	icl_mbus_init(dev_priv);
-	/* 8. CHICKEN_DCPR_1 */
-	I915_WRITE(GEN8_CHICKEN_DCPR_1, I915_READ(GEN8_CHICKEN_DCPR_1) |
-					CNL_DDI_CLOCK_REG_ACCESS_ON);
 }
 static void icl_display_core_uninit(struct drm_i915_private *dev_priv)
 	 *    disabled at this point.
 	 */
 	mutex_lock(&power_domains->lock);
-	well = lookup_power_well(dev_priv, ICL_DISP_PW_1);
+	well = lookup_power_well(dev_priv, SKL_DISP_PW_1);
 	intel_power_well_disable(dev_priv, well);
 	mutex_unlock(&power_domains->lock);
 static void chv_phy_control_init(struct drm_i915_private *dev_priv)
 {
 	struct i915_power_well *cmn_bc =
-		lookup_power_well(dev_priv, PUNIT_POWER_WELL_DPIO_CMN_BC);
+		lookup_power_well(dev_priv, VLV_DISP_PW_DPIO_CMN_BC);
 	struct i915_power_well *cmn_d =
-		lookup_power_well(dev_priv, PUNIT_POWER_WELL_DPIO_CMN_D);
+		lookup_power_well(dev_priv, CHV_DISP_PW_DPIO_CMN_D);
 	/*
 	 * DISPLAY_PHY_CONTROL can get corrupted if read. As a
 	 * override and set the lane powerdown bits accding to the
 	 * current lane status.
 	 */
-	if (cmn_bc->ops->is_enabled(dev_priv, cmn_bc)) {
+	if (cmn_bc->desc->ops->is_enabled(dev_priv, cmn_bc)) {
 		uint32_t status = I915_READ(DPLL(PIPE_A));
 		unsigned int mask;
 		dev_priv->chv_phy_assert[DPIO_PHY0] = true;
 	}
-	if (cmn_d->ops->is_enabled(dev_priv, cmn_d)) {
+	if (cmn_d->desc->ops->is_enabled(dev_priv, cmn_d)) {
 		uint32_t status = I915_READ(DPIO_PHY_STATUS);
 		unsigned int mask;
 static void vlv_cmnlane_wa(struct drm_i915_private *dev_priv)
 {
 	struct i915_power_well *cmn =
-		lookup_power_well(dev_priv, PUNIT_POWER_WELL_DPIO_CMN_BC);
+		lookup_power_well(dev_priv, VLV_DISP_PW_DPIO_CMN_BC);
 	struct i915_power_well *disp2d =
-		lookup_power_well(dev_priv, PUNIT_POWER_WELL_DISP2D);
+		lookup_power_well(dev_priv, VLV_DISP_PW_DISP2D);
 	/* If the display might be already active skip this */
-	if (cmn->ops->is_enabled(dev_priv, cmn) &&
+	if (cmn->desc->ops->is_enabled(dev_priv, cmn) &&
-	    disp2d->ops->is_enabled(dev_priv, disp2d) &&
+	    disp2d->desc->ops->is_enabled(dev_priv, disp2d) &&
 	    I915_READ(DPIO_CTL) & DPIO_CMNRST)
 		return;
 	DRM_DEBUG_KMS("toggling display PHY side reset\n");
 	/* cmnlane needs DPLL registers */
-	disp2d->ops->enable(dev_priv, disp2d);
+	disp2d->desc->ops->enable(dev_priv, disp2d);
 	/*
 	 * From VLV2A0_DP_eDP_HDMI_DPIO_driver_vbios_notes_11.docx:
 	 * Simply ungating isn't enough to reset the PHY enough to get
 	 * ports and lanes running.
 	 */
-	cmn->ops->disable(dev_priv, cmn);
+	cmn->desc->ops->disable(dev_priv, cmn);
 }
+static void intel_power_domains_verify_state(struct drm_i915_private *dev_priv);
 /**
  * intel_power_domains_init_hw - initialize hardware power domain state
  * @dev_priv: i915 device instance
  *
  * This function initializes the hardware power domain state and enables all
  * power wells belonging to the INIT power domain. Power wells in other
- * domains (and not in the INIT domain) are referenced or disabled during the
+ * domains (and not in the INIT domain) are referenced or disabled by
- * modeset state HW readout. After that the reference count of each power well
+ * intel_modeset_readout_hw_state(). After that the reference count of each
- * must match its HW enabled state, see intel_power_domains_verify_state().
+ * power well must match its HW enabled state, see
+ * intel_power_domains_verify_state().
+ *
+ * It will return with power domains disabled (to be enabled later by
+ * intel_power_domains_enable()) and must be paired with
+ * intel_power_domains_fini_hw().
  */
 void intel_power_domains_init_hw(struct drm_i915_private *dev_priv, bool resume)
 {
 		mutex_unlock(&power_domains->lock);
 	}
-	/* For now, we need the power well to be always enabled. */
+	/*
-	intel_display_set_init_power(dev_priv, true);
+	 * Keep all power wells enabled for any dependent HW access during
+	 * initialization and to make sure we keep BIOS enabled display HW
+	 * resources powered until display HW readout is complete. We drop
+	 * this reference in intel_power_domains_enable().
+	 */
+	intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
 	/* Disable power support if the user asked so. */
 	if (!i915_modparams.disable_power_well)
 		intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
 	intel_power_domains_sync_hw(dev_priv);
 	power_domains->initializing = false;
 }
+/**
+ * intel_power_domains_fini_hw - deinitialize hw power domain state
+ * @dev_priv: i915 device instance
+ *
+ * De-initializes the display power domain HW state. It also ensures that the
+ * device stays powered up so that the driver can be reloaded.
+ *
+ * It must be called with power domains already disabled (after a call to
+ * intel_power_domains_disable()) and must be paired with
+ * intel_power_domains_init_hw().
+ */
+void intel_power_domains_fini_hw(struct drm_i915_private *dev_priv)
+{
+	/* Keep the power well enabled, but cancel its rpm wakeref. */
+	intel_runtime_pm_put(dev_priv);
+	/* Remove the refcount we took to keep power well support disabled. */
+	if (!i915_modparams.disable_power_well)
+		intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
+	intel_power_domains_verify_state(dev_priv);
+}
+/**
+ * intel_power_domains_enable - enable toggling of display power wells
+ * @dev_priv: i915 device instance
+ *
+ * Enable the ondemand enabling/disabling of the display power wells. Note that
+ * power wells not belonging to POWER_DOMAIN_INIT are allowed to be toggled
+ * only at specific points of the display modeset sequence, thus they are not
+ * affected by the intel_power_domains_enable()/disable() calls. The purpose
+ * of these function is to keep the rest of power wells enabled until the end
+ * of display HW readout (which will acquire the power references reflecting
+ * the current HW state).
+ */
+void intel_power_domains_enable(struct drm_i915_private *dev_priv)
+{
+	intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
+	intel_power_domains_verify_state(dev_priv);
+}
+/**
+ * intel_power_domains_disable - disable toggling of display power wells
+ * @dev_priv: i915 device instance
+ *
+ * Disable the ondemand enabling/disabling of the display power wells. See
+ * intel_power_domains_enable() for which power wells this call controls.
+ */
+void intel_power_domains_disable(struct drm_i915_private *dev_priv)
+{
+	intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
+	intel_power_domains_verify_state(dev_priv);
+}
 /**
  * intel_power_domains_suspend - suspend power domain state
  * @dev_priv: i915 device instance
+ * @suspend_mode: specifies the target suspend state (idle, mem, hibernation)
  *
  * This function prepares the hardware power domain state before entering
- * system suspend. It must be paired with intel_power_domains_init_hw().
+ * system suspend.
+ *
+ * It must be called with power domains already disabled (after a call to
+ * intel_power_domains_disable()) and paired with intel_power_domains_resume().
  */
-void intel_power_domains_suspend(struct drm_i915_private *dev_priv)
+void intel_power_domains_suspend(struct drm_i915_private *dev_priv,
+				 enum i915_drm_suspend_mode suspend_mode)
 {
+	struct i915_power_domains *power_domains = &dev_priv->power_domains;
+	intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
+	/*
+	 * In case of suspend-to-idle (aka S0ix) on a DMC platform without DC9
+	 * support don't manually deinit the power domains. This also means the
+	 * CSR/DMC firmware will stay active, it will power down any HW
+	 * resources as required and also enable deeper system power states
+	 * that would be blocked if the firmware was inactive.
+	 */
+	if (!(dev_priv->csr.allowed_dc_mask & DC_STATE_EN_DC9) &&
+	    suspend_mode == I915_DRM_SUSPEND_IDLE &&
+	    dev_priv->csr.dmc_payload != NULL) {
+		intel_power_domains_verify_state(dev_priv);
+		return;
+	}
 	/*
 	 * Even if power well support was disabled we still want to disable
-	 * power wells while we are system suspended.
+	 * power wells if power domains must be deinitialized for suspend.
 	 */
-	if (!i915_modparams.disable_power_well)
+	if (!i915_modparams.disable_power_well) {
 		intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
+		intel_power_domains_verify_state(dev_priv);
+	}
 	if (IS_ICELAKE(dev_priv))
 		icl_display_core_uninit(dev_priv);
 		skl_display_core_uninit(dev_priv);
 	else if (IS_GEN9_LP(dev_priv))
 		bxt_display_core_uninit(dev_priv);
+	power_domains->display_core_suspended = true;
 }
+/**
+ * intel_power_domains_resume - resume power domain state
+ * @dev_priv: i915 device instance
+ *
+ * This function resume the hardware power domain state during system resume.
+ *
+ * It will return with power domain support disabled (to be enabled later by
+ * intel_power_domains_enable()) and must be paired with
+ * intel_power_domains_suspend().
+ */
+void intel_power_domains_resume(struct drm_i915_private *dev_priv)
+{
+	struct i915_power_domains *power_domains = &dev_priv->power_domains;
+	if (power_domains->display_core_suspended) {
+		intel_power_domains_init_hw(dev_priv, true);
+		power_domains->display_core_suspended = false;
+	} else {
+		intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
+	}
+	intel_power_domains_verify_state(dev_priv);
+}
+#if IS_ENABLED(CONFIG_DRM_I915_DEBUG_RUNTIME_PM)
 static void intel_power_domains_dump_info(struct drm_i915_private *dev_priv)
 {
 	struct i915_power_domains *power_domains = &dev_priv->power_domains;
 		enum intel_display_power_domain domain;
 		DRM_DEBUG_DRIVER("%-25s %d\n",
-				 power_well->name, power_well->count);
+				 power_well->desc->name, power_well->count);
-		for_each_power_domain(domain, power_well->domains)
+		for_each_power_domain(domain, power_well->desc->domains)
 			DRM_DEBUG_DRIVER("  %-23s %d\n",
 					 intel_display_power_domain_str(domain),
 					 power_domains->domain_use_count[domain]);
  * acquiring reference counts for any power wells in use and disabling the
  * ones left on by BIOS but not required by any active output.
  */
-void intel_power_domains_verify_state(struct drm_i915_private *dev_priv)
+static void intel_power_domains_verify_state(struct drm_i915_private *dev_priv)
 {
 	struct i915_power_domains *power_domains = &dev_priv->power_domains;
 	struct i915_power_well *power_well;
 		 * and PW1 power wells) are under FW control, so ignore them,
 		 * since their state can change asynchronously.
 		 */
-		if (!power_well->domains)
+		if (!power_well->desc->domains)
 			continue;
-		enabled = power_well->ops->is_enabled(dev_priv, power_well);
+		enabled = power_well->desc->ops->is_enabled(dev_priv,
-		if ((power_well->count || power_well->always_on) != enabled)
+							    power_well);
+		if ((power_well->count || power_well->desc->always_on) !=
+		    enabled)
 			DRM_ERROR("power well %s state mismatch (refcount %d/enabled %d)",
-				  power_well->name, power_well->count, enabled);
+				  power_well->desc->name,
+				  power_well->count, enabled);
 		domains_count = 0;
-		for_each_power_domain(domain, power_well->domains)
+		for_each_power_domain(domain, power_well->desc->domains)
 			domains_count += power_domains->domain_use_count[domain];
 		if (power_well->count != domains_count) {
 			DRM_ERROR("power well %s refcount/domain refcount mismatch "
 				  "(refcount %d/domains refcount %d)\n",
-				  power_well->name, power_well->count,
+				  power_well->desc->name, power_well->count,
 				  domains_count);
 			dump_domain_info = true;
 		}
 	mutex_unlock(&power_domains->lock);
 }
+#else
+static void intel_power_domains_verify_state(struct drm_i915_private *dev_priv)
+{
+}
+#endif
 /**
  * intel_runtime_pm_get - grab a runtime pm reference
  * @dev_priv: i915 device instance
  * This function enables runtime pm at the end of the driver load sequence.
  *
  * Note that this function does currently not enable runtime pm for the
- * subordinate display power domains. That is only done on the first modeset
+ * subordinate display power domains. That is done by
- * using intel_display_set_init_power().
+ * intel_power_domains_enable().
  */
 void intel_runtime_pm_enable(struct drm_i915_private *dev_priv)
 {
 	struct pci_dev *pdev = dev_priv->drm.pdev;
 	struct device *kdev = &pdev->dev;
+	/*
+	 * Disable the system suspend direct complete optimization, which can
+	 * leave the device suspended skipping the driver's suspend handlers
+	 * if the device was already runtime suspended. This is needed due to
+	 * the difference in our runtime and system suspend sequence and
+	 * becaue the HDA driver may require us to enable the audio power
+	 * domain during system suspend.
+	 */
+	dev_pm_set_driver_flags(kdev, DPM_FLAG_NEVER_SKIP);
 	pm_runtime_set_autosuspend_delay(kdev, 10000); /* 10s */
 	pm_runtime_mark_last_busy(kdev);
 	 */
 	pm_runtime_put_autosuspend(kdev);
 }
+void intel_runtime_pm_disable(struct drm_i915_private *dev_priv)
+{
+	struct pci_dev *pdev = dev_priv->drm.pdev;
+	struct device *kdev = &pdev->dev;
+	/* Transfer rpm ownership back to core */
+	WARN(pm_runtime_get_sync(&dev_priv->drm.pdev->dev) < 0,
+	     "Failed to pass rpm ownership back to core\n");
+	pm_runtime_dont_use_autosuspend(kdev);
+	if (!HAS_RUNTIME_PM(dev_priv))
+		pm_runtime_put(kdev);
+}

drivers/gpu/drm/i915/intel_sprite.c+7−6

 	bool need_vlv_dsi_wa = (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) &&
 		intel_crtc_has_type(new_crtc_state, INTEL_OUTPUT_DSI);
 	DEFINE_WAIT(wait);
+	u32 psr_status;
 	vblank_start = adjusted_mode->crtc_vblank_start;
 	if (adjusted_mode->flags & DRM_MODE_FLAG_INTERLACE)
 	 * VBL interrupts will start the PSR exit and prevent a PSR
 	 * re-entry as well.
 	 */
-	if (intel_psr_wait_for_idle(new_crtc_state))
+	if (intel_psr_wait_for_idle(new_crtc_state, &psr_status))
-		DRM_ERROR("PSR idle timed out, atomic update may fail\n");
+		DRM_ERROR("PSR idle timed out 0x%x, atomic update may fail\n",
+			  psr_status);
 	local_irq_disable();
 }
 static int
-intel_check_sprite_plane(struct intel_plane *plane,
+intel_check_sprite_plane(struct intel_crtc_state *crtc_state,
-			 struct intel_crtc_state *crtc_state,
 			 struct intel_plane_state *state)
 {
+	struct intel_plane *plane = to_intel_plane(state->base.plane);
 	struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
 	struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
 	struct drm_framebuffer *fb = state->base.fb;
 	case DRM_FORMAT_XBGR8888:
 	case DRM_FORMAT_ARGB8888:
 	case DRM_FORMAT_ABGR8888:
-		if (modifier == I915_FORMAT_MOD_Yf_TILED_CCS ||
+		if (is_ccs_modifier(modifier))
-		    modifier == I915_FORMAT_MOD_Y_TILED_CCS)
 			return true;
 		/* fall through */
 	case DRM_FORMAT_RGB565:

drivers/gpu/drm/i915/intel_uc_fw.c+1−1

@@ -222,7 +222,7 @@ int intel_uc_fw_upload(struct intel_uc_fw *uc_fw, · 221 unmodified lines
222	goto fail;	222	goto fail;
223	}	223	}
224		224
225	ggtt_pin_bias = to_i915(uc_fw->obj->base.dev)->guc.ggtt_pin_bias;	225	ggtt_pin_bias = to_i915(uc_fw->obj->base.dev)->ggtt.pin_bias;
226	vma = i915_gem_object_ggtt_pin(uc_fw->obj, NULL, 0, 0,	226	vma = i915_gem_object_ggtt_pin(uc_fw->obj, NULL, 0, 0,
227	PIN_OFFSET_BIAS \| ggtt_pin_bias);	227	PIN_OFFSET_BIAS \| ggtt_pin_bias);
228	if (IS_ERR(vma)) {	228	if (IS_ERR(vma)) {

drivers/gpu/drm/i915/intel_uncore.c+92−38

@@ -283,14 +283,24 @@ fw_domains_reset(struct drm_i915_private *i915, · 282 unmodified lines
283	fw_domain_reset(i915, d);	283	fw_domain_reset(i915, d);
284	}	284	}
285		285
		286	static inline u32 gt_thread_status(struct drm_i915_private *dev_priv)
		287	{
		288	u32 val;
		289
		290	val = __raw_i915_read32(dev_priv, GEN6_GT_THREAD_STATUS_REG);
		291	val &= GEN6_GT_THREAD_STATUS_CORE_MASK;
		292
		293	return val;
		294	}
		295
286	static void __gen6_gt_wait_for_thread_c0(struct drm_i915_private *dev_priv)	296	static void __gen6_gt_wait_for_thread_c0(struct drm_i915_private *dev_priv)
287	{	297	{
288	/* w/a for a sporadic read returning 0 by waiting for the GT	298	/*
		299	* w/a for a sporadic read returning 0 by waiting for the GT
289	* thread to wake up.	300	* thread to wake up.
290	*/	301	*/
291	if (wait_for_atomic_us((__raw_i915_read32(dev_priv, GEN6_GT_THREAD_STATUS_REG) &	302	WARN_ONCE(wait_for_atomic_us(gt_thread_status(dev_priv) == 0, 5000),
292	GEN6_GT_THREAD_STATUS_CORE_MASK) == 0, 500))	303	"GT thread status wait timed out\n");
293	DRM_ERROR("GT thread status wait timed out\n");
294	}	304	}
295		305
296	static void fw_domains_get_with_thread_status(struct drm_i915_private *dev_priv,	306	static void fw_domains_get_with_thread_status(struct drm_i915_private *dev_priv,
@@ -1729,7 +1739,7 @@ static void gen3_stop_engine(struct intel_engine_cs *engine) · 1432 unmodified lines
1729	}	1739	}
1730		1740
1731	static void i915_stop_engines(struct drm_i915_private *dev_priv,	1741	static void i915_stop_engines(struct drm_i915_private *dev_priv,
1732	unsigned engine_mask)	1742	unsigned int engine_mask)
1733	{	1743	{
1734	struct intel_engine_cs *engine;	1744	struct intel_engine_cs *engine;
1735	enum intel_engine_id id;	1745	enum intel_engine_id id;
@@ -1749,7 +1759,9 @@ static bool i915_in_reset(struct pci_dev *pdev) · 13 unmodified lines
1749	return gdrst & GRDOM_RESET_STATUS;	1759	return gdrst & GRDOM_RESET_STATUS;
1750	}	1760	}
1751		1761
1752	static int i915_do_reset(struct drm_i915_private *dev_priv, unsigned engine_mask)	1762	static int i915_do_reset(struct drm_i915_private *dev_priv,
		1763	unsigned int engine_mask,
		1764	unsigned int retry)
1753	{	1765	{
1754	struct pci_dev *pdev = dev_priv->drm.pdev;	1766	struct pci_dev *pdev = dev_priv->drm.pdev;
1755	int err;	1767	int err;
@@ -1776,7 +1788,9 @@ static bool g4x_reset_complete(struct pci_dev *pdev) · 20 unmodified lines
1776	return (gdrst & GRDOM_RESET_ENABLE) == 0;	1788	return (gdrst & GRDOM_RESET_ENABLE) == 0;
1777	}	1789	}
1778		1790
1779	static int g33_do_reset(struct drm_i915_private *dev_priv, unsigned engine_mask)	1791	static int g33_do_reset(struct drm_i915_private *dev_priv,
		1792	unsigned int engine_mask,
		1793	unsigned int retry)
1780	{	1794	{
1781	struct pci_dev *pdev = dev_priv->drm.pdev;	1795	struct pci_dev *pdev = dev_priv->drm.pdev;
1782		1796
@@ -1784,7 +1798,9 @@ static int g33_do_reset(struct drm_i915_private *dev_priv, unsigned engine_mask) · 1 unmodified line
1784	return wait_for(g4x_reset_complete(pdev), 500);	1798	return wait_for(g4x_reset_complete(pdev), 500);
1785	}	1799	}
1786		1800
1787	static int g4x_do_reset(struct drm_i915_private *dev_priv, unsigned engine_mask)	1801	static int g4x_do_reset(struct drm_i915_private *dev_priv,
		1802	unsigned int engine_mask,
		1803	unsigned int retry)
1788	{	1804	{
1789	struct pci_dev *pdev = dev_priv->drm.pdev;	1805	struct pci_dev *pdev = dev_priv->drm.pdev;
1790	int ret;	1806	int ret;
@@ -1821,7 +1837,8 @@ out: · 30 unmodified lines
1821	}	1837	}
1822		1838
1823	static int ironlake_do_reset(struct drm_i915_private *dev_priv,	1839	static int ironlake_do_reset(struct drm_i915_private *dev_priv,
1824	unsigned engine_mask)	1840	unsigned int engine_mask,
		1841	unsigned int retry)
1825	{	1842	{
1826	int ret;	1843	int ret;
1827		1844
@@ -1877,6 +1894,7 @@ static int gen6_hw_domain_reset(struct drm_i915_private *dev_priv, · 49 unmodified lines
1877	* gen6_reset_engines - reset individual engines	1894	* gen6_reset_engines - reset individual engines
1878	* @dev_priv: i915 device	1895	* @dev_priv: i915 device
1879	* @engine_mask: mask of intel_ring_flag() engines or ALL_ENGINES for full reset	1896	* @engine_mask: mask of intel_ring_flag() engines or ALL_ENGINES for full reset
		1897	* @retry: the count of of previous attempts to reset.
1880	*	1898	*
1881	* This function will reset the individual engines that are set in engine_mask.	1899	* This function will reset the individual engines that are set in engine_mask.
1882	* If you provide ALL_ENGINES as mask, full global domain reset will be issued.	1900	* If you provide ALL_ENGINES as mask, full global domain reset will be issued.
@@ -1887,7 +1905,8 @@ static int gen6_hw_domain_reset(struct drm_i915_private *dev_priv, · 4 unmodified lines
1887	* Returns 0 on success, nonzero on error.	1905	* Returns 0 on success, nonzero on error.
1888	*/	1906	*/
1889	static int gen6_reset_engines(struct drm_i915_private *dev_priv,	1907	static int gen6_reset_engines(struct drm_i915_private *dev_priv,
1890	unsigned engine_mask)	1908	unsigned int engine_mask,
		1909	unsigned int retry)
1891	{	1910	{
1892	struct intel_engine_cs *engine;	1911	struct intel_engine_cs *engine;
1893	const u32 hw_engine_mask[I915_NUM_ENGINES] = {	1912	const u32 hw_engine_mask[I915_NUM_ENGINES] = {
@@ -1926,7 +1945,7 @@ static int gen6_reset_engines(struct drm_i915_private *dev_priv, · 32 unmodified lines
1926	* Returns 0 on success, nonzero on error.	1945	* Returns 0 on success, nonzero on error.
1927	*/	1946	*/
1928	static int gen11_reset_engines(struct drm_i915_private *dev_priv,	1947	static int gen11_reset_engines(struct drm_i915_private *dev_priv,
1929	unsigned engine_mask)	1948	unsigned int engine_mask)
1930	{	1949	{
1931	struct intel_engine_cs *engine;	1950	struct intel_engine_cs *engine;
1932	const u32 hw_engine_mask[I915_NUM_ENGINES] = {	1951	const u32 hw_engine_mask[I915_NUM_ENGINES] = {
@@ -2066,7 +2085,7 @@ int __intel_wait_for_register(struct drm_i915_private *dev_priv, · 133 unmodified lines
2066	return ret;	2085	return ret;
2067	}	2086	}
2068		2087
2069	static int gen8_reset_engine_start(struct intel_engine_cs *engine)	2088	static int gen8_engine_reset_prepare(struct intel_engine_cs *engine)
2070	{	2089	{
2071	struct drm_i915_private *dev_priv = engine->i915;	2090	struct drm_i915_private *dev_priv = engine->i915;
2072	int ret;	2091	int ret;
@@ -2086,7 +2105,7 @@ static int gen8_reset_engine_start(struct intel_engine_cs *engine) · 13 unmodified lines
2086	return ret;	2105	return ret;
2087	}	2106	}
2088		2107
2089	static void gen8_reset_engine_cancel(struct intel_engine_cs *engine)	2108	static void gen8_engine_reset_cancel(struct intel_engine_cs *engine)
2090	{	2109	{
2091	struct drm_i915_private *dev_priv = engine->i915;	2110	struct drm_i915_private *dev_priv = engine->i915;
2092		2111
@@ -2094,33 +2113,56 @@ static void gen8_reset_engine_cancel(struct intel_engine_cs *engine) · 1 unmodified line
2094	_MASKED_BIT_DISABLE(RESET_CTL_REQUEST_RESET));	2113	_MASKED_BIT_DISABLE(RESET_CTL_REQUEST_RESET));
2095	}	2114	}
2096		2115
		2116	static int reset_engines(struct drm_i915_private *i915,
		2117	unsigned int engine_mask,
		2118	unsigned int retry)
		2119	{
		2120	if (INTEL_GEN(i915) >= 11)
		2121	return gen11_reset_engines(i915, engine_mask);
		2122	else
		2123	return gen6_reset_engines(i915, engine_mask, retry);
		2124	}
		2125
2097	static int gen8_reset_engines(struct drm_i915_private *dev_priv,	2126	static int gen8_reset_engines(struct drm_i915_private *dev_priv,
2098	unsigned engine_mask)	2127	unsigned int engine_mask,
		2128	unsigned int retry)
2099	{	2129	{
2100	struct intel_engine_cs *engine;	2130	struct intel_engine_cs *engine;
		2131	const bool reset_non_ready = retry >= 1;
2101	unsigned int tmp;	2132	unsigned int tmp;
2102	int ret;	2133	int ret;
2103		2134
2104	for_each_engine_masked(engine, dev_priv, engine_mask, tmp) {	2135	for_each_engine_masked(engine, dev_priv, engine_mask, tmp) {
2105	if (gen8_reset_engine_start(engine)) {	2136	ret = gen8_engine_reset_prepare(engine);
2106	ret = -EIO;	2137	if (ret && !reset_non_ready)
2107	goto not_ready;	2138	goto skip_reset;
2108	}	2139
		2140	/*
		2141	* If this is not the first failed attempt to prepare,
		2142	* we decide to proceed anyway.
		2143	*
		2144	* By doing so we risk context corruption and with
		2145	* some gens (kbl), possible system hang if reset
		2146	* happens during active bb execution.
		2147	*
		2148	* We rather take context corruption instead of
		2149	* failed reset with a wedged driver/gpu. And
		2150	* active bb execution case should be covered by
		2151	* i915_stop_engines we have before the reset.
		2152	*/
2109	}	2153	}
2110		2154
2111	if (INTEL_GEN(dev_priv) >= 11)	2155	ret = reset_engines(dev_priv, engine_mask, retry);
2112	ret = gen11_reset_engines(dev_priv, engine_mask);
2113	else
2114	ret = gen6_reset_engines(dev_priv, engine_mask);
2115		2156
2116	not_ready:	2157	skip_reset:
2117	for_each_engine_masked(engine, dev_priv, engine_mask, tmp)	2158	for_each_engine_masked(engine, dev_priv, engine_mask, tmp)
2118	gen8_reset_engine_cancel(engine);	2159	gen8_engine_reset_cancel(engine);
2119		2160
2120	return ret;	2161	return ret;
2121	}	2162	}
2122		2163
2123	typedef int (reset_func)(struct drm_i915_private , unsigned engine_mask);	2164	typedef int (reset_func)(struct drm_i915_private ,
		2165	unsigned int engine_mask, unsigned int retry);
2124		2166
2125	static reset_func intel_get_gpu_reset(struct drm_i915_private *dev_priv)	2167	static reset_func intel_get_gpu_reset(struct drm_i915_private *dev_priv)
2126	{	2168	{
@@ -2143,12 +2185,15 @@ static reset_func intel_get_gpu_reset(struct drm_i915_private *dev_priv) · 16 unmodified lines
2143	return NULL;	2185	return NULL;
2144	}	2186	}
2145		2187
2146	int intel_gpu_reset(struct drm_i915_private *dev_priv, unsigned engine_mask)	2188	int intel_gpu_reset(struct drm_i915_private *dev_priv,
		2189	const unsigned int engine_mask)
2147	{	2190	{
2148	reset_func reset = intel_get_gpu_reset(dev_priv);	2191	reset_func reset = intel_get_gpu_reset(dev_priv);
2149	int retry;	2192	unsigned int retry;
2150	int ret;	2193	int ret;
2151		2194
		2195	GEM_BUG_ON(!engine_mask);
		2196
2152	/*	2197	/*
2153	* We want to perform per-engine reset from atomic context (e.g.	2198	* We want to perform per-engine reset from atomic context (e.g.
2154	* softirq), which imposes the constraint that we cannot sleep.	2199	* softirq), which imposes the constraint that we cannot sleep.
@@ -2190,8 +2235,9 @@ int intel_gpu_reset(struct drm_i915_private *dev_priv, unsigned engine_mask) · 35 unmodified lines
2190		2235
2191	ret = -ENODEV;	2236	ret = -ENODEV;
2192	if (reset) {	2237	if (reset) {
2193	GEM_TRACE("engine_mask=%x\n", engine_mask);	2238	ret = reset(dev_priv, engine_mask, retry);
2194	ret = reset(dev_priv, engine_mask);	2239	GEM_TRACE("engine_mask=%x, ret=%d, retry=%d\n",
		2240	engine_mask, ret, retry);
2195	}	2241	}
2196	if (ret != -ETIMEDOUT \|\| engine_mask != ALL_ENGINES)	2242	if (ret != -ETIMEDOUT \|\| engine_mask != ALL_ENGINES)
2197	break;	2243	break;
@@ -2237,20 +2283,28 @@ bool intel_uncore_unclaimed_mmio(struct drm_i915_private *dev_priv) · 39 unmodified lines
2237	bool	2283	bool
2238	intel_uncore_arm_unclaimed_mmio_detection(struct drm_i915_private *dev_priv)	2284	intel_uncore_arm_unclaimed_mmio_detection(struct drm_i915_private *dev_priv)
2239	{	2285	{
2240	if (unlikely(i915_modparams.mmio_debug \|\|	2286	bool ret = false;
2241	dev_priv->uncore.unclaimed_mmio_check <= 0))	2287
2242	return false;	2288	spin_lock_irq(&dev_priv->uncore.lock);
		2289
		2290	if (unlikely(dev_priv->uncore.unclaimed_mmio_check <= 0))
		2291	goto out;
2243		2292
2244	if (unlikely(intel_uncore_unclaimed_mmio(dev_priv))) {	2293	if (unlikely(intel_uncore_unclaimed_mmio(dev_priv))) {
2245	DRM_DEBUG("Unclaimed register detected, "	2294	if (!i915_modparams.mmio_debug) {
2246	"enabling oneshot unclaimed register reporting. "	2295	DRM_DEBUG("Unclaimed register detected, "
2247	"Please use i915.mmio_debug=N for more information.\n");	2296	"enabling oneshot unclaimed register reporting. "
2248	i915_modparams.mmio_debug++;	2297	"Please use i915.mmio_debug=N for more information.\n");
		2298	i915_modparams.mmio_debug++;
		2299	}
2249	dev_priv->uncore.unclaimed_mmio_check--;	2300	dev_priv->uncore.unclaimed_mmio_check--;
2250	return true;	2301	ret = true;
2251	}	2302	}
2252		2303
2253	return false;	2304	out:
		2305	spin_unlock_irq(&dev_priv->uncore.lock);
		2306
		2307	return ret;
2254	}	2308	}
2255		2309
2256	static enum forcewake_domains	2310	static enum forcewake_domains

drivers/gpu/drm/i915/intel_wopcm.c+6−0

 	u32 guc_wopcm_rsvd;
 	int err;
+	if (!USES_GUC(dev_priv))
+		return 0;
 	GEM_BUG_ON(!wopcm->size);
+	if (i915_inject_load_failure())
+		return -E2BIG;
 	if (guc_fw_size >= wopcm->size) {
 		DRM_ERROR("GuC FW (%uKiB) is too big to fit in WOPCM.",
 			  guc_fw_size / 1024);

drivers/gpu/drm/i915/selftests/huge_pages.c+6−5

 	if (IS_ERR(obj))
 		return ERR_CAST(obj);
-	cmd = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	err = i915_gem_object_set_to_wc_domain(obj, true);
+	if (err)
+		goto err;
+	cmd = i915_gem_object_pin_map(obj, I915_MAP_WC);
 	if (IS_ERR(cmd)) {
 		err = PTR_ERR(cmd);
 		goto err;
 	}
 	*cmd = MI_BATCH_BUFFER_END;
+	i915_gem_chipset_flush(i915);
 	i915_gem_object_unpin_map(obj);
-	err = i915_gem_object_set_to_gtt_domain(obj, false);
-	if (err)
-		goto err;
 	batch = i915_vma_instance(obj, vma->vm, NULL);
 	if (IS_ERR(batch)) {
 		err = PTR_ERR(batch);

drivers/gpu/drm/i915/selftests/i915_gem.c+221−0

@@ -0,0 +1,221 @@
	1	/*
	2	* SPDX-License-Identifier: MIT
	3	*
	4	* Copyright © 2018 Intel Corporation
	5	*/
	6
	7	#include <linux/random.h>
	8
	9	#include "../i915_selftest.h"
	10
	11	#include "mock_context.h"
	12	#include "igt_flush_test.h"
	13
	14	static int switch_to_context(struct drm_i915_private *i915,
	15	struct i915_gem_context *ctx)
	16	{
	17	struct intel_engine_cs *engine;
	18	enum intel_engine_id id;
	19	int err = 0;
	20
	21	intel_runtime_pm_get(i915);
	22
	23	for_each_engine(engine, i915, id) {
	24	struct i915_request *rq;
	25
	26	rq = i915_request_alloc(engine, ctx);
	27	if (IS_ERR(rq)) {
	28	err = PTR_ERR(rq);
	29	break;
	30	}
	31
	32	i915_request_add(rq);
	33	}
	34
	35	intel_runtime_pm_put(i915);
	36
	37	return err;
	38	}
	39
	40	static void trash_stolen(struct drm_i915_private *i915)
	41	{
	42	struct i915_ggtt *ggtt = &i915->ggtt;
	43	const u64 slot = ggtt->error_capture.start;
	44	const resource_size_t size = resource_size(&i915->dsm);
	45	unsigned long page;
	46	u32 prng = 0x12345678;
	47
	48	for (page = 0; page < size; page += PAGE_SIZE) {
	49	const dma_addr_t dma = i915->dsm.start + page;
	50	u32 __iomem *s;
	51	int x;
	52
	53	ggtt->vm.insert_page(&ggtt->vm, dma, slot, I915_CACHE_NONE, 0);
	54
	55	s = io_mapping_map_atomic_wc(&ggtt->iomap, slot);
	56	for (x = 0; x < PAGE_SIZE / sizeof(u32); x++) {
	57	prng = next_pseudo_random32(prng);
	58	iowrite32(prng, &s[x]);
	59	}
	60	io_mapping_unmap_atomic(s);
	61	}
	62
	63	ggtt->vm.clear_range(&ggtt->vm, slot, PAGE_SIZE);
	64	}
	65
	66	static void simulate_hibernate(struct drm_i915_private *i915)
	67	{
	68	intel_runtime_pm_get(i915);
	69
	70	/*
	71	* As a final sting in the tail, invalidate stolen. Under a real S4,
	72	* stolen is lost and needs to be refilled on resume. However, under
	73	* CI we merely do S4-device testing (as full S4 is too unreliable
	74	* for automated testing across a cluster), so to simulate the effect
	75	* of stolen being trashed across S4, we trash it ourselves.
	76	*/
	77	trash_stolen(i915);
	78
	79	intel_runtime_pm_put(i915);
	80	}
	81
	82	static int pm_prepare(struct drm_i915_private *i915)
	83	{
	84	int err = 0;
	85
	86	if (i915_gem_suspend(i915)) {
	87	pr_err("i915_gem_suspend failed\n");
	88	err = -EINVAL;
	89	}
	90
	91	return err;
	92	}
	93
	94	static void pm_suspend(struct drm_i915_private *i915)
	95	{
	96	intel_runtime_pm_get(i915);
	97
	98	i915_gem_suspend_gtt_mappings(i915);
	99	i915_gem_suspend_late(i915);
	100
	101	intel_runtime_pm_put(i915);
	102	}
	103
	104	static void pm_hibernate(struct drm_i915_private *i915)
	105	{
	106	intel_runtime_pm_get(i915);
	107
	108	i915_gem_suspend_gtt_mappings(i915);
	109
	110	i915_gem_freeze(i915);
	111	i915_gem_freeze_late(i915);
	112
	113	intel_runtime_pm_put(i915);
	114	}
	115
	116	static void pm_resume(struct drm_i915_private *i915)
	117	{
	118	/*
	119	* Both suspend and hibernate follow the same wakeup path and assume
	120	* that runtime-pm just works.
	121	*/
	122	intel_runtime_pm_get(i915);
	123
	124	intel_engines_sanitize(i915);
	125	i915_gem_sanitize(i915);
	126	i915_gem_resume(i915);
	127
	128	intel_runtime_pm_put(i915);
	129	}
	130
	131	static int igt_gem_suspend(void *arg)
	132	{
	133	struct drm_i915_private *i915 = arg;
	134	struct i915_gem_context *ctx;
	135	struct drm_file *file;
	136	int err;
	137
	138	file = mock_file(i915);
	139	if (IS_ERR(file))
	140	return PTR_ERR(file);
	141
	142	err = -ENOMEM;
	143	mutex_lock(&i915->drm.struct_mutex);
	144	ctx = live_context(i915, file);
	145	if (!IS_ERR(ctx))
	146	err = switch_to_context(i915, ctx);
	147	mutex_unlock(&i915->drm.struct_mutex);
	148	if (err)
	149	goto out;
	150
	151	err = pm_prepare(i915);
	152	if (err)
	153	goto out;
	154
	155	pm_suspend(i915);
	156
	157	/* Here be dragons! Note that with S3RST any S3 may become S4! */
	158	simulate_hibernate(i915);
	159
	160	pm_resume(i915);
	161
	162	mutex_lock(&i915->drm.struct_mutex);
	163	err = switch_to_context(i915, ctx);
	164	if (igt_flush_test(i915, I915_WAIT_LOCKED))
	165	err = -EIO;
	166	mutex_unlock(&i915->drm.struct_mutex);
	167	out:
	168	mock_file_free(i915, file);
	169	return err;
	170	}
	171
	172	static int igt_gem_hibernate(void *arg)
	173	{
	174	struct drm_i915_private *i915 = arg;
	175	struct i915_gem_context *ctx;
	176	struct drm_file *file;
	177	int err;
	178
	179	file = mock_file(i915);
	180	if (IS_ERR(file))
	181	return PTR_ERR(file);
	182
	183	err = -ENOMEM;
	184	mutex_lock(&i915->drm.struct_mutex);
	185	ctx = live_context(i915, file);
	186	if (!IS_ERR(ctx))
	187	err = switch_to_context(i915, ctx);
	188	mutex_unlock(&i915->drm.struct_mutex);
	189	if (err)
	190	goto out;
	191
	192	err = pm_prepare(i915);
	193	if (err)
	194	goto out;
	195
	196	pm_hibernate(i915);
	197
	198	/* Here be dragons! */
	199	simulate_hibernate(i915);
	200
	201	pm_resume(i915);
	202
	203	mutex_lock(&i915->drm.struct_mutex);
	204	err = switch_to_context(i915, ctx);
	205	if (igt_flush_test(i915, I915_WAIT_LOCKED))
	206	err = -EIO;
	207	mutex_unlock(&i915->drm.struct_mutex);
	208	out:
	209	mock_file_free(i915, file);
	210	return err;
	211	}
	212
	213	int i915_gem_live_selftests(struct drm_i915_private *i915)
	214	{
	215	static const struct i915_subtest tests[] = {
	216	SUBTEST(igt_gem_suspend),
	217	SUBTEST(igt_gem_hibernate),
	218	};
	219
	220	return i915_subtests(tests, i915);
	221	}

drivers/gpu/drm/i915/selftests/i915_gem_coherency.c+17−21

 {
 	unsigned int needs_clflush;
 	struct page *page;
-	u32 *map;
+	void *map;
+	u32 *cpu;
 	int err;
 	err = i915_gem_obj_prepare_shmem_write(obj, &needs_clflush);
 	page = i915_gem_object_get_page(obj, offset >> PAGE_SHIFT);
 	map = kmap_atomic(page);
+	cpu = map + offset_in_page(offset);
-	if (needs_clflush & CLFLUSH_BEFORE) {
+	if (needs_clflush & CLFLUSH_BEFORE)
-		mb();
+		drm_clflush_virt_range(cpu, sizeof(*cpu));
-		clflush(map+offset_in_page(offset) / sizeof(*map));
-		mb();
-	}
-	map[offset_in_page(offset) / sizeof(*map)] = v;
+	*cpu = v;
-	if (needs_clflush & CLFLUSH_AFTER) {
+	if (needs_clflush & CLFLUSH_AFTER)
-		mb();
+		drm_clflush_virt_range(cpu, sizeof(*cpu));
-		clflush(map+offset_in_page(offset) / sizeof(*map));
-		mb();
-	}
 	kunmap_atomic(map);
 	i915_gem_obj_finish_shmem_access(obj);
 	return 0;
 }
 {
 	unsigned int needs_clflush;
 	struct page *page;
-	u32 *map;
+	void *map;
+	u32 *cpu;
 	int err;
 	err = i915_gem_obj_prepare_shmem_read(obj, &needs_clflush);
 	page = i915_gem_object_get_page(obj, offset >> PAGE_SHIFT);
 	map = kmap_atomic(page);
+	cpu = map + offset_in_page(offset);
-	if (needs_clflush & CLFLUSH_BEFORE) {
+	if (needs_clflush & CLFLUSH_BEFORE)
-		mb();
+		drm_clflush_virt_range(cpu, sizeof(*cpu));
-		clflush(map+offset_in_page(offset) / sizeof(*map));
-		mb();
-	}
-	*v = map[offset_in_page(offset) / sizeof(*map)];
+	*v = *cpu;
-	kunmap_atomic(map);
+	kunmap_atomic(map);
 	i915_gem_obj_finish_shmem_access(obj);
 	return 0;
 }

drivers/gpu/drm/i915/selftests/i915_gem_object.c+1−1

@@ -282,7 +282,7 @@ static int check_partial_mapping(struct drm_i915_gem_object *obj, · 281 unmodified lines
282	view.partial.offset,	282	view.partial.offset,
283	view.partial.size,	283	view.partial.size,
284	vma->size >> PAGE_SHIFT,	284	vma->size >> PAGE_SHIFT,
285	tile_row_pages(obj),	285	tile->tiling ? tile_row_pages(obj) : 0,
286	vma->fence ? vma->fence->id : -1, tile->tiling, tile->stride,	286	vma->fence ? vma->fence->id : -1, tile->tiling, tile->stride,
287	offset >> PAGE_SHIFT,	287	offset >> PAGE_SHIFT,
288	(unsigned int)offset_in_page(offset),	288	(unsigned int)offset_in_page(offset),

drivers/gpu/drm/i915/selftests/i915_live_selftests.h+1−0

@@ -17,6 +17,7 @@ selftest(objects, i915_gem_object_live_selftests) · 16 unmodified lines
17	selftest(dmabuf, i915_gem_dmabuf_live_selftests)	17	selftest(dmabuf, i915_gem_dmabuf_live_selftests)
18	selftest(coherency, i915_gem_coherency_live_selftests)	18	selftest(coherency, i915_gem_coherency_live_selftests)
19	selftest(gtt, i915_gem_gtt_live_selftests)	19	selftest(gtt, i915_gem_gtt_live_selftests)
		20	selftest(gem, i915_gem_live_selftests)
20	selftest(evict, i915_gem_evict_live_selftests)	21	selftest(evict, i915_gem_evict_live_selftests)
21	selftest(hugepages, i915_gem_huge_page_live_selftests)	22	selftest(hugepages, i915_gem_huge_page_live_selftests)
22	selftest(contexts, i915_gem_context_live_selftests)	23	selftest(contexts, i915_gem_context_live_selftests)

drivers/gpu/drm/i915/selftests/intel_guc.c+38−0

@@ -65,6 +65,40 @@ static int check_all_doorbells(struct intel_guc *guc) · 64 unmodified lines
65	return 0;	65	return 0;
66	}	66	}
67		67
		68	static int ring_doorbell_nop(struct intel_guc_client *client)
		69	{
		70	struct guc_process_desc *desc = __get_process_desc(client);
		71	int err;
		72
		73	client->use_nop_wqi = true;
		74
		75	spin_lock_irq(&client->wq_lock);
		76
		77	guc_wq_item_append(client, 0, 0, 0, 0);
		78	guc_ring_doorbell(client);
		79
		80	spin_unlock_irq(&client->wq_lock);
		81
		82	client->use_nop_wqi = false;
		83
		84	/* if there are no issues GuC will update the WQ head and keep the
		85	* WQ in active status
		86	*/
		87	err = wait_for(READ_ONCE(desc->head) == READ_ONCE(desc->tail), 10);
		88	if (err) {
		89	pr_err("doorbell %u ring failed!\n", client->doorbell_id);
		90	return -EIO;
		91	}
		92
		93	if (desc->wq_status != WQ_STATUS_ACTIVE) {
		94	pr_err("doorbell %u ring put WQ in bad state (%u)!\n",
		95	client->doorbell_id, desc->wq_status);
		96	return -EIO;
		97	}
		98
		99	return 0;
		100	}
		101
68	/*	102	/*
69	* Basic client sanity check, handy to validate create_clients.	103	* Basic client sanity check, handy to validate create_clients.
70	*/	104	*/
@@ -332,6 +366,10 @@ static int igt_guc_doorbells(void *arg) · 261 unmodified lines
332	err = check_all_doorbells(guc);	366	err = check_all_doorbells(guc);
333	if (err)	367	if (err)
334	goto out;	368	goto out;
		369
		370	err = ring_doorbell_nop(clients[i]);
		371	if (err)
		372	goto out;
335	}	373	}
336		374
337	out:	375	out:

drivers/gpu/drm/i915/selftests/intel_hangcheck.c+89−12

@@ -1018,8 +1018,41 @@ static int evict_vma(void *data) · 1017 unmodified lines
1018	return err;	1018	return err;
1019	}	1019	}
1020		1020
		1021	static int evict_fence(void *data)
		1022	{
		1023	struct evict_vma *arg = data;
		1024	struct drm_i915_private *i915 = arg->vma->vm->i915;
		1025	int err;
		1026
		1027	complete(&arg->completion);
		1028
		1029	mutex_lock(&i915->drm.struct_mutex);
		1030
		1031	/* Mark the fence register as dirty to force the mmio update. */
		1032	err = i915_gem_object_set_tiling(arg->vma->obj, I915_TILING_Y, 512);
		1033	if (err) {
		1034	pr_err("Invalid Y-tiling settings; err:%d\n", err);
		1035	goto out_unlock;
		1036	}
		1037
		1038	err = i915_vma_pin_fence(arg->vma);
		1039	if (err) {
		1040	pr_err("Unable to pin Y-tiled fence; err:%d\n", err);
		1041	goto out_unlock;
		1042	}
		1043
		1044	i915_vma_unpin_fence(arg->vma);
		1045
		1046	out_unlock:
		1047	mutex_unlock(&i915->drm.struct_mutex);
		1048
		1049	return err;
		1050	}
		1051
1021	static int __igt_reset_evict_vma(struct drm_i915_private *i915,	1052	static int __igt_reset_evict_vma(struct drm_i915_private *i915,
1022	struct i915_address_space *vm)	1053	struct i915_address_space *vm,
		1054	int (fn)(void ),
		1055	unsigned int flags)
1023	{	1056	{
1024	struct drm_i915_gem_object *obj;	1057	struct drm_i915_gem_object *obj;
1025	struct task_struct *tsk = NULL;	1058	struct task_struct *tsk = NULL;
@@ -1040,12 +1073,20 @@ static int __igt_reset_evict_vma(struct drm_i915_private *i915, · 14 unmodified lines
1040	if (err)	1073	if (err)
1041	goto unlock;	1074	goto unlock;
1042		1075
1043	obj = i915_gem_object_create_internal(i915, PAGE_SIZE);	1076	obj = i915_gem_object_create_internal(i915, SZ_1M);
1044	if (IS_ERR(obj)) {	1077	if (IS_ERR(obj)) {
1045	err = PTR_ERR(obj);	1078	err = PTR_ERR(obj);
1046	goto fini;	1079	goto fini;
1047	}	1080	}
1048		1081
		1082	if (flags & EXEC_OBJECT_NEEDS_FENCE) {
		1083	err = i915_gem_object_set_tiling(obj, I915_TILING_X, 512);
		1084	if (err) {
		1085	pr_err("Invalid X-tiling settings; err:%d\n", err);
		1086	goto out_obj;
		1087	}
		1088	}
		1089
1049	arg.vma = i915_vma_instance(obj, vm, NULL);	1090	arg.vma = i915_vma_instance(obj, vm, NULL);
1050	if (IS_ERR(arg.vma)) {	1091	if (IS_ERR(arg.vma)) {
1051	err = PTR_ERR(arg.vma);	1092	err = PTR_ERR(arg.vma);
@@ -1059,11 +1100,28 @@ static int __igt_reset_evict_vma(struct drm_i915_private *i915, · 7 unmodified lines
1059	}	1100	}
1060		1101
1061	err = i915_vma_pin(arg.vma, 0, 0,	1102	err = i915_vma_pin(arg.vma, 0, 0,
1062	i915_vma_is_ggtt(arg.vma) ? PIN_GLOBAL : PIN_USER);	1103	i915_vma_is_ggtt(arg.vma) ?
1063	if (err)	1104	PIN_GLOBAL \| PIN_MAPPABLE :
		1105	PIN_USER);
		1106	if (err) {
		1107	i915_request_add(rq);
1064	goto out_obj;	1108	goto out_obj;
		1109	}
		1110
		1111	if (flags & EXEC_OBJECT_NEEDS_FENCE) {
		1112	err = i915_vma_pin_fence(arg.vma);
		1113	if (err) {
		1114	pr_err("Unable to pin X-tiled fence; err:%d\n", err);
		1115	i915_vma_unpin(arg.vma);
		1116	i915_request_add(rq);
		1117	goto out_obj;
		1118	}
		1119	}
		1120
		1121	err = i915_vma_move_to_active(arg.vma, rq, flags);
1065		1122
1066	err = i915_vma_move_to_active(arg.vma, rq, EXEC_OBJECT_WRITE);	1123	if (flags & EXEC_OBJECT_NEEDS_FENCE)
		1124	i915_vma_unpin_fence(arg.vma);
1067	i915_vma_unpin(arg.vma);	1125	i915_vma_unpin(arg.vma);
1068		1126
1069	i915_request_get(rq);	1127	i915_request_get(rq);
@@ -1086,7 +1144,7 @@ static int __igt_reset_evict_vma(struct drm_i915_private *i915, · 16 unmodified lines
1086		1144
1087	init_completion(&arg.completion);	1145	init_completion(&arg.completion);
1088		1146
1089	tsk = kthread_run(evict_vma, &arg, "igt/evict_vma");	1147	tsk = kthread_run(fn, &arg, "igt/evict_vma");
1090	if (IS_ERR(tsk)) {	1148	if (IS_ERR(tsk)) {
1091	err = PTR_ERR(tsk);	1149	err = PTR_ERR(tsk);
1092	tsk = NULL;	1150	tsk = NULL;
@@ -1137,29 +1195,47 @@ static int igt_reset_evict_ggtt(void *arg) · 44 unmodified lines
1137	{	1195	{
1138	struct drm_i915_private *i915 = arg;	1196	struct drm_i915_private *i915 = arg;
1139		1197
1140	return __igt_reset_evict_vma(i915, &i915->ggtt.vm);	1198	return __igt_reset_evict_vma(i915, &i915->ggtt.vm,
		1199	evict_vma, EXEC_OBJECT_WRITE);
1141	}	1200	}
1142		1201
1143	static int igt_reset_evict_ppgtt(void *arg)	1202	static int igt_reset_evict_ppgtt(void *arg)
1144	{	1203	{
1145	struct drm_i915_private *i915 = arg;	1204	struct drm_i915_private *i915 = arg;
1146	struct i915_gem_context *ctx;	1205	struct i915_gem_context *ctx;
		1206	struct drm_file *file;
1147	int err;	1207	int err;
1148		1208
		1209	file = mock_file(i915);
		1210	if (IS_ERR(file))
		1211	return PTR_ERR(file);
		1212
1149	mutex_lock(&i915->drm.struct_mutex);	1213	mutex_lock(&i915->drm.struct_mutex);
1150	ctx = kernel_context(i915);	1214	ctx = live_context(i915, file);
1151	mutex_unlock(&i915->drm.struct_mutex);	1215	mutex_unlock(&i915->drm.struct_mutex);
1152	if (IS_ERR(ctx))	1216	if (IS_ERR(ctx)) {
1153	return PTR_ERR(ctx);	1217	err = PTR_ERR(ctx);
		1218	goto out;
		1219	}
1154		1220
1155	err = 0;	1221	err = 0;
1156	if (ctx->ppgtt) /* aliasing == global gtt locking, covered above */	1222	if (ctx->ppgtt) /* aliasing == global gtt locking, covered above */
1157	err = __igt_reset_evict_vma(i915, &ctx->ppgtt->vm);	1223	err = __igt_reset_evict_vma(i915, &ctx->ppgtt->vm,
		1224	evict_vma, EXEC_OBJECT_WRITE);
1158		1225
1159	kernel_context_close(ctx);	1226	out:
		1227	mock_file_free(i915, file);
1160	return err;	1228	return err;
1161	}	1229	}
1162		1230
		1231	static int igt_reset_evict_fence(void *arg)
		1232	{
		1233	struct drm_i915_private *i915 = arg;
		1234
		1235	return __igt_reset_evict_vma(i915, &i915->ggtt.vm,
		1236	evict_fence, EXEC_OBJECT_NEEDS_FENCE);
		1237	}
		1238
1163	static int wait_for_others(struct drm_i915_private *i915,	1239	static int wait_for_others(struct drm_i915_private *i915,
1164	struct intel_engine_cs *exclude)	1240	struct intel_engine_cs *exclude)
1165	{	1241	{
@@ -1409,6 +1485,7 @@ int intel_hangcheck_live_selftests(struct drm_i915_private *i915) · 243 unmodified lines
1409	SUBTEST(igt_reset_wait),	1485	SUBTEST(igt_reset_wait),
1410	SUBTEST(igt_reset_evict_ggtt),	1486	SUBTEST(igt_reset_evict_ggtt),
1411	SUBTEST(igt_reset_evict_ppgtt),	1487	SUBTEST(igt_reset_evict_ppgtt),
		1488	SUBTEST(igt_reset_evict_fence),
1412	SUBTEST(igt_handle_error),	1489	SUBTEST(igt_handle_error),
1413	};	1490	};
1414	bool saved_hangcheck;	1491	bool saved_hangcheck;

drivers/gpu/drm/i915/selftests/mock_context.c+3−8

 	INIT_RADIX_TREE(&ctx->handles_vma, GFP_KERNEL);
 	INIT_LIST_HEAD(&ctx->handles_list);
+	INIT_LIST_HEAD(&ctx->hw_id_link);
 	for (n = 0; n < ARRAY_SIZE(ctx->__engine); n++) {
 		struct intel_context *ce = &ctx->__engine[n];
 		ce->gem_context = ctx;
 	}
-	ret = ida_simple_get(&i915->contexts.hw_ida,
+	ret = i915_gem_context_pin_hw_id(ctx);
-			     0, MAX_CONTEXT_HW_ID, GFP_KERNEL);
 	if (ret < 0)
 		goto err_handles;
-	ctx->hw_id = ret;
 	if (name) {
 		ctx->name = kstrdup(name, GFP_KERNEL);
 void mock_init_contexts(struct drm_i915_private *i915)
 {
-	INIT_LIST_HEAD(&i915->contexts.list);
+	init_contexts(i915);
-	ida_init(&i915->contexts.hw_ida);
-	INIT_WORK(&i915->contexts.free_work, contexts_free_worker);
-	init_llist_head(&i915->contexts.free_list);
 }
 struct i915_gem_context *

drivers/gpu/drm/i915/selftests/mock_gtt.c+2−0

@@ -118,6 +118,8 @@ void mock_init_ggtt(struct drm_i915_private *i915) · 117 unmodified lines
118	ggtt->vm.vma_ops.clear_pages = clear_pages;	118	ggtt->vm.vma_ops.clear_pages = clear_pages;
119		119
120	i915_address_space_init(&ggtt->vm, i915);	120	i915_address_space_init(&ggtt->vm, i915);
		121
		122	ggtt->vm.is_ggtt = true;
121	}	123	}
122		124
123	void mock_fini_ggtt(struct drm_i915_private *i915)	125	void mock_fini_ggtt(struct drm_i915_private *i915)

include/drm/i915_pciids.h+1−0

@@ -386,6 +386,7 @@ · 385 unmodified lines
386	INTEL_VGA_DEVICE(0x3E91, info), /* SRV GT2 */ \	386	INTEL_VGA_DEVICE(0x3E91, info), /* SRV GT2 */ \
387	INTEL_VGA_DEVICE(0x3E92, info), /* SRV GT2 */ \	387	INTEL_VGA_DEVICE(0x3E92, info), /* SRV GT2 */ \
388	INTEL_VGA_DEVICE(0x3E96, info), /* SRV GT2 */ \	388	INTEL_VGA_DEVICE(0x3E96, info), /* SRV GT2 */ \
		389	INTEL_VGA_DEVICE(0x3E98, info), /* SRV GT2 */ \
389	INTEL_VGA_DEVICE(0x3E9A, info) /* SRV GT2 */	390	INTEL_VGA_DEVICE(0x3E9A, info) /* SRV GT2 */
390		391
391	/* CFL H */	392	/* CFL H */

include/uapi/drm/i915_drm.h+22−0

@@ -529,6 +529,28 @@ typedef struct drm_i915_irq_wait { · 528 unmodified lines
529	*/	529	*/
530	#define I915_PARAM_CS_TIMESTAMP_FREQUENCY 51	530	#define I915_PARAM_CS_TIMESTAMP_FREQUENCY 51
531		531
		532	/*
		533	* Once upon a time we supposed that writes through the GGTT would be
		534	* immediately in physical memory (once flushed out of the CPU path). However,
		535	* on a few different processors and chipsets, this is not necessarily the case
		536	* as the writes appear to be buffered internally. Thus a read of the backing
		537	* storage (physical memory) via a different path (with different physical tags
		538	* to the indirect write via the GGTT) will see stale values from before
		539	* the GGTT write. Inside the kernel, we can for the most part keep track of
		540	* the different read/write domains in use (e.g. set-domain), but the assumption
		541	* of coherency is baked into the ABI, hence reporting its true state in this
		542	* parameter.
		543	*
		544	* Reports true when writes via mmap_gtt are immediately visible following an
		545	* lfence to flush the WCB.
		546	*
		547	* Reports false when writes via mmap_gtt are indeterminately delayed in an in
		548	* internal buffer and are _not_ immediately visible to third parties accessing
		549	* directly via mmap_cpu/mmap_wc. Use of mmap_gtt as part of an IPC
		550	* communications channel when reporting false is strongly disadvised.
		551	*/
		552	#define I915_PARAM_MMAP_GTT_COHERENT 52
		553
532	typedef struct drm_i915_getparam {	554	typedef struct drm_i915_getparam {
533	__s32 param;	555	__s32 param;
534	/*	556	/*

Merge tag 'drm-intel-next-2018-09-06-2' of git://anongit.freedesktop.org/drm/drm-intel into drm-next

Keyboard shortcuts

General

Issues and pull requests

Issues, pull requests and Inbox

Repository code